Compare commits

...
39 Commits
Author SHA1 Message Date
glenn schrooyenandClaude Opus 5 d57122ec74 DATA-02: a real Belgian price year, so AC4 stops being unmeetable
checks / unit (pull_request) Successful in 3m36s
OPT-04 reported AC4 - "net-positive across 12 months of replayed prices" -
as NOT MET and replayed a synthetic year, because scenarios/ held only P1
meter captures. Right call, wrong diagnosis: the criterion was being carried
beside a 30-day METER criterion as though both needed waiting. EPEX BE
day-ahead is public data, not a capture. It needed a fetch.

8760 hourly points, 2025-09-01 to 2026-09-01 Brussels-local, canonical three
fields. Licence checked BEFORE committing: Energy-Charts publishes the BE
zone under CC BY 4.0 from Bundesnetzagentur | SMARD.de and other zones under
private-use-only terms, so this one is redistributable with attribution. The
fetcher refuses to write a file whose per-response license_info stops saying
so. Cross-checked against SMARD's own 4996/DE: 96 of 96 quarter-hours, max
difference 0.00 EUR/MWh.

Raw wholesale, and the file says so. all_in: false, with the markup/tax
assumption carried as numbers to APPLY rather than numbers applied -
TARIFF-02's trap in data form. Section 7B applies it in one visible place and
asserts arithmetically that its additive terms cancel out of every daily
spread, so the trade count turns on the VAT multiplier alone; the raw replay
is run too, so how much rests on the guess is reported (110 vs 88).

DST by construction: iterate in UTC, render through price.brussels().
2025-10-26 is 25 points, 2026-03-29 is 23, verified by counting per Brussels
day rather than asserting 24, and by checking October's 02:00 appears twice
at two offsets with two prices while March's never appears. Both bugs the
point-and-gap accounting caught were real: a UTC-bounded window made the
first day 22 hours, and an inclusive-end off-by-one dropped the last day
silently. A day nobody asked for is not a day with no prices.

THE RESULT, AT THE SHIPPED DEFAULT, WHATEVER IT IS. At 0.10 widened 1.5x to
0.1500: 110 round trips, +24.88 EUR net of the cycle cost. Unwidened, 173.
Nothing tuned - the threshold and the 2.5/1.5 factors are untouched.

And what the owner's decision now has under it: the four reports that a
static ~0.0840 spread does not clear 0.10 all STAND. What was missing is that
a dynamic site clears the widened requirement on 121 of 365 days, median
all-in spread 0.1130, so a median day would clear at 0.0754 - 156 trips,
+35.42 EUR. Reported, not applied.

The synthetic year stays, because the real one cannot grade section 17: not
one of 365 days has a negative DEAREST hour, so a zero there counts something
that could not happen. 7B asserts that degeneracy rather than reporting the
zero.

Size measured, not assumed: 893 KB plain but 74.6 KB as a git object against
56.5 KB gzipped. Git already compresses; 18 KB is not worth making the
provenance block - the licence claim a reviewer must check - unreadable in a
diff.

Nothing wired into the add-on; generic_dayahead is still the live path.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-31 11:01:42 +02:00
admin 4601682815 SAFETY-24: an energy floor under the peak guard's forced discharge (#21)
SAFETY-07 could command a discharge to hold a billed quarter down and nothing below it consulted the state of charge - the BMS was the sole energy backstop. decide() now takes soc and soc_floor, and discharge_permitted() is the one place the question is answered: public, no default arguments, and consulted only AFTER a forced discharge is actually being asked for. The denied action is a 0 W floor - one-directional, forbids charging, structurally cannot cap the discharge the loop is already doing.

Reviewed independently. The trade-off is ruled honest and was made a check rather than a footnote: on a pack deliberately sized so the floor must bind, the worst quarter goes 4338 -> 5030 W while end SoC goes 9.6 -> 20.0 % and cycles pinned at the BMS cut-out go 1227 -> 0. Reproduced on the reviewer's own replay, including the 304.2 Wh withheld against 292.8 Wh between the two floors - half a cycle at the rail apart, so the floor is doing the arithmetic it claims.

The guard's central guarantee was re-derived across 19,800 combinations of load x elapsed x battery x SoC x floor: 0 negative floors, 0 claims with hi other than +inf, 0 decisions both floor-limited and saturated, 0 arbiter resolutions lowered or contradicted. Rule 8's <= is right: at equality the EMS floor could never be the thing that stops the discharge.

Eleven author mutations plus nine reviewer ones, all red under python -B with a purged __pycache__ and real byte deltas. One survivor is an equivalent mutant and provably so: a float(soc or 100) defaulting above the call site is dead code, because a Controller with no SoC never reaches decide() at all - inputs_ok is False and cycle() stops at the safety claim first. That property is now asserted on the service call, replacing an assertion whose sentence was wider than its check.

Merged after two rebases. The second one broke a grader neither card could have seen alone: test_peakguard graded the FIRST .decide( in main.py and required exactly one, which stopped being true when OPT-04 added arbitrage.decide(). It failed loudly rather than grading the wrong call. Fixed by selecting peakguard.decide by qualified name and requiring exactly one of that, verified still red on a decoy and on the maintenance_soc_floor swap.

Follow-ups filed as SAFETY-27, not addressed here: arbitrage_soc_floor has no rule-8 twin against bms_soc_floor, and this card's own 'revisit if a third floor appears' trigger has fired.
2026-08-31 10:24:36 +02:00
glenn schrooyenandClaude Opus 5 9234016419 Merge release/1.0 (OPT-04) into SAFETY-24, and fix one grader the merge broke
checks / unit (pull_request) Successful in 3m36s
OPT-04 landed after the previous rebase. Conflicts were CHANGELOG.md and one
IMPORT LINE in validate.py; both refusal blocks and both option sets survive
untouched, and DOCS.md, main.py and config.yaml auto-merged.

⚠️ AND THE MERGE BROKE A CHECK THAT NEITHER CARD COULD HAVE SEEN ALONE.
test_peakguard.py graded "the first `.decide(` call in main.py" and asserted
there was exactly one. That was true when SAFETY-24 was written; OPT-04 added
`arbitrage.decide(...)`, so the merged tree had two and the assertion failed.
It failed loudly rather than grading the wrong call - the guard did its job -
but it cannot ship red.

Fixed by selecting the call by QUALIFIED NAME (`peakguard.decide`) instead of
by position, and requiring exactly one of THAT. Other modules may now call
their own decide(); this grader can no longer be moved by one of them. Verified
still discriminating: a decoy `peakguard.decide` added to main.py turns it red
on 7 assertions, and the `maintenance_soc_floor` swap still turns it red on 6.

Also folded in the review's wording finding. An assertion read "a CONTROLLER
whose SoC entity does not answer hands decide() None" while driving no
Controller. The wiring property is stronger than the one claimed, and is now
asserted on the SERVICE CALL: a Controller with no SoC never reaches the guard
at all, because `inputs_ok` is False and cycle() stops at the safety claim
first - so a `float(soc or 100)` defaulting line above the call site cannot
disarm the floor. The original decide()-level check is kept, reworded to say
what it actually grades.

Full suite green on the merged tree: 15 files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-31 07:39:40 +02:00
admin b7272169b1 SAFETY-26: pin the negative-price boundary from below, not just from above (#23)
Closes the survivor found reviewing OPT-04: price < 0 weakened to < -0.001 left every suite green while a -0.0005 EUR/kWh sell hour discharged. Pinned as a sweep rather than a point - five magnitudes spanning >10000x down to -0.000001, the last place the canonical 6-decimal format expresses - with the well-formedness of each stimulus asserted first so section 17 is the only rung that can refuse, and the same five sign-flipped asserted to discharge so it is a boundary at exactly zero. app/ untouched.
2026-08-31 07:37:46 +02:00
glenn schrooyenandClaude Opus 5 2b8b8cb97f SAFETY-26: pin the negative-price refusal from below, not only from above
checks / unit (pull_request) Successful in 3m15s
OPT-04's reviewer weakened `if price < 0` to `if price < -0.001` and every
suite stayed green while a -0.0005 EUR/kWh sell hour discharged. The shipped
code was right the whole time; the CHECK could not tell correct from
nearly-correct. The only probe near the boundary sat at 0.0000, which catches a
`<=` and says nothing about a tolerance invented on the other side - and a
tolerance on the other side is §17's named regression walking through with the
suite passing. A dearest hour at -0.0005 is an ordinary Belgian day-ahead
print, not a fuzz value.

`app/` IS NOT TOUCHED. `if price < 0` is already correct and changing it would
be the defect; this card closes the check, not the behaviour. The 0.0000 probe
stays and is now labelled as the half that pins the boundary from above.

The new probe is a SWEEP rather than a point, because one probe at -0.0005
closes `-0.001` and nothing else - a mutation reaching for `-0.01` would be the
identical defect one decimal further out. Five prices span four orders of
magnitude down to -0.000001, the last place the canonical six-decimal format
can express; each is asserted to survive that rounding as strictly negative,
to still be the DEAREST hour, and to still clear the widened threshold, so §17
is the only rung that can refuse. The same five magnitudes with the sign
flipped all discharge, so this is a boundary at exactly zero rather than a
refusal of small numbers.

AC4, the look: every price/quantity boundary in arbitrage.py and
static_tariff.py was given the same tolerance mutation, both directions where
both exist. Two survive, both in static_tariff.py and both NAMED RATHER THAN
FIXED - they are TARIFF-03's rungs and want probes in its own file. `spread ==
0.0` loosened to `<= 0.001` flips a real 0.0005 spread from allowed to refused
at min_spread_eur_kwh=0.0, a documented supported value, and otherwise prints a
false reason about a tariff that is not flat. `len(firm) < 2` loosened to `< 1`
survives at +0 bytes - TEST-15's risky delta - and is masked only by the flat
rung, so the two mask each other and a later edit to either opens a hole
nothing reports.

Mutation verified under `python -B` with a recursive __pycache__ purge, byte
delta printed (+5 B), the mutated line asserted changed, and - per the
reviewer's addition to TEST-15 - the tree re-read from disk and asserted to be
the tree that was mutated before each run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-31 02:07:56 +02:00
admin 8e589fee3f OPT-04: price arbitrage, at priority 3, shipped switched off (#22)
app/arbitrage.py consumes TARIFF-02s canonical prices, TARIFF-03s Arbitrage verdict and FCST-03s firm/provisional split rather than re-implementing them. An idle decision files NO claim - a 0 W set at P_ARBITRAGE would out-rank the loop and kill self-consumption. Peak shaving is honoured twice: TARIFF-03s required peak_guard_active interlock, and P_PEAK 50 clamping P_ARBITRAGE 30 in the arbiter. The fetch lives in run_tariff_refresh(), asserted on the shipped AST to be absent from cycle() and read_inputs(). Selling into a negative-price hour is refused. arbitrage_enabled ships false.
2026-08-31 01:53:31 +02:00
glenn schrooyenandClaude Opus 5 bdd7f18a54 OPT-04: charge cheap, discharge expensive - and only when it pays
checks / unit (pull_request) Successful in 3m16s
Four cards built the price stack and nothing traded on it. app/arbitrage.py is
§10.1's priority 3, and it consumes all four rather than re-deciding anything:
whether a trade is permitted at all, on which two hours, against which widened
threshold is TariffPlan.arbitrage()'s answer, and that answer cannot be
obtained without first saying whether the peak guard is holding. §9.1's "peak
shaving outranks arbitrage" is therefore honoured structurally - there is no
path to a spread that skips the question - and again in the arbiter, where
P_PEAK (50) clamps a claim filed at P_ARBITRAGE (30).

A guard that never ran is not a guard that is clear. On meter_source off the
peak guard is never evaluated; that third state is carried as None and refuses,
because charging at the rail creates the billed quantity while nothing measures
the quarter. validate.py refuses that configuration too, along with arbitrage
armed with no price source and a SoC floor at or above the ceiling.

§17: never export into a negative price. The guard is on the hour being sold
into, not on the spread - a day priced -0.30 .. -0.05 clears any threshold and
its dearest hour is still negative. Buying in a negative hour stays allowed.

resolve_tariff() may open an HTTPS connection and the tick feeds the ESP32
heartbeat, so the fetch runs in its own task at a 30 min cadence and the tick
reads an attribute and an age bound. test_arbitrage.py section 8 asserts that
structurally on the shipped AST: a fetch put back on cycle() is red.

AC4 is NOT met and is not restated - this repository holds no price history.
Section 7 replays a synthetic year and asserts the property that does not need
real prices: every executed round trip cleared its own widened requirement, and
zero discharges landed in the 522 negative-priced hours. At the shipped 0.10
EUR/kWh threshold a realistic Belgian day/night site does not trade at all.
Reported, not tuned: that number and FCST-03's widening factors are one product
decision and it is the owner's.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-31 01:36:09 +02:00
glenn schrooyenandClaude Opus 5 a1376e44e4 Merge release/1.0 (FCST-03) into SAFETY-24 - CHANGELOG only
checks / unit (pull_request) Successful in 3m28s
FCST-03 landed while this was in review, so both branches had prepended an
Unreleased stanza. Only CHANGELOG.md conflicted, exactly as predicted;
app/, config.yaml, validate.py and DOCS.md all auto-merged, and this card
touches neither price_forecast.py nor static_tariff.py.

Both stanzas kept verbatim, SAFETY-24's above FCST-03's - no text from
either author edited. Full suite green on the merged tree: 14 files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-31 00:43:58 +02:00
glenn schrooyenandClaude Opus 5 e709554857 SAFETY-24: an energy floor under the peak guard's forced discharge
checks / unit (pull_request) Successful in 3m29s
SAFETY-07 can command a discharge to hold a billed quarter down, and
nothing below it consulted the state of charge. On the 2026-08-20 replay
it asks for 878 Wh in a day. Relying on the BMS is not absurd - it is a
real floor and it works - but it is the pack's protection rather than the
operator's, and a month of aggressive quarters leaves the battery
repeatedly at that cut-out with no EMS-side record of why. The first
symptom is a pack never available when wanted, which is indistinguishable
from the SoC drift MAINT-01 exists to correct.

decide() now takes soc and soc_floor, both required and neither
defaulted, and discharge_permitted() is the one place the question is
answered. Above the floor nothing changes. At or below it the guard files
a 0 W floor instead of a sized one - one-directional, so charging stays
blocked and no discharge is capped - and reports floor_limited with the
reason it bound. It is not a trip: every stop in 11.2's table still
outranks the claim, and the floor is never negative, so 9.1's asymmetry
survives as a property of the object (0 negative floors over 1296 swept
combinations, SoC included).

The SoC is graded only after `wanted > 0` is established, so it is never
evaluated where it governs nothing. An unreadable SoC is floor-limited,
not a full battery - a full battery is exactly the value that would
disarm the floor, and this project shipped that defaulting once already.

Failing to shave is not silent: one notification per episode
(goodwe_peak_soc_floor), distinct from the saturation alarm because the
two ask for opposite operator actions, plus its own status row and an
action-log line.

Verified by replay. The 2026-08-20 capture is driven against a modelled
pack sized from the day's own measured appetite, so it genuinely
exhausts: without the floor it reaches the BMS cut-out and holds there
for 1227 guard cycles; with it the pack stops at 20.0 % and never touches
the BMS. The energy withheld equals the energy between the two floors to
within one cycle at the rail. The honest half is reported too - the worst
quarter goes 4338 W -> 5030 W, because a floor-limited guard shaves less
on purpose.

Two new options, mirrored once in peakguard.PEAK_OPTION_DEFAULTS
(TARIFF-02's pattern), imported by validate.py and main.py rather than
typed as literals and pinned to config.yaml by a check that reads the
file - so CFG-03 still tracks three places. validate.py rule 8 refuses
peak_soc_floor <= bms_soc_floor at startup, in CFG-01/MAINT-04's manner.
maintenance_soc_floor is deliberately not reused and not renamed: it is a
drain target, not a spend limit, and the rename is recorded as a decision
to revisit if a third floor appears.

Eleven mutations of the shipped source were run against the checks; all
eleven go red and there are no survivors.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-31 00:29:36 +02:00
admin a30bde1ab1 FCST-03: provisional prices - yesterday flagged as yesterday, and unable to move anything (#20)
app/price_forecast.py fills unknown hours from yesterday, flagged provisional by four independent mechanisms: _carried() has no provisional parameter, published points pass through by identity, _audit() re-derives every flag and raises rather than repairs, and an AST check refuses a non-literal provisional. Gate blocks irreversible actions on provisional hours. Arbitrage refusal 3 rewritten to fewer-than-two-FIRM-hours so filling does not switch arbitrage off every morning, with the 11.4 fallback still refused by a truer route. Real-time products get a measured 2.5x wider threshold, following the declared PriceProduct.
2026-08-31 00:23:41 +02:00
glenn schrooyenandClaude Opus 5 71cd7849e4 Merge release/1.0 (SAFETY-07) into FCST-03, and close two degenerate stimuli
checks / unit (pull_request) Successful in 3m13s
CHANGELOG.md was the entire collision - SAFETY-07 lands in peakguard.py, main.py
and the arbiter, none of which this branch touches. Both entries kept.

X1. `_firm_or_die` guarding only `cheapest` survived, because the check passed
the SAME provisional point as both cheapest and dearest: a guard on either role
alone raises, so the stimulus could not tell "the buy hour is guarded" from
"both are". Now one call per role with a firm point in the other seat, plus the
positive arm so the two refusals are discriminating.

X6. `firm_points()` returning the carried hours too survived, because its two
consumers - the §11.4 refusal and cheap_now - guard provenance independently and
both keep answering correctly. The accessor is now graded directly: exactly the
published points, by identity and in order.

Both are the degeneracy rule this file states at the top, in a new dress - a
stimulus that cannot distinguish the two things it grades.

⚠️ AND THE REASON THE M9 MUTATION "SURVIVED" IN MY EARLIER REPORT: a stale
__pycache__. Reverting `self.provisional` to `here.provisional` changes neither
the file size nor, at one-second resolution, the mtime, so CPython kept the
MUTATED bytecode and a green run afterwards proved nothing about that line. Every
mutation here now clears __pycache__ first; re-run that way, M9 is caught. The
committed source was always correct - the checks were not being run against it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-31 00:15:32 +02:00
admin e7c95bfd8a SAFETY-07: cap the loop before it sets the month's billed capacity peak (#18)
The first thing in this system that can STOP a runaway rather than alarm on one. Each cycle it projects the open 900 s block from P1 telegrams and, over the ceiling, files one arbiter claim: a lower bound at P_PEAK, sized as excess = 900*(projected-ceiling)/remaining added to the MEASURED battery power. No writes, no I/O, no clock, no state between calls.

Replayed independently by the reviewer on a separate harness: 2026-08-20's billed quarter 5029.8 W -> 3485.3 W, both days 189/190 -> 190/190 under the 3500 W soft limit, 878 Wh modelled discharge on the failure day and 0 Wh with 0 engagements on the healthy one.

Verified as properties rather than witnesses: the lo-limit claim cannot cap discharge or export (hi is inf; 0 of 54 loop/floor pairs lowered the target; no floor in 0..5000 W reduces a full-rate discharge; decide() emitted 0 negative floors over 630 input combinations), and the 900/remaining gain is exactly 1.0 at the top of the block, growing, bounded at the end by the achievable_w clamp so it can never file lo > hi.

Round one held this PR on three wiring mutations that left all twelve suites green while a default install parked at 0 W in the billed quarter - including the suppression re-gated on the optional peak_forecast_entity, verbatim the historical defect this card's own comment described. All three are closed. The conjunction moved out of main.py into peakguard.shaping_suspended() so the check can assert on the shipped expression, and part 7 now executes the mutations against the real Controller.cycle(). The reviewer confirmed the harness cannot go vacuous: re-inlining the conjunction in main.py with behaviour-identical code turns B1 and B2 RED rather than green, because they assert the mutation takes effect.

Eight further reviewer mutations this round, all red, no survivors. One note recorded, not blocking: the parsed-syntax grade on decide(batt_w=...) is defeated by a decoy .decide() call or by overwriting self.batt before the call, and it is the behavioural checks beside it that catch both.

AC 3 remains TARIFF-01's and is NOT met - do not close the card as AC 3 met. SAFETY-24, the missing SoC backstop on the forced discharge, is named in peakguard.py with the 878 Wh figure and must land before this reaches a client install.
2026-08-30 23:56:20 +02:00
glenn schrooyenandClaude Opus 5 dd40c56ab4 FCST-03: yesterday's prices, flagged as yesterday's, and unable to move anything
checks / unit (pull_request) Successful in 2m43s
§8.3 fills the hours a day-ahead feed has not published yet with yesterday's
profile and flags them provisional, and forbids any irreversible action on such
a price. The whole risk of this ticket is one line - a carried point and a
published point are three fields and a bool, so `provisional=y.provisional`
copies yesterday's False onto today's guess and produces stale data presented as
current. That is SAFETY-18's frozen-but-fresh meter one layer up.

So the flag is never passed through. Carried points come from a constructor with
no provisional parameter; published points pass through BY IDENTITY, so no copy
exists that could lose one; an audit re-derives the expected flag for every
emitted point and raises rather than repairing; and the check walks the module's
own AST so the pass-through idiom cannot come back looking ordinary. A published
point may itself be provisional (eneco_eddy's imbalance print is), so the rule
for a published hour is "whatever the feed said", never False.

The fill keys on the Brussels wall clock rather than t-24h, so the 23- and
25-hour days are right; a late publication rebuilds the series, moving the
number and the flag together.

AC2 and AC3 landed on TARIFF-03's existing verdict surface rather than beside
it. Refusal 3 became "fewer than two FIRM hours" - refusing on "any point is
provisional" would have switched arbitrage off every morning on every dynamic
site - and refusal 5 is new: the spread must clear a threshold widened by the
widest active uncertainty source (max, never a product). The permitting variant
now refuses to be constructed around a provisional point at all, and the
cheap-window flag became per-hour for the same reason.

The irreversible action is a value carrying the hours it commits, derived from
maintenance.py's own DRAIN_TIMEOUT_MIN; the gate blocks a maintenance drain
whose window reaches into a carried hour, and the phase machine stays idle. It
is the price-justified entry point only - a gate that blocked on no_data would
stop the monthly cycle on every default install.

Zero new config options; TEST-14's signature scan ported to cover both Gate's
and Arbitrage's constructors, closing _Refused.reason's open hole.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 23:54:07 +02:00
glenn schrooyenandClaude Opus 5 c2ed509565 Merge release/1.0 (TARIFF-03) into SAFETY-07
checks / unit (pull_request) Successful in 3m12s
The anticipated conflict again, and only it: both cards prepend to CHANGELOG.md's
Unreleased section. Both entries kept, SAFETY-07 first. app/main.py and DOCS.md
auto-merged, so the overlap claim on #18 still holds - config.yaml and
validate.py remain untouched by this card even after TARIFF-02 and TARIFF-03
widened both.

Full suite green on the merge result, test_price.py and test_static_tariff.py
included.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 23:43:35 +02:00
glenn schrooyenandClaude Opus 5 2d570d84ce SAFETY-07 review: defend the three wiring mutations that survived
The reviewer's finding was correct and the shape of it matters: everything the
first cut graded was in peakguard.py, and grading the module is not the same as
grading the guard. Three one-line changes to main.py's CALL SITE left all twelve
suites green while a default install parked at 0 W in the quarter that sets the
month's bill.

B2 was the serious one. My own comment described that defect - "that entity is
optional and empty on a default install, so until now the suspension could not
fire on the site that needed it" - and nothing could regress it. A comment that
documents a fix nobody can test is the same shape as a corrected comment nobody
tests, which this project has already been caught by once.

WHAT CHANGED

The suppression conjunction moved out of main.py and into
peakguard.shaping_suspended(), for the reason watchdog.loop_is_following() and
freeze.looks_frozen() live where they do: the check that grades a safety
predicate must assert on THE SHIPPED EXPRESSION, not on a transcription that can
drift apart from it in silence. Its docstring carries both terms and why neither
is sufficient alone. main.py's call site now says READ THAT FUNCTION FIRST, and
records that a reviewer mutated it back to its historical broken form with every
suite staying green.

test_peakguard.py part 7 gains a section that EXECUTES the three mutations
rather than describing them. Each one is put in place on the shipped module and
the REAL Controller.cycle() is driven with it, on the dangerous fixture: a
default install (no peak_forecast_entity, so peak_fc is None and peak_at_risk is
False forever) with maintenance in its CHARGE phase, so a P_MAINT_SHAPE `hi <= 0`
limit is live, and the guard genuinely engaged on the measurement. That the
mutations are monkeypatched onto the shipped module is itself load-bearing: if
main.py inlined either expression again the patch would do nothing and these
checks would pass vacuously.

    SHIPPED   suspended=True    loop -> 1498 W, limited by peak    +1497.8 W
    B1        suspended=False   CONTRADICTION between limits           0.0 W
    B2        suspended=False   CONTRADICTION between limits           0.0 W
    B2b (forecast term dropped instead) - caught by its own check

B3 is graded three ways, because the single-cycle form is only INFLATED and the
failure is what happens on the next cycle:

  * the value main.py actually hands decide() is recorded off the real call, with
    the measured battery at 0 W and the command standing at +3000 W;
  * the one-cycle difference is asserted to be exactly the standing command - the
    whole defect in one line, watts that were asked for and never arrived counted
    as if they had;
  * both forms are then run forward against the REAL controller's own projection
    with an inverter that never follows:

        intent-sized floor    4482 5000 5000 5000 ...  SATURATED at cycle 2
        delivered-power floor 1482 1482 1482 1482 ...  stationary

  * and the call site is graded on parsed syntax, so a rename cannot quietly
    reintroduce it: `decide(batt_w=...)` must unparse to `self.batt`.

SAFETY-24 is now named in peakguard.py, in its own section, with the number that
makes it real - 878 Wh of forced discharge on the 2026-08-20 replay - and the
statement that this must not go to a client install before that ticket lands. A
ticket nobody can find from the code is a ticket nobody reads before shipping.

RED AND GREEN

Fifteen mutations run against the shipped modules, each reverted after: the six
new wiring ones (B1, B2, B2b, B3, B3b, and deleting the SAFETY-24 pointer) and
the eight from the first round, re-run to confirm they still bite. All fifteen
turn test_peakguard.py red. Full suite green, test_price.py and TARIFF-03's
included.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 23:40:39 +02:00
admin cf5094c608 TARIFF-03: the static day/night profile as a first-class product and the 11.4 fallback (#19)
app/static_tariff.py: StaticProfile derives STATIC vs FIXED from its own prices, one generator serves both the configured product and the degradation fallback differing only in the provisional flag. TariffPlan.arbitrage() is a refusal type behind a required keyword-only peak_guard_active, and a flat tariff is refused by arithmetic rather than by matching PriceProduct.FIXED. main.py touched only in read_inputs() else-branch, one field, one status row - nothing in cycle(), the arbiter or the peak path; an empty profile is bit-for-bit the old behaviour. Rule 7 tripwire re-aimed rather than silenced.
2026-08-30 23:24:53 +02:00
glenn schrooyenandClaude Opus 5 9f8b721a4d Merge release/1.0 (TARIFF-02) into SAFETY-07
checks / unit (pull_request) Successful in 3m9s
Conflict was the one anticipated: both cards prepend to CHANGELOG.md's
Unreleased section. Both entries kept, SAFETY-07 first. Nothing else of
TARIFF-02's surface (app/price.py, validate.py rule 7, config.yaml, DOCS.md)
overlaps this card - config.yaml and validate.py are untouched here, so the
CFG-03 check still reads a config.yaml that only TARIFF-02 has widened.

Full suite green on the merge result, test_price.py included.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 23:13:29 +02:00
glenn schrooyenandClaude Opus 5 303525cfc5 TARIFF-03: static day/night and fully-fixed tariffs, first-class
checks / unit (pull_request) Successful in 2m43s
§9.2 puts a large share of the Belgian installed base on a static profile, and
an EMS that only works under dynamic pricing is an EMS for a minority. A new
`static_profile` option describes a two-period day/night tariff - or a single
fixed price - and app/static_tariff.py turns it into the same canonical
{start, eur_per_kwh, provisional} series the fetched providers emit.

Not a sixth provider plugin, for two mechanical reasons. capabilities() is a
classmethod, so a plugin's PriceProduct is fixed when the class is written -
but §9.2 makes STATIC and FIXED two different products and which one a site is
on is a property of the operator's data; StaticProfile.product derives it from
the number of distinct prices. And the plugin framework exists to stop a
FETCHED figure being marked up twice, which a number typed as all-in cannot be.
So the profile sits where §11.4 already says it does - under the providers, as
the fallback. Configured product and feed-is-down are one generator differing
in one flag, so the degradation path cannot rot unused.

"No arbitrage" is a type. TariffPlan.arbitrage() returns a variant whose
ALLOWED arm carries the spread and the cheapest/dearest hours and whose REFUSED
arm carries a reason and nothing else - .spread_eur_kwh on a refusal raises
rather than answering 0.0, so an optimiser cannot compute a trade from one.
Four refusals, all derived: peak guard holding, no price data, plan
provisional, or the series is FLAT. The last is AC2: a fixed tariff disables
arbitrage by arithmetic, not by matching on an enum. peak_guard_active is a
required keyword argument, so §10.1's priority 2 over 3 cannot be skipped.

Time-window shifting reaches the one shiftable load that exists: main.py's
cheap-window flag falls back to the static profile, and the UI gained a Tariff
row. Cheap is STRICTLY below the seven-day mean - `<=` would call every hour of
a flat tariff cheap, and 48 h taken on a Friday is two-thirds weekend.

A profile must tile every day type exactly once, checked at startup; a gap is
refused with the range named rather than filled in. `22:00-07:00 mon-fri` does
not cover Saturday morning, and that trap is what the check catches.

validate.py rule 7's stated reason expired exactly as TARIFF-02 predicted: the
test_price.py section 12 tripwire went red naming static_tariff.py. The message
now says what is true instead, and the tripwire is re-aimed rather than removed
- consumer set pinned, expired phrase asserted gone. Rule 7 also grew a second
refusal for static_profile alongside the HA price pair, the same silent-loser
conflict and a live one.

CFG-03 is not widened: static_profile went into price.py's PRICE_OPTION_DEFAULTS
- the one in-code mirror pinned to config.yaml by a check - and the new module
carries no defaults dict of its own.

Measured on the two committed days of this house's real P1 meter. 2026-08-20
(Thursday): 15.220 kWh imported, 9.606 day / 5.614 night, EUR 4.7185 under
day/night vs EUR 4.6725 flat, shifting saves EUR 0.8069 = 9.606 kWh x the
0.0840 spread, derived twice. 2026-08-23 (Sunday): 0.728 kWh, all of it
night-priced, which is the weekend row proving itself. §17 names six Belgian
scenarios; this is the seventh, and it is two days rather than a season - the
file says so rather than implying an annual figure.

All 12 suites green. Eleven mutations of static_tariff.py were each caught by
at least one check.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 23:07:53 +02:00
glenn schrooyenandClaude Opus 5 616b62d3e9 SAFETY-07: cap the loop before it sets the month's billed capacity peak
checks / unit (pull_request) Successful in 3m7s
TARIFF-01 measured the quarter and said the consumer would be this card. It is
one new module, app/peakguard.py, and one new claim.

THE MECHANISM. Every cycle the guard projects the open 900 s block to its
end-of-interval average - from P1 telegrams only, never from the command - and
compares it to a ceiling. Over the ceiling it files ONE arbiter claim: a lower
bound on the setpoint at P_PEAK. That single shape is both behaviours 10.1 asks
for, a hard clamp on charging and pre-emptive discharge, and it cannot cap
discharge or export, because a floor cannot. 9.1's asymmetry becomes a property
of the object instead of a rule to remember.

    excess = 900 x (projected - ceiling) / seconds remaining
    floor  = MEASURED battery power + excess

The 900/remaining gain is 1.0 at the top of the quarter and grows, so the guard
is gentlest where its projection rests on the least evidence. The floor is sized
from delivered battery power, not the standing command: a floor built on intent
re-adds its own unmet ask every cycle and winds to the rail whenever the inverter
fails to follow, and test_peakguard.py part 1 shows the two diverging.

IT TRIPS, AND THE CAPTURES DECIDED THAT. On the healthy reference day it engages
on zero of 95 quarters. The action is level-triggered, unlatched and re-derived
every cycle, so a false engagement costs one cycle of discharge rather than a
parked battery - the cost SAFETY-18 could not bound and this one can. Replayed on
2026-08-20 the billed quarter goes from 5029.8 W, above the inverter's own 5000 W
rating, to 3485 W, under the soft limit and without saturating. Quarters under
the limit across both committed days: 189/190 -> 190/190. Two days is not thirty
and AC 3 remains TARIFF-01's open item.

BLINDNESS IS CHECKED BEFORE THE PROJECTION IS GRADED, and it is a different
action. A fresh averager projects a confident 0 W, which is under every ceiling
there is; a meter gap longer than meter_max_age_s makes the projection read LOW;
a frozen meter reading export projects 0 W at full coverage. All three become a
floor of 0 W - no importing while the billed quantity is unobservable - which is
one-directional and cannot cost pack energy. That is the backstop SAFETY-18 could
not be: its alarm arrives at 300 s of a 900 s quarter and commands nothing. The
frozen term is freeze.looks_frozen() itself, on this cycle's standing command.

THE CEILING IS peak_cap_w, RAISED ONLY BY AN OBSERVED READING. NO_RECORD and
NOT_OBSERVED both leave it at the cap, so an unreadable peak store makes the
guard stricter and never looser. The gate is the reading's STATE, not whether a
number happens to be attached: today a `peak_w is not None` test behaves
identically, and depending on that coincidence is how a future last-known-peak
field would raise the ceiling off a read that failed. headroom_w() is
deliberately not consumed - cap minus peak turns a zero peak into FULL freedom.

TWO DEFECTS FOUND BUILDING IT, both in the interaction rather than the
arithmetic, and both caught by running the thing rather than by reading it:

  * A LIMIT does not out-vote another LIMIT. arbiter.resolve() applies every
    limit at or above the winner's priority and takes the most restrictive, so
    the peak floor (lo >= 0) and the maintenance charge-only shaping (hi <= 0)
    resolved to a CONTRADICTION and 0 W - the battery parked in the exact quarter
    it should shave. Fixed by suppression, not priority: main.py drops the
    maintenance shaping while the guard is engaged, which is what that branch
    already did for the optional peak_forecast_entity - a key empty on a default
    install, so until now the suspension could not fire on the site that needed
    it.
  * Thresholding `excess > 0` released the claim the instant it began working,
    producing seventeen engage/release cycles in one 900 s quarter and leaving
    that quarter 49 W over the limit it had spent fifteen minutes holding. That
    is 10.2's oscillation. The threshold is now the floor itself, which is the
    same arithmetic read as a level rather than as a switch.

PeakTracker.project() no longer reads a private name with a default.
getattr(averager, "_last_net", 0.0) failed quiet AND disarming - a rename credits
0 W held, the projection reads low, the money guard does not engage.
QuarterAverager.held_net_w is a documented public property now, read without a
default, and Projection carries it. TEST-13 item 4.

NO NEW OPTION AND NO FOURTH DEFAULTS SITE. The ceiling is peak_cap_w, the
coverage threshold is meter_max_age_s, the freeze predicate is meter_freeze_s,
and the achievable discharge comes from max_w and the device rating. config.yaml
and validate.py are untouched; 14's capacity_peak_limit_kw stays declined. On
meter_source: off there are no telegrams, so the guard does not run at all and
the status page says NOT COVERED - running it there blinded it permanently and
stopped every legacy install from charging, which is what the first cut did until
test_clamp.py's export case caught it.

test_peakguard.py: eight parts, all figures derived from the shipped config.yaml
and the committed captures. Part 8 runs seven one-line defeats of the guard and
reports which check bites; six do, and the seventh is reported as NOT caught,
because max(cap, peak) makes a zero peak the strictest ceiling there is and there
is nothing left to catch.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 23:05:05 +02:00
admin 91ea676af4 TARIFF-02: price provider plugins + the canonical all-in price format (#16)
Five providers behind a registry, canonical all-in {start, eur_per_kwh, provisional}. Plugins return RawPoints and declare capabilities().sources; assemble() adds the complement once. register() refuses at import any plugin whose class code mentions a framework key, PRICE_OPTION_DEFAULTS, price_option or _o - closing the three routes review used to double-count. PriceRead.series/no_data as an absence type; all-zero is a series, no_data forbids arbitrage. Additive only: main.py untouched, zero deletions.
2026-08-30 22:35:49 +02:00
adminandClaude Opus 5 862f0fa574 Merge release/1.0 (TARIFF-01) into TARIFF-02
checks / unit (pull_request) Successful in 2m39s
TARIFF-01 (#17) landed on release/1.0 first, so this branch was two
commits behind and Gitea reported the PR unmergeable. Integrating rather
than rebasing: the two reviewed commits stay addressable, and the branch
is protected against force-push anyway.

ONE CONFLICT, AND IT IS THE BORING HALF OF THE ONE REVIEW PREDICTED.
Review of #16 expected collisions in four places - the tail of
`options:`, the tail of `schema:`, the rule list inside
validate_startup_config(), and the tail of validate.py. None of them
happened: TARIFF-01 touched neither config.yaml nor validate.py. It
added app/tariff.py, test_tariff.py, two P1 scenario captures, and edits
to main.py and p1.py, none of which this branch goes near.

The only conflict was CHANGELOG.md, where both branches prepended a
block under the same `## Unreleased` heading. Both blocks kept, TARIFF-02
above TARIFF-01 since it lands second. Nothing was dropped or reworded on
either side, and the resolution adds no line of its own beyond the blank
one between them.

Checked rather than assumed after the merge:
  - all ELEVEN test files pass, including TARIFF-01's own test_tariff.py
    (which this branch had never run) and the 93 s test_loop.py;
  - the 24-mutation battery still has 0 survivors;
  - no semantic overlap in config: TARIFF-01's peak tracker reads the
    peak_cap_w / peak_forecast_entity options that already existed, so
    the price options added here neither shadow nor duplicate anything.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 19:42:16 +02:00
adminandClaude Opus 5 e102cde841 TARIFF-02 review: close the three other doors to the markup, and stop overclaiming
checks / unit (pull_request) Successful in 2m34s
Review of PR #16 wrote a double-counting plugin four ways. PluginOptions
blocked one. The one that decided it touched nothing but
PRICE_OPTION_DEFAULTS - a public dict in the module every plugin must
already import for register() - so "the number is not in scope" was
simply false, and an absolute guarantee a reviewer trusts is worse than
a stated limit, because they stop checking.

register() now refuses, AT IMPORT, any plugin whose own code mentions a
framework-owned name: FRAMEWORK_KEYS plus PRICE_OPTION_DEFAULTS,
price_option and _o. code_leaks() parses the class and drops every
docstring node before searching, because every plugin is REQUIRED to
name its sourced components in prose - engie_dynamic says "markup" in
its docstring and a raw text scan false-positives on the documentation
the same file mandates. A check that fires on required documentation is
a check somebody deletes. test_price.py asserts that a raw scan really
would have false-positived, so the strip is proven load-bearing rather
than asserted.

It fails OPEN if inspect.getsource() cannot find the .py, which the
add-on image always can. Refusing to start a battery controller because
a static analysis could not run would trade a real outage for a
hypothetical accounting error - so instead the checks assert source was
found for all five shipped plugins, and a build that lost it is a red
check rather than a quietly disarmed guard.

All three reachable attacks are written out as plugins in the suite and
each must be refused at registration. The cost is measured, not
asserted: honest wholesale 0.188, every attack 0.219 - one markup high,
every hour, invisible in every log this add-on writes.

The guarantee is now stated at its true size everywhere it appears: the
framework adds the complement exactly once, GIVEN A CORRECT sources
DECLARATION. A plugin that declares {ENERGY} while its feed is all-in
still doubles while touching no framework name, and no code can catch
that - the declaration is an unverifiable claim about somebody else's
tariff. Measured in the suite (0.276) so the limit is a number rather
than a caveat. The docstring/declaration check verifies that a plugin's
two claims AGREE, not that either is true; that now says so.

The three surviving mutations:

M3 - _NoData's reason taking a default. This is the identical survivor
left open on RIG-01's _Unread.cause: the shape was borrowed from
sim/riglock.py including its blind spot, which is what borrowing a shape
does if nobody re-runs the mutations against the copy. Closed by
SIGNATURE - no constructor or handler of an outcome type may carry a
default - so the next one is red without anyone writing a new case.

M5 - the ENERGY-required guard was untested. Its consequence is that
adds becomes {ENERGY, TAXES} and the emitted price silently omits the
energy it is pricing.

M7 - deleting the TLS refusal went green AND the check named for it
still passed, because the old parser stripped a fixed len("https://")
and mangled http://api.energy-charts.info/p into the host
pi.energy-charts.info, which then failed the ALLOW-LIST. An assertion
that does not test what it names, inside a security check. Fixed on both
sides: the scheme is parsed as a scheme, so an http URL now resolves to
its real and declared host and only the TLS rule can refuse it; and the
check asserts the REASON. Fixing it surfaced M7b - the no-scheme guard
was survivable the same way - found by mutating my own fix, and closed
the same way.

Also carried forward: rule 7's refusal is correct forever but its stated
reason ("the provider decides nothing") expires the moment TARIFF-03 or
OPT-04 lands. test_price.py section 12 now greps app/ by AST for a
load_prices consumer and goes red when one appears, telling whoever
lands it to update the message in the same commit. The first draft of
that tripwire used a substring and matched the comment in validate.py
explaining the tripwire - the same false-positive class as the plugin
docstrings, hit twice in one ticket, and fixed the same way.

Documented rather than coded around: vars(read).get("points", []) and
bool(read) both still get past the absence type, and neither is worth
code - defining __bool__ would turn a line that reads like a guard into
one that IS a guard by accident, which is how .get(k, default) got here
in the first place.

Stated plainly in DOCS/CHANGELOG/PR: every fixture is synthetic, so the
five feed URLs and JSON shapes are verified against a belief about what
those suppliers publish rather than against the feeds. A wrong shape
degrades to no-price-data, not to a wrong price.

Review's own DST verification recorded: 149,040 hourly instants,
2020-2036, zero mismatches against zoneinfo. The residual is political,
not arithmetic, and the docstring now says so.

24 mutations driven through the suite, 0 survivors. Whole add-on suite
green. No deleted lines outside this ticket's own files; main.py still
untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 19:35:36 +02:00
admin 0dbd485b5a TARIFF-01: measure the month's billed capacity peak, and make it survive a restart (#17)
The instrument SAFETY-07 will read: the highest clock-aligned 15-minute average offtake of the calendar month, persisted in /data/tariff.db across two retention tiers, with a monthly peak that recompute_month() proves has not diverged from the 15-minute tier. Files no claim, applies no clamp, commands nothing.

AC 1 was already built and unwired - p1.QuarterAverager on the base already does clock-aligned 900 s blocks, time-weighted over irregular spacing, offtake-only, per-phase, with the max_hold_s gap rule; verified on 9aadf46 rather than taken on the PR's word, and the p1.py diff removes zero lines. So this is persistence and reduction rather than a second source of truth for the quantity the money depends on.

Block rather than rolling, and the reading is provable: a 900 s burst centred on a :15 boundary bills at 2000 W and reads 4000 W through a trailing window - exactly 2.00x, the supremum - and since the aligned windows are a subset of all trailing windows, a rolling figure can only ever read HIGH, never low. On the real captures, 1.09x and 1.05x.

Reviewed independently: the encoded EU DST rule graded against zoneinfo at minute resolution over eight transitions, all of 2026 hourly and every month boundary - 0 mismatches - and the wrong-arm fromutc() mutation that differs only inside the one-hour transition window now goes red, which is the class that escaped the first draft. All four persistence behaviours produced rather than read, including a hard os._exit with no close() losing neither a banked quarter nor the open checkpoint. The SAFETY-20 syntax grader discriminates on names; two evasions remain in its import half.

AC 3 is NOT met and the check says so: no 30-day dataset exists. The two days that do exist replay to 189/190 quarters = 99.5 % under the 3500 W soft limit, uncontrolled - the baseline SAFETY-07 must improve on - and 2026-08-20's peak reproduces independently at 5029.8 W, a billed quarter over the live install's own 5000 W rating. Do not close the card as AC 3 met; start the 30-day capture.

Four reviewer mutations survive, recorded on the PR. The one to act on: record_block() ignoring its policy argument stays green, because every block reaching the store in the check is single-phase where NET, SUM and MAX collapse - the third instance of the degeneracy trap this file already documents twice, sitting on the exact line the pending per-phase product decision will change.

No conflict with TARIFF-02 (#16) beyond CHANGELOG.md, verified from both sides.
2026-08-30 19:33:57 +02:00
glenn schrooyenandClaude Opus 5 f4914a174a TARIFF-01: measure the month's billed peak, and make it survive a restart
checks / unit (pull_request) Successful in 2m38s
The 15-minute block average was already built and unwired. p1.QuarterAverager
computes clock-aligned 900 s blocks, time-weighted over irregular telegram
spacing, offtake-only and per-phase; P1Ingest.blocks was a 96-entry list in
memory that a restart erased. So this is the persistence and the monthly
reduction, not a second accumulator - a second one would be a second source of
truth for the exact quantity the money depends on.

BLOCKS, NOT A SLIDING WINDOW. The card says "rolling"; 9.1 bills "the highest
15-minute average offtake" and states its own worked case per QUARTER - one
4998 W quarter moving a monthly peak from 4832 to 5013 W. A trailing window
reports a peak the meter never bills, and reports it HIGH: a burst straddling
:15 fills the window completely while contributing to two block averages at
partial weight.

DELIVERED, NOT INTENDED. tariff.py only ever sees P1Sample - the meter's own
reading - and never a setpoint. SAFETY-20's open question does not arise for a
capacity tariff: the utility integrates the current that flowed.

AN UNREADABLE STORE IS NOT A ZERO PEAK. peak() answers OBSERVED / NO_RECORD /
NOT_OBSERVED with peak_w None unless observed, which is riglock.holder()'s fix
applied to something that spends money. A peak read as 0 W hands SAFETY-07
maximum headroom and disarms the guard at the moment its own storage breaks.

Restart mid-block resumes the accumulator through a new snapshot()/restore()
pair, checkpointed every 15 s and cleared in the same breath as a block close.
The held sample keeps its ORIGINAL timestamp, so an outage is integrated under
the same max_hold_s rule a running process would apply. quarter_start is the
primary key and a re-bank may only ever RAISE a figure.

Calendar months are LOCAL: a quarter starting 22:15 UTC on 31 August bills in
September.

No new option (the limit is the existing peak_cap_w), so config.yaml and
validate.py are untouched and CFG-03 is not widened. No second peak entity:
peak_forecast_entity stays the input to maintenance_charge_floor(), because
repointing that consumer changes what moves energy and belongs to SAFETY-07.

AC 3 is NOT met. It asks for 30 days of replay; two days of this house's real
meter exist and are committed here. test_tariff.py part 9 replays them, reports
the percentage next to the sample size in days, and asserts nothing about 95 %.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 19:13:58 +02:00
adminandClaude Opus 5 852fc7ced6 TARIFF-02: one canonical price format, and a seam that keeps supplier names out of the optimiser
checks / unit (pull_request) Successful in 2m33s
Five provider plugins - generic_dayahead, engie_dynamic,
totalenergies_dynamic, eneco_eddy, manual_csv - all emitting exactly
{"prices": [{"start", "eur_per_kwh", "provisional"}]}, tz-aware at the
Brussels offset, EUR/kWh, all-in. Nothing consumes it yet; the seam
lands first, per the PM's ordering.

It is built in goodwe_controller/app/ rather than at specs.md 2.2's
ems/ingest/prices/, per the product owner's decision of 2026-08-30. The
behaviour 9.3 / 8.3 / 11.4 specifies is unchanged.

Three things shape the design, and each is the mechanism rather than a
comment saying to be careful:

Double-counting the markup is unreachable. A plugin does not return a
price - it returns the figure its source published and declares which of
{energy, markup, taxes} that figure already contains. assemble() adds
the complement, once, in the framework. A plugin is never handed
price_markup_eur_kwh: PluginOptions raises FrameworkOption on it, so the
line that adds the margin a second time cannot be written. Engie and
TotalEnergies publish a retail figure with their margin inside; adding
the configured 3 c/kWh on top would have made every arbitrage decision
wrong by that amount with nothing reporting it.

The product type is declared, not inferred from the plugin's name.
eneco_eddy is real-time/imbalance and gets 8.3's widened thresholds
through capabilities().product and volatility_regime(), never through a
name comparison - which is AC2. test_price.py asserts mechanically that
no plugin id appears anywhere in the framework's code, and drives a
sixth provider defined inside the test file to the right regime with
price.py untouched.

"No price data" is a type, not an empty list - the same rework
sim/riglock.py took under TEST-04. A fetch returns PriceRead.series(...)
or PriceRead.no_data(reason), and only the first carries fields. An
empty series is not constructible; no_data has no .points, no .at(), and
.get("points", []) raises rather than returning the default. A day of
free electricity and a day with no prices are different instructions -
arbitrage is pointless under one and forbidden under the other - and an
implementation returning [] cannot tell them apart. Every site-level
failure returns no_data; nothing raises into the control tick.

Also: injection stays a separate scalar (export_price_eur_kwh, AC4),
negative prices survive the boundary validator (17's regression case),
egress is allow-listed against each plugin's own declaration (16), and
no tzdata is used - the shipped Alpine image has none, so Brussels
offsets come from the EU rule and instants are carried as UTC.

validate.py grows rule 7: an unknown provider id, manual_csv with no
path, and - the HA overlap the PM told us to decide about -
price_provider set alongside price_now_entity/price_avg_entity, which is
two price sources where the HA pair silently wins. The HA sensors are
kept and still work alone; TARIFF-03 and OPT-04 are what replace them.

CFG-03 is not widened: the price defaults have ONE in-code mirror,
PRICE_OPTION_DEFAULTS, which validate.py imports rather than restating
and which test_price.py pins to config.yaml by reading the file.

test_price.py is runnable, assert-free in the house style, and every
verdict it prints is derived. It grades the markup arithmetic with
0.031/0.057 rather than the shipped 0.03/0.06, because 0.06 is exactly
twice 0.03 and a degenerate pair would make double-counting
indistinguishable from correct addition - and it asserts that
non-degeneracy as its first section. The DST checks run on the 23-hour
and 25-hour days, not on a normal day where a fixed 24-slot
implementation and a correct one agree. Thirteen mutations were driven
through it; all thirteen go red.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 19:05:12 +02:00
admin 9aadf469bc TEST-06: pin the cycle() ordering, the NOT COVERED string and the shipped meter_freeze_s (#14)
Closes the three mutations that survived SAFETY-18 suite green. Ordering pinned by consequence (N8 red above the assignment, N3 green below it); NOT COVERED asserted in both directions; shipped meter_freeze_s and its schema bounds pinned. notes_of()/all_notes_of() split so this guard and the whole notification surface are separately nameable, with the three surface-wide claims decided per scenario after SAFETY-03. app/ change is comment-only (AST-identical).
2026-08-30 18:39:07 +02:00
glenn schrooyenandClaude Opus 5 4b521b2df0 ci: trigger a run on the rebased head
checks / unit (pull_request) Successful in 2m33s
No content change. The gate never created a run for 0942228d - the newest
run stayed on d31a12b7, the pre-rebase head - so `checks / unit` had
nothing to report and sat pending. Why no run was created is CI-02 and
belongs to CI-01's owner; this commit only asks for one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 18:11:21 +02:00
glenn schrooyenandClaude Opus 5 0942228d65 Merge release/1.0 (SAFETY-03) into TEST-06
checks / unit (pull_request) Canceled after 0s
Two conflicts, both predicted, both mine as the later merger.

app/main.py - #12 inserted `await self.check_output_clamp()` on the line
immediately above `await self.check_meter_frozen()`, so its hunk abuts my
comment correction. Resolved so EACH CALL CARRIES ITS OWN BOUNDARY,
because the two guards do not share one:

  - check_output_clamp() reads `self.clamp_last`, which is None until
    write_setpoint() has run, so its boundary IS the write;
  - check_meter_frozen() grades `self.target`, assigned further up, so
    its boundary is the assignment.

Letting my correction generalise across both would have been worse than
the vague comment it replaces, because it would read as deliberate. The
AST of app/main.py is identical to merged release/1.0 (+1251 bytes, all
comment).

test_freeze.py - #12 independently added `frozen_notes()` keyed on the
same id. `notes_of()` subsumes it: same id, plus the title fallback that
keeps the id's own check independently falsifiable, plus `all_notes_of()`
for claims about the whole surface.

THE THREE SURFACE-WIDE ASSERTIONS, DECIDED ONE AT A TIME. Measured on the
merged tree:

  scenario                              surface  ids             freeze
  frozen meter, railed command             2     frozen + clamp     1
  TEL-05 converged, -14.2 W                0     -                  0
  moving meter, a real 8 kW load           1     clamp              0
  uncovered transport, no unchanged_s      1     clamp              0

  - the wiring scenario STAYS SURFACE-WIDE and now asserts exactly TWO
    alarms with BOTH IDS NAMED. The answer to "is this still meant to
    raise one alarm" is no, it raises two, and a third guard has to come
    back to this line. That is the one assertion in the file that
    accounts for the controller's entire alarm output;
  - the CONVERGED case stays surface-wide and stays silent: at the
    operating point the whole add-on should have nothing to say, that is
    still true, and narrowing it would give away a real property;
  - the other two NARROW to notes_of(). The clamp alarm there is correct
    and is SAFETY-03's subject: a real 8 kW load does drive the loop past
    the rating, and an uncovered transport does not stop it. Asserting
    whole-surface silence would make this file fail whenever another
    guard correctly does its job. Each still owns the load-bearing half -
    THE FREEZE DETECTOR SAID NOTHING - and the surface is PRINTED so a
    reader sees what else fired without this file grading it.

Re-verified on the merged tree: M1 RED 4, M2 RED 5, M3 RED 3 + RED 2,
N6 RED 1, N3 green (correctly), N8 RED 4, R1 RED 2, R2 RED 1. All nine
suites green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 17:45:09 +02:00
admin 6aabb832fc SAFETY-03: alarm when a computed setpoint exceeds the rating, do not just clip it (#12)
The clamp already existed; the alarm did not, and specs 10.3 records a silently clipped integrator that reached 14,768 W while every dashboard read a legal 5000 W. OUTPUT_CLAMP now alarms on every engagement, with its own reason code, notification_id, status row and counter, distinct from ordinary saturation. Clamp lives in write_setpoint(), the one path every write takes; the alarm runs after it, out of the write path. Not a trip: the clamped value is written and the loop keeps running.

Reviewed over two rounds. Round one held the PR on three findings: clamp_to_rating() returned early from the max_w stage so the device bound was never applied when the first stage engaged (the bigger the overshoot, the less the guard did), and two checks that could not fail - range(CLEAR_CYCLES - 1) issued zero clean cycles at CLEAR_CYCLES=1, and REASSERT_S + 9 passed for any value including an hour of silence. All three fixed in 4e66583 and verified independently: both clamp stages now run on symmetric, asymmetric and wider-device ratings with no escapes; the 492-point sweep was shown to bite where the witness values cannot (a positive-side-only device bound gives 181 escapes with both witnesses passing, and a single-grid-point hole gives 1); and the rewritten checks pin the requirement rather than the constant - CLEAR_CYCLES red at 1 and at 51, REASSERT_S red at <=5 and at >=300.

Author's six mutations all reproduced red, including the cross-suite one where removing the event-log coalescing turns test_freeze red - this guard's log had been evicting SAFETY-18's METER FROZEN? row. Headline claim reproduced: keying the alarm on Decision.reason instead of the i_clamped/clamped booleans takes the 10.3 replay from 2 engagements to 0. AC2 verified with an independent saturation-without-clamping scenario: sat_count 10 at 21.6% of the rating, zero clamp engagements.

Two reviewer mutations survive, both recorded and neither blocking: describe() dropping the bound name (the bound field itself is pinned), and abs() on limit_w (unreachable under the int(100,5000) schema).

First merge in this repository to pass through CI-01's gate rather than around it: 46383c5 merged release/1.0 in so the required context could run at all, and checks / unit (pull_request) reports success in 2m32s.

Note for TEST-06 (#14), which rebases behind this: it conflicts in BOTH app/main.py and test_freeze.py, and its 8 red assertions are its own service-keyed notification counting - keying those four call sites on notification_id turns the whole suite green against this app/ with no change here.
2026-08-30 17:22:30 +02:00
glenn schrooyenandClaude Opus 5 d31a12b7c6 TEST-06: the id-selection note said 'the only assertion'; there are two
checks / unit (pull_request) Successful in 2m32s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 17:10:42 +02:00
glenn schrooyen 2983d7c3b0 Merge remote-tracking branch 'origin/release/1.0' into TEST-06
checks / unit (pull_request) Canceled after 0s
2026-08-30 17:05:35 +02:00
glenn schrooyenandClaude Opus 5 611d3dbcdf TEST-06 review follow-ups: two notification surfaces, the real ordering boundary, the schema bounds
1. notes_of() could not tell this guard's alarm from anybody else's. It
   selected by SERVICE, so against #12's app/ (which adds
   goodwe_output_clamp) eight assertions went red, AC1's core check among
   them. Blanket id-keying is not the repair and was measured, not
   guessed:

       selector                    vs #12's app/   id renamed
       by service (as committed)     8 red           1 red
       by id alone                   3 red           6 red
       by id or title (shipped)      3 red           1 red

   all_notes_of() is the whole surface; notes_of() is this guard's,
   recognised by the id it should carry OR the title it writes - the `or`
   is what keeps the one assertion whose subject IS the id able to fail on
   its own. The three surface-wide claims stay surface-wide, each with a
   comment naming which surface it is about and why the next guard's
   author must decide rather than narrow them. They are the three that go
   red against #12, which is the correct signal.

2. The ordering comment named the wrong boundary. N3 - moving the call to
   immediately before write_setpoint() - is green and correctly so:
   self.target is assigned two lines earlier, write_setpoint() clamps only
   a local and can skip the write entirely. So the invariant is "after
   `self.target = resolution.target_w`", and the detector grades pre-clamp
   INTENT, not the wire. Corrected in test_freeze.py and in main.py's own
   note above the call (comment-only; I merge second and own the
   conflict). N8 - moving the call above the assignment - is RED 4.

3. The schema range's own bounds are now pinned. int(60,3600) ->
   int(1,86400) survived green: an operator could set meter_freeze_s:
   86400 and the guard is dead with nothing red. Both bounds are
   re-derived rather than restated.

4. find_hwsim() crashed the whole suite on a checkout one level below the
   root (HERE.parents[1] -> IndexError, reproduced at C:/goodwe_controller).
   Guarded. And part 3 now names its transport in the summary: the rig
   hwsim and the stub are not equivalent - the rig path failed 1 run in 3
   - so a green log has to say which one backed it.

Also: the "revived transport re-rails" check is labelled a PRECONDITION
like its sibling; freeze_rows() records that flipping p1_enabled exercises
the row's content and never its gate; and the note about the deleted
assertion is corrected - it was REDUNDANT, not undecidable. The reviewer
falsified it in one line (move the hass.call above `if not fired: return`,
RED 9). "I could not construct a falsifier, therefore it cannot fail" was
exactly the inference this file exists to distrust.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 17:05:28 +02:00
glenn schrooyenandClaude Opus 5 46383c5ae5 Merge release/1.0 (CI-01) into SAFETY-03 so the required check can run
checks / unit (pull_request) Successful in 2m32s
Branch protection on release/1.0 now requires the context
`checks / unit (pull_request)`, and CI-01's workflow arrived after this
branch was cut - so no run had ever been triggered for this PR and the
required context would have sat pending forever. Clean merge, no
conflicts: .gitattributes, .gitea/ci/Dockerfile and
.gitea/workflows/checks.yml only. No add-on source or check is touched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 16:03:32 +02:00
admin 5e966ca59f CI-01: this repository's first automated check (#13) 2026-08-30 16:00:42 +02:00
glenn schrooyenandClaude Opus 5 4e66583a52 SAFETY-03 review fixes: both clamp stages always run, and two checks that could not fail
1. BEHAVIOURAL, and a regression this PR introduced. clamp_to_rating() returned
early from the max_w stage, so the device bound was applied only when the rating
bound had NOT fired - the bigger the overshoot, the less the guard did. At max_w
5000 against a +-4000 device, +9999 emitted +5000 W, outside the device range,
while +4500 was correctly held to +4000. It undid the pre-PR guarantee "never
send a value the device will reject outright" and undercut this module's own
case for max_w-not-min(), which rests on that stage catching a validation that
did not run. Both stages now always run; the result names both bounds when both
bite (max_w+device). Asserted over a 492-point sweep, because the defect was
invisible to any single-value check sitting inside one stage.

2. The hysteresis check issued range(CLEAR_CYCLES - 1) clean cycles, which at
CLEAR_CYCLES = 1 issues zero, passes, and prints "0 clean cycles do NOT clear
it" - green for the chatter setting the constant exists to prevent. Now: ONE
clean cycle must not clear the alarm, a literal that does not move with the
constant, and the de-assert count is measured by looping until it clears.

3. The re-assert check stimulated at REASSERT_S + 9, green for any value
including 3600 - the hour of silence the module promises not to produce. Now
stated in wall-clock terms: a loop pinned across the 900 s capacity-tariff
quarter must speak at least 5 times and on no more than 20% of cycles. Pinned in
both directions.

Red confirmed for each: the restored early return (3 checks), CLEAR_CYCLES = 1
(1), REASSERT_S = 3600 (1) and REASSERT_S = 0 (3).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 15:52:09 +02:00
glenn schrooyenandClaude Opus 5 c34b5a2922 TEST-06: close the three gaps that survived SAFETY-18's suite green
None of the three is a bug in the shipped code. They are gaps in what the
suite could detect, and the next person to touch freeze.py had less
protection than the passing count implied.

1. WHERE check_meter_frozen() runs in cycle(). Moving it to the top left
   the whole file green. Pinned by CONSEQUENCE, not by source order, so a
   refactor that keeps the semantics stays green:
     - the alarm must land on the SAME cycle the command reaches the rail
       (at the top it grades the previous command against this cycle's
       meter and arrives a cycle late);
     - the dead-poll-task composition: poll task dies, stale_input_s is
       crossed, the arbiter commands 0 W, and on THAT cycle the alarm is
       already clear - because the detector ran after the 0 W write.
   wired() now returns a per-cycle trace, which is what makes "on which
   cycle" an observation rather than a hope.

2. The NOT COVERED status row. The old check only asserted that an
   uncovered transport is SILENT, which a page claiming `armed` on a guard
   that cannot run also satisfies. The string is now asserted in both
   directions: the uncovered row must say NOT COVERED, name the
   meter_source and homewizard_local, and warn; the covered row must still
   say armed, or "does not say armed" would be satisfied by deleting the
   armed branch.

3. meter_freeze_s. Pinned in test_validate.py by MAINT-04's regex
   technique (no PyYAML in the image), and in test_freeze.py by its
   consequences: the shipped value drives the replays, must stay inside
   the 900 s billing quarter, and 3600/hold_s must stay inside the shipped
   deadband. A Controller built with the key ABSENT must land on the same
   number, which pins main.py's fallback without touching CFG-03.

All three demonstrated red before being accepted green, and every new
check was mutated to confirm it can fail; one candidate assertion ("no new
notification while parked at 0 W") was DELETED because no mutation could
break it, and replaced by the exact notification count across a revived
transport, which can.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 15:50:08 +02:00
glenn schrooyenandClaude Opus 5 9c1a23328a CI-01: this repository's first automated check
checks / unit (pull_request) Successful in 2m39s
Before this commit `admin/goodwe-addon` had no `.gitea/workflows/` directory at
all. Eight runnable test files - `test_loop.py` alone drives the real
`run_control()` for 93 s of wall clock - and not one of them had ever been run
by a machine. Every merge here was a human reading a diff, and the product
owner's requirement that a green CI gate a merge could not be met on a
repository with no CI to be green.

One job, `unit`, with no `paths:` filter, so it reports on every PR and can
therefore be a REQUIRED status check: Gitea leaves a required context it never
receives pending forever, which is the failure mode on the other side of this
ticket. Concurrency is scoped per PR from the start rather than to a bare group
name - the bare name is what let the infra repo's run 521 be superseded while
still pending, so its four jobs concluded `cancelled` with `started_at` at the
Unix epoch and its PR merged on no evidence at all.

The tests run in `.gitea/ci/Dockerfile`: `alpine:3` plus `py3-aiohttp` and
`py3-paho-mqtt` from apk, matching goodwe_controller/Dockerfile exactly, because
pip would install glibc wheels of a library the shipped image gets from Alpine.
The runner itself cannot host them - bare python3, no aiohttp, no pip - and
mutating a shared long-lived runner with `apk add` is invisible to anyone
reading this repo.

Two things measured while building it and recorded in the file rather than
learned again: the checkout must sit two levels below the mount root or
test_freeze.py's `find_hwsim()` dies on `IndexError: 1` before its first
assertion, and test_freeze's hwsim section is silently absent here because
admin/goodwe is private - so its PASS is a weaker claim than it looks, and the
file says so.

Verified green on release/1.0 content before this workflow was written: 8 of 8
files pass, ~4 minutes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 15:46:30 +02:00
glenn schrooyenandClaude Opus 5 c040aac4b8 SAFETY-03: alarm when a computed setpoint exceeds the rating, do not just clip it
The clamp was never the missing half. specs.md 10.3 records an unclamped
integrator climbing ~130 W per 4 s past 10 kW and reporting 14 768 W pinned at
the 5 kW rail; clipped silently that is a flat, legal-looking 5000 W hold and
nothing says the loop asked for three times it.

write_setpoint() - the one path every write takes - now clamps to the VALIDATED
max_w and reports whether the raw value differed; check_output_clamp() raises
OUTPUT_CLAMP as a notification, an ERROR, an event row and a red status banner.
Not a trip: the clamped value is written and the loop keeps running.

The rating is max_w and nothing is min()ed at runtime - CFG-01 already refuses a
max_w outside the device range at startup. The device range is applied second as
defence in depth; it cannot bite after a successful validation, so if it ever
does it alarms too and names the bound. No new config key.

Decision grew i_clamped / clamped booleans, and that is the load-bearing part:
`reason` is one string that slew-limiting and the saturation freeze overwrite,
so an alarm keyed on it is silent on exactly the cycles a winding loop produces.
Mutating main.py back onto the string drops the 10.3 replay from 2 engagements
to 0.

Distinct from saturation by construction - no shared reason code, notification
id, status row or counter - and checked in the negative, after first asserting
the saturation freeze really did engage.

Two things found by running it rather than by reading it: a row per engagement
flushed the 40-row event log and evicted SAFETY-18's own METER FROZEN? row (now
coalesced per episode with an exact count), and a naive clamp turns a nan into a
command at the full rating because every comparison against nan is False.

test_clamp.py is new: assert-based, no framework. Six author mutations
reproduced red, including "clamp kept, alarm deleted". Two of the check's own
hypotheses about the freeze/clamp interaction were refused by its sweep and
rewritten to what it measured.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 15:22:22 +02:00
admin 0cdaa3bd3e SAFETY-18: alarm on a meter frozen at a wrong-but-fresh value (#11)
A meter answering 200 OK with a document that never changes reads healthy to every freshness mechanism in the stack while the loop rails. Detected by a conjunction - unchanged_s >= meter_freeze_s AND |command| within 5% of max_w - which fails in both directions for independent reasons. Alarm only: no claim, no latch, no 0 W.

Reviewed: all six author mutations reproduced red (redder than reported), four more added, three survivors recorded for follow-up. Capture figures re-derived independently. Attack point 4 (dead poll task) measured, not just reasoned. Known follow-up: a device rating that clips max_w by >5% silently disarms the rail term while the status page says armed - does not affect the live install, where dev_max == max_w == 5000.
2026-08-30 03:20:38 +02:00
32 changed files with 60307 additions and 67 deletions
+8
View File
@@ -8,3 +8,11 @@
* text=auto eol=lf
*.png binary
*.gz binary
# `.gitea/workflows/*.yml` steps embed shell heredocs; a `\r` on the terminator
# line means the heredoc never closes and the job fails on a syntax error that is
# invisible in the diff. A Windows checkout with core.autocrlf=true would
# reintroduce it on the next edit. CI-01.
*.yml text eol=lf
*.yaml text eol=lf
Dockerfile text eol=lf
+22
View File
@@ -0,0 +1,22 @@
# The image the add-on's tests run in (CI-01). This repo had NO CI at all before
# that ticket - eight runnable test files, none of them ever run by a machine.
#
# ⚠️ THE DEPENDENCIES ARE THE SHIPPED ONES, FROM APK, AND THAT IS THE POINT.
# goodwe_controller/Dockerfile installs `py3-aiohttp py3-paho-mqtt` from Alpine
# packages rather than pip, because Alpine ships musl and there are no musl
# wheels for aiohttp - pip would compile it from source on a client's Raspberry
# Pi. Testing against pip-installed glibc wheels would be testing a different
# add-on from the one that ships. Same base, same package manager, same names.
#
# ⚠️ NOT `FROM ghcr.io/home-assistant/...-base`. The real Dockerfile takes its
# base from the Supervisor's `BUILD_FROM` build-arg, which is per-architecture
# and not resolvable outside a Supervisor build. `alpine:3` is what those images
# are, minus the s6 supervision the tests do not use - and it is already cached
# on this runner's host, so the gate does not depend on a registry pull.
#
# ⚠️ The runner container itself CANNOT run these tests: it is Alpine with a bare
# python3, no aiohttp, no pip (`No module named pip`; checked on the box, not
# assumed). An image is not gold-plating here, it is the only option that does
# not mutate a shared long-lived runner.
FROM alpine:3
RUN apk add --no-cache python3 py3-aiohttp py3-paho-mqtt
+136
View File
@@ -0,0 +1,136 @@
# CI-01: this repository's FIRST automated check. Before this file, admin/goodwe-addon
# had no `.gitea/workflows/` directory at all - eight runnable test files, ~100 s
# of real coverage including a wall-clock cadence test, and not one machine ever
# ran them. Every merge here was a human reading a diff.
#
# ⚠️ NO `paths:` FILTER, AND ITS ABSENCE IS LOAD-BEARING. Branch protection on
# release/1.0 requires the context `checks / unit (pull_request)`. Gitea leaves a
# required context it never receives PENDING forever, so the moment this workflow
# learns to sit a PR out, that PR becomes unmergeable. Condition a STEP if you
# must; never the workflow.
#
# ⚠️ THE RUNNER IS SHARED WITH THE OTHER REPO'S RIG. `goodwe-runner` on
# 192.168.2.147 carries the labels `self-hosted` and `goodwe-rig` and is
# registered `:host`, and it is the only runner there is - so this job wears the
# rig label without wanting a rig. Nothing below starts, stops, or reads the
# `goodwe-sim` compose project or takes `sim/riglock.py`; it runs one short
# container and exits. It does occupy the runner's single slot (`capacity: 1`),
# so it QUEUES behind a rig campaign - a delayed verdict, never a destroyed one.
#
# NO JS ACTIONS: the runner image has no node, so `actions/checkout` cannot run.
# Checkout is seven lines of git below.
name: checks
on:
pull_request:
branches: [release/1.0, master]
workflow_dispatch:
concurrency:
# ⚠️ PER PULL REQUEST, NOT A BARE NAME. A shared group holds one running and
# one pending run, so a bare name lets a second PR supersede a first PR's run
# before any runner has picked it up - which is exactly what happened to the
# other repo's run 521 (four jobs, `started_at` at the Unix epoch,
# `runner_id: 0`, concluded `cancelled`, and the PR merged anyway). Do not
# replace this with a constant.
group: addon-checks-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
unit:
runs-on: [self-hosted, goodwe-rig]
timeout-minutes: 70 # ~4 to run; the rest is queueing for the single slot
env:
# ⚠️ UNDER /home/homelab, NOT $GITHUB_WORKSPACE (ENV-01, and re-measured
# during CI-01). act's workspace is `/root/.cache/act/<id>/hostexecutor`, a
# path that exists only INSIDE the runner container. Hand it to the host's
# snap-confined dockerd and the build fails with "failed to read
# dockerfile", or - worse, and this is the documented case - a bind mount
# resolves to an EMPTY DIRECTORY with no error at all. /home/homelab is
# mirrored 1:1 into the runner and is under $HOME, where snap confinement
# accepts it. The same probe under /tmp failed and under $HOME succeeded.
WS: /home/homelab/ci/addon-checks-${{ github.run_number }}
steps:
# ⚠️ THE CHECKOUT IS TWO LEVELS BELOW THE MOUNT ROOT, DELIBERATELY.
# test_freeze.py's find_hwsim() reads `HERE.parents[1] / "goodwe" / "sim"`,
# looking for the infra repo as a SIBLING of this one. Check out flat -
# goodwe_controller directly at the mount root - and `parents[1]` does not
# exist, so the test dies on `IndexError: 1` before its first assertion.
# Measured during CI-01; the flat layout was tried first and did exactly
# that. $WS/goodwe-addon/goodwe_controller gives parents[1] == $WS.
- name: checkout into a path the host shares (git; no node for actions/checkout)
env:
URL: ${{ github.server_url }}/${{ github.repository }}.git
TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -eu
rm -rf "$WS" && mkdir -p "$WS/goodwe-addon" && cd "$WS/goodwe-addon"
git init -q .
git remote add origin "$(echo "$URL" | sed "s#://#://x-access-token:${TOKEN}@#")"
git fetch -q --depth 1 origin "$GITHUB_REF" || git fetch -q --depth 1 origin "$GITHUB_SHA"
git checkout -qf FETCH_HEAD
echo "checked out $(git log --oneline -1)"
- name: build the test image (shipped deps, from apk - see .gitea/ci/Dockerfile)
run: |
set -eu
docker build -q -t goodwe-addon-checks:1 "$WS/goodwe-addon/.gitea/ci" >/dev/null
docker run --rm goodwe-addon-checks:1 python3 -c \
'import aiohttp, paho.mqtt.client as m, sys; print("python", sys.version.split()[0], "aiohttp", aiohttp.__version__)'
# ⚠️ THE FILE LIST IS DERIVED FROM THE TREE, NOT TYPED HERE. A hand-written
# list is how a new test file joins the repo and is never run - the exact
# shape of failure this ticket exists to close. The step globs `test_*.py`,
# prints the count, and REFUSES A COUNT OF ZERO: a loop over an empty glob
# exits 0 and would report a green gate that examined nothing.
#
# ⚠️ test_loop.py TAKES ~93 SECONDS OF WALL CLOCK ON PURPOSE (its own
# docstring says so, and says why: it drives the real `run_control()`
# against a real wall-clock floor, and mocking the clock would leave the
# assertion testing the mock). It is not hung. Do not "fix" it with a
# shorter timeout - a 60 s limit was tried during CI-01 and reported
# exactly the false hang the docstring predicts.
#
# ⚠️ WHAT THIS RUN DOES NOT COVER, stated because a pass here would
# otherwise over-claim. test_freeze.py's third section wants the infra
# repo's `sim/hwsim.py` to drive a real HTTP meter through the real
# transport. admin/goodwe is PRIVATE (checked via the API) and this
# workflow's GITHUB_TOKEN is minted for THIS repository only, so hwsim is
# absent here and `find_hwsim()` returns None. The rest of the file runs
# and passes. It does so SILENTLY - the file prints nothing to say the
# hwsim section was skipped - so "test_freeze.py: PASS" below is a weaker
# claim than it looks. That belongs to whoever owns test_freeze.py next;
# naming it here is the least this gate can do about it.
- name: the add-on's own tests
run: |
set -eu
docker run --rm -v "$WS:/w:ro" goodwe-addon-checks:1 sh -eu -c '
cd /w/goodwe-addon/goodwe_controller
set -- test_*.py
if [ "$1" = "test_*.py" ]; then
echo "NO test_*.py FOUND - the mount is empty or the layout moved. NOTHING has been checked."
exit 1
fi
echo "$# test file(s) to run"
rc=0
for f in "$@"; do
start=$(date +%s)
if python3 "$f" > /tmp/out 2>&1; then
echo "PASS $f ($(( $(date +%s) - start ))s)"
else
echo "FAIL $f ($(( $(date +%s) - start ))s)"
tail -40 /tmp/out
rc=1
fi
done
exit $rc
'
- name: clean up
if: always()
run: |
rm -rf "$WS" || true
# Root-owned by this runner inside a directory the `homelab` user owns,
# and sudo on that box is interactive-only, so the user cannot clean
# this up and the job that made it has to.
ls -1dt /home/homelab/ci/addon-checks-*/ 2>/dev/null | tail -n +6 | xargs -r rm -rf || true
+620
View File
@@ -2,6 +2,626 @@
## Unreleased
**OPT-04's AC4 is met: twelve months of real EPEX BE day-ahead prices are now in
the repository, and the strategy has been replayed against them (DATA-02).**
AC4 asked for "net-positive across 12 months of replayed prices". OPT-04 shipped
it as NOT MET and replayed a synthetic year instead, because `scenarios/` held
only P1 meter captures. That was the right call at the time and the wrong
diagnosis of why: the criterion was being carried alongside a 30-day *meter*
criterion as though both needed waiting, and EPEX BE day-ahead is **public data,
not a capture**. It needed a fetch, not a month.
`scenarios/epex-be-dayahead-2025-09-01_2026-09-01.json` — **8760 hourly points**,
2025-09-01 to 2026-09-01 Brussels-local, in §9.3's canonical three-field shape.
`scenarios/fetch_epex_be.py` produced it and `--verify` re-derives every claim it
makes about itself without touching the network.
**Licensing was checked before anything was committed.** The source is
Fraunhofer ISE's Energy-Charts API, which publishes the **BE** zone under
**CC BY 4.0 from Bundesnetzagentur | SMARD.de** and other zones under
private-use-only terms — so this zone is redistributable with attribution and the
data is committed rather than left as a script plus a checksum. The fetcher
**refuses to write a file** whose per-response `license_info` stops saying
CC BY 4.0. Cross-checked against the upstream: SMARD's own `4996/DE` series
matched this feed on all 96 quarter-hours of 2026-04-10, maximum difference
0.00 EUR/MWh.
**The figures are raw wholesale and the file says so.** `"all_in": false`, with
the markup/tax assumption carried in `provenance.all_in_assumption` as *numbers
to apply* rather than numbers already applied — TARIFF-02's double-counting trap
in data form. `test_arbitrage.py` section 7B applies it in one visible place and
**asserts arithmetically** that its two additive terms cancel out of every daily
spread, so the trade counts depend on the VAT multiplier alone and not on the
guessed markup. The same replay is run on raw wholesale as well, so how much of
the answer rests on the guess is reported (110 trades all-in, 88 raw).
**DST is handled by construction, not by a special case.** The fetcher iterates
in UTC and renders through `price.brussels()`, so **2025-10-26 comes out 25
points and 2026-03-29 comes out 23** — verified by counting per Brussels day
rather than asserting 24, and by checking that October's 02:00 appears twice at
`+02:00` and `+01:00` with two different prices while March's 02:00 does not
appear at all. The window is bounded at *local* midnight: bounding it at 00:00
UTC produced a 22-hour first day, and a 24-hour day silently missing from the
end, both of which the point-and-gap accounting caught.
**The result, at the shipped default, whatever it is.** At
`cycle_cost_threshold_eur` **0.10** widened by FCST-03's no-weather **1.5×** to a
required 0.1500 EUR/kWh, the real year completes **110 round trips** and is
**net-positive at +24.88 EUR** of benefit net of the cycle cost (2 kWh a trade).
Unwidened it completes 173. **Nothing was tuned:** the shipped default and the
2.5/1.5 factors are untouched, and every figure comes out of the same `decide()`.
**What the product owner's open decision now has under it.** The four
independent reports that a static day/night spread of ~0.0840 does not clear
0.10 **all stand** — a static-tariff site still never trades. What was missing
was the other half: a *dynamic* site's daily spread clears the widened
requirement on **121 of 365 days**, median daily all-in spread **0.1130**. The
threshold at which a median day would clear is **0.0754**, and at that value the
same year completes 156 trips for +35.42 EUR. Both numbers are reported; neither
is applied.
**And the synthetic year stays, because the real one cannot grade §17.** Not one
of the 365 real days has a negative *dearest* hour — 372 negative wholesale hours
and 0 days where the strategy would have sold into one — so "0 discharges into a
negative-price hour" over real prices is a count of something that could not
happen, and deleting the guard would not change it. Section 7B asserts that
degeneracy instead of quietly reporting the zero; section 7's generator is still
the only thing here that puts a negative price where the strategy would sell.
**Size, measured rather than assumed.** Plain JSON: 893 KB in the working tree
but **74.6 KB as a git object**, against 56.5 KB for a committed `.json.gz`. Git
already compresses blobs, so gzipping saves 18 KB and costs a reviewer the
ability to read the provenance block — including the licence claim they are
meant to check — in the diff. Plain, and `load()` still accepts `.gz` for a
future dataset that earns it.
**Nothing is wired into the add-on.** `generic_dayahead` remains the live fetch
path; nothing under `app/` imports the script, and the dataset is for replay.
**The capacity-peak guard's forced discharge now has an EMS-side energy floor
(SAFETY-24).** SAFETY-07 can command a discharge to hold a billed quarter down,
and nothing below it consulted the state of charge — `soc_floor` belongs to the
maintenance cycle, and freeze.py already recorded that for the grid-following
loop "the BMS is the sole energy backstop". On the 2026-08-20 replay the guard
asks for **878 Wh in a day**. Relying on the BMS is not absurd — it is a real
floor and it works — but it is the pack's protection rather than the operator's,
and a month of aggressive quarters leaves the battery repeatedly at that cut-out
with no EMS-side record of why.
`decide()` now takes `soc` and `soc_floor`, both **required** and neither
defaulted. Above the floor nothing changes. At or below it the guard files a
**0 W** floor instead of a sized one — one-directional, so charging stays blocked
and no discharge is capped — and reports `floor_limited` with the reason it
bound. It is not a trip: every stop in §11.2's table still outranks the claim,
and the floor can never be negative, so §9.1's "discharge and export are never
capped by the peak guard" survives as a property of the object (swept: 0 negative
floors across 1296 combinations of load, elapsed, battery, SoC and floor).
**Failing to shave is not silent.** A floor-limited cycle raises one notification
per episode (`goodwe_peak_soc_floor`), distinct from the saturation alarm because
the two ask the operator for opposite things, and renders its own status row.
An **unreadable SoC is floor-limited, not a full battery** — a full battery is
exactly the value that would disarm the floor, and this project has shipped that
defaulting once before (`| float(100)` in the stage-5 loop).
**Verified by replay, not by argument.** The 2026-08-20 capture is driven against
a modelled pack sized from the day's own measured appetite, so it genuinely
exhausts: without the floor it reaches the BMS cut-out and sits there for 1227
guard cycles; with it the pack stops at 20.0 % and never touches the BMS. The
energy withheld equals the energy between the two floors to within one cycle at
the rail. The honest half is reported too — the worst quarter goes from 4338 W to
5030 W, because a floor-limited guard shaves less on purpose.
**Two new options, mirrored once.** `peak_soc_floor` (20) and `bms_soc_floor`
(10), whose defaults live in `peakguard.PEAK_OPTION_DEFAULTS` — TARIFF-02's
pattern — imported by `validate.py` and `main.py` rather than typed as literals,
and pinned to `config.yaml` by a check that reads the file. CFG-03 still tracks
three places. `validate.py` rule 8 refuses `peak_soc_floor <= bms_soc_floor` at
startup, in CFG-01/MAINT-04's manner. `maintenance_soc_floor` is deliberately
**not** reused and deliberately **not** renamed: it is a drain target, not a
spend limit, and the rename is recorded as a decision to revisit if a third
floor ever appears.
**Price arbitrage - the first strategy that trades on a price (OPT-04).**
Four cards built the price stack and nothing traded on it. `app/arbitrage.py`
is §10.1's priority 3: charge in the cheapest published hour, discharge in the
dearest, **and only when the spread covers `cycle_cost_threshold_eur`**. It is
off by default (`arbitrage_enabled: false`) - it is the first strategy in this
add-on that commands the battery against the meter on purpose.
**It consumes the four cards rather than re-deciding anything.** Whether a
trade is permitted at all, on which two hours, and against which widened
threshold is `TariffPlan.arbitrage()`'s answer (TARIFF-03), and that answer
cannot be obtained without first stating whether the peak guard is holding. So
§9.1's "give peak shaving priority over arbitrage" is honoured
**structurally**: there is no code path to a tradeable spread that skips the
question. And because the claim is filed at `P_ARBITRAGE` (30) while peakguard
files its floor at `P_PEAK` (50), the arbiter clamps an arbitrage charge even
if that interlock were bypassed. Two independent mechanisms, neither
re-implemented here.
**A guard that never ran is not a guard that is clear.** On `meter_source: off`
there is no telegram stream, so the peak guard is never evaluated. That third
state is carried as `None` and it **refuses**: charging at the rail creates the
billed quantity while nothing measures the quarter it lands in. `validate.py`
refuses that configuration at startup too, along with arbitrage armed on a site
with no price source at all, and a SoC floor at or above the ceiling.
**§17: never export into a negative price.** The guard is on the hour being
sold into, not on the spread - a day priced -0.30 ... -0.05 has a 0.25 EUR/kWh
spread that clears any shipped threshold and a dearest hour that is still
negative. Buying in a negative hour stays allowed; being paid to charge is the
right trade.
**Prices are fetched in the background, never on the tick.** `resolve_tariff()`
may open an HTTPS connection, and the control tick is what feeds the ESP32
heartbeat - a supplier's TLS timeout there is a supplier able to push the
add-on towards layer 1's 30 s veto. A separate task refreshes every 30 minutes;
the tick reads an attribute and an age bound, and a plan whose refresh has
stopped answering retires rather than standing as if it were current.
**What is honestly not measured.** The acceptance criterion asked for
net-positivity across 12 months of *replayed* prices. This repository holds no
price history - `scenarios/` is P1 meter data - so that criterion is **not met,
and is not restated**. What `test_arbitrage.py` section 7 does instead is
replay a synthetic year and assert the property that does not need real prices:
every executed round trip cleared its own widened requirement, on all of them,
across a year containing 522 hours priced below zero and zero discharges into
one. And at the shipped 0.10 EUR/kWh threshold a realistic Belgian day/night
site **does not trade at all** - reported, not tuned. That number, and the
widening factors beside it, are one product decision and it is open.
**Provisional prices, and what may not rest on one (FCST-03).**
Day-ahead prices arrive once a day, around 14–16 h CET. Until they do, §8.3 says
the unknown hours use yesterday's profile and are flagged `provisional: true` —
and **no irreversible action may be taken on a provisional price.**
`app/price_forecast.py` does the fill, and the whole module is shaped by one
defect: a carried-forward point and a published one are the same three fields
and a bool, so the natural line `PricePoint(..., provisional=y.provisional)`
copies yesterday's `False` onto today's guess and produces stale data presented
as current. That is the meter-side SAFETY-18 failure one layer up — the reading
arrives on time, looks fresh, and is wrong.
**So the flag is never passed through.** Carried points are built by a
constructor that has no `provisional` parameter at all (the literal `True` lives
in one place, and there is nothing for a caller to forget or copy); published
points are passed through **by identity**, so there is no copy that could lose a
flag; and an audit re-derives the expected flag for every emitted point from the
set of hours the feed actually published and *raises* rather than repairing one.
A published point may itself be provisional — `eneco_eddy`'s current-quarter
imbalance print is — so the audit's rule for a published hour is "whatever the
feed said", never "False".
**The fill keys on the Brussels wall clock, not on `t − 24 h`.** A 23-hour day
has no 02:00 and a 25-hour day has two, so a fixed 24-hour subtraction shifts
every carried hour by one across a transition and misprices a whole day with
nothing to show for it. Re-running the fill after a late publication replaces
the carried hours with the real print: the number and the flag move together
because the series is rebuilt, never patched.
**Arbitrage now drops provisional hours instead of refusing the whole day.**
The refusal that read "the plan is provisional" would have switched arbitrage off
every morning on every dynamic site once carry-forward filling landed — a
part-firm series is the *normal* state before the print. What §8.3 forbids is
acting **on** a provisional price, so the provisional hours are dropped and the
spread, the cheapest hour and the dearest hour are computed over the published
ones only; fewer than two firm hours is still a refusal, which is where the
§11.4 fallback (wholly provisional) lands. The permitting variant now refuses to
be *constructed* around a provisional point at all. The cheap-window flag became
per-hour for the same reason, so a site's maintenance charge floor does not go
dark every morning.
**Real-time / imbalance products get measurably wider thresholds.** One widening
path, driven by a set of active uncertainty sources: the product's own (§9.2 row
2, derived from the declared `PriceProduct` and never from a plugin id) and
§11.4's "no weather data" row, which the spec widens by a different trigger and
which is the same mechanism. The combination rule is `max`, not a product — two
independent doubts must not compound into a threshold nothing can clear. The
caller supplies the base round-trip cost threshold, so OPT-04 still owns what
the number *is*; this owns only how much wider it gets and why, and the allowed
verdict reports the threshold it cleared.
**An irreversible action is a value, not a judgement call at each call site.**
`Irreversible` carries the hours an action commits — the maintenance drain's
window is derived from `maintenance.DRAIN_TIMEOUT_MIN` rather than typed twice —
and the gate looks at exactly those hours, blocking on a provisional one, on an
hour the series does not cover, and on no price data at all: three different
reasons, none of them an answer. ⚠️ It is the entry point for a start being
*justified by a price*; the calendar path main.py runs today is untouched,
because a gate that blocked on `no_data` would stop the monthly cycle on every
default install — §12's opening sentence.
**Zero new options.** No `config.yaml` key, no new defaults dict, no `o.get(k,
X)` in main.py: the module reads no options at all, so CFG-03 did not widen.
**The grid-following loop is now capped against the month's billed capacity peak
(SAFETY-07).** New `app/peakguard.py`. TARIFF-01 measured the quarter; this is the
consumer it was built for. Every cycle the guard projects the open quarter,
compares it to a ceiling and files ONE arbiter claim: a lower bound on the
setpoint at `P_PEAK`. That shape is both behaviours §10.1 asks for — a hard clamp
on charging and pre-emptive discharge — and it cannot cap discharge or export,
because a floor cannot. `excess = 900 × (projected − ceiling) ÷ remaining`, and
the floor is the MEASURED battery power plus that: a floor built on the standing
command winds itself to the rail whenever the inverter fails to follow, and one
built on delivered power cannot.
**It trips where SAFETY-18 only notifies, and the captures decided that.** On the
healthy reference day it engages on zero of 95 quarters; the action is
level-triggered, unlatched and re-derived every cycle, so a false engagement costs
one cycle of discharge rather than a parked battery. Replayed on 2026-08-20 the
billed quarter goes from **5029.8 W — above the inverter's own 5000 W rating — to
3485 W**, under the soft limit and without saturating. Quarters under the limit
across both committed days: 189/190 → 190/190. Two days is not thirty; AC 3 is
still TARIFF-01's open item and this does not claim it.
**Blindness forbids importing, and it is checked before the projection is
graded.** A fresh averager projects a confident 0 W, which is under every ceiling
there is; a meter gap longer than `meter_max_age_s` makes the projection read LOW;
and a frozen meter reading export projects 0 W at full coverage. All three become
a floor of 0 W, which is one-directional and cannot cost pack energy. That is the
backstop SAFETY-18 could not be — its alarm arrives at 300 s of a 900 s quarter
and commands nothing. The frozen term is `freeze.looks_frozen()` itself, evaluated
on this cycle's standing command, not a second copy of the conjunction.
**The ceiling is `peak_cap_w`, raised only by an OBSERVED reading.** `NO_RECORD`
and `NOT_OBSERVED` both leave it at the cap, so an unreadable peak store makes the
guard STRICTER. The gate is the reading's STATE, not whether a number happens to
be attached to it: today a `peak_w is not None` test behaves identically, and
depending on that coincidence is how a future last-known-peak field would silently
raise the ceiling off a read that failed. `headroom_w()` is deliberately not
consumed — `cap − peak` turns a zero peak into FULL freedom.
**Two defects found building it, both in the interaction rather than the
arithmetic.** A LIMIT does not out-vote another LIMIT — `arbiter.resolve()` applies
every limit at or above the winner's priority and takes the most restrictive — so
the peak floor (`lo ≥ 0`) and the maintenance charge-only shaping (`hi ≤ 0`)
resolved to a CONTRADICTION and 0 W, parking the battery in the exact quarter it
should shave. Fixed by suppression rather than priority: main.py drops the
maintenance shaping while the guard is engaged, which is what that branch already
did for the optional `peak_forecast_entity` — a key that is empty on a default
install, so until now the suspension could not fire on the site that needed it.
And thresholding `excess > 0` released the claim the instant it started working,
producing seventeen engage/release cycles in one 900 s quarter (§10.2's
oscillation) and leaving that quarter 49 W over; the threshold is now the floor
itself, which is the same arithmetic read as a level rather than as a switch.
**`PeakTracker.project()` no longer reads a private name with a default.**
`getattr(averager, "_last_net", 0.0)` failed quiet AND in the disarming direction:
a rename credits 0 W held, the projection reads low, and the money guard does not
engage. `QuarterAverager.held_net_w` is now a documented public property, read
without a default, and `Projection` carries it. TEST-13 item 4.
**No new option, and no fourth defaults site.** The ceiling is `peak_cap_w`, the
coverage threshold is `meter_max_age_s`, the freeze predicate is `meter_freeze_s`,
and the achievable discharge comes from `max_w` and the device rating.
`config.yaml` and `validate.py` are untouched; §14's `capacity_peak_limit_kw`
stays declined as the kW/W mismatch trap it is. On `meter_source: off` there are
no telegrams, so the guard does not run at all and the status page says NOT
COVERED — running it there would have blinded it permanently and stopped every
legacy install from charging, which is what the first cut did until
`test_clamp.py`'s export case caught it.
**Static day/night and fully-fixed tariffs, first-class (TARIFF-03).**
§9.2 puts a large share of the Belgian installed base on a static profile, and
an EMS that only works under dynamic pricing is an EMS for a minority. A new
`static_profile` option describes a two-period day/night tariff — or a single
fixed price — as JSON, and `app/static_tariff.py` turns it into the same
canonical `{start, eur_per_kwh, provisional}` series the fetched providers emit,
so nothing downstream learns that a tariff can be static.
**It is not a sixth provider plugin, and the reason is mechanical.**
`PriceProvider.capabilities()` is a classmethod, so a plugin's `PriceProduct` is
fixed when the class is written — but §9.2 makes STATIC and FIXED two different
products with two different required behaviours, and which one a site is on is a
property of the operator's data, not of any class. `StaticProfile.product`
derives it from the number of distinct prices instead. The second reason: the
plugin framework exists to stop a *fetched* figure being marked up twice, and a
number the operator typed as all-in has no publisher to disagree with.
**So the profile sits where §11.4 already says it does — under the providers, as
the fallback.** The configured-product path and the feed-is-down path are the
same generator differing in one flag: `provisional`. A static-tariff site runs
that path every day, so the degradation path for a dynamic site cannot rot
unused.
**"No arbitrage" is a type, not a boolean somebody has to remember to check.**
`TariffPlan.arbitrage()` returns a variant: the *allowed* arm carries the
spread and the cheapest and dearest hours, and the *refused* arm carries a
reason and nothing else — `.spread_eur_kwh` on a refusal raises rather than
answering 0.0. An optimiser cannot compute a trade from a refusal because it
cannot get the inputs out of one. Four refusals, all derived: the peak guard is
holding, there is no price data, the plan is provisional, or **the series is
flat**. That last one is AC2 — a fully-fixed tariff disables arbitrage by
arithmetic rather than by matching on an enum, and a dynamic feed that happens
to print one price all day is refused for the same true reason.
**Peak shaving outranks arbitrage, expressed so OPT-04 cannot skip it.**
`arbitrage()` takes `peak_guard_active` as a required keyword argument, so the
spread is not obtainable without first answering whether §10.1's priority 2 is
holding. §9.1's capacity tariff is a network cost charged whatever the energy
product is.
**Time-window shifting reaches the one shiftable load that exists.** main.py's
cheap-window flag — previously fed only by the `price_now_entity` /
`price_avg_entity` pair — now falls back to the static profile, and the Web UI
gained a **Tariff** row naming the window, the price, when it next changes and
why it is or is not cheap. The rule is *strictly* below the seven-day mean:
under a flat tariff `<=` would call every hour cheap and a fixed-tariff site
would top up from the grid at any hour, which is the opposite of
self-consumption only. Seven days rather than 48 hours because a 48 h mean taken
on a Friday is two-thirds weekend, and the same Tuesday 09:00 would come out
cheap or dear depending on when you asked.
**A profile must tile every day type exactly once, checked at startup.** A gap
is refused with the uncovered range named rather than filled in — an invented
price is one an optimiser would trade on — and an overlap is refused because
otherwise one of two prices wins silently on list order. The trap this catches
in practice: `22:00–07:00 mon-fri` does **not** cover Saturday morning, because
a window applies on the day the clock reads.
**validate.py rule 7's stated reason expired, exactly as TARIFF-02 predicted.**
That ticket left a tripwire in `test_price.py` section 12 asserting that nothing
in `app/` consumed `load_prices()`, with instructions to fix the refusal's
wording in the same commit when it went red. It went red naming
`static_tariff.py`. The message now says what is true instead — the series *is*
consumed, but main.py acts on the static side of it and not on a fetched one —
and the tripwire is re-aimed rather than removed: the consumer set is pinned, so
a second consumer (OPT-04) goes red again, and the expired phrase is asserted
gone so a bad merge cannot restore it. Rule 7 also grew a second refusal:
`static_profile` alongside the Home Assistant price pair is the same
silent-loser conflict, and a live one.
**No new place for an option default (CFG-03 is not widened).**
`static_profile` was added to `app/price.py`'s `PRICE_OPTION_DEFAULTS` — the one
in-code mirror TARIFF-02 created and pinned to `config.yaml` by a check — rather
than to a defaults dict in the new module, which has none. main.py grew no
`o.get("static_profile", …)` fallback. That is TARIFF-02's pattern, stated
here so a reviewer does not have to infer it.
**Measured, on the data that exists.** `test_static_tariff.py` replays the two
committed days of this house's real P1 meter (`scenarios/p1-net-2026-08-{20,23}`)
priced under a Belgian day/night profile. 2026-08-20 is a Thursday: 15.220 kWh
imported, 9.606 in the day window and 5.614 in the night one, EUR 4.7185 under
day/night against EUR 4.6725 flat, and shifting the day-window energy into the
night window saves EUR 0.8069 — which is 9.606 kWh x the 0.0840 spread, derived
twice and required to agree. 2026-08-23 is a Sunday: 0.728 kWh, **all** of it
night-priced, which is the weekend row proving itself on real data. §17 names
six Belgian scenarios and a static-profile case is not among them; this is the
seventh, and it is two captured days rather than a season — the file says so
rather than implying an annual figure.
Also worth knowing before OPT-04: on realistic Belgian day/night numbers the
whole-tariff spread is about 0.08 EUR/kWh, **below** the shipped
`cycle_cost_threshold_eur` of 0.10. A correct optimiser will decline to cycle
for the tariff alone on such a site. That is a product finding, not a defect.
**Price provider plugins, and one canonical all-in price format (TARIFF-02).**
`app/price.py` ships five providers - `generic_dayahead` (wholesale EPEX BE),
`engie_dynamic`, `totalenergies_dynamic`, `eneco_eddy` and the offline
`manual_csv` - all emitting exactly `{"prices": [{"start", "eur_per_kwh",
"provisional"}]}`, tz-aware at the Brussels offset, EUR/kWh, all-in. **Nothing
consumes it yet**: this is the seam, landed first, so that the arbitrage work
that follows never learns a supplier's name.
**Double-counting the markup is refused rather than discouraged, given a
correct declaration.** A plugin does not return a price; it returns the figure
its source published and declares which of {energy, markup, taxes} that figure
contains. The framework adds the complement, once. Engie and TotalEnergies
publish a retail figure with their margin already inside; adding the configured
3 c/kWh on top would make every future arbitrage decision wrong by that amount
with nothing reporting it.
Two mechanisms, because one was not enough. `PluginOptions` raises on the
framework-owned keys - and review of PR #16 then wrote a double-counting plugin
three more ways, one of them touching nothing but `PRICE_OPTION_DEFAULTS`, a
public dict in the module every plugin already imports. So `register()` now also
**refuses at import any plugin whose own code mentions a framework key**
(`code_leaks()`, docstrings stripped via `ast` so a plugin's mandated
`SOURCES:` line does not false-positive). Honest wholesale baseline 0.188
EUR/kWh; every attack landed 0.219.
**The guarantee's precise shape**, stated because an overclaim is worse than no
claim: the framework adds the complement exactly once *given a correct `sources`
declaration*. A plugin that declares `{energy}` while its feed is really all-in
still comes out one markup high while touching no framework name, and no code
can catch that - the declaration is the plugin's unverifiable claim about
somebody else's tariff. What is mechanised is reachability; what stays human is
truthfulness.
**The product type is declared, not inferred from the plugin's name.**
`eneco_eddy` is a real-time/imbalance product and gets widened arbitrage
thresholds; that reaches a consumer through `capabilities().product` and
`volatility_regime()`, never through a name comparison. `test_price.py` asserts
mechanically that no plugin id appears anywhere in the framework's own code,
and drives a sixth provider defined inside the test file to the correct regime
with `price.py` untouched.
**"No price data" is a type, not an empty list.** Following the same rework
`sim/riglock.py` took, a fetch returns `PriceRead.series(...)` or
`PriceRead.no_data(reason)`, and only the first carries fields. An empty series
is not constructible; `no_data` has no `.points` to iterate, no `.at()`, and
`.get("points", [])` raises rather than returning the default. **A day of free
electricity and a day with no prices are different instructions** - arbitrage is
pointless under the first and forbidden under the second - and an implementation
that returns `[]` on a failed fetch cannot tell them apart. Every site-level
failure returns `no_data`; nothing raises into the control tick.
**What is not verified, and cannot be by an offline suite:** every fixture is
synthetic, so the five feed URLs and JSON shapes are checked against a belief
about what those suppliers publish, not against the feeds. The parse layer is
exercised; the contract with each supplier is not. Expect the first live fetch
of each to need a shape fix, and note that a wrong shape degrades to
no-price-data rather than to a wrong price.
**`manual_csv` is the documented offline fallback** and needs no network, no
supplier account and no HTTP stack. See DOCS.md for the format. Its `start`
column must carry a UTC offset - a bare local timestamp is refused, because one
hour every October names two moments.
**Injection is priced separately** (`export_price_eur_kwh`), a scalar rather
than a second series, and negative values are accepted. Consumption plugins
cannot reach it.
**`price_provider` may not be combined with `price_now_entity` /
`price_avg_entity`** - the add-on refuses to start on that pair. The Home
Assistant sensors are the existing delegation of pricing to HA and still work
alone; with both configured they would silently keep deciding while the status
page named a supplier that decided nothing.
**No timezone database is used.** The shipped Alpine image has no `tzdata`, so
Brussels offsets are computed from the EU rule and every instant is carried as
UTC and rendered at the offset that applies to it. The checks drive both 2026
transitions - the 23-hour and 25-hour days - rather than a normal day, where a
fixed 24-slot implementation and a correct one agree.
**The capacity-tariff peak is now measured, banked and survives a restart
(TARIFF-01).** New `app/tariff.py`. The 15-minute block average itself was
already built and unwired - `p1.QuarterAverager` computes clock-aligned 900 s
blocks, time-weighted over irregular telegram spacing, offtake-only and
per-phase - and `P1Ingest.blocks` was a 96-entry list a restart erased. This
adds the SQLite behind it (`/data/tariff.db`): one row per completed quarter in
the indefinite 15-min tier, one row per calendar month in the permanent peak
tier, and no `DELETE` against either anywhere in the module.
**Blocks, not a sliding window.** The card says "rolling"; 9.1 bills "the
highest 15-minute average offtake", stated per QUARTER. "Rolling" here means
continuously updated, not a trailing-15-minutes window - a trailing window
reports a peak the meter never bills, and reports it HIGH, because a burst
straddling :15 fills it completely while contributing to two block averages at
partial weight.
**A restart mid-block resumes it, and a restart that outlives the block banks
it.** `QuarterAverager` grew `snapshot()`/`restore()`; the tracker checkpoints
the open block every 15 s (worst case 1.7 % of one quarter) and clears the
checkpoint in the same breath as a block close. The held sample keeps its
ORIGINAL timestamp across the restart, so the outage is integrated under the
same `max_hold_s` rule a running process would apply and uncovered seconds drag
the closed block DOWN rather than being held forward. `quarter_start` is the
primary key and a re-bank may only ever RAISE a figure, so a replay cannot
double-count and a partial re-run cannot lower a quarter already measured.
**An unreadable store is not a zero peak.** `peak()` answers `OBSERVED` /
`NO_RECORD` / `NOT_OBSERVED` and carries `peak_w = None` unless observed - the
`riglock.holder()` fix applied to something that spends money. A peak read as
0 W hands SAFETY-07 maximum headroom and silently disarms the guard at the
moment its own storage breaks, so there is no accessor that defaults it: a
caller who forgets gets a `TypeError`, and the status page prints a different
row for "no quarter yet" than for "the database did not answer".
**Calendar months are LOCAL, blocks are UTC.** A quarter starting 22:15 UTC on
31 August bills in September. Block alignment still needs no timezone.
**No new option, and no second peak entity.** The soft limit is the existing
`peak_cap_w`; `config.yaml` and `validate.py` are untouched, so CFG-03 is not
widened. `peak_forecast_entity` stays the input to `maintenance_charge_floor()`
- repointing that consumer changes what moves energy and belongs to SAFETY-07.
**It measures and nothing else.** No claim, no clamp, no command; graded on the
parsed syntax in `test_tariff.py`, which also proves the same grader lights up
on `main.py`. Per SAFETY-20, the quantity is DELIVERED power - `tariff.py` only
ever sees `P1Sample`, the meter's own reading, never a setpoint.
**AC 3 is not met and the check says so.** It asks for 30 days of replay; two
days of this house's real meter exist (2026-08-20, 2026-08-23), now committed as
`scenarios/p1-net-*.jsonl.gz`. `test_tariff.py` part 9 replays them, reports
the percentage under the shipped limit next to the sample size in days, and
asserts nothing about the 95 % figure.
**A computed setpoint over the rating now alarms instead of being clipped in
silence (SAFETY-03).** The clamp was never the missing half - specs.md 10.3
records an unclamped integrator climbing ~130 W per 4 s past 10 kW and reporting
**14 768 W** while pinned at the 5 kW rail, and a silent clamp turns exactly that
into a flat, legal-looking 5000 W hold. `write_setpoint()` - the one path every
write takes, cycle, heartbeat, operator button and shutdown alike - now clamps to
the **validated** `max_w` and reports whether the raw value differed, and
`Controller.check_output_clamp()` raises `OUTPUT_CLAMP`: a persistent
notification (`goodwe_output_clamp`), an `ERROR`, an event row and a red status
banner. **Not a trip**: the clamped value is written and the loop keeps running,
per 11.2's "clamp *and alarm*".
**The bound is `max_w` and nothing is `min()`ed at runtime.** Section 14 requires
the loop clamp and the driver's `setpoint_limit_w` to agree, validated at startup
with a refusal on mismatch - which CFG-01 already ships. The device range is
still applied second, as defence in depth; after a successful validation it
cannot engage, so if it ever does, that is louder than the first stage and it
alarms too, naming the bound. No new config key: the guard's limit is the one
the operator already set, and its cadence is a module constant.
**`Decision` grew `i_clamped` / `clamped` booleans, and that is the load-bearing
part.** `reason` is one string that later stages overwrite - a cycle that clamps
the integrator and is then slew limited reports `slew-limited`, and one that
clamps while saturated reports `saturated-freeze`. Both *are* the winding case,
so an alarm keyed on the string is silent on exactly the cycles it exists for:
mutating main.py back onto `"clamp" in decision.reason` drops the 10.3 replay
from 2 engagements to **0**. The flags are set where the clamp happens and are
never overwritten. On a default install (`integrator_max_w` following `max_w`)
the integrator bound is the one that bites and the output clamp is unreachable,
so the integrator flag is the whole signal.
**Distinct from the saturation freeze, and checked in the negative.** A clamp is
a *software* bug signal; saturation is the *hardware* failing to follow a legal
command, and its action is to freeze the integrator, not to alarm. They share no
reason code, notification id, status row or counter. `test_clamp.py` drives a
loop that saturates for 10 consecutive cycles at 1.4 % of the rating and asserts
zero clamp notifications, zero engagements, no log row and no status row - after
first asserting that the saturation freeze *really did* engage, so the silence is
an observation rather than a scenario that did nothing.
**The action log is coalesced per episode, and that was a real bug.**
`log_event()` keeps 40 rows; one row per engagement flushed the entire ring
within 40 cycles of a winding loop, and the first thing it evicted was
SAFETY-18's own `METER FROZEN?` row - this guard silencing another guard's log.
Caught by `test_freeze.py` going red, not by inspection. One row per episode now,
carrying the exact engagement count read off the counter.
**A non-finite setpoint is refused, and it was worse than passing one through.**
`nan` is loss of input (11.1 rule 1: command zero, keep commanding), not an
out-of-range value. Measured: because every comparison against `nan` is False, a
naive `max(-limit, min(limit, nan))` keeps its *first* argument and turns a `nan`
into a command **at the full rating**, then reports it as a clamp engagement.
**Review fixes (PR #12).** Three, and the first was a behavioural regression
this PR introduced: `clamp_to_rating()` **returned early** from the `max_w`
stage, so the device bound - the "applied second, defence in depth" half - was
reached only when the rating bound had NOT fired. The bigger the overshoot, the
less the guard did. At `max_w` 5000 against a +-4000 device, +9999 emitted
**+5000 W, outside the device range**, while a mere +4500 was correctly held to
+4000. That undid the pre-PR guarantee "never send a value the device will
reject outright" and undercut this module's own case for clamping to `max_w`
rather than a runtime `min()`, which rests entirely on that stage catching a
validation that did not run. Both stages now always run, the result names both
bounds when both bite (`max_w+device`), and the property is asserted over a
492-point sweep rather than on witness values - the defect was invisible to any
single-value check that happened to sit inside one stage. No reachable
production path was found for it (two validation gates and the arbiter bound sit
in front), but latent is not harmless.
Two harness defects, both of the kind where a check cannot fail for the value it
forbids: the hysteresis check issued `range(CLEAR_CYCLES - 1)` clean cycles,
which at `CLEAR_CYCLES = 1` issues **zero** and passes while printing
"0 clean cycles do NOT clear it" - green for the chatter setting the constant
exists to prevent. And the re-assert check stimulated at `REASSERT_S + 9`, green
for **any** value including 3600, i.e. the hour of silence the module promises
not to produce. Both are now written in wall-clock terms that do not move when
the constant does: **one** clean cycle must not clear the alarm, and a loop
pinned across the 900 s capacity-tariff quarter must speak at least 5 times and
on no more than 20 % of cycles. `CLEAR_CYCLES = 1`, `REASSERT_S = 3600` and
`REASSERT_S = 0` each go red.
**`test_clamp.py`** (new, assert-based, no framework) covers the clamp
arithmetic and its boundaries - exactly +-`max_w` is *not* an engagement - the
alarm's edge/re-assert/hysteresis cadence, the 10.3 runaway replayed through the
real `Controller.cycle()` against an unclamped transcription, the saturation
negative, and the direct-write path. Six author mutations reproduced red,
including "clamp kept, alarm deleted" (the historical build) and "alarm keyed on
`reason`". Two of the check's own hypotheses about how the freeze and the clamp
interact were **refused by the sweep** and rewritten to what it measured: a
frozen cycle *can* clamp, and does so exactly when the integrator reverses.
`test_freeze.py`'s notification assertions now filter by `notification_id`
rather than by service - the add-on raises more than one kind of notification,
and its railing scenarios legitimately raise this one too.
**Cheap-window maintenance charging was unreachable at the shipped defaults, and
now the config that makes it unreachable is refused (MAINT-04).** `config.yaml`
shipped `max_w: 2000` against `maintenance_charge_w: 2500`. The cheap-window
+368 -8
View File
@@ -420,7 +420,7 @@ help - see "Why the tuning is what it is".
| `maintenance_start_hour` | 10 | Hour of day a due cycle begins |
| `maintenance_discharge_w` | 2000 | Drain rate (exports the surplus). Above `max_w` it is **clamped** to `max_w` — the drain phase SETs the setpoint and the arbiter bounds it, saying so on the status page (`limited by supervised(max_w)`). Warned about, not refused: the cycle still completes, just slower |
| `maintenance_charge_w` | 2000 | Charge ceiling, capped again by peak headroom. Must stay **at or below `max_w`** or the cheap-window shaping (`hi = -maintenance_charge_w`) contradicts the supervised limit (`lo = -max_w`), the arbiter resolves it to **0 W**, and maintenance silently never charges in a cheap window — **the add-on refuses to start** |
| `maintenance_soc_floor` | 11 | Drain target — stay just above the inverter's own floor |
| `maintenance_soc_floor` | 11 | Drain target — stay just above the inverter's own floor. ⚠️ **This is not `peak_soc_floor`.** It is the depth the monthly recalibration deliberately goes to, once, on purpose; `peak_soc_floor` is the depth a money-saving discharge may never reach on any day. The right values differ (11 vs 20 as shipped), which is why they are two keys |
| `maintenance_soc_target` | 99 | Charge target |
| `maintenance_hold_min` | 120 | Hold at full so the BMS can balance |
@@ -430,6 +430,8 @@ help - see "Why the tuning is what it is".
|---|---|
| `peak_forecast_entity` | Quarter-hour demand forecast, for capacity-tariff markets. Empty = no cap |
| `peak_cap_w` | The site's capacity-tariff target |
| `peak_soc_floor` (20) | **The percent below which the capacity-peak guard will not discharge the battery.** Not optional and not the same thing as `maintenance_soc_floor` — see below |
| `bms_soc_floor` (10) | The pack's **own** cut-out percent, the number the BMS enforces whatever you set above. `peak_soc_floor` must be **strictly above** it or the add-on refuses to start |
| `price_now_entity`, `price_avg_entity` | Dynamic tariff. Empty = never force a paid grid top-up |
On a capacity-tariff site the maintenance charge is capped by the headroom left
@@ -439,6 +441,11 @@ maintenance schedule.
#### Capacity-tariff exposure of the grid-following loop
⚠️ **This section described the exposure BEFORE SAFETY-07. The loop is now
capped — see "The capacity-peak guard" below — and what follows is kept because it
is still the clearest statement of WHAT is being guarded and why the number is
worth what it is.**
⚠️ **`peak_cap_w` caps MAINTENANCE charging. It does not cap the loop.** The
grid-following loop is bounded by `max_w` and by the device rating, and by
nothing else. That is deliberate — the loop exists to *reduce* what crosses the
@@ -463,16 +470,162 @@ What exists today, and what does not:
|---|---|
| `max_w` bounds the magnitude | ✅ — but the exposure IS the magnitude |
| Maintenance charging capped under `peak_cap_w` | ✅ |
| **Loop capped under `peak_cap_w`** | ❌ — not built. This is SAFETY-07 |
| **Loop capped under `peak_cap_w`** | ✅ — SAFETY-07, on every `meter_source` but `off` |
| Frozen meter detected | ✅ on `homewizard_local` only, **alarm only** |
| Frozen meter *stopped* | ❌ — the alarm does not command anything |
| Frozen meter *stopped* | ✅ — SAFETY-07 forbids importing while the meter is frozen |
| SoC guard on the loop | ❌ — the BMS is the only energy backstop |
So the frozen-meter alarm below **narrows the window; it does not close it.** On
a capacity-tariff site where an unattended runaway is unacceptable, the mitigation
today is operational: set `max_w` no higher than the loop actually needs (the
tuning was measured at 2000 W and a 1.78 kW load step is tracked fine there), fit
the e-stop, and watch for the alarm.
On the legacy `meter_source: off` path there are no P1 telegrams, so there is no
quarter-hour average to guard, and the status page says **NOT COVERED**. There
the mitigation is still operational: set `max_w` no higher than the loop actually
needs (the tuning was measured at 2000 W and a 1.78 kW load step is tracked fine
there), fit the e-stop, and watch for the alarm.
### The capacity-peak guard
**What it does.** Every cycle the add-on projects the open quarter to its
end-of-interval average — from P1 telegrams only, never from the command — and
compares it to a ceiling. Over the ceiling it files one claim: a **lower bound on
the setpoint**, at `P_PEAK`. That single shape is both behaviours §10.1 asks for:
a hard clamp on charging while the bound sits above the standing charge, and
pre-emptive discharge while it sits above zero. It can never cap discharge or
export, because a floor cannot.
**The number.** `excess = 900 × (projected − ceiling) ÷ seconds remaining`, and
the floor is the **measured** battery power plus that. The 900/remaining factor
is the design rather than an accident: it is 1.0 at the top of the quarter and
grows, so the guard is gentlest when its projection rests on the least evidence
and becomes aggressive only once the quarter is nearly measured.
**The ceiling** is `peak_cap_w`, raised to the month's banked peak when that
already stands higher — once a month's bill is set at 5013 W, holding later
quarters to 3500 W buys nothing and costs real charging. **That relaxation is
granted only by an OBSERVED reading:** a peak store that cannot be read leaves the
ceiling at the cap, so a broken instrument makes the guard stricter, never looser.
**When it cannot see, it forbids importing.** Three states are not evidence and
are checked before the projection is graded: no open quarter (a fresh averager
projects a confident 0 W, which is under every ceiling there is), a meter gap
longer than `meter_max_age_s`, and SAFETY-18's frozen-meter conjunction. In all
three the claim becomes a floor of **0 W**. That is the backstop for the frozen
meter: the alarm arrives at `meter_freeze_s` (300 s) of a 900 s billing quarter
and commands nothing, and this stops the import.
**It trips, where the freeze detector only notifies,** and the captures are why.
On the healthy reference day (2026-08-23) it engages on **zero** of 95 quarters.
The action is level-triggered and unlatched — re-derived from a fresh projection
every cycle, released the instant the projection drops back — so a false
engagement costs one cycle of discharge rather than a parked battery. And it is a
LIMIT, so every stop in the watchdog table still outranks it.
**Replayed on 2026-08-20**, the day the vendor controller failed: the billed
quarter measured **5029.8 W**, above the inverter's own 5000 W rating, and the
guard brings it to **3485 W** — under the soft limit, without ever asking for more
than the inverter can give. Across both committed days the quarters under the
limit go from 189/190 to 190/190. Two days is not thirty and does not close AC 3;
it is the comparison two days can support.
**What it alarms on is saturation, not engagement.** An engagement is an ordinary
control action and goes to the action log and the status page. A *saturated*
guard — asking for everything the inverter has while the quarter is still
projected over — raises one notification (`goodwe_peak_saturated`), because that
is the one state where the money will be spent and shedding load is the only
thing left that changes the bill.
**No new option for the CEILING.** It is `peak_cap_w`, which already shipped. The
coverage threshold is `meter_max_age_s`, the freeze predicate is
`meter_freeze_s`, and the achievable discharge comes from `max_w` and the device
rating — every one of them a key that already exists.
#### The energy floor under the forced discharge (`peak_soc_floor`)
The guard above can **command a discharge**, and until SAFETY-24 nothing in the
add-on bounded how far down it took the pack — the BMS was the only backstop.
On the 2026-08-20 replay the guard asks for **878 Wh in a day**.
That is not absurd: the BMS floor is real and it works. But it is *the pack's*
protection, not *yours*. A month of aggressive quarters can leave the battery
repeatedly parked at that cut-out with no record on this side of why, and the
first symptom — a battery that is never available when you want it — looks
exactly like the SoC drift the monthly maintenance cycle exists to correct.
- **`peak_soc_floor` (20 %)** is where the forced discharge stops. Below it the
guard files a **0 W** floor instead of a sized one: charging stays blocked
(that can only ever lower your offtake), and no further discharge is asked for.
- **It does not stop the ordinary self-consumption loop.** That loop still has no
SoC guard of its own. This bounds what the *peak guard* spends for money.
- **When it binds you are told**, once per episode, with notification
`goodwe_peak_soc_floor` and a status-page row reading `PEAK GUARD
FLOOR-LIMITED`: the quarter **will** go over the ceiling, so shedding load is
the only thing left that changes the bill. Failing to shave is never silent —
the euro figure is the whole point of the guard.
- **An unreadable SoC is treated as floor-limited, not as a full battery.** A
full battery is exactly the value that would disarm this floor.
- **`bms_soc_floor` (10 %)** is the pack's own cut-out — register `0x0560` on the
reference GW5000S-BP, consistent with DoD 90. It exists so the add-on can
**refuse to start** when `peak_soc_floor` is at or below it: such a floor can
never be the thing that stops the discharge, and a guard that advertises a
backstop it does not have is the one thing this project will not ship. Equal is
refused too — equal buys zero margin.
Why 20 %: the reference pack is 13.8 kWh usable, so 10 points of margin over the
BMS cut-out is about 1.6× the guard's worst measured day. If your pack is
smaller, or your quarters are worse, raise it.
### The winding-loop alarm (the output clamp)
Every setpoint this add-on writes is clamped to `max_w` before it leaves the
process, and **when that clamp actually engages, it alarms.** The clamp on its
own is not the feature - this project already clipped, and the clipping is what
hid the defect the guard exists for: with the inverter switched off entirely an
unclamped integrator climbed ~130 W per 4 s past 10 kW and, pinned at the 5 kW
rail, reported **14 768 W**. Clipped silently, that is a flat, legal-looking
5000 W hold on every dashboard, and nothing anywhere says the loop asked for
three times it.
**What "the rating" is.** The bound is `max_w`, and only `max_w`. It is already
validated against the setpoint device's own range at startup - the add-on
*refuses to start* if `max_w` exceeds what the device accepts (and again once
ESPHome is up and the real rating is known), so by the time the loop runs the two
numbers agree. The device range is applied a second time inside the write as
defence in depth; after a successful validation it cannot bite, and if it ever
does, that engagement alarms too and names the bound.
**Clamping is not saturation, and the two never share a channel.**
| | what it means | what happens |
|---|---|---|
| **clamp** | the *computed* value exceeded the rating | a software bug signal - something upstream is winding. **Alarm.** |
| **saturation** | the inverter is not following a perfectly *legal* command | a hardware condition - freeze the integrator. **No alarm.** |
A loop can be legally saturated all day: a cold pack, a curtailed inverter, an
export limit. Routing the clamp through a saturation-shaped alarm puts the bug
signal inside the noise, which is the same silence as clipping quietly. They
share no reason code, no notification id, no status row and no counter.
**It is an alarm, not a trip.** The clamped value is written and the loop keeps
running - this is not the sign-disagreement failsafe, which commands 0 W. You
get a persistent notification (`goodwe_output_clamp`, replaced rather than
stacked), an `ERROR` in the log with the reason code `OUTPUT_CLAMP`, a row in
the event list, and a red status banner while it is engaged. Once it clears, a
warning row keeps the engagement count so the evidence does not vanish with the
condition. Consecutive engagements share one event row carrying the exact count,
because the event list keeps only 40 rows and a row per cycle would flush every
other guard's entries out of it.
**There is nothing to configure**, deliberately. The bound is `max_w`, which you
already set; giving the guard a limit of its own would be a second number that
can disagree with the first, which is exactly the failure the startup validation
exists to prevent. The alarm's cadence - first engagement, then re-asserted every
30 s while it persists, cleared after 3 clean cycles - is fixed for the same
reason `RAIL_FRAC` is: it describes the notification bus, not the plant.
WARNING: **on a default install the engagement you will see is the integrator's,
not the output's.** The control law bounds the integrator to `max_w` and *then*
derives the command from it, so the command itself can never come out over the
rating unless `integrator_max_w` has been deliberately raised above `max_w`.
Both are watched. A guard that watched only the emitted value would be silent on
every standard configuration.
### The frozen-meter alarm
@@ -526,6 +679,213 @@ unchanged is 12 W, which is inside the 15 W deadband — so the command there is
parked at 0 W and the two halves of the rule cannot both be true. At the schema
floor of 60 s that figure is 60 W and only the margin is left.
### Energy price providers
> **A fetched price moves energy only when you switch arbitrage on.** This
> release ships the five provider plugins, the canonical all-in format they
> emit, the config that selects one, and — new in OPT-04 — a price-arbitrage
> strategy that trades on them. With `arbitrage_enabled: false`, which is the
> shipped default, turning a provider on still fetches, validates and publishes
> prices and changes no charging decision.
>
> `static_profile` below needs no network at all, so its cheap window is
> computed on every cycle and reaches the maintenance cycle's grid top-up
> whether or not arbitrage is armed.
| option | meaning |
|---|---|
| `price_provider` | `off` (default) or one of the five plugins below |
| `price_markup_eur_kwh` | Your supplier's margin, EUR/kWh. **Added only when the chosen plugin's feed does not already contain it** |
| `price_taxes_eur_kwh` | Levies and VAT, EUR/kWh. Same rule |
| `export_price_eur_kwh` | The **injection** price. A separate scalar, not a series — see below |
| `price_csv_path` | Where `manual_csv` reads its file |
| `price_feed_url` | Override the plugin's default feed. Only a host the plugin itself declares is accepted |
| `price_horizon_h` | How far ahead to ask for, hours |
| `price_timeout_s` | Per-request timeout |
| `static_profile` | A static day/night or fully-fixed tariff. Empty by default — see below |
**Which plugin sources what.** This is the one thing to get right, because
adding your markup to a price that already contains it makes every future
arbitrage decision wrong by that amount, in a direction nothing reports.
| plugin | product | the published figure contains | the add-on adds |
|---|---|---|---|
| `generic_dayahead` | day-ahead | energy only (wholesale EPEX BE) | markup **and** taxes |
| `engie_dynamic` | day-ahead | energy + Engie's markup | taxes |
| `totalenergies_dynamic` | day-ahead | energy + TotalEnergies' markup | taxes |
| `eneco_eddy` | **real-time / imbalance** | everything, all-in | nothing |
| `manual_csv` | day-ahead | everything, all-in | nothing |
You cannot get this wrong by editing the markup: each plugin *declares* what its
feed contains and the add-on adds only the rest. Set a markup under a plugin
that already carries one and it is **ignored, with a line in the log saying so**
— never applied twice.
That holds **given a correct declaration**, and it is worth knowing where the
line is. What the add-on enforces mechanically is that a plugin cannot obtain
the markup or apply it: the numbers are not reachable from plugin code, and a
plugin whose code mentions them fails to load at all. What it cannot check is
whether a plugin described its supplier's feed truthfully — that is a claim
about somebody else's tariff, and no code can verify it. If you write a plugin,
the `sources` declaration is the part a reviewer has to read.
`eneco_eddy` is a real-time/imbalance product rather than a day-ahead one. It is
flagged as high-volatility, which widens arbitrage thresholds once the
optimiser lands. That comes from the plugin's own declaration, not from its
name.
**`manual_csv` — the offline fallback.** It needs no internet, no supplier
account and no HTTP stack at all, and it is the supported way to run this
add-on price-aware on a site with no outbound connectivity. Write a file and
point `price_csv_path` at it:
```csv
start,eur_per_kwh,provisional
2026-01-15T00:00:00+01:00,0.1834,false
2026-01-15T01:00:00+01:00,-0.0210,true
2026-01-15T02:00:00+01:00,0.3115,false
```
- `start` **must carry its UTC offset** (`+01:00` in winter, `+02:00` in
summer). A bare `2026-01-15 00:00` is refused, because for one hour every
October a local wall clock names two different moments.
- `eur_per_kwh` is **all-in** — energy *plus* markup *plus* taxes. This plugin
has no way to know what your numbers already include, so it assumes
everything and adds nothing.
- Negative prices are valid and are kept. They are the best charging hours of
the year.
- `provisional` is optional; omit the column and every hour is treated as
settled.
**When there are no prices at all** — provider off, feed unreachable, CSV
missing — the add-on does not guess. It reports *no price data*, which is a
distinct state from "prices, all of them zero": the plan is marked provisional
and arbitrage is disabled, per the degradation matrix. It never falls back to a
price of zero, which would read as free electricity.
**Injection is priced separately** (`export_price_eur_kwh`). In Belgium it is
often near zero, which by itself strongly favours self-consumption over export.
It is a single number rather than a curve because no supported supplier
publishes a dynamic injection tariff; negative values are accepted.
**The old Home Assistant price sensors.** `price_now_entity` and
`price_avg_entity` still work and are unchanged: the add-on compares them and
uses the result as a cheap-window hint for the maintenance charge. They are
**not compatible with `price_provider`, and not with `static_profile` either**
— configure two of them and the add-on refuses to start, naming the conflict,
because the add-on reads exactly one and the loser is silent. Use one.
### Static day/night and fully-fixed tariffs (`static_profile`)
Most Belgian households are **not** on a dynamic contract, and an EMS that only
works under hourly pricing is an EMS for a minority. `static_profile` is how a
two-period day/night tariff or a single fixed price becomes a first-class
configuration rather than "turn the price feature off and hope".
It is one option, holding a JSON object, on one line:
```json
{"periods": [
{"name":"day", "from":"07:00","to":"22:00","days":"mon-fri","eur_per_kwh":0.34},
{"name":"night", "from":"22:00","to":"07:00","days":"mon-fri","eur_per_kwh":0.26},
{"name":"night", "from":"00:00","to":"24:00","days":"sat,sun", "eur_per_kwh":0.26}]}
```
A **fully fixed** tariff is one period covering the whole week:
```json
{"periods": [
{"name":"fixed","from":"00:00","to":"24:00","days":"all","eur_per_kwh":0.31}]}
```
- `eur_per_kwh` is **all-in** — energy *plus* your supplier's markup *plus*
taxes — exactly like `manual_csv`. `price_markup_eur_kwh` and
`price_taxes_eur_kwh` are **not** added to it.
- `days` takes `mon`…`sun`, ranges (`mon-fri`, `fri-mon`), comma lists
(`sat,sun`) and the words `weekday`, `weekend`, `all`.
- Times are **local Belgian time** and windows are half-open: `07:00`–`22:00`
includes 07:00 and excludes 22:00. A window may cross midnight.
- The clock changes twice a year and the add-on handles it. It generates prices
against absolute instants, so a spring-forward day is 23 hours and an
autumn-back day is 25.
> ⚠️ **A window belongs to the day the clock reads, not the day it started.**
> `22:00`–`07:00` on `mon-fri` covers Friday 23:00 but **not** Saturday 03:00 —
> Saturday 03:00 is Saturday's. That is why the example above has a third row
> for the weekend. Leave it out and the add-on **refuses to start**, naming the
> uncovered hours.
**Every day type must be covered exactly once.** The periods have to tile all
seven days with no gap and no overlap, and the add-on checks that at startup. A
gap is refused rather than filled in, because a made-up price is one the
optimiser would trade on; an overlap is refused because otherwise one of the two
prices silently wins on the order you typed them in.
**What it changes today.** The add-on works out which window you are in and
whether that window is *cheap* — strictly below the tariff's seven-day average.
The Web UI shows both on a new **Tariff** row, and the maintenance cycle's
cheap-window grid top-up uses it, which is exactly the "shift load into the
cheap window" behaviour a day/night tariff asks for. Nothing else moves energy
on a price yet.
Under a **fully fixed** tariff every hour equals the average, so there is *no*
cheap window by construction and the add-on runs self-consumption only — which
is the correct behaviour for that product, not a limitation. Peak shaving, the
capacity-tariff tracker, the reserve floor and the maintenance schedule are all
unaffected: the capacity tariff is a network cost charged whatever your energy
contract is.
**Recommended even on a dynamic contract.** When a price feed is unreachable the
add-on falls back to your static profile, marks the plan provisional and
disables arbitrage. With `static_profile` empty there is nothing to fall back
to, and the site simply has no tariff until the feed returns. It is not required
— the add-on warns rather than refusing — but filling it in is what makes that
fallback work.
### Price arbitrage (`arbitrage_enabled`)
§10.1 priority 3: charge in the cheapest published hour, discharge in the
dearest, **and only when the spread covers what the round trip costs.** Off by
default — this is the first strategy in the add-on that commands the battery
*against* the meter on purpose, so it is armed deliberately or not at all.
| option | meaning |
|---|---|
| `arbitrage_enabled` | `false` by default. Arms the strategy |
| `cycle_cost_threshold_eur` | What one full round trip costs per kWh — pack wear plus conversion losses. A spread below this is not traded |
| `arbitrage_soc_floor` | The reserve arbitrage may not spend, % |
| `arbitrage_soc_ceiling` | The headroom it may not overfill, % |
**What it needs, and what the add-on refuses.** Arbitrage requires a price
source (`price_provider` or `static_profile`) and a P1 meter (`meter_source`).
Without a price source there is nothing to tell cheap from expensive; without
P1 the capacity-tariff peak guard never runs, and §9.1 puts peak shaving above
arbitrage — a charge that nothing measures is exactly what that guard exists to
prevent. Both combinations are **refused at startup** rather than left looking
armed.
**Peak shaving always wins.** The strategy cannot obtain a spread at all
without first answering whether the peak guard is holding, and even if it
could, the guard's floor is filed at a higher arbiter priority and clamps an
arbitrage charge. Discharging is never capped by it — only offtake is billed.
**It never exports into a negative price** (§17). A negative hour is the best
possible hour to *charge* in and that is still allowed; selling into one gives
energy away and pays for the privilege.
**At the shipped threshold a static day/night site does not trade.** The
realistic Belgian day/night spread is about 0.08 EUR/kWh and the default
threshold is 0.10, so the honest answer on such a site is "no opportunity", and
the status page says so with the numbers. Whether 0.10 is the right default is
a product decision, not a bug: lower it only if you know what a cycle costs
your pack.
**The prices are fetched in the background**, never on the control tick — a
supplier's TLS timeout on that path would be a supplier able to push the add-on
towards the firmware's 30 s failsafe. The plan refreshes every 30 minutes and
retires itself if the refresh stops answering, at which point arbitrage stops
with it.
### Site
| option | meaning |
+9
View File
@@ -36,6 +36,15 @@ P_SAFETY = 100 # stopped, inputs missing, shutting down, device limits
P_MAINTENANCE = 60 # maintenance owns the actuator outright (drain, hold)
P_PEAK = 50 # capacity-tariff peak shaving - costs real money
P_MAINT_SHAPE = 40 # maintenance shaping the loop (charge-only)
# OPT-04. §10.1 priority 3 - price arbitrage. ⚠️ ITS POSITION IS THE SPEC, NOT A
# preference: P_PEAK (50) is ABOVE it so clause 2 lets the capacity-tariff floor
# clamp an arbitrage charge, which is §9.1's "give peak shaving priority over
# arbitrage whenever the two conflict" expressed as arithmetic rather than as a
# rule somebody has to remember. P_MAINT_SHAPE (40) is above it so a speculative
# spread cannot override the pack's balancing cycle. P_LOOP (10) is below it
# because trading against the meter is the entire point - the grid-following
# loop would otherwise cancel every charge the moment it created import.
P_ARBITRAGE = 30 # price arbitrage - trades against the meter on purpose
P_LOOP = 10 # ordinary grid-following
SET = "set"
+429
View File
@@ -0,0 +1,429 @@
"""OPT-04 - §10.1 priority 3: charge cheap, discharge expensive, and only when
the round trip is worth more than it costs.
WHERE THIS LIVES, AND WHY IT IS NOT WHERE THE CARD SAYS. The functional notes
on the Trello card reference specs.md §2.2's `ems/optimizer/...` layout. That
module layout does not exist and is not being built; the product owner's
decision of 2026-08-30 is that everything ships in `goodwe_controller/app/`,
beside price.py, static_tariff.py, peakguard.py and arbiter.py. The BEHAVIOUR
§10.1 / §9.1 / §8.3 / §17 describes is unchanged and still authoritative.
WHAT THIS MODULE IS, AND - MORE IMPORTANTLY - WHAT IT IS NOT
============================================================
Four cards landed before this one and this one CONSUMES all four. Almost every
question an arbitrage strategy has to answer is already answered somewhere
else, by an object built for it, and the single biggest way to get this ticket
wrong is to answer any of them again here:
price.py (TARIFF-02) what a price IS, all-in, and what a product is.
`Uncertainty`, `WIDEN_FACTOR`, `required_spread`.
static_tariff.py (TARIFF-03) `TariffPlan.arbitrage()` - THE VERDICT. Whether
a trade is permitted at all, on which two hours,
against which widened threshold. Five refusal
reasons live there, not here.
price_forecast.py (FCST-03) provisional carrying, and the rule that no
irreversible action rests on a provisional
price (§8.3).
peakguard.py (SAFETY-07) the capacity-tariff floor, filed as a `lo`
LIMIT at P_PEAK on the arbiter.
So what is left for this file is genuinely small, and that is the point:
* WHICH HOUR IS IT NOW - is this instant the buy hour, the sell hour, or
neither.
* MAY THE BATTERY DO IT - state of charge, and §17's negative-price rule.
* HOW HARD - one number, filed as one arbiter claim.
Everything else is delegated, and delegated by CONSTRUCTION rather than by
convention: `TariffPlan.arbitrage()` will not hand over a spread at all unless
it is told whether the peak guard is holding, what a cycle costs, and what is
uncertain. There is no code path here that can reach a tradeable number without
having answered §10.1's priority ordering first.
AC3 - PEAK SHAVING WINS, AND IT WINS TWICE
==========================================
§9.1: "Give peak shaving priority over arbitrage whenever the two conflict."
That is honoured through the two mechanisms that already exist, and NEITHER of
them is re-implemented here:
1. THE INTERLOCK. `plan.arbitrage(peak_guard_active=...)` is keyword-only and
REQUIRED. When the guard is engaged the verdict is a refusal carrying the
§10.1 wording, so `decide()` below never sees a spread at all. It is not
possible to write a caller here that skips the question - the argument has
no default and the spread is unreachable without it.
2. THE ARBITER. Even if (1) were somehow bypassed, `claim()` files a SET at
P_ARBITRAGE (30) and peakguard files a LIMIT at P_PEAK (50). arbiter.py
clause 2 applies every limit whose priority is >= the winning set's, most
restrictive wins - so the peak floor CLAMPS an arbitrage charge whatever
this module thinks. 50 >= 30 is what makes that true, and it is why the
priority below sits under P_PEAK rather than beside it.
Two independent mechanisms for one rule is deliberate. The first is the one
that reports honestly; the second is the one that holds when the first is
wrong.
⚠️ AND THE GUARD MUST BE ABLE TO SEE. On `meter_source: off` there is no
telegram stream, so peakguard.decide() is never run and `peak_guard_active` is
not FALSE - it is UNKNOWN. Charging a battery at max_w creates offtake, offtake
is the billed quantity (§9.1), and doing it deliberately while nothing measures
the quarter is the exact shape of failure §9.1 exists to prevent. So the third
state is carried explicitly as `None` and it REFUSES. A failed read is not an
observation, and "the guard did not run" is not "the guard is clear".
§17's NEGATIVE-PRICE CASE, STATED ONCE
======================================
"Negative-price hours - arbitrage must not export." A negative price means the
market pays the site to consume. Discharging the battery into such an hour
gives away stored energy AND forgoes being paid to take more in; if the house
load is covered it becomes export, which at a negative price is paid-for
disposal. The buy side is untouched - a negative hour is the best possible hour
to charge in, and this file's guard is deliberately asymmetric for that reason.
⚠️ THE GUARD IS ON THE HOUR BEING SOLD INTO, NOT ON THE SPREAD. A series whose
prices are -0.30 and -0.05 has a 0.25 EUR/kWh spread that clears any shipped
threshold, and its DEAREST hour is still negative. A guard written as
"spread > 0" or "the prices are not all negative" passes that case and exports.
WHAT `cycle_cost_threshold_eur` IS, AND WHAT THIS FILE REFUSES TO DO WITH IT
===========================================================================
§10.1 gates arbitrage on "expected round-trip benefit exceeds
`cycle_cost_threshold_eur`". It ships at 0.10 EUR/kWh. On realistic Belgian
day/night numbers the static spread is about 0.08 EUR/kWh, and FCST-03's
widening factors (2.5 real-time, 1.5 no-weather) are honest policy defaults
with nothing to derive them from.
⚠️ SO A STATIC-TARIFF SITE DOES NOT TRADE AT SHIPPED DEFAULTS, AND THAT IS
REPORTED RATHER THAN TUNED AWAY. The threshold is a parameter here, the
behaviour is correct at any value of it, and no number in this file was chosen
to make a trade happen. Whether 0.10 is the right shipped default - and whether
2.5/1.5 are - is one product decision and it belongs to the owner, not to this
card. test_arbitrage.py section 7 reports what the shipped default does over a
replayed year rather than adjusting anything so that it looks better.
THE ACCOUNTING BASIS, AND THE TERM THAT IS DELIBERATELY ABSENT
=============================================================
The card's technical notes ask for `benefit = value_out(d) - cost_in(c)/η`, and
then say of the round-trip efficiency: "η is NOT a §14 key - it needs one, or
must be derived from measured charge/discharge energy in the state store. DO
NOT HARDCODE IT." Both halves are honoured by not having the term at all:
* THE SELL SIDE IS PRICED AS DISPLACED IMPORT, NOT AS EXPORT. The two hours
compared are both all-in CONSUMPTION prices, which is the basis the notes
require ("credit displacement at all-in import prices, not at export price,
or the number flatters itself"). `export_price_eur_kwh` is therefore not
read here at all - §9.3's near-zero Belgian injection price is precisely
why the economically sound trade on this market is charge-cheap-to-
displace-expensive-import, and §17's guard below is the same fact stated as
a rule.
* AND THERE IS NO η, BECAUSE THERE IS NOTHING HONEST TO PUT IN IT. No config
key exists, the state store carries no measured charge/discharge energy
pair to derive one from, and a hardcoded 0.85 is exactly what the notes
forbid - it would also DOUBLE-COUNT, because `cycle_cost_threshold_eur` is
already defined as what a round trip costs per kWh and conversion loss is
part of that cost. When a measured η exists it belongs beside the threshold
as a second term, and this file's arithmetic is one subtraction, in one
place, ready for it.
HYSTERESIS (§10.2)
==================
The switching decision is "which hour is it", and an hour is 60 minutes wide,
so the action cannot chatter within one. The other two terms are monotone
inside an hour in the only direction that matters: a charge raises SoC towards
the ceiling and a discharge lowers it towards the floor, so each can stop once
and cannot restart in the same hour. No timer is needed and none is invented.
"""
import logging
import math
from dataclasses import dataclass
from datetime import datetime
from .arbiter import P_ARBITRAGE, Claim
from .price import Absence, price_option
_LOG = logging.getLogger("goodwe.arbitrage")
SOURCE = "arbitrage"
# §10.2 reason codes. Written out because the action log is a user-facing
# artefact and a code assembled from an f-string cannot be grepped for.
CHARGE = "ARBITRAGE_CHARGE"
DISCHARGE = "ARBITRAGE_DISCHARGE"
# Why nothing was done. These are the reasons this MODULE owns; the five
# refusals that belong to the verdict itself are carried through verbatim from
# static_tariff.py rather than restated, so a refusal's wording has exactly one
# author.
IDLE_NO_PLAN = "no tariff plan has been resolved yet"
IDLE_NOT_COVERED = (
"NOT COVERED: meter_source is off, so the capacity-tariff peak guard never "
"runs and nothing measures the quarter this charge would land in. §9.1 puts "
"peak shaving above arbitrage, and an unmeasured peak cannot be shaved - "
"so the honest answer is that arbitrage is not covered on this install, not "
"that the guard is clear")
IDLE_NO_SOC = (
"state of charge is not being read, and a battery whose fill nobody knows "
"cannot be told to empty or to fill")
IDLE_OFF_PEAK_HOUR = "this hour is neither the buy hour nor the sell hour"
NEGATIVE_PRICE = (
"§17: the sell hour is priced BELOW ZERO, so discharging into it would give "
"energy away and pay for the privilege. Arbitrage must not export at a "
"negative price")
@dataclass(frozen=True)
class ArbitrageDecision:
"""What the strategy concluded this cycle, with what it concluded it from.
⚠️ NO FIELD HAS A DEFAULT, and that is TEST-14's rule rather than a style
choice: an outcome type's required field quietly gaining a default is a
mutation that survived review in three files in this repository. The
signature is scanned in test_price_forecast.py alongside `Gate`'s and
`Arbitrage`'s constructors, so a default added here later goes red without
anybody remembering to write a case for it.
`action` is None whenever nothing is to be done, and `power_w` is then None
too - not 0.0. Zero watts is a COMMAND (it parks the battery); "no claim"
is the absence of one, and the arbiter must be free to let the loop win.
"""
action: str | None # CHARGE | DISCHARGE | None
power_w: float | None # signed: negative charges, positive discharges
reason: str
spread_eur_kwh: float | None
required_eur_kwh: float | None
price_now_eur_kwh: float | None
@property
def trading(self) -> bool:
return self.action is not None
def explain(self) -> str:
if not self.trading:
return f"arbitrage idle - {self.reason}"
return self.reason
def _idle(reason, *, spread=None, required=None, price=None) -> ArbitrageDecision:
if not reason or not isinstance(reason, str):
raise ValueError(
"an idle arbitrage decision needs a reason a human can act on - "
"§10.2 requires every plan change to carry one, and 'arbitrage did "
"nothing' with no reason is indistinguishable from a bug.")
return ArbitrageDecision(action=None, power_w=None, reason=reason,
spread_eur_kwh=spread, required_eur_kwh=required,
price_now_eur_kwh=price)
def enabled(opts: dict) -> bool:
"""Is the strategy switched on at all? Off in the shipped configuration.
⚠️ OFF BY DEFAULT ON PURPOSE. Every other card in this repository ships its
machinery inert and lets an operator arm it; this one MOVES ENERGY against
the meter on a real battery, so it is the last one that should be an
exception to that.
"""
v = price_option(opts, "arbitrage_enabled")
return v is True or str(v).strip().lower() in ("true", "yes", "on", "1")
def threshold_eur_kwh(opts: dict) -> float:
"""§10.1's `cycle_cost_threshold_eur`, from the one place option defaults live."""
v = float(price_option(opts, "cycle_cost_threshold_eur"))
if not math.isfinite(v) or v < 0:
raise ValueError(
f"cycle_cost_threshold_eur is {v!r}. It is what a round trip COSTS, "
f"in EUR/kWh, so it is non-negative and finite - a negative "
f"threshold is one every spread clears, which is arbitrage with the "
f"brake removed.")
return v
def soc_bounds(opts: dict) -> tuple:
"""(floor, ceiling) in percent. The reserve arbitrage may not spend."""
return (float(price_option(opts, "arbitrage_soc_floor")),
float(price_option(opts, "arbitrage_soc_ceiling")))
def guard_state(peak_decision):
"""The peak guard's answer as the three-state `decide()` takes: T / F / None.
THE ONE PLACE THIS CONJUNCTION IS WRITTEN, and it is public for the reason
`peakguard.shaping_suspended()`, `watchdog.loop_is_following()` and
`freeze.looks_frozen()` are: the check that grades it has to assert on THE
SHIPPED EXPRESSION. A predicate transcribed into its own test is one copy
plus a future disagreement about what the check means, and the copy that
drifts is always the grading one - this repository has three separate
records of exactly that, one of which left every suite green while a
default install parked at 0 W in the quarter that sets the month's bill.
⚠️ `None` IS NOT `False` AND COLLAPSING THE TWO IS THE WHOLE HAZARD. On
`meter_source: off` main.py never runs peakguard.decide() - there is no
telegram stream, so there is no open quarter to project - and the decision
it hands here is None. That is "the guard never ran", not "the guard is
clear", and answering False would arm a strategy whose charging creates the
billed quantity (§9.1) while nothing measures the quarter it lands in.
"""
if peak_decision is None:
return None
return bool(peak_decision.engaged)
def _point_at(plan, when: datetime):
"""The price point covering `when`, or None when the series does not cover it.
⚠️ None IS NOT A PRICE AND IS NOT ZERO. `TariffPlan` raises `Absence` here
rather than answering 0.0, for price.py's reason - "no data" becoming "free"
is the defect the whole outcome-type family exists to remove - and this
turns that into "do nothing", never into "it is cheap".
"""
try:
return plan.read.match(series=lambda r: r.at(when),
no_data=lambda _r: None)
except Absence:
return None
def decide(plan, now: datetime, *, peak_guard_active, soc,
min_spread_eur_kwh, uncertainty, charge_w, discharge_w,
soc_floor_pct, soc_ceiling_pct) -> ArbitrageDecision:
"""Should the battery trade this cycle, and how hard.
⚠️ `peak_guard_active` IS A THREE-STATE ARGUMENT AND ALL THREE ARE REAL.
True (the guard is holding), False (it ran and is clear), and None (it never
ran, because nothing measures this install's quarter). See the module note:
None refuses, because the alternative is a strategy that creates the billed
quantity while the thing that would have limited it is switched off.
⚠️ EVERY OTHER ARGUMENT IS KEYWORD-ONLY AND HAS NO DEFAULT, for
`TariffPlan.arbitrage()`'s reason. A defaulted `soc_floor_pct` is a reserve
floor this module asserted rather than one the operator configured, and the
first time it is wrong the battery is at 0 %.
"""
if peak_guard_active is None:
return _idle(IDLE_NOT_COVERED)
if plan is None:
return _idle(IDLE_NO_PLAN)
verdict = plan.arbitrage(peak_guard_active=peak_guard_active,
min_spread_eur_kwh=min_spread_eur_kwh,
uncertainty=uncertainty)
return verdict.match(
refused=lambda v: _idle(v.reason),
allowed=lambda v: _act(plan, now, v, soc, charge_w, discharge_w,
soc_floor_pct, soc_ceiling_pct))
def _act(plan, now, v, soc, charge_w, discharge_w, floor_pct, ceiling_pct):
"""The permitted case: which hour is this, and may the pack do it."""
here = _point_at(plan, now)
if here is None:
return _idle(
f"the series carries a tradeable spread but no price for this "
f"instant ({now.isoformat()}), which is §11.4 for this hour rather "
f"than a reason to reach for a neighbouring one",
spread=v.spread_eur_kwh, required=v.required_spread_eur_kwh)
price = here.eur_per_kwh
buying = here.start == v.cheapest.start
selling = here.start == v.dearest.start
if not (buying or selling):
return _idle(IDLE_OFF_PEAK_HOUR, spread=v.spread_eur_kwh,
required=v.required_spread_eur_kwh, price=price)
# ⚠️ BELT AND BRACES ON §8.3, AND IT IS NOT DEAD CODE. `_firm_or_die` makes
# the two points on the verdict firm at construction, but the point matched
# HERE is whatever `at()` returned for this instant, and the identity of the
# two rests on the series being the one the verdict was built from. An hour
# that is provisional is one nobody published a price for, and no
# irreversible action may rest on it.
if here.provisional:
return _idle(
f"§8.3: the price for this hour ({here.start.isoformat()}) is "
f"CARRIED FORWARD, not published, and no irreversible action may be "
f"taken on a provisional price",
spread=v.spread_eur_kwh, required=v.required_spread_eur_kwh,
price=price)
if soc is None:
return _idle(IDLE_NO_SOC, spread=v.spread_eur_kwh,
required=v.required_spread_eur_kwh, price=price)
common = dict(spread=v.spread_eur_kwh, required=v.required_spread_eur_kwh,
price=price)
if selling:
# §17. THE NAMED REGRESSION CASE. Checked BEFORE the SoC question, so a
# negative price reports the reason that actually forbids the trade
# rather than one that happens to fire first on some other site.
if price < 0:
return _idle(f"{NEGATIVE_PRICE} - this hour is "
f"{price:+.4f} EUR/kWh", **common)
if soc <= floor_pct:
return _idle(
f"the sell hour is here at {price:.4f} EUR/kWh but the pack is "
f"at {soc:g} %, at or under the {floor_pct:g} % reserve floor "
f"arbitrage may not spend (§10.1 priority 4)", **common)
return ArbitrageDecision(
action=DISCHARGE, power_w=abs(float(discharge_w)),
reason=(f"{DISCHARGE}: this is the dearest firm hour at "
f"{price:.4f} EUR/kWh, {v.spread_eur_kwh:.4f} above the "
f"cheapest and clear of the {v.required_spread_eur_kwh:.4f} "
f"this cycle had to cover; discharging "
f"{abs(float(discharge_w)):.0f} W from {soc:g} %"),
spread_eur_kwh=v.spread_eur_kwh,
required_eur_kwh=v.required_spread_eur_kwh,
price_now_eur_kwh=price)
if soc >= ceiling_pct:
return _idle(
f"the buy hour is here at {price:.4f} EUR/kWh but the pack is at "
f"{soc:g} %, at or over the {ceiling_pct:g} % ceiling - there is "
f"nowhere to put the cheap energy", **common)
return ArbitrageDecision(
action=CHARGE, power_w=-abs(float(charge_w)),
reason=(f"{CHARGE}: this is the cheapest firm hour at "
f"{price:.4f} EUR/kWh, {v.spread_eur_kwh:.4f} below the dearest "
f"and clear of the {v.required_spread_eur_kwh:.4f} this cycle "
f"had to cover; charging {abs(float(charge_w)):.0f} W from "
f"{soc:g} %"),
spread_eur_kwh=v.spread_eur_kwh,
required_eur_kwh=v.required_spread_eur_kwh,
price_now_eur_kwh=price)
def claim(decision: ArbitrageDecision) -> Claim | None:
"""The decision as one arbiter claim, or None when nothing is to be traded.
A SET at P_ARBITRAGE, and the priority is the whole §10.1 ordering:
P_PEAK 50 > this, so peakguard's `lo` floor CLAMPS an arbitrage
charge (arbiter clause 2 applies every limit at or
above the winner's priority). AC3, mechanically.
P_MAINT_SHAPE 40 > this, so a maintenance charge window's `hi` shaping is
not overridden by a price opportunity. §12.4's "money
outranks the schedule" is about the PEAK guard, which
is measured and billed; a speculative spread is not
that, and letting it out-rank the pack's balancing
cycle would be this card quietly rewriting §12.
P_LOOP 10 < this, because trading against the meter is the point:
the grid-following loop would otherwise cancel every
charge the moment it created import.
⚠️ NONE MEANS NO CLAIM, NOT A 0 W CLAIM. Filing `Claim.set(..., 0.0)` here
would out-rank the loop with a park order every cycle the strategy had
nothing to say, i.e. it would silently disable self-consumption on any site
that armed arbitrage.
"""
if not decision.trading or decision.power_w is None:
return None
return Claim.set(SOURCE, P_ARBITRAGE, float(decision.power_w),
decision.action)
+284
View File
@@ -0,0 +1,284 @@
"""SAFETY-03 - the output clamp, and the alarm that makes it audible.
WHAT IT CATCHES, in one sentence: a control law that computes a setpoint the
actuator cannot deliver, which every layer below it will happily clip to
something legal-looking while the thing that produced it goes on winding.
THE CLAMP IS NOT THE DELIVERABLE. specs.md 10.3 is explicit that this project
already clipped, and that the clipping is what hid the defect:
with the inverter switched off entirely, an unclamped integrator climbed
~130 W per 4 s to over 10 kW and, pinned at the 5 kW rail, reported
14 768 W
A clamp alone turns that into a silent, perfectly flat 5000 W hold. Every
dashboard reads "commanded 5000 W", which is a legal number, and nothing
anywhere says the loop asked for three times it. 11.2's watchdog row says
"Clamp AND ALARM - the loop is winding"; the second verb is this module.
------------------------------------------------------------------------------
WHAT "THE RATING" IS HERE, AND WHY IT IS ONE NUMBER AND NOT A MIN()
The card names two bounds and warns against choosing between them:
- `DriverCapabilities.setpoint_limit_w` (3.2), the inverter's own magnitude
rating. In this add-on that arrives as the setpoint entity's `min`/`max`
from `hass.limits()` - `Controller.dev_min` / `dev_max`.
- `optimizer.loop.clamp_w` (14), the configured loop clamp. In this add-on
that is the `max_w` option, which `Tuning.max_w` and the arbiter's
`supervised` limit both already use.
14 says these two must AGREE, validated at startup with a refusal on
mismatch - and that gate already ships: `validate_startup_config()` refuses
`max_w > dev_max` and `-max_w < dev_min` (validate.py, problem 1), and
run_control() re-runs it once ESPHome is up and the real rating is known.
So this guard clamps to `max_w`, THE VALIDATED VALUE, and does not `min()` it
with the device rating at runtime. That is the card's instruction and it is also
the only reading that keeps `max_w` meaningful: a silent runtime `min()` is
precisely the "helper bounded to +-500 W under a loop clamped to 1500 W" failure
the 14 warning box is about.
WARNING - THE DEVICE BOUND IS STILL APPLIED, SECOND, AND IT IS NOT REDUNDANT.
After a successful validation `+-max_w` lies inside `[dev_min, dev_max]`, so the
second stage is arithmetically a no-op and can never engage. If it DOES engage,
the validation did not run or no longer holds - `hass.limits()` returns
(None, None) whenever ESPHome is not up yet, and the range checks are skipped
rather than guessed at - and that is a louder fault than the first stage, not a
quieter one. It therefore alarms too, through the same channel, with the bound
named. This is NOT SAFETY-20; nothing here changes what `freeze.py` measures its
rail against.
------------------------------------------------------------------------------
WHERE THE WIND ACTUALLY SHOWS UP - READ THIS BEFORE "SIMPLIFYING" THE CALLER
`control.compute()` bounds the integrator to `integrator_max_w or max_w` and
THEN derives the command from it, so on a default install (`integrator_max_w`
unset, i.e. following `max_w`) the command can never exceed `max_w` and the
actuator-side clamp below IS UNREACHABLE. control.py says so in as many words:
SAFETY-03 ... must watch for THIS, not for "clamped" below.
An alarm wired only to this module's `ClampResult.engaged` would therefore be
silent on the exact configuration the whole ticket is about. The winding is
visible one layer up, at the integrator bound, and that is why `Decision` now
carries `i_clamped` / `clamped` BOOLEANS and the caller ORs them in here.
AND WHY BOOLEANS RATHER THAN `Decision.reason`. `reason` is a single string that
later stages overwrite: a cycle that clamps the integrator and is then slew
limited reports "slew-limited", and one that clamps while saturated reports
"saturated-freeze". Both are exactly the winding-into-a-rail case, and on the
string the clamp signal is destroyed before anything can alarm on it. The two
flags are set where the clamp happens and are never overwritten. A check in
test_clamp.py drives that precise combination.
------------------------------------------------------------------------------
DISTINCT FROM SATURATION - THIS IS ACCEPTANCE CRITERION 2, NOT A NICETY
clamp the COMPUTED value exceeded the rating. A software bug signal:
something upstream is winding. Alarm, every engagement.
saturation |command - inverter AC power| > `saturation_w` for
`saturation_cycles`. The HARDWARE cannot follow a perfectly legal
command. Action is "freeze the integrator" (11.2), not an alarm.
A loop can be legally saturated all day - a cold pack, a curtailed inverter, an
export limit - so a saturation-shaped alarm is one an operator learns to ignore.
Routing the clamp through it puts the bug signal inside the noise, which is the
same silence 10.3 is complaining about. They share no reason code, no
notification id, no status row and no counter.
------------------------------------------------------------------------------
ANTI-FLOOD, WITHOUT GOING SILENT
A genuinely winding loop clamps on EVERY cycle for as long as it winds. One HA
notification per cycle is how an operator learns to dismiss notifications
unread, and this add-on already made that argument once (see the fixed
`notification_id` note in main.check_meter_frozen).
So the two surfaces are split, deliberately:
- EVERY engagement is recorded, never deduplicated, and that record is what
says how hard the loop is pushing. 15's action log is this add-on's
`Controller.events`, and it gets a row from `write_setpoint()` on every
engaged WRITE; `engagements` here counts every engaged CYCLE. The two layers
are counted separately on purpose - the heartbeat re-writes an
already-clamped value between cycles, so a per-write count would inflate and
a per-cycle log would miss an out-of-cycle write.
- THE NOTIFICATION fires on the rising edge and then RE-ASSERTS every
`REASSERT_S`, so a loop pinned on the rail for an hour does not go quiet.
- It DE-ASSERTS only after `CLEAR_CYCLES` consecutive clean cycles, which is
the 10.2 hysteresis: a command sitting exactly on the rail dithers in and
out of engagement by one quantisation step, and without this the alarm
chatters.
`REASSERT_S` and `CLEAR_CYCLES` are module constants, not options, for the same
reason `freeze.RAIL_FRAC` is: they describe the event bus, not the plant, and
every knob 10.3 asked to keep exposed is a LOOP TUNING knob and is untouched by
this module. No new config key is added anywhere - CFG-03 already has three
places a default has to be kept in step and this ticket does not earn a fourth.
------------------------------------------------------------------------------
NOT A TRIP
Clamping does not stop the loop and does not command 0 W. The clamped value is
written and control continues, which is what 11.2 asks for ("Clamp and alarm")
and what distinguishes this from SAFETY-01/02, whose action is 0 W. It follows
that this must NOT drive the failsafe status - `publish()` keeps reporting
"running", because the controller is.
"""
import math
from dataclasses import dataclass
from datetime import datetime
# The reason code, distinct from any saturation-freeze code by construction:
# nothing else in this add-on uses it. See "DISTINCT FROM SATURATION".
REASON_CODE = "OUTPUT_CLAMP"
# Re-assert the notification this often while the clamp stays engaged. 30 s is
# the planning cadence (2.3), i.e. slow enough not to flood and fast enough
# that a pinned loop is never quiet for a whole billing quarter.
REASSERT_S = 30.0
# Consecutive clean cycles before the alarm de-asserts (10.2 hysteresis).
CLEAR_CYCLES = 3
@dataclass(frozen=True)
class ClampResult:
"""What the guard did, with the RAW value preserved.
AC 2 turns on "the raw computed value differed from the clamped one", so the
raw value has to survive the call - a guard that returns only the applied
number cannot be asked the question the acceptance criterion asks.
"""
requested_w: float
applied_w: float
limit_w: float
engaged: bool
bound: str = "" # "" | "max_w" | "device" - which stage bit
def describe(self) -> str:
return (f"{REASON_CODE}: the computed setpoint {self.requested_w:+.0f} W "
f"was clamped to {self.applied_w:+.0f} W by the {self.bound} "
f"bound (rating {self.limit_w:.0f} W) - something upstream is "
f"winding")
def is_commandable(value) -> bool:
"""False for None, NaN and the infinities.
A non-finite setpoint is LOSS OF INPUT, not an out-of-range value, and
11.1 rule 1 says loss of input means command zero and keep commanding. It
must not reach the clamp arithmetic: `min(limit, nan)` returns nan on
CPython, so a nan would sail straight through a naive clamp and then report
`engaged` True on the `applied != requested` test - because nan != nan - i.e.
a transport bug filed in the winding-loop counter, with a nan on the wire.
"""
if value is None:
return False
try:
return math.isfinite(float(value))
except (TypeError, ValueError):
return False
def clamp_to_rating(requested_w, limit_w, dev_min=None, dev_max=None) -> ClampResult:
"""Bound a computed setpoint to the validated rating. The ONLY clamp.
`limit_w` the validated magnitude bound - `max_w`. Symmetric, because
3.2 calls `setpoint_limit_w` a MAGNITUDE clamp: it binds import
(+) and export (-) identically.
`dev_min` / `dev_max` the device's own asymmetric range, applied second and
expected never to bite. See the module note.
Public, and the predicate lives here rather than in the test, for the reason
`freeze.looks_frozen()` gives: a safety expression transcribed into its own
check is one copy plus a future disagreement about what the check means.
EXACTLY +-`limit_w` IS NOT AN ENGAGEMENT. The bound is inclusive; a loop
resting on the rail is doing what it was told, and alarming there would fire
on every healthy full-power hour.
"""
requested = float(requested_w)
limit = abs(float(limit_w))
# ⚠️ BOTH STAGES ALWAYS RUN. THIS USED TO `return` FROM THE FIRST ONE, AND
# THAT WAS A REGRESSION AGAINST THE CODE IT REPLACED. `write_setpoint()`
# previously clamped to the device range unconditionally, under the comment
# "never send a value the device will reject outright"; short-circuiting here
# meant the device bound was applied only when the rating bound had NOT
# fired, i.e. THE BIGGER THE OVERSHOOT THE LESS THE GUARD DID. Measured on
# the shipped function at max_w 5000 against a device of +-4000:
#
# +9999 -> +5000 bound='max_w' <- outside the device range
# +4500 -> +4000 bound='device'
#
# It also undercut this module's own argument for clamping to `max_w` rather
# than a runtime `min()`, which rests entirely on this stage catching the
# case where the startup validation did not run.
applied = max(-limit, min(limit, requested))
bound = "max_w" if applied != requested else ""
# Second stage. Arithmetically dead after a successful startup validation -
# and therefore worth alarming on precisely BECAUSE it fired.
if dev_min is not None and dev_max is not None:
bounded = max(float(dev_min), min(float(dev_max), applied))
if bounded != applied:
bound = "device" if not bound else "max_w+device"
applied = bounded
return ClampResult(requested, applied, limit, applied != requested, bound)
class ClampWatchdog:
"""One instance per controller. Counts every engagement, alarms sparsely.
`note()` is called once per write with whether the clamp bit; the caller ORs
in the loop's own `i_clamped` / `clamped` flags, which is where the winding
is actually visible on a default install (see the module note).
"""
def __init__(self):
self.active = False
self.engagements = 0 # every engagement, never deduplicated
self.episodes = 0 # rising edges
self.since: datetime | None = None
self.last_alarm: datetime | None = None
self.reason = ""
self.clean_cycles = 0
def note(self, engaged: bool, reason: str, now: datetime) -> bool:
"""Advance one cycle. True when the caller should raise the alarm.
`now` is the caller's clock, the same discipline as
`SignWatchdog.evaluate()` and `FreezeWatchdog.evaluate()`.
"""
if engaged:
self.engagements += 1
self.clean_cycles = 0
self.reason = reason
if not self.active:
self.active = True
self.since = now
self.episodes += 1
self.last_alarm = now
return True
# Still engaged: re-assert on the slow cadence, not every cycle.
if (self.last_alarm is None
or (now - self.last_alarm).total_seconds() >= REASSERT_S):
self.last_alarm = now
return True
return False
if not self.active:
return False
self.clean_cycles += 1
if self.clean_cycles >= CLEAR_CYCLES:
self.active = False
self.since = None
self.last_alarm = None
self.reason = ""
self.clean_cycles = 0
return False
+23 -3
View File
@@ -64,6 +64,23 @@ class Decision:
# and quantisation. Carry it back in as `i_w` next cycle; that is what keeps
# it a separate quantity from the command.
i_w: float = 0.0
# SAFETY-03. Did a clamp actually engage this cycle - i.e. did the RAW
# computed value differ from the bounded one?
#
# ⚠️ THESE EXIST BECAUSE `reason` CANNOT ANSWER THAT QUESTION. `reason` is
# one string and every later stage overwrites it: a cycle that clamps the
# integrator and is then slew limited reports "slew-limited", and one that
# clamps while saturated reports "saturated-freeze". Both ARE the winding
# case, and on the string the clamp signal is gone before anything can alarm
# on it. These two are written where the clamp happens and are never
# overwritten, so they mean exactly what they say and nothing else.
#
# `i_clamped` is the one that fires on a default install: the integrator
# bound is reached first, so `clamped` is unreachable unless
# `integrator_max_w` is deliberately set above `max_w`. Alarm on BOTH - see
# the comment at the bound below, and clamp.py.
i_clamped: bool = False
clamped: bool = False
def compute(
@@ -188,7 +205,8 @@ def compute(
# bound is not. It is what makes the worst-case unwind time finite and
# knowable instead of a function of how long the error happened to stand.
bounded = max(-limit, min(limit, i_w))
if bounded != i_w:
i_clamped = bounded != i_w
if i_clamped:
# ⚠️ SAFETY-03 (alarm whenever the loop winds into a rail) must watch
# for THIS, not for "clamped" below. At the default limit == max_w the
# integrator bound is reached first and the command derived from it can
@@ -209,7 +227,8 @@ def compute(
# --- ORDER MATTERS: clamp -> slew -> freeze ----------------------------
# An early draft applied a floor after the clamp and let demand escape it.
target = max(-tuning.max_w, min(tuning.max_w, want))
if target != want:
clamped = target != want
if clamped:
reason = "clamped"
slewed = max(prev_w - tuning.slew_w, min(prev_w + tuning.slew_w, target))
@@ -235,7 +254,8 @@ def compute(
step = max(1, int(tuning.step_w))
target = round(target / step) * step
return Decision(float(target), sat_count, frozen, reason, float(i_w))
return Decision(float(target), sat_count, frozen, reason, float(i_w),
i_clamped, clamped)
def maintenance_charge_floor(
+774 -19
View File
@@ -28,19 +28,30 @@ import contextlib
import json
import logging
import signal
from datetime import datetime, timezone
from datetime import datetime, timedelta, timezone
import aiohttp
from . import arbitrage
from .arbiter import (
P_LOOP, P_MAINT_SHAPE, P_MAINTENANCE, P_SAFETY, Claim, resolve,
P_LOOP, P_MAINT_SHAPE, P_MAINTENANCE, P_PEAK, P_SAFETY, Claim, resolve,
)
from .control import Tuning, compute, maintenance_charge_floor, peak_at_risk
from .freeze import FreezeWatchdog, FreezeWatchdogConfig, rail_w
from .clamp import (
REASON_CODE as CLAMP_REASON, ClampResult, ClampWatchdog, clamp_to_rating,
is_commandable,
)
from .freeze import (
FreezeWatchdog, FreezeWatchdogConfig, looks_frozen, rail_w,
)
from .hass import HomeAssistant
from .maintenance import IDLE, MaintConfig, Maintenance
from .mqtt import MqttPublisher
from . import peakguard
from .p1 import P1Ingest, build_source, is_enabled
from .price import Uncertainty, price_option
from .static_tariff import resolve_tariff, static_status
from .tariff import PeakStore, PeakTracker
from .validate import ConfigRefused, refuse, validate_startup_config
from .watchdog import SignWatchdog, SignWatchdogConfig, loop_is_following
from . import web
@@ -48,6 +59,34 @@ from . import web
OPTIONS_PATH = "/data/options.json"
_LOG = logging.getLogger("goodwe")
# --------------------------------------------------------------------------- #
# OPT-04: THE BACKGROUND TARIFF REFRESH, AND WHY IT IS NOT OPTIONAL
#
# `static_tariff.resolve_tariff()` may perform an HTTPS fetch. The control tick
# is what feeds the ESP32 heartbeat, and layer 1 commands 0 W if that heartbeat
# stops for ~30 s - so a supplier's TLS timeout on the tick is a SUPPLIER able
# to push this add-on into the failsafe. Every module in the price stack says
# so in its own words (static_tariff.static_plan, resolve_tariff, main's
# read_inputs) and this is the task those notes were waiting for.
#
# The tick therefore never awaits a fetch. It reads `self.tariff_plan`, an
# attribute this task assigns, and nothing else.
#
# ⚠️ THE CADENCE IS NOT A GUESS AT HOW OFTEN PRICES CHANGE. A day-ahead feed
# publishes ONCE, at 14-16 h CET, and §8.3's carried hours become firm at that
# moment; 30 minutes is short enough that the switch is picked up inside the
# hour it happens and long enough to be a polite 48 fetches a day. Nothing in
# the control law depends on it.
TARIFF_REFRESH_S = 1800.0
# ⚠️ AND THE AGE BOUND IS NOT THE SAME QUESTION. A refresh that FAILS still
# stores a plan - resolve_tariff() degrades to the §11.4 static fallback, which
# is provisional and refuses arbitrage on its own. This bound covers the other
# case: the refresh task itself dying, leaving a plan that was true hours ago
# standing as if it were current. Three missed refreshes is the threshold, for
# `p1.max_age_s`'s reason - one missed poll is not a fault, a run of them is.
# A failed read is not an observation, and neither is a stale one.
TARIFF_MAX_AGE_S = 3 * TARIFF_REFRESH_S
def load_options() -> dict:
try:
@@ -59,7 +98,8 @@ def load_options() -> dict:
class Controller:
def __init__(self, opts: dict, hass: HomeAssistant, store, mqtt_pub, *, stop):
def __init__(self, opts: dict, hass: HomeAssistant, store, mqtt_pub, *, stop,
peak_db_path: str = "/data/tariff.db"):
self.o = opts
self.hass = hass
self.store = store
@@ -140,12 +180,75 @@ class Controller:
self.freeze_wd = FreezeWatchdog(
FreezeWatchdogConfig(hold_s=float(opts.get("meter_freeze_s", 300))))
# SAFETY-03. The output clamp's alarm - see clamp.py for why the clamp
# itself is the cheap half.
#
# ⚠️ NO CONFIG OF ITS OWN, on purpose. The bound it clamps to is `max_w`,
# which is already an option, already validated against the device's
# rating at startup, and already used by Tuning and the arbiter. Giving
# this a second key would be a fourth place a limit has to be kept in
# step (CFG-03 already tracks three) and, worse, would recreate exactly
# the two-numbers-that-disagree failure the §14 warning box describes.
# The anti-flood cadence is a module constant for the same reason
# freeze.RAIL_FRAC is: it describes the event bus, not the plant.
self.clamp_wd = ClampWatchdog()
# P1 ingestion (TEL-01). `meter_source: off` keeps the original
# single-entity meter_entity path, so an existing install is unchanged
# until it opts in.
self.p1 = P1Ingest(phases=int(opts.get("meter_phases", 1)),
max_age_s=float(opts.get("meter_max_age_s", 30)))
self.p1_enabled = is_enabled(opts)
# TARIFF-01. The capacity-tariff peak: measured from the meter, banked
# in SQLite, restored across a restart. It is an INSTRUMENT - it files no
# claim and clamps nothing. SAFETY-07 is what will read it.
#
# ⚠️ NO NEW OPTION. The soft limit is `peak_cap_w`, already shipped,
# already validated, already read by control.peak_at_risk(). CFG-03
# tracks three places option defaults live; a fourth, for a limit that
# already exists, is the mistake the clamp watchdog declined to make ~30
# lines up. config.yaml and validate.py are untouched by this ticket.
#
# ⚠️ AND NO SECOND PEAK ENTITY. `peak_forecast_entity` is the existing
# HA-side delegation and it stays the input to maintenance_charge_floor():
# repointing that consumer at this measurement CHANGES WHAT MOVES ENERGY
# during a maintenance charge, which belongs to SAFETY-07 with its own
# acceptance, not to a measurement ticket. What this adds is the MEASURED
# MONTH PEAK - a different quantity from a forecast entity, published
# nowhere as a duplicate of it.
self.peak_store = PeakStore(peak_db_path)
self.peak = PeakTracker(self.peak_store,
cap_w=float(opts.get("peak_cap_w", 3500)))
self.peak.restore(self.p1.averager)
_LOG.info("capacity-tariff peak store: %s", self.peak.restore_state)
# SAFETY-07. The guard TARIFF-01 was built for. It is stateless by
# design (peakguard.py holds no history between calls), so the only
# state here is what an EDGE needs: what the last cycle concluded, so
# the action log and the notification fire on transitions rather than
# once per cycle for a quarter of an hour.
#
# ⚠️ THE CEILING HAS NO CONFIG OF ITS OWN, for the reason the clamp
# watchdog gives ~90 lines up and TARIFF-01 repeats: it is `peak_cap_w`,
# already shipped and already validated, and every other term of the
# projection is read off something the arbiter or the averager already
# owns.
#
# ⚠️ SAFETY-24'S FLOOR DOES HAVE ONE, AND IT COULD NOT NOT HAVE. How far
# down the guard may spend the pack for money is a number only the site
# knows, and refusing one set below the pack's own cut-out means knowing
# that cut-out too. So this card adds exactly two keys and reads their
# defaults from `peakguard.PEAK_OPTION_DEFAULTS` - TARIFF-02's mirror
# pattern - rather than typing literals here. THAT is what keeps CFG-03
# at three places: the divergence validate.py's rule 6 documents
# (`maintenance_charge_w` 2000 there, 2500 here) is exactly what a
# literal in this file would repeat.
self.peak_guard = None
self._peak_engaged = False
self._peak_saturated = False
self._peak_floor_limited = False
self._maint_shape_suspended = False
# Set by amain() once the transport is built, so the status page can show
# what only the transport knows (homewizard_local's unchanged_s). Stays
# None when P1 is off, or under a transport that has no such counter.
@@ -166,11 +269,30 @@ class Controller:
self.grid = self.soc = self.batt = None
self.peak_fc = None
self.cheap = False
# TARIFF-03. One line of prose for the status page saying WHICH tariff
# window we are in and why it is or is not cheap - so "Cheap window:
# no" is an evidenced answer rather than a bare no.
self.tariff_words = "no static profile"
# OPT-04. The tariff plan the BACKGROUND task last resolved, and when.
# cycle() reads these two attributes and never awaits a fetch - see
# TARIFF_REFRESH_S. `None` means no refresh has completed yet, which is
# not "no prices" and certainly not "free": arbitrage.decide() reports
# it as its own reason and does nothing.
self.tariff_plan = None
self.tariff_at = None
self.arb = None
self._arb_action = None
self.last_write = None
self.last_write_at = None
self.last_write_ok = None
self.inputs_bad_since = None
self.dev_min = self.dev_max = None
# SAFETY-03. What the last write's clamp did, and what the loop's own
# bounds did last cycle. `loop_clamped` is a (engaged, note) pair so the
# alarm can say WHICH bound bit without re-deriving it from a string.
self.clamp_last: ClampResult | None = None
self.loop_clamped: tuple[bool, str] = (False, "")
self.events: list[str] = []
self.stopping = False
@@ -214,9 +336,31 @@ class Controller:
now_p = await self.hass.number(now_e)
avg_p = await self.hass.number(avg_e)
self.cheap = (now_p is not None and avg_p is not None and now_p <= avg_p)
self.tariff_words = "Home Assistant price entities"
else:
# Fixed-tariff site: never force a paid grid top-up, wait for sun.
self.cheap = False
# TARIFF-03. §9.2 makes a static day/night tariff a first-class
# product, and its required EMS handling is TIME-WINDOW SHIFTING.
# The add-on has exactly one shiftable load in 0.5.x - the
# maintenance cycle's cheap-window grid top-up, ~15 lines below in
# cycle() - so this is where the window reaches something that moves
# energy. With `static_profile` empty this returns False and the
# behaviour is bit-for-bit what it has always been: "fixed-tariff
# site, never force a paid grid top-up, wait for sun".
#
# ⚠️ SYNCHRONOUS AND I/O-FREE, ON PURPOSE. The provider-backed
# resolver (static_tariff.resolve_tariff) can perform an HTTP fetch,
# and this coroutine is on the path that feeds the ESP32 heartbeat -
# a supplier's TLS timeout here would be a supplier able to push the
# add-on towards layer 1's 30 s veto. Generation from a static
# profile is pure arithmetic, so §2.3's "regenerate on the tick"
# applies and there is nothing to block on. Putting a fetched series
# on this path needs a BACKGROUND refresh and belongs with OPT-04.
#
# ⚠️ AND IT CANNOT BE BOTH. validate.py refuses static_profile
# alongside the two Home Assistant entities, because this `if` picks
# one and the loser is silent.
self.cheap, self.tariff_words = static_status(
o, datetime.now(timezone.utc))
if self.inputs_ok:
if self.inputs_bad_since is not None:
@@ -229,11 +373,37 @@ class Controller:
entity = self.o.get("setpoint_entity", "")
if not entity:
return
# Never send a value the device will reject outright. A rejected write is
# silent from the controller's point of view, and the inverter then keeps
# doing whatever it was already doing.
if self.dev_min is not None and self.dev_max is not None:
value = max(self.dev_min, min(self.dev_max, value))
# --- SAFETY-03: THE CLAMP, AND IT IS ON THE ONLY PATH TO THE WIRE -----
# Every write in this add-on comes through here - the cycle, the
# heartbeat, the operator's stop button and shutdown() - so no optimizer,
# maintenance or manual path can route around it. That placement is the
# first acceptance criterion, and it is why the guard is not in
# control.py: a guard the control law owns is a guard every other claimer
# bypasses.
#
# ⚠️ A NON-FINITE VALUE IS LOSS OF INPUT, NOT AN OUT-OF-RANGE ONE. §11.1
# rule 1: command zero and keep commanding. It must not reach the clamp
# arithmetic - see is_commandable() for what a nan does to it - and it
# must not be counted as a clamp engagement, or a transport bug lands in
# the winding-loop counter.
if not is_commandable(value):
self.log_event(f"non-finite setpoint {value!r} - commanding 0 W")
_LOG.error("refusing to write a non-finite setpoint %r - commanding "
"0 W and continuing to command. This is a bug upstream, "
"not a limit.", value)
value = 0.0
result = clamp_to_rating(value, float(self.o.get("max_w", 2000)),
self.dev_min, self.dev_max)
self.clamp_last = result
if result.engaged:
# ⚠️ EVERY engagement, never deduplicated: this is the action log
# (§15), and it is a different surface from the notification, which
# IS deduplicated. The count is what says how hard the loop pushed;
# the notification only says that it did.
self.log_event(result.describe())
value = result.applied_w
changed = (self.last_write is None or value != self.last_write)
if not (changed or force):
return
@@ -324,6 +494,146 @@ class Controller:
self.log_event("inputs missing - commanding 0 W")
claims.append(Claim.set("safety", P_SAFETY, 0.0, "inputs missing"))
# --- SAFETY-07: the quarter-peak projection guard --------------------
# ONE `lo` limit at P_PEAK, and peakguard.py carries the whole argument.
# What is decided HERE and nowhere else is what the guard is allowed to
# see and what it is allowed to ask for:
#
# ⚠️ `frozen` IS EVALUATED ON THIS CYCLE'S STANDING COMMAND, NOT ON
# `self.freeze_wd.active`. The watchdog object is advanced at the END of
# cycle() (check_meter_frozen), so reading its level here would grade
# this quarter against the PREVIOUS cycle's conjunction - one cycle of
# lag, in a guard whose entire job is to act inside a 900 s quarter.
# `standing` is the value SAFETY-02 already computed above for exactly
# this meaning: the command on the wire that this meter reading is a
# response to. The predicate itself is freeze.py's shipped
# `looks_frozen()`, not a copy of it - see loop_is_following() on what a
# transcribed safety predicate costs.
#
# ⚠️ `achievable_w` BOUNDS THE FLOOR SO THIS CLAIM CANNOT CONTRADICT THE
# SAFETY LIMITS ABOVE. `lo > hi` in the arbiter is a CONTRADICTION and
# resolves to 0 W - the battery parked, in the one minute of the month it
# should be discharging hardest. maintenance_charge_w had to learn the
# same lesson (config.yaml, validate.py rule 6); this claim is built so
# the config cannot express it.
#
# ⚠️ AND IT IS FILED EVERY CYCLE, unconditionally, like SAFETY-02's. A
# claim is not sticky. Filed once and skipped next cycle, the floor
# lapses and the actuator goes silently back to whatever was pushing the
# peak. Which of the higher-priority claims outranks it is arbiter.py's
# business, not this block's - and the answer is already right: a
# maintenance DRAIN or HOLD owns the setpoint at P_MAINTENANCE and
# commands discharge or zero, neither of which can create offtake, while
# a maintenance CHARGE is only ever a P_MAINT_SHAPE limit (40) and is
# therefore bound by this floor. That is §9.1's "money outranks the
# maintenance schedule", now as a consequence of the priorities.
#
# ⚠️ AND IT IS NOT RUN AT ALL WITHOUT P1. `meter_source: off` is the
# legacy single-entity path: there is no telegram stream, so
# `p1.averager` has no open block, so the guard would be BLIND on every
# cycle forever - and blind means "no charging". That is not a
# conservative default, it is a controller that silently stops charging
# on every install that has not opted in to P1 yet; test_clamp.py's
# export case caught it doing exactly that. The billed quantity simply
# is not measured on that path, and the honest report is NOT COVERED -
# the same answer, for the same reason, that the status page already
# gives for SAFETY-18 on a transport that cannot see `unchanged_s`.
dev_top = self.dev_max if self.dev_max is not None else max_w
self.peak_guard = None if not self.p1_enabled else peakguard.decide(
self.peak.project(self.p1.averager, now),
batt_w=self.batt,
cap_w=float(self.o.get("peak_cap_w", 3500)),
reading=self.peak.peak(now),
achievable_w=min(max_w, abs(float(dev_top))),
# ⚠️ SAFETY-24. `self.soc` IS THE MEASURED READING AND MAY BE None,
# and it is handed over as None rather than defaulted here. An early
# draft of the stage-5 loop read SoC as `| float(100)` and thereby
# invented a full battery (NOTES.md 4.4b); a full battery is
# precisely the value that disarms this floor. decide() treats None
# as floor-limited, which is the direction that costs a shave rather
# than the pack.
soc=self.soc,
soc_floor=float(self.o.get(
"peak_soc_floor",
peakguard.PEAK_OPTION_DEFAULTS["peak_soc_floor"])),
max_hold_s=self.p1.averager.max_hold_s,
frozen=looks_frozen(getattr(self.p1_source, "unchanged_s", None),
standing, max_w, self.freeze_wd.cfg),
)
peak_claim = (peakguard.claim(self.peak_guard)
if self.peak_guard is not None else None)
if peak_claim is not None:
claims.append(peak_claim)
# ⚠️ EDGE-LOGGED, not level-logged. The action log holds 40 lines; a
# guard that appended one every 15 s for a whole quarter would push the
# cause of the event off the end of the log that is meant to explain it.
# SAFETY-03 logs every engagement because a clamp engagement IS the
# count that matters there; here the count is the seconds, and the
# engagement's own reason line already carries them.
engaged_now = self.peak_guard is not None and self.peak_guard.engaged
if engaged_now != self._peak_engaged:
self._peak_engaged = engaged_now
if self.peak_guard is not None:
self.log_event(self.peak_guard.explain())
# --- OPT-04: price arbitrage (§10.1 priority 3) ----------------------
# ONE `set` claim at P_ARBITRAGE, and app/arbitrage.py carries the whole
# argument. What is decided HERE and nowhere else is what the strategy
# is allowed to see:
#
# ⚠️ IT IS PLACED AFTER THE PEAK BLOCK BECAUSE IT READS ITS CONCLUSION.
# `engaged_now` is this cycle's peak verdict, computed ~20 lines up.
# Evaluated before it, this would hand the strategy the PREVIOUS cycle's
# answer - one cycle of lag on the interlock that implements §9.1's
# "peak shaving outranks arbitrage", inside a 900 s quarter.
#
# ⚠️ AND THE THIRD STATE IS NOT A TIDINESS. Without P1 the guard is not
# CLEAR, it never RAN, and `None` says so. arbitrage.decide() refuses on
# it, because charging at max_w creates the billed quantity while
# nothing is measuring the quarter it lands in. `False` here would be
# this line asserting a measurement that was never taken - the defect
# `PeakReading` and `quarter_ceiling_w()` both exist to remove.
#
# ⚠️ THE THREE-STATE COLLAPSE LIVES IN arbitrage.guard_state(), NOT
# HERE, and that is not tidiness. It is the same lesson
# `shaping_suspended()` records: inline, this expression was reachable
# by no check, and mutating it back to a bare `engaged_now` left every
# suite in this repository green while a `meter_source: off` install
# traded blind. Read guard_state() before touching this line.
#
# ⚠️ NO FETCH ON THIS PATH. `fresh_tariff_plan()` is an attribute read
# and an age comparison; the HTTPS call lives in run_tariff_refresh().
# See TARIFF_REFRESH_S: a supplier's TLS timeout here would be a
# supplier able to push this add-on towards layer 1's 30 s veto.
#
# ⚠️ AND THE UNCERTAINTY SET IS NOT EMPTY BY DEFAULT. §11.4's
# no-weather row widens the threshold, and this add-on carries no
# weather forecast at all - §8.1's PV forecast is not built. Passing
# `frozenset()` would be main.py asserting a forecast is healthy when
# there is not one, which is exactly what static_tariff's `uncertainty`
# argument has no default in order to prevent.
self.arb = None
if arbitrage.enabled(self.o):
floor_pct, ceiling_pct = arbitrage.soc_bounds(self.o)
self.arb = arbitrage.decide(
self.fresh_tariff_plan(now), now,
peak_guard_active=arbitrage.guard_state(self.peak_guard),
soc=self.soc,
min_spread_eur_kwh=arbitrage.threshold_eur_kwh(self.o),
uncertainty=frozenset({Uncertainty.NO_WEATHER}),
charge_w=max_w, discharge_w=max_w,
soc_floor_pct=floor_pct, soc_ceiling_pct=ceiling_pct)
arb_claim = arbitrage.claim(self.arb)
if arb_claim is not None:
claims.append(arb_claim)
# Edge-logged, for the peak guard's reason: the action log holds 40
# lines and a trade lasts a whole hour, so a line per cycle would
# flush the context that explains it. The transition is what an
# operator needs - "it started buying at 03:00", not 240 copies.
if self.arb.action != self._arb_action:
self._arb_action = self.arb.action
self.log_event(self.arb.explain())
# --- maintenance -----------------------------------------------------
if result.owns_setpoint:
claims.append(Claim.set("maintenance", P_MAINTENANCE,
@@ -335,9 +645,44 @@ class Controller:
# claimed, so the loop can discharge and shave the peak. When a
# dedicated peak-shaving strategy arrives it will claim SET at
# P_PEAK and outrank this limit without any code here changing.
if peak_at_risk(self.peak_fc, float(self.o.get("peak_cap_w", 3500))):
self.log_event("peak at risk - maintenance charge shaping suspended")
# ⚠️ AND SAFETY-07 IS THE OTHER HALF OF THIS CONDITION, NOT A
# SECOND MECHANISM. The shaping is `hi <= 0`; the peak guard's claim
# is `lo >= 0`. Both are limits, arbiter.py applies EVERY limit at or
# above the winner's priority, and most restrictive wins - so a peak
# event during a maintenance charge window files `lo > hi`, which is
# a CONTRADICTION and resolves to 0 W. The battery would park in the
# exact quarter it should be shaving. Priority does not save it:
# clause 2 has no notion of a higher-priority limit overriding a
# lower one, and it should not - "most restrictive" is what makes the
# safety bounds unconditional.
#
# So the higher-priority claimant SUPPRESSES the lower one rather
# than out-voting it, which is what this branch already did for
# `peak_forecast_entity`. That entity is optional and empty on a
# default install, so until now the suspension could not fire on the
# site that needed it. `self.peak_guard` is measured from the meter
# and is always there.
#
# ⚠️ THE CONJUNCTION LIVES IN peakguard.py, NOT HERE, and that is not
# tidiness. It was inline, and a reviewer mutated it back to its
# historical broken form - re-gated on `peak_forecast_entity` alone -
# and every suite in this repository stayed green while a default
# install parked at 0 W in the quarter that sets the month's bill.
# Read shaping_suspended() before touching this: both terms are
# there, and why neither is sufficient alone.
if peakguard.shaping_suspended(
self.peak_guard, self.peak_fc,
float(self.o.get("peak_cap_w", 3500))):
# Edge-logged: level-logging here would write one line per cycle
# for as long as the event lasts and flush the 40-line action log
# of the very context that explains it.
if not self._maint_shape_suspended:
self._maint_shape_suspended = True
self.log_event("peak at risk - maintenance charge shaping "
"suspended so the peak claim is not "
"contradicted")
else:
self._maint_shape_suspended = False
hi = 0.0
reason = "charge-only"
if self.cheap:
@@ -379,11 +724,29 @@ class Controller:
self.sat_count = decision.sat_count
self.i_w = decision.i_w
self.loop_w = decision.target_w
# --- SAFETY-03 ---------------------------------------------------
# ⚠️ READ THE BOOLEANS, NOT `decision.reason`. On a default install
# the integrator bound is what bites and the output clamp is
# unreachable (control.py says so), and `reason` is then overwritten
# by whichever of slew-limiting or the saturation freeze ran after
# it - so a string test is silent on the two cases a winding loop
# most often presents as. These flags are set where the clamp
# happens and nothing downstream touches them.
self.loop_clamped = (
decision.i_clamped or decision.clamped,
(f"the loop asked for {decision.i_w:+.0f} W"
+ (" (integrator bound)" if decision.i_clamped else "")
+ (" (output bound)" if decision.clamped else "")),
)
claims.append(Claim.set("loop", P_LOOP, decision.target_w, decision.reason))
else:
# Stopped or blind: no accumulation may survive the outage.
self.i_w = 0.0
self.loop_w = None
# A loop that is not running cannot be winding. Clearing this here
# rather than letting the last value stand is what makes the alarm
# de-assert when control is stopped.
self.loop_clamped = (False, "")
resolution = resolve(claims)
if resolution.contradiction:
@@ -406,14 +769,257 @@ class Controller:
self.reason = resolution.explain()
await self.write_setpoint(self.target)
# --- SAFETY-18, and it runs LAST on purpose --------------------------
# Both terms of the conjunction describe the state that now exists: the
# command that has just gone on the wire, and how long the meter that
# chose it has been standing still. Evaluated before write_setpoint()
# it would be reasoning about the PREVIOUS command against THIS cycle's
# meter - the one combination that is true of neither moment.
# --- SAFETY-03, and its boundary is the WRITE ------------------------
# ⚠️ NOT THE SAME BOUNDARY AS SAFETY-18's BELOW, THOUGH THE CALLS ABUT.
# This one reads `self.clamp_last`, which is None until write_setpoint()
# has actually run, so it has to follow the write. The one below grades
# `self.target`, which is assigned further up - see check_output_clamp()
# and check_meter_frozen() for each. Two guards, two orderings, adjacent
# lines: do not "tidy" them into one rule.
await self.check_output_clamp()
# --- SAFETY-18, and its boundary is the ASSIGNMENT --------------------
# ⚠️ THE BOUNDARY IS `self.target = resolution.target_w` ABOVE, NOT
# write_setpoint(). This comment used to say "the command that has just
# gone on the wire", which is wrong twice over and was corrected by
# TEST-06's reviewer: `check_meter_frozen()` grades `self.target` - the
# arbiter's resolved intent - and `write_setpoint()` clamps only a local
# `value` and can skip the write entirely, so that number may never have
# gone anywhere.
#
# What both terms of the conjunction must describe is the SAME moment:
# the command this cycle resolved, against how long the meter that chose
# it has been standing still. Evaluated before
# `self.target = resolution.target_w` it would grade the PREVIOUS
# cycle's command against THIS cycle's meter - the one combination that
# was true of neither moment. Moving it between the assignment and
# write_setpoint() is therefore harmless; moving it above the assignment
# is not, and test_freeze.py part 5 fails on exactly that.
await self.check_meter_frozen()
# --- SAFETY-07's alarm, and it is NOT the engagement ------------------
# Its boundary is unconstrained: it grades `self.peak_guard`, which this
# method assigned above and nothing since has touched.
await self.check_peak_saturated()
# --- SAFETY-24's alarm, and it is the OTHER way the money gets spent --
# Same boundary argument as check_peak_saturated(): it grades
# `self.peak_guard`, which cycle() assigned above and nothing since has
# touched. Kept as its own method rather than folded into that one,
# because the two states are mutually exclusive and want OPPOSITE
# operator actions - saturation says "shed load, the inverter is already
# flat out"; this says "the pack is at its floor and this quarter is not
# being shaved at all".
await self.check_peak_floor_limited()
# --- TARIFF-01, and it is LAST on purpose -----------------------------
# Banking a quarter is bookkeeping about a moment that has already
# happened; a slow SD card must never sit between the meter and the
# setpoint. Its position is unconstrained in the way SAFETY-03's and
# SAFETY-18's are not - it grades neither `target` nor `standing` and
# reads only `self.p1`, which no line in this method writes.
self.peak.sync(self.p1)
async def check_output_clamp(self) -> None:
"""SAFETY-03. Alarm whenever a clamp actually engaged this cycle.
Runs AFTER write_setpoint() for the same reason SAFETY-18 does: the
clamp is arithmetic done inside the write, so before the write there is
nothing yet to report.
⚠️ PUBLISHING IS NOT IN THE WRITE PATH. The clamp itself is synchronous
inside write_setpoint(); this - the awaits, the notification, anything
that can block on Home Assistant - is out here. A stalled supervisor must
never be able to delay or drop a frame.
⚠️ THIS IS NOT A TRIP. No claim is filed, nothing is latched, 0 W is not
commanded and `publish()` still reports "running". §11.2's action for
this row is "clamp and alarm", and the clamped value is written and the
loop keeps running. Turning this into a trip means coming here and
changing that deliberately, not discovering it in the field.
"""
engaged_write = bool(self.clamp_last and self.clamp_last.engaged)
engaged_loop, loop_note = self.loop_clamped
engaged = engaged_write or engaged_loop
if engaged:
where = (self.clamp_last.describe() if engaged_write
else f"{CLAMP_REASON}: {loop_note}, above the "
f"{float(self.o.get('max_w', 2000)):.0f} W rating, and "
f"was clamped to {self.target:+.0f} W")
reason = (f"{where}. This is a BUG SIGNAL, not a limit being used: "
f"something upstream computed a setpoint the actuator "
f"cannot deliver.")
else:
reason = ""
fired = self.clamp_wd.note(engaged, reason, datetime.now(timezone.utc))
# --- THE ACTION LOG, AND IT IS COALESCED PER EPISODE ------------------
# Every engagement is recorded, but consecutive ones are recorded by
# UPDATING one row's count rather than inserting a row each.
#
# ⚠️ THAT IS NOT COSMETIC, AND IT WAS FOUND THE HARD WAY. `log_event()`
# keeps 40 rows; a winding loop clamps on every cycle, so a row per
# engagement flushes the whole ring within 40 cycles - and the first
# thing it evicted, in test_freeze.py, was SAFETY-18's "METER FROZEN?"
# row. A guard whose log drowns out the other guards' logs has moved the
# flood rather than removed it. The count here is `clamp_wd.engagements`,
# so nothing is lost: the row says exactly how many, and the number is
# read off the counter rather than off the number of rows.
#
# This lives here rather than only in write_setpoint() because on a
# default install the actuator-side clamp is unreachable - compute() has
# already bounded the integrator - so the only engagements that exist are
# loop-side ones. Measured: 60 cycles of the §10.3 runaway produce 0
# write-path engagements and 2 loop-side ones.
if engaged and not engaged_write:
tag = f"{CLAMP_REASON} episode {self.clamp_wd.episodes}"
row = (f"{tag}: {self.clamp_wd.engagements} engagement(s) - "
f"{reason}")
if self.events and tag in self.events[0]:
stamp = self.events[0].split(" ", 1)[0]
self.events[0] = f"{stamp} {row}"
else:
self.log_event(row)
if not fired:
return
_LOG.error(
"SAFETY-03 - %s Silently clipping this is how a winding integrator "
"reached 14 768 W on this project before: the clamp makes every "
"dashboard read a legal number while the thing producing it goes on "
"climbing. The command IS clamped and the loop is STILL RUNNING - "
"this is an alarm, not a trip. Engagements so far: %d.",
self.clamp_wd.reason, self.clamp_wd.engagements)
# A fixed notification_id, for the reason check_meter_frozen() gives -
# and a DIFFERENT one from every other alarm this add-on raises, because
# a clamp is not a frozen meter and neither is a saturation freeze.
await self.hass.call("persistent_notification", "create", {
"notification_id": "goodwe_output_clamp",
"title": "GoodWe: a computed setpoint exceeded the rating",
"message": (
f"{self.clamp_wd.reason}\n\nThe command has been clamped and "
f"the controller is STILL RUNNING. This is NOT the inverter "
f"failing to follow a legal command (that is the saturation "
f"freeze, which is normal and does not alarm) - the value asked "
f"for was itself out of range."),
})
async def check_peak_saturated(self) -> None:
"""SAFETY-07. Alarm ONLY when the guard has run out of inverter.
⚠️ AN ENGAGEMENT IS NOT AN ALARM, AND THE DIFFERENCE IS §10.3's DOCTRINE
READ CAREFULLY. "Treat an out-of-range computed setpoint as a bug signal
- clamp it, then alarm" is SAFETY-03's rule and it is right there: the
loop computing 14 768 W is a WINDING INTEGRATOR, a defect, and clipping
it silently is what let it keep climbing behind a dashboard full of legal
numbers. A peak floor above the inverter's rating is not a defect. It is
a house load bigger than the inverter - 2026-08-20 15:45 ran ~5900 W
against a 5000 W unit - and alarming on the ordinary case is how an
operator learns to dismiss the notification that matters. So an
engagement is an ordinary control action: it goes to the §15 action log
and the status page, and no further.
What DOES earn the operator's attention is saturation: the guard asked
for more discharge than the inverter can give, so the quarter is going
over the ceiling anyway and the controller has already done everything
available to it. That is the one state where the money is going to be
spent and a human is the only remaining actuator.
Edge-triggered, and de-asserting arms it again - the same shape as
FreezeWatchdog's `evaluate()`, and for the same reason: one notification
per episode, not one per cycle.
"""
g = self.peak_guard
saturated = bool(g is not None and g.saturated)
was, self._peak_saturated = self._peak_saturated, saturated
if not saturated or was:
return
self.log_event("PEAK GUARD SATURATED - " + g.reason)
_LOG.error(
"SAFETY-07 - %s. The guard is asking for everything the inverter "
"has and the quarter is still projected over the ceiling: this "
"quarter will set or raise the month's billed capacity peak. The "
"controller is STILL RUNNING and still shaving; there is nothing "
"further it can do. Shed load.", g.reason)
# A fixed notification_id, and a DIFFERENT one from every other alarm
# here - check_meter_frozen() gives the reasoning. A saturated peak guard
# is not a frozen meter and is not an out-of-range setpoint.
await self.hass.call("persistent_notification", "create", {
"notification_id": "goodwe_peak_saturated",
"title": "GoodWe: the capacity-peak guard has run out of inverter",
"message": (
f"{g.reason}.\n\nThe battery is being commanded to everything "
f"it can give and the quarter is STILL projected over the "
f"{g.ceiling_w:.0f} W ceiling, so this quarter will set or raise "
f"the month's billed capaciteitstarief peak. This is not a "
f"controller fault - the load is simply larger than the "
f"inverter. Shedding load in the next few minutes is the only "
f"thing that changes the bill."),
})
async def check_peak_floor_limited(self) -> None:
"""SAFETY-24. Alarm when the EMS SoC floor stops the guard shaving.
⚠️ FAILING TO SHAVE MUST NOT BE SILENT, AND THAT IS THE WHOLE POINT.
The guard exists for one euro figure - the capaciteitstarief bills the
month's single worst quarter - so a cycle in which it CHOSE not to shave
is a cycle in which money is being spent on purpose. Under-shaving
quietly would leave the operator with a bill and a status page full of
green.
The distinction from check_peak_saturated() is the operator's action,
which is why these are two notifications and not one:
saturated the guard asked for everything the inverter has and it
was not enough. SHED LOAD - the controller has already
done all it can, and it is still discharging.
floor-limited the guard asked for NOTHING, because the pack is at or
under the EMS floor, or its SoC could not be read.
Nothing is being discharged for this quarter at all.
They cannot both be true: peakguard.decide()'s floor-limited branch
returns before the `achievable_w` clamp that sets `saturated`.
Edge-triggered, and de-asserting arms it again - FreezeWatchdog's shape,
for FreezeWatchdog's reason: one notification per episode, not one per
cycle for a quarter of an hour.
"""
g = self.peak_guard
limited = bool(g is not None and g.floor_limited)
was, self._peak_floor_limited = self._peak_floor_limited, limited
if not limited or was:
return
# No prefix: `reason` already opens with PEAK GUARD FLOOR-LIMITED, and
# a second copy is the kind of stutter that makes a 40-line action log
# read like boilerplate. The saturated path needs its prefix because its
# reason carries the word in the middle of a sentence, not at the front.
self.log_event(g.reason)
_LOG.error(
"SAFETY-24 - %s. The controller is STILL RUNNING and charging is "
"still blocked, but it is not discharging to hold this quarter, so "
"the quarter will set or raise the month's billed capacity peak. "
"Shed load, or lower peak_soc_floor if this pack can afford it.",
g.reason)
# A fixed notification_id, and a DIFFERENT one from every other alarm
# here - check_meter_frozen() gives the reasoning. A battery at its
# energy floor is not a saturated inverter and is not a frozen meter,
# and an operator who has to dismiss three identical-looking cards
# learns to dismiss the one that mattered.
await self.hass.call("persistent_notification", "create", {
"notification_id": "goodwe_peak_soc_floor",
"title": "GoodWe: the capacity-peak guard has run out of battery",
"message": (
f"{g.reason}.\n\nTHIS QUARTER WILL GO OVER THE "
f"{g.ceiling_w:.0f} W ceiling and will set or raise the "
f"month's billed capaciteitstarief peak, because the peak guard "
f"is not allowed to discharge any further: {g.soc_floor_reason}."
f" This is not a controller fault - it is the EMS-side energy "
f"floor (peak_soc_floor) doing what it is set to do, and it sits "
f"deliberately above the pack's own BMS cut-out. Charging stays "
f"blocked. Shedding load is the only thing that changes the bill "
f"now; if this pack can afford a deeper discharge, lower "
f"peak_soc_floor."),
})
async def check_meter_frozen(self) -> None:
"""SAFETY-18. Alarm on a meter that answers forever without changing.
@@ -454,6 +1060,57 @@ class Controller:
})
# -- tasks ---------------------------------------------------------------
# -- OPT-04: the background tariff refresh -------------------------------
def fresh_tariff_plan(self, now):
"""The stored plan if it is still current, else None. NEVER a stale one.
⚠️ THIS IS THE ONE PLACE THE AGE IS JUDGED, and it is public for
`loop_is_following()`'s reason: a check has to be able to assert on the
shipped expression rather than on a transcription that drifts. Both
terms are required - a plan with no timestamp is one nothing stored, and
a timestamped plan older than TARIFF_MAX_AGE_S is a reading from a task
that has stopped answering.
"""
if self.tariff_plan is None or self.tariff_at is None:
return None
if (now - self.tariff_at).total_seconds() > TARIFF_MAX_AGE_S:
return None
return self.tariff_plan
async def run_tariff_refresh(self) -> None:
"""Resolve the tariff off the control path, forever. NEVER on the tick.
⚠️ IT CATCHES EVERYTHING AND KEEPS GOING, for MqttPublisher's reason:
a supplier's outage, a certificate expiry, a malformed feed - all of it
is pricing, and the battery does not care. What it must never do is
raise out of here and take the task down, because then the plan freezes
at its last value and looks current forever. That is what
`fresh_tariff_plan()`'s age bound is the backstop for, and this is the
half that keeps the task alive so the bound rarely matters.
⚠️ AND A FAILURE DOES NOT CLEAR THE PLAN. `resolve_tariff()` already has
a §11.4 answer for a dead feed - degrade to the static profile, mark it
provisional, arbitrage refuses - and that answer is better than the
absence of one. Zeroing the plan here would replace a coarse tariff with
no tariff, which is the opposite of what §11.4 asks for.
"""
horizon = float(price_option(self.o, "price_horizon_h"))
while not self.stopping:
now = datetime.now(timezone.utc)
try:
plan = await resolve_tariff(
self.o, now, now + timedelta(hours=horizon), now=now)
self.tariff_plan, self.tariff_at = plan, now
_LOG.info("tariff refreshed: %s", plan.words)
except Exception as err: # noqa: BLE001
_LOG.warning(
"tariff refresh failed (%s); keeping the plan resolved at "
"%s. It retires on its own after %.0f s and arbitrage "
"stops with it.", err, self.tariff_at, TARIFF_MAX_AGE_S)
with contextlib.suppress(asyncio.TimeoutError):
await asyncio.wait_for(self.stop_event.wait(),
timeout=TARIFF_REFRESH_S)
async def run_control(self) -> None:
entity = self.o.get("setpoint_entity", "")
if entity:
@@ -668,6 +1325,68 @@ class Controller:
f"{self.freeze_wd.cfg.hold_s:.0f} s unchanged "
f"while the command is over "
f"{rail_w(self.o.get('max_w', 2000)):.0f} W)"})
# TARIFF-01. The month's billed peak, and - the load-bearing half -
# a DIFFERENT row when the store could not answer. "no peak yet" is
# a normal state on the 1st; "unreadable" is a broken instrument, and
# a status page that spelled them the same way would let a 0 W read
# look like a quiet month right up until SAFETY-07 shaved nothing.
_pk = self.peak.peak()
if _pk.is_unreadable:
out.append({"ok": False, "warn": True,
"text": "capacity-tariff peak: " + _pk.explain()})
else:
# §9.1's "warning entity at 80 %", as a status row rather than
# a second entity - see the constructor on not doubling up.
_frac = self.peak.warning_frac() or 0.0
_ok = _frac < 0.8
out.append({"ok": _ok, "warn": not _ok,
"text": _pk.explain()
+ f" - {_frac * 100:.0f} % of the "
f"{self.peak.cap_w:.0f} W soft limit"
+ f"; {self.peak.recorded} quarter(s) banked"
+ f"; on start: {self.peak.restore_state}"})
# SAFETY-07. Three rows, not one, because "idle" and "cannot see"
# are opposite states and a page that spelled them the same way
# would show a green line over a guard that is switched off. `None`
# is a fourth state on purpose: before the first cycle the guard has
# concluded nothing, and saying "idle" then would be a claim about a
# decision that has not been taken.
_pg = self.peak_guard
if _pg is None:
out.append({"ok": True, "warn": True,
"text": "peak guard: no decision yet - the first "
"control cycle has not run"})
elif _pg.saturated:
out.append({"ok": False, "warn": False,
"text": "PEAK GUARD SATURATED - " + _pg.reason})
elif _pg.floor_limited:
# SAFETY-24. NOT ok, like saturation: in both states the money
# is going to be spent and a human is the only actuator left. A
# separate row from saturation because the two ask for opposite
# things - see check_peak_floor_limited().
out.append({"ok": False, "warn": False,
"text": _pg.explain()})
elif _pg.blind:
out.append({"ok": False, "warn": True, "text": _pg.explain()})
else:
out.append({"ok": not _pg.engaged, "warn": _pg.engaged,
"text": _pg.explain()})
else:
# ⚠️ OUTSIDE the `p1_enabled` block, and that placement is the point.
# SAFETY-07 is NOT COVERED on the legacy single-entity path - the
# quarter-hour average is measured from P1 telegrams and this path
# has none - and a row that only rendered where the guard runs would
# leave the one install that is unprotected showing nothing at all.
# Same answer, same reason, as the freeze detector's NOT COVERED row
# a few lines up.
out.append({"ok": False, "warn": True,
"text": f"peak guard: NOT COVERED on meter_source "
f"{o.get('meter_source')} - the quarter-hour "
f"average is measured from P1 telegrams and this "
f"path has none, so a runaway cannot be capped"})
rows = [("battery SoC", self.soc, o.get("soc_entity")),
("battery power", self.batt, o.get("batt_entity"))]
if not self.p1_enabled:
@@ -694,6 +1413,23 @@ class Controller:
out.append({"ok": True, "warn": False,
"text": f"setpoint {sp} (range {self.dev_min:g}…{self.dev_max:g} W)"})
# SAFETY-03. Red while a clamp is engaged, and the engagement COUNT is
# shown even after it clears - a loop that wound once and stopped is
# still a bug, and a row that vanishes with the condition takes the
# evidence with it. Reported only when there is something to report:
# unlike the freeze detector this guard has no "not covered" state, it
# runs on every write on every transport.
if self.clamp_wd.active:
out.append({"ok": False, "warn": False,
"text": "OUTPUT CLAMP ENGAGED - " + self.clamp_wd.reason
+ f" Engagements: {self.clamp_wd.engagements}."})
elif self.clamp_wd.engagements:
out.append({"ok": False, "warn": True,
"text": f"output clamp: not engaged now, but it has "
f"engaged {self.clamp_wd.engagements} time(s) in "
f"{self.clamp_wd.episodes} episode(s) since "
f"start - see the event log"})
if self.last_write_ok is False:
out.append({"ok": False, "warn": False,
"text": "last write to the inverter was REJECTED - check the log"})
@@ -758,6 +1494,12 @@ class Controller:
("Maintenance phase", self.maint.phase),
("Maintenance due", "now" if next_due is None else next_due.strftime("%Y-%m-%d")),
("Cheap window", "yes" if self.cheap else "no"),
("Tariff", self.tariff_words),
# OPT-04. The strategy's own sentence, not a re-derivation of it -
# every refusal it can make already carries a reason a human can
# act on, and "Arbitrage: no" would throw all of them away.
("Arbitrage", "off (arbitrage_enabled)" if self.arb is None
else self.arb.explain()),
]
return {
"banner": banner, "level": level, "rows": rows, "checks": checks,
@@ -885,6 +1627,19 @@ async def amain() -> None:
tasks = [asyncio.create_task(controller.run_control())]
# OPT-04. The tariff refresh runs as its OWN long-lived task, for the
# reason P1 ingestion does and then some: `resolve_tariff()` may open an
# HTTPS connection to a supplier, and the control tick is what feeds the
# ESP32 heartbeat. A fetch on that path is a supplier's TLS timeout able
# to push this add-on towards layer 1's 30 s veto. See TARIFF_REFRESH_S.
#
# ⚠️ STARTED ONLY WHEN ARBITRAGE IS ARMED. Nothing else in the add-on
# reads the resolved plan - `cheap` still comes from static_status() on
# the tick, which is pure - so on a default install this task would be
# network traffic in aid of an attribute nobody reads.
if arbitrage.enabled(opts):
tasks.append(asyncio.create_task(controller.run_tariff_refresh()))
# P1 ingestion runs as its own long-lived task. ⚠️ It must not be driven
# off the control loop: telegrams arrive every ~5 s and the loop would
# decimate them, so the 15-minute average - the capacity-tariff billing
+68
View File
@@ -286,6 +286,26 @@ class QuarterAverager:
"""Offtake watt-seconds accumulated in the open block so far."""
return self._acc
@property
def held_net_w(self) -> float:
"""The signed net power currently being held forward, in W.
PUBLIC BECAUSE SAFETY-07 DEPENDS ON IT, AND THAT IS THE WHOLE REASON.
`PeakTracker.project()` extrapolates the rest of the quarter from this
value and used to reach it as `getattr(averager, "_last_net", 0.0)`. A
default on a private name fails QUIET and fails in the DISARMING
direction: rename `_last_net` and every projection silently credits 0 W
held, which reads LOW, which puts the peak guard under the ceiling
exactly when the meter says it is over. Filed as TEST-13 item 4 and
fixed here rather than inherited - a money guard may not rest on a read
that can go wrong without a symptom.
Positive is offtake, matching `P1Sample.net_w`. It is the raw held
value, NOT floored at zero: the flooring is the accumulator's rule (only
offtake is billed) and belongs where it is applied, not here.
"""
return self._last_net
@property
def offtake_avg_w(self) -> float:
"""Average offtake over the part of the open block seen so far."""
@@ -353,6 +373,54 @@ class QuarterAverager:
self._last_pp = sample.per_phase_w
return closed
# -- surviving a restart (TARIFF-01) ------------------------------------
def snapshot(self) -> dict:
"""The open block's whole state, as JSON-safe primitives.
⚠️ `last_t` is the HELD SAMPLE'S OWN timestamp and is stored as it is,
not refreshed to "now". That is what makes a restart integrate the gap
under exactly the `max_hold_s` rule a running process would have used:
a short restart is a plateau, a long one leaves uncovered seconds that
drag the closed block's average down. Rewriting it to the restore time
would silently credit the whole outage at the last value seen before it.
"""
return {
"block": self._block,
"acc": self._acc,
"pp_acc": list(self._pp_acc),
"elapsed": self._elapsed,
"last_t": self._last_t,
"last_net": self._last_net,
"last_pp": list(self._last_pp) if self._last_pp is not None else None,
"phases": self.phases,
}
def restore(self, state: dict) -> bool:
"""Put a `snapshot()` back. False if it does not fit this averager.
A phase-count change is a reconfiguration, not a resumption: the
per-phase accumulator would be reshaped mid-block and the two halves
would not be the same measurement. Refusing is the honest answer, and
the caller reports starting fresh rather than pretending it resumed.
"""
try:
if int(state.get("phases", self.phases)) != self.phases:
return False
pp_acc = list(state.get("pp_acc") or [])
if len(pp_acc) != self.phases:
return False
self._block = None if state["block"] is None else int(state["block"])
self._acc = float(state["acc"])
self._pp_acc = [float(v) for v in pp_acc]
self._elapsed = float(state["elapsed"])
self._last_t = None if state["last_t"] is None else float(state["last_t"])
self._last_net = float(state["last_net"])
lp = state.get("last_pp")
self._last_pp = tuple(float(v) for v in lp) if lp is not None else None
return True
except (KeyError, TypeError, ValueError):
return False
# --------------------------------------------------------------------------- #
# what the rest of the add-on talks to
+725
View File
@@ -0,0 +1,725 @@
"""SAFETY-07 - the quarter-peak projection guard, and the only thing in this
add-on that is allowed to spend the loop's freedom to save money.
WHAT IT IS. TARIFF-01 built the INSTRUMENT: `tariff.py` measures the number the
Belgian capaciteitstarief bills on - the highest 15-minute average offtake of the
calendar month - and banks it where a restart cannot reach. It files no claim and
commands nothing, and says so in its own first paragraph. This module is the
CONSUMER it named. It turns that measurement into one arbiter claim and nothing
else: no writes, no I/O, no clock, no state between calls. Given a projection and
two measured numbers it returns a decision; `main.py` files it.
------------------------------------------------------------------------------
THE HAZARD, WITH THE EURO FIGURE
specs.md:583 records a measured case: ONE 4998 W quarter moved a monthly peak
from 4832 W to 5013 W and billed the whole month at it. The capacity tariff has
no averaging and no forgiveness - the month's single worst quarter is the bill.
SAFETY-18 shipped the ALARM for the way this system produces such a quarter by
itself: a meter that answers 200 OK forever with a document that never changes,
which every freshness mechanism in this stack reads as healthy while the loop
integrates a constant error to the rail and holds it there. freeze.py's own
table: at the live install's `max_w: 5000`, a meter frozen at >= 3000 W rails the
command in 75 s. SAFETY-18 notifies at `meter_freeze_s` (300 s default), against
a 900 s billing quarter. That is about nine minutes of human reaction time and
nothing else - freeze.py says so in the words "THIS CUT DOES NOT CAP THAT."
This is the cap. It is deliberately NOT a second freeze detector: it consumes
freeze.py's shipped predicate rather than transcribing it.
------------------------------------------------------------------------------
THE MECHANISM: ONE `lo` LIMIT AT `P_PEAK`, AND WHY THAT SHAPE
specs.md 10.1 row 2 asks for exactly two behaviours - "pre-emptive discharge as
the projected quarter average approaches the soft limit; hard clamp on charging
above it" - and 9.1 constrains the lever: "Discharge and export are never capped
by the peak guard; only charging and import are."
A LOWER BOUND on the setpoint is both of those in one object, and it is the only
shape that is:
* incapable of capping discharge - a floor never stops the loop discharging
HARDER, so 9.1's asymmetry is structural here rather than remembered;
* incapable of capping export, for the same reason;
* a hard clamp on charging whenever the floor is above the standing charge;
* pre-emptive discharge whenever the floor is above zero.
It is a LIMIT, not a SET, and that matters at both ends of the priority table.
`P_PEAK` (50) is above `P_LOOP` (10), so it binds the grid-following loop. It is
BELOW `P_MAINTENANCE` (60) and `P_SAFETY` (100), so a maintenance phase that owns
the actuator and every layer-1 stop still outrank it: arbiter.py's clause 2 only
applies limits whose priority is >= the WINNING SET's, so a P_SAFETY 0 W stop is
not dragged off zero by this floor. That is the correct order - a controller
pushing power the wrong way is wrong however much money it is trying to save.
⚠️ AND A LIMIT DOES NOT OUT-VOTE ANOTHER LIMIT. This was assumed, and it is
false: arbiter.py clause 2 applies EVERY limit at or above the winner's priority
and takes the MOST RESTRICTIVE, with no notion of priority between them - which
is exactly what makes the safety bounds unconditional and must not be changed.
So this floor (`lo >= 0`) and the maintenance charge-only shaping (`hi <= 0`) do
not resolve in P_PEAK's favour when both are filed. They resolve to `lo > hi`, a
CONTRADICTION, which arbiter.py answers with 0 W - the battery parked in the one
quarter of the month it should be shaving hardest. The rule for two limits that
can cross is therefore SUPPRESSION, not priority: main.py drops the maintenance
shaping for as long as this guard is engaged, which is the same thing that branch
already did for `peak_forecast_entity` and is why 9.1's "money outranks the
maintenance schedule" reads as a suppression there rather than as a priority
here. The floor's clamp to `achievable_w` closes the only other pair that could
cross - the P_SAFETY rating bounds.
AND IT IS WHY OPT-04 NEEDS NO CHANGE HERE. 9.1 requires peak shaving to outrank
arbitrage. Arbitrage does not exist yet, but it will arrive as a SET below
P_PEAK, and arbiter.py clause 2 will bind it with this floor on the day it lands,
with no edit to this file. Expressing the priority as a claim rather than as an
if/else in main.py is the whole reason that holds.
------------------------------------------------------------------------------
THE FLOOR IS DERIVED, AND EVERY TERM OF IT IS MEASURED
Let `ceiling_w` be the quarter average that must not be exceeded and `remaining`
the seconds left in the open block. `project()` gives
projected = (acc + held * remaining) / 900
so the held level that would land the quarter exactly on the ceiling is
`held + 900 * (ceiling - projected) / remaining`, and the watts that must come
off the meter are the difference between the two:
excess_w = 900 * (projected - ceiling) / remaining
That single expression carries the whole control law, and two properties of it
are the argument for this design:
1. THE CLAIM IS FILED WHILE THE FLOOR IS POSITIVE, NOT WHILE THE EXCESS IS.
`excess` is what must come off the METER, so it falls to zero the moment the
battery is already taking it - and a guard that disengaged there would hand
the actuator back, watch the offtake spring straight to the raw load, and
engage again next cycle. That is not a hypothetical: it is what the first
cut of this file did to 2026-08-20's billed quarter on replay - seventeen
engage/release cycles in 900 s, and the quarter still finished 49 W over the
limit it had spent a quarter of an hour holding. 10.2 names the cost
exactly: "an EMS that oscillates is worse than no EMS, both for hardware
wear and - per 9.1 - for the peak it is trying to avoid."
`batt_w + excess` IS the quantity to threshold: the offtake there would be
with the battery at zero, less the level this quarter can still afford. Read
as a LEVEL rather than as a switch, the same arithmetic gives a claim that
is filed for as long as a positive floor is needed and decays continuously
to nothing as the load falls. So there is still no hysteresis constant in
this file - not because oscillation was ruled out by argument, but because
the thresholded quantity is the one that does not oscillate.
2. THE GAIN IS 900/remaining, WHICH IS 1.0 AT THE START OF THE QUARTER AND
GROWS. This is the answer to the degeneracy at the top of a block, where the
projection is an extrapolation from almost no evidence: the guard is at its
GENTLEST exactly when its evidence is weakest, and becomes aggressive only
once most of the quarter is behind it and the projection is nearly a
measurement. The formula supplies that; no minimum-evidence threshold was
invented, because inventing one would delay the guard in the case it exists
for - a sustained rail from second zero.
The floor itself is then
floor_w = batt_w + excess_w
`batt_w` IS THE MEASURED BATTERY POWER, NOT THE STANDING COMMAND. Both were
available. The measured one is right for two independent reasons. First, 9.1's
settled constraint is that the utility bills what FLOWED - the arithmetic
`grid = load - batt` is true of delivered power and false of intent. Second, and
concretely: when the inverter cannot follow (the saturation case control.py
already freezes the integrator for), a floor built on the command asks for
`command + excess` every cycle while the command never arrives, and winds itself
to the rail; a floor built on the measurement asks for `delivered + excess`,
which stops rising the moment delivery stops rising. The delivered form cannot
wind up. The intent form can.
------------------------------------------------------------------------------
THE CEILING IS `peak_cap_w`, RAISED ONLY BY AN OBSERVATION
11.2's row is "Quarter peak | Projected quarter average approaches THE MONTHLY
PEAK | Clamp the command", and 9.1's soft limit is `peak_cap_w`. Both matter,
because the money is spent on `max(month's peak so far, this quarter)`:
ceiling_w = max(peak_cap_w, the month's banked peak)
Once a month's peak already stands at 5013 W, holding every later quarter to
3500 W buys nothing - the bill for that month is already 5013 W - while the
holding itself costs real charging the site wanted. Raising the ceiling to a peak
that has already been paid for is what stops one bad quarter from parking the
battery's charging for three weeks.
AND THAT RELAXATION IS GRANTED ONLY BY AN `OBSERVED` READING. `NO_RECORD` and
`NOT_OBSERVED` both leave the ceiling at `peak_cap_w`, the tighter of the two, so
a peak store that cannot be read makes this guard STRICTER and never looser. That
direction is the whole point: tariff.py's `PeakReading` exists because a database
that did not answer must not become a number, and the number it would have become
here - a zero peak - is exactly the one that reads as "no peak recorded, full
freedom". It is also why this module does NOT consume `headroom_w()`, whose
disarming failure it would otherwise inherit: a peak read as 0 W returns FULL
headroom, and full headroom is a guard that is not there.
------------------------------------------------------------------------------
BLINDNESS COMES FIRST, AND IT IS A DIFFERENT ACTION
A projection is evidence only if the quantity under it was actually measured.
Three ways it is not, all of them checked BEFORE the projection is graded,
because grading a stimulus where the quantity it grades is degenerate is how a
guard passes its own tests while being switched off:
* NO OPEN BLOCK. `project()` answers a fresh averager with
`Projection(0.0, 0.0, 0.0, 0.0, False, None, 0.0)` - a ZERO projection, which
compares as comfortably under any ceiling. That is not "the quarter is fine",
it is "there is no quarter". Graded as a number it silently disarms the guard
on exactly the startup path where nothing else is watching.
* COVERAGE. A meter gap longer than one `max_hold_s` window leaves seconds that
`QuarterAverager` deliberately does not accumulate (p1.py: holding a dead
meter's last value forward is how a fabricated peak gets onto a permanent
record). Those seconds make the projection read LOW, which is the
under-guarding direction. The threshold is `max_hold_s` itself - the point
p1.py already stops trusting a held sample - not a new number.
* FROZEN. SAFETY-18's conjunction, evaluated through `freeze.looks_frozen()`
rather than re-written here. A frozen meter is the worst case of all: the
samples keep ARRIVING, so coverage stays perfect and the projection is
confident - and if the stuck value is an export, the projection is a
confident ZERO while the real meter runs at the rail. This is the case
SAFETY-07 exists to backstop, and it is the case a projection cannot see.
THE BLIND ACTION IS `floor_w = 0.0`: the battery may not IMPORT while the billed
quantity is unobservable. It is the minimal action that closes the hazard, and it
is one-directional - forbidding charging can only ever lower delivered offtake,
never raise it - so a false blindness costs a charge the site wanted and cannot
cost energy the pack did not want to give. Contrast the graded branch, which can
force discharge; blindness deliberately does not earn that, because there is by
definition no measurement to size it from.
------------------------------------------------------------------------------
WHY THIS ONE TRIPS WHERE SAFETY-18 ONLY NOTIFIES
SAFETY-18 shipped notification-first, and config.yaml says why in one line: "A
false trip parks the battery; a false alarm costs an alert." That reasoning is
right for a DETECTOR whose negative case had to be established on a rig run. It
does not carry to this guard, and the two committed captures say so:
* MEASURED FALSE-ENGAGEMENT RATE ON THE HEALTHY DAY: zero. 2026-08-23's worst
quarter is a few hundred watts against a 3500 W ceiling and nothing in its 95
quarters comes near engaging. test_peakguard.py re-measures that every run
from the committed capture rather than quoting it here.
* THE ACTION IS LEVEL-TRIGGERED AND UNLATCHED. It is re-derived from a fresh
projection every cycle and releases the instant the projection drops back
under the ceiling. There is no state to clear and no operator to wake. A
false engagement costs one cycle of discharge, not a parked battery - which
is precisely the cost SAFETY-18 could not bound and this one can.
* THE ACTION IS A LIMIT, NOT A SET. Every safety stop in 11.2's table outranks
it, so the worst thing a bug in this file can do is fail to relax a floor -
and a floor is bounded above by `achievable_w`, which main.py takes from the
same `max_w` and device rating the arbiter already enforces.
So it trips. It is also the only guard here whose trigger IS the billed quantity
itself, measured from parsed P1 telegrams, rather than a proxy for it.
------------------------------------------------------------------------------
SATURATION IS REPORTED, NOT ALARMED AS A BUG
10.3's doctrine - "treat an out-of-range computed setpoint as a bug signal, clamp
it AND alarm" - is SAFETY-03's, and it is right there: a loop computing 14 768 W
is a winding integrator, i.e. a defect. A peak floor above the inverter's rating
is NOT a defect. It is the ordinary physics of a 5900 W house load on a 5000 W
inverter, which is literally what 2026-08-20 15:45 was. So the floor is clamped
to `achievable_w` and the decision carries `saturated`, and main.py raises ONE
notification on that - not because the controller is broken, but because it is
the one case where the money WILL be spent and the controller has already done
everything it can. An engagement that is not saturated is an ordinary control
action and goes to the action log, like the clamp's engagements do.
------------------------------------------------------------------------------
NO NEW OPTION. CFG-03 STILL TRACKS THREE PLACES.
`peak_cap_w` already ships, is already validated `int(500,15000)`, and is already
read by `control.peak_at_risk()`. `max_hold_s` comes off the averager the caller
already owns; `achievable_w` off `max_w` and the device rating the arbiter
already enforces; the freeze predicate off `meter_freeze_s`. config.yaml and
validate.py are untouched by this ticket, exactly as TARIFF-01 left them, and
14's `capacity_peak_limit_kw` stays declined - a kW key beside a W key is the
mismatch trap 14's own warning box describes.
------------------------------------------------------------------------------
SAFETY-24: THE ENERGY BACKSTOP THIS GUARD SHIPPED WITHOUT
SAFETY-07 left this file saying, in its own words, that the graded branch can
FORCE DISCHARGE and that nothing below it consults the state of charge - "the
BMS is the sole energy backstop" (freeze.py). On the 2026-08-20 replay the guard
asks for 878 Wh in a day, and it keeps asking for as long as the meter says the
quarter is over the ceiling. That is the behaviour working as designed, and it
is unbounded from below by anything in the EMS.
⚠️ STATE THE RISK FAIRLY, because overstating it is how a real one gets ignored.
Relying on the BMS is not absurd: it is a genuine floor and it works. But it is
THE PACK's protection, not the OPERATOR's. A month of aggressive quarters can
leave the battery repeatedly parked at that floor with no EMS-side record of
why, and the first symptom - a pack that is never available when it is wanted -
is indistinguishable from the SoC drift MAINT-01 exists to correct.
So `decide()` now takes `soc` and `soc_floor`, and `discharge_permitted()` is
the one place the question is answered. Four things about the shape of it:
1. IT IS NOT A TRIP, AND IT MUST NOT BECOME ONE. SAFETY-07's claim is a LIMIT
and every stop in 11.2's table still outranks it. This floor is about
ENERGY - how far down the guard may spend the pack for money - not about
last-resort safety. The action when it binds is therefore to stop ASKING
for more discharge, not to command anything.
2. THE ACTION IS `floor_w = 0.0`, WHICH IS THE BLIND BRANCH'S ACTION FOR THE
BLIND BRANCH'S REASON. Zero is one-directional: it forbids CHARGING, which
can only ever lower delivered offtake, and it forbids nothing else. It
cannot pull the loop off a discharge it is already doing, because a `lo`
limit structurally cannot cap discharge - the property this module's whole
shape rests on, and one a NEGATIVE floor would destroy. Nothing here can
produce a negative floor: the only two values `floor_w` ever takes are 0.0
and `min(wanted, achievable)` with `wanted > 0` already established.
⚠️ AND IT DOES NOT BOUND THE LOOP. Self-consumption may still drain the
pack past this floor, exactly as SAFETY-18 recorded. That is not fixed
here and must not be claimed: this card bounds THE PEAK GUARD's OWN forced
discharge, which is the thing SAFETY-07 introduced. A loop-side SoC guard
is a different lever with a different right value.
3. IT IS GRADED ONLY WHERE IT IS NOT DEGENERATE. The SoC question is asked
AFTER `wanted > 0` is established - i.e. only where a forced discharge is
actually being asked for. Asked earlier it would grade a quantity that
governs nothing, and would report "floor-limited" on a quiet afternoon
where the guard was never going to discharge anyway.
4. AN UNREADABLE SoC IS NOT A FULL BATTERY. `soc=None` is floor-limited, not
permitted. This is `PeakReading`'s lesson in the one place it bites
hardest: a full battery is exactly the value that would disarm this floor,
so a failed read defaulting to one would silently restore the unbounded
discharge this card exists to remove. Both new arguments are REQUIRED
keyword-only and neither has a default, which is TEST-14 read as a rule
rather than as a story: a required field that gains a default is how three
separate files in this repo shipped a quiet fallback.
AND FAILING TO SHAVE MUST NOT BE SILENT. The whole point of the guard is the
euro figure, so a floor-limited cycle carries `floor_limited=True` and main.py
raises ONE notification on the edge, saying the quarter will be exceeded - the
same shape, and the same argument, as `saturated`. The two are mutually
exclusive by construction: a floor-limited decision asks for nothing, so it
cannot have run out of inverter.
------------------------------------------------------------------------------
CFG-03: TWO NEW OPTIONS, MIRRORED ONCE, AND WHY BOTH ARE NEEDED
SAFETY-07 added none and said so. This card cannot: an EMS-side floor is a
number only the site knows, and refusing one set below the pack's own cut-out
means knowing that cut-out too.
* `peak_soc_floor` - the EMS floor. ⚠️ NOT `maintenance_soc_floor`. That is
the maintenance cycle's DRAIN TARGET: the depth a monthly recalibration
deliberately goes to, once, under supervision. This is the depth a
money-saving discharge may never reach, on any day of the month. Sharing
one key would couple two features whose right values differ and would widen
CFG-03 rather than narrow it.
* `bms_soc_floor` - the pack's own cut-out (register 0x0560 on the reference
unit, 10 %). It is here so validate.py rule 8 can REFUSE a `peak_soc_floor`
at or below it. That configuration is CFG-01 rule (a): the operator states
a floor, the BMS enforces a different one, and the status page reports an
energy backstop that can never bind.
Both defaults are mirrored ONCE, in `PEAK_OPTION_DEFAULTS` below, which is
TARIFF-02's `PRICE_OPTION_DEFAULTS` pattern - imported by validate.py and by
main.py, pinned to config.yaml by test_peakguard.py rather than transcribed.
This module still reads no option itself: `decide()` takes both as ARGUMENTS,
makes no `.get()` call, and the mirror is a default table, not a read.
⚠️ A DECISION RECORDED, NOT A RENAME DONE. Now that a second SoC floor exists,
`maintenance_soc_floor` is the ambiguous name of the two - it reads like "the
SoC floor" and is not. Renaming it is an option migration on a live install for
a clarity gain, so the decision recorded here is: KEEP IT, and carry the
distinction in DOCS.md and in this comment. Revisit if a THIRD floor appears -
a loop-side guard would be the third - at which point the migration buys enough
to be worth its cost.
------------------------------------------------------------------------------
AND IT READS NO PER-PHASE QUANTITY, ON PURPOSE
The sum/max/net reduction is an open product question with the owner. TARIFF-01
put it behind `billed_w()` and defaulted to NET. Everything this module consumes
- `PeakReading.peak_w`, `Projection.projected_w` - is already reduced by that one
function, so whichever way the owner answers, this file does not change and does
not need re-testing. Nothing here indexes a phase.
"""
from dataclasses import dataclass
from .arbiter import P_PEAK, Claim
from .control import peak_at_risk
from .tariff import QUARTER_S
# Why the projection may not be graded. Reasons, not booleans: the status page
# and the action log both have to say WHICH, and "blind" alone is unactionable.
BLIND_NO_BLOCK = "no open quarter - the meter has produced no sample yet"
BLIND_COVERAGE = "meter gap"
BLIND_FROZEN = "SAFETY-18: the meter is answering without changing"
# Where the ceiling came from. Only the second is a relaxation, and only an
# OBSERVED reading may produce it.
CEILING_CAP = "peak_cap_w"
CEILING_MONTH_PEAK = "the month's banked peak"
# SAFETY-24. Why the forced discharge may not be sized. Reasons, not booleans,
# for BLIND_*'s reason: the status page, the action log and the operator
# notification all have to say WHICH, and "floor-limited" alone is unactionable.
FLOOR_SOC_UNREADABLE = ("the battery state of charge was not measured this "
"cycle, and an unreadable SoC is not a full battery")
FLOOR_AT_SOC_FLOOR = "state of charge is at or below the peak guard's SoC floor"
SOURCE = "peak"
# ⚠️ CFG-03. THE ONE IN-CODE MIRROR OF config.yaml FOR THIS CARD'S OPTIONS, and
# it is TARIFF-02's `PRICE_OPTION_DEFAULTS` pattern for TARIFF-02's reason: the
# add-on already carries option defaults in three places (config.yaml's
# `options:`, config.yaml's `schema:`, and main.py's literal `.get()`
# fallbacks), and a fourth is the open ticket. validate.py and main.py both
# import THIS dict rather than typing a literal, and test_peakguard.py pins it
# to config.yaml by READING the file - so a drift goes red instead of shipping.
#
# ⚠️ NOT A READ. `decide()` takes both values as arguments and this module makes
# no `.get()` call at all; the dict is a default table for its two callers.
PEAK_OPTION_DEFAULTS = {
# The EMS-side floor on SAFETY-07's forced discharge, in percent.
#
# 20 is DERIVED from the two numbers this repository already has, not
# chosen. The reference pack is 300 Ah at 51.2 V with DoD 90 - 13.8 kWh
# usable, the figure validate.py's rule 6 note already quotes - and the
# 2026-08-20 replay models 878 Wh of guard discharge in a day, which is
# 6.4 % of that. A floor 10 points above the pack's own 10 % cut-out
# therefore leaves the guard roughly 1.6 of its worst observed days of room
# before it stops, which is the smallest margin that lets the guard do its
# job on a bad day and still never reach the BMS.
"peak_soc_floor": 20,
# The pack's OWN cut-out, in percent: what the BMS enforces regardless of
# anything here. On the reference unit this is register 0x0560, read as 10,
# and it is consistent with DoD 90 on a full pack. It exists so validate.py
# rule 8 can refuse a `peak_soc_floor` at or below it - see that rule.
"bms_soc_floor": 10,
}
@dataclass(frozen=True)
class PeakDecision:
"""What the guard concluded this cycle, with everything it concluded it from.
Every field is derived from a measurement taken in the same cycle. There is
no `floor_w_or(default)` and there must not be one, for `PeakReading`'s
reason: an accessor that turns "not measured" into a number re-creates the
defect the type exists to remove.
"""
engaged: bool
floor_w: float | None # the `lo` the claim carries; None = no claim
ceiling_w: float
ceiling_from: str
blind: str | None = None
projected_w: float | None = None
excess_w: float = 0.0
wanted_w: float | None = None # the floor before `achievable_w` clamped it
saturated: bool = False
# SAFETY-24. True when a forced discharge WAS wanted and the EMS-side SoC
# floor refused to size it. Mutually exclusive with `saturated` by
# construction: this branch asks for nothing, so it cannot have run out of
# inverter. `soc_floor_reason` says WHICH of the two ways it bound.
floor_limited: bool = False
soc: float | None = None
soc_floor: float | None = None
soc_floor_reason: str | None = None
reason: str = ""
def explain(self) -> str:
if self.floor_limited:
return self.reason
if not self.engaged:
if self.projected_w is None:
return f"peak guard idle - {self.reason}"
return (f"peak guard idle - quarter projects "
f"{self.projected_w:.0f} W under a {self.ceiling_w:.0f} W "
f"ceiling ({self.ceiling_from})")
return self.reason
def quarter_ceiling_w(cap_w, reading) -> tuple[float, str]:
"""The average this quarter may not exceed, and where the number came from.
ONLY an OBSERVED reading may raise it. `NO_RECORD` (the store answered, this
month has nothing yet) and `NOT_OBSERVED` (the store did not answer) both
return the cap, which is the tighter answer. A failed read therefore makes
the guard stricter, never looser - see the module note.
"""
cap = float(cap_w)
if (reading is not None and getattr(reading, "is_observed", False)
and reading.peak_w is not None and float(reading.peak_w) > cap):
return float(reading.peak_w), CEILING_MONTH_PEAK
return cap, CEILING_CAP
def excess_w(projection, ceiling_w) -> float:
"""Watts that must come off the meter NOW to land the quarter on the ceiling.
The whole control law, and the only place it is written down:
excess = 900 * (projected - ceiling) / remaining
Positive means the quarter is projected over; zero or negative means it is
not. `remaining <= 0` is not a large number, it is an undefined one - a
quarter with no seconds left cannot be influenced - so it answers 0.0 rather
than dividing.
"""
remaining = QUARTER_S - float(projection.elapsed_s)
if remaining <= 0.0:
return 0.0
return QUARTER_S * (float(projection.projected_w) - float(ceiling_w)) / remaining
def uncovered_s(projection) -> float:
"""Seconds of the open block the meter did not actually cover.
Derived from what `Projection` already carries rather than re-measured:
`covered_frac` is `elapsed / elapsed_wall`, and `elapsed_s` IS that wall.
"""
return max(0.0, float(projection.elapsed_s)
* (1.0 - float(projection.covered_frac)))
def blindness(projection, max_hold_s, frozen: bool) -> str | None:
"""Why this projection may not be graded, or None if it may be.
Evaluated BEFORE the projection is compared to anything. Each of the three is
a way for the projected number to be confident and wrong, and the first - a
fresh averager's all-zero `Projection` - is confident, wrong AND under every
ceiling there is.
"""
if frozen:
return BLIND_FROZEN
if projection.block_start is None:
return BLIND_NO_BLOCK
gap = uncovered_s(projection)
if max_hold_s is not None and gap > float(max_hold_s):
return (f"{BLIND_COVERAGE}: {gap:.0f} s of this quarter uncovered, past "
f"the {float(max_hold_s):.0f} s the averager will hold a sample")
return None
def discharge_permitted(soc, soc_floor) -> str | None:
"""SAFETY-24. May the guard size a forced discharge, or why may it not?
Returns None when it may, and otherwise the REASON it may not - which the
decision, the status page, the action log and the operator notification all
carry, because "floor-limited" on its own tells nobody what to do.
THE ONE PLACE THIS QUESTION IS ANSWERED, and it is public for the reason
`freeze.looks_frozen()`, `watchdog.loop_is_following()` and
`shaping_suspended()` are: a safety predicate transcribed into its own check
is one copy plus a future disagreement about what the check means, and the
copy that drifts is always the grading one.
⚠️ `soc is None` IS NOT PERMISSION. A failed read is not an observation, and
a FULL BATTERY is exactly the value that would disarm this floor - so a
`float(soc or 100)` here would restore the unbounded discharge this card
removes, silently, on the one install whose SoC sensor is misnamed. There is
deliberately no default argument on either parameter.
⚠️ AND `soc_floor is None` IS NOT PERMISSION EITHER, for the same reason one
step further out: a caller that has no floor configured has not thereby been
granted an unbounded one.
Boundary: `<=`, not `<`. A pack sitting exactly ON the floor has already
spent everything this guard was allowed to spend; permitting one more cycle
there would make the floor mean "one cycle below the floor".
"""
if soc is None:
return FLOOR_SOC_UNREADABLE
if soc_floor is None:
return ("no peak guard SoC floor is configured, so there is nothing to "
"bound a forced discharge with")
if float(soc) <= float(soc_floor):
return FLOOR_AT_SOC_FLOOR
return None
def decide(projection, *, batt_w, cap_w, reading, achievable_w, soc, soc_floor,
max_hold_s=None, frozen: bool = False) -> PeakDecision:
"""The whole guard. Pure: no clock, no I/O, no state carried between calls.
`batt_w` MEASURED battery power, positive = discharge. None when the
sensor for it is missing - and then there is nothing to size a
floor from, so no floor is invented.
`achievable_w` the largest discharge the arbiter could actually grant
(`min(max_w, device max)`). The floor is clamped to it so this
claim can NEVER produce `lo > hi` in the resolver - a
contradiction there resolves to 0 W, which would park the
battery in the exact minute it should be discharging hardest.
`soc` SAFETY-24. MEASURED state of charge, percent. None when the
sensor for it did not answer - which is floor-limited, not
permission.
`soc_floor` SAFETY-24. The percent below which this guard may not spend
the pack for money. REQUIRED, and deliberately without a
default: a default here would be CFG-03's fourth defaults
site and TEST-14's quiet fallback in one line.
"""
ceiling, from_where = quarter_ceiling_w(cap_w, reading)
ceil_top = abs(float(achievable_w))
blind = blindness(projection, max_hold_s, frozen)
if blind is not None:
# 0.0, not a sized floor. There is no trustworthy measurement to size one
# from; the claim that remains is "do not add to the offtake you cannot
# see", which is one-directional and cannot cost pack energy.
return PeakDecision(
engaged=True, floor_w=0.0, ceiling_w=ceiling, ceiling_from=from_where,
blind=blind, projected_w=None,
reason=(f"PEAK GUARD BLIND ({blind}) - the battery may not import "
f"while the billed quarter cannot be measured; commanding no "
f"charging (floor 0 W)"))
projected = float(projection.projected_w)
if batt_w is None:
# main.py's P_SAFETY stop already owns a cycle with a missing input, at
# 0 W. Filing nothing is the honest answer: a floor sized from a
# measurement that is not there is a made-up number.
return PeakDecision(
engaged=False, floor_w=None, ceiling_w=ceiling,
ceiling_from=from_where, projected_w=projected,
reason="no battery measurement to size a floor from")
if QUARTER_S - float(projection.elapsed_s) <= 0.0:
# A block with no seconds left cannot be influenced. This is degenerate
# rather than urgent - `excess_w` answers 0.0 there and a floor equal to
# the battery's own current output would be a claim that says nothing.
return PeakDecision(
engaged=False, floor_w=None, ceiling_w=ceiling,
ceiling_from=from_where, projected_w=projected,
reason="the quarter has no seconds left to influence")
ex = excess_w(projection, ceiling)
wanted = float(batt_w) + ex
if wanted <= 0.0:
# ⚠️ THE TEST IS ON THE FLOOR, NOT ON `excess > 0`, AND THE REPLAY IS WHY.
# `excess` is the watts that must come off the METER, so it goes to zero
# as soon as the battery is already removing them - and a guard that
# disengaged there handed the actuator straight back, the offtake sprang
# back to the raw load, and the next cycle engaged again. On 2026-08-20's
# billed quarter that bang-bang cycled 17 times in 900 s and left the
# quarter 49 W over the limit it had just spent a quarter of an hour
# holding. §10.2: "an EMS that oscillates is worse than no EMS, both for
# hardware wear and - per §9.1 - for the peak it is trying to avoid."
#
# `batt_w + excess` is the offtake that would exist with the battery at
# zero, minus the level the quarter can afford. Testing THAT is the same
# arithmetic read as a level rather than as a switch: the claim is filed
# for as long as a positive floor is needed and decays continuously to
# nothing as the load falls, so there is still no hysteresis constant -
# the quantity being thresholded is simply the right one.
return PeakDecision(
engaged=False, floor_w=None, ceiling_w=ceiling,
ceiling_from=from_where, projected_w=projected, excess_w=ex,
wanted_w=wanted,
reason="the load is under the level this quarter can afford")
# SAFETY-24 - AND ONLY HERE. `wanted > 0` is now established, so a forced
# discharge really is being asked for and the SoC is the quantity that
# decides whether it may be sized. Asked one line earlier it would grade a
# number that governs nothing on every quiet cycle of the day, and would
# report "floor-limited" over a guard that was never going to discharge.
denied = discharge_permitted(soc, soc_floor)
if denied is not None:
# 0.0, not `wanted`, and not a negative number. Zero forbids CHARGING -
# one-directional, cannot cost pack energy - and forbids nothing else;
# a `lo` limit structurally cannot cap the discharge the loop is already
# doing. See the module note: this bounds THE GUARD's forced discharge,
# not the grid-following loop, which still has no SoC guard.
return PeakDecision(
engaged=True, floor_w=0.0, ceiling_w=ceiling,
ceiling_from=from_where, projected_w=projected, excess_w=ex,
wanted_w=wanted, floor_limited=True, soc=soc, soc_floor=soc_floor,
soc_floor_reason=denied,
reason=(
f"PEAK GUARD FLOOR-LIMITED: the quarter projects "
f"{projected:.0f} W against a {ceiling:.0f} W ceiling "
f"({from_where}) and {wanted:+.0f} W of discharge would hold "
f"it, but {denied}"
+ ("" if soc is None else f" ({float(soc):.0f} %")
+ ("" if soc is None or soc_floor is None
else f" vs {float(soc_floor):.0f} %")
+ ("" if soc is None else ")")
+ " - commanding no forced discharge, so THIS QUARTER WILL GO "
"OVER THE CEILING. Charging stays blocked (floor 0 W)"))
floor = min(wanted, ceil_top)
saturated = floor < wanted
return PeakDecision(
engaged=True, floor_w=floor, ceiling_w=ceiling, ceiling_from=from_where,
blind=None, projected_w=projected, excess_w=ex, wanted_w=wanted,
saturated=saturated, soc=soc, soc_floor=soc_floor,
reason=(
f"PEAK GUARD: quarter projects {projected:.0f} W against a "
f"{ceiling:.0f} W ceiling ({from_where}) with "
f"{QUARTER_S - projection.elapsed_s:.0f} s left, so {ex:.0f} W must "
f"come off the meter; battery measured {float(batt_w):+.0f} W, "
f"floor {floor:+.0f} W"
+ (f" - SATURATED, {wanted:+.0f} W was needed and the inverter can "
f"only give {ceil_top:.0f} W" if saturated else "")))
def shaping_suspended(decision, peak_forecast_w, cap_w) -> bool:
"""Must the maintenance charge-only shaping stand down this cycle?
THE ONE PLACE THIS CONJUNCTION IS WRITTEN, and it is public for the reason
`watchdog.loop_is_following()` and `freeze.looks_frozen()` are: the check
that grades it has to assert on THE SHIPPED EXPRESSION. A safety predicate
transcribed into its own test is one copy plus a future disagreement about
what the check means, and the copy that drifts is always the grading one.
It was inline in `main.py` and nothing could reach it; a reviewer mutated it
back to its historical broken form and all twelve suites stayed green.
⚠️ WHY SUPPRESSION AND NOT PRIORITY. The shaping is a LIMIT (`hi <= 0`) and
the peak guard's claim is a LIMIT (`lo >= 0`). arbiter.py clause 2 applies
EVERY limit at or above the winner's priority and takes the MOST RESTRICTIVE,
with no notion of one limit outranking another - which is precisely what
makes the safety bounds unconditional and must not be changed. Two crossing
limits are therefore a CONTRADICTION, which resolves to 0 W: the battery
PARKED, in the one quarter of the month that sets the bill. P_PEAK 50 >
P_MAINT_SHAPE 40 does not save it and never could.
⚠️ AND THE FIRST TERM IS NOT ENOUGH ON ITS OWN. `peak_forecast_entity` is
OPTIONAL and is empty on a default install, so `peak_forecast_w` is None
there and `peak_at_risk()` answers False forever. Before this card the
suspension therefore could not fire on the very site that needed it. The
second term is the MEASURED guard, which is always present wherever P1 is.
Deleting either term - or re-gating the whole thing on the forecast - parks
a default install at 0 W, silently, in the billed quarter. test_peakguard.py
part 7 drives the real cycle with each of those mutations in place.
"""
if peak_at_risk(peak_forecast_w, float(cap_w)):
return True
return decision is not None and decision.engaged
def claim(decision: PeakDecision) -> Claim | None:
"""The decision as one arbiter claim, or None when the guard is idle.
A LIMIT with only `lo` set. `hi` stays `+inf`, which is 9.1's "discharge and
export are never capped by the peak guard" expressed as a fact about the
object rather than as a rule somebody has to remember.
"""
if not decision.engaged or decision.floor_w is None:
return None
# Three reasons, not two. A `lo = 0 W` claim reaches the arbiter from THREE
# different states now - blind, floor-limited, and a genuine floor that
# happens to be zero - and the arbiter's explain() is what an operator reads
# when the battery stops charging. Spelling them the same way would make the
# SAFETY-24 case indistinguishable from the SAFETY-18 one on the one line
# that gets read.
return Claim.limit(SOURCE, P_PEAK, lo=float(decision.floor_w),
reason=("blind" if decision.blind
else "SoC floor" if decision.floor_limited
else "quarter peak"))
File diff suppressed because it is too large Load Diff
+465
View File
@@ -0,0 +1,465 @@
"""FCST-03 - the hours the feed has not published yet, and what may not rest on them.
WHERE THIS LIVES. specs.md §2.2 puts price ingest under `ems/ingest/prices/`.
That layout does not exist and is not being built; the product owner's decision
of 2026-08-30 is that everything ships in `goodwe_controller/app/`. The
BEHAVIOUR §8.3 describes is unchanged and still authoritative - only the path
moved. Read `ems/...` as `app/...`.
§8.3, in full, because every line below is one clause of it:
"Day-ahead prices arrive once daily. Until they do, unknown hours use
yesterday's profile and are flagged `provisional: true` - and NO
IRREVERSIBLE ACTION MAY BE TAKEN ON A PROVISIONAL PRICE. Real-time /
imbalance products are treated as high-volatility: arbitrage thresholds
widen."
Three separate things, and this module owns the first two. The third - the
widening arithmetic - lives in price.py beside the product enum it is derived
from (`Uncertainty`, `widen_factor`, `required_spread`), and is spent by
`static_tariff.TariffPlan.arbitrage()`, because that is where the verdict this
repo already refuses to hand out casually is produced. FCST-03 added a refusal
to that existing ladder rather than a second mechanism beside it.
THE DEFECT THIS MODULE IS SHAPED BY: A COPY THAT KEEPS THE WRONG FLAG
=====================================================================
A point filled from yesterday and a point published this morning are the SAME
SHAPE. Three floats and a bool; only the bool tells them apart. And the natural
way to write a carry-forward fill is:
PricePoint(start=t, eur_per_kwh=y.eur_per_kwh, provisional=y.provisional)
^^^^^^^^^^^^^
which copies yesterday's `False` onto today's guess and produces exactly the
thing the ticket exists to prevent: STALE DATA PRESENTED AS CURRENT. It reads
fine, it type-checks, it round-trips through §9.3's JSON, and every price on
the status page looks like a price. This is the meter-side SAFETY-18 defect one
layer up: the reading arrives on time, looks fresh, and is wrong - and a
freshness check that only asks "did something arrive" cannot see it.
So the flag is not passed through anywhere in this file:
* A CARRIED POINT IS BUILT BY `_carried()`, WHICH HAS NO PARAMETER FOR THE
FLAG. `provisional=True` is a literal in one place. There is no argument a
caller can pass, no keyword to forget, and no variable that could hold
yesterday's value at the moment the point is constructed.
* A PUBLISHED POINT IS NEVER COPIED AT ALL. It is passed through BY IDENTITY -
the object the provider produced is the object in the output series - so
there is no copy that could lose or gain a flag. `filled.points[i] is
fetched.points[j]` is a checkable property, and test_price_forecast.py
checks it.
* `_audit()` RE-DERIVES the expected flag for every output point from the set
of hours the feed actually published, and RAISES if any point disagrees.
It is a post-condition, not a filter: it never fixes a flag, because a
module that quietly corrects its own provenance is one whose provenance
cannot be trusted.
* and test_price_forecast.py walks THIS FILE'S AST and fails if any
`PricePoint(...)` in it passes anything but a literal for `provisional`, so
the pass-through idiom cannot come back in a later edit and pass review by
looking ordinary.
⚠️ "PUBLISHED" DOES NOT MEAN "FIRM". `eneco_eddy` is an imbalance product whose
current-quarter print is genuinely provisional and says so per point (§9.2,
TARIFF-02). Such a point is published AND provisional, and the audit's rule for
a published point is "the flag is whatever the feed said", never "False". A
provenance check that forced published points to False would be a bug that
looks like a guarantee.
THE FILL, AND WHY IT KEYS ON THE WALL CLOCK
===========================================
For every whole hour in the requested horizon that the feed has not published,
take yesterday's value for THE SAME BRUSSELS WALL-CLOCK HOUR - not `t - 86400
s`. On the two days a year that matter those are different instants, and the
difference is not academic: a 23-hour day has no 02:00 at all and a 25-hour day
has two, so a fill that subtracts a fixed 24 h shifts every carried hour by one
across a transition and misprices the whole day in a way nothing reports.
On the 25-hour day the two 02:00 hours - `02:00+02:00` and `02:00+01:00` - both
key to local hour 2 and are BOTH filled from yesterday's single 02:00 value.
That is deliberate: they are both guesses, they are both flagged, and nothing
irreversible may rest on either. If yesterday itself was a 25-hour day, the
first of its two 02:00 points is the one used, deterministically.
An hour with no published price AND no yesterday value is LEFT OUT of the
series. It is not filled with a neighbour, a mean, or a zero. §11.4 applies to
that hour - `_Series.at()` raises `Absence` for it - and the caller that wanted
a tariff for the whole horizon falls back to the static profile (TARIFF-03),
which is the row the spec actually writes for "no price data". If NOTHING can
be produced, the result is `PriceRead.no_data(reason)` and not an empty series,
because an empty series is not constructible on purpose (price.py trap 3).
AC2: THE IRREVERSIBLE-ACTION GATE
=================================
"The distinction should be an explicit property of the action, not a judgement
call at each call site" - so an irreversible action is a value, `Irreversible`,
carrying the number of hours it COMMITS. `price_permits()` looks at exactly
those hours and answers with a `Gate` variant that carries the evidence on the
permitting arm and a reason on the blocking one, the `_Outcome` shape price.py
borrowed from `sim/riglock.py`.
The shipped example is the maintenance drain (§12): starting one commits hours
of forced discharge that cannot be undone if the price turns out to have been
yesterday's. Its commitment window is DERIVED from `maintenance.DRAIN_TIMEOUT_MIN`
rather than typed here, so the gate and the phase machine cannot drift apart.
⚠️ WHAT DOES NOT GO THROUGH THIS GATE, AND WHY THAT MATTERS MORE THAN WHAT
DOES. The maintenance cycle is a CALENDAR obligation (§12.2: a cycle that
silently never fires is indistinguishable from one that is not due yet), and a
site with `price_provider: off` - the shipped default - has no price series at
all. Routing every maintenance start through a gate that blocks on `no_data`
would stop the monthly cycle on every default install, permanently and
silently: the exact failure §12 opens by naming. So `begin_phase()` is the
entry point for a start that is being JUSTIFIED BY A PRICE, main.py's existing
calendar path is untouched by this ticket, and OPT-04/MAINT-05 own the decision
of which one a site runs. Nothing here disables anything that runs today.
Reversible actions - ordinary self-consumption modulation, the loop that
actually holds the grid at zero - are not gated at all and cannot be: §6 says
absent machinery must not disable the loop.
"""
import logging
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from enum import Enum
from .maintenance import CHARGE_TIMEOUT_MIN, DRAIN_TIMEOUT_MIN
from .price import PricePoint, PriceRead, _Outcome, brussels
_LOG = logging.getLogger("goodwe.price.forecast")
class Provenance(Enum):
"""Where a point in a filled series came from. INTERNAL - §9.3's wire format
carries the boolean and nothing else, so this never leaves the module.
It exists so `_audit()` has a vocabulary to state its post-condition in, and
so the two cases have names in the log rather than being "the ones with the
flag" and "the others".
"""
PUBLISHED = "published" # the provider printed this hour. Flag: the
# feed's own - an imbalance print may be
# published AND provisional (§9.2).
CARRIED = "carried" # yesterday's same-wall-clock hour (§8.3).
# Flag: ALWAYS True, by construction.
# --------------------------------------------------------------------------- #
# The fill
# --------------------------------------------------------------------------- #
def _utc_hour(instant: datetime) -> datetime:
u = instant.astimezone(timezone.utc)
return u.replace(minute=0, second=0, microsecond=0)
def _carried(start_utc: datetime, eur_per_kwh: float) -> PricePoint:
"""A point filled from yesterday's profile. §8.3: ALWAYS provisional.
⚠️ THERE IS NO `provisional` PARAMETER, AND THAT IS THE MECHANISM. This is
the only constructor of a carried point, the flag is a literal, and no
caller - including a future one editing in a hurry - has anything to pass,
default, or copy from the point being carried.
"""
return PricePoint(start=brussels(start_utc),
eur_per_kwh=round(float(eur_per_kwh), 6),
provisional=True)
def _audit(points, published) -> None:
"""Post-condition: every flag matches the point's TRUE provenance (AC1).
`published` maps a UTC hour to the point the provider produced for it.
⚠️ RAISES, NEVER REPAIRS. A module that silently corrects its own
provenance is one whose provenance means nothing; the only safe response to
"this point's flag disagrees with where it came from" is to refuse to emit
the series at all.
"""
seen = set()
for p in points:
u = _utc_hour(p.start)
if u in seen:
raise ValueError(f"the filled series repeats {u.isoformat()}; a "
f"duplicated hour is how a DST day gets mispriced")
seen.add(u)
src = published.get(u)
prov = Provenance.CARRIED if src is None else Provenance.PUBLISHED
if src is None:
if p.provisional is not True:
raise ValueError(
f"{u.isoformat()} is {prov.value} and is flagged "
f"provisional={p.provisional!r}. §8.3: an hour filled from "
f"yesterday's profile is provisional whether or not the "
f"number turns out to be right - the flag is a claim about "
f"PROVENANCE, not about accuracy.")
else:
if p is not src:
raise ValueError(
f"{u.isoformat()} is {prov.value} but the series carries a "
f"COPY of that point rather than the point itself. A copy "
f"is where the provisional flag gets lost; published points "
f"pass through by identity.")
def fill_from_yesterday(read: PriceRead, *, yesterday: PriceRead,
since: datetime, until: datetime,
stamp: datetime) -> PriceRead:
"""§8.3: fill the hours the feed has not published with yesterday's profile.
Every argument is required and keyword-only past the first. In particular
`yesterday` has NO DEFAULT: a fill with no yesterday to fill from is a
different situation from a fill that found nothing useful in one, and a
default `None` would make the two arrive looking identical.
Returns a `PriceRead`:
* a series, whose points are the provider's own objects for the hours it
published and `_carried()` points for the rest, ordered by instant;
* `no_data` when neither source can produce a single hour - never an
empty series, which is not constructible (price.py trap 3).
⚠️ IDEMPOTENT AND RE-RUNNABLE, WHICH IS THE §5.4 LATE-PUBLICATION CONTRACT.
Day-ahead publication at 14-16 h CET can be late or fail, so the fetcher
retries and this runs again against the fresher read. Published hours always
win over carried ones, so a re-run REPLACES a carried point with the real
print and the hour becomes firm - the flag and the number move together
because the whole series is rebuilt rather than patched in place.
"""
for name, value in (("since", since), ("until", until), ("stamp", stamp)):
if value.tzinfo is None or value.utcoffset() is None:
raise ValueError(f"{name} must be tz-aware; a naive instant is "
f"ambiguous for one hour every October")
published = read.match(
series=lambda r: {_utc_hour(p.start): p for p in r.points},
no_data=lambda _r: {})
# Yesterday's profile, keyed by BRUSSELS WALL-CLOCK HOUR. First point wins
# when yesterday itself was a 25-hour day and printed local hour 2 twice.
by_local_hour = {}
yesterday.match(
series=lambda r: [by_local_hour.setdefault(brussels(p.start).hour, p)
for p in r.points],
no_data=lambda _r: [])
out = []
u = _utc_hour(since)
end = until.astimezone(timezone.utc)
while u < end:
if u in published:
out.append(published[u]) # BY IDENTITY. No copy.
else:
src = by_local_hour.get(brussels(u).hour)
if src is not None:
out.append(_carried(u, src.eur_per_kwh))
u += timedelta(hours=1)
if not out:
reason = read.match(
series=lambda r: (f"the {len(r.points)} published hours all fall "
f"outside {since.isoformat()}..{until.isoformat()}"),
no_data=lambda r: r.reason)
return PriceRead.no_data(
f"§8.3 carry-forward produced no hours: {reason}, and yesterday's "
f"profile has nothing to fill them with either. §11.4 applies: "
f"static profile, plan provisional, no arbitrage")
_audit(out, published)
carried = sum(1 for p in out if _utc_hour(p.start) not in published)
if carried:
_LOG.info("§8.3: %d of %d hours carried forward from yesterday's "
"profile and flagged provisional; %d published",
carried, len(out), len(out) - carried)
product = read.match(series=lambda r: r.product,
no_data=lambda _r: yesterday.product)
provider = read.match(series=lambda r: r.provider,
no_data=lambda r: yesterday.provider)
return PriceRead.series(out, product, provider, stamp)
# --------------------------------------------------------------------------- #
# AC2: irreversibility as a property of the ACTION, and the gate that reads it
# --------------------------------------------------------------------------- #
@dataclass(frozen=True)
class Irreversible:
"""An action that cannot be undone once started, and how far it commits.
`commits_h` is the horizon the action ties up - the hours whose prices must
therefore be firm before it may be justified by a price at all.
"""
name: str # the §10.2 reason code
commits_h: float
def __post_init__(self):
if not self.name or not isinstance(self.name, str):
raise ValueError("an irreversible action needs a reason code (§10.2)")
if isinstance(self.commits_h, bool) or \
not isinstance(self.commits_h, (int, float)) or \
not self.commits_h > 0:
raise ValueError(
f"commits_h must be a positive number of hours, not "
f"{self.commits_h!r}. An action that commits nothing is a "
f"reversible one and does not belong here.")
# ⚠️ DERIVED FROM THE PHASE MACHINE'S OWN BOUNDS, NOT TYPED HERE. maintenance.py
# already states how long each phase may run before it is a fault; a second copy
# of those numbers here would be a place for them to drift apart, and the drift
# would show up as a gate that checked the wrong hours.
MAINTENANCE_DRAIN = Irreversible("MAINTENANCE_DRAIN", DRAIN_TIMEOUT_MIN / 60.0)
MAINTENANCE_CHARGE = Irreversible("MAINTENANCE_CHARGE", CHARGE_TIMEOUT_MIN / 60.0)
class Gate(_Outcome):
"""May an irreversible action proceed on the price evidence available?
* `permitted` - carries `.action` and `.hours`, the firm points that were
actually checked. The evidence exists only on this arm.
* `blocked` - carries `.reason`. Nothing else; `.hours` raises `Absence`.
No `__bool__`, both `match()` arms required and keyword-only, for the reason
price.py's `_Outcome` gives: `if gate:` must not look like a guard while
being none.
"""
def match(self, *, permitted, blocked):
return {"permitted": permitted, "blocked": blocked}[self.kind](self)
@staticmethod
def permit(action, hours):
return _Permitted(action, hours)
@staticmethod
def block(reason):
if not reason or not isinstance(reason, str):
raise ValueError(
"Gate.block() takes a reason a human can act on. §6 requires "
"every decision to explain itself, and 'the maintenance cycle "
"did not start' with no reason is indistinguishable from a bug.")
return _Blocked(reason)
class _Permitted(Gate):
kind = "permitted"
def __init__(self, action, hours):
if not isinstance(action, Irreversible):
raise TypeError(f"action must be an Irreversible, not {action!r}")
pts = tuple(hours)
if not pts:
raise ValueError(
"a permission granted over ZERO checked hours is a permission "
"granted on no evidence - the shape this whole file exists to "
"refuse. Block instead.")
if any(p.provisional for p in pts):
raise ValueError(
"a permitted gate cannot carry a provisional hour: §8.3 is the "
"one rule this type has.")
self.action = action
self.hours = pts
self.words = (f"{action.name} permitted on {len(pts)} firm hours "
f"({pts[0].start.isoformat()} ..)")
class _Blocked(Gate):
kind = "blocked"
def __init__(self, reason):
self.reason = reason
self.words = f"blocked - {reason}"
def committed_hours(now: datetime, action: Irreversible) -> tuple:
"""The whole hours `action` started at `now` would run through.
Split out because the gate and its caller must agree on what "the hours it
commits" means, and because a window that quietly excluded the last hour
would be a gate that permits an action on an hour it never looked at.
"""
if not isinstance(action, Irreversible):
raise TypeError(f"action must be an Irreversible, not {action!r}")
if now.tzinfo is None or now.utcoffset() is None:
raise ValueError("now must be tz-aware")
out = []
u = _utc_hour(now)
end = now.astimezone(timezone.utc) + timedelta(hours=action.commits_h)
while u < end:
out.append(u)
u += timedelta(hours=1)
return tuple(out)
def price_permits(read: PriceRead, now: datetime,
action: Irreversible) -> Gate:
"""§8.3's rule, as a value: may this irreversible action rest on this price?
Blocks when the read observed nothing (§11.4 - an action justified by a
price nobody has is an action justified by nothing), when any hour it
commits has no price at all, and when any of those hours is provisional.
Names the offending hour, so an operator reading the log knows which one.
"""
hours = committed_hours(now, action)
return read.match(
series=lambda r: _walk(r, hours, action),
no_data=lambda r: Gate.block(
f"{action.name} is gated on a price and there is no price data: "
f"{r.reason}. §11.4 - and a missing price is not a cheap one"))
def _walk(series, hours, action) -> Gate:
pts = []
for u in hours:
if not series.covers(u):
return Gate.block(
f"{action.name} commits {action.commits_h:g} h from "
f"{hours[0].isoformat()} and no price covers "
f"{u.isoformat()}. An hour outside the series is an hour with "
f"no price (§11.4 for that hour), not a free one")
p = series.at(u)
if p.provisional:
return Gate.block(
f"{action.name} commits {action.commits_h:g} h from "
f"{hours[0].isoformat()} and {u.isoformat()} is PROVISIONAL - "
f"carried from yesterday's profile or standing in for a dead "
f"feed. §8.3: no irreversible action may be taken on a "
f"provisional price. It becomes available when the day-ahead "
f"print lands (~14-16 h CET, §5.4)")
pts.append(p)
return Gate.permit(action, pts)
def begin_phase(maint, now: datetime, read: PriceRead,
action: Irreversible) -> Gate:
"""Start a maintenance phase BECAUSE OF A PRICE, or refuse and say why.
The AC2 example made real: `maint.force_start(now)` - the call that moves
the phase machine out of `idle` and commits the pack to hours of forced
discharge - happens on the permitting arm and cannot happen on the other
one, because the blocking arm has no code path to it.
⚠️ READ THE MODULE DOCSTRING'S LAST WARNING BEFORE WIRING THIS IN. This is
the price-justified entry point, not the calendar one; a site with no price
provider must keep starting its monthly cycle the way it does today, or
§12's opening sentence comes true - a cycle that silently never fires.
"""
gate = price_permits(read, now, action)
return gate.match(permitted=lambda g: _go(maint, now, g),
blocked=lambda g: _no(g))
def _go(maint, now, gate) -> Gate:
maint.force_start(now)
_LOG.info("%s: %s", gate.action.name, gate.words)
return gate
def _no(gate) -> Gate:
_LOG.warning("irreversible action refused - %s", gate.reason)
return gate
File diff suppressed because it is too large Load Diff
+702
View File
@@ -0,0 +1,702 @@
"""TARIFF-01 - the monthly capacity-tariff peak, and the record that outlives it.
WHAT THIS IS, AND WHAT IT DELIBERATELY IS NOT. This module is an INSTRUMENT. It
measures the number the Belgian capaciteitstarief bills on - the highest
15-minute average offtake of the calendar month (Fluvius / Resa / Ores) - and it
writes that number down somewhere a restart cannot reach. It makes no claim,
applies no clamp, and commands nothing. The guard that will read it is SAFETY-07,
and it belongs in `arbiter.py` as a `P_PEAK` claim when it is built. Keeping the
measurement and the actuation apart is the whole reason this file is small.
WHAT WAS ALREADY BUILT, AND WHY THIS CARD IS NOT 8 SP OF NEW ARITHMETIC.
`p1.QuarterAverager` already computes AC 1 in full: clock-aligned 900 s blocks,
time-weighted over irregular telegram spacing, offtake-only, per-phase, with a
`max_hold_s` gap rule. It was unwired and unpersisted - `P1Ingest.blocks` is a
96-entry list in memory that a restart erases. So this file is the persistence
and the monthly reduction, not a second accumulator. Adding one would be adding
a second source of truth for the exact quantity the money depends on.
BLOCKS, NOT A SLIDING WINDOW. The Description says "rolling"; AC 1 says
"15-min block average". They are different numbers and the utility bills the
second one. §9.1 calls the billed unit the "highest 15-minute average offtake",
and §9.1's own worked case - "one 4998 W quarter moved a monthly peak from
4832 -> 5013 W" - is stated per QUARTER, an object a sliding window does not
have. "Rolling" here means CONTINUOUSLY UPDATED (the open block's partial
average is readable at every sample), not a trailing-15-minutes window. A
trailing window would report a peak the meter never bills, and it reports it
HIGHER than the real one: a load burst straddling :15 fills a trailing window
completely while contributing to two block averages at partial weight. Guarding
that would spend battery defending an interval nobody charges for.
DELIVERED POWER, NOT INTENDED POWER. SAFETY-20 has an open question about
whether guards should grade `Decision.target` (pre-clamp intent) or `standing`
(what actually reached the wire). For a capacity tariff there is no question to
ask: the utility's meter integrates the current that flowed. This module never
sees a setpoint at all - it consumes `P1Sample`, which is the meter's own
reading - and that is on purpose. A peak computed from intent would be wrong in
the expensive direction (a clamped 9 kW request that delivered 5 kW would
record 9 kW, and every consumer would shave against a peak that never existed).
AN UNREADABLE STORE IS NOT A ZERO PEAK. This is the defect this repo spent a
day removing from `sim/riglock.py`, where `holder()` was rewritten to answer
`observed / not_observed / gone` instead of a partial dict every caller
`.get()`s a default out of. The same trap is here and it is worse, because the
degenerate value is not merely wrong, it is DISARMING: a peak read as 0 W gives
SAFETY-07 the maximum possible headroom and silently switches the guard off at
the moment its own storage is broken. So `peak()` returns a `PeakReading` whose
`state` is one of OBSERVED / NO_RECORD / NOT_OBSERVED, `peak_w` is `None`
unless the state is OBSERVED, and there is no `.get(..., 0)` shaped accessor
anywhere on it. NO_RECORD ("this month has no completed block yet", a normal
state on the 1st) and NOT_OBSERVED ("the database did not answer") are
different types of thing and are never spelled the same way.
CONFIG: NONE ADDED, ON PURPOSE. CFG-03 is open because the option defaults
already live in three places held apart by a tripwire in `test_validate.py`. The
soft limit this measurement is graded against already exists as `peak_cap_w`
(config.yaml, default 3500), already validated, already read by
`control.peak_at_risk()`. §14 names a `tariffs.capacity_peak_limit_kw: 2.5` - in
KILOwatts, against watts everywhere else - and adding it would have made a fourth
defaults site AND imported the unit mismatch §14's own warning box says must be
caught at startup. So this module takes its limit as a plain constructor
argument in WATTS and config.yaml is untouched by this ticket.
THE DATABASE. SQLite in /data, the add-on's only durable volume, alongside
`store.py`'s state.json. Two tiers from §15, and their retention is the point:
quarter_blocks one row per completed 900 s block INDEFINITE
monthly_peak one row per calendar month PERMANENT
Neither is ever pruned here, and no sweep in this file can prune them - there is
no DELETE statement against either table anywhere in this module. §15's asymmetry
is why: raw samples vanish after 7 days, so a 15-minute aggregate that was not
written AS IT HAPPENED can never be recomputed. `monthly_peak` is maintained as a
derived record - upserted on every block close, and `recompute_month()` re-derives
it from `quarter_blocks` so the two can be PROVEN not to have diverged rather
than assumed not to have.
RESTART MID-BLOCK. The open block's accumulator is memory. A restart at minute 12
that dropped it would under-read the rest of the quarter, and SAFETY-07's clamp
would release exactly when it was most needed. So the accumulator is checkpointed
(`QuarterAverager.snapshot()`) into a single row, at most `CHECKPOINT_S` stale,
and restored on startup. Three cases, all of them exercised in `test_tariff.py`:
* the checkpoint's block is the block containing `now` -> restore it verbatim,
including the held sample's ORIGINAL timestamp, so the first sample after the
restart integrates the gap under the same `max_hold_s` rule a running process
would have applied. A restart longer than `max_hold_s` therefore leaves
uncovered seconds, and a closed block is always divided by the full 900 s, so
the gap DRAGS THE BILLED AVERAGE DOWN instead of being averaged away;
* the checkpoint's block ended while we were down -> that block really happened
and its measured part is real, so it is closed at `acc / 900` and recorded,
then a fresh block is opened. Discarding it would lose measured energy;
* no checkpoint, or one that cannot be read -> start clean and SAY SO. The
return value of `restore()` names which of the three happened; it is never
silently the first one.
Double counting is answered structurally rather than by care: `quarter_start` is
the PRIMARY KEY, and the recorded average may only ever be RAISED by a later
write (`ON CONFLICT ... WHERE excluded.offtake_avg_w > ...`). Replaying the same
block twice is therefore a no-op, and a restart that reprocesses a block with
LESS coverage than the first pass cannot lower a figure already banked.
MONTH BOUNDARY. Calendar months are LOCAL (Europe/Brussels); blocks are stored in
UTC. In August those differ by two hours, so a block starting 2026-08-31 22:15
UTC belongs to SEPTEMBER and putting it in August would move a month's bill.
Block alignment itself needs no timezone - every Belgian offset is a whole number
of hours, so the 900 s UTC grid lands on :00/:15/:30/:45 local on both sides of a
DST transition, which is the argument `QuarterAverager` already carries.
"""
import json
import logging
import os
import sqlite3
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone, tzinfo
_LOG = logging.getLogger("goodwe.tariff")
QUARTER_S = 900
# How stale the partial-block checkpoint may be. This is the ONLY thing a crash
# can lose from an open block, and the loss is bounded and one-directional: the
# lost seconds were already integrated into `_acc`, so dropping them makes the
# restored block read LOW, never high. At 15 s against a 900 s block the
# worst-case under-read of one quarter's average is 15/900 = 1.7 % of that
# quarter's own offtake - see test_tariff.py, which computes that bound from
# these two constants rather than quoting it.
CHECKPOINT_S = 15.0
# -- reading states. Three, and never fewer. -------------------------------- #
OBSERVED = "observed" # the store answered and there is a peak
NO_RECORD = "no_record" # the store answered and this month has none yet
NOT_OBSERVED = "not_observed" # the store did not answer. NOT a zero peak.
# --------------------------------------------------------------------------- #
# local time, without a hard dependency on the tz database
# --------------------------------------------------------------------------- #
class _EUTimezone(tzinfo):
"""Europe/Brussels by the EU harmonised rule, for hosts with no tzdata.
⚠️ THIS IS A FALLBACK AND IT IS NOT SILENT. `zoneinfo` is tried first and is
what runs on the add-on's container. The fallback exists because the
alternative - quietly using UTC when tzdata is missing - misfiles every block
in the last two hours of a month into the wrong month's bill, and does it
without a symptom. Encoding the rule (CET, CEST from 01:00 UTC on the last
Sunday of March to 01:00 UTC on the last Sunday of October) is 20 lines and
is checked against zoneinfo, date by date, wherever zoneinfo is available.
"""
# The rule, in both frames. `fromutc` is overridden rather than inherited
# because the base implementation calls back into `utcoffset` with a
# datetime whose tzinfo is self - and the first version of this class
# answered that by calling `astimezone`, which calls `utcoffset`, which
# recursed until the stack ran out. A tzinfo must answer from the value it
# is handed, never by converting it.
@staticmethod
def _last_sunday(year: int, month: int, hour: int) -> datetime:
d = datetime(year, month + 1, 1) - timedelta(days=1)
return (d - timedelta(days=(d.weekday() + 1) % 7)).replace(hour=hour)
def _dst_local(self, naive: datetime) -> bool:
"""From a LOCAL wall clock: 02:00 CET last Sunday March -> 03:00 CEST
last Sunday October. The one ambiguous hour each autumn resolves to
standard time, which cannot move a month boundary."""
y = naive.year
return self._last_sunday(y, 3, 2) <= naive < self._last_sunday(y, 10, 3)
def _dst_utc(self, naive_utc: datetime) -> bool:
"""From UTC: both EU transitions happen at 01:00 UTC, everywhere."""
y = naive_utc.year
return self._last_sunday(y, 3, 1) <= naive_utc < self._last_sunday(y, 10, 1)
def fromutc(self, dt: datetime) -> datetime:
return dt + timedelta(hours=2 if self._dst_utc(dt.replace(tzinfo=None)) else 1)
def utcoffset(self, dt):
if dt is None:
return timedelta(hours=1)
return timedelta(hours=2 if self._dst_local(dt.replace(tzinfo=None)) else 1)
def dst(self, dt):
if dt is None:
return timedelta(0)
return timedelta(hours=1 if self._dst_local(dt.replace(tzinfo=None)) else 0)
def tzname(self, dt):
return "CEST" if dt is not None and self._dst_local(dt.replace(tzinfo=None)) else "CET"
def _billing_tz():
try:
from zoneinfo import ZoneInfo
tz = ZoneInfo("Europe/Brussels")
# Prove it resolves before trusting it: a ZoneInfo built from an empty
# tzdata answers, and answering wrongly is the failure being avoided.
if datetime(2026, 7, 1, tzinfo=timezone.utc).astimezone(tz).hour == 2:
return tz
_LOG.warning("zoneinfo Europe/Brussels resolved but gave the wrong "
"summer offset - falling back to the encoded EU rule")
except Exception as err: # noqa: BLE001 - any failure
_LOG.info("no usable tz database (%s) - using the encoded EU rule", err)
return _EUTimezone()
BILLING_TZ = _billing_tz()
def month_key(when: datetime) -> str:
"""The calendar month a UTC instant is billed in, as 'YYYY-MM' LOCAL time."""
if when.tzinfo is None:
when = when.replace(tzinfo=timezone.utc)
local = when.astimezone(BILLING_TZ)
return f"{local.year:04d}-{local.month:02d}"
# --------------------------------------------------------------------------- #
# the per-phase reduction policy - one function, deliberately
# --------------------------------------------------------------------------- #
# §9.1 requires the average be computed per-phase-aware, but does NOT say whether
# the BILLED figure on a three-phase connection is the sum of the phases or the
# worst single phase. §5.2 requires the per-phase data; nothing states the
# reduction. So it is one function with one argument, and correcting it later is
# a one-line change plus a test parameter rather than a change to the
# accumulator. §17's "Three-phase unbalanced load" scenario is the case that
# tells the three apart - on a balanced load all three agree, which is exactly
# why a balanced fixture must not be what this is tested on.
POLICY_NET = "net" # default: the connection's net offtake, what the meter bills
POLICY_SUM = "sum" # sum of the per-phase offtakes (differs when a phase exports)
POLICY_MAX = "max" # the worst single phase
def billed_w(block, policy: str = POLICY_NET) -> float:
"""Reduce one closed block to the single figure the month is billed on."""
pp = getattr(block, "per_phase_offtake_avg_w", None)
if policy == POLICY_NET or not pp:
return float(block.offtake_avg_w)
if policy == POLICY_SUM:
return float(sum(pp))
if policy == POLICY_MAX:
return float(max(pp))
raise ValueError(f"unknown per-phase reduction policy: {policy!r}")
# --------------------------------------------------------------------------- #
# what a read of the store answers
# --------------------------------------------------------------------------- #
@dataclass(frozen=True)
class PeakReading:
"""An answer about a month's peak, or an honest refusal to give one.
⚠️ There is no `peak_w_or(default)` and there must never be one. Every
accessor that lets a caller turn "the database did not answer" into a number
re-creates the defect this type exists to remove. Callers ask `is_observed`
first, and a caller that forgets gets `None` and a TypeError, which is loud.
"""
state: str
month: str
peak_w: float | None = None
quarter_start: datetime | None = None
per_phase_w: tuple | None = None
detail: str = ""
@property
def is_observed(self) -> bool:
return self.state == OBSERVED
@property
def is_unreadable(self) -> bool:
return self.state == NOT_OBSERVED
def explain(self) -> str:
if self.state == OBSERVED:
at = self.quarter_start.isoformat() if self.quarter_start else "?"
return f"{self.month}: peak {self.peak_w:.0f} W, set by the quarter at {at}"
if self.state == NO_RECORD:
return f"{self.month}: no completed quarter recorded yet"
return (f"{self.month}: PEAK UNREADABLE ({self.detail}) - this is NOT a "
f"zero peak and must not be used as headroom")
# --------------------------------------------------------------------------- #
# the store
# --------------------------------------------------------------------------- #
_SCHEMA = """
CREATE TABLE IF NOT EXISTS quarter_blocks (
quarter_start TEXT PRIMARY KEY, -- ISO-8601 UTC, aligned to 900 s
month TEXT NOT NULL, -- 'YYYY-MM', LOCAL calendar month
offtake_avg_w REAL NOT NULL,
per_phase_json TEXT
);
CREATE INDEX IF NOT EXISTS ix_quarter_month ON quarter_blocks(month);
CREATE TABLE IF NOT EXISTS monthly_peak (
month TEXT PRIMARY KEY,
peak_w REAL NOT NULL,
quarter_start TEXT NOT NULL,
per_phase_json TEXT
);
CREATE TABLE IF NOT EXISTS partial_block (
id INTEGER PRIMARY KEY CHECK (id = 1),
state_json TEXT NOT NULL
);
"""
class PeakStore:
"""SQLite behind /data. Every read answers with a state, never with a default."""
def __init__(self, path: str = "/data/tariff.db"):
self.path = path
self.ok = False
self.last_error: str | None = None
self._db: sqlite3.Connection | None = None
self._open()
def _open(self) -> None:
parent = os.path.dirname(self.path)
# ⚠️ THE DIRECTORY IS NOT CREATED. /data is mounted by the Supervisor and
# always exists on a real install, so `makedirs` here could only ever
# succeed where the durable volume is MISSING - manufacturing a database
# on the container's ephemeral layer that looks durable, reads fine, and
# is thrown away on the next restart. That is a peak store that silently
# forgets, which is worse than one that says it cannot answer.
if parent and not os.path.isdir(parent):
self.last_error = f"no durable volume at {parent}"
# INFO, not ERROR: on a dev host or in a test container there is no
# /data and nothing is wrong. A store that OPENED and then broke is
# the error case, and it is logged as one below.
_LOG.info("no peak store: %s - the monthly peak will read "
"NOT_OBSERVED", self.last_error)
return
try:
self._db = sqlite3.connect(self.path, timeout=5.0,
check_same_thread=False)
# WAL: a power cut mid-write must not leave a torn database that
# reads as "no peak this month" - the exact shape of the defect.
self._db.execute("PRAGMA journal_mode=WAL")
self._db.executescript(_SCHEMA)
self._db.commit()
self.ok = True
self.last_error = None
except sqlite3.Error as err:
self._db = None
self.ok = False
self.last_error = str(err)
# ⚠️ Deliberately NOT raised. §6's fail-open rule: an aggregation
# fault must not stop a controller that is regulating a battery. The
# cost of continuing is a peak that reads NOT_OBSERVED, and every
# reader is built to treat that as "do not use me", not as 0 W.
_LOG.error("peak store unusable at %s (%s) - the monthly peak will "
"read NOT_OBSERVED, which is not a zero peak", self.path, err)
def close(self) -> None:
if self._db is not None:
try:
self._db.close()
finally:
self._db = None
self.ok = False
# -- writing ------------------------------------------------------------
def record_block(self, block, policy: str = POLICY_NET) -> bool:
"""Persist one completed block and fold it into its month's peak.
Both tables move in ONE transaction. A crash between them would leave a
banked quarter that the monthly peak does not know about, and the peak is
the record the money is read off.
"""
if self._db is None:
return False
w = billed_w(block, policy)
start = block.start.astimezone(timezone.utc).isoformat()
month = month_key(block.start)
pp = getattr(block, "per_phase_offtake_avg_w", None)
pp_json = json.dumps(list(pp)) if pp else None
try:
with self._db:
# A block may only ever be RAISED. A restart that reprocesses a
# quarter it had already banked will have LESS coverage than the
# first pass (its first seconds are gone), so an unconditional
# REPLACE would silently lower a recorded figure.
self._db.execute(
"INSERT INTO quarter_blocks(quarter_start, month, offtake_avg_w, per_phase_json)"
" VALUES(?,?,?,?)"
" ON CONFLICT(quarter_start) DO UPDATE SET"
" offtake_avg_w=excluded.offtake_avg_w,"
" per_phase_json=excluded.per_phase_json"
" WHERE excluded.offtake_avg_w > quarter_blocks.offtake_avg_w",
(start, month, w, pp_json))
self._db.execute(
"INSERT INTO monthly_peak(month, peak_w, quarter_start, per_phase_json)"
" VALUES(?,?,?,?)"
" ON CONFLICT(month) DO UPDATE SET"
" peak_w=excluded.peak_w,"
" quarter_start=excluded.quarter_start,"
" per_phase_json=excluded.per_phase_json"
" WHERE excluded.peak_w > monthly_peak.peak_w",
(month, w, start, pp_json))
return True
except sqlite3.Error as err:
self.last_error = str(err)
_LOG.error("could not record quarter %s (%s)", start, err)
return False
# -- reading ------------------------------------------------------------
def peak(self, when: datetime | None = None, month: str | None = None) -> PeakReading:
"""The billed peak for a month. Three possible answers, never a default."""
if month is None:
month = month_key(when or datetime.now(timezone.utc))
if self._db is None:
return PeakReading(NOT_OBSERVED, month,
detail=self.last_error or "store never opened")
try:
row = self._db.execute(
"SELECT peak_w, quarter_start, per_phase_json FROM monthly_peak"
" WHERE month=?", (month,)).fetchone()
except sqlite3.Error as err:
self.last_error = str(err)
return PeakReading(NOT_OBSERVED, month, detail=str(err))
if row is None:
return PeakReading(NO_RECORD, month)
pp = tuple(json.loads(row[2])) if row[2] else None
return PeakReading(OBSERVED, month, peak_w=float(row[0]),
quarter_start=datetime.fromisoformat(row[1]),
per_phase_w=pp)
def recompute_month(self, month: str) -> PeakReading:
"""Re-derive the peak from `quarter_blocks` - the divergence check.
`monthly_peak` is a derived record, so it can be PROVEN equal to the
tier it derives from instead of trusted to be. Read-only.
"""
if self._db is None:
return PeakReading(NOT_OBSERVED, month,
detail=self.last_error or "store never opened")
try:
row = self._db.execute(
"SELECT offtake_avg_w, quarter_start, per_phase_json FROM quarter_blocks"
" WHERE month=? ORDER BY offtake_avg_w DESC, quarter_start ASC LIMIT 1",
(month,)).fetchone()
except sqlite3.Error as err:
self.last_error = str(err)
return PeakReading(NOT_OBSERVED, month, detail=str(err))
if row is None:
return PeakReading(NO_RECORD, month)
pp = tuple(json.loads(row[2])) if row[2] else None
return PeakReading(OBSERVED, month, peak_w=float(row[0]),
quarter_start=datetime.fromisoformat(row[1]),
per_phase_w=pp)
def blocks_in(self, month: str) -> list:
"""(quarter_start, offtake_avg_w) for a month, oldest first. [] on error
is safe here ONLY because no caller reads emptiness as a peak - AC 3's
counter reports the sample size it actually got."""
if self._db is None:
return []
try:
return [(datetime.fromisoformat(a), float(b)) for a, b in self._db.execute(
"SELECT quarter_start, offtake_avg_w FROM quarter_blocks"
" WHERE month=? ORDER BY quarter_start", (month,))]
except sqlite3.Error as err:
self.last_error = str(err)
return []
# -- the open block's checkpoint ----------------------------------------
def save_partial(self, state: dict) -> bool:
if self._db is None:
return False
try:
with self._db:
self._db.execute(
"INSERT INTO partial_block(id, state_json) VALUES(1,?)"
" ON CONFLICT(id) DO UPDATE SET state_json=excluded.state_json",
(json.dumps(state),))
return True
except (sqlite3.Error, TypeError, ValueError) as err:
self.last_error = str(err)
return False
def load_partial(self) -> dict | None:
if self._db is None:
return None
try:
row = self._db.execute(
"SELECT state_json FROM partial_block WHERE id=1").fetchone()
return json.loads(row[0]) if row else None
except (sqlite3.Error, ValueError) as err:
self.last_error = str(err)
return None
def clear_partial(self) -> None:
# The one DELETE in this module, and it is against the checkpoint - a
# scratch row, not a retention tier. Neither quarter_blocks nor
# monthly_peak is deleted from anywhere in this file.
if self._db is None:
return
try:
with self._db:
self._db.execute("DELETE FROM partial_block WHERE id=1")
except sqlite3.Error as err:
self.last_error = str(err)
# --------------------------------------------------------------------------- #
# the tracker: what main.py holds
# --------------------------------------------------------------------------- #
@dataclass(frozen=True)
class Projection:
"""The open block's end-of-quarter estimate, with the reasons to distrust it.
`elapsed_s` travels WITH the number rather than beside it, because early in a
quarter a projection is a 10-second extrapolation over 900 s and a consumer
that cannot see that will react to noise. `covered_frac` is separately useful:
it is below 1.0 when the meter had a gap, which is the "unreliable quarter"
the functional note asks be marked rather than reported low.
"""
projected_w: float
elapsed_w: float
elapsed_s: float
covered_frac: float
complete: bool
block_start: datetime | None
held_w: float
def explain(self) -> str:
return (f"quarter projects {self.projected_w:.0f} W "
f"({self.elapsed_s:.0f} s elapsed, {self.covered_frac * 100:.0f} % covered"
+ ("" if self.complete else ", INCOMPLETE") + ")")
class PeakTracker:
"""Drives the store from `P1Ingest`. Measures; never commands."""
def __init__(self, store: PeakStore, cap_w: float = 3500.0,
policy: str = POLICY_NET, checkpoint_s: float = CHECKPOINT_S):
self.store = store
self.cap_w = float(cap_w)
self.policy = policy
self.checkpoint_s = float(checkpoint_s)
self.recorded = 0
self.restore_state = "not attempted"
self._seen: set = set()
self._last_ckpt = 0.0
# -- startup ------------------------------------------------------------
def restore(self, averager, now: datetime | None = None) -> str:
"""Put the open block back. Returns which of the three cases happened."""
now = now or datetime.now(timezone.utc)
snap = self.store.load_partial()
if not snap:
self.restore_state = ("fresh: no checkpoint" if self.store.ok
else "fresh: store unreadable")
return self.restore_state
block = snap.get("block")
if block is None:
self.restore_state = "fresh: checkpoint had no open block"
return self.restore_state
current = int(now.timestamp() // QUARTER_S) * QUARTER_S
if block == current:
if not averager.restore(snap):
# A reshaped accumulator (meter_phases changed) is a different
# measurement, not a resumable one. Say so; do not half-resume.
self.store.clear_partial()
self.restore_state = "fresh: checkpoint does not fit this averager"
return self.restore_state
self.restore_state = (
f"resumed the open block at {snap.get('elapsed', 0.0):.0f} s "
f"of {QUARTER_S} s ({snap.get('acc', 0.0) / QUARTER_S:.0f} W banked)")
return self.restore_state
# The block ended while the add-on was down. Its measured part really
# happened, so bank it at acc/900 rather than discard measured energy.
# It reads LOW because the seconds after the shutdown were never covered,
# and that is the honest figure: coverage a closed block did not get.
closed = _ClosedBlock(
start=datetime.fromtimestamp(block, timezone.utc),
offtake_avg_w=float(snap.get("acc", 0.0)) / QUARTER_S,
per_phase_offtake_avg_w=(tuple(v / QUARTER_S for v in snap["pp_acc"])
if snap.get("pp_acc") and snap.get("last_pp") else None))
banked = self.store.record_block(closed, self.policy)
self.store.clear_partial()
self.restore_state = (
f"closed a stale block from {closed.start.isoformat()} at "
f"{closed.offtake_avg_w:.0f} W (partial coverage), "
+ ("banked" if banked else "NOT banked - store unwritable"))
return self.restore_state
# -- steady state -------------------------------------------------------
def sync(self, ingest, now_mono: float | None = None) -> int:
"""Bank any blocks that closed, and refresh the checkpoint. Returns the
number of blocks newly recorded on this call."""
import time as _t
n = 0
for block in getattr(ingest, "blocks", ()):
key = block.start.timestamp()
if key in self._seen:
continue
if self.store.record_block(block, self.policy):
self._seen.add(key)
self.recorded += 1
n += 1
if n:
# A block just closed, so the checkpoint's block is gone: clearing it
# in the same breath is what stops a crash here from replaying it.
self.store.clear_partial()
self._last_ckpt = 0.0
self.checkpoint(ingest, now_mono or _t.monotonic())
return n
def checkpoint(self, ingest, now_mono: float, force: bool = False) -> bool:
if not force and (now_mono - self._last_ckpt) < self.checkpoint_s:
return False
avg = getattr(ingest, "averager", None)
if avg is None:
return False
self._last_ckpt = now_mono
return self.store.save_partial(avg.snapshot())
# -- reading ------------------------------------------------------------
def peak(self, when: datetime | None = None) -> PeakReading:
return self.store.peak(when)
def headroom_w(self, when: datetime | None = None) -> float | None:
"""Watts left under the soft limit, or None when it cannot be known.
⚠️ `None`, not `self.cap_w`. An unreadable peak returning full headroom
is the disarming default this whole module is shaped against.
"""
r = self.peak(when)
if r.state == NO_RECORD:
return self.cap_w
if not r.is_observed:
return None
return max(0.0, self.cap_w - r.peak_w)
def warning_frac(self, when: datetime | None = None) -> float | None:
"""Peak as a fraction of the soft limit - §9.1's 80 % warning entity."""
r = self.peak(when)
if r.state == NO_RECORD:
return 0.0
if not r.is_observed or self.cap_w <= 0:
return None
return r.peak_w / self.cap_w
@staticmethod
def project(averager, now: datetime) -> Projection:
"""End-of-quarter average if the last sample held for the rest of it.
⚠️ The two divisors are DIFFERENT and only coincide at the end of the
block. `elapsed_w` divides by the seconds actually covered (what the
meter has shown so far); `projected_w` divides by the full 900 s and
credits the held value over what remains. At `elapsed_s == 900` and full
coverage they are equal - which is precisely why nothing may be tested
only at that point.
"""
start = averager.block_start
if start is None:
# ⚠️ A ZERO PROJECTION, AND IT IS NOT "THE QUARTER IS FINE". It is
# "there is no quarter". `block_start is None` is the flag that says
# which, and SAFETY-07 grades that flag BEFORE it grades the number -
# see peakguard.blindness(). A consumer that reads only
# `projected_w` here reads 0 W, which is under every ceiling there
# is, on exactly the startup path where nothing else is watching.
return Projection(0.0, 0.0, 0.0, 0.0, False, None, 0.0)
elapsed_wall = max(0.0, (now - start).total_seconds())
elapsed = averager.elapsed_s
acc = averager.partial_ws
remaining = max(0.0, QUARTER_S - elapsed_wall)
# ⚠️ NO `getattr` DEFAULT, AND THAT IS THE FIX, NOT A TIDY-UP. This read
# was `getattr(averager, "_last_net", 0.0)`: a private name with a
# fallback, so renaming the attribute would have kept every projection
# green while crediting 0 W held - a projection that reads LOW, which is
# the UNDER-GUARDING direction for SAFETY-07's floor. TEST-13 item 4.
# `held_net_w` is now a documented public property on QuarterAverager and
# is read without a default, so the same rename raises AttributeError on
# the first cycle instead of quietly disarming the money guard.
held = max(averager.held_net_w, 0.0)
return Projection(
projected_w=(acc + held * remaining) / QUARTER_S,
elapsed_w=(acc / elapsed) if elapsed > 0 else 0.0,
elapsed_s=elapsed_wall,
covered_frac=(elapsed / elapsed_wall) if elapsed_wall > 0 else 0.0,
complete=elapsed_wall <= 0 or elapsed >= elapsed_wall - 1e-9,
block_start=start,
# Carried so a consumer can see WHAT was extrapolated, not only the
# extrapolation. It is the signed held value; `projected_w` uses the
# offtake-only form above, and the two differ whenever the site is
# exporting - which is the healthy operating point here.
held_w=averager.held_net_w)
@dataclass(frozen=True)
class _ClosedBlock:
"""A block reconstructed from a checkpoint. Shaped like `p1.QuarterBlock` so
`record_block` has one input type; kept private so nothing else makes one."""
start: datetime
offtake_avg_w: float
per_phase_offtake_avg_w: tuple | None
+283
View File
@@ -116,6 +116,40 @@ NOT READY banner already names the problem, (b) does not apply.
(13.8 kWh usable at 2000 W is ~6.9 h against the 720 min DRAIN_TIMEOUT).
It is warned about below, not refused.
7. A price configuration that cannot mean what it says. [(a)]
Three cases, all in `_price_problems()`: an unknown `price_provider` id
(refused rather than defaulted to 'off', per the §14 warning box);
`price_provider` set alongside `price_now_entity`/`price_avg_entity`,
which is TWO PRICE SOURCES ON ONE SITE where the Home Assistant pair
silently wins; and `manual_csv` with no `price_csv_path`, which turns the
documented offline fallback into a permanent no-price-data condition
reached by accident. Same shape as 1, 4 and 5 every time - a source the
operator chose, overridden by a layer they cannot see.
⚠️ `price_provider: "off"` IS NOT REFUSED AND MUST NOT BECOME SO. §11.4
makes no-price-data a supported state, and §9.2 says a large share of the
installed base is on a fixed or static tariff. Refusing to start without a
price feed would lock those sites out of the add-on entirely.
8. `peak_soc_floor` at or below `bms_soc_floor`. [(a)]
SAFETY-24, and the same shape as 1, 4, 5 and 6. The peak guard can COMMAND
a discharge to hold a billed quarter down (SAFETY-07), and `peak_soc_floor`
is the EMS-side bound on how far down it may take the pack. The BMS has its
own cut-out and enforces it regardless. Set the EMS floor at or below that
cut-out and THE BMS WINS: the EMS bound can never be the thing that stops
the discharge, so the add-on carries an energy backstop that is incapable
of binding while its status page reports one - rule 5's "a disarmed guard
that advertises itself as armed", on the guard whose failure mode is a pack
that is quietly never available when it is wanted.
⚠️ `<=`, NOT `<`, and the equality case is the interesting one. At exactly
equal, the EMS floor and the pack's cut-out act at the same percent, so the
EMS floor buys ZERO margin - it can never prevent the pack reaching the BMS
limit, which is the entire thing this option exists to prevent. That is not
"close to the rail", it is the disarmed case with an extra step. Rule 6
permits equality because equality there leaves one reachable point that
works; here it leaves none.
WHAT IS DELIBERATELY *NOT* REFUSED, because it is untidy rather than unsafe:
- `integrator_max_w` above `max_w`. DOCS.md is right that the only thing the
@@ -136,11 +170,19 @@ WHAT IS DELIBERATELY *NOT* REFUSED, because it is untidy rather than unsafe:
- `maintenance_discharge_w` above `max_w`. Clamped, not contradicted, and the
clamp names itself in the arbiter's explain() - see rule 6. Warned about,
not refused.
- `peak_soc_floor` below `maintenance_soc_floor`, or above it. They are
DIFFERENT QUANTITIES (SAFETY-24: a drain target versus a spend limit) and
every ordering of the two is a legitimate site configuration. Refusing on
an ordering would be inventing a relationship the system does not have.
"""
import logging
from .arbitrage import enabled as arbitrage_on
from .p1 import is_enabled, validate_meter_options
from .peakguard import PEAK_OPTION_DEFAULTS
from .price import PRICE_OPTION_DEFAULTS, PROVIDERS
from .static_tariff import ProfileInvalid, parse_static_profile
_LOG = logging.getLogger("goodwe.config")
@@ -349,6 +391,39 @@ def validate_startup_config(opts: dict, dev_min=None, dev_max=None) -> list[str]
f"two numbers. Lower maintenance_charge_w to {max_w:.0f} W or less, "
f"or raise max_w to at least {charge_w:.0f} W.")
# --- 7. two price sources on one site ------------------------------------
problems.extend(_price_problems(opts))
# --- 8. an EMS energy floor the BMS does not already enforce -------------
# ⚠️ THE FALLBACKS ARE peakguard.PEAK_OPTION_DEFAULTS, NOT LITERALS, and
# that is CFG-03 read as a rule rather than as a complaint. Rule 6's own
# comment documents a LIVE divergence in this repo caused by typing 2000
# here and 2500 in main.py; this card refuses to add a second instance of
# it. The dict is the single in-code mirror, pinned to config.yaml by
# test_peakguard.py, and main.py reads the same one.
soc_floor = _number(opts, "peak_soc_floor",
PEAK_OPTION_DEFAULTS["peak_soc_floor"])
bms_floor = _number(opts, "bms_soc_floor",
PEAK_OPTION_DEFAULTS["bms_soc_floor"])
if soc_floor is None or bms_floor is None:
problems.append(
"peak_soc_floor and bms_soc_floor must both be numbers (got "
f"{opts.get('peak_soc_floor')!r} / {opts.get('bms_soc_floor')!r})")
elif soc_floor <= bms_floor:
problems.append(
f"peak_soc_floor is {soc_floor:.0f} % but the pack's own cut-out "
f"(bms_soc_floor) is {bms_floor:.0f} %, so SAFETY-07's forced "
"discharge has no EMS-side bound at all - THE BMS WINS, and it is "
"the pack's protection rather than yours. The guard would report a "
"SoC floor on its status page while being incapable of ever being "
"the thing that stops the discharge, and the symptom of that is a "
"battery repeatedly sitting at the BMS limit with no record of why "
f"- which looks exactly like the SoC drift the maintenance cycle "
f"exists to correct. Raise peak_soc_floor above {bms_floor:.0f} % "
f"(the shipped default is "
f"{PEAK_OPTION_DEFAULTS['peak_soc_floor']:.0f} %), or correct "
"bms_soc_floor if it does not match this pack.")
# --- warnings: real, but not worth refusing over -------------------------
integ = opts.get("integrator_max_w")
integ_w = None if not integ else _number(opts, "integrator_max_w", None)
@@ -382,6 +457,214 @@ def validate_startup_config(opts: dict, dev_min=None, dev_max=None) -> list[str]
return problems
def _price_problems(opts: dict) -> list[str]:
"""Rule 7 (TARIFF-02): the price configuration, refused where it is untrue.
⚠️ NOTE WHAT IS *NOT* HERE. `price_provider: "off"` is not refused and never
will be: §11.4 makes no-price-data a first-class supported state (static
profile, provisional plan, no arbitrage), and an add-on that will not start
without a price feed is an add-on that a Belgian fixed-tariff site - a large
share of the installed base, per §9.2 - cannot run at all.
Four refusals and two log lines (TARIFF-03 added one refusal, one parse and
two notices), and every refusal is still case (a) from this module's
docstring: a source the operator SET being overridden by a layer they cannot
see. The additions are `static_profile` alongside the Home Assistant price
pair - the same silent-loser shape as provider-plus-pair, and a LIVE one -
and a `static_profile` that does not tile the day, which would leave hours
with no price at all.
"""
problems: list[str] = []
pid = str(opts.get("price_provider", "") or "off").strip() or "off"
if pid != "off" and pid not in PROVIDERS:
problems.append(
f"price_provider is {pid!r}, which is not a shipped plugin. The "
f"ids that exist are: off, {', '.join(sorted(PROVIDERS))}. "
"Refused rather than defaulted, per the §14 warning box - a "
"silent fall back to 'off' would look exactly like a working "
"provider that happens never to find a cheap hour.")
return problems # nothing below can be judged against an unknown id
# ⚠️ THE HA-SIDE OVERLAP THE PM TOLD US TO DECIDE ABOUT, DECIDED HERE.
# price_now_entity/price_avg_entity are the original delegation of pricing
# to Home Assistant: main.py reduces them to ONE BOOLEAN, `cheap`, read by
# the maintenance cheap-window charge floor. They are KEPT - retiring them
# would take a live install's only price signal away - but they may not run
# alongside another price source, because main.py reads exactly one and the
# loser is silent. That is (a), and it is the "silently double up" the PM
# named on 2026-08-30.
#
# ⚠️ THE REASON THIS REFUSAL GIVES WAS REWRITTEN BY TARIFF-03, AND THE OLD
# WORDING IS WHY. Until TARIFF-03 this message said the configured provider
# "would decide nothing", which was true only because nothing consumed the
# series; test_price.py section 12 carried a tripwire that went red the
# moment a consumer landed, with instructions to fix the wording here in the
# same commit. It landed, it went red, and this is that fix. What is true
# NOW, stated exactly and no wider:
#
# * app/static_tariff.py consumes load_prices(). resolve_tariff() turns a
# provider series into a TariffPlan, a cheap-window flag and an
# arbitrage verdict, so the seam is no longer inert.
# * main.py reads the STATIC side of that module every cycle
# (static_status()), which makes static_profile a real writer of `cheap`.
# * main.py does NOT call resolve_tariff() on the control tick, and that
# is still deliberate: a provider fetch is network I/O and the tick is
# what feeds the ESP32 heartbeat. OPT-04 put it on a BACKGROUND REFRESH
# instead (Controller.run_tariff_refresh), and the tick reads the plan
# that task last stored - an attribute read, with nothing to block on.
# * so a configured PROVIDER now DOES shape an energy decision, but only
# through app/arbitrage.py and only while `arbitrage_enabled` is on.
# With it off - the shipped configuration - the series is still fetched,
# validated and published, and still moves nothing.
#
# Hence three refusals below rather than two, and none of them claims more
# than that list. ⚠️ THE MIDDLE CLAUSE OF THE FIRST MESSAGE IS WHAT OPT-04
# CHANGED, exactly as the note it replaced predicted - the RULE is
# untouched, because the reason it exists (main.py reads the Home Assistant
# pair FIRST and the other source is silent) is unchanged.
ha_price = [k for k in ("price_now_entity", "price_avg_entity")
if str(opts.get(k, "") or "").strip()]
if pid != "off" and ha_price:
problems.append(
f"price_provider is {pid!r} AND {' and '.join(ha_price)} "
f"{'is' if len(ha_price) == 1 else 'are'} set, which is two price "
"sources on one site. THE HOME ASSISTANT ENTITIES WIN: they are "
"what main.py reduces to the `cheap` flag the maintenance cycle "
"reads, and read_inputs() never looks at the other source once "
"they are set. The provider series is consumed - "
"app/static_tariff.py builds a tariff plan and an arbitrage "
"verdict, and app/arbitrage.py trades on it off a background "
"refresh - but the `cheap` flag those entities own is a separate "
"decision they would silently keep. Clear "
"price_now_entity and price_avg_entity to use the "
f"{pid!r} plugin, or set price_provider to 'off' to keep the "
"Home Assistant sensors.")
# ⚠️ THE SAME RULE FOR THE SOURCE TARIFF-03 ADDED, AND THIS ONE IS A LIVE
# CONFLICT RATHER THAN A LATENT ONE. main.py's read_inputs() takes `cheap`
# from the Home Assistant pair when it is set and from the static profile
# otherwise - one flag, first writer wins, and the operator who filled in a
# day/night tariff has no way to see that it lost.
if ha_price and str(opts.get("static_profile", "") or "").strip():
problems.append(
f"static_profile is set AND {' and '.join(ha_price)} "
f"{'is' if len(ha_price) == 1 else 'are'} set, which is two "
"tariff sources feeding one cheap-window flag. THE HOME ASSISTANT "
"ENTITIES WIN: main.py reads them first and never looks at the "
"static profile, so the day/night tariff you typed would decide "
"nothing and nothing would say so. Clear price_now_entity and "
"price_avg_entity to use static_profile, or clear static_profile "
"to keep the Home Assistant sensors.")
# ⚠️ THE PROFILE ITSELF, PARSED AT STARTUP RATHER THAN AT 02:00. A profile
# that does not tile the day has hours with no price, and the only ways to
# serve those are to invent a number or to fail at the moment the clock
# reaches them. §14's warning box - refuse to start on a mismatch rather
# than clamp quietly - is the same instruction one level down.
raw_profile = opts.get("static_profile", "")
if str(raw_profile or "").strip() or isinstance(raw_profile, (list, dict)):
try:
profile = parse_static_profile(raw_profile)
except ProfileInvalid as e:
profile = None
problems.append(f"static_profile cannot be used: {e}")
if profile is not None and profile.product is not None \
and len(set(profile.prices)) == 1:
_LOG.info(
"static_profile carries one distinct price (%.4f EUR/kWh "
"all-in), so this site is priced as FULLY FIXED. Arbitrage is "
"off by arithmetic - a flat series has no spread - and the "
"add-on runs self-consumption only, which is §9.2's handling "
"for this product. Peak shaving, reserve management and the "
"maintenance cycle are unaffected.", profile.prices[0])
# ⚠️ A WARNING, NOT A REFUSAL, AND THE ASYMMETRY IS DELIBERATE. §11.4 says a
# site with no price data falls back to the static profile; a dynamic site
# with none configured has nothing to fall back to when its feed dies. That
# is a real gap worth naming - but refusing to start over it would break
# every install TARIFF-02 shipped, none of which could have had one.
if pid != "off" and not str(opts.get("static_profile", "") or "").strip():
_LOG.warning(
"price_provider is %r but static_profile is empty. §11.4's "
"degradation row is 'no price data -> fall back to the static "
"profile'; with none configured, a feed outage leaves this site "
"with no tariff at all rather than with a coarse one. Not refused, "
"but filling it in is what makes that row work.", pid)
# ⚠️ OPT-04. THE THREE WAYS `arbitrage_enabled: true` CAN BE A LIE, refused
# rather than logged, because a strategy that is armed in the UI and cannot
# fire is indistinguishable from one that fires and finds no opportunity -
# which is the §14 warning box's case exactly. None of these can break an
# existing install: the option is new and ships `false`.
# ⚠️ CHECKED WHETHER OR NOT ARBITRAGE IS ARMED. A threshold that is not a
# non-negative number is a configuration error the moment it is typed, and
# deferring the complaint until somebody turns the strategy on would report
# it at the worst possible moment - the one where energy starts moving.
thresh = _number(opts, "cycle_cost_threshold_eur",
PRICE_OPTION_DEFAULTS["cycle_cost_threshold_eur"])
if thresh is None or thresh < 0:
problems.append(
f"cycle_cost_threshold_eur is "
f"{opts.get('cycle_cost_threshold_eur')!r} and must be a "
f"non-negative number of EUR/kWh - it is what one round trip COSTS "
f"(§10.1). A negative threshold is one that every spread clears, "
f"including a spread of nothing, which is arbitrage with the brake "
f"removed rather than arbitrage tuned aggressively.")
if arbitrage_on(opts):
if pid == "off" and not str(opts.get("static_profile", "") or "").strip():
problems.append(
"arbitrage_enabled is on but price_provider is 'off' and "
"static_profile is empty, so this site has NO PRICES AT ALL. "
"§10.1's priority 3 is 'charge cheap, discharge expensive' and "
"there is nothing here to tell cheap from expensive, so the "
"strategy would refuse on every cycle forever while the "
"configuration said it was running. Configure a price_provider "
"or a static_profile, or turn arbitrage_enabled off.")
if not is_enabled(opts):
problems.append(
"arbitrage_enabled is on but meter_source is 'off', so there "
"is no P1 telegram stream, the capacity-tariff peak guard "
"never runs, and nothing measures the 15-minute average this "
"strategy's charging would land in. §9.1 gives peak shaving "
"priority over arbitrage; an unmeasured peak cannot be shaved, "
"so arbitrage would be deliberately creating the billed "
"quantity with the one guard that limits it switched off. "
"app/arbitrage.py refuses to trade in that state anyway - this "
"refuses the configuration that asks it to. Set meter_source, "
"or turn arbitrage_enabled off.")
floor = _number(opts, "arbitrage_soc_floor",
PRICE_OPTION_DEFAULTS["arbitrage_soc_floor"])
ceiling = _number(opts, "arbitrage_soc_ceiling",
PRICE_OPTION_DEFAULTS["arbitrage_soc_ceiling"])
if floor is None or ceiling is None:
problems.append(
f"arbitrage_soc_floor {opts.get('arbitrage_soc_floor')!r} and "
f"arbitrage_soc_ceiling {opts.get('arbitrage_soc_ceiling')!r} "
f"must both be numbers - they are percentages of pack charge.")
elif floor >= ceiling:
problems.append(
f"arbitrage_soc_floor is {floor:g} % and arbitrage_soc_ceiling "
f"is {ceiling:g} %, so there is no band left between them: "
f"every state of charge is either at or under the floor (no "
f"discharge) or at or over the ceiling (no charge), and the "
f"strategy can never act. The floor is the reserve it may not "
f"spend and the ceiling is the headroom it may not overfill, "
f"so the floor must be the SMALLER number.")
if pid == "manual_csv" and not str(opts.get("price_csv_path", "") or "").strip():
problems.append(
"price_provider is 'manual_csv' but price_csv_path is empty, so "
"the offline fallback has no file to read. Every fetch would "
"return no-price-data with a filesystem error as its reason - "
"which is §11.4 behaviour arrived at by accident rather than by "
"configuration. Point it at the CSV, or set price_provider to "
"'off', which says the same thing on purpose.")
return problems
def _number(opts: dict, key, default):
raw = opts.get(key, default)
if raw is None or raw == "":
+177
View File
@@ -162,9 +162,147 @@ options:
# --- tariff / capacity tariff (all optional) ------------------------------
peak_forecast_entity: ""
peak_cap_w: 3500
# --- SAFETY-24: the energy backstop under SAFETY-07's forced discharge -----
# The peak guard can COMMAND a discharge to hold a billed quarter under
# peak_cap_w. Until this card it was bounded from below by nothing but the
# pack's own BMS cut-out - a real floor, but the PACK's protection rather than
# the OPERATOR's. On the 2026-08-20 replay the guard asks for 878 Wh in a day.
#
# peak_soc_floor is the percent below which that discharge is not sized. When
# it binds the guard reports FLOOR-LIMITED, keeps charging blocked (a 0 W
# floor), and the add-on raises ONE notification saying the quarter will be
# exceeded - failing to shave is never silent, because the euro figure is the
# whole point of the guard.
#
# ⚠️ THIS IS NOT maintenance_soc_floor AND MUST NOT BE MERGED WITH IT.
# maintenance_soc_floor is the DRAIN TARGET of the monthly recalibration: the
# depth the cycle deliberately goes to, once a month, on purpose. This is the
# depth a money-saving discharge may never reach on any day. The right values
# differ, and sharing one key would couple two features that want to move
# independently.
#
# ⚠️ 20 IS DERIVED, NOT PICKED. The reference pack is 13.8 kWh usable and the
# worst modelled guard day is 878 Wh, i.e. 6.4 % - so 10 points above the
# pack's own cut-out leaves the guard ~1.6 of its worst observed days of room
# and still never reaches the BMS.
peak_soc_floor: 20
# The pack's OWN cut-out percent - what the BMS enforces whatever is set
# above. Register 0x0560 reads 10 on the reference GW5000S-BP, consistent with
# DoD 90. It is here so validate.py can REFUSE a peak_soc_floor at or below
# it: such a floor can never bind, the BMS wins, and the status page would
# report an energy backstop that does not exist.
bms_soc_floor: 10
# ⚠️ THE OLD PRICE PATH, AND IT IS NOT DEAD YET. These two entities are the
# add-on delegating the whole price question to Home Assistant: main.py reads
# both and derives ONE BOOLEAN, `cheap = now <= avg`, which is consumed by
# nothing except the maintenance cycle's cheap-window charge floor. That is
# all they have ever done. They are kept, because retiring them would take
# the maintenance cycle's only price signal away on a live install before
# anything replaces it - TARIFF-03 (static profile) and OPT-04 (arbitrage)
# are the cards that replace it, and neither has landed.
#
# ⚠️ They may NOT be combined with price_provider below. Two price sources on
# one site, one of which silently wins, is exactly the "double up" the PM
# named; validate.py rule 7 refuses that configuration at startup rather than
# letting an operator believe the provider they configured is driving
# something. See app/validate.py.
price_now_entity: ""
price_avg_entity: ""
# --- price providers (TARIFF-02, specs §9.3 / §14 `tariffs:`) --------------
# `off` is the shipped default and means the §11.4 no-price-data row applies:
# static profile, plan provisional, NO ARBITRAGE. Nothing in 0.5.x consumes
# the series yet - OPT-04 is the consumer - so turning this on today fetches,
# validates and publishes prices and changes no energy decision.
#
# ⚠️ THESE DEFAULTS ARE MIRRORED ONCE, IN app/price.py's
# PRICE_OPTION_DEFAULTS, and test_price.py reads THIS FILE and asserts the two
# agree key for key. Do not add a third copy in main.py - CFG-03 is open
# because that has already happened three times.
#
# markup/taxes are the §14 `tariffs.energy_price.*` figures and are added by
# the FRAMEWORK, only for the components the chosen plugin declares its feed
# does not already contain. engie_dynamic, totalenergies_dynamic, eneco_eddy
# and manual_csv publish a figure that already carries the markup, so setting
# it under those is ignored - and logged as ignored, never applied twice.
price_provider: "off"
price_markup_eur_kwh: 0.03
price_taxes_eur_kwh: 0.06
price_csv_path: "/share/goodwe_prices.csv"
price_feed_url: ""
price_horizon_h: 48
price_timeout_s: 10
# --- static / fully-fixed tariffs (TARIFF-03, specs §9.2 / §11.4) ----------
# §14's `tariffs.static_profile`, and "" is its `null`. §9.2: "a large share
# of the installed base is on [a static profile]" - this is what makes those
# sites first-class rather than "price_provider: off, good luck".
#
# A JSON object, on one line, whose periods must tile every day type exactly
# once - no gap and no overlap. A gap is REFUSED at startup rather than
# defaulted, because an invented price is one an optimiser would trade on.
# A fully-fixed tariff is ONE period covering 00:00-24:00, all days; the
# add-on derives "fixed" from there being one distinct price, so there is no
# box to tick and none to tick wrongly.
#
# {"periods": [
# {"name":"day", "from":"07:00","to":"22:00","days":"mon-fri","eur_per_kwh":0.34},
# {"name":"night","from":"22:00","to":"07:00","days":"mon-fri","eur_per_kwh":0.26},
# {"name":"night","from":"00:00","to":"24:00","days":"sat,sun", "eur_per_kwh":0.26}]}
#
# ⚠️ A window written "22:00-07:00 mon-fri" does NOT cover Saturday morning -
# a period applies on the day the CLOCK reads, so the weekend row above is
# required, not decorative. See DOCS.md.
#
# eur_per_kwh is ALL-IN (energy + markup + taxes, §9.3), the same basis the
# dynamic providers are assembled onto. price_markup_eur_kwh and
# price_taxes_eur_kwh are NOT added to it.
#
# ⚠️ RECOMMENDED EVEN ON A DYNAMIC SITE. §11.4's degradation row is "no price
# data -> fall back to the static profile"; with this empty there is nothing
# to fall back to and a dead feed leaves the site with no tariff at all.
# validate.py warns about exactly that.
static_profile: ""
# §9.3: injection is priced SEPARATELY, and in Belgium it is often near zero -
# which by itself strongly favours self-consumption over export. A scalar, not
# a second series: no shipped supplier publishes a dynamic injection curve.
export_price_eur_kwh: 0.04
# --- price arbitrage (OPT-04, specs §10.1 priority 3 / §17) ---------------
# Charge in the cheapest published hour, discharge in the dearest, and ONLY
# when the spread covers what the round trip costs. Off in the shipped
# configuration: this is the first strategy in the add-on that moves energy
# against the meter on purpose, so it is armed deliberately or not at all.
#
# ⚠️ IT NEEDS A PRICE SOURCE AND A P1 METER, and validate.py refuses the
# combinations that cannot work rather than letting them look armed. With
# `price_provider: off` and no `static_profile` there is no series to trade
# on; with `meter_source: off` the capacity-tariff peak guard never runs, and
# §9.1 puts peak shaving above arbitrage - a charge nothing measures is the
# exact failure that guard exists to prevent.
#
# ⚠️ AT THE SHIPPED THRESHOLD A STATIC DAY/NIGHT SITE DOES NOT TRADE. The
# realistic Belgian day/night spread is about 0.08 EUR/kWh and the threshold
# below is 0.10, so the verdict is "the firm spread does not cover this
# cycle's cost" and nothing happens - correctly. That is a product decision
# (this number, and app/price.py's widening factors) and it is open with the
# owner; the code is correct at any value and no number was picked to make a
# trade appear.
arbitrage_enabled: false
# §10.1's `cycle_cost_threshold_eur`: what one full round trip costs per kWh
# - the wear on the pack plus the conversion losses. A spread below this is
# a cycle that loses money and pack life, so it is not taken.
cycle_cost_threshold_eur: 0.10
# The reserve arbitrage may not spend, and the headroom it may not overfill.
# Deliberately NOT the maintenance_soc_* keys: those describe a monthly
# balancing cycle, and pointing a trading strategy at them would couple two
# unrelated policies through one number.
arbitrage_soc_floor: 20
arbitrage_soc_ceiling: 95
# --- site ------------------------------------------------------------------
estop_fitted: false
log_level: info
@@ -236,8 +374,47 @@ schema:
peak_forecast_entity: str?
peak_cap_w: int(500,15000)
# SAFETY-24. No "off" value: a guard that can command a discharge does not get
# a supported way to have no energy bound at all. The ranges OVERLAP on
# purpose - the schema cannot express "above bms_soc_floor", so validate.py
# rule 8 is what refuses the pair, with a message naming both numbers.
peak_soc_floor: int(5,60)
bms_soc_floor: int(0,50)
price_now_entity: str?
price_avg_entity: str?
# ⚠️ The provider list is written out rather than left as `str`, so the
# Supervisor rejects a typo in the UI. validate.py ALSO refuses an unknown id
# against app/price.py's PROVIDERS registry, because sim/inject.py and the
# checks reach validate_startup_config() with a dict the Supervisor never saw.
# Adding a sixth plugin means adding it here and to PROVIDERS - and nowhere
# else, which is AC2.
price_provider: list(off|generic_dayahead|engie_dynamic|totalenergies_dynamic|eneco_eddy|manual_csv)
price_markup_eur_kwh: float(0,1)
price_taxes_eur_kwh: float(0,1)
price_csv_path: str?
price_feed_url: str?
price_horizon_h: int(1,72)
price_timeout_s: int(2,60)
# A JSON string rather than a nested schema, deliberately: it is ONE option
# key (§14 has one), it survives config.yaml's flat options block, and
# parse_static_profile() accepts an already-parsed dict too, so a future
# nested schema is a config.yaml change and no code change at all.
static_profile: str?
# ⚠️ The lower bound is NEGATIVE on purpose. Belgian injection tariffs have
# gone below zero (you pay to export at midday in summer), and a schema that
# bottoms out at 0 would make that site inexpressible.
export_price_eur_kwh: float(-1,1)
# OPT-04. ⚠️ The threshold's lower bound is 0 and not negative: it is a COST,
# and a negative one is a threshold every spread clears - arbitrage with the
# brake removed. app/arbitrage.py refuses it in code too, because
# validate_startup_config() is also sim/inject.py's entry point and can be
# handed any dict at all.
arbitrage_enabled: bool
cycle_cost_threshold_eur: float(0,1)
arbitrage_soc_floor: int(5,95)
arbitrage_soc_ceiling: int(10,100)
estop_fitted: bool
log_level: list(trace|debug|info|warning|error)
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,551 @@
#!/usr/bin/env python3
"""DATA-02: fetch 12 months of EPEX BE day-ahead prices, in §9.3's shape.
python3 scenarios/fetch_epex_be.py --start 2025-09-01 --end 2026-09-01 \
--out scenarios/epex-be-dayahead-2025-09-01_2026-09-01.json
Run from `goodwe_controller/`. Rewrites the output file from scratch; the
committed dataset is exactly what this produces, and `--verify` re-derives every
claim the file makes about itself without going to the network.
WHAT THIS IS NOT
================
NOT wired into the add-on, and it must never be. `price.py`'s `GenericDayAhead`
is the live path and it has its own egress check; this is a one-shot archival
fetch that produces a file for REPLAY, run by hand by a developer. Nothing under
`app/` imports it.
THE SOURCE, AND WHY THIS ONE
============================
`https://api.energy-charts.info/price?bzn=BE` - Fraunhofer ISE's Energy-Charts
API. No key, no registration.
⚠️ THE LICENCE WAS CHECKED BEFORE ANY DATA WAS COMMITTED, because for this
source it is NOT uniform across bidding zones. The API's own OpenAPI document
(`https://api.energy-charts.info/openapi.json`, /price) splits the zones in two:
"The data for the following bidding zones is licensed as CC BY 4.0 from
Bundesnetzagentur | SMARD.de and is published without changes:
AT, BE (Belgium), CH, CZ, DE-LU, ..."
"The data for the other bidding zones is for private and internal use only.
The utilization of any data, whether in its raw or derived form, for
external or commercial purposes is expressly prohibited. Should you require
licensing for market-related data, please direct your inquiries to the
original data providers, including but not limited to EPEX SPOT SE."
BE is in the FIRST list, so redistribution is permitted with attribution and the
dataset is committed rather than left as a fetch script and a checksum. Every
response also carries the licence per-response in `license_info`, and this
script REFUSES to write a file if that field ever stops saying CC BY 4.0 - a
licence checked once by a human and assumed forever is a licence that changes
silently. Had BE been in the second list, the correct output of this card would
have been this script and a checksum and no data.
Attribution, carried in the dataset's own `provenance` block and reproduced
here: data CC BY 4.0, Bundesnetzagentur | SMARD.de, retrieved via Energy-Charts
(Fraunhofer ISE), https://energy-charts.info, published without changes.
Cross-checked against the upstream on 2026-08-31: SMARD's own chart_data series
`4996/DE` over 2026-04-10 matched this API's BE series on all 96 quarter-hours
with a maximum absolute difference of 0.00 EUR/MWh. "Published without changes"
is a claim worth checking once; `--verify` does not repeat it, because it is a
claim about the SOURCE and not about the file.
RAW WHOLESALE, AND THE ASSUMPTION THAT IS NOT BAKED IN
======================================================
⚠️ WHAT COMES BACK IS RAW WHOLESALE, EUR/MWh, DAY-AHEAD AUCTION CLEARING PRICE.
It is NOT all-in: no supplier markup, no distribution or transmission tariff, no
levies, no VAT. §9.3's canonical point is all-in, and TARIFF-02's whole trap is
a figure that already contains a component being handed to something that adds
it again.
So this file does NOT add one. `eur_per_kwh` in the output is the wholesale
figure converted from EUR/MWh, `provenance.all_in` is `false` beside it, and the
markup/tax assumption a consumer needs lives in `provenance.all_in_assumption`
as NUMBERS TO APPLY rather than numbers already applied. test_arbitrage.py
section 7 applies them where a reader can watch it happen.
The assumption's SHAPE is the part that is not arbitrary, and it decides which
conclusions depend on the guessed numbers at all:
all_in(h) = (wholesale(h) + markup + network_and_levies) * (1 + vat)
* The two ADDITIVE terms cancel out of a spread: max - min is unchanged by
them. So how often a day clears `cycle_cost_threshold_eur` does NOT depend
on the markup guess - only on the VAT multiplier.
* They do NOT cancel out of §17's negative-price guard, which compares ONE
hour's all-in price against zero. A wholesale hour at -0.02 EUR/kWh is
comfortably positive all-in once ~0.12 of fixed charges is added.
Both are asserted arithmetically in section 7 rather than believed, and the
trade count is reported on the raw spread as well, so a reader can see exactly
how much of the answer rests on the guess.
The numbers themselves (2026 Belgian residential dynamic contract, order of
magnitude) are a STATED GUESS, labelled as one everywhere they appear. They are
not a measurement and this card had nothing to measure them with.
HOURS, QUARTER-HOURS, AND THE SEAM IN THE MIDDLE OF THE YEAR
============================================================
⚠️ THE SOURCE CHANGES RESOLUTION PART-WAY THROUGH ANY RECENT 12 MONTHS. EPEX
day-ahead moved to a 15-minute market time unit in October 2025 and the feed
follows: 2025-09-10 comes back as 24 hourly points, 2025-12-10 as 96
quarter-hourly ones. An importer that assumed either one silently loses three
quarters of the year, or reads a quarter as an hour.
The canonical format is hourly, so quarter-hours are aggregated: an hour's price
is the unweighted mean of its four quarters. That is a STATED APPROXIMATION of
the hourly product, not the hourly product, and it errs in the direction that
matters here - averaging discards intra-hour variation, so the observed spread
is a LOWER bound on what a 15-minute strategy would see.
`provenance.resolution` records how many hours came from each source shape.
⚠️ AND A PARTIAL HOUR IS DROPPED, NOT AVERAGED OVER WHAT ARRIVED. An hour with
three of four quarters is an hour nobody published a complete price for; the
mean of three would be an invented number carrying no flag. Dropped hours are
listed in `provenance.gaps` with their timestamps, because a missing hour in a
feed is not a zero price, and a zero price is exactly the value that would make
everything look free.
SIZE: WHY THIS IS PLAIN JSON AND NOT A `.gz`, MEASURED
======================================================
The directory's existing captures are gzipped, so the default would have been to
gzip this too. Measured on the committed file instead of assumed:
working tree, plain JSON 893 171 bytes
what GIT actually stores for it (zlib, loose object) 74 590 bytes
what git would store for a committed `.json.gz` 56 504 bytes
⚠️ GIT ALREADY COMPRESSES BLOBS, so gzipping buys 18 KB - 24 % of an object
that is already 75 KB, in a repository whose three existing P1 captures cost
27 KB, 136 KB and 140 KB. Eighteen kilobytes is not worth making the provenance
block unreadable in a pull request, and the licence claim in that block is
precisely the thing a reviewer of this card has to read and check. Plain JSON
also greps, diffs line-by-line when a future refetch changes something, and is
what `--verify` opens. `load()` still handles `.gz` for a future dataset large
enough to want it; this one is not.
The P1 captures are gzipped for the opposite reason and it is not an
inconsistency: they are hundreds of thousands of telegram lines that no human
reads, and there is no metadata in them to review.
TIME
====
Every instant is carried as UTC and rendered with `app.price.brussels()`, the
module that already owns the EU DST rule without a tz database. There is no
second answer to that question here and there must not be: the point of
`brussels()` is that the 02:00-03:00 hour which happens twice in October is two
instants, and a 24-slot local-hour importer collapses them.
So a year is 8760 hours only if it contains no transition, or contains both and
they cancel - 25 in October, 23 in March. `--verify` therefore asserts the
PER-DAY counts, not the total, because the total is exactly the number a broken
importer also produces.
"""
import argparse
import gzip
import hashlib
import json
import os
import sys
import time
import urllib.error
import urllib.request
from collections import Counter, defaultdict
from datetime import datetime, timedelta, timezone
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from app.price import brussels # noqa: E402
API = "https://api.energy-charts.info/price"
BZN = "BE"
LICENCE_MUST_CONTAIN = "CC BY 4.0"
UTC = timezone.utc
ATTRIBUTION = ("Data licensed CC BY 4.0 by Bundesnetzagentur | SMARD.de, "
"retrieved via the Energy-Charts API (Fraunhofer ISE), "
"https://energy-charts.info. Published without changes.")
# ⚠️ A GUESS, LABELLED AS ONE, AND APPLIED BY THE CONSUMER - NOT HERE.
# Order-of-magnitude 2026 Belgian residential dynamic contract. Nothing in this
# repository measured any part of it.
ALL_IN_ASSUMPTION = {
"form": "all_in = (wholesale + markup + network_and_levies) * (1 + vat)",
"markup_eur_per_kwh": 0.020,
"network_and_levies_eur_per_kwh": 0.100,
"vat_fraction": 0.06,
"status": "ASSUMPTION, NOT MEASURED. Stated so a consumer applies it "
"deliberately; the committed eur_per_kwh has none of it applied.",
"note": "The two additive terms cancel out of any spread, so an arbitrage "
"trade count depends on vat_fraction only. They do NOT cancel out "
"of the negative-price guard (17), which compares one hour's "
"all-in price against zero.",
}
PACE_S = 3.0 # between chunks, unconditionally
RETRIES = 6
def _get(url: str) -> dict:
"""One GET, backing off on 429 and 5xx.
⚠️ THIS IS A FREE PUBLIC SERVICE AND IT RATE-LIMITS. The first run of this
script was refused with 429 on its fifth chunk. Backing off is the polite
behaviour; it is also the correct one, because the alternative - catching
the error and carrying on - would produce a file that is silently missing a
month and says nothing about it. A refused request is not an empty month.
"""
req = urllib.request.Request(
url,
headers={"User-Agent": "goodwe-addon/DATA-02 (one-shot archival fetch)"})
wait = 5.0
for attempt in range(1, RETRIES + 1):
try:
with urllib.request.urlopen(req, timeout=90) as r:
return json.loads(r.read().decode("utf-8"))
except urllib.error.HTTPError as e:
if e.code not in (429, 500, 502, 503, 504) or attempt == RETRIES:
raise
print(f" HTTP {e.code}; backing off {wait:.0f}s "
f"(attempt {attempt}/{RETRIES})", file=sys.stderr)
time.sleep(wait)
wait *= 2
raise SystemExit(f"unreachable: {url}")
def local_midnight_utc(day: str) -> datetime:
"""The INSTANT at which Brussels midnight begins on `day` (YYYY-MM-DD).
⚠️ THE WINDOW IS LOCAL, NOT UTC, AND THE FIRST RUN OF THIS SCRIPT PROVED WHY.
Bounding it at 00:00 UTC starts the year at 02:00 CEST, which makes the first
Brussels day 22 hours long - and 22 is not one of the two lengths a
transition produces, so `report()` refused the file. A dataset whose first
day is short is a dataset whose "days per year" arithmetic is quietly wrong
everywhere downstream.
Derived from `brussels()` by trying the only two offsets Belgium has and
keeping the one that round-trips, rather than by restating the EU rule here.
"""
for hours in (1, 2):
guess = datetime.fromisoformat(f"{day}T00:00:00").replace(
tzinfo=timezone(timedelta(hours=hours))).astimezone(UTC)
if brussels(guess).isoformat() == f"{day}T00:00:00+0{hours}:00":
return guess
raise SystemExit(f"no Brussels midnight for {day!r} - brussels() disagrees "
f"with itself, which is a bug in price.py, not here.")
def _months(start: datetime, end: datetime):
"""[start, end) in month-sized chunks, so a bad chunk names its own window."""
a = start
while a < end:
b = min(end, (a.replace(day=1) + timedelta(days=32)).replace(day=1))
yield a, b
a = b
def fetch(start: datetime, end: datetime):
"""Raw {unix_second: EUR/MWh} over [start, end), deduplicated.
⚠️ A `null` price is DROPPED, never coerced. The API returns null for an
interval it holds no result for, and `or 0.0` is the version of this line
that ships a year of free electricity.
"""
seen, licences, nulls = {}, Counter(), 0
for a, b in _months(start, end):
# ⚠️ THE API'S `end` IS AN INCLUSIVE LOCAL DAY, AND THE WINDOW'S IS AN
# EXCLUSIVE INSTANT. Ask for the day containing the LAST hour of the
# chunk, derived from the instant rather than by subtracting a calendar
# day from the bound: `b - 1 day` drops the final local day whenever the
# window ends at local midnight, which is exactly how this window ends.
# The first run lost 2026-08-31 that way, silently - 8736 points instead
# of 8760 - and the points-plus-gaps accounting in report() is what
# noticed. A day nobody asked for is not a day with no prices.
first = brussels(a).date().isoformat()
last = brussels(b - timedelta(hours=1)).date().isoformat()
url = f"{API}?bzn={BZN}&start={first}&end={last}"
d = _get(url)
lic = str(d.get("license_info", ""))
licences[lic] += 1
if LICENCE_MUST_CONTAIN not in lic:
raise SystemExit(
f"REFUSING TO WRITE. {url} came back under licence {lic!r}, "
f"which does not say {LICENCE_MUST_CONTAIN!r}. The BE zone was "
f"redistributable when this card ran; if that has changed, the "
f"right output is this script and a checksum, not the data.")
unit = str(d.get("unit", ""))
if "MWh" not in unit:
raise SystemExit(
f"REFUSING TO WRITE. {url} reports unit {unit!r}, not EUR/MWh. "
f"A silent unit change is a factor of 1000 on every price.")
for t, p in zip(d["unix_seconds"], d["price"]):
if p is None:
nulls += 1
continue
if start.timestamp() <= t < end.timestamp():
seen[int(t)] = float(p)
print(f" {a.date()} .. {b.date()} {len(d['price']):5d} intervals",
file=sys.stderr)
time.sleep(PACE_S)
return seen, licences, nulls
def to_hourly(raw: dict):
"""Hourly means keyed by UTC hour. Returns (hours, gaps, shape_counts).
An hour is emitted only when every interval the source used for it arrived:
1 of 1 where the source was hourly, 4 of 4 where it was quarter-hourly.
Anything else is a gap and is named as one.
"""
buckets = defaultdict(dict)
for t, p in raw.items():
buckets[t - (t % 3600)][t % 3600] = p
hours, gaps, shapes = {}, [], Counter()
for h in sorted(buckets):
got = buckets[h]
if set(got) == {0}:
shapes["hourly"] += 1
hours[h] = got[0]
elif set(got) == {0, 900, 1800, 2700}:
shapes["quarter_hourly_mean"] += 1
hours[h] = sum(got.values()) / 4.0
else:
gaps.append({
"hour_utc": datetime.fromtimestamp(h, UTC).isoformat(),
"intervals_present": sorted(got),
"why": "incomplete hour - dropped rather than averaged over "
"what arrived; a partial read is not an observation",
})
return hours, gaps, shapes
def canonical(hours: dict) -> list:
"""9.3 points. THREE FIELDS, rendered at the Brussels offset for the instant."""
return [{"start": brussels(datetime.fromtimestamp(h, UTC)).isoformat(),
"eur_per_kwh": round(hours[h] / 1000.0, 6) + 0.0,
"provisional": False}
for h in sorted(hours)]
def expected_hours(start: datetime, end: datetime) -> int:
"""Whole hours in the window. Derived, rather than 8760 typed in."""
return int((end - start).total_seconds() // 3600)
def per_local_day(points) -> Counter:
"""Points per BRUSSELS calendar day - the count that exposes a DST bug."""
n = Counter()
for p in points:
n[p["start"][:10]] += 1
return n
def digest(points) -> str:
"""A checksum over the POINTS, so reformatting the envelope does not move it."""
h = hashlib.sha256()
for p in points:
h.update(f"{p['start']}|{p['eur_per_kwh']:.6f}|{int(p['provisional'])}\n"
.encode())
return h.hexdigest()
def load(path: str) -> dict:
op = gzip.open if path.endswith(".gz") else open
with op(path, "rt", encoding="utf-8") as f:
return json.load(f)
def report(doc, *, start=None, end=None) -> int:
"""Re-derive every claim the file makes about itself. Returns failures."""
pts = doc["prices"]
prov = doc["provenance"]
bad = 0
def ck(name, cond):
nonlocal bad
print(f" {'ok ' if cond else 'FAIL'} {name}")
if not cond:
bad += 1
ck(f"the checksum over the {len(pts)} points matches the recorded one "
f"({prov['sha256_of_points'][:16]}...)",
digest(pts) == prov["sha256_of_points"])
ck("every point has exactly the three canonical fields and nothing else",
all(set(p) == {"start", "eur_per_kwh", "provisional"} for p in pts))
ck("every start carries a UTC offset, and it is +01:00 or +02:00 - a naive "
"timestamp here is a wall clock plus an assumption",
bool(pts) and all(p["start"][-6:] in ("+01:00", "+02:00") for p in pts))
ck("the series is strictly increasing in time, so no instant is duplicated",
all(datetime.fromisoformat(a["start"]) < datetime.fromisoformat(b["start"])
for a, b in zip(pts, pts[1:])))
ck("consecutive points are exactly one hour apart AS INSTANTS, so the two "
"02:00s of the October day are two hours rather than one",
all((datetime.fromisoformat(b["start"])
- datetime.fromisoformat(a["start"])) == timedelta(hours=1)
for a, b in zip(pts, pts[1:])))
days = per_local_day(pts)
odd = {d: n for d, n in days.items() if n != 24}
ck(f"exactly two Brussels days are not 24 hours long, and they are the "
f"transitions: {sorted(odd.items())}",
sorted(odd.values()) == [23, 25] and len(odd) == 2)
spring = [d for d, n in odd.items() if n == 23]
autumn = [d for d, n in odd.items() if n == 25]
ck(f"the 23-hour day {spring} is in March and the 25-hour day {autumn} is "
f"in October, which is the direction the EU rule actually runs",
len(spring) == 1 and len(autumn) == 1
and spring[0][5:7] == "03" and autumn[0][5:7] == "10")
if autumn:
twice = [p["start"] for p in pts
if p["start"].startswith(autumn[0] + "T02")]
ck(f"...and 02:00 on {autumn[0]} occurs TWICE at two different offsets "
f"{twice} - exactly the case a 24-slot local-hour array collapses",
len(twice) == 2 and twice[0][-6:] == "+02:00"
and twice[1][-6:] == "+01:00")
ck(f"...and the two carry DIFFERENT prices, so the 25th hour is a real "
f"observation and not the same one written down twice",
len({p["eur_per_kwh"] for p in pts
if p["start"].startswith(autumn[0] + "T02")}) == 2)
if spring:
ck(f"...and 02:00 on {spring[0]} does not occur at all, because that "
f"wall clock reading never happened in Belgium",
not any(p["start"].startswith(spring[0] + "T02") for p in pts))
if start and end:
want = expected_hours(start, end)
ck(f"{len(pts)} points for a window of {want} whole hours, and the "
f"{len(prov['gaps'])} recorded gaps account for the difference "
f"exactly", len(pts) + len(prov["gaps"]) == want)
# ⚠️ A ZERO IS NOT AUTOMATICALLY A DEFECT HERE, AND SAYING IT WAS WOULD BE
# THE WRONG LESSON. The Belgian market really does clear at 0.00 EUR/MWh in
# solar-glut hours; this window contains such hours. What must be impossible
# is a zero that arrived by COERCION - a dropped read turned into a free
# hour. That is excluded structurally rather than by outlawing the value:
# nulls are dropped, incomplete hours become gaps, and the accounting above
# shows points + gaps covering every hour of the window, so every emitted
# point stands for intervals the source actually published.
zeros = [p["start"] for p in pts if p["eur_per_kwh"] == 0.0]
ck(f"the {len(zeros)} hours priced at exactly zero match the count recorded "
f"at build time, so no zero appeared between the fetch and the file",
len(zeros) == prov["exact_zero_hours"])
ck("...and none of them is a coerced read: every emitted hour is backed by "
"published intervals, which is what the points-plus-gaps accounting "
"above establishes",
len(prov["gaps"]) == 0 or all("intervals_present" in g
for g in prov["gaps"]))
ck("the figures are declared RAW WHOLESALE, and the all-in assumption is "
"recorded as an assumption rather than applied",
prov["all_in"] is False
and "ASSUMPTION" in prov["all_in_assumption"]["status"])
return bad
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("--start", default="2025-09-01")
ap.add_argument("--end", default="2026-09-01",
help="exclusive; both bounds are BRUSSELS-local midnights")
ap.add_argument("--out", default=None)
ap.add_argument("--verify", default=None,
help="re-derive the claims of an existing file; no network")
a = ap.parse_args()
if a.verify:
doc = load(a.verify)
print(f"verifying {a.verify}")
s = datetime.fromisoformat(doc["provenance"]["window_utc"]["start"])
e = datetime.fromisoformat(doc["provenance"]["window_utc"]["end"])
bad = report(doc, start=s, end=e)
print("OK" if not bad else f"{bad} FAILED")
return 1 if bad else 0
start = local_midnight_utc(a.start)
end = local_midnight_utc(a.end)
out = a.out or f"scenarios/epex-be-dayahead-{a.start}_{a.end}.json"
print(f"fetching {BZN} day-ahead {start.date()} .. {end.date()}",
file=sys.stderr)
raw, licences, nulls = fetch(start, end)
hours, gaps, shapes = to_hourly(raw)
pts = canonical(hours)
doc = {
"provenance": {
"card": "DATA-02",
"what": "EPEX BE day-ahead auction clearing prices, hourly",
"source": f"{API}?bzn={BZN} (Energy-Charts API, Fraunhofer ISE)",
"upstream": "Bundesnetzagentur | SMARD.de",
"licence": "CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)",
"licence_checked": "before committing anything; the API's OpenAPI "
"document lists BE among the CC BY 4.0 zones and "
"the remaining zones as private/internal use "
"only, so redistribution of THIS zone is "
"permitted with attribution",
"licence_as_returned": sorted(licences),
"attribution": ATTRIBUTION,
"cross_check": "SMARD chart_data 4996/DE over 2026-04-10 matched "
"this feed on all 96 quarter-hours, max abs "
"difference 0.00 EUR/MWh (2026-08-31)",
"retrieved_utc": datetime.now(UTC).replace(microsecond=0).isoformat(),
"fetched_by": "scenarios/fetch_epex_be.py",
"window_utc": {"start": start.isoformat(), "end": end.isoformat()},
"unit": "EUR per kWh (source publishes EUR/MWh; divided by 1000)",
"all_in": False,
"all_in_note": "RAW WHOLESALE. No supplier markup, no network or "
"transmission tariff, no levies, no VAT. 9.3's "
"canonical point is all-in and this is NOT; the "
"consumer applies the assumption below, visibly.",
"all_in_assumption": ALL_IN_ASSUMPTION,
"resolution": {
"note": "EPEX day-ahead moved to a 15-minute market time unit "
"in Oct 2025, part-way through this window.",
"hours_from_hourly_source": shapes["hourly"],
"hours_from_quarter_hourly_mean": shapes["quarter_hourly_mean"],
"aggregation": "unweighted mean of the four quarters; a stated "
"approximation of the hourly product, and a "
"LOWER bound on the spread a 15-minute strategy "
"would have seen",
},
"timezone": "starts rendered at the Brussels offset by "
"app.price.brussels(); instants carried in UTC, so the "
"transition days come out 23 and 25 points by "
"construction rather than by a special case",
"nulls_dropped": nulls,
"gaps": gaps,
"gaps_note": "hours the source did not publish completely. Dropped, "
"never zero-filled: a missing hour is not a free hour.",
"points": len(pts),
"exact_zero_hours": sum(1 for p in pts if p["eur_per_kwh"] == 0.0),
"negative_hours": sum(1 for p in pts if p["eur_per_kwh"] < 0),
"zero_note": "REAL CLEARINGS AT 0.00 EUR/MWh, not dropped reads. "
"Nulls are dropped and incomplete hours become gaps, "
"so points + gaps == hours in the window and no point "
"stands in for something that failed to arrive.",
"sha256_of_points": digest(pts),
},
"prices": pts,
}
with open(out, "w", encoding="utf-8", newline="\n") as f:
json.dump(doc, f, indent=1, ensure_ascii=False)
f.write("\n")
print(f"wrote {out} {len(pts)} points {os.path.getsize(out)} bytes",
file=sys.stderr)
print(f"verifying {out}")
return 1 if report(doc, start=start, end=end) else 0
if __name__ == "__main__":
raise SystemExit(main())
File diff suppressed because it is too large Load Diff
+550
View File
@@ -0,0 +1,550 @@
"""Runnable check for SAFETY-03, the output clamp and its winding-loop alarm.
python3 test_clamp.py
No framework, asserts only, same house style as test_freeze.py.
Five parts, and parts 3 and 4 are the ones that carry the acceptance criteria:
1. THE CLAMP ARITHMETIC, on the shipped `clamp_to_rating()` - boundaries,
symmetry, the device second stage, and the non-finite cases. Table driven.
2. THE ALARM'S CADENCE - rising edge, re-assert, hysteresis - on the shipped
`ClampWatchdog`. Every engagement counted, notifications not flooded.
3. AC 1 + AC 3, WIRED. A computed setpoint that exceeds the rating, driven
through the REAL `Controller.cycle()`: nothing over the rating reaches the
device, and the alarm fires. Includes the historical runaway from
specs.md 10.3 replayed against an unclamped transcription, so the check
says what the clamp is actually worth in watts rather than asserting a
constant.
4. AC 2, AND IT IS THE LOAD-BEARING HALF. A loop that SATURATES WITHOUT
CLAMPING - the inverter refusing to follow a perfectly legal command - must
raise NO clamp alarm. If it does, the bug signal is inside the noise again
and the whole ticket has bought nothing.
5. NO PATH BYPASSES IT. write_setpoint() is the only route to the wire, so it
is called directly with an out-of-range value the way the heartbeat and the
operator's buttons call it.
------------------------------------------------------------------------------
WHY PART 4 ALSO ASSERTS ON `Decision`'s BOOLEANS AND NOT ON `reason`
`Decision.reason` is one string and the stages after the clamp overwrite it. The
check below drives a cycle that clamps the integrator AND is then slew limited,
and asserts that `reason` does NOT say so while `i_clamped` does. That is not a
style preference - an alarm keyed on the string is silent on exactly the cycles a
winding loop produces, and this is the check that would go red if someone
"simplified" main.py back onto it.
------------------------------------------------------------------------------
WHAT THIS CHECK DOES NOT COVER
- The firmware-side magnitude clamp (the card's "two clamps, not one"). This
add-on's `write_setpoint()` is the add-on half; the ESP32 half is DRIVER-07's
and is not in this repo's Python.
- A rig run. Nothing here needs one: the clamp is arithmetic on the write path
and part 3 drives the shipped `Controller.cycle()` against a stub Home
Assistant, which is the same surface test_freeze.py part 1-2 use. The wire-
level invariant the card asks for (`|emitted| <= rating` on every frame) is
asserted here over every value that reached `set_number()`, which is the last
thing this process controls; proving it at the inverter needs the witness log
and belongs with DRIVER-07.
"""
import asyncio
import math
import sys
from datetime import datetime, timedelta, timezone
from app.clamp import (
CLEAR_CYCLES, REASON_CODE, REASSERT_S, ClampWatchdog, clamp_to_rating,
is_commandable,
)
from app.control import Tuning, compute
from app.main import Controller
fails = []
T0 = datetime(2026, 8, 30, 9, 0, 0, tzinfo=timezone.utc)
# The live install's numbers. NOTES.md and freeze.py both say max_w: 5000 there,
# which is also the device rating, so this is the bound the alarm is written at.
LIVE_MAX_W = 5000.0
CLAMP_ID = "goodwe_output_clamp"
def check(name, cond):
print(f" {'ok ' if cond else 'FAIL'} {name}")
if not cond:
fails.append(name)
# ==============================================================================
print("1. the clamp arithmetic, on the shipped expression")
# (requested, limit, expected applied, expected engaged, expected bound)
TABLE = [
(5001.0, 5000.0, 5000.0, True, "max_w"), # +limit+1
(-5001.0, 5000.0, -5000.0, True, "max_w"), # -limit-1, the export side
(5000.0, 5000.0, 5000.0, False, ""), # exactly +limit: NOT engaged
(-5000.0, 5000.0, -5000.0, False, ""), # exactly -limit: NOT engaged
(0.0, 5000.0, 0.0, False, ""), # the failsafe value
(1234.0, 5000.0, 1234.0, False, ""),
(14768.0, 5000.0, 5000.0, True, "max_w"), # 10.3's reported runaway
]
for requested, limit, want_applied, want_engaged, want_bound in TABLE:
r = clamp_to_rating(requested, limit)
check(f"{requested:+.0f} W against a {limit:.0f} W rating -> "
f"{r.applied_w:+.0f} W, engaged={r.engaged}, bound={r.bound!r}",
r.applied_w == want_applied and r.engaged is want_engaged
and r.bound == want_bound)
# It is a MAGNITUDE clamp (3.2), so import and export must be bound identically.
# Derived over a sweep rather than asserted on two hand-picked values.
asym = [m for m in range(0, 12001, 37)
if clamp_to_rating(m, LIVE_MAX_W).applied_w
!= -clamp_to_rating(-m, LIVE_MAX_W).applied_w]
check(f"symmetric over {len(range(0, 12001, 37))} magnitudes "
f"({len(asym)} asymmetric)", not asym)
# The engagement flag has to mean "the RAW value differed", which is the words
# AC 2 uses. Re-derived from the result's own fields, not from the table.
mismatch = [m for m in range(0, 12001, 37)
for r in [clamp_to_rating(m, LIVE_MAX_W)]
if r.engaged != (r.requested_w != r.applied_w)]
check("engaged is exactly 'raw differs from applied', over the same sweep",
not mismatch)
# The device second stage. Only reachable when max_w is ABOVE the device range,
# i.e. when validate_startup_config() did not run or no longer holds.
r = clamp_to_rating(3000.0, 5000.0, dev_min=-2000.0, dev_max=2000.0)
check(f"a device narrower than max_w still bites, and says which bound "
f"({r.applied_w:+.0f} W, {r.bound!r})",
r.applied_w == 2000.0 and r.engaged and r.bound == "device")
r = clamp_to_rating(3000.0, 5000.0, dev_min=-5000.0, dev_max=5000.0)
check("...and is a no-op once the startup validation holds",
r.applied_w == 3000.0 and not r.engaged)
# ⚠️ REGRESSION, REVIEW-FOUND. THE FIRST STAGE USED TO `return` AND THE DEVICE
# STAGE WAS THEN NEVER REACHED, so the harder the overshoot the less the guard
# did: at max_w 5000 against a +-4000 device, +9999 emitted +5000 - OUTSIDE THE
# DEVICE RANGE - while a mere +4500 was correctly held to +4000. That inverts
# the guard and it was a regression against the pre-PR `write_setpoint()`, whose
# comment read "never send a value the device will reject outright".
#
# The property is asserted over a sweep, not on the two witness values, because
# the defect was invisible to every single-value check that happened to sit
# inside one stage.
DEV_LO, DEV_HI = -4000.0, 4000.0
escapes = [(m, clamp_to_rating(m, 5000.0, DEV_LO, DEV_HI).applied_w)
for m in range(-15000, 15001, 61)
for a in [clamp_to_rating(m, 5000.0, DEV_LO, DEV_HI).applied_w]
if not (DEV_LO <= a <= DEV_HI)]
big = clamp_to_rating(9999.0, 5000.0, DEV_LO, DEV_HI)
small = clamp_to_rating(4500.0, 5000.0, DEV_LO, DEV_HI)
print(f" max_w 5000 vs a device of {DEV_LO:+.0f}..{DEV_HI:+.0f}: "
f"+9999 -> {big.applied_w:+.0f} W ({big.bound}), "
f"+4500 -> {small.applied_w:+.0f} W ({small.bound})")
check(f"BOTH stages always run - {big.requested_w:+.0f} W emits "
f"{big.applied_w:+.0f} W, inside the device's {DEV_HI:+.0f} W, and names "
f"both bounds ({big.bound!r})",
big.applied_w == DEV_HI and big.bound == "max_w+device")
check(f"...and it holds over the whole range, so no overshoot escapes the "
f"device ({len(escapes)} escapes in {len(range(-15000, 15001, 61))})",
not escapes)
check("...a big overshoot is not clamped LESS than a small one",
abs(big.applied_w) <= abs(small.applied_w) + 0.0)
print(" non-finite input is loss of input, not an out-of-range value")
for bad in (None, float("nan"), float("inf"), float("-inf"), "abc", object()):
check(f"is_commandable({bad!r:.30}) is False", is_commandable(bad) is False)
for good in (0, -5000.0, 5000, 12.5):
check(f"is_commandable({good!r}) is True", is_commandable(good) is True)
# ⚠️ WHY THAT GUARD EXISTS, AND THE ANSWER IS WORSE THAN "it passes a nan
# through". MEASURED HERE, not recalled: every comparison against a nan is
# False, so `min(limit, nan)` keeps its FIRST argument and a naive clamp turns a
# nan into a command at the FULL RATING - and then reports engaged=True, because
# limit != nan, so the winding-loop counter gets a transport bug filed in it.
# The number below is computed by the same expression the guard replaces.
nan = float("nan")
naive = max(-LIVE_MAX_W, min(LIVE_MAX_W, nan))
check(f"a naive clamp turns a nan into {naive:+,.0f} W and calls it "
f"engaged={naive != nan} - hence the guard",
naive == LIVE_MAX_W and (naive != nan) is True)
# ==============================================================================
print()
print("2. the alarm's cadence: every engagement counted, no flood")
wd = ClampWatchdog()
t = T0
fired = [wd.note(True, "winding", t + timedelta(seconds=i)) for i in range(10)]
check(f"the rising edge fires ({fired[0]})", fired[0] is True)
check(f"the next 9 one-second cycles fire {sum(fired[1:])} more times "
f"(re-assert is {REASSERT_S:.0f} s)", sum(fired[1:]) == 0)
check(f"but every one of them is counted: engagements={wd.engagements}",
wd.engagements == 10)
check(f"and it is one episode, not ten: episodes={wd.episodes}",
wd.episodes == 1)
# --- THE RE-ASSERT CADENCE, PINNED IN BOTH DIRECTIONS ------------------------
# ⚠️ REVIEW-FOUND. This used to read `wd.note(True, ..., T0 + REASSERT_S + 9)`,
# which is green for ANY value of the constant - REASSERT_S = 3600 passed, and an
# hour of silence is precisely what the module promises not to do. A check whose
# stimulus is written in terms of the constant it is grading cannot grade it.
#
# The requirements are stated in wall-clock seconds and derived from what the
# alarm is FOR, so they do not move when the constant does:
#
# floor - a loop pinned on the rail must speak at least MIN_PER_QUARTER times
# inside the 900 s quarter that bills the capacity tariff. That window
# is the same one freeze.py and DOCS.md reason about.
# ceiling - it must not speak on more than MAX_FRACTION of cycles, or it is
# the per-cycle flood the split exists to remove.
QUARTER_S = 900.0
MIN_PER_QUARTER = 5
MAX_FRACTION = 0.20
pinned = ClampWatchdog()
alarms = sum(1 for i in range(int(QUARTER_S))
if pinned.note(True, "pinned", T0 + timedelta(seconds=i)))
print(f" pinned on the rail for {QUARTER_S:.0f} s at 1 s cycles: "
f"{pinned.engagements} engagements, {alarms} alarm(s) "
f"({alarms / QUARTER_S:.1%} of cycles)")
check(f"a loop pinned for a whole billing quarter speaks at least "
f"{MIN_PER_QUARTER} times (it spoke {alarms})",
alarms >= MIN_PER_QUARTER)
check(f"...and on no more than {MAX_FRACTION:.0%} of cycles (it was "
f"{alarms / QUARTER_S:.1%})", alarms <= MAX_FRACTION * QUARTER_S)
check("...while every single cycle is still counted",
pinned.engagements == int(QUARTER_S))
# --- THE HYSTERESIS, PINNED AGAINST THE CHATTER DIRECTION --------------------
# ⚠️ REVIEW-FOUND. This used to issue `range(CLEAR_CYCLES - 1)` clean cycles,
# which at CLEAR_CYCLES = 1 issues ZERO of them and passes while printing
# "0 clean cycles do NOT clear it". Green for every value >= 1 - including 1,
# which IS the chatter the constant exists to prevent. The stimulus is a literal
# now: ONE clean cycle must not clear the alarm, whatever the constant says.
wd2 = ClampWatchdog()
wd2.note(True, "winding", T0)
wd2.note(False, "", T0 + timedelta(seconds=1))
check(f"ONE clean cycle does NOT clear the alarm (active={wd2.active}) - "
f"a command dithering across the rail must not chatter",
wd2.active is True)
survived = 1
while wd2.active and survived < 50:
wd2.note(False, "", T0 + timedelta(seconds=1 + survived))
survived += 1
print(f" it took {survived} consecutive clean cycles to de-assert "
f"(CLEAR_CYCLES = {CLEAR_CYCLES})")
check(f"...and it does de-assert, after {survived} of them, rather than "
f"latching", wd2.active is False and survived == CLEAR_CYCLES)
check("...and the engagement count survives the clear, so the evidence does",
wd2.engagements == 1)
check("a clean cycle on an idle watchdog says nothing",
wd2.note(False, "", T0 + timedelta(seconds=300)) is False)
# ==============================================================================
print()
print("3. AC 1 + AC 3, wired: over the rating both clamps AND alarms")
class _Store:
data = {}
def set(self, *a):
pass
def get_time(self, *a):
return None
class _Pub:
enabled = False
def publish(self, *a, **k):
pass
def close(self):
pass
class _Hass:
"""Stub HA. `track` False models an inverter that is switched off entirely,
which is the 10.3 runaway's own setup."""
def __init__(self, grid, batt=0.0, track=True):
self.grid = grid
self.batt = batt
self.track = track
self.calls = []
self.writes = []
async def limits(self, entity):
return (-LIVE_MAX_W, LIVE_MAX_W, 1.0)
async def number(self, entity, invert=False):
return {"sensor.grid": self.grid, "sensor.soc": 50.0,
"sensor.batt": self.batt}.get(entity)
async def set_number(self, entity, value):
self.writes.append(float(value))
if self.track:
self.batt = float(value)
return True
async def call(self, domain, service, data):
self.calls.append((domain, service, data))
return True
def wired(grid, cycles=40, batt=0.0, track=True, **over):
async def go():
hass = _Hass(grid, batt=batt, track=track)
c = Controller(dict(
{"setpoint_entity": "number.sp", "meter_entity": "sensor.grid",
"soc_entity": "sensor.soc", "batt_entity": "sensor.batt",
"meter_source": "off", "max_w": LIVE_MAX_W, "gain": 0.6,
"slew_w": 1000, "deadband_w": 15, "target_grid_w": 0,
"step_w": 10, "heartbeat_s": 10, "stale_input_s": 15},
**over), hass, _Store(), _Pub(), stop=asyncio.Event())
c.auto = True
c.dev_min, c.dev_max = -LIVE_MAX_W, LIVE_MAX_W
# The red status row exists only WHILE the clamp is engaged, and the
# hysteresis clears it a few cycles later - so it is captured live
# rather than looked for after the run, when it is correctly gone.
c.red_snapshot = []
for _ in range(cycles):
await c.read_inputs()
await c.cycle()
if c.clamp_wd.active and not c.red_snapshot:
c.red_snapshot = c.checks()
return hass, c
return asyncio.run(go())
def clamp_notes(hass):
return [d for dom, svc, d in hass.calls
if (dom, svc) == ("persistent_notification", "create")
and d.get("notification_id") == CLAMP_ID]
# THE 10.3 RUNAWAY, replayed. "With the inverter switched off entirely, an
# unclamped integrator climbed ~130 W per 4 s to over 10 kW and, pinned at the
# 5 kW rail, reported 14 768 W." `track=False` is the inverter switched off: it
# reports 0 W however hard it is commanded, so the error never closes.
hass_r, ctl_r = wired(6000.0, cycles=60, track=False)
emitted = [abs(w) for w in hass_r.writes]
# What the SAME error would have done to an accumulator with no bound - the
# transcription is deliberately the unclamped velocity form 10.3 describes, so
# the figure below is computed here and not quoted from the spec.
runaway, err = 0.0, 6000.0
for _ in range(60):
runaway += 0.6 * err
print(f" unclamped, the same 60 cycles reach {runaway:,.0f} W; "
f"the largest value that reached the device is {max(emitted):,.0f} W")
check(f"an unbounded accumulator really does run away here "
f"({runaway:,.0f} W >> {LIVE_MAX_W:.0f} W)", runaway > 3 * LIVE_MAX_W)
check(f"AC 1: NOT ONE of the {len(emitted)} values that reached the device "
f"exceeds the {LIVE_MAX_W:.0f} W rating",
emitted and max(emitted) <= LIVE_MAX_W)
# ⚠️ AND THE CLAMP IS NOT THE ONLY THING HOLDING THAT LINE HERE, which is worth
# saying out loud rather than letting the number above imply otherwise. Against
# an inverter reporting 0 W the saturation freeze engages after
# `saturation_cycles` and pins the command well BELOW the rail - that is why the
# largest emitted value is what it is and not 5000 W. The clamp is what makes it
# bounded; the freeze is what makes it small. A check that read
# `max(emitted) == max_w` would have been asserting the freeze, not the clamp.
check(f"...and the saturation freeze is what pins it at {max(emitted):,.0f} W "
f"rather than the rail", ctl_r.sat_count >= ctl_r.tuning.saturation_cycles)
notes_r = clamp_notes(hass_r)
print(f" the loop clamped on {ctl_r.clamp_wd.engagements} of {60} cycles, "
f"in {ctl_r.clamp_wd.episodes} episode(s), raising {len(notes_r)} "
f"notification(s)")
check("AC 3: and it ALARMED - silently clipping this is the defect",
ctl_r.clamp_wd.engagements > 0 and len(notes_r) >= 1)
check("...with its own notification_id, not shared with any other guard",
all(d["notification_id"] == CLAMP_ID for d in notes_r))
check("...one episode, re-asserted rather than one per cycle",
ctl_r.clamp_wd.episodes == 1 and len(notes_r) < ctl_r.clamp_wd.engagements)
check("...and it names itself a bug signal, not a limit in use",
bool(notes_r) and "out of range" in notes_r[0]["message"])
rows_r = [e for e in ctl_r.events if REASON_CODE in e]
check(f"the action log records every engagement, coalesced into one row per "
f"episode ({len(rows_r)} row(s), {ctl_r.clamp_wd.engagements} "
f"engagement(s))",
len(rows_r) == ctl_r.clamp_wd.episodes
and f"{ctl_r.clamp_wd.engagements} engagement(s)" in rows_r[0])
# ⚠️ REGRESSION, AND IT WAS A REAL BUG BEFORE IT WAS A CHECK. `log_event()` keeps
# 40 rows. A row per engagement flushes that ring inside 40 cycles of a winding
# loop, and the first thing it evicted was SAFETY-18's own "METER FROZEN?" row -
# i.e. this guard silencing another guard's log. Driven here with a plant that
# TRACKS the command, so the saturation freeze never engages and the integrator
# clamps on essentially every cycle.
hass_l, ctl_l = wired(9000.0, cycles=80, track=True)
rows_l = [e for e in ctl_l.events if REASON_CODE in e]
other_l = [e for e in ctl_l.events if REASON_CODE not in e]
print(f" 80 winding cycles: {ctl_l.clamp_wd.engagements} engagement(s) -> "
f"{len(rows_l)} log row(s), leaving room for {len(other_l)} other event(s)")
check(f"a long winding episode does NOT flood the 40-row event log "
f"({len(rows_l)} rows for {ctl_l.clamp_wd.engagements} engagements)",
ctl_l.clamp_wd.engagements > 40 and len(rows_l) <= ctl_l.clamp_wd.episodes)
check("...and the count in the row is the counter's, so nothing is lost",
f"{ctl_l.clamp_wd.engagements} engagement(s)" in rows_l[-1])
# The RED row exists only while engaged, and the episode above is short, so it
# is captured mid-run rather than asserted after the hysteresis has cleared it.
check("the status page goes red WHILE it is engaged",
any(not c["ok"] and not c["warn"] and "OUTPUT CLAMP ENGAGED" in c["text"]
for c in ctl_r.red_snapshot))
check("...and once clear, a warn row still says how many times it engaged",
any(c["warn"] and f"{ctl_r.clamp_wd.engagements} time(s)" in c["text"]
for c in ctl_r.checks()))
# ⚠️ NOT A TRIP. 11.2's action is "clamp and alarm"; the loop keeps running and
# keeps writing the clamped value. If a later cut changes that it must come here.
check(f"it is NOT a trip - the command stands at {ctl_r.target:+.0f} W, not 0 W",
ctl_r.target != 0.0 and ctl_r.target == hass_r.writes[-1])
check("...and the controller still reports itself running, not failsafe",
ctl_r.auto and not ctl_r.sign_wd.tripped)
# The export side, because the clamp is a magnitude clamp and a sign bug in it
# would be invisible on the import case alone.
hass_x, ctl_x = wired(-6000.0, cycles=60, track=False)
check(f"the export side clamps and alarms too "
f"({min(hass_x.writes):+,.0f} W, {len(clamp_notes(hass_x))} note(s), "
f"{ctl_x.clamp_wd.engagements} engagement(s))",
min(hass_x.writes) < 0 and abs(min(hass_x.writes)) <= LIVE_MAX_W
and clamp_notes(hass_x) and ctl_x.clamp_wd.engagements > 0)
# ==============================================================================
print()
print("4. AC 2: saturation WITHOUT clamping must NOT raise a clamp alarm")
# A legal command the inverter will not follow. The meter sits just outside the
# deadband so the loop moves a little and then rests; `batt` is pinned far from
# whatever is commanded, which is the saturation condition (|cmd - actual| >
# saturation_w) and nothing else. The integrator never approaches the rating.
hass_s, ctl_s = wired(60.0, cycles=40, batt=2500.0, track=False)
peak = max(abs(w) for w in hass_s.writes)
print(f" saturated for {ctl_s.sat_count} consecutive cycles; the command "
f"peaked at {peak:.0f} W, {peak / LIVE_MAX_W:.1%} of the rating")
# ⚠️ THE POSITIVE HALF FIRST. A negative check that saw nothing happen is not an
# observation - if the saturation freeze never engaged, "no clamp alarm" is
# vacuous and this check is worthless.
check(f"the saturation freeze REALLY DID engage (sat_count="
f"{ctl_s.sat_count} >= {ctl_s.tuning.saturation_cycles})",
ctl_s.sat_count >= ctl_s.tuning.saturation_cycles)
check("...and the event log says so, in the saturation's own words",
any("saturation freeze" in e for e in ctl_s.events))
check(f"...while the command stayed well inside the rating "
f"({peak:.0f} W < {LIVE_MAX_W:.0f} W)", peak < LIVE_MAX_W)
# Now the negative, which is only worth anything because of the three above.
check("AC 2: NO clamp notification was raised", not clamp_notes(hass_s))
check("AC 2: the clamp counted ZERO engagements", ctl_s.clamp_wd.engagements == 0)
check("AC 2: no OUTPUT_CLAMP row in the action log",
not [e for e in ctl_s.events if REASON_CODE in e])
check("AC 2: and no clamp row on the status page",
not [c for c in ctl_s.checks() if "output clamp" in c["text"].lower()])
print(" ...and the two alarms do not share a notification_id")
ids_clamp = {d["notification_id"] for d in clamp_notes(hass_r)}
ids_other = {d.get("notification_id") for dom, svc, d in hass_r.calls
if svc == "create"} - ids_clamp
check(f"clamp ids {sorted(ids_clamp)} are disjoint from every other alarm's "
f"{sorted(x for x in ids_other if x)}", not (ids_clamp & ids_other))
print(" ...and the alarm cannot be keyed on Decision.reason")
# A cycle that clamps the integrator AND is then slew limited. `reason` reports
# the LAST stage, so a string-keyed alarm is silent on it; the boolean is not.
t = Tuning(max_w=2000.0, slew_w=100.0, gain=0.6)
d = compute(prev_w=1000.0, grid_w=6000.0, actual_w=1000.0, tuning=t, i_w=1900.0)
print(f" reason={d.reason!r} i_clamped={d.i_clamped} clamped={d.clamped}")
check("the integrator DID clamp on that cycle", d.i_clamped is True)
check("...but `reason` says 'slew-limited' - the clamp was overwritten "
"downstream, so a string-keyed alarm is SILENT here",
d.reason == "slew-limited" and "clamp" not in d.reason)
# And the same for the OUTPUT clamp, on a config that lets the integrator run
# above the rail - then both flags fire and `reason` still reports neither.
t2 = Tuning(max_w=2000.0, integrator_max_w=3000.0, slew_w=100.0, gain=0.6)
d2 = compute(prev_w=1000.0, grid_w=6000.0, actual_w=1000.0, tuning=t2, i_w=2900.0)
print(f" reason={d2.reason!r} i_clamped={d2.i_clamped} clamped={d2.clamped}")
check("both clamps engaged, and `reason` reports neither",
d2.i_clamped and d2.clamped and "clamp" not in d2.reason)
# ⚠️ HOW THE FREEZE AND THE CLAMP INTERACT, MEASURED RATHER THAN ASSUMED, AND
# THE MEASUREMENT REFUSED TWO GUESSES BEFORE THIS ONE. Written first as "a
# frozen cycle never clamps" (false), then as "only the i_w == 0 reseed
# exception clamps" (also false - at i_w == 0 with a matching actual the cycle
# is not saturated at all, so it is never frozen). What the sweep actually
# shows: a frozen cycle clamps exactly when the integrator REVERSES. control.py
# permits that on purpose - "it may fall, cross zero, or reverse outright" -
# because the alternative deadlocks the loop, and a reversal is a full swing
# that can clear the bound on the far side in one step.
#
# That matters for this ticket rather than being trivia: it is the one shape
# where the saturation freeze and a clamp engagement coincide, so it is exactly
# where an alarm that shared a channel with saturation would lose the clamp.
grid_sweep = (-9000.0, -3000.0, 3000.0, 9000.0)
grid = [(pp, g) for pp in (-1900.0, -500.0, 0.0, 500.0, 1900.0)
for g in grid_sweep]
runs = {(pp, g): compute(pp, g, 0.0, t, sat_count=5, i_w=pp) for pp, g in grid}
frozen = [k for k, d in runs.items() if d.frozen]
frozen_clamps = [k for k in frozen if runs[k].i_clamped]
# The property, stated independently of the control law's own branches: the
# correction pushes AGAINST the standing integrator, and the resulting swing
# clears the bound.
reversals = [(pp, g) for pp, g in frozen
if pp * g < 0 and abs(pp + t.gain * g) > t.max_w]
print(f" {len(frozen)} of {len(grid)} sweep cells are frozen; "
f"those that also clamp: {frozen_clamps}")
check(f"a frozen cycle CAN clamp, and only by reversing "
f"({len(frozen_clamps)} of {len(frozen)} frozen cells)",
frozen_clamps and frozen_clamps == reversals)
check("...and a frozen cycle pushing the way it already leans never clamps - "
"that is the freeze doing its job",
not [k for k in frozen if runs[k].i_clamped and k[0] * k[1] > 0])
check("a healthy cycle sets neither flag",
not any([compute(0.0, 100.0, 0.0, t).i_clamped,
compute(0.0, 100.0, 0.0, t).clamped]))
# ==============================================================================
print()
print("5. no path reaches the wire around the clamp")
async def _direct():
hass = _Hass(0.0)
c = Controller({"setpoint_entity": "number.sp", "max_w": 2000,
"meter_source": "off"},
hass, _Store(), _Pub(), stop=asyncio.Event())
c.dev_min, c.dev_max = -LIVE_MAX_W, LIVE_MAX_W
# The shape the heartbeat, shutdown() and handle_action() all use.
await c.write_setpoint(9999.0, force=True)
await c.write_setpoint(-9999.0, force=True)
await c.write_setpoint(float("nan"), force=True)
await c.write_setpoint(1500.0, force=True)
return hass, c
hass_d, ctl_d = asyncio.run(_direct())
print(f" asked for 9999 / -9999 / nan / 1500 W; the device saw "
f"{hass_d.writes}")
check("a direct out-of-range write is clamped, not passed through",
hass_d.writes[:2] == [2000.0, -2000.0])
check("a nan is refused and becomes 0 W, per 11.1 rule 1",
hass_d.writes[2] == 0.0)
check("...and is NOT counted as a clamp engagement - it is a transport bug",
sum(1 for e in ctl_d.events if REASON_CODE in e) == 2)
check("an in-range value is untouched", hass_d.writes[3] == 1500.0)
check("every emitted frame satisfies |emitted| <= the rating",
all(abs(w) <= 2000.0 for w in hass_d.writes))
print()
if fails:
print(f"{len(fails)} FAILED: {', '.join(fails)}")
sys.exit(1)
print("all checks passed")
+510 -37
View File
@@ -2,7 +2,10 @@
python3 test_freeze.py
Three parts, and the last two are the ones that matter:
Six parts. Parts 2 and 3 are the acceptance argument; parts 5 and 6 exist
because TEST-06 found three mutations that survived parts 1-4 green - the
evaluation POSITION inside cycle(), the NOT COVERED status string, and the
shipped `meter_freeze_s` itself:
1. the predicate's boundaries and the edge-trigger, table-driven;
2. SILENT ON REAL CONVERGED DATA. `scenarios/p1-age-2026-08-26.jsonl.gz` is
@@ -18,6 +21,14 @@ Three parts, and the last two are the ones that matter:
repo is next to this one (or `$GOODWE_SIM`), and an equivalent local stub
otherwise. Both are exercised; the run says which one it used.
4. THE WIRING: the real `Controller.cycle()`, asserted on the service call
that leaves the process;
5. WHERE IN `cycle()` the detector is evaluated - by consequence, not by
source order. It runs after `write_setpoint()`, and that is what makes two
compositions safe (see the section for both);
6. WHAT THE STATUS PAGE SAYS about an uncovered transport, and the shipped
threshold read back out of `config.yaml`.
⚠️ Part 3 runs in WALL CLOCK - `unchanged_s` is `time.monotonic()` and the
transport really polls - so it drives the detector at `hold_s=10`, not the
shipped 300. The THRESHOLD is config; the MECHANISM is what is under test, and
@@ -57,6 +68,7 @@ import asyncio
import gzip
import json
import os
import re
import subprocess
import sys
import time
@@ -79,6 +91,29 @@ HERE = Path(__file__).resolve().parent
# The live install's numbers, which are the ones the exposure is written at.
LIVE_MAX_W = 5000.0
# --------------------------------------------------------------------------- #
# THE SHIPPED THRESHOLD, READ FROM config.yaml (TEST-06 / AC3)
# --------------------------------------------------------------------------- #
# ⚠️ `meter_freeze_s: 300` used to appear in this file only as the literal 300,
# retyped six times, so `300 -> 3600` in config.yaml left every check here
# green. Everything below is now driven off the SHIPPED value, and the value
# itself is pinned in test_validate.py the way MAINT-04 pins the other defaults.
#
# Regex, not PyYAML: the add-on image installs py3-aiohttp/py3-paho-mqtt only,
# and a check that skips on ImportError is not a check.
_CFG_TXT = (HERE / "config.yaml").read_text(encoding="utf-8")
def _cfg_default(key):
"""The single ` key: <number>` default line in config.yaml's options block."""
found = re.findall(rf"^ {key}: (-?\d+(?:\.\d+)?)\s*$", _CFG_TXT, re.M)
return float(found[0]) if len(found) == 1 else None
SHIPPED_HOLD_S = _cfg_default("meter_freeze_s")
SHIPPED_DEADBAND_W = _cfg_default("deadband_w")
SHIPPED_MAX_W = _cfg_default("max_w")
def check(name, cond):
print(f" {'ok ' if cond else 'FAIL'} {name}")
@@ -135,6 +170,35 @@ check("the reason names both terms",
"unchanged over 300" in wd.reason and "command over 4750" in wd.reason)
# --------------------------------------------------------------------------- #
# 1b. the SHIPPED threshold, and the two properties it has to have (AC3)
# --------------------------------------------------------------------------- #
# ⚠️ This is the number the whole ticket is an argument about, and it was pinned
# nowhere: `meter_freeze_s: 300 -> 3600` used to leave this file and
# test_validate.py both green. test_validate.py pins the literal in MAINT-04's
# manner; here it is pinned by its CONSEQUENCES, so a schema-legal value that
# gives up the argument goes red even though it is a legal value.
print("the shipped meter_freeze_s (AC3)")
check(f"config.yaml's numeric defaults were read (freeze {SHIPPED_HOLD_S}, "
f"deadband {SHIPPED_DEADBAND_W}, max_w {SHIPPED_MAX_W})",
None not in (SHIPPED_HOLD_S, SHIPPED_DEADBAND_W, SHIPPED_MAX_W))
check("the code default and the shipped default are the same number - "
"FreezeWatchdogConfig() is a default, config.yaml is the ship",
FreezeWatchdogConfig().hold_s == SHIPPED_HOLD_S)
check("this file's `tuning()` really is config.yaml's deadband, so the "
"mutual-exclusion arithmetic below is about the shipped law",
tuning().deadband_w == SHIPPED_DEADBAND_W)
# The upper bound is money: config.yaml's own comment says 300 s is "still
# inside the 900 s quarter that bills the capacity tariff". A hold that outlives
# the quarter notifies after the peak is already billed.
QUARTER_S = 900.0
check(f"the alarm still lands inside the {QUARTER_S:.0f} s billing quarter "
f"({SHIPPED_HOLD_S:.0f} s hold)", SHIPPED_HOLD_S < QUARTER_S)
# The lower bound is the conjunction itself - see the derivation below, which
# now runs off the shipped numbers rather than off two retyped literals.
# --------------------------------------------------------------------------- #
# 2. silent on 40 minutes of real converged behaviour
# --------------------------------------------------------------------------- #
@@ -183,10 +247,10 @@ def replay(hold_s, max_w, cycle_floor_s=15.0):
for mw in (2000.0, LIVE_MAX_W):
wd_r, peak_cmd, peak_unchanged, fired = replay(300.0, mw)
wd_r, peak_cmd, peak_unchanged, fired = replay(SHIPPED_HOLD_S, mw)
print(f" max_w {mw:.0f}: worst |command| {peak_cmd:.0f} W (rail "
f"{rail_w(mw):.0f} W), worst unchanged {peak_unchanged:.1f} s "
f"(hold 300 s), episodes {wd_r.episodes}")
f"(hold {SHIPPED_HOLD_S:.0f} s), episodes {wd_r.episodes}")
check(f"max_w {mw:.0f}: never alarms on the converged capture", fired is None)
check(f"max_w {mw:.0f}: and the rail term had margin to spare",
peak_cmd < rail_w(mw))
@@ -194,7 +258,7 @@ for mw in (2000.0, LIVE_MAX_W):
# The margin on the OTHER term, as numbers rather than as trust. Both figures
# are quoted in config.yaml and freeze.py, so they are re-measured every run and
# a capture swapped underneath them cannot leave the prose saying the old thing.
_, _, worst_unchanged, _ = replay(300.0, LIVE_MAX_W)
_, _, worst_unchanged, _ = replay(SHIPPED_HOLD_S, LIVE_MAX_W)
gaps, last, prev = [], rows[0]["t"], None
for r in rows:
if prev is None or r["pwr"] != prev:
@@ -208,7 +272,8 @@ check("worst observed unchanged_s is the 8 s the docs quote",
7.5 < worst_unchanged < 8.5)
check("longest distinct-reading interval is the 15.4 s the docs quote",
15.0 < max(gaps) < 16.0)
check("300 s is at least 15x the worst of the two", 300.0 >= 15 * max(gaps))
check(f"the shipped {SHIPPED_HOLD_S:.0f} s is at least 15x the worst of the two",
SHIPPED_HOLD_S >= 15 * max(gaps))
# --------------------------------------------------------------------------- #
# 2b. TEL-05's case, reconstructed, because the capture cannot contain it
@@ -226,22 +291,26 @@ check("300 s is at least 15x the worst of the two", 300.0 >= 15 * max(gaps))
# -14.2 W) rather than invented: one hour of a perfectly healthy converged meter
# that happens not to move.
print("TEL-05's low-power case - a converged meter that legitimately repeats")
tel05 = FreezeWatchdog(FreezeWatchdogConfig(hold_s=300))
tel05 = FreezeWatchdog(FreezeWatchdogConfig(hold_s=SHIPPED_HOLD_S))
t5 = tuning(LIVE_MAX_W)
tgt, iw, sc = 0.0, 0.0, 0
bare_would_fire = False
tel05_cmd = 0.0
for k in range(3600):
# Long enough that the REJECTED bare threshold has to get its chance even if
# someone raises meter_freeze_s - the mutation must be caught by the pins above,
# not by this case going red for an incidental reason.
TEL05_S = max(3600, int(SHIPPED_HOLD_S) + 300)
for k in range(TEL05_S):
if k % 15 == 0: # cycle_floor_s, the only trigger left
d = compute(prev_w=tgt, grid_w=-14.2, actual_w=tgt, tuning=t5,
sat_count=sc, i_w=iw)
tgt, iw, sc = d.target_w, d.i_w, d.sat_count
tel05_cmd = max(tel05_cmd, abs(tgt))
# The REJECTED design, written out so the case can be shown to be real.
bare_would_fire = bare_would_fire or float(k) >= 300
bare_would_fire = bare_would_fire or float(k) >= SHIPPED_HOLD_S
tel05.evaluate(float(k), tgt, LIVE_MAX_W, T0 + timedelta(seconds=k))
print(f" an hour at -14.2 W: worst |command| {tel05_cmd:.0f} W, "
f"unchanged_s reaches 3599 s, episodes {tel05.episodes}")
print(f" {TEL05_S / 60:.0f} min at -14.2 W: worst |command| {tel05_cmd:.0f} W, "
f"unchanged_s reaches {TEL05_S - 1} s, episodes {tel05.episodes}")
check("the bare unchanged_s threshold TEL-05 rejected WOULD fire here",
bare_would_fire)
check("the conjunction does not - this is what the rail term buys",
@@ -255,8 +324,11 @@ check("...and the command stayed nowhere near the rail",
# figure is inside the deadband, so the command there is parked at 0 W. If
# someone lowers meter_freeze_s or narrows the deadband, this is the check that
# should stop them.
DEADBAND_W = 15.0
slowest_healthy_w = 3600.0 / 300.0
DEADBAND_W = SHIPPED_DEADBAND_W # config.yaml's, not a retyped 15
slowest_healthy_w = 3600.0 / SHIPPED_HOLD_S
print(f" at the shipped {SHIPPED_HOLD_S:.0f} s hold, the slowest healthy "
f"meter that still looks unchanged is {slowest_healthy_w:.1f} W, against "
f"a {DEADBAND_W:.0f} W deadband")
check("a healthy meter slow enough to look unchanged is inside the deadband",
slowest_healthy_w < DEADBAND_W)
_park = compute(prev_w=0.0, grid_w=-slowest_healthy_w, actual_w=0.0,
@@ -277,11 +349,19 @@ check("...and the same replay DOES alarm once both terms are made reachable",
# 3. a genuinely frozen meter, over real HTTP, through the real transport
# --------------------------------------------------------------------------- #
def find_hwsim():
"""The rig's own fake meter, if the infra repo is reachable."""
"""The rig's own fake meter, if the infra repo is reachable.
⚠️ `HERE.parents[1]` only exists when this package sits at least two levels
below the filesystem root - it raised IndexError and took the WHOLE suite
down on a checkout at `C:/goodwe_controller`, which is a failure of the
lookup, not of the thing being tested. Guarded, so a shallow checkout falls
through to the stub like any other missing sibling repo.
"""
cands = []
if os.environ.get("GOODWE_SIM"):
cands.append(Path(os.environ["GOODWE_SIM"]) / "hwsim.py")
cands.append(HERE.parents[1] / "goodwe" / "sim" / "hwsim.py")
if len(HERE.parents) > 1:
cands.append(HERE.parents[1] / "goodwe" / "sim" / "hwsim.py")
for c in cands:
if c.exists():
return c
@@ -379,7 +459,15 @@ def with_server(argv, port, fn):
hw = find_hwsim()
print(f"frozen meter over HTTP - using {'the rig hwsim at ' + str(hw) if hw else 'the local stub'}")
# ⚠️ WHICH TRANSPORT BACKED PART 3 IS PART OF THE RESULT, and it is repeated in
# the summary at the end of the run. The two are not equivalent: pointed at the
# real `sim/hwsim.py` the rig path failed 1 run in 3 (a 16 s window against a
# 10 s hold, when hwsim.serve() is slow to start), while the stub is tuned never
# to fail. A green log that does not say which one ran is a green log that can
# have quietly substituted the weaker stand-in - the same shape as an assertion
# that cannot fail, from the other end.
PART3_TRANSPORT = f"the rig hwsim at {hw}" if hw else "the local stub"
print(f"frozen meter over HTTP - using {PART3_TRANSPORT}")
if hw:
def argv_for(mode):
return [HWSIM_DRIVER, "8791" if mode == "freeze" else "8792", mode,
@@ -454,11 +542,17 @@ class _Hass:
# runaway under test never happens - which would have made this a check
# that passes for the wrong reason.
self.batt = 0.0
# ⚠️ The dead-poll-task case (AC1's second half): every input reads
# None, exactly as it does when the transport task has died while the
# controller keeps cycling.
self.dead = False
async def limits(self, entity):
return (-5000.0, 5000.0, 1.0)
async def number(self, entity, invert=False):
if self.dead:
return None
return {"sensor.grid": self.grid, "sensor.soc": 50.0,
"sensor.batt": self.batt}.get(entity)
@@ -479,29 +573,113 @@ class _Source:
self.unchanged_s = unchanged_s
# ⚠️ meter_freeze_s is deliberately NOT in here. A controller built with the key
# absent must land on the shipped number by itself, and that is asserted below -
# supplying it here would test this dict instead of the shipped default.
STALE_INPUT_S = 15.0
def _opts(**over):
o = {"setpoint_entity": "number.sp", "meter_entity": "sensor.grid",
"soc_entity": "sensor.soc", "batt_entity": "sensor.batt",
"meter_source": "off", "max_w": LIVE_MAX_W, "gain": 0.6,
"slew_w": 1000, "deadband_w": SHIPPED_DEADBAND_W,
"target_grid_w": -10, "step_w": 10, "heartbeat_s": 10,
"stale_input_s": STALE_INPUT_S}
o.update(over)
return o
# ⚠️ TWO SURFACES, AND EVERY ASSERTION BELOW SAYS WHICH ONE IT IS ABOUT.
# The first cut of this helper selected by SERVICE, which measured "any guard's
# notification" while every assertion built on it was named for THIS guard's.
# That was true only because `check_meter_frozen()` was the sole
# `persistent_notification.create` in `app/`; SAFETY-03 (#12) adds a second one
# (`goodwe_output_clamp`) raised by scenarios that rail the command on purpose,
# and eight assertions here went red against it - AC1's core check included.
#
# Blanket id-keying is NOT the repair. Three of those eight are claims about the
# WHOLE notification surface, and narrowing them to this guard's id would have
# silently shrunk a claim rather than fixed a conflation - which is the defect
# class this card exists to close. So both surfaces are available by name, and
# each assertion picks one deliberately.
FREEZE_NOTE_ID = "goodwe_meter_frozen"
def all_notes_of(hass):
"""EVERY HA persistent notification raised this run, whoever raised it."""
return [d for dom, svc, d in hass.calls
if (dom, svc) == ("persistent_notification", "create")]
def _is_freeze_note(d):
"""Is this notification the freeze detector's?
⚠️ THE ID **OR** THE TITLE, and the `or` is measured rather than tidy.
Keyed on the id alone, renaming the id empties every list in this file and
SIX assertions collapse together - so the one check whose subject IS the id
could no longer fail on its own, in the file about assertions that cannot
fail. Recognising the guard by what it says as well leaves the rename
exactly one red: the id's own check. Measured both ways:
selector vs #12's app/ id renamed
by service (the first cut) 8 red 1 red
by id alone 3 red 6 red
by id or title (this) 3 red 1 red
The union cannot re-admit another guard's alarm unless that guard also
calls itself frozen, which would be a confusable alarm on its own account.
"""
return (d.get("notification_id") == FREEZE_NOTE_ID
or "frozen" in str(d.get("title", "")).lower())
def notes_of(hass):
"""Only THIS guard's - see _is_freeze_note()."""
return [d for d in all_notes_of(hass) if _is_freeze_note(d)]
def notes_by_text_of(hass):
"""This guard's, recognised by what it SAYS and never by its id.
Exists for the two assertions whose subject IS the id: selected by the id,
both would be true by construction.
"""
return [d for d in all_notes_of(hass)
if "frozen" in str(d.get("title", "")).lower()]
def wired(unchanged_s, grid, cycles=60, **over):
"""Returns (hass, controller, trace).
`trace` is one row per cycle - (command after the cycle, the watchdog's
level, notifications raised so far) - which is what AC1 needs: the
QUESTION IS NOT WHETHER THE ALARM HAPPENS BUT ON WHICH CYCLE.
"""
async def go():
hass = _Hass(grid)
c = Controller(dict(
{"setpoint_entity": "number.sp", "meter_entity": "sensor.grid",
"soc_entity": "sensor.soc", "batt_entity": "sensor.batt",
"meter_source": "off", "max_w": LIVE_MAX_W, "gain": 0.6,
"slew_w": 1000, "deadband_w": 15, "target_grid_w": -10,
"step_w": 10, "heartbeat_s": 10, "stale_input_s": 15},
**over), hass, _Store(), _Pub(), stop=asyncio.Event())
c = Controller(_opts(**over), hass, _Store(), _Pub(),
stop=asyncio.Event())
c.auto = True
c.dev_min, c.dev_max = -5000.0, 5000.0
c.p1_source = _Source(unchanged_s)
trace = []
for _ in range(cycles):
await c.read_inputs()
await c.cycle()
return hass, c
trace.append((c.target, c.freeze_wd.active, len(notes_of(hass))))
return hass, c, trace
return asyncio.run(go())
hass_f, ctl_f = wired(3600.0, 4000.0)
notes = [d for dom, svc, d in hass_f.calls
if (dom, svc) == ("persistent_notification", "create")]
# ⚠️ SAFETY-03 (#12) arrived at the same diagnosis independently and added a
# `frozen_notes()` keyed on the id. `notes_of()` above subsumes it - same id,
# plus the title fallback that keeps the id's own check independently
# falsifiable, plus `all_notes_of()` for the claims that are about the whole
# surface rather than about this guard.
hass_f, ctl_f, trace_f = wired(3600.0, 4000.0)
notes = notes_of(hass_f) # THIS guard's
all_notes = all_notes_of(hass_f) # the whole surface
# The status row lives behind `p1_enabled`, which is what a homewizard_local
# install has. Flipped here rather than plumbing a whole P1 transport in - the
# row's own content is what is under test.
@@ -509,9 +687,28 @@ ctl_f.p1_enabled = True
rows = [c for c in ctl_f.checks() if "METER MAY BE FROZEN" in c["text"]]
print(f" frozen at +4000 W: command ran to {ctl_f.target:.0f} W, "
f"{len(notes)} notification(s), {len(rows)} red status row(s)")
check("the shipped cycle() raises exactly one HA notification", len(notes) == 1)
# SURFACE-WIDE, deliberately, and the second guard has now landed. SAFETY-03
# ships `goodwe_output_clamp`, and this scenario - a runaway loop winding into
# the rail - raises it legitimately: the loop asks for more than the rating and
# is clamped. So the answer to "is this scenario still meant to raise exactly
# one alarm" is NO, it raises exactly two, and BOTH ARE NAMED here rather than
# the claim being narrowed until it went green again. A third guard has to come
# back to this line, which is the whole point of keeping one assertion in the
# file that accounts for the controller's entire alarm output.
_ids = sorted(d.get("notification_id") for d in all_notes)
print(f" the whole notification surface this run: {_ids}")
check("the whole notification surface carried exactly two alarms this run - "
"SAFETY-18's and SAFETY-03's, both named",
_ids == ["goodwe_meter_frozen", "goodwe_output_clamp"])
# THIS GUARD'S, keyed on the id.
check("...and the freeze detector is the one that raised it", len(notes) == 1)
# ⚠️ Selected by TEXT, not by id - see notes_by_text_of(). One of the two
# assertions whose subject IS the id (the other is the stability check in part
# 5), so it must not be selected by the thing it is checking, and it has to be
# able to fail while everything else stays green.
check("...with a fixed notification_id so repeats replace, not stack",
notes and notes[0].get("notification_id") == "goodwe_meter_frozen")
[d.get("notification_id") for d in notes_by_text_of(hass_f)]
== [FREEZE_NOTE_ID])
check("...and it says the controller is still running",
bool(notes) and "STILL RUNNING" in notes[0]["message"])
check("the status page carries a failed row saying so",
@@ -522,17 +719,293 @@ check("the event log names it", any("METER FROZEN" in e for e in ctl_f.events))
check("and it does NOT command 0 W - this cut is an alarm only",
abs(ctl_f.target) >= rail_w(LIVE_MAX_W))
hass_n, ctl_n = wired(3600.0, -14.2)
check("wired up, TEL-05's converged case still raises nothing",
not [1 for dom, svc, _ in hass_n.calls if svc == "create"])
hass_m, ctl_m = wired(2.0, 8000.0)
check("wired up, a real railed load against a moving meter raises nothing",
not [1 for dom, svc, _ in hass_m.calls if svc == "create"]
and abs(ctl_m.target) >= rail_w(LIVE_MAX_W))
check("an uncovered transport (no unchanged_s) raises nothing, and says so",
not [1 for dom, svc, _ in wired(None, 4000.0)[0].calls if svc == "create"])
# ⚠️ THE THREE SCENARIOS THAT SAFETY-03 FORCED A DECISION ON, decided one at a
# time and not reflexively narrowed to green. Measured on the merged tree:
#
# scenario surface ids freeze
# frozen meter, railed command (above) 2 frozen + clamp 1
# TEL-05 converged, -14.2 W 0 - 0
# moving meter, a real 8 kW load 1 clamp 0
# uncovered transport, no unchanged_s 1 clamp 0
#
# CONVERGED stays SURFACE-WIDE. At the operating point this controller exists to
# hold, the correct behaviour is that the whole add-on has nothing to say - not
# merely that the freeze guard has nothing to say. Nothing about SAFETY-03
# weakens that claim, and it still holds, so narrowing it would give away a real
# property for nothing.
#
# The other two NARROW to notes_of(), and the reason is the same in both: the
# clamp alarm there is CORRECT and is SAFETY-03's subject, not this file's. A
# genuine 8 kW load really does drive the loop past the rating; an uncovered
# transport really does not stop it doing so. Asserting silence across the whole
# surface would make this file fail whenever a different guard correctly does
# its job - a claim about somebody else's behaviour wearing this file's name.
# What each still owns is the load-bearing half: THE FREEZE DETECTOR SAID
# NOTHING. The surface is printed rather than asserted, so a reader can still
# see what else fired without this file grading it.
hass_n, ctl_n, _ = wired(3600.0, -14.2)
check("wired up, TEL-05's converged case leaves the WHOLE SURFACE silent - at "
"the operating point the add-on should have nothing at all to say",
not all_notes_of(hass_n))
hass_m, ctl_m, _ = wired(2.0, 8000.0)
print(f" real 8 kW load, moving meter: surface "
f"{sorted(d.get('notification_id') for d in all_notes_of(hass_m))}, "
f"freeze {len(notes_of(hass_m))}")
check("wired up, a real railed load against a moving meter raises nothing FROM "
"THIS GUARD (the clamp alarm there is SAFETY-03's and is correct)",
not notes_of(hass_m) and abs(ctl_m.target) >= rail_w(LIVE_MAX_W))
hass_u = wired(None, 4000.0)[0]
print(f" uncovered transport: surface "
f"{sorted(d.get('notification_id') for d in all_notes_of(hass_u))}, "
f"freeze {len(notes_of(hass_u))}")
check("an uncovered transport (no unchanged_s) raises nothing FROM THIS GUARD, "
"which is the honest report of a detector that cannot see",
not notes_of(hass_u))
# --------------------------------------------------------------------------- #
# 5. WHERE in cycle() the detector is evaluated (TEST-06 / AC1)
# --------------------------------------------------------------------------- #
# ⚠️ WHERE THE BOUNDARY ACTUALLY IS - AND IT IS NOT write_setpoint().
# The first version of this comment said the detector runs "after
# write_setpoint()" and grades "the command that has just gone on the wire".
# TEST-06's reviewer disproved both halves by mutation:
#
# N3 move check_meter_frozen() to immediately BEFORE write_setpoint()
# -> green, and CORRECTLY green
#
# because `self.target = resolution.target_w` is assigned two lines above the
# write, `write_setpoint()` clamps only a local `value`, and it can skip the
# write entirely (`if not (changed or force): return`). So what the detector
# grades is `self.target` - the arbiter's resolved intent, PRE device clamp and
# possibly never emitted - and the invariant these checks pin is:
#
# check_meter_frozen() runs AFTER `self.target = resolution.target_w`
#
# That distinction is not pedantry. A check more robust than its own
# explanation is how the next person moves the call ABOVE the assignment
# believing they had preserved the stated rule.
#
# Both properties below are asserted by an OBSERVABLE CONSEQUENCE rather than by
# source order, so semantic-preserving refactors stay green (measured by the
# reviewer: renaming the method, and calling it from a helper that itself runs
# last, both survive - the suite drives `await c.cycle()` and never names the
# method):
#
# (a) BOTH TERMS MUST DESCRIBE THE SAME MOMENT. The command it grades has to
# be the one THIS cycle resolved, judged against how long the meter that
# chose it has stood still. Evaluated above the assignment it grades the
# PREVIOUS cycle's command against THIS cycle's meter - a pairing that was
# never true of any moment - and the alarm arrives a whole cycle late.
# (b) THE DEAD-POLL-TASK COMPOSITION. When the poll task dies the inputs go
# stale, the arbiter resolves 0 W, and the detector - evaluated after that
# resolution lands in self.target - correctly sees an unrailed command and
# CLEARS. Evaluated above the assignment it would still be looking at the
# railed command from before the failsafe, and would keep the alarm
# standing over a controller that is already commanding 0 W.
print("WHERE the detector is evaluated in cycle() - the ordering AC1 protects")
_railed = [i for i, (t, _, _) in enumerate(trace_f)
if abs(t) >= rail_w(LIVE_MAX_W)]
_alarmed = [i for i, (_, _, n) in enumerate(trace_f) if n >= 1]
_first_railed = _railed[0] if _railed else None
_first_alarm = _alarmed[0] if _alarmed else None
print(f" ramp to the rail: first railed command on cycle {_first_railed}, "
f"first notification on cycle {_first_alarm}, over {len(trace_f)} cycles")
# Without this the equality below could be satisfied on cycle 0, where "the same
# cycle" and "one cycle later" are indistinguishable.
check("the command takes several cycles to reach the rail, so 'same cycle' and "
"'one cycle later' are different observations",
_first_railed is not None and _first_railed >= 1
and len(trace_f) > _first_railed + 1)
check("the alarm lands on the SAME cycle the command reaches the rail - "
"evaluated after `self.target = resolution.target_w` it grades the "
"command THIS cycle resolved; evaluated above that assignment it grades "
"the previous one and arrives a cycle late",
_first_alarm is not None and _first_alarm == _first_railed)
def dead_poll(after_cycles=20, dead_cycles=4, revived_cycles=20):
"""Alarm on a railed command, then kill the poll task and keep cycling.
The real thing: the transport task dies, every input reads None, and after
`stale_input_s` the arbiter's P_SAFETY stop resolves 0 W. `inputs_bad_since`
is backdated rather than slept through - the clock is wall clock, and the
mechanism rather than the wait is what is under test.
"""
async def go():
hass = _Hass(4000.0)
c = Controller(_opts(), hass, _Store(), _Pub(), stop=asyncio.Event())
c.auto = True
c.dev_min, c.dev_max = -5000.0, 5000.0
c.p1_source = _Source(3600.0)
for _ in range(after_cycles):
await c.read_inputs()
await c.cycle()
before = (c.target, c.freeze_wd.active, len(notes_of(hass)))
hass.dead = True
trace = []
for _ in range(dead_cycles):
await c.read_inputs()
if c.inputs_bad_since is not None:
c.inputs_bad_since = (datetime.now(timezone.utc)
- timedelta(seconds=STALE_INPUT_S + 1))
await c.cycle()
trace.append((c.target, c.freeze_wd.active, len(notes_of(hass))))
# ...and then the transport comes back, against a meter that is still
# frozen. The loop re-rails and this is a NEW episode - the alarm is
# allowed, indeed required, to speak again.
hass.dead = False
revived = []
for _ in range(revived_cycles):
await c.read_inputs()
await c.cycle()
revived.append((c.target, c.freeze_wd.active, len(notes_of(hass))))
# ⚠️ by TEXT: "the second episode REPLACED the first" is a claim about
# the id, so the list it is checked against must not be selected by the
# id - that would make it true by construction.
return before, trace, revived, notes_by_text_of(hass), c
return asyncio.run(go())
_before, _dead, _revived, _freeze_notes, _ctl_d = dead_poll()
_zeroed = [i for i, (t, _, _) in enumerate(_dead) if t == 0.0]
print(f" poll task killed: alarming at {_before[0]:+.0f} W "
f"(active={_before[1]}, {_before[2]} notification(s)) -> "
+ ", ".join(f"cycle {i}: {t:+.0f} W active={a} notes={n}"
for i, (t, a, n) in enumerate(_dead)))
check("the episode really was standing before the poll task died - otherwise "
"the release below is vacuous",
_before[1] and _before[2] == 1 and abs(_before[0]) >= rail_w(LIVE_MAX_W))
check("a dead poll task drives the command to 0 W",
bool(_zeroed) and _zeroed[0] == 0)
check("...and on THAT SAME cycle the alarm is already clear, because the "
"detector ran after 0 W was resolved into self.target, not before it",
bool(_zeroed) and not _dead[_zeroed[0]][1])
check("...and it stays clear for as long as the failsafe holds it there",
_dead and not any(a for _, a, _ in _dead))
# ⚠️ "and it raises nothing new while parked" is NOT asserted here, and the
# reason matters more than the deletion. It was dropped as REDUNDANT, not as
# undecidable: TEST-06's reviewer falsified it in one line - move the
# `hass.call` above `if not fired: return` so the detector notifies every cycle,
# and notifications appear in exactly these parked cycles (that mutation takes
# this file RED 9). My first note here claimed no mutation could break it, which
# was the very inference this file exists to distrust, made about this file.
# The deletion stands because the exact count below catches that same mutation
# nine times over - no coverage was lost - and the honest form of the claim is
# "I tried to break it and here is what happened", never "I could not, so it
# cannot".
# ⚠️ AND THE OTHER HALF, or "no second notification" is a claim no mutation can
# break: when the transport comes back against a meter that is STILL frozen the
# loop re-rails, and that is a new episode which MUST speak. What must not
# happen is a second notification stacking up beside the first.
_re_railed = [i for i, (t, _, _) in enumerate(_revived)
if abs(t) >= rail_w(LIVE_MAX_W)]
print(f" transport revived: command re-rails on cycle "
f"{_re_railed[0] if _re_railed else None}, "
f"{_ctl_d.freeze_wd.episodes} episode(s), {len(_freeze_notes)} "
f"notification(s) from this guard, "
f"ids {sorted({n.get('notification_id') for n in _freeze_notes})}")
# A PRECONDITION, not a discriminating check - the same role as "the command
# takes several cycles to reach the rail" above, and labelled the same way:
# without it the two checks below are satisfied by a controller that never came
# back. The crudest falsifier (never file the loop claim) takes most of this
# file red, which is the signature of a precondition rather than a check; the
# narrowest one measured is a stale-input stop that never releases.
check("PRECONDITION: a revived transport against a still-frozen meter re-rails "
"the command",
bool(_re_railed))
check("...which is a SECOND episode, and it alarms again on the cycle it rails",
_ctl_d.freeze_wd.episodes == 2
and bool(_re_railed) and _revived[_re_railed[0]][2] == _before[2] + 1)
# ⚠️ STABILITY, not the value - the value is pinned once, above. Renaming the
# id must fail that check and NOT this one; the two say different things and
# each has to be able to go red on its own.
check("...on the same notification_id, so the second REPLACES the first "
"instead of stacking a second one beside it",
len(_freeze_notes) == 2
and len({n.get("notification_id") for n in _freeze_notes}) == 1)
# --------------------------------------------------------------------------- #
# 6. what the status page SAYS about coverage (TEST-06 / AC2)
# --------------------------------------------------------------------------- #
# ⚠️ The check above only ever asserted that an uncovered transport is SILENT,
# which a status page claiming `armed` on a guard that cannot run also satisfies.
# The AC was that the detector reports uncovered transports *and says so*, so
# the string is what gets asserted here, in both directions: the uncovered row
# must not claim `armed`, and the covered row must still claim it - or "does not
# say armed" would be satisfiable by deleting the armed branch outright.
print("what the status page SAYS about coverage")
def freeze_rows(unchanged_s, grid, **over):
"""The freeze-detector rows of the real checks(), on a p1 install.
⚠️ WHAT THIS DOES NOT COVER. The rows live behind `if self.p1_enabled:`, and
the flag is set here by hand rather than by building a real P1 transport -
inherited from part 4's fixture. So everything below proves "WHEN the row is
produced it says the right thing", and nothing below proves "the row is
produced on homewizard_local and not otherwise". That gate is uncovered;
AC2 is about the string, and widening the fixture belongs to whoever covers
the gate.
"""
_h, ctl, _t = wired(unchanged_s, grid, cycles=3, **over)
# p1_enabled is what a homewizard_local install has; the row's own content
# is what is under test, not the transport that feeds it.
ctl.p1_enabled = True
return ctl, [r for r in ctl.checks()
if r["text"].startswith("freeze detector")]
_ctl_u, _rows_u = freeze_rows(None, 4000.0, meter_source="ha_dsmr")
_ctl_c, _rows_c = freeze_rows(2.0, -14.2)
print(f" uncovered: {_rows_u[0]['text'] if _rows_u else '(no row at all)'}")
print(f" covered: {_rows_c[0]['text'] if _rows_c else '(no row at all)'}")
check("an uncovered transport gets exactly one freeze-detector row",
len(_rows_u) == 1)
check("...and it says NOT COVERED",
bool(_rows_u) and "NOT COVERED" in _rows_u[0]["text"])
check("...and it does NOT claim the guard is armed - that would be the status "
"page lying on behalf of a guard that cannot run",
bool(_rows_u) and "armed" not in _rows_u[0]["text"])
check("...and it names the meter_source it is not covering, so the operator "
"knows which install this is",
bool(_rows_u) and "ha_dsmr" in _rows_u[0]["text"])
check("...and it names homewizard_local as the transport that would be covered",
bool(_rows_u) and "homewizard_local" in _rows_u[0]["text"])
check("...flagged as a warning rather than a silent ok",
bool(_rows_u) and not _rows_u[0]["ok"] and _rows_u[0]["warn"])
check("a COVERED transport gets exactly one freeze-detector row",
len(_rows_c) == 1)
check("...and that one DOES say armed - without this, 'the uncovered row does "
"not say armed' is satisfied by deleting the armed branch",
bool(_rows_c) and "armed" in _rows_c[0]["text"]
and "NOT COVERED" not in _rows_c[0]["text"])
check("...and it quotes the hold and the rail it is armed at, derived from the "
"config the controller was actually built with",
bool(_rows_c)
and f"{_ctl_c.freeze_wd.cfg.hold_s:.0f} s unchanged" in _rows_c[0]["text"]
and f"{rail_w(LIVE_MAX_W):.0f} W" in _rows_c[0]["text"])
check("...as an ok row, since a covered armed guard is not a warning",
bool(_rows_c) and _rows_c[0]["ok"] and not _rows_c[0]["warn"])
# AC3's other half, through the built object rather than through a literal: the
# opts dict does NOT carry meter_freeze_s, so this is the number an install that
# leaves the key at its config.yaml default actually lands on.
print(f" a Controller built with meter_freeze_s ABSENT holds "
f"{_ctl_c.freeze_wd.cfg.hold_s:.0f} s (config.yaml ships "
f"{SHIPPED_HOLD_S:.0f} s)")
check("a Controller built with the key absent lands on the SHIPPED hold, so "
"config.yaml and main.py's fallback cannot drift apart unnoticed",
_ctl_c.freeze_wd.cfg.hold_s == SHIPPED_HOLD_S)
print()
print(f"part 3 (the live HTTP transport) ran against: {PART3_TRANSPORT}")
if fails:
print(f"{len(fails)} FAILED: {', '.join(fails)}")
sys.exit(1)
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+911
View File
@@ -0,0 +1,911 @@
"""Runnable check for FCST-03 - provisional prices. `python3 test_price_forecast.py`
WHAT THIS FILE IS DEFENDING, in the order the ticket's acceptance criteria name
it:
AC1 Every price point carries a `provisional` flag MATCHING ITS TRUE
PROVENANCE - a point filled from yesterday's profile is provisional even
when the number happens to be right, a point the feed published keeps
whatever flag the feed gave it, and an unmarked carry-forward is not
constructible rather than merely discouraged.
AC2 An irreversible action gated on a provisional price is REJECTED, with the
spec's own example - a maintenance-phase transition - actually refused:
the phase machine is still in `idle` afterwards.
AC3 Real-time / imbalance prices get MEASURABLY WIDER thresholds. Measured,
by finding the smallest spread each regime will accept and comparing the
two numbers - not by reading a constant back out of the module that
defines it.
THREE RULES THIS FILE HOLDS ITSELF TO, because the reviewer will:
* EVERY VERDICT IS DERIVED. The carried/published split is re-derived here
from the set of hours the fixture published, by different arithmetic than
app/price_forecast.py uses; the DST day lengths come from the offset
difference rather than from a 23 or a 25 typed into a check; the widening
factor is MEASURED by bisecting the accept/refuse boundary and then
compared with the declared one.
* NO STIMULUS IS EVALUATED WHERE THE QUANTITY IT GRADES IS DEGENERATE, and
this ticket is full of that trap:
- YESTERDAY AND TODAY MUST PRICE EVERY HOUR DIFFERENTLY, or a fill that
carried the wrong day forward is invisible. Asserted first, as a check.
- THE PROVISIONAL HOURS MUST CONTAIN THE CHEAPEST HOUR, or "arbitrage
ignores provisional hours" grades nothing: the firm-only spread and the
whole-series spread would be the same number. Asserted before it is
used, and it is the realistic shape of the danger - yesterday's cheap
hour luring a charge that today's prices do not justify.
- THE TWO WIDENING FACTORS MUST DIFFER, or `max`, `sum` and `product` are
indistinguishable combination rules. Asserted, then all three are
computed and only one matches what the code does.
- THE LOCAL-HOUR FILL AND A NAIVE `t - 24 h` FILL MUST DISAGREE on the
transition days, or the DST check passes for both implementations.
Asserted, then the correct one is graded.
- PRICES ARE WRITTEN TO SIX DECIMALS with distinct 3-, 4- and 6-dp
roundings, so an implementation that rounds differently is visible.
(TARIFF-03's fixtures were 3-dp, where those three are one number.)
* A FAILED READ IS NOT AN OBSERVATION. A missing price is not a zero price
and a stale one is not a fresh one: the gate blocks on `no_data`, blocks on
an hour the series does not cover, and blocks on a provisional hour - three
different reasons, none of them an answer.
"""
import ast
import inspect
import os
import sys
from datetime import datetime, timedelta, timezone
from app import price as _price
from app import price_forecast as _fc
from app import static_tariff as _st
from app.maintenance import (
CHARGE_TIMEOUT_MIN, DRAIN_TIMEOUT_MIN, DRAIN, IDLE, MaintConfig, Maintenance,
)
from app.price import (
Absence, PricePoint, PriceProduct, PriceRead, PROVIDERS, Uncertainty,
WIDEN_FACTOR, brussels, brussels_offset, plan_is_provisional,
required_spread, uncertainty_of, widen_factor,
)
from app.price_forecast import (
MAINTENANCE_CHARGE, MAINTENANCE_DRAIN, Gate, Irreversible, committed_hours,
fill_from_yesterday, price_permits, begin_phase,
)
from app.arbitrage import ArbitrageDecision
from app.static_tariff import Arbitrage, TariffPlan
UTC = timezone.utc
HERE = os.path.dirname(os.path.abspath(__file__))
fails = []
def check(name, cond):
print(f" {'ok ' if cond else 'FAIL'} {name}")
if not cond:
fails.append(name)
def raises(fn, exc):
"""True only if fn() raises EXACTLY `exc`. A different exception is False -
'it was refused' and 'it broke on the way to being refused' must not grade
the same."""
try:
fn()
except exc:
return True
except Exception:
return False
return False
class _Raised:
def __init__(self, e):
self.e = e
def __eq__(self, other):
return False
def __hash__(self):
return id(self)
def __bool__(self):
return False
def __repr__(self):
return f"<raised {type(self.e).__name__}: {self.e}>"
def derived(fn):
"""A value a check grades. An exception becomes `_Raised`, which equals
nothing and is falsy - NOT a default."""
try:
return fn()
except Exception as e: # noqa: BLE001
return _Raised(e)
def near(a, b, tol=1e-9):
try:
return abs(float(a) - float(b)) <= tol
except (TypeError, ValueError):
return False
# --------------------------------------------------------------------------- #
# Fixtures.
#
# ⚠️ SIX-DECIMAL PRICES, DELIBERATELY. TARIFF-03's fixtures were 0.3410 /
# 0.2570, where rounding to 3, 4 and 6 decimals gives the same number - so an
# implementation that rounded differently from the one being graded survived
# every check. These are chosen so those three roundings differ.
# --------------------------------------------------------------------------- #
def y_price(h):
"""Yesterday's all-in EUR/kWh for local hour h."""
return round(0.123437 + 0.010700 * h, 6)
def t_price(h):
"""Today's PUBLISHED all-in EUR/kWh for local hour h. Different from
yesterday's at every hour, on purpose - see the degeneracy note above."""
return round(0.206562 + 0.009300 * h, 6)
def local_midnight(y, m, d) -> datetime:
"""The UTC instant of Brussels local midnight on that date, derived from
`brussels_offset` rather than from a table of dates."""
naive = datetime(y, m, d, 0, 0)
for hours in (1, 2):
guess = naive.replace(tzinfo=UTC) - timedelta(hours=hours)
if brussels_offset(guess).utcoffset(None) == timedelta(hours=hours):
return guess
raise AssertionError(f"no consistent Brussels offset for {naive}")
def hours_between(a, b):
out, u = [], a
while u < b:
out.append(u)
u += timedelta(hours=1)
return out
def series_of(instants, price_of, product, provider, *, provisional):
return PriceRead.series(
[PricePoint(start=brussels(u), eur_per_kwh=price_of(u),
provisional=provisional) for u in instants],
product, provider, instants[0])
# A plain winter day: local 2026-01-15, 24 hours, no transition in sight.
D0 = local_midnight(2026, 1, 15)
D1 = local_midnight(2026, 1, 16)
YEST0 = local_midnight(2026, 1, 14)
DAY = hours_between(D0, D1)
YDAY = hours_between(YEST0, D0)
YESTERDAY = series_of(YDAY, lambda u: y_price(brussels(u).hour),
PriceProduct.DAY_AHEAD, "generic_dayahead",
provisional=False)
# The feed has published the first 12 hours of today and nothing after - the
# ordinary state of a dynamic site before the 14-16 h CET print (§5.4).
PUBLISHED_N = 12
TODAY_PART = series_of(DAY[:PUBLISHED_N], lambda u: t_price(brussels(u).hour),
PriceProduct.DAY_AHEAD, "generic_dayahead",
provisional=False)
print()
print("0. the fixtures are not degenerate (checked before anything is graded)")
check("yesterday and today price EVERY shared hour differently, so a fill that "
"carried the wrong day forward would be visible in the value and not "
"only in the flag",
all(y_price(h) != t_price(h) for h in range(24)))
_roundings = derived(lambda: {round(y_price(3), n) for n in (3, 4, 6)})
check(f"...and a fixture price rounds to three DIFFERENT numbers at 3, 4 and 6 "
f"decimals {sorted(_roundings)}, so a divergent rounding cannot hide",
len(_roundings) == 3)
check("...and both widening factors differ, so max / sum / product are three "
"distinguishable answers",
len(set(WIDEN_FACTOR.values())) == len(WIDEN_FACTOR) >= 2)
# --------------------------------------------------------------------------- #
print()
print("1. AC1 - the flag matches PROVENANCE, and an unmarked fill is not "
"constructible")
FILLED = fill_from_yesterday(TODAY_PART, yesterday=YESTERDAY,
since=D0, until=D1, stamp=D0)
_pub_starts = {u for u in DAY[:PUBLISHED_N]}
_expected_prov = {u: (u not in _pub_starts) for u in DAY}
_got_prov = derived(lambda: {p.start.astimezone(UTC): p.provisional
for p in FILLED.points})
check(f"the filled series covers every hour of the day "
f"{len(FILLED.points) if FILLED.kind == 'series' else FILLED}",
FILLED.kind == "series" and len(FILLED.points) == len(DAY))
check("...and EVERY point's flag equals what its provenance says it should be, "
"re-derived here from the set of hours the fixture published",
_got_prov == _expected_prov)
_carried = derived(lambda: [p for p in FILLED.points if p.provisional])
check(f"...which is {len(DAY) - PUBLISHED_N} carried hours and {PUBLISHED_N} "
f"published ones, counted rather than asserted",
len(_carried) == len(DAY) - PUBLISHED_N)
check("...and the carried hours carry YESTERDAY's number for the same wall "
"clock hour, not today's and not a neighbour's",
all(near(p.eur_per_kwh, y_price(brussels(p.start).hour))
for p in _carried))
check("...while the published hours carry TODAY's, so the two halves are not "
"the same data wearing two flags",
all(near(p.eur_per_kwh, t_price(brussels(p.start).hour))
for p in FILLED.points if not p.provisional))
# ⚠️ THE COPY IS THE DEFECT. A published point that survives as a COPY is one
# whose flag could have been rewritten on the way through; identity is the only
# property that rules that out for every field at once.
check("a published point is passed through BY IDENTITY - the object the "
"provider produced is the object in the filled series",
all(any(p is q for q in TODAY_PART.points)
for p in FILLED.points if not p.provisional))
check("...and the canonical §9.3 dict carries the flag out to the wire for "
"both kinds",
FILLED.canonical()["prices"][0]["provisional"] is False
and FILLED.canonical()["prices"][-1]["provisional"] is True)
check("...and plan_is_provisional() sees the series as provisional overall, "
"because at least one hour is",
plan_is_provisional(FILLED) is True)
# The audit is a post-condition that RAISES. Feed it the exact defect.
_stale_copy = PricePoint(start=FILLED.points[-1].start,
eur_per_kwh=FILLED.points[-1].eur_per_kwh,
provisional=False) # the copy-the-flag bug
check("the provenance audit REFUSES a carried hour flagged non-provisional - "
"the 'stale but plausible' point, which is SAFETY-18's frozen-but-fresh "
"meter one layer up",
raises(lambda: _fc._audit([_stale_copy], {}), ValueError))
check("...and refuses a published hour that arrived as a COPY rather than as "
"the published object, which is where a flag gets lost",
raises(lambda: _fc._audit(
[PricePoint(start=TODAY_PART.points[0].start,
eur_per_kwh=TODAY_PART.points[0].eur_per_kwh,
provisional=TODAY_PART.points[0].provisional)],
{D0: TODAY_PART.points[0]}), ValueError))
check("...and never repairs a flag: the audit's only outcomes are silence and "
"an exception, so a series it passed was already correct",
_fc._audit(list(FILLED.points), {u: TODAY_PART.points[i]
for i, u in enumerate(DAY[:PUBLISHED_N])})
is None)
# ⚠️ A PUBLISHED POINT MAY BE PROVISIONAL. eneco_eddy's current-quarter print
# is settled after the fact and says so (§9.2). "Published" is not "firm".
_eddy = series_of(DAY[:2], lambda u: t_price(brussels(u).hour),
PriceProduct.REALTIME, "eneco_eddy", provisional=True)
_eddy_filled = derived(lambda: fill_from_yesterday(
_eddy, yesterday=YESTERDAY, since=D0, until=D1, stamp=D0))
check("a PUBLISHED point that the feed itself flagged provisional keeps that "
"flag - the audit checks provenance, and does not force published hours "
"to False",
_eddy_filled.kind == "series"
and all(p.provisional for p in _eddy_filled.points[:2])
and _eddy_filled.points[0] is _eddy.points[0])
# The structural half: the constructor has nothing to pass, and the file is
# scanned so the pass-through idiom cannot come back looking ordinary.
check("_carried() has NO `provisional` parameter, so a caller has nothing to "
"forget, default, or copy from the point being carried",
"provisional" not in inspect.signature(_fc._carried).parameters)
_SRC = derived(lambda: open(os.path.join(HERE, "app", "price_forecast.py"),
encoding="utf-8").read())
def _nonliteral_provisional(src):
"""Every `PricePoint(...)` in the module whose `provisional` is not a
literal. AST, not a substring: the module docstring quotes the defect it
exists to prevent, and a text scan would match its own documentation."""
out = []
for node in ast.walk(ast.parse(src)):
if isinstance(node, ast.Call) and \
getattr(node.func, "id", None) == "PricePoint":
kw = {k.arg: k.value for k in node.keywords}
v = kw.get("provisional")
if not isinstance(v, ast.Constant):
out.append(ast.dump(node)[:80])
return out
_leaks = derived(lambda: _nonliteral_provisional(_SRC))
check(f"no PricePoint built in app/price_forecast.py takes its `provisional` "
f"from an expression - it is a literal at every construction site, so "
f"`provisional=y.provisional` cannot reappear in a later edit {_leaks}",
_leaks == [])
# The scan must be able to fail, or it grades nothing.
check("...and that scan FAILS on the defect it names, so it is a check and not "
"a decoration",
_nonliteral_provisional(
"PricePoint(start=t, eur_per_kwh=v, provisional=y.provisional)") != [])
# --------------------------------------------------------------------------- #
print()
print("2. the fill keys on the WALL CLOCK, so the 23- and 25-hour days are right")
for label, (y, m, d) in (("spring forward (23 h)", (2026, 3, 29)),
("fall back (25 h)", (2026, 10, 25))):
a = local_midnight(y, m, d)
b = local_midnight(y, m, d + 1)
yb = local_midnight(y, m, d - 1)
yesterday = series_of(hours_between(yb, a),
lambda u: y_price(brussels(u).hour),
PriceProduct.DAY_AHEAD, "generic_dayahead",
provisional=False)
# Nothing published at all yet: the whole day is carried forward.
nothing = PriceRead.no_data("the day-ahead print has not landed yet")
got = derived(lambda: fill_from_yesterday(nothing, yesterday=yesterday,
since=a, until=b, stamp=a))
# ⚠️ DERIVED FROM THE OFFSETS, not from a 23 or a 25 typed here.
off_a = brussels_offset(a).utcoffset(None).total_seconds() / 3600
off_b = brussels_offset(b - timedelta(seconds=1)).utcoffset(None) \
.total_seconds() / 3600
expect = int(24 - (off_b - off_a))
check(f"{label}: the day is {expect} hours long, derived from the offset "
f"change {off_a:+.0f} -> {off_b:+.0f}, and the fill emits exactly "
f"that many",
got.kind == "series" and len(got.points) == expect)
check(f"{label}: every hour of it is flagged provisional - nothing was "
f"published",
got.kind == "series" and all(p.provisional for p in got.points))
# The discriminating part: a naive `t - 24 h` fill would give a DIFFERENT
# answer here, so this check separates the two implementations.
y_by_utc = {u: y_price(brussels(u).hour) for u in hours_between(yb, a)}
naive = [y_by_utc.get(u - timedelta(hours=24)) for u in hours_between(a, b)]
wall = [y_price(brussels(u).hour) for u in hours_between(a, b)]
check(f"{label}: the wall-clock fill and a naive t-24h fill DISAGREE, so "
f"the check below can tell them apart",
naive != wall)
check(f"{label}: ...and the emitted values are the WALL-CLOCK ones",
got.kind == "series"
and [round(p.eur_per_kwh, 6) for p in got.points] == wall)
# The 25-hour day's two 02:00 hours are two distinct instants, both filled.
_oct = local_midnight(2026, 10, 25)
_oct_end = local_midnight(2026, 10, 26)
_oct_fill = fill_from_yesterday(
PriceRead.no_data("nothing yet"),
yesterday=series_of(hours_between(local_midnight(2026, 10, 24), _oct),
lambda u: y_price(brussels(u).hour),
PriceProduct.DAY_AHEAD, "generic_dayahead",
provisional=False),
since=_oct, until=_oct_end, stamp=_oct)
_twos = [p for p in _oct_fill.points if brussels(p.start).hour == 2]
check("the 25-hour day has TWO local 02:00 points, at different offsets, and "
"both are filled and both are flagged - a 24-slot implementation would "
"collide them into one",
len(_twos) == 2
and _twos[0].start.utcoffset() != _twos[1].start.utcoffset()
and all(p.provisional for p in _twos))
# --------------------------------------------------------------------------- #
print()
print("3. §5.4 - a late publication REPLACES a carried hour and re-flags it")
_late = fill_from_yesterday(
series_of(DAY, lambda u: t_price(brussels(u).hour),
PriceProduct.DAY_AHEAD, "generic_dayahead", provisional=False),
yesterday=YESTERDAY, since=D0, until=D1, stamp=D1)
_h20 = DAY[20]
_before = derived(lambda: FILLED.at(_h20))
_after = derived(lambda: _late.at(_h20))
check("before the print, hour 20 is provisional and carries YESTERDAY's number",
_before.provisional is True and near(_before.eur_per_kwh, y_price(20)))
check("...after it, the same hour is FIRM and carries today's - the flag and "
"the number move together, because the series is rebuilt and not patched",
_after.provisional is False and near(_after.eur_per_kwh, t_price(20)))
check("...and the two numbers actually differ, so 'it was replaced' is a "
"measurement and not a restatement",
not near(_before.eur_per_kwh, _after.eur_per_kwh))
check("...and re-running the fill over an already-complete series changes "
"nothing: it is idempotent",
[p.canonical() for p in fill_from_yesterday(
_late, yesterday=YESTERDAY, since=D0, until=D1, stamp=D1).points]
== [p.canonical() for p in _late.points])
# Nothing to fill from, and nothing published: no_data, never an empty series.
_nothing = derived(lambda: fill_from_yesterday(
PriceRead.no_data("the feed timed out"),
yesterday=PriceRead.no_data("and there is no yesterday either"),
since=D0, until=D1, stamp=D0))
check("with neither a published hour nor a yesterday, the result is no_data - "
"not an empty series, and not a series of zeros",
_nothing.kind == "no_data")
check("...so it has no .points to iterate and no .get() to default out of",
raises(lambda: _nothing.points, Absence)
and raises(lambda: _nothing.get("points", []), Absence))
check("...and its reason names §11.4 for an operator",
"§11.4" in _nothing.reason)
check("a naive `since` is refused rather than silently interpreted as UTC",
raises(lambda: fill_from_yesterday(TODAY_PART, yesterday=YESTERDAY,
since=datetime(2026, 1, 15),
until=D1, stamp=D0), ValueError))
check("...and `yesterday` has no default, so a fill with nothing to fill from "
"cannot be written by accident",
raises(lambda: fill_from_yesterday(TODAY_PART, since=D0, until=D1,
stamp=D0), TypeError))
# --------------------------------------------------------------------------- #
print()
print("4. AC2 - an irreversible action is REJECTED on a provisional price")
class FakeStore:
"""Same surface as app/store.Store, no disk. Copied from test_maintenance."""
def __init__(self):
self.data = {"phase": IDLE, "phase_started": None,
"last_completed": None, "last_start_attempt": None}
def set(self, k, v):
self.data[k] = v
def set_time(self, k, when):
self.data[k] = when
def get_time(self, k):
return self.data.get(k)
def save(self):
pass
def maint():
return Maintenance(MaintConfig(enabled=True), FakeStore())
check("the drain's commitment window is DERIVED from the phase machine's own "
"timeout, so the gate and maintenance.py cannot drift apart",
near(MAINTENANCE_DRAIN.commits_h, DRAIN_TIMEOUT_MIN / 60.0)
and near(MAINTENANCE_CHARGE.commits_h, CHARGE_TIMEOUT_MIN / 60.0))
# The shipped `maintenance_start_hour` is 10:00 local, so a drain started on
# schedule runs into the evening - hours a morning-only day-ahead print has not
# covered. That is the ticket's scenario and it is where the window is placed.
START = D0 + timedelta(hours=MaintConfig().start_hour)
_hours = derived(lambda: committed_hours(START, MAINTENANCE_DRAIN))
check(f"...and it commits {len(_hours) if not isinstance(_hours, _Raised) else _hours} "
f"whole hours, one per hour of the timeout",
len(_hours) == int(DRAIN_TIMEOUT_MIN / 60))
check("an action that commits nothing is not an irreversible one and cannot be "
"declared", raises(lambda: Irreversible("X", 0), ValueError)
and raises(lambda: Irreversible("", 4), ValueError))
# The stimulus: the first hours are firm, the drain's window reaches into the
# carried ones. THIS is the ticket's scenario - a cycle started at 10:00 on
# prices that were published for the morning and guessed for the evening.
_m = maint()
_gate_bad = derived(lambda: begin_phase(_m, START, FILLED, MAINTENANCE_DRAIN))
_first_prov = derived(lambda: next(u for u in _hours if FILLED.at(u).provisional))
check("the drain's window really does reach past the published hours while "
"staying INSIDE the series, so the refusal below is about provenance and "
"not about a short series",
FILLED.covers(_hours[-1])
and any(not FILLED.at(u).provisional for u in _hours)
and any(FILLED.at(u).provisional for u in _hours))
check("a maintenance DRAIN whose window includes a carried-forward hour is "
"BLOCKED", _gate_bad.kind == "blocked")
check("...and the phase machine did not move: it is still idle, so the "
"rejection is of the ACTION and not merely of a verdict object",
_m.phase == IDLE)
check("...and the reason names §8.3, the offending hour, and when the real "
"price is expected",
"§8.3" in _gate_bad.reason
and _first_prov.isoformat() in _gate_bad.reason
and "14-16" in _gate_bad.reason)
# The positive arm, so the block above is discriminating rather than universal.
_firm_long = series_of(hours_between(START,
START + timedelta(hours=len(_hours) + 2)),
lambda u: t_price(brussels(u).hour % 24),
PriceProduct.DAY_AHEAD, "generic_dayahead",
provisional=False)
_m2 = maint()
_gate_ok = derived(lambda: begin_phase(_m2, START, _firm_long,
MAINTENANCE_DRAIN))
check("...while the SAME transition on a fully published series is PERMITTED, "
"and the phase machine actually moves to drain",
_gate_ok.kind == "permitted" and _m2.phase == DRAIN)
check("...and the permitting arm carries the evidence it checked - one firm "
"point per committed hour",
_gate_ok.kind == "permitted" and len(_gate_ok.hours) == len(_hours)
and not any(p.provisional for p in _gate_ok.hours))
# The other two ways a price can fail to be an observation.
_m3 = maint()
_gate_none = begin_phase(_m3, START, PriceRead.no_data("the feed timed out"),
MAINTENANCE_DRAIN)
check("no price data at all BLOCKS an irreversible action too - a missing "
"price is not a cheap one (§11.4)",
_gate_none.kind == "blocked" and "§11.4" in _gate_none.reason
and _m3.phase == IDLE)
_short = series_of(DAY[:4], lambda u: t_price(brussels(u).hour),
PriceProduct.DAY_AHEAD, "generic_dayahead",
provisional=False)
_gate_short = price_permits(_short, D0, MAINTENANCE_DRAIN)
check("...and so does an hour the series does not cover, which is §11.4 for "
"that hour rather than a reason to reach for a neighbour",
_gate_short.kind == "blocked" and "no price covers" in _gate_short.reason)
# The variant type itself, held to the same shape as PriceRead and Arbitrage.
check("a BLOCKED gate carries a reason and nothing else: .hours and .action "
"raise Absence rather than answering with an empty tuple",
raises(lambda: _gate_bad.hours, Absence)
and raises(lambda: _gate_bad.action, Absence))
check("...it is not iterable and has no .get(), the other two routes to an "
"empty, harmless-looking loop",
raises(lambda: list(_gate_bad), Absence)
and raises(lambda: _gate_bad.get("hours", []), Absence))
check("...match() demands BOTH arms, so the block cannot be fallen through",
raises(lambda: _gate_bad.match(permitted=lambda g: 1), TypeError))
check("...and there is no falsy variant, so `if gate:` cannot look like a "
"guard while being none",
bool(_gate_bad) is True and bool(_gate_ok) is True)
# A genuinely carried point - the flagged one, not the mis-flagged copy above.
_prov_pt = FILLED.points[-1]
check("the point the two constructor checks below are handed really is "
"provisional, or they would be grading nothing",
_prov_pt.provisional is True)
check("a permitted gate cannot be built over a provisional hour or over no "
"hours at all - §8.3 is the one rule this type has",
raises(lambda: Gate.permit(MAINTENANCE_DRAIN, [_prov_pt]), ValueError)
and raises(lambda: Gate.permit(MAINTENANCE_DRAIN, []), ValueError))
# TEST-14, PORTED. A default on any of these constructors reopens the exact
# hole `_NoData.reason` had - and static_tariff's `_Refused.reason` still has,
# because nothing scanned it. Scanned by SIGNATURE, so a default added later is
# red without anyone remembering to write a new case.
def _defaulted(fn):
try:
ps = list(inspect.signature(fn).parameters.values())
except (TypeError, ValueError) as e: # pragma: no cover
return _Raised(e)
return [q.name for q in ps
if q.name != "self" and q.default is not inspect.Parameter.empty]
_ctors = {"Gate.permit": Gate.permit, "Gate.block": Gate.block,
"Gate.match": Gate.match,
"_Permitted.__init__": _fc._Permitted.__init__,
"_Blocked.__init__": _fc._Blocked.__init__,
# ⚠️ static_tariff's outcome type is scanned HERE because nothing
# scanned it before: TEST-14 records `_Refused.reason` gaining a
# default as a mutation that survived every check in that file.
"Arbitrage.allow": Arbitrage.allow, "Arbitrage.refuse": Arbitrage.refuse,
"Arbitrage.match": Arbitrage.match,
"_Allowed.__init__": _st._Allowed.__init__,
"_Refused.__init__": _st._Refused.__init__,
# ⚠️ OPT-04's outcome type is scanned HERE, in the same dict, rather
# than by a hand-written case in test_arbitrage.py - which is TEST-14's
# actual instruction. A new outcome type extends the SCAN; a
# hand-written case is the thing that gets forgotten, and the whole
# reason this dict exists instead of five separate checks.
"ArbitrageDecision.__init__": ArbitrageDecision.__init__}
_with_defaults = derived(lambda: {n: d for n, d in
((n, _defaulted(f)) for n, f in _ctors.items())
if d})
check(f"no constructor or handler of an outcome type - Gate's OR Arbitrage's - "
f"carries a DEFAULT, so a verdict can never be built out of nothing "
f"{_with_defaults}", _with_defaults == {})
check("...so a block with no reason is a TypeError at the call site, not a "
"blank status line",
raises(lambda: Gate.block(), TypeError)
and raises(lambda: Gate.block(""), ValueError))
check("...and the same holds for the refusal FCST-03 added to Arbitrage",
raises(lambda: Arbitrage.refuse(), TypeError))
# ⚠️ ONE CALL PER ROLE, AND THE FIRST DRAFT SHOWED WHY. Passing the provisional
# point as BOTH `cheapest` and `dearest` is a stimulus that cannot distinguish
# "the constructor guards the cheapest hour" from "the constructor guards both
# hours": a guard on either one alone raises, and the check goes green for an
# implementation that leaves the sell hour unguarded. That is the degeneracy
# rule this file states at the top, in a new dress - so the two roles are
# stimulated SEPARATELY, each with a firm point in the other seat.
_firm_pt = TODAY_PART.points[0]
check("the two points these constructor checks use really differ in provenance, "
"or neither of them grades anything",
_prov_pt.provisional is True and _firm_pt.provisional is False)
check("an ALLOWED arbitrage verdict cannot be constructed with a provisional "
"CHEAPEST hour - the buy hour - even directly: §8.3 is enforced at the "
"constructor, not only in the ladder above it",
raises(lambda: Arbitrage.allow(0.1, _prov_pt, _firm_pt, "normal",
PriceProduct.DAY_AHEAD, 0.0, frozenset()),
ValueError))
check("...nor with a provisional DEAREST hour - the sell hour - which a guard "
"written for `cheapest` alone would have let through",
raises(lambda: Arbitrage.allow(0.1, _firm_pt, _prov_pt, "normal",
PriceProduct.DAY_AHEAD, 0.0, frozenset()),
ValueError))
check("...while two firm points build the verdict perfectly well, so the two "
"refusals above are discriminating rather than universal",
Arbitrage.allow(0.1, _firm_pt, _firm_pt, "normal",
PriceProduct.DAY_AHEAD, 0.0, frozenset()).kind
== "allowed")
# --------------------------------------------------------------------------- #
print()
print("5. AC3 - real-time thresholds are MEASURABLY wider (measured, not read)")
def plan_with_spread(spread, product, provider):
"""A two-hour series with exactly `spread` between its two prices."""
lo, hi = 0.150000, round(0.150000 + spread, 6)
pts = [PricePoint(start=brussels(D0), eur_per_kwh=lo, provisional=False),
PricePoint(start=brussels(D0 + timedelta(hours=1)),
eur_per_kwh=hi, provisional=False)]
return TariffPlan(PriceRead.series(pts, product, provider, D0), None,
"provider")
def min_accepted_spread(product, sources, base, provider="generic_dayahead"):
"""The SMALLEST spread this configuration will trade on, found by bisecting
the accept/refuse boundary of the real verdict. Nothing here reads a
factor out of the module it is grading."""
lo, hi = 0.0, 4.0
for _ in range(60):
mid = (lo + hi) / 2
v = plan_with_spread(mid, product, provider).arbitrage(
peak_guard_active=False, min_spread_eur_kwh=base,
uncertainty=sources)
if v.kind == "allowed":
hi = mid
else:
lo = mid
return hi
BASE = 0.104000 # a caller's own round-trip cost threshold, EUR/kWh
_normal = derived(lambda: min_accepted_spread(PriceProduct.DAY_AHEAD,
frozenset(), BASE))
_wide = derived(lambda: min_accepted_spread(PriceProduct.REALTIME,
frozenset(), BASE))
_weather = derived(lambda: min_accepted_spread(
PriceProduct.DAY_AHEAD, frozenset({Uncertainty.NO_WEATHER}), BASE))
_both = derived(lambda: min_accepted_spread(
PriceProduct.REALTIME, frozenset({Uncertainty.NO_WEATHER}), BASE))
check(f"a day-ahead product trades on any spread above its own threshold "
f"{_normal:.6f}, which is the base {BASE} it was given",
near(_normal, BASE, 1e-6))
check(f"a REAL-TIME / imbalance product needs {_wide:.6f} - MEASURABLY wider, "
f"by a factor of {(_wide / _normal) if _normal else 0:.3f} measured from "
f"two verdict boundaries and not read from any constant",
_wide > _normal + 1e-6)
check(f"...and that measured factor equals the declared one for "
f"REALTIME_PRODUCT ({WIDEN_FACTOR[Uncertainty.REALTIME_PRODUCT]}), so "
f"the declaration and the behaviour are the same thing",
near(_wide / _normal, WIDEN_FACTOR[Uncertainty.REALTIME_PRODUCT], 1e-4))
check(f"§11.4's no-weather row reaches the SAME widening path by a different "
f"trigger, and lands at {_weather:.6f}",
near(_weather / _normal, WIDEN_FACTOR[Uncertainty.NO_WEATHER], 1e-4))
# ⚠️ THE COMBINATION RULE, ON A STIMULUS WHERE THE THREE CANDIDATES DIFFER.
_f1 = WIDEN_FACTOR[Uncertainty.REALTIME_PRODUCT]
_f2 = WIDEN_FACTOR[Uncertainty.NO_WEATHER]
check(f"with BOTH sources active the threshold is max(f1,f2)={max(_f1, _f2)} x "
f"base and NOT the product ({_f1 * _f2}) or the sum ({_f1 + _f2}) - two "
f"independent doubts must not compound into a threshold nothing can "
f"clear",
near(_both / _normal, max(_f1, _f2), 1e-4)
and not near(_both / _normal, _f1 * _f2, 1e-3)
and not near(_both / _normal, _f1 + _f2, 1e-3))
# The regime must follow the DECLARED product, never the plugin id.
_wide_named_dayahead = derived(lambda: min_accepted_spread(
PriceProduct.REALTIME, frozenset(), BASE, provider="generic_dayahead"))
_normal_named_eddy = derived(lambda: min_accepted_spread(
PriceProduct.DAY_AHEAD, frozenset(), BASE, provider="eneco_eddy"))
check("the widening follows the DECLARED PriceProduct even when the provider "
"is called 'generic_dayahead', and does NOT follow the name "
"'eneco_eddy' when the declared product is day-ahead - so a sixth "
"supplier is widened by its declaration and not by an id in a list",
near(_wide_named_dayahead, _wide, 1e-9)
and near(_normal_named_eddy, _normal, 1e-9))
check("...and the shipped plugin §9.2 names IS the one that declares REALTIME, "
"so the mechanism is pointed at the real product",
PROVIDERS["eneco_eddy"].capabilities().product is PriceProduct.REALTIME
and uncertainty_of(PriceProduct.REALTIME)
== frozenset({Uncertainty.REALTIME_PRODUCT})
and uncertainty_of(PriceProduct.DAY_AHEAD) == frozenset())
check("a string in the uncertainty set is REFUSED rather than widening by 1.0 "
"and reporting that it widened",
raises(lambda: widen_factor({"realtime"}), TypeError)
and raises(lambda: required_spread(0.1, {"realtime"}), TypeError))
check("...and a negative or non-finite base threshold is refused, since it is "
"one every spread clears",
raises(lambda: required_spread(-0.01, frozenset()), ValueError)
and raises(lambda: required_spread(float("nan"), frozenset()), ValueError))
check("required_spread() with nothing uncertain is the base itself, so the "
"widening is additive machinery and not a permanent surcharge",
near(required_spread(BASE, frozenset()), BASE))
# --------------------------------------------------------------------------- #
print()
print("6. the refusal ladder - provisional hours are DROPPED, not traded on")
# ⚠️ THE PROVISIONAL HOURS CARRY THE CHEAPEST PRICE. That is the realistic
# danger (yesterday's cheap night luring a charge) AND the non-degenerate
# stimulus: the firm-only spread and the whole-series spread differ, so a check
# on the spread can tell which set was used.
_cheap_stale = [PricePoint(start=brussels(DAY[i]),
eur_per_kwh=0.041250, provisional=True)
for i in range(3)]
_firm_rest = [PricePoint(start=brussels(DAY[i]),
eur_per_kwh=round(0.212500 + 0.013750 * i, 6),
provisional=False) for i in range(3, 12)]
_mixed = TariffPlan(PriceRead.series(_cheap_stale + _firm_rest,
PriceProduct.DAY_AHEAD,
"generic_dayahead", D0), None, "provider")
_all_spread = derived(lambda: _mixed.spread_eur_kwh())
_firm_spread = derived(lambda: round(
max(p.eur_per_kwh for p in _firm_rest)
- min(p.eur_per_kwh for p in _firm_rest), 6))
check(f"the fixture is non-degenerate: the whole-series spread "
f"{_all_spread:.6f} and the firm-only spread {_firm_spread:.6f} are "
f"different numbers, so a verdict can be traced to one of them",
not near(_all_spread, _firm_spread, 1e-6))
_v = derived(lambda: _mixed.arbitrage(peak_guard_active=False,
min_spread_eur_kwh=0.0,
uncertainty=frozenset()))
check("a PART-published series still allows arbitrage - §8.3 forbids acting on "
"a provisional price, not trading at all until 16:00",
_v.kind == "allowed")
check(f"...on the FIRM spread {_firm_spread:.6f}, not the one yesterday's "
f"carried-forward cheap hour would have invented",
near(_v.spread_eur_kwh, _firm_spread, 1e-9))
check("...and the cheapest and dearest hours it hands an optimiser are both "
"PUBLISHED hours - the two points ARE the trade",
_v.cheapest.provisional is False and _v.dearest.provisional is False)
check("...and it reports the threshold it cleared and why, rather than leaving "
"OPT-04 to re-derive it from a product enum",
near(_v.required_spread_eur_kwh, 0.0) and _v.uncertainty == frozenset())
_all_prov = TariffPlan(PriceRead.series(_cheap_stale, PriceProduct.DAY_AHEAD,
"generic_dayahead", D0), None,
"fallback")
_vp = derived(lambda: _all_prov.arbitrage(peak_guard_active=False,
min_spread_eur_kwh=0.0,
uncertainty=frozenset()))
check("a WHOLLY provisional series - the §11.4 fallback, or a day before any "
"print - is still refused, and the reason counts the firm hours",
_vp.kind == "refused" and "0 of 3" in _vp.reason and "§8.3" in _vp.reason)
check("...and the refusal carries no numbers to trade on",
raises(lambda: _vp.spread_eur_kwh, Absence)
and raises(lambda: _vp.cheapest, Absence))
_one_firm = TariffPlan(
PriceRead.series(_cheap_stale + _firm_rest[:1], PriceProduct.DAY_AHEAD,
"generic_dayahead", D0), None, "provider")
check("one firm hour is not enough either: a buy hour and a sell hour must "
"BOTH be published before a cycle can be committed",
_one_firm.arbitrage(peak_guard_active=False, min_spread_eur_kwh=0.0,
uncertainty=frozenset()).kind == "refused")
check("§10.1 still holds above all of it: the peak guard refuses the same "
"series that is otherwise allowed",
_mixed.arbitrage(peak_guard_active=True, min_spread_eur_kwh=0.0,
uncertainty=frozenset()).kind == "refused")
check("the three questions cannot be skipped: all of peak_guard_active, "
"min_spread_eur_kwh and uncertainty are required and keyword-only",
raises(lambda: _mixed.arbitrage(peak_guard_active=False), TypeError)
and raises(lambda: _mixed.arbitrage(peak_guard_active=False,
min_spread_eur_kwh=0.0), TypeError)
and raises(lambda: _mixed.arbitrage(False, 0.0, frozenset()), TypeError))
check("...nor answered with a truthy 'unknown' or a negative threshold",
raises(lambda: _mixed.arbitrage(peak_guard_active=None,
min_spread_eur_kwh=0.0,
uncertainty=frozenset()), TypeError)
and raises(lambda: _mixed.arbitrage(peak_guard_active=False,
min_spread_eur_kwh=-0.1,
uncertainty=frozenset()), TypeError))
# The cheap-window flag is the other consumer of provisionality, and it is the
# one a live install's maintenance charge floor already reads.
_firm_hour = _firm_rest[0].start
_prov_hour = _cheap_stale[0].start
check("the cheap-window flag is decided per HOUR: a provisional hour is never "
"cheap, whatever its (stale) number says",
_mixed.cheap_now(_prov_hour) is False)
check("...while a firm hour below the firm mean still is, so carry-forward "
"filling does not switch the maintenance charge floor off every morning",
_mixed.cheap_now(_firm_hour) is True)
# ⚠️ GRADED DIRECTLY, BECAUSE ITS CONSUMERS CANNOT GRADE IT. The §11.4 refusal
# and `cheap_now` both guard provenance independently, so `firm_points()`
# returning the carried hours TOO still leaves both of them answering correctly
# - a stimulus that cannot distinguish an over-reporting accessor from a
# correct one. So the tuple itself is compared with the published points, by
# identity and in order, rather than only the mean it feeds.
check("firm_points() returns EXACTLY the published points, in order - not the "
"carried ones as well, which its two consumers would have absorbed "
"without either of them changing its answer",
list(_mixed.firm_points()) == _firm_rest
and all(a is b for a, b in zip(_mixed.firm_points(), _firm_rest)))
check("...and the firm mean it compares against is derived from published "
"hours only",
near(sum(p.eur_per_kwh for p in _mixed.firm_points())
/ len(_mixed.firm_points()),
sum(p.eur_per_kwh for p in _firm_rest) / len(_firm_rest)))
# --------------------------------------------------------------------------- #
print()
print("7. CFG-03 and validate.py rule 7 - what this ticket did NOT widen")
_defaults = derived(lambda: sorted(_price.PRICE_OPTION_DEFAULTS))
_cfg = derived(lambda: open(os.path.join(HERE, "config.yaml"),
encoding="utf-8").read())
# ⚠️ THIS LIST IS A DRIFT PIN, NOT A BAN ON EVER ADDING AN OPTION. FCST-03 added
# none, and said so here. OPT-04 added FOUR - and added them to this ONE dict
# and to config.yaml, which is precisely the behaviour CFG-03 asks for: the
# alternative is a fourth freehand copy in main.py's `o.get(k, X)` fallbacks.
# The pin is what makes such an addition a decision somebody signs rather than a
# thing that appears; whoever adds the fifth comes here and says which card and
# why, as this line does.
_OPT04_KEYS = ["arbitrage_enabled", "cycle_cost_threshold_eur",
"arbitrage_soc_floor", "arbitrage_soc_ceiling"]
check(f"the in-code mirror holds exactly TARIFF-02's, TARIFF-03's and OPT-04's "
f"keys - FCST-03 still added none of its own {_defaults}",
_defaults == sorted(["price_provider", "price_markup_eur_kwh",
"price_taxes_eur_kwh", "export_price_eur_kwh",
"price_csv_path", "price_feed_url",
"price_horizon_h", "price_timeout_s",
"static_profile"] + _OPT04_KEYS))
# ...and the four are genuinely NEW, not a rename of something that was already
# there, or the line above would pass while the mirror silently lost a key.
check("...and OPT-04's four keys are additions rather than replacements: the "
"nine TARIFF-02/03 keys are all still present",
isinstance(_defaults, list)
and len(set(_defaults) - set(_OPT04_KEYS)) == 9)
check("...and every one of them is still in config.yaml, which stays the "
"source of truth",
isinstance(_cfg, str)
and all(f"\n {k}:" in _cfg for k in _price.PRICE_OPTION_DEFAULTS))
def _names(path):
try:
tree = ast.parse(open(path, encoding="utf-8").read())
except (OSError, SyntaxError): # pragma: no cover
return set()
out = set()
for node in ast.walk(tree):
if isinstance(node, ast.ImportFrom):
out |= {a.name for a in node.names}
if isinstance(node, ast.Name):
out.add(node.id)
if isinstance(node, ast.Attribute):
out.add(node.attr)
return out
_FC = os.path.join(HERE, "app", "price_forecast.py")
_used = derived(lambda: _names(_FC))
check("app/price_forecast.py reads NO options at all - it takes a PriceRead "
"and instants, so there is no fourth place for an option default to live",
isinstance(_used, set)
and not ({"price_option", "PRICE_OPTION_DEFAULTS", "PluginOptions"}
& _used))
# ⚠️ RULE 7'S TRIPWIRE, RESPECTED RATHER THAN TRIPPED. test_price.py section 12
# pins the set of modules that consume `load_prices()` to exactly
# ["static_tariff.py"], and goes red when a second one appears. This module
# deliberately does NOT fetch: it transforms a read somebody else obtained, so
# rule 7's message about what a configured provider does or does not drive is
# unchanged by this ticket and needed no rewording.
check("...and it does not consume load_prices(), so validate.py rule 7's "
"pinned consumer list is still true and its wording still accurate",
"load_prices" not in _used)
print()
print("FAILED" if fails else "OK")
for f in fails:
print(" -", f)
sys.exit(1 if fails else 0)
+861
View File
@@ -0,0 +1,861 @@
"""Runnable check for static day/night and fully-fixed tariffs. `python3 test_static_tariff.py`
WHAT THIS FILE IS DEFENDING, in the order the ticket's acceptance criteria name
it:
AC1 `static_profile` (§14) drives TIME-WINDOW SHIFTING for a two-period
tariff - the right window at the right local instant, on both sides of
both 2026 DST transitions, and reaching the one shiftable load the add-on
actually has.
AC2 Fully-fixed pricing DISABLES ARBITRAGE and leaves self-consumption only.
AC3 The Belgian scenario suite carries a static-profile regression case.
THREE RULES THIS FILE HOLDS ITSELF TO, because the reviewer will:
* EVERY VERDICT IS DERIVED. No `check("the split is right", True)`. Where a
check needs an expected number it is computed HERE, by different arithmetic
than app/static_tariff.py uses - the window split is re-derived by summing
over samples with an independent local-hour test, the profile mean by a
second pass, the saving by two routes that must agree.
* NO STIMULUS IS EVALUATED WHERE THE QUANTITY IT GRADES IS DEGENERATE. A
two-period tariff has this trap BUILT IN and it is the main hazard here:
- THE PERIOD BOUNDARY IS THE DEGENERACY. At 07:00 and at 22:00 the "which
window" question and the "is it cheap" question agree under almost any
bug, and a probe at 12:00 cannot tell a correct wrap-around from one
that treats 22:00-07:00 as 22:00-24:00. So probes sit at 06:59 / 07:00 /
07:01 and 21:59 / 22:00 / 22:01, and a Saturday 03:00 probe exists
specifically because the mon-fri night window MUST NOT cover it.
- MEAN-CHEAP AND MINIMUM-CHEAP AGREE ON A TWO-LEVEL TARIFF. Under two
prices "below the mean" and "is the cheapest price" are the same set, so
a two-period profile cannot tell `cheap_now` from `price == min`.
Section 3 therefore runs a THREE-level profile where those two answers
differ, and asserts they differ before grading anything with it.
- DAY AND NIGHT MUST HAVE DIFFERENT PRICES AND NEITHER MAY BE ZERO, or the
whole shifting question collapses. Asserted as checks in their own
right, first, rather than assumed.
- THE DST CHECKS RUN ON THE 23-HOUR AND 25-HOUR DAYS, not on a normal day
where a 24-slot implementation and a correct one agree exactly.
- THE REPLAY DAY MUST CARRY IMPORT IN BOTH WINDOWS. A day whose whole
import fell in one window grades no split at all; section 6 asserts that
before it reports a saving.
* A FAILED READ IS NOT AN OBSERVATION. A missing static_profile is not a zero
price, and zero is exactly the value that makes everything look free. So
`static_plan` with no profile is checked to produce a read with NO
`.points`, and `price_at` on it must RAISE rather than answer 0.0.
"""
import gzip
import json
import sys
from datetime import datetime, timedelta, timezone
from pathlib import Path
from app.price import (
Absence, PricePoint, PriceProduct, PriceRead, arbitrage_allowed, brussels,
plan_is_provisional,
)
from app.static_tariff import (
MEAN_WINDOW_DAYS, ProfileInvalid, StaticProfile, TariffPlan,
parse_static_profile, resolve_tariff, static_plan, static_status,
)
from app.validate import validate_startup_config
HERE = Path(__file__).resolve().parent
UTC = timezone.utc
fails = []
def check(name, cond):
print(f" {'ok ' if cond else 'FAIL'} {name}")
if not cond:
fails.append(name)
def raises(fn, exc):
"""True only if fn() raises EXACTLY `exc`. A different exception is False -
'it was refused' and 'it broke on the way to being refused' must not grade
the same."""
try:
fn()
except exc:
return True
except Exception:
return False
return False
def reason(fn):
try:
fn()
except Exception as e: # noqa: BLE001
return str(e)
return ""
class _Raised:
def __init__(self, e):
self.e = e
def __eq__(self, other):
return False
def __hash__(self):
return id(self)
def __bool__(self):
return False
def __repr__(self):
return f"<raised {type(self.e).__name__}: {self.e}>"
def derived(fn):
"""A value a check grades. An exception becomes `_Raised`, which equals
nothing and is falsy - NOT a default. See test_price.py for why."""
try:
return fn()
except Exception as e: # noqa: BLE001
return _Raised(e)
def near(a, b, tol=1e-9):
try:
return abs(float(a) - float(b)) <= tol
except (TypeError, ValueError):
return False
# --------------------------------------------------------------------------- #
# The fixtures. Belgian day/night numbers, and their degeneracies asserted.
# --------------------------------------------------------------------------- #
DAY_P = 0.3410 # all-in EUR/kWh, day
NIGHT_P = 0.2570 # all-in EUR/kWh, night
FIXED_P = 0.3070 # all-in EUR/kWh, single-tariff site
TWO_PERIOD = {"periods": [
{"name": "day", "from": "07:00", "to": "22:00", "days": "mon-fri",
"eur_per_kwh": DAY_P},
{"name": "night", "from": "22:00", "to": "07:00", "days": "mon-fri",
"eur_per_kwh": NIGHT_P},
{"name": "night", "from": "00:00", "to": "24:00", "days": "sat,sun",
"eur_per_kwh": NIGHT_P},
]}
FULLY_FIXED = {"periods": [
{"name": "fixed", "from": "00:00", "to": "24:00", "days": "all",
"eur_per_kwh": FIXED_P},
]}
# ⚠️ THREE LEVELS, AND SECTION 3 EXISTS BECAUSE OF IT. Under two prices "cheaper
# than the mean" and "is the minimum price" are the same set of hours, so a
# two-period profile cannot distinguish `cheap_now`'s rule from `price == min`.
# Here the middle level sits BELOW the mean without being the minimum (the four
# expensive evening hours pull the mean up past it), so those two rules give
# DIFFERENT answers at 12:00 - and section 3 proves they differ before grading
# anything with it.
THREE_LEVEL = {"periods": [
{"name": "peak", "from": "17:00", "to": "21:00", "days": "all",
"eur_per_kwh": 0.4800},
{"name": "day", "from": "07:00", "to": "17:00", "days": "all",
"eur_per_kwh": 0.2900},
{"name": "evening", "from": "21:00", "to": "24:00", "days": "all",
"eur_per_kwh": 0.2900},
{"name": "night", "from": "00:00", "to": "07:00", "days": "all",
"eur_per_kwh": 0.1900},
]}
def opts(profile=None, **over):
o = {"price_provider": "off",
"static_profile": "" if profile is None else json.dumps(profile)}
o.update(over)
return o
# --------------------------------------------------------------------------- #
print("0. the arithmetic this file grades with must not be degenerate")
# ⚠️ These ARE the guard against a section that passes for the wrong reason. If
# any goes red, every shifting assertion below is measuring nothing.
check("day and night prices differ, so 'used the wrong window' is visible",
DAY_P != NIGHT_P)
check("...and neither is zero, so 'dropped the price entirely' is visible",
DAY_P != 0 and NIGHT_P != 0)
check("...and the fixed price is neither of them, so a fixed-tariff result "
"cannot be a day/night result in disguise",
FIXED_P not in (DAY_P, NIGHT_P))
check("...and the day/night spread is not the fixed price, so 'reported the "
"spread where the price was wanted' is visible",
not near(DAY_P - NIGHT_P, FIXED_P))
_lv = sorted({p["eur_per_kwh"] for p in THREE_LEVEL["periods"]})
check(f"the three-level fixture really has three levels {_lv}", len(_lv) == 3)
# --------------------------------------------------------------------------- #
print()
print("1. AC1 - the two-period profile puts the right window on the right hour")
TWO = parse_static_profile(json.dumps(TWO_PERIOD))
check("a JSON string parses (what config.yaml's flat `str?` option carries)",
isinstance(TWO, StaticProfile))
check("...and so does an already-parsed dict (what /data/options.json would "
"carry under a nested schema, and what sim/inject.py hands over)",
isinstance(parse_static_profile(TWO_PERIOD), StaticProfile))
check("...and a bare list of periods, without the wrapper",
isinstance(parse_static_profile(TWO_PERIOD["periods"]), StaticProfile))
# ⚠️ THE PRODUCT IS DERIVED FROM THE DATA. Nobody ticks a box saying 'static'.
check("the product is derived from the number of distinct prices, not declared: "
"two levels -> STATIC", TWO.product is PriceProduct.STATIC)
check("...and one level -> FIXED, from the same code, with no flag anywhere",
parse_static_profile(FULLY_FIXED).product is PriceProduct.FIXED)
# ...and it is really the PRICES, not the row count: three rows, one price.
_three_rows_one_price = parse_static_profile({"periods": [
dict(p, eur_per_kwh=FIXED_P) for p in TWO_PERIOD["periods"]]})
check("...and THREE rows carrying ONE price is still FIXED, so the derivation "
"is on the prices and not on len(periods)",
_three_rows_one_price.product is PriceProduct.FIXED
and len(_three_rows_one_price.periods) == 3)
# ⚠️ PROBES AT THE BOUNDARY, NOT IN THE MIDDLE OF A WINDOW. 2026-01-15 is a
# Thursday and Belgium is on CET (+01:00) then, so local == UTC + 1 h.
THU = datetime(2026, 1, 15, tzinfo=UTC)
def at_local(day, hh, mm=0):
"""A UTC instant whose BRUSSELS wall clock reads hh:mm on `day`.
Built by search rather than by adding an offset, so this helper cannot
inherit the same offset bug it is meant to detect.
"""
for minutes in range(-180, 24 * 60 + 180):
u = day + timedelta(minutes=minutes)
loc = brussels(u)
if loc.hour == hh and loc.minute == mm and loc.date() == \
brussels(day + timedelta(hours=12)).date():
return u
raise AssertionError(f"no instant reads {hh:02d}:{mm:02d} local on {day}")
_probes = [(6, 59, "night"), (7, 0, "day"), (7, 1, "day"),
(21, 59, "day"), (22, 0, "night"), (22, 1, "night"),
(12, 0, "day"), (3, 0, "night")]
_got = [(hh, mm, TWO.at(at_local(THU, hh, mm)).name) for hh, mm, _ in _probes]
check(f"the half-open boundaries land exactly - 06:59 night, 07:00 day, "
f"21:59 day, 22:00 night {_got}",
all(name == want for (hh, mm, want), (_, __, name)
in zip(_probes, _got)))
check("...and the 22:00-07:00 window really wraps midnight: Thursday 03:00 is "
"night, which a [22:00,24:00) reading would call uncovered",
TWO.at(at_local(THU, 3)).name == "night")
# ⚠️ THE DAY-TYPE TRAP. A `mon-fri` night window must NOT bleed into Saturday
# morning - a period applies on the day the CLOCK reads. The weekend row is what
# covers Saturday 03:00, and it is a DIFFERENT Period object.
SAT = datetime(2026, 1, 17, tzinfo=UTC) # a Saturday
_sat3 = TWO.at(at_local(SAT, 3))
_thu3 = TWO.at(at_local(THU, 3))
check("Saturday 03:00 is served by the WEEKEND row, not by Friday's night "
"window bleeding over - same name and price, different period",
_sat3.name == _thu3.name and _sat3 is not _thu3
and 5 in _sat3.days and 5 not in _thu3.days)
check("...and Saturday 12:00 is night-priced, because a Belgian weekend tariff "
"runs all day - a weekday-shaped lookup would call it day",
near(TWO.price_at(at_local(SAT, 12)), NIGHT_P))
check("...while Thursday 12:00 is day-priced, so the two are really different "
"answers and not one answer twice",
near(TWO.price_at(at_local(THU, 12)), DAY_P))
# next_change: the shifting primitive - "how long until the cheap window".
_nc = TWO.next_change(at_local(THU, 12))
check(f"next_change from Thursday 12:00 is the 22:00 flip "
f"{None if _nc is None else brussels(_nc).isoformat()}",
_nc is not None and brussels(_nc).hour == 22 and brussels(_nc).minute == 0)
_nc2 = TWO.next_change(at_local(SAT, 12))
check("...and from Saturday noon it is Monday 07:00, because the whole weekend "
"is one window - a scan that stopped at midnight would say 24 h",
_nc2 is not None and brussels(_nc2).hour == 7
and brussels(_nc2).weekday() == 0)
# --------------------------------------------------------------------------- #
print()
print("2. AC2 - a fully-fixed tariff disables arbitrage, by arithmetic")
FIX = parse_static_profile(FULLY_FIXED)
# ⚠️ The window OPENS A DAY EARLY. `at_local(THU, 0)` is 23:00 UTC on the 14th -
# Belgium is an hour ahead - so a plan starting at THU would not cover midnight
# local, and `price_at` would correctly raise for it. That raise is the right
# behaviour (an hour outside the series is an hour with no price) but it is not
# what these sections are grading.
SPAN_A, SPAN_B = THU - timedelta(days=1), THU + timedelta(days=2)
p_fixed = static_plan(opts(FULLY_FIXED), SPAN_A, SPAN_B)
p_two = static_plan(opts(TWO_PERIOD), SPAN_A, SPAN_B)
check("the fixed plan really produced a series (so the refusal below is about "
"the tariff, not about a failed read)",
p_fixed.has_prices and len(p_fixed.read.points) == 72)
_spread_fixed = derived(p_fixed.spread_eur_kwh)
_spread_two = derived(p_two.spread_eur_kwh)
check(f"...and its spread is exactly zero {_spread_fixed}",
near(_spread_fixed, 0.0))
check(f"...while the two-period spread is the day/night difference "
f"{_spread_two}, re-derived here as {DAY_P - NIGHT_P:.4f}",
near(_spread_two, DAY_P - NIGHT_P, 1e-9))
_v_fixed = p_fixed.arbitrage(peak_guard_active=False, min_spread_eur_kwh=0.0, uncertainty=frozenset())
_v_two = p_two.arbitrage(peak_guard_active=False, min_spread_eur_kwh=0.0, uncertainty=frozenset())
check("a fully-fixed tariff REFUSES arbitrage", _v_fixed.kind == "refused")
check("...for the true reason - the series is flat, so there is nothing to buy "
"low and sell high, not because a FIXED enum was matched on",
"flat" in _v_fixed.reason and "self-consumption only" in _v_fixed.reason)
check("...and a two-period tariff, on the same code path, ALLOWS it - so the "
"refusal is discriminating rather than universal",
_v_two.kind == "allowed")
# ⚠️ NOT KEYED ON THE PRODUCT ENUM. A DAY-AHEAD series that happens to print one
# price all day must be refused for the same reason, or the rule is a label
# check wearing arithmetic's clothes.
_flat_dayahead = TariffPlan(
PriceRead.series(
[PricePoint(start=brussels(THU + timedelta(hours=i)),
eur_per_kwh=0.2222, provisional=False) for i in range(24)],
PriceProduct.DAY_AHEAD, "generic_dayahead", THU),
None, "provider")
check("a DAY_AHEAD series that happens to be flat is refused too, so the rule "
"is arithmetic and not a product label",
_flat_dayahead.arbitrage(peak_guard_active=False, min_spread_eur_kwh=0.0, uncertainty=frozenset()).kind == "refused")
# The numbers an optimiser needs exist ONLY on the permitting variant.
check("the allowed verdict carries the spread, the cheapest hour and the "
"dearest hour - the whole input set for a trade",
near(_v_two.spread_eur_kwh, DAY_P - NIGHT_P)
and near(_v_two.cheapest.eur_per_kwh, NIGHT_P)
and near(_v_two.dearest.eur_per_kwh, DAY_P))
check("...and a REFUSED verdict carries none of them: .spread_eur_kwh raises "
"Absence rather than answering 0.0, which is the number that would make "
"every hour look equally good",
raises(lambda: _v_fixed.spread_eur_kwh, Absence)
and raises(lambda: _v_fixed.cheapest, Absence))
check("...and it is not iterable and has no .get(), the other two routes from "
"a refusal to an empty, harmless-looking loop",
raises(lambda: list(_v_fixed), Absence)
and raises(lambda: _v_fixed.get("spread_eur_kwh", 0.0), Absence))
check("...and match() demands BOTH arms, so the refusal cannot be fallen "
"through", raises(lambda: _v_fixed.match(allowed=lambda v: 1), TypeError))
check("...and there is no falsy variant, so `if verdict:` cannot look like a "
"guard while being none", bool(_v_fixed) is True and bool(_v_two) is True)
# §10.1 priority 2 over priority 3, expressed so OPT-04 cannot skip it.
check("peak shaving outranks arbitrage: the SAME allowed plan is refused when "
"the peak guard is holding",
p_two.arbitrage(peak_guard_active=True, min_spread_eur_kwh=0.0, uncertainty=frozenset()).kind == "refused")
check("...and the reason names the ordering rather than the tariff",
"priority 2" in p_two.arbitrage(peak_guard_active=True, min_spread_eur_kwh=0.0, uncertainty=frozenset()).reason)
check("...and the question cannot be skipped: peak_guard_active is required "
"and keyword-only, so a spread is not obtainable without answering it",
raises(lambda: p_two.arbitrage(), TypeError)
and raises(lambda: p_two.arbitrage(False), TypeError))
check("...nor answered with a truthy 'unknown'",
raises(lambda: p_two.arbitrage(peak_guard_active=None), TypeError)
and raises(lambda: p_two.arbitrage(peak_guard_active="no"), TypeError))
# --------------------------------------------------------------------------- #
print()
print("3. the cheap-window rule, on a tariff where it is not degenerate")
# ⚠️ WHY THREE LEVELS. Under two prices, "below the mean" and "equal to the
# minimum" select the SAME hours, so a two-period profile cannot tell
# cheap_now()'s actual rule from `price == min(prices)`. This fixture's `day`
# level sits between the two, and the check below PROVES the two rules disagree
# before anything is graded with it.
THREE = parse_static_profile(THREE_LEVEL)
p3 = static_plan(opts(THREE_LEVEL), SPAN_A,
SPAN_A + timedelta(days=MEAN_WINDOW_DAYS))
_mean3 = p3.mean_eur_kwh()
_min3 = min(p.eur_per_kwh for p in p3.read.points)
_day3 = 0.2900
check(f"the three-level fixture is non-degenerate: the minimum {_min3:.4f} is "
f"strictly below the middle level {_day3:.4f}, which is strictly below "
f"the mean {_mean3:.4f} - so 'below the mean' and 'is the minimum' select "
f"DIFFERENT sets of hours and this fixture can tell them apart",
_min3 < _day3 < _mean3)
check("...so cheap_now() calls the 07:00-17:00 MIDDLE level cheap - it is under "
"the mean without being the cheapest price of the day, which a "
"`price == min(prices)` rule would get wrong and a `price < mean` rule "
"gets right", p3.cheap_now(at_local(THU, 12)) is True)
check("...and the middle level really is not the minimum, so that check graded "
"the difference rather than restating it",
not near(p3.price_at(at_local(THU, 12)), _min3))
check("...and calls the 00:00-07:00 minimum cheap",
p3.cheap_now(at_local(THU, 3)) is True)
check("...and the 17:00-21:00 peak not cheap",
p3.cheap_now(at_local(THU, 18)) is False)
# ⚠️ STRICTLY BELOW, AND THAT IS AC2 IN THE SHIFTING HALF. On a flat series
# `<=` would make EVERY hour a cheap window and a fixed-tariff site would top
# the battery up from the grid at any hour - the opposite of self-consumption
# only.
_fixed_hours = [at_local(THU, h) for h in range(0, 24, 3)]
check(f"a fully-fixed tariff has NO cheap window at any hour of the day "
f"{[h for h in range(0, 24, 3) if p_fixed.cheap_now(at_local(THU, h))]} - "
f"which is what `<` buys and `<=` would destroy",
not any(p_fixed.cheap_now(t) for t in _fixed_hours))
check("...and the price really is defined at every one of those hours, so the "
"'no cheap window' answer is a priced no and not a missing read",
all(near(p_fixed.price_at(t), FIXED_P) for t in _fixed_hours))
# ...and on the two-period tariff the cheap hours are exactly the night ones.
_cheap_h = {h for h in range(24) if p_two.cheap_now(at_local(THU, h))}
_night_h = {h for h in range(24) if TWO.at(at_local(THU, h)).name == "night"}
check(f"on the day/night tariff the cheap hours ARE the night hours, derived "
f"two ways {sorted(_cheap_h)}",
_cheap_h == _night_h and 0 < len(_cheap_h) < 24)
# ⚠️ THE MEAN IS OVER SEVEN DAYS, NOT OVER THE HOUR YOU ASKED AT. A 48 h mean on
# a Friday afternoon is ~2/3 weekend, so the same Tuesday 09:00 would come out
# cheap or dear depending on when the question was put.
FRI = datetime(2026, 1, 16, tzinfo=UTC)
_from_thu = static_status(opts(TWO_PERIOD), at_local(THU, 12))
_from_fri = static_status(opts(TWO_PERIOD), at_local(FRI, 15))
_mean_thu = static_plan(opts(TWO_PERIOD), at_local(THU, 12),
at_local(THU, 12) + timedelta(days=MEAN_WINDOW_DAYS)
).mean_eur_kwh()
_mean_fri = static_plan(opts(TWO_PERIOD), at_local(FRI, 15),
at_local(FRI, 15) + timedelta(days=MEAN_WINDOW_DAYS)
).mean_eur_kwh()
check(f"the 7-day mean is stable across the day it is asked on "
f"({_mean_thu:.4f} vs {_mean_fri:.4f}), so a window's cheapness is a "
f"property of the tariff and not of the clock",
near(_mean_thu, _mean_fri, 5e-4))
_mean48_thu = static_plan(opts(TWO_PERIOD), at_local(THU, 12),
at_local(THU, 12) + timedelta(days=2)).mean_eur_kwh()
_mean48_fri = static_plan(opts(TWO_PERIOD), at_local(FRI, 15),
at_local(FRI, 15) + timedelta(days=2)).mean_eur_kwh()
check(f"...and the 48 h mean this replaces is NOT stable ({_mean48_thu:.4f} vs "
f"{_mean48_fri:.4f}), so the seven days are buying something real",
not near(_mean48_thu, _mean48_fri, 5e-4))
# --------------------------------------------------------------------------- #
print()
print("4. DST - 23 and 25 hour days, where a 24-slot implementation is wrong")
# ⚠️ 2026: CEST from 01:00 UTC Sunday 29 March, CET from 01:00 UTC Sunday
# 25 October (price.py's brussels_offset, checked in PR #16 against zoneinfo
# over 149,040 instants). Both are SUNDAYS, so the weekend row prices them -
# which is itself worth knowing, because the wrap-around weekday row is not what
# is under test here.
SPRING = datetime(2026, 3, 29, tzinfo=UTC)
AUTUMN = datetime(2026, 10, 25, tzinfo=UTC)
def local_day_points(profile, day_utc):
"""Hourly points whose BRUSSELS date is that of `day_utc` + 12 h."""
target = brussels(day_utc + timedelta(hours=12)).date()
pts = profile.series(day_utc - timedelta(hours=6),
day_utc + timedelta(hours=30), provisional=False)
return [p for p in pts if p.start.date() == target]
_spring = derived(lambda: local_day_points(TWO, SPRING))
_autumn = derived(lambda: local_day_points(TWO, AUTUMN))
check(f"the spring-forward local day emits 23 points, not 24 "
f"({len(_spring) if not isinstance(_spring, _Raised) else _spring})",
not isinstance(_spring, _Raised) and len(_spring) == 23)
check(f"the autumn-back local day emits 25 points, not 24 "
f"({len(_autumn) if not isinstance(_autumn, _Raised) else _autumn})",
not isinstance(_autumn, _Raised) and len(_autumn) == 25)
check("...and the two 02:00 points in October are DIFFERENT instants at "
"DIFFERENT offsets, which a naive local-hour key would collide",
len({p.start.utcoffset() for p in _autumn if p.start.hour == 2}) == 2)
check("...and every emitted start is strictly increasing in UTC, which is what "
"price.py's boundary check demands of a fetched feed too",
all(a.start.astimezone(UTC) < b.start.astimezone(UTC)
for a, b in zip(_autumn, _autumn[1:])))
# A weekday profile across a transition, so the wrap-around path sees one too.
_wrapdays = local_day_points(TWO, datetime(2026, 3, 30, tzinfo=UTC))
check("a normal Monday still emits 24, so the 23/25 above are the transitions "
"and not a systematic miscount", len(_wrapdays) == 24)
# --------------------------------------------------------------------------- #
print()
print("5. a failed read is not an observation, and a gap is not a price")
_none = static_plan(opts(None), THU, THU + timedelta(days=1))
check("no static_profile produces a NO-DATA read, not an empty series and not "
"a zero one", not _none.has_prices)
check("...and reading a price off it RAISES rather than answering 0.0 - the "
"value that would make every hour look free",
raises(lambda: _none.price_at(THU), Absence)
and raises(_none.mean_eur_kwh, Absence)
and raises(_none.spread_eur_kwh, Absence))
check("...and its underlying read has no .points at all, so `min(read.points)` "
"cannot run against it",
raises(lambda: _none.read.points, Absence))
check("...and arbitrage is refused, per §11.4",
_none.arbitrage(peak_guard_active=False, min_spread_eur_kwh=0.0, uncertainty=frozenset()).kind == "refused"
and arbitrage_allowed(_none.read) is False)
check("...and static_status() says so in words rather than reporting a price",
static_status(opts(None), THU) == (False, "no static profile"))
# ⚠️ THE TILING REFUSAL. An uncovered hour has no price and the only ways to
# serve it are to invent one or to fail when the clock reaches it.
_gap = {"periods": [
{"name": "day", "from": "07:00", "to": "22:00", "days": "all",
"eur_per_kwh": DAY_P},
{"name": "night", "from": "23:00", "to": "07:00", "days": "all",
"eur_per_kwh": NIGHT_P}]}
check("a profile with a 22:00-23:00 gap is REFUSED, not defaulted",
raises(lambda: parse_static_profile(_gap), ProfileInvalid))
check("...and names the uncovered range so the operator can fix it",
"22:00-23:00" in reason(lambda: parse_static_profile(_gap)))
_overlap = {"periods": [
{"name": "day", "from": "07:00", "to": "22:00", "days": "all",
"eur_per_kwh": DAY_P},
{"name": "night", "from": "21:00", "to": "07:00", "days": "all",
"eur_per_kwh": NIGHT_P}]}
check("...and an overlap is refused too - two prices for one hour means one "
"wins on list order",
raises(lambda: parse_static_profile(_overlap), ProfileInvalid)
and "2 prices" in reason(lambda: parse_static_profile(_overlap)))
# THE CLASSIC MISTAKE the docstring warns about: no weekend row.
_noweekend = {"periods": [
{"name": "day", "from": "07:00", "to": "22:00", "days": "mon-fri",
"eur_per_kwh": DAY_P},
{"name": "night", "from": "22:00", "to": "07:00", "days": "mon-fri",
"eur_per_kwh": NIGHT_P}]}
check("a weekday-only profile is refused because Saturday and Sunday have no "
"price - the mistake a wrap-around window invites",
raises(lambda: parse_static_profile(_noweekend), ProfileInvalid)
and "sat" in reason(lambda: parse_static_profile(_noweekend)))
# ...and the shipped shape is accepted, so the rule is not simply "refuse all".
check("...while the three-row Belgian shape IS accepted, so the tiling rule "
"discriminates rather than refusing everything",
isinstance(parse_static_profile(TWO_PERIOD), StaticProfile))
_bad = [
("not JSON at all", "day 07:00-22:00 0.34"),
("a price in cents", {"periods": [dict(FULLY_FIXED["periods"][0],
eur_per_kwh=30.7)]}),
("a non-numeric price", {"periods": [dict(FULLY_FIXED["periods"][0],
eur_per_kwh="cheap")]}),
("a missing days selector", {"periods": [
{"name": "x", "from": "00:00", "to": "24:00", "eur_per_kwh": 0.3}]}),
("an unknown day name", {"periods": [dict(FULLY_FIXED["periods"][0],
days="mon-frd")]}),
("a decimal hour", {"periods": [dict(FULLY_FIXED["periods"][0],
**{"from": "7.5"})]}),
("an unsupported timezone", dict(FULLY_FIXED, timezone="Europe/Madrid")),
("an empty periods list", {"periods": []}),
]
for label, value in _bad:
payload = value if isinstance(value, str) else json.dumps(value)
check(f"refused: {label}",
raises(lambda p=payload: parse_static_profile(p), ProfileInvalid))
check("...and every one of those refusals says something an operator can act "
"on, rather than just failing",
all(len(reason(lambda p=(v if isinstance(v, str) else json.dumps(v)):
parse_static_profile(p))) > 40 for _, v in _bad))
for label, value in (("empty", ""), ("null", "null"), ("absent", None),
("whitespace", " ")):
check(f"...while {label!r} means 'no profile' rather than an error",
parse_static_profile(value) is None)
# --------------------------------------------------------------------------- #
print()
print("6. AC3 - the static-profile regression case, on the Belgian data that exists")
# ⚠️ WHAT THE "BELGIAN SCENARIO SUITE" ACTUALLY IS IN THIS REPO. specs §17 names
# six scenarios; none of them is a static-profile case, which is why AC3 asks
# for a seventh. What exists as DATA is two days of this house's real P1 net
# power, captured from the live meter and committed as
# `scenarios/p1-net-2026-08-{20,23}.jsonl.gz` - the same fixtures test_tariff.py
# part 9 replays for the capacity tariff. Nothing longer was ever captured.
#
# So this case is measured against REAL BELGIAN HOUSEHOLD LOAD, and the two days
# happen to be one weekday and one weekend day, which is exactly the pair a
# day/night tariff needs to be graded on. It is NOT a 30-day replay and this
# section does not claim to be one; the numbers it prints are the numbers those
# two days actually produce.
REPLAY = sorted((HERE / "scenarios").glob("p1-net-*.jsonl.gz"))
print(f" replay fixtures found: {len(REPLAY)} "
f"({', '.join(p.name for p in REPLAY) or 'none'})")
check("the committed replay fixtures are present - a regression fixture CI "
"cannot see is not a regression fixture", len(REPLAY) >= 2)
MAX_GAP_S = 60.0 # a longer hole in the capture is not counted as energy
def replay_import(path):
"""(kWh imported per local tariff window, total kWh, span hours, weekday).
⚠️ THE WINDOW IS DECIDED HERE BY AN INDEPENDENT ROUTE - the Brussels-local
hour of the sample, tested against the fixture's own 07:00/22:00 boundaries
- rather than by asking the module under test. The two must agree; that
agreement is the check.
"""
per_window = {}
total = 0.0
first = last = None
prev = None
days = set()
with gzip.open(path, "rt", encoding="utf-8") as fh:
for line in fh:
row = json.loads(line)
t = datetime.fromisoformat(row["t"])
w = float(row["w"])
if first is None:
first = t
last = t
if prev is not None:
dt = (t - prev[0]).total_seconds()
if 0 < dt <= MAX_GAP_S and prev[1] > 0:
kwh = prev[1] * dt / 3_600_000.0
loc = brussels(prev[0])
days.add(loc.weekday())
window = ("day" if loc.weekday() < 5
and 7 <= loc.hour < 22 else "night")
per_window[window] = per_window.get(window, 0.0) + kwh
total += kwh
prev = (t, w)
span = (last - first).total_seconds() / 3600.0 if first else 0.0
return per_window, total, span, days
def cost_under(path, profile):
"""EUR the imported energy cost, priced sample by sample by the MODULE."""
total = 0.0
prev = None
with gzip.open(path, "rt", encoding="utf-8") as fh:
for line in fh:
row = json.loads(line)
t = datetime.fromisoformat(row["t"])
w = float(row["w"])
if prev is not None:
dt = (t - prev[0]).total_seconds()
if 0 < dt <= MAX_GAP_S and prev[1] > 0:
total += (prev[1] * dt / 3_600_000.0) * profile.price_at(prev[0])
prev = (t, w)
return total
replay_days = 0.0
graded = 0
for path in REPLAY:
windows, total_kwh, span_h, weekdays = replay_import(path)
replay_days += span_h / 24.0
day_kwh = windows.get("day", 0.0)
night_kwh = windows.get("night", 0.0)
kind = "weekday" if weekdays and min(weekdays) < 5 else "weekend"
print(f" {path.name}: {span_h:.1f} h, {kind} "
f"(weekday indices {sorted(weekdays)}), {total_kwh:.3f} kWh imported "
f"= {day_kwh:.3f} day + {night_kwh:.3f} night")
check(f"{path.name}: the two routes to the total agree",
near(day_kwh + night_kwh, total_kwh, 1e-9))
# The module's own pricing must reproduce the independently-split volumes.
c_two = cost_under(path, TWO)
c_expect = day_kwh * DAY_P + night_kwh * NIGHT_P
check(f"{path.name}: the module prices the real day at "
f"EUR {c_two:.4f}, which equals the independently-split "
f"{day_kwh:.3f} kWh x {DAY_P} + {night_kwh:.3f} kWh x {NIGHT_P} = "
f"EUR {c_expect:.4f}", near(c_two, c_expect, 1e-9))
c_fix = cost_under(path, FIX)
check(f"{path.name}: under a FULLY FIXED tariff the same energy costs "
f"EUR {c_fix:.4f} = {total_kwh:.3f} kWh x {FIXED_P}, one price for "
f"every hour", near(c_fix, total_kwh * FIXED_P, 1e-9))
# ⚠️ THE SHIFTING FIGURE, AND THE DEGENERACY GUARD IN FRONT OF IT. A day
# whose entire import landed in one window grades nothing: the saving would
# be 0 under a correct implementation AND under one that ignored the clock.
if day_kwh > 0 and night_kwh > 0:
graded += 1
saving = c_two - total_kwh * NIGHT_P
check(f"{path.name}: this day DOES import in both windows "
f"({day_kwh:.3f} / {night_kwh:.3f} kWh), so the split grades "
f"something", True)
check(f"{path.name}: shifting every day-window kWh into the night "
f"window saves EUR {saving:.4f}, which is exactly "
f"{day_kwh:.3f} kWh x the {DAY_P - NIGHT_P:.4f} spread - two "
f"routes to one number",
near(saving, day_kwh * (DAY_P - NIGHT_P), 1e-9) and saving > 0)
# AC2 on the same real data: under a fixed tariff there is nothing to
# shift, and that is arithmetic rather than a special case.
check(f"{path.name}: under the FIXED tariff the identical shift saves "
f"exactly nothing, so self-consumption-only is the right "
f"behaviour for that product",
near(c_fix - total_kwh * FIXED_P, 0.0, 1e-12))
else:
print(f" {path.name}: import fell in ONE window only "
f"({day_kwh:.3f} day / {night_kwh:.3f} night) - this day is a "
f"day-typing witness, not a shifting one, and no saving is "
f"claimed from it")
print()
print(f" AC3 MEASURED: {replay_days:.1f} DAYS of real Belgian household "
f"P1 data, {graded} of {len(REPLAY)} days carrying import in BOTH tariff "
f"windows and therefore grading the split.")
print(f" AC3 ASKS FOR A REGRESSION CASE IN THE BELGIAN SUITE, AND THIS IS "
f"ONE - but the suite holds two captured days, not a season, so nothing "
f"here is a claim about annual savings. §17's six named scenarios do not "
f"include a static-profile case; this is the seventh, and TEST-01 should "
f"say so.")
check("the replay produced a measurable split over a stated sample size, "
"rather than a criterion quietly redefined to fit the data",
graded >= 1 and replay_days < 30)
check("...and BOTH day types are present in the fixtures, which is what makes "
"the weekend row testable on real data at all",
len({("weekday" if min(replay_import(p)[3]) < 5 else "weekend")
for p in REPLAY}) == 2)
# --------------------------------------------------------------------------- #
print()
print("7. §11.4 - the profile is the fallback, and the fallback is provisional")
async def _dead(url, timeout_s):
raise OSError("the supplier's feed is down")
def run(coro):
import asyncio
return asyncio.run(coro)
_cfg = opts(TWO_PERIOD, price_provider="generic_dayahead",
price_feed_url="https://api.energy-charts.info/price?bzn=BE",
price_timeout_s=1, price_markup_eur_kwh=0.03,
price_taxes_eur_kwh=0.06)
_fb = run(resolve_tariff(_cfg, THU, THU + timedelta(days=1), transport=_dead))
check("a dead feed falls back to the static profile rather than to nothing",
_fb.origin == "fallback" and _fb.has_prices)
check("...with the SAME prices the configured-product path produces, because "
"it is one generator and not a second implementation",
[p.eur_per_kwh for p in _fb.read.points]
== [p.eur_per_kwh for p in static_plan(
opts(TWO_PERIOD), THU, THU + timedelta(days=1)).read.points])
check("...marked PROVISIONAL, which is the one flag that differs",
_fb.provisional and plan_is_provisional(_fb.read))
check("...and arbitrage is refused on it, per §11.4, even though a full series "
"exists and has a real spread",
_fb.arbitrage(peak_guard_active=False, min_spread_eur_kwh=0.0, uncertainty=frozenset()).kind == "refused"
and near(_fb.spread_eur_kwh(), DAY_P - NIGHT_P))
check("...while the SAME profile as the configured product allows it - so the "
"refusal is about provisionality and not about the profile",
static_plan(opts(TWO_PERIOD), THU, THU + timedelta(days=1)
).arbitrage(peak_guard_active=False, min_spread_eur_kwh=0.0, uncertainty=frozenset()).kind == "allowed")
check("...and the provisional plan has no cheap window either: §8.3 forbids an "
"irreversible action on a provisional price, and a grid top-up is one",
not any(_fb.cheap_now(at_local(THU, h)) for h in range(24)))
_fb_none = run(resolve_tariff(opts(None, price_provider="generic_dayahead",
price_timeout_s=1),
THU, THU + timedelta(days=1), transport=_dead))
check("a dead feed with NO static_profile has nothing to fall back to, and "
"says that rather than inventing a tariff",
_fb_none.origin == "none" and not _fb_none.has_prices)
# A live feed still wins over the profile.
_body = json.dumps({"data": [
{"start": (THU + timedelta(hours=i)).isoformat(), "price_eur_mwh": 40 + i}
for i in range(24)]}).encode()
async def _live(url, timeout_s):
return _body
_live_plan = run(resolve_tariff(_cfg, THU, THU + timedelta(days=1),
transport=_live))
check("a live feed is used in preference to the static profile",
_live_plan.origin == "provider"
and _live_plan.product is PriceProduct.DAY_AHEAD)
check("...and arbitrage is allowed on it, with the spread derived from the "
"feed rather than from the profile",
_live_plan.arbitrage(peak_guard_active=False, min_spread_eur_kwh=0.0, uncertainty=frozenset()).kind == "allowed"
and near(_live_plan.arbitrage(peak_guard_active=False, min_spread_eur_kwh=0.0, uncertainty=frozenset()).spread_eur_kwh,
0.023, 1e-9))
# --------------------------------------------------------------------------- #
print()
print("8. the startup gate, and CFG-03")
def vbase(**over):
o = {"setpoint_entity": "number.goodwe_setpoint", "max_w": 2000,
"soc_entity": "sensor.soc", "batt_entity": "sensor.batt",
"meter_source": "off",
"maintenance_charge_w": 2000, "maintenance_discharge_w": 2000}
o.update(over)
return o
def why(problems, needle):
return any(needle in p for p in problems)
check("a good static_profile starts",
validate_startup_config(
vbase(static_profile=json.dumps(TWO_PERIOD)), -5000, 5000) == [])
check("a gapped one is REFUSED at startup, not at 22:00 on the day it bites",
why(validate_startup_config(vbase(static_profile=json.dumps(_gap)),
-5000, 5000),
"static_profile cannot be used"))
check("...and the refusal names the uncovered range",
why(validate_startup_config(vbase(static_profile=json.dumps(_gap)),
-5000, 5000), "22:00-23:00"))
check("static_profile alongside the Home Assistant price entities is refused - "
"main.py reads one of them and the loser is silent",
why(validate_startup_config(
vbase(static_profile=json.dumps(TWO_PERIOD),
price_now_entity="sensor.now", price_avg_entity="sensor.avg"),
-5000, 5000), "HOME ASSISTANT"))
check("...and the Home Assistant pair ALONE still starts, so nothing is taken "
"away from a live install",
validate_startup_config(vbase(price_now_entity="sensor.now",
price_avg_entity="sensor.avg"),
-5000, 5000) == [])
check("...and an empty static_profile is never refused: §9.2 puts a large share "
"of the installed base on a fixed tariff, and §11.4 makes no-price-data a "
"supported state",
validate_startup_config(vbase(static_profile=""), -5000, 5000) == []
and validate_startup_config(vbase(), -5000, 5000) == [])
check("...and a fully-fixed profile starts, since a one-price site is a "
"supported product and not a misconfiguration",
validate_startup_config(vbase(static_profile=json.dumps(FULLY_FIXED)),
-5000, 5000) == [])
# ⚠️ CFG-03 IS NOT WIDENED BY THIS TICKET. `static_profile` was added to
# app/price.py's PRICE_OPTION_DEFAULTS - the ONE in-code mirror TARIFF-02
# created and pinned to config.yaml - rather than to a second defaults dict in
# the new module. That mirror is enforced by test_price.py section 1, which
# reads config.yaml; this check is the other half of the claim: the new module
# must carry NO defaults dict of its own.
_SRC = (HERE / "app" / "static_tariff.py").read_text(encoding="utf-8")
check("the new module holds no option-defaults dict of its own, so this ticket "
"added ZERO new places an option default lives (CFG-03)",
"OPTION_DEFAULTS" not in _SRC.replace("PRICE_OPTION_DEFAULTS", ""))
check("...and it reads the option through price.price_option(), the framework's "
"single read path, rather than through opts.get(key, <a number>)",
"price_option(opts" in _SRC and 'opts.get("static_profile"' not in _SRC)
_MAIN = (HERE / "app" / "main.py").read_text(encoding="utf-8")
check("...and main.py grew no `o.get(\"static_profile\", <a default>)` fallback "
"either - it calls static_status(o, now), and the option is read once, "
"inside the framework",
'get("static_profile"' not in _MAIN and "static_status(" in _MAIN)
# --------------------------------------------------------------------------- #
print()
if fails:
print(f"{len(fails)} FAILED: " + "; ".join(fails))
sys.exit(1)
print("all checks passed")
+910
View File
@@ -0,0 +1,910 @@
"""Runnable check for TARIFF-01, the capacity-tariff measurement.
python3 test_tariff.py
Ten parts. What each is for, and - where it matters - what it is deliberately
NOT allowed to be tested at:
1. THE BLOCK ARITHMETIC, against a worked example written out segment by
segment. The fixture is chosen to be non-degenerate on three axes at once:
irregular sample spacing, a sample that STRADDLES the :15 boundary, and a
gap longer than `max_hold_s` so the block's coverage is short of 900 s.
2. THE 900 s DEGENERACY, named and then avoided. `acc/elapsed`, `acc/900` and
the end-of-quarter projection are THREE DIFFERENT FORMULAS that all agree
at exactly full coverage of a complete block. A check run only there proves
nothing about any of them, so part 2 first shows them agreeing at that
point (so the trap is documented rather than asserted around), then shows
part 1's fixture separating all three - and every later part uses fixtures
that are away from it.
3. RESTART MID-BLOCK, through the real `QuarterAverager.snapshot()`/`restore()`
and a real SQLite file: a restarted run must land on the same closed block
as an uninterrupted one, and a run that DOES NOT restore must land lower.
Without that second half, "restore works" is satisfied by a fixture where
restoring changes nothing.
4. MONTH ROLLOVER, in LOCAL time, including a restart that straddles it.
5. AN UNREADABLE STORE IS NOT A ZERO PEAK - the acceptance argument. Three
states, three different answers, and no accessor that turns "the database
did not answer" into a number.
6. THE TIMEZONE FALLBACK, graded against `zoneinfo` day by day.
7. THE PER-PHASE REDUCTION on an UNBALANCED load. §17's three-phase case. On a
balanced load net/sum/max all coincide, which is why a balanced fixture is
worth nothing here.
8. `monthly_peak` vs a recompute from `quarter_blocks` - the derived record
proven equal to the tier it derives from, and proven never to be lowered.
9. AC 3, ON THE REPLAY DATA THAT ACTUALLY EXISTS. See the honesty note below.
10. THE WIRING, through the real `Controller` and its real status page.
------------------------------------------------------------------------------
⚠️ AC 3 IS NOT MET, AND PART 9 DOES NOT PRETEND OTHERWISE.
AC 3 asks for "the monthly peak held under the soft limit in >= 95 % of
intervals over a 30-day simulation replay". THERE IS NO 30-DAY DATASET. The
infra repo's `sim/scenarios/` holds exactly two days of this house's real meter
- 2026-08-20 (the Illusmart failure day) and 2026-08-23 (the healthy reference)
- exported on 2026-08-25 because they were about to age out of HA's recorder,
and nothing longer was ever captured. `sim/scenarios/README.md` lists the whole
inventory; the rest is RS485 bus frames, OTA witness logs and a 578-row rollback
poll, none of which carries grid power over calendar time.
So part 9 replays the two days that exist, reports the percentage over them, and
states the sample size in days next to it. It is a MEASUREMENT, not a verdict:
nothing in it exits non-zero on the 95 % figure, because a 2-day run cannot pass
or fail a 30-day criterion and a check that claimed it could would be the more
expensive of the two mistakes. Both days are also *uncontrolled* - the add-on
was not shaving peaks on either - so the figure they give is the BASELINE the
guard has to improve on, which is the useful thing a 2-day replay can say.
The two days are committed here as `scenarios/p1-net-<date>.jsonl.gz`
(`{"t": iso8601, "w": signed_watts}` per line, derived from the HA recorder
export) for the reason `scenarios/p1-age-2026-08-26.jsonl.gz` already is: the
infra repo is private, CI's token is minted for this repository only, and a
regression fixture that CI cannot see is not a regression fixture.
Closing AC 3 needs 30 days of capture. That is TEST-01's replay harness, and it
cannot start before somebody starts recording.
"""
import asyncio
import gzip
import json
import shutil
import sys
import tempfile
from datetime import datetime, timedelta, timezone
from pathlib import Path
from app.main import Controller
from app.p1 import P1Ingest, QuarterAverager, QUARTER_S, make_sample, split_signed
from app.tariff import (
BILLING_TZ, CHECKPOINT_S, NO_RECORD, NOT_OBSERVED, POLICY_MAX,
POLICY_NET, POLICY_SUM, PeakStore, PeakTracker, _EUTimezone, billed_w,
month_key,
)
fails = []
HERE = Path(__file__).resolve().parent
TMP = Path(tempfile.mkdtemp(prefix="goodwe-tariff-"))
# The live install's numbers: NOTES.md and freeze.py both record max_w 5000
# there. `peak_cap_w` ships at 3500 in config.yaml and is the SOFT LIMIT AC 3
# grades against - read out of the shipped file in part 10 rather than typed.
LIVE_MAX_W = 5000.0
def check(name, cond):
print(f" {'ok ' if cond else 'FAIL'} {name}")
if not cond:
fails.append(name)
def near(a, b, tol=1e-6):
return a is not None and b is not None and abs(a - b) <= tol
def sample(t: datetime, net_w: float, phases=1, phase_net=None):
"""One P1 telegram, through the SHIPPED derivation.
`split_signed` and `make_sample` are what the real transports call, so the
sign convention under test is the one that ships rather than a local
re-derivation of it.
"""
imp, exp = split_signed(net_w)
pi = pe = None
if phase_net is not None:
pairs = [split_signed(v) for v in phase_net]
pi = [a for a, _ in pairs]
pe = [b for _, b in pairs]
return make_sample("ha_signed", imp, exp, phases=phases,
phase_import_w=pi, phase_export_w=pe,
ingest_ts=t, ingest_mono=t.timestamp())
def fresh_store(name: str) -> PeakStore:
return PeakStore(str(TMP / f"{name}.db"))
# B is a clock-aligned quarter boundary: proved, not assumed, because every
# other number in part 1 rests on it.
B = datetime(2026, 8, 30, 9, 0, 0, tzinfo=timezone.utc)
# ==============================================================================
print("1. the block arithmetic, on a worked example")
check(f"the fixture's start {B.isoformat()} really is 900 s aligned",
int(B.timestamp()) % QUARTER_S == 0)
# (t seconds after B, net W). Deliberately irregular, with an export, a gap
# longer than max_hold_s, and a last sample that straddles the boundary.
MAX_HOLD_S = 30.0
FIXTURE = [(0, 1000.0), (20, 2000.0), (30, -500.0), (50, 3000.0),
(250, 4000.0), (880, 1200.0), (1000, 900.0)]
# The worked example, written as the segments a person would compute by hand:
# each HELD value and how many seconds of the FIRST block it is evidence for.
# Two rules do all the work: a sample is held forward until the next one, capped
# at MAX_HOLD_S; and export is 0 W of offtake, never a negative contribution.
SEGMENTS = [
(1000.0, 20), # 0 -> 20, a normal telegram interval, held in full
(2000.0, 10), # 20 -> 30, likewise
(0.0, 20), # 30 -> 50: -500 W is EXPORT, so 0 W of offtake for 20 s
(3000.0, 30), # 50 -> 250 is a 200 s GAP; only the first 30 s is evidence
(4000.0, 30), # 250 -> 880 is a 630 s gap; again only 30 s is evidence
(1200.0, 20), # 880 -> 900: the STRADDLE's first half, up to the boundary
]
# The straddle's second half: the same held 1200 W, from the boundary until its
# hold expires at 880 + 30. It must be credited to the NEXT block, not to this
# one and not to neither.
SPLIT_TAIL_W, SPLIT_TAIL_S = 1200.0, 10
EXP_WS = sum(w * s for w, s in SEGMENTS)
EXP_COVERED_S = sum(s for _, s in SEGMENTS)
EXP_BLOCK_AVG = EXP_WS / QUARTER_S # what the utility bills
EXP_ELAPSED_AVG = EXP_WS / EXP_COVERED_S # what a /elapsed divisor gives
NAIVE_MEAN = sum(max(w, 0.0) for _, w in FIXTURE[:-1]) / (len(FIXTURE) - 1)
avg = QuarterAverager(phases=1, max_hold_s=MAX_HOLD_S)
closed = []
for dt_s, w in FIXTURE:
closed += avg.add(sample(B + timedelta(seconds=dt_s), w))
print(f" {len(SEGMENTS)} held segments -> {EXP_WS:,.0f} Ws over "
f"{EXP_COVERED_S:.0f} s of the {QUARTER_S} s block")
check(f"exactly one block closed", len(closed) == 1)
check(f"...starting at the fixture's own boundary {B.isoformat()}",
bool(closed) and closed[0].start == B)
check(f"...averaging {EXP_BLOCK_AVG:.2f} W - the worked example's "
f"{EXP_WS:,.0f} Ws over the FULL {QUARTER_S} s",
bool(closed) and near(closed[0].offtake_avg_w, EXP_BLOCK_AVG, 1e-9))
# The three ways of being wrong, each of which produces a plausible number.
print(f" the same data read three other ways: /elapsed = {EXP_ELAPSED_AVG:.2f} W, "
f"naive mean of samples = {NAIVE_MEAN:.2f} W, "
f"billed = {EXP_BLOCK_AVG:.2f} W")
check("...and /elapsed is NOT the billed figure - a gap must drag the average "
"down, not be averaged away",
abs(EXP_ELAPSED_AVG - EXP_BLOCK_AVG) > 1.0)
check("...and a naive mean over samples is NOT the billed figure either - it "
"weights a burst of fast telegrams like a slow one",
abs(NAIVE_MEAN - EXP_BLOCK_AVG) > 1.0)
# The straddle: the 4000 W sample at t=400 is held ACROSS 900. Its evidence
# expires at 430, so the second block gets none of it - and the second block
# must still have been OPENED at 900, not at 1000.
check(f"the block after the straddle opens at the boundary, not at the sample",
avg.block_start == B + timedelta(seconds=QUARTER_S))
print(f" the straddling sample is SPLIT at the boundary: "
f"{SEGMENTS[-1][0]:.0f} W x {SEGMENTS[-1][1]} s into the first block, "
f"x {SPLIT_TAIL_S} s into the second")
check(f"...and the second block is credited the tail "
f"({SPLIT_TAIL_W * SPLIT_TAIL_S:,.0f} Ws), so a sample spanning :15 is "
f"attributed to BOTH blocks in proportion and wholly to neither",
near(avg.partial_ws, SPLIT_TAIL_W * SPLIT_TAIL_S, 1e-9))
# ==============================================================================
print()
print("2. the 900 s degeneracy - where three different formulas agree")
# A complete block at a constant power, fully covered. Here acc/elapsed,
# acc/900 and the projection are ALL the same number. Anything checked only
# here is checked against nothing.
HELD_W = 2400.0
deg = QuarterAverager(phases=1, max_hold_s=MAX_HOLD_S)
deg_closed = []
for i in range(0, QUARTER_S + 1, 5):
deg_closed += deg.add(sample(B + timedelta(seconds=i), HELD_W))
d_block = deg_closed[0].offtake_avg_w
deg2 = QuarterAverager(phases=1, max_hold_s=MAX_HOLD_S)
LAST_S = QUARTER_S - 5
for i in range(0, LAST_S + 1, 5):
deg2.add(sample(B + timedelta(seconds=i), HELD_W))
d_partial = deg2.offtake_avg_w
# Graded AT the last sample: full coverage behind it, a held value in front.
d_proj = PeakTracker.project(deg2, B + timedelta(seconds=LAST_S)).projected_w
print(f" at full coverage of a whole block: acc/900 = {d_block:.2f} W, "
f"acc/elapsed = {d_partial:.2f} W, projection = {d_proj:.2f} W")
check("all three formulas coincide at exactly 900 s of full coverage - so a "
"check placed only there distinguishes none of them",
near(d_block, HELD_W, 1e-6) and near(d_partial, HELD_W, 1e-6)
and near(d_proj, HELD_W, 1e-6))
# Away from it, on part 1's fixture at a point mid-block, they must separate.
mid = QuarterAverager(phases=1, max_hold_s=MAX_HOLD_S)
for dt_s, w in FIXTURE[:-1]:
mid.add(sample(B + timedelta(seconds=dt_s), w))
NOW = B + timedelta(seconds=500)
p = PeakTracker.project(mid, NOW)
print(f" at t=500 s on part 1's fixture: acc/elapsed = {p.elapsed_w:.2f} W, "
f"projection = {p.projected_w:.2f} W, coverage = {p.covered_frac * 100:.1f} %")
check("mid-block, with a gap in it, the elapsed average and the projection are "
"DIFFERENT numbers", abs(p.elapsed_w - p.projected_w) > 1.0)
check("...and the projection carries elapsed_s, so a consumer can see it is an "
f"extrapolation over {QUARTER_S - p.elapsed_s:.0f} s it has not seen",
near(p.elapsed_s, 500.0, 1e-6))
check("...and a gap shows up as coverage below 1.0 rather than as a confident "
"low number", p.covered_frac < 1.0 and not p.complete)
# ==============================================================================
print()
print("3. restart mid-block: the same block, or a lower one, never a lost one")
RUN_W = 2000.0
STEP_S = 1
RESTART_AT_S = 600 # minute 10 of the quarter - the functional note's case
def feed(avg, t0_s, t1_s, w=RUN_W):
out = []
for i in range(t0_s, t1_s + 1, STEP_S):
out += avg.add(sample(B + timedelta(seconds=i), w))
return out
# (a) the control: no restart at all.
ctrl = QuarterAverager(phases=1, max_hold_s=MAX_HOLD_S)
ctrl_closed = feed(ctrl, 0, QUARTER_S)
CTRL_W = ctrl_closed[0].offtake_avg_w
# (b) restarted at minute 10, checkpoint restored through a real SQLite file.
st = fresh_store("restart")
tr = PeakTracker(st, cap_w=LIVE_MAX_W)
a1 = QuarterAverager(phases=1, max_hold_s=MAX_HOLD_S)
feed(a1, 0, RESTART_AT_S)
check("the checkpoint round-trips through SQLite", st.save_partial(a1.snapshot()))
a2 = QuarterAverager(phases=1, max_hold_s=MAX_HOLD_S) # a brand-new process
restored = tr.restore(a2, B + timedelta(seconds=RESTART_AT_S + 1))
print(f" restore() said: {restored}")
check("...and restore() reports RESUMING rather than starting fresh",
restored.startswith("resumed"))
r_closed = feed(a2, RESTART_AT_S + STEP_S, QUARTER_S)
RESTART_W = r_closed[0].offtake_avg_w if r_closed else None
# (c) the mutation guard: the SAME restart with no restore.
a3 = QuarterAverager(phases=1, max_hold_s=MAX_HOLD_S)
n_closed = feed(a3, RESTART_AT_S + STEP_S, QUARTER_S)
NORESTORE_W = n_closed[0].offtake_avg_w if n_closed else None
print(f" uninterrupted {CTRL_W:.2f} W | restored {RESTART_W:.2f} W | "
f"restart with no restore {NORESTORE_W:.2f} W")
check(f"a restart at {RESTART_AT_S} s lands on the SAME billed block as an "
f"uninterrupted run", near(RESTART_W, CTRL_W, 1e-6))
check("...and without the restore it lands LOWER - so 'the restore works' is "
"not satisfied by a fixture where restoring changes nothing",
NORESTORE_W is not None and NORESTORE_W < CTRL_W - 1.0)
# Derived, not asserted against a literal: the lost part is exactly the seconds
# before the restart, at the power that was flowing.
DROPPED_S = RESTART_AT_S + STEP_S # the resumed feed starts one step later
check(f"...and it is lower by exactly the {DROPPED_S} s it dropped "
f"({RUN_W * DROPPED_S / QUARTER_S:.2f} W)",
near(CTRL_W - NORESTORE_W, RUN_W * DROPPED_S / QUARTER_S, 1e-6))
# (d) a LONG outage: restored, but the meter was away past max_hold_s. The
# uncovered seconds must drag the block DOWN, not be held forward.
OUTAGE_S = 200
a4 = QuarterAverager(phases=1, max_hold_s=MAX_HOLD_S)
st4 = fresh_store("outage")
st4.save_partial(a1.snapshot())
PeakTracker(st4, cap_w=LIVE_MAX_W).restore(a4, B + timedelta(seconds=RESTART_AT_S + OUTAGE_S))
o_closed = feed(a4, RESTART_AT_S + OUTAGE_S, QUARTER_S)
OUTAGE_UNCOVERED_S = OUTAGE_S - MAX_HOLD_S
print(f" a {OUTAGE_S} s outage inside the quarter, of which "
f"{OUTAGE_UNCOVERED_S:.0f} s is past the {MAX_HOLD_S:.0f} s hold: "
f"{o_closed[0].offtake_avg_w:.2f} W")
check(f"...reads lower than the uninterrupted block by exactly the uncovered "
f"seconds ({RUN_W * OUTAGE_UNCOVERED_S / QUARTER_S:.2f} W) - a gap is "
f"never held forward across the whole outage",
near(CTRL_W - o_closed[0].offtake_avg_w,
RUN_W * OUTAGE_UNCOVERED_S / QUARTER_S, 1e-6))
# (e) double counting is answered by the primary key, not by care.
st5 = fresh_store("dedupe")
st5.record_block(ctrl_closed[0])
st5.record_block(ctrl_closed[0])
st5.record_block(ctrl_closed[0])
rows = st5.blocks_in(month_key(B))
check("banking the same quarter three times leaves ONE row - quarter_start is "
"the primary key, so a replay or a restart cannot double-count",
len(rows) == 1 and near(rows[0][1], CTRL_W, 1e-9))
# ...and a later pass with LESS coverage may not lower a banked figure.
from app.tariff import _ClosedBlock as _CB
st5.record_block(_CB(start=B, offtake_avg_w=CTRL_W / 2, per_phase_offtake_avg_w=None))
after = st5.blocks_in(month_key(B))
check(f"...and re-banking the same quarter at half the figure leaves it at "
f"{CTRL_W:.0f} W - a restart's partial re-run must never lower a "
f"quarter that was already measured in full",
len(after) == 1 and near(after[0][1], CTRL_W, 1e-9))
# (f) the checkpoint's own staleness bound, derived from the constant.
print(f" CHECKPOINT_S = {CHECKPOINT_S:.0f} s, so a crash can lose at most "
f"{CHECKPOINT_S / QUARTER_S * 100:.1f} % of one quarter's average")
check("the checkpoint bound is small enough that its worst case is under 2 % "
"of a quarter", CHECKPOINT_S / QUARTER_S < 0.02)
# ==============================================================================
print()
print("4. month rollover, in LOCAL time, and across a restart that straddles it")
# CEST is UTC+2, so the last two hours of 31 August UTC are already September.
AUG_LATE = datetime(2026, 8, 31, 21, 45, tzinfo=timezone.utc)
SEP_EARLY = datetime(2026, 8, 31, 22, 15, tzinfo=timezone.utc)
print(f" {AUG_LATE.isoformat()} is local "
f"{AUG_LATE.astimezone(BILLING_TZ).isoformat()} -> {month_key(AUG_LATE)}")
print(f" {SEP_EARLY.isoformat()} is local "
f"{SEP_EARLY.astimezone(BILLING_TZ).isoformat()} -> {month_key(SEP_EARLY)}")
check("a quarter starting 21:45 UTC on 31 Aug bills in AUGUST",
month_key(AUG_LATE) == "2026-08")
check("a quarter starting 22:15 UTC on 31 Aug bills in SEPTEMBER - filing it "
"under August would move a whole month's bill",
month_key(SEP_EARLY) == "2026-09")
check("...and the two are genuinely on opposite sides of the boundary, not "
"the same answer twice", month_key(AUG_LATE) != month_key(SEP_EARLY))
# Winter, where the offset is +1 and the boundary sits an hour later.
JAN_LATE = datetime(2026, 1, 31, 22, 45, tzinfo=timezone.utc)
check("in WINTER the same trap is an hour later: 22:45 UTC on 31 Jan is still "
"January, and 23:15 UTC is February",
month_key(JAN_LATE) == "2026-01"
and month_key(JAN_LATE + timedelta(minutes=30)) == "2026-02")
st6 = fresh_store("months")
AUG_PEAK_W, SEP_PEAK_W = 4998.0, 900.0
st6.record_block(_CB(AUG_LATE, AUG_PEAK_W, None))
st6.record_block(_CB(SEP_EARLY, SEP_PEAK_W, None))
r_aug, r_sep = st6.peak(month="2026-08"), st6.peak(month="2026-09")
print(f" {r_aug.explain()}")
print(f" {r_sep.explain()}")
check(f"August's peak is the {AUG_PEAK_W:.0f} W quarter (specs.md:583's own "
f"measured case) and September's is not",
r_aug.is_observed and near(r_aug.peak_w, AUG_PEAK_W)
and r_sep.is_observed and near(r_sep.peak_w, SEP_PEAK_W))
check("...so a new month starts from its OWN quarters, not from the previous "
"month's peak", r_sep.peak_w < r_aug.peak_w)
# The restart that straddles the boundary: a checkpoint whose open block is in
# August, restored when `now` is already September.
st7 = fresh_store("straddle")
tr7 = PeakTracker(st7, cap_w=LIVE_MAX_W)
straddler = QuarterAverager(phases=1, max_hold_s=MAX_HOLD_S)
for i in range(0, 400, STEP_S):
straddler.add(sample(AUG_LATE + timedelta(seconds=i), RUN_W))
st7.save_partial(straddler.snapshot())
after_boot = QuarterAverager(phases=1, max_hold_s=MAX_HOLD_S)
said = tr7.restore(after_boot, SEP_EARLY + timedelta(minutes=3))
print(f" restore() across the boundary said: {said}")
aug7, sep7 = st7.peak(month="2026-08"), st7.peak(month="2026-09")
check("a restart that straddles the month boundary BANKS the dead August "
"quarter rather than dropping measured energy",
said.startswith("closed a stale block") and aug7.is_observed)
check(f"...at its partial-coverage figure ({aug7.peak_w:.2f} W = "
f"{RUN_W * 399 / QUARTER_S:.2f} W, the seconds it actually covered)",
near(aug7.peak_w, RUN_W * 399 / QUARTER_S, 1e-6))
check("...and September reads NO_RECORD, which is a different answer from "
"August's peak AND from a zero",
sep7.state == NO_RECORD and sep7.peak_w is None)
check("...and the checkpoint is cleared, so a second boot cannot re-bank it",
st7.load_partial() is None)
# ==============================================================================
print()
print("5. AN UNREADABLE STORE IS NOT A ZERO PEAK (the acceptance argument)")
# Three stores, three states, and the three must not answer alike.
good = fresh_store("states")
good_tr = PeakTracker(good, cap_w=LIVE_MAX_W)
empty_r = good_tr.peak(B)
gone = fresh_store("gone")
gone_tr = PeakTracker(gone, cap_w=LIVE_MAX_W)
gone.record_block(_CB(B, 3000.0, None))
gone.close() # the database goes away under a live reader
gone_r = gone_tr.peak(B)
never = PeakStore(str(TMP / "no-such-dir" / "x.db"))
never_tr = PeakTracker(never, cap_w=LIVE_MAX_W)
never_r = never_tr.peak(B)
# A store that is OPEN and whose query FAILS is a third code path - a closed
# handle short-circuits before the SQL ever runs, so without this the
# `except sqlite3.Error` arm of peak() is never executed by anything.
broken = fresh_store("broken-table")
broken.record_block(_CB(B, 3000.0, None))
broken._db.execute("DROP TABLE monthly_peak")
broken._db.commit()
broken_r = PeakTracker(broken, cap_w=LIVE_MAX_W).peak(B)
for label, r in (("fresh store, no quarter yet", empty_r),
("store that broke under a live reader", gone_r),
("store that never opened", never_r),
("store open, query failed", broken_r)):
print(f" {label:38s} -> {r.state:13s} {r.explain()}")
check("a fresh store with no quarter yet says NO_RECORD - a normal state on "
"the 1st of the month", empty_r.state == NO_RECORD)
check("a store that broke says NOT_OBSERVED", gone_r.state == NOT_OBSERVED)
check("a store that never opened says NOT_OBSERVED", never_r.state == NOT_OBSERVED)
check("a store whose QUERY fails says NOT_OBSERVED too - a different code path "
"from a closed handle, and the one a half-corrupt database takes",
broken_r.state == NOT_OBSERVED and broken_r.peak_w is None)
# ...and losing the DERIVED record does not lose the peak, which is the whole
# argument for keeping monthly_peak recomputable from the 15-min tier.
_rec = broken.recompute_month(month_key(B))
print(f" ...but with the 15-min tier intact, recompute_month() still "
f"answers: {_rec.explain()}")
check("...while recompute_month() STILL answers from the surviving 15-min "
"tier - the permanent record is derived, so losing it is recoverable",
_rec.is_observed and near(_rec.peak_w, 3000.0))
broken._db.execute("DROP TABLE quarter_blocks")
broken._db.commit()
check("...and once BOTH tiers are gone it says NOT_OBSERVED rather than "
"raising into the control loop",
broken.recompute_month(month_key(B)).state == NOT_OBSERVED)
check("...and NO_RECORD and NOT_OBSERVED are DIFFERENT states, not two "
"spellings of the same one", empty_r.state != gone_r.state)
check("...and they read differently to a human, too",
empty_r.explain() != gone_r.explain()
and "NOT" in gone_r.explain() and "zero peak" in gone_r.explain())
check("an unreadable peak is None, not 0.0 - there is no number to mistake",
gone_r.peak_w is None and never_r.peak_w is None)
check("...and NO_RECORD carries no number either", empty_r.peak_w is None)
_typed = False
try:
float(gone_r.peak_w)
except TypeError:
_typed = True
check("...and a caller who forgets to ask gets a TypeError rather than a "
"silently disarming zero", _typed)
check("PeakReading exposes no accessor that defaults a missing peak to a "
"number - no .get(), no _or() ",
not [a for a in dir(gone_r)
if a.startswith(("get", "value")) or a.endswith("_or")])
# The consequence, which is the reason any of this matters.
print(f" headroom under a {LIVE_MAX_W:.0f} W limit: "
f"no quarter yet -> {good_tr.headroom_w(B)}, "
f"unreadable -> {gone_tr.headroom_w(B)}")
check("an unreadable store yields NO headroom figure - the disarming default "
"(full headroom, guard silently off) is unreachable",
gone_tr.headroom_w(B) is None and never_tr.headroom_w(B) is None)
check(f"...where a genuinely empty month yields the full {LIVE_MAX_W:.0f} W, "
f"which is correct and is a different answer",
near(good_tr.headroom_w(B), LIVE_MAX_W))
check("the 80 % warning fraction is likewise None when nothing was observed",
gone_tr.warning_frac(B) is None and good_tr.warning_frac(B) == 0.0)
# And a real peak reduces headroom, so "always None" does not pass this part.
good.record_block(_CB(B, LIVE_MAX_W * 0.9, None))
print(f" after one {LIVE_MAX_W * 0.9:.0f} W quarter: "
f"headroom {good_tr.headroom_w(B):.0f} W, "
f"warning {good_tr.warning_frac(B) * 100:.0f} % of the limit")
check("a real observed peak gives a real headroom, so this part is not passed "
"by a store that answers None to everything",
near(good_tr.headroom_w(B), LIVE_MAX_W * 0.1, 1e-6))
check(f"...and it is over §9.1's 80 % warning threshold",
good_tr.warning_frac(B) > 0.8)
# A CORRUPT database file is the third way a store fails, and it must land in
# NOT_OBSERVED too rather than raising into the control loop.
corrupt_path = TMP / "corrupt.db"
corrupt_path.write_bytes(b"this is not a database" * 64)
corrupt = PeakStore(str(corrupt_path))
c_r = PeakTracker(corrupt, cap_w=LIVE_MAX_W).peak(B)
print(f" a corrupt database file -> {c_r.state}")
check("a corrupt database reads NOT_OBSERVED and does not raise into the "
"control loop (§6 fail-open)", c_r.state == NOT_OBSERVED)
# ==============================================================================
print()
print("6. the timezone fallback, graded against zoneinfo")
_zi = None
try:
from zoneinfo import ZoneInfo
_zi = ZoneInfo("Europe/Brussels")
except Exception as err: # noqa: BLE001
print(f" zoneinfo unavailable here ({err}) - the fallback IS the tz")
if _zi is None:
check("zoneinfo is absent, so the encoded rule cannot be graded here - "
"reported, not silently passed", True)
else:
fb = _EUTimezone()
# ⚠️ HOURLY, NOT 3-HOURLY, ACROSS THE TRANSITION DAYS. Both EU transitions
# happen at 01:00 UTC, so a rule that is wrong BY AN HOUR differs from the
# right one only between 01:00 and 02:00 UTC. A 3-hourly sweep steps over
# that window entirely and grades nothing - measured: a mutation moving the
# UTC threshold from 01:00 to 02:00 survived the whole file at 3-hourly.
probes = [datetime(2026, 1, 1, 12, tzinfo=timezone.utc)]
probes += [datetime(2026, m, d, h, mi, tzinfo=timezone.utc)
for m in (3, 10) for d in (28, 29, 30, 31)
for h in range(24) for mi in (0, 30)]
probes += [datetime(2026, m, 1, 0, 30, tzinfo=timezone.utc) for m in range(1, 13)]
# ⚠️ COMPARE THE CONVERTED WALL CLOCK, NOT ITS utcoffset(). `astimezone`
# goes through `fromutc`, but the RESULT carries local wall values, so
# asking it for `.utcoffset()` routes straight back into the LOCAL arm and
# never grades `fromutc` at all. Measured: a mutation moving the UTC
# threshold by an hour survived 397 offset-comparing probes and dies
# immediately on the wall clock they produce.
def _wallclock(t, tz):
return t.astimezone(tz).replace(tzinfo=None)
bad = [t for t in probes if _wallclock(t, fb) != _wallclock(t, _zi)]
print(f" {len(probes)} probes across both DST transitions and every "
f"month boundary: {len(bad)} disagreement(s) in the LOCAL WALL CLOCK "
f"each one converts to")
check("the encoded EU rule converts every probe to the same local wall "
"clock as zoneinfo, including the two transition weekends", not bad)
# ...and the probes are not all the same offset, or the check is vacuous.
offsets = {t.astimezone(_zi).utcoffset() for t in probes}
check(f"...and the probe set spans BOTH offsets ({sorted(str(o) for o in offsets)}), "
f"so agreeing is not free", len(offsets) == 2)
check(f"BILLING_TZ resolved to zoneinfo on this host, not the fallback",
not isinstance(BILLING_TZ, _EUTimezone))
# ⚠️ THE PROBES ABOVE ONLY EXERCISE `fromutc()`. `astimezone` on a UTC
# instant never calls `utcoffset()` with a local wall time, so the local
# arm of the rule was ungraded - a mutation that moved it by an hour
# survived the whole file. These probes drive it directly. Noon is used
# because it is unambiguous on every day of the year, including the two
# transition days where 02:00-03:00 local is missing or repeated.
#
# ⚠️ AND THE LOCAL PROBES MUST INCLUDE 03:00-03:59 ON THE OCTOBER SUNDAY.
# That hour is where a local rule that ends DST an hour late still claims
# CEST while zoneinfo has already returned to CET; at noon both answers
# agree and the probe is worth nothing. Measured: at noon-only, a mutation
# moving the local thresholds from 02:00/03:00 to 03:00/04:00 survived.
#
# The MARCH gap (02:00-02:59 local, which does not exist) is deliberately
# NOT probed: Python hands a nonexistent local time to `utcoffset` as an
# ordinary value and any answer is defensible, so grading it would pin an
# arbitrary choice. It cannot move a month boundary - it is the last Sunday
# of March, never the 1st of a month - which is the only thing this tz is
# used for.
wall = [datetime(2026, m, d, 12) for m, d in
[(3, 27), (3, 28), (3, 29), (3, 30), (10, 24), (10, 25),
(10, 26), (10, 27)]]
wall += [datetime(2026, 10, 25, h, mi) for h in range(0, 6) for mi in (0, 30)
if h != 2] # 02:00 local is the ambiguous fold
wall += [datetime(2026, 3, 29, h, mi) for h in (0, 1, 3, 4, 5) for mi in (0, 30)]
wall += [datetime(2026, m, 1, 12) for m in range(1, 13)]
bad_w = [w for w in wall
if w.replace(tzinfo=fb).utcoffset() != w.replace(tzinfo=_zi).utcoffset()]
w_offsets = {w.replace(tzinfo=_zi).utcoffset() for w in wall}
print(f" {len(wall)} LOCAL-wall probes through utcoffset(): "
f"{len(bad_w)} disagreement(s), spanning "
f"{sorted(str(o) for o in w_offsets)}")
check("utcoffset() on local wall time matches zoneinfo across both "
"transition weekends - the arm astimezone() never reaches", not bad_w)
check("...and those probes span both offsets too", len(w_offsets) == 2)
bad_n = [w for w in wall
if w.replace(tzinfo=fb).tzname() != w.replace(tzinfo=_zi).tzname()]
check("...and tzname() agrees with zoneinfo at every one of them (CET/CEST)",
not bad_n)
# ==============================================================================
print()
print("7. the per-phase reduction, on an UNBALANCED three-phase load (§17)")
# One phase exporting is what separates the three policies. On a balanced load
# they all give the same number, which is why this fixture is not balanced.
PH = (3000.0, 500.0, -200.0)
NET = sum(PH)
u = QuarterAverager(phases=3, max_hold_s=MAX_HOLD_S)
u_closed = []
for i in range(0, QUARTER_S + 1, 5):
u_closed += u.add(sample(B + timedelta(seconds=i), NET, phases=3, phase_net=PH))
blk = u_closed[0]
vals = {p: billed_w(blk, p) for p in (POLICY_NET, POLICY_SUM, POLICY_MAX)}
print(f" phases {PH} (net {NET:+.0f} W) -> "
+ ", ".join(f"{k}={v:.0f} W" for k, v in vals.items()))
check(f"the NET policy bills the connection's net offtake ({NET:.0f} W)",
near(vals[POLICY_NET], NET, 1e-6))
check(f"the SUM policy clips each phase first, so the exporting phase does not "
f"net off ({sum(max(v, 0.0) for v in PH):.0f} W)",
near(vals[POLICY_SUM], sum(max(v, 0.0) for v in PH), 1e-6))
check(f"the MAX policy bills the worst single phase ({max(PH):.0f} W)",
near(vals[POLICY_MAX], max(PH), 1e-6))
check("...and on THIS fixture all three are different numbers - which is the "
"whole reason the reduction is a named policy and not an assumption",
len({round(v, 6) for v in vals.values()}) == 3)
check("the shipped default is NET, the meter's own billed register",
billed_w(blk) == vals[POLICY_NET])
bal = QuarterAverager(phases=3, max_hold_s=MAX_HOLD_S)
b_closed = []
for i in range(0, QUARTER_S + 1, 5):
b_closed += bal.add(sample(B + timedelta(seconds=i), 3000.0, phases=3,
phase_net=(1000.0, 1000.0, 1000.0)))
b_vals = {p: billed_w(b_closed[0], p) for p in (POLICY_NET, POLICY_SUM, POLICY_MAX)}
check(f"...and on a BALANCED load NET and SUM coincide ({b_vals[POLICY_NET]:.0f} W), "
f"so a balanced fixture could not have told them apart",
near(b_vals[POLICY_NET], b_vals[POLICY_SUM], 1e-6))
# ==============================================================================
print()
print("8. monthly_peak is DERIVED, and provably has not diverged")
st8 = fresh_store("derive")
QUARTERS = [(0, 1200.0), (1, 4998.0), (2, 3357.0), (3, 800.0), (4, 4832.0)]
for q, w in QUARTERS:
st8.record_block(_CB(B + timedelta(seconds=q * QUARTER_S), w, None))
MAXQ = max(w for _, w in QUARTERS)
stored = st8.peak(month=month_key(B))
derived = st8.recompute_month(month_key(B))
print(f" {len(QUARTERS)} quarters banked; stored peak {stored.peak_w:.0f} W, "
f"recomputed from quarter_blocks {derived.peak_w:.0f} W")
check(f"the stored monthly peak is the largest quarter ({MAXQ:.0f} W - "
f"specs.md:583's own figure)",
stored.is_observed and near(stored.peak_w, MAXQ))
check("...and re-deriving it from the 15-min tier gives the same number and "
"the same quarter, so the two records are proven equal rather than "
"assumed to be",
derived.is_observed and near(derived.peak_w, stored.peak_w)
and derived.quarter_start == stored.quarter_start)
check("...and the two measured regressions from §9.1 land on DIFFERENT peaks - "
f"the sustained {MAXQ:.0f} W quarter, not the oscillating 3357 W one",
not near(stored.peak_w, 3357.0))
# A quarter arriving LATER and lower must not lower the month.
st8.record_block(_CB(B + timedelta(seconds=9 * QUARTER_S), 100.0, None))
check("a later, smaller quarter does not lower the month's peak",
near(st8.peak(month=month_key(B)).peak_w, MAXQ))
check("...and the recompute agrees with it after that write, too",
near(st8.recompute_month(month_key(B)).peak_w, MAXQ))
# No pruning: §15 says these two tiers are indefinite / permanent.
src = (HERE / "app" / "tariff.py").read_text(encoding="utf-8")
check("§15's retention is structural: no DELETE against quarter_blocks or "
"monthly_peak exists anywhere in tariff.py",
"DELETE FROM quarter_blocks" not in src
and "DELETE FROM monthly_peak" not in src
and "DROP TABLE" not in src)
# ==============================================================================
print()
print("9. AC 3, on the replay data that actually exists")
REPLAY = sorted((HERE / "scenarios").glob("p1-net-*.jsonl.gz"))
print(f" replay fixtures found: {len(REPLAY)} "
f"({', '.join(p.name for p in REPLAY) or 'none'})")
check("the committed replay fixtures are present - a regression fixture CI "
"cannot see is not a regression fixture", len(REPLAY) >= 2)
# ⚠️ `peak_cap_w:` appears TWICE in config.yaml - once under `options:` with the
# default, once under `schema:` with `int(500,15000)`. Taking the last match
# would blow up on the schema line, and taking any match that parses would be
# luck; the FIRST is the options block, which is the number an install lands on.
SHIPPED_CAP_W = None
for line in (HERE / "config.yaml").read_text(encoding="utf-8").splitlines():
if line.strip().startswith("peak_cap_w:"):
try:
SHIPPED_CAP_W = float(line.split(":", 1)[1].strip())
except ValueError:
continue
break
check("config.yaml's shipped peak_cap_w was read out of the file rather than "
"typed here", SHIPPED_CAP_W is not None)
replay_days = 0
total_q = 0
under = 0
for path in REPLAY:
st9 = fresh_store("replay-" + path.stem)
ing = P1Ingest(phases=1, max_age_s=MAX_HOLD_S)
tr9 = PeakTracker(st9, cap_w=SHIPPED_CAP_W or LIVE_MAX_W)
first = last = None
with gzip.open(path, "rt", encoding="utf-8") as fh:
for line in fh:
row = json.loads(line)
t = datetime.fromisoformat(row["t"])
first = first or t
last = t
ing.submit(sample(t, float(row["w"])))
tr9.sync(ing)
months = {month_key(b.start) for b in ing.blocks}
span_h = (last - first).total_seconds() / 3600.0
qs = [w for _, w in st9.blocks_in(sorted(months)[0])] if months else []
peak = max(qs) if qs else 0.0
ok_q = sum(1 for w in qs if w <= (SHIPPED_CAP_W or LIVE_MAX_W))
total_q += len(qs)
under += ok_q
replay_days += span_h / 24.0
print(f" {path.name}: {span_h:.1f} h, {len(qs)} closed quarters, "
f"month peak {peak:.0f} W, "
f"{ok_q}/{len(qs)} = {100.0 * ok_q / max(len(qs), 1):.1f} % under the "
f"{SHIPPED_CAP_W:.0f} W soft limit")
check(f"{path.name}: the month's peak equals its own largest quarter",
near(st9.peak(month=sorted(months)[0]).peak_w, peak, 1e-9))
check(f"{path.name}: a full day of real telegrams produces ~96 quarters, "
f"got {len(qs)}", 90 <= len(qs) <= 96)
PCT = 100.0 * under / max(total_q, 1)
print()
print(f" AC 3 MEASURED: {under}/{total_q} quarters = {PCT:.1f} % under the "
f"{SHIPPED_CAP_W:.0f} W soft limit, over {replay_days:.1f} DAYS of "
f"replay, UNCONTROLLED (no peak shaving was running on either day).")
print(f" AC 3 ASKS FOR 30 DAYS. {replay_days:.1f} days exist. "
f"AC 3 IS NOT MET and this run does not claim it is - see the module "
f"docstring. The figure above is the BASELINE SAFETY-07 has to improve.")
check("the replay produced a measurable number over a stated sample size, "
"rather than a criterion quietly redefined to fit the data",
total_q > 100 and replay_days < 30)
# ==============================================================================
print()
print("10. the wiring: the real Controller, and what its status page says")
class _Store:
data = {}
def set(self, *a):
pass
def get_time(self, *a):
return None
class _Pub:
enabled = False
def publish(self, *a, **k):
pass
def close(self):
pass
class _Hass:
async def limits(self, entity):
return (-LIVE_MAX_W, LIVE_MAX_W, 1.0)
async def number(self, entity, invert=False):
return None
async def set_number(self, entity, value):
return True
async def call(self, domain, service, data):
return True
def built(db_path, opts=None):
return Controller(dict(
{"setpoint_entity": "number.sp", "meter_source": "ha_signed",
"p1_net_entity": "sensor.net", "meter_phases": 1,
"max_w": LIVE_MAX_W, "gain": 0.6, "slew_w": 1000, "deadband_w": 15,
"target_grid_w": 0, "step_w": 10, "heartbeat_s": 10,
"stale_input_s": 15}, **(opts or {})),
_Hass(), _Store(), _Pub(), stop=asyncio.Event(),
peak_db_path=str(db_path))
c = built(TMP / "wired.db")
check(f"a Controller built with peak_cap_w ABSENT takes the SHIPPED "
f"{SHIPPED_CAP_W:.0f} W, so config.yaml and main.py's fallback cannot "
f"drift apart unnoticed", c.peak.cap_w == SHIPPED_CAP_W)
# Drive one real quarter through the real ingest, then the real cycle's sync.
for i in range(0, QUARTER_S + 1, 5):
c.p1.submit(sample(B + timedelta(seconds=i), 4998.0))
n = c.peak.sync(c.p1)
wr = c.peak.peak(B)
print(f" after one 4998 W quarter through the real ingest: "
f"{n} banked, {wr.explain()}")
check("the closed quarter reached SQLite through the tracker the cycle calls",
n == 1 and wr.is_observed and near(wr.peak_w, 4998.0, 1e-6))
# ⚠️ `cycle()` calls sync() ONCE A SECOND against a `blocks` list that keeps a
# closed quarter for a DAY. Without a dedupe that is ~86 000 re-banks of the
# same row per block, and `recorded` - the number the status page prints as
# "quarters banked" - would be a count of loop iterations rather than of
# quarters. The database stays correct either way, which is exactly why this
# needs its own check instead of being caught by the peak.
again = c.peak.sync(c.p1)
third = c.peak.sync(c.p1)
print(f" sync() called three times over the same closed quarter: "
f"{n}, {again}, {third} newly banked; tracker reports "
f"{c.peak.recorded} quarter(s)")
check("...and syncing the same quarter again banks nothing new, so the count "
"the status page shows is quarters and not cycles",
again == 0 and third == 0 and c.peak.recorded == 1)
check("...and the peak is unchanged by the repeat",
near(c.peak.peak(B).peak_w, 4998.0, 1e-6))
rows = [r for r in c.checks() if "capacity-tariff" in r["text"] or "peak" in r["text"]]
peak_rows = [r for r in rows if "peak" in r["text"] and "quarter(s) banked" in r["text"]]
print(" status row: " + (peak_rows[0]["text"] if peak_rows else "(none)"))
check("the status page carries exactly one capacity-tariff peak row",
len(peak_rows) == 1)
check("...naming the peak, the soft limit and the quarters banked, all derived "
"from the controller that was actually built",
bool(peak_rows) and "4998 W" in peak_rows[0]["text"]
and f"{SHIPPED_CAP_W:.0f} W soft limit" in peak_rows[0]["text"]
and "1 quarter(s) banked" in peak_rows[0]["text"])
check("...and it is FLAGGED, because 4998 W is over §9.1's 80 % warning point "
f"on a {SHIPPED_CAP_W:.0f} W limit",
bool(peak_rows) and not peak_rows[0]["ok"] and peak_rows[0]["warn"])
# The half that matters: the same page with a store that cannot answer.
c2 = built(TMP / "no-such-dir" / "nope.db")
u_rows = [r for r in c2.checks() if "capacity-tariff peak" in r["text"]]
print(" status row (unreadable store): "
+ (u_rows[0]["text"] if u_rows else "(none)"))
check("an unreadable store gets its OWN row, saying UNREADABLE", len(u_rows) == 1)
check("...and it does NOT read as a quiet month at 0 W - without this, a broken "
"instrument looks exactly like a month with no peak",
bool(u_rows) and "UNREADABLE" in u_rows[0]["text"]
and "0 %" not in u_rows[0]["text"] and u_rows[0]["warn"])
check("...and the healthy row and the unreadable row are different text",
bool(u_rows) and bool(peak_rows)
and u_rows[0]["text"] != peak_rows[0]["text"])
# THE MEASUREMENT IS OF DELIVERED POWER (SAFETY-20). Graded on the parsed
# SYNTAX, not on the text: the module docstring argues this at length and names
# every forbidden word while doing so, so a grep over the source would be
# satisfied by deleting the explanation and would fail on writing one. What must
# be true is that no NAME the code evaluates refers to a command.
import ast as _ast
_tree = _ast.parse(src)
_names = {n.id for n in _ast.walk(_tree) if isinstance(n, _ast.Name)}
_names |= {n.attr for n in _ast.walk(_tree) if isinstance(n, _ast.Attribute)}
_names |= {n.name for n in _ast.walk(_tree)
if isinstance(n, (_ast.FunctionDef, _ast.ClassDef))}
_FORBIDDEN = {"setpoint", "write_setpoint", "target", "Decision", "i_clamped",
"clamped", "Claim", "resolve", "P_PEAK", "arbiter"}
_found = sorted(_names & _FORBIDDEN)
_imports = sorted(m.module or "" for m in _ast.walk(_tree)
if isinstance(m, _ast.ImportFrom))
print(f" tariff.py evaluates {len(_names)} distinct names and imports "
f"from {_imports or ['(stdlib only)']}")
check(f"none of them is a command-side name {sorted(_FORBIDDEN)} - the utility "
f"bills what FLOWED, so this reads the meter and nothing else"
+ (f"; found {_found}" if _found else ""), not _found)
check("...and it imports nothing from arbiter, clamp or control: it files no "
"claim, applies no clamp, and commands nothing - SAFETY-07's job",
not [m for m in _imports if m in ("arbiter", "clamp", "control",
".arbiter", ".clamp", ".control")])
# ...and the guard is not vacuous: the same grader run over main.py, which DOES
# command, must light up.
_main = {n.attr for n in _ast.walk(
_ast.parse((HERE / "app" / "main.py").read_text(encoding="utf-8")))
if isinstance(n, _ast.Attribute)}
check(f"the same grader over main.py finds {sorted(_main & _FORBIDDEN)}, so it "
f"is capable of failing", bool(_main & _FORBIDDEN))
# ==============================================================================
print()
shutil.rmtree(TMP, ignore_errors=True)
if fails:
print(f"{len(fails)} FAILED: {', '.join(fails)}")
sys.exit(1)
print("all checks passed")
+85
View File
@@ -253,6 +253,91 @@ check("...and the shipped charge rate is at or under the shipped ceiling",
check("...and the shipped discharge rate too, so it warns about nothing",
_shipped["maintenance_discharge_w"] <= _shipped["max_w"])
# --------------------------------------------------------------------------- #
print("the shipped meter_freeze_s, pinned (TEST-06 / AC3)")
# ⚠️ WHY A NUMBER GETS ITS OWN CHECK. `meter_freeze_s: 300` is the threshold
# SAFETY-18's entire argument is about, and before this it was pinned NOWHERE:
# 300 -> 3600 left test_freeze.py AND this file green, because the schema
# accepts int(60,3600) and no check ever read the shipped value. The schema
# bounds what is CONFIGURABLE; this pins what is SHIPPED, which is a different
# question and the one the docs answer.
#
# Read by regex like _shipped above, for the same reason: no PyYAML in the
# image, and a check that skips on ImportError is not a check.
_freeze_s = [float(v) for v in re.findall(
r"^ meter_freeze_s: (\d+(?:\.\d+)?)\s*$", _txt, re.M)]
_freeze_schema = re.findall(r"^ meter_freeze_s: int\((\d+),(\d+)\)\s*$",
_txt, re.M)
check(f"config.yaml carries exactly one meter_freeze_s default {_freeze_s}",
len(_freeze_s) == 1)
check("...and exactly one schema range for it", len(_freeze_schema) == 1)
_hold_s = _freeze_s[0] if _freeze_s else None
_lo, _hi = ((float(_freeze_schema[0][0]), float(_freeze_schema[0][1]))
if _freeze_schema else (None, None))
check("the shipped meter_freeze_s is still the measured 300 s that freeze.py, "
"config.yaml's own comment and test_freeze.py are all written at",
_hold_s == 300.0)
check(f"...and it is inside its own schema range ({_lo}-{_hi} s)",
None not in (_lo, _hi, _hold_s) and _lo <= _hold_s <= _hi)
# The two properties the number has to have, DERIVED rather than restated - so
# they go red on a value that is merely schema-legal. Both come straight from
# config.yaml's own comment on the key.
#
# upper: the alert must land inside the 900 s quarter that bills the capacity
# tariff, or the notification arrives after the money is spent;
# lower: the slowest a HEALTHY meter can tick a 1 Wh register and still look
# unchanged for hold_s is 3600/hold_s W, and that must stay inside the
# deadband or the conjunction's two terms stop being mutually
# exclusive (freeze.py, "WHERE hold_s COMES FROM").
_QUARTER_S = 900.0
_shipped_deadband = [float(v) for v in re.findall(
r"^ deadband_w: (\d+(?:\.\d+)?)\s*$", _txt, re.M)]
check(f"config.yaml carries exactly one deadband_w default {_shipped_deadband}",
len(_shipped_deadband) == 1)
_db = _shipped_deadband[0] if _shipped_deadband else None
_slowest_healthy_w = 3600.0 / _hold_s if _hold_s else None
print(f" shipped meter_freeze_s {_hold_s} s: alert inside the "
f"{_QUARTER_S:.0f} s billing quarter, and a healthy meter slow enough to "
f"look unchanged for it is {_slowest_healthy_w} W against a "
f"{_db} W deadband")
check(f"...the alarm still lands inside the {_QUARTER_S:.0f} s billing quarter",
_hold_s is not None and _hold_s < _QUARTER_S)
check("...and a healthy meter that looks unchanged for it is still inside the "
"deadband, so the two terms stay mutually exclusive",
_slowest_healthy_w is not None and _db is not None
and _slowest_healthy_w < _db)
# ⚠️ AND THE RANGE'S OWN BOUNDS. "300 sits inside the range" is worth nothing
# while the range itself is unpinned: widened to `int(1,86400)` an operator can
# configure `meter_freeze_s: 86400` and the guard is dead for a day at a time,
# with every check in this repo still green. That mutation survived the first
# cut of this PR and is what this block closes. Both numbers are argued for in
# config.yaml's own comment and in freeze.py, and each is re-derived here:
#
# 60 s the floor freeze.py calls NOT A RECOMMENDATION - there a healthy
# meter can look unchanged at 60 W, four times the shipped deadband, so
# the conjunction's two terms stop being mutually exclusive and only
# the margin is left;
# 3600 s the ceiling - already four times the billing quarter, so the top of
# the range has given up detecting a freeze inside the quarter that
# bills it. It is the last bound between a supported configuration and
# a disabled guard, and moving it is a decision rather than a tidy-up.
print(f" schema range {_lo}-{_hi} s: at the floor a healthy meter that looks "
f"unchanged is "
+ (f"{3600.0 / _lo:.0f} W" if _lo else "?")
+ f" against a {_db} W deadband; the ceiling is "
+ (f"{_hi / _QUARTER_S:.1f}x" if _hi else "?")
+ " the billing quarter")
check(f"the schema range itself is still the argued 60-3600 s (found "
f"{_lo}-{_hi})", (_lo, _hi) == (60.0, 3600.0))
check("...a floor that gives up the mutual exclusion and keeps only the "
"margin, which is why config.yaml calls it not a recommendation",
None not in (_lo, _db) and 3600.0 / _lo > _db)
check("...and a ceiling already past the billing quarter, so no value the "
"schema still accepts can alarm inside the quarter that bills it",
_hi is not None and _hi > _QUARTER_S)
# --------------------------------------------------------------------------- #
print("the KNOWN cross-file default divergence, pinned rather than commented")
# ⚠️ WHAT THIS GUARDS, AND WHY IT ASSERTS THE BUG RATHER THAN THE FIX.
+10
View File
@@ -274,6 +274,16 @@ from app.main import Controller # noqa: E402
class _Hass:
def __init__(self):
self.writes = []
self.calls = []
# ⚠️ ADDED FOR SAFETY-03, AND THE REASON IS WORTH READING. The e2e below
# runs a DELIBERATELY INVERTED loop (gain -0.6) into a steady 2000 W import,
# which is a loop that winds straight into the rail - so the output-clamp
# alarm fires here, and the stub not having `call` is how that was
# discovered. This scenario is now a second, independent witness that the
# clamp alarm sees a real winding loop; test_clamp.py asserts it explicitly.
async def call(self, domain, service, data):
self.calls.append((domain, service, data))
async def limits(self, entity):
return (-5000.0, 5000.0, 1.0)