Compare commits

...
38 Commits
Author SHA1 Message Date
glenn schrooyenandClaude Opus 5 9c65ac2cf8 TARIFF-18: price_taxes_eur_kwh stops shipping at half the real figure
checks / unit (pull_request) Successful in 7m48s
`price_taxes_eur_kwh` shipped at 0.06 EUR/kWh with no recorded provenance -
no card, no date, no component list - and roughly HALF the true Flemish
all-in non-energy figure. It now ships at 0.11736, and the rows it is made
of are in the code rather than in this message:

  Fluvius distribution (Antwerpen)              5,35   c EUR/kWh
  Bijzondere accijns, "Alle verbruik"           4,876
  Bijdrage groene stroom en WKK Vlaanderen      1,51
  Energiebijdrage (0 EUR/MWh since 2026-08-01)  0
                                               ------
                                               11,736 c EUR/kWh = 0.11736

Eneco residential card for Flanders, September 2026, incl. 6 % VAT.
`app/price.py`'s FLEMISH_TAXES_COMPONENTS_EUR_KWH carries the rows and
test_price.py section 1b re-adds them - and also compares the total against
the card's own figures typed in the test file, so an edit that moved the
rows and the default together is still caught.

WHAT THIS IS NOT. It is not an arbitrage-correctness fix and it cannot be.
This scalar is a LEVEL component: an additive constant cancels exactly in
`max - min` and leaves the argmin where it was. TARIFF-17 shipped that proof
and it is unchanged - test_price.py 16b drives six flat scalars spanning and
exceeding the whole per-DSO range and gets the identical cheapest hour and
the identical spread to the last digit. DOCS and the CHANGELOG say so in
those words.

WHAT IT IS. A display and guard fix:

  * every number an operator reads, which at half the true value made the
    add-on look considerably better than it is; and
  * §17's negative-price guard in arbitrage.py - the one rung that tests an
    ABSOLUTE all-in price rather than a difference. It fires when
    `spot < -(markup + taxes)`, so its boundary moves -0.09 -> -0.14736
    EUR/kWh of spot. test_arbitrage.py section 2c drives both sides through
    the shipped decide() and pins the boundary from below as well.

The default is Antwerpen's real row, near the LOW end of the 0.117-0.131
per-DSO band, and that is deliberate: the error directions are not
symmetric. Too high carries a genuinely negative hour above zero and the
add-on exports into paid-for disposal - §17's own case. Too low only
declines a discharge that would have paid.

TARIFF-17's INTERACTION, WHICH HAD TO BE DECIDED. Nothing in that rung
hardcoded 0.06; it read PRICE_OPTION_DEFAULTS, so moving the default moves
what it refuses. The Supervisor sets `App.options = deepmerge(config.yaml
defaults, stored options)`, so a saved blob carries the key whether it was
typed or not - every stored 0.06 became "non-default" with nobody editing
anything, and a working Walloon or Brussels site with a taxes table would
have met SystemExit(78) on the upgrade over an inert number. price.py now
carries SUPERSEDED_TAXES_DEFAULTS_EUR_KWH = (0.06,) and validate.py reads a
superseded default as untouched too. The rung is NOT weakened: 0.1198 beside
a table is still refused by name, and the refusal now names both figures.

config.yaml and PRICE_OPTION_DEFAULTS moved together; CFG-03's tripwire
(test_price.py section 1, test_defaults_pin.py) stayed green by agreeing.
test_price.py section 0's fourth check was REWRITTEN rather than re-pointed:
it used to justify the file's own grading constants by observing that the
shipped pair was degenerate (0.06 is exactly twice 0.03). TARIFF-18 ends
that coincidence, so the sentence is no longer true and is not kept.

29/29 test files green. Mutation ledger on the PR.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-13 16:24:56 +02:00
admin 7796f168a9 Merge pull request 'TARIFF-17: a charge that varies by hour gets somewhere to live' (#85) from TARIFF-17 into release/1.0
price.py modelled taxes as one flat scalar. In Flanders that is correct - Fluvius distribution is a single rate - but the Walloon DSOs publish day/night bands on the same card: ORES 13,27 vs 7,39, AIESH 15,17 vs 8,21, ORES IMPACT 16,578 vs 5,090 c EUR/kWh. The bands are ANTI-CORRELATED with what the battery wants to do - cheap network hours coincide with cheap spot hours - so flattening them systematically UNDERSTATES the spread and declines cycles that are genuinely profitable.

The flat scalar was never a wrong number. It sits in the additive bucket and provably cannot change which hour is cheapest; the defect was that a component belonging in the order-changing bucket had nowhere to live. That framing is the ticket, and it is what the regression grades.

Reproduced on one ordinary Belgian winter Thursday, same feed, same markup, taxes the only argument differing: five flat scalars spanning AND exceeding the table's own range - 0.0000 / 0.0739 / 0.1198 / 0.1327 / 1.0000 - all give cheapest hour 13 and spread 0.1500 to the last digit, while the ORES table gives hour 03 and 0.1788. Asserted through the shipped decision, static_tariff.arbitrage().cheapest, not through assemble()'s output. Before this existed the shipped tree answered assemble(..., <table>) -> TypeError.

The fixture differs in exactly ONE way and that is its own precondition - same caps object, same RawPoint list, same markup, by identity - and the flat arm is a FAMILY rather than a single scalar, so what the comparison measures cannot be a level effect. That closes the shape that produced a whole-suite survivor in TARIFF-11, where a fixture varied two things at once.

GRAMMAR REUSED, NOT DUPLICATED. parse_taxes_profile() is parse_static_profile() with option= and means= as default arguments, so every pre-existing call site and sentence is byte-identical. Graded two ways: one check parses every module under app/ and asserts the 14 grammar names are defined in exactly one of them, with an AST pin that the wrapper's whole body is one call; another DERIVES all thirteen refusal sentences for the new option from the old option's own sentence. Copying the tiling check into price.py is red.

A WHOLE CARD TYPED IN CENTS IS ITS OWN RUNG, and not a restatement of the parser's plausibility bound. ORES's 13,27 trips that bound - but SIBELGA'S ENTIRE BRUSSELS CARD (9,39 / 7,10) IS UNDER IT and would have started as a 9.39 EUR/kWh levy. Graded by its own reason, and the mutation is red on the Sibelga line only.

Malformed tables are refused at STARTUP by validate.py across three rungs. If one ever gets past, load_prices() returns a section 11.4 no-data read naming the key and logs an error - no silent fall back to the scalar, no traceback into the tick.

Both plugins that need the framework to add taxes move 13 -> 03 (custom_formula {ENERGY,MARKUP} and generic_dayahead {ENERGY}); manual_csv, which is all-in, stays at 13. Enumerated from PROVIDERS rather than typed.

Evidence: 29/29 green. Thirteen mutations, all red, and no row is an edit-and-revert - each gets a fresh rsync'd copy with no __pycache__, mounted :ro, then deleted, which closes habit 18 structurally rather than by discipline. Hoisting the lookup out of assemble()'s loop (+17 B) keeps the number right while stopping it varying by hour: red on six named checks in test_price.py and GREEN IN THE OTHER 28 FILES.

Three things named rather than rounded off: one mutant is equivalent and unseparable by any fixture, because StaticProfile.at() calls brussels() on whatever instant it is given; DOCS' bucket table is prose that nothing asserts, though the numbers it quotes are printed by the checks on every run; and the card's own 0.2240/0.2422 figures were NOT verified and were not repeated - replaced with a claim that is derivable from the contract price, that the spread is proportional to factor and the offset drops out.

One design point is explicitly the owner's, not the agent's: a table beside a NON-DEFAULT price_taxes_eur_kwh is a refusal, so an operator adopting the table must set that key back to 0.06. The alternative - warn, table wins - is the silent-loser shape this repo refuses everywhere else, so the refusal was taken and flagged rather than quietly chosen.
2026-09-13 15:40:17 +02:00
glenn schrooyenandClaude Opus 5 2fac0aaf06 TARIFF-17: a price component that varies by hour has somewhere to live
checks / unit (pull_request) Successful in 7m25s
`price.py` modelled the whole non-energy part of a Belgian bill as ONE flat
scalar. This adds `price_taxes_profile`, which carries the TAXES component as
a period table, and makes `assemble()` evaluate it at each point's own instant.

⚠️ THE FLAT SCALAR WAS NOT A WRONG NUMBER, AND THIS IS NOT A CORRECTION OF ONE.
The optimiser reads exactly two quantities off a series - the spread
`max - min` over the firm points, and which hour is cheapest - and an additive
constant identical in every hour cancels out of the first and leaves the second
where it was. It is provably inert. Measured, and printed by test_price.py 16b
on every run: with a flat taxes figure of 0.00, 0.0739, 0.1198, 0.1327 or 1.00
EUR/kWh the same 24-hour feed is cheapest at 13:00 in all five cases and its
spread is 0.1500 EUR/kWh in all five cases. What was missing is anywhere for an
HOUR-VARYING non-energy component to live. This is a missing capability, not a
bad constant.

Belgium has one. Fluvius distribution is a single rate - its tariff-card row
reads `Normaal | Uitsl. nacht`, where exclusive night is a separate physical
circuit and not a night register - so nothing changes for the reference site in
Flanders. The Walloon DSOs and Sibelga are on the same card with four columns,
`Enkelvoudig | Dag | Nacht | Uitsl. nacht`, and the gap is real money (c/kWh):
ORES 13,27 / 7,39; AIESH 15,17 / 8,21; Tecteo RESA 12,20 / 7,01; Sibelga
9,39 / 7,10; ORES IMPACT 2026 16,578 / 5,090.

⚠️ AND THE BANDS ARE ANTI-CORRELATED WITH WHAT THE BATTERY WANTS TO DO. The
cheap network hours are the cheap spot hours, so flattening them does not shift
the curve - it UNDERSTATES the spread and declines cycles that would have paid.
Same feed, same markup, the only difference being the taxes argument:

    flat 0.0000 / 0.0739 / 0.1198 / 0.1327 / 1.0000
                                       -> cheapest hour 13, spread 0.1500
    ORES dag 0.1327 / nacht 0.0739     -> cheapest hour 03, spread 0.1788

The regression check grades WHICH HOUR IS CHEAPEST, not the spread: an
implementation that reads the table once and adds one number to every point is
the LEVEL bucket wearing a period table - right magnitude, wrong shape - and a
spread assertion passes on it. Measured: hoisting the lookup out of
`assemble()`'s loop (+17 B) is red on six named checks in test_price.py and
GREEN in the other 28 files.

ONE GRAMMAR, NOT TWO. `static_tariff.py` already shipped the periods, the day
selectors, the tiling rule and the DST handling, so `parse_taxes_profile()` is
the same parser with the option name and the "what this number means" clause as
arguments. A second tiling rule would be a second set of 23- and 25-hour-day
bugs. test_static_tariff.py section 10 pins that two ways: by parsing every
module under `app/` and asserting the grammar is defined in exactly one of
them, and by deriving each of thirteen refusal sentences for the new option
from the old option's own sentence rather than transcribing them.

`Component.TAXES` semantics, `code_leaks()` and the double-counting guarantee
are untouched. The new key joins FRAMEWORK_KEYS, BANNED_IN_PLUGIN_CODE is
derived from that set, and a plugin that names it is refused at `register()`.
It reaches both plugins that take taxes - `custom_formula` ({ENERGY, MARKUP})
and `generic_dayahead` ({ENERGY}) - and not `manual_csv`, which is all-in: the
table is routed through `capabilities().adds`, exactly as the scalar is.

⚠️ A MALFORMED TABLE IS REFUSED AT STARTUP, NOT AT THE NEXT FETCH. It is read
on the ~30 s planning path, and a table that first fails at 03:00 is a site
that stops pricing at 03:00, which §11.4 turns into a quiet fallback
indistinguishable from a healthy fixed-tariff day. validate.py refuses three
things: a table that does not tile the day; a table set beside a NON-DEFAULT
`price_taxes_eur_kwh`, because the table wins and the scalar would then be a
number the operator typed that decides nothing; and a whole card typed in
CENTS, which is how these figures are printed. That last rung is not a
restatement of the parser's +-10 EUR/kWh bound - ORES's 13,27 trips the parser,
but Sibelga's entire card, 9,39 and 7,10, is under it and would otherwise have
started as a 9.39 EUR/kWh levy. Both rungs are graded by the REASON each gives,
and mutating them together (TEST-16) moves three rows.

NOTHING CHANGES FOR A SITE THAT DOES NOT SET IT. The option ships "", and the
scalar path is unchanged arithmetically rather than merely in intent:
test_price.py 16f reproduces the pre-TARIFF-17 expression and compares 12
capability/scalar combinations point for point with `==`.

13 mutations, every one red on a named check, ledger in test_price.py below
section 16k with byte deltas and the one equivalent mutant named. 29/29 green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-13 15:31:15 +02:00
admin c518007caf Merge pull request 'TARIFF-11: the add-on stops flattering itself' (#84) from TARIFF-11 into release/1.0
Four places where the add-on's account of itself was wrong, every one in the direction that flatters it, and none of them pinned by any check because nothing graded the strings a human reads.

TariffPlan.words counted points and called them hours - 192 h for 192 quarter-hourly points, which is 48 hours; measured across a 40-hour soak the error ranged 1.79x to 5.33x with the firm/carried mix, and horizon_h was 48.0 at every sample while words never said 48. The section 8.3 refusal did the same and reached the status page through arb.explain(): 'only 1 of 96 hours carry a published price' for 96 points spanning 24. The plan's shape reached no HTTP surface at all - rows['Tariff'] was static_status()'s own sentence and read 'no static profile' at all 802 soak samples, while the plan was provisional for 76% of them and carried mixed 900/3600 s spacing for 70%. And main.py's 'if and only if' comment was false: cycle() returns on missing inputs before self.arb = None, so a site with trading ON rendered 'off (arbitrage_enabled)' - naming an option that was on.

A FIFTH instance was found that the card did not list, in price_forecast.py's '%d of %d hours' - carried hours over points - which DOCS quotes verbatim, so leaving it would have re-broken the doc.

THE CHECK IS THE DELIVERABLE, not the fixes. It renders Controller.status_payload()['rows'], parses each number out of its OWN slot, and compares it against a figure computed from the fixture's two construction parameters - never from the property the row was built out of. The slot parser is calibrated first against a correct row AND against a row wrong exactly as the defect was. Fixtures are a SET of four plan shapes and the check computes and prints the table of field pairs no fixture separates. origin and product are graded by disagreement over identical point lists, the Arbitrage row's states by inequality, the end-to-end row against the captured document's own length.

The two Tariff sources stay distinguishable: one row, two explicitly timestamped halves, with the age taken from tariff_at - which TARIFF-12 deliberately does not move on a kept plan - and measured by the caller rather than by a second clock inside the renderer. The same plan object at 0 s and 4000 s must render differently, and hardcoding the age to 0 is red.

FCST-04's Tariff-to-DOCS pin was honoured rather than routed around: wiring the plan in took test_price_history.py red until DOCS carried the pinned sentence, the row's five facts, the two-clock warning and a four-row table of the Arbitrage renderings. The trap fired exactly as designed.

Evidence: 29/29 green, re-verified after merging TARIFF-13. Sixteen mutations, no survivors; one is a zero-byte edit transposing firm and carried, and it is red.

TWO MUTATIONS BIT THE AGENT'S OWN CHECKS AND BOTH WERE CLOSED. One was a survivor of the whole suite at -190 B: the origin fixture gave the fallback plan a distinct provider string as well as a distinct origin, so the rows differed for two reasons and dropping origin entirely changed nothing observable - habit 4, occurring inside the fix for habit 4. The provider string is now held equal as its own precondition. The other produced a traceback and no named red, because a datetime comparison raised inside an f-string OPERAND rather than inside the condition - FCST-06's finding, one layer over.

Left for its own card, named rather than folded in: TARIFF-09's price_feed_url override warning still predicts the wrong degraded state - on a cold store the unpublished hours are ABSENT, not carried and flagged - and absence is the worse state.
2026-09-13 14:07:07 +02:00
admin a11597e0b3 Merge remote-tracking branch 'origin/release/1.0' into TARIFF-11
checks / unit (pull_request) Successful in 7m20s
# Conflicts:
#	goodwe_controller/CHANGELOG.md
2026-09-13 13:51:35 +02:00
adminandClaude Opus 5 610725bed9 TARIFF-11: the plan's account of itself, corrected and put on the page
Four defects of one shape - the add-on's report of its own tariff plan was
wrong, always in the direction that flattered it - and one check, which is
the deliverable.

1. `TariffPlan.words` counted POINTS and called them HOURS. The live add-on
   logged `dynamic_dayahead from generic_dayahead, 132 h` for 132
   quarter-hourly points = 33 hours. Belgium moved to a 15-minute MTU on
   2025-10-01, so this was wrong every time it was printed, and NOT by a
   constant factor: a 40-hour soak measured 1.79x .. 5.33x, because §8.3's
   carried points are hourly while the firm ones are quarter-hourly.
   `words` now reports both quantities, plus the firm/carried split and the
   spacing, in FCST-04's *Price history* wording so the two rows can be
   compared.

2. §8.3's firm-pair refusal and the flat-tariff refusal did the same, and
   they reach an operator through `self.arb.explain()`. This is exactly the
   point<->hour conflation TARIFF-14 removed from `at()`, left standing in
   the prose. `price_forecast.py`'s §8.3 carry-forward log line carried the
   third instance - a carried-HOUR count over a POINT count - and DOCS
   quotes it verbatim, so it is corrected here too.

3. The plan reached no HTTP surface at all. `rows["Tariff"]` was
   `static_status()`'s 30 s-tick sentence and never saw the provider plan -
   it read 'no static profile' at all 802 soak samples on a site whose feed
   was answering. It now carries the plan's shape. ⚠️ The two sources are
   kept distinguishable: the plan half is stamped with its own age, measured
   from `tariff_at`, because it can be up to TARIFF_MAX_AGE_S (5400 s) old.
   FCST-04 pins that row to DOCS as a pair; DOCS moves in this commit.

4. `main.py`'s "if and only if" was false. `cycle()` returns at the input
   hold before `self.arb = None`, so an armed site read
   `off (arbitrage_enabled)` - naming an option that is on - and, after one
   completed cycle, showed a decision up to 15 s stale as a current one. The
   row now reads the OPTION and separates three facts; `cycle_at` and
   `arb_at` make "this cycle reached no decision" an observation.

THE DELIVERABLE IS THE CHECK. All four survived 29 green suites because
nothing graded the strings a human is shown. `test_price.py` section 12
renders through the shipped `status_payload()`, parses each number out of
its OWN SLOT and compares it against a figure computed from the fixture's
construction parameters - not from the property the row was built out of
(habit 4), and not as a substring of a sentence the same file wrote (habit
14). The fixture set is four plan shapes and the check computes and prints
the field pairs no fixture separates. The *Arbitrage* row's states are
driven through real cycles with a sensor answering None and graded by
INEQUALITY.

No behaviour changed: no threshold, refusal rung, claim or setpoint moved.

MUTATIONS: 16, no survivors, one at +0 B (the risky delta) and red. Two were
found by the battery INSIDE THIS CARD'S OWN CHECK: a -190 B survivor whose
fixture varied two things at once (habit 4), and a mutation whose red was a
TRACEBACK in an f-string operand and therefore named nothing (FCST-06). Both
closed; the table is on the PR.

All 29 test files green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-13 13:51:05 +02:00
admin b4a3801f68 Merge pull request 'TARIFF-13: the count of what was measured is as durable as the measurement' (#83) from TARIFF-13 into release/1.0
A 40-hour soak drove a restart and caught the capacity-tariff row asserting '0 quarter(s) banked' for a month in which 120 quarters had been measured. The peak and its attributed quarter persisted; the count did not. It is the count, not the peak, that tells an operator how much of the billing month the measurement actually covers - a peak of 365 W over 120 quarters and the same peak over 0 quarters are very different claims about the same number, and after a restart the page made the weaker claim while displaying the stronger one's value.

Reproduced through a real Controller and P1Ingest with a real restart, never by reading code: 12 -> 0 with the store still holding 12 rows. Fixed durable rather than removed - the count is the useful half of the row. PeakStore.banked_count(month) queries quarter_blocks and `recorded` became a property over it. After the fix the printed count matches the store on both sides, 12 -> 13, and 13 is CORRECT: the restart's restore() banks the stale open block, which the old process-wide counter never saw.

TWO DEFECTS THE TICKET DID NOT NAME, both closed by the same change: the count was never month-scoped, so a fresh October reported a quarter banked from September; and it missed blocks banked by restore(). An unreadable store now returns None and renders 'an unreadable number of quarter(s) banked' - rendering it as 0 was rejected because it reads as a real zero, which is TARIFF-01's own disarming-value argument applied to the coverage figure beside the peak.

THE TICKET'S SECOND FINDING WAS FALSE AND THE AGENT DISPROVED IT. I wrote that ties at the same wattage are re-attributed to the most recent matching quarter. The upsert is strictly '>', so an exact tie never writes at all - six exactly-equal 365.0 W quarters re-attributed zero times. The soak's drift was a real new maximum below the row's whole-watt display resolution: 365.0000001 W is a new peak to the database and the same '365 W' to the reader. A plausible diagnosis, checked rather than implemented.

Tie attribution is nevertheless decided deliberately - EARLIEST wins - on what the row is for rather than on money: wattage is what is billed, and an operator investigating one event needs a stable answer. It is also what recompute_month() already did, and the two readers genuinely diverged: fed newest-first, monthly_peak answered 09:45 and the recompute 09:00, breaking the invariant part 8 exists to prove. peak_w is untouched and SAFETY-07 reads identical watts.

Evidence: 29/29 green. Eleven mutations, all red, purged before each with the fingerprint verified after every revert. One was a REAL SURVIVOR at -3 B: banked_count() has two None paths and the first battery reached only one, because a never-opened handle short-circuits before any SQL. Closed by dropping the table out from under an OPEN store, behind a precondition asserting the same call answered 4 first.

main.py is untouched - the row picks the fix up unedited because `recorded` is now a property. The residual is named rather than hidden: `recorded` answers for now() while the peak follows checks(when=...). Production has exactly one call site, self.checks() with no argument, where both are the same month, so the live page is correct and durable; they part only under the suite's when= affordance, and that mismatch is strictly smaller than the process-wide counter it replaces. recorded_in(when) is shipped and the row's move onto it is one token.
2026-09-13 13:03:30 +02:00
glenn schrooyenandClaude Opus 5 a991438e40 TARIFF-13: the coverage figure beside the peak stops resetting at a restart
checks / unit (pull_request) Successful in 7m27s
The capacity-tariff status row printed a peak read from SQLite next to a count
of banked quarters read from PROCESS MEMORY - `PeakTracker.recorded`, an `int`
set to 0 in `__init__` and counted up inside `sync()`. A 40-hour soak caught
what that means at a driven restart, on both instances independently:

  before: 2026-09: peak 365 W, set by the quarter at 2026-09-11T18:45:00+00:00
          - 10 % of the 3500 W soft limit; 120 quarter(s) banked
  after : 2026-09: peak 365 W, set by the quarter at 2026-09-11T18:45:00+00:00
          - 10 % of the 3500 W soft limit;   0 quarter(s) banked

Reproduced here before anything changed, through the real Controller and a real
restart - a second Controller object against the same file on disk - at 12 -> 0
with 12 rows standing in `quarter_blocks`.

The peak is durable and was never wrong. The COUNT is what says how much of the
billing month that peak covers, and 365 W over 120 quarters and 365 W over 0
quarters are different claims about the same number. Mid-month, after every
restart, the page made the weaker claim while displaying the stronger one's
value. That is TARIFF-01's own argument - an unreadable store must not read as a
zero peak, because a zero peak is disarming - one column over.

`PeakStore.banked_count(month)` asks the database, and `recorded` is a property
over it. Two more defects in the same variable fell out once it was driven, and
neither was in the ticket: it was never scoped to a MONTH (on 1 October the row
still showed September's tally beside October's empty month - measured), and it
missed the block `restore()` banks from the checkpoint, which never passes
through `sync()`.

⚠️ NOT 0, AND NOT None, WHEN THE STORE WILL NOT ANSWER. `banked_count()` returns
None and the row renders `an unreadable number of quarter(s) banked`.
`f"{None} quarter(s) banked"` was rejected because it reads as the same zero.
`0 quarter(s) banked` stays true and reachable - it is what a month nothing has
been written to says, and the two are asserted to be different text.

⚠️ THE DEDUPE IS NOT REMOVED WITH THE COUNTER. `_seen` still stops `cycle()`'s
once-a-second `sync()` re-banking a block `P1Ingest.blocks` keeps for a day. The
check that graded it asserted `recorded == 1` - the dedupe and the figure
reporting it were the same variable - and now grades `sync()`'s RETURN, because
counting rows cannot see a re-bank at all (PRIMARY KEY no-op).

THE TIE RULE, DECIDED RATHER THAN EMERGENT: at equal wattage the EARLIEST
quarter keeps the attribution. No bill and no guard moves - `peak_w` is still
only ever raised, the `>` term is untouched, SAFETY-07 reads the same watts.
The two readers DID disagree: the upsert's strict `>` kept whichever tied block
was WRITTEN first while `recompute_month()` has always kept the earliest, the
same only while blocks arrive in time order, and measured to diverge (09:45
against 09:00) fed newest-first.

⚠️ AND THE SOAK'S DRIFTING QUARTER WAS NOT A TIE. Six exactly equal 365.0 W
quarters re-attributed zero times on the shipped tree - `>` is strict. What
moved it was a real new maximum BELOW THE ROW'S OWN RESOLUTION: the row renders
whole watts, so 365.0000001 W is a new peak to the database and the same 365 W
to the reader. Left alone deliberately, and pinned so it is not re-diagnosed.

test_tariff.py parts 11 and 12. 12 mutations, all red, listed on the PR; M2 -
`banked_count()`'s sqlite3.Error arm returning 0 - was a SURVIVOR at -3 B on the
first battery, because a handle that never opened short-circuits before any SQL
runs and nothing reached the except arm. Closed with an open store whose table
is dropped under it, with a precondition that the same call answered 4 first.

main.py is NOT touched - TARIFF-11 holds its status rows. The row picks this up
without editing because `recorded` is an attribute access and is now a property.
The residual is stated in DECISIONS §3: `recorded` answers for now() while the
peak follows `checks(when=...)`, and the shipped install has exactly one call
site - `self.checks()` with no argument, main.py:4359 - where they are the same
month, so the live page is correct and durable. `recorded_in(when)` is shipped
and the row's move onto it is one token, for whoever holds main.py next; part 10
pins the present behaviour against a row count taken from `quarter_blocks` so
the gap cannot be quietly forgotten.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-13 12:54:15 +02:00
admin 068b83f57c Merge pull request 'TARIFF-15: custom_formula, graded against the card it was copied from' (#82) from TARIFF-15 into release/1.0
The owner asked for a custom formula for exotic suppliers. TARIFF-07's research established why it is the only shape that can work: there is no machine-readable source of Belgian tariff coefficients anywhere - CREG's tariff-card pages 404, VREG was unreachable, and every coefficient found came from a supplier's own PDF, one image-only. Coefficients are per-contract and live behind a klantenzone login. So no scraping and no per-supplier plugin can work; the customer is the only party who can see their own numbers, and the card they read them from is the only artefact that can grade them.

The shape is the market's published arithmetic rather than our approximation: both readable cards print the formula, the reference index assumed, AND the resulting price. Through shipped code, Eneco (0.102 x 133.9905 + 1) x 1.06 = 15.54705 against a card printing 15,55; ENGIE (0.1 x 61.96 + 0.9) x 1.06 = 7.52176 against 7,522.

That is also the safety mechanism. validate.py recomputes the card's own worked example at startup and REFUSES TO START unless it matches to 0.01. An offset inflated by network charges cannot pass, because the card's printed estimate is by construction the energy component alone. The refusal names how far out, in which direction, by what ratio, and then names the expensive cause: network charges folded into `offset` bills you for it twice, which is why the check cannot be switched off.

NO EXPRESSION EVALUATION, guaranteed three structural ways - coefficients must be numbers, an unknown field is refused rather than ignored, and there is no evaluator in the module - and checked by an AST walk rather than a grep, because the module's own docstring contains the text eval(s, {"__builtins__": {}}) explaining why that is not a sandbox, and a text scan would fire on the documentation it mandates. That is code_leaks()'s lesson applied to a second file. A mutation plants a real eval in the shipped module and turns it red.

TWO CORRECTIONS TO THE TICKET, both the agent's and both right. The plausibility band's floor was specified on the all-in figure; the shipped 0.06 of taxes is 3x the 0.02 floor, so a hundredfold error (0.0016 -> 0.0616) sails over it. The floor now sits on the ENERGY figure and the ceiling on the all-in. And the ticket claimed the card check catches 10x/100x/1000x errors: it catches a MISMATCH between the two sides, and a unit error applied to both cancels out of a comparison between them. The wording was narrowed everywhere rather than restated, and the band added as a separate refusal on a separate quantity.

Stated limits rather than glossed: the check grades the coefficients against the card, not the card against reality - a consistent-but-wrong pair, both numbers from the wrong product, passes, and DOCS says so in the same sentence as the claim. Only TWO real tariffs could be reproduced end to end, not three: the repo's third real formula (Eneco injection) has no printed price beside it, so there is nothing to grade it against and none was fabricated.

Evidence: 29/29 green, 347 checks in test_price.py. 22 mutations, no survivors. One was a -13 B SURVIVOR on the first attempt - the check graded the preview's heading rather than the row, the third instance this week of a check anchored to something it does not own - and is now graded on a rendered row built from the capture. One mutation passes the all-in check by declaring ALL_IN, exactly as trap 1 says no code can catch; what goes red is the declaration-agreement check plus the card reproductions.

The riskiest edit is flagged rather than buried: test_price.py's summary block had to move, because a section appended after `if fails: sys.exit(1)` would have left every new check's failure unreported with exit 0. Verified on the base that sections 13 and 14 were always before it, so no earlier evidence is affected.
2026-09-13 03:58:46 +02:00
Claude Opus 5 ac9854886b TARIFF-15: your own tariff card, as two coefficients the card itself grades
checks / unit (pull_request) Successful in 7m14s
A Belgian dynamic contract prices energy as `B x index + A`, and the two
coefficients exist on exactly one artefact: the customer's own tariff card.
There is no machine-readable source for them anywhere - CREG's consumer
tariff-card pages 404, CREG Scan is a JS wizard, the annual report has the
supplier list without per-product coefficients, VREG was unreachable, and
every coefficient found in research came off a supplier's own PDF, one
image-only. So nothing can be scraped and a per-supplier plugin would have no
source for its numbers, which is why TARIFF-07 withdrew three of them. The
customer is the only party who can see their own coefficients; they type them.

New provider `custom_formula`, one new option `price_formula` (a JSON string,
the `static_profile` precedent exactly), and a new module
`app/custom_formula.py`. The index path is `generic_dayahead`'s, not a second
one: the URL/override decision is now one shared `dayahead_feed_url()` both
plugins call, same feed, same Brussels-calendar range, same columnar parser,
same declared egress hosts.

It declares {ENERGY, MARKUP}, NOT ALL_IN. A card's Tariefformule line is the
energy component alone - Eneco tabulates network tariffs and levies
separately, ENGIE states the formula excludes network, distribution,
transport, levies and excise - so ALL_IN would lie for every operator who
typed what their card says and would drop the taxes from every hour
(0.2125 -> 0.1555 EUR/kWh, measured).

THE CARD IS THE GRADER, and that is why this is safe to ship. The framework
already mechanises that a PLUGIN cannot double-count. This introduces a risk
none of those guards can see because it is config, not code: the OPERATOR
double-counting, by folding network charges into `offset` while
price_taxes_eur_kwh carries them too. So validate.py recomputes the card's own
printed price from the operator's coefficients at startup and refuses to start
unless the two agree to within 0.01, naming the discrepancy. An inflated
offset cannot pass: 15.55 becomes 28.27. The same check catches VAT omitted,
VAT doubled, a transposed digit and a unit mismatch between the two sides.

And the limit, in the same sentence as the claim: it grades the coefficients
against the card, NOT the card against reality. A consistent-but-wrong pair
passes. A second refusal covers the one unit error the first cannot see - the
one applied to both sides at once - by bounding the card's own point to
0.02-1.00 EUR/kWh. Its FLOOR is applied to the energy figure rather than the
all-in one because the shipped 0.06 of taxes is three times that floor and
would carry a hundredfold error over it alone; found by writing the mutation,
not by review (M10, +0 B).

The market's own arithmetic, reproduced through the shipped code:

  Eneco (0.102 x 133.9905 + 1) x 1.06 = 15.5471   card prints 15,55
  ENGIE (0.1   x  61.96   + 0.9) x 1.06 = 7.5218  card prints 7,522

NO EXPRESSION EVALUATION OF ANY KIND. The string is settable over HTTP through
the add-on's own ingress page, so eval on it is RCE in a root container
holding SUPERVISOR_TOKEN and driving an inverter over RS485 - but the deciding
argument is that an expression is UNVERIFIABLE: two coefficients can be graded
against the card, an arbitrary expression cannot be graded against anything,
so it would make the invisible-wrong-coefficient problem strictly worse. An
unknown field is REFUSED rather than ignored, so `"expr"` is told it does not
exist. Future shapes become named fields, never a grammar. test_price.py 15f
walks the module's AST for eval/exec/compile/__import__/literal_eval and
exercises the scanner's red direction on every run.

Refused at startup: card absent or out by >0.01; `units` absent (no default);
factor == 0 (every spread 0, arbitrage silently never fires - the message
points at static_profile); factor < 0 (inverts the market); vat_pct outside
0-30 (0 stays legitimate); non-finite or non-numeric; unknown field, index or
resolution; the plausibility band. A card older than ~6 months WARNS and does
not refuse - ENGIE freezes coefficients for a contract's duration. Negative
output prices are not refused anywhere.

A startup preview logs the next 24 h as `index EUR/MWh -> all-in EUR/kWh` with
min/median/max and the day's spread, through a new default-silent
`PriceProvider.startup_preview()` hook - only the plugin knows what went in to
its own arithmetic. Emitted once per process, and a preview that raises cannot
cost the site its prices.

`custom_formula` joins the config.yaml enum through the DERIVED set
PROVIDERS | RETIRED_PROVIDERS | {"off"}, not a hardcoded list. Both
translations describe the option and the new id. DOCS.md replaces the
"planned" sentence with the shape, a worked example from a real card, a real
startup transcript and the limit of the check stated beside the claim.

22 mutations, one per check added, no survivors - MUTATION.md carries the
ledger. M13 (the preview's index column) was a -13 B survivor of the whole
suite on the first attempt because the check graded the HEADING and not the
row; it is graded on a rendered row now. All 29 suites green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-13 03:49:08 +02:00
admin aade2c79fd Merge pull request 'TARIFF-07: withdraw three plugins that never fetched a price' (#81) from TARIFF-07 into release/1.0
engie_dynamic, totalenergies_dynamic and eneco_eddy had fabricated endpoints. Re-verified before deletion, 2026-09-13: api.engie.be/dynamic/be answers 404 from a Kong gateway with no route; api.totalenergies.be presents a certificate for CN=pge-apigee-dev.direct-energie.com with the host absent from its SAN, so TLS closes before a request is sent; api.eneco.be/eddy/prices answers 404 with a one-byte body. And 'Eddy' is not an Eneco product at all - theirs is 'Zon & Wind Dynamisch', indexed to Belpex/EPEX DAY-AHEAD rather than imbalance.

They are structurally impossible, not merely unfinished: supplier coefficients are per-contract and live behind the customer's klantenzone login, so no public unauthenticated endpoint could ever return a given customer's price. There is no regulator index either - CREG's tariff-card pages 404, VREG was unreachable, and every coefficient found in research came from a supplier's own PDF, one of them image-only. The three invented URLs are the predictable failure mode of a task with no canonical index to fall back on.

THE SCHEMA ENUM KEEPS ALL SIX IDS ON PURPOSE. config.yaml's price_provider is a list(...) enum validated by the Supervisor BEFORE the add-on starts, so removing the three would make an existing config schema-invalid, produce an opaque 'expected one of ...', and validate.py would never run - the refusal would never be seen on exactly the sites it exists for. PROVIDERS is two; the enum stays six; both translations label them withdrawn; the schema check is derived from PROVIDERS | RETIRED_PROVIDERS | {off}; and register() refuses to re-register a retired id so the registry and the retired table cannot both claim one.

The refusal tells a stranded operator the truth rather than aliasing them somewhere: that this is not a typo - the add-on offered the id, accepted it, and said nothing at every startup since; that it has never fetched a price anywhere, with the endpoint and its observed response named; that section 11.4 turned every failure into a silent fallback, so what the site has actually been running is the static profile, or nothing at all when none is set; and what to do instead, stating plainly that generic_dayahead APPROXIMATES a Belgian dynamic tariff and cannot reproduce a supplier's B x spot + A formula. There is deliberately no alias, because aliasing would require the contract's own two coefficients - which are in no public index, so the add-on would have to invent the numbers it prices a battery against.

False claims corrected across DOCS.md, price_forecast.py, price_history.py, static_tariff.py, DECISIONS/FCST-04 and CHANGELOG.md: eneco_eddy was described as a 'Belgian imbalance product' in all of them and is not. The quarter-hourly examples now cite Belgian day-ahead, which genuinely settles per quarter since the 15-minute MTU of 2025-10-01 - the mechanism was right, only the example was invented. export_price_eur_kwh's docstring claimed no shipped supplier publishes a dynamic injection curve; Eneco publishes 0,1 x BELPEX-H - 1,188 on the same card as its consumption formula, identical on the January and September 2026 cards, and it is VAT-exempt while consumption is at 6%. _points_from_json() is deleted: it read a shape no reachable feed publishes and was graded only against fixtures built from the same guess.

PriceProduct.REALTIME and the widening machinery are KEPT although eneco_eddy was their sole declarer, with a DECISIONS note recording that no shipped plugin declares it until one does - otherwise a later reviewer finds a mechanism with zero callers and reasonably deletes the AC2 seam.

ONE CHECK HAD QUIETLY STOPPED BEING ONE. test_price.py 4b proved code_leaks()'s docstring strip is load-bearing by using a real shipped docstring that a raw scan false-positives on - and that docstring belonged to engie_dynamic. Deleting the plugin would have left the check PASSING on a tree with the strip removed. The stimulus is rebuilt inside the test file and the shipped set asserted clean separately.

BACKWARD-INCOMPATIBLE: a config legal on the current version is refused on the next. That is the point of the refusal, and the CHANGELOG says so in the shape PKG-07 established for slew_w.

Evidence: 29/29 green, with the baseline established at 8d09b30 before any edit. Nine mutations, nine red, two of them zero-byte edits; two counts are lower bounds because the run hit a traceback mid-suite.
2026-09-13 02:54:59 +02:00
adminandClaude Opus 5 f620f7cf80 TARIFF-07: three plugins that never fetched a price are withdrawn, and a site configured on one is told so
checks / unit (pull_request) Successful in 7m12s
engie_dynamic, totalenergies_dynamic and eneco_eddy are removed from PROVIDERS
and from every surface that listed them. They did not stop working - they never
worked. Re-verified by hand on 2026-09-13 before deleting anything: 404 with
Kong's "no Route matched with those values"; a TLS handshake that fails because
api.totalenergies.be is absent from the certificate's SAN list; 404 with an
empty body. All three are also structurally impossible - the coefficients are
per-contract and behind a customer login, there is no regulator index, and
"Eddy" is not an Eneco product at all.

⚠️ BACKWARD-INCOMPATIBLE. A config legal on the current version is refused on
the next. That is the point of the change: §11.4 turned every failed fetch into
a silent fall back, so a site on one of these ids has been running on
static_profile - or on nothing - since the day it was set, with nothing on its
status page ever differing from a working provider that never found a cheap
hour. validate.py now refuses a retired id with a message naming the id, the
endpoint, what that endpoint answers, which tariff has really been in force,
and what to set instead. No version number is picked here; that depends on a
fact about the live box.

⚠️ config.yaml's price_provider enum deliberately KEEPS all three ids. The
Supervisor validates the options file against that enum before the add-on
starts, so removing them would make an existing config schema-invalid and the
refusal above would never be reached on exactly the sites that need it.
PROVIDERS is two; the enum stays six; test_price.py asserts the schema offers
exactly PROVIDERS | RETIRED_PROVIDERS | {"off"}, derived rather than typed, and
register() refuses to re-register a retired id.

PriceProduct.REALTIME, _WIDE, Uncertainty.REALTIME_PRODUCT and WIDEN_FACTOR's
2.5 are KEPT with no shipped declarer - it is the AC2 seam, and a reviewer
finding a mechanism with zero callers would reasonably delete it. Recorded in
DECISIONS/TARIFF-07-withdrawing-three-plugins.md and pinned by a check.

False claims corrected: every "Belgian imbalance product" claim about
eneco_eddy (DOCS.md, price_forecast.py, price_history.py, static_tariff.py,
FCST-04, CHANGELOG - the two released entries annotated in place rather than
rewritten); the quarter-hourly examples now cite Belgian day-ahead, which
really does settle per quarter since the 15-minute MTU; and
export_price_eur_kwh's "no shipped supplier publishes a dynamic injection
curve", which is false - Eneco publishes 0,1 x BELPEX-H - 1,188 and ENGIE
(B x eSpot_15) - A, same two-coefficient shape with its own coefficients and
its own VAT treatment, so the scalar is a simplification and not the market.

DOCS.md's provider table is two rows and now says honestly what a Belgian
dynamic customer should do today, including that generic_dayahead cannot
express B x spot + A and that price_markup_eur_kwh: float(0,1) could not hold a
negative offset. The formula provider is referenced as planned, not built.

_points_from_json() went with the plugins it served: it read a shape no feed
anybody here has ever reached publishes. The one place it survives is a
negative stimulus, named row_shape_body().

Nine mutations driven, all red; table in the decision record. 29/29 suites
green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-13 02:46:15 +02:00
admin 8d09b30f16 Merge pull request 'TARIFF-14: the window a point covers is the point's, not an assumed hour' (#80) from TARIFF-14 into release/1.0
_Series.at() hardcoded a one-hour coverage window and returned the FIRST point whose start fell inside it. Belgium moved to a 15-minute MTU on 2025-10-01 and the live feed returns 96 points at 900 s, so on the real series at() answered with the wrong point for 45 of every 60 minutes - and then drifted, because from the second hour on the first match was an earlier point still inside its own one-hour window. Reproduced against shipped code: asking for 03:00 returned the 02:15 point.

It reached the trade decision directly. arbitrage._act() calls _point_at(plan, now) -> series.at(now), then tests here.start == cheapest.start / == dearest.start to decide whether this interval is the buy or the sell one. With a stale and drifting point that identity test almost never matches the extremum, so the strategy idled through exactly the interval it exists to trade. It also tested here.provisional and the section 17 negative-price guard on the wrong point's price.

Nothing caught it because no test drove at() on a sub-hourly series: the DST tests count points on 23- and 25-hour days but never ask for a quarter-hour, and every price fixture before TARIFF-08 was the invented hourly row shape. static_tariff.py emits hourly points, so static sites are immune - the defect was reachable only by the dynamic path TARIFF-08/09 had just made work.

Each point's window now runs to the next point's start, CAPPED AT ONE HOUR. The cap is not cosmetic: uncapped, the rule would have stretched the point before a three-hour FCST-04 hole across the whole hole, introducing hole-filling as a side effect of fixing a lookup. The last point takes the span of the point before it, because a flat one-hour tail answers at(00:30) tomorrow with today's 23:45 price - reaching for a neighbour across a day boundary, which section 11.4 forbids.

That tail choice was independently validated by TARIFF-12, which merged during this work. A served plan can now be up to TARIFF_MAX_AGE_S (5400 s) old, and price_horizon_h permits a 1 h horizon - so now can walk past the end of a short series. Under the shipped rule at() raises Absence and the site falls to the static profile; under the rejected flat tail the same stale plan would have gone on trading against an expired :45 quarter for up to 45 minutes. Re-read the rule if TARIFF_MAX_AGE_S is ever widened.

covers() shared the bug exactly - it has no rule of its own and delegates to at() - so one change fixes both. window() did not: it grades a start against a half-open range and never asks how long a point lasts. at() now depends on sortedness AND ESTABLISHES IT ITSELF, sorting once in _Series.__init__, because nothing else guaranteed it: _validate_raw() covers only the load_prices() path, _parse_points() only the store read, and PriceRead.series() never enforced anything. A series handed its points reversed now answers identically, as a check.

Evidence: 29/29 green on the merged tree. Eight mutations, all red. The killing one restores the shipped rule (+68 B) and is red on 24 named checks across both files, including the one named THE PROPERTY THE DEFECT BROKE - an arbitrage-level check driving the shipped decide() over a 96-point day whose cheapest is a :45 and dearest a :15, asserting as its own check that the old and new rules return different points. The tail mutation is a ZERO-BYTE edit and is caught. It is stated honestly that five of the eight leave test_arbitrage.py green, because those are at()-level properties no arbitrage fixture reaches.

Named and not fixed: price_forecast._walk() steps section 8.3's gate in whole hours, so on a sub-hourly series it reads one point per hour and never sees the :15/:30/:45 flags. Unreachable today - FCST-04 carries whole hours and the feed publishes whole days - but that is an assumption about producers, not a property of the gate.
2026-09-13 01:56:04 +02:00
glenn schrooyenandClaude Opus 5 028c3e53fb Merge release/1.0 (TARIFF-12) into TARIFF-14
checks / unit (pull_request) Successful in 7m13s
Second base move while this card was open: 559f0eb -> a6e64fb. Only
CHANGELOG.md conflicted; test_arbitrage.py auto-merged (TARIFF-12 used
section 8b, this card uses 10, so the two do not collide).

CHANGELOG: both entries under the single ## Unreleased in MERGE ORDER -
TARIFF-12's row and section precede this branch's, because it landed
first. No version number is picked; PKG-06/07's note stands untouched.

WHAT TARIFF-12 CHANGES FOR THIS CARD - written out in decision note §8
rather than left silent, because the question is a fair one.

A served plan can now be up to TARIFF_MAX_AGE_S = 3 * TARIFF_REFRESH_S
= 5400 s old, where before it was at most one refresh interval, and
at() is what answers "what does it cost right now" out of that older
plan.

* PER-POINT SPAN: unaffected, structurally. Every window is derived
  from the neighbours' ABSOLUTE start instants - datetimes in a frozen
  dataclass in a tuple, which cannot move - and _cover_table() runs
  once at construction over data that cannot change. A plan's AGE is a
  property of fetched_at/tariff_at, never of a window. The index cannot
  go stale, which is a small positive under TARIFF-12: a kept plan now
  lives longer and there is nothing in it that needed recomputing.

* ONE-HOUR CAP: unaffected. It bites only at a hole; walking further
  into a series crosses more windows, each resolved on its own
  neighbours.

* THE TAIL: reachable for the first time, and §2's choice survives it.
  At the shipped 48 h horizon the tail is ~46.5 h beyond anything a
  90-minute-stale plan can be asked about. But price_horizon_h is
  int(1,72), so a 1 h horizon is a SUPPORTED configuration, and a
  90-minute serve-stale window against a 1 h series puts `now` past the
  end. Under the chosen rule a quarter-hourly 1 h series ends at :45,
  its window closes on the hour, at() raises Absence, _act() idles and
  §11.4 sends the site to the static profile. Under the REJECTED flat
  one-hour tail the same stale plan would have traded against the :45
  print for up to 45 minutes after the interval it describes had ended.
  So TARIFF-12 does not weaken the decision - it is the first thing
  that makes it reachable, and it selects the same answer. The rule
  should be re-read if TARIFF_MAX_AGE_S is ever widened again.

* TariffPlan.from_feed is a pure predicate on `origin` and TARIFF-12
  keeps or replaces a WHOLE PLAN OBJECT. It never splices two reads
  into one series, so it cannot hand at() an unsorted or duplicated
  point list.

RE-GRADED ON THIS BASE, not carried over: all 29 test files green;
app/price.py byte-identical to the TARIFF-10 merge (90d8ca31f7c0109d,
85953 B) because TARIFF-12 does not touch it; M1 re-driven at +68 B,
red on the same 24 named checks across both files; M3 re-driven at
ZERO BYTES, red on the same 4. Each purged before the run and reverted
with the fingerprint re-checked, from a card-specific harness at
/home/homelab/.cache/t14-mut/ with a mktemp backup inside it.

This branch contributes zero diff to app/main.py and app/static_tariff.py.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-13 01:47:17 +02:00
admin a6e64fb66e Merge pull request 'TARIFF-12: a degradation is not a refresh' (#79) from TARIFF-12 into release/1.0
run_tariff_refresh()'s docstring promised 'A FAILURE DOES NOT CLEAR THE PLAN' and guarded the wrong thing. resolve_tariff() deliberately never raises on a feed failure - load_prices() returns no_data rather than letting a traceback into the planning tick - so a dead feed returned a degraded TariffPlan, the try SUCCEEDED, and the good plan was overwritten. The except branch that promised otherwise was reachable only by unexpected exceptions; a supplier outage, the case the docstring names first, never reached it. With static_profile unset - the shipped default for a dynamic site - a complete 48-hour plan was replaced by origin='none', 0 points.

A 40-hour soak drove a 90-minute outage and measured it: the plan was gone at the FIRST failed fetch, ~27 min after the last good one, with TARIFF_MAX_AGE_S (5400 s) governing nothing. There was no serve-stale window; a transient blip had the same effect as a long outage.

The fix keeps a plan that is still fresh, via fresh_tariff_plan() - reused rather than transcribed, and pinned on the AST. The §11.4 static fallback does NOT replace a fresh provider plan: every point _degrade() builds is provisional, so firm_points() is empty and arbitrage and cheap_now are already off, while a day-ahead price for a given hour does not change once published - a 30-minute-old observation is exactly as true about 19:00 as one fetched now. The fallback is strictly less actionable and no more current. Deliberately NOT symmetrical: a standing plan that is itself a degradation IS replaced, or a static site's plan would freeze at the window it was first generated for. tariff_at does not move on a kept plan, so TARIFF_MAX_AGE_S now genuinely governs and a kept plan stays distinguishable from a fresh one.

Rebased onto TARIFF-10 and re-verified rather than assumed. The interaction was confirmed by driving it: nothing on this path reads arbitrage_enabled, and the static-profile case was driven twice consecutively because a guard that latched there would freeze a site's plan for the life of the install. One honesty fix followed from TARIFF-10 - the kept-plan log said 'arbitrage stops with it', which is false on a site that fetches without trading, and now reads 'arbitrage, where it is armed, stops with it'.

Evidence: 29/29 green on the merged tree, including TARIFF-10's new reachability section. Eleven mutations, all re-driven on the merged tree because cached reds from the old base are not evidence about the new one; eleven red, no survivors, every revert fingerprinted after the run as well as before.

Two process findings recorded rather than quietly fixed. Two drafts of one check were red against a CORRECT tree because they substring-matched the docstring that states the property - a text search over a function grades its prose as much as its behaviour, which is habit 4 in a third costume; the check now scans AST symbols and non-docstring constants, with positive controls. And the mutation harness was overwritten mid-session by a concurrent agent's harness sharing the scratchpad, pointing it at a different worktree - MUTATION.md habit 18's second bullet almost verbatim. It surfaced as an error rather than a silent wrong result, and the per-run revert fingerprints are what prove the worktree was never left mutated.
2026-09-12 21:39:24 +02:00
glenn schrooyenandClaude Opus 5 435e1b6f7c Merge release/1.0 (TARIFF-10) into TARIFF-14
checks / unit (pull_request) Successful in 7m10s
CI on PR #80 was NOT a test failure. The workflow's "grade the merge, not
the branch" step refused the PR because it does not apply, and said so
itself: "REFUSED: refs/pull/80/head does not merge into release/1.0.
NOTHING HAS BEEN TESTED. This is not a test failure - no test ran."
Conflicts were CHANGELOG.md and test_price.py; app/price.py auto-merged.

CONFLICTS RESOLVED

* CHANGELOG.md - both entries belong under ## Unreleased. TARIFF-10's
  table row and section come first because it landed first. No version
  number is picked here; PKG-06/07's note stands untouched.

* test_price.py - BOTH cards appended a section "13". Nothing is
  dropped: TARIFF-10's stays 13, this card's is renumbered 14, and
  TARIFF-10's temp-directory teardown stays at the end of the file. The
  two share no fixture, option dict or helper name.

WHAT TARIFF-10 CHANGES FOR THIS CARD - nothing in the reasoning, and the
check is recorded rather than asserted (decision note §7).

In app/price.py the two do not meet: TARIFF-10 added provider_id() and
provider_configured() beside price_option() and routed load_prices()
through them - the option-reading layer, touching nothing in _Series,
at(), covers(), window() or _cover_table(). The merged at() is
byte-identical to the one the mutations were driven against.

Behaviourally TARIFF-10 makes run_tariff_refresh() run with arbitrage
OFF, so a provider-configured site now has a standing plan - and
therefore a _Series - where before it had none. It does NOT add an at()
consumer to the 30 s tick: static_status() builds a STATIC plan (hourly,
immune), main.py calls only plan.words on the fetched plan, and
price_forecast.price_permits()/begin_phase() have no caller in main.py
at all. So TARIFF-10 neither creates nor closes any exposure this card
is about - it enlarges the defect's reach the moment such a site arms
trading, which strengthens the fix without changing it.

RE-GRADED ON THE MERGED TREE, not carried over

* all 29 test files green;
* all EIGHT mutations re-driven, purged before each run, each reverted
  with the fingerprint re-checked. Same verdicts and same red-check
  counts as before the merge: M1 (the shipped one-hour scan, +68 B) red
  on 24 named checks across both files; M3 (the ZERO-BYTE tail rule) red
  on 4; M2/M4/M6 on 1 each, M5 on 2, M7 on 16, M8 on 14.
* Clean app/price.py on the merged tree: 90d8ca31f7c0109d, 85953 B.
* None of TARIFF-10's 650 new lines of checks moves under any of the
  eight - they grade resolve_tariff()'s reachability from a
  configuration, not the lookup.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-12 21:38:28 +02:00
Claude f5ea39784d Merge release/1.0 (TARIFF-10) into TARIFF-12
checks / unit (pull_request) Successful in 7m16s
CHANGELOG.md was the only conflict: both entries belong under the same
`## Unreleased` heading, TARIFF-10's first because it merged first. No
version number picked.

app/main.py auto-merged, and was re-read because of it. TARIFF-10 changed
WHICH installs run `run_tariff_refresh()` (`arbitrage.enabled(opts)` ->
`tariff_refresh_wanted(opts)`); this card changed what that task does on
a failed fetch. Nothing in the keep/replace reasoning changed, and the
two newly reachable configurations are now DRIVEN in test_arbitrage.py
8b rather than reasoned about:

  * a provider configured with trading DISARMED is the case the guard
    was designed for, and needs no second fixture because nothing on
    this path reads `arbitrage_enabled` at all - asserted over the AST's
    symbols and non-docstring constants, after two drafts of that check
    went red on a CORRECT tree for matching the word "arbitrage" in a
    docstring and in a log line;
  * `price_provider: off` + a `static_profile` resolves to the
    CONFIGURED product (`origin: "static"`), so `standing.from_feed` is
    false on every refresh and the guard never latches - driven twice in
    a row, because a guard that latched there would freeze a static
    site's plan at its first window for the life of the install.

Also: the kept-plan log line said "arbitrage stops with it", which after
TARIFF-10 is false on a site that fetches without trading; it now says
"arbitrage, where it is armed, stops with it". The keep decision is
strictly downstream of `resolve_tariff()`, so the §8.3 carry into
price_history - TARIFF-10's whole point - is unaffected either way.

All 29 suites green on the merged tree, including TARIFF-10's new
test_price.py section 13. All eleven mutations RE-DRIVEN on the merged
tree rather than carried over: eleven red, no survivors, every revert
fingerprinted. M11 is new - making the guard conditional on
`arbitrage.enabled(self.o)` (+86 B) is red on five named checks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-12 21:31:27 +02:00
glenn schrooyenandClaude Opus 5 7f2c4e20e1 TARIFF-14: the point whose own interval covers the instant
checks / unit (pull_request) Failing after 2s
`_Series.at()` asked `s <= u < s + 1 h` of every point IN LIST ORDER and
returned the FIRST match. On the hourly shape every fixture in this
repository carried before TARIFF-08 that is right by accident. Belgium
moved to a 15-minute MTU on 2025-10-01 and the live day-ahead feed
returns 96 points at 900 s, where it is wrong twice over - measured
against the shipped code at 5059e28:

    series: 02:00=0.10  02:15=0.11  02:30=0.12  02:45=0.13  03:00=0.14

    ask +15min -> 02:00  0.10   <-- WRONG (wanted 02:15)
    ask +60min -> 02:15  0.11   <-- WRONG (wanted 03:00)
    ask +75min -> 02:30  0.12   <-- WRONG (wanted 03:15)

Inside the first hour the :00 point wins because its hour has not run
out. From the second hour on the answer is not even the top of the hour
- the first point still inside its own hour wins - so the error
compounds with depth into the series.

`arbitrage._act()` tests `here.start == v.cheapest.start` /
`== v.dearest.start` to decide whether this interval is the buy one or
the sell one. That identity almost never held, so the strategy idled
through exactly the interval it exists to trade, and read §8.3's
`here.provisional` and §17's `price < 0` off the same wrong point.
`static_tariff.py` emits hourly points, so static sites were never
affected; this is reachable only down the dynamic path TARIFF-08/09
made work.

THE RULE: a point covers from its own start until the next point's
start, and never more than one hour. Quarter-hourly takes 900 s,
hourly is unchanged, and a MIXED series - FCST-04's carry-forward puts
hourly carried points beside quarter-hourly firm ones, spacing
[900, 3600] in 70 % of soak samples - takes each point's OWN gap.

The one-hour cap is load-bearing: FCST-04 leaves an hour OUT when
nothing published it and yesterday has no value, and `at()` raising
Absence for it is what sends the caller to the static profile (§11.4).
Uncapped, "until the next point's start" would stretch the point before
a three-hour hole across the whole hole and answer every instant in it
with a stale price.

THE LAST POINT is given the same span as the point before it (one hour
when it is the only point). A flat one-hour tail would answer 00:30
TOMORROW with today's 23:45 price. The cost - a mixed series ending at
the firm/carried seam loses its last 45 minutes to Absence - is argued
in DECISIONS/TARIFF-14-the-window-a-point-covers.md §2 rather than
buried; the direction is the safe one.

`at()` now reads a point's NEIGHBOURS, so it would depend on
sortedness. It establishes that itself: the lookup index is sorted by
UTC start once, at construction, so no caller's list order reaches it.
A series handed its 96 points in reverse answers identically, and that
is a check. `covers()` shared the defect exactly - it has no rule of
its own, it asks `at()` - and is fixed by the same change; `window()`
did not, and its docstring now says why.

Nothing about §9.3 or a point's `start` changed. PricePoint still has
three fields; the span is derived from the neighbours, never stored.

Graded by test_price.py section 13 (900 s, 3600 s, mixed, holes, the
tail, caller order, duplicate starts, covers()/window(), the no-data
contract) and end to end by test_arbitrage.py section 10, which drives
decide() over a 96-point quarter-hourly day whose cheapest and dearest
quarters are deliberately NOT on a whole hour, plus §17, §8.3 and the
mixed series. Eight mutations, all red, including a ZERO-BYTE one on
the tail rule; byte deltas and what the arbitrage file could not see
are in the decision note §6. All 29 test files green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-12 21:22:28 +02:00
admin 559f0eb022 Merge pull request 'TARIFF-10: a configured provider fetches prices, whether or not trading is armed' (#78) from TARIFF-10 into release/1.0
run_tariff_refresh() was created only under `if arbitrage.enabled(opts)`, and it is the sole caller of resolve_tariff(), which is the sole caller of price_history.carry(). arbitrage_enabled defaults to false, so on a shipped install with a provider configured nothing was ever fetched, no plan resolved, /data/price_history.json never written, FCST-04's Price history row read `empty` forever, and TARIFF-09's warning never fired. Every price check constructs its own resolve_tariff() call, so the suites exercised a path production never reached.

Measured, not inferred: a 40-hour soak ran two instances side by side - the shipped default made zero feed requests and never created its store across all 802 samples, while an identical instance with arbitrage armed resolved a 132-point plan from the same feed. DOCS.md:1348 has claimed the fixed behaviour all along.

The product owner's decision is Option A: a configured provider fetches. Trading stays separately gated, and that is graded directly rather than inferred - the real control loop is driven on a resolved provider plan with arbitrage_enabled the only option moving: off gives +180 W self-consumption, on gives -5000 W at the cheapest hour and +5000 W elsewhere. The armed rows are what make the unarmed row evidence.

ONE ACCEPTANCE CRITERION WAS DELIBERATELY NOT IMPLEMENTED LITERALLY, and the deviation is right. The criterion said no fetch task at all when price_provider is off, whatever arbitrage_enabled says. That breaks a configuration validate.py explicitly accepts - provider off, static_profile set, arbitrage armed - by leaving tariff_plan at None forever, silently disarming a strategy the UI reports as running. Driven both ways. The guard is therefore `provider_configured(opts) or arbitrage.enabled(opts)`; the criterion's stated harm is still fully met, because with the provider off resolve_tariff() answers out of the static profile and opens no socket, and the true shipped default starts no task at all. DECISIONS section 3 records the trade.

cheap_now() is NOT wired, deliberately. On a provider-only site static_status() returns False on every cycle, so adopting it would take the maintenance cycle's paid grid top-up from never to sometimes - a change to what spends money, and it needs its own decision rather than riding along on this one.

Evidence: 29/29 green. 13 mutations over 14 runs, each on a fresh tree copy, purged, :ro, -B. The shipped defect at -4 B is red on 7 named checks. The reachability check grades reachability from a CONFIGURATION rather than from the `if` statement - the `if` is exactly what someone meant to write. The instrument itself was mutated too, both +0 B, both red. One real survivor was found and closed in the same commit: provider_id() dropping .strip() left four price suites green. And one fixture defect was found and fixed - five minutes of P1 telegrams made the peak guard blind depending on where the wall clock sat in the billing quarter, so the armed baseline failed to trade on some runs.
2026-09-12 21:07:55 +02:00
Claude c81289527f TARIFF-12: a returned degradation is not a refresh
checks / unit (pull_request) Successful in 7m12s
`run_tariff_refresh()` promised in its own docstring that "a failure does
not clear the plan", and enforced it with `except Exception` - a branch a
supplier outage never reaches. `load_prices()` states the reason in its
own words: it "RETURNS, NEVER RAISES, for every failure a site can
actually have", because a degradation arriving as a traceback into the
planning tick would stop the controller regulating over an expired
certificate. So a dead feed RETURNS a §11.4 plan, the `try` succeeds, and
the assignment below it overwrote the good plan unconditionally. With
`static_profile` unset - the shipped default, and the normal state of a
dynamic-tariff site - `_degrade()` returns `TariffPlan(no_data, None,
"none")`: a complete 48-hour plan replaced by nothing, at the FIRST
failed fetch, ~27 minutes after the last good one. `TARIFF_MAX_AGE_S`
(5400 s) never governed anything and there was no serve-stale window at
all.

Fourth card in one family: FCST-04's M4 (`not ok` only), FCST-05's HIGH
(`(JSONDecodeError, OSError)` only), FCST-06 (`not self.ok` only), and
now `except Exception` only, for a path that never raises.

The question is now asked once, after the attempt, from the evidence
that exists - FCST-06's answer, one module over. The evidence is
`TariffPlan.from_feed` (did this resolution observe the configured feed)
and the shipped `fresh_tariff_plan()`, reused rather than transcribed. A
degradation replaces a standing observation only once that observation
has aged out, so `TARIFF_MAX_AGE_S` is now the serve-stale window it
reads like.

A §11.4 static fallback does not replace a fresh provider plan either,
although it carries real prices: every point `_degrade()` builds is
`provisional=True`, so `firm_points()` is empty, `arbitrage()` refuses
and `cheap_now()` is False - everything a consumer may act on is already
off - while a day-ahead price for a given hour does not change once
published. The rule is deliberately not symmetrical: a standing plan
that is itself a degradation IS replaced, or an outage would freeze the
operator's static profile at the window it was first generated for.

`tariff_at` does not move on a kept plan. It is the moment of the fetch
that produced the plan standing, `fresh_tariff_plan()` judges the age
from it, and an unmoved timestamp is the only thing that tells a kept
plan from a freshly fetched one.

test_arbitrage.py section 8b drives the whole chain with nothing stubbed
but the socket: the transport raises the way aiohttp does, load_prices()
returns no_data, _degrade() builds the §11.4 plan, and the shipped
run_tariff_refresh() decides. A check that patched resolve_tariff to
RAISE would have passed against the defect. Ten mutations, ten reds, no
survivors; 29/29 suites green.

Out of scope and unchanged: the persisted price store still does not
back the plan during an outage (FCST-03), and the status banner and the
Arbitrage row's lag are separately carded.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-12 21:06:45 +02:00
adminandClaude Opus 5 e6c45a61e7 TARIFF-10: configuring a price provider is what fetches prices
checks / unit (pull_request) Successful in 7m19s
amain() started the background tariff refresh only when arbitrage_enabled was
on, and that option ships false. run_tariff_refresh is the only caller of
resolve_tariff(), which is the only caller of price_history.carry() - so on a
shipped install with a price provider configured nothing was ever fetched: no
plan resolved, /data/price_history.json never created, the Price history status
row reading `empty` for the life of the install, and TARIFF-09's short-horizon
warning unable to fire because there was no fetch to warn about. Measured over
a 40 h two-instance soak on one feed: 0 requests and `present: false` across all
802 samples, against a resolved 132-point plan on the instance with arbitrage
armed.

git log -S puts the refresh task, its arbitrage.enabled() guard AND the DOCS.md
sentence saying a configured provider still fetches in the same commit - OPT-04,
bdd7f18; config.yaml has carried the claim since TARIFF-02, 852fc7c. The prose
and the code disagreed from the moment both existed and nothing compared them.
config.yaml and both translations additionally carried a claim that had rotted
the other way - "nothing consumes the price series yet", untrue since OPT-04
shipped the consumer. All four surfaces now say what the add-on does, and a
check pairs DOCS against the DRIVEN behaviour so they cannot drift again.

The gate is tariff_refresh_wanted() = provider_configured(opts) or
arbitrage.enabled(opts), and the `or` is a STRICT WIDENING: every configuration
that started the task before still starts it. The second term is not there for
symmetry. price_provider: off + a static_profile + arbitrage_enabled: true is a
configuration validate.py explicitly ACCEPTS and it trades - measured, at
cycle_cost_threshold_eur 0.05 on a 0.20/0.284 day/night profile,
ARBITRAGE_CHARGE -5000 W. Gating on the provider alone would leave that site's
tariff_plan at None forever and answer "no tariff plan has been resolved yet" on
every cycle, silently disarming a strategy the UI says is running.

Nothing acquires a periodic HTTPS call. With price_provider off, resolve_tariff()
answers out of the static profile without opening a socket (driven and counted:
0 requests), and the true shipped default - no provider, no trading - starts no
task at all. What decides whether a request leaves the machine is price_provider
and only that. The literal reading of the ticket's AC2 - drop the `or` - is one
token and is recorded in DECISIONS §3 with what it costs, for the owner.

AND NOTHING TRADES THAT DID NOT TRADE BEFORE. cycle()'s
`if arbitrage.enabled(self.o):` is untouched, and that is graded directly rather
than inferred from the gate having moved: Controller.cycle() is driven on a
resolved provider plan with the peak guard asserted clear, and arbitrage_enabled
is the only option that moves between the rows - false/false give +180 W
("loop -> 180 W") on both the cheap-now and dear-now plans, true/true give
-5000 W and +5000 W ("arbitrage -> ...").

TariffPlan.cheap_now() is deliberately NOT wired, decided rather than inherited
(DECISIONS §4). A provider-backed plan now exists on every configured site, so
Cheap window could read it - and Cheap window feeds the maintenance cycle's grid
top-up, the one place a price moves energy without trading being armed. On a
provider-only site static_status() returns False on every cycle, so adopting it
would take a paid grid top-up from never to sometimes. self.cheap stays on
static_status(); what adoption would take is listed rather than left implied.

THE CHECK THAT WOULD HAVE CAUGHT THIS is test_price.py section 13, and it grades
REACHABILITY FROM A CONFIGURATION rather than the shape of an `if` - the `if` was
exactly what somebody meant to write. amain()'s guards are EVALUATED against real
option dicts, whatever they are spelled as; the call graph under app/ is walked
to see whether the tasks those guards start arrive at resolve_tariff(), wherever
it has moved to; and then the fetch is DRIVEN with a counting transport, because
a graph is an argument and a counted request is an observation. What the
instrument cannot see is printed on every run, largest hole first.

Evidence: 29/29 green. 13 mutations in 14 runs - M10 re-run as M10b once it was
closed - each applied to a fresh cp -a of the clean worktree rather than
reverted (MUTATION.md habit 18), purged, :ro, -B. The
shipped defect - the gate back at arbitrage.enabled(opts), -4 B - is red on
seven named checks. Dropping the `or` (-27 B) is red on the strict-widening row;
removing cycle()'s trading gate (+8 B) is red on both AC3 rows. The instrument
is mutated too (habit 16): _reaches() always True and _task_sites() returning []
are both +0 B and both red.

Two findings are recorded rather than rounded off. provider_id() dropping
.strip() (-8 B) was a real SURVIVOR across four price suites - nothing fed a
padded provider id - and is closed in this commit by the padded-id pair; and the
first draft of the AC3 fixture fed five minutes of P1 telegrams, so whether the
peak guard was blind depended on where the wall clock sat inside the billing
quarter and the armed baseline failed to trade on some runs. It is 900 s now,
and the guard's clear state is a check rather than an assumption.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-12 20:58:48 +02:00
admin 88c59a9918 Merge pull request 'TARIFF-09: ask the feed for the horizon the operator configured' (#77) from TARIFF-09 into release/1.0
generic_dayahead.fetch() accepted since and until and ignored both, requesting the bare URL - so the feed returned one Brussels day against a 48 h horizon. Everything past today was carry-forward, provisional, and dropped from the trade set by 8.3, which means the optimiser never saw tomorrow as firm at any hour of the day. That is not what FCST-04's fill was built for: it covers the pre-publication window until the ~13-14 h CET auction, after which a healthy feed should hand over tomorrow in full.

Measured before, 2026-09-10T09:00+02:00 on the live feed: 15 firm hours and 33 carried of 48, firm local days = [2026-09-10] only, buy and sell both on the same day. After, against a real two-day body: 39 firm and 9 carried, firm local days = two, buy 13:45 on 09-09 and sell 19:45 on 09-10. At 23:00 the difference is the whole verdict - before, 1/47 and arbitrage REFUSED at spread 0.0464; after, 25/23 and allowed. The 9 hours still carried are the tail running into a third Brussels day, which no day-ahead market publishes.

The range is derived on the BRUSSELS calendar, both ends graded and both driven red by mutation - the API takes dates while since/until are instants, so deriving either end from the UTC date loses the last one or two hours of every day. That is _bdate()'s lesson from FCST-04 one layer up, except that here the hours are never fetched at all rather than misfiled.

price_feed_url is an EXACT URL, used verbatim, nothing appended - and the deciding argument came from the codebase rather than from taste: generic_dayahead declares a second egress host, web-api.tp.entsoe.eu, whose date parameters are not start/end. An override to the other DECLARED host is exactly where merging would append parameters that host ignores and produce the short horizon anyway. The cost is made visible by a fetch-time WARNING naming the consequence and the start..end it would have asked for, with a control asserting the default path stays silent, and dated_feed_url() refuses to append to a base that already carries a query string, so blind appending is structurally unreachable.

The plugin contract is deliberately NOT enforced yet. Obliging the three unverified-endpoint plugins of TARIFF-07 to pass a range would be satisfied by an invented parameter name - TARIFF-08's defect moved from the response to the request and shipped as a guarantee. The roster of plugins that consume the range is pinned as a literal instead, so the next supplier has to decide.

Evidence: 29/29 green. 11 mutations, no survivors, each applied to a fresh copy of the clean tree rather than reverted. Includes the Brussels/UTC boundary pair, one at +1 B, and one at +0 B that changes a single price digit inside the committed capture and is caught by the re-derived sha256. Two live fetches only, 66 s apart, respecting the feed's rate limit; the second returned bytes identical to TARIFF-08's capture, so the pre-publication case is a second recorded URL for the same bytes with the identity re-derived rather than asserted.

MUTATION.md habit 20: a fixture grades the response, and nothing about the REQUEST is reachable from one. canned() had recorded the URL since TARIFF-02 and no check ever asserted on it, which is how a plugin that never asked for a date range went unnoticed for four tickets.
2026-09-10 10:44:15 +02:00
Claude Opus 5 17b7b447c4 TARIFF-09: ask the day-ahead feed for the horizon it was handed
checks / unit (pull_request) Successful in 7m10s
`generic_dayahead.fetch()` accepted `since` and `until` and requested the
bare `?bzn=BE`, which answers ONE Brussels day. Everything past today was
then filled from yesterday's profile by FCST-04, flagged provisional, and
§8.3 drops provisional hours from the trade set -- so the optimiser never
saw tomorrow as firm, at any hour of any day, and nothing reported it.

Measured on 4257e38 at 2026-09-10T09:00+02:00, 48 h horizon, live feed,
yesterday seeded through the shipped read path:

    60 firm points (15 whole hours) + 33 carried hours of 48 requested
    carried: 24 h on 2026-09-11, 9 h on 2026-09-12
    arbitrage(0.10): buy 11:45 and sell 19:45 -- BOTH on 2026-09-10
    firm local days = [2026-09-10]

The ceiling was structural: with a one-day window, at hour H a 48 h horizon
is at most 24-H firm and 24+H carried. At 23:00 that is 1 and 47, and the
remaining firm spread (0.0464 EUR/kWh) no longer clears the default 0.10
threshold, so arbitrage is refused outright. FCST-04's fill covers the
window until the ~13:00-14:00 CET auction; it was never meant to cover
every hour of every day.

The data exists and the endpoint serves it: verified 2026-09-10T07:45:49Z,
`?bzn=BE&start=2026-09-09&end=2026-09-10` returns 192 quarter-hours across
two local days. Committed unmodified beside TARIFF-08's captures, with the
URL, retrieval time and sha256 recorded and RE-DERIVED by the replay.

THE RANGE IS DERIVED ON THE BRUSSELS CALENDAR, not the UTC one. 22:30 UTC
in summer is already tomorrow in Belgium, so a UTC-derived start asks for
the day the controller has just left and drops the day it is standing in,
for the last one or two hours of every day. That is `_bdate()`'s lesson one
layer up with a worse consequence: `_bdate()` misfiles an hour, this never
fetches it. `until` is exclusive, so local midnight belongs to the day
before it; an inverted window collapses to one day rather than a backwards
range. Both ends are graded and both are driven red by mutation.

After, same code path and window, a feed serving both days: 39 firm hours
and 9 carried (was 15 and 33), firm across TWO Brussels days, and the
permitted cycle buys on one day and sells on the next.

`price_feed_url` IS THE EXACT URL AND THE PLUGIN APPENDS NOTHING. `start`/
`end` are this feed's spelling and the provider declares a second host
whose parameters are different, so merging would produce a request nobody
typed and the short horizon anyway. The cost is real, so it is not silent:
a WARNING on the shipped logger at fetch time names the consequence and the
start..end that would otherwise have been asked for. `dated_feed_url()`
refuses to append to a base that already carries a query string, so the
blind-append shape cannot come back by edit.
DECISIONS/TARIFF-09-the-range-and-the-override.md argues the alternative.

Before publication nothing regresses and it is graded, not assumed: at
09:46 CEST the two-day request came back byte-identical to the one-day
body, and that stays a `series` of what exists with FCST-04 filling the
rest. A `no_data` there would switch a working site's arbitrage off every
morning -- the mutation that adds that rule is red by name.

The plugin contract is still NOT enforced: three shipped plugins point at
endpoints nobody has verified, and a rule obliging them to pass a range
would be satisfied by an invented parameter name -- TARIFF-08's defect
moved from the response to the request. Which plugins consume the range is
pinned as a literal instead.

11 mutations, all red, no survivors; each applied to a fresh copy of the
tree rather than reverted, through tools/mutation_run.sh. One is +0 B (a
single price digit inside the committed capture) and the re-derived sha256
catches it. MUTATION.md habit 20 records why a fixture could never have
reached this: a captured body is the answer to a URL, and no property of a
body says which URL was asked for. All 29 suites green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-10 10:35:12 +02:00
admin 4257e38e5a Merge pull request 'TARIFF-08: read the feed the add-on actually ships pointed at' (#76) from TARIFF-08 into release/1.0
_points_from_json() expected doc['data'] to be a list of {start, price_eur_mwh}. api.energy-charts.info/price?bzn=BE returns a columnar document - parallel unix_seconds and price arrays - so the shipped parser raised SourceUnavailable('the feed carried no data list') against every real response. generic_dayahead had never parsed one. With the three fabricated supplier URLs of TARIFF-07, that left manual_csv as the only working provider in a five-provider table.

It survived to release because every fixture fed the invented shape: test_price.py, test_price_history.py and test_static_tariff.py all built {'data':[{'start':...,'price_eur_mwh':...}]}, so the suite validated the parser against the parser's own assumption - a closed loop that cannot notice the assumption is wrong. Section 11.4 then made the failure silent: a parse error degrades to the static profile, which is correct and says nothing about the cause.

Verified live 2026-09-10, end to end: fetch -> parse -> assemble() -> TariffPlan returns 96 quarter-hourly points at 900 s, origin provider, product DAY_AHEAD, arbitrage allowed with a firm spread of 0.4645 EUR/kWh. First time the pipeline has completed against a real feed. Belgium moved to a 15-minute MTU on 2025-10-01, so FCST-04's H1 fix - the fill was decimating sub-hourly points to one per hour - was never theoretical; it is the shape of the live feed, and it is what makes this deliverable.

The columnar shape is read by a SECOND function, not a sniff, and feeding the row shape to generic_dayahead is now a graded no_data so the sniff cannot return quietly. Equal length refused in both directions with both lengths in the reason; the unit refused if not EUR/MWh, because MAX_PLAUSIBLE_EUR_KWH only bounds prices that are too large and a EUR/kWh switch would otherwise be invisible; nulls dropped with a count, an all-null body named as no_data; a deprecated feed warns while it still works.

Two real bodies captured as fixtures - unmodified bytes, with URL, retrieval time, unit, licence-as-returned and sha256 recorded, and the checksums RE-DERIVED by the replay rather than asserted. The invented fixtures are renamed synthetic_row_feed at every call site so the closed loop cannot re-form. Only one of four network providers has a capture, and a registry-derived check says so in its own sentence: the other three have nothing to capture, which is TARIFF-07.

Evidence: 29/29 green. 12 mutations, no survivors, purged tree and -B per TEST-15. One first-draft line was decoration and the mutation said so - the parser called brussels() per instant while assemble() already renders, so removing it changed nothing. The line was deleted rather than fenced with a check, and the mutation re-aimed at assemble() where the rule lives. MUTATION.md habit 19.

Follow-up filed as TARIFF-09: generic_dayahead ignores the since/until it is handed, so the feed supplies 24 h of a 48 h horizon and tomorrow is never firm.
2026-09-10 09:14:50 +02:00
adminandClaude Opus 5 7a5281e97f Merge release/1.0 (FCST-06) into TARIFF-08
checks / unit (pull_request) Successful in 7m10s
One conflict, in CHANGELOG.md's `## Unreleased` block, exactly as expected
from two branches landing beside each other: both `_this branch_` table rows
and both sections are kept, FCST-06's first. Nothing else conflicted -
FCST-06's `test_price_history.py` section 5d and this branch's columnar
`feed_body()` merged cleanly - and all 29 test files are green on the result.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-10 09:06:39 +02:00
adminandClaude Opus 5 6eba18968c TARIFF-08: the day-ahead feed is read in the shape it actually publishes
checks / unit (pull_request) Failing after 3s
`generic_dayahead` could not read one byte of `api.energy-charts.info`, and
never had been able to. `_points_from_json()` asks for `doc["data"]`, a list of
`{start, price_eur_mwh}` objects; that document does not exist. Fetched live on
2026-09-10 the feed returns COLUMNAR parallel arrays and no `data` key at all -
`unix_seconds` (96 UTC epoch seconds, 900 s apart), `price` (96 floats),
`unit: 'EUR / MWh'`. Driving the shipped parser against that body:
`SourceUnavailable: the feed carried no 'data' list`. It is the only shipped
provider whose endpoint anybody has reached, so dynamic pricing did not work.

AND §11.4 MADE IT SILENT, which is why it survived to release: a parse error
becomes no_data, becomes the static profile, becomes arbitrage off - the
CORRECT degradation, saying nothing about the cause. A site would have sat on
the fallback indefinitely.

AND THE SUITE COULD NOT SEE IT, because every fixture was built from the
parser's own assumption. test_price.py, test_price_history.py and
test_static_tariff.py each built `json.dumps({"data": [...]})`, a shape invented
here and published by nobody, so the checks graded the parser against the
parser's guess - a closed loop no coverage inside it could ever have escaped.
MUTATION.md habit 19 records it as the fourteenth instance of habit 4 and the
first where the fixture and the code under test shared a SHAPE, not a number.

WHAT CHANGED

* `_points_from_columnar_json()` - a SECOND function, not a sniff. A parser that
  decides which document it received is one that can decide wrong, and a
  wrong-but-parseable response then becomes a price. Feeding the row shape to
  `generic_dayahead` is now a graded no_data.
* Unequal arrays are refused in both directions, with the two lengths in the
  reason. `zip()` stops at the shorter list silently, and a truncated `price`
  array would have produced a real series missing the evening peak - traded on
  as complete, where no data at all switches arbitrage off.
* A unit that is not EUR/MWh is refused. The plugin converts with a fixed
  0.001, and `MAX_PLAUSIBLE_EUR_KWH` only bounds prices that are too LARGE.
* `unix_seconds` are read as UTC instants explicitly; §9.3's Brussels rendering
  stays in `assemble()` rather than being copied into the plugin, where a
  mutation could not tell it from a no-op (M4 proved exactly that on a first
  draft, and the line was removed rather than fenced).
* A published null is dropped and logged with a count; an all-null body is
  no_data named by this parser rather than by the boundary validator.

FIXTURES

Two unmodified response bodies under `scenarios/`, captured by hand on
2026-09-10: quarter-hourly (2026-09-10, 96 points at 900 s) and hourly
(2025-09-10, 24 at 3600 s - the same endpoint before Belgium's 15-minute MTU).
`energy-charts-be-price-captures.json` records the URL, retrieval time, unit,
licence-as-returned and a sha256 of each, and the replay RE-DERIVES the
checksums, so a fixture edited to make a check pass stops being a capture. Data
CC BY 4.0, Bundesnetzagentur | SMARD.de, via Energy-Charts (Fraunhofer ISE),
published without changes - the same BE-zone licence check DATA-02 established.

The surviving invented fixtures are renamed `synthetic_row_feed` at every call
site. `engie_dynamic`, `totalenergies_dynamic` and `eneco_eddy` are untouched
and have no capture: their URLs are unverified and withdrawing them is
TARIFF-07's open decision. Nothing should read their green checks as evidence
that those feeds parse.

MUTATIONS - 12, all red, no survivors. Purged tree, mounted `:ro`, `python3 -B`
(TEST-15). M1 drop the equal-length refusal; M2 make it one-sided; M3 drop the
unit check; M4 `assemble()` stops rendering at the Brussels offset; M5 read
`unix_seconds` as a naive local clock; M6 zero-fill a null instead of dropping
it; M7 drop the all-null refusal; M8 sniff for the row shape; M9 drop the
deprecated warning; M10 drop the dropped-nulls warning; M11 edit one price in
the committed capture; M12 point `generic_dayahead` back at the row parser -
red across all three price suites.

END TO END, LIVE, 2026-09-10: `resolve_tariff()` returns a `TariffPlan`,
`origin='provider'`, 96 quarter-hourly points, spacing [900] s, no decimation,
first point 2026-09-10T00:00:00+02:00 at 0.30608 EUR/kWh all-in. FCST-04's H1
fix was never theoretical. All 29 test files green.

STILL OPEN: `generic_dayahead` asks for no date range, so the default URL
returns one published day - a full 48 h horizon is reached only through §8.3's
carry-forward fill.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-10 08:57:32 +02:00
admin 958f5cd218 Merge pull request 'FCST-06: cannot-persist is one question, asked after the attempt' (#75) from FCST-06 into release/1.0
The module claimed 'a store that cannot persist records nothing' and enforced it against `not self.ok` only. A store with ok=True whose _save() fails - a full volume, a read-only remount - kept answering yesterday() from memory until the next restart, which is the silently-forgetting cache the docstring rejects, reached by the condition the rule did not name.

Reproduced against real volumes rather than a monkeypatch: /data bind-mounted read-only (EROFS) and a full tmpfs (ENOSPC), with a healthy-/data control asserted either side. Before: yesterday() returned a 24-point series while the disk held {}.

Fixed in the code, not the docstring - weakening the doc would have documented a cache that forgets without saying so as intended.

The design answer is the part worth keeping: 'can this store persist' CANNOT be answered before the attempt, because a directory that was writable a moment ago is not a write that completed, so os.access or a probe write is a third symptom test wearing a predicate's name. It is answered after, once: _save() -> bool, true only if the file now holds what memory holds. The `not self.ok` test in _remember() is GONE rather than joined by a second one, and a check asserts its absence from the source. This is the same shape as FCST-04's M4 and FCST-05's HIGH - a rule enforced on the conditions someone thought of - and it is the first of the three to be closed by removing a test rather than adding one.

What is undone is the MERGE, not the store: a volume that fills at 14:00 leaves the yesterday written at 13:30 still answering. The snapshot is a deepcopy - measured, a shallow copy let a store holding 12 persisted points answer 24.

Evidence: 29/29 green. Six mutations, no survivors, purged tree each run with the mutated line greped off disk. One of them exposed an instrument defect now recorded in MUTATION.md: it printed 1 FAIL when the true figure was 3, because an f-string operand called len(read.points) and .points on a no_data raises Absence by design, killing the harness mid-section. One check earlier and it would have read as a survivor. A mutation's red count is a lower bound until the run is seen to reach the end of its section.
2026-09-10 08:52:09 +02:00
adminandClaude Opus 5 c45893ffad FCST-06: "cannot persist" becomes one question, asked once, about the outcome
checks / unit (pull_request) Successful in 7m14s
price_history.py's docstring states its rule unconditionally - "A store that
cannot persist now records nothing, and both of its answers say the same
thing" - and FCST-04 enforced it against `not self.ok` only, which is one
SYMPTOM: the durable volume is missing. `ok` is about the DIRECTORY, and a
read-only remount and a full disk both leave it True. Reproduced on 5059e28
against two real volumes, a /data bind-mounted read-only and a full tmpfs:

    last_error = 'could not write ([Errno 30] Read-only file system)'
    words      = '2026-01-14 (24 points over 24 h); 2026-01-15 (24 points
                  over 24 h) [could not write (...)]'
    yesterday()= series, 24 points
    ON DISK    = {}

`words` and `yesterday()` were describing two different stores, and the 24
points were gone on the next restart with nothing saying so - the
silently-forgetting cache that rule exists to refuse, reached by the condition
it did not name. No shipped path reaches it: /data is the Supervisor's durable
volume and this module is its only writer. The defect is that a guarantee the
module states about itself did not hold in one condition.

Route taken: `_remember()` refuses to retain what it could not write, and the
docstring's claim stands as written. It is now ONE derived predicate rather
than two places testing two symptoms. "Can this store persist?" cannot be
answered BEFORE the attempt - a directory that existed and was writable a
moment ago is not a write that completed - so it is answered after it, once,
by the only thing holding the evidence: `_save()` returns whether the file now
holds what memory holds. No volume is one False, an OSError is another.
`_remember()` keeps only what that one answer says was kept, and the
`not self.ok` test in `_remember()` is GONE rather than joined by a second one.
FCST-04's M4 and FCST-05's HIGH were the same shape, so a third cause now needs
no third branch.

What is undone is the MERGE, not the store. The state restored is the last one
that IS on disk, so a site whose volume fills at 14:00 goes on answering the
yesterday it wrote at 13:30 - that record survives a restart, so answering it
is not a promise the store cannot keep. It stops adding to it. The snapshot is
copy.deepcopy: a one-level copy shares the day record the merge rebinds
`points` on, and measured with one, a store holding 12 persisted points
answered 24 after a failed merge.

Not fatal, deliberately: §6 and this module's opening - a broken cache must not
be a broken controller, and an add-on that will not start because /data filled
up is a worse defect than this one. `_save()` reports and returns; the shipped
resolve_tariff() path still gives a provider plan with no §11.4 degradation.
What changed is that the failure is now acted on rather than only logged.

test_price_history.py section 5d drives it on two REAL failures with different
syscalls and different errnos - a directory where the file goes (os.replace,
EISDIR) and a volume moved out from under an open store (mkstemp, ENOENT) -
because the rule is enforced on the outcome and one cause would grade a
coincidence. DOCS.md's "where there is no /data" sentence is widened to the
condition the code now holds to.

MUTATION.md gains what M4 exposed: a mutation whose red is a TRACEBACK reports
however many checks ran before it and grades none after it. M4 first printed
1 FAIL; the true figure is 3. An f-string operand read `len(read.points)`, and
`.points` on a no_data raises `Absence` by design - why()'s rule arriving in
what a check PRINTS rather than in what it asserts. `npts()` is now beside
why(), and a red count is a lower bound until the run is seen to reach the end
of its section.

6 mutations, purged tree, every one RED on named checks, no survivors:
M1 self._days = before -> dict(self._days)          +10 B  9 red
M2 _save() OSError handler returns True             -1 B   9 red
M3 _save() returns True on `not self.ok`            -1 B   3 red
M4 self._days = before -> self._days = {}           -4 B   3 red
M5 copy.deepcopy -> dict                            -9 B   3 red
M6 `if not self.ok: return` back in _remember()     +43 B  1 red

All 29 test files green. Nothing deployed; no option added; config.yaml and
app/ behaviour on a healthy /data untouched. No version number picked - that
is the open decision the Unreleased block names.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-10 08:43:17 +02:00
admin 5059e28860 Merge pull request 'FCST-05: one byte stops stopping the add-on, and the write path gets the read path's standard' (#74) from FCST-05 into release/1.0
Both defects were found by the QA pass on merged FCST-04, and both are the same shape: a corrupt price cache taking the controller with it.

HIGH: _load() caught (JSONDecodeError, OSError) and UnicodeDecodeError is neither - it is a ValueError. One non-UTF-8 byte in /data/price_history.json escaped to Controller.__init__, which is unguarded, and the add-on did not start. The except is now (ValueError, OSError) - the content family beside the bytes family - rather than the two members known today, because a third ValueError subclass nobody thought of is exactly how this arrived. The new fixture writes real bytes and asserts the old fixture's string IS valid UTF-8, so the pair cannot collapse back into one.

MEDIUM: a malformed `start` under the day the feed was writing raised out of _remember() on every refresh forever - the add-on stayed up while the plan retired after 90 min and arbitrage stayed off, with no self-heal, and `words` reported the store healthy throughout. Read and write paths now share one validator, and the merge key is the validated instant (which also fixes a lexicographic sort putting the 25-hour day's second 02:00 first).

Also: the row counts what it says it counts, and two false claims of strengthening in _audit's docstring are corrected - a differential over the realisable domain returns 0 weaker, 0 stronger, and that differential is now a committed check with FCST-03's audit frozen beside it. PROC-01: a right decision with a fabricated reason is still a defect.

_audit completeness was NOT attempted, and the reason is measured rather than asserted: the rule as stated takes the horizon checks red on the shipped fill and a healthy feed, because `published` is the whole read while the fill emits only [since, until). The rule is wrong, not merely absent; making it right needs the window in the signature. Separate card.

Evidence: 29/29 green. 12 mutations, all red, each on a fresh tree copy fingerprinted against clean. One survivor was found and closed during the work - two rungs reported into one last_error, so a -1 byte deletion on the half of the fix that tells an operator the cache self-healed survived the whole suite. TEST-16's masking pattern, inside a check written to close a false-health defect. Recorded in MUTATION.md.
2026-09-10 01:13:41 +02:00
glenn schrooyenandClaude Opus 5 375a1e29d2 FCST-05: one byte stops stopping the add-on, and the write path gets the read path's standard
checks / unit (pull_request) Successful in 7m10s
HIGH - a non-UTF-8 byte in /data/price_history.json stopped the add-on
from starting. `_load()` caught `(json.JSONDecodeError, OSError)`.
`UnicodeDecodeError` is `ValueError -> UnicodeError -> UnicodeDecodeError`:
neither of those, and `issubclass(UnicodeDecodeError, OSError)` is False. So
it escaped `_load()` -> `_open()` -> `PriceHistory.__init__`, which
`Controller.__init__` does not guard - breaking `_load()`'s own stated
contract, "a corrupt cache must not stop the controller". The handler now
names the ValueError FAMILY (the file's content) beside OSError (its bytes),
because a third subclass nobody has thought of is how this arrived the first
time. The suite could not see it: its "unreadable file" fixture writes
"{ this is not json", which is valid UTF-8 and fails one step later, in
`json`. A fixture holding a genuinely undecodable byte is now beside it,
with the difference between the two asserted as a precondition.

MEDIUM - a malformed `start` under the day the feed is writing to killed the
tariff plan permanently. `_remember()` merged with `{q["start"]: q ...}` and
sorted on `q["start"]`, trusting a value read off /data to be a hashable,
sortable string. An int, null, a dict or a list raised TypeError out of
`carry()` -> `resolve_tariff()` -> `run_tariff_refresh`, which catches
Exception - so the add-on stayed UP while every refresh raised forever, the
plan retiring after TARIFF_MAX_AGE_S with arbitrage and the cheap window off
for good, and no self-heal because `_remember()` raised before `_save()`.
The read path was properly defensive throughout, which is why `yesterday()`
survived and the existing fixture passed - it files its malformed rows under
a day the feed is NOT writing to. Both paths now share one validator,
`_point_of()`. A row that is not a point is dropped, reported, and written
back out clean.

...and the status row said the store was healthy while the plan was dead:
`words` read '2026-01-15 (1 h)'. `_load()` now counts the rows inside a kept
record as well as the record itself, so the store stops reporting plain
health from the moment it is opened; the note names what was FOUND at open
and what was DROPPED by the refresh, and clears once the file is rewritten.

...and the merge key is the validated INSTANT rather than its spelling,
which is what rule 4 already said. The lexicographic sort beside the string
key also put the second 02:00 of the 25-hour October day (+01:00) before the
first (+02:00); stored order only, since `_parse_points()` re-sorts.

LOW - the status row counted POINTS and printed them under the HOUR's name:
'2026-01-13 (96 h)' for a 24-hour day on the quarter-hourly feed this add-on
already supports. It is now '96 points over 24 h' - two facts, each under
its own unit.

LOW - two stated reasons were false. `_audit`'s "WHY IT IS STRICTER" claimed
the rework refused the same published point emitted twice and a carried
point sharing a published hour, "which the old count never looked at". A
differential over the whole domain the audit can express finds ZERO
divergence in both directions: the old `seen` count already refused the
first (one point twice is one hour twice) and the old IDENTITY rule already
refused the second. The reworked audit is EQUAL to FCST-03's, not stronger;
the only behavioural change is the intended relaxation, and it stays. The
differential is committed as a check rather than left as prose.

...and the audit's real limit is named: it does not check COMPLETENESS. Fed
the decimation defect FCST-04 was raised for it ACCEPTS, and it accepts an
empty output. So `_audit` is not what stops that recurring - the count and
identity checks are, and the text now points at them. Not made complete:
`published` is the whole read while the fill emits only [since, until), so
the rule as stated is wrong, and building it takes the 1/12/48 h horizon
checks red on a healthy feed. Measured, not assumed.

12 mutations, purged tree, every one RED, no survivors. All 29 test files
green. Nothing deployed; no option added; config.yaml untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-10 01:04:30 +02:00
admin c2787da539 Merge pull request 'FCST-04: the carry-forward fill reaches the plan, and the day it carries survives a restart' (#73) from FCST-04 into release/1.0
Review found 2 HIGH, 4 MEDIUM, 12 LOW - all driven to a concrete failure, all fixed on the branch.

H1 was a regression this PR introduced: fill_from_yesterday keyed the published set as {_utc_hour(p): p}, so a sub-hourly feed lost three points in four. eneco_eddy is the Belgian imbalance product and settles per quarter. Measured on 96 quarter-hourly firm points: 96 -> 15 points, spread 0.143 -> 0.014 EUR/kWh. The fix passes every published point through by identity and carries only whole hours with no coverage at all; _audit's one-per-hour rule became two rules that subsume it, so it caught MORE after the change than before - a repeated published point and a carried point sharing an hour with a real print both now raise, and neither was checked before.

H2 falsified the branch's own 'a broken cache is not a broken controller': a day record that parsed but was not a dict raised out of words(), which is unguarded in Controller.__init__, so the add-on did not start and could not self-heal.

Two of the branch's own checks were decoration and are recorded as such in DECISIONS rather than quietly repaired: the prune mutation survived because the fixture seeded one day and never reached the code it graded, and the AST ordering check compared ast.walk BFS depth rather than source order - shipped and reversed scored identically. Both are now driven by behaviour.

Evidence: 29/29 test files green. 20 mutations driven across the rework, fresh tree copy and __pycache__ purge before each per MUTATION.md/TEST-15. No survivors.
2026-09-10 00:03:08 +02:00
glenn schrooyenandClaude Opus 5 b59f5ce31b FCST-04: the carry-forward stops decimating a quarter-hourly feed, and a broken cache stops stopping the add-on
checks / unit (pull_request) Successful in 7m22s
Rework of the FCST-04 branch against review. Every finding below was proven by
a runnable script before it was touched and re-run after; every check added or
repaired was driven red by a mutation, `__pycache__` purged first (MUTATION.md,
TEST-15). 29/29 test files green.

⚠️ H1 - A SUB-HOURLY FEED WAS DECIMATED TO ONE POINT AN HOUR, AND THE SPREAD
WENT WITH IT. `fill_from_yesterday()` keyed the published set as
`{_utc_hour(p.start): p for p in r.points}` - LAST POINT OF THE HOUR WINS, the
other three dropped. `eneco_eddy` is the Belgian imbalance product and settles
per QUARTER, and `_validate_raw()` accepts any strictly-increasing spacing, so
this is a shape the product supports today. Measured on a 96-point quarter-hourly
feed - ALL FIRM, FULLY PUBLISHED, nothing to carry: the plan reached the
optimiser as 15 points, the arbitrage spread fell from 0.143 to 0.014 EUR/kWh,
and the cheapest "hour" became a `:45` point whose `at()` window runs into the
next one. Not the windowing effect this card is about; a feed losing 84 % of its
own prints, with no flag anywhere reporting it. The fill now decides only WHICH
WHOLE HOURS NEED CARRYING: every published point inside the window passes
through BY IDENTITY, and a carried point is added only for a whole hour with no
published coverage at all. Same feed, same fixture: 60 of 60 firm points kept,
spread 0.134 over the hours still ahead.

⚠️ AND `_audit()` GOT STRICTER, NOT LOOSER - that was the condition on doing it
at all. The old post-condition refused any repeated `_utc_hour`, a count that
cannot express four points in one hour. Two rules replace it and SUBSUME it: a
published point must BE one of that hour's published points by identity and may
be emitted only once; a carried point may stand only in an hour absent from the
published set, must be flagged True, and may not repeat. So the duplicated
carried hour it existed to catch still raises, the same published point emitted
twice now raises, and a carried point sharing an hour with a real print now
raises too - which the count never looked at. Three mutations, three reds.

⚠️ H2 - A DAY RECORD THAT WAS NOT A RECORD STOPPED THE ADD-ON FROM STARTING.
`_load()` validated `version` and that `days` is a dict, never that each VALUE
is one. `{"version": 1, "days": {"2026-01-14": "oops"}}` loaded as ok=True,
last_error=None, and then `rec.get(...)` raised AttributeError out of `words`,
`yesterday()` and `resolve_tariff()` alike. `PriceHistory(...)` and `.words` are
both unguarded in `Controller.__init__`, so the controller did not come up - and
it never self-healed, because `_remember()` raised on the same record before it
could reach `_save()`. That falsifies the CHANGELOG's "A BROKEN CACHE IS NOT A
BROKEN CONTROLLER" in the one direction that matters. The asymmetry that
produced it is visible in the file: `_parse_points()` is carefully defensive
about every field of a POINT and `_load()` was not defensive about the record
holding them. Unusable records are now dropped and REPORTED (`words` says how
many and why), and `words` is total by construction as a second line.

M3 - `yesterday()` WAS OFF BY ONE DAY FOR 00:00-00:59 AFTER SPRING-FORWARD.
`brussels()` renders at the FIXED offset for that instant, so `brussels(stamp) -
timedelta(days=1)` subtracts 24 ABSOLUTE hours - and the Brussels day before the
March transition is 23 hours long. At 2026-03-30T00:30+02:00 it returned
2026-03-28's prices as "yesterday", or `no_data` naming a date that is not
yesterday. The arithmetic is now on the local DATE, which has no offset to lose,
and both transitions have a check - the module's headline claim had none.

M4 - WITH NO /data THE STORE STILL FILLED, IN MEMORY. `_save()` short-circuited
on `not ok` while `_remember()` populated `self._days` anyway: `words` said
"unavailable" and `yesterday()` answered A SERIES that is gone on the next
restart. The module docstring rejects exactly that ("a yesterday that silently
forgets") as its reason for not calling makedirs, so the code now matches the
docstring rather than contradicting it.

M5 - THE PRUNE MUTATION SURVIVED BECAUSE THE STIMULUS WAS DEGENERATE. §8's
clock-jump check seeded ONE day, so `if len(self._days) <= KEEP_DAYS: return`
fired and the body it is named after never ran: pruning by arithmetic against
the clock - which empties the store on exactly the morning the fill is for -
passed the whole 29-file suite. The fixture is now written to a file and
reopened (`_load()` does not prune, which is how a store legitimately holds more
than the cap) with KEEP_DAYS + 2 days, and the mutant is red.

M6 - THE AST ORDERING CHECK COULD NOT CATCH A REVERSAL, AND TWO DOCUMENTS CITED
IT AS A GUARANTEE. `_calls.index("_remember") < _calls.index("yesterday")` grades
`ast.walk` BFS DEPTH: `self._remember(...)` is a statement and
`self.yesterday(...)` is an ARGUMENT of the fill, one level deeper, so
`_remember` came out first whichever statement was written first - shipped and
reversed both scored True. It is now line numbers AND, the half that matters,
the behaviour: a feed carrying yesterday alongside today into an EMPTY store
must fill the whole horizon; reversed, the lookup finds nothing and the plan is
only the published hours. Reversing the two statements is red on both.

L14/L9 - KEEP_DAYS = 3 IS THE STEADY STATE EXACTLY, so a newest-N cap has no
slack: a feed publishing to D+2, or ONE stray point in the wrong year - which
sorts LAST and can never age out - pushes yesterday out for good. `_prune()` now
PINS the two days a lookup can actually ask for on top of the cap. Pinning is
additive, so it can only RETAIN and the WITNESS-03 property (nothing is deleted
by clock arithmetic) is preserved by construction; `stamp` stops being a dead
parameter; the store is bounded at KEEP_DAYS + 2.

L8 - the horizon end was not floored while `since` was, so a 48 h request made
at any instant off the hour returned 49 points. §1's count was green only
because its NOW was exactly :00:00, a shape production never has; the fixture
now sits at :17:43. L10 - a failed `_save()` leaked its temp file; at 48 writes
a day a persistent write failure leaked 48 inodes a day onto the volume that was
already the problem. L15 - `{"version": 1.0}` passed, since `1.0 == 1`. L16 -
the `_remember` tree walk scanned `app/*.py` only and matched a bare string; it
now parses `app/`, `tools/`, `scenarios/` and `estop/` and looks for the NAME,
so a comment does not fail it and a call in `tools/` cannot pass unseen.

CLAIM-ACCURACY DEFECTS, WHICH THIS REPO TREATS AS REAL:

* L7. DOCS said "*Tariff* on the status page reads `provisional` through the
  morning". FALSE - `tariff_words` is assigned only from the HA price-entity
  branch and from `static_status()`, both on the 30 s tick, and the plan's own
  sentence reaches `_LOG.info`. The doc now says what an operator can see (the
  Price history row, the Arbitrage row's own refusal sentence, the refresh log
  line) and a check pins the PAIRING - wiring the plan into the row is fine,
  correcting the doc is fine, doing one without the other is red.
* L11. "fills EVERY unpublished hour" is conditional: a partial yesterday leaves
  the local hours it never held out of the series (§11.4 for those hours, and
  `at()` raises rather than answering). Correct behaviour, overclaimed wording,
  and untested - all three fixed.
* L12. DECISIONS §1 leaned on `TariffPlan.cheap_now()`, which has NO PRODUCTION
  CALLER on the provider plan (`self.cheap` comes from `static_status()`, which
  builds its own). The conclusion holds on `arbitrage()`'s firm-pair refusal
  alone, which does reach control; the evidence is now stated at the reach it
  has.
* L13. §1's "passed through by identity" check compared `eur_per_kwh` between
  two SEPARATE `resolve_tariff()` calls, where `is` could never have held. The
  value comparison is kept and now says what it grades; identity is graded on a
  read the file constructs and still holds.
* L17. `PriceHistory.words` and the new status row had no check at all - and
  `words` is one of the two things H2 crashed. Both now have one.
* L18. main.py's inline note carried the "widens what the optimiser can SEE,
  never what it may act on" guarantee without the windowing caveat that the
  CHANGELOG, DOCS and `resolve_tariff()`'s docstring all carry. It is the one
  place a maintainer reads it unqualified.

EVIDENCE. All eight reviewer proofs re-run: p1/p1b return 29 March, p4 keeps
60 of 60 firm points at spread 0.134, p5 starts and reports, p6 D leaves no
temp, p6 E refuses 1.0, p6 F records nothing, p7 returns 48 for 48. p2's script
still scores True/True because it re-implements the OLD algorithm - the
reversal is now driven as a mutation instead, and is red. p3's script always
showed the shipped code correct; what was wrong was the fixture, and its mutant
is now red. Twenty mutations driven, no survivors, tree fingerprinted and
`__pycache__` purged before each: decimation (2 suites red), audit x3, unfloored
`until` (2), unvalidated day records (7 checks), instant-subtraction yesterday,
in-memory fill on a volume-less store, clock-arithmetic prune, cap without the
pin, leaked temp, loose schema type, empty `words` (5), reversed `carry()` (2),
a `_remember` leak in tools/ spelled two ways, a neighbour-filling fallback, a
copy instead of identity (2), the DOCS claim in both directions, and the caveat
removed from main.py. Baseline and the unmutated tree both green.

No option added, config.yaml untouched, and the CHANGELOG entry still picks no
version: PKG-07's rule, and only the deployer knows whether 0.9.0's image has
been pulled.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-09 23:54:23 +02:00
glenn schrooyenandClaude Opus 5 0618194d9c FCST-04: the carry-forward fill reaches the plan, and the day it carries survives a restart
checks / unit (pull_request) Successful in 7m11s
FCST-03 shipped `fill_from_yesterday()` complete and checked, and shipped the
consumers written for it - `TariffPlan.firm_points()`, `cheap_now()`'s refusal
of a provisional hour, and `arbitrage()`'s refusal 3, which drops provisional
hours from the trade set rather than refusing the whole series. It shipped NO
CALLER. Nothing in `app/` named the fill, so no series ever contained a carried
point and every one of those consumers was exercising its all-firm branch.

⚠️ THE DEFECT THAT PRODUCED WAS NOT AN ERROR ANYWHERE. `load_prices()` returns
what the feed published and nothing widened it to the requested horizon, so on
a dynamic site between midnight and the 14-16 h CET publication the plan
covered the rest of today and nothing more - measured on the fixture here, 24 h
against a 48 h request. `arbitrage()` then took `max - min` over that truncated
window and reported a spread. The number was real, the window was half the one
configured, and nothing said so: tomorrow evening's peak, the hour a morning
charge would be sold into, was not in the series to be compared against
anything. A missing hour is not a wrong price - it is an hour the optimiser
never learns exists.

WHAT THIS ADDS. `app/price_history.py` keeps the PUBLISHED prices of the last
KEEP_DAYS Brussels days in /data/price_history.json; `resolve_tariff()` gains a
required keyword-only `history` and fills every unpublished hour of the horizon
from yesterday's same-wall-clock hour, flagged. Persistence rather than a
variable on the refresh task, because the window the fill exists for is
midnight -> publication and an add-on restarted at 08:00 would otherwise spend
all of it on the truncated series. Not in state.json: the refresh runs every
1800 s and that file holds the maintenance history and SAFETY-02's latch.

⚠️ ONE CHANGE HERE MOVES ENERGY, AND IT IS NOT THE ONE THE CARD IS NAMED FOR.
Carried hours are provisional, so §8.3 keeps them out of the trade set: the
fill widens what the optimiser can SEE, never what it may DO, and
test_price_history.py checks that so it cannot be overclaimed later. What does
change a trade is that the filled series is windowed to [since, until), so
hours that have ALREADY ELAPSED are no longer in the spread. Before, a cheap
night that had gone sat in the series as the day's cheapest hour - and a cycle
cannot buy in an hour that is past, so the trade was quietly never taken.
Section 2 grades this by finding the cheapest firm hour with and without the
fill and showing it MOVES, from 02:00 UTC to 08:00 UTC on the fixture.

⚠️ A DEAD FEED IS STILL §11.4, NOT YESTERDAY. The fill runs only on a `series`
read. The two are indistinguishable to the optimiser - both 100 % provisional,
so arbitrage refuses on the firm-pair rung and `cheap_now()` is False - so
preferring yesterday's feed over the `static_profile` an operator configured
FOR that case would change a documented degradation for a difference only the
status page can see. That, and two more, in
DECISIONS/FCST-04-the-day-that-is-carried.md.

⚠️ THE ANTI-DECAY RULE IS AN ORDERING, NOT A VALIDATION, and saying so is more
useful than a guarantee that would not hold. Feeding the filled series back in
is how this rots - day 2 carries day 1's guesses, day 3 carries day 2's carry
of them, behind a `provisional` flag that stays TRUE while the NUMBER decays.
No value-level check can catch it: a carried point and `eneco_eddy`'s
published-provisional quarter are the same three fields with the same flag. So
`carry()` is the only public entry, it records BEFORE it fills, the filled
series exists only as a return value the object never sees again, and
test_price_history.py walks app/ for any other module naming `_remember` and
reads `carry()`'s AST to pin the order.

EVIDENCE. 29/29 test files pass in the CI image. Eight mutations driven against
app/price_history.py and app/static_tariff.py, __pycache__ purged before each
run per MUTATION.md/TEST-15: newest-record lookup (5 red), recording the filled
series (1), replace-instead-of-merge (10), UTC-date keying (10), removing the
fill call entirely (12), a lost price field defaulting to 0.0 (2), prune by
clock arithmetic (1), a degraded reason that stops naming its cause (1). No
survivors; baseline and revert both green. Three of those first crashed the
harness instead of reporting a FAIL - "it was refused" and "it broke on the way
to being refused" must not grade the same - so `why()` and `_priced()` were
added and the run repeated.

No option added, config.yaml untouched, and the CHANGELOG entry deliberately
picks no version: PKG-07's rule, and only the deployer knows whether 0.9.0's
image has been pulled.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-09 19:37:45 +02:00
admin 587751b115 Merge pull request 'INSTR-01: grade diff()'s suppression rule at the level it is written at, one case per clause' (#72) from INSTR-01 into release/1.0 2026-09-09 01:05:36 +02:00
glenn schrooyenandClaude Opus 5 3de4453956 INSTR-01: match the finding KIND, because M12 was not holding this file to its own rule
checks / unit (pull_request) Successful in 7m7s
Review APPROVED f6f3f22 plainly. Three LOW notes, all applied - the first is a
real gap and the other two are one line each.

⚠️ M12's ASSERTION WAS KIND-BLIND. It matched on the key and ignored the
finding kind, so renaming `diff()`'s `"VANISHED STORE KEY"` to
`"GONE STORE KEY"` left M12 GREEN - review measured it. A vanished latch
quietly downgraded to a softer heading is exactly what part 3 of this same file
requires every mutation to be graded against ("the SPECIFIC finding it should
produce"), so the file was not holding itself to its own doctrine. Inherited
from SAFETY-44's original M12, not introduced here, and fixed now:

    ("VANISHED STORE KEY", "unit/shed/sign_tripped") in _m12_new

Driven: the kind rename is exit 1 and M12 is among the reds where it was green
before, and C5's removal still turns M12 red, so the strengthening did not cost
the discrimination it exists for.

C4 IS NAMED IN THE CLAUSE MAP. It has no 5e row deliberately - part 3's M9
reaches `vanished_unit is not None` end-to-end and a synthetic row would
duplicate that rather than add separability - but C5 lives in the same `if`,
and listing one guard while omitting the other reads as an oversight rather
than a boundary. Review's own recommendation was not to add the row.

AND THE PRINTED SET SAYS WHAT IT LOSES. The finding regex captures the first
TOKEN of each text, so two findings of one kind collapse into one tuple; the
line now says "first token only" so nobody reads it as an inventory. Harmless
for M12 - the difference is one-directional and the pair it asserts on is exact
- but the display was lossier than it looked. Unused `_m12_out` dropped.

28/28 files pass. `rehearse_roster.py` is untouched by this round; review
confirmed its AST is identical between 91b0d5e and f6f3f22, so every
measurement made against the shipped rule still stands.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-09 00:58:03 +02:00
glenn schrooyenandClaude Opus 5 f6f3f227c1 INSTR-01: restore M12 on the leg I generalised past, and measure the clause that had no row
checks / unit (pull_request) Successful in 7m6s
Review found two things this PR asserted about itself that were false. Both are
mine and both are worse than the defect the card was filed for.

⚠️ "NO FIXTURE CAN RESTORE IT, MEASURED BOTH WAYS" WAS NOT TRUE, AND TWO
MEASUREMENTS ARE NOT AN IMPOSSIBILITY PROOF. I deleted M12 - SAFETY-44's
working end-to-end control for the off-roster narrowing - on the argument that
the provenance clause made it unrestorable. The structural claim behind it,
that `run()` builds both snapshots from one state file so "already latched" and
"made by this transition" are incompatible, IS TRUE OF SAME-STEP AND FALSE OF
STAGED: `snapshot_over()` starts the after-leg over the file the BEFORE-leg
wrote, so a transition-made destination is available there. Both arrangements I
measured were same-step shaped. Review found the third.

RESTORED, and it discriminates with all six clauses in:

    staged leg, legacy+shed, shipped     _m12_new = VANISHED unit/shed/sign_tripped
    ...C5 narrowing removed              _m12_new = []          ⚠️ suppressed

⚠️ AND MY FIRST RESTORATION DIDN'T DISCRIMINATE EITHER, for a reason worth
recording: `data.pop(k, False)` RETURNS the value, so shed lost the KEY and
kept the LATCH - `_grade_store_migration()` saw `on_roster_latched`, never
re-armed garage, and there was no transition-made destination. The finding was
reported whether or not C5 existed and the control looked restored. `and False`
is the difference between "the key is gone" and "the LATCH is gone", and only
the second reaches the re-arm this arrangement needs.

⚠️ WHAT ACTUALLY BROKE WAS M12's exit-0 PRECONDITION, NOT M12. A legacy-shaped
fixture is never clean. That precondition is the one SAFETY-44's own commit
called "belt-and-braces, load-bearing only if the exit-0 requirement is ever
relaxed", saying what closed the gap is `_m12_new` BEING A SET DIFFERENCE - and
a stable dirty baseline is fine for a set difference. So I deleted a control on
the strength of a precondition the same commit had already called not
load-bearing. That is removing coverage rather than failing to add it, which is
the worse of the two directions.

⭐ AND THE BARE-KEY CLAUSE HAD NO ROW AT ALL. Review deleted
`k.rsplit("/", 1)[-1] == bare` and the whole 28-file suite stayed GREEN. 5e
mutated five clauses; the comprehension carries more, and this one is
load-bearing: a vanished SAFETY-02 `sign_tripped` was suppressed by a SAFETY-42
`delivery_tripped` on a roster unit - a DIFFERENT FAILSAFE's key. The rule's own
comment says "THE TEST IS NOT 'the name appears somewhere'. It is: the SAME bare
key", and nothing measured that sentence. `unit/*/delivery_tripped` is
machine-read, so it is reachable on any real two-latch store. R7 added; C2 now
flips exactly R7.

All six clause-removals now flip exactly their own row:

    C1 machine_read   -> ['R6-machine-read']    C5 off-roster  -> ['R2-narrowing']
    C2 same bare key  -> ['R7-bare-key']        C6 injectivity -> ['R5-injectivity']
    C3 same value     -> ['R4-value']           C7 provenance  -> ['R3-provenance']

THREE DANGLING CITATIONS TO A CONTROL THIS PR HAD DELETED, one of them in the
shipped tool eight lines below the paragraph condemning exactly that ("a
citation is the worst possible carrier, because it reads as having been
checked"). The tool no longer names a check; it states the property - every
clause has a row, and each removal flips its own.

A STALE SAFETY-44 BLOCK saying both fixes were "REVERTED OUT OF THIS CARD,
because THIS SUITE CANNOT MEASURE EITHER ONE" - describing a tree this PR
replaced. Rewritten as a map of where each clause is now measured. A comment
that outlives its subject is the same defect as a citation that outlives its
check.

AND A SELF-CONTRADICTION about the driver, in the file and in the last commit
message: "THE DRIVER IS WRITTEN INTO THE TREE UNDER TEST" eight lines above
"THE DRIVER GOES IN A TEMPDIR, NOT IN THE TREE". The code does the tempdir; the
first sentence stated the read-only-CI bug the same commit says it fixed, as if
it were the design. It now says the driver IMPORTS the tree and is WRITTEN to a
tempdir.

ONE HOLE REMAINS OPEN AND IS STATED AS UNMEASURED: attribution. `sorted()`
decides which vanished key claims a destination, so a deleted key can sort
first and be named as the one that migrated while the key that really moved is
reported VANISHED. Review measured that `sorted(` -> `list(` leaves the suite
green - every 5e case has at most one candidate, so no row can see ordering.
Still exit 1, so a wrong sentence rather than a hidden loss. Recorded rather
than left looking guarded.

28/28 files pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-09 00:29:00 +02:00
glenn schrooyenandClaude Opus 5 91b0d5e4dc INSTR-01: grade the suppression rule at the level it is written at, one case per clause
checks / unit (pull_request) Successful in 7m4s
SAFETY-44 produced THREE inert controls for this one rule, each inert
differently, and split the two remaining fixes out rather than ship them
unmeasured. This is that card, and the first thing it did was explain why the
end-to-end route could never have worked.

⚠️ `moved_to` IS ONE COMPREHENSION CARRYING FIVE CONDITIONS WITH FIVE DIFFERENT
REASONS TO EXIST, AND `run()` BUILDS BOTH SNAPSHOTS FROM ONE STATE FILE. So
"the destination was already latched" and "the destination was made by this
transition" are the same file answering two incompatible questions: the
arrangement that reaches one forecloses the other. That is why rebuilding a
fixture to reach clause N kept moving the others, and it is not a fixture
problem.

⭐ SO THE RULE IS GRADED WHERE IT IS WRITTEN - `diff()` on constructed
Snapshots, ONE CASE PER CLAUSE. Six rows, and on the SHIPPED tree before any
fix they already separated the two known defects:

    R1 rename (legitimate)   migrated                      correct
    R2 narrowing             vanished                      correct
    R3 provenance            MIGRATED                      ⚠️ the defect
    R4 same value            vanished                      correct
    R5 injectivity           BOTH migrated                 ⚠️ the defect
    R6 machine_read          vanished                      correct

The two one-line fixes SAFETY-44 reverted are back, and R3/R5 flip while
R1/R2/R4/R6 do not - so the legitimate rename is still suppressed and this is
not an over-correction.

⭐ AND EACH CLAUSE REMOVAL FLIPS **EXACTLY ITS OWN ROW**, which is the
acceptance criterion this card was filed with:

    C1 machine_read  removed -> ['R6-machine-read']
    C3 same value    removed -> ['R4-value']
    C5 off-roster    removed -> ['R2-narrowing']
    C6 injectivity   removed -> ['R5-injectivity']
    C7 provenance    removed -> ['R3-provenance']

A removal that flips nothing is an unmeasured clause; a removal that flips
someone else's row means the cases are not separable after all. Neither
happened.

⚠️ AND M12 IS DELETED, WHICH IS THE MOST USEFUL THING THIS CARD FOUND. M12 was
SAFETY-44's end-to-end control for the narrowing and it was a REAL control -
red when the narrowing was removed. Adding the provenance clause disarmed it,
and NO FIXTURE CAN RESTORE IT, measured both ways: with the sibling latched in
its own right, C7 rejects the destination before C5 is consulted, so the
mutation is reported whether or not C5 exists; making the destination
transition-made means a flat latch, which makes the shipped run dirty and
breaks M12's own precondition. The second half of the card's acceptance
criterion - "M12 still turns red with the new clauses in" - is therefore
UNSATISFIABLE, and the honest answer is to remove the control rather than keep
one that has become inert. R2 measures C5 strictly better. Part 5 keeps the
end-to-end coverage; discrimination lives in 5e.

`state-both-latched.json` goes with it. Nothing referenced it once M12 was
gone, and the first draft of that paragraph said it was "kept, because 5e's
R2/R3 are derived from the arrangements it represents" - a nice sentence and
not a reference. An unused fixture a comment calls load-bearing is an inert
check one directory over.

TWO THINGS THE DRIVER GETS RIGHT ON PURPOSE:
  - it is written to a TEMPDIR, not beside the tool. `checks / unit` mounts the
    checkout READ-ONLY, so writing into the tree works on a dev box and is
    `OSError: Read-only file system` in CI - green for whoever wrote it and red
    on every PR. Caught by running it in the check image before committing.
  - the TABLE stays in the test file and the driver goes into the tree under
    test, so a mutation to the tool cannot also move the answer it is graded
    against.

28/28 files pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-08 23:51:35 +02:00
admin 806390d8b1 Merge pull request 'SAFETY-44: the latch migration takes the roster, and the guard stops being defeated by the key the defect wrote' (#71) from SAFETY-44 into release/1.0 2026-09-08 19:16:32 +02:00
41 changed files with 16362 additions and 587 deletions
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,190 @@
# FCST-04 — what "yesterday" is, and what the fill is allowed to reach
Four decisions, recorded because each had a defensible alternative that a later
reader would otherwise re-open from scratch.
---
## 1. The fill runs on a `series` read only. A dead feed still goes to §11.4.
`fill_from_yesterday()` accepts a `no_data` read and will happily build a
100 %-carried series from yesterday alone. `resolve_tariff()` does not let it:
a feed that published nothing still falls to `_degrade()` and §11.4's own row —
*"No price data → fall back to the static profile, mark the plan provisional,
no arbitrage."*
**Why not prefer yesterday's real prices to a coarse day/night profile?**
Because it buys nothing the optimiser can use, and it silently replaces a
fallback the operator configured.
Measured, not assumed — and stated at the reach it actually has:
- **This one reaches control.** A fully-carried series is 100 % provisional, so
`TariffPlan.arbitrage()` finds fewer than two firm hours and refuses on
§8.3's firm-pair rung. That is the verdict the optimiser reads: `arbitrage.py`
calls `plan.arbitrage(...)` on `fresh_tariff_plan()`, and it is the only path
by which a provider plan moves energy. The §11.4 static fallback refuses on
the same rung, for the same reason.
- **This one does not, and saying so is the point.** `TariffPlan.cheap_now()`
also returns False on a fully-carried series, because `firm_points()` is
empty — but it has **no production caller on the provider plan**.
`self.cheap` comes from `static_status()`, which builds its *own* static plan
and calls `cheap_now()` on that. So it is a property of the type, checked in
`test_static_tariff.py`, and it is not evidence about the shipped control
path. The conclusion below holds on the first bullet alone.
The two paths therefore differ only in which numbers the status page shows.
So the choice is between two identical control behaviours, and the tie-breaker
is that `static_profile` is a documented option an operator fills in **for this
case**. Overriding it for a cosmetic difference is a behaviour change nobody
asked for.
⚠️ **The consequence, stated so it is not discovered later as a bug:**
`fill_from_yesterday()`'s `no_data` branch is not reached from production.
It stays checked in `test_price_forecast.py`. Turning it on is a product
decision — it would want a rule about how old a carried profile may be before it
stops being offered at all — and not a wiring one.
---
## 2. "Yesterday" is a date, not the newest record.
`newest()` is the obvious accessor and it is the one that decays. An add-on down
from Friday to Monday would answer Friday's prices to a fill whose only
vocabulary is `provisional: true` — a flag that says "this is a guess", never
"this guess is three days old". Nothing downstream can tell the difference and
nothing reports it.
So `yesterday(stamp)` computes the Brussels date before `stamp`'s Brussels date
and looks up **that** record. Anything else is `no_data` with a reason naming
both the date wanted and the dates held.
The cost is real and accepted: after an outage of more than a day, the first
morning back has no fill. That is the honest state — there is no yesterday —
and the horizon is short for one morning rather than wrong for several.
---
## 3. What is recorded is the provider's read, and the mechanism is ordering.
Feeding the filled series back into the store is how this rots: day 2 carries
day 1's guesses, day 3 carries day 2's carry of them, and after a week of a dead
feed the "yesterday" being carried is a fossil of the last real publication.
Every point still carries `provisional=True`, so the **flag never lies** — what
decays is the number, behind a flag that is telling the truth.
⚠️ **No value-level check can catch that**, and claiming one would be worse than
claiming none. A carried point and a published-provisional point (a real-time
product's current interval, §9.2) are the same three fields carrying the same
flag. The
difference is provenance, and provenance is not in the value.
The defence is therefore an **ordering**, not a validation:
- `carry()` is the only public entry point. It records its argument, then
returns the fill.
- The filled series exists only as that return value. The object never sees it
again.
- `_remember()` is private and called from exactly one place.
- `test_price_history.py` parses every shipped module under `app/`, `tools/`,
`scenarios/` and `estop/` and fails if any of them **uses the name**
`_remember` — an attribute, a bare name, a `def`, or the string constant a
`getattr` would need. By name rather than by substring, so a comment that
merely mentions the rule does not fail it and a call in `tools/` cannot pass
unseen.
- and it pins the order two ways: the **line numbers** of the two calls in
`carry()`, and — the one that matters — the **behaviour**. A feed carrying
yesterday alongside today, into an empty store, must fill the whole horizon;
reversed, the lookup finds nothing and the plan is only the hours the feed
published. Driven, so a reversal is red.
⚠️ **The first version of that ordering check graded nothing, and this file
said it did.** It compared positions in `ast.walk`, which is breadth-first:
`self._remember(...)` is a statement and `self.yesterday(...)` is an *argument*
of `fill_from_yesterday(...)`, one level deeper, so `_remember` came out first
whichever statement was written first. Measured — shipped and reversed both
scored the same. A decorative check that a document cites as a guarantee is
worse than no check, which is why the correction is recorded here rather than
quietly made.
That ordering has a second benefit worth keeping: a feed that returns yesterday
alongside today refreshes yesterday's record from live data *before* it is
consumed as the profile.
---
## What this ticket does not claim
It does not make a trade possible that yesterday's prices justify. Carried hours
are dropped from the trade set by §8.3, and that is not a limitation to be fixed
later — it is the rule.
The one change that **does** move energy came from the windowing, not the
carrying: the filled series covers `[since, until)`, so hours that have already
elapsed are no longer in the spread. A cheap night that has gone used to be the
series' cheapest hour, and a cycle cannot buy in an hour that is past — so the
trade was silently never taken. `test_price_history.py` section 2 grades that by
finding the cheapest firm hour with and without the fill and showing it moves.
⚠️ **And section 1's identity check now grades identity.** It used to compare
`eur_per_kwh` between `_before` and `_after`, which come from two *separate*
`resolve_tariff()` calls — each parsing the feed body into its own objects, so
`is` could never have held between them however the fill were written. FCST-03's
whole anti-copy argument is `filled.points[i] is fetched.points[j]`, and floats
cannot grade it. The value comparison is kept and now says what it is (the fill
moved the window, it did not touch a price); identity is graded on a read the
file constructs and still holds a reference to.
---
## 4. A sub-hourly feed passes through whole; the fill only decides which hours are missing.
⚠️ **This section originally said `eneco_eddy` is "the Belgian imbalance
product". That was false and [[TARIFF-07]] withdrew the plugin: Eneco's dynamic
product is "Zon & Wind Dynamisch" and is indexed to Belpex/EPEX **day-ahead**,
not to imbalance, and `api.eneco.be/eddy/prices` never existed. The mechanism
below is unaffected, because the real quarter-hourly feed is the one still
shipped.**
Belgian **day-ahead** settles per **quarter** — the 15-minute MTU took effect
on 2025-10-01, so `generic_dayahead` reads 96 points a day rather than 24 — and
`price.py`'s `_validate_raw()` accepts any strictly-increasing spacing, so a
published hour may legitimately hold four points.
FCST-04's first caller keyed the published set as `{_utc_hour(p.start): p for p
in r.points}`. It reads perfectly ordinary and it keeps the **last** point of
each hour. Measured on a 96-point quarter-hourly feed — all firm, fully
published, nothing to carry: the plan reached the optimiser as **15 points**,
the arbitrage spread fell from **0.143 to 0.014 EUR/kWh**, and the cheapest
"hour" became a `:45` point whose `at()` window runs into the next hour. That is
not the windowing effect above; it is a feed losing 84 % of its own prints, and
no `provisional` flag would ever have reported it.
**The alternative was rejected:** refusing sub-hourly feeds at `_validate_raw()`
would have made the count correct by breaking the one supported product that
legitimately publishes per quarter.
**What was done instead** keeps FCST-03's existing contract rather than weakening
it: *every* published point inside the window passes through **by identity**,
and a carried point is added only for a whole hour with **no** published
coverage at all. A carried hour is still a single hourly point whatever the
feed's own spacing is, because yesterday's profile is only known per wall-clock
hour — and carried hours are provisional, so they are out of the trade set
anyway.
⚠️ **`_audit()` got stricter, not looser, and that was the condition on doing
this at all.** The old post-condition refused any repeated `_utc_hour` — a count
that cannot express four points in one hour. It is replaced by two rules that
**subsume** it:
- a **published** point must *be* one of that hour's published points, by
identity, and may be emitted only once;
- a **carried** point may stand only in an hour that does not appear in the
published set at all, must be flagged `True`, and may not repeat.
So a duplicated carried hour — the DST mispricing the count existed to catch —
still raises; so does the same published point emitted twice; and a carried
point sharing an hour with a real print now raises too, which the count never
looked at. Each of those three is driven by its own mutation in
`test_price_forecast.py`.
@@ -0,0 +1,125 @@
# FCST-06 — "cannot persist" is one question, and it is asked about the outcome
Two decisions. The first is the one the ticket put up for judgement; the second
is the one that decides what a store answers *after* it has failed to write.
---
## 1. One derived predicate, not two places testing two symptoms — yes
`price_history.py`'s module docstring states, unconditionally:
> ⚠️ AND THAT RULE IS CARRIED OUT IN `_remember()`, NOT ONLY IN `_save()`. …
> A store that cannot persist now records nothing, and both of its answers say
> the same thing.
It was enforced against `not self.ok`, and `ok` is **about the directory**: it
says `/data` was there when the store opened. A read-only remount and a full
volume both leave it `True`. So the second way to be a store that cannot
persist — `ok=True`, the directory present, `_save()` failing — was outside the
one condition the rule named, and in that state `words` printed two full days
beside the write error while `yesterday()` answered 24 real points, from memory,
until a restart dropped them without a word.
**The ticket's question was whether "cannot persist" should be one derived
predicate rather than two places each testing a different symptom. It should,
and the reason is not tidiness — it is that the enumeration cannot be
completed.** FCST-04's M4 enforced the rule on the symptom that was in front of
it. FCST-05's HIGH was the same shape — an `except` clause naming the one
content-failure member somebody had met (`json.JSONDecodeError`) instead of the
`ValueError` family — and its fix was to name the family. This is the third instance in
one module, and it is the last place in this file where the rule can be spelt
twice. A fourth cause — an `OSError` subclass, a filesystem that accepts the
`mkstemp` and refuses the `os.replace`, a container whose volume is unmounted
mid-refresh — needs no fourth branch now, because none of them is asked about.
### Where the one question can be asked, which is the whole design
**"Can this store persist?" cannot be answered before the attempt.** A
directory that exists and was writable a moment ago is not a write that
completed; there is no pre-check that is not itself a guess, and adding one
(`os.access`, a probe write) would be a *third* symptom test — precisely what
the ticket rules out. So the predicate is an **outcome**:
_save() -> bool # True only if the file now holds what memory holds
Both ways of not persisting are the same answer from the same call: no durable
volume returns `False` without touching the disk, an `OSError` out of the write
returns `False` after recording why. `_remember()` reads that one answer and
keeps only what it says was kept. **The `not self.ok` test in `_remember()` is
gone** — not moved, gone — because there is nothing left for a second test to
add, and a check in `test_price_history.py` 5d asserts its absence from the
source.
⚠️ **That last check is deliberately a source check, and habit 4 is why it is
allowed to be.** The claim "there are not two places testing two symptoms" is a
claim about the *shape of the code*, not about a behaviour — the behaviour is
identical whether the rule is enforced once or twice. Every other check in 5d
is driven.
### What was NOT done, and why
- **Not a `can_persist()` helper.** It would have to answer before the attempt,
which is the thing that cannot be done honestly, and it would be a second
place to keep in step with the first.
- **Not a probe write at `_open()`.** It would make `ok` mean something new, it
would write to `/data` on every start, and a volume that is writable at 08:00
and full at 14:00 would still defeat it. A predicate that can go stale is a
symptom test wearing a predicate's name.
- **Not making a failed write fatal.** §6, and this module's own opening: a
broken cache must not be a broken controller. An add-on that will not start
because `/data` filled up is a worse defect than the one being fixed. The
store degrades; it does not raise. `_save()` reports and returns.
---
## 2. The MERGE is undone, not the store
When `_save()` answers no, `_remember()` restores the state the read started
from — **not** an empty one.
The alternative considered was to clear `self._days` outright, on the argument
that a store which cannot write is a store that should stop answering. It is
wrong, and measurably so: the state being restored is the last one that **is on
disk**, so answering it is not a promise the store cannot keep. A site whose
volume fills at 14:00 goes on answering the yesterday it wrote at 13:30 —
through the restart, because that record is in the file. What it stops doing is
*adding* to it.
Clearing would throw away a good yesterday that survives a restart in order to
punish a write that failed, and it would make the morning fill worse on exactly
the day the operator is already dealing with a full disk.
That gives the file the invariant its rule was always describing, stated once
in the class docstring:
> `self._days` **never holds a point the file does not.**
⚠️ Not *"mirrors the file"* — `_usable_days()` drops records the file still
holds, and that asymmetry is deliberate. The only direction that can **invent a
yesterday** is the one named.
⚠️ **And the snapshot is `copy.deepcopy`, not `dict(self._days)`.** A one-level
copy shares the day *records*, and the merge rebinds `points` on the ones it
touches — so the "restored" store would answer the merge that never reached the
file. Measured: with a shallow copy, a store holding 12 persisted points
answered **24** after a failed merge. It would also be correct only for as long
as nobody makes the merge mutate a `points` list or a row in place, which is the
same "correct for the conditions someone thought of" this ticket is about. The
store holds at most `KEEP_DAYS + 2` records and writes 48 times a day; a copy
that is right by construction is worth more than the allocations.
---
## What this does not change
- **No shipped path reaches the defect.** `/data` is the Supervisor's durable
volume and this module is its only writer, so on a real install `_save()`
succeeds and nothing above is on the hot path. The fix is that a guarantee the
module states about itself now holds in the condition it did not.
- **The failure was already visible and still is** — `_LOG.error("could not
persist price history: %s")` and `last_error`, reported by both `words` and
`yesterday()`'s reason. What changed is that it is now *acted on*.
- **Nothing about the prices.** The series served is the feed's own, every
carried point is still flagged provisional, and a broken cache is still a
short horizon rather than a §11.4 degradation.
@@ -0,0 +1,230 @@
# TARIFF-07 — three plugins that never fetched a price, and the one mechanism they leave behind
**Status: DECIDED 2026-09-13 — withdraw `engie_dynamic`,
`totalenergies_dynamic` and `eneco_eddy`. Keep `PriceProduct.REALTIME` and the
widening machinery. Refuse a retired id by name rather than aliasing it.**
Filed with TARIFF-07. Follows [[TARIFF-08]], which found the same class of
defect one layer down — a parser graded against its own guess — and left the
question of these three open.
---
## 1. The endpoints, verified rather than taken on report
The card said the three endpoints were dead. They were re-checked by hand on
**2026-09-13** before anything was deleted, because "the endpoint is dead" is
exactly the kind of claim this repository keeps finding to be second-hand:
| id | endpoint | observed |
|---|---|---|
| `engie_dynamic` | `https://api.engie.be/dynamic/be` | **HTTP 404**, body `{"message":"no Route matched with those values","request_id":"..."}` — a Kong gateway with no route configured |
| `totalenergies_dynamic` | `https://api.totalenergies.be/prices/dynamic` | **TLS failure**. The handshake completes and the server presents `CN=pge-apigee-dev.direct-energie.com`; `api.totalenergies.be` is not in the SAN list, so the connection is closed before any request is sent |
| `eneco_eddy` | `https://api.eneco.be/eddy/prices` | **HTTP 404**, `content-length: 1`, `server: Microsoft-IIS/10.0` — an empty body from a live host that does not serve this path |
All three failed. Had any one of them answered, this card would have stopped
and asked, because an endpoint that answers changes the decision from
"withdraw" to "fix the parser".
## 2. Why this is not "unfinished", and why no replacement can be written
An unfinished plugin has a right endpoint and a wrong parser. These have
neither, and the reason is structural:
* **ENGIE's dynamic price is `(B × eSpot_15) − A`, and `B` and `A` are
PER-CONTRACT.** They live behind the customer's klantenzone login. No
unauthenticated public URL can return *a given customer's* price, so the
endpoint this plugin wanted cannot exist for anybody to build.
* **There is no regulator index to read the coefficients from.** CREG's
tariff-card pages 404; VREG was unreachable. Every coefficient found in
research came from a supplier's own PDF, which is a document, not a feed.
* **"Eddy" is not an Eneco product at all.** Eneco's dynamic product is
**"Zon & Wind Dynamisch"**, and it is indexed to **Belpex/EPEX day-ahead** —
not to imbalance. The plugin was wrong about the product, the product type
and the URL simultaneously.
So the three were not withdrawn for being broken. They were withdrawn for
describing tariffs that no public endpoint can serve.
## 3. What a site configured on one of these has actually been running
⚠️ **This is the half that decides what the migration is.** §11.4 turns a
failed fetch into a silent fall back: static profile, plan provisional, no
arbitrage. That is the *correct* degradation, and it says nothing at all about
the cause. Every fetch these three ever made raised `SourceUnavailable`.
Therefore a site configured on one of them has been running on
`static_profile` — **or on nothing at all, where no profile was set** — since
the day the id was entered, and **nothing on its status page has ever differed
from a working provider that simply never found a cheap hour**.
The migration is therefore **not** "keep their prices working". There were no
prices. It is *"tell them their prices never worked"*, which is why the answer
is a refusal and not a fallback.
### Why not a silent alias to `generic_dayahead`
It is the friendlier-looking option and it is wrong. These published **retail**
prices; `generic_dayahead` reads **wholesale**. Aliasing needs the site's own
`B` and `A`, which §2 has just established nobody has. An alias would have to
**invent the numbers a battery is priced against** — the same defect as a
parser graded against its own guess, one layer further from anything that
reports. The scalar `price_markup_eur_kwh` cannot stand in for `B` either: it
is an addend, not a multiplier, and it is `float(0,1)` so it cannot even hold a
negative `A`.
## 4. ⚠️ The `config.yaml` schema is deliberately WIDER than the registry
`price_provider` is a `list(off|…)` enum, and the Supervisor validates the
options file against it **before the add-on starts**.
So the obvious edit — delete the three ids from the enum along with the classes
— has a consequence that runs exactly opposite to this card's purpose:
```
operator's config says price_provider: eneco_eddy
|
+-- id NOT in the schema enum
|
+-- Supervisor rejects the options file
|
+-- add-on never starts
|
+-- validate.py never runs
|
+-- the named refusal is NEVER SEEN
```
The operator gets an opaque `expected one of ...` from a layer they did not
configure, on **exactly the sites the withdrawal exists to inform**.
**Decision: keep all three ids in the schema enum.** `PROVIDERS` shrinks to
two; the enum stays at six. The cost is that the dropdown still offers three
dead ids to a *new* operator — and that is the cheaper failure, because
choosing one is refused at the next startup **by name**, with the endpoint, the
observation and the alternative, which is strictly more than a schema
rejection would say. Both translations label them withdrawn in the option
description.
`app/price.py`'s `RETIRED_PROVIDERS` is the list. `test_price.py` section 1
asserts the schema offers exactly `PROVIDERS | RETIRED_PROVIDERS | {"off"}` —
derived, not typed — and `register()` refuses to re-register a retired id, so
the registry and the retired table cannot both claim one and the refusal cannot
be taken away by an `@register` somebody adds back.
## 5. ⚠️ `PriceProduct.REALTIME` and the widening path are KEPT with no shipped declarer
`eneco_eddy` was the **sole** declarer of `PriceProduct.REALTIME`. Deleting it
orphans:
* `_WIDE = frozenset({PriceProduct.REALTIME})`
* `Uncertainty.REALTIME_PRODUCT`
* `WIDEN_FACTOR[Uncertainty.REALTIME_PRODUCT] = 2.5`
**They stay.** The reasoning, stated here so the next reader finds intent
rather than an apparent dead branch:
1. A reviewer finding a mechanism with **zero callers** would reasonably delete
it. That is good instinct applied to a missing fact, and this file is the
fact.
2. It is the **AC2 seam**. The whole point of `capabilities().product` is that
a new supplier is a new class and nothing else: declare `REALTIME`, get the
wide regime, edit no framework code. Deleting the enum member would mean the
first genuine imbalance product has to **rebuild** that seam — and rebuilding
it is precisely the "editing the optimiser to add a supplier" that AC2
forbids.
3. It costs nothing. Two enum members and a dict entry.
**⚠️ NO SHIPPED PLUGIN DECLARES `PriceProduct.REALTIME` UNTIL ONE DOES.** Both
shipped plugins (`generic_dayahead`, `manual_csv`) are day-ahead. §9.2's row 2
is unoccupied. That is not the same as it having no mechanism, and the
difference is what this section exists to record.
It is **pinned, not merely documented**. `test_price.py` section 5 asserts:
* that **no shipped plugin** declares `REALTIME` — so if one ever does, the
check goes red and the right fix is to **edit this record**, not to delete
the seam;
* that the widening still works **from a declaration alone** —
`volatility_regime`, `uncertainty_of` and `widen_factor` are graded on a
`ProviderCapabilities` the test file builds;
* and section 10 registers `_NewSupplier`, a class defined **outside
`price.py`** that declares `REALTIME` and is asserted to land in the wide
regime. That class is now the only live declarer in the tree, which makes the
seam something this suite exercises end to end rather than a constant it
reads.
## 6. What else went, and one thing that did not
`_points_from_json()` went with the three. It read a `{"data": [{...}]}`
row-per-point document that **no feed anybody here has ever reached
publishes** — the shape was invented alongside these plugins, and the only
bodies ever fed to it were fixtures written from the same guess. TARIFF-08
found that loop on the response side for `generic_dayahead`; this is the rest
of it. `needs_credentials` went too: the three were its only setters and
nothing read it.
The shape survives in **one** place and in one direction: `row_shape_body()` in
`test_price.py`, feeding `generic_dayahead` so that section 9 grades a row
document as a **failure** for it rather than a second dialect it might sniff.
It is named for what it is.
⚠️ **One check had to be rebuilt rather than deleted, and saying so matters.**
`test_price.py` section 4b proved that `code_leaks()`'s docstring strip is
load-bearing by finding a shipped docstring that a raw text scan would
false-positive on. The docstring that did that was **`engie_dynamic`'s** — it
spelled `price_markup_eur_kwh` in full. With it gone, no shipped docstring
spells a full framework key, so the check would have **passed on a tree with
the strip deleted**: a check that had quietly stopped being one, which is the
third failure mode in `MUTATION.md`. The stimulus is now constructed in the
test file (`_DocumentedPlugin`), and the shipped set is asserted clean
separately.
## 7. Mutations driven
Every check added or rebuilt here was driven red. Harness:
`/home/homelab/.cache/t07-mut/`, which rebuilds the tree from the worktree and
purges `__pycache__` per `MUTATION.md`'s rule.
| | mutation | Δ | red on |
|---|---|---|---|
| M1 | delete the retired branch from `_price_problems()` | −2785 B | 7 checks in `test_price.py` |
| M2 | move the retired branch **after** the unknown-id branch | **+0 B** | the same 7 |
| M3 | make `been_running` constant (drop the empty-`static_profile` case) | −212 B | 2 |
| M4 | drop the "WHAT TO DO" half of the refusal | −874 B | 1 |
| M5 | `RETIRED_PROVIDERS = {}` | −1099 B | 5, then a traceback |
| M6 | drop the three ids from `config.yaml`'s enum — §4's trap | −47 B | 2 |
| M7 | delete `register()`'s retired guard | −1101 B | 2, then a traceback |
| M8 | `_WIDE = frozenset()` | −23 B | 3 + 4 across two suites |
| M9 | `WIDEN_FACTOR[REALTIME_PRODUCT]` 2.5 → 1.5 | **+0 B** | 1 + 1 |
⚠️ **M2 and M9 have a byte delta of zero**, which `MUTATION.md` names as the
risky case. The harness rebuilds the tree from source on every run and purges
`__pycache__`, so there is no inherited cache to read; and stale bytecode
manufactures **survivors**, never reds, so a red from a `+0 B` mutation is not
at risk from that defect. M2's grep confirmed the branch had moved on disk.
⚠️ **M5 and M7's red counts are LOWER BOUNDS.** Both runs ended in a traceback
with checks after it ungraded — FCST-06's rule: read the output, not the count.
M7's traceback is itself informative: with the guard gone the re-registration
**succeeds**, `PROVIDERS` is polluted with a class that has no fixture, and the
suite dies downstream. That is the failure the guard prevents, arriving loudly.
## 8. What this card did not do
* **It did not pick a version number.** The change is backward-incompatible — a
config legal on the current version is refused on the next — and the
CHANGELOG says so prominently, in the shape PKG-07 established for `slew_w`.
Which number it belongs to depends on whether the current image has already
been built and pulled, which is a fact about the live box that this branch
cannot observe. Whoever deploys decides.
* **It did not implement the formula provider.** `B × spot + A` with per-site
coefficients is the real answer for a Belgian dynamic customer and is a
separate card. `DOCS.md` references it as planned and is explicit about what
`generic_dayahead` can and cannot express in the meantime — in particular
that it cannot represent a negative offset, and that
`price_markup_eur_kwh: float(0,1)` could not hold one anyway.
* **It did not rewrite released CHANGELOG history.** The two historical entries
that called `eneco_eddy` an imbalance product are annotated in place with a
correction naming this card, rather than silently edited. The record of what
the project believed is part of the record.
@@ -0,0 +1,141 @@
# TARIFF-09 — asking the feed for the horizon, and what `price_feed_url` means
Two decisions. The first had no defensible alternative once it was measured;
the second did, and the alternative is argued below rather than dismissed.
---
## 0. The measurement this card exists for
`generic_dayahead.fetch()` accepted `since` and `until` and requested the bare
`https://api.energy-charts.info/price?bzn=BE`. That URL names no range, and the
feed answers **one Brussels day**.
Measured on the merged tree (`4257e38`) at 2026-09-10 09:00 CEST, 48 h horizon,
live feed, yesterday seeded through the shipped read path:
```
60 firm points (15 whole hours) + 33 carried hours of 48 requested
carried: 24 h on 2026-09-11, 9 h on 2026-09-12
arbitrage(0.10): allowed, spread 0.4645, buy 11:45 and sell 19:45 -- BOTH on 2026-09-10
firm local days = [2026-09-10]
```
Everything past today was carry-forward, flagged provisional, and §8.3 drops
provisional hours from the trade set. **The optimiser never saw tomorrow as
firm, at any hour.** The ceiling was structural rather than incidental: with a
one-day window, at hour *H* of the local day a 48 h horizon is at most
**24−H hours firm and 24+H carried** — 15/33 at 09:00, and **1/47 at 23:00**,
where the remaining firm spread (0.0464 EUR/kWh) no longer even clears the
default 0.10 round-trip threshold and arbitrage is refused outright.
That is not what FCST-04's fill was built for. FCST-04 covers the
pre-publication window until the ~13:00–14:00 CET auction, after which a
healthy feed hands over tomorrow in full.
**The data exists and the endpoint serves it.** Verified 2026-09-10T07:45:49Z:
`?bzn=BE&start=2026-09-09&end=2026-09-10` returns **192 quarter-hours across
two local days**, committed unmodified as
`scenarios/energy-charts-be-price-2026-09-09_2026-09-10-twoday.json`. The
plugin simply never asked.
After the change, same code path, same window, a feed serving both days:
**39 firm hours and 9 carried**, firm across **two** Brussels days, and the
permitted cycle buys on one day and sells on the other. Graded in
`test_price_history.py` section 1c against both feed states.
---
## 1. The range is derived on the **Brussels** calendar, from `since`/`until`
`since` and `until` are **instants**; the API takes **dates**. Those are not
the same question, and the conversion is where the day gets lost.
`brussels_date_range()` returns two inclusive `date`s:
* the first is `brussels(since).date()`. **Not `since.date()`, and not the UTC
date.** 22:30 UTC in summer is already tomorrow in Belgium, so a UTC-derived
start asks for the day the controller has just left and drops the day it is
standing in — for the last one or two hours of *every* day. This is
`price_history._bdate()`'s lesson one layer up, with a worse consequence:
`_bdate()` misfiles an hour, this never fetches it.
* the last is `brussels(until).date()`, **stepped back a day when `until`
falls exactly on local midnight**, because `until` is exclusive and that
instant is the first tick of a day nobody asked for. Without it every
whole-hour horizon requests one extra day — harmless on this feed, but it
makes the request a claim the caller never made.
* an empty or inverted window collapses to a single day rather than producing
a backwards range, which no feed has a defined answer for.
Both ends are graded, and both are driven red by mutation — including the two
that swap `brussels()` for the UTC date (`test_price.py` section 2c).
---
## 2. `price_feed_url` is the **exact URL**, used as given. The plugin appends
nothing.
The alternative — merge our `start`/`end` into whatever query string the
operator supplied — was considered and rejected. Three reasons, in the order
they decided it:
1. **`start`/`end` are energy-charts' spelling, and this provider declares a
second host.** `generic_dayahead.capabilities().egress_hosts` is
`("api.energy-charts.info", "web-api.tp.entsoe.eu")`. That second host's
day-ahead date parameters have not been verified by this card and are not
guessed at here — but they are not `start`/`end`. So an override to the
*other declared host* — a case the declaration itself anticipates — is
exactly where merging appends two parameters that host ignores, producing
the short horizon anyway, with a URL nobody typed as the receipt.
2. **Merging means re-serialising a URL somebody else wrote.** Parsing and
rebuilding reorders parameters, collapses duplicates and re-escapes values.
"Override the feed" is what the option already says it means, and an
override that is silently rewritten is not one.
3. **Blind appending is not on the table at all.** `dated_feed_url()` refuses
to add a query string to a base that already has one, so
`?bzn=BE?bzn=BE&start=…` cannot be produced by this module even by a later
edit that folds the parameters back into the constant.
⚠️ **The cost is real and it is therefore NOT a doc footnote.** An overriding
operator gets exactly the short horizon this card exists to fix. So the plugin
emits a `WARNING` on the shipped `goodwe.price` logger at **fetch time**, naming
the consequence — one Brussels day, the rest carried and flagged, §8.3 dropping
them from the trade set — and naming the `start..end` it *would* have asked
for. §11.4 is what makes a price defect quiet; an operator who has taken the
short horizon has to be able to find out from the log that they did.
The warning is graded, with a control that the default path stays silent, and
a mutation that downgrades it to `debug()` takes that check red.
**Fetch time rather than startup**, because that is where the URL actually in
use is known. The trade-off is one line per refresh (the tariff task, not the
30 s control tick). If the status page ever grows a price-source row, this is
the sentence it should carry.
---
## 3. The plugin contract: `fetch(self, opts, since, until, transport)`
The signature promises every plugin the range, and until this card **none of
the four networked plugins used it**. The obvious follow-up is to enforce the
promise. It is **not** enforced, and the reason is TARIFF-08's:
Three of the five shipped plugins point at endpoints nobody has ever verified
(TARIFF-07 is the open decision on withdrawing them). A rule obliging them to
pass a date range would be satisfied by an **invented parameter name** — which
is the parser-graded-against-its-own-guess defect moved from the response to
the request, and it would ship looking like a guarantee.
What exists instead is a **roster pin** in `test_price.py` section 2c: which
plugins reference their `since`/`until` is asserted as a literal, derived from
the registry and the AST. A sixth supplier, or a change to any of the three,
has to edit that line and make the decision rather than inherit the silence.
It is named as a source-shape pin, not as evidence that any feed parses;
`generic_dayahead`'s behaviour is graded separately by the URL it really
requests.
**Recommendation for whoever picks up TARIFF-07:** enforce it at the moment
those three endpoints are either verified or withdrawn, not before. If they are
withdrawn, `generic_dayahead` and `manual_csv` are the whole roster and the
rule is one line — `manual_csv` reads a whole file and is windowed downstream,
so it would be the single documented exemption.
@@ -0,0 +1,277 @@
# TARIFF-10 — fetching a price and trading on one are two decisions
**Status: DECIDED 2026-09-12 — Option A. If a price provider is configured,
prices are fetched. Trading stays a separate switch (`arbitrage_enabled`), and
this card moves nothing that moves energy.**
Filed with TARIFF-10. Follows [[TARIFF-09]] and [[FCST-04]]; §4 settles the
limitation FCST-04 recorded and did not decide.
---
## 1. The defect, and why it was invisible
`app/main.py`'s `amain()` read:
```
if arbitrage.enabled(opts):
tasks.append(asyncio.create_task(controller.run_tariff_refresh()))
```
`run_tariff_refresh` is the **only** caller of `resolve_tariff()`, which is the
**only** caller of `price_history.carry()`. `arbitrage_enabled` ships `false`.
So on a shipped install with a provider configured:
* no HTTPS request was ever made to the feed;
* no plan was ever resolved, so `self.tariff_plan` stayed `None` forever;
* `/data/price_history.json` was never created;
* the *Price history* status row read `empty` for the life of the install;
* TARIFF-09's short-horizon warning could not fire, because it is logged on a
fetch and there was no fetch.
Measured, not inferred — a 40 h soak, two instances, one feed:
| | shipped default + provider | same, arbitrage armed |
|---|---|---|
| feed requests | **0** | fetching |
| `/data/price_history.json` | `present: false`, **802/802 samples** | written |
| resolved plan | none | 132 points |
⚠️ **AND FOUR SURFACES ALREADY DESCRIBED THE FIXED BEHAVIOUR.** `DOCS.md`
§*Energy price providers*: *"With `arbitrage_enabled: false`, which is the
shipped default, turning a provider on still fetches, validates and publishes
prices and changes no charging decision."* `config.yaml` and both translations
said the same, and additionally carried a claim that had gone stale in the other
direction — *"nothing consumes the price series yet"*, untrue since OPT-04
shipped the consumer. The code was the only surface that disagreed with all of
them, and no check compared the two.
The dating is worth being exact about, because it says this was never a
regression. `git log -S` puts the refresh task, its `arbitrage.enabled()` guard
**and** the DOCS sentence in the *same commit* — OPT-04, `bdd7f18`;
`config.yaml`'s version of the claim goes back further, to TARIFF-02,
`852fc7c`. The prose and the code disagreed from the moment both existed.
## 2. Why a check on the `if` would not have caught it
The `if` was exactly what somebody meant to write, and its comment argued for
it honestly:
> ⚠️ STARTED ONLY WHEN ARBITRAGE IS ARMED. Nothing else in the add-on reads the
> resolved plan … so on a default install this task would be network traffic in
> aid of an attribute nobody reads.
That was true when OPT-04 wrote it and stopped being true when FCST-04 gave the
plan a *store* and a *status row*. A pin on the expression would have gone red
on the fix and green on the defect. What was wrong is the **set of
configurations from which a fetch is reachable**, so that is what
`test_price.py` section 13 grades:
1. the guards on every `asyncio.create_task` in `amain()` are **evaluated**
against real option dicts — whatever the guard is spelled as;
2. the call graph under `app/` is walked to see whether the coroutines those
guards start can arrive at `resolve_tariff()` — wherever it has moved to;
3. and then the fetch is **driven**, with a counting transport, a real
`Controller` and a real store path, because a reachability graph is an
argument and a counted request is an observation.
⚠️ **Step 3 grades the gate only because it starts the task the way `amain()`
would.** The driver asks `_fetchers(opts)` — the evaluated guard — before it
calls the coroutine. Measured: with the shipped defect restored **and** that one
line defeated (`if False and "run_tariff_refresh" not in names:`), the four
driven rows go **green** while only the structural rows stay red. Without it
they are decoration, which is MUTATION.md's habit 4 in a harness.
What the instrument **cannot** see is printed on every run: a coroutine started
by anything but `create_task`; a guard that is not an enclosing `if` in
`amain()`; anything outside `app/`; and — because the graph is keyed by **name**
— two same-named functions as one node, which makes every *reachable* answer an
**upper** bound and every *not reachable* answer the strong one.
## 3. The gate, and why it is an `or`
```
def tariff_refresh_wanted(opts: dict) -> bool:
return provider_configured(opts) or arbitrage.enabled(opts)
```
The first term is what this card adds. **The second term is the old gate, kept
verbatim, and it is not there for symmetry.**
`price_provider: off` + a `static_profile` + `arbitrage_enabled: true` is a
configuration `validate.py` explicitly **accepts** (rule 7 refuses only the
no-prices-at-all case), and it **trades**. Measured on this branch:
```
opts: price_provider off, arbitrage_enabled true,
cycle_cost_threshold_eur 0.05,
static_profile 0.20 night / 0.284 day
plan : static_profile from static_profile, 48 h, 48 points
WITH the plan -> ARBITRAGE_CHARGE -5000.0 W
"cheapest firm hour at 0.2000 EUR/kWh, 0.0840 below the
dearest and clear of the 0.0750 this cycle had to cover"
WITHOUT the plan -> None "no tariff plan has been resolved yet"
```
Gating the refresh on the provider **alone** — which is the literal reading of
the ticket's second acceptance criterion — would leave that site's
`tariff_plan` at `None` forever and answer *"no tariff plan has been resolved
yet"* on every cycle. That is a strategy the UI says is running and that cannot
fire: the exact state `validate.py`'s OPT-04 rules exist to refuse, arrived at
from inside the code instead of from the configuration.
So the gate is a **strict widening**: every configuration that started the task
before still starts it, and `test_price.py` grades that as its own row rather
than leaving it to prose.
⚠️ **The AC's stated harm is still fully met, and it is worth being exact about
this rather than quietly declaring victory.** The criterion reads *"with
`price_provider: off`: no fetch task at all, whatever `arbitrage_enabled` says
— an operator with no provider must not acquire a periodic HTTPS call."* The
letter of it is not met in one configuration (provider off, trading armed),
because that configuration keeps the task it already had. The harm it names is:
* with `price_provider` off, `resolve_tariff()` returns `static_plan()` — pure
arithmetic, **no socket**. Driven and counted: **0 requests**.
* the true shipped default — no provider, no trading — starts **no task at
all**.
Nobody *acquires* a periodic HTTPS call from this card, and the only thing that
decides whether a request leaves the machine is `price_provider`. **If the
owner wants the literal reading instead, it is a one-token change** (drop the
`or` term) **and it costs the site in the measurement above its trading.** That
is the trade, stated rather than chosen silently.
## 4. `TariffPlan.cheap_now()` — available, and deliberately NOT consumed
**Decision: do not wire it in this card. `self.cheap` stays on
`static_status()`.**
FCST-04 recorded that `cheap_now()` has no production caller on the provider
plan: `main.read_inputs()` assigns `self.cheap, self.tariff_words =
static_status(o, now)`, which builds its **own** static plan and calls
`cheap_now()` on that. Under Option A a provider-backed plan now exists on every
configured site, so the limitation stops being structural and becomes a choice.
**It is not cosmetic, and that is the whole reason for the decision.**
`self.cheap` is read in `cycle()` by the maintenance cycle's cheap-window grid
top-up — the add-on's one shiftable load, and the one place a *price* reaches
something that **moves energy** on an install that has not armed trading.
⚠️ **And the direction it would move is not "a slightly different window".**
`static_status()` returns `(False, "no static profile")` whenever
`static_profile` is empty, so today a provider-only site — a plausible shape
for a dynamic-tariff install, and the shape this card makes work — has
`self.cheap = False` on **every** cycle and never forces a paid grid top-up at
all. Wiring `cheap_now()` would take that from *never* to *sometimes*. That is
a strictly larger behaviour change than this card's headline, and it belongs to
whoever owns the maintenance cycle rather than to the ticket that made it
reachable.
Three further reasons, in the order they would bite:
1. **Two sources, one attribute, and the loser is silent.** `validate.py`
already refuses `static_profile` alongside the Home Assistant price
entities *because* `read_inputs()` picks one and the other disappears. A
provider plan would be a third writer of the same attribute with no rule
saying which wins.
2. **The freshness question is not answered for this consumer.** The plan
retires after `TARIFF_MAX_AGE_S` and `fresh_tariff_plan()` returns `None`.
Arbitrage has an answer for that (refuse). The maintenance top-up has none:
"the plan expired" would have to mean *not cheap*, and *not cheap* on this
path means **wait for sun**, which is a real decision nobody has taken.
3. **The comparison basis changes, and so does the answer on a bad feed.**
`cheap_now()` is strictly-below-the-mean-of-the-**firm**-hours, and it is
`False` when THIS hour is provisional or is not in the series at all
(FCST-03's rule: this hour decides, not the whole plan). So a feed that
published nothing, or an hour the §8.3 fill could not cover, reads *not
cheap* where a `static_profile` reads whatever the clock says. Defensible
either way — and it is a change to the shipped maintenance cycle that needs
its own measurement rather than a side effect of this one.
**What adopting it would take**, so the next card does not start from scratch:
* a rule for which source wins when both a provider plan and a
`static_profile` are configured, enforced in `validate.py` the way the
entity/profile conflict already is;
* a decision on what an **expired** or **absent** plan means for the grid
top-up, stated as a default rather than left to `None`;
* `read_inputs()` splitting `self.cheap` from `self.tariff_words`, which are
currently one assignment from one function;
* a check that the maintenance top-up actually fires and does not fire on the
provider plan — driven through `cycle()`, not asserted on the attribute;
* and the *Tariff* status row, which FCST-04 pinned as a **pair** with
`DOCS.md` in `test_price_history.py` §9: it does not report the provider
plan, DOCS does not claim it does, and moving one without the other is
already red.
## 5. The *Arbitrage* status row, re-read and left alone
`main.status_payload()` renders `("Arbitrage", "off (arbitrage_enabled)" if
self.arb is None else self.arb.explain())`. `self.arb` is assigned only inside
`if arbitrage.enabled(self.o):` and `arbitrage.decide()` never returns `None`,
so `self.arb is None` **iff** the option is off. The string still names the
option that is actually off, and under Option A it gains a job it did not have:
on a site with a provider configured it is the row that tells an operator the
prices arriving on the *Price history* row above are moving nothing. Unchanged,
with a note at the call site saying it was checked.
## 6. What this card did not do
* It did not touch `run_tariff_refresh()`'s body (TARIFF-12 is there).
* It did not change `cycle()`'s `if arbitrage.enabled(self.o):`, the sizing,
the thresholds, the SoC bounds or the peak interlock.
* It did not wire the provider plan into the *Tariff* status row or into
`self.cheap` — §4.
* It did not pick a version number (PKG-06).
## 7. Mutation ledger
Every mutation applied to a **fresh `cp -a`** of the clean worktree rather than
reverted (habit 18: there is no revert to verify and no shared scratch path),
`__pycache__` purged, tree mounted `:ro`, run under `python3 -B`.
| id | mutation | Δ bytes | result |
|---|---|---|---|
| M1 | the shipped defect: gate back to `arbitrage.enabled(opts)` | **-4 B** | **RED**, 7 named checks |
| M2 | drop the `or arbitrage.enabled(opts)` term | -27 B | **RED**, 2 named checks (strict-widening + the static-profile row) |
| M3 | `tariff_refresh_wanted` -> always `True` | +8 B | **RED**, the AC2 reachability row |
| M4 | `provider_configured`: `"off"` counts as configured | -6 B | **RED**, the AC2 reachability row |
| M5 | `cycle()`'s trading gate -> `if True or arbitrage.enabled(self.o)` | +8 B | **RED**, both AC3 rows (`-5000 W` with trading off) |
| M6 | instrument: `_reaches()` always `True` | **+0 B** | **RED**, 4 checks incl. its own calibration row |
| M7 | instrument: `_task_sites()` returns `[]` | **+0 B** | **RED**, 10 checks incl. "it found 0 sites" |
| M8 | fixture: 5 minutes of telegrams instead of a whole quarter | **+0 B** | **RED**, the peak-guard precondition and both armed rows |
| M9 | M1 **plus** the harness's guard-driven start defeated | -4 B | the four DRIVEN rows go **GREEN** — see §2 |
| M10 | `provider_id()` drops `.strip()` | -8 B | **survivor**, then CLOSED — see below |
| M11 | `provider_configured()` -> `return True` | -32 B | **RED**, 4 checks |
| M12 | DOCS.md's sentence reverted to the old, vaguer claim | +0 B (docs) | **RED**, the AC4 pairing row (`fetches = True, DOCS says = False`) |
| M13 | the Dutch translation left behind while English moved | +0 B (docs) | **RED**, the both-languages row |
⚠️ **M10 was a real survivor and is recorded as one.** Dropping `.strip()` left
`test_price.py`, `test_price_history.py`, `test_static_tariff.py` and
`test_validate.py` all green: nothing in the repository fed a `price_provider`
with surrounding whitespace, and the `.strip()` predates this card — it was
carried into `provider_id()` from the three call sites it replaced. The
Supervisor's `list(off|…)` schema cannot produce `" off "`, but
`validate_startup_config()` is also `sim/inject.py`'s entry point and takes any
dict at all. Closed in the same commit by the padded-id pair in section 13;
re-run as **M10b**, red on two named checks.
⚠️ **M8 is the fixture defect this card walked into and then fenced.** The
first draft fed five minutes of telegrams. Whether that **covers** the current
billing quarter depends on where the wall clock sits inside it, so the peak
guard was blind and engaged on some runs and not others — and `decide()` refuses
outright while the guard is holding (§9.1 over §10.1). On those runs the armed
baseline did not trade, and "trading off never trades" would have been graded
on a fixture in which nothing could trade either way. It is now 900 s of
telegrams, and the guard's clear state is a **check** rather than an assumption.
## 8. What is still open
* The literal reading of AC2 (§3). One token, and it costs a measured trading
configuration its plan. **Owner's call.**
* `cheap_now()` (§4). Owner's call, with the adoption list above.
* The *Tariff* status row still shows the static-profile sentence on a
provider-backed site; the provider plan's own line goes to the log. Pinned as
a pair with `DOCS.md` by FCST-04 and untouched here.
@@ -0,0 +1,252 @@
# TARIFF-12 — a returned degradation is not a refresh
Three decisions. The first is what the ticket put up for judgement; the second
and third are what fall out of it and would otherwise be decided by accident.
---
## 0. What was actually wrong, stated once
`run_tariff_refresh()`'s docstring:
> ⚠️ **AND A FAILURE DOES NOT CLEAR THE PLAN.** … Zeroing the plan here would
> replace a coarse tariff with no tariff, which is the opposite of what §11.4
> asks for.
The guarantee was real. The condition it was enforced on was not:
```python
try:
plan = await resolve_tariff(...)
self.tariff_plan, self.tariff_at = plan, now # unconditional
except Exception as err:
_LOG.warning("tariff refresh failed (%s); keeping the plan ...")
```
`load_prices()` says in its own docstring that it **"RETURNS, NEVER RAISES, for
every failure a site can actually have"**, because a degradation arriving as a
traceback into the planning tick would stop the controller regulating over an
expired certificate. So a supplier outage — the case the docstring names first —
is a **return**: the `try` succeeds, `_degrade()` hands back a §11.4 plan, and
the good plan is overwritten. The `except` is reachable only by something
nobody expected, and nothing in the shipped path produces it.
This is the fourth card in one family: FCST-04's M4 (`not ok` only), FCST-05's
HIGH (`(JSONDecodeError, OSError)` only), FCST-06 (`not self.ok` only), and now
`except Exception` only, for a path that never raises. **A guarantee enforced on
the condition someone thought of, rather than the condition that occurs.**
Reproduced before the fix, driving the shipped chain with nothing stubbed but
the socket — 48 h horizon, `static_profile` empty, one good fetch then a
`ClientConnectorError` 1800 s later, `TARIFF_MAX_AGE_S` = 5400 s:
```
after a good fetch origin='provider' n_points=48 'dynamic_dayahead from generic_dayahead, 48 h'
after the failed fetch origin='none' n_points=0 'no tariff - generic_dayahead: ClientConnectorError: ...'
tariff_at moved: True
```
and after it:
```
after the failed fetch origin='provider' n_points=48 'dynamic_dayahead from generic_dayahead, 48 h'
tariff_at moved: False
```
---
## 1. The question is asked ONCE, after the attempt, from the evidence that exists
FCST-06's answer, one module over, and for the same reason: **the enumeration
of failure conditions cannot be completed.** There is no pre-check that a feed
will answer which is not itself a guess, and adding one would be a second
symptom test beside the `except`. So the predicate is an **outcome**, read off
the object the attempt produced:
```python
standing = self.fresh_tariff_plan(now)
keep = (not plan.from_feed
and standing is not None and standing.from_feed)
```
Two facts, both of which exist by the time the question is asked:
* **`plan.from_feed`** — did *this* resolution observe the configured provider?
`TariffPlan.origin` already names the four ways a plan is obtained; this
collapses them to the one question a caller holding a previous plan has to
ask. It lives on `TariffPlan` rather than as an `origin == "provider"` at the
call site for `fresh_tariff_plan()`'s own stated reason — *a check has to be
able to assert on the shipped expression rather than on a transcription that
drifts* — and because `origin`'s vocabulary belongs to `static_tariff.py`.
* **`fresh_tariff_plan(now)`** — is a standing observation still current. It
already existed, it is already the one place the age is judged, and it is
**reused, not transcribed**. `test_arbitrage.py` 8b pins that reuse on the
AST; a copy of the comparison behaves identically until the day the two
drift, which is a claim about the code's shape, so a shape check is what
grades it (FCST-06's precedent for the one source check among driven ones).
### What was NOT done
* **Not `except Exception` widened, or a second `if` beside it.** That is the
defect's own shape — a place where the rule is spelt twice, so a fifth cause
needs a fifth branch. The `except` stays, relabelled *"not the outage path"*,
keeps the plan on the same terms, and is now honestly the
something-nobody-expected branch.
* **Not the persisted price store as a cache.** Out of scope by the product
owner's call, and it is the right call: FCST-03 exists precisely to stop
stale prices being acted on, and "recent" is not "current". Keeping the last
**fetched** plan until it ages out is a different promise from serving the
store, and that one needs its own ticket.
* **Not a `no_data`-only guard.** `not plan.has_prices` is the tempting
spelling and it is wrong — see decision 2. It leaves the §11.4 fallback free
to replace a fresh observation, and it is a 1-byte edit from the shipped
line, which is why it is in the mutation table rather than in a comment.
---
## 2. A §11.4 static fallback does NOT replace a fresh provider plan
The ticket asked this directly, because the fallback is *not* nothing: it is
the operator's own configured profile, with real prices, for the requested
window. The answer is still no, and the reason is not that it is worse data —
it is that **nothing may be acted on it, and it is no more current.**
* Every point `_degrade()` builds carries `provisional=True`. So
`firm_points()` is empty, `arbitrage()` finds fewer than two firm hours and
refuses, and `cheap_now()` is False. Both of the two things `TariffPlan`
exposes for a consumer to act on are already off.
* A day-ahead price for a given hour **does not change once published**. A plan
fetched 30 minutes ago is exactly as true about 19:00 as one fetched now; at
most `TARIFF_MAX_AGE_S` of its horizon has elapsed. The observation is not
stale in any sense the fallback is fresher than.
So replacing the observation with the fallback trades a plan with firm hours
for a plan with none, and buys no currency with it. The fallback takes over
when the observation **retires** — which is exactly the row §11.4 describes,
arrived at when there is no longer an observation to prefer.
### And the rule is deliberately NOT symmetrical
`standing.from_feed` is a term in its own right. A standing plan that is itself
a degradation **is** replaced by the next degradation, and its timestamp moves.
Without that term:
* an outage on a static-profile site would freeze the operator's profile at the
window it was first generated for, and the guard would be *"never overwrite"*
wearing a bound's name;
* and on a `price_provider: off` install — the shipped default — the plan would
flip between kept and re-derived every `TARIFF_MAX_AGE_S`, for no behavioural
gain, changing which reason the status page prints on a two-hour cycle.
Each of the three terms is the only false one in some check in 8b, which is
MUTATION.md habit 17's test for whether a suite can tell an `AND` from an `OR`.
---
## 3. `tariff_at` does not move on a kept plan
The docstring's other half is that a plan must not *"freeze at its last value
and look current forever"*. `tariff_at` is what makes that work: it is the
moment of the fetch that produced the plan standing, and `fresh_tariff_plan()`
judges the age from it.
Re-stamping it on a kept plan would be this task asserting the feed answered
when it did not. The plan would be re-certified as current at every refresh, an
outage of any length would never retire it, and `TARIFF_MAX_AGE_S` would be
**exactly as dead as it was before this card** — reachable only by the refresh
task itself dying. Leaving it alone is what makes the bound govern: the plan is
served for the age bound after its own fetch, and no longer.
It is also the whole of **"distinguishable from a freshly-fetched one"**. A
kept plan and a fetched plan are the same object shape; what separates them is
that one's timestamp advanced and the other's did not, and every consumer that
asks how old the plan is already reads that number. The kept case additionally
logs at WARNING, naming the degradation's own reason, the fetch it is keeping,
its age, and the deadline it retires at.
⚠️ **What this does not give the operator, and why it is not here.** The status
page has no row for "the last refresh failed" — the banner/level not moving
when the plan is lost is separately carded, and the ticket rules it out of
scope. This card's change makes that surface *more* truthful, not less: during
the first `TARIFF_MAX_AGE_S` of an outage there is now a real plan behind the
Arbitrage row instead of a no-plan refusal, so the row says something true for
longer. The no-plan state is entered later by up to 5400 s; it is not removed,
and the 30-minute lag reported on that row is untouched by anything here.
---
## 4. Merged after TARIFF-10 — what the wider gate changed, and what it did not
TARIFF-10 landed on `release/1.0` while this card was open. It changed **which
installs run this task** — `if arbitrage.enabled(opts)` became
`tariff_refresh_wanted(opts)`, i.e. `provider_configured(opts) or
arbitrage.enabled(opts)` — while this card changed **what the task does on a
failed fetch**. Git auto-merged the two hunks, which is exactly when the
interaction needs reading rather than trusting.
**Nothing in the decisions above changed.** Two configurations are newly
reachable, and both are now driven in `test_arbitrage.py` 8b rather than
reasoned about:
* **A provider configured with trading DISARMED** — TARIFF-10's headline case.
This is precisely the configuration the guard was designed for: the plans are
`origin == "provider"`, and a dead feed must not delete one that is still
inside `TARIFF_MAX_AGE_S`. It needs no second fixture, because **nothing on
this path reads `arbitrage_enabled` at all**, and that is asserted rather
than assumed — over the AST's symbols and the non-docstring string constants,
because two earlier drafts of that check were **red against a correct tree**:
the first matched the docstring paragraph that *says* `arbitrage_enabled` is
not read, the second matched the log line's "arbitrage, where it is armed".
A substring test over a function grades its prose as much as its behaviour,
which is habit 4 in a third costume. A fixture that set the option and
asserted no difference would have been the other defect — comparing a thing
against itself.
* **`price_provider: off` + a `static_profile`** — the configuration TARIFF-10's
`or` exists to protect, and the one where a latching guard would do real harm.
`resolve_tariff()` returns `static_plan(provisional=False)`, whose origin is
`"static"`: the CONFIGURED product, not a §11.4 row. So `standing.from_feed`
is false on every refresh, `keep` is false on every refresh, and the plan is
regenerated for the current window every 30 minutes exactly as before this
card. **Driven, twice in a row**, because a guard that latched here would
freeze a static site's plan at its first window for the life of the install —
and mutation M3, which drops the `standing.from_feed` term, is red on that
check as well as on the fallback one.
Two further facts settle the rest of the interaction:
* **This decision is strictly downstream of every side effect.** The §8.3 carry
into `price_history` happens inside `resolve_tariff()`, on the `series`
branch, before `keep` is computed — so which of two plans is kept can neither
add nor lose a recorded day, and TARIFF-10's *Price history* row is
unaffected either way.
* **The kept-plan log line was reworded.** It said "arbitrage stops with it",
which after TARIFF-10 is false on a site that fetches without trading; it now
says "arbitrage, **where it is armed**, stops with it".
All eleven mutations were **re-driven on the merged tree** rather than carried
over — the surrounding code changed, so a red against the old base is not
evidence about the new one. Eleven red, no survivors, every revert
fingerprinted. The added member is M11: making the guard conditional on
`arbitrage.enabled(self.o)` (+86 B), red on five named checks.
⚠️ **And one process note that belongs in this file.** This card's mutation
harness was written to a shared scratchpad path and was **overwritten mid-run
by another agent's harness for a different card**, pointing at a different
worktree. That is MUTATION.md habit 18's second bullet almost verbatim. It was
caught because the harness fingerprints the tree after every revert and the
runs are what proved this worktree was never left mutated; the harness now
lives at a card-specific path. **A harness that edits source must not share a
name, a directory or a backup path with another one.**
---
## What this does not change
* **Nothing on the 30 s tick.** `cycle()` still reads two attributes and awaits
no fetch; the AST pin in `test_arbitrage.py` section 8 is unchanged and green.
* **Nothing about the prices themselves.** No series is invented, extended,
re-stamped or carried. The only question this card answers is *which of two
plans the controller holds*.
* **A `price_provider: off` install is bit-for-bit unaffected.** Its standing
plan is never `origin == "provider"`, so `keep` is false on every refresh it
will ever run, and the assignment happens exactly as it did before.
@@ -0,0 +1,225 @@
# TARIFF-13 — the coverage of a peak, and which quarter gets the credit
Two decisions. §1 is the defect the ticket was raised on; §2 is the smaller
question it asked to be settled deliberately rather than left as an accident.
§3 is what this branch deliberately did **not** do, and why.
---
## 0. What was actually wrong, stated once
`main.py`'s capacity-tariff status row prints a peak read from SQLite and a
coverage figure read from **process memory**:
```python
_pk = self.peak.peak(when) # SQLite, durable
...
+ f"; {self.peak.recorded} quarter(s) banked" # an int on the tracker
```
`PeakTracker.recorded` was `self.recorded = 0` in `__init__`, incremented once
per block inside `sync()`. A 40-hour soak caught what that means at a driven
restart, on both instances independently:
```
before: 2026-09: peak 365 W, set by the quarter at 2026-09-11T18:45:00+00:00
- 10 % of the 3500 W soft limit; 120 quarter(s) banked
after : 2026-09: peak 365 W, set by the quarter at 2026-09-11T18:45:00+00:00
- 10 % of the 3500 W soft limit; 0 quarter(s) banked
```
Reproduced here before anything was changed, through the real `Controller`, the
real `P1Ingest`, and a **real restart** — a second `Controller` object against
the same file on disk, never a re-read of the first:
```
before: 2026-09: peak 365 W, set by the quarter at 2026-09-10T09:00:00+00:00
- 10 % of the 3500 W soft limit; 12 quarter(s) banked
after : 2026-09: peak 365 W, set by the quarter at 2026-09-10T09:00:00+00:00
- 10 % of the 3500 W soft limit; 0 quarter(s) banked
peak survives the restart: True
count survives the restart: False (12 -> 0, store held 12 rows)
```
**TARIFF-01's argument, one column over.** That module's whole design is that
an unreadable store must not read as a zero peak, *because a zero peak is
disarming*. A coverage figure of zero is disarming in exactly the same way and
about the same number: the count is what says how much of the billing month the
watt figure actually covers, and 365 W over 120 quarters and 365 W over 0
quarters are different claims about the same 365 W. After a restart the page
made the weaker claim while displaying the stronger one's value. It did not
under-report the peak — it under-reported **its own evidence**.
Two further defects fell out of the same variable once it was driven, neither
of them in the ticket:
* **it was never scoped to a month.** It counted every block the *process* had
banked, so on 1 October the row read September's tally beside October's "no
completed quarter recorded yet". Measured, on the shipped tree:
`2026-10: no completed quarter recorded yet - 0 % of the 3500 W soft limit;
1 quarter(s) banked`.
* **it missed the block `restore()` banks.** A stale block closed from the
checkpoint goes into `quarter_blocks` without passing through `sync()`, so
the counter never saw it.
---
## 1. Decision: the count is READ FROM THE TIER THE PEAK IS READ FROM
The acceptance criterion allowed either route — make the count durable, or stop
printing a number that silently resets. **Durable**, because the count is the
useful half of the row and removing it would have cost the operator the one
thing that says whether the peak means anything yet.
`PeakStore.banked_count(month)` asks the database `SELECT COUNT(*) FROM
quarter_blocks WHERE month=?`, and `PeakTracker.recorded` is a property over it
rather than an attribute. Three consequences, and all three are the defects
above:
* a restart re-reads the same rows, so the figure survives it;
* the count is per **month**, so it agrees with the peak it is printed beside;
* a block banked by `restore()` is a row like any other and is counted.
Three things this route deliberately does **not** do:
* **it does not count `len(blocks_in(month))`.** `blocks_in()` returns `[]` on a
read error — safe there only because no caller reads its emptiness as a peak,
which its own docstring says — so taking its length would rebuild the
disarming zero one call further out. `banked_count()` returns **`None`** when
the database will not answer.
* **it does not count `monthly_peak`.** That tier holds one row per *month* and
would answer `1` for every month ever measured — wrong by exactly the amount
that makes it look plausible. The two tables are proven equal on the **peak**
by `recompute_month()`; they were never equal on the **count**.
* **it does not remove the dedupe.** `_seen` is still what stops `cycle()`'s
once-a-second `sync()` re-banking a block that `P1Ingest.blocks` keeps for a
day, and `sync()`'s return value is still the number newly banked. What moved
is only where the *status page* reads its figure. ⚠️ The check that graded the
dedupe used to assert `recorded == 1` — the dedupe and the figure reporting it
were the same variable. Counting rows in `quarter_blocks` cannot see a
re-bank at all (it is a PRIMARY KEY no-op), so that check is now graded on
`sync()`'s **return**, with the store's tally as the second half.
### ⚠️ The unreadable case renders a WORD, not a number
`banked_count()` returns `int | None`; `recorded_in()` renders the `None` as
the module constant `COUNT_UNREADABLE = "an unreadable number of"`, so the
sentence reads
```
... ; an unreadable number of quarter(s) banked
```
`0` was never available. `None` was rejected too: `f"{None} quarter(s) banked"`
renders **"None quarter(s) banked"**, which an operator reads as the same zero.
This is the `PeakReading` trade — no `peak_w_or(default)`, ever — applied to the
coverage figure: a caller that formats it gets a true sentence in both states,
and a caller that does arithmetic on it gets a `TypeError`, which is loud.
"0 quarter(s) banked" remains a **true and reachable** sentence: it is what a
month nothing has been written to says, and part 11 asserts that the two states
are spelled differently.
---
## 2. Decision: at EQUAL wattage, the EARLIEST quarter keeps the attribution
**First, not most recent.** The reasoning, in the order it matters:
1. **It is not a money question.** The utility bills the wattage; the timestamp
is not part of the bill. So the decision is settled entirely on what the
status row is *for*.
2. **The row answers "when did my monthly peak happen".** An operator reading it
goes and looks at one event — what was running, what the weather was. A
stable answer is the one that can be investigated; "most recent wins"
guarantees the answer moves every time the wattage recurs, and an operator
who looked yesterday is sent somewhere else today for no reason.
3. **It is what one reader already did unconditionally.** `recompute_month()`
is `ORDER BY offtake_avg_w DESC, quarter_start ASC LIMIT 1` — earliest wins.
Choosing "most recent" would have meant changing both readers; choosing
"earliest" meant making the other one say out loud what it was already doing
by accident.
### ⚠️ And the two readers DID diverge, which is why this is a code change
The `monthly_peak` upsert was `WHERE excluded.peak_w > monthly_peak.peak_w` —
strictly greater, so an exact tie never wrote and the row kept whichever tied
block was **written first**. That coincides with *earliest* only while blocks
arrive in time order. Fed four equal 365 W blocks newest-first, measured on the
shipped tree:
```
ascending arrival monthly_peak -> 09:00:00 recompute_month -> 09:00:00 agree=True
DESCENDING arrival monthly_peak -> 09:45:00 recompute_month -> 09:00:00 agree=False
```
That is a divergence in the derived record that part 8 of `test_tariff.py`
exists to prove cannot happen — latent, because nothing in the shipped path
banks out of order today. The upsert now carries the tie term explicitly:
```sql
WHERE excluded.peak_w > monthly_peak.peak_w
OR (excluded.peak_w = monthly_peak.peak_w
AND excluded.quarter_start < monthly_peak.quarter_start)
```
**`peak_w` is untouched.** The `>` term is exactly as it was, no tie can move
the billed number, and SAFETY-07's guard reads the same watt figure it read
before. The second term moves only the timestamp.
### ⚠️ AND THE SOAK'S DRIFT WAS NOT A TIE AT ALL
The ticket's reading — "ties at the same wattage are re-attributed to the most
recent matching quarter" — is **false**, and driving it is what showed that.
Six exactly-equal 365.0 W quarters on the shipped tree moved the attribution
**zero** times, because `>` is strict.
What the soak saw was a genuine new maximum **smaller than the row's own
resolution**. The row renders `{peak_w:.0f}` — whole watts — so 365.0000001 W is
a new peak to the database and the same `365 W` to the reader:
```
q0 fed 365.0 -> renders 365 W, attributed 09:00:00
q1 fed 365.0000001 -> renders 365 W, attributed 09:15:00
q2 fed 365.0000002 -> renders 365 W, attributed 09:30:00
```
**Decision: left alone, deliberately.** It is the instrument behaving correctly
— the peak really did rise — and the alternative, printing more decimal places,
puts noise in the one row an operator reads at the moment the guard engages, to
explain a movement that is real. Both cases are pinned in part 12 so that
neither can be changed by accident, and the distinction is recorded here so the
next reader does not re-diagnose it as a tie.
---
## 3. What this branch did NOT do, and who owns it
**`main.py`'s status row was not touched.** TARIFF-11 holds `main.py`'s status
rows on a concurrent branch, and the standing instruction is to stop rather than
race there. Everything above is in `app/tariff.py`, and the row picks the fix up
without editing, because `self.peak.recorded` is an attribute access and is now
a property.
**The residual, stated with its scope.** `recorded` answers for
`datetime.now()`. The peak beside it follows `checks(when=...)`. The shipped
install has **exactly one** call site — `self.checks()` with no argument, at
`main.py:4359` — where the two are the same month, so **the live page is correct
and durable**. They part company only under the `when=` affordance TEST-17 added
for the suite, and the mismatch there is strictly smaller than the one it
replaces (this month's tally, rather than a process-wide total across all
months).
`PeakTracker.recorded_in(when)` is the scoped form and is shipped. Moving the
row onto it is:
```python
- + f"; {self.peak.recorded} quarter(s) banked"
+ + f"; {self.peak.recorded_in(when)} quarter(s) banked"
```
one token, on one line, inside the block TARIFF-11 owns. `test_tariff.py`
part 10 **pins the present behaviour** — graded against a row count taken
straight from `quarter_blocks`, not against the property that produces it — so
the gap goes red if it changes and cannot be quietly forgotten.
@@ -0,0 +1,349 @@
# TARIFF-14 — how long one price point covers, and what the last one does
One defect and one genuinely open decision. The defect had no defensible
alternative once it was measured; the decision — the **last** point's window —
did, and the alternative is argued below rather than dismissed.
---
## 0. The measurement this card exists for
`_Series.at()` asked `s <= u < s + 1 h` of every point **in list order** and
returned the **first** match. Run against the shipped code at `5059e28` on a
quarter-hourly series:
```
series: 02:00=0.10 02:15=0.11 02:30=0.12 02:45=0.13 03:00=0.14 ...
ask + 0min -> 02:00 0.10
ask +15min -> 02:00 0.10 <-- WRONG (wanted 02:15)
ask +30min -> 02:00 0.10 <-- WRONG
ask +45min -> 02:00 0.10 <-- WRONG
ask +60min -> 02:15 0.11 <-- WRONG (wanted 03:00)
ask +75min -> 02:30 0.12 <-- WRONG (wanted 03:15)
```
**Two failure modes, not one.** Inside the first hour the `:00` point wins
because its hour has not run out. From the second hour on the answer is not
even the top of the hour — the first point still inside its own hour wins — so
**the error compounds with depth into the series**.
Same code, same run, after the fix:
```
ask + 0min -> 02:00 0.10 ask +60min -> 03:00 0.14
ask +15min -> 02:15 0.11 ask +75min -> 03:15 0.15
ask +30min -> 02:30 0.12 ask +90min -> 03:30 0.16
ask +45min -> 02:45 0.13 ask+120min -> 04:00 0.18
```
An hourly series is **unchanged at every instant**, before and after — which is
the whole reason nothing caught this. Every price fixture in this repository
before TARIFF-08 was the invented hourly row shape, `static_tariff.py` emits
hourly points, and the DST checks count points on the 23- and 25-hour days
without ever asking `at()` for a quarter-hour. **The suite had been exercising
the one resolution on which the broken rule and the right one agree.**
Belgium moved to a 15-minute MTU on 2025-10-01 and the live day-ahead feed
returns **96 points at 900 s**, so this is reachable only down the dynamic path
TARIFF-08/09 just made work. Static sites are immune.
## 0.1 What it cost, which is not "a slightly wrong price"
`arbitrage._act()` calls `_point_at(plan, now)` → `series.at(now)` and then
tests `here.start == v.cheapest.start` / `== v.dearest.start` to decide whether
this interval is the buy one or the sell one. That is an **identity test
between the matched point and the point the verdict was built from**, and on a
quarter-hourly series it almost never matched — so the strategy idled through
exactly the interval it exists to trade, reporting *"this hour is neither the
buy hour nor the sell hour"*. It also read `here.provisional` (§8.3) and §17's
`price < 0` off that same wrong point.
---
## 1. The rule
> **A point covers from its own start until the next point's start, and never
> more than one hour.**
* **900 s** → each point covers its own quarter.
* **3600 s** → each point covers its own hour. Identical to what shipped.
* **MIXED** → **each point takes its own gap.** FCST-04's carry-forward emits
hourly carried points beside quarter-hourly firm ones, and the soak measured
spacing `[900, 3600]` in **70 % of samples** — so a window derived from one
global spacing would be wrong in the **normal steady state**, not in an edge
case.
### 1.1 Why the one-hour cap is not decoration
FCST-04 **leaves holes on purpose**: an hour with no published price and no
yesterday value is left *out* of the series, and `price_forecast.py`'s module
note names `at()` raising `Absence` for it as the behaviour that sends the
caller to the static profile (§11.4 for that hour).
Uncapped, "until the next point's start" stretches the point **before** a
three-hour hole across the whole hole and answers every instant in it with a
stale price. Filling holes is the defect this module exists to refuse, and it
would have been introduced **by the fix for a different one**. Graded in
`test_price.py` 14d; the mutation that removes the cap is red there.
---
## 2. The decision: what the **last** point covers
There is no next start, and §9.3 says nothing about how wide the final interval
is — a `PricePoint` has **three fields** and must not learn a fourth, because
that is the wire format every store, replay, fixture and captured body in this
repository is written against. So the tail is a **choice**, and both candidates
are defensible.
**Chosen: the last point is given the same span as the point before it** (one
hour when it is the only point).
| | flat one hour | the span in front of it (**chosen**) |
|---|---|---|
| 96-point 900 s day, `at(23:50)` | answers 23:45 | answers 23:45 |
| the same day, `at(00:30)` **tomorrow** | **answers with today's 23:45 price** | `Absence` |
| 24-point hourly day, `at(23:50)` | answers 23:00 | answers 23:00 |
| mixed series ending at the firm/carried seam | answers for an hour | `Absence` after 15 min |
**The second row is why.** A flat one-hour tail reaches across a day boundary
and prices tomorrow morning with yesterday's last print — which is
"reaching for a neighbouring one", the single thing the `Absence` message has
always refused to do. The chosen rule never extends a point past a span the
series itself demonstrates.
### 2.1 The cost of the choice, stated rather than buried
On a **mixed** series whose last point is a carried **hourly** one sitting
behind quarter-hourly firm ones, the tail is **900 s**, and the last 45 minutes
of the horizon raise `Absence` for an hour something did carry a value for.
That is accepted, not tolerated, and the direction is the safe one:
* the caller falls back to the static profile (§11.4) instead of trading on a
span the series never demonstrated;
* the point is **carried**, so §8.3 forbids any irreversible action resting on
it regardless;
* it is at the far end of a 48 h horizon, where nothing is decided at 30 s.
Under-covering costs a missed decision. Over-covering prices a real interval
with a number that does not describe it. Those are not symmetric.
### 2.2 The degenerate case
A series of **one** point has no gap in front of it to copy and gets **one
hour** — the shipped behaviour, kept deliberately. `test_price.py` 14e states
it as its own check so it cannot decay into an accident.
---
## 3. Does `at()` now depend on sortedness? Yes — and it establishes it itself
The old rule read one point at a time and did not care about order. The new one
reads a point's **neighbours**, so it *would* depend on the series being
ascending.
It does not, because **the lookup index is sorted by UTC start at construction**
(`_cover_table()`, called once from `_Series.__init__`). No caller's list order
can reach `at()`.
That matters because the three things that "guarantee" order today each
guarantee it somewhere else:
| | what it guarantees | where |
|---|---|---|
| `_validate_raw()` | strictly increasing starts | the `load_prices()` path only |
| `price_history._parse_points()` | re-sorts by instant | the store read path only |
| FCST-04's fill | emits ascending | that one producer only |
| `PriceRead.series()` | **nothing** | every other caller, including tests |
Three producers agreeing today is not an invariant; it is a coincidence with
good references. `test_price.py` 14f drives a series handed its 96 points in
**reverse** order and asserts `at()` answers identically at every one of them —
red the moment the sort is removed.
**Duplicated starts** (refused by `_validate_raw()`, constructible by hand) give
the earlier list entry a zero-width window and the later one the real span. One
of the pair becomes unreachable through `at()`; **nothing raises**, at
construction or on the 30 s control path. Stated as a check rather than left to
be discovered.
---
## 4. `covers()` shared it. `window()` did not
* **`covers()`** has no window rule of its own — it calls `at()` — so it shared
the defect exactly and the same change fixes it. It is kept as a delegation
for that reason: a second copy of the rule is a second place for it to be
wrong.
* **`window(since, until)`** grades a point's **start** against a half-open
range and never asks how long a point lasts, so it carries no resolution
assumption at all. It is **not** a coverage query — a point that starts before
`since` and runs into it is absent from its result by design. That is now
said in its docstring and pinned by a check, because the next reader will ask.
It has **no callers** in `app/`.
---
## 5. What this card does NOT fix, with its shape
`price_forecast._walk()` — §8.3's irreversible-action gate — iterates
`committed_hours()`, which steps in **whole hours**, and asks `series.at(u)` at
each. On a quarter-hourly series that now reliably returns the hour's `:00`
point (it previously returned something earlier and arbitrary), so the gate
reads **one point per hour** and never looks at the `:15`/`:30`/`:45` flags.
A partially-provisional hour would therefore be gated on its first quarter
alone. **Two things keep it off every shipped install, and both are
circumstantial rather than structural:** `price_permits()`/`begin_phase()` have
**no caller in `app/main.py`** — the §12 price-justified entry point is built
and not yet wired — and FCST-04 carries **whole** hours while
`generic_dayahead` publishes whole days, so the four quarters of an hour are
firm or carried together anyway. Both are facts about the *callers and
producers*, not properties of the *gate*, so this is written down here rather
than left for the first sub-hourly feed or the first wiring-up to find. Fixing it means teaching
`committed_hours()` about resolution, which is a data-model question this card
deliberately does not open.
---
## 6. Mutations
Eight, each purged before the run (TEST-15), each reverted with the tree
fingerprint checked afterwards (MUTATION.md habit 18), backup at a `mktemp`
path per invocation.
⚠️ **Re-driven in full on the MERGED tree** after TARIFF-10 landed on
`release/1.0` (base `88c59a9` -> `559f0eb`). All eight give the same verdicts
and the same red-check counts as before the merge, which is the answer that
matters: TARIFF-10 added 650 lines of checks to `test_price.py` and none of
them moves under any of these edits, because they grade `resolve_tariff()`'s
*reachability from a configuration* and not the lookup. Clean `app/price.py` on
the merged tree = **`90d8ca31f7c0109d`, 85953 B** (pre-merge it was
`30d1a56f0117e571`, 83742 B; the difference is TARIFF-10's `provider_id()` /
`provider_configured()`, which this card does not touch).
| | mutation | Δ | red in |
|---|---|---|---|
| M1 | `at()` back to the shipped fixed one-hour scan over `self.points` | +68 B | **24 named checks**: `test_price.py` 14b ×4, 14c ×3, 14d, 14e ×3, 14f, 14g; `test_arbitrage.py` 10b ×6, 10c ×3, 10d ×2 |
| M2 | drop the one-hour cap | −68 B | `test_price.py` 14d |
| M3 | flat one-hour tail instead of the span in front | **+0 B** | `test_price.py` 14e ×3, 14g — 4 checks |
| M4 | trust the caller's list order — drop the sort | −61 B | `test_price.py` 14f |
| M5 | `covers()` grows its own hour-wide rule | +95 B | `test_price.py` 14e, 14g |
| M6 | `window()` becomes a coverage query | −37 B | `test_price.py` 14g |
| M7 | the window's end becomes inclusive | +1 B | `test_price.py` 14b ×4, 14c ×4, 14d, 14e ×4, 14f ×2, 14g — 16 checks |
| M8 | the window's start becomes exclusive | −1 B | `test_price.py` 14b ×4, 14c ×4, 14d, 14f ×2, 14g, 14h, plus the pre-existing section 6 check — 14 checks |
**M3 is the zero-byte one and it is the tail decision itself** — the case
MUTATION.md names as the risky size, caught by the check written for it.
**What the arbitrage file does NOT see, with sizes.** M2, M4, M5, M6 and M8
leave `test_arbitrage.py` green, and that is honest rather than a gap to round
off: they are `at()`-level properties (holes, caller order, `covers()`,
`window()`, the start boundary) that no arbitrage fixture reaches, because the
probes are seven minutes into a slot and the plans have no holes. The one that
matters end to end — M1, the shipped rule — is red in **both** files, on the
named check `THE PROPERTY THE DEFECT BROKE`.
---
## 7. TARIFF-10 landed on the base while this was open — what it changes here
Base moved `88c59a9` → `559f0eb` mid-card. The question is not whether the
branches merge; it is whether TARIFF-10 makes any sentence above **false**.
**In `app/price.py` the two do not meet.** TARIFF-10 added `provider_id()` and
`provider_configured()` beside `price_option()` and routed `load_prices()`
through them. That is the option-reading layer; it touches nothing in
`_Series`, `at()`, `covers()`, `window()` or `_cover_table()`. Git merged the
file without a conflict, and the merged `at()` is byte-identical to the one the
mutations above were driven against.
**Behaviourally it widens who has a series at all, and that cuts one way only.**
`run_tariff_refresh()` now runs on installs with `arbitrage_enabled` **off**, so
a provider-configured site has a standing plan — and therefore a `_Series` —
where before it had none. **It does not add a new `at()` consumer to the 30 s
tick**, which is the claim worth checking rather than assuming:
| caller of `at()`/`covers()` | reached by a non-trading install? |
|---|---|
| `arbitrage._point_at()` (`arbitrage.py:300`) | no — trading only |
| `static_tariff.TariffPlan.price_at()` / `.words` `here` (`:750`, `:819`) | on the tick these are reached through `static_status()`, which builds a **`static_plan`** — hourly points, immune. `main.py` calls only `plan.words` on the fetched plan, which counts points and never looks one up |
| `price_forecast._walk()` (`:562`, `:568`) | **`price_permits()` and `begin_phase()` have no caller in `app/main.py` at all** — §12's price-justified entry point is built and not yet wired |
So TARIFF-10 neither creates nor closes any exposure this card is about. What it
does do is make the defect's reach larger the moment such a site arms trading,
because the series it would trade against now exists on every configured
install rather than only on armed ones. That strengthens the case for the fix
and changes nothing in its reasoning.
**In `test_price.py` both cards appended a section 13.** Nothing is dropped:
TARIFF-10's stays **13**, this card's is renumbered **14** (which is also this
card's number), and the temp-directory teardown TARIFF-10 put at the end of the
file stays at the end of the file. The two sections share no fixture, no option
dict and no helper name; TARIFF-10's grades `resolve_tariff()`'s *reachability
from a configuration* and drives real fetches with a counting transport, and
**none of its checks moves under any of the eight mutations above** — measured,
not assumed.
---
## 8. TARIFF-12 landed too — a plan may now be 90 minutes old when it is asked
Base moved `559f0eb` → `a6e64fb` while this was open. TARIFF-12 stops a
returned §11.4 degradation from overwriting a standing plan that is still
inside `TARIFF_MAX_AGE_S`, so **a served plan can now be up to
`3 × TARIFF_REFRESH_S = 5400 s` old** where before it was at most one refresh
interval. `at()` is what answers *"what does it cost right now"* out of that
older plan, so the question is fair and the answer is written out rather than
assumed.
**Per-point span: unaffected, structurally.** Every window is derived from the
neighbouring points' **absolute `start` instants**. Those are `datetime`s inside
a frozen dataclass inside a tuple — they cannot move, and `_cover_table()` runs
once at construction over data that cannot change afterwards. A plan's *age* is
a property of `fetched_at` / `tariff_at`, never of a window. **The index cannot
go stale**, which is a small positive under TARIFF-12 rather than a risk: a kept
plan now lives longer, and there is nothing in it that needed recomputing.
**The one-hour cap: unaffected.** It bites only where a gap exceeds an hour,
i.e. at a hole FCST-04 left. Walking further into a series crosses more windows;
each is still resolved on its own neighbours.
**The tail: reachable for the first time, and the choice made in §2 is the one
that survives it.** This is the half worth spelling out.
* At the **shipped 48 h horizon** the tail is unreachable by staleness. A plan
is served for at most 90 minutes, so `now` is at most 90 minutes into a
~48 h series — the last point is still about 46½ hours away.
* But `price_horizon_h` is `int(1,72)` in `config.yaml`, so an operator **can**
configure a **1 h** horizon. A 90-minute serve-stale window against a 1 h
series puts `now` genuinely past the end. That is not hypothetical; it is a
supported configuration meeting a supported outage.
* Under the **chosen** rule a quarter-hourly 1 h series ends at `:45` and that
point's window closes at the top of the next hour: `at()` raises `Absence`,
`_point_at()` returns None, `_act()` idles with its "no price for this
instant" reason, and §11.4 sends the site to the static profile.
* Under the **rejected** flat one-hour tail the same stale plan would have gone
on trading against the `:45` print for **up to 45 minutes after the interval
it describes had ended** — pricing an outage with a number for a quarter that
is over.
So TARIFF-12 does not weaken the tail decision. It is the first thing that makes
it reachable on a shipped configuration, and it independently selects the same
answer. ⚠️ **Note which direction that runs**: the more stale a plan is allowed
to be, the more the tail rule is doing real work, so it should be re-read if
`TARIFF_MAX_AGE_S` is ever widened again.
**`TariffPlan.from_feed` cannot produce a series this code has not seen.** It is
a pure predicate on `origin` (`== "provider"`), and TARIFF-12's decision keeps or
replaces a **whole plan object**. It never splices two reads into one series, so
it cannot hand `at()` an unsorted or duplicated point list — the case §3 covers
defensively and that nothing shipped produces.
**Re-graded on this base.** All 29 files green. `app/price.py` is byte-identical
to the TARIFF-10 merge (`90d8ca31f7c0109d`, 85953 B) because TARIFF-12 does not
touch it. **M1** re-driven: +68 B, red on the same **24 named checks** across
both files. **M3** re-driven: **+0 B**, red on the same 4. This branch
contributes **zero diff** to `app/main.py` and `app/static_tariff.py`.
@@ -0,0 +1,166 @@
# TARIFF-15 — the operator's own tariff card, and why the card has to grade it
**Status: DECIDED 2026-09-13 — ship `custom_formula` with FIXED coefficients and
a REQUIRED `card` block whose printed price the add-on reproduces at startup or
refuses to run. No expression evaluation, ever. Declare `{ENERGY, MARKUP}`,
never `ALL_IN`.**
Filed with TARIFF-15. Follows [[TARIFF-07]], which withdrew three per-supplier
plugins and named a formula provider as the answer it could not give.
---
## 1. Why a formula rather than more plugins
TARIFF-07 established the negative half: a per-supplier plugin has no source
for its numbers. This card establishes the positive one.
* **There is no machine-readable source of Belgian tariff coefficients
anywhere.** CREG's consumer tariff-card pages 404. CREG Scan is a
JavaScript-only wizard. The annual report carries the supplier list and no
per-product coefficients. VREG was unreachable. Every coefficient found in
research came from a supplier's **own PDF**, one of them image-only.
* So there is nothing to scrape, and fourteen plugins would still have no
source for their numbers — they would be fourteen copies of the invention
TARIFF-07 deleted three of.
* **The customer is the only party who can see their own coefficients**, and
the card they read them from **is the only artefact that can grade them**.
That second clause is the whole design; everything below follows from it.
## 2. The shape is the market's published arithmetic
Both suppliers whose 2026 cards were readable print three things on one line:
the formula, the reference index it assumes, **and the resulting price**.
| | computed | the card prints |
|---|---|---|
| Eneco `(0.102 × 133.9905 + 1) × 1.06` | **15.5471** | **15,55** |
| ENGIE `(0.1 × 61.96 + 0.9) × 1.06` | **7.5218** | **7,522** |
Two independent suppliers, reproduced to the last printed digit, through the
shipped code (`test_price.py` 15a runs both, and then runs both again end to
end through `load_prices()`).
That third column is not a nicety. It is the only externally-published number
in this whole feature, and it is what makes the design safe.
## 3. The double-counting risk, and which half of it is new
`price.py`'s trap 1 already mechanises that a **plugin** cannot double-count:
`PluginOptions` refuses the framework keys, `code_leaks()` refuses a plugin
whose *code* mentions them, `assemble()` adds the complement exactly once.
`custom_formula` introduces a risk none of those can see, **because it is not
in code**: the **operator** double-counting, by folding their network charges
into `offset` while `price_taxes_eur_kwh` also carries them. `code_leaks()`
scans code. This is config, typed by a person reading a PDF.
It is a real reading of a real card, not a hypothetical. A Belgian card's
`Tariefformule` line is the **energy component alone** — Eneco tabulates
network tariffs in one table and levies in a second; ENGIE states in so many
words that the formula excludes network, distribution, transport, grid levies
and excise. An operator who wants "the price I actually pay" and adds ~12
c€/kWh to `offset` has done something entirely reasonable and is now billed for
network charges twice, on every hour, with §11.4 making the whole class of
price defect invisible.
**Decision: the card grades it.** At startup `validate.py` recomputes the
card's own printed price from the operator's coefficients and refuses to start
unless the two agree to within `0.01` — which is the last digit a card prints.
An inflated `offset` cannot pass, because the printed estimate is by
construction the energy component: 15.55 becomes 28.27 and the add-on stops and
says why.
**And the limit, stated in the same sentence as the claim**, exactly as
`price.py` states trap 1's: *it grades the coefficients against the card, not
the card against reality.* A consistent-but-wrong pair — both numbers off the
wrong product, or off a card the contract has since rolled over from — agrees
with itself perfectly and passes. Nothing here or anywhere else can catch that.
### 3b. The one unit error the card check cannot see
A factor applied to **both** sides — coefficients *and* printed price — cancels
out of a comparison between them. That is a second quantity, so it is a second
refusal: the card's own point must land inside 0.02–1.00 EUR/kWh.
⚠️ **Its floor is applied to the ENERGY figure and not to the all-in one, and
that asymmetry was found by a mutation rather than by reasoning.** The shipped
`price_taxes_eur_kwh` was 0.06 — three times the floor — so a card read a
hundred times too small comes out at 0.0016 EUR/kWh of energy, is carried over
0.02 by the flat tax addend alone, and passes a band applied to the total. The
hundredfold error is exactly what the band exists for. Graded from both
directions in `test_price.py` 15e/15i and driven red as M10. *(Annotated
2026-09-13: TARIFF-18 raised the default to 0.11736, nearly six times the
floor. The asymmetry is unchanged in kind and larger in degree — the flat
addend that carries a hundredfold-small card over 0.02 got bigger.)*
## 4. Fixed coefficients. No expression evaluation of any kind
**Decided: never a grammar.** Two arguments, and the second is the one that
decides it.
* **Security.** The add-on serves `POST config/save` → `POST
/addons/self/options` on its own ingress page, so `price_formula` is
**settable over HTTP**. `eval` on it is remote code execution in a root
container holding `SUPERVISOR_TOKEN`, with `homeassistant_api` and
`hassio_api`, driving an inverter over RS485. `eval(s, {"__builtins__": {}})`
is not a sandbox and this repository is not going to pretend otherwise.
* **⚠️ AND THE DECIDING ARGUMENT IS NOT SECURITY: AN EXPRESSION IS
UNVERIFIABLE.** Two coefficients can be graded against the card's printed
estimate. An arbitrary expression cannot be graded against **anything** — so
it makes the invisible-wrong-coefficient problem *strictly worse*, which is
the exact failure this provider exists to prevent. An expression would buy
the operator freedom to be wrong in a way nothing could report.
All 24 surveyed Belgian products, both injection legs and aWATTar AT fit
`B × index + A`, so an expression buys nothing the market needs either.
**Future shapes become NAMED FIELDS** — `clamp_lo`, `factor_negative`,
`offset_bands` — each individually refusable and each individually gradeable
against a card. `parse_price_formula()` therefore **refuses an unknown key**
rather than ignoring it: an operator who types `"expr"` is told it does not
exist, instead of being priced on the fields they also typed with the one they
believed in dropped in silence.
`test_price.py` 15f walks the module's AST for `eval`, `exec`, `compile`,
`__import__` and `literal_eval`, and exercises the scanner's **red** direction
on every run so it cannot become a scanner that scans nothing.
## 5. `{ENERGY, MARKUP}`, not `ALL_IN`
The contract's formula **is** the supplier's markup — that is what `B` and `A`
are — but it is not the levies. Declaring `ALL_IN` would lie on behalf of every
operator who typed exactly what their card says and would drop
`price_taxes_eur_kwh` from every hour: measured at 0.2125 → 0.1555 EUR/kWh.
`price_markup_eur_kwh` is therefore **ignored** under this provider, and logged
as ignored, which is the framework's existing rule rather than a new one.
## 6. What was reused rather than rebuilt
The index path is `generic_dayahead`'s: same EPEX BE feed, same
Brussels-calendar range, same columnar parser, same declared egress hosts. The
override/range decision moved into one shared `dayahead_feed_url()` that both
plugins call, because two copies of that rule would be two places for the
short-horizon defect TARIFF-09 fixed to come back in only one of them.
`resolution: hourly` averages each hour's members. ⚠️ **That is not
`_Series.at()`'s hourly assumption returning (TARIFF-14):** it decides how many
points are *emitted*; the window each point covers is still derived from its
neighbours in `_cover_table()`. A feed that is already hourly averages one
member per hour — the identity — so the option cannot silently assume the
granularity it was handed. Graded.
## 7. What this card did NOT do
* **The injection leg.** Eneco uses `0,102` on consumption and `0,100` on
injection, and injection is VAT-exempt where consumption is at 6 % — a second
per-contract formula, not a second copy of this one. Separate card.
* **The hour-varying network charge that bites Wallonia.** Separate card.
* **Raising `price_taxes_eur_kwh` from 0.06 to ~0.12.** Separate card, and an
owner's decision. Note §3b: the band's floor is deliberately insensitive to
whatever that number becomes. *(Annotated 2026-09-13: that card is
**TARIFF-18** and it shipped on this branch at **0.11736** — the Eneco
residential card for Flanders, September 2026, incl. 6 % VAT, summed from
Fluvius Antwerpen distribution 5,35 + bijzondere accijns 4,876 + bijdrage
groene stroom en WKK 1,51 c€/kWh. §3b held: the floor is still on the energy
figure and nothing in this module needed a new number.)*
+691 -17
View File
@@ -116,7 +116,7 @@ whole rendered list, in both languages.
| | |
|---|---|
| **All 71 options are always shown** | The frontend keeps a field when `name in options \|\| required`, and every key is in `options:`. There is no "advanced" fold |
| **All 72 options are always shown** | The frontend keeps a field when `name in options \|\| required`, and every key is in `options:`. There is no "advanced" fold |
| **Order comes from `schema:`** | `App.schema_ui` is `UiOptions(...)(data["schema"])`, so the **schema half** of `config.yaml` decides the order the tab renders in — not the `options:` half |
| **There are no section headers** | Add-on options are a flat list: no header primitive, no per-option advanced flag, no units on the input, no sliders. Grouping is approximated by **order** plus the `Group · ` prefix in each label, and by nothing else |
| **Groups, in task order** | Hardware → Meter → Limits → Tariff → Maintenance → Safety → Advanced |
@@ -1346,24 +1346,39 @@ does, and only after you have read why.
> release ships the five provider plugins, the canonical all-in format they
> emit, the config that selects one, and — new in OPT-04 — a price-arbitrage
> strategy that trades on them. With `arbitrage_enabled: false`, which is the
> shipped default, turning a provider on still fetches, validates and publishes
> prices and changes no charging decision.
> shipped default, **selecting a provider is what starts the fetch**: the feed
> is read every 30 minutes, validated, and the day recorded in
> `/data/price_history.json`, which the *Price history* status row reports and
> the add-on log names at every refresh. No charging decision changes.
>
> ⚠️ **That has been the documented behaviour since OPT-04 and was not the
> actual behaviour until TARIFF-10.** The background refresh was started only
> when `arbitrage_enabled` was on — and it ships `false` — so an install with a
> provider configured fetched nothing at all, never created the store, and read
> *Price history: empty* for as long as it ran. If you configured a provider and saw an empty
> Price history row, that is what you were looking at. Trading is unchanged and
> is still `arbitrage_enabled` alone.
>
> `static_profile` below needs no network at all, so its cheap window is
> computed on every cycle and reaches the maintenance cycle's grid top-up
> whether or not arbitrage is armed.
> whether or not arbitrage is armed. ⚠️ **And it is still the only source of
> that window.** A provider-backed plan now exists on every configured site,
> but *Cheap window* and the maintenance cycle's grid top-up read
> `static_profile` and nothing else — a deliberate limit, recorded with what it
> would take to change in `DECISIONS/TARIFF-10-fetching-is-not-trading.md`.
| option | meaning |
|---|---|
| `price_provider` | `off` (default) or one of the five plugins below |
| `price_provider` | `off` (default) or one of the three plugins below |
| `price_markup_eur_kwh` | Your supplier's margin, EUR/kWh. **Added only when the chosen plugin's feed does not already contain it** |
| `price_taxes_eur_kwh` | Levies and VAT, EUR/kWh. Same rule |
| `price_taxes_eur_kwh` | Distribution, levies and VAT, EUR/kWh. Same rule. Defaults to **0.11736** — a real Flemish figure, and **yours is probably different**: see *What `price_taxes_eur_kwh` is made of* below |
| `export_price_eur_kwh` | The **injection** price. A separate scalar, not a series — see below |
| `price_csv_path` | Where `manual_csv` reads its file |
| `price_feed_url` | Override the plugin's default feed. Only a host the plugin itself declares is accepted |
| `price_feed_url` | Override the plugin's default feed. Used **exactly as written** — the add-on appends nothing to it, including the date range, so an override gets whatever horizon that URL returns. Only a host the plugin itself declares is accepted |
| `price_horizon_h` | How far ahead to ask for, hours |
| `price_timeout_s` | Per-request timeout |
| `static_profile` | A static day/night or fully-fixed tariff. Empty by default — see below |
| `price_formula` | For `custom_formula` only: your own contract's two coefficients, off your own tariff card. Empty by default — see below |
**Which plugin sources what.** This is the one thing to get right, because
adding your markup to a price that already contains it makes every future
@@ -1372,11 +1387,86 @@ arbitrage decision wrong by that amount, in a direction nothing reports.
| plugin | product | the published figure contains | the add-on adds |
|---|---|---|---|
| `generic_dayahead` | day-ahead | energy only (wholesale EPEX BE) | markup **and** taxes |
| `engie_dynamic` | day-ahead | energy + Engie's markup | taxes |
| `totalenergies_dynamic` | day-ahead | energy + TotalEnergies' markup | taxes |
| `eneco_eddy` | **real-time / imbalance** | everything, all-in | nothing |
| `custom_formula` | day-ahead | energy **and** your contract's markup — your card's `Tariefformule` line | taxes |
| `manual_csv` | day-ahead | everything, all-in | nothing |
⚠️ **Three plugins were withdrawn in TARIFF-07, and if your configuration names
one of them, read this.** `engie_dynamic`, `totalenergies_dynamic` and
`eneco_eddy` are gone. They did not stop working — **they never worked.** Each
pointed at an endpoint that does not exist and never did:
| id | its endpoint | what that endpoint actually answers |
|---|---|---|
| `engie_dynamic` | `https://api.engie.be/dynamic/be` | **404**, gateway error `no Route matched with those values` |
| `totalenergies_dynamic` | `https://api.totalenergies.be/prices/dynamic` | **TLS failure** — that hostname is not in the server's certificate |
| `eneco_eddy` | `https://api.eneco.be/eddy/prices` | **404**, empty body |
Every fetch any of them ever made failed, and a failed fetch falls back to the
static tariff **silently** — so a site set to one of these has been running on
`static_profile`, or on nothing at all, since the day it was configured, with
nothing on the status page saying so. The add-on now **refuses to start** on one
of these ids and tells you all of the above by name. That is deliberate: a
silent fallback is what hid this for as long as it lasted.
There is no automatic migration and there cannot be one. These were *retail*
tariffs of the form `B x spot + A` with **per-contract** coefficients that live
behind your supplier's customer login; there is no public index to read them
from. Any alias would have to invent the numbers your battery is priced against.
(`eneco_eddy` was doubly wrong: **"Eddy" is not an Eneco product at all** —
Eneco's dynamic product is *Zon & Wind Dynamisch*, and it is indexed to
Belpex/EPEX **day-ahead**, not to imbalance.)
#### So what should a Belgian dynamic-tariff customer do today?
**Honestly: `generic_dayahead` gets you close, and you should know exactly how
close.** It reads the wholesale EPEX BE day-ahead curve and adds your
`price_markup_eur_kwh` and `price_taxes_eur_kwh` as **flat** scalars. Your real
Belgian dynamic contract is a **formula**:
```
consumption: price = B x spot + A <- B and A are in YOUR contract
```
`generic_dayahead` cannot express that. Two gaps, and the second is the
harder one — **both are closed by `custom_formula`**, which is what the
rest of this section is about:
* **No multiplier.** `B` is typically slightly above 1 (it scales the wholesale
price before anything is added). A flat markup cannot scale anything, so the
*shape* of your curve is right but its *amplitude* is not — the peaks and
troughs the optimiser trades against are compressed relative to your bill.
* **No negative offset.** `A` is frequently **negative** on injection and can be
on consumption. `price_markup_eur_kwh` is `float(0,1)` in the schema — it
**cannot hold a negative number at all**, so that contract shape is currently
inexpressible rather than merely approximate.
What that means in practice: arbitrage decisions made on `generic_dayahead` are
made against a curve that is *correlated* with your bill, not equal to it. For
deciding **which hours** are cheap that is usually good enough; for deciding
**whether a cycle pays for itself** — which is what `cycle_cost_threshold_eur`
compares against — it is an approximation, and one that errs by however far `B`
is from 1.
**`custom_formula` is the answer to exactly this, and it shipped in TARIFF-15.**
You type your contract's two coefficients once, off your own tariff card, and
the add-on evaluates `B x index + A` against the same EPEX BE day-ahead curve
`generic_dayahead` reads. Both gaps above close: `B` is a real multiplier, and
`A` may be negative. It is documented in full in **[Your own tariff formula
(`custom_formula`)](#your-own-tariff-formula-custom_formula)** below.
So there are three honest options rather than two:
1. **`custom_formula`** — your contract's own arithmetic, if you can read the
two coefficients off your tariff card. This is the one to use.
2. **`generic_dayahead`** with your markup and taxes set as well as you can —
accepting the approximation above; or
3. **`manual_csv`**, where you compute the all-in prices yourself from your own
contract's formula and hand the add-on the answers. Nothing is approximated,
but nothing is automatic either.
If you are on a **fixed** or **day/night** tariff — most Belgian households —
neither of those applies: use `static_profile` and leave `price_provider: off`.
You cannot get this wrong by editing the markup: each plugin *declares* what its
feed contains and the add-on adds only the rest. Set a markup under a plugin
that already carries one and it is **ignored, with a line in the log saying so**
@@ -1390,12 +1480,225 @@ whether a plugin described its supplier's feed truthfully — that is a claim
about somebody else's tariff, and no code can verify it. If you write a plugin,
the `sources` declaration is the part a reviewer has to read.
`eneco_eddy` is a real-time/imbalance product rather than a day-ahead one. It is
flagged as high-volatility, which widens arbitrage thresholds once the
optimiser lands. That comes from the plugin's own declaration, not from its
name.
**Real-time / imbalance products still have a place in the machinery, but no
plugin declares one.** A plugin that declares `PriceProduct.REALTIME` is flagged
high-volatility and gets widened arbitrage thresholds; that comes from the
plugin's own declaration, never from its name. All three shipped plugins are
day-ahead, so nothing exercises that path on a real install today — it is kept
on purpose so the first genuine imbalance product does not have to rebuild it.
See `DECISIONS/TARIFF-07-withdrawing-three-plugins.md`.
**`manual_csv` — the offline fallback.** It needs no internet, no supplier
**`generic_dayahead` and the shape of the public feed.** It reads Fraunhofer
ISE's Energy-Charts API (`https://api.energy-charts.info/price?bzn=BE`, no key
and no registration), which publishes a *columnar* document — parallel
`unix_seconds` and `price` arrays, in EUR/MWh, with the unit and the licence
stated in the response. Two things follow that are worth knowing before you
point `price_feed_url` somewhere else:
* The timestamps are **UTC instants**. They are rendered at the Brussels offset
that applies to each one, so the hour that happens twice every October is two
points rather than one overwritten one.
* Belgium moved to a **15-minute market time unit** on 2025-10-01, and the feed
followed. A published day is **96 points, not 24**. Nothing in the add-on
assumes either spacing, and nothing aggregates the quarters away.
* The request **names the days it wants**, derived from the horizon on the
**Brussels** calendar — `?bzn=BE&start=2026-09-09&end=2026-09-10`, two
inclusive local dates. Without that the feed answers a single day, and
everything past today would be carried forward from yesterday's profile and
flagged provisional — which §8.3 excludes from arbitrage, so tomorrow would
never be tradeable at any hour. The calendar is Brussels rather than UTC
because 22:30 UTC in summer is already tomorrow in Belgium.
**What you get before the auction publishes.** The day-ahead auction clears at
about 13:00–14:00 CET. Ask for tomorrow before that and the feed returns fewer
days than requested — that is normal, not a failure. The add-on keeps the
series of what exists and fills the remaining hours from yesterday's profile,
flagged provisional; arbitrage runs on the firm hours and ignores the flagged
ones. So a dynamic site's plan is part firm and part carried every morning and
fully firm once tomorrow is published, and neither state is an error.
⚠️ **If you set `price_feed_url`, you keep the short horizon.** The override is
sent exactly as you wrote it and the add-on adds no `start`/`end` to it, because
those are this feed's parameter names and an override may point somewhere with
different ones — rewriting a URL you typed is how a request nobody intended gets
made. The consequence is a **`WARNING` in the log on every price refresh**,
naming the range that would otherwise have been requested. If you only wanted a
different host and not a different horizon, the fix is to clear the option.
If the arrays ever come back different lengths, or the unit stops saying
EUR/MWh, the fetch is refused outright rather than trusted in part: a price
series missing its last hours is a series missing the evening peak, and it
would be traded on as though it were complete. A refused fetch is the §11.4
row — static profile, provisional plan, no arbitrage — and the reason is in the
log.
**A refused fetch does not take the plan you already have.** The last plan that
actually came from the feed keeps standing until it is 90 minutes old (three
missed refreshes), and only then does the §11.4 row above take over. The
add-on logs a `WARNING` naming the failure, the fetch it is keeping, its age
and the moment it retires; the plan's own age is **not** reset by the failed
refresh, so a kept plan still expires on schedule and is never presented as a
current one. One failed fetch is a blip, not a lost tariff.
#### Your own tariff formula (`custom_formula`)
**What it is for.** A Belgian dynamic contract prices energy as a formula —
your supplier's tariff card prints it as `Tariefformule` — and the two
coefficients in it belong to *your contract*. There is no machine-readable
source for them anywhere: CREG's consumer tariff-card pages return 404, CREG
Scan is a JavaScript wizard, the annual report lists suppliers without
per-product coefficients, and VREG was unreachable. Every coefficient found in
research came off a supplier's own PDF, one of them image-only. **You are the
only party who can see your own coefficients**, so you type them, once.
Set `price_provider: custom_formula` and fill in `price_formula` with one line
of JSON:
```json
{"index": "epex_be", "resolution": "hourly",
"factor": 0.102, "offset": 1.00,
"units": "c_eur_per_kwh", "vat_pct": 6,
"card": {"month": "2026-09", "index_eur_per_mwh": 133.9905,
"expect_incl_vat": 15.55}}
```
The contract it expresses is:
```
price = (factor x index_eur_per_mwh + offset) x (1 + vat_pct / 100)
```
expressed in `units`, and **converted to EUR/kWh by the add-on**. You type what
your card prints, in the card's own units; every unit conversion and the VAT
multiplication are the add-on's job, because those are exactly the two steps
where a human silently loses a factor of 10, 100 or 1.06.
| field | what it is |
|---|---|
| `index` | the reference series. `epex_be` — the EPEX BE day-ahead curve in EUR/MWh, the same feed `generic_dayahead` reads |
| `resolution` | `quarter_hourly` (the 96 points the feed publishes) or `hourly` (each hour's quarters averaged into one point). Your card says which series your contract is written against |
| `factor` | `B` — the multiplier on the index. Must be a number, and positive |
| `offset` | `A` — the fixed part, in the same units. **May be negative** |
| `units` | `c_eur_per_kwh`, `eur_per_kwh` or `eur_per_mwh`. **Required, with no default** |
| `vat_pct` | the VAT percentage. `6` for Belgian residential electricity — not `0.06` and not `106`. `0` is legitimate |
| `card` | the month the card is dated, the reference index it assumed, and the price it prints. **Required** — see below |
##### A worked example, from a real card
Eneco's September 2026 card prints the formula, the reference index it assumed
*and* the resulting price. With `factor: 0.102`, `offset: 1.00`, `vat_pct: 6`
and the card's own index of `133.9905` EUR/MWh:
```
(0.102 x 133.9905 + 1.00) x 1.06 = 15.5471 c€/kWh the card prints 15,55
```
ENGIE's card, in the same shape with its own numbers:
```
(0.100 x 61.96 + 0.90) x 1.06 = 7.5218 c€/kWh the card prints 7,522
```
Both reproduce to the last digit the card prints, through the shipped code —
`test_price.py` section 15a runs exactly these two.
##### ⚠️ The `card` block is required, and it is the safety mechanism
Because your card prints the index it assumed **and** the price that comes out,
the add-on can check your coefficients against your supplier's own worked
example. At startup it recomputes that printed price from the numbers you
typed and **refuses to start** unless the two agree to within `0.01` — naming
the discrepancy, the ratio, and the likely causes.
That catches, in one check: VAT left out, VAT applied twice, a transposed
digit, and a **unit mismatch** between your coefficients and the price you
typed beside them (a factor of 10, 100 or 1000).
And it catches the expensive one. **Your card's formula line is the energy
component only.** Eneco tabulates network tariffs in one table and levies in a
second; ENGIE states that the formula excludes network, distribution,
transport, grid levies and excise. The add-on therefore declares that
`custom_formula` supplies *energy and markup*, and adds `price_taxes_eur_kwh`
on top — once. If you fold your network charges into `offset`, you are billed
for them twice, and nothing on the status page would ever show it. It cannot
happen: ~12 c€/kWh of network charge moves the computed 15.55 to about 27, and
the add-on refuses to start and says so.
⚠️ **And the limit of that check, in the same breath as the claim: it grades
your coefficients against your card, not your card against reality.** Two
numbers copied off the wrong product — or off a card your contract has since
rolled over from — agree with each other perfectly and pass. Nothing can catch
that. What is mechanised is that your coefficients and your card's printed
price *agree*; whether the card is the right card is yours to know. The add-on
warns (and does not refuse) when `card.month` is more than about six months
old, because ENGIE freezes coefficients for a contract's duration and an old
card is often exactly right.
One more check runs beside it: the card's own point must land inside the range
a Belgian retail tariff can occupy (0.02–1.00 EUR/kWh). That exists because a
unit error applied to **both** sides at once cancels out of the comparison
above and cannot cancel out of this one. It bounds the card's own point, **not
every hour** — a live hour may legitimately fall outside it, and a negative
day-ahead hour, which is the best charging opportunity of the year, is never
refused.
##### ⚠️ Fixed coefficients. No expression is evaluated, ever
`price_formula` takes numbers. There is no field that accepts an expression,
and an unrecognised field is **refused** rather than ignored, so typing
`"expr": "..."` tells you it does not exist instead of quietly pricing you on
the other fields.
Two reasons, and the second is the decisive one:
* This string is settable over HTTP — the add-on's own ingress configuration
page posts it to the Supervisor — so evaluating it would be remote code
execution inside a root container that holds a Supervisor token and drives
an inverter. (`eval` with an emptied `__builtins__` is not a sandbox.)
* **An expression could not be graded against anything.** Two coefficients can
be checked against your card's printed price; an arbitrary expression
cannot, so it would make the invisible-wrong-coefficient problem strictly
worse — which is the exact failure this provider exists to prevent.
All 24 surveyed Belgian products, both injection legs and aWATTar AT fit
`B x index + A`, so an expression would buy nothing the market needs. If a
future contract needs a different shape, it gets a **named field**
(`clamp_lo`, `factor_negative`, `offset_bands`) that can be refused on its
own. Never a grammar.
##### What you see at startup
The first time the feed answers, the add-on logs a preview of the next 24
hours as `index EUR/MWh -> all-in EUR/kWh`, followed by the minimum, median,
maximum and the day's spread. It is there so that a formula off by a factor of
1000 is visible in one line rather than in a month of bills:
```
custom_formula preview - the next 24 h of YOUR tariff, index EUR/MWh -> all-in
EUR/kWh, computed as (0.102 x index + 1) x (1 + 6/100) c_eur_per_kwh, converted
to EUR/kWh and with price_taxes_eur_kwh added by the framework:
2026-09-10T00:00:00+02:00 216.080 -> 0.30423
2026-09-10T00:15:00+02:00 216.080 -> 0.30423
... 72 further point(s) not listed; the figures below cover ALL 96
min 0.17277 / median 0.28031 / max 0.67499 EUR/kWh, spread 0.50222 EUR/kWh
over 96 point(s). ⚠️ If those are a thousand times what your bill says,
`units` is wrong; if the spread is 0, `factor` is.
```
(That transcript is the shipped code run against the committed
`scenarios/energy-charts-be-price-2026-09-10-quarterhourly.json` capture with
the default markup and taxes, not an illustration.)
##### What it does not do
The **injection** leg is a separate card. Eneco uses `0,102` on consumption and
`0,100` on injection, and injection is VAT-exempt where consumption is at 6 %,
so it is a second per-contract formula rather than a second copy of this one.
`export_price_eur_kwh` is still the scalar you enter by hand.
#### `manual_csv` — the offline fallback
It needs no internet, no supplier
account and no HTTP stack at all, and it is the supported way to run this
add-on price-aware on a site with no outbound connectivity. Write a file and
point `price_csv_path` at it:
@@ -1424,10 +1727,141 @@ distinct state from "prices, all of them zero": the plan is marked provisional
and arbitrage is disabled, per the degradation matrix. It never falls back to a
price of zero, which would read as free electricity.
#### The hours the feed has not published yet (FCST-04)
A day-ahead auction publishes tomorrow's prices once a day, at **14–16 h CET**.
Before that, the feed is serving today and nothing else — so a `price_horizon_h`
of 48 asks for 48 hours and gets back the ten or fifteen that are left in the
day. Until FCST-04 those other hours were simply **absent** from the plan: not
priced badly, not flagged, just missing, and nothing said so.
Now the add-on keeps the **published prices of the last few days** in
`/data/price_history.json` and fills an unpublished hour of the horizon with
yesterday's price for the same Brussels wall-clock hour, flagged
`provisional: true`. Two things follow, and the second one matters more:
- **The plan says it is guessing** — `provisional: true` on every carried hour,
which is what the optimiser reads. Before, a half-length plan looked
completely settled.
- **Nothing may trade on a carried hour.** §8.3 forbids acting on a provisional
price, so arbitrage drops those hours from the trade set entirely — it needs a
buy hour and a sell hour that were *both* published. Carrying a price forward
widens what the optimiser can **see**; it never widens what it may **do**.
**"An unpublished hour", not "every unpublished hour"** — the fill can only
carry an hour yesterday actually published. If yesterday's record is itself
partial (a first day, an outage, a feed that only ever serves the rest of the
day), the local hours it never held are **left out of the series**: §11.4
applies to those hours, and asking the plan for one raises rather than
answering. An hour with no price is not a free hour, and the fill does not
invent a neighbour, a mean or a zero for it.
**A quarter-hourly feed keeps all four of its prints.** Belgian day-ahead
settles per quarter — the 15-minute MTU took effect on 2025-10-01, so
`generic_dayahead` reads 96 points a day — and the fill decides *which whole
hours need carrying*, never what a published hour may contain. Every point the provider published inside the
horizon is passed through untouched; a carried point is added only for a whole
hour the feed did not print at all.
**What you can actually see on the status page.** *Price history* (below) says
what is held. *Arbitrage* carries the strategy's own sentence, which names the
firm/provisional split whenever that split is what refused a trade. And
**Tariff** on the status page reads `provisional` through the morning, because
since TARIFF-11 that row carries the provider plan itself:
```
Tariff: no static profile (recomputed this tick); provider plan fetched 412 s
ago for a 48 h horizon: dynamic_dayahead from generic_dayahead,
60 points over 33 h, 36 firm / 24 carried, 900+3600 s spacing,
provisional
```
⚠️ **The two halves of that row are not computed at the same moment, and the
row says so.** Everything before the semicolon is the static-profile or Home
Assistant price-entity sentence, regenerated on the **30 s tick** from options
alone — no network. Everything after it is the **background refresh's**
resolution, which runs every 1800 s and is served for up to 5400 s before it is
retired; `fetched N s ago` is the age of the fetch that produced it and is the
only figure in the row that tells you whether the feed is still answering. On a
site with a provider configured and no usable plan the row says that too,
rather than falling silently back to the static half.
Read the plan half as five separate facts: which *product* the feed publishes
and which *plugin* read it, how many **points** it carries and how many **whole
hours** those points cover, how many of the points are **firm** (published for
their own interval) against **carried** (filled from yesterday under §8.3), the
**spacing** of the points in seconds — `900+3600` is the normal mixed steady
state, quarter-hourly firm points beside hourly carried ones — and `provisional`
whenever any point is carried.
⚠️ **Points and hours are two different numbers and this row used to print one
of them under the other's name.** `words` was `f"… {len(points)} h"`, so a
33-hour plan of quarter-hourly points logged `dynamic_dayahead from
generic_dayahead, 132 h`. It is the same conflation the *Price history* row
carried before FCST-04 and the same one `at()` carried before TARIFF-14, and it
is **not a constant factor**: measured over a 40-hour soak the overstatement
ranged **1.79× to 5.33×**, because the carried points are hourly while the firm
ones are quarter-hourly, so the ratio moves with the mix as the day goes on.
The same sentence is in the **add-on log** at every refresh, alongside `§8.3:
33 of 48 hours carried forward from yesterday's profile and flagged
provisional; the feed published 60 points over the other 15 hours`.
The §8.3 refusal an operator reads when a trade is declined counts in the same
two units — `§8.3: only 1 of 96 points carry a published price - 1 of 24 hours`
— where it used to say `only 1 of 96 hours` about a 24-hour day.
The plan is also now cut to the horizon you asked for, which means **hours that
have already elapsed are no longer in it**. That is a real change to arbitrage:
a cheap night that has already gone used to sit in the series as the cheapest
hour of the day, and a cycle can never buy in an hour that is past — so the
trade was quietly never taken. The spread is now taken over hours still ahead.
A new status row, **Price history**, says which days are held, how many price
points each holds and how many whole hours those cover — `2026-01-13 (96 points
over 24 h)`. ⚠️ **Those are two different numbers and the row used to print one
of them under the other's name**: it counted points and labelled them `h`, so a
quarter-hourly feed read `96 h` for a 24-hour day. The figure that says whether
tomorrow morning's fill has a whole day to carry is the **hour** one. If it
reads `empty` or `unavailable`, nothing is carried, the horizon
is as short as the feed makes it, and every other guarantee is unchanged — a
cache that cannot answer is not allowed to be a controller that cannot regulate.
The store is rebuilt by the next successful fetch; on a brand-new install the
first fill is possible the day after the first refresh.
The same applies to a file that is *partly* broken: a day record this build
cannot read is dropped, the row says how many were dropped and why, and the rest
of the file still loads. And a store that **cannot persist records nothing** —
not even in memory. That covers a missing `/data`, and it also covers a `/data`
that is there and cannot be **written**: a full volume, a read-only remount. An
in-memory yesterday would answer for one run and vanish on the next restart,
which is a cache that forgets without saying so; the row naming the reason and
*no prices carried* are the honest pair.
⚠️ **What a failed write does not do is throw away what is already in the
file.** A volume that fills at 14:00 leaves the yesterday written at 13:30 still
answering, because that record survives a restart — the store stops *adding* to
it, nothing more. And it never stops the add-on: a write that cannot complete is
logged and reported on the status row, never raised. A cache that cannot answer
is not allowed to be a controller that cannot regulate, and that applies to the
volume being full exactly as it applies to the volume being absent.
**Injection is priced separately** (`export_price_eur_kwh`). In Belgium it is
often near zero, which by itself strongly favours self-consumption over export.
It is a single number rather than a curve because no supported supplier
publishes a dynamic injection tariff; negative values are accepted.
Negative values are accepted — you can be paid to *not* export at midday in
summer.
⚠️ It is a single number rather than a curve, and the reason given here until
TARIFF-07 — *"no supported supplier publishes a dynamic injection tariff"* — was
**false**. Suppliers do. Eneco publishes `0,1 x BELPEX-H − 1,188` on the same
tariff card as its consumption formula (identical on the January and the
September 2026 cards), and ENGIE publishes one in the same shape,
`(B x eSpot_15) − A`. What is true is narrower: an injection curve is the **same
two-coefficient formula** as the consumption curve but with **its own**
coefficients and **its own VAT treatment** — injection is VAT-exempt where
consumption is at 6% — so it is a second per-contract formula rather than a copy
of the first, and the add-on cannot evaluate either one yet. Until the formula
provider ships, **this scalar is a simplification you enter by hand — an average
of what you are actually paid — and not a reflection of the market.**
**The old Home Assistant price sensors.** `price_now_entity` and
`price_avg_entity` still work and are unchanged: the add-on compares them and
@@ -1436,6 +1870,211 @@ uses the result as a cheap-window hint for the maintenance charge. They are
— configure two of them and the add-on refuses to start, naming the conflict,
because the add-on reads exactly one and the loser is silent. Use one.
### The three buckets — which price components can move a decision (TARIFF-17)
**Read this before adding a price component of any kind.** The optimiser reads
exactly two quantities off a price series, and nothing else:
- the **spread**, `max − min` over the *firm* points, weighed against
`cycle_cost_threshold_eur`; and
- **which hour is cheapest**, `min(firm, key=…)` — the hour a battery would
buy in.
Every component of a Belgian bill reaches those two through one of three
buckets, and which bucket it is in is a property of its *shape*, not of its
size:
| bucket | what it looks like | reaches the decision? |
|---|---|---|
| **LEVEL** | additive, **identical every hour** | **No — provably.** An additive constant cancels exactly in `max − min`, and adding it to every point leaves the argmin where it was |
| **SPREAD** | **multiplicative** on the spot index | **Yes.** It scales the spread against a fixed `cycle_cost_threshold_eur` |
| **ORDER** | **varies** by hour, block or day-type | **Yes, and worst.** It changes *which hour is cheapest* |
Measured on the real arithmetic (`test_price.py` section 16b, printed on every
run): with a flat taxes figure of 0.00, 0.0739, 0.11736 (the shipped default),
0.1198, 0.1327 or 1.00 EUR/kWh, the same 24-hour feed is cheapest at **13:00 in
all six cases** and its spread is **0.1500 EUR/kWh in all six cases**. That is
the LEVEL bucket doing nothing, and it is not a rounding artefact — it is
arithmetic.
Three consequences worth having in mind:
1. **`price_taxes_eur_kwh` is not a wrong number and never was.** It sits in
the LEVEL bucket and *cannot* distort either quantity. What it also cannot
do is express a component that varies by hour — which is why
`price_taxes_profile` exists. The defect TARIFF-17 closed was a **missing
capability**, not a bad constant.
2. **`custom_formula`'s `factor` is a SPREAD component, and its `offset` is
LEVEL.** The contract price is `(factor × index + offset) × (1 + VAT)`, so
the spread over a feed is `factor × (1 + VAT) × (max index − min index)` —
the offset drops out entirely, and the spread is *proportional* to `factor`.
Moving `factor` from 0.100 to 0.10812 therefore moves the spread by exactly
**+8.1 %** on any feed whatever. A coefficient typed one digit wrong is not
a small pricing error; it is a different trading threshold. The VAT rate is
multiplicative on the whole expression, so it is SPREAD too.
3. **A check that grades the spread is grading the wrong bucket.** An
implementation that reads an hour-varying table *once* and adds the same
number to every point produces a series of the right magnitude and the wrong
shape — the LEVEL bucket wearing a period table. The spread would look
plausible; the buy hour would be wrong. So the regression check for an
ORDER-bucket component asserts **which hour is cheapest**, not how big the
spread is (`test_price.py` 16c and 16d).
⚠️ **Where the buckets are today.** `price_markup_eur_kwh`,
`price_taxes_eur_kwh` and `custom_formula`'s `offset` are LEVEL.
`custom_formula`'s `factor` and its VAT rate are SPREAD. `price_taxes_profile`
is the only ORDER component the add-on has. Anything added next belongs in one
of the three, and saying which is part of adding it.
### What `price_taxes_eur_kwh` is made of, and why yours is different (TARIFF-18)
**It ships at `0.11736` EUR/kWh.** That is the all-in non-energy part of a
Flemish residential bill — everything on your invoice that is *not* the energy
itself — read off the Eneco residential card for Flanders, **September 2026**,
**including 6 % VAT**:
| component | c€/kWh |
|---|---|
| Fluvius distribution (**Antwerpen**) | 5,35 |
| Bijzondere accijns, *"Alle verbruik"* — no volume bands | 4,876 |
| Bijdrage groene stroom en WKK Vlaanderen | 1,51 |
| Energiebijdrage | **0** |
| **total** | **11,736 c€/kWh = 0.11736 €/kWh** |
The rows are in the code, in `app/price.py`'s
`FLEMISH_TAXES_COMPONENTS_EUR_KWH`, and `test_price.py` section 1b re-adds them
and refuses a default that is not their sum. It replaces a shipped `0.06`,
which was roughly **half** the true figure and had no recorded provenance at
all.
⚠️ **`energiebijdrage` is zero and that is not a mistake.** *"Vanaf 1 augustus
2026 bedraagt de energiebijdrage 0 EUR/MWh"* — it was merged into the
bijzondere accijns. If you are re-deriving this total from a guide written
before that date you will find a separate energiebijdrage line; adding it would
count it twice. It is carried as a **zero row** rather than left out for
exactly that reason.
⚠️ **THIS IS A PER-DSO FIGURE AND YOURS IS PROBABLY NOT 0.11736.** Only the
distribution row moves. Across the eight Fluvius DSOs it runs **5,28 c€/kWh**
(Midden-Vlaanderen) to **6,70** (West), so the Flemish all-in total runs about
**0.117 to 0.131 €/kWh** — a 1,42 c€/kWh spread, which on 4 000 kWh a year is
about €57. The other two rows are Flanders-wide and do not move. **Look up your
own distribution row, add the same accijns and groenestroombijdrage, and type
the total.**
⚠️ **A WALLOON OR BRUSSELS SITE SHOULD NOT USE THIS SCALAR AT ALL.** Their
distribution tariff has a **day and a night band** — `Enkelvoudig | Dag |
Nacht | Uitsl. nacht` on the same card, where Fluvius prints a single rate —
and a flat figure flattens exactly the component that is anti-correlated with
what the battery wants to do. Use **`price_taxes_profile`** (the next section);
it is the only ORDER-bucket component the add-on has.
⚠️ **THIS IS A DISPLAY AND GUARD FIX. IT DOES NOT IMPROVE ARBITRAGE, AND IT
CANNOT.** `price_taxes_eur_kwh` is a **LEVEL** component — added identically to
every hour — so it **cancels exactly** in `max − min` and leaves the argmin
where it was. It does not change which hour is cheapest, it does not change the
spread, and it does not make any trade happen that did not happen before. See
*The three buckets* above, where six flat scalars spanning and exceeding the
whole per-DSO range give the identical cheapest hour and the identical spread
to the last digit. What raising it **does** change is exactly two things:
* **every figure an operator reads** — the all-in prices on the status page,
`custom_formula`'s startup preview and the recorded price history. At half
the true non-energy component they made the add-on look considerably better
than it is.
* **§17's negative-price guard** (`arbitrage.py`), which is the one rung that
tests an **absolute** all-in price rather than a difference. The guard fires
when `spot < −(markup + taxes)`, so its boundary moved from **−0.09000** to
**−0.14736 €/kWh** of spot. A spot hour at −0.11868 €/kWh used to read
**−0.02868 €/kWh** all-in and the discharge was refused as paid-for disposal;
at the real tax level it reads **+0.02868** and the discharge is allowed,
because the operator really does pay those levies and that hour really was
profitable (`test_arbitrage.py` section 2c drives both, and pins the boundary
from below as well).
⚠️ **Which is why the default sits near the LOW end of the per-DSO band rather
than in the middle of it.** The two error directions are not symmetric. Set too
**high**, the figure carries a genuinely negative hour up above zero and the
add-on exports into paid-for disposal — §17's own regression case. Set too
**low**, it only declines a discharge that would have paid. Where one default
has to stand in for eight DSOs, it belongs at the bottom.
**If you set this yourself, nothing above affects you.** A site with its own
`price_taxes_eur_kwh` keeps it; the default only answers where the option is
absent from the stored options blob.
### Levies and VAT by time of day (`price_taxes_profile`)
**Leave this empty unless your network tariff really does have a day and a
night rate**, which in Flanders it does not. Fluvius distribution is a single
rate: its row on the tariff card reads `Normaal | Uitsl. nacht`, and
*exclusive night* is a **separate physical circuit** (a night-storage heater on
its own meter), not the night register of a dual-rate meter. For a Flemish site
the flat `price_taxes_eur_kwh` is correct, and this option changes nothing.
**Wallonia and Brussels are different, and the same card proves it.** The
Walloon DSOs and Sibelga get four columns — `Enkelvoudig | Dag | Nacht |
Uitsl. nacht` — and the day/night gap is real money (c€/kWh):
| DSO | enkelvoudig | dag | nacht | gap |
|---|---|---|---|---|
| ORES | 11,98 | 13,27 | 7,39 | **5,88** |
| AIESH | 13,64 | 15,17 | 8,21 | **6,96** |
| Tecteo RESA | 11,06 | 12,20 | 7,01 | 5,19 |
| Sibelga (Brussels) | 9,39 | 9,39 | 7,10 | 2,29 |
| ORES IMPACT (2026) | — | 16,578 | 5,090 | **11,49** |
⚠️ **The bands are anti-correlated with what the battery wants to do.** The
cheap network hours are the cheap spot hours. So flattening them does not
merely shift the curve — it **systematically understates the true spread**, and
the add-on declines cycles that are genuinely profitable. On ORES's bands over
one ordinary winter weekday the measured effect is: cheapest hour **13:00 →
03:00**, spread **0.1500 → 0.1788 EUR/kWh**.
It is written in **exactly the same grammar as `static_profile`** — the same
periods, the same day selectors, the same rule that every day type is covered
once with no gap and no overlap, the same DST handling. It is literally the
same parser; there is no second tiling rule, because a second one would be a
second set of 23- and 25-hour-day bugs.
```json
{"periods": [
{"name":"dag", "from":"07:00","to":"22:00","days":"mon-fri","eur_per_kwh":0.1327},
{"name":"nacht","from":"22:00","to":"07:00","days":"mon-fri","eur_per_kwh":0.0739},
{"name":"nacht","from":"00:00","to":"24:00","days":"sat,sun", "eur_per_kwh":0.0739}]}
```
- ⚠️ **`eur_per_kwh` here is the LEVIES ALONE, not an all-in price.** It is the
non-energy part of the bill for that band, and the add-on adds it to whatever
your price source publishes — exactly as it adds the flat figure. This is the
one place the two options differ in meaning while sharing a grammar, and the
refusal you get for leaving the number out says which one is wanted.
- ⚠️ **The card prints cents. This option takes EUR.** ORES prints `13,27`;
type `0.1327`. A whole card typed in cents is refused at startup, naming both
bands and the factor — including the cards (Sibelga's 9,39 / 7,10) whose
figures are small enough to slip past the ±10 EUR/kWh bound the parser holds
every price to.
- ⚠️ **Set this *or* `price_taxes_eur_kwh`, not both.** The table wins when it
is set, so a scalar left at a non-default value beside it would be a number
you typed that decides nothing. That is refused at startup rather than
silently resolved; leave `price_taxes_eur_kwh` at its shipped `0.11736` when
you fill this in. ⚠️ **A scalar still sitting on the superseded `0.06` is
also accepted**, and deliberately so — TARIFF-18 moved that default, and your
stored options blob carries the old number whether you typed it or not, so
refusing it would have stopped working Walloon and Brussels sites on an
upgrade over a figure the table overrides anyway.
- A **malformed table is refused at startup**, not at the next fetch. It is
read on the ~30 s planning path, and a table that first fails at 03:00 is a
site that stops pricing at 03:00 — which §11.4 turns into a quiet fallback
that looks exactly like a healthy fixed-tariff day.
- It applies to the components the framework **adds**: `generic_dayahead`
(which declares `{ENERGY}`) and `custom_formula` (which declares
`{ENERGY, MARKUP}`) both get it. `manual_csv` publishes an all-in figure and
is untouched — the table is routed through the same `capabilities().adds`
rule as the scalar, not bolted on.
- It does **not** apply to `static_profile`, whose prices are already all-in,
nor to `export_price_eur_kwh`.
### Static day/night and fully-fixed tariffs (`static_profile`)
Most Belgian households are **not** on a dynamic contract, and an EMS that only
@@ -1547,6 +2186,41 @@ towards the firmware's 30 s failsafe. The plan refreshes every 30 minutes and
retires itself if the refresh stops answering, at which point arbitrage stops
with it.
⚠️ **That refresh is not gated on this switch** (TARIFF-10). It runs when a
`price_provider` is configured *or* when arbitrage is armed, so turning
arbitrage off does not turn the price feed off — and a site with no provider
that arms arbitrage on a `static_profile` keeps the plan it has always had. The
only thing that decides whether a request leaves the machine is
`price_provider`: with it `off`, the refresh resolves the static profile in
memory and opens no connection.
**The *Arbitrage* status row says which of three things is true** — trading is
off, trading is on and this cycle reached no decision, or a decision was
reached — and it stamps every decision with its age. Until TARIFF-11 it said
one of two, because the first two rendered identically. They have three
different remedies:
| the row opens with | what is true | what to do |
|---|---|---|
| **off** — *"arbitrage_enabled is false, so nothing trades"* | Trading is not armed. Nothing will happen, and nothing is wrong | Arm it, if you meant to |
| **ON (arbitrage_enabled), but NO DECISION has been reached yet** | Trading **is** armed, and no cycle has run as far as the arbitrage step. The row then names the control loop's own reason, usually *inputs missing* | Fix the input the loop names |
| **decided N s ago:** … | The strategy answered this cycle, and `N` is how old that answer is | Read the sentence that follows — it is the strategy's own |
| **no decision this cycle (reason); the last one was taken N s ago:** … | Trading is armed, a decision stands from an earlier cycle, and **this** cycle stopped before reaching the step | As row 2 — the standing decision is history, not a verdict on now |
A fifth rendering — *"…no decision this cycle …, and none stands from an
earlier one"* — covers the case where trading was armed after a cycle had
already run without it.
⚠️ **The second row used to render exactly like the first one**, naming
`arbitrage_enabled` on a site where `arbitrage_enabled` is `true`. A cycle with
a missing input returns after roughly `stale_input_s` (15 s by default) without
reaching the arbitrage step at all, so on an armed site the row reported *off*
at startup and, once one decision had been taken, went on showing that decision
for up to 15 s after the site stopped deciding anything. The age on the third
row is printed **always**, including when it is zero: a row that reports
staleness only once it is stale cannot be told from one that never reports
it.
### Site
| option | meaning |
+202
View File
@@ -127,6 +127,43 @@ Two rules come out of it, and both are cheap:
function, that pair is the first thing to try — TEST-16's two survivors were
adjacent lines in one function and neither was individually live.
⚠️ **AND FCST-05 WALKED INTO IT AGAIN, IN ITS OWN NEW CHECK, WHICH IS WHY THE
RULE IS “mutate every check you add”.** `price_history.py` reports a malformed
point row from **two** places into **one** field: `_usable_days()` records what
it FOUND when the file was opened, and `_remember()` records what it DROPPED on
the next refresh. Both land in `last_error`, and `words` prints `last_error`.
The first draft of the new check asked `"malformed point row" in words` — which
whichever rung is still standing satisfies — so deleting `_remember()`'s note
entirely was a **−1 B SURVIVOR of the whole suite**, on the half of the fix that
tells an operator the cache **self-healed**. It is graded on the TENSE now
(`… in <day>` at open, `… dropped from <day>` after the refresh), the two are
asserted to DIFFER, and the paired mutation — both notes deleted, −91 B — is
red on 15 named checks. Same shape as TEST-16 and the same fix: **grade the
reason, not the verdict**, and when two rungs answer one stimulus, mutate them
together.
⚠️ **AND FCST-06 FOUND THE COUNT ITSELF LYING — LOW, NOT SILENT.** A mutation
whose red is a **traceback** does not report one red; it reports however many
checks ran before the traceback, and grades **none** of the ones after it. The
new section drove M4 (`self._days = before` → `self._days = {}`) and the run
printed **1 FAIL**. The true figure is **3**: an f-string in the next check read
`len(read.points)`, and `.points` on a `no_data` **raises `Absence` by design**,
so the harness died mid-section with the remaining checks ungraded. That is
`why()`'s and `derived()`'s rule — *"it answered nothing" and "it broke on the
way to answering" must not grade the same* — arriving in the **printed operand**
of a check rather than in its condition, which is the half that is easy to miss
because it looks like formatting. Two things follow:
* **a mutation report's red COUNT is only a lower bound until the run is seen to
have reached the end of its section.** Read the output, not the count;
* **the total accessor has to cover what a check PRINTS, not only what it
asserts.** `test_price_history.py` 5d added `npts()` beside `why()` for
exactly this, and re-driving M4 against it moved 1 FAIL to 3.
Worse, had the traceback landed one check earlier the mutation would have
produced **no named red at all** — a result that reads like a broken harness and
gets retried rather than read.
## The third way a check lies: its DATA cannot reach the failure it names
Stale bytecode manufactures survivors from a mutation that never ran. Masking
@@ -658,6 +695,84 @@ Cheap habits fall out of it, and all of them are now used in this repository:
that two of it cannot silently overwrite each other's work. A fixed path
in a tool that edits source is a shared mutable global with a filename.
19. ⚠️ **A FIXTURE IN A SHAPE NOBODY PUBLISHES GRADES THE PARSER AGAINST THE
PARSER — FOURTEENTH INSTANCE OF HABIT 4, AND THE FIRST WHERE THE FIXTURE
AND THE CODE UNDER TEST SHARED A *SHAPE* RATHER THAN A NUMBER.** TARIFF-08.
`price.py`'s `_points_from_json()` read `doc["data"]` as a list of
`{start, price_eur_mwh}` objects. `api.energy-charts.info` publishes
parallel `unix_seconds` / `price` arrays and has no `data` key at all, so
`generic_dayahead` raised `SourceUnavailable` on **every live fetch it ever
made** — and the suite was green, because all three price suites built
their bodies with `json.dumps({"data": [...]})`. The fixture was derived
from the parser's guess, so no mutation of the parser and no amount of
coverage inside that loop could ever have reached the failure. §11.4 then
turned it into a *correct* silent degradation: parse error -> `no_data` ->
static profile -> arbitrage off, with nothing naming the cause.
Habit 2 with a network on the end of it: **the reference has to be data the
code under test did not shape.** For a parser that means BYTES, captured
from the source and committed unmodified, with the URL, the retrieval time
and a checksum beside them - `scenarios/energy-charts-be-price-*.json` and
its `-captures.json` provenance, re-derived by the replay so an edited
fixture stops being a capture. Two granularities, because the same endpoint
answers hourly before 2025-10-01 and quarter-hourly after it and a fixture
at one spacing cannot grade an assumption about the other.
⚠️ **AND THE SURVIVING SYNTHETIC FIXTURES ARE NAMED FOR WHAT THEY ARE.**
Three shipped plugins point at endpoints nobody has verified (TARIFF-07),
so no capture exists for them; their fixture builder is called
`synthetic_row_feed` at every call site, because the next reader's default
assumption will otherwise be that a green check means the feed parses.
⚠️ **ONE OF THIS CARD'S OWN NEW LINES WAS DECORATION AND THE BATTERY SAID
SO.** The columnar parser first rendered its instants with `brussels()`
before returning them. `assemble()` already renders every plugin's points,
so removing the parser's call changed nothing observable - **M4 came back a
survivor on a `:ro`, purged tree**, which is not the direction stale
bytecode fabricates. The line was removed rather than fenced with a check:
a second copy of a rule that already has an owner is indistinguishable from
a no-op, and the mutation that proves the rendering is real belongs on
`assemble()`, where it is now aimed.
20. ⚠️ **A FIXTURE GRADES THE RESPONSE. NOTHING ABOUT THE REQUEST IS REACHABLE
FROM ONE, AND THE TRANSPORT HAD BEEN RECORDING IT ALL ALONG.** TARIFF-09,
and it is habit 19's closed loop with the other half still shut.
`generic_dayahead.fetch()` took `since` and `until` and asked for the bare
`?bzn=BE`, which answers ONE Brussels day - so §8.3 dropped every hour past
today from the trade set as provisional and the optimiser never saw
tomorrow as firm, at any hour of any day. Measured at 09:00: **15 firm
hours and 33 carried of 48**, both ends of the permitted cycle inside one
calendar day; at 23:00, **1 and 47**, with the firm spread no longer
clearing the round-trip cost.
TARIFF-08 had just fixed the RESPONSE side of this same plugin by
committing real bytes. It could not have caught this: a captured body is
the answer to a URL, and **no property of any body says which URL was
asked for**. `canned()` in `test_price.py` had recorded `(url, timeout_s)`
on every call since TARIFF-02 and not one check had ever asserted on the
first element - so the harness *knew* the plugin was ignoring its arguments
and nothing asked it. The fix is one line of habit: **where a plugin takes
arguments that shape a request, assert on the REQUEST off the injected
transport, not only on what comes back.** Section 2c does, and the
shipped-defect mutation (restore the bare URL, `-173 B`) is red on six
named checks in `test_price.py` and two in `test_price_history.py`.
⚠️ **AND A MUTATION THAT TURNS A READ INTO `no_data` CRASHES A SUITE
INSTEAD OF NAMING A CHECK, UNLESS THE CHECK EXPECTS IT.** `_NoData.points`
raises `Absence` by design, so `derived(...) < derived(...)` over two
absent reads is a `TypeError` and `[p for p in plan.read.points]` is a
traceback - habit 5's shape with an exception in place of a hang. The
mutation that matters here is the "defensive" one somebody writes after
reading this card: *refuse when the feed returned fewer days than we asked
for*. It is exactly what a healthy feed does every morning before the
~13:00-14:00 CET auction, and switching arbitrage off for it would be a
worse defect than the one being fixed. It is red BY NAME in
`test_price.py` (33 checks, the AC's own line among them) and takes
`test_price_history.py` down at **FCST-04's section 1**, whose `_before`
read is unguarded - a red, but an unnamed one. That is recorded rather than
retrofitted: guarding another card's section is a change to what its
checks report, and it belongs to whoever next touches it.
And the mutation-table consequence: **a variant is not covered because a
near-miss of it is.** DEV-02's table listed the *actuator* variant of the
per-unit claim defect (`device`/`supervised` filed without `inv`), which the
@@ -665,6 +780,92 @@ static routing pin catches. The *bound* variant — right actuator, wrong
numbers — is a different defect with a different fix, and it was absent. Both
are in the table now.
## TARIFF-15 — the card-graded formula, mutation by mutation
Twenty-two mutations, one per check the card added, **run** rather than
reasoned about. Harness: `/home/homelab/.cache/t15-mut/run.py`, backups beside
it rather than in a shared scratchpad — two agents' harnesses collided in one
this week. Each run restores from backup, purges `__pycache__` **on the host**
first, and grades through the CI image with the tree bind-mounted `:ro`, so the
interpreter cannot write a cache into the tree under test at all (TEST-15's
structural immunity, borrowed).
**No survivors.** Two entries are `+0 B` and are called out as such, per the
reporting rule.
| | mutation | file | Δ | red |
|---|---|---|---|---|
| M1 | `CARD_TOLERANCE` widened to 100 — the self-check stops checking | `custom_formula.py` | +1 B | **8** |
| M2 | `CARD_TOLERANCE` tightened to 0 — the other direction, so the tolerance is a number and not a shape | `custom_formula.py` | −1 B | **16** |
| M3 | `units` becomes optional | `custom_formula.py` | +10 B | 1 |
| M4 | the `factor == 0` rung removed | `custom_formula.py` | −8 B | 1 |
| M5 | the `factor < 0` rung removed | `custom_formula.py` | −7 B | 1 |
| M6 | the `vat_pct` band widened to ±1000 | `custom_formula.py` | +6 B | 2 |
| M7 | the non-finite rung removed from `_finite()` | `custom_formula.py` | −26 B | 2 |
| M8 | the non-numeric rung removed from `_finite()` | `custom_formula.py` | −16 B | 4 |
| M9 | an unknown top-level field ignored instead of refused | `custom_formula.py` | −2 B | 1 |
| M10 | the plausibility **floor** applied to the all-in figure instead of the energy one | `custom_formula.py` | **+0 B** | 2 |
| M11 | `hourly` stops averaging and emits the feed's own points | `custom_formula.py` | +12 B | 3 |
| M12 | the hourly bucket takes the FIRST quarter instead of the mean | `custom_formula.py` | −11 B | 1 |
| M13 | the preview drops the index column from each row | `custom_formula.py` | −13 B | 1 |
| M14 | the preview is emitted on every refresh rather than once | `price.py` | −8 B | 1 |
| M15 | `custom_formula` declares `ALL_IN` — the double-counting lie | `price.py` | −41 B | **10** |
| M16 | a stale card never warns | `custom_formula.py` | −8 B | 1 |
| M17 | a stale card REFUSES instead of warning | `validate.py` | −3 B | 1 |
| M18 | the empty-formula refusal removed from the startup gate | `validate.py` | −38 B | 1 |
| M19 | the index scaled to EUR/kWh before the formula — the factor-1000 | `price.py` | +2 B | **7** |
| M20 | an `eval` call planted in `app/custom_formula.py` | `custom_formula.py` | +14 B | 1 |
| M21 | `custom_formula` stops passing the caller's range to the feed | `price.py` | **+0 B** | 1 |
| M22 | **PAIRED**: M1 *and* the plausibility band removed, together | `custom_formula.py` | −60 B | **12** |
### The four that are worth reading rather than counting
**M13 was a SURVIVOR of the whole suite on the first attempt, at −13 B.** The
check read `"index EUR/MWh -> all-in EUR/kWh" in log` — which is the preview's
**heading**. Deleting the index column from every **row** left the heading
saying it prints something it no longer printed, and nothing went red. That is
this file's third failure mode arriving in the check written to close it: the
assertion ran, the sentence was true, and the data could not reach the failure
it named. It is graded on a rendered ROW now, built here from the captured
document (`' 216.080 -> 0.30123'`), and M13 is red.
**M10 is `+0 B` and it is a real defect, not a synthetic one.** It came out of
writing the mutation rather than out of review: `implausible_all_in()`'s floor
was applied to the all-in figure, and `price_taxes_eur_kwh` shipped at 0.06 —
**three times the 0.02 floor** — so a card read a hundred times too small
(0.0016 EUR/kWh of energy) is carried over the floor by the flat tax addend
alone and passes. The hundredfold error is exactly what that band exists for.
The floor is applied to the **energy** figure now and the ceiling to the all-in
one, both directions are graded, and the trap is stated as a check in its own
right so the asymmetry cannot be "tidied up" later. *(Annotated 2026-09-13:
TARIFF-18 raised that default to 0.11736, **nearly six times** the floor. The
addend that does the carrying got BIGGER, so the trap is wider than when M10
was measured and the asymmetry is more load-bearing rather than less.)*
**M15 is the one the framework CANNOT catch, and the reds say which half does.**
Declaring `ALL_IN` makes `assemble()` add nothing, and section 3's all-in check
is *derived from the declaration* — so it passes on the lie, exactly as
`price.py`'s trap 1 says no code can catch a misdeclaration. What goes red is
the **docstring/declaration agreement** check, the card reproductions, and the
`sources` pin: ten reds, none of them from the arithmetic. That is the
guarantee at its true size — the two claims are held to agree, and whether
either is true about somebody else's tariff remains human.
**M22 is the paired run (TEST-16).** The card self-check and the plausibility
band both answer a unit error, so each could be masking the other. Weakened
**together** they produce twelve reds rather than the eight M1 alone produces,
and the band's own checks stay red — so neither rung is doing the other's work.
### What this battery does NOT establish
It does not establish that a card is the right card. Every mutation above moves
the relationship between the coefficients and the printed price; **none of them
can move the relationship between the printed price and the operator's actual
contract**, because nothing in this repository observes that. A
consistent-but-wrong pair passes every one of these checks, by construction,
and `DECISIONS/TARIFF-15-the-card-is-the-grader.md` §3 says so in the same
sentence as the claim.
## Where each surface stands
* **Local and dev-harness runs — the exposed one.** This is where FCST-03 hit
@@ -715,6 +916,7 @@ commit and PR bodies.
| **SAFETY-03** reviewer survivor: `abs()` dropped from `clamp_to_rating`'s limit (6aabb83) | **re-verified — still a survivor** | re-run the same way, all 15 green. Also as recorded: unreachable under the `int(100, 5000)` schema. |
| **MAINT-05 round 3**, `migrate_maint_keys()` marker gate `if rep.ok:` → `if rep.ok or rep.kept:` (`+12 B`) | **was a survivor — now CLOSED** | measured against head `65cddf3` under the purge, tree mounted `:ro`, `19/19 PASS`. It stamps the schema marker on a migration that LOST state, defeating the older `lost` guard as well as the new `misdirected` one, because every check that reached the marker did so on a store whose `kept` list was empty. Closed by `test_maintenance.py` **9d**; the mutation is now red on three named checks. |
| **MAINT-05 round 3**, `migrate_unit_keys()` marker gate `if rep.ok:` → `if rep.ok or rep.kept:` (`+12 B`) | **found here — SAFETY-02's half, now CLOSED** | the SAME mutation one function up, on the LATCH migration rather than the history one. Measured a survivor against this branch under the purge, `:ro`, `19/19 PASS`, then closed by `test_inverter.py` part 10's new fixture — a store with `unit/inverter-a/sign_reason` already present (so `kept` is non-empty) whose `sign_tripped: true` write is dropped. Shipped: marker withheld. Mutant: marker stamped on a **disarmed latch**, and the next start skips the migration that would have re-armed it. |
| **TARIFF-09** 11 mutations over `app/price.py` and the two-day capture, all red, no survivors | **not at risk** | direction, and method: every mutation was applied to a FRESH `cp -a` of the clean worktree rather than reverted, so there is no revert to verify (habit 18) and no shared scratch path two runs could collide on. Each ran through `tools/mutation_run.sh`, which purges and fingerprints. Byte deltas recorded on the PR; one is **+0 B** - a single price digit edited inside the committed capture, the risky delta - and the re-derived sha256 catches it. |
| **TARIFF-01**'s other three survivors, **SAFETY-03**'s `describe()` survivor, **TARIFF-02**'s M3/M5/M7, **SAFETY-18**'s three | **still unverified** | recorded before the purge discipline, with no byte delta on the record, so a `+0 B` same-second revert cannot be excluded. All are survivors, which is the direction this defect fabricates. TARIFF-02's and SAFETY-18's were **closed by later commits whose claims are reds**, so what is unverified is the *diagnosis*, not the current state of the code. |
**Nothing here says a past result is wrong.** The two survivors re-run above
+1 -1
View File
@@ -24,7 +24,7 @@ since the status page shipped; this file has five more chances to get it wrong.
⚠️ AND IT IS BUILT FOR A PHONE FIRST. This is the screen someone stands in
front of the inverter with: one column, big touch targets, sections that start
collapsed so 71 options do not arrive as one wall, and the refusal block pinned
collapsed so 72 options do not arrive as one wall, and the refusal block pinned
where it cannot be scrolled past.
"""
+673
View File
@@ -0,0 +1,673 @@
"""TARIFF-15 - the customer's own tariff card, as two fixed coefficients.
WHAT THIS IS FOR, in one sentence: a Belgian dynamic contract prices energy as
`B x index + A` with coefficients that exist in exactly one place - the tariff
card in the customer's klantenzone - so the customer types those two numbers
and the add-on does every unit conversion and the VAT multiplication itself.
WHY A FORMULA AND NOT FOURTEEN MORE PLUGINS (§9.2, TARIFF-07)
=============================================================
There is NO machine-readable source of Belgian tariff coefficients anywhere.
CREG's consumer tariff-card pages 404, CREG Scan is a JS-only wizard, the
annual report carries the supplier list and no per-product coefficients, and
VREG was unreachable. Every coefficient found in research came off a supplier's
own PDF, one of them image-only. So there is nothing to scrape and a
per-supplier plugin would have no source for its numbers - fourteen plugins
would be fourteen copies of the same invention, which is the defect TARIFF-07
withdrew three plugins for. THE CUSTOMER IS THE ONLY PARTY WHO CAN SEE THEIR
OWN COEFFICIENTS, and the card they read them from is the only artefact that
can grade them.
THE SHAPE IS THE MARKET'S PUBLISHED ARITHMETIC, NOT OUR APPROXIMATION
====================================================================
Both suppliers whose 2026 cards were readable print the formula, the reference
index they assumed AND the resulting price, so the arithmetic below is
reproducible against something they published rather than against our own
guess:
Eneco (0.102 x 133.9905 + 1.00) x 1.06 = 15.5471 card prints 15,55
ENGIE (0.100 x 61.96 + 0.90) x 1.06 = 7.5218 card prints 7,522
Two independent suppliers, reproduced to the last printed digit, in
c EUR/kWh. test_price.py section 13 runs both through the SHIPPED code.
⚠️ THE DOUBLE-COUNTING MECHANISM, AND WHY THIS DESIGN IS SAFE AT ALL
====================================================================
This provider declares `sources = {ENERGY, MARKUP}` and NOT `ALL_IN`. A Belgian
card's `Tariefformule` line is THE ENERGY COMPONENT ONLY: Eneco puts network
tariffs in one table and levies in a second, and ENGIE states in so many words
that the formula "does not include fixed network/distribution costs, transport,
grid levies, or excise duties". Declaring ALL_IN would lie on behalf of every
user who typed what their card says, and would silently drop ~12 c EUR/kWh.
price.py already mechanises that a PLUGIN cannot double-count (`PluginOptions`,
`FRAMEWORK_KEYS`, `code_leaks()`). This provider introduces a risk none of those
can see, because it is not in the code: THE OPERATOR double-counting, by folding
their network charges into `offset` while `price_taxes_eur_kwh` carries them as
well. `code_leaks()` scans code. This is config.
So the card is the grader:
At startup validate.py computes
`(factor x card.index_eur_per_mwh + offset) x (1 + vat_pct/100)` and
REFUSES TO START unless it equals `card.expect_incl_vat` to within
CARD_TOLERANCE, naming the discrepancy.
An `offset` inflated by network charges CANNOT PASS: the card's printed estimate
is by construction the energy component alone, so adding ~12 c EUR/kWh moves
15.55 to ~27 and the add-on refuses. The same check catches VAT omitted, VAT
applied twice, a transposed digit, and a UNIT MISMATCH BETWEEN THE TWO SIDES -
coefficients read off the formula line while the expected price is typed in
another convention, which is the factor-10/100/1000 class as it actually
happens.
⚠️ THE ONE UNIT ERROR THE CARD CHECK CANNOT SEE IS THE ONE APPLIED TO BOTH
SIDES AT ONCE, and saying so is the difference between a guard and a slogan:
scaling the coefficients AND the expected price by the same 100 cancels out of
a comparison between them. That is what `implausible_all_in()` is for, and it
is a SEPARATE refusal on a SEPARATE quantity - the card's own point, all-in,
against the band a Belgian retail tariff can occupy.
⚠️ AND THE LIMIT, IN THE SAME BREATH AS THE CLAIM, exactly as price.py's trap 1
states its own: THE CHECK GRADES THE COEFFICIENTS AGAINST THE CARD, NOT THE CARD
AGAINST REALITY. A consistent-but-wrong pair - both numbers copied off the wrong
product, or off last year's card - reproduces its own printed price perfectly
and passes. Nothing here or anywhere else can catch that; what is mechanised is
that the two coefficients and the printed estimate AGREE, which is the whole of
the unit/VAT/decimal-point class and none of the "wrong contract" class.
⚠️ FIXED COEFFICIENTS. NO EXPRESSION EVALUATION OF ANY KIND.
============================================================
All 24 surveyed Belgian products, both injection legs and aWATTar AT fit
`B x index + A`, so an expression buys nothing the market needs. Against that:
* the add-on serves `POST config/save` -> `POST /addons/self/options` on its
own ingress page, so this string is SETTABLE OVER HTTP. `eval` on it is
remote code execution in a root container holding SUPERVISOR_TOKEN, with
`homeassistant_api` and `hassio_api`, driving an inverter over RS485.
(`eval(s, {"__builtins__": {}})` is not a sandbox and never was.)
* ⚠️ AND THE DECIDING ARGUMENT IS NOT SECURITY - AN EXPRESSION IS
UNVERIFIABLE. Two coefficients can be graded against the card's printed
estimate. An arbitrary expression cannot be graded against anything, so it
makes the invisible-wrong-coefficient problem STRICTLY WORSE, which is the
exact failure this provider exists to prevent.
Future shapes become NAMED FIELDS - `clamp_lo`, `factor_negative`,
`offset_bands` - each individually refusable and each individually gradeable
against a card. Never a grammar. `parse_price_formula()` therefore REFUSES AN
UNKNOWN KEY rather than ignoring it, so a user who types `"expr"` is told it
does not exist instead of being silently priced on the fields they also typed.
test_price.py section 13f walks this module's AST and fails if `eval`, `exec`,
`compile`, `__import__` or `literal_eval` ever appears in it.
WHERE THE NUMBERS COME FROM AND WHERE THEY GO
=============================================
`index` names the reference series. `epex_be` is the EPEX BE day-ahead curve in
EUR/MWh, which is what `generic_dayahead` already fetches - same feed, same
`brussels_date_range()` request, same columnar parser, same declared egress
hosts. No new parser and no new failure mode.
`resolution` says which series the contract is written against:
* `quarter_hourly` - the 96 points the feed publishes since Belgium's
15-minute MTU (2025-10-01), emitted as they are;
* `hourly` - the four quarters of each hour averaged into one point.
⚠️ NEITHER IS AN HOURLY ASSUMPTION IN `_Series.at()`'s SENSE (TARIFF-14). The
hourly option changes how many POINTS are emitted and their spacing; the window
each point covers is still derived from its neighbours by `_cover_table()`, and
nothing here tells anything downstream how long a point lasts. A feed that is
already hourly averages one member per hour, which is the identity - so the
option cannot silently assume the granularity it was handed.
`units` is NOT bureaucracy. Two suppliers publish the same shape in different
units and the market aggregator prints both conventions in one column, so the
add-on has to be told which one the card is in - and the unit conversion is the
add-on's job precisely because it is one of the two steps that silently costs a
factor of 10, 100 or 1000. The other is VAT, which is also the add-on's.
"""
import json
import math
from dataclasses import dataclass
from datetime import date, datetime, timezone
class FormulaInvalid(Exception):
"""`price_formula` cannot be used. Raised by the parser, refused by
validate.py at startup - never at 03:00 on a fetch."""
# --------------------------------------------------------------------------- #
# The enumerations. Every one of these is a CLOSED set, and that is the point:
# an unknown value is refused by name rather than guessed at.
# --------------------------------------------------------------------------- #
# unit -> how many EUR/kWh one of it is. ⚠️ THE WHOLE FACTOR-1000 CLASS LIVES
# HERE, so the table is tiny, explicit, and has no "default".
UNITS: dict = {
"c_eur_per_kwh": 0.01, # eurocent per kWh - Eneco and ENGIE print this
"eur_per_kwh": 1.0, # some aggregators print this
"eur_per_mwh": 0.001, # the wholesale convention, occasionally reused
}
RESOLUTIONS: tuple = ("hourly", "quarter_hourly")
# index id -> the human name of the series it refers to. One entry today; a
# second index is a new row here plus a fetch path, not a new grammar.
INDEXES: dict = {
"epex_be": "EPEX BE day-ahead, EUR/MWh",
}
# ⚠️ 0.01 IN THE CARD'S OWN UNITS, WHICH IS THE LAST DIGIT BOTH REAL CARDS
# PRINT. Eneco prints 15,55 against a computed 15.5471 (0.0029 out) and ENGIE
# prints 7,522 against a computed 7.5218 (0.0002 out), so the tolerance has to
# absorb the card's own rounding and nothing more. Every error class this check
# exists for - a factor of 10, of 100, of 1000, a missing or doubled 1.06, a
# transposed digit, ~12 c EUR/kWh of network charge folded into `offset` - is
# at least two orders of magnitude bigger than this.
CARD_TOLERANCE = 0.01
# The all-in plausibility band, EUR/kWh. `MAX_PLAUSIBLE_EUR_KWH` in price.py
# bounds only prices that are too LARGE; this is the other end, and it is the
# one the factor-1000 error falls off.
PLAUSIBLE_LO_EUR_KWH = 0.02
PLAUSIBLE_HI_EUR_KWH = 1.00
# ~6 months. ⚠️ A WARNING, NEVER A REFUSAL: ENGIE freezes a contract's
# coefficients for its duration, so a card two years old can be exactly right
# and a site refused over one would be a site that cannot start.
CARD_STALE_DAYS = 183
_CARD_KEYS = ("month", "index_eur_per_mwh", "expect_incl_vat")
_TOP_KEYS = ("index", "resolution", "factor", "offset", "units", "vat_pct",
"card")
@dataclass(frozen=True)
class FormulaSpec:
"""One contract's published arithmetic. SEVEN numbers and no code.
⚠️ EVERY FIELD IS A FIXED, NAMED SCALAR. There is deliberately no field
that holds an expression, a callable, a lambda or a template, and there is
no place to put one: `parse_price_formula()` refuses an unrecognised key.
"""
index: str
resolution: str
factor: float
offset: float
units: str
vat_pct: float
card_month: str # "YYYY-MM", as the card is dated
card_index_eur_per_mwh: float
card_expect_incl_vat: float # in `units`, as the card prints it
@property
def unit_to_eur_kwh(self) -> float:
return UNITS[self.units]
# --------------------------------------------------------------------------- #
# The arithmetic. Two functions, and the SECOND one is the contract.
# --------------------------------------------------------------------------- #
def in_card_units(spec: FormulaSpec, index_eur_per_mwh: float) -> float:
"""`(factor x index + offset) x (1 + vat_pct/100)`, in the card's units.
This is the line the card prints, spelled exactly as the card spells it,
and it is the only place the coefficients are combined. The card
self-check and the price path both go through here, so a mutation to it
cannot move the price without also moving the self-check.
"""
return ((spec.factor * float(index_eur_per_mwh) + spec.offset)
* (1.0 + spec.vat_pct / 100.0))
def price_eur_kwh(spec: FormulaSpec, index_eur_per_mwh: float) -> float:
"""The energy+markup price for one index value, in EUR/kWh.
⚠️ THE UNIT CONVERSION IS THE ADD-ON'S JOB AND IT HAPPENS HERE, ONCE. The
user types what their card prints, in the card's own units; every step
from there to EUR/kWh is code, because those steps are exactly the ones
that silently cost a factor of 10, 100 or 1000 when a human does them.
"""
return in_card_units(spec, index_eur_per_mwh) * spec.unit_to_eur_kwh
def card_discrepancy(spec: FormulaSpec) -> float:
"""How far the coefficients land from the card's own printed estimate.
In the card's units, signed: positive means the formula computes MORE than
the card prints, which is the direction an `offset` carrying network
charges moves it.
"""
return in_card_units(spec, spec.card_index_eur_per_mwh) \
- spec.card_expect_incl_vat
# --------------------------------------------------------------------------- #
# Reading the option
# --------------------------------------------------------------------------- #
def _finite(value, field: str) -> float:
"""A real, finite number - or a refusal naming the field.
⚠️ SAME CLASS AS `_validate_raw()`'s NaN RULE, one layer earlier. A NaN
factor makes every price NaN, every comparison False, and an arbitrage
strategy that silently never fires; `True` is an `int` in Python and would
price the whole day at the index.
"""
if isinstance(value, bool) or not isinstance(value, (int, float)):
raise FormulaInvalid(
f"price_formula.{field} is {value!r}, which is not a number. "
f"Write the coefficient as a bare JSON number - 0.102, not "
f"\"0.102\" and not an expression: this provider takes FIXED "
f"coefficients and evaluates nothing.")
if not math.isfinite(float(value)):
raise FormulaInvalid(
f"price_formula.{field} is {value!r}, which is not finite. A NaN "
f"or an infinity here makes every price NaN, every price "
f"comparison False, and an arbitrage strategy that never fires "
f"while reporting that it is armed.")
return float(value)
def _card(raw) -> tuple:
"""The `card` block: the estimate the coefficients are graded against."""
if raw is None:
raise FormulaInvalid(
"price_formula has no \"card\" block, and it is REQUIRED. It is "
"the only thing that can tell a correct pair of coefficients from "
"a plausible-looking wrong one: the add-on recomputes your "
"supplier's own printed example from the numbers you typed and "
"refuses to start if it does not come out. Copy the three figures "
"off the same tariff card as the coefficients: "
"{\"month\": \"2026-09\", \"index_eur_per_mwh\": 133.9905, "
"\"expect_incl_vat\": 15.55}.")
if not isinstance(raw, dict):
raise FormulaInvalid(
f"price_formula.card is {type(raw).__name__}, not an object with "
f"{', '.join(_CARD_KEYS)}.")
unknown = sorted(set(raw) - set(_CARD_KEYS))
if unknown:
raise FormulaInvalid(
f"price_formula.card has no field(s) {unknown}. The fields it "
f"takes are {list(_CARD_KEYS)}; a key that is merely ignored is a "
f"number the operator believes took effect.")
month = str(raw.get("month", "") or "").strip()
if not _month_date(month):
raise FormulaInvalid(
f"price_formula.card.month is {raw.get('month')!r}; write the "
f"month the card is dated as \"YYYY-MM\", e.g. \"2026-09\". It is "
f"not used in any price - it is what lets the add-on tell you "
f"your card has gone stale.")
return (month,
_finite(raw.get("index_eur_per_mwh"), "card.index_eur_per_mwh"),
_finite(raw.get("expect_incl_vat"), "card.expect_incl_vat"))
def _month_date(month: str):
"""`"2026-09"` -> the first of that month, or None if it is not one."""
try:
year, mon = str(month).split("-")
return date(int(year), int(mon), 1)
except (TypeError, ValueError):
return None
def parse_price_formula(value):
"""`price_formula` -> a `FormulaSpec`, or None when unset.
Accepts what the option can actually arrive as, exactly as
`parse_static_profile()` does:
* `None` / `""` / `"null"` - unset.
* a JSON string - what config.yaml's flat `str?` option
carries on a Supervisor install.
* a dict - already parsed, from a future nested
schema, from sim/inject.py, or a check.
Raises `FormulaInvalid`; never returns a half-built spec.
⚠️ THE CARD SELF-CHECK IS PART OF PARSING, not a separate step a caller
may forget. Both callers - validate.py's startup gate and the plugin's own
fetch - go through this one function, so there is no route to a price that
has not been graded against the card.
"""
if value is None:
return None
if isinstance(value, str):
text = value.strip()
if not text or text.lower() in ("null", "none", "off"):
return None
try:
value = json.loads(text)
except ValueError as e:
raise FormulaInvalid(
f"price_formula is not valid JSON ({e}). It is a JSON object "
f"on one line, e.g.\n"
f' {{"index":"epex_be","resolution":"hourly","factor":0.102,'
f'"offset":1.00,"units":"c_eur_per_kwh","vat_pct":6,'
f'"card":{{"month":"2026-09","index_eur_per_mwh":133.9905,'
f'"expect_incl_vat":15.55}}}}\n'
f"Leave it empty unless price_provider is 'custom_formula'.")
if not isinstance(value, dict):
raise FormulaInvalid(
f"price_formula is {type(value).__name__}, not a JSON object with "
f"{', '.join(_TOP_KEYS)}.")
# ⚠️ AN UNKNOWN KEY IS REFUSED, NOT IGNORED, AND THIS IS THE HALF THAT
# KEEPS THE "NEVER A GRAMMAR" PROMISE HONEST. A future shape is a NAMED
# field - clamp_lo, factor_negative, offset_bands - added here and
# refusable on its own. Ignoring unknown keys would mean a user who typed
# "expr" or "formula" gets priced on the fields they also typed, with the
# one they believed in dropped in silence.
unknown = sorted(set(value) - set(_TOP_KEYS))
if unknown:
raise FormulaInvalid(
f"price_formula has no field(s) {unknown}. The fields it takes "
f"are {list(_TOP_KEYS)}. ⚠️ There is deliberately NO field that "
f"takes an expression: this provider multiplies and adds two "
f"FIXED coefficients and evaluates nothing, because two "
f"coefficients can be graded against your card's printed price "
f"and an arbitrary expression cannot be graded against anything. "
f"If your contract needs a shape this cannot express, it needs a "
f"new named field here - say which shape.")
index = str(value.get("index", "") or "").strip()
if index not in INDEXES:
raise FormulaInvalid(
f"price_formula.index is {value.get('index')!r}. The reference "
f"series this add-on can fetch are "
f"{', '.join(f'{k!r} ({v})' for k, v in sorted(INDEXES.items()))}. "
f"If your card names a different index, the add-on cannot get it "
f"and guessing which series your supplier meant would price your "
f"battery against the wrong curve.")
resolution = str(value.get("resolution", "") or "").strip()
if resolution not in RESOLUTIONS:
raise FormulaInvalid(
f"price_formula.resolution is {value.get('resolution')!r}; it "
f"must be one of {list(RESOLUTIONS)}. Belgium has been on a "
f"15-minute MTU since 2025-10-01 and the feed publishes 96 points "
f"a day; a contract written against the HOURLY series is priced "
f"on the mean of each hour's quarters. Which one your contract "
f"uses is on your card and cannot be inferred from the feed.")
# ⚠️ `units` IS REQUIRED AND HAS NO DEFAULT. A default here would be the
# factor-1000 class with a friendly face: the add-on would silently assume
# a convention the card never stated, and the card self-check would then
# be graded in units nobody chose.
units = str(value.get("units", "") or "").strip()
if units not in UNITS:
raise FormulaInvalid(
f"price_formula.units is {value.get('units')!r} and is REQUIRED - "
f"there is no default, on purpose. It must be one of "
f"{sorted(UNITS)}. Your card prints the formula in ONE of these "
f"and two suppliers use two different conventions for the same "
f"shape, so assuming one would be a factor of 100 or 1000 on "
f"every hour with nothing able to notice.")
factor = _finite(value.get("factor"), "factor")
offset = _finite(value.get("offset"), "offset")
vat_pct = _finite(value.get("vat_pct"), "vat_pct")
# ⚠️ ZERO IS NOT "NO INDEXATION", IT IS A FIXED TARIFF WEARING A DYNAMIC
# ONE'S CLOTHES. Every hour comes out at the same price, so every spread is
# 0, arbitrage can never clear any threshold, and a strategy that never
# fires is indistinguishable from one that is switched off (§14's warning
# box). The add-on already has a first-class way to say "one price": point
# the operator at it rather than let them build it by accident.
if factor == 0.0:
raise FormulaInvalid(
"price_formula.factor is 0, so the index would not enter the "
"price at all: every hour of every day comes out at "
f"{offset:g} x (1 + {vat_pct:g}/100), every spread is exactly 0, "
"and arbitrage can never clear any threshold - a strategy that "
"silently never fires rather than one that is off. If your tariff "
"really is one flat price, that is `static_profile` with a single "
"period covering 00:00-24:00 on all days and price_provider "
"'off'; the add-on then says so at startup instead of leaving you "
"to find out from a year of unchanged bills.")
# ⚠️ A NEGATIVE FACTOR INVERTS THE MARKET, and the direction is the danger:
# the dearest hour becomes the cheapest one, so the battery is told to
# charge into the evening peak and to discharge at 03:00. That is not a
# mispriced site, it is a site actively trading against itself.
if factor < 0.0:
raise FormulaInvalid(
f"price_formula.factor is {factor:g}, which is NEGATIVE. That "
f"inverts the market: the dearest hour of the day comes out as "
f"the cheapest one, so the battery would charge at the evening "
f"peak and discharge at 03:00 - trading against itself on every "
f"cycle. No consumption tariff is indexed negatively; if your "
f"card shows a minus sign it is on the OFFSET (`A`), which is "
f"allowed here and common on injection.")
# ⚠️ 0 IS LEGITIMATE AND MUST STAY SO: injection is VAT-exempt in Belgium,
# and some cards are already quoted VAT-inclusive. The bound exists for the
# 106 / 1.06 / 6 confusion, not to make VAT compulsory.
if not 0.0 <= vat_pct <= 30.0:
raise FormulaInvalid(
f"price_formula.vat_pct is {vat_pct:g} and must be between 0 and "
f"30. It is a PERCENTAGE, so Belgian residential electricity is "
f"6, not 0.06 and not 106. Zero is legitimate and is not refused: "
f"injection is VAT-exempt, and a card that already quotes "
f"VAT-inclusive coefficients needs no second multiplication.")
month, card_index, card_expect = _card(value.get("card"))
spec = FormulaSpec(index=index, resolution=resolution, factor=factor,
offset=offset, units=units, vat_pct=vat_pct,
card_month=month, card_index_eur_per_mwh=card_index,
card_expect_incl_vat=card_expect)
delta = card_discrepancy(spec)
if abs(delta) > CARD_TOLERANCE:
raise FormulaInvalid(card_refusal(spec, delta))
return spec
def card_refusal(spec: FormulaSpec, delta: float) -> str:
"""The sentence an operator reads when their formula misses its own card.
⚠️ IT NAMES THE DISCREPANCY AND THE SIX WAYS IT HAPPENS. "Invalid
configuration" would be true and useless: every one of these errors looks
exactly like a working tariff from the status page, which is why the check
exists at all.
"""
got = in_card_units(spec, spec.card_index_eur_per_mwh)
ratio = (got / spec.card_expect_incl_vat
if spec.card_expect_incl_vat else float("inf"))
return (
f"price_formula does not reproduce its own card, so the add-on will "
f"NOT start on it. Your contract's arithmetic is "
f"price = (factor x index_eur_per_mwh + offset) x (1 + vat_pct/100); "
f"on the card's own index that is "
f"({spec.factor:g} x {spec.card_index_eur_per_mwh:g} + "
f"{spec.offset:g}) x (1 + {spec.vat_pct:g}/100) = {got:.4f} "
f"{spec.units}, but card.expect_incl_vat says "
f"{spec.card_expect_incl_vat:g} - OUT BY {delta:+.4f} {spec.units} "
f"(ratio {ratio:.4g}; the tolerance is {CARD_TOLERANCE}, which is the "
f"last digit a card prints). One of the two sides is wrong and the "
f"add-on cannot tell which, so it refuses rather than price a battery "
f"on it. The usual causes, in the order they happen: the coefficients "
f"and the expected price are in DIFFERENT units (ratio 10, 100 or "
f"1000 - `units` must be the one the formula line is printed in); VAT "
f"left out or applied twice (ratio 1.06 or 1/1.06); a transposed "
f"digit; the two sides copied off DIFFERENT cards or different "
f"months; or - the expensive one - network charges, levies or excise "
f"folded into `offset`. ⚠️ THEY DO NOT BELONG THERE: a Belgian card's "
f"Tariefformule line is the ENERGY component alone (Eneco tabulates "
f"network tariffs and levies separately; ENGIE says in so many words "
f"that the formula excludes network, distribution, transport, levies "
f"and excise), the add-on adds price_taxes_eur_kwh on top of it, and "
f"folding ~12 c EUR/kWh of network charge into `offset` bills you for "
f"it twice. That is the error this check exists to catch, and it is "
f"why it cannot be switched off.")
# --------------------------------------------------------------------------- #
# The two questions the PLUGIN cannot ask, because they need framework numbers
# --------------------------------------------------------------------------- #
def implausible_all_in(spec: FormulaSpec, taxes_eur_kwh: float):
"""The card's own point, if its LEVEL cannot be a Belgian retail price.
Returns a refusal sentence or None. ⚠️ CALLED FROM validate.py AND FROM
NOWHERE ELSE, because it reads a FRAMEWORK-owned number: a plugin that
could see `taxes` could add it a second time, which is price.py's trap 1.
⚠️ THE LOW END IS GRADED ON THE ENERGY COMPONENT AND THE HIGH END ON THE
ALL-IN FIGURE, AND THAT ASYMMETRY IS THE CHECK RATHER THAN AN OVERSIGHT.
`price_taxes_eur_kwh` ships at 0.11736 (TARIFF-18; it was 0.06 before, and
the argument was already decisive at that figure), which is nearly SIX
times the low bound - so a card read a HUNDRED times too small comes out at
0.0016 EUR/kWh of energy, is carried over the floor by the flat tax addend
alone, and passes a band applied to the all-in total. The hundredfold error
is precisely what this exists to catch, so the floor is applied where it
can still see it. ⚠️ Raising the default made this trap WIDER, not
narrower: the addend that does the carrying got bigger. The ceiling is
applied to the all-in figure because that is the number the site is really
billed at, and taxes only push it up.
⚠️ AND THE LIMIT, WITH THE CLAIM. This grades the CARD'S OWN POINT - one
price, at the monthly index the card assumed - and not every hour of every
day. A live EPEX hour can and should fall outside this band (a negative
hour is the best charging opportunity of the year, and §17 keeps a
regression case for it), so the band is deliberately NOT applied to the
fetched series. What it catches is a formula whose whole LEVEL is wrong,
which is the factor-of-1000 class arriving from the end
`MAX_PLAUSIBLE_EUR_KWH` does not bound.
"""
energy = price_eur_kwh(spec, spec.card_index_eur_per_mwh)
all_in = energy + float(taxes_eur_kwh)
if energy >= PLAUSIBLE_LO_EUR_KWH and all_in <= PLAUSIBLE_HI_EUR_KWH:
return None
return (
f"price_formula's own card works out at {energy:.5f} EUR/kWh of "
f"energy ({spec.card_expect_incl_vat:g} {spec.units} as the card "
f"prints it) and {all_in:.5f} EUR/kWh all-in once "
f"price_taxes_eur_kwh is added, which is outside the plausible band "
f"{PLAUSIBLE_LO_EUR_KWH}-{PLAUSIBLE_HI_EUR_KWH} EUR/kWh for a Belgian "
f"retail tariff. The card self-check passed, so the coefficients and "
f"the printed price AGREE - they simply agree on a number no Belgian "
f"bill carries, which is what a whole card read in the wrong units "
f"looks like (a factor of 10, 100 or 1000 applied to BOTH sides at "
f"once cancels out of that check and cannot cancel out of this one). "
f"Check `units` against the unit printed beside your formula. "
f"⚠️ This bounds the CARD'S OWN POINT, not every hour: a live hour may "
f"legitimately fall outside the band, and a negative day-ahead hour - "
f"the best charging opportunity of the year - is never refused.")
def stale_card(spec: FormulaSpec, today=None):
"""A warning when the card is older than ~6 months, or None. NEVER a
refusal: ENGIE freezes a contract's coefficients for its duration, so an
old card can be exactly right and refusing one would stop the add-on on a
correctly configured site."""
dated = _month_date(spec.card_month)
now = today or datetime.now(timezone.utc).date()
if dated is None or (now - dated).days <= CARD_STALE_DAYS:
return None
return (
f"price_formula's card is dated {spec.card_month}, which is "
f"{(now - dated).days} days ago. Suppliers reprice: if your contract "
f"has been renewed or rolled over since then, the coefficients here "
f"are last year's and every price this add-on computes is wrong by "
f"however far they moved - and the card self-check cannot see it, "
f"because it grades your coefficients against the card you typed, not "
f"the card against your current contract. Not refused: ENGIE freezes "
f"coefficients for the contract's duration, so an old card is often "
f"exactly right. Worth re-reading your klantenzone once.")
# --------------------------------------------------------------------------- #
# Resolution
# --------------------------------------------------------------------------- #
def to_resolution(pairs, resolution: str) -> list:
"""`[(instant, index EUR/MWh)]` -> the series the CONTRACT is written on.
`quarter_hourly` hands the feed's own points back, ascending.
`hourly` averages each hour's members into one point AT THE TOP OF THE
HOUR.
⚠️ THE BUCKET IS A UTC HOUR AND THAT IS EXACT FOR BELGIUM, not an
approximation: CET and CEST are both WHOLE-hour offsets, so a Brussels hour
and a UTC hour start at the same instant on every day of the year including
both transition days. Keying by a naive local hour is what collides the two
02:00s in October; keying by the UTC instant cannot.
⚠️ AND THIS IS NOT `_Series.at()`'s HOURLY ASSUMPTION COMING BACK
(TARIFF-14). It decides how many points are EMITTED, not how long one
covers - the coverage window is still derived from the neighbours in
`_cover_table()`. A feed that already publishes hourly points averages one
member per hour, i.e. the identity, so this cannot silently assume the
granularity it was handed.
⚠️ AN INCOMPLETE HOUR IS AVERAGED OVER WHAT IS THERE rather than dropped or
zero-filled. A horizon that starts at 09:20 has two quarters in its first
hour; their mean is the best statement available about that hour, and
dropping it would leave a hole `at()` answers with `Absence` for an hour
the feed did publish.
"""
ordered = sorted(pairs, key=lambda r: r[0].astimezone(timezone.utc))
if resolution == "quarter_hourly":
return list(ordered)
if resolution != "hourly":
raise FormulaInvalid(f"unknown resolution {resolution!r}")
buckets: dict = {}
for start, value in ordered:
key = start.astimezone(timezone.utc).replace(
minute=0, second=0, microsecond=0)
buckets.setdefault(key, []).append(value)
return [(k, sum(v) / len(v)) for k, v in sorted(buckets.items())]
# --------------------------------------------------------------------------- #
# The startup preview
# --------------------------------------------------------------------------- #
PREVIEW_HOURS = 24
PREVIEW_MAX_ROWS = 24
def preview_lines(spec: FormulaSpec, rows) -> list:
"""`[(instant, index EUR/MWh, all-in EUR/kWh)]` -> lines for the log.
THE DEFECT THIS EXISTS TO MAKE VISIBLE: every error this module refuses is
one the operator cannot see from the status page, and the two numbers that
would have shown it - the index that went IN and the price that came OUT -
are never printed beside each other anywhere else. A factor of 1000 is
obvious in one line of `61.96 -> 0.13522` and invisible in a plan.
⚠️ IT PRINTS A BOUNDED NUMBER OF ROWS AND SAYS SO. At `quarter_hourly` the
next 24 h is 96 points; the cap is a row COUNT rather than "every fourth
point", because sampling every fourth point would assume the granularity
TARIFF-14 spent a card removing the assumption of.
"""
if not rows:
return []
shown = rows[:PREVIEW_MAX_ROWS]
prices = sorted(r[2] for r in rows)
mid = len(prices) // 2
median = (prices[mid] if len(prices) % 2
else (prices[mid - 1] + prices[mid]) / 2.0)
out = [
f"custom_formula preview - the next {PREVIEW_HOURS} h of YOUR tariff, "
f"index EUR/MWh -> all-in EUR/kWh, computed as "
f"({spec.factor:g} x index + {spec.offset:g}) x "
f"(1 + {spec.vat_pct:g}/100) {spec.units}, converted to EUR/kWh and "
f"with price_taxes_eur_kwh added by the framework:"]
for start, index, all_in in shown:
out.append(f" {start.isoformat()} {index:9.3f} -> {all_in:9.5f}")
if len(rows) > len(shown):
out.append(f" ... {len(rows) - len(shown)} further point(s) not "
f"listed; the figures below cover ALL {len(rows)}")
out.append(
f" min {prices[0]:.5f} / median {median:.5f} / max {prices[-1]:.5f} "
f"EUR/kWh, spread {prices[-1] - prices[0]:.5f} EUR/kWh over "
f"{len(rows)} point(s). ⚠️ If those are a thousand times what your "
f"bill says, `units` is wrong; if the spread is 0, `factor` is.")
return out
+356 -20
View File
@@ -67,7 +67,8 @@ from .mqtt import (MqttPublisher, announced_units, unit_slugs,
from . import peakguard
from . import packshare
from .p1 import P1Ingest, build_source, is_enabled
from .price import Uncertainty, price_option
from .price import Uncertainty, price_option, provider_configured
from .price_history import PriceHistory
from .static_tariff import resolve_tariff, static_status
from .tariff import PeakStore, PeakTracker
from .store import migrate_maint_keys, migrate_unit_keys
@@ -137,6 +138,60 @@ TARIFF_REFRESH_S = 1800.0
TARIFF_MAX_AGE_S = 3 * TARIFF_REFRESH_S
def tariff_refresh_wanted(opts: dict) -> bool:
"""TARIFF-10. Does this install need the background tariff refresh at all?
THE DEFECT THIS PREVENTS, and it shipped: the gate below used to be
`arbitrage.enabled(opts)` alone. That option defaults to `false`, and
`run_tariff_refresh` is the only caller of `resolve_tariff()`, which is the
only caller of `price_history.carry()` - so on a shipped install with a
provider configured NOTHING was ever fetched, no plan was ever resolved,
`/data/price_history.json` was never created, the *Price history* row read
`empty` forever, and TARIFF-09's short-horizon warning could not fire
because there was no fetch to warn about. Measured over a 40 h soak of two
instances side by side: zero feed requests and `present: false` across all
802 samples on the shipped default, against a resolved plan from the same
feed on the instance with arbitrage armed. ⚠️ AND `DOCS.md` HAS
DESCRIBED THE BEHAVIOUR BELOW SINCE THE COMMIT THAT WROTE THE GATE:
OPT-04 (bdd7f18) added the refresh task, the `arbitrage.enabled()` guard on
it, and the sentence saying a configured provider still fetches, all in one
change. config.yaml has said it since TARIFF-02 (852fc7c).
Two terms, and they answer two different questions:
provider_configured(opts) -> "the operator asked for a price feed", so
fetch it, validate it and record the day
(§8.3). This is the term TARIFF-10 ADDS.
arbitrage.enabled(opts) -> "the operator armed trading". This term is
what the gate USED to be, kept verbatim.
⚠️ IT IS AN `or`, SO THE GATE IS A STRICT WIDENING. Every configuration
that started this task before still starts it; none that did not start it
stops anything. That matters because the second term is not redundant:
`price_provider: off` + a `static_profile` + `arbitrage_enabled: true` is a
configuration validate.py explicitly ACCEPTS, and it trades - measured, at
`cycle_cost_threshold_eur: 0.05` on a 0.20/0.284 day/night profile,
`ARBITRAGE_CHARGE -5000 W`. Dropping that term to make the gate read
`provider_configured(opts)` alone would leave `self.tariff_plan` at None on
that site forever and `decide()` would answer "no tariff plan has been
resolved yet" on every cycle - silently disarming a strategy the UI says is
running, which is the exact failure validate.py's OPT-04 rules exist to
refuse.
⚠️ AND NO CONFIGURATION ACQUIRES A PERIODIC HTTPS CALL FROM THIS TICKET.
With `price_provider` off, `resolve_tariff()` returns `static_plan()` - pure
arithmetic, no socket - so the second term starts a task that fetches
nothing. The true shipped default (no provider, no arbitrage) starts no
task at all. What is fetched is decided by the FIRST term only.
⚠️ THIS IS NOT A TRADING SWITCH. It decides whether prices are LOOKED UP.
Whether they move energy is `arbitrage.enabled()` at the one call site in
`cycle()`, and TARIFF-10 did not touch it: `arbitrage_enabled: false` still
means no trade, ever, on any configuration.
"""
return provider_configured(opts) or arbitrage.enabled(opts)
def load_options() -> dict:
try:
with open(OPTIONS_PATH, "r", encoding="utf-8") as fh:
@@ -163,9 +218,110 @@ def pack_energy_of(inv):
inv.capacity_kwh, inv.max_w)
def tariff_plan_row(plan, tick_words: str, *, age_s, horizon_h: float,
provider: bool, max_age_s: float) -> str:
"""The *Tariff* status row: the 30 s tick's sentence AND the provider plan.
THE DEFECT THIS PREVENTS (TARIFF-11). `origin`, `product`, the horizon,
the firm/carried split and the spacing appeared in NO status row. The row
was `static_status()`'s sentence alone, computed on the 30 s I/O-free tick
from its own static plan, and it NEVER SAW the provider plan - it read
`'no static profile'` at all 802 samples of a 40-hour soak on a site whose
feed was answering. The only external evidence a plan existed at all was
one `_LOG.info` line, and that line carried the point<->hour mislabel
`TariffPlan.words` is fixed for.
⚠️ TWO SOURCES, TWO CLOCKS, AND THE ROW SAYS SO IN WORDS. These sentences
are NOT computed at the same moment and presenting them as though they
were would be a new version of the defect being fixed:
`tick_words` static_status() / the Home Assistant price entities,
regenerated on THIS tick, every 30 s, I/O-free;
`plan` the background refresh's resolution, up to
TARIFF_REFRESH_S old before it is retried and up to
TARIFF_MAX_AGE_S (5400 s since TARIFF-12) old before
`fresh_tariff_plan()` retires it.
So the plan half is stamped with its OWN age in seconds, measured from
`tariff_at` - the moment of the fetch that produced it, which TARIFF-12
deliberately does not move on a kept plan.
⚠️ AND `age_s` IS THE CALLER'S MEASUREMENT, NOT THIS FUNCTION'S. A clock
read in here would be a second clock disagreeing with the one that decided
the plan was fresh enough to pass at all.
⚠️ "NO PLAN" IS SAID OUT LOUD ON A SITE THAT CONFIGURED A PROVIDER. A
dynamic site whose refresh has never completed, or whose plan has aged
past `max_age_s`, is the exact state the soak measured; rendering it as
the bare static sentence is how it stayed invisible for 802 samples.
"""
if plan is None:
if not provider:
return tick_words
return (f"{tick_words} (recomputed this tick); NO provider plan - "
f"none has been resolved yet, or the last one is older than "
f"{max_age_s:.0f} s")
return (f"{tick_words} (recomputed this tick); provider plan fetched "
f"{age_s:.0f} s ago for a {horizon_h:.0f} h horizon: {plan.words}")
def arbitrage_row(*, on: bool, decision, decided_at, cycle_at, now,
why: str) -> str:
"""The *Arbitrage* status row: THREE states, which used to render as two.
THE DEFECT THIS PREVENTS (TARIFF-11). The row was
`"off (arbitrage_enabled)" if self.arb is None else self.arb.explain()`,
under a comment asserting that `self.arb is None` if and only if
`arbitrage.enabled(self.o)` was False. IT IS NOT: `cycle()` returns at the
input hold - inputs missing, `bad_for < stale_input_s`, default 15 s -
BEFORE it reaches `self.arb = None`. Driven: `arbitrage_enabled: true`
plus a sensor answering `None` gives `arbitrage.enabled(opts) = True`,
`self.arb = None` (its `__init__` value, untouched) and a row reading
`off (arbitrage_enabled)` - NAMING AN OPTION THAT IS ON. Past the first
completed cycle the same early return leaves the PREVIOUS decision
standing instead, which the row then rendered as a current one.
THE THREE FACTS, and they are three because they have three remedies:
trading is off the operator has not armed it. Nothing is
wrong and nothing will happen.
no decision this cycle it is armed, and the cycle did not get as far
as the arbitrage step. `why` is the control
loop's own reason - usually "inputs missing".
a decision the strategy's own sentence, with the age of
the cycle that produced it.
⚠️ `on` IS READ FROM THE OPTIONS, NOT INFERRED FROM `decision is None`.
Inferring it is the whole defect: the option is the only thing that
answers "is trading armed", and it is available on every render.
⚠️ THE AGE IS ALWAYS PRINTED, INCLUDING WHEN IT IS ZERO. A row that
reports staleness only once it is stale cannot be told, on any single
render, from one that never reports it - and the bound TARIFF-10's
comment claimed did not exist is ~15 s of it.
"""
if not on:
return "off - `arbitrage_enabled` is false, so nothing trades"
if decided_at is None:
return (f"ON (`arbitrage_enabled`), but NO DECISION has been reached "
f"yet - no cycle has run as far as the arbitrage step. The "
f"loop's own reason: {why or 'none recorded'}")
age = (now - decided_at).total_seconds()
stale = cycle_at is not None and decided_at < cycle_at
head = (f"no decision this cycle ({why or 'no reason recorded'}); the "
f"last one was taken {age:.0f} s ago: " if stale else
f"decided {age:.0f} s ago: ")
if decision is None:
return (f"ON (`arbitrage_enabled`), no decision this cycle "
f"({why or 'no reason recorded'}), and none stands from an "
f"earlier one")
return head + decision.explain()
class Controller:
def __init__(self, opts: dict, hass: HomeAssistant, store, mqtt_pub, *, stop,
peak_db_path: str = "/data/tariff.db"):
peak_db_path: str = "/data/tariff.db",
price_history_path: str = "/data/price_history.json"):
self.o = opts
self.hass = hass
self.store = store
@@ -582,6 +738,42 @@ class Controller:
self.peak.restore(self.p1.averager)
_LOG.info("capacity-tariff peak store: %s", self.peak.restore_state)
# FCST-04. The §8.3 carry-forward store: the published prices of the
# last few Brussels days, so the hours a day-ahead feed has not printed
# yet can be filled from yesterday's profile and FLAGGED, instead of
# being absent from the series entirely.
#
# ⚠️ IT IS AN INSTRUMENT'S BACKING FILE, NOT A STRATEGY. It commands
# nothing and claims nothing; what it changes is how many hours the
# plan covers. Before this, a dynamic site between midnight and the
# 14-16 h CET publication had a series covering only the rest of today
# - typically 10-24 of the 48 hours `price_horizon_h` asked for - and
# `TariffPlan.arbitrage()` took its spread over that truncated window
# without anything reporting that it had. Every carried hour is
# provisional, so refusal 3 drops it from the trade set: CARRYING a
# price widens what the optimiser can SEE, never what it may act on.
#
# ⚠️ AND THE CAVEAT THAT BELONGS WITH THAT SENTENCE, because this is
# the one place a maintainer reads it on its own. THE OTHER HALF OF
# THIS CHANGE DOES MOVE ENERGY, and it is the windowing rather than the
# carrying: the filled series covers `[since, until)`, so hours that
# have ALREADY ELAPSED are no longer in the spread. A cheap night that
# has gone used to sit in the series as the day's cheapest hour, and no
# cycle can buy in an hour that is past - so the trade was quietly
# never taken. `resolve_tariff()`'s docstring, DOCS.md and the CHANGELOG
# all carry this beside the guarantee; it may not appear here without it.
#
# ⚠️ A BROKEN OR ABSENT STORE IS NOT AN ERROR HERE. It answers `no_data`,
# the fill carries nothing, and the plan is the pre-FCST-04 one minus
# the elapsed hours - short, but honest and still refusing on the same
# rules. It must not be an error in the other sense either: a day
# record this build cannot read is DROPPED by `PriceHistory._load()`
# and reported in `.words`, because both the constructor above and the
# `.words` read below are unguarded, and a cache that raises here is an
# add-on that does not start.
self.price_history = PriceHistory(price_history_path)
_LOG.info("§8.3 price history: %s", self.price_history.words)
# SAFETY-07. The guard TARIFF-01 was built for. It is stateless by
# design (peakguard.py holds no history between calls), so the only
# state here is what an EDGE needs: what the last cycle concluded, so
@@ -704,6 +896,19 @@ class Controller:
self.tariff_plan = None
self.tariff_at = None
self.arb = None
# TARIFF-11. WHEN the standing arbitrage decision was taken, or None if
# this controller has never reached the arbitrage step. ⚠️ `self.arb is
# None` DOES NOT MEAN "TRADING IS OFF" AND NEVER DID - see
# `arbitrage_row()`. It also means "cycle() returned before the step
# ran", and at that point `self.arb` is either this attribute's initial
# None (startup) or the PREVIOUS cycle's decision (an input hold), and
# the status row rendered all three identically.
self.arb_at = None
# TARIFF-11. When the most recent cycle() STARTED. `arb_at < cycle_at`
# is the exact, derived statement of "the latest cycle did not get as
# far as the arbitrage step", which is the fact the *Arbitrage* row
# used to render identically to "trading is switched off".
self.cycle_at = None
self._arb_action = None
self.inputs_bad_since = None
# SAFETY-03. What the loop's OWN bounds did last cycle - a
@@ -1473,6 +1678,24 @@ class Controller:
self.cheap, self.tariff_words = static_status(
o, datetime.now(timezone.utc))
# ⚠️ TARIFF-11. THE PROVIDER PLAN REACHES THE *Tariff* ROW HERE, AND
# THIS IS THE ONLY PLACE THE TWO SOURCES MEET. Everything above is the
# 30 s I/O-free tick; `fresh_tariff_plan()` is an attribute read and an
# age comparison against TARIFF_MAX_AGE_S - still no fetch on this path,
# which is the rule the whole branch above exists to keep. What the row
# gains is the plan's SHAPE (origin, product, horizon, the firm/carried
# split, the spacing) and what it must never gain is the suggestion
# that the two halves were computed at the same instant, so the plan
# half carries its own age in seconds. See `tariff_plan_row()`.
_tick = datetime.now(timezone.utc)
_standing = self.fresh_tariff_plan(_tick)
self.tariff_words = tariff_plan_row(
_standing, self.tariff_words,
age_s=(None if _standing is None
else (_tick - self.tariff_at).total_seconds()),
horizon_h=float(price_option(o, "price_horizon_h")),
provider=provider_configured(o), max_age_s=TARIFF_MAX_AGE_S)
if self.inputs_ok:
if self.inputs_bad_since is not None:
self.log_event("inputs recovered")
@@ -1826,6 +2049,11 @@ class Controller:
method's only job is to state honestly what each strategy wants.
"""
now = datetime.now(timezone.utc)
# ⚠️ BEFORE THE FIRST `return` OF THIS METHOD, ON PURPOSE (TARIFF-11).
# It stamps that a cycle RAN, which is a different fact from any of the
# decisions the cycle may or may not reach, and the status row needs
# the two separately.
self.cycle_at = now
# ⚠️ EACH PACK ON ITS OWN READING, NOT "the SoC". §12's recalibration
# drains and refills ONE pack and grades ONE coulomb counter, so each
@@ -2485,7 +2713,13 @@ class Controller:
# `frozenset()` would be main.py asserting a forecast is healthy when
# there is not one, which is exactly what static_tariff's `uncertainty`
# argument has no default in order to prevent.
self.arb = None
# ⚠️ TARIFF-11: BOTH ATTRIBUTES, AND `arb_at` IS WHAT MAKES THE ROW
# HONEST. Reaching this line is the only proof that this cycle got as
# far as the arbitrage step; the early return at the input hold above
# leaves BOTH standing at whatever the last completed cycle left, and
# `arb_at` is then the age the status row reports rather than a
# decision presented as current.
self.arb, self.arb_at = None, now
if arbitrage.enabled(self.o):
floor_pct, ceiling_pct = arbitrage.soc_bounds(self.o)
charge_w, discharge_w = self.trade_power_w(floor_pct, ceiling_pct)
@@ -3442,25 +3676,99 @@ class Controller:
raise out of here and take the task down, because then the plan freezes
at its last value and looks current forever. That is what
`fresh_tariff_plan()`'s age bound is the backstop for, and this is the
half that keeps the task alive so the bound rarely matters.
half that keeps the task alive. ⚠️ THE BOUND IS NO LONGER A BACKSTOP
ONLY: since TARIFF-12 it is also the serve-stale window an outage runs
on, and it is consulted below on every refresh.
⚠️ AND A FAILURE DOES NOT CLEAR THE PLAN. `resolve_tariff()` already has
a §11.4 answer for a dead feed - degrade to the static profile, mark it
provisional, arbitrage refuses - and that answer is better than the
absence of one. Zeroing the plan here would replace a coarse tariff with
no tariff, which is the opposite of what §11.4 asks for.
⚠️ AND A FAILURE DOES NOT CLEAR THE PLAN - WHICH IS A CLAIM ABOUT WHAT
`resolve_tariff()` RETURNS, NOT ABOUT WHAT IT RAISES (TARIFF-12). This
paragraph was enforced by the `except` below alone, and the `except`
below is the one path a supplier outage never takes. `load_prices()`
says so in its own words - it "RETURNS, NEVER RAISES, for every failure
a site can actually have" - because a degradation arriving as a
traceback into the planning tick would stop the controller regulating
over an expired certificate. So a dead feed is a RETURN: the `try`
succeeds, `_degrade()` hands back a §11.4 plan, and the assignment used
to overwrite the good one unconditionally. Measured in a 40 h soak with
`static_profile` unset (the shipped default, and the normal state of a
dynamic-tariff site): the FIRST failed fetch replaced a complete 48 h
plan with `TariffPlan(no_data, None, "none")` - 0 points - 27 minutes
after the last good one, and `TARIFF_MAX_AGE_S` governed nothing. A
transient blip cost the same as a 90-minute outage.
⚠️ SO THE QUESTION IS ASKED ONCE, AFTER THE ATTEMPT, FROM THE EVIDENCE
THAT EXISTS - FCST-06's answer, in this file. The evidence is
`plan.from_feed` (did this attempt observe the feed) and
`fresh_tariff_plan(now)` (is an observation still current). A
degradation replaces a standing observation only once that observation
has aged out, so `TARIFF_MAX_AGE_S` is now the serve-stale window it
reads like rather than a bound nothing could reach.
⚠️ AND A §11.4 STATIC FALLBACK DOES NOT REPLACE A FRESH PROVIDER PLAN
EITHER, although it carries real prices. Every point `_degrade()`
builds is `provisional=True`, so `firm_points()` is empty,
`arbitrage()` refuses and `cheap_now()` is False - everything a
consumer may act on is already off. A day-ahead price for a given hour
does not change once published, so a plan fetched half an hour ago is
exactly as true about that hour as one fetched now. The fallback is
less actionable and no more current; it takes over when the observation
retires, which is what §11.4 is for. The reverse is not symmetrical: a
standing plan that is ITSELF a degradation is replaced by a fresh one,
or an outage would freeze the operator's static profile at the window
it was generated for.
⚠️ AND NOTHING HERE READS `arbitrage_enabled`, WHICH TARIFF-10 MAKES
WORTH SAYING. Since that card this task also runs on a site with a
provider configured and trading DISARMED, so "keep the plan" is a
decision about what the add-on has LOOKED UP, not about what it may
trade on. It is also strictly downstream of `resolve_tariff()`: the
§8.3 carry into `price_history` happens on the `series` branch inside
that call, before this decision exists, so which of two plans is kept
cannot add or lose a recorded day.
⚠️ AND `tariff_at` DOES NOT MOVE ON A KEPT PLAN. It is the moment the
fetch that produced the standing plan happened, and
`fresh_tariff_plan()` judges the age from it - re-stamping it here
would be this task asserting the feed answered when it did not, and the
plan would "freeze at its last value and look current forever", which
is the hazard the paragraph above this one names. An unmoved
`tariff_at` is also the only thing that tells a kept plan from a
freshly fetched one, so it is what the check asserts.
"""
horizon = float(price_option(self.o, "price_horizon_h"))
while not self.stopping:
now = datetime.now(timezone.utc)
try:
plan = await resolve_tariff(
self.o, now, now + timedelta(hours=horizon), now=now)
self.tariff_plan, self.tariff_at = plan, now
_LOG.info("tariff refreshed: %s", plan.words)
self.o, now, now + timedelta(hours=horizon), now=now,
history=self.price_history)
standing = self.fresh_tariff_plan(now)
# Three terms, and each of them is the only false one in some
# check: the feed answered (replace), nothing current stands
# (take the degradation - this is what makes the age bound
# govern), or what stands is itself a degradation (replace).
keep = (not plan.from_feed
and standing is not None and standing.from_feed)
if keep:
_LOG.warning(
"tariff refresh degraded (%s); KEEPING the plan "
"resolved at %s, %.0f s ago. tariff_at is NOT moved, "
"so it retires %.0f s after that fetch; arbitrage, "
"where it is armed, stops with it. A kept plan is "
"never presented as a current one.", plan.words,
self.tariff_at,
(now - self.tariff_at).total_seconds(),
TARIFF_MAX_AGE_S)
else:
self.tariff_plan, self.tariff_at = plan, now
_LOG.info("tariff refreshed: %s", plan.words)
except Exception as err: # noqa: BLE001
# ⚠️ NOT THE OUTAGE PATH - see the docstring. `load_prices()`
# returns on every failure a site can have, so what reaches
# here is something nobody expected, and the plan is kept for
# the same reason and on the same terms.
_LOG.warning(
"tariff refresh failed (%s); keeping the plan resolved at "
"tariff refresh raised (%s); keeping the plan resolved at "
"%s. It retires on its own after %.0f s and arbitrage "
"stops with it.", err, self.tariff_at, TARIFF_MAX_AGE_S)
with contextlib.suppress(asyncio.TimeoutError):
@@ -4304,11 +4612,34 @@ class Controller:
("Maintenance due", "now" if next_due is None else next_due.strftime("%Y-%m-%d")),
("Cheap window", "yes" if self.cheap else "no"),
("Tariff", self.tariff_words),
# FCST-04. WHAT §8.3 HAS TO CARRY FORWARD, which is the difference
# between a 48-hour plan and a 12-hour one on a weekday morning.
# It is its own row rather than a clause of "Tariff" because it
# answers a different question: "Tariff" says what the plan IS, and
# this says what the plan could still be filled from if the feed is
# late again tomorrow.
("Price history", self.price_history.words),
# OPT-04. The strategy's own sentence, not a re-derivation of it -
# every refusal it can make already carries a reason a human can
# act on, and "Arbitrage: no" would throw all of them away.
("Arbitrage", "off (arbitrage_enabled)" if self.arb is None
else self.arb.explain()),
#
# ⚠️ TARIFF-10 RE-READ THIS STRING AND LEFT IT ALONE UNDER A
# COMMENT THAT WAS FALSE, AND TARIFF-11 IS THAT CORRECTION. The
# claim was that `self.arb is None` **if and only if**
# `arbitrage.enabled(self.o)` was False. Only the forward direction
# holds: `cycle()` returns at the input hold (`bad_for <
# stale_input_s`, default 15 s) BEFORE it reaches `self.arb = None`
# at all, so on an armed site `self.arb` is `__init__`'s None at
# startup and the PREVIOUS cycle's decision during a later hold -
# and the row rendered `off (arbitrage_enabled)` on a site whose
# `arbitrage_enabled` is `true`. `arbitrage_row()` reads the option
# instead of inferring it, and separates "trading is off" from "no
# decision was reached this cycle"; `cycle_at` and `arb_at` are what
# make the second one an observation rather than a guess.
("Arbitrage", arbitrage_row(
on=arbitrage.enabled(self.o), decision=self.arb,
decided_at=self.arb_at, cycle_at=self.cycle_at,
now=datetime.now(timezone.utc), why=self.reason)),
]
return {
"banner": banner, "level": level, "rows": rows, "checks": checks,
@@ -4559,11 +4890,16 @@ async def amain() -> None:
# ESP32 heartbeat. A fetch on that path is a supplier's TLS timeout able
# to push this add-on towards layer 1's 30 s veto. See TARIFF_REFRESH_S.
#
# ⚠️ STARTED ONLY WHEN ARBITRAGE IS ARMED. Nothing else in the add-on
# reads the resolved plan - `cheap` still comes from static_status() on
# the tick, which is pure - so on a default install this task would be
# network traffic in aid of an attribute nobody reads.
if arbitrage.enabled(opts):
# ⚠️ STARTED WHEN A PROVIDER IS CONFIGURED, *OR* WHEN TRADING IS ARMED
# - TARIFF-10, and it used to be the second term alone. `arbitrage_
# enabled` ships `false`, so a site that had selected a price feed
# fetched nothing at all; the whole argument, the measurement and what
# the `or` is protecting are in `tariff_refresh_wanted()` above.
#
# ⚠️ IT DOES NOT ARM ANYTHING. Fetching a price and trading on one are
# two decisions; this is the first, and `cycle()`'s
# `if arbitrage.enabled(self.o):` is still the only gate on the second.
if tariff_refresh_wanted(opts):
tasks.append(asyncio.create_task(controller.run_tariff_refresh()))
# P1 ingestion runs as its own long-lived task. ⚠️ It must not be driven
File diff suppressed because it is too large Load Diff
+197 -36
View File
@@ -48,23 +48,42 @@ So the flag is not passed through anywhere in this file:
the object the provider produced is the object in the output series - so
there is no copy that could lose or gain a flag. `filled.points[i] is
fetched.points[j]` is a checkable property, and test_price_forecast.py
checks it.
checks it. EVERY published point in the window is passed through, not one
per hour: see the sub-hourly note below.
* `_audit()` RE-DERIVES the expected flag for every output point from the set
of hours the feed actually published, and RAISES if any point disagrees.
It is a post-condition, not a filter: it never fixes a flag, because a
module that quietly corrects its own provenance is one whose provenance
cannot be trusted.
⚠️ IT GRADES THE POINTS THAT ARE EMITTED AND NOTHING ABOUT THE ONES THAT
ARE NOT (FCST-05). `_audit()` is a provenance check, NOT a completeness
check: fed FCST-04's own decimation defect - four published points in an
hour, only the last emitted - it ACCEPTS, and it accepts an empty output
too. Measured, and the FCST-03 audit it replaced was equally blind, so
this is a limit rather than a regression. What actually guards against a
decimated series is the point COUNT and the pass-through IDENTITY checks
in `test_price_forecast.py` (section on the quarter-hourly feed) and
`test_price_history.py`; see `_audit()`'s own docstring for why the
completeness rule is not expressible from its two arguments.
* and test_price_forecast.py walks THIS FILE'S AST and fails if any
`PricePoint(...)` in it passes anything but a literal for `provisional`, so
the pass-through idiom cannot come back in a later edit and pass review by
looking ordinary.
⚠️ "PUBLISHED" DOES NOT MEAN "FIRM". `eneco_eddy` is an imbalance product whose
current-quarter print is genuinely provisional and says so per point (§9.2,
TARIFF-02). Such a point is published AND provisional, and the audit's rule for
a published point is "the flag is whatever the feed said", never "False". A
provenance check that forced published points to False would be a bug that
looks like a guarantee.
⚠️ "PUBLISHED" DOES NOT MEAN "FIRM". §9.3's canonical format carries
`provisional` PER POINT, not per series, precisely because a real-time /
imbalance product's current-interval print is settled after the fact and is
genuinely provisional while it is published. Such a point is published AND
provisional, and the audit's rule for a published point is "the flag is
whatever the feed said", never "False". A provenance check that forced
published points to False would be a bug that looks like a guarantee.
⚠️ NO SHIPPED PLUGIN EMITS ONE TODAY (TARIFF-07 withdrew the only declarer of
`PriceProduct.REALTIME`), and that is exactly why this paragraph stays. The
rule is a property of the FORMAT, not of any supplier; weakening it to
"published implies False" while nothing exercises it is how the first real
imbalance feed would arrive mispriced and flagged firm.
THE FILL, AND WHY IT KEYS ON THE WALL CLOCK
@@ -90,6 +109,25 @@ which is the row the spec actually writes for "no price data". If NOTHING can
be produced, the result is `PriceRead.no_data(reason)` and not an empty series,
because an empty series is not constructible on purpose (price.py trap 3).
⚠️ A SUB-HOURLY FEED IS NOT DECIMATED TO ONE POINT AN HOUR, and the hourly walk
below is about WHICH HOURS NEED FILLING - never about what a published hour is
allowed to contain. BELGIAN DAY-AHEAD ITSELF SETTLES PER QUARTER since the
15-minute MTU took effect on 2025-10-01 - `generic_dayahead` reads 96 points a
day, not 24 - `_validate_raw()` accepts any strictly-increasing spacing, and a
published hour may therefore hold four points. So EVERY published point inside
the window is passed through, in order, and a carried point is added only for a
whole hour with NO published coverage at all.
The alternative was measured rather than reasoned about, because FCST-04's
first caller shipped it: keying the published set as `{_utc_hour(p.start): p
for p in r.points}` reads perfectly ordinary and keeps the LAST point of each
hour, dropping the other three. On a 96-point quarter-hourly feed - all firm,
fully published, nothing to carry - the series came out 15 points long, the
arbitrage spread collapsed from 0.143 to 0.014 EUR/kWh, and the cheapest
"hour" became a `:45` point whose `at()` window runs into the next hour. A feed
losing 84 % of its own prints is not a windowing effect and is not something a
provisional flag would ever have reported.
AC2: THE IRREVERSIBLE-ACTION GATE
=================================
@@ -141,7 +179,7 @@ class Provenance(Enum):
flag" and "the others".
"""
PUBLISHED = "published" # the provider printed this hour. Flag: the
# feed's own - an imbalance print may be
# feed's own - a real-time print may be
# published AND provisional (§9.2).
CARRIED = "carried" # yesterday's same-wall-clock hour (§8.3).
# Flag: ALWAYS True, by construction.
@@ -156,6 +194,25 @@ def _utc_hour(instant: datetime) -> datetime:
return u.replace(minute=0, second=0, microsecond=0)
def _published_by_hour(read: PriceRead) -> dict:
"""UTC hour -> the provider's OWN points that start inside it, in order.
⚠️ A TUPLE PER HOUR, NOT A POINT, AND THAT IS THE WHOLE OF THE SUB-HOURLY
RULE. `{_utc_hour(p.start): p for p in r.points}` reads identically, is one
character shorter, and silently keeps one point per hour - the last one
written. The mapping answers two DIFFERENT questions here: "does this hour
need filling at all?" is `u in published`, and "what does the output carry
for it?" is all of `published[u]`. Collapsing them is how a quarter-hourly
feed loses three quarters of its prints.
"""
out: dict = {}
read.match(
series=lambda r: [out.setdefault(_utc_hour(p.start), []).append(p)
for p in r.points],
no_data=lambda _r: [])
return {u: tuple(ps) for u, ps in out.items()}
def _carried(start_utc: datetime, eur_per_kwh: float) -> PricePoint:
"""A point filled from yesterday's profile. §8.3: ALWAYS provisional.
@@ -172,37 +229,122 @@ def _carried(start_utc: datetime, eur_per_kwh: float) -> PricePoint:
def _audit(points, published) -> None:
"""Post-condition: every flag matches the point's TRUE provenance (AC1).
`published` maps a UTC hour to the point the provider produced for it.
`published` maps a UTC hour to the TUPLE of points the provider produced
for it (`_published_by_hour()`). An hour ABSENT from that mapping is an hour
the feed did not print, and a carried point is the only thing that may
stand in it.
The rule, in the two forms an output point can take:
* PUBLISHED - the point must BE one of `published[u]`, by identity, and
may be emitted only once. Its flag is whatever the feed said, never
forced to False: an imbalance print is published AND provisional (§9.2,
TARIFF-02), and a check that "corrected" it would be a bug wearing a
guarantee.
* CARRIED - the hour must not appear in `published` AT ALL, the flag must
be exactly True, and that hour may hold no second point.
⚠️ WHAT REPLACED "AT MOST ONE POINT PER HOUR", AND WHAT THAT DID AND DID
NOT CHANGE. This audit began as FCST-03's `seen` set: any repeated
`_utc_hour` raised, beside an identity rule `p is not src`. A bare hour
count cannot express a quarter-hourly feed, where four distinct published
points share one hour legitimately - so the count was replaced by the two
rules above, which subsume it.
⚠️ AND IT IS NOT STRICTER. FCST-04's version of this paragraph claimed two
strengthenings and BOTH WERE FALSE; FCST-05 measured it. A differential
over the whole domain this audit can express - every published mask over 3
hours crossed with every output series of length 0-3 drawn from
{published, published-provisional, carried, carried-unflagged,
copy-of-published} - found ZERO divergence in EITHER direction:
OLD raised, NEW accepted (WEAKER): 0
OLD accepted, NEW raised (STRONGER): 0
The two specific claims, one at a time:
* "so does the same published point emitted twice" - the OLD `seen`
count already raised on that, because emitting one point twice is one
HOUR twice. Preserved by the identity/`emitted` rules, not added.
* "a carried point may no longer share an hour with a published one,
which the count never checked at all" - true of the COUNT, and the old
IDENTITY rule raised on it anyway: a carried point in a published hour
took the `else` branch and failed `p is not src`. The message was about
a copy rather than about a carry, which is the only thing that moved.
The one real behavioural change is the intended RELAXATION - several
distinct published points in one hour, which the old audit could not
express at all and refused. That relaxation is why this was rewritten, and
it stays. A right decision with a fabricated reason is still a defect here.
⚠️ AND THE REAL LIMIT, WHICH IS NOT A REGRESSION AND IS NOT SMALL: THIS
AUDIT DOES NOT CHECK COMPLETENESS. It grades every point that IS emitted
and says nothing about the ones that are not. Fed the H1 decimation
directly - four published points in an hour, output emitting only the last
- it ACCEPTS; so does two of four, and so does an EMPTY output. So `_audit`
is NOT what stops H1 recurring. What stops it is the point COUNT and the
IDENTITY checks in `test_price_history.py` and `test_price_forecast.py`,
which compare the emitted series against the feed's own prints.
It is not made complete here, and the reason is structural rather than
effort: `published` is `_published_by_hour(read)` over the WHOLE read,
unwindowed, while `fill_from_yesterday()` emits only [since, until). A
published point outside the window is legitimately absent, so "every
published point appears" is not a true post-condition of these two
arguments - it would need the window passed in, which is a different
signature and a different check.
⚠️ AND THAT IS MEASURED, NOT ASSUMED. FCST-05 built the rule - raise unless
every point of every `published[u]` is emitted, +162 B - and ran it. It
takes the 1 h, 12 h and 48 h HORIZON checks in test_price_forecast.py RED,
on the shipped fill and a healthy feed: at an off-the-hour instant the
feed's already-elapsed prints are outside [since, until) and are correctly
dropped, and the rule calls that a provenance failure. (The differential
beside those checks scores the same rule as 49 STRONGER cases, so it is a
real change of behaviour and not a no-op.) The rule as stated is therefore
not merely unimplemented, it is WRONG - and stating a false post-condition
is MUTATION.md habit 16's defect: an instrument that asserts a completeness
it does not have is worse than no instrument.
⚠️ RAISES, NEVER REPAIRS. A module that silently corrects its own
provenance is one whose provenance means nothing; the only safe response to
"this point's flag disagrees with where it came from" is to refuse to emit
the series at all.
"""
seen = set()
carried_hours = set()
emitted = set()
for p in points:
u = _utc_hour(p.start)
if u in seen:
raise ValueError(f"the filled series repeats {u.isoformat()}; a "
f"duplicated hour is how a DST day gets mispriced")
seen.add(u)
src = published.get(u)
prov = Provenance.CARRIED if src is None else Provenance.PUBLISHED
if src is None:
if u in carried_hours:
raise ValueError(
f"the filled series repeats the carried hour "
f"{u.isoformat()}; a duplicated hour is how a DST day gets "
f"mispriced")
carried_hours.add(u)
if p.provisional is not True:
raise ValueError(
f"{u.isoformat()} is {prov.value} and is flagged "
f"provisional={p.provisional!r}. §8.3: an hour filled from "
f"yesterday's profile is provisional whether or not the "
f"number turns out to be right - the flag is a claim about "
f"PROVENANCE, not about accuracy.")
f"{u.isoformat()} is {Provenance.CARRIED.value} and is "
f"flagged provisional={p.provisional!r}. §8.3: an hour "
f"filled from yesterday's profile is provisional whether "
f"or not the number turns out to be right - the flag is a "
f"claim about PROVENANCE, not about accuracy.")
else:
if p is not src:
if not any(p is q for q in src):
raise ValueError(
f"{u.isoformat()} is {prov.value} but the series carries a "
f"COPY of that point rather than the point itself. A copy "
f"is where the provisional flag gets lost; published points "
f"pass through by identity.")
f"{u.isoformat()} is {Provenance.PUBLISHED.value} but the "
f"series carries a COPY of that point rather than the "
f"point itself. A copy is where the provisional flag gets "
f"lost; published points pass through by identity.")
if id(p) in emitted:
raise ValueError(
f"the filled series emits the published point at "
f"{p.start.isoformat()} TWICE. The provider's own points "
f"are strictly increasing (`_validate_raw`), so a repeat "
f"is never a second print - it is a duplicated hour, which "
f"is how a DST day gets mispriced")
emitted.add(id(p))
def fill_from_yesterday(read: PriceRead, *, yesterday: PriceRead,
@@ -217,11 +359,21 @@ def fill_from_yesterday(read: PriceRead, *, yesterday: PriceRead,
Returns a `PriceRead`:
* a series, whose points are the provider's own objects for the hours it
published and `_carried()` points for the rest, ordered by instant;
* a series, carrying the provider's own objects - ALL of them, which on
a quarter-hourly feed is four inside one hour - for the hours it
published, and one `_carried()` point for each remaining whole hour,
ordered by instant;
* `no_data` when neither source can produce a single hour - never an
empty series, which is not constructible (price.py trap 3).
⚠️ THE WINDOW IS WHOLE HOURS AT BOTH ENDS. `since` is floored to the hour
that CONTAINS it, because the current hour is still a tradeable hour; and
so is `until`, which is the half FCST-04's first draft left out. With
`since` floored and `until` compared raw, a 48 h horizon asked for at 08:17
emitted 49 hours - the hour of `until` was still `< until` - and the count
was one too many for every `now` that is not exactly on the hour, which is
every `now` a running controller has.
⚠️ IDEMPOTENT AND RE-RUNNABLE, WHICH IS THE §5.4 LATE-PUBLICATION CONTRACT.
Day-ahead publication at 14-16 h CET can be late or fail, so the fetcher
retries and this runs again against the fresher read. Published hours always
@@ -234,9 +386,7 @@ def fill_from_yesterday(read: PriceRead, *, yesterday: PriceRead,
raise ValueError(f"{name} must be tz-aware; a naive instant is "
f"ambiguous for one hour every October")
published = read.match(
series=lambda r: {_utc_hour(p.start): p for p in r.points},
no_data=lambda _r: {})
published = _published_by_hour(read)
# Yesterday's profile, keyed by BRUSSELS WALL-CLOCK HOUR. First point wins
# when yesterday itself was a 25-hour day and printed local hour 2 twice.
@@ -248,11 +398,12 @@ def fill_from_yesterday(read: PriceRead, *, yesterday: PriceRead,
out = []
u = _utc_hour(since)
end = until.astimezone(timezone.utc)
end = _utc_hour(until)
while u < end:
if u in published:
out.append(published[u]) # BY IDENTITY. No copy.
else:
got = published.get(u)
if got:
out.extend(got) # BY IDENTITY, and ALL of them. No copy,
else: # and no decimation to one an hour.
src = by_local_hour.get(brussels(u).hour)
if src is not None:
out.append(_carried(u, src.eur_per_kwh))
@@ -271,9 +422,19 @@ def fill_from_yesterday(read: PriceRead, *, yesterday: PriceRead,
_audit(out, published)
carried = sum(1 for p in out if _utc_hour(p.start) not in published)
if carried:
# ⚠️ THREE QUANTITIES, THREE NAMES (TARIFF-11). This line read
# "%d of %d hours ... ; %d published" with `carried` (a count of
# carried HOURS - the fill emits exactly one point per unpublished
# hour) over `len(out)` (a count of POINTS, four per hour wherever the
# feed printed) and `len(out) - carried` (POINTS again) under the same
# noun. On the Belgian quarter-hourly feed that reads "33 of 141
# hours" about a 48-hour horizon. The fill's unit is the HOUR and the
# feed's is the POINT; nothing derived from one answers for the other.
total_h = len({_utc_hour(p.start) for p in out})
_LOG.info("§8.3: %d of %d hours carried forward from yesterday's "
"profile and flagged provisional; %d published",
carried, len(out), len(out) - carried)
"profile and flagged provisional; the feed published %d "
"points over the other %d hours",
carried, total_h, len(out) - carried, total_h - carried)
product = read.match(series=lambda r: r.product,
no_data=lambda _r: yesterday.product)
+865
View File
@@ -0,0 +1,865 @@
"""FCST-04 - the day the carry-forward carries, kept somewhere a restart cannot reach.
WHAT WAS MISSING, AND WHY IT WAS INVISIBLE. FCST-03 shipped
`price_forecast.fill_from_yesterday()` complete and checked, and shipped the
consumers that were written to expect it - `TariffPlan.firm_points()`,
`cheap_now()`'s refusal of a provisional hour, and `arbitrage()`'s refusal 3,
which drops provisional hours from the trade set instead of refusing the whole
series. Every one of those was live. The FILL ITSELF HAD NO CALLER: nothing in
`app/` referenced `fill_from_yesterday`, so no series ever contained a carried
point and every one of those consumers was exercising its all-firm branch.
The defect that produced is not an error anywhere. `load_prices()` returns
whatever the feed published and nothing widens it to the requested horizon, so
on a dynamic site between midnight and the 14-16 h CET day-ahead publication
the series covers the rest of TODAY and nothing more - typically 10-24 of the
48 hours `price_horizon_h` asked for. `TariffPlan.arbitrage()` then computes
`max - min` over that truncated window and reports a spread. The number is
real, the window is half the one the operator configured, and NOTHING SAYS SO:
tomorrow evening's peak - the hour a morning charge would be sold into - is
simply not in the series, so the spread it would have created cannot clear a
threshold it was never compared against. A missing hour is not a wrong price;
it is an hour the optimiser never learns exists.
So this module is the half FCST-03 left out, and it is deliberately the SMALL
half: the fill's arithmetic, its DST rule and its provenance audit all stay in
`price_forecast.py`, untouched. What lives here is the one thing that
arithmetic needs and could not have: A YESTERDAY THAT SURVIVES A RESTART.
WHY PERSISTENCE, AND NOT A VARIABLE ON THE REFRESH TASK
======================================================
An in-memory "the last full day I saw" costs nothing and is wrong exactly when
it matters. The window the fill exists for is midnight -> publication; an
add-on restarted at 08:00 - a config change, an HA update, a power cut - would
come up with no yesterday at all and spend that whole window on the truncated
series, which is the pre-FCST-03 behaviour arriving silently on the mornings
after an operator touched something. The fill's own value is a function of
surviving a restart, so the day it carries has to.
⚠️ AND NOT IN `/data/state.json`. That file holds the maintenance history and
SAFETY-02's sign latch, and `Store.set()` rewrites the whole of it on every
write. The tariff refresh runs every `TARIFF_REFRESH_S` (1800 s), so folding a
price cache into it would put 48 extra whole-file rewrites a day through the
file whose corruption reads as "no maintenance ever ran" and "never tripped".
The cache is worth having; it is not worth putting anywhere near that. Separate
file, separate blast radius.
⚠️ AND THE DIRECTORY IS NOT CREATED, which is `tariff.PeakStore`'s rule and its
reason applies unchanged: `/data` is mounted by the Supervisor and always
exists on a real install, so `makedirs` here could only ever succeed where the
durable volume is MISSING - manufacturing a cache on the container's ephemeral
layer that writes fine, reads fine, and is thrown away on the next restart.
That is a yesterday that silently forgets, which is worse than one that says it
cannot answer.
⚠️ AND THAT RULE IS CARRIED OUT IN `_remember()`, NOT ONLY IN `_save()`. FCST-04
first shipped it half-done: `_save()` short-circuited on `not self.ok` while
`_remember()` filled `self._days` anyway, so a store with no durable volume
answered `words` = "unavailable" and `yesterday()` = A SERIES - an in-memory
yesterday that works all day and is gone on the next restart, which is exactly
the silently-forgetting cache this rule exists to refuse. A store that cannot
persist now records nothing, and both of its answers say the same thing.
⚠️ AND "CANNOT PERSIST" IS ONE QUESTION, ASKED ONCE, ABOUT THE OUTCOME -
FCST-06. The paragraph above enforced that rule against `not self.ok`, which is
one SYMPTOM of not persisting: the durable volume is missing. There is a second,
and FCST-06 measured it. With `ok=True` and the directory present, `_save()` can
still fail - a read-only remount (`[Errno 30] Read-only file system`), a full
volume (`[Errno 28] No space left on device`) - and it logs, records
`last_error` and returns. `_remember()` had already populated `self._days`, so
`words` read `'2026-01-14 (24 points over 24 h); 2026-01-15 (24 points over
24 h) [could not write (...)]'` while the file on disk held `{}`, and
`yesterday()` answered 24 REAL POINTS for the rest of the process's life. Gone
on the next restart, and nothing said it would be - the same silently-forgetting
cache, reached by the condition the rule did not name.
⚠️ SO IT IS NOT TWO RULES, AND IT IS DELIBERATELY NOT TWO TESTS. "Can this
store persist?" CANNOT BE ANSWERED BEFORE THE ATTEMPT - a directory that existed
and was writable a moment ago is not a write that completed - so it is answered
AFTER it, once, by the only thing holding the evidence: `_save()` returns
whether the file now holds what memory holds. A missing volume is one of its
False cases and an `OSError` from the write is another, and `_remember()` KEEPS
ONLY WHAT THAT ONE ANSWER SAYS WAS KEPT. There is no `not self.ok` test in
`_remember()` any more, because there is nothing left for a second test to add -
and a third symptom nobody has thought of is how this defect arrived both times.
⚠️ AND WHAT IS KEPT IS THE LAST STATE THAT IS ON DISK, NOT AN EMPTY ONE. The
MERGE is undone, not the store. A site whose volume fills at 14:00 goes on
answering the yesterday it wrote at 13:30 - that record survives a restart, so
answering it is not a promise the store cannot keep. What it stops doing is
adding to it. The invariant the rule was always describing, stated once:
`self._days` NEVER HOLDS A POINT THE FILE DOES NOT. (Not "mirrors the file" -
`_usable_days()` drops records the file still holds. The only direction that can
invent a yesterday is the one named.)
THE FOUR RULES THIS FILE IS SHAPED BY
=====================================
1. IT RECORDS WHAT THE PROVIDER PUBLISHED, NEVER WHAT THE FILL PRODUCED.
Feeding a filled series back in is the defect that makes the whole mechanism
rot: day 2 carries day 1's guesses, day 3 carries day 2's carry of them, and
after a week of a dead feed the "yesterday" being carried forward is a
fossil of the last real publication with nothing anywhere reporting its age.
Every point still carries `provisional=True`, so the flag never lies - what
decays is the NUMBER, silently, behind a flag that is telling the truth.
⚠️ NO VALUE-LEVEL CHECK CAN CATCH THAT, and saying so is more useful than a
guarantee that would not hold. A carried point and a published-provisional
point (a real-time product's current interval, §9.2) are the same three
fields with
the same flag; the difference is provenance, and provenance is not in the
value. So the mechanism is ORDERING, not validation: `carry()` is the only
public entry point, it remembers the argument and returns the fill, and the
filled series exists only as a return value that `carry()` itself never
sees again. `_remember()` is private, is called from exactly one place, and
`test_price_history.py` PARSES every shipped module under `app/`, `tools/`,
`scenarios/` and `estop/` and fails if any of them uses the NAME - an
attribute, a bare name, a def, or the string constant a `getattr` needs. By
name rather than by substring, so a comment about the rule does not fail it
and a call from `tools/` cannot pass unseen.
⚠️ AND THE ORDER IS GRADED BY DRIVING IT, NOT BY READING THE SOURCE. The
first version of that check compared positions in `ast.walk`, which is
breadth-first: `self._remember(...)` is a statement and `self.yesterday(...)`
is an ARGUMENT of the fill, one level deeper, so `_remember` came out first
whichever statement was written first - shipped and reversed scored the
same. What grades it now is the behaviour: a feed carrying yesterday
alongside today, into an EMPTY store, must fill the whole horizon, and
reversed it fills nothing.
2. "YESTERDAY" MEANS YESTERDAY, NOT "THE NEWEST DAY I HAVE".
`newest()` is the natural accessor and it is the one that decays: an add-on
down from Friday to Monday would answer Friday's prices to a fill that flags
them as one day old. The lookup is BY DATE - the Brussels day before the
Brussels day of the instant asked about - and a record for any other date is
NO_RECORD with a reason naming both dates. A three-day-old profile is not a
worse yesterday; it is not a yesterday.
3. A STORE THAT CANNOT ANSWER SAYS SO, IN THE TYPE.
`yesterday()` returns a `PriceRead`: a `series` when a full record exists,
`no_data(reason)` for every other outcome - no file, no record for that
date, an unreadable file, a record that parses to nothing. There is no
empty-series answer and no `.get(..., [])`-shaped accessor, for
`price.py`'s reason: an empty series is how "the cache is broken" gets to
wear the shape of "yesterday was free". `fill_from_yesterday()` takes a
`PriceRead` and handles `no_data` on its own, so the degraded case needs no
branch at the call site - it fills nothing and the horizon stays short,
which is exactly the behaviour that shipped before this ticket.
4. HOURS ACCUMULATE WITHIN A DAY; THEY DO NOT REPLACE IT.
A feed that serves only the REMAINING hours of the day (a legitimate shape -
nothing in the plugin contract promises a whole day) would, under
replace-the-record, overwrite a complete 24-hour profile with the two points
left at 22:00 and hand the next morning a yesterday with two hours in it. So
a record is MERGED by instant: a later read updates the hours it carries and
leaves the rest, and a revised price for an hour already held wins because
it is the newer statement about the same hour.
⚠️ KEYED BY INSTANT, NOT BY LOCAL HOUR, AND THAT IS THE DST DECISION. Storing
`hour -> price` would collapse the two 02:00s of a 25-hour October day into
one, inventing a second and different collapsing rule beside the one
`fill_from_yesterday()` already documents ("the first of its two 02:00 points
is the one used, deterministically"). Storing the instants keeps both, in
order, and lets the fill's own rule be the only one in the system.
WHAT THIS DOES NOT DO, ON PURPOSE
=================================
It does not extend the fill to a feed that published NOTHING. `resolve_tariff()`
applies `carry()` only on a `series` read; a `no_data` read still falls to
`_degrade()` and §11.4's row - "No price data -> fall back to the static
profile, mark the plan provisional, no arbitrage" - which is the fallback the
operator configured `static_profile` FOR. Preferring yesterday's feed over it
would change a documented degradation without being asked, and it would buy
nothing measurable: a fully-carried series is 100 % provisional, so
`arbitrage()` finds fewer than two firm hours and refuses, and `cheap_now()` is
False, exactly as on the §11.4 fallback. The two differ only in which numbers
the status page shows. `fill_from_yesterday()` handles a `no_data` read anyway,
and that branch is therefore not reached from production - it stays checked,
and turning it on is a product decision, not a wiring one. See
DECISIONS/FCST-04-the-day-that-is-carried.md.
"""
import copy
import json
import logging
import os
import tempfile
from datetime import datetime, timedelta, timezone
from .price import PricePoint, PriceProduct, PriceRead, brussels
from .price_forecast import fill_from_yesterday
_LOG = logging.getLogger("goodwe.price.history")
# How many Brussels days of published prices the SORT-ORDER cap keeps. The fill
# needs exactly one - yesterday - and rule 2 refuses anything older, so this is
# not a horizon: it is the number of days a record may sit unused before
# pruning removes it, and it is small because a bigger cache would only hold
# days no lookup can ask for. Three covers today, yesterday, and the day a
# fetch spanning midnight writes ahead of itself.
#
# ⚠️ IT IS A CAP, NOT THE RETENTION RULE, AND THREE HAS NO SLACK. The steady
# state is already {yesterday, today, tomorrow}; a feed that publishes to D+2,
# or ONE stray point in the wrong year - which sorts LAST and is therefore
# immortal under a newest-N cap - pushes yesterday out of a three-slot store
# and keeps it out, permanently and silently. So `_prune()` PINS the two days a
# lookup can actually ask for on top of the cap; see its docstring. The store
# is then bounded at KEEP_DAYS + 2 records rather than at KEEP_DAYS.
KEEP_DAYS = 3
SCHEMA = 1
def _bdate(instant: datetime) -> str:
"""The Brussels calendar date of an instant, as `YYYY-MM-DD`.
⚠️ BRUSSELS, NOT UTC. 23:30 UTC in July is already tomorrow in Belgium, and
a cache keyed on the UTC date would file the last 90-120 minutes of every
day under the wrong one - so a lookup on the following morning would find a
"yesterday" missing its own evening hours, which are the expensive ones.
"""
return brussels(instant).strftime("%Y-%m-%d")
def _yesterday_of(instant: datetime) -> str:
"""The Brussels date of the day BEFORE `instant`'s Brussels day.
⚠️ THE ARITHMETIC IS ON THE LOCAL DATE, NOT ON THE INSTANT, AND THAT IS A
DST DEFECT THIS FILE SHIPPED. `brussels()` renders at the FIXED offset in
force at that instant, so `brussels(stamp) - timedelta(days=1)` subtracts 24
ABSOLUTE hours from a wall clock - and the Brussels day before a
spring-forward is 23 hours long. Measured at 2026-03-30T00:30+02:00, the
first hour of the morning after the March transition: it returned
2026-03-28, so the fill carried a TWO-DAY-OLD profile flagged as yesterday's
(or, with only the last three days held, answered `no_data` naming a date
that is not yesterday).
A calendar day minus one day is `date` arithmetic, and a `date` has no
offset to lose. The 25-hour October day is right for the same reason.
"""
return (brussels(instant).date() - timedelta(days=1)).strftime("%Y-%m-%d")
def _point_of(row):
"""One stored row -> a `PricePoint`, or `None` if the row is not one.
THE ONE VALIDATOR, AND THE ASYMMETRY IT CLOSES (FCST-05's MEDIUM). This
file used to hold TWO standards for what a stored point is. The READ path
(`_parse_points()`) was properly defensive - every field checked, a
malformed row dropped and reported. The WRITE path was
`{q["start"]: q for q in raw if isinstance(q, dict) and "start" in q}`
followed by `sorted(..., key=lambda q: q["start"])`, which trusts
`q["start"]` to be a hashable, sortable string it never checked. Measured,
with the malformed row under the day the feed is currently writing to:
start is an int -> TypeError: '<' not supported between instances
of 'str' and 'int'
start is a dict -> TypeError: cannot use 'dict' as a dict key
start is null -> TypeError: '<' not supported between 'str' and
'NoneType'
start is a list -> TypeError: cannot use 'list' as a dict key
⚠️ AND THE BLAST RADIUS IS PERMANENT, WHICH IS WHY IT IS NOT MERELY UNTIDY.
`_remember()` is reached through `carry()` -> `resolve_tariff()` ->
`run_tariff_refresh`, which catches `Exception` - so the add-on stays UP
and EVERY refresh raises forever. The plan retires after
`TARIFF_MAX_AGE_S`, and arbitrage and the cheap window are off from then
on. There is no self-heal, because `_remember()` raises before it reaches
`_save()`. `_usable_days()` only required `rec["points"]` to be a LIST; the
elements were unchecked on the write path, and the existing fixture put its
malformed row under a day the feed was NOT writing to, so `_remember()`
never touched it.
So both paths call this, and neither of them has a standard of its own.
Returning `None` rather than raising is the contract: a caller decides
whether a bad row is dropped (both of them do) and how it is reported.
"""
if not isinstance(row, dict):
return None
try:
start = datetime.fromisoformat(row["start"])
value = float(row["eur_per_kwh"])
flag = row["provisional"]
except (KeyError, TypeError, ValueError):
return None
if start.tzinfo is None or start.utcoffset() is None \
or not isinstance(flag, bool):
return None
return PricePoint(start=start, eur_per_kwh=value, provisional=flag)
def _held(rec) -> tuple:
"""`(points, whole hours they cover)` for a day record. `(0, 0)` otherwise.
⚠️ POINTS AND HOURS ARE TWO NUMBERS, AND FCST-04's STATUS ROW PRINTED THE
FIRST UNDER THE SECOND'S NAME. The row was `f"{d} ({_n_points(rec)} h)"`,
which counts POINTS: on the quarter-hourly feed this add-on already
supports it read `'2026-01-13 (96 h)'` - 96 "h" for a 24-hour day. That is
the points/hours conflation H1 was raised for, on the row H1's own ticket
shipped. Both numbers are reported now, so an operator can see a day that
is 24 points over 24 h and a day that is 96 points over 24 h and tell them
apart, and neither number is wearing the other's unit.
⚠️ TOLERANT, AND IT IS THE SECOND LINE OF DEFENCE, NOT THE FIRST. `_load()`
drops a malformed day record and says so, so this should never see one. It
is written this way because `words` is read from `Controller.__init__` and
from every status render: a status LINE that raises takes the add-on down,
and "the cache is broken" must never be able to mean "the controller does
not start". That is exactly what FCST-04 shipped - `{"days": {"2026-01-14":
"oops"}}` gave `ok=True`, `last_error=None`, and an `AttributeError` out of
`words`, `yesterday()` and `resolve_tariff()` alike, with no self-heal
because `_remember()` raised before it could rewrite the file.
⚠️ AND IT COUNTS ONLY ROWS THAT ARE POINTS. A row `_point_of()` refuses is
not held prices, and a row count that included it would report a store as
healthier than it is - which is the second half of FCST-05's MEDIUM: the
status row said `'2026-01-15 (1 h)'` about a record whose single row was
killing every tariff refresh. What is dropped is named in `last_error`, and
`words` prints that beside the counts.
"""
if not isinstance(rec, dict):
return (0, 0)
raw = rec.get("points")
if not isinstance(raw, (list, tuple)):
return (0, 0)
n, hours = 0, set()
for row in raw:
pt = _point_of(row)
if pt is None:
continue
n += 1
hours.add(pt.start.astimezone(timezone.utc)
.replace(minute=0, second=0, microsecond=0))
return (n, len(hours))
def _row_of(day: str, points: int, hours: int) -> str:
"""One day's cell of the status row: what is held, in BOTH of its units.
⚠️ THE UNIT IS SPELT OUT ON EACH NUMBER. FCST-04's cell was
`f"{day} ({points} h)"` - the point count wearing the hour's name, which on
a quarter-hourly feed reads `'2026-01-13 (96 h)'` for a 24-hour day. What
an operator needs from this row is whether tomorrow morning's fill has a
whole day to carry, and that is the HOUR figure; the point count says how
finely it is priced. Neither answers for the other.
"""
return (f"{day} ({points} point{'' if points == 1 else 's'} "
f"over {hours} h)")
def _bad_rows(rec) -> int:
"""How many rows of a day record are not points. Total; never raises."""
if not isinstance(rec, dict):
return 0
raw = rec.get("points")
if not isinstance(raw, (list, tuple)):
return 0
return sum(1 for row in raw if _point_of(row) is None)
class PriceHistory:
"""The published price series of the last few Brussels days, in /data.
⚠️ `ok` IS ABOUT THE FILE, NOT ABOUT THE DATA. A healthy store with no
record for yesterday is `ok` and answers `no_data` - a normal state on a
site's first day, and a different thing from a store that could not be
opened. Both are visible: `ok`/`last_error` describe the file, and
`yesterday()`'s reason describes the answer.
⚠️ AND `ok` IS NOT "CAN PERSIST" (FCST-06). It says the directory was
there when the store opened, which a full volume and a read-only remount
both leave true. The one thing that answers "can this store persist" is
`_save()`'s return value, and `self._days` NEVER HOLDS A POINT THE FILE DOES
NOT, because `_remember()` undoes a merge that answer refused. That is what
makes `words` and `yesterday()` describe the same store in EVERY degraded
state rather than only in the ones somebody enumerated.
"""
def __init__(self, path: str = "/data/price_history.json"):
self.path = path
self.ok = False
self.last_error: str | None = None
self._days: dict = {}
self._open()
# -- the file ------------------------------------------------------------
def _open(self) -> None:
parent = os.path.dirname(self.path)
# See the module note: NOT created. A cache on the ephemeral layer is
# one that forgets without saying so.
if parent and not os.path.isdir(parent):
self.last_error = f"no durable volume at {parent}"
# INFO, not ERROR: on a dev host or in a test container there is no
# /data and nothing is wrong.
_LOG.info("no price history: %s - §8.3 carry-forward has nothing "
"to carry and the horizon stays as short as the feed",
self.last_error)
return
self.ok = True
self._load()
def _load(self) -> None:
try:
with open(self.path, "r", encoding="utf-8") as fh:
blob = json.load(fh)
except FileNotFoundError:
_LOG.info("no price history yet at %s - the first refresh writes "
"one, and the fill has nothing to carry until the day "
"after that", self.path)
return
except (ValueError, OSError) as err:
# A corrupt cache must not stop the controller, and must not be
# silently re-created either: the record is discarded, the reason
# is kept, and `yesterday()` reports it rather than answering
# "no record", which would read as a normal first day.
#
# ⚠️ `ValueError`, NOT `json.JSONDecodeError`, AND THAT IS
# FCST-05's HIGH. Reading this file has two failure families: the
# BYTES (`OSError`) and the CONTENT - and the content family has at
# least two members, because `open(..., encoding="utf-8")` decodes
# before `json` ever sees a character. `UnicodeDecodeError` is
# `ValueError -> UnicodeError -> UnicodeDecodeError`; it is NOT a
# `json.JSONDecodeError` and it is NOT an `OSError`
# (`issubclass(UnicodeDecodeError, OSError)` is False). So ONE
# non-UTF-8 byte in /data/price_history.json escaped `_load()` ->
# `_open()` -> `PriceHistory.__init__`, which is unguarded at
# main.py's `Controller.__init__`, AND THE ADD-ON DID NOT START.
# Measured: `UnicodeDecodeError: 'utf-8' codec can't decode byte
# 0xff in position 39`. Valid-UTF-8-invalid-JSON, a UTF-8 BOM and a
# non-dict day record all degraded correctly - it was the one byte
# class left uncovered, which is the same class as the defect the
# `_usable_days()` rework was raised for.
#
# The tuple names the FAMILY rather than the two members known
# today, because "the file's content is not readable as JSON" is
# exactly what this handler says, and a third `ValueError` subclass
# nobody has thought of is how this defect arrived the first time.
self.last_error = f"unreadable ({err})"
_LOG.warning("price history at %s is %s - starting from empty. "
"§8.3 carry-forward has nothing to carry until the "
"next refresh has written a full day",
self.path, self.last_error)
return
found = blob.get("version") if isinstance(blob, dict) else None
# ⚠️ THE TYPE IS PART OF THE VERSION. `1.0 == 1` in Python, so a float
# written by some other tool would pass a bare `!=` and be read as
# schema 1 - a version check that cannot tell 1 from 1.0 is one that
# would not notice a writer this build has never met.
if not isinstance(blob, dict) or isinstance(found, bool) \
or not isinstance(found, int) or found != SCHEMA:
self.last_error = f"schema {found!r}, not {SCHEMA}"
_LOG.warning("price history at %s has %s - discarding it rather "
"than guessing what its points mean",
self.path, self.last_error)
return
days = blob.get("days")
if not isinstance(days, dict):
self.last_error = "no `days` object"
_LOG.warning("price history at %s has %s", self.path, self.last_error)
return
self._days = self._usable_days(days)
def _usable_days(self, days: dict) -> dict:
"""The day records that are shaped like day records. The rest are dropped.
⚠️ THE ASYMMETRY THAT PRODUCED THE DEFECT: `_parse_points()` below is
carefully defensive about every field of a POINT, and `_load()` was not
defensive about the record holding them at all. `{"version": 1, "days":
{"2026-01-14": "oops"}}` therefore loaded as `ok=True`,
`last_error=None` - and then `rec.get(...)` raised `AttributeError` out
of `words`, `yesterday()` and `resolve_tariff()`. `PriceHistory(...)`
and `.words` are both unguarded in `Controller.__init__`, so THE ADD-ON
DID NOT START; and it never self-healed, because `_remember()` raised on
the same record before it could reach `_save()`.
A broken cache is a short horizon. It is not a controller that will not
regulate a battery, and §6's rule - absent machinery must not disable
the loop - does not stop applying because the machinery is a JSON file.
Dropped, not repaired: an unusable record has nothing in it to keep, and
the next `_remember()` writes a fresh one under that date.
⚠️ AND THE ROWS INSIDE A KEPT RECORD ARE COUNTED, NOT ONLY THE RECORD
(FCST-05). This checked that `rec["points"]` is a LIST and stopped
there, so a record holding one row with a `start` of `7` loaded as
healthy: `last_error=None`, and `words` said `'2026-01-15 (1 h)'` about
a store whose next `_remember()` would raise on that row and go on
raising forever. The rows are NOT dropped here - `_parse_points()` is
the read path's one filter and stays so - but they are COUNTED, so the
store stops reporting health it does not have from the moment it is
opened. `_remember()` drops them on the next refresh and `_save()`
rewrites the file, after which a restart clears the note.
"""
kept, dropped, malformed = {}, [], {}
for key, rec in days.items():
if not isinstance(key, str) or not isinstance(rec, dict) \
or not isinstance(rec.get("points"), list):
dropped.append(str(key))
continue
bad = _bad_rows(rec)
if bad:
malformed[key] = bad
kept[key] = rec
notes = []
if dropped:
notes.append(f"{len(dropped)} unusable day record(s) dropped "
f"({', '.join(sorted(dropped))})")
if malformed:
notes.append(
f"{sum(malformed.values())} malformed point row(s) in "
+ ", ".join(f"{d} ({n})" for d, n in sorted(malformed.items())))
if notes:
# Recorded, not silent: `yesterday()` and `words` both report
# `last_error`, and a cache that quietly discards half a file reads
# to an operator exactly like a normal first day.
self.last_error = "; ".join(notes)
_LOG.warning("price history at %s: %s - the controller starts "
"anyway and the horizon is as short as the feed makes "
"it", self.path, self.last_error)
return kept
def _save(self) -> bool:
"""Write `self._days` out. True only if the file now holds it.
⚠️ THE RETURN VALUE IS THE "CAN THIS STORE PERSIST?" PREDICATE, AND
IT IS THE ONLY ONE (FCST-06). Both ways of not persisting end here and
are the same answer: no durable volume returns False without touching
the disk, an `OSError` out of the write returns False after recording
why. `_remember()` acts on that one answer, so the second place that
used to test the first symptom on its own - and missed the second - has
nothing left to test. See the module docstring's fifth warning.
⚠️ AND IT REPORTS, IT DOES NOT RAISE. §6 and this module's opening: a
broken cache must not be a broken controller. An add-on that will not
start because /data filled up is a worse defect than the one FCST-06
fixed, so a failed write is logged, kept in `last_error` - where `words`
and `yesterday()` both read it - and RETURNED, never thrown.
⚠️ THE TEMP FILE IS REMOVED ON EVERY FAILING PATH. A write that dies
between `mkstemp` and `os.replace` used to leave its temp behind, and
the refresh task runs 48 times a day: a persistent write failure - a
full volume being the likeliest cause - leaked 48 inodes a day onto the
volume that was already the problem.
"""
if not self.ok:
return False
tmp = None
try:
fd, tmp = tempfile.mkstemp(dir=os.path.dirname(self.path))
with os.fdopen(fd, "w", encoding="utf-8") as fh:
json.dump({"version": SCHEMA, "days": self._days}, fh, indent=1)
# Atomic: a power cut mid-write must not leave a truncated file
# that parses as an empty day.
os.replace(tmp, self.path)
tmp = None # it IS the file now; do not unlink
except OSError as err:
self.last_error = f"could not write ({err})"
_LOG.error("could not persist price history: %s", err)
return False
finally:
if tmp is not None:
try:
os.unlink(tmp)
except OSError: # nothing further to try, and a
pass # failed cleanup is not a fault
return True
# -- the one public entry point -----------------------------------------
def carry(self, read: PriceRead, *, since: datetime, until: datetime,
stamp: datetime) -> PriceRead:
"""Remember what this read published, then §8.3-fill the missing hours.
THE ORDER IS DELIBERATE AND IS THE POINT OF THE METHOD. The read is
recorded FIRST, so a feed that hands back yesterday alongside today
refreshes yesterday's record from live data before it is consumed as
the profile; and the filled series - the one thing that must never be
recorded - exists only as this method's return value, which this object
never sees again. See rule 1, including how that order is graded.
Returns whatever `fill_from_yesterday()` returns: a series over the
whole hours of [since, until), carrying EVERY point the provider
published inside it - by identity, and four of them in one hour on a
quarter-hourly feed - plus one carried point for each remaining whole
hour FOR WHICH YESTERDAY HELD THAT WALL-CLOCK HOUR. An hour neither
source can price is left out (§11.4 for that hour), and `no_data` comes
back when that is every hour.
"""
self._remember(read, stamp)
return fill_from_yesterday(read, yesterday=self.yesterday(stamp),
since=since, until=until, stamp=stamp)
# -- the halves, private because rule 1 is an ordering ------------------
def _remember(self, read: PriceRead, stamp: datetime) -> None:
"""File a provider read's points under their own Brussels days.
⚠️ PRIVATE, AND CALLED FROM EXACTLY ONE PLACE. A caller that can reach
this can record a filled series, which is the decay in rule 1. There is
no value-level defence against that - a carried point and a published
provisional point are the same three fields - so the defence is that
nothing outside this file names it, and `test_price_history.py` checks
the tree for that rather than trusting the underscore.
"""
points = read.match(series=lambda r: r.points, no_data=lambda _r: ())
if not points:
return
product = read.match(series=lambda r: r.product.value,
no_data=lambda _r: None)
provider = read.match(series=lambda r: r.provider,
no_data=lambda _r: None)
# ⚠️ THE STATE THIS READ STARTED FROM, KEPT SO IT CAN BE PUT BACK
# (FCST-06). Everything below merges into `self._days` IN PLACE, and
# whether any of it may be KEPT is not knowable until `_save()` has
# tried - so the merge runs, and it is undone if the file did not take
# it. There is no `not self.ok` branch above any more: a store with no
# durable volume is one of the answers `_save()` gives, not a separate
# symptom with a test of its own. Module docstring, fifth warning.
#
# ⚠️ DEEP, NOT `dict(self._days)`. A one-level copy shares the day
# RECORDS, and the loop below rebinds keys on the ones it touches - so
# the "previous" state would hand back records the merge had already
# edited, and a store with a good yesterday on disk would go on
# answering the merge of it that never reached the file. It would also
# be right only for as long as nobody makes this loop mutate a `points`
# list or a row in place. A copy that is right by construction outlives
# that, and the store holds at most KEEP_DAYS + 2 records, 48 times a
# day.
before = copy.deepcopy(self._days)
touched = set()
junked = {}
for p in points:
day = _bdate(p.start)
rec = self._days.setdefault(
day, {"product": product, "provider": provider, "points": []})
# Rule 4: MERGE by instant. A later read updates the hours it
# carries and leaves the rest; a revised price for an hour already
# held wins, because it is the newer statement about that hour.
#
# ⚠️ EVERY HELD ROW GOES THROUGH `_point_of()` (FCST-05's MEDIUM).
# This was `{q["start"]: q for q in raw if isinstance(q, dict) and
# "start" in q}` sorted by `q["start"]` - a key and a sort key that
# trusted a value read off disk to be a hashable, sortable string.
# An `int`, `null`, a `dict` or a `list` there raised `TypeError`
# out of `carry()` on the day the feed was writing to, and
# `run_tariff_refresh` caught it, so the add-on stayed up while
# EVERY refresh failed for good. A row that is not a point is now
# dropped and reported here, exactly as the read path drops it, and
# `_save()` below writes the record back clean - which is the
# self-heal the raising version could never reach.
#
# ⚠️ AND THE MERGE KEY IS THE VALIDATED INSTANT, NOT ITS SPELLING.
# Aware datetimes hash and compare by instant, so this is rule 4 as
# rule 4 is written. The old string key merged by SPELLING, and the
# old lexicographic sort put the second 02:00 of the 25-hour
# October day (`+01:00`) BEFORE the first (`+02:00`). Only the
# stored order was affected - `_parse_points()` re-sorts by instant
# on the way out - but the store now holds what it says it holds.
raw = rec.get("points")
held = {}
for q in (raw if isinstance(raw, list) else ()):
kept = _point_of(q)
if kept is None:
junked[day] = junked.get(day, 0) + 1
continue
held[kept.start] = q
held[p.start] = p.canonical()
order = sorted(held, key=lambda t: t.astimezone(timezone.utc))
rec["points"] = [held[t] for t in order]
rec["product"], rec["provider"] = product, provider
touched.add(day)
if junked:
# Said, not swallowed. `words` prints `last_error`, so a store that
# has just discarded rows off disk stops reporting plain health -
# and after `_save()` below the file is clean, so a restart clears
# this. `_save()` overwrites it if the write itself fails, which is
# the more urgent fact of the two - and on that path the drop is
# undone with the rest of the merge, so those rows are still in the
# file and the next write that completes drops them again.
self.last_error = (
f"{sum(junked.values())} malformed point row(s) dropped from "
+ ", ".join(f"{d} ({n})" for d, n in sorted(junked.items())))
_LOG.warning("price history at %s: %s - the record is rewritten "
"without them and the refresh continues",
self.path, self.last_error)
self._prune(stamp)
if not self._save():
# ⚠️ A STORE THAT CANNOT PERSIST RECORDS NOTHING - the rule, asked
# ONCE as a question about the outcome instead of tested twice
# against two symptoms of it. Keeping the merge here is exactly the
# cache the module docstring refuses: `yesterday()` would answer a
# series the file does not contain, all day, until a restart
# dropped it without a word - and `words` and `yesterday()` would
# be describing two different stores. The reason is already in
# `last_error` (from `_save()`, or from `_open()` when there is no
# volume at all), so both answers now describe what is on disk.
#
# It does NOT raise, and §6 is why: a broken cache must not be a
# broken controller. `carry()` still returns the fill, the horizon
# is as short as the feed makes it, and the add-on regulates.
self._days = before
return
_LOG.debug("price history: %d point(s) filed under %s; %d day(s) held",
len(points), ", ".join(sorted(touched)), len(self._days))
def _prune(self, stamp: datetime) -> None:
"""Drop records no lookup can reach. A sort-order cap, plus two pins.
⚠️ NOTHING IS DELETED BY ARITHMETIC AGAINST THE CLOCK. A clock that
jumped - the failure mode WITNESS-03 spent a session on - would
otherwise delete every record in one pass and leave the fill with
nothing on the morning after, which is the one morning it is for.
Deleting on a bad clock is unrecoverable; holding a few days too many
is not, so the cap is `sorted(...)[-KEEP_DAYS:]` and stays that way.
⚠️ AND A CAP ALONE IS DEFEATED BY ONE POINT IN THE WRONG YEAR. Sorting
is lexicographic on the date string, so a stray far-future record sorts
LAST and is immortal - it consumes a slot for good, and with the steady
state already {yesterday, today, tomorrow} filling all three, yesterday
is what falls out. A feed that publishes to D+2 does the same thing
without any stray at all. So the two days `yesterday()` can ACTUALLY be
asked for are pinned on top of the cap.
`stamp` is READ here, which it was not before: the pin is the only use
of it, and it is additive. Pinning can only RETAIN a record, never
delete one, so a jumped `stamp` pins two dates that are not held and
changes nothing - the WITNESS-03 property is preserved by construction
rather than by remembering to check it.
"""
if len(self._days) <= KEEP_DAYS:
return
keep = set(sorted(self._days)[-KEEP_DAYS:])
keep |= {_bdate(stamp), _yesterday_of(stamp)}
for day in [d for d in self._days if d not in keep]:
del self._days[day]
def yesterday(self, stamp: datetime) -> PriceRead:
"""The published series of the Brussels day BEFORE `stamp`'s, or no_data.
⚠️ THE DATE IS COMPUTED, NOT LOOKED UP. See rule 2: `newest()` would
answer Friday's prices to a Monday fill, and the fill would flag them as
one day old because that is the only thing its flag can say.
"""
if stamp.tzinfo is None or stamp.utcoffset() is None:
raise ValueError(
f"yesterday({stamp!r}): a naive datetime is not an instant, "
f"and 'which Brussels day was yesterday' is exactly the "
f"question a wall-clock reading cannot answer.")
want = _yesterday_of(stamp)
if self.last_error and not self._days:
return PriceRead.no_data(
f"the price history at {self.path} is {self.last_error}, so "
f"§8.3 has no profile to carry forward for {want}; the plan "
f"covers only the hours the feed published")
rec = self._days.get(want)
if rec is None:
have = ", ".join(sorted(self._days)) or "no days at all"
return PriceRead.no_data(
f"no published prices recorded for {want} (held: {have}), so "
f"§8.3 has no profile to carry forward. A record for another "
f"day is not a yesterday")
points, bad = _parse_points(rec.get("points"))
if not points:
why = bad or "it is empty"
return PriceRead.no_data(
f"the {want} record holds no usable point ({why}), and an "
f"empty series is not a free day")
try:
product = PriceProduct(rec.get("product"))
except ValueError:
return PriceRead.no_data(
f"the {want} record names product {rec.get('product')!r}, "
f"which is not a PriceProduct. The optimiser switches on that "
f"enum, so a record that cannot name one cannot be a series")
if bad:
_LOG.warning("price history: %s in the %s record were dropped; "
"%d point(s) carried forward", bad, want, len(points))
return PriceRead.series(points, product,
str(rec.get("provider") or "price_history"),
stamp)
# -- observability -------------------------------------------------------
@property
def words(self) -> str:
"""One line for the status page. Says what is held, or why nothing is.
⚠️ THIS MUST NOT RAISE, EVER. It is read once from `Controller.__init__`
and again on every status render, and neither call site guards it - so
an exception here is an add-on that does not start. It went out that
way: a day record that was a string instead of a dict raised
`AttributeError` from the `len(...)` below. `_load()` now drops such a
record and `_held()` is total, so this has two independent reasons not
to fail on a broken file.
⚠️ AND IT COUNTED POINTS AND CALLED THEM HOURS (FCST-05's LOW). The row
was `f"{d} ({_n_points(rec)} h)"`; on the quarter-hourly feed this
add-on already supports that reads `'2026-01-13 (96 h)'` for a 24-hour
day. Both numbers are printed now - see `_held()` - because they are
two different facts and only one of them was ever being shown.
"""
if not self.ok:
return f"unavailable - {self.last_error}"
if not self._days:
why = f" - {self.last_error}" if self.last_error else ""
return f"empty{why} - nothing to carry forward yet"
held = "; ".join(_row_of(d, *_held(self._days[d]))
for d in sorted(self._days, key=str))
# A file that PARTLY loaded says so, or the dropped days read as days
# that were simply never written.
return held + (f" [{self.last_error}]" if self.last_error else "")
def _parse_points(raw) -> tuple:
"""Canonical dicts -> PricePoints, and a phrase naming what was dropped.
⚠️ A MALFORMED POINT IS DROPPED, NOT DEFAULTED, and the drop is reported.
`float(row.get("eur_per_kwh", 0))` is the shape of the whole family of
defects this repo has been removing: a cache that lost a field would answer
"that hour was free" to a fill that flags it as an ordinary guess.
"""
if not isinstance(raw, list):
return (), "the `points` field is not a list"
out, dropped = [], 0
for row in raw:
# ⚠️ `_point_of()` IS THE ONE VALIDATOR, and this loop used to BE it.
# The write path had its own, weaker standard; see `_point_of()` for
# what that cost. One standard, two callers, no asymmetry to re-open.
pt = _point_of(row)
if pt is None:
dropped += 1
continue
out.append(pt)
out.sort(key=lambda p: p.start.astimezone(timezone.utc))
return tuple(out), (f"{dropped} malformed point(s)" if dropped else "")
class NoHistory:
"""The explicit "this caller keeps no history" - a NAME, not a `None`.
⚠️ IT EXISTS SO THE PARAMETER CAN BE REQUIRED. `history=None` meaning "skip
the fill" is the shape this repo keeps removing: a caller who forgets the
argument gets the truncated-horizon behaviour of the bug this ticket is
about, silently, and the code reads as though the fill is happening. With
no default there is nothing to forget, and a caller with no durable volume
- a check, a replay harness - has to say `NoHistory()` and be visible in
the diff.
It carries out the ordering faithfully: it records nothing and carries
nothing, so `carry()` returns the provider's read unchanged and the plan is
exactly what it was before FCST-04.
"""
ok = False
last_error = "no history is kept by this caller"
words = "not kept"
def carry(self, read: PriceRead, *, since, until, stamp) -> PriceRead:
return read
def yesterday(self, stamp: datetime) -> PriceRead:
return PriceRead.no_data(
"this caller keeps no price history (NoHistory), so §8.3 has no "
"profile to carry forward")
+335 -47
View File
@@ -13,7 +13,7 @@ is an EMS for a minority." The §9.2 product table maps four products onto four
EMS behaviours, and TARIFF-02 shipped the machinery for the top two only:
Dynamic day-ahead hourly EPEX BE + markup full arbitrage
Real-time/imbalance e.g. Eneco Eddy Flex arbitrage, wider thresholds
Real-time/imbalance no shipped plugin (TARIFF-07) arbitrage, wider thresh.
Static day/night two-period tariff TIME-WINDOW SHIFTING <- here
Fully fixed single price SELF-CONSUMPTION ONLY <- here
@@ -207,7 +207,7 @@ from datetime import datetime, timedelta, timezone
from .price import (
MAX_PLAUSIBLE_EUR_KWH, Absence, PricePoint, PriceProduct, PriceRead,
Uncertainty, _Outcome, brussels, load_prices, price_option,
required_spread, uncertainty_of, widen_factor,
provider_configured, required_spread, uncertainty_of, widen_factor,
)
_LOG = logging.getLogger("goodwe.tariff.static")
@@ -222,6 +222,12 @@ _DAY_ALIASES = {
MINUTES_PER_DAY = 1440
# TARIFF-17. The option key whose value is a period table of the TAXES
# component. One spelling, here, because price.py, validate.py and config.yaml
# all name it and `provider_id()`'s note says what three spellings of one
# decision cost.
TAXES_PROFILE_OPTION = "price_taxes_profile"
# The mean that `cheap_now()` grades against is taken over this many local days.
# Seven, so each weekday appears exactly once - see the module docstring. It is
# a MODULE CONSTANT and not an option on purpose: CFG-03 already tracks three
@@ -320,13 +326,37 @@ def _days(value, field: str) -> frozenset:
return frozenset(out)
def _price(value, field: str) -> float:
"""The all-in EUR/kWh for a period, bounded exactly as price.py bounds a feed."""
# What a period's `eur_per_kwh` MEANS, per option. The grammar is one grammar
# (TARIFF-17); the quantity it carries is not, so the "you left this out"
# sentence is the one thing the two tables do not share.
_PERIOD_MEANS = {
"static_profile":
"an all-in eur_per_kwh (energy + supplier markup + taxes, §9.3) - the "
"same basis the dynamic providers are assembled onto, so the two are "
"comparable",
}
_TAXES_MEANS = (
"the TAXES component alone in eur_per_kwh - the non-energy part of the "
"bill for that band (distribution, levies, VAT), NOT an all-in price. The "
"framework adds it to what the feed publishes, exactly as the scalar "
"price_taxes_eur_kwh is added"
)
_PERIOD_MEANS[TAXES_PROFILE_OPTION] = _TAXES_MEANS
def _price(value, field: str, means: str = None) -> float:
"""One period's EUR/kWh, bounded exactly as price.py bounds a feed.
`means` names the quantity for the error sentence only. TARIFF-17: the same
grammar carries an ALL-IN price under `static_profile` and a TAXES-ONLY
component under `price_taxes_profile`, and an operator who confuses the two
is out by an energy price - so the refusal says which one is wanted.
"""
if means is None:
means = _PERIOD_MEANS["static_profile"]
if value is None:
raise ProfileInvalid(
f"{field} is missing. Every period needs an all-in eur_per_kwh "
f"(energy + supplier markup + taxes, §9.3) - the same basis the "
f"dynamic providers are assembled onto, so the two are comparable.")
f"{field} is missing. Every period needs {means}.")
if isinstance(value, bool) or not isinstance(value, (int, float, str)):
raise ProfileInvalid(f"{field} is {value!r}, which is not a number")
try:
@@ -383,14 +413,19 @@ class Period:
class StaticProfile:
"""A validated two-or-more-period tariff. Immutable, pure, I/O-free."""
def __init__(self, periods):
def __init__(self, periods, *, option: str = "static_profile"):
self.periods = tuple(periods)
# ⚠️ TARIFF-17: the OPTION NAME travels with the profile so that every
# refusal names the key the operator actually typed. It is a default
# argument on purpose - every pre-existing call site and every
# pre-existing sentence is byte-identical without it.
self.option = option
if not self.periods:
raise ProfileInvalid(
"static_profile has no periods. A profile with no periods is "
"not a fixed tariff, it is an absent one - leave the option "
"empty to say that on purpose (§11.4 then applies).")
_check_tiling(self.periods)
f"{option} has no periods. A profile with no periods is "
f"not a fixed tariff, it is an absent one - leave the option "
f"empty to say that on purpose (§11.4 then applies).")
_check_tiling(self.periods, option=option)
# -- what kind of tariff this is, DERIVED --------------------------------
@property
@@ -477,7 +512,7 @@ class StaticProfile:
u += timedelta(hours=1)
if not out:
raise ProfileInvalid(
f"static_profile.series({since.isoformat()}, "
f"{self.option}.series({since.isoformat()}, "
f"{until.isoformat()}) spans no whole hour. An empty series is "
f"not constructible downstream on purpose (price.py trap 3).")
return out
@@ -489,7 +524,7 @@ class StaticProfile:
return f"{kind}: " + "; ".join(p.words for p in self.periods)
def _check_tiling(periods) -> None:
def _check_tiling(periods, *, option: str = "static_profile") -> None:
"""Every day type must be covered by EXACTLY ONE period, every minute.
⚠️ REFUSES BOTH DIRECTIONS. A gap means an hour with no price, and the only
@@ -506,22 +541,22 @@ def _check_tiling(periods) -> None:
if n > 1 and over_from is None:
over_from = (minute, n)
if n != 0 and gap_from is not None:
_raise_gap(weekday, gap_from, minute)
_raise_gap(weekday, gap_from, minute, option)
if n <= 1 and over_from is not None:
_raise_overlap(weekday, over_from, minute, periods)
_raise_overlap(weekday, over_from, minute, periods, option)
if gap_from is not None:
_raise_gap(weekday, gap_from, MINUTES_PER_DAY)
_raise_gap(weekday, gap_from, MINUTES_PER_DAY, option)
if over_from is not None:
_raise_overlap(weekday, over_from, MINUTES_PER_DAY, periods)
_raise_overlap(weekday, over_from, MINUTES_PER_DAY, periods, option)
def _hhmm(m):
return f"{m // 60:02d}:{m % 60:02d}"
def _raise_gap(weekday, start, end):
def _raise_gap(weekday, start, end, option="static_profile"):
raise ProfileInvalid(
f"static_profile leaves {DAY_NAMES[weekday]} "
f"{option} leaves {DAY_NAMES[weekday]} "
f"{_hhmm(start)}-{_hhmm(end)} with no period, so those hours have no "
f"price. Refused rather than defaulted: an invented price is one an "
f"optimiser would trade on, and §11.4 already says what a missing price "
@@ -529,19 +564,28 @@ def _raise_gap(weekday, start, end):
f"\"22:00-07:00 mon-fri\" does NOT cover Saturday morning.")
def _raise_overlap(weekday, over, end, periods):
def _raise_overlap(weekday, over, end, periods, option="static_profile"):
start, n = over
names = [p.name for p in periods if p.covers(weekday, start)]
raise ProfileInvalid(
f"static_profile gives {DAY_NAMES[weekday]} {_hhmm(start)}-"
f"{option} gives {DAY_NAMES[weekday]} {_hhmm(start)}-"
f"{_hhmm(end)} {n} prices at once ({', '.join(names)}). Two prices for "
f"one hour means one of them silently wins on list order. Narrow one of "
f"the windows or split the day selector.")
def parse_static_profile(value):
def parse_static_profile(value, *, option: str = "static_profile",
means: str = None):
"""`static_profile` -> a `StaticProfile`, or None when unset.
⚠️ TARIFF-17 MADE `option`/`means` PARAMETERS RATHER THAN FORKING THIS
FUNCTION, and that is the whole of the reuse. A second period parser would
be a second tiling rule, a second day-type rule and a second set of DST
bugs, diverging first on the 23- and 25-hour days where nobody looks. The
two arguments move only the OPTION NAME and the "what this number is"
clause through the refusals; the grammar, the tiling check and the wrapping
rule are one implementation with one set of checks over it.
Accepts what the option can actually arrive as:
* `None` / `""` / `"null"` - unset. §14 ships `static_profile: null`.
* a JSON string - what config.yaml's flat `str?` option
@@ -555,6 +599,8 @@ def parse_static_profile(value):
"""
if value is None:
return None
if means is None:
means = _PERIOD_MEANS.get(option, _PERIOD_MEANS["static_profile"])
if isinstance(value, str):
text = value.strip()
if not text or text.lower() in ("null", "none", "off"):
@@ -563,7 +609,7 @@ def parse_static_profile(value):
value = json.loads(text)
except ValueError as e:
raise ProfileInvalid(
f"static_profile is not valid JSON ({e}). It is a JSON object "
f"{option} is not valid JSON ({e}). It is a JSON object "
f"with a \"periods\" list, on one line, e.g.\n"
f' {{"periods": [{{"name":"day","from":"07:00","to":"22:00",'
f'"days":"mon-fri","eur_per_kwh":0.34}}, ...]}}\n'
@@ -573,7 +619,7 @@ def parse_static_profile(value):
tz = str(value.get("timezone", "") or "").strip().lower()
if tz not in SUPPORTED_TZ:
raise ProfileInvalid(
f"static_profile.timezone is {value.get('timezone')!r}. Only "
f"{option}.timezone is {value.get('timezone')!r}. Only "
f"Europe/Brussels is supported: the shipped image carries no tz "
f"database (see app/price.py), so any other zone could not be "
f"honoured and would be silently treated as Brussels - which is "
@@ -581,24 +627,24 @@ def parse_static_profile(value):
rows = value.get("periods")
if rows is None:
raise ProfileInvalid(
"static_profile has no \"periods\" list. Even a fully-fixed "
"tariff is written as one period covering 00:00-24:00 all days "
"- one shape for both products, so nothing downstream branches "
"on which kind of site this is.")
f"{option} has no \"periods\" list. Even a fully-fixed "
f"tariff is written as one period covering 00:00-24:00 all days "
f"- one shape for both products, so nothing downstream branches "
f"on which kind of site this is.")
else:
rows = value
if not isinstance(rows, (list, tuple)):
raise ProfileInvalid(
f"static_profile.periods is {type(rows).__name__}, not a list")
f"{option}.periods is {type(rows).__name__}, not a list")
periods = []
for i, row in enumerate(rows, start=1):
if not isinstance(row, dict):
raise ProfileInvalid(
f"static_profile.periods[{i}] is {type(row).__name__}, not an "
f"{option}.periods[{i}] is {type(row).__name__}, not an "
f"object with from/to/days/eur_per_kwh")
where = f"static_profile.periods[{i}]"
where = f"{option}.periods[{i}]"
name = str(row.get("name", "") or f"period{i}").strip() or f"period{i}"
start = _minutes(row.get("from", row.get("start")), f"{where}.from")
end = _minutes(row.get("to", row.get("end")), f"{where}.to")
@@ -617,8 +663,47 @@ def parse_static_profile(value):
end_min=end,
days=_days(row.get("days"), f"{where}.days"),
eur_per_kwh=_price(row.get("eur_per_kwh"),
f"{where}.eur_per_kwh")))
return StaticProfile(periods)
f"{where}.eur_per_kwh",
means)))
return StaticProfile(periods, option=option)
def parse_taxes_profile(value):
"""`price_taxes_profile` -> a `StaticProfile` of TAX values, or None.
TARIFF-17. THE DEFECT THIS EXISTS TO CLOSE: `price.py` modelled the whole
non-energy part of a Belgian bill as ONE scalar. In Flanders that is
correct - Fluvius distribution is a single rate and its tariff card offers
only `Normaal | Uitsl. nacht`, where *exclusive night* is a separate
physical circuit rather than a night register. The Walloon and Brussels
DSOs are on the same card with four columns - `Enkelvoudig | Dag | Nacht |
Uitsl. nacht` - and the day/night gap is real money:
ORES dag 13,27 nacht 7,39 -> 5,88 c EUR/kWh
AIESH dag 15,17 nacht 8,21 -> 6,96
Tecteo RESA dag 12,20 nacht 7,01 -> 5,19
Sibelga (BXL) dag 9,39 nacht 7,10 -> 2,29
ORES IMPACT dag 16,578 nacht 5,090 -> 11,49 (2026)
⚠️ AND THE BANDS ARE ANTI-CORRELATED WITH WHAT THE BATTERY WANTS TO DO.
The cheap network hours are the cheap spot hours, so flattening them does
not merely shift the curve - it SYSTEMATICALLY UNDERSTATES the true spread
and makes the add-on decline cycles that are genuinely profitable. See
DOCS.md "The three buckets" for why the flat scalar could never have done
this: an additive constant, identical every hour, cancels exactly in
`max - min` and cannot move `min(firm, key=...)` either.
⚠️ THE VALUE IS THE TAXES COMPONENT, NOT AN ALL-IN PRICE. `static_profile`
uses the same grammar for an all-in figure; the two are one parser and two
meanings, and `_price()`'s refusal names which one it wanted.
⚠️ NOT PARSED ON THE FETCH PATH. validate.py refuses a malformed table at
STARTUP, by the rule `price.py` trap 3 states for `static_profile`: a table
that cannot be parsed at 03:00 is a site that stops pricing at 03:00, and
§11.4 makes that quiet.
"""
return parse_static_profile(value, option=TAXES_PROFILE_OPTION,
means=_TAXES_MEANS)
# --------------------------------------------------------------------------- #
@@ -728,6 +813,33 @@ class TariffPlan:
self.origin = origin
# -- derived facts -------------------------------------------------------
@property
def from_feed(self) -> bool:
"""Is this series an OBSERVATION of the configured provider's feed?
TARIFF-12. `origin` names four ways a plan is obtained; this collapses
them to the one question a caller holding a PREVIOUS plan has to ask
about a new one - did this resolution see the feed, or is it standing
in for one that did not? Only `"provider"` saw it. `"fallback"` and
`"none"` are the two §11.4 rows, and `"static"` is a site with no
feed to observe at all, where the question does not arise.
⚠️ IT IS NOT `has_prices`, AND THE DIFFERENCE IS THE WHOLE POINT. A
§11.4 fallback HAS prices - the operator's own static profile - but
every point `_degrade()` builds carries `provisional=True`, so
`firm_points()` is empty, `arbitrage()` refuses and `cheap_now()` is
False. It is a real tariff and it is not an observation, and a caller
deciding whether to REPLACE an observation with it needs the second
fact, not the first.
⚠️ AND IT IS A PROPERTY HERE RATHER THAN AN `origin == "provider"` AT
THE CALL SITE, for `main.fresh_tariff_plan()`'s stated reason: a check
has to be able to assert on the shipped expression rather than on a
transcription that drifts, and `origin`'s vocabulary belongs to this
module.
"""
return self.origin == "provider"
@property
def has_prices(self) -> bool:
return self.read.match(series=lambda _r: True, no_data=lambda _r: False)
@@ -744,6 +856,63 @@ class TariffPlan:
series=lambda r: any(p.provisional for p in r.points),
no_data=lambda _r: True)
# -- the plan's SHAPE, in the two units it actually has (TARIFF-11) ------
# ⚠️ POINTS AND HOURS ARE TWO NUMBERS AND THIS PLAN HAD ONLY EVER
# REPORTED ONE OF THEM, UNDER THE OTHER'S NAME. `words` read
# `f"... {len(self.read.points)} h"`, which counts POINTS: on the
# quarter-hourly Belgian feed (15-minute MTU, 2025-10-01) a 33-hour plan
# logged `dynamic_dayahead from generic_dayahead, 132 h`. It is the same
# conflation TARIFF-14 removed from `at()` and FCST-04 removed from the
# *Price history* row, left standing in the prose. ⚠️ AND THE ERROR IS
# NOT A FLAT 4x: a 40 h soak measured 1.79x .. 5.33x, because §8.3's
# carried points are HOURLY while the firm ones are quarter-hourly, so
# the ratio moves with the firm/carried mix as the day goes on. Nothing
# derived from ONE of these numbers can answer for the other.
@property
def n_points(self) -> int:
"""How many points the series carries. A `no_data` read carries none."""
return self.read.match(series=lambda r: len(r.points),
no_data=lambda _r: 0)
@property
def hours(self) -> int:
"""How many distinct WHOLE CLOCK HOURS those points touch.
Same rule as `price_history._held()`, deliberately - the *Tariff* and
*Price history* rows describe the same feed and an operator reading
`132 points over 33 h` beside `96 points over 24 h` must be able to
compare them. A point is attributed to the UTC hour its own start
falls in; hours the feed never printed are simply not in the set, so
this counts what is PRICED rather than what was asked for.
"""
return self.read.match(series=lambda r: _whole_hours(r.points),
no_data=lambda _r: 0)
@property
def n_firm(self) -> int:
"""Points carrying a published price for their own interval (§8.3)."""
return len(self.firm_points())
@property
def n_carried(self) -> int:
"""Points filled from yesterday's profile, or from §11.4's fallback."""
return self.n_points - self.n_firm
@property
def spacings_s(self) -> tuple:
"""The distinct gaps between consecutive points, ascending, in seconds.
⚠️ MIXED IS THE NORMAL STEADY STATE, NOT AN EDGE CASE. FCST-04's
carry-forward emits HOURLY carried points beside QUARTER-HOURLY firm
ones and the soak measured `[900, 3600]` in 70 % of samples - so a
surface reporting one global spacing would be wrong about the usual
case. `price._cover_table()` is where that rule is enforced; this
only reports it.
"""
return self.read.match(
series=lambda r: _spacings(r.points), no_data=lambda _r: ())
def price_at(self, instant: datetime) -> float:
"""EUR/kWh now. Raises `Absence` when there is no price - never 0.0."""
return self.read.match(
@@ -899,20 +1068,30 @@ class TariffPlan:
pts = self.read.points
firm = tuple(p for p in pts if not p.provisional)
if len(firm) < 2:
# ⚠️ POINTS, AND THE HOUR FIGURE BESIDE IT RATHER THAN INSTEAD OF
# IT (TARIFF-11). This sentence read "only 1 of 96 hours carry a
# published price" about 96 quarter-hourly points, which are 24
# hours - the point<->hour conflation TARIFF-14 removed from
# `at()`, still standing here, and this one reaches an OPERATOR
# through main.py's *Arbitrage* row and not only the log.
return Arbitrage.refuse(
f"§8.3: only {len(firm)} of {len(pts)} hours carry a published "
f"price - the rest are provisional (origin {self.origin!r}), "
f"and no irreversible action may be taken on a provisional "
f"price. A buy hour and a sell hour must BOTH be firm before a "
f"cycle can be committed. Load shifting, self-consumption and "
f"peak shaving continue")
f"§8.3: only {len(firm)} of {len(pts)} points carry a "
f"published price - {_whole_hours(firm)} of "
f"{_whole_hours(pts)} hours - and the rest are provisional "
f"(origin {self.origin!r}), so no irreversible action may be "
f"taken on them. A buy interval and a sell interval must BOTH "
f"be firm before a cycle can be committed. Load shifting, "
f"self-consumption and peak shaving continue")
spread = round(max(p.eur_per_kwh for p in firm)
- min(p.eur_per_kwh for p in firm), 6)
if spread == 0.0:
# ⚠️ POINTS AND HOURS, SEPARATELY, for the §8.3 rung's reason -
# this sentence counted points and called them "firm hours" too.
return Arbitrage.refuse(
f"the tariff is flat - every one of the {len(firm)} firm hours "
f"costs {firm[0].eur_per_kwh:.4f} EUR/kWh, so buying low and "
f"the tariff is flat - every one of the {len(firm)} firm "
f"points, spanning {_whole_hours(firm)} hours, costs "
f"{firm[0].eur_per_kwh:.4f} EUR/kWh, so buying low and "
f"selling high is arithmetically impossible. §9.2: a fully "
f"fixed product is self-consumption only")
@@ -942,15 +1121,77 @@ class TariffPlan:
@property
def words(self) -> str:
"""The plan's whole shape in one sentence - LOG *AND* STATUS PAGE.
⚠️ THIS STRING IS OPERATOR-FACING SINCE TARIFF-11, which is a reason
for its wording to matter MORE rather than less. It used to reach only
`_LOG.info("tariff refreshed: %s")`; main.py's *Tariff* row now carries
it, age-stamped, so every number in it is read by somebody who cannot
see the object.
⚠️ EVERY QUANTITY IS NAMED IN ITS OWN UNIT. `{n} points over {h} h` is
FCST-04's *Price history* wording on purpose - the two rows describe
the same feed and have to be comparable - and the firm/carried split
and the spacing are here because they were the two facts the soak
found an operator could not learn at all: the plan was `provisional`
for 76 % of a 40-hour soak and mixed 900/3600 s for 70 %, and no HTTP
surface said either.
"""
if not self.has_prices:
return f"no tariff - {self.read.reason}"
tag = {"provider": self.read.provider, "static": "static_profile",
"fallback": "static_profile (feed down)"}.get(self.origin,
self.origin)
return (f"{self.product.value} from {tag}, {len(self.read.points)} h"
gaps = self.spacings_s
return (f"{self.product.value} from {tag}, "
f"{self.n_points} points over {self.hours} h, "
f"{self.n_firm} firm / {self.n_carried} carried"
+ (f", {'+'.join(str(g) for g in gaps)} s spacing"
if gaps else "")
+ (", provisional" if self.provisional else ""))
def _whole_hours(points) -> int:
"""How many distinct whole CLOCK HOURS a point list touches. Never raises.
THE DEFECT THIS PREVENTS (TARIFF-11). Three sentences in this module
reported `len(points)` under the name "hours": `TariffPlan.words`, §8.3's
firm-pair refusal and the flat-tariff refusal. On the hourly series that
was the only resolution any fixture carried before TARIFF-08 the two
numbers are identical, which is why it shipped; on the Belgian
quarter-hourly feed (15-minute MTU, 2025-10-01) `words` read
`dynamic_dayahead from generic_dayahead, 132 h` for 33 hours and the
refusal read `only 1 of 96 hours` for 24.
⚠️ IT IS NOT `n / 4`, AND A CONSTANT FACTOR IS THE WRONG FIX. §8.3's
carry-forward emits ONE HOURLY point per unpublished hour beside the
feed's four-per-hour firm ones, so the measured overstatement over a
40-hour soak ranged 1.79x .. 5.33x as the mix moved. The hour count has
to be COUNTED.
⚠️ IT COUNTS PRICED HOURS, NOT THE HORIZON. FCST-04 leaves an hour out
of the series when neither the feed nor yesterday can price it, so this
is at most the horizon and is smaller whenever the plan has holes - which
is the honest direction for a figure an operator reads as "how far ahead
do I have prices".
"""
return len({p.start.astimezone(timezone.utc)
.replace(minute=0, second=0, microsecond=0) for p in points})
def _spacings(points) -> tuple:
"""Distinct gaps between consecutive points, ascending, in whole seconds.
Sorted by START, not by list order: `price._cover_table()` establishes
that order for `at()` rather than assuming it, and a surface that reported
the gaps of an unsorted list would print negative spacings about a series
that answers correctly.
"""
starts = sorted(p.start.astimezone(timezone.utc) for p in points)
return tuple(sorted({int((b - a).total_seconds())
for a, b in zip(starts, starts[1:])}))
def _absent(reason):
raise Absence(
f"this plan observed no prices ({reason}). A number invented here is "
@@ -1019,7 +1260,7 @@ def static_status(opts: dict, now: datetime):
async def resolve_tariff(opts: dict, since: datetime, until: datetime,
transport=None, now=None) -> TariffPlan:
transport=None, now=None, *, history) -> TariffPlan:
"""The full §9.2 + §11.4 resolution. MAY PERFORM I/O - see `static_plan`.
⚠️ NOT ON THE 30 s TICK, and that is a deliberate boundary rather than an
@@ -1033,17 +1274,64 @@ async def resolve_tariff(opts: dict, since: datetime, until: datetime,
price_provider off + profile -> the CONFIGURED product (§9.2 rows 3-4)
price_provider off + none -> no prices at all (§11.4, nothing to
fall back to)
provider series -> the feed (§9.2 rows 1-2)
provider series -> the feed, §8.3-FILLED (§9.2 rows 1-2)
provider no_data + profile -> §11.4 FALLBACK, provisional, no arbitrage
provider no_data + none -> no prices at all
⚠️ `history` IS REQUIRED, KEYWORD-ONLY, AND HAS NO DEFAULT (FCST-04). It is
the §8.3 carry-forward store - `price_history.PriceHistory`, or the explicit
`price_history.NoHistory()` for a caller that keeps none. A default of
`None` meaning "skip the fill" would give a caller who forgot the argument
the TRUNCATED HORIZON this ticket exists to remove, silently, while the code
read as though the fill were happening. There is nothing to forget now.
⚠️ AND THE FILL RUNS ONLY ON A `series` READ. A feed that published nothing
still falls to `_degrade()` and §11.4's own row - the static profile the
operator configured FOR that case - rather than to yesterday's feed. The two
are indistinguishable to the optimiser (both are 100 % provisional, so
`arbitrage()` finds fewer than two firm hours and refuses, and `cheap_now()`
is False), so preferring yesterday would change a documented degradation for
a difference only the status page can see. See price_history.py's closing
note and DECISIONS/FCST-04-the-day-that-is-carried.md.
⚠️ THE FILL IS PER WHOLE HOUR AND ITS COVERAGE IS CONDITIONAL. Every point
the provider published inside the window passes through BY IDENTITY - four
of them in one hour on the quarter-hourly day-ahead feed `generic_dayahead`
reads (Belgium's 15-minute MTU, 2025-10-01) - and a
carried point is added only for a whole hour the feed did not print at all
AND yesterday did. An hour neither source can price is left out, which is
§11.4 for that hour: `plan.price_at()` raises rather than answering, and an
hour with no price is not a free one.
⚠️ THE FILLED SERIES IS WINDOWED TO [since, until), AND THAT IS A CHANGE.
`load_prices()` returns whatever the feed sent, past hours included, and
`TariffPlan.arbitrage()` took its spread over all of it - so an hour that
had already gone could be the cheapest hour of a cycle nothing can now buy
in. `fill_from_yesterday()` walks the requested horizon and emits exactly
that, so the spread is over hours that are still ahead. Both directions of
the difference are real: the window gains the unpublished hours it was
missing and loses the elapsed ones it should never have had.
"""
stamp = now or datetime.now(timezone.utc)
pid = str(price_option(opts, "price_provider") or "off").strip()
if pid in ("", "off"):
if not provider_configured(opts):
return static_plan(opts, since, until, provisional=False, now=stamp)
read = await load_prices(opts, since, until, transport)
return read.match(
series=lambda r: _provider_plan(opts, r, since, until, stamp, history),
no_data=lambda r: _degrade(opts, r, since, until, stamp))
def _provider_plan(opts, read, since, until, stamp, history) -> TariffPlan:
"""§8.3: record what the feed published, carry yesterday into the gaps.
The fill can itself return `no_data` - a feed whose points all fall outside
the horizon, with nothing in the history to fill it from - and that is the
§11.4 row arrived at one step later, so it degrades exactly as a failed
fetch does rather than being special-cased here.
"""
filled = history.carry(read, since=since, until=until, stamp=stamp)
return filled.match(
series=lambda r: TariffPlan(r, _quiet_profile(opts), "provider"),
no_data=lambda r: _degrade(opts, r, since, until, stamp))
+124 -3
View File
@@ -132,6 +132,18 @@ OBSERVED = "observed" # the store answered and there is a peak
NO_RECORD = "no_record" # the store answered and this month has none yet
NOT_OBSERVED = "not_observed" # the store did not answer. NOT a zero peak.
# TARIFF-13. What the COVERAGE figure reads when the store cannot count.
#
# ⚠️ NOT 0, AND NOT `None`. This is the reading-state argument of TARIFF-01
# applied to the number printed BESIDE the peak rather than to the peak. A
# coverage figure of 0 says "this month has been measured over no quarters at
# all", which is the strongest possible claim that the peak next to it means
# nothing - and `f"{None} quarter(s) banked"` renders "None quarter(s) banked",
# which an operator reads as that same zero. The one honest answer to "how many
# quarters have I banked" when the table will not answer is that it is not
# known, and it has to SAY so in the sentence the operator reads.
COUNT_UNREADABLE = "an unreadable number of"
# --------------------------------------------------------------------------- #
# local time, without a hard dependency on the tz database
@@ -388,6 +400,23 @@ class PeakStore:
" per_phase_json=excluded.per_phase_json"
" WHERE excluded.offtake_avg_w > quarter_blocks.offtake_avg_w",
(start, month, w, pp_json))
# TARIFF-13. THE TIE RULE, DECIDED RATHER THAN EMERGENT:
# at EQUAL wattage the EARLIEST quarter keeps the attribution.
# See DECISIONS/TARIFF-13-the-coverage-of-a-peak.md §2.
#
# `peak_w` itself is still only ever RAISED - the `>` term is
# untouched and no tie can move the billed number - so SAFETY-07
# reads exactly the watt figure it read before. The second term
# moves only the TIMESTAMP the status page attributes it to.
#
# ⚠️ WITHOUT THE SECOND TERM THE TWO READERS DISAGREE, measured:
# `>` alone keeps whichever tied block was WRITTEN FIRST, while
# `recompute_month()`'s `ORDER BY offtake_avg_w DESC,
# quarter_start ASC` keeps the EARLIEST. Those coincide only
# while blocks arrive in time order. Fed four equal 365 W blocks
# newest-first, `monthly_peak` answered 09:45 and the recompute
# answered 09:00 - a divergence in the derived record part 8 of
# test_tariff.py exists to prove cannot happen.
self._db.execute(
"INSERT INTO monthly_peak(month, peak_w, quarter_start, per_phase_json)"
" VALUES(?,?,?,?)"
@@ -395,7 +424,9 @@ class PeakStore:
" peak_w=excluded.peak_w,"
" quarter_start=excluded.quarter_start,"
" per_phase_json=excluded.per_phase_json"
" WHERE excluded.peak_w > monthly_peak.peak_w",
" WHERE excluded.peak_w > monthly_peak.peak_w"
" OR (excluded.peak_w = monthly_peak.peak_w"
" AND excluded.quarter_start < monthly_peak.quarter_start)",
(month, w, start, pp_json))
return True
except sqlite3.Error as err:
@@ -449,6 +480,49 @@ class PeakStore:
quarter_start=datetime.fromisoformat(row[1]),
per_phase_w=pp)
def banked_count(self, month: str) -> int | None:
"""How many quarters this month has banked. `None` when unknowable.
TARIFF-13. THE DEFECT THIS PREVENTS: the status page printed a coverage
figure held in PROCESS MEMORY - `PeakTracker.recorded`, incremented once
per block banked by THIS process - beside a peak read from SQLite. The
peak outlives a restart and the counter does not, so a driven restart
mid-month took a real reading of
2026-09: peak 365 W ... ; 120 quarter(s) banked
to the same peak, the same attributed quarter and `0 quarter(s) banked`,
on both instances independently. The count is what tells an operator how
much of the billing month the number beside it actually covers: 365 W
over 120 quarters and 365 W over 0 quarters are different claims about
the same watt figure, and after a restart the page made the weaker claim
while displaying the stronger one's value. Reading the count from the
same durable tier the peak comes from is what makes the two agree.
⚠️ `None`, NOT 0, and NOT `len(blocks_in(month))`. `blocks_in()` returns
`[]` on a read error - safe there only because no caller reads its
emptiness as a peak (§ its own docstring) - so counting its length would
rebuild the disarming zero one call further out. This asks the database
for a COUNT and refuses to answer when the database refuses.
⚠️ Counts `quarter_blocks`, the tier with one row per closed block, and
NOT `monthly_peak`, which has one row per MONTH and would answer 1 for
every month that has ever been measured. The two tables are proven equal
on the PEAK by `recompute_month()`; they were never equal on the COUNT
and are not meant to be.
"""
if self._db is None:
return None
try:
row = self._db.execute(
"SELECT COUNT(*) FROM quarter_blocks WHERE month=?",
(month,)).fetchone()
except sqlite3.Error as err:
self.last_error = str(err)
_LOG.error("could not count quarters for %s (%s)", month, err)
return None
return int(row[0])
def blocks_in(self, month: str) -> list:
"""(quarter_start, offtake_avg_w) for a month, oldest first. [] on error
is safe here ONLY because no caller reads emptiness as a peak - AC 3's
@@ -539,11 +613,59 @@ class PeakTracker:
self.cap_w = float(cap_w)
self.policy = policy
self.checkpoint_s = float(checkpoint_s)
self.recorded = 0
self.restore_state = "not attempted"
self._seen: set = set()
self._last_ckpt = 0.0
# -- coverage -----------------------------------------------------------
#
# TARIFF-13. `recorded` USED TO BE AN `int` ATTRIBUTE ON THIS OBJECT,
# starting at 0 and incremented in `sync()`. It is a property over the store
# now, and the three things that changes are all defects it was carrying:
#
# * it RESET TO 0 ON EVERY RESTART, beside a peak that did not - see
# `PeakStore.banked_count()` for the driven reading;
# * it was NOT SCOPED TO A MONTH. It counted every block this process had
# banked, so on 1 October the row still read September's tally beside
# October's "no completed quarter recorded yet";
# * it MISSED the block `restore()` banks. A stale block closed from the
# checkpoint goes into `quarter_blocks` and never went through `sync()`,
# so the counter never saw it. Counting the table cannot miss it.
#
# ⚠️ REMOVING THE COUNTER DOES NOT REMOVE THE DEDUPE. `_seen` is still what
# stops `cycle()`'s once-a-second `sync()` re-banking a closed quarter that
# `P1Ingest.blocks` keeps for a DAY, and `sync()`'s RETURN value is still
# the count of blocks newly banked on that call. What changed is only where
# the STATUS PAGE's figure is read from. The check that grades the dedupe
# must therefore grade `sync()`'s return, not this property, which now
# answers 1 from the database however many times the block was offered.
def recorded_in(self, when: datetime | None = None):
"""Quarters banked in `when`'s billing month - `int`, or the sentinel.
⚠️ The return type is `int | str` on purpose, and the `str` is
`COUNT_UNREADABLE`. A caller that formats this into a sentence gets a
true sentence in both cases; a caller that wants to do arithmetic on it
gets a TypeError, which is loud. That is the same trade `PeakReading`
makes by having no `peak_w_or(default)`.
"""
n = self.store.banked_count(month_key(when or datetime.now(timezone.utc)))
return COUNT_UNREADABLE if n is None else n
@property
def recorded(self):
"""Quarters banked in the month being billed TODAY.
⚠️ This is the live status page's reading, where the peak beside it is
also read for `datetime.now()`. `checks(when=X)` for an X in another
month is a test affordance (TEST-17); the row scoped that way gets this
month's count beside that month's peak, which is strictly better than
the process-wide counter it replaces but is still a mismatch.
`recorded_in(when)` is the scoped form the row should read once
`main.py`'s status rows are free to change - TARIFF-11 holds them.
"""
return self.recorded_in()
# -- startup ------------------------------------------------------------
def restore(self, averager, now: datetime | None = None) -> str:
"""Put the open block back. Returns which of the three cases happened."""
@@ -600,7 +722,6 @@ class PeakTracker:
continue
if self.store.record_block(block, self.policy):
self._seen.add(key)
self.recorded += 1
n += 1
if n:
# A block just closed, so the checkpoint's block is gone: clearing it
+254 -11
View File
@@ -118,8 +118,14 @@ NOT READY banner already names the problem, (b) does not apply.
It is warned about below, not refused.
7. A price configuration that cannot mean what it says. [(a)]
Three cases, all in `_price_problems()`: an unknown `price_provider` id
(refused rather than defaulted to 'off', per the §14 warning box);
Four cases, all in `_price_problems()`: an unknown `price_provider` id
(refused rather than defaulted to 'off', per the §14 warning box); a
RETIRED id - one this add-on really did ship and has been accepting ever
since, whose plugin never once fetched a price because its endpoint does
not exist (TARIFF-07), which is case (a) in its purest form: §11.4's
silent fall back to the static profile IS the layer the operator cannot
see, and it has been overriding their chosen source since the day they
set it;
`price_provider` set alongside `price_now_entity`/`price_avg_entity`,
which is TWO PRICE SOURCES ON ONE SITE where the Home Assistant pair
silently wins; and `manual_csv` with no `price_csv_path`, which turns the
@@ -318,12 +324,18 @@ import logging
from .arbitrage import enabled as arbitrage_on
from .control import SLEW_RAIL_FRAC, LOOP_TICK_S, PLANT_SETTLE_S
from .custom_formula import (
FormulaInvalid, implausible_all_in, parse_price_formula, stale_card)
from .inverter import roster_entries
from .maintenance import MAINT_OPTION_DEFAULTS
from .p1 import is_enabled, validate_meter_options
from .peakguard import PEAK_OPTION_DEFAULTS
from .price import PRICE_OPTION_DEFAULTS, PROVIDERS
from .static_tariff import ProfileInvalid, parse_static_profile
from .price import (
PRICE_OPTION_DEFAULTS, PROVIDERS, RETIRED_PROVIDERS,
SUPERSEDED_TAXES_DEFAULTS_EUR_KWH)
from .static_tariff import (
ProfileInvalid, TAXES_PROFILE_OPTION, parse_static_profile,
parse_taxes_profile)
from .watchdog import SIGN_OPTION_DEFAULTS
_LOG = logging.getLogger("goodwe.config")
@@ -1242,17 +1254,74 @@ def _price_problems(opts: dict) -> list[str]:
without a price feed is an add-on that a Belgian fixed-tariff site - a large
share of the installed base, per §9.2 - cannot run at all.
Four refusals and two log lines (TARIFF-03 added one refusal, one parse and
two notices), and every refusal is still case (a) from this module's
docstring: a source the operator SET being overridden by a layer they cannot
see. The additions are `static_profile` alongside the Home Assistant price
pair - the same silent-loser shape as provider-plus-pair, and a LIVE one -
and a `static_profile` that does not tile the day, which would leave hours
with no price at all.
Five refusals and two log lines (TARIFF-03 added one refusal, one parse and
two notices; TARIFF-07 added the retired-id one; TARIFF-15 added three -
a `custom_formula` with no formula, a formula that does not parse or does
not reproduce its own card, and one whose card lands outside the plausible
band - plus two notices, a formula nothing reads and a stale card), and
every refusal is still case (a) from this module's docstring: a source the
operator SET being overridden by a layer they cannot see. The additions are `static_profile`
alongside the Home Assistant price pair - the same silent-loser shape as
provider-plus-pair, and a LIVE one - and a `static_profile` that does not
tile the day, which would leave hours with no price at all.
⚠️ THE RETIRED-ID BRANCH IS NOT A FRIENDLIER SPELLING OF THE UNKNOWN-ID
ONE, AND MUST NOT COLLAPSE INTO IT. They answer different questions. An
unknown id is a TYPO: the operator meant something else and the fix is to
spell it right. A retired id is a CHOICE THE ADD-ON ACCEPTED - it was in
the schema, it was in the dropdown, it resolved to a class, and every
startup since said nothing. What that operator needs is not "this id is
wrong" but "this id has never once returned a price, and here is what your
site has actually been running instead" - because §11.4 made the failure
silent and NOTHING on their status page has ever distinguished it from a
working provider that never finds a cheap hour. The retired branch is
therefore FIRST, and it names the id, the endpoint, the observation, the
tariff that has really been in force, and the move to make.
"""
problems: list[str] = []
pid = str(opts.get("price_provider", "") or "off").strip() or "off"
if pid in RETIRED_PROVIDERS:
# ⚠️ BEFORE the unknown-id branch, and the order is load-bearing: a
# retired id is ALSO not in PROVIDERS, so the generic message would
# win on a reordering and the operator would be told they mistyped
# something they chose on purpose.
# ⚠️ AND THE SECOND SENTENCE CHANGES WITH `static_profile`, because
# "you have been on the static profile" is FALSE on a site that never
# set one. §11.4 falls back to the static profile; with no profile to
# fall back to, the site has had no tariff at all - a worse state, and
# one this message must not paper over.
been_running = (
f"the static day/night tariff in static_profile"
if str(opts.get("static_profile", "") or "").strip()
else "NOTHING AT ALL - static_profile is empty too, so there was "
"no tariff to fall back to either")
problems.append(
f"price_provider is {pid!r}, which WAS a shipped plugin and has "
f"been WITHDRAWN (TARIFF-07). This is not a typo and you did not "
f"mistype it - the add-on offered this id, accepted it, and has "
f"said nothing at every startup since. "
f"⚠️ IT HAS NEVER FETCHED A PRICE, HERE OR ANYWHERE: "
f"{RETIRED_PROVIDERS[pid]}. Every fetch it ever made ended in "
f"SourceUnavailable, and §11.4 turns that into a SILENT fall back "
f"- so what this site has actually been running since the day the "
f"id was set is {been_running}. Nothing you could see would have "
f"differed from a working provider that simply never found a cheap "
f"hour, which is why you are being told now rather than left on "
f"it. WHAT TO DO: set price_provider to 'generic_dayahead' for the "
f"wholesale EPEX BE day-ahead curve, to which the add-on adds your "
f"price_markup_eur_kwh and price_taxes_eur_kwh - that APPROXIMATES "
f"a Belgian dynamic tariff and cannot reproduce your supplier's "
f"B x spot + A formula, so the level will be off even when the "
f"shape is right; or 'manual_csv' to enter all-in prices you "
f"compute yourself; or 'off' to run on static_profile "
f"deliberately. There is deliberately NO alias to your old "
f"supplier: aliasing would need your contract's own two "
f"coefficients, which live behind your klantenzone login and are "
f"in no public index, so the add-on would have to invent the "
f"numbers it prices your battery against.")
return problems # nothing below can be judged against a dead id
if pid != "off" and pid not in PROVIDERS:
problems.append(
f"price_provider is {pid!r}, which is not a shipped plugin. The "
@@ -1355,6 +1424,180 @@ def _price_problems(opts: dict) -> list[str]:
"for this product. Peak shaving, reserve management and the "
"maintenance cycle are unaffected.", profile.prices[0])
# ⚠️ TARIFF-17: THE TAXES PERIOD TABLE, PARSED AT STARTUP RATHER THAN AT
# 03:00, FOR `static_profile`'s REASON ONE OPTION OVER. This table is read
# by `price.assemble()` on the ~30 s planning path; a table that only fails
# when a fetch happens is a site that stops pricing mid-afternoon, and
# §11.4 turns that into a quiet fallback that looks exactly like a healthy
# fixed-tariff day. So the parse happens here, once, where a refusal is a
# banner naming the key.
raw_taxes_profile = opts.get(TAXES_PROFILE_OPTION, "")
taxes_table = None
if str(raw_taxes_profile or "").strip() \
or isinstance(raw_taxes_profile, (list, dict)):
try:
taxes_table = parse_taxes_profile(raw_taxes_profile)
except ProfileInvalid as e:
problems.append(f"{TAXES_PROFILE_OPTION} cannot be used: {e}")
if taxes_table is not None:
# ⚠️ TWO WRITERS OF ONE QUANTITY, ONE OF WHICH SILENTLY LOSES - the
# exact case (a) this module exists for, and the same refusal shape as
# the price-entity/static_profile pair above. The table decides when it
# is set, so a non-default scalar left beside it is a number the
# operator typed that changes nothing, on the quantity §11.4 makes
# hardest to notice. It is compared against the SHIPPED DEFAULTS rather
# than against presence, because the Supervisor writes every schema key
# into the options file whether the operator touched it or not - so
# "the operator did not type this" has to be read off the VALUE.
#
# ⚠️ TARIFF-18: "UNTOUCHED" IS A SET, NOT ONE NUMBER, AND THE DEFECT
# THAT MAKES IT ONE IS AN UPGRADE THAT STOPS A BATTERY CONTROLLER
# STARTING. The Supervisor sets `App.options = deepmerge(config.yaml
# defaults, stored options)`, so a saved blob carries every schema key
# whether the operator typed it or not - the reference install's export
# is 70 of 71. When TARIFF-18 moved this default 0.06 -> 0.11736, every
# such blob went from "equal to the shipped default" to "not equal to
# it" without anybody editing anything, and a site that also carries a
# taxes table would have met `SystemExit(78)` on the upgrade. The
# sentence below is about a number the OPERATOR TYPED; a superseded
# shipped default is not one, so it still reads as untouched. The set
# is in price.py beside the default it supersedes, because the two only
# ever move together.
scalar = _number(opts, "price_taxes_eur_kwh",
PRICE_OPTION_DEFAULTS["price_taxes_eur_kwh"])
shipped = PRICE_OPTION_DEFAULTS["price_taxes_eur_kwh"]
untouched = (shipped,) + tuple(SUPERSEDED_TAXES_DEFAULTS_EUR_KWH)
superseded = ", ".join(
str(u) for u in SUPERSEDED_TAXES_DEFAULTS_EUR_KWH)
if scalar is not None \
and not any(abs(scalar - u) <= 1e-12 for u in untouched):
problems.append(
f"{TAXES_PROFILE_OPTION} is set AND price_taxes_eur_kwh is "
f"{scalar} rather than its shipped {shipped}. THE TABLE WINS: "
f"the add-on reads the per-period figures and never looks at "
f"the scalar, so the number you typed there would decide "
f"nothing and nothing would say so. Set price_taxes_eur_kwh "
f"back to {shipped}, or clear {TAXES_PROFILE_OPTION} to keep "
f"the flat figure. (A scalar still sitting on a SUPERSEDED "
f"default - {superseded} - is accepted and NOT refused: "
f"that is a number this add-on wrote into your options "
f"blob, not one you typed.)")
# ⚠️ THE UNITS RUNG, AND IT IS A SECOND QUANTITY RATHER THAN A
# RESTATEMENT. Every Belgian DSO prints these bands in c EUR/kWh
# (ORES `13,27`), and the whole table is copied off one card - so the
# factor-100 error arrives on EVERY row at once and nothing internal to
# the table disagrees with itself. The parser's own +-10 EUR/kWh bound
# catches a 13,27 typed raw, but NOT ORES IMPACT's night band of 5,090,
# which would be a 5.09 EUR/kWh levy sailing through. Belgian
# non-energy components run roughly 0.05-0.25 EUR/kWh, so anything at
# or above 1.0 is the wrong unit and not a tariff.
steep = sorted({p.eur_per_kwh for p in taxes_table.periods
if abs(p.eur_per_kwh) >= 1.0})
if steep:
problems.append(
f"{TAXES_PROFILE_OPTION} carries {steep} EUR/kWh. Belgian "
f"levies and distribution run about 0.05-0.25 EUR/kWh, so "
f"these are almost certainly the CENT figures off the tariff "
f"card (ORES prints `13,27`, which is 0.1327 EUR/kWh) typed "
f"without the conversion - a factor of 100 that would make "
f"every hour look like an emergency. Divide by 100, or clear "
f"the option.")
negative = sorted({p.eur_per_kwh for p in taxes_table.periods
if p.eur_per_kwh < 0})
if negative:
# ⚠️ A WARNING, NOT A REFUSAL. A negative non-energy component is
# implausible rather than impossible - a prosumer rebate is
# expressible this way - and refusing a site over it would be this
# module asserting a fact about Belgian tariff design it cannot
# observe. §14's warning box governs CONTRADICTIONS, and this is
# not one.
_LOG.warning(
"%s carries negative levies %s EUR/kWh. That is accepted, but "
"a tax component below zero is usually a sign error: the "
"add-on will SUBTRACT it from the published price in those "
"hours.", TAXES_PROFILE_OPTION, negative)
# ⚠️ TARIFF-15: THE FORMULA, GRADED AGAINST THE OPERATOR'S OWN CARD, AT
# STARTUP RATHER THAN AT 03:00.
#
# THE DEFECT THIS EXISTS TO PREVENT, and it is not one `code_leaks()` can
# see because it is not in code: an operator folds their NETWORK CHARGES
# into `offset` while `price_taxes_eur_kwh` carries them as well, and the
# site is billed against a price ~12 c EUR/kWh high on every hour for as
# long as nobody re-reads the configuration. §11.4 makes every price
# defect quiet, so nothing on the status page would ever differ.
#
# The card is what grades it. A Belgian tariff card prints the formula,
# the reference index it assumed AND the resulting price, so the add-on
# recomputes that printed price from the numbers the operator typed and
# refuses to start unless the two agree to within CARD_TOLERANCE. An
# inflated `offset` cannot pass - the printed estimate is the energy
# component alone, so ~12 c EUR/kWh moves 15.55 to ~27. The same check
# catches VAT omitted, VAT applied twice, a transposed digit, and a unit
# MISMATCH between the coefficients and the printed estimate. The one unit
# error it cannot see is the one applied to BOTH sides at once, because
# that cancels out of a comparison between them - which is why
# `implausible_all_in()` below is a second refusal on a second quantity
# rather than a restatement of this one.
#
# ⚠️ AND ITS LIMIT, IN THE SAME SENTENCE AS THE CLAIM, exactly as price.py
# states trap 1's: IT GRADES THE COEFFICIENTS AGAINST THE CARD, NOT THE
# CARD AGAINST REALITY. A consistent-but-wrong pair - both numbers off the
# wrong product - reproduces its own printed price and passes. Nothing can
# catch that.
raw_formula = opts.get("price_formula", "")
has_formula = bool(str(raw_formula or "").strip()) \
or isinstance(raw_formula, dict)
if pid == "custom_formula" and not has_formula:
problems.append(
"price_provider is 'custom_formula' but price_formula is empty, "
"so the add-on has no coefficients to price anything with. Every "
"fetch would return no-price-data and §11.4 would fall back "
"silently - which is the shape of defect this provider exists to "
"remove, arrived at by leaving one field blank. Copy the "
"Tariefformule line off your supplier's tariff card, e.g. "
"{\"index\":\"epex_be\",\"resolution\":\"hourly\","
"\"factor\":0.102,\"offset\":1.00,"
"\"units\":\"c_eur_per_kwh\",\"vat_pct\":6,"
"\"card\":{\"month\":\"2026-09\","
"\"index_eur_per_mwh\":133.9905,\"expect_incl_vat\":15.55}} "
"- see DOCS.md. Or set price_provider to 'generic_dayahead' for "
"the wholesale curve with a flat markup.")
elif has_formula:
try:
spec = parse_price_formula(raw_formula)
except FormulaInvalid as e:
spec = None
problems.append(f"price_formula cannot be used: {e}")
if spec is not None and pid != "custom_formula":
# ⚠️ A WARNING AND NOT A REFUSAL. This is not two price sources
# fighting over one flag - it is one that is simply not wired to
# anything - so the §14 warning box does not apply. But an option
# an operator filled in and that decides nothing is exactly the
# silent loser this module's case (a) is about, so it is said out
# loud rather than left to be discovered from a bill.
_LOG.warning(
"price_formula is filled in but price_provider is %r, so the "
"coefficients decide NOTHING - only 'custom_formula' reads "
"them. Set price_provider to 'custom_formula' to use them, or "
"clear price_formula so the configuration says what the "
"add-on is doing.", pid)
if spec is not None and pid == "custom_formula":
taxes = _number(opts, "price_taxes_eur_kwh",
PRICE_OPTION_DEFAULTS["price_taxes_eur_kwh"])
band = implausible_all_in(spec, taxes if taxes is not None else 0.0)
if band:
problems.append(band)
aged = stale_card(spec)
if aged:
# ⚠️ WARN, NEVER REFUSE. ENGIE freezes a contract's
# coefficients for its duration, so a two-year-old card can be
# exactly right and a site refused over one is a site that
# cannot start.
_LOG.warning("%s", aged)
# ⚠️ A WARNING, NOT A REFUSAL, AND THE ASYMMETRY IS DELIBERATE. §11.4 says a
# site with no price data falls back to the static profile; a dynamic site
# with none configured has nothing to fall back to when its feed dies. That
+173 -10
View File
@@ -265,9 +265,17 @@ options:
# TARIFF - what energy costs, and whether to trade on it
# ==========================================================================
# `off` is the shipped default and means the §11.4 no-price-data row applies:
# static profile, plan provisional, NO ARBITRAGE. Nothing in 0.5.x consumes
# the series yet - OPT-04 is the consumer - so turning this on today fetches,
# validates and publishes prices and changes no energy decision.
# static profile, plan provisional, NO ARBITRAGE.
#
# ⚠️ SELECTING A PROVIDER IS WHAT STARTS THE FETCH, AND IT IS NOT
# `arbitrage_enabled` (TARIFF-10). With a provider set and trading off - the
# shipped default - the add-on fetches the feed every 30 minutes, validates
# it, records the day in /data/price_history.json and reports what it holds
# on the *Price history* status row. It changes NO energy decision: the cheap
# window the maintenance cycle reads still comes from `static_profile`, and
# §10.1's arbitrage is gated on `arbitrage_enabled` alone. Until TARIFF-10
# the fetch was gated on `arbitrage_enabled` too, so this option did nothing
# whatsoever on a shipped install - the store was never even created.
#
# ⚠️ THESE DEFAULTS ARE MIRRORED ONCE, IN app/price.py's
# PRICE_OPTION_DEFAULTS, and test_price.py reads THIS FILE and asserts the two
@@ -276,12 +284,36 @@ options:
#
# markup/taxes are the §14 `tariffs.energy_price.*` figures and are added by
# the FRAMEWORK, only for the components the chosen plugin declares its feed
# does not already contain. engie_dynamic, totalenergies_dynamic, eneco_eddy
# and manual_csv publish a figure that already carries the markup, so setting
# it under those is ignored - and logged as ignored, never applied twice.
# does not already contain. manual_csv carries an all-in figure, so setting
# either under it is ignored - and logged as ignored, never applied twice.
# `custom_formula` (TARIFF-15) carries energy AND markup - your contract's
# formula IS the markup - so it takes price_taxes_eur_kwh only, and your
# price_markup_eur_kwh is ignored under it, with a line in the log saying so.
price_provider: "off"
price_markup_eur_kwh: 0.03
price_taxes_eur_kwh: 0.06
# TARIFF-18. The all-in Flemish non-energy figure, EUR/kWh, incl. 6 % VAT.
# Its components are named in app/price.py's FLEMISH_TAXES_COMPONENTS_EUR_KWH
# and test_price.py section 1b re-adds them:
#
# Fluvius distribution (Antwerpen) 5,35 c EUR/kWh
# Bijzondere accijns, "Alle verbruik" 4,876
# Bijdrage groene stroom en WKK Vlaanderen 1,51
# Energiebijdrage (0 EUR/MWh since 2026-08-01) 0
# ------
# 11,736 c EUR/kWh = 0.11736
#
# ⚠️ THIS IS PER-DSO AND YOURS IS PROBABLY DIFFERENT. Only the distribution
# row moves: across the eight Fluvius DSOs it runs 5,28 c EUR/kWh
# (Midden-Vlaanderen) to 6,70 (West), so the Flemish total runs about 0.117
# to 0.131. Antwerpen is near the LOW end, which is the direction this
# default should err in - it is inert on which hour is cheapest, but §17's
# negative-price guard reads the ABSOLUTE all-in price, and a figure set too
# high would carry a genuinely negative hour above zero.
#
# ⚠️ WALLONIA AND BRUSSELS DO NOT BELONG IN THIS SCALAR AT ALL. Their
# distribution tariff has a day and a night band; use price_taxes_profile.
price_taxes_eur_kwh: 0.11736
# §9.3: injection is priced SEPARATELY, and in Belgium it is often near zero -
# which by itself strongly favours self-consumption over export. A scalar, not
@@ -317,7 +349,100 @@ options:
# to fall back to and a dead feed leaves the site with no tariff at all.
# validate.py warns about exactly that.
static_profile: ""
# TARIFF-17. THE TAXES COMPONENT AS A DAY/NIGHT TABLE, for the sites whose
# network tariff actually has one. "" means the flat price_taxes_eur_kwh
# above decides, which is what every install has had until now and is CORRECT
# IN FLANDERS: Fluvius distribution is a single rate, and its row on the
# tariff card offers only `Normaal | Uitsl. nacht` - exclusive night being a
# separate physical circuit, not the night register of a dual-rate meter.
#
# The Walloon and Brussels DSOs are on the same card with FOUR columns -
# `Enkelvoudig | Dag | Nacht | Uitsl. nacht` - and the gap is real money
# (c EUR/kWh): ORES dag 13,27 / nacht 7,39; AIESH 15,17 / 8,21; Tecteo RESA
# 12,20 / 7,01; Sibelga 9,39 / 7,10; ORES IMPACT 2026 16,578 / 5,090.
#
# ⚠️ WHY THIS IS NOT COSMETIC. The optimiser reads the SPREAD (max - min) and
# which hour is cheapest. A flat number is additive and identical every hour,
# so it cancels out of both - it cannot distort anything, and it also cannot
# express this. These bands are ANTI-CORRELATED with what the battery wants
# to do (cheap network hours are cheap spot hours), so flattening them
# UNDERSTATES the true spread and declines cycles that would have paid. See
# DOCS.md, "The three buckets".
#
# Same JSON grammar as static_profile above - the same tiling rule, the same
# day selectors, the same DST handling, because it is literally the same
# parser. ⚠️ THE VALUE IS THE TAXES COMPONENT ALONE, NOT AN ALL-IN PRICE:
#
# {"periods": [
# {"name":"dag", "from":"07:00","to":"22:00","days":"mon-fri","eur_per_kwh":0.1327},
# {"name":"nacht","from":"22:00","to":"07:00","days":"mon-fri","eur_per_kwh":0.0739},
# {"name":"nacht","from":"00:00","to":"24:00","days":"sat,sun", "eur_per_kwh":0.0739}]}
#
# ⚠️ SET THIS *OR* price_taxes_eur_kwh, NOT BOTH. Two writers of one quantity,
# one of which silently wins, is refused at startup: leave the scalar at its
# shipped default when you fill this in. A scalar still sitting on the
# SUPERSEDED 0.06 is accepted too (TARIFF-18 moved that default, and your
# stored options carry the old number whether you typed it or not) - the
# refusal is about a figure you chose, not about one this add-on wrote.
price_taxes_profile: ""
# TARIFF-15. YOUR OWN TARIFF CARD, as two fixed coefficients. Used only when
# `price_provider: custom_formula`; "" is unset.
#
# A Belgian dynamic contract prices energy as `B x index + A`, and the two
# coefficients exist in exactly ONE place - the tariff card in your
# klantenzone. There is no machine-readable source for them anywhere (CREG's
# tariff-card pages 404, CREG Scan is a JS wizard, VREG was unreachable), so
# no plugin and no scraper can ever know YOUR numbers. You are the only party
# who can see them.
#
# A JSON object, on one line. You type what your card prints, IN THE CARD'S
# OWN UNITS; every unit conversion and the VAT multiplication are the
# add-on's job, because those are exactly the two steps that silently cost a
# factor of 10, 100 or 1.06.
#
# {"index":"epex_be","resolution":"hourly","factor":0.102,"offset":1.00,
# "units":"c_eur_per_kwh","vat_pct":6,
# "card":{"month":"2026-09","index_eur_per_mwh":133.9905,
# "expect_incl_vat":15.55}}
#
# price = (factor x index_eur_per_mwh + offset) x (1 + vat_pct/100), in
# `units`, converted to EUR/kWh by the add-on.
#
# ⚠️ THE `card` BLOCK IS REQUIRED AND IT IS THE SAFETY MECHANISM. Your card
# prints the formula, the reference index it assumed AND the resulting price.
# At startup the add-on recomputes that printed price from the numbers you
# typed and REFUSES TO START unless the two agree to within 0.01 - naming the
# discrepancy. That catches VAT left out or applied twice, a transposed
# digit, a unit MISMATCH between the coefficients and the printed price
# (10x/100x/1000x), and the expensive one: NETWORK CHARGES FOLDED INTO
# `offset`. A unit error applied to BOTH sides at once cancels out of that
# comparison; the add-on refuses that separately, by checking the card's own
# point against the band a Belgian retail tariff can occupy. Your card's formula line is the ENERGY component
# alone; the add-on adds price_taxes_eur_kwh on top, so folding ~12 c/kWh of
# network charge into `offset` would bill you for it twice - and it cannot
# pass, because it moves the computed 15.55 to about 27.
#
# ⚠️ AND THE LIMIT OF THAT CHECK, IN THE SAME BREATH: it grades your
# COEFFICIENTS AGAINST YOUR CARD, not your card against reality. Two numbers
# copied off the wrong product agree with each other perfectly and pass.
#
# ⚠️ FIXED COEFFICIENTS ONLY - the add-on evaluates NO expression of any kind
# and there is no field that takes one. An unknown key is refused rather than
# ignored. See DOCS.md for why (it is not only security: two coefficients can
# be graded against your card and an arbitrary expression cannot be graded
# against anything).
price_formula: ""
price_csv_path: "/share/goodwe_prices.csv"
# ⚠️ AN OVERRIDE IS THE EXACT URL (TARIFF-09). The plugin sends it verbatim
# and appends nothing - including the `start`/`end` date range it derives from
# the horizon for its own feed - so an override gets whatever range that URL
# answers, which for a bare day-ahead endpoint is ONE Brussels day. Every hour
# past it is then carried from yesterday and flagged provisional, and §8.3
# keeps provisional hours out of the trade set. The plugin logs a WARNING
# naming that consequence on every fetch that uses an override; clearing this
# option is what restores the full horizon.
price_feed_url: ""
price_horizon_h: 48
price_timeout_s: 10
@@ -344,6 +469,12 @@ options:
# configuration: this is the first strategy in the add-on that moves energy
# against the meter on purpose, so it is armed deliberately or not at all.
#
# ⚠️ IT IS NOT WHAT TURNS THE PRICE FEED ON (TARIFF-10). `price_provider`
# above decides whether prices are FETCHED; this decides whether they move
# energy. Leaving this off on a site with a provider configured is a
# supported, documented state: prices are fetched, validated and recorded,
# and nothing trades.
#
# ⚠️ IT NEEDS A PRICE SOURCE AND A P1 METER, and validate.py refuses the
# combinations that cannot work rather than letting them look armed. With
# `price_provider: off` and no `static_profile` there is no series to trade
@@ -736,9 +867,32 @@ schema:
# Supervisor rejects a typo in the UI. validate.py ALSO refuses an unknown id
# against app/price.py's PROVIDERS registry, because sim/inject.py and the
# checks reach validate_startup_config() with a dict the Supervisor never saw.
# Adding a sixth plugin means adding it here and to PROVIDERS - and nowhere
# else, which is AC2.
price_provider: list(off|generic_dayahead|engie_dynamic|totalenergies_dynamic|eneco_eddy|manual_csv)
# Adding a plugin means adding it here and to PROVIDERS - and nowhere else,
# which is AC2.
#
# ⚠️ THREE OF THESE FIVE IDS NO LONGER RESOLVE TO A PLUGIN, AND LEAVING THEM
# HERE IS A DECISION (TARIFF-07). engie_dynamic, totalenergies_dynamic and
# eneco_eddy were withdrawn - their endpoints do not exist and never did - and
# the obvious edit is to delete them from this line too. DO NOT. The Supervisor
# validates the options file against this schema BEFORE the add-on starts, so
# an id that is not listed here makes an EXISTING config schema-invalid: the
# operator gets an opaque "expected one of ..." from the Supervisor, the add-on
# never runs, and validate.py's retired-id refusal - the message that tells
# them their prices never worked and what to do instead - IS NEVER REACHED, on
# exactly the sites that need it. Schema tightness would cost the only thing
# this withdrawal is FOR.
#
# The cost of keeping them is that the dropdown still offers three dead ids to
# a NEW operator. That is the cheaper failure: choosing one is refused at the
# next startup by name, with the endpoint, the observation and the alternative
# - which is strictly more than a typo-shaped schema rejection would say. Both
# translations label them withdrawn in the option description as well.
#
# app/price.py's RETIRED_PROVIDERS is the list, test_price.py section 1 asserts
# this line offers exactly PROVIDERS + RETIRED_PROVIDERS + 'off' - derived, not
# typed - and register() refuses to re-register a retired id, so the two
# cannot both claim one.
price_provider: list(off|generic_dayahead|custom_formula|engie_dynamic|totalenergies_dynamic|eneco_eddy|manual_csv)
price_markup_eur_kwh: float(0,1)
price_taxes_eur_kwh: float(0,1)
@@ -752,6 +906,15 @@ schema:
# parse_static_profile() accepts an already-parsed dict too, so a future
# nested schema is a config.yaml change and no code change at all.
static_profile: str?
# TARIFF-17. A JSON string for `static_profile`'s reason, and parsed by
# `static_tariff.parse_taxes_profile()` - the SAME period grammar, so a
# second tiling rule and a second set of DST bugs do not exist to diverge.
price_taxes_profile: str?
# TARIFF-15. A JSON string for the same reason `static_profile` is one: it is
# ONE option key, it survives config.yaml's flat options block, and
# parse_price_formula() accepts an already-parsed dict too, so a future
# nested schema is a config.yaml change and no code change at all.
price_formula: str?
price_csv_path: str?
price_feed_url: str?
price_horizon_h: int(1,72)
@@ -0,0 +1 @@
{"license_info":"CC BY 4.0 (creativecommons.org/licenses/by/4.0) from Bundesnetzagentur | SMARD.de","unix_seconds":[1757455200,1757458800,1757462400,1757466000,1757469600,1757473200,1757476800,1757480400,1757484000,1757487600,1757491200,1757494800,1757498400,1757502000,1757505600,1757509200,1757512800,1757516400,1757520000,1757523600,1757527200,1757530800,1757534400,1757538000],"price":[73.0,61.31,64.32,62.49,67.0,64.48,94.52,149.26,132.43,91.61,84.8,79.61,69.23,66.75,68.91,69.31,80.29,66.91,105.2,161.05,116.73,84.2,64.1,42.41],"unit":"EUR / MWh","deprecated":false}
@@ -0,0 +1 @@
{"license_info":"CC BY 4.0 (creativecommons.org/licenses/by/4.0) from Bundesnetzagentur | SMARD.de","unix_seconds":[1788904800,1788905700,1788906600,1788907500,1788908400,1788909300,1788910200,1788911100,1788912000,1788912900,1788913800,1788914700,1788915600,1788916500,1788917400,1788918300,1788919200,1788920100,1788921000,1788921900,1788922800,1788923700,1788924600,1788925500,1788926400,1788927300,1788928200,1788929100,1788930000,1788930900,1788931800,1788932700,1788933600,1788934500,1788935400,1788936300,1788937200,1788938100,1788939000,1788939900,1788940800,1788941700,1788942600,1788943500,1788944400,1788945300,1788946200,1788947100,1788948000,1788948900,1788949800,1788950700,1788951600,1788952500,1788953400,1788954300,1788955200,1788956100,1788957000,1788957900,1788958800,1788959700,1788960600,1788961500,1788962400,1788963300,1788964200,1788965100,1788966000,1788966900,1788967800,1788968700,1788969600,1788970500,1788971400,1788972300,1788973200,1788974100,1788975000,1788975900,1788976800,1788977700,1788978600,1788979500,1788980400,1788981300,1788982200,1788983100,1788984000,1788984900,1788985800,1788986700,1788987600,1788988500,1788989400,1788990300,1788991200,1788992100,1788993000,1788993900,1788994800,1788995700,1788996600,1788997500,1788998400,1788999300,1789000200,1789001100,1789002000,1789002900,1789003800,1789004700,1789005600,1789006500,1789007400,1789008300,1789009200,1789010100,1789011000,1789011900,1789012800,1789013700,1789014600,1789015500,1789016400,1789017300,1789018200,1789019100,1789020000,1789020900,1789021800,1789022700,1789023600,1789024500,1789025400,1789026300,1789027200,1789028100,1789029000,1789029900,1789030800,1789031700,1789032600,1789033500,1789034400,1789035300,1789036200,1789037100,1789038000,1789038900,1789039800,1789040700,1789041600,1789042500,1789043400,1789044300,1789045200,1789046100,1789047000,1789047900,1789048800,1789049700,1789050600,1789051500,1789052400,1789053300,1789054200,1789055100,1789056000,1789056900,1789057800,1789058700,1789059600,1789060500,1789061400,1789062300,1789063200,1789064100,1789065000,1789065900,1789066800,1789067700,1789068600,1789069500,1789070400,1789071300,1789072200,1789073100,1789074000,1789074900,1789075800,1789076700],"price":[150.93,151.41,150.99,147.59,153.25,147.26,144.04,141.53,147.79,146.53,143.55,139.93,145.58,139.56,139.11,138.06,138.92,138.52,138.04,138.65,137.29,139.03,144.59,150.34,154.12,174.5,178.43,186.62,199.35,200.21,200.11,197.43,220.07,202.0,196.01,174.03,198.32,167.49,152.54,142.05,161.5,145.45,128.61,100.85,139.18,115.83,94.41,80.39,111.25,89.61,84.99,74.38,80.53,70.58,62.27,57.1,62.24,64.95,68.99,71.19,66.9,87.32,109.0,123.12,68.48,125.53,155.99,161.9,138.77,160.57,167.35,173.0,142.2,172.83,196.19,211.26,205.85,218.77,217.08,222.0,210.98,208.46,208.88,210.02,209.77,202.61,200.27,191.49,194.39,195.04,197.26,190.92,188.39,198.05,203.05,191.84,216.08,216.08,214.92,200.0,223.37,201.43,196.32,190.0,199.53,194.83,180.53,172.23,179.88,175.77,167.1,173.89,163.29,165.99,171.41,178.73,170.27,177.37,187.71,201.07,193.62,207.61,218.27,226.77,227.17,234.05,236.99,229.94,248.93,227.5,216.08,195.67,215.99,195.01,177.1,155.09,174.91,152.71,139.59,123.83,140.77,123.19,114.96,94.5,102.77,96.63,94.57,116.34,114.63,126.82,129.89,123.95,105.0,119.11,121.0,136.79,123.41,135.01,135.0,156.68,146.88,152.16,162.02,181.78,154.76,184.28,194.3,215.76,197.23,234.15,253.03,296.58,345.32,389.16,482.99,559.0,551.11,446.41,358.97,295.9,271.39,268.06,241.43,238.01,239.7,226.41,223.05,211.56,218.21,210.61,197.78,171.79],"unit":"EUR / MWh","deprecated":false}
@@ -0,0 +1 @@
{"license_info":"CC BY 4.0 (creativecommons.org/licenses/by/4.0) from Bundesnetzagentur | SMARD.de","unix_seconds":[1788991200,1788992100,1788993000,1788993900,1788994800,1788995700,1788996600,1788997500,1788998400,1788999300,1789000200,1789001100,1789002000,1789002900,1789003800,1789004700,1789005600,1789006500,1789007400,1789008300,1789009200,1789010100,1789011000,1789011900,1789012800,1789013700,1789014600,1789015500,1789016400,1789017300,1789018200,1789019100,1789020000,1789020900,1789021800,1789022700,1789023600,1789024500,1789025400,1789026300,1789027200,1789028100,1789029000,1789029900,1789030800,1789031700,1789032600,1789033500,1789034400,1789035300,1789036200,1789037100,1789038000,1789038900,1789039800,1789040700,1789041600,1789042500,1789043400,1789044300,1789045200,1789046100,1789047000,1789047900,1789048800,1789049700,1789050600,1789051500,1789052400,1789053300,1789054200,1789055100,1789056000,1789056900,1789057800,1789058700,1789059600,1789060500,1789061400,1789062300,1789063200,1789064100,1789065000,1789065900,1789066800,1789067700,1789068600,1789069500,1789070400,1789071300,1789072200,1789073100,1789074000,1789074900,1789075800,1789076700],"price":[216.08,216.08,214.92,200.0,223.37,201.43,196.32,190.0,199.53,194.83,180.53,172.23,179.88,175.77,167.1,173.89,163.29,165.99,171.41,178.73,170.27,177.37,187.71,201.07,193.62,207.61,218.27,226.77,227.17,234.05,236.99,229.94,248.93,227.5,216.08,195.67,215.99,195.01,177.1,155.09,174.91,152.71,139.59,123.83,140.77,123.19,114.96,94.5,102.77,96.63,94.57,116.34,114.63,126.82,129.89,123.95,105.0,119.11,121.0,136.79,123.41,135.01,135.0,156.68,146.88,152.16,162.02,181.78,154.76,184.28,194.3,215.76,197.23,234.15,253.03,296.58,345.32,389.16,482.99,559.0,551.11,446.41,358.97,295.9,271.39,268.06,241.43,238.01,239.7,226.41,223.05,211.56,218.21,210.61,197.78,171.79],"unit":"EUR / MWh","deprecated":false}
@@ -0,0 +1,82 @@
{
"card": "TARIFF-08, extended by TARIFF-09",
"what": "UNMODIFIED response bodies from the public EPEX BE day-ahead feed, captured so the SHIPPED parser can be replayed against what the source really publishes",
"why": "every price fixture in the suite was built from a `{\"data\": [{\"start\": ..., \"price_eur_mwh\": ...}]}` document that this feed has never published. The suite therefore graded `_points_from_json` against its own assumption - a closed loop that could not notice the assumption was wrong - and §11.4 turned the resulting failure into a silent fall back to the static profile. A captured real body is what breaks that loop, so these files are bytes, not a reconstruction: nothing here is pretty-printed, re-keyed or trimmed. TARIFF-09 adds a body from a request that names a DATE RANGE, for the same reason one layer out: the shipped plugin took `since`/`until` and asked for the bare default URL, so the feed answered ONE Brussels day and every hour past today was carried forward and dropped from the trade set as provisional. A two-day body captured from the range request is what makes \"asking for the range gets the range\" checkable against something the plugin did not shape.",
"source": "Energy-Charts API (Fraunhofer ISE), api.energy-charts.info",
"upstream": "Bundesnetzagentur | SMARD.de",
"licence": "CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)",
"licence_note": "the BE bidding zone is in the API's CC BY 4.0 list, so redistribution is permitted with attribution. DATA-02 established this against the API's own OpenAPI document before committing `epex-be-dayahead-2025-09-01_2026-09-01.json`; both bodies below carry the same `license_info` string, recorded per file and asserted by the replay test.",
"attribution": "Data licensed CC BY 4.0 by Bundesnetzagentur | SMARD.de, retrieved via the Energy-Charts API (Fraunhofer ISE), https://energy-charts.info. Published without changes.",
"retrieved_utc": "2026-09-10T06:35:00+00:00 (TARIFF-08's two), 2026-09-10T07:45:49+00:00 (TARIFF-09's)",
"retrieved_with": "curl, by hand; no code in `app/` fetches these",
"shape": "COLUMNAR: parallel `unix_seconds` (UTC epoch seconds) and `price` arrays, plus `unit`, `license_info` and `deprecated`. There is no `data` key and no per-point object.",
"replayed_by": "test_price.py sections 2b (TARIFF-08) and 2c (TARIFF-09)",
"files": {
"energy-charts-be-price-2026-09-10-quarterhourly.json": {
"url": "https://api.energy-charts.info/price?bzn=BE",
"note": "the SHIPPED default URL of `generic_dayahead`, verbatim and with no parameters, so the replay drives the bytes the add-on's own request produces",
"covers_local_day": "2026-09-10 (Europe/Brussels)",
"granularity_s": 900,
"intervals": 96,
"bytes": 1877,
"sha256": "176309d0020ad208320797385ef68b6a98ac248218539416cddf6579dd060487",
"unit_as_returned": "EUR / MWh",
"licence_as_returned": "CC BY 4.0 (creativecommons.org/licenses/by/4.0) from Bundesnetzagentur | SMARD.de",
"deprecated_as_returned": false,
"top_level_keys": [
"deprecated",
"license_info",
"price",
"unit",
"unix_seconds"
],
"also_returned_verbatim_by": {
"url": "https://api.energy-charts.info/price?bzn=BE&start=2026-09-10&end=2026-09-11",
"retrieved_utc": "2026-09-10T07:46:55+00:00",
"note": "TARIFF-09's PRE-PUBLICATION case, and it needed no second file because the answer was BYTE-IDENTICAL to the bare-URL capture above - the same 1877 bytes and the same sha256. Asked for two days at 09:46 CEST, before the ~13:00-14:00 CET auction publishes 2026-09-11, the feed returned the ONE day that exists. That is the short answer this card requires to stay a `series` of what exists rather than becoming `no_data`, and the replay re-derives the identity from the bytes rather than restating it."
}
},
"energy-charts-be-price-2025-09-10-hourly.json": {
"url": "https://api.energy-charts.info/price?bzn=BE&start=2025-09-10&end=2025-09-10",
"note": "a day BEFORE Belgium's 15-minute MTU (2025-10-01), so the same endpoint answers HOURLY. Kept because the parser must not assume either spacing, and a fixture at only one granularity cannot grade that",
"covers_local_day": "2025-09-10 (Europe/Brussels)",
"granularity_s": 3600,
"intervals": 24,
"bytes": 572,
"sha256": "03fe41690810df224611f4016899f856457f67a3c12c7654da17b1331ff2eff4",
"unit_as_returned": "EUR / MWh",
"licence_as_returned": "CC BY 4.0 (creativecommons.org/licenses/by/4.0) from Bundesnetzagentur | SMARD.de",
"deprecated_as_returned": false,
"top_level_keys": [
"deprecated",
"license_info",
"price",
"unit",
"unix_seconds"
]
},
"energy-charts-be-price-2026-09-09_2026-09-10-twoday.json": {
"url": "https://api.energy-charts.info/price?bzn=BE&start=2026-09-09&end=2026-09-10",
"retrieved_utc": "2026-09-10T07:45:49+00:00",
"note": "TARIFF-09. A RANGE request, and the counterfactual the ticket is built on: `start`/`end` are INCLUSIVE Brussels calendar dates and this one answers 192 quarter-hours across TWO local days, where the bare `?bzn=BE` of the same endpoint answers 96 across one. The shipped `generic_dayahead` now derives these two dates from the `since`/`until` it is handed.",
"covers_local_days": [
"2026-09-09",
"2026-09-10"
],
"granularity_s": 900,
"intervals": 192,
"bytes": 3579,
"sha256": "4d83f5efb4294a4a682562eac619bf35e45c95edde957bfa94c402e1777cc9b3",
"unit_as_returned": "EUR / MWh",
"licence_as_returned": "CC BY 4.0 (creativecommons.org/licenses/by/4.0) from Bundesnetzagentur | SMARD.de",
"deprecated_as_returned": false,
"top_level_keys": [
"deprecated",
"license_info",
"price",
"unit",
"unix_seconds"
]
}
}
}
+744 -1
View File
@@ -92,7 +92,9 @@ from app.price import (
PRICE_OPTION_DEFAULTS, Absence, PricePoint, PriceProduct, PriceRead,
Uncertainty, WIDEN_FACTOR, brussels, required_spread, widen_factor,
)
from app.static_tariff import TariffPlan, parse_static_profile, static_plan
from app.static_tariff import (
TariffPlan, parse_static_profile, resolve_tariff, static_plan,
)
from app.validate import validate_startup_config
UTC = timezone.utc
@@ -525,6 +527,124 @@ check(f"...while THE SAME MAGNITUDES with the sign flipped all DISCHARGE, so "
# header. Nothing in app/ changed - these rungs were correct all along.
# --------------------------------------------------------------------------- #
print()
print("2c. TARIFF-18 - where the \u00a717 boundary sits in SPOT terms, and the "
"shipped default that decides it")
# ⚠️ WHAT THIS GRADES AND WHAT IT DOES NOT CLAIM. `price_taxes_eur_kwh` is a
# LEVEL component: it is added identically to every hour, so it cancels exactly
# in `max - min` and cannot move which hour is cheapest. test_price.py 16b
# measures that as a FAMILY - five flat scalars spanning and exceeding a real
# per-DSO range, identical argmin and identical spread to the last digit - and
# nothing here re-litigates it. Raising the shipped default 0.06 -> 0.11736 is
# therefore NOT an arbitrage-correctness fix, and this section does not say it
# is.
#
# What it IS: §17's guard is the one rung in this module that reads an
# ABSOLUTE price rather than a difference, so it is the one decision a level
# component can reach. The boundary is `spot < -(markup + taxes)`, and moving
# the taxes default moves it. Below, driven through the shipped `decide()`.
#
# ⚠️ AND THE TWO ERROR DIRECTIONS ARE NOT SYMMETRIC, WHICH IS WHY THE DEFAULT
# SITS AT THE BOTTOM OF THE PER-DSO BAND RATHER THAN IN THE MIDDLE. A taxes
# figure set too HIGH carries a genuinely negative hour up above zero and the
# add-on exports into paid-for disposal - §17's own regression case. A figure
# set too LOW only declines a discharge that would have paid. Where one default
# must stand in for eight Fluvius DSOs, it belongs at the low end.
_T18_MARKUP = PRICE_OPTION_DEFAULTS["price_markup_eur_kwh"]
_T18_TAXES = PRICE_OPTION_DEFAULTS["price_taxes_eur_kwh"]
_T18_OLD_TAXES = 0.06 # what config.yaml shipped before TARIFF-18
# ⚠️ THE PRECONDITION FIRST (MUTATION.md habit 1). Every comparison below is
# "the old level and the new level answer differently". If the two levels were
# ever the same number, all of it would be one figure graded against itself -
# DEV-02's `(-5000, 5000)` against `(-5000, 5000)`, one card over.
check(f"the shipped taxes default {_T18_TAXES} and the superseded "
f"{_T18_OLD_TAXES} are DIFFERENT numbers, and differ by more than the "
f"0.01 EUR/kWh step this fixture's spot series is built on",
abs(_T18_TAXES - _T18_OLD_TAXES) > 0.01)
# The §17 boundary, computed HERE from the two option values rather than read
# off arbitrage.py - habit 2. all-in = spot + markup + taxes, so the guard
# fires exactly when spot < -(markup + taxes).
_T18_B_NEW = round(-(_T18_MARKUP + _T18_TAXES), 6)
_T18_B_OLD = round(-(_T18_MARKUP + _T18_OLD_TAXES), 6)
_T18_PROBE = round((_T18_B_NEW + _T18_B_OLD) / 2, 6)
check(f"the guard's spot-price boundary moves {_T18_B_OLD:+.5f} -> "
f"{_T18_B_NEW:+.5f} EUR/kWh, and the probe {_T18_PROBE:+.5f} lies "
f"STRICTLY BETWEEN them - so one level answers each way and the probe is "
f"not on either boundary",
_T18_B_NEW < _T18_PROBE < _T18_B_OLD)
# A whole-day-negative spot series whose DEAREST hour is the probe. Same shape
# as section 2's: the case a guard written "not all prices are negative" lets
# through, in SPOT rather than all-in terms.
_T18_SPOT = [round(-0.40 + 0.010 * i, 6) for i in range(24)]
_T18_SPOT[DEAR_H] = _T18_PROBE
check(f"the spot day's dearest hour is slot {DEAR_H} at {_T18_PROBE:+.5f} and "
f"every other hour is below it, so the hour the guard is asked about is "
f"the one the probe was chosen for",
_T18_SPOT.index(max(_T18_SPOT)) == DEAR_H
and _T18_SPOT.count(max(_T18_SPOT)) == 1
and all(p < 0 for p in _T18_SPOT))
def _t18_all_in(taxes):
"""spot + markup + taxes, added here, by this file's own arithmetic."""
return [round(p + _T18_MARKUP + taxes, 6) for p in _T18_SPOT]
_T18_OLD_SERIES = _t18_all_in(_T18_OLD_TAXES)
_T18_NEW_SERIES = _t18_all_in(_T18_TAXES)
print(f" dearest hour all-in: old level {_T18_OLD_SERIES[DEAR_H]:+.5f}, "
f"shipped level {_T18_NEW_SERIES[DEAR_H]:+.5f} EUR/kWh")
# ⚠️ THE LEVEL BUCKET RESTATED ON THIS FIXTURE, BEFORE ANYTHING IS CONCLUDED
# FROM IT. If the two arms differed in spread or in argmin, the verdicts below
# could be attributed to that instead, and this section would be claiming an
# arbitrage effect it must not claim.
check(f"the two levels leave the SPREAD identical to the last digit "
f"({round(max(_T18_OLD_SERIES) - min(_T18_OLD_SERIES), 6)} EUR/kWh) and "
f"the cheapest hour in the same slot - so nothing below is an optimiser "
f"effect",
round(max(_T18_OLD_SERIES) - min(_T18_OLD_SERIES), 6)
== round(max(_T18_NEW_SERIES) - min(_T18_NEW_SERIES), 6)
and _T18_OLD_SERIES.index(min(_T18_OLD_SERIES))
== _T18_NEW_SERIES.index(min(_T18_NEW_SERIES)))
_T18_SELL_OLD = act(plan_of(_T18_OLD_SERIES), DEAR_H, thresh=_shipped,
unc=NO_WEATHER)
_T18_SELL_NEW = act(plan_of(_T18_NEW_SERIES), DEAR_H, thresh=_shipped,
unc=NO_WEATHER)
check(f"at the SUPERSEDED 0.06 the same spot hour is "
f"{_T18_OLD_SERIES[DEAR_H]:+.5f} all-in, so \u00a717 REFUSES the discharge "
f"- and the reason names \u00a717 and that price, not merely that nothing "
f"happened",
_T18_SELL_OLD.action is None
and "§17" in _T18_SELL_OLD.reason
and f"{_T18_OLD_SERIES[DEAR_H]:+.4f}" in _T18_SELL_OLD.reason)
check(f"...and at the SHIPPED {_T18_TAXES} the SAME spot hour is "
f"{_T18_NEW_SERIES[DEAR_H]:+.5f} all-in and the battery DISCHARGES. That "
f"is the whole of what this card changes in a decision: the operator "
f"really does pay those levies, so this hour was never paid-for disposal "
f"and the old default was declining a trade that paid",
_T18_SELL_NEW.action == DISCHARGE)
# ⚠️ AND THE OTHER SIDE OF THE SHIPPED BOUNDARY, or the pair above grades
# "bigger taxes discharge more" rather than "the boundary is at -(markup +
# taxes)". One step below the shipped boundary is still refused.
_T18_BELOW = list(_T18_SPOT)
_T18_BELOW[DEAR_H] = round(_T18_B_NEW - 0.001, 6)
_T18_SELL_BELOW = act(
plan_of([round(p + _T18_MARKUP + _T18_TAXES, 6) for p in _T18_BELOW]),
DEAR_H, thresh=_shipped, unc=NO_WEATHER)
check(f"a spot hour at {_T18_BELOW[DEAR_H]:+.5f} - one tenth of a cent BELOW "
f"the shipped boundary - is still refused by \u00a717, so the boundary is "
f"where -(markup + taxes) puts it and not merely somewhere lower than "
f"before",
_T18_SELL_BELOW.action is None and "§17" in _T18_SELL_BELOW.reason)
# --------------------------------------------------------------------------- #
print()
print("3. AC3 - peak shaving outranks arbitrage, through both mechanisms")
@@ -1284,6 +1404,323 @@ check("main.py reads every arbitrage option through app/arbitrage.py or "
"arbitrage_soc_ceiling")))
# --------------------------------------------------------------------------- #
print()
print("8b. TARIFF-12 - a RETURNED degradation must not clear a fresh plan")
# ⚠️ THE STIMULUS IS A RETURN, NOT A RAISE, AND THAT IS THE WHOLE CHECK.
# `run_tariff_refresh()` guarded the plan with `except Exception` alone, and
# `load_prices()` states in its own docstring that it "RETURNS, NEVER RAISES,
# for every failure a site can actually have" - so the one case the guard was
# written for, a supplier outage, never reached it. Everything below drives the
# SHIPPED chain and stubs nothing but the socket: the transport raises the way
# aiohttp does, `load_prices()` turns that into a `no_data` read, `_degrade()`
# turns THAT into a §11.4 plan, and `run_tariff_refresh()` decides what to do
# with it. A check that patched `resolve_tariff` to raise would have passed
# against the defective code.
import asyncio # noqa: E402
from app import price as _price # noqa: E402
from app.price_history import NoHistory # noqa: E402
class ClientConnectorError(Exception):
"""aiohttp's own class name, so the §11.4 reason this drives is the
sentence the 40 h soak recorded at refresh 56, verbatim."""
_T12_FEED_UP = {"up": True}
async def _t12_transport(url, timeout_s):
"""The socket, and nothing else, is what this replaces."""
if not _T12_FEED_UP["up"]:
raise ClientConnectorError(
"Cannot connect to host api.energy-charts.info:443 ssl:default")
t0 = datetime.now(UTC).replace(minute=0, second=0, microsecond=0)
return json.dumps({
"unix_seconds": [int((t0 + timedelta(hours=i)).timestamp())
for i in range(48)],
"price": [80.0 + 40.0 * (i % 6) for i in range(48)],
"unit": "EUR / MWh"}).encode()
class _T12Controller:
"""Just enough Controller to run the SHIPPED refresh coroutine.
Both methods are bound off the shipped class rather than copied, for
`fresh_tariff_plan()`'s own stated reason: a transcription drifts.
"""
run_tariff_refresh = _main.Controller.run_tariff_refresh
fresh_tariff_plan = _main.Controller.fresh_tariff_plan
def __init__(self, opts):
self.o = opts
self.tariff_plan = None
self.tariff_at = None
self.price_history = NoHistory()
self.stop_event = asyncio.Event()
self.stop_event.set() # so the inter-refresh wait returns
self._ticks = 0
@property
def stopping(self):
self._ticks -= 1
return self._ticks < 0
def refresh_once(self):
self._ticks = 1
asyncio.run(self.run_tariff_refresh())
return self
def _t12_opts(profile=""):
# `price_timeout_s` is read with a plain `.get()` by every plugin, so it
# has to be present - an options dict that relies on price_option()'s
# defaults for it fetches with `float(None)` and this section would grade
# a TypeError instead of an outage.
return {"price_provider": "generic_dayahead", "price_timeout_s": 10,
"price_horizon_h": 48, "static_profile": profile}
_T12_HORIZON_H = 48
_T12_STALE_S = _main.TARIFF_MAX_AGE_S
_t12_saved_transport = _price.https_get
_price.https_get = _t12_transport
# -- the accepted baseline (habit 7): one good fetch, varied one thing at a
# -- time from here.
_T12_FEED_UP["up"] = True
_t12_a = derived(lambda: _T12Controller(_t12_opts()).refresh_once())
_t12_good = derived(lambda: _t12_a.tariff_plan)
check(f"BASELINE: a fetch that succeeds stores an OBSERVATION - "
f"{getattr(_t12_good, 'words', _t12_good)!r}",
getattr(_t12_good, "origin", None) == "provider"
and _t12_good.from_feed is True
and len(_t12_good.read.points) == _T12_HORIZON_H)
# -- the stimulus, graded before it is used on anything (habit 6/7) ----------
_T12_FEED_UP["up"] = False
_t12_now = datetime.now(UTC)
_t12_deg = derived(lambda: asyncio.run(resolve_tariff(
_t12_opts(), _t12_now, _t12_now + timedelta(hours=_T12_HORIZON_H),
now=_t12_now, history=NoHistory())))
check("STIMULUS: a dead feed makes resolve_tariff() RETURN a degraded plan - "
"it does not raise, so `except Exception` is not what was protecting "
"the plan and the defect was never on that branch",
isinstance(_t12_deg, TariffPlan))
check(f"...and the returned plan is the §11.4 no-profile row the soak saw - "
f"{getattr(_t12_deg, 'words', _t12_deg)!r}",
isinstance(_t12_deg, TariffPlan) and _t12_deg.origin == "none"
and _t12_deg.from_feed is False and _t12_deg.has_prices is False
and "ClientConnectorError" in _t12_deg.words)
# -- AC1: the degradation must not replace a plan still inside the bound -----
# ⚠️ 1800 s IS STATED HERE, NOT DERIVED FROM THE MODULE BEING GRADED - section
# 8's own lesson. A probe at `TARIFF_MAX_AGE_S - 1` moves with a mutation of
# the constant; half an hour is inside any reading of a 5400 s bound, and it is
# the interval the soak actually ran at.
_t12_b = derived(lambda: _T12Controller(_t12_opts()))
_t12_b.tariff_plan, _t12_b.tariff_at = _t12_good, datetime.now(UTC) - \
timedelta(seconds=1800)
_t12_at_before = _t12_b.tariff_at
check(f"PRECONDITION: the standing plan is 1800 s old and TARIFF_MAX_AGE_S is "
f"{_T12_STALE_S:.0f} s, so it IS fresh when the failed refresh runs - "
f"without this the next check grades an expired plan",
1800 < _T12_STALE_S
and _t12_b.fresh_tariff_plan(datetime.now(UTC)) is _t12_good)
_t12_b.refresh_once()
check("a RETURNED degradation does not replace a plan still inside "
"TARIFF_MAX_AGE_S - the 48 h series is still there, by identity, and "
"not a 0-point `origin: none` read",
_t12_b.tariff_plan is _t12_good
and _t12_b.tariff_plan.has_prices is True
and len(_t12_b.tariff_plan.read.points) == _T12_HORIZON_H)
check("...and `tariff_at` did NOT move, so the kept plan still ages out from "
"the fetch that produced it and is distinguishable from a freshly "
"fetched one. Re-stamping it here is how a plan freezes at its last "
"value and looks current forever",
_t12_b.tariff_at == _t12_at_before)
check("...and the two timestamps a successful refresh and a kept plan produce "
"really are different values, or the check above compares a number "
"against itself",
_t12_a.tariff_at != _t12_at_before)
# -- AC3: TARIFF_MAX_AGE_S governs - the bound is where the module says ------
_t12_c = derived(lambda: _T12Controller(_t12_opts()))
_t12_c.tariff_plan, _t12_c.tariff_at = _t12_good, datetime.now(UTC) - \
timedelta(seconds=_T12_STALE_S + 5)
_t12_c.refresh_once()
check(f"a plan that has aged PAST {_T12_STALE_S:.0f} s IS replaced by the "
f"degradation - the plan is served stale for the age bound and no "
f"longer, so TARIFF_MAX_AGE_S governs something rather than being "
f"unreachable",
_t12_c.tariff_plan is not _t12_good
and _t12_c.tariff_plan.from_feed is False
and _t12_c.tariff_plan.has_prices is False)
_t12_d = derived(lambda: _T12Controller(_t12_opts()))
_t12_d.tariff_plan, _t12_d.tariff_at = _t12_good, datetime.now(UTC) - \
timedelta(seconds=_T12_STALE_S - 60)
_t12_d.refresh_once()
check("...and one minute SHORT of the bound it is still kept, so the line "
"above is a bound rather than 'the degradation always wins'",
_t12_d.tariff_plan is _t12_good)
_t12_e = derived(lambda: _T12Controller(_t12_opts()).refresh_once())
check("a degradation with NO plan standing at all is stored - §11.4 still "
"reaches an install that has never had a good fetch, and the guard is "
"not 'never overwrite'",
_t12_e.tariff_plan is not None
and _t12_e.tariff_plan.from_feed is False
and _t12_e.tariff_at is not None)
# -- the §11.4 STATIC FALLBACK, which is the judgement this card had to make -
_t12_f = derived(lambda: _T12Controller(_t12_opts(STATIC_BE)))
_t12_fb = derived(lambda: asyncio.run(resolve_tariff(
_t12_opts(STATIC_BE), _t12_now,
_t12_now + timedelta(hours=_T12_HORIZON_H), now=_t12_now,
history=NoHistory())))
check(f"PRECONDITION: with a static_profile configured the SAME dead feed "
f"degrades to a §11.4 FALLBACK carrying real prices, not to the "
f"no-profile row - {getattr(_t12_fb, 'words', _t12_fb)!r}",
isinstance(_t12_fb, TariffPlan) and _t12_fb.origin == "fallback"
and _t12_fb.has_prices is True and _t12_fb.from_feed is False)
check("...and every one of its points is provisional, so firm_points() is "
"empty, arbitrage() refuses and cheap_now() is False - it is a real "
"tariff and nothing may be acted on it. That is why it does not replace "
"a fresh observation",
isinstance(_t12_fb, TariffPlan) and _t12_fb.provisional is True
and _t12_fb.firm_points() == ()
and _t12_fb.arbitrage(peak_guard_active=False, min_spread_eur_kwh=0.0,
uncertainty=NO_WEATHER).kind == "refused")
_t12_f.tariff_plan, _t12_f.tariff_at = _t12_good, datetime.now(UTC) - \
timedelta(seconds=1800)
_t12_f_at = _t12_f.tariff_at
_t12_f.refresh_once()
check("a §11.4 static FALLBACK does not replace a fresh provider plan either, "
"although it has prices - a day-ahead price for an hour does not change "
"once published, so the 30-minute-old observation is no less true and "
"is the only one of the two that carries a firm hour",
_t12_f.tariff_plan is _t12_good and _t12_f.tariff_at == _t12_f_at)
# -- ...and the rule is NOT symmetrical: a degradation replaces a degradation.
_t12_g = derived(lambda: _T12Controller(_t12_opts(STATIC_BE)))
_t12_g.tariff_plan, _t12_g.tariff_at = _t12_fb, datetime.now(UTC) - \
timedelta(seconds=1800)
_t12_g_at = _t12_g.tariff_at
_t12_g.refresh_once()
check("a standing plan that is ITSELF a §11.4 fallback IS replaced by the "
"next one, and its timestamp moves - otherwise an outage would freeze "
"the operator's static profile at the window it was first generated "
"for, and the guard would be 'never overwrite' wearing a bound's name",
_t12_g.tariff_plan is not _t12_fb and _t12_g.tariff_plan.origin ==
"fallback" and _t12_g.tariff_at != _t12_g_at)
# -- ...and a recovered feed replaces a kept plan, which is the third term ---
_T12_FEED_UP["up"] = True
_t12_h = derived(lambda: _T12Controller(_t12_opts()))
_t12_h.tariff_plan, _t12_h.tariff_at = _t12_good, datetime.now(UTC) - \
timedelta(seconds=1800)
_t12_h_at = _t12_h.tariff_at
_t12_h.refresh_once()
check("when the feed answers again the observation replaces the kept plan and "
"`tariff_at` moves - the guard keeps a plan against a DEGRADATION, not "
"against a fetch that worked",
_t12_h.tariff_plan is not _t12_good
and _t12_h.tariff_plan.from_feed is True
and _t12_h.tariff_at != _t12_h_at)
# -- TARIFF-10 widened WHICH INSTALLS RUN THIS TASK, so the configurations it
# -- newly keeps alive are graded here rather than reasoned about.
# ⚠️ `price_provider: off` + a `static_profile` + `arbitrage_enabled: true` is a
# configuration validate.py ACCEPTS and TARIFF-10's `or` exists to protect. On
# it `resolve_tariff()` returns `static_plan(provisional=False)` - the
# CONFIGURED product, origin "static" - so `standing.from_feed` is false on
# every refresh and this card's guard must never fire. Confirmed by driving it,
# not by reading the origin table: a guard that quietly latched here would
# freeze a static site's plan at its first window for the life of the install.
_t12_i = derived(lambda: _T12Controller(
{"price_provider": "off", "price_timeout_s": 10, "price_horizon_h": 48,
"static_profile": STATIC_BE}).refresh_once())
_t12_i_plan, _t12_i_at = _t12_i.tariff_plan, _t12_i.tariff_at
_t12_i.refresh_once()
check("PRECONDITION: a `price_provider: off` + static_profile site resolves to "
"the CONFIGURED product, not to a §11.4 row - origin 'static', prices, "
"and from_feed false because there is no feed to have observed",
getattr(_t12_i_plan, "origin", None) == "static"
and _t12_i_plan.has_prices is True and _t12_i_plan.from_feed is False)
check("...and TARIFF-10 now keeps this task alive on exactly that site, where "
"this card's guard must NEVER latch: the second refresh replaces the "
"first and the timestamp moves, so the plan is regenerated for the "
"current window every 30 minutes as it always was",
_t12_i.tariff_plan is not _t12_i_plan
and _t12_i.tariff_plan.origin == "static"
and _t12_i.tariff_at != _t12_i_at)
# ⚠️ AND THE OTHER CONFIGURATION TARIFF-10 ADDED - a provider configured with
# trading DISARMED - is graded by every driven check above without a second
# fixture, because nothing on this path reads `arbitrage_enabled` at all. That
# is a claim about the code rather than about a behaviour, so it is asserted as
# one (habit 4's exception, FCST-06's precedent): a fixture that set the option
# and asserted no difference would be comparing a thing against itself.
_T12_TRADING_OPTS = ("arbitrage_enabled", "cycle_cost_threshold_eur",
"arbitrage_soc_floor", "arbitrage_soc_ceiling")
# ⚠️ AND IT IS WHAT THE FUNCTION READS THAT IS SCANNED, NOT ITS TEXT. Two
# earlier drafts of this check were red against a CORRECT tree: the first
# because the docstring paragraph that SAYS `arbitrage_enabled` is not read is
# itself an occurrence of the string, the second because the log line says
# "arbitrage, where it is armed, stops with it" - which is a sentence, not a
# read. A substring test over a function grades its prose as much as its
# behaviour (habit 4). So the symbols are taken from the AST, and the string
# constants are taken separately and with the docstring dropped: naming
# arbitrage in a message is fine, READING a trading option is the thing claimed
# not to happen.
_t12_syms = derived(lambda: {getattr(n, "id", None) or getattr(n, "attr", None)
for n in ast.walk(_refresh)
if isinstance(n, (ast.Name, ast.Attribute))})
_t12_consts = derived(lambda: {c.value for b in _refresh.body[1:]
for c in ast.walk(b)
if isinstance(c, ast.Constant)
and isinstance(c.value, str)})
check("run_tariff_refresh() READS no trading option and calls nothing in "
"app/arbitrage.py, so keeping a plan is a decision about what the add-on "
"has LOOKED UP - TARIFF-10's own distinction, and why the checks above "
"cover a provider-configured site whether or not arbitrage is armed",
isinstance(_t12_syms, set)
and not any("arbitrage" in (n or "") for n in _t12_syms)
and not (_t12_consts & set(_T12_TRADING_OPTS))
# ...and both scanners can see what IS there, or they grade nothing: the
# one option this task does read, and a name it really does call.
and "price_horizon_h" in _t12_consts
and "fresh_tariff_plan" in _t12_syms
# ...and the docstring really was dropped, or the first draft's defect is
# back and the constant set is grading prose again.
and not any("RETURNS, NEVER RAISES" in c for c in _t12_consts))
# -- the vocabulary, graded on its own so a mutation of it has a named home --
check("TariffPlan.from_feed is true of exactly the observation - a §11.4 "
"fallback, a no-data read and a CONFIGURED static profile are all false, "
"and it is not a synonym for has_prices",
_t12_good.from_feed is True and _t12_fb.from_feed is False
and _t12_deg.from_feed is False
and static_plan({"static_profile": STATIC_BE}, D0,
D0 + timedelta(days=2), now=D0).from_feed is False
and static_plan({"static_profile": STATIC_BE}, D0,
D0 + timedelta(days=2), now=D0).has_prices is True)
# ⚠️ A SHAPE CLAIM, AND IT IS ALLOWED TO BE ONE (FCST-06's precedent). "The age
# is judged in ONE place" is a claim about the code, not about a behaviour: a
# transcribed copy of the same comparison behaves identically until the day the
# two drift. Every other check in this section is driven.
check("run_tariff_refresh() asks fresh_tariff_plan() rather than re-deriving "
"the age bound - one place judges it, which is what that method's own "
"docstring is for",
_refresh is not None
and {n.func.attr for n in ast.walk(_refresh)
if isinstance(n, ast.Call) and isinstance(n.func, ast.Attribute)}
>= {"fresh_tariff_plan"})
_price.https_get = _t12_saved_transport
# --------------------------------------------------------------------------- #
print()
print("9. the startup gate - the configurations that cannot work are refused")
@@ -1376,6 +1813,312 @@ check("soc_bounds() returns the shipped pair, floor first",
float(PRICE_OPTION_DEFAULTS["arbitrage_soc_ceiling"])))
# --------------------------------------------------------------------------- #
print()
print("10. TARIFF-14 - the QUARTER-HOURLY day, end to end through decide()")
# ⚠️ THIS IS THE PROPERTY THE DEFECT ACTUALLY BROKE, WHICH IS WHY IT IS GRADED
# HERE AND NOT ONLY ON `at()`. `_act()` decides whether this interval is the
# buy one or the sell one with `here.start == v.cheapest.start` - an IDENTITY
# test between the point `_point_at()` matched and the point the verdict was
# built from. `_Series.at()` used to return the first point whose start was
# within one hour BEFORE the instant, in list order, so on a 900 s series that
# identity almost never held: the strategy idled through exactly the interval
# it exists to trade, and read `here.provisional` and §17's `price < 0` off a
# point that was not the one being traded.
#
# Belgium moved to a 15-minute MTU on 2025-10-01 and the live day-ahead feed
# returns 96 points at 900 s. `static_tariff.py` emits HOURLY points, so every
# fixture above this line - and every static site - is immune; this is
# reachable only down the dynamic path TARIFF-08/09 made work.
QD0 = datetime(2026, 1, 15, 0, 0, tzinfo=timezone(timedelta(hours=1)))
QUARTER = timedelta(minutes=15)
def qplan_of(prices, *, provisional=False, start=QD0, gaps=None):
"""A TariffPlan over `prices` spaced by `gaps` seconds (900 s by default).
`gaps` is what makes the MIXED fixture below expressible: FCST-04's
carry-forward emits HOURLY carried points beside QUARTER-HOURLY firm ones,
and the soak measured spacing [900, 3600] in 70 % of samples.
"""
g = [900] * (len(prices) - 1) if gaps is None else list(gaps)
flags = ([bool(provisional)] * len(prices)
if isinstance(provisional, bool) else list(provisional))
pts, t = [], start
for i, (p, f) in enumerate(zip(prices, flags)):
pts.append(PricePoint(start=brussels(t), eur_per_kwh=round(float(p), 6),
provisional=bool(f)))
if i < len(g):
t += timedelta(seconds=g[i])
return TariffPlan(PriceRead.series(pts, PriceProduct.DAY_AHEAD,
"generic_dayahead", start),
None, "provider")
def qprobe(i, *, start=QD0, gaps=None):
"""An instant 7 minutes into slot `i` - inside every window, never on a
boundary, and NOT the midpoint of an hour either, so an implementation that
answered per hour and one that answers per quarter differ here."""
g = [900] * i if gaps is None else list(gaps)[:i]
return start + timedelta(seconds=sum(g)) + timedelta(minutes=7)
def shipped_at(series, u):
"""THE RULE TARIFF-14 REPLACED, kept here as the COUNTERFACTUAL ONLY.
The first point whose start is within one hour before `u`, in list order.
Nothing below is graded against it; it exists so that the fixtures can be
shown to SEPARATE the two rules. A check whose data cannot reach the
failure it names is decoration (MUTATION.md), and on an hourly series the
two rules agree at every instant - which is why 29 files stayed green.
"""
uu = u.astimezone(UTC)
for p in series.points:
s = p.start.astimezone(UTC)
if s <= uu < s + timedelta(hours=1):
return p
return None
# The hourly day interpolated across its quarters, with the two extremes
# planted OFF the hour boundary on purpose.
QBASE = []
for _i in range(24):
_a, _b = DAY[_i], DAY[(_i + 1) % 24]
for _q in range(4):
QBASE.append(round(_a + (_b - _a) * _q / 4 + 0.000001 * (_i * 4 + _q),
6))
Q_CHEAP_I = 4 * 5 + 3 # 05:45 - a :45 quarter, not a whole hour
Q_DEAR_I = 4 * 19 + 1 # 19:15 - a :15 quarter, not a whole hour
QDAY = list(QBASE)
QDAY[Q_CHEAP_I] = round(min(QBASE) - 0.030, 6)
QDAY[Q_DEAR_I] = round(max(QBASE) + 0.030, 6)
QPLAN = qplan_of(QDAY)
QPTS = QPLAN.read.points
print("10a. the fixture is not degenerate, and it SEPARATES the two rules")
_qgaps = sorted({int((b.start - a.start).total_seconds())
for a, b in zip(QPTS, QPTS[1:])})
check(f"the day is 96 points at {_qgaps} s - the shape the live feed has "
f"returned since Belgium's 2025-10-01 move to a 15-minute MTU",
len(QPTS) == 96 and _qgaps == [900])
check(f"the cheapest quarter is slot {Q_CHEAP_I} and the dearest slot "
f"{Q_DEAR_I} - neither first, neither last, {abs(Q_CHEAP_I - Q_DEAR_I)} "
f"slots apart, and NEITHER ON A WHOLE HOUR, without which this section "
f"would grade nothing an hourly fixture does not already grade",
QDAY.index(min(QDAY)) == Q_CHEAP_I and QDAY.index(max(QDAY)) == Q_DEAR_I
and Q_CHEAP_I % 4 != 0 and Q_DEAR_I % 4 != 0
and Q_CHEAP_I not in (0, 95) and Q_DEAR_I not in (0, 95)
and abs(Q_CHEAP_I - Q_DEAR_I) > 1)
check("...and all 96 prices are distinct, so 'the cheapest quarter' names one "
"quarter and no tie-break is what any check below is really grading",
len(set(QDAY)) == 96)
check(f"...and its spread is {round(max(QDAY) - min(QDAY), 6):.4f} EUR/kWh, "
f"clear of the widened threshold, so what follows grades the strategy "
f"and not the spread rung",
round(max(QDAY) - min(QDAY), 6) > required_spread(_shipped, NO_WEATHER))
# ⚠️ HABIT 4, AND THE ONLY REASON THIS SECTION IS WORTH ITS LINES. State, as a
# check, that the shipped rule and the fixed one give DIFFERENT points at the
# two probes that matter. Without it, every check below could be passing on a
# fixture the defect could not have changed.
_cf_buy = derived(lambda: shipped_at(QPLAN.read, qprobe(Q_CHEAP_I)))
_cf_sell = derived(lambda: shipped_at(QPLAN.read, qprobe(Q_DEAR_I)))
check(f"the shipped rule and this one give DIFFERENT points at both probes - "
f"buy: {derived(lambda: _cf_buy.eur_per_kwh)} instead of "
f"{QDAY[Q_CHEAP_I]}, sell: {derived(lambda: _cf_sell.eur_per_kwh)} "
f"instead of {QDAY[Q_DEAR_I]} - so the fixture can reach the failure it "
f"names",
_cf_buy is not QPTS[Q_CHEAP_I] and _cf_sell is not QPTS[Q_DEAR_I]
and _cf_buy is not None and _cf_sell is not None)
print()
print("10b. the buy/sell identity test matches the actual extremum interval")
_qv = QPLAN.arbitrage(peak_guard_active=False, min_spread_eur_kwh=_shipped,
uncertainty=NO_WEATHER)
check("the verdict's cheapest and dearest are the two QUARTERS, not the hours "
"containing them - `static_tariff` reduces over every firm point and "
"carries no resolution assumption of its own",
derived(lambda: _qv.cheapest) is QPTS[Q_CHEAP_I]
and derived(lambda: _qv.dearest) is QPTS[Q_DEAR_I])
_pa_buy = derived(lambda: _arb._point_at(QPLAN, qprobe(Q_CHEAP_I)))
_pa_sell = derived(lambda: _arb._point_at(QPLAN, qprobe(Q_DEAR_I)))
check("THE PROPERTY THE DEFECT BROKE: at the buy probe `_point_at()` returns "
"the verdict's OWN cheapest point, by identity - which is what makes "
"`_act()`'s `here.start == v.cheapest.start` true, and what made it "
"false on every quarter-hourly series the add-on has ever fetched",
_pa_buy is derived(lambda: _qv.cheapest))
check("...and the same at the sell probe, against the verdict's dearest",
_pa_sell is derived(lambda: _qv.dearest))
_qbuy = decide(QPLAN, qprobe(Q_CHEAP_I), peak_guard_active=False, soc=50.0,
min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
soc_ceiling_pct=95.0)
_qsell = decide(QPLAN, qprobe(Q_DEAR_I), peak_guard_active=False, soc=50.0,
min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
soc_ceiling_pct=95.0)
check("...so decide() CHARGES in the cheapest quarter and DISCHARGES in the "
"dearest one, which is the trade the strategy exists to make and the "
"one it slept through",
_qbuy.action == CHARGE and _qsell.action == DISCHARGE)
check(f"...at THAT QUARTER's price and not at the price of the hour "
f"containing it (buy {_qbuy.price_now_eur_kwh}, sell "
f"{_qsell.price_now_eur_kwh})",
near(_qbuy.price_now_eur_kwh, QDAY[Q_CHEAP_I])
and near(_qsell.price_now_eur_kwh, QDAY[Q_DEAR_I]))
check("...and both reasons quote that quarter's price, so the operator's log "
"names the interval that was actually traded",
f"{QDAY[Q_CHEAP_I]:.4f}" in _qbuy.reason
and f"{QDAY[Q_DEAR_I]:.4f}" in _qsell.reason)
# The negative arm. Without it "it acted" could be "it acts everywhere".
_QOFF_I = Q_CHEAP_I + 2
check(f"the off-peak probe's own price differs from its hour's :00 price "
f"({QDAY[_QOFF_I]} vs {QDAY[4 * (_QOFF_I // 4)]}), so the check below "
f"can tell which point the decision read",
QDAY[_QOFF_I] != QDAY[4 * (_QOFF_I // 4)]
and _QOFF_I not in (Q_CHEAP_I, Q_DEAR_I))
_qoff = decide(QPLAN, qprobe(_QOFF_I), peak_guard_active=False, soc=50.0,
min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
soc_ceiling_pct=95.0)
check("a quarter that is neither extremum still idles, for the reason that "
"names the right rule, and reports ITS OWN price - not the price of the "
"quarter the shipped scan would have matched",
_qoff.action is None and _qoff.reason == _arb.IDLE_OFF_PEAK_HOUR
and near(_qoff.price_now_eur_kwh, QDAY[_QOFF_I]))
print()
print("10c. §17 and §8.3 are evaluated on the QUARTER being traded")
# ⚠️ §17 AT 900 s. The whole day is below zero and the dearest QUARTER is the
# least negative one - section 2's shape, at the resolution the live feed
# actually publishes. Under the shipped scan this probe never reached the
# guard at all: the identity test failed first and the decision idled as
# "neither the buy hour nor the sell hour", so §17 was not so much wrong as
# never consulted.
_shift = round(max(QDAY) + 0.020, 6)
QNEG = [round(p - _shift, 6) for p in QDAY]
QNEG_PLAN = qplan_of(QNEG)
check(f"the negative day's dearest quarter is slot {Q_DEAR_I} and it is BELOW "
f"ZERO ({max(QNEG):+.4f}), and its spread still clears the widened "
f"threshold - so the refusal below is §17's and not the spread rung's",
QNEG.index(max(QNEG)) == Q_DEAR_I and max(QNEG) < 0
and round(max(QNEG) - min(QNEG), 6) > required_spread(_shipped,
NO_WEATHER))
_qneg_sell = decide(QNEG_PLAN, qprobe(Q_DEAR_I), peak_guard_active=False,
soc=50.0, min_spread_eur_kwh=_shipped,
uncertainty=NO_WEATHER, charge_w=MAX_W, discharge_w=MAX_W,
soc_floor_pct=20.0, soc_ceiling_pct=95.0)
check("selling into a negative-price QUARTER is refused, and the reason names "
"§17 and THAT QUARTER's price",
_qneg_sell.action is None and "§17" in _qneg_sell.reason
and f"{max(QNEG):+.4f}" in _qneg_sell.reason)
# ...and the other side, so the refusal is about the sign and nothing else.
QPOS = list(QNEG)
QPOS[Q_DEAR_I] = round(-max(QNEG), 6)
check("...while the SAME quarter-hourly series with that ONE price's SIGN "
"FLIPPED discharges instead",
decide(qplan_of(QPOS), qprobe(Q_DEAR_I), peak_guard_active=False,
soc=50.0, min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
soc_ceiling_pct=95.0).action == DISCHARGE
and sum(1 for a, b in zip(QNEG, QPOS) if a != b) == 1)
# §8.3's flag, read off the covering point. `_act`'s own provisional guard
# stays unreachable here for section 4's reason - the extrema are firm by
# `_firm_or_die` and starts are unique - and THAT ARGUMENT IS EXACTLY WHAT THE
# DEFECT BROKE at 900 s, because the point `at()` returned was not the
# verdict's point at all. It is restated here at the resolution that matters.
check("the point matched for the buy quarter IS the verdict's own cheapest "
"point and is FIRM, so `_act`'s §8.3 guard is unreachable at 900 s for "
"the same construction section 4 states at 3600 s",
_pa_buy is derived(lambda: _qv.cheapest)
and derived(lambda: _pa_buy.provisional) is False
and len({p.start for p in QPTS}) == len(QPTS))
print()
print("10d. the MIXED series FCST-04 actually emits, traded end to end")
# ⚠️ 70 % OF SOAK SAMPLES, NOT AN EDGE CASE. The carry-forward emits HOURLY
# carried points beside QUARTER-HOURLY firm ones, so a window derived from one
# global spacing would be wrong in the NORMAL steady state. Here: 12 carried
# hourly points for the morning, then firm quarters for the rest of the day.
MIX_N_H, MIX_N_Q = 12, 48
MIX_GAPS = [3600] * MIX_N_H + [900] * (MIX_N_Q - 1)
MIX_PRICES = [QDAY[4 * i] for i in range(MIX_N_H)] + QDAY[4 * MIX_N_H:]
MIX_FLAGS = [True] * MIX_N_H + [False] * MIX_N_Q
MIX_PLAN = qplan_of(MIX_PRICES, provisional=MIX_FLAGS, gaps=MIX_GAPS)
MIX_PTS = MIX_PLAN.read.points
_mix_gaps = sorted({int((b.start - a.start).total_seconds())
for a, b in zip(MIX_PTS, MIX_PTS[1:])})
check(f"the mixed plan really is mixed - spacing {_mix_gaps} s, "
f"{MIX_N_H} carried and {MIX_N_Q} firm points, so neither half is what "
f"the other half's window rule would give",
_mix_gaps == [900, 3600]
and sum(1 for p in MIX_PTS if p.provisional) == MIX_N_H
and sum(1 for p in MIX_PTS if not p.provisional) == MIX_N_Q)
_mix_cheap_i = MIX_N_H + (MIX_PRICES[MIX_N_H:].index(min(MIX_PRICES[MIX_N_H:])))
_mix_dear_i = MIX_N_H + (MIX_PRICES[MIX_N_H:].index(max(MIX_PRICES[MIX_N_H:])))
check(f"...and its firm half's extremes are quarters {_mix_cheap_i} and "
f"{_mix_dear_i}, neither on a whole hour, so the identity test below is "
f"about the quarter and not about the hour",
(_mix_cheap_i - MIX_N_H) % 4 != 0 and (_mix_dear_i - MIX_N_H) % 4 != 0)
_mv = MIX_PLAN.arbitrage(peak_guard_active=False, min_spread_eur_kwh=_shipped,
uncertainty=NO_WEATHER)
_mix_buy_t = qprobe(_mix_cheap_i, gaps=MIX_GAPS)
_mix_sell_t = qprobe(_mix_dear_i, gaps=MIX_GAPS)
check("the verdict over a MIXED series still names firm quarters, and "
"`_point_at()` matches them by identity at their own probes",
derived(lambda: _mv.cheapest) is MIX_PTS[_mix_cheap_i]
and derived(lambda: _mv.dearest) is MIX_PTS[_mix_dear_i]
and derived(lambda: _arb._point_at(MIX_PLAN, _mix_buy_t))
is MIX_PTS[_mix_cheap_i]
and derived(lambda: _arb._point_at(MIX_PLAN, _mix_sell_t))
is MIX_PTS[_mix_dear_i])
check("...so the mixed day CHARGES and DISCHARGES in its firm quarters, at "
"their own prices",
decide(MIX_PLAN, _mix_buy_t, peak_guard_active=False, soc=50.0,
min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
soc_ceiling_pct=95.0).action == CHARGE
and decide(MIX_PLAN, _mix_sell_t, peak_guard_active=False, soc=50.0,
min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
soc_ceiling_pct=95.0).action == DISCHARGE)
# An instant inside a CARRIED hour of the same series. The covering point is
# the carried one; the shipped scan answered a point from the FIRM block
# earlier in the list, which is §8.3's flag read off the wrong point.
_mix_carried_t = QD0 + timedelta(hours=MIX_N_H - 1, minutes=37)
check("an instant inside a CARRIED hour of the mixed series matches the "
"CARRIED point, whose provisional flag is what §8.3 reads - and the "
"strategy does nothing there, because a carried point is not in the "
"firm trade set",
derived(lambda: _arb._point_at(MIX_PLAN, _mix_carried_t))
is MIX_PTS[MIX_N_H - 1]
and MIX_PTS[MIX_N_H - 1].provisional is True
and decide(MIX_PLAN, _mix_carried_t, peak_guard_active=False, soc=50.0,
min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
soc_ceiling_pct=95.0).action is None)
print()
print("10e. the HOURLY day is unchanged, which is the regression half")
# ⚠️ A REGRESSION GUARD, NOT A WITNESS. On an hourly series the shipped rule
# and this one agree at every instant, which is precisely why 29 files stayed
# green through the defect. It is here so that a fix which got 900 s right by
# breaking 3600 s cannot pass.
_h_buy = act(PLAN, CHEAP_H, thresh=_shipped, unc=NO_WEATHER)
_h_sell = act(PLAN, DEAR_H, thresh=_shipped, unc=NO_WEATHER)
check("the hourly fixture still charges in its cheapest hour and discharges "
"in its dearest, at those hours' prices",
_h_buy.action == CHARGE and _h_sell.action == DISCHARGE
and near(_h_buy.price_now_eur_kwh, DAY[CHEAP_H])
and near(_h_sell.price_now_eur_kwh, DAY[DEAR_H]))
check("...and a probe 59 minutes into an hourly slot still matches that slot, "
"so the hourly window did not shrink under the fix",
derived(lambda: _arb._point_at(
PLAN, D0 + timedelta(hours=CHEAP_H, minutes=59)))
is derived(lambda: PLAN.read.points[CHEAP_H]))
# --------------------------------------------------------------------------- #
print()
if fails:
+1 -1
View File
@@ -7,7 +7,7 @@ WHAT UX-01 AC 4 ASKED FOR, IN ITS OWN WORDS: "every `schema:` key absent from
check, not a grep."
WHAT SHIPPED. Clause one is VACUOUS: `options:` and `schema:` declare the same
71 keys, so "every schema key absent from options" quantifies over the empty set
72 keys, so "every schema key absent from options" quantifies over the empty set
and is true of any tree, including one with no defaults at all. Clause two -
"the table changed goes red" - does work, six times over, once per mirror in its
own suite. Clause three, THE ONE WITH THE CONTENT, was never written. The only
File diff suppressed because it is too large Load Diff
+302 -21
View File
@@ -59,6 +59,7 @@ Read MUTATION.md before you report a survivor.
import ast
import inspect
import itertools
import os
import sys
from datetime import datetime, timedelta, timezone
@@ -272,31 +273,266 @@ check("the provenance audit REFUSES a carried hour flagged non-provisional - "
"the 'stale but plausible' point, which is SAFETY-18's frozen-but-fresh "
"meter one layer up",
raises(lambda: _fc._audit([_stale_copy], {}), ValueError))
# ⚠️ `published` MAPS AN HOUR TO A TUPLE, not to a point (FCST-04 H1). An hour
# may legitimately hold four published points - see the quarter-hourly block
# below - so the audit takes the whole set and checks membership by identity.
check("...and refuses a published hour that arrived as a COPY rather than as "
"the published object, which is where a flag gets lost",
raises(lambda: _fc._audit(
[PricePoint(start=TODAY_PART.points[0].start,
eur_per_kwh=TODAY_PART.points[0].eur_per_kwh,
provisional=TODAY_PART.points[0].provisional)],
{D0: TODAY_PART.points[0]}), ValueError))
{D0: (TODAY_PART.points[0],)}), ValueError))
check("...and never repairs a flag: the audit's only outcomes are silence and "
"an exception, so a series it passed was already correct",
_fc._audit(list(FILLED.points), {u: TODAY_PART.points[i]
_fc._audit(list(FILLED.points), {u: (TODAY_PART.points[i],)
for i, u in enumerate(DAY[:PUBLISHED_N])})
is None)
# ⚠️ A PUBLISHED POINT MAY BE PROVISIONAL. eneco_eddy's current-quarter print
# is settled after the fact and says so (§9.2). "Published" is not "firm".
_eddy = series_of(DAY[:2], lambda u: t_price(brussels(u).hour),
PriceProduct.REALTIME, "eneco_eddy", provisional=True)
# ⚠️ A PUBLISHED POINT MAY BE PROVISIONAL. A real-time / imbalance product's
# current interval is settled after the fact and says so per point (§9.2).
# "Published" is not "firm", and §9.3 carries the flag PER POINT for exactly
# this. No shipped plugin declares REALTIME since TARIFF-07, so the series is
# declared here - which is the only place the case can come from now, and is
# why this stimulus did not go with the plugin.
_rt = series_of(DAY[:2], lambda u: t_price(brussels(u).hour),
PriceProduct.REALTIME, "_realtime_not_shipped",
provisional=True)
_eddy_filled = derived(lambda: fill_from_yesterday(
_eddy, yesterday=YESTERDAY, since=D0, until=D1, stamp=D0))
_rt, yesterday=YESTERDAY, since=D0, until=D1, stamp=D0))
check("a PUBLISHED point that the feed itself flagged provisional keeps that "
"flag - the audit checks provenance, and does not force published hours "
"to False",
_eddy_filled.kind == "series"
and all(p.provisional for p in _eddy_filled.points[:2])
and _eddy_filled.points[0] is _eddy.points[0])
and _eddy_filled.points[0] is _rt.points[0])
# ⚠️ AND A SUB-HOURLY FEED IS NOT DECIMATED TO ONE POINT AN HOUR (FCST-04 H1).
# BELGIAN DAY-AHEAD ITSELF SETTLES PER QUARTER - the 15-minute MTU of
# 2025-10-01, which `generic_dayahead` reads as 96 points a day - and
# `_validate_raw()` accepts any strictly-increasing spacing, so a published
# hour can hold four points. The
# fill walks WHOLE HOURS to decide what needs carrying; that must not become a
# rule about what a published hour may contain. Measured on FCST-04's first
# draft, which keyed the published set `{_utc_hour(p.start): p ...}`: a
# 96-point feed came out 15 points long and its arbitrage spread fell from
# 0.143 to 0.014 EUR/kWh, on an ALL-FIRM, fully-published series.
_QN = 16 # four hours of quarters
_QUARTERS = PriceRead.series(
[PricePoint(start=brussels(D0 + timedelta(minutes=15 * n)),
eur_per_kwh=round(0.100000 + 0.011300 * n, 6),
provisional=False)
for n in range(_QN)],
PriceProduct.DAY_AHEAD, "generic_dayahead", D0)
_QHOURS = {_fc._utc_hour(p.start) for p in _QUARTERS.points}
check(f"the quarter-hourly fixture really is sub-hourly - {_QN} points across "
f"{len(_QHOURS)} hours - or every check below passes for an hourly feed "
f"too and grades nothing",
len(_QUARTERS.points) == _QN and len(_QHOURS) == _QN // 4)
_qfill = derived(lambda: fill_from_yesterday(_QUARTERS, yesterday=YESTERDAY,
since=D0, until=D1, stamp=D0))
_qfirm = derived(lambda: [p for p in _qfill.points if not p.provisional])
check(f"every one of the {_QN} published points survives the fill BY IDENTITY "
f"({len(_qfirm)} kept). One-per-hour keying would leave {_QN // 4}, and "
f"the spread over the hours the feed actually published would collapse "
f"with them",
len(_qfirm) == _QN
and all(any(p is q for q in _QUARTERS.points) for p in _qfirm))
check(f"...and the {len(DAY) - len(_QHOURS)} whole hours it published NOTHING "
f"for are carried, ONE point each: a carried hour is hourly whatever the "
f"feed's spacing is, because yesterday's profile is only known per "
f"wall-clock hour",
_qfill.kind == "series"
and len([p for p in _qfill.points if p.provisional])
== len(DAY) - len(_QHOURS))
check("...and the filled series is still ordered by instant across the two "
"kinds, so `at()` and `window()` are not reading a shuffled list",
_qfill.kind == "series"
and [p.start.astimezone(UTC) for p in _qfill.points]
== sorted(p.start.astimezone(UTC) for p in _qfill.points))
_PUB_Q = _fc._published_by_hour(_QUARTERS)
# ⚠️ `derived`, NOT A RAW CALL. Under the decimating mutation this check
# exists for, `_audit` RAISES here - and a harness that dies on the mutation it
# is grading reports a traceback instead of a FAIL, which MUTATION.md forbids.
check("the audit ACCEPTS four published points inside one hour, which is the "
"case the repeated-hour COUNT it replaced could not express",
derived(lambda: _fc._audit(list(_QUARTERS.points), _PUB_Q)) is None)
check("...and still REFUSES a repeated CARRIED hour, which is the DST "
"mispricing that count existed to catch",
raises(lambda: _fc._audit([_fc._carried(D0, 0.11),
_fc._carried(D0, 0.12)], {}), ValueError))
check("...and STILL refuses the SAME published point emitted twice - the "
"provider's points are strictly increasing, so a repeat is never a "
"second print. The old hour COUNT already refused this (one point twice "
"is one hour twice); what is graded here is that the relaxation did not "
"lose it, not that it added it",
raises(lambda: _fc._audit([_QUARTERS.points[0], _QUARTERS.points[0]],
_PUB_Q), ValueError))
check("...and STILL refuses a CARRIED point standing in an hour the feed DID "
"publish. The old COUNT never looked at it; the old IDENTITY rule did, "
"and raised - so this is preserved behaviour, not new strictness. "
"FCST-04's text called it new here and in `_audit`'s docstring, and a "
"measured differential says otherwise",
raises(lambda: _fc._audit([_fc._carried(D0, 0.11)], _PUB_Q), ValueError))
# ⚠️ AND WHAT THE AUDIT DOES NOT DO, STATED WHERE A READER OF THESE THREE
# CHECKS WOULD OTHERWISE ASSUME IT (FCST-05, MUTATION.md habit 16). `_audit` is
# a PROVENANCE post-condition, not a COMPLETENESS one: it grades every point
# that IS emitted and says nothing about points that are not. So it accepts the
# very decimation FCST-04 was raised for. That is not a regression - FCST-03's
# audit was equally blind - but it means the audit is NOT the thing that stops
# H1 coming back. The count and identity checks above this line are.
check("the audit ACCEPTS the H1 decimation itself - four published points in "
"one hour, only the last emitted - and accepts an EMPTY output. It is a "
"provenance check, not a completeness one, and the checks that guard "
"provenance completeness are the COUNT and IDENTITY ones above",
derived(lambda: _fc._audit([_QUARTERS.points[3]], _PUB_Q)) is None
and derived(lambda: _fc._audit([], _PUB_Q)) is None)
# --------------------------------------------------------------------------- #
# ⚠️ AND THE CLAIM "STRICTER" IS ITSELF A CHECK NOW (FCST-05).
#
# FCST-04 rewrote `_audit` and wrote, in the module and in the three checks
# above, that the result was STRICTER than FCST-03's. It is not. Two of the
# stated reasons were false - the old `seen` count already refused the same
# point emitted twice, and the old IDENTITY rule already refused a carried
# point in a published hour - and a right decision with a fabricated reason is
# a defect in this repository (MUTATION.md, and the reason DECISIONS files
# exist). So the claim is measured here rather than asserted in prose:
# FCST-03's audit is reproduced below from dd40c56 - its RULES verbatim, its
# message TEXT shortened, and nothing here reads a message - and the two are
# run over the whole domain `_audit` can express, in BOTH directions.
#
# THE DOMAIN, AND ITS SIZE. Every published mask over 3 hours (2**3) crossed
# with every output series of length 0-3 drawn from 3 hours x 5 point kinds.
# The nominal cross-product is 28 928; three of those kinds - published,
# published-provisional, copy-of-published - DO NOT EXIST for an hour the feed
# did not publish, and those combinations are SKIPPED rather than faked, which
# leaves 11 990 realisable comparisons. A fabricated "published point" for an
# unpublished hour is not in either audit's domain, and inventing one would
# grade the fixture rather than the code.
#
# ⚠️ THE OLD COPY IS A LITERAL, NOT A CALL. MUTATION.md habit 2: a differential
# whose two sides both come from the shipped module compares a thing against
# itself and moves with every edit. This side is frozen text.
def _audit_fcst03(points, published):
"""FCST-03's `_audit`, verbatim from dd40c56. Its `published` maps a UTC
hour to ONE point, which is exactly the expressiveness this ticket relaxed.
"""
seen = set()
for p in points:
u = _fc._utc_hour(p.start)
if u in seen:
raise ValueError("the filled series repeats " + u.isoformat())
seen.add(u)
src = published.get(u)
if src is None:
if p.provisional is not True:
raise ValueError(u.isoformat() + " is carried and is flagged "
"provisional=" + repr(p.provisional))
else:
if p is not src:
raise ValueError(u.isoformat() + " is published but the series "
"carries a COPY of that point")
def _verdict(fn, pts, pub):
try:
fn(list(pts), pub)
return "accept"
except ValueError:
return "raise"
_DH = [D0 + timedelta(hours=6 + i) for i in range(3)]
_weaker, _stronger, _compared = [], [], 0
for _mask in itertools.product((0, 1), repeat=3):
_pub = {u: PricePoint(start=u, eur_per_kwh=0.10 + i, provisional=False)
for i, (u, _on) in enumerate(zip(_DH, _mask)) if _on}
_old_pub = dict(_pub)
_new_pub = {u: (q,) for u, q in _pub.items()}
# The five shapes an output point can take at a given hour. Three of them
# do not exist for an hour the feed did not publish, and those combinations
# are skipped rather than faked - a fabricated "published point" for an
# unpublished hour is not in either audit's domain.
_kinds = []
for _u in _DH:
_src = _pub.get(_u)
_kinds.append(("published", _src))
_kinds.append(("published-provisional",
None if _src is None else
PricePoint(start=_src.start,
eur_per_kwh=_src.eur_per_kwh,
provisional=True)))
_kinds.append(("carried", _fc._carried(_u, 0.5)))
_kinds.append(("carried-unflagged",
PricePoint(start=_u, eur_per_kwh=0.5,
provisional=False)))
_kinds.append(("copy-of-published",
None if _src is None else
PricePoint(start=_src.start,
eur_per_kwh=_src.eur_per_kwh,
provisional=_src.provisional)))
for _n in range(4):
for _combo in itertools.product(_kinds, repeat=_n):
if any(k[1] is None for k in _combo):
continue
_pts = [k[1] for k in _combo]
_compared += 1
_o = _verdict(_audit_fcst03, _pts, _old_pub)
_nw = _verdict(_fc._audit, _pts, _new_pub)
if _o == "raise" and _nw == "accept":
_weaker.append((_mask, [k[0] for k in _combo]))
if _o == "accept" and _nw == "raise":
_stronger.append((_mask, [k[0] for k in _combo]))
# ⚠️ THE PRECONDITION FIRST (MUTATION.md habit 1). A differential over a
# domain that cannot reach the two disputed cases would score 0-0 for the wrong
# reason and read exactly like this one, so the two cases are driven against
# the OLD audit by name before the totals are graded.
_probe = PricePoint(start=_DH[0], eur_per_kwh=0.1, provisional=False)
check(f"the differential really covers the two claims it is here to settle, "
f"or it is {_compared} runs of nothing: FCST-03's own audit raises on "
f"BOTH 'the same published point twice' and 'a carried point in a "
f"published hour', which is what FCST-04 said it did not",
_compared > 3000
and _verdict(_audit_fcst03, [_probe, _probe], {_DH[0]: _probe}) == "raise"
and _verdict(_audit_fcst03, [_fc._carried(_DH[0], 0.5)],
{_DH[0]: _probe}) == "raise")
check(f"...and over all {_compared} of them the reworked audit is NOT WEAKER "
f"than FCST-03's: {len(_weaker)} case(s) the old one raised and this one "
f"accepts {_weaker[:2]}",
_weaker == [])
check(f"...and NOT STRONGER either: {len(_stronger)} case(s) the old one "
f"accepted and this one raises {_stronger[:2]}. FCST-04's docstring and "
f"two of these checks said it was stricter and named two reasons, and "
f"BOTH were already refused by the old count and the old identity rule. "
f"The only behavioural change is the RELAXATION below, which is what the "
f"rework was for",
_stronger == [])
_QP = [PricePoint(start=_DH[0] + timedelta(minutes=15 * i),
eur_per_kwh=0.1 + i, provisional=False) for i in range(4)]
check("...and the relaxation is real and is NOT expressible to the old audit: "
"four distinct published points inside one hour, which FCST-03's count "
"refused as a repeated hour and this one accepts",
_verdict(_fc._audit, _QP, {_DH[0]: tuple(_QP)}) == "accept"
and _verdict(_audit_fcst03, _QP, {_DH[0]: _QP[0]}) == "raise")
# ⚠️ THE HORIZON IS WHOLE HOURS AT BOTH ENDS (FCST-04 L8). `since` is floored
# and `until` was not, so a 48 h request made at any instant off the hour came
# back 49 hours long. Graded at an OFF-THE-HOUR instant, because on the hour
# the two implementations agree and the check would grade nothing.
_off = D0 + timedelta(hours=8, minutes=17, seconds=43)
for _h in (1, 12, 48):
_n = derived(lambda h=_h: len(fill_from_yesterday(
TODAY_PART, yesterday=YESTERDAY, since=_off,
until=_off + timedelta(hours=h), stamp=_off).points))
check(f"a {_h} h horizon asked for at {_off.isoformat()} - 17 min 43 s "
f"past the hour, the shape a running controller always has - is "
f"{_h} points long, not {_h + 1}",
_n == _h)
# The structural half: the constructor has nothing to pass, and the file is
# scanned so the pass-through idiom cannot come back looking ordinary.
@@ -713,13 +949,20 @@ _normal_named_eddy = derived(lambda: min_accepted_spread(
PriceProduct.DAY_AHEAD, frozenset(), BASE, provider="eneco_eddy"))
check("the widening follows the DECLARED PriceProduct even when the provider "
"is called 'generic_dayahead', and does NOT follow the name "
"'eneco_eddy' when the declared product is day-ahead - so a sixth "
"supplier is widened by its declaration and not by an id in a list",
"'eneco_eddy' when the declared product is day-ahead - so a supplier is "
"widened by its declaration and not by an id in a list",
near(_wide_named_dayahead, _wide, 1e-9)
and near(_normal_named_eddy, _normal, 1e-9))
check("...and the shipped plugin §9.2 names IS the one that declares REALTIME, "
"so the mechanism is pointed at the real product",
PROVIDERS["eneco_eddy"].capabilities().product is PriceProduct.REALTIME
# ⚠️ 'eneco_eddy' ABOVE IS NOW A RETIRED ID AND THE CHECK IS STRONGER FOR IT.
# It was chosen because it was the one plugin that really did declare REALTIME,
# so the check read "the name does not override the declaration". It declares
# nothing now, which makes the string a pure name with no product behind it -
# exactly the input an id-sniffing implementation would still widen.
check("...and NO shipped plugin declares REALTIME since TARIFF-07, so the "
"widening path is reached by declaration alone - kept deliberately, see "
"DECISIONS/TARIFF-07-withdrawing-three-plugins.md",
[p for p, c in PROVIDERS.items()
if c.capabilities().product is PriceProduct.REALTIME] == []
and uncertainty_of(PriceProduct.REALTIME)
== frozenset({Uncertainty.REALTIME_PRODUCT})
and uncertainty_of(PriceProduct.DAY_AHEAD) == frozenset())
@@ -858,19 +1101,57 @@ _cfg = derived(lambda: open(os.path.join(HERE, "config.yaml"),
# why, as this line does.
_OPT04_KEYS = ["arbitrage_enabled", "cycle_cost_threshold_eur",
"arbitrage_soc_floor", "arbitrage_soc_ceiling"]
check(f"the in-code mirror holds exactly TARIFF-02's, TARIFF-03's and OPT-04's "
f"keys - FCST-03 still added none of its own {_defaults}",
# TARIFF-15 added the fifth, and came here to say so as this comment instructs:
# `price_formula` is the `custom_formula` provider's only option - one JSON
# string carrying one contract's coefficients - and it lives in this dict and
# in config.yaml and nowhere else, which is the arrangement `static_profile`
# already has for the same reason.
_T15_KEYS = ["price_formula"]
# TARIFF-17 added the sixth, and comes here to say so as the comment above
# instructs. `price_taxes_profile` is the hour-varying spelling of
# `price_taxes_eur_kwh` - a period table of the TAXES component, for the Walloon
# and Brussels DSOs whose distribution tariff really does have a day and a night
# band (ORES 13,27 / 7,39 c EUR/kWh). It is an ADDITION and not a replacement:
# the scalar is untouched, every site on it is unaffected, and validate.py
# refuses the configuration that sets both. It lives in this dict and in
# config.yaml and nowhere else, which is `static_profile`'s arrangement for
# `static_profile`'s reason.
_T17_KEYS = ["price_taxes_profile"]
check(f"the in-code mirror holds exactly TARIFF-02's, TARIFF-03's, OPT-04's, "
f"TARIFF-15's and TARIFF-17's keys - FCST-03 still added none of its own "
f"{_defaults}",
_defaults == sorted(["price_provider", "price_markup_eur_kwh",
"price_taxes_eur_kwh", "export_price_eur_kwh",
"price_csv_path", "price_feed_url",
"price_horizon_h", "price_timeout_s",
"static_profile"] + _OPT04_KEYS))
# ...and the four are genuinely NEW, not a rename of something that was already
# there, or the line above would pass while the mirror silently lost a key.
check("...and OPT-04's four keys are additions rather than replacements: the "
"nine TARIFF-02/03 keys are all still present",
"static_profile"]
+ _OPT04_KEYS + _T15_KEYS + _T17_KEYS))
# ...and the later keys are genuinely NEW, not a rename of something that was
# already there, or the line above would pass while the mirror silently lost a
# key.
check("...and OPT-04's four, TARIFF-15's one and TARIFF-17's one are additions "
"rather than replacements: the nine TARIFF-02/03 keys are all still "
"present",
isinstance(_defaults, list)
and len(set(_defaults) - set(_OPT04_KEYS)) == 9)
and len(set(_defaults) - set(_OPT04_KEYS) - set(_T15_KEYS)
- set(_T17_KEYS)) == 9)
# ⚠️ AND THE ONE TARIFF-17 IS THE HOUR-VARYING TWIN OF IS STILL THERE, BY NAME.
# The line above counts nine, and a rename of `price_taxes_eur_kwh` to
# `price_taxes_profile` would also count nine. The scalar path being unchanged
# is this card's first acceptance criterion, and a count cannot grade it.
# ⚠️ THE LITERAL 0.11736 IS TARIFF-18's, AND IT IS TYPED HERE ON PURPOSE. The
# claim is "the scalar is still its own key, holding its own real number", and
# reading that number out of the dict under test would compare it against
# itself. TARIFF-18 raised it from 0.06; if it moves again this line goes red
# and whoever moved it comes here and says which card.
check("...and `price_taxes_eur_kwh` is still its own key beside the table, so "
"TARIFF-17 widened the framework rather than renaming the scalar - and "
"it still holds TARIFF-18's Flemish figure",
isinstance(_defaults, list)
and "price_taxes_eur_kwh" in _defaults
and "price_taxes_profile" in _defaults
and _price.PRICE_OPTION_DEFAULTS["price_taxes_eur_kwh"] == 0.11736
and _price.PRICE_OPTION_DEFAULTS["price_taxes_profile"] == "")
check("...and every one of them is still in config.yaml, which stays the "
"source of truth",
isinstance(_cfg, str)
File diff suppressed because it is too large Load Diff
+287 -109
View File
@@ -411,70 +411,91 @@ check("...and `phase` is NOT among them, because its value 'idle' IS the "
"phase" not in _m9_vanished)
shutil.rmtree(Path(_m9).parent, ignore_errors=True)
# ⚠️ AND `M12` BOUNDS THE **MOVED-NOT-LOST** RULE, WHICH NOTHING DID - AND ITS
# FIRST DRAFT DID NOT EITHER, WHICH IS THE WORSE FAILURE OF THE TWO.
# ⭐ `M12` IS BACK, AND THE CLAIM THAT KILLED IT WAS WRONG.
#
# The rule cited M3 as its bound, and the commit message cited it too. M3 drops
# `sign_reason` from `LEGACY_UNIT_KEYS`; M9, the one actually meant, vanishes
# only FLAT keys, so neither can satisfy the `unit/` gate the rule sits behind.
# THE BRANCH HAD NO CONTROL AT ALL.
# M12 is SAFETY-44's end-to-end control for the off-roster narrowing: a mutant
# makes one ROSTER unit's latch key vanish while a sibling holds the same
# value, and it must go red when the narrowing is removed. INSTR-01's first
# draft DELETED it, on the claim that the provenance clause made it
# unrestorable and that "NO FIXTURE CAN RESTORE IT, measured both ways".
#
# ⚠️ THE FIRST FIX FOR THAT WAS INERT, AND REVIEW MEASURED IT: run M12's mutant
# over `state-legacy.json` and the findings diff against the UNMUTATED tree is
# EMPTY. It "passed" because that fixture's STAGED leg is already exit 1 on
# pre-existing litter, and `"migrated unit/shed/" not in out` was trivially
# true because a pre-DEV-02 FLAT store has no shed latch to lose - after
# migration only the FIRST roster entry is ever latched. A control written
# against a fixture that cannot hold the failure is habit 4 wearing the clothes
# of its own fix, and it is WORSE than the honest "no control" it replaced,
# because it reads as checked.
# ⚠️ TWO MEASUREMENTS ARE NOT AN IMPOSSIBILITY PROOF, AND THIS ONE WAS FALSE.
# Both arrangements I measured were SAME-STEP shaped, and the structural
# argument - `run()` builds both snapshots from one state file, so "already
# latched" and "made by this transition" are incompatible - is TRUE OF
# SAME-STEP AND FALSE OF STAGED. `snapshot_over()` starts the after-leg over
# the file the BEFORE-leg wrote, so on the staged leg a transition-made
# destination is available and the two questions stop being incompatible.
# Review found the third arrangement I had generalised past.
#
# ⭐ SO THE FIXTURE CAME FIRST. `state-both-latched.json` is a store at
# SCHEMA_VERSION with `unit/garage/*` AND `unit/shed/*` both latched, by ONE
# site event so the reasons match too - the arrangement in which the
# suppression is most dangerous, and the one no existing fixture could produce.
# The mutant then makes ONE roster unit's latch key VANISH while its sibling's
# survives at the same value.
# HOW IT WORKS. The fixture is `state-legacy.json` plus a second roster unit
# already latched. The mutant makes `shed` POP its own latch key and come up
# untripped - a truer SAFETY-02 loss than the original, which left `tripped`
# True in memory. On the staged leg, losing shed's latch makes
# `_grade_store_migration()` fall through `on_roster_latched` and RE-ARM
# `unit/garage/sign_tripped`, which is absent from `before.data` - so the
# destination is transition-made and C7 lets it through. The off-roster
# `inverter-a` litter is present in BOTH snapshots on this leg, so it never
# vanishes and never claims the destination, and C6 stays out of the way.
#
# THE DECISIVE MEASUREMENT, which is what the previous draft could not make:
# reverting the `vanished_unit not in after.units` narrowing - restoring exactly
# the rule review proved hides a real loss - turns this RED. Before, it was
# 28/28 green.
# ⚠️ AND WHAT ACTUALLY BROKE WAS THE PRECONDITION, NOT THE CONTROL. M12's
# `exit == EXIT_OK` baseline does not survive a legacy-shaped fixture. That
# precondition is the one SAFETY-44's own commit called "belt-and-braces,
# load-bearing only if the exit-0 requirement is ever relaxed", saying that
# what closed the first draft's gap is `_m12_new` BEING A SET DIFFERENCE. A
# stable dirty baseline is fine for a set difference. So the control was
# deleted on the strength of a precondition the same commit had already
# characterised as not load-bearing - which is a worse mistake than the
# original inert check, because it removed coverage rather than failing to add
# it.
_M12_OLD = 'tripped=bool(data.get(self.key("sign_tripped"), False)),'
_M12_NEW = ('tripped=bool(data.pop(self.key("sign_tripped"), False)'
+ chr(10) + ' if self.name == "shed"'
+ chr(10) + ' else data.get(self.key("sign_tripped"), False)),')
# ⚠️ THE MUTANT POPS **AND COMES UP UNTRIPPED**, which is the whole point and
# which the first draft got wrong: `data.pop(k, False)` RETURNS the value, so
# shed stayed latched in memory, `_grade_store_migration()` saw
# `on_roster_latched` and never re-armed garage - no transition-made
# destination, and the finding was reported whether or not C5 existed. The
# control looked restored and discriminated nothing. `and False` is the
# difference between "the key is gone" and "the LATCH is gone", and only the
# second reaches the re-arm this arrangement needs.
_M12_NEW = ('tripped=bool(data.pop(self.key("sign_tripped"), False) and False'
+ chr(10) + ' if self.name == "shed" else'
+ chr(10) + ' data.get(self.key("sign_tripped"), False)),')
def m12_findings(tree):
"""The finding SET of a staged-leg run over the legacy+shed fixture."""
code, out = run(Path(tree), "state-legacy-plus-shed.json", "roster.yaml",
"--paths", "staged")
return code, set(re.findall(r"⚠️ ([A-Z ]+): (\S+)", out)), out
_m12 = mutant_tree([("app/inverter.py", _M12_OLD, _M12_NEW)])
_m12_code, _m12_out = run(Path(_m12), "state-both-latched.json", "roster.yaml")
_m12_clean_code, _m12_clean_out = run(HERE, "state-both-latched.json", "roster.yaml")
_m12_new = (set(re.findall(r"⚠️ ([A-Z ]+): (\S+)", _m12_out))
- set(re.findall(r"⚠️ ([A-Z ]+): (\S+)", _m12_clean_out)))
print(f" both-latched store, shipped -> exit {_m12_clean_code}; "
f"one unit's latch key deleted -> exit {_m12_code}")
print(f" findings the mutation ADDS: {sorted(_m12_new)}")
# ⚠️ THE SECOND CONJUNCT IS IMPLIED BY THE FIRST TODAY - on a clean run the
# finding regex matches nothing - and the commit message that introduced it
# overclaimed by saying the exit code "cannot say" it. It is belt-and-braces,
# load-bearing only if the exit-0 requirement is ever relaxed. WHAT ACTUALLY
# CLOSED THE FIRST DRAFT'S GAP IS `_m12_new` BEING A SET DIFFERENCE: the
# mutation must ADD a finding, which is the check the narrowing's removal turns
# red. Said plainly here because the wrong half was credited once already.
check("⚠️ PRECONDITION: the fixture is clean on the shipped tree AND does not "
"already report the finding this mutation exists to produce - the second "
"is implied by the first today and kept as a belt-and-braces guard",
_m12_clean_code == RR.EXIT_OK
and not any(k == "unit/shed/sign_tripped"
for _, k in re.findall(r"⚠️ ([A-Z ]+): (\S+)", _m12_clean_out)))
check("⭐ M12: a ROSTER unit's latch key that VANISHES is a finding even though "
"a sibling on the same roster still holds True - the moved-not-lost rule "
"suppresses a RENAME (a key leaving a name this roster does not carry), "
"never a LOSS",
_m12_code == RR.EXIT_FINDING
and any(k == "unit/shed/sign_tripped" for _, k in _m12_new))
check("...and the mutation ADDS findings rather than merely coinciding with "
"ones the fixture already had - an empty diff here is what made the "
"previous draft inert",
bool(_m12_new))
_m12_clean_code, _m12_clean, _ = m12_findings(HERE)
_m12_code, _m12_mut, _ = m12_findings(Path(_m12))
_m12_new = _m12_mut - _m12_clean
print(f" staged leg, legacy+shed: shipped exit {_m12_clean_code}, "
f"one roster unit's latch popped -> exit {_m12_code}")
# ⚠️ THE PRINTED SET IS LOSSIER THAN IT LOOKS: the regex captures the first
# TOKEN of each finding's text, so two findings of one kind collapse into one
# tuple. Harmless here - the difference is one-directional and the pair M12
# asserts on is exact - but do not read this line as an inventory.
print(f" findings the mutation ADDS (first token only): {sorted(_m12_new)}")
check("⚠️ PRECONDITION: the baseline is STABLE and does not already carry the "
"finding this mutation exists to produce. It is NOT clean - a legacy "
"fixture never is - and the SET DIFFERENCE is what makes that fine, "
"which is the sentence the deletion should have read before acting on it",
not any(k == "unit/shed/sign_tripped" for _, k in _m12_clean))
# ⚠️ AND IT MATCHES THE **KIND** AS WELL AS THE KEY, which SAFETY-44's original
# did not. Review measured it: renaming the finding kind in `diff()` from
# "VANISHED STORE KEY" to "GONE STORE KEY" left M12 GREEN. A vanished latch
# quietly downgraded to a softer heading is exactly the shape part 3 requires
# every mutation to be graded against - "the SPECIFIC finding it should produce"
# - and this file was not holding itself to its own rule.
check("⭐ M12 RESTORED: a ROSTER unit's latch key that vanishes is a "
"VANISHED STORE KEY finding even though a sibling holds True - and this "
"is the STAGED leg, where `snapshot_over()` supplies a transition-made "
"destination, so the provenance clause does not foreclose the narrowing",
("VANISHED STORE KEY", "unit/shed/sign_tripped") in _m12_new)
shutil.rmtree(Path(_m12).parent, ignore_errors=True)
@@ -715,60 +736,39 @@ check("...and no leg's text has swallowed the CLOSING SUMMARY, which repeats "
"red whenever any leg was",
not any("FINDING(S) - this transition is not clean" in v
for v in _L.values()))
# ⚠️ TWO FURTHER HOLES IN THIS RULE ARE **KNOWN, MEASURED BY REVIEW, AND NOT
# FIXED HERE** - deliberately, and this paragraph is the whole of the entry.
# ⚠️ THIS BLOCK USED TO SAY THOSE HOLES WERE KNOWN AND NOT FIXED, AND IT IS
# NOW STALE IN THE OTHER DIRECTION - INSTR-01 fixed them, and left the
# paragraph saying "BOTH ARE REVERTED OUT OF THIS CARD, because THIS SUITE
# CANNOT MEASURE EITHER ONE" describing a tree that no longer exists. Review
# found it. A comment that outlives its subject is the same defect as a
# citation that outlives its check.
#
# INJECTIVITY. `moved_to` is "any machine-read key with the same bare name
# and the same value", so N vanished keys can all claim the SAME destination.
# Review measured a rename carrying `inverter-a` -> `garage` correctly while
# DROPPING `inverter-b` -> `shed`: both reported as migrated to
# `unit/garage/sign_tripped`, zero store-key findings, `shed` unlatched.
# WHERE EACH ONE NOW LIVES, and every clause of `moved_to` has a row in 5e:
#
# PROVENANCE. A destination is accepted regardless of whether the transition
# put the value there, so a roster unit already latched in its own right is a
# free suppression slot. Review measured a build deleting `old-a` while
# renaming `old-b` -> `shed`: exit 0, NO FINDINGS, `old-a` reported as having
# moved into a key that already held True.
# INJECTIVITY (C6) R5, and removing the clause flips exactly R5
# PROVENANCE (C7) R3, likewise
# SAME VALUE (C3) R4
# SAME BARE KEY(C2) R7 - which had NO row until review deleted the clause
# and the whole 28-file suite stayed green
# MACHINE-READ (C1) R6
# OFF-ROSTER (C5) R2, and M12 end-to-end on the staged leg
# PREFIX GATE (C4) NO 5e ROW, deliberately - `vanished_unit is not None`
# is reached end-to-end by part 3's M9, and a synthetic
# row would duplicate that rather than add separability.
# Listed because C5 lives in the same `if` and omitting
# one guard while naming the other reads as an oversight.
#
# ATTRIBUTION. Which vanished key gets a destination is decided by
# `sorted()`, so on a build renaming `inverter-b` -> `garage` while DELETING
# `inverter-a`, the deleted key sorts first and takes it: the report names the
# DELETED key as migrated and the one that really moved as VANISHED. The run
# is still exit 1, so this is a wrong sentence rather than a hidden loss - but
# an operator chasing the named key looks at the wrong inverter.
#
# VALUE MATCHING. `and after.data.get(k) == value` survives the suite and is
# load-bearing: without it the rule degenerates to BARE-NAME matching, and
# review measured a deleted `unit/old-x/sign_reason` reported as having
# migrated into a `sign_reason` holding a COMPLETELY DIFFERENT SENTENCE.
# Essentially every off-roster vanished `unit/*` key would be suppressed.
#
# ⚠️ BOTH FIXES ARE ONE LINE AND BOTH WERE WRITTEN, AND BOTH ARE REVERTED OUT
# OF THIS CARD, because THIS SUITE CANNOT MEASURE EITHER ONE. Three attempts:
# a check over part 5's destinations passed with the whole injectivity
# mechanism deleted (its two destinations are `sign_reason` and `sign_tripped`
# - different bare keys, which can never contend); a dedicated fixture
# discriminated nothing (both snapshots read one state file, so nothing
# vanishes and there are no migrated lines either way); and rebuilding M12's
# fixture to reach the provenance case disarmed M12 for the NARROWING, so all
# three clause-removals came back green at once.
#
# ⚠️ THAT LAST ONE IS THE REASON THIS IS A SPLIT AND NOT A JUDGEMENT CALL: a
# later fix to the same function silently disarmed the control for an earlier
# one, and I only saw it because the removal was re-driven. Shipping two more
# unmeasured clauses into the instrument, on a card whose subject is checks
# that cannot reach what they name, would be the fourth instance in one PR.
# They go on their own card, with review's measurements as the starting
# evidence and a fixture built BEFORE the fix.
#
# ⚠️ ALL FOUR ARE CLAUSES OF **ONE COMPREHENSION**, WHICH IS THE STRUCTURAL
# POINT AND NOT A LIST OF CHORES. `diff()`'s `moved_to` now carries four
# conditions with four different reasons to exist, against one fixture set that
# can reach only some of them at a time - so rebuilding a fixture to reach
# clause N moves the others. That is exactly how (3) happened, and it is what
# INSTR-01 has to design against rather than patch around.
#
# What IS measured here is the narrowing, by M12 above, both ways.
# ⚠️ ONE HOLE IS STILL OPEN AND IS **UNMEASURED**, stated so nobody reads
# `sorted()` as guarded: ATTRIBUTION. Which vanished key gets a destination is
# decided by `sorted()` order, so on a build renaming `inverter-b` -> `garage`
# while DELETING `inverter-a`, the deleted key sorts first and takes it - the
# report names the DELETED key as migrated and the one that really moved as
# VANISHED. Review measured that replacing `sorted(` with `list(` leaves the
# suite green: every 5e case has at most one candidate destination, so no row
# can see ordering. The run is still exit 1, so this is a WRONG SENTENCE rather
# than a hidden loss - an operator chasing the named key looks at the wrong
# inverter. Fixing it needs the destination's provenance rather than
# first-come, which is more than a rename diff currently knows.
check("⚠️ and the key that MOVED is not reported as lost: the before-snapshot "
"holds `unit/inverter-a/sign_tripped = True` and the after-snapshot "
"holds it under `garage`, three lines apart in the same report. A "
@@ -873,6 +873,184 @@ check("the SAME state file onto a roster whose first entry IS `inverter-a` "
code_ok == RR.EXIT_OK and "LATCH CAME UP DISARMED" not in out_ok)
# --------------------------------------------------------------------------- #
print()
print("5e. INSTR-01: `diff()`'s suppression rule, ONE FIXTURE PER CLAUSE")
# --------------------------------------------------------------------------- #
# ⚠️ THIS SECTION EXISTS BECAUSE SAFETY-44 PRODUCED THREE INERT CONTROLS FOR
# THIS ONE RULE, EACH INERT DIFFERENTLY:
#
# 1. a check over part 5's real destinations passed with the whole mechanism
# deleted - its two destinations are `sign_reason` and `sign_tripped`,
# DIFFERENT bare keys, which can never contend;
# 2. a dedicated end-to-end fixture discriminated nothing - both snapshots
# read ONE state file, so nothing vanishes and there are no migrated lines
# with the clause or without it;
# 3. rebuilding M12's fixture to reach a new clause DISARMED M12 for an older
# one, and all three clause-removals went green at once.
#
# ⭐ (3) IS THE STRUCTURAL FINDING AND IT IS WHY THIS SECTION IS SHAPED THIS WAY.
# `moved_to` is ONE comprehension carrying FIVE conditions with five different
# reasons to exist, and an END-TO-END fixture cannot separate them: both
# snapshots come from one state file, so the arrangement that reaches clause N
# forecloses clause M. Rebuilding the fixture to reach one moves the others.
#
# So the rule is graded at the level it is written at - `diff()` on constructed
# Snapshots - with ONE CASE PER CLAUSE. Nothing here is a fiction: the inputs
# are the same three fields `diff()` reads (`data`, `machine_read`, `units`),
# and the end-to-end controls above - M12 on the STAGED leg, and part 5 -
# still grade the same rule through a real start. This is the separability
# those cannot have, not a replacement for them.
CLAUSE_CASES = {
# name: (units, machine_read, before, after)
"R1-rename-legit": (
["garage"], ["unit/garage/sign_tripped"],
{"unit/old/sign_tripped": True}, {"unit/garage/sign_tripped": True}),
"R2-narrowing": (
["garage", "shed"], ["unit/garage/sign_tripped", "unit/shed/sign_tripped"],
{"unit/shed/sign_tripped": True}, {"unit/garage/sign_tripped": True}),
"R3-provenance": (
["garage"], ["unit/garage/sign_tripped"],
{"unit/old/sign_tripped": True, "unit/garage/sign_tripped": True},
{"unit/garage/sign_tripped": True}),
"R4-value": (
["garage"], ["unit/garage/sign_reason"],
{"unit/old/sign_reason": "X-unique"},
{"unit/garage/sign_reason": "Y-different"}),
"R5-injectivity": (
["garage"], ["unit/garage/sign_tripped"],
{"unit/old-a/sign_tripped": True, "unit/old-b/sign_tripped": True},
{"unit/garage/sign_tripped": True}),
"R6-machine-read": (
["garage"], [],
{"unit/old/sign_tripped": True}, {"unit/garage/sign_tripped": True}),
# ⚠️ R7 EXISTS BECAUSE THE BARE-KEY MATCH HAD NO ROW AT ALL, and review
# deleted it and left the whole 28-file suite GREEN. The rule's own comment
# says "THE TEST IS NOT 'the name appears somewhere'. It is: the SAME bare
# key" - and nothing measured that sentence. It is load-bearing: a vanished
# SAFETY-02 `sign_tripped` was suppressed by a SAFETY-42
# `delivery_tripped` on a roster unit, i.e. by a DIFFERENT FAILSAFE's key.
# `unit/*/delivery_tripped` is machine-read, so this is reachable on any
# real two-latch store rather than a contrivance.
"R7-bare-key": (
["garage"], ["unit/garage/delivery_tripped"],
{"unit/old/sign_tripped": True},
{"unit/garage/delivery_tripped": True}),
}
# What the SHIPPED rule must answer. `migrated` is the suppression firing.
CLAUSE_EXPECT = {
"R1-rename-legit": ([], ["unit/old/sign_tripped"]),
"R2-narrowing": (["unit/shed/sign_tripped"], []),
"R3-provenance": (["unit/old/sign_tripped"], []),
"R4-value": (["unit/old/sign_reason"], []),
"R5-injectivity": (["unit/old-b/sign_tripped"], ["unit/old-a/sign_tripped"]),
"R6-machine-read": (["unit/old/sign_tripped"], []),
"R7-bare-key": (["unit/old/sign_tripped"], []),
}
# ⚠️ THE DRIVER **IMPORTS** THE TREE UNDER TEST AND THE TABLE STAYS HERE, so
# expectation and implementation sit on opposite sides of the subprocess and
# a mutation to the tool cannot also move the answer it is graded against.
# (It is WRITTEN to a tempdir - see `clause_table()`. The first draft of this
# sentence said "written into the tree", which is the read-only-CI bug the
# same commit says it fixed, stated as if it were the design.)
CLAUSE_DRIVER = """
import json, sys
import rehearse_roster as RR
spec = json.load(sys.stdin)
out = {}
for name, case in spec.items():
units, mread, before, after = case
b = object.__new__(RR.Snapshot); a = object.__new__(RR.Snapshot)
for snap, data in ((b, before), (a, after)):
snap.label = name
snap.data = dict(data)
snap.units = list(units)
snap.machine_read = set(mread)
snap.entities = {}
snap.latched = {u: False for u in units}
snap.latch_reason = {u: "" for u in units}
snap.flat_tripped = False
snap.history = {}
snap.read = set()
snap.stranded_at_default = []
snap.vanished_at_default = []
snap.vanished_but_migrated = []
f = RR.diff(b, a)
out[name] = [
sorted(x.text.split(" = ")[0] for x in f
if x.kind == "VANISHED STORE KEY"),
sorted(t.split(" = ")[0] for t in a.vanished_but_migrated),
]
print(json.dumps(out))
"""
def clause_table(tree) -> dict:
"""Run CLAUSE_CASES through `diff()` in `tree`, as the tool would see it."""
# ⚠️ THE DRIVER GOES IN A TEMPDIR, NOT IN THE TREE. `checks / unit` mounts
# the checkout READ-ONLY, so writing beside the tool works on a dev box and
# is `OSError: Read-only file system` in CI - a difference that would have
# turned this section red on every PR and green for whoever wrote it.
tools = Path(tree) / "tools"
tmp = tempfile.mkdtemp(prefix="clause-drv-")
drv = Path(tmp) / "_clause_driver.py"
drv.write_text(CLAUSE_DRIVER, encoding="utf-8")
proc = subprocess.run(
[sys.executable, str(drv)],
input=json.dumps(CLAUSE_CASES), capture_output=True, text=True,
cwd=tmp,
env={**os.environ, "PYTHONDONTWRITEBYTECODE": "1",
"PYTHONPATH": f"{tools}{os.pathsep}{Path(tree)}"})
shutil.rmtree(tmp, ignore_errors=True)
assert proc.returncode == 0, (
"the clause driver did not run, so every row below would be "
f"fabricated:\n{proc.stdout}\n{proc.stderr}")
return {k: (v[0], v[1]) for k, v in json.loads(proc.stdout).items()}
_ct = clause_table(HERE)
for _name in sorted(CLAUSE_CASES):
_got, _want = _ct[_name], CLAUSE_EXPECT[_name]
print(f" {_name:<18} vanished={_got[0]} migrated={_got[1]}")
check(f"{_name}: the shipped rule answers as specified", list(_got) == [list(_want[0]), list(_want[1])])
check("⚠️ ...and the LEGITIMATE case is still suppressed - a rule that reports "
"every rename is a rule an operator turns off, which is the failure "
"CONTROL-C exists to prevent",
_ct["R1-rename-legit"][1] == ["unit/old/sign_tripped"])
# ⭐ AND EACH CLAUSE REMOVAL MUST FLIP **EXACTLY ITS OWN ROW**. That second half
# is the acceptance criterion this card was filed with: a removal that flips
# nothing is an unmeasured clause, and a removal that flips someone else's row
# means the cases are not separable after all - which is (3) above, caught.
CLAUSE_MUTATIONS = [
("C2 same bare key", "R7-bare-key",
' if k.rsplit("/", 1)[-1] == bare and after.data.get(k) == value',
" if True and after.data.get(k) == value"),
("C1 machine_read", "R6-machine-read",
" k for k in after.machine_read",
" k for k in after.data"),
("C3 same value", "R4-value",
" and after.data.get(k) == value\n", "\n"),
("C5 off-roster", "R2-narrowing",
" if vanished_unit is not None and vanished_unit not in after.units \\\n"
" and moved_to:",
" if vanished_unit is not None and moved_to:"),
("C6 injectivity", "R5-injectivity",
" and k not in claimed)", ")"),
("C7 provenance", "R3-provenance",
" and before.data.get(k) != value", " and True"),
]
for _label, _owns, _old, _new in CLAUSE_MUTATIONS:
_t = mutant_tree([("tools/rehearse_roster.py", _old, _new)])
_mt = clause_table(_t)
_flipped = sorted(n for n in CLAUSE_CASES if _mt[n] != _ct[n])
print(f" {_label:<16} removed -> rows that changed: {_flipped}")
check(f"⭐ removing {_label} flips {_owns} - the clause is load-bearing and "
f"this row is what measures it",
_flipped == [_owns])
shutil.rmtree(Path(_t).parent, ignore_errors=True)
# --------------------------------------------------------------------------- #
print()
print("6. THE REVERSE DIRECTION: the retreat, and the return from it")
+220 -9
View File
@@ -59,6 +59,7 @@ is TEST-15, and it has already turned one live mutation into a false survivor.
Read MUTATION.md before you report a survivor.
"""
import ast
import gzip
import json
import sys
@@ -69,9 +70,11 @@ from app.price import (
Absence, PricePoint, PriceProduct, PriceRead, arbitrage_allowed, brussels,
plan_is_provisional, required_spread, uncertainty_of,
)
from app.price_history import NoHistory
from app.static_tariff import (
MEAN_WINDOW_DAYS, ProfileInvalid, StaticProfile, TariffPlan,
parse_static_profile, resolve_tariff, static_plan, static_status,
MEAN_WINDOW_DAYS, ProfileInvalid, StaticProfile, TAXES_PROFILE_OPTION,
TariffPlan, parse_static_profile, parse_taxes_profile, resolve_tariff,
static_plan, static_status,
)
from app.validate import validate_startup_config
@@ -86,6 +89,16 @@ def check(name, cond):
fails.append(name)
def _reason_of(fn):
"""The message fn() raised, or "" - so a check can assert WHICH rule
refused rather than only that something did (MUTATION.md habit 6)."""
try:
fn()
except Exception as e: # noqa: BLE001
return str(e)
return ""
def raises(fn, exc):
"""True only if fn() raises EXACTLY `exc`. A different exception is False -
'it was refused' and 'it broke on the way to being refused' must not grade
@@ -742,11 +755,14 @@ def run(coro):
return asyncio.run(coro)
# ⚠️ NO `price_feed_url` (TARIFF-09): an override is used verbatim and the
# plugin appends no date range to it, so setting one - even to the feed's own
# address - exercises the override path rather than the shipped default.
_cfg = opts(TWO_PERIOD, price_provider="generic_dayahead",
price_feed_url="https://api.energy-charts.info/price?bzn=BE",
price_timeout_s=1, price_markup_eur_kwh=0.03,
price_taxes_eur_kwh=0.06)
_fb = run(resolve_tariff(_cfg, THU, THU + timedelta(days=1), transport=_dead))
_fb = run(resolve_tariff(_cfg, THU, THU + timedelta(days=1), transport=_dead,
history=NoHistory()))
check("a dead feed falls back to the static profile rather than to nothing",
_fb.origin == "fallback" and _fb.has_prices)
check("...with the SAME prices the configured-product path produces, because "
@@ -769,15 +785,27 @@ check("...and the provisional plan has no cheap window either: §8.3 forbids an
not any(_fb.cheap_now(at_local(THU, h)) for h in range(24)))
_fb_none = run(resolve_tariff(opts(None, price_provider="generic_dayahead",
price_timeout_s=1),
THU, THU + timedelta(days=1), transport=_dead))
THU, THU + timedelta(days=1), transport=_dead,
history=NoHistory()))
check("a dead feed with NO static_profile has nothing to fall back to, and "
"says that rather than inventing a tariff",
_fb_none.origin == "none" and not _fb_none.has_prices)
# A live feed still wins over the profile.
_body = json.dumps({"data": [
{"start": (THU + timedelta(hours=i)).isoformat(), "price_eur_mwh": 40 + i}
for i in range(24)]}).encode()
# ⚠️ COLUMNAR (TARIFF-08). `api.energy-charts.info` publishes parallel
# `unix_seconds`/`price` arrays and no `data` key; the `{"data": [...]}` body
# this used to build was invented by the parser it was grading, which is how a
# provider that could never parse a live response shipped green. Real captured
# bodies are replayed in test_price.py section 2b - the prices here are
# synthetic, the SHAPE is not.
_body = json.dumps({
"unix_seconds": [int((THU + timedelta(hours=i)).timestamp())
for i in range(24)],
"price": [40 + i for i in range(24)],
"unit": "EUR / MWh",
"license_info": "CC BY 4.0 (creativecommons.org/licenses/by/4.0) "
"from Bundesnetzagentur | SMARD.de",
"deprecated": False}).encode()
async def _live(url, timeout_s):
@@ -785,7 +813,7 @@ async def _live(url, timeout_s):
_live_plan = run(resolve_tariff(_cfg, THU, THU + timedelta(days=1),
transport=_live))
transport=_live, history=NoHistory()))
check("a live feed is used in preference to the static profile",
_live_plan.origin == "provider"
and _live_plan.product is PriceProduct.DAY_AHEAD)
@@ -1114,6 +1142,189 @@ check("...and the table really does exercise both rungs and the allowing path, "
{want for _n, _p, want in _TABLE9} == {"flat", "firm-pair", "allowed"})
# --------------------------------------------------------------------------- #
print()
print("10. TARIFF-17 - ONE period grammar, not two. A second tiling rule is a "
"second set of DST bugs")
# ⚠️ THE ACCEPTANCE CRITERION THIS GRADES, and why it needs two checks rather
# than one. `price_taxes_profile` carries the TAXES component as a period table
# so that a Walloon day/night distribution band can reach the optimiser. The
# obvious way to build it is a second parser next to this one - and the two
# would agree on every ordinary day and diverge on the 23- and 25-hour days,
# where nobody looks and where §11.4 makes the divergence quiet.
#
# So the claim is an IDENTITY ("it is this grammar"), and MUTATION.md habit 10
# says pin an identity structurally and a meaning behaviourally. Both are here:
# 10a the grammar family is DEFINED IN EXACTLY ONE MODULE, computed by
# parsing every file under app/ rather than by a list somebody typed;
# 10b the two options produce the SAME REFUSAL SENTENCES, derived from each
# other, which is what a fork would break first.
print()
print("10a. the grammar is defined in exactly one module under app/, and the "
"scope is COMPUTED")
# ⚠️ A HAND-WRITTEN SCOPE IS THE SAME DEFECT AS A HAND-WRITTEN FIXTURE
# (habit 12): it passes while the copy sits in the file nobody added to the
# list. Every `app/*.py` is parsed; the family is taken from THIS module's own
# top-level names, so it cannot drift from what the grammar actually is.
#
# ⚠️ AND THE FAMILY LIST HAS ONE DELIBERATE OMISSION, STATED HERE RATHER THAN
# ROUNDED OFF (habit 13). `_days` is NOT in it, because `maintenance.py` has a
# `_days` of its own - a NAMESAKE, not a copy - and the check below would go
# red on it and be deleted by whoever hit it. The namesake is asserted to BE a
# namesake, by signature, immediately afterwards, so the omission is graded
# rather than assumed.
_T17_FAMILY = ("Period", "StaticProfile", "ProfileInvalid", "_check_tiling",
"_raise_gap", "_raise_overlap", "_minutes", "_price",
"parse_static_profile", "DAY_NAMES", "MINUTES_PER_DAY",
"_DAY_INDEX", "_DAY_ALIASES", "SUPPORTED_TZ")
def _t17_defs(path):
"""Top-level names a module DEFINES - classes, functions and assignments."""
tree = ast.parse(path.read_text(encoding="utf-8"))
out = set()
for node in tree.body:
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef,
ast.ClassDef)):
out.add(node.name)
elif isinstance(node, ast.Assign):
out |= {t.id for t in node.targets if isinstance(t, ast.Name)}
return out
_T17_APP = sorted((HERE / "app").glob("*.py"))
_T17_WHERE = {name: sorted(p.name for p in _T17_APP if name in _t17_defs(p))
for name in _T17_FAMILY}
for _n, _w in sorted(_T17_WHERE.items()):
print(f" {_n:<22} defined in {_w}")
check(f"every one of the {len(_T17_FAMILY)} grammar names is defined in "
f"EXACTLY ONE of the {len(_T17_APP)} modules under app/ - a copy of "
f"`_check_tiling` in price.py would take this red",
all(len(w) == 1 for w in _T17_WHERE.values()))
check("...and that module is static_tariff.py in every case, so the grammar "
"did not move house either",
{w[0] for w in _T17_WHERE.values() if w} == {"static_tariff.py"})
check("...and the pin is not vacuous: each of those names really is a "
"top-level definition, so the scan found something rather than nothing",
all(w for w in _T17_WHERE.values()) and len(_T17_APP) > 5)
def _t17_sig(module, name):
tree = ast.parse((HERE / "app" / module).read_text(encoding="utf-8"))
fn = next(n for n in tree.body
if isinstance(n, ast.FunctionDef) and n.name == name)
return [a.arg for a in fn.args.args]
_T17_DAYS = (_t17_sig("static_tariff.py", "_days"),
_t17_sig("maintenance.py", "_days"))
print(f" _days: static_tariff{_T17_DAYS[0]} vs maintenance{_T17_DAYS[1]}")
check(f"...and the ONE name held out of the family is held out honestly: "
f"`maintenance._days{tuple(_T17_DAYS[1])}` renders an age in words and "
f"is not a second copy of the grammar's "
f"`_days{tuple(_T17_DAYS[0])}` day selector",
_T17_DAYS[0] != _T17_DAYS[1] and _T17_DAYS[0] == ["value", "field"]
and _T17_DAYS[1] == ["n"])
# ⚠️ WHAT THIS PIN CANNOT SEE, WITH ITS SHAPE (habit 13). It grades WHERE the
# grammar is defined, never WHAT it means: `_check_tiling` could be rewritten
# in place to accept a gap and this would still pass. Section 5 and 10b are
# what grade the meaning. It also covers `app/` only, and a second grammar
# living under `tools/` or in a check file is outside it.
_T17_BODY = ast.parse(
(HERE / "app" / "static_tariff.py").read_text(encoding="utf-8"))
_T17_WRAP = next(
(n for n in _T17_BODY.body
if isinstance(n, ast.FunctionDef) and n.name == "parse_taxes_profile"),
None)
_T17_CALLS = ([ast.unparse(n.func) for n in ast.walk(_T17_WRAP)
if isinstance(n, ast.Call)] if _T17_WRAP else [])
check(f"...and `parse_taxes_profile` is a WRAPPER and not a fork: its whole "
f"body calls {_T17_CALLS}, pinned on the AST rather than on the source "
f"text, because `\"parse_static_profile\" in src` is true of a copy that "
f"merely mentions it",
_T17_CALLS == ["parse_static_profile"])
print()
print("10b. and the two options give the SAME refusals - derived from each "
"other, not typed twice")
# ⚠️ EVERY EXPECTED SENTENCE HERE IS THE OTHER OPTION'S OWN SENTENCE with the
# key substituted. Nothing is transcribed, so a fork that reworded one refusal
# - or that quietly stopped refusing something - is red on the row it broke and
# not on all of them.
_T17_BAD = [
("a gap", {"periods": [{"name": "d", "from": "07:00", "to": "22:00",
"days": "all", "eur_per_kwh": 0.13}]}),
("an overlap", {"periods": [
{"name": "a", "from": "00:00", "to": "24:00", "days": "all",
"eur_per_kwh": 0.13},
{"name": "b", "from": "07:00", "to": "22:00", "days": "all",
"eur_per_kwh": 0.07}]}),
("a bad HH:MM", {"periods": [{"name": "d", "from": "7.5", "to": "22:00",
"days": "all", "eur_per_kwh": 0.13}]}),
("a bad day", {"periods": [{"name": "d", "from": "00:00", "to": "24:00",
"days": "funday", "eur_per_kwh": 0.13}]}),
("missing days", {"periods": [{"name": "d", "from": "00:00", "to": "24:00",
"eur_per_kwh": 0.13}]}),
("a zero-length window", {"periods": [
{"name": "d", "from": "07:00", "to": "07:00", "days": "all",
"eur_per_kwh": 0.13}]}),
("no periods list", {"timezone": "europe/brussels"}),
("an empty periods list", {"periods": []}),
("a foreign timezone", {"timezone": "utc", "periods": []}),
("periods that are not a list", {"periods": 3}),
("a row that is not an object", {"periods": ["dag"]}),
("invalid JSON", "{not json"),
("a 24:00 start", {"periods": [{"name": "d", "from": "24:00", "to": "07:00",
"days": "all", "eur_per_kwh": 0.13}]}),
]
_T17_ROWS = []
for _what, _bad in _T17_BAD:
_a = _reason_of(lambda: parse_static_profile(_bad))
_b = _reason_of(lambda: parse_taxes_profile(_bad))
_T17_ROWS.append((_what, _a, _b,
_a.replace("static_profile", TAXES_PROFILE_OPTION) == _b))
for _what, _a, _b, _same in _T17_ROWS:
print(f" {'ok ' if _same else 'XX '}{_what:<28} {_b[:74]}")
check(f"all {len(_T17_ROWS)} malformations are refused IDENTICALLY under both "
f"options, sentence for sentence, once the key name is substituted - "
f"which is what one parser means and a fork could not hold",
all(row[3] for row in _T17_ROWS))
check("...and every one of them really did refuse, or 'the two agree' would be "
"two empty strings agreeing",
all(row[1] and row[2] for row in _T17_ROWS))
check("...and the sentences are not all the same sentence, or the row above "
"would be one comparison repeated 13 times",
len({row[2] for row in _T17_ROWS}) == len(_T17_ROWS))
# ⚠️ THE ONE REFUSAL THAT IS DELIBERATELY DIFFERENT, and it is different
# because the two options carry different QUANTITIES in the same grammar. A
# period without `eur_per_kwh` gets a sentence naming what was wanted, and an
# operator who typed an ALL-IN price into the taxes table is out by an energy
# price with nothing reporting it.
_T17_NOPRICE = {"periods": [{"name": "d", "from": "00:00", "to": "24:00",
"days": "all"}]}
_T17_M_S = _reason_of(lambda: parse_static_profile(_T17_NOPRICE))
_T17_M_T = _reason_of(lambda: parse_taxes_profile(_T17_NOPRICE))
print(f" static_profile : {_T17_M_S[-96:]}")
print(f" taxes table : {_T17_M_T[-96:]}")
check("the 'you left the number out' refusal is the one sentence the two do "
"NOT share, and each names its own quantity: an all-in price for "
"static_profile, the taxes component alone for the table",
"an all-in eur_per_kwh" in _T17_M_S
and "an all-in eur_per_kwh" not in _T17_M_T
and "TAXES component alone" in _T17_M_T
and "NOT an all-in price" in _T17_M_T
and _T17_M_S.replace("static_profile", TAXES_PROFILE_OPTION) != _T17_M_T)
check("...and both still name the ROW the operator has to go and fix, which is "
"the half they do share",
"periods[1].eur_per_kwh is missing" in _T17_M_S
and "periods[1].eur_per_kwh is missing" in _T17_M_T)
# --------------------------------------------------------------------------- #
print()
if fails:
+359 -14
View File
@@ -2,8 +2,8 @@
python3 test_tariff.py
Ten parts. What each is for, and - where it matters - what it is deliberately
NOT allowed to be tested at:
Twelve parts. What each is for, and - where it matters - what it is
deliberately NOT allowed to be tested at:
1. THE BLOCK ARITHMETIC, against a worked example written out segment by
segment. The fixture is chosen to be non-degenerate on three axes at once:
@@ -33,6 +33,17 @@ NOT allowed to be tested at:
proven equal to the tier it derives from, and proven never to be lowered.
9. AC 3, ON THE REPLAY DATA THAT ACTUALLY EXISTS. See the honesty note below.
10. THE WIRING, through the real `Controller` and its real status page.
11. TARIFF-13: THE COVERAGE FIGURE PRINTED BESIDE THE PEAK, across a REAL
restart - a second `Controller` against the same file. The peak was
always durable; the count of banked quarters was an `int` on the
tracker and reset to 0, so the page asserted "0 quarter(s) banked"
for a month it had measured 120 of. An unreadable store must not
produce a coverage figure of 0 either - part 5's argument, one
column over.
12. TARIFF-13: WHICH QUARTER A TIED PEAK IS ATTRIBUTED TO, decided
(earliest wins) rather than left emergent, and pinned on BOTH
readers with a newest-first fixture - ascending arrival cannot tell
"the earliest quarter" from "the first row written".
------------------------------------------------------------------------------
⚠️ PART 9 IS A TWO-DAY MEASUREMENT AND SAYS SO. THE 30-DAY ANSWER IS ELSEWHERE
@@ -106,9 +117,9 @@ from pathlib import Path
from app.main import Controller
from app.p1 import P1Ingest, QuarterAverager, QUARTER_S, make_sample, split_signed
from app.tariff import (
BILLING_TZ, CHECKPOINT_S, NO_RECORD, NOT_OBSERVED, POLICY_MAX,
POLICY_NET, POLICY_SUM, PeakStore, PeakTracker, _EUTimezone, billed_w,
month_key,
BILLING_TZ, CHECKPOINT_S, COUNT_UNREADABLE, NO_RECORD, NOT_OBSERVED,
POLICY_MAX, POLICY_NET, POLICY_SUM, PeakStore, PeakTracker, _ClosedBlock,
_EUTimezone, billed_w, month_key,
)
fails = []
@@ -932,18 +943,28 @@ check("the closed quarter reached SQLite through the tracker the cycle calls",
# ⚠️ `cycle()` calls sync() ONCE A SECOND against a `blocks` list that keeps a
# closed quarter for a DAY. Without a dedupe that is ~86 000 re-banks of the
# same row per block, and `recorded` - the number the status page prints as
# "quarters banked" - would be a count of loop iterations rather than of
# quarters. The database stays correct either way, which is exactly why this
# needs its own check instead of being caught by the peak.
# same row per block, and a coverage figure derived from the number of times a
# block was OFFERED would be a count of loop iterations rather than of quarters.
# The database stays correct either way, which is exactly why this needs its own
# check instead of being caught by the peak.
#
# ⚠️ GRADED ON `sync()`'s RETURN, and on the STORE's tally for the month the
# quarter is in - TARIFF-13. It used to be graded on `PeakTracker.recorded`,
# which was then an `int` incremented inside this very loop, so the dedupe and
# the figure that reported it were the same variable. They are not any more:
# `recorded_in()` counts rows in `quarter_blocks`, where re-banking the same
# `quarter_start` is a PRIMARY KEY no-op and could never have shown up as a
# second quarter. A check written against it alone would now pass however many
# times sync() re-banked, which is why the return value is the load-bearing
# half of this assertion and is asserted first.
again = c.peak.sync(c.p1)
third = c.peak.sync(c.p1)
print(f" sync() called three times over the same closed quarter: "
f"{n}, {again}, {third} newly banked; tracker reports "
f"{c.peak.recorded} quarter(s)")
f"{n}, {again}, {third} newly banked; the store holds "
f"{c.peak.recorded_in(B)} quarter(s) for {month_key(B)}")
check("...and syncing the same quarter again banks nothing new, so the count "
"the status page shows is quarters and not cycles",
again == 0 and third == 0 and c.peak.recorded == 1)
again == 0 and third == 0 and c.peak.recorded_in(B) == 1)
check("...and the peak is unchanged by the repeat",
near(c.peak.peak(B).peak_w, 4998.0, 1e-6))
@@ -957,11 +978,33 @@ peak_rows = [r for r in rows if "peak" in r["text"] and "quarter(s) banked" in r
print(" status row: " + (peak_rows[0]["text"] if peak_rows else "(none)"))
check("the status page carries exactly one capacity-tariff peak row",
len(peak_rows) == 1)
check("...naming the peak, the soft limit and the quarters banked, all derived "
check("...naming the peak, the soft limit and a coverage figure, all derived "
"from the controller that was actually built",
bool(peak_rows) and "4998 W" in peak_rows[0]["text"]
and f"{SHIPPED_CAP_W:.0f} W soft limit" in peak_rows[0]["text"]
and "1 quarter(s) banked" in peak_rows[0]["text"])
and "quarter(s) banked" in peak_rows[0]["text"])
# ⚠️ AND THE COVERAGE FIGURE IN THAT ROW IS SCOPED TO THE LIVE BILLING MONTH,
# NOT TO `when` - TARIFF-13, and it is a KNOWN GAP stated rather than rounded
# off. `main.py`'s row reads `self.peak.recorded`, which answers for
# `datetime.now()`; the peak beside it follows `when`. The two agree on the
# ONLY call site a shipped install has - `checks()` with no argument, at
# main.py:4359 - which is why the live page is correct and durable; they part
# company only under this suite's `when=` affordance. `PeakTracker` already
# carries the scoped form, `recorded_in(when)`, and moving the row onto it is a
# one-token edit inside main.py's status-row block, which TARIFF-11 holds. This
# check PINS the present behaviour so the gap cannot be forgotten, and part 11
# is the authority on the figure itself.
_live_month = month_key(datetime.now(timezone.utc))
_live_rows = len(c.peak.store.blocks_in(_live_month))
print(f" the row read at {month_key(B)} carries the tally for "
f"{_live_month} ({_live_rows} quarter(s)), which is this fixture's "
f"known scoping gap")
check("the row's coverage figure is the LIVE month's tally - graded against a "
"row count taken straight from quarter_blocks, so this pins the gap "
"rather than restating the property that produces it",
bool(peak_rows)
and f"; {_live_rows} quarter(s) banked" in peak_rows[0]["text"])
check("...and it is FLAGGED, because 4998 W is over §9.1's 80 % warning point "
f"on a {SHIPPED_CAP_W:.0f} W limit",
bool(peak_rows) and not peak_rows[0]["ok"] and peak_rows[0]["warn"])
@@ -1149,6 +1192,308 @@ check(f"...and the same IMPORT grader over main.py finds {_main_imports}, so "
bool(_main_imports))
# ==============================================================================
print()
print("11. TARIFF-13: what the instrument says about its OWN COVERAGE, "
"across a restart")
# THE DEFECT THIS PART PREVENTS. The status row prints a peak read from SQLite
# beside a count of banked quarters, and the count used to be an `int` attribute
# on `PeakTracker` that started at 0 in `__init__`. A 40-hour soak caught what
# that means at a driven restart, on both instances independently:
#
# before: 2026-09: peak 365 W, set by the quarter at 2026-09-11T18:45:00+00:00
# - 10 % of the 3500 W soft limit; 120 quarter(s) banked
# after : 2026-09: peak 365 W, set by the quarter at 2026-09-11T18:45:00+00:00
# - 10 % of the 3500 W soft limit; 0 quarter(s) banked
#
# The peak and its attributed quarter persist; the coverage beside them does
# not. §9.1's peak is what the utility bills, but the COUNT is what says how
# much of the billing month the figure covers - 365 W over 120 quarters and
# 365 W over 0 quarters are different claims about the same watt figure, and
# after a restart the page made the weaker claim while showing the stronger
# one's value. TARIFF-01's argument is that an unreadable store must not read as
# a zero peak because a zero peak is DISARMING; a zero COVERAGE beside a live
# peak is the same error one column over.
#
# ⚠️ THE RESTART HERE IS A REAL ONE - a second `Controller` against the same
# file on disk, never a re-read of the first object. 11b asserts that the second
# object has banked nothing itself, so the number it prints cannot have come
# from process memory. Without that precondition this whole part is satisfied by
# a counter that was simply never reset.
_T13 = TMP / "t13-restart.db"
# The fixture lives in the month being billed TODAY, and the row is read with NO
# `when` argument - the LIVE status page, the surface that misinformed. That
# also makes the check stable on every day of the year, which a fixed calendar
# date is not (TEST-17, one part up, is the same lesson).
_now13 = datetime.now(timezone.utc)
_mon13 = month_key(_now13)
# Anchor on the first 900 s boundary of the billing month, so all N quarters are
# inside `_mon13` however close to a month edge the suite happens to run.
_ms = datetime(int(_mon13[:4]), int(_mon13[5:]), 1, tzinfo=timezone.utc)
_q0 = datetime.fromtimestamp(
(int(_ms.timestamp()) // QUARTER_S + 1) * QUARTER_S, timezone.utc)
_N13 = 6 # > 1, so "count == rows" cannot be satisfied by 0 == 0
_W13 = 1234.0
c13 = built(_T13)
for _q in range(_N13):
_base = _q0 + timedelta(seconds=QUARTER_S * _q)
for _i in range(0, QUARTER_S + 1, 5):
c13.p1.submit(sample(_base + timedelta(seconds=_i), _W13))
c13.peak.sync(c13.p1)
# Ground truth from the OTHER tier, which the tracker did not supply - the count
# under test must be graded against something that is not itself (habit 2).
_truth_before = len(c13.peak.store.blocks_in(_mon13))
_row_before = _peak_rows(c13.checks())[0]["text"]
print(" before restart: " + _row_before)
check(f"precondition: the fixture banked {_N13} quarters in {_mon13}, so a "
f"count of 0 and a count of the truth are different numbers",
_truth_before == _N13 and _N13 > 1)
check(f"...and the row says so before the restart",
f"{_N13} quarter(s) banked" in _row_before)
c13.peak.store.close()
# THE RESTART.
c13b = built(_T13)
_row_after = _peak_rows(c13b.checks())[0]["text"]
_truth_after = len(c13b.peak.store.blocks_in(_mon13))
print(" after restart: " + _row_after)
print(f" rows in quarter_blocks: {_truth_before} before, "
f"{_truth_after} after; the restarted tracker has banked "
f"{len(c13b.peak._seen)} block(s) of its own through sync()")
check("precondition: the second Controller is a REAL restart - it shares no "
"object with the first and has banked nothing through sync(), so any "
"count it prints came off the disk and not out of memory",
c13b.peak is not c13.peak and len(c13b.peak._seen) == 0)
check("the peak and its attributed quarter survive the restart - the durable "
"half, asserted so that the coverage claim below is not graded against a "
"reading that had already collapsed",
_row_before.split(" - ")[0] == _row_after.split(" - ")[0]
and f"{_W13:.0f} W" in _row_after)
check("...and the COUNT BESIDE IT DOES NOT COLLAPSE TO ZERO, which is the "
"defect: a coverage figure that resets while the value beside it "
"persists actively misinforms",
"; 0 quarter(s) banked" not in _row_after)
check(f"...and it is the store's own tally, {_truth_after}, not a "
f"process-lifetime counter - graded against a COUNT OF ROWS in the "
f"other tier rather than against the property under test",
f"{_truth_after} quarter(s) banked" in _row_after and _truth_after >= _N13)
# ⚠️ The restarted run banks one MORE than it drove, and that is correct: the
# checkpointed open block was stale, so `restore()` closed and banked it. The
# in-memory counter never saw that block at all - it only ever counted what went
# through `sync()` - so this is a third way the old figure under-read, and it is
# asserted rather than tolerated. Driven at a chosen `now` so it cannot depend
# on when the suite runs.
_t13s = fresh_store("t13-stale")
_tr13 = PeakTracker(_t13s, cap_w=SHIPPED_CAP_W)
_av13 = QuarterAverager(phases=1)
_sb = _q0 + timedelta(seconds=QUARTER_S * 40)
for _i in range(0, 600, 5):
list(_av13.add(sample(_sb + timedelta(seconds=_i), 2000.0)))
_t13s.save_partial(_av13.snapshot())
_before_stale = _tr13.recorded_in(_sb)
_state = _tr13.restore(QuarterAverager(phases=1),
now=_sb + timedelta(seconds=QUARTER_S * 3))
print(f" stale-block restore: {_state}")
print(f" quarters banked in {month_key(_sb)}: {_before_stale} -> "
f"{_tr13.recorded_in(_sb)}")
check("a block closed by restore() rather than by sync() is counted too - the "
"old counter incremented only in sync() and missed it entirely",
_before_stale == 0 and _tr13.recorded_in(_sb) == 1
and "banked" in _state)
# --- the count is scoped to the MONTH it is printed beside --------------------
# ⚠️ TWO MONTHS AT DIFFERENT COUNTS. Equal counts would be satisfied by a
# cross-month total, by a constant, and by reading either month for the other.
_t13m = fresh_store("t13-months")
_MA = datetime(2026, 4, 6, 9, 0, tzinfo=timezone.utc) # 2026-04
_MB = datetime(2026, 5, 6, 9, 0, tzinfo=timezone.utc) # 2026-05
_trm = PeakTracker(_t13m, cap_w=SHIPPED_CAP_W)
for _m, _k in ((_MA, 3), (_MB, 5)):
for _q in range(_k):
_t13m.record_block(_ClosedBlock(
start=_m + timedelta(seconds=QUARTER_S * _q),
offtake_avg_w=900.0 + _q, per_phase_offtake_avg_w=None))
print(f" banked by month: {month_key(_MA)}={_trm.recorded_in(_MA)}, "
f"{month_key(_MB)}={_trm.recorded_in(_MB)}, "
f"a month never written={_trm.recorded_in(datetime(2026, 7, 6, 9, 0, tzinfo=timezone.utc))}")
check("precondition: the two months were given DIFFERENT numbers of quarters, "
"so a count that answered either one for the other would be visible here",
month_key(_MA) != month_key(_MB))
check("each month reports ITS OWN quarters and not the pair's total - the old "
"counter was per PROCESS, so on 1 October the row still showed "
"September's tally beside October's empty month",
_trm.recorded_in(_MA) == 3 and _trm.recorded_in(_MB) == 5)
check("...and a month with no quarters reports 0, which is TRUE of a month "
"nothing was written to - the sentinel below is for a store that will "
"not answer, and the two states must not be spelled the same way",
_trm.recorded_in(datetime(2026, 7, 6, 9, 0, tzinfo=timezone.utc)) == 0)
# --- and a store that cannot answer does NOT report zero ----------------------
# The other half of part 5's argument, applied to the coverage figure. `0
# quarter(s) banked` and `an unreadable number of quarter(s) banked` are
# opposite states: the first says the month has been measured over nothing, the
# second says the instrument cannot say. Spelling them the same way is exactly
# what this card was raised about.
_t13u = PeakStore(str(TMP / "no-such-dir" / "nope13.db"))
_tru = PeakTracker(_t13u, cap_w=SHIPPED_CAP_W)
print(f" unreadable store: banked_count -> {_t13u.banked_count(_mon13)!r}, "
f"rendered -> {_tru.recorded_in(_now13)!r}")
check("an unreadable store answers None to a COUNT, never 0 - the store-level "
"half",
_t13u.banked_count(_mon13) is None)
check("...and the rendered figure says it is unreadable instead of naming a "
"number, so the sentence cannot read as a measured zero",
_tru.recorded_in(_now13) == COUNT_UNREADABLE
and f"{_tru.recorded_in(_now13)} quarter(s) banked"
== "an unreadable number of quarter(s) banked")
check("...and that sentence is not merely NON-zero by accident: the string a "
"count of 0 would produce is different text",
f"{_tru.recorded_in(_now13)} quarter(s) banked"
!= f"{0} quarter(s) banked")
# ⚠️ A STORE THAT IS OPEN AND WHOSE COUNT QUERY FAILS IS A SECOND CODE PATH, and
# the checks above cannot reach it - a handle that never opened short-circuits
# on `self._db is None` before any SQL runs, so `banked_count()`'s
# `except sqlite3.Error` arm was executed by nothing. Found by this card's own
# mutation battery, not by review: returning 0 instead of None from that arm
# survived the whole suite at -3 B. It is the same third-path argument part 5
# makes for `peak()`, and it is the arm a half-corrupt database on a real
# install actually takes.
_t13b = fresh_store("t13-broken")
for _q in range(4):
_t13b.record_block(_ClosedBlock(start=_q0 + timedelta(seconds=QUARTER_S * _q),
offtake_avg_w=800.0 + _q,
per_phase_offtake_avg_w=None))
_trb = PeakTracker(_t13b, cap_w=SHIPPED_CAP_W)
_count_ok = _t13b.banked_count(_mon13)
_t13b._db.execute("DROP TABLE quarter_blocks")
_t13b._db.commit()
_count_broken = _t13b.banked_count(_mon13)
print(f" open store, count query FAILS: {_count_ok} -> {_count_broken!r}, "
f"rendered -> {_trb.recorded_in(_now13)!r}")
check(f"precondition: before the table was dropped the same call answered "
f"{_count_ok}, so 'it returns nothing now' is a change and not the "
f"store's resting state",
_count_ok == 4)
check("a store whose COUNT QUERY fails answers None too - a different code "
"path from a handle that never opened, and the one a half-corrupt "
"database takes",
_count_broken is None)
check("...and it renders as unreadable rather than as 0, so a half-corrupt "
"database cannot print a coverage figure that reads like a measured "
"month with nothing in it",
_trb.recorded_in(_now13) == COUNT_UNREADABLE)
# --- the count reads the BLOCK tier, not the month tier -----------------------
# `monthly_peak` holds one row per MONTH, so counting it answers 1 for every
# month that has ever been measured - a figure that is wrong by exactly the
# amount that makes it look plausible.
check("the coverage figure counts quarter_blocks and not monthly_peak, which "
"would answer 1 for every measured month however many quarters it holds",
_trm.recorded_in(_MB) == 5
and _t13m.peak(_MB).is_observed and _t13m.recompute_month(month_key(_MB)).is_observed)
# ==============================================================================
print()
print("12. TARIFF-13: which quarter a TIED peak is attributed to")
# THE DECISION, recorded in DECISIONS/TARIFF-13-the-coverage-of-a-peak.md §2:
# at equal wattage the EARLIEST quarter keeps the attribution. The wattage is
# what is billed, so this is not a money question - it is the answer to "when
# did my monthly peak happen", and an operator investigating that goes and looks
# at one event. A stable answer is the useful one; "most recent wins" guarantees
# the answer moves every time the wattage recurs.
#
# ⚠️ AND THE SOAK'S DRIFT WAS NOT A TIE. The 40-hour run showed 365 W attributed
# to three different quarters with no restart and no new peak, and the obvious
# reading - "ties re-attribute to the most recent" - is false, measured: the
# upsert's term is strictly `>`, so an exact tie never wrote. What moved was a
# genuine new maximum SMALLER THAN THE ROW'S OWN RESOLUTION. The row renders
# `{peak_w:.0f}`, whole watts; 365.0000001 W is a new peak to the database and
# the same "365 W" to the reader. Both cases are pinned below, because the fix
# for one is not the fix for the other.
_t13t = fresh_store("t13-ties")
_TB = datetime(2026, 3, 9, 9, 0, tzinfo=timezone.utc)
def _bank(store, q, w):
store.record_block(_ClosedBlock(start=_TB + timedelta(seconds=QUARTER_S * q),
offtake_avg_w=w, per_phase_offtake_avg_w=None))
def _qt(n):
return _TB + timedelta(seconds=QUARTER_S * n)
for _q in (0, 1, 2, 3):
_bank(_t13t, _q, 365.0)
_asc = _t13t.peak(_TB)
print(f" four EXACTLY equal 365 W quarters, oldest first -> attributed "
f"{_asc.quarter_start.isoformat()}")
check("an exact tie keeps the EARLIEST quarter: the attribution does not move "
"when the same wattage recurs",
_asc.quarter_start == _qt(0) and near(_asc.peak_w, 365.0))
# ⚠️ FED NEWEST-FIRST, which is what separates "the earliest quarter" from
# "whichever row was written first". Ascending arrival cannot tell those apart -
# they are the same row - so an ascending-only fixture grades neither.
_t13r = fresh_store("t13-ties-rev")
for _q in (3, 2, 1, 0):
_bank(_t13r, _q, 365.0)
_desc = _t13r.peak(_TB)
_desc_rc = _t13r.recompute_month(month_key(_TB))
print(f" the same four fed NEWEST first -> monthly_peak attributed "
f"{_desc.quarter_start.isoformat()}, recompute_month attributed "
f"{_desc_rc.quarter_start.isoformat()}")
check("...and it is the earliest QUARTER, not the first row WRITTEN - fed "
"newest-first the answer is still the oldest quarter",
_desc.quarter_start == _qt(0))
check("...and BOTH readers give it, so the derived record has not diverged "
"from the tier it derives from on the tie rule either (part 8's "
"invariant, at equal wattage)",
_desc_rc.quarter_start == _desc.quarter_start
and near(_desc_rc.peak_w, _desc.peak_w))
# The other direction, and the reason the tie rule may never touch `peak_w`: a
# STRICTLY HIGHER quarter must still take the attribution, whenever it arrives.
_t13h = fresh_store("t13-higher")
for _q, _w in ((0, 365.0), (1, 366.0), (2, 365.0)):
_bank(_t13h, _q, _w)
_hi = _t13h.peak(_TB)
print(f" 365, then 366, then 365 -> peak {_hi.peak_w:.0f} W at "
f"{_hi.quarter_start.isoformat()}")
check("a strictly higher quarter still takes both the peak and the "
"attribution - the tie rule moves the TIMESTAMP and never the watts",
near(_hi.peak_w, 366.0) and _hi.quarter_start == _qt(1))
# And the mechanism the soak actually saw: an increase below the row's own
# resolution. Not a tie, and no tie rule can address it.
_t13d = fresh_store("t13-subdisplay")
_seen13 = []
for _q, _w in enumerate((365.0, 365.0000001, 365.0000002)):
_bank(_t13d, _q, _w)
_seen13.append(_t13d.peak(_TB).quarter_start)
print(f" 365.0, 365.0000001, 365.0000002 W all render "
f"{_t13d.peak(_TB).peak_w:.0f} W and attribute to "
f"{[s.time().isoformat() for s in _seen13]}")
check("an increase BELOW the row's whole-watt resolution is a real new maximum "
"and does move the attribution - so the soak's drifting quarter was a "
"rounded display, not a tie, and is left alone deliberately",
len(set(_seen13)) == 3
and len({f"{w:.0f}" for w in (365.0, 365.0000001, 365.0000002)}) == 1)
# ==============================================================================
print()
shutil.rmtree(TMP, ignore_errors=True)
+11 -1
View File
@@ -147,6 +147,16 @@ DERIVED = {
# where the ratio came from - a citation alone would leave an operator
# reading "0" on the tab with no idea it is not "off".
"slew_w": ("SAFETY-39", "DERIVED", "0 MEANS FOLLOW"),
# TARIFF-18. `price_taxes_eur_kwh` joins the table for `slew_w`'s reason:
# it stopped being a figure somebody picked and became one SUMMED FROM
# NAMED CARD ROWS. Three needles, because three different things about it
# can go wrong on the Configuration tab and only the first is provenance:
# where the number came from, that the operator's own DSO gives a
# different one, and that changing it moves no trade - which is the claim
# an operator is most likely to assume the opposite of when a number
# doubles.
"price_taxes_eur_kwh": ("TARIFF-18", "YOUR OWN DSO",
"MOVES NO TRADING DECISION"),
}
_uncited = []
for key, needles in sorted(DERIVED.items()):
@@ -362,7 +372,7 @@ print("\n7. no behaviour and no value moved")
# test_peakguard.py's. This line is the cheap one that says the reorder did not
# lose or duplicate a key on the way.
check("no option was lost, gained or duplicated by the reordering: the two "
"halves of config.yaml hold the same 71 keys, once each",
"halves of config.yaml hold the same 72 keys, once each",
sorted(_opt_order) == sorted(ORDER) and len(set(ORDER)) == len(ORDER))
check("...and `inverters: []` is still shipped in the options half, which is "
"what Supervisor 2026.08.0 refuses every pre-0.6.0 options blob without "
+187
View File
@@ -18,6 +18,8 @@ import asyncio
import sys
from app.maintenance import MAINT_OPTION_DEFAULTS, MaintConfig
from app.price import (
PRICE_OPTION_DEFAULTS, SUPERSEDED_TAXES_DEFAULTS_EUR_KWH)
from app.control import LOOP_TICK_S, PLANT_SETTLE_S, SLEW_RAIL_FRAC
from app.validate import (
ACTUATOR_WRITE_KEYS, FIRMWARE_INGEST_VETO_S, ConfigRefused,
@@ -1366,6 +1368,191 @@ check("...and the flat single-inverter config it always refused is refused "
-5000, 5000) == [])
# --------------------------------------------------------------------------- #
print()
print("TARIFF-17 - the taxes period table is refused HERE, at startup, and not "
"at 03:00")
# ⚠️ WHY IT IS REFUSED IN THIS MODULE AND NOT AT FETCH TIME. `price.assemble()`
# reads this table on the ~30 s planning path. A table that only fails when a
# fetch happens is a site that stops pricing mid-afternoon, and §11.4 turns
# that into a quiet fallback indistinguishable on the status page from a
# healthy fixed-tariff day. That is `static_profile`'s argument, one option
# over, and this is the same rule.
_T17_DAG, _T17_NACHT = 0.1327, 0.0739
_T18_SHIPPED_TAXES = PRICE_OPTION_DEFAULTS["price_taxes_eur_kwh"]
_T18_SUPERSEDED = tuple(SUPERSEDED_TAXES_DEFAULTS_EUR_KWH)
_T17_OK = ('{"periods":['
'{"name":"dag","from":"07:00","to":"22:00","days":"mon-fri",'
f'"eur_per_kwh":{_T17_DAG}}},'
'{"name":"nacht","from":"22:00","to":"07:00","days":"mon-fri",'
f'"eur_per_kwh":{_T17_NACHT}}},'
'{"name":"nacht","from":"00:00","to":"24:00","days":"sat,sun",'
f'"eur_per_kwh":{_T17_NACHT}}}]}}')
# ⚠️ THE ACCEPTED BASELINE COMES FIRST (MUTATION.md habit 7). Every refusal
# below varies ONE thing from this line, so a refusal on the next line has
# nowhere else to have come from.
check("ORES's real dag/nacht table STARTS - it is a valid configuration, and "
"the refusals below are one edit away from this one",
validate_startup_config(base(price_taxes_profile=_T17_OK),
-5000, 5000) == [])
check("...and so does the shipped \"\", which is every install that exists: "
"the option is an addition, not a requirement",
validate_startup_config(base(price_taxes_profile=""), -5000, 5000) == []
and validate_startup_config(base(), -5000, 5000) == [])
_T17_GAP = ('{"periods":[{"name":"dag","from":"07:00","to":"22:00",'
f'"days":"all","eur_per_kwh":{_T17_DAG}}}]}}')
_T17_P_GAP = validate_startup_config(base(price_taxes_profile=_T17_GAP),
-5000, 5000)
print(f" {(_T17_P_GAP or [''])[0][:120]}")
check("a table that leaves 00:00-07:00 unpriced is REFUSED, by the same tiling "
"rule static_profile has - and the refusal names the KEY and the HOURS, "
"not merely that something was wrong",
why(_T17_P_GAP, "price_taxes_profile cannot be used:")
and why(_T17_P_GAP, "leaves mon 00:00-07:00 with no period"))
check("...and it is refused for THAT reason and not as a side effect of "
"something else this fixture happens to break: exactly one problem",
len(_T17_P_GAP) == 1)
# ⚠️ TWO WRITERS OF ONE QUANTITY, ONE OF WHICH SILENTLY WINS. The table decides
# when it is set, so a scalar left at a NON-DEFAULT value beside it is a number
# the operator typed that changes nothing - on the quantity §11.4 makes hardest
# to notice.
_T17_P_BOTH = validate_startup_config(
base(price_taxes_profile=_T17_OK, price_taxes_eur_kwh=0.1198), -5000, 5000)
print(f" {(_T17_P_BOTH or [''])[0][:120]}")
check("the table AND a non-default price_taxes_eur_kwh is refused, and the "
"refusal says WHICH ONE WINS and what to do - not merely that the two "
"are incompatible",
why(_T17_P_BOTH, "price_taxes_profile is set AND price_taxes_eur_kwh is "
"0.1198")
and why(_T17_P_BOTH, "THE TABLE WINS"))
check("...and the SHIPPED scalar beside the table is NOT refused, or every "
"Walloon site would have to clear a key it never touched. The refusal is "
"about a value the operator chose, and the shipped default is the one "
"they did not",
validate_startup_config(
base(price_taxes_profile=_T17_OK,
price_taxes_eur_kwh=_T18_SHIPPED_TAXES),
-5000, 5000) == []
and len(_T17_P_BOTH) == 1)
# ⚠️ TARIFF-18 - "UNTOUCHED" IS A SET, AND THIS IS THE HALF THAT DECIDES
# WHETHER AN UPGRADE STOPS A RUNNING SITE. The Supervisor sets `App.options =
# deepmerge(config.yaml defaults, stored options)`, so a saved blob carries
# every schema key whether the operator typed it or not - the reference
# install's export is 70 of 71 (CHANGELOG 0.9.0). When TARIFF-18 moved this
# default 0.06 -> 0.11736, every such blob went from "equal to the shipped
# default" to "not equal to it" WITHOUT ANYBODY EDITING ANYTHING. A site that
# also carries a taxes table would have met `SystemExit(78)` on the upgrade,
# over a number it never typed and that decides nothing. So a SUPERSEDED
# default still reads as untouched.
#
# ⚠️ AND THE PRECONDITION FIRST (MUTATION.md habit 1): if the shipped default
# and the superseded one were ever the same number, the two checks below would
# be one check written twice, and dropping the superseded set entirely would
# leave both green.
check(f"the shipped taxes default {_T18_SHIPPED_TAXES} and the superseded "
f"{_T18_SUPERSEDED} are DIFFERENT numbers, so 'the shipped one is "
f"accepted' and 'the superseded one is accepted' are two claims",
_T18_SHIPPED_TAXES not in _T18_SUPERSEDED and len(_T18_SUPERSEDED) >= 1)
check(f"...and the superseded set is non-empty ({_T18_SUPERSEDED}), or the "
f"loop below quantifies over nothing and passes on a tree that dropped "
f"the set entirely - UX-01 AC 4's vacuous clause, one file over",
len(_T18_SUPERSEDED) >= 1)
for _old in _T18_SUPERSEDED:
check(f"...and a blob that still carries the SUPERSEDED default {_old} "
f"beside the table STARTS. That value was written into the options "
f"file by this add-on, not typed by the operator, so refusing it "
f"would stop a working site on an upgrade over an inert number",
validate_startup_config(base(price_taxes_profile=_T17_OK,
price_taxes_eur_kwh=_old),
-5000, 5000) == [])
# ⚠️ AND THE RUNG IS STILL A RUNG. A superseded default is not a licence for
# any value: 0.1198 is ORES's enkelvoudig rate, a number an operator plausibly
# types, and it is still refused beside the table. Without this line the set
# above could be widened to "anything" and nothing would report.
check("...while a value that is NEITHER the shipped default NOR a superseded "
"one is still REFUSED beside the table, naming it - the set of "
"'untouched' figures widened by exactly the old defaults and by nothing "
"else",
why(_T17_P_BOTH, "price_taxes_profile is set AND price_taxes_eur_kwh is "
"0.1198")
and 0.1198 not in _T18_SUPERSEDED
and abs(0.1198 - _T18_SHIPPED_TAXES) > 1e-9)
# ⚠️ AND THE REFUSAL SAYS SO, so an operator who HAS typed a number is told
# both which figure to return to and that the old default is not the problem.
check("...and the refusal names the SHIPPED default to return to AND says the "
"superseded default is accepted, so the sentence is actionable on both "
"sides of the upgrade",
bool(_T18_SUPERSEDED)
and why(_T17_P_BOTH, f"back to {_T18_SHIPPED_TAXES}")
and why(_T17_P_BOTH, "SUPERSEDED")
and all(why(_T17_P_BOTH, str(u)) for u in _T18_SUPERSEDED))
check("...and the same non-default scalar WITHOUT the table still starts, "
"which is what makes the line above about the PAIR rather than about "
"0.1198 being disliked",
validate_startup_config(base(price_taxes_eur_kwh=0.1198),
-5000, 5000) == [])
# ⚠️ THE UNITS RUNG, A SECOND QUANTITY RATHER THAN A RESTATEMENT. Every Belgian
# DSO prints these bands in c EUR/kWh, and the whole table is copied off ONE
# card - so a factor-100 error lands on every row at once and nothing inside
# the table disagrees with itself.
#
# ⚠️ AND THE TWO RUNGS ARE GRADED SEPARATELY, BY THE REASON EACH GIVES, because
# the parser's own +-10 EUR/kWh bound catches SOME of these cards and not
# others - and a check that only ever exercised the loud ones would pass with
# this rung deleted. ORES's 13,27 is refused by the parser; SIBELGA's whole
# card - 9,39 dag against 7,10 nacht, the Brussels DSO - is under 10 in both
# bands and reaches this rung alone.
_T17_ORES_C = ('{"periods":['
'{"name":"dag","from":"07:00","to":"22:00","days":"mon-fri",'
'"eur_per_kwh":13.27},'
'{"name":"nacht","from":"22:00","to":"07:00","days":"mon-fri",'
'"eur_per_kwh":7.39},'
'{"name":"nacht","from":"00:00","to":"24:00","days":"sat,sun",'
'"eur_per_kwh":7.39}]}')
_T17_SIB_C = ('{"periods":['
'{"name":"dag","from":"07:00","to":"22:00","days":"mon-fri",'
'"eur_per_kwh":9.39},'
'{"name":"nacht","from":"22:00","to":"07:00","days":"mon-fri",'
'"eur_per_kwh":7.10},'
'{"name":"nacht","from":"00:00","to":"24:00","days":"sat,sun",'
'"eur_per_kwh":7.10}]}')
_T17_P_ORES = validate_startup_config(base(price_taxes_profile=_T17_ORES_C),
-5000, 5000)
_T17_P_SIB = validate_startup_config(base(price_taxes_profile=_T17_SIB_C),
-5000, 5000)
print(f" ORES : {(_T17_P_ORES or [''])[0][:110]}")
print(f" Sibelga : {(_T17_P_SIB or [''])[0][:110]}")
check("ORES's card typed in CENTS is refused by the PARSER's bound, and the "
"sentence is the parser's - `beyond +-10.0`, naming the row",
why(_T17_P_ORES, "price_taxes_profile cannot be used:")
and why(_T17_P_ORES, "beyond +-10.0"))
check("...and SIBELGA's whole card - 9,39 and 7,10, BOTH under that bound - is "
"refused by THIS rung instead, naming both bands and the factor. Without "
"it a Brussels site would price a 9.39 EUR/kWh levy and start",
why(_T17_P_SIB, "price_taxes_profile carries [7.1, 9.39] EUR/kWh")
and why(_T17_P_SIB, "factor of 100"))
check("...and the two refusals are DIFFERENT sentences, so the rungs cannot "
"mask each other: deleting the units rung leaves the Sibelga line red "
"while the ORES line goes on passing for the parser's reason",
_T17_P_ORES != _T17_P_SIB
and not why(_T17_P_ORES, "factor of 100")
and not why(_T17_P_SIB, "beyond +-10.0"))
check("...and BOTH cards divided by 100 start, which is habit 7: the refusals "
"above are about the units and not about Brussels or Wallonia",
validate_startup_config(
base(price_taxes_profile=_T17_ORES_C.replace("13.27", "0.1327")
.replace("7.39", "0.0739")), -5000, 5000) == []
and validate_startup_config(
base(price_taxes_profile=_T17_SIB_C.replace("9.39", "0.0939")
.replace("7.10", "0.0710")), -5000, 5000) == [])
# --------------------------------------------------------------------------- #
print()
if fails:
@@ -1,10 +0,0 @@
{
"_comment": "SAFETY-44 SYNTHETIC FIXTURE - not a copy of any live state file; this session cannot read that one and did not try. It exists for ONE control: a store in which TWO ROSTER UNITS ARE LATCHED AT THE SAME VALUE. `state-legacy.json` cannot serve it - it is pre-DEV-02 and flat, so after migration only the FIRST roster entry is ever latched, and a rule that suppresses a vanished key when another key holds the same value can never be exercised against it. That is exactly how M12's first draft came to pass on the UNMUTATED tree. Both units are latched by ONE site event, so the reasons match too, which is the arrangement in which the suppression is most dangerous.",
"schema_version": 1,
"maint_schema_version": 1,
"auto": true,
"unit/garage/sign_tripped": true,
"unit/garage/sign_reason": "SIGN_DISAGREEMENT: grid meter reversed, both units",
"unit/shed/sign_tripped": true,
"unit/shed/sign_reason": "SIGN_DISAGREEMENT: grid meter reversed, both units"
}
@@ -0,0 +1,12 @@
{
"_comment": "INSTR-01 SYNTHETIC FIXTURE - `state-legacy.json` with a SECOND roster unit already latched. Not a copy of any live state file. It exists to restore M12 END-TO-END on the STAGED leg, which is the arrangement SAFETY-44 and INSTR-01's first draft both missed: `snapshot_over()` starts the after-leg over the file the BEFORE-leg wrote, so a transition-made destination is available there and the two questions the provenance clause and the narrowing ask are NOT incompatible. Every same-step arrangement makes them incompatible; that is what was measured and over-generalised to 'no fixture can'.",
"phase": "idle",
"phase_started": "2026-08-07T10:00:00+00:00",
"last_completed": "2026-08-07T18:42:00+00:00",
"last_start_attempt": "2026-08-07T10:00:00+00:00",
"sign_tripped": true,
"sign_reason": "SIGN_DISAGREEMENT: commanded -1800 W against a meter reading -1450 W for 190 s",
"auto": true,
"unit/shed/sign_tripped": true,
"unit/shed/sign_reason": "SIGN_DISAGREEMENT: shed, latched before the switch"
}
+23 -3
View File
@@ -545,6 +545,12 @@ def diff(before: Snapshot, after: Snapshot) -> list:
# name and whose roster start writes them under another reported two
# VANISHED findings about a pair of defaults, and CONTROL-C went red on a
# build that was CORRECT.
# ⚠️ `claimed` MAKES THE MOVE INJECTIVE. Without it N vanished keys can all
# claim the SAME destination - and `DEFAULTS["sign_tripped"] is False`, so
# `True` is the ONLY non-default value a latch can hold, which means they
# collide by CONSTRUCTION rather than by coincidence. Two keys cannot both
# have become one key; the second claimant has not moved anywhere.
claimed = set()
for key in sorted(set(before.data) - set(after.data)):
value = before.data[key]
bare = key.rsplit("/", 1)[-1]
@@ -573,11 +579,24 @@ def diff(before: Snapshot, after: Snapshot) -> list:
# here. Neither mutation exercises this branch. The sentence "what keeps
# this from being a softening of the check" was not supported by the
# thing it named - and a citation is the worst possible carrier for that,
# because it reads as having been checked. The branch is now bounded by
# M12, written for it, in test_rehearse.py.
# because it reads as having been checked.
#
# ⚠️ AND THE REPLACEMENT CITATION WENT STALE THE SAME WAY. It read "the
# branch is now bounded by M12" through a card that DELETED M12 and a
# card that restored it on a different leg - eight lines below the
# paragraph condemning exactly this. Every clause of this comprehension
# is now graded by a row in test_rehearse.py part 5e, one case per
# clause, and each clause-removal must flip exactly its own row. That
# is a property of the section, not a pointer at a named check.
# ⚠️ AND THE DESTINATION MUST BE ONE **THIS TRANSITION MADE**
# (`before.data.get(k) != value`) AND NOT ALREADY SPOKEN FOR
# (`k not in claimed`). Without the first, a roster unit that was
# already latched in its own right is a free suppression slot that
# nobody moved into.
moved_to = sorted(
k for k in after.machine_read
if k.rsplit("/", 1)[-1] == bare and after.data.get(k) == value)
if k.rsplit("/", 1)[-1] == bare and after.data.get(k) == value
and before.data.get(k) != value and k not in claimed)
# ⚠️ AND THE VANISHED KEY'S OWN UNIT MUST BE **OFF THE ROSTER**, WHICH
# THE FIRST VERSION OF THIS RULE DID NOT REQUIRE AND WHICH IS WHAT MADE
# IT A SOFTENING. The match was bare-key-plus-value over the whole of
@@ -600,6 +619,7 @@ def diff(before: Snapshot, after: Snapshot) -> list:
vanished_unit = key.split("/")[1] if key.startswith("unit/") else None
if vanished_unit is not None and vanished_unit not in after.units \
and moved_to:
claimed.add(moved_to[0])
after.vanished_but_migrated.append(
f"{key} = {value!r} -> {moved_to[0]}")
continue
+14 -8
View File
@@ -3,8 +3,8 @@
# ⚠️ WHAT THIS FILE IS FOR, AND WHAT HOME ASSISTANT ACTUALLY DOES WITH IT.
# Without a `translations/` directory the Configuration tab renders the raw
# option KEYS - `unit_sign_disagree_w`, `saturation_cycles`, `integrator_max_w`.
# The frontend keeps a field when `e.name in options || e.required`, and all 71
# keys are in `options:` (59 of them are `required` as well), so ALL SEVENTY-ONE are
# The frontend keeps a field when `e.name in options || e.required`, and all 72
# keys are in `options:` (59 of them are `required` as well), so ALL SEVENTY-TWO are
# rendered, always, in `schema:` order, with no description. This file is what
# turns that into sentences.
#
@@ -143,25 +143,31 @@ configuration:
description: "DERIVED, not picked (SAFETY-24): the reference pack holds 13.8 kWh usable and the worst modelled peak-shaving day asks for 878 Wh, which is 6.4 %, so 20 % sits ten points above the pack's own cut-out and still leaves the guard about 1.6 of its worst observed days of room. ⚠️ IT BOUNDS THE PEAK GUARD ONLY. It is not a reserve for the whole pack: ordinary self-consumption discharges straight past it and goes on down to the battery cut-out, and raising this number does not change that by a tenth of a percent. It must stay strictly above the battery cut-out or the add-on refuses to start. Do not change it without re-deriving it."
price_provider:
name: "Tariff · Dynamic price source"
description: "Where day-ahead or real-time prices come from. 'off' is what ships, and means this site runs on the static tariff below with no trading. ⚠️ Switching it on today fetches, checks and publishes prices and changes no energy decision - nothing consumes the price series yet. It may not be combined with the two legacy Home Assistant price entities further down: two price sources on one site, one of which silently wins, is refused at startup rather than left looking armed."
description: "Where day-ahead or real-time prices come from. 'off' is what ships, and means this site runs on the static tariff below with no trading. ⚠️ SWITCHING THIS ON IS WHAT STARTS THE PRICE FETCH - not the trading switch further down. With a source selected the add-on fetches the feed every half hour, checks it, records the day and shows what it holds on the Price history row, whether or not price trading is armed. On its own it still moves no energy: trading is the separate switch, and the cheap window the monthly maintenance cycle uses comes from the fixed or day/night tariff below. It may not be combined with the two legacy Home Assistant price entities further down: two price sources on one site, one of which silently wins, is refused at startup rather than left looking armed. ⚠️ THREE IDS IN THIS LIST ARE WITHDRAWN AND WILL NOT START: engie_dynamic, totalenergies_dynamic and eneco_eddy. Their endpoints do not exist and never did, so those plugins never once fetched a price - a site set to one of them has silently been running on the fixed or day/night tariff below, or on nothing, since the day it was set. They are still offered here only so that such a site gets that explanation at startup instead of an unreadable schema error. Choose generic_dayahead for the wholesale curve with a flat markup, custom_formula to enter your own contract's B x index + A coefficients off your tariff card, manual_csv for a file you write yourself, or off."
price_markup_eur_kwh:
name: "Tariff · Supplier markup (EUR/kWh)"
description: "Your supplier's margin. The add-on adds it ONLY when the chosen price source does not already include it - under engie_dynamic, totalenergies_dynamic, eneco_eddy and manual_csv it is ignored, and logged as ignored, never applied twice."
description: "Your supplier's margin. The add-on adds it ONLY when the chosen price source does not already include it - under manual_csv, whose figures are all-in, it is ignored, and logged as ignored, never applied twice."
price_taxes_eur_kwh:
name: "Tariff · Levies and VAT (EUR/kWh)"
description: "Taxes and levies per kWh, added under the same rule as the supplier markup: only when the chosen price source does not already carry them."
name: "Tariff · Distribution, levies and VAT (EUR/kWh)"
description: "Everything on your bill that is not the energy itself - distribution, levies, excise and VAT - per kWh, added under the same rule as the supplier markup: only when the chosen price source does not already carry them. ⚠️ THIS NUMBER IS NOT A PREFERENCE AND IT IS NOT YOUR INVERTER'S. It ships at 0.11736, DERIVED in TARIFF-18 from the Eneco residential card for Flanders of September 2026 including 6% VAT: Fluvius distribution for Antwerpen 5,35 plus bijzondere accijns 4,876 plus bijdrage groene stroom en WKK 1,51 c/kWh. The energiebijdrage is 0 since 1 August 2026 - it was merged into the excise, so a figure copied from an older guide would count it twice. ⚠️ CHECK IT AGAINST YOUR OWN DSO. Only the distribution row moves: across the eight Fluvius DSOs it runs from 5,28 c/kWh in Midden-Vlaanderen to 6,70 in West-Vlaanderen, so the Flemish total runs from about 0.117 to 0.131. In Wallonia or Brussels the distribution tariff has a separate day and night rate and does not belong in this field at all - use the levies-by-time-of-day option below instead. ⚠️ CHANGING IT MOVES NO TRADING DECISION: the same amount is added to every hour, so it cancels out of both the cheapest hour and the spread. What it does move is every price you are shown, and the rule that refuses to discharge into an hour whose all-in price is below zero."
export_price_eur_kwh:
name: "Tariff · Injection price (EUR/kWh)"
description: "What you are paid for feeding a kWh back into the grid, all in. One number rather than a series: no supplier here publishes a dynamic injection curve. In Belgium it is often close to zero, which on its own strongly favours using your own production over exporting it. It may be negative - at midday in summer you can be paying to export."
static_profile:
name: "Tariff · Fixed or day/night tariff (one line of JSON)"
description: "Your fixed or day/night contract, written as a single line of JSON. RECOMMENDED EVEN ON A DYNAMIC CONTRACT: this is what the add-on falls back to when the price feed dies, and with it empty a dead feed leaves the site with no tariff at all. The periods must cover every day type exactly once, with no gap and no overlap - a gap is refused at startup rather than filled in, because an invented price is one a strategy would trade on. A fully fixed tariff is one period covering 00:00-24:00 on all days. Prices are ALL IN; the markup and levies above are not added to them. Note that a window written 22:00-07:00 for mon-fri does NOT cover Saturday morning, so a weekend row is required rather than decorative."
price_taxes_profile:
name: "Tariff · Levies and VAT by time of day (one line of JSON)"
description: "Leave this empty unless your network tariff really does charge a different day and night rate - in Flanders it does not, Fluvius distribution is one rate, and the flat figure above is correct. In Wallonia and Brussels it does: ORES charges 13,27 c/kWh by day against 7,39 at night, AIESH 15,17 against 8,21, Sibelga 9,39 against 7,10. Written as a single line of JSON in exactly the same shape as the fixed or day/night tariff above, with the same rule that the periods must cover every day type once with no gap and no overlap. ⚠️ THE NUMBERS HERE ARE THE LEVIES ALONE, NOT A FULL PRICE - the add-on adds them to whatever your price source publishes, exactly as it adds the flat figure. Why it matters: a flat figure is the same in every hour, so it moves every hour equally and cannot change which hour is cheapest - and a night network rate is cheap in the same hours the market is cheap, so flattening it makes the add-on think the day is flatter than it is and turn down battery cycles that would have paid for themselves. Set this or the flat figure above, not both: the add-on refuses to start if the two disagree."
price_formula:
name: "Tariff · Your own tariff formula (one line of JSON)"
description: "For price source custom_formula only: your own contract's two coefficients, written as a single line of JSON. Your supplier's tariff card prints the formula as B x index + A - those two numbers exist only on your card, there is no public index of them anywhere, and you are the only person who can read them. Type what the card prints, IN THE UNITS THE CARD PRINTS: every unit conversion and the VAT step are the add-on's job, because those are the two places a human silently loses a factor of 10, 100 or 1.06. Example: {\"index\":\"epex_be\",\"resolution\":\"hourly\",\"factor\":0.102,\"offset\":1.00,\"units\":\"c_eur_per_kwh\",\"vat_pct\":6,\"card\":{\"month\":\"2026-09\",\"index_eur_per_mwh\":133.9905,\"expect_incl_vat\":15.55}}. ⚠️ THE card BLOCK IS REQUIRED AND IT IS WHAT MAKES THIS SAFE: your card also prints the reference index it assumed and the price that comes out, so the add-on recomputes that price from your numbers and REFUSES TO START if it does not come out, telling you by how much. That catches a wrong unit, VAT left out or counted twice, a transposed digit, and the expensive one - network charges folded into the offset, which the add-on would then bill you for a second time. ⚠️ AND ITS LIMIT, IN THE SAME BREATH: it checks your coefficients against your card, not your card against reality, so two numbers copied off the wrong product agree with each other and pass. Fixed numbers only - no expression of any kind is evaluated, and an unrecognised field is refused rather than ignored."
price_csv_path:
name: "Tariff · Price CSV file"
description: "For price source manual_csv only: where in /share the add-on reads your own price file from."
price_feed_url:
name: "Tariff · Price feed URL override"
description: "Leave empty to use the chosen source's own feed. Only a host that source itself declares is accepted, so this cannot be pointed anywhere."
description: "Leave empty to use the chosen source's own feed. Only a host that source itself declares is accepted, so this cannot be pointed anywhere. WHAT YOU TYPE IS SENT EXACTLY AS WRITTEN: the add-on adds nothing to it, not even the date range it normally asks its own feed for, so an override gets whatever horizon that URL returns - often a single day, with the rest of the plan carried over from yesterday and excluded from price arbitrage. A warning in the log says so on every price refresh. Clear this option to get the full horizon back."
price_horizon_h:
name: "Tariff · How far ahead to fetch prices (hours)"
description: "How many hours of prices to ask the source for."
@@ -176,7 +182,7 @@ configuration:
description: "The average the current price is compared against. Only meaningful together with the current-price entity above, and like it, may not be combined with a dynamic price source."
arbitrage_enabled:
name: "Tariff · Trade on the price spread"
description: "Charge in the cheapest published hour and discharge in the dearest, but only when the spread covers what the round trip costs. OFF in the shipped configuration, deliberately: this is the first strategy in the add-on that moves energy against the meter on purpose, so it is armed on purpose or not at all. It needs both a price source and a P1 meter, and the add-on refuses the combinations that cannot work rather than letting them look armed."
description: "Charge in the cheapest published hour and discharge in the dearest, but only when the spread covers what the round trip costs. OFF in the shipped configuration, deliberately: this is the first strategy in the add-on that moves energy against the meter on purpose, so it is armed on purpose or not at all. ⚠️ LEAVING IT OFF DOES NOT TURN THE PRICE FEED OFF. A price source selected above is fetched, checked and recorded either way; this switch decides only whether the add-on ACTS on those prices. It needs both a price source and a P1 meter, and the add-on refuses the combinations that cannot work rather than letting them look armed."
cycle_cost_threshold_eur:
name: "Tariff · Cost of one full charge/discharge cycle (EUR/kWh)"
description: "What one full round trip costs per kWh: wear on the pack plus conversion losses. A price spread smaller than this loses money and battery life, so it is not traded. ⚠️ THIS ONE IS NOT DERIVED FROM A MEASUREMENT. 0.10 is a product decision and it is still an open question with the owner (OPT-08). At this value a fixed day/night site does not trade at all - the realistic Belgian day/night spread is about 0.08 EUR/kWh - and that is the correct verdict at this number, not a fault."
+12 -6
View File
@@ -123,25 +123,31 @@ configuration:
description: "AFGELEID, niet gekozen (SAFETY-24): de referentiebatterij heeft 13,8 kWh bruikbaar en de zwaarste doorgerekende afvlakdag vraagt 878 Wh, oftewel 6,4 %, dus 20 % ligt tien punten boven de uitschakelgrens van de batterij zelf en laat de bewaking nog ongeveer 1,6 van haar zwaarste waargenomen dagen speling. ⚠️ DIT BEGRENST ALLEEN DE PIEKBEWAKING. Het is geen reserve voor de hele batterij: gewoon eigenverbruik ontlaadt er dwars doorheen en gaat door tot aan de uitschakelgrens van de batterij, en dit getal verhogen verandert daar geen tiende procent aan. Het moet strikt boven de uitschakelgrens van de batterij blijven, anders start de add-on niet op. Verander het niet zonder de afleiding opnieuw te maken."
price_provider:
name: "Tarief · Bron van dynamische prijzen"
description: "Waar day-ahead- of realtimeprijzen vandaan komen. 'off' is de standaardinstelling en betekent dat deze installatie op het vaste tarief hieronder draait, zonder handel. ⚠️ Dit vandaag aanzetten haalt prijzen op, controleert en publiceert ze, en verandert geen enkele energiebeslissing - er is nog niets dat de prijsreeks gebruikt. Het mag niet gecombineerd worden met de twee oude Home Assistant-prijsentiteiten verderop: twee prijsbronnen op één installatie, waarvan er stilzwijgend één wint, wordt bij het opstarten geweigerd in plaats van schijnbaar actief gelaten."
description: "Waar day-ahead- of realtimeprijzen vandaan komen. 'off' is de standaardinstelling en betekent dat deze installatie op het vaste tarief hieronder draait, zonder handel. ⚠️ DIT AANZETTEN IS WAT HET OPHALEN VAN PRIJZEN START - niet de handelsschakelaar verderop. Met een bron ingesteld haalt de add-on de feed elk half uur op, controleert die, bewaart de dag en toont op de rij Prijsgeschiedenis wat er bewaard is, of prijshandel nu aan staat of niet. Op zichzelf verplaatst dat nog geen energie: handelen is de aparte schakelaar, en het goedkope venster dat de maandelijkse onderhoudscyclus gebruikt komt uit het vaste of dag/nacht-tarief hieronder. Het mag niet gecombineerd worden met de twee oude Home Assistant-prijsentiteiten verderop: twee prijsbronnen op één installatie, waarvan er stilzwijgend één wint, wordt bij het opstarten geweigerd in plaats van schijnbaar actief gelaten. ⚠️ DRIE IDS IN DEZE LIJST ZIJN INGETROKKEN EN STARTEN NIET MEER: engie_dynamic, totalenergies_dynamic en eneco_eddy. Hun endpoints bestaan niet en hebben nooit bestaan, dus die plugins hebben nooit één prijs opgehaald - een installatie die op zo'n bron stond, draait sindsdien stilzwijgend op het vaste of dag/nacht-tarief hieronder, of op niets. Ze staan hier alleen nog zodat zo'n installatie bij het opstarten die uitleg krijgt in plaats van een onleesbare schemafout. Kies generic_dayahead voor de groothandelscurve met een vaste marge, custom_formula om de coëfficiënten B x index + A van uw eigen contract van uw tariefkaart over te typen, manual_csv voor een bestand dat u zelf schrijft, of off."
price_markup_eur_kwh:
name: "Tarief · Marge van de leverancier (EUR/kWh)"
description: "De marge van uw energieleverancier. De add-on telt die ALLEEN op wanneer de gekozen prijsbron ze niet al bevat - bij engie_dynamic, totalenergies_dynamic, eneco_eddy en manual_csv wordt ze genegeerd, en dat wordt ook gelogd, zodat ze nooit dubbel wordt aangerekend."
description: "De marge van uw energieleverancier. De add-on telt die ALLEEN op wanneer de gekozen prijsbron ze niet al bevat - bij manual_csv, waarvan de bedragen alles inbegrepen zijn, wordt ze genegeerd, en dat wordt ook gelogd, zodat ze nooit dubbel wordt aangerekend."
price_taxes_eur_kwh:
name: "Tarief · Heffingen en btw (EUR/kWh)"
description: "Belastingen en heffingen per kWh, opgeteld volgens dezelfde regel als de marge: alleen wanneer de gekozen prijsbron ze nog niet bevat."
name: "Tarief · Distributie, heffingen en btw (EUR/kWh)"
description: "Alles op uw factuur dat niet de energie zelf is - distributie, heffingen, accijns en btw - per kWh, opgeteld volgens dezelfde regel als de marge: alleen wanneer de gekozen prijsbron ze nog niet bevat. ⚠️ DIT GETAL IS GEEN VOORKEUR EN HET IS NIET DAT VAN UW OMVORMER. Het staat standaard op 0.11736, AFGELEID in TARIFF-18 van de Eneco-kaart voor residentiele klanten in Vlaanderen van september 2026, inclusief 6% btw: Fluvius-distributie voor Antwerpen 5,35 plus bijzondere accijns 4,876 plus bijdrage groene stroom en WKK 1,51 c/kWh. De energiebijdrage is sinds 1 augustus 2026 nul - ze is opgegaan in de accijns, dus een bedrag uit een oudere gids telt ze dubbel. ⚠️ CONTROLEER HET BIJ UW EIGEN DISTRIBUTIENETBEHEERDER. Alleen de distributieregel verschilt: over de acht Fluvius-netgebieden loopt die van 5,28 c/kWh in Midden-Vlaanderen tot 6,70 in West-Vlaanderen, zodat het Vlaamse totaal ongeveer van 0.117 tot 0.131 loopt. In Wallonie en Brussel heeft het distributietarief een apart dag- en nachttarief en hoort het hier helemaal niet thuis - gebruik dan de optie voor heffingen per tijdstip hieronder. ⚠️ DIT WIJZIGEN VERANDERT GEEN ENKELE HANDELSBESLISSING: hetzelfde bedrag komt bij elk uur, dus het valt weg uit zowel het goedkoopste uur als het prijsverschil. Wat het wel verandert is elke prijs die u te zien krijgt, en de regel die weigert te ontladen in een uur waarvan de totaalprijs onder nul ligt."
export_price_eur_kwh:
name: "Tarief · Injectievergoeding (EUR/kWh)"
description: "Wat u krijgt voor een kWh die u op het net zet, alles inbegrepen. Eén getal en geen reeks: geen enkele leverancier hier publiceert een dynamische injectiecurve. In België ligt dit vaak dicht bij nul, wat op zich al sterk pleit voor eigenverbruik in plaats van injectie. De waarde mag negatief zijn - op een zomerse middag betaalt u soms om te injecteren."
static_profile:
name: "Tarief · Vast of dag/nacht-tarief (één regel JSON)"
description: "Uw vaste of dag/nacht-contract, geschreven als één regel JSON. AANBEVOLEN ZELFS BIJ EEN DYNAMISCH CONTRACT: hierop valt de add-on terug wanneer de prijsfeed uitvalt, en staat dit leeg dan blijft de installatie bij een dode feed helemaal zonder tarief achter. De periodes moeten elk dagtype precies één keer bedekken, zonder gat en zonder overlap - een gat wordt bij het opstarten geweigerd in plaats van opgevuld, want een verzonnen prijs is er één waarop een strategie zou handelen. Een volledig vast tarief is één periode van 00:00 tot 24:00 op alle dagen. De prijzen zijn ALLES INBEGREPEN; de marge en heffingen hierboven worden er niet bij opgeteld. Let op: een venster van 22:00 tot 07:00 voor ma-vr dekt zaterdagochtend NIET, dus een regel voor het weekend is noodzakelijk en niet decoratief."
price_taxes_profile:
name: "Tarief · Heffingen en btw per dagdeel (één regel JSON)"
description: "Laat dit leeg tenzij uw netbeheerder werkelijk een ander dag- en nachttarief aanrekent - in Vlaanderen is dat niet zo, de Fluvius-distributie is één tarief, en het vaste bedrag hierboven klopt. In Wallonië en Brussel is dat wél zo: ORES rekent 13,27 c/kWh overdag tegen 7,39 's nachts, AIESH 15,17 tegen 8,21, Sibelga 9,39 tegen 7,10. Geschreven als één regel JSON in precies dezelfde vorm als het vaste of dag/nacht-tarief hierboven, met dezelfde regel dat de periodes elk dagtype één keer moeten bedekken zonder gat en zonder overlap. ⚠️ DE BEDRAGEN HIER ZIJN ALLEEN DE HEFFINGEN, GEEN VOLLEDIGE PRIJS - de add-on telt ze op bij wat uw prijsbron publiceert, net zoals bij het vaste bedrag. Waarom het uitmaakt: een vast bedrag is in elk uur hetzelfde, verschuift dus elk uur evenveel en kan niet veranderen welk uur het goedkoopste is - en een nachttarief van het net is goedkoop in dezelfde uren waarin de markt goedkoop is, dus door het plat te slaan lijkt de dag vlakker dan hij is en weigert de add-on batterijcycli die zichzelf zouden terugverdienen. Vul dit in óf het vaste bedrag hierboven, niet allebei: de add-on weigert te starten als de twee elkaar tegenspreken."
price_formula:
name: "Tarief · Uw eigen tariefformule (één regel JSON)"
description: "Alleen voor prijsbron custom_formula: de twee coëfficiënten van uw eigen contract, als één regel JSON. De tariefkaart van uw leverancier drukt de formule af als B x index + A - die twee getallen staan alleen op uw kaart, er bestaat nergens een publieke lijst van, en u bent de enige die ze kan lezen. Typ wat de kaart afdrukt, IN DE EENHEID DIE DE KAART GEBRUIKT: elke eenheidsomrekening en de btw-stap zijn het werk van de add-on, want dat zijn precies de twee plaatsen waar een mens stilzwijgend een factor 10, 100 of 1,06 verliest. Voorbeeld: {\"index\":\"epex_be\",\"resolution\":\"hourly\",\"factor\":0.102,\"offset\":1.00,\"units\":\"c_eur_per_kwh\",\"vat_pct\":6,\"card\":{\"month\":\"2026-09\",\"index_eur_per_mwh\":133.9905,\"expect_incl_vat\":15.55}}. ⚠️ HET BLOK card IS VERPLICHT EN MAAKT DIT VEILIG: uw kaart vermeldt ook de referentie-index die ze veronderstelt en de prijs die eruit volgt, dus de add-on berekent die prijs opnieuw uit uw getallen en START NIET als het resultaat niet klopt - met vermelding van het verschil. Dat vangt een verkeerde eenheid, btw die ontbreekt of dubbel geteld is, een omgewisseld cijfer, en de dure fout: nettarieven die in de offset verwerkt zitten en die de add-on er dan nog een tweede keer bovenop zou rekenen. ⚠️ EN DE GRENS DAARVAN, IN DEZELFDE ZIN: ze toetst uw coëfficiënten aan uw kaart, niet uw kaart aan de werkelijkheid - twee getallen van het verkeerde product kloppen onderling en komen er dus door. Alleen vaste getallen: er wordt geen enkele expressie geëvalueerd, en een onbekend veld wordt geweigerd in plaats van genegeerd."
price_csv_path:
name: "Tarief · CSV-bestand met prijzen"
description: "Alleen voor prijsbron manual_csv: waar in /share de add-on uw eigen prijsbestand leest."
price_feed_url:
name: "Tarief · Afwijkende URL voor de prijsfeed"
description: "Laat leeg om de eigen feed van de gekozen bron te gebruiken. Alleen een host die die bron zelf opgeeft, wordt aanvaard, zodat dit niet zomaar ergens naar kan verwijzen."
description: "Laat leeg om de eigen feed van de gekozen bron te gebruiken. Alleen een host die die bron zelf opgeeft, wordt aanvaard, zodat dit niet zomaar ergens naar kan verwijzen. WAT U HIER TYPT WORDT LETTERLIJK VERSTUURD: de add-on voegt er niets aan toe, ook niet het datumbereik dat ze normaal aan haar eigen feed vraagt. Een afwijkende URL levert dus de horizon op die die URL zelf teruggeeft - vaak één dag, waarna de rest van het plan uit de prijzen van gisteren wordt overgenomen en buiten de prijsarbitrage valt. Bij elke prijsvernieuwing verschijnt daarover een waarschuwing in het logboek. Maak dit veld leeg om de volledige horizon terug te krijgen."
price_horizon_h:
name: "Tarief · Hoe ver vooruit prijzen ophalen (uren)"
description: "Hoeveel uur aan prijzen er bij de bron opgevraagd wordt."
@@ -156,7 +162,7 @@ configuration:
description: "Het gemiddelde waarmee de huidige prijs vergeleken wordt. Alleen zinvol samen met de entiteit hierboven en, net als die, niet te combineren met een dynamische prijsbron."
arbitrage_enabled:
name: "Tarief · Handelen op het prijsverschil"
description: "Laden in het goedkoopste gepubliceerde uur en ontladen in het duurste, maar alleen wanneer het verschil de kosten van de volledige cyclus dekt. STAAT UIT in de standaardinstelling, met opzet: dit is de eerste strategie in de add-on die bewust energie tegen de meter in verplaatst, dus ze wordt bewust aangezet of helemaal niet. Ze heeft zowel een prijsbron als een P1-meter nodig, en de add-on weigert combinaties die niet kunnen werken in plaats van ze schijnbaar actief te laten."
description: "Laden in het goedkoopste gepubliceerde uur en ontladen in het duurste, maar alleen wanneer het verschil de kosten van de volledige cyclus dekt. STAAT UIT in de standaardinstelling, met opzet: dit is de eerste strategie in de add-on die bewust energie tegen de meter in verplaatst, dus ze wordt bewust aangezet of helemaal niet. ⚠️ DIT UIT LATEN STAAN ZET DE PRIJSFEED NIET UIT. Een prijsbron die hierboven gekozen is, wordt hoe dan ook opgehaald, gecontroleerd en bewaard; deze schakelaar bepaalt alleen of de add-on op die prijzen HANDELT. Ze heeft zowel een prijsbron als een P1-meter nodig, en de add-on weigert combinaties die niet kunnen werken in plaats van ze schijnbaar actief te laten."
cycle_cost_threshold_eur:
name: "Tarief · Kostprijs van één volledige laad/ontlaadcyclus (EUR/kWh)"
description: "Wat één volledige cyclus per kWh kost: slijtage van de batterij plus omzettingsverliezen. Een prijsverschil dat hier onder ligt, kost geld en levensduur en wordt dus niet verhandeld. ⚠️ DIT GETAL IS NIET AFGELEID UIT EEN METING. 0,10 is een productbeslissing en staat nog open bij de eigenaar (OPT-08). Bij deze waarde handelt een installatie met een vast dag/nachttarief helemaal niet - het realistische Belgische dag/nachtverschil is ongeveer 0,08 EUR/kWh - en dat is bij dit getal het juiste antwoord, geen fout."