admin/goodwe-addon
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

FCST-04: the carry-forward stops decimating a quarter-hourly feed, and a broken cache stops stopping the add-on
checks / unit (pull_request) Successful in 7m22s
Details

Browse Source 
Rework of the FCST-04 branch against review. Every finding below was proven by
a runnable script before it was touched and re-run after; every check added or
repaired was driven red by a mutation, `__pycache__` purged first (MUTATION.md,
TEST-15). 29/29 test files green.

⚠️ H1 - A SUB-HOURLY FEED WAS DECIMATED TO ONE POINT AN HOUR, AND THE SPREAD
WENT WITH IT. `fill_from_yesterday()` keyed the published set as
`{_utc_hour(p.start): p for p in r.points}` - LAST POINT OF THE HOUR WINS, the
other three dropped. `eneco_eddy` is the Belgian imbalance product and settles
per QUARTER, and `_validate_raw()` accepts any strictly-increasing spacing, so
this is a shape the product supports today. Measured on a 96-point quarter-hourly
feed - ALL FIRM, FULLY PUBLISHED, nothing to carry: the plan reached the
optimiser as 15 points, the arbitrage spread fell from 0.143 to 0.014 EUR/kWh,
and the cheapest "hour" became a `:45` point whose `at()` window runs into the
next one. Not the windowing effect this card is about; a feed losing 84 % of its
own prints, with no flag anywhere reporting it. The fill now decides only WHICH
WHOLE HOURS NEED CARRYING: every published point inside the window passes
through BY IDENTITY, and a carried point is added only for a whole hour with no
published coverage at all. Same feed, same fixture: 60 of 60 firm points kept,
spread 0.134 over the hours still ahead.

⚠️ AND `_audit()` GOT STRICTER, NOT LOOSER - that was the condition on doing it
at all. The old post-condition refused any repeated `_utc_hour`, a count that
cannot express four points in one hour. Two rules replace it and SUBSUME it: a
published point must BE one of that hour's published points by identity and may
be emitted only once; a carried point may stand only in an hour absent from the
published set, must be flagged True, and may not repeat. So the duplicated
carried hour it existed to catch still raises, the same published point emitted
twice now raises, and a carried point sharing an hour with a real print now
raises too - which the count never looked at. Three mutations, three reds.

⚠️ H2 - A DAY RECORD THAT WAS NOT A RECORD STOPPED THE ADD-ON FROM STARTING.
`_load()` validated `version` and that `days` is a dict, never that each VALUE
is one. `{"version": 1, "days": {"2026-01-14": "oops"}}` loaded as ok=True,
last_error=None, and then `rec.get(...)` raised AttributeError out of `words`,
`yesterday()` and `resolve_tariff()` alike. `PriceHistory(...)` and `.words` are
both unguarded in `Controller.__init__`, so the controller did not come up - and
it never self-healed, because `_remember()` raised on the same record before it
could reach `_save()`. That falsifies the CHANGELOG's "A BROKEN CACHE IS NOT A
BROKEN CONTROLLER" in the one direction that matters. The asymmetry that
produced it is visible in the file: `_parse_points()` is carefully defensive
about every field of a POINT and `_load()` was not defensive about the record
holding them. Unusable records are now dropped and REPORTED (`words` says how
many and why), and `words` is total by construction as a second line.

M3 - `yesterday()` WAS OFF BY ONE DAY FOR 00:00-00:59 AFTER SPRING-FORWARD.
`brussels()` renders at the FIXED offset for that instant, so `brussels(stamp) -
timedelta(days=1)` subtracts 24 ABSOLUTE hours - and the Brussels day before the
March transition is 23 hours long. At 2026-03-30T00:30+02:00 it returned
2026-03-28's prices as "yesterday", or `no_data` naming a date that is not
yesterday. The arithmetic is now on the local DATE, which has no offset to lose,
and both transitions have a check - the module's headline claim had none.

M4 - WITH NO /data THE STORE STILL FILLED, IN MEMORY. `_save()` short-circuited
on `not ok` while `_remember()` populated `self._days` anyway: `words` said
"unavailable" and `yesterday()` answered A SERIES that is gone on the next
restart. The module docstring rejects exactly that ("a yesterday that silently
forgets") as its reason for not calling makedirs, so the code now matches the
docstring rather than contradicting it.

M5 - THE PRUNE MUTATION SURVIVED BECAUSE THE STIMULUS WAS DEGENERATE. §8's
clock-jump check seeded ONE day, so `if len(self._days) <= KEEP_DAYS: return`
fired and the body it is named after never ran: pruning by arithmetic against
the clock - which empties the store on exactly the morning the fill is for -
passed the whole 29-file suite. The fixture is now written to a file and
reopened (`_load()` does not prune, which is how a store legitimately holds more
than the cap) with KEEP_DAYS + 2 days, and the mutant is red.

M6 - THE AST ORDERING CHECK COULD NOT CATCH A REVERSAL, AND TWO DOCUMENTS CITED
IT AS A GUARANTEE. `_calls.index("_remember") < _calls.index("yesterday")` grades
`ast.walk` BFS DEPTH: `self._remember(...)` is a statement and
`self.yesterday(...)` is an ARGUMENT of the fill, one level deeper, so
`_remember` came out first whichever statement was written first - shipped and
reversed both scored True. It is now line numbers AND, the half that matters,
the behaviour: a feed carrying yesterday alongside today into an EMPTY store
must fill the whole horizon; reversed, the lookup finds nothing and the plan is
only the published hours. Reversing the two statements is red on both.

L14/L9 - KEEP_DAYS = 3 IS THE STEADY STATE EXACTLY, so a newest-N cap has no
slack: a feed publishing to D+2, or ONE stray point in the wrong year - which
sorts LAST and can never age out - pushes yesterday out for good. `_prune()` now
PINS the two days a lookup can actually ask for on top of the cap. Pinning is
additive, so it can only RETAIN and the WITNESS-03 property (nothing is deleted
by clock arithmetic) is preserved by construction; `stamp` stops being a dead
parameter; the store is bounded at KEEP_DAYS + 2.

L8 - the horizon end was not floored while `since` was, so a 48 h request made
at any instant off the hour returned 49 points. §1's count was green only
because its NOW was exactly :00:00, a shape production never has; the fixture
now sits at :17:43. L10 - a failed `_save()` leaked its temp file; at 48 writes
a day a persistent write failure leaked 48 inodes a day onto the volume that was
already the problem. L15 - `{"version": 1.0}` passed, since `1.0 == 1`. L16 -
the `_remember` tree walk scanned `app/*.py` only and matched a bare string; it
now parses `app/`, `tools/`, `scenarios/` and `estop/` and looks for the NAME,
so a comment does not fail it and a call in `tools/` cannot pass unseen.

CLAIM-ACCURACY DEFECTS, WHICH THIS REPO TREATS AS REAL:

* L7. DOCS said "*Tariff* on the status page reads `provisional` through the
  morning". FALSE - `tariff_words` is assigned only from the HA price-entity
  branch and from `static_status()`, both on the 30 s tick, and the plan's own
  sentence reaches `_LOG.info`. The doc now says what an operator can see (the
  Price history row, the Arbitrage row's own refusal sentence, the refresh log
  line) and a check pins the PAIRING - wiring the plan into the row is fine,
  correcting the doc is fine, doing one without the other is red.
* L11. "fills EVERY unpublished hour" is conditional: a partial yesterday leaves
  the local hours it never held out of the series (§11.4 for those hours, and
  `at()` raises rather than answering). Correct behaviour, overclaimed wording,
  and untested - all three fixed.
* L12. DECISIONS §1 leaned on `TariffPlan.cheap_now()`, which has NO PRODUCTION
  CALLER on the provider plan (`self.cheap` comes from `static_status()`, which
  builds its own). The conclusion holds on `arbitrage()`'s firm-pair refusal
  alone, which does reach control; the evidence is now stated at the reach it
  has.
* L13. §1's "passed through by identity" check compared `eur_per_kwh` between
  two SEPARATE `resolve_tariff()` calls, where `is` could never have held. The
  value comparison is kept and now says what it grades; identity is graded on a
  read the file constructs and still holds.
* L17. `PriceHistory.words` and the new status row had no check at all - and
  `words` is one of the two things H2 crashed. Both now have one.
* L18. main.py's inline note carried the "widens what the optimiser can SEE,
  never what it may act on" guarantee without the windowing caveat that the
  CHANGELOG, DOCS and `resolve_tariff()`'s docstring all carry. It is the one
  place a maintainer reads it unqualified.

EVIDENCE. All eight reviewer proofs re-run: p1/p1b return 29 March, p4 keeps
60 of 60 firm points at spread 0.134, p5 starts and reports, p6 D leaves no
temp, p6 E refuses 1.0, p6 F records nothing, p7 returns 48 for 48. p2's script
still scores True/True because it re-implements the OLD algorithm - the
reversal is now driven as a mutation instead, and is red. p3's script always
showed the shipped code correct; what was wrong was the fixture, and its mutant
is now red. Twenty mutations driven, no survivors, tree fingerprinted and
`__pycache__` purged before each: decimation (2 suites red), audit x3, unfloored
`until` (2), unvalidated day records (7 checks), instant-subtraction yesterday,
in-memory fill on a volume-less store, clock-arithmetic prune, cap without the
pin, leaked temp, loose schema type, empty `words` (5), reversed `carry()` (2),
a `_remember` leak in tools/ spelled two ways, a neighbour-filling fallback, a
copy instead of identity (2), the DOCS claim in both directions, and the caveat
removed from main.py. Baseline and the unmutated tree both green.

No option added, config.yaml untouched, and the CHANGELOG entry still picks no
version: PKG-07's rule, and only the deployer knows whether 0.9.0's image has
been pulled.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
This commit is contained in:
glenn schrooyen
2026-09-09 23:54:23 +02:00
co-authored by Claude Opus 5
parent 0618194d9c
commit b59f5ce31b
9 changed files with 1222 additions and 112 deletions
Download Patch File Download Diff File Expand all files Collapse all files
goodwe_controller/CHANGELOG.md
+39 -9
View File
@@ -34,10 +34,26 @@ spread it would have created was never compared against anything.
**What changes:**
* `app/price_history.py` (new) keeps the **published** prices of the last
`KEEP_DAYS` Brussels days in `/data/price_history.json`, and
`resolve_tariff()` now fills every unpublished hour of the horizon from
yesterday's same-wall-clock hour, flagged provisional.
* `app/price_history.py` (new) keeps the **published** prices of the last few
Brussels days in `/data/price_history.json`, and `resolve_tariff()` now fills
an unpublished hour of the horizon from yesterday's same-wall-clock hour,
flagged provisional. ⚠️ **"An" unpublished hour, not "every" one:** the fill
can carry only an hour yesterday actually published, so a partial yesterday
leaves the local hours it never held **out of the series** - §11.4 for those
hours, and asking for one raises rather than answering. Correct behaviour,
and the wording is now the behaviour.
* ⚠️ **A sub-hourly feed keeps every print.** `eneco_eddy` is the imbalance
product and settles per **quarter**; the fill walks whole hours to decide
what needs carrying, and that is not a rule about what a published hour may
contain. Every published point inside the horizon passes through by identity,
and a carried point is added only for a whole hour with no published coverage
at all. The first draft keyed the published set by hour - last point wins -
and a 96-point quarter-hourly feed reached the optimiser as 15 points with
its spread down from 0.143 to 0.014 EUR/kWh, on an all-firm, fully-published
series.
* The horizon is **whole hours at both ends**: `since` is floored to the hour
that contains it, and so is `until`. Asked for 48 hours at 08:17, the plan is
48 hours long and not 49.
* `resolve_tariff(history=...)` is **required and keyword-only**. A caller with
no durable volume passes the explicit `NoHistory()`. A default of `None`
meaning "skip the fill" would hand a caller who forgot the argument exactly
@@ -64,11 +80,25 @@ difference only the status page can see.
`DECISIONS/FCST-04-the-day-that-is-carried.md` carries that and two more.
⚠️ **A BROKEN CACHE IS NOT A BROKEN CONTROLLER.** No `/data`, an unreadable
file, a schema this build does not know, a malformed point - four states, four
reasons, none of them a price and none of them an empty series. In every one the
plan is still a real provider series and the controller still regulates; the
horizon is just as short as it was before this card. **No option was added and
`config.yaml` is untouched.**
file, a schema this build does not know (including a `version` of `1.0`, which
`== 1`), a **day record that is not a record**, a malformed point - each its own
state with its own reason, none of them a price and none of them an empty
series. In every one the plan is still a real provider series and the controller
still regulates; the horizon is just as short as it was before this card.
`.words` never raises, because `Controller.__init__` reads it unguarded and a
status line that throws is an add-on that does not start.
⚠️ **AND WHERE THERE IS NO `/data`, NOTHING IS RECORDED AT ALL** - not even in
memory. An in-memory yesterday answers for one run and is gone on the next
restart, which is a cache that forgets without saying so; the honest pair is
`unavailable` on the status row and no prices carried.
⚠️ **"YESTERDAY" IS A LOCAL DATE, AND THE ARITHMETIC IS ON THE DATE.**
`brussels()` renders at a fixed offset, so subtracting `timedelta(days=1)` from
it subtracts 24 **absolute** hours - and the Brussels day before a
spring-forward is 23 hours long. Between 00:00 and 00:59 on the morning after
the March transition that lands on the day *before* yesterday. **No option was
added and `config.yaml` is untouched.**
## 0.9.0
goodwe_controller/DECISIONS/FCST-04-the-day-that-is-carried.md
+92 -9
View File
@@ -1,6 +1,6 @@
# FCST-04 — what "yesterday" is, and what the fill is allowed to reach
Three decisions, recorded because each had a defensible alternative that a later
Four decisions, recorded because each had a defensible alternative that a later
reader would otherwise re-open from scratch.
---
@@ -17,11 +17,23 @@ no arbitrage."*
Because it buys nothing the optimiser can use, and it silently replaces a
fallback the operator configured.
Measured, not assumed: a fully-carried series is 100 % provisional, so
`TariffPlan.arbitrage()` finds fewer than two firm hours and refuses on §8.3's
firm-pair rung, and `cheap_now()` returns False because `firm_points()` is
empty. The §11.4 static fallback produces *exactly* those two answers. The two
paths differ only in which numbers the status page shows.
Measured, not assumed — and stated at the reach it actually has:
- **This one reaches control.** A fully-carried series is 100 % provisional, so
`TariffPlan.arbitrage()` finds fewer than two firm hours and refuses on
§8.3's firm-pair rung. That is the verdict the optimiser reads: `arbitrage.py`
calls `plan.arbitrage(...)` on `fresh_tariff_plan()`, and it is the only path
by which a provider plan moves energy. The §11.4 static fallback refuses on
the same rung, for the same reason.
- **This one does not, and saying so is the point.** `TariffPlan.cheap_now()`
also returns False on a fully-carried series, because `firm_points()` is
empty — but it has **no production caller on the provider plan**.
`self.cheap` comes from `static_status()`, which builds its *own* static plan
and calls `cheap_now()` on that. So it is a property of the type, checked in
`test_static_tariff.py`, and it is not evidence about the shipped control
path. The conclusion below holds on the first bullet alone.
The two paths therefore differ only in which numbers the status page shows.
So the choice is between two identical control behaviours, and the tie-breaker
is that `static_profile` is a documented option an operator fills in **for this
@@ -74,9 +86,26 @@ The defence is therefore an **ordering**, not a validation:
- The filled series exists only as that return value. The object never sees it
again.
- `_remember()` is private and called from exactly one place.
- `test_price_history.py` walks `app/` and fails if any other module names
`_remember`, and reads `carry()`'s AST to confirm the record happens before
the lookup — so a later edit that reverses them is red rather than plausible.
- `test_price_history.py` parses every shipped module under `app/`, `tools/`,
`scenarios/` and `estop/` and fails if any of them **uses the name**
`_remember` — an attribute, a bare name, a `def`, or the string constant a
`getattr` would need. By name rather than by substring, so a comment that
merely mentions the rule does not fail it and a call in `tools/` cannot pass
unseen.
- and it pins the order two ways: the **line numbers** of the two calls in
`carry()`, and — the one that matters — the **behaviour**. A feed carrying
yesterday alongside today, into an empty store, must fill the whole horizon;
reversed, the lookup finds nothing and the plan is only the hours the feed
published. Driven, so a reversal is red.
⚠️ **The first version of that ordering check graded nothing, and this file
said it did.** It compared positions in `ast.walk`, which is breadth-first:
`self._remember(...)` is a statement and `self.yesterday(...)` is an *argument*
of `fill_from_yesterday(...)`, one level deeper, so `_remember` came out first
whichever statement was written first. Measured — shipped and reversed both
scored the same. A decorative check that a document cites as a guarantee is
worse than no check, which is why the correction is recorded here rather than
quietly made.
That ordering has a second benefit worth keeping: a feed that returns yesterday
alongside today refreshes yesterday's record from live data *before* it is
@@ -96,3 +125,57 @@ elapsed are no longer in the spread. A cheap night that has gone used to be the
series' cheapest hour, and a cycle cannot buy in an hour that is past — so the
trade was silently never taken. `test_price_history.py` section 2 grades that by
finding the cheapest firm hour with and without the fill and showing it moves.
⚠️ **And section 1's identity check now grades identity.** It used to compare
`eur_per_kwh` between `_before` and `_after`, which come from two *separate*
`resolve_tariff()` calls — each parsing the feed body into its own objects, so
`is` could never have held between them however the fill were written. FCST-03's
whole anti-copy argument is `filled.points[i] is fetched.points[j]`, and floats
cannot grade it. The value comparison is kept and now says what it is (the fill
moved the window, it did not touch a price); identity is graded on a read the
file constructs and still holds a reference to.
---
## 4. A sub-hourly feed passes through whole; the fill only decides which hours are missing.
`eneco_eddy` is the Belgian imbalance product and settles **per quarter**, and
`price.py`'s `_validate_raw()` accepts any strictly-increasing spacing — so a
published hour may legitimately hold four points.
FCST-04's first caller keyed the published set as `{_utc_hour(p.start): p for p
in r.points}`. It reads perfectly ordinary and it keeps the **last** point of
each hour. Measured on a 96-point quarter-hourly feed — all firm, fully
published, nothing to carry: the plan reached the optimiser as **15 points**,
the arbitrage spread fell from **0.143 to 0.014 EUR/kWh**, and the cheapest
"hour" became a `:45` point whose `at()` window runs into the next hour. That is
not the windowing effect above; it is a feed losing 84 % of its own prints, and
no `provisional` flag would ever have reported it.
**The alternative was rejected:** refusing sub-hourly feeds at `_validate_raw()`
would have made the count correct by breaking the one supported product that
legitimately publishes per quarter.
**What was done instead** keeps FCST-03's existing contract rather than weakening
it: *every* published point inside the window passes through **by identity**,
and a carried point is added only for a whole hour with **no** published
coverage at all. A carried hour is still a single hourly point whatever the
feed's own spacing is, because yesterday's profile is only known per wall-clock
hour — and carried hours are provisional, so they are out of the trade set
anyway.
⚠️ **`_audit()` got stricter, not looser, and that was the condition on doing
this at all.** The old post-condition refused any repeated `_utc_hour` — a count
that cannot express four points in one hour. It is replaced by two rules that
**subsume** it:
- a **published** point must *be* one of that hour's published points, by
identity, and may be emitted only once;
- a **carried** point may stand only in an hour that does not appear in the
published set at all, must be flagged `True`, and may not repeat.
So a duplicated carried hour — the DST mispricing the count existed to catch —
still raises; so does the same published point emitted twice; and a carried
point sharing an hour with a real print now raises too, which the count never
looked at. Each of those three is driven by its own mutation in
`test_price_forecast.py`.
goodwe_controller/DOCS.md
+34 -4
View File
@@ -1433,18 +1433,41 @@ day. Until FCST-04 those other hours were simply **absent** from the plan: not
priced badly, not flagged, just missing, and nothing said so.
Now the add-on keeps the **published prices of the last few days** in
`/data/price_history.json` and fills every unpublished hour of the horizon with
`/data/price_history.json` and fills an unpublished hour of the horizon with
yesterday's price for the same Brussels wall-clock hour, flagged
`provisional: true`. Two things follow, and the second one matters more:
- **The plan says it is guessing.** *Tariff* on the status page reads
`provisional` through the morning, because half its hours are. Before, a
half-length plan looked completely settled.
- **The plan says it is guessing** — `provisional: true` on every carried hour,
which is what the optimiser reads. Before, a half-length plan looked
completely settled.
- **Nothing may trade on a carried hour.** §8.3 forbids acting on a provisional
price, so arbitrage drops those hours from the trade set entirely — it needs a
buy hour and a sell hour that were *both* published. Carrying a price forward
widens what the optimiser can **see**; it never widens what it may **do**.
**"An unpublished hour", not "every unpublished hour"** — the fill can only
carry an hour yesterday actually published. If yesterday's record is itself
partial (a first day, an outage, a feed that only ever serves the rest of the
day), the local hours it never held are **left out of the series**: §11.4
applies to those hours, and asking the plan for one raises rather than
answering. An hour with no price is not a free hour, and the fill does not
invent a neighbour, a mean or a zero for it.
**A quarter-hourly feed keeps all four of its prints.** `eneco_eddy` settles
per quarter, and the fill decides *which whole hours need carrying* — never what
a published hour may contain. Every point the provider published inside the
horizon is passed through untouched; a carried point is added only for a whole
hour the feed did not print at all.
**What you can actually see on the status page.** *Price history* (below) says
what is held. *Arbitrage* carries the strategy's own sentence, which names the
firm/provisional split whenever that split is what refused a trade. *Tariff*
does **not** report the fill: it is the static-profile or Home Assistant
price-entity sentence, regenerated on the 30 s tick. The provider plan's own
line — `dynamic_dayahead from generic_dayahead, 48 h, provisional` — goes to the
**add-on log** at every refresh, alongside `§8.3: 33 of 48 hours carried forward
from yesterday's profile and flagged provisional; 15 published`.
The plan is also now cut to the horizon you asked for, which means **hours that
have already elapsed are no longer in it**. That is a real change to arbitrage:
a cheap night that has already gone used to sit in the series as the cheapest
@@ -1458,6 +1481,13 @@ cache that cannot answer is not allowed to be a controller that cannot regulate.
The store is rebuilt by the next successful fetch; on a brand-new install the
first fill is possible the day after the first refresh.
The same applies to a file that is *partly* broken: a day record this build
cannot read is dropped, the row says how many were dropped and why, and the rest
of the file still loads. And where there is no `/data` at all the store records
**nothing** — not even in memory. An in-memory yesterday would answer for one
run and vanish on the next restart, which is a cache that forgets without saying
so; `unavailable` and *no prices carried* are the honest pair.
**Injection is priced separately** (`export_price_eur_kwh`). In Belgium it is
often near zero, which by itself strongly favours self-consumption over export.
It is a single number rather than a curve because no supported supplier
goodwe_controller/app/main.py
+19 -4
View File
@@ -596,12 +596,27 @@ class Controller:
# - typically 10-24 of the 48 hours `price_horizon_h` asked for - and
# `TariffPlan.arbitrage()` took its spread over that truncated window
# without anything reporting that it had. Every carried hour is
# provisional, so refusal 3 drops it from the trade set: this widens
# what the optimiser can SEE, never what it may act on.
# provisional, so refusal 3 drops it from the trade set: CARRYING a
# price widens what the optimiser can SEE, never what it may act on.
#
# ⚠️ AND THE CAVEAT THAT BELONGS WITH THAT SENTENCE, because this is
# the one place a maintainer reads it on its own. THE OTHER HALF OF
# THIS CHANGE DOES MOVE ENERGY, and it is the windowing rather than the
# carrying: the filled series covers `[since, until)`, so hours that
# have ALREADY ELAPSED are no longer in the spread. A cheap night that
# has gone used to sit in the series as the day's cheapest hour, and no
# cycle can buy in an hour that is past - so the trade was quietly
# never taken. `resolve_tariff()`'s docstring, DOCS.md and the CHANGELOG
# all carry this beside the guarantee; it may not appear here without it.
#
# ⚠️ A BROKEN OR ABSENT STORE IS NOT AN ERROR HERE. It answers `no_data`,
# the fill carries nothing, and the plan is exactly the pre-FCST-04
# one - short, but honest and still refusing on the same rules.
# the fill carries nothing, and the plan is the pre-FCST-04 one minus
# the elapsed hours - short, but honest and still refusing on the same
# rules. It must not be an error in the other sense either: a day
# record this build cannot read is DROPPED by `PriceHistory._load()`
# and reported in `.words`, because both the constructor above and the
# `.words` read below are unguarded, and a cache that raises here is an
# add-on that does not start.
self.price_history = PriceHistory(price_history_path)
_LOG.info("§8.3 price history: %s", self.price_history.words)
goodwe_controller/app/price_forecast.py
+106 -27
View File
@@ -48,7 +48,8 @@ So the flag is not passed through anywhere in this file:
the object the provider produced is the object in the output series - so
there is no copy that could lose or gain a flag. `filled.points[i] is
fetched.points[j]` is a checkable property, and test_price_forecast.py
checks it.
checks it. EVERY published point in the window is passed through, not one
per hour: see the sub-hourly note below.
* `_audit()` RE-DERIVES the expected flag for every output point from the set
of hours the feed actually published, and RAISES if any point disagrees.
It is a post-condition, not a filter: it never fixes a flag, because a
@@ -90,6 +91,24 @@ which is the row the spec actually writes for "no price data". If NOTHING can
be produced, the result is `PriceRead.no_data(reason)` and not an empty series,
because an empty series is not constructible on purpose (price.py trap 3).
⚠️ A SUB-HOURLY FEED IS NOT DECIMATED TO ONE POINT AN HOUR, and the hourly walk
below is about WHICH HOURS NEED FILLING - never about what a published hour is
allowed to contain. `eneco_eddy` is the Belgian imbalance product and settles
per QUARTER, `_validate_raw()` accepts any strictly-increasing spacing, and a
published hour may therefore hold four points. So EVERY published point inside
the window is passed through, in order, and a carried point is added only for a
whole hour with NO published coverage at all.
The alternative was measured rather than reasoned about, because FCST-04's
first caller shipped it: keying the published set as `{_utc_hour(p.start): p
for p in r.points}` reads perfectly ordinary and keeps the LAST point of each
hour, dropping the other three. On a 96-point quarter-hourly feed - all firm,
fully published, nothing to carry - the series came out 15 points long, the
arbitrage spread collapsed from 0.143 to 0.014 EUR/kWh, and the cheapest
"hour" became a `:45` point whose `at()` window runs into the next hour. A feed
losing 84 % of its own prints is not a windowing effect and is not something a
provisional flag would ever have reported.
AC2: THE IRREVERSIBLE-ACTION GATE
=================================
@@ -156,6 +175,25 @@ def _utc_hour(instant: datetime) -> datetime:
return u.replace(minute=0, second=0, microsecond=0)
def _published_by_hour(read: PriceRead) -> dict:
"""UTC hour -> the provider's OWN points that start inside it, in order.
⚠️ A TUPLE PER HOUR, NOT A POINT, AND THAT IS THE WHOLE OF THE SUB-HOURLY
RULE. `{_utc_hour(p.start): p for p in r.points}` reads identically, is one
character shorter, and silently keeps one point per hour - the last one
written. The mapping answers two DIFFERENT questions here: "does this hour
need filling at all?" is `u in published`, and "what does the output carry
for it?" is all of `published[u]`. Collapsing them is how a quarter-hourly
feed loses three quarters of its prints.
"""
out: dict = {}
read.match(
series=lambda r: [out.setdefault(_utc_hour(p.start), []).append(p)
for p in r.points],
no_data=lambda _r: [])
return {u: tuple(ps) for u, ps in out.items()}
def _carried(start_utc: datetime, eur_per_kwh: float) -> PricePoint:
"""A point filled from yesterday's profile. §8.3: ALWAYS provisional.
@@ -172,37 +210,69 @@ def _carried(start_utc: datetime, eur_per_kwh: float) -> PricePoint:
def _audit(points, published) -> None:
"""Post-condition: every flag matches the point's TRUE provenance (AC1).
`published` maps a UTC hour to the point the provider produced for it.
`published` maps a UTC hour to the TUPLE of points the provider produced
for it (`_published_by_hour()`). An hour ABSENT from that mapping is an hour
the feed did not print, and a carried point is the only thing that may
stand in it.
The rule, in the two forms an output point can take:
* PUBLISHED - the point must BE one of `published[u]`, by identity, and
may be emitted only once. Its flag is whatever the feed said, never
forced to False: an imbalance print is published AND provisional (§9.2,
TARIFF-02), and a check that "corrected" it would be a bug wearing a
guarantee.
* CARRIED - the hour must not appear in `published` AT ALL, the flag must
be exactly True, and that hour may hold no second point.
⚠️ WHAT REPLACED "AT MOST ONE POINT PER HOUR", AND WHY IT IS STRICTER. This
audit began as a `seen` set: any repeated `_utc_hour` raised, because a
duplicated hour is how a DST day gets mispriced. A bare hour count cannot
express a quarter-hourly feed, where four distinct published points share
one hour legitimately - so the count was replaced by the two rules above,
which subsume it. A repeated CARRIED hour still raises; so does the same
published point emitted twice; and a carried point may no longer share an
hour with a published one, which the count never checked at all.
⚠️ RAISES, NEVER REPAIRS. A module that silently corrects its own
provenance is one whose provenance means nothing; the only safe response to
"this point's flag disagrees with where it came from" is to refuse to emit
the series at all.
"""
seen = set()
carried_hours = set()
emitted = set()
for p in points:
u = _utc_hour(p.start)
if u in seen:
raise ValueError(f"the filled series repeats {u.isoformat()}; a "
f"duplicated hour is how a DST day gets mispriced")
seen.add(u)
src = published.get(u)
prov = Provenance.CARRIED if src is None else Provenance.PUBLISHED
if src is None:
if u in carried_hours:
raise ValueError(
f"the filled series repeats the carried hour "
f"{u.isoformat()}; a duplicated hour is how a DST day gets "
f"mispriced")
carried_hours.add(u)
if p.provisional is not True:
raise ValueError(
f"{u.isoformat()} is {prov.value} and is flagged "
f"provisional={p.provisional!r}. §8.3: an hour filled from "
f"yesterday's profile is provisional whether or not the "
f"number turns out to be right - the flag is a claim about "
f"PROVENANCE, not about accuracy.")
f"{u.isoformat()} is {Provenance.CARRIED.value} and is "
f"flagged provisional={p.provisional!r}. §8.3: an hour "
f"filled from yesterday's profile is provisional whether "
f"or not the number turns out to be right - the flag is a "
f"claim about PROVENANCE, not about accuracy.")
else:
if p is not src:
if not any(p is q for q in src):
raise ValueError(
f"{u.isoformat()} is {prov.value} but the series carries a "
f"COPY of that point rather than the point itself. A copy "
f"is where the provisional flag gets lost; published points "
f"pass through by identity.")
f"{u.isoformat()} is {Provenance.PUBLISHED.value} but the "
f"series carries a COPY of that point rather than the "
f"point itself. A copy is where the provisional flag gets "
f"lost; published points pass through by identity.")
if id(p) in emitted:
raise ValueError(
f"the filled series emits the published point at "
f"{p.start.isoformat()} TWICE. The provider's own points "
f"are strictly increasing (`_validate_raw`), so a repeat "
f"is never a second print - it is a duplicated hour, which "
f"is how a DST day gets mispriced")
emitted.add(id(p))
def fill_from_yesterday(read: PriceRead, *, yesterday: PriceRead,
@@ -217,11 +287,21 @@ def fill_from_yesterday(read: PriceRead, *, yesterday: PriceRead,
Returns a `PriceRead`:
* a series, whose points are the provider's own objects for the hours it
published and `_carried()` points for the rest, ordered by instant;
* a series, carrying the provider's own objects - ALL of them, which on
a quarter-hourly feed is four inside one hour - for the hours it
published, and one `_carried()` point for each remaining whole hour,
ordered by instant;
* `no_data` when neither source can produce a single hour - never an
empty series, which is not constructible (price.py trap 3).
⚠️ THE WINDOW IS WHOLE HOURS AT BOTH ENDS. `since` is floored to the hour
that CONTAINS it, because the current hour is still a tradeable hour; and
so is `until`, which is the half FCST-04's first draft left out. With
`since` floored and `until` compared raw, a 48 h horizon asked for at 08:17
emitted 49 hours - the hour of `until` was still `< until` - and the count
was one too many for every `now` that is not exactly on the hour, which is
every `now` a running controller has.
⚠️ IDEMPOTENT AND RE-RUNNABLE, WHICH IS THE §5.4 LATE-PUBLICATION CONTRACT.
Day-ahead publication at 14-16 h CET can be late or fail, so the fetcher
retries and this runs again against the fresher read. Published hours always
@@ -234,9 +314,7 @@ def fill_from_yesterday(read: PriceRead, *, yesterday: PriceRead,
raise ValueError(f"{name} must be tz-aware; a naive instant is "
f"ambiguous for one hour every October")
published = read.match(
series=lambda r: {_utc_hour(p.start): p for p in r.points},
no_data=lambda _r: {})
published = _published_by_hour(read)
# Yesterday's profile, keyed by BRUSSELS WALL-CLOCK HOUR. First point wins
# when yesterday itself was a 25-hour day and printed local hour 2 twice.
@@ -248,11 +326,12 @@ def fill_from_yesterday(read: PriceRead, *, yesterday: PriceRead,
out = []
u = _utc_hour(since)
end = until.astimezone(timezone.utc)
end = _utc_hour(until)
while u < end:
if u in published:
out.append(published[u]) # BY IDENTITY. No copy.
else:
got = published.get(u)
if got:
out.extend(got) # BY IDENTITY, and ALL of them. No copy,
else: # and no decimation to one an hour.
src = by_local_hour.get(brussels(u).hour)
if src is not None:
out.append(_carried(u, src.eur_per_kwh))
goodwe_controller/app/price_history.py
+190 -25
View File
@@ -53,6 +53,14 @@ layer that writes fine, reads fine, and is thrown away on the next restart.
That is a yesterday that silently forgets, which is worse than one that says it
cannot answer.
⚠️ AND THAT RULE IS CARRIED OUT IN `_remember()`, NOT ONLY IN `_save()`. FCST-04
first shipped it half-done: `_save()` short-circuited on `not self.ok` while
`_remember()` filled `self._days` anyway, so a store with no durable volume
answered `words` = "unavailable" and `yesterday()` = A SERIES - an in-memory
yesterday that works all day and is gone on the next restart, which is exactly
the silently-forgetting cache this rule exists to refuse. A store that cannot
persist now records nothing, and both of its answers say the same thing.
THE FOUR RULES THIS FILE IS SHAPED BY
=====================================
@@ -73,8 +81,20 @@ THE FOUR RULES THIS FILE IS SHAPED BY
public entry point, it remembers the argument and returns the fill, and the
filled series exists only as a return value that `carry()` itself never
sees again. `_remember()` is private, is called from exactly one place, and
`test_price_history.py` walks the tree and fails if any module outside this
file names it.
`test_price_history.py` PARSES every shipped module under `app/`, `tools/`,
`scenarios/` and `estop/` and fails if any of them uses the NAME - an
attribute, a bare name, a def, or the string constant a `getattr` needs. By
name rather than by substring, so a comment about the rule does not fail it
and a call from `tools/` cannot pass unseen.
⚠️ AND THE ORDER IS GRADED BY DRIVING IT, NOT BY READING THE SOURCE. The
first version of that check compared positions in `ast.walk`, which is
breadth-first: `self._remember(...)` is a statement and `self.yesterday(...)`
is an ARGUMENT of the fill, one level deeper, so `_remember` came out first
whichever statement was written first - shipped and reversed scored the
same. What grades it now is the behaviour: a feed carrying yesterday
alongside today, into an EMPTY store, must fill the whole horizon, and
reversed it fills nothing.
2. "YESTERDAY" MEANS YESTERDAY, NOT "THE NEWEST DAY I HAVE".
`newest()` is the natural accessor and it is the one that decays: an add-on
@@ -139,12 +159,20 @@ from .price_forecast import fill_from_yesterday
_LOG = logging.getLogger("goodwe.price.history")
# How many Brussels days of published prices are kept. The fill needs exactly
# one - yesterday - and rule 2 refuses anything older, so this is not a horizon:
# it is the number of days a record may sit unused before pruning removes it,
# and it is small because a bigger cache would only hold days no lookup can ask
# for. Three covers today, yesterday, and the day a fetch spanning midnight
# writes ahead of itself.
# How many Brussels days of published prices the SORT-ORDER cap keeps. The fill
# needs exactly one - yesterday - and rule 2 refuses anything older, so this is
# not a horizon: it is the number of days a record may sit unused before
# pruning removes it, and it is small because a bigger cache would only hold
# days no lookup can ask for. Three covers today, yesterday, and the day a
# fetch spanning midnight writes ahead of itself.
#
# ⚠️ IT IS A CAP, NOT THE RETENTION RULE, AND THREE HAS NO SLACK. The steady
# state is already {yesterday, today, tomorrow}; a feed that publishes to D+2,
# or ONE stray point in the wrong year - which sorts LAST and is therefore
# immortal under a newest-N cap - pushes yesterday out of a three-slot store
# and keeps it out, permanently and silently. So `_prune()` PINS the two days a
# lookup can actually ask for on top of the cap; see its docstring. The store
# is then bounded at KEEP_DAYS + 2 records rather than at KEEP_DAYS.
KEEP_DAYS = 3
SCHEMA = 1
@@ -161,6 +189,44 @@ def _bdate(instant: datetime) -> str:
return brussels(instant).strftime("%Y-%m-%d")
def _yesterday_of(instant: datetime) -> str:
"""The Brussels date of the day BEFORE `instant`'s Brussels day.
⚠️ THE ARITHMETIC IS ON THE LOCAL DATE, NOT ON THE INSTANT, AND THAT IS A
DST DEFECT THIS FILE SHIPPED. `brussels()` renders at the FIXED offset in
force at that instant, so `brussels(stamp) - timedelta(days=1)` subtracts 24
ABSOLUTE hours from a wall clock - and the Brussels day before a
spring-forward is 23 hours long. Measured at 2026-03-30T00:30+02:00, the
first hour of the morning after the March transition: it returned
2026-03-28, so the fill carried a TWO-DAY-OLD profile flagged as yesterday's
(or, with only the last three days held, answered `no_data` naming a date
that is not yesterday).
A calendar day minus one day is `date` arithmetic, and a `date` has no
offset to lose. The 25-hour October day is right for the same reason.
"""
return (brussels(instant).date() - timedelta(days=1)).strftime("%Y-%m-%d")
def _n_points(rec) -> int:
"""How many points a day record holds. 0 for anything that is not one.
⚠️ TOLERANT, AND IT IS THE SECOND LINE OF DEFENCE, NOT THE FIRST. `_load()`
drops a malformed day record and says so, so this should never see one. It
is written this way because `words` is read from `Controller.__init__` and
from every status render: a status LINE that raises takes the add-on down,
and "the cache is broken" must never be able to mean "the controller does
not start". That is exactly what FCST-04 shipped - `{"days": {"2026-01-14":
"oops"}}` gave `ok=True`, `last_error=None`, and an `AttributeError` out of
`words`, `yesterday()` and `resolve_tariff()` alike, with no self-heal
because `_remember()` raised before it could rewrite the file.
"""
if not isinstance(rec, dict):
return 0
pts = rec.get("points")
return len(pts) if isinstance(pts, (list, tuple)) else 0
class PriceHistory:
"""The published price series of the last few Brussels days, in /data.
@@ -214,8 +280,13 @@ class PriceHistory:
"next refresh has written a full day",
self.path, self.last_error)
return
if not isinstance(blob, dict) or blob.get("version") != SCHEMA:
found = blob.get("version") if isinstance(blob, dict) else None
found = blob.get("version") if isinstance(blob, dict) else None
# ⚠️ THE TYPE IS PART OF THE VERSION. `1.0 == 1` in Python, so a float
# written by some other tool would pass a bare `!=` and be read as
# schema 1 - a version check that cannot tell 1 from 1.0 is one that
# would not notice a writer this build has never met.
if not isinstance(blob, dict) or isinstance(found, bool) \
or not isinstance(found, int) or found != SCHEMA:
self.last_error = f"schema {found!r}, not {SCHEMA}"
_LOG.warning("price history at %s has %s - discarding it rather "
"than guessing what its points mean",
@@ -226,11 +297,55 @@ class PriceHistory:
self.last_error = "no `days` object"
_LOG.warning("price history at %s has %s", self.path, self.last_error)
return
self._days = days
self._days = self._usable_days(days)
def _usable_days(self, days: dict) -> dict:
"""The day records that are shaped like day records. The rest are dropped.
⚠️ THE ASYMMETRY THAT PRODUCED THE DEFECT: `_parse_points()` below is
carefully defensive about every field of a POINT, and `_load()` was not
defensive about the record holding them at all. `{"version": 1, "days":
{"2026-01-14": "oops"}}` therefore loaded as `ok=True`,
`last_error=None` - and then `rec.get(...)` raised `AttributeError` out
of `words`, `yesterday()` and `resolve_tariff()`. `PriceHistory(...)`
and `.words` are both unguarded in `Controller.__init__`, so THE ADD-ON
DID NOT START; and it never self-healed, because `_remember()` raised on
the same record before it could reach `_save()`.
A broken cache is a short horizon. It is not a controller that will not
regulate a battery, and §6's rule - absent machinery must not disable
the loop - does not stop applying because the machinery is a JSON file.
Dropped, not repaired: an unusable record has nothing in it to keep, and
the next `_remember()` writes a fresh one under that date.
"""
kept, dropped = {}, []
for key, rec in days.items():
if not isinstance(key, str) or not isinstance(rec, dict) \
or not isinstance(rec.get("points"), list):
dropped.append(str(key))
continue
kept[key] = rec
if dropped:
# Recorded, not silent: `yesterday()` and `words` both report
# `last_error`, and a cache that quietly discards half a file reads
# to an operator exactly like a normal first day.
self.last_error = (f"{len(dropped)} unusable day record(s) dropped "
f"({', '.join(sorted(dropped))})")
_LOG.warning("price history at %s: %s - the controller starts "
"anyway and the horizon is as short as the feed makes "
"it", self.path, self.last_error)
return kept
def _save(self) -> None:
if not self.ok:
return
# ⚠️ THE TEMP FILE IS REMOVED ON EVERY FAILING PATH. A write that dies
# between `mkstemp` and `os.replace` used to leave its temp behind, and
# the refresh task runs 48 times a day: a persistent write failure -
# a full volume being the likeliest cause - leaked 48 inodes a day onto
# the volume that was already the problem.
tmp = None
try:
fd, tmp = tempfile.mkstemp(dir=os.path.dirname(self.path))
with os.fdopen(fd, "w", encoding="utf-8") as fh:
@@ -238,9 +353,16 @@ class PriceHistory:
# Atomic: a power cut mid-write must not leave a truncated file
# that parses as an empty day.
os.replace(tmp, self.path)
tmp = None # it IS the file now; do not unlink
except OSError as err:
self.last_error = f"could not write ({err})"
_LOG.error("could not persist price history: %s", err)
finally:
if tmp is not None:
try:
os.unlink(tmp)
except OSError: # nothing further to try, and a
pass # failed cleanup is not a fault
# -- the one public entry point -----------------------------------------
def carry(self, read: PriceRead, *, since: datetime, until: datetime,
@@ -252,12 +374,15 @@ class PriceHistory:
refreshes yesterday's record from live data before it is consumed as
the profile; and the filled series - the one thing that must never be
recorded - exists only as this method's return value, which this object
never sees again. See rule 1.
never sees again. See rule 1, including how that order is graded.
Returns whatever `fill_from_yesterday()` returns: a series over
[since, until) whose published hours are the provider's own point
objects and whose remaining hours are carried and flagged, or
`no_data` when neither source can produce a single hour.
Returns whatever `fill_from_yesterday()` returns: a series over the
whole hours of [since, until), carrying EVERY point the provider
published inside it - by identity, and four of them in one hour on a
quarter-hourly feed - plus one carried point for each remaining whole
hour FOR WHICH YESTERDAY HELD THAT WALL-CLOCK HOUR. An hour neither
source can price is left out (§11.4 for that hour), and `no_data` comes
back when that is every hour.
"""
self._remember(read, stamp)
return fill_from_yesterday(read, yesterday=self.yesterday(stamp),
@@ -274,6 +399,15 @@ class PriceHistory:
nothing outside this file names it, and `test_price_history.py` checks
the tree for that rather than trusting the underscore.
"""
if not self.ok:
# ⚠️ A STORE THAT CANNOT PERSIST RECORDS NOTHING. See the module
# docstring's third warning: filling `self._days` here would give a
# site with no /data an in-memory yesterday that answers a series
# all day and is gone on the next restart - a cache that forgets
# without saying so, which is the one thing that rule refuses. The
# honest degraded state is `words` = "unavailable" and `yesterday()`
# = `no_data`, and those two must agree.
return
points = read.match(series=lambda r: r.points, no_data=lambda _r: ())
if not points:
return
@@ -290,7 +424,8 @@ class PriceHistory:
# Rule 4: MERGE by instant. A later read updates the hours it
# carries and leaves the rest; a revised price for an hour already
# held wins, because it is the newer statement about that hour.
held = {q["start"]: q for q in rec.get("points", [])
raw = rec.get("points")
held = {q["start"]: q for q in (raw if isinstance(raw, list) else ())
if isinstance(q, dict) and "start" in q}
held[p.start.isoformat()] = p.canonical()
rec["points"] = sorted(held.values(), key=lambda q: q["start"])
@@ -303,16 +438,34 @@ class PriceHistory:
len(points), ", ".join(sorted(touched)), len(self._days))
def _prune(self, stamp: datetime) -> None:
"""Keep the newest KEEP_DAYS records. Older ones can never be a lookup.
"""Drop records no lookup can reach. A sort-order cap, plus two pins.
Pruning is by SORT ORDER, not by arithmetic against `stamp`: a clock
that jumped - the failure mode WITNESS-03 spent a session on - would
⚠️ NOTHING IS DELETED BY ARITHMETIC AGAINST THE CLOCK. A clock that
jumped - the failure mode WITNESS-03 spent a session on - would
otherwise delete every record in one pass and leave the fill with
nothing on the morning after, which is the one morning it is for.
Deleting on a bad clock is unrecoverable; holding a few days too many
is not, so the cap is `sorted(...)[-KEEP_DAYS:]` and stays that way.
⚠️ AND A CAP ALONE IS DEFEATED BY ONE POINT IN THE WRONG YEAR. Sorting
is lexicographic on the date string, so a stray far-future record sorts
LAST and is immortal - it consumes a slot for good, and with the steady
state already {yesterday, today, tomorrow} filling all three, yesterday
is what falls out. A feed that publishes to D+2 does the same thing
without any stray at all. So the two days `yesterday()` can ACTUALLY be
asked for are pinned on top of the cap.
`stamp` is READ here, which it was not before: the pin is the only use
of it, and it is additive. Pinning can only RETAIN a record, never
delete one, so a jumped `stamp` pins two dates that are not held and
changes nothing - the WITNESS-03 property is preserved by construction
rather than by remembering to check it.
"""
if len(self._days) <= KEEP_DAYS:
return
for day in sorted(self._days)[:-KEEP_DAYS]:
keep = set(sorted(self._days)[-KEEP_DAYS:])
keep |= {_bdate(stamp), _yesterday_of(stamp)}
for day in [d for d in self._days if d not in keep]:
del self._days[day]
def yesterday(self, stamp: datetime) -> PriceRead:
@@ -327,7 +480,7 @@ class PriceHistory:
f"yesterday({stamp!r}): a naive datetime is not an instant, "
f"and 'which Brussels day was yesterday' is exactly the "
f"question a wall-clock reading cannot answer.")
want = _bdate(brussels(stamp) - timedelta(days=1))
want = _yesterday_of(stamp)
if self.last_error and not self._days:
return PriceRead.no_data(
f"the price history at {self.path} is {self.last_error}, so "
@@ -363,14 +516,26 @@ class PriceHistory:
# -- observability -------------------------------------------------------
@property
def words(self) -> str:
"""One line for the status page. Says what is held, or why nothing is."""
"""One line for the status page. Says what is held, or why nothing is.
⚠️ THIS MUST NOT RAISE, EVER. It is read once from `Controller.__init__`
and again on every status render, and neither call site guards it - so
an exception here is an add-on that does not start. It went out that
way: a day record that was a string instead of a dict raised
`AttributeError` from the `len(...)` below. `_load()` now drops such a
record and `_n_points()` is total, so this has two independent reasons
not to fail on a broken file.
"""
if not self.ok:
return f"unavailable - {self.last_error}"
if not self._days:
why = f" - {self.last_error}" if self.last_error else ""
return f"empty{why} - nothing to carry forward yet"
return "; ".join(f"{d} ({len(self._days[d].get('points') or ())} h)"
for d in sorted(self._days))
held = "; ".join(f"{d} ({_n_points(self._days[d])} h)"
for d in sorted(self._days, key=str))
# A file that PARTLY loaded says so, or the dropped days read as days
# that were simply never written.
return held + (f" [{self.last_error}]" if self.last_error else "")
def _parse_points(raw) -> tuple:
goodwe_controller/app/static_tariff.py
+8
View File
@@ -1053,6 +1053,14 @@ async def resolve_tariff(opts: dict, since: datetime, until: datetime,
a difference only the status page can see. See price_history.py's closing
note and DECISIONS/FCST-04-the-day-that-is-carried.md.
⚠️ THE FILL IS PER WHOLE HOUR AND ITS COVERAGE IS CONDITIONAL. Every point
the provider published inside the window passes through BY IDENTITY - four
of them in one hour on a quarter-hourly product like `eneco_eddy` - and a
carried point is added only for a whole hour the feed did not print at all
AND yesterday did. An hour neither source can price is left out, which is
§11.4 for that hour: `plan.price_at()` raises rather than answering, and an
hour with no price is not a free one.
⚠️ THE FILLED SERIES IS WINDOWED TO [since, until), AND THAT IS A CHANGE.
`load_prices()` returns whatever the feed sent, past hours included, and
`TariffPlan.arbitrage()` took its spread over all of it - so an hour that
goodwe_controller/test_price_forecast.py
+80 -2
View File
@@ -272,16 +272,19 @@ check("the provenance audit REFUSES a carried hour flagged non-provisional - "
"the 'stale but plausible' point, which is SAFETY-18's frozen-but-fresh "
"meter one layer up",
raises(lambda: _fc._audit([_stale_copy], {}), ValueError))
# ⚠️ `published` MAPS AN HOUR TO A TUPLE, not to a point (FCST-04 H1). An hour
# may legitimately hold four published points - see the quarter-hourly block
# below - so the audit takes the whole set and checks membership by identity.
check("...and refuses a published hour that arrived as a COPY rather than as "
"the published object, which is where a flag gets lost",
raises(lambda: _fc._audit(
[PricePoint(start=TODAY_PART.points[0].start,
eur_per_kwh=TODAY_PART.points[0].eur_per_kwh,
provisional=TODAY_PART.points[0].provisional)],
{D0: TODAY_PART.points[0]}), ValueError))
{D0: (TODAY_PART.points[0],)}), ValueError))
check("...and never repairs a flag: the audit's only outcomes are silence and "
"an exception, so a series it passed was already correct",
_fc._audit(list(FILLED.points), {u: TODAY_PART.points[i]
_fc._audit(list(FILLED.points), {u: (TODAY_PART.points[i],)
for i, u in enumerate(DAY[:PUBLISHED_N])})
is None)
@@ -298,6 +301,81 @@ check("a PUBLISHED point that the feed itself flagged provisional keeps that "
and all(p.provisional for p in _eddy_filled.points[:2])
and _eddy_filled.points[0] is _eddy.points[0])
# ⚠️ AND A SUB-HOURLY FEED IS NOT DECIMATED TO ONE POINT AN HOUR (FCST-04 H1).
# `eneco_eddy` settles per QUARTER and `_validate_raw()` accepts any
# strictly-increasing spacing, so a published hour can hold four points. The
# fill walks WHOLE HOURS to decide what needs carrying; that must not become a
# rule about what a published hour may contain. Measured on FCST-04's first
# draft, which keyed the published set `{_utc_hour(p.start): p ...}`: a
# 96-point feed came out 15 points long and its arbitrage spread fell from
# 0.143 to 0.014 EUR/kWh, on an ALL-FIRM, fully-published series.
_QN = 16 # four hours of quarters
_QUARTERS = PriceRead.series(
[PricePoint(start=brussels(D0 + timedelta(minutes=15 * n)),
eur_per_kwh=round(0.100000 + 0.011300 * n, 6),
provisional=False)
for n in range(_QN)],
PriceProduct.REALTIME, "eneco_eddy", D0)
_QHOURS = {_fc._utc_hour(p.start) for p in _QUARTERS.points}
check(f"the quarter-hourly fixture really is sub-hourly - {_QN} points across "
f"{len(_QHOURS)} hours - or every check below passes for an hourly feed "
f"too and grades nothing",
len(_QUARTERS.points) == _QN and len(_QHOURS) == _QN // 4)
_qfill = derived(lambda: fill_from_yesterday(_QUARTERS, yesterday=YESTERDAY,
since=D0, until=D1, stamp=D0))
_qfirm = derived(lambda: [p for p in _qfill.points if not p.provisional])
check(f"every one of the {_QN} published points survives the fill BY IDENTITY "
f"({len(_qfirm)} kept). One-per-hour keying would leave {_QN // 4}, and "
f"the spread over the hours the feed actually published would collapse "
f"with them",
len(_qfirm) == _QN
and all(any(p is q for q in _QUARTERS.points) for p in _qfirm))
check(f"...and the {len(DAY) - len(_QHOURS)} whole hours it published NOTHING "
f"for are carried, ONE point each: a carried hour is hourly whatever the "
f"feed's spacing is, because yesterday's profile is only known per "
f"wall-clock hour",
_qfill.kind == "series"
and len([p for p in _qfill.points if p.provisional])
== len(DAY) - len(_QHOURS))
check("...and the filled series is still ordered by instant across the two "
"kinds, so `at()` and `window()` are not reading a shuffled list",
_qfill.kind == "series"
and [p.start.astimezone(UTC) for p in _qfill.points]
== sorted(p.start.astimezone(UTC) for p in _qfill.points))
_PUB_Q = _fc._published_by_hour(_QUARTERS)
# ⚠️ `derived`, NOT A RAW CALL. Under the decimating mutation this check
# exists for, `_audit` RAISES here - and a harness that dies on the mutation it
# is grading reports a traceback instead of a FAIL, which MUTATION.md forbids.
check("the audit ACCEPTS four published points inside one hour, which is the "
"case the repeated-hour COUNT it replaced could not express",
derived(lambda: _fc._audit(list(_QUARTERS.points), _PUB_Q)) is None)
check("...and still REFUSES a repeated CARRIED hour, which is the DST "
"mispricing that count existed to catch",
raises(lambda: _fc._audit([_fc._carried(D0, 0.11),
_fc._carried(D0, 0.12)], {}), ValueError))
check("...and refuses the SAME published point emitted twice - the provider's "
"points are strictly increasing, so a repeat is never a second print",
raises(lambda: _fc._audit([_QUARTERS.points[0], _QUARTERS.points[0]],
_PUB_Q), ValueError))
check("...and refuses a CARRIED point standing in an hour the feed DID "
"publish, which the old count never looked at",
raises(lambda: _fc._audit([_fc._carried(D0, 0.11)], _PUB_Q), ValueError))
# ⚠️ THE HORIZON IS WHOLE HOURS AT BOTH ENDS (FCST-04 L8). `since` is floored
# and `until` was not, so a 48 h request made at any instant off the hour came
# back 49 hours long. Graded at an OFF-THE-HOUR instant, because on the hour
# the two implementations agree and the check would grade nothing.
_off = D0 + timedelta(hours=8, minutes=17, seconds=43)
for _h in (1, 12, 48):
_n = derived(lambda h=_h: len(fill_from_yesterday(
TODAY_PART, yesterday=YESTERDAY, since=_off,
until=_off + timedelta(hours=h), stamp=_off).points))
check(f"a {_h} h horizon asked for at {_off.isoformat()} - 17 min 43 s "
f"past the hour, the shape a running controller always has - is "
f"{_h} points long, not {_h + 1}",
_n == _h)
# The structural half: the constructor has nothing to pass, and the file is
# scanned so the pass-through idiom cannot come back looking ordinary.
check("_carried() has NO `provisional` parameter, so a caller has nothing to "
goodwe_controller/test_price_history.py
+654 -32
View File
@@ -25,9 +25,14 @@ cache cannot decay by being fed its own output.
that stays true while the NUMBER rots. Graded by driving `carry()` twice
and reading the file back.
AC4 A STORE THAT CANNOT ANSWER SAYS SO. No file, no record for yesterday, an
unreadable file, a record whose points are malformed - four different
unreadable file, a schema this build does not know, A DAY RECORD THAT IS
NOT A RECORD, a record whose points are malformed - six different
reasons, none of them a price, none of them an empty series, and the
controller keeps its pre-FCST-04 plan through all four.
controller keeps its pre-FCST-04 plan through all of them. ⚠️ THE
CRASHING ONE IS THE POINT: `{"days": {"2026-01-14": "oops"}}` loaded as
healthy and then raised `AttributeError` out of every accessor, and both
`PriceHistory(...)` and `.words` are unguarded in `Controller.__init__`
- so the add-on did not start, and never self-healed.
THE RULES THIS FILE HOLDS ITSELF TO, because the reviewer will:
@@ -49,13 +54,33 @@ THE RULES THIS FILE HOLDS ITSELF TO, because the reviewer will:
- THE MERGE FIXTURE MUST REVISE A PRICE TO A DIFFERENT NUMBER, or merge
and replace are indistinguishable.
- THE PRUNE FIXTURE MUST HOLD MORE DAYS THAN KEEP_DAYS, or pruning is not
exercised at all.
exercised at all. ⚠️ THIS FILE BROKE ITS OWN RULE AND IT COST A LIVE
MUTATION: §8's clock-jump check seeded ONE day, `_prune`'s early
return fired, and the body it is named after never ran. The mutant that
pruned by arithmetic against the clock - which empties the store on
exactly the morning the fill is for - survived the whole 29-file suite.
The fixture is now WRITTEN TO A FILE and reopened, because `_load()`
does not prune and that is how a store legitimately holds more days
than the cap.
- NOW MUST NOT BE ON AN HOUR BOUNDARY. `datetime.now()` never is, and a
horizon count taken at exactly :00:00 was green for a fill that emitted
HORIZON + 1 hours at every other instant.
- A SUB-HOURLY FIXTURE MUST REALLY BE SUB-HOURLY, or "every published
print survives" passes for an hourly feed and grades nothing.
* A FAILED READ IS NOT AN OBSERVATION. Every degraded store is graded on the
TYPE it answers with (`no_data` and its reason), never on a falsy value,
and the malformed-point case is checked for the specific defect - a price
defaulted to 0.0 - rather than for a count.
* AND A STRUCTURAL CHECK MUST BE ABLE TO FAIL. §3's ordering check compared
`ast.walk` positions, which grade NESTING DEPTH: `self._remember(...)` is a
statement and `self.yesterday(...)` is an argument of the fill, so
`_remember` came out first whichever statement was written first - measured
by reversing them and watching the check stay green, while DECISIONS §3 and
the module docstring both cited it as making a reversal "red rather than
plausible". It is now line numbers AND the behaviour itself.
⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run
`tools/mutation_run.sh`, which does. See MUTATION.md and TEST-15.
"""
@@ -72,7 +97,7 @@ from datetime import datetime, timedelta, timezone
from app import price_history as _ph
from app.price import (
PricePoint, PriceProduct, PriceRead, brussels,
Absence, PricePoint, PriceProduct, PriceRead, brussels,
)
from app.price_history import KEEP_DAYS, NoHistory, PriceHistory, SCHEMA
from app.static_tariff import resolve_tariff
@@ -111,6 +136,42 @@ def raises(fn, exc):
return False
class _Raised:
"""What a check saw instead of a value. Equals nothing, is falsy, and is
NOT a default."""
def __init__(self, e):
self.e = e
def __eq__(self, other):
return False
def __hash__(self):
return id(self)
def __bool__(self):
return False
def __repr__(self):
return f"<raised {type(self.e).__name__}: {self.e}>"
def derived(fn):
"""A value a check grades. An exception becomes `_Raised`.
⚠️ NOT A DEFAULT, AND NOT A BARE CALL EITHER. MUTATION.md's rule is that
"it was refused" and "it broke on the way to being refused" must not grade
the same - and the accessors below are exactly the ones the unvalidated
day record used to raise from, so a raw call here would take the harness
down on the mutation it is meant to catch and report nothing at all.
Copied from test_price_forecast.py.
"""
try:
return fn()
except Exception as e: # noqa: BLE001
return _Raised(e)
def run(coro):
return asyncio.run(coro)
@@ -169,13 +230,30 @@ def at_local(bdate: str, hh: int) -> datetime:
return next(u for u in bday_hours(bdate) if brussels(u).hour == hh)
NOW = at_local(TODAY_D, 9)
# ⚠️ AND IT IS NOT ON THE HOUR. A controller's `now` is
# `datetime.now(timezone.utc)`; it is :17:43 past as often as anything else and
# it is never :00:00. The first draft of this file used the exact hour, and
# §1's `_n_after == HORIZON_H` was green only because of it: `since` was
# floored to its hour while `until` was compared raw, so a 48 h request made at
# any other instant came back 49 hours long. A fixture has to have the shape
# production has, or a count grades an alignment the running system never gets.
NOW = at_local(TODAY_D, 9) + timedelta(minutes=17, seconds=43)
# Where the WINDOW starts: the whole hour that contains NOW. The hour a
# controller is standing in is still a tradeable hour, so the fill emits from
# here and not from NOW - and every count below is derived against this rather
# than against the instant.
SINCE_HOUR = NOW.replace(minute=0, second=0, microsecond=0)
# Today's feed, in EUR/MWh, keyed by BRUSSELS local hour and all distinct.
# ⚠️ THE CHEAPEST HOUR IS 03:00 LOCAL - ALREADY ELAPSED AT 09:00 - and that is
# AC2's stimulus, asserted below rather than assumed.
TODAY_MWH = {h: 120.0 + h for h in range(24)}
TODAY_MWH[3] = 5.0 # the cheap night that has already gone
TODAY_MWH[9] = 250.0 # the hour NOW falls in - dear, so the
# cheapest hour AHEAD is strictly
# later than NOW and "it moved to an
# hour still ahead" is exact
TODAY_MWH[19] = 260.0 # the dear evening still ahead
# Yesterday's published prices, DIFFERENT AT EVERY LOCAL HOUR from today's, so
@@ -230,9 +308,16 @@ check(f"a Brussels day is 24 hours long in the fixture helper, on both fixture "
len(TODAY_H) == 24 and len(YDAY_H) == 24
and {brussels(u).hour for u in TODAY_H} == set(range(24)))
check(f"NOW is {NOW.isoformat()} - deliberately NOT on an hour boundary. A "
f"horizon count taken at exactly :00:00 is green for a fill that emits "
f"HORIZON + 1 hours at every instant a controller actually runs at",
(NOW.minute, NOW.second, NOW.microsecond) != (0, 0, 0))
# The feed serves TODAY IN FULL - which is what a day-ahead feed does - so the
# hours it publishes that are still AHEAD are a strict subset of the horizon.
_ahead = [u for u in TODAY_H if u >= NOW]
# ⚠️ AGAINST SINCE_HOUR, NOT NOW: the 09:00 point started before NOW and its
# hour still contains NOW, so it is a published hour of the window.
_ahead = [u for u in TODAY_H if u >= SINCE_HOUR]
check(f"at 09:00 the hours the feed published that have NOT elapsed are a "
f"STRICT SUBSET of the {HORIZON_H} h horizon ({len(_ahead)} of "
f"{HORIZON_H}), so 'the fill widened the plan' and 'the fill did "
@@ -254,8 +339,8 @@ check(f"AC2's stimulus is real: the cheapest hour of the whole published day is
f"({brussels(_cheapest_ahead).hour:02d}:00). Without this the windowed "
f"and unwindowed answers would be the same point and AC2 would grade "
f"nothing",
_cheapest_all != _cheapest_ahead and _cheapest_all < NOW
and _cheapest_ahead >= NOW)
_cheapest_all != _cheapest_ahead and _cheapest_all < SINCE_HOUR
and _cheapest_ahead > NOW)
check(f"KEEP_DAYS is {KEEP_DAYS}, and the prune fixture below files more days "
f"than that, so pruning is exercised rather than skipped", KEEP_DAYS < 6)
@@ -314,11 +399,33 @@ check("AC1: exactly the hours the feed published are FIRM, and exactly the "
and len(_got_carried) == HORIZON_H - len(_ahead))
_before_by_start = {p.start.astimezone(UTC): p for p in _before.read.points}
check("...and every published point is the PROVIDER'S OWN OBJECT, passed "
"through by identity - a copy is where the flag gets lost (FCST-03)",
# ⚠️ THIS IS A VALUE CHECK AND SAYS SO. `_before` and `_after` come from two
# separate `resolve_tariff()` calls, each of which parsed the feed body into
# its OWN objects, so `is` could never hold between them however the fill were
# written - a check claiming identity here would be claiming a property its
# own fixture makes untestable. What it does grade is that the fill moved the
# WINDOW and did not touch a PRICE.
check("...and every published point carries the same NUMBER as the unfilled "
"plan's for that instant: the fill changed which hours are in the "
"series, not what any of them costs",
all(p.eur_per_kwh == _before_by_start[p.start.astimezone(UTC)].eur_per_kwh
for p in _after.read.points if not p.provisional))
# ...and identity itself, graded on a read this file constructed and still
# holds a reference to. FCST-03's whole anti-copy argument is
# `filled.points[i] is fetched.points[j]`, and floats cannot grade it.
_h1b = store()
_ph.PriceHistory._remember(_h1b, series_of(YDAY_H, YESTERDAY_MWH), NOW)
_read_id = series_of(TODAY_H, TODAY_MWH)
_filled_id = _h1b.carry(_read_id, since=NOW, until=_until, stamp=NOW)
_firm_id = [p for p in _filled_id.points if not p.provisional]
check(f"...and IDENTITY, graded as identity: all {len(_firm_id)} firm points "
f"of a filled series ARE the provider's own objects (`is`), not copies "
f"carrying the same numbers. A copy is where the provisional flag gets "
f"lost (FCST-03)",
len(_firm_id) == len(_ahead)
and all(any(p is q for q in _read_id.points) for p in _firm_id))
# The carried numbers are yesterday's, at the same BRUSSELS wall-clock hour.
# ⚠️ THE LOOKUP IS BUILT BY SEARCH over yesterday's own instants, not by
# subtracting 24 h from today's - a fill that subtracted a fixed day is exactly
@@ -336,6 +443,36 @@ check(f"...and every carried number is YESTERDAY's price for the same BRUSSELS "
f"({sum(_carried_ok)}/{len(_carried_ok)})",
_carried_ok and all(_carried_ok))
# ⚠️ "FILLS EVERY UNPUBLISHED HOUR" IS CONDITIONAL, AND THIS IS THE CONDITION.
# A yesterday that is itself partial fills only the local hours it HOLDS; the
# rest are left out, which is §11.4 for those hours and emphatically not a
# zero. The behaviour is right and the wording was not - the CHANGELOG and DOCS
# said "every", unqualified, and nothing here exercised it.
_hp = store()
_PART_LOCAL = list(range(12)) # yesterday published 00:00..11:00 only
_ph.PriceHistory._remember(
_hp, series_of([at_local(YDAY_D, h) for h in _PART_LOCAL], YESTERDAY_MWH),
NOW)
_plan_part = run(resolve_tariff(CFG, NOW, _until,
transport=transport_for(_body_today), now=NOW,
history=_hp))
_window_hours = [SINCE_HOUR + timedelta(hours=n) for n in range(HORIZON_H)]
_ahead_utc = {u.astimezone(UTC) for u in _ahead}
_expect_covered = {u for u in _window_hours
if u in _ahead_utc or brussels(u).hour in _PART_LOCAL}
_got_covered = {p.start.astimezone(UTC) for p in _plan_part.read.points}
check(f"a PARTIAL yesterday fills only the local hours it holds: "
f"{len(_got_covered)} of the {HORIZON_H} horizon hours are covered, "
f"re-derived from the fixture as {len(_expect_covered)}. The hours "
f"yesterday never published stay ABSENT",
_got_covered == _expect_covered and 0 < len(_got_covered) < HORIZON_H)
_gap = sorted(set(_window_hours) - _expect_covered)[0]
check(f"...and an uncovered hour ({_gap.isoformat()}) raises `Absence` rather "
f"than answering a price. §11.4 applies to that hour: a hole in the "
f"series is not a free hour, and this is why the wording had to stop "
f"saying 'every'",
raises(lambda: _plan_part.read.at(_gap), Absence))
check("the plan reports itself PROVISIONAL now, which it did not before - the "
"old plan claimed full firmness over a horizon it covered half of",
_after.provisional and not _before.provisional)
@@ -370,8 +507,7 @@ check("...and the new one is the cheapest hour the feed published that has NOT "
check("...and no point of the filled series starts before the requested "
"`since` hour: the window is [since, until) and the past is not in it",
min(p.start.astimezone(UTC) for p in _after.read.points)
>= NOW.replace(minute=0, second=0, microsecond=0))
min(p.start.astimezone(UTC) for p in _after.read.points) >= SINCE_HOUR)
# ⚠️ THE HONEST LIMIT OF THIS TICKET, checked so it cannot be overclaimed
# later: the CARRIED hours buy no trade. §8.3 forbids acting on a provisional
@@ -386,6 +522,76 @@ check("the carried hours are NOT in the trade set - the verdict's cheapest and "
and len(_firm_after) == len(_ahead))
# --------------------------------------------------------------------------- #
print()
print("2b. a QUARTER-HOURLY feed keeps every print, and its spread with them")
# ⚠️ THE DEFECT THIS SECTION EXISTS FOR. `eneco_eddy` is the Belgian imbalance
# product and settles per QUARTER; `_validate_raw()` accepts any
# strictly-increasing spacing, so a published hour may hold four points.
# FCST-04's first draft keyed the published set `{_utc_hour(p.start): p ...}` -
# LAST POINT OF THE HOUR WINS, the other three dropped - and on an all-firm,
# fully-published 96-point feed the plan came out 15 points long with its
# arbitrage spread down from 0.143 to 0.014 EUR/kWh. That is not the windowing
# effect AC2 is about; it is prints being thrown away, and no flag reports it.
_QSTEP_MIN = 15
_qrows = []
for _n in range(len(TODAY_H) * (60 // _QSTEP_MIN)):
_qu = TODAY_H[0] + timedelta(minutes=_QSTEP_MIN * _n)
# A spiky within-the-hour profile, so decimation is visible in the SPREAD
# and not only in a count.
_qrows.append({"start": _qu.isoformat(),
"price_eur_mwh": 100.0 + 40.0 * (_n % 4) + brussels(_qu).hour})
_qbody = json.dumps({"data": _qrows}).encode()
check(f"the fixture really is sub-hourly - {len(_qrows)} points across "
f"{len(TODAY_H)} Brussels hours - or every check below passes for an "
f"hourly feed too and grades nothing",
len(_qrows) == 4 * len(TODAY_H)
and len({_r["start"] for _r in _qrows}) == len(_qrows))
_q_before = run(resolve_tariff(CFG, NOW, _until, transport=transport_for(_qbody),
now=NOW, history=NoHistory()))
_hq = store()
_ph.PriceHistory._remember(_hq, series_of(YDAY_H, YESTERDAY_MWH), NOW)
_q_after = run(resolve_tariff(CFG, NOW, _until, transport=transport_for(_qbody),
now=NOW, history=_hq))
# What the feed published INSIDE the window, derived from the unfilled plan.
_q_win = [p for p in _q_before.read.points
if p.start.astimezone(UTC) >= SINCE_HOUR]
_q_firm = [p for p in _q_after.read.points if not p.provisional]
check(f"every print the window contains survives the fill: {len(_q_firm)} firm "
f"points against the {len(_q_win)} the feed published in that window. "
f"One-per-hour keying leaves {len(_q_win) // 4}",
len(_q_win) > 0 and len(_q_firm) == len(_q_win))
check("...and they are the SAME INSTANTS AND PRICES the feed published, so "
"the count is not being made up by four hourly points wearing "
"quarter-hourly starts",
{(p.start.astimezone(UTC), p.eur_per_kwh) for p in _q_firm}
== {(p.start.astimezone(UTC), p.eur_per_kwh) for p in _q_win})
check("...and no two of them share a whole hour by accident: the fixture puts "
"four distinct prices inside every published hour, which is what the "
"decimated version could not represent",
len({p.start.astimezone(UTC).replace(minute=0) for p in _q_firm})
== len(_q_firm) // 4 > 0)
_q_spread_want = round(max(p.eur_per_kwh for p in _q_win)
- min(p.eur_per_kwh for p in _q_win), 6)
_qv = _q_after.arbitrage(**_A)
_q_spread_got = _qv.match(allowed=lambda v: v.spread_eur_kwh,
refused=lambda _v: None)
check(f"...and the spread is the fixture's own max - min over that window "
f"({_q_spread_want} EUR/kWh, measured {_q_spread_got}), not the "
f"flattened figure a decimated series produces",
_qv.kind == "allowed" and _q_spread_got is not None
and abs(_q_spread_got - _q_spread_want) < 1e-9)
check("...and the cheapest firm point is one the FEED published, at its own "
"sub-hourly instant - not a re-derived hourly one",
_qv.kind == "allowed"
and any(_qv.cheapest is q for q in _q_after.read.points)
and any(_qv.cheapest.start.astimezone(UTC) == q.start.astimezone(UTC)
for q in _q_win))
# --------------------------------------------------------------------------- #
print()
print("3. AC3 - the cache records the FEED, never its own output")
@@ -423,26 +629,114 @@ check(f"...and the fill is idempotent across the two rounds, which is §5.4's "
_canon(_filled2) and _canon(_filled2) == _canon(_filled2b))
# The ordering is the mechanism, so the mechanism is checked in the tree.
#
# ⚠️ EVERY SHIPPED PYTHON FILE, AND BY NAME RATHER THAN BY SUBSTRING. The first
# draft scanned `app/*.py` only - not `tools/`, not `scenarios/`, not the
# repo's `estop/` - and matched the bare string, so a module that merely
# MENTIONED `_remember` in a comment failed the check while a real call from
# `tools/` passed it unseen. This parses each file and looks for the name being
# USED: an attribute, a bare name, a def, or a string constant, which is how
# `getattr` reaches a private method.
_APP = os.path.join(HERE, "app")
_leaks = []
for name in sorted(os.listdir(_APP)):
if not name.endswith(".py") or name == "price_history.py":
continue
src = open(os.path.join(_APP, name), encoding="utf-8").read()
if "_remember" in src:
_leaks.append(name)
def _uses_remember(path):
"""Does this module USE the name `_remember`? Prose does not count."""
try:
tree = ast.parse(open(path, encoding="utf-8").read())
except (OSError, SyntaxError):
# ⚠️ NOT False. A file this scan cannot read is a file it has not
# cleared, and "unreadable" must never grade the same as "clean".
return True
for n in ast.walk(tree):
if isinstance(n, ast.Attribute) and n.attr == "_remember":
return True
if isinstance(n, ast.Name) and n.id == "_remember":
return True
if isinstance(n, (ast.FunctionDef, ast.AsyncFunctionDef)) \
and n.name == "_remember":
return True
if isinstance(n, ast.Constant) and n.value == "_remember":
return True
return False
_SHIPPED = []
for _root in (_APP, os.path.join(HERE, "tools"), os.path.join(HERE, "scenarios"),
os.path.join(os.path.dirname(HERE), "estop")):
for _dirpath, _dirnames, _filenames in os.walk(_root):
for _f in sorted(_filenames):
if _f.endswith(".py") and _f != "price_history.py":
_SHIPPED.append(os.path.join(_dirpath, _f))
check(f"the scan has a roster to walk ({len(_SHIPPED)} shipped modules across "
f"app/, tools/, scenarios/ and estop/), and it reaches beyond app/ - an "
f"empty or app-only glob passes for a leak in any other directory",
len(_SHIPPED) > 10
and any(os.sep + "tools" + os.sep in p for p in _SHIPPED))
_leaks = [os.path.relpath(p, HERE) for p in _SHIPPED if _uses_remember(p)]
check(f"AC3's mechanism is an ORDERING, not a validation - a carried point and "
f"a published-provisional point are the same three fields - so nothing "
f"outside app/price_history.py may name `_remember` {_leaks or 'none do'}",
not _leaks)
_probe = os.path.join(tmpdir(), "probe.py")
open(_probe, "w", encoding="utf-8").write("def f(h):\n return h._remember(1)\n")
_probe_ok = os.path.join(tmpdir(), "probe_ok.py")
open(_probe_ok, "w", encoding="utf-8").write(
'"""A docstring about _remember."""\n# and a comment about _remember\nX = 1\n')
check("...and that scan FAILS on a real call and PASSES on a file that only "
"talks about the rule, so it is a check and not a decoration",
_uses_remember(_probe) and not _uses_remember(_probe_ok))
# ⚠️ `ast.walk` IS BREADTH-FIRST, SO A LIST OF CALL NAMES GRADES NESTING DEPTH
# AND NOT SOURCE ORDER. The first draft of this check was
# `_calls.index("_remember") < _calls.index("yesterday")`: `self._remember(...)`
# is a statement while `self.yesterday(...)` is an ARGUMENT of
# `fill_from_yesterday(...)`, one level deeper, so `_remember` was emitted
# first whichever statement came first. Measured by writing the reversal out
# and grading it: shipped and reversed both scored True, while DECISIONS §3 and
# price_history.py's docstring both cited the check as making a reversal "red
# rather than plausible". Line numbers here, and the behaviour itself below.
_carry_src = inspect.getsource(PriceHistory.carry)
_tree = ast.parse(_carry_src.strip())
_calls = [n.func.attr for n in ast.walk(_tree)
if isinstance(n, ast.Call) and isinstance(n.func, ast.Attribute)]
check(f"...and `carry()` records BEFORE it fills, so a feed carrying yesterday "
f"alongside today refreshes the profile before consuming it {_calls}",
_calls.index("_remember") < _calls.index("yesterday"))
_lines = {}
for _n in ast.walk(_tree):
if isinstance(_n, ast.Call) and isinstance(_n.func, ast.Attribute):
_lines[_n.func.attr] = min(_lines.get(_n.func.attr, 10 ** 9), _n.lineno)
check(f"`carry()` names `_remember` on an EARLIER LINE than `yesterday` "
f"{_lines} - source order, which is what 'it records before it fills' "
f"actually means",
"_remember" in _lines and "yesterday" in _lines
and _lines["_remember"] < _lines["yesterday"])
# ...and the BEHAVIOUR the ordering exists for, driven rather than read. A feed
# that hands back yesterday alongside today must refresh yesterday's record
# BEFORE it is consumed as the profile. Into an EMPTY store that is the whole
# difference: record-then-fill covers the horizon, fill-then-record covers
# nothing, because at the moment of the fill there is no yesterday yet.
_h_ord = store() # empty: no record from an earlier day
_two_day = PriceRead.series(
[PricePoint(start=brussels(u),
eur_per_kwh=allin(YESTERDAY_MWH[brussels(u).hour]),
provisional=False) for u in YDAY_H]
+ [PricePoint(start=brussels(u),
eur_per_kwh=allin(TODAY_MWH[brussels(u).hour]),
provisional=False) for u in TODAY_H],
PriceProduct.DAY_AHEAD, "generic_dayahead", NOW)
_ord_filled = _h_ord.carry(_two_day, since=NOW, until=_until, stamp=NOW)
_ord_n = _ord_filled.match(series=lambda r: len(r.points), no_data=lambda _r: 0)
check(f"...and the ordering is DRIVEN, not only read: a feed carrying "
f"YESTERDAY alongside today, into an empty store, fills the whole "
f"{HORIZON_H} h horizon ({_ord_n} h) because the record was written "
f"before the lookup. Reversed, the lookup finds nothing and the plan is "
f"the {len(_ahead)} published hours the window keeps",
_ord_n == HORIZON_H)
check("...and the hours it carried carry YESTERDAY's numbers - the record that "
"same call had just written - not today's",
_ord_filled.kind == "series"
and all(abs(p.eur_per_kwh
- allin(YESTERDAY_MWH[brussels(p.start).hour])) < 1e-9
for p in _ord_filled.points if p.provisional))
# --------------------------------------------------------------------------- #
@@ -496,7 +790,72 @@ check(f"...and the record is keyed on the BRUSSELS date, so those 24 hours are "
# --------------------------------------------------------------------------- #
print()
print("5. AC4 - four degraded stores, four reasons, no prices")
print("4b. the two mornings a year when 'yesterday' is a DATE and not 24 hours")
# ⚠️ `brussels()` RETURNS A FIXED-OFFSET DATETIME, so `brussels(stamp) -
# timedelta(days=1)` subtracts 24 ABSOLUTE HOURS from a wall clock. The
# Brussels day before a spring-forward is 23 hours long, so between 00:00 and
# 00:59 on the morning after it that lands on the day BEFORE yesterday: the
# fill then carries a two-day-old profile behind a flag whose only vocabulary
# is "this is a guess", or - once pruning has dropped that day - answers
# `no_data` naming a date that is not yesterday. Measured at
# 2026-03-30T00:30+02:00 on the shipped branch: it returned 2026-03-28's
# prices. DST is this module's headline claim and it had NO CHECK.
for _label, (_dm2, _dm1, _d0) in (
("spring forward - yesterday was 23 h long",
("2026-03-28", "2026-03-29", "2026-03-30")),
("fall back - yesterday was 25 h long",
("2026-10-24", "2026-10-25", "2026-10-26"))):
_lens = [len(bday_hours(_d)) for _d in (_dm2, _dm1, _d0)]
check(f"{_label}: the fixture days are {_lens} hours long, so one of them "
f"really is not 24 and a DATE subtraction and an INSTANT subtraction "
f"can be told apart at all",
_lens[1] != 24 and _lens[0] == 24 and _lens[2] == 24)
_hd = store()
_tag = {}
for _i, _d in enumerate((_dm2, _dm1, _d0)):
# One price band per DAY, so the answer names which day it came from.
_tag[_d] = 0.100000 * (_i + 1)
_ph.PriceHistory._remember(
_hd,
PriceRead.series(
[PricePoint(start=brussels(u),
eur_per_kwh=round(_tag[_d] + 0.000100
* brussels(u).hour, 6),
provisional=False)
for u in bday_hours(_d)],
PriceProduct.DAY_AHEAD, "generic_dayahead", at_local(_d, 12)),
at_local(_d, 12))
check(f"{_label}: all three days are held, so the check below is about the "
f"DATE arithmetic and not about a record having been pruned away",
sorted(_hd._days) == sorted((_dm2, _dm1, _d0)))
_stamp_dst = at_local(_d0, 0) + timedelta(minutes=30)
_yd = _hd.yesterday(_stamp_dst)
_band = _yd.match(
series=lambda r: round(min(p.eur_per_kwh for p in r.points), 4),
no_data=lambda _r: None)
check(f"{_label}: at 00:30 local on {_d0}, yesterday() answers {_dm1}'s "
f"prices (band {_band}, wanted {round(_tag[_dm1], 4)}) and not "
f"{_dm2}'s (band {round(_tag[_dm2], 4)}) {why(_yd)!r}",
_yd.kind == "series" and _band == round(_tag[_dm1], 4))
check(f"{_label}: ...and it holds {_lens[1]} points, the length that "
f"Brussels day actually was - a 24-slot answer would be the wrong "
f"day wearing the right date",
_yd.kind == "series" and len(_yd.points) == _lens[1])
# The control, one hour later: an implementation that is right only at
# 01:30 is the one being caught, so both are graded.
_yd2 = _hd.yesterday(at_local(_d0, 1) + timedelta(minutes=30))
check(f"{_label}: ...and at 01:30 local it is still {_dm1} - the defect "
f"shows only in the first local hour, which is why an hourly refresh "
f"could hit it and never be noticed",
_yd2.kind == "series"
and round(min(p.eur_per_kwh for p in _yd2.points), 4)
== round(_tag[_dm1], 4))
# --------------------------------------------------------------------------- #
print()
print("5. AC4 - degraded stores, one reason each, no prices")
_missing = PriceHistory(os.path.join(tmpdir(), "gone", "price_history.json"))
check("no durable volume: the store is NOT ok, and it does not create the "
@@ -505,6 +864,28 @@ check("no durable volume: the store is NOT ok, and it does not create the "
not _missing.ok and "no durable volume" in (_missing.last_error or "")
and not os.path.isdir(os.path.dirname(_missing.path)))
# ⚠️ AND IT DOES NOT FILL IN MEMORY EITHER. FCST-04 shipped this half-done:
# `_save()` short-circuited on `not ok` while `_remember()` populated
# `self._days` anyway, so a site with no /data answered `words` = "unavailable"
# and `yesterday()` = A SERIES - an in-memory profile that works all day and is
# gone on the next restart. price_history.py's own third warning rejects
# exactly that ("a yesterday that silently forgets") as its reason for not
# calling makedirs, so the code and the docstring have to agree.
_carried_nowhere = _missing.carry(series_of(TODAY_H, TODAY_MWH),
since=NOW, until=_until, stamp=NOW)
check("a store with no durable volume records NOTHING, not even in memory: "
"after a full carry it still holds no day, `yesterday()` still answers "
"no_data, and `words` still says unavailable. A cache that forgets on "
"restart without saying so is worse than one that says it cannot answer",
_missing._days == {}
and _missing.yesterday(NOW + timedelta(days=1)).kind == "no_data"
and _missing.words.startswith("unavailable"))
check("...and the carry still returns a real series - the fill windows the "
"feed and carries nothing, which is the pre-FCST-04 horizon and not a "
"degradation",
_carried_nowhere.kind == "series"
and len(_carried_nowhere.points) == len(_ahead))
_d6 = tmpdir()
_p6 = os.path.join(_d6, "price_history.json")
open(_p6, "w", encoding="utf-8").write("{ this is not json")
@@ -554,6 +935,127 @@ check("a record naming a product that is not a PriceProduct is no_data, not a "
"series with a string in the field the optimiser switches on",
PriceHistory(_p9).yesterday(NOW).kind == "no_data")
# ⚠️ AND THE ONE THAT STOPPED THE ADD-ON FROM STARTING. `_load()` validated
# `version` and that `days` is a dict, and never that each VALUE is one. With
# `{"version": 1, "days": {"2026-01-14": "oops"}}` the store reported ok=True
# and last_error=None, and then `rec.get(...)` raised `AttributeError` out of
# `words`, `yesterday()` and `resolve_tariff()` alike. `PriceHistory(...)` and
# `.words` are both unguarded in `Controller.__init__`, so the controller did
# not come up - and it never self-healed, because `_remember()` raised on the
# same record before it could reach `_save()`. The asymmetry that produced it
# is visible in the file: `_parse_points()` is carefully defensive about every
# field of a POINT, and `_load()` was not defensive about the record at all.
_p14 = os.path.join(tmpdir(), "price_history.json")
json.dump({"version": SCHEMA, "days": {
"2026-01-14": "oops",
"2026-01-13": {"product": "dynamic_dayahead", "provider": "generic_dayahead",
"points": [_good]}}},
open(_p14, "w", encoding="utf-8"))
_h14 = PriceHistory(_p14)
check("a day record that is not a dict is DROPPED and REPORTED, and the rest "
"of the file still loads - one bad record is not a bad file",
_h14.ok and "unusable day record" in (_h14.last_error or "")
and sorted(_h14._days) == ["2026-01-13"])
_w14 = derived(lambda: _h14.words)
_y14 = derived(lambda: _h14.yesterday(NOW).kind)
_plan14 = derived(lambda: run(resolve_tariff(
CFG, NOW, _until, transport=transport_for(_body_today),
now=NOW, history=_h14)).origin)
check(f"...and NOTHING raises on it: not `words` ({_w14!r}), not `yesterday()` "
f"({_y14!r}), not the shipped `resolve_tariff()` path ({_plan14!r}). A "
f"broken cache is a short horizon, not a controller that will not start",
isinstance(_w14, str) and _y14 == "no_data" and _plan14 == "provider")
for _shape in ("oops", [1, 2, 3], 7, None, {"points": "not a list"}):
_pX = os.path.join(tmpdir(), "price_history.json")
json.dump({"version": SCHEMA, "days": {"2026-01-14": _shape}},
open(_pX, "w", encoding="utf-8"))
_hX = PriceHistory(_pX)
_wX = derived(lambda h=_hX: h.words)
_yX = derived(lambda h=_hX: h.yesterday(NOW).kind)
check(f"...and the same holds for a day record of {type(_shape).__name__} "
f"{_shape!r}: dropped, reported, and every accessor answers "
f"({_wX!r}, {_yX!r})",
_hX._days == {} and "unusable day record" in (_hX.last_error or "")
and isinstance(_wX, str) and _yX == "no_data")
_pF = os.path.join(tmpdir(), "price_history.json")
json.dump({"version": float(SCHEMA), "days": {}},
open(_pF, "w", encoding="utf-8"))
check(f"a `version` of {float(SCHEMA)!r} is refused although it EQUALS "
f"{SCHEMA} - a schema check that cannot tell 1 from 1.0 is one that "
f"would not notice a writer this build has never met",
"schema" in (PriceHistory(_pF).last_error or ""))
# --------------------------------------------------------------------------- #
print()
print("5b. `words` - the status row, and one of the two things a bad record "
"used to crash")
def words_of(h):
return derived(lambda: h.words)
_w_missing = words_of(_missing)
check(f"no durable volume: the row says UNAVAILABLE and names the reason "
f"{_w_missing!r}",
isinstance(_w_missing, str) and _w_missing.startswith("unavailable")
and "no durable volume" in _w_missing)
_h_words = store()
check("a healthy but EMPTY store says something different, because it is a "
"different state and a normal one on a site's first day",
isinstance(words_of(_h_words), str)
and words_of(_h_words).startswith("empty")
and "nothing to carry forward" in words_of(_h_words))
_ph.PriceHistory._remember(_h_words, series_of(YDAY_H, YESTERDAY_MWH), NOW)
check(f"...and a store holding a day names the DATE and the HOUR COUNT "
f"{words_of(_h_words)!r} - 'a cache exists' and 'the cache holds "
f"yesterday, all 24 hours of it' are different facts, and only the "
f"second says the horizon will be full tomorrow morning",
words_of(_h_words) == f"{YDAY_D} (24 h)")
check(f"a file that only PARTLY loaded says so in the row {_w14!r}, or the "
f"days it dropped read as days that were simply never written",
isinstance(_w14, str) and (_h14.last_error or "") in _w14
and "2026-01-13" in _w14)
check("NoHistory() has a sentence of its own rather than an empty cell",
NoHistory().words == "not kept")
check("...and `words` never raises, whatever the file holds. It is read once "
"from Controller.__init__ and again on every status render, and neither "
"call site guards it - an exception here is an add-on that does not start",
all(isinstance(words_of(h), str)
for h in (_missing, _h6, _h7, _h14, _h_words)))
# ⚠️ AND A FAILED WRITE LEAVES NOTHING BEHIND. `_save()` writes to a `mkstemp`
# and renames - atomic, so a power cut cannot leave a truncated file that
# parses as an empty day - and a failure BETWEEN the two used to leave the temp
# in place. The refresh task writes 48 times a day, so a persistent write
# failure leaked 48 inodes a day onto the volume whose being full is the
# likeliest reason the write failed at all.
_dleak = tmpdir()
_pleak = os.path.join(_dleak, "price_history.json")
_hleak = PriceHistory(_pleak)
check("the leak fixture starts clean, or 'nothing left behind' is not a "
"measurement",
_hleak.ok and os.listdir(_dleak) == [])
os.mkdir(_pleak) # a DIRECTORY where the file goes: the temp is
# written fine and `os.replace` cannot complete
_ph.PriceHistory._remember(_hleak, series_of(YDAY_H, YESTERDAY_MWH), NOW)
check(f"a write that cannot complete SAYS so rather than passing quietly "
f"{(_hleak.last_error or '')[:48]!r}",
"could not write" in (_hleak.last_error or ""))
_left = sorted(n for n in os.listdir(_dleak) if n != "price_history.json")
check(f"...and leaves NOTHING behind {_left} - at 48 writes a day a leaked "
f"temp per write fills the volume that was already the problem",
_left == [])
_dok = tmpdir()
_hok = PriceHistory(os.path.join(_dok, "price_history.json"))
_ph.PriceHistory._remember(_hok, series_of(YDAY_H, YESTERDAY_MWH), NOW)
check(f"...and a SUCCESSFUL write leaves exactly the one file "
f"{sorted(os.listdir(_dok))}, so the cleanup did not start deleting the "
f"cache it just wrote",
sorted(os.listdir(_dok)) == ["price_history.json"]
and _hok.last_error is None)
# ...and every one of those leaves the CONTROLLER regulating.
#
# ⚠️ THE PLAN IS NOT BIT-FOR-BIT THE PRE-FCST-04 ONE, and the first draft of
@@ -671,14 +1173,88 @@ check(f"six days filed, {KEEP_DAYS} held {_held} - a bigger cache would only "
check("...and yesterday is one of them, which is the only day that matters",
_h12.yesterday(NOW).kind == "series")
_h13 = store()
_ph.PriceHistory._remember(
_h13, series_of(YDAY_H, YESTERDAY_MWH), NOW)
# ⚠️ THE STIMULUS HAS TO REACH `_prune`'s BODY, AND THE FIRST DRAFT DID NOT.
# It seeded ONE day and then pruned with a jumped stamp: `if len(self._days) <=
# KEEP_DAYS: return` fired, the body never ran, and the check named after that
# body was green for an implementation that pruned by clock arithmetic.
# Measured - the body was replaced with "delete every day older than stamp
# minus KEEP_DAYS days", the guard left in place, and the whole 29-file suite
# stayed green while the mutant emptied the store on a jumped clock.
#
# A store holding more days than the cap is a REAL state, not a contrivance:
# `_load()` does not prune, so this is exactly what a restart sees after a file
# written by a build with a larger cap, or after a feed that spanned several
# days. So the fixture is written, opened, and only then pruned.
_FILED = KEEP_DAYS + 2
_p13 = os.path.join(tmpdir(), "price_history.json")
_days13 = {}
for _back in range(_FILED - 1, -1, -1):
_d13 = (datetime.fromisoformat(TODAY_D)
- timedelta(days=_back)).strftime("%Y-%m-%d")
_days13[_d13] = {
"product": PriceProduct.DAY_AHEAD.value,
"provider": "generic_dayahead",
"points": [PricePoint(start=brussels(u),
eur_per_kwh=allin(YESTERDAY_MWH[brussels(u).hour]),
provisional=False).canonical()
for u in bday_hours(_d13)]}
json.dump({"version": SCHEMA, "days": _days13},
open(_p13, "w", encoding="utf-8"))
_h13 = PriceHistory(_p13)
check(f"the prune fixture holds {len(_h13._days)} days, MORE than KEEP_DAYS "
f"({KEEP_DAYS}), so the early return cannot swallow the stimulus - this "
f"file's own rule, and the one its first draft broke",
len(_h13._days) == _FILED > KEEP_DAYS)
_ph.PriceHistory._prune(_h13, NOW + timedelta(days=4000))
check("pruning is by SORT ORDER, not by arithmetic against the clock: a clock "
"that jumped four thousand days does not delete the record, which would "
"empty the store on exactly the morning the fill is for",
_h13.yesterday(NOW).kind == "series")
check(f"pruning does not delete by ARITHMETIC AGAINST THE CLOCK: a clock that "
f"jumped four thousand days keeps the newest {KEEP_DAYS} of the "
f"{_FILED} records {sorted(_h13._days)} instead of deleting all of them, "
f"which would empty the store on exactly the morning the fill is for",
len(_h13._days) == KEEP_DAYS and _h13.yesterday(NOW).kind == "series")
# --------------------------------------------------------------------------- #
print()
print("8b. the two days a lookup can ask for are PINNED, so a far-off record "
"cannot evict yesterday")
# ⚠️ KEEP_DAYS = 3 IS THE STEADY STATE EXACTLY - {yesterday, today, tomorrow} -
# so a newest-N cap has NO SLACK. A feed that publishes to D+2 makes four days
# and the cap drops the oldest, which is yesterday: the only day the fill can
# use. One stray point in the wrong year does the same thing permanently,
# because a lexicographic sort puts it LAST and nothing can ever age it out.
_h15 = store()
_SPAN_D = (YDAY_D, TODAY_D, "2026-01-16", "2026-01-17")
_span = [u for _d in _SPAN_D for u in bday_hours(_d)]
check(f"the spanning fixture really covers {len(_SPAN_D)} Brussels days, more "
f"than KEEP_DAYS ({KEEP_DAYS}), or the cap is not under pressure and "
f"this grades nothing",
len(_SPAN_D) > KEEP_DAYS
and len({brussels(u).strftime("%Y-%m-%d") for u in _span}) == len(_SPAN_D))
_ph.PriceHistory._remember(_h15, series_of(_span, YESTERDAY_MWH), NOW)
check(f"a feed spanning {len(_SPAN_D)} days does NOT evict yesterday: "
f"{sorted(_h15._days)} held, and {YDAY_D} is one of them. Under the "
f"newest-{KEEP_DAYS} cap alone it is the first record dropped, and the "
f"fill loses its only usable day for good",
YDAY_D in _h15._days and _h15.yesterday(NOW).kind == "series")
_h16 = store()
for _d16 in (YDAY_D, TODAY_D, "2026-01-16"):
_ph.PriceHistory._remember(_h16, series_of(bday_hours(_d16), TODAY_MWH), NOW)
_STRAY_D = "2099-07-01"
_ph.PriceHistory._remember(
_h16, series_of(bday_hours(_STRAY_D)[:1], TODAY_MWH), NOW)
check(f"the stray fixture is really far future and really sorts last "
f"{sorted(_h16._days)[-1]!r}, or 'it can never age out' is not what is "
f"being graded",
sorted(_h16._days)[-1] == _STRAY_D)
check(f"...and ONE stray point in {_STRAY_D} does not push yesterday out "
f"either. It is not deleted - nothing is deleted by arithmetic here - "
f"it simply cannot take yesterday's slot",
YDAY_D in _h16._days and _h16.yesterday(NOW).kind == "series")
check(f"...while the store stays BOUNDED at KEEP_DAYS + 2 = {KEEP_DAYS + 2} "
f"records ({len(_h16._days)} held), so the pin is a pin and not a leak",
len(_h16._days) <= KEEP_DAYS + 2)
# --------------------------------------------------------------------------- #
@@ -708,11 +1284,57 @@ _kw = [k.arg for n in ast.walk(_refresh) if isinstance(n, ast.Call)
check(f"the shipped refresh task passes a history {_kw} - the wiring is what "
f"this ticket IS, so it is checked in the tree rather than described",
"history" in _kw)
check("the FCST-04 note in main.py carries the WINDOWING caveat beside the "
"'widens what the optimiser can SEE, never what it may act on' "
"guarantee. That note is the one place a maintainer reads that sentence "
"unqualified, and the CHANGELOG, DOCS and resolve_tariff()'s docstring "
"all carry the pair - so it may not appear here on its own",
"never what it may act on" in _MAIN and "ALREADY ELAPSED" in _MAIN)
check("...and it is the CONTROLLER'S store, not a NoHistory() smuggled in to "
"make the call typecheck",
"history=self.price_history" in _MAIN
and "NoHistory" not in _MAIN)
check("...and `words` reaches the STATUS PAGE as its own row, so what the fill "
"has to carry from is a fact an operator can read rather than one that "
"only exists in a log line",
'("Price history", self.price_history.words)' in _MAIN)
_rows_fn = next(n for n in ast.walk(_main_tree)
if isinstance(n, ast.FunctionDef)
and n.name == "status_payload")
_row_labels = [e.elts[0].value for e in ast.walk(_rows_fn)
if isinstance(e, ast.Tuple) and len(e.elts) == 2
and isinstance(e.elts[0], ast.Constant)
and isinstance(e.elts[0].value, str)]
check(f"...and it is in the ROW LIST rather than in a comment about one "
f"{[r for r in _row_labels if 'rice' in r or 'ariff' in r]}",
"Price history" in _row_labels)
# ⚠️ AND WHAT THE STATUS PAGE DOES **NOT** SAY, PINNED AS A PAIR. DOCS.md
# claimed "*Tariff* on the status page reads `provisional` through the
# morning". It does not and never did: `self.tariff_words` is assigned only
# from the Home Assistant price-entity branch and from `static_status()`, both
# on the 30 s tick, and the provider plan's own sentence reaches `_LOG.info`
# and nothing else. This grades the PAIRING rather than the current state - so
# wiring the plan into that row is fine and correcting the doc is fine, and
# doing one without the other is red.
_MAIN_LINES = _MAIN.splitlines()
_tw_lines = sorted({n.lineno for n in ast.walk(_main_tree)
if isinstance(n, ast.Assign)
for t in n.targets
for a in ast.walk(t)
if isinstance(a, ast.Attribute) and a.attr == "tariff_words"})
_row_from_plan = any("plan" in _MAIN_LINES[i - 1] for i in _tw_lines)
_DOCS = open(os.path.join(HERE, "DOCS.md"), encoding="utf-8").read()
_docs_claims = "**Tariff** on the status page reads `provisional`" in _DOCS
check(f"the *Tariff* row and what DOCS says about it AGREE: fed from the "
f"provider plan = {_row_from_plan} (assigned on lines {_tw_lines}), DOCS "
f"claims it reads `provisional` through the morning = {_docs_claims}. It "
f"shipped as False against True - an overclaim in a doc is a defect "
f"here, and this is the pair that has to move together",
len(_tw_lines) >= 2 and _row_from_plan == _docs_claims)
# NoHistory is inert, and says so by identity rather than by producing a series.
_nh_read = series_of(TODAY_H, TODAY_MWH)
check("NoHistory() carries nothing and returns the read BY IDENTITY, so the "