From 50332d4190a9519a2f4a85dc8c298994206d5df5 Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Sun, 1 Nov 2026 01:19:29 +0100 Subject: [PATCH] MAINT-20 part 1: DECISIONS note for the forecast fill (FCST-03) Co-Authored-By: Claude --- .../FCST-03-the-hours-not-yet-published.md | 108 ++++++++++++++++++ 1 file changed, 108 insertions(+) create mode 100644 goodwe_controller/DECISIONS/FCST-03-the-hours-not-yet-published.md diff --git a/goodwe_controller/DECISIONS/FCST-03-the-hours-not-yet-published.md b/goodwe_controller/DECISIONS/FCST-03-the-hours-not-yet-published.md new file mode 100644 index 0000000..a238a1e --- /dev/null +++ b/goodwe_controller/DECISIONS/FCST-03-the-hours-not-yet-published.md @@ -0,0 +1,108 @@ +# FCST-03 - the hours the feed has not published, and what may not rest on them + +**Status: SHIPPED.** Tests: `test_price_forecast.py`, `test_price_history.py`. +Written down by MAINT-20 from the long module comment of `app/price_forecast.py`. +Follows specs.md 8.3: "Day-ahead prices arrive once daily. Until they do, unknown +hours use yesterday's profile and are flagged `provisional: true`, and no +irreversible action may be taken on a provisional price. Real-time / imbalance +products are treated as high-volatility: arbitrage thresholds widen." This module +owns the first two; the widening is in price.py (`Uncertainty`, `widen_factor`, +`required_spread`), spent by `static_tariff.TariffPlan.arbitrage()` as one more +refusal on its existing ladder. Persistence of yesterday: [[FCST-04]]. + +## The flag is never passed through + +A carried point and a published one are the same three floats and a bool. +`PricePoint(..., provisional=y.provisional)` copies yesterday's `False` onto +today's guess: stale data presented as current (SAFETY-18's defect one layer up). +So: + +- `_carried()` has no parameter for the flag; `provisional=True` is a literal in + one place. +- A published point is passed through by identity (`filled.points[i] is + fetched.points[j]`, checked by the test), never copied. +- `_audit()` re-derives the expected flag for every emitted point from the hours + the feed published and raises on a mismatch. It never corrects (a module that + quietly fixes its own provenance cannot be trusted). +- test_price_forecast.py walks this file's AST and fails if any `PricePoint(...)` + passes anything but a literal for `provisional`. + +"Published" does not mean "firm": 9.3 carries `provisional` per point because a +real-time product's current-interval print is published AND provisional, so the +audit's rule for a published point is "whatever the feed said". No shipped +plugin emits one since [[TARIFF-07]]; the rule is a property of the format and +weakening it would let the first imbalance feed arrive mispriced and flagged firm. + +## The audit is a provenance check, not a completeness check (FCST-05) + +Rules: a PUBLISHED point must be one of `published[u]` by identity and be +emitted once, with the feed's own flag; a CARRIED point's hour must not be in +`published` at all, flag exactly True, and no second point in that hour. This +replaced FCST-03's "at most one point per hour" count, which cannot express a +quarter-hourly feed. FCST-04 claimed two strengthenings; both were false: a +differential over every published mask on 3 hours crossed with every output of +length 0-3 found zero divergence either way (OLD raised/NEW accepted 0, OLD +accepted/NEW raised 0). The one real change is the intended relaxation (several +distinct published points per hour). + +The audit does not check completeness: fed the decimation defect (four points in +an hour, only the last emitted), two of four, or an empty output, it accepts. +What stops decimation is the point COUNT and IDENTITY checks in the two test +files. It cannot be made complete from its arguments: `published` is over the +whole unwindowed read while the fill emits `[since, until)`, so a published point +outside the window is legitimately absent. FCST-05 built "every published point +is emitted" (+162 B) and it took the 1 h, 12 h and 48 h horizon checks red on a +healthy feed (49 cases scored stronger), so the rule as stated is wrong, and +asserting a completeness the instrument lacks is MUTATION.md habit 16's defect. + +## The fill keys on the wall clock + +For each whole hour of the horizon the feed has not published, take yesterday's +value for the SAME BRUSSELS WALL-CLOCK HOUR, not `t - 86400 s`: a 23-hour day has +no 02:00 and a 25-hour day has two, so a fixed 24 h shifts every carried hour +across a transition. On a 25-hour day both 02:00s are filled from yesterday's +single 02:00 (both guesses, both flagged); if yesterday was a 25-hour day, the +first of its two 02:00 points is used. An hour with no published price and no +yesterday value is left out, not filled with a neighbour, mean or zero: 11.4 +applies (`Absence`, then the static profile). If nothing can be produced the +result is `no_data`, not an empty series. + +A sub-hourly feed is not decimated: the hourly walk decides which hours need +filling, never what a published hour may hold. Belgian day-ahead settles per +quarter since 2025-10-01 (96 points a day), so every published point in the +window is passed through and a carried one is added only for an hour with no +published coverage. `{_utc_hour(p.start): p for p in r.points}` reads ordinary +and keeps the last point of each hour; FCST-04's first caller shipped it, the +series came out 15 points long, the spread fell from 0.143 to 0.014 EUR/kWh and +the cheapest "hour" became a `:45` point. `_published_by_hour()` holds a tuple +per hour for that reason. + +The window is whole hours at both ends: `since` floored (the current hour is +still tradeable) and `until` floored too; with `until` compared raw a 48 h horizon +asked at 08:17 emitted 49 hours. `yesterday` has no default (a missing yesterday +and a useless one must not look alike). The fill is idempotent and re-runnable +(5.4 late publication, 14-16 h CET): published hours win, and the series is +rebuilt, not patched. TARIFF-11 log line: carried HOURS, total hours and +published POINTS are three different quantities (a quarter-hourly feed made it +read "33 of 141 hours" for 48 h). + +## AC2: the irreversible-action gate + +An irreversible action is a value, `Irreversible`, carrying the hours it commits. +`price_permits()` looks at exactly those hours and answers with a `Gate` carrying +evidence on the permitting arm and a reason on the blocking one (the `_Outcome` +shape). It blocks when the read observed nothing (11.4), an hour has no price, or +an hour is provisional, and names the hour. The maintenance drain's window is +derived from `maintenance.DRAIN_TIMEOUT_MIN` / `CHARGE_TIMEOUT_MIN`, so the gate +and the phase machine cannot drift. `_window_hours()` is separate so the gate and +its caller agree on "the hours it commits". + +What is NOT gated matters more: the maintenance cycle is a calendar obligation +(12.2: a cycle that silently never fires looks like one not yet due), and a site +with `price_provider: off` (the default) has no series. Routing every start +through a gate that blocks on `no_data` would stop the monthly cycle on every +default install. So `begin_phase()` is the entry point for a start JUSTIFIED BY A +PRICE; main.py's calendar path is untouched, and OPT-04 / MAINT-05 own which a +site runs. Reversible actions (self-consumption modulation, the loop) are not +gated: 6 says absent machinery must not disable the loop. Read this before wiring +`begin_phase()` in.