MAINT-20 part 1: DECISIONS note for the forecast fill (FCST-03)

Co-Authored-By: Claude <noreply@anthropic.com>

goodwe_controller/DECISIONS/FCST-03-the-hours-not-yet-published.md

@@ -0,0 +1,108 @@
+# FCST-03 - the hours the feed has not published, and what may not rest on them
+
+**Status: SHIPPED.** Tests: `test_price_forecast.py`, `test_price_history.py`.
+Written down by MAINT-20 from the long module comment of `app/price_forecast.py`.
+Follows specs.md 8.3: "Day-ahead prices arrive once daily. Until they do, unknown
+hours use yesterday's profile and are flagged `provisional: true`, and no
+irreversible action may be taken on a provisional price. Real-time / imbalance
+products are treated as high-volatility: arbitrage thresholds widen." This module
+owns the first two; the widening is in price.py (`Uncertainty`, `widen_factor`,
+`required_spread`), spent by `static_tariff.TariffPlan.arbitrage()` as one more
+refusal on its existing ladder. Persistence of yesterday: [[FCST-04]].
+
+## The flag is never passed through
+
+A carried point and a published one are the same three floats and a bool.
+`PricePoint(..., provisional=y.provisional)` copies yesterday's `False` onto
+today's guess: stale data presented as current (SAFETY-18's defect one layer up).
+So:
+
+- `_carried()` has no parameter for the flag; `provisional=True` is a literal in
+  one place.
+- A published point is passed through by identity (`filled.points[i] is
+  fetched.points[j]`, checked by the test), never copied.
+- `_audit()` re-derives the expected flag for every emitted point from the hours
+  the feed published and raises on a mismatch. It never corrects (a module that
+  quietly fixes its own provenance cannot be trusted).
+- test_price_forecast.py walks this file's AST and fails if any `PricePoint(...)`
+  passes anything but a literal for `provisional`.
+
+"Published" does not mean "firm": 9.3 carries `provisional` per point because a
+real-time product's current-interval print is published AND provisional, so the
+audit's rule for a published point is "whatever the feed said". No shipped
+plugin emits one since [[TARIFF-07]]; the rule is a property of the format and
+weakening it would let the first imbalance feed arrive mispriced and flagged firm.
+
+## The audit is a provenance check, not a completeness check (FCST-05)
+
+Rules: a PUBLISHED point must be one of `published[u]` by identity and be
+emitted once, with the feed's own flag; a CARRIED point's hour must not be in
+`published` at all, flag exactly True, and no second point in that hour. This
+replaced FCST-03's "at most one point per hour" count, which cannot express a
+quarter-hourly feed. FCST-04 claimed two strengthenings; both were false: a
+differential over every published mask on 3 hours crossed with every output of
+length 0-3 found zero divergence either way (OLD raised/NEW accepted 0, OLD
+accepted/NEW raised 0). The one real change is the intended relaxation (several
+distinct published points per hour).
+
+The audit does not check completeness: fed the decimation defect (four points in
+an hour, only the last emitted), two of four, or an empty output, it accepts.
+What stops decimation is the point COUNT and IDENTITY checks in the two test
+files. It cannot be made complete from its arguments: `published` is over the
+whole unwindowed read while the fill emits `[since, until)`, so a published point
+outside the window is legitimately absent. FCST-05 built "every published point
+is emitted" (+162 B) and it took the 1 h, 12 h and 48 h horizon checks red on a
+healthy feed (49 cases scored stronger), so the rule as stated is wrong, and
+asserting a completeness the instrument lacks is MUTATION.md habit 16's defect.
+
+## The fill keys on the wall clock
+
+For each whole hour of the horizon the feed has not published, take yesterday's
+value for the SAME BRUSSELS WALL-CLOCK HOUR, not `t - 86400 s`: a 23-hour day has
+no 02:00 and a 25-hour day has two, so a fixed 24 h shifts every carried hour
+across a transition. On a 25-hour day both 02:00s are filled from yesterday's
+single 02:00 (both guesses, both flagged); if yesterday was a 25-hour day, the
+first of its two 02:00 points is used. An hour with no published price and no
+yesterday value is left out, not filled with a neighbour, mean or zero: 11.4
+applies (`Absence`, then the static profile). If nothing can be produced the
+result is `no_data`, not an empty series.
+
+A sub-hourly feed is not decimated: the hourly walk decides which hours need
+filling, never what a published hour may hold. Belgian day-ahead settles per
+quarter since 2025-10-01 (96 points a day), so every published point in the
+window is passed through and a carried one is added only for an hour with no
+published coverage. `{_utc_hour(p.start): p for p in r.points}` reads ordinary
+and keeps the last point of each hour; FCST-04's first caller shipped it, the
+series came out 15 points long, the spread fell from 0.143 to 0.014 EUR/kWh and
+the cheapest "hour" became a `:45` point. `_published_by_hour()` holds a tuple
+per hour for that reason.
+
+The window is whole hours at both ends: `since` floored (the current hour is
+still tradeable) and `until` floored too; with `until` compared raw a 48 h horizon
+asked at 08:17 emitted 49 hours. `yesterday` has no default (a missing yesterday
+and a useless one must not look alike). The fill is idempotent and re-runnable
+(5.4 late publication, 14-16 h CET): published hours win, and the series is
+rebuilt, not patched. TARIFF-11 log line: carried HOURS, total hours and
+published POINTS are three different quantities (a quarter-hourly feed made it
+read "33 of 141 hours" for 48 h).
+
+## AC2: the irreversible-action gate
+
+An irreversible action is a value, `Irreversible`, carrying the hours it commits.
+`price_permits()` looks at exactly those hours and answers with a `Gate` carrying
+evidence on the permitting arm and a reason on the blocking one (the `_Outcome`
+shape). It blocks when the read observed nothing (11.4), an hour has no price, or
+an hour is provisional, and names the hour. The maintenance drain's window is
+derived from `maintenance.DRAIN_TIMEOUT_MIN` / `CHARGE_TIMEOUT_MIN`, so the gate
+and the phase machine cannot drift. `_window_hours()` is separate so the gate and
+its caller agree on "the hours it commits".
+
+What is NOT gated matters more: the maintenance cycle is a calendar obligation
+(12.2: a cycle that silently never fires looks like one not yet due), and a site
+with `price_provider: off` (the default) has no series. Routing every start
+through a gate that blocks on `no_data` would stop the monthly cycle on every
+default install. So `begin_phase()` is the entry point for a start JUSTIFIED BY A
+PRICE; main.py's calendar path is untouched, and OPT-04 / MAINT-05 own which a
+site runs. Reversible actions (self-consumption modulation, the loop) are not
+gated: 6 says absent machinery must not disable the loop. Read this before wiring
+`begin_phase()` in.