FCST-03: yesterday's prices, flagged as yesterday's, and unable to move anything

§8.3 fills the hours a day-ahead feed has not published yet with yesterday's
profile and flags them provisional, and forbids any irreversible action on such
a price. The whole risk of this ticket is one line - a carried point and a
published point are three fields and a bool, so `provisional=y.provisional`
copies yesterday's False onto today's guess and produces stale data presented as
current. That is SAFETY-18's frozen-but-fresh meter one layer up.

So the flag is never passed through. Carried points come from a constructor with
no provisional parameter; published points pass through BY IDENTITY, so no copy
exists that could lose one; an audit re-derives the expected flag for every
emitted point and raises rather than repairing; and the check walks the module's
own AST so the pass-through idiom cannot come back looking ordinary. A published
point may itself be provisional (eneco_eddy's imbalance print is), so the rule
for a published hour is "whatever the feed said", never False.

The fill keys on the Brussels wall clock rather than t-24h, so the 23- and
25-hour days are right; a late publication rebuilds the series, moving the
number and the flag together.

AC2 and AC3 landed on TARIFF-03's existing verdict surface rather than beside
it. Refusal 3 became "fewer than two FIRM hours" - refusing on "any point is
provisional" would have switched arbitrage off every morning on every dynamic
site - and refusal 5 is new: the spread must clear a threshold widened by the
widest active uncertainty source (max, never a product). The permitting variant
now refuses to be constructed around a provisional point at all, and the
cheap-window flag became per-hour for the same reason.

The irreversible action is a value carrying the hours it commits, derived from
maintenance.py's own DRAIN_TIMEOUT_MIN; the gate blocks a maintenance drain
whose window reaches into a carried hour, and the phase machine stays idle. It
is the price-justified entry point only - a gate that blocked on no_data would
stop the monthly cycle on every default install.

Zero new config options; TEST-14's signature scan ported to cover both Gate's
and Arbitrage's constructors, closing _Refused.reason's open hole.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC

goodwe_controller/app/static_tariff.py

@@ -77,25 +77,49 @@ price.py borrowed from `sim/riglock.py`. An optimiser cannot compute a trade
 from a refusal, because it cannot get the inputs out of one.
 
 And the verdict is only obtainable from `TariffPlan.arbitrage(...)`, which takes
-`peak_guard_active` as a REQUIRED KEYWORD ARGUMENT. §9.1 and §10.1 put peak
+`peak_guard_active`, `min_spread_eur_kwh` and `uncertainty` as REQUIRED KEYWORD
+ARGUMENTS, none of them defaulted. §9.1 and §10.1 put peak
 shaving (priority 2) above price arbitrage (priority 3) - the capacity tariff is
 a network cost charged whatever the energy product is - and the way that
 ordering is made to survive OPT-04 landing is that OPT-04 cannot obtain a spread
 without first stating whether the peak guard is holding. It is not free to
 forget the question; the call does not compile without an answer.
 
-⚠️ FOUR REFUSALS, IN THIS ORDER, EACH DERIVED FROM SOMETHING MEASURED:
+⚠️ FIVE REFUSALS, IN THIS ORDER, EACH DERIVED FROM SOMETHING MEASURED:
 
   1. the peak guard is active           §10.1: priority 2 outranks priority 3
   2. no price data at all               §11.4, and no static_profile to fall to
-  3. the plan is provisional            §11.4 fallback / §8.3 unknown hours
-  4. the series is FLAT                 max - min == 0, so there is nothing to
-                                        arbitrage. THIS is AC2: a fully-fixed
-                                        tariff disables arbitrage by ARITHMETIC,
-                                        not by a special case keyed on
-                                        `PriceProduct.FIXED`. A dynamic feed
+  3. fewer than TWO FIRM HOURS          §8.3. FCST-03 rewrote this one: it used
+                                        to be "any point in the series is
+                                        provisional", which would have switched
+                                        arbitrage off every morning on every
+                                        dynamic site once carry-forward filling
+                                        landed (a series is part-firm and
+                                        part-carried until the 14-16 h CET
+                                        publication). What §8.3 forbids is
+                                        acting ON a provisional price, so the
+                                        provisional hours are DROPPED and
+                                        everything below is computed over the
+                                        firm ones. A wholly provisional series -
+                                        the §11.4 fallback - still has no firm
+                                        pair and is still refused.
+  4. the FIRM series is FLAT            max - min == 0, so there is nothing to
+                                        arbitrage. THIS is TARIFF-03's AC2: a
+                                        fully-fixed tariff disables arbitrage by
+                                        ARITHMETIC, not by a special case keyed
+                                        on `PriceProduct.FIXED`. A dynamic feed
                                         that happens to print one price all day
                                         gets refused for the same true reason.
+  5. the spread does not clear the      §8.3/§9.2/§11.4, new in FCST-03. The
+     WIDENED threshold                  caller states its round-trip cost
+                                        threshold; this widens it by the widest
+                                        active uncertainty source - a real-time
+                                        / imbalance PRODUCT (derived from the
+                                        declared `PriceProduct`, never from a
+                                        plugin id) or the caller's own
+                                        `NO_WEATHER`. See price.py's
+                                        `required_spread()`; the combination
+                                        rule is `max`, not a product.
 
 Nothing here decides a trade. The spread is handed over; OPT-04 owns
 `cycle_cost_threshold_eur` (§10.1/§14) and what to do about it. ⚠️ Worth knowing
@@ -182,7 +206,8 @@ from datetime import datetime, timedelta, timezone
 
 from .price import (
     MAX_PLAUSIBLE_EUR_KWH, Absence, PricePoint, PriceProduct, PriceRead,
-    _Outcome, brussels, load_prices, price_option,
+    Uncertainty, _Outcome, brussels, load_prices, price_option,
+    required_spread, uncertainty_of, widen_factor,
 )
 
 _LOG = logging.getLogger("goodwe.tariff.static")
@@ -619,8 +644,9 @@ class Arbitrage(_Outcome):
         return {"allowed": allowed, "refused": refused}[self.kind](self)
 
     @staticmethod
-    def allow(spread, cheapest, dearest, regime, product):
-        return _Allowed(spread, cheapest, dearest, regime, product)
+    def allow(spread, cheapest, dearest, regime, product, required, sources):
+        return _Allowed(spread, cheapest, dearest, regime, product, required,
+                        sources)
 
     @staticmethod
     def refuse(reason):
@@ -632,17 +658,43 @@ class Arbitrage(_Outcome):
         return _Refused(reason)
 
 
+# ⚠️ FIRM BY CONSTRUCTION, ASSERTED RATHER THAN TRUSTED (FCST-03). §8.3 forbids
+# an irreversible action on a provisional price, and the two points on the
+# permitting variant ARE the action: they are the hours a strategy buys and
+# sells in. A provisional point reaching this far is a defect in the refusal
+# ladder above, and it fails loudly at the moment of construction rather than
+# quietly at the meter a day later.
+def _firm_or_die(cheapest, dearest):
+    for role, pt in (("cheapest", cheapest), ("dearest", dearest)):
+        if getattr(pt, "provisional", True):
+            raise ValueError(
+                f"the {role} hour handed to an arbitrage strategy is "
+                f"PROVISIONAL ({getattr(pt, 'start', '?')}). §8.3: no "
+                f"irreversible action may be taken on a provisional price - "
+                f"an ALLOWED verdict cannot be built out of one.")
+
+
 class _Allowed(Arbitrage):
     kind = "allowed"
 
-    def __init__(self, spread, cheapest, dearest, regime, product):
+    def __init__(self, spread, cheapest, dearest, regime, product, required,
+                 sources):
+        _firm_or_die(cheapest, dearest)
         self.spread_eur_kwh = spread
         self.cheapest = cheapest          # the PricePoint at the low end
         self.dearest = dearest            # the PricePoint at the high end
         self.regime = regime              # §8.3 "wide" | "normal"
         self.product = product
+        # FCST-03. The threshold this spread actually cleared, and why it was
+        # the size it was. An optimiser that wants to know how much margin it
+        # has does not get to re-derive it from a product enum.
+        self.required_spread_eur_kwh = required
+        self.uncertainty = frozenset(sources)
         self.words = (f"spread {spread:.4f} EUR/kWh over "
-                      f"{product.value} ({regime} thresholds)")
+                      f"{product.value} ({regime} thresholds, cleared "
+                      f"{required:.4f} EUR/kWh)")
+
+
 
 
 class _Refused(Arbitrage):
@@ -724,36 +776,100 @@ class TariffPlan:
             no_data=lambda r: _absent(r.reason))
 
     # -- the two things a consumer is allowed to act on ----------------------
+    def firm_points(self) -> tuple:
+        """The points that carry a PUBLISHED price for their own hour (§8.3).
+
+        The complement of the carried-forward fill FCST-03 adds. Empty on a
+        §11.4 fallback plan and on a `no_data` read - and empty is honest here,
+        because it is a count of observations, not a value invented in place of
+        one. Never used to answer "what does this hour cost"; only to answer
+        "may something irreversible rest on it".
+        """
+        return self.read.match(
+            series=lambda r: tuple(p for p in r.points if not p.provisional),
+            no_data=lambda _r: ())
+
     def cheap_now(self, instant: datetime) -> bool:
         """Is now a cheap window? STRICTLY below the mean - see the docstring.
 
-        False on a provisional plan: §8.3 forbids an irreversible action on a
-        provisional price, and a grid top-up of a battery is about as
-        irreversible as this add-on gets. False when there is no price at all,
-        which is the §11.4 row and also main.py's long-standing "fixed-tariff
-        site: never force a paid grid top-up, wait for sun".
+        ⚠️ FALSE ON A PROVISIONAL HOUR, AND IT IS THIS HOUR THAT DECIDES
+        (FCST-03). §8.3 forbids an irreversible action on a provisional price
+        and a paid grid top-up of a battery is about as irreversible as this
+        add-on gets - but the price that would be paid is THIS hour's, so a
+        firm hour does not stop being cheap because tomorrow's hours have not
+        been published yet. Before FCST-03 the whole plan had to be firm, which
+        was the same answer for the only two series that existed then (all firm,
+        or all provisional) and would have switched the cheap window off every
+        morning once carry-forward filling landed.
+
+        The mean is taken over the FIRM hours for the same reason: yesterday's
+        carried values are not observations of what today costs, and a
+        comparison basis half-built out of them grades this hour against a
+        number nobody published. On an all-firm series - every series that
+        existed before FCST-03 - it is the same mean to the last digit.
+
+        False when there is no price at all, which is the §11.4 row and also
+        main.py's long-standing "fixed-tariff site: never force a paid grid
+        top-up, wait for sun".
         """
-        if not self.has_prices or self.provisional:
+        firm = self.firm_points()
+        if not firm:
             return False
         try:
-            return self.price_at(instant) < self.mean_eur_kwh()
+            here = self.read.match(series=lambda r: r.at(instant),
+                                   no_data=lambda r: _absent(r.reason))
         except Absence:
             # An instant outside the series is an hour with no price (§11.4 for
             # that hour), not a cheap one.
             return False
+        if here.provisional:
+            return False
+        return here.eur_per_kwh < (sum(p.eur_per_kwh for p in firm) / len(firm))
 
-    def arbitrage(self, *, peak_guard_active: bool) -> Arbitrage:
+    def arbitrage(self, *, peak_guard_active: bool, min_spread_eur_kwh,
+                  uncertainty) -> Arbitrage:
         """May OPT-04 trade, and on what numbers. See the module docstring.
 
-        ⚠️ `peak_guard_active` IS REQUIRED AND KEYWORD-ONLY. §9.1/§10.1 put peak
-        shaving above arbitrage, and this is how that ordering survives a
-        strategy that has not been written yet: the spread is not obtainable
-        without answering the question.
+        ⚠️ ALL THREE ARGUMENTS ARE REQUIRED AND KEYWORD-ONLY, for one reason
+        each, and none of them has a default:
+
+          `peak_guard_active`   §9.1/§10.1 put peak shaving above arbitrage,
+                                and this is how that ordering survives a
+                                strategy that has not been written yet: the
+                                spread is not obtainable without answering the
+                                question.
+          `min_spread_eur_kwh`  the caller's own round-trip cost threshold
+                                (§10.1's `cycle_cost_threshold_eur`). A verdict
+                                that says "allowed" without being told what
+                                would make a cycle worth doing is a verdict
+                                about arithmetic, not about money. Pass 0.0 to
+                                mean "any positive spread", explicitly.
+          `uncertainty`         the §11.4 doubts the CALLER can see and this
+                                object cannot - today that is
+                                `Uncertainty.NO_WEATHER`. Defaulting it to the
+                                empty set would be this module asserting the
+                                weather forecast is healthy, which is a fact it
+                                has no way to observe. The product's own
+                                real-time uncertainty (§9.2 row 2) is derived
+                                here and unioned in, so AC3 cannot be forgotten
+                                by a caller who passes an empty set.
         """
         if not isinstance(peak_guard_active, bool):
             raise TypeError(
                 f"peak_guard_active must be a bool, not {peak_guard_active!r}. "
                 f"A truthy 'unknown' is how §10.1's priority 2 gets skipped.")
+        if isinstance(min_spread_eur_kwh, bool) or                 not isinstance(min_spread_eur_kwh, (int, float)) or                 not math.isfinite(min_spread_eur_kwh) or                 min_spread_eur_kwh < 0:
+            raise TypeError(
+                f"min_spread_eur_kwh must be a finite, non-negative number of "
+                f"EUR/kWh, not {min_spread_eur_kwh!r}. A negative threshold is "
+                f"one every spread clears.")
+        caller_sources = frozenset(uncertainty)
+        if any(not isinstance(u, Uncertainty) for u in caller_sources):
+            raise TypeError(
+                f"uncertainty must be Uncertainty members, not "
+                f"{sorted(map(repr, caller_sources))} - a string widens by "
+                f"nothing while reporting that it widened.")
+
         if peak_guard_active:
             return Arbitrage.refuse(
                 "the capacity-tariff peak guard is holding, and §10.1 puts peak "
@@ -765,26 +881,64 @@ class TariffPlan:
                 f"§11.4: {self.read.reason}. No series exists, so no spread "
                 f"exists; self-consumption, peak shaving and reserve management "
                 f"(§10.1 priorities 1, 2 and 4) are unaffected")
-        if self.provisional:
+
+        # ⚠️ REFUSAL 3, REWRITTEN BY FCST-03 FROM "ANY POINT IS PROVISIONAL" TO
+        # "THE FIRM POINTS ARE NOT ENOUGH TO TRADE ON", and the difference is
+        # the whole ticket. §8.3 fills the hours a day-ahead feed has not
+        # published yet with yesterday's profile and flags them - so from
+        # midnight until the 14-16 h CET publication, a normal healthy site has
+        # a series that is PART firm and PART carried. Refusing on "any point
+        # provisional" would switch arbitrage off every morning of every day on
+        # every dynamic site, which is not what §8.3 says and is a behaviour
+        # nothing would ever report. What §8.3 forbids is acting ON a
+        # provisional price - so the provisional hours are DROPPED from the
+        # trade set and everything below is computed over the firm ones only.
+        # A fully provisional series (the §11.4 fallback, or a pre-publication
+        # day with no fetch at all) has no firm pair left and is refused, which
+        # is the previous behaviour arrived at by a truer route.
+        pts = self.read.points
+        firm = tuple(p for p in pts if not p.provisional)
+        if len(firm) < 2:
             return Arbitrage.refuse(
-                f"§11.4: the plan is provisional (origin {self.origin!r}), and "
-                f"§8.3 forbids an irreversible action on a provisional price. "
-                f"Load shifting and peak shaving continue")
-        spread = self.spread_eur_kwh()
+                f"§8.3: only {len(firm)} of {len(pts)} hours carry a published "
+                f"price - the rest are provisional (origin {self.origin!r}), "
+                f"and no irreversible action may be taken on a provisional "
+                f"price. A buy hour and a sell hour must BOTH be firm before a "
+                f"cycle can be committed. Load shifting, self-consumption and "
+                f"peak shaving continue")
+
+        spread = round(max(p.eur_per_kwh for p in firm)
+                       - min(p.eur_per_kwh for p in firm), 6)
         if spread == 0.0:
             return Arbitrage.refuse(
-                f"the tariff is flat - every one of the "
-                f"{len(self.read.points)} hours costs "
-                f"{self.read.points[0].eur_per_kwh:.4f} EUR/kWh, so buying low "
-                f"and selling high is arithmetically impossible. §9.2: a fully "
+                f"the tariff is flat - every one of the {len(firm)} firm hours "
+                f"costs {firm[0].eur_per_kwh:.4f} EUR/kWh, so buying low and "
+                f"selling high is arithmetically impossible. §9.2: a fully "
                 f"fixed product is self-consumption only")
-        pts = self.read.points
+
+        # REFUSAL 5, NEW IN FCST-03 (AC3). One widening path, driven by the set
+        # of active uncertainty sources: the product's own (§9.2 row 2, derived
+        # from the DECLARED product and never from a plugin id) unioned with
+        # whatever the caller can see (§11.4's no-weather row).
+        sources = uncertainty_of(self.read.product) | caller_sources
+        need = required_spread(min_spread_eur_kwh, sources)
+        if spread < need:
+            return Arbitrage.refuse(
+                f"the firm spread is {spread:.4f} EUR/kWh and this cycle needs "
+                f"{need:.4f} - {min_spread_eur_kwh:.4f} widened by "
+                f"{widen_factor(sources):.2f}x because "
+                f"{', '.join(sorted(u.value for u in sources)) or 'nothing'} "
+                f"is uncertain (§8.3, §9.2, §11.4). A cycle whose spread does "
+                f"not cover its own round-trip cost costs money and pack life")
+
         return Arbitrage.allow(
             spread=spread,
-            cheapest=min(pts, key=lambda p: p.eur_per_kwh),
-            dearest=max(pts, key=lambda p: p.eur_per_kwh),
+            cheapest=min(firm, key=lambda p: p.eur_per_kwh),
+            dearest=max(firm, key=lambda p: p.eur_per_kwh),
             regime=self.read.regime,
-            product=self.read.product)
+            product=self.read.product,
+            required=need,
+            sources=sources)
 
     @property
     def words(self) -> str: