FCST-05: one byte stops stopping the add-on, and the write path gets the read path's standard

HIGH - a non-UTF-8 byte in /data/price_history.json stopped the add-on
from starting. `_load()` caught `(json.JSONDecodeError, OSError)`.
`UnicodeDecodeError` is `ValueError -> UnicodeError -> UnicodeDecodeError`:
neither of those, and `issubclass(UnicodeDecodeError, OSError)` is False. So
it escaped `_load()` -> `_open()` -> `PriceHistory.__init__`, which
`Controller.__init__` does not guard - breaking `_load()`'s own stated
contract, "a corrupt cache must not stop the controller". The handler now
names the ValueError FAMILY (the file's content) beside OSError (its bytes),
because a third subclass nobody has thought of is how this arrived the first
time. The suite could not see it: its "unreadable file" fixture writes
"{ this is not json", which is valid UTF-8 and fails one step later, in
`json`. A fixture holding a genuinely undecodable byte is now beside it,
with the difference between the two asserted as a precondition.

MEDIUM - a malformed `start` under the day the feed is writing to killed the
tariff plan permanently. `_remember()` merged with `{q["start"]: q ...}` and
sorted on `q["start"]`, trusting a value read off /data to be a hashable,
sortable string. An int, null, a dict or a list raised TypeError out of
`carry()` -> `resolve_tariff()` -> `run_tariff_refresh`, which catches
Exception - so the add-on stayed UP while every refresh raised forever, the
plan retiring after TARIFF_MAX_AGE_S with arbitrage and the cheap window off
for good, and no self-heal because `_remember()` raised before `_save()`.
The read path was properly defensive throughout, which is why `yesterday()`
survived and the existing fixture passed - it files its malformed rows under
a day the feed is NOT writing to. Both paths now share one validator,
`_point_of()`. A row that is not a point is dropped, reported, and written
back out clean.

...and the status row said the store was healthy while the plan was dead:
`words` read '2026-01-15 (1 h)'. `_load()` now counts the rows inside a kept
record as well as the record itself, so the store stops reporting plain
health from the moment it is opened; the note names what was FOUND at open
and what was DROPPED by the refresh, and clears once the file is rewritten.

...and the merge key is the validated INSTANT rather than its spelling,
which is what rule 4 already said. The lexicographic sort beside the string
key also put the second 02:00 of the 25-hour October day (+01:00) before the
first (+02:00); stored order only, since `_parse_points()` re-sorts.

LOW - the status row counted POINTS and printed them under the HOUR's name:
'2026-01-13 (96 h)' for a 24-hour day on the quarter-hourly feed this add-on
already supports. It is now '96 points over 24 h' - two facts, each under
its own unit.

LOW - two stated reasons were false. `_audit`'s "WHY IT IS STRICTER" claimed
the rework refused the same published point emitted twice and a carried
point sharing a published hour, "which the old count never looked at". A
differential over the whole domain the audit can express finds ZERO
divergence in both directions: the old `seen` count already refused the
first (one point twice is one hour twice) and the old IDENTITY rule already
refused the second. The reworked audit is EQUAL to FCST-03's, not stronger;
the only behavioural change is the intended relaxation, and it stays. The
differential is committed as a check rather than left as prose.

...and the audit's real limit is named: it does not check COMPLETENESS. Fed
the decimation defect FCST-04 was raised for it ACCEPTS, and it accepts an
empty output. So `_audit` is not what stops that recurring - the count and
identity checks are, and the text now points at them. Not made complete:
`published` is the whole read while the fill emits only [since, until), so
the rule as stated is wrong, and building it takes the 1/12/48 h horizon
checks red on a healthy feed. Measured, not assumed.

12 mutations, purged tree, every one RED, no survivors. All 29 test files
green. Nothing deployed; no option added; config.yaml untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU

goodwe_controller/CHANGELOG.md

@@ -13,6 +13,89 @@ if it has, it needs 0.10.0. Whoever deploys decides.
 | | |
 |---|---|
 | _this branch_ | FCST-04 - the carry-forward fill reaches the plan, and the day it carries survives a restart |
+| _this branch_ | FCST-05 - one byte stops stopping the add-on, and the write path gets the read path's standard |
+
+### FCST-05 - what FCST-04's own contract still did not hold
+
+**A single non-UTF-8 byte in `/data/price_history.json` stopped the add-on from
+starting.** `_load()` caught `(json.JSONDecodeError, OSError)`.
+`UnicodeDecodeError` is `ValueError -> UnicodeError -> UnicodeDecodeError` -
+neither of those, and `issubclass(UnicodeDecodeError, OSError)` is **False** -
+so it escaped `_load()` -> `_open()` -> `PriceHistory.__init__`, which
+`Controller.__init__` does not guard. Reproduced: `UnicodeDecodeError: 'utf-8'
+codec can't decode byte 0xff`, and **the controller did not come up**. That is the same class as the day-record defect the store was reworked
+for, in the one byte class the rework left uncovered: valid-UTF-8-invalid-JSON,
+a UTF-8 BOM and a non-dict day record all degraded correctly. The suite could
+not see it because its "unreadable file" fixture writes `"{ this is not json"`,
+which is **valid UTF-8**. The handler now names the `ValueError` family - the
+file's CONTENT - beside `OSError`, the file's BYTES, and a fixture holding a
+genuinely undecodable byte is in `test_price_history.py` section 5.
+
+**A malformed `start` under the day the feed is writing to killed the tariff
+plan permanently.** `_remember()` merged a day's points with
+`{q["start"]: q ...}` and sorted on `q["start"]`, trusting a value read off
+`/data` to be a hashable, sortable string. An `int`, `null`, a `dict` or a
+`list` there raised `TypeError`; `run_tariff_refresh` catches `Exception`, so
+the add-on stayed **up** while **every refresh raised, forever** - the plan
+retiring after `TARIFF_MAX_AGE_S` and arbitrage and the cheap window off for
+good, with no self-heal because `_remember()` raised before reaching `_save()`.
+The read path (`_parse_points()`) was properly defensive throughout, which is
+why `yesterday()` survived and the existing fixture passed: it files its
+malformed rows under a day the feed is **not** writing to, so `_remember()`
+never opened that record. **The two paths now share one validator**,
+`_point_of()`, and neither has a standard of its own. A row that is not a point
+is dropped and reported, and `_save()` rewrites the record clean.
+
+⚠️ **AND THE STATUS ROW SAID THE STORE WAS HEALTHY WHILE THE PLAN WAS DEAD.**
+`words` read `'2026-01-15 (1 h)'` about the record that was killing every
+refresh. `_load()` now counts the rows inside a kept record as well as the
+record itself, so the store stops reporting plain health from the moment it is
+opened; the note names what was **found** at open and what was **dropped** by
+the refresh, and clears itself once the file has been rewritten.
+
+⚠️ **AND THE ROW COUNTED POINTS AND CALLED THEM HOURS.** `f"{d} ({_n_points(d)}
+h)"` on the quarter-hourly feed this add-on already supports reads
+`'2026-01-13 (96 h)'` for a 24-hour day - the same points/hours conflation the
+sub-hourly defect above was about, on the row that defect's own card shipped.
+The cell is now `2026-01-13 (96 points over 24 h)`: two facts, each under its
+own unit, and the hour figure is the one that says whether tomorrow morning's
+fill has a whole day to carry.
+
+⚠️ **AND THE MERGE KEY IS THE INSTANT RATHER THAN ITS SPELLING**, which is what
+rule 4 already said. Aware datetimes hash and compare by instant. The string
+key merged by spelling, and the lexicographic sort beside it put the second
+02:00 of the 25-hour October day (`+01:00`) **before** the first (`+02:00`);
+only the stored order was affected, because `_parse_points()` re-sorts by
+instant on the way out.
+
+⚠️ **TWO STATED REASONS WERE FALSE, AND THIS REPOSITORY TREATS THAT AS A
+DEFECT.** `price_forecast._audit()`'s "WHY IT IS STRICTER" section claimed the
+rework refused two things FCST-03's audit did not. A differential over the
+whole domain the audit can express - every published mask over 3 hours crossed
+with every output series of length 0-3 - found **zero divergence in both
+directions**: the reworked audit is exactly **EQUAL** to FCST-03's, not
+stronger. The old `seen` count already refused the same published point emitted
+twice (one point twice is one hour twice), and the old **identity** rule
+already refused a carried point sharing a published hour. The only behavioural
+change is the intended **relaxation** - several distinct published points in
+one hour - and that stays. The docstring and the two check texts now say what
+is true, and the differential is committed as a check in
+`test_price_forecast.py` rather than left as prose.
+
+⚠️ **AND `_audit` IS NOT WHAT STOPS THE DECIMATION DEFECT RECURRING.** It does
+not check **completeness**: fed the decimation directly - four published points
+in an hour, output emitting only the last - it **accepts**, and it accepts an
+empty output. That is a limit, not a regression, and the module docstring said
+otherwise by presenting the audit as the post-condition guarding provenance.
+What actually guards it is the point **count** and the pass-through **identity**
+checks in `test_price_forecast.py` and `test_price_history.py`, and the text now
+points at them. It is **not** made complete: `published` is the whole read while
+the fill emits only `[since, until)`, so "every published point appears" is not
+a true post-condition of those two arguments. Measured - the rule as stated
+takes the 1 h, 12 h and 48 h horizon checks red on legitimate series.
+
+**No option was added, `config.yaml` is untouched, and nothing was deployed.**
+
 
 **FCST-03 shipped the fill and no caller.** `price_forecast.fill_from_yesterday()`
 was complete and checked, and so were the consumers written for it -
@@ -58,8 +141,9 @@ spread it would have created was never compared against anything.
   no durable volume passes the explicit `NoHistory()`. A default of `None`
   meaning "skip the fill" would hand a caller who forgot the argument exactly
   the truncated horizon this card removes, silently.
-* A new status row, **Price history**, names the days held and their hour
-  counts.
+* A new status row, **Price history**, names the days held, how many points
+  each holds and how many whole hours they cover (see FCST-05: it shipped
+  counting points and printing them under the hour's name).
 
 ⚠️ **ONE CHANGE HERE MOVES ENERGY, AND IT IS NOT THE ONE THE CARD IS NAMED FOR.**
 Carried hours are provisional, so §8.3 keeps them out of the trade set: the fill

goodwe_controller/DOCS.md

@@ -1474,8 +1474,13 @@ a cheap night that has already gone used to sit in the series as the cheapest
 hour of the day, and a cycle can never buy in an hour that is past — so the
 trade was quietly never taken. The spread is now taken over hours still ahead.
 
-A new status row, **Price history**, says which days are held and how many hours
-each has. If it reads `empty` or `unavailable`, nothing is carried, the horizon
+A new status row, **Price history**, says which days are held, how many price
+points each holds and how many whole hours those cover — `2026-01-13 (96 points
+over 24 h)`. ⚠️ **Those are two different numbers and the row used to print one
+of them under the other's name**: it counted points and labelled them `h`, so a
+quarter-hourly feed read `96 h` for a 24-hour day. The figure that says whether
+tomorrow morning's fill has a whole day to carry is the **hour** one. If it
+reads `empty` or `unavailable`, nothing is carried, the horizon
 is as short as the feed makes it, and every other guarantee is unchanged — a
 cache that cannot answer is not allowed to be a controller that cannot regulate.
 The store is rebuilt by the next successful fetch; on a brand-new install the

goodwe_controller/MUTATION.md

@@ -127,6 +127,21 @@ Two rules come out of it, and both are cheap:
    function, that pair is the first thing to try — TEST-16's two survivors were
    adjacent lines in one function and neither was individually live.
 
+⚠️ **AND FCST-05 WALKED INTO IT AGAIN, IN ITS OWN NEW CHECK, WHICH IS WHY THE
+RULE IS “mutate every check you add”.** `price_history.py` reports a malformed
+point row from **two** places into **one** field: `_usable_days()` records what
+it FOUND when the file was opened, and `_remember()` records what it DROPPED on
+the next refresh. Both land in `last_error`, and `words` prints `last_error`.
+The first draft of the new check asked `"malformed point row" in words` — which
+whichever rung is still standing satisfies — so deleting `_remember()`'s note
+entirely was a **−1 B SURVIVOR of the whole suite**, on the half of the fix that
+tells an operator the cache **self-healed**. It is graded on the TENSE now
+(`… in <day>` at open, `… dropped from <day>` after the refresh), the two are
+asserted to DIFFER, and the paired mutation — both notes deleted, −91 B — is
+red on 15 named checks. Same shape as TEST-16 and the same fix: **grade the
+reason, not the verdict**, and when two rungs answer one stimulus, mutate them
+together.
+
 ## The third way a check lies: its DATA cannot reach the failure it names
 
 Stale bytecode manufactures survivors from a mutation that never ran. Masking

goodwe_controller/app/price_forecast.py

@@ -55,6 +55,17 @@ So the flag is not passed through anywhere in this file:
     It is a post-condition, not a filter: it never fixes a flag, because a
     module that quietly corrects its own provenance is one whose provenance
     cannot be trusted.
+
+    ⚠️ IT GRADES THE POINTS THAT ARE EMITTED AND NOTHING ABOUT THE ONES THAT
+    ARE NOT (FCST-05). `_audit()` is a provenance check, NOT a completeness
+    check: fed FCST-04's own decimation defect - four published points in an
+    hour, only the last emitted - it ACCEPTS, and it accepts an empty output
+    too. Measured, and the FCST-03 audit it replaced was equally blind, so
+    this is a limit rather than a regression. What actually guards against a
+    decimated series is the point COUNT and the pass-through IDENTITY checks
+    in `test_price_forecast.py` (section on the quarter-hourly feed) and
+    `test_price_history.py`; see `_audit()`'s own docstring for why the
+    completeness rule is not expressible from its two arguments.
   * and test_price_forecast.py walks THIS FILE'S AST and fails if any
     `PricePoint(...)` in it passes anything but a literal for `provisional`, so
     the pass-through idiom cannot come back in a later edit and pass review by
@@ -225,14 +236,67 @@ def _audit(points, published) -> None:
       * CARRIED - the hour must not appear in `published` AT ALL, the flag must
         be exactly True, and that hour may hold no second point.
 
-    ⚠️ WHAT REPLACED "AT MOST ONE POINT PER HOUR", AND WHY IT IS STRICTER. This
-    audit began as a `seen` set: any repeated `_utc_hour` raised, because a
-    duplicated hour is how a DST day gets mispriced. A bare hour count cannot
-    express a quarter-hourly feed, where four distinct published points share
-    one hour legitimately - so the count was replaced by the two rules above,
-    which subsume it. A repeated CARRIED hour still raises; so does the same
-    published point emitted twice; and a carried point may no longer share an
-    hour with a published one, which the count never checked at all.
+    ⚠️ WHAT REPLACED "AT MOST ONE POINT PER HOUR", AND WHAT THAT DID AND DID
+    NOT CHANGE. This audit began as FCST-03's `seen` set: any repeated
+    `_utc_hour` raised, beside an identity rule `p is not src`. A bare hour
+    count cannot express a quarter-hourly feed, where four distinct published
+    points share one hour legitimately - so the count was replaced by the two
+    rules above, which subsume it.
+
+    ⚠️ AND IT IS NOT STRICTER. FCST-04's version of this paragraph claimed two
+    strengthenings and BOTH WERE FALSE; FCST-05 measured it. A differential
+    over the whole domain this audit can express - every published mask over 3
+    hours crossed with every output series of length 0-3 drawn from
+    {published, published-provisional, carried, carried-unflagged,
+    copy-of-published} - found ZERO divergence in EITHER direction:
+
+        OLD raised, NEW accepted (WEAKER):   0
+        OLD accepted, NEW raised (STRONGER): 0
+
+    The two specific claims, one at a time:
+
+      * "so does the same published point emitted twice" - the OLD `seen`
+        count already raised on that, because emitting one point twice is one
+        HOUR twice. Preserved by the identity/`emitted` rules, not added.
+      * "a carried point may no longer share an hour with a published one,
+        which the count never checked at all" - true of the COUNT, and the old
+        IDENTITY rule raised on it anyway: a carried point in a published hour
+        took the `else` branch and failed `p is not src`. The message was about
+        a copy rather than about a carry, which is the only thing that moved.
+
+    The one real behavioural change is the intended RELAXATION - several
+    distinct published points in one hour, which the old audit could not
+    express at all and refused. That relaxation is why this was rewritten, and
+    it stays. A right decision with a fabricated reason is still a defect here.
+
+    ⚠️ AND THE REAL LIMIT, WHICH IS NOT A REGRESSION AND IS NOT SMALL: THIS
+    AUDIT DOES NOT CHECK COMPLETENESS. It grades every point that IS emitted
+    and says nothing about the ones that are not. Fed the H1 decimation
+    directly - four published points in an hour, output emitting only the last
+    - it ACCEPTS; so does two of four, and so does an EMPTY output. So `_audit`
+    is NOT what stops H1 recurring. What stops it is the point COUNT and the
+    IDENTITY checks in `test_price_history.py` and `test_price_forecast.py`,
+    which compare the emitted series against the feed's own prints.
+
+    It is not made complete here, and the reason is structural rather than
+    effort: `published` is `_published_by_hour(read)` over the WHOLE read,
+    unwindowed, while `fill_from_yesterday()` emits only [since, until). A
+    published point outside the window is legitimately absent, so "every
+    published point appears" is not a true post-condition of these two
+    arguments - it would need the window passed in, which is a different
+    signature and a different check.
+
+    ⚠️ AND THAT IS MEASURED, NOT ASSUMED. FCST-05 built the rule - raise unless
+    every point of every `published[u]` is emitted, +162 B - and ran it. It
+    takes the 1 h, 12 h and 48 h HORIZON checks in test_price_forecast.py RED,
+    on the shipped fill and a healthy feed: at an off-the-hour instant the
+    feed's already-elapsed prints are outside [since, until) and are correctly
+    dropped, and the rule calls that a provenance failure. (The differential
+    beside those checks scores the same rule as 49 STRONGER cases, so it is a
+    real change of behaviour and not a no-op.) The rule as stated is therefore
+    not merely unimplemented, it is WRONG - and stating a false post-condition
+    is MUTATION.md habit 16's defect: an instrument that asserts a completeness
+    it does not have is worse than no instrument.
 
     ⚠️ RAISES, NEVER REPAIRS. A module that silently corrects its own
     provenance is one whose provenance means nothing; the only safe response to

goodwe_controller/app/price_history.py

@@ -208,8 +208,65 @@ def _yesterday_of(instant: datetime) -> str:
     return (brussels(instant).date() - timedelta(days=1)).strftime("%Y-%m-%d")
 
 
-def _n_points(rec) -> int:
-    """How many points a day record holds. 0 for anything that is not one.
+def _point_of(row):
+    """One stored row -> a `PricePoint`, or `None` if the row is not one.
+
+    THE ONE VALIDATOR, AND THE ASYMMETRY IT CLOSES (FCST-05's MEDIUM). This
+    file used to hold TWO standards for what a stored point is. The READ path
+    (`_parse_points()`) was properly defensive - every field checked, a
+    malformed row dropped and reported. The WRITE path was
+    `{q["start"]: q for q in raw if isinstance(q, dict) and "start" in q}`
+    followed by `sorted(..., key=lambda q: q["start"])`, which trusts
+    `q["start"]` to be a hashable, sortable string it never checked. Measured,
+    with the malformed row under the day the feed is currently writing to:
+
+        start is an int   -> TypeError: '<' not supported between instances
+                             of 'str' and 'int'
+        start is a dict   -> TypeError: cannot use 'dict' as a dict key
+        start is null     -> TypeError: '<' not supported between 'str' and
+                             'NoneType'
+        start is a list   -> TypeError: cannot use 'list' as a dict key
+
+    ⚠️ AND THE BLAST RADIUS IS PERMANENT, WHICH IS WHY IT IS NOT MERELY UNTIDY.
+    `_remember()` is reached through `carry()` -> `resolve_tariff()` ->
+    `run_tariff_refresh`, which catches `Exception` - so the add-on stays UP
+    and EVERY refresh raises forever. The plan retires after
+    `TARIFF_MAX_AGE_S`, and arbitrage and the cheap window are off from then
+    on. There is no self-heal, because `_remember()` raises before it reaches
+    `_save()`. `_usable_days()` only required `rec["points"]` to be a LIST; the
+    elements were unchecked on the write path, and the existing fixture put its
+    malformed row under a day the feed was NOT writing to, so `_remember()`
+    never touched it.
+
+    So both paths call this, and neither of them has a standard of its own.
+    Returning `None` rather than raising is the contract: a caller decides
+    whether a bad row is dropped (both of them do) and how it is reported.
+    """
+    if not isinstance(row, dict):
+        return None
+    try:
+        start = datetime.fromisoformat(row["start"])
+        value = float(row["eur_per_kwh"])
+        flag = row["provisional"]
+    except (KeyError, TypeError, ValueError):
+        return None
+    if start.tzinfo is None or start.utcoffset() is None \
+            or not isinstance(flag, bool):
+        return None
+    return PricePoint(start=start, eur_per_kwh=value, provisional=flag)
+
+
+def _held(rec) -> tuple:
+    """`(points, whole hours they cover)` for a day record. `(0, 0)` otherwise.
+
+    ⚠️ POINTS AND HOURS ARE TWO NUMBERS, AND FCST-04's STATUS ROW PRINTED THE
+    FIRST UNDER THE SECOND'S NAME. The row was `f"{d} ({_n_points(rec)} h)"`,
+    which counts POINTS: on the quarter-hourly feed this add-on already
+    supports it read `'2026-01-13 (96 h)'` - 96 "h" for a 24-hour day. That is
+    the points/hours conflation H1 was raised for, on the row H1's own ticket
+    shipped. Both numbers are reported now, so an operator can see a day that
+    is 24 points over 24 h and a day that is 96 points over 24 h and tell them
+    apart, and neither number is wearing the other's unit.
 
     ⚠️ TOLERANT, AND IT IS THE SECOND LINE OF DEFENCE, NOT THE FIRST. `_load()`
     drops a malformed day record and says so, so this should never see one. It
@@ -220,11 +277,52 @@ def _n_points(rec) -> int:
     "oops"}}` gave `ok=True`, `last_error=None`, and an `AttributeError` out of
     `words`, `yesterday()` and `resolve_tariff()` alike, with no self-heal
     because `_remember()` raised before it could rewrite the file.
+
+    ⚠️ AND IT COUNTS ONLY ROWS THAT ARE POINTS. A row `_point_of()` refuses is
+    not held prices, and a row count that included it would report a store as
+    healthier than it is - which is the second half of FCST-05's MEDIUM: the
+    status row said `'2026-01-15 (1 h)'` about a record whose single row was
+    killing every tariff refresh. What is dropped is named in `last_error`, and
+    `words` prints that beside the counts.
     """
+    if not isinstance(rec, dict):
+        return (0, 0)
+    raw = rec.get("points")
+    if not isinstance(raw, (list, tuple)):
+        return (0, 0)
+    n, hours = 0, set()
+    for row in raw:
+        pt = _point_of(row)
+        if pt is None:
+            continue
+        n += 1
+        hours.add(pt.start.astimezone(timezone.utc)
+                  .replace(minute=0, second=0, microsecond=0))
+    return (n, len(hours))
+
+
+def _row_of(day: str, points: int, hours: int) -> str:
+    """One day's cell of the status row: what is held, in BOTH of its units.
+
+    ⚠️ THE UNIT IS SPELT OUT ON EACH NUMBER. FCST-04's cell was
+    `f"{day} ({points} h)"` - the point count wearing the hour's name, which on
+    a quarter-hourly feed reads `'2026-01-13 (96 h)'` for a 24-hour day. What
+    an operator needs from this row is whether tomorrow morning's fill has a
+    whole day to carry, and that is the HOUR figure; the point count says how
+    finely it is priced. Neither answers for the other.
+    """
+    return (f"{day} ({points} point{'' if points == 1 else 's'} "
+            f"over {hours} h)")
+
+
+def _bad_rows(rec) -> int:
+    """How many rows of a day record are not points. Total; never raises."""
     if not isinstance(rec, dict):
         return 0
-    pts = rec.get("points")
-    return len(pts) if isinstance(pts, (list, tuple)) else 0
+    raw = rec.get("points")
+    if not isinstance(raw, (list, tuple)):
+        return 0
+    return sum(1 for row in raw if _point_of(row) is None)
 
 
 class PriceHistory:
@@ -269,11 +367,33 @@ class PriceHistory:
                       "one, and the fill has nothing to carry until the day "
                       "after that", self.path)
             return
-        except (json.JSONDecodeError, OSError) as err:
+        except (ValueError, OSError) as err:
             # A corrupt cache must not stop the controller, and must not be
             # silently re-created either: the record is discarded, the reason
             # is kept, and `yesterday()` reports it rather than answering
             # "no record", which would read as a normal first day.
+            #
+            # ⚠️ `ValueError`, NOT `json.JSONDecodeError`, AND THAT IS
+            # FCST-05's HIGH. Reading this file has two failure families: the
+            # BYTES (`OSError`) and the CONTENT - and the content family has at
+            # least two members, because `open(..., encoding="utf-8")` decodes
+            # before `json` ever sees a character. `UnicodeDecodeError` is
+            # `ValueError -> UnicodeError -> UnicodeDecodeError`; it is NOT a
+            # `json.JSONDecodeError` and it is NOT an `OSError`
+            # (`issubclass(UnicodeDecodeError, OSError)` is False). So ONE
+            # non-UTF-8 byte in /data/price_history.json escaped `_load()` ->
+            # `_open()` -> `PriceHistory.__init__`, which is unguarded at
+            # main.py's `Controller.__init__`, AND THE ADD-ON DID NOT START.
+            # Measured: `UnicodeDecodeError: 'utf-8' codec can't decode byte
+            # 0xff in position 39`. Valid-UTF-8-invalid-JSON, a UTF-8 BOM and a
+            # non-dict day record all degraded correctly - it was the one byte
+            # class left uncovered, which is the same class as the defect the
+            # `_usable_days()` rework was raised for.
+            #
+            # The tuple names the FAMILY rather than the two members known
+            # today, because "the file's content is not readable as JSON" is
+            # exactly what this handler says, and a third `ValueError` subclass
+            # nobody has thought of is how this defect arrived the first time.
             self.last_error = f"unreadable ({err})"
             _LOG.warning("price history at %s is %s - starting from empty. "
                          "§8.3 carry-forward has nothing to carry until the "
@@ -318,20 +438,41 @@ class PriceHistory:
 
         Dropped, not repaired: an unusable record has nothing in it to keep, and
         the next `_remember()` writes a fresh one under that date.
+
+        ⚠️ AND THE ROWS INSIDE A KEPT RECORD ARE COUNTED, NOT ONLY THE RECORD
+        (FCST-05). This checked that `rec["points"]` is a LIST and stopped
+        there, so a record holding one row with a `start` of `7` loaded as
+        healthy: `last_error=None`, and `words` said `'2026-01-15 (1 h)'` about
+        a store whose next `_remember()` would raise on that row and go on
+        raising forever. The rows are NOT dropped here - `_parse_points()` is
+        the read path's one filter and stays so - but they are COUNTED, so the
+        store stops reporting health it does not have from the moment it is
+        opened. `_remember()` drops them on the next refresh and `_save()`
+        rewrites the file, after which a restart clears the note.
         """
-        kept, dropped = {}, []
+        kept, dropped, malformed = {}, [], {}
         for key, rec in days.items():
             if not isinstance(key, str) or not isinstance(rec, dict) \
                     or not isinstance(rec.get("points"), list):
                 dropped.append(str(key))
                 continue
+            bad = _bad_rows(rec)
+            if bad:
+                malformed[key] = bad
             kept[key] = rec
+        notes = []
         if dropped:
+            notes.append(f"{len(dropped)} unusable day record(s) dropped "
+                         f"({', '.join(sorted(dropped))})")
+        if malformed:
+            notes.append(
+                f"{sum(malformed.values())} malformed point row(s) in "
+                + ", ".join(f"{d} ({n})" for d, n in sorted(malformed.items())))
+        if notes:
             # Recorded, not silent: `yesterday()` and `words` both report
             # `last_error`, and a cache that quietly discards half a file reads
             # to an operator exactly like a normal first day.
-            self.last_error = (f"{len(dropped)} unusable day record(s) dropped "
-                               f"({', '.join(sorted(dropped))})")
+            self.last_error = "; ".join(notes)
             _LOG.warning("price history at %s: %s - the controller starts "
                          "anyway and the horizon is as short as the feed makes "
                          "it", self.path, self.last_error)
@@ -417,6 +558,7 @@ class PriceHistory:
                               no_data=lambda _r: None)
 
         touched = set()
+        junked = {}
         for p in points:
             day = _bdate(p.start)
             rec = self._days.setdefault(
@@ -424,14 +566,52 @@ class PriceHistory:
             # Rule 4: MERGE by instant. A later read updates the hours it
             # carries and leaves the rest; a revised price for an hour already
             # held wins, because it is the newer statement about that hour.
+            #
+            # ⚠️ EVERY HELD ROW GOES THROUGH `_point_of()` (FCST-05's MEDIUM).
+            # This was `{q["start"]: q for q in raw if isinstance(q, dict) and
+            # "start" in q}` sorted by `q["start"]` - a key and a sort key that
+            # trusted a value read off disk to be a hashable, sortable string.
+            # An `int`, `null`, a `dict` or a `list` there raised `TypeError`
+            # out of `carry()` on the day the feed was writing to, and
+            # `run_tariff_refresh` caught it, so the add-on stayed up while
+            # EVERY refresh failed for good. A row that is not a point is now
+            # dropped and reported here, exactly as the read path drops it, and
+            # `_save()` below writes the record back clean - which is the
+            # self-heal the raising version could never reach.
+            #
+            # ⚠️ AND THE MERGE KEY IS THE VALIDATED INSTANT, NOT ITS SPELLING.
+            # Aware datetimes hash and compare by instant, so this is rule 4 as
+            # rule 4 is written. The old string key merged by SPELLING, and the
+            # old lexicographic sort put the second 02:00 of the 25-hour
+            # October day (`+01:00`) BEFORE the first (`+02:00`). Only the
+            # stored order was affected - `_parse_points()` re-sorts by instant
+            # on the way out - but the store now holds what it says it holds.
             raw = rec.get("points")
-            held = {q["start"]: q for q in (raw if isinstance(raw, list) else ())
-                    if isinstance(q, dict) and "start" in q}
-            held[p.start.isoformat()] = p.canonical()
-            rec["points"] = sorted(held.values(), key=lambda q: q["start"])
+            held = {}
+            for q in (raw if isinstance(raw, list) else ()):
+                kept = _point_of(q)
+                if kept is None:
+                    junked[day] = junked.get(day, 0) + 1
+                    continue
+                held[kept.start] = q
+            held[p.start] = p.canonical()
+            order = sorted(held, key=lambda t: t.astimezone(timezone.utc))
+            rec["points"] = [held[t] for t in order]
             rec["product"], rec["provider"] = product, provider
             touched.add(day)
 
+        if junked:
+            # Said, not swallowed. `words` prints `last_error`, so a store that
+            # has just discarded rows off disk stops reporting plain health -
+            # and after `_save()` below the file is clean, so a restart clears
+            # this. `_save()` overwrites it if the write itself fails, which is
+            # the more urgent fact of the two.
+            self.last_error = (
+                f"{sum(junked.values())} malformed point row(s) dropped from "
+                + ", ".join(f"{d} ({n})" for d, n in sorted(junked.items())))
+            _LOG.warning("price history at %s: %s - the record is rewritten "
+                         "without them and the refresh continues",
+                         self.path, self.last_error)
         self._prune(stamp)
         self._save()
         _LOG.debug("price history: %d point(s) filed under %s; %d day(s) held",
@@ -523,15 +703,21 @@ class PriceHistory:
         an exception here is an add-on that does not start. It went out that
         way: a day record that was a string instead of a dict raised
         `AttributeError` from the `len(...)` below. `_load()` now drops such a
-        record and `_n_points()` is total, so this has two independent reasons
-        not to fail on a broken file.
+        record and `_held()` is total, so this has two independent reasons not
+        to fail on a broken file.
+
+        ⚠️ AND IT COUNTED POINTS AND CALLED THEM HOURS (FCST-05's LOW). The row
+        was `f"{d} ({_n_points(rec)} h)"`; on the quarter-hourly feed this
+        add-on already supports that reads `'2026-01-13 (96 h)'` for a 24-hour
+        day. Both numbers are printed now - see `_held()` - because they are
+        two different facts and only one of them was ever being shown.
         """
         if not self.ok:
             return f"unavailable - {self.last_error}"
         if not self._days:
             why = f" - {self.last_error}" if self.last_error else ""
             return f"empty{why} - nothing to carry forward yet"
-        held = "; ".join(f"{d} ({_n_points(self._days[d])} h)"
+        held = "; ".join(_row_of(d, *_held(self._days[d]))
                          for d in sorted(self._days, key=str))
         # A file that PARTLY loaded says so, or the dropped days read as days
         # that were simply never written.
@@ -550,21 +736,14 @@ def _parse_points(raw) -> tuple:
         return (), "the `points` field is not a list"
     out, dropped = [], 0
     for row in raw:
-        if not isinstance(row, dict):
+        # ⚠️ `_point_of()` IS THE ONE VALIDATOR, and this loop used to BE it.
+        # The write path had its own, weaker standard; see `_point_of()` for
+        # what that cost. One standard, two callers, no asymmetry to re-open.
+        pt = _point_of(row)
+        if pt is None:
             dropped += 1
             continue
-        try:
-            start = datetime.fromisoformat(row["start"])
-            value = float(row["eur_per_kwh"])
-            flag = row["provisional"]
-        except (KeyError, TypeError, ValueError):
-            dropped += 1
-            continue
-        if start.tzinfo is None or start.utcoffset() is None \
-                or not isinstance(flag, bool):
-            dropped += 1
-            continue
-        out.append(PricePoint(start=start, eur_per_kwh=value, provisional=flag))
+        out.append(pt)
     out.sort(key=lambda p: p.start.astimezone(timezone.utc))
     return tuple(out), (f"{dropped} malformed point(s)" if dropped else "")
 

goodwe_controller/test_price_forecast.py

@@ -59,6 +59,7 @@ Read MUTATION.md before you report a survivor.
 
 import ast
 import inspect
+import itertools
 import os
 import sys
 from datetime import datetime, timedelta, timezone
@@ -354,14 +355,164 @@ check("...and still REFUSES a repeated CARRIED hour, which is the DST "
       "mispricing that count existed to catch",
       raises(lambda: _fc._audit([_fc._carried(D0, 0.11),
                                  _fc._carried(D0, 0.12)], {}), ValueError))
-check("...and refuses the SAME published point emitted twice - the provider's "
-      "points are strictly increasing, so a repeat is never a second print",
+check("...and STILL refuses the SAME published point emitted twice - the "
+      "provider's points are strictly increasing, so a repeat is never a "
+      "second print. The old hour COUNT already refused this (one point twice "
+      "is one hour twice); what is graded here is that the relaxation did not "
+      "lose it, not that it added it",
       raises(lambda: _fc._audit([_QUARTERS.points[0], _QUARTERS.points[0]],
                                 _PUB_Q), ValueError))
-check("...and refuses a CARRIED point standing in an hour the feed DID "
-      "publish, which the old count never looked at",
+check("...and STILL refuses a CARRIED point standing in an hour the feed DID "
+      "publish. The old COUNT never looked at it; the old IDENTITY rule did, "
+      "and raised - so this is preserved behaviour, not new strictness. "
+      "FCST-04's text called it new here and in `_audit`'s docstring, and a "
+      "measured differential says otherwise",
       raises(lambda: _fc._audit([_fc._carried(D0, 0.11)], _PUB_Q), ValueError))
 
+# ⚠️ AND WHAT THE AUDIT DOES NOT DO, STATED WHERE A READER OF THESE THREE
+# CHECKS WOULD OTHERWISE ASSUME IT (FCST-05, MUTATION.md habit 16). `_audit` is
+# a PROVENANCE post-condition, not a COMPLETENESS one: it grades every point
+# that IS emitted and says nothing about points that are not. So it accepts the
+# very decimation FCST-04 was raised for. That is not a regression - FCST-03's
+# audit was equally blind - but it means the audit is NOT the thing that stops
+# H1 coming back. The count and identity checks above this line are.
+check("the audit ACCEPTS the H1 decimation itself - four published points in "
+      "one hour, only the last emitted - and accepts an EMPTY output. It is a "
+      "provenance check, not a completeness one, and the checks that guard "
+      "provenance completeness are the COUNT and IDENTITY ones above",
+      derived(lambda: _fc._audit([_QUARTERS.points[3]], _PUB_Q)) is None
+      and derived(lambda: _fc._audit([], _PUB_Q)) is None)
+
+
+# --------------------------------------------------------------------------- #
+# ⚠️ AND THE CLAIM "STRICTER" IS ITSELF A CHECK NOW (FCST-05).
+#
+# FCST-04 rewrote `_audit` and wrote, in the module and in the three checks
+# above, that the result was STRICTER than FCST-03's. It is not. Two of the
+# stated reasons were false - the old `seen` count already refused the same
+# point emitted twice, and the old IDENTITY rule already refused a carried
+# point in a published hour - and a right decision with a fabricated reason is
+# a defect in this repository (MUTATION.md, and the reason DECISIONS files
+# exist). So the claim is measured here rather than asserted in prose:
+# FCST-03's audit is reproduced below from dd40c56 - its RULES verbatim, its
+# message TEXT shortened, and nothing here reads a message - and the two are
+# run over the whole domain `_audit` can express, in BOTH directions.
+#
+# THE DOMAIN, AND ITS SIZE. Every published mask over 3 hours (2**3) crossed
+# with every output series of length 0-3 drawn from 3 hours x 5 point kinds.
+# The nominal cross-product is 28 928; three of those kinds - published,
+# published-provisional, copy-of-published - DO NOT EXIST for an hour the feed
+# did not publish, and those combinations are SKIPPED rather than faked, which
+# leaves 11 990 realisable comparisons. A fabricated "published point" for an
+# unpublished hour is not in either audit's domain, and inventing one would
+# grade the fixture rather than the code.
+#
+# ⚠️ THE OLD COPY IS A LITERAL, NOT A CALL. MUTATION.md habit 2: a differential
+# whose two sides both come from the shipped module compares a thing against
+# itself and moves with every edit. This side is frozen text.
+def _audit_fcst03(points, published):
+    """FCST-03's `_audit`, verbatim from dd40c56. Its `published` maps a UTC
+    hour to ONE point, which is exactly the expressiveness this ticket relaxed.
+    """
+    seen = set()
+    for p in points:
+        u = _fc._utc_hour(p.start)
+        if u in seen:
+            raise ValueError("the filled series repeats " + u.isoformat())
+        seen.add(u)
+        src = published.get(u)
+        if src is None:
+            if p.provisional is not True:
+                raise ValueError(u.isoformat() + " is carried and is flagged "
+                                 "provisional=" + repr(p.provisional))
+        else:
+            if p is not src:
+                raise ValueError(u.isoformat() + " is published but the series "
+                                 "carries a COPY of that point")
+
+
+def _verdict(fn, pts, pub):
+    try:
+        fn(list(pts), pub)
+        return "accept"
+    except ValueError:
+        return "raise"
+
+
+_DH = [D0 + timedelta(hours=6 + i) for i in range(3)]
+_weaker, _stronger, _compared = [], [], 0
+for _mask in itertools.product((0, 1), repeat=3):
+    _pub = {u: PricePoint(start=u, eur_per_kwh=0.10 + i, provisional=False)
+            for i, (u, _on) in enumerate(zip(_DH, _mask)) if _on}
+    _old_pub = dict(_pub)
+    _new_pub = {u: (q,) for u, q in _pub.items()}
+    # The five shapes an output point can take at a given hour. Three of them
+    # do not exist for an hour the feed did not publish, and those combinations
+    # are skipped rather than faked - a fabricated "published point" for an
+    # unpublished hour is not in either audit's domain.
+    _kinds = []
+    for _u in _DH:
+        _src = _pub.get(_u)
+        _kinds.append(("published", _src))
+        _kinds.append(("published-provisional",
+                       None if _src is None else
+                       PricePoint(start=_src.start,
+                                  eur_per_kwh=_src.eur_per_kwh,
+                                  provisional=True)))
+        _kinds.append(("carried", _fc._carried(_u, 0.5)))
+        _kinds.append(("carried-unflagged",
+                       PricePoint(start=_u, eur_per_kwh=0.5,
+                                  provisional=False)))
+        _kinds.append(("copy-of-published",
+                       None if _src is None else
+                       PricePoint(start=_src.start,
+                                  eur_per_kwh=_src.eur_per_kwh,
+                                  provisional=_src.provisional)))
+    for _n in range(4):
+        for _combo in itertools.product(_kinds, repeat=_n):
+            if any(k[1] is None for k in _combo):
+                continue
+            _pts = [k[1] for k in _combo]
+            _compared += 1
+            _o = _verdict(_audit_fcst03, _pts, _old_pub)
+            _nw = _verdict(_fc._audit, _pts, _new_pub)
+            if _o == "raise" and _nw == "accept":
+                _weaker.append((_mask, [k[0] for k in _combo]))
+            if _o == "accept" and _nw == "raise":
+                _stronger.append((_mask, [k[0] for k in _combo]))
+
+# ⚠️ THE PRECONDITION FIRST (MUTATION.md habit 1). A differential over a
+# domain that cannot reach the two disputed cases would score 0-0 for the wrong
+# reason and read exactly like this one, so the two cases are driven against
+# the OLD audit by name before the totals are graded.
+_probe = PricePoint(start=_DH[0], eur_per_kwh=0.1, provisional=False)
+check(f"the differential really covers the two claims it is here to settle, "
+      f"or it is {_compared} runs of nothing: FCST-03's own audit raises on "
+      f"BOTH 'the same published point twice' and 'a carried point in a "
+      f"published hour', which is what FCST-04 said it did not",
+      _compared > 3000
+      and _verdict(_audit_fcst03, [_probe, _probe], {_DH[0]: _probe}) == "raise"
+      and _verdict(_audit_fcst03, [_fc._carried(_DH[0], 0.5)],
+                   {_DH[0]: _probe}) == "raise")
+check(f"...and over all {_compared} of them the reworked audit is NOT WEAKER "
+      f"than FCST-03's: {len(_weaker)} case(s) the old one raised and this one "
+      f"accepts {_weaker[:2]}",
+      _weaker == [])
+check(f"...and NOT STRONGER either: {len(_stronger)} case(s) the old one "
+      f"accepted and this one raises {_stronger[:2]}. FCST-04's docstring and "
+      f"two of these checks said it was stricter and named two reasons, and "
+      f"BOTH were already refused by the old count and the old identity rule. "
+      f"The only behavioural change is the RELAXATION below, which is what the "
+      f"rework was for",
+      _stronger == [])
+_QP = [PricePoint(start=_DH[0] + timedelta(minutes=15 * i),
+                  eur_per_kwh=0.1 + i, provisional=False) for i in range(4)]
+check("...and the relaxation is real and is NOT expressible to the old audit: "
+      "four distinct published points inside one hour, which FCST-03's count "
+      "refused as a repeated hour and this one accepts",
+      _verdict(_fc._audit, _QP, {_DH[0]: tuple(_QP)}) == "accept"
+      and _verdict(_audit_fcst03, _QP, {_DH[0]: _QP[0]}) == "raise")
+
 # ⚠️ THE HORIZON IS WHOLE HOURS AT BOTH ENDS (FCST-04 L8). `since` is floored
 # and `until` was not, so a 48 h request made at any instant off the hour came
 # back 49 hours long. Graded at an OFF-THE-HOUR instant, because on the hour

goodwe_controller/test_price_history.py

@@ -34,6 +34,18 @@ cache cannot decay by being fed its own output.
        `PriceHistory(...)` and `.words` are unguarded in `Controller.__init__`
        - so the add-on did not start, and never self-healed.
 
+  AC5  (FCST-05) THE CACHE'S OWN CONTRACT HOLDS FOR EVERY BYTE AND EVERY ROW.
+       A file holding one non-UTF-8 byte no longer stops the add-on from
+       starting - `_load()` caught `(json.JSONDecodeError, OSError)` and
+       `UnicodeDecodeError` is neither, so it escaped `PriceHistory.__init__`
+       and `Controller.__init__` does not guard it. And a malformed `start`
+       under the day the feed is WRITING to no longer kills the tariff plan
+       for good: the write path shares `_point_of()` with the read path,
+       drops the row, says so, and `_save()` rewrites the record clean.
+       ⚠️ THE STATUS ROW IS PART OF THIS. It said `'2026-01-15 (1 h)'` about a
+       store whose every refresh was raising - health it did not have, in a
+       unit that was not its own (it counts POINTS).
+
 THE RULES THIS FILE HOLDS ITSELF TO, because the reviewer will:
 
   * EVERY VERDICT IS DERIVED. The carried/published split is re-derived here
@@ -262,6 +274,13 @@ YESTERDAY_MWH = {h: 400.0 + 3 * h for h in range(24)}
 
 MARKUP, TAXES = 0.03, 0.06
 
+# ⚠️ FCST-05's HIGH, AS BYTES. A cache that is structurally valid JSON up to
+# the point where one byte is not UTF-8 - which is what a truncated write, a
+# half-flushed page or a foreign writer produces. Written with `open(..., "wb")`
+# and never through a text handle, or the fixture cannot hold the byte it is
+# named for. `0xff` is not a valid UTF-8 start byte in any position.
+_UTF8_BOMB = b'{"version": 1, "days": {"2026-01-14": "\xff\xfe"}}'
+
 
 def allin(mwh):
     """EUR/MWh as published -> the all-in EUR/kWh `assemble()` produces."""
@@ -896,6 +915,55 @@ check("an unreadable file: the reason is kept and reported, and the record is "
       and _h6.yesterday(NOW).kind == "no_data"
       and "unreadable" in why(_h6.yesterday(NOW)))
 
+# ⚠️ AND THE BYTE CLASS THAT FIXTURE CANNOT REACH - FCST-05's HIGH. The row
+# above is INVALID JSON but it is PERFECTLY VALID UTF-8, so it is decoded and
+# then refused by `json`, which raises `json.JSONDecodeError`. A file holding
+# one byte that is not UTF-8 at all fails EARLIER, in the decode `open(...,
+# encoding="utf-8")` performs before `json` sees a character, and
+# `UnicodeDecodeError` is `ValueError -> UnicodeError -> UnicodeDecodeError`:
+# NOT a `JSONDecodeError`, and NOT an `OSError` either. `_load()` caught
+# `(json.JSONDecodeError, OSError)`, so it escaped `_load()` -> `_open()` ->
+# `PriceHistory.__init__` - unguarded in `Controller.__init__` - AND THE ADD-ON
+# DID NOT START, on a cache whose own contract says "a corrupt cache must not
+# stop the controller".
+#
+# The precondition is asserted first, because it is the whole reason the check
+# above could not see this: if `"{ this is not json"` were undecodable, the two
+# fixtures would be one fixture and this would grade nothing (MUTATION.md
+# habit 1).
+check("the two unreadable-file fixtures are DIFFERENT FAILURES: the one above "
+      "is valid UTF-8 that is not JSON, and the one below is not UTF-8 at all. "
+      "A file that is merely bad JSON cannot reach the decode step, which is "
+      "why the suite had this hole",
+      open(_p6, "rb").read().decode("utf-8") == "{ this is not json"
+      and raises(lambda: _UTF8_BOMB.decode("utf-8"), UnicodeDecodeError))
+
+_d6b = tmpdir()
+_p6b = os.path.join(_d6b, "price_history.json")
+open(_p6b, "wb").write(_UTF8_BOMB)
+_h6b = derived(lambda: PriceHistory(_p6b))
+check(f"a single non-UTF-8 byte in the cache does NOT stop the store from "
+      f"being constructed {_h6b!r}. `PriceHistory(...)` is unguarded in "
+      f"`Controller.__init__`, so an exception here is an add-on that does not "
+      f"start - which is what shipped",
+      isinstance(_h6b, PriceHistory))
+check(f"...and it degrades the SAME WAY the bad-JSON file does: the reason is "
+      f"kept, named and reported "
+      f"{(_h6b.last_error if isinstance(_h6b, PriceHistory) else _h6b)!r}",
+      isinstance(_h6b, PriceHistory)
+      and "unreadable" in (_h6b.last_error or "")
+      and "codec can't decode byte" in (_h6b.last_error or "")
+      and _h6b.yesterday(NOW).kind == "no_data"
+      and "unreadable" in why(_h6b.yesterday(NOW)))
+_w6b = derived(lambda: _h6b.words)
+_plan6b = derived(lambda: run(resolve_tariff(
+    CFG, NOW, _until, transport=transport_for(_body_today),
+    now=NOW, history=_h6b)).origin)
+check(f"...and NOTHING downstream raises on it either: not `words` ({_w6b!r}), "
+      f"not the shipped `resolve_tariff()` path ({_plan6b!r}). §6: absent "
+      f"machinery must not disable the loop, and a JSON file is machinery",
+      isinstance(_w6b, str) and _plan6b == "provider")
+
 _p7 = os.path.join(tmpdir(), "price_history.json")
 json.dump({"version": SCHEMA + 99, "days": {}}, open(_p7, "w", encoding="utf-8"))
 _h7 = PriceHistory(_p7)
@@ -1008,11 +1076,49 @@ check("a healthy but EMPTY store says something different, because it is a "
       and words_of(_h_words).startswith("empty")
       and "nothing to carry forward" in words_of(_h_words))
 _ph.PriceHistory._remember(_h_words, series_of(YDAY_H, YESTERDAY_MWH), NOW)
-check(f"...and a store holding a day names the DATE and the HOUR COUNT "
-      f"{words_of(_h_words)!r} - 'a cache exists' and 'the cache holds "
-      f"yesterday, all 24 hours of it' are different facts, and only the "
-      f"second says the horizon will be full tomorrow morning",
-      words_of(_h_words) == f"{YDAY_D} (24 h)")
+check(f"...and a store holding a day names the DATE, the POINT COUNT and the "
+      f"HOUR COUNT {words_of(_h_words)!r} - 'a cache exists' and 'the cache "
+      f"holds yesterday, all 24 hours of it' are different facts, and only "
+      f"the second says the horizon will be full tomorrow morning",
+      words_of(_h_words) == f"{YDAY_D} (24 points over 24 h)")
+
+# ⚠️ AND THE TWO NUMBERS ARE NOT THE SAME NUMBER - FCST-05's LOW. The row
+# shipped as `f"{d} ({_n_points(rec)} h)"`, which counts POINTS and prints them
+# under the HOUR's name. On an hourly feed the two are equal and nothing shows;
+# on the quarter-hourly feed this add-on already supports (`_published_by_hour`
+# keeps a TUPLE per hour precisely for it, and §7 below drives one) the row
+# read '2026-01-13 (96 h)' - 96 "h" for a 24-hour day. That is the points/hours
+# conflation H1 was raised for, on the row H1's own ticket shipped.
+#
+# MUTATION.md's rule: this check asserts a DIFFERENCE, so the fixture is run
+# with the two numbers actually made different. 96 != 24, asserted first.
+_h_q = store()
+_QUARTERS = [u + timedelta(minutes=15 * q) for u in YDAY_H for q in range(4)]
+_ph.PriceHistory._remember(
+    _h_q, PriceRead.series(
+        [PricePoint(start=brussels(u),
+                    eur_per_kwh=allin(YESTERDAY_MWH[brussels(u).hour]
+                                      + brussels(u).minute),
+                    provisional=False) for u in _QUARTERS],
+        PriceProduct.DAY_AHEAD, "generic_dayahead", NOW),
+    NOW)
+_qw = words_of(_h_q)
+check(f"the quarter-hourly fixture really is sub-hourly and its two counts "
+      f"really differ ({len(_QUARTERS)} points, "
+      f"{len({brussels(u).hour for u in _QUARTERS})} local hours), or 'points' "
+      f"and 'hours' are the same observation and the row grades nothing",
+      len(_QUARTERS) == 96
+      and len({brussels(u).hour for u in _QUARTERS}) == 24
+      and len({u.minute for u in _QUARTERS}) == 4)
+check(f"...and the row reports BOTH, each under its own unit: {_qw!r}. "
+      f"A day of 96 quarter-hourly prints is 96 points over 24 h, and the "
+      f"figure that says whether tomorrow's fill has a whole day to carry is "
+      f"the HOUR one",
+      _qw == f"{YDAY_D} (96 points over 24 h)")
+check(f"...and specifically the row does NOT say '96 h' {_qw!r} - a point "
+      f"count wearing the hour's name is H1's defect, and this row is where "
+      f"an operator would have read it",
+      isinstance(_qw, str) and "(96 h)" not in _qw and " (96 points " in _qw)
 check(f"a file that only PARTLY loaded says so in the row {_w14!r}, or the "
       f"days it dropped read as days that were simply never written",
       isinstance(_w14, str) and (_h14.last_error or "") in _w14
@@ -1023,7 +1129,7 @@ check("...and `words` never raises, whatever the file holds. It is read once "
       "from Controller.__init__ and again on every status render, and neither "
       "call site guards it - an exception here is an add-on that does not start",
       all(isinstance(words_of(h), str)
-          for h in (_missing, _h6, _h7, _h14, _h_words)))
+          for h in (_missing, _h6, _h6b, _h7, _h14, _h_words)))
 
 # ⚠️ AND A FAILED WRITE LEAVES NOTHING BEHIND. `_save()` writes to a `mkstemp`
 # and renames - atomic, so a power cut cannot leave a truncated file that
@@ -1067,6 +1173,7 @@ check(f"...and a SUCCESSFUL write leaves exactly the one file "
 # provider series with no invented points and no §11.4 degradation.
 for _label, _hist in (("no durable volume", _missing),
                       ("unreadable file", _h6),
+                      ("a non-UTF-8 byte in the file", _h6b),
                       ("wrong schema", _h7)):
     _plan = run(resolve_tariff(CFG, NOW, _until,
                                transport=transport_for(_body_today), now=NOW,
@@ -1080,6 +1187,198 @@ for _label, _hist in (("no durable volume", _missing),
           and not any(p.provisional for p in _plan.read.points))
 
 
+# --------------------------------------------------------------------------- #
+print()
+print("5c. FCST-05 - the WRITE path is as defensive as the read path")
+
+# THE DEFECT. `_remember()` merged a day's held points with
+#
+#     held = {q["start"]: q for q in raw
+#             if isinstance(q, dict) and "start" in q}
+#     rec["points"] = sorted(held.values(), key=lambda q: q["start"])
+#
+# which trusts a value read off /data to be a HASHABLE, SORTABLE STRING. It is
+# whatever is in the file. `_usable_days()` required `rec["points"]` to be a
+# LIST and never looked at the elements, and `_parse_points()` - the READ path -
+# was properly defensive, so `yesterday()` survived and the §5 `_p8` fixture
+# passed. Two standards for what a stored point is; the weaker one on the path
+# that WRITES.
+#
+# ⚠️ AND THE PLACEMENT IS THE WHOLE FIXTURE. `_p8` files its malformed rows
+# under a day the feed is NOT writing to, so `_remember()` never touches that
+# record and the defect is invisible. What reaches it is a malformed row under
+# TODAY - the record the running feed merges into on every refresh - so that is
+# asserted before anything is graded (MUTATION.md habit 1), and the harmless
+# placement is run as the accepted baseline it varies from (habit 7).
+#
+# ⚠️ AND THE BLAST RADIUS IS WHY IT IS NOT MERELY UNTIDY. `carry()` is reached
+# from `resolve_tariff()` from `run_tariff_refresh`, which catches `Exception`:
+# the add-on stays UP and EVERY refresh raises, forever. The plan retires after
+# TARIFF_MAX_AGE_S and arbitrage and the cheap window are off permanently, with
+# no self-heal, because `_remember()` raises before it reaches `_save()`.
+
+_BAD_STARTS = (
+    ("an int", 7),                     # TypeError: '<' between 'str' and 'int'
+    ("a dict", {"a": 1}),              # TypeError: unhashable as a dict key
+    ("null", None),                    # TypeError: '<' between 'str', 'NoneType'
+    ("a list", [1, 2]),                # TypeError: unhashable as a dict key
+    ("a string that is not an instant", "nope"),   # sorts fine, never a point
+)
+
+
+def _seeded(day, bad_start):
+    """A store on disk holding ONE malformed row under `day`, plus a good one."""
+    d = tmpdir()
+    path = os.path.join(d, "price_history.json")
+    good = {"start": brussels(at_local(day, 5)).isoformat(),
+            "eur_per_kwh": allin(YESTERDAY_MWH[5]), "provisional": False}
+    json.dump({"version": SCHEMA, "days": {day: {
+        "product": "dynamic_dayahead", "provider": "generic_dayahead",
+        "points": [good, {"start": bad_start, "eur_per_kwh": 0.2,
+                          "provisional": False}]}}},
+              open(path, "w", encoding="utf-8"))
+    return path, PriceHistory(path)
+
+
+_TODAY_FEED = series_of(TODAY_H, TODAY_MWH)
+check(f"the fixture files its malformed row under {TODAY_D}, which is exactly "
+      f"the Brussels day the feed being carried writes to "
+      f"{sorted({_ph._bdate(p.start) for p in _TODAY_FEED.points})} - under any "
+      f"other day `_remember()` never opens the record and this whole section "
+      f"grades nothing, which is why the suite already had this hole",
+      {_ph._bdate(p.start) for p in _TODAY_FEED.points} == {TODAY_D})
+
+# The accepted baseline: the SAME row, one day over. Shipped and broken alike
+# pass this, and that is the point of running it (habit 7).
+_pB, _hB = _seeded(YDAY_D, 7)
+_cB = derived(lambda: _hB.carry(_TODAY_FEED, since=NOW, until=_until, stamp=NOW))
+check("baseline: the same malformed row under a day the feed is NOT writing "
+      "to is carried without incident - one variable moves between this and "
+      "every check below, and it is the DAY",
+      not isinstance(_cB, _Raised) and _cB.kind == "series")
+
+for _shape, _bad in _BAD_STARTS:
+    _pM, _hM = _seeded(TODAY_D, _bad)
+    _wload = words_of(_hM)
+    _c1 = derived(lambda h=_hM: h.carry(_TODAY_FEED, since=NOW, until=_until,
+                                        stamp=NOW))
+    _saw = repr(_c1) if isinstance(_c1, _Raised) else _c1.kind
+    check(f"a `start` that is {_shape} ({_bad!r}) under TODAY's own record "
+          f"does not take the refresh down: `carry()` returns a series "
+          f"({_saw}). It raised "
+          f"`TypeError` out of the shipped path, and `run_tariff_refresh` "
+          f"catches `Exception`, so the add-on stayed up while every refresh "
+          f"failed for good",
+          not isinstance(_c1, _Raised) and _c1.kind == "series")
+    # ⚠️ THE VERDICT IS NOT THE CLAIM. "It did not raise" is satisfied by a
+    # store that silently kept the poison row; what this section promises is
+    # that the row is GONE FROM THE FILE, so the next start is clean.
+    _disk = json.load(open(_pM, encoding="utf-8"))
+    _rows = _disk["days"][TODAY_D]["points"]
+    # DERIVED, not a constant: the surviving rows are exactly the DISTINCT
+    # instants the feed published for that day plus the fixture's one good
+    # seeded row - which is 05:00, an hour the feed also prints, so it merges.
+    _want = len({p.start.astimezone(UTC) for p in _TODAY_FEED.points}
+                | {at_local(TODAY_D, 5).astimezone(UTC)})
+    check(f"...and the row is DROPPED FROM /data, not carried: {len(_rows)} "
+          f"row(s) on disk against {_want} distinct instants, every one of "
+          f"them a point. `_remember()` used to raise BEFORE `_save()`, so "
+          f"there was no self-heal at all - the store came back poisoned on "
+          f"every restart",
+          all(_ph._point_of(r) is not None for r in _rows)
+          and len(_rows) == _want)
+    # ...and the store stops reporting health, at BOTH moments it can be read:
+    # when it is opened, and after the refresh that cleaned it.
+    # ⚠️ THE COUNTS ARE ASSERTED TOO, NOT ONLY THE NOTE. The seeded record holds
+    # ONE point and one row that is not one, so a row printing "2 points" would
+    # be reporting the poison as held prices - which is the half of this defect
+    # that reached the operator: `words` said "2026-01-15 (1 h)" while every
+    # tariff refresh was dying on that record.
+    check(f"...and the status row does not call that store healthy - at open "
+          f"{_wload!r}: ONE point held, and the row that is not a point named "
+          f"rather than counted",
+          isinstance(_wload, str)
+          and _wload.startswith(f"{TODAY_D} (1 point over 1 h) [")
+          and "1 malformed point row(s) in" in _wload)
+    # ⚠️ AND THE TWO NOTES ARE GRADED APART, BECAUSE THEY MASK EACH OTHER
+    # OTHERWISE (MUTATION.md TEST-16). `_usable_days()` records what it FOUND
+    # at open and `_remember()` records what it DROPPED on the refresh; both
+    # land in `last_error`, so a check asking only "does the row mention a
+    # malformed point" is answered by whichever rung is still standing - the
+    # first draft of this check was exactly that, and deleting `_remember()`'s
+    # note left the whole 29-file suite green at -1 B. The tense is the
+    # difference an operator reads: "in" is still there, "dropped from" is
+    # gone, and after the refresh only the second is true.
+    _wafter = words_of(_hM)
+    check(f"...nor after the refresh that dropped it {_wafter!r}. It said "
+          f"'{TODAY_D} (1 h)' - the status row reporting a full cache while "
+          f"every tariff refresh was dying on it",
+          isinstance(_wafter, str)
+          and "malformed point row(s) dropped from" in _wafter
+          and TODAY_D in _wafter)
+    check(f"...and the row MOVED from what was found to what was dropped "
+          f"{_wload.split('[')[-1]!r} -> {_wafter.split('[')[-1]!r}, so the "
+          f"two notes are not one note read twice",
+          isinstance(_wload, str) and isinstance(_wafter, str)
+          and "malformed point row(s) in" in _wload
+          and "malformed point row(s) in" not in _wafter
+          and "malformed point row(s) dropped from" not in _wload)
+    # ...and a SECOND refresh over the same store is clean, which is the state
+    # a restarted add-on comes up in.
+    _hM2 = PriceHistory(_pM)
+    check(f"...and the store reopened from that file is CLEAN: the whole day "
+          f"held and nothing to report {words_of(_hM2)!r}, so the note clears "
+          f"itself rather than becoming permanent furniture",
+          words_of(_hM2) == f"{TODAY_D} ({_want} points over {_want} h)"
+          and _hM2.last_error is None)
+
+# ...and the shipped path, end to end, through the caller that catches
+# `Exception` and would otherwise have hidden this for good.
+_pP, _hP = _seeded(TODAY_D, 7)
+_o1 = derived(lambda: run(resolve_tariff(
+    CFG, NOW, _until, transport=transport_for(_body_today), now=NOW,
+    history=_hP)).origin)
+_o2 = derived(lambda: run(resolve_tariff(
+    CFG, NOW, _until, transport=transport_for(_body_today), now=NOW,
+    history=_hP)).origin)
+check(f"the SHIPPED `resolve_tariff()` path survives it TWICE ({_o1!r}, "
+      f"{_o2!r}) - once for the refresh that finds the poison and once for "
+      f"the one after it. A guard that refuses once and dies on the next call "
+      f"is a delay, not a guard (MUTATION.md habit 9)",
+      _o1 == "provider" and _o2 == "provider")
+
+# ⚠️ AND THE MERGE KEY IS THE INSTANT, WHICH IS WHAT RULE 4 SAYS. The old key
+# was `q["start"]` - the SPELLING - so the same instant written at two offsets
+# was held twice, and the lexicographic sort beside it put the second 02:00 of
+# the 25-hour October day (`+01:00`) BEFORE the first (`+02:00`). Aware
+# datetimes hash and compare by instant, so the validated `start` is the key.
+_d15 = tmpdir()
+_p15 = os.path.join(_d15, "price_history.json")
+_alt = at_local(YDAY_D, 6)
+json.dump({"version": SCHEMA, "days": {YDAY_D: {
+    "product": "dynamic_dayahead", "provider": "generic_dayahead",
+    "points": [{"start": _alt.astimezone(UTC).isoformat(),
+                "eur_per_kwh": 0.999, "provisional": False}]}}},
+          open(_p15, "w", encoding="utf-8"))
+_h15 = PriceHistory(_p15)
+check(f"the two spellings really are one instant and really are two strings "
+      f"({_alt.astimezone(UTC).isoformat()!r} vs "
+      f"{brussels(_alt).isoformat()!r}), or 'merged by instant' and 'merged by "
+      f"spelling' are the same observation",
+      _alt.astimezone(UTC).isoformat() != brussels(_alt).isoformat()
+      and _alt.astimezone(UTC) == brussels(_alt))
+_ph.PriceHistory._remember(
+    _h15, series_of([_alt], YESTERDAY_MWH), NOW)
+_y15 = _h15.yesterday(NOW)
+_n15 = len(_y15.points) if _y15.kind == "series" else _y15.kind
+_v15 = _y15.points[0].eur_per_kwh if _y15.kind == "series" else None
+check(f"...and a revision of that hour REPLACES it rather than sitting beside "
+      f"it: {_n15} point held, priced {_v15} and not 0.999. Rule 4 says "
+      f"'MERGE BY INSTANT'; the key was the string",
+      _y15.kind == "series" and len(_y15.points) == 1
+      and abs(_y15.points[0].eur_per_kwh - allin(YESTERDAY_MWH[6])) < 1e-9)
+
+
 # --------------------------------------------------------------------------- #
 print()
 print("6. rule 4 - hours accumulate within a day; they do not replace it")