goodwe-addon/goodwe_controller/app/price_history.py

"""FCST-04 - the day the carry-forward carries, kept somewhere a restart cannot reach.

WHAT WAS MISSING, AND WHY IT WAS INVISIBLE. FCST-03 shipped
`price_forecast.fill_from_yesterday()` complete and checked, and shipped the
consumers that were written to expect it - `TariffPlan.firm_points()`,
`cheap_now()`'s refusal of a provisional hour, and `arbitrage()`'s refusal 3,
which drops provisional hours from the trade set instead of refusing the whole
series. Every one of those was live. The FILL ITSELF HAD NO CALLER: nothing in
`app/` referenced `fill_from_yesterday`, so no series ever contained a carried
point and every one of those consumers was exercising its all-firm branch.

The defect that produced is not an error anywhere. `load_prices()` returns
whatever the feed published and nothing widens it to the requested horizon, so
on a dynamic site between midnight and the 14-16 h CET day-ahead publication
the series covers the rest of TODAY and nothing more - typically 10-24 of the
48 hours `price_horizon_h` asked for. `TariffPlan.arbitrage()` then computes
`max - min` over that truncated window and reports a spread. The number is
real, the window is half the one the operator configured, and NOTHING SAYS SO:
tomorrow evening's peak - the hour a morning charge would be sold into - is
simply not in the series, so the spread it would have created cannot clear a
threshold it was never compared against. A missing hour is not a wrong price;
it is an hour the optimiser never learns exists.

So this module is the half FCST-03 left out, and it is deliberately the SMALL
half: the fill's arithmetic, its DST rule and its provenance audit all stay in
`price_forecast.py`, untouched. What lives here is the one thing that
arithmetic needs and could not have: A YESTERDAY THAT SURVIVES A RESTART.


WHY PERSISTENCE, AND NOT A VARIABLE ON THE REFRESH TASK
======================================================
An in-memory "the last full day I saw" costs nothing and is wrong exactly when
it matters. The window the fill exists for is midnight -> publication; an
add-on restarted at 08:00 - a config change, an HA update, a power cut - would
come up with no yesterday at all and spend that whole window on the truncated
series, which is the pre-FCST-03 behaviour arriving silently on the mornings
after an operator touched something. The fill's own value is a function of
surviving a restart, so the day it carries has to.

⚠️ AND NOT IN `/data/state.json`. That file holds the maintenance history and
SAFETY-02's sign latch, and `Store.set()` rewrites the whole of it on every
write. The tariff refresh runs every `TARIFF_REFRESH_S` (1800 s), so folding a
price cache into it would put 48 extra whole-file rewrites a day through the
file whose corruption reads as "no maintenance ever ran" and "never tripped".
The cache is worth having; it is not worth putting anywhere near that. Separate
file, separate blast radius.

⚠️ AND THE DIRECTORY IS NOT CREATED, which is `tariff.PeakStore`'s rule and its
reason applies unchanged: `/data` is mounted by the Supervisor and always
exists on a real install, so `makedirs` here could only ever succeed where the
durable volume is MISSING - manufacturing a cache on the container's ephemeral
layer that writes fine, reads fine, and is thrown away on the next restart.
That is a yesterday that silently forgets, which is worse than one that says it
cannot answer.

⚠️ AND THAT RULE IS CARRIED OUT IN `_remember()`, NOT ONLY IN `_save()`. FCST-04
first shipped it half-done: `_save()` short-circuited on `not self.ok` while
`_remember()` filled `self._days` anyway, so a store with no durable volume
answered `words` = "unavailable" and `yesterday()` = A SERIES - an in-memory
yesterday that works all day and is gone on the next restart, which is exactly
the silently-forgetting cache this rule exists to refuse. A store that cannot
persist now records nothing, and both of its answers say the same thing.


THE FOUR RULES THIS FILE IS SHAPED BY
=====================================

1. IT RECORDS WHAT THE PROVIDER PUBLISHED, NEVER WHAT THE FILL PRODUCED.
   Feeding a filled series back in is the defect that makes the whole mechanism
   rot: day 2 carries day 1's guesses, day 3 carries day 2's carry of them, and
   after a week of a dead feed the "yesterday" being carried forward is a
   fossil of the last real publication with nothing anywhere reporting its age.
   Every point still carries `provisional=True`, so the flag never lies - what
   decays is the NUMBER, silently, behind a flag that is telling the truth.

   ⚠️ NO VALUE-LEVEL CHECK CAN CATCH THAT, and saying so is more useful than a
   guarantee that would not hold. A carried point and a published-provisional
   point (`eneco_eddy`'s current quarter, §9.2) are the same three fields with
   the same flag; the difference is provenance, and provenance is not in the
   value. So the mechanism is ORDERING, not validation: `carry()` is the only
   public entry point, it remembers the argument and returns the fill, and the
   filled series exists only as a return value that `carry()` itself never
   sees again. `_remember()` is private, is called from exactly one place, and
   `test_price_history.py` PARSES every shipped module under `app/`, `tools/`,
   `scenarios/` and `estop/` and fails if any of them uses the NAME - an
   attribute, a bare name, a def, or the string constant a `getattr` needs. By
   name rather than by substring, so a comment about the rule does not fail it
   and a call from `tools/` cannot pass unseen.

   ⚠️ AND THE ORDER IS GRADED BY DRIVING IT, NOT BY READING THE SOURCE. The
   first version of that check compared positions in `ast.walk`, which is
   breadth-first: `self._remember(...)` is a statement and `self.yesterday(...)`
   is an ARGUMENT of the fill, one level deeper, so `_remember` came out first
   whichever statement was written first - shipped and reversed scored the
   same. What grades it now is the behaviour: a feed carrying yesterday
   alongside today, into an EMPTY store, must fill the whole horizon, and
   reversed it fills nothing.

2. "YESTERDAY" MEANS YESTERDAY, NOT "THE NEWEST DAY I HAVE".
   `newest()` is the natural accessor and it is the one that decays: an add-on
   down from Friday to Monday would answer Friday's prices to a fill that flags
   them as one day old. The lookup is BY DATE - the Brussels day before the
   Brussels day of the instant asked about - and a record for any other date is
   NO_RECORD with a reason naming both dates. A three-day-old profile is not a
   worse yesterday; it is not a yesterday.

3. A STORE THAT CANNOT ANSWER SAYS SO, IN THE TYPE.
   `yesterday()` returns a `PriceRead`: a `series` when a full record exists,
   `no_data(reason)` for every other outcome - no file, no record for that
   date, an unreadable file, a record that parses to nothing. There is no
   empty-series answer and no `.get(..., [])`-shaped accessor, for
   `price.py`'s reason: an empty series is how "the cache is broken" gets to
   wear the shape of "yesterday was free". `fill_from_yesterday()` takes a
   `PriceRead` and handles `no_data` on its own, so the degraded case needs no
   branch at the call site - it fills nothing and the horizon stays short,
   which is exactly the behaviour that shipped before this ticket.

4. HOURS ACCUMULATE WITHIN A DAY; THEY DO NOT REPLACE IT.
   A feed that serves only the REMAINING hours of the day (a legitimate shape -
   nothing in the plugin contract promises a whole day) would, under
   replace-the-record, overwrite a complete 24-hour profile with the two points
   left at 22:00 and hand the next morning a yesterday with two hours in it. So
   a record is MERGED by instant: a later read updates the hours it carries and
   leaves the rest, and a revised price for an hour already held wins because
   it is the newer statement about the same hour.

⚠️ KEYED BY INSTANT, NOT BY LOCAL HOUR, AND THAT IS THE DST DECISION. Storing
`hour -> price` would collapse the two 02:00s of a 25-hour October day into
one, inventing a second and different collapsing rule beside the one
`fill_from_yesterday()` already documents ("the first of its two 02:00 points
is the one used, deterministically"). Storing the instants keeps both, in
order, and lets the fill's own rule be the only one in the system.

WHAT THIS DOES NOT DO, ON PURPOSE
=================================
It does not extend the fill to a feed that published NOTHING. `resolve_tariff()`
applies `carry()` only on a `series` read; a `no_data` read still falls to
`_degrade()` and §11.4's row - "No price data -> fall back to the static
profile, mark the plan provisional, no arbitrage" - which is the fallback the
operator configured `static_profile` FOR. Preferring yesterday's feed over it
would change a documented degradation without being asked, and it would buy
nothing measurable: a fully-carried series is 100 % provisional, so
`arbitrage()` finds fewer than two firm hours and refuses, and `cheap_now()` is
False, exactly as on the §11.4 fallback. The two differ only in which numbers
the status page shows. `fill_from_yesterday()` handles a `no_data` read anyway,
and that branch is therefore not reached from production - it stays checked,
and turning it on is a product decision, not a wiring one. See
DECISIONS/FCST-04-the-day-that-is-carried.md.
"""

import json
import logging
import os
import tempfile
from datetime import datetime, timedelta, timezone

from .price import PricePoint, PriceProduct, PriceRead, brussels
from .price_forecast import fill_from_yesterday

_LOG = logging.getLogger("goodwe.price.history")

# How many Brussels days of published prices the SORT-ORDER cap keeps. The fill
# needs exactly one - yesterday - and rule 2 refuses anything older, so this is
# not a horizon: it is the number of days a record may sit unused before
# pruning removes it, and it is small because a bigger cache would only hold
# days no lookup can ask for. Three covers today, yesterday, and the day a
# fetch spanning midnight writes ahead of itself.
#
# ⚠️ IT IS A CAP, NOT THE RETENTION RULE, AND THREE HAS NO SLACK. The steady
# state is already {yesterday, today, tomorrow}; a feed that publishes to D+2,
# or ONE stray point in the wrong year - which sorts LAST and is therefore
# immortal under a newest-N cap - pushes yesterday out of a three-slot store
# and keeps it out, permanently and silently. So `_prune()` PINS the two days a
# lookup can actually ask for on top of the cap; see its docstring. The store
# is then bounded at KEEP_DAYS + 2 records rather than at KEEP_DAYS.
KEEP_DAYS = 3

SCHEMA = 1


def _bdate(instant: datetime) -> str:
    """The Brussels calendar date of an instant, as `YYYY-MM-DD`.

    ⚠️ BRUSSELS, NOT UTC. 23:30 UTC in July is already tomorrow in Belgium, and
    a cache keyed on the UTC date would file the last 90-120 minutes of every
    day under the wrong one - so a lookup on the following morning would find a
    "yesterday" missing its own evening hours, which are the expensive ones.
    """
    return brussels(instant).strftime("%Y-%m-%d")


def _yesterday_of(instant: datetime) -> str:
    """The Brussels date of the day BEFORE `instant`'s Brussels day.

    ⚠️ THE ARITHMETIC IS ON THE LOCAL DATE, NOT ON THE INSTANT, AND THAT IS A
    DST DEFECT THIS FILE SHIPPED. `brussels()` renders at the FIXED offset in
    force at that instant, so `brussels(stamp) - timedelta(days=1)` subtracts 24
    ABSOLUTE hours from a wall clock - and the Brussels day before a
    spring-forward is 23 hours long. Measured at 2026-03-30T00:30+02:00, the
    first hour of the morning after the March transition: it returned
    2026-03-28, so the fill carried a TWO-DAY-OLD profile flagged as yesterday's
    (or, with only the last three days held, answered `no_data` naming a date
    that is not yesterday).

    A calendar day minus one day is `date` arithmetic, and a `date` has no
    offset to lose. The 25-hour October day is right for the same reason.
    """
    return (brussels(instant).date() - timedelta(days=1)).strftime("%Y-%m-%d")


def _n_points(rec) -> int:
    """How many points a day record holds. 0 for anything that is not one.

    ⚠️ TOLERANT, AND IT IS THE SECOND LINE OF DEFENCE, NOT THE FIRST. `_load()`
    drops a malformed day record and says so, so this should never see one. It
    is written this way because `words` is read from `Controller.__init__` and
    from every status render: a status LINE that raises takes the add-on down,
    and "the cache is broken" must never be able to mean "the controller does
    not start". That is exactly what FCST-04 shipped - `{"days": {"2026-01-14":
    "oops"}}` gave `ok=True`, `last_error=None`, and an `AttributeError` out of
    `words`, `yesterday()` and `resolve_tariff()` alike, with no self-heal
    because `_remember()` raised before it could rewrite the file.
    """
    if not isinstance(rec, dict):
        return 0
    pts = rec.get("points")
    return len(pts) if isinstance(pts, (list, tuple)) else 0


class PriceHistory:
    """The published price series of the last few Brussels days, in /data.

    ⚠️ `ok` IS ABOUT THE FILE, NOT ABOUT THE DATA. A healthy store with no
    record for yesterday is `ok` and answers `no_data` - a normal state on a
    site's first day, and a different thing from a store that could not be
    opened. Both are visible: `ok`/`last_error` describe the file, and
    `yesterday()`'s reason describes the answer.
    """

    def __init__(self, path: str = "/data/price_history.json"):
        self.path = path
        self.ok = False
        self.last_error: str | None = None
        self._days: dict = {}
        self._open()

    # -- the file ------------------------------------------------------------
    def _open(self) -> None:
        parent = os.path.dirname(self.path)
        # See the module note: NOT created. A cache on the ephemeral layer is
        # one that forgets without saying so.
        if parent and not os.path.isdir(parent):
            self.last_error = f"no durable volume at {parent}"
            # INFO, not ERROR: on a dev host or in a test container there is no
            # /data and nothing is wrong.
            _LOG.info("no price history: %s - §8.3 carry-forward has nothing "
                      "to carry and the horizon stays as short as the feed",
                      self.last_error)
            return
        self.ok = True
        self._load()

    def _load(self) -> None:
        try:
            with open(self.path, "r", encoding="utf-8") as fh:
                blob = json.load(fh)
        except FileNotFoundError:
            _LOG.info("no price history yet at %s - the first refresh writes "
                      "one, and the fill has nothing to carry until the day "
                      "after that", self.path)
            return
        except (json.JSONDecodeError, OSError) as err:
            # A corrupt cache must not stop the controller, and must not be
            # silently re-created either: the record is discarded, the reason
            # is kept, and `yesterday()` reports it rather than answering
            # "no record", which would read as a normal first day.
            self.last_error = f"unreadable ({err})"
            _LOG.warning("price history at %s is %s - starting from empty. "
                         "§8.3 carry-forward has nothing to carry until the "
                         "next refresh has written a full day",
                         self.path, self.last_error)
            return
        found = blob.get("version") if isinstance(blob, dict) else None
        # ⚠️ THE TYPE IS PART OF THE VERSION. `1.0 == 1` in Python, so a float
        # written by some other tool would pass a bare `!=` and be read as
        # schema 1 - a version check that cannot tell 1 from 1.0 is one that
        # would not notice a writer this build has never met.
        if not isinstance(blob, dict) or isinstance(found, bool) \
                or not isinstance(found, int) or found != SCHEMA:
            self.last_error = f"schema {found!r}, not {SCHEMA}"
            _LOG.warning("price history at %s has %s - discarding it rather "
                         "than guessing what its points mean",
                         self.path, self.last_error)
            return
        days = blob.get("days")
        if not isinstance(days, dict):
            self.last_error = "no `days` object"
            _LOG.warning("price history at %s has %s", self.path, self.last_error)
            return
        self._days = self._usable_days(days)

    def _usable_days(self, days: dict) -> dict:
        """The day records that are shaped like day records. The rest are dropped.

        ⚠️ THE ASYMMETRY THAT PRODUCED THE DEFECT: `_parse_points()` below is
        carefully defensive about every field of a POINT, and `_load()` was not
        defensive about the record holding them at all. `{"version": 1, "days":
        {"2026-01-14": "oops"}}` therefore loaded as `ok=True`,
        `last_error=None` - and then `rec.get(...)` raised `AttributeError` out
        of `words`, `yesterday()` and `resolve_tariff()`. `PriceHistory(...)`
        and `.words` are both unguarded in `Controller.__init__`, so THE ADD-ON
        DID NOT START; and it never self-healed, because `_remember()` raised on
        the same record before it could reach `_save()`.

        A broken cache is a short horizon. It is not a controller that will not
        regulate a battery, and §6's rule - absent machinery must not disable
        the loop - does not stop applying because the machinery is a JSON file.

        Dropped, not repaired: an unusable record has nothing in it to keep, and
        the next `_remember()` writes a fresh one under that date.
        """
        kept, dropped = {}, []
        for key, rec in days.items():
            if not isinstance(key, str) or not isinstance(rec, dict) \
                    or not isinstance(rec.get("points"), list):
                dropped.append(str(key))
                continue
            kept[key] = rec
        if dropped:
            # Recorded, not silent: `yesterday()` and `words` both report
            # `last_error`, and a cache that quietly discards half a file reads
            # to an operator exactly like a normal first day.
            self.last_error = (f"{len(dropped)} unusable day record(s) dropped "
                               f"({', '.join(sorted(dropped))})")
            _LOG.warning("price history at %s: %s - the controller starts "
                         "anyway and the horizon is as short as the feed makes "
                         "it", self.path, self.last_error)
        return kept

    def _save(self) -> None:
        if not self.ok:
            return
        # ⚠️ THE TEMP FILE IS REMOVED ON EVERY FAILING PATH. A write that dies
        # between `mkstemp` and `os.replace` used to leave its temp behind, and
        # the refresh task runs 48 times a day: a persistent write failure -
        # a full volume being the likeliest cause - leaked 48 inodes a day onto
        # the volume that was already the problem.
        tmp = None
        try:
            fd, tmp = tempfile.mkstemp(dir=os.path.dirname(self.path))
            with os.fdopen(fd, "w", encoding="utf-8") as fh:
                json.dump({"version": SCHEMA, "days": self._days}, fh, indent=1)
            # Atomic: a power cut mid-write must not leave a truncated file
            # that parses as an empty day.
            os.replace(tmp, self.path)
            tmp = None                      # it IS the file now; do not unlink
        except OSError as err:
            self.last_error = f"could not write ({err})"
            _LOG.error("could not persist price history: %s", err)
        finally:
            if tmp is not None:
                try:
                    os.unlink(tmp)
                except OSError:             # nothing further to try, and a
                    pass                    # failed cleanup is not a fault

    # -- the one public entry point -----------------------------------------
    def carry(self, read: PriceRead, *, since: datetime, until: datetime,
              stamp: datetime) -> PriceRead:
        """Remember what this read published, then §8.3-fill the missing hours.

        THE ORDER IS DELIBERATE AND IS THE POINT OF THE METHOD. The read is
        recorded FIRST, so a feed that hands back yesterday alongside today
        refreshes yesterday's record from live data before it is consumed as
        the profile; and the filled series - the one thing that must never be
        recorded - exists only as this method's return value, which this object
        never sees again. See rule 1, including how that order is graded.

        Returns whatever `fill_from_yesterday()` returns: a series over the
        whole hours of [since, until), carrying EVERY point the provider
        published inside it - by identity, and four of them in one hour on a
        quarter-hourly feed - plus one carried point for each remaining whole
        hour FOR WHICH YESTERDAY HELD THAT WALL-CLOCK HOUR. An hour neither
        source can price is left out (§11.4 for that hour), and `no_data` comes
        back when that is every hour.
        """
        self._remember(read, stamp)
        return fill_from_yesterday(read, yesterday=self.yesterday(stamp),
                                   since=since, until=until, stamp=stamp)

    # -- the halves, private because rule 1 is an ordering ------------------
    def _remember(self, read: PriceRead, stamp: datetime) -> None:
        """File a provider read's points under their own Brussels days.

        ⚠️ PRIVATE, AND CALLED FROM EXACTLY ONE PLACE. A caller that can reach
        this can record a filled series, which is the decay in rule 1. There is
        no value-level defence against that - a carried point and a published
        provisional point are the same three fields - so the defence is that
        nothing outside this file names it, and `test_price_history.py` checks
        the tree for that rather than trusting the underscore.
        """
        if not self.ok:
            # ⚠️ A STORE THAT CANNOT PERSIST RECORDS NOTHING. See the module
            # docstring's third warning: filling `self._days` here would give a
            # site with no /data an in-memory yesterday that answers a series
            # all day and is gone on the next restart - a cache that forgets
            # without saying so, which is the one thing that rule refuses. The
            # honest degraded state is `words` = "unavailable" and `yesterday()`
            # = `no_data`, and those two must agree.
            return
        points = read.match(series=lambda r: r.points, no_data=lambda _r: ())
        if not points:
            return
        product = read.match(series=lambda r: r.product.value,
                             no_data=lambda _r: None)
        provider = read.match(series=lambda r: r.provider,
                              no_data=lambda _r: None)

        touched = set()
        for p in points:
            day = _bdate(p.start)
            rec = self._days.setdefault(
                day, {"product": product, "provider": provider, "points": []})
            # Rule 4: MERGE by instant. A later read updates the hours it
            # carries and leaves the rest; a revised price for an hour already
            # held wins, because it is the newer statement about that hour.
            raw = rec.get("points")
            held = {q["start"]: q for q in (raw if isinstance(raw, list) else ())
                    if isinstance(q, dict) and "start" in q}
            held[p.start.isoformat()] = p.canonical()
            rec["points"] = sorted(held.values(), key=lambda q: q["start"])
            rec["product"], rec["provider"] = product, provider
            touched.add(day)

        self._prune(stamp)
        self._save()
        _LOG.debug("price history: %d point(s) filed under %s; %d day(s) held",
                   len(points), ", ".join(sorted(touched)), len(self._days))

    def _prune(self, stamp: datetime) -> None:
        """Drop records no lookup can reach. A sort-order cap, plus two pins.

        ⚠️ NOTHING IS DELETED BY ARITHMETIC AGAINST THE CLOCK. A clock that
        jumped - the failure mode WITNESS-03 spent a session on - would
        otherwise delete every record in one pass and leave the fill with
        nothing on the morning after, which is the one morning it is for.
        Deleting on a bad clock is unrecoverable; holding a few days too many
        is not, so the cap is `sorted(...)[-KEEP_DAYS:]` and stays that way.

        ⚠️ AND A CAP ALONE IS DEFEATED BY ONE POINT IN THE WRONG YEAR. Sorting
        is lexicographic on the date string, so a stray far-future record sorts
        LAST and is immortal - it consumes a slot for good, and with the steady
        state already {yesterday, today, tomorrow} filling all three, yesterday
        is what falls out. A feed that publishes to D+2 does the same thing
        without any stray at all. So the two days `yesterday()` can ACTUALLY be
        asked for are pinned on top of the cap.

        `stamp` is READ here, which it was not before: the pin is the only use
        of it, and it is additive. Pinning can only RETAIN a record, never
        delete one, so a jumped `stamp` pins two dates that are not held and
        changes nothing - the WITNESS-03 property is preserved by construction
        rather than by remembering to check it.
        """
        if len(self._days) <= KEEP_DAYS:
            return
        keep = set(sorted(self._days)[-KEEP_DAYS:])
        keep |= {_bdate(stamp), _yesterday_of(stamp)}
        for day in [d for d in self._days if d not in keep]:
            del self._days[day]

    def yesterday(self, stamp: datetime) -> PriceRead:
        """The published series of the Brussels day BEFORE `stamp`'s, or no_data.

        ⚠️ THE DATE IS COMPUTED, NOT LOOKED UP. See rule 2: `newest()` would
        answer Friday's prices to a Monday fill, and the fill would flag them as
        one day old because that is the only thing its flag can say.
        """
        if stamp.tzinfo is None or stamp.utcoffset() is None:
            raise ValueError(
                f"yesterday({stamp!r}): a naive datetime is not an instant, "
                f"and 'which Brussels day was yesterday' is exactly the "
                f"question a wall-clock reading cannot answer.")
        want = _yesterday_of(stamp)
        if self.last_error and not self._days:
            return PriceRead.no_data(
                f"the price history at {self.path} is {self.last_error}, so "
                f"§8.3 has no profile to carry forward for {want}; the plan "
                f"covers only the hours the feed published")
        rec = self._days.get(want)
        if rec is None:
            have = ", ".join(sorted(self._days)) or "no days at all"
            return PriceRead.no_data(
                f"no published prices recorded for {want} (held: {have}), so "
                f"§8.3 has no profile to carry forward. A record for another "
                f"day is not a yesterday")
        points, bad = _parse_points(rec.get("points"))
        if not points:
            why = bad or "it is empty"
            return PriceRead.no_data(
                f"the {want} record holds no usable point ({why}), and an "
                f"empty series is not a free day")
        try:
            product = PriceProduct(rec.get("product"))
        except ValueError:
            return PriceRead.no_data(
                f"the {want} record names product {rec.get('product')!r}, "
                f"which is not a PriceProduct. The optimiser switches on that "
                f"enum, so a record that cannot name one cannot be a series")
        if bad:
            _LOG.warning("price history: %s in the %s record were dropped; "
                         "%d point(s) carried forward", bad, want, len(points))
        return PriceRead.series(points, product,
                                str(rec.get("provider") or "price_history"),
                                stamp)

    # -- observability -------------------------------------------------------
    @property
    def words(self) -> str:
        """One line for the status page. Says what is held, or why nothing is.

        ⚠️ THIS MUST NOT RAISE, EVER. It is read once from `Controller.__init__`
        and again on every status render, and neither call site guards it - so
        an exception here is an add-on that does not start. It went out that
        way: a day record that was a string instead of a dict raised
        `AttributeError` from the `len(...)` below. `_load()` now drops such a
        record and `_n_points()` is total, so this has two independent reasons
        not to fail on a broken file.
        """
        if not self.ok:
            return f"unavailable - {self.last_error}"
        if not self._days:
            why = f" - {self.last_error}" if self.last_error else ""
            return f"empty{why} - nothing to carry forward yet"
        held = "; ".join(f"{d} ({_n_points(self._days[d])} h)"
                         for d in sorted(self._days, key=str))
        # A file that PARTLY loaded says so, or the dropped days read as days
        # that were simply never written.
        return held + (f" [{self.last_error}]" if self.last_error else "")


def _parse_points(raw) -> tuple:
    """Canonical dicts -> PricePoints, and a phrase naming what was dropped.

    ⚠️ A MALFORMED POINT IS DROPPED, NOT DEFAULTED, and the drop is reported.
    `float(row.get("eur_per_kwh", 0))` is the shape of the whole family of
    defects this repo has been removing: a cache that lost a field would answer
    "that hour was free" to a fill that flags it as an ordinary guess.
    """
    if not isinstance(raw, list):
        return (), "the `points` field is not a list"
    out, dropped = [], 0
    for row in raw:
        if not isinstance(row, dict):
            dropped += 1
            continue
        try:
            start = datetime.fromisoformat(row["start"])
            value = float(row["eur_per_kwh"])
            flag = row["provisional"]
        except (KeyError, TypeError, ValueError):
            dropped += 1
            continue
        if start.tzinfo is None or start.utcoffset() is None \
                or not isinstance(flag, bool):
            dropped += 1
            continue
        out.append(PricePoint(start=start, eur_per_kwh=value, provisional=flag))
    out.sort(key=lambda p: p.start.astimezone(timezone.utc))
    return tuple(out), (f"{dropped} malformed point(s)" if dropped else "")


class NoHistory:
    """The explicit "this caller keeps no history" - a NAME, not a `None`.

    ⚠️ IT EXISTS SO THE PARAMETER CAN BE REQUIRED. `history=None` meaning "skip
    the fill" is the shape this repo keeps removing: a caller who forgets the
    argument gets the truncated-horizon behaviour of the bug this ticket is
    about, silently, and the code reads as though the fill is happening. With
    no default there is nothing to forget, and a caller with no durable volume
    - a check, a replay harness - has to say `NoHistory()` and be visible in
    the diff.

    It carries out the ordering faithfully: it records nothing and carries
    nothing, so `carry()` returns the provider's read unchanged and the plan is
    exactly what it was before FCST-04.
    """

    ok = False
    last_error = "no history is kept by this caller"
    words = "not kept"

    def carry(self, read: PriceRead, *, since, until, stamp) -> PriceRead:
        return read

    def yesterday(self, stamp: datetime) -> PriceRead:
        return PriceRead.no_data(
            "this caller keeps no price history (NoHistory), so §8.3 has no "
            "profile to carry forward")