TARIFF-08: the day-ahead feed is read in the shape it actually publishes

`generic_dayahead` could not read one byte of `api.energy-charts.info`, and
never had been able to. `_points_from_json()` asks for `doc["data"]`, a list of
`{start, price_eur_mwh}` objects; that document does not exist. Fetched live on
2026-09-10 the feed returns COLUMNAR parallel arrays and no `data` key at all -
`unix_seconds` (96 UTC epoch seconds, 900 s apart), `price` (96 floats),
`unit: 'EUR / MWh'`. Driving the shipped parser against that body:
`SourceUnavailable: the feed carried no 'data' list`. It is the only shipped
provider whose endpoint anybody has reached, so dynamic pricing did not work.

AND §11.4 MADE IT SILENT, which is why it survived to release: a parse error
becomes no_data, becomes the static profile, becomes arbitrage off - the
CORRECT degradation, saying nothing about the cause. A site would have sat on
the fallback indefinitely.

AND THE SUITE COULD NOT SEE IT, because every fixture was built from the
parser's own assumption. test_price.py, test_price_history.py and
test_static_tariff.py each built `json.dumps({"data": [...]})`, a shape invented
here and published by nobody, so the checks graded the parser against the
parser's guess - a closed loop no coverage inside it could ever have escaped.
MUTATION.md habit 19 records it as the fourteenth instance of habit 4 and the
first where the fixture and the code under test shared a SHAPE, not a number.

WHAT CHANGED

* `_points_from_columnar_json()` - a SECOND function, not a sniff. A parser that
  decides which document it received is one that can decide wrong, and a
  wrong-but-parseable response then becomes a price. Feeding the row shape to
  `generic_dayahead` is now a graded no_data.
* Unequal arrays are refused in both directions, with the two lengths in the
  reason. `zip()` stops at the shorter list silently, and a truncated `price`
  array would have produced a real series missing the evening peak - traded on
  as complete, where no data at all switches arbitrage off.
* A unit that is not EUR/MWh is refused. The plugin converts with a fixed
  0.001, and `MAX_PLAUSIBLE_EUR_KWH` only bounds prices that are too LARGE.
* `unix_seconds` are read as UTC instants explicitly; §9.3's Brussels rendering
  stays in `assemble()` rather than being copied into the plugin, where a
  mutation could not tell it from a no-op (M4 proved exactly that on a first
  draft, and the line was removed rather than fenced).
* A published null is dropped and logged with a count; an all-null body is
  no_data named by this parser rather than by the boundary validator.

FIXTURES

Two unmodified response bodies under `scenarios/`, captured by hand on
2026-09-10: quarter-hourly (2026-09-10, 96 points at 900 s) and hourly
(2025-09-10, 24 at 3600 s - the same endpoint before Belgium's 15-minute MTU).
`energy-charts-be-price-captures.json` records the URL, retrieval time, unit,
licence-as-returned and a sha256 of each, and the replay RE-DERIVES the
checksums, so a fixture edited to make a check pass stops being a capture. Data
CC BY 4.0, Bundesnetzagentur | SMARD.de, via Energy-Charts (Fraunhofer ISE),
published without changes - the same BE-zone licence check DATA-02 established.

The surviving invented fixtures are renamed `synthetic_row_feed` at every call
site. `engie_dynamic`, `totalenergies_dynamic` and `eneco_eddy` are untouched
and have no capture: their URLs are unverified and withdrawing them is
TARIFF-07's open decision. Nothing should read their green checks as evidence
that those feeds parse.

MUTATIONS - 12, all red, no survivors. Purged tree, mounted `:ro`, `python3 -B`
(TEST-15). M1 drop the equal-length refusal; M2 make it one-sided; M3 drop the
unit check; M4 `assemble()` stops rendering at the Brussels offset; M5 read
`unix_seconds` as a naive local clock; M6 zero-fill a null instead of dropping
it; M7 drop the all-null refusal; M8 sniff for the row shape; M9 drop the
deprecated warning; M10 drop the dropped-nulls warning; M11 edit one price in
the committed capture; M12 point `generic_dayahead` back at the row parser -
red across all three price suites.

END TO END, LIVE, 2026-09-10: `resolve_tariff()` returns a `TariffPlan`,
`origin='provider'`, 96 quarter-hourly points, spacing [900] s, no decimation,
first point 2026-09-10T00:00:00+02:00 at 0.30608 EUR/kWh all-in. FCST-04's H1
fix was never theoretical. All 29 test files green.

STILL OPEN: `generic_dayahead` asks for no date range, so the default URL
returns one published day - a full 48 h horizon is reached only through §8.3's
carry-forward fill.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU

goodwe_controller/app/price.py

@@ -979,9 +979,166 @@ def _points_from_json(body, price_key, scale, provisional_default):
     return out
 
 
+
+# --------------------------------------------------------------------------- #
+# TARIFF-08: the COLUMNAR document, which is what the public EPEX feed really
+# publishes.
+#
+# ⚠️ THIS IS A SECOND SHAPE, READ BY A SECOND FUNCTION ON PURPOSE.
+# `_points_from_json()` above reads a row-per-point document: `{"data": [{...},
+# {...}]}`. `https://api.energy-charts.info/price?bzn=BE` does not publish that
+# and never did - it publishes PARALLEL ARRAYS:
+#
+#     {"unix_seconds": [1788991200, 1788992100, ...],   <- UTC epoch seconds
+#      "price":        [216.08,     209.90,     ...],   <- one per instant
+#      "unit": "EUR / MWh", "license_info": "...", "deprecated": false}
+#
+# The shipped parser was pointed at that URL and asked for `doc["data"]`, so
+# every fetch ended `SourceUnavailable: the feed carried no 'data' list` ->
+# §11.4 -> static profile, arbitrage off, and nothing in the status page said
+# why. Verified live 2026-09-10 against both fixtures under `scenarios/`.
+#
+# ⚠️ AND IT IS NOT ONE FUNCTION THAT SNIFFS FOR BOTH. A parser that decides
+# which document it got is a parser that can decide wrong, and a wrong-but-
+# parseable response then becomes a price. Two shapes, two functions, and each
+# plugin names the one its own feed publishes.
+#
+# The columnar shape was already known in this repository: DATA-02's
+# `scenarios/fetch_epex_be.py` has read `zip(d["unix_seconds"], d["price"])`
+# since 2026-08-31. Only the SHIPPED path never learned it, because every
+# fixture in the suite was built from the invented row shape - the parser was
+# graded against its own assumption. See test_price.py section 2b.
+# --------------------------------------------------------------------------- #
+
+COLUMNAR_TIME_KEY = "unix_seconds"
+COLUMNAR_PRICE_KEY = "price"
+COLUMNAR_UNIT_KEY = "unit"
+
+
+def _columnar_unit(text) -> str:
+    """`"EUR / MWh"` -> `"eur/mwh"`. Spacing and case are not information."""
+    return "".join(str(text).split()).lower()
+
+
+def _points_from_columnar_json(body, expect_unit, scale,
+                               provisional_default):
+    """Parse a COLUMNAR price document: parallel `unix_seconds` / `price`.
+
+    Defects this refuses, each of which has a way of becoming a number rather
+    than an error:
+
+      * ⚠️ UNEQUAL ARRAYS ARE `SourceUnavailable`, NOT A SHORT SERIES. `zip()`
+        stops at the shorter array and says nothing, so a `price` list that
+        arrived truncated would produce a REAL price series with its tail
+        missing - and the tail of a Belgian day is the evening peak. A series
+        that is missing the expensive hours is worse than no series: §11.4's
+        no-data row switches arbitrage OFF, whereas a truncated series is
+        traded on. Nothing downstream can tell the two apart, so the refusal
+        has to happen here.
+      * ⚠️ A UNIT THAT IS NOT THE DECLARED ONE IS REFUSED. `scale` is the
+        caller's standing assumption about the unit; if the feed ever
+        publishes EUR/kWh under the same keys, every price silently comes out
+        a THOUSAND times too small. `_validate_raw()` cannot catch that -
+        `MAX_PLAUSIBLE_EUR_KWH` only bounds prices that are too LARGE.
+        DATA-02's fetcher refuses on the same field for the same reason.
+      * ⚠️ `unix_seconds` ARE UTC INSTANTS, SAID SO EXPLICITLY.
+        `datetime.fromtimestamp(t, timezone.utc)`. Without that second
+        argument CPython returns a NAIVE datetime in the HOST's zone, and a
+        naive datetime is a wall clock reading plus an assumption about whose
+        clock it was. `brussels()` and `_validate_raw()` both refuse one, so
+        the mistake surfaces as a refused fetch rather than as a mispriced
+        day - but they refuse ALL naive datetimes and can say nothing about
+        whether this one was the right instant. That part is decided here.
+
+        §9.3's rendering AT THE BRUSSELS OFFSET is `assemble()`'s, once, for
+        every plugin - `brussels(p.start)` there is the single line that does
+        it, and calling it here as well would be a second copy of a rule that
+        already has an owner, indistinguishable from a no-op under mutation.
+        A plugin's job is the right INSTANT; the framework's is the rendering.
+
+    A `null` price is DROPPED and logged, not fatal, and that is a different
+    judgement from the one above: an unequal pair of arrays means the pairing
+    of instant to price is unreliable, whereas a null means the source stated
+    it published no price for that one instant. A dropped instant leaves a
+    hole, and `_Series.at()` already answers a hole with `Absence` - §11.4 for
+    that hour. `scenarios/fetch_epex_be.py` dropped 0 nulls over the 12 months
+    to 2026-08-31, so this is a shape the source allows rather than one it uses.
+    """
+    try:
+        text = body.decode("utf-8") if isinstance(body, (bytes, bytearray)) \
+            else str(body)
+        doc = json.loads(text)
+    except (ValueError, UnicodeDecodeError) as e:
+        raise SourceUnavailable(f"the feed did not parse as JSON: {e}")
+    if not isinstance(doc, dict):
+        raise SourceUnavailable(
+            f"the feed is a {type(doc).__name__}, not a columnar object with "
+            f"{COLUMNAR_TIME_KEY!r} and {COLUMNAR_PRICE_KEY!r} arrays")
+    times = doc.get(COLUMNAR_TIME_KEY)
+    prices = doc.get(COLUMNAR_PRICE_KEY)
+    for key, col in ((COLUMNAR_TIME_KEY, times), (COLUMNAR_PRICE_KEY, prices)):
+        if not isinstance(col, list):
+            raise SourceUnavailable(
+                f"the feed carried no {key!r} array; this provider reads the "
+                f"columnar shape (parallel {COLUMNAR_TIME_KEY}/"
+                f"{COLUMNAR_PRICE_KEY} arrays), and a row-per-point document "
+                f"is a different feed, not this one with a field missing")
+    if len(times) != len(prices):
+        raise SourceUnavailable(
+            f"{COLUMNAR_TIME_KEY} has {len(times)} entries and "
+            f"{COLUMNAR_PRICE_KEY} has {len(prices)}; refusing to zip them. "
+            f"The short array would truncate the series silently, and a price "
+            f"series missing its last hours is one missing the evening peak - "
+            f"traded on as if complete, where no data at all would have "
+            f"switched arbitrage off (§11.4)")
+    unit = _columnar_unit(doc.get(COLUMNAR_UNIT_KEY, ""))
+    if unit != _columnar_unit(expect_unit):
+        raise SourceUnavailable(
+            f"the feed reports {doc.get(COLUMNAR_UNIT_KEY)!r}, not "
+            f"{expect_unit!r}. This provider converts with a fixed factor of "
+            f"{scale}, so a changed unit is a factor of 1000 on every price "
+            f"with nothing downstream able to notice it")
+    if doc.get("deprecated"):
+        # Not a refusal: today's prices are still prices. But §11.4 makes a
+        # dead feed look exactly like a quiet one, so the warning that the
+        # source is going away has to be emitted while it still works.
+        _LOG.warning("the price feed reports deprecated=%r; it still answers, "
+                     "but this endpoint is scheduled to stop",
+                     doc.get("deprecated"))
+    out = []
+    dropped = []
+    for i, (t, v) in enumerate(zip(times, prices)):
+        if v is None:
+            dropped.append(i)
+            continue
+        if isinstance(t, bool) or not isinstance(t, (int, float)) or \
+                not math.isfinite(t):
+            raise SourceUnavailable(
+                f"{COLUMNAR_TIME_KEY}[{i}] is {t!r}, not epoch seconds")
+        if isinstance(v, bool) or not isinstance(v, (int, float)) or \
+                not math.isfinite(v):
+            raise SourceUnavailable(
+                f"{COLUMNAR_PRICE_KEY}[{i}] is {v!r}, which is not a price")
+        start = datetime.fromtimestamp(float(t), timezone.utc)
+        out.append(RawPoint(start=start, eur_per_kwh=float(v) * scale,
+                            provisional=bool(provisional_default)))
+    if dropped:
+        _LOG.warning("the feed published no price for %d of %d instants "
+                     "(index %s); those instants are left OUT of the series, "
+                     "never zero-filled - a missing hour is not a free hour",
+                     len(dropped), len(times),
+                     ", ".join(str(i) for i in dropped[:5])
+                     + (", ..." if len(dropped) > 5 else ""))
+    if not out:
+        raise SourceUnavailable(
+            f"the feed carried {len(times)} instants and a price for none of "
+            f"them")
+    return out
+
+
 @register
 class GenericDayAhead(PriceProvider):
-    """Public EPEX BE day-ahead feed, in EUR/MWh.
+    """Public EPEX BE day-ahead feed, COLUMNAR, in EUR/MWh.
 
     SOURCES: energy
     The framework ADDS: markup, taxes
@@ -990,6 +1147,19 @@ class GenericDayAhead(PriceProvider):
     is the wholesale clearing price and nothing else, so both user-entered
     components have to be added to it - and because this plugin cannot read
     either number, it cannot be the one that adds them twice.
+
+    ⚠️ IT READS THE COLUMNAR SHAPE, AND SAYS SO IN ITS OWN `fetch()`. Until
+    TARIFF-08 it called `_points_from_json()` and asked
+    `api.energy-charts.info` for a `"data"` list it has never published, so
+    every fetch this provider ever made ended in `SourceUnavailable` and §11.4
+    turned that into a silent fall back to the static profile. See
+    `_points_from_columnar_json()` for the shape and for why there are two
+    parsers rather than one that guesses.
+
+    ⚠️ THE FEED IS QUARTER-HOURLY SINCE BELGIUM'S 15-MINUTE MTU (2025-10-01):
+    96 points a day at 900 s, not 24 at 3600 s. Nothing here assumes either -
+    the instants come from the document - and both granularities are replayed
+    from real captured bodies in test_price.py section 2b.
     """
 
     id = "generic_dayahead"
@@ -1006,7 +1176,9 @@ class GenericDayAhead(PriceProvider):
             "https://api.energy-charts.info/price?bzn=BE"
         check_egress(url, self.capabilities())
         body = await transport(url, float(opts.get("price_timeout_s")))
-        return _points_from_json(body, "price_eur_mwh", 0.001, False)
+        return _points_from_columnar_json(body, expect_unit="EUR / MWh",
+                                          scale=0.001,
+                                          provisional_default=False)
 
 
 @register