goodwe-addon/goodwe_controller/app/price.py

"""TARIFF-02 - price provider plugins, and the one canonical shape they all emit.

WHERE THIS LIVES, AND WHY IT IS NOT WHERE THE SPEC SAYS. specs.md §2.2 puts this
at `ems/ingest/prices/`. That module layout does not exist and is not being
built; the product owner's decision of 2026-08-30 is that everything ships in
`goodwe_controller/app/`, next to arbiter.py, clamp.py, freeze.py and
validate.py. The BEHAVIOUR specs.md §9.3 / §8.3 / §11.4 describes is unchanged
and still authoritative - only the path moved.

WHAT THIS IS FOR, in one sentence: so that a new Belgian supplier is a new class
and one line in `PROVIDERS`, and the optimiser that eventually consumes prices
never learns a supplier's name.

THE CANONICAL FORMAT (§9.3), exactly three fields and no more:

    { "prices": [ { "start": "2026-01-15T00:00:00+01:00",
                    "eur_per_kwh": 0.1834,
                    "provisional": false } ] }

`start` is tz-aware at the Brussels offset for that instant, the price is EUR
per kWh and ALL-IN (energy + markup + taxes), and `provisional` is per point,
not per series (§8.3: unknown hours are flagged, and no irreversible action may
be taken on a provisional price).


THE THREE THINGS THIS MODULE IS SHAPED BY
=========================================

1. DOUBLE-COUNTING MARKUP IS REFUSED, NOT DISCOURAGED - GIVEN A CORRECT
   DECLARATION.
   `generic_dayahead` reads a raw EPEX figure and the user's markup and taxes
   have to be added to it. Engie and TotalEnergies publish a figure that already
   HAS the markup in it. Add the configured markup to that one too and every
   arbitrage decision on that site is made against a price that is 3 c/kWh
   wrong, in a direction nothing reports.

   A comment saying "do not add markup here" is not a mechanism. So:

     * A plugin does not return a price. It returns `RawPoint`s carrying the
       figure ITS SOURCE PUBLISHED, and declares in `capabilities().sources`
       which of {ENERGY, MARKUP, TAXES} that figure already contains.
     * `assemble()` - the framework, never a plugin - adds exactly the
       components the plugin declared it did NOT source, from
       `price_markup_eur_kwh` / `price_taxes_eur_kwh`.
     * ⚠️ A PLUGIN CANNOT REACH THOSE TWO NUMBERS BY ANY ROUTE, and the word
       "any" was earned the hard way. `PluginOptions` raises `FrameworkOption`
       on them, which closes the documented path - but review of PR #16 wrote a
       double-counting plugin THREE more ways, one of them touching nothing but
       `PRICE_OPTION_DEFAULTS`, a public dict in the module every plugin must
       already import for `PriceProvider` and `register`. One closed door in a
       room with three is not a guarantee. So `register()` now also REFUSES ANY
       PLUGIN WHOSE OWN CODE MENTIONS A FRAMEWORK KEY - see `code_leaks()`.

   ⚠️ THE PRECISE SHAPE OF THE GUARANTEE, because an overclaim here is worse
   than no claim: **the framework adds the complement exactly once, GIVEN A
   CORRECT `sources` DECLARATION.** A plugin that declares `{ENERGY}` while its
   feed is really all-in still comes out one markup high, and NO CODE CAN CATCH
   THAT - the declaration is the plugin's unverifiable claim about somebody
   else's tariff. What is mechanised is that a plugin cannot obtain the numbers
   and cannot apply them; what remains human is whether it described its feed
   truthfully. The docstring/declaration check in test_price.py verifies that
   the plugin's two claims AGREE, not that either is true, and it is worth
   knowing which of those two things you are reading.

2. THE PRODUCT TYPE IS DECLARED, NOT INFERRED FROM THE PLUGIN'S NAME.
   §9.2 classes real-time / imbalance products separately from day-ahead ones
   and §8.3 says those are treated as high-volatility with widened arbitrage
   thresholds. If the optimiser had to write `if provider_id == "<supplier>"`
   to know that, then adding a supplier would mean editing the optimiser -
   which is precisely what AC2 forbids. So `capabilities().product` is a
   `PriceProduct`, it rides along on the series, and `volatility_regime()`
   reads the enum. A plugin id appears nowhere in this file except in
   `PROVIDERS`, in `RETIRED_PROVIDERS`, and in the class that answers to it.

   ⚠️ NO SHIPPED PLUGIN DECLARES `PriceProduct.REALTIME` (TARIFF-07). Both
   shipped plugins are day-ahead, so `_WIDE`, `Uncertainty.REALTIME_PRODUCT`
   and `WIDEN_FACTOR`'s 2.5 currently have no shipped declarer. They are KEPT
   deliberately - see DECISIONS/TARIFF-07-withdrawing-three-plugins.md - and
   test_price.py section 10 keeps them live by declaring REALTIME from a class
   defined outside this file, which is AC2's own stimulus.

3. "NO PRICE DATA" IS A TYPE, NOT AN EMPTY LIST.
   §11.4: no price data -> fall back to the static profile, mark the plan
   provisional, NO ARBITRAGE. That is a different instruction from "prices
   exist and they are all zero", under which arbitrage is merely pointless
   rather than forbidden - and the two are indistinguishable if a failed fetch
   returns `[]`, or `None`, or a dict a caller `.get()`s a default out of.

   ⚠️ THIS REPO SPENT 2026-08-30 REMOVING EXACTLY THAT DEFECT. `sim/riglock.py`
   in the infra repo used to return a partial dict from `holder()`; TEST-04
   rewrote it so a look at the lock is `observed / not_observed / gone` and only
   the observed variant carries fields. `_Outcome` below is that shape,
   deliberately, down to the fatal `.get()`. Read that file's `class Absence`
   block before changing anything here.

   Concretely:

     * `PriceRead.series(...)` REFUSES AN EMPTY POINT LIST. A series with no
       prices in it cannot be constructed, so "I have no prices" cannot arrive
       wearing the shape of an answer.
     * `PriceRead.no_data(reason)` carries `.reason` AND NOTHING ELSE. It has no
       `.points` to iterate, no `.product` to switch on, no `.at()` to call, and
       `.get("points", [])` raises. `min(read.points)` - the line that would
       silently make "no data" mean "everything is free, charge now" - does not
       run against it.
     * `.match(*, series, no_data)` takes both handlers as REQUIRED keyword
       arguments, so a caller who forgets the no-data case gets a TypeError at
       the call site rather than a fall-through.


TIMEZONE, AND WHY THERE IS NO `zoneinfo` HERE
=============================================
§9.3's example is `+01:00`, and Belgium runs CET/CEST. Two constraints:

  * The shipped image is Alpine with `py3-aiohttp` from apk and NO `tzdata`
    package (goodwe_controller/Dockerfile, and .gitea/ci/Dockerfile is the same
    base by design). `ZoneInfo("Europe/Brussels")` raises
    `ZoneInfoNotFoundError` there. A price module that only works on a machine
    with a tz database is a price module that fails on the client's Pi.
  * Fixed offsets have no fold. Every instant here is carried as UTC and
    RENDERED at the offset that applies to it, so the 02:00-03:00 hour that
    happens twice in October is two different instants that render as
    `02:00+02:00` and `02:00+01:00`. A naive-local-hour key collides those two
    and misprices an hour of a 25-hour day; an implementation on a fixed
    24-slot array is wrong on both transition days. test_price.py drives both
    2026 transitions and counts the points rather than asserting 24.

The EU rule is fixed in law and needs no database: CEST from 01:00 UTC on the
last Sunday of March until 01:00 UTC on the last Sunday of October. Review of
PR #16 cross-checked this against `zoneinfo`'s Europe/Brussels over 149,040
hourly instants, 2020-2036 inclusive, and found ZERO mismatches - so the
residual risk here is not arithmetic, it is political: if the EU ever abolishes
the seasonal change, `brussels_offset()` needs editing. That is the same
exposure as a machine whose `tzdata` has gone stale, and it is one edit in one
function rather than a package upgrade.
"""

import ast
import csv
import inspect
import io
import json
import logging
import math
from abc import ABC, abstractmethod
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from enum import Enum
from textwrap import dedent
from urllib.parse import urlencode

_LOG = logging.getLogger("goodwe.price")


# --------------------------------------------------------------------------- #
# Brussels wall time, without a tz database. See the module docstring.
# --------------------------------------------------------------------------- #

CET = timezone(timedelta(hours=1))
CEST = timezone(timedelta(hours=2))


def _last_sunday(year: int, month: int) -> int:
    """Day-of-month of the last Sunday. March and October both have 31 days."""
    day = 31
    while datetime(year, month, day).weekday() != 6:   # 6 == Sunday
        day -= 1
    return day


def brussels_offset(instant: datetime) -> timezone:
    """The offset Belgium is on at this UTC instant. CET or CEST, never else."""
    u = instant.astimezone(timezone.utc)
    y = u.year
    start = datetime(y, 3, _last_sunday(y, 3), 1, tzinfo=timezone.utc)
    end = datetime(y, 10, _last_sunday(y, 10), 1, tzinfo=timezone.utc)
    return CEST if start <= u < end else CET


def brussels(instant: datetime) -> datetime:
    """Render a UTC instant at the Brussels offset for that instant.

    ⚠️ Takes an INSTANT, and refuses a naive one. A naive datetime is not a
    moment in time, it is a wall clock reading plus an assumption, and the
    assumption is wrong for one hour twice a year.
    """
    if instant.tzinfo is None or instant.utcoffset() is None:
        raise ValueError(
            f"brussels({instant!r}): a naive datetime is not an instant. Parse "
            f"the source's offset, or say UTC explicitly - guessing here is how "
            f"one hour of a transition day gets priced twice.")
    return instant.astimezone(brussels_offset(instant))


def brussels_date_range(since: datetime, until: datetime) -> tuple:
    """The Brussels calendar days a half-open window [since, until) touches.

    Returns `(first, last)`, two `date`s, BOTH INCLUSIVE - which is the shape
    `api.energy-charts.info` takes: `?bzn=BE&start=2026-09-09&end=2026-09-10`
    answered 192 quarter-hours across exactly those two local days, measured
    2026-09-10T07:45:49Z and committed as
    `scenarios/energy-charts-be-price-2026-09-09_2026-09-10-twoday.json`.

    THE DEFECT THIS EXISTS TO PREVENT (TARIFF-09). `generic_dayahead.fetch()`
    accepted `since` and `until` and requested the bare default URL, so the
    feed answered ONE Brussels day. Everything past today was then carried
    forward from yesterday's profile, flagged provisional, and §8.3 dropped all
    of it from the trade set - so the optimiser never saw tomorrow as firm, at
    any hour. Measured on the merged tree at 2026-09-10T09:00+02:00 over a 48 h
    horizon: 15 firm whole hours and 33 carried, and both ends of the only
    permitted cycle inside ONE calendar day. The ceiling was structural: at
    hour H of the local day a one-day window gives at most 24-H firm hours and
    24+H carried - 1/47 at 23:00, where the firm spread no longer even clears
    the round-trip cost. That is not what FCST-04's fill is for; it covers the
    pre-publication window until the ~13:00-14:00 CET auction, after which a
    healthy feed hands over tomorrow in full.

    ⚠️ THE CALENDAR IS BRUSSELS, NEVER UTC, AND THIS IS `_bdate()`'s LESSON ONE
    LAYER UP. `since`/`until` are INSTANTS; the API takes DATES, and the two are
    not the same question. 22:30 UTC in summer is already tomorrow in Belgium,
    so a range derived from `since.date()` asks for the wrong FIRST day for the
    last one or two hours of every single day - and the day it drops is the one
    the controller is standing in. price_history.py made this mistake at the
    cache layer and documents the same arithmetic; here it decides what is
    fetched at all, so getting it wrong loses the hours rather than misfiling
    them.

    ⚠️ `until` IS EXCLUSIVE, SO LOCAL MIDNIGHT BELONGS TO THE DAY BEFORE IT.
    A 48 h horizon from local midnight ends at local midnight two days later,
    and that instant is the first tick of a day the caller did not ask for.
    Taking `brussels(until).date()` unconditionally would request a THIRD day
    on every whole-hour horizon. This feed answers such a request without
    complaint, so nothing would report it - which is the reason to get it right
    here rather than to rely on the server's tolerance: the range is a claim
    about what the caller asked for, and it should be true.

    ⚠️ AND AN EMPTY OR INVERTED WINDOW COLLAPSES TO ONE DAY RATHER THAN TO A
    BACKWARDS RANGE. `start` after `end` is a request no feed has a defined
    answer for; some return everything, some return an error, and neither is a
    thing to hand a battery controller.
    """
    first = brussels(since).date()
    edge = brussels(until)
    last = edge.date()
    if (edge.hour, edge.minute, edge.second, edge.microsecond) == (0, 0, 0, 0):
        last = last - timedelta(days=1)
    if last < first:
        last = first
    return first, last


def dated_feed_url(base: str, **params) -> str:
    """`base` plus a query string. Refuses a `base` that already carries one.

    ⚠️ THE REFUSAL IS THE POINT, NOT THE FORMATTING. Appending `&start=...` to
    a URL that may already hold its own query string is how a plugin produces a
    request nobody intended - `?bzn=BE?bzn=BE&start=...`, or a duplicated
    parameter whose winner is the server's choice rather than ours. So the base
    is required to be parameterless and the parameters are added exactly once,
    by `urlencode`, which is also what keeps a date out of the escaping
    question. A future edit that folds the query back into the constant is a
    refused fetch with a sentence, not a silently malformed URL.

    ⚠️ IT IS NOT REACHED BY AN OPERATOR'S `price_feed_url`. See
    `GenericDayAhead.fetch()`: an override is used exactly as given.
    """
    if "?" in base or "#" in base:
        raise SourceUnavailable(
            f"the feed base {base!r} already carries a query string or "
            f"fragment, and this builder appends parameters. Adding a second "
            f"'?' produces a URL the server parses differently from the one "
            f"anybody intended; put the fixed parameters in the call instead")
    return f"{base}?{urlencode(params)}" if params else base


# --------------------------------------------------------------------------- #
# Absence, borrowed wholesale from sim/riglock.py (TEST-04).
# --------------------------------------------------------------------------- #

class Absence(Exception):
    """A value was asked of a read that never observed one."""


class FrameworkOption(Exception):
    """A plugin reached for an option the FRAMEWORK owns. See trap 1."""


class SourceUnavailable(Exception):
    """A plugin could not get its data. Raised by plugins, caught by the
    framework, and turned into `PriceRead.no_data(...)`.

    ⚠️ It must never escape into the control tick. §11.4's row is a returned
    condition, not a traceback: the loop keeps regulating on P1 whatever the
    price situation is, and a price fetch that could stop it would be a price
    fetch that can stop the battery.
    """


class _Outcome:
    """One outcome of one read. The absent variant CARRIES NO FIELDS."""

    kind = "?"
    words = "?"          # human phrasing. Cosmetic, never an input to a decision.

    def __getattr__(self, name):
        if name.startswith("__") and name.endswith("__"):
            raise AttributeError(name)
        raise Absence(
            f"{type(self).__name__} has no {name!r}. This read did not observe "
            f"a price series, and a value invented here is the bug this "
            f"representation exists to prevent - 'no data' becoming 'free'. "
            f"Handle the {self.kind!r} case explicitly; see match().")

    def get(self, *_a, **_k):
        raise Absence(
            f"{type(self).__name__}.get() does not exist: a default is how a "
            f"failed fetch becomes an observation. Use .match(...) and handle "
            f"both outcomes.")

    def __iter__(self):
        # ⚠️ DELIBERATE. `for p in read:` and `list(read)` are the other two
        # idioms that turn an absent read into an empty, harmless-looking loop.
        raise Absence(
            f"{type(self).__name__} is not iterable: an empty loop body is how "
            f"'we have no prices' becomes 'we checked every hour and none were "
            f"expensive'. Use .match(series=..., no_data=...).")

    def __repr__(self):
        return f"{type(self).__name__}({self.words!r})"


# --------------------------------------------------------------------------- #
# The vocabulary
# --------------------------------------------------------------------------- #

class Component(Enum):
    """The three parts of an all-in Belgian consumption price (§9.3)."""
    ENERGY = "energy"
    MARKUP = "markup"
    TAXES = "taxes"


ALL_IN = frozenset(Component)


class PriceProduct(Enum):
    """The §9.2 product table. The optimiser switches on THIS, never on a name."""
    DAY_AHEAD = "dynamic_dayahead"
    REALTIME = "realtime_imbalance"
    STATIC = "static_profile"
    FIXED = "fixed"


# ⚠️ §8.3: "Real-time / imbalance products are treated as high-volatility:
# arbitrage thresholds widen." This maps product -> regime, and it is the ONLY
# place in the shipped code that says which products are volatile. OPT-04 reads
# it; it does not re-derive it, and it certainly does not check plugin ids.
_WIDE = frozenset({PriceProduct.REALTIME})


def volatility_regime(product: PriceProduct) -> str:
    """"wide" | "normal" - the arbitrage threshold regime for a product (§8.3).

    Deliberately a plain string and deliberately tiny: OPT-04 owns what the
    widened numbers ARE. This owns only which products get them, because that
    is a property of the product and belongs with the product enum.
    """
    if not isinstance(product, PriceProduct):
        raise TypeError(f"volatility_regime() takes a PriceProduct, not "
                        f"{product!r} - a name is exactly what AC2 forbids.")
    return "wide" if product in _WIDE else "normal"


# --------------------------------------------------------------------------- #
# FCST-03: ONE widening path, driven by a SET of active uncertainty sources.
#
# Two spec rows widen the same number by two different triggers:
#
#   §9.2 / §8.3  real-time / imbalance product   -> "arbitrage thresholds widen"
#   §11.4        no weather data                 -> "widen arbitrage thresholds"
#
# They are one mechanism reached twice, not two mechanisms, so there is one
# function and a set of reasons. ⚠️ THE COMBINATION RULE IS `max`, NOT A
# PRODUCT: multiplying 2.5 by 1.5 would make two independent doubts compound
# into a threshold nothing on a Belgian tariff can ever clear, and an arbitrage
# strategy that can never fire is indistinguishable from one that is switched
# off. The widest single reason wins.
#
# ⚠️ HONEST ABOUT WHERE THE NUMBERS COME FROM. specs.md says thresholds widen
# and does not say by how much; there is no capture to derive a factor from,
# because no arbitrage strategy has ever run here. These are POLICY DEFAULTS,
# chosen so that (a) a real-time product needs a materially larger spread than
# a settled day-ahead one before a cycle is worth its wear, and (b) the two
# factors DIFFER, so `max`, `sum` and `product` are three distinguishable
# answers rather than the same number - a check that grades a combination rule
# where all the candidates agree grades nothing. When OPT-04 lands
# `cycle_cost_threshold_eur` (§10.1/§14) these become the natural second option
# beside it, in PRICE_OPTION_DEFAULTS and nowhere else (CFG-03).
class Uncertainty(Enum):
    """Why an arbitrage threshold is currently wider than the tariff alone says."""
    REALTIME_PRODUCT = "realtime_product"    # §9.2 row 2, §8.3
    NO_WEATHER = "no_weather"                # §11.4 "no weather data"


WIDEN_FACTOR = {
    Uncertainty.REALTIME_PRODUCT: 2.5,
    Uncertainty.NO_WEATHER: 1.5,
}


def uncertainty_of(product: PriceProduct) -> frozenset:
    """The uncertainty a PRODUCT contributes on its own (§9.2 row 2).

    Derived from the declared product, never from a plugin id - the same rule
    `volatility_regime()` exists for. A sixth supplier declaring
    `PriceProduct.REALTIME` is widened without anyone editing this file.
    """
    if not isinstance(product, PriceProduct):
        raise TypeError(f"uncertainty_of() takes a PriceProduct, not "
                        f"{product!r} - a plugin name is what §9.3 forbids.")
    return (frozenset({Uncertainty.REALTIME_PRODUCT})
            if product in _WIDE else frozenset())


def widen_factor(sources) -> float:
    """The multiplier the widest active source asks for. 1.0 when there are none.

    ⚠️ A STRING IS REFUSED. `widen_factor({"realtime"})` would otherwise be a
    silent 1.0 - a widening that reports itself as applied and is not, which is
    the shape of defect this repo keeps finding.
    """
    srcs = frozenset(sources)
    bad = [s for s in srcs if not isinstance(s, Uncertainty)]
    if bad:
        raise TypeError(f"widen_factor() takes Uncertainty members, not {bad!r}. "
                        f"A string here would widen by 1.0 and say it widened.")
    return max((WIDEN_FACTOR[s] for s in srcs), default=1.0)


def required_spread(base_eur_kwh: float, sources) -> float:
    """The spread an arbitrage cycle must clear, given what is uncertain (§8.3).

    `base_eur_kwh` is the caller's own round-trip cost threshold - OPT-04 owns
    that number (§10.1's `cycle_cost_threshold_eur`), this owns only how much
    wider it gets and why.
    """
    if isinstance(base_eur_kwh, bool) or             not isinstance(base_eur_kwh, (int, float)) or             not math.isfinite(base_eur_kwh) or base_eur_kwh < 0:
        raise ValueError(
            f"required_spread() takes a non-negative, finite base threshold in "
            f"EUR/kWh, not {base_eur_kwh!r}. A negative or NaN base makes every "
            f"spread clear it, which is arbitrage with the brake removed.")
    return round(float(base_eur_kwh) * widen_factor(sources), 6)


@dataclass(frozen=True)
class PricePoint:
    """One canonical point. THREE FIELDS. §9.3 lists three; this has three."""

    start: datetime        # tz-aware, rendered at the Brussels offset
    eur_per_kwh: float     # ALL-IN: energy + markup + taxes
    provisional: bool

    def canonical(self) -> dict:
        return {"start": self.start.isoformat(),
                "eur_per_kwh": self.eur_per_kwh,
                "provisional": self.provisional}


@dataclass(frozen=True)
class RawPoint:
    """What a plugin returns: the figure its source published, unmodified.

    ⚠️ NOT a PricePoint, and the type difference is the guard. A plugin cannot
    hand back something already claiming to be all-in, because the only
    constructor of `PricePoint` that `load_prices()` uses is `assemble()`.
    """

    start: datetime            # tz-aware instant
    eur_per_kwh: float         # as published, containing exactly caps.sources
    provisional: bool


@dataclass(frozen=True)
class ProviderCapabilities:
    """What a plugin declares about itself, before anyone asks it for prices."""

    product: PriceProduct
    sources: frozenset          # which Components the published figure contains
    horizon_h: int = 48         # aligns with the §8.1/§8.2 forecast horizon
    egress_hosts: tuple = ()    # §16 allow-list, declared not discovered

    def __post_init__(self):
        if not isinstance(self.product, PriceProduct):
            raise ValueError(f"product must be a PriceProduct, not "
                             f"{self.product!r}")
        if Component.ENERGY not in self.sources:
            raise ValueError(
                "capabilities().sources must contain Component.ENERGY - a feed "
                "that does not publish an energy price is not a price feed.")
        extra = set(self.sources) - set(ALL_IN)
        if extra:
            raise ValueError(f"unknown price components declared: {extra}")

    @property
    def adds(self) -> frozenset:
        """The components the FRAMEWORK must add. The complement of `sources`."""
        return frozenset(ALL_IN - self.sources)


# --------------------------------------------------------------------------- #
# The read: a series, or no data. Nothing in between.
# --------------------------------------------------------------------------- #

class PriceRead(_Outcome):
    """What one attempt to get prices said. Two outcomes, and only one answers.

      * `series`  - we have prices. Carries `.points` (NON-EMPTY, by
                    construction), `.product`, `.provider`, `.fetched_at`.
      * `no_data` - we do not. Carries `.reason` and NOTHING ELSE. §11.4: static
                    profile, plan provisional, no arbitrage.

    ⚠️ A SERIES OF ALL-ZERO PRICES IS A `series`, NOT `no_data`, and that
    distinction is the whole point. Zero prices mean arbitrage is pointless;
    no data means arbitrage is FORBIDDEN. An implementation that returns `[]`
    on a failed fetch cannot tell a caller which of those two it is in.
    """

    def match(self, *, series, no_data):
        """Fold to a value. Both handlers required and keyword-only.

        Handlers take one argument: this read. Required-ness is what makes the
        no-data case impossible to forget rather than merely easy to remember.
        """
        return {"series": series, "no_data": no_data}[self.kind](self)

    @staticmethod
    def series(points, product, provider, fetched_at):
        return _Series(points, product, provider, fetched_at)

    @staticmethod
    def no_data(reason):
        if not reason or not isinstance(reason, str):
            raise ValueError(
                "PriceRead.no_data() takes a reason a human can act on. It is "
                "the only field this variant has; an empty one makes an "
                "operator's only evidence a blank status line.")
        return _NoData(reason)


# --------------------------------------------------------------------------- #
# How long ONE point covers - TARIFF-14
# --------------------------------------------------------------------------- #

# ⚠️ A POINT DOES NOT CARRY ITS OWN DURATION AND MUST NOT LEARN TO. §9.3 lists
# THREE fields and `PricePoint` has three; a fourth would change the wire
# format every store, replay, fixture and captured body in this repository is
# written against. The span is DERIVED from the neighbours instead - here,
# once, at construction.
MAX_POINT_SPAN = timedelta(hours=1)


def _cover_table(points) -> tuple:
    """`(start_utc, end_utc, point)` per point, ascending. The `at()` index.

    THE DEFECT THIS PREVENTS (TARIFF-14, HIGH). `at()` asked
    `s <= u < s + 1 h` of every point IN LIST ORDER and returned the FIRST
    match. On an hourly series - the only resolution any fixture carried
    before TARIFF-08 - that is right by accident. Belgium moved to a
    15-minute MTU on 2025-10-01 and the live day-ahead feed returns 96 points
    at 900 s, where it is wrong twice over. Measured on shipped `5059e28`:

        series: 02:00=0.10  02:15=0.11  02:30=0.12  02:45=0.13  03:00=0.14

        at(02:15) -> 02:00   the :00 point's HOUR has not run out yet
        at(03:15) -> 02:30   and from the second hour on it is not even the
                             top of the hour: the first point still inside
                             its own hour wins, so the error COMPOUNDS with
                             depth into the series

    `arbitrage._act()` compares `here.start` against `v.cheapest.start` and
    `v.dearest.start` to decide whether this interval is the buy or the sell
    one. That identity test almost never matched the extremum, so the
    strategy idled through exactly the interval it exists to trade - and read
    `here.provisional` and §17's `price < 0` guard off the wrong point while
    doing it. `static_tariff.py` emits hourly points, so static sites never
    saw it; this is reachable only down the dynamic path TARIFF-08/09 made
    work.

    THE RULE, in one sentence:

        A POINT COVERS FROM ITS OWN START UNTIL THE NEXT POINT'S START, AND
        NEVER MORE THAN ONE HOUR.

      * 900 s  -> each point covers its own quarter, so `at(02:15)` is the
        02:15 point;
      * 3600 s -> each point covers its own hour, unchanged from shipped;
      * MIXED  -> EACH POINT TAKES ITS OWN GAP. FCST-04's carry-forward emits
        HOURLY carried points beside QUARTER-HOURLY firm ones, and the soak
        measured spacing `[900, 3600]` in 70 % of samples - so mixed is the
        normal steady state, and a window derived from one global spacing
        would be wrong THERE rather than in an edge case.

    ⚠️ WHY THE ONE-HOUR CAP, WHICH IS NOT DECORATION. FCST-04 LEAVES HOLES ON
    PURPOSE: an hour with no published price AND no yesterday value is left
    OUT of the series, and `price_forecast.py`'s module note names `at()`
    raising `Absence` for it as the behaviour that sends the caller to the
    static profile (§11.4 for that hour). Uncapped, "until the next point's
    start" would stretch the point BEFORE a three-hour hole across the whole
    hole and answer every instant in it with a stale price. Filling holes is
    the defect this module exists to refuse, and it would have been
    introduced by the fix for a different one.

    ⚠️ THE LAST POINT'S WINDOW IS DECIDED, NOT INHERITED. There is no next
    start and nothing in §9.3 says how wide the final interval is, so THE
    LAST POINT IS GIVEN THE SAME SPAN AS THE POINT BEFORE IT - one hour when
    it is the only point. Three consequences, all intended:

      * on a 96-point 900 s day the 23:45 point covers 23:45 .. 00:00, so
        `at(23:50)` ANSWERS and `at(00:00)` raises `Absence`. A flat one-hour
        tail would have answered 00:30 TOMORROW with today's last price,
        which is reaching for a neighbour across a day boundary;
      * on an hourly series the tail is an hour, exactly as it shipped, so no
        static or hourly site changes behaviour;
      * on a MIXED series whose last point is a carried HOURLY one behind
        quarter-hourly firm ones, the tail is 900 s and the last 45 minutes
        of the horizon raise `Absence`. That is the SAFE direction and it is
        accepted rather than tolerated: the caller falls back to the static
        profile (§11.4) instead of trading on a span the series never
        demonstrated, and a carried point is provisional, so §8.3 forbids any
        irreversible action resting on it regardless.

    ⚠️ ORDER IS ESTABLISHED HERE, NOT ASSUMED - and that is the difference
    between this and the rule it replaces. `at()` now reads neighbours, so it
    WOULD depend on sortedness; the table is therefore sorted by UTC start at
    construction, and no caller's list order can reach it. `_validate_raw()`
    rejects out-of-order and duplicate starts on the `load_prices()` path,
    `price_history._parse_points()` re-sorts what it reads off disk, and
    FCST-04's fill emits ascending - but `PriceRead.series()` itself has
    never enforced any of that, and an assumption that holds today and is
    unstated is how this class of defect recurs.

    ⚠️ NOTHING HERE RAISES ON A MALFORMED SERIES. A duplicated start - not
    reachable through `load_prices()`, which refuses it - gives the earlier
    list entry a ZERO-width window and the later one the real span, so one of
    the pair becomes unreachable through `at()` and neither the 30 s control
    path nor construction raises.
    """
    rows = sorted(((p.start.astimezone(timezone.utc), i, p)
                   for i, p in enumerate(points)),
                  key=lambda r: (r[0], r[1]))
    out = []
    for n, (start, _i, point) in enumerate(rows):
        if n + 1 < len(rows):
            span = rows[n + 1][0] - start          # -> the next point's start
        elif n:
            span = start - rows[n - 1][0]          # -> the tail, see above
        else:
            span = MAX_POINT_SPAN                  # -> a series of one point
        if span > MAX_POINT_SPAN:
            span = MAX_POINT_SPAN
        out.append((start, start + span, point))
    return tuple(out)


class _Series(PriceRead):
    kind = "series"

    def __init__(self, points, product, provider, fetched_at):
        points = list(points)
        # ⚠️ THE REFUSAL THAT MAKES THE TYPE WORTH HAVING. An empty series is
        # how "the fetch failed" gets to wear the shape of "we looked and there
        # is nothing expensive today". It is not constructible.
        if not points:
            raise ValueError(
                "PriceRead.series() refuses an empty point list. A fetch that "
                "produced no points is PriceRead.no_data(reason) - a different "
                "variant with different fields, on purpose. See §11.4.")
        if not isinstance(product, PriceProduct):
            raise ValueError(
                f"product must be a PriceProduct, not {product!r}. The "
                f"optimiser switches on this enum instead of on a plugin name; "
                f"a string here reopens exactly the hardcoding AC2 forbids.")
        self.points = tuple(points)
        self.product = product
        self.provider = provider
        self.fetched_at = fetched_at
        self.regime = volatility_regime(product)
        # ⚠️ TARIFF-14. Built ONCE, here, so that `at()` scans an index whose
        # order it established itself rather than one it hopes it was handed.
        # See `_cover_table()` for the window rule and for what the tail does.
        self._cover = _cover_table(self.points)
        self.words = (f"{len(self.points)} points from {provider} "
                      f"({product.value}, {self.regime} thresholds)")

    def canonical(self) -> dict:
        """The §9.3 wire format. Exactly `{"prices": [...]}`."""
        return {"prices": [p.canonical() for p in self.points]}

    def at(self, instant: datetime) -> PricePoint:
        """The point whose OWN interval covers `instant`, at ANY resolution.

        ⚠️ RETURNS A POINT OR RAISES; it never returns None. `series.at(t) or
        fallback` is the same defaulting bug one level down, so the falsy
        answer does not exist. Ask `covers()` first if you mean to branch.

        ⚠️ TARIFF-14: THE WINDOW IS NOT ONE HOUR WIDE ANY MORE, AND WAS NEVER
        ENTITLED TO BE. It runs to the next point's start, capped at an hour -
        `_cover_table()` above holds the rule, the tail decision and the
        reproduction on a quarter-hourly series. A fixed `s + 1 h` here
        returned the FIRST point still inside its own hour, which on the live
        900 s feed is neither the covering point nor, past the first hour,
        even the top of the hour.

        A linear scan over the index, which is 192 entries on the shipped 48 h
        horizon. This runs on the 30 s control path through
        `arbitrage._point_at()`; a bisect would be correct and is not worth
        the second way to be wrong at this size.
        """
        u = instant.astimezone(timezone.utc)
        for start, end, point in self._cover:
            if start <= u < end:
                return point
        raise Absence(
            f"no price point covers {u.isoformat()}; the series runs "
            f"{self.points[0].start.isoformat()} .. "
            f"{self.points[-1].start.isoformat()}. An instant outside the "
            f"series - or inside a hole FCST-04 left because nothing "
            f"published that hour - is an instant with no price, which is "
            f"§11.4 for it, not a reason to reach for a neighbouring one.")

    def covers(self, instant: datetime) -> bool:
        """Whether `at(instant)` would answer.

        ⚠️ SHARED TARIFF-14 EXACTLY, AND IS FIXED BY THE SAME CHANGE. It has
        no window rule of its own - it asks `at()` - so on the 900 s feed it
        answered True for instants no point covered and False for none of
        them, because the hour-wide window is never SHORTER than the truth.
        It is kept as a delegation for that reason: a second copy of the rule
        is a second place for it to be wrong.
        """
        try:
            self.at(instant)
            return True
        except Absence:
            return False

    def window(self, since: datetime, until: datetime) -> tuple:
        """Points whose START falls in [since, until). May legitimately be empty.

        ⚠️ DID NOT SHARE TARIFF-14, AND THE REASON IS WORTH STATING RATHER
        THAN REDISCOVERING. This grades a point's `start` against a half-open
        range and never asks how long a point lasts, so it carries no
        resolution assumption to be wrong about - the same call answers the
        same set at 900 s, at 3600 s and mixed. It is therefore NOT a
        coverage query: a point that starts before `since` and runs into it
        is absent from this tuple by design. `at()`/`covers()` are the
        coverage questions.
        """
        a = since.astimezone(timezone.utc)
        b = until.astimezone(timezone.utc)
        return tuple(p for p in self.points
                     if a <= p.start.astimezone(timezone.utc) < b)


class _NoData(PriceRead):
    """We have no prices, and the §11.4 row applies.

    ⚠️ THIS CLASS HAS NO `.points` AND THAT IS THE POINT. `min(read.points)`,
    `for p in read.points`, `read.get("points", [])` and `len(read.points)` all
    raise `Absence` here rather than quietly reporting that the cheapest hour of
    the day costs nothing.

    ⚠️ TWO WAYS PAST IT SURVIVE, BOTH FOUND IN REVIEW OF PR #16, AND NEITHER IS
    WORTH CODE:

      * `vars(read).get("points", [])` returns `[]`. `vars()` bypasses
        `__getattr__` entirely; nothing short of `__slots__` plus a metaclass
        would stop it, and anyone writing `vars()` on a variant type has left
        the road on purpose.
      * `bool(read)` is `True` for BOTH variants, so `if read:` looks like a
        check and is not one. Defining `__bool__` to be False here would make
        `if read:` "work" - and that is exactly the wrong fix: it would turn a
        line that reads like a guard into one that IS a guard by accident,
        which is how `.get(k, default)` got into this codebase in the first
        place. Use `.match(...)`. There is no falsy variant on purpose.
    """

    kind = "no_data"

    def __init__(self, reason):
        self.reason = reason
        self.words = f"no price data - {reason}"


def arbitrage_allowed(read: PriceRead) -> bool:
    """§11.4, stated once, as a function so no caller has to remember it.

    Not the arbitrage strategy - that is OPT-04. This is only the veto, which
    belongs with the type that expresses the condition.
    """
    return read.match(series=lambda _r: True, no_data=lambda _r: False)


def plan_is_provisional(read: PriceRead) -> bool:
    """§8.3 / §11.4: provisional if there is no data at all, or if ANY point in
    the series is itself provisional. Derived from the points, never asserted."""
    return read.match(
        series=lambda r: any(p.provisional for p in r.points),
        no_data=lambda _r: True)


# --------------------------------------------------------------------------- #
# Options: what a plugin may see, and what it may not
# --------------------------------------------------------------------------- #

# ⚠️ CFG-03, AND HOW THIS TICKET AVOIDS WIDENING IT. Option defaults in this
# add-on already live in three places - config.yaml, main.py's `o.get(k, X)`
# fallbacks, and a dataclass default - held apart only by a tripwire in
# test_validate.py. This module adds ONE more location, not another three:
#
#   * every price option's fallback is here, and only here;
#   * validate.py imports this dict rather than restating any number;
#   * main.py gets no `o.get("price_...", X)` fallbacks at all;
#   * there is no dataclass carrying a default copy of any of these;
#   * test_price.py READS config.yaml and asserts the shipped default equals
#     this dict, key for key, so the two cannot drift silently.
#
# config.yaml remains the source of truth. This is the single in-code mirror,
# pinned to it by a check, which is the `validate.py` pattern rather than a
# fourth freehand copy.
PRICE_OPTION_DEFAULTS = {
    "price_provider": "off",
    "price_markup_eur_kwh": 0.03,      # §14 tariffs.energy_price.markup_eur_kwh
    "price_taxes_eur_kwh": 0.06,       # §14 tariffs.energy_price.taxes_eur_kwh
    "export_price_eur_kwh": 0.04,      # §14 tariffs.export_price_eur_kwh (AC4)
    "price_csv_path": "/share/goodwe_prices.csv",
    "price_feed_url": "",
    "price_horizon_h": 48,
    "price_timeout_s": 10,
    # TARIFF-03. §14 `tariffs.static_profile: null`. A JSON object describing a
    # static day/night or fully-fixed tariff; "" is §14's `null`. Parsed and
    # validated by app/static_tariff.py, refused at startup by validate.py.
    #
    # ⚠️ IT LIVES HERE RATHER THAN IN static_tariff.py FOR CFG-03's SAKE. This
    # dict is the ONE in-code mirror of config.yaml, pinned to it by
    # test_price.py section 1. A second defaults dict in the new module would be
    # a fourth place option defaults live, which is exactly the open ticket.
    "static_profile": "",
    # OPT-04. §10.1 priority 3, and §14's `optimizer.cycle_cost_threshold_eur`.
    #
    # ⚠️ THEY LIVE HERE FOR CFG-03's SAKE, exactly as `static_profile` does, and
    # FCST-03's note above `WIDEN_FACTOR` names this as the place they would
    # land. `arbitrage_soc_floor` / `arbitrage_soc_ceiling` are the reserve
    # arbitrage may not spend and the headroom it may not overfill; they are
    # NOT the maintenance SoC keys, which describe a balancing cycle rather
    # than a trading reserve and would silently couple two unrelated policies.
    #
    # ⚠️ AND `cycle_cost_threshold_eur` IS 0.10 BECAUSE §10.1's NAME FOR IT SAYS
    # SO, NOT BECAUSE IT MAKES ANYTHING TRADE. On realistic Belgian day/night
    # numbers the static spread is ~0.08 EUR/kWh, so a static-tariff site does
    # NOT trade at this default. That is a reported result (test_arbitrage.py
    # section 7), and whether 0.10 - or FCST-03's 2.5/1.5 - is right is one
    # product decision that belongs to the owner. No number here was chosen to
    # make arbitrage happen.
    "arbitrage_enabled": False,
    "cycle_cost_threshold_eur": 0.10,
    "arbitrage_soc_floor": 20,
    "arbitrage_soc_ceiling": 95,
}

# ⚠️ The keys the FRAMEWORK owns. A plugin that reads one of these is a plugin
# that can double-count, so `PluginOptions` refuses them by name. Trap 1.
FRAMEWORK_KEYS = frozenset({
    "price_markup_eur_kwh", "price_taxes_eur_kwh", "export_price_eur_kwh",
})


class PluginOptions:
    """The options dict, minus the keys only the framework may read.

    Reading `price_markup_eur_kwh` here raises `FrameworkOption` rather than
    returning 0.03.

    ⚠️ THIS IS ONE OF THE TWO DOORS, NOT THE ONLY ONE. `_o` is reachable, and
    `price_option()` and `PRICE_OPTION_DEFAULTS` are public names in this
    module. This class alone is a convention; it is `register()`'s
    `code_leaks()` scan that makes it an inability. Do not weaken either half
    on the strength of the other.
    """

    def __init__(self, opts: dict):
        self._o = dict(opts or {})

    def get(self, key, default=None):
        if key in FRAMEWORK_KEYS:
            raise FrameworkOption(
                f"{key!r} belongs to the framework, not to a plugin. A plugin "
                f"returns RawPoints carrying what its SOURCE published and "
                f"declares capabilities().sources; assemble() adds the missing "
                f"components exactly once. If a plugin could read this number "
                f"it could add it a second time, and the resulting price would "
                f"be wrong by one markup with nothing anywhere reporting it.")
        v = self._o.get(key, None)
        if v is None or v == "":
            return PRICE_OPTION_DEFAULTS.get(key, default)
        return v

    def __getitem__(self, key):
        return self.get(key)

    def __contains__(self, key):
        return key not in FRAMEWORK_KEYS and key in self._o


def price_option(opts: dict, key: str):
    """A framework-side option read. One place, defaults from one dict."""
    if key not in PRICE_OPTION_DEFAULTS:
        raise KeyError(f"{key!r} is not a price option; add it to "
                       f"PRICE_OPTION_DEFAULTS and to config.yaml together.")
    v = (opts or {}).get(key, None)
    if v is None or v == "":
        return PRICE_OPTION_DEFAULTS[key]
    return v


def provider_id(opts: dict) -> str:
    """The selected plugin id, normalised. `"off"` when none is selected.

    ⚠️ ONE SPELLING OF THIS EXPRESSION, AND TARIFF-10 IS WHY IT IS A FUNCTION.
    `str(price_option(opts, "price_provider") or "off").strip()` was written out
    twice - in `load_prices()` below and in `static_tariff.resolve_tariff()` -
    and main.py's startup gate would have been a third. "Is a provider
    configured" is now a DECISION the add-on takes (it is what starts the
    background refresh), and a decision spelled out at three call sites is three
    places to disagree about whether `"  off  "` or `None` counts.
    ⚠️ `validate.py` deliberately keeps its own reader: it validates the RAW
    options blob, including one the Supervisor never saw, so it must not route
    through `price_option()`'s defaults. Note what this does NOT do: it does not
    decide
    whether the id names a shipped plugin - `load_prices()` and validate.py's
    rule 7 own that, and answering "unknown" here would turn a typo into
    "no provider" instead of into a startup refusal.
    """
    return str(price_option(opts, "price_provider") or "off").strip()


def provider_configured(opts: dict) -> bool:
    """Has the operator selected a price feed at all? (TARIFF-10)

    THE DEFECT THIS PREVENTS: the background tariff refresh used to start only
    when `arbitrage_enabled` was on, which ships `false` - so a site with a
    provider configured fetched NOTHING, ever. `/data/price_history.json` was
    never created, the *Price history* status row read `empty` forever, and
    TARIFF-09's short-horizon warning could not fire because no fetch happened
    to warn about. Measured over a 40 h two-instance soak: zero feed requests
    and `present: false` across all 802 samples on the shipped default, against
    a resolved plan on the same feed with arbitrage armed.

    ⚠️ THIS IS NOT A TRADING SWITCH AND MUST NEVER BECOME ONE. It answers
    "fetch prices?"; `arbitrage.enabled()` answers "move energy on them?", and
    §10.1's priority 3 is gated on the second alone. See
    `main.tariff_refresh_wanted()` for the one place the two meet.
    """
    return provider_id(opts) not in ("", "off")


def export_price_eur_kwh(opts: dict) -> float:
    """AC4 / §9.3: the injection price is a SEPARATE SCALAR, not a second series.

    In Belgium it is often near zero, which by itself favours self-consumption
    over export. It is deliberately not routed through the plugin framework -
    but ⚠️ NOT because nothing dynamic exists to route. That claim stood here
    until TARIFF-07 and it was FALSE:

      * Eneco publishes an injection formula on the same tariff card as its
        consumption formula - `0,1 x BELPEX-H - 1,188` - identical on the
        January and the September 2026 cards;
      * ENGIE publishes one in the same two-coefficient shape,
        `(B x eSpot_15) - A`.

    So a supplier DOES publish a dynamic injection curve. What is true is
    narrower and is the actual reason this is a scalar: an injection curve is
    the SAME `B x spot - A` shape as the consumption curve but carries ITS OWN
    coefficients and its own VAT treatment - injection is VAT-exempt where
    consumption is at 6% - so it is a second per-contract formula, not a second
    copy of the first. Nothing in this add-on can evaluate either one yet
    (`generic_dayahead` cannot express `B x spot + A` at all; the formula
    provider is a separate card). Until it can, this scalar is a SIMPLIFICATION
    an operator enters by hand - an average of what they are actually paid -
    and not a reflection of the market. When the formula provider lands, an
    injection curve gets a `PriceProvider` of its own and this scalar becomes
    its FIXED-product default; the seam is already the right shape for that.

    ⚠️ It is NOT reachable from a plugin: `export_price_eur_kwh` is in
    FRAMEWORK_KEYS, so a consumption plugin cannot quietly fold an injection
    price into a consumption price.
    """
    return float(price_option(opts, "export_price_eur_kwh"))


# --------------------------------------------------------------------------- #
# The plugin interface
# --------------------------------------------------------------------------- #

MAX_PLAUSIBLE_EUR_KWH = 10.0    # see _validate_raw


class PriceProvider(ABC):
    """One supplier. Declares what it is, then returns what its source said.

    ⚠️ `fetch()` DOES NOT RETURN A PRICE. It returns `RawPoint`s carrying the
    published figure. The all-in arithmetic happens in `assemble()`, once, in
    the framework, using `capabilities().sources`. See trap 1.
    """

    id = "?"

    @classmethod
    @abstractmethod
    def capabilities(cls) -> ProviderCapabilities:
        ...

    @abstractmethod
    async def fetch(self, opts, since: datetime, until: datetime,
                    transport) -> list:
        """Return `RawPoint`s, or raise `SourceUnavailable(reason)`.

        `transport(url, timeout_s) -> bytes` is injected so the checks never
        touch a network and `manual_csv` never needs one. `opts` is a
        `PluginOptions`, so the markup and taxes are not reachable from here.

        ⚠️ `since`/`until` ARE THE RANGE THE CALLER WANTS, AND A PLUGIN THAT
        IGNORES THEM GETS WHATEVER ITS FEED DEFAULTS TO (TARIFF-09). That is
        not a small difference: `generic_dayahead` ignored both, the public
        EPEX feed answered ONE Brussels day, and §8.3 then dropped every hour
        past today from the trade set as provisional - so the optimiser never
        saw tomorrow as firm, at any hour of any day, and nothing reported it.

        This promise is NOT mechanically enforced. A rule obliging every
        plugin to pass a range would be satisfied by an INVENTED parameter
        name - the parser-graded-against-its-own-guess defect, moved from the
        response to the request, and the defect TARIFF-08 found on the
        response side. What exists instead is a roster pin in test_price.py:
        which plugins consume the range and which do not is asserted as a
        literal, so a third supplier forces the decision rather than
        inheriting the silence.
        """


# ⚠️ THE NAMES A PLUGIN'S OWN CODE MAY NOT MENTION. FRAMEWORK_KEYS is the
# obvious half; the other three are the routes review of PR #16 actually used to
# write a double-counting plugin after `PluginOptions` was in place:
#
#   `_o`                     - PluginOptions' private dict, reachable as
#                              `opts._o["price_markup_eur_kwh"]`.
#   `price_option`           - a PUBLIC framework helper that takes the raw
#                              options dict and answers with the markup.
#   `PRICE_OPTION_DEFAULTS`  - a PUBLIC module constant. This is the one that
#                              decided the review: no private access, no options
#                              dict, no helper - just a dict in the module every
#                              plugin already imports for `register`.
#
# Any of them turns +0.088 into +0.119 on a wholesale feed with nothing
# anywhere reporting it.
BANNED_IN_PLUGIN_CODE = tuple(sorted(FRAMEWORK_KEYS)) + (
    "PRICE_OPTION_DEFAULTS", "price_option", "_o")


def code_leaks(cls) -> list:
    """Framework-owned names appearing in a plugin's own CODE. Empty is clean.

    ⚠️ DOCSTRINGS ARE STRIPPED BEFORE THE SEARCH, AND THAT IS NOT COSMETIC.
    Every shipped plugin's docstring is REQUIRED to name the components it
    sources, so `generic_dayahead` says "markup" in prose and would
    false-positive against `price_markup_eur_kwh` on a raw text scan. The
    check that fires on the documentation it also mandates is the check that
    gets deleted. So the
    class is parsed, every docstring node is dropped, and the search runs on
    `ast.unparse` of what is left - i.e. on code, which is the only thing that
    can actually add a number twice.

    ⚠️ FAIL-OPEN ON MISSING SOURCE, DELIBERATELY. `inspect.getsource` needs the
    .py file, which the add-on image has (Dockerfile COPYs `app/`) but a frozen
    or zipapp build would not. Refusing to start a battery controller because a
    STATIC ANALYSIS could not run would trade a real outage for a hypothetical
    accounting error, so an unreadable source returns `[]` and logs. CI proves
    the scan really runs: test_price.py asserts source was found for every
    shipped plugin, so a build that silently lost it is a red check, not a
    quietly disarmed guard.
    """
    try:
        src = dedent(inspect.getsource(cls))
        tree = ast.parse(src)
    except (OSError, TypeError, IndentationError, SyntaxError) as e:
        _LOG.warning("could not read %s's source for the framework-key scan "
                     "(%s); the double-counting guard is NOT enforced for it",
                     getattr(cls, "id", cls), e)
        return []
    for node in ast.walk(tree):
        body = getattr(node, "body", None)
        if isinstance(body, list) and body and isinstance(body[0], ast.Expr)                 and isinstance(body[0].value, ast.Constant)                 and isinstance(body[0].value.value, str):
            body.pop(0)
    code = ast.unparse(tree)
    return sorted({name for name in BANNED_IN_PLUGIN_CODE if name in code})


# --------------------------------------------------------------------------- #
# TARIFF-07: the ids that USED to resolve here, and what an operator on one of
# them has actually been running.
#
# ⚠️ THE DEFECT THIS EXISTS TO PREVENT: A RETIRED ID READ AS A TYPO. Without
# this table `price_provider: "eneco_eddy"` reaches validate.py's unknown-id
# branch and is refused as "not a shipped plugin" - true, and useless. The
# operator did not mistype it; it WAS a shipped plugin, they chose it on
# purpose, and the add-on has accepted it every startup since. What they need
# told is not "that id is wrong" but "that id never fetched a price, and here
# is what you have been running instead" - because §11.4 turned every failed
# fetch into a silent fall back to `static_profile`, so NOTHING they can see
# has ever differed from a working provider that finds no cheap hour.
#
# ⚠️ AND WHY THERE IS NO SILENT ALIAS. Mapping these to `generic_dayahead`
# would be the friendlier-looking option and it is the wrong one: these three
# published RETAIL prices (`B x spot + A`, per contract) and `generic_dayahead`
# reads WHOLESALE. Aliasing needs the site's own two coefficients, which nobody
# has - they live behind the customer's klantenzone login, and there is no
# regulator index to read them from (CREG's tariff-card pages 404; VREG was
# unreachable; every coefficient found in research came from a supplier's own
# PDF). An alias would therefore have to INVENT the numbers it prices a battery
# against, which is the same class of defect as the parser graded against its
# own guess - just one layer further from anything that reports.
#
# Each value is the ID-SPECIFIC half of the refusal: the endpoint, what it was
# observed to answer on 2026-09-13, and why no replacement plugin can be built.
# validate.py owns the common framing. Verified by hand before deletion, not
# taken on report - see DECISIONS/TARIFF-07-withdrawing-three-plugins.md.
# --------------------------------------------------------------------------- #
RETIRED_PROVIDERS: dict = {
    "engie_dynamic": (
        "its endpoint https://api.engie.be/dynamic/be does not exist - the "
        "gateway there answers 404 with Kong's \"no Route matched with those "
        "values\". No public endpoint could replace it: ENGIE's dynamic price "
        "is (B x eSpot_15) - A with PER-CONTRACT coefficients that live behind "
        "your klantenzone login, so no unauthenticated URL can return YOUR "
        "price"),
    "totalenergies_dynamic": (
        "its endpoint https://api.totalenergies.be/prices/dynamic does not "
        "exist - the TLS handshake fails outright because that hostname is "
        "absent from the certificate the server presents. The coefficients are "
        "per-contract there too"),
    "eneco_eddy": (
        "its endpoint https://api.eneco.be/eddy/prices does not exist - it "
        "answers 404 with an empty body - and \"Eddy\" is not an Eneco "
        "product at all; their dynamic product is \"Zon & Wind Dynamisch\", "
        "which is indexed to Belpex/EPEX DAY-AHEAD and whose coefficients are "
        "likewise per-contract"),
}


PROVIDERS: dict = {}


def register(cls):
    """Register a provider by its own declared id. Decorator.

    ⚠️ THE ONLY NAME->CLASS MAPPING IN THE SYSTEM. AC2 is "adding a plugin needs
    no optimiser changes", and this is where that is bought: config resolves an
    id here, the optimiser is handed a `_Series` carrying a `PriceProduct`, and
    nothing downstream ever sees the id except to print it.
    """
    if cls.id in PROVIDERS:
        raise ValueError(f"duplicate provider id {cls.id!r}")
    if cls.id in RETIRED_PROVIDERS:
        # ⚠️ AT IMPORT, NOT AT FETCH, and for the same reason the leak check is.
        # A retired id that is registered again is the one state in which
        # validate.py's retired-id refusal goes quiet WITHOUT anyone deciding
        # it should: `pid in PROVIDERS` becomes true, the named message is
        # never reached, and the operator is back to a provider that looks
        # like it works. Re-shipping an id is legitimate - the endpoint could
        # become real - but it is a DECISION, and taking the id out of
        # RETIRED_PROVIDERS in the same commit is what records it.
        raise ValueError(
            f"{cls.id!r} is in RETIRED_PROVIDERS and cannot also be a shipped "
            f"plugin. A site configured on it is refused at startup with a "
            f"message naming the id; registering it silently takes that "
            f"message away. If this id really ships again, remove it from "
            f"RETIRED_PROVIDERS in the same commit - and say in the CHANGELOG "
            f"what changed about the endpoint.")
    cls.capabilities()          # fail at import, not at 15:00 on fetch day
    leaks = code_leaks(cls)
    if leaks:
        # ⚠️ AT IMPORT, NOT AT FETCH. A plugin that can reach the markup is a
        # plugin that can bill wrong for a month before anyone notices, so it
        # does not get to exist - the add-on fails to start and names the line.
        raise FrameworkOption(
            f"{cls.id!r} mentions {', '.join(leaks)} in its own code. A plugin "
            f"returns what its SOURCE published and declares "
            f"capabilities().sources; the framework adds the complement, once. "
            f"Reaching any of {BANNED_IN_PLUGIN_CODE} lets it be added twice, "
            f"and a price that is one markup high is invisible in every log "
            f"this add-on writes. Remove the reference; if the plugin genuinely "
            f"needs a per-site number, add a NON-framework option key for it.")
    PROVIDERS[cls.id] = cls
    return cls


def _validate_raw(points, caps):
    """Reject at the boundary, so a malformed feed cannot become a bad decision.

    ⚠️ NEGATIVE PRICES ARE VALID AND ARE NOT CLAMPED (§17 keeps a negative-price
    regression case; Belgian day-ahead goes negative on windy Sundays and those
    hours are the single best charging opportunity of the year). What is
    rejected is what cannot be a price at all: NaN, infinity, an implausible
    magnitude, a naive datetime, and duplicate or out-of-order starts.
    """
    if not isinstance(caps, ProviderCapabilities):
        raise SourceUnavailable("the plugin did not declare capabilities")
    if not points:
        raise SourceUnavailable("the source returned no points")
    seen = None
    for p in points:
        if not isinstance(p, RawPoint):
            raise SourceUnavailable(f"a plugin returned {type(p).__name__}, not "
                                    f"a RawPoint - see PriceProvider.fetch()")
        if p.start.tzinfo is None or p.start.utcoffset() is None:
            raise SourceUnavailable(f"point at {p.start!r} has no offset")
        if isinstance(p.eur_per_kwh, bool) or \
                not isinstance(p.eur_per_kwh, (int, float)) or \
                not math.isfinite(p.eur_per_kwh):
            raise SourceUnavailable(f"point at {p.start.isoformat()} has a "
                                    f"non-finite price {p.eur_per_kwh!r}")
        if abs(p.eur_per_kwh) > MAX_PLAUSIBLE_EUR_KWH:
            raise SourceUnavailable(
                f"point at {p.start.isoformat()} is {p.eur_per_kwh} EUR/kWh, "
                f"beyond +-{MAX_PLAUSIBLE_EUR_KWH} - the usual cause is a feed "
                f"in EUR/MWh being read as EUR/kWh, which is a factor of 1000 "
                f"and would make every hour look like an emergency")
        u = p.start.astimezone(timezone.utc)
        if seen is not None and u <= seen:
            raise SourceUnavailable(
                f"points are not strictly increasing at {u.isoformat()} - a "
                f"duplicated hour on a DST day is the usual cause, and keying "
                f"prices by a naive local hour is how it happens")
        seen = u


def assemble(caps: ProviderCapabilities, raw, markup: float,
             taxes: float) -> list:
    """RawPoints -> canonical all-in PricePoints. THE ONLY PLACE MARKUP IS ADDED.

    Adds exactly `caps.adds` - the complement of what the plugin declared its
    source already contains. A plugin that publishes an all-in figure gets
    nothing added; a raw EPEX figure gets both.
    """
    add = 0.0
    if Component.MARKUP in caps.adds:
        add += float(markup)
    if Component.TAXES in caps.adds:
        add += float(taxes)
    return [PricePoint(start=brussels(p.start),
                       eur_per_kwh=round(p.eur_per_kwh + add, 6),
                       provisional=bool(p.provisional))
            for p in raw]


# --------------------------------------------------------------------------- #
# Transport - one place, HTTPS only, host checked against the declaration
# --------------------------------------------------------------------------- #

async def https_get(url: str, timeout_s: float) -> bytes:
    """The default transport. §16: outbound is allow-listed and TLS'd.

    Imported lazily so `manual_csv` - the offline fallback - never needs aiohttp
    to be importable at all. AC3 says it works fully offline; that includes a
    machine where the HTTP stack is missing.
    """
    import aiohttp
    t = aiohttp.ClientTimeout(total=float(timeout_s))
    async with aiohttp.ClientSession(timeout=t) as s:
        async with s.get(url) as r:
            r.raise_for_status()
            return await r.read()


def check_egress(url: str, caps: ProviderCapabilities) -> str:
    """§16: a plugin may only reach a host it DECLARED. Enforced, not documented.

    Returns the host, so a caller that forgets to call this is not the same
    shape as one that did.

    ⚠️ THE SCHEME IS SPLIT OFF BEFORE THE HOST IS PARSED, AND THE ORDER MATTERS
    FOR THE TEST MORE THAN FOR THE CODE. The first version stripped a fixed
    `len("https://")` from the front, which on an `http://` URL ate two
    characters of the hostname instead - `http://api.energy-charts.info/p`
    became the host `pi.energy-charts.info`, which then failed the ALLOW-LIST.
    So a non-TLS feed was refused, the check named "plain http is refused"
    passed, and deleting the TLS rule entirely left it passing. Review of
    PR #16 found that by mutation: an assertion that does not test what it
    names, inside a security check - this project's own recurring defect. Now
    the scheme is parsed as a scheme, the host of an `http://` URL comes out
    correct and DECLARED, and the TLS rule is the only thing that can refuse
    it - which is what makes the check able to fail.
    """
    if not url:
        raise SourceUnavailable("no feed URL configured (price_feed_url)")
    scheme, sep, rest = url.partition("://")
    if not sep:
        raise SourceUnavailable(
            f"not an absolute URL, so no host can be checked against the "
            f"§16 allow-list: {url!r}")
    if scheme.lower() != "https":
        raise SourceUnavailable(
            f"refusing a non-TLS price feed: {url!r} uses {scheme!r}. §16 "
            f"allow-lists outbound destinations and SEC-01 requires TLS on "
            f"every one of them.")
    host = rest.split("/", 1)[0].split("?", 1)[0].split("@")[-1]
    host = host.split(":", 1)[0].lower()
    if host not in {h.lower() for h in caps.egress_hosts}:
        raise SourceUnavailable(
            f"{host!r} is not in this provider's declared egress_hosts "
            f"{caps.egress_hosts} - §16 allow-lists outbound destinations, and "
            f"a URL from the options file is not a declaration")
    return host


# --------------------------------------------------------------------------- #
# The shipped plugins.
#
# ⚠️ EACH DOCSTRING STATES WHICH COMPONENTS IT SOURCES AND WHICH THE FRAMEWORK
# ADDS, in the two fixed lines `SOURCES: ...` / `The framework ADDS: ...`. That
# is not decoration - it is the human-readable half of `capabilities().sources`,
# and test_price.py PARSES those lines out of the docstring and compares them
# with the declaration, so a plugin whose prose and whose code disagree about
# what its feed contains is a red check rather than a 3 c/kWh error nobody sees.
# --------------------------------------------------------------------------- #

# TARIFF-08: the COLUMNAR document, which is what the public EPEX feed really
# publishes.
#
# ⚠️ THIS IS THE ONLY PARSER LEFT, AND THE SHAPE IT DOES **NOT** READ IS WHY.
# Until TARIFF-07 a second function, `_points_from_json()`, read a row-per-point
# document: `{"data": [{...}, {...}]}`. NO FEED ANYBODY REACHED EVER PUBLISHED
# THAT SHAPE - it was invented alongside the three withdrawn plugins, and the
# only bodies ever fed to it were the fixtures written from the same guess. It
# went with them. `https://api.energy-charts.info/price?bzn=BE` publishes
# PARALLEL ARRAYS:
#
#     {"unix_seconds": [1788991200, 1788992100, ...],   <- UTC epoch seconds
#      "price":        [216.08,     209.90,     ...],   <- one per instant
#      "unit": "EUR / MWh", "license_info": "...", "deprecated": false}
#
# The row parser was pointed at that URL and asked for `doc["data"]`, so
# every fetch ended `SourceUnavailable: the feed carried no 'data' list` ->
# §11.4 -> static profile, arbitrage off, and nothing in the status page said
# why. Verified live 2026-09-10 against both fixtures under `scenarios/`.
#
# ⚠️ AND IT IS NOT ONE FUNCTION THAT SNIFFS FOR BOTH. A parser that decides
# which document it got is a parser that can decide wrong, and a wrong-but-
# parseable response then becomes a price. Two shapes, two functions, and each
# plugin names the one its own feed publishes.
#
# The columnar shape was already known in this repository: DATA-02's
# `scenarios/fetch_epex_be.py` has read `zip(d["unix_seconds"], d["price"])`
# since 2026-08-31. Only the SHIPPED path never learned it, because every
# fixture in the suite was built from the invented row shape - the parser was
# graded against its own assumption. See test_price.py section 2b.
# --------------------------------------------------------------------------- #

COLUMNAR_TIME_KEY = "unix_seconds"
COLUMNAR_PRICE_KEY = "price"
COLUMNAR_UNIT_KEY = "unit"


def _columnar_unit(text) -> str:
    """`"EUR / MWh"` -> `"eur/mwh"`. Spacing and case are not information."""
    return "".join(str(text).split()).lower()


def _points_from_columnar_json(body, expect_unit, scale,
                               provisional_default):
    """Parse a COLUMNAR price document: parallel `unix_seconds` / `price`.

    Defects this refuses, each of which has a way of becoming a number rather
    than an error:

      * ⚠️ UNEQUAL ARRAYS ARE `SourceUnavailable`, NOT A SHORT SERIES. `zip()`
        stops at the shorter array and says nothing, so a `price` list that
        arrived truncated would produce a REAL price series with its tail
        missing - and the tail of a Belgian day is the evening peak. A series
        that is missing the expensive hours is worse than no series: §11.4's
        no-data row switches arbitrage OFF, whereas a truncated series is
        traded on. Nothing downstream can tell the two apart, so the refusal
        has to happen here.
      * ⚠️ A UNIT THAT IS NOT THE DECLARED ONE IS REFUSED. `scale` is the
        caller's standing assumption about the unit; if the feed ever
        publishes EUR/kWh under the same keys, every price silently comes out
        a THOUSAND times too small. `_validate_raw()` cannot catch that -
        `MAX_PLAUSIBLE_EUR_KWH` only bounds prices that are too LARGE.
        DATA-02's fetcher refuses on the same field for the same reason.
      * ⚠️ `unix_seconds` ARE UTC INSTANTS, SAID SO EXPLICITLY.
        `datetime.fromtimestamp(t, timezone.utc)`. Without that second
        argument CPython returns a NAIVE datetime in the HOST's zone, and a
        naive datetime is a wall clock reading plus an assumption about whose
        clock it was. `brussels()` and `_validate_raw()` both refuse one, so
        the mistake surfaces as a refused fetch rather than as a mispriced
        day - but they refuse ALL naive datetimes and can say nothing about
        whether this one was the right instant. That part is decided here.

        §9.3's rendering AT THE BRUSSELS OFFSET is `assemble()`'s, once, for
        every plugin - `brussels(p.start)` there is the single line that does
        it, and calling it here as well would be a second copy of a rule that
        already has an owner, indistinguishable from a no-op under mutation.
        A plugin's job is the right INSTANT; the framework's is the rendering.

    A `null` price is DROPPED and logged, not fatal, and that is a different
    judgement from the one above: an unequal pair of arrays means the pairing
    of instant to price is unreliable, whereas a null means the source stated
    it published no price for that one instant. A dropped instant leaves a
    hole, and `_Series.at()` already answers a hole with `Absence` - §11.4 for
    that hour. `scenarios/fetch_epex_be.py` dropped 0 nulls over the 12 months
    to 2026-08-31, so this is a shape the source allows rather than one it uses.
    """
    try:
        text = body.decode("utf-8") if isinstance(body, (bytes, bytearray)) \
            else str(body)
        doc = json.loads(text)
    except (ValueError, UnicodeDecodeError) as e:
        raise SourceUnavailable(f"the feed did not parse as JSON: {e}")
    if not isinstance(doc, dict):
        raise SourceUnavailable(
            f"the feed is a {type(doc).__name__}, not a columnar object with "
            f"{COLUMNAR_TIME_KEY!r} and {COLUMNAR_PRICE_KEY!r} arrays")
    times = doc.get(COLUMNAR_TIME_KEY)
    prices = doc.get(COLUMNAR_PRICE_KEY)
    for key, col in ((COLUMNAR_TIME_KEY, times), (COLUMNAR_PRICE_KEY, prices)):
        if not isinstance(col, list):
            raise SourceUnavailable(
                f"the feed carried no {key!r} array; this provider reads the "
                f"columnar shape (parallel {COLUMNAR_TIME_KEY}/"
                f"{COLUMNAR_PRICE_KEY} arrays), and a row-per-point document "
                f"is a different feed, not this one with a field missing")
    if len(times) != len(prices):
        raise SourceUnavailable(
            f"{COLUMNAR_TIME_KEY} has {len(times)} entries and "
            f"{COLUMNAR_PRICE_KEY} has {len(prices)}; refusing to zip them. "
            f"The short array would truncate the series silently, and a price "
            f"series missing its last hours is one missing the evening peak - "
            f"traded on as if complete, where no data at all would have "
            f"switched arbitrage off (§11.4)")
    unit = _columnar_unit(doc.get(COLUMNAR_UNIT_KEY, ""))
    if unit != _columnar_unit(expect_unit):
        raise SourceUnavailable(
            f"the feed reports {doc.get(COLUMNAR_UNIT_KEY)!r}, not "
            f"{expect_unit!r}. This provider converts with a fixed factor of "
            f"{scale}, so a changed unit is a factor of 1000 on every price "
            f"with nothing downstream able to notice it")
    if doc.get("deprecated"):
        # Not a refusal: today's prices are still prices. But §11.4 makes a
        # dead feed look exactly like a quiet one, so the warning that the
        # source is going away has to be emitted while it still works.
        _LOG.warning("the price feed reports deprecated=%r; it still answers, "
                     "but this endpoint is scheduled to stop",
                     doc.get("deprecated"))
    out = []
    dropped = []
    for i, (t, v) in enumerate(zip(times, prices)):
        if v is None:
            dropped.append(i)
            continue
        if isinstance(t, bool) or not isinstance(t, (int, float)) or \
                not math.isfinite(t):
            raise SourceUnavailable(
                f"{COLUMNAR_TIME_KEY}[{i}] is {t!r}, not epoch seconds")
        if isinstance(v, bool) or not isinstance(v, (int, float)) or \
                not math.isfinite(v):
            raise SourceUnavailable(
                f"{COLUMNAR_PRICE_KEY}[{i}] is {v!r}, which is not a price")
        start = datetime.fromtimestamp(float(t), timezone.utc)
        out.append(RawPoint(start=start, eur_per_kwh=float(v) * scale,
                            provisional=bool(provisional_default)))
    if dropped:
        _LOG.warning("the feed published no price for %d of %d instants "
                     "(index %s); those instants are left OUT of the series, "
                     "never zero-filled - a missing hour is not a free hour",
                     len(dropped), len(times),
                     ", ".join(str(i) for i in dropped[:5])
                     + (", ..." if len(dropped) > 5 else ""))
    if not out:
        raise SourceUnavailable(
            f"the feed carried {len(times)} instants and a price for none of "
            f"them")
    return out


@register
class GenericDayAhead(PriceProvider):
    """Public EPEX BE day-ahead feed, COLUMNAR, in EUR/MWh.

    SOURCES: energy
    The framework ADDS: markup, taxes

    This is the plugin the double-counting trap is about. The published figure
    is the wholesale clearing price and nothing else, so both user-entered
    components have to be added to it - and because this plugin cannot read
    either number, it cannot be the one that adds them twice.

    ⚠️ IT READS THE COLUMNAR SHAPE, AND SAYS SO IN ITS OWN `fetch()`. Until
    TARIFF-08 it called the row-per-point parser (withdrawn by TARIFF-07) and
    asked
    `api.energy-charts.info` for a `"data"` list it has never published, so
    every fetch this provider ever made ended in `SourceUnavailable` and §11.4
    turned that into a silent fall back to the static profile. See
    `_points_from_columnar_json()` for the shape and for why there are two
    parsers rather than one that guesses.

    ⚠️ THE FEED IS QUARTER-HOURLY SINCE BELGIUM'S 15-MINUTE MTU (2025-10-01):
    96 points a day at 900 s, not 24 at 3600 s. Nothing here assumes either -
    the instants come from the document - and both granularities are replayed
    from real captured bodies in test_price.py section 2b.
    """

    id = "generic_dayahead"

    @classmethod
    def capabilities(cls):
        return ProviderCapabilities(
            product=PriceProduct.DAY_AHEAD,
            sources=frozenset({Component.ENERGY}),
            egress_hosts=("api.energy-charts.info", "web-api.tp.entsoe.eu"))

    FEED_BASE = "https://api.energy-charts.info/price"
    BZN = "BE"

    async def fetch(self, opts, since, until, transport):
        """Request the range the CALLER asked for, on the Brussels calendar.

        ⚠️ THE ARGUMENTS ARE USED. Until TARIFF-09 this method took `since` and
        `until` and asked for the bare `?bzn=BE`, which answers ONE Brussels
        day - so a 48 h horizon was 15 firm hours and 33 carried at 09:00, and
        1 firm and 47 carried at 23:00, with the whole trade set inside a
        single calendar day at every hour of every day. See
        `brussels_date_range()` for the measurement and for why the calendar
        is Brussels rather than UTC.

        ⚠️ AN OVERRIDE IS THE EXACT URL AND THIS METHOD APPENDS NOTHING TO IT,
        WHICH COSTS THE OVERRIDING SITE THE RANGE. That is a decision, not an
        oversight, and the sentence below is why it is not a footnote:

          * `price_feed_url` says "override the plugin's default feed", and an
            override that is silently rewritten is not one. Merging parameters
            into an operator's query string means parsing and re-serialising a
            URL they wrote - which reorders, de-duplicates and re-escapes it,
            and can hand the server a request nobody typed.
          * `start`/`end` are ENERGY-CHARTS' spelling, and this provider
            DECLARES A SECOND HOST: `web-api.tp.entsoe.eu`. Nothing here has
            verified that host's day-ahead parameters and nothing here should
            guess them - but they are not `start`/`end`, so an override to the
            other DECLARED host is exactly the case where appending our
            vocabulary adds two parameters that host ignores and yields the
            short horizon anyway, with nothing saying so.
          * and blind appending is refused outright: `dated_feed_url()` will
            not add a query string to a base that has one.

        So the cost is real and it is REPORTED, at fetch time, on the shipped
        logger, naming the horizon consequence rather than the substitution.
        §11.4 is what makes a price defect quiet; an operator who has taken the
        short horizon has to be able to find out from the log that they did.
        """
        override = str(opts.get("price_feed_url") or "").strip()
        if override:
            url = override
            _LOG.warning(
                "price_feed_url is set, so %s is requesting %s exactly as "
                "given and is adding NO date range to it. The public feed "
                "answers a single Brussels day to a request that names no "
                "range, and every hour past that day is then carried forward "
                "from yesterday's profile and flagged provisional - §8.3 "
                "drops provisional hours from the trade set, so arbitrage "
                "would never see tomorrow as firm. Clear price_feed_url to "
                "let this plugin ask for the %s..%s it was given.",
                self.id, url, *(d.isoformat()
                                for d in brussels_date_range(since, until)))
        else:
            first, last = brussels_date_range(since, until)
            url = dated_feed_url(self.FEED_BASE, bzn=self.BZN,
                                 start=first.isoformat(),
                                 end=last.isoformat())
        check_egress(url, self.capabilities())
        body = await transport(url, float(opts.get("price_timeout_s")))
        return _points_from_columnar_json(body, expect_unit="EUR / MWh",
                                          scale=0.001,
                                          provisional_default=False)


@register
class ManualCsv(PriceProvider):
    """A file on disk. AC3: the documented fallback, fully offline.

    SOURCES: energy, markup, taxes
    The framework ADDS: nothing

    ⚠️ THE CSV IS ALL-IN, AND DOCS.md says so. It has to be: this plugin has no
    way to know what the numbers a human typed already contain, and guessing
    would be the double-counting defect with a friendlier face. `load_prices()`
    warns when markup/taxes are configured non-zero under a plugin that sources
    them, so the ignored numbers are visible rather than silently discarded.

    ⚠️ NO TRANSPORT, NO aiohttp, NO NETWORK, NO tz DATABASE. It is the escape
    hatch for a site with no internet and the ingest path the TEST-01 replay
    harness will share, so it must not acquire a dependency on anything that
    needs one. `transport` is accepted and deliberately unused.

    Format - header required, two columns or three:

        start,eur_per_kwh,provisional
        2026-01-15T00:00:00+01:00,0.1834,false
    """

    id = "manual_csv"

    @classmethod
    def capabilities(cls):
        return ProviderCapabilities(
            product=PriceProduct.DAY_AHEAD,
            sources=ALL_IN,
            egress_hosts=())          # declared empty: this one reaches nothing

    async def fetch(self, opts, since, until, transport):
        path = opts.get("price_csv_path")
        try:
            with open(path, "r", encoding="utf-8") as fh:
                text = fh.read()
        except OSError as e:
            raise SourceUnavailable(f"cannot read {path}: {e}")
        return self.parse(text)

    @staticmethod
    def parse(text: str) -> list:
        """Split out so the offline check can drive it without a filesystem."""
        rdr = csv.DictReader(io.StringIO(text))
        if not rdr.fieldnames or "start" not in rdr.fieldnames or \
                "eur_per_kwh" not in rdr.fieldnames:
            raise SourceUnavailable(
                f"the CSV needs a header with at least 'start' and "
                f"'eur_per_kwh'; found {rdr.fieldnames!r}")
        out = []
        for n, row in enumerate(rdr, start=2):
            try:
                start = datetime.fromisoformat((row["start"] or "").strip())
                value = float((row["eur_per_kwh"] or "").strip())
            except (TypeError, ValueError) as e:
                raise SourceUnavailable(f"CSV line {n}: {e}")
            if start.tzinfo is None:
                raise SourceUnavailable(
                    f"CSV line {n}: {row['start']!r} has no UTC offset. Write "
                    f"'2026-01-15T00:00:00+01:00', not '2026-01-15 00:00' - a "
                    f"local wall clock is ambiguous for one hour every October.")
            flag = (row.get("provisional") or "false").strip().lower()
            out.append(RawPoint(start=start, eur_per_kwh=value,
                                provisional=flag in ("1", "true", "yes")))
        return out


# --------------------------------------------------------------------------- #
# The one entry point
# --------------------------------------------------------------------------- #

async def load_prices(opts: dict, since: datetime, until: datetime,
                      transport=None) -> PriceRead:
    """Resolve the configured provider, fetch, assemble all-in, return a read.

    ⚠️ RETURNS, NEVER RAISES, for every failure a site can actually have. §11.4
    makes "no price data" a degradation row, and a degradation row that arrives
    as a traceback into the 30 s planning tick is a controller that stops
    regulating a battery because a supplier's TLS certificate expired.

    The one exception is `FrameworkOption`, which is re-raised: that is not a
    site condition, it is a plugin trying to read the markup, and turning it
    into "no price data" would hide the exact defect trap 1 exists to catch.
    """
    pid = provider_id(opts)
    if not provider_configured(opts):
        return PriceRead.no_data(
            "no price provider configured (price_provider is 'off'); "
            "§11.4: static profile, plan provisional, no arbitrage")
    cls = PROVIDERS.get(pid)
    if cls is None:
        return PriceRead.no_data(
            f"unknown price provider {pid!r}; shipped: "
            f"{', '.join(sorted(PROVIDERS))}")

    caps = cls.capabilities()
    markup = float(price_option(opts, "price_markup_eur_kwh"))
    taxes = float(price_option(opts, "price_taxes_eur_kwh"))

    # ⚠️ Not a refusal, but not silent either. A plugin whose feed is already
    # all-in ignores these two numbers, and an operator who typed them has every
    # reason to believe they took effect.
    ignored = [n for c, n, v in ((Component.MARKUP, "price_markup_eur_kwh", markup),
                                 (Component.TAXES, "price_taxes_eur_kwh", taxes))
               if c not in caps.adds and v]
    if ignored:
        _LOG.warning("%s publishes a figure that already contains %s, so the "
                     "configured %s ignored - adding them would double-count",
                     pid,
                     "/".join(sorted(c.value for c in
                                     caps.sources - {Component.ENERGY})),
                     " and ".join(ignored) +
                     (" is" if len(ignored) == 1 else " are"))

    if transport is None:
        transport = https_get
    try:
        raw = await cls().fetch(PluginOptions(opts), since, until, transport)
        _validate_raw(raw, caps)
        points = assemble(caps, raw, markup, taxes)
    except FrameworkOption:
        raise                       # a plugin bug, not a site condition
    except SourceUnavailable as e:
        return PriceRead.no_data(f"{pid}: {e}")
    except Exception as e:          # noqa: BLE001 - see the docstring
        return PriceRead.no_data(f"{pid}: {type(e).__name__}: {e}")

    return PriceRead.series(points, caps.product, pid,
                            datetime.now(timezone.utc))