"""TARIFF-02 - price provider plugins, and the one canonical shape they all emit. WHERE THIS LIVES, AND WHY IT IS NOT WHERE THE SPEC SAYS. specs.md §2.2 puts this at `ems/ingest/prices/`. That module layout does not exist and is not being built; the product owner's decision of 2026-08-30 is that everything ships in `goodwe_controller/app/`, next to arbiter.py, clamp.py, freeze.py and validate.py. The BEHAVIOUR specs.md §9.3 / §8.3 / §11.4 describes is unchanged and still authoritative - only the path moved. WHAT THIS IS FOR, in one sentence: so that a new Belgian supplier is a new class and one line in `PROVIDERS`, and the optimiser that eventually consumes prices never learns a supplier's name. THE CANONICAL FORMAT (§9.3), exactly three fields and no more: { "prices": [ { "start": "2026-01-15T00:00:00+01:00", "eur_per_kwh": 0.1834, "provisional": false } ] } `start` is tz-aware at the Brussels offset for that instant, the price is EUR per kWh and ALL-IN (energy + markup + taxes), and `provisional` is per point, not per series (§8.3: unknown hours are flagged, and no irreversible action may be taken on a provisional price). THE THREE THINGS THIS MODULE IS SHAPED BY ========================================= 1. DOUBLE-COUNTING MARKUP IS REFUSED, NOT DISCOURAGED - GIVEN A CORRECT DECLARATION. `generic_dayahead` reads a raw EPEX figure and the user's markup and taxes have to be added to it. Engie and TotalEnergies publish a figure that already HAS the markup in it. Add the configured markup to that one too and every arbitrage decision on that site is made against a price that is 3 c/kWh wrong, in a direction nothing reports. A comment saying "do not add markup here" is not a mechanism. So: * A plugin does not return a price. It returns `RawPoint`s carrying the figure ITS SOURCE PUBLISHED, and declares in `capabilities().sources` which of {ENERGY, MARKUP, TAXES} that figure already contains. * `assemble()` - the framework, never a plugin - adds exactly the components the plugin declared it did NOT source, from `price_markup_eur_kwh` / `price_taxes_eur_kwh`. * ⚠️ A PLUGIN CANNOT REACH THOSE TWO NUMBERS BY ANY ROUTE, and the word "any" was earned the hard way. `PluginOptions` raises `FrameworkOption` on them, which closes the documented path - but review of PR #16 wrote a double-counting plugin THREE more ways, one of them touching nothing but `PRICE_OPTION_DEFAULTS`, a public dict in the module every plugin must already import for `PriceProvider` and `register`. One closed door in a room with three is not a guarantee. So `register()` now also REFUSES ANY PLUGIN WHOSE OWN CODE MENTIONS A FRAMEWORK KEY - see `code_leaks()`. ⚠️ THE PRECISE SHAPE OF THE GUARANTEE, because an overclaim here is worse than no claim: **the framework adds the complement exactly once, GIVEN A CORRECT `sources` DECLARATION.** A plugin that declares `{ENERGY}` while its feed is really all-in still comes out one markup high, and NO CODE CAN CATCH THAT - the declaration is the plugin's unverifiable claim about somebody else's tariff. What is mechanised is that a plugin cannot obtain the numbers and cannot apply them; what remains human is whether it described its feed truthfully. The docstring/declaration check in test_price.py verifies that the plugin's two claims AGREE, not that either is true, and it is worth knowing which of those two things you are reading. 2. THE PRODUCT TYPE IS DECLARED, NOT INFERRED FROM THE PLUGIN'S NAME. §9.2 classes real-time / imbalance products separately from day-ahead ones and §8.3 says those are treated as high-volatility with widened arbitrage thresholds. If the optimiser had to write `if provider_id == ""` to know that, then adding a supplier would mean editing the optimiser - which is precisely what AC2 forbids. So `capabilities().product` is a `PriceProduct`, it rides along on the series, and `volatility_regime()` reads the enum. A plugin id appears nowhere in this file except in `PROVIDERS`, in `RETIRED_PROVIDERS`, and in the class that answers to it. ⚠️ NO SHIPPED PLUGIN DECLARES `PriceProduct.REALTIME` (TARIFF-07). All three shipped plugins are day-ahead, so `_WIDE`, `Uncertainty.REALTIME_PRODUCT` and `WIDEN_FACTOR`'s 2.5 currently have no shipped declarer. They are KEPT deliberately - see DECISIONS/TARIFF-07-withdrawing-three-plugins.md - and test_price.py section 10 keeps them live by declaring REALTIME from a class defined outside this file, which is AC2's own stimulus. 3. "NO PRICE DATA" IS A TYPE, NOT AN EMPTY LIST. §11.4: no price data -> fall back to the static profile, mark the plan provisional, NO ARBITRAGE. That is a different instruction from "prices exist and they are all zero", under which arbitrage is merely pointless rather than forbidden - and the two are indistinguishable if a failed fetch returns `[]`, or `None`, or a dict a caller `.get()`s a default out of. ⚠️ THIS REPO SPENT 2026-08-30 REMOVING EXACTLY THAT DEFECT. `sim/riglock.py` in the infra repo used to return a partial dict from `holder()`; TEST-04 rewrote it so a look at the lock is `observed / not_observed / gone` and only the observed variant carries fields. `_Outcome` below is that shape, deliberately, down to the fatal `.get()`. Read that file's `class Absence` block before changing anything here. Concretely: * `PriceRead.series(...)` REFUSES AN EMPTY POINT LIST. A series with no prices in it cannot be constructed, so "I have no prices" cannot arrive wearing the shape of an answer. * `PriceRead.no_data(reason)` carries `.reason` AND NOTHING ELSE. It has no `.points` to iterate, no `.product` to switch on, no `.at()` to call, and `.get("points", [])` raises. `min(read.points)` - the line that would silently make "no data" mean "everything is free, charge now" - does not run against it. * `.match(*, series, no_data)` takes both handlers as REQUIRED keyword arguments, so a caller who forgets the no-data case gets a TypeError at the call site rather than a fall-through. TIMEZONE, AND WHY THERE IS NO `zoneinfo` HERE ============================================= §9.3's example is `+01:00`, and Belgium runs CET/CEST. Two constraints: * The shipped image is Alpine with `py3-aiohttp` from apk and NO `tzdata` package (goodwe_controller/Dockerfile, and .gitea/ci/Dockerfile is the same base by design). `ZoneInfo("Europe/Brussels")` raises `ZoneInfoNotFoundError` there. A price module that only works on a machine with a tz database is a price module that fails on the client's Pi. * Fixed offsets have no fold. Every instant here is carried as UTC and RENDERED at the offset that applies to it, so the 02:00-03:00 hour that happens twice in October is two different instants that render as `02:00+02:00` and `02:00+01:00`. A naive-local-hour key collides those two and misprices an hour of a 25-hour day; an implementation on a fixed 24-slot array is wrong on both transition days. test_price.py drives both 2026 transitions and counts the points rather than asserting 24. The EU rule is fixed in law and needs no database: CEST from 01:00 UTC on the last Sunday of March until 01:00 UTC on the last Sunday of October. Review of PR #16 cross-checked this against `zoneinfo`'s Europe/Brussels over 149,040 hourly instants, 2020-2036 inclusive, and found ZERO mismatches - so the residual risk here is not arithmetic, it is political: if the EU ever abolishes the seasonal change, `brussels_offset()` needs editing. That is the same exposure as a machine whose `tzdata` has gone stale, and it is one edit in one function rather than a package upgrade. """ import ast import csv import inspect import io import json import logging import math from abc import ABC, abstractmethod from dataclasses import dataclass from datetime import datetime, timedelta, timezone from enum import Enum from textwrap import dedent from urllib.parse import urlencode from . import messages from .custom_formula import ( EXPORT_FORMULA_OPTION, FormulaInvalid, PREVIEW_HOURS, parse_export_formula, parse_price_formula, preview_lines, price_eur_kwh, to_resolution) _LOG = logging.getLogger("goodwe.price") # --------------------------------------------------------------------------- # # Brussels wall time, without a tz database. See the module docstring. # --------------------------------------------------------------------------- # CET = timezone(timedelta(hours=1)) CEST = timezone(timedelta(hours=2)) def _last_sunday(year: int, month: int) -> int: """Day-of-month of the last Sunday. March and October both have 31 days.""" day = 31 while datetime(year, month, day).weekday() != 6: # 6 == Sunday day -= 1 return day def brussels_offset(instant: datetime) -> timezone: """The offset Belgium is on at this UTC instant. CET or CEST, never else.""" u = instant.astimezone(timezone.utc) y = u.year start = datetime(y, 3, _last_sunday(y, 3), 1, tzinfo=timezone.utc) end = datetime(y, 10, _last_sunday(y, 10), 1, tzinfo=timezone.utc) return CEST if start <= u < end else CET def brussels(instant: datetime) -> datetime: """Render a UTC instant at the Brussels offset for that instant. ⚠️ Takes an INSTANT, and refuses a naive one. A naive datetime is not a moment in time, it is a wall clock reading plus an assumption, and the assumption is wrong for one hour twice a year. """ if instant.tzinfo is None or instant.utcoffset() is None: raise ValueError( f"brussels({instant!r}): a naive datetime is not an instant. Parse " f"the source's offset, or say UTC explicitly - guessing here is how " f"one hour of a transition day gets priced twice.") return instant.astimezone(brussels_offset(instant)) def brussels_date_range(since: datetime, until: datetime) -> tuple: """The Brussels calendar days a half-open window [since, until) touches. Returns `(first, last)`, two `date`s, BOTH INCLUSIVE - which is the shape `api.energy-charts.info` takes: `?bzn=BE&start=2026-09-09&end=2026-09-10` answered 192 quarter-hours across exactly those two local days, measured 2026-09-10T07:45:49Z and committed as `scenarios/energy-charts-be-price-2026-09-09_2026-09-10-twoday.json`. THE DEFECT THIS EXISTS TO PREVENT (TARIFF-09). `generic_dayahead.fetch()` accepted `since` and `until` and requested the bare default URL, so the feed answered ONE Brussels day. Everything past today was then carried forward from yesterday's profile, flagged provisional, and §8.3 dropped all of it from the trade set - so the optimiser never saw tomorrow as firm, at any hour. Measured on the merged tree at 2026-09-10T09:00+02:00 over a 48 h horizon: 15 firm whole hours and 33 carried, and both ends of the only permitted cycle inside ONE calendar day. The ceiling was structural: at hour H of the local day a one-day window gives at most 24-H firm hours and 24+H carried - 1/47 at 23:00, where the firm spread no longer even clears the round-trip cost. That is not what FCST-04's fill is for; it covers the pre-publication window until the ~13:00-14:00 CET auction, after which a healthy feed hands over tomorrow in full. ⚠️ THE CALENDAR IS BRUSSELS, NEVER UTC, AND THIS IS `_bdate()`'s LESSON ONE LAYER UP. `since`/`until` are INSTANTS; the API takes DATES, and the two are not the same question. 22:30 UTC in summer is already tomorrow in Belgium, so a range derived from `since.date()` asks for the wrong FIRST day for the last one or two hours of every single day - and the day it drops is the one the controller is standing in. price_history.py made this mistake at the cache layer and documents the same arithmetic; here it decides what is fetched at all, so getting it wrong loses the hours rather than misfiling them. ⚠️ `until` IS EXCLUSIVE, SO LOCAL MIDNIGHT BELONGS TO THE DAY BEFORE IT. A 48 h horizon from local midnight ends at local midnight two days later, and that instant is the first tick of a day the caller did not ask for. Taking `brussels(until).date()` unconditionally would request a THIRD day on every whole-hour horizon. This feed answers such a request without complaint, so nothing would report it - which is the reason to get it right here rather than to rely on the server's tolerance: the range is a claim about what the caller asked for, and it should be true. ⚠️ AND AN EMPTY OR INVERTED WINDOW COLLAPSES TO ONE DAY RATHER THAN TO A BACKWARDS RANGE. `start` after `end` is a request no feed has a defined answer for; some return everything, some return an error, and neither is a thing to hand a battery controller. """ first = brussels(since).date() edge = brussels(until) last = edge.date() if (edge.hour, edge.minute, edge.second, edge.microsecond) == (0, 0, 0, 0): last = last - timedelta(days=1) if last < first: last = first return first, last def dated_feed_url(base: str, **params) -> str: """`base` plus a query string. Refuses a `base` that already carries one. ⚠️ THE REFUSAL IS THE POINT, NOT THE FORMATTING. Appending `&start=...` to a URL that may already hold its own query string is how a plugin produces a request nobody intended - `?bzn=BE?bzn=BE&start=...`, or a duplicated parameter whose winner is the server's choice rather than ours. So the base is required to be parameterless and the parameters are added exactly once, by `urlencode`, which is also what keeps a date out of the escaping question. A future edit that folds the query back into the constant is a refused fetch with a sentence, not a silently malformed URL. ⚠️ IT IS NOT REACHED BY AN OPERATOR'S `price_feed_url`. See `GenericDayAhead.fetch()`: an override is used exactly as given. """ if "?" in base or "#" in base: raise SourceUnavailable( f"the feed base {base!r} already carries a query string or " f"fragment, and this builder appends parameters. Adding a second " f"'?' produces a URL the server parses differently from the one " f"anybody intended; put the fixed parameters in the call instead") return f"{base}?{urlencode(params)}" if params else base # --------------------------------------------------------------------------- # # Absence, borrowed wholesale from sim/riglock.py (TEST-04). # --------------------------------------------------------------------------- # class Absence(Exception): """A value was asked of a read that never observed one.""" class FrameworkOption(Exception): """A plugin reached for an option the FRAMEWORK owns. See trap 1.""" class SourceUnavailable(Exception): """A plugin could not get its data. Raised by plugins, caught by the framework, and turned into `PriceRead.no_data(...)`. ⚠️ It must never escape into the control tick. §11.4's row is a returned condition, not a traceback: the loop keeps regulating on P1 whatever the price situation is, and a price fetch that could stop it would be a price fetch that can stop the battery. """ class _Outcome: """One outcome of one read. The absent variant CARRIES NO FIELDS.""" kind = "?" words = "?" # human phrasing. Cosmetic, never an input to a decision. def __getattr__(self, name): if name.startswith("__") and name.endswith("__"): raise AttributeError(name) raise Absence( f"{type(self).__name__} has no {name!r}. This read did not observe " f"a price series, and a value invented here is the bug this " f"representation exists to prevent - 'no data' becoming 'free'. " f"Handle the {self.kind!r} case explicitly; see match().") def get(self, *_a, **_k): raise Absence( f"{type(self).__name__}.get() does not exist: a default is how a " f"failed fetch becomes an observation. Use .match(...) and handle " f"both outcomes.") def __iter__(self): # ⚠️ DELIBERATE. `for p in read:` and `list(read)` are the other two # idioms that turn an absent read into an empty, harmless-looking loop. raise Absence( f"{type(self).__name__} is not iterable: an empty loop body is how " f"'we have no prices' becomes 'we checked every hour and none were " f"expensive'. Use .match(series=..., no_data=...).") def __repr__(self): return f"{type(self).__name__}({self.words!r})" # --------------------------------------------------------------------------- # # The vocabulary # --------------------------------------------------------------------------- # class Component(Enum): """The three parts of an all-in Belgian consumption price (§9.3).""" ENERGY = "energy" MARKUP = "markup" TAXES = "taxes" ALL_IN = frozenset(Component) class PriceProduct(Enum): """The §9.2 product table. The optimiser switches on THIS, never on a name.""" DAY_AHEAD = "dynamic_dayahead" REALTIME = "realtime_imbalance" STATIC = "static_profile" FIXED = "fixed" # ⚠️ §8.3: "Real-time / imbalance products are treated as high-volatility: # arbitrage thresholds widen." This maps product -> regime, and it is the ONLY # place in the shipped code that says which products are volatile. OPT-04 reads # it; it does not re-derive it, and it certainly does not check plugin ids. _WIDE = frozenset({PriceProduct.REALTIME}) def volatility_regime(product: PriceProduct) -> str: """"wide" | "normal" - the arbitrage threshold regime for a product (§8.3). Deliberately a plain string and deliberately tiny: OPT-04 owns what the widened numbers ARE. This owns only which products get them, because that is a property of the product and belongs with the product enum. """ if not isinstance(product, PriceProduct): raise TypeError(f"volatility_regime() takes a PriceProduct, not " f"{product!r} - a name is exactly what AC2 forbids.") return "wide" if product in _WIDE else "normal" # --------------------------------------------------------------------------- # # FCST-03: ONE widening path, driven by a SET of active uncertainty sources. # # Two spec rows widen the same number by two different triggers: # # §9.2 / §8.3 real-time / imbalance product -> "arbitrage thresholds widen" # §11.4 no weather data -> "widen arbitrage thresholds" # # They are one mechanism reached twice, not two mechanisms, so there is one # function and a set of reasons. ⚠️ THE COMBINATION RULE IS `max`, NOT A # PRODUCT: multiplying 2.5 by 1.5 would make two independent doubts compound # into a threshold nothing on a Belgian tariff can ever clear, and an arbitrage # strategy that can never fire is indistinguishable from one that is switched # off. The widest single reason wins. # # ⚠️ HONEST ABOUT WHERE THE NUMBERS COME FROM. specs.md says thresholds widen # and does not say by how much; there is no capture to derive a factor from, # because no arbitrage strategy has ever run here. These are POLICY DEFAULTS, # chosen so that (a) a real-time product needs a materially larger spread than # a settled day-ahead one before a cycle is worth its wear, and (b) the two # factors DIFFER, so `max`, `sum` and `product` are three distinguishable # answers rather than the same number - a check that grades a combination rule # where all the candidates agree grades nothing. When OPT-04 lands # `cycle_cost_threshold_eur` (§10.1/§14) these become the natural second option # beside it, in PRICE_OPTION_DEFAULTS and nowhere else (CFG-03). class Uncertainty(Enum): """Why an arbitrage threshold is currently wider than the tariff alone says.""" REALTIME_PRODUCT = "realtime_product" # §9.2 row 2, §8.3 NO_WEATHER = "no_weather" # §11.4 "no weather data" WIDEN_FACTOR = { Uncertainty.REALTIME_PRODUCT: 2.5, Uncertainty.NO_WEATHER: 1.5, } def uncertainty_of(product: PriceProduct) -> frozenset: """The uncertainty a PRODUCT contributes on its own (§9.2 row 2). Derived from the declared product, never from a plugin id - the same rule `volatility_regime()` exists for. A sixth supplier declaring `PriceProduct.REALTIME` is widened without anyone editing this file. """ if not isinstance(product, PriceProduct): raise TypeError(f"uncertainty_of() takes a PriceProduct, not " f"{product!r} - a plugin name is what §9.3 forbids.") return (frozenset({Uncertainty.REALTIME_PRODUCT}) if product in _WIDE else frozenset()) def widen_factor(sources) -> float: """The multiplier the widest active source asks for. 1.0 when there are none. ⚠️ A STRING IS REFUSED. `widen_factor({"realtime"})` would otherwise be a silent 1.0 - a widening that reports itself as applied and is not, which is the shape of defect this repo keeps finding. """ srcs = frozenset(sources) bad = [s for s in srcs if not isinstance(s, Uncertainty)] if bad: raise TypeError(f"widen_factor() takes Uncertainty members, not {bad!r}. " f"A string here would widen by 1.0 and say it widened.") return max((WIDEN_FACTOR[s] for s in srcs), default=1.0) def required_spread(base_eur_kwh: float, sources) -> float: """The spread an arbitrage cycle must clear, given what is uncertain (§8.3). `base_eur_kwh` is the caller's own round-trip cost threshold - OPT-04 owns that number (§10.1's `cycle_cost_threshold_eur`), this owns only how much wider it gets and why. """ if isinstance(base_eur_kwh, bool) or not isinstance(base_eur_kwh, (int, float)) or not math.isfinite(base_eur_kwh) or base_eur_kwh < 0: raise ValueError( f"required_spread() takes a non-negative, finite base threshold in " f"EUR/kWh, not {base_eur_kwh!r}. A negative or NaN base makes every " f"spread clear it, which is arbitrage with the brake removed.") return round(float(base_eur_kwh) * widen_factor(sources), 6) @dataclass(frozen=True) class PricePoint: """One canonical point. THREE FIELDS. §9.3 lists three; this has three.""" start: datetime # tz-aware, rendered at the Brussels offset eur_per_kwh: float # ALL-IN: energy + markup + taxes provisional: bool def canonical(self) -> dict: return {"start": self.start.isoformat(), "eur_per_kwh": self.eur_per_kwh, "provisional": self.provisional} @dataclass(frozen=True) class RawPoint: """What a plugin returns: the figure its source published, unmodified. ⚠️ NOT a PricePoint, and the type difference is the guard. A plugin cannot hand back something already claiming to be all-in, because the only constructor of `PricePoint` that `load_prices()` uses is `assemble()`. """ start: datetime # tz-aware instant eur_per_kwh: float # as published, containing exactly caps.sources provisional: bool @dataclass(frozen=True) class ProviderCapabilities: """What a plugin declares about itself, before anyone asks it for prices.""" product: PriceProduct sources: frozenset # which Components the published figure contains horizon_h: int = 48 # aligns with the §8.1/§8.2 forecast horizon egress_hosts: tuple = () # §16 allow-list, declared not discovered def __post_init__(self): if not isinstance(self.product, PriceProduct): raise ValueError(f"product must be a PriceProduct, not " f"{self.product!r}") if Component.ENERGY not in self.sources: raise ValueError( "capabilities().sources must contain Component.ENERGY - a feed " "that does not publish an energy price is not a price feed.") extra = set(self.sources) - set(ALL_IN) if extra: raise ValueError(f"unknown price components declared: {extra}") @property def adds(self) -> frozenset: """The components the FRAMEWORK must add. The complement of `sources`.""" return frozenset(ALL_IN - self.sources) # --------------------------------------------------------------------------- # # The read: a series, or no data. Nothing in between. # --------------------------------------------------------------------------- # class PriceRead(_Outcome): """What one attempt to get prices said. Two outcomes, and only one answers. * `series` - we have prices. Carries `.points` (NON-EMPTY, by construction), `.product`, `.provider`, `.fetched_at`. * `no_data` - we do not. Carries `.reason` and NOTHING ELSE. §11.4: static profile, plan provisional, no arbitrage. ⚠️ A SERIES OF ALL-ZERO PRICES IS A `series`, NOT `no_data`, and that distinction is the whole point. Zero prices mean arbitrage is pointless; no data means arbitrage is FORBIDDEN. An implementation that returns `[]` on a failed fetch cannot tell a caller which of those two it is in. """ def match(self, *, series, no_data): """Fold to a value. Both handlers required and keyword-only. Handlers take one argument: this read. Required-ness is what makes the no-data case impossible to forget rather than merely easy to remember. """ return {"series": series, "no_data": no_data}[self.kind](self) @staticmethod def series(points, product, provider, fetched_at): return _Series(points, product, provider, fetched_at) @staticmethod def no_data(reason): if not reason or not isinstance(reason, str): raise ValueError( "PriceRead.no_data() takes a reason a human can act on. It is " "the only field this variant has; an empty one makes an " "operator's only evidence a blank status line.") return _NoData(reason) # --------------------------------------------------------------------------- # # How long ONE point covers - TARIFF-14 # --------------------------------------------------------------------------- # # ⚠️ A POINT DOES NOT CARRY ITS OWN DURATION AND MUST NOT LEARN TO. §9.3 lists # THREE fields and `PricePoint` has three; a fourth would change the wire # format every store, replay, fixture and captured body in this repository is # written against. The span is DERIVED from the neighbours instead - here, # once, at construction. MAX_POINT_SPAN = timedelta(hours=1) def _cover_table(points) -> tuple: """`(start_utc, end_utc, point)` per point, ascending. The `at()` index. THE DEFECT THIS PREVENTS (TARIFF-14, HIGH). `at()` asked `s <= u < s + 1 h` of every point IN LIST ORDER and returned the FIRST match. On an hourly series - the only resolution any fixture carried before TARIFF-08 - that is right by accident. Belgium moved to a 15-minute MTU on 2025-10-01 and the live day-ahead feed returns 96 points at 900 s, where it is wrong twice over. Measured on shipped `5059e28`: series: 02:00=0.10 02:15=0.11 02:30=0.12 02:45=0.13 03:00=0.14 at(02:15) -> 02:00 the :00 point's HOUR has not run out yet at(03:15) -> 02:30 and from the second hour on it is not even the top of the hour: the first point still inside its own hour wins, so the error COMPOUNDS with depth into the series `arbitrage._act()` compares `here.start` against `v.cheapest.start` and `v.dearest.start` to decide whether this interval is the buy or the sell one. That identity test almost never matched the extremum, so the strategy idled through exactly the interval it exists to trade - and read `here.provisional` and §17's `price < 0` guard off the wrong point while doing it. `static_tariff.py` emits hourly points, so static sites never saw it; this is reachable only down the dynamic path TARIFF-08/09 made work. THE RULE, in one sentence: A POINT COVERS FROM ITS OWN START UNTIL THE NEXT POINT'S START, AND NEVER MORE THAN ONE HOUR. * 900 s -> each point covers its own quarter, so `at(02:15)` is the 02:15 point; * 3600 s -> each point covers its own hour, unchanged from shipped; * MIXED -> EACH POINT TAKES ITS OWN GAP. FCST-04's carry-forward emits HOURLY carried points beside QUARTER-HOURLY firm ones, and the soak measured spacing `[900, 3600]` in 70 % of samples - so mixed is the normal steady state, and a window derived from one global spacing would be wrong THERE rather than in an edge case. ⚠️ WHY THE ONE-HOUR CAP, WHICH IS NOT DECORATION. FCST-04 LEAVES HOLES ON PURPOSE: an hour with no published price AND no yesterday value is left OUT of the series, and `price_forecast.py`'s module note names `at()` raising `Absence` for it as the behaviour that sends the caller to the static profile (§11.4 for that hour). Uncapped, "until the next point's start" would stretch the point BEFORE a three-hour hole across the whole hole and answer every instant in it with a stale price. Filling holes is the defect this module exists to refuse, and it would have been introduced by the fix for a different one. ⚠️ THE LAST POINT'S WINDOW IS DECIDED, NOT INHERITED. There is no next start and nothing in §9.3 says how wide the final interval is, so THE LAST POINT IS GIVEN THE SAME SPAN AS THE POINT BEFORE IT - one hour when it is the only point. Three consequences, all intended: * on a 96-point 900 s day the 23:45 point covers 23:45 .. 00:00, so `at(23:50)` ANSWERS and `at(00:00)` raises `Absence`. A flat one-hour tail would have answered 00:30 TOMORROW with today's last price, which is reaching for a neighbour across a day boundary; * on an hourly series the tail is an hour, exactly as it shipped, so no static or hourly site changes behaviour; * on a MIXED series whose last point is a carried HOURLY one behind quarter-hourly firm ones, the tail is 900 s and the last 45 minutes of the horizon raise `Absence`. That is the SAFE direction and it is accepted rather than tolerated: the caller falls back to the static profile (§11.4) instead of trading on a span the series never demonstrated, and a carried point is provisional, so §8.3 forbids any irreversible action resting on it regardless. ⚠️ ORDER IS ESTABLISHED HERE, NOT ASSUMED - and that is the difference between this and the rule it replaces. `at()` now reads neighbours, so it WOULD depend on sortedness; the table is therefore sorted by UTC start at construction, and no caller's list order can reach it. `_validate_raw()` rejects out-of-order and duplicate starts on the `load_prices()` path, `price_history._parse_points()` re-sorts what it reads off disk, and FCST-04's fill emits ascending - but `PriceRead.series()` itself has never enforced any of that, and an assumption that holds today and is unstated is how this class of defect recurs. ⚠️ NOTHING HERE RAISES ON A MALFORMED SERIES. A duplicated start - not reachable through `load_prices()`, which refuses it - gives the earlier list entry a ZERO-width window and the later one the real span, so one of the pair becomes unreachable through `at()` and neither the 30 s control path nor construction raises. """ rows = sorted(((p.start.astimezone(timezone.utc), i, p) for i, p in enumerate(points)), key=lambda r: (r[0], r[1])) out = [] for n, (start, _i, point) in enumerate(rows): if n + 1 < len(rows): span = rows[n + 1][0] - start # -> the next point's start elif n: span = start - rows[n - 1][0] # -> the tail, see above else: span = MAX_POINT_SPAN # -> a series of one point if span > MAX_POINT_SPAN: span = MAX_POINT_SPAN out.append((start, start + span, point)) return tuple(out) class _Series(PriceRead): kind = "series" def __init__(self, points, product, provider, fetched_at): points = list(points) # ⚠️ THE REFUSAL THAT MAKES THE TYPE WORTH HAVING. An empty series is # how "the fetch failed" gets to wear the shape of "we looked and there # is nothing expensive today". It is not constructible. if not points: raise ValueError( "PriceRead.series() refuses an empty point list. A fetch that " "produced no points is PriceRead.no_data(reason) - a different " "variant with different fields, on purpose. See §11.4.") if not isinstance(product, PriceProduct): raise ValueError( f"product must be a PriceProduct, not {product!r}. The " f"optimiser switches on this enum instead of on a plugin name; " f"a string here reopens exactly the hardcoding AC2 forbids.") self.points = tuple(points) self.product = product self.provider = provider self.fetched_at = fetched_at self.regime = volatility_regime(product) # ⚠️ TARIFF-14. Built ONCE, here, so that `at()` scans an index whose # order it established itself rather than one it hopes it was handed. # See `_cover_table()` for the window rule and for what the tail does. self._cover = _cover_table(self.points) self.words = (f"{len(self.points)} points from {provider} " f"({product.value}, {self.regime} thresholds)") def canonical(self) -> dict: """The §9.3 wire format. Exactly `{"prices": [...]}`.""" return {"prices": [p.canonical() for p in self.points]} def at(self, instant: datetime) -> PricePoint: """The point whose OWN interval covers `instant`, at ANY resolution. ⚠️ RETURNS A POINT OR RAISES; it never returns None. `series.at(t) or fallback` is the same defaulting bug one level down, so the falsy answer does not exist. Ask `covers()` first if you mean to branch. ⚠️ TARIFF-14: THE WINDOW IS NOT ONE HOUR WIDE ANY MORE, AND WAS NEVER ENTITLED TO BE. It runs to the next point's start, capped at an hour - `_cover_table()` above holds the rule, the tail decision and the reproduction on a quarter-hourly series. A fixed `s + 1 h` here returned the FIRST point still inside its own hour, which on the live 900 s feed is neither the covering point nor, past the first hour, even the top of the hour. A linear scan over the index, which is 192 entries on the shipped 48 h horizon. This runs on the 30 s control path through `arbitrage._point_at()`; a bisect would be correct and is not worth the second way to be wrong at this size. """ u = instant.astimezone(timezone.utc) for start, end, point in self._cover: if start <= u < end: return point raise Absence( f"no price point covers {u.isoformat()}; the series runs " f"{self.points[0].start.isoformat()} .. " f"{self.points[-1].start.isoformat()}. An instant outside the " f"series - or inside a hole FCST-04 left because nothing " f"published that hour - is an instant with no price, which is " f"§11.4 for it, not a reason to reach for a neighbouring one.") def covers(self, instant: datetime) -> bool: """Whether `at(instant)` would answer. ⚠️ SHARED TARIFF-14 EXACTLY, AND IS FIXED BY THE SAME CHANGE. It has no window rule of its own - it asks `at()` - so on the 900 s feed it answered True for instants no point covered and False for none of them, because the hour-wide window is never SHORTER than the truth. It is kept as a delegation for that reason: a second copy of the rule is a second place for it to be wrong. """ try: self.at(instant) return True except Absence: return False def last_end_before(self, instant: datetime): """When the last price that has run out by `instant` stopped covering, or None if no price has (PRICE-01: "no prices since 00:00"). Each point's end is the one `_cover_table()` worked out, so a hole in the series is reported from where the hole starts and not from the end of the day. """ u = instant.astimezone(timezone.utc) ends = [end for _start, end, _p in self._cover if end <= u] return max(ends) if ends else None def window(self, since: datetime, until: datetime) -> tuple: """Points whose START falls in [since, until). May legitimately be empty. ⚠️ DID NOT SHARE TARIFF-14, AND THE REASON IS WORTH STATING RATHER THAN REDISCOVERING. This grades a point's `start` against a half-open range and never asks how long a point lasts, so it carries no resolution assumption to be wrong about - the same call answers the same set at 900 s, at 3600 s and mixed. It is therefore NOT a coverage query: a point that starts before `since` and runs into it is absent from this tuple by design. `at()`/`covers()` are the coverage questions. """ a = since.astimezone(timezone.utc) b = until.astimezone(timezone.utc) return tuple(p for p in self.points if a <= p.start.astimezone(timezone.utc) < b) class _NoData(PriceRead): """We have no prices, and the §11.4 row applies. ⚠️ THIS CLASS HAS NO `.points` AND THAT IS THE POINT. `min(read.points)`, `for p in read.points`, `read.get("points", [])` and `len(read.points)` all raise `Absence` here rather than quietly reporting that the cheapest hour of the day costs nothing. ⚠️ TWO WAYS PAST IT SURVIVE, BOTH FOUND IN REVIEW OF PR #16, AND NEITHER IS WORTH CODE: * `vars(read).get("points", [])` returns `[]`. `vars()` bypasses `__getattr__` entirely; nothing short of `__slots__` plus a metaclass would stop it, and anyone writing `vars()` on a variant type has left the road on purpose. * `bool(read)` is `True` for BOTH variants, so `if read:` looks like a check and is not one. Defining `__bool__` to be False here would make `if read:` "work" - and that is exactly the wrong fix: it would turn a line that reads like a guard into one that IS a guard by accident, which is how `.get(k, default)` got into this codebase in the first place. Use `.match(...)`. There is no falsy variant on purpose. """ kind = "no_data" def __init__(self, reason): self.reason = reason self.words = f"no price data - {reason}" def arbitrage_allowed(read: PriceRead) -> bool: """§11.4, stated once, as a function so no caller has to remember it. Not the arbitrage strategy - that is OPT-04. This is only the veto, which belongs with the type that expresses the condition. """ return read.match(series=lambda _r: True, no_data=lambda _r: False) def plan_is_provisional(read: PriceRead) -> bool: """§8.3 / §11.4: provisional if there is no data at all, or if ANY point in the series is itself provisional. Derived from the points, never asserted.""" return read.match( series=lambda r: any(p.provisional for p in r.points), no_data=lambda _r: True) # --------------------------------------------------------------------------- # # Options: what a plugin may see, and what it may not # --------------------------------------------------------------------------- # # --------------------------------------------------------------------------- # # TARIFF-18: what `price_taxes_eur_kwh` IS MADE OF # --------------------------------------------------------------------------- # # ⚠️ THE DEFECT THIS TABLE PREVENTS IS THE ONE THE SHIPPED 0.06 WAS: A NUMBER # NOBODY COULD SOURCE. 0.06 EUR/kWh is roughly HALF the real Flemish non-energy # figure, and because no reader could see what it was made of there was nothing # to argue with. Every row below is one line of the Eneco residential card for # Flanders, September 2026, INCLUDING 6 % VAT, converted from the c EUR/kWh the # card prints to the EUR/kWh this option takes. `price_taxes_eur_kwh`'s default # is their sum, and test_price.py section 1b re-adds them and refuses a default # that is not exactly that. # # ⚠️ THIS IS A PER-DSO FIGURE, AND IT IS THE DISTRIBUTION ROW THAT MOVES. Across # the eight Fluvius DSOs that row runs 5,28 c EUR/kWh (Midden-Vlaanderen) to # 6,70 (West), so the all-in Flemish total runs about 0.117 to 0.131 EUR/kWh. # A site outside Antwerpen should type its own total; the other two rows are # Flanders-wide and do not move with the DSO. # # ⚠️ AND THE DEFAULT SITS NEAR THE LOW END OF THAT BAND ON PURPOSE, BECAUSE THE # TWO ERROR DIRECTIONS ARE NOT SYMMETRIC. This scalar is a LEVEL component: it # cancels exactly in `max - min` and cannot move which hour is cheapest # (test_price.py 16b measures that as a family). The one DECISION it reaches is # §17's negative-price guard in arbitrage.py, which tests the ABSOLUTE all-in # price of the sell hour. A figure set too HIGH carries a genuinely negative # hour up above zero, and the add-on exports into paid-for disposal - the exact # case §17 exists to refuse. A figure set too LOW only declines a trade that # would have paid. So where a default must stand in for eight different DSOs, # it belongs at the bottom of the band and not in the middle of it. # # ⚠️ `energiebijdrage` IS ZERO, AND IT IS A ROW HERE RATHER THAN AN ABSENCE. # "Vanaf 1 augustus 2026 bedraagt de energiebijdrage 0 EUR/MWh" - it was merged # into the bijzondere accijns. Anyone re-deriving this total from a guide # written before that date will find a separate energiebijdrage line and add it # on top of an accijns that already contains it. The zero row is here so that # the double-count is visible as a decision rather than as an omission. FLEMISH_TAXES_COMPONENTS_EUR_KWH = ( ("Fluvius distribution, Antwerpen", 0.0535), ("Bijzondere accijns, 'Alle verbruik' - no volume bands", 0.04876), ("Bijdrage groene stroom en WKK Vlaanderen", 0.0151), ("Energiebijdrage - 0 EUR/MWh since 2026-08-01, inside the accijns", 0.0), ) # ⚠️ TARIFF-18, AND IT IS NOT HOUSEKEEPING - IT DECIDES WHICH CONFIGS START. # TARIFF-17's rung in validate.py refuses a site that sets `price_taxes_profile` # AND leaves `price_taxes_eur_kwh` at something other than the shipped default, # because the table wins and the scalar would decide nothing. It compares # against the DEFAULT rather than against presence, because the Supervisor sets # `App.options = deepmerge(config.yaml defaults, stored options)` and a saved # blob therefore carries every schema key whether the operator typed it or not # (measured: the reference install's export is 70 of 71 keys - CHANGELOG 0.9.0, # `tools/supervisor_selfapi_probe.py` §D). # # So moving the default turns a previously ACCEPTED configuration into # `SystemExit(78)`. A site that stored 0.06 - which is every install that has # ever saved its configuration - and that also carries a taxes table would stop # starting on the upgrade, over a number it never typed and that decides # nothing. That refusal would be TRUE and USELESS: the rung's subject is a # number the OPERATOR typed, and a superseded shipped default is not one. It # still reads as "untouched" here, and the rung's sentence names both figures. SUPERSEDED_TAXES_DEFAULTS_EUR_KWH = (0.06,) # TARIFF-02 .. TARIFF-17 # ⚠️ CFG-03, AND HOW THIS TICKET AVOIDS WIDENING IT. Option defaults in this # add-on already live in three places - config.yaml, main.py's `o.get(k, X)` # fallbacks, and a dataclass default - held apart only by a tripwire in # test_validate.py. This module adds ONE more location, not another three: # # * every price option's fallback is here, and only here; # * validate.py imports this dict rather than restating any number; # * main.py gets no `o.get("price_...", X)` fallbacks at all; # * there is no dataclass carrying a default copy of any of these; # * test_price.py READS config.yaml and asserts the shipped default equals # this dict, key for key, so the two cannot drift silently. # # config.yaml remains the source of truth. This is the single in-code mirror, # pinned to it by a check, which is the `validate.py` pattern rather than a # fourth freehand copy. PRICE_OPTION_DEFAULTS = { "price_provider": "off", "price_markup_eur_kwh": 0.03, # §14 tariffs.energy_price.markup_eur_kwh # TARIFF-18: the sum of FLEMISH_TAXES_COMPONENTS_EUR_KWH above, to the # card's own last digit. §14 tariffs.energy_price.taxes_eur_kwh. "price_taxes_eur_kwh": 0.11736, "export_price_eur_kwh": 0.04, # §14 tariffs.export_price_eur_kwh (AC4) "price_csv_path": "/share/goodwe_prices.csv", "price_feed_url": "", # PRICE-01. The bearer token of the owner's price server. A secret: it is # in SECRET_OPTIONS below, so no text built from an error reaches a log, # the status page or a notification with it in. "price_service_token": "", "price_horizon_h": 48, "price_timeout_s": 10, # TARIFF-03. §14 `tariffs.static_profile: null`. A JSON object describing a # static day/night or fully-fixed tariff; "" is §14's `null`. Parsed and # validated by app/static_tariff.py, refused at startup by validate.py. # # ⚠️ IT LIVES HERE RATHER THAN IN static_tariff.py FOR CFG-03's SAKE. This # dict is the ONE in-code mirror of config.yaml, pinned to it by # test_price.py section 1. A second defaults dict in the new module would be # a fourth place option defaults live, which is exactly the open ticket. "static_profile": "", # TARIFF-17. A JSON object in `static_profile`'s OWN period grammar whose # per-period `eur_per_kwh` is the TAXES COMPONENT for that band, not an # all-in price; "" is unset and the scalar above decides. Parsed and # validated by app/static_tariff.py (`parse_taxes_profile`), refused at # startup by validate.py - the `static_profile` precedent exactly, and here # for the same CFG-03 reason: this dict is the ONE in-code mirror of # config.yaml. # # ⚠️ IT IS IN FRAMEWORK_KEYS. A plugin that could read this could add the # taxes a second time, per hour, which is the double-counting defect trap 1 # exists to refuse - wearing a period table instead of a scalar. "price_taxes_profile": "", # TARIFF-15. A JSON object carrying ONE contract's two coefficients, the # units they are printed in, its VAT rate and the card figures the add-on # grades them against; "" is unset. Parsed and validated by # app/custom_formula.py, refused at startup by validate.py - the # `static_profile` precedent exactly, and here for the same CFG-03 reason: # this dict is the ONE in-code mirror of config.yaml. "price_formula": "", # TARIFF-16. The INJECTION leg, in `price_formula`'s OWN grammar and read # by `custom_formula.parse_export_formula()` - the same parser with its # option name as an argument, so a second unit table and a second unknown- # key rule do not exist to diverge. "" is unset and the scalar # `export_price_eur_kwh` above decides, which is every site today. # # ⚠️ IT IS IN FRAMEWORK_KEYS, for the reason the scalar is: a CONSUMPTION # plugin that could read an injection curve could fold it into a # consumption price, and nothing downstream could tell. # # ⚠️ AND IT LIVES HERE RATHER THAN IN custom_formula.py FOR CFG-03's SAKE, # exactly as `price_formula` and `price_taxes_profile` do: this dict is the # ONE in-code mirror of config.yaml. "price_export_formula": "", # OPT-04. §10.1 priority 3, and §14's `optimizer.cycle_cost_threshold_eur`. # # ⚠️ THEY LIVE HERE FOR CFG-03's SAKE, exactly as `static_profile` does, and # FCST-03's note above `WIDEN_FACTOR` names this as the place they would # land. `arbitrage_soc_floor` / `arbitrage_soc_ceiling` are the reserve # arbitrage may not spend and the headroom it may not overfill; they are # NOT the maintenance SoC keys, which describe a balancing cycle rather # than a trading reserve and would silently couple two unrelated policies. # # ⚠️ AND `cycle_cost_threshold_eur` IS 0.10 BECAUSE §10.1's NAME FOR IT SAYS # SO, NOT BECAUSE IT MAKES ANYTHING TRADE. On realistic Belgian day/night # numbers the static spread is ~0.08 EUR/kWh, so a static-tariff site does # NOT trade at this default. That is a reported result (test_arbitrage.py # section 7), and whether 0.10 - or FCST-03's 2.5/1.5 - is right is one # product decision that belongs to the owner. No number here was chosen to # make arbitrage happen. "arbitrage_enabled": False, "cycle_cost_threshold_eur": 0.10, "arbitrage_soc_floor": 20, "arbitrage_soc_ceiling": 95, } # ⚠️ The keys the FRAMEWORK owns. A plugin that reads one of these is a plugin # that can double-count, so `PluginOptions` refuses them by name. Trap 1. FRAMEWORK_KEYS = frozenset({ "price_markup_eur_kwh", "price_taxes_eur_kwh", "export_price_eur_kwh", # TARIFF-17. The hour-varying spelling of `price_taxes_eur_kwh`, and it # belongs here for the same reason the scalar does: it is a component the # FRAMEWORK adds exactly once, by `caps.adds`. `BANNED_IN_PLUGIN_CODE` is # derived from this set, so `code_leaks()` refuses a plugin that names it # without anything else being edited. "price_taxes_profile", # TARIFF-16. The hour-varying spelling of `export_price_eur_kwh`, and it # belongs here for the same reason the scalar does: a plugin that could # read an INJECTION curve could fold it into the CONSUMPTION series it # emits, and `assemble()` has no way to notice a component that arrived # inside a RawPoint. `BANNED_IN_PLUGIN_CODE` is derived from this set, so # `code_leaks()` refuses a plugin that names it with nothing else edited. EXPORT_FORMULA_OPTION, }) # PRICE-01. Options whose VALUE must never be printed. `scrub()` removes them # from any text the price code builds out of an error. SECRET_OPTIONS = frozenset({"price_service_token"}) def scrub(text, opts) -> str: """`text` with every secret option's value replaced by `***`. The price code builds reasons out of library errors, and a library may put a request header into its error text. Taking the value out here is the last line of defence; the plugin also never puts it in a message itself. """ out = str(text) for key in SECRET_OPTIONS: secret = str((opts or {}).get(key) or "").strip() if len(secret) >= 4: out = out.replace(secret, "***") return out class PluginOptions: """The options dict, minus the keys only the framework may read. Reading `price_markup_eur_kwh` here raises `FrameworkOption` rather than returning 0.03. ⚠️ THIS IS ONE OF THE TWO DOORS, NOT THE ONLY ONE. `_o` is reachable, and `price_option()` and `PRICE_OPTION_DEFAULTS` are public names in this module. This class alone is a convention; it is `register()`'s `code_leaks()` scan that makes it an inability. Do not weaken either half on the strength of the other. """ def __init__(self, opts: dict): self._o = dict(opts or {}) def get(self, key, default=None): if key in FRAMEWORK_KEYS: raise FrameworkOption( f"{key!r} belongs to the framework, not to a plugin. A plugin " f"returns RawPoints carrying what its SOURCE published and " f"declares capabilities().sources; assemble() adds the missing " f"components exactly once. If a plugin could read this number " f"it could add it a second time, and the resulting price would " f"be wrong by one markup with nothing anywhere reporting it.") v = self._o.get(key, None) if v is None or v == "": return PRICE_OPTION_DEFAULTS.get(key, default) return v def __getitem__(self, key): return self.get(key) def __contains__(self, key): return key not in FRAMEWORK_KEYS and key in self._o def price_option(opts: dict, key: str): """A framework-side option read. One place, defaults from one dict.""" if key not in PRICE_OPTION_DEFAULTS: raise KeyError(f"{key!r} is not a price option; add it to " f"PRICE_OPTION_DEFAULTS and to config.yaml together.") v = (opts or {}).get(key, None) if v is None or v == "": return PRICE_OPTION_DEFAULTS[key] return v def provider_id(opts: dict) -> str: """The selected plugin id, normalised. `"off"` when none is selected. ⚠️ ONE SPELLING OF THIS EXPRESSION, AND TARIFF-10 IS WHY IT IS A FUNCTION. `str(price_option(opts, "price_provider") or "off").strip()` was written out twice - in `load_prices()` below and in `static_tariff.resolve_tariff()` - and main.py's startup gate would have been a third. "Is a provider configured" is now a DECISION the add-on takes (it is what starts the background refresh), and a decision spelled out at three call sites is three places to disagree about whether `" off "` or `None` counts. ⚠️ `validate.py` deliberately keeps its own reader: it validates the RAW options blob, including one the Supervisor never saw, so it must not route through `price_option()`'s defaults. Note what this does NOT do: it does not decide whether the id names a shipped plugin - `load_prices()` and validate.py's rule 7 own that, and answering "unknown" here would turn a typo into "no provider" instead of into a startup refusal. """ return str(price_option(opts, "price_provider") or "off").strip() def provider_configured(opts: dict) -> bool: """Has the operator selected a price feed at all? (TARIFF-10) THE DEFECT THIS PREVENTS: the background tariff refresh used to start only when `arbitrage_enabled` was on, which ships `false` - so a site with a provider configured fetched NOTHING, ever. `/data/price_history.json` was never created, the *Price history* status row read `empty` forever, and TARIFF-09's short-horizon warning could not fire because no fetch happened to warn about. Measured over a 40 h two-instance soak: zero feed requests and `present: false` across all 802 samples on the shipped default, against a resolved plan on the same feed with arbitrage armed. ⚠️ THIS IS NOT A TRADING SWITCH AND MUST NEVER BECOME ONE. It answers "fetch prices?"; `arbitrage.enabled()` answers "move energy on them?", and §10.1's priority 3 is gated on the second alone. See `main.tariff_refresh_wanted()` for the one place the two meet. """ return provider_id(opts) not in ("", "off") def export_price_eur_kwh(opts: dict) -> float: """AC4 / §9.3: the injection price is a SEPARATE SCALAR, not a second series. In Belgium it is often near zero, which by itself favours self-consumption over export. It is deliberately not routed through the plugin framework - but ⚠️ NOT because nothing dynamic exists to route. That claim stood here until TARIFF-07 and it was FALSE: * Eneco publishes an injection formula on the same tariff card as its consumption formula - `0,1 x BELPEX-H - 1,188` - identical on the January and the September 2026 cards; * ENGIE publishes one in the same two-coefficient shape, `(B x eSpot_15) - A`. So a supplier DOES publish a dynamic injection curve. What is true is narrower and is the actual reason this is a scalar: an injection curve is the SAME `B x spot - A` shape as the consumption curve but carries ITS OWN coefficients and its own VAT treatment - injection is VAT-exempt where consumption is at 6% - so it is a second per-contract formula, not a second copy of the first. ⚠️ TARIFF-16 LANDED THE INJECTION LEG, AND THIS FUNCTION IS UNCHANGED. The curve lives in `price_export_formula` and is read by `export_formula()` below; this reader still answers the FLAT injection price, still from one option, still with no formula anywhere in its arithmetic. That is the arrangement this docstring predicted before either existed - "an injection curve gets a `PriceProvider` of its own and this scalar becomes its FIXED-product default" - and it is why a site with a flat injection price sees nothing change: `price_export_formula` ships "", `export_price_at()` falls through to this function, and this function's body is byte-identical to the one TARIFF-02 shipped. ⚠️ It is NOT reachable from a plugin: `export_price_eur_kwh` is in FRAMEWORK_KEYS, so a consumption plugin cannot quietly fold an injection price into a consumption price. `price_export_formula` is in the same set for the same reason. ⚠️ AND NOTHING IN THIS ADD-ON READS EITHER ONE TO TAKE A DECISION. `arbitrage.py` prices the sell side as DISPLACED IMPORT and says so in its own comment; both hours it compares are all-in consumption prices. This reader has no caller under `app/` and neither does `export_price_at()`. They are what an operator is shown and what a future export-aware strategy would call - not a number any trade turns on today. """ return float(price_option(opts, "export_price_eur_kwh")) def export_formula(opts: dict): """The configured `price_export_formula` as a `FormulaSpec`, or None. TARIFF-16. Raises `custom_formula.FormulaInvalid` on a malformed leg - and ⚠️ THAT EXCEPTION IS NOT MEANT TO REACH A RUNNING SITE, by the rule `taxes_profile()` states above and `static_profile` established: validate.py parses the same option at STARTUP and refuses to start on it, because a formula that only fails when something reads it is a defect discovered at the worst possible moment and §11.4 makes every price defect quiet. ⚠️ THE PARSE IS THE SAME ONE THE CONSUMPTION LEG GETS. There is no second parser, no second unit table and no second resolution rule - `parse_export_formula()` is `parse_price_formula()` with its `option` argument moved, so every refusal an operator can hit on this leg is a refusal that already had checks over it. """ return parse_export_formula(price_option(opts, EXPORT_FORMULA_OPTION)) def export_price_at(opts: dict, index_eur_per_mwh: float) -> float: """What the site is PAID for one exported kWh at one index value, EUR/kWh. THE SEAM, AND ITS HONEST SCOPE. This is the one function a future export-aware strategy calls, and it has NO CALLER UNDER `app/` today - stated here rather than implied, because the add-on takes no decision on an injection price (see `export_price_eur_kwh()` above and `arbitrage.py`'s accounting-basis comment). What it buys now is that OPT-06 and anything after it inherit a shape the market fits, instead of a scalar that cannot hold its own factor, a negative offset, or a VAT rate different from the consumption leg's. ⚠️ THE FLAT PATH IS THE FALLTHROUGH AND IT IGNORES THE INDEX, which is CORRECT rather than a simplification: a site with a flat injection price is paid the same for every kWh whatever the market did, and that is what the scalar means. With `price_export_formula` empty - every site today - this returns exactly `export_price_eur_kwh(opts)` for every index. ⚠️ A NEGATIVE ANSWER IS A REAL ANSWER. Eneco's `0,1 x BELPEX-H - 1,188` is below zero for any BELPEX-H under 11,88 EUR/MWh, which is the middle of a sunny day and precisely when a PV household exports. Nothing here floors it: a caller that silently clamped it to zero would hide the hours in which exporting COSTS money, which is the one thing an injection price is for. """ spec = export_formula(opts) if spec is None: return export_price_eur_kwh(opts) return price_eur_kwh(spec, index_eur_per_mwh) def taxes_profile(opts: dict): """The configured `price_taxes_profile` as a period table, or None. TARIFF-17. Raises `static_tariff.ProfileInvalid` on a malformed table - and ⚠️ THAT EXCEPTION IS NOT MEANT TO REACH A RUNNING SITE. validate.py parses the same option at STARTUP and refuses to start on it, for the reason `static_profile` is refused there: a table that only fails when a fetch happens is a site that stops pricing at 03:00, and §11.4 turns that into a quiet fallback nothing on the status page distinguishes from a healthy fixed-tariff day. `load_prices()` still handles it, loudly, because a guard whose only proof is "the other guard ran first" is not a guard. ⚠️ THE IMPORT IS LAZY AND THE CYCLE IS THE REASON. `static_tariff.py` imports this module at module scope; importing it back at module scope here would make which file is imported first decide whether the add-on starts. `sys.modules` makes the repeat cost a dict lookup. """ from .static_tariff import parse_taxes_profile return parse_taxes_profile(price_option(opts, "price_taxes_profile")) # --------------------------------------------------------------------------- # # The plugin interface # --------------------------------------------------------------------------- # MAX_PLAUSIBLE_EUR_KWH = 10.0 # see _validate_raw class PriceProvider(ABC): """One supplier. Declares what it is, then returns what its source said. ⚠️ `fetch()` DOES NOT RETURN A PRICE. It returns `RawPoint`s carrying the published figure. The all-in arithmetic happens in `assemble()`, once, in the framework, using `capabilities().sources`. See trap 1. """ id = "?" # PRICE-01. The ids to try, in order, when this source cannot answer, and # whether the last answer is kept for a day without any source. Empty and # False for every plugin that existed before, which is what keeps them # exactly as they were: `price_chain.py` is not used for them at all. FALLBACKS: tuple = () KEEPS_LAST_GOOD: bool = False @classmethod @abstractmethod def capabilities(cls) -> ProviderCapabilities: ... @abstractmethod async def fetch(self, opts, since: datetime, until: datetime, transport) -> list: """Return `RawPoint`s, or raise `SourceUnavailable(reason)`. `transport(url, timeout_s) -> bytes` is injected so the checks never touch a network and `manual_csv` never needs one. `opts` is a `PluginOptions`, so the markup and taxes are not reachable from here. ⚠️ `since`/`until` ARE THE RANGE THE CALLER WANTS, AND A PLUGIN THAT IGNORES THEM GETS WHATEVER ITS FEED DEFAULTS TO (TARIFF-09). That is not a small difference: `generic_dayahead` ignored both, the public EPEX feed answered ONE Brussels day, and §8.3 then dropped every hour past today from the trade set as provisional - so the optimiser never saw tomorrow as firm, at any hour of any day, and nothing reported it. This promise is NOT mechanically enforced. A rule obliging every plugin to pass a range would be satisfied by an INVENTED parameter name - the parser-graded-against-its-own-guess defect, moved from the response to the request, and the defect TARIFF-08 found on the response side. What exists instead is a roster pin in test_price.py: which plugins consume the range and which do not is asserted as a literal, so a third supplier forces the decision rather than inheriting the silence. """ def startup_preview(self, points) -> list: """Lines to log ONCE, the first time this provider answers (TARIFF-15). `points` are the ASSEMBLED, all-in `PricePoint`s - what the site is really priced on - and the default is to say nothing, so no existing plugin changes behaviour. ⚠️ IT IS A HOOK RATHER THAN A LINE IN `load_prices()` BECAUSE THE FRAMEWORK CANNOT KNOW WHAT IS WORTH PREVIEWING. Only the plugin knows what went IN to its own arithmetic - for `custom_formula` the index in EUR/MWh - and printing that beside the all-in figure is the whole point: a factor of 1000 is obvious in one line of `61.96 -> 0.13522` and invisible in a plan. The framework supplies the one number a plugin may never read, by handing back the assembled result rather than the tax rate. ⚠️ IT MUST NOT RAISE INTO THE CONTROL PATH and it is not trusted not to: `_emit_startup_preview()` catches everything. A log line is not a reason to stop regulating a battery. """ return [] # ⚠️ THE NAMES A PLUGIN'S OWN CODE MAY NOT MENTION. FRAMEWORK_KEYS is the # obvious half; the other three are the routes review of PR #16 actually used to # write a double-counting plugin after `PluginOptions` was in place: # # `_o` - PluginOptions' private dict, reachable as # `opts._o["price_markup_eur_kwh"]`. # `price_option` - a PUBLIC framework helper that takes the raw # options dict and answers with the markup. # `PRICE_OPTION_DEFAULTS` - a PUBLIC module constant. This is the one that # decided the review: no private access, no options # dict, no helper - just a dict in the module every # plugin already imports for `register`. # # Any of them turns +0.088 into +0.119 on a wholesale feed with nothing # anywhere reporting it. BANNED_IN_PLUGIN_CODE = tuple(sorted(FRAMEWORK_KEYS)) + ( "PRICE_OPTION_DEFAULTS", "price_option", "_o") def code_leaks(cls) -> list: """Framework-owned names appearing in a plugin's own CODE. Empty is clean. ⚠️ DOCSTRINGS ARE STRIPPED BEFORE THE SEARCH, AND THAT IS NOT COSMETIC. Every shipped plugin's docstring is REQUIRED to name the components it sources, so `generic_dayahead` says "markup" in prose and would false-positive against `price_markup_eur_kwh` on a raw text scan. The check that fires on the documentation it also mandates is the check that gets deleted. So the class is parsed, every docstring node is dropped, and the search runs on `ast.unparse` of what is left - i.e. on code, which is the only thing that can actually add a number twice. ⚠️ FAIL-OPEN ON MISSING SOURCE, DELIBERATELY. `inspect.getsource` needs the .py file, which the add-on image has (Dockerfile COPYs `app/`) but a frozen or zipapp build would not. Refusing to start a battery controller because a STATIC ANALYSIS could not run would trade a real outage for a hypothetical accounting error, so an unreadable source returns `[]` and logs. CI proves the scan really runs: test_price.py asserts source was found for every shipped plugin, so a build that silently lost it is a red check, not a quietly disarmed guard. """ try: src = dedent(inspect.getsource(cls)) tree = ast.parse(src) except (OSError, TypeError, IndentationError, SyntaxError) as e: _LOG.warning("Could not read the code of price source %s (%s), so its " "double-counting check is skipped.", getattr(cls, "id", cls), e) return [] for node in ast.walk(tree): body = getattr(node, "body", None) if isinstance(body, list) and body and isinstance(body[0], ast.Expr) and isinstance(body[0].value, ast.Constant) and isinstance(body[0].value.value, str): body.pop(0) code = ast.unparse(tree) return sorted({name for name in BANNED_IN_PLUGIN_CODE if name in code}) # --------------------------------------------------------------------------- # # TARIFF-07: the ids that USED to resolve here, and what an operator on one of # them has actually been running. # # ⚠️ THE DEFECT THIS EXISTS TO PREVENT: A RETIRED ID READ AS A TYPO. Without # this table `price_provider: "eneco_eddy"` reaches validate.py's unknown-id # branch and is refused as "not a shipped plugin" - true, and useless. The # operator did not mistype it; it WAS a shipped plugin, they chose it on # purpose, and the add-on has accepted it every startup since. What they need # told is not "that id is wrong" but "that id never fetched a price, and here # is what you have been running instead" - because §11.4 turned every failed # fetch into a silent fall back to `static_profile`, so NOTHING they can see # has ever differed from a working provider that finds no cheap hour. # # ⚠️ AND WHY THERE IS NO SILENT ALIAS. Mapping these to `generic_dayahead` # would be the friendlier-looking option and it is the wrong one: these three # published RETAIL prices (`B x spot + A`, per contract) and `generic_dayahead` # reads WHOLESALE. Aliasing needs the site's own two coefficients, which nobody # has - they live behind the customer's klantenzone login, and there is no # regulator index to read them from (CREG's tariff-card pages 404; VREG was # unreachable; every coefficient found in research came from a supplier's own # PDF). An alias would therefore have to INVENT the numbers it prices a battery # against, which is the same class of defect as the parser graded against its # own guess - just one layer further from anything that reports. # # Each value is the ID-SPECIFIC half of the refusal: the endpoint, what it was # observed to answer on 2026-09-13, and why no replacement plugin can be built. # validate.py owns the common framing. Verified by hand before deletion, not # taken on report - see DECISIONS/TARIFF-07-withdrawing-three-plugins.md. # --------------------------------------------------------------------------- # RETIRED_PROVIDERS: dict = { "engie_dynamic": ( "its endpoint https://api.engie.be/dynamic/be does not exist - the " "gateway there answers 404 with Kong's \"no Route matched with those " "values\". No public endpoint could replace it: ENGIE's dynamic price " "is (B x eSpot_15) - A with PER-CONTRACT coefficients that live behind " "your klantenzone login, so no unauthenticated URL can return YOUR " "price"), "totalenergies_dynamic": ( "its endpoint https://api.totalenergies.be/prices/dynamic does not " "exist - the TLS handshake fails outright because that hostname is " "absent from the certificate the server presents. The coefficients are " "per-contract there too"), "eneco_eddy": ( "its endpoint https://api.eneco.be/eddy/prices does not exist - it " "answers 404 with an empty body - and \"Eddy\" is not an Eneco " "product at all; their dynamic product is \"Zon & Wind Dynamisch\", " "which is indexed to Belpex/EPEX DAY-AHEAD and whose coefficients are " "likewise per-contract"), } PROVIDERS: dict = {} def register(cls): """Register a provider by its own declared id. Decorator. ⚠️ THE ONLY NAME->CLASS MAPPING IN THE SYSTEM. AC2 is "adding a plugin needs no optimiser changes", and this is where that is bought: config resolves an id here, the optimiser is handed a `_Series` carrying a `PriceProduct`, and nothing downstream ever sees the id except to print it. """ if cls.id in PROVIDERS: raise ValueError(f"duplicate provider id {cls.id!r}") if cls.id in RETIRED_PROVIDERS: # ⚠️ AT IMPORT, NOT AT FETCH, and for the same reason the leak check is. # A retired id that is registered again is the one state in which # validate.py's retired-id refusal goes quiet WITHOUT anyone deciding # it should: `pid in PROVIDERS` becomes true, the named message is # never reached, and the operator is back to a provider that looks # like it works. Re-shipping an id is legitimate - the endpoint could # become real - but it is a DECISION, and taking the id out of # RETIRED_PROVIDERS in the same commit is what records it. raise ValueError( f"{cls.id!r} is in RETIRED_PROVIDERS and cannot also be a shipped " f"plugin. A site configured on it is refused at startup with a " f"message naming the id; registering it silently takes that " f"message away. If this id really ships again, remove it from " f"RETIRED_PROVIDERS in the same commit - and say in the CHANGELOG " f"what changed about the endpoint.") cls.capabilities() # fail at import, not at 15:00 on fetch day leaks = code_leaks(cls) if leaks: # ⚠️ AT IMPORT, NOT AT FETCH. A plugin that can reach the markup is a # plugin that can bill wrong for a month before anyone notices, so it # does not get to exist - the add-on fails to start and names the line. raise FrameworkOption( f"{cls.id!r} mentions {', '.join(leaks)} in its own code. A plugin " f"returns what its SOURCE published and declares " f"capabilities().sources; the framework adds the complement, once. " f"Reaching any of {BANNED_IN_PLUGIN_CODE} lets it be added twice, " f"and a price that is one markup high is invisible in every log " f"this add-on writes. Remove the reference; if the plugin genuinely " f"needs a per-site number, add a NON-framework option key for it.") PROVIDERS[cls.id] = cls return cls def _validate_raw(points, caps): """Reject at the boundary, so a malformed feed cannot become a bad decision. ⚠️ NEGATIVE PRICES ARE VALID AND ARE NOT CLAMPED (§17 keeps a negative-price regression case; Belgian day-ahead goes negative on windy Sundays and those hours are the single best charging opportunity of the year). What is rejected is what cannot be a price at all: NaN, infinity, an implausible magnitude, a naive datetime, and duplicate or out-of-order starts. """ if not isinstance(caps, ProviderCapabilities): raise SourceUnavailable("the plugin did not declare capabilities") if not points: raise SourceUnavailable("the source returned no points") seen = None for p in points: if not isinstance(p, RawPoint): raise SourceUnavailable(f"a plugin returned {type(p).__name__}, not " f"a RawPoint - see PriceProvider.fetch()") if p.start.tzinfo is None or p.start.utcoffset() is None: raise SourceUnavailable(f"point at {p.start!r} has no offset") if isinstance(p.eur_per_kwh, bool) or \ not isinstance(p.eur_per_kwh, (int, float)) or \ not math.isfinite(p.eur_per_kwh): raise SourceUnavailable(f"point at {p.start.isoformat()} has a " f"non-finite price {p.eur_per_kwh!r}") if abs(p.eur_per_kwh) > MAX_PLAUSIBLE_EUR_KWH: raise SourceUnavailable( f"point at {p.start.isoformat()} is {p.eur_per_kwh} EUR/kWh, " f"beyond +-{MAX_PLAUSIBLE_EUR_KWH}, so the feed is probably in " f"EUR/MWh") u = p.start.astimezone(timezone.utc) if seen is not None and u <= seen: raise SourceUnavailable( f"points are not strictly increasing at {u.isoformat()} - a " f"duplicated hour on a DST day is the usual cause, and keying " f"prices by a naive local hour is how it happens") seen = u def taxes_at(taxes, instant: datetime) -> float: """The TAXES component at ONE instant. A scalar, or a period table. TARIFF-17. THE DEFECT THIS EXISTS TO CLOSE, and it is a MISSING CAPABILITY rather than a wrong constant: `price_taxes_eur_kwh` is additive and identical every hour, so it cancels exactly out of `max - min` and cannot move `min(firm, key=...)`. It is not a distortion - it is provably inert on both quantities the optimiser reads. What was missing is anywhere for an HOUR-VARYING non-energy component to live, and the Walloon and Brussels DSOs bill one: ORES `dag 13,27 / nacht 7,39` c EUR/kWh, a 5,88 c gap whose cheap band coincides with the cheap spot hours. Flattening it does not shift the curve, it UNDERSTATES the spread and declines profitable cycles. See DOCS.md "The three buckets". ⚠️ DUCK-TYPED ON `price_at`, NOT ON `isinstance`. price.py must not import static_tariff.py - static_tariff.py imports THIS module, and a module-level cycle would make the import order load-bearing. `taxes_profile()` below is the one place the real parser is reached, and it imports it lazily. ⚠️ THE LOOKUP IS ON THE ~30 s PLANNING PATH through `assemble()`, so it stays a linear scan over the two-to-four periods a Belgian tariff has (`StaticProfile.at()`), with no parsing, no allocation and no I/O. A malformed table never reaches here: validate.py refuses it at STARTUP. """ at = getattr(taxes, "price_at", None) if at is None: return float(taxes) return float(at(instant)) def assemble(caps: ProviderCapabilities, raw, markup: float, taxes) -> list: """RawPoints -> canonical all-in PricePoints. THE ONLY PLACE MARKUP IS ADDED. Adds exactly `caps.adds` - the complement of what the plugin declared its source already contains. A plugin that publishes an all-in figure gets nothing added; a raw EPEX figure gets both. ⚠️ `taxes` IS A SCALAR *OR* A PERIOD TABLE (TARIFF-17), AND IT IS EVALUATED AT EACH POINT'S OWN INSTANT. Hoisting the lookup out of the loop - reading it once and adding the same number to every point - is the mutation that collapses the whole feature back into the LEVEL bucket: the assembled series would still differ from the untaxed one by the right MAGNITUDE, and the cheapest hour would be wrong. `test_price.py` section 24 is the check that goes red for it, and it grades WHICH HOUR IS CHEAPEST rather than the spread, because a spread-magnitude assertion passes on the hoisted version. ⚠️ THE SCALAR PATH IS UNCHANGED, ARITHMETICALLY AND NOT ONLY IN INTENT. `taxes_at()` on a float returns that float at every instant, and `0.0 + markup` is exactly `markup` in IEEE 754 for every finite markup, so a site on a scalar gets the identical bytes it got before this landed. """ add_markup = float(markup) if Component.MARKUP in caps.adds else 0.0 wants_taxes = Component.TAXES in caps.adds out = [] for p in raw: local = brussels(p.start) add = add_markup + (taxes_at(taxes, local) if wants_taxes else 0.0) out.append(PricePoint(start=local, eur_per_kwh=round(p.eur_per_kwh + add, 6), provisional=bool(p.provisional))) return out # --------------------------------------------------------------------------- # # Transport - one place, HTTPS only, host checked against the declaration # --------------------------------------------------------------------------- # async def https_get(url: str, timeout_s: float, headers=None) -> bytes: """The default transport. §16: outbound is allow-listed and TLS'd. Imported lazily so `manual_csv` - the offline fallback - never needs aiohttp to be importable at all. AC3 says it works fully offline; that includes a machine where the HTTP stack is missing. PRICE-01. `headers` is None for every plugin but `price_service`, and with None this is the code it has always been, byte for byte. With headers it answers in the SAFE words of `status_words()` and never lets a library error through: an error from a request that carried a token is the one place a token could be printed by somebody else's code. """ import aiohttp t = aiohttp.ClientTimeout(total=float(timeout_s)) if headers is None: async with aiohttp.ClientSession(timeout=t) as s: async with s.get(url) as r: r.raise_for_status() return await r.read() try: async with aiohttp.ClientSession(timeout=t) as s: async with s.get(url, headers=headers) as r: if r.status != 200: raise SourceUnavailable(status_words(r.status)) return await r.read() except SourceUnavailable: raise except Exception as e: # noqa: BLE001 raise SourceUnavailable( f"the price service did not answer ({type(e).__name__})") # PRICE-01. The one phrase a refused token is reported with. The fallback chain # reads it to tell "the owner must act" from "the server is down", so it is a # constant both sides import. AUTH_REFUSED = "the price service refused the token" def status_words(status: int) -> str: """What an HTTP status from the price service is called. No header text.""" if status in (401, 403): return f"{AUTH_REFUSED} ({status})" if status == 429: return "the price service says too many requests (429)" if status >= 500: return f"the price service has an error ({status})" return f"the price service answered {status}" def check_egress(url: str, caps: ProviderCapabilities) -> str: """§16: a plugin may only reach a host it DECLARED. Enforced, not documented. Returns the host, so a caller that forgets to call this is not the same shape as one that did. ⚠️ THE SCHEME IS SPLIT OFF BEFORE THE HOST IS PARSED, AND THE ORDER MATTERS FOR THE TEST MORE THAN FOR THE CODE. The first version stripped a fixed `len("https://")` from the front, which on an `http://` URL ate two characters of the hostname instead - `http://api.energy-charts.info/p` became the host `pi.energy-charts.info`, which then failed the ALLOW-LIST. So a non-TLS feed was refused, the check named "plain http is refused" passed, and deleting the TLS rule entirely left it passing. Review of PR #16 found that by mutation: an assertion that does not test what it names, inside a security check - this project's own recurring defect. Now the scheme is parsed as a scheme, the host of an `http://` URL comes out correct and DECLARED, and the TLS rule is the only thing that can refuse it - which is what makes the check able to fail. """ if not url: raise SourceUnavailable("no feed URL configured (price_feed_url)") scheme, sep, rest = url.partition("://") if not sep: raise SourceUnavailable( f"not an absolute URL, so no host can be checked against the " f"§16 allow-list: {url!r}") if scheme.lower() != "https": raise SourceUnavailable( f"refusing a non-TLS price feed: {url!r} uses {scheme!r}. §16 " f"allow-lists outbound destinations and SEC-01 requires TLS on " f"every one of them.") host = rest.split("/", 1)[0].split("?", 1)[0].split("@")[-1] host = host.split(":", 1)[0].lower() if host not in {h.lower() for h in caps.egress_hosts}: raise SourceUnavailable( f"{host!r} is not in this provider's declared egress_hosts " f"{caps.egress_hosts} - §16 allow-lists outbound destinations, and " f"a URL from the options file is not a declaration") return host # --------------------------------------------------------------------------- # # The shipped plugins. # # ⚠️ EACH DOCSTRING STATES WHICH COMPONENTS IT SOURCES AND WHICH THE FRAMEWORK # ADDS, in the two fixed lines `SOURCES: ...` / `The framework ADDS: ...`. That # is not decoration - it is the human-readable half of `capabilities().sources`, # and test_price.py PARSES those lines out of the docstring and compares them # with the declaration, so a plugin whose prose and whose code disagree about # what its feed contains is a red check rather than a 3 c/kWh error nobody sees. # --------------------------------------------------------------------------- # # TARIFF-08: the COLUMNAR document, which is what the public EPEX feed really # publishes. # # ⚠️ THIS IS THE ONLY PARSER LEFT, AND THE SHAPE IT DOES **NOT** READ IS WHY. # Until TARIFF-07 a second function, `_points_from_json()`, read a row-per-point # document: `{"data": [{...}, {...}]}`. NO FEED ANYBODY REACHED EVER PUBLISHED # THAT SHAPE - it was invented alongside the three withdrawn plugins, and the # only bodies ever fed to it were the fixtures written from the same guess. It # went with them. `https://api.energy-charts.info/price?bzn=BE` publishes # PARALLEL ARRAYS: # # {"unix_seconds": [1788991200, 1788992100, ...], <- UTC epoch seconds # "price": [216.08, 209.90, ...], <- one per instant # "unit": "EUR / MWh", "license_info": "...", "deprecated": false} # # The row parser was pointed at that URL and asked for `doc["data"]`, so # every fetch ended `SourceUnavailable: the feed carried no 'data' list` -> # §11.4 -> static profile, arbitrage off, and nothing in the status page said # why. Verified live 2026-09-10 against both fixtures under `scenarios/`. # # ⚠️ AND IT IS NOT ONE FUNCTION THAT SNIFFS FOR BOTH. A parser that decides # which document it got is a parser that can decide wrong, and a wrong-but- # parseable response then becomes a price. Two shapes, two functions, and each # plugin names the one its own feed publishes. # # The columnar shape was already known in this repository: DATA-02's # `scenarios/fetch_epex_be.py` has read `zip(d["unix_seconds"], d["price"])` # since 2026-08-31. Only the SHIPPED path never learned it, because every # fixture in the suite was built from the invented row shape - the parser was # graded against its own assumption. See test_price.py section 2b. # --------------------------------------------------------------------------- # COLUMNAR_TIME_KEY = "unix_seconds" COLUMNAR_PRICE_KEY = "price" COLUMNAR_UNIT_KEY = "unit" def _columnar_unit(text) -> str: """`"EUR / MWh"` -> `"eur/mwh"`. Spacing and case are not information.""" return "".join(str(text).split()).lower() def _points_from_columnar_json(body, expect_unit, scale, provisional_default): """Parse a COLUMNAR price document: parallel `unix_seconds` / `price`. Defects this refuses, each of which has a way of becoming a number rather than an error: * ⚠️ UNEQUAL ARRAYS ARE `SourceUnavailable`, NOT A SHORT SERIES. `zip()` stops at the shorter array and says nothing, so a `price` list that arrived truncated would produce a REAL price series with its tail missing - and the tail of a Belgian day is the evening peak. A series that is missing the expensive hours is worse than no series: §11.4's no-data row switches arbitrage OFF, whereas a truncated series is traded on. Nothing downstream can tell the two apart, so the refusal has to happen here. * ⚠️ A UNIT THAT IS NOT THE DECLARED ONE IS REFUSED. `scale` is the caller's standing assumption about the unit; if the feed ever publishes EUR/kWh under the same keys, every price silently comes out a THOUSAND times too small. `_validate_raw()` cannot catch that - `MAX_PLAUSIBLE_EUR_KWH` only bounds prices that are too LARGE. DATA-02's fetcher refuses on the same field for the same reason. * ⚠️ `unix_seconds` ARE UTC INSTANTS, SAID SO EXPLICITLY. `datetime.fromtimestamp(t, timezone.utc)`. Without that second argument CPython returns a NAIVE datetime in the HOST's zone, and a naive datetime is a wall clock reading plus an assumption about whose clock it was. `brussels()` and `_validate_raw()` both refuse one, so the mistake surfaces as a refused fetch rather than as a mispriced day - but they refuse ALL naive datetimes and can say nothing about whether this one was the right instant. That part is decided here. §9.3's rendering AT THE BRUSSELS OFFSET is `assemble()`'s, once, for every plugin - `brussels(p.start)` there is the single line that does it, and calling it here as well would be a second copy of a rule that already has an owner, indistinguishable from a no-op under mutation. A plugin's job is the right INSTANT; the framework's is the rendering. A `null` price is DROPPED and logged, not fatal, and that is a different judgement from the one above: an unequal pair of arrays means the pairing of instant to price is unreliable, whereas a null means the source stated it published no price for that one instant. A dropped instant leaves a hole, and `_Series.at()` already answers a hole with `Absence` - §11.4 for that hour. `scenarios/fetch_epex_be.py` dropped 0 nulls over the 12 months to 2026-08-31, so this is a shape the source allows rather than one it uses. """ try: text = body.decode("utf-8") if isinstance(body, (bytes, bytearray)) \ else str(body) doc = json.loads(text) except (ValueError, UnicodeDecodeError) as e: raise SourceUnavailable(f"the feed did not parse as JSON: {e}") if not isinstance(doc, dict): raise SourceUnavailable( f"the feed is a {type(doc).__name__}, not a columnar object with " f"{COLUMNAR_TIME_KEY!r} and {COLUMNAR_PRICE_KEY!r} arrays") times = doc.get(COLUMNAR_TIME_KEY) prices = doc.get(COLUMNAR_PRICE_KEY) for key, col in ((COLUMNAR_TIME_KEY, times), (COLUMNAR_PRICE_KEY, prices)): if not isinstance(col, list): raise SourceUnavailable( f"the feed carried no {key!r} array; this provider reads the " f"columnar shape (parallel {COLUMNAR_TIME_KEY}/" f"{COLUMNAR_PRICE_KEY} arrays), and a row-per-point document " f"is a different feed, not this one with a field missing") if len(times) != len(prices): raise SourceUnavailable( f"{COLUMNAR_TIME_KEY} has {len(times)} entries and " f"{COLUMNAR_PRICE_KEY} has {len(prices)}, so the feed is cut short " f"and none of it is used") unit = _columnar_unit(doc.get(COLUMNAR_UNIT_KEY, "")) if unit != _columnar_unit(expect_unit): raise SourceUnavailable( f"the feed reports {doc.get(COLUMNAR_UNIT_KEY)!r}, not " f"{expect_unit!r}, so its prices cannot be converted") if doc.get("deprecated"): # Not a refusal: today's prices are still prices. But §11.4 makes a # dead feed look exactly like a quiet one, so the warning that the # source is going away has to be emitted while it still works. _LOG.warning("The price feed says it will stop (deprecated=%r), " "though it still answers for now.", doc.get("deprecated")) out = [] dropped = [] for i, (t, v) in enumerate(zip(times, prices)): if v is None: dropped.append(i) continue if isinstance(t, bool) or not isinstance(t, (int, float)) or \ not math.isfinite(t): raise SourceUnavailable( f"{COLUMNAR_TIME_KEY}[{i}] is {t!r}, not epoch seconds") if isinstance(v, bool) or not isinstance(v, (int, float)) or \ not math.isfinite(v): raise SourceUnavailable( f"{COLUMNAR_PRICE_KEY}[{i}] is {v!r}, which is not a price") start = datetime.fromtimestamp(float(t), timezone.utc) out.append(RawPoint(start=start, eur_per_kwh=float(v) * scale, provisional=bool(provisional_default))) if dropped: _LOG.warning("The price feed gave no price for %d of %s, at index " "%s. Those are left out, not counted as free.", len(dropped), messages.count(len(times), "time"), ", ".join(str(i) for i in dropped[:5]) + (" and more" if len(dropped) > 5 else "")) if not out: raise SourceUnavailable( f"the feed carried {len(times)} instants and a price for none of " f"them") return out def dayahead_feed_url(plugin_id, opts, since, until, base, bzn) -> str: """The day-ahead URL for a window, or the operator's override, verbatim. ⚠️ SHARED BY `generic_dayahead` AND `custom_formula`, ON PURPOSE (TARIFF-15). They read the SAME feed for the same range; two copies of this rule would be two places for the Brussels-calendar derivation and the override decision to disagree, and the one that is wrong would be wrong silently - §11.4 turns a short horizon into provisional hours, not into an error. It takes `plugin_id` rather than reading one, so the framework's AC2 rule (no plugin id in shared code) still holds here. ⚠️ AN OVERRIDE IS THE EXACT URL AND NOTHING IS APPENDED TO IT, WHICH COSTS THE OVERRIDING SITE THE RANGE. That is a decision, not an oversight, and the sentence below is why it is not a footnote: * `price_feed_url` says "override the plugin's default feed", and an override that is silently rewritten is not one. Merging parameters into an operator's query string means parsing and re-serialising a URL they wrote - which reorders, de-duplicates and re-escapes it, and can hand the server a request nobody typed. * `start`/`end` are ENERGY-CHARTS' spelling, and both providers DECLARE A SECOND HOST: `web-api.tp.entsoe.eu`. Nothing here has verified that host's day-ahead parameters and nothing here should guess them - but they are not `start`/`end`, so an override to the other DECLARED host is exactly the case where appending our vocabulary adds two parameters that host ignores and yields the short horizon anyway, with nothing saying so. * and blind appending is refused outright: `dated_feed_url()` will not add a query string to a base that has one. So the cost is real and it is REPORTED, at fetch time, on the shipped logger, naming the horizon consequence rather than the substitution. §11.4 is what makes a price defect quiet; an operator who has taken the short horizon has to be able to find out from the log that they did. """ override = str(opts.get("price_feed_url") or "").strip() if override: _LOG.warning( "price_feed_url is set, so %s asks for %s as given, and " "tomorrow's prices may stay provisional. Clear it to ask for %s " "to %s.", plugin_id, override, *(d.isoformat() for d in brussels_date_range(since, until))) _LOG.debug( "The public feed answers one Brussels day to a request that names " "no range. Every later hour is then copied from yesterday and " "marked provisional, and §8.3 leaves provisional hours out of " "trading.") return override first, last = brussels_date_range(since, until) return dated_feed_url(base, bzn=bzn, start=first.isoformat(), end=last.isoformat()) @register class GenericDayAhead(PriceProvider): """Public EPEX BE day-ahead feed, COLUMNAR, in EUR/MWh. SOURCES: energy The framework ADDS: markup, taxes This is the plugin the double-counting trap is about. The published figure is the wholesale clearing price and nothing else, so both user-entered components have to be added to it - and because this plugin cannot read either number, it cannot be the one that adds them twice. ⚠️ IT READS THE COLUMNAR SHAPE, AND SAYS SO IN ITS OWN `fetch()`. Until TARIFF-08 it called the row-per-point parser (withdrawn by TARIFF-07) and asked `api.energy-charts.info` for a `"data"` list it has never published, so every fetch this provider ever made ended in `SourceUnavailable` and §11.4 turned that into a silent fall back to the static profile. See `_points_from_columnar_json()` for the shape and for why there are two parsers rather than one that guesses. ⚠️ THE FEED IS QUARTER-HOURLY SINCE BELGIUM'S 15-MINUTE MTU (2025-10-01): 96 points a day at 900 s, not 24 at 3600 s. Nothing here assumes either - the instants come from the document - and both granularities are replayed from real captured bodies in test_price.py section 2b. """ id = "generic_dayahead" @classmethod def capabilities(cls): return ProviderCapabilities( product=PriceProduct.DAY_AHEAD, sources=frozenset({Component.ENERGY}), egress_hosts=("api.energy-charts.info", "web-api.tp.entsoe.eu")) FEED_BASE = "https://api.energy-charts.info/price" BZN = "BE" async def fetch(self, opts, since, until, transport): """Request the range the CALLER asked for, on the Brussels calendar. ⚠️ THE ARGUMENTS ARE USED, and `dayahead_feed_url()` above is where they are turned into a URL. Until TARIFF-09 this method took `since` and `until` and asked for the bare `?bzn=BE`, which answers ONE Brussels day - so a 48 h horizon was 15 firm hours and 33 carried at 09:00, and 1 firm and 47 carried at 23:00, with the whole trade set inside a single calendar day at every hour of every day. ⚠️ THE URL DECISION MOVED OUT OF THIS METHOD IN TARIFF-15 AND DID NOT CHANGE. `custom_formula` reads the same feed with the same range and the same parser, and a second copy of the override rule would be a second place for it to be wrong - which is the reason `covers()` delegates to `at()` rather than restating its window. The reasoning, the warning and the exact URL are all in `dayahead_feed_url()`. """ url = dayahead_feed_url(self.id, opts, since, until, self.FEED_BASE, self.BZN) check_egress(url, self.capabilities()) body = await transport(url, float(opts.get("price_timeout_s"))) return _points_from_columnar_json(body, expect_unit="EUR / MWh", scale=0.001, provisional_default=False) @register class PriceService(PriceProvider): """The owner's own price server (PRICE-01), COLUMNAR, in EUR/MWh. SOURCES: energy The framework ADDS: markup, taxes The server answers the energy-charts layout (`unix_seconds`, `price`, `unit`) plus `covers`, the complete Brussels days it holds. The price is the wholesale energy price and nothing else, so markup and taxes are added by the framework exactly as for `generic_dayahead`. The host is fixed here, HTTPS only, and a bearer token (the option `price_service_token`) goes in the `Authorization` header. The token is not put into any message this class builds. `FALLBACKS` names the next source down. `app/price_chain.py` walks it and then the stored prices; this class only fetches. """ id = "price_service" HOST = "prices.kammenstraatha.duckdns.org" URL = "https://prices.kammenstraatha.duckdns.org/v1/dayahead/BE" FALLBACKS = ("generic_dayahead",) KEEPS_LAST_GOOD = True @classmethod def capabilities(cls): return ProviderCapabilities( product=PriceProduct.DAY_AHEAD, sources=frozenset({Component.ENERGY}), egress_hosts=(cls.HOST,)) async def fetch(self, opts, since, until, transport): token = str(opts.get("price_service_token") or "").strip() if not token: raise SourceUnavailable("no token is set for the price service") if any(c.isspace() or not c.isprintable() for c in token): raise SourceUnavailable( "the price service token has a space or a line break in it") check_egress(self.URL, self.capabilities()) try: body = await transport( self.URL, float(opts.get("price_timeout_s")), headers={"Authorization": "Bearer " + token}) except SourceUnavailable: raise except Exception as e: # noqa: BLE001 raise SourceUnavailable( f"the price service did not answer ({type(e).__name__})") points = _points_from_columnar_json(body, expect_unit="EUR / MWh", scale=0.001, provisional_default=False) covers = _covered_days(body) if covers is None: return points whole = [p for p in points if brussels(p.start).date().isoformat() in covers] if not whole: raise SourceUnavailable( "the price service sent no complete day") return whole def _covered_days(body): """The `covers` list of a price service answer as a set, or None if absent. A day the server does not list as complete is not used: an hour missing inside a day is a hole, and a hole is not a free hour. """ try: text = body.decode("utf-8") if isinstance(body, (bytes, bytearray)) \ else str(body) covers = json.loads(text).get("covers") except (ValueError, UnicodeDecodeError, AttributeError): return None if covers is None: return None if not isinstance(covers, list) or \ not all(isinstance(d, str) for d in covers): raise SourceUnavailable("the price service sent a bad list of days") return set(covers) @register class ManualCsv(PriceProvider): """A file on disk. AC3: the documented fallback, fully offline. SOURCES: energy, markup, taxes The framework ADDS: nothing ⚠️ THE CSV IS ALL-IN, AND DOCS.md says so. It has to be: this plugin has no way to know what the numbers a human typed already contain, and guessing would be the double-counting defect with a friendlier face. `load_prices()` warns when markup/taxes are configured non-zero under a plugin that sources them, so the ignored numbers are visible rather than silently discarded. ⚠️ NO TRANSPORT, NO aiohttp, NO NETWORK, NO tz DATABASE. It is the escape hatch for a site with no internet and the ingest path the TEST-01 replay harness will share, so it must not acquire a dependency on anything that needs one. `transport` is accepted and deliberately unused. Format - header required, two columns or three: start,eur_per_kwh,provisional 2026-01-15T00:00:00+01:00,0.1834,false """ id = "manual_csv" @classmethod def capabilities(cls): return ProviderCapabilities( product=PriceProduct.DAY_AHEAD, sources=ALL_IN, egress_hosts=()) # declared empty: this one reaches nothing async def fetch(self, opts, since, until, transport): path = opts.get("price_csv_path") try: with open(path, "r", encoding="utf-8") as fh: text = fh.read() except OSError as e: raise SourceUnavailable(f"cannot read {path}: {e}") return self.parse(text) @staticmethod def parse(text: str) -> list: """Split out so the offline check can drive it without a filesystem.""" rdr = csv.DictReader(io.StringIO(text)) if not rdr.fieldnames or "start" not in rdr.fieldnames or \ "eur_per_kwh" not in rdr.fieldnames: raise SourceUnavailable( f"the CSV needs a header with at least 'start' and " f"'eur_per_kwh'; found {rdr.fieldnames!r}") out = [] for n, row in enumerate(rdr, start=2): try: start = datetime.fromisoformat((row["start"] or "").strip()) value = float((row["eur_per_kwh"] or "").strip()) except (TypeError, ValueError) as e: raise SourceUnavailable(f"CSV line {n}: {e}") if start.tzinfo is None: raise SourceUnavailable( f"CSV line {n}: {row['start']!r} has no UTC offset. Write " f"'2026-01-15T00:00:00+01:00', not '2026-01-15 00:00' - a " f"local wall clock is ambiguous for one hour every October.") flag = (row.get("provisional") or "false").strip().lower() out.append(RawPoint(start=start, eur_per_kwh=value, provisional=flag in ("1", "true", "yes"))) return out @register class CustomFormula(PriceProvider): """The operator's OWN tariff card, as two fixed coefficients. TARIFF-15. SOURCES: energy, markup The framework ADDS: taxes ⚠️ IT DECLARES `{ENERGY, MARKUP}` AND NOT `ALL_IN`, AND THAT IS THE WHOLE SAFETY ARGUMENT. A Belgian card's `Tariefformule` line is the ENERGY component only - Eneco tabulates network tariffs and levies separately, ENGIE states that the formula excludes network, distribution, transport, grid levies and excise - so declaring ALL_IN would lie on behalf of every operator who typed exactly what their card says, and would silently drop ~12 c EUR/kWh from every hour. The framework adds `price_taxes_eur_kwh` on top, once, and this class cannot see or apply it (trap 1). The risk that introduces is one no guard in this file can see, because it is not in code: THE OPERATOR folding their network charges into `offset` while the tax option carries them too. `app/custom_formula.py`'s card self-check is what refuses that - the card's printed estimate is the energy component alone, so an inflated offset moves 15.55 to ~27 and the add-on will not start. Its limit is stated in the same breath there: it grades the coefficients against the card, NOT the card against reality. ⚠️ NO EXPRESSION EVALUATION OF ANY KIND, AND THE COEFFICIENTS ARE NUMBERS BECAUSE THE PARSER REFUSES ANYTHING ELSE. This string is settable over HTTP through the add-on's own ingress config page, so `eval` on it would be RCE in a root container holding SUPERVISOR_TOKEN - and, the deciding argument, an arbitrary expression could not be graded against the card at all. See app/custom_formula.py. ⚠️ THE INDEX PATH IS `generic_dayahead`'s, NOT A SECOND ONE. Same feed, same `dayahead_feed_url()` request on the Brussels calendar, same columnar parser, same declared egress hosts. A second parser would be a second failure mode for a document that already has one that works. """ id = "custom_formula" FEED_BASE = "https://api.energy-charts.info/price" BZN = "BE" # Written by fetch(), read by startup_preview(), and by nothing else. spec = None index_by_instant: dict = {} @classmethod def capabilities(cls): return ProviderCapabilities( product=PriceProduct.DAY_AHEAD, sources=frozenset({Component.ENERGY, Component.MARKUP}), egress_hosts=("api.energy-charts.info", "web-api.tp.entsoe.eu")) async def fetch(self, opts, since, until, transport): """Fetch the index, apply the card's arithmetic, return RawPoints. ⚠️ `since`/`until` ARE USED, through the shared URL builder - see `dayahead_feed_url()` and TARIFF-09 for what ignoring them costs. """ try: spec = parse_price_formula(opts.get("price_formula")) except FormulaInvalid as e: raise SourceUnavailable(f"price_formula: {e}") if spec is None: raise SourceUnavailable( "“Dynamic price source” (price_provider) is 'custom_formula', " "but “Your own tariff formula” (price_formula) is empty. " "Restart the add-on to check the settings again") url = dayahead_feed_url(self.id, opts, since, until, self.FEED_BASE, self.BZN) check_egress(url, self.capabilities()) body = await transport(url, float(opts.get("price_timeout_s"))) # ⚠️ `scale=1.0`, SO THESE RawPoints CARRY EUR/**MWh** IN A FIELD # SPELLED `eur_per_kwh`. That is a trap and it is contained on purpose: # the list is destructured on the very next line and never leaves this # method, so nothing downstream can read the wrong unit off it. The # alternative - scaling to EUR/kWh and multiplying by 1000 again to # feed the formula - is two conversions where the contract needs none, # and every extra conversion here is another place to lose a factor of # 1000. If this list ever DID escape, `_validate_raw()` would refuse it # at `MAX_PLAUSIBLE_EUR_KWH` rather than price a day on it. index = _points_from_columnar_json(body, expect_unit="EUR / MWh", scale=1.0, provisional_default=False) pairs = to_resolution([(p.start, p.eur_per_kwh) for p in index], spec.resolution) self.spec = spec self.index_by_instant = {t.astimezone(timezone.utc): v for t, v in pairs} # ⚠️ `provisional=False` FOR THE SAME REASON `generic_dayahead` SAYS # SO: the day-ahead auction is settled once published, and the columnar # document carries no per-point flag to read one from. FCST-04's # carry-forward is what marks the hours nobody published yet. return [RawPoint(start=t, eur_per_kwh=price_eur_kwh(spec, v), provisional=False) for t, v in pairs] def startup_preview(self, points) -> list: """`index EUR/MWh -> all-in EUR/kWh` for the first 24 h, once. ⚠️ THE INDEX COLUMN IS WHY THIS HOOK EXISTS. The all-in price alone says nothing about which of the two numbers is wrong; printed beside the index it came from, a factor of 1000 is one glance. See `preview_lines()` for the rows and the min/median/max/spread line. """ if self.spec is None or not points: return [] cut = points[0].start + timedelta(hours=PREVIEW_HOURS) rows = [] for point in points: if point.start >= cut: break index = self.index_by_instant.get( point.start.astimezone(timezone.utc)) if index is not None: rows.append((point.start, index, point.eur_per_kwh)) return preview_lines(self.spec, rows) # --------------------------------------------------------------------------- # # The one entry point # --------------------------------------------------------------------------- # # Provider ids whose startup preview has already been emitted in this process. # ⚠️ NOT A PLUGIN NAME IN THE FRAMEWORK: it is filled at runtime from whatever # id was resolved, and starts empty. AC2 is intact. _PREVIEWED: set = set() def _emit_startup_preview(plugin, pid: str, points) -> None: """Log a plugin's preview once per process, and never fail because of it. ⚠️ THE ID IS RECORDED BEFORE THE HOOK RUNS, DELIBERATELY. A preview that raises must not be retried every 30 minutes for the life of the add-on - that would turn a cosmetic defect into a log flood on the one path §11.4 already makes hard to read. """ if pid in _PREVIEWED: return _PREVIEWED.add(pid) try: lines = list(plugin.startup_preview(points)) except Exception as e: # noqa: BLE001 _LOG.warning("%s could not show its start-up preview (%s: %s). Prices " "are not affected.", pid, type(e).__name__, e) return for line in lines: _LOG.info("%s", line) async def load_prices(opts: dict, since: datetime, until: datetime, transport=None) -> PriceRead: """Resolve the configured provider, fetch, assemble all-in, return a read. ⚠️ RETURNS, NEVER RAISES, for every failure a site can actually have. §11.4 makes "no price data" a degradation row, and a degradation row that arrives as a traceback into the 30 s planning tick is a controller that stops regulating a battery because a supplier's TLS certificate expired. The one exception is `FrameworkOption`, which is re-raised: that is not a site condition, it is a plugin trying to read the markup, and turning it into "no price data" would hide the exact defect trap 1 exists to catch. """ pid = provider_id(opts) if not provider_configured(opts): return PriceRead.no_data( "no price feed is set (price_provider is 'off'), so the fixed " "tariff is used, the plan is estimated and there is no trading") cls = PROVIDERS.get(pid) if cls is None: return PriceRead.no_data( f"unknown price provider {pid!r}. The known ones are " f"{', '.join(sorted(PROVIDERS))}") caps = cls.capabilities() markup = float(price_option(opts, "price_markup_eur_kwh")) # ⚠️ TARIFF-17: THE TABLE WINS OVER THE SCALAR, AND validate.py REFUSES THE # CASE WHERE BOTH SAY SOMETHING. Two writers of one quantity, one of which # silently loses, is the shape validate.py exists to refuse - so the # precedence here is only ever exercised on a configuration that has # already been accepted, and it is spelled out rather than left implicit. taxes_key = "price_taxes_eur_kwh" try: table = taxes_profile(opts) except Exception as e: # noqa: BLE001 # Not silent, and not a traceback into the 30 s tick. validate.py # should have refused this at startup; if the add-on is running with it # anyway, the operator is told which key and what is wrong with it. _LOG.error("price_taxes_profile cannot be used (%s: %s), so no taxes " "are added to this price update. Fix the table or clear it.", type(e).__name__, e) _LOG.debug("price_taxes_eur_kwh is not used instead, because a " "silently different tax basis is the fault this setting " "exists to remove.") return PriceRead.no_data( f"price_taxes_profile cannot be used: {e}") if table is not None: taxes = table taxes_key = "price_taxes_profile" else: taxes = float(price_option(opts, "price_taxes_eur_kwh")) # ⚠️ Not a refusal, but not silent either. A plugin whose feed is already # all-in ignores these two numbers, and an operator who typed them has every # reason to believe they took effect. ignored = [n for c, n, v in ((Component.MARKUP, "price_markup_eur_kwh", markup), (Component.TAXES, taxes_key, taxes)) if c not in caps.adds and v] if ignored: _LOG.warning("%s publishes a figure that already contains %s, so the " "configured %s ignored - adding them would double-count", pid, "/".join(sorted(c.value for c in caps.sources - {Component.ENERGY})), " and ".join(ignored) + (" is" if len(ignored) == 1 else " are")) if transport is None: transport = https_get plugin = cls() try: raw = await plugin.fetch(PluginOptions(opts), since, until, transport) _validate_raw(raw, caps) points = assemble(caps, raw, markup, taxes) except FrameworkOption: raise # a plugin bug, not a site condition except SourceUnavailable as e: return PriceRead.no_data(scrub(f"{pid}: {e}", opts)) except Exception as e: # noqa: BLE001 - see the docstring return PriceRead.no_data( scrub(f"{pid}: {type(e).__name__}: {e}", opts)) _emit_startup_preview(plugin, pid, points) return PriceRead.series(points, caps.product, pid, datetime.now(timezone.utc))