glenn schrooyenandClaude
9ac3a9d2bc
A new price source, `price_provider: price_service`: the owner's price server,
one fixed HTTPS host, a bearer token (new password option
`price_service_token`), wholesale energy only so markup and taxes are added as
for generic_dayahead.
The chain is server, then energy-charts, then the prices the history store
kept. A step counts only if it has a price for now. Each step down and back up
is one log line. With every source down a site plans from what it saved.
No price for now: one status row, one Home plan line and one notification, in
English and Dutch, cleared when a price returns. With this provider the
maintenance charge gate reads the provider's prices and keeps the Home
Assistant rule: the current 15-minute price is at or below today's average.
The token is hidden on the settings page, refused when missing, and taken out
of any error text. Other providers, `off` and the Home Assistant pair take the
road they always took (scripted-day probe against 0ce7d35: one line differs,
the refusal that lists the known providers). See
DECISIONS/PRICE-01-own-price-server.md.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bdqr7PiaHcBnzPituu2yTC
2343 lines
115 KiB
Python
2343 lines
115 KiB
Python
"""TARIFF-02 - price provider plugins, and the one canonical shape they all emit.
|
|
|
|
WHERE THIS LIVES, AND WHY IT IS NOT WHERE THE SPEC SAYS. specs.md §2.2 puts this
|
|
at `ems/ingest/prices/`. That module layout does not exist and is not being
|
|
built; the product owner's decision of 2026-08-30 is that everything ships in
|
|
`goodwe_controller/app/`, next to arbiter.py, clamp.py, freeze.py and
|
|
validate.py. The BEHAVIOUR specs.md §9.3 / §8.3 / §11.4 describes is unchanged
|
|
and still authoritative - only the path moved.
|
|
|
|
WHAT THIS IS FOR, in one sentence: so that a new Belgian supplier is a new class
|
|
and one line in `PROVIDERS`, and the optimiser that eventually consumes prices
|
|
never learns a supplier's name.
|
|
|
|
THE CANONICAL FORMAT (§9.3), exactly three fields and no more:
|
|
|
|
{ "prices": [ { "start": "2026-01-15T00:00:00+01:00",
|
|
"eur_per_kwh": 0.1834,
|
|
"provisional": false } ] }
|
|
|
|
`start` is tz-aware at the Brussels offset for that instant, the price is EUR
|
|
per kWh and ALL-IN (energy + markup + taxes), and `provisional` is per point,
|
|
not per series (§8.3: unknown hours are flagged, and no irreversible action may
|
|
be taken on a provisional price).
|
|
|
|
|
|
THE THREE THINGS THIS MODULE IS SHAPED BY
|
|
=========================================
|
|
|
|
1. DOUBLE-COUNTING MARKUP IS REFUSED, NOT DISCOURAGED - GIVEN A CORRECT
|
|
DECLARATION.
|
|
`generic_dayahead` reads a raw EPEX figure and the user's markup and taxes
|
|
have to be added to it. Engie and TotalEnergies publish a figure that already
|
|
HAS the markup in it. Add the configured markup to that one too and every
|
|
arbitrage decision on that site is made against a price that is 3 c/kWh
|
|
wrong, in a direction nothing reports.
|
|
|
|
A comment saying "do not add markup here" is not a mechanism. So:
|
|
|
|
* A plugin does not return a price. It returns `RawPoint`s carrying the
|
|
figure ITS SOURCE PUBLISHED, and declares in `capabilities().sources`
|
|
which of {ENERGY, MARKUP, TAXES} that figure already contains.
|
|
* `assemble()` - the framework, never a plugin - adds exactly the
|
|
components the plugin declared it did NOT source, from
|
|
`price_markup_eur_kwh` / `price_taxes_eur_kwh`.
|
|
* ⚠️ A PLUGIN CANNOT REACH THOSE TWO NUMBERS BY ANY ROUTE, and the word
|
|
"any" was earned the hard way. `PluginOptions` raises `FrameworkOption`
|
|
on them, which closes the documented path - but review of PR #16 wrote a
|
|
double-counting plugin THREE more ways, one of them touching nothing but
|
|
`PRICE_OPTION_DEFAULTS`, a public dict in the module every plugin must
|
|
already import for `PriceProvider` and `register`. One closed door in a
|
|
room with three is not a guarantee. So `register()` now also REFUSES ANY
|
|
PLUGIN WHOSE OWN CODE MENTIONS A FRAMEWORK KEY - see `code_leaks()`.
|
|
|
|
⚠️ THE PRECISE SHAPE OF THE GUARANTEE, because an overclaim here is worse
|
|
than no claim: **the framework adds the complement exactly once, GIVEN A
|
|
CORRECT `sources` DECLARATION.** A plugin that declares `{ENERGY}` while its
|
|
feed is really all-in still comes out one markup high, and NO CODE CAN CATCH
|
|
THAT - the declaration is the plugin's unverifiable claim about somebody
|
|
else's tariff. What is mechanised is that a plugin cannot obtain the numbers
|
|
and cannot apply them; what remains human is whether it described its feed
|
|
truthfully. The docstring/declaration check in test_price.py verifies that
|
|
the plugin's two claims AGREE, not that either is true, and it is worth
|
|
knowing which of those two things you are reading.
|
|
|
|
2. THE PRODUCT TYPE IS DECLARED, NOT INFERRED FROM THE PLUGIN'S NAME.
|
|
§9.2 classes real-time / imbalance products separately from day-ahead ones
|
|
and §8.3 says those are treated as high-volatility with widened arbitrage
|
|
thresholds. If the optimiser had to write `if provider_id == "<supplier>"`
|
|
to know that, then adding a supplier would mean editing the optimiser -
|
|
which is precisely what AC2 forbids. So `capabilities().product` is a
|
|
`PriceProduct`, it rides along on the series, and `volatility_regime()`
|
|
reads the enum. A plugin id appears nowhere in this file except in
|
|
`PROVIDERS`, in `RETIRED_PROVIDERS`, and in the class that answers to it.
|
|
|
|
⚠️ NO SHIPPED PLUGIN DECLARES `PriceProduct.REALTIME` (TARIFF-07). All
|
|
three shipped plugins are day-ahead, so `_WIDE`,
|
|
`Uncertainty.REALTIME_PRODUCT` and `WIDEN_FACTOR`'s 2.5 currently have no
|
|
shipped declarer. They are KEPT
|
|
deliberately - see DECISIONS/TARIFF-07-withdrawing-three-plugins.md - and
|
|
test_price.py section 10 keeps them live by declaring REALTIME from a class
|
|
defined outside this file, which is AC2's own stimulus.
|
|
|
|
3. "NO PRICE DATA" IS A TYPE, NOT AN EMPTY LIST.
|
|
§11.4: no price data -> fall back to the static profile, mark the plan
|
|
provisional, NO ARBITRAGE. That is a different instruction from "prices
|
|
exist and they are all zero", under which arbitrage is merely pointless
|
|
rather than forbidden - and the two are indistinguishable if a failed fetch
|
|
returns `[]`, or `None`, or a dict a caller `.get()`s a default out of.
|
|
|
|
⚠️ THIS REPO SPENT 2026-08-30 REMOVING EXACTLY THAT DEFECT. `sim/riglock.py`
|
|
in the infra repo used to return a partial dict from `holder()`; TEST-04
|
|
rewrote it so a look at the lock is `observed / not_observed / gone` and only
|
|
the observed variant carries fields. `_Outcome` below is that shape,
|
|
deliberately, down to the fatal `.get()`. Read that file's `class Absence`
|
|
block before changing anything here.
|
|
|
|
Concretely:
|
|
|
|
* `PriceRead.series(...)` REFUSES AN EMPTY POINT LIST. A series with no
|
|
prices in it cannot be constructed, so "I have no prices" cannot arrive
|
|
wearing the shape of an answer.
|
|
* `PriceRead.no_data(reason)` carries `.reason` AND NOTHING ELSE. It has no
|
|
`.points` to iterate, no `.product` to switch on, no `.at()` to call, and
|
|
`.get("points", [])` raises. `min(read.points)` - the line that would
|
|
silently make "no data" mean "everything is free, charge now" - does not
|
|
run against it.
|
|
* `.match(*, series, no_data)` takes both handlers as REQUIRED keyword
|
|
arguments, so a caller who forgets the no-data case gets a TypeError at
|
|
the call site rather than a fall-through.
|
|
|
|
|
|
TIMEZONE, AND WHY THERE IS NO `zoneinfo` HERE
|
|
=============================================
|
|
§9.3's example is `+01:00`, and Belgium runs CET/CEST. Two constraints:
|
|
|
|
* The shipped image is Alpine with `py3-aiohttp` from apk and NO `tzdata`
|
|
package (goodwe_controller/Dockerfile, and .gitea/ci/Dockerfile is the same
|
|
base by design). `ZoneInfo("Europe/Brussels")` raises
|
|
`ZoneInfoNotFoundError` there. A price module that only works on a machine
|
|
with a tz database is a price module that fails on the client's Pi.
|
|
* Fixed offsets have no fold. Every instant here is carried as UTC and
|
|
RENDERED at the offset that applies to it, so the 02:00-03:00 hour that
|
|
happens twice in October is two different instants that render as
|
|
`02:00+02:00` and `02:00+01:00`. A naive-local-hour key collides those two
|
|
and misprices an hour of a 25-hour day; an implementation on a fixed
|
|
24-slot array is wrong on both transition days. test_price.py drives both
|
|
2026 transitions and counts the points rather than asserting 24.
|
|
|
|
The EU rule is fixed in law and needs no database: CEST from 01:00 UTC on the
|
|
last Sunday of March until 01:00 UTC on the last Sunday of October. Review of
|
|
PR #16 cross-checked this against `zoneinfo`'s Europe/Brussels over 149,040
|
|
hourly instants, 2020-2036 inclusive, and found ZERO mismatches - so the
|
|
residual risk here is not arithmetic, it is political: if the EU ever abolishes
|
|
the seasonal change, `brussels_offset()` needs editing. That is the same
|
|
exposure as a machine whose `tzdata` has gone stale, and it is one edit in one
|
|
function rather than a package upgrade.
|
|
"""
|
|
|
|
import ast
|
|
import csv
|
|
import inspect
|
|
import io
|
|
import json
|
|
import logging
|
|
import math
|
|
from abc import ABC, abstractmethod
|
|
from dataclasses import dataclass
|
|
from datetime import datetime, timedelta, timezone
|
|
from enum import Enum
|
|
from textwrap import dedent
|
|
from urllib.parse import urlencode
|
|
|
|
from . import messages
|
|
from .custom_formula import (
|
|
EXPORT_FORMULA_OPTION, FormulaInvalid, PREVIEW_HOURS, parse_export_formula,
|
|
parse_price_formula, preview_lines, price_eur_kwh, to_resolution)
|
|
|
|
_LOG = logging.getLogger("goodwe.price")
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# Brussels wall time, without a tz database. See the module docstring.
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
CET = timezone(timedelta(hours=1))
|
|
CEST = timezone(timedelta(hours=2))
|
|
|
|
|
|
def _last_sunday(year: int, month: int) -> int:
|
|
"""Day-of-month of the last Sunday. March and October both have 31 days."""
|
|
day = 31
|
|
while datetime(year, month, day).weekday() != 6: # 6 == Sunday
|
|
day -= 1
|
|
return day
|
|
|
|
|
|
def brussels_offset(instant: datetime) -> timezone:
|
|
"""The offset Belgium is on at this UTC instant. CET or CEST, never else."""
|
|
u = instant.astimezone(timezone.utc)
|
|
y = u.year
|
|
start = datetime(y, 3, _last_sunday(y, 3), 1, tzinfo=timezone.utc)
|
|
end = datetime(y, 10, _last_sunday(y, 10), 1, tzinfo=timezone.utc)
|
|
return CEST if start <= u < end else CET
|
|
|
|
|
|
def brussels(instant: datetime) -> datetime:
|
|
"""Render a UTC instant at the Brussels offset for that instant.
|
|
|
|
⚠️ Takes an INSTANT, and refuses a naive one. A naive datetime is not a
|
|
moment in time, it is a wall clock reading plus an assumption, and the
|
|
assumption is wrong for one hour twice a year.
|
|
"""
|
|
if instant.tzinfo is None or instant.utcoffset() is None:
|
|
raise ValueError(
|
|
f"brussels({instant!r}): a naive datetime is not an instant. Parse "
|
|
f"the source's offset, or say UTC explicitly - guessing here is how "
|
|
f"one hour of a transition day gets priced twice.")
|
|
return instant.astimezone(brussels_offset(instant))
|
|
|
|
|
|
def brussels_date_range(since: datetime, until: datetime) -> tuple:
|
|
"""The Brussels calendar days a half-open window [since, until) touches.
|
|
|
|
Returns `(first, last)`, two `date`s, BOTH INCLUSIVE - which is the shape
|
|
`api.energy-charts.info` takes: `?bzn=BE&start=2026-09-09&end=2026-09-10`
|
|
answered 192 quarter-hours across exactly those two local days, measured
|
|
2026-09-10T07:45:49Z and committed as
|
|
`scenarios/energy-charts-be-price-2026-09-09_2026-09-10-twoday.json`.
|
|
|
|
THE DEFECT THIS EXISTS TO PREVENT (TARIFF-09). `generic_dayahead.fetch()`
|
|
accepted `since` and `until` and requested the bare default URL, so the
|
|
feed answered ONE Brussels day. Everything past today was then carried
|
|
forward from yesterday's profile, flagged provisional, and §8.3 dropped all
|
|
of it from the trade set - so the optimiser never saw tomorrow as firm, at
|
|
any hour. Measured on the merged tree at 2026-09-10T09:00+02:00 over a 48 h
|
|
horizon: 15 firm whole hours and 33 carried, and both ends of the only
|
|
permitted cycle inside ONE calendar day. The ceiling was structural: at
|
|
hour H of the local day a one-day window gives at most 24-H firm hours and
|
|
24+H carried - 1/47 at 23:00, where the firm spread no longer even clears
|
|
the round-trip cost. That is not what FCST-04's fill is for; it covers the
|
|
pre-publication window until the ~13:00-14:00 CET auction, after which a
|
|
healthy feed hands over tomorrow in full.
|
|
|
|
⚠️ THE CALENDAR IS BRUSSELS, NEVER UTC, AND THIS IS `_bdate()`'s LESSON ONE
|
|
LAYER UP. `since`/`until` are INSTANTS; the API takes DATES, and the two are
|
|
not the same question. 22:30 UTC in summer is already tomorrow in Belgium,
|
|
so a range derived from `since.date()` asks for the wrong FIRST day for the
|
|
last one or two hours of every single day - and the day it drops is the one
|
|
the controller is standing in. price_history.py made this mistake at the
|
|
cache layer and documents the same arithmetic; here it decides what is
|
|
fetched at all, so getting it wrong loses the hours rather than misfiling
|
|
them.
|
|
|
|
⚠️ `until` IS EXCLUSIVE, SO LOCAL MIDNIGHT BELONGS TO THE DAY BEFORE IT.
|
|
A 48 h horizon from local midnight ends at local midnight two days later,
|
|
and that instant is the first tick of a day the caller did not ask for.
|
|
Taking `brussels(until).date()` unconditionally would request a THIRD day
|
|
on every whole-hour horizon. This feed answers such a request without
|
|
complaint, so nothing would report it - which is the reason to get it right
|
|
here rather than to rely on the server's tolerance: the range is a claim
|
|
about what the caller asked for, and it should be true.
|
|
|
|
⚠️ AND AN EMPTY OR INVERTED WINDOW COLLAPSES TO ONE DAY RATHER THAN TO A
|
|
BACKWARDS RANGE. `start` after `end` is a request no feed has a defined
|
|
answer for; some return everything, some return an error, and neither is a
|
|
thing to hand a battery controller.
|
|
"""
|
|
first = brussels(since).date()
|
|
edge = brussels(until)
|
|
last = edge.date()
|
|
if (edge.hour, edge.minute, edge.second, edge.microsecond) == (0, 0, 0, 0):
|
|
last = last - timedelta(days=1)
|
|
if last < first:
|
|
last = first
|
|
return first, last
|
|
|
|
|
|
def dated_feed_url(base: str, **params) -> str:
|
|
"""`base` plus a query string. Refuses a `base` that already carries one.
|
|
|
|
⚠️ THE REFUSAL IS THE POINT, NOT THE FORMATTING. Appending `&start=...` to
|
|
a URL that may already hold its own query string is how a plugin produces a
|
|
request nobody intended - `?bzn=BE?bzn=BE&start=...`, or a duplicated
|
|
parameter whose winner is the server's choice rather than ours. So the base
|
|
is required to be parameterless and the parameters are added exactly once,
|
|
by `urlencode`, which is also what keeps a date out of the escaping
|
|
question. A future edit that folds the query back into the constant is a
|
|
refused fetch with a sentence, not a silently malformed URL.
|
|
|
|
⚠️ IT IS NOT REACHED BY AN OPERATOR'S `price_feed_url`. See
|
|
`GenericDayAhead.fetch()`: an override is used exactly as given.
|
|
"""
|
|
if "?" in base or "#" in base:
|
|
raise SourceUnavailable(
|
|
f"the feed base {base!r} already carries a query string or "
|
|
f"fragment, and this builder appends parameters. Adding a second "
|
|
f"'?' produces a URL the server parses differently from the one "
|
|
f"anybody intended; put the fixed parameters in the call instead")
|
|
return f"{base}?{urlencode(params)}" if params else base
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# Absence, borrowed wholesale from sim/riglock.py (TEST-04).
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
class Absence(Exception):
|
|
"""A value was asked of a read that never observed one."""
|
|
|
|
|
|
class FrameworkOption(Exception):
|
|
"""A plugin reached for an option the FRAMEWORK owns. See trap 1."""
|
|
|
|
|
|
class SourceUnavailable(Exception):
|
|
"""A plugin could not get its data. Raised by plugins, caught by the
|
|
framework, and turned into `PriceRead.no_data(...)`.
|
|
|
|
⚠️ It must never escape into the control tick. §11.4's row is a returned
|
|
condition, not a traceback: the loop keeps regulating on P1 whatever the
|
|
price situation is, and a price fetch that could stop it would be a price
|
|
fetch that can stop the battery.
|
|
"""
|
|
|
|
|
|
class _Outcome:
|
|
"""One outcome of one read. The absent variant CARRIES NO FIELDS."""
|
|
|
|
kind = "?"
|
|
words = "?" # human phrasing. Cosmetic, never an input to a decision.
|
|
|
|
def __getattr__(self, name):
|
|
if name.startswith("__") and name.endswith("__"):
|
|
raise AttributeError(name)
|
|
raise Absence(
|
|
f"{type(self).__name__} has no {name!r}. This read did not observe "
|
|
f"a price series, and a value invented here is the bug this "
|
|
f"representation exists to prevent - 'no data' becoming 'free'. "
|
|
f"Handle the {self.kind!r} case explicitly; see match().")
|
|
|
|
def get(self, *_a, **_k):
|
|
raise Absence(
|
|
f"{type(self).__name__}.get() does not exist: a default is how a "
|
|
f"failed fetch becomes an observation. Use .match(...) and handle "
|
|
f"both outcomes.")
|
|
|
|
def __iter__(self):
|
|
# ⚠️ DELIBERATE. `for p in read:` and `list(read)` are the other two
|
|
# idioms that turn an absent read into an empty, harmless-looking loop.
|
|
raise Absence(
|
|
f"{type(self).__name__} is not iterable: an empty loop body is how "
|
|
f"'we have no prices' becomes 'we checked every hour and none were "
|
|
f"expensive'. Use .match(series=..., no_data=...).")
|
|
|
|
def __repr__(self):
|
|
return f"{type(self).__name__}({self.words!r})"
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# The vocabulary
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
class Component(Enum):
|
|
"""The three parts of an all-in Belgian consumption price (§9.3)."""
|
|
ENERGY = "energy"
|
|
MARKUP = "markup"
|
|
TAXES = "taxes"
|
|
|
|
|
|
ALL_IN = frozenset(Component)
|
|
|
|
|
|
class PriceProduct(Enum):
|
|
"""The §9.2 product table. The optimiser switches on THIS, never on a name."""
|
|
DAY_AHEAD = "dynamic_dayahead"
|
|
REALTIME = "realtime_imbalance"
|
|
STATIC = "static_profile"
|
|
FIXED = "fixed"
|
|
|
|
|
|
# ⚠️ §8.3: "Real-time / imbalance products are treated as high-volatility:
|
|
# arbitrage thresholds widen." This maps product -> regime, and it is the ONLY
|
|
# place in the shipped code that says which products are volatile. OPT-04 reads
|
|
# it; it does not re-derive it, and it certainly does not check plugin ids.
|
|
_WIDE = frozenset({PriceProduct.REALTIME})
|
|
|
|
|
|
def volatility_regime(product: PriceProduct) -> str:
|
|
""""wide" | "normal" - the arbitrage threshold regime for a product (§8.3).
|
|
|
|
Deliberately a plain string and deliberately tiny: OPT-04 owns what the
|
|
widened numbers ARE. This owns only which products get them, because that
|
|
is a property of the product and belongs with the product enum.
|
|
"""
|
|
if not isinstance(product, PriceProduct):
|
|
raise TypeError(f"volatility_regime() takes a PriceProduct, not "
|
|
f"{product!r} - a name is exactly what AC2 forbids.")
|
|
return "wide" if product in _WIDE else "normal"
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# FCST-03: ONE widening path, driven by a SET of active uncertainty sources.
|
|
#
|
|
# Two spec rows widen the same number by two different triggers:
|
|
#
|
|
# §9.2 / §8.3 real-time / imbalance product -> "arbitrage thresholds widen"
|
|
# §11.4 no weather data -> "widen arbitrage thresholds"
|
|
#
|
|
# They are one mechanism reached twice, not two mechanisms, so there is one
|
|
# function and a set of reasons. ⚠️ THE COMBINATION RULE IS `max`, NOT A
|
|
# PRODUCT: multiplying 2.5 by 1.5 would make two independent doubts compound
|
|
# into a threshold nothing on a Belgian tariff can ever clear, and an arbitrage
|
|
# strategy that can never fire is indistinguishable from one that is switched
|
|
# off. The widest single reason wins.
|
|
#
|
|
# ⚠️ HONEST ABOUT WHERE THE NUMBERS COME FROM. specs.md says thresholds widen
|
|
# and does not say by how much; there is no capture to derive a factor from,
|
|
# because no arbitrage strategy has ever run here. These are POLICY DEFAULTS,
|
|
# chosen so that (a) a real-time product needs a materially larger spread than
|
|
# a settled day-ahead one before a cycle is worth its wear, and (b) the two
|
|
# factors DIFFER, so `max`, `sum` and `product` are three distinguishable
|
|
# answers rather than the same number - a check that grades a combination rule
|
|
# where all the candidates agree grades nothing. When OPT-04 lands
|
|
# `cycle_cost_threshold_eur` (§10.1/§14) these become the natural second option
|
|
# beside it, in PRICE_OPTION_DEFAULTS and nowhere else (CFG-03).
|
|
class Uncertainty(Enum):
|
|
"""Why an arbitrage threshold is currently wider than the tariff alone says."""
|
|
REALTIME_PRODUCT = "realtime_product" # §9.2 row 2, §8.3
|
|
NO_WEATHER = "no_weather" # §11.4 "no weather data"
|
|
|
|
|
|
WIDEN_FACTOR = {
|
|
Uncertainty.REALTIME_PRODUCT: 2.5,
|
|
Uncertainty.NO_WEATHER: 1.5,
|
|
}
|
|
|
|
|
|
def uncertainty_of(product: PriceProduct) -> frozenset:
|
|
"""The uncertainty a PRODUCT contributes on its own (§9.2 row 2).
|
|
|
|
Derived from the declared product, never from a plugin id - the same rule
|
|
`volatility_regime()` exists for. A sixth supplier declaring
|
|
`PriceProduct.REALTIME` is widened without anyone editing this file.
|
|
"""
|
|
if not isinstance(product, PriceProduct):
|
|
raise TypeError(f"uncertainty_of() takes a PriceProduct, not "
|
|
f"{product!r} - a plugin name is what §9.3 forbids.")
|
|
return (frozenset({Uncertainty.REALTIME_PRODUCT})
|
|
if product in _WIDE else frozenset())
|
|
|
|
|
|
def widen_factor(sources) -> float:
|
|
"""The multiplier the widest active source asks for. 1.0 when there are none.
|
|
|
|
⚠️ A STRING IS REFUSED. `widen_factor({"realtime"})` would otherwise be a
|
|
silent 1.0 - a widening that reports itself as applied and is not, which is
|
|
the shape of defect this repo keeps finding.
|
|
"""
|
|
srcs = frozenset(sources)
|
|
bad = [s for s in srcs if not isinstance(s, Uncertainty)]
|
|
if bad:
|
|
raise TypeError(f"widen_factor() takes Uncertainty members, not {bad!r}. "
|
|
f"A string here would widen by 1.0 and say it widened.")
|
|
return max((WIDEN_FACTOR[s] for s in srcs), default=1.0)
|
|
|
|
|
|
def required_spread(base_eur_kwh: float, sources) -> float:
|
|
"""The spread an arbitrage cycle must clear, given what is uncertain (§8.3).
|
|
|
|
`base_eur_kwh` is the caller's own round-trip cost threshold - OPT-04 owns
|
|
that number (§10.1's `cycle_cost_threshold_eur`), this owns only how much
|
|
wider it gets and why.
|
|
"""
|
|
if isinstance(base_eur_kwh, bool) or not isinstance(base_eur_kwh, (int, float)) or not math.isfinite(base_eur_kwh) or base_eur_kwh < 0:
|
|
raise ValueError(
|
|
f"required_spread() takes a non-negative, finite base threshold in "
|
|
f"EUR/kWh, not {base_eur_kwh!r}. A negative or NaN base makes every "
|
|
f"spread clear it, which is arbitrage with the brake removed.")
|
|
return round(float(base_eur_kwh) * widen_factor(sources), 6)
|
|
|
|
|
|
@dataclass(frozen=True)
|
|
class PricePoint:
|
|
"""One canonical point. THREE FIELDS. §9.3 lists three; this has three."""
|
|
|
|
start: datetime # tz-aware, rendered at the Brussels offset
|
|
eur_per_kwh: float # ALL-IN: energy + markup + taxes
|
|
provisional: bool
|
|
|
|
def canonical(self) -> dict:
|
|
return {"start": self.start.isoformat(),
|
|
"eur_per_kwh": self.eur_per_kwh,
|
|
"provisional": self.provisional}
|
|
|
|
|
|
@dataclass(frozen=True)
|
|
class RawPoint:
|
|
"""What a plugin returns: the figure its source published, unmodified.
|
|
|
|
⚠️ NOT a PricePoint, and the type difference is the guard. A plugin cannot
|
|
hand back something already claiming to be all-in, because the only
|
|
constructor of `PricePoint` that `load_prices()` uses is `assemble()`.
|
|
"""
|
|
|
|
start: datetime # tz-aware instant
|
|
eur_per_kwh: float # as published, containing exactly caps.sources
|
|
provisional: bool
|
|
|
|
|
|
@dataclass(frozen=True)
|
|
class ProviderCapabilities:
|
|
"""What a plugin declares about itself, before anyone asks it for prices."""
|
|
|
|
product: PriceProduct
|
|
sources: frozenset # which Components the published figure contains
|
|
horizon_h: int = 48 # aligns with the §8.1/§8.2 forecast horizon
|
|
egress_hosts: tuple = () # §16 allow-list, declared not discovered
|
|
|
|
def __post_init__(self):
|
|
if not isinstance(self.product, PriceProduct):
|
|
raise ValueError(f"product must be a PriceProduct, not "
|
|
f"{self.product!r}")
|
|
if Component.ENERGY not in self.sources:
|
|
raise ValueError(
|
|
"capabilities().sources must contain Component.ENERGY - a feed "
|
|
"that does not publish an energy price is not a price feed.")
|
|
extra = set(self.sources) - set(ALL_IN)
|
|
if extra:
|
|
raise ValueError(f"unknown price components declared: {extra}")
|
|
|
|
@property
|
|
def adds(self) -> frozenset:
|
|
"""The components the FRAMEWORK must add. The complement of `sources`."""
|
|
return frozenset(ALL_IN - self.sources)
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# The read: a series, or no data. Nothing in between.
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
class PriceRead(_Outcome):
|
|
"""What one attempt to get prices said. Two outcomes, and only one answers.
|
|
|
|
* `series` - we have prices. Carries `.points` (NON-EMPTY, by
|
|
construction), `.product`, `.provider`, `.fetched_at`.
|
|
* `no_data` - we do not. Carries `.reason` and NOTHING ELSE. §11.4: static
|
|
profile, plan provisional, no arbitrage.
|
|
|
|
⚠️ A SERIES OF ALL-ZERO PRICES IS A `series`, NOT `no_data`, and that
|
|
distinction is the whole point. Zero prices mean arbitrage is pointless;
|
|
no data means arbitrage is FORBIDDEN. An implementation that returns `[]`
|
|
on a failed fetch cannot tell a caller which of those two it is in.
|
|
"""
|
|
|
|
def match(self, *, series, no_data):
|
|
"""Fold to a value. Both handlers required and keyword-only.
|
|
|
|
Handlers take one argument: this read. Required-ness is what makes the
|
|
no-data case impossible to forget rather than merely easy to remember.
|
|
"""
|
|
return {"series": series, "no_data": no_data}[self.kind](self)
|
|
|
|
@staticmethod
|
|
def series(points, product, provider, fetched_at):
|
|
return _Series(points, product, provider, fetched_at)
|
|
|
|
@staticmethod
|
|
def no_data(reason):
|
|
if not reason or not isinstance(reason, str):
|
|
raise ValueError(
|
|
"PriceRead.no_data() takes a reason a human can act on. It is "
|
|
"the only field this variant has; an empty one makes an "
|
|
"operator's only evidence a blank status line.")
|
|
return _NoData(reason)
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# How long ONE point covers - TARIFF-14
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
# ⚠️ A POINT DOES NOT CARRY ITS OWN DURATION AND MUST NOT LEARN TO. §9.3 lists
|
|
# THREE fields and `PricePoint` has three; a fourth would change the wire
|
|
# format every store, replay, fixture and captured body in this repository is
|
|
# written against. The span is DERIVED from the neighbours instead - here,
|
|
# once, at construction.
|
|
MAX_POINT_SPAN = timedelta(hours=1)
|
|
|
|
|
|
def _cover_table(points) -> tuple:
|
|
"""`(start_utc, end_utc, point)` per point, ascending. The `at()` index.
|
|
|
|
THE DEFECT THIS PREVENTS (TARIFF-14, HIGH). `at()` asked
|
|
`s <= u < s + 1 h` of every point IN LIST ORDER and returned the FIRST
|
|
match. On an hourly series - the only resolution any fixture carried
|
|
before TARIFF-08 - that is right by accident. Belgium moved to a
|
|
15-minute MTU on 2025-10-01 and the live day-ahead feed returns 96 points
|
|
at 900 s, where it is wrong twice over. Measured on shipped `5059e28`:
|
|
|
|
series: 02:00=0.10 02:15=0.11 02:30=0.12 02:45=0.13 03:00=0.14
|
|
|
|
at(02:15) -> 02:00 the :00 point's HOUR has not run out yet
|
|
at(03:15) -> 02:30 and from the second hour on it is not even the
|
|
top of the hour: the first point still inside
|
|
its own hour wins, so the error COMPOUNDS with
|
|
depth into the series
|
|
|
|
`arbitrage._act()` compares `here.start` against `v.cheapest.start` and
|
|
`v.dearest.start` to decide whether this interval is the buy or the sell
|
|
one. That identity test almost never matched the extremum, so the
|
|
strategy idled through exactly the interval it exists to trade - and read
|
|
`here.provisional` and §17's `price < 0` guard off the wrong point while
|
|
doing it. `static_tariff.py` emits hourly points, so static sites never
|
|
saw it; this is reachable only down the dynamic path TARIFF-08/09 made
|
|
work.
|
|
|
|
THE RULE, in one sentence:
|
|
|
|
A POINT COVERS FROM ITS OWN START UNTIL THE NEXT POINT'S START, AND
|
|
NEVER MORE THAN ONE HOUR.
|
|
|
|
* 900 s -> each point covers its own quarter, so `at(02:15)` is the
|
|
02:15 point;
|
|
* 3600 s -> each point covers its own hour, unchanged from shipped;
|
|
* MIXED -> EACH POINT TAKES ITS OWN GAP. FCST-04's carry-forward emits
|
|
HOURLY carried points beside QUARTER-HOURLY firm ones, and the soak
|
|
measured spacing `[900, 3600]` in 70 % of samples - so mixed is the
|
|
normal steady state, and a window derived from one global spacing
|
|
would be wrong THERE rather than in an edge case.
|
|
|
|
⚠️ WHY THE ONE-HOUR CAP, WHICH IS NOT DECORATION. FCST-04 LEAVES HOLES ON
|
|
PURPOSE: an hour with no published price AND no yesterday value is left
|
|
OUT of the series, and `price_forecast.py`'s module note names `at()`
|
|
raising `Absence` for it as the behaviour that sends the caller to the
|
|
static profile (§11.4 for that hour). Uncapped, "until the next point's
|
|
start" would stretch the point BEFORE a three-hour hole across the whole
|
|
hole and answer every instant in it with a stale price. Filling holes is
|
|
the defect this module exists to refuse, and it would have been
|
|
introduced by the fix for a different one.
|
|
|
|
⚠️ THE LAST POINT'S WINDOW IS DECIDED, NOT INHERITED. There is no next
|
|
start and nothing in §9.3 says how wide the final interval is, so THE
|
|
LAST POINT IS GIVEN THE SAME SPAN AS THE POINT BEFORE IT - one hour when
|
|
it is the only point. Three consequences, all intended:
|
|
|
|
* on a 96-point 900 s day the 23:45 point covers 23:45 .. 00:00, so
|
|
`at(23:50)` ANSWERS and `at(00:00)` raises `Absence`. A flat one-hour
|
|
tail would have answered 00:30 TOMORROW with today's last price,
|
|
which is reaching for a neighbour across a day boundary;
|
|
* on an hourly series the tail is an hour, exactly as it shipped, so no
|
|
static or hourly site changes behaviour;
|
|
* on a MIXED series whose last point is a carried HOURLY one behind
|
|
quarter-hourly firm ones, the tail is 900 s and the last 45 minutes
|
|
of the horizon raise `Absence`. That is the SAFE direction and it is
|
|
accepted rather than tolerated: the caller falls back to the static
|
|
profile (§11.4) instead of trading on a span the series never
|
|
demonstrated, and a carried point is provisional, so §8.3 forbids any
|
|
irreversible action resting on it regardless.
|
|
|
|
⚠️ ORDER IS ESTABLISHED HERE, NOT ASSUMED - and that is the difference
|
|
between this and the rule it replaces. `at()` now reads neighbours, so it
|
|
WOULD depend on sortedness; the table is therefore sorted by UTC start at
|
|
construction, and no caller's list order can reach it. `_validate_raw()`
|
|
rejects out-of-order and duplicate starts on the `load_prices()` path,
|
|
`price_history._parse_points()` re-sorts what it reads off disk, and
|
|
FCST-04's fill emits ascending - but `PriceRead.series()` itself has
|
|
never enforced any of that, and an assumption that holds today and is
|
|
unstated is how this class of defect recurs.
|
|
|
|
⚠️ NOTHING HERE RAISES ON A MALFORMED SERIES. A duplicated start - not
|
|
reachable through `load_prices()`, which refuses it - gives the earlier
|
|
list entry a ZERO-width window and the later one the real span, so one of
|
|
the pair becomes unreachable through `at()` and neither the 30 s control
|
|
path nor construction raises.
|
|
"""
|
|
rows = sorted(((p.start.astimezone(timezone.utc), i, p)
|
|
for i, p in enumerate(points)),
|
|
key=lambda r: (r[0], r[1]))
|
|
out = []
|
|
for n, (start, _i, point) in enumerate(rows):
|
|
if n + 1 < len(rows):
|
|
span = rows[n + 1][0] - start # -> the next point's start
|
|
elif n:
|
|
span = start - rows[n - 1][0] # -> the tail, see above
|
|
else:
|
|
span = MAX_POINT_SPAN # -> a series of one point
|
|
if span > MAX_POINT_SPAN:
|
|
span = MAX_POINT_SPAN
|
|
out.append((start, start + span, point))
|
|
return tuple(out)
|
|
|
|
|
|
class _Series(PriceRead):
|
|
kind = "series"
|
|
|
|
def __init__(self, points, product, provider, fetched_at):
|
|
points = list(points)
|
|
# ⚠️ THE REFUSAL THAT MAKES THE TYPE WORTH HAVING. An empty series is
|
|
# how "the fetch failed" gets to wear the shape of "we looked and there
|
|
# is nothing expensive today". It is not constructible.
|
|
if not points:
|
|
raise ValueError(
|
|
"PriceRead.series() refuses an empty point list. A fetch that "
|
|
"produced no points is PriceRead.no_data(reason) - a different "
|
|
"variant with different fields, on purpose. See §11.4.")
|
|
if not isinstance(product, PriceProduct):
|
|
raise ValueError(
|
|
f"product must be a PriceProduct, not {product!r}. The "
|
|
f"optimiser switches on this enum instead of on a plugin name; "
|
|
f"a string here reopens exactly the hardcoding AC2 forbids.")
|
|
self.points = tuple(points)
|
|
self.product = product
|
|
self.provider = provider
|
|
self.fetched_at = fetched_at
|
|
self.regime = volatility_regime(product)
|
|
# ⚠️ TARIFF-14. Built ONCE, here, so that `at()` scans an index whose
|
|
# order it established itself rather than one it hopes it was handed.
|
|
# See `_cover_table()` for the window rule and for what the tail does.
|
|
self._cover = _cover_table(self.points)
|
|
self.words = (f"{len(self.points)} points from {provider} "
|
|
f"({product.value}, {self.regime} thresholds)")
|
|
|
|
def canonical(self) -> dict:
|
|
"""The §9.3 wire format. Exactly `{"prices": [...]}`."""
|
|
return {"prices": [p.canonical() for p in self.points]}
|
|
|
|
def at(self, instant: datetime) -> PricePoint:
|
|
"""The point whose OWN interval covers `instant`, at ANY resolution.
|
|
|
|
⚠️ RETURNS A POINT OR RAISES; it never returns None. `series.at(t) or
|
|
fallback` is the same defaulting bug one level down, so the falsy
|
|
answer does not exist. Ask `covers()` first if you mean to branch.
|
|
|
|
⚠️ TARIFF-14: THE WINDOW IS NOT ONE HOUR WIDE ANY MORE, AND WAS NEVER
|
|
ENTITLED TO BE. It runs to the next point's start, capped at an hour -
|
|
`_cover_table()` above holds the rule, the tail decision and the
|
|
reproduction on a quarter-hourly series. A fixed `s + 1 h` here
|
|
returned the FIRST point still inside its own hour, which on the live
|
|
900 s feed is neither the covering point nor, past the first hour,
|
|
even the top of the hour.
|
|
|
|
A linear scan over the index, which is 192 entries on the shipped 48 h
|
|
horizon. This runs on the 30 s control path through
|
|
`arbitrage._point_at()`; a bisect would be correct and is not worth
|
|
the second way to be wrong at this size.
|
|
"""
|
|
u = instant.astimezone(timezone.utc)
|
|
for start, end, point in self._cover:
|
|
if start <= u < end:
|
|
return point
|
|
raise Absence(
|
|
f"no price point covers {u.isoformat()}; the series runs "
|
|
f"{self.points[0].start.isoformat()} .. "
|
|
f"{self.points[-1].start.isoformat()}. An instant outside the "
|
|
f"series - or inside a hole FCST-04 left because nothing "
|
|
f"published that hour - is an instant with no price, which is "
|
|
f"§11.4 for it, not a reason to reach for a neighbouring one.")
|
|
|
|
def covers(self, instant: datetime) -> bool:
|
|
"""Whether `at(instant)` would answer.
|
|
|
|
⚠️ SHARED TARIFF-14 EXACTLY, AND IS FIXED BY THE SAME CHANGE. It has
|
|
no window rule of its own - it asks `at()` - so on the 900 s feed it
|
|
answered True for instants no point covered and False for none of
|
|
them, because the hour-wide window is never SHORTER than the truth.
|
|
It is kept as a delegation for that reason: a second copy of the rule
|
|
is a second place for it to be wrong.
|
|
"""
|
|
try:
|
|
self.at(instant)
|
|
return True
|
|
except Absence:
|
|
return False
|
|
|
|
def last_end_before(self, instant: datetime):
|
|
"""When the last price that has run out by `instant` stopped covering,
|
|
or None if no price has (PRICE-01: "no prices since 00:00").
|
|
|
|
Each point's end is the one `_cover_table()` worked out, so a hole in
|
|
the series is reported from where the hole starts and not from the end
|
|
of the day.
|
|
"""
|
|
u = instant.astimezone(timezone.utc)
|
|
ends = [end for _start, end, _p in self._cover if end <= u]
|
|
return max(ends) if ends else None
|
|
|
|
def window(self, since: datetime, until: datetime) -> tuple:
|
|
"""Points whose START falls in [since, until). May legitimately be empty.
|
|
|
|
⚠️ DID NOT SHARE TARIFF-14, AND THE REASON IS WORTH STATING RATHER
|
|
THAN REDISCOVERING. This grades a point's `start` against a half-open
|
|
range and never asks how long a point lasts, so it carries no
|
|
resolution assumption to be wrong about - the same call answers the
|
|
same set at 900 s, at 3600 s and mixed. It is therefore NOT a
|
|
coverage query: a point that starts before `since` and runs into it
|
|
is absent from this tuple by design. `at()`/`covers()` are the
|
|
coverage questions.
|
|
"""
|
|
a = since.astimezone(timezone.utc)
|
|
b = until.astimezone(timezone.utc)
|
|
return tuple(p for p in self.points
|
|
if a <= p.start.astimezone(timezone.utc) < b)
|
|
|
|
|
|
class _NoData(PriceRead):
|
|
"""We have no prices, and the §11.4 row applies.
|
|
|
|
⚠️ THIS CLASS HAS NO `.points` AND THAT IS THE POINT. `min(read.points)`,
|
|
`for p in read.points`, `read.get("points", [])` and `len(read.points)` all
|
|
raise `Absence` here rather than quietly reporting that the cheapest hour of
|
|
the day costs nothing.
|
|
|
|
⚠️ TWO WAYS PAST IT SURVIVE, BOTH FOUND IN REVIEW OF PR #16, AND NEITHER IS
|
|
WORTH CODE:
|
|
|
|
* `vars(read).get("points", [])` returns `[]`. `vars()` bypasses
|
|
`__getattr__` entirely; nothing short of `__slots__` plus a metaclass
|
|
would stop it, and anyone writing `vars()` on a variant type has left
|
|
the road on purpose.
|
|
* `bool(read)` is `True` for BOTH variants, so `if read:` looks like a
|
|
check and is not one. Defining `__bool__` to be False here would make
|
|
`if read:` "work" - and that is exactly the wrong fix: it would turn a
|
|
line that reads like a guard into one that IS a guard by accident,
|
|
which is how `.get(k, default)` got into this codebase in the first
|
|
place. Use `.match(...)`. There is no falsy variant on purpose.
|
|
"""
|
|
|
|
kind = "no_data"
|
|
|
|
def __init__(self, reason):
|
|
self.reason = reason
|
|
self.words = f"no price data - {reason}"
|
|
|
|
|
|
def arbitrage_allowed(read: PriceRead) -> bool:
|
|
"""§11.4, stated once, as a function so no caller has to remember it.
|
|
|
|
Not the arbitrage strategy - that is OPT-04. This is only the veto, which
|
|
belongs with the type that expresses the condition.
|
|
"""
|
|
return read.match(series=lambda _r: True, no_data=lambda _r: False)
|
|
|
|
|
|
def plan_is_provisional(read: PriceRead) -> bool:
|
|
"""§8.3 / §11.4: provisional if there is no data at all, or if ANY point in
|
|
the series is itself provisional. Derived from the points, never asserted."""
|
|
return read.match(
|
|
series=lambda r: any(p.provisional for p in r.points),
|
|
no_data=lambda _r: True)
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# Options: what a plugin may see, and what it may not
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# TARIFF-18: what `price_taxes_eur_kwh` IS MADE OF
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
# ⚠️ THE DEFECT THIS TABLE PREVENTS IS THE ONE THE SHIPPED 0.06 WAS: A NUMBER
|
|
# NOBODY COULD SOURCE. 0.06 EUR/kWh is roughly HALF the real Flemish non-energy
|
|
# figure, and because no reader could see what it was made of there was nothing
|
|
# to argue with. Every row below is one line of the Eneco residential card for
|
|
# Flanders, September 2026, INCLUDING 6 % VAT, converted from the c EUR/kWh the
|
|
# card prints to the EUR/kWh this option takes. `price_taxes_eur_kwh`'s default
|
|
# is their sum, and test_price.py section 1b re-adds them and refuses a default
|
|
# that is not exactly that.
|
|
#
|
|
# ⚠️ THIS IS A PER-DSO FIGURE, AND IT IS THE DISTRIBUTION ROW THAT MOVES. Across
|
|
# the eight Fluvius DSOs that row runs 5,28 c EUR/kWh (Midden-Vlaanderen) to
|
|
# 6,70 (West), so the all-in Flemish total runs about 0.117 to 0.131 EUR/kWh.
|
|
# A site outside Antwerpen should type its own total; the other two rows are
|
|
# Flanders-wide and do not move with the DSO.
|
|
#
|
|
# ⚠️ AND THE DEFAULT SITS NEAR THE LOW END OF THAT BAND ON PURPOSE, BECAUSE THE
|
|
# TWO ERROR DIRECTIONS ARE NOT SYMMETRIC. This scalar is a LEVEL component: it
|
|
# cancels exactly in `max - min` and cannot move which hour is cheapest
|
|
# (test_price.py 16b measures that as a family). The one DECISION it reaches is
|
|
# §17's negative-price guard in arbitrage.py, which tests the ABSOLUTE all-in
|
|
# price of the sell hour. A figure set too HIGH carries a genuinely negative
|
|
# hour up above zero, and the add-on exports into paid-for disposal - the exact
|
|
# case §17 exists to refuse. A figure set too LOW only declines a trade that
|
|
# would have paid. So where a default must stand in for eight different DSOs,
|
|
# it belongs at the bottom of the band and not in the middle of it.
|
|
#
|
|
# ⚠️ `energiebijdrage` IS ZERO, AND IT IS A ROW HERE RATHER THAN AN ABSENCE.
|
|
# "Vanaf 1 augustus 2026 bedraagt de energiebijdrage 0 EUR/MWh" - it was merged
|
|
# into the bijzondere accijns. Anyone re-deriving this total from a guide
|
|
# written before that date will find a separate energiebijdrage line and add it
|
|
# on top of an accijns that already contains it. The zero row is here so that
|
|
# the double-count is visible as a decision rather than as an omission.
|
|
FLEMISH_TAXES_COMPONENTS_EUR_KWH = (
|
|
("Fluvius distribution, Antwerpen", 0.0535),
|
|
("Bijzondere accijns, 'Alle verbruik' - no volume bands", 0.04876),
|
|
("Bijdrage groene stroom en WKK Vlaanderen", 0.0151),
|
|
("Energiebijdrage - 0 EUR/MWh since 2026-08-01, inside the accijns", 0.0),
|
|
)
|
|
|
|
# ⚠️ TARIFF-18, AND IT IS NOT HOUSEKEEPING - IT DECIDES WHICH CONFIGS START.
|
|
# TARIFF-17's rung in validate.py refuses a site that sets `price_taxes_profile`
|
|
# AND leaves `price_taxes_eur_kwh` at something other than the shipped default,
|
|
# because the table wins and the scalar would decide nothing. It compares
|
|
# against the DEFAULT rather than against presence, because the Supervisor sets
|
|
# `App.options = deepmerge(config.yaml defaults, stored options)` and a saved
|
|
# blob therefore carries every schema key whether the operator typed it or not
|
|
# (measured: the reference install's export is 70 of 71 keys - CHANGELOG 0.9.0,
|
|
# `tools/supervisor_selfapi_probe.py` §D).
|
|
#
|
|
# So moving the default turns a previously ACCEPTED configuration into
|
|
# `SystemExit(78)`. A site that stored 0.06 - which is every install that has
|
|
# ever saved its configuration - and that also carries a taxes table would stop
|
|
# starting on the upgrade, over a number it never typed and that decides
|
|
# nothing. That refusal would be TRUE and USELESS: the rung's subject is a
|
|
# number the OPERATOR typed, and a superseded shipped default is not one. It
|
|
# still reads as "untouched" here, and the rung's sentence names both figures.
|
|
SUPERSEDED_TAXES_DEFAULTS_EUR_KWH = (0.06,) # TARIFF-02 .. TARIFF-17
|
|
|
|
|
|
# ⚠️ CFG-03, AND HOW THIS TICKET AVOIDS WIDENING IT. Option defaults in this
|
|
# add-on already live in three places - config.yaml, main.py's `o.get(k, X)`
|
|
# fallbacks, and a dataclass default - held apart only by a tripwire in
|
|
# test_validate.py. This module adds ONE more location, not another three:
|
|
#
|
|
# * every price option's fallback is here, and only here;
|
|
# * validate.py imports this dict rather than restating any number;
|
|
# * main.py gets no `o.get("price_...", X)` fallbacks at all;
|
|
# * there is no dataclass carrying a default copy of any of these;
|
|
# * test_price.py READS config.yaml and asserts the shipped default equals
|
|
# this dict, key for key, so the two cannot drift silently.
|
|
#
|
|
# config.yaml remains the source of truth. This is the single in-code mirror,
|
|
# pinned to it by a check, which is the `validate.py` pattern rather than a
|
|
# fourth freehand copy.
|
|
PRICE_OPTION_DEFAULTS = {
|
|
"price_provider": "off",
|
|
"price_markup_eur_kwh": 0.03, # §14 tariffs.energy_price.markup_eur_kwh
|
|
# TARIFF-18: the sum of FLEMISH_TAXES_COMPONENTS_EUR_KWH above, to the
|
|
# card's own last digit. §14 tariffs.energy_price.taxes_eur_kwh.
|
|
"price_taxes_eur_kwh": 0.11736,
|
|
"export_price_eur_kwh": 0.04, # §14 tariffs.export_price_eur_kwh (AC4)
|
|
"price_csv_path": "/share/goodwe_prices.csv",
|
|
"price_feed_url": "",
|
|
# PRICE-01. The bearer token of the owner's price server. A secret: it is
|
|
# in SECRET_OPTIONS below, so no text built from an error reaches a log,
|
|
# the status page or a notification with it in.
|
|
"price_service_token": "",
|
|
"price_horizon_h": 48,
|
|
"price_timeout_s": 10,
|
|
# TARIFF-03. §14 `tariffs.static_profile: null`. A JSON object describing a
|
|
# static day/night or fully-fixed tariff; "" is §14's `null`. Parsed and
|
|
# validated by app/static_tariff.py, refused at startup by validate.py.
|
|
#
|
|
# ⚠️ IT LIVES HERE RATHER THAN IN static_tariff.py FOR CFG-03's SAKE. This
|
|
# dict is the ONE in-code mirror of config.yaml, pinned to it by
|
|
# test_price.py section 1. A second defaults dict in the new module would be
|
|
# a fourth place option defaults live, which is exactly the open ticket.
|
|
"static_profile": "",
|
|
# TARIFF-17. A JSON object in `static_profile`'s OWN period grammar whose
|
|
# per-period `eur_per_kwh` is the TAXES COMPONENT for that band, not an
|
|
# all-in price; "" is unset and the scalar above decides. Parsed and
|
|
# validated by app/static_tariff.py (`parse_taxes_profile`), refused at
|
|
# startup by validate.py - the `static_profile` precedent exactly, and here
|
|
# for the same CFG-03 reason: this dict is the ONE in-code mirror of
|
|
# config.yaml.
|
|
#
|
|
# ⚠️ IT IS IN FRAMEWORK_KEYS. A plugin that could read this could add the
|
|
# taxes a second time, per hour, which is the double-counting defect trap 1
|
|
# exists to refuse - wearing a period table instead of a scalar.
|
|
"price_taxes_profile": "",
|
|
# TARIFF-15. A JSON object carrying ONE contract's two coefficients, the
|
|
# units they are printed in, its VAT rate and the card figures the add-on
|
|
# grades them against; "" is unset. Parsed and validated by
|
|
# app/custom_formula.py, refused at startup by validate.py - the
|
|
# `static_profile` precedent exactly, and here for the same CFG-03 reason:
|
|
# this dict is the ONE in-code mirror of config.yaml.
|
|
"price_formula": "",
|
|
# TARIFF-16. The INJECTION leg, in `price_formula`'s OWN grammar and read
|
|
# by `custom_formula.parse_export_formula()` - the same parser with its
|
|
# option name as an argument, so a second unit table and a second unknown-
|
|
# key rule do not exist to diverge. "" is unset and the scalar
|
|
# `export_price_eur_kwh` above decides, which is every site today.
|
|
#
|
|
# ⚠️ IT IS IN FRAMEWORK_KEYS, for the reason the scalar is: a CONSUMPTION
|
|
# plugin that could read an injection curve could fold it into a
|
|
# consumption price, and nothing downstream could tell.
|
|
#
|
|
# ⚠️ AND IT LIVES HERE RATHER THAN IN custom_formula.py FOR CFG-03's SAKE,
|
|
# exactly as `price_formula` and `price_taxes_profile` do: this dict is the
|
|
# ONE in-code mirror of config.yaml.
|
|
"price_export_formula": "",
|
|
# OPT-04. §10.1 priority 3, and §14's `optimizer.cycle_cost_threshold_eur`.
|
|
#
|
|
# ⚠️ THEY LIVE HERE FOR CFG-03's SAKE, exactly as `static_profile` does, and
|
|
# FCST-03's note above `WIDEN_FACTOR` names this as the place they would
|
|
# land. `arbitrage_soc_floor` / `arbitrage_soc_ceiling` are the reserve
|
|
# arbitrage may not spend and the headroom it may not overfill; they are
|
|
# NOT the maintenance SoC keys, which describe a balancing cycle rather
|
|
# than a trading reserve and would silently couple two unrelated policies.
|
|
#
|
|
# ⚠️ AND `cycle_cost_threshold_eur` IS 0.10 BECAUSE §10.1's NAME FOR IT SAYS
|
|
# SO, NOT BECAUSE IT MAKES ANYTHING TRADE. On realistic Belgian day/night
|
|
# numbers the static spread is ~0.08 EUR/kWh, so a static-tariff site does
|
|
# NOT trade at this default. That is a reported result (test_arbitrage.py
|
|
# section 7), and whether 0.10 - or FCST-03's 2.5/1.5 - is right is one
|
|
# product decision that belongs to the owner. No number here was chosen to
|
|
# make arbitrage happen.
|
|
"arbitrage_enabled": False,
|
|
"cycle_cost_threshold_eur": 0.10,
|
|
"arbitrage_soc_floor": 20,
|
|
"arbitrage_soc_ceiling": 95,
|
|
}
|
|
|
|
# ⚠️ The keys the FRAMEWORK owns. A plugin that reads one of these is a plugin
|
|
# that can double-count, so `PluginOptions` refuses them by name. Trap 1.
|
|
FRAMEWORK_KEYS = frozenset({
|
|
"price_markup_eur_kwh", "price_taxes_eur_kwh", "export_price_eur_kwh",
|
|
# TARIFF-17. The hour-varying spelling of `price_taxes_eur_kwh`, and it
|
|
# belongs here for the same reason the scalar does: it is a component the
|
|
# FRAMEWORK adds exactly once, by `caps.adds`. `BANNED_IN_PLUGIN_CODE` is
|
|
# derived from this set, so `code_leaks()` refuses a plugin that names it
|
|
# without anything else being edited.
|
|
"price_taxes_profile",
|
|
# TARIFF-16. The hour-varying spelling of `export_price_eur_kwh`, and it
|
|
# belongs here for the same reason the scalar does: a plugin that could
|
|
# read an INJECTION curve could fold it into the CONSUMPTION series it
|
|
# emits, and `assemble()` has no way to notice a component that arrived
|
|
# inside a RawPoint. `BANNED_IN_PLUGIN_CODE` is derived from this set, so
|
|
# `code_leaks()` refuses a plugin that names it with nothing else edited.
|
|
EXPORT_FORMULA_OPTION,
|
|
})
|
|
|
|
|
|
# PRICE-01. Options whose VALUE must never be printed. `scrub()` removes them
|
|
# from any text the price code builds out of an error.
|
|
SECRET_OPTIONS = frozenset({"price_service_token"})
|
|
|
|
|
|
def scrub(text, opts) -> str:
|
|
"""`text` with every secret option's value replaced by `***`.
|
|
|
|
The price code builds reasons out of library errors, and a library may put
|
|
a request header into its error text. Taking the value out here is the
|
|
last line of defence; the plugin also never puts it in a message itself.
|
|
"""
|
|
out = str(text)
|
|
for key in SECRET_OPTIONS:
|
|
secret = str((opts or {}).get(key) or "").strip()
|
|
if len(secret) >= 4:
|
|
out = out.replace(secret, "***")
|
|
return out
|
|
|
|
|
|
class PluginOptions:
|
|
"""The options dict, minus the keys only the framework may read.
|
|
|
|
Reading `price_markup_eur_kwh` here raises `FrameworkOption` rather than
|
|
returning 0.03.
|
|
|
|
⚠️ THIS IS ONE OF THE TWO DOORS, NOT THE ONLY ONE. `_o` is reachable, and
|
|
`price_option()` and `PRICE_OPTION_DEFAULTS` are public names in this
|
|
module. This class alone is a convention; it is `register()`'s
|
|
`code_leaks()` scan that makes it an inability. Do not weaken either half
|
|
on the strength of the other.
|
|
"""
|
|
|
|
def __init__(self, opts: dict):
|
|
self._o = dict(opts or {})
|
|
|
|
def get(self, key, default=None):
|
|
if key in FRAMEWORK_KEYS:
|
|
raise FrameworkOption(
|
|
f"{key!r} belongs to the framework, not to a plugin. A plugin "
|
|
f"returns RawPoints carrying what its SOURCE published and "
|
|
f"declares capabilities().sources; assemble() adds the missing "
|
|
f"components exactly once. If a plugin could read this number "
|
|
f"it could add it a second time, and the resulting price would "
|
|
f"be wrong by one markup with nothing anywhere reporting it.")
|
|
v = self._o.get(key, None)
|
|
if v is None or v == "":
|
|
return PRICE_OPTION_DEFAULTS.get(key, default)
|
|
return v
|
|
|
|
def __getitem__(self, key):
|
|
return self.get(key)
|
|
|
|
def __contains__(self, key):
|
|
return key not in FRAMEWORK_KEYS and key in self._o
|
|
|
|
|
|
def price_option(opts: dict, key: str):
|
|
"""A framework-side option read. One place, defaults from one dict."""
|
|
if key not in PRICE_OPTION_DEFAULTS:
|
|
raise KeyError(f"{key!r} is not a price option; add it to "
|
|
f"PRICE_OPTION_DEFAULTS and to config.yaml together.")
|
|
v = (opts or {}).get(key, None)
|
|
if v is None or v == "":
|
|
return PRICE_OPTION_DEFAULTS[key]
|
|
return v
|
|
|
|
|
|
def provider_id(opts: dict) -> str:
|
|
"""The selected plugin id, normalised. `"off"` when none is selected.
|
|
|
|
⚠️ ONE SPELLING OF THIS EXPRESSION, AND TARIFF-10 IS WHY IT IS A FUNCTION.
|
|
`str(price_option(opts, "price_provider") or "off").strip()` was written out
|
|
twice - in `load_prices()` below and in `static_tariff.resolve_tariff()` -
|
|
and main.py's startup gate would have been a third. "Is a provider
|
|
configured" is now a DECISION the add-on takes (it is what starts the
|
|
background refresh), and a decision spelled out at three call sites is three
|
|
places to disagree about whether `" off "` or `None` counts.
|
|
⚠️ `validate.py` deliberately keeps its own reader: it validates the RAW
|
|
options blob, including one the Supervisor never saw, so it must not route
|
|
through `price_option()`'s defaults. Note what this does NOT do: it does not
|
|
decide
|
|
whether the id names a shipped plugin - `load_prices()` and validate.py's
|
|
rule 7 own that, and answering "unknown" here would turn a typo into
|
|
"no provider" instead of into a startup refusal.
|
|
"""
|
|
return str(price_option(opts, "price_provider") or "off").strip()
|
|
|
|
|
|
def provider_configured(opts: dict) -> bool:
|
|
"""Has the operator selected a price feed at all? (TARIFF-10)
|
|
|
|
THE DEFECT THIS PREVENTS: the background tariff refresh used to start only
|
|
when `arbitrage_enabled` was on, which ships `false` - so a site with a
|
|
provider configured fetched NOTHING, ever. `/data/price_history.json` was
|
|
never created, the *Price history* status row read `empty` forever, and
|
|
TARIFF-09's short-horizon warning could not fire because no fetch happened
|
|
to warn about. Measured over a 40 h two-instance soak: zero feed requests
|
|
and `present: false` across all 802 samples on the shipped default, against
|
|
a resolved plan on the same feed with arbitrage armed.
|
|
|
|
⚠️ THIS IS NOT A TRADING SWITCH AND MUST NEVER BECOME ONE. It answers
|
|
"fetch prices?"; `arbitrage.enabled()` answers "move energy on them?", and
|
|
§10.1's priority 3 is gated on the second alone. See
|
|
`main.tariff_refresh_wanted()` for the one place the two meet.
|
|
"""
|
|
return provider_id(opts) not in ("", "off")
|
|
|
|
|
|
def export_price_eur_kwh(opts: dict) -> float:
|
|
"""AC4 / §9.3: the injection price is a SEPARATE SCALAR, not a second series.
|
|
|
|
In Belgium it is often near zero, which by itself favours self-consumption
|
|
over export. It is deliberately not routed through the plugin framework -
|
|
but ⚠️ NOT because nothing dynamic exists to route. That claim stood here
|
|
until TARIFF-07 and it was FALSE:
|
|
|
|
* Eneco publishes an injection formula on the same tariff card as its
|
|
consumption formula - `0,1 x BELPEX-H - 1,188` - identical on the
|
|
January and the September 2026 cards;
|
|
* ENGIE publishes one in the same two-coefficient shape,
|
|
`(B x eSpot_15) - A`.
|
|
|
|
So a supplier DOES publish a dynamic injection curve. What is true is
|
|
narrower and is the actual reason this is a scalar: an injection curve is
|
|
the SAME `B x spot - A` shape as the consumption curve but carries ITS OWN
|
|
coefficients and its own VAT treatment - injection is VAT-exempt where
|
|
consumption is at 6% - so it is a second per-contract formula, not a second
|
|
copy of the first.
|
|
|
|
⚠️ TARIFF-16 LANDED THE INJECTION LEG, AND THIS FUNCTION IS UNCHANGED. The
|
|
curve lives in `price_export_formula` and is read by `export_formula()`
|
|
below; this reader still answers the FLAT injection price, still from one
|
|
option, still with no formula anywhere in its arithmetic. That is the
|
|
arrangement this docstring predicted before either existed - "an injection
|
|
curve gets a `PriceProvider` of its own and this scalar becomes its
|
|
FIXED-product default" - and it is why a site with a flat injection price
|
|
sees nothing change: `price_export_formula` ships "", `export_price_at()`
|
|
falls through to this function, and this function's body is byte-identical
|
|
to the one TARIFF-02 shipped.
|
|
|
|
⚠️ It is NOT reachable from a plugin: `export_price_eur_kwh` is in
|
|
FRAMEWORK_KEYS, so a consumption plugin cannot quietly fold an injection
|
|
price into a consumption price. `price_export_formula` is in the same set
|
|
for the same reason.
|
|
|
|
⚠️ AND NOTHING IN THIS ADD-ON READS EITHER ONE TO TAKE A DECISION.
|
|
`arbitrage.py` prices the sell side as DISPLACED IMPORT and says so in its
|
|
own comment; both hours it compares are all-in consumption prices. This
|
|
reader has no caller under `app/` and neither does `export_price_at()`.
|
|
They are what an operator is shown and what a future export-aware strategy
|
|
would call - not a number any trade turns on today.
|
|
"""
|
|
return float(price_option(opts, "export_price_eur_kwh"))
|
|
|
|
|
|
def export_formula(opts: dict):
|
|
"""The configured `price_export_formula` as a `FormulaSpec`, or None.
|
|
|
|
TARIFF-16. Raises `custom_formula.FormulaInvalid` on a malformed leg - and
|
|
⚠️ THAT EXCEPTION IS NOT MEANT TO REACH A RUNNING SITE, by the rule
|
|
`taxes_profile()` states above and `static_profile` established:
|
|
validate.py parses the same option at STARTUP and refuses to start on it,
|
|
because a formula that only fails when something reads it is a defect
|
|
discovered at the worst possible moment and §11.4 makes every price
|
|
defect quiet.
|
|
|
|
⚠️ THE PARSE IS THE SAME ONE THE CONSUMPTION LEG GETS. There is no second
|
|
parser, no second unit table and no second resolution rule -
|
|
`parse_export_formula()` is `parse_price_formula()` with its `option`
|
|
argument moved, so every refusal an operator can hit on this leg is a
|
|
refusal that already had checks over it.
|
|
"""
|
|
return parse_export_formula(price_option(opts, EXPORT_FORMULA_OPTION))
|
|
|
|
|
|
def export_price_at(opts: dict, index_eur_per_mwh: float) -> float:
|
|
"""What the site is PAID for one exported kWh at one index value, EUR/kWh.
|
|
|
|
THE SEAM, AND ITS HONEST SCOPE. This is the one function a future
|
|
export-aware strategy calls, and it has NO CALLER UNDER `app/` today -
|
|
stated here rather than implied, because the add-on takes no decision on an
|
|
injection price (see `export_price_eur_kwh()` above and `arbitrage.py`'s
|
|
accounting-basis comment). What it buys now is that OPT-06 and anything
|
|
after it inherit a shape the market fits, instead of a scalar that cannot
|
|
hold its own factor, a negative offset, or a VAT rate different from the
|
|
consumption leg's.
|
|
|
|
⚠️ THE FLAT PATH IS THE FALLTHROUGH AND IT IGNORES THE INDEX, which is
|
|
CORRECT rather than a simplification: a site with a flat injection price is
|
|
paid the same for every kWh whatever the market did, and that is what the
|
|
scalar means. With `price_export_formula` empty - every site today - this
|
|
returns exactly `export_price_eur_kwh(opts)` for every index.
|
|
|
|
⚠️ A NEGATIVE ANSWER IS A REAL ANSWER. Eneco's `0,1 x BELPEX-H - 1,188` is
|
|
below zero for any BELPEX-H under 11,88 EUR/MWh, which is the middle of a
|
|
sunny day and precisely when a PV household exports. Nothing here floors
|
|
it: a caller that silently clamped it to zero would hide the hours in which
|
|
exporting COSTS money, which is the one thing an injection price is for.
|
|
"""
|
|
spec = export_formula(opts)
|
|
if spec is None:
|
|
return export_price_eur_kwh(opts)
|
|
return price_eur_kwh(spec, index_eur_per_mwh)
|
|
|
|
|
|
def taxes_profile(opts: dict):
|
|
"""The configured `price_taxes_profile` as a period table, or None.
|
|
|
|
TARIFF-17. Raises `static_tariff.ProfileInvalid` on a malformed table - and
|
|
⚠️ THAT EXCEPTION IS NOT MEANT TO REACH A RUNNING SITE. validate.py parses
|
|
the same option at STARTUP and refuses to start on it, for the reason
|
|
`static_profile` is refused there: a table that only fails when a fetch
|
|
happens is a site that stops pricing at 03:00, and §11.4 turns that into a
|
|
quiet fallback nothing on the status page distinguishes from a healthy
|
|
fixed-tariff day. `load_prices()` still handles it, loudly, because a guard
|
|
whose only proof is "the other guard ran first" is not a guard.
|
|
|
|
⚠️ THE IMPORT IS LAZY AND THE CYCLE IS THE REASON. `static_tariff.py`
|
|
imports this module at module scope; importing it back at module scope here
|
|
would make which file is imported first decide whether the add-on starts.
|
|
`sys.modules` makes the repeat cost a dict lookup.
|
|
"""
|
|
from .static_tariff import parse_taxes_profile
|
|
return parse_taxes_profile(price_option(opts, "price_taxes_profile"))
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# The plugin interface
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
MAX_PLAUSIBLE_EUR_KWH = 10.0 # see _validate_raw
|
|
|
|
|
|
class PriceProvider(ABC):
|
|
"""One supplier. Declares what it is, then returns what its source said.
|
|
|
|
⚠️ `fetch()` DOES NOT RETURN A PRICE. It returns `RawPoint`s carrying the
|
|
published figure. The all-in arithmetic happens in `assemble()`, once, in
|
|
the framework, using `capabilities().sources`. See trap 1.
|
|
"""
|
|
|
|
id = "?"
|
|
|
|
# PRICE-01. The ids to try, in order, when this source cannot answer, and
|
|
# whether the last answer is kept for a day without any source. Empty and
|
|
# False for every plugin that existed before, which is what keeps them
|
|
# exactly as they were: `price_chain.py` is not used for them at all.
|
|
FALLBACKS: tuple = ()
|
|
KEEPS_LAST_GOOD: bool = False
|
|
|
|
@classmethod
|
|
@abstractmethod
|
|
def capabilities(cls) -> ProviderCapabilities:
|
|
...
|
|
|
|
@abstractmethod
|
|
async def fetch(self, opts, since: datetime, until: datetime,
|
|
transport) -> list:
|
|
"""Return `RawPoint`s, or raise `SourceUnavailable(reason)`.
|
|
|
|
`transport(url, timeout_s) -> bytes` is injected so the checks never
|
|
touch a network and `manual_csv` never needs one. `opts` is a
|
|
`PluginOptions`, so the markup and taxes are not reachable from here.
|
|
|
|
⚠️ `since`/`until` ARE THE RANGE THE CALLER WANTS, AND A PLUGIN THAT
|
|
IGNORES THEM GETS WHATEVER ITS FEED DEFAULTS TO (TARIFF-09). That is
|
|
not a small difference: `generic_dayahead` ignored both, the public
|
|
EPEX feed answered ONE Brussels day, and §8.3 then dropped every hour
|
|
past today from the trade set as provisional - so the optimiser never
|
|
saw tomorrow as firm, at any hour of any day, and nothing reported it.
|
|
|
|
This promise is NOT mechanically enforced. A rule obliging every
|
|
plugin to pass a range would be satisfied by an INVENTED parameter
|
|
name - the parser-graded-against-its-own-guess defect, moved from the
|
|
response to the request, and the defect TARIFF-08 found on the
|
|
response side. What exists instead is a roster pin in test_price.py:
|
|
which plugins consume the range and which do not is asserted as a
|
|
literal, so a third supplier forces the decision rather than
|
|
inheriting the silence.
|
|
"""
|
|
|
|
def startup_preview(self, points) -> list:
|
|
"""Lines to log ONCE, the first time this provider answers (TARIFF-15).
|
|
|
|
`points` are the ASSEMBLED, all-in `PricePoint`s - what the site is
|
|
really priced on - and the default is to say nothing, so no existing
|
|
plugin changes behaviour.
|
|
|
|
⚠️ IT IS A HOOK RATHER THAN A LINE IN `load_prices()` BECAUSE THE
|
|
FRAMEWORK CANNOT KNOW WHAT IS WORTH PREVIEWING. Only the plugin knows
|
|
what went IN to its own arithmetic - for `custom_formula` the index in
|
|
EUR/MWh - and printing that beside the all-in figure is the whole
|
|
point: a factor of 1000 is obvious in one line of
|
|
`61.96 -> 0.13522` and invisible in a plan. The framework supplies the
|
|
one number a plugin may never read, by handing back the assembled
|
|
result rather than the tax rate.
|
|
|
|
⚠️ IT MUST NOT RAISE INTO THE CONTROL PATH and it is not trusted not
|
|
to: `_emit_startup_preview()` catches everything. A log line is not a
|
|
reason to stop regulating a battery.
|
|
"""
|
|
return []
|
|
|
|
|
|
# ⚠️ THE NAMES A PLUGIN'S OWN CODE MAY NOT MENTION. FRAMEWORK_KEYS is the
|
|
# obvious half; the other three are the routes review of PR #16 actually used to
|
|
# write a double-counting plugin after `PluginOptions` was in place:
|
|
#
|
|
# `_o` - PluginOptions' private dict, reachable as
|
|
# `opts._o["price_markup_eur_kwh"]`.
|
|
# `price_option` - a PUBLIC framework helper that takes the raw
|
|
# options dict and answers with the markup.
|
|
# `PRICE_OPTION_DEFAULTS` - a PUBLIC module constant. This is the one that
|
|
# decided the review: no private access, no options
|
|
# dict, no helper - just a dict in the module every
|
|
# plugin already imports for `register`.
|
|
#
|
|
# Any of them turns +0.088 into +0.119 on a wholesale feed with nothing
|
|
# anywhere reporting it.
|
|
BANNED_IN_PLUGIN_CODE = tuple(sorted(FRAMEWORK_KEYS)) + (
|
|
"PRICE_OPTION_DEFAULTS", "price_option", "_o")
|
|
|
|
|
|
def code_leaks(cls) -> list:
|
|
"""Framework-owned names appearing in a plugin's own CODE. Empty is clean.
|
|
|
|
⚠️ DOCSTRINGS ARE STRIPPED BEFORE THE SEARCH, AND THAT IS NOT COSMETIC.
|
|
Every shipped plugin's docstring is REQUIRED to name the components it
|
|
sources, so `generic_dayahead` says "markup" in prose and would
|
|
false-positive against `price_markup_eur_kwh` on a raw text scan. The
|
|
check that fires on the documentation it also mandates is the check that
|
|
gets deleted. So the
|
|
class is parsed, every docstring node is dropped, and the search runs on
|
|
`ast.unparse` of what is left - i.e. on code, which is the only thing that
|
|
can actually add a number twice.
|
|
|
|
⚠️ FAIL-OPEN ON MISSING SOURCE, DELIBERATELY. `inspect.getsource` needs the
|
|
.py file, which the add-on image has (Dockerfile COPYs `app/`) but a frozen
|
|
or zipapp build would not. Refusing to start a battery controller because a
|
|
STATIC ANALYSIS could not run would trade a real outage for a hypothetical
|
|
accounting error, so an unreadable source returns `[]` and logs. CI proves
|
|
the scan really runs: test_price.py asserts source was found for every
|
|
shipped plugin, so a build that silently lost it is a red check, not a
|
|
quietly disarmed guard.
|
|
"""
|
|
try:
|
|
src = dedent(inspect.getsource(cls))
|
|
tree = ast.parse(src)
|
|
except (OSError, TypeError, IndentationError, SyntaxError) as e:
|
|
_LOG.warning("Could not read the code of price source %s (%s), so its "
|
|
"double-counting check is skipped.",
|
|
getattr(cls, "id", cls), e)
|
|
return []
|
|
for node in ast.walk(tree):
|
|
body = getattr(node, "body", None)
|
|
if isinstance(body, list) and body and isinstance(body[0], ast.Expr) and isinstance(body[0].value, ast.Constant) and isinstance(body[0].value.value, str):
|
|
body.pop(0)
|
|
code = ast.unparse(tree)
|
|
return sorted({name for name in BANNED_IN_PLUGIN_CODE if name in code})
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# TARIFF-07: the ids that USED to resolve here, and what an operator on one of
|
|
# them has actually been running.
|
|
#
|
|
# ⚠️ THE DEFECT THIS EXISTS TO PREVENT: A RETIRED ID READ AS A TYPO. Without
|
|
# this table `price_provider: "eneco_eddy"` reaches validate.py's unknown-id
|
|
# branch and is refused as "not a shipped plugin" - true, and useless. The
|
|
# operator did not mistype it; it WAS a shipped plugin, they chose it on
|
|
# purpose, and the add-on has accepted it every startup since. What they need
|
|
# told is not "that id is wrong" but "that id never fetched a price, and here
|
|
# is what you have been running instead" - because §11.4 turned every failed
|
|
# fetch into a silent fall back to `static_profile`, so NOTHING they can see
|
|
# has ever differed from a working provider that finds no cheap hour.
|
|
#
|
|
# ⚠️ AND WHY THERE IS NO SILENT ALIAS. Mapping these to `generic_dayahead`
|
|
# would be the friendlier-looking option and it is the wrong one: these three
|
|
# published RETAIL prices (`B x spot + A`, per contract) and `generic_dayahead`
|
|
# reads WHOLESALE. Aliasing needs the site's own two coefficients, which nobody
|
|
# has - they live behind the customer's klantenzone login, and there is no
|
|
# regulator index to read them from (CREG's tariff-card pages 404; VREG was
|
|
# unreachable; every coefficient found in research came from a supplier's own
|
|
# PDF). An alias would therefore have to INVENT the numbers it prices a battery
|
|
# against, which is the same class of defect as the parser graded against its
|
|
# own guess - just one layer further from anything that reports.
|
|
#
|
|
# Each value is the ID-SPECIFIC half of the refusal: the endpoint, what it was
|
|
# observed to answer on 2026-09-13, and why no replacement plugin can be built.
|
|
# validate.py owns the common framing. Verified by hand before deletion, not
|
|
# taken on report - see DECISIONS/TARIFF-07-withdrawing-three-plugins.md.
|
|
# --------------------------------------------------------------------------- #
|
|
RETIRED_PROVIDERS: dict = {
|
|
"engie_dynamic": (
|
|
"its endpoint https://api.engie.be/dynamic/be does not exist - the "
|
|
"gateway there answers 404 with Kong's \"no Route matched with those "
|
|
"values\". No public endpoint could replace it: ENGIE's dynamic price "
|
|
"is (B x eSpot_15) - A with PER-CONTRACT coefficients that live behind "
|
|
"your klantenzone login, so no unauthenticated URL can return YOUR "
|
|
"price"),
|
|
"totalenergies_dynamic": (
|
|
"its endpoint https://api.totalenergies.be/prices/dynamic does not "
|
|
"exist - the TLS handshake fails outright because that hostname is "
|
|
"absent from the certificate the server presents. The coefficients are "
|
|
"per-contract there too"),
|
|
"eneco_eddy": (
|
|
"its endpoint https://api.eneco.be/eddy/prices does not exist - it "
|
|
"answers 404 with an empty body - and \"Eddy\" is not an Eneco "
|
|
"product at all; their dynamic product is \"Zon & Wind Dynamisch\", "
|
|
"which is indexed to Belpex/EPEX DAY-AHEAD and whose coefficients are "
|
|
"likewise per-contract"),
|
|
}
|
|
|
|
|
|
PROVIDERS: dict = {}
|
|
|
|
|
|
def register(cls):
|
|
"""Register a provider by its own declared id. Decorator.
|
|
|
|
⚠️ THE ONLY NAME->CLASS MAPPING IN THE SYSTEM. AC2 is "adding a plugin needs
|
|
no optimiser changes", and this is where that is bought: config resolves an
|
|
id here, the optimiser is handed a `_Series` carrying a `PriceProduct`, and
|
|
nothing downstream ever sees the id except to print it.
|
|
"""
|
|
if cls.id in PROVIDERS:
|
|
raise ValueError(f"duplicate provider id {cls.id!r}")
|
|
if cls.id in RETIRED_PROVIDERS:
|
|
# ⚠️ AT IMPORT, NOT AT FETCH, and for the same reason the leak check is.
|
|
# A retired id that is registered again is the one state in which
|
|
# validate.py's retired-id refusal goes quiet WITHOUT anyone deciding
|
|
# it should: `pid in PROVIDERS` becomes true, the named message is
|
|
# never reached, and the operator is back to a provider that looks
|
|
# like it works. Re-shipping an id is legitimate - the endpoint could
|
|
# become real - but it is a DECISION, and taking the id out of
|
|
# RETIRED_PROVIDERS in the same commit is what records it.
|
|
raise ValueError(
|
|
f"{cls.id!r} is in RETIRED_PROVIDERS and cannot also be a shipped "
|
|
f"plugin. A site configured on it is refused at startup with a "
|
|
f"message naming the id; registering it silently takes that "
|
|
f"message away. If this id really ships again, remove it from "
|
|
f"RETIRED_PROVIDERS in the same commit - and say in the CHANGELOG "
|
|
f"what changed about the endpoint.")
|
|
cls.capabilities() # fail at import, not at 15:00 on fetch day
|
|
leaks = code_leaks(cls)
|
|
if leaks:
|
|
# ⚠️ AT IMPORT, NOT AT FETCH. A plugin that can reach the markup is a
|
|
# plugin that can bill wrong for a month before anyone notices, so it
|
|
# does not get to exist - the add-on fails to start and names the line.
|
|
raise FrameworkOption(
|
|
f"{cls.id!r} mentions {', '.join(leaks)} in its own code. A plugin "
|
|
f"returns what its SOURCE published and declares "
|
|
f"capabilities().sources; the framework adds the complement, once. "
|
|
f"Reaching any of {BANNED_IN_PLUGIN_CODE} lets it be added twice, "
|
|
f"and a price that is one markup high is invisible in every log "
|
|
f"this add-on writes. Remove the reference; if the plugin genuinely "
|
|
f"needs a per-site number, add a NON-framework option key for it.")
|
|
PROVIDERS[cls.id] = cls
|
|
return cls
|
|
|
|
|
|
def _validate_raw(points, caps):
|
|
"""Reject at the boundary, so a malformed feed cannot become a bad decision.
|
|
|
|
⚠️ NEGATIVE PRICES ARE VALID AND ARE NOT CLAMPED (§17 keeps a negative-price
|
|
regression case; Belgian day-ahead goes negative on windy Sundays and those
|
|
hours are the single best charging opportunity of the year). What is
|
|
rejected is what cannot be a price at all: NaN, infinity, an implausible
|
|
magnitude, a naive datetime, and duplicate or out-of-order starts.
|
|
"""
|
|
if not isinstance(caps, ProviderCapabilities):
|
|
raise SourceUnavailable("the plugin did not declare capabilities")
|
|
if not points:
|
|
raise SourceUnavailable("the source returned no points")
|
|
seen = None
|
|
for p in points:
|
|
if not isinstance(p, RawPoint):
|
|
raise SourceUnavailable(f"a plugin returned {type(p).__name__}, not "
|
|
f"a RawPoint - see PriceProvider.fetch()")
|
|
if p.start.tzinfo is None or p.start.utcoffset() is None:
|
|
raise SourceUnavailable(f"point at {p.start!r} has no offset")
|
|
if isinstance(p.eur_per_kwh, bool) or \
|
|
not isinstance(p.eur_per_kwh, (int, float)) or \
|
|
not math.isfinite(p.eur_per_kwh):
|
|
raise SourceUnavailable(f"point at {p.start.isoformat()} has a "
|
|
f"non-finite price {p.eur_per_kwh!r}")
|
|
if abs(p.eur_per_kwh) > MAX_PLAUSIBLE_EUR_KWH:
|
|
raise SourceUnavailable(
|
|
f"point at {p.start.isoformat()} is {p.eur_per_kwh} EUR/kWh, "
|
|
f"beyond +-{MAX_PLAUSIBLE_EUR_KWH}, so the feed is probably in "
|
|
f"EUR/MWh")
|
|
u = p.start.astimezone(timezone.utc)
|
|
if seen is not None and u <= seen:
|
|
raise SourceUnavailable(
|
|
f"points are not strictly increasing at {u.isoformat()} - a "
|
|
f"duplicated hour on a DST day is the usual cause, and keying "
|
|
f"prices by a naive local hour is how it happens")
|
|
seen = u
|
|
|
|
|
|
def taxes_at(taxes, instant: datetime) -> float:
|
|
"""The TAXES component at ONE instant. A scalar, or a period table.
|
|
|
|
TARIFF-17. THE DEFECT THIS EXISTS TO CLOSE, and it is a MISSING CAPABILITY
|
|
rather than a wrong constant: `price_taxes_eur_kwh` is additive and
|
|
identical every hour, so it cancels exactly out of `max - min` and cannot
|
|
move `min(firm, key=...)`. It is not a distortion - it is provably inert on
|
|
both quantities the optimiser reads. What was missing is anywhere for an
|
|
HOUR-VARYING non-energy component to live, and the Walloon and Brussels
|
|
DSOs bill one: ORES `dag 13,27 / nacht 7,39` c EUR/kWh, a 5,88 c gap whose
|
|
cheap band coincides with the cheap spot hours. Flattening it does not
|
|
shift the curve, it UNDERSTATES the spread and declines profitable cycles.
|
|
See DOCS.md "The three buckets".
|
|
|
|
⚠️ DUCK-TYPED ON `price_at`, NOT ON `isinstance`. price.py must not import
|
|
static_tariff.py - static_tariff.py imports THIS module, and a module-level
|
|
cycle would make the import order load-bearing. `taxes_profile()` below is
|
|
the one place the real parser is reached, and it imports it lazily.
|
|
|
|
⚠️ THE LOOKUP IS ON THE ~30 s PLANNING PATH through `assemble()`, so it
|
|
stays a linear scan over the two-to-four periods a Belgian tariff has
|
|
(`StaticProfile.at()`), with no parsing, no allocation and no I/O. A
|
|
malformed table never reaches here: validate.py refuses it at STARTUP.
|
|
"""
|
|
at = getattr(taxes, "price_at", None)
|
|
if at is None:
|
|
return float(taxes)
|
|
return float(at(instant))
|
|
|
|
|
|
def assemble(caps: ProviderCapabilities, raw, markup: float,
|
|
taxes) -> list:
|
|
"""RawPoints -> canonical all-in PricePoints. THE ONLY PLACE MARKUP IS ADDED.
|
|
|
|
Adds exactly `caps.adds` - the complement of what the plugin declared its
|
|
source already contains. A plugin that publishes an all-in figure gets
|
|
nothing added; a raw EPEX figure gets both.
|
|
|
|
⚠️ `taxes` IS A SCALAR *OR* A PERIOD TABLE (TARIFF-17), AND IT IS EVALUATED
|
|
AT EACH POINT'S OWN INSTANT. Hoisting the lookup out of the loop - reading
|
|
it once and adding the same number to every point - is the mutation that
|
|
collapses the whole feature back into the LEVEL bucket: the assembled
|
|
series would still differ from the untaxed one by the right MAGNITUDE, and
|
|
the cheapest hour would be wrong. `test_price.py` section 24 is the check
|
|
that goes red for it, and it grades WHICH HOUR IS CHEAPEST rather than the
|
|
spread, because a spread-magnitude assertion passes on the hoisted version.
|
|
|
|
⚠️ THE SCALAR PATH IS UNCHANGED, ARITHMETICALLY AND NOT ONLY IN INTENT.
|
|
`taxes_at()` on a float returns that float at every instant, and
|
|
`0.0 + markup` is exactly `markup` in IEEE 754 for every finite markup, so
|
|
a site on a scalar gets the identical bytes it got before this landed.
|
|
"""
|
|
add_markup = float(markup) if Component.MARKUP in caps.adds else 0.0
|
|
wants_taxes = Component.TAXES in caps.adds
|
|
out = []
|
|
for p in raw:
|
|
local = brussels(p.start)
|
|
add = add_markup + (taxes_at(taxes, local) if wants_taxes else 0.0)
|
|
out.append(PricePoint(start=local,
|
|
eur_per_kwh=round(p.eur_per_kwh + add, 6),
|
|
provisional=bool(p.provisional)))
|
|
return out
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# Transport - one place, HTTPS only, host checked against the declaration
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
async def https_get(url: str, timeout_s: float, headers=None) -> bytes:
|
|
"""The default transport. §16: outbound is allow-listed and TLS'd.
|
|
|
|
Imported lazily so `manual_csv` - the offline fallback - never needs aiohttp
|
|
to be importable at all. AC3 says it works fully offline; that includes a
|
|
machine where the HTTP stack is missing.
|
|
|
|
PRICE-01. `headers` is None for every plugin but `price_service`, and with
|
|
None this is the code it has always been, byte for byte. With headers it
|
|
answers in the SAFE words of `status_words()` and never lets a library
|
|
error through: an error from a request that carried a token is the one
|
|
place a token could be printed by somebody else's code.
|
|
"""
|
|
import aiohttp
|
|
t = aiohttp.ClientTimeout(total=float(timeout_s))
|
|
if headers is None:
|
|
async with aiohttp.ClientSession(timeout=t) as s:
|
|
async with s.get(url) as r:
|
|
r.raise_for_status()
|
|
return await r.read()
|
|
try:
|
|
async with aiohttp.ClientSession(timeout=t) as s:
|
|
async with s.get(url, headers=headers) as r:
|
|
if r.status != 200:
|
|
raise SourceUnavailable(status_words(r.status))
|
|
return await r.read()
|
|
except SourceUnavailable:
|
|
raise
|
|
except Exception as e: # noqa: BLE001
|
|
raise SourceUnavailable(
|
|
f"the price service did not answer ({type(e).__name__})")
|
|
|
|
|
|
# PRICE-01. The one phrase a refused token is reported with. The fallback chain
|
|
# reads it to tell "the owner must act" from "the server is down", so it is a
|
|
# constant both sides import.
|
|
AUTH_REFUSED = "the price service refused the token"
|
|
|
|
|
|
def status_words(status: int) -> str:
|
|
"""What an HTTP status from the price service is called. No header text."""
|
|
if status in (401, 403):
|
|
return f"{AUTH_REFUSED} ({status})"
|
|
if status == 429:
|
|
return "the price service says too many requests (429)"
|
|
if status >= 500:
|
|
return f"the price service has an error ({status})"
|
|
return f"the price service answered {status}"
|
|
|
|
|
|
def check_egress(url: str, caps: ProviderCapabilities) -> str:
|
|
"""§16: a plugin may only reach a host it DECLARED. Enforced, not documented.
|
|
|
|
Returns the host, so a caller that forgets to call this is not the same
|
|
shape as one that did.
|
|
|
|
⚠️ THE SCHEME IS SPLIT OFF BEFORE THE HOST IS PARSED, AND THE ORDER MATTERS
|
|
FOR THE TEST MORE THAN FOR THE CODE. The first version stripped a fixed
|
|
`len("https://")` from the front, which on an `http://` URL ate two
|
|
characters of the hostname instead - `http://api.energy-charts.info/p`
|
|
became the host `pi.energy-charts.info`, which then failed the ALLOW-LIST.
|
|
So a non-TLS feed was refused, the check named "plain http is refused"
|
|
passed, and deleting the TLS rule entirely left it passing. Review of
|
|
PR #16 found that by mutation: an assertion that does not test what it
|
|
names, inside a security check - this project's own recurring defect. Now
|
|
the scheme is parsed as a scheme, the host of an `http://` URL comes out
|
|
correct and DECLARED, and the TLS rule is the only thing that can refuse
|
|
it - which is what makes the check able to fail.
|
|
"""
|
|
if not url:
|
|
raise SourceUnavailable("no feed URL configured (price_feed_url)")
|
|
scheme, sep, rest = url.partition("://")
|
|
if not sep:
|
|
raise SourceUnavailable(
|
|
f"not an absolute URL, so no host can be checked against the "
|
|
f"§16 allow-list: {url!r}")
|
|
if scheme.lower() != "https":
|
|
raise SourceUnavailable(
|
|
f"refusing a non-TLS price feed: {url!r} uses {scheme!r}. §16 "
|
|
f"allow-lists outbound destinations and SEC-01 requires TLS on "
|
|
f"every one of them.")
|
|
host = rest.split("/", 1)[0].split("?", 1)[0].split("@")[-1]
|
|
host = host.split(":", 1)[0].lower()
|
|
if host not in {h.lower() for h in caps.egress_hosts}:
|
|
raise SourceUnavailable(
|
|
f"{host!r} is not in this provider's declared egress_hosts "
|
|
f"{caps.egress_hosts} - §16 allow-lists outbound destinations, and "
|
|
f"a URL from the options file is not a declaration")
|
|
return host
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# The shipped plugins.
|
|
#
|
|
# ⚠️ EACH DOCSTRING STATES WHICH COMPONENTS IT SOURCES AND WHICH THE FRAMEWORK
|
|
# ADDS, in the two fixed lines `SOURCES: ...` / `The framework ADDS: ...`. That
|
|
# is not decoration - it is the human-readable half of `capabilities().sources`,
|
|
# and test_price.py PARSES those lines out of the docstring and compares them
|
|
# with the declaration, so a plugin whose prose and whose code disagree about
|
|
# what its feed contains is a red check rather than a 3 c/kWh error nobody sees.
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
# TARIFF-08: the COLUMNAR document, which is what the public EPEX feed really
|
|
# publishes.
|
|
#
|
|
# ⚠️ THIS IS THE ONLY PARSER LEFT, AND THE SHAPE IT DOES **NOT** READ IS WHY.
|
|
# Until TARIFF-07 a second function, `_points_from_json()`, read a row-per-point
|
|
# document: `{"data": [{...}, {...}]}`. NO FEED ANYBODY REACHED EVER PUBLISHED
|
|
# THAT SHAPE - it was invented alongside the three withdrawn plugins, and the
|
|
# only bodies ever fed to it were the fixtures written from the same guess. It
|
|
# went with them. `https://api.energy-charts.info/price?bzn=BE` publishes
|
|
# PARALLEL ARRAYS:
|
|
#
|
|
# {"unix_seconds": [1788991200, 1788992100, ...], <- UTC epoch seconds
|
|
# "price": [216.08, 209.90, ...], <- one per instant
|
|
# "unit": "EUR / MWh", "license_info": "...", "deprecated": false}
|
|
#
|
|
# The row parser was pointed at that URL and asked for `doc["data"]`, so
|
|
# every fetch ended `SourceUnavailable: the feed carried no 'data' list` ->
|
|
# §11.4 -> static profile, arbitrage off, and nothing in the status page said
|
|
# why. Verified live 2026-09-10 against both fixtures under `scenarios/`.
|
|
#
|
|
# ⚠️ AND IT IS NOT ONE FUNCTION THAT SNIFFS FOR BOTH. A parser that decides
|
|
# which document it got is a parser that can decide wrong, and a wrong-but-
|
|
# parseable response then becomes a price. Two shapes, two functions, and each
|
|
# plugin names the one its own feed publishes.
|
|
#
|
|
# The columnar shape was already known in this repository: DATA-02's
|
|
# `scenarios/fetch_epex_be.py` has read `zip(d["unix_seconds"], d["price"])`
|
|
# since 2026-08-31. Only the SHIPPED path never learned it, because every
|
|
# fixture in the suite was built from the invented row shape - the parser was
|
|
# graded against its own assumption. See test_price.py section 2b.
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
COLUMNAR_TIME_KEY = "unix_seconds"
|
|
COLUMNAR_PRICE_KEY = "price"
|
|
COLUMNAR_UNIT_KEY = "unit"
|
|
|
|
|
|
def _columnar_unit(text) -> str:
|
|
"""`"EUR / MWh"` -> `"eur/mwh"`. Spacing and case are not information."""
|
|
return "".join(str(text).split()).lower()
|
|
|
|
|
|
def _points_from_columnar_json(body, expect_unit, scale,
|
|
provisional_default):
|
|
"""Parse a COLUMNAR price document: parallel `unix_seconds` / `price`.
|
|
|
|
Defects this refuses, each of which has a way of becoming a number rather
|
|
than an error:
|
|
|
|
* ⚠️ UNEQUAL ARRAYS ARE `SourceUnavailable`, NOT A SHORT SERIES. `zip()`
|
|
stops at the shorter array and says nothing, so a `price` list that
|
|
arrived truncated would produce a REAL price series with its tail
|
|
missing - and the tail of a Belgian day is the evening peak. A series
|
|
that is missing the expensive hours is worse than no series: §11.4's
|
|
no-data row switches arbitrage OFF, whereas a truncated series is
|
|
traded on. Nothing downstream can tell the two apart, so the refusal
|
|
has to happen here.
|
|
* ⚠️ A UNIT THAT IS NOT THE DECLARED ONE IS REFUSED. `scale` is the
|
|
caller's standing assumption about the unit; if the feed ever
|
|
publishes EUR/kWh under the same keys, every price silently comes out
|
|
a THOUSAND times too small. `_validate_raw()` cannot catch that -
|
|
`MAX_PLAUSIBLE_EUR_KWH` only bounds prices that are too LARGE.
|
|
DATA-02's fetcher refuses on the same field for the same reason.
|
|
* ⚠️ `unix_seconds` ARE UTC INSTANTS, SAID SO EXPLICITLY.
|
|
`datetime.fromtimestamp(t, timezone.utc)`. Without that second
|
|
argument CPython returns a NAIVE datetime in the HOST's zone, and a
|
|
naive datetime is a wall clock reading plus an assumption about whose
|
|
clock it was. `brussels()` and `_validate_raw()` both refuse one, so
|
|
the mistake surfaces as a refused fetch rather than as a mispriced
|
|
day - but they refuse ALL naive datetimes and can say nothing about
|
|
whether this one was the right instant. That part is decided here.
|
|
|
|
§9.3's rendering AT THE BRUSSELS OFFSET is `assemble()`'s, once, for
|
|
every plugin - `brussels(p.start)` there is the single line that does
|
|
it, and calling it here as well would be a second copy of a rule that
|
|
already has an owner, indistinguishable from a no-op under mutation.
|
|
A plugin's job is the right INSTANT; the framework's is the rendering.
|
|
|
|
A `null` price is DROPPED and logged, not fatal, and that is a different
|
|
judgement from the one above: an unequal pair of arrays means the pairing
|
|
of instant to price is unreliable, whereas a null means the source stated
|
|
it published no price for that one instant. A dropped instant leaves a
|
|
hole, and `_Series.at()` already answers a hole with `Absence` - §11.4 for
|
|
that hour. `scenarios/fetch_epex_be.py` dropped 0 nulls over the 12 months
|
|
to 2026-08-31, so this is a shape the source allows rather than one it uses.
|
|
"""
|
|
try:
|
|
text = body.decode("utf-8") if isinstance(body, (bytes, bytearray)) \
|
|
else str(body)
|
|
doc = json.loads(text)
|
|
except (ValueError, UnicodeDecodeError) as e:
|
|
raise SourceUnavailable(f"the feed did not parse as JSON: {e}")
|
|
if not isinstance(doc, dict):
|
|
raise SourceUnavailable(
|
|
f"the feed is a {type(doc).__name__}, not a columnar object with "
|
|
f"{COLUMNAR_TIME_KEY!r} and {COLUMNAR_PRICE_KEY!r} arrays")
|
|
times = doc.get(COLUMNAR_TIME_KEY)
|
|
prices = doc.get(COLUMNAR_PRICE_KEY)
|
|
for key, col in ((COLUMNAR_TIME_KEY, times), (COLUMNAR_PRICE_KEY, prices)):
|
|
if not isinstance(col, list):
|
|
raise SourceUnavailable(
|
|
f"the feed carried no {key!r} array; this provider reads the "
|
|
f"columnar shape (parallel {COLUMNAR_TIME_KEY}/"
|
|
f"{COLUMNAR_PRICE_KEY} arrays), and a row-per-point document "
|
|
f"is a different feed, not this one with a field missing")
|
|
if len(times) != len(prices):
|
|
raise SourceUnavailable(
|
|
f"{COLUMNAR_TIME_KEY} has {len(times)} entries and "
|
|
f"{COLUMNAR_PRICE_KEY} has {len(prices)}, so the feed is cut short "
|
|
f"and none of it is used")
|
|
unit = _columnar_unit(doc.get(COLUMNAR_UNIT_KEY, ""))
|
|
if unit != _columnar_unit(expect_unit):
|
|
raise SourceUnavailable(
|
|
f"the feed reports {doc.get(COLUMNAR_UNIT_KEY)!r}, not "
|
|
f"{expect_unit!r}, so its prices cannot be converted")
|
|
if doc.get("deprecated"):
|
|
# Not a refusal: today's prices are still prices. But §11.4 makes a
|
|
# dead feed look exactly like a quiet one, so the warning that the
|
|
# source is going away has to be emitted while it still works.
|
|
_LOG.warning("The price feed says it will stop (deprecated=%r), "
|
|
"though it still answers for now.", doc.get("deprecated"))
|
|
out = []
|
|
dropped = []
|
|
for i, (t, v) in enumerate(zip(times, prices)):
|
|
if v is None:
|
|
dropped.append(i)
|
|
continue
|
|
if isinstance(t, bool) or not isinstance(t, (int, float)) or \
|
|
not math.isfinite(t):
|
|
raise SourceUnavailable(
|
|
f"{COLUMNAR_TIME_KEY}[{i}] is {t!r}, not epoch seconds")
|
|
if isinstance(v, bool) or not isinstance(v, (int, float)) or \
|
|
not math.isfinite(v):
|
|
raise SourceUnavailable(
|
|
f"{COLUMNAR_PRICE_KEY}[{i}] is {v!r}, which is not a price")
|
|
start = datetime.fromtimestamp(float(t), timezone.utc)
|
|
out.append(RawPoint(start=start, eur_per_kwh=float(v) * scale,
|
|
provisional=bool(provisional_default)))
|
|
if dropped:
|
|
_LOG.warning("The price feed gave no price for %d of %s, at index "
|
|
"%s. Those are left out, not counted as free.",
|
|
len(dropped), messages.count(len(times), "time"),
|
|
", ".join(str(i) for i in dropped[:5])
|
|
+ (" and more" if len(dropped) > 5 else ""))
|
|
if not out:
|
|
raise SourceUnavailable(
|
|
f"the feed carried {len(times)} instants and a price for none of "
|
|
f"them")
|
|
return out
|
|
|
|
|
|
def dayahead_feed_url(plugin_id, opts, since, until, base, bzn) -> str:
|
|
"""The day-ahead URL for a window, or the operator's override, verbatim.
|
|
|
|
⚠️ SHARED BY `generic_dayahead` AND `custom_formula`, ON PURPOSE
|
|
(TARIFF-15). They read the SAME feed for the same range; two copies of this
|
|
rule would be two places for the Brussels-calendar derivation and the
|
|
override decision to disagree, and the one that is wrong would be wrong
|
|
silently - §11.4 turns a short horizon into provisional hours, not into an
|
|
error. It takes `plugin_id` rather than reading one, so the framework's
|
|
AC2 rule (no plugin id in shared code) still holds here.
|
|
|
|
⚠️ AN OVERRIDE IS THE EXACT URL AND NOTHING IS APPENDED TO IT, WHICH COSTS
|
|
THE OVERRIDING SITE THE RANGE. That is a decision, not an oversight, and
|
|
the sentence below is why it is not a footnote:
|
|
|
|
* `price_feed_url` says "override the plugin's default feed", and an
|
|
override that is silently rewritten is not one. Merging parameters
|
|
into an operator's query string means parsing and re-serialising a
|
|
URL they wrote - which reorders, de-duplicates and re-escapes it,
|
|
and can hand the server a request nobody typed.
|
|
* `start`/`end` are ENERGY-CHARTS' spelling, and both providers DECLARE
|
|
A SECOND HOST: `web-api.tp.entsoe.eu`. Nothing here has verified that
|
|
host's day-ahead parameters and nothing here should guess them - but
|
|
they are not `start`/`end`, so an override to the other DECLARED host
|
|
is exactly the case where appending our vocabulary adds two parameters
|
|
that host ignores and yields the short horizon anyway, with nothing
|
|
saying so.
|
|
* and blind appending is refused outright: `dated_feed_url()` will not
|
|
add a query string to a base that has one.
|
|
|
|
So the cost is real and it is REPORTED, at fetch time, on the shipped
|
|
logger, naming the horizon consequence rather than the substitution.
|
|
§11.4 is what makes a price defect quiet; an operator who has taken the
|
|
short horizon has to be able to find out from the log that they did.
|
|
"""
|
|
override = str(opts.get("price_feed_url") or "").strip()
|
|
if override:
|
|
_LOG.warning(
|
|
"price_feed_url is set, so %s asks for %s as given, and "
|
|
"tomorrow's prices may stay provisional. Clear it to ask for %s "
|
|
"to %s.",
|
|
plugin_id, override, *(d.isoformat()
|
|
for d in brussels_date_range(since, until)))
|
|
_LOG.debug(
|
|
"The public feed answers one Brussels day to a request that names "
|
|
"no range. Every later hour is then copied from yesterday and "
|
|
"marked provisional, and §8.3 leaves provisional hours out of "
|
|
"trading.")
|
|
return override
|
|
first, last = brussels_date_range(since, until)
|
|
return dated_feed_url(base, bzn=bzn, start=first.isoformat(),
|
|
end=last.isoformat())
|
|
|
|
|
|
@register
|
|
class GenericDayAhead(PriceProvider):
|
|
"""Public EPEX BE day-ahead feed, COLUMNAR, in EUR/MWh.
|
|
|
|
SOURCES: energy
|
|
The framework ADDS: markup, taxes
|
|
|
|
This is the plugin the double-counting trap is about. The published figure
|
|
is the wholesale clearing price and nothing else, so both user-entered
|
|
components have to be added to it - and because this plugin cannot read
|
|
either number, it cannot be the one that adds them twice.
|
|
|
|
⚠️ IT READS THE COLUMNAR SHAPE, AND SAYS SO IN ITS OWN `fetch()`. Until
|
|
TARIFF-08 it called the row-per-point parser (withdrawn by TARIFF-07) and
|
|
asked
|
|
`api.energy-charts.info` for a `"data"` list it has never published, so
|
|
every fetch this provider ever made ended in `SourceUnavailable` and §11.4
|
|
turned that into a silent fall back to the static profile. See
|
|
`_points_from_columnar_json()` for the shape and for why there are two
|
|
parsers rather than one that guesses.
|
|
|
|
⚠️ THE FEED IS QUARTER-HOURLY SINCE BELGIUM'S 15-MINUTE MTU (2025-10-01):
|
|
96 points a day at 900 s, not 24 at 3600 s. Nothing here assumes either -
|
|
the instants come from the document - and both granularities are replayed
|
|
from real captured bodies in test_price.py section 2b.
|
|
"""
|
|
|
|
id = "generic_dayahead"
|
|
|
|
@classmethod
|
|
def capabilities(cls):
|
|
return ProviderCapabilities(
|
|
product=PriceProduct.DAY_AHEAD,
|
|
sources=frozenset({Component.ENERGY}),
|
|
egress_hosts=("api.energy-charts.info", "web-api.tp.entsoe.eu"))
|
|
|
|
FEED_BASE = "https://api.energy-charts.info/price"
|
|
BZN = "BE"
|
|
|
|
async def fetch(self, opts, since, until, transport):
|
|
"""Request the range the CALLER asked for, on the Brussels calendar.
|
|
|
|
⚠️ THE ARGUMENTS ARE USED, and `dayahead_feed_url()` above is where
|
|
they are turned into a URL. Until TARIFF-09 this method took `since`
|
|
and `until` and asked for the bare `?bzn=BE`, which answers ONE
|
|
Brussels day - so a 48 h horizon was 15 firm hours and 33 carried at
|
|
09:00, and 1 firm and 47 carried at 23:00, with the whole trade set
|
|
inside a single calendar day at every hour of every day.
|
|
|
|
⚠️ THE URL DECISION MOVED OUT OF THIS METHOD IN TARIFF-15 AND DID NOT
|
|
CHANGE. `custom_formula` reads the same feed with the same range and
|
|
the same parser, and a second copy of the override rule would be a
|
|
second place for it to be wrong - which is the reason `covers()`
|
|
delegates to `at()` rather than restating its window. The reasoning,
|
|
the warning and the exact URL are all in `dayahead_feed_url()`.
|
|
"""
|
|
url = dayahead_feed_url(self.id, opts, since, until,
|
|
self.FEED_BASE, self.BZN)
|
|
check_egress(url, self.capabilities())
|
|
body = await transport(url, float(opts.get("price_timeout_s")))
|
|
return _points_from_columnar_json(body, expect_unit="EUR / MWh",
|
|
scale=0.001,
|
|
provisional_default=False)
|
|
|
|
|
|
@register
|
|
class PriceService(PriceProvider):
|
|
"""The owner's own price server (PRICE-01), COLUMNAR, in EUR/MWh.
|
|
|
|
SOURCES: energy
|
|
The framework ADDS: markup, taxes
|
|
|
|
The server answers the energy-charts layout (`unix_seconds`, `price`,
|
|
`unit`) plus `covers`, the complete Brussels days it holds. The price is
|
|
the wholesale energy price and nothing else, so markup and taxes are added
|
|
by the framework exactly as for `generic_dayahead`.
|
|
|
|
The host is fixed here, HTTPS only, and a bearer token (the option
|
|
`price_service_token`) goes in the `Authorization` header. The token is not
|
|
put into any message this class builds.
|
|
|
|
`FALLBACKS` names the next source down. `app/price_chain.py` walks it and
|
|
then the stored prices; this class only fetches.
|
|
"""
|
|
|
|
id = "price_service"
|
|
|
|
HOST = "prices.kammenstraatha.duckdns.org"
|
|
URL = "https://prices.kammenstraatha.duckdns.org/v1/dayahead/BE"
|
|
FALLBACKS = ("generic_dayahead",)
|
|
KEEPS_LAST_GOOD = True
|
|
|
|
@classmethod
|
|
def capabilities(cls):
|
|
return ProviderCapabilities(
|
|
product=PriceProduct.DAY_AHEAD,
|
|
sources=frozenset({Component.ENERGY}),
|
|
egress_hosts=(cls.HOST,))
|
|
|
|
async def fetch(self, opts, since, until, transport):
|
|
token = str(opts.get("price_service_token") or "").strip()
|
|
if not token:
|
|
raise SourceUnavailable("no token is set for the price service")
|
|
if any(c.isspace() or not c.isprintable() for c in token):
|
|
raise SourceUnavailable(
|
|
"the price service token has a space or a line break in it")
|
|
check_egress(self.URL, self.capabilities())
|
|
try:
|
|
body = await transport(
|
|
self.URL, float(opts.get("price_timeout_s")),
|
|
headers={"Authorization": "Bearer " + token})
|
|
except SourceUnavailable:
|
|
raise
|
|
except Exception as e: # noqa: BLE001
|
|
raise SourceUnavailable(
|
|
f"the price service did not answer ({type(e).__name__})")
|
|
points = _points_from_columnar_json(body, expect_unit="EUR / MWh",
|
|
scale=0.001,
|
|
provisional_default=False)
|
|
covers = _covered_days(body)
|
|
if covers is None:
|
|
return points
|
|
whole = [p for p in points
|
|
if brussels(p.start).date().isoformat() in covers]
|
|
if not whole:
|
|
raise SourceUnavailable(
|
|
"the price service sent no complete day")
|
|
return whole
|
|
|
|
|
|
def _covered_days(body):
|
|
"""The `covers` list of a price service answer as a set, or None if absent.
|
|
|
|
A day the server does not list as complete is not used: an hour missing
|
|
inside a day is a hole, and a hole is not a free hour.
|
|
"""
|
|
try:
|
|
text = body.decode("utf-8") if isinstance(body, (bytes, bytearray)) \
|
|
else str(body)
|
|
covers = json.loads(text).get("covers")
|
|
except (ValueError, UnicodeDecodeError, AttributeError):
|
|
return None
|
|
if covers is None:
|
|
return None
|
|
if not isinstance(covers, list) or \
|
|
not all(isinstance(d, str) for d in covers):
|
|
raise SourceUnavailable("the price service sent a bad list of days")
|
|
return set(covers)
|
|
|
|
|
|
@register
|
|
class ManualCsv(PriceProvider):
|
|
"""A file on disk. AC3: the documented fallback, fully offline.
|
|
|
|
SOURCES: energy, markup, taxes
|
|
The framework ADDS: nothing
|
|
|
|
⚠️ THE CSV IS ALL-IN, AND DOCS.md says so. It has to be: this plugin has no
|
|
way to know what the numbers a human typed already contain, and guessing
|
|
would be the double-counting defect with a friendlier face. `load_prices()`
|
|
warns when markup/taxes are configured non-zero under a plugin that sources
|
|
them, so the ignored numbers are visible rather than silently discarded.
|
|
|
|
⚠️ NO TRANSPORT, NO aiohttp, NO NETWORK, NO tz DATABASE. It is the escape
|
|
hatch for a site with no internet and the ingest path the TEST-01 replay
|
|
harness will share, so it must not acquire a dependency on anything that
|
|
needs one. `transport` is accepted and deliberately unused.
|
|
|
|
Format - header required, two columns or three:
|
|
|
|
start,eur_per_kwh,provisional
|
|
2026-01-15T00:00:00+01:00,0.1834,false
|
|
"""
|
|
|
|
id = "manual_csv"
|
|
|
|
@classmethod
|
|
def capabilities(cls):
|
|
return ProviderCapabilities(
|
|
product=PriceProduct.DAY_AHEAD,
|
|
sources=ALL_IN,
|
|
egress_hosts=()) # declared empty: this one reaches nothing
|
|
|
|
async def fetch(self, opts, since, until, transport):
|
|
path = opts.get("price_csv_path")
|
|
try:
|
|
with open(path, "r", encoding="utf-8") as fh:
|
|
text = fh.read()
|
|
except OSError as e:
|
|
raise SourceUnavailable(f"cannot read {path}: {e}")
|
|
return self.parse(text)
|
|
|
|
@staticmethod
|
|
def parse(text: str) -> list:
|
|
"""Split out so the offline check can drive it without a filesystem."""
|
|
rdr = csv.DictReader(io.StringIO(text))
|
|
if not rdr.fieldnames or "start" not in rdr.fieldnames or \
|
|
"eur_per_kwh" not in rdr.fieldnames:
|
|
raise SourceUnavailable(
|
|
f"the CSV needs a header with at least 'start' and "
|
|
f"'eur_per_kwh'; found {rdr.fieldnames!r}")
|
|
out = []
|
|
for n, row in enumerate(rdr, start=2):
|
|
try:
|
|
start = datetime.fromisoformat((row["start"] or "").strip())
|
|
value = float((row["eur_per_kwh"] or "").strip())
|
|
except (TypeError, ValueError) as e:
|
|
raise SourceUnavailable(f"CSV line {n}: {e}")
|
|
if start.tzinfo is None:
|
|
raise SourceUnavailable(
|
|
f"CSV line {n}: {row['start']!r} has no UTC offset. Write "
|
|
f"'2026-01-15T00:00:00+01:00', not '2026-01-15 00:00' - a "
|
|
f"local wall clock is ambiguous for one hour every October.")
|
|
flag = (row.get("provisional") or "false").strip().lower()
|
|
out.append(RawPoint(start=start, eur_per_kwh=value,
|
|
provisional=flag in ("1", "true", "yes")))
|
|
return out
|
|
|
|
|
|
@register
|
|
class CustomFormula(PriceProvider):
|
|
"""The operator's OWN tariff card, as two fixed coefficients. TARIFF-15.
|
|
|
|
SOURCES: energy, markup
|
|
The framework ADDS: taxes
|
|
|
|
⚠️ IT DECLARES `{ENERGY, MARKUP}` AND NOT `ALL_IN`, AND THAT IS THE WHOLE
|
|
SAFETY ARGUMENT. A Belgian card's `Tariefformule` line is the ENERGY
|
|
component only - Eneco tabulates network tariffs and levies separately,
|
|
ENGIE states that the formula excludes network, distribution, transport,
|
|
grid levies and excise - so declaring ALL_IN would lie on behalf of every
|
|
operator who typed exactly what their card says, and would silently drop
|
|
~12 c EUR/kWh from every hour. The framework adds `price_taxes_eur_kwh`
|
|
on top, once, and this class cannot see or apply it (trap 1).
|
|
|
|
The risk that introduces is one no guard in this file can see, because it
|
|
is not in code: THE OPERATOR folding their network charges into `offset`
|
|
while the tax option carries them too. `app/custom_formula.py`'s card
|
|
self-check is what refuses that - the card's printed estimate is the
|
|
energy component alone, so an inflated offset moves 15.55 to ~27 and the
|
|
add-on will not start. Its limit is stated in the same breath there: it
|
|
grades the coefficients against the card, NOT the card against reality.
|
|
|
|
⚠️ NO EXPRESSION EVALUATION OF ANY KIND, AND THE COEFFICIENTS ARE NUMBERS
|
|
BECAUSE THE PARSER REFUSES ANYTHING ELSE. This string is settable over
|
|
HTTP through the add-on's own ingress config page, so `eval` on it would
|
|
be RCE in a root container holding SUPERVISOR_TOKEN - and, the deciding
|
|
argument, an arbitrary expression could not be graded against the card at
|
|
all. See app/custom_formula.py.
|
|
|
|
⚠️ THE INDEX PATH IS `generic_dayahead`'s, NOT A SECOND ONE. Same feed,
|
|
same `dayahead_feed_url()` request on the Brussels calendar, same columnar
|
|
parser, same declared egress hosts. A second parser would be a second
|
|
failure mode for a document that already has one that works.
|
|
"""
|
|
|
|
id = "custom_formula"
|
|
|
|
FEED_BASE = "https://api.energy-charts.info/price"
|
|
BZN = "BE"
|
|
|
|
# Written by fetch(), read by startup_preview(), and by nothing else.
|
|
spec = None
|
|
index_by_instant: dict = {}
|
|
|
|
@classmethod
|
|
def capabilities(cls):
|
|
return ProviderCapabilities(
|
|
product=PriceProduct.DAY_AHEAD,
|
|
sources=frozenset({Component.ENERGY, Component.MARKUP}),
|
|
egress_hosts=("api.energy-charts.info", "web-api.tp.entsoe.eu"))
|
|
|
|
async def fetch(self, opts, since, until, transport):
|
|
"""Fetch the index, apply the card's arithmetic, return RawPoints.
|
|
|
|
⚠️ `since`/`until` ARE USED, through the shared URL builder - see
|
|
`dayahead_feed_url()` and TARIFF-09 for what ignoring them costs.
|
|
"""
|
|
try:
|
|
spec = parse_price_formula(opts.get("price_formula"))
|
|
except FormulaInvalid as e:
|
|
raise SourceUnavailable(f"price_formula: {e}")
|
|
if spec is None:
|
|
raise SourceUnavailable(
|
|
"“Dynamic price source” (price_provider) is 'custom_formula', "
|
|
"but “Your own tariff formula” (price_formula) is empty. "
|
|
"Restart the add-on to check the settings again")
|
|
url = dayahead_feed_url(self.id, opts, since, until,
|
|
self.FEED_BASE, self.BZN)
|
|
check_egress(url, self.capabilities())
|
|
body = await transport(url, float(opts.get("price_timeout_s")))
|
|
# ⚠️ `scale=1.0`, SO THESE RawPoints CARRY EUR/**MWh** IN A FIELD
|
|
# SPELLED `eur_per_kwh`. That is a trap and it is contained on purpose:
|
|
# the list is destructured on the very next line and never leaves this
|
|
# method, so nothing downstream can read the wrong unit off it. The
|
|
# alternative - scaling to EUR/kWh and multiplying by 1000 again to
|
|
# feed the formula - is two conversions where the contract needs none,
|
|
# and every extra conversion here is another place to lose a factor of
|
|
# 1000. If this list ever DID escape, `_validate_raw()` would refuse it
|
|
# at `MAX_PLAUSIBLE_EUR_KWH` rather than price a day on it.
|
|
index = _points_from_columnar_json(body, expect_unit="EUR / MWh",
|
|
scale=1.0,
|
|
provisional_default=False)
|
|
pairs = to_resolution([(p.start, p.eur_per_kwh) for p in index],
|
|
spec.resolution)
|
|
self.spec = spec
|
|
self.index_by_instant = {t.astimezone(timezone.utc): v
|
|
for t, v in pairs}
|
|
# ⚠️ `provisional=False` FOR THE SAME REASON `generic_dayahead` SAYS
|
|
# SO: the day-ahead auction is settled once published, and the columnar
|
|
# document carries no per-point flag to read one from. FCST-04's
|
|
# carry-forward is what marks the hours nobody published yet.
|
|
return [RawPoint(start=t, eur_per_kwh=price_eur_kwh(spec, v),
|
|
provisional=False)
|
|
for t, v in pairs]
|
|
|
|
def startup_preview(self, points) -> list:
|
|
"""`index EUR/MWh -> all-in EUR/kWh` for the first 24 h, once.
|
|
|
|
⚠️ THE INDEX COLUMN IS WHY THIS HOOK EXISTS. The all-in price alone
|
|
says nothing about which of the two numbers is wrong; printed beside
|
|
the index it came from, a factor of 1000 is one glance. See
|
|
`preview_lines()` for the rows and the min/median/max/spread line.
|
|
"""
|
|
if self.spec is None or not points:
|
|
return []
|
|
cut = points[0].start + timedelta(hours=PREVIEW_HOURS)
|
|
rows = []
|
|
for point in points:
|
|
if point.start >= cut:
|
|
break
|
|
index = self.index_by_instant.get(
|
|
point.start.astimezone(timezone.utc))
|
|
if index is not None:
|
|
rows.append((point.start, index, point.eur_per_kwh))
|
|
return preview_lines(self.spec, rows)
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# The one entry point
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
# Provider ids whose startup preview has already been emitted in this process.
|
|
# ⚠️ NOT A PLUGIN NAME IN THE FRAMEWORK: it is filled at runtime from whatever
|
|
# id was resolved, and starts empty. AC2 is intact.
|
|
_PREVIEWED: set = set()
|
|
|
|
|
|
def _emit_startup_preview(plugin, pid: str, points) -> None:
|
|
"""Log a plugin's preview once per process, and never fail because of it.
|
|
|
|
⚠️ THE ID IS RECORDED BEFORE THE HOOK RUNS, DELIBERATELY. A preview that
|
|
raises must not be retried every 30 minutes for the life of the add-on -
|
|
that would turn a cosmetic defect into a log flood on the one path §11.4
|
|
already makes hard to read.
|
|
"""
|
|
if pid in _PREVIEWED:
|
|
return
|
|
_PREVIEWED.add(pid)
|
|
try:
|
|
lines = list(plugin.startup_preview(points))
|
|
except Exception as e: # noqa: BLE001
|
|
_LOG.warning("%s could not show its start-up preview (%s: %s). Prices "
|
|
"are not affected.",
|
|
pid, type(e).__name__, e)
|
|
return
|
|
for line in lines:
|
|
_LOG.info("%s", line)
|
|
|
|
|
|
|
|
async def load_prices(opts: dict, since: datetime, until: datetime,
|
|
transport=None) -> PriceRead:
|
|
"""Resolve the configured provider, fetch, assemble all-in, return a read.
|
|
|
|
⚠️ RETURNS, NEVER RAISES, for every failure a site can actually have. §11.4
|
|
makes "no price data" a degradation row, and a degradation row that arrives
|
|
as a traceback into the 30 s planning tick is a controller that stops
|
|
regulating a battery because a supplier's TLS certificate expired.
|
|
|
|
The one exception is `FrameworkOption`, which is re-raised: that is not a
|
|
site condition, it is a plugin trying to read the markup, and turning it
|
|
into "no price data" would hide the exact defect trap 1 exists to catch.
|
|
"""
|
|
pid = provider_id(opts)
|
|
if not provider_configured(opts):
|
|
return PriceRead.no_data(
|
|
"no price feed is set (price_provider is 'off'), so the fixed "
|
|
"tariff is used, the plan is estimated and there is no trading")
|
|
cls = PROVIDERS.get(pid)
|
|
if cls is None:
|
|
return PriceRead.no_data(
|
|
f"unknown price provider {pid!r}. The known ones are "
|
|
f"{', '.join(sorted(PROVIDERS))}")
|
|
|
|
caps = cls.capabilities()
|
|
markup = float(price_option(opts, "price_markup_eur_kwh"))
|
|
|
|
# ⚠️ TARIFF-17: THE TABLE WINS OVER THE SCALAR, AND validate.py REFUSES THE
|
|
# CASE WHERE BOTH SAY SOMETHING. Two writers of one quantity, one of which
|
|
# silently loses, is the shape validate.py exists to refuse - so the
|
|
# precedence here is only ever exercised on a configuration that has
|
|
# already been accepted, and it is spelled out rather than left implicit.
|
|
taxes_key = "price_taxes_eur_kwh"
|
|
try:
|
|
table = taxes_profile(opts)
|
|
except Exception as e: # noqa: BLE001
|
|
# Not silent, and not a traceback into the 30 s tick. validate.py
|
|
# should have refused this at startup; if the add-on is running with it
|
|
# anyway, the operator is told which key and what is wrong with it.
|
|
_LOG.error("price_taxes_profile cannot be used (%s: %s), so no taxes "
|
|
"are added to this price update. Fix the table or clear it.",
|
|
type(e).__name__, e)
|
|
_LOG.debug("price_taxes_eur_kwh is not used instead, because a "
|
|
"silently different tax basis is the fault this setting "
|
|
"exists to remove.")
|
|
return PriceRead.no_data(
|
|
f"price_taxes_profile cannot be used: {e}")
|
|
if table is not None:
|
|
taxes = table
|
|
taxes_key = "price_taxes_profile"
|
|
else:
|
|
taxes = float(price_option(opts, "price_taxes_eur_kwh"))
|
|
|
|
# ⚠️ Not a refusal, but not silent either. A plugin whose feed is already
|
|
# all-in ignores these two numbers, and an operator who typed them has every
|
|
# reason to believe they took effect.
|
|
ignored = [n for c, n, v in ((Component.MARKUP, "price_markup_eur_kwh", markup),
|
|
(Component.TAXES, taxes_key, taxes))
|
|
if c not in caps.adds and v]
|
|
if ignored:
|
|
_LOG.warning("%s publishes a figure that already contains %s, so the "
|
|
"configured %s ignored - adding them would double-count",
|
|
pid,
|
|
"/".join(sorted(c.value for c in
|
|
caps.sources - {Component.ENERGY})),
|
|
" and ".join(ignored) +
|
|
(" is" if len(ignored) == 1 else " are"))
|
|
|
|
if transport is None:
|
|
transport = https_get
|
|
plugin = cls()
|
|
try:
|
|
raw = await plugin.fetch(PluginOptions(opts), since, until, transport)
|
|
_validate_raw(raw, caps)
|
|
points = assemble(caps, raw, markup, taxes)
|
|
except FrameworkOption:
|
|
raise # a plugin bug, not a site condition
|
|
except SourceUnavailable as e:
|
|
return PriceRead.no_data(scrub(f"{pid}: {e}", opts))
|
|
except Exception as e: # noqa: BLE001 - see the docstring
|
|
return PriceRead.no_data(
|
|
scrub(f"{pid}: {type(e).__name__}: {e}", opts))
|
|
|
|
_emit_startup_preview(plugin, pid, points)
|
|
return PriceRead.series(points, caps.product, pid,
|
|
datetime.now(timezone.utc))
|