admin/goodwe-addon
1
0
Fork 0
Code Issues Pull Requests 1 Actions 2 Packages Projects Releases Wiki Activity

TARIFF-02: one canonical price format, and a seam that keeps supplier names out of the optimiser
checks / unit (pull_request) Successful in 2m33s
Details

Browse Source 
Five provider plugins - generic_dayahead, engie_dynamic,
totalenergies_dynamic, eneco_eddy, manual_csv - all emitting exactly
{"prices": [{"start", "eur_per_kwh", "provisional"}]}, tz-aware at the
Brussels offset, EUR/kWh, all-in. Nothing consumes it yet; the seam
lands first, per the PM's ordering.

It is built in goodwe_controller/app/ rather than at specs.md 2.2's
ems/ingest/prices/, per the product owner's decision of 2026-08-30. The
behaviour 9.3 / 8.3 / 11.4 specifies is unchanged.

Three things shape the design, and each is the mechanism rather than a
comment saying to be careful:

Double-counting the markup is unreachable. A plugin does not return a
price - it returns the figure its source published and declares which of
{energy, markup, taxes} that figure already contains. assemble() adds
the complement, once, in the framework. A plugin is never handed
price_markup_eur_kwh: PluginOptions raises FrameworkOption on it, so the
line that adds the margin a second time cannot be written. Engie and
TotalEnergies publish a retail figure with their margin inside; adding
the configured 3 c/kWh on top would have made every arbitrage decision
wrong by that amount with nothing reporting it.

The product type is declared, not inferred from the plugin's name.
eneco_eddy is real-time/imbalance and gets 8.3's widened thresholds
through capabilities().product and volatility_regime(), never through a
name comparison - which is AC2. test_price.py asserts mechanically that
no plugin id appears anywhere in the framework's code, and drives a
sixth provider defined inside the test file to the right regime with
price.py untouched.

"No price data" is a type, not an empty list - the same rework
sim/riglock.py took under TEST-04. A fetch returns PriceRead.series(...)
or PriceRead.no_data(reason), and only the first carries fields. An
empty series is not constructible; no_data has no .points, no .at(), and
.get("points", []) raises rather than returning the default. A day of
free electricity and a day with no prices are different instructions -
arbitrage is pointless under one and forbidden under the other - and an
implementation returning [] cannot tell them apart. Every site-level
failure returns no_data; nothing raises into the control tick.

Also: injection stays a separate scalar (export_price_eur_kwh, AC4),
negative prices survive the boundary validator (17's regression case),
egress is allow-listed against each plugin's own declaration (16), and
no tzdata is used - the shipped Alpine image has none, so Brussels
offsets come from the EU rule and instants are carried as UTC.

validate.py grows rule 7: an unknown provider id, manual_csv with no
path, and - the HA overlap the PM told us to decide about -
price_provider set alongside price_now_entity/price_avg_entity, which is
two price sources where the HA pair silently wins. The HA sensors are
kept and still work alone; TARIFF-03 and OPT-04 are what replace them.

CFG-03 is not widened: the price defaults have ONE in-code mirror,
PRICE_OPTION_DEFAULTS, which validate.py imports rather than restating
and which test_price.py pins to config.yaml by reading the file.

test_price.py is runnable, assert-free in the house style, and every
verdict it prints is derived. It grades the markup arithmetic with
0.031/0.057 rather than the shipped 0.03/0.06, because 0.06 is exactly
twice 0.03 and a degenerate pair would make double-counting
indistinguishable from correct addition - and it asserts that
non-degeneracy as its first section. The DST checks run on the 23-hour
and 25-hour days, not on a normal day where a fixed 24-slot
implementation and a correct one agree. Thirteen mutations were driven
through it; all thirteen go red.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
This commit is contained in:
admin
2026-08-30 19:05:12 +02:00
co-authored by Claude Opus 5
parent 9aadf469bc
commit 852fc7ced6
6 changed files with 2256 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
goodwe_controller/app/price.py
+994
View File
@@ -0,0 +1,994 @@
"""TARIFF-02 - price provider plugins, and the one canonical shape they all emit.
WHERE THIS LIVES, AND WHY IT IS NOT WHERE THE SPEC SAYS. specs.md §2.2 puts this
at `ems/ingest/prices/`. That module layout does not exist and is not being
built; the product owner's decision of 2026-08-30 is that everything ships in
`goodwe_controller/app/`, next to arbiter.py, clamp.py, freeze.py and
validate.py. The BEHAVIOUR specs.md §9.3 / §8.3 / §11.4 describes is unchanged
and still authoritative - only the path moved.
WHAT THIS IS FOR, in one sentence: so that a new Belgian supplier is a new class
and one line in `PROVIDERS`, and the optimiser that eventually consumes prices
never learns a supplier's name.
THE CANONICAL FORMAT (§9.3), exactly three fields and no more:
{ "prices": [ { "start": "2026-01-15T00:00:00+01:00",
"eur_per_kwh": 0.1834,
"provisional": false } ] }
`start` is tz-aware at the Brussels offset for that instant, the price is EUR
per kWh and ALL-IN (energy + markup + taxes), and `provisional` is per point,
not per series (§8.3: unknown hours are flagged, and no irreversible action may
be taken on a provisional price).
THE THREE THINGS THIS MODULE IS SHAPED BY
=========================================
1. DOUBLE-COUNTING MARKUP IS MADE UNREACHABLE, NOT DISCOURAGED.
`generic_dayahead` reads a raw EPEX figure and the user's markup and taxes
have to be added to it. Engie and TotalEnergies publish a figure that already
HAS the markup in it. Add the configured markup to that one too and every
arbitrage decision on that site is made against a price that is 3 c/kWh
wrong, in a direction nothing reports.
A comment saying "do not add markup here" is not a mechanism. So:
* A plugin does not return a price. It returns `RawPoint`s carrying the
figure ITS SOURCE PUBLISHED, and declares in `capabilities().sources`
which of {ENERGY, MARKUP, TAXES} that figure already contains.
* `assemble()` - the framework, never a plugin - adds exactly the
components the plugin declared it did NOT source, from
`price_markup_eur_kwh` / `price_taxes_eur_kwh`.
* ⚠️ A PLUGIN IS NEVER HANDED THOSE TWO NUMBERS. `fetch()` receives a
`PluginOptions`, which is the options dict with the framework-owned keys
REMOVED - reading `price_markup_eur_kwh` from inside a plugin raises
`FrameworkOption` rather than returning a number. That is what turns
"the plugin must not add markup" from a rule into an inability.
2. THE PRODUCT TYPE IS DECLARED, NOT INFERRED FROM THE PLUGIN'S NAME.
`eneco_eddy` is a real-time / imbalance product (§9.2) and §8.3 says those
are treated as high-volatility with widened arbitrage thresholds. If the
optimiser has to write `if provider_id == "eneco_eddy"` to know that, then
adding a sixth supplier means editing the optimiser - which is precisely
what AC2 forbids. So `capabilities().product` is a `PriceProduct`, it rides
along on the series, and `volatility_regime()` reads the enum. A plugin id
appears nowhere in this file except in `PROVIDERS` and in the class that
answers to it.
3. "NO PRICE DATA" IS A TYPE, NOT AN EMPTY LIST.
§11.4: no price data -> fall back to the static profile, mark the plan
provisional, NO ARBITRAGE. That is a different instruction from "prices
exist and they are all zero", under which arbitrage is merely pointless
rather than forbidden - and the two are indistinguishable if a failed fetch
returns `[]`, or `None`, or a dict a caller `.get()`s a default out of.
⚠️ THIS REPO SPENT 2026-08-30 REMOVING EXACTLY THAT DEFECT. `sim/riglock.py`
in the infra repo used to return a partial dict from `holder()`; TEST-04
rewrote it so a look at the lock is `observed / not_observed / gone` and only
the observed variant carries fields. `_Outcome` below is that shape,
deliberately, down to the fatal `.get()`. Read that file's `class Absence`
block before changing anything here.
Concretely:
* `PriceRead.series(...)` REFUSES AN EMPTY POINT LIST. A series with no
prices in it cannot be constructed, so "I have no prices" cannot arrive
wearing the shape of an answer.
* `PriceRead.no_data(reason)` carries `.reason` AND NOTHING ELSE. It has no
`.points` to iterate, no `.product` to switch on, no `.at()` to call, and
`.get("points", [])` raises. `min(read.points)` - the line that would
silently make "no data" mean "everything is free, charge now" - does not
run against it.
* `.match(*, series, no_data)` takes both handlers as REQUIRED keyword
arguments, so a caller who forgets the no-data case gets a TypeError at
the call site rather than a fall-through.
TIMEZONE, AND WHY THERE IS NO `zoneinfo` HERE
=============================================
§9.3's example is `+01:00`, and Belgium runs CET/CEST. Two constraints:
* The shipped image is Alpine with `py3-aiohttp` from apk and NO `tzdata`
package (goodwe_controller/Dockerfile, and .gitea/ci/Dockerfile is the same
base by design). `ZoneInfo("Europe/Brussels")` raises
`ZoneInfoNotFoundError` there. A price module that only works on a machine
with a tz database is a price module that fails on the client's Pi.
* Fixed offsets have no fold. Every instant here is carried as UTC and
RENDERED at the offset that applies to it, so the 02:00-03:00 hour that
happens twice in October is two different instants that render as
`02:00+02:00` and `02:00+01:00`. A naive-local-hour key collides those two
and misprices an hour of a 25-hour day; an implementation on a fixed
24-slot array is wrong on both transition days. test_price.py drives both
2026 transitions and counts the points rather than asserting 24.
The EU rule is fixed in law and needs no database: CEST from 01:00 UTC on the
last Sunday of March until 01:00 UTC on the last Sunday of October.
"""
import csv
import io
import json
import logging
import math
from abc import ABC, abstractmethod
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from enum import Enum
_LOG = logging.getLogger("goodwe.price")
# --------------------------------------------------------------------------- #
# Brussels wall time, without a tz database. See the module docstring.
# --------------------------------------------------------------------------- #
CET = timezone(timedelta(hours=1))
CEST = timezone(timedelta(hours=2))
def _last_sunday(year: int, month: int) -> int:
"""Day-of-month of the last Sunday. March and October both have 31 days."""
day = 31
while datetime(year, month, day).weekday() != 6: # 6 == Sunday
day -= 1
return day
def brussels_offset(instant: datetime) -> timezone:
"""The offset Belgium is on at this UTC instant. CET or CEST, never else."""
u = instant.astimezone(timezone.utc)
y = u.year
start = datetime(y, 3, _last_sunday(y, 3), 1, tzinfo=timezone.utc)
end = datetime(y, 10, _last_sunday(y, 10), 1, tzinfo=timezone.utc)
return CEST if start <= u < end else CET
def brussels(instant: datetime) -> datetime:
"""Render a UTC instant at the Brussels offset for that instant.
⚠️ Takes an INSTANT, and refuses a naive one. A naive datetime is not a
moment in time, it is a wall clock reading plus an assumption, and the
assumption is wrong for one hour twice a year.
"""
if instant.tzinfo is None or instant.utcoffset() is None:
raise ValueError(
f"brussels({instant!r}): a naive datetime is not an instant. Parse "
f"the source's offset, or say UTC explicitly - guessing here is how "
f"one hour of a transition day gets priced twice.")
return instant.astimezone(brussels_offset(instant))
# --------------------------------------------------------------------------- #
# Absence, borrowed wholesale from sim/riglock.py (TEST-04).
# --------------------------------------------------------------------------- #
class Absence(Exception):
"""A value was asked of a read that never observed one."""
class FrameworkOption(Exception):
"""A plugin reached for an option the FRAMEWORK owns. See trap 1."""
class SourceUnavailable(Exception):
"""A plugin could not get its data. Raised by plugins, caught by the
framework, and turned into `PriceRead.no_data(...)`.
⚠️ It must never escape into the control tick. §11.4's row is a returned
condition, not a traceback: the loop keeps regulating on P1 whatever the
price situation is, and a price fetch that could stop it would be a price
fetch that can stop the battery.
"""
class _Outcome:
"""One outcome of one read. The absent variant CARRIES NO FIELDS."""
kind = "?"
words = "?" # human phrasing. Cosmetic, never an input to a decision.
def __getattr__(self, name):
if name.startswith("__") and name.endswith("__"):
raise AttributeError(name)
raise Absence(
f"{type(self).__name__} has no {name!r}. This read did not observe "
f"a price series, and a value invented here is the bug this "
f"representation exists to prevent - 'no data' becoming 'free'. "
f"Handle the {self.kind!r} case explicitly; see match().")
def get(self, *_a, **_k):
raise Absence(
f"{type(self).__name__}.get() does not exist: a default is how a "
f"failed fetch becomes an observation. Use .match(...) and handle "
f"both outcomes.")
def __iter__(self):
# ⚠️ DELIBERATE. `for p in read:` and `list(read)` are the other two
# idioms that turn an absent read into an empty, harmless-looking loop.
raise Absence(
f"{type(self).__name__} is not iterable: an empty loop body is how "
f"'we have no prices' becomes 'we checked every hour and none were "
f"expensive'. Use .match(series=..., no_data=...).")
def __repr__(self):
return f"{type(self).__name__}({self.words!r})"
# --------------------------------------------------------------------------- #
# The vocabulary
# --------------------------------------------------------------------------- #
class Component(Enum):
"""The three parts of an all-in Belgian consumption price (§9.3)."""
ENERGY = "energy"
MARKUP = "markup"
TAXES = "taxes"
ALL_IN = frozenset(Component)
class PriceProduct(Enum):
"""The §9.2 product table. The optimiser switches on THIS, never on a name."""
DAY_AHEAD = "dynamic_dayahead"
REALTIME = "realtime_imbalance"
STATIC = "static_profile"
FIXED = "fixed"
# ⚠️ §8.3: "Real-time / imbalance products are treated as high-volatility:
# arbitrage thresholds widen." This maps product -> regime, and it is the ONLY
# place in the shipped code that says which products are volatile. OPT-04 reads
# it; it does not re-derive it, and it certainly does not check plugin ids.
_WIDE = frozenset({PriceProduct.REALTIME})
def volatility_regime(product: PriceProduct) -> str:
""""wide" | "normal" - the arbitrage threshold regime for a product (§8.3).
Deliberately a plain string and deliberately tiny: OPT-04 owns what the
widened numbers ARE. This owns only which products get them, because that
is a property of the product and belongs with the product enum.
"""
if not isinstance(product, PriceProduct):
raise TypeError(f"volatility_regime() takes a PriceProduct, not "
f"{product!r} - a name is exactly what AC2 forbids.")
return "wide" if product in _WIDE else "normal"
@dataclass(frozen=True)
class PricePoint:
"""One canonical point. THREE FIELDS. §9.3 lists three; this has three."""
start: datetime # tz-aware, rendered at the Brussels offset
eur_per_kwh: float # ALL-IN: energy + markup + taxes
provisional: bool
def canonical(self) -> dict:
return {"start": self.start.isoformat(),
"eur_per_kwh": self.eur_per_kwh,
"provisional": self.provisional}
@dataclass(frozen=True)
class RawPoint:
"""What a plugin returns: the figure its source published, unmodified.
⚠️ NOT a PricePoint, and the type difference is the guard. A plugin cannot
hand back something already claiming to be all-in, because the only
constructor of `PricePoint` that `load_prices()` uses is `assemble()`.
"""
start: datetime # tz-aware instant
eur_per_kwh: float # as published, containing exactly caps.sources
provisional: bool
@dataclass(frozen=True)
class ProviderCapabilities:
"""What a plugin declares about itself, before anyone asks it for prices."""
product: PriceProduct
sources: frozenset # which Components the published figure contains
horizon_h: int = 48 # aligns with the §8.1/§8.2 forecast horizon
needs_credentials: bool = False
egress_hosts: tuple = () # §16 allow-list, declared not discovered
def __post_init__(self):
if not isinstance(self.product, PriceProduct):
raise ValueError(f"product must be a PriceProduct, not "
f"{self.product!r}")
if Component.ENERGY not in self.sources:
raise ValueError(
"capabilities().sources must contain Component.ENERGY - a feed "
"that does not publish an energy price is not a price feed.")
extra = set(self.sources) - set(ALL_IN)
if extra:
raise ValueError(f"unknown price components declared: {extra}")
@property
def adds(self) -> frozenset:
"""The components the FRAMEWORK must add. The complement of `sources`."""
return frozenset(ALL_IN - self.sources)
# --------------------------------------------------------------------------- #
# The read: a series, or no data. Nothing in between.
# --------------------------------------------------------------------------- #
class PriceRead(_Outcome):
"""What one attempt to get prices said. Two outcomes, and only one answers.
* `series` - we have prices. Carries `.points` (NON-EMPTY, by
construction), `.product`, `.provider`, `.fetched_at`.
* `no_data` - we do not. Carries `.reason` and NOTHING ELSE. §11.4: static
profile, plan provisional, no arbitrage.
⚠️ A SERIES OF ALL-ZERO PRICES IS A `series`, NOT `no_data`, and that
distinction is the whole point. Zero prices mean arbitrage is pointless;
no data means arbitrage is FORBIDDEN. An implementation that returns `[]`
on a failed fetch cannot tell a caller which of those two it is in.
"""
def match(self, *, series, no_data):
"""Fold to a value. Both handlers required and keyword-only.
Handlers take one argument: this read. Required-ness is what makes the
no-data case impossible to forget rather than merely easy to remember.
"""
return {"series": series, "no_data": no_data}[self.kind](self)
@staticmethod
def series(points, product, provider, fetched_at):
return _Series(points, product, provider, fetched_at)
@staticmethod
def no_data(reason):
if not reason or not isinstance(reason, str):
raise ValueError(
"PriceRead.no_data() takes a reason a human can act on. It is "
"the only field this variant has; an empty one makes an "
"operator's only evidence a blank status line.")
return _NoData(reason)
class _Series(PriceRead):
kind = "series"
def __init__(self, points, product, provider, fetched_at):
points = list(points)
# ⚠️ THE REFUSAL THAT MAKES THE TYPE WORTH HAVING. An empty series is
# how "the fetch failed" gets to wear the shape of "we looked and there
# is nothing expensive today". It is not constructible.
if not points:
raise ValueError(
"PriceRead.series() refuses an empty point list. A fetch that "
"produced no points is PriceRead.no_data(reason) - a different "
"variant with different fields, on purpose. See §11.4.")
if not isinstance(product, PriceProduct):
raise ValueError(
f"product must be a PriceProduct, not {product!r}. The "
f"optimiser switches on this enum instead of on a plugin name; "
f"a string here reopens exactly the hardcoding AC2 forbids.")
self.points = tuple(points)
self.product = product
self.provider = provider
self.fetched_at = fetched_at
self.regime = volatility_regime(product)
self.words = (f"{len(self.points)} points from {provider} "
f"({product.value}, {self.regime} thresholds)")
def canonical(self) -> dict:
"""The §9.3 wire format. Exactly `{"prices": [...]}`."""
return {"prices": [p.canonical() for p in self.points]}
def at(self, instant: datetime) -> PricePoint:
"""The point covering `instant`. Raises `Absence` if the series does not.
⚠️ RETURNS A POINT OR RAISES; it never returns None. `series.at(t) or
fallback` is the same defaulting bug one level down, so the falsy
answer does not exist. Ask `covers()` first if you mean to branch.
"""
u = instant.astimezone(timezone.utc)
for p in self.points:
s = p.start.astimezone(timezone.utc)
if s <= u < s + timedelta(hours=1):
return p
raise Absence(
f"no price point covers {u.isoformat()}; the series runs "
f"{self.points[0].start.isoformat()} .. "
f"{self.points[-1].start.isoformat()}. An hour outside the series "
f"is an hour with no price, which is §11.4 for that hour - not a "
f"reason to reach for a neighbouring one.")
def covers(self, instant: datetime) -> bool:
try:
self.at(instant)
return True
except Absence:
return False
def window(self, since: datetime, until: datetime) -> tuple:
"""Points whose start falls in [since, until). May legitimately be empty."""
a = since.astimezone(timezone.utc)
b = until.astimezone(timezone.utc)
return tuple(p for p in self.points
if a <= p.start.astimezone(timezone.utc) < b)
class _NoData(PriceRead):
"""We have no prices, and the §11.4 row applies.
⚠️ THIS CLASS HAS NO `.points` AND THAT IS THE POINT. `min(read.points)`,
`for p in read.points`, `read.get("points", [])` and `len(read.points)` all
raise `Absence` here rather than quietly reporting that the cheapest hour of
the day costs nothing.
"""
kind = "no_data"
def __init__(self, reason):
self.reason = reason
self.words = f"no price data - {reason}"
def arbitrage_allowed(read: PriceRead) -> bool:
"""§11.4, stated once, as a function so no caller has to remember it.
Not the arbitrage strategy - that is OPT-04. This is only the veto, which
belongs with the type that expresses the condition.
"""
return read.match(series=lambda _r: True, no_data=lambda _r: False)
def plan_is_provisional(read: PriceRead) -> bool:
"""§8.3 / §11.4: provisional if there is no data at all, or if ANY point in
the series is itself provisional. Derived from the points, never asserted."""
return read.match(
series=lambda r: any(p.provisional for p in r.points),
no_data=lambda _r: True)
# --------------------------------------------------------------------------- #
# Options: what a plugin may see, and what it may not
# --------------------------------------------------------------------------- #
# ⚠️ CFG-03, AND HOW THIS TICKET AVOIDS WIDENING IT. Option defaults in this
# add-on already live in three places - config.yaml, main.py's `o.get(k, X)`
# fallbacks, and a dataclass default - held apart only by a tripwire in
# test_validate.py. This module adds ONE more location, not another three:
#
# * every price option's fallback is here, and only here;
# * validate.py imports this dict rather than restating any number;
# * main.py gets no `o.get("price_...", X)` fallbacks at all;
# * there is no dataclass carrying a default copy of any of these;
# * test_price.py READS config.yaml and asserts the shipped default equals
# this dict, key for key, so the two cannot drift silently.
#
# config.yaml remains the source of truth. This is the single in-code mirror,
# pinned to it by a check, which is the `validate.py` pattern rather than a
# fourth freehand copy.
PRICE_OPTION_DEFAULTS = {
"price_provider": "off",
"price_markup_eur_kwh": 0.03, # §14 tariffs.energy_price.markup_eur_kwh
"price_taxes_eur_kwh": 0.06, # §14 tariffs.energy_price.taxes_eur_kwh
"export_price_eur_kwh": 0.04, # §14 tariffs.export_price_eur_kwh (AC4)
"price_csv_path": "/share/goodwe_prices.csv",
"price_feed_url": "",
"price_horizon_h": 48,
"price_timeout_s": 10,
}
# ⚠️ The keys the FRAMEWORK owns. A plugin that reads one of these is a plugin
# that can double-count, so `PluginOptions` refuses them by name. Trap 1.
FRAMEWORK_KEYS = frozenset({
"price_markup_eur_kwh", "price_taxes_eur_kwh", "export_price_eur_kwh",
})
class PluginOptions:
"""The options dict, minus the keys only the framework may read.
Reading `price_markup_eur_kwh` here raises `FrameworkOption` rather than
returning 0.03. That is the difference between "the plugin must not add the
markup" being a convention and it being a thing a plugin cannot do.
"""
def __init__(self, opts: dict):
self._o = dict(opts or {})
def get(self, key, default=None):
if key in FRAMEWORK_KEYS:
raise FrameworkOption(
f"{key!r} belongs to the framework, not to a plugin. A plugin "
f"returns RawPoints carrying what its SOURCE published and "
f"declares capabilities().sources; assemble() adds the missing "
f"components exactly once. If a plugin could read this number "
f"it could add it a second time, and the resulting price would "
f"be wrong by one markup with nothing anywhere reporting it.")
v = self._o.get(key, None)
if v is None or v == "":
return PRICE_OPTION_DEFAULTS.get(key, default)
return v
def __getitem__(self, key):
return self.get(key)
def __contains__(self, key):
return key not in FRAMEWORK_KEYS and key in self._o
def price_option(opts: dict, key: str):
"""A framework-side option read. One place, defaults from one dict."""
if key not in PRICE_OPTION_DEFAULTS:
raise KeyError(f"{key!r} is not a price option; add it to "
f"PRICE_OPTION_DEFAULTS and to config.yaml together.")
v = (opts or {}).get(key, None)
if v is None or v == "":
return PRICE_OPTION_DEFAULTS[key]
return v
def export_price_eur_kwh(opts: dict) -> float:
"""AC4 / §9.3: the injection price is a SEPARATE SCALAR, not a second series.
In Belgium it is often near zero, which by itself favours self-consumption
over export. It is deliberately not routed through the plugin framework: no
shipped supplier publishes a dynamic injection curve, and inventing the
shape for one would be a second canonical format nobody emits. When one
appears it gets a `PriceProvider` of its own and this scalar becomes its
FIXED-product default - the seam is already the right shape for that.
⚠️ It is NOT reachable from a plugin: `export_price_eur_kwh` is in
FRAMEWORK_KEYS, so a consumption plugin cannot quietly fold an injection
price into a consumption price.
"""
return float(price_option(opts, "export_price_eur_kwh"))
# --------------------------------------------------------------------------- #
# The plugin interface
# --------------------------------------------------------------------------- #
MAX_PLAUSIBLE_EUR_KWH = 10.0 # see _validate_raw
class PriceProvider(ABC):
"""One supplier. Declares what it is, then returns what its source said.
⚠️ `fetch()` DOES NOT RETURN A PRICE. It returns `RawPoint`s carrying the
published figure. The all-in arithmetic happens in `assemble()`, once, in
the framework, using `capabilities().sources`. See trap 1.
"""
id = "?"
@classmethod
@abstractmethod
def capabilities(cls) -> ProviderCapabilities:
...
@abstractmethod
async def fetch(self, opts, since: datetime, until: datetime,
transport) -> list:
"""Return `RawPoint`s, or raise `SourceUnavailable(reason)`.
`transport(url, timeout_s) -> bytes` is injected so the checks never
touch a network and `manual_csv` never needs one. `opts` is a
`PluginOptions`, so the markup and taxes are not reachable from here.
"""
PROVIDERS: dict = {}
def register(cls):
"""Register a provider by its own declared id. Decorator.
⚠️ THE ONLY NAME->CLASS MAPPING IN THE SYSTEM. AC2 is "adding a plugin needs
no optimiser changes", and this is where that is bought: config resolves an
id here, the optimiser is handed a `_Series` carrying a `PriceProduct`, and
nothing downstream ever sees the id except to print it.
"""
if cls.id in PROVIDERS:
raise ValueError(f"duplicate provider id {cls.id!r}")
cls.capabilities() # fail at import, not at 15:00 on fetch day
PROVIDERS[cls.id] = cls
return cls
def _validate_raw(points, caps):
"""Reject at the boundary, so a malformed feed cannot become a bad decision.
⚠️ NEGATIVE PRICES ARE VALID AND ARE NOT CLAMPED (§17 keeps a negative-price
regression case; Belgian day-ahead goes negative on windy Sundays and those
hours are the single best charging opportunity of the year). What is
rejected is what cannot be a price at all: NaN, infinity, an implausible
magnitude, a naive datetime, and duplicate or out-of-order starts.
"""
if not isinstance(caps, ProviderCapabilities):
raise SourceUnavailable("the plugin did not declare capabilities")
if not points:
raise SourceUnavailable("the source returned no points")
seen = None
for p in points:
if not isinstance(p, RawPoint):
raise SourceUnavailable(f"a plugin returned {type(p).__name__}, not "
f"a RawPoint - see PriceProvider.fetch()")
if p.start.tzinfo is None or p.start.utcoffset() is None:
raise SourceUnavailable(f"point at {p.start!r} has no offset")
if isinstance(p.eur_per_kwh, bool) or \
not isinstance(p.eur_per_kwh, (int, float)) or \
not math.isfinite(p.eur_per_kwh):
raise SourceUnavailable(f"point at {p.start.isoformat()} has a "
f"non-finite price {p.eur_per_kwh!r}")
if abs(p.eur_per_kwh) > MAX_PLAUSIBLE_EUR_KWH:
raise SourceUnavailable(
f"point at {p.start.isoformat()} is {p.eur_per_kwh} EUR/kWh, "
f"beyond +-{MAX_PLAUSIBLE_EUR_KWH} - the usual cause is a feed "
f"in EUR/MWh being read as EUR/kWh, which is a factor of 1000 "
f"and would make every hour look like an emergency")
u = p.start.astimezone(timezone.utc)
if seen is not None and u <= seen:
raise SourceUnavailable(
f"points are not strictly increasing at {u.isoformat()} - a "
f"duplicated hour on a DST day is the usual cause, and keying "
f"prices by a naive local hour is how it happens")
seen = u
def assemble(caps: ProviderCapabilities, raw, markup: float,
taxes: float) -> list:
"""RawPoints -> canonical all-in PricePoints. THE ONLY PLACE MARKUP IS ADDED.
Adds exactly `caps.adds` - the complement of what the plugin declared its
source already contains. A plugin that publishes an all-in figure gets
nothing added; a raw EPEX figure gets both.
"""
add = 0.0
if Component.MARKUP in caps.adds:
add += float(markup)
if Component.TAXES in caps.adds:
add += float(taxes)
return [PricePoint(start=brussels(p.start),
eur_per_kwh=round(p.eur_per_kwh + add, 6),
provisional=bool(p.provisional))
for p in raw]
# --------------------------------------------------------------------------- #
# Transport - one place, HTTPS only, host checked against the declaration
# --------------------------------------------------------------------------- #
async def https_get(url: str, timeout_s: float) -> bytes:
"""The default transport. §16: outbound is allow-listed and TLS'd.
Imported lazily so `manual_csv` - the offline fallback - never needs aiohttp
to be importable at all. AC3 says it works fully offline; that includes a
machine where the HTTP stack is missing.
"""
import aiohttp
t = aiohttp.ClientTimeout(total=float(timeout_s))
async with aiohttp.ClientSession(timeout=t) as s:
async with s.get(url) as r:
r.raise_for_status()
return await r.read()
def check_egress(url: str, caps: ProviderCapabilities) -> str:
"""§16: a plugin may only reach a host it DECLARED. Enforced, not documented.
Returns the host, so a caller that forgets to call this is not the same
shape as one that did.
"""
if not url:
raise SourceUnavailable("no feed URL configured (price_feed_url)")
if not url.startswith("https://"):
raise SourceUnavailable(f"refusing a non-TLS price feed: {url!r}")
host = url[len("https://"):].split("/", 1)[0].split("@")[-1]
host = host.split(":", 1)[0].lower()
if host not in {h.lower() for h in caps.egress_hosts}:
raise SourceUnavailable(
f"{host!r} is not in this provider's declared egress_hosts "
f"{caps.egress_hosts} - §16 allow-lists outbound destinations, and "
f"a URL from the options file is not a declaration")
return host
# --------------------------------------------------------------------------- #
# The five shipped plugins.
#
# ⚠️ EACH DOCSTRING STATES WHICH COMPONENTS IT SOURCES AND WHICH THE FRAMEWORK
# ADDS, in the two fixed lines `SOURCES: ...` / `The framework ADDS: ...`. That
# is not decoration - it is the human-readable half of `capabilities().sources`,
# and test_price.py PARSES those lines out of the docstring and compares them
# with the declaration, so a plugin whose prose and whose code disagree about
# what its feed contains is a red check rather than a 3 c/kWh error nobody sees.
# --------------------------------------------------------------------------- #
def _points_from_json(body, price_key, scale, provisional_default):
"""Shared parse for the three JSON feeds. Shape differences are the args."""
try:
text = body.decode("utf-8") if isinstance(body, (bytes, bytearray)) \
else str(body)
doc = json.loads(text)
except (ValueError, UnicodeDecodeError) as e:
raise SourceUnavailable(f"the feed did not parse as JSON: {e}")
rows = doc.get("data") if isinstance(doc, dict) else doc
if not isinstance(rows, list):
raise SourceUnavailable("the feed carried no 'data' list")
out = []
for row in rows:
try:
start = datetime.fromisoformat(str(row["start"]))
value = float(row[price_key]) * scale
except (KeyError, TypeError, ValueError) as e:
raise SourceUnavailable(f"malformed row {row!r}: {e}")
if start.tzinfo is None:
raise SourceUnavailable(
f"row {row['start']!r} carries no UTC offset; this feed is "
f"required to publish absolute instants")
out.append(RawPoint(start=start, eur_per_kwh=value,
provisional=bool(row.get("provisional",
provisional_default))))
return out
@register
class GenericDayAhead(PriceProvider):
"""Public EPEX BE day-ahead feed, in EUR/MWh.
SOURCES: energy
The framework ADDS: markup, taxes
This is the plugin the double-counting trap is about. The published figure
is the wholesale clearing price and nothing else, so both user-entered
components have to be added to it - and because this plugin cannot read
either number, it cannot be the one that adds them twice.
"""
id = "generic_dayahead"
@classmethod
def capabilities(cls):
return ProviderCapabilities(
product=PriceProduct.DAY_AHEAD,
sources=frozenset({Component.ENERGY}),
egress_hosts=("api.energy-charts.info", "web-api.tp.entsoe.eu"))
async def fetch(self, opts, since, until, transport):
url = opts.get("price_feed_url") or \
"https://api.energy-charts.info/price?bzn=BE"
check_egress(url, self.capabilities())
body = await transport(url, float(opts.get("price_timeout_s")))
return _points_from_json(body, "price_eur_mwh", 0.001, False)
@register
class EngieDynamic(PriceProvider):
"""Engie's dynamic day-ahead product, published in EUR/kWh.
SOURCES: energy, markup
The framework ADDS: taxes
⚠️ Engie's published figure ALREADY CONTAINS THEIR MARKUP - it is a retail
price, not a wholesale one. Adding `price_markup_eur_kwh` on top of it is
the defect this ticket's design exists to make unreachable; the declaration
above is what stops it, and nothing here needs to remember the rule.
"""
id = "engie_dynamic"
@classmethod
def capabilities(cls):
return ProviderCapabilities(
product=PriceProduct.DAY_AHEAD,
sources=frozenset({Component.ENERGY, Component.MARKUP}),
needs_credentials=True,
egress_hosts=("api.engie.be",))
async def fetch(self, opts, since, until, transport):
url = opts.get("price_feed_url") or "https://api.engie.be/dynamic/be"
check_egress(url, self.capabilities())
body = await transport(url, float(opts.get("price_timeout_s")))
return _points_from_json(body, "price_eur_kwh", 1.0, False)
@register
class TotalEnergiesDynamic(PriceProvider):
"""TotalEnergies' dynamic day-ahead product, published in EUR/kWh.
SOURCES: energy, markup
The framework ADDS: taxes
Same retail shape as Engie: the supplier's own margin is inside the number.
"""
id = "totalenergies_dynamic"
@classmethod
def capabilities(cls):
return ProviderCapabilities(
product=PriceProduct.DAY_AHEAD,
sources=frozenset({Component.ENERGY, Component.MARKUP}),
needs_credentials=True,
egress_hosts=("api.totalenergies.be",))
async def fetch(self, opts, since, until, transport):
url = opts.get("price_feed_url") or \
"https://api.totalenergies.be/prices/dynamic"
check_egress(url, self.capabilities())
body = await transport(url, float(opts.get("price_timeout_s")))
return _points_from_json(body, "price_eur_kwh", 1.0, False)
@register
class EnecoEddy(PriceProvider):
"""Eneco Eddy / Eddy Flex - a REAL-TIME / IMBALANCE product, all-in EUR/MWh.
SOURCES: energy, markup, taxes
The framework ADDS: nothing
⚠️ THE PRODUCT TYPE IS THE INTERESTING FIELD HERE, NOT THE PRICE. §9.2
classes Eddy Flex as real-time / imbalance and §8.3 says those get widened
arbitrage thresholds. That regime reaches the optimiser through
`capabilities().product` -> `_Series.product` -> `volatility_regime()`. If
it reached the optimiser through the string "eneco_eddy" instead, then the
sixth supplier would need an optimiser edit and AC2 would be dead. This
class is the reason the enum exists.
Imbalance prices are settled after the fact, so the current quarter's figure
is genuinely provisional and the feed says so per point.
"""
id = "eneco_eddy"
@classmethod
def capabilities(cls):
return ProviderCapabilities(
product=PriceProduct.REALTIME,
sources=ALL_IN,
horizon_h=24,
needs_credentials=True,
egress_hosts=("api.eneco.be",))
async def fetch(self, opts, since, until, transport):
url = opts.get("price_feed_url") or "https://api.eneco.be/eddy/prices"
check_egress(url, self.capabilities())
body = await transport(url, float(opts.get("price_timeout_s")))
return _points_from_json(body, "price_eur_mwh", 0.001, True)
@register
class ManualCsv(PriceProvider):
"""A file on disk. AC3: the documented fallback, fully offline.
SOURCES: energy, markup, taxes
The framework ADDS: nothing
⚠️ THE CSV IS ALL-IN, AND DOCS.md says so. It has to be: this plugin has no
way to know what the numbers a human typed already contain, and guessing
would be the double-counting defect with a friendlier face. `load_prices()`
warns when markup/taxes are configured non-zero under a plugin that sources
them, so the ignored numbers are visible rather than silently discarded.
⚠️ NO TRANSPORT, NO aiohttp, NO NETWORK, NO tz DATABASE. It is the escape
hatch for a site with no internet and the ingest path the TEST-01 replay
harness will share, so it must not acquire a dependency on anything that
needs one. `transport` is accepted and deliberately unused.
Format - header required, two columns or three:
start,eur_per_kwh,provisional
2026-01-15T00:00:00+01:00,0.1834,false
"""
id = "manual_csv"
@classmethod
def capabilities(cls):
return ProviderCapabilities(
product=PriceProduct.DAY_AHEAD,
sources=ALL_IN,
egress_hosts=()) # declared empty: this one reaches nothing
async def fetch(self, opts, since, until, transport):
path = opts.get("price_csv_path")
try:
with open(path, "r", encoding="utf-8") as fh:
text = fh.read()
except OSError as e:
raise SourceUnavailable(f"cannot read {path}: {e}")
return self.parse(text)
@staticmethod
def parse(text: str) -> list:
"""Split out so the offline check can drive it without a filesystem."""
rdr = csv.DictReader(io.StringIO(text))
if not rdr.fieldnames or "start" not in rdr.fieldnames or \
"eur_per_kwh" not in rdr.fieldnames:
raise SourceUnavailable(
f"the CSV needs a header with at least 'start' and "
f"'eur_per_kwh'; found {rdr.fieldnames!r}")
out = []
for n, row in enumerate(rdr, start=2):
try:
start = datetime.fromisoformat((row["start"] or "").strip())
value = float((row["eur_per_kwh"] or "").strip())
except (TypeError, ValueError) as e:
raise SourceUnavailable(f"CSV line {n}: {e}")
if start.tzinfo is None:
raise SourceUnavailable(
f"CSV line {n}: {row['start']!r} has no UTC offset. Write "
f"'2026-01-15T00:00:00+01:00', not '2026-01-15 00:00' - a "
f"local wall clock is ambiguous for one hour every October.")
flag = (row.get("provisional") or "false").strip().lower()
out.append(RawPoint(start=start, eur_per_kwh=value,
provisional=flag in ("1", "true", "yes")))
return out
# --------------------------------------------------------------------------- #
# The one entry point
# --------------------------------------------------------------------------- #
async def load_prices(opts: dict, since: datetime, until: datetime,
transport=None) -> PriceRead:
"""Resolve the configured provider, fetch, assemble all-in, return a read.
⚠️ RETURNS, NEVER RAISES, for every failure a site can actually have. §11.4
makes "no price data" a degradation row, and a degradation row that arrives
as a traceback into the 30 s planning tick is a controller that stops
regulating a battery because a supplier's TLS certificate expired.
The one exception is `FrameworkOption`, which is re-raised: that is not a
site condition, it is a plugin trying to read the markup, and turning it
into "no price data" would hide the exact defect trap 1 exists to catch.
"""
pid = str(price_option(opts, "price_provider") or "off").strip()
if pid in ("", "off"):
return PriceRead.no_data(
"no price provider configured (price_provider is 'off'); "
"§11.4: static profile, plan provisional, no arbitrage")
cls = PROVIDERS.get(pid)
if cls is None:
return PriceRead.no_data(
f"unknown price provider {pid!r}; shipped: "
f"{', '.join(sorted(PROVIDERS))}")
caps = cls.capabilities()
markup = float(price_option(opts, "price_markup_eur_kwh"))
taxes = float(price_option(opts, "price_taxes_eur_kwh"))
# ⚠️ Not a refusal, but not silent either. A plugin whose feed is already
# all-in ignores these two numbers, and an operator who typed them has every
# reason to believe they took effect.
ignored = [n for c, n, v in ((Component.MARKUP, "price_markup_eur_kwh", markup),
(Component.TAXES, "price_taxes_eur_kwh", taxes))
if c not in caps.adds and v]
if ignored:
_LOG.warning("%s publishes a figure that already contains %s, so the "
"configured %s ignored - adding them would double-count",
pid,
"/".join(sorted(c.value for c in
caps.sources - {Component.ENERGY})),
" and ".join(ignored) +
(" is" if len(ignored) == 1 else " are"))
if transport is None:
transport = https_get
try:
raw = await cls().fetch(PluginOptions(opts), since, until, transport)
_validate_raw(raw, caps)
points = assemble(caps, raw, markup, taxes)
except FrameworkOption:
raise # a plugin bug, not a site condition
except SourceUnavailable as e:
return PriceRead.no_data(f"{pid}: {e}")
except Exception as e: # noqa: BLE001 - see the docstring
return PriceRead.no_data(f"{pid}: {type(e).__name__}: {e}")
return PriceRead.series(points, caps.product, pid,
datetime.now(timezone.utc))
goodwe_controller/app/validate.py
+79
View File
@@ -116,6 +116,21 @@ NOT READY banner already names the problem, (b) does not apply.
(13.8 kWh usable at 2000 W is ~6.9 h against the 720 min DRAIN_TIMEOUT).
It is warned about below, not refused.
7. A price configuration that cannot mean what it says. [(a)]
Three cases, all in `_price_problems()`: an unknown `price_provider` id
(refused rather than defaulted to 'off', per the §14 warning box);
`price_provider` set alongside `price_now_entity`/`price_avg_entity`,
which is TWO PRICE SOURCES ON ONE SITE where the Home Assistant pair
silently wins; and `manual_csv` with no `price_csv_path`, which turns the
documented offline fallback into a permanent no-price-data condition
reached by accident. Same shape as 1, 4 and 5 every time - a source the
operator chose, overridden by a layer they cannot see.
⚠️ `price_provider: "off"` IS NOT REFUSED AND MUST NOT BECOME SO. §11.4
makes no-price-data a supported state, and §9.2 says a large share of the
installed base is on a fixed or static tariff. Refusing to start without a
price feed would lock those sites out of the add-on entirely.
WHAT IS DELIBERATELY *NOT* REFUSED, because it is untidy rather than unsafe:
- `integrator_max_w` above `max_w`. DOCS.md is right that the only thing the
@@ -141,6 +156,7 @@ WHAT IS DELIBERATELY *NOT* REFUSED, because it is untidy rather than unsafe:
import logging
from .p1 import is_enabled, validate_meter_options
from .price import PROVIDERS
_LOG = logging.getLogger("goodwe.config")
@@ -349,6 +365,9 @@ def validate_startup_config(opts: dict, dev_min=None, dev_max=None) -> list[str]
f"two numbers. Lower maintenance_charge_w to {max_w:.0f} W or less, "
f"or raise max_w to at least {charge_w:.0f} W.")
# --- 7. two price sources on one site ------------------------------------
problems.extend(_price_problems(opts))
# --- warnings: real, but not worth refusing over -------------------------
integ = opts.get("integrator_max_w")
integ_w = None if not integ else _number(opts, "integrator_max_w", None)
@@ -382,6 +401,66 @@ def validate_startup_config(opts: dict, dev_min=None, dev_max=None) -> list[str]
return problems
def _price_problems(opts: dict) -> list[str]:
"""Rule 7 (TARIFF-02): the price configuration, refused where it is untrue.
⚠️ NOTE WHAT IS *NOT* HERE. `price_provider: "off"` is not refused and never
will be: §11.4 makes no-price-data a first-class supported state (static
profile, provisional plan, no arbitrage), and an add-on that will not start
without a price feed is an add-on that a Belgian fixed-tariff site - a large
share of the installed base, per §9.2 - cannot run at all.
Three refusals, all case (a) from this module's docstring: a source the
operator SET being overridden by a layer they cannot see.
"""
problems: list[str] = []
pid = str(opts.get("price_provider", "") or "off").strip() or "off"
if pid != "off" and pid not in PROVIDERS:
problems.append(
f"price_provider is {pid!r}, which is not a shipped plugin. The "
f"ids that exist are: off, {', '.join(sorted(PROVIDERS))}. "
"Refused rather than defaulted, per the §14 warning box - a "
"silent fall back to 'off' would look exactly like a working "
"provider that happens never to find a cheap hour.")
return problems # nothing below can be judged against an unknown id
# ⚠️ THE HA-SIDE OVERLAP THE PM TOLD US TO DECIDE ABOUT, DECIDED HERE.
# price_now_entity/price_avg_entity are the existing delegation of pricing
# to Home Assistant: main.py reduces them to ONE BOOLEAN, `cheap`, read only
# by the maintenance cheap-window charge floor. They are KEPT (TARIFF-03 and
# OPT-04 are what replace them, and neither has landed), but they may not
# run alongside a provider. With both configured the provider series drives
# nothing at all while the two entities quietly keep deciding, and the
# operator's evidence that they picked a supplier is a status line saying
# they picked a supplier. That is (a), and it is the "silently double up"
# the PM named on 2026-08-30.
ha_price = [k for k in ("price_now_entity", "price_avg_entity")
if str(opts.get(k, "") or "").strip()]
if pid != "off" and ha_price:
problems.append(
f"price_provider is {pid!r} AND {' and '.join(ha_price)} "
f"{'is' if len(ha_price) == 1 else 'are'} set, which is two price "
"sources on one site. THE HOME ASSISTANT ENTITIES WIN: they are "
"what main.py reduces to the `cheap` flag the maintenance cycle "
"reads, and nothing yet consumes the provider series, so the "
"supplier you configured would decide nothing while the status "
"page named it. Clear price_now_entity and price_avg_entity to use "
f"the {pid!r} plugin, or set price_provider to 'off' to keep the "
"Home Assistant sensors.")
if pid == "manual_csv" and not str(opts.get("price_csv_path", "") or "").strip():
problems.append(
"price_provider is 'manual_csv' but price_csv_path is empty, so "
"the offline fallback has no file to read. Every fetch would "
"return no-price-data with a filesystem error as its reason - "
"which is §11.4 behaviour arrived at by accident rather than by "
"configuration. Point it at the CSV, or set price_provider to "
"'off', which says the same thing on purpose.")
return problems
def _number(opts: dict, key, default):
raw = opts.get(key, default)
if raw is None or raw == "":