admin/goodwe-addon
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

MAINT-20 part 1: custom_formula.py comments cut; new TARIFF-16 note

Browse Source 
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
glenn schrooyen
2026-11-01 01:17:46 +01:00
co-authored by Claude
parent a3f725e8a4
commit eda6d0aed1
2 changed files with 194 additions and 518 deletions
Download Patch File Download Diff File Expand all files Collapse all files
goodwe_controller/DECISIONS/TARIFF-16-the-injection-leg.md
+102
View File
@@ -0,0 +1,102 @@
# TARIFF-16 - the injection leg of a dynamic contract (`price_export_formula`)
**Status: SHIPPED.** Tests: `test_price.py` (17 series). Written down by
MAINT-20 from `app/custom_formula.py` and `app/price.py`. Follows [[TARIFF-15]],
whose parser it reuses.
## Why a formula, not the scalar
`price.py` modelled injection as one scalar. The market publishes a line a
scalar cannot hold:
Eneco Zon & Wind Dynamisch 0,1 x BELPEX-H - 1,188 c EUR/kWh
ENGIE Dynamic (B x eSpot_15) - A
Aspiravi (non-dynamic) 0,07 x Belpex - 2
1. Its own factor (Eneco 0,102 on consumption, 0,100 on injection; Aspiravi 0,07
against about 0,10): not the consumption `B` with a sign flip.
2. A negative offset (`- 1,188`, `- 2`); `price_markup_eur_kwh: float(0,1)`
cannot express one.
3. VAT-exempt ("Injectietarieven zijn vrijgesteld van btw") against 6 % on
consumption: representable only because `vat_pct` is per spec.
4. It goes negative with no floor: Eneco's is below zero for any BELPEX-H under
11,88 EUR/MWh, the hours a PV household exports in. You can be charged to
export.
`price_export_formula` is `custom_formula`'s parser with one argument moved
(`option`, a row of `_LEGS`), as TARIFF-17 did for `parse_static_profile()`. A
second parser would be a second `_finite()`, unknown-key rule, unit table and
`to_resolution()`, diverging first on the 23- and 25-hour days. Every earlier
call site passes no `option` and reads byte-identical sentences. The flat scalar
`export_price_eur_kwh()` is untouched and still answers when the option is "".
## What it does not affect
`arbitrage.py` does not read the injection price at all (9.3: near-zero Belgian
injection; it prices the sell side as displaced import), so no trade today turns
on this number and it is not a wrong-trade fix. It changes what an operator is
told and what a future export-aware strategy (OPT-06) rests on.
## The card cannot always grade it
Eneco's injection formula is printed with no price beside it. The `card` block
is required on the consumption leg and optional here (`FormulaSpec.card_graded`
is how callers ask; `card_discrepancy()` raises on an ungraded spec rather than
returning 0.0). The self-check runs wherever a printed estimate exists, and
validate.py says at startup that a leg is ungraded. Inventing an estimate would
grade the arithmetic against itself (MUTATION.md habit 4).
## `implausible_injection()`: bounds on the coefficients, not the price
The consumption band (`implausible_all_in()`) cannot be reused: it adds
`price_taxes_eur_kwh`, a consumption component, and its floor `energy >= 0.02`
would refuse the real market's negative hours. But that floor catches a whole
card read 100x too small, which the card check cannot see (it cancels). A band
spanning zero cannot catch 0.00132 EUR/kWh, so the bound moves onto the
coefficients, index-free and sign-free:
- pass-through `factor x UNITS[units] / 0.001` (one EUR/MWh is 0.001 EUR/kWh):
1.00 for Eneco's injection leg, 0.70 for Aspiravi's; 100x off reads 0.01 or 100;
- offset magnitude, on the absolute value (1,188 c = 0.01188; a hundredfold error
is 1.188 EUR/kWh).
Two rungs can both answer one stimulus, so they are graded on their reasons and
mutated together (MUTATION.md, TEST-16); test_price.py 17d builds fixtures so
each is the only one firing. It bounds the shape of the coefficients, never
their truth.
## The consumption band's asymmetry (`implausible_all_in()`)
The low end is graded on the energy component and the high end on the all-in
figure. `price_taxes_eur_kwh` ships at 0.11736 (TARIFF-18; 0.06 before), nearly
six times the low bound, so a card read 100x too small (0.0016 EUR/kWh) is
carried over the floor by the flat tax addend alone and would pass a band applied
to the all-in total. Raising the default made the trap wider. The ceiling is on
the all-in figure because that is what the site is billed. It grades the card's
own point, not every hour: live EPEX hours may fall outside (a negative hour is
the best charging opportunity of the year, 17 keeps a regression case). It is
called from validate.py only, since it reads the framework-owned taxes number.
## Smaller rules
- `CARD_TOLERANCE` is 0.01 in the card's units, the last digit both real cards
print (Eneco 15,55 against 15.5471; ENGIE 7,522 against 7.5218). Every error
class it exists for is at least two orders of magnitude bigger.
- The card age warning (about 6 months) is never a refusal: ENGIE freezes a
contract's coefficients, so an old card can be exactly right.
- `units` is required with no default (a default is the factor-1000 class).
A zero factor is a fixed tariff in a dynamic one's clothes (every spread 0,
arbitrage never fires; 14's warning box), so the operator is pointed at the
first-class way to say one price. A negative factor inverts the market (charge
into the evening peak). `vat_pct: 0` is legitimate (injection is exempt; some
cards are quoted VAT-inclusive); the bound exists for the 106 / 1.06 / 6
confusion. An unknown key is refused, not ignored, so a user who types "expr"
is not priced on the fields they also typed. A non-finite factor would make
every price NaN; `True` is an `int` and would price the day at the index.
- `to_resolution()`: the bucket is a UTC hour, exact for Belgium (CET and CEST
are whole-hour offsets, so keying by UTC cannot collide the two 02:00s). It
decides how many points are emitted, not how long one covers ([[TARIFF-14]]). An
incomplete hour is averaged over what is there, not dropped or zero-filled.
- The startup preview prints the index that went in beside the price that came
out (`61.96 -> 0.13522`): every error this module refuses is invisible on the
status page, and a factor of 1000 is one glance there.
goodwe_controller/app/custom_formula.py
+92 -518
View File
@@ -1,174 +1,18 @@
"""TARIFF-15 - the customer's own tariff card, as two fixed coefficients.
WHAT THIS IS FOR, in one sentence: a Belgian dynamic contract prices energy as
`B x index + A` with coefficients that exist in exactly one place - the tariff
card in the customer's klantenzone - so the customer types those two numbers
and the add-on does every unit conversion and the VAT multiplication itself.
A Belgian dynamic contract prices energy as `B x index + A` with coefficients
that exist only on the customer's card. The customer types those two numbers;
the add-on does every unit conversion and the VAT multiplication. Fixed
coefficients, no expression evaluation of any kind (the string is settable over
HTTP, and an expression cannot be graded against the card). An unknown key is
refused, never ignored. validate.py refuses to start unless the formula
reproduces the card's printed estimate within CARD_TOLERANCE: that grades the
coefficients against the card, not the card against reality.
WHY A FORMULA AND NOT FOURTEEN MORE PLUGINS (§9.2, TARIFF-07)
=============================================================
There is NO machine-readable source of Belgian tariff coefficients anywhere.
CREG's consumer tariff-card pages 404, CREG Scan is a JS-only wizard, the
annual report carries the supplier list and no per-product coefficients, and
VREG was unreachable. Every coefficient found in research came off a supplier's
own PDF, one of them image-only. So there is nothing to scrape and a
per-supplier plugin would have no source for its numbers - fourteen plugins
would be fourteen copies of the same invention, which is the defect TARIFF-07
withdrew three plugins for. THE CUSTOMER IS THE ONLY PARTY WHO CAN SEE THEIR
OWN COEFFICIENTS, and the card they read them from is the only artefact that
can grade them.
THE SHAPE IS THE MARKET'S PUBLISHED ARITHMETIC, NOT OUR APPROXIMATION
====================================================================
Both suppliers whose 2026 cards were readable print the formula, the reference
index they assumed AND the resulting price, so the arithmetic below is
reproducible against something they published rather than against our own
guess:
Eneco (0.102 x 133.9905 + 1.00) x 1.06 = 15.5471 card prints 15,55
ENGIE (0.100 x 61.96 + 0.90) x 1.06 = 7.5218 card prints 7,522
Two independent suppliers, reproduced to the last printed digit, in
c EUR/kWh. test_price.py section 13 runs both through the SHIPPED code.
⚠️ THE DOUBLE-COUNTING MECHANISM, AND WHY THIS DESIGN IS SAFE AT ALL
====================================================================
This provider declares `sources = {ENERGY, MARKUP}` and NOT `ALL_IN`. A Belgian
card's `Tariefformule` line is THE ENERGY COMPONENT ONLY: Eneco puts network
tariffs in one table and levies in a second, and ENGIE states in so many words
that the formula "does not include fixed network/distribution costs, transport,
grid levies, or excise duties". Declaring ALL_IN would lie on behalf of every
user who typed what their card says, and would silently drop ~12 c EUR/kWh.
price.py already mechanises that a PLUGIN cannot double-count (`PluginOptions`,
`FRAMEWORK_KEYS`, `code_leaks()`). This provider introduces a risk none of those
can see, because it is not in the code: THE OPERATOR double-counting, by folding
their network charges into `offset` while `price_taxes_eur_kwh` carries them as
well. `code_leaks()` scans code. This is config.
So the card is the grader:
At startup validate.py computes
`(factor x card.index_eur_per_mwh + offset) x (1 + vat_pct/100)` and
REFUSES TO START unless it equals `card.expect_incl_vat` to within
CARD_TOLERANCE, naming the discrepancy.
An `offset` inflated by network charges CANNOT PASS: the card's printed estimate
is by construction the energy component alone, so adding ~12 c EUR/kWh moves
15.55 to ~27 and the add-on refuses. The same check catches VAT omitted, VAT
applied twice, a transposed digit, and a UNIT MISMATCH BETWEEN THE TWO SIDES -
coefficients read off the formula line while the expected price is typed in
another convention, which is the factor-10/100/1000 class as it actually
happens.
⚠️ THE ONE UNIT ERROR THE CARD CHECK CANNOT SEE IS THE ONE APPLIED TO BOTH
SIDES AT ONCE, and saying so is the difference between a guard and a slogan:
scaling the coefficients AND the expected price by the same 100 cancels out of
a comparison between them. That is what `implausible_all_in()` is for, and it
is a SEPARATE refusal on a SEPARATE quantity - the card's own point, all-in,
against the band a Belgian retail tariff can occupy.
⚠️ AND THE LIMIT, IN THE SAME BREATH AS THE CLAIM, exactly as price.py's trap 1
states its own: THE CHECK GRADES THE COEFFICIENTS AGAINST THE CARD, NOT THE CARD
AGAINST REALITY. A consistent-but-wrong pair - both numbers copied off the wrong
product, or off last year's card - reproduces its own printed price perfectly
and passes. Nothing here or anywhere else can catch that; what is mechanised is
that the two coefficients and the printed estimate AGREE, which is the whole of
the unit/VAT/decimal-point class and none of the "wrong contract" class.
⚠️ FIXED COEFFICIENTS. NO EXPRESSION EVALUATION OF ANY KIND.
============================================================
All 24 surveyed Belgian products, both injection legs and aWATTar AT fit
`B x index + A`, so an expression buys nothing the market needs. Against that:
* the add-on serves `POST config/save` -> `POST /addons/self/options` on its
own ingress page, so this string is SETTABLE OVER HTTP. `eval` on it is
remote code execution in a root container holding SUPERVISOR_TOKEN, with
`homeassistant_api` and `hassio_api`, driving an inverter over RS485.
(`eval(s, {"__builtins__": {}})` is not a sandbox and never was.)
* ⚠️ AND THE DECIDING ARGUMENT IS NOT SECURITY - AN EXPRESSION IS
UNVERIFIABLE. Two coefficients can be graded against the card's printed
estimate. An arbitrary expression cannot be graded against anything, so it
makes the invisible-wrong-coefficient problem STRICTLY WORSE, which is the
exact failure this provider exists to prevent.
Future shapes become NAMED FIELDS - `clamp_lo`, `factor_negative`,
`offset_bands` - each individually refusable and each individually gradeable
against a card. Never a grammar. `parse_price_formula()` therefore REFUSES AN
UNKNOWN KEY rather than ignoring it, so a user who types `"expr"` is told it
does not exist instead of being silently priced on the fields they also typed.
test_price.py section 13f walks this module's AST and fails if `eval`, `exec`,
`compile`, `__import__` or `literal_eval` ever appears in it.
WHERE THE NUMBERS COME FROM AND WHERE THEY GO
=============================================
`index` names the reference series. `epex_be` is the EPEX BE day-ahead curve in
EUR/MWh, which is what `generic_dayahead` already fetches - same feed, same
`brussels_date_range()` request, same columnar parser, same declared egress
hosts. No new parser and no new failure mode.
`resolution` says which series the contract is written against:
* `quarter_hourly` - the 96 points the feed publishes since Belgium's
15-minute MTU (2025-10-01), emitted as they are;
* `hourly` - the four quarters of each hour averaged into one point.
⚠️ NEITHER IS AN HOURLY ASSUMPTION IN `_Series.at()`'s SENSE (TARIFF-14). The
hourly option changes how many POINTS are emitted and their spacing; the window
each point covers is still derived from its neighbours by `_cover_table()`, and
nothing here tells anything downstream how long a point lasts. A feed that is
already hourly averages one member per hour, which is the identity - so the
option cannot silently assume the granularity it was handed.
`units` is NOT bureaucracy. Two suppliers publish the same shape in different
units and the market aggregator prints both conventions in one column, so the
add-on has to be told which one the card is in - and the unit conversion is the
add-on's job precisely because it is one of the two steps that silently costs a
factor of 10, 100 or 1000. The other is VAT, which is also the add-on's.
TARIFF-16 - THE SECOND LEG, AND WHY IT IS THIS PARSER AND NOT A SECOND ONE
==========================================================================
A Belgian dynamic contract has TWO published lines, not one, and the second is
what the site is PAID for export:
Eneco Zon & Wind Dynamisch 0,1 x BELPEX-H - 1,188 c EUR/kWh
ENGIE Dynamic (B x eSpot_15) - A
Aspiravi (non-dynamic) 0,07 x Belpex - 2
Same `B x index + A` shape - which is why `price_export_formula` is this
module's parser with ONE argument moved (`option`), exactly as TARIFF-17 made
`parse_static_profile()`'s option name a parameter rather than forking the
period grammar. What differs between the two legs is FOUR numbers and a
handful of sentences, and all of them are rows of `_LEGS`:
* its OWN factor - Eneco 0,102 on consumption against 0,100 on injection,
Aspiravi 0,07 against ~0,10. NOT the consumption `B` with a sign flip;
* a NEGATIVE offset, which `factor`/`offset` already allow and which
`price_markup_eur_kwh: float(0,1)` cannot express at all;
* `vat_pct: 0` - "Injectietarieven zijn vrijgesteld van btw" - against 6 on
consumption, which is a SIX PERCENT ASYMMETRY between the buy leg and the
sell leg and is representable only because `vat_pct` is PER SPEC;
* a price that goes BELOW ZERO with no floor, in the hours a PV household
exports. `implausible_injection()` is the band that accepts that and still
refuses an implausible one; `implausible_all_in()` is NOT it, and its
docstring says why.
⚠️ AND THE CARD CANNOT ALWAYS GRADE IT. Eneco's injection formula is printed
with NO price beside it, so on that card there is nothing to reproduce. The
`card` block is therefore OPTIONAL on this leg and REQUIRED on the other, the
self-check runs wherever there is a printed estimate, and validate.py says at
startup when a leg is ungraded. Inventing an estimate to grade against would
be grading arithmetic against itself, which is the defect MUTATION.md habit 4
is about; TARIFF-15 demonstrated two tariffs end to end rather than three for
the same reason.
⚠️ NOTHING TRADES ON THIS TODAY, AND SAYING SO IS PART OF SHIPPING IT.
`arbitrage.py` does not read `export_price_eur_kwh` at all - its own comment
says so, citing §9.3's near-zero Belgian injection price - and both hours it
compares are all-in CONSUMPTION prices. So this is not a wrong-trade defect
and must not be described as one. What it changes is what an operator is told,
and what a future export-aware strategy would rest on.
"""
The provider declares `{ENERGY, MARKUP}`, never `ALL_IN` (a card's formula is
the energy component only). TARIFF-16 added the injection leg through this same
parser. DECISIONS/TARIFF-15-the-card-is-the-grader.md and
DECISIONS/TARIFF-16-the-injection-leg.md."""
import json
import math
@@ -183,13 +27,11 @@ class FormulaInvalid(Exception):
validate.py at startup - never at 03:00 on a fetch."""
# --------------------------------------------------------------------------- #
# The enumerations. Every one of these is a CLOSED set, and that is the point:
# an unknown value is refused by name rather than guessed at.
# --------------------------------------------------------------------------- #
# The enumerations. Each is a CLOSED set: an unknown value is refused by name,
# not guessed.
# unit -> how many EUR/kWh one of it is. ⚠️ THE WHOLE FACTOR-1000 CLASS LIVES
# HERE, so the table is tiny, explicit, and has no "default".
# unit -> EUR/kWh per one of it. The whole factor-1000 class lives here: tiny,
# explicit, no default.
UNITS: dict = {
"c_eur_per_kwh": 0.01, # eurocent per kWh - Eneco and ENGIE print this
"eur_per_kwh": 1.0, # some aggregators print this
@@ -198,54 +40,42 @@ UNITS: dict = {
RESOLUTIONS: tuple = ("hourly", "quarter_hourly")
# index id -> the human name of the series it refers to. One entry today; a
# second index is a new row here plus a fetch path, not a new grammar.
# index id -> the series it names. A second index is a new row here plus a
# fetch path.
INDEXES: dict = {
"epex_be": "Belgian day-ahead price, EUR/MWh",
}
# ⚠️ 0.01 IN THE CARD'S OWN UNITS, WHICH IS THE LAST DIGIT BOTH REAL CARDS
# PRINT. Eneco prints 15,55 against a computed 15.5471 (0.0029 out) and ENGIE
# prints 7,522 against a computed 7.5218 (0.0002 out), so the tolerance has to
# absorb the card's own rounding and nothing more. Every error class this check
# exists for - a factor of 10, of 100, of 1000, a missing or doubled 1.06, a
# transposed digit, ~12 c EUR/kWh of network charge folded into `offset` - is
# at least two orders of magnitude bigger than this.
# 0.01 in the card's own units, the last digit both real cards print (Eneco
# 15,55 vs 15.5471; ENGIE 7,522 vs 7.5218): the card's rounding and nothing
# more.
# DECISIONS/TARIFF-16-the-injection-leg.md.
CARD_TOLERANCE = 0.01
# The all-in plausibility band, EUR/kWh. `MAX_PLAUSIBLE_EUR_KWH` in price.py
# bounds only prices that are too LARGE; this is the other end, and it is the
# one the factor-1000 error falls off.
# The all-in plausibility band, EUR/kWh: the low end, where a factor-1000 error
# falls off.
PLAUSIBLE_LO_EUR_KWH = 0.02
PLAUSIBLE_HI_EUR_KWH = 1.00
# ~6 months. ⚠️ A WARNING, NEVER A REFUSAL: ENGIE freezes a contract's
# coefficients for its duration, so a card two years old can be exactly right
# and a site refused over one would be a site that cannot start.
# ~6 months. A WARNING, never a refusal: ENGIE freezes a contract's
# coefficients.
CARD_STALE_DAYS = 183
_CARD_KEYS = ("month", "index_eur_per_mwh", "expect_incl_vat")
_TOP_KEYS = ("index", "resolution", "factor", "offset", "units", "vat_pct",
"card")
# --------------------------------------------------------------------------- #
# TARIFF-16 - the SECOND leg, and the one parameter that carries it
# --------------------------------------------------------------------------- #
EXPORT_FORMULA_OPTION = "price_export_formula"
# ⚠️ THE PASS-THROUGH FRACTION, WHICH IS WHY THE NEXT TWO BOUNDS CAN BE
# INDEX-FREE. One EUR/MWh IS 0.001 EUR/kWh, so `factor x UNITS[units] / 0.001`
# is the dimensionless share of the wholesale price the contract passes
# through - 1.00 for Eneco's injection leg (0,100 c EUR/kWh per EUR/MWh), 0.70
# for Aspiravi's (0,07). A coefficient typed a hundred times too small or too
# large moves this by a hundred, whatever the index and whatever the sign of
# the price.
# The pass-through fraction, `factor x UNITS[units] / 0.001`: index-free and
# sign-free, so a coefficient typed 100x off moves it by 100 whatever the
# price.
EUR_KWH_PER_EUR_MWH = 0.001
# The two plausibility bounds for an INJECTION leg. ⚠️ NEITHER LOOKS AT THE
# SIGN OF THE PRICE, AND THAT IS THE WHOLE POINT - see
# `implausible_injection()`.
# The two plausibility bounds for an INJECTION leg; neither looks at the sign
# of the price.
PASSTHROUGH_LO = 0.10 # 10% of the wholesale price
PASSTHROUGH_HI = 10.0 # 1000% of it
INJECTION_OFFSET_MAX_EUR_KWH = 0.20
@@ -254,18 +84,8 @@ INJECTION_OFFSET_MAX_EUR_KWH = 0.20
@dataclass(frozen=True)
class _Leg:
"""What ONE option's coefficients price, as sentences its refusals read.
⚠️ TARIFF-16 MADE `option` A PARAMETER RATHER THAN FORKING THE PARSER,
exactly as TARIFF-17 made `option`/`means` parameters of
`parse_static_profile()` rather than forking the period grammar. A second
copy of this parser would be a second `_finite()`, a second unknown-key
rule, a second unit table and a second `to_resolution()` - and the two
would diverge first on the 23- and 25-hour days, where nobody looks.
Everything that differs between the consumption leg and the injection leg
is a STRING in this table; the arithmetic, the enumerations, the card
comparison and the resolution rule are ONE implementation with one set of
checks over it.
"""
`option` is a parameter, not a fork of the parser (TARIFF-16, as TARIFF-17 did
for `parse_static_profile()`)."""
option: str
what: str # "consumption" / "injection", for a sentence
@@ -316,12 +136,8 @@ _LEGS = {
@dataclass(frozen=True)
class FormulaSpec:
"""One contract's published arithmetic. SEVEN numbers and no code.
⚠️ EVERY FIELD IS A FIXED, NAMED SCALAR. There is deliberately no field
that holds an expression, a callable, a lambda or a template, and there is
no place to put one: `parse_price_formula()` refuses an unrecognised key.
"""
"""One contract's published arithmetic: SEVEN numbers and no code. Every field is
a fixed named scalar; `parse_price_formula()` refuses an unrecognised key."""
index: str
resolution: str
@@ -332,8 +148,8 @@ class FormulaSpec:
card_month: str = "" # "YYYY-MM", as the card is dated
card_index_eur_per_mwh: float = None
card_expect_incl_vat: float = None # in `units`, as the card prints it
# TARIFF-16. Which OPTION this spec was read from, so a refusal, a log line
# and `card_graded` name the leg the operator actually typed.
# TARIFF-16. Which OPTION this spec was read from, so refusals name the leg
# typed.
option: str = "price_formula"
@property
@@ -342,24 +158,14 @@ class FormulaSpec:
@property
def card_graded(self) -> bool:
"""Is there a printed estimate to grade these coefficients against?
⚠️ FALSE IS A REAL, SHIPPED STATE ON THE INJECTION LEG and is reported
rather than defaulted: Eneco's injection formula is printed with no
price beside it. Never False on `price_formula`, where `_card()`
refuses an absent block.
"""
"""Is there a printed estimate to grade these coefficients against? False is a
real state on the injection leg (Eneco prints none); never False on `price_formula`."""
return self.card_expect_incl_vat is not None
@property
def passthrough(self) -> float:
"""The dimensionless share of the wholesale price this passes through.
`factor x UNITS[units] / 0.001`. 1.00 for Eneco's injection leg, 0.70
for Aspiravi's. ⚠️ INDEX-FREE AND SIGN-FREE, which is what lets
`implausible_injection()` bound a leg whose price legitimately goes
below zero.
"""
"""The dimensionless share of the wholesale price this passes through. Index-free
and sign-free, which lets `implausible_injection()` bound a leg that goes negative."""
return self.factor * self.unit_to_eur_kwh / EUR_KWH_PER_EUR_MWH
@property
@@ -368,46 +174,27 @@ class FormulaSpec:
return self.offset * self.unit_to_eur_kwh
# --------------------------------------------------------------------------- #
# The arithmetic. Two functions, and the SECOND one is the contract.
# --------------------------------------------------------------------------- #
def in_card_units(spec: FormulaSpec, index_eur_per_mwh: float) -> float:
"""`(factor x index + offset) x (1 + vat_pct/100)`, in the card's units.
This is the line the card prints, spelled exactly as the card spells it,
and it is the only place the coefficients are combined. The card
self-check and the price path both go through here, so a mutation to it
cannot move the price without also moving the self-check.
"""
"""`(factor x index + offset) x (1 + vat_pct/100)`, in the card's units: the line
the card prints. The only place the coefficients are combined, so the card
check and the price path cannot diverge."""
return ((spec.factor * float(index_eur_per_mwh) + spec.offset)
* (1.0 + spec.vat_pct / 100.0))
def price_eur_kwh(spec: FormulaSpec, index_eur_per_mwh: float) -> float:
"""The energy+markup price for one index value, in EUR/kWh.
⚠️ THE UNIT CONVERSION IS THE ADD-ON'S JOB AND IT HAPPENS HERE, ONCE. The
user types what their card prints, in the card's own units; every step
from there to EUR/kWh is code, because those steps are exactly the ones
that silently cost a factor of 10, 100 or 1000 when a human does them.
"""
"""The energy+markup price for one index value, in EUR/kWh. The unit conversion
happens here, once: those steps are the ones that cost a factor of 10, 100 or 1000."""
return in_card_units(spec, index_eur_per_mwh) * spec.unit_to_eur_kwh
def card_discrepancy(spec: FormulaSpec) -> float:
"""How far the coefficients land from the card's own printed estimate.
In the card's units, signed: positive means the formula computes MORE than
the card prints, which is the direction an `offset` carrying network
charges moves it.
⚠️ RAISES ON A SPEC WITH NO CARD RATHER THAN RETURNING 0.0 (TARIFF-16). An
ungraded injection leg has no printed estimate, and a discrepancy of zero
is exactly what "the coefficients reproduce the card" looks like - so a
silent zero here would report the strongest possible result for the one
state in which nothing was compared at all.
"""
"""How far the coefficients land from the card's printed estimate, in the card's
units, signed: positive means MORE than the card prints (the direction an
`offset` carrying network charges moves it). Raises on a spec with no card
rather than returning 0.0 (TARIFF-16)."""
if not spec.card_graded:
raise FormulaInvalid(
f"{spec.option} has no card block, so there is no printed "
@@ -417,18 +204,10 @@ def card_discrepancy(spec: FormulaSpec) -> float:
- spec.card_expect_incl_vat
# --------------------------------------------------------------------------- #
# Reading the option
# --------------------------------------------------------------------------- #
def _finite(value, field: str, option: str = "price_formula") -> float:
"""A real, finite number - or a refusal naming the field.
⚠️ SAME CLASS AS `_validate_raw()`'s NaN RULE, one layer earlier. A NaN
factor makes every price NaN, every comparison False, and an arbitrage
strategy that silently never fires; `True` is an `int` in Python and would
price the whole day at the index.
"""
"""A real, finite number, or a refusal naming the field (NaN would price every hour NaN; `True` is an `int`)."""
if isinstance(value, bool) or not isinstance(value, (int, float)):
raise FormulaInvalid(
f"{option}.{field} is {value!r}, which is not a number. Write a "
@@ -441,15 +220,9 @@ def _finite(value, field: str, option: str = "price_formula") -> float:
def _card(raw, leg: _Leg = None) -> tuple:
"""The `card` block: the estimate the coefficients are graded against.
⚠️ REQUIRED ON THE CONSUMPTION LEG AND OPTIONAL ON THE INJECTION ONE, and
the asymmetry is a fact about the CARDS rather than a relaxed rule. See
`_LEGS[EXPORT_FORMULA_OPTION].card_note`: Eneco's injection formula is
printed with no price beside it, so on that card there is nothing to grade
against. `("", None, None)` is the ungraded answer and
`FormulaSpec.card_graded` is how every caller asks.
"""
"""The `card` block: the estimate the coefficients are graded against. Required on
the consumption leg, optional on the injection one because Eneco prints none
(`("", None, None)`; `FormulaSpec.card_graded`)."""
if leg is None:
leg = _LEGS["price_formula"]
if raw is None:
@@ -491,31 +264,8 @@ def _month_date(month: str):
def parse_price_formula(value, *, option: str = "price_formula"):
"""`price_formula` -> a `FormulaSpec`, or None when unset.
⚠️ `option` IS TARIFF-16'S WHOLE REUSE, and it is TARIFF-17's move. It
selects a row of `_LEGS` and nothing else: the enumerations, the
arithmetic, `to_resolution()`, the unknown-key rule and the card
comparison are shared, and every pre-existing call site - which passes no
`option` at all - lands on the consumption row and reads byte-identical
sentences. `parse_export_formula()` is a two-line wrapper, not a fork.
Accepts what the option can actually arrive as, exactly as
`parse_static_profile()` does:
* `None` / `""` / `"null"` - unset.
* a JSON string - what config.yaml's flat `str?` option
carries on a Supervisor install.
* a dict - already parsed, from a future nested
schema, from sim/inject.py, or a check.
Raises `FormulaInvalid`; never returns a half-built spec.
⚠️ THE CARD SELF-CHECK IS PART OF PARSING, not a separate step a caller
may forget. Both callers - validate.py's startup gate and the plugin's own
fetch - go through this one function, so there is no route to a price that
has not been graded against the card.
"""
"""`price_formula` -> a `FormulaSpec`, or None when unset. `option` selects a row of
`_LEGS`; everything else is shared, so `parse_export_formula()` is a wrapper."""
leg = _LEGS[option]
if value is None:
return None
@@ -534,12 +284,8 @@ def parse_price_formula(value, *, option: str = "price_formula"):
f"{option} is {type(value).__name__}, not a JSON object with "
f"{', '.join(_TOP_KEYS)}.")
# ⚠️ AN UNKNOWN KEY IS REFUSED, NOT IGNORED, AND THIS IS THE HALF THAT
# KEEPS THE "NEVER A GRAMMAR" PROMISE HONEST. A future shape is a NAMED
# field - clamp_lo, factor_negative, offset_bands - added here and
# refusable on its own. Ignoring unknown keys would mean a user who typed
# "expr" or "formula" gets priced on the fields they also typed, with the
# one they believed in dropped in silence.
# An unknown key is refused, not ignored: a future shape is a NAMED field,
# never a grammar.
unknown = sorted(set(value) - set(_TOP_KEYS))
if unknown:
raise FormulaInvalid(
@@ -559,10 +305,8 @@ def parse_price_formula(value, *, option: str = "price_formula"):
f"{option}.resolution is {value.get('resolution')!r}. Choose one "
f"of {list(RESOLUTIONS)}, as your tariff card says.")
# ⚠️ `units` IS REQUIRED AND HAS NO DEFAULT. A default here would be the
# factor-1000 class with a friendly face: the add-on would silently assume
# a convention the card never stated, and the card self-check would then
# be graded in units nobody chose.
# `units` is required with no default: a default is the factor-1000 class
# with a friendly face.
units = str(value.get("units", "") or "").strip()
if units not in UNITS:
raise FormulaInvalid(
@@ -573,29 +317,22 @@ def parse_price_formula(value, *, option: str = "price_formula"):
offset = _finite(value.get("offset"), "offset", option)
vat_pct = _finite(value.get("vat_pct"), "vat_pct", option)
# ⚠️ ZERO IS NOT "NO INDEXATION", IT IS A FIXED TARIFF WEARING A DYNAMIC
# ONE'S CLOTHES. Every hour comes out at the same price, so every spread is
# 0, arbitrage can never clear any threshold, and a strategy that never
# fires is indistinguishable from one that is switched off (§14's warning
# box). The add-on already has a first-class way to say "one price": point
# the operator at it rather than let them build it by accident.
# Zero is a fixed tariff in a dynamic one's clothes (arbitrage never
# fires); point the operator at the first-class fixed price.
if factor == 0.0:
raise FormulaInvalid(
f"{option}.factor is 0, so every hour gets the same {leg.what} "
f"price{leg.zero_factor}. For one flat price, {leg.flat}.")
# ⚠️ A NEGATIVE FACTOR INVERTS THE MARKET, and the direction is the danger:
# the dearest hour becomes the cheapest one, so the battery is told to
# charge into the evening peak and to discharge at 03:00. That is not a
# mispriced site, it is a site actively trading against itself.
# A negative factor inverts the market: charge into the evening peak,
# discharge at 03:00.
if factor < 0.0:
raise FormulaInvalid(
f"{option}.factor is {factor:g}, which turns the market upside "
f"down. {leg.neg_factor}.")
# ⚠️ 0 IS LEGITIMATE AND MUST STAY SO: injection is VAT-exempt in Belgium,
# and some cards are already quoted VAT-inclusive. The bound exists for the
# 106 / 1.06 / 6 confusion, not to make VAT compulsory.
# 0 is legitimate (injection is VAT-exempt; some cards are VAT-inclusive);
# the bound is for the 106 / 1.06 / 6 confusion.
if not 0.0 <= vat_pct <= 30.0:
raise FormulaInvalid(
f"{option}.vat_pct is {vat_pct:g}. Write a percentage between 0 "
@@ -607,13 +344,10 @@ def parse_price_formula(value, *, option: str = "price_formula"):
card_month=month, card_index_eur_per_mwh=card_index,
card_expect_incl_vat=card_expect, option=option)
# ⚠️ THE CARD SELF-CHECK STILL RUNS WHEREVER THERE IS A CARD TO RUN IT
# AGAINST, on either leg, with the same tolerance. What TARIFF-16 adds is
# the honest answer for the leg where the card prints no estimate: NOT a
# weakened comparison and NOT an estimate computed here, but two bounds on
# the COEFFICIENTS that need no printed price - see
# `implausible_injection()` - and a startup line saying the leg is
# ungraded.
# The card self-check still runs wherever there is a card, with the same
# tolerance. Where the card prints no estimate, `implausible_injection()`
# bounds
# the coefficients and a startup line says the leg is ungraded.
if spec.card_graded:
delta = card_discrepancy(spec)
if abs(delta) > CARD_TOLERANCE:
@@ -626,60 +360,15 @@ def parse_price_formula(value, *, option: str = "price_formula"):
def parse_export_formula(value):
"""`price_export_formula` -> the INJECTION leg's `FormulaSpec`, or None.
TARIFF-16. THE DEFECT THIS EXISTS TO CLOSE: `price.py` modelled injection
as ONE scalar, and the market publishes a line with four properties a
scalar cannot hold. Verbatim off the cards:
Eneco Zon & Wind Dynamisch 0,1 x BELPEX-H - 1,188 c EUR/kWh
"Het injectietarief is dynamisch en kan per uur wijzigen"
- identical on the January and the September 2026 cards
ENGIE Dynamic (B x eSpot_15) - A c EUR/kWh
Aspiravi (non-dynamic) 0,07 x Belpex - 2
1. ITS OWN FACTOR. Eneco uses 0,102 on consumption and 0,100 on
injection; Aspiravi 0,07 against ~0,10 on its consumption leg. It is
NOT the consumption `B` with a sign flip.
2. A NEGATIVE OFFSET - `- 1,188`, `- 2`. Note that
`price_markup_eur_kwh: float(0,1)` cannot express a negative markup
at all, so the offset has no home on the consumption side either.
3. VAT-EXEMPT. "Injectietarieven zijn vrijgesteld van btw", while
consumption is at 6% - a SIX PERCENT ASYMMETRY between the buy leg
and the sell leg, which a single scalar cannot even state.
4. IT GOES NEGATIVE, WITH NO FLOOR. Eneco's injection price is negative
for any BELPEX-H below 11,88 EUR/MWh, and a PV household exports
precisely in the hours most likely to be negative. You can be charged
to export.
⚠️ AND WHAT THIS DOES **NOT** AFFECT TODAY, SAID HERE SO NOBODY READS THE
ABOVE AS A TRADING BUG. `arbitrage.py` does not read
`export_price_eur_kwh` at all - its own comment says so, and gives §9.3's
near-zero Belgian injection price as the reason the economically sound
trade on this market is charge-cheap-to-displace-expensive-IMPORT. Both
hours it compares are all-in CONSUMPTION prices. So no trade the add-on
takes today is decided by this number, and this card is not a correction
of a wrong trade. What it changes is what an operator is told, and what
any future export-aware strategy would rest on: OPT-06 and any export
logic would otherwise inherit a scalar that cannot represent the market,
and the 6% VAT asymmetry would be invisible in the arithmetic.
⚠️ THE SCALAR IS NOT REPLACED. `export_price_eur_kwh()` is untouched and
still answers for every site with a flat injection price, which is every
site that exists today; this option ships "". That is `price.py`'s own
prediction coming true rather than a new design - "when it does land, an
injection curve gets a `PriceProvider` of its own and this scalar becomes
its FIXED-product default; the seam is already the right shape for that."
"""
"""`price_export_formula` -> the INJECTION leg's `FormulaSpec`, or None (TARIFF-16).
Same parser with `option` moved; the flat `export_price_eur_kwh()` still answers
when unset. Nothing trades on it today. DECISIONS/TARIFF-16-the-injection-leg.md."""
return parse_price_formula(value, option=EXPORT_FORMULA_OPTION)
def card_refusal(spec: FormulaSpec, delta: float) -> str:
"""The sentence an operator reads when their formula misses its own card.
It names both numbers and what to check. The usual causes are in DOCS.md,
because each of these errors looks like a working tariff on the status page.
"""
"""The sentence an operator reads when their formula misses its own card: both
numbers and what to check (causes are in DOCS.md)."""
got = in_card_units(spec, spec.card_index_eur_per_mwh)
leg = _LEGS[spec.option]
return (
@@ -688,53 +377,14 @@ def card_refusal(spec: FormulaSpec, delta: float) -> str:
f"{spec.card_expect_incl_vat:g}. Check {leg.card_causes}.")
# --------------------------------------------------------------------------- #
# The two questions the PLUGIN cannot ask, because they need framework numbers
# --------------------------------------------------------------------------- #
def implausible_all_in(spec: FormulaSpec, taxes_eur_kwh: float):
"""The card's own point, if its LEVEL cannot be a Belgian retail price.
Returns a refusal sentence or None. ⚠️ CALLED FROM validate.py AND FROM
NOWHERE ELSE, because it reads a FRAMEWORK-owned number: a plugin that
could see `taxes` could add it a second time, which is price.py's trap 1.
⚠️ THE LOW END IS GRADED ON THE ENERGY COMPONENT AND THE HIGH END ON THE
ALL-IN FIGURE, AND THAT ASYMMETRY IS THE CHECK RATHER THAN AN OVERSIGHT.
`price_taxes_eur_kwh` ships at 0.11736 (TARIFF-18; it was 0.06 before, and
the argument was already decisive at that figure), which is nearly SIX
times the low bound - so a card read a HUNDRED times too small comes out at
0.0016 EUR/kWh of energy, is carried over the floor by the flat tax addend
alone, and passes a band applied to the all-in total. The hundredfold error
is precisely what this exists to catch, so the floor is applied where it
can still see it. ⚠️ Raising the default made this trap WIDER, not
narrower: the addend that does the carrying got bigger. The ceiling is
applied to the all-in figure because that is the number the site is really
billed at, and taxes only push it up.
⚠️ AND THE LIMIT, WITH THE CLAIM. This grades the CARD'S OWN POINT - one
price, at the monthly index the card assumed - and not every hour of every
day. A live EPEX hour can and should fall outside this band (a negative
hour is the best charging opportunity of the year, and §17 keeps a
regression case for it), so the band is deliberately NOT applied to the
fetched series. What it catches is a formula whose whole LEVEL is wrong,
which is the factor-of-1000 class arriving from the end
`MAX_PLAUSIBLE_EUR_KWH` does not bound.
⚠️ AND IT IS THE CONSUMPTION LEG'S BAND ONLY - TARIFF-16 DELIBERATELY DID
NOT POINT IT AT THE INJECTION LEG. Two reasons, and the second is an
acceptance criterion rather than a preference:
* it adds `taxes_eur_kwh`, and `price_taxes_eur_kwh` is a CONSUMPTION
component. Nothing is added to what a site is PAID for export, so the
`all_in` half would be adding a number the injection leg never carries;
* its floor is `energy >= 0.02 EUR/kWh`, and a REAL Eneco injection price
is negative for any BELPEX-H below 11,88 EUR/MWh - the hours a PV
household actually exports in. Applied to injection this floor would
refuse the market, and refuse it hardest in the hours where knowing the
number matters most. `implausible_injection()` is the injection band
and it is deliberately NOT this function with a lower floor.
"""
"""The card's own point, if its LEVEL cannot be a Belgian retail price. Refusal
sentence or None. Called from validate.py only (it reads the framework-owned
taxes). The low end is graded on the energy component and the high end on the
all-in figure; the consumption leg's band only. It grades the card's own point,
not the fetched series. DECISIONS/TARIFF-16-the-injection-leg.md."""
energy = price_eur_kwh(spec, spec.card_index_eur_per_mwh)
all_in = energy + float(taxes_eur_kwh)
if energy >= PLAUSIBLE_LO_EUR_KWH and all_in <= PLAUSIBLE_HI_EUR_KWH:
@@ -747,49 +397,10 @@ def implausible_all_in(spec: FormulaSpec, taxes_eur_kwh: float):
def implausible_injection(spec: FormulaSpec):
"""The INJECTION leg's plausibility band. Returns a refusal or None.
⚠️ A NEGATIVE INJECTION PRICE IS ACCEPTED, NOT REFUSED, AND NEITHER RUNG
BELOW LOOKS AT THE SIGN OR THE LEVEL OF A PRICE. Eneco's published line
`0,1 x BELPEX-H - 1,188` is negative for any BELPEX-H under 11,88 EUR/MWh;
those are the midday hours a PV household exports in, so a band with a
floor above zero - which is what `implausible_all_in()` has, for the
consumption leg's own good reasons - would refuse the real market and
would hide it in exactly the hours it costs money. You CAN be charged to
export. The add-on says so rather than declining to represent it.
THE PROBLEM THAT LEAVES, STATED BEFORE THE SOLUTION. The consumption
floor was not decoration: it is what catches a whole card read a hundred
times too small, the one unit error the card self-check cannot see because
it cancels out of a comparison between the two sides. A band that spans
zero cannot catch it - 0.00132 EUR/kWh is inside any interval that
contains -0.03. So the bound moves OFF the price and ONTO the
coefficients, where it is index-free and sign-free:
* PASS-THROUGH. `factor x UNITS[units] / 0.001` is the dimensionless
share of the wholesale price the contract passes through, because one
EUR/MWh IS 0.001 EUR/kWh. Eneco's injection leg is 1.00 and Aspiravi's
is 0.70. A factor typed 100x too small reads 0.01 and 100x too large
reads 100, whatever the index and whatever the sign of the result.
* OFFSET MAGNITUDE. Eneco's is 1,188 c EUR/kWh = 0.01188 EUR/kWh and
Aspiravi's is 2 c = 0.02. A hundredfold error is 1.188 EUR/kWh, which
is not an injection tariff in any direction. The bound is on the
ABSOLUTE value, so a negative offset of a believable size passes and
that is the normal case on this leg.
⚠️ TWO RUNGS THAT CAN BOTH ANSWER ONE STIMULUS, SO THEY ARE GRADED ON
THEIR REASONS AND MUTATED TOGETHER (MUTATION.md, TEST-16). A whole card
read 100x too large trips both. The fixtures in test_price.py 17d are
built so that each rung is the ONLY one that fires on its own stimulus.
⚠️ AND WHAT THIS CANNOT SEE, WITH THE CLAIM (habit 13). It bounds the
SHAPE of the coefficients, never their truth: a contract really written
against a different index, or last year's coefficients, passes both rungs
and reproduces any card typed beside them. Where a card DOES print an
estimate the self-check grades the pair against it; where it does not -
which is Eneco's injection line - these two bounds are the whole of what
is mechanised, and validate.py says so at startup.
"""
"""The INJECTION leg's plausibility band. Refusal or None. A negative price is
accepted; the bounds are on the coefficients (pass-through and offset
magnitude), since a band spanning zero cannot catch a card 100x too small. It
bounds the shape, not the truth. DECISIONS/TARIFF-16-the-injection-leg.md."""
if not PASSTHROUGH_LO <= spec.passthrough <= PASSTHROUGH_HI:
return (
f"{spec.option}.factor passes through {spec.passthrough:.4g}x the "
@@ -806,10 +417,7 @@ def implausible_injection(spec: FormulaSpec):
def stale_card(spec: FormulaSpec, today=None):
"""A warning when the card is older than ~6 months, or None. NEVER a
refusal: ENGIE freezes a contract's coefficients for its duration, so an
old card can be exactly right and refusing one would stop the add-on on a
correctly configured site."""
"""A warning when the card is older than ~6 months, or None. Never a refusal (ENGIE freezes coefficients)."""
dated = _month_date(spec.card_month)
now = today or datetime.now(timezone.utc).date()
if dated is None or (now - dated).days <= CARD_STALE_DAYS:
@@ -820,36 +428,14 @@ def stale_card(spec: FormulaSpec, today=None):
f"contract changed since, copy the new figures from your supplier.")
# --------------------------------------------------------------------------- #
# Resolution
# --------------------------------------------------------------------------- #
def to_resolution(pairs, resolution: str) -> list:
"""`[(instant, index EUR/MWh)]` -> the series the CONTRACT is written on.
`quarter_hourly` hands the feed's own points back, ascending.
`hourly` averages each hour's members into one point AT THE TOP OF THE
HOUR.
⚠️ THE BUCKET IS A UTC HOUR AND THAT IS EXACT FOR BELGIUM, not an
approximation: CET and CEST are both WHOLE-hour offsets, so a Brussels hour
and a UTC hour start at the same instant on every day of the year including
both transition days. Keying by a naive local hour is what collides the two
02:00s in October; keying by the UTC instant cannot.
⚠️ AND THIS IS NOT `_Series.at()`'s HOURLY ASSUMPTION COMING BACK
(TARIFF-14). It decides how many points are EMITTED, not how long one
covers - the coverage window is still derived from the neighbours in
`_cover_table()`. A feed that already publishes hourly points averages one
member per hour, i.e. the identity, so this cannot silently assume the
granularity it was handed.
⚠️ AN INCOMPLETE HOUR IS AVERAGED OVER WHAT IS THERE rather than dropped or
zero-filled. A horizon that starts at 09:20 has two quarters in its first
hour; their mean is the best statement available about that hour, and
dropping it would leave a hole `at()` answers with `Absence` for an hour
the feed did publish.
"""
`quarter_hourly` returns the feed's points, `hourly` averages each UTC hour
(exact for Belgium: whole-hour offsets) at the top of the hour. An incomplete
hour is averaged over what is there. It sets how many points are emitted, not
how long one covers (TARIFF-14)."""
ordered = sorted(pairs, key=lambda r: r[0].astimezone(timezone.utc))
if resolution == "quarter_hourly":
return list(ordered)
@@ -863,28 +449,16 @@ def to_resolution(pairs, resolution: str) -> list:
return [(k, sum(v) / len(v)) for k, v in sorted(buckets.items())]
# --------------------------------------------------------------------------- #
# The startup preview
# --------------------------------------------------------------------------- #
PREVIEW_HOURS = 24
PREVIEW_MAX_ROWS = 24
def preview_lines(spec: FormulaSpec, rows) -> list:
"""`[(instant, index EUR/MWh, all-in EUR/kWh)]` -> lines for the log.
THE DEFECT THIS EXISTS TO MAKE VISIBLE: every error this module refuses is
one the operator cannot see from the status page, and the two numbers that
would have shown it - the index that went IN and the price that came OUT -
are never printed beside each other anywhere else. A factor of 1000 is
obvious in one line of `61.96 -> 0.13522` and invisible in a plan.
⚠️ IT PRINTS A BOUNDED NUMBER OF ROWS AND SAYS SO. At `quarter_hourly` the
next 24 h is 96 points; the cap is a row COUNT rather than "every fourth
point", because sampling every fourth point would assume the granularity
TARIFF-14 spent a card removing the assumption of.
"""
"""`[(instant, index EUR/MWh, all-in EUR/kWh)]` -> lines for the log: the index
that went IN beside the price that came OUT. Prints a bounded row COUNT, not
every fourth point (that would assume the granularity)."""
if not rows:
return []
shown = rows[:PREVIEW_MAX_ROWS]