TARIFF-15: your own tariff card, as two coefficients the card itself grades
checks / unit (pull_request) Successful in 7m14s
checks / unit (pull_request) Successful in 7m14s
A Belgian dynamic contract prices energy as `B x index + A`, and the two
coefficients exist on exactly one artefact: the customer's own tariff card.
There is no machine-readable source for them anywhere - CREG's consumer
tariff-card pages 404, CREG Scan is a JS wizard, the annual report has the
supplier list without per-product coefficients, VREG was unreachable, and
every coefficient found in research came off a supplier's own PDF, one
image-only. So nothing can be scraped and a per-supplier plugin would have no
source for its numbers, which is why TARIFF-07 withdrew three of them. The
customer is the only party who can see their own coefficients; they type them.
New provider `custom_formula`, one new option `price_formula` (a JSON string,
the `static_profile` precedent exactly), and a new module
`app/custom_formula.py`. The index path is `generic_dayahead`'s, not a second
one: the URL/override decision is now one shared `dayahead_feed_url()` both
plugins call, same feed, same Brussels-calendar range, same columnar parser,
same declared egress hosts.
It declares {ENERGY, MARKUP}, NOT ALL_IN. A card's Tariefformule line is the
energy component alone - Eneco tabulates network tariffs and levies
separately, ENGIE states the formula excludes network, distribution,
transport, levies and excise - so ALL_IN would lie for every operator who
typed what their card says and would drop the taxes from every hour
(0.2125 -> 0.1555 EUR/kWh, measured).
THE CARD IS THE GRADER, and that is why this is safe to ship. The framework
already mechanises that a PLUGIN cannot double-count. This introduces a risk
none of those guards can see because it is config, not code: the OPERATOR
double-counting, by folding network charges into `offset` while
price_taxes_eur_kwh carries them too. So validate.py recomputes the card's own
printed price from the operator's coefficients at startup and refuses to start
unless the two agree to within 0.01, naming the discrepancy. An inflated
offset cannot pass: 15.55 becomes 28.27. The same check catches VAT omitted,
VAT doubled, a transposed digit and a unit mismatch between the two sides.
And the limit, in the same sentence as the claim: it grades the coefficients
against the card, NOT the card against reality. A consistent-but-wrong pair
passes. A second refusal covers the one unit error the first cannot see - the
one applied to both sides at once - by bounding the card's own point to
0.02-1.00 EUR/kWh. Its FLOOR is applied to the energy figure rather than the
all-in one because the shipped 0.06 of taxes is three times that floor and
would carry a hundredfold error over it alone; found by writing the mutation,
not by review (M10, +0 B).
The market's own arithmetic, reproduced through the shipped code:
Eneco (0.102 x 133.9905 + 1) x 1.06 = 15.5471 card prints 15,55
ENGIE (0.1 x 61.96 + 0.9) x 1.06 = 7.5218 card prints 7,522
NO EXPRESSION EVALUATION OF ANY KIND. The string is settable over HTTP through
the add-on's own ingress page, so eval on it is RCE in a root container
holding SUPERVISOR_TOKEN and driving an inverter over RS485 - but the deciding
argument is that an expression is UNVERIFIABLE: two coefficients can be graded
against the card, an arbitrary expression cannot be graded against anything,
so it would make the invisible-wrong-coefficient problem strictly worse. An
unknown field is REFUSED rather than ignored, so `"expr"` is told it does not
exist. Future shapes become named fields, never a grammar. test_price.py 15f
walks the module's AST for eval/exec/compile/__import__/literal_eval and
exercises the scanner's red direction on every run.
Refused at startup: card absent or out by >0.01; `units` absent (no default);
factor == 0 (every spread 0, arbitrage silently never fires - the message
points at static_profile); factor < 0 (inverts the market); vat_pct outside
0-30 (0 stays legitimate); non-finite or non-numeric; unknown field, index or
resolution; the plausibility band. A card older than ~6 months WARNS and does
not refuse - ENGIE freezes coefficients for a contract's duration. Negative
output prices are not refused anywhere.
A startup preview logs the next 24 h as `index EUR/MWh -> all-in EUR/kWh` with
min/median/max and the day's spread, through a new default-silent
`PriceProvider.startup_preview()` hook - only the plugin knows what went in to
its own arithmetic. Emitted once per process, and a preview that raises cannot
cost the site its prices.
`custom_formula` joins the config.yaml enum through the DERIVED set
PROVIDERS | RETIRED_PROVIDERS | {"off"}, not a hardcoded list. Both
translations describe the option and the new id. DOCS.md replaces the
"planned" sentence with the shape, a worked example from a real card, a real
startup transcript and the limit of the check stated beside the claim.
22 mutations, one per check added, no survivors - MUTATION.md carries the
ledger. M13 (the preview's index column) was a -13 B survivor of the whole
suite on the first attempt because the check graded the HEADING and not the
row; it is graded on a rendered row now. All 29 suites green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
This commit is contained in:
Claude Opus 5
@@ -21,6 +21,7 @@ if it has, it needs 0.10.0. Whoever deploys decides.
|
||||
| _this branch_ | TARIFF-12 - a supplier outage stops deleting the plan on its first failed fetch, and TARIFF_MAX_AGE_S starts governing |
|
||||
| _this branch_ | TARIFF-14 - the price a trade is decided against is the price of the interval being traded |
|
||||
| _this branch_ | TARIFF-07 - three plugins that never fetched a price are withdrawn, and a site configured on one is told so **⚠️ BACKWARD-INCOMPATIBLE** |
|
||||
| _this branch_ | TARIFF-15 - `custom_formula`: your own tariff card's two coefficients, graded against the card's own printed price |
|
||||
|
||||
⚠️ **A CONFIG THAT STARTS ON THE CURRENT VERSION CAN BE REFUSED ON THE NEXT
|
||||
ONE.** TARIFF-07 withdraws `engie_dynamic`, `totalenergies_dynamic` and
|
||||
@@ -52,7 +53,152 @@ the customer's supplier login; CREG's tariff-card pages 404 and VREG was
|
||||
unreachable, so there is no public index to read them from. An alias to
|
||||
`generic_dayahead` — which reads *wholesale* — would have to invent the numbers
|
||||
a battery is priced against. A formula provider is the planned answer and is a
|
||||
separate card.
|
||||
separate card. *(Annotated 2026-09-13: that card is **TARIFF-15**, below, and it
|
||||
shipped on this branch as `custom_formula`. It is still not an alias — the
|
||||
operator types their own coefficients; nothing is invented on their behalf.)*
|
||||
|
||||
### TARIFF-15 - your own tariff card, as two coefficients
|
||||
|
||||
**New price provider `custom_formula`, and one new option, `price_formula`.**
|
||||
A Belgian dynamic contract prices energy as `B × index + A`; the two
|
||||
coefficients live on the customer's own tariff card and **nowhere else**.
|
||||
There is no machine-readable source for them anywhere — CREG's consumer
|
||||
tariff-card pages 404, CREG Scan is a JavaScript wizard, the annual report
|
||||
carries the supplier list without per-product coefficients, VREG was
|
||||
unreachable, and every coefficient found in research came off a supplier's own
|
||||
PDF, one image-only. So there is nothing to scrape and a per-supplier plugin
|
||||
would have no source for its numbers — which is precisely why TARIFF-07
|
||||
withdrew three of them. **The customer is the only party who can see their own
|
||||
coefficients.** They type them, once:
|
||||
|
||||
```json
|
||||
{"index": "epex_be", "resolution": "hourly",
|
||||
"factor": 0.102, "offset": 1.00,
|
||||
"units": "c_eur_per_kwh", "vat_pct": 6,
|
||||
"card": {"month": "2026-09", "index_eur_per_mwh": 133.9905,
|
||||
"expect_incl_vat": 15.55}}
|
||||
```
|
||||
|
||||
`price = (factor × index_eur_per_mwh + offset) × (1 + vat_pct/100)`, in
|
||||
`units`, **converted to EUR/kWh by the add-on**. The operator types what the
|
||||
card prints, in the card's own units; every unit conversion and the VAT
|
||||
multiplication are the add-on's job, because those are the two steps that
|
||||
silently cost a factor of 10, 100 or 1.06.
|
||||
|
||||
**The index is `generic_dayahead`'s, not a second one** — same EPEX BE feed,
|
||||
same Brussels-calendar range request, same columnar parser, same declared
|
||||
egress hosts. The URL decision is now one function both plugins call, so a
|
||||
short-horizon or override bug cannot exist in one and not the other.
|
||||
`resolution: quarter_hourly` emits the feed's 96 points as they are;
|
||||
`hourly` averages each hour's members. Neither is `_Series.at()`'s hourly
|
||||
assumption returning: it decides how many points are **emitted**, and the
|
||||
window each one covers is still derived from its neighbours (TARIFF-14).
|
||||
|
||||
⚠️ **It declares `{ENERGY, MARKUP}`, not `ALL_IN`, and that is the whole safety
|
||||
argument.** A Belgian card's `Tariefformule` line is the **energy component
|
||||
only** — Eneco tabulates network tariffs in one table and levies in a second,
|
||||
ENGIE states that the formula excludes network, distribution, transport, grid
|
||||
levies and excise. Declaring `ALL_IN` would lie on behalf of every operator who
|
||||
typed exactly what their card says, and would drop `price_taxes_eur_kwh` from
|
||||
every hour. Measured: 0.2125 → 0.1555 EUR/kWh, silently.
|
||||
|
||||
⚠️ **THE CARD IS THE GRADER, AND IT IS WHY THIS IS SAFE TO SHIP.** The
|
||||
framework already mechanises that a *plugin* cannot double-count
|
||||
(`PluginOptions`, `FRAMEWORK_KEYS`, `code_leaks()`). This provider introduces a
|
||||
risk none of those can see, because it is **config, not code**: the *operator*
|
||||
double-counting, by folding network charges into `offset` while
|
||||
`price_taxes_eur_kwh` carries them too. So:
|
||||
|
||||
> At startup `validate.py` computes
|
||||
> `(factor × card.index_eur_per_mwh + offset) × (1 + vat_pct/100)` and
|
||||
> **refuses to start** unless it equals `card.expect_incl_vat` to within
|
||||
> `0.01`, naming the discrepancy, the ratio and the likely causes.
|
||||
|
||||
An inflated `offset` **cannot pass**: the printed estimate is the energy
|
||||
component alone, so ~12 c€/kWh of network charge moves the computed 15.55 to
|
||||
28.27 and the add-on stops. The same check catches VAT omitted, VAT applied
|
||||
twice, a transposed digit, and a unit mismatch between the coefficients and the
|
||||
printed price.
|
||||
|
||||
⚠️ **And the limit, in the same sentence as the claim: it grades the
|
||||
coefficients against the card, not the card against reality.** A
|
||||
consistent-but-wrong pair — both numbers off the wrong product — reproduces its
|
||||
own printed price perfectly and passes. Nothing can catch that. A **second**
|
||||
refusal covers the one unit error the first cannot see, the one applied to both
|
||||
sides at once: the card's own point must land inside 0.02–1.00 EUR/kWh. Its
|
||||
floor is applied to the **energy** figure rather than the all-in one, because
|
||||
the shipped 0.06 of taxes is three times that floor and would carry a
|
||||
hundredfold error over it on its own — measured, and driven red by a mutation.
|
||||
|
||||
**The market's own arithmetic, reproduced to the digit the supplier prints.**
|
||||
Both cards that were readable print the formula, the index they assumed *and*
|
||||
the resulting price:
|
||||
|
||||
| | computed by the shipped code | the card prints |
|
||||
|---|---|---|
|
||||
| Eneco `(0.102 × 133.9905 + 1) × 1.06` | **15.5471** | **15,55** |
|
||||
| ENGIE `(0.1 × 61.96 + 0.9) × 1.06` | **7.5218** | **7,522** |
|
||||
|
||||
⚠️ **No expression evaluation of any kind, and a check that fails if one ever
|
||||
appears.** `price_formula` takes numbers; there is no field that accepts an
|
||||
expression and an **unrecognised field is refused rather than ignored**, so
|
||||
typing `"expr"` is told it does not exist instead of quietly pricing the site
|
||||
on the fields beside it. Two reasons, and the second decides it:
|
||||
|
||||
* the string is settable over HTTP — the add-on's own ingress page posts it to
|
||||
`/addons/self/options` — so `eval` on it is RCE in a root container holding
|
||||
`SUPERVISOR_TOKEN`, with `homeassistant_api` and `hassio_api`, driving an
|
||||
inverter over RS485;
|
||||
* **an expression is unverifiable.** Two coefficients can be graded against the
|
||||
card's printed price; an arbitrary expression cannot be graded against
|
||||
anything, so it would make the invisible-wrong-coefficient problem *strictly
|
||||
worse* — the exact failure this provider exists to prevent.
|
||||
|
||||
All 24 surveyed Belgian products, both injection legs and aWATTar AT fit
|
||||
`B × index + A`. Future shapes become **named fields** (`clamp_lo`,
|
||||
`factor_negative`, `offset_bands`), each individually refusable and each
|
||||
gradeable against a card. Never a grammar. `test_price.py` 15f walks
|
||||
`app/custom_formula.py`'s AST for `eval`/`exec`/`compile`/`__import__`/
|
||||
`literal_eval` and exercises the scanner's **red** direction on every run.
|
||||
|
||||
**Refused at startup, not at 03:00.** `card` absent or failing by more than
|
||||
0.01; `units` absent (no default, on purpose — the same convention is printed
|
||||
two ways by two suppliers); `factor == 0`, which makes every spread 0 and
|
||||
arbitrage silently never fire (the message points at `static_profile`, which is
|
||||
the first-class way to say "one price"); `factor < 0`, which inverts the market
|
||||
so the battery charges at the evening peak; `vat_pct` outside 0–30 (**0 stays
|
||||
legitimate** — injection is VAT-exempt and some cards are already VAT-inclusive);
|
||||
a non-finite or non-numeric coefficient; an unknown field; an unknown `index` or
|
||||
`resolution`; and the plausibility band above. A `card.month` older than about
|
||||
six months **warns and does not refuse** — ENGIE freezes coefficients for a
|
||||
contract's duration, so an old card is often exactly right.
|
||||
|
||||
⚠️ **Negative output prices are NOT refused.** They are legal on EPEX and they
|
||||
are the best charging hours of the year (§17 keeps the regression case). The
|
||||
card check bounds the formula's *magnitude* without constraining the sign of
|
||||
any hour.
|
||||
|
||||
**A startup preview, so a factor of 1000 is visible in one line.** The first
|
||||
time the feed answers, the add-on logs the next 24 h as
|
||||
`index EUR/MWh -> all-in EUR/kWh` with the min, median, max and the day's
|
||||
spread. It is a new framework hook (`PriceProvider.startup_preview()`, default
|
||||
silent) rather than a line in `load_prices()`, because only the plugin knows
|
||||
what went *in* to its own arithmetic; it is emitted once per process and it
|
||||
cannot stop a fetch — a preview that raises is caught and logged.
|
||||
|
||||
**Surfaces.** `config.yaml` gains `price_formula` (options **and** schema) and
|
||||
`custom_formula` joins the `price_provider` enum through the **derived** set
|
||||
`PROVIDERS | RETIRED_PROVIDERS | {"off"}` rather than a hand-typed list; both
|
||||
translations describe the option and the new id. `DOCS.md` replaces the
|
||||
"planned" sentence with the shape, a worked example from a real card, the
|
||||
startup transcript, and the limit of the check stated in the same sentence as
|
||||
the claim.
|
||||
|
||||
**Not in scope, and each is a card of its own:** the injection leg's own
|
||||
coefficients (Eneco uses 0,102 on consumption against 0,100 on injection, and
|
||||
injection is VAT-exempt where consumption is at 6 %); the hour-varying network
|
||||
charge that bites Wallonia; raising the `price_taxes_eur_kwh` default from 0.06
|
||||
to ~0.12.
|
||||
|
||||
### TARIFF-07 - three plugins that never fetched a price, withdrawn
|
||||
|
||||
@@ -130,6 +276,8 @@ which **approximates** a retail dynamic tariff and **cannot express
|
||||
`B × spot + A`** — it has no multiplier, and it cannot represent a negative
|
||||
offset, which `price_markup_eur_kwh: float(0,1)` could not hold in any case. A
|
||||
formula provider is named as the planned answer and is a separate card.
|
||||
*(Annotated 2026-09-13: TARIFF-15 replaced that sentence. The table is three
|
||||
rows and the prose points at `custom_formula`.)*
|
||||
|
||||
⚠️ **One check had to be rebuilt rather than deleted.** `test_price.py` 4b
|
||||
proved `code_leaks()`'s docstring strip is load-bearing by finding a shipped
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
# TARIFF-15 — the operator's own tariff card, and why the card has to grade it
|
||||
|
||||
**Status: DECIDED 2026-09-13 — ship `custom_formula` with FIXED coefficients and
|
||||
a REQUIRED `card` block whose printed price the add-on reproduces at startup or
|
||||
refuses to run. No expression evaluation, ever. Declare `{ENERGY, MARKUP}`,
|
||||
never `ALL_IN`.**
|
||||
|
||||
Filed with TARIFF-15. Follows [[TARIFF-07]], which withdrew three per-supplier
|
||||
plugins and named a formula provider as the answer it could not give.
|
||||
|
||||
---
|
||||
|
||||
## 1. Why a formula rather than more plugins
|
||||
|
||||
TARIFF-07 established the negative half: a per-supplier plugin has no source
|
||||
for its numbers. This card establishes the positive one.
|
||||
|
||||
* **There is no machine-readable source of Belgian tariff coefficients
|
||||
anywhere.** CREG's consumer tariff-card pages 404. CREG Scan is a
|
||||
JavaScript-only wizard. The annual report carries the supplier list and no
|
||||
per-product coefficients. VREG was unreachable. Every coefficient found in
|
||||
research came from a supplier's **own PDF**, one of them image-only.
|
||||
* So there is nothing to scrape, and fourteen plugins would still have no
|
||||
source for their numbers — they would be fourteen copies of the invention
|
||||
TARIFF-07 deleted three of.
|
||||
* **The customer is the only party who can see their own coefficients**, and
|
||||
the card they read them from **is the only artefact that can grade them**.
|
||||
That second clause is the whole design; everything below follows from it.
|
||||
|
||||
## 2. The shape is the market's published arithmetic
|
||||
|
||||
Both suppliers whose 2026 cards were readable print three things on one line:
|
||||
the formula, the reference index it assumes, **and the resulting price**.
|
||||
|
||||
| | computed | the card prints |
|
||||
|---|---|---|
|
||||
| Eneco `(0.102 × 133.9905 + 1) × 1.06` | **15.5471** | **15,55** |
|
||||
| ENGIE `(0.1 × 61.96 + 0.9) × 1.06` | **7.5218** | **7,522** |
|
||||
|
||||
Two independent suppliers, reproduced to the last printed digit, through the
|
||||
shipped code (`test_price.py` 15a runs both, and then runs both again end to
|
||||
end through `load_prices()`).
|
||||
|
||||
That third column is not a nicety. It is the only externally-published number
|
||||
in this whole feature, and it is what makes the design safe.
|
||||
|
||||
## 3. The double-counting risk, and which half of it is new
|
||||
|
||||
`price.py`'s trap 1 already mechanises that a **plugin** cannot double-count:
|
||||
`PluginOptions` refuses the framework keys, `code_leaks()` refuses a plugin
|
||||
whose *code* mentions them, `assemble()` adds the complement exactly once.
|
||||
|
||||
`custom_formula` introduces a risk none of those can see, **because it is not
|
||||
in code**: the **operator** double-counting, by folding their network charges
|
||||
into `offset` while `price_taxes_eur_kwh` also carries them. `code_leaks()`
|
||||
scans code. This is config, typed by a person reading a PDF.
|
||||
|
||||
It is a real reading of a real card, not a hypothetical. A Belgian card's
|
||||
`Tariefformule` line is the **energy component alone** — Eneco tabulates
|
||||
network tariffs in one table and levies in a second; ENGIE states in so many
|
||||
words that the formula excludes network, distribution, transport, grid levies
|
||||
and excise. An operator who wants "the price I actually pay" and adds ~12
|
||||
c€/kWh to `offset` has done something entirely reasonable and is now billed for
|
||||
network charges twice, on every hour, with §11.4 making the whole class of
|
||||
price defect invisible.
|
||||
|
||||
**Decision: the card grades it.** At startup `validate.py` recomputes the
|
||||
card's own printed price from the operator's coefficients and refuses to start
|
||||
unless the two agree to within `0.01` — which is the last digit a card prints.
|
||||
An inflated `offset` cannot pass, because the printed estimate is by
|
||||
construction the energy component: 15.55 becomes 28.27 and the add-on stops and
|
||||
says why.
|
||||
|
||||
**And the limit, stated in the same sentence as the claim**, exactly as
|
||||
`price.py` states trap 1's: *it grades the coefficients against the card, not
|
||||
the card against reality.* A consistent-but-wrong pair — both numbers off the
|
||||
wrong product, or off a card the contract has since rolled over from — agrees
|
||||
with itself perfectly and passes. Nothing here or anywhere else can catch that.
|
||||
|
||||
### 3b. The one unit error the card check cannot see
|
||||
|
||||
A factor applied to **both** sides — coefficients *and* printed price — cancels
|
||||
out of a comparison between them. That is a second quantity, so it is a second
|
||||
refusal: the card's own point must land inside 0.02–1.00 EUR/kWh.
|
||||
|
||||
⚠️ **Its floor is applied to the ENERGY figure and not to the all-in one, and
|
||||
that asymmetry was found by a mutation rather than by reasoning.** The shipped
|
||||
`price_taxes_eur_kwh` is 0.06 — three times the floor — so a card read a
|
||||
hundred times too small comes out at 0.0016 EUR/kWh of energy, is carried over
|
||||
0.02 by the flat tax addend alone, and passes a band applied to the total. The
|
||||
hundredfold error is exactly what the band exists for. Graded from both
|
||||
directions in `test_price.py` 15e/15i and driven red as M10.
|
||||
|
||||
## 4. Fixed coefficients. No expression evaluation of any kind
|
||||
|
||||
**Decided: never a grammar.** Two arguments, and the second is the one that
|
||||
decides it.
|
||||
|
||||
* **Security.** The add-on serves `POST config/save` → `POST
|
||||
/addons/self/options` on its own ingress page, so `price_formula` is
|
||||
**settable over HTTP**. `eval` on it is remote code execution in a root
|
||||
container holding `SUPERVISOR_TOKEN`, with `homeassistant_api` and
|
||||
`hassio_api`, driving an inverter over RS485. `eval(s, {"__builtins__": {}})`
|
||||
is not a sandbox and this repository is not going to pretend otherwise.
|
||||
* **⚠️ AND THE DECIDING ARGUMENT IS NOT SECURITY: AN EXPRESSION IS
|
||||
UNVERIFIABLE.** Two coefficients can be graded against the card's printed
|
||||
estimate. An arbitrary expression cannot be graded against **anything** — so
|
||||
it makes the invisible-wrong-coefficient problem *strictly worse*, which is
|
||||
the exact failure this provider exists to prevent. An expression would buy
|
||||
the operator freedom to be wrong in a way nothing could report.
|
||||
|
||||
All 24 surveyed Belgian products, both injection legs and aWATTar AT fit
|
||||
`B × index + A`, so an expression buys nothing the market needs either.
|
||||
|
||||
**Future shapes become NAMED FIELDS** — `clamp_lo`, `factor_negative`,
|
||||
`offset_bands` — each individually refusable and each individually gradeable
|
||||
against a card. `parse_price_formula()` therefore **refuses an unknown key**
|
||||
rather than ignoring it: an operator who types `"expr"` is told it does not
|
||||
exist, instead of being priced on the fields they also typed with the one they
|
||||
believed in dropped in silence.
|
||||
|
||||
`test_price.py` 15f walks the module's AST for `eval`, `exec`, `compile`,
|
||||
`__import__` and `literal_eval`, and exercises the scanner's **red** direction
|
||||
on every run so it cannot become a scanner that scans nothing.
|
||||
|
||||
## 5. `{ENERGY, MARKUP}`, not `ALL_IN`
|
||||
|
||||
The contract's formula **is** the supplier's markup — that is what `B` and `A`
|
||||
are — but it is not the levies. Declaring `ALL_IN` would lie on behalf of every
|
||||
operator who typed exactly what their card says and would drop
|
||||
`price_taxes_eur_kwh` from every hour: measured at 0.2125 → 0.1555 EUR/kWh.
|
||||
`price_markup_eur_kwh` is therefore **ignored** under this provider, and logged
|
||||
as ignored, which is the framework's existing rule rather than a new one.
|
||||
|
||||
## 6. What was reused rather than rebuilt
|
||||
|
||||
The index path is `generic_dayahead`'s: same EPEX BE feed, same
|
||||
Brussels-calendar range, same columnar parser, same declared egress hosts. The
|
||||
override/range decision moved into one shared `dayahead_feed_url()` that both
|
||||
plugins call, because two copies of that rule would be two places for the
|
||||
short-horizon defect TARIFF-09 fixed to come back in only one of them.
|
||||
|
||||
`resolution: hourly` averages each hour's members. ⚠️ **That is not
|
||||
`_Series.at()`'s hourly assumption returning (TARIFF-14):** it decides how many
|
||||
points are *emitted*; the window each point covers is still derived from its
|
||||
neighbours in `_cover_table()`. A feed that is already hourly averages one
|
||||
member per hour — the identity — so the option cannot silently assume the
|
||||
granularity it was handed. Graded.
|
||||
|
||||
## 7. What this card did NOT do
|
||||
|
||||
* **The injection leg.** Eneco uses `0,102` on consumption and `0,100` on
|
||||
injection, and injection is VAT-exempt where consumption is at 6 % — a second
|
||||
per-contract formula, not a second copy of this one. Separate card.
|
||||
* **The hour-varying network charge that bites Wallonia.** Separate card.
|
||||
* **Raising `price_taxes_eur_kwh` from 0.06 to ~0.12.** Separate card, and an
|
||||
owner's decision. Note §3b: the band's floor is deliberately insensitive to
|
||||
whatever that number becomes.
|
||||
+179
-10
@@ -116,7 +116,7 @@ whole rendered list, in both languages.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **All 71 options are always shown** | The frontend keeps a field when `name in options \|\| required`, and every key is in `options:`. There is no "advanced" fold |
|
||||
| **All 72 options are always shown** | The frontend keeps a field when `name in options \|\| required`, and every key is in `options:`. There is no "advanced" fold |
|
||||
| **Order comes from `schema:`** | `App.schema_ui` is `UiOptions(...)(data["schema"])`, so the **schema half** of `config.yaml` decides the order the tab renders in — not the `options:` half |
|
||||
| **There are no section headers** | Add-on options are a flat list: no header primitive, no per-option advanced flag, no units on the input, no sliders. Grouping is approximated by **order** plus the `Group · ` prefix in each label, and by nothing else |
|
||||
| **Groups, in task order** | Hardware → Meter → Limits → Tariff → Maintenance → Safety → Advanced |
|
||||
@@ -1369,7 +1369,7 @@ does, and only after you have read why.
|
||||
|
||||
| option | meaning |
|
||||
|---|---|
|
||||
| `price_provider` | `off` (default) or one of the two plugins below |
|
||||
| `price_provider` | `off` (default) or one of the three plugins below |
|
||||
| `price_markup_eur_kwh` | Your supplier's margin, EUR/kWh. **Added only when the chosen plugin's feed does not already contain it** |
|
||||
| `price_taxes_eur_kwh` | Levies and VAT, EUR/kWh. Same rule |
|
||||
| `export_price_eur_kwh` | The **injection** price. A separate scalar, not a series — see below |
|
||||
@@ -1378,6 +1378,7 @@ does, and only after you have read why.
|
||||
| `price_horizon_h` | How far ahead to ask for, hours |
|
||||
| `price_timeout_s` | Per-request timeout |
|
||||
| `static_profile` | A static day/night or fully-fixed tariff. Empty by default — see below |
|
||||
| `price_formula` | For `custom_formula` only: your own contract's two coefficients, off your own tariff card. Empty by default — see below |
|
||||
|
||||
**Which plugin sources what.** This is the one thing to get right, because
|
||||
adding your markup to a price that already contains it makes every future
|
||||
@@ -1386,6 +1387,7 @@ arbitrage decision wrong by that amount, in a direction nothing reports.
|
||||
| plugin | product | the published figure contains | the add-on adds |
|
||||
|---|---|---|---|
|
||||
| `generic_dayahead` | day-ahead | energy only (wholesale EPEX BE) | markup **and** taxes |
|
||||
| `custom_formula` | day-ahead | energy **and** your contract's markup — your card's `Tariefformule` line | taxes |
|
||||
| `manual_csv` | day-ahead | everything, all-in | nothing |
|
||||
|
||||
⚠️ **Three plugins were withdrawn in TARIFF-07, and if your configuration names
|
||||
@@ -1425,7 +1427,9 @@ Belgian dynamic contract is a **formula**:
|
||||
consumption: price = B x spot + A <- B and A are in YOUR contract
|
||||
```
|
||||
|
||||
The add-on cannot express that yet. Two gaps, and the second is the harder one:
|
||||
`generic_dayahead` cannot express that. Two gaps, and the second is the
|
||||
harder one — **both are closed by `custom_formula`**, which is what the
|
||||
rest of this section is about:
|
||||
|
||||
* **No multiplier.** `B` is typically slightly above 1 (it scales the wholesale
|
||||
price before anything is added). A flat markup cannot scale anything, so the
|
||||
@@ -1443,13 +1447,20 @@ deciding **which hours** are cheap that is usually good enough; for deciding
|
||||
compares against — it is an approximation, and one that errs by however far `B`
|
||||
is from 1.
|
||||
|
||||
A **formula provider** (`B x spot + A`, with your own coefficients entered once)
|
||||
is the planned answer and is a separate card. It is not implemented here. Until
|
||||
it ships, the two supported honest options are:
|
||||
**`custom_formula` is the answer to exactly this, and it shipped in TARIFF-15.**
|
||||
You type your contract's two coefficients once, off your own tariff card, and
|
||||
the add-on evaluates `B x index + A` against the same EPEX BE day-ahead curve
|
||||
`generic_dayahead` reads. Both gaps above close: `B` is a real multiplier, and
|
||||
`A` may be negative. It is documented in full in **[Your own tariff formula
|
||||
(`custom_formula`)](#your-own-tariff-formula-custom_formula)** below.
|
||||
|
||||
1. **`generic_dayahead`** with your markup and taxes set as well as you can —
|
||||
So there are three honest options rather than two:
|
||||
|
||||
1. **`custom_formula`** — your contract's own arithmetic, if you can read the
|
||||
two coefficients off your tariff card. This is the one to use.
|
||||
2. **`generic_dayahead`** with your markup and taxes set as well as you can —
|
||||
accepting the approximation above; or
|
||||
2. **`manual_csv`**, where you compute the all-in prices yourself from your own
|
||||
3. **`manual_csv`**, where you compute the all-in prices yourself from your own
|
||||
contract's formula and hand the add-on the answers. Nothing is approximated,
|
||||
but nothing is automatic either.
|
||||
|
||||
@@ -1472,7 +1483,7 @@ the `sources` declaration is the part a reviewer has to read.
|
||||
**Real-time / imbalance products still have a place in the machinery, but no
|
||||
plugin declares one.** A plugin that declares `PriceProduct.REALTIME` is flagged
|
||||
high-volatility and gets widened arbitrage thresholds; that comes from the
|
||||
plugin's own declaration, never from its name. Both shipped plugins are
|
||||
plugin's own declaration, never from its name. All three shipped plugins are
|
||||
day-ahead, so nothing exercises that path on a real install today — it is kept
|
||||
on purpose so the first genuine imbalance product does not have to rebuild it.
|
||||
See `DECISIONS/TARIFF-07-withdrawing-three-plugins.md`.
|
||||
@@ -1529,7 +1540,165 @@ and the moment it retires; the plan's own age is **not** reset by the failed
|
||||
refresh, so a kept plan still expires on schedule and is never presented as a
|
||||
current one. One failed fetch is a blip, not a lost tariff.
|
||||
|
||||
**`manual_csv` — the offline fallback.** It needs no internet, no supplier
|
||||
#### Your own tariff formula (`custom_formula`)
|
||||
|
||||
**What it is for.** A Belgian dynamic contract prices energy as a formula —
|
||||
your supplier's tariff card prints it as `Tariefformule` — and the two
|
||||
coefficients in it belong to *your contract*. There is no machine-readable
|
||||
source for them anywhere: CREG's consumer tariff-card pages return 404, CREG
|
||||
Scan is a JavaScript wizard, the annual report lists suppliers without
|
||||
per-product coefficients, and VREG was unreachable. Every coefficient found in
|
||||
research came off a supplier's own PDF, one of them image-only. **You are the
|
||||
only party who can see your own coefficients**, so you type them, once.
|
||||
|
||||
Set `price_provider: custom_formula` and fill in `price_formula` with one line
|
||||
of JSON:
|
||||
|
||||
```json
|
||||
{"index": "epex_be", "resolution": "hourly",
|
||||
"factor": 0.102, "offset": 1.00,
|
||||
"units": "c_eur_per_kwh", "vat_pct": 6,
|
||||
"card": {"month": "2026-09", "index_eur_per_mwh": 133.9905,
|
||||
"expect_incl_vat": 15.55}}
|
||||
```
|
||||
|
||||
The contract it expresses is:
|
||||
|
||||
```
|
||||
price = (factor x index_eur_per_mwh + offset) x (1 + vat_pct / 100)
|
||||
```
|
||||
|
||||
expressed in `units`, and **converted to EUR/kWh by the add-on**. You type what
|
||||
your card prints, in the card's own units; every unit conversion and the VAT
|
||||
multiplication are the add-on's job, because those are exactly the two steps
|
||||
where a human silently loses a factor of 10, 100 or 1.06.
|
||||
|
||||
| field | what it is |
|
||||
|---|---|
|
||||
| `index` | the reference series. `epex_be` — the EPEX BE day-ahead curve in EUR/MWh, the same feed `generic_dayahead` reads |
|
||||
| `resolution` | `quarter_hourly` (the 96 points the feed publishes) or `hourly` (each hour's quarters averaged into one point). Your card says which series your contract is written against |
|
||||
| `factor` | `B` — the multiplier on the index. Must be a number, and positive |
|
||||
| `offset` | `A` — the fixed part, in the same units. **May be negative** |
|
||||
| `units` | `c_eur_per_kwh`, `eur_per_kwh` or `eur_per_mwh`. **Required, with no default** |
|
||||
| `vat_pct` | the VAT percentage. `6` for Belgian residential electricity — not `0.06` and not `106`. `0` is legitimate |
|
||||
| `card` | the month the card is dated, the reference index it assumed, and the price it prints. **Required** — see below |
|
||||
|
||||
##### A worked example, from a real card
|
||||
|
||||
Eneco's September 2026 card prints the formula, the reference index it assumed
|
||||
*and* the resulting price. With `factor: 0.102`, `offset: 1.00`, `vat_pct: 6`
|
||||
and the card's own index of `133.9905` EUR/MWh:
|
||||
|
||||
```
|
||||
(0.102 x 133.9905 + 1.00) x 1.06 = 15.5471 c€/kWh the card prints 15,55
|
||||
```
|
||||
|
||||
ENGIE's card, in the same shape with its own numbers:
|
||||
|
||||
```
|
||||
(0.100 x 61.96 + 0.90) x 1.06 = 7.5218 c€/kWh the card prints 7,522
|
||||
```
|
||||
|
||||
Both reproduce to the last digit the card prints, through the shipped code —
|
||||
`test_price.py` section 15a runs exactly these two.
|
||||
|
||||
##### ⚠️ The `card` block is required, and it is the safety mechanism
|
||||
|
||||
Because your card prints the index it assumed **and** the price that comes out,
|
||||
the add-on can check your coefficients against your supplier's own worked
|
||||
example. At startup it recomputes that printed price from the numbers you
|
||||
typed and **refuses to start** unless the two agree to within `0.01` — naming
|
||||
the discrepancy, the ratio, and the likely causes.
|
||||
|
||||
That catches, in one check: VAT left out, VAT applied twice, a transposed
|
||||
digit, and a **unit mismatch** between your coefficients and the price you
|
||||
typed beside them (a factor of 10, 100 or 1000).
|
||||
|
||||
And it catches the expensive one. **Your card's formula line is the energy
|
||||
component only.** Eneco tabulates network tariffs in one table and levies in a
|
||||
second; ENGIE states that the formula excludes network, distribution,
|
||||
transport, grid levies and excise. The add-on therefore declares that
|
||||
`custom_formula` supplies *energy and markup*, and adds `price_taxes_eur_kwh`
|
||||
on top — once. If you fold your network charges into `offset`, you are billed
|
||||
for them twice, and nothing on the status page would ever show it. It cannot
|
||||
happen: ~12 c€/kWh of network charge moves the computed 15.55 to about 27, and
|
||||
the add-on refuses to start and says so.
|
||||
|
||||
⚠️ **And the limit of that check, in the same breath as the claim: it grades
|
||||
your coefficients against your card, not your card against reality.** Two
|
||||
numbers copied off the wrong product — or off a card your contract has since
|
||||
rolled over from — agree with each other perfectly and pass. Nothing can catch
|
||||
that. What is mechanised is that your coefficients and your card's printed
|
||||
price *agree*; whether the card is the right card is yours to know. The add-on
|
||||
warns (and does not refuse) when `card.month` is more than about six months
|
||||
old, because ENGIE freezes coefficients for a contract's duration and an old
|
||||
card is often exactly right.
|
||||
|
||||
One more check runs beside it: the card's own point must land inside the range
|
||||
a Belgian retail tariff can occupy (0.02–1.00 EUR/kWh). That exists because a
|
||||
unit error applied to **both** sides at once cancels out of the comparison
|
||||
above and cannot cancel out of this one. It bounds the card's own point, **not
|
||||
every hour** — a live hour may legitimately fall outside it, and a negative
|
||||
day-ahead hour, which is the best charging opportunity of the year, is never
|
||||
refused.
|
||||
|
||||
##### ⚠️ Fixed coefficients. No expression is evaluated, ever
|
||||
|
||||
`price_formula` takes numbers. There is no field that accepts an expression,
|
||||
and an unrecognised field is **refused** rather than ignored, so typing
|
||||
`"expr": "..."` tells you it does not exist instead of quietly pricing you on
|
||||
the other fields.
|
||||
|
||||
Two reasons, and the second is the decisive one:
|
||||
|
||||
* This string is settable over HTTP — the add-on's own ingress configuration
|
||||
page posts it to the Supervisor — so evaluating it would be remote code
|
||||
execution inside a root container that holds a Supervisor token and drives
|
||||
an inverter. (`eval` with an emptied `__builtins__` is not a sandbox.)
|
||||
* **An expression could not be graded against anything.** Two coefficients can
|
||||
be checked against your card's printed price; an arbitrary expression
|
||||
cannot, so it would make the invisible-wrong-coefficient problem strictly
|
||||
worse — which is the exact failure this provider exists to prevent.
|
||||
|
||||
All 24 surveyed Belgian products, both injection legs and aWATTar AT fit
|
||||
`B x index + A`, so an expression would buy nothing the market needs. If a
|
||||
future contract needs a different shape, it gets a **named field**
|
||||
(`clamp_lo`, `factor_negative`, `offset_bands`) that can be refused on its
|
||||
own. Never a grammar.
|
||||
|
||||
##### What you see at startup
|
||||
|
||||
The first time the feed answers, the add-on logs a preview of the next 24
|
||||
hours as `index EUR/MWh -> all-in EUR/kWh`, followed by the minimum, median,
|
||||
maximum and the day's spread. It is there so that a formula off by a factor of
|
||||
1000 is visible in one line rather than in a month of bills:
|
||||
|
||||
```
|
||||
custom_formula preview - the next 24 h of YOUR tariff, index EUR/MWh -> all-in
|
||||
EUR/kWh, computed as (0.102 x index + 1) x (1 + 6/100) c_eur_per_kwh, converted
|
||||
to EUR/kWh and with price_taxes_eur_kwh added by the framework:
|
||||
2026-09-10T00:00:00+02:00 216.080 -> 0.30423
|
||||
2026-09-10T00:15:00+02:00 216.080 -> 0.30423
|
||||
... 72 further point(s) not listed; the figures below cover ALL 96
|
||||
min 0.17277 / median 0.28031 / max 0.67499 EUR/kWh, spread 0.50222 EUR/kWh
|
||||
over 96 point(s). ⚠️ If those are a thousand times what your bill says,
|
||||
`units` is wrong; if the spread is 0, `factor` is.
|
||||
```
|
||||
|
||||
(That transcript is the shipped code run against the committed
|
||||
`scenarios/energy-charts-be-price-2026-09-10-quarterhourly.json` capture with
|
||||
the default markup and taxes, not an illustration.)
|
||||
|
||||
##### What it does not do
|
||||
|
||||
The **injection** leg is a separate card. Eneco uses `0,102` on consumption and
|
||||
`0,100` on injection, and injection is VAT-exempt where consumption is at 6 %,
|
||||
so it is a second per-contract formula rather than a second copy of this one.
|
||||
`export_price_eur_kwh` is still the scalar you enter by hand.
|
||||
|
||||
#### `manual_csv` — the offline fallback
|
||||
|
||||
It needs no internet, no supplier
|
||||
account and no HTTP stack at all, and it is the supported way to run this
|
||||
add-on price-aware on a site with no outbound connectivity. Write a file and
|
||||
point `price_csv_path` at it:
|
||||
|
||||
@@ -780,6 +780,89 @@ static routing pin catches. The *bound* variant — right actuator, wrong
|
||||
numbers — is a different defect with a different fix, and it was absent. Both
|
||||
are in the table now.
|
||||
|
||||
## TARIFF-15 — the card-graded formula, mutation by mutation
|
||||
|
||||
Twenty-two mutations, one per check the card added, **run** rather than
|
||||
reasoned about. Harness: `/home/homelab/.cache/t15-mut/run.py`, backups beside
|
||||
it rather than in a shared scratchpad — two agents' harnesses collided in one
|
||||
this week. Each run restores from backup, purges `__pycache__` **on the host**
|
||||
first, and grades through the CI image with the tree bind-mounted `:ro`, so the
|
||||
interpreter cannot write a cache into the tree under test at all (TEST-15's
|
||||
structural immunity, borrowed).
|
||||
|
||||
**No survivors.** Two entries are `+0 B` and are called out as such, per the
|
||||
reporting rule.
|
||||
|
||||
| | mutation | file | Δ | red |
|
||||
|---|---|---|---|---|
|
||||
| M1 | `CARD_TOLERANCE` widened to 100 — the self-check stops checking | `custom_formula.py` | +1 B | **8** |
|
||||
| M2 | `CARD_TOLERANCE` tightened to 0 — the other direction, so the tolerance is a number and not a shape | `custom_formula.py` | −1 B | **16** |
|
||||
| M3 | `units` becomes optional | `custom_formula.py` | +10 B | 1 |
|
||||
| M4 | the `factor == 0` rung removed | `custom_formula.py` | −8 B | 1 |
|
||||
| M5 | the `factor < 0` rung removed | `custom_formula.py` | −7 B | 1 |
|
||||
| M6 | the `vat_pct` band widened to ±1000 | `custom_formula.py` | +6 B | 2 |
|
||||
| M7 | the non-finite rung removed from `_finite()` | `custom_formula.py` | −26 B | 2 |
|
||||
| M8 | the non-numeric rung removed from `_finite()` | `custom_formula.py` | −16 B | 4 |
|
||||
| M9 | an unknown top-level field ignored instead of refused | `custom_formula.py` | −2 B | 1 |
|
||||
| M10 | the plausibility **floor** applied to the all-in figure instead of the energy one | `custom_formula.py` | **+0 B** | 2 |
|
||||
| M11 | `hourly` stops averaging and emits the feed's own points | `custom_formula.py` | +12 B | 3 |
|
||||
| M12 | the hourly bucket takes the FIRST quarter instead of the mean | `custom_formula.py` | −11 B | 1 |
|
||||
| M13 | the preview drops the index column from each row | `custom_formula.py` | −13 B | 1 |
|
||||
| M14 | the preview is emitted on every refresh rather than once | `price.py` | −8 B | 1 |
|
||||
| M15 | `custom_formula` declares `ALL_IN` — the double-counting lie | `price.py` | −41 B | **10** |
|
||||
| M16 | a stale card never warns | `custom_formula.py` | −8 B | 1 |
|
||||
| M17 | a stale card REFUSES instead of warning | `validate.py` | −3 B | 1 |
|
||||
| M18 | the empty-formula refusal removed from the startup gate | `validate.py` | −38 B | 1 |
|
||||
| M19 | the index scaled to EUR/kWh before the formula — the factor-1000 | `price.py` | +2 B | **7** |
|
||||
| M20 | an `eval` call planted in `app/custom_formula.py` | `custom_formula.py` | +14 B | 1 |
|
||||
| M21 | `custom_formula` stops passing the caller's range to the feed | `price.py` | **+0 B** | 1 |
|
||||
| M22 | **PAIRED**: M1 *and* the plausibility band removed, together | `custom_formula.py` | −60 B | **12** |
|
||||
|
||||
### The four that are worth reading rather than counting
|
||||
|
||||
**M13 was a SURVIVOR of the whole suite on the first attempt, at −13 B.** The
|
||||
check read `"index EUR/MWh -> all-in EUR/kWh" in log` — which is the preview's
|
||||
**heading**. Deleting the index column from every **row** left the heading
|
||||
saying it prints something it no longer printed, and nothing went red. That is
|
||||
this file's third failure mode arriving in the check written to close it: the
|
||||
assertion ran, the sentence was true, and the data could not reach the failure
|
||||
it named. It is graded on a rendered ROW now, built here from the captured
|
||||
document (`' 216.080 -> 0.30123'`), and M13 is red.
|
||||
|
||||
**M10 is `+0 B` and it is a real defect, not a synthetic one.** It came out of
|
||||
writing the mutation rather than out of review: `implausible_all_in()`'s floor
|
||||
was applied to the all-in figure, and `price_taxes_eur_kwh` ships at 0.06 —
|
||||
**three times the 0.02 floor** — so a card read a hundred times too small
|
||||
(0.0016 EUR/kWh of energy) is carried over the floor by the flat tax addend
|
||||
alone and passes. The hundredfold error is exactly what that band exists for.
|
||||
The floor is applied to the **energy** figure now and the ceiling to the all-in
|
||||
one, both directions are graded, and the trap is stated as a check in its own
|
||||
right so the asymmetry cannot be "tidied up" later.
|
||||
|
||||
**M15 is the one the framework CANNOT catch, and the reds say which half does.**
|
||||
Declaring `ALL_IN` makes `assemble()` add nothing, and section 3's all-in check
|
||||
is *derived from the declaration* — so it passes on the lie, exactly as
|
||||
`price.py`'s trap 1 says no code can catch a misdeclaration. What goes red is
|
||||
the **docstring/declaration agreement** check, the card reproductions, and the
|
||||
`sources` pin: ten reds, none of them from the arithmetic. That is the
|
||||
guarantee at its true size — the two claims are held to agree, and whether
|
||||
either is true about somebody else's tariff remains human.
|
||||
|
||||
**M22 is the paired run (TEST-16).** The card self-check and the plausibility
|
||||
band both answer a unit error, so each could be masking the other. Weakened
|
||||
**together** they produce twelve reds rather than the eight M1 alone produces,
|
||||
and the band's own checks stay red — so neither rung is doing the other's work.
|
||||
|
||||
### What this battery does NOT establish
|
||||
|
||||
It does not establish that a card is the right card. Every mutation above moves
|
||||
the relationship between the coefficients and the printed price; **none of them
|
||||
can move the relationship between the printed price and the operator's actual
|
||||
contract**, because nothing in this repository observes that. A
|
||||
consistent-but-wrong pair passes every one of these checks, by construction,
|
||||
and `DECISIONS/TARIFF-15-the-card-is-the-grader.md` §3 says so in the same
|
||||
sentence as the claim.
|
||||
|
||||
## Where each surface stands
|
||||
|
||||
* **Local and dev-harness runs — the exposed one.** This is where FCST-03 hit
|
||||
|
||||
@@ -24,7 +24,7 @@ since the status page shipped; this file has five more chances to get it wrong.
|
||||
|
||||
⚠️ AND IT IS BUILT FOR A PHONE FIRST. This is the screen someone stands in
|
||||
front of the inverter with: one column, big touch targets, sections that start
|
||||
collapsed so 71 options do not arrive as one wall, and the refusal block pinned
|
||||
collapsed so 72 options do not arrive as one wall, and the refusal block pinned
|
||||
where it cannot be scrolled past.
|
||||
"""
|
||||
|
||||
|
||||
@@ -0,0 +1,670 @@
|
||||
"""TARIFF-15 - the customer's own tariff card, as two fixed coefficients.
|
||||
|
||||
WHAT THIS IS FOR, in one sentence: a Belgian dynamic contract prices energy as
|
||||
`B x index + A` with coefficients that exist in exactly one place - the tariff
|
||||
card in the customer's klantenzone - so the customer types those two numbers
|
||||
and the add-on does every unit conversion and the VAT multiplication itself.
|
||||
|
||||
WHY A FORMULA AND NOT FOURTEEN MORE PLUGINS (§9.2, TARIFF-07)
|
||||
=============================================================
|
||||
There is NO machine-readable source of Belgian tariff coefficients anywhere.
|
||||
CREG's consumer tariff-card pages 404, CREG Scan is a JS-only wizard, the
|
||||
annual report carries the supplier list and no per-product coefficients, and
|
||||
VREG was unreachable. Every coefficient found in research came off a supplier's
|
||||
own PDF, one of them image-only. So there is nothing to scrape and a
|
||||
per-supplier plugin would have no source for its numbers - fourteen plugins
|
||||
would be fourteen copies of the same invention, which is the defect TARIFF-07
|
||||
withdrew three plugins for. THE CUSTOMER IS THE ONLY PARTY WHO CAN SEE THEIR
|
||||
OWN COEFFICIENTS, and the card they read them from is the only artefact that
|
||||
can grade them.
|
||||
|
||||
THE SHAPE IS THE MARKET'S PUBLISHED ARITHMETIC, NOT OUR APPROXIMATION
|
||||
====================================================================
|
||||
Both suppliers whose 2026 cards were readable print the formula, the reference
|
||||
index they assumed AND the resulting price, so the arithmetic below is
|
||||
reproducible against something they published rather than against our own
|
||||
guess:
|
||||
|
||||
Eneco (0.102 x 133.9905 + 1.00) x 1.06 = 15.5471 card prints 15,55
|
||||
ENGIE (0.100 x 61.96 + 0.90) x 1.06 = 7.5218 card prints 7,522
|
||||
|
||||
Two independent suppliers, reproduced to the last printed digit, in
|
||||
c EUR/kWh. test_price.py section 13 runs both through the SHIPPED code.
|
||||
|
||||
⚠️ THE DOUBLE-COUNTING MECHANISM, AND WHY THIS DESIGN IS SAFE AT ALL
|
||||
====================================================================
|
||||
This provider declares `sources = {ENERGY, MARKUP}` and NOT `ALL_IN`. A Belgian
|
||||
card's `Tariefformule` line is THE ENERGY COMPONENT ONLY: Eneco puts network
|
||||
tariffs in one table and levies in a second, and ENGIE states in so many words
|
||||
that the formula "does not include fixed network/distribution costs, transport,
|
||||
grid levies, or excise duties". Declaring ALL_IN would lie on behalf of every
|
||||
user who typed what their card says, and would silently drop ~12 c EUR/kWh.
|
||||
|
||||
price.py already mechanises that a PLUGIN cannot double-count (`PluginOptions`,
|
||||
`FRAMEWORK_KEYS`, `code_leaks()`). This provider introduces a risk none of those
|
||||
can see, because it is not in the code: THE OPERATOR double-counting, by folding
|
||||
their network charges into `offset` while `price_taxes_eur_kwh` carries them as
|
||||
well. `code_leaks()` scans code. This is config.
|
||||
|
||||
So the card is the grader:
|
||||
|
||||
At startup validate.py computes
|
||||
`(factor x card.index_eur_per_mwh + offset) x (1 + vat_pct/100)` and
|
||||
REFUSES TO START unless it equals `card.expect_incl_vat` to within
|
||||
CARD_TOLERANCE, naming the discrepancy.
|
||||
|
||||
An `offset` inflated by network charges CANNOT PASS: the card's printed estimate
|
||||
is by construction the energy component alone, so adding ~12 c EUR/kWh moves
|
||||
15.55 to ~27 and the add-on refuses. The same check catches VAT omitted, VAT
|
||||
applied twice, a transposed digit, and a UNIT MISMATCH BETWEEN THE TWO SIDES -
|
||||
coefficients read off the formula line while the expected price is typed in
|
||||
another convention, which is the factor-10/100/1000 class as it actually
|
||||
happens.
|
||||
|
||||
⚠️ THE ONE UNIT ERROR THE CARD CHECK CANNOT SEE IS THE ONE APPLIED TO BOTH
|
||||
SIDES AT ONCE, and saying so is the difference between a guard and a slogan:
|
||||
scaling the coefficients AND the expected price by the same 100 cancels out of
|
||||
a comparison between them. That is what `implausible_all_in()` is for, and it
|
||||
is a SEPARATE refusal on a SEPARATE quantity - the card's own point, all-in,
|
||||
against the band a Belgian retail tariff can occupy.
|
||||
|
||||
⚠️ AND THE LIMIT, IN THE SAME BREATH AS THE CLAIM, exactly as price.py's trap 1
|
||||
states its own: THE CHECK GRADES THE COEFFICIENTS AGAINST THE CARD, NOT THE CARD
|
||||
AGAINST REALITY. A consistent-but-wrong pair - both numbers copied off the wrong
|
||||
product, or off last year's card - reproduces its own printed price perfectly
|
||||
and passes. Nothing here or anywhere else can catch that; what is mechanised is
|
||||
that the two coefficients and the printed estimate AGREE, which is the whole of
|
||||
the unit/VAT/decimal-point class and none of the "wrong contract" class.
|
||||
|
||||
⚠️ FIXED COEFFICIENTS. NO EXPRESSION EVALUATION OF ANY KIND.
|
||||
============================================================
|
||||
All 24 surveyed Belgian products, both injection legs and aWATTar AT fit
|
||||
`B x index + A`, so an expression buys nothing the market needs. Against that:
|
||||
|
||||
* the add-on serves `POST config/save` -> `POST /addons/self/options` on its
|
||||
own ingress page, so this string is SETTABLE OVER HTTP. `eval` on it is
|
||||
remote code execution in a root container holding SUPERVISOR_TOKEN, with
|
||||
`homeassistant_api` and `hassio_api`, driving an inverter over RS485.
|
||||
(`eval(s, {"__builtins__": {}})` is not a sandbox and never was.)
|
||||
* ⚠️ AND THE DECIDING ARGUMENT IS NOT SECURITY - AN EXPRESSION IS
|
||||
UNVERIFIABLE. Two coefficients can be graded against the card's printed
|
||||
estimate. An arbitrary expression cannot be graded against anything, so it
|
||||
makes the invisible-wrong-coefficient problem STRICTLY WORSE, which is the
|
||||
exact failure this provider exists to prevent.
|
||||
|
||||
Future shapes become NAMED FIELDS - `clamp_lo`, `factor_negative`,
|
||||
`offset_bands` - each individually refusable and each individually gradeable
|
||||
against a card. Never a grammar. `parse_price_formula()` therefore REFUSES AN
|
||||
UNKNOWN KEY rather than ignoring it, so a user who types `"expr"` is told it
|
||||
does not exist instead of being silently priced on the fields they also typed.
|
||||
test_price.py section 13f walks this module's AST and fails if `eval`, `exec`,
|
||||
`compile`, `__import__` or `literal_eval` ever appears in it.
|
||||
|
||||
WHERE THE NUMBERS COME FROM AND WHERE THEY GO
|
||||
=============================================
|
||||
`index` names the reference series. `epex_be` is the EPEX BE day-ahead curve in
|
||||
EUR/MWh, which is what `generic_dayahead` already fetches - same feed, same
|
||||
`brussels_date_range()` request, same columnar parser, same declared egress
|
||||
hosts. No new parser and no new failure mode.
|
||||
|
||||
`resolution` says which series the contract is written against:
|
||||
|
||||
* `quarter_hourly` - the 96 points the feed publishes since Belgium's
|
||||
15-minute MTU (2025-10-01), emitted as they are;
|
||||
* `hourly` - the four quarters of each hour averaged into one point.
|
||||
|
||||
⚠️ NEITHER IS AN HOURLY ASSUMPTION IN `_Series.at()`'s SENSE (TARIFF-14). The
|
||||
hourly option changes how many POINTS are emitted and their spacing; the window
|
||||
each point covers is still derived from its neighbours by `_cover_table()`, and
|
||||
nothing here tells anything downstream how long a point lasts. A feed that is
|
||||
already hourly averages one member per hour, which is the identity - so the
|
||||
option cannot silently assume the granularity it was handed.
|
||||
|
||||
`units` is NOT bureaucracy. Two suppliers publish the same shape in different
|
||||
units and the market aggregator prints both conventions in one column, so the
|
||||
add-on has to be told which one the card is in - and the unit conversion is the
|
||||
add-on's job precisely because it is one of the two steps that silently costs a
|
||||
factor of 10, 100 or 1000. The other is VAT, which is also the add-on's.
|
||||
"""
|
||||
|
||||
import json
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
from datetime import date, datetime, timezone
|
||||
|
||||
|
||||
class FormulaInvalid(Exception):
|
||||
"""`price_formula` cannot be used. Raised by the parser, refused by
|
||||
validate.py at startup - never at 03:00 on a fetch."""
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# The enumerations. Every one of these is a CLOSED set, and that is the point:
|
||||
# an unknown value is refused by name rather than guessed at.
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
# unit -> how many EUR/kWh one of it is. ⚠️ THE WHOLE FACTOR-1000 CLASS LIVES
|
||||
# HERE, so the table is tiny, explicit, and has no "default".
|
||||
UNITS: dict = {
|
||||
"c_eur_per_kwh": 0.01, # eurocent per kWh - Eneco and ENGIE print this
|
||||
"eur_per_kwh": 1.0, # some aggregators print this
|
||||
"eur_per_mwh": 0.001, # the wholesale convention, occasionally reused
|
||||
}
|
||||
|
||||
RESOLUTIONS: tuple = ("hourly", "quarter_hourly")
|
||||
|
||||
# index id -> the human name of the series it refers to. One entry today; a
|
||||
# second index is a new row here plus a fetch path, not a new grammar.
|
||||
INDEXES: dict = {
|
||||
"epex_be": "EPEX BE day-ahead, EUR/MWh",
|
||||
}
|
||||
|
||||
# ⚠️ 0.01 IN THE CARD'S OWN UNITS, WHICH IS THE LAST DIGIT BOTH REAL CARDS
|
||||
# PRINT. Eneco prints 15,55 against a computed 15.5471 (0.0029 out) and ENGIE
|
||||
# prints 7,522 against a computed 7.5218 (0.0002 out), so the tolerance has to
|
||||
# absorb the card's own rounding and nothing more. Every error class this check
|
||||
# exists for - a factor of 10, of 100, of 1000, a missing or doubled 1.06, a
|
||||
# transposed digit, ~12 c EUR/kWh of network charge folded into `offset` - is
|
||||
# at least two orders of magnitude bigger than this.
|
||||
CARD_TOLERANCE = 0.01
|
||||
|
||||
# The all-in plausibility band, EUR/kWh. `MAX_PLAUSIBLE_EUR_KWH` in price.py
|
||||
# bounds only prices that are too LARGE; this is the other end, and it is the
|
||||
# one the factor-1000 error falls off.
|
||||
PLAUSIBLE_LO_EUR_KWH = 0.02
|
||||
PLAUSIBLE_HI_EUR_KWH = 1.00
|
||||
|
||||
# ~6 months. ⚠️ A WARNING, NEVER A REFUSAL: ENGIE freezes a contract's
|
||||
# coefficients for its duration, so a card two years old can be exactly right
|
||||
# and a site refused over one would be a site that cannot start.
|
||||
CARD_STALE_DAYS = 183
|
||||
|
||||
_CARD_KEYS = ("month", "index_eur_per_mwh", "expect_incl_vat")
|
||||
_TOP_KEYS = ("index", "resolution", "factor", "offset", "units", "vat_pct",
|
||||
"card")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class FormulaSpec:
|
||||
"""One contract's published arithmetic. SEVEN numbers and no code.
|
||||
|
||||
⚠️ EVERY FIELD IS A FIXED, NAMED SCALAR. There is deliberately no field
|
||||
that holds an expression, a callable, a lambda or a template, and there is
|
||||
no place to put one: `parse_price_formula()` refuses an unrecognised key.
|
||||
"""
|
||||
|
||||
index: str
|
||||
resolution: str
|
||||
factor: float
|
||||
offset: float
|
||||
units: str
|
||||
vat_pct: float
|
||||
card_month: str # "YYYY-MM", as the card is dated
|
||||
card_index_eur_per_mwh: float
|
||||
card_expect_incl_vat: float # in `units`, as the card prints it
|
||||
|
||||
@property
|
||||
def unit_to_eur_kwh(self) -> float:
|
||||
return UNITS[self.units]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# The arithmetic. Two functions, and the SECOND one is the contract.
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
def in_card_units(spec: FormulaSpec, index_eur_per_mwh: float) -> float:
|
||||
"""`(factor x index + offset) x (1 + vat_pct/100)`, in the card's units.
|
||||
|
||||
This is the line the card prints, spelled exactly as the card spells it,
|
||||
and it is the only place the coefficients are combined. The card
|
||||
self-check and the price path both go through here, so a mutation to it
|
||||
cannot move the price without also moving the self-check.
|
||||
"""
|
||||
return ((spec.factor * float(index_eur_per_mwh) + spec.offset)
|
||||
* (1.0 + spec.vat_pct / 100.0))
|
||||
|
||||
|
||||
def price_eur_kwh(spec: FormulaSpec, index_eur_per_mwh: float) -> float:
|
||||
"""The energy+markup price for one index value, in EUR/kWh.
|
||||
|
||||
⚠️ THE UNIT CONVERSION IS THE ADD-ON'S JOB AND IT HAPPENS HERE, ONCE. The
|
||||
user types what their card prints, in the card's own units; every step
|
||||
from there to EUR/kWh is code, because those steps are exactly the ones
|
||||
that silently cost a factor of 10, 100 or 1000 when a human does them.
|
||||
"""
|
||||
return in_card_units(spec, index_eur_per_mwh) * spec.unit_to_eur_kwh
|
||||
|
||||
|
||||
def card_discrepancy(spec: FormulaSpec) -> float:
|
||||
"""How far the coefficients land from the card's own printed estimate.
|
||||
|
||||
In the card's units, signed: positive means the formula computes MORE than
|
||||
the card prints, which is the direction an `offset` carrying network
|
||||
charges moves it.
|
||||
"""
|
||||
return in_card_units(spec, spec.card_index_eur_per_mwh) \
|
||||
- spec.card_expect_incl_vat
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Reading the option
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
def _finite(value, field: str) -> float:
|
||||
"""A real, finite number - or a refusal naming the field.
|
||||
|
||||
⚠️ SAME CLASS AS `_validate_raw()`'s NaN RULE, one layer earlier. A NaN
|
||||
factor makes every price NaN, every comparison False, and an arbitrage
|
||||
strategy that silently never fires; `True` is an `int` in Python and would
|
||||
price the whole day at the index.
|
||||
"""
|
||||
if isinstance(value, bool) or not isinstance(value, (int, float)):
|
||||
raise FormulaInvalid(
|
||||
f"price_formula.{field} is {value!r}, which is not a number. "
|
||||
f"Write the coefficient as a bare JSON number - 0.102, not "
|
||||
f"\"0.102\" and not an expression: this provider takes FIXED "
|
||||
f"coefficients and evaluates nothing.")
|
||||
if not math.isfinite(float(value)):
|
||||
raise FormulaInvalid(
|
||||
f"price_formula.{field} is {value!r}, which is not finite. A NaN "
|
||||
f"or an infinity here makes every price NaN, every price "
|
||||
f"comparison False, and an arbitrage strategy that never fires "
|
||||
f"while reporting that it is armed.")
|
||||
return float(value)
|
||||
|
||||
|
||||
def _card(raw) -> tuple:
|
||||
"""The `card` block: the estimate the coefficients are graded against."""
|
||||
if raw is None:
|
||||
raise FormulaInvalid(
|
||||
"price_formula has no \"card\" block, and it is REQUIRED. It is "
|
||||
"the only thing that can tell a correct pair of coefficients from "
|
||||
"a plausible-looking wrong one: the add-on recomputes your "
|
||||
"supplier's own printed example from the numbers you typed and "
|
||||
"refuses to start if it does not come out. Copy the three figures "
|
||||
"off the same tariff card as the coefficients: "
|
||||
"{\"month\": \"2026-09\", \"index_eur_per_mwh\": 133.9905, "
|
||||
"\"expect_incl_vat\": 15.55}.")
|
||||
if not isinstance(raw, dict):
|
||||
raise FormulaInvalid(
|
||||
f"price_formula.card is {type(raw).__name__}, not an object with "
|
||||
f"{', '.join(_CARD_KEYS)}.")
|
||||
unknown = sorted(set(raw) - set(_CARD_KEYS))
|
||||
if unknown:
|
||||
raise FormulaInvalid(
|
||||
f"price_formula.card has no field(s) {unknown}. The fields it "
|
||||
f"takes are {list(_CARD_KEYS)}; a key that is merely ignored is a "
|
||||
f"number the operator believes took effect.")
|
||||
month = str(raw.get("month", "") or "").strip()
|
||||
if not _month_date(month):
|
||||
raise FormulaInvalid(
|
||||
f"price_formula.card.month is {raw.get('month')!r}; write the "
|
||||
f"month the card is dated as \"YYYY-MM\", e.g. \"2026-09\". It is "
|
||||
f"not used in any price - it is what lets the add-on tell you "
|
||||
f"your card has gone stale.")
|
||||
return (month,
|
||||
_finite(raw.get("index_eur_per_mwh"), "card.index_eur_per_mwh"),
|
||||
_finite(raw.get("expect_incl_vat"), "card.expect_incl_vat"))
|
||||
|
||||
|
||||
def _month_date(month: str):
|
||||
"""`"2026-09"` -> the first of that month, or None if it is not one."""
|
||||
try:
|
||||
year, mon = str(month).split("-")
|
||||
return date(int(year), int(mon), 1)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def parse_price_formula(value):
|
||||
"""`price_formula` -> a `FormulaSpec`, or None when unset.
|
||||
|
||||
Accepts what the option can actually arrive as, exactly as
|
||||
`parse_static_profile()` does:
|
||||
|
||||
* `None` / `""` / `"null"` - unset.
|
||||
* a JSON string - what config.yaml's flat `str?` option
|
||||
carries on a Supervisor install.
|
||||
* a dict - already parsed, from a future nested
|
||||
schema, from sim/inject.py, or a check.
|
||||
|
||||
Raises `FormulaInvalid`; never returns a half-built spec.
|
||||
|
||||
⚠️ THE CARD SELF-CHECK IS PART OF PARSING, not a separate step a caller
|
||||
may forget. Both callers - validate.py's startup gate and the plugin's own
|
||||
fetch - go through this one function, so there is no route to a price that
|
||||
has not been graded against the card.
|
||||
"""
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, str):
|
||||
text = value.strip()
|
||||
if not text or text.lower() in ("null", "none", "off"):
|
||||
return None
|
||||
try:
|
||||
value = json.loads(text)
|
||||
except ValueError as e:
|
||||
raise FormulaInvalid(
|
||||
f"price_formula is not valid JSON ({e}). It is a JSON object "
|
||||
f"on one line, e.g.\n"
|
||||
f' {{"index":"epex_be","resolution":"hourly","factor":0.102,'
|
||||
f'"offset":1.00,"units":"c_eur_per_kwh","vat_pct":6,'
|
||||
f'"card":{{"month":"2026-09","index_eur_per_mwh":133.9905,'
|
||||
f'"expect_incl_vat":15.55}}}}\n'
|
||||
f"Leave it empty unless price_provider is 'custom_formula'.")
|
||||
if not isinstance(value, dict):
|
||||
raise FormulaInvalid(
|
||||
f"price_formula is {type(value).__name__}, not a JSON object with "
|
||||
f"{', '.join(_TOP_KEYS)}.")
|
||||
|
||||
# ⚠️ AN UNKNOWN KEY IS REFUSED, NOT IGNORED, AND THIS IS THE HALF THAT
|
||||
# KEEPS THE "NEVER A GRAMMAR" PROMISE HONEST. A future shape is a NAMED
|
||||
# field - clamp_lo, factor_negative, offset_bands - added here and
|
||||
# refusable on its own. Ignoring unknown keys would mean a user who typed
|
||||
# "expr" or "formula" gets priced on the fields they also typed, with the
|
||||
# one they believed in dropped in silence.
|
||||
unknown = sorted(set(value) - set(_TOP_KEYS))
|
||||
if unknown:
|
||||
raise FormulaInvalid(
|
||||
f"price_formula has no field(s) {unknown}. The fields it takes "
|
||||
f"are {list(_TOP_KEYS)}. ⚠️ There is deliberately NO field that "
|
||||
f"takes an expression: this provider multiplies and adds two "
|
||||
f"FIXED coefficients and evaluates nothing, because two "
|
||||
f"coefficients can be graded against your card's printed price "
|
||||
f"and an arbitrary expression cannot be graded against anything. "
|
||||
f"If your contract needs a shape this cannot express, it needs a "
|
||||
f"new named field here - say which shape.")
|
||||
|
||||
index = str(value.get("index", "") or "").strip()
|
||||
if index not in INDEXES:
|
||||
raise FormulaInvalid(
|
||||
f"price_formula.index is {value.get('index')!r}. The reference "
|
||||
f"series this add-on can fetch are "
|
||||
f"{', '.join(f'{k!r} ({v})' for k, v in sorted(INDEXES.items()))}. "
|
||||
f"If your card names a different index, the add-on cannot get it "
|
||||
f"and guessing which series your supplier meant would price your "
|
||||
f"battery against the wrong curve.")
|
||||
|
||||
resolution = str(value.get("resolution", "") or "").strip()
|
||||
if resolution not in RESOLUTIONS:
|
||||
raise FormulaInvalid(
|
||||
f"price_formula.resolution is {value.get('resolution')!r}; it "
|
||||
f"must be one of {list(RESOLUTIONS)}. Belgium has been on a "
|
||||
f"15-minute MTU since 2025-10-01 and the feed publishes 96 points "
|
||||
f"a day; a contract written against the HOURLY series is priced "
|
||||
f"on the mean of each hour's quarters. Which one your contract "
|
||||
f"uses is on your card and cannot be inferred from the feed.")
|
||||
|
||||
# ⚠️ `units` IS REQUIRED AND HAS NO DEFAULT. A default here would be the
|
||||
# factor-1000 class with a friendly face: the add-on would silently assume
|
||||
# a convention the card never stated, and the card self-check would then
|
||||
# be graded in units nobody chose.
|
||||
units = str(value.get("units", "") or "").strip()
|
||||
if units not in UNITS:
|
||||
raise FormulaInvalid(
|
||||
f"price_formula.units is {value.get('units')!r} and is REQUIRED - "
|
||||
f"there is no default, on purpose. It must be one of "
|
||||
f"{sorted(UNITS)}. Your card prints the formula in ONE of these "
|
||||
f"and two suppliers use two different conventions for the same "
|
||||
f"shape, so assuming one would be a factor of 100 or 1000 on "
|
||||
f"every hour with nothing able to notice.")
|
||||
|
||||
factor = _finite(value.get("factor"), "factor")
|
||||
offset = _finite(value.get("offset"), "offset")
|
||||
vat_pct = _finite(value.get("vat_pct"), "vat_pct")
|
||||
|
||||
# ⚠️ ZERO IS NOT "NO INDEXATION", IT IS A FIXED TARIFF WEARING A DYNAMIC
|
||||
# ONE'S CLOTHES. Every hour comes out at the same price, so every spread is
|
||||
# 0, arbitrage can never clear any threshold, and a strategy that never
|
||||
# fires is indistinguishable from one that is switched off (§14's warning
|
||||
# box). The add-on already has a first-class way to say "one price": point
|
||||
# the operator at it rather than let them build it by accident.
|
||||
if factor == 0.0:
|
||||
raise FormulaInvalid(
|
||||
"price_formula.factor is 0, so the index would not enter the "
|
||||
"price at all: every hour of every day comes out at "
|
||||
f"{offset:g} x (1 + {vat_pct:g}/100), every spread is exactly 0, "
|
||||
"and arbitrage can never clear any threshold - a strategy that "
|
||||
"silently never fires rather than one that is off. If your tariff "
|
||||
"really is one flat price, that is `static_profile` with a single "
|
||||
"period covering 00:00-24:00 on all days and price_provider "
|
||||
"'off'; the add-on then says so at startup instead of leaving you "
|
||||
"to find out from a year of unchanged bills.")
|
||||
|
||||
# ⚠️ A NEGATIVE FACTOR INVERTS THE MARKET, and the direction is the danger:
|
||||
# the dearest hour becomes the cheapest one, so the battery is told to
|
||||
# charge into the evening peak and to discharge at 03:00. That is not a
|
||||
# mispriced site, it is a site actively trading against itself.
|
||||
if factor < 0.0:
|
||||
raise FormulaInvalid(
|
||||
f"price_formula.factor is {factor:g}, which is NEGATIVE. That "
|
||||
f"inverts the market: the dearest hour of the day comes out as "
|
||||
f"the cheapest one, so the battery would charge at the evening "
|
||||
f"peak and discharge at 03:00 - trading against itself on every "
|
||||
f"cycle. No consumption tariff is indexed negatively; if your "
|
||||
f"card shows a minus sign it is on the OFFSET (`A`), which is "
|
||||
f"allowed here and common on injection.")
|
||||
|
||||
# ⚠️ 0 IS LEGITIMATE AND MUST STAY SO: injection is VAT-exempt in Belgium,
|
||||
# and some cards are already quoted VAT-inclusive. The bound exists for the
|
||||
# 106 / 1.06 / 6 confusion, not to make VAT compulsory.
|
||||
if not 0.0 <= vat_pct <= 30.0:
|
||||
raise FormulaInvalid(
|
||||
f"price_formula.vat_pct is {vat_pct:g} and must be between 0 and "
|
||||
f"30. It is a PERCENTAGE, so Belgian residential electricity is "
|
||||
f"6, not 0.06 and not 106. Zero is legitimate and is not refused: "
|
||||
f"injection is VAT-exempt, and a card that already quotes "
|
||||
f"VAT-inclusive coefficients needs no second multiplication.")
|
||||
|
||||
month, card_index, card_expect = _card(value.get("card"))
|
||||
spec = FormulaSpec(index=index, resolution=resolution, factor=factor,
|
||||
offset=offset, units=units, vat_pct=vat_pct,
|
||||
card_month=month, card_index_eur_per_mwh=card_index,
|
||||
card_expect_incl_vat=card_expect)
|
||||
|
||||
delta = card_discrepancy(spec)
|
||||
if abs(delta) > CARD_TOLERANCE:
|
||||
raise FormulaInvalid(card_refusal(spec, delta))
|
||||
return spec
|
||||
|
||||
|
||||
def card_refusal(spec: FormulaSpec, delta: float) -> str:
|
||||
"""The sentence an operator reads when their formula misses its own card.
|
||||
|
||||
⚠️ IT NAMES THE DISCREPANCY AND THE SIX WAYS IT HAPPENS. "Invalid
|
||||
configuration" would be true and useless: every one of these errors looks
|
||||
exactly like a working tariff from the status page, which is why the check
|
||||
exists at all.
|
||||
"""
|
||||
got = in_card_units(spec, spec.card_index_eur_per_mwh)
|
||||
ratio = (got / spec.card_expect_incl_vat
|
||||
if spec.card_expect_incl_vat else float("inf"))
|
||||
return (
|
||||
f"price_formula does not reproduce its own card, so the add-on will "
|
||||
f"NOT start on it. Your contract's arithmetic is "
|
||||
f"price = (factor x index_eur_per_mwh + offset) x (1 + vat_pct/100); "
|
||||
f"on the card's own index that is "
|
||||
f"({spec.factor:g} x {spec.card_index_eur_per_mwh:g} + "
|
||||
f"{spec.offset:g}) x (1 + {spec.vat_pct:g}/100) = {got:.4f} "
|
||||
f"{spec.units}, but card.expect_incl_vat says "
|
||||
f"{spec.card_expect_incl_vat:g} - OUT BY {delta:+.4f} {spec.units} "
|
||||
f"(ratio {ratio:.4g}; the tolerance is {CARD_TOLERANCE}, which is the "
|
||||
f"last digit a card prints). One of the two sides is wrong and the "
|
||||
f"add-on cannot tell which, so it refuses rather than price a battery "
|
||||
f"on it. The usual causes, in the order they happen: the coefficients "
|
||||
f"and the expected price are in DIFFERENT units (ratio 10, 100 or "
|
||||
f"1000 - `units` must be the one the formula line is printed in); VAT "
|
||||
f"left out or applied twice (ratio 1.06 or 1/1.06); a transposed "
|
||||
f"digit; the two sides copied off DIFFERENT cards or different "
|
||||
f"months; or - the expensive one - network charges, levies or excise "
|
||||
f"folded into `offset`. ⚠️ THEY DO NOT BELONG THERE: a Belgian card's "
|
||||
f"Tariefformule line is the ENERGY component alone (Eneco tabulates "
|
||||
f"network tariffs and levies separately; ENGIE says in so many words "
|
||||
f"that the formula excludes network, distribution, transport, levies "
|
||||
f"and excise), the add-on adds price_taxes_eur_kwh on top of it, and "
|
||||
f"folding ~12 c EUR/kWh of network charge into `offset` bills you for "
|
||||
f"it twice. That is the error this check exists to catch, and it is "
|
||||
f"why it cannot be switched off.")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# The two questions the PLUGIN cannot ask, because they need framework numbers
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
def implausible_all_in(spec: FormulaSpec, taxes_eur_kwh: float):
|
||||
"""The card's own point, if its LEVEL cannot be a Belgian retail price.
|
||||
|
||||
Returns a refusal sentence or None. ⚠️ CALLED FROM validate.py AND FROM
|
||||
NOWHERE ELSE, because it reads a FRAMEWORK-owned number: a plugin that
|
||||
could see `taxes` could add it a second time, which is price.py's trap 1.
|
||||
|
||||
⚠️ THE LOW END IS GRADED ON THE ENERGY COMPONENT AND THE HIGH END ON THE
|
||||
ALL-IN FIGURE, AND THAT ASYMMETRY IS THE CHECK RATHER THAN AN OVERSIGHT.
|
||||
`price_taxes_eur_kwh` ships at 0.06, which is three times the low bound -
|
||||
so a card read a HUNDRED times too small comes out at 0.0016 EUR/kWh of
|
||||
energy, is carried over the floor by the flat tax addend alone, and passes
|
||||
a band applied to the all-in total. The hundredfold error is precisely
|
||||
what this exists to catch, so the floor is applied where it can still see
|
||||
it. The ceiling is applied to the all-in figure because that is the number
|
||||
the site is really billed at, and taxes only push it up.
|
||||
|
||||
⚠️ AND THE LIMIT, WITH THE CLAIM. This grades the CARD'S OWN POINT - one
|
||||
price, at the monthly index the card assumed - and not every hour of every
|
||||
day. A live EPEX hour can and should fall outside this band (a negative
|
||||
hour is the best charging opportunity of the year, and §17 keeps a
|
||||
regression case for it), so the band is deliberately NOT applied to the
|
||||
fetched series. What it catches is a formula whose whole LEVEL is wrong,
|
||||
which is the factor-of-1000 class arriving from the end
|
||||
`MAX_PLAUSIBLE_EUR_KWH` does not bound.
|
||||
"""
|
||||
energy = price_eur_kwh(spec, spec.card_index_eur_per_mwh)
|
||||
all_in = energy + float(taxes_eur_kwh)
|
||||
if energy >= PLAUSIBLE_LO_EUR_KWH and all_in <= PLAUSIBLE_HI_EUR_KWH:
|
||||
return None
|
||||
return (
|
||||
f"price_formula's own card works out at {energy:.5f} EUR/kWh of "
|
||||
f"energy ({spec.card_expect_incl_vat:g} {spec.units} as the card "
|
||||
f"prints it) and {all_in:.5f} EUR/kWh all-in once "
|
||||
f"price_taxes_eur_kwh is added, which is outside the plausible band "
|
||||
f"{PLAUSIBLE_LO_EUR_KWH}-{PLAUSIBLE_HI_EUR_KWH} EUR/kWh for a Belgian "
|
||||
f"retail tariff. The card self-check passed, so the coefficients and "
|
||||
f"the printed price AGREE - they simply agree on a number no Belgian "
|
||||
f"bill carries, which is what a whole card read in the wrong units "
|
||||
f"looks like (a factor of 10, 100 or 1000 applied to BOTH sides at "
|
||||
f"once cancels out of that check and cannot cancel out of this one). "
|
||||
f"Check `units` against the unit printed beside your formula. "
|
||||
f"⚠️ This bounds the CARD'S OWN POINT, not every hour: a live hour may "
|
||||
f"legitimately fall outside the band, and a negative day-ahead hour - "
|
||||
f"the best charging opportunity of the year - is never refused.")
|
||||
|
||||
|
||||
def stale_card(spec: FormulaSpec, today=None):
|
||||
"""A warning when the card is older than ~6 months, or None. NEVER a
|
||||
refusal: ENGIE freezes a contract's coefficients for its duration, so an
|
||||
old card can be exactly right and refusing one would stop the add-on on a
|
||||
correctly configured site."""
|
||||
dated = _month_date(spec.card_month)
|
||||
now = today or datetime.now(timezone.utc).date()
|
||||
if dated is None or (now - dated).days <= CARD_STALE_DAYS:
|
||||
return None
|
||||
return (
|
||||
f"price_formula's card is dated {spec.card_month}, which is "
|
||||
f"{(now - dated).days} days ago. Suppliers reprice: if your contract "
|
||||
f"has been renewed or rolled over since then, the coefficients here "
|
||||
f"are last year's and every price this add-on computes is wrong by "
|
||||
f"however far they moved - and the card self-check cannot see it, "
|
||||
f"because it grades your coefficients against the card you typed, not "
|
||||
f"the card against your current contract. Not refused: ENGIE freezes "
|
||||
f"coefficients for the contract's duration, so an old card is often "
|
||||
f"exactly right. Worth re-reading your klantenzone once.")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Resolution
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
def to_resolution(pairs, resolution: str) -> list:
|
||||
"""`[(instant, index EUR/MWh)]` -> the series the CONTRACT is written on.
|
||||
|
||||
`quarter_hourly` hands the feed's own points back, ascending.
|
||||
`hourly` averages each hour's members into one point AT THE TOP OF THE
|
||||
HOUR.
|
||||
|
||||
⚠️ THE BUCKET IS A UTC HOUR AND THAT IS EXACT FOR BELGIUM, not an
|
||||
approximation: CET and CEST are both WHOLE-hour offsets, so a Brussels hour
|
||||
and a UTC hour start at the same instant on every day of the year including
|
||||
both transition days. Keying by a naive local hour is what collides the two
|
||||
02:00s in October; keying by the UTC instant cannot.
|
||||
|
||||
⚠️ AND THIS IS NOT `_Series.at()`'s HOURLY ASSUMPTION COMING BACK
|
||||
(TARIFF-14). It decides how many points are EMITTED, not how long one
|
||||
covers - the coverage window is still derived from the neighbours in
|
||||
`_cover_table()`. A feed that already publishes hourly points averages one
|
||||
member per hour, i.e. the identity, so this cannot silently assume the
|
||||
granularity it was handed.
|
||||
|
||||
⚠️ AN INCOMPLETE HOUR IS AVERAGED OVER WHAT IS THERE rather than dropped or
|
||||
zero-filled. A horizon that starts at 09:20 has two quarters in its first
|
||||
hour; their mean is the best statement available about that hour, and
|
||||
dropping it would leave a hole `at()` answers with `Absence` for an hour
|
||||
the feed did publish.
|
||||
"""
|
||||
ordered = sorted(pairs, key=lambda r: r[0].astimezone(timezone.utc))
|
||||
if resolution == "quarter_hourly":
|
||||
return list(ordered)
|
||||
if resolution != "hourly":
|
||||
raise FormulaInvalid(f"unknown resolution {resolution!r}")
|
||||
buckets: dict = {}
|
||||
for start, value in ordered:
|
||||
key = start.astimezone(timezone.utc).replace(
|
||||
minute=0, second=0, microsecond=0)
|
||||
buckets.setdefault(key, []).append(value)
|
||||
return [(k, sum(v) / len(v)) for k, v in sorted(buckets.items())]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# The startup preview
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
PREVIEW_HOURS = 24
|
||||
PREVIEW_MAX_ROWS = 24
|
||||
|
||||
|
||||
def preview_lines(spec: FormulaSpec, rows) -> list:
|
||||
"""`[(instant, index EUR/MWh, all-in EUR/kWh)]` -> lines for the log.
|
||||
|
||||
THE DEFECT THIS EXISTS TO MAKE VISIBLE: every error this module refuses is
|
||||
one the operator cannot see from the status page, and the two numbers that
|
||||
would have shown it - the index that went IN and the price that came OUT -
|
||||
are never printed beside each other anywhere else. A factor of 1000 is
|
||||
obvious in one line of `61.96 -> 0.13522` and invisible in a plan.
|
||||
|
||||
⚠️ IT PRINTS A BOUNDED NUMBER OF ROWS AND SAYS SO. At `quarter_hourly` the
|
||||
next 24 h is 96 points; the cap is a row COUNT rather than "every fourth
|
||||
point", because sampling every fourth point would assume the granularity
|
||||
TARIFF-14 spent a card removing the assumption of.
|
||||
"""
|
||||
if not rows:
|
||||
return []
|
||||
shown = rows[:PREVIEW_MAX_ROWS]
|
||||
prices = sorted(r[2] for r in rows)
|
||||
mid = len(prices) // 2
|
||||
median = (prices[mid] if len(prices) % 2
|
||||
else (prices[mid - 1] + prices[mid]) / 2.0)
|
||||
out = [
|
||||
f"custom_formula preview - the next {PREVIEW_HOURS} h of YOUR tariff, "
|
||||
f"index EUR/MWh -> all-in EUR/kWh, computed as "
|
||||
f"({spec.factor:g} x index + {spec.offset:g}) x "
|
||||
f"(1 + {spec.vat_pct:g}/100) {spec.units}, converted to EUR/kWh and "
|
||||
f"with price_taxes_eur_kwh added by the framework:"]
|
||||
for start, index, all_in in shown:
|
||||
out.append(f" {start.isoformat()} {index:9.3f} -> {all_in:9.5f}")
|
||||
if len(rows) > len(shown):
|
||||
out.append(f" ... {len(rows) - len(shown)} further point(s) not "
|
||||
f"listed; the figures below cover ALL {len(rows)}")
|
||||
out.append(
|
||||
f" min {prices[0]:.5f} / median {median:.5f} / max {prices[-1]:.5f} "
|
||||
f"EUR/kWh, spread {prices[-1] - prices[0]:.5f} EUR/kWh over "
|
||||
f"{len(rows)} point(s). ⚠️ If those are a thousand times what your "
|
||||
f"bill says, `units` is wrong; if the spread is 0, `factor` is.")
|
||||
return out
|
||||
+272
-58
@@ -72,9 +72,10 @@ THE THREE THINGS THIS MODULE IS SHAPED BY
|
||||
reads the enum. A plugin id appears nowhere in this file except in
|
||||
`PROVIDERS`, in `RETIRED_PROVIDERS`, and in the class that answers to it.
|
||||
|
||||
⚠️ NO SHIPPED PLUGIN DECLARES `PriceProduct.REALTIME` (TARIFF-07). Both
|
||||
shipped plugins are day-ahead, so `_WIDE`, `Uncertainty.REALTIME_PRODUCT`
|
||||
and `WIDEN_FACTOR`'s 2.5 currently have no shipped declarer. They are KEPT
|
||||
⚠️ NO SHIPPED PLUGIN DECLARES `PriceProduct.REALTIME` (TARIFF-07). All
|
||||
three shipped plugins are day-ahead, so `_WIDE`,
|
||||
`Uncertainty.REALTIME_PRODUCT` and `WIDEN_FACTOR`'s 2.5 currently have no
|
||||
shipped declarer. They are KEPT
|
||||
deliberately - see DECISIONS/TARIFF-07-withdrawing-three-plugins.md - and
|
||||
test_price.py section 10 keeps them live by declaring REALTIME from a class
|
||||
defined outside this file, which is AC2's own stimulus.
|
||||
@@ -149,6 +150,10 @@ from enum import Enum
|
||||
from textwrap import dedent
|
||||
from urllib.parse import urlencode
|
||||
|
||||
from .custom_formula import (
|
||||
FormulaInvalid, PREVIEW_HOURS, parse_price_formula, preview_lines,
|
||||
price_eur_kwh, to_resolution)
|
||||
|
||||
_LOG = logging.getLogger("goodwe.price")
|
||||
|
||||
|
||||
@@ -846,6 +851,13 @@ PRICE_OPTION_DEFAULTS = {
|
||||
# test_price.py section 1. A second defaults dict in the new module would be
|
||||
# a fourth place option defaults live, which is exactly the open ticket.
|
||||
"static_profile": "",
|
||||
# TARIFF-15. A JSON object carrying ONE contract's two coefficients, the
|
||||
# units they are printed in, its VAT rate and the card figures the add-on
|
||||
# grades them against; "" is unset. Parsed and validated by
|
||||
# app/custom_formula.py, refused at startup by validate.py - the
|
||||
# `static_profile` precedent exactly, and here for the same CFG-03 reason:
|
||||
# this dict is the ONE in-code mirror of config.yaml.
|
||||
"price_formula": "",
|
||||
# OPT-04. §10.1 priority 3, and §14's `optimizer.cycle_cost_threshold_eur`.
|
||||
#
|
||||
# ⚠️ THEY LIVE HERE FOR CFG-03's SAKE, exactly as `static_profile` does, and
|
||||
@@ -983,11 +995,20 @@ def export_price_eur_kwh(opts: dict) -> float:
|
||||
the SAME `B x spot - A` shape as the consumption curve but carries ITS OWN
|
||||
coefficients and its own VAT treatment - injection is VAT-exempt where
|
||||
consumption is at 6% - so it is a second per-contract formula, not a second
|
||||
copy of the first. Nothing in this add-on can evaluate either one yet
|
||||
(`generic_dayahead` cannot express `B x spot + A` at all; the formula
|
||||
provider is a separate card). Until it can, this scalar is a SIMPLIFICATION
|
||||
an operator enters by hand - an average of what they are actually paid -
|
||||
and not a reflection of the market. When the formula provider lands, an
|
||||
copy of the first.
|
||||
|
||||
⚠️ TARIFF-15 LANDED THE EVALUATOR AND DID NOT LAND THIS CURVE, and the
|
||||
difference is the point rather than an omission. `custom_formula` can now
|
||||
express `B x index + A` exactly - it is what it exists for - but it is a
|
||||
CONSUMPTION provider: it declares {ENERGY, MARKUP}, the framework adds the
|
||||
consumption taxes, and an injection leg carries its OWN coefficients
|
||||
(Eneco: 0,102 consumption against 0,100 injection) and its own VAT
|
||||
treatment (injection is exempt; consumption is at 6%). Pointing this
|
||||
scalar at that machinery would mean pricing an injection curve with a
|
||||
consumption card, which is the wrong-card defect the card self-check
|
||||
exists to refuse. So until the injection card lands this scalar remains a
|
||||
SIMPLIFICATION an operator enters by hand - an average of what they are
|
||||
actually paid - and not a reflection of the market. When it does land, an
|
||||
injection curve gets a `PriceProvider` of its own and this scalar becomes
|
||||
its FIXED-product default; the seam is already the right shape for that.
|
||||
|
||||
@@ -1046,6 +1067,28 @@ class PriceProvider(ABC):
|
||||
inheriting the silence.
|
||||
"""
|
||||
|
||||
def startup_preview(self, points) -> list:
|
||||
"""Lines to log ONCE, the first time this provider answers (TARIFF-15).
|
||||
|
||||
`points` are the ASSEMBLED, all-in `PricePoint`s - what the site is
|
||||
really priced on - and the default is to say nothing, so no existing
|
||||
plugin changes behaviour.
|
||||
|
||||
⚠️ IT IS A HOOK RATHER THAN A LINE IN `load_prices()` BECAUSE THE
|
||||
FRAMEWORK CANNOT KNOW WHAT IS WORTH PREVIEWING. Only the plugin knows
|
||||
what went IN to its own arithmetic - for `custom_formula` the index in
|
||||
EUR/MWh - and printing that beside the all-in figure is the whole
|
||||
point: a factor of 1000 is obvious in one line of
|
||||
`61.96 -> 0.13522` and invisible in a plan. The framework supplies the
|
||||
one number a plugin may never read, by handing back the assembled
|
||||
result rather than the tax rate.
|
||||
|
||||
⚠️ IT MUST NOT RAISE INTO THE CONTROL PATH and it is not trusted not
|
||||
to: `_emit_startup_preview()` catches everything. A log line is not a
|
||||
reason to stop regulating a battery.
|
||||
"""
|
||||
return []
|
||||
|
||||
|
||||
# ⚠️ THE NAMES A PLUGIN'S OWN CODE MAY NOT MENTION. FRAMEWORK_KEYS is the
|
||||
# obvious half; the other three are the routes review of PR #16 actually used to
|
||||
@@ -1491,6 +1534,60 @@ def _points_from_columnar_json(body, expect_unit, scale,
|
||||
return out
|
||||
|
||||
|
||||
def dayahead_feed_url(plugin_id, opts, since, until, base, bzn) -> str:
|
||||
"""The day-ahead URL for a window, or the operator's override, verbatim.
|
||||
|
||||
⚠️ SHARED BY `generic_dayahead` AND `custom_formula`, ON PURPOSE
|
||||
(TARIFF-15). They read the SAME feed for the same range; two copies of this
|
||||
rule would be two places for the Brussels-calendar derivation and the
|
||||
override decision to disagree, and the one that is wrong would be wrong
|
||||
silently - §11.4 turns a short horizon into provisional hours, not into an
|
||||
error. It takes `plugin_id` rather than reading one, so the framework's
|
||||
AC2 rule (no plugin id in shared code) still holds here.
|
||||
|
||||
⚠️ AN OVERRIDE IS THE EXACT URL AND NOTHING IS APPENDED TO IT, WHICH COSTS
|
||||
THE OVERRIDING SITE THE RANGE. That is a decision, not an oversight, and
|
||||
the sentence below is why it is not a footnote:
|
||||
|
||||
* `price_feed_url` says "override the plugin's default feed", and an
|
||||
override that is silently rewritten is not one. Merging parameters
|
||||
into an operator's query string means parsing and re-serialising a
|
||||
URL they wrote - which reorders, de-duplicates and re-escapes it,
|
||||
and can hand the server a request nobody typed.
|
||||
* `start`/`end` are ENERGY-CHARTS' spelling, and both providers DECLARE
|
||||
A SECOND HOST: `web-api.tp.entsoe.eu`. Nothing here has verified that
|
||||
host's day-ahead parameters and nothing here should guess them - but
|
||||
they are not `start`/`end`, so an override to the other DECLARED host
|
||||
is exactly the case where appending our vocabulary adds two parameters
|
||||
that host ignores and yields the short horizon anyway, with nothing
|
||||
saying so.
|
||||
* and blind appending is refused outright: `dated_feed_url()` will not
|
||||
add a query string to a base that has one.
|
||||
|
||||
So the cost is real and it is REPORTED, at fetch time, on the shipped
|
||||
logger, naming the horizon consequence rather than the substitution.
|
||||
§11.4 is what makes a price defect quiet; an operator who has taken the
|
||||
short horizon has to be able to find out from the log that they did.
|
||||
"""
|
||||
override = str(opts.get("price_feed_url") or "").strip()
|
||||
if override:
|
||||
_LOG.warning(
|
||||
"price_feed_url is set, so %s is requesting %s exactly as "
|
||||
"given and is adding NO date range to it. The public feed "
|
||||
"answers a single Brussels day to a request that names no "
|
||||
"range, and every hour past that day is then carried forward "
|
||||
"from yesterday's profile and flagged provisional - §8.3 "
|
||||
"drops provisional hours from the trade set, so arbitrage "
|
||||
"would never see tomorrow as firm. Clear price_feed_url to "
|
||||
"let this plugin ask for the %s..%s it was given.",
|
||||
plugin_id, override, *(d.isoformat()
|
||||
for d in brussels_date_range(since, until)))
|
||||
return override
|
||||
first, last = brussels_date_range(since, until)
|
||||
return dated_feed_url(base, bzn=bzn, start=first.isoformat(),
|
||||
end=last.isoformat())
|
||||
|
||||
|
||||
@register
|
||||
class GenericDayAhead(PriceProvider):
|
||||
"""Public EPEX BE day-ahead feed, COLUMNAR, in EUR/MWh.
|
||||
@@ -1533,57 +1630,22 @@ class GenericDayAhead(PriceProvider):
|
||||
async def fetch(self, opts, since, until, transport):
|
||||
"""Request the range the CALLER asked for, on the Brussels calendar.
|
||||
|
||||
⚠️ THE ARGUMENTS ARE USED. Until TARIFF-09 this method took `since` and
|
||||
`until` and asked for the bare `?bzn=BE`, which answers ONE Brussels
|
||||
day - so a 48 h horizon was 15 firm hours and 33 carried at 09:00, and
|
||||
1 firm and 47 carried at 23:00, with the whole trade set inside a
|
||||
single calendar day at every hour of every day. See
|
||||
`brussels_date_range()` for the measurement and for why the calendar
|
||||
is Brussels rather than UTC.
|
||||
⚠️ THE ARGUMENTS ARE USED, and `dayahead_feed_url()` above is where
|
||||
they are turned into a URL. Until TARIFF-09 this method took `since`
|
||||
and `until` and asked for the bare `?bzn=BE`, which answers ONE
|
||||
Brussels day - so a 48 h horizon was 15 firm hours and 33 carried at
|
||||
09:00, and 1 firm and 47 carried at 23:00, with the whole trade set
|
||||
inside a single calendar day at every hour of every day.
|
||||
|
||||
⚠️ AN OVERRIDE IS THE EXACT URL AND THIS METHOD APPENDS NOTHING TO IT,
|
||||
WHICH COSTS THE OVERRIDING SITE THE RANGE. That is a decision, not an
|
||||
oversight, and the sentence below is why it is not a footnote:
|
||||
|
||||
* `price_feed_url` says "override the plugin's default feed", and an
|
||||
override that is silently rewritten is not one. Merging parameters
|
||||
into an operator's query string means parsing and re-serialising a
|
||||
URL they wrote - which reorders, de-duplicates and re-escapes it,
|
||||
and can hand the server a request nobody typed.
|
||||
* `start`/`end` are ENERGY-CHARTS' spelling, and this provider
|
||||
DECLARES A SECOND HOST: `web-api.tp.entsoe.eu`. Nothing here has
|
||||
verified that host's day-ahead parameters and nothing here should
|
||||
guess them - but they are not `start`/`end`, so an override to the
|
||||
other DECLARED host is exactly the case where appending our
|
||||
vocabulary adds two parameters that host ignores and yields the
|
||||
short horizon anyway, with nothing saying so.
|
||||
* and blind appending is refused outright: `dated_feed_url()` will
|
||||
not add a query string to a base that has one.
|
||||
|
||||
So the cost is real and it is REPORTED, at fetch time, on the shipped
|
||||
logger, naming the horizon consequence rather than the substitution.
|
||||
§11.4 is what makes a price defect quiet; an operator who has taken the
|
||||
short horizon has to be able to find out from the log that they did.
|
||||
⚠️ THE URL DECISION MOVED OUT OF THIS METHOD IN TARIFF-15 AND DID NOT
|
||||
CHANGE. `custom_formula` reads the same feed with the same range and
|
||||
the same parser, and a second copy of the override rule would be a
|
||||
second place for it to be wrong - which is the reason `covers()`
|
||||
delegates to `at()` rather than restating its window. The reasoning,
|
||||
the warning and the exact URL are all in `dayahead_feed_url()`.
|
||||
"""
|
||||
override = str(opts.get("price_feed_url") or "").strip()
|
||||
if override:
|
||||
url = override
|
||||
_LOG.warning(
|
||||
"price_feed_url is set, so %s is requesting %s exactly as "
|
||||
"given and is adding NO date range to it. The public feed "
|
||||
"answers a single Brussels day to a request that names no "
|
||||
"range, and every hour past that day is then carried forward "
|
||||
"from yesterday's profile and flagged provisional - §8.3 "
|
||||
"drops provisional hours from the trade set, so arbitrage "
|
||||
"would never see tomorrow as firm. Clear price_feed_url to "
|
||||
"let this plugin ask for the %s..%s it was given.",
|
||||
self.id, url, *(d.isoformat()
|
||||
for d in brussels_date_range(since, until)))
|
||||
else:
|
||||
first, last = brussels_date_range(since, until)
|
||||
url = dated_feed_url(self.FEED_BASE, bzn=self.BZN,
|
||||
start=first.isoformat(),
|
||||
end=last.isoformat())
|
||||
url = dayahead_feed_url(self.id, opts, since, until,
|
||||
self.FEED_BASE, self.BZN)
|
||||
check_egress(url, self.capabilities())
|
||||
body = await transport(url, float(opts.get("price_timeout_s")))
|
||||
return _points_from_columnar_json(body, expect_unit="EUR / MWh",
|
||||
@@ -1660,10 +1722,160 @@ class ManualCsv(PriceProvider):
|
||||
return out
|
||||
|
||||
|
||||
@register
|
||||
class CustomFormula(PriceProvider):
|
||||
"""The operator's OWN tariff card, as two fixed coefficients. TARIFF-15.
|
||||
|
||||
SOURCES: energy, markup
|
||||
The framework ADDS: taxes
|
||||
|
||||
⚠️ IT DECLARES `{ENERGY, MARKUP}` AND NOT `ALL_IN`, AND THAT IS THE WHOLE
|
||||
SAFETY ARGUMENT. A Belgian card's `Tariefformule` line is the ENERGY
|
||||
component only - Eneco tabulates network tariffs and levies separately,
|
||||
ENGIE states that the formula excludes network, distribution, transport,
|
||||
grid levies and excise - so declaring ALL_IN would lie on behalf of every
|
||||
operator who typed exactly what their card says, and would silently drop
|
||||
~12 c EUR/kWh from every hour. The framework adds `price_taxes_eur_kwh`
|
||||
on top, once, and this class cannot see or apply it (trap 1).
|
||||
|
||||
The risk that introduces is one no guard in this file can see, because it
|
||||
is not in code: THE OPERATOR folding their network charges into `offset`
|
||||
while the tax option carries them too. `app/custom_formula.py`'s card
|
||||
self-check is what refuses that - the card's printed estimate is the
|
||||
energy component alone, so an inflated offset moves 15.55 to ~27 and the
|
||||
add-on will not start. Its limit is stated in the same breath there: it
|
||||
grades the coefficients against the card, NOT the card against reality.
|
||||
|
||||
⚠️ NO EXPRESSION EVALUATION OF ANY KIND, AND THE COEFFICIENTS ARE NUMBERS
|
||||
BECAUSE THE PARSER REFUSES ANYTHING ELSE. This string is settable over
|
||||
HTTP through the add-on's own ingress config page, so `eval` on it would
|
||||
be RCE in a root container holding SUPERVISOR_TOKEN - and, the deciding
|
||||
argument, an arbitrary expression could not be graded against the card at
|
||||
all. See app/custom_formula.py.
|
||||
|
||||
⚠️ THE INDEX PATH IS `generic_dayahead`'s, NOT A SECOND ONE. Same feed,
|
||||
same `dayahead_feed_url()` request on the Brussels calendar, same columnar
|
||||
parser, same declared egress hosts. A second parser would be a second
|
||||
failure mode for a document that already has one that works.
|
||||
"""
|
||||
|
||||
id = "custom_formula"
|
||||
|
||||
FEED_BASE = "https://api.energy-charts.info/price"
|
||||
BZN = "BE"
|
||||
|
||||
# Written by fetch(), read by startup_preview(), and by nothing else.
|
||||
spec = None
|
||||
index_by_instant: dict = {}
|
||||
|
||||
@classmethod
|
||||
def capabilities(cls):
|
||||
return ProviderCapabilities(
|
||||
product=PriceProduct.DAY_AHEAD,
|
||||
sources=frozenset({Component.ENERGY, Component.MARKUP}),
|
||||
egress_hosts=("api.energy-charts.info", "web-api.tp.entsoe.eu"))
|
||||
|
||||
async def fetch(self, opts, since, until, transport):
|
||||
"""Fetch the index, apply the card's arithmetic, return RawPoints.
|
||||
|
||||
⚠️ `since`/`until` ARE USED, through the shared URL builder - see
|
||||
`dayahead_feed_url()` and TARIFF-09 for what ignoring them costs.
|
||||
"""
|
||||
try:
|
||||
spec = parse_price_formula(opts.get("price_formula"))
|
||||
except FormulaInvalid as e:
|
||||
raise SourceUnavailable(f"price_formula: {e}")
|
||||
if spec is None:
|
||||
raise SourceUnavailable(
|
||||
"price_provider is 'custom_formula' but price_formula is "
|
||||
"empty, so there are no coefficients to price anything with. "
|
||||
"validate.py refuses this configuration at startup; reaching "
|
||||
"it here means the options blob changed under a running "
|
||||
"add-on")
|
||||
url = dayahead_feed_url(self.id, opts, since, until,
|
||||
self.FEED_BASE, self.BZN)
|
||||
check_egress(url, self.capabilities())
|
||||
body = await transport(url, float(opts.get("price_timeout_s")))
|
||||
# ⚠️ `scale=1.0`, SO THESE RawPoints CARRY EUR/**MWh** IN A FIELD
|
||||
# SPELLED `eur_per_kwh`. That is a trap and it is contained on purpose:
|
||||
# the list is destructured on the very next line and never leaves this
|
||||
# method, so nothing downstream can read the wrong unit off it. The
|
||||
# alternative - scaling to EUR/kWh and multiplying by 1000 again to
|
||||
# feed the formula - is two conversions where the contract needs none,
|
||||
# and every extra conversion here is another place to lose a factor of
|
||||
# 1000. If this list ever DID escape, `_validate_raw()` would refuse it
|
||||
# at `MAX_PLAUSIBLE_EUR_KWH` rather than price a day on it.
|
||||
index = _points_from_columnar_json(body, expect_unit="EUR / MWh",
|
||||
scale=1.0,
|
||||
provisional_default=False)
|
||||
pairs = to_resolution([(p.start, p.eur_per_kwh) for p in index],
|
||||
spec.resolution)
|
||||
self.spec = spec
|
||||
self.index_by_instant = {t.astimezone(timezone.utc): v
|
||||
for t, v in pairs}
|
||||
# ⚠️ `provisional=False` FOR THE SAME REASON `generic_dayahead` SAYS
|
||||
# SO: the day-ahead auction is settled once published, and the columnar
|
||||
# document carries no per-point flag to read one from. FCST-04's
|
||||
# carry-forward is what marks the hours nobody published yet.
|
||||
return [RawPoint(start=t, eur_per_kwh=price_eur_kwh(spec, v),
|
||||
provisional=False)
|
||||
for t, v in pairs]
|
||||
|
||||
def startup_preview(self, points) -> list:
|
||||
"""`index EUR/MWh -> all-in EUR/kWh` for the first 24 h, once.
|
||||
|
||||
⚠️ THE INDEX COLUMN IS WHY THIS HOOK EXISTS. The all-in price alone
|
||||
says nothing about which of the two numbers is wrong; printed beside
|
||||
the index it came from, a factor of 1000 is one glance. See
|
||||
`preview_lines()` for the rows and the min/median/max/spread line.
|
||||
"""
|
||||
if self.spec is None or not points:
|
||||
return []
|
||||
cut = points[0].start + timedelta(hours=PREVIEW_HOURS)
|
||||
rows = []
|
||||
for point in points:
|
||||
if point.start >= cut:
|
||||
break
|
||||
index = self.index_by_instant.get(
|
||||
point.start.astimezone(timezone.utc))
|
||||
if index is not None:
|
||||
rows.append((point.start, index, point.eur_per_kwh))
|
||||
return preview_lines(self.spec, rows)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# The one entry point
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
# Provider ids whose startup preview has already been emitted in this process.
|
||||
# ⚠️ NOT A PLUGIN NAME IN THE FRAMEWORK: it is filled at runtime from whatever
|
||||
# id was resolved, and starts empty. AC2 is intact.
|
||||
_PREVIEWED: set = set()
|
||||
|
||||
|
||||
def _emit_startup_preview(plugin, pid: str, points) -> None:
|
||||
"""Log a plugin's preview once per process, and never fail because of it.
|
||||
|
||||
⚠️ THE ID IS RECORDED BEFORE THE HOOK RUNS, DELIBERATELY. A preview that
|
||||
raises must not be retried every 30 minutes for the life of the add-on -
|
||||
that would turn a cosmetic defect into a log flood on the one path §11.4
|
||||
already makes hard to read.
|
||||
"""
|
||||
if pid in _PREVIEWED:
|
||||
return
|
||||
_PREVIEWED.add(pid)
|
||||
try:
|
||||
lines = list(plugin.startup_preview(points))
|
||||
except Exception as e: # noqa: BLE001
|
||||
_LOG.warning("%s's startup preview raised (%s: %s); prices are "
|
||||
"unaffected - this is a log line, not a decision",
|
||||
pid, type(e).__name__, e)
|
||||
return
|
||||
for line in lines:
|
||||
_LOG.info("%s", line)
|
||||
|
||||
|
||||
|
||||
async def load_prices(opts: dict, since: datetime, until: datetime,
|
||||
transport=None) -> PriceRead:
|
||||
"""Resolve the configured provider, fetch, assemble all-in, return a read.
|
||||
@@ -1709,8 +1921,9 @@ async def load_prices(opts: dict, since: datetime, until: datetime,
|
||||
|
||||
if transport is None:
|
||||
transport = https_get
|
||||
plugin = cls()
|
||||
try:
|
||||
raw = await cls().fetch(PluginOptions(opts), since, until, transport)
|
||||
raw = await plugin.fetch(PluginOptions(opts), since, until, transport)
|
||||
_validate_raw(raw, caps)
|
||||
points = assemble(caps, raw, markup, taxes)
|
||||
except FrameworkOption:
|
||||
@@ -1720,5 +1933,6 @@ async def load_prices(opts: dict, since: datetime, until: datetime,
|
||||
except Exception as e: # noqa: BLE001 - see the docstring
|
||||
return PriceRead.no_data(f"{pid}: {type(e).__name__}: {e}")
|
||||
|
||||
_emit_startup_preview(plugin, pid, points)
|
||||
return PriceRead.series(points, caps.product, pid,
|
||||
datetime.now(timezone.utc))
|
||||
|
||||
@@ -324,6 +324,8 @@ import logging
|
||||
|
||||
from .arbitrage import enabled as arbitrage_on
|
||||
from .control import SLEW_RAIL_FRAC, LOOP_TICK_S, PLANT_SETTLE_S
|
||||
from .custom_formula import (
|
||||
FormulaInvalid, implausible_all_in, parse_price_formula, stale_card)
|
||||
from .inverter import roster_entries
|
||||
from .maintenance import MAINT_OPTION_DEFAULTS
|
||||
from .p1 import is_enabled, validate_meter_options
|
||||
@@ -1249,9 +1251,12 @@ def _price_problems(opts: dict) -> list[str]:
|
||||
share of the installed base, per §9.2 - cannot run at all.
|
||||
|
||||
Five refusals and two log lines (TARIFF-03 added one refusal, one parse and
|
||||
two notices; TARIFF-07 added the retired-id one), and every refusal is still
|
||||
case (a) from this module's docstring: a source the operator SET being
|
||||
overridden by a layer they cannot see. The additions are `static_profile`
|
||||
two notices; TARIFF-07 added the retired-id one; TARIFF-15 added three -
|
||||
a `custom_formula` with no formula, a formula that does not parse or does
|
||||
not reproduce its own card, and one whose card lands outside the plausible
|
||||
band - plus two notices, a formula nothing reads and a stale card), and
|
||||
every refusal is still case (a) from this module's docstring: a source the
|
||||
operator SET being overridden by a layer they cannot see. The additions are `static_profile`
|
||||
alongside the Home Assistant price pair - the same silent-loser shape as
|
||||
provider-plus-pair, and a LIVE one - and a `static_profile` that does not
|
||||
tile the day, which would leave hours with no price at all.
|
||||
@@ -1415,6 +1420,85 @@ def _price_problems(opts: dict) -> list[str]:
|
||||
"for this product. Peak shaving, reserve management and the "
|
||||
"maintenance cycle are unaffected.", profile.prices[0])
|
||||
|
||||
# ⚠️ TARIFF-15: THE FORMULA, GRADED AGAINST THE OPERATOR'S OWN CARD, AT
|
||||
# STARTUP RATHER THAN AT 03:00.
|
||||
#
|
||||
# THE DEFECT THIS EXISTS TO PREVENT, and it is not one `code_leaks()` can
|
||||
# see because it is not in code: an operator folds their NETWORK CHARGES
|
||||
# into `offset` while `price_taxes_eur_kwh` carries them as well, and the
|
||||
# site is billed against a price ~12 c EUR/kWh high on every hour for as
|
||||
# long as nobody re-reads the configuration. §11.4 makes every price
|
||||
# defect quiet, so nothing on the status page would ever differ.
|
||||
#
|
||||
# The card is what grades it. A Belgian tariff card prints the formula,
|
||||
# the reference index it assumed AND the resulting price, so the add-on
|
||||
# recomputes that printed price from the numbers the operator typed and
|
||||
# refuses to start unless the two agree to within CARD_TOLERANCE. An
|
||||
# inflated `offset` cannot pass - the printed estimate is the energy
|
||||
# component alone, so ~12 c EUR/kWh moves 15.55 to ~27. The same check
|
||||
# catches VAT omitted, VAT applied twice, a transposed digit, and a unit
|
||||
# MISMATCH between the coefficients and the printed estimate. The one unit
|
||||
# error it cannot see is the one applied to BOTH sides at once, because
|
||||
# that cancels out of a comparison between them - which is why
|
||||
# `implausible_all_in()` below is a second refusal on a second quantity
|
||||
# rather than a restatement of this one.
|
||||
#
|
||||
# ⚠️ AND ITS LIMIT, IN THE SAME SENTENCE AS THE CLAIM, exactly as price.py
|
||||
# states trap 1's: IT GRADES THE COEFFICIENTS AGAINST THE CARD, NOT THE
|
||||
# CARD AGAINST REALITY. A consistent-but-wrong pair - both numbers off the
|
||||
# wrong product - reproduces its own printed price and passes. Nothing can
|
||||
# catch that.
|
||||
raw_formula = opts.get("price_formula", "")
|
||||
has_formula = bool(str(raw_formula or "").strip()) \
|
||||
or isinstance(raw_formula, dict)
|
||||
if pid == "custom_formula" and not has_formula:
|
||||
problems.append(
|
||||
"price_provider is 'custom_formula' but price_formula is empty, "
|
||||
"so the add-on has no coefficients to price anything with. Every "
|
||||
"fetch would return no-price-data and §11.4 would fall back "
|
||||
"silently - which is the shape of defect this provider exists to "
|
||||
"remove, arrived at by leaving one field blank. Copy the "
|
||||
"Tariefformule line off your supplier's tariff card, e.g. "
|
||||
"{\"index\":\"epex_be\",\"resolution\":\"hourly\","
|
||||
"\"factor\":0.102,\"offset\":1.00,"
|
||||
"\"units\":\"c_eur_per_kwh\",\"vat_pct\":6,"
|
||||
"\"card\":{\"month\":\"2026-09\","
|
||||
"\"index_eur_per_mwh\":133.9905,\"expect_incl_vat\":15.55}} "
|
||||
"- see DOCS.md. Or set price_provider to 'generic_dayahead' for "
|
||||
"the wholesale curve with a flat markup.")
|
||||
elif has_formula:
|
||||
try:
|
||||
spec = parse_price_formula(raw_formula)
|
||||
except FormulaInvalid as e:
|
||||
spec = None
|
||||
problems.append(f"price_formula cannot be used: {e}")
|
||||
if spec is not None and pid != "custom_formula":
|
||||
# ⚠️ A WARNING AND NOT A REFUSAL. This is not two price sources
|
||||
# fighting over one flag - it is one that is simply not wired to
|
||||
# anything - so the §14 warning box does not apply. But an option
|
||||
# an operator filled in and that decides nothing is exactly the
|
||||
# silent loser this module's case (a) is about, so it is said out
|
||||
# loud rather than left to be discovered from a bill.
|
||||
_LOG.warning(
|
||||
"price_formula is filled in but price_provider is %r, so the "
|
||||
"coefficients decide NOTHING - only 'custom_formula' reads "
|
||||
"them. Set price_provider to 'custom_formula' to use them, or "
|
||||
"clear price_formula so the configuration says what the "
|
||||
"add-on is doing.", pid)
|
||||
if spec is not None and pid == "custom_formula":
|
||||
taxes = _number(opts, "price_taxes_eur_kwh",
|
||||
PRICE_OPTION_DEFAULTS["price_taxes_eur_kwh"])
|
||||
band = implausible_all_in(spec, taxes if taxes is not None else 0.0)
|
||||
if band:
|
||||
problems.append(band)
|
||||
aged = stale_card(spec)
|
||||
if aged:
|
||||
# ⚠️ WARN, NEVER REFUSE. ENGIE freezes a contract's
|
||||
# coefficients for its duration, so a two-year-old card can be
|
||||
# exactly right and a site refused over one is a site that
|
||||
# cannot start.
|
||||
_LOG.warning("%s", aged)
|
||||
|
||||
# ⚠️ A WARNING, NOT A REFUSAL, AND THE ASYMMETRY IS DELIBERATE. §11.4 says a
|
||||
# site with no price data falls back to the static profile; a dynamic site
|
||||
# with none configured has nothing to fall back to when its feed dies. That
|
||||
|
||||
@@ -286,6 +286,9 @@ options:
|
||||
# the FRAMEWORK, only for the components the chosen plugin declares its feed
|
||||
# does not already contain. manual_csv carries an all-in figure, so setting
|
||||
# either under it is ignored - and logged as ignored, never applied twice.
|
||||
# `custom_formula` (TARIFF-15) carries energy AND markup - your contract's
|
||||
# formula IS the markup - so it takes price_taxes_eur_kwh only, and your
|
||||
# price_markup_eur_kwh is ignored under it, with a line in the log saying so.
|
||||
price_provider: "off"
|
||||
price_markup_eur_kwh: 0.03
|
||||
price_taxes_eur_kwh: 0.06
|
||||
@@ -324,6 +327,54 @@ options:
|
||||
# to fall back to and a dead feed leaves the site with no tariff at all.
|
||||
# validate.py warns about exactly that.
|
||||
static_profile: ""
|
||||
|
||||
# TARIFF-15. YOUR OWN TARIFF CARD, as two fixed coefficients. Used only when
|
||||
# `price_provider: custom_formula`; "" is unset.
|
||||
#
|
||||
# A Belgian dynamic contract prices energy as `B x index + A`, and the two
|
||||
# coefficients exist in exactly ONE place - the tariff card in your
|
||||
# klantenzone. There is no machine-readable source for them anywhere (CREG's
|
||||
# tariff-card pages 404, CREG Scan is a JS wizard, VREG was unreachable), so
|
||||
# no plugin and no scraper can ever know YOUR numbers. You are the only party
|
||||
# who can see them.
|
||||
#
|
||||
# A JSON object, on one line. You type what your card prints, IN THE CARD'S
|
||||
# OWN UNITS; every unit conversion and the VAT multiplication are the
|
||||
# add-on's job, because those are exactly the two steps that silently cost a
|
||||
# factor of 10, 100 or 1.06.
|
||||
#
|
||||
# {"index":"epex_be","resolution":"hourly","factor":0.102,"offset":1.00,
|
||||
# "units":"c_eur_per_kwh","vat_pct":6,
|
||||
# "card":{"month":"2026-09","index_eur_per_mwh":133.9905,
|
||||
# "expect_incl_vat":15.55}}
|
||||
#
|
||||
# price = (factor x index_eur_per_mwh + offset) x (1 + vat_pct/100), in
|
||||
# `units`, converted to EUR/kWh by the add-on.
|
||||
#
|
||||
# ⚠️ THE `card` BLOCK IS REQUIRED AND IT IS THE SAFETY MECHANISM. Your card
|
||||
# prints the formula, the reference index it assumed AND the resulting price.
|
||||
# At startup the add-on recomputes that printed price from the numbers you
|
||||
# typed and REFUSES TO START unless the two agree to within 0.01 - naming the
|
||||
# discrepancy. That catches VAT left out or applied twice, a transposed
|
||||
# digit, a unit MISMATCH between the coefficients and the printed price
|
||||
# (10x/100x/1000x), and the expensive one: NETWORK CHARGES FOLDED INTO
|
||||
# `offset`. A unit error applied to BOTH sides at once cancels out of that
|
||||
# comparison; the add-on refuses that separately, by checking the card's own
|
||||
# point against the band a Belgian retail tariff can occupy. Your card's formula line is the ENERGY component
|
||||
# alone; the add-on adds price_taxes_eur_kwh on top, so folding ~12 c/kWh of
|
||||
# network charge into `offset` would bill you for it twice - and it cannot
|
||||
# pass, because it moves the computed 15.55 to about 27.
|
||||
#
|
||||
# ⚠️ AND THE LIMIT OF THAT CHECK, IN THE SAME BREATH: it grades your
|
||||
# COEFFICIENTS AGAINST YOUR CARD, not your card against reality. Two numbers
|
||||
# copied off the wrong product agree with each other perfectly and pass.
|
||||
#
|
||||
# ⚠️ FIXED COEFFICIENTS ONLY - the add-on evaluates NO expression of any kind
|
||||
# and there is no field that takes one. An unknown key is refused rather than
|
||||
# ignored. See DOCS.md for why (it is not only security: two coefficients can
|
||||
# be graded against your card and an arbitrary expression cannot be graded
|
||||
# against anything).
|
||||
price_formula: ""
|
||||
price_csv_path: "/share/goodwe_prices.csv"
|
||||
# ⚠️ AN OVERRIDE IS THE EXACT URL (TARIFF-09). The plugin sends it verbatim
|
||||
# and appends nothing - including the `start`/`end` date range it derives from
|
||||
@@ -782,7 +833,7 @@ schema:
|
||||
# this line offers exactly PROVIDERS + RETIRED_PROVIDERS + 'off' - derived, not
|
||||
# typed - and register() refuses to re-register a retired id, so the two
|
||||
# cannot both claim one.
|
||||
price_provider: list(off|generic_dayahead|engie_dynamic|totalenergies_dynamic|eneco_eddy|manual_csv)
|
||||
price_provider: list(off|generic_dayahead|custom_formula|engie_dynamic|totalenergies_dynamic|eneco_eddy|manual_csv)
|
||||
price_markup_eur_kwh: float(0,1)
|
||||
price_taxes_eur_kwh: float(0,1)
|
||||
|
||||
@@ -796,6 +847,11 @@ schema:
|
||||
# parse_static_profile() accepts an already-parsed dict too, so a future
|
||||
# nested schema is a config.yaml change and no code change at all.
|
||||
static_profile: str?
|
||||
# TARIFF-15. A JSON string for the same reason `static_profile` is one: it is
|
||||
# ONE option key, it survives config.yaml's flat options block, and
|
||||
# parse_price_formula() accepts an already-parsed dict too, so a future
|
||||
# nested schema is a config.yaml change and no code change at all.
|
||||
price_formula: str?
|
||||
price_csv_path: str?
|
||||
price_feed_url: str?
|
||||
price_horizon_h: int(1,72)
|
||||
|
||||
@@ -7,7 +7,7 @@ WHAT UX-01 AC 4 ASKED FOR, IN ITS OWN WORDS: "every `schema:` key absent from
|
||||
check, not a grep."
|
||||
|
||||
WHAT SHIPPED. Clause one is VACUOUS: `options:` and `schema:` declare the same
|
||||
71 keys, so "every schema key absent from options" quantifies over the empty set
|
||||
72 keys, so "every schema key absent from options" quantifies over the empty set
|
||||
and is true of any tree, including one with no defaults at all. Clause two -
|
||||
"the table changed goes red" - does work, six times over, once per mirror in its
|
||||
own suite. Clause three, THE ONE WITH THE CONTENT, was never written. The only
|
||||
|
||||
+632
-15
@@ -69,6 +69,12 @@ from app.price import (
|
||||
export_price_eur_kwh, load_prices, plan_is_provisional, register,
|
||||
volatility_regime, uncertainty_of, widen_factor, Uncertainty,
|
||||
)
|
||||
from app.custom_formula import (
|
||||
CARD_TOLERANCE, FormulaInvalid, INDEXES, PLAUSIBLE_HI_EUR_KWH,
|
||||
PLAUSIBLE_LO_EUR_KWH, RESOLUTIONS, UNITS, card_discrepancy,
|
||||
implausible_all_in, in_card_units, parse_price_formula, preview_lines,
|
||||
price_eur_kwh, stale_card, to_resolution)
|
||||
from app import custom_formula as _cf
|
||||
from app import price as _price
|
||||
from app import arbitrage as _arb
|
||||
from app import main as _main
|
||||
@@ -376,6 +382,27 @@ def columnar_feed(values, provisional_at=None, unit="EUR / MWh",
|
||||
return json.dumps(doc).encode()
|
||||
|
||||
|
||||
# TARIFF-15. THE TWO REAL TARIFF CARDS, as the operator would type them.
|
||||
# ⚠️ THESE ARE NOT INVENTED. Both suppliers print the formula, the reference
|
||||
# index they assumed AND the resulting price on the same card, so section 13
|
||||
# can grade the SHIPPED code against a number somebody else published - which
|
||||
# is the only kind of expectation this file is allowed to have (2b's rule,
|
||||
# applied to arithmetic rather than to bytes).
|
||||
ENECO_CARD = {"index": "epex_be", "resolution": "quarter_hourly",
|
||||
"factor": 0.102, "offset": 1.00, "units": "c_eur_per_kwh",
|
||||
"vat_pct": 6,
|
||||
"card": {"month": "2026-09", "index_eur_per_mwh": 133.9905,
|
||||
"expect_incl_vat": 15.55}}
|
||||
ENGIE_CARD = {"index": "epex_be", "resolution": "quarter_hourly",
|
||||
"factor": 0.100, "offset": 0.90, "units": "c_eur_per_kwh",
|
||||
"vat_pct": 6,
|
||||
"card": {"month": "2026-09", "index_eur_per_mwh": 61.96,
|
||||
"expect_incl_vat": 7.522}}
|
||||
# The energy+markup component of the shared EPEX fixture under Eneco's card,
|
||||
# spelled out here as the card spells it: c EUR/kWh, then /100 to EUR/kWh.
|
||||
FORMULA_EUR_KWH = [(0.102 * v + 1.00) * 1.06 / 100.0 for v in EPEX_EUR_MWH]
|
||||
|
||||
|
||||
CSV_BODY = ("start,eur_per_kwh,provisional\n"
|
||||
"2026-01-15T00:00:00+01:00,0.1834,false\n"
|
||||
"2026-01-15T01:00:00+01:00,-0.0210,true\n"
|
||||
@@ -390,12 +417,13 @@ CSV_BODY = ("start,eur_per_kwh,provisional\n"
|
||||
FIXTURES = {
|
||||
"generic_dayahead": (columnar_feed(EPEX_EUR_MWH),
|
||||
[v / 1000 for v in EPEX_EUR_MWH]),
|
||||
"custom_formula": (columnar_feed(EPEX_EUR_MWH), FORMULA_EUR_KWH),
|
||||
"manual_csv": (None, [0.1834, -0.0210, 0.3115]),
|
||||
}
|
||||
|
||||
check(f"the two plugins that survive TARIFF-07 are registered "
|
||||
check(f"the plugins that ship after TARIFF-07 and TARIFF-15 are registered "
|
||||
f"{sorted(PROVIDERS)}",
|
||||
set(PROVIDERS) == {"generic_dayahead", "manual_csv"})
|
||||
set(PROVIDERS) == {"generic_dayahead", "custom_formula", "manual_csv"})
|
||||
check(f"...and the three withdrawn ids resolve to NOTHING "
|
||||
f"{sorted(RETIRED_PROVIDERS)}",
|
||||
set(RETIRED_PROVIDERS) == {"engie_dynamic", "totalenergies_dynamic",
|
||||
@@ -453,6 +481,13 @@ def load_any(pid):
|
||||
body, _ = FIXTURES[pid]
|
||||
if pid == "manual_csv":
|
||||
return load(pid, {"price_csv_path": CSV_PATH}, transport=refuse_transport)
|
||||
if pid == "custom_formula":
|
||||
# ⚠️ THE SAME BODY AS generic_dayahead, ON PURPOSE. Both read the same
|
||||
# feed, so feeding them different bytes would hide the fact that what
|
||||
# differs between them is the ARITHMETIC and the declaration - which
|
||||
# is exactly what section 3 grades.
|
||||
return load(pid, {"price_formula": json.dumps(ENECO_CARD)},
|
||||
transport=canned(body))
|
||||
return load(pid, transport=canned(body))
|
||||
|
||||
|
||||
@@ -744,7 +779,11 @@ check("...and a feed announcing deprecated=true says so WHILE IT STILL WORKS - "
|
||||
# sixth supplier cannot be added without this line noticing.
|
||||
_networked = sorted(pid for pid, cls in PROVIDERS.items()
|
||||
if cls.capabilities().egress_hosts)
|
||||
_replayed = ["generic_dayahead"]
|
||||
# ⚠️ TARIFF-15 PUT A SECOND PLUGIN ON THIS LIST AND IT IS REPLAYED FROM THE
|
||||
# SAME CAPTURED BYTES (section 13c), because it reads the same endpoint with
|
||||
# the same parser. A networked plugin with no capture is what this line exists
|
||||
# to forbid.
|
||||
_replayed = ["custom_formula", "generic_dayahead"]
|
||||
check(f"the networked plugins are {_networked}, and EVERY one of them is "
|
||||
f"replayed from a captured real body ({_replayed}) - no shipped plugin "
|
||||
f"reaches a network this file has never seen answer (TARIFF-07)",
|
||||
@@ -1012,9 +1051,9 @@ def _uses_range(cls):
|
||||
_uses = sorted(pid for pid, cls in PROVIDERS.items() if _uses_range(cls))
|
||||
check(f"the plugins that CONSUME their `since`/`until` are {_uses}; the rest "
|
||||
f"take the range and ignore it, which is what TARIFF-09 fixed for "
|
||||
f"generic_dayahead - and after TARIFF-07 the only one left that does not "
|
||||
f"is the offline plugin, for the honest reason below",
|
||||
_uses == ["generic_dayahead"])
|
||||
f"generic_dayahead - and after TARIFF-07 and TARIFF-15 the only one left "
|
||||
f"that does not is the offline plugin, for the honest reason below",
|
||||
_uses == ["custom_formula", "generic_dayahead"])
|
||||
check("...and `manual_csv` is honestly among the ones that do not: it reads a "
|
||||
"whole file and the fill windows it downstream, so a range parameter "
|
||||
"would have nowhere to go",
|
||||
@@ -1094,21 +1133,29 @@ check(f"a negative day-ahead price passes through, all-in {_neg}",
|
||||
# from it would mean feeding it an invented body carrying an invented flag,
|
||||
# which is exactly the closed loop TARIFF-08 exists to remove. It is graded on
|
||||
# what its real body can grade instead, immediately below.
|
||||
for pid in sorted(set(READS) - {"generic_dayahead"}):
|
||||
# ⚠️ `custom_formula` IS EXCLUDED FOR THE SAME REASON AND NOT FOR A NEW ONE
|
||||
# (TARIFF-15): it reads the SAME columnar document, which has no per-point
|
||||
# flag, and applies arithmetic to it - arithmetic cannot invent a provenance
|
||||
# the document never carried.
|
||||
_SETTLED_FEED = {"generic_dayahead", "custom_formula"}
|
||||
for pid in sorted(set(READS) - _SETTLED_FEED):
|
||||
read = READS[pid]
|
||||
flags = derived(lambda r=read: [p.provisional for p in r.points])
|
||||
check(f"{pid}: provisional is per point and mixed {flags}",
|
||||
isinstance(flags, list) and any(flags) and not all(flags))
|
||||
check(f"...so plan_is_provisional is derived from those points",
|
||||
derived(lambda r=read: plan_is_provisional(r)) is True)
|
||||
_gen_flags = derived(lambda: [p.provisional
|
||||
for p in READS["generic_dayahead"].points])
|
||||
check(f"generic_dayahead's settled auction comes out settled at every point "
|
||||
f"{_gen_flags}",
|
||||
isinstance(_gen_flags, list) and len(_gen_flags) == len(EPEX_EUR_MWH)
|
||||
and not any(_gen_flags))
|
||||
check("...so plan_is_provisional is False for it, derived from the same points",
|
||||
derived(lambda: plan_is_provisional(READS["generic_dayahead"])) is False)
|
||||
for pid in sorted(_SETTLED_FEED):
|
||||
_gen_flags = derived(lambda p=pid: [pt.provisional
|
||||
for pt in READS[p].points])
|
||||
check(f"{pid}'s settled auction comes out settled at every point "
|
||||
f"{_gen_flags}",
|
||||
isinstance(_gen_flags, list)
|
||||
and len(_gen_flags) == len(EPEX_EUR_MWH)
|
||||
and not any(_gen_flags))
|
||||
check(f"...so plan_is_provisional is False for {pid}, derived from the "
|
||||
f"same points",
|
||||
derived(lambda p=pid: plan_is_provisional(READS[p])) is False)
|
||||
# ...and the same function says False when nothing is provisional. Without this
|
||||
# arm, "always True" would pass every check above.
|
||||
_settled = PriceRead.series(
|
||||
@@ -3150,6 +3197,576 @@ check("...and it never invents a neighbour: the instant one second before the "
|
||||
for _d in _tmpdirs_t10:
|
||||
shutil.rmtree(_d, ignore_errors=True)
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
print()
|
||||
print("15. TARIFF-15 - the operator's own tariff card, as two coefficients")
|
||||
#
|
||||
# WHAT THIS SECTION IS DEFENDING. `custom_formula` lets an operator type the
|
||||
# `B x index + A` line off their own supplier's tariff card, because there is
|
||||
# no machine-readable source of Belgian tariff coefficients anywhere and the
|
||||
# customer is the only party who can see their own. That makes the card the
|
||||
# ONLY artefact that can grade them - which is exactly what the startup check
|
||||
# does, and what the checks below grade it doing.
|
||||
#
|
||||
# ⚠️ EVERY EXPECTATION HERE IS SOMEBODY ELSE'S PUBLISHED NUMBER. The two cards
|
||||
# print the formula, the reference index they assumed AND the resulting price,
|
||||
# so this file compares the SHIPPED code against 15,55 and 7,522 rather than
|
||||
# against its own restatement of the same arithmetic. That is section 2b's
|
||||
# rule - grade against something nobody here wrote - applied to arithmetic.
|
||||
|
||||
print()
|
||||
print("15a. the two REAL cards reproduce their PRINTED prices, through the "
|
||||
"shipped code")
|
||||
|
||||
# ⚠️ NON-DEGENERACY FIRST (habit 1). Two cards that agreed on every field
|
||||
# would be one card counted twice, and the pair is what says the arithmetic is
|
||||
# general rather than tuned to one supplier.
|
||||
check(f"the two cards really are two - factor {ENECO_CARD['factor']} vs "
|
||||
f"{ENGIE_CARD['factor']}, offset {ENECO_CARD['offset']} vs "
|
||||
f"{ENGIE_CARD['offset']}, index "
|
||||
f"{ENECO_CARD['card']['index_eur_per_mwh']} vs "
|
||||
f"{ENGIE_CARD['card']['index_eur_per_mwh']}",
|
||||
ENECO_CARD["factor"] != ENGIE_CARD["factor"]
|
||||
and ENECO_CARD["offset"] != ENGIE_CARD["offset"]
|
||||
and ENECO_CARD["card"]["index_eur_per_mwh"]
|
||||
!= ENGIE_CARD["card"]["index_eur_per_mwh"]
|
||||
and ENECO_CARD["card"]["expect_incl_vat"]
|
||||
!= ENGIE_CARD["card"]["expect_incl_vat"])
|
||||
|
||||
# The suppliers' own printed prices, transcribed from the cards and from
|
||||
# nowhere in app/.
|
||||
PRINTED = {"Eneco (0.102 x 133.9905 + 1) x 1.06": (ENECO_CARD, 15.55),
|
||||
"ENGIE (0.1 x 61.96 + 0.9) x 1.06": (ENGIE_CARD, 7.522)}
|
||||
for _what, (_card, _printed) in sorted(PRINTED.items()):
|
||||
_spec = derived(lambda c=_card: parse_price_formula(json.dumps(c)))
|
||||
_got = derived(lambda sp=_spec: in_card_units(
|
||||
sp, sp.card_index_eur_per_mwh))
|
||||
check(f"{_what} = {_got} c EUR/kWh through the shipped module, which "
|
||||
f"rounds to the {_printed} the card prints",
|
||||
isinstance(_got, float)
|
||||
and round(_got, len(str(_printed).split('.')[-1])) == _printed)
|
||||
|
||||
# ...and the same two cards end to end through `load_prices()`, i.e. through
|
||||
# the registered plugin, the columnar parser and `assemble()` - not through a
|
||||
# module function this file called directly. The feed body carries the card's
|
||||
# OWN index value, so the point that comes out must be the card's own printed
|
||||
# price in EUR/kWh plus the framework's taxes and nothing else.
|
||||
for _what, (_card, _printed) in sorted(PRINTED.items()):
|
||||
_idx = _card["card"]["index_eur_per_mwh"]
|
||||
_read = derived(lambda c=_card, i=_idx: load(
|
||||
"custom_formula", {"price_formula": json.dumps(c)},
|
||||
transport=canned(columnar_feed([i]))))
|
||||
_all_in = derived(lambda r=_read: r.points[0].eur_per_kwh)
|
||||
_want = round(_printed / 100.0 + TAXES, 6)
|
||||
check(f"{_what}: end to end through load_prices() the hour comes out at "
|
||||
f"{_all_in} EUR/kWh all-in, which is the card's printed "
|
||||
f"{_printed} c EUR/kWh plus price_taxes_eur_kwh and nothing else",
|
||||
isinstance(_all_in, float) and abs(_all_in - _want) < 5e-5)
|
||||
|
||||
print()
|
||||
print("15b. the card self-check REFUSES at startup, naming the discrepancy")
|
||||
# ⚠️ THIS IS THE WHOLE SAFETY ARGUMENT AND IT IS GRADED ON THE REASON, NOT THE
|
||||
# VERDICT (MUTATION.md, TEST-16). Several rules here can refuse the same blob -
|
||||
# a wrong `units`, a missing key, a bad number - so a check that only asked
|
||||
# "was it refused" would pass on a tree where the card comparison is deleted.
|
||||
|
||||
|
||||
def spoilt(**over):
|
||||
"""Eneco's card with one field replaced. Everything else is the real one."""
|
||||
c = json.loads(json.dumps(ENECO_CARD))
|
||||
for k, v in over.items():
|
||||
if k.startswith("card_"):
|
||||
c["card"][k[5:]] = v
|
||||
else:
|
||||
c[k] = v
|
||||
return c
|
||||
|
||||
|
||||
def refusal(blob):
|
||||
return _reason(lambda: parse_price_formula(json.dumps(blob)))
|
||||
|
||||
|
||||
_honest = refusal(ENECO_CARD)
|
||||
check("the CONTROL first: the real card is NOT refused, or every refusal "
|
||||
"below would be graded against a rule that refuses everything",
|
||||
_honest == "")
|
||||
|
||||
# ⚠️ THE EXPENSIVE ONE, AND THE REASON THIS PROVIDER IS SAFE TO SHIP. A
|
||||
# Belgian card's Tariefformule line is the ENERGY component alone; the add-on
|
||||
# adds price_taxes_eur_kwh on top. An operator who folds ~12 c EUR/kWh of
|
||||
# network charge into `offset` is billed for it twice, and `code_leaks()`
|
||||
# cannot see it because it is CONFIG, not code. The card is what catches it:
|
||||
# the printed estimate is the energy component, so the inflated offset cannot
|
||||
# reproduce it.
|
||||
_NETWORK_C_KWH = 12.0
|
||||
_double = refusal(spoilt(offset=ENECO_CARD["offset"] + _NETWORK_C_KWH))
|
||||
_double_price = derived(lambda: in_card_units(
|
||||
_cf.FormulaSpec(index="epex_be", resolution="hourly", factor=0.102,
|
||||
offset=ENECO_CARD["offset"] + _NETWORK_C_KWH,
|
||||
units="c_eur_per_kwh", vat_pct=6, card_month="2026-09",
|
||||
card_index_eur_per_mwh=133.9905,
|
||||
card_expect_incl_vat=15.55), 133.9905))
|
||||
check(f"network charges folded into `offset` move the card's own price to "
|
||||
f"{_double_price} c EUR/kWh against a printed 15.55, so it CANNOT pass "
|
||||
f"- the counterfactual first, or the refusal below grades nothing",
|
||||
isinstance(_double_price, float) and abs(_double_price - 15.55) > 10)
|
||||
check(f"...and it is refused, by the CARD rule, naming the discrepancy and "
|
||||
f"the network-charge cause ({_double[:60]!r})",
|
||||
"does not reproduce its own card" in _double and "OUT BY" in _double
|
||||
and "folded into `offset`" in _double)
|
||||
|
||||
_CORRUPTIONS = {
|
||||
"VAT left out (vat_pct 0)": spoilt(vat_pct=0),
|
||||
"VAT applied twice (the card's price already carries it)":
|
||||
spoilt(card_expect_incl_vat=round(15.55 * 1.06, 4)),
|
||||
"the coefficients in c EUR and the expected price in EUR":
|
||||
spoilt(card_expect_incl_vat=0.1555),
|
||||
"the index typed in EUR/kWh instead of EUR/MWh":
|
||||
spoilt(card_index_eur_per_mwh=0.1339905),
|
||||
"a transposed digit in the expected price (15.05 for 15.55)":
|
||||
spoilt(card_expect_incl_vat=15.05),
|
||||
"a transposed digit in the factor (0.120 for 0.102)":
|
||||
spoilt(factor=0.120),
|
||||
}
|
||||
for _name, _blob in sorted(_CORRUPTIONS.items()):
|
||||
_why = refusal(_blob)
|
||||
check(f"{_name}: refused, and by the CARD rule rather than by another one "
|
||||
f"({_why[:48]!r})",
|
||||
"does not reproduce its own card" in _why and "OUT BY" in _why)
|
||||
|
||||
# ⚠️ THE TOLERANCE IS A REAL EDGE AND IT IS GRADED FROM BOTH SIDES, or
|
||||
# "within 0.01" and "within anything" would be the same check.
|
||||
_inside = refusal(spoilt(card_expect_incl_vat=round(
|
||||
in_card_units(parse_price_formula(json.dumps(ENECO_CARD)), 133.9905)
|
||||
+ CARD_TOLERANCE * 0.5, 6)))
|
||||
_outside = refusal(spoilt(card_expect_incl_vat=round(
|
||||
in_card_units(parse_price_formula(json.dumps(ENECO_CARD)), 133.9905)
|
||||
+ CARD_TOLERANCE * 2.0, 6)))
|
||||
check(f"half a tolerance out passes and twice a tolerance out does not, so "
|
||||
f"{CARD_TOLERANCE} is the number being enforced rather than a shape",
|
||||
_inside == "" and "does not reproduce its own card" in _outside)
|
||||
|
||||
# A card that is absent is refused, and NOT by the generic unknown-field rule.
|
||||
check(f"a formula with NO card block is refused, and the message says why the "
|
||||
f"card is required rather than merely that a field is missing",
|
||||
"no \"card\" block" in refusal({k: v for k, v in ENECO_CARD.items()
|
||||
if k != "card"}))
|
||||
|
||||
print()
|
||||
print("15c. the index path is generic_dayahead's, replayed from the SAME "
|
||||
"captured bytes")
|
||||
# A networked plugin with no capture is what section 2b's roster forbids. This
|
||||
# one reads the same endpoint with the same parser, so it is replayed from the
|
||||
# same real bodies rather than from a fixture written here.
|
||||
_CF_OPTS = {"price_formula": json.dumps(ENECO_CARD)}
|
||||
_cf_replay = {n: load("custom_formula", _CF_OPTS, transport=canned(b))
|
||||
for n, b in _REAL.items()}
|
||||
check(f"the SHIPPED custom_formula parses all three captured real bodies "
|
||||
f"{[r.words for r in _cf_replay.values()]}",
|
||||
all(r.kind == "series" for r in _cf_replay.values()))
|
||||
_cf_counts = derived(lambda: {n: len(r.points)
|
||||
for n, r in _cf_replay.items()})
|
||||
check(f"...and emits one point per published instant at quarter_hourly "
|
||||
f"{_cf_counts}",
|
||||
_cf_counts == {n: len(json.loads(b.decode("utf-8"))["price"])
|
||||
for n, b in _REAL.items()})
|
||||
# The URL is the same one, derived from the same helper. Behaviour, off the
|
||||
# injected transport, not source text.
|
||||
_cf_t = canned(_REAL[_TD])
|
||||
run(load_prices(opts(price_provider="custom_formula", **_CF_OPTS),
|
||||
_w0, _w0 + timedelta(hours=48), transport=_cf_t))
|
||||
check(f"...and asks for the same Brussels range generic_dayahead asks for "
|
||||
f"{_cf_t.calls[0][0]!r}",
|
||||
_cf_t.calls[0][0] == asked_url(_w0, _w0 + timedelta(hours=48)))
|
||||
|
||||
# The price at each instant is the document's own index through the card's
|
||||
# arithmetic - computed HERE from the captured bytes, by different arithmetic.
|
||||
_doc = json.loads(_REAL[_QH].decode("utf-8"))
|
||||
_want_cf = [round((0.102 * v + 1.00) * 1.06 / 100.0 + TAXES, 6)
|
||||
for v in _doc["price"]]
|
||||
_got_cf = derived(lambda: [p.eur_per_kwh
|
||||
for p in _cf_replay[_QH].points])
|
||||
check(f"every one of the {len(_want_cf)} points is (0.102 x index + 1) x 1.06 "
|
||||
f"c EUR/kWh plus taxes, against the captured document's own index "
|
||||
f"column (first three {_got_cf[:3] if isinstance(_got_cf, list) else _got_cf})",
|
||||
_got_cf == _want_cf)
|
||||
# ...and the index really varies over that day, or one constant would grade the
|
||||
# whole column.
|
||||
check(f"...and the captured index really varies across the day "
|
||||
f"({min(_doc['price'])} .. {max(_doc['price'])} EUR/MWh), so that "
|
||||
f"comparison is 96 different numbers and not one repeated",
|
||||
len(set(_doc["price"])) > 50)
|
||||
|
||||
print()
|
||||
print("15d. resolution: the contract's series, not the feed's")
|
||||
# ⚠️ THIS IS NOT TARIFF-14's HOURLY ASSUMPTION COMING BACK. It decides how many
|
||||
# points are EMITTED; the window each one covers is still derived from its
|
||||
# neighbours by _cover_table(). Both halves are graded.
|
||||
_hourly_card = dict(ENECO_CARD, resolution="hourly")
|
||||
_h_read = load("custom_formula", {"price_formula": json.dumps(_hourly_card)},
|
||||
transport=canned(_REAL[_QH]))
|
||||
_q_read = _cf_replay[_QH]
|
||||
check(f"the same 96-point body is 24 points at `hourly` and 96 at "
|
||||
f"`quarter_hourly` ({len(_h_read.points)} vs {len(_q_read.points)}), so "
|
||||
f"the option is doing something",
|
||||
len(_h_read.points) == 24 and len(_q_read.points) == 96)
|
||||
# The hourly value is the MEAN of that hour's quarters, computed here from the
|
||||
# captured document rather than from the module.
|
||||
_by_hour = {}
|
||||
for _t, _v in zip(_doc["unix_seconds"], _doc["price"]):
|
||||
_by_hour.setdefault(
|
||||
datetime.fromtimestamp(_t, UTC).replace(minute=0), []).append(_v)
|
||||
_want_h = [round((0.102 * (sum(v) / len(v)) + 1.00) * 1.06 / 100.0 + TAXES, 6)
|
||||
for _k, v in sorted(_by_hour.items())]
|
||||
_got_h = derived(lambda: [p.eur_per_kwh for p in _h_read.points])
|
||||
check(f"...and each hourly point is the MEAN of its four quarters through the "
|
||||
f"card's arithmetic (first three {_got_h[:3] if isinstance(_got_h, list) else _got_h})",
|
||||
_got_h == _want_h)
|
||||
# ⚠️ NON-DEGENERACY: an hour whose four quarters were equal would make "the
|
||||
# mean" and "the first quarter" the same number, and the check above would
|
||||
# grade nothing.
|
||||
_differs = sum(1 for _k, v in _by_hour.items() if len(set(v)) > 1)
|
||||
check(f"...and {_differs} of the {len(_by_hour)} hours really have quarters "
|
||||
f"that DIFFER, so the mean is distinguishable from taking the first one",
|
||||
_differs >= 20)
|
||||
# An already-hourly feed averages one member per hour - the identity - so the
|
||||
# option cannot silently assume the granularity it was handed.
|
||||
_h1_pairs = [(datetime.fromtimestamp(t, UTC), v) for t, v in
|
||||
zip(json.loads(_REAL[_H1].decode("utf-8"))["unix_seconds"],
|
||||
json.loads(_REAL[_H1].decode("utf-8"))["price"])]
|
||||
check("an ALREADY hourly feed is the identity under `hourly`, so the option "
|
||||
"assumes nothing about the granularity it was handed",
|
||||
derived(lambda: to_resolution(_h1_pairs, "hourly")) == _h1_pairs)
|
||||
# and TARIFF-14: the window is still the neighbours', not an assumed hour.
|
||||
_h_pts = _h_read.points
|
||||
check("...and the hourly series' coverage is still DERIVED from the "
|
||||
"neighbours - at() answers 59 minutes into a point and the next point "
|
||||
"at 60, which is _cover_table()'s rule and not a rule this card added",
|
||||
derived(lambda: _h_read.at(_h_pts[0].start + timedelta(minutes=59))
|
||||
is _h_pts[0]) is True
|
||||
and derived(lambda: _h_read.at(_h_pts[0].start + timedelta(minutes=60))
|
||||
is _h_pts[1]) is True)
|
||||
|
||||
print()
|
||||
print("15e. every refusal in the table, and the two that are WARNINGS")
|
||||
_TABLE = {
|
||||
"units absent": (spoilt(units=""), "units", "is REQUIRED"),
|
||||
"units not one of the enumerated ones":
|
||||
(spoilt(units="cents"), "units", "must be one of"),
|
||||
"factor == 0 (every spread becomes 0, arbitrage never fires)":
|
||||
(spoilt(factor=0), "factor", "static_profile"),
|
||||
"factor < 0 (the dearest hour becomes the cheapest)":
|
||||
(spoilt(factor=-0.102), "factor", "inverts the market"),
|
||||
"vat_pct above 30 (the 6 / 1.06 / 106 confusion)":
|
||||
(spoilt(vat_pct=106), "vat_pct", "between 0 and 30"),
|
||||
"vat_pct below 0": (spoilt(vat_pct=-6), "vat_pct", "between 0 and 30"),
|
||||
"a non-numeric coefficient": (spoilt(factor="0.102"), "factor",
|
||||
"not a number"),
|
||||
"a coefficient that is an EXPRESSION rather than a number":
|
||||
(spoilt(factor="0.1*x"), "factor", "evaluates nothing"),
|
||||
"a non-finite coefficient": (spoilt(offset=float("inf")), "offset",
|
||||
"not finite"),
|
||||
"an unknown index": (spoilt(index="belpex_h"), "index", "cannot get it"),
|
||||
"an unknown resolution": (spoilt(resolution="daily"), "resolution",
|
||||
"must be one of"),
|
||||
"an unknown top-level field (which is how 'expr' is answered)":
|
||||
(spoilt(expr="0.102*spot+1"), "expr", "NO field that takes an "
|
||||
"expression"),
|
||||
"an unknown field inside the card": ({**ENECO_CARD,
|
||||
"card": dict(ENECO_CARD["card"],
|
||||
vat="6")},
|
||||
"vat", "no field(s)"),
|
||||
}
|
||||
for _name, (_blob, _names, _needle) in sorted(_TABLE.items()):
|
||||
_why = refusal(_blob)
|
||||
check(f"{_name}: refused, naming the field and the consequence "
|
||||
f"({_why[:48]!r})",
|
||||
_names in _why and _needle in _why)
|
||||
|
||||
# ⚠️ AND THE ARM THAT MAKES THOSE MEAN SOMETHING: a NaN is not merely refused
|
||||
# by "not a number", and `True` is an int in Python.
|
||||
check("NaN is refused as NOT FINITE rather than as not a number, so the two "
|
||||
"rungs are distinguishable",
|
||||
"not finite" in refusal(spoilt(factor=float("nan"))))
|
||||
check("...and a bool is refused as not a number, because `True` is an int "
|
||||
"here and would price the whole day at the index",
|
||||
"not a number" in refusal(spoilt(factor=True)))
|
||||
|
||||
# The plausibility band. ⚠️ IT IS A SEPARATE QUANTITY FROM THE CARD CHECK: a
|
||||
# unit error applied to BOTH sides cancels out of the card comparison and
|
||||
# cannot cancel out of this one.
|
||||
_both_sides = spoilt(factor=0.00102, offset=0.01,
|
||||
card_expect_incl_vat=0.1555)
|
||||
_bs_spec = derived(lambda: parse_price_formula(json.dumps(_both_sides)))
|
||||
check("a unit error applied to BOTH sides passes the card self-check - stated "
|
||||
"as a check, because it is the limit of that guard and not a bug in it",
|
||||
not isinstance(_bs_spec, _Raised)
|
||||
and abs(card_discrepancy(_bs_spec)) <= CARD_TOLERANCE)
|
||||
_band = derived(lambda: implausible_all_in(_bs_spec, 0.0))
|
||||
check(f"...and the all-in band is what catches it, naming the band "
|
||||
f"({str(_band)[:48]!r})",
|
||||
isinstance(_band, str) and "plausible band" in _band)
|
||||
check("...and the REAL card is inside the band, so the band is not simply "
|
||||
"refusing everything",
|
||||
implausible_all_in(parse_price_formula(json.dumps(ENECO_CARD)),
|
||||
0.06) is None)
|
||||
check(f"...and the band really is {PLAUSIBLE_LO_EUR_KWH}-"
|
||||
f"{PLAUSIBLE_HI_EUR_KWH} EUR/kWh at both ends, graded from inside and "
|
||||
f"outside each",
|
||||
implausible_all_in(parse_price_formula(json.dumps(ENECO_CARD)),
|
||||
PLAUSIBLE_HI_EUR_KWH) is not None
|
||||
and implausible_all_in(parse_price_formula(json.dumps(
|
||||
spoilt(factor=0.0001, offset=0.0009,
|
||||
card_expect_incl_vat=round((0.0001 * 133.9905 + 0.0009)
|
||||
* 1.06, 6)))), 0.0) is not None)
|
||||
|
||||
# ⚠️ A STALE CARD IS A WARNING AND MUST NEVER BECOME A REFUSAL: ENGIE freezes
|
||||
# a contract's coefficients for its duration, so a two-year-old card can be
|
||||
# exactly right and a site refused over one is a site that cannot start.
|
||||
_old_card = spoilt(card_month="2019-01")
|
||||
check("an OLD card is not refused - the parse succeeds",
|
||||
refusal(_old_card) == "")
|
||||
check("...and it is warned about, naming the month and saying why it is not "
|
||||
"refused",
|
||||
"2019-01" in (stale_card(parse_price_formula(json.dumps(_old_card)))
|
||||
or "")
|
||||
and "Not refused" in (stale_card(
|
||||
parse_price_formula(json.dumps(_old_card))) or ""))
|
||||
check("...while a card from this month draws no warning, so the rule is the "
|
||||
"AGE and not the presence of the field",
|
||||
stale_card(parse_price_formula(json.dumps(ENECO_CARD)),
|
||||
today=date(2026, 9, 13)) is None)
|
||||
|
||||
print()
|
||||
print("15f. NO EXPRESSION EVALUATION OF ANY KIND - and the check that says so")
|
||||
# WHY THIS IS A CHECK AND NOT A PROMISE. This string is settable over HTTP: the
|
||||
# add-on serves `POST config/save` -> `POST /addons/self/options` on its own
|
||||
# ingress page, so `eval` on it is remote code execution in a root container
|
||||
# holding SUPERVISOR_TOKEN, with homeassistant_api and hassio_api, driving an
|
||||
# inverter over RS485. And the deciding argument is not security: two
|
||||
# coefficients can be GRADED against the card's printed price and an arbitrary
|
||||
# expression cannot be graded against anything, so an expression would make
|
||||
# the invisible-wrong-coefficient problem strictly worse.
|
||||
_EVAL_NAMES = ("eval", "exec", "compile", "__import__", "literal_eval",
|
||||
"eval_expr", "evaluate", "Interpreter", "simple_eval")
|
||||
|
||||
|
||||
def eval_calls(source):
|
||||
"""Every call in `source` whose target NAME is an evaluator. AST, not grep.
|
||||
|
||||
⚠️ IT WALKS THE TREE RATHER THAN MATCHING TEXT, so `eval` inside a comment
|
||||
or a docstring - and this file's own prose about eval - is not a hit, and
|
||||
`ast.literal_eval(...)` written as `literal_eval(...)` is.
|
||||
"""
|
||||
found = []
|
||||
for node in ast.walk(ast.parse(source)):
|
||||
if not isinstance(node, ast.Call):
|
||||
continue
|
||||
fn = node.func
|
||||
name = (fn.id if isinstance(fn, ast.Name)
|
||||
else fn.attr if isinstance(fn, ast.Attribute) else "")
|
||||
if name in _EVAL_NAMES:
|
||||
found.append(name)
|
||||
return sorted(set(found))
|
||||
|
||||
|
||||
_CF_SRC = open(os.path.join(HERE, "app", "custom_formula.py"),
|
||||
encoding="utf-8").read()
|
||||
check(f"app/custom_formula.py evaluates NOTHING: {eval_calls(_CF_SRC) or '[]'}",
|
||||
eval_calls(_CF_SRC) == [])
|
||||
check("...and neither does the plugin class that reads it",
|
||||
eval_calls(dedent(inspect.getsource(PROVIDERS["custom_formula"]))) == [])
|
||||
# ⚠️ THE RED DIRECTION, EXERCISED ON EVERY RUN. A scanner that found nothing
|
||||
# because it scans nothing is the third failure mode in MUTATION.md, and this
|
||||
# is the cheap way to keep it honest.
|
||||
check("...and the scanner CAN go red: planted `eval` and `literal_eval` calls "
|
||||
"are both found",
|
||||
eval_calls("def f(s):\n return eval(s)\n") == ["eval"]
|
||||
and eval_calls("import ast\ndef f(s):\n"
|
||||
" return ast.literal_eval(s)\n") == ["literal_eval"])
|
||||
# and the structural half: the coefficients are NUMBERS, so there is nothing
|
||||
# for an evaluator to be reached from even if one appeared.
|
||||
check("the spec carries only fixed scalars - no field holds a string that is "
|
||||
"later interpreted",
|
||||
derived(lambda: sorted(
|
||||
type(v).__name__ for v in vars(parse_price_formula(
|
||||
json.dumps(ENECO_CARD))).values())) ==
|
||||
["float"] * 5 + ["str"] * 4)
|
||||
check("...and a field that LOOKS like an expression is refused as not a "
|
||||
"number rather than parsed as one",
|
||||
"not a number" in refusal(spoilt(factor="0.102 * spot")))
|
||||
|
||||
print()
|
||||
print("15g. it declares {ENERGY, MARKUP}, so the framework adds taxes ONCE")
|
||||
_cf_caps = PROVIDERS["custom_formula"].capabilities()
|
||||
check(f"custom_formula sources {sorted(c.value for c in _cf_caps.sources)} - "
|
||||
f"a Belgian card's Tariefformule line is the ENERGY component and the "
|
||||
f"contract's own markup, and nothing else",
|
||||
_cf_caps.sources == frozenset({Component.ENERGY, Component.MARKUP}))
|
||||
check(f"...so the framework adds exactly "
|
||||
f"{sorted(c.value for c in _cf_caps.adds)}",
|
||||
_cf_caps.adds == frozenset({Component.TAXES}))
|
||||
# THE COST OF THE ALTERNATIVE, MEASURED, so the declaration is not grading
|
||||
# itself. Declaring ALL_IN would drop the taxes silently - on every hour, on
|
||||
# behalf of an operator who typed exactly what their card says.
|
||||
_as_declared = derived(lambda: assemble(
|
||||
_cf_caps, [RawPoint(T0, 0.1555, False)], MARKUP, TAXES)[0].eur_per_kwh)
|
||||
_as_all_in = derived(lambda: assemble(
|
||||
ProviderCapabilities(product=PriceProduct.DAY_AHEAD, sources=ALL_IN),
|
||||
[RawPoint(T0, 0.1555, False)], MARKUP, TAXES)[0].eur_per_kwh)
|
||||
check(f"declaring ALL_IN would price every hour {_as_declared} -> "
|
||||
f"{_as_all_in} EUR/kWh, exactly one taxes component low, on behalf of "
|
||||
f"an operator who typed what their card said",
|
||||
isinstance(_as_declared, float) and isinstance(_as_all_in, float)
|
||||
and abs((_as_declared - _as_all_in) - TAXES) < 1e-9)
|
||||
check("and the configured markup is reported as IGNORED under it, because the "
|
||||
"contract's formula already IS the markup",
|
||||
(lambda: (lambda b, h: (
|
||||
logging.getLogger("goodwe.price").addHandler(h),
|
||||
load_any("custom_formula"),
|
||||
logging.getLogger("goodwe.price").removeHandler(h),
|
||||
"custom_formula" in b.getvalue()
|
||||
and "double-count" in b.getvalue())[-1])(
|
||||
*(lambda b: (b, logging.StreamHandler(b)))(io.StringIO())))())
|
||||
|
||||
print()
|
||||
print("15h. the startup preview - a factor of 1000 visible in one line")
|
||||
_prev_buf = io.StringIO()
|
||||
_prev_h = logging.StreamHandler(_prev_buf)
|
||||
_prev_log = logging.getLogger("goodwe.price")
|
||||
_prev_level = _prev_log.level
|
||||
# ⚠️ THE PREVIEW IS AT INFO AND THE DEFAULT ROOT LEVEL IS WARNING, so a check
|
||||
# that forgot this line would grade an empty buffer and read as "the preview
|
||||
# is missing". main.py runs the add-on at INFO - that is where `tariff
|
||||
# refreshed: %s` is emitted from - so this makes the check's logger match the
|
||||
# shipped one rather than relaxing anything.
|
||||
_prev_log.setLevel(logging.INFO)
|
||||
_prev_log.addHandler(_prev_h)
|
||||
_price._PREVIEWED.discard("custom_formula")
|
||||
load("custom_formula", _CF_OPTS, transport=canned(_REAL[_QH]))
|
||||
_first_run = _prev_buf.getvalue()
|
||||
load("custom_formula", _CF_OPTS, transport=canned(_REAL[_QH]))
|
||||
_second_run = _prev_buf.getvalue()[len(_first_run):]
|
||||
_prev_log.removeHandler(_prev_h)
|
||||
_prev_log.setLevel(_prev_level)
|
||||
check("the preview says it prints the index in EUR/MWh beside the all-in "
|
||||
"EUR/kWh",
|
||||
"index EUR/MWh -> all-in EUR/kWh" in _first_run)
|
||||
# ⚠️ AND THE ROW, NOT ONLY THE HEADING. The check above was the whole of this
|
||||
# for one draft, and deleting the index column from the ROW was a -13 B
|
||||
# SURVIVOR of the entire suite: the sentence stayed true while the thing it
|
||||
# describes was gone. MUTATION.md's third failure mode, in the check written
|
||||
# to close it. The expectation is built here from the captured document.
|
||||
_row_index = _doc["price"][0]
|
||||
_row_price = (0.102 * _row_index + 1.00) * 1.06 / 100.0 + TAXES
|
||||
_want_row = f"{_row_index:9.3f} -> {_row_price:9.5f}"
|
||||
check(f"...and each ROW really carries the index BESIDE its own price "
|
||||
f"({_want_row!r}), which is the pair a heading alone does not print",
|
||||
_want_row in _first_run)
|
||||
check("...and the min/median/max and the day's spread, in one line",
|
||||
"min " in _first_run and "median " in _first_run
|
||||
and "max " in _first_run and "spread " in _first_run)
|
||||
_want_min = round(min((0.102 * v + 1.00) * 1.06 / 100.0 + TAXES
|
||||
for v in _doc["price"][:96]), 5)
|
||||
check(f"...and the figures are the SERIES' own, not a restatement: the "
|
||||
f"minimum printed is {_want_min:.5f}, computed here from the captured "
|
||||
f"document",
|
||||
f"min {_want_min:.5f}" in _first_run)
|
||||
check("...and it is emitted ONCE per process, not on every half-hourly "
|
||||
"refresh - a log line must not become a log flood on the one path "
|
||||
"§11.4 already makes hard to read",
|
||||
"index EUR/MWh -> all-in EUR/kWh" not in _second_run)
|
||||
# ⚠️ AND IT CANNOT STOP A BATTERY. A preview that raises is a log line that
|
||||
# failed, not a fetch that failed.
|
||||
class _BoomPreview(PriceProvider):
|
||||
"""A plugin whose preview raises. AC: prices are unaffected.
|
||||
|
||||
SOURCES: energy
|
||||
The framework ADDS: markup, taxes
|
||||
"""
|
||||
|
||||
id = "_preview_boom"
|
||||
|
||||
@classmethod
|
||||
def capabilities(cls):
|
||||
return ProviderCapabilities(product=PriceProduct.DAY_AHEAD,
|
||||
sources=frozenset({Component.ENERGY}))
|
||||
|
||||
async def fetch(self, opts, since, until, transport):
|
||||
return [RawPoint(T0, 0.1, False)]
|
||||
|
||||
def startup_preview(self, points):
|
||||
raise RuntimeError("the preview is broken")
|
||||
|
||||
|
||||
PROVIDERS["_preview_boom"] = _BoomPreview
|
||||
_boom_read = derived(lambda: load("_preview_boom", transport=refuse_transport))
|
||||
del PROVIDERS["_preview_boom"]
|
||||
check("a preview that RAISES does not cost the site its prices - the series "
|
||||
"still comes back",
|
||||
not isinstance(_boom_read, _Raised) and _boom_read.kind == "series")
|
||||
|
||||
print()
|
||||
print("15i. the startup gate refuses it, so nothing is discovered at 03:00")
|
||||
check("custom_formula with a correct card starts",
|
||||
validate_startup_config(vbase(price_provider="custom_formula",
|
||||
price_formula=json.dumps(ENECO_CARD)),
|
||||
-5000, 5000) == [])
|
||||
check("custom_formula with NO formula is refused, naming the option and "
|
||||
"showing the shape",
|
||||
why(validate_startup_config(vbase(price_provider="custom_formula"),
|
||||
-5000, 5000),
|
||||
"price_formula is empty"))
|
||||
check("a formula that does not reproduce its card is refused AT STARTUP, "
|
||||
"with the discrepancy named",
|
||||
why(validate_startup_config(
|
||||
vbase(price_provider="custom_formula",
|
||||
price_formula=json.dumps(spoilt(offset=13.0))),
|
||||
-5000, 5000), "does not reproduce its own card"))
|
||||
check("...and a formula that is not JSON at all is refused with the shape it "
|
||||
"wanted, rather than with a traceback",
|
||||
why(validate_startup_config(
|
||||
vbase(price_provider="custom_formula", price_formula="0.102*spot+1"),
|
||||
-5000, 5000), "not valid JSON"))
|
||||
check("...and a card outside the plausible band is refused too, WITH THE "
|
||||
"SHIPPED taxes in place - the flat 0.06 must not be able to carry a "
|
||||
"hundredfold error over the floor",
|
||||
why(validate_startup_config(
|
||||
vbase(price_provider="custom_formula",
|
||||
price_formula=json.dumps(_both_sides)),
|
||||
-5000, 5000), "plausible band"))
|
||||
_masked = implausible_all_in(_bs_spec,
|
||||
PRICE_OPTION_DEFAULTS["price_taxes_eur_kwh"])
|
||||
check(f"...and that is a REAL trap, measured: the same card plus the shipped "
|
||||
f"{PRICE_OPTION_DEFAULTS['price_taxes_eur_kwh']} of taxes totals "
|
||||
f"{price_eur_kwh(_bs_spec, _bs_spec.card_index_eur_per_mwh) + PRICE_OPTION_DEFAULTS['price_taxes_eur_kwh']:.5f} "
|
||||
f"EUR/kWh, which is INSIDE the band - a floor applied to the all-in "
|
||||
f"figure would have passed it",
|
||||
isinstance(_masked, str)
|
||||
and PLAUSIBLE_LO_EUR_KWH <= (
|
||||
price_eur_kwh(_bs_spec, _bs_spec.card_index_eur_per_mwh)
|
||||
+ PRICE_OPTION_DEFAULTS["price_taxes_eur_kwh"])
|
||||
<= PLAUSIBLE_HI_EUR_KWH)
|
||||
# ⚠️ AND THE ONE ROW OF THE TABLE THAT IS A WARNING RATHER THAN A REFUSAL,
|
||||
# graded AT THE GATE and not only at the parser - which is where the two could
|
||||
# quietly become the same thing.
|
||||
check("...while a card older than six months still STARTS: ENGIE freezes a "
|
||||
"contract's coefficients for its duration, so an old card is often "
|
||||
"exactly right and a site refused over one is a site that cannot run",
|
||||
validate_startup_config(
|
||||
vbase(price_provider="custom_formula",
|
||||
price_formula=json.dumps(_old_card)), -5000, 5000) == [])
|
||||
|
||||
# ⚠️ THE ID IS IN THE SCHEMA, or the Supervisor refuses the options blob before
|
||||
# the add-on runs and none of the sentences above is ever reached.
|
||||
check("the id is offered by config.yaml's schema, so a site configured on it "
|
||||
"reaches these messages rather than a Supervisor schema error",
|
||||
"custom_formula" in _schema_ids)
|
||||
|
||||
print()
|
||||
if fails:
|
||||
print(f"{len(fails)} FAILED: " + "; ".join(fails))
|
||||
|
||||
@@ -1101,19 +1101,27 @@ _cfg = derived(lambda: open(os.path.join(HERE, "config.yaml"),
|
||||
# why, as this line does.
|
||||
_OPT04_KEYS = ["arbitrage_enabled", "cycle_cost_threshold_eur",
|
||||
"arbitrage_soc_floor", "arbitrage_soc_ceiling"]
|
||||
check(f"the in-code mirror holds exactly TARIFF-02's, TARIFF-03's and OPT-04's "
|
||||
f"keys - FCST-03 still added none of its own {_defaults}",
|
||||
# TARIFF-15 added the fifth, and came here to say so as this comment instructs:
|
||||
# `price_formula` is the `custom_formula` provider's only option - one JSON
|
||||
# string carrying one contract's coefficients - and it lives in this dict and
|
||||
# in config.yaml and nowhere else, which is the arrangement `static_profile`
|
||||
# already has for the same reason.
|
||||
_T15_KEYS = ["price_formula"]
|
||||
check(f"the in-code mirror holds exactly TARIFF-02's, TARIFF-03's, OPT-04's "
|
||||
f"and TARIFF-15's keys - FCST-03 still added none of its own "
|
||||
f"{_defaults}",
|
||||
_defaults == sorted(["price_provider", "price_markup_eur_kwh",
|
||||
"price_taxes_eur_kwh", "export_price_eur_kwh",
|
||||
"price_csv_path", "price_feed_url",
|
||||
"price_horizon_h", "price_timeout_s",
|
||||
"static_profile"] + _OPT04_KEYS))
|
||||
# ...and the four are genuinely NEW, not a rename of something that was already
|
||||
# there, or the line above would pass while the mirror silently lost a key.
|
||||
check("...and OPT-04's four keys are additions rather than replacements: the "
|
||||
"nine TARIFF-02/03 keys are all still present",
|
||||
"static_profile"] + _OPT04_KEYS + _T15_KEYS))
|
||||
# ...and the later keys are genuinely NEW, not a rename of something that was
|
||||
# already there, or the line above would pass while the mirror silently lost a
|
||||
# key.
|
||||
check("...and OPT-04's four and TARIFF-15's one are additions rather than "
|
||||
"replacements: the nine TARIFF-02/03 keys are all still present",
|
||||
isinstance(_defaults, list)
|
||||
and len(set(_defaults) - set(_OPT04_KEYS)) == 9)
|
||||
and len(set(_defaults) - set(_OPT04_KEYS) - set(_T15_KEYS)) == 9)
|
||||
check("...and every one of them is still in config.yaml, which stays the "
|
||||
"source of truth",
|
||||
isinstance(_cfg, str)
|
||||
|
||||
@@ -362,7 +362,7 @@ print("\n7. no behaviour and no value moved")
|
||||
# test_peakguard.py's. This line is the cheap one that says the reorder did not
|
||||
# lose or duplicate a key on the way.
|
||||
check("no option was lost, gained or duplicated by the reordering: the two "
|
||||
"halves of config.yaml hold the same 71 keys, once each",
|
||||
"halves of config.yaml hold the same 72 keys, once each",
|
||||
sorted(_opt_order) == sorted(ORDER) and len(set(ORDER)) == len(ORDER))
|
||||
check("...and `inverters: []` is still shipped in the options half, which is "
|
||||
"what Supervisor 2026.08.0 refuses every pre-0.6.0 options blob without "
|
||||
|
||||
@@ -3,8 +3,8 @@
|
||||
# ⚠️ WHAT THIS FILE IS FOR, AND WHAT HOME ASSISTANT ACTUALLY DOES WITH IT.
|
||||
# Without a `translations/` directory the Configuration tab renders the raw
|
||||
# option KEYS - `unit_sign_disagree_w`, `saturation_cycles`, `integrator_max_w`.
|
||||
# The frontend keeps a field when `e.name in options || e.required`, and all 71
|
||||
# keys are in `options:` (59 of them are `required` as well), so ALL SEVENTY-ONE are
|
||||
# The frontend keeps a field when `e.name in options || e.required`, and all 72
|
||||
# keys are in `options:` (59 of them are `required` as well), so ALL SEVENTY-TWO are
|
||||
# rendered, always, in `schema:` order, with no description. This file is what
|
||||
# turns that into sentences.
|
||||
#
|
||||
@@ -143,7 +143,7 @@ configuration:
|
||||
description: "DERIVED, not picked (SAFETY-24): the reference pack holds 13.8 kWh usable and the worst modelled peak-shaving day asks for 878 Wh, which is 6.4 %, so 20 % sits ten points above the pack's own cut-out and still leaves the guard about 1.6 of its worst observed days of room. ⚠️ IT BOUNDS THE PEAK GUARD ONLY. It is not a reserve for the whole pack: ordinary self-consumption discharges straight past it and goes on down to the battery cut-out, and raising this number does not change that by a tenth of a percent. It must stay strictly above the battery cut-out or the add-on refuses to start. Do not change it without re-deriving it."
|
||||
price_provider:
|
||||
name: "Tariff · Dynamic price source"
|
||||
description: "Where day-ahead or real-time prices come from. 'off' is what ships, and means this site runs on the static tariff below with no trading. ⚠️ SWITCHING THIS ON IS WHAT STARTS THE PRICE FETCH - not the trading switch further down. With a source selected the add-on fetches the feed every half hour, checks it, records the day and shows what it holds on the Price history row, whether or not price trading is armed. On its own it still moves no energy: trading is the separate switch, and the cheap window the monthly maintenance cycle uses comes from the fixed or day/night tariff below. It may not be combined with the two legacy Home Assistant price entities further down: two price sources on one site, one of which silently wins, is refused at startup rather than left looking armed. ⚠️ THREE IDS IN THIS LIST ARE WITHDRAWN AND WILL NOT START: engie_dynamic, totalenergies_dynamic and eneco_eddy. Their endpoints do not exist and never did, so those plugins never once fetched a price - a site set to one of them has silently been running on the fixed or day/night tariff below, or on nothing, since the day it was set. They are still offered here only so that such a site gets that explanation at startup instead of an unreadable schema error. Choose generic_dayahead, manual_csv or off."
|
||||
description: "Where day-ahead or real-time prices come from. 'off' is what ships, and means this site runs on the static tariff below with no trading. ⚠️ SWITCHING THIS ON IS WHAT STARTS THE PRICE FETCH - not the trading switch further down. With a source selected the add-on fetches the feed every half hour, checks it, records the day and shows what it holds on the Price history row, whether or not price trading is armed. On its own it still moves no energy: trading is the separate switch, and the cheap window the monthly maintenance cycle uses comes from the fixed or day/night tariff below. It may not be combined with the two legacy Home Assistant price entities further down: two price sources on one site, one of which silently wins, is refused at startup rather than left looking armed. ⚠️ THREE IDS IN THIS LIST ARE WITHDRAWN AND WILL NOT START: engie_dynamic, totalenergies_dynamic and eneco_eddy. Their endpoints do not exist and never did, so those plugins never once fetched a price - a site set to one of them has silently been running on the fixed or day/night tariff below, or on nothing, since the day it was set. They are still offered here only so that such a site gets that explanation at startup instead of an unreadable schema error. Choose generic_dayahead for the wholesale curve with a flat markup, custom_formula to enter your own contract's B x index + A coefficients off your tariff card, manual_csv for a file you write yourself, or off."
|
||||
price_markup_eur_kwh:
|
||||
name: "Tariff · Supplier markup (EUR/kWh)"
|
||||
description: "Your supplier's margin. The add-on adds it ONLY when the chosen price source does not already include it - under manual_csv, whose figures are all-in, it is ignored, and logged as ignored, never applied twice."
|
||||
@@ -156,6 +156,9 @@ configuration:
|
||||
static_profile:
|
||||
name: "Tariff · Fixed or day/night tariff (one line of JSON)"
|
||||
description: "Your fixed or day/night contract, written as a single line of JSON. RECOMMENDED EVEN ON A DYNAMIC CONTRACT: this is what the add-on falls back to when the price feed dies, and with it empty a dead feed leaves the site with no tariff at all. The periods must cover every day type exactly once, with no gap and no overlap - a gap is refused at startup rather than filled in, because an invented price is one a strategy would trade on. A fully fixed tariff is one period covering 00:00-24:00 on all days. Prices are ALL IN; the markup and levies above are not added to them. Note that a window written 22:00-07:00 for mon-fri does NOT cover Saturday morning, so a weekend row is required rather than decorative."
|
||||
price_formula:
|
||||
name: "Tariff · Your own tariff formula (one line of JSON)"
|
||||
description: "For price source custom_formula only: your own contract's two coefficients, written as a single line of JSON. Your supplier's tariff card prints the formula as B x index + A - those two numbers exist only on your card, there is no public index of them anywhere, and you are the only person who can read them. Type what the card prints, IN THE UNITS THE CARD PRINTS: every unit conversion and the VAT step are the add-on's job, because those are the two places a human silently loses a factor of 10, 100 or 1.06. Example: {\"index\":\"epex_be\",\"resolution\":\"hourly\",\"factor\":0.102,\"offset\":1.00,\"units\":\"c_eur_per_kwh\",\"vat_pct\":6,\"card\":{\"month\":\"2026-09\",\"index_eur_per_mwh\":133.9905,\"expect_incl_vat\":15.55}}. ⚠️ THE card BLOCK IS REQUIRED AND IT IS WHAT MAKES THIS SAFE: your card also prints the reference index it assumed and the price that comes out, so the add-on recomputes that price from your numbers and REFUSES TO START if it does not come out, telling you by how much. That catches a wrong unit, VAT left out or counted twice, a transposed digit, and the expensive one - network charges folded into the offset, which the add-on would then bill you for a second time. ⚠️ AND ITS LIMIT, IN THE SAME BREATH: it checks your coefficients against your card, not your card against reality, so two numbers copied off the wrong product agree with each other and pass. Fixed numbers only - no expression of any kind is evaluated, and an unrecognised field is refused rather than ignored."
|
||||
price_csv_path:
|
||||
name: "Tariff · Price CSV file"
|
||||
description: "For price source manual_csv only: where in /share the add-on reads your own price file from."
|
||||
|
||||
@@ -123,7 +123,7 @@ configuration:
|
||||
description: "AFGELEID, niet gekozen (SAFETY-24): de referentiebatterij heeft 13,8 kWh bruikbaar en de zwaarste doorgerekende afvlakdag vraagt 878 Wh, oftewel 6,4 %, dus 20 % ligt tien punten boven de uitschakelgrens van de batterij zelf en laat de bewaking nog ongeveer 1,6 van haar zwaarste waargenomen dagen speling. ⚠️ DIT BEGRENST ALLEEN DE PIEKBEWAKING. Het is geen reserve voor de hele batterij: gewoon eigenverbruik ontlaadt er dwars doorheen en gaat door tot aan de uitschakelgrens van de batterij, en dit getal verhogen verandert daar geen tiende procent aan. Het moet strikt boven de uitschakelgrens van de batterij blijven, anders start de add-on niet op. Verander het niet zonder de afleiding opnieuw te maken."
|
||||
price_provider:
|
||||
name: "Tarief · Bron van dynamische prijzen"
|
||||
description: "Waar day-ahead- of realtimeprijzen vandaan komen. 'off' is de standaardinstelling en betekent dat deze installatie op het vaste tarief hieronder draait, zonder handel. ⚠️ DIT AANZETTEN IS WAT HET OPHALEN VAN PRIJZEN START - niet de handelsschakelaar verderop. Met een bron ingesteld haalt de add-on de feed elk half uur op, controleert die, bewaart de dag en toont op de rij Prijsgeschiedenis wat er bewaard is, of prijshandel nu aan staat of niet. Op zichzelf verplaatst dat nog geen energie: handelen is de aparte schakelaar, en het goedkope venster dat de maandelijkse onderhoudscyclus gebruikt komt uit het vaste of dag/nacht-tarief hieronder. Het mag niet gecombineerd worden met de twee oude Home Assistant-prijsentiteiten verderop: twee prijsbronnen op één installatie, waarvan er stilzwijgend één wint, wordt bij het opstarten geweigerd in plaats van schijnbaar actief gelaten. ⚠️ DRIE IDS IN DEZE LIJST ZIJN INGETROKKEN EN STARTEN NIET MEER: engie_dynamic, totalenergies_dynamic en eneco_eddy. Hun endpoints bestaan niet en hebben nooit bestaan, dus die plugins hebben nooit één prijs opgehaald - een installatie die op zo'n bron stond, draait sindsdien stilzwijgend op het vaste of dag/nacht-tarief hieronder, of op niets. Ze staan hier alleen nog zodat zo'n installatie bij het opstarten die uitleg krijgt in plaats van een onleesbare schemafout. Kies generic_dayahead, manual_csv of off."
|
||||
description: "Waar day-ahead- of realtimeprijzen vandaan komen. 'off' is de standaardinstelling en betekent dat deze installatie op het vaste tarief hieronder draait, zonder handel. ⚠️ DIT AANZETTEN IS WAT HET OPHALEN VAN PRIJZEN START - niet de handelsschakelaar verderop. Met een bron ingesteld haalt de add-on de feed elk half uur op, controleert die, bewaart de dag en toont op de rij Prijsgeschiedenis wat er bewaard is, of prijshandel nu aan staat of niet. Op zichzelf verplaatst dat nog geen energie: handelen is de aparte schakelaar, en het goedkope venster dat de maandelijkse onderhoudscyclus gebruikt komt uit het vaste of dag/nacht-tarief hieronder. Het mag niet gecombineerd worden met de twee oude Home Assistant-prijsentiteiten verderop: twee prijsbronnen op één installatie, waarvan er stilzwijgend één wint, wordt bij het opstarten geweigerd in plaats van schijnbaar actief gelaten. ⚠️ DRIE IDS IN DEZE LIJST ZIJN INGETROKKEN EN STARTEN NIET MEER: engie_dynamic, totalenergies_dynamic en eneco_eddy. Hun endpoints bestaan niet en hebben nooit bestaan, dus die plugins hebben nooit één prijs opgehaald - een installatie die op zo'n bron stond, draait sindsdien stilzwijgend op het vaste of dag/nacht-tarief hieronder, of op niets. Ze staan hier alleen nog zodat zo'n installatie bij het opstarten die uitleg krijgt in plaats van een onleesbare schemafout. Kies generic_dayahead voor de groothandelscurve met een vaste marge, custom_formula om de coëfficiënten B x index + A van uw eigen contract van uw tariefkaart over te typen, manual_csv voor een bestand dat u zelf schrijft, of off."
|
||||
price_markup_eur_kwh:
|
||||
name: "Tarief · Marge van de leverancier (EUR/kWh)"
|
||||
description: "De marge van uw energieleverancier. De add-on telt die ALLEEN op wanneer de gekozen prijsbron ze niet al bevat - bij manual_csv, waarvan de bedragen alles inbegrepen zijn, wordt ze genegeerd, en dat wordt ook gelogd, zodat ze nooit dubbel wordt aangerekend."
|
||||
@@ -136,6 +136,9 @@ configuration:
|
||||
static_profile:
|
||||
name: "Tarief · Vast of dag/nacht-tarief (één regel JSON)"
|
||||
description: "Uw vaste of dag/nacht-contract, geschreven als één regel JSON. AANBEVOLEN ZELFS BIJ EEN DYNAMISCH CONTRACT: hierop valt de add-on terug wanneer de prijsfeed uitvalt, en staat dit leeg dan blijft de installatie bij een dode feed helemaal zonder tarief achter. De periodes moeten elk dagtype precies één keer bedekken, zonder gat en zonder overlap - een gat wordt bij het opstarten geweigerd in plaats van opgevuld, want een verzonnen prijs is er één waarop een strategie zou handelen. Een volledig vast tarief is één periode van 00:00 tot 24:00 op alle dagen. De prijzen zijn ALLES INBEGREPEN; de marge en heffingen hierboven worden er niet bij opgeteld. Let op: een venster van 22:00 tot 07:00 voor ma-vr dekt zaterdagochtend NIET, dus een regel voor het weekend is noodzakelijk en niet decoratief."
|
||||
price_formula:
|
||||
name: "Tarief · Uw eigen tariefformule (één regel JSON)"
|
||||
description: "Alleen voor prijsbron custom_formula: de twee coëfficiënten van uw eigen contract, als één regel JSON. De tariefkaart van uw leverancier drukt de formule af als B x index + A - die twee getallen staan alleen op uw kaart, er bestaat nergens een publieke lijst van, en u bent de enige die ze kan lezen. Typ wat de kaart afdrukt, IN DE EENHEID DIE DE KAART GEBRUIKT: elke eenheidsomrekening en de btw-stap zijn het werk van de add-on, want dat zijn precies de twee plaatsen waar een mens stilzwijgend een factor 10, 100 of 1,06 verliest. Voorbeeld: {\"index\":\"epex_be\",\"resolution\":\"hourly\",\"factor\":0.102,\"offset\":1.00,\"units\":\"c_eur_per_kwh\",\"vat_pct\":6,\"card\":{\"month\":\"2026-09\",\"index_eur_per_mwh\":133.9905,\"expect_incl_vat\":15.55}}. ⚠️ HET BLOK card IS VERPLICHT EN MAAKT DIT VEILIG: uw kaart vermeldt ook de referentie-index die ze veronderstelt en de prijs die eruit volgt, dus de add-on berekent die prijs opnieuw uit uw getallen en START NIET als het resultaat niet klopt - met vermelding van het verschil. Dat vangt een verkeerde eenheid, btw die ontbreekt of dubbel geteld is, een omgewisseld cijfer, en de dure fout: nettarieven die in de offset verwerkt zitten en die de add-on er dan nog een tweede keer bovenop zou rekenen. ⚠️ EN DE GRENS DAARVAN, IN DEZELFDE ZIN: ze toetst uw coëfficiënten aan uw kaart, niet uw kaart aan de werkelijkheid - twee getallen van het verkeerde product kloppen onderling en komen er dus door. Alleen vaste getallen: er wordt geen enkele expressie geëvalueerd, en een onbekend veld wordt geweigerd in plaats van genegeerd."
|
||||
price_csv_path:
|
||||
name: "Tarief · CSV-bestand met prijzen"
|
||||
description: "Alleen voor prijsbron manual_csv: waar in /share de add-on uw eigen prijsbestand leest."
|
||||
|
||||
Reference in New Issue
Block a user