Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f620f7cf80 | ||
|
|
8d09b30f16 | ||
|
|
028c3e53fb | ||
|
|
a6e64fb66e | ||
|
|
435e1b6f7c | ||
|
|
7f2c4e20e1 |
@@ -19,6 +19,129 @@ if it has, it needs 0.10.0. Whoever deploys decides.
|
||||
| _this branch_ | TARIFF-09 - the day-ahead feed is asked for the horizon it was given, so tomorrow arrives firm instead of carried |
|
||||
| _this branch_ | TARIFF-10 - configuring a price provider is what fetches prices; trading stays its own switch |
|
||||
| _this branch_ | TARIFF-12 - a supplier outage stops deleting the plan on its first failed fetch, and TARIFF_MAX_AGE_S starts governing |
|
||||
| _this branch_ | TARIFF-14 - the price a trade is decided against is the price of the interval being traded |
|
||||
| _this branch_ | TARIFF-07 - three plugins that never fetched a price are withdrawn, and a site configured on one is told so **⚠️ BACKWARD-INCOMPATIBLE** |
|
||||
|
||||
⚠️ **A CONFIG THAT STARTS ON THE CURRENT VERSION CAN BE REFUSED ON THE NEXT
|
||||
ONE.** TARIFF-07 withdraws `engie_dynamic`, `totalenergies_dynamic` and
|
||||
`eneco_eddy`. `price_provider` set to any of those three is now **refused at
|
||||
startup** — `SystemExit(78)` (`EX_CONFIG`), the add-on does not start — where
|
||||
it was accepted before. That is as backward-incompatible as it gets, and it is
|
||||
**the point of the change** rather than a side effect of it: see the next
|
||||
paragraph for why starting was the worse outcome. Same shape as 0.8.0's
|
||||
`slew_w` rule, and the same instruction follows — **read the live options and
|
||||
check `price_provider` against the withdrawn list BEFORE `ha addons update`.**
|
||||
That check is NOT performed by this branch and cannot be.
|
||||
|
||||
⚠️ **THE REFUSAL IS THE MIGRATION, BECAUSE THERE WAS NOTHING TO MIGRATE.** All
|
||||
three pointed at endpoints that **do not exist and never did** — verified by
|
||||
hand on 2026-09-13: a 404 from a gateway with no route, a TLS hostname the
|
||||
certificate does not cover, and a 404 with an empty body. Every fetch any of
|
||||
them ever made failed, and §11.4 turns a failed fetch into a **silent** fall
|
||||
back to the static profile. So a site configured on one of these has been
|
||||
running on `static_profile` — **or on nothing at all, where none was set** —
|
||||
since the day the id was entered, with nothing on its status page ever
|
||||
differing from a working provider that never found a cheap hour. The refusal
|
||||
names the id, the endpoint, what that endpoint actually answers, which tariff
|
||||
has really been in force, and what to set instead. **Starting quietly is what
|
||||
hid this; it is not an option that was on the table.**
|
||||
|
||||
⚠️ **THERE IS NO ALIAS AND THERE WILL NOT BE ONE.** These were retail tariffs
|
||||
of the form `B × spot + A` with **per-contract** coefficients that live behind
|
||||
the customer's supplier login; CREG's tariff-card pages 404 and VREG was
|
||||
unreachable, so there is no public index to read them from. An alias to
|
||||
`generic_dayahead` — which reads *wholesale* — would have to invent the numbers
|
||||
a battery is priced against. A formula provider is the planned answer and is a
|
||||
separate card.
|
||||
|
||||
### TARIFF-07 - three plugins that never fetched a price, withdrawn
|
||||
|
||||
**`engie_dynamic`, `totalenergies_dynamic` and `eneco_eddy` are gone from
|
||||
`PROVIDERS` and from every surface that listed them.** They did not stop
|
||||
working. **They never worked.** Verified by hand on 2026-09-13, before anything
|
||||
was deleted:
|
||||
|
||||
| id | endpoint | observed |
|
||||
|---|---|---|
|
||||
| `engie_dynamic` | `https://api.engie.be/dynamic/be` | **404**, Kong `no Route matched with those values` |
|
||||
| `totalenergies_dynamic` | `https://api.totalenergies.be/prices/dynamic` | **TLS failure** — cert is `CN=pge-apigee-dev.direct-energie.com`, host not in the SAN list |
|
||||
| `eneco_eddy` | `https://api.eneco.be/eddy/prices` | **404**, empty body |
|
||||
|
||||
They are also **structurally impossible**, not merely unfinished. ENGIE's
|
||||
`(B × eSpot_15) − A` coefficients are per-contract and sit behind the
|
||||
klantenzone login, so no public unauthenticated endpoint could return a given
|
||||
customer's price; there is no regulator index to read coefficients from; and
|
||||
**"Eddy" is not an Eneco product at all** — Eneco's dynamic product is *Zon &
|
||||
Wind Dynamisch*, indexed to Belpex/EPEX **day-ahead**, not imbalance.
|
||||
|
||||
**The schema keeps all three ids on purpose, and that is the whole reason the
|
||||
refusal is reachable.** `price_provider` is a `list(off|…)` enum and the
|
||||
Supervisor validates the options file against it *before the add-on runs*. Drop
|
||||
a retired id from the enum and an existing config naming it becomes
|
||||
schema-invalid: the operator gets an opaque `expected one of ...`, the add-on
|
||||
never starts, and `validate.py`'s message is never seen — on exactly the sites
|
||||
that need it. `PROVIDERS` is two; the enum stays six. `test_price.py` asserts
|
||||
the schema offers exactly `PROVIDERS | RETIRED_PROVIDERS | {"off"}`, derived
|
||||
rather than typed, and `register()` refuses to re-register a retired id so the
|
||||
refusal cannot be taken away by an `@register` somebody adds back.
|
||||
|
||||
**`PriceProduct.REALTIME` and the widening path survive with no shipped
|
||||
declarer.** `eneco_eddy` was their sole declarer, so `_WIDE`,
|
||||
`Uncertainty.REALTIME_PRODUCT` and `WIDEN_FACTOR`'s 2.5 now have none. They are
|
||||
kept: it is the AC2 seam — declare `REALTIME`, get the wide regime, edit no
|
||||
framework code — and the first real imbalance product would otherwise have to
|
||||
rebuild it. ⚠️ **No shipped plugin declares `PriceProduct.REALTIME` until one
|
||||
does**, `test_price.py` section 5 pins that so the emptiness reads as intent
|
||||
rather than a dead branch, and section 10 keeps the path live from a class
|
||||
declared outside `price.py`. See
|
||||
`DECISIONS/TARIFF-07-withdrawing-three-plugins.md`.
|
||||
|
||||
⚠️ **FOUR FALSE CLAIMS ARE CORRECTED, AND THEY ARE WHY THIS CARD EXISTS.**
|
||||
|
||||
* `eneco_eddy` was called a "Belgian imbalance product" in `DOCS.md`,
|
||||
`app/price_forecast.py`, `app/price_history.py`, `app/static_tariff.py`,
|
||||
`DECISIONS/FCST-04-the-day-that-is-carried.md` and twice in this file. It is
|
||||
**false** — Eneco's dynamic product is day-ahead-indexed. The two historical
|
||||
entries here are **annotated in place** rather than rewritten; the record of
|
||||
what the project believed is part of the record.
|
||||
* The quarter-hourly examples those surfaces used `eneco_eddy` for are now
|
||||
**Belgian day-ahead itself**, which really does settle per quarter since the
|
||||
15-minute MTU of 2025-10-01 — `generic_dayahead` reads 96 points a day. The
|
||||
mechanism was always right; only the example was invented.
|
||||
* `export_price_eur_kwh`'s docstring claimed *"no shipped supplier publishes a
|
||||
dynamic injection curve"*. **Also false.** Eneco publishes
|
||||
`0,1 × BELPEX-H − 1,188` on the same card as its consumption formula,
|
||||
identical on the January and September 2026 cards, and ENGIE publishes
|
||||
`(B × eSpot_15) − A`. What is true: an injection curve is the **same
|
||||
two-coefficient shape** with **its own** coefficients and its own VAT
|
||||
treatment — injection is VAT-exempt, consumption is at 6% — so the scalar is
|
||||
a **simplification an operator enters by hand, not a reflection of the
|
||||
market**. `DOCS.md` says the same.
|
||||
* `_points_from_json()` is deleted with the plugins it served. It read a
|
||||
`{"data": [...]}` document **no feed anybody here has ever reached
|
||||
publishes** — invented alongside these three, and graded only against
|
||||
fixtures written from the same guess. TARIFF-08 found that loop on the
|
||||
response side for `generic_dayahead`; this is the rest of it.
|
||||
|
||||
**`DOCS.md` now answers the question a Belgian dynamic customer actually has.**
|
||||
The provider table is two rows. The new prose is explicit that
|
||||
`generic_dayahead` gives wholesale EPEX BE plus flat markup and taxes scalars,
|
||||
which **approximates** a retail dynamic tariff and **cannot express
|
||||
`B × spot + A`** — it has no multiplier, and it cannot represent a negative
|
||||
offset, which `price_markup_eur_kwh: float(0,1)` could not hold in any case. A
|
||||
formula provider is named as the planned answer and is a separate card.
|
||||
|
||||
⚠️ **One check had to be rebuilt rather than deleted.** `test_price.py` 4b
|
||||
proved `code_leaks()`'s docstring strip is load-bearing by finding a shipped
|
||||
docstring a raw scan false-positives on — and that docstring was
|
||||
`engie_dynamic`'s. With it gone the check would have **passed on a tree with
|
||||
the strip deleted**: a check that had quietly stopped being one. The stimulus
|
||||
is constructed in the test file now, and the shipped set asserted clean
|
||||
separately.
|
||||
|
||||
Nine mutations driven, all red; the table is in the decision record. Two carry
|
||||
a **+0 B** delta and two end in a traceback whose red count is therefore a
|
||||
lower bound — both flagged there, per `MUTATION.md`.
|
||||
|
||||
### TARIFF-10 - a configured provider is fetched, whether or not you trade
|
||||
|
||||
@@ -184,6 +307,91 @@ driven, and a mutation that makes the guard depend on `arbitrage.enabled()` is
|
||||
red on five named checks.
|
||||
|
||||
|
||||
### TARIFF-14 - the trade was decided against the wrong price
|
||||
|
||||
**On the only price feed that works, `_Series.at()` returned the wrong point,
|
||||
and the error grew with depth into the series.** It asked
|
||||
`s <= u < s + 1 h` of every point **in list order** and returned the **first**
|
||||
match. On the hourly shape every fixture in this repository carried before
|
||||
TARIFF-08 that is right by accident; Belgium moved to a **15-minute MTU on
|
||||
2025-10-01** and the live day-ahead feed returns **96 points at 900 s**, where
|
||||
it is wrong twice over. Measured against the shipped code at `5059e28`:
|
||||
|
||||
```
|
||||
series: 02:00=0.10 02:15=0.11 02:30=0.12 02:45=0.13 03:00=0.14 ...
|
||||
|
||||
ask + 0min -> 02:00 0.10
|
||||
ask +15min -> 02:00 0.10 <-- WRONG (wanted 02:15)
|
||||
ask +45min -> 02:00 0.10 <-- WRONG
|
||||
ask +60min -> 02:15 0.11 <-- WRONG (wanted 03:00)
|
||||
ask +75min -> 02:30 0.12 <-- WRONG (wanted 03:15)
|
||||
```
|
||||
|
||||
Inside the first hour the `:00` point wins because its hour has not run out.
|
||||
**From the second hour on the answer is not even the top of the hour** - the
|
||||
first point still inside its own hour wins - so the error compounds.
|
||||
|
||||
**What it cost is not "a slightly wrong price".** `arbitrage._act()` tests
|
||||
`here.start == v.cheapest.start` / `== v.dearest.start` to decide whether this
|
||||
interval is the buy one or the sell one. That identity almost never held, so
|
||||
**the strategy idled through exactly the interval it exists to trade**,
|
||||
reporting *"this hour is neither the buy hour nor the sell hour"* - and read
|
||||
§8.3's `here.provisional` and §17's `price < 0` off the same wrong point.
|
||||
`static_tariff.py` emits hourly points, so **static sites were never affected**;
|
||||
this was reachable only down the dynamic path TARIFF-08/09 made work.
|
||||
|
||||
**The rule now, in one sentence: a point covers from its own start until the
|
||||
next point's start, and never more than one hour.**
|
||||
|
||||
| | before | after |
|
||||
|---|---|---|
|
||||
| 900 s | `at(02:15)` -> the 02:00 point | -> the 02:15 point |
|
||||
| 3600 s | `at(02:30)` -> the 02:00 point | **unchanged** |
|
||||
| **mixed** `[900, 3600]` | each point one hour wide | **each point takes its own gap** |
|
||||
|
||||
The mixed case is not an edge case: FCST-04's carry-forward emits **hourly
|
||||
carried points beside quarter-hourly firm ones**, and the soak measured spacing
|
||||
`[900, 3600]` in **70 % of samples**. A window derived from one global spacing
|
||||
would be wrong in the normal steady state.
|
||||
|
||||
**The one-hour cap is load-bearing.** FCST-04 leaves an hour *out* of the
|
||||
series when nothing published it and yesterday has no value either, and
|
||||
`at()` raising `Absence` for that hour is what sends the caller to the static
|
||||
profile (§11.4). Uncapped, "until the next point's start" would stretch the
|
||||
point before a three-hour hole across the whole hole and answer every instant
|
||||
in it with a stale price - filling holes, introduced by the fix for a different
|
||||
defect.
|
||||
|
||||
**The last point's window is a decision and it is written down.**
|
||||
`DECISIONS/TARIFF-14-the-window-a-point-covers.md` §2 argues it: the last point
|
||||
is given **the same span as the point before it** (one hour when it is the only
|
||||
point). A flat one-hour tail would answer **00:30 tomorrow with today's 23:45
|
||||
price**, which is reaching for a neighbour across a day boundary. The cost of
|
||||
the choice - a mixed series ending at the firm/carried seam loses its last 45
|
||||
minutes to `Absence` - is stated there rather than buried, and the direction is
|
||||
the safe one.
|
||||
|
||||
**`at()` now reads a point's neighbours, so it would depend on the series being
|
||||
sorted - and it establishes that itself** rather than trusting the three
|
||||
producers that happen to guarantee it today. The lookup index is sorted by UTC
|
||||
start once, at construction; a series handed its 96 points in reverse order
|
||||
answers identically, which is a check. `covers()` shared the defect exactly
|
||||
(it has no rule of its own - it asks `at()`) and is fixed by the same change;
|
||||
`window()` did not (it grades a point's **start** against a range and asks
|
||||
nothing about duration), and its docstring now says so.
|
||||
|
||||
**Nothing about §9.3 or a point's `start` changed.** `PricePoint` still has
|
||||
three fields; the span is derived from the neighbours, not stored.
|
||||
|
||||
Graded by `test_price.py` section 14 - 900 s, 3600 s, **mixed**, holes, the
|
||||
tail, caller order, `covers()`/`window()` - and end to end by
|
||||
`test_arbitrage.py` section 10, which drives `decide()` over a 96-point
|
||||
quarter-hourly day whose cheapest and dearest quarters are deliberately **not**
|
||||
on a whole hour. Eight mutations, all red, including a **zero-byte** one on the
|
||||
tail rule; the table with byte deltas and the checks that were NOT reached is
|
||||
in the decision note §6.
|
||||
|
||||
|
||||
### TARIFF-09 - the feed is asked for the range the caller wanted
|
||||
|
||||
**`generic_dayahead` took `since` and `until` and requested the bare feed URL,
|
||||
@@ -510,7 +718,12 @@ spread it would have created was never compared against anything.
|
||||
leaves the local hours it never held **out of the series** - §11.4 for those
|
||||
hours, and asking for one raises rather than answering. Correct behaviour,
|
||||
and the wording is now the behaviour.
|
||||
* ⚠️ **A sub-hourly feed keeps every print.** `eneco_eddy` is the imbalance
|
||||
* ⚠️ **A sub-hourly feed keeps every print.** [⚠️ **The claim that follows is
|
||||
FALSE and TARIFF-07 corrected it: `eneco_eddy` was not an imbalance product,
|
||||
was not an Eneco product at all, and its endpoint never existed. The real
|
||||
quarter-hourly feed is Belgian DAY-AHEAD since the 15-minute MTU of
|
||||
2025-10-01, which is what `generic_dayahead` reads. The behaviour described
|
||||
below is unchanged.**] `eneco_eddy` is the imbalance
|
||||
product and settles per **quarter**; the fill walks whole hours to decide
|
||||
what needs carrying, and that is not a rule about what a published hour may
|
||||
contain. Every published point inside the horizon passes through by identity,
|
||||
@@ -3726,6 +3939,10 @@ somebody else's tariff. What is mechanised is reachability; what stays human is
|
||||
truthfulness.
|
||||
|
||||
**The product type is declared, not inferred from the plugin's name.**
|
||||
[⚠️ **FALSE, corrected by TARIFF-07: `eneco_eddy` was never a real-time /
|
||||
imbalance product — Eneco's dynamic product is day-ahead-indexed — and the
|
||||
plugin has been withdrawn. The declaration mechanism described here is
|
||||
unchanged and is now exercised only from outside `price.py`.**]
|
||||
`eneco_eddy` is a real-time/imbalance product and gets widened arbitrage
|
||||
thresholds; that reaches a consumer through `capabilities().product` and
|
||||
`volatility_regime()`, never through a name comparison. `test_price.py` asserts
|
||||
|
||||
@@ -75,8 +75,9 @@ Every point still carries `provisional=True`, so the **flag never lies** — wha
|
||||
decays is the number, behind a flag that is telling the truth.
|
||||
|
||||
⚠️ **No value-level check can catch that**, and claiming one would be worse than
|
||||
claiming none. A carried point and a published-provisional point (`eneco_eddy`'s
|
||||
current quarter, §9.2) are the same three fields carrying the same flag. The
|
||||
claiming none. A carried point and a published-provisional point (a real-time
|
||||
product's current interval, §9.2) are the same three fields carrying the same
|
||||
flag. The
|
||||
difference is provenance, and provenance is not in the value.
|
||||
|
||||
The defence is therefore an **ordering**, not a validation:
|
||||
@@ -139,8 +140,16 @@ file constructs and still holds a reference to.
|
||||
|
||||
## 4. A sub-hourly feed passes through whole; the fill only decides which hours are missing.
|
||||
|
||||
`eneco_eddy` is the Belgian imbalance product and settles **per quarter**, and
|
||||
`price.py`'s `_validate_raw()` accepts any strictly-increasing spacing — so a
|
||||
⚠️ **This section originally said `eneco_eddy` is "the Belgian imbalance
|
||||
product". That was false and [[TARIFF-07]] withdrew the plugin: Eneco's dynamic
|
||||
product is "Zon & Wind Dynamisch" and is indexed to Belpex/EPEX **day-ahead**,
|
||||
not to imbalance, and `api.eneco.be/eddy/prices` never existed. The mechanism
|
||||
below is unaffected, because the real quarter-hourly feed is the one still
|
||||
shipped.**
|
||||
|
||||
Belgian **day-ahead** settles per **quarter** — the 15-minute MTU took effect
|
||||
on 2025-10-01, so `generic_dayahead` reads 96 points a day rather than 24 — and
|
||||
`price.py`'s `_validate_raw()` accepts any strictly-increasing spacing, so a
|
||||
published hour may legitimately hold four points.
|
||||
|
||||
FCST-04's first caller keyed the published set as `{_utc_hour(p.start): p for p
|
||||
|
||||
@@ -0,0 +1,230 @@
|
||||
# TARIFF-07 — three plugins that never fetched a price, and the one mechanism they leave behind
|
||||
|
||||
**Status: DECIDED 2026-09-13 — withdraw `engie_dynamic`,
|
||||
`totalenergies_dynamic` and `eneco_eddy`. Keep `PriceProduct.REALTIME` and the
|
||||
widening machinery. Refuse a retired id by name rather than aliasing it.**
|
||||
|
||||
Filed with TARIFF-07. Follows [[TARIFF-08]], which found the same class of
|
||||
defect one layer down — a parser graded against its own guess — and left the
|
||||
question of these three open.
|
||||
|
||||
---
|
||||
|
||||
## 1. The endpoints, verified rather than taken on report
|
||||
|
||||
The card said the three endpoints were dead. They were re-checked by hand on
|
||||
**2026-09-13** before anything was deleted, because "the endpoint is dead" is
|
||||
exactly the kind of claim this repository keeps finding to be second-hand:
|
||||
|
||||
| id | endpoint | observed |
|
||||
|---|---|---|
|
||||
| `engie_dynamic` | `https://api.engie.be/dynamic/be` | **HTTP 404**, body `{"message":"no Route matched with those values","request_id":"..."}` — a Kong gateway with no route configured |
|
||||
| `totalenergies_dynamic` | `https://api.totalenergies.be/prices/dynamic` | **TLS failure**. The handshake completes and the server presents `CN=pge-apigee-dev.direct-energie.com`; `api.totalenergies.be` is not in the SAN list, so the connection is closed before any request is sent |
|
||||
| `eneco_eddy` | `https://api.eneco.be/eddy/prices` | **HTTP 404**, `content-length: 1`, `server: Microsoft-IIS/10.0` — an empty body from a live host that does not serve this path |
|
||||
|
||||
All three failed. Had any one of them answered, this card would have stopped
|
||||
and asked, because an endpoint that answers changes the decision from
|
||||
"withdraw" to "fix the parser".
|
||||
|
||||
## 2. Why this is not "unfinished", and why no replacement can be written
|
||||
|
||||
An unfinished plugin has a right endpoint and a wrong parser. These have
|
||||
neither, and the reason is structural:
|
||||
|
||||
* **ENGIE's dynamic price is `(B × eSpot_15) − A`, and `B` and `A` are
|
||||
PER-CONTRACT.** They live behind the customer's klantenzone login. No
|
||||
unauthenticated public URL can return *a given customer's* price, so the
|
||||
endpoint this plugin wanted cannot exist for anybody to build.
|
||||
* **There is no regulator index to read the coefficients from.** CREG's
|
||||
tariff-card pages 404; VREG was unreachable. Every coefficient found in
|
||||
research came from a supplier's own PDF, which is a document, not a feed.
|
||||
* **"Eddy" is not an Eneco product at all.** Eneco's dynamic product is
|
||||
**"Zon & Wind Dynamisch"**, and it is indexed to **Belpex/EPEX day-ahead** —
|
||||
not to imbalance. The plugin was wrong about the product, the product type
|
||||
and the URL simultaneously.
|
||||
|
||||
So the three were not withdrawn for being broken. They were withdrawn for
|
||||
describing tariffs that no public endpoint can serve.
|
||||
|
||||
## 3. What a site configured on one of these has actually been running
|
||||
|
||||
⚠️ **This is the half that decides what the migration is.** §11.4 turns a
|
||||
failed fetch into a silent fall back: static profile, plan provisional, no
|
||||
arbitrage. That is the *correct* degradation, and it says nothing at all about
|
||||
the cause. Every fetch these three ever made raised `SourceUnavailable`.
|
||||
|
||||
Therefore a site configured on one of them has been running on
|
||||
`static_profile` — **or on nothing at all, where no profile was set** — since
|
||||
the day the id was entered, and **nothing on its status page has ever differed
|
||||
from a working provider that simply never found a cheap hour**.
|
||||
|
||||
The migration is therefore **not** "keep their prices working". There were no
|
||||
prices. It is *"tell them their prices never worked"*, which is why the answer
|
||||
is a refusal and not a fallback.
|
||||
|
||||
### Why not a silent alias to `generic_dayahead`
|
||||
|
||||
It is the friendlier-looking option and it is wrong. These published **retail**
|
||||
prices; `generic_dayahead` reads **wholesale**. Aliasing needs the site's own
|
||||
`B` and `A`, which §2 has just established nobody has. An alias would have to
|
||||
**invent the numbers a battery is priced against** — the same defect as a
|
||||
parser graded against its own guess, one layer further from anything that
|
||||
reports. The scalar `price_markup_eur_kwh` cannot stand in for `B` either: it
|
||||
is an addend, not a multiplier, and it is `float(0,1)` so it cannot even hold a
|
||||
negative `A`.
|
||||
|
||||
## 4. ⚠️ The `config.yaml` schema is deliberately WIDER than the registry
|
||||
|
||||
`price_provider` is a `list(off|…)` enum, and the Supervisor validates the
|
||||
options file against it **before the add-on starts**.
|
||||
|
||||
So the obvious edit — delete the three ids from the enum along with the classes
|
||||
— has a consequence that runs exactly opposite to this card's purpose:
|
||||
|
||||
```
|
||||
operator's config says price_provider: eneco_eddy
|
||||
|
|
||||
+-- id NOT in the schema enum
|
||||
|
|
||||
+-- Supervisor rejects the options file
|
||||
|
|
||||
+-- add-on never starts
|
||||
|
|
||||
+-- validate.py never runs
|
||||
|
|
||||
+-- the named refusal is NEVER SEEN
|
||||
```
|
||||
|
||||
The operator gets an opaque `expected one of ...` from a layer they did not
|
||||
configure, on **exactly the sites the withdrawal exists to inform**.
|
||||
|
||||
**Decision: keep all three ids in the schema enum.** `PROVIDERS` shrinks to
|
||||
two; the enum stays at six. The cost is that the dropdown still offers three
|
||||
dead ids to a *new* operator — and that is the cheaper failure, because
|
||||
choosing one is refused at the next startup **by name**, with the endpoint, the
|
||||
observation and the alternative, which is strictly more than a schema
|
||||
rejection would say. Both translations label them withdrawn in the option
|
||||
description.
|
||||
|
||||
`app/price.py`'s `RETIRED_PROVIDERS` is the list. `test_price.py` section 1
|
||||
asserts the schema offers exactly `PROVIDERS | RETIRED_PROVIDERS | {"off"}` —
|
||||
derived, not typed — and `register()` refuses to re-register a retired id, so
|
||||
the registry and the retired table cannot both claim one and the refusal cannot
|
||||
be taken away by an `@register` somebody adds back.
|
||||
|
||||
## 5. ⚠️ `PriceProduct.REALTIME` and the widening path are KEPT with no shipped declarer
|
||||
|
||||
`eneco_eddy` was the **sole** declarer of `PriceProduct.REALTIME`. Deleting it
|
||||
orphans:
|
||||
|
||||
* `_WIDE = frozenset({PriceProduct.REALTIME})`
|
||||
* `Uncertainty.REALTIME_PRODUCT`
|
||||
* `WIDEN_FACTOR[Uncertainty.REALTIME_PRODUCT] = 2.5`
|
||||
|
||||
**They stay.** The reasoning, stated here so the next reader finds intent
|
||||
rather than an apparent dead branch:
|
||||
|
||||
1. A reviewer finding a mechanism with **zero callers** would reasonably delete
|
||||
it. That is good instinct applied to a missing fact, and this file is the
|
||||
fact.
|
||||
2. It is the **AC2 seam**. The whole point of `capabilities().product` is that
|
||||
a new supplier is a new class and nothing else: declare `REALTIME`, get the
|
||||
wide regime, edit no framework code. Deleting the enum member would mean the
|
||||
first genuine imbalance product has to **rebuild** that seam — and rebuilding
|
||||
it is precisely the "editing the optimiser to add a supplier" that AC2
|
||||
forbids.
|
||||
3. It costs nothing. Two enum members and a dict entry.
|
||||
|
||||
**⚠️ NO SHIPPED PLUGIN DECLARES `PriceProduct.REALTIME` UNTIL ONE DOES.** Both
|
||||
shipped plugins (`generic_dayahead`, `manual_csv`) are day-ahead. §9.2's row 2
|
||||
is unoccupied. That is not the same as it having no mechanism, and the
|
||||
difference is what this section exists to record.
|
||||
|
||||
It is **pinned, not merely documented**. `test_price.py` section 5 asserts:
|
||||
|
||||
* that **no shipped plugin** declares `REALTIME` — so if one ever does, the
|
||||
check goes red and the right fix is to **edit this record**, not to delete
|
||||
the seam;
|
||||
* that the widening still works **from a declaration alone** —
|
||||
`volatility_regime`, `uncertainty_of` and `widen_factor` are graded on a
|
||||
`ProviderCapabilities` the test file builds;
|
||||
* and section 10 registers `_NewSupplier`, a class defined **outside
|
||||
`price.py`** that declares `REALTIME` and is asserted to land in the wide
|
||||
regime. That class is now the only live declarer in the tree, which makes the
|
||||
seam something this suite exercises end to end rather than a constant it
|
||||
reads.
|
||||
|
||||
## 6. What else went, and one thing that did not
|
||||
|
||||
`_points_from_json()` went with the three. It read a `{"data": [{...}]}`
|
||||
row-per-point document that **no feed anybody here has ever reached
|
||||
publishes** — the shape was invented alongside these plugins, and the only
|
||||
bodies ever fed to it were fixtures written from the same guess. TARIFF-08
|
||||
found that loop on the response side for `generic_dayahead`; this is the rest
|
||||
of it. `needs_credentials` went too: the three were its only setters and
|
||||
nothing read it.
|
||||
|
||||
The shape survives in **one** place and in one direction: `row_shape_body()` in
|
||||
`test_price.py`, feeding `generic_dayahead` so that section 9 grades a row
|
||||
document as a **failure** for it rather than a second dialect it might sniff.
|
||||
It is named for what it is.
|
||||
|
||||
⚠️ **One check had to be rebuilt rather than deleted, and saying so matters.**
|
||||
`test_price.py` section 4b proved that `code_leaks()`'s docstring strip is
|
||||
load-bearing by finding a shipped docstring that a raw text scan would
|
||||
false-positive on. The docstring that did that was **`engie_dynamic`'s** — it
|
||||
spelled `price_markup_eur_kwh` in full. With it gone, no shipped docstring
|
||||
spells a full framework key, so the check would have **passed on a tree with
|
||||
the strip deleted**: a check that had quietly stopped being one, which is the
|
||||
third failure mode in `MUTATION.md`. The stimulus is now constructed in the
|
||||
test file (`_DocumentedPlugin`), and the shipped set is asserted clean
|
||||
separately.
|
||||
|
||||
## 7. Mutations driven
|
||||
|
||||
Every check added or rebuilt here was driven red. Harness:
|
||||
`/home/homelab/.cache/t07-mut/`, which rebuilds the tree from the worktree and
|
||||
purges `__pycache__` per `MUTATION.md`'s rule.
|
||||
|
||||
| | mutation | Δ | red on |
|
||||
|---|---|---|---|
|
||||
| M1 | delete the retired branch from `_price_problems()` | −2785 B | 7 checks in `test_price.py` |
|
||||
| M2 | move the retired branch **after** the unknown-id branch | **+0 B** | the same 7 |
|
||||
| M3 | make `been_running` constant (drop the empty-`static_profile` case) | −212 B | 2 |
|
||||
| M4 | drop the "WHAT TO DO" half of the refusal | −874 B | 1 |
|
||||
| M5 | `RETIRED_PROVIDERS = {}` | −1099 B | 5, then a traceback |
|
||||
| M6 | drop the three ids from `config.yaml`'s enum — §4's trap | −47 B | 2 |
|
||||
| M7 | delete `register()`'s retired guard | −1101 B | 2, then a traceback |
|
||||
| M8 | `_WIDE = frozenset()` | −23 B | 3 + 4 across two suites |
|
||||
| M9 | `WIDEN_FACTOR[REALTIME_PRODUCT]` 2.5 → 1.5 | **+0 B** | 1 + 1 |
|
||||
|
||||
⚠️ **M2 and M9 have a byte delta of zero**, which `MUTATION.md` names as the
|
||||
risky case. The harness rebuilds the tree from source on every run and purges
|
||||
`__pycache__`, so there is no inherited cache to read; and stale bytecode
|
||||
manufactures **survivors**, never reds, so a red from a `+0 B` mutation is not
|
||||
at risk from that defect. M2's grep confirmed the branch had moved on disk.
|
||||
|
||||
⚠️ **M5 and M7's red counts are LOWER BOUNDS.** Both runs ended in a traceback
|
||||
with checks after it ungraded — FCST-06's rule: read the output, not the count.
|
||||
M7's traceback is itself informative: with the guard gone the re-registration
|
||||
**succeeds**, `PROVIDERS` is polluted with a class that has no fixture, and the
|
||||
suite dies downstream. That is the failure the guard prevents, arriving loudly.
|
||||
|
||||
## 8. What this card did not do
|
||||
|
||||
* **It did not pick a version number.** The change is backward-incompatible — a
|
||||
config legal on the current version is refused on the next — and the
|
||||
CHANGELOG says so prominently, in the shape PKG-07 established for `slew_w`.
|
||||
Which number it belongs to depends on whether the current image has already
|
||||
been built and pulled, which is a fact about the live box that this branch
|
||||
cannot observe. Whoever deploys decides.
|
||||
* **It did not implement the formula provider.** `B × spot + A` with per-site
|
||||
coefficients is the real answer for a Belgian dynamic customer and is a
|
||||
separate card. `DOCS.md` references it as planned and is explicit about what
|
||||
`generic_dayahead` can and cannot express in the meantime — in particular
|
||||
that it cannot represent a negative offset, and that
|
||||
`price_markup_eur_kwh: float(0,1)` could not hold one anyway.
|
||||
* **It did not rewrite released CHANGELOG history.** The two historical entries
|
||||
that called `eneco_eddy` an imbalance product are annotated in place with a
|
||||
correction naming this card, rather than silently edited. The record of what
|
||||
the project believed is part of the record.
|
||||
@@ -0,0 +1,349 @@
|
||||
# TARIFF-14 — how long one price point covers, and what the last one does
|
||||
|
||||
One defect and one genuinely open decision. The defect had no defensible
|
||||
alternative once it was measured; the decision — the **last** point's window —
|
||||
did, and the alternative is argued below rather than dismissed.
|
||||
|
||||
---
|
||||
|
||||
## 0. The measurement this card exists for
|
||||
|
||||
`_Series.at()` asked `s <= u < s + 1 h` of every point **in list order** and
|
||||
returned the **first** match. Run against the shipped code at `5059e28` on a
|
||||
quarter-hourly series:
|
||||
|
||||
```
|
||||
series: 02:00=0.10 02:15=0.11 02:30=0.12 02:45=0.13 03:00=0.14 ...
|
||||
|
||||
ask + 0min -> 02:00 0.10
|
||||
ask +15min -> 02:00 0.10 <-- WRONG (wanted 02:15)
|
||||
ask +30min -> 02:00 0.10 <-- WRONG
|
||||
ask +45min -> 02:00 0.10 <-- WRONG
|
||||
ask +60min -> 02:15 0.11 <-- WRONG (wanted 03:00)
|
||||
ask +75min -> 02:30 0.12 <-- WRONG (wanted 03:15)
|
||||
```
|
||||
|
||||
**Two failure modes, not one.** Inside the first hour the `:00` point wins
|
||||
because its hour has not run out. From the second hour on the answer is not
|
||||
even the top of the hour — the first point still inside its own hour wins — so
|
||||
**the error compounds with depth into the series**.
|
||||
|
||||
Same code, same run, after the fix:
|
||||
|
||||
```
|
||||
ask + 0min -> 02:00 0.10 ask +60min -> 03:00 0.14
|
||||
ask +15min -> 02:15 0.11 ask +75min -> 03:15 0.15
|
||||
ask +30min -> 02:30 0.12 ask +90min -> 03:30 0.16
|
||||
ask +45min -> 02:45 0.13 ask+120min -> 04:00 0.18
|
||||
```
|
||||
|
||||
An hourly series is **unchanged at every instant**, before and after — which is
|
||||
the whole reason nothing caught this. Every price fixture in this repository
|
||||
before TARIFF-08 was the invented hourly row shape, `static_tariff.py` emits
|
||||
hourly points, and the DST checks count points on the 23- and 25-hour days
|
||||
without ever asking `at()` for a quarter-hour. **The suite had been exercising
|
||||
the one resolution on which the broken rule and the right one agree.**
|
||||
|
||||
Belgium moved to a 15-minute MTU on 2025-10-01 and the live day-ahead feed
|
||||
returns **96 points at 900 s**, so this is reachable only down the dynamic path
|
||||
TARIFF-08/09 just made work. Static sites are immune.
|
||||
|
||||
## 0.1 What it cost, which is not "a slightly wrong price"
|
||||
|
||||
`arbitrage._act()` calls `_point_at(plan, now)` → `series.at(now)` and then
|
||||
tests `here.start == v.cheapest.start` / `== v.dearest.start` to decide whether
|
||||
this interval is the buy one or the sell one. That is an **identity test
|
||||
between the matched point and the point the verdict was built from**, and on a
|
||||
quarter-hourly series it almost never matched — so the strategy idled through
|
||||
exactly the interval it exists to trade, reporting *"this hour is neither the
|
||||
buy hour nor the sell hour"*. It also read `here.provisional` (§8.3) and §17's
|
||||
`price < 0` off that same wrong point.
|
||||
|
||||
---
|
||||
|
||||
## 1. The rule
|
||||
|
||||
> **A point covers from its own start until the next point's start, and never
|
||||
> more than one hour.**
|
||||
|
||||
* **900 s** → each point covers its own quarter.
|
||||
* **3600 s** → each point covers its own hour. Identical to what shipped.
|
||||
* **MIXED** → **each point takes its own gap.** FCST-04's carry-forward emits
|
||||
hourly carried points beside quarter-hourly firm ones, and the soak measured
|
||||
spacing `[900, 3600]` in **70 % of samples** — so a window derived from one
|
||||
global spacing would be wrong in the **normal steady state**, not in an edge
|
||||
case.
|
||||
|
||||
### 1.1 Why the one-hour cap is not decoration
|
||||
|
||||
FCST-04 **leaves holes on purpose**: an hour with no published price and no
|
||||
yesterday value is left *out* of the series, and `price_forecast.py`'s module
|
||||
note names `at()` raising `Absence` for it as the behaviour that sends the
|
||||
caller to the static profile (§11.4 for that hour).
|
||||
|
||||
Uncapped, "until the next point's start" stretches the point **before** a
|
||||
three-hour hole across the whole hole and answers every instant in it with a
|
||||
stale price. Filling holes is the defect this module exists to refuse, and it
|
||||
would have been introduced **by the fix for a different one**. Graded in
|
||||
`test_price.py` 14d; the mutation that removes the cap is red there.
|
||||
|
||||
---
|
||||
|
||||
## 2. The decision: what the **last** point covers
|
||||
|
||||
There is no next start, and §9.3 says nothing about how wide the final interval
|
||||
is — a `PricePoint` has **three fields** and must not learn a fourth, because
|
||||
that is the wire format every store, replay, fixture and captured body in this
|
||||
repository is written against. So the tail is a **choice**, and both candidates
|
||||
are defensible.
|
||||
|
||||
**Chosen: the last point is given the same span as the point before it** (one
|
||||
hour when it is the only point).
|
||||
|
||||
| | flat one hour | the span in front of it (**chosen**) |
|
||||
|---|---|---|
|
||||
| 96-point 900 s day, `at(23:50)` | answers 23:45 | answers 23:45 |
|
||||
| the same day, `at(00:30)` **tomorrow** | **answers with today's 23:45 price** | `Absence` |
|
||||
| 24-point hourly day, `at(23:50)` | answers 23:00 | answers 23:00 |
|
||||
| mixed series ending at the firm/carried seam | answers for an hour | `Absence` after 15 min |
|
||||
|
||||
**The second row is why.** A flat one-hour tail reaches across a day boundary
|
||||
and prices tomorrow morning with yesterday's last print — which is
|
||||
"reaching for a neighbouring one", the single thing the `Absence` message has
|
||||
always refused to do. The chosen rule never extends a point past a span the
|
||||
series itself demonstrates.
|
||||
|
||||
### 2.1 The cost of the choice, stated rather than buried
|
||||
|
||||
On a **mixed** series whose last point is a carried **hourly** one sitting
|
||||
behind quarter-hourly firm ones, the tail is **900 s**, and the last 45 minutes
|
||||
of the horizon raise `Absence` for an hour something did carry a value for.
|
||||
|
||||
That is accepted, not tolerated, and the direction is the safe one:
|
||||
|
||||
* the caller falls back to the static profile (§11.4) instead of trading on a
|
||||
span the series never demonstrated;
|
||||
* the point is **carried**, so §8.3 forbids any irreversible action resting on
|
||||
it regardless;
|
||||
* it is at the far end of a 48 h horizon, where nothing is decided at 30 s.
|
||||
|
||||
Under-covering costs a missed decision. Over-covering prices a real interval
|
||||
with a number that does not describe it. Those are not symmetric.
|
||||
|
||||
### 2.2 The degenerate case
|
||||
|
||||
A series of **one** point has no gap in front of it to copy and gets **one
|
||||
hour** — the shipped behaviour, kept deliberately. `test_price.py` 14e states
|
||||
it as its own check so it cannot decay into an accident.
|
||||
|
||||
---
|
||||
|
||||
## 3. Does `at()` now depend on sortedness? Yes — and it establishes it itself
|
||||
|
||||
The old rule read one point at a time and did not care about order. The new one
|
||||
reads a point's **neighbours**, so it *would* depend on the series being
|
||||
ascending.
|
||||
|
||||
It does not, because **the lookup index is sorted by UTC start at construction**
|
||||
(`_cover_table()`, called once from `_Series.__init__`). No caller's list order
|
||||
can reach `at()`.
|
||||
|
||||
That matters because the three things that "guarantee" order today each
|
||||
guarantee it somewhere else:
|
||||
|
||||
| | what it guarantees | where |
|
||||
|---|---|---|
|
||||
| `_validate_raw()` | strictly increasing starts | the `load_prices()` path only |
|
||||
| `price_history._parse_points()` | re-sorts by instant | the store read path only |
|
||||
| FCST-04's fill | emits ascending | that one producer only |
|
||||
| `PriceRead.series()` | **nothing** | every other caller, including tests |
|
||||
|
||||
Three producers agreeing today is not an invariant; it is a coincidence with
|
||||
good references. `test_price.py` 14f drives a series handed its 96 points in
|
||||
**reverse** order and asserts `at()` answers identically at every one of them —
|
||||
red the moment the sort is removed.
|
||||
|
||||
**Duplicated starts** (refused by `_validate_raw()`, constructible by hand) give
|
||||
the earlier list entry a zero-width window and the later one the real span. One
|
||||
of the pair becomes unreachable through `at()`; **nothing raises**, at
|
||||
construction or on the 30 s control path. Stated as a check rather than left to
|
||||
be discovered.
|
||||
|
||||
---
|
||||
|
||||
## 4. `covers()` shared it. `window()` did not
|
||||
|
||||
* **`covers()`** has no window rule of its own — it calls `at()` — so it shared
|
||||
the defect exactly and the same change fixes it. It is kept as a delegation
|
||||
for that reason: a second copy of the rule is a second place for it to be
|
||||
wrong.
|
||||
* **`window(since, until)`** grades a point's **start** against a half-open
|
||||
range and never asks how long a point lasts, so it carries no resolution
|
||||
assumption at all. It is **not** a coverage query — a point that starts before
|
||||
`since` and runs into it is absent from its result by design. That is now
|
||||
said in its docstring and pinned by a check, because the next reader will ask.
|
||||
It has **no callers** in `app/`.
|
||||
|
||||
---
|
||||
|
||||
## 5. What this card does NOT fix, with its shape
|
||||
|
||||
`price_forecast._walk()` — §8.3's irreversible-action gate — iterates
|
||||
`committed_hours()`, which steps in **whole hours**, and asks `series.at(u)` at
|
||||
each. On a quarter-hourly series that now reliably returns the hour's `:00`
|
||||
point (it previously returned something earlier and arbitrary), so the gate
|
||||
reads **one point per hour** and never looks at the `:15`/`:30`/`:45` flags.
|
||||
|
||||
A partially-provisional hour would therefore be gated on its first quarter
|
||||
alone. **Two things keep it off every shipped install, and both are
|
||||
circumstantial rather than structural:** `price_permits()`/`begin_phase()` have
|
||||
**no caller in `app/main.py`** — the §12 price-justified entry point is built
|
||||
and not yet wired — and FCST-04 carries **whole** hours while
|
||||
`generic_dayahead` publishes whole days, so the four quarters of an hour are
|
||||
firm or carried together anyway. Both are facts about the *callers and
|
||||
producers*, not properties of the *gate*, so this is written down here rather
|
||||
than left for the first sub-hourly feed or the first wiring-up to find. Fixing it means teaching
|
||||
`committed_hours()` about resolution, which is a data-model question this card
|
||||
deliberately does not open.
|
||||
|
||||
---
|
||||
|
||||
## 6. Mutations
|
||||
|
||||
Eight, each purged before the run (TEST-15), each reverted with the tree
|
||||
fingerprint checked afterwards (MUTATION.md habit 18), backup at a `mktemp`
|
||||
path per invocation.
|
||||
|
||||
⚠️ **Re-driven in full on the MERGED tree** after TARIFF-10 landed on
|
||||
`release/1.0` (base `88c59a9` -> `559f0eb`). All eight give the same verdicts
|
||||
and the same red-check counts as before the merge, which is the answer that
|
||||
matters: TARIFF-10 added 650 lines of checks to `test_price.py` and none of
|
||||
them moves under any of these edits, because they grade `resolve_tariff()`'s
|
||||
*reachability from a configuration* and not the lookup. Clean `app/price.py` on
|
||||
the merged tree = **`90d8ca31f7c0109d`, 85953 B** (pre-merge it was
|
||||
`30d1a56f0117e571`, 83742 B; the difference is TARIFF-10's `provider_id()` /
|
||||
`provider_configured()`, which this card does not touch).
|
||||
|
||||
| | mutation | Δ | red in |
|
||||
|---|---|---|---|
|
||||
| M1 | `at()` back to the shipped fixed one-hour scan over `self.points` | +68 B | **24 named checks**: `test_price.py` 14b ×4, 14c ×3, 14d, 14e ×3, 14f, 14g; `test_arbitrage.py` 10b ×6, 10c ×3, 10d ×2 |
|
||||
| M2 | drop the one-hour cap | −68 B | `test_price.py` 14d |
|
||||
| M3 | flat one-hour tail instead of the span in front | **+0 B** | `test_price.py` 14e ×3, 14g — 4 checks |
|
||||
| M4 | trust the caller's list order — drop the sort | −61 B | `test_price.py` 14f |
|
||||
| M5 | `covers()` grows its own hour-wide rule | +95 B | `test_price.py` 14e, 14g |
|
||||
| M6 | `window()` becomes a coverage query | −37 B | `test_price.py` 14g |
|
||||
| M7 | the window's end becomes inclusive | +1 B | `test_price.py` 14b ×4, 14c ×4, 14d, 14e ×4, 14f ×2, 14g — 16 checks |
|
||||
| M8 | the window's start becomes exclusive | −1 B | `test_price.py` 14b ×4, 14c ×4, 14d, 14f ×2, 14g, 14h, plus the pre-existing section 6 check — 14 checks |
|
||||
|
||||
**M3 is the zero-byte one and it is the tail decision itself** — the case
|
||||
MUTATION.md names as the risky size, caught by the check written for it.
|
||||
|
||||
**What the arbitrage file does NOT see, with sizes.** M2, M4, M5, M6 and M8
|
||||
leave `test_arbitrage.py` green, and that is honest rather than a gap to round
|
||||
off: they are `at()`-level properties (holes, caller order, `covers()`,
|
||||
`window()`, the start boundary) that no arbitrage fixture reaches, because the
|
||||
probes are seven minutes into a slot and the plans have no holes. The one that
|
||||
matters end to end — M1, the shipped rule — is red in **both** files, on the
|
||||
named check `THE PROPERTY THE DEFECT BROKE`.
|
||||
|
||||
---
|
||||
|
||||
## 7. TARIFF-10 landed on the base while this was open — what it changes here
|
||||
|
||||
Base moved `88c59a9` → `559f0eb` mid-card. The question is not whether the
|
||||
branches merge; it is whether TARIFF-10 makes any sentence above **false**.
|
||||
|
||||
**In `app/price.py` the two do not meet.** TARIFF-10 added `provider_id()` and
|
||||
`provider_configured()` beside `price_option()` and routed `load_prices()`
|
||||
through them. That is the option-reading layer; it touches nothing in
|
||||
`_Series`, `at()`, `covers()`, `window()` or `_cover_table()`. Git merged the
|
||||
file without a conflict, and the merged `at()` is byte-identical to the one the
|
||||
mutations above were driven against.
|
||||
|
||||
**Behaviourally it widens who has a series at all, and that cuts one way only.**
|
||||
`run_tariff_refresh()` now runs on installs with `arbitrage_enabled` **off**, so
|
||||
a provider-configured site has a standing plan — and therefore a `_Series` —
|
||||
where before it had none. **It does not add a new `at()` consumer to the 30 s
|
||||
tick**, which is the claim worth checking rather than assuming:
|
||||
|
||||
| caller of `at()`/`covers()` | reached by a non-trading install? |
|
||||
|---|---|
|
||||
| `arbitrage._point_at()` (`arbitrage.py:300`) | no — trading only |
|
||||
| `static_tariff.TariffPlan.price_at()` / `.words` `here` (`:750`, `:819`) | on the tick these are reached through `static_status()`, which builds a **`static_plan`** — hourly points, immune. `main.py` calls only `plan.words` on the fetched plan, which counts points and never looks one up |
|
||||
| `price_forecast._walk()` (`:562`, `:568`) | **`price_permits()` and `begin_phase()` have no caller in `app/main.py` at all** — §12's price-justified entry point is built and not yet wired |
|
||||
|
||||
So TARIFF-10 neither creates nor closes any exposure this card is about. What it
|
||||
does do is make the defect's reach larger the moment such a site arms trading,
|
||||
because the series it would trade against now exists on every configured
|
||||
install rather than only on armed ones. That strengthens the case for the fix
|
||||
and changes nothing in its reasoning.
|
||||
|
||||
**In `test_price.py` both cards appended a section 13.** Nothing is dropped:
|
||||
TARIFF-10's stays **13**, this card's is renumbered **14** (which is also this
|
||||
card's number), and the temp-directory teardown TARIFF-10 put at the end of the
|
||||
file stays at the end of the file. The two sections share no fixture, no option
|
||||
dict and no helper name; TARIFF-10's grades `resolve_tariff()`'s *reachability
|
||||
from a configuration* and drives real fetches with a counting transport, and
|
||||
**none of its checks moves under any of the eight mutations above** — measured,
|
||||
not assumed.
|
||||
|
||||
---
|
||||
|
||||
## 8. TARIFF-12 landed too — a plan may now be 90 minutes old when it is asked
|
||||
|
||||
Base moved `559f0eb` → `a6e64fb` while this was open. TARIFF-12 stops a
|
||||
returned §11.4 degradation from overwriting a standing plan that is still
|
||||
inside `TARIFF_MAX_AGE_S`, so **a served plan can now be up to
|
||||
`3 × TARIFF_REFRESH_S = 5400 s` old** where before it was at most one refresh
|
||||
interval. `at()` is what answers *"what does it cost right now"* out of that
|
||||
older plan, so the question is fair and the answer is written out rather than
|
||||
assumed.
|
||||
|
||||
**Per-point span: unaffected, structurally.** Every window is derived from the
|
||||
neighbouring points' **absolute `start` instants**. Those are `datetime`s inside
|
||||
a frozen dataclass inside a tuple — they cannot move, and `_cover_table()` runs
|
||||
once at construction over data that cannot change afterwards. A plan's *age* is
|
||||
a property of `fetched_at` / `tariff_at`, never of a window. **The index cannot
|
||||
go stale**, which is a small positive under TARIFF-12 rather than a risk: a kept
|
||||
plan now lives longer, and there is nothing in it that needed recomputing.
|
||||
|
||||
**The one-hour cap: unaffected.** It bites only where a gap exceeds an hour,
|
||||
i.e. at a hole FCST-04 left. Walking further into a series crosses more windows;
|
||||
each is still resolved on its own neighbours.
|
||||
|
||||
**The tail: reachable for the first time, and the choice made in §2 is the one
|
||||
that survives it.** This is the half worth spelling out.
|
||||
|
||||
* At the **shipped 48 h horizon** the tail is unreachable by staleness. A plan
|
||||
is served for at most 90 minutes, so `now` is at most 90 minutes into a
|
||||
~48 h series — the last point is still about 46½ hours away.
|
||||
* But `price_horizon_h` is `int(1,72)` in `config.yaml`, so an operator **can**
|
||||
configure a **1 h** horizon. A 90-minute serve-stale window against a 1 h
|
||||
series puts `now` genuinely past the end. That is not hypothetical; it is a
|
||||
supported configuration meeting a supported outage.
|
||||
* Under the **chosen** rule a quarter-hourly 1 h series ends at `:45` and that
|
||||
point's window closes at the top of the next hour: `at()` raises `Absence`,
|
||||
`_point_at()` returns None, `_act()` idles with its "no price for this
|
||||
instant" reason, and §11.4 sends the site to the static profile.
|
||||
* Under the **rejected** flat one-hour tail the same stale plan would have gone
|
||||
on trading against the `:45` print for **up to 45 minutes after the interval
|
||||
it describes had ended** — pricing an outage with a number for a quarter that
|
||||
is over.
|
||||
|
||||
So TARIFF-12 does not weaken the tail decision. It is the first thing that makes
|
||||
it reachable on a shipped configuration, and it independently selects the same
|
||||
answer. ⚠️ **Note which direction that runs**: the more stale a plan is allowed
|
||||
to be, the more the tail rule is doing real work, so it should be re-read if
|
||||
`TARIFF_MAX_AGE_S` is ever widened again.
|
||||
|
||||
**`TariffPlan.from_feed` cannot produce a series this code has not seen.** It is
|
||||
a pure predicate on `origin` (`== "provider"`), and TARIFF-12's decision keeps or
|
||||
replaces a **whole plan object**. It never splices two reads into one series, so
|
||||
it cannot hand `at()` an unsorted or duplicated point list — the case §3 covers
|
||||
defensively and that nothing shipped produces.
|
||||
|
||||
**Re-graded on this base.** All 29 files green. `app/price.py` is byte-identical
|
||||
to the TARIFF-10 merge (`90d8ca31f7c0109d`, 85953 B) because TARIFF-12 does not
|
||||
touch it. **M1** re-driven: +68 B, red on the same **24 named checks** across
|
||||
both files. **M3** re-driven: **+0 B**, red on the same 4. This branch
|
||||
contributes **zero diff** to `app/main.py` and `app/static_tariff.py`.
|
||||
+95
-13
@@ -1369,7 +1369,7 @@ does, and only after you have read why.
|
||||
|
||||
| option | meaning |
|
||||
|---|---|
|
||||
| `price_provider` | `off` (default) or one of the five plugins below |
|
||||
| `price_provider` | `off` (default) or one of the two plugins below |
|
||||
| `price_markup_eur_kwh` | Your supplier's margin, EUR/kWh. **Added only when the chosen plugin's feed does not already contain it** |
|
||||
| `price_taxes_eur_kwh` | Levies and VAT, EUR/kWh. Same rule |
|
||||
| `export_price_eur_kwh` | The **injection** price. A separate scalar, not a series — see below |
|
||||
@@ -1386,11 +1386,76 @@ arbitrage decision wrong by that amount, in a direction nothing reports.
|
||||
| plugin | product | the published figure contains | the add-on adds |
|
||||
|---|---|---|---|
|
||||
| `generic_dayahead` | day-ahead | energy only (wholesale EPEX BE) | markup **and** taxes |
|
||||
| `engie_dynamic` | day-ahead | energy + Engie's markup | taxes |
|
||||
| `totalenergies_dynamic` | day-ahead | energy + TotalEnergies' markup | taxes |
|
||||
| `eneco_eddy` | **real-time / imbalance** | everything, all-in | nothing |
|
||||
| `manual_csv` | day-ahead | everything, all-in | nothing |
|
||||
|
||||
⚠️ **Three plugins were withdrawn in TARIFF-07, and if your configuration names
|
||||
one of them, read this.** `engie_dynamic`, `totalenergies_dynamic` and
|
||||
`eneco_eddy` are gone. They did not stop working — **they never worked.** Each
|
||||
pointed at an endpoint that does not exist and never did:
|
||||
|
||||
| id | its endpoint | what that endpoint actually answers |
|
||||
|---|---|---|
|
||||
| `engie_dynamic` | `https://api.engie.be/dynamic/be` | **404**, gateway error `no Route matched with those values` |
|
||||
| `totalenergies_dynamic` | `https://api.totalenergies.be/prices/dynamic` | **TLS failure** — that hostname is not in the server's certificate |
|
||||
| `eneco_eddy` | `https://api.eneco.be/eddy/prices` | **404**, empty body |
|
||||
|
||||
Every fetch any of them ever made failed, and a failed fetch falls back to the
|
||||
static tariff **silently** — so a site set to one of these has been running on
|
||||
`static_profile`, or on nothing at all, since the day it was configured, with
|
||||
nothing on the status page saying so. The add-on now **refuses to start** on one
|
||||
of these ids and tells you all of the above by name. That is deliberate: a
|
||||
silent fallback is what hid this for as long as it lasted.
|
||||
|
||||
There is no automatic migration and there cannot be one. These were *retail*
|
||||
tariffs of the form `B x spot + A` with **per-contract** coefficients that live
|
||||
behind your supplier's customer login; there is no public index to read them
|
||||
from. Any alias would have to invent the numbers your battery is priced against.
|
||||
(`eneco_eddy` was doubly wrong: **"Eddy" is not an Eneco product at all** —
|
||||
Eneco's dynamic product is *Zon & Wind Dynamisch*, and it is indexed to
|
||||
Belpex/EPEX **day-ahead**, not to imbalance.)
|
||||
|
||||
#### So what should a Belgian dynamic-tariff customer do today?
|
||||
|
||||
**Honestly: `generic_dayahead` gets you close, and you should know exactly how
|
||||
close.** It reads the wholesale EPEX BE day-ahead curve and adds your
|
||||
`price_markup_eur_kwh` and `price_taxes_eur_kwh` as **flat** scalars. Your real
|
||||
Belgian dynamic contract is a **formula**:
|
||||
|
||||
```
|
||||
consumption: price = B x spot + A <- B and A are in YOUR contract
|
||||
```
|
||||
|
||||
The add-on cannot express that yet. Two gaps, and the second is the harder one:
|
||||
|
||||
* **No multiplier.** `B` is typically slightly above 1 (it scales the wholesale
|
||||
price before anything is added). A flat markup cannot scale anything, so the
|
||||
*shape* of your curve is right but its *amplitude* is not — the peaks and
|
||||
troughs the optimiser trades against are compressed relative to your bill.
|
||||
* **No negative offset.** `A` is frequently **negative** on injection and can be
|
||||
on consumption. `price_markup_eur_kwh` is `float(0,1)` in the schema — it
|
||||
**cannot hold a negative number at all**, so that contract shape is currently
|
||||
inexpressible rather than merely approximate.
|
||||
|
||||
What that means in practice: arbitrage decisions made on `generic_dayahead` are
|
||||
made against a curve that is *correlated* with your bill, not equal to it. For
|
||||
deciding **which hours** are cheap that is usually good enough; for deciding
|
||||
**whether a cycle pays for itself** — which is what `cycle_cost_threshold_eur`
|
||||
compares against — it is an approximation, and one that errs by however far `B`
|
||||
is from 1.
|
||||
|
||||
A **formula provider** (`B x spot + A`, with your own coefficients entered once)
|
||||
is the planned answer and is a separate card. It is not implemented here. Until
|
||||
it ships, the two supported honest options are:
|
||||
|
||||
1. **`generic_dayahead`** with your markup and taxes set as well as you can —
|
||||
accepting the approximation above; or
|
||||
2. **`manual_csv`**, where you compute the all-in prices yourself from your own
|
||||
contract's formula and hand the add-on the answers. Nothing is approximated,
|
||||
but nothing is automatic either.
|
||||
|
||||
If you are on a **fixed** or **day/night** tariff — most Belgian households —
|
||||
neither of those applies: use `static_profile` and leave `price_provider: off`.
|
||||
|
||||
You cannot get this wrong by editing the markup: each plugin *declares* what its
|
||||
feed contains and the add-on adds only the rest. Set a markup under a plugin
|
||||
that already carries one and it is **ignored, with a line in the log saying so**
|
||||
@@ -1404,10 +1469,13 @@ whether a plugin described its supplier's feed truthfully — that is a claim
|
||||
about somebody else's tariff, and no code can verify it. If you write a plugin,
|
||||
the `sources` declaration is the part a reviewer has to read.
|
||||
|
||||
`eneco_eddy` is a real-time/imbalance product rather than a day-ahead one. It is
|
||||
flagged as high-volatility, which widens arbitrage thresholds once the
|
||||
optimiser lands. That comes from the plugin's own declaration, not from its
|
||||
name.
|
||||
**Real-time / imbalance products still have a place in the machinery, but no
|
||||
plugin declares one.** A plugin that declares `PriceProduct.REALTIME` is flagged
|
||||
high-volatility and gets widened arbitrage thresholds; that comes from the
|
||||
plugin's own declaration, never from its name. Both shipped plugins are
|
||||
day-ahead, so nothing exercises that path on a real install today — it is kept
|
||||
on purpose so the first genuine imbalance product does not have to rebuild it.
|
||||
See `DECISIONS/TARIFF-07-withdrawing-three-plugins.md`.
|
||||
|
||||
**`generic_dayahead` and the shape of the public feed.** It reads Fraunhofer
|
||||
ISE's Energy-Charts API (`https://api.energy-charts.info/price?bzn=BE`, no key
|
||||
@@ -1519,9 +1587,10 @@ applies to those hours, and asking the plan for one raises rather than
|
||||
answering. An hour with no price is not a free hour, and the fill does not
|
||||
invent a neighbour, a mean or a zero for it.
|
||||
|
||||
**A quarter-hourly feed keeps all four of its prints.** `eneco_eddy` settles
|
||||
per quarter, and the fill decides *which whole hours need carrying* — never what
|
||||
a published hour may contain. Every point the provider published inside the
|
||||
**A quarter-hourly feed keeps all four of its prints.** Belgian day-ahead
|
||||
settles per quarter — the 15-minute MTU took effect on 2025-10-01, so
|
||||
`generic_dayahead` reads 96 points a day — and the fill decides *which whole
|
||||
hours need carrying*, never what a published hour may contain. Every point the provider published inside the
|
||||
horizon is passed through untouched; a carried point is added only for a whole
|
||||
hour the feed did not print at all.
|
||||
|
||||
@@ -1571,8 +1640,21 @@ volume being full exactly as it applies to the volume being absent.
|
||||
|
||||
**Injection is priced separately** (`export_price_eur_kwh`). In Belgium it is
|
||||
often near zero, which by itself strongly favours self-consumption over export.
|
||||
It is a single number rather than a curve because no supported supplier
|
||||
publishes a dynamic injection tariff; negative values are accepted.
|
||||
Negative values are accepted — you can be paid to *not* export at midday in
|
||||
summer.
|
||||
|
||||
⚠️ It is a single number rather than a curve, and the reason given here until
|
||||
TARIFF-07 — *"no supported supplier publishes a dynamic injection tariff"* — was
|
||||
**false**. Suppliers do. Eneco publishes `0,1 x BELPEX-H − 1,188` on the same
|
||||
tariff card as its consumption formula (identical on the January and the
|
||||
September 2026 cards), and ENGIE publishes one in the same shape,
|
||||
`(B x eSpot_15) − A`. What is true is narrower: an injection curve is the **same
|
||||
two-coefficient formula** as the consumption curve but with **its own**
|
||||
coefficients and **its own VAT treatment** — injection is VAT-exempt where
|
||||
consumption is at 6% — so it is a second per-contract formula rather than a copy
|
||||
of the first, and the add-on cannot evaluate either one yet. Until the formula
|
||||
provider ships, **this scalar is a simplification you enter by hand — an average
|
||||
of what you are actually paid — and not a reflection of the market.**
|
||||
|
||||
**The old Home Assistant price sensors.** `price_now_entity` and
|
||||
`price_avg_entity` still work and are unchanged: the add-on compares them and
|
||||
|
||||
+289
-169
@@ -63,14 +63,21 @@ THE THREE THINGS THIS MODULE IS SHAPED BY
|
||||
knowing which of those two things you are reading.
|
||||
|
||||
2. THE PRODUCT TYPE IS DECLARED, NOT INFERRED FROM THE PLUGIN'S NAME.
|
||||
`eneco_eddy` is a real-time / imbalance product (§9.2) and §8.3 says those
|
||||
are treated as high-volatility with widened arbitrage thresholds. If the
|
||||
optimiser has to write `if provider_id == "eneco_eddy"` to know that, then
|
||||
adding a sixth supplier means editing the optimiser - which is precisely
|
||||
what AC2 forbids. So `capabilities().product` is a `PriceProduct`, it rides
|
||||
along on the series, and `volatility_regime()` reads the enum. A plugin id
|
||||
appears nowhere in this file except in `PROVIDERS` and in the class that
|
||||
answers to it.
|
||||
§9.2 classes real-time / imbalance products separately from day-ahead ones
|
||||
and §8.3 says those are treated as high-volatility with widened arbitrage
|
||||
thresholds. If the optimiser had to write `if provider_id == "<supplier>"`
|
||||
to know that, then adding a supplier would mean editing the optimiser -
|
||||
which is precisely what AC2 forbids. So `capabilities().product` is a
|
||||
`PriceProduct`, it rides along on the series, and `volatility_regime()`
|
||||
reads the enum. A plugin id appears nowhere in this file except in
|
||||
`PROVIDERS`, in `RETIRED_PROVIDERS`, and in the class that answers to it.
|
||||
|
||||
⚠️ NO SHIPPED PLUGIN DECLARES `PriceProduct.REALTIME` (TARIFF-07). Both
|
||||
shipped plugins are day-ahead, so `_WIDE`, `Uncertainty.REALTIME_PRODUCT`
|
||||
and `WIDEN_FACTOR`'s 2.5 currently have no shipped declarer. They are KEPT
|
||||
deliberately - see DECISIONS/TARIFF-07-withdrawing-three-plugins.md - and
|
||||
test_price.py section 10 keeps them live by declaring REALTIME from a class
|
||||
defined outside this file, which is AC2's own stimulus.
|
||||
|
||||
3. "NO PRICE DATA" IS A TYPE, NOT AN EMPTY LIST.
|
||||
§11.4: no price data -> fall back to the static profile, mark the plan
|
||||
@@ -480,7 +487,6 @@ class ProviderCapabilities:
|
||||
product: PriceProduct
|
||||
sources: frozenset # which Components the published figure contains
|
||||
horizon_h: int = 48 # aligns with the §8.1/§8.2 forecast horizon
|
||||
needs_credentials: bool = False
|
||||
egress_hosts: tuple = () # §16 allow-list, declared not discovered
|
||||
|
||||
def __post_init__(self):
|
||||
@@ -541,6 +547,121 @@ class PriceRead(_Outcome):
|
||||
return _NoData(reason)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# How long ONE point covers - TARIFF-14
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
# ⚠️ A POINT DOES NOT CARRY ITS OWN DURATION AND MUST NOT LEARN TO. §9.3 lists
|
||||
# THREE fields and `PricePoint` has three; a fourth would change the wire
|
||||
# format every store, replay, fixture and captured body in this repository is
|
||||
# written against. The span is DERIVED from the neighbours instead - here,
|
||||
# once, at construction.
|
||||
MAX_POINT_SPAN = timedelta(hours=1)
|
||||
|
||||
|
||||
def _cover_table(points) -> tuple:
|
||||
"""`(start_utc, end_utc, point)` per point, ascending. The `at()` index.
|
||||
|
||||
THE DEFECT THIS PREVENTS (TARIFF-14, HIGH). `at()` asked
|
||||
`s <= u < s + 1 h` of every point IN LIST ORDER and returned the FIRST
|
||||
match. On an hourly series - the only resolution any fixture carried
|
||||
before TARIFF-08 - that is right by accident. Belgium moved to a
|
||||
15-minute MTU on 2025-10-01 and the live day-ahead feed returns 96 points
|
||||
at 900 s, where it is wrong twice over. Measured on shipped `5059e28`:
|
||||
|
||||
series: 02:00=0.10 02:15=0.11 02:30=0.12 02:45=0.13 03:00=0.14
|
||||
|
||||
at(02:15) -> 02:00 the :00 point's HOUR has not run out yet
|
||||
at(03:15) -> 02:30 and from the second hour on it is not even the
|
||||
top of the hour: the first point still inside
|
||||
its own hour wins, so the error COMPOUNDS with
|
||||
depth into the series
|
||||
|
||||
`arbitrage._act()` compares `here.start` against `v.cheapest.start` and
|
||||
`v.dearest.start` to decide whether this interval is the buy or the sell
|
||||
one. That identity test almost never matched the extremum, so the
|
||||
strategy idled through exactly the interval it exists to trade - and read
|
||||
`here.provisional` and §17's `price < 0` guard off the wrong point while
|
||||
doing it. `static_tariff.py` emits hourly points, so static sites never
|
||||
saw it; this is reachable only down the dynamic path TARIFF-08/09 made
|
||||
work.
|
||||
|
||||
THE RULE, in one sentence:
|
||||
|
||||
A POINT COVERS FROM ITS OWN START UNTIL THE NEXT POINT'S START, AND
|
||||
NEVER MORE THAN ONE HOUR.
|
||||
|
||||
* 900 s -> each point covers its own quarter, so `at(02:15)` is the
|
||||
02:15 point;
|
||||
* 3600 s -> each point covers its own hour, unchanged from shipped;
|
||||
* MIXED -> EACH POINT TAKES ITS OWN GAP. FCST-04's carry-forward emits
|
||||
HOURLY carried points beside QUARTER-HOURLY firm ones, and the soak
|
||||
measured spacing `[900, 3600]` in 70 % of samples - so mixed is the
|
||||
normal steady state, and a window derived from one global spacing
|
||||
would be wrong THERE rather than in an edge case.
|
||||
|
||||
⚠️ WHY THE ONE-HOUR CAP, WHICH IS NOT DECORATION. FCST-04 LEAVES HOLES ON
|
||||
PURPOSE: an hour with no published price AND no yesterday value is left
|
||||
OUT of the series, and `price_forecast.py`'s module note names `at()`
|
||||
raising `Absence` for it as the behaviour that sends the caller to the
|
||||
static profile (§11.4 for that hour). Uncapped, "until the next point's
|
||||
start" would stretch the point BEFORE a three-hour hole across the whole
|
||||
hole and answer every instant in it with a stale price. Filling holes is
|
||||
the defect this module exists to refuse, and it would have been
|
||||
introduced by the fix for a different one.
|
||||
|
||||
⚠️ THE LAST POINT'S WINDOW IS DECIDED, NOT INHERITED. There is no next
|
||||
start and nothing in §9.3 says how wide the final interval is, so THE
|
||||
LAST POINT IS GIVEN THE SAME SPAN AS THE POINT BEFORE IT - one hour when
|
||||
it is the only point. Three consequences, all intended:
|
||||
|
||||
* on a 96-point 900 s day the 23:45 point covers 23:45 .. 00:00, so
|
||||
`at(23:50)` ANSWERS and `at(00:00)` raises `Absence`. A flat one-hour
|
||||
tail would have answered 00:30 TOMORROW with today's last price,
|
||||
which is reaching for a neighbour across a day boundary;
|
||||
* on an hourly series the tail is an hour, exactly as it shipped, so no
|
||||
static or hourly site changes behaviour;
|
||||
* on a MIXED series whose last point is a carried HOURLY one behind
|
||||
quarter-hourly firm ones, the tail is 900 s and the last 45 minutes
|
||||
of the horizon raise `Absence`. That is the SAFE direction and it is
|
||||
accepted rather than tolerated: the caller falls back to the static
|
||||
profile (§11.4) instead of trading on a span the series never
|
||||
demonstrated, and a carried point is provisional, so §8.3 forbids any
|
||||
irreversible action resting on it regardless.
|
||||
|
||||
⚠️ ORDER IS ESTABLISHED HERE, NOT ASSUMED - and that is the difference
|
||||
between this and the rule it replaces. `at()` now reads neighbours, so it
|
||||
WOULD depend on sortedness; the table is therefore sorted by UTC start at
|
||||
construction, and no caller's list order can reach it. `_validate_raw()`
|
||||
rejects out-of-order and duplicate starts on the `load_prices()` path,
|
||||
`price_history._parse_points()` re-sorts what it reads off disk, and
|
||||
FCST-04's fill emits ascending - but `PriceRead.series()` itself has
|
||||
never enforced any of that, and an assumption that holds today and is
|
||||
unstated is how this class of defect recurs.
|
||||
|
||||
⚠️ NOTHING HERE RAISES ON A MALFORMED SERIES. A duplicated start - not
|
||||
reachable through `load_prices()`, which refuses it - gives the earlier
|
||||
list entry a ZERO-width window and the later one the real span, so one of
|
||||
the pair becomes unreachable through `at()` and neither the 30 s control
|
||||
path nor construction raises.
|
||||
"""
|
||||
rows = sorted(((p.start.astimezone(timezone.utc), i, p)
|
||||
for i, p in enumerate(points)),
|
||||
key=lambda r: (r[0], r[1]))
|
||||
out = []
|
||||
for n, (start, _i, point) in enumerate(rows):
|
||||
if n + 1 < len(rows):
|
||||
span = rows[n + 1][0] - start # -> the next point's start
|
||||
elif n:
|
||||
span = start - rows[n - 1][0] # -> the tail, see above
|
||||
else:
|
||||
span = MAX_POINT_SPAN # -> a series of one point
|
||||
if span > MAX_POINT_SPAN:
|
||||
span = MAX_POINT_SPAN
|
||||
out.append((start, start + span, point))
|
||||
return tuple(out)
|
||||
|
||||
|
||||
class _Series(PriceRead):
|
||||
kind = "series"
|
||||
|
||||
@@ -564,6 +685,10 @@ class _Series(PriceRead):
|
||||
self.provider = provider
|
||||
self.fetched_at = fetched_at
|
||||
self.regime = volatility_regime(product)
|
||||
# ⚠️ TARIFF-14. Built ONCE, here, so that `at()` scans an index whose
|
||||
# order it established itself rather than one it hopes it was handed.
|
||||
# See `_cover_table()` for the window rule and for what the tail does.
|
||||
self._cover = _cover_table(self.points)
|
||||
self.words = (f"{len(self.points)} points from {provider} "
|
||||
f"({product.value}, {self.regime} thresholds)")
|
||||
|
||||
@@ -572,25 +697,47 @@ class _Series(PriceRead):
|
||||
return {"prices": [p.canonical() for p in self.points]}
|
||||
|
||||
def at(self, instant: datetime) -> PricePoint:
|
||||
"""The point covering `instant`. Raises `Absence` if the series does not.
|
||||
"""The point whose OWN interval covers `instant`, at ANY resolution.
|
||||
|
||||
⚠️ RETURNS A POINT OR RAISES; it never returns None. `series.at(t) or
|
||||
fallback` is the same defaulting bug one level down, so the falsy
|
||||
answer does not exist. Ask `covers()` first if you mean to branch.
|
||||
|
||||
⚠️ TARIFF-14: THE WINDOW IS NOT ONE HOUR WIDE ANY MORE, AND WAS NEVER
|
||||
ENTITLED TO BE. It runs to the next point's start, capped at an hour -
|
||||
`_cover_table()` above holds the rule, the tail decision and the
|
||||
reproduction on a quarter-hourly series. A fixed `s + 1 h` here
|
||||
returned the FIRST point still inside its own hour, which on the live
|
||||
900 s feed is neither the covering point nor, past the first hour,
|
||||
even the top of the hour.
|
||||
|
||||
A linear scan over the index, which is 192 entries on the shipped 48 h
|
||||
horizon. This runs on the 30 s control path through
|
||||
`arbitrage._point_at()`; a bisect would be correct and is not worth
|
||||
the second way to be wrong at this size.
|
||||
"""
|
||||
u = instant.astimezone(timezone.utc)
|
||||
for p in self.points:
|
||||
s = p.start.astimezone(timezone.utc)
|
||||
if s <= u < s + timedelta(hours=1):
|
||||
return p
|
||||
for start, end, point in self._cover:
|
||||
if start <= u < end:
|
||||
return point
|
||||
raise Absence(
|
||||
f"no price point covers {u.isoformat()}; the series runs "
|
||||
f"{self.points[0].start.isoformat()} .. "
|
||||
f"{self.points[-1].start.isoformat()}. An hour outside the series "
|
||||
f"is an hour with no price, which is §11.4 for that hour - not a "
|
||||
f"reason to reach for a neighbouring one.")
|
||||
f"{self.points[-1].start.isoformat()}. An instant outside the "
|
||||
f"series - or inside a hole FCST-04 left because nothing "
|
||||
f"published that hour - is an instant with no price, which is "
|
||||
f"§11.4 for it, not a reason to reach for a neighbouring one.")
|
||||
|
||||
def covers(self, instant: datetime) -> bool:
|
||||
"""Whether `at(instant)` would answer.
|
||||
|
||||
⚠️ SHARED TARIFF-14 EXACTLY, AND IS FIXED BY THE SAME CHANGE. It has
|
||||
no window rule of its own - it asks `at()` - so on the 900 s feed it
|
||||
answered True for instants no point covered and False for none of
|
||||
them, because the hour-wide window is never SHORTER than the truth.
|
||||
It is kept as a delegation for that reason: a second copy of the rule
|
||||
is a second place for it to be wrong.
|
||||
"""
|
||||
try:
|
||||
self.at(instant)
|
||||
return True
|
||||
@@ -598,7 +745,17 @@ class _Series(PriceRead):
|
||||
return False
|
||||
|
||||
def window(self, since: datetime, until: datetime) -> tuple:
|
||||
"""Points whose start falls in [since, until). May legitimately be empty."""
|
||||
"""Points whose START falls in [since, until). May legitimately be empty.
|
||||
|
||||
⚠️ DID NOT SHARE TARIFF-14, AND THE REASON IS WORTH STATING RATHER
|
||||
THAN REDISCOVERING. This grades a point's `start` against a half-open
|
||||
range and never asks how long a point lasts, so it carries no
|
||||
resolution assumption to be wrong about - the same call answers the
|
||||
same set at 900 s, at 3600 s and mixed. It is therefore NOT a
|
||||
coverage query: a point that starts before `since` and runs into it
|
||||
is absent from this tuple by design. `at()`/`covers()` are the
|
||||
coverage questions.
|
||||
"""
|
||||
a = since.astimezone(timezone.utc)
|
||||
b = until.astimezone(timezone.utc)
|
||||
return tuple(p for p in self.points
|
||||
@@ -811,11 +968,28 @@ def export_price_eur_kwh(opts: dict) -> float:
|
||||
"""AC4 / §9.3: the injection price is a SEPARATE SCALAR, not a second series.
|
||||
|
||||
In Belgium it is often near zero, which by itself favours self-consumption
|
||||
over export. It is deliberately not routed through the plugin framework: no
|
||||
shipped supplier publishes a dynamic injection curve, and inventing the
|
||||
shape for one would be a second canonical format nobody emits. When one
|
||||
appears it gets a `PriceProvider` of its own and this scalar becomes its
|
||||
FIXED-product default - the seam is already the right shape for that.
|
||||
over export. It is deliberately not routed through the plugin framework -
|
||||
but ⚠️ NOT because nothing dynamic exists to route. That claim stood here
|
||||
until TARIFF-07 and it was FALSE:
|
||||
|
||||
* Eneco publishes an injection formula on the same tariff card as its
|
||||
consumption formula - `0,1 x BELPEX-H - 1,188` - identical on the
|
||||
January and the September 2026 cards;
|
||||
* ENGIE publishes one in the same two-coefficient shape,
|
||||
`(B x eSpot_15) - A`.
|
||||
|
||||
So a supplier DOES publish a dynamic injection curve. What is true is
|
||||
narrower and is the actual reason this is a scalar: an injection curve is
|
||||
the SAME `B x spot - A` shape as the consumption curve but carries ITS OWN
|
||||
coefficients and its own VAT treatment - injection is VAT-exempt where
|
||||
consumption is at 6% - so it is a second per-contract formula, not a second
|
||||
copy of the first. Nothing in this add-on can evaluate either one yet
|
||||
(`generic_dayahead` cannot express `B x spot + A` at all; the formula
|
||||
provider is a separate card). Until it can, this scalar is a SIMPLIFICATION
|
||||
an operator enters by hand - an average of what they are actually paid -
|
||||
and not a reflection of the market. When the formula provider lands, an
|
||||
injection curve gets a `PriceProvider` of its own and this scalar becomes
|
||||
its FIXED-product default; the seam is already the right shape for that.
|
||||
|
||||
⚠️ It is NOT reachable from a plugin: `export_price_eur_kwh` is in
|
||||
FRAMEWORK_KEYS, so a consumption plugin cannot quietly fold an injection
|
||||
@@ -862,16 +1036,14 @@ class PriceProvider(ABC):
|
||||
past today from the trade set as provisional - so the optimiser never
|
||||
saw tomorrow as firm, at any hour of any day, and nothing reported it.
|
||||
|
||||
This promise is NOT mechanically enforced, and the honest reason is
|
||||
TARIFF-08's: three of the five shipped plugins point at endpoints
|
||||
nobody has verified (TARIFF-07 is the open decision on withdrawing
|
||||
them), so nobody knows how those feeds spell a date range. A rule
|
||||
obliging them to pass one would be satisfied by an INVENTED parameter
|
||||
This promise is NOT mechanically enforced. A rule obliging every
|
||||
plugin to pass a range would be satisfied by an INVENTED parameter
|
||||
name - the parser-graded-against-its-own-guess defect, moved from the
|
||||
response to the request. What exists instead is a roster pin in
|
||||
test_price.py: which plugins consume the range and which do not is
|
||||
asserted as a literal, so a sixth supplier, or a change to one of the
|
||||
three, forces the decision rather than inheriting the silence.
|
||||
response to the request, and the defect TARIFF-08 found on the
|
||||
response side. What exists instead is a roster pin in test_price.py:
|
||||
which plugins consume the range and which do not is asserted as a
|
||||
literal, so a third supplier forces the decision rather than
|
||||
inheriting the silence.
|
||||
"""
|
||||
|
||||
|
||||
@@ -899,9 +1071,10 @@ def code_leaks(cls) -> list:
|
||||
|
||||
⚠️ DOCSTRINGS ARE STRIPPED BEFORE THE SEARCH, AND THAT IS NOT COSMETIC.
|
||||
Every shipped plugin's docstring is REQUIRED to name the components it
|
||||
sources, so `engie_dynamic` says "markup" in prose and would false-positive
|
||||
against `price_markup_eur_kwh` on a raw text scan. The check that fires on
|
||||
the documentation it also mandates is the check that gets deleted. So the
|
||||
sources, so `generic_dayahead` says "markup" in prose and would
|
||||
false-positive against `price_markup_eur_kwh` on a raw text scan. The
|
||||
check that fires on the documentation it also mandates is the check that
|
||||
gets deleted. So the
|
||||
class is parsed, every docstring node is dropped, and the search runs on
|
||||
`ast.unparse` of what is left - i.e. on code, which is the only thing that
|
||||
can actually add a number twice.
|
||||
@@ -911,8 +1084,8 @@ def code_leaks(cls) -> list:
|
||||
or zipapp build would not. Refusing to start a battery controller because a
|
||||
STATIC ANALYSIS could not run would trade a real outage for a hypothetical
|
||||
accounting error, so an unreadable source returns `[]` and logs. CI proves
|
||||
the scan really runs: test_price.py asserts source was found for all five
|
||||
shipped plugins, so a build that silently lost it is a red check, not a
|
||||
the scan really runs: test_price.py asserts source was found for every
|
||||
shipped plugin, so a build that silently lost it is a red check, not a
|
||||
quietly disarmed guard.
|
||||
"""
|
||||
try:
|
||||
@@ -931,6 +1104,58 @@ def code_leaks(cls) -> list:
|
||||
return sorted({name for name in BANNED_IN_PLUGIN_CODE if name in code})
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# TARIFF-07: the ids that USED to resolve here, and what an operator on one of
|
||||
# them has actually been running.
|
||||
#
|
||||
# ⚠️ THE DEFECT THIS EXISTS TO PREVENT: A RETIRED ID READ AS A TYPO. Without
|
||||
# this table `price_provider: "eneco_eddy"` reaches validate.py's unknown-id
|
||||
# branch and is refused as "not a shipped plugin" - true, and useless. The
|
||||
# operator did not mistype it; it WAS a shipped plugin, they chose it on
|
||||
# purpose, and the add-on has accepted it every startup since. What they need
|
||||
# told is not "that id is wrong" but "that id never fetched a price, and here
|
||||
# is what you have been running instead" - because §11.4 turned every failed
|
||||
# fetch into a silent fall back to `static_profile`, so NOTHING they can see
|
||||
# has ever differed from a working provider that finds no cheap hour.
|
||||
#
|
||||
# ⚠️ AND WHY THERE IS NO SILENT ALIAS. Mapping these to `generic_dayahead`
|
||||
# would be the friendlier-looking option and it is the wrong one: these three
|
||||
# published RETAIL prices (`B x spot + A`, per contract) and `generic_dayahead`
|
||||
# reads WHOLESALE. Aliasing needs the site's own two coefficients, which nobody
|
||||
# has - they live behind the customer's klantenzone login, and there is no
|
||||
# regulator index to read them from (CREG's tariff-card pages 404; VREG was
|
||||
# unreachable; every coefficient found in research came from a supplier's own
|
||||
# PDF). An alias would therefore have to INVENT the numbers it prices a battery
|
||||
# against, which is the same class of defect as the parser graded against its
|
||||
# own guess - just one layer further from anything that reports.
|
||||
#
|
||||
# Each value is the ID-SPECIFIC half of the refusal: the endpoint, what it was
|
||||
# observed to answer on 2026-09-13, and why no replacement plugin can be built.
|
||||
# validate.py owns the common framing. Verified by hand before deletion, not
|
||||
# taken on report - see DECISIONS/TARIFF-07-withdrawing-three-plugins.md.
|
||||
# --------------------------------------------------------------------------- #
|
||||
RETIRED_PROVIDERS: dict = {
|
||||
"engie_dynamic": (
|
||||
"its endpoint https://api.engie.be/dynamic/be does not exist - the "
|
||||
"gateway there answers 404 with Kong's \"no Route matched with those "
|
||||
"values\". No public endpoint could replace it: ENGIE's dynamic price "
|
||||
"is (B x eSpot_15) - A with PER-CONTRACT coefficients that live behind "
|
||||
"your klantenzone login, so no unauthenticated URL can return YOUR "
|
||||
"price"),
|
||||
"totalenergies_dynamic": (
|
||||
"its endpoint https://api.totalenergies.be/prices/dynamic does not "
|
||||
"exist - the TLS handshake fails outright because that hostname is "
|
||||
"absent from the certificate the server presents. The coefficients are "
|
||||
"per-contract there too"),
|
||||
"eneco_eddy": (
|
||||
"its endpoint https://api.eneco.be/eddy/prices does not exist - it "
|
||||
"answers 404 with an empty body - and \"Eddy\" is not an Eneco "
|
||||
"product at all; their dynamic product is \"Zon & Wind Dynamisch\", "
|
||||
"which is indexed to Belpex/EPEX DAY-AHEAD and whose coefficients are "
|
||||
"likewise per-contract"),
|
||||
}
|
||||
|
||||
|
||||
PROVIDERS: dict = {}
|
||||
|
||||
|
||||
@@ -944,6 +1169,22 @@ def register(cls):
|
||||
"""
|
||||
if cls.id in PROVIDERS:
|
||||
raise ValueError(f"duplicate provider id {cls.id!r}")
|
||||
if cls.id in RETIRED_PROVIDERS:
|
||||
# ⚠️ AT IMPORT, NOT AT FETCH, and for the same reason the leak check is.
|
||||
# A retired id that is registered again is the one state in which
|
||||
# validate.py's retired-id refusal goes quiet WITHOUT anyone deciding
|
||||
# it should: `pid in PROVIDERS` becomes true, the named message is
|
||||
# never reached, and the operator is back to a provider that looks
|
||||
# like it works. Re-shipping an id is legitimate - the endpoint could
|
||||
# become real - but it is a DECISION, and taking the id out of
|
||||
# RETIRED_PROVIDERS in the same commit is what records it.
|
||||
raise ValueError(
|
||||
f"{cls.id!r} is in RETIRED_PROVIDERS and cannot also be a shipped "
|
||||
f"plugin. A site configured on it is refused at startup with a "
|
||||
f"message naming the id; registering it silently takes that "
|
||||
f"message away. If this id really ships again, remove it from "
|
||||
f"RETIRED_PROVIDERS in the same commit - and say in the CHANGELOG "
|
||||
f"what changed about the endpoint.")
|
||||
cls.capabilities() # fail at import, not at 15:00 on fetch day
|
||||
leaks = code_leaks(cls)
|
||||
if leaks:
|
||||
@@ -1082,7 +1323,7 @@ def check_egress(url: str, caps: ProviderCapabilities) -> str:
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# The five shipped plugins.
|
||||
# The shipped plugins.
|
||||
#
|
||||
# ⚠️ EACH DOCSTRING STATES WHICH COMPONENTS IT SOURCES AND WHICH THE FRAMEWORK
|
||||
# ADDS, in the two fixed lines `SOURCES: ...` / `The framework ADDS: ...`. That
|
||||
@@ -1092,49 +1333,22 @@ def check_egress(url: str, caps: ProviderCapabilities) -> str:
|
||||
# what its feed contains is a red check rather than a 3 c/kWh error nobody sees.
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
def _points_from_json(body, price_key, scale, provisional_default):
|
||||
"""Shared parse for the three JSON feeds. Shape differences are the args."""
|
||||
try:
|
||||
text = body.decode("utf-8") if isinstance(body, (bytes, bytearray)) \
|
||||
else str(body)
|
||||
doc = json.loads(text)
|
||||
except (ValueError, UnicodeDecodeError) as e:
|
||||
raise SourceUnavailable(f"the feed did not parse as JSON: {e}")
|
||||
rows = doc.get("data") if isinstance(doc, dict) else doc
|
||||
if not isinstance(rows, list):
|
||||
raise SourceUnavailable("the feed carried no 'data' list")
|
||||
out = []
|
||||
for row in rows:
|
||||
try:
|
||||
start = datetime.fromisoformat(str(row["start"]))
|
||||
value = float(row[price_key]) * scale
|
||||
except (KeyError, TypeError, ValueError) as e:
|
||||
raise SourceUnavailable(f"malformed row {row!r}: {e}")
|
||||
if start.tzinfo is None:
|
||||
raise SourceUnavailable(
|
||||
f"row {row['start']!r} carries no UTC offset; this feed is "
|
||||
f"required to publish absolute instants")
|
||||
out.append(RawPoint(start=start, eur_per_kwh=value,
|
||||
provisional=bool(row.get("provisional",
|
||||
provisional_default))))
|
||||
return out
|
||||
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# TARIFF-08: the COLUMNAR document, which is what the public EPEX feed really
|
||||
# publishes.
|
||||
#
|
||||
# ⚠️ THIS IS A SECOND SHAPE, READ BY A SECOND FUNCTION ON PURPOSE.
|
||||
# `_points_from_json()` above reads a row-per-point document: `{"data": [{...},
|
||||
# {...}]}`. `https://api.energy-charts.info/price?bzn=BE` does not publish that
|
||||
# and never did - it publishes PARALLEL ARRAYS:
|
||||
# ⚠️ THIS IS THE ONLY PARSER LEFT, AND THE SHAPE IT DOES **NOT** READ IS WHY.
|
||||
# Until TARIFF-07 a second function, `_points_from_json()`, read a row-per-point
|
||||
# document: `{"data": [{...}, {...}]}`. NO FEED ANYBODY REACHED EVER PUBLISHED
|
||||
# THAT SHAPE - it was invented alongside the three withdrawn plugins, and the
|
||||
# only bodies ever fed to it were the fixtures written from the same guess. It
|
||||
# went with them. `https://api.energy-charts.info/price?bzn=BE` publishes
|
||||
# PARALLEL ARRAYS:
|
||||
#
|
||||
# {"unix_seconds": [1788991200, 1788992100, ...], <- UTC epoch seconds
|
||||
# "price": [216.08, 209.90, ...], <- one per instant
|
||||
# "unit": "EUR / MWh", "license_info": "...", "deprecated": false}
|
||||
#
|
||||
# The shipped parser was pointed at that URL and asked for `doc["data"]`, so
|
||||
# The row parser was pointed at that URL and asked for `doc["data"]`, so
|
||||
# every fetch ended `SourceUnavailable: the feed carried no 'data' list` ->
|
||||
# §11.4 -> static profile, arbitrage off, and nothing in the status page said
|
||||
# why. Verified live 2026-09-10 against both fixtures under `scenarios/`.
|
||||
@@ -1290,7 +1504,8 @@ class GenericDayAhead(PriceProvider):
|
||||
either number, it cannot be the one that adds them twice.
|
||||
|
||||
⚠️ IT READS THE COLUMNAR SHAPE, AND SAYS SO IN ITS OWN `fetch()`. Until
|
||||
TARIFF-08 it called `_points_from_json()` and asked
|
||||
TARIFF-08 it called the row-per-point parser (withdrawn by TARIFF-07) and
|
||||
asked
|
||||
`api.energy-charts.info` for a `"data"` list it has never published, so
|
||||
every fetch this provider ever made ended in `SourceUnavailable` and §11.4
|
||||
turned that into a silent fall back to the static profile. See
|
||||
@@ -1376,101 +1591,6 @@ class GenericDayAhead(PriceProvider):
|
||||
provisional_default=False)
|
||||
|
||||
|
||||
@register
|
||||
class EngieDynamic(PriceProvider):
|
||||
"""Engie's dynamic day-ahead product, published in EUR/kWh.
|
||||
|
||||
SOURCES: energy, markup
|
||||
The framework ADDS: taxes
|
||||
|
||||
⚠️ Engie's published figure ALREADY CONTAINS THEIR MARKUP - it is a retail
|
||||
price, not a wholesale one. Adding `price_markup_eur_kwh` on top of it is
|
||||
the defect this ticket's design exists to make unreachable; the declaration
|
||||
above is what stops it, and nothing here needs to remember the rule.
|
||||
"""
|
||||
|
||||
id = "engie_dynamic"
|
||||
|
||||
@classmethod
|
||||
def capabilities(cls):
|
||||
return ProviderCapabilities(
|
||||
product=PriceProduct.DAY_AHEAD,
|
||||
sources=frozenset({Component.ENERGY, Component.MARKUP}),
|
||||
needs_credentials=True,
|
||||
egress_hosts=("api.engie.be",))
|
||||
|
||||
async def fetch(self, opts, since, until, transport):
|
||||
url = opts.get("price_feed_url") or "https://api.engie.be/dynamic/be"
|
||||
check_egress(url, self.capabilities())
|
||||
body = await transport(url, float(opts.get("price_timeout_s")))
|
||||
return _points_from_json(body, "price_eur_kwh", 1.0, False)
|
||||
|
||||
|
||||
@register
|
||||
class TotalEnergiesDynamic(PriceProvider):
|
||||
"""TotalEnergies' dynamic day-ahead product, published in EUR/kWh.
|
||||
|
||||
SOURCES: energy, markup
|
||||
The framework ADDS: taxes
|
||||
|
||||
Same retail shape as Engie: the supplier's own margin is inside the number.
|
||||
"""
|
||||
|
||||
id = "totalenergies_dynamic"
|
||||
|
||||
@classmethod
|
||||
def capabilities(cls):
|
||||
return ProviderCapabilities(
|
||||
product=PriceProduct.DAY_AHEAD,
|
||||
sources=frozenset({Component.ENERGY, Component.MARKUP}),
|
||||
needs_credentials=True,
|
||||
egress_hosts=("api.totalenergies.be",))
|
||||
|
||||
async def fetch(self, opts, since, until, transport):
|
||||
url = opts.get("price_feed_url") or \
|
||||
"https://api.totalenergies.be/prices/dynamic"
|
||||
check_egress(url, self.capabilities())
|
||||
body = await transport(url, float(opts.get("price_timeout_s")))
|
||||
return _points_from_json(body, "price_eur_kwh", 1.0, False)
|
||||
|
||||
|
||||
@register
|
||||
class EnecoEddy(PriceProvider):
|
||||
"""Eneco Eddy / Eddy Flex - a REAL-TIME / IMBALANCE product, all-in EUR/MWh.
|
||||
|
||||
SOURCES: energy, markup, taxes
|
||||
The framework ADDS: nothing
|
||||
|
||||
⚠️ THE PRODUCT TYPE IS THE INTERESTING FIELD HERE, NOT THE PRICE. §9.2
|
||||
classes Eddy Flex as real-time / imbalance and §8.3 says those get widened
|
||||
arbitrage thresholds. That regime reaches the optimiser through
|
||||
`capabilities().product` -> `_Series.product` -> `volatility_regime()`. If
|
||||
it reached the optimiser through the string "eneco_eddy" instead, then the
|
||||
sixth supplier would need an optimiser edit and AC2 would be dead. This
|
||||
class is the reason the enum exists.
|
||||
|
||||
Imbalance prices are settled after the fact, so the current quarter's figure
|
||||
is genuinely provisional and the feed says so per point.
|
||||
"""
|
||||
|
||||
id = "eneco_eddy"
|
||||
|
||||
@classmethod
|
||||
def capabilities(cls):
|
||||
return ProviderCapabilities(
|
||||
product=PriceProduct.REALTIME,
|
||||
sources=ALL_IN,
|
||||
horizon_h=24,
|
||||
needs_credentials=True,
|
||||
egress_hosts=("api.eneco.be",))
|
||||
|
||||
async def fetch(self, opts, since, until, transport):
|
||||
url = opts.get("price_feed_url") or "https://api.eneco.be/eddy/prices"
|
||||
check_egress(url, self.capabilities())
|
||||
body = await transport(url, float(opts.get("price_timeout_s")))
|
||||
return _points_from_json(body, "price_eur_mwh", 0.001, True)
|
||||
|
||||
|
||||
@register
|
||||
class ManualCsv(PriceProvider):
|
||||
"""A file on disk. AC3: the documented fallback, fully offline.
|
||||
|
||||
@@ -71,12 +71,19 @@ So the flag is not passed through anywhere in this file:
|
||||
the pass-through idiom cannot come back in a later edit and pass review by
|
||||
looking ordinary.
|
||||
|
||||
⚠️ "PUBLISHED" DOES NOT MEAN "FIRM". `eneco_eddy` is an imbalance product whose
|
||||
current-quarter print is genuinely provisional and says so per point (§9.2,
|
||||
TARIFF-02). Such a point is published AND provisional, and the audit's rule for
|
||||
a published point is "the flag is whatever the feed said", never "False". A
|
||||
provenance check that forced published points to False would be a bug that
|
||||
looks like a guarantee.
|
||||
⚠️ "PUBLISHED" DOES NOT MEAN "FIRM". §9.3's canonical format carries
|
||||
`provisional` PER POINT, not per series, precisely because a real-time /
|
||||
imbalance product's current-interval print is settled after the fact and is
|
||||
genuinely provisional while it is published. Such a point is published AND
|
||||
provisional, and the audit's rule for a published point is "the flag is
|
||||
whatever the feed said", never "False". A provenance check that forced
|
||||
published points to False would be a bug that looks like a guarantee.
|
||||
|
||||
⚠️ NO SHIPPED PLUGIN EMITS ONE TODAY (TARIFF-07 withdrew the only declarer of
|
||||
`PriceProduct.REALTIME`), and that is exactly why this paragraph stays. The
|
||||
rule is a property of the FORMAT, not of any supplier; weakening it to
|
||||
"published implies False" while nothing exercises it is how the first real
|
||||
imbalance feed would arrive mispriced and flagged firm.
|
||||
|
||||
|
||||
THE FILL, AND WHY IT KEYS ON THE WALL CLOCK
|
||||
@@ -104,8 +111,9 @@ because an empty series is not constructible on purpose (price.py trap 3).
|
||||
|
||||
⚠️ A SUB-HOURLY FEED IS NOT DECIMATED TO ONE POINT AN HOUR, and the hourly walk
|
||||
below is about WHICH HOURS NEED FILLING - never about what a published hour is
|
||||
allowed to contain. `eneco_eddy` is the Belgian imbalance product and settles
|
||||
per QUARTER, `_validate_raw()` accepts any strictly-increasing spacing, and a
|
||||
allowed to contain. BELGIAN DAY-AHEAD ITSELF SETTLES PER QUARTER since the
|
||||
15-minute MTU took effect on 2025-10-01 - `generic_dayahead` reads 96 points a
|
||||
day, not 24 - `_validate_raw()` accepts any strictly-increasing spacing, and a
|
||||
published hour may therefore hold four points. So EVERY published point inside
|
||||
the window is passed through, in order, and a carried point is added only for a
|
||||
whole hour with NO published coverage at all.
|
||||
@@ -171,7 +179,7 @@ class Provenance(Enum):
|
||||
flag" and "the others".
|
||||
"""
|
||||
PUBLISHED = "published" # the provider printed this hour. Flag: the
|
||||
# feed's own - an imbalance print may be
|
||||
# feed's own - a real-time print may be
|
||||
# published AND provisional (§9.2).
|
||||
CARRIED = "carried" # yesterday's same-wall-clock hour (§8.3).
|
||||
# Flag: ALWAYS True, by construction.
|
||||
|
||||
@@ -107,7 +107,8 @@ THE FOUR RULES THIS FILE IS SHAPED BY
|
||||
|
||||
⚠️ NO VALUE-LEVEL CHECK CAN CATCH THAT, and saying so is more useful than a
|
||||
guarantee that would not hold. A carried point and a published-provisional
|
||||
point (`eneco_eddy`'s current quarter, §9.2) are the same three fields with
|
||||
point (a real-time product's current interval, §9.2) are the same three
|
||||
fields with
|
||||
the same flag; the difference is provenance, and provenance is not in the
|
||||
value. So the mechanism is ORDERING, not validation: `carry()` is the only
|
||||
public entry point, it remembers the argument and returns the fill, and the
|
||||
|
||||
@@ -13,7 +13,7 @@ is an EMS for a minority." The §9.2 product table maps four products onto four
|
||||
EMS behaviours, and TARIFF-02 shipped the machinery for the top two only:
|
||||
|
||||
Dynamic day-ahead hourly EPEX BE + markup full arbitrage
|
||||
Real-time/imbalance e.g. Eneco Eddy Flex arbitrage, wider thresholds
|
||||
Real-time/imbalance no shipped plugin (TARIFF-07) arbitrage, wider thresh.
|
||||
Static day/night two-period tariff TIME-WINDOW SHIFTING <- here
|
||||
Fully fixed single price SELF-CONSUMPTION ONLY <- here
|
||||
|
||||
@@ -1082,7 +1082,8 @@ async def resolve_tariff(opts: dict, since: datetime, until: datetime,
|
||||
|
||||
⚠️ THE FILL IS PER WHOLE HOUR AND ITS COVERAGE IS CONDITIONAL. Every point
|
||||
the provider published inside the window passes through BY IDENTITY - four
|
||||
of them in one hour on a quarter-hourly product like `eneco_eddy` - and a
|
||||
of them in one hour on the quarter-hourly day-ahead feed `generic_dayahead`
|
||||
reads (Belgium's 15-minute MTU, 2025-10-01) - and a
|
||||
carried point is added only for a whole hour the feed did not print at all
|
||||
AND yesterday did. An hour neither source can price is left out, which is
|
||||
§11.4 for that hour: `plan.price_at()` raises rather than answering, and an
|
||||
|
||||
@@ -118,8 +118,14 @@ NOT READY banner already names the problem, (b) does not apply.
|
||||
It is warned about below, not refused.
|
||||
|
||||
7. A price configuration that cannot mean what it says. [(a)]
|
||||
Three cases, all in `_price_problems()`: an unknown `price_provider` id
|
||||
(refused rather than defaulted to 'off', per the §14 warning box);
|
||||
Four cases, all in `_price_problems()`: an unknown `price_provider` id
|
||||
(refused rather than defaulted to 'off', per the §14 warning box); a
|
||||
RETIRED id - one this add-on really did ship and has been accepting ever
|
||||
since, whose plugin never once fetched a price because its endpoint does
|
||||
not exist (TARIFF-07), which is case (a) in its purest form: §11.4's
|
||||
silent fall back to the static profile IS the layer the operator cannot
|
||||
see, and it has been overriding their chosen source since the day they
|
||||
set it;
|
||||
`price_provider` set alongside `price_now_entity`/`price_avg_entity`,
|
||||
which is TWO PRICE SOURCES ON ONE SITE where the Home Assistant pair
|
||||
silently wins; and `manual_csv` with no `price_csv_path`, which turns the
|
||||
@@ -322,7 +328,7 @@ from .inverter import roster_entries
|
||||
from .maintenance import MAINT_OPTION_DEFAULTS
|
||||
from .p1 import is_enabled, validate_meter_options
|
||||
from .peakguard import PEAK_OPTION_DEFAULTS
|
||||
from .price import PRICE_OPTION_DEFAULTS, PROVIDERS
|
||||
from .price import PRICE_OPTION_DEFAULTS, PROVIDERS, RETIRED_PROVIDERS
|
||||
from .static_tariff import ProfileInvalid, parse_static_profile
|
||||
from .watchdog import SIGN_OPTION_DEFAULTS
|
||||
|
||||
@@ -1242,17 +1248,71 @@ def _price_problems(opts: dict) -> list[str]:
|
||||
without a price feed is an add-on that a Belgian fixed-tariff site - a large
|
||||
share of the installed base, per §9.2 - cannot run at all.
|
||||
|
||||
Four refusals and two log lines (TARIFF-03 added one refusal, one parse and
|
||||
two notices), and every refusal is still case (a) from this module's
|
||||
docstring: a source the operator SET being overridden by a layer they cannot
|
||||
see. The additions are `static_profile` alongside the Home Assistant price
|
||||
pair - the same silent-loser shape as provider-plus-pair, and a LIVE one -
|
||||
and a `static_profile` that does not tile the day, which would leave hours
|
||||
with no price at all.
|
||||
Five refusals and two log lines (TARIFF-03 added one refusal, one parse and
|
||||
two notices; TARIFF-07 added the retired-id one), and every refusal is still
|
||||
case (a) from this module's docstring: a source the operator SET being
|
||||
overridden by a layer they cannot see. The additions are `static_profile`
|
||||
alongside the Home Assistant price pair - the same silent-loser shape as
|
||||
provider-plus-pair, and a LIVE one - and a `static_profile` that does not
|
||||
tile the day, which would leave hours with no price at all.
|
||||
|
||||
⚠️ THE RETIRED-ID BRANCH IS NOT A FRIENDLIER SPELLING OF THE UNKNOWN-ID
|
||||
ONE, AND MUST NOT COLLAPSE INTO IT. They answer different questions. An
|
||||
unknown id is a TYPO: the operator meant something else and the fix is to
|
||||
spell it right. A retired id is a CHOICE THE ADD-ON ACCEPTED - it was in
|
||||
the schema, it was in the dropdown, it resolved to a class, and every
|
||||
startup since said nothing. What that operator needs is not "this id is
|
||||
wrong" but "this id has never once returned a price, and here is what your
|
||||
site has actually been running instead" - because §11.4 made the failure
|
||||
silent and NOTHING on their status page has ever distinguished it from a
|
||||
working provider that never finds a cheap hour. The retired branch is
|
||||
therefore FIRST, and it names the id, the endpoint, the observation, the
|
||||
tariff that has really been in force, and the move to make.
|
||||
"""
|
||||
problems: list[str] = []
|
||||
pid = str(opts.get("price_provider", "") or "off").strip() or "off"
|
||||
|
||||
if pid in RETIRED_PROVIDERS:
|
||||
# ⚠️ BEFORE the unknown-id branch, and the order is load-bearing: a
|
||||
# retired id is ALSO not in PROVIDERS, so the generic message would
|
||||
# win on a reordering and the operator would be told they mistyped
|
||||
# something they chose on purpose.
|
||||
# ⚠️ AND THE SECOND SENTENCE CHANGES WITH `static_profile`, because
|
||||
# "you have been on the static profile" is FALSE on a site that never
|
||||
# set one. §11.4 falls back to the static profile; with no profile to
|
||||
# fall back to, the site has had no tariff at all - a worse state, and
|
||||
# one this message must not paper over.
|
||||
been_running = (
|
||||
f"the static day/night tariff in static_profile"
|
||||
if str(opts.get("static_profile", "") or "").strip()
|
||||
else "NOTHING AT ALL - static_profile is empty too, so there was "
|
||||
"no tariff to fall back to either")
|
||||
problems.append(
|
||||
f"price_provider is {pid!r}, which WAS a shipped plugin and has "
|
||||
f"been WITHDRAWN (TARIFF-07). This is not a typo and you did not "
|
||||
f"mistype it - the add-on offered this id, accepted it, and has "
|
||||
f"said nothing at every startup since. "
|
||||
f"⚠️ IT HAS NEVER FETCHED A PRICE, HERE OR ANYWHERE: "
|
||||
f"{RETIRED_PROVIDERS[pid]}. Every fetch it ever made ended in "
|
||||
f"SourceUnavailable, and §11.4 turns that into a SILENT fall back "
|
||||
f"- so what this site has actually been running since the day the "
|
||||
f"id was set is {been_running}. Nothing you could see would have "
|
||||
f"differed from a working provider that simply never found a cheap "
|
||||
f"hour, which is why you are being told now rather than left on "
|
||||
f"it. WHAT TO DO: set price_provider to 'generic_dayahead' for the "
|
||||
f"wholesale EPEX BE day-ahead curve, to which the add-on adds your "
|
||||
f"price_markup_eur_kwh and price_taxes_eur_kwh - that APPROXIMATES "
|
||||
f"a Belgian dynamic tariff and cannot reproduce your supplier's "
|
||||
f"B x spot + A formula, so the level will be off even when the "
|
||||
f"shape is right; or 'manual_csv' to enter all-in prices you "
|
||||
f"compute yourself; or 'off' to run on static_profile "
|
||||
f"deliberately. There is deliberately NO alias to your old "
|
||||
f"supplier: aliasing would need your contract's own two "
|
||||
f"coefficients, which live behind your klantenzone login and are "
|
||||
f"in no public index, so the add-on would have to invent the "
|
||||
f"numbers it prices your battery against.")
|
||||
return problems # nothing below can be judged against a dead id
|
||||
|
||||
if pid != "off" and pid not in PROVIDERS:
|
||||
problems.append(
|
||||
f"price_provider is {pid!r}, which is not a shipped plugin. The "
|
||||
|
||||
@@ -284,9 +284,8 @@ options:
|
||||
#
|
||||
# markup/taxes are the §14 `tariffs.energy_price.*` figures and are added by
|
||||
# the FRAMEWORK, only for the components the chosen plugin declares its feed
|
||||
# does not already contain. engie_dynamic, totalenergies_dynamic, eneco_eddy
|
||||
# and manual_csv publish a figure that already carries the markup, so setting
|
||||
# it under those is ignored - and logged as ignored, never applied twice.
|
||||
# does not already contain. manual_csv carries an all-in figure, so setting
|
||||
# either under it is ignored - and logged as ignored, never applied twice.
|
||||
price_provider: "off"
|
||||
price_markup_eur_kwh: 0.03
|
||||
price_taxes_eur_kwh: 0.06
|
||||
@@ -758,8 +757,31 @@ schema:
|
||||
# Supervisor rejects a typo in the UI. validate.py ALSO refuses an unknown id
|
||||
# against app/price.py's PROVIDERS registry, because sim/inject.py and the
|
||||
# checks reach validate_startup_config() with a dict the Supervisor never saw.
|
||||
# Adding a sixth plugin means adding it here and to PROVIDERS - and nowhere
|
||||
# else, which is AC2.
|
||||
# Adding a plugin means adding it here and to PROVIDERS - and nowhere else,
|
||||
# which is AC2.
|
||||
#
|
||||
# ⚠️ THREE OF THESE FIVE IDS NO LONGER RESOLVE TO A PLUGIN, AND LEAVING THEM
|
||||
# HERE IS A DECISION (TARIFF-07). engie_dynamic, totalenergies_dynamic and
|
||||
# eneco_eddy were withdrawn - their endpoints do not exist and never did - and
|
||||
# the obvious edit is to delete them from this line too. DO NOT. The Supervisor
|
||||
# validates the options file against this schema BEFORE the add-on starts, so
|
||||
# an id that is not listed here makes an EXISTING config schema-invalid: the
|
||||
# operator gets an opaque "expected one of ..." from the Supervisor, the add-on
|
||||
# never runs, and validate.py's retired-id refusal - the message that tells
|
||||
# them their prices never worked and what to do instead - IS NEVER REACHED, on
|
||||
# exactly the sites that need it. Schema tightness would cost the only thing
|
||||
# this withdrawal is FOR.
|
||||
#
|
||||
# The cost of keeping them is that the dropdown still offers three dead ids to
|
||||
# a NEW operator. That is the cheaper failure: choosing one is refused at the
|
||||
# next startup by name, with the endpoint, the observation and the alternative
|
||||
# - which is strictly more than a typo-shaped schema rejection would say. Both
|
||||
# translations label them withdrawn in the option description as well.
|
||||
#
|
||||
# app/price.py's RETIRED_PROVIDERS is the list, test_price.py section 1 asserts
|
||||
# this line offers exactly PROVIDERS + RETIRED_PROVIDERS + 'off' - derived, not
|
||||
# typed - and register() refuses to re-register a retired id, so the two
|
||||
# cannot both claim one.
|
||||
price_provider: list(off|generic_dayahead|engie_dynamic|totalenergies_dynamic|eneco_eddy|manual_csv)
|
||||
price_markup_eur_kwh: float(0,1)
|
||||
price_taxes_eur_kwh: float(0,1)
|
||||
|
||||
@@ -1695,6 +1695,312 @@ check("soc_bounds() returns the shipped pair, floor first",
|
||||
float(PRICE_OPTION_DEFAULTS["arbitrage_soc_ceiling"])))
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
print()
|
||||
print("10. TARIFF-14 - the QUARTER-HOURLY day, end to end through decide()")
|
||||
# ⚠️ THIS IS THE PROPERTY THE DEFECT ACTUALLY BROKE, WHICH IS WHY IT IS GRADED
|
||||
# HERE AND NOT ONLY ON `at()`. `_act()` decides whether this interval is the
|
||||
# buy one or the sell one with `here.start == v.cheapest.start` - an IDENTITY
|
||||
# test between the point `_point_at()` matched and the point the verdict was
|
||||
# built from. `_Series.at()` used to return the first point whose start was
|
||||
# within one hour BEFORE the instant, in list order, so on a 900 s series that
|
||||
# identity almost never held: the strategy idled through exactly the interval
|
||||
# it exists to trade, and read `here.provisional` and §17's `price < 0` off a
|
||||
# point that was not the one being traded.
|
||||
#
|
||||
# Belgium moved to a 15-minute MTU on 2025-10-01 and the live day-ahead feed
|
||||
# returns 96 points at 900 s. `static_tariff.py` emits HOURLY points, so every
|
||||
# fixture above this line - and every static site - is immune; this is
|
||||
# reachable only down the dynamic path TARIFF-08/09 made work.
|
||||
|
||||
QD0 = datetime(2026, 1, 15, 0, 0, tzinfo=timezone(timedelta(hours=1)))
|
||||
QUARTER = timedelta(minutes=15)
|
||||
|
||||
|
||||
def qplan_of(prices, *, provisional=False, start=QD0, gaps=None):
|
||||
"""A TariffPlan over `prices` spaced by `gaps` seconds (900 s by default).
|
||||
|
||||
`gaps` is what makes the MIXED fixture below expressible: FCST-04's
|
||||
carry-forward emits HOURLY carried points beside QUARTER-HOURLY firm ones,
|
||||
and the soak measured spacing [900, 3600] in 70 % of samples.
|
||||
"""
|
||||
g = [900] * (len(prices) - 1) if gaps is None else list(gaps)
|
||||
flags = ([bool(provisional)] * len(prices)
|
||||
if isinstance(provisional, bool) else list(provisional))
|
||||
pts, t = [], start
|
||||
for i, (p, f) in enumerate(zip(prices, flags)):
|
||||
pts.append(PricePoint(start=brussels(t), eur_per_kwh=round(float(p), 6),
|
||||
provisional=bool(f)))
|
||||
if i < len(g):
|
||||
t += timedelta(seconds=g[i])
|
||||
return TariffPlan(PriceRead.series(pts, PriceProduct.DAY_AHEAD,
|
||||
"generic_dayahead", start),
|
||||
None, "provider")
|
||||
|
||||
|
||||
def qprobe(i, *, start=QD0, gaps=None):
|
||||
"""An instant 7 minutes into slot `i` - inside every window, never on a
|
||||
boundary, and NOT the midpoint of an hour either, so an implementation that
|
||||
answered per hour and one that answers per quarter differ here."""
|
||||
g = [900] * i if gaps is None else list(gaps)[:i]
|
||||
return start + timedelta(seconds=sum(g)) + timedelta(minutes=7)
|
||||
|
||||
|
||||
def shipped_at(series, u):
|
||||
"""THE RULE TARIFF-14 REPLACED, kept here as the COUNTERFACTUAL ONLY.
|
||||
|
||||
The first point whose start is within one hour before `u`, in list order.
|
||||
Nothing below is graded against it; it exists so that the fixtures can be
|
||||
shown to SEPARATE the two rules. A check whose data cannot reach the
|
||||
failure it names is decoration (MUTATION.md), and on an hourly series the
|
||||
two rules agree at every instant - which is why 29 files stayed green.
|
||||
"""
|
||||
uu = u.astimezone(UTC)
|
||||
for p in series.points:
|
||||
s = p.start.astimezone(UTC)
|
||||
if s <= uu < s + timedelta(hours=1):
|
||||
return p
|
||||
return None
|
||||
|
||||
|
||||
# The hourly day interpolated across its quarters, with the two extremes
|
||||
# planted OFF the hour boundary on purpose.
|
||||
QBASE = []
|
||||
for _i in range(24):
|
||||
_a, _b = DAY[_i], DAY[(_i + 1) % 24]
|
||||
for _q in range(4):
|
||||
QBASE.append(round(_a + (_b - _a) * _q / 4 + 0.000001 * (_i * 4 + _q),
|
||||
6))
|
||||
Q_CHEAP_I = 4 * 5 + 3 # 05:45 - a :45 quarter, not a whole hour
|
||||
Q_DEAR_I = 4 * 19 + 1 # 19:15 - a :15 quarter, not a whole hour
|
||||
QDAY = list(QBASE)
|
||||
QDAY[Q_CHEAP_I] = round(min(QBASE) - 0.030, 6)
|
||||
QDAY[Q_DEAR_I] = round(max(QBASE) + 0.030, 6)
|
||||
QPLAN = qplan_of(QDAY)
|
||||
QPTS = QPLAN.read.points
|
||||
|
||||
print("10a. the fixture is not degenerate, and it SEPARATES the two rules")
|
||||
_qgaps = sorted({int((b.start - a.start).total_seconds())
|
||||
for a, b in zip(QPTS, QPTS[1:])})
|
||||
check(f"the day is 96 points at {_qgaps} s - the shape the live feed has "
|
||||
f"returned since Belgium's 2025-10-01 move to a 15-minute MTU",
|
||||
len(QPTS) == 96 and _qgaps == [900])
|
||||
check(f"the cheapest quarter is slot {Q_CHEAP_I} and the dearest slot "
|
||||
f"{Q_DEAR_I} - neither first, neither last, {abs(Q_CHEAP_I - Q_DEAR_I)} "
|
||||
f"slots apart, and NEITHER ON A WHOLE HOUR, without which this section "
|
||||
f"would grade nothing an hourly fixture does not already grade",
|
||||
QDAY.index(min(QDAY)) == Q_CHEAP_I and QDAY.index(max(QDAY)) == Q_DEAR_I
|
||||
and Q_CHEAP_I % 4 != 0 and Q_DEAR_I % 4 != 0
|
||||
and Q_CHEAP_I not in (0, 95) and Q_DEAR_I not in (0, 95)
|
||||
and abs(Q_CHEAP_I - Q_DEAR_I) > 1)
|
||||
check("...and all 96 prices are distinct, so 'the cheapest quarter' names one "
|
||||
"quarter and no tie-break is what any check below is really grading",
|
||||
len(set(QDAY)) == 96)
|
||||
check(f"...and its spread is {round(max(QDAY) - min(QDAY), 6):.4f} EUR/kWh, "
|
||||
f"clear of the widened threshold, so what follows grades the strategy "
|
||||
f"and not the spread rung",
|
||||
round(max(QDAY) - min(QDAY), 6) > required_spread(_shipped, NO_WEATHER))
|
||||
# ⚠️ HABIT 4, AND THE ONLY REASON THIS SECTION IS WORTH ITS LINES. State, as a
|
||||
# check, that the shipped rule and the fixed one give DIFFERENT points at the
|
||||
# two probes that matter. Without it, every check below could be passing on a
|
||||
# fixture the defect could not have changed.
|
||||
_cf_buy = derived(lambda: shipped_at(QPLAN.read, qprobe(Q_CHEAP_I)))
|
||||
_cf_sell = derived(lambda: shipped_at(QPLAN.read, qprobe(Q_DEAR_I)))
|
||||
check(f"the shipped rule and this one give DIFFERENT points at both probes - "
|
||||
f"buy: {derived(lambda: _cf_buy.eur_per_kwh)} instead of "
|
||||
f"{QDAY[Q_CHEAP_I]}, sell: {derived(lambda: _cf_sell.eur_per_kwh)} "
|
||||
f"instead of {QDAY[Q_DEAR_I]} - so the fixture can reach the failure it "
|
||||
f"names",
|
||||
_cf_buy is not QPTS[Q_CHEAP_I] and _cf_sell is not QPTS[Q_DEAR_I]
|
||||
and _cf_buy is not None and _cf_sell is not None)
|
||||
|
||||
print()
|
||||
print("10b. the buy/sell identity test matches the actual extremum interval")
|
||||
_qv = QPLAN.arbitrage(peak_guard_active=False, min_spread_eur_kwh=_shipped,
|
||||
uncertainty=NO_WEATHER)
|
||||
check("the verdict's cheapest and dearest are the two QUARTERS, not the hours "
|
||||
"containing them - `static_tariff` reduces over every firm point and "
|
||||
"carries no resolution assumption of its own",
|
||||
derived(lambda: _qv.cheapest) is QPTS[Q_CHEAP_I]
|
||||
and derived(lambda: _qv.dearest) is QPTS[Q_DEAR_I])
|
||||
_pa_buy = derived(lambda: _arb._point_at(QPLAN, qprobe(Q_CHEAP_I)))
|
||||
_pa_sell = derived(lambda: _arb._point_at(QPLAN, qprobe(Q_DEAR_I)))
|
||||
check("THE PROPERTY THE DEFECT BROKE: at the buy probe `_point_at()` returns "
|
||||
"the verdict's OWN cheapest point, by identity - which is what makes "
|
||||
"`_act()`'s `here.start == v.cheapest.start` true, and what made it "
|
||||
"false on every quarter-hourly series the add-on has ever fetched",
|
||||
_pa_buy is derived(lambda: _qv.cheapest))
|
||||
check("...and the same at the sell probe, against the verdict's dearest",
|
||||
_pa_sell is derived(lambda: _qv.dearest))
|
||||
_qbuy = decide(QPLAN, qprobe(Q_CHEAP_I), peak_guard_active=False, soc=50.0,
|
||||
min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
|
||||
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
|
||||
soc_ceiling_pct=95.0)
|
||||
_qsell = decide(QPLAN, qprobe(Q_DEAR_I), peak_guard_active=False, soc=50.0,
|
||||
min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
|
||||
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
|
||||
soc_ceiling_pct=95.0)
|
||||
check("...so decide() CHARGES in the cheapest quarter and DISCHARGES in the "
|
||||
"dearest one, which is the trade the strategy exists to make and the "
|
||||
"one it slept through",
|
||||
_qbuy.action == CHARGE and _qsell.action == DISCHARGE)
|
||||
check(f"...at THAT QUARTER's price and not at the price of the hour "
|
||||
f"containing it (buy {_qbuy.price_now_eur_kwh}, sell "
|
||||
f"{_qsell.price_now_eur_kwh})",
|
||||
near(_qbuy.price_now_eur_kwh, QDAY[Q_CHEAP_I])
|
||||
and near(_qsell.price_now_eur_kwh, QDAY[Q_DEAR_I]))
|
||||
check("...and both reasons quote that quarter's price, so the operator's log "
|
||||
"names the interval that was actually traded",
|
||||
f"{QDAY[Q_CHEAP_I]:.4f}" in _qbuy.reason
|
||||
and f"{QDAY[Q_DEAR_I]:.4f}" in _qsell.reason)
|
||||
# The negative arm. Without it "it acted" could be "it acts everywhere".
|
||||
_QOFF_I = Q_CHEAP_I + 2
|
||||
check(f"the off-peak probe's own price differs from its hour's :00 price "
|
||||
f"({QDAY[_QOFF_I]} vs {QDAY[4 * (_QOFF_I // 4)]}), so the check below "
|
||||
f"can tell which point the decision read",
|
||||
QDAY[_QOFF_I] != QDAY[4 * (_QOFF_I // 4)]
|
||||
and _QOFF_I not in (Q_CHEAP_I, Q_DEAR_I))
|
||||
_qoff = decide(QPLAN, qprobe(_QOFF_I), peak_guard_active=False, soc=50.0,
|
||||
min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
|
||||
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
|
||||
soc_ceiling_pct=95.0)
|
||||
check("a quarter that is neither extremum still idles, for the reason that "
|
||||
"names the right rule, and reports ITS OWN price - not the price of the "
|
||||
"quarter the shipped scan would have matched",
|
||||
_qoff.action is None and _qoff.reason == _arb.IDLE_OFF_PEAK_HOUR
|
||||
and near(_qoff.price_now_eur_kwh, QDAY[_QOFF_I]))
|
||||
|
||||
print()
|
||||
print("10c. §17 and §8.3 are evaluated on the QUARTER being traded")
|
||||
# ⚠️ §17 AT 900 s. The whole day is below zero and the dearest QUARTER is the
|
||||
# least negative one - section 2's shape, at the resolution the live feed
|
||||
# actually publishes. Under the shipped scan this probe never reached the
|
||||
# guard at all: the identity test failed first and the decision idled as
|
||||
# "neither the buy hour nor the sell hour", so §17 was not so much wrong as
|
||||
# never consulted.
|
||||
_shift = round(max(QDAY) + 0.020, 6)
|
||||
QNEG = [round(p - _shift, 6) for p in QDAY]
|
||||
QNEG_PLAN = qplan_of(QNEG)
|
||||
check(f"the negative day's dearest quarter is slot {Q_DEAR_I} and it is BELOW "
|
||||
f"ZERO ({max(QNEG):+.4f}), and its spread still clears the widened "
|
||||
f"threshold - so the refusal below is §17's and not the spread rung's",
|
||||
QNEG.index(max(QNEG)) == Q_DEAR_I and max(QNEG) < 0
|
||||
and round(max(QNEG) - min(QNEG), 6) > required_spread(_shipped,
|
||||
NO_WEATHER))
|
||||
_qneg_sell = decide(QNEG_PLAN, qprobe(Q_DEAR_I), peak_guard_active=False,
|
||||
soc=50.0, min_spread_eur_kwh=_shipped,
|
||||
uncertainty=NO_WEATHER, charge_w=MAX_W, discharge_w=MAX_W,
|
||||
soc_floor_pct=20.0, soc_ceiling_pct=95.0)
|
||||
check("selling into a negative-price QUARTER is refused, and the reason names "
|
||||
"§17 and THAT QUARTER's price",
|
||||
_qneg_sell.action is None and "§17" in _qneg_sell.reason
|
||||
and f"{max(QNEG):+.4f}" in _qneg_sell.reason)
|
||||
# ...and the other side, so the refusal is about the sign and nothing else.
|
||||
QPOS = list(QNEG)
|
||||
QPOS[Q_DEAR_I] = round(-max(QNEG), 6)
|
||||
check("...while the SAME quarter-hourly series with that ONE price's SIGN "
|
||||
"FLIPPED discharges instead",
|
||||
decide(qplan_of(QPOS), qprobe(Q_DEAR_I), peak_guard_active=False,
|
||||
soc=50.0, min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
|
||||
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
|
||||
soc_ceiling_pct=95.0).action == DISCHARGE
|
||||
and sum(1 for a, b in zip(QNEG, QPOS) if a != b) == 1)
|
||||
# §8.3's flag, read off the covering point. `_act`'s own provisional guard
|
||||
# stays unreachable here for section 4's reason - the extrema are firm by
|
||||
# `_firm_or_die` and starts are unique - and THAT ARGUMENT IS EXACTLY WHAT THE
|
||||
# DEFECT BROKE at 900 s, because the point `at()` returned was not the
|
||||
# verdict's point at all. It is restated here at the resolution that matters.
|
||||
check("the point matched for the buy quarter IS the verdict's own cheapest "
|
||||
"point and is FIRM, so `_act`'s §8.3 guard is unreachable at 900 s for "
|
||||
"the same construction section 4 states at 3600 s",
|
||||
_pa_buy is derived(lambda: _qv.cheapest)
|
||||
and derived(lambda: _pa_buy.provisional) is False
|
||||
and len({p.start for p in QPTS}) == len(QPTS))
|
||||
|
||||
print()
|
||||
print("10d. the MIXED series FCST-04 actually emits, traded end to end")
|
||||
# ⚠️ 70 % OF SOAK SAMPLES, NOT AN EDGE CASE. The carry-forward emits HOURLY
|
||||
# carried points beside QUARTER-HOURLY firm ones, so a window derived from one
|
||||
# global spacing would be wrong in the NORMAL steady state. Here: 12 carried
|
||||
# hourly points for the morning, then firm quarters for the rest of the day.
|
||||
MIX_N_H, MIX_N_Q = 12, 48
|
||||
MIX_GAPS = [3600] * MIX_N_H + [900] * (MIX_N_Q - 1)
|
||||
MIX_PRICES = [QDAY[4 * i] for i in range(MIX_N_H)] + QDAY[4 * MIX_N_H:]
|
||||
MIX_FLAGS = [True] * MIX_N_H + [False] * MIX_N_Q
|
||||
MIX_PLAN = qplan_of(MIX_PRICES, provisional=MIX_FLAGS, gaps=MIX_GAPS)
|
||||
MIX_PTS = MIX_PLAN.read.points
|
||||
_mix_gaps = sorted({int((b.start - a.start).total_seconds())
|
||||
for a, b in zip(MIX_PTS, MIX_PTS[1:])})
|
||||
check(f"the mixed plan really is mixed - spacing {_mix_gaps} s, "
|
||||
f"{MIX_N_H} carried and {MIX_N_Q} firm points, so neither half is what "
|
||||
f"the other half's window rule would give",
|
||||
_mix_gaps == [900, 3600]
|
||||
and sum(1 for p in MIX_PTS if p.provisional) == MIX_N_H
|
||||
and sum(1 for p in MIX_PTS if not p.provisional) == MIX_N_Q)
|
||||
_mix_cheap_i = MIX_N_H + (MIX_PRICES[MIX_N_H:].index(min(MIX_PRICES[MIX_N_H:])))
|
||||
_mix_dear_i = MIX_N_H + (MIX_PRICES[MIX_N_H:].index(max(MIX_PRICES[MIX_N_H:])))
|
||||
check(f"...and its firm half's extremes are quarters {_mix_cheap_i} and "
|
||||
f"{_mix_dear_i}, neither on a whole hour, so the identity test below is "
|
||||
f"about the quarter and not about the hour",
|
||||
(_mix_cheap_i - MIX_N_H) % 4 != 0 and (_mix_dear_i - MIX_N_H) % 4 != 0)
|
||||
_mv = MIX_PLAN.arbitrage(peak_guard_active=False, min_spread_eur_kwh=_shipped,
|
||||
uncertainty=NO_WEATHER)
|
||||
_mix_buy_t = qprobe(_mix_cheap_i, gaps=MIX_GAPS)
|
||||
_mix_sell_t = qprobe(_mix_dear_i, gaps=MIX_GAPS)
|
||||
check("the verdict over a MIXED series still names firm quarters, and "
|
||||
"`_point_at()` matches them by identity at their own probes",
|
||||
derived(lambda: _mv.cheapest) is MIX_PTS[_mix_cheap_i]
|
||||
and derived(lambda: _mv.dearest) is MIX_PTS[_mix_dear_i]
|
||||
and derived(lambda: _arb._point_at(MIX_PLAN, _mix_buy_t))
|
||||
is MIX_PTS[_mix_cheap_i]
|
||||
and derived(lambda: _arb._point_at(MIX_PLAN, _mix_sell_t))
|
||||
is MIX_PTS[_mix_dear_i])
|
||||
check("...so the mixed day CHARGES and DISCHARGES in its firm quarters, at "
|
||||
"their own prices",
|
||||
decide(MIX_PLAN, _mix_buy_t, peak_guard_active=False, soc=50.0,
|
||||
min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
|
||||
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
|
||||
soc_ceiling_pct=95.0).action == CHARGE
|
||||
and decide(MIX_PLAN, _mix_sell_t, peak_guard_active=False, soc=50.0,
|
||||
min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
|
||||
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
|
||||
soc_ceiling_pct=95.0).action == DISCHARGE)
|
||||
# An instant inside a CARRIED hour of the same series. The covering point is
|
||||
# the carried one; the shipped scan answered a point from the FIRM block
|
||||
# earlier in the list, which is §8.3's flag read off the wrong point.
|
||||
_mix_carried_t = QD0 + timedelta(hours=MIX_N_H - 1, minutes=37)
|
||||
check("an instant inside a CARRIED hour of the mixed series matches the "
|
||||
"CARRIED point, whose provisional flag is what §8.3 reads - and the "
|
||||
"strategy does nothing there, because a carried point is not in the "
|
||||
"firm trade set",
|
||||
derived(lambda: _arb._point_at(MIX_PLAN, _mix_carried_t))
|
||||
is MIX_PTS[MIX_N_H - 1]
|
||||
and MIX_PTS[MIX_N_H - 1].provisional is True
|
||||
and decide(MIX_PLAN, _mix_carried_t, peak_guard_active=False, soc=50.0,
|
||||
min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
|
||||
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
|
||||
soc_ceiling_pct=95.0).action is None)
|
||||
|
||||
print()
|
||||
print("10e. the HOURLY day is unchanged, which is the regression half")
|
||||
# ⚠️ A REGRESSION GUARD, NOT A WITNESS. On an hourly series the shipped rule
|
||||
# and this one agree at every instant, which is precisely why 29 files stayed
|
||||
# green through the defect. It is here so that a fix which got 900 s right by
|
||||
# breaking 3600 s cannot pass.
|
||||
_h_buy = act(PLAN, CHEAP_H, thresh=_shipped, unc=NO_WEATHER)
|
||||
_h_sell = act(PLAN, DEAR_H, thresh=_shipped, unc=NO_WEATHER)
|
||||
check("the hourly fixture still charges in its cheapest hour and discharges "
|
||||
"in its dearest, at those hours' prices",
|
||||
_h_buy.action == CHARGE and _h_sell.action == DISCHARGE
|
||||
and near(_h_buy.price_now_eur_kwh, DAY[CHEAP_H])
|
||||
and near(_h_sell.price_now_eur_kwh, DAY[DEAR_H]))
|
||||
check("...and a probe 59 minutes into an hourly slot still matches that slot, "
|
||||
"so the hourly window did not shrink under the fix",
|
||||
derived(lambda: _arb._point_at(
|
||||
PLAN, D0 + timedelta(hours=CHEAP_H, minutes=59)))
|
||||
is derived(lambda: PLAN.read.points[CHEAP_H]))
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
print()
|
||||
if fails:
|
||||
|
||||
+648
-103
@@ -59,6 +59,7 @@ from textwrap import dedent
|
||||
|
||||
from app.price import (
|
||||
ALL_IN, Absence, BANNED_IN_PLUGIN_CODE, Component, FrameworkOption,
|
||||
RETIRED_PROVIDERS,
|
||||
MAX_PLAUSIBLE_EUR_KWH, code_leaks,
|
||||
PRICE_OPTION_DEFAULTS, PROVIDERS, PluginOptions, PricePoint, PriceProduct,
|
||||
PriceProvider, PriceRead, ProviderCapabilities, RawPoint, SourceUnavailable,
|
||||
@@ -66,7 +67,7 @@ from app.price import (
|
||||
brussels_offset, check_egress, dated_feed_url,
|
||||
price_option,
|
||||
export_price_eur_kwh, load_prices, plan_is_provisional, register,
|
||||
volatility_regime,
|
||||
volatility_regime, uncertainty_of, widen_factor, Uncertainty,
|
||||
)
|
||||
from app import price as _price
|
||||
from app import arbitrage as _arb
|
||||
@@ -263,13 +264,31 @@ _unschemad = sorted(k for k in PRICE_OPTION_DEFAULTS
|
||||
if not any(ln.startswith(f" {k}:") for ln in _schema_block))
|
||||
check("...and every one of them is in the schema block too", _unschemad == [])
|
||||
# ⚠️ Derived from the registry, not typed: the schema's provider list must offer
|
||||
# exactly the plugins that exist, or the UI and the code disagree about what is
|
||||
# installable.
|
||||
# exactly the plugins that exist PLUS the retired ids, or the UI and the code
|
||||
# disagree about what is installable.
|
||||
#
|
||||
# ⚠️ THE RETIRED IDS ARE IN THE SCHEMA ON PURPOSE AND THIS IS THE CHECK THAT
|
||||
# KEEPS THEM THERE (TARIFF-07). The Supervisor validates the options file
|
||||
# against this line BEFORE the add-on runs. Drop a retired id from it and an
|
||||
# EXISTING config naming that id becomes schema-invalid: the operator gets an
|
||||
# opaque "expected one of ..." and validate.py's retired-id refusal - the whole
|
||||
# point of the withdrawal - is never reached, on exactly the sites that need
|
||||
# it. So the schema is deliberately WIDER than the registry, by exactly
|
||||
# RETIRED_PROVIDERS and by nothing else. Both halves are graded: an id that is
|
||||
# in neither set, and a retired id that has gone missing.
|
||||
_line = next(ln for ln in _schema_block if ln.startswith(" price_provider:"))
|
||||
_schema_ids = set(_line.split("list(", 1)[1].rstrip(")").split("|"))
|
||||
check(f"the schema offers exactly the registered plugins plus 'off' "
|
||||
f"{sorted(_schema_ids)}",
|
||||
_schema_ids == set(PROVIDERS) | {"off"})
|
||||
_wanted = set(PROVIDERS) | set(RETIRED_PROVIDERS) | {"off"}
|
||||
check(f"the schema offers exactly the registered plugins, the retired ids and "
|
||||
f"'off' {sorted(_schema_ids)}",
|
||||
_schema_ids == _wanted)
|
||||
check(f"...and every retired id is STILL offered, so a config already naming "
|
||||
f"one reaches validate.py rather than the Supervisor's schema error "
|
||||
f"{sorted(set(RETIRED_PROVIDERS) - _schema_ids) or 'none missing'}",
|
||||
set(RETIRED_PROVIDERS) <= _schema_ids and set(RETIRED_PROVIDERS) != set())
|
||||
check("...and no retired id is ALSO a live plugin, which would silently take "
|
||||
"that refusal away",
|
||||
set(RETIRED_PROVIDERS) & set(PROVIDERS) == set())
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
@@ -290,41 +309,42 @@ T0 = datetime(2026, 1, 15, 0, 0, tzinfo=UTC)
|
||||
# what it claims to.
|
||||
EPEX_EUR_MWH = [61.4, -95.0, 183.0] # EUR/MWh, one deeply negative
|
||||
RETAIL_EUR_KWH = [0.1341, 0.0872, 0.2610] # EUR/kWh, markup already inside
|
||||
EDDY_EUR_MWH = [141.0, 96.2, 268.4] # EUR/MWh, all-in
|
||||
|
||||
|
||||
def synthetic_row_feed(key, values, provisional_at=1):
|
||||
"""An INVENTED row-per-point body. `provisional_at` makes ONE point
|
||||
provisional and the rest not, so `plan_is_provisional` is graded against a
|
||||
mixed series - an all-true or all-false fixture would return the same
|
||||
answer under any bug.
|
||||
# ⚠️ THE INVENTED ROW-PER-POINT FIXTURE IS GONE, AND SO IS ITS PARSER
|
||||
# (TARIFF-07). Until this card every price fixture here was a
|
||||
# `{"data": [{"start": ..., "price_eur_mwh": ...}]}` document that
|
||||
# `_points_from_json()` was written to read and that NO FEED ANYBODY REACHED
|
||||
# HAS EVER PUBLISHED. The three plugins fed from it - `engie_dynamic`,
|
||||
# `totalenergies_dynamic`, `eneco_eddy` - had no captured body because their
|
||||
# endpoints do not exist, so the loop was closed twice over: an invented parser
|
||||
# graded against an invented body, on behalf of plugins that could never fetch.
|
||||
# Both plugins that remain are replayed from something real - `generic_dayahead`
|
||||
# from captured bytes (section 2b), `manual_csv` from the CSV a human types.
|
||||
#
|
||||
# The shape itself survives in ONE place and in one direction only: as the body
|
||||
# section 9 feeds `generic_dayahead` to prove that a row-per-point document is a
|
||||
# FAILURE for it rather than a second dialect it might sniff. It is named for
|
||||
# what it is.
|
||||
|
||||
⚠️ THE NAME SAYS `synthetic` BECAUSE TARIFF-08 IS WHAT HAPPENS WHEN IT DOES
|
||||
NOT. This shape - `{"data": [{"start": ..., "price_eur_mwh": ...}]}` - was
|
||||
invented by whoever wrote `_points_from_json()`, and until 2026-09-10 it
|
||||
was the ONLY shape any price check in this repository had ever fed a
|
||||
parser, `generic_dayahead` included. The public EPEX feed publishes
|
||||
parallel arrays and no `data` key at all, so every live fetch that plugin
|
||||
ever made raised `SourceUnavailable` while the suite stayed green: the
|
||||
checks were grading the parser against the parser's own assumption. §11.4
|
||||
then turned the failure into a quiet fall back to the static profile, so a
|
||||
site would have sat on it forever.
|
||||
|
||||
The three plugins still fed from here - `engie_dynamic`,
|
||||
`totalenergies_dynamic`, `eneco_eddy` - have NO captured body, because
|
||||
their endpoints are unverified (TARIFF-07 is the open decision on whether
|
||||
they are withdrawn). This fixture therefore grades that their DECLARATIONS
|
||||
and the framework arithmetic are right; it grades NOTHING about what those
|
||||
feeds publish, and it must not be read as evidence that they parse.
|
||||
`generic_dayahead` is replayed from real bytes instead - section 2b.
|
||||
def row_shape_body(key, values):
|
||||
"""A `{"data": [...]}` document. ⚠️ EXISTS ONLY TO BE REFUSED.
|
||||
|
||||
No feed this repository has ever reached publishes this, and the parser
|
||||
that read it was withdrawn with the three plugins it served (TARIFF-07).
|
||||
It is kept as a NEGATIVE stimulus: if `_points_from_columnar_json()` ever
|
||||
grows a sniff for `data`, section 9's "the body is the OTHER shape" row
|
||||
turns from `no_data` into a series and a check goes red, rather than
|
||||
TARIFF-08's scope rule being widened by a review nobody remembers.
|
||||
"""
|
||||
return json.dumps({"data": [
|
||||
{"start": (T0 + timedelta(hours=i)).isoformat(),
|
||||
key: v,
|
||||
"provisional": (i == provisional_at)}
|
||||
{"start": (T0 + timedelta(hours=i)).isoformat(), key: v,
|
||||
"provisional": False}
|
||||
for i, v in enumerate(values)]}).encode()
|
||||
|
||||
|
||||
|
||||
def columnar_feed(values, provisional_at=None, unit="EUR / MWh",
|
||||
step_min=60, start=None, prices=None):
|
||||
"""A body in the shape `api.energy-charts.info` really publishes.
|
||||
@@ -370,19 +390,36 @@ CSV_BODY = ("start,eur_per_kwh,provisional\n"
|
||||
FIXTURES = {
|
||||
"generic_dayahead": (columnar_feed(EPEX_EUR_MWH),
|
||||
[v / 1000 for v in EPEX_EUR_MWH]),
|
||||
"engie_dynamic": (synthetic_row_feed("price_eur_kwh", RETAIL_EUR_KWH),
|
||||
list(RETAIL_EUR_KWH)),
|
||||
"totalenergies_dynamic": (synthetic_row_feed("price_eur_kwh",
|
||||
RETAIL_EUR_KWH),
|
||||
list(RETAIL_EUR_KWH)),
|
||||
"eneco_eddy": (synthetic_row_feed("price_eur_mwh", EDDY_EUR_MWH),
|
||||
[v / 1000 for v in EDDY_EUR_MWH]),
|
||||
"manual_csv": (None, [0.1834, -0.0210, 0.3115]),
|
||||
}
|
||||
|
||||
check(f"all five plugins the ticket lists are registered {sorted(PROVIDERS)}",
|
||||
set(PROVIDERS) == {"generic_dayahead", "engie_dynamic",
|
||||
"totalenergies_dynamic", "eneco_eddy", "manual_csv"})
|
||||
check(f"the two plugins that survive TARIFF-07 are registered "
|
||||
f"{sorted(PROVIDERS)}",
|
||||
set(PROVIDERS) == {"generic_dayahead", "manual_csv"})
|
||||
check(f"...and the three withdrawn ids resolve to NOTHING "
|
||||
f"{sorted(RETIRED_PROVIDERS)}",
|
||||
set(RETIRED_PROVIDERS) == {"engie_dynamic", "totalenergies_dynamic",
|
||||
"eneco_eddy"}
|
||||
and not (set(RETIRED_PROVIDERS) & set(PROVIDERS)))
|
||||
def _readd_retired():
|
||||
return register(type("_ReAdd", (PriceProvider,), {
|
||||
"id": "eneco_eddy",
|
||||
"capabilities": classmethod(lambda c: ProviderCapabilities(
|
||||
product=PriceProduct.DAY_AHEAD,
|
||||
sources=frozenset({Component.ENERGY}))),
|
||||
"fetch": lambda s, o, a, b, t: None}))
|
||||
|
||||
|
||||
# ⚠️ GRADED ON THE REASON, NOT THE VERDICT. `register()` already refuses a
|
||||
# DUPLICATE id, so "it raised ValueError" would pass on a tree where the
|
||||
# retired guard is deleted and the id happens to be live again - the masking
|
||||
# shape MUTATION.md's TEST-16 section is about.
|
||||
_readd_why = _reason(_readd_retired)
|
||||
check(f"re-registering a retired id is refused, and refused for the RETIRED "
|
||||
f"reason rather than the duplicate one ({_readd_why[:60]!r})",
|
||||
"RETIRED_PROVIDERS" in _readd_why and "eneco_eddy" in _readd_why)
|
||||
check("...and the refusal did not take effect anyway",
|
||||
"eneco_eddy" not in PROVIDERS)
|
||||
check("...and the fixture set covers every one of them",
|
||||
set(FIXTURES) == set(PROVIDERS))
|
||||
|
||||
@@ -486,13 +523,13 @@ print("2b. TARIFF-08 - the SHIPPED parser, replayed against REAL captured "
|
||||
# checksums are RE-DERIVED here so a fixture edited to make a check pass stops
|
||||
# being a capture and starts being a failure.
|
||||
#
|
||||
# ⚠️ THE OTHER THREE NETWORKED PLUGINS HAVE NO CAPTURE AND THIS FILE DOES NOT
|
||||
# PRETEND OTHERWISE. `engie_dynamic`, `totalenergies_dynamic` and `eneco_eddy`
|
||||
# point at endpoints nobody has verified; TARIFF-07 is the open decision on
|
||||
# whether they are withdrawn. Their fixtures are named `synthetic_row_feed` so
|
||||
# no reader can mistake them for evidence that those feeds parse, and the check
|
||||
# below states, derived from the shipped `PROVIDERS` registry, exactly which
|
||||
# plugins are replayed from real bytes and which are not.
|
||||
# ⚠️ THERE IS NOW EXACTLY ONE NETWORKED PLUGIN, WHICH IS TARIFF-07's DOING.
|
||||
# `engie_dynamic`, `totalenergies_dynamic` and `eneco_eddy` had no capture
|
||||
# because their endpoints do not exist - verified by hand on 2026-09-13: a 404
|
||||
# from a gateway with no route, a TLS hostname mismatch, and a 404 with an empty
|
||||
# body. They were withdrawn rather than left fed by an invented fixture. The
|
||||
# check below is derived from the shipped `PROVIDERS` registry, so a plugin
|
||||
# added with no capture forces the question rather than inheriting silence.
|
||||
|
||||
_CAPTURES = json.load(open(
|
||||
os.path.join(HERE, "scenarios", "energy-charts-be-price-captures.json"),
|
||||
@@ -708,11 +745,10 @@ check("...and a feed announcing deprecated=true says so WHILE IT STILL WORKS - "
|
||||
_networked = sorted(pid for pid, cls in PROVIDERS.items()
|
||||
if cls.capabilities().egress_hosts)
|
||||
_replayed = ["generic_dayahead"]
|
||||
check(f"the networked plugins are {_networked}; {_replayed} is replayed from a "
|
||||
f"captured real body and {sorted(set(_networked) - set(_replayed))} have "
|
||||
f"no capture because their endpoints are unverified (TARIFF-07)",
|
||||
_networked == sorted(["generic_dayahead", "engie_dynamic",
|
||||
"totalenergies_dynamic", "eneco_eddy"]))
|
||||
check(f"the networked plugins are {_networked}, and EVERY one of them is "
|
||||
f"replayed from a captured real body ({_replayed}) - no shipped plugin "
|
||||
f"reaches a network this file has never seen answer (TARIFF-07)",
|
||||
_networked == _replayed)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
@@ -976,7 +1012,8 @@ def _uses_range(cls):
|
||||
_uses = sorted(pid for pid, cls in PROVIDERS.items() if _uses_range(cls))
|
||||
check(f"the plugins that CONSUME their `since`/`until` are {_uses}; the rest "
|
||||
f"take the range and ignore it, which is what TARIFF-09 fixed for "
|
||||
f"generic_dayahead and what TARIFF-07 has to decide for the others",
|
||||
f"generic_dayahead - and after TARIFF-07 the only one left that does not "
|
||||
f"is the offline plugin, for the honest reason below",
|
||||
_uses == ["generic_dayahead"])
|
||||
check("...and `manual_csv` is honestly among the ones that do not: it reads a "
|
||||
"whole file and the fill windows it downstream, so a range parameter "
|
||||
@@ -1001,28 +1038,45 @@ for pid, read in sorted(READS.items()):
|
||||
f"-> {got}",
|
||||
got == want)
|
||||
|
||||
# ⚠️ THE ONE THAT MATTERS. generic_dayahead is the only plugin whose feed is
|
||||
# wholesale, so it is the only one that gets both components - and the three
|
||||
# retail plugins must each get strictly less. Derived by comparing the two
|
||||
# plugins' output on the SAME energy figure rather than by restating numbers.
|
||||
# ⚠️ THE ONE THAT MATTERS. generic_dayahead is the only shipped plugin whose
|
||||
# feed is wholesale, so it is the only one that gets both components - and a
|
||||
# feed declaring more must get strictly less. Derived by comparing output on the
|
||||
# SAME energy figure rather than by restating numbers.
|
||||
#
|
||||
# ⚠️ THE MIDDLE RUNG - a feed sourcing {ENERGY, MARKUP}, which gets TAXES only -
|
||||
# HAS NO SHIPPED DECLARER SINCE TARIFF-07, and is graded through `assemble()`
|
||||
# with a hand-built declaration instead of being dropped. It is the only rung
|
||||
# that distinguishes "adds the complement" from "adds everything or nothing",
|
||||
# so deleting it with the three retail plugins would have left the partial case
|
||||
# ungraded on the exact function that must not get it wrong.
|
||||
def added(pid):
|
||||
return derived(lambda: round(READS[pid].points[0].eur_per_kwh
|
||||
- FIXTURES[pid][1][0], 6))
|
||||
|
||||
|
||||
_generic_add, _engie_add, _eddy_add = (added("generic_dayahead"),
|
||||
added("engie_dynamic"),
|
||||
added("eneco_eddy"))
|
||||
def added_for(sources):
|
||||
"""What assemble() adds to ONE point declared as sourcing `sources`."""
|
||||
caps = ProviderCapabilities(product=PriceProduct.DAY_AHEAD,
|
||||
sources=frozenset(sources))
|
||||
e = RETAIL_EUR_KWH[0]
|
||||
return derived(lambda: round(
|
||||
assemble(caps, [RawPoint(T0, e, False)], MARKUP, TAXES)[0].eur_per_kwh
|
||||
- e, 6))
|
||||
|
||||
|
||||
_generic_add = added("generic_dayahead")
|
||||
_csv_add = added("manual_csv")
|
||||
_retail_add = added_for({Component.ENERGY, Component.MARKUP})
|
||||
check(f"the wholesale feed gets markup+taxes ({_generic_add})",
|
||||
isinstance(_generic_add, float)
|
||||
and abs(_generic_add - (MARKUP + TAXES)) < 1e-9)
|
||||
check(f"a retail feed that already carries the markup gets taxes only "
|
||||
f"({_engie_add}), one markup less than the wholesale one",
|
||||
isinstance(_engie_add, float) and isinstance(_generic_add, float)
|
||||
and abs(_engie_add - TAXES) < 1e-9
|
||||
and abs((_generic_add - _engie_add) - MARKUP) < 1e-9)
|
||||
check(f"an all-in feed gets nothing added ({_eddy_add})",
|
||||
isinstance(_eddy_add, float) and abs(_eddy_add) < 1e-9)
|
||||
check(f"a feed that already carries the markup gets taxes ONLY "
|
||||
f"({_retail_add}), exactly one markup less than the wholesale one",
|
||||
isinstance(_retail_add, float) and isinstance(_generic_add, float)
|
||||
and abs(_retail_add - TAXES) < 1e-9
|
||||
and abs((_generic_add - _retail_add) - MARKUP) < 1e-9)
|
||||
check(f"an all-in feed gets nothing added ({_csv_add})",
|
||||
isinstance(_csv_add, float) and abs(_csv_add) < 1e-9)
|
||||
|
||||
# Negative prices survive. A framework that clamped them would turn the best
|
||||
# charging hour of the year into an average one, silently.
|
||||
@@ -1089,7 +1143,13 @@ class _Rogue(PriceProvider):
|
||||
return [RawPoint(T0, RETAIL_EUR_KWH[0], False)]
|
||||
|
||||
|
||||
_honest_caps = PROVIDERS["engie_dynamic"].capabilities()
|
||||
# The honest counterpart to _Rogue: the SAME declaration _Rogue makes, on a
|
||||
# feed that really does publish only energy. No shipped plugin is needed for it
|
||||
# and none is used - the comparison is between two declarations, not two
|
||||
# suppliers.
|
||||
_honest_caps = ProviderCapabilities(
|
||||
product=PriceProduct.DAY_AHEAD,
|
||||
sources=frozenset({Component.ENERGY, Component.MARKUP}))
|
||||
_honest = assemble(_honest_caps, [RawPoint(T0, RETAIL_EUR_KWH[0], False)],
|
||||
MARKUP, TAXES)[0].eur_per_kwh
|
||||
_rogue_raw = run(_Rogue().fetch(PluginOptions({}), T0, T0, None))
|
||||
@@ -1142,12 +1202,12 @@ check(f"every plugin's docstring names exactly the components it declares "
|
||||
_buf = io.StringIO()
|
||||
_h = logging.StreamHandler(_buf)
|
||||
logging.getLogger("goodwe.price").addHandler(_h)
|
||||
load_any("eneco_eddy")
|
||||
load_any("manual_csv")
|
||||
load_any("generic_dayahead")
|
||||
logging.getLogger("goodwe.price").removeHandler(_h)
|
||||
_warned = _buf.getvalue()
|
||||
check("an all-in plugin logs that the configured markup/taxes are ignored",
|
||||
"double-count" in _warned and "eneco_eddy" in _warned)
|
||||
"double-count" in _warned and "manual_csv" in _warned)
|
||||
check("...and the wholesale plugin, which really does use them, does not",
|
||||
"generic_dayahead" not in _warned)
|
||||
|
||||
@@ -1174,24 +1234,54 @@ _srclen = derived(lambda: {pid: len(inspect.getsource(cls))
|
||||
for pid, cls in PROVIDERS.items()})
|
||||
check(f"the scan really read every shipped plugin's source, so its fail-open "
|
||||
f"path is not silently in use {_srclen}",
|
||||
isinstance(_srclen, dict) and len(_srclen) == 5
|
||||
and all(n > 200 for n in _srclen.values()))
|
||||
isinstance(_srclen, dict) and set(_srclen) == set(PROVIDERS)
|
||||
and _srclen != {} and all(n > 200 for n in _srclen.values()))
|
||||
_dirty = derived(lambda: {pid: code_leaks(cls)
|
||||
for pid, cls in PROVIDERS.items()
|
||||
if code_leaks(cls)})
|
||||
check(f"...and all five shipped plugins are clean {_dirty}", _dirty == {})
|
||||
check(f"...and every shipped plugin is clean {_dirty}", _dirty == {})
|
||||
|
||||
# THE DOCSTRING STRIP IS LOAD-BEARING, AND THIS PROVES IT RATHER THAN ASSERTING
|
||||
# IT. Every plugin is REQUIRED to name its sourced components in prose, so
|
||||
# engie_dynamic's docstring contains "markup" - a raw text scan false-positives
|
||||
# on the documentation the same file mandates, and a check that fires on
|
||||
# required documentation is a check somebody deletes.
|
||||
_raw_hits = derived(lambda: sorted(
|
||||
pid for pid, cls in PROVIDERS.items()
|
||||
if any(b in inspect.getsource(cls) for b in _banned)))
|
||||
check(f"a RAW text scan would false-positive on the mandated docstrings "
|
||||
f"{_raw_hits}, which is why code_leaks() strips them before searching",
|
||||
isinstance(_raw_hits, list) and _raw_hits != [] and _dirty == {})
|
||||
# IT. Every plugin is REQUIRED to name its sourced components in prose, and a
|
||||
# plugin whose feed already carries the markup naturally writes the option key
|
||||
# out to say the framework must not add it again - which a RAW text scan reads
|
||||
# as a leak. A check that fires on the documentation the same file mandates is
|
||||
# a check somebody deletes.
|
||||
#
|
||||
# ⚠️ THE STIMULUS IS BUILT HERE NOW, AND SAYING SO IS THE POINT. Until
|
||||
# TARIFF-07 this asserted on the SHIPPED set, because `engie_dynamic`'s
|
||||
# docstring happened to spell `price_markup_eur_kwh` in full. It was withdrawn,
|
||||
# no remaining shipped docstring spells a full key, and grading the shipped set
|
||||
# would therefore now pass on a tree with the strip DELETED - a check that had
|
||||
# quietly stopped being one. So the false-positive is constructed, and the
|
||||
# shipped set is asserted to be clean separately, above.
|
||||
class _DocumentedPlugin(PriceProvider):
|
||||
"""A plugin documenting, correctly, that price_markup_eur_kwh is not added.
|
||||
|
||||
SOURCES: energy, markup
|
||||
The framework ADDS: taxes
|
||||
"""
|
||||
|
||||
id = "_documented_not_shipped"
|
||||
|
||||
@classmethod
|
||||
def capabilities(cls):
|
||||
return ProviderCapabilities(
|
||||
product=PriceProduct.DAY_AHEAD,
|
||||
sources=frozenset({Component.ENERGY, Component.MARKUP}))
|
||||
|
||||
async def fetch(self, o, since, until, transport):
|
||||
return [RawPoint(T0, RETAIL_EUR_KWH[0], False)]
|
||||
|
||||
|
||||
_raw_hit = derived(lambda: sorted(
|
||||
b for b in _banned if b in inspect.getsource(_DocumentedPlugin)))
|
||||
check(f"a RAW text scan false-positives on a mandated docstring {_raw_hit}, "
|
||||
f"which is why code_leaks() strips docstrings before searching - and on "
|
||||
f"the same class code_leaks() reports {code_leaks(_DocumentedPlugin)}",
|
||||
_raw_hit == ["price_markup_eur_kwh"]
|
||||
and code_leaks(_DocumentedPlugin) == []
|
||||
and _dirty == {})
|
||||
|
||||
|
||||
class _AttackPrivateDict(PriceProvider):
|
||||
@@ -1296,20 +1386,50 @@ check(f"a MIS-DECLARED all-in feed still doubles ({_misdeclared} vs "
|
||||
# --------------------------------------------------------------------------- #
|
||||
print()
|
||||
print("5. product type is declared, not inferred from the name (AC2, trap 2)")
|
||||
# ⚠️ NO SHIPPED PLUGIN DECLARES `PriceProduct.REALTIME`, AND THAT IS RECORDED
|
||||
# HERE RATHER THAN LEFT TO BE REDISCOVERED (TARIFF-07). `eneco_eddy` was its
|
||||
# sole declarer and it is withdrawn, so `_WIDE`, `Uncertainty.REALTIME_PRODUCT`
|
||||
# and `WIDEN_FACTOR`'s 2.5 have no shipped caller. They are KEPT - a reviewer
|
||||
# finding a mechanism with zero callers would reasonably delete it, and the
|
||||
# first real imbalance product would then have to rebuild the AC2 seam - and
|
||||
# this pin is what tells that reviewer the emptiness is intentional. See
|
||||
# DECISIONS/TARIFF-07-withdrawing-three-plugins.md.
|
||||
#
|
||||
# ⚠️ IT IS A PIN, NOT A PROHIBITION. If a plugin that really is real-time ever
|
||||
# ships, this check goes red and the right fix is to EDIT it - and the decision
|
||||
# record with it - not to delete the seam.
|
||||
_products = {pid: cls.capabilities().product for pid, cls in PROVIDERS.items()}
|
||||
check(f"eneco_eddy declares itself real-time/imbalance per §9.2 "
|
||||
f"({_products['eneco_eddy'].value})",
|
||||
_products["eneco_eddy"] is PriceProduct.REALTIME)
|
||||
check("...and it is the only shipped one that is",
|
||||
[p for p, v in _products.items() if v is PriceProduct.REALTIME]
|
||||
== ["eneco_eddy"])
|
||||
_realtime = sorted(p for p, v in _products.items()
|
||||
if v is PriceProduct.REALTIME)
|
||||
check(f"no SHIPPED plugin declares real-time/imbalance {_realtime or '[]'} - "
|
||||
f"§9.2's row 2 is unoccupied since TARIFF-07 withdrew eneco_eddy, and "
|
||||
f"the widening path below is kept anyway, on purpose",
|
||||
_realtime == [])
|
||||
_regimes = {pid: volatility_regime(v) for pid, v in _products.items()}
|
||||
check(f"...so §8.3's widened thresholds reach it through the ENUM {_regimes}",
|
||||
_regimes["eneco_eddy"] == "wide"
|
||||
and set(v for p, v in _regimes.items() if p != "eneco_eddy") == {"normal"})
|
||||
check("the regime travels on the series a consumer is handed",
|
||||
derived(lambda: (READS["eneco_eddy"].regime,
|
||||
READS["generic_dayahead"].regime)) == ("wide", "normal"))
|
||||
check(f"...so every shipped plugin is in the NORMAL regime {_regimes}",
|
||||
set(_regimes.values()) == {"normal"})
|
||||
|
||||
# ...and the machinery is still LIVE, graded from a declaration this file makes
|
||||
# rather than from a plugin. This is the same stimulus AC2 promises a sixth
|
||||
# supplier: declare REALTIME, get the wide regime, edit nothing in price.py.
|
||||
_rt_caps = ProviderCapabilities(product=PriceProduct.REALTIME,
|
||||
sources=frozenset({Component.ENERGY}))
|
||||
check(f"the widening path still works from a declaration alone "
|
||||
f"({volatility_regime(_rt_caps.product)}), which is why §9.2 row 2 "
|
||||
f"having no occupant is not the same as it having no mechanism",
|
||||
volatility_regime(_rt_caps.product) == "wide"
|
||||
and volatility_regime(PriceProduct.DAY_AHEAD) == "normal")
|
||||
check(f"...and it is the PRODUCT that carries it: uncertainty_of() answers "
|
||||
f"{sorted(u.value for u in uncertainty_of(PriceProduct.REALTIME))} for "
|
||||
f"real-time and nothing for day-ahead, and widen_factor() turns that "
|
||||
f"into {widen_factor(uncertainty_of(PriceProduct.REALTIME))}",
|
||||
uncertainty_of(PriceProduct.REALTIME)
|
||||
== frozenset({Uncertainty.REALTIME_PRODUCT})
|
||||
and uncertainty_of(PriceProduct.DAY_AHEAD) == frozenset()
|
||||
and widen_factor(uncertainty_of(PriceProduct.REALTIME)) == 2.5
|
||||
and widen_factor(uncertainty_of(PriceProduct.DAY_AHEAD)) == 1.0)
|
||||
|
||||
|
||||
def _accepts_name():
|
||||
"""Would volatility_regime() take a plugin id? If it would, somebody can
|
||||
start passing one and AC2 rots without a single check going red."""
|
||||
@@ -1360,11 +1480,17 @@ class _NewSupplier(PriceProvider):
|
||||
async def fetch(self, o, since, until, transport):
|
||||
body = await transport("https://api.example.invalid/p",
|
||||
float(o.get("price_timeout_s")))
|
||||
return _price._points_from_json(body, "price_eur_mwh", 0.001, False)
|
||||
return _price._points_from_columnar_json(body, expect_unit="EUR / MWh",
|
||||
scale=0.001,
|
||||
provisional_default=False)
|
||||
|
||||
|
||||
# ⚠️ THIS CLASS IS ALSO THE ONLY LIVE DECLARER OF `PriceProduct.REALTIME` IN
|
||||
# THE TREE (TARIFF-07), which makes the widened-threshold path a thing this
|
||||
# suite exercises end to end - registry, series, regime - and not merely a
|
||||
# constant section 5 reads. Deleting it would strand the seam twice over.
|
||||
_sixth = load("_test_sixth_supplier",
|
||||
transport=canned(synthetic_row_feed("price_eur_mwh", EPEX_EUR_MWH)))
|
||||
transport=canned(columnar_feed(EPEX_EUR_MWH)))
|
||||
check("a plugin invented outside price.py resolves through the registry",
|
||||
_sixth.kind == "series" and _sixth.provider == "_test_sixth_supplier")
|
||||
_sixth_regime = derived(lambda: _sixth.regime)
|
||||
@@ -1506,8 +1632,9 @@ _failures = {
|
||||
# check rather than by a review.
|
||||
"the body is the OTHER shape": load(
|
||||
"generic_dayahead",
|
||||
transport=canned(synthetic_row_feed("price_eur_mwh", EPEX_EUR_MWH))),
|
||||
transport=canned(row_shape_body("price_eur_mwh", EPEX_EUR_MWH))),
|
||||
"the provider id is unknown": load("no_such_supplier"),
|
||||
"the provider id is RETIRED": load("eneco_eddy"),
|
||||
"no provider is configured": load("off"),
|
||||
"the csv is missing": load("manual_csv",
|
||||
{"price_csv_path": "/no/such/file.csv"},
|
||||
@@ -1773,7 +1900,7 @@ check("a userinfo prefix cannot smuggle a declared host past the check",
|
||||
check("an operator-supplied URL to another supplier's host is refused",
|
||||
load("generic_dayahead",
|
||||
{"price_feed_url": "https://api.eneco.be/eddy/prices"},
|
||||
transport=canned(synthetic_row_feed("price_eur_mwh", EPEX_EUR_MWH))
|
||||
transport=canned(columnar_feed(EPEX_EUR_MWH))
|
||||
).kind == "no_data")
|
||||
_declared = {pid: cls.capabilities().egress_hosts
|
||||
for pid, cls in PROVIDERS.items()}
|
||||
@@ -1785,7 +1912,7 @@ check("the timeout the plugin passes to the transport is the configured one, "
|
||||
"not a hardcoded default",
|
||||
(lambda t: (load("generic_dayahead", {"price_timeout_s": 7},
|
||||
transport=t), t.calls[0][1] == 7.0)[1])(
|
||||
canned(synthetic_row_feed("price_eur_mwh", EPEX_EUR_MWH))))
|
||||
canned(columnar_feed(EPEX_EUR_MWH))))
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
@@ -1835,10 +1962,71 @@ check("...and so does a configured provider on its own",
|
||||
check("an unknown provider id is refused rather than defaulted to 'off'",
|
||||
why(validate_startup_config(vbase(price_provider="nordpool"), -5000, 5000),
|
||||
"not a shipped plugin"))
|
||||
|
||||
# ⚠️ TARIFF-07: A RETIRED ID IS REFUSED BY A DIFFERENT MESSAGE, AND EVERY CHECK
|
||||
# HERE GRADES THE REASON RATHER THAN THE VERDICT. The failure this section
|
||||
# exists to catch is NOT "a retired id starts" - it never will, because it is
|
||||
# absent from PROVIDERS and the unknown-id branch would catch it. It is the
|
||||
# retired branch COLLAPSING INTO the unknown-id one: same refusal, same exit
|
||||
# code, and an operator told they mistyped a name they chose from a dropdown
|
||||
# this add-on offered them. The two messages are asserted to DIFFER and each
|
||||
# needle is asserted to be absent from the other, which is MUTATION.md's
|
||||
# "grade the reason, not the verdict" applied to two rungs that answer one
|
||||
# stimulus.
|
||||
_unknown = validate_startup_config(vbase(price_provider="nordpool"),
|
||||
-5000, 5000)
|
||||
_retired = {pid: validate_startup_config(vbase(price_provider=pid),
|
||||
-5000, 5000)
|
||||
for pid in sorted(RETIRED_PROVIDERS)}
|
||||
check(f"every retired id is refused at startup {sorted(_retired)}",
|
||||
len(_retired) == 3 and all(len(v) == 1 for v in _retired.values()))
|
||||
check("...and NOT by the unknown-id message, which would tell an operator they "
|
||||
"mistyped an id this add-on offered them and accepted for months",
|
||||
all("not a shipped plugin" not in v[0] for v in _retired.values())
|
||||
and "WITHDRAWN" not in _unknown[0])
|
||||
check("...and each message NAMES ITS OWN ID, so the operator can tell it is "
|
||||
"about their configuration and not a generic notice",
|
||||
all(repr(pid) in v[0] for pid, v in _retired.items()))
|
||||
check("...and (a) says the plugin never fetched a price, naming the endpoint "
|
||||
"that does not exist",
|
||||
all("NEVER FETCHED A PRICE" in v[0] for v in _retired.values())
|
||||
and "api.engie.be/dynamic/be" in _retired["engie_dynamic"][0]
|
||||
and "api.totalenergies.be/prices/dynamic"
|
||||
in _retired["totalenergies_dynamic"][0]
|
||||
and "api.eneco.be/eddy/prices" in _retired["eneco_eddy"][0])
|
||||
check("...and (b) says what the site has ACTUALLY been running instead - the "
|
||||
"half §11.4 made invisible",
|
||||
all("SILENT" in v[0] and "static_profile" in v[0]
|
||||
for v in _retired.values()))
|
||||
check("...and (c) says what to do, naming a live plugin and refusing to "
|
||||
"pretend an alias could exist",
|
||||
all("generic_dayahead" in v[0] and "manual_csv" in v[0]
|
||||
and "NO alias" in v[0] for v in _retired.values()))
|
||||
|
||||
# ⚠️ AND THE SENTENCE CHANGES WITH THE SITE. "You have been on the static
|
||||
# profile" is FALSE where no profile was ever set, and that site is in a WORSE
|
||||
# state - no tariff at all - so the message must not flatten the two. Graded by
|
||||
# asserting the two renderings DIFFER, because a check that only looked for
|
||||
# "static_profile" would pass on a message that says the same thing to both.
|
||||
_PROF = ('{"periods": [{"from":"00:00","to":"24:00","days":"all",'
|
||||
'"eur_per_kwh":0.31}]}')
|
||||
_with_profile = validate_startup_config(
|
||||
vbase(price_provider="eneco_eddy", static_profile=_PROF), -5000, 5000)[0]
|
||||
_without = _retired["eneco_eddy"][0]
|
||||
check("a retired id on a site WITH a static profile is told it has been "
|
||||
"running that profile",
|
||||
"the static day/night tariff in static_profile" in _with_profile)
|
||||
check("...and a site WITHOUT one is told it has been running nothing at all, "
|
||||
"which is a different and worse state",
|
||||
"NOTHING AT ALL" in _without
|
||||
and "the static day/night tariff in static_profile" not in _without)
|
||||
check("...so the two renderings are not the same sentence",
|
||||
_with_profile != _without)
|
||||
|
||||
# ⚠️ THE HA OVERLAP, DECIDED. The two entities keep working alone; combining
|
||||
# them with a provider is what is refused, because the entities win silently.
|
||||
_p = validate_startup_config(
|
||||
vbase(price_provider="engie_dynamic", price_now_entity="sensor.now",
|
||||
vbase(price_provider="generic_dayahead", price_now_entity="sensor.now",
|
||||
price_avg_entity="sensor.avg"), -5000, 5000)
|
||||
check("a provider AND the Home Assistant price entities is refused - two price "
|
||||
"sources on one site", len(_p) == 1)
|
||||
@@ -2601,6 +2789,363 @@ check("...and BOTH translations tell an operator which switch does which - "
|
||||
and "ZET DE PRIJSFEED NIET UIT" in _T10_TR["nl"])
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
print()
|
||||
print("14. TARIFF-14 - the point whose OWN interval covers the instant")
|
||||
# ⚠️ THE RESOLUTION IS THE WHOLE SECTION, AND ITS ABSENCE IS WHY THE DEFECT
|
||||
# SHIPPED. Every price fixture in this repository before TARIFF-08 was the
|
||||
# invented HOURLY row shape, and on an hourly series `s <= u < s + 1 h` and
|
||||
# "until the next point's start" AGREE AT EVERY INSTANT - so the suite had
|
||||
# been exercising the one resolution that cannot expose the bug. Section 8
|
||||
# counts points on the 23- and 25-hour days and never asks `at()` for a
|
||||
# quarter-hour. Belgium moved to a 15-minute MTU on 2025-10-01 and the live
|
||||
# day-ahead feed returns 96 points at 900 s.
|
||||
|
||||
# ⚠️ BRUSSELS MIDNIGHT, NOT UTC MIDNIGHT, so that every wall-clock time in
|
||||
# a check name below is the time the series really renders - "00:15" and
|
||||
# "23:45" are what `canonical()` writes, not an offset away from it.
|
||||
Q0 = datetime(2026, 1, 15, 0, 0, tzinfo=timezone(timedelta(hours=1)))
|
||||
HOUR = timedelta(hours=1)
|
||||
|
||||
|
||||
def spaced(gaps, first=0.10, step=0.01, provisional=None, start=Q0):
|
||||
"""A series whose consecutive gaps are exactly `gaps`, in SECONDS.
|
||||
|
||||
`len(points) == len(gaps) + 1`. Prices ascend by `step`, so a point can be
|
||||
named by VALUE as well as by identity and a sweep that compared only
|
||||
prices could still tell two points apart.
|
||||
"""
|
||||
n = len(gaps) + 1
|
||||
flags = [False] * n if provisional is None else list(provisional)
|
||||
pts, t = [], start
|
||||
for i in range(n):
|
||||
pts.append(PricePoint(brussels(t), round(first + step * i, 6),
|
||||
bool(flags[i])))
|
||||
if i < len(gaps):
|
||||
t += timedelta(seconds=gaps[i])
|
||||
return PriceRead.series(pts, PriceProduct.DAY_AHEAD, "tariff14", start)
|
||||
|
||||
|
||||
def gaps_of(s):
|
||||
u = [p.start.astimezone(UTC) for p in s.points]
|
||||
return [int((b - a).total_seconds()) for a, b in zip(u, u[1:])]
|
||||
|
||||
|
||||
# The three shapes the acceptance criteria name, plus the two degenerate ones.
|
||||
Q = spaced([900] * 95) # the shipped 96-point Belgian day
|
||||
H = spaced([3600] * 23) # what every older fixture carried
|
||||
# ⚠️ FCST-04'S STEADY STATE, NOT AN EDGE CASE. Eight firm quarter-hours, then
|
||||
# the carried HOURLY points the fill adds for the hours nobody published. The
|
||||
# seam between them is a 900 s gap and the tail is 3600 s.
|
||||
MIX = spaced([900] * 8 + [3600] * 3,
|
||||
provisional=[False] * 9 + [True] * 3)
|
||||
# The same seam made the END of the series - the last point is the FIRST
|
||||
# carried hourly one, and the gap behind it is 900 s. This is the fixture the
|
||||
# tail decision is graded on.
|
||||
MIX_TAIL = spaced([900] * 8, provisional=[False] * 8 + [True])
|
||||
ONE = spaced([])
|
||||
|
||||
print("14a. the fixtures are not degenerate - asserted before anything else")
|
||||
check(f"the quarter-hourly fixture is the live shape - 96 points, spacing "
|
||||
f"{sorted(set(gaps_of(Q)))} s",
|
||||
len(Q.points) == 96 and set(gaps_of(Q)) == {900})
|
||||
check(f"the hourly fixture is 24 points at {sorted(set(gaps_of(H)))} s - and "
|
||||
f"it is a REGRESSION guard, not a witness: the shipped rule and this "
|
||||
f"one agree at every instant of it, which is exactly why nothing caught "
|
||||
f"this",
|
||||
len(H.points) == 24 and set(gaps_of(H)) == {3600})
|
||||
check(f"...while the MIXED fixture really is mixed, spacing "
|
||||
f"{sorted(set(gaps_of(MIX)))} s - the pair the FCST-04 soak measured in "
|
||||
f"70 % of samples, so a window derived from ONE global spacing would be "
|
||||
f"wrong in the NORMAL steady state",
|
||||
set(gaps_of(MIX)) == {900, 3600}
|
||||
and set(gaps_of(MIX_TAIL)) == {900}
|
||||
and gaps_of(MIX)[-1] == 3600 and gaps_of(MIX_TAIL)[-1] == 900)
|
||||
check("...and every point of every fixture carries a DISTINCT price, so 'the "
|
||||
"covering point' names one point and a wrong answer cannot coincide "
|
||||
"with the right one",
|
||||
all(len({p.eur_per_kwh for p in s.points}) == len(s.points)
|
||||
for s in (Q, H, MIX, MIX_TAIL)))
|
||||
check("...and the mixed fixtures carry BOTH firm and carried points, which is "
|
||||
"what makes them FCST-04's output rather than a spacing exercise",
|
||||
all(any(p.provisional for p in s.points)
|
||||
and any(not p.provisional for p in s.points)
|
||||
for s in (MIX, MIX_TAIL)))
|
||||
|
||||
print()
|
||||
print("14b. the reading the shipped scan got wrong, stated as literals")
|
||||
# ⚠️ ASSERTED AGAINST TYPED-OUT INDICES AND PRICES, not against a window this
|
||||
# file recomputed by price.py's own rule. The sweep below is exhaustive and
|
||||
# the sweep is derived; these four are the anchors that keep it honest.
|
||||
_q15 = derived(lambda: Q.at(Q0 + timedelta(minutes=15)))
|
||||
check(f"at(00:15) on the 900 s series is the 00:15 point at 0.11 EUR/kWh - the "
|
||||
f"shipped scan answered the 00:00 point at 0.10, because the :00 "
|
||||
f"point's HOUR had not run out yet (saw {_q15!r})",
|
||||
_q15 is Q.points[1] and derived(lambda: _q15.eur_per_kwh) == 0.11)
|
||||
_q195 = derived(lambda: Q.at(Q0 + timedelta(minutes=195)))
|
||||
check(f"at(03:15) is the 03:15 point at 0.23 - the shipped scan answered "
|
||||
f"02:30 at 0.20 here, and that is the SECOND failure mode: from the "
|
||||
f"second hour on the first point still inside its own hour wins, so the "
|
||||
f"error COMPOUNDS with depth into the series (saw {_q195!r})",
|
||||
_q195 is Q.points[13] and derived(lambda: _q195.eur_per_kwh) == 0.23)
|
||||
check("at(00:30) and at(00:45) are their own quarters too, so the answer is "
|
||||
"not merely 'one point later'",
|
||||
derived(lambda: Q.at(Q0 + timedelta(minutes=30))) is Q.points[2]
|
||||
and derived(lambda: Q.at(Q0 + timedelta(minutes=45))) is Q.points[3])
|
||||
check("on the MIXED series the quarter-hourly half answers per QUARTER and "
|
||||
"the carried hourly half answers per HOUR, in one series - at(00:15) is "
|
||||
"the 00:15 point and at(02:30) is the 02:00 one",
|
||||
derived(lambda: MIX.at(Q0 + timedelta(minutes=15))) is MIX.points[1]
|
||||
and derived(lambda: MIX.at(Q0 + timedelta(minutes=150))) is MIX.points[8]
|
||||
and derived(lambda: MIX.points[8].start.astimezone(UTC)) ==
|
||||
Q0 + timedelta(hours=2))
|
||||
|
||||
print()
|
||||
print("14c. the exhaustive sweep, at 900 s, 3600 s and mixed")
|
||||
|
||||
|
||||
def windows(s):
|
||||
"""`(start_utc, width)` per point, computed HERE from the gaps.
|
||||
|
||||
Deliberately NOT read off `_Series._cover`: a check built from the table it
|
||||
grades compares a thing against itself. The rule is restated - a point runs
|
||||
to the next start, capped at an hour; the last point copies the width in
|
||||
front of it; a lone point gets an hour.
|
||||
"""
|
||||
u = [p.start.astimezone(UTC) for p in s.points]
|
||||
out = []
|
||||
for i, t in enumerate(u):
|
||||
if i + 1 < len(u):
|
||||
g = u[i + 1] - t
|
||||
elif i:
|
||||
g = t - u[i - 1]
|
||||
else:
|
||||
g = HOUR
|
||||
out.append((t, g if g < HOUR else HOUR))
|
||||
return out
|
||||
|
||||
|
||||
def sweep(s):
|
||||
"""Every probe whose covering point is unambiguous, and what `at()` said."""
|
||||
wrong = []
|
||||
for i, (t, w) in enumerate(windows(s)):
|
||||
for off in (timedelta(0), w / 2, w - timedelta(seconds=1)):
|
||||
got = derived(lambda: s.at(t + off))
|
||||
if got is not s.points[i]:
|
||||
wrong.append(f"{(t + off).isoformat()} -> {got!r}, "
|
||||
f"wanted {s.points[i].eur_per_kwh}")
|
||||
return wrong
|
||||
|
||||
|
||||
for _name, _s in (("900 s", Q), ("3600 s", H), ("mixed 900/3600", MIX),
|
||||
("mixed, 900 s tail", MIX_TAIL)):
|
||||
_w = derived(lambda s=_s: sweep(s))
|
||||
check(f"{_name}: all {3 * len(_s.points)} probes inside the series return "
|
||||
f"the point whose OWN interval covers them "
|
||||
f"({_w if _w != [] else 'no misses'})", _w == [])
|
||||
|
||||
print()
|
||||
print("14d. the one-hour cap, which is what keeps FCST-04's holes holes")
|
||||
# ⚠️ NOT DECORATION. FCST-04 leaves an hour OUT when nothing published it and
|
||||
# yesterday has no value either, and price_forecast.py's module note names
|
||||
# `at()` raising Absence for it as the behaviour that sends the caller to the
|
||||
# static profile (§11.4 for that hour). Uncapped, "until the next point's
|
||||
# start" would stretch the point before a three-hour hole across the whole
|
||||
# hole - filling holes with a stale price, which is the defect this module
|
||||
# exists to refuse, introduced by the fix for a different one.
|
||||
HOLE = PriceRead.series(
|
||||
[PricePoint(brussels(Q0 + timedelta(hours=i)), 0.10 + i / 100.0, False)
|
||||
for i in (0, 1, 4, 5)],
|
||||
PriceProduct.DAY_AHEAD, "tariff14", Q0)
|
||||
_in_hole = [Q0 + timedelta(hours=2), Q0 + timedelta(hours=2, minutes=30),
|
||||
Q0 + timedelta(hours=3), Q0 + timedelta(hours=3, minutes=59)]
|
||||
check(f"a three-hour hole stays a hole: all {len(_in_hole)} instants inside it "
|
||||
f"raise Absence, while the point BEFORE it still covers its own hour and "
|
||||
f"the point after it answers from its own start",
|
||||
all(raises(lambda u=u: HOLE.at(u), Absence) for u in _in_hole)
|
||||
and derived(lambda: HOLE.at(Q0 + timedelta(hours=1, minutes=59)))
|
||||
is HOLE.points[1]
|
||||
and derived(lambda: HOLE.at(Q0 + timedelta(hours=4))) is HOLE.points[2])
|
||||
check(f"...and the cap is stated once, as MAX_POINT_SPAN = "
|
||||
f"{_price.MAX_POINT_SPAN}, rather than spelled into the lookup",
|
||||
_price.MAX_POINT_SPAN == HOUR)
|
||||
# ⚠️ AND THE OTHER HALF OF THE HOLE, WHICH IS THE ONE §8.3 READS. FCST-04
|
||||
# fills a whole hour nobody published with ONE carried point, so a
|
||||
# quarter-hourly day can carry an hourly carried point in the middle of it.
|
||||
# The instant inside that hour is covered by the CARRIED point - and the
|
||||
# shipped scan answered the FIRM quarter 52 minutes earlier, because that was
|
||||
# the first point still inside its own hour. `_act()`'s `here.provisional`
|
||||
# guard and `price_forecast._walk()`'s `series.at(u).provisional` both read the
|
||||
# flag off whatever that lookup returned, so the flag was FIRM on an hour
|
||||
# nobody published.
|
||||
SANDWICH = PriceRead.series(
|
||||
[PricePoint(brussels(Q0 + timedelta(minutes=15 * i)),
|
||||
round(0.10 + i / 1000.0, 6), False) for i in range(44)]
|
||||
+ [PricePoint(brussels(Q0 + timedelta(hours=11)), 0.500, True)]
|
||||
+ [PricePoint(brussels(Q0 + timedelta(hours=12, minutes=15 * i)),
|
||||
round(0.20 + i / 1000.0, 6), False) for i in range(4)],
|
||||
PriceProduct.DAY_AHEAD, "tariff14", Q0)
|
||||
_sand_t = Q0 + timedelta(hours=11, minutes=37)
|
||||
|
||||
|
||||
def shipped_at(s, u):
|
||||
"""The rule TARIFF-14 replaced, as the COUNTERFACTUAL only - nothing is
|
||||
graded against it. It is here so the fixture can be SHOWN to separate the
|
||||
two rules; a check whose data cannot reach the failure it names grades
|
||||
nothing, and on an hourly series these two agree at every instant."""
|
||||
uu = u.astimezone(UTC)
|
||||
for p in s.points:
|
||||
st = p.start.astimezone(UTC)
|
||||
if st <= uu < st + HOUR:
|
||||
return p
|
||||
return None
|
||||
|
||||
|
||||
check("the sandwich fixture is FCST-04's real output - 44 firm quarters, ONE "
|
||||
"carried hourly point filling the hour nobody published, then firm "
|
||||
"quarters again",
|
||||
len(SANDWICH.points) == 49
|
||||
and SANDWICH.points[44].provisional is True
|
||||
and sum(1 for p in SANDWICH.points if p.provisional) == 1)
|
||||
_sand_now = derived(lambda: SANDWICH.at(_sand_t))
|
||||
_sand_old = derived(lambda: shipped_at(SANDWICH, _sand_t))
|
||||
check(f"an instant inside the carried hour is covered by the CARRIED point, "
|
||||
f"so §8.3's flag is read off the hour it belongs to. The shipped scan "
|
||||
f"answered the FIRM quarter 52 minutes earlier "
|
||||
f"({derived(lambda: _sand_old.start.isoformat())}, provisional="
|
||||
f"{derived(lambda: _sand_old.provisional)}), which is a carried hour "
|
||||
f"wearing a published hour's flag",
|
||||
_sand_now is SANDWICH.points[44]
|
||||
and derived(lambda: _sand_now.provisional) is True
|
||||
and _sand_old is SANDWICH.points[43]
|
||||
and derived(lambda: _sand_old.provisional) is False)
|
||||
|
||||
print()
|
||||
print("14e. the LAST point's window - decided and documented, not inherited")
|
||||
# There is no next start and §9.3 says nothing about how wide the final
|
||||
# interval is, so THE LAST POINT IS GIVEN THE SAME SPAN AS THE POINT BEFORE IT
|
||||
# (one hour when it is the only point). The alternative - a flat one-hour tail
|
||||
# - is defensible for an hourly series and wrong for the live one: it answers
|
||||
# 00:30 TOMORROW with today's last price, which is reaching for a neighbour
|
||||
# across a day boundary.
|
||||
_ql = Q.points[-1].start.astimezone(UTC)
|
||||
_hl = H.points[-1].start.astimezone(UTC)
|
||||
_ml = MIX_TAIL.points[-1].start.astimezone(UTC)
|
||||
check("the 900 s day's last point covers its own QUARTER: at(+14:59) answers "
|
||||
"and at(+15:00) raises, so today's 23:45 price cannot be returned for "
|
||||
"00:30 tomorrow",
|
||||
derived(lambda: Q.at(_ql + timedelta(minutes=14, seconds=59)))
|
||||
is Q.points[-1]
|
||||
and raises(lambda: Q.at(_ql + timedelta(minutes=15)), Absence))
|
||||
check("the hourly series' last point still covers an HOUR, exactly as it "
|
||||
"shipped - no static or hourly site changes behaviour",
|
||||
derived(lambda: H.at(_hl + timedelta(minutes=59))) is H.points[-1]
|
||||
and raises(lambda: H.at(_hl + HOUR), Absence))
|
||||
check("the MIXED series ending at the firm/carried seam takes the 900 s in "
|
||||
"front of it, so its last 45 minutes raise Absence. That is the SAFE "
|
||||
"direction and it is CHOSEN: §11.4 sends the caller to the static "
|
||||
"profile rather than trading on a span the series never demonstrated, "
|
||||
"and the point is carried, so §8.3 forbids irreversible action on it "
|
||||
"anyway",
|
||||
derived(lambda: MIX_TAIL.at(_ml + timedelta(minutes=14, seconds=59)))
|
||||
is MIX_TAIL.points[-1]
|
||||
and raises(lambda: MIX_TAIL.at(_ml + timedelta(minutes=15)), Absence)
|
||||
and MIX_TAIL.points[-1].provisional)
|
||||
# ⚠️ HABIT 1. Without this the three checks above would all pass under a
|
||||
# single global tail rule of 15 minutes, or of an hour, depending which two
|
||||
# you read - so the claim that the tail FOLLOWS THE SERIES is asserted as its
|
||||
# own check, at one instant where the three fixtures must disagree.
|
||||
_t30 = [derived(lambda s=s, l=l: s.covers(l + timedelta(minutes=30)))
|
||||
for s, l in ((Q, _ql), (H, _hl), (MIX_TAIL, _ml))]
|
||||
check(f"...and those tails are genuinely three readings of one rule and not "
|
||||
f"one constant: 30 minutes past its own last point the hourly series "
|
||||
f"answers and both 900 s ones do not {_t30}",
|
||||
_t30 == [False, True, False])
|
||||
check("a series of ONE point has no gap in front of it to copy, so it gets an "
|
||||
"hour - the shipped behaviour, kept deliberately for the degenerate case",
|
||||
derived(lambda: ONE.at(Q0 + timedelta(minutes=59, seconds=59)))
|
||||
is ONE.points[0]
|
||||
and raises(lambda: ONE.at(Q0 + HOUR), Absence))
|
||||
|
||||
print()
|
||||
print("14f. order is established by the lookup, not assumed of the caller")
|
||||
# ⚠️ `at()` now reads a point's NEIGHBOURS, so it WOULD depend on sortedness.
|
||||
# The index is sorted at construction so that it cannot. `_validate_raw()`
|
||||
# refuses out-of-order and duplicate starts on the `load_prices()` path
|
||||
# (section 9 above), `price_history._parse_points()` re-sorts what it reads off
|
||||
# disk and FCST-04's fill emits ascending - but `PriceRead.series()` itself has
|
||||
# never enforced any of it, and an assumption that holds today and is unstated
|
||||
# is how this class of defect recurs.
|
||||
_REV = PriceRead.series(tuple(reversed(Q.points)), PriceProduct.DAY_AHEAD,
|
||||
"tariff14", Q0)
|
||||
check("the reversed fixture really is reversed, so the check below is not "
|
||||
"grading two identical series",
|
||||
_REV.points[0] is Q.points[-1] and _REV.points[-1] is Q.points[0]
|
||||
and _REV.points != Q.points)
|
||||
_rev_bad = derived(lambda: [p.start.isoformat() for p in Q.points
|
||||
if _REV.at(p.start) is not p])
|
||||
check(f"...and a series handed its 96 points in REVERSE order answers at() "
|
||||
f"identically at every one of them "
|
||||
f"({_rev_bad if _rev_bad != [] else 'no misses'})",
|
||||
_rev_bad == [])
|
||||
check("...while the load path refuses an unsorted feed outright, so the two "
|
||||
"defences are stated rather than one being quietly relied on",
|
||||
raises(lambda: _price._validate_raw(
|
||||
[RawPoint(T0 + timedelta(minutes=15), 0.1, False),
|
||||
RawPoint(T0, 0.2, False)], _caps), SourceUnavailable))
|
||||
check("a DUPLICATED start does not raise anywhere - not at construction and "
|
||||
"not on the 30 s control path. The later list entry takes the real span "
|
||||
"and the earlier one becomes unreachable through at(); load_prices() "
|
||||
"refuses the shape outright, and this says what happens to anyone who "
|
||||
"builds one by hand",
|
||||
derived(lambda: PriceRead.series(
|
||||
[PricePoint(brussels(Q0), 0.10, False),
|
||||
PricePoint(brussels(Q0), 0.20, False),
|
||||
PricePoint(brussels(Q0 + timedelta(minutes=15)), 0.30, False)],
|
||||
PriceProduct.DAY_AHEAD, "tariff14", Q0).at(Q0).eur_per_kwh) == 0.20)
|
||||
|
||||
print()
|
||||
print("14g. covers() shared it; window() did not")
|
||||
check("covers() shared the defect EXACTLY, because it has no window rule of "
|
||||
"its own - it asks at() - so the one change fixes both. On the 900 s "
|
||||
"day it is now True 14:59 past the last point and False 15:00 past it, "
|
||||
"where an hour-wide window said True for another 45 minutes",
|
||||
derived(lambda: Q.covers(_ql + timedelta(minutes=14, seconds=59))) is True
|
||||
and derived(lambda: Q.covers(_ql + timedelta(minutes=15))) is False
|
||||
and derived(lambda: Q.covers(_ql + timedelta(minutes=30))) is False
|
||||
and derived(lambda: Q.covers(Q0 + timedelta(minutes=15))) is True)
|
||||
check("window() did NOT share it, and the reason is worth a check rather than "
|
||||
"a rediscovery: it grades a point's START against a half-open range and "
|
||||
"never asks how long a point lasts, so it carries no resolution "
|
||||
"assumption at all. 00:07:30 .. 00:22:30 holds exactly the 00:15 point "
|
||||
"- while at(00:07:30) is the 00:00 one, which is what makes it a "
|
||||
"DIFFERENT question and not a coverage query",
|
||||
derived(lambda: Q.window(Q0 + timedelta(minutes=7, seconds=30),
|
||||
Q0 + timedelta(minutes=22, seconds=30)))
|
||||
== (Q.points[1],)
|
||||
and derived(lambda: Q.at(Q0 + timedelta(minutes=7, seconds=30)))
|
||||
is Q.points[0])
|
||||
|
||||
print()
|
||||
print("14h. the 'no data is not free' contract, unchanged")
|
||||
check("at() answers with a PricePoint or raises Absence - never None, at any "
|
||||
"resolution, so `series.at(t) or fallback` cannot become a price of zero",
|
||||
all(isinstance(derived(lambda s=s: s.at(s.points[0].start)), PricePoint)
|
||||
for s in (Q, H, MIX, MIX_TAIL, ONE))
|
||||
and raises(lambda: Q.at(Q0 - timedelta(seconds=1)), Absence)
|
||||
and raises(lambda: Q.at(Q0 + timedelta(days=2)), Absence))
|
||||
_absence_words = _reason(lambda: Q.at(Q0 + timedelta(days=2)))
|
||||
check(f"...and the refusal still names the range the series runs, so an "
|
||||
f"operator can tell 'outside the horizon' from 'into a hole'",
|
||||
Q.points[0].start.isoformat() in _absence_words
|
||||
and Q.points[-1].start.isoformat() in _absence_words)
|
||||
check("...and it never invents a neighbour: the instant one second before the "
|
||||
"series starts is refused, not answered with the first point",
|
||||
raises(lambda: Q.at(Q0 - timedelta(seconds=1)), Absence)
|
||||
and raises(lambda: MIX.at(Q0 - timedelta(seconds=1)), Absence)
|
||||
and raises(lambda: H.at(Q0 - timedelta(seconds=1)), Absence))
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
for _d in _tmpdirs_t10:
|
||||
shutil.rmtree(_d, ignore_errors=True)
|
||||
|
||||
@@ -289,22 +289,29 @@ check("...and never repairs a flag: the audit's only outcomes are silence and "
|
||||
for i, u in enumerate(DAY[:PUBLISHED_N])})
|
||||
is None)
|
||||
|
||||
# ⚠️ A PUBLISHED POINT MAY BE PROVISIONAL. eneco_eddy's current-quarter print
|
||||
# is settled after the fact and says so (§9.2). "Published" is not "firm".
|
||||
_eddy = series_of(DAY[:2], lambda u: t_price(brussels(u).hour),
|
||||
PriceProduct.REALTIME, "eneco_eddy", provisional=True)
|
||||
# ⚠️ A PUBLISHED POINT MAY BE PROVISIONAL. A real-time / imbalance product's
|
||||
# current interval is settled after the fact and says so per point (§9.2).
|
||||
# "Published" is not "firm", and §9.3 carries the flag PER POINT for exactly
|
||||
# this. No shipped plugin declares REALTIME since TARIFF-07, so the series is
|
||||
# declared here - which is the only place the case can come from now, and is
|
||||
# why this stimulus did not go with the plugin.
|
||||
_rt = series_of(DAY[:2], lambda u: t_price(brussels(u).hour),
|
||||
PriceProduct.REALTIME, "_realtime_not_shipped",
|
||||
provisional=True)
|
||||
_eddy_filled = derived(lambda: fill_from_yesterday(
|
||||
_eddy, yesterday=YESTERDAY, since=D0, until=D1, stamp=D0))
|
||||
_rt, yesterday=YESTERDAY, since=D0, until=D1, stamp=D0))
|
||||
check("a PUBLISHED point that the feed itself flagged provisional keeps that "
|
||||
"flag - the audit checks provenance, and does not force published hours "
|
||||
"to False",
|
||||
_eddy_filled.kind == "series"
|
||||
and all(p.provisional for p in _eddy_filled.points[:2])
|
||||
and _eddy_filled.points[0] is _eddy.points[0])
|
||||
and _eddy_filled.points[0] is _rt.points[0])
|
||||
|
||||
# ⚠️ AND A SUB-HOURLY FEED IS NOT DECIMATED TO ONE POINT AN HOUR (FCST-04 H1).
|
||||
# `eneco_eddy` settles per QUARTER and `_validate_raw()` accepts any
|
||||
# strictly-increasing spacing, so a published hour can hold four points. The
|
||||
# BELGIAN DAY-AHEAD ITSELF SETTLES PER QUARTER - the 15-minute MTU of
|
||||
# 2025-10-01, which `generic_dayahead` reads as 96 points a day - and
|
||||
# `_validate_raw()` accepts any strictly-increasing spacing, so a published
|
||||
# hour can hold four points. The
|
||||
# fill walks WHOLE HOURS to decide what needs carrying; that must not become a
|
||||
# rule about what a published hour may contain. Measured on FCST-04's first
|
||||
# draft, which keyed the published set `{_utc_hour(p.start): p ...}`: a
|
||||
@@ -316,7 +323,7 @@ _QUARTERS = PriceRead.series(
|
||||
eur_per_kwh=round(0.100000 + 0.011300 * n, 6),
|
||||
provisional=False)
|
||||
for n in range(_QN)],
|
||||
PriceProduct.REALTIME, "eneco_eddy", D0)
|
||||
PriceProduct.DAY_AHEAD, "generic_dayahead", D0)
|
||||
_QHOURS = {_fc._utc_hour(p.start) for p in _QUARTERS.points}
|
||||
check(f"the quarter-hourly fixture really is sub-hourly - {_QN} points across "
|
||||
f"{len(_QHOURS)} hours - or every check below passes for an hourly feed "
|
||||
@@ -942,13 +949,20 @@ _normal_named_eddy = derived(lambda: min_accepted_spread(
|
||||
PriceProduct.DAY_AHEAD, frozenset(), BASE, provider="eneco_eddy"))
|
||||
check("the widening follows the DECLARED PriceProduct even when the provider "
|
||||
"is called 'generic_dayahead', and does NOT follow the name "
|
||||
"'eneco_eddy' when the declared product is day-ahead - so a sixth "
|
||||
"supplier is widened by its declaration and not by an id in a list",
|
||||
"'eneco_eddy' when the declared product is day-ahead - so a supplier is "
|
||||
"widened by its declaration and not by an id in a list",
|
||||
near(_wide_named_dayahead, _wide, 1e-9)
|
||||
and near(_normal_named_eddy, _normal, 1e-9))
|
||||
check("...and the shipped plugin §9.2 names IS the one that declares REALTIME, "
|
||||
"so the mechanism is pointed at the real product",
|
||||
PROVIDERS["eneco_eddy"].capabilities().product is PriceProduct.REALTIME
|
||||
# ⚠️ 'eneco_eddy' ABOVE IS NOW A RETIRED ID AND THE CHECK IS STRONGER FOR IT.
|
||||
# It was chosen because it was the one plugin that really did declare REALTIME,
|
||||
# so the check read "the name does not override the declaration". It declares
|
||||
# nothing now, which makes the string a pure name with no product behind it -
|
||||
# exactly the input an id-sniffing implementation would still widen.
|
||||
check("...and NO shipped plugin declares REALTIME since TARIFF-07, so the "
|
||||
"widening path is reached by declaration alone - kept deliberately, see "
|
||||
"DECISIONS/TARIFF-07-withdrawing-three-plugins.md",
|
||||
[p for p, c in PROVIDERS.items()
|
||||
if c.capabilities().product is PriceProduct.REALTIME] == []
|
||||
and uncertainty_of(PriceProduct.REALTIME)
|
||||
== frozenset({Uncertainty.REALTIME_PRODUCT})
|
||||
and uncertainty_of(PriceProduct.DAY_AHEAD) == frozenset())
|
||||
|
||||
@@ -779,9 +779,10 @@ check("the carried hours are NOT in the trade set - the verdict's cheapest and "
|
||||
print()
|
||||
print("2b. a QUARTER-HOURLY feed keeps every print, and its spread with them")
|
||||
|
||||
# ⚠️ THE DEFECT THIS SECTION EXISTS FOR. `eneco_eddy` is the Belgian imbalance
|
||||
# product and settles per QUARTER; `_validate_raw()` accepts any
|
||||
# strictly-increasing spacing, so a published hour may hold four points.
|
||||
# ⚠️ THE DEFECT THIS SECTION EXISTS FOR. Belgian DAY-AHEAD settles per QUARTER
|
||||
# - the 15-minute MTU took effect on 2025-10-01, so `generic_dayahead` reads 96
|
||||
# points a day - and `_validate_raw()` accepts any strictly-increasing spacing,
|
||||
# so a published hour may hold four points.
|
||||
# FCST-04's first draft keyed the published set `{_utc_hour(p.start): p ...}` -
|
||||
# LAST POINT OF THE HOUR WINS, the other three dropped - and on an all-firm,
|
||||
# fully-published 96-point feed the plan came out 15 points long with its
|
||||
|
||||
@@ -143,10 +143,10 @@ configuration:
|
||||
description: "DERIVED, not picked (SAFETY-24): the reference pack holds 13.8 kWh usable and the worst modelled peak-shaving day asks for 878 Wh, which is 6.4 %, so 20 % sits ten points above the pack's own cut-out and still leaves the guard about 1.6 of its worst observed days of room. ⚠️ IT BOUNDS THE PEAK GUARD ONLY. It is not a reserve for the whole pack: ordinary self-consumption discharges straight past it and goes on down to the battery cut-out, and raising this number does not change that by a tenth of a percent. It must stay strictly above the battery cut-out or the add-on refuses to start. Do not change it without re-deriving it."
|
||||
price_provider:
|
||||
name: "Tariff · Dynamic price source"
|
||||
description: "Where day-ahead or real-time prices come from. 'off' is what ships, and means this site runs on the static tariff below with no trading. ⚠️ SWITCHING THIS ON IS WHAT STARTS THE PRICE FETCH - not the trading switch further down. With a source selected the add-on fetches the feed every half hour, checks it, records the day and shows what it holds on the Price history row, whether or not price trading is armed. On its own it still moves no energy: trading is the separate switch, and the cheap window the monthly maintenance cycle uses comes from the fixed or day/night tariff below. It may not be combined with the two legacy Home Assistant price entities further down: two price sources on one site, one of which silently wins, is refused at startup rather than left looking armed."
|
||||
description: "Where day-ahead or real-time prices come from. 'off' is what ships, and means this site runs on the static tariff below with no trading. ⚠️ SWITCHING THIS ON IS WHAT STARTS THE PRICE FETCH - not the trading switch further down. With a source selected the add-on fetches the feed every half hour, checks it, records the day and shows what it holds on the Price history row, whether or not price trading is armed. On its own it still moves no energy: trading is the separate switch, and the cheap window the monthly maintenance cycle uses comes from the fixed or day/night tariff below. It may not be combined with the two legacy Home Assistant price entities further down: two price sources on one site, one of which silently wins, is refused at startup rather than left looking armed. ⚠️ THREE IDS IN THIS LIST ARE WITHDRAWN AND WILL NOT START: engie_dynamic, totalenergies_dynamic and eneco_eddy. Their endpoints do not exist and never did, so those plugins never once fetched a price - a site set to one of them has silently been running on the fixed or day/night tariff below, or on nothing, since the day it was set. They are still offered here only so that such a site gets that explanation at startup instead of an unreadable schema error. Choose generic_dayahead, manual_csv or off."
|
||||
price_markup_eur_kwh:
|
||||
name: "Tariff · Supplier markup (EUR/kWh)"
|
||||
description: "Your supplier's margin. The add-on adds it ONLY when the chosen price source does not already include it - under engie_dynamic, totalenergies_dynamic, eneco_eddy and manual_csv it is ignored, and logged as ignored, never applied twice."
|
||||
description: "Your supplier's margin. The add-on adds it ONLY when the chosen price source does not already include it - under manual_csv, whose figures are all-in, it is ignored, and logged as ignored, never applied twice."
|
||||
price_taxes_eur_kwh:
|
||||
name: "Tariff · Levies and VAT (EUR/kWh)"
|
||||
description: "Taxes and levies per kWh, added under the same rule as the supplier markup: only when the chosen price source does not already carry them."
|
||||
|
||||
@@ -123,10 +123,10 @@ configuration:
|
||||
description: "AFGELEID, niet gekozen (SAFETY-24): de referentiebatterij heeft 13,8 kWh bruikbaar en de zwaarste doorgerekende afvlakdag vraagt 878 Wh, oftewel 6,4 %, dus 20 % ligt tien punten boven de uitschakelgrens van de batterij zelf en laat de bewaking nog ongeveer 1,6 van haar zwaarste waargenomen dagen speling. ⚠️ DIT BEGRENST ALLEEN DE PIEKBEWAKING. Het is geen reserve voor de hele batterij: gewoon eigenverbruik ontlaadt er dwars doorheen en gaat door tot aan de uitschakelgrens van de batterij, en dit getal verhogen verandert daar geen tiende procent aan. Het moet strikt boven de uitschakelgrens van de batterij blijven, anders start de add-on niet op. Verander het niet zonder de afleiding opnieuw te maken."
|
||||
price_provider:
|
||||
name: "Tarief · Bron van dynamische prijzen"
|
||||
description: "Waar day-ahead- of realtimeprijzen vandaan komen. 'off' is de standaardinstelling en betekent dat deze installatie op het vaste tarief hieronder draait, zonder handel. ⚠️ DIT AANZETTEN IS WAT HET OPHALEN VAN PRIJZEN START - niet de handelsschakelaar verderop. Met een bron ingesteld haalt de add-on de feed elk half uur op, controleert die, bewaart de dag en toont op de rij Prijsgeschiedenis wat er bewaard is, of prijshandel nu aan staat of niet. Op zichzelf verplaatst dat nog geen energie: handelen is de aparte schakelaar, en het goedkope venster dat de maandelijkse onderhoudscyclus gebruikt komt uit het vaste of dag/nacht-tarief hieronder. Het mag niet gecombineerd worden met de twee oude Home Assistant-prijsentiteiten verderop: twee prijsbronnen op één installatie, waarvan er stilzwijgend één wint, wordt bij het opstarten geweigerd in plaats van schijnbaar actief gelaten."
|
||||
description: "Waar day-ahead- of realtimeprijzen vandaan komen. 'off' is de standaardinstelling en betekent dat deze installatie op het vaste tarief hieronder draait, zonder handel. ⚠️ DIT AANZETTEN IS WAT HET OPHALEN VAN PRIJZEN START - niet de handelsschakelaar verderop. Met een bron ingesteld haalt de add-on de feed elk half uur op, controleert die, bewaart de dag en toont op de rij Prijsgeschiedenis wat er bewaard is, of prijshandel nu aan staat of niet. Op zichzelf verplaatst dat nog geen energie: handelen is de aparte schakelaar, en het goedkope venster dat de maandelijkse onderhoudscyclus gebruikt komt uit het vaste of dag/nacht-tarief hieronder. Het mag niet gecombineerd worden met de twee oude Home Assistant-prijsentiteiten verderop: twee prijsbronnen op één installatie, waarvan er stilzwijgend één wint, wordt bij het opstarten geweigerd in plaats van schijnbaar actief gelaten. ⚠️ DRIE IDS IN DEZE LIJST ZIJN INGETROKKEN EN STARTEN NIET MEER: engie_dynamic, totalenergies_dynamic en eneco_eddy. Hun endpoints bestaan niet en hebben nooit bestaan, dus die plugins hebben nooit één prijs opgehaald - een installatie die op zo'n bron stond, draait sindsdien stilzwijgend op het vaste of dag/nacht-tarief hieronder, of op niets. Ze staan hier alleen nog zodat zo'n installatie bij het opstarten die uitleg krijgt in plaats van een onleesbare schemafout. Kies generic_dayahead, manual_csv of off."
|
||||
price_markup_eur_kwh:
|
||||
name: "Tarief · Marge van de leverancier (EUR/kWh)"
|
||||
description: "De marge van uw energieleverancier. De add-on telt die ALLEEN op wanneer de gekozen prijsbron ze niet al bevat - bij engie_dynamic, totalenergies_dynamic, eneco_eddy en manual_csv wordt ze genegeerd, en dat wordt ook gelogd, zodat ze nooit dubbel wordt aangerekend."
|
||||
description: "De marge van uw energieleverancier. De add-on telt die ALLEEN op wanneer de gekozen prijsbron ze niet al bevat - bij manual_csv, waarvan de bedragen alles inbegrepen zijn, wordt ze genegeerd, en dat wordt ook gelogd, zodat ze nooit dubbel wordt aangerekend."
|
||||
price_taxes_eur_kwh:
|
||||
name: "Tarief · Heffingen en btw (EUR/kWh)"
|
||||
description: "Belastingen en heffingen per kWh, opgeteld volgens dezelfde regel als de marge: alleen wanneer de gekozen prijsbron ze nog niet bevat."
|
||||
|
||||
Reference in New Issue
Block a user