Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e6c45a61e7 | ||
|
|
88c59a9918 | ||
|
|
17b7b447c4 | ||
|
|
4257e38e5a |
@@ -16,6 +16,169 @@ if it has, it needs 0.10.0. Whoever deploys decides.
|
||||
| _this branch_ | FCST-05 - one byte stops stopping the add-on, and the write path gets the read path's standard |
|
||||
| _this branch_ | FCST-06 - "cannot persist" stops meaning "has no /data" and starts meaning what it says |
|
||||
| _this branch_ | TARIFF-08 - the day-ahead feed is read in the shape it actually publishes, and the fixtures stop agreeing with the parser |
|
||||
| _this branch_ | TARIFF-09 - the day-ahead feed is asked for the horizon it was given, so tomorrow arrives firm instead of carried |
|
||||
| _this branch_ | TARIFF-10 - configuring a price provider is what fetches prices; trading stays its own switch |
|
||||
|
||||
### TARIFF-10 - a configured provider is fetched, whether or not you trade
|
||||
|
||||
**`amain()` started the background tariff refresh only when
|
||||
`arbitrage_enabled` was on, and that option ships `false`.**
|
||||
`run_tariff_refresh` is the only caller of `resolve_tariff()`, which is the
|
||||
only caller of `price_history.carry()` - so on a shipped install with a price
|
||||
provider configured **nothing was ever fetched**: no plan resolved,
|
||||
`/data/price_history.json` never created, the *Price history* status row
|
||||
reading `empty` for the life of the install, and TARIFF-09's short-horizon
|
||||
warning unable to fire because there was no fetch to warn about.
|
||||
|
||||
Measured over a 40 h soak of two instances side by side on the same feed:
|
||||
|
||||
| | shipped default + provider | same, arbitrage armed |
|
||||
|---|---|---|
|
||||
| feed requests | **0** | fetching |
|
||||
| `/data/price_history.json` | `present: false` on **all 802 samples** | written |
|
||||
| resolved plan | none | 132 points |
|
||||
|
||||
⚠️ **`DOCS.md` has described the fixed behaviour since the commit that wrote
|
||||
the gate.** OPT-04 (`bdd7f18`) added the refresh task, the
|
||||
`arbitrage.enabled()` guard on it, and the sentence *"turning a provider on
|
||||
still fetches, validates and publishes prices and changes no charging
|
||||
decision"* — all in one change. `config.yaml` has said the same since
|
||||
TARIFF-02 (`852fc7c`). The documentation and the code have therefore disagreed
|
||||
for as long as both existed, and nothing compared them. `config.yaml` and both translations carried the same
|
||||
claim, plus a second one that had gone stale in the other direction (*"nothing
|
||||
consumes the price series yet"*, untrue since OPT-04). All four now say what
|
||||
the add-on does.
|
||||
|
||||
**The gate is `tariff_refresh_wanted()`, and it is a STRICT WIDENING.**
|
||||
|
||||
```
|
||||
provider_configured(opts) or arbitrage.enabled(opts)
|
||||
```
|
||||
|
||||
Every configuration that started the refresh before still starts it. The second
|
||||
term is not redundant and is not there for symmetry: `price_provider: off` with
|
||||
a `static_profile` and `arbitrage_enabled: true` is a configuration
|
||||
`validate.py` explicitly **accepts**, and it trades - measured, at
|
||||
`cycle_cost_threshold_eur: 0.05` on a 0.20/0.284 EUR/kWh day/night profile,
|
||||
`ARBITRAGE_CHARGE -5000 W`. Gating the refresh on the provider alone would
|
||||
leave that site's `tariff_plan` at `None` forever and answer *"no tariff plan
|
||||
has been resolved yet"* on every cycle - silently disarming a strategy the UI
|
||||
says is running.
|
||||
|
||||
⚠️ **Nothing acquires a periodic HTTPS call from this.** With `price_provider`
|
||||
off, `resolve_tariff()` answers out of the static profile without opening a
|
||||
socket; and the true shipped default - no provider, no trading - starts **no
|
||||
task at all**. What decides whether a request leaves the machine is
|
||||
`price_provider`, and only that.
|
||||
|
||||
⚠️ **AND NOTHING TRADES THAT DID NOT TRADE BEFORE.** `cycle()`'s
|
||||
`if arbitrage.enabled(self.o):` is untouched. Graded directly rather than
|
||||
inferred from the gate having moved: `Controller.cycle()` is driven on a
|
||||
resolved provider plan whose cheapest firm hour is now, with the peak guard
|
||||
asserted clear, and `arbitrage_enabled` is the single option that moves between
|
||||
the rows.
|
||||
|
||||
| `arbitrage_enabled` | cheapest hour is now | commanded |
|
||||
|---|---|---|
|
||||
| `false` | yes | **+180 W** (`loop -> 180 W`) |
|
||||
| `false` | no | **+180 W** (`loop -> 180 W`) |
|
||||
| `true` | yes | -5000 W (`arbitrage -> -5000 W`) |
|
||||
| `true` | no | +5000 W (`arbitrage -> 5000 W`) |
|
||||
|
||||
**`TariffPlan.cheap_now()` is deliberately NOT wired, and that is now a
|
||||
decision rather than an inheritance.** A provider-backed plan exists on every
|
||||
configured site from this commit, so *Cheap window* - which feeds the
|
||||
maintenance cycle's grid top-up, i.e. the one place a price reaches something
|
||||
that **moves energy** without trading being armed - could now read it. It does
|
||||
not: `self.cheap` still comes from `static_status()`, and on a provider-only
|
||||
site (`static_profile` empty) that is `False` on every cycle, so no paid grid
|
||||
top-up is ever forced. Wiring it would take that from *never* to *sometimes*.
|
||||
`DECISIONS/TARIFF-10-fetching-is-not-trading.md` §4 carries the argument and
|
||||
the list of what adopting it would take.
|
||||
|
||||
**The check that would have caught this** is `test_price.py` section 13, and it
|
||||
grades **reachability from a configuration**, not the shape of an `if`. The
|
||||
guards in `amain()` are *evaluated* against real option dicts, whatever they are
|
||||
spelled as; the call graph under `app/` is walked to see whether the tasks those
|
||||
guards start can arrive at `resolve_tariff()`, wherever it has moved to; and
|
||||
then the fetch is **driven** with a counting transport, because a graph is an
|
||||
argument and a counted request is an observation. The shipped defect - the gate
|
||||
back at `arbitrage.enabled(opts)`, **-4 B** - is red on seven named checks.
|
||||
|
||||
### TARIFF-09 - the feed is asked for the range the caller wanted
|
||||
|
||||
**`generic_dayahead` took `since` and `until` and requested the bare feed URL,
|
||||
which answers one Brussels day.** Everything past today was then filled from
|
||||
yesterday's profile by FCST-04, flagged provisional, and §8.3 drops provisional
|
||||
hours from the trade set - so **the optimiser never saw tomorrow as firm, at any
|
||||
hour of any day.** Measured on `4257e38` at 2026-09-10 09:00 CEST over the
|
||||
shipped 48 h horizon, live feed, yesterday seeded:
|
||||
|
||||
| | 48 h horizon, one-day feed |
|
||||
|---|---|
|
||||
| firm | **15 whole hours** (60 quarter-hour points) |
|
||||
| carried, provisional | **33 hours** - 24 on D+1, 9 on D+2 |
|
||||
| permitted cycle | buy 11:45, sell 19:45 - **both on 2026-09-10** |
|
||||
| firm local days | `[2026-09-10]` |
|
||||
|
||||
The ceiling was structural, not incidental: with a one-day window, at hour *H*
|
||||
of the local day a 48 h horizon is at most **24−H hours firm and 24+H carried**.
|
||||
At 23:00 that is **1 firm hour and 47 carried**, and the remaining firm spread
|
||||
(0.0464 EUR/kWh) no longer clears the default 0.10 round-trip threshold at all -
|
||||
so arbitrage was refused outright. This is **not** what FCST-04's fill is for:
|
||||
that covers the window until the ~13:00-14:00 CET auction, after which a healthy
|
||||
feed publishes tomorrow in full.
|
||||
|
||||
**The data exists and the endpoint serves it.** Verified 2026-09-10T07:45:49Z:
|
||||
`?bzn=BE&start=2026-09-09&end=2026-09-10` returns 192 quarter-hours across two
|
||||
local days, committed unmodified beside TARIFF-08's captures. The plugin simply
|
||||
never asked.
|
||||
|
||||
**The range is now derived from `since`/`until` on the BRUSSELS calendar**, as
|
||||
two inclusive dates. Not the UTC calendar: 22:30 UTC in summer is already
|
||||
tomorrow in Belgium, so a UTC-derived start asks for the day the controller has
|
||||
just left and drops the day it is standing in, for the last one or two hours of
|
||||
every day. That is `price_history._bdate()`'s lesson one layer up, with a worse
|
||||
consequence - `_bdate()` misfiles an hour, this never fetches it. `until` is
|
||||
exclusive, so a horizon ending exactly at local midnight asks for one day fewer
|
||||
than the naive reading gives, and an inverted window collapses to a single day
|
||||
rather than a backwards range.
|
||||
|
||||
Same code path, same window, a feed serving both days:
|
||||
|
||||
| | before | after |
|
||||
|---|---|---|
|
||||
| firm hours of 48 | 15 | **39** |
|
||||
| carried, provisional | 33 | 9 |
|
||||
| firm local days | 1 | **2** |
|
||||
| permitted cycle | one calendar day | **buys on one day, sells on the next** |
|
||||
|
||||
The 9 remaining carried hours are the tail of the horizon that runs into a third
|
||||
Brussels day, which no day-ahead market publishes. FCST-04 still fills them and
|
||||
still flags them; what changed is what is firm, never what may be acted on.
|
||||
|
||||
**Before publication nothing regresses, and that is graded rather than
|
||||
assumed.** Asking for tomorrow before the auction returns fewer days than
|
||||
requested - measured, at 09:46 CEST the two-day request came back byte-identical
|
||||
to the one-day body - and that stays a `series` of what exists, with FCST-04
|
||||
filling the remainder. A `no_data` there would switch a working site's arbitrage
|
||||
off every morning.
|
||||
|
||||
⚠️ **`price_feed_url` is the EXACT URL and the plugin appends nothing to it, so
|
||||
an overriding site keeps the short horizon.** That is a decision, not an
|
||||
oversight: `start`/`end` are this feed's spelling, the provider declares a second
|
||||
host whose parameters are different, and rewriting a URL an operator typed is
|
||||
how a request nobody intended gets made. **It is not silent** - the plugin logs a
|
||||
`WARNING` at fetch time naming the consequence and the `start..end` it would
|
||||
otherwise have asked for. See
|
||||
`DECISIONS/TARIFF-09-the-range-and-the-override.md`.
|
||||
|
||||
The plugin contract - `fetch(self, opts, since, until, transport)` hands every
|
||||
plugin the range - is still **not** enforced, because three shipped plugins point
|
||||
at endpoints nobody has verified and a rule obliging them to pass a range would
|
||||
be satisfied by an invented parameter name. Which plugins consume the range is
|
||||
pinned as a literal in `test_price.py` instead, so the next supplier has to
|
||||
decide rather than inherit the silence.
|
||||
|
||||
### FCST-06 - the second way to be a store that cannot persist
|
||||
|
||||
|
||||
@@ -0,0 +1,141 @@
|
||||
# TARIFF-09 — asking the feed for the horizon, and what `price_feed_url` means
|
||||
|
||||
Two decisions. The first had no defensible alternative once it was measured;
|
||||
the second did, and the alternative is argued below rather than dismissed.
|
||||
|
||||
---
|
||||
|
||||
## 0. The measurement this card exists for
|
||||
|
||||
`generic_dayahead.fetch()` accepted `since` and `until` and requested the bare
|
||||
`https://api.energy-charts.info/price?bzn=BE`. That URL names no range, and the
|
||||
feed answers **one Brussels day**.
|
||||
|
||||
Measured on the merged tree (`4257e38`) at 2026-09-10 09:00 CEST, 48 h horizon,
|
||||
live feed, yesterday seeded through the shipped read path:
|
||||
|
||||
```
|
||||
60 firm points (15 whole hours) + 33 carried hours of 48 requested
|
||||
carried: 24 h on 2026-09-11, 9 h on 2026-09-12
|
||||
arbitrage(0.10): allowed, spread 0.4645, buy 11:45 and sell 19:45 -- BOTH on 2026-09-10
|
||||
firm local days = [2026-09-10]
|
||||
```
|
||||
|
||||
Everything past today was carry-forward, flagged provisional, and §8.3 drops
|
||||
provisional hours from the trade set. **The optimiser never saw tomorrow as
|
||||
firm, at any hour.** The ceiling was structural rather than incidental: with a
|
||||
one-day window, at hour *H* of the local day a 48 h horizon is at most
|
||||
**24−H hours firm and 24+H carried** — 15/33 at 09:00, and **1/47 at 23:00**,
|
||||
where the remaining firm spread (0.0464 EUR/kWh) no longer even clears the
|
||||
default 0.10 round-trip threshold and arbitrage is refused outright.
|
||||
|
||||
That is not what FCST-04's fill was built for. FCST-04 covers the
|
||||
pre-publication window until the ~13:00–14:00 CET auction, after which a
|
||||
healthy feed hands over tomorrow in full.
|
||||
|
||||
**The data exists and the endpoint serves it.** Verified 2026-09-10T07:45:49Z:
|
||||
`?bzn=BE&start=2026-09-09&end=2026-09-10` returns **192 quarter-hours across
|
||||
two local days**, committed unmodified as
|
||||
`scenarios/energy-charts-be-price-2026-09-09_2026-09-10-twoday.json`. The
|
||||
plugin simply never asked.
|
||||
|
||||
After the change, same code path, same window, a feed serving both days:
|
||||
**39 firm hours and 9 carried**, firm across **two** Brussels days, and the
|
||||
permitted cycle buys on one day and sells on the other. Graded in
|
||||
`test_price_history.py` section 1c against both feed states.
|
||||
|
||||
---
|
||||
|
||||
## 1. The range is derived on the **Brussels** calendar, from `since`/`until`
|
||||
|
||||
`since` and `until` are **instants**; the API takes **dates**. Those are not
|
||||
the same question, and the conversion is where the day gets lost.
|
||||
|
||||
`brussels_date_range()` returns two inclusive `date`s:
|
||||
|
||||
* the first is `brussels(since).date()`. **Not `since.date()`, and not the UTC
|
||||
date.** 22:30 UTC in summer is already tomorrow in Belgium, so a UTC-derived
|
||||
start asks for the day the controller has just left and drops the day it is
|
||||
standing in — for the last one or two hours of *every* day. This is
|
||||
`price_history._bdate()`'s lesson one layer up, with a worse consequence:
|
||||
`_bdate()` misfiles an hour, this never fetches it.
|
||||
* the last is `brussels(until).date()`, **stepped back a day when `until`
|
||||
falls exactly on local midnight**, because `until` is exclusive and that
|
||||
instant is the first tick of a day nobody asked for. Without it every
|
||||
whole-hour horizon requests one extra day — harmless on this feed, but it
|
||||
makes the request a claim the caller never made.
|
||||
* an empty or inverted window collapses to a single day rather than producing
|
||||
a backwards range, which no feed has a defined answer for.
|
||||
|
||||
Both ends are graded, and both are driven red by mutation — including the two
|
||||
that swap `brussels()` for the UTC date (`test_price.py` section 2c).
|
||||
|
||||
---
|
||||
|
||||
## 2. `price_feed_url` is the **exact URL**, used as given. The plugin appends
|
||||
nothing.
|
||||
|
||||
The alternative — merge our `start`/`end` into whatever query string the
|
||||
operator supplied — was considered and rejected. Three reasons, in the order
|
||||
they decided it:
|
||||
|
||||
1. **`start`/`end` are energy-charts' spelling, and this provider declares a
|
||||
second host.** `generic_dayahead.capabilities().egress_hosts` is
|
||||
`("api.energy-charts.info", "web-api.tp.entsoe.eu")`. That second host's
|
||||
day-ahead date parameters have not been verified by this card and are not
|
||||
guessed at here — but they are not `start`/`end`. So an override to the
|
||||
*other declared host* — a case the declaration itself anticipates — is
|
||||
exactly where merging appends two parameters that host ignores, producing
|
||||
the short horizon anyway, with a URL nobody typed as the receipt.
|
||||
2. **Merging means re-serialising a URL somebody else wrote.** Parsing and
|
||||
rebuilding reorders parameters, collapses duplicates and re-escapes values.
|
||||
"Override the feed" is what the option already says it means, and an
|
||||
override that is silently rewritten is not one.
|
||||
3. **Blind appending is not on the table at all.** `dated_feed_url()` refuses
|
||||
to add a query string to a base that already has one, so
|
||||
`?bzn=BE?bzn=BE&start=…` cannot be produced by this module even by a later
|
||||
edit that folds the parameters back into the constant.
|
||||
|
||||
⚠️ **The cost is real and it is therefore NOT a doc footnote.** An overriding
|
||||
operator gets exactly the short horizon this card exists to fix. So the plugin
|
||||
emits a `WARNING` on the shipped `goodwe.price` logger at **fetch time**, naming
|
||||
the consequence — one Brussels day, the rest carried and flagged, §8.3 dropping
|
||||
them from the trade set — and naming the `start..end` it *would* have asked
|
||||
for. §11.4 is what makes a price defect quiet; an operator who has taken the
|
||||
short horizon has to be able to find out from the log that they did.
|
||||
|
||||
The warning is graded, with a control that the default path stays silent, and
|
||||
a mutation that downgrades it to `debug()` takes that check red.
|
||||
|
||||
**Fetch time rather than startup**, because that is where the URL actually in
|
||||
use is known. The trade-off is one line per refresh (the tariff task, not the
|
||||
30 s control tick). If the status page ever grows a price-source row, this is
|
||||
the sentence it should carry.
|
||||
|
||||
---
|
||||
|
||||
## 3. The plugin contract: `fetch(self, opts, since, until, transport)`
|
||||
|
||||
The signature promises every plugin the range, and until this card **none of
|
||||
the four networked plugins used it**. The obvious follow-up is to enforce the
|
||||
promise. It is **not** enforced, and the reason is TARIFF-08's:
|
||||
|
||||
Three of the five shipped plugins point at endpoints nobody has ever verified
|
||||
(TARIFF-07 is the open decision on withdrawing them). A rule obliging them to
|
||||
pass a date range would be satisfied by an **invented parameter name** — which
|
||||
is the parser-graded-against-its-own-guess defect moved from the response to
|
||||
the request, and it would ship looking like a guarantee.
|
||||
|
||||
What exists instead is a **roster pin** in `test_price.py` section 2c: which
|
||||
plugins reference their `since`/`until` is asserted as a literal, derived from
|
||||
the registry and the AST. A sixth supplier, or a change to any of the three,
|
||||
has to edit that line and make the decision rather than inherit the silence.
|
||||
It is named as a source-shape pin, not as evidence that any feed parses;
|
||||
`generic_dayahead`'s behaviour is graded separately by the URL it really
|
||||
requests.
|
||||
|
||||
**Recommendation for whoever picks up TARIFF-07:** enforce it at the moment
|
||||
those three endpoints are either verified or withdrawn, not before. If they are
|
||||
withdrawn, `generic_dayahead` and `manual_csv` are the whole roster and the
|
||||
rule is one line — `manual_csv` reads a whole file and is windowed downstream,
|
||||
so it would be the single documented exemption.
|
||||
@@ -0,0 +1,277 @@
|
||||
# TARIFF-10 — fetching a price and trading on one are two decisions
|
||||
|
||||
**Status: DECIDED 2026-09-12 — Option A. If a price provider is configured,
|
||||
prices are fetched. Trading stays a separate switch (`arbitrage_enabled`), and
|
||||
this card moves nothing that moves energy.**
|
||||
|
||||
Filed with TARIFF-10. Follows [[TARIFF-09]] and [[FCST-04]]; §4 settles the
|
||||
limitation FCST-04 recorded and did not decide.
|
||||
|
||||
---
|
||||
|
||||
## 1. The defect, and why it was invisible
|
||||
|
||||
`app/main.py`'s `amain()` read:
|
||||
|
||||
```
|
||||
if arbitrage.enabled(opts):
|
||||
tasks.append(asyncio.create_task(controller.run_tariff_refresh()))
|
||||
```
|
||||
|
||||
`run_tariff_refresh` is the **only** caller of `resolve_tariff()`, which is the
|
||||
**only** caller of `price_history.carry()`. `arbitrage_enabled` ships `false`.
|
||||
|
||||
So on a shipped install with a provider configured:
|
||||
|
||||
* no HTTPS request was ever made to the feed;
|
||||
* no plan was ever resolved, so `self.tariff_plan` stayed `None` forever;
|
||||
* `/data/price_history.json` was never created;
|
||||
* the *Price history* status row read `empty` for the life of the install;
|
||||
* TARIFF-09's short-horizon warning could not fire, because it is logged on a
|
||||
fetch and there was no fetch.
|
||||
|
||||
Measured, not inferred — a 40 h soak, two instances, one feed:
|
||||
|
||||
| | shipped default + provider | same, arbitrage armed |
|
||||
|---|---|---|
|
||||
| feed requests | **0** | fetching |
|
||||
| `/data/price_history.json` | `present: false`, **802/802 samples** | written |
|
||||
| resolved plan | none | 132 points |
|
||||
|
||||
⚠️ **AND FOUR SURFACES ALREADY DESCRIBED THE FIXED BEHAVIOUR.** `DOCS.md`
|
||||
§*Energy price providers*: *"With `arbitrage_enabled: false`, which is the
|
||||
shipped default, turning a provider on still fetches, validates and publishes
|
||||
prices and changes no charging decision."* `config.yaml` and both translations
|
||||
said the same, and additionally carried a claim that had gone stale in the other
|
||||
direction — *"nothing consumes the price series yet"*, untrue since OPT-04
|
||||
shipped the consumer. The code was the only surface that disagreed with all of
|
||||
them, and no check compared the two.
|
||||
|
||||
The dating is worth being exact about, because it says this was never a
|
||||
regression. `git log -S` puts the refresh task, its `arbitrage.enabled()` guard
|
||||
**and** the DOCS sentence in the *same commit* — OPT-04, `bdd7f18`;
|
||||
`config.yaml`'s version of the claim goes back further, to TARIFF-02,
|
||||
`852fc7c`. The prose and the code disagreed from the moment both existed.
|
||||
|
||||
## 2. Why a check on the `if` would not have caught it
|
||||
|
||||
The `if` was exactly what somebody meant to write, and its comment argued for
|
||||
it honestly:
|
||||
|
||||
> ⚠️ STARTED ONLY WHEN ARBITRAGE IS ARMED. Nothing else in the add-on reads the
|
||||
> resolved plan … so on a default install this task would be network traffic in
|
||||
> aid of an attribute nobody reads.
|
||||
|
||||
That was true when OPT-04 wrote it and stopped being true when FCST-04 gave the
|
||||
plan a *store* and a *status row*. A pin on the expression would have gone red
|
||||
on the fix and green on the defect. What was wrong is the **set of
|
||||
configurations from which a fetch is reachable**, so that is what
|
||||
`test_price.py` section 13 grades:
|
||||
|
||||
1. the guards on every `asyncio.create_task` in `amain()` are **evaluated**
|
||||
against real option dicts — whatever the guard is spelled as;
|
||||
2. the call graph under `app/` is walked to see whether the coroutines those
|
||||
guards start can arrive at `resolve_tariff()` — wherever it has moved to;
|
||||
3. and then the fetch is **driven**, with a counting transport, a real
|
||||
`Controller` and a real store path, because a reachability graph is an
|
||||
argument and a counted request is an observation.
|
||||
|
||||
⚠️ **Step 3 grades the gate only because it starts the task the way `amain()`
|
||||
would.** The driver asks `_fetchers(opts)` — the evaluated guard — before it
|
||||
calls the coroutine. Measured: with the shipped defect restored **and** that one
|
||||
line defeated (`if False and "run_tariff_refresh" not in names:`), the four
|
||||
driven rows go **green** while only the structural rows stay red. Without it
|
||||
they are decoration, which is MUTATION.md's habit 4 in a harness.
|
||||
|
||||
What the instrument **cannot** see is printed on every run: a coroutine started
|
||||
by anything but `create_task`; a guard that is not an enclosing `if` in
|
||||
`amain()`; anything outside `app/`; and — because the graph is keyed by **name**
|
||||
— two same-named functions as one node, which makes every *reachable* answer an
|
||||
**upper** bound and every *not reachable* answer the strong one.
|
||||
|
||||
## 3. The gate, and why it is an `or`
|
||||
|
||||
```
|
||||
def tariff_refresh_wanted(opts: dict) -> bool:
|
||||
return provider_configured(opts) or arbitrage.enabled(opts)
|
||||
```
|
||||
|
||||
The first term is what this card adds. **The second term is the old gate, kept
|
||||
verbatim, and it is not there for symmetry.**
|
||||
|
||||
`price_provider: off` + a `static_profile` + `arbitrage_enabled: true` is a
|
||||
configuration `validate.py` explicitly **accepts** (rule 7 refuses only the
|
||||
no-prices-at-all case), and it **trades**. Measured on this branch:
|
||||
|
||||
```
|
||||
opts: price_provider off, arbitrage_enabled true,
|
||||
cycle_cost_threshold_eur 0.05,
|
||||
static_profile 0.20 night / 0.284 day
|
||||
plan : static_profile from static_profile, 48 h, 48 points
|
||||
WITH the plan -> ARBITRAGE_CHARGE -5000.0 W
|
||||
"cheapest firm hour at 0.2000 EUR/kWh, 0.0840 below the
|
||||
dearest and clear of the 0.0750 this cycle had to cover"
|
||||
WITHOUT the plan -> None "no tariff plan has been resolved yet"
|
||||
```
|
||||
|
||||
Gating the refresh on the provider **alone** — which is the literal reading of
|
||||
the ticket's second acceptance criterion — would leave that site's
|
||||
`tariff_plan` at `None` forever and answer *"no tariff plan has been resolved
|
||||
yet"* on every cycle. That is a strategy the UI says is running and that cannot
|
||||
fire: the exact state `validate.py`'s OPT-04 rules exist to refuse, arrived at
|
||||
from inside the code instead of from the configuration.
|
||||
|
||||
So the gate is a **strict widening**: every configuration that started the task
|
||||
before still starts it, and `test_price.py` grades that as its own row rather
|
||||
than leaving it to prose.
|
||||
|
||||
⚠️ **The AC's stated harm is still fully met, and it is worth being exact about
|
||||
this rather than quietly declaring victory.** The criterion reads *"with
|
||||
`price_provider: off`: no fetch task at all, whatever `arbitrage_enabled` says
|
||||
— an operator with no provider must not acquire a periodic HTTPS call."* The
|
||||
letter of it is not met in one configuration (provider off, trading armed),
|
||||
because that configuration keeps the task it already had. The harm it names is:
|
||||
|
||||
* with `price_provider` off, `resolve_tariff()` returns `static_plan()` — pure
|
||||
arithmetic, **no socket**. Driven and counted: **0 requests**.
|
||||
* the true shipped default — no provider, no trading — starts **no task at
|
||||
all**.
|
||||
|
||||
Nobody *acquires* a periodic HTTPS call from this card, and the only thing that
|
||||
decides whether a request leaves the machine is `price_provider`. **If the
|
||||
owner wants the literal reading instead, it is a one-token change** (drop the
|
||||
`or` term) **and it costs the site in the measurement above its trading.** That
|
||||
is the trade, stated rather than chosen silently.
|
||||
|
||||
## 4. `TariffPlan.cheap_now()` — available, and deliberately NOT consumed
|
||||
|
||||
**Decision: do not wire it in this card. `self.cheap` stays on
|
||||
`static_status()`.**
|
||||
|
||||
FCST-04 recorded that `cheap_now()` has no production caller on the provider
|
||||
plan: `main.read_inputs()` assigns `self.cheap, self.tariff_words =
|
||||
static_status(o, now)`, which builds its **own** static plan and calls
|
||||
`cheap_now()` on that. Under Option A a provider-backed plan now exists on every
|
||||
configured site, so the limitation stops being structural and becomes a choice.
|
||||
|
||||
**It is not cosmetic, and that is the whole reason for the decision.**
|
||||
`self.cheap` is read in `cycle()` by the maintenance cycle's cheap-window grid
|
||||
top-up — the add-on's one shiftable load, and the one place a *price* reaches
|
||||
something that **moves energy** on an install that has not armed trading.
|
||||
|
||||
⚠️ **And the direction it would move is not "a slightly different window".**
|
||||
`static_status()` returns `(False, "no static profile")` whenever
|
||||
`static_profile` is empty, so today a provider-only site — a plausible shape
|
||||
for a dynamic-tariff install, and the shape this card makes work — has
|
||||
`self.cheap = False` on **every** cycle and never forces a paid grid top-up at
|
||||
all. Wiring `cheap_now()` would take that from *never* to *sometimes*. That is
|
||||
a strictly larger behaviour change than this card's headline, and it belongs to
|
||||
whoever owns the maintenance cycle rather than to the ticket that made it
|
||||
reachable.
|
||||
|
||||
Three further reasons, in the order they would bite:
|
||||
|
||||
1. **Two sources, one attribute, and the loser is silent.** `validate.py`
|
||||
already refuses `static_profile` alongside the Home Assistant price
|
||||
entities *because* `read_inputs()` picks one and the other disappears. A
|
||||
provider plan would be a third writer of the same attribute with no rule
|
||||
saying which wins.
|
||||
2. **The freshness question is not answered for this consumer.** The plan
|
||||
retires after `TARIFF_MAX_AGE_S` and `fresh_tariff_plan()` returns `None`.
|
||||
Arbitrage has an answer for that (refuse). The maintenance top-up has none:
|
||||
"the plan expired" would have to mean *not cheap*, and *not cheap* on this
|
||||
path means **wait for sun**, which is a real decision nobody has taken.
|
||||
3. **The comparison basis changes, and so does the answer on a bad feed.**
|
||||
`cheap_now()` is strictly-below-the-mean-of-the-**firm**-hours, and it is
|
||||
`False` when THIS hour is provisional or is not in the series at all
|
||||
(FCST-03's rule: this hour decides, not the whole plan). So a feed that
|
||||
published nothing, or an hour the §8.3 fill could not cover, reads *not
|
||||
cheap* where a `static_profile` reads whatever the clock says. Defensible
|
||||
either way — and it is a change to the shipped maintenance cycle that needs
|
||||
its own measurement rather than a side effect of this one.
|
||||
|
||||
**What adopting it would take**, so the next card does not start from scratch:
|
||||
|
||||
* a rule for which source wins when both a provider plan and a
|
||||
`static_profile` are configured, enforced in `validate.py` the way the
|
||||
entity/profile conflict already is;
|
||||
* a decision on what an **expired** or **absent** plan means for the grid
|
||||
top-up, stated as a default rather than left to `None`;
|
||||
* `read_inputs()` splitting `self.cheap` from `self.tariff_words`, which are
|
||||
currently one assignment from one function;
|
||||
* a check that the maintenance top-up actually fires and does not fire on the
|
||||
provider plan — driven through `cycle()`, not asserted on the attribute;
|
||||
* and the *Tariff* status row, which FCST-04 pinned as a **pair** with
|
||||
`DOCS.md` in `test_price_history.py` §9: it does not report the provider
|
||||
plan, DOCS does not claim it does, and moving one without the other is
|
||||
already red.
|
||||
|
||||
## 5. The *Arbitrage* status row, re-read and left alone
|
||||
|
||||
`main.status_payload()` renders `("Arbitrage", "off (arbitrage_enabled)" if
|
||||
self.arb is None else self.arb.explain())`. `self.arb` is assigned only inside
|
||||
`if arbitrage.enabled(self.o):` and `arbitrage.decide()` never returns `None`,
|
||||
so `self.arb is None` **iff** the option is off. The string still names the
|
||||
option that is actually off, and under Option A it gains a job it did not have:
|
||||
on a site with a provider configured it is the row that tells an operator the
|
||||
prices arriving on the *Price history* row above are moving nothing. Unchanged,
|
||||
with a note at the call site saying it was checked.
|
||||
|
||||
## 6. What this card did not do
|
||||
|
||||
* It did not touch `run_tariff_refresh()`'s body (TARIFF-12 is there).
|
||||
* It did not change `cycle()`'s `if arbitrage.enabled(self.o):`, the sizing,
|
||||
the thresholds, the SoC bounds or the peak interlock.
|
||||
* It did not wire the provider plan into the *Tariff* status row or into
|
||||
`self.cheap` — §4.
|
||||
* It did not pick a version number (PKG-06).
|
||||
|
||||
## 7. Mutation ledger
|
||||
|
||||
Every mutation applied to a **fresh `cp -a`** of the clean worktree rather than
|
||||
reverted (habit 18: there is no revert to verify and no shared scratch path),
|
||||
`__pycache__` purged, tree mounted `:ro`, run under `python3 -B`.
|
||||
|
||||
| id | mutation | Δ bytes | result |
|
||||
|---|---|---|---|
|
||||
| M1 | the shipped defect: gate back to `arbitrage.enabled(opts)` | **-4 B** | **RED**, 7 named checks |
|
||||
| M2 | drop the `or arbitrage.enabled(opts)` term | -27 B | **RED**, 2 named checks (strict-widening + the static-profile row) |
|
||||
| M3 | `tariff_refresh_wanted` -> always `True` | +8 B | **RED**, the AC2 reachability row |
|
||||
| M4 | `provider_configured`: `"off"` counts as configured | -6 B | **RED**, the AC2 reachability row |
|
||||
| M5 | `cycle()`'s trading gate -> `if True or arbitrage.enabled(self.o)` | +8 B | **RED**, both AC3 rows (`-5000 W` with trading off) |
|
||||
| M6 | instrument: `_reaches()` always `True` | **+0 B** | **RED**, 4 checks incl. its own calibration row |
|
||||
| M7 | instrument: `_task_sites()` returns `[]` | **+0 B** | **RED**, 10 checks incl. "it found 0 sites" |
|
||||
| M8 | fixture: 5 minutes of telegrams instead of a whole quarter | **+0 B** | **RED**, the peak-guard precondition and both armed rows |
|
||||
| M9 | M1 **plus** the harness's guard-driven start defeated | -4 B | the four DRIVEN rows go **GREEN** — see §2 |
|
||||
| M10 | `provider_id()` drops `.strip()` | -8 B | **survivor**, then CLOSED — see below |
|
||||
| M11 | `provider_configured()` -> `return True` | -32 B | **RED**, 4 checks |
|
||||
| M12 | DOCS.md's sentence reverted to the old, vaguer claim | +0 B (docs) | **RED**, the AC4 pairing row (`fetches = True, DOCS says = False`) |
|
||||
| M13 | the Dutch translation left behind while English moved | +0 B (docs) | **RED**, the both-languages row |
|
||||
|
||||
⚠️ **M10 was a real survivor and is recorded as one.** Dropping `.strip()` left
|
||||
`test_price.py`, `test_price_history.py`, `test_static_tariff.py` and
|
||||
`test_validate.py` all green: nothing in the repository fed a `price_provider`
|
||||
with surrounding whitespace, and the `.strip()` predates this card — it was
|
||||
carried into `provider_id()` from the three call sites it replaced. The
|
||||
Supervisor's `list(off|…)` schema cannot produce `" off "`, but
|
||||
`validate_startup_config()` is also `sim/inject.py`'s entry point and takes any
|
||||
dict at all. Closed in the same commit by the padded-id pair in section 13;
|
||||
re-run as **M10b**, red on two named checks.
|
||||
|
||||
⚠️ **M8 is the fixture defect this card walked into and then fenced.** The
|
||||
first draft fed five minutes of telegrams. Whether that **covers** the current
|
||||
billing quarter depends on where the wall clock sits inside it, so the peak
|
||||
guard was blind and engaged on some runs and not others — and `decide()` refuses
|
||||
outright while the guard is holding (§9.1 over §10.1). On those runs the armed
|
||||
baseline did not trade, and "trading off never trades" would have been graded
|
||||
on a fixture in which nothing could trade either way. It is now 900 s of
|
||||
telegrams, and the guard's clear state is a **check** rather than an assumption.
|
||||
|
||||
## 8. What is still open
|
||||
|
||||
* The literal reading of AC2 (§3). One token, and it costs a measured trading
|
||||
configuration its plan. **Owner's call.**
|
||||
* `cheap_now()` (§4). Owner's call, with the adoption list above.
|
||||
* The *Tariff* status row still shows the static-profile sentence on a
|
||||
provider-backed site; the provider plan's own line goes to the log. Pinned as
|
||||
a pair with `DOCS.md` by FCST-04 and untouched here.
|
||||
@@ -1346,12 +1346,26 @@ does, and only after you have read why.
|
||||
> release ships the five provider plugins, the canonical all-in format they
|
||||
> emit, the config that selects one, and — new in OPT-04 — a price-arbitrage
|
||||
> strategy that trades on them. With `arbitrage_enabled: false`, which is the
|
||||
> shipped default, turning a provider on still fetches, validates and publishes
|
||||
> prices and changes no charging decision.
|
||||
> shipped default, **selecting a provider is what starts the fetch**: the feed
|
||||
> is read every 30 minutes, validated, and the day recorded in
|
||||
> `/data/price_history.json`, which the *Price history* status row reports and
|
||||
> the add-on log names at every refresh. No charging decision changes.
|
||||
>
|
||||
> ⚠️ **That has been the documented behaviour since OPT-04 and was not the
|
||||
> actual behaviour until TARIFF-10.** The background refresh was started only
|
||||
> when `arbitrage_enabled` was on — and it ships `false` — so an install with a
|
||||
> provider configured fetched nothing at all, never created the store, and read
|
||||
> *Price history: empty* for as long as it ran. If you configured a provider and saw an empty
|
||||
> Price history row, that is what you were looking at. Trading is unchanged and
|
||||
> is still `arbitrage_enabled` alone.
|
||||
>
|
||||
> `static_profile` below needs no network at all, so its cheap window is
|
||||
> computed on every cycle and reaches the maintenance cycle's grid top-up
|
||||
> whether or not arbitrage is armed.
|
||||
> whether or not arbitrage is armed. ⚠️ **And it is still the only source of
|
||||
> that window.** A provider-backed plan now exists on every configured site,
|
||||
> but *Cheap window* and the maintenance cycle's grid top-up read
|
||||
> `static_profile` and nothing else — a deliberate limit, recorded with what it
|
||||
> would take to change in `DECISIONS/TARIFF-10-fetching-is-not-trading.md`.
|
||||
|
||||
| option | meaning |
|
||||
|---|---|
|
||||
@@ -1360,7 +1374,7 @@ does, and only after you have read why.
|
||||
| `price_taxes_eur_kwh` | Levies and VAT, EUR/kWh. Same rule |
|
||||
| `export_price_eur_kwh` | The **injection** price. A separate scalar, not a series — see below |
|
||||
| `price_csv_path` | Where `manual_csv` reads its file |
|
||||
| `price_feed_url` | Override the plugin's default feed. Only a host the plugin itself declares is accepted |
|
||||
| `price_feed_url` | Override the plugin's default feed. Used **exactly as written** — the add-on appends nothing to it, including the date range, so an override gets whatever horizon that URL returns. Only a host the plugin itself declares is accepted |
|
||||
| `price_horizon_h` | How far ahead to ask for, hours |
|
||||
| `price_timeout_s` | Per-request timeout |
|
||||
| `static_profile` | A static day/night or fully-fixed tariff. Empty by default — see below |
|
||||
@@ -1408,6 +1422,29 @@ point `price_feed_url` somewhere else:
|
||||
* Belgium moved to a **15-minute market time unit** on 2025-10-01, and the feed
|
||||
followed. A published day is **96 points, not 24**. Nothing in the add-on
|
||||
assumes either spacing, and nothing aggregates the quarters away.
|
||||
* The request **names the days it wants**, derived from the horizon on the
|
||||
**Brussels** calendar — `?bzn=BE&start=2026-09-09&end=2026-09-10`, two
|
||||
inclusive local dates. Without that the feed answers a single day, and
|
||||
everything past today would be carried forward from yesterday's profile and
|
||||
flagged provisional — which §8.3 excludes from arbitrage, so tomorrow would
|
||||
never be tradeable at any hour. The calendar is Brussels rather than UTC
|
||||
because 22:30 UTC in summer is already tomorrow in Belgium.
|
||||
|
||||
**What you get before the auction publishes.** The day-ahead auction clears at
|
||||
about 13:00–14:00 CET. Ask for tomorrow before that and the feed returns fewer
|
||||
days than requested — that is normal, not a failure. The add-on keeps the
|
||||
series of what exists and fills the remaining hours from yesterday's profile,
|
||||
flagged provisional; arbitrage runs on the firm hours and ignores the flagged
|
||||
ones. So a dynamic site's plan is part firm and part carried every morning and
|
||||
fully firm once tomorrow is published, and neither state is an error.
|
||||
|
||||
⚠️ **If you set `price_feed_url`, you keep the short horizon.** The override is
|
||||
sent exactly as you wrote it and the add-on adds no `start`/`end` to it, because
|
||||
those are this feed's parameter names and an override may point somewhere with
|
||||
different ones — rewriting a URL you typed is how a request nobody intended gets
|
||||
made. The consequence is a **`WARNING` in the log on every price refresh**,
|
||||
naming the range that would otherwise have been requested. If you only wanted a
|
||||
different host and not a different horizon, the fix is to clear the option.
|
||||
|
||||
If the arrays ever come back different lengths, or the unit stops saying
|
||||
EUR/MWh, the fetch is refused outright rather than trusted in part: a price
|
||||
@@ -1647,6 +1684,14 @@ towards the firmware's 30 s failsafe. The plan refreshes every 30 minutes and
|
||||
retires itself if the refresh stops answering, at which point arbitrage stops
|
||||
with it.
|
||||
|
||||
⚠️ **That refresh is not gated on this switch** (TARIFF-10). It runs when a
|
||||
`price_provider` is configured *or* when arbitrage is armed, so turning
|
||||
arbitrage off does not turn the price feed off — and a site with no provider
|
||||
that arms arbitrage on a `static_profile` keeps the plan it has always had. The
|
||||
only thing that decides whether a request leaves the machine is
|
||||
`price_provider`: with it `off`, the refresh resolves the static profile in
|
||||
memory and opens no connection.
|
||||
|
||||
### Site
|
||||
|
||||
| option | meaning |
|
||||
|
||||
@@ -734,6 +734,45 @@ Cheap habits fall out of it, and all of them are now used in this repository:
|
||||
a no-op, and the mutation that proves the rendering is real belongs on
|
||||
`assemble()`, where it is now aimed.
|
||||
|
||||
20. ⚠️ **A FIXTURE GRADES THE RESPONSE. NOTHING ABOUT THE REQUEST IS REACHABLE
|
||||
FROM ONE, AND THE TRANSPORT HAD BEEN RECORDING IT ALL ALONG.** TARIFF-09,
|
||||
and it is habit 19's closed loop with the other half still shut.
|
||||
`generic_dayahead.fetch()` took `since` and `until` and asked for the bare
|
||||
`?bzn=BE`, which answers ONE Brussels day - so §8.3 dropped every hour past
|
||||
today from the trade set as provisional and the optimiser never saw
|
||||
tomorrow as firm, at any hour of any day. Measured at 09:00: **15 firm
|
||||
hours and 33 carried of 48**, both ends of the permitted cycle inside one
|
||||
calendar day; at 23:00, **1 and 47**, with the firm spread no longer
|
||||
clearing the round-trip cost.
|
||||
|
||||
TARIFF-08 had just fixed the RESPONSE side of this same plugin by
|
||||
committing real bytes. It could not have caught this: a captured body is
|
||||
the answer to a URL, and **no property of any body says which URL was
|
||||
asked for**. `canned()` in `test_price.py` had recorded `(url, timeout_s)`
|
||||
on every call since TARIFF-02 and not one check had ever asserted on the
|
||||
first element - so the harness *knew* the plugin was ignoring its arguments
|
||||
and nothing asked it. The fix is one line of habit: **where a plugin takes
|
||||
arguments that shape a request, assert on the REQUEST off the injected
|
||||
transport, not only on what comes back.** Section 2c does, and the
|
||||
shipped-defect mutation (restore the bare URL, `-173 B`) is red on six
|
||||
named checks in `test_price.py` and two in `test_price_history.py`.
|
||||
|
||||
⚠️ **AND A MUTATION THAT TURNS A READ INTO `no_data` CRASHES A SUITE
|
||||
INSTEAD OF NAMING A CHECK, UNLESS THE CHECK EXPECTS IT.** `_NoData.points`
|
||||
raises `Absence` by design, so `derived(...) < derived(...)` over two
|
||||
absent reads is a `TypeError` and `[p for p in plan.read.points]` is a
|
||||
traceback - habit 5's shape with an exception in place of a hang. The
|
||||
mutation that matters here is the "defensive" one somebody writes after
|
||||
reading this card: *refuse when the feed returned fewer days than we asked
|
||||
for*. It is exactly what a healthy feed does every morning before the
|
||||
~13:00-14:00 CET auction, and switching arbitrage off for it would be a
|
||||
worse defect than the one being fixed. It is red BY NAME in
|
||||
`test_price.py` (33 checks, the AC's own line among them) and takes
|
||||
`test_price_history.py` down at **FCST-04's section 1**, whose `_before`
|
||||
read is unguarded - a red, but an unnamed one. That is recorded rather than
|
||||
retrofitted: guarding another card's section is a change to what its
|
||||
checks report, and it belongs to whoever next touches it.
|
||||
|
||||
And the mutation-table consequence: **a variant is not covered because a
|
||||
near-miss of it is.** DEV-02's table listed the *actuator* variant of the
|
||||
per-unit claim defect (`device`/`supervised` filed without `inv`), which the
|
||||
@@ -791,6 +830,7 @@ commit and PR bodies.
|
||||
| **SAFETY-03** reviewer survivor: `abs()` dropped from `clamp_to_rating`'s limit (6aabb83) | **re-verified — still a survivor** | re-run the same way, all 15 green. Also as recorded: unreachable under the `int(100, 5000)` schema. |
|
||||
| **MAINT-05 round 3**, `migrate_maint_keys()` marker gate `if rep.ok:` → `if rep.ok or rep.kept:` (`+12 B`) | **was a survivor — now CLOSED** | measured against head `65cddf3` under the purge, tree mounted `:ro`, `19/19 PASS`. It stamps the schema marker on a migration that LOST state, defeating the older `lost` guard as well as the new `misdirected` one, because every check that reached the marker did so on a store whose `kept` list was empty. Closed by `test_maintenance.py` **9d**; the mutation is now red on three named checks. |
|
||||
| **MAINT-05 round 3**, `migrate_unit_keys()` marker gate `if rep.ok:` → `if rep.ok or rep.kept:` (`+12 B`) | **found here — SAFETY-02's half, now CLOSED** | the SAME mutation one function up, on the LATCH migration rather than the history one. Measured a survivor against this branch under the purge, `:ro`, `19/19 PASS`, then closed by `test_inverter.py` part 10's new fixture — a store with `unit/inverter-a/sign_reason` already present (so `kept` is non-empty) whose `sign_tripped: true` write is dropped. Shipped: marker withheld. Mutant: marker stamped on a **disarmed latch**, and the next start skips the migration that would have re-armed it. |
|
||||
| **TARIFF-09** 11 mutations over `app/price.py` and the two-day capture, all red, no survivors | **not at risk** | direction, and method: every mutation was applied to a FRESH `cp -a` of the clean worktree rather than reverted, so there is no revert to verify (habit 18) and no shared scratch path two runs could collide on. Each ran through `tools/mutation_run.sh`, which purges and fingerprints. Byte deltas recorded on the PR; one is **+0 B** - a single price digit edited inside the committed capture, the risky delta - and the re-derived sha256 catches it. |
|
||||
| **TARIFF-01**'s other three survivors, **SAFETY-03**'s `describe()` survivor, **TARIFF-02**'s M3/M5/M7, **SAFETY-18**'s three | **still unverified** | recorded before the purge discipline, with no byte delta on the record, so a `+0 B` same-second revert cannot be excluded. All are survivors, which is the direction this defect fabricates. TARIFF-02's and SAFETY-18's were **closed by later commits whose claims are reds**, so what is unverified is the *diagnosis*, not the current state of the code. |
|
||||
|
||||
**Nothing here says a past result is wrong.** The two survivors re-run above
|
||||
|
||||
@@ -67,7 +67,7 @@ from .mqtt import (MqttPublisher, announced_units, unit_slugs,
|
||||
from . import peakguard
|
||||
from . import packshare
|
||||
from .p1 import P1Ingest, build_source, is_enabled
|
||||
from .price import Uncertainty, price_option
|
||||
from .price import Uncertainty, price_option, provider_configured
|
||||
from .price_history import PriceHistory
|
||||
from .static_tariff import resolve_tariff, static_status
|
||||
from .tariff import PeakStore, PeakTracker
|
||||
@@ -138,6 +138,60 @@ TARIFF_REFRESH_S = 1800.0
|
||||
TARIFF_MAX_AGE_S = 3 * TARIFF_REFRESH_S
|
||||
|
||||
|
||||
def tariff_refresh_wanted(opts: dict) -> bool:
|
||||
"""TARIFF-10. Does this install need the background tariff refresh at all?
|
||||
|
||||
THE DEFECT THIS PREVENTS, and it shipped: the gate below used to be
|
||||
`arbitrage.enabled(opts)` alone. That option defaults to `false`, and
|
||||
`run_tariff_refresh` is the only caller of `resolve_tariff()`, which is the
|
||||
only caller of `price_history.carry()` - so on a shipped install with a
|
||||
provider configured NOTHING was ever fetched, no plan was ever resolved,
|
||||
`/data/price_history.json` was never created, the *Price history* row read
|
||||
`empty` forever, and TARIFF-09's short-horizon warning could not fire
|
||||
because there was no fetch to warn about. Measured over a 40 h soak of two
|
||||
instances side by side: zero feed requests and `present: false` across all
|
||||
802 samples on the shipped default, against a resolved plan from the same
|
||||
feed on the instance with arbitrage armed. ⚠️ AND `DOCS.md` HAS
|
||||
DESCRIBED THE BEHAVIOUR BELOW SINCE THE COMMIT THAT WROTE THE GATE:
|
||||
OPT-04 (bdd7f18) added the refresh task, the `arbitrage.enabled()` guard on
|
||||
it, and the sentence saying a configured provider still fetches, all in one
|
||||
change. config.yaml has said it since TARIFF-02 (852fc7c).
|
||||
|
||||
Two terms, and they answer two different questions:
|
||||
|
||||
provider_configured(opts) -> "the operator asked for a price feed", so
|
||||
fetch it, validate it and record the day
|
||||
(§8.3). This is the term TARIFF-10 ADDS.
|
||||
arbitrage.enabled(opts) -> "the operator armed trading". This term is
|
||||
what the gate USED to be, kept verbatim.
|
||||
|
||||
⚠️ IT IS AN `or`, SO THE GATE IS A STRICT WIDENING. Every configuration
|
||||
that started this task before still starts it; none that did not start it
|
||||
stops anything. That matters because the second term is not redundant:
|
||||
`price_provider: off` + a `static_profile` + `arbitrage_enabled: true` is a
|
||||
configuration validate.py explicitly ACCEPTS, and it trades - measured, at
|
||||
`cycle_cost_threshold_eur: 0.05` on a 0.20/0.284 day/night profile,
|
||||
`ARBITRAGE_CHARGE -5000 W`. Dropping that term to make the gate read
|
||||
`provider_configured(opts)` alone would leave `self.tariff_plan` at None on
|
||||
that site forever and `decide()` would answer "no tariff plan has been
|
||||
resolved yet" on every cycle - silently disarming a strategy the UI says is
|
||||
running, which is the exact failure validate.py's OPT-04 rules exist to
|
||||
refuse.
|
||||
|
||||
⚠️ AND NO CONFIGURATION ACQUIRES A PERIODIC HTTPS CALL FROM THIS TICKET.
|
||||
With `price_provider` off, `resolve_tariff()` returns `static_plan()` - pure
|
||||
arithmetic, no socket - so the second term starts a task that fetches
|
||||
nothing. The true shipped default (no provider, no arbitrage) starts no
|
||||
task at all. What is fetched is decided by the FIRST term only.
|
||||
|
||||
⚠️ THIS IS NOT A TRADING SWITCH. It decides whether prices are LOOKED UP.
|
||||
Whether they move energy is `arbitrage.enabled()` at the one call site in
|
||||
`cycle()`, and TARIFF-10 did not touch it: `arbitrage_enabled: false` still
|
||||
means no trade, ever, on any configuration.
|
||||
"""
|
||||
return provider_configured(opts) or arbitrage.enabled(opts)
|
||||
|
||||
|
||||
def load_options() -> dict:
|
||||
try:
|
||||
with open(OPTIONS_PATH, "r", encoding="utf-8") as fh:
|
||||
@@ -4353,6 +4407,14 @@ class Controller:
|
||||
# OPT-04. The strategy's own sentence, not a re-derivation of it -
|
||||
# every refusal it can make already carries a reason a human can
|
||||
# act on, and "Arbitrage: no" would throw all of them away.
|
||||
#
|
||||
# ⚠️ TARIFF-10 RE-READ THIS STRING AND LEFT IT ALONE, which is worth
|
||||
# saying because that card is about the two switches being
|
||||
# different. `self.arb` is None if and only if
|
||||
# `arbitrage.enabled(self.o)` was False, so `off (arbitrage_enabled)`
|
||||
# still names the option that is actually off - and on a site with a
|
||||
# provider configured it is now the row that says the prices
|
||||
# arriving on the *Price history* row above are moving nothing.
|
||||
("Arbitrage", "off (arbitrage_enabled)" if self.arb is None
|
||||
else self.arb.explain()),
|
||||
]
|
||||
@@ -4605,11 +4667,16 @@ async def amain() -> None:
|
||||
# ESP32 heartbeat. A fetch on that path is a supplier's TLS timeout able
|
||||
# to push this add-on towards layer 1's 30 s veto. See TARIFF_REFRESH_S.
|
||||
#
|
||||
# ⚠️ STARTED ONLY WHEN ARBITRAGE IS ARMED. Nothing else in the add-on
|
||||
# reads the resolved plan - `cheap` still comes from static_status() on
|
||||
# the tick, which is pure - so on a default install this task would be
|
||||
# network traffic in aid of an attribute nobody reads.
|
||||
if arbitrage.enabled(opts):
|
||||
# ⚠️ STARTED WHEN A PROVIDER IS CONFIGURED, *OR* WHEN TRADING IS ARMED
|
||||
# - TARIFF-10, and it used to be the second term alone. `arbitrage_
|
||||
# enabled` ships `false`, so a site that had selected a price feed
|
||||
# fetched nothing at all; the whole argument, the measurement and what
|
||||
# the `or` is protecting are in `tariff_refresh_wanted()` above.
|
||||
#
|
||||
# ⚠️ IT DOES NOT ARM ANYTHING. Fetching a price and trading on one are
|
||||
# two decisions; this is the first, and `cycle()`'s
|
||||
# `if arbitrage.enabled(self.o):` is still the only gate on the second.
|
||||
if tariff_refresh_wanted(opts):
|
||||
tasks.append(asyncio.create_task(controller.run_tariff_refresh()))
|
||||
|
||||
# P1 ingestion runs as its own long-lived task. ⚠️ It must not be driven
|
||||
|
||||
@@ -140,6 +140,7 @@ from dataclasses import dataclass
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from enum import Enum
|
||||
from textwrap import dedent
|
||||
from urllib.parse import urlencode
|
||||
|
||||
_LOG = logging.getLogger("goodwe.price")
|
||||
|
||||
@@ -184,6 +185,87 @@ def brussels(instant: datetime) -> datetime:
|
||||
return instant.astimezone(brussels_offset(instant))
|
||||
|
||||
|
||||
def brussels_date_range(since: datetime, until: datetime) -> tuple:
|
||||
"""The Brussels calendar days a half-open window [since, until) touches.
|
||||
|
||||
Returns `(first, last)`, two `date`s, BOTH INCLUSIVE - which is the shape
|
||||
`api.energy-charts.info` takes: `?bzn=BE&start=2026-09-09&end=2026-09-10`
|
||||
answered 192 quarter-hours across exactly those two local days, measured
|
||||
2026-09-10T07:45:49Z and committed as
|
||||
`scenarios/energy-charts-be-price-2026-09-09_2026-09-10-twoday.json`.
|
||||
|
||||
THE DEFECT THIS EXISTS TO PREVENT (TARIFF-09). `generic_dayahead.fetch()`
|
||||
accepted `since` and `until` and requested the bare default URL, so the
|
||||
feed answered ONE Brussels day. Everything past today was then carried
|
||||
forward from yesterday's profile, flagged provisional, and §8.3 dropped all
|
||||
of it from the trade set - so the optimiser never saw tomorrow as firm, at
|
||||
any hour. Measured on the merged tree at 2026-09-10T09:00+02:00 over a 48 h
|
||||
horizon: 15 firm whole hours and 33 carried, and both ends of the only
|
||||
permitted cycle inside ONE calendar day. The ceiling was structural: at
|
||||
hour H of the local day a one-day window gives at most 24-H firm hours and
|
||||
24+H carried - 1/47 at 23:00, where the firm spread no longer even clears
|
||||
the round-trip cost. That is not what FCST-04's fill is for; it covers the
|
||||
pre-publication window until the ~13:00-14:00 CET auction, after which a
|
||||
healthy feed hands over tomorrow in full.
|
||||
|
||||
⚠️ THE CALENDAR IS BRUSSELS, NEVER UTC, AND THIS IS `_bdate()`'s LESSON ONE
|
||||
LAYER UP. `since`/`until` are INSTANTS; the API takes DATES, and the two are
|
||||
not the same question. 22:30 UTC in summer is already tomorrow in Belgium,
|
||||
so a range derived from `since.date()` asks for the wrong FIRST day for the
|
||||
last one or two hours of every single day - and the day it drops is the one
|
||||
the controller is standing in. price_history.py made this mistake at the
|
||||
cache layer and documents the same arithmetic; here it decides what is
|
||||
fetched at all, so getting it wrong loses the hours rather than misfiling
|
||||
them.
|
||||
|
||||
⚠️ `until` IS EXCLUSIVE, SO LOCAL MIDNIGHT BELONGS TO THE DAY BEFORE IT.
|
||||
A 48 h horizon from local midnight ends at local midnight two days later,
|
||||
and that instant is the first tick of a day the caller did not ask for.
|
||||
Taking `brussels(until).date()` unconditionally would request a THIRD day
|
||||
on every whole-hour horizon. This feed answers such a request without
|
||||
complaint, so nothing would report it - which is the reason to get it right
|
||||
here rather than to rely on the server's tolerance: the range is a claim
|
||||
about what the caller asked for, and it should be true.
|
||||
|
||||
⚠️ AND AN EMPTY OR INVERTED WINDOW COLLAPSES TO ONE DAY RATHER THAN TO A
|
||||
BACKWARDS RANGE. `start` after `end` is a request no feed has a defined
|
||||
answer for; some return everything, some return an error, and neither is a
|
||||
thing to hand a battery controller.
|
||||
"""
|
||||
first = brussels(since).date()
|
||||
edge = brussels(until)
|
||||
last = edge.date()
|
||||
if (edge.hour, edge.minute, edge.second, edge.microsecond) == (0, 0, 0, 0):
|
||||
last = last - timedelta(days=1)
|
||||
if last < first:
|
||||
last = first
|
||||
return first, last
|
||||
|
||||
|
||||
def dated_feed_url(base: str, **params) -> str:
|
||||
"""`base` plus a query string. Refuses a `base` that already carries one.
|
||||
|
||||
⚠️ THE REFUSAL IS THE POINT, NOT THE FORMATTING. Appending `&start=...` to
|
||||
a URL that may already hold its own query string is how a plugin produces a
|
||||
request nobody intended - `?bzn=BE?bzn=BE&start=...`, or a duplicated
|
||||
parameter whose winner is the server's choice rather than ours. So the base
|
||||
is required to be parameterless and the parameters are added exactly once,
|
||||
by `urlencode`, which is also what keeps a date out of the escaping
|
||||
question. A future edit that folds the query back into the constant is a
|
||||
refused fetch with a sentence, not a silently malformed URL.
|
||||
|
||||
⚠️ IT IS NOT REACHED BY AN OPERATOR'S `price_feed_url`. See
|
||||
`GenericDayAhead.fetch()`: an override is used exactly as given.
|
||||
"""
|
||||
if "?" in base or "#" in base:
|
||||
raise SourceUnavailable(
|
||||
f"the feed base {base!r} already carries a query string or "
|
||||
f"fragment, and this builder appends parameters. Adding a second "
|
||||
f"'?' produces a URL the server parses differently from the one "
|
||||
f"anybody intended; put the fixed parameters in the call instead")
|
||||
return f"{base}?{urlencode(params)}" if params else base
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Absence, borrowed wholesale from sim/riglock.py (TEST-04).
|
||||
# --------------------------------------------------------------------------- #
|
||||
@@ -684,6 +766,47 @@ def price_option(opts: dict, key: str):
|
||||
return v
|
||||
|
||||
|
||||
def provider_id(opts: dict) -> str:
|
||||
"""The selected plugin id, normalised. `"off"` when none is selected.
|
||||
|
||||
⚠️ ONE SPELLING OF THIS EXPRESSION, AND TARIFF-10 IS WHY IT IS A FUNCTION.
|
||||
`str(price_option(opts, "price_provider") or "off").strip()` was written out
|
||||
twice - in `load_prices()` below and in `static_tariff.resolve_tariff()` -
|
||||
and main.py's startup gate would have been a third. "Is a provider
|
||||
configured" is now a DECISION the add-on takes (it is what starts the
|
||||
background refresh), and a decision spelled out at three call sites is three
|
||||
places to disagree about whether `" off "` or `None` counts.
|
||||
⚠️ `validate.py` deliberately keeps its own reader: it validates the RAW
|
||||
options blob, including one the Supervisor never saw, so it must not route
|
||||
through `price_option()`'s defaults. Note what this does NOT do: it does not
|
||||
decide
|
||||
whether the id names a shipped plugin - `load_prices()` and validate.py's
|
||||
rule 7 own that, and answering "unknown" here would turn a typo into
|
||||
"no provider" instead of into a startup refusal.
|
||||
"""
|
||||
return str(price_option(opts, "price_provider") or "off").strip()
|
||||
|
||||
|
||||
def provider_configured(opts: dict) -> bool:
|
||||
"""Has the operator selected a price feed at all? (TARIFF-10)
|
||||
|
||||
THE DEFECT THIS PREVENTS: the background tariff refresh used to start only
|
||||
when `arbitrage_enabled` was on, which ships `false` - so a site with a
|
||||
provider configured fetched NOTHING, ever. `/data/price_history.json` was
|
||||
never created, the *Price history* status row read `empty` forever, and
|
||||
TARIFF-09's short-horizon warning could not fire because no fetch happened
|
||||
to warn about. Measured over a 40 h two-instance soak: zero feed requests
|
||||
and `present: false` across all 802 samples on the shipped default, against
|
||||
a resolved plan on the same feed with arbitrage armed.
|
||||
|
||||
⚠️ THIS IS NOT A TRADING SWITCH AND MUST NEVER BECOME ONE. It answers
|
||||
"fetch prices?"; `arbitrage.enabled()` answers "move energy on them?", and
|
||||
§10.1's priority 3 is gated on the second alone. See
|
||||
`main.tariff_refresh_wanted()` for the one place the two meet.
|
||||
"""
|
||||
return provider_id(opts) not in ("", "off")
|
||||
|
||||
|
||||
def export_price_eur_kwh(opts: dict) -> float:
|
||||
"""AC4 / §9.3: the injection price is a SEPARATE SCALAR, not a second series.
|
||||
|
||||
@@ -731,6 +854,24 @@ class PriceProvider(ABC):
|
||||
`transport(url, timeout_s) -> bytes` is injected so the checks never
|
||||
touch a network and `manual_csv` never needs one. `opts` is a
|
||||
`PluginOptions`, so the markup and taxes are not reachable from here.
|
||||
|
||||
⚠️ `since`/`until` ARE THE RANGE THE CALLER WANTS, AND A PLUGIN THAT
|
||||
IGNORES THEM GETS WHATEVER ITS FEED DEFAULTS TO (TARIFF-09). That is
|
||||
not a small difference: `generic_dayahead` ignored both, the public
|
||||
EPEX feed answered ONE Brussels day, and §8.3 then dropped every hour
|
||||
past today from the trade set as provisional - so the optimiser never
|
||||
saw tomorrow as firm, at any hour of any day, and nothing reported it.
|
||||
|
||||
This promise is NOT mechanically enforced, and the honest reason is
|
||||
TARIFF-08's: three of the five shipped plugins point at endpoints
|
||||
nobody has verified (TARIFF-07 is the open decision on withdrawing
|
||||
them), so nobody knows how those feeds spell a date range. A rule
|
||||
obliging them to pass one would be satisfied by an INVENTED parameter
|
||||
name - the parser-graded-against-its-own-guess defect, moved from the
|
||||
response to the request. What exists instead is a roster pin in
|
||||
test_price.py: which plugins consume the range and which do not is
|
||||
asserted as a literal, so a sixth supplier, or a change to one of the
|
||||
three, forces the decision rather than inheriting the silence.
|
||||
"""
|
||||
|
||||
|
||||
@@ -1171,9 +1312,63 @@ class GenericDayAhead(PriceProvider):
|
||||
sources=frozenset({Component.ENERGY}),
|
||||
egress_hosts=("api.energy-charts.info", "web-api.tp.entsoe.eu"))
|
||||
|
||||
FEED_BASE = "https://api.energy-charts.info/price"
|
||||
BZN = "BE"
|
||||
|
||||
async def fetch(self, opts, since, until, transport):
|
||||
url = opts.get("price_feed_url") or \
|
||||
"https://api.energy-charts.info/price?bzn=BE"
|
||||
"""Request the range the CALLER asked for, on the Brussels calendar.
|
||||
|
||||
⚠️ THE ARGUMENTS ARE USED. Until TARIFF-09 this method took `since` and
|
||||
`until` and asked for the bare `?bzn=BE`, which answers ONE Brussels
|
||||
day - so a 48 h horizon was 15 firm hours and 33 carried at 09:00, and
|
||||
1 firm and 47 carried at 23:00, with the whole trade set inside a
|
||||
single calendar day at every hour of every day. See
|
||||
`brussels_date_range()` for the measurement and for why the calendar
|
||||
is Brussels rather than UTC.
|
||||
|
||||
⚠️ AN OVERRIDE IS THE EXACT URL AND THIS METHOD APPENDS NOTHING TO IT,
|
||||
WHICH COSTS THE OVERRIDING SITE THE RANGE. That is a decision, not an
|
||||
oversight, and the sentence below is why it is not a footnote:
|
||||
|
||||
* `price_feed_url` says "override the plugin's default feed", and an
|
||||
override that is silently rewritten is not one. Merging parameters
|
||||
into an operator's query string means parsing and re-serialising a
|
||||
URL they wrote - which reorders, de-duplicates and re-escapes it,
|
||||
and can hand the server a request nobody typed.
|
||||
* `start`/`end` are ENERGY-CHARTS' spelling, and this provider
|
||||
DECLARES A SECOND HOST: `web-api.tp.entsoe.eu`. Nothing here has
|
||||
verified that host's day-ahead parameters and nothing here should
|
||||
guess them - but they are not `start`/`end`, so an override to the
|
||||
other DECLARED host is exactly the case where appending our
|
||||
vocabulary adds two parameters that host ignores and yields the
|
||||
short horizon anyway, with nothing saying so.
|
||||
* and blind appending is refused outright: `dated_feed_url()` will
|
||||
not add a query string to a base that has one.
|
||||
|
||||
So the cost is real and it is REPORTED, at fetch time, on the shipped
|
||||
logger, naming the horizon consequence rather than the substitution.
|
||||
§11.4 is what makes a price defect quiet; an operator who has taken the
|
||||
short horizon has to be able to find out from the log that they did.
|
||||
"""
|
||||
override = str(opts.get("price_feed_url") or "").strip()
|
||||
if override:
|
||||
url = override
|
||||
_LOG.warning(
|
||||
"price_feed_url is set, so %s is requesting %s exactly as "
|
||||
"given and is adding NO date range to it. The public feed "
|
||||
"answers a single Brussels day to a request that names no "
|
||||
"range, and every hour past that day is then carried forward "
|
||||
"from yesterday's profile and flagged provisional - §8.3 "
|
||||
"drops provisional hours from the trade set, so arbitrage "
|
||||
"would never see tomorrow as firm. Clear price_feed_url to "
|
||||
"let this plugin ask for the %s..%s it was given.",
|
||||
self.id, url, *(d.isoformat()
|
||||
for d in brussels_date_range(since, until)))
|
||||
else:
|
||||
first, last = brussels_date_range(since, until)
|
||||
url = dated_feed_url(self.FEED_BASE, bzn=self.BZN,
|
||||
start=first.isoformat(),
|
||||
end=last.isoformat())
|
||||
check_egress(url, self.capabilities())
|
||||
body = await transport(url, float(opts.get("price_timeout_s")))
|
||||
return _points_from_columnar_json(body, expect_unit="EUR / MWh",
|
||||
@@ -1362,8 +1557,8 @@ async def load_prices(opts: dict, since: datetime, until: datetime,
|
||||
site condition, it is a plugin trying to read the markup, and turning it
|
||||
into "no price data" would hide the exact defect trap 1 exists to catch.
|
||||
"""
|
||||
pid = str(price_option(opts, "price_provider") or "off").strip()
|
||||
if pid in ("", "off"):
|
||||
pid = provider_id(opts)
|
||||
if not provider_configured(opts):
|
||||
return PriceRead.no_data(
|
||||
"no price provider configured (price_provider is 'off'); "
|
||||
"§11.4: static profile, plan provisional, no arbitrage")
|
||||
|
||||
@@ -207,7 +207,7 @@ from datetime import datetime, timedelta, timezone
|
||||
from .price import (
|
||||
MAX_PLAUSIBLE_EUR_KWH, Absence, PricePoint, PriceProduct, PriceRead,
|
||||
Uncertainty, _Outcome, brussels, load_prices, price_option,
|
||||
required_spread, uncertainty_of, widen_factor,
|
||||
provider_configured, required_spread, uncertainty_of, widen_factor,
|
||||
)
|
||||
|
||||
_LOG = logging.getLogger("goodwe.tariff.static")
|
||||
@@ -1071,8 +1071,7 @@ async def resolve_tariff(opts: dict, since: datetime, until: datetime,
|
||||
missing and loses the elapsed ones it should never have had.
|
||||
"""
|
||||
stamp = now or datetime.now(timezone.utc)
|
||||
pid = str(price_option(opts, "price_provider") or "off").strip()
|
||||
if pid in ("", "off"):
|
||||
if not provider_configured(opts):
|
||||
return static_plan(opts, since, until, provisional=False, now=stamp)
|
||||
|
||||
read = await load_prices(opts, since, until, transport)
|
||||
|
||||
@@ -265,9 +265,17 @@ options:
|
||||
# TARIFF - what energy costs, and whether to trade on it
|
||||
# ==========================================================================
|
||||
# `off` is the shipped default and means the §11.4 no-price-data row applies:
|
||||
# static profile, plan provisional, NO ARBITRAGE. Nothing in 0.5.x consumes
|
||||
# the series yet - OPT-04 is the consumer - so turning this on today fetches,
|
||||
# validates and publishes prices and changes no energy decision.
|
||||
# static profile, plan provisional, NO ARBITRAGE.
|
||||
#
|
||||
# ⚠️ SELECTING A PROVIDER IS WHAT STARTS THE FETCH, AND IT IS NOT
|
||||
# `arbitrage_enabled` (TARIFF-10). With a provider set and trading off - the
|
||||
# shipped default - the add-on fetches the feed every 30 minutes, validates
|
||||
# it, records the day in /data/price_history.json and reports what it holds
|
||||
# on the *Price history* status row. It changes NO energy decision: the cheap
|
||||
# window the maintenance cycle reads still comes from `static_profile`, and
|
||||
# §10.1's arbitrage is gated on `arbitrage_enabled` alone. Until TARIFF-10
|
||||
# the fetch was gated on `arbitrage_enabled` too, so this option did nothing
|
||||
# whatsoever on a shipped install - the store was never even created.
|
||||
#
|
||||
# ⚠️ THESE DEFAULTS ARE MIRRORED ONCE, IN app/price.py's
|
||||
# PRICE_OPTION_DEFAULTS, and test_price.py reads THIS FILE and asserts the two
|
||||
@@ -318,6 +326,14 @@ options:
|
||||
# validate.py warns about exactly that.
|
||||
static_profile: ""
|
||||
price_csv_path: "/share/goodwe_prices.csv"
|
||||
# ⚠️ AN OVERRIDE IS THE EXACT URL (TARIFF-09). The plugin sends it verbatim
|
||||
# and appends nothing - including the `start`/`end` date range it derives from
|
||||
# the horizon for its own feed - so an override gets whatever range that URL
|
||||
# answers, which for a bare day-ahead endpoint is ONE Brussels day. Every hour
|
||||
# past it is then carried from yesterday and flagged provisional, and §8.3
|
||||
# keeps provisional hours out of the trade set. The plugin logs a WARNING
|
||||
# naming that consequence on every fetch that uses an override; clearing this
|
||||
# option is what restores the full horizon.
|
||||
price_feed_url: ""
|
||||
price_horizon_h: 48
|
||||
price_timeout_s: 10
|
||||
@@ -344,6 +360,12 @@ options:
|
||||
# configuration: this is the first strategy in the add-on that moves energy
|
||||
# against the meter on purpose, so it is armed deliberately or not at all.
|
||||
#
|
||||
# ⚠️ IT IS NOT WHAT TURNS THE PRICE FEED ON (TARIFF-10). `price_provider`
|
||||
# above decides whether prices are FETCHED; this decides whether they move
|
||||
# energy. Leaving this off on a site with a provider configured is a
|
||||
# supported, documented state: prices are fetched, validated and recorded,
|
||||
# and nothing trades.
|
||||
#
|
||||
# ⚠️ IT NEEDS A PRICE SOURCE AND A P1 METER, and validate.py refuses the
|
||||
# combinations that cannot work rather than letting them look armed. With
|
||||
# `price_provider: off` and no `static_profile` there is no series to trade
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
{"license_info":"CC BY 4.0 (creativecommons.org/licenses/by/4.0) from Bundesnetzagentur | SMARD.de","unix_seconds":[1788904800,1788905700,1788906600,1788907500,1788908400,1788909300,1788910200,1788911100,1788912000,1788912900,1788913800,1788914700,1788915600,1788916500,1788917400,1788918300,1788919200,1788920100,1788921000,1788921900,1788922800,1788923700,1788924600,1788925500,1788926400,1788927300,1788928200,1788929100,1788930000,1788930900,1788931800,1788932700,1788933600,1788934500,1788935400,1788936300,1788937200,1788938100,1788939000,1788939900,1788940800,1788941700,1788942600,1788943500,1788944400,1788945300,1788946200,1788947100,1788948000,1788948900,1788949800,1788950700,1788951600,1788952500,1788953400,1788954300,1788955200,1788956100,1788957000,1788957900,1788958800,1788959700,1788960600,1788961500,1788962400,1788963300,1788964200,1788965100,1788966000,1788966900,1788967800,1788968700,1788969600,1788970500,1788971400,1788972300,1788973200,1788974100,1788975000,1788975900,1788976800,1788977700,1788978600,1788979500,1788980400,1788981300,1788982200,1788983100,1788984000,1788984900,1788985800,1788986700,1788987600,1788988500,1788989400,1788990300,1788991200,1788992100,1788993000,1788993900,1788994800,1788995700,1788996600,1788997500,1788998400,1788999300,1789000200,1789001100,1789002000,1789002900,1789003800,1789004700,1789005600,1789006500,1789007400,1789008300,1789009200,1789010100,1789011000,1789011900,1789012800,1789013700,1789014600,1789015500,1789016400,1789017300,1789018200,1789019100,1789020000,1789020900,1789021800,1789022700,1789023600,1789024500,1789025400,1789026300,1789027200,1789028100,1789029000,1789029900,1789030800,1789031700,1789032600,1789033500,1789034400,1789035300,1789036200,1789037100,1789038000,1789038900,1789039800,1789040700,1789041600,1789042500,1789043400,1789044300,1789045200,1789046100,1789047000,1789047900,1789048800,1789049700,1789050600,1789051500,1789052400,1789053300,1789054200,1789055100,1789056000,1789056900,1789057800,1789058700,1789059600,1789060500,1789061400,1789062300,1789063200,1789064100,1789065000,1789065900,1789066800,1789067700,1789068600,1789069500,1789070400,1789071300,1789072200,1789073100,1789074000,1789074900,1789075800,1789076700],"price":[150.93,151.41,150.99,147.59,153.25,147.26,144.04,141.53,147.79,146.53,143.55,139.93,145.58,139.56,139.11,138.06,138.92,138.52,138.04,138.65,137.29,139.03,144.59,150.34,154.12,174.5,178.43,186.62,199.35,200.21,200.11,197.43,220.07,202.0,196.01,174.03,198.32,167.49,152.54,142.05,161.5,145.45,128.61,100.85,139.18,115.83,94.41,80.39,111.25,89.61,84.99,74.38,80.53,70.58,62.27,57.1,62.24,64.95,68.99,71.19,66.9,87.32,109.0,123.12,68.48,125.53,155.99,161.9,138.77,160.57,167.35,173.0,142.2,172.83,196.19,211.26,205.85,218.77,217.08,222.0,210.98,208.46,208.88,210.02,209.77,202.61,200.27,191.49,194.39,195.04,197.26,190.92,188.39,198.05,203.05,191.84,216.08,216.08,214.92,200.0,223.37,201.43,196.32,190.0,199.53,194.83,180.53,172.23,179.88,175.77,167.1,173.89,163.29,165.99,171.41,178.73,170.27,177.37,187.71,201.07,193.62,207.61,218.27,226.77,227.17,234.05,236.99,229.94,248.93,227.5,216.08,195.67,215.99,195.01,177.1,155.09,174.91,152.71,139.59,123.83,140.77,123.19,114.96,94.5,102.77,96.63,94.57,116.34,114.63,126.82,129.89,123.95,105.0,119.11,121.0,136.79,123.41,135.01,135.0,156.68,146.88,152.16,162.02,181.78,154.76,184.28,194.3,215.76,197.23,234.15,253.03,296.58,345.32,389.16,482.99,559.0,551.11,446.41,358.97,295.9,271.39,268.06,241.43,238.01,239.7,226.41,223.05,211.56,218.21,210.61,197.78,171.79],"unit":"EUR / MWh","deprecated":false}
|
||||
@@ -1,16 +1,16 @@
|
||||
{
|
||||
"card": "TARIFF-08",
|
||||
"card": "TARIFF-08, extended by TARIFF-09",
|
||||
"what": "UNMODIFIED response bodies from the public EPEX BE day-ahead feed, captured so the SHIPPED parser can be replayed against what the source really publishes",
|
||||
"why": "every price fixture in the suite was built from a `{\"data\": [{\"start\": ..., \"price_eur_mwh\": ...}]}` document that this feed has never published. The suite therefore graded `_points_from_json` against its own assumption - a closed loop that could not notice the assumption was wrong - and §11.4 turned the resulting failure into a silent fall back to the static profile. A captured real body is what breaks that loop, so these files are bytes, not a reconstruction: nothing here is pretty-printed, re-keyed or trimmed.",
|
||||
"why": "every price fixture in the suite was built from a `{\"data\": [{\"start\": ..., \"price_eur_mwh\": ...}]}` document that this feed has never published. The suite therefore graded `_points_from_json` against its own assumption - a closed loop that could not notice the assumption was wrong - and §11.4 turned the resulting failure into a silent fall back to the static profile. A captured real body is what breaks that loop, so these files are bytes, not a reconstruction: nothing here is pretty-printed, re-keyed or trimmed. TARIFF-09 adds a body from a request that names a DATE RANGE, for the same reason one layer out: the shipped plugin took `since`/`until` and asked for the bare default URL, so the feed answered ONE Brussels day and every hour past today was carried forward and dropped from the trade set as provisional. A two-day body captured from the range request is what makes \"asking for the range gets the range\" checkable against something the plugin did not shape.",
|
||||
"source": "Energy-Charts API (Fraunhofer ISE), api.energy-charts.info",
|
||||
"upstream": "Bundesnetzagentur | SMARD.de",
|
||||
"licence": "CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)",
|
||||
"licence_note": "the BE bidding zone is in the API's CC BY 4.0 list, so redistribution is permitted with attribution. DATA-02 established this against the API's own OpenAPI document before committing `epex-be-dayahead-2025-09-01_2026-09-01.json`; both bodies below carry the same `license_info` string, recorded per file and asserted by the replay test.",
|
||||
"attribution": "Data licensed CC BY 4.0 by Bundesnetzagentur | SMARD.de, retrieved via the Energy-Charts API (Fraunhofer ISE), https://energy-charts.info. Published without changes.",
|
||||
"retrieved_utc": "2026-09-10T06:35:00+00:00",
|
||||
"retrieved_utc": "2026-09-10T06:35:00+00:00 (TARIFF-08's two), 2026-09-10T07:45:49+00:00 (TARIFF-09's)",
|
||||
"retrieved_with": "curl, by hand; no code in `app/` fetches these",
|
||||
"shape": "COLUMNAR: parallel `unix_seconds` (UTC epoch seconds) and `price` arrays, plus `unit`, `license_info` and `deprecated`. There is no `data` key and no per-point object.",
|
||||
"replayed_by": "test_price.py section 2b",
|
||||
"replayed_by": "test_price.py sections 2b (TARIFF-08) and 2c (TARIFF-09)",
|
||||
"files": {
|
||||
"energy-charts-be-price-2026-09-10-quarterhourly.json": {
|
||||
"url": "https://api.energy-charts.info/price?bzn=BE",
|
||||
@@ -29,7 +29,12 @@
|
||||
"price",
|
||||
"unit",
|
||||
"unix_seconds"
|
||||
]
|
||||
],
|
||||
"also_returned_verbatim_by": {
|
||||
"url": "https://api.energy-charts.info/price?bzn=BE&start=2026-09-10&end=2026-09-11",
|
||||
"retrieved_utc": "2026-09-10T07:46:55+00:00",
|
||||
"note": "TARIFF-09's PRE-PUBLICATION case, and it needed no second file because the answer was BYTE-IDENTICAL to the bare-URL capture above - the same 1877 bytes and the same sha256. Asked for two days at 09:46 CEST, before the ~13:00-14:00 CET auction publishes 2026-09-11, the feed returned the ONE day that exists. That is the short answer this card requires to stay a `series` of what exists rather than becoming `no_data`, and the replay re-derives the identity from the bytes rather than restating it."
|
||||
}
|
||||
},
|
||||
"energy-charts-be-price-2025-09-10-hourly.json": {
|
||||
"url": "https://api.energy-charts.info/price?bzn=BE&start=2025-09-10&end=2025-09-10",
|
||||
@@ -49,6 +54,29 @@
|
||||
"unit",
|
||||
"unix_seconds"
|
||||
]
|
||||
},
|
||||
"energy-charts-be-price-2026-09-09_2026-09-10-twoday.json": {
|
||||
"url": "https://api.energy-charts.info/price?bzn=BE&start=2026-09-09&end=2026-09-10",
|
||||
"retrieved_utc": "2026-09-10T07:45:49+00:00",
|
||||
"note": "TARIFF-09. A RANGE request, and the counterfactual the ticket is built on: `start`/`end` are INCLUSIVE Brussels calendar dates and this one answers 192 quarter-hours across TWO local days, where the bare `?bzn=BE` of the same endpoint answers 96 across one. The shipped `generic_dayahead` now derives these two dates from the `since`/`until` it is handed.",
|
||||
"covers_local_days": [
|
||||
"2026-09-09",
|
||||
"2026-09-10"
|
||||
],
|
||||
"granularity_s": 900,
|
||||
"intervals": 192,
|
||||
"bytes": 3579,
|
||||
"sha256": "4d83f5efb4294a4a682562eac619bf35e45c95edde957bfa94c402e1777cc9b3",
|
||||
"unit_as_returned": "EUR / MWh",
|
||||
"licence_as_returned": "CC BY 4.0 (creativecommons.org/licenses/by/4.0) from Bundesnetzagentur | SMARD.de",
|
||||
"deprecated_as_returned": false,
|
||||
"top_level_keys": [
|
||||
"deprecated",
|
||||
"license_info",
|
||||
"price",
|
||||
"unit",
|
||||
"unix_seconds"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -51,20 +51,28 @@ import io
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
from datetime import datetime, timedelta, timezone
|
||||
import tempfile
|
||||
from datetime import date, datetime, timedelta, timezone
|
||||
from textwrap import dedent
|
||||
|
||||
from app.price import (
|
||||
ALL_IN, Absence, BANNED_IN_PLUGIN_CODE, Component, FrameworkOption,
|
||||
MAX_PLAUSIBLE_EUR_KWH, code_leaks,
|
||||
PRICE_OPTION_DEFAULTS, PROVIDERS, PluginOptions, PricePoint, PriceProduct,
|
||||
PriceProvider, PriceRead, ProviderCapabilities, RawPoint, SourceUnavailable,
|
||||
arbitrage_allowed, assemble, brussels, brussels_offset, check_egress,
|
||||
arbitrage_allowed, assemble, brussels, brussels_date_range,
|
||||
brussels_offset, check_egress, dated_feed_url,
|
||||
price_option,
|
||||
export_price_eur_kwh, load_prices, plan_is_provisional, register,
|
||||
volatility_regime,
|
||||
)
|
||||
from app import price as _price
|
||||
from app import arbitrage as _arb
|
||||
from app import main as _main
|
||||
from app.p1 import make_sample, split_signed
|
||||
from app.static_tariff import TariffPlan
|
||||
from app.validate import validate_startup_config
|
||||
|
||||
fails = []
|
||||
@@ -500,14 +508,14 @@ _REAL = {n: _capture(n) for n in _CAPTURES["files"]}
|
||||
_bad_sha = sorted(n for n, b in _REAL.items()
|
||||
if hashlib.sha256(b).hexdigest()
|
||||
!= _CAPTURES["files"][n]["sha256"])
|
||||
check(f"both captured bodies are the bytes the provenance file recorded "
|
||||
f"{sorted(_REAL)}",
|
||||
len(_REAL) == 2 and _bad_sha == [])
|
||||
check(f"all three captured bodies are the bytes the provenance file "
|
||||
f"recorded {sorted(_REAL)}",
|
||||
len(_REAL) == 3 and _bad_sha == [])
|
||||
|
||||
# The document really is columnar, derived from the bytes rather than asserted:
|
||||
# if it ever grows a `data` key this check says so before any parser does.
|
||||
_shapes = {n: sorted(json.loads(b.decode("utf-8"))) for n, b in _REAL.items()}
|
||||
check(f"...and both are COLUMNAR - parallel arrays, no 'data' key "
|
||||
check(f"...and every one is COLUMNAR - parallel arrays, no 'data' key "
|
||||
f"{sorted({tuple(v) for v in _shapes.values()})}",
|
||||
all(v == ["deprecated", "license_info", "price", "unit", "unix_seconds"]
|
||||
for v in _shapes.values()))
|
||||
@@ -707,6 +715,275 @@ check(f"the networked plugins are {_networked}; {_replayed} is replayed from a "
|
||||
"totalenergies_dynamic", "eneco_eddy"]))
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
print()
|
||||
print("2c. TARIFF-09 - the RANGE the caller asked for, on the Brussels "
|
||||
"calendar")
|
||||
#
|
||||
# ⚠️ WHAT WAS WRONG, AND WHY NO CHECK IN THIS FILE COULD SEE IT.
|
||||
# `generic_dayahead.fetch()` accepted `since` and `until` and requested the
|
||||
# bare `https://api.energy-charts.info/price?bzn=BE`, which answers ONE
|
||||
# Brussels day. Everything past today was then filled from yesterday's profile
|
||||
# by FCST-04, flagged provisional, and §8.3 dropped all of it from the trade
|
||||
# set - so the optimiser never saw tomorrow as firm, at any hour of any day.
|
||||
# Measured on the merged tree at 2026-09-10T09:00+02:00 over a 48 h horizon:
|
||||
# 15 firm whole hours and 33 carried, both ends of the only permitted cycle
|
||||
# inside ONE calendar day; and at 23:00, 1 firm hour and 47 carried, where the
|
||||
# firm spread no longer clears the round-trip cost at all.
|
||||
#
|
||||
# The reason section 2b could not catch it is that section 2b fed the parser
|
||||
# BYTES and never looked at the REQUEST. `canned()` has always recorded the URL
|
||||
# it was asked for and nothing asserted on it - so the transport knew the
|
||||
# plugin was ignoring its arguments and no check ever asked. The checks below
|
||||
# assert on the URL, and the two-day body they replay is a capture of what that
|
||||
# URL really answers.
|
||||
|
||||
_TD = "energy-charts-be-price-2026-09-09_2026-09-10-twoday.json"
|
||||
|
||||
# ⚠️ NON-DEGENERACY FIRST (habit 1): the two-day capture must really cover TWO
|
||||
# Brussels days and the one-day capture ONE, or "the range reached the feed"
|
||||
# is graded against two fixtures that are the same shape.
|
||||
_bdays = {n: sorted({brussels(datetime.fromtimestamp(t, UTC)).date().isoformat()
|
||||
for t in json.loads(b.decode("utf-8"))["unix_seconds"]})
|
||||
for n, b in _REAL.items()}
|
||||
check(f"the captures span the local-day counts this section needs - "
|
||||
f"{ {n: len(v) for n, v in _bdays.items()} } - or the range check has "
|
||||
f"nothing to distinguish {_bdays[_TD]} from {_bdays[_QH]}",
|
||||
len(_bdays[_TD]) == 2 and len(_bdays[_QH]) == 1
|
||||
and len(_bdays[_H1]) == 1)
|
||||
_npts = {n: len(json.loads(b.decode("utf-8"))["price"])
|
||||
for n, b in _REAL.items()}
|
||||
check(f"...and the two-day body carries 192 quarter-hours where the one-day "
|
||||
f"body carries 96, which is the counterfactual this card is built on "
|
||||
f"{_npts}",
|
||||
len(json.loads(_REAL[_TD].decode("utf-8"))["price"]) == 192
|
||||
and len(json.loads(_REAL[_QH].decode("utf-8"))["price"]) == 96)
|
||||
|
||||
# --- the URL the SHIPPED plugin asks for ----------------------------------- #
|
||||
|
||||
|
||||
def asked_url(since, until, extra=None):
|
||||
"""The URL `generic_dayahead` really requests for a window. Behaviour,
|
||||
not source text: it comes back off the injected transport."""
|
||||
t = canned(_REAL[_TD])
|
||||
o = opts(price_provider="generic_dayahead", **(extra or {}))
|
||||
run(load_prices(o, since, until, transport=t))
|
||||
return t.calls[0][0]
|
||||
|
||||
|
||||
# The expectation is built HERE, from the window, by different arithmetic than
|
||||
# price.py uses: a literal string with the two dates spelled out.
|
||||
_w0 = datetime.fromisoformat("2026-09-09T09:17:43+02:00")
|
||||
check(f"a 48 h horizon from 09:17 local asks the feed for the three Brussels "
|
||||
f"days it touches -> {asked_url(_w0, _w0 + timedelta(hours=48))!r}",
|
||||
asked_url(_w0, _w0 + timedelta(hours=48))
|
||||
== "https://api.energy-charts.info/price?bzn=BE"
|
||||
"&start=2026-09-09&end=2026-09-11")
|
||||
check("...and the shipped default no longer asks the bare URL that answers a "
|
||||
"single day",
|
||||
asked_url(_w0, _w0 + timedelta(hours=48))
|
||||
!= "https://api.energy-charts.info/price?bzn=BE")
|
||||
|
||||
# ⚠️ THE BRUSSELS/UTC BOUNDARY, WHICH IS THE TRAP. `since`/`until` are
|
||||
# INSTANTS and the API takes DATES. At 22:30 UTC in September Belgium is
|
||||
# already on the NEXT calendar day, so a range derived from `since.date()` -
|
||||
# or from `since.astimezone(utc).date()`, which is the same thing - asks for
|
||||
# the day the controller has just left and drops the day it is standing in.
|
||||
# This is `_bdate()`'s lesson one layer up: price_history.py MISFILES an hour
|
||||
# when it gets this wrong; here the hours are never fetched at all.
|
||||
_edge = datetime.fromisoformat("2026-09-10T22:30:00+00:00")
|
||||
check(f"the precondition: {_edge.isoformat()} really is a different day in "
|
||||
f"Brussels ({brussels(_edge).date().isoformat()}) than in UTC "
|
||||
f"({_edge.date().isoformat()}), or the boundary check grades nothing",
|
||||
brussels(_edge).date().isoformat() != _edge.date().isoformat())
|
||||
check(f"...so the range starts on the BRUSSELS day, 2026-09-11, and not on "
|
||||
f"the UTC day 2026-09-10 -> "
|
||||
f"{asked_url(_edge, _edge + timedelta(hours=12))!r}",
|
||||
asked_url(_edge, _edge + timedelta(hours=12))
|
||||
== "https://api.energy-charts.info/price?bzn=BE"
|
||||
"&start=2026-09-11&end=2026-09-11")
|
||||
|
||||
# ...and the SAME trap on the other end. `until` at 22:30 UTC is already
|
||||
# 00:30 of the next Brussels day, so a UTC-derived END stops one day short and
|
||||
# the last hours of the horizon are never fetched. Both ends of the range
|
||||
# derivation are graded, or a mutation only has to get one of them right.
|
||||
_edge2_a = datetime.fromisoformat("2026-09-09T12:00:00+00:00")
|
||||
_edge2_b = datetime.fromisoformat("2026-09-10T22:30:00+00:00")
|
||||
check(f"the precondition: {_edge2_b.isoformat()} is "
|
||||
f"{brussels(_edge2_b).date().isoformat()} in Brussels and "
|
||||
f"{_edge2_b.date().isoformat()} in UTC, so the two answers differ",
|
||||
brussels(_edge2_b).date().isoformat() != _edge2_b.date().isoformat())
|
||||
check(f"...so the range ENDS on the Brussels day 2026-09-11, not on the UTC "
|
||||
f"day 2026-09-10 -> {asked_url(_edge2_a, _edge2_b)!r}",
|
||||
asked_url(_edge2_a, _edge2_b)
|
||||
== "https://api.energy-charts.info/price?bzn=BE"
|
||||
"&start=2026-09-09&end=2026-09-11")
|
||||
|
||||
# ⚠️ `until` IS EXCLUSIVE, so local midnight belongs to the day BEFORE it. A
|
||||
# 24 h horizon from local midnight is ONE day, not two.
|
||||
_mid = datetime.fromisoformat("2026-09-09T00:00:00+02:00")
|
||||
check(f"a horizon ending exactly at local midnight asks for one day, because "
|
||||
f"`until` is exclusive -> {asked_url(_mid, _mid + timedelta(hours=24))!r}",
|
||||
asked_url(_mid, _mid + timedelta(hours=24))
|
||||
== "https://api.energy-charts.info/price?bzn=BE"
|
||||
"&start=2026-09-09&end=2026-09-09")
|
||||
check(f"...while one second past midnight is two days, so the rule is the "
|
||||
f"exclusive END and not a fixed subtraction -> "
|
||||
f"{asked_url(_mid, _mid + timedelta(hours=24, seconds=1))!r}",
|
||||
asked_url(_mid, _mid + timedelta(hours=24, seconds=1))
|
||||
== "https://api.energy-charts.info/price?bzn=BE"
|
||||
"&start=2026-09-09&end=2026-09-10")
|
||||
check(f"an empty or inverted window collapses to a single day rather than "
|
||||
f"asking for a backwards range no feed has a defined answer for -> "
|
||||
f"{asked_url(_mid, _mid - timedelta(hours=5))!r}",
|
||||
asked_url(_mid, _mid - timedelta(hours=5))
|
||||
== "https://api.energy-charts.info/price?bzn=BE"
|
||||
"&start=2026-09-09&end=2026-09-09")
|
||||
|
||||
# The helper on its own, over the DST transitions, where a fixed-offset
|
||||
# assumption and a correct one disagree.
|
||||
_mar = datetime.fromisoformat("2026-03-29T00:30:00+00:00") # 02:30 CEST
|
||||
_bdr = derived(lambda: brussels_date_range(_mar, _mar + timedelta(hours=1)))
|
||||
check(f"brussels_date_range() is right across the spring transition, where "
|
||||
f"00:30 UTC is already 02:30 local -> {_bdr}",
|
||||
_bdr == (date(2026, 3, 29), date(2026, 3, 29)))
|
||||
|
||||
# --- the two-day body, end to end through the SHIPPED parser --------------- #
|
||||
|
||||
_two = load("generic_dayahead", transport=canned(_REAL[_TD]))
|
||||
check(f"the shipped generic_dayahead parses the real two-day body "
|
||||
f"{_two.words}", _two.kind == "series")
|
||||
def _series_days(read):
|
||||
return derived(lambda: sorted({brussels(p.start).date().isoformat()
|
||||
for p in read.points}))
|
||||
|
||||
|
||||
_two_days = _series_days(_two)
|
||||
check(f"...and the series it produces spans TWO Brussels local days "
|
||||
f"{_two_days}, where the one-day body's spans one "
|
||||
f"{_series_days(_REPLAY[_QH])}",
|
||||
_two_days == ["2026-09-09", "2026-09-10"]
|
||||
and len(_two_days) == 2)
|
||||
check(f"...one point per published interval, 192 of them, no decimation and "
|
||||
f"no day dropped ({derived(lambda: len(_two.points))})",
|
||||
derived(lambda: len(_two.points)) == 192)
|
||||
|
||||
# --- the SHORT ANSWER, which is normal and must stay a series -------------- #
|
||||
#
|
||||
# ⚠️ ASKING FOR TOMORROW BEFORE THE ~13:00-14:00 CET AUCTION RETURNS FEWER DAYS
|
||||
# THAN REQUESTED, AND THAT IS NOT AN ERROR. Measured: at 2026-09-10T07:46:55Z
|
||||
# the request `?bzn=BE&start=2026-09-10&end=2026-09-11` came back BYTE-IDENTICAL
|
||||
# to the bare-URL capture - one day, 96 quarter-hours. A `no_data` there would
|
||||
# switch a working site's arbitrage off every morning, so the requirement is
|
||||
# that the short body stays a `series` of what EXISTS and FCST-04's fill covers
|
||||
# the rest with flagged, provisional hours. The identity is re-derived from the
|
||||
# bytes rather than restated.
|
||||
_prepub = _CAPTURES["files"][_QH]["also_returned_verbatim_by"]
|
||||
check(f"the pre-publication range request {_prepub['url']!r} returned the "
|
||||
f"SAME BYTES as the bare default URL - one day where two were asked for",
|
||||
hashlib.sha256(_REAL[_QH]).hexdigest()
|
||||
== _CAPTURES["files"][_QH]["sha256"]
|
||||
and "start=2026-09-10&end=2026-09-11" in _prepub["url"])
|
||||
check(f"...and the shipped parser answers that short body with a SERIES of "
|
||||
f"what exists, never no_data - {_REPLAY[_QH].words}",
|
||||
_REPLAY[_QH].kind == "series"
|
||||
and derived(lambda: len(_REPLAY[_QH].points)) == 96)
|
||||
# ⚠️ THE COUNTS ARE TAKEN THROUGH `derived()` AND COMPARED ONLY WHEN BOTH ARE
|
||||
# NUMBERS. A mutation that turns the short answer into `no_data` makes
|
||||
# `.points` raise `Absence` by design, and `_Raised < _Raised` is a TypeError -
|
||||
# which takes the file down with a traceback instead of reporting a FAIL. A
|
||||
# check whose failure mode is a crash has no name attached (habit 5's shape,
|
||||
# with an exception in place of a hang).
|
||||
_n_short = derived(lambda: len(_REPLAY[_QH].points))
|
||||
_n_two = derived(lambda: len(_two.points))
|
||||
check(f"...which is strictly fewer points than the two-day body "
|
||||
f"({_n_short} < {_n_two}), so 'short' is a real property of this "
|
||||
f"stimulus and not a name for it",
|
||||
isinstance(_n_short, int) and isinstance(_n_two, int)
|
||||
and _n_short < _n_two)
|
||||
|
||||
# --- price_feed_url: the exact URL, and the cost said out loud ------------- #
|
||||
#
|
||||
# ⚠️ THE DECISION. An override is used VERBATIM and this plugin appends
|
||||
# nothing to it. Merging our `start`/`end` into an operator's query string
|
||||
# would mean re-serialising a URL they wrote, and `start`/`end` are
|
||||
# energy-charts' spelling - and this provider also DECLARES a SECOND host,
|
||||
# `web-api.tp.entsoe.eu`, whose date parameters are not those and are not
|
||||
# verified here, so appending our vocabulary there yields two ignored
|
||||
# parameters and the short horizon anyway. The cost is
|
||||
# that an overriding site silently gets the horizon this card exists to fix,
|
||||
# so it is NOT silent: a warning names the consequence at fetch time.
|
||||
_ovr = "https://api.energy-charts.info/price?bzn=BE&somethingelse=1"
|
||||
check(f"an override is requested EXACTLY as given, with nothing appended -> "
|
||||
f"{asked_url(_w0, _w0 + timedelta(hours=48), {'price_feed_url': _ovr})!r}",
|
||||
asked_url(_w0, _w0 + timedelta(hours=48),
|
||||
{"price_feed_url": _ovr}) == _ovr)
|
||||
|
||||
_obuf = io.StringIO()
|
||||
_oh = logging.StreamHandler(_obuf)
|
||||
logging.getLogger("goodwe.price").addHandler(_oh)
|
||||
asked_url(_w0, _w0 + timedelta(hours=48), {"price_feed_url": _ovr})
|
||||
_ovr_log = _obuf.getvalue()
|
||||
_obuf2 = io.StringIO()
|
||||
_oh2 = logging.StreamHandler(_obuf2)
|
||||
logging.getLogger("goodwe.price").removeHandler(_oh)
|
||||
logging.getLogger("goodwe.price").addHandler(_oh2)
|
||||
asked_url(_w0, _w0 + timedelta(hours=48))
|
||||
_dflt_log = _obuf2.getvalue()
|
||||
logging.getLogger("goodwe.price").removeHandler(_oh2)
|
||||
check(f"...and the operator is TOLD what it costs them, at fetch time, naming "
|
||||
f"the horizon consequence rather than only the substitution",
|
||||
"price_feed_url" in _ovr_log and _ovr in _ovr_log
|
||||
and "provisional" in _ovr_log and "2026-09-11" in _ovr_log)
|
||||
# ⚠️ THE CONTROL. Without it, "warns about everything" passes the line above.
|
||||
check("...while the DEFAULT path says nothing, so the warning is about the "
|
||||
"override and not about fetching",
|
||||
"price_feed_url is set" not in _dflt_log)
|
||||
|
||||
# The builder refuses to append to a base that already has a query string,
|
||||
# which is the "URL nobody intended" this decision is meant to make
|
||||
# unreachable. Driven, not read off the source.
|
||||
check(f"dated_feed_url() refuses a base that already carries a query string, "
|
||||
f"rather than producing a second '?' - "
|
||||
f"{_reason(lambda: dated_feed_url('https://h/p?a=1', b='2'))!r}",
|
||||
raises(lambda: dated_feed_url("https://h/p?a=1", b="2"),
|
||||
SourceUnavailable))
|
||||
check("...and a clean base still builds, so the refusal is the existing query "
|
||||
"string and not the builder",
|
||||
dated_feed_url("https://h/p", b="2") == "https://h/p?b=2")
|
||||
|
||||
# --- the plugin contract: who consumes the range, pinned ------------------- #
|
||||
#
|
||||
# ⚠️ THIS IS A SOURCE-SHAPE PIN AND IS NAMED AS ONE (habit 4). The claim is
|
||||
# about the CODE - which plugins reference the range arguments they are handed
|
||||
# - which is the same kind of claim `code_leaks()` makes, and it is not
|
||||
# evidence that any feed parses. `generic_dayahead`'s BEHAVIOUR is graded above
|
||||
# by the URL it really requests.
|
||||
#
|
||||
# It exists because `fetch(self, opts, since, until, transport)` promises every
|
||||
# plugin the range and the promise is NOT enforced. Enforcing it would oblige
|
||||
# the three plugins whose endpoints nobody has verified (TARIFF-07) to invent a
|
||||
# date-parameter vocabulary - TARIFF-08's defect moved from the response to the
|
||||
# request. So the split is pinned instead, and a sixth supplier has to change
|
||||
# this line rather than inherit the silence.
|
||||
def _uses_range(cls):
|
||||
src = dedent(inspect.getsource(cls.fetch))
|
||||
tree = ast.parse(src)
|
||||
names = {n.id for n in ast.walk(tree) if isinstance(n, ast.Name)}
|
||||
return bool(names & {"since", "until"})
|
||||
|
||||
|
||||
_uses = sorted(pid for pid, cls in PROVIDERS.items() if _uses_range(cls))
|
||||
check(f"the plugins that CONSUME their `since`/`until` are {_uses}; the rest "
|
||||
f"take the range and ignore it, which is what TARIFF-09 fixed for "
|
||||
f"generic_dayahead and what TARIFF-07 has to decide for the others",
|
||||
_uses == ["generic_dayahead"])
|
||||
check("...and `manual_csv` is honestly among the ones that do not: it reads a "
|
||||
"whole file and the fill windows it downstream, so a range parameter "
|
||||
"would have nowhere to go",
|
||||
"manual_csv" in sorted(set(PROVIDERS) - set(_uses)))
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
print()
|
||||
print("3. all-in: energy + markup + taxes, added exactly once (AC1, trap 1)")
|
||||
@@ -1674,6 +1951,660 @@ check("'off' is not refused, on any of the other configurations",
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
print()
|
||||
print("13. TARIFF-10 - a configured provider is FETCHED, and trading is still "
|
||||
"its own switch")
|
||||
# WHAT SHIPPED, AND WHY THIS SECTION IS NOT A CHECK ON AN `if`. `amain()` read
|
||||
#
|
||||
# if arbitrage.enabled(opts):
|
||||
# tasks.append(asyncio.create_task(controller.run_tariff_refresh()))
|
||||
#
|
||||
# `run_tariff_refresh` is the only caller of `resolve_tariff()`, which is the
|
||||
# only caller of `price_history.carry()`, and `arbitrage_enabled` ships FALSE.
|
||||
# So on a shipped install with a provider configured NOTHING was fetched, no
|
||||
# plan was resolved, /data/price_history.json was never created, the *Price
|
||||
# history* row read `empty` for the life of the install, and TARIFF-09's
|
||||
# short-horizon warning could not fire because no fetch happened to warn about.
|
||||
# Measured over a 40 h two-instance soak: zero feed requests and
|
||||
# `present: false` across all 802 samples.
|
||||
#
|
||||
# ⚠️ A CHECK THAT PINS THE `if` WOULD NOT HAVE CAUGHT IT, because the `if` was
|
||||
# EXACTLY WHAT SOMEBODY MEANT TO WRITE. What was wrong is the SET OF
|
||||
# CONFIGURATIONS FROM WHICH A FETCH CAN BE REACHED, so that is what is graded:
|
||||
# the guards in `amain()` are EVALUATED against real option dicts (whatever they
|
||||
# are spelled as), the call graph under `app/` is walked to see whether the
|
||||
# tasks those guards start can arrive at `resolve_tariff` (wherever it has been
|
||||
# moved to), and then the fetch is DRIVEN and the requests counted, because a
|
||||
# reachability graph is an argument and a counted request is an observation.
|
||||
|
||||
|
||||
# -- the predicate the gate is built on, before anything is wired to it -------
|
||||
# ⚠️ ONE SPELLING, AND IT NOW HAS THREE READERS. `str(price_option(opts,
|
||||
# "price_provider") or "off").strip()` was written out TWICE - in
|
||||
# `load_prices()` and in `static_tariff.resolve_tariff()` - and main.py's
|
||||
# startup gate would have been a third. "Is a provider configured" is now a
|
||||
# DECISION rather than a detail, because it is what starts a background task,
|
||||
# so the three readers have to agree about the edge cases and `provider_id()`
|
||||
# is the one place they can. (`validate.py` keeps its own reader on purpose: it
|
||||
# validates the RAW options blob and must not route through the defaults.)
|
||||
check("`provider_configured()` is False on the shipped default, on an explicit "
|
||||
"'off', and on a key that is absent, empty or None - `price_option()` "
|
||||
"maps the last three onto the default and none of them may read as a "
|
||||
"configured feed",
|
||||
not any(_price.provider_configured(o) for o in
|
||||
({}, {"price_provider": "off"}, {"price_provider": ""},
|
||||
{"price_provider": None})))
|
||||
check("...and True on each shipped plugin id, so it is a predicate and not a "
|
||||
f"constant {sorted(PROVIDERS)}",
|
||||
all(_price.provider_configured({"price_provider": pid})
|
||||
for pid in PROVIDERS))
|
||||
# ⚠️ WHITESPACE IS THE EDGE THE `.strip()` IS FOR, AND NOTHING GRADED IT. The
|
||||
# Supervisor's `list(off|...)` schema cannot produce `" off "`, but
|
||||
# `validate_startup_config()` is also sim/inject.py's entry point and takes any
|
||||
# dict at all - and dropping the `.strip()` left all four price suites green
|
||||
# (measured, -8 B) until this pair was written.
|
||||
check("...and a padded id is normalised on both sides of the decision: "
|
||||
"' off ' is still off, and ' generic_dayahead ' is still that plugin",
|
||||
_price.provider_id({"price_provider": " off "}) == "off"
|
||||
and not _price.provider_configured({"price_provider": " off "})
|
||||
and _price.provider_id({"price_provider": " generic_dayahead "})
|
||||
== "generic_dayahead"
|
||||
and _price.provider_configured(
|
||||
{"price_provider": " generic_dayahead "}))
|
||||
check("...and `load_prices()` answers `no_data` for exactly the dicts "
|
||||
"`provider_configured()` calls unconfigured, so the startup gate and the "
|
||||
"fetch cannot disagree about what 'a provider is set' means",
|
||||
all((run(load_prices(o, datetime(2026, 1, 1, tzinfo=UTC),
|
||||
datetime(2026, 1, 2, tzinfo=UTC),
|
||||
transport=None)).kind == "no_data")
|
||||
== (not _price.provider_configured(o))
|
||||
for o in ({}, {"price_provider": "off"},
|
||||
{"price_provider": " off "},
|
||||
{"price_provider": "manual_csv",
|
||||
"price_csv_path": CSV_PATH})))
|
||||
|
||||
|
||||
def _coroutine_name(node):
|
||||
"""`controller.run_tariff_refresh()` -> `run_tariff_refresh`."""
|
||||
if isinstance(node, ast.Call):
|
||||
if isinstance(node.func, ast.Attribute):
|
||||
return node.func.attr
|
||||
if isinstance(node.func, ast.Name):
|
||||
return node.func.id
|
||||
return ast.unparse(node)
|
||||
|
||||
|
||||
def _task_sites(node, guards=()):
|
||||
"""Every `create_task(...)` under `node`, with the `if`s that guard it.
|
||||
|
||||
The guard is kept as SOURCE and evaluated later against an options dict, so
|
||||
this reports what the tree says rather than what any one ticket expected to
|
||||
find there. An `else` branch is recorded as `not (<test>)`.
|
||||
"""
|
||||
out = []
|
||||
for child in ast.iter_child_nodes(node):
|
||||
if isinstance(child, ast.If):
|
||||
test = ast.unparse(child.test)
|
||||
for stmt in child.body:
|
||||
out += _task_sites(stmt, guards + (test,))
|
||||
for stmt in child.orelse:
|
||||
out += _task_sites(stmt, guards + (f"not ({test})",))
|
||||
continue
|
||||
if isinstance(child, ast.Call) and \
|
||||
ast.unparse(child.func).endswith("create_task"):
|
||||
if child.args:
|
||||
out.append((guards, _coroutine_name(child.args[0])))
|
||||
continue
|
||||
out += _task_sites(child, guards)
|
||||
return out
|
||||
|
||||
|
||||
def _call_graph(app_dir):
|
||||
"""name -> the names it calls, over every module in `app/`.
|
||||
|
||||
⚠️ IT IS KEYED BY NAME AND THEREFORE OVER-APPROXIMATES, AND THAT IS THE
|
||||
SAFE DIRECTION HERE. Two functions called `run` in two modules are one node;
|
||||
a method call on an unrelated object joins the same node. So a `reachable`
|
||||
answer is an upper bound and a `NOT reachable` answer is the strong one -
|
||||
which is why every positive row below is also DRIVEN, with the requests
|
||||
counted, rather than being left to this graph (MUTATION.md habit 13).
|
||||
"""
|
||||
graph = {}
|
||||
for name in sorted(os.listdir(app_dir)):
|
||||
if not name.endswith(".py"):
|
||||
continue
|
||||
tree = ast.parse(open(os.path.join(app_dir, name),
|
||||
encoding="utf-8").read())
|
||||
for node in ast.walk(tree):
|
||||
if not isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
|
||||
continue
|
||||
called = set()
|
||||
for inner in ast.walk(node):
|
||||
if isinstance(inner, ast.Call):
|
||||
fn = inner.func
|
||||
if isinstance(fn, ast.Attribute):
|
||||
called.add(fn.attr)
|
||||
elif isinstance(fn, ast.Name):
|
||||
called.add(fn.id)
|
||||
graph.setdefault(node.name, set()).update(called)
|
||||
return graph
|
||||
|
||||
|
||||
def _reaches(graph, start, target):
|
||||
"""Can `start` arrive at `target` through the call graph?"""
|
||||
seen, stack = set(), [start]
|
||||
while stack:
|
||||
cur = stack.pop()
|
||||
if cur in seen:
|
||||
continue
|
||||
seen.add(cur)
|
||||
for nxt in graph.get(cur, ()):
|
||||
if nxt == target:
|
||||
return True
|
||||
stack.append(nxt)
|
||||
return False
|
||||
|
||||
|
||||
def _started(sites, opts):
|
||||
"""[(coroutine, guards, verdict)] for `opts`. Verdict True/False/'?'.
|
||||
|
||||
`'?'` is a guard this cannot decide from the options alone - `p1_source is
|
||||
not None` is one - and it is reported rather than assumed either way. A
|
||||
fetch-reaching task whose guard is undecidable would make every row below
|
||||
vacuous, so that is a check of its own.
|
||||
"""
|
||||
out = []
|
||||
for guards, name in sites:
|
||||
verdict = True
|
||||
for guard in guards:
|
||||
try:
|
||||
answer = bool(eval(guard, vars(_main), {"opts": opts}))
|
||||
except NameError:
|
||||
verdict = "?"
|
||||
break
|
||||
if not answer:
|
||||
verdict = False
|
||||
break
|
||||
out.append((name, guards, verdict))
|
||||
return out
|
||||
|
||||
|
||||
_MAIN_TREE = derived(lambda: ast.parse(open(os.path.join(_APP, "main.py"),
|
||||
encoding="utf-8").read()))
|
||||
_AMAIN = derived(lambda: next(n for n in ast.walk(_MAIN_TREE)
|
||||
if isinstance(n, ast.AsyncFunctionDef)
|
||||
and n.name == "amain"))
|
||||
_SITES = derived(lambda: _task_sites(_AMAIN))
|
||||
_GRAPH = derived(lambda: _call_graph(_APP))
|
||||
_FETCH_ENTRY = "resolve_tariff"
|
||||
|
||||
print(f" create_task sites found in amain(): "
|
||||
f"{[(n, list(g)) for g, n in _SITES]}")
|
||||
print(f" call graph: {len(_GRAPH)} named functions across app/")
|
||||
print(" ⚠️ what this instrument CANNOT see, largest hole first: a coroutine "
|
||||
"started by anything other than `create_task` (gather, ensure_future, a "
|
||||
"thread, a timer); a guard that is not an enclosing `if` in amain() (a "
|
||||
"while, a try/except, an early return above the site, or a refusal "
|
||||
"INSIDE the coroutine); any module outside app/; and - because the graph "
|
||||
"is keyed by NAME - two same-named functions as one node, which makes "
|
||||
"every `reachable` answer an UPPER bound.")
|
||||
|
||||
# -- the instrument is calibrated before anything is graded with it (habit 1) --
|
||||
check(f"the scanner finds amain()'s create_task sites at all - it found "
|
||||
f"{len(_SITES) if not isinstance(_SITES, _Raised) else _SITES}, and at "
|
||||
f"zero every row below would be vacuously true",
|
||||
not isinstance(_SITES, _Raised) and len(_SITES) >= 2)
|
||||
check(f"...and the call graph can tell reaching from not-reaching: "
|
||||
f"`run_tariff_refresh` arrives at {_FETCH_ENTRY!r} and `run_control` - "
|
||||
f"the control tick, which must never await a fetch - does not. A graph "
|
||||
f"that answered the same for both would pass every row below however the "
|
||||
f"gate was wired",
|
||||
_reaches(_GRAPH, "run_tariff_refresh", _FETCH_ENTRY)
|
||||
and not _reaches(_GRAPH, "run_control", _FETCH_ENTRY))
|
||||
|
||||
|
||||
def _fetchers(opts):
|
||||
"""The coroutines amain() STARTS, for `opts`, that can reach the feed."""
|
||||
return sorted(n for n, _, v in _started(_SITES, opts)
|
||||
if v is True and _reaches(_GRAPH, n, _FETCH_ENTRY))
|
||||
|
||||
|
||||
def _undecidable_fetchers(opts):
|
||||
return sorted(n for n, _, v in _started(_SITES, opts)
|
||||
if v == "?" and _reaches(_GRAPH, n, _FETCH_ENTRY))
|
||||
|
||||
|
||||
# The four configurations. `_T10_BASE` carries no price keys at all, so the
|
||||
# shipped defaults in PRICE_OPTION_DEFAULTS are what answer - which is the
|
||||
# install the defect was measured on.
|
||||
_T10_BASE = {"setpoint_entity": "number.sp", "max_w": 5000,
|
||||
"soc_entity": "sensor.soc", "batt_entity": "sensor.batt",
|
||||
"meter_source": "off", "maintenance_charge_w": 2000,
|
||||
"maintenance_discharge_w": 2000}
|
||||
_T10_PROFILE = json.dumps({"periods": [
|
||||
{"name": "night", "days": "all", "from": "22:00", "to": "07:00",
|
||||
"eur_per_kwh": 0.20},
|
||||
{"name": "day", "days": "all", "from": "07:00", "to": "22:00",
|
||||
"eur_per_kwh": 0.284}]})
|
||||
_T10_CASES = (
|
||||
("shipped default: no provider, no trading", {}, False),
|
||||
("PROVIDER configured, trading OFF (the defect)",
|
||||
{"price_provider": "generic_dayahead"}, True),
|
||||
("PROVIDER configured, trading ON",
|
||||
{"price_provider": "generic_dayahead", "arbitrage_enabled": True}, True),
|
||||
("no provider, trading ON on a static profile",
|
||||
{"arbitrage_enabled": True, "static_profile": _T10_PROFILE}, True),
|
||||
)
|
||||
print(" startup reachability of resolve_tariff(), by configuration:")
|
||||
for _label, _over, _want in _T10_CASES:
|
||||
print(f" {'reachable' if _fetchers(dict(_T10_BASE, **_over)) else '- '}"
|
||||
f" {_label}")
|
||||
|
||||
check(f"the shipped defaults really do leave `arbitrage_enabled` off "
|
||||
f"({PRICE_OPTION_DEFAULTS['arbitrage_enabled']!r}) and `price_provider` "
|
||||
f"off ({PRICE_OPTION_DEFAULTS['price_provider']!r}) - the precondition "
|
||||
f"every row below varies from, and if a later card changes either "
|
||||
f"default these rows stop meaning what they say. Section 1 is what pins "
|
||||
f"this dict to config.yaml",
|
||||
PRICE_OPTION_DEFAULTS["price_provider"] == "off"
|
||||
and PRICE_OPTION_DEFAULTS["arbitrage_enabled"] is False
|
||||
and not _arb.enabled(dict(_T10_BASE))
|
||||
and not _price.provider_configured(dict(_T10_BASE)))
|
||||
check(f"AC1: with a `price_provider` set and `arbitrage_enabled` at its shipped "
|
||||
f"default, the startup path REACHES the feed "
|
||||
f"{_fetchers(dict(_T10_BASE, price_provider='generic_dayahead'))}. This "
|
||||
f"is the shipped defect: it read `[]`, so nothing was ever fetched",
|
||||
_fetchers(dict(_T10_BASE, price_provider="generic_dayahead")) != [])
|
||||
check(f"AC2: with `price_provider: off` and trading off - the true shipped "
|
||||
f"default - NO task reaches the feed "
|
||||
f"{_fetchers(dict(_T10_BASE))}, so an install that asked for no prices "
|
||||
f"acquires no periodic work from this card",
|
||||
_fetchers(dict(_T10_BASE)) == [])
|
||||
check("...and the guard on the fetch-reaching task is decidable from the "
|
||||
"OPTIONS alone. A gate moved onto a local variable would make every row "
|
||||
"in this section vacuous rather than red",
|
||||
_undecidable_fetchers(dict(_T10_BASE, price_provider="generic_dayahead"))
|
||||
== [])
|
||||
# ⚠️ THE `or` IS LOAD-BEARING AND THIS IS THE ROW THAT SAYS SO. `price_provider:
|
||||
# off` + a `static_profile` + `arbitrage_enabled: true` is a configuration
|
||||
# validate.py ACCEPTS and that trades (measured: ARBITRAGE_CHARGE -5000 W at
|
||||
# `cycle_cost_threshold_eur: 0.05` on a 0.20/0.284 day/night profile). Gating
|
||||
# the refresh on the provider ALONE would leave that site's `tariff_plan` at
|
||||
# None forever and silently disarm a strategy the UI says is running.
|
||||
_T10_OLD = {lbl: _arb.enabled(dict(_T10_BASE, **over))
|
||||
for lbl, over, _ in _T10_CASES}
|
||||
_T10_NEW = {lbl: bool(_fetchers(dict(_T10_BASE, **over)))
|
||||
for lbl, over, _ in _T10_CASES}
|
||||
print(f" old gate (arbitrage.enabled alone) -> {list(_T10_OLD.values())}")
|
||||
print(f" shipped gate -> {list(_T10_NEW.values())}")
|
||||
check(f"the gate is a STRICT WIDENING: every configuration that started the "
|
||||
f"refresh before still starts it, so nothing was taken away from a live "
|
||||
f"install {[l for l in _T10_OLD if _T10_OLD[l] and not _T10_NEW[l]]}",
|
||||
all(_T10_NEW[l] for l in _T10_OLD if _T10_OLD[l]))
|
||||
check("...and it is a widening rather than a no-op: at least one configuration "
|
||||
"fetches now that did not before, which is the whole ticket",
|
||||
any(_T10_NEW[l] and not _T10_OLD[l] for l in _T10_OLD))
|
||||
|
||||
|
||||
# -- and the fetch is DRIVEN, because a graph is an argument and a counted ----
|
||||
# -- request is an observation (habit 4) -------------------------------------
|
||||
_CAPTURE = derived(lambda: json.load(open(
|
||||
os.path.join(HERE, "scenarios",
|
||||
"energy-charts-be-price-2026-09-09_2026-09-10-twoday.json"),
|
||||
encoding="utf-8")))
|
||||
|
||||
|
||||
def _restamped_body(now):
|
||||
"""The COMMITTED capture, with only its instants moved onto `now`.
|
||||
|
||||
⚠️ THE SHAPE AND THE PRICES ARE THE CAPTURE'S (TARIFF-08, habit 19). What
|
||||
is synthetic is the CLOCK and nothing else: `run_tariff_refresh` reads the
|
||||
wall clock and there is no seam to inject one, so a 2026-09-09 capture
|
||||
would fall outside every horizon it asks for. Section 2b replays the
|
||||
capture unmodified against the parser; this one is grading the WIRING.
|
||||
"""
|
||||
base = int(now.timestamp()) // 3600 * 3600
|
||||
doc = dict(_CAPTURE)
|
||||
doc["unix_seconds"] = [base + 900 * i
|
||||
for i in range(len(_CAPTURE["price"]))]
|
||||
return json.dumps(doc).encode()
|
||||
|
||||
|
||||
class _T10Hass:
|
||||
async def limits(self, entity):
|
||||
return (-5000.0, 5000.0, 1.0)
|
||||
|
||||
async def number(self, entity, invert=False):
|
||||
return {"sensor.grid": 0.0, "sensor.soc": 55.0,
|
||||
"sensor.batt": 0.0}.get(entity)
|
||||
|
||||
async def set_number(self, entity, value):
|
||||
return True
|
||||
|
||||
async def call(self, domain, service, data):
|
||||
return True
|
||||
|
||||
|
||||
class _T10Store:
|
||||
data = {}
|
||||
|
||||
def set(self, *a):
|
||||
pass
|
||||
|
||||
def get_time(self, key):
|
||||
return None
|
||||
|
||||
|
||||
class _T10Pub:
|
||||
enabled = False
|
||||
|
||||
def publish(self, *a, **k):
|
||||
pass
|
||||
|
||||
def close(self):
|
||||
pass
|
||||
|
||||
|
||||
def _drive_startup(over):
|
||||
"""Start exactly what amain() would start for `opts`, and watch the wire.
|
||||
|
||||
⚠️ THE TASK IS STARTED ONLY IF THE SHIPPED GUARD SAYS SO. That is the whole
|
||||
point: the coroutine is not called because this file knows its name, it is
|
||||
called because `_fetchers()` evaluated amain()'s own guard against these
|
||||
options. A gate that goes back to `arbitrage.enabled(opts)` leaves the
|
||||
coroutine uncalled and the request count at zero.
|
||||
|
||||
⚠️ AND IT RUNS UNDER A DEADLINE (habit 5). The refresh loop is infinite by
|
||||
design; the failure mode of a mutation that never resolves a plan is a HANG,
|
||||
which reads like flaky infrastructure rather than like a named check.
|
||||
"""
|
||||
opts = dict(_T10_BASE, **over)
|
||||
tmp = tempfile.mkdtemp(prefix="tariff10-")
|
||||
_tmpdirs_t10.append(tmp)
|
||||
store_path = os.path.join(tmp, "price_history.json")
|
||||
seen = []
|
||||
|
||||
async def _counting_get(url, timeout_s):
|
||||
seen.append(url)
|
||||
return _restamped_body(datetime.now(timezone.utc))
|
||||
|
||||
real_get = _price.https_get
|
||||
_price.https_get = _counting_get
|
||||
try:
|
||||
async def _go():
|
||||
stop = asyncio.Event()
|
||||
ctl = _main.Controller(opts, _T10Hass(), _T10Store(), _T10Pub(),
|
||||
stop=stop,
|
||||
peak_db_path=os.path.join(tmp, "tariff.db"),
|
||||
price_history_path=store_path)
|
||||
names = _fetchers(opts)
|
||||
if "run_tariff_refresh" not in names:
|
||||
# Nothing amain() starts reaches the feed. Anything ELSE that
|
||||
# does is a task this harness cannot drive, and it is a FAILURE
|
||||
# rather than a shrug - see the check below.
|
||||
return ctl, names
|
||||
task = asyncio.create_task(ctl.run_tariff_refresh())
|
||||
for _ in range(250): # ~5 s ceiling
|
||||
if ctl.tariff_at is not None:
|
||||
break
|
||||
await asyncio.sleep(0.02)
|
||||
ctl.stopping = True
|
||||
stop.set()
|
||||
await asyncio.wait_for(task, timeout=5)
|
||||
return ctl, names
|
||||
ctl, names = asyncio.run(_go())
|
||||
finally:
|
||||
_price.https_get = real_get
|
||||
try:
|
||||
n_points = len(list(ctl.tariff_plan.read.points))
|
||||
except Exception: # noqa: BLE001 - Absence
|
||||
n_points = 0
|
||||
return {"requests": seen, "started": names,
|
||||
"store_present": os.path.exists(store_path),
|
||||
"points": n_points,
|
||||
"history": ctl.price_history.words}
|
||||
|
||||
|
||||
_tmpdirs_t10 = []
|
||||
_on = derived(lambda: _drive_startup({"price_provider": "generic_dayahead"}))
|
||||
_off = derived(lambda: _drive_startup({}))
|
||||
_static_armed = derived(lambda: _drive_startup(
|
||||
{"arbitrage_enabled": True, "static_profile": _T10_PROFILE}))
|
||||
print(f" provider + trading OFF -> {len(_on['requests'])} request(s), "
|
||||
f"store written {_on['store_present']}, {_on['points']} point plan, "
|
||||
f"Price history row: {_on['history'][:70]!r}")
|
||||
print(f" no provider -> {len(_off['requests'])} request(s), "
|
||||
f"store written {_off['store_present']}, "
|
||||
f"Price history row: {_off['history'][:40]!r}")
|
||||
|
||||
check(f"AC1, DRIVEN: a shipped-default install with a provider configured "
|
||||
f"actually ASKS THE FEED - {len(_on['requests'])} request(s), "
|
||||
f"{_on['requests'][:1]}. The soak measured ZERO here over 40 hours",
|
||||
len(_on["requests"]) >= 1)
|
||||
check(f"...and a plan RESOLVES from it: {_on['points']} points, against the "
|
||||
f"{len(_CAPTURE['price'])} the capture publishes",
|
||||
_on["points"] == len(_CAPTURE["price"]))
|
||||
check(f"...and /data/price_history.json is WRITTEN - present={_on['store_present']}. "
|
||||
f"The soak measured `present: false` on all 802 samples",
|
||||
_on["store_present"] is True)
|
||||
check(f"...and the *Price history* status row POPULATES rather than reading "
|
||||
f"`empty` forever: {_on['history'][:60]!r}",
|
||||
isinstance(_on["history"], str)
|
||||
and not _on["history"].startswith("empty")
|
||||
and not _on["history"].startswith("unavailable"))
|
||||
check(f"AC2, DRIVEN: `price_provider: off` issues NO request and creates no "
|
||||
f"store - {len(_off['requests'])} request(s), "
|
||||
f"present={_off['store_present']}",
|
||||
_off["requests"] == [] and _off["store_present"] is False)
|
||||
check(f"...and neither does the one configuration that keeps its task without "
|
||||
f"a provider - trading armed on a static profile makes "
|
||||
f"{len(_static_armed['requests'])} request(s), because "
|
||||
f"`resolve_tariff()` answers a provider-off site out of the static "
|
||||
f"profile without opening a socket. NOBODY acquires a periodic HTTPS "
|
||||
f"call from this card",
|
||||
_static_armed["requests"] == []
|
||||
and "run_tariff_refresh" in _static_armed["started"])
|
||||
check(f"...and no OTHER startup task can reach the feed - if one appears, this "
|
||||
f"harness cannot drive it and every driven row above stops covering the "
|
||||
f"startup path {_on['started']}",
|
||||
_on["started"] == ["run_tariff_refresh"])
|
||||
|
||||
|
||||
# -- AC3, and it is the property this card is most dangerous to: ARBITRAGE -----
|
||||
# -- OFF STILL MEANS NO TRADE, EVER. Graded by DRIVING `Controller.cycle()` ---
|
||||
# -- with a provider plan resolved - the state TARIFF-10 newly creates - and ---
|
||||
# -- moving exactly one option (habit 7). ------------------------------------
|
||||
#
|
||||
# ⚠️ THE BASELINE IS THE HALF THAT MAKES THIS A CHECK. "No trade" is satisfied
|
||||
# by a fixture in which nothing could have traded anyway - a flat series, a SoC
|
||||
# outside the bounds, a peak guard that never ran - and a check like that stays
|
||||
# green however the gate is wired. So the SAME plan, the SAME SoC, the SAME P1
|
||||
# history and the SAME threshold are run with `arbitrage_enabled: true`, and
|
||||
# that run must COMMAND THE BATTERY. If the armed row stops trading, the
|
||||
# unarmed row stops being evidence and both go red together.
|
||||
|
||||
|
||||
def _t10_sample(when, net_w):
|
||||
"""One P1 telegram through the shipped derivation."""
|
||||
imp, exp = split_signed(net_w)
|
||||
return make_sample("ha_signed", imp, exp, phases=1,
|
||||
ingest_ts=when, ingest_mono=when.timestamp())
|
||||
|
||||
|
||||
class _T10TradeHass(_T10Hass):
|
||||
def __init__(self):
|
||||
self.writes = []
|
||||
|
||||
async def set_number(self, entity, value):
|
||||
self.writes.append(float(value))
|
||||
return True
|
||||
|
||||
|
||||
def _t10_plan(now, cheap_now):
|
||||
"""A 24 h FIRM series whose cheapest hour is now, or whose dearest is.
|
||||
|
||||
Firm, because refusal 3 drops provisional hours from the trade set and a
|
||||
plan of carried hours cannot trade whatever the gate says - which would
|
||||
make the armed baseline below vacuous exactly as a flat series would.
|
||||
"""
|
||||
top = now.replace(minute=0, second=0, microsecond=0)
|
||||
lo, hi = 0.05, 0.60
|
||||
return TariffPlan(
|
||||
PriceRead.series(
|
||||
[PricePoint(start=top + timedelta(hours=i),
|
||||
eur_per_kwh=(lo if (i < 3) == cheap_now else hi),
|
||||
provisional=False)
|
||||
for i in range(24)],
|
||||
PriceProduct.DAY_AHEAD, "generic_dayahead", now),
|
||||
None, "provider")
|
||||
|
||||
|
||||
def _t10_trade_cycle(armed, cheap_now):
|
||||
"""One real `Controller.cycle()` on a resolved provider plan."""
|
||||
tmp = tempfile.mkdtemp(prefix="tariff10t-")
|
||||
_tmpdirs_t10.append(tmp)
|
||||
opts = dict(_T10_BASE, meter_source="ha_dsmr", meter_entity="sensor.grid",
|
||||
heartbeat_s=10, price_provider="generic_dayahead",
|
||||
cycle_cost_threshold_eur=0.10, arbitrage_enabled=armed)
|
||||
|
||||
async def _go():
|
||||
hass = _T10TradeHass()
|
||||
ctl = _main.Controller(opts, hass, _T10Store(), _T10Pub(),
|
||||
stop=asyncio.Event(),
|
||||
peak_db_path=os.path.join(tmp, "tariff.db"),
|
||||
price_history_path=os.path.join(tmp, "ph.json"))
|
||||
ctl.auto = True
|
||||
ctl.dev_min, ctl.dev_max = -5000.0, 5000.0
|
||||
now = datetime.now(timezone.utc)
|
||||
# ⚠️ A WHOLE QUARTER OF TELEGRAMS, AND FIVE MINUTES IS NOT ENOUGH.
|
||||
# Without any, `p1_enabled` is on, `inputs_ok` is False and cycle()
|
||||
# returns before it reaches the strategy at all. With only five
|
||||
# minutes, whether the quarter is COVERED depends on where the wall
|
||||
# clock happens to sit inside it: run late in a quarter and the guard
|
||||
# is blind, engages conservatively, and `decide()` refuses for §9.1's
|
||||
# reason instead of trading - which would make the armed baseline
|
||||
# below vacuous on some runs and not others. Measured: the same four
|
||||
# rows came back `peak guard is holding` at one time of day and
|
||||
# `ARBITRAGE_CHARGE` at another. 900 s at the shipped 5 s telegram
|
||||
# cadence covers any quarter from any starting phase.
|
||||
for k in range(181):
|
||||
ctl.p1.submit(_t10_sample(now - timedelta(seconds=5 * (180 - k)),
|
||||
300.0))
|
||||
ctl.tariff_plan, ctl.tariff_at = _t10_plan(now, cheap_now), now
|
||||
await ctl.read_inputs()
|
||||
ctl.tariff_plan, ctl.tariff_at = _t10_plan(now, cheap_now), now
|
||||
await ctl.cycle()
|
||||
return ctl, hass
|
||||
ctl, hass = asyncio.run(_go())
|
||||
return {"arb": None if ctl.arb is None else ctl.arb.action,
|
||||
"target": ctl.target, "reason": ctl.reason,
|
||||
"writes": hass.writes,
|
||||
"guard": None if ctl.peak_guard is None else ctl.peak_guard.engaged}
|
||||
|
||||
|
||||
_T10_TRADE = derived(lambda: {(a, c): _t10_trade_cycle(a, c)
|
||||
for a in (False, True) for c in (True, False)})
|
||||
for _key in sorted(_T10_TRADE, key=str):
|
||||
_r = _T10_TRADE[_key]
|
||||
print(f" arbitrage_enabled={str(_key[0]):5s} cheapest hour is now="
|
||||
f"{str(_key[1]):5s} -> arb={str(_r['arb']):16s} "
|
||||
f"commanded {_r['target']:+.0f} W, reason {_r['reason'][:40]!r}")
|
||||
|
||||
# ⚠️ THE PRECONDITION, AND IT IS THE MASKING HAZARD IN THIS FIXTURE (habit 1).
|
||||
# `decide()` refuses outright while the peak guard is engaged, §9.1 outranking
|
||||
# §10.1 - so a fixture whose guard is holding answers "no trade" in EVERY row,
|
||||
# the armed one included, and the AC3 rows below would grade the guard rather
|
||||
# than the gate. It is asserted, not assumed.
|
||||
check(f"the peak guard is CLEAR in all four rows, so what refuses a trade "
|
||||
f"below is the arbitrage gate and not §9.1's interlock "
|
||||
f"{ {k: _T10_TRADE[k]['guard'] for k in sorted(_T10_TRADE, key=str)} }",
|
||||
all(_T10_TRADE[k]["guard"] is False for k in _T10_TRADE))
|
||||
check("the ARMED baseline actually trades on the plan TARIFF-10 now resolves - "
|
||||
"it charges in the cheap hour and discharges in the dear one. Without "
|
||||
"this row, 'trading off never trades' would be satisfied by a fixture "
|
||||
"that could not have traded either way",
|
||||
_T10_TRADE[(True, True)]["arb"] == "ARBITRAGE_CHARGE"
|
||||
and _T10_TRADE[(True, False)]["arb"] == "ARBITRAGE_DISCHARGE")
|
||||
check(f"...and the site is actually COMMANDED there, which is the only "
|
||||
f"observable that moves energy: {_T10_TRADE[(True, True)]['target']:+.0f} W "
|
||||
f"on the cheap hour against "
|
||||
f"{_T10_TRADE[(True, False)]['target']:+.0f} W on the dear one",
|
||||
_T10_TRADE[(True, True)]["target"] < -1000.0
|
||||
and _T10_TRADE[(True, False)]["target"] > 1000.0)
|
||||
check(f"AC3: with `arbitrage_enabled: false` and the SAME resolved provider "
|
||||
f"plan, the strategy is never consulted and NOTHING trades - "
|
||||
f"arb={_T10_TRADE[(False, True)]['arb']!r} / "
|
||||
f"{_T10_TRADE[(False, False)]['arb']!r} in both directions",
|
||||
_T10_TRADE[(False, True)]["arb"] is None
|
||||
and _T10_TRADE[(False, False)]["arb"] is None)
|
||||
check(f"...and the command is the LOOP's, not a trade that merely went "
|
||||
f"unnamed: {_T10_TRADE[(False, True)]['target']:+.0f} W / "
|
||||
f"{_T10_TRADE[(False, False)]['target']:+.0f} W, reason "
|
||||
f"{_T10_TRADE[(False, True)]['reason'][:30]!r} - the same small "
|
||||
f"self-consumption figure on both plans, where the armed rows differ by "
|
||||
f"10 kW between them",
|
||||
all(abs(_T10_TRADE[(False, c)]["target"]) < 1000.0
|
||||
and "arbitrage" not in _T10_TRADE[(False, c)]["reason"]
|
||||
for c in (True, False)))
|
||||
check("...and the trading gate is untouched by this card: it is still "
|
||||
"`arbitrage.enabled()` and it is still the only thing the price "
|
||||
"strategy is gated on - the four rows above differ in that option and "
|
||||
"in nothing else",
|
||||
_arb.enabled({"arbitrage_enabled": True})
|
||||
and not _arb.enabled({})
|
||||
and not _arb.enabled({"price_provider": "generic_dayahead"}))
|
||||
|
||||
|
||||
# -- AC4: the four surfaces that DESCRIBE this, pinned as a PAIR with it ------
|
||||
# ⚠️ THIS IS A TEXT CHECK AND IT IS WEAK ON PURPOSE (test_translations.py §3's
|
||||
# argument). It cannot know a sentence is TRUE. What it CAN do is refuse to let
|
||||
# the code and the prose move independently: the left side of each pairing is
|
||||
# DRIVEN above, the right side is read off the file, and the two must agree. It
|
||||
# shipped as False against True - the docs described Option A for as long as
|
||||
# both existed - and an overclaim in a doc is a defect in this repository.
|
||||
_T10_DOCS = derived(lambda: open(os.path.join(HERE, "DOCS.md"),
|
||||
encoding="utf-8").read())
|
||||
_T10_CFG = derived(lambda: open(os.path.join(HERE, "config.yaml"),
|
||||
encoding="utf-8").read())
|
||||
_T10_TR = derived(lambda: {lang: open(os.path.join(HERE, "translations",
|
||||
f"{lang}.yaml"),
|
||||
encoding="utf-8").read()
|
||||
for lang in ("en", "nl")})
|
||||
_T10_FETCHES = bool(_on["requests"])
|
||||
_T10_DOCS_SAYS = "selecting a provider is what starts the fetch" \
|
||||
in _T10_DOCS.lower()
|
||||
check(f"AC4: the BEHAVIOUR and DOCS.md agree about what a configured provider "
|
||||
f"does on a shipped-default install - it fetches = {_T10_FETCHES}, DOCS "
|
||||
f"says it fetches = {_T10_DOCS_SAYS}. These two are the pair that has to "
|
||||
f"move together",
|
||||
_T10_FETCHES == _T10_DOCS_SAYS)
|
||||
# The claim that had rotted in the OTHER direction: OPT-04 shipped the
|
||||
# consumer, so "nothing consumes the price series yet" had been false for a
|
||||
# release before this card made the first half false too.
|
||||
_T10_STALE = {"config.yaml": "consumes\n # the series yet" in _T10_CFG
|
||||
or "nothing consumes the price series" in _T10_CFG,
|
||||
"en.yaml": "nothing consumes the price series yet"
|
||||
in _T10_TR["en"],
|
||||
"nl.yaml": "er is nog niets dat de prijsreeks gebruikt"
|
||||
in _T10_TR["nl"]}
|
||||
check(f"...and no surface still says the price series has no consumer - untrue "
|
||||
f"since OPT-04, and it sat beside the claim this card is fixing in all "
|
||||
f"three files {[k for k, v in _T10_STALE.items() if v]}",
|
||||
not any(_T10_STALE.values()))
|
||||
check("...and BOTH translations tell an operator which switch does which - "
|
||||
"the provider option says it is what starts the fetch, and the trading "
|
||||
"option says turning it off does not turn the feed off. One language "
|
||||
"updated and not the other is how this surface rots",
|
||||
"STARTS THE PRICE FETCH" in _T10_TR["en"]
|
||||
and "DOES NOT TURN THE PRICE FEED OFF" in _T10_TR["en"]
|
||||
and "HET OPHALEN VAN PRIJZEN START" in _T10_TR["nl"]
|
||||
and "ZET DE PRIJSFEED NIET UIT" in _T10_TR["nl"])
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
for _d in _tmpdirs_t10:
|
||||
shutil.rmtree(_d, ignore_errors=True)
|
||||
|
||||
print()
|
||||
if fails:
|
||||
print(f"{len(fails)} FAILED: " + "; ".join(fails))
|
||||
|
||||
@@ -315,8 +315,12 @@ def transport_for(body):
|
||||
return _t
|
||||
|
||||
|
||||
# ⚠️ NO `price_feed_url` (TARIFF-09). An override is used VERBATIM and the
|
||||
# plugin appends no date range to it, so a CFG that sets one - even to the
|
||||
# feed's own address - would exercise the override path in every check in this
|
||||
# file and never the shipped default. The override has its own grading in
|
||||
# test_price.py section 2c.
|
||||
CFG = {"price_provider": "generic_dayahead",
|
||||
"price_feed_url": "https://api.energy-charts.info/price?bzn=BE",
|
||||
"price_timeout_s": 1,
|
||||
"price_markup_eur_kwh": MARKUP,
|
||||
"price_taxes_eur_kwh": TAXES,
|
||||
@@ -517,6 +521,220 @@ check("...and the plan is still a provider plan, not a §11.4 fallback: the "
|
||||
_after.origin == "provider" and _after.product is PriceProduct.DAY_AHEAD)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
print()
|
||||
print("1c. TARIFF-09 - the feed is now ASKED for the horizon, and tomorrow "
|
||||
"comes back FIRM")
|
||||
#
|
||||
# ⚠️ WHAT SECTION 1 COULD NOT SEE, AND WHY. Everything above grades the FILL
|
||||
# against a feed that published one day, and it was right to: FCST-04 covers
|
||||
# the window between midnight and the ~13:00-14:00 CET auction. But
|
||||
# `generic_dayahead.fetch()` accepted `since`/`until` and requested the bare
|
||||
# `?bzn=BE`, which answers one Brussels day AT EVERY HOUR OF EVERY DAY - so the
|
||||
# one-day feed was not the pre-publication state, it was the ONLY state, and
|
||||
# §8.3 dropped every hour past today from the trade set as provisional. The
|
||||
# optimiser never saw tomorrow as firm.
|
||||
#
|
||||
# Measured on the merged tree (4257e38) at 2026-09-10T09:00+02:00 over a 48 h
|
||||
# horizon, live feed, yesterday seeded: 15 firm whole hours and 33 carried,
|
||||
# and BOTH ends of the permitted cycle inside 2026-09-10. At 23:00 it was 1
|
||||
# firm and 47 carried, and the firm spread no longer cleared the round-trip
|
||||
# cost at all. The ceiling was structural - at hour H a one-day window gives at
|
||||
# most 24-H firm hours - which is why no amount of carrying could fix it.
|
||||
#
|
||||
# The two plans below differ in ONE thing: what the feed answered. The code
|
||||
# path, the window, the store and the instant are identical, so the difference
|
||||
# is the horizon and nothing else (habit 7).
|
||||
|
||||
TOMORROW_D = "2026-01-16"
|
||||
TOMORROW_H = bday_hours(TOMORROW_D)
|
||||
|
||||
# ⚠️ TOMORROW'S PRICES MUST NOT OVERLAP TODAY'S OR YESTERDAY'S, or a point's
|
||||
# day of origin is not recoverable from its number and every claim below is
|
||||
# about a coincidence. Asserted, not assumed.
|
||||
TOMORROW_MWH = {h: 300.0 + h for h in range(24)}
|
||||
TOMORROW_MWH[18] = 900.0 # the dear evening TOMORROW: the global peak of
|
||||
# the 48 h window, and it is on the second day
|
||||
|
||||
PRICE_AT = {u: TODAY_MWH[brussels(u).hour] for u in TODAY_H}
|
||||
PRICE_AT.update({u: TOMORROW_MWH[brussels(u).hour] for u in TOMORROW_H})
|
||||
|
||||
|
||||
def body_at(instants):
|
||||
"""A columnar body priced PER INSTANT, so two days can differ at the same
|
||||
local hour. `feed_body()` keys on the local hour alone and cannot."""
|
||||
return json.dumps({
|
||||
"unix_seconds": [int(u.timestamp()) for u in instants],
|
||||
"price": [PRICE_AT[u] for u in instants],
|
||||
"unit": "EUR / MWh",
|
||||
"license_info": "CC BY 4.0 (creativecommons.org/licenses/by/4.0) "
|
||||
"from Bundesnetzagentur | SMARD.de",
|
||||
"deprecated": False}).encode()
|
||||
|
||||
|
||||
def recording(body):
|
||||
"""A transport that serves bytes and remembers the URL it was asked for."""
|
||||
seen = []
|
||||
|
||||
async def _t(url, timeout_s):
|
||||
seen.append(url)
|
||||
return body
|
||||
_t.seen = seen
|
||||
return _t
|
||||
|
||||
|
||||
_vals_today = {TODAY_MWH[h] for h in range(24)}
|
||||
_vals_tmrw = {TOMORROW_MWH[h] for h in range(24)}
|
||||
_vals_yday = {YESTERDAY_MWH[h] for h in range(24)}
|
||||
check(f"the three days' price tables share no value at all "
|
||||
f"({len(_vals_today)}/{len(_vals_tmrw)}/{len(_vals_yday)} distinct), so "
|
||||
f"which day a point came from is recoverable from the point",
|
||||
not (_vals_today & _vals_tmrw) and not (_vals_tmrw & _vals_yday)
|
||||
and not (_vals_today & _vals_yday))
|
||||
check(f"...and both fixture days are 24 Brussels hours long "
|
||||
f"({len(TODAY_H)}/{len(TOMORROW_H)}), found by search rather than by "
|
||||
f"adding 24 UTC hours",
|
||||
len(TODAY_H) == 24 and len(TOMORROW_H) == 24)
|
||||
|
||||
# THE TWO FEED STATES, one code path.
|
||||
_t_two = recording(body_at(TODAY_H + TOMORROW_H))
|
||||
_h_two = store()
|
||||
_ph.PriceHistory._remember(_h_two, series_of(YDAY_H, YESTERDAY_MWH), NOW)
|
||||
_pub = run(resolve_tariff(CFG, NOW, _until, transport=_t_two, now=NOW,
|
||||
history=_h_two))
|
||||
|
||||
_t_one = recording(body_at(TODAY_H))
|
||||
_h_one = store()
|
||||
_ph.PriceHistory._remember(_h_one, series_of(YDAY_H, YESTERDAY_MWH), NOW)
|
||||
_pre = run(resolve_tariff(CFG, NOW, _until, transport=_t_one, now=NOW,
|
||||
history=_h_one))
|
||||
|
||||
# ⚠️ THE REQUEST, WHICH IS THE THING THIS CARD CHANGED. Both calls ask for the
|
||||
# same three Brussels days; only the answer differs. The expected dates are
|
||||
# spelled out here rather than derived from the module that builds them.
|
||||
check(f"the shipped plugin asks the feed for the Brussels days the horizon "
|
||||
f"touches, on both calls -> {_t_two.seen}",
|
||||
_t_two.seen == _t_one.seen
|
||||
== ["https://api.energy-charts.info/price?bzn=BE"
|
||||
"&start=2026-01-15&end=2026-01-17"])
|
||||
check("...and it is no longer the bare URL that answers a single day, which "
|
||||
"is what made the one-day state permanent rather than pre-publication",
|
||||
"start=" in _t_two.seen[0] and "end=" in _t_two.seen[0])
|
||||
|
||||
|
||||
def split(plan):
|
||||
"""(firm, carried), and ([], []) for a plan that observed no series.
|
||||
|
||||
⚠️ THE EMPTY ANSWER IS DELIBERATE. A mutation that turns the short
|
||||
pre-publication answer into `no_data` makes `plan.read.points` raise
|
||||
`Absence` by design, and an unguarded read here would take this file down
|
||||
with a traceback rather than reporting a named FAIL - "the suite is red"
|
||||
and "the check that watches this is red" are different claims
|
||||
(MUTATION.md, reporting a mutation, point 3). With this guard the counts
|
||||
below go red BY NAME, which is what a mutation report can cite.
|
||||
"""
|
||||
if not plan.has_prices:
|
||||
return [], []
|
||||
firm = [p for p in plan.read.points if not p.provisional]
|
||||
carried = [p for p in plan.read.points if p.provisional]
|
||||
return firm, carried
|
||||
|
||||
|
||||
def local_days(points):
|
||||
return sorted({brussels(p.start).strftime("%Y-%m-%d") for p in points})
|
||||
|
||||
|
||||
_f2, _c2 = split(_pub)
|
||||
_f1, _c1 = split(_pre)
|
||||
|
||||
# The expected counts are re-derived from the fixture by set arithmetic: the
|
||||
# horizon hours the feed printed, versus the horizon hours it did not.
|
||||
_window = [SINCE_HOUR + timedelta(hours=n) for n in range(HORIZON_H)]
|
||||
_two_pub_utc = {u.astimezone(UTC) for u in TODAY_H + TOMORROW_H}
|
||||
_one_pub_utc = {u.astimezone(UTC) for u in TODAY_H}
|
||||
_want_f2 = len([u for u in _window if u.astimezone(UTC) in _two_pub_utc])
|
||||
_want_f1 = len([u for u in _window if u.astimezone(UTC) in _one_pub_utc])
|
||||
check(f"WITH TOMORROW PUBLISHED the 48 h plan is {len(_f2)} firm hours and "
|
||||
f"{len(_c2)} carried; with only today, {len(_f1)} firm and {len(_c1)} "
|
||||
f"carried. Re-derived from the fixture: {_want_f2} and {_want_f1}",
|
||||
len(_f2) == _want_f2 and len(_f1) == _want_f1
|
||||
and len(_f2) + len(_c2) == HORIZON_H
|
||||
and len(_f1) + len(_c1) == HORIZON_H
|
||||
and len(_f2) > len(_f1))
|
||||
check(f"...and the firm hours now span TWO Brussels local days "
|
||||
f"{local_days(_f2)}, where the one-day answer spans exactly one "
|
||||
f"{local_days(_f1)}. This is the acceptance criterion, and the ONE thing "
|
||||
f"no amount of carry-forward could ever produce: a carried hour is "
|
||||
f"provisional by construction",
|
||||
len(local_days(_f2)) == 2 and len(local_days(_f1)) == 1
|
||||
and local_days(_f2) == [TODAY_D, TOMORROW_D])
|
||||
|
||||
# ⚠️ THE TAIL IS STILL CARRIED, WHICH IS FCST-04 STILL DOING ITS JOB. A 48 h
|
||||
# horizon asked for at 09:17 runs 9 hours into a THIRD Brussels day, and no
|
||||
# day-ahead market publishes that far. The fix widens what is firm; it does not
|
||||
# claim the market publishes more than it does.
|
||||
check(f"FCST-04 still fills the remainder: the {len(_c2)} hours past the "
|
||||
f"market's own horizon are carried from yesterday and flagged, on a "
|
||||
f"third Brussels day {local_days(_c2)} - so the plan still covers all "
|
||||
f"{HORIZON_H} hours",
|
||||
len(_c2) > 0 and local_days(_c2) == ["2026-01-17"]
|
||||
and all(p.provisional for p in _c2))
|
||||
check("...and the plan is still a provider plan reporting itself provisional, "
|
||||
"not a §11.4 fallback",
|
||||
_pub.origin == "provider" and _pub.provisional)
|
||||
|
||||
# ⚠️ THE PRE-PUBLICATION ANSWER IS A SERIES OF WHAT EXISTS, NOT `no_data`.
|
||||
# Asking for tomorrow before the auction returns fewer days than requested.
|
||||
# A `no_data` there would switch a working site's arbitrage off every morning.
|
||||
check(f"the short answer stays a SERIES of what exists - {_pre.read.kind}, "
|
||||
f"{len(_pre.read.points)} hours, origin {_pre.origin!r} - and is never "
|
||||
f"no_data and never a truncated series presented as complete "
|
||||
f"({len(_c1)} of its hours are flagged provisional)",
|
||||
_pre.read.kind == "series" and _pre.origin == "provider"
|
||||
and len(_pre.read.points) == HORIZON_H and len(_c1) > 0)
|
||||
|
||||
# --- the trade set, which is where the whole thing lands ------------------- #
|
||||
_A9 = dict(peak_guard_active=False, min_spread_eur_kwh=0.0,
|
||||
uncertainty=frozenset())
|
||||
_v2 = _pub.arbitrage(**_A9)
|
||||
_v1 = _pre.arbitrage(**_A9)
|
||||
check("both plans still ALLOW a trade, so the two verdicts below are compared "
|
||||
"against each other and not against a refusal",
|
||||
_v2.kind == "allowed" and _v1.kind == "allowed")
|
||||
_v2 = _v2 if _v2.kind == "allowed" else None
|
||||
_v1 = _v1 if _v1.kind == "allowed" else None
|
||||
|
||||
# ⚠️ NON-DEGENERACY, DERIVED. The global cheapest firm hour must be on TODAY
|
||||
# and the global dearest on TOMORROW, or "the trade set spans two days" is a
|
||||
# property of the arithmetic rather than of the fix.
|
||||
def _day_of(v, end):
|
||||
"""The Brussels day of one end of a permitting verdict, or "" for a
|
||||
refusal - so a mutation that turns the verdict into a refusal is a named
|
||||
FAIL below rather than an `Absence` traceback."""
|
||||
return "" if v is None else brussels(getattr(v, end).start).strftime(
|
||||
"%Y-%m-%d")
|
||||
|
||||
|
||||
_lo2, _hi2 = _day_of(_v2, "cheapest"), _day_of(_v2, "dearest")
|
||||
_lo1, _hi1 = _day_of(_v1, "cheapest"), _day_of(_v1, "dearest")
|
||||
check(f"WITH TOMORROW FIRM the trade set contains hours from BOTH days: "
|
||||
f"buy on {_lo2}, sell on {_hi2}",
|
||||
sorted({_lo2, _hi2}) == [TODAY_D, TOMORROW_D])
|
||||
check(f"...where the one-day answer's buy AND sell are both inside {_lo1} - "
|
||||
f"a trade set that is a subset of ONE calendar day, structurally, which "
|
||||
f"is the defect this card is about",
|
||||
_lo1 == _hi1 == TODAY_D)
|
||||
_s2 = None if _v2 is None else _v2.spread_eur_kwh
|
||||
_s1 = None if _v1 is None else _v1.spread_eur_kwh
|
||||
check(f"...and the cross-day spread is strictly larger ({_s2} vs {_s1} "
|
||||
f"EUR/kWh), so the second day is not merely visible but PRICED",
|
||||
_s2 is not None and _s1 is not None and _s2 > _s1)
|
||||
check("...and both ends of the permitted cycle are FIRM, which "
|
||||
"`_firm_or_die()` asserts at construction and §8.3 requires",
|
||||
_v2 is not None and not _v2.cheapest.provisional
|
||||
and not _v2.dearest.provisional)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
print()
|
||||
print("2. AC2 - the spread stopped being taken over hours that have gone")
|
||||
|
||||
@@ -743,8 +743,10 @@ def run(coro):
|
||||
return asyncio.run(coro)
|
||||
|
||||
|
||||
# ⚠️ NO `price_feed_url` (TARIFF-09): an override is used verbatim and the
|
||||
# plugin appends no date range to it, so setting one - even to the feed's own
|
||||
# address - exercises the override path rather than the shipped default.
|
||||
_cfg = opts(TWO_PERIOD, price_provider="generic_dayahead",
|
||||
price_feed_url="https://api.energy-charts.info/price?bzn=BE",
|
||||
price_timeout_s=1, price_markup_eur_kwh=0.03,
|
||||
price_taxes_eur_kwh=0.06)
|
||||
_fb = run(resolve_tariff(_cfg, THU, THU + timedelta(days=1), transport=_dead,
|
||||
|
||||
@@ -143,7 +143,7 @@ configuration:
|
||||
description: "DERIVED, not picked (SAFETY-24): the reference pack holds 13.8 kWh usable and the worst modelled peak-shaving day asks for 878 Wh, which is 6.4 %, so 20 % sits ten points above the pack's own cut-out and still leaves the guard about 1.6 of its worst observed days of room. ⚠️ IT BOUNDS THE PEAK GUARD ONLY. It is not a reserve for the whole pack: ordinary self-consumption discharges straight past it and goes on down to the battery cut-out, and raising this number does not change that by a tenth of a percent. It must stay strictly above the battery cut-out or the add-on refuses to start. Do not change it without re-deriving it."
|
||||
price_provider:
|
||||
name: "Tariff · Dynamic price source"
|
||||
description: "Where day-ahead or real-time prices come from. 'off' is what ships, and means this site runs on the static tariff below with no trading. ⚠️ Switching it on today fetches, checks and publishes prices and changes no energy decision - nothing consumes the price series yet. It may not be combined with the two legacy Home Assistant price entities further down: two price sources on one site, one of which silently wins, is refused at startup rather than left looking armed."
|
||||
description: "Where day-ahead or real-time prices come from. 'off' is what ships, and means this site runs on the static tariff below with no trading. ⚠️ SWITCHING THIS ON IS WHAT STARTS THE PRICE FETCH - not the trading switch further down. With a source selected the add-on fetches the feed every half hour, checks it, records the day and shows what it holds on the Price history row, whether or not price trading is armed. On its own it still moves no energy: trading is the separate switch, and the cheap window the monthly maintenance cycle uses comes from the fixed or day/night tariff below. It may not be combined with the two legacy Home Assistant price entities further down: two price sources on one site, one of which silently wins, is refused at startup rather than left looking armed."
|
||||
price_markup_eur_kwh:
|
||||
name: "Tariff · Supplier markup (EUR/kWh)"
|
||||
description: "Your supplier's margin. The add-on adds it ONLY when the chosen price source does not already include it - under engie_dynamic, totalenergies_dynamic, eneco_eddy and manual_csv it is ignored, and logged as ignored, never applied twice."
|
||||
@@ -161,7 +161,7 @@ configuration:
|
||||
description: "For price source manual_csv only: where in /share the add-on reads your own price file from."
|
||||
price_feed_url:
|
||||
name: "Tariff · Price feed URL override"
|
||||
description: "Leave empty to use the chosen source's own feed. Only a host that source itself declares is accepted, so this cannot be pointed anywhere."
|
||||
description: "Leave empty to use the chosen source's own feed. Only a host that source itself declares is accepted, so this cannot be pointed anywhere. WHAT YOU TYPE IS SENT EXACTLY AS WRITTEN: the add-on adds nothing to it, not even the date range it normally asks its own feed for, so an override gets whatever horizon that URL returns - often a single day, with the rest of the plan carried over from yesterday and excluded from price arbitrage. A warning in the log says so on every price refresh. Clear this option to get the full horizon back."
|
||||
price_horizon_h:
|
||||
name: "Tariff · How far ahead to fetch prices (hours)"
|
||||
description: "How many hours of prices to ask the source for."
|
||||
@@ -176,7 +176,7 @@ configuration:
|
||||
description: "The average the current price is compared against. Only meaningful together with the current-price entity above, and like it, may not be combined with a dynamic price source."
|
||||
arbitrage_enabled:
|
||||
name: "Tariff · Trade on the price spread"
|
||||
description: "Charge in the cheapest published hour and discharge in the dearest, but only when the spread covers what the round trip costs. OFF in the shipped configuration, deliberately: this is the first strategy in the add-on that moves energy against the meter on purpose, so it is armed on purpose or not at all. It needs both a price source and a P1 meter, and the add-on refuses the combinations that cannot work rather than letting them look armed."
|
||||
description: "Charge in the cheapest published hour and discharge in the dearest, but only when the spread covers what the round trip costs. OFF in the shipped configuration, deliberately: this is the first strategy in the add-on that moves energy against the meter on purpose, so it is armed on purpose or not at all. ⚠️ LEAVING IT OFF DOES NOT TURN THE PRICE FEED OFF. A price source selected above is fetched, checked and recorded either way; this switch decides only whether the add-on ACTS on those prices. It needs both a price source and a P1 meter, and the add-on refuses the combinations that cannot work rather than letting them look armed."
|
||||
cycle_cost_threshold_eur:
|
||||
name: "Tariff · Cost of one full charge/discharge cycle (EUR/kWh)"
|
||||
description: "What one full round trip costs per kWh: wear on the pack plus conversion losses. A price spread smaller than this loses money and battery life, so it is not traded. ⚠️ THIS ONE IS NOT DERIVED FROM A MEASUREMENT. 0.10 is a product decision and it is still an open question with the owner (OPT-08). At this value a fixed day/night site does not trade at all - the realistic Belgian day/night spread is about 0.08 EUR/kWh - and that is the correct verdict at this number, not a fault."
|
||||
|
||||
@@ -123,7 +123,7 @@ configuration:
|
||||
description: "AFGELEID, niet gekozen (SAFETY-24): de referentiebatterij heeft 13,8 kWh bruikbaar en de zwaarste doorgerekende afvlakdag vraagt 878 Wh, oftewel 6,4 %, dus 20 % ligt tien punten boven de uitschakelgrens van de batterij zelf en laat de bewaking nog ongeveer 1,6 van haar zwaarste waargenomen dagen speling. ⚠️ DIT BEGRENST ALLEEN DE PIEKBEWAKING. Het is geen reserve voor de hele batterij: gewoon eigenverbruik ontlaadt er dwars doorheen en gaat door tot aan de uitschakelgrens van de batterij, en dit getal verhogen verandert daar geen tiende procent aan. Het moet strikt boven de uitschakelgrens van de batterij blijven, anders start de add-on niet op. Verander het niet zonder de afleiding opnieuw te maken."
|
||||
price_provider:
|
||||
name: "Tarief · Bron van dynamische prijzen"
|
||||
description: "Waar day-ahead- of realtimeprijzen vandaan komen. 'off' is de standaardinstelling en betekent dat deze installatie op het vaste tarief hieronder draait, zonder handel. ⚠️ Dit vandaag aanzetten haalt prijzen op, controleert en publiceert ze, en verandert geen enkele energiebeslissing - er is nog niets dat de prijsreeks gebruikt. Het mag niet gecombineerd worden met de twee oude Home Assistant-prijsentiteiten verderop: twee prijsbronnen op één installatie, waarvan er stilzwijgend één wint, wordt bij het opstarten geweigerd in plaats van schijnbaar actief gelaten."
|
||||
description: "Waar day-ahead- of realtimeprijzen vandaan komen. 'off' is de standaardinstelling en betekent dat deze installatie op het vaste tarief hieronder draait, zonder handel. ⚠️ DIT AANZETTEN IS WAT HET OPHALEN VAN PRIJZEN START - niet de handelsschakelaar verderop. Met een bron ingesteld haalt de add-on de feed elk half uur op, controleert die, bewaart de dag en toont op de rij Prijsgeschiedenis wat er bewaard is, of prijshandel nu aan staat of niet. Op zichzelf verplaatst dat nog geen energie: handelen is de aparte schakelaar, en het goedkope venster dat de maandelijkse onderhoudscyclus gebruikt komt uit het vaste of dag/nacht-tarief hieronder. Het mag niet gecombineerd worden met de twee oude Home Assistant-prijsentiteiten verderop: twee prijsbronnen op één installatie, waarvan er stilzwijgend één wint, wordt bij het opstarten geweigerd in plaats van schijnbaar actief gelaten."
|
||||
price_markup_eur_kwh:
|
||||
name: "Tarief · Marge van de leverancier (EUR/kWh)"
|
||||
description: "De marge van uw energieleverancier. De add-on telt die ALLEEN op wanneer de gekozen prijsbron ze niet al bevat - bij engie_dynamic, totalenergies_dynamic, eneco_eddy en manual_csv wordt ze genegeerd, en dat wordt ook gelogd, zodat ze nooit dubbel wordt aangerekend."
|
||||
@@ -141,7 +141,7 @@ configuration:
|
||||
description: "Alleen voor prijsbron manual_csv: waar in /share de add-on uw eigen prijsbestand leest."
|
||||
price_feed_url:
|
||||
name: "Tarief · Afwijkende URL voor de prijsfeed"
|
||||
description: "Laat leeg om de eigen feed van de gekozen bron te gebruiken. Alleen een host die die bron zelf opgeeft, wordt aanvaard, zodat dit niet zomaar ergens naar kan verwijzen."
|
||||
description: "Laat leeg om de eigen feed van de gekozen bron te gebruiken. Alleen een host die die bron zelf opgeeft, wordt aanvaard, zodat dit niet zomaar ergens naar kan verwijzen. WAT U HIER TYPT WORDT LETTERLIJK VERSTUURD: de add-on voegt er niets aan toe, ook niet het datumbereik dat ze normaal aan haar eigen feed vraagt. Een afwijkende URL levert dus de horizon op die die URL zelf teruggeeft - vaak één dag, waarna de rest van het plan uit de prijzen van gisteren wordt overgenomen en buiten de prijsarbitrage valt. Bij elke prijsvernieuwing verschijnt daarover een waarschuwing in het logboek. Maak dit veld leeg om de volledige horizon terug te krijgen."
|
||||
price_horizon_h:
|
||||
name: "Tarief · Hoe ver vooruit prijzen ophalen (uren)"
|
||||
description: "Hoeveel uur aan prijzen er bij de bron opgevraagd wordt."
|
||||
@@ -156,7 +156,7 @@ configuration:
|
||||
description: "Het gemiddelde waarmee de huidige prijs vergeleken wordt. Alleen zinvol samen met de entiteit hierboven en, net als die, niet te combineren met een dynamische prijsbron."
|
||||
arbitrage_enabled:
|
||||
name: "Tarief · Handelen op het prijsverschil"
|
||||
description: "Laden in het goedkoopste gepubliceerde uur en ontladen in het duurste, maar alleen wanneer het verschil de kosten van de volledige cyclus dekt. STAAT UIT in de standaardinstelling, met opzet: dit is de eerste strategie in de add-on die bewust energie tegen de meter in verplaatst, dus ze wordt bewust aangezet of helemaal niet. Ze heeft zowel een prijsbron als een P1-meter nodig, en de add-on weigert combinaties die niet kunnen werken in plaats van ze schijnbaar actief te laten."
|
||||
description: "Laden in het goedkoopste gepubliceerde uur en ontladen in het duurste, maar alleen wanneer het verschil de kosten van de volledige cyclus dekt. STAAT UIT in de standaardinstelling, met opzet: dit is de eerste strategie in de add-on die bewust energie tegen de meter in verplaatst, dus ze wordt bewust aangezet of helemaal niet. ⚠️ DIT UIT LATEN STAAN ZET DE PRIJSFEED NIET UIT. Een prijsbron die hierboven gekozen is, wordt hoe dan ook opgehaald, gecontroleerd en bewaard; deze schakelaar bepaalt alleen of de add-on op die prijzen HANDELT. Ze heeft zowel een prijsbron als een P1-meter nodig, en de add-on weigert combinaties die niet kunnen werken in plaats van ze schijnbaar actief te laten."
|
||||
cycle_cost_threshold_eur:
|
||||
name: "Tarief · Kostprijs van één volledige laad/ontlaadcyclus (EUR/kWh)"
|
||||
description: "Wat één volledige cyclus per kWh kost: slijtage van de batterij plus omzettingsverliezen. Een prijsverschil dat hier onder ligt, kost geld en levensduur en wordt dus niet verhandeld. ⚠️ DIT GETAL IS NIET AFGELEID UIT EEN METING. 0,10 is een productbeslissing en staat nog open bij de eigenaar (OPT-08). Bij deze waarde handelt een installatie met een vast dag/nachttarief helemaal niet - het realistische Belgische dag/nachtverschil is ongeveer 0,08 EUR/kWh - en dat is bij dit getal het juiste antwoord, geen fout."
|
||||
|
||||
Reference in New Issue
Block a user