Claude Opus 5
17b7b447c4
checks / unit (pull_request) Successful in 7m10s
`generic_dayahead.fetch()` accepted `since` and `until` and requested the
bare `?bzn=BE`, which answers ONE Brussels day. Everything past today was
then filled from yesterday's profile by FCST-04, flagged provisional, and
§8.3 drops provisional hours from the trade set -- so the optimiser never
saw tomorrow as firm, at any hour of any day, and nothing reported it.
Measured on 4257e38 at 2026-09-10T09:00+02:00, 48 h horizon, live feed,
yesterday seeded through the shipped read path:
60 firm points (15 whole hours) + 33 carried hours of 48 requested
carried: 24 h on 2026-09-11, 9 h on 2026-09-12
arbitrage(0.10): buy 11:45 and sell 19:45 -- BOTH on 2026-09-10
firm local days = [2026-09-10]
The ceiling was structural: with a one-day window, at hour H a 48 h horizon
is at most 24-H firm and 24+H carried. At 23:00 that is 1 and 47, and the
remaining firm spread (0.0464 EUR/kWh) no longer clears the default 0.10
threshold, so arbitrage is refused outright. FCST-04's fill covers the
window until the ~13:00-14:00 CET auction; it was never meant to cover
every hour of every day.
The data exists and the endpoint serves it: verified 2026-09-10T07:45:49Z,
`?bzn=BE&start=2026-09-09&end=2026-09-10` returns 192 quarter-hours across
two local days. Committed unmodified beside TARIFF-08's captures, with the
URL, retrieval time and sha256 recorded and RE-DERIVED by the replay.
THE RANGE IS DERIVED ON THE BRUSSELS CALENDAR, not the UTC one. 22:30 UTC
in summer is already tomorrow in Belgium, so a UTC-derived start asks for
the day the controller has just left and drops the day it is standing in,
for the last one or two hours of every day. That is `_bdate()`'s lesson one
layer up with a worse consequence: `_bdate()` misfiles an hour, this never
fetches it. `until` is exclusive, so local midnight belongs to the day
before it; an inverted window collapses to one day rather than a backwards
range. Both ends are graded and both are driven red by mutation.
After, same code path and window, a feed serving both days: 39 firm hours
and 9 carried (was 15 and 33), firm across TWO Brussels days, and the
permitted cycle buys on one day and sells on the next.
`price_feed_url` IS THE EXACT URL AND THE PLUGIN APPENDS NOTHING. `start`/
`end` are this feed's spelling and the provider declares a second host
whose parameters are different, so merging would produce a request nobody
typed and the short horizon anyway. The cost is real, so it is not silent:
a WARNING on the shipped logger at fetch time names the consequence and the
start..end that would otherwise have been asked for. `dated_feed_url()`
refuses to append to a base that already carries a query string, so the
blind-append shape cannot come back by edit.
DECISIONS/TARIFF-09-the-range-and-the-override.md argues the alternative.
Before publication nothing regresses and it is graded, not assumed: at
09:46 CEST the two-day request came back byte-identical to the one-day
body, and that stays a `series` of what exists with FCST-04 filling the
rest. A `no_data` there would switch a working site's arbitrage off every
morning -- the mutation that adds that rule is red by name.
The plugin contract is still NOT enforced: three shipped plugins point at
endpoints nobody has verified, and a rule obliging them to pass a range
would be satisfied by an invented parameter name -- TARIFF-08's defect
moved from the response to the request. Which plugins consume the range is
pinned as a literal instead.
11 mutations, all red, no survivors; each applied to a fresh copy of the
tree rather than reverted, through tools/mutation_run.sh. One is +0 B (a
single price digit inside the committed capture) and the re-derived sha256
catches it. MUTATION.md habit 20 records why a fixture could never have
reached this: a captured body is the answer to a URL, and no property of a
body says which URL was asked for. All 29 suites green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
142 lines
7.2 KiB
Markdown
142 lines
7.2 KiB
Markdown
# TARIFF-09 — asking the feed for the horizon, and what `price_feed_url` means
|
||
|
||
Two decisions. The first had no defensible alternative once it was measured;
|
||
the second did, and the alternative is argued below rather than dismissed.
|
||
|
||
---
|
||
|
||
## 0. The measurement this card exists for
|
||
|
||
`generic_dayahead.fetch()` accepted `since` and `until` and requested the bare
|
||
`https://api.energy-charts.info/price?bzn=BE`. That URL names no range, and the
|
||
feed answers **one Brussels day**.
|
||
|
||
Measured on the merged tree (`4257e38`) at 2026-09-10 09:00 CEST, 48 h horizon,
|
||
live feed, yesterday seeded through the shipped read path:
|
||
|
||
```
|
||
60 firm points (15 whole hours) + 33 carried hours of 48 requested
|
||
carried: 24 h on 2026-09-11, 9 h on 2026-09-12
|
||
arbitrage(0.10): allowed, spread 0.4645, buy 11:45 and sell 19:45 -- BOTH on 2026-09-10
|
||
firm local days = [2026-09-10]
|
||
```
|
||
|
||
Everything past today was carry-forward, flagged provisional, and §8.3 drops
|
||
provisional hours from the trade set. **The optimiser never saw tomorrow as
|
||
firm, at any hour.** The ceiling was structural rather than incidental: with a
|
||
one-day window, at hour *H* of the local day a 48 h horizon is at most
|
||
**24−H hours firm and 24+H carried** — 15/33 at 09:00, and **1/47 at 23:00**,
|
||
where the remaining firm spread (0.0464 EUR/kWh) no longer even clears the
|
||
default 0.10 round-trip threshold and arbitrage is refused outright.
|
||
|
||
That is not what FCST-04's fill was built for. FCST-04 covers the
|
||
pre-publication window until the ~13:00–14:00 CET auction, after which a
|
||
healthy feed hands over tomorrow in full.
|
||
|
||
**The data exists and the endpoint serves it.** Verified 2026-09-10T07:45:49Z:
|
||
`?bzn=BE&start=2026-09-09&end=2026-09-10` returns **192 quarter-hours across
|
||
two local days**, committed unmodified as
|
||
`scenarios/energy-charts-be-price-2026-09-09_2026-09-10-twoday.json`. The
|
||
plugin simply never asked.
|
||
|
||
After the change, same code path, same window, a feed serving both days:
|
||
**39 firm hours and 9 carried**, firm across **two** Brussels days, and the
|
||
permitted cycle buys on one day and sells on the other. Graded in
|
||
`test_price_history.py` section 1c against both feed states.
|
||
|
||
---
|
||
|
||
## 1. The range is derived on the **Brussels** calendar, from `since`/`until`
|
||
|
||
`since` and `until` are **instants**; the API takes **dates**. Those are not
|
||
the same question, and the conversion is where the day gets lost.
|
||
|
||
`brussels_date_range()` returns two inclusive `date`s:
|
||
|
||
* the first is `brussels(since).date()`. **Not `since.date()`, and not the UTC
|
||
date.** 22:30 UTC in summer is already tomorrow in Belgium, so a UTC-derived
|
||
start asks for the day the controller has just left and drops the day it is
|
||
standing in — for the last one or two hours of *every* day. This is
|
||
`price_history._bdate()`'s lesson one layer up, with a worse consequence:
|
||
`_bdate()` misfiles an hour, this never fetches it.
|
||
* the last is `brussels(until).date()`, **stepped back a day when `until`
|
||
falls exactly on local midnight**, because `until` is exclusive and that
|
||
instant is the first tick of a day nobody asked for. Without it every
|
||
whole-hour horizon requests one extra day — harmless on this feed, but it
|
||
makes the request a claim the caller never made.
|
||
* an empty or inverted window collapses to a single day rather than producing
|
||
a backwards range, which no feed has a defined answer for.
|
||
|
||
Both ends are graded, and both are driven red by mutation — including the two
|
||
that swap `brussels()` for the UTC date (`test_price.py` section 2c).
|
||
|
||
---
|
||
|
||
## 2. `price_feed_url` is the **exact URL**, used as given. The plugin appends
|
||
nothing.
|
||
|
||
The alternative — merge our `start`/`end` into whatever query string the
|
||
operator supplied — was considered and rejected. Three reasons, in the order
|
||
they decided it:
|
||
|
||
1. **`start`/`end` are energy-charts' spelling, and this provider declares a
|
||
second host.** `generic_dayahead.capabilities().egress_hosts` is
|
||
`("api.energy-charts.info", "web-api.tp.entsoe.eu")`. That second host's
|
||
day-ahead date parameters have not been verified by this card and are not
|
||
guessed at here — but they are not `start`/`end`. So an override to the
|
||
*other declared host* — a case the declaration itself anticipates — is
|
||
exactly where merging appends two parameters that host ignores, producing
|
||
the short horizon anyway, with a URL nobody typed as the receipt.
|
||
2. **Merging means re-serialising a URL somebody else wrote.** Parsing and
|
||
rebuilding reorders parameters, collapses duplicates and re-escapes values.
|
||
"Override the feed" is what the option already says it means, and an
|
||
override that is silently rewritten is not one.
|
||
3. **Blind appending is not on the table at all.** `dated_feed_url()` refuses
|
||
to add a query string to a base that already has one, so
|
||
`?bzn=BE?bzn=BE&start=…` cannot be produced by this module even by a later
|
||
edit that folds the parameters back into the constant.
|
||
|
||
⚠️ **The cost is real and it is therefore NOT a doc footnote.** An overriding
|
||
operator gets exactly the short horizon this card exists to fix. So the plugin
|
||
emits a `WARNING` on the shipped `goodwe.price` logger at **fetch time**, naming
|
||
the consequence — one Brussels day, the rest carried and flagged, §8.3 dropping
|
||
them from the trade set — and naming the `start..end` it *would* have asked
|
||
for. §11.4 is what makes a price defect quiet; an operator who has taken the
|
||
short horizon has to be able to find out from the log that they did.
|
||
|
||
The warning is graded, with a control that the default path stays silent, and
|
||
a mutation that downgrades it to `debug()` takes that check red.
|
||
|
||
**Fetch time rather than startup**, because that is where the URL actually in
|
||
use is known. The trade-off is one line per refresh (the tariff task, not the
|
||
30 s control tick). If the status page ever grows a price-source row, this is
|
||
the sentence it should carry.
|
||
|
||
---
|
||
|
||
## 3. The plugin contract: `fetch(self, opts, since, until, transport)`
|
||
|
||
The signature promises every plugin the range, and until this card **none of
|
||
the four networked plugins used it**. The obvious follow-up is to enforce the
|
||
promise. It is **not** enforced, and the reason is TARIFF-08's:
|
||
|
||
Three of the five shipped plugins point at endpoints nobody has ever verified
|
||
(TARIFF-07 is the open decision on withdrawing them). A rule obliging them to
|
||
pass a date range would be satisfied by an **invented parameter name** — which
|
||
is the parser-graded-against-its-own-guess defect moved from the response to
|
||
the request, and it would ship looking like a guarantee.
|
||
|
||
What exists instead is a **roster pin** in `test_price.py` section 2c: which
|
||
plugins reference their `since`/`until` is asserted as a literal, derived from
|
||
the registry and the AST. A sixth supplier, or a change to any of the three,
|
||
has to edit that line and make the decision rather than inherit the silence.
|
||
It is named as a source-shape pin, not as evidence that any feed parses;
|
||
`generic_dayahead`'s behaviour is graded separately by the URL it really
|
||
requests.
|
||
|
||
**Recommendation for whoever picks up TARIFF-07:** enforce it at the moment
|
||
those three endpoints are either verified or withdrawn, not before. If they are
|
||
withdrawn, `generic_dayahead` and `manual_csv` are the whole roster and the
|
||
rule is one line — `manual_csv` reads a whole file and is windowed downstream,
|
||
so it would be the single documented exemption.
|