From 17b7b447c43fc0185aa24f1fd39ffeb576d76694 Mon Sep 17 00:00:00 2001 From: Claude Opus 5 Date: Thu, 10 Sep 2026 10:35:12 +0200 Subject: [PATCH] TARIFF-09: ask the day-ahead feed for the horizon it was handed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `generic_dayahead.fetch()` accepted `since` and `until` and requested the bare `?bzn=BE`, which answers ONE Brussels day. Everything past today was then filled from yesterday's profile by FCST-04, flagged provisional, and §8.3 drops provisional hours from the trade set -- so the optimiser never saw tomorrow as firm, at any hour of any day, and nothing reported it. Measured on 4257e38 at 2026-09-10T09:00+02:00, 48 h horizon, live feed, yesterday seeded through the shipped read path: 60 firm points (15 whole hours) + 33 carried hours of 48 requested carried: 24 h on 2026-09-11, 9 h on 2026-09-12 arbitrage(0.10): buy 11:45 and sell 19:45 -- BOTH on 2026-09-10 firm local days = [2026-09-10] The ceiling was structural: with a one-day window, at hour H a 48 h horizon is at most 24-H firm and 24+H carried. At 23:00 that is 1 and 47, and the remaining firm spread (0.0464 EUR/kWh) no longer clears the default 0.10 threshold, so arbitrage is refused outright. FCST-04's fill covers the window until the ~13:00-14:00 CET auction; it was never meant to cover every hour of every day. The data exists and the endpoint serves it: verified 2026-09-10T07:45:49Z, `?bzn=BE&start=2026-09-09&end=2026-09-10` returns 192 quarter-hours across two local days. Committed unmodified beside TARIFF-08's captures, with the URL, retrieval time and sha256 recorded and RE-DERIVED by the replay. THE RANGE IS DERIVED ON THE BRUSSELS CALENDAR, not the UTC one. 22:30 UTC in summer is already tomorrow in Belgium, so a UTC-derived start asks for the day the controller has just left and drops the day it is standing in, for the last one or two hours of every day. That is `_bdate()`'s lesson one layer up with a worse consequence: `_bdate()` misfiles an hour, this never fetches it. `until` is exclusive, so local midnight belongs to the day before it; an inverted window collapses to one day rather than a backwards range. Both ends are graded and both are driven red by mutation. After, same code path and window, a feed serving both days: 39 firm hours and 9 carried (was 15 and 33), firm across TWO Brussels days, and the permitted cycle buys on one day and sells on the next. `price_feed_url` IS THE EXACT URL AND THE PLUGIN APPENDS NOTHING. `start`/ `end` are this feed's spelling and the provider declares a second host whose parameters are different, so merging would produce a request nobody typed and the short horizon anyway. The cost is real, so it is not silent: a WARNING on the shipped logger at fetch time names the consequence and the start..end that would otherwise have been asked for. `dated_feed_url()` refuses to append to a base that already carries a query string, so the blind-append shape cannot come back by edit. DECISIONS/TARIFF-09-the-range-and-the-override.md argues the alternative. Before publication nothing regresses and it is graded, not assumed: at 09:46 CEST the two-day request came back byte-identical to the one-day body, and that stays a `series` of what exists with FCST-04 filling the rest. A `no_data` there would switch a working site's arbitrage off every morning -- the mutation that adds that rule is red by name. The plugin contract is still NOT enforced: three shipped plugins point at endpoints nobody has verified, and a rule obliging them to pass a range would be satisfied by an invented parameter name -- TARIFF-08's defect moved from the response to the request. Which plugins consume the range is pinned as a literal instead. 11 mutations, all red, no survivors; each applied to a fresh copy of the tree rather than reverted, through tools/mutation_run.sh. One is +0 B (a single price digit inside the committed capture) and the re-derived sha256 catches it. MUTATION.md habit 20 records why a fixture could never have reached this: a captured body is the answer to a URL, and no property of a body says which URL was asked for. All 29 suites green. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU --- goodwe_controller/CHANGELOG.md | 76 +++++ .../TARIFF-09-the-range-and-the-override.md | 141 +++++++++ goodwe_controller/DOCS.md | 25 +- goodwe_controller/MUTATION.md | 40 +++ goodwe_controller/app/price.py | 158 +++++++++- goodwe_controller/config.yaml | 8 + ...be-price-2026-09-09_2026-09-10-twoday.json | 1 + .../energy-charts-be-price-captures.json | 38 ++- goodwe_controller/test_price.py | 283 +++++++++++++++++- goodwe_controller/test_price_history.py | 220 +++++++++++++- goodwe_controller/test_static_tariff.py | 4 +- goodwe_controller/translations/en.yaml | 2 +- goodwe_controller/translations/nl.yaml | 2 +- 13 files changed, 980 insertions(+), 18 deletions(-) create mode 100644 goodwe_controller/DECISIONS/TARIFF-09-the-range-and-the-override.md create mode 100644 goodwe_controller/scenarios/energy-charts-be-price-2026-09-09_2026-09-10-twoday.json diff --git a/goodwe_controller/CHANGELOG.md b/goodwe_controller/CHANGELOG.md index f373f53..973b73f 100644 --- a/goodwe_controller/CHANGELOG.md +++ b/goodwe_controller/CHANGELOG.md @@ -16,6 +16,82 @@ if it has, it needs 0.10.0. Whoever deploys decides. | _this branch_ | FCST-05 - one byte stops stopping the add-on, and the write path gets the read path's standard | | _this branch_ | FCST-06 - "cannot persist" stops meaning "has no /data" and starts meaning what it says | | _this branch_ | TARIFF-08 - the day-ahead feed is read in the shape it actually publishes, and the fixtures stop agreeing with the parser | +| _this branch_ | TARIFF-09 - the day-ahead feed is asked for the horizon it was given, so tomorrow arrives firm instead of carried | + +### TARIFF-09 - the feed is asked for the range the caller wanted + +**`generic_dayahead` took `since` and `until` and requested the bare feed URL, +which answers one Brussels day.** Everything past today was then filled from +yesterday's profile by FCST-04, flagged provisional, and §8.3 drops provisional +hours from the trade set - so **the optimiser never saw tomorrow as firm, at any +hour of any day.** Measured on `4257e38` at 2026-09-10 09:00 CEST over the +shipped 48 h horizon, live feed, yesterday seeded: + +| | 48 h horizon, one-day feed | +|---|---| +| firm | **15 whole hours** (60 quarter-hour points) | +| carried, provisional | **33 hours** - 24 on D+1, 9 on D+2 | +| permitted cycle | buy 11:45, sell 19:45 - **both on 2026-09-10** | +| firm local days | `[2026-09-10]` | + +The ceiling was structural, not incidental: with a one-day window, at hour *H* +of the local day a 48 h horizon is at most **24−H hours firm and 24+H carried**. +At 23:00 that is **1 firm hour and 47 carried**, and the remaining firm spread +(0.0464 EUR/kWh) no longer clears the default 0.10 round-trip threshold at all - +so arbitrage was refused outright. This is **not** what FCST-04's fill is for: +that covers the window until the ~13:00-14:00 CET auction, after which a healthy +feed publishes tomorrow in full. + +**The data exists and the endpoint serves it.** Verified 2026-09-10T07:45:49Z: +`?bzn=BE&start=2026-09-09&end=2026-09-10` returns 192 quarter-hours across two +local days, committed unmodified beside TARIFF-08's captures. The plugin simply +never asked. + +**The range is now derived from `since`/`until` on the BRUSSELS calendar**, as +two inclusive dates. Not the UTC calendar: 22:30 UTC in summer is already +tomorrow in Belgium, so a UTC-derived start asks for the day the controller has +just left and drops the day it is standing in, for the last one or two hours of +every day. That is `price_history._bdate()`'s lesson one layer up, with a worse +consequence - `_bdate()` misfiles an hour, this never fetches it. `until` is +exclusive, so a horizon ending exactly at local midnight asks for one day fewer +than the naive reading gives, and an inverted window collapses to a single day +rather than a backwards range. + +Same code path, same window, a feed serving both days: + +| | before | after | +|---|---|---| +| firm hours of 48 | 15 | **39** | +| carried, provisional | 33 | 9 | +| firm local days | 1 | **2** | +| permitted cycle | one calendar day | **buys on one day, sells on the next** | + +The 9 remaining carried hours are the tail of the horizon that runs into a third +Brussels day, which no day-ahead market publishes. FCST-04 still fills them and +still flags them; what changed is what is firm, never what may be acted on. + +**Before publication nothing regresses, and that is graded rather than +assumed.** Asking for tomorrow before the auction returns fewer days than +requested - measured, at 09:46 CEST the two-day request came back byte-identical +to the one-day body - and that stays a `series` of what exists, with FCST-04 +filling the remainder. A `no_data` there would switch a working site's arbitrage +off every morning. + +⚠️ **`price_feed_url` is the EXACT URL and the plugin appends nothing to it, so +an overriding site keeps the short horizon.** That is a decision, not an +oversight: `start`/`end` are this feed's spelling, the provider declares a second +host whose parameters are different, and rewriting a URL an operator typed is +how a request nobody intended gets made. **It is not silent** - the plugin logs a +`WARNING` at fetch time naming the consequence and the `start..end` it would +otherwise have asked for. See +`DECISIONS/TARIFF-09-the-range-and-the-override.md`. + +The plugin contract - `fetch(self, opts, since, until, transport)` hands every +plugin the range - is still **not** enforced, because three shipped plugins point +at endpoints nobody has verified and a rule obliging them to pass a range would +be satisfied by an invented parameter name. Which plugins consume the range is +pinned as a literal in `test_price.py` instead, so the next supplier has to +decide rather than inherit the silence. ### FCST-06 - the second way to be a store that cannot persist diff --git a/goodwe_controller/DECISIONS/TARIFF-09-the-range-and-the-override.md b/goodwe_controller/DECISIONS/TARIFF-09-the-range-and-the-override.md new file mode 100644 index 0000000..a3e046e --- /dev/null +++ b/goodwe_controller/DECISIONS/TARIFF-09-the-range-and-the-override.md @@ -0,0 +1,141 @@ +# TARIFF-09 — asking the feed for the horizon, and what `price_feed_url` means + +Two decisions. The first had no defensible alternative once it was measured; +the second did, and the alternative is argued below rather than dismissed. + +--- + +## 0. The measurement this card exists for + +`generic_dayahead.fetch()` accepted `since` and `until` and requested the bare +`https://api.energy-charts.info/price?bzn=BE`. That URL names no range, and the +feed answers **one Brussels day**. + +Measured on the merged tree (`4257e38`) at 2026-09-10 09:00 CEST, 48 h horizon, +live feed, yesterday seeded through the shipped read path: + +``` +60 firm points (15 whole hours) + 33 carried hours of 48 requested +carried: 24 h on 2026-09-11, 9 h on 2026-09-12 +arbitrage(0.10): allowed, spread 0.4645, buy 11:45 and sell 19:45 -- BOTH on 2026-09-10 +firm local days = [2026-09-10] +``` + +Everything past today was carry-forward, flagged provisional, and §8.3 drops +provisional hours from the trade set. **The optimiser never saw tomorrow as +firm, at any hour.** The ceiling was structural rather than incidental: with a +one-day window, at hour *H* of the local day a 48 h horizon is at most +**24−H hours firm and 24+H carried** — 15/33 at 09:00, and **1/47 at 23:00**, +where the remaining firm spread (0.0464 EUR/kWh) no longer even clears the +default 0.10 round-trip threshold and arbitrage is refused outright. + +That is not what FCST-04's fill was built for. FCST-04 covers the +pre-publication window until the ~13:00–14:00 CET auction, after which a +healthy feed hands over tomorrow in full. + +**The data exists and the endpoint serves it.** Verified 2026-09-10T07:45:49Z: +`?bzn=BE&start=2026-09-09&end=2026-09-10` returns **192 quarter-hours across +two local days**, committed unmodified as +`scenarios/energy-charts-be-price-2026-09-09_2026-09-10-twoday.json`. The +plugin simply never asked. + +After the change, same code path, same window, a feed serving both days: +**39 firm hours and 9 carried**, firm across **two** Brussels days, and the +permitted cycle buys on one day and sells on the other. Graded in +`test_price_history.py` section 1c against both feed states. + +--- + +## 1. The range is derived on the **Brussels** calendar, from `since`/`until` + +`since` and `until` are **instants**; the API takes **dates**. Those are not +the same question, and the conversion is where the day gets lost. + +`brussels_date_range()` returns two inclusive `date`s: + +* the first is `brussels(since).date()`. **Not `since.date()`, and not the UTC + date.** 22:30 UTC in summer is already tomorrow in Belgium, so a UTC-derived + start asks for the day the controller has just left and drops the day it is + standing in — for the last one or two hours of *every* day. This is + `price_history._bdate()`'s lesson one layer up, with a worse consequence: + `_bdate()` misfiles an hour, this never fetches it. +* the last is `brussels(until).date()`, **stepped back a day when `until` + falls exactly on local midnight**, because `until` is exclusive and that + instant is the first tick of a day nobody asked for. Without it every + whole-hour horizon requests one extra day — harmless on this feed, but it + makes the request a claim the caller never made. +* an empty or inverted window collapses to a single day rather than producing + a backwards range, which no feed has a defined answer for. + +Both ends are graded, and both are driven red by mutation — including the two +that swap `brussels()` for the UTC date (`test_price.py` section 2c). + +--- + +## 2. `price_feed_url` is the **exact URL**, used as given. The plugin appends + nothing. + +The alternative — merge our `start`/`end` into whatever query string the +operator supplied — was considered and rejected. Three reasons, in the order +they decided it: + +1. **`start`/`end` are energy-charts' spelling, and this provider declares a + second host.** `generic_dayahead.capabilities().egress_hosts` is + `("api.energy-charts.info", "web-api.tp.entsoe.eu")`. That second host's + day-ahead date parameters have not been verified by this card and are not + guessed at here — but they are not `start`/`end`. So an override to the + *other declared host* — a case the declaration itself anticipates — is + exactly where merging appends two parameters that host ignores, producing + the short horizon anyway, with a URL nobody typed as the receipt. +2. **Merging means re-serialising a URL somebody else wrote.** Parsing and + rebuilding reorders parameters, collapses duplicates and re-escapes values. + "Override the feed" is what the option already says it means, and an + override that is silently rewritten is not one. +3. **Blind appending is not on the table at all.** `dated_feed_url()` refuses + to add a query string to a base that already has one, so + `?bzn=BE?bzn=BE&start=…` cannot be produced by this module even by a later + edit that folds the parameters back into the constant. + +⚠️ **The cost is real and it is therefore NOT a doc footnote.** An overriding +operator gets exactly the short horizon this card exists to fix. So the plugin +emits a `WARNING` on the shipped `goodwe.price` logger at **fetch time**, naming +the consequence — one Brussels day, the rest carried and flagged, §8.3 dropping +them from the trade set — and naming the `start..end` it *would* have asked +for. §11.4 is what makes a price defect quiet; an operator who has taken the +short horizon has to be able to find out from the log that they did. + +The warning is graded, with a control that the default path stays silent, and +a mutation that downgrades it to `debug()` takes that check red. + +**Fetch time rather than startup**, because that is where the URL actually in +use is known. The trade-off is one line per refresh (the tariff task, not the +30 s control tick). If the status page ever grows a price-source row, this is +the sentence it should carry. + +--- + +## 3. The plugin contract: `fetch(self, opts, since, until, transport)` + +The signature promises every plugin the range, and until this card **none of +the four networked plugins used it**. The obvious follow-up is to enforce the +promise. It is **not** enforced, and the reason is TARIFF-08's: + +Three of the five shipped plugins point at endpoints nobody has ever verified +(TARIFF-07 is the open decision on withdrawing them). A rule obliging them to +pass a date range would be satisfied by an **invented parameter name** — which +is the parser-graded-against-its-own-guess defect moved from the response to +the request, and it would ship looking like a guarantee. + +What exists instead is a **roster pin** in `test_price.py` section 2c: which +plugins reference their `since`/`until` is asserted as a literal, derived from +the registry and the AST. A sixth supplier, or a change to any of the three, +has to edit that line and make the decision rather than inherit the silence. +It is named as a source-shape pin, not as evidence that any feed parses; +`generic_dayahead`'s behaviour is graded separately by the URL it really +requests. + +**Recommendation for whoever picks up TARIFF-07:** enforce it at the moment +those three endpoints are either verified or withdrawn, not before. If they are +withdrawn, `generic_dayahead` and `manual_csv` are the whole roster and the +rule is one line — `manual_csv` reads a whole file and is windowed downstream, +so it would be the single documented exemption. diff --git a/goodwe_controller/DOCS.md b/goodwe_controller/DOCS.md index 5b4495b..dc9258d 100644 --- a/goodwe_controller/DOCS.md +++ b/goodwe_controller/DOCS.md @@ -1360,7 +1360,7 @@ does, and only after you have read why. | `price_taxes_eur_kwh` | Levies and VAT, EUR/kWh. Same rule | | `export_price_eur_kwh` | The **injection** price. A separate scalar, not a series — see below | | `price_csv_path` | Where `manual_csv` reads its file | -| `price_feed_url` | Override the plugin's default feed. Only a host the plugin itself declares is accepted | +| `price_feed_url` | Override the plugin's default feed. Used **exactly as written** — the add-on appends nothing to it, including the date range, so an override gets whatever horizon that URL returns. Only a host the plugin itself declares is accepted | | `price_horizon_h` | How far ahead to ask for, hours | | `price_timeout_s` | Per-request timeout | | `static_profile` | A static day/night or fully-fixed tariff. Empty by default — see below | @@ -1408,6 +1408,29 @@ point `price_feed_url` somewhere else: * Belgium moved to a **15-minute market time unit** on 2025-10-01, and the feed followed. A published day is **96 points, not 24**. Nothing in the add-on assumes either spacing, and nothing aggregates the quarters away. +* The request **names the days it wants**, derived from the horizon on the + **Brussels** calendar — `?bzn=BE&start=2026-09-09&end=2026-09-10`, two + inclusive local dates. Without that the feed answers a single day, and + everything past today would be carried forward from yesterday's profile and + flagged provisional — which §8.3 excludes from arbitrage, so tomorrow would + never be tradeable at any hour. The calendar is Brussels rather than UTC + because 22:30 UTC in summer is already tomorrow in Belgium. + +**What you get before the auction publishes.** The day-ahead auction clears at +about 13:00–14:00 CET. Ask for tomorrow before that and the feed returns fewer +days than requested — that is normal, not a failure. The add-on keeps the +series of what exists and fills the remaining hours from yesterday's profile, +flagged provisional; arbitrage runs on the firm hours and ignores the flagged +ones. So a dynamic site's plan is part firm and part carried every morning and +fully firm once tomorrow is published, and neither state is an error. + +⚠️ **If you set `price_feed_url`, you keep the short horizon.** The override is +sent exactly as you wrote it and the add-on adds no `start`/`end` to it, because +those are this feed's parameter names and an override may point somewhere with +different ones — rewriting a URL you typed is how a request nobody intended gets +made. The consequence is a **`WARNING` in the log on every price refresh**, +naming the range that would otherwise have been requested. If you only wanted a +different host and not a different horizon, the fix is to clear the option. If the arrays ever come back different lengths, or the unit stops saying EUR/MWh, the fetch is refused outright rather than trusted in part: a price diff --git a/goodwe_controller/MUTATION.md b/goodwe_controller/MUTATION.md index 9654f29..1a9de8d 100644 --- a/goodwe_controller/MUTATION.md +++ b/goodwe_controller/MUTATION.md @@ -734,6 +734,45 @@ Cheap habits fall out of it, and all of them are now used in this repository: a no-op, and the mutation that proves the rendering is real belongs on `assemble()`, where it is now aimed. +20. ⚠️ **A FIXTURE GRADES THE RESPONSE. NOTHING ABOUT THE REQUEST IS REACHABLE + FROM ONE, AND THE TRANSPORT HAD BEEN RECORDING IT ALL ALONG.** TARIFF-09, + and it is habit 19's closed loop with the other half still shut. + `generic_dayahead.fetch()` took `since` and `until` and asked for the bare + `?bzn=BE`, which answers ONE Brussels day - so §8.3 dropped every hour past + today from the trade set as provisional and the optimiser never saw + tomorrow as firm, at any hour of any day. Measured at 09:00: **15 firm + hours and 33 carried of 48**, both ends of the permitted cycle inside one + calendar day; at 23:00, **1 and 47**, with the firm spread no longer + clearing the round-trip cost. + + TARIFF-08 had just fixed the RESPONSE side of this same plugin by + committing real bytes. It could not have caught this: a captured body is + the answer to a URL, and **no property of any body says which URL was + asked for**. `canned()` in `test_price.py` had recorded `(url, timeout_s)` + on every call since TARIFF-02 and not one check had ever asserted on the + first element - so the harness *knew* the plugin was ignoring its arguments + and nothing asked it. The fix is one line of habit: **where a plugin takes + arguments that shape a request, assert on the REQUEST off the injected + transport, not only on what comes back.** Section 2c does, and the + shipped-defect mutation (restore the bare URL, `-173 B`) is red on six + named checks in `test_price.py` and two in `test_price_history.py`. + + ⚠️ **AND A MUTATION THAT TURNS A READ INTO `no_data` CRASHES A SUITE + INSTEAD OF NAMING A CHECK, UNLESS THE CHECK EXPECTS IT.** `_NoData.points` + raises `Absence` by design, so `derived(...) < derived(...)` over two + absent reads is a `TypeError` and `[p for p in plan.read.points]` is a + traceback - habit 5's shape with an exception in place of a hang. The + mutation that matters here is the "defensive" one somebody writes after + reading this card: *refuse when the feed returned fewer days than we asked + for*. It is exactly what a healthy feed does every morning before the + ~13:00-14:00 CET auction, and switching arbitrage off for it would be a + worse defect than the one being fixed. It is red BY NAME in + `test_price.py` (33 checks, the AC's own line among them) and takes + `test_price_history.py` down at **FCST-04's section 1**, whose `_before` + read is unguarded - a red, but an unnamed one. That is recorded rather than + retrofitted: guarding another card's section is a change to what its + checks report, and it belongs to whoever next touches it. + And the mutation-table consequence: **a variant is not covered because a near-miss of it is.** DEV-02's table listed the *actuator* variant of the per-unit claim defect (`device`/`supervised` filed without `inv`), which the @@ -791,6 +830,7 @@ commit and PR bodies. | **SAFETY-03** reviewer survivor: `abs()` dropped from `clamp_to_rating`'s limit (6aabb83) | **re-verified — still a survivor** | re-run the same way, all 15 green. Also as recorded: unreachable under the `int(100, 5000)` schema. | | **MAINT-05 round 3**, `migrate_maint_keys()` marker gate `if rep.ok:` → `if rep.ok or rep.kept:` (`+12 B`) | **was a survivor — now CLOSED** | measured against head `65cddf3` under the purge, tree mounted `:ro`, `19/19 PASS`. It stamps the schema marker on a migration that LOST state, defeating the older `lost` guard as well as the new `misdirected` one, because every check that reached the marker did so on a store whose `kept` list was empty. Closed by `test_maintenance.py` **9d**; the mutation is now red on three named checks. | | **MAINT-05 round 3**, `migrate_unit_keys()` marker gate `if rep.ok:` → `if rep.ok or rep.kept:` (`+12 B`) | **found here — SAFETY-02's half, now CLOSED** | the SAME mutation one function up, on the LATCH migration rather than the history one. Measured a survivor against this branch under the purge, `:ro`, `19/19 PASS`, then closed by `test_inverter.py` part 10's new fixture — a store with `unit/inverter-a/sign_reason` already present (so `kept` is non-empty) whose `sign_tripped: true` write is dropped. Shipped: marker withheld. Mutant: marker stamped on a **disarmed latch**, and the next start skips the migration that would have re-armed it. | +| **TARIFF-09** 11 mutations over `app/price.py` and the two-day capture, all red, no survivors | **not at risk** | direction, and method: every mutation was applied to a FRESH `cp -a` of the clean worktree rather than reverted, so there is no revert to verify (habit 18) and no shared scratch path two runs could collide on. Each ran through `tools/mutation_run.sh`, which purges and fingerprints. Byte deltas recorded on the PR; one is **+0 B** - a single price digit edited inside the committed capture, the risky delta - and the re-derived sha256 catches it. | | **TARIFF-01**'s other three survivors, **SAFETY-03**'s `describe()` survivor, **TARIFF-02**'s M3/M5/M7, **SAFETY-18**'s three | **still unverified** | recorded before the purge discipline, with no byte delta on the record, so a `+0 B` same-second revert cannot be excluded. All are survivors, which is the direction this defect fabricates. TARIFF-02's and SAFETY-18's were **closed by later commits whose claims are reds**, so what is unverified is the *diagnosis*, not the current state of the code. | **Nothing here says a past result is wrong.** The two survivors re-run above diff --git a/goodwe_controller/app/price.py b/goodwe_controller/app/price.py index 0f4513e..aeb63fd 100644 --- a/goodwe_controller/app/price.py +++ b/goodwe_controller/app/price.py @@ -140,6 +140,7 @@ from dataclasses import dataclass from datetime import datetime, timedelta, timezone from enum import Enum from textwrap import dedent +from urllib.parse import urlencode _LOG = logging.getLogger("goodwe.price") @@ -184,6 +185,87 @@ def brussels(instant: datetime) -> datetime: return instant.astimezone(brussels_offset(instant)) +def brussels_date_range(since: datetime, until: datetime) -> tuple: + """The Brussels calendar days a half-open window [since, until) touches. + + Returns `(first, last)`, two `date`s, BOTH INCLUSIVE - which is the shape + `api.energy-charts.info` takes: `?bzn=BE&start=2026-09-09&end=2026-09-10` + answered 192 quarter-hours across exactly those two local days, measured + 2026-09-10T07:45:49Z and committed as + `scenarios/energy-charts-be-price-2026-09-09_2026-09-10-twoday.json`. + + THE DEFECT THIS EXISTS TO PREVENT (TARIFF-09). `generic_dayahead.fetch()` + accepted `since` and `until` and requested the bare default URL, so the + feed answered ONE Brussels day. Everything past today was then carried + forward from yesterday's profile, flagged provisional, and §8.3 dropped all + of it from the trade set - so the optimiser never saw tomorrow as firm, at + any hour. Measured on the merged tree at 2026-09-10T09:00+02:00 over a 48 h + horizon: 15 firm whole hours and 33 carried, and both ends of the only + permitted cycle inside ONE calendar day. The ceiling was structural: at + hour H of the local day a one-day window gives at most 24-H firm hours and + 24+H carried - 1/47 at 23:00, where the firm spread no longer even clears + the round-trip cost. That is not what FCST-04's fill is for; it covers the + pre-publication window until the ~13:00-14:00 CET auction, after which a + healthy feed hands over tomorrow in full. + + ⚠️ THE CALENDAR IS BRUSSELS, NEVER UTC, AND THIS IS `_bdate()`'s LESSON ONE + LAYER UP. `since`/`until` are INSTANTS; the API takes DATES, and the two are + not the same question. 22:30 UTC in summer is already tomorrow in Belgium, + so a range derived from `since.date()` asks for the wrong FIRST day for the + last one or two hours of every single day - and the day it drops is the one + the controller is standing in. price_history.py made this mistake at the + cache layer and documents the same arithmetic; here it decides what is + fetched at all, so getting it wrong loses the hours rather than misfiling + them. + + ⚠️ `until` IS EXCLUSIVE, SO LOCAL MIDNIGHT BELONGS TO THE DAY BEFORE IT. + A 48 h horizon from local midnight ends at local midnight two days later, + and that instant is the first tick of a day the caller did not ask for. + Taking `brussels(until).date()` unconditionally would request a THIRD day + on every whole-hour horizon. This feed answers such a request without + complaint, so nothing would report it - which is the reason to get it right + here rather than to rely on the server's tolerance: the range is a claim + about what the caller asked for, and it should be true. + + ⚠️ AND AN EMPTY OR INVERTED WINDOW COLLAPSES TO ONE DAY RATHER THAN TO A + BACKWARDS RANGE. `start` after `end` is a request no feed has a defined + answer for; some return everything, some return an error, and neither is a + thing to hand a battery controller. + """ + first = brussels(since).date() + edge = brussels(until) + last = edge.date() + if (edge.hour, edge.minute, edge.second, edge.microsecond) == (0, 0, 0, 0): + last = last - timedelta(days=1) + if last < first: + last = first + return first, last + + +def dated_feed_url(base: str, **params) -> str: + """`base` plus a query string. Refuses a `base` that already carries one. + + ⚠️ THE REFUSAL IS THE POINT, NOT THE FORMATTING. Appending `&start=...` to + a URL that may already hold its own query string is how a plugin produces a + request nobody intended - `?bzn=BE?bzn=BE&start=...`, or a duplicated + parameter whose winner is the server's choice rather than ours. So the base + is required to be parameterless and the parameters are added exactly once, + by `urlencode`, which is also what keeps a date out of the escaping + question. A future edit that folds the query back into the constant is a + refused fetch with a sentence, not a silently malformed URL. + + ⚠️ IT IS NOT REACHED BY AN OPERATOR'S `price_feed_url`. See + `GenericDayAhead.fetch()`: an override is used exactly as given. + """ + if "?" in base or "#" in base: + raise SourceUnavailable( + f"the feed base {base!r} already carries a query string or " + f"fragment, and this builder appends parameters. Adding a second " + f"'?' produces a URL the server parses differently from the one " + f"anybody intended; put the fixed parameters in the call instead") + return f"{base}?{urlencode(params)}" if params else base + + # --------------------------------------------------------------------------- # # Absence, borrowed wholesale from sim/riglock.py (TEST-04). # --------------------------------------------------------------------------- # @@ -731,6 +813,24 @@ class PriceProvider(ABC): `transport(url, timeout_s) -> bytes` is injected so the checks never touch a network and `manual_csv` never needs one. `opts` is a `PluginOptions`, so the markup and taxes are not reachable from here. + + ⚠️ `since`/`until` ARE THE RANGE THE CALLER WANTS, AND A PLUGIN THAT + IGNORES THEM GETS WHATEVER ITS FEED DEFAULTS TO (TARIFF-09). That is + not a small difference: `generic_dayahead` ignored both, the public + EPEX feed answered ONE Brussels day, and §8.3 then dropped every hour + past today from the trade set as provisional - so the optimiser never + saw tomorrow as firm, at any hour of any day, and nothing reported it. + + This promise is NOT mechanically enforced, and the honest reason is + TARIFF-08's: three of the five shipped plugins point at endpoints + nobody has verified (TARIFF-07 is the open decision on withdrawing + them), so nobody knows how those feeds spell a date range. A rule + obliging them to pass one would be satisfied by an INVENTED parameter + name - the parser-graded-against-its-own-guess defect, moved from the + response to the request. What exists instead is a roster pin in + test_price.py: which plugins consume the range and which do not is + asserted as a literal, so a sixth supplier, or a change to one of the + three, forces the decision rather than inheriting the silence. """ @@ -1171,9 +1271,63 @@ class GenericDayAhead(PriceProvider): sources=frozenset({Component.ENERGY}), egress_hosts=("api.energy-charts.info", "web-api.tp.entsoe.eu")) + FEED_BASE = "https://api.energy-charts.info/price" + BZN = "BE" + async def fetch(self, opts, since, until, transport): - url = opts.get("price_feed_url") or \ - "https://api.energy-charts.info/price?bzn=BE" + """Request the range the CALLER asked for, on the Brussels calendar. + + ⚠️ THE ARGUMENTS ARE USED. Until TARIFF-09 this method took `since` and + `until` and asked for the bare `?bzn=BE`, which answers ONE Brussels + day - so a 48 h horizon was 15 firm hours and 33 carried at 09:00, and + 1 firm and 47 carried at 23:00, with the whole trade set inside a + single calendar day at every hour of every day. See + `brussels_date_range()` for the measurement and for why the calendar + is Brussels rather than UTC. + + ⚠️ AN OVERRIDE IS THE EXACT URL AND THIS METHOD APPENDS NOTHING TO IT, + WHICH COSTS THE OVERRIDING SITE THE RANGE. That is a decision, not an + oversight, and the sentence below is why it is not a footnote: + + * `price_feed_url` says "override the plugin's default feed", and an + override that is silently rewritten is not one. Merging parameters + into an operator's query string means parsing and re-serialising a + URL they wrote - which reorders, de-duplicates and re-escapes it, + and can hand the server a request nobody typed. + * `start`/`end` are ENERGY-CHARTS' spelling, and this provider + DECLARES A SECOND HOST: `web-api.tp.entsoe.eu`. Nothing here has + verified that host's day-ahead parameters and nothing here should + guess them - but they are not `start`/`end`, so an override to the + other DECLARED host is exactly the case where appending our + vocabulary adds two parameters that host ignores and yields the + short horizon anyway, with nothing saying so. + * and blind appending is refused outright: `dated_feed_url()` will + not add a query string to a base that has one. + + So the cost is real and it is REPORTED, at fetch time, on the shipped + logger, naming the horizon consequence rather than the substitution. + §11.4 is what makes a price defect quiet; an operator who has taken the + short horizon has to be able to find out from the log that they did. + """ + override = str(opts.get("price_feed_url") or "").strip() + if override: + url = override + _LOG.warning( + "price_feed_url is set, so %s is requesting %s exactly as " + "given and is adding NO date range to it. The public feed " + "answers a single Brussels day to a request that names no " + "range, and every hour past that day is then carried forward " + "from yesterday's profile and flagged provisional - §8.3 " + "drops provisional hours from the trade set, so arbitrage " + "would never see tomorrow as firm. Clear price_feed_url to " + "let this plugin ask for the %s..%s it was given.", + self.id, url, *(d.isoformat() + for d in brussels_date_range(since, until))) + else: + first, last = brussels_date_range(since, until) + url = dated_feed_url(self.FEED_BASE, bzn=self.BZN, + start=first.isoformat(), + end=last.isoformat()) check_egress(url, self.capabilities()) body = await transport(url, float(opts.get("price_timeout_s"))) return _points_from_columnar_json(body, expect_unit="EUR / MWh", diff --git a/goodwe_controller/config.yaml b/goodwe_controller/config.yaml index b7864a5..2ea139a 100644 --- a/goodwe_controller/config.yaml +++ b/goodwe_controller/config.yaml @@ -318,6 +318,14 @@ options: # validate.py warns about exactly that. static_profile: "" price_csv_path: "/share/goodwe_prices.csv" + # ⚠️ AN OVERRIDE IS THE EXACT URL (TARIFF-09). The plugin sends it verbatim + # and appends nothing - including the `start`/`end` date range it derives from + # the horizon for its own feed - so an override gets whatever range that URL + # answers, which for a bare day-ahead endpoint is ONE Brussels day. Every hour + # past it is then carried from yesterday and flagged provisional, and §8.3 + # keeps provisional hours out of the trade set. The plugin logs a WARNING + # naming that consequence on every fetch that uses an override; clearing this + # option is what restores the full horizon. price_feed_url: "" price_horizon_h: 48 price_timeout_s: 10 diff --git a/goodwe_controller/scenarios/energy-charts-be-price-2026-09-09_2026-09-10-twoday.json b/goodwe_controller/scenarios/energy-charts-be-price-2026-09-09_2026-09-10-twoday.json new file mode 100644 index 0000000..ab76514 --- /dev/null +++ b/goodwe_controller/scenarios/energy-charts-be-price-2026-09-09_2026-09-10-twoday.json @@ -0,0 +1 @@ +{"license_info":"CC BY 4.0 (creativecommons.org/licenses/by/4.0) from Bundesnetzagentur | SMARD.de","unix_seconds":[1788904800,1788905700,1788906600,1788907500,1788908400,1788909300,1788910200,1788911100,1788912000,1788912900,1788913800,1788914700,1788915600,1788916500,1788917400,1788918300,1788919200,1788920100,1788921000,1788921900,1788922800,1788923700,1788924600,1788925500,1788926400,1788927300,1788928200,1788929100,1788930000,1788930900,1788931800,1788932700,1788933600,1788934500,1788935400,1788936300,1788937200,1788938100,1788939000,1788939900,1788940800,1788941700,1788942600,1788943500,1788944400,1788945300,1788946200,1788947100,1788948000,1788948900,1788949800,1788950700,1788951600,1788952500,1788953400,1788954300,1788955200,1788956100,1788957000,1788957900,1788958800,1788959700,1788960600,1788961500,1788962400,1788963300,1788964200,1788965100,1788966000,1788966900,1788967800,1788968700,1788969600,1788970500,1788971400,1788972300,1788973200,1788974100,1788975000,1788975900,1788976800,1788977700,1788978600,1788979500,1788980400,1788981300,1788982200,1788983100,1788984000,1788984900,1788985800,1788986700,1788987600,1788988500,1788989400,1788990300,1788991200,1788992100,1788993000,1788993900,1788994800,1788995700,1788996600,1788997500,1788998400,1788999300,1789000200,1789001100,1789002000,1789002900,1789003800,1789004700,1789005600,1789006500,1789007400,1789008300,1789009200,1789010100,1789011000,1789011900,1789012800,1789013700,1789014600,1789015500,1789016400,1789017300,1789018200,1789019100,1789020000,1789020900,1789021800,1789022700,1789023600,1789024500,1789025400,1789026300,1789027200,1789028100,1789029000,1789029900,1789030800,1789031700,1789032600,1789033500,1789034400,1789035300,1789036200,1789037100,1789038000,1789038900,1789039800,1789040700,1789041600,1789042500,1789043400,1789044300,1789045200,1789046100,1789047000,1789047900,1789048800,1789049700,1789050600,1789051500,1789052400,1789053300,1789054200,1789055100,1789056000,1789056900,1789057800,1789058700,1789059600,1789060500,1789061400,1789062300,1789063200,1789064100,1789065000,1789065900,1789066800,1789067700,1789068600,1789069500,1789070400,1789071300,1789072200,1789073100,1789074000,1789074900,1789075800,1789076700],"price":[150.93,151.41,150.99,147.59,153.25,147.26,144.04,141.53,147.79,146.53,143.55,139.93,145.58,139.56,139.11,138.06,138.92,138.52,138.04,138.65,137.29,139.03,144.59,150.34,154.12,174.5,178.43,186.62,199.35,200.21,200.11,197.43,220.07,202.0,196.01,174.03,198.32,167.49,152.54,142.05,161.5,145.45,128.61,100.85,139.18,115.83,94.41,80.39,111.25,89.61,84.99,74.38,80.53,70.58,62.27,57.1,62.24,64.95,68.99,71.19,66.9,87.32,109.0,123.12,68.48,125.53,155.99,161.9,138.77,160.57,167.35,173.0,142.2,172.83,196.19,211.26,205.85,218.77,217.08,222.0,210.98,208.46,208.88,210.02,209.77,202.61,200.27,191.49,194.39,195.04,197.26,190.92,188.39,198.05,203.05,191.84,216.08,216.08,214.92,200.0,223.37,201.43,196.32,190.0,199.53,194.83,180.53,172.23,179.88,175.77,167.1,173.89,163.29,165.99,171.41,178.73,170.27,177.37,187.71,201.07,193.62,207.61,218.27,226.77,227.17,234.05,236.99,229.94,248.93,227.5,216.08,195.67,215.99,195.01,177.1,155.09,174.91,152.71,139.59,123.83,140.77,123.19,114.96,94.5,102.77,96.63,94.57,116.34,114.63,126.82,129.89,123.95,105.0,119.11,121.0,136.79,123.41,135.01,135.0,156.68,146.88,152.16,162.02,181.78,154.76,184.28,194.3,215.76,197.23,234.15,253.03,296.58,345.32,389.16,482.99,559.0,551.11,446.41,358.97,295.9,271.39,268.06,241.43,238.01,239.7,226.41,223.05,211.56,218.21,210.61,197.78,171.79],"unit":"EUR / MWh","deprecated":false} \ No newline at end of file diff --git a/goodwe_controller/scenarios/energy-charts-be-price-captures.json b/goodwe_controller/scenarios/energy-charts-be-price-captures.json index 601b58a..8a5351c 100644 --- a/goodwe_controller/scenarios/energy-charts-be-price-captures.json +++ b/goodwe_controller/scenarios/energy-charts-be-price-captures.json @@ -1,16 +1,16 @@ { - "card": "TARIFF-08", + "card": "TARIFF-08, extended by TARIFF-09", "what": "UNMODIFIED response bodies from the public EPEX BE day-ahead feed, captured so the SHIPPED parser can be replayed against what the source really publishes", - "why": "every price fixture in the suite was built from a `{\"data\": [{\"start\": ..., \"price_eur_mwh\": ...}]}` document that this feed has never published. The suite therefore graded `_points_from_json` against its own assumption - a closed loop that could not notice the assumption was wrong - and §11.4 turned the resulting failure into a silent fall back to the static profile. A captured real body is what breaks that loop, so these files are bytes, not a reconstruction: nothing here is pretty-printed, re-keyed or trimmed.", + "why": "every price fixture in the suite was built from a `{\"data\": [{\"start\": ..., \"price_eur_mwh\": ...}]}` document that this feed has never published. The suite therefore graded `_points_from_json` against its own assumption - a closed loop that could not notice the assumption was wrong - and §11.4 turned the resulting failure into a silent fall back to the static profile. A captured real body is what breaks that loop, so these files are bytes, not a reconstruction: nothing here is pretty-printed, re-keyed or trimmed. TARIFF-09 adds a body from a request that names a DATE RANGE, for the same reason one layer out: the shipped plugin took `since`/`until` and asked for the bare default URL, so the feed answered ONE Brussels day and every hour past today was carried forward and dropped from the trade set as provisional. A two-day body captured from the range request is what makes \"asking for the range gets the range\" checkable against something the plugin did not shape.", "source": "Energy-Charts API (Fraunhofer ISE), api.energy-charts.info", "upstream": "Bundesnetzagentur | SMARD.de", "licence": "CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)", "licence_note": "the BE bidding zone is in the API's CC BY 4.0 list, so redistribution is permitted with attribution. DATA-02 established this against the API's own OpenAPI document before committing `epex-be-dayahead-2025-09-01_2026-09-01.json`; both bodies below carry the same `license_info` string, recorded per file and asserted by the replay test.", "attribution": "Data licensed CC BY 4.0 by Bundesnetzagentur | SMARD.de, retrieved via the Energy-Charts API (Fraunhofer ISE), https://energy-charts.info. Published without changes.", - "retrieved_utc": "2026-09-10T06:35:00+00:00", + "retrieved_utc": "2026-09-10T06:35:00+00:00 (TARIFF-08's two), 2026-09-10T07:45:49+00:00 (TARIFF-09's)", "retrieved_with": "curl, by hand; no code in `app/` fetches these", "shape": "COLUMNAR: parallel `unix_seconds` (UTC epoch seconds) and `price` arrays, plus `unit`, `license_info` and `deprecated`. There is no `data` key and no per-point object.", - "replayed_by": "test_price.py section 2b", + "replayed_by": "test_price.py sections 2b (TARIFF-08) and 2c (TARIFF-09)", "files": { "energy-charts-be-price-2026-09-10-quarterhourly.json": { "url": "https://api.energy-charts.info/price?bzn=BE", @@ -29,7 +29,12 @@ "price", "unit", "unix_seconds" - ] + ], + "also_returned_verbatim_by": { + "url": "https://api.energy-charts.info/price?bzn=BE&start=2026-09-10&end=2026-09-11", + "retrieved_utc": "2026-09-10T07:46:55+00:00", + "note": "TARIFF-09's PRE-PUBLICATION case, and it needed no second file because the answer was BYTE-IDENTICAL to the bare-URL capture above - the same 1877 bytes and the same sha256. Asked for two days at 09:46 CEST, before the ~13:00-14:00 CET auction publishes 2026-09-11, the feed returned the ONE day that exists. That is the short answer this card requires to stay a `series` of what exists rather than becoming `no_data`, and the replay re-derives the identity from the bytes rather than restating it." + } }, "energy-charts-be-price-2025-09-10-hourly.json": { "url": "https://api.energy-charts.info/price?bzn=BE&start=2025-09-10&end=2025-09-10", @@ -49,6 +54,29 @@ "unit", "unix_seconds" ] + }, + "energy-charts-be-price-2026-09-09_2026-09-10-twoday.json": { + "url": "https://api.energy-charts.info/price?bzn=BE&start=2026-09-09&end=2026-09-10", + "retrieved_utc": "2026-09-10T07:45:49+00:00", + "note": "TARIFF-09. A RANGE request, and the counterfactual the ticket is built on: `start`/`end` are INCLUSIVE Brussels calendar dates and this one answers 192 quarter-hours across TWO local days, where the bare `?bzn=BE` of the same endpoint answers 96 across one. The shipped `generic_dayahead` now derives these two dates from the `since`/`until` it is handed.", + "covers_local_days": [ + "2026-09-09", + "2026-09-10" + ], + "granularity_s": 900, + "intervals": 192, + "bytes": 3579, + "sha256": "4d83f5efb4294a4a682562eac619bf35e45c95edde957bfa94c402e1777cc9b3", + "unit_as_returned": "EUR / MWh", + "licence_as_returned": "CC BY 4.0 (creativecommons.org/licenses/by/4.0) from Bundesnetzagentur | SMARD.de", + "deprecated_as_returned": false, + "top_level_keys": [ + "deprecated", + "license_info", + "price", + "unit", + "unix_seconds" + ] } } } diff --git a/goodwe_controller/test_price.py b/goodwe_controller/test_price.py index c0a4975..fbdf04b 100644 --- a/goodwe_controller/test_price.py +++ b/goodwe_controller/test_price.py @@ -52,14 +52,16 @@ import json import logging import os import sys -from datetime import datetime, timedelta, timezone +from datetime import date, datetime, timedelta, timezone +from textwrap import dedent from app.price import ( ALL_IN, Absence, BANNED_IN_PLUGIN_CODE, Component, FrameworkOption, MAX_PLAUSIBLE_EUR_KWH, code_leaks, PRICE_OPTION_DEFAULTS, PROVIDERS, PluginOptions, PricePoint, PriceProduct, PriceProvider, PriceRead, ProviderCapabilities, RawPoint, SourceUnavailable, - arbitrage_allowed, assemble, brussels, brussels_offset, check_egress, + arbitrage_allowed, assemble, brussels, brussels_date_range, + brussels_offset, check_egress, dated_feed_url, price_option, export_price_eur_kwh, load_prices, plan_is_provisional, register, volatility_regime, @@ -500,14 +502,14 @@ _REAL = {n: _capture(n) for n in _CAPTURES["files"]} _bad_sha = sorted(n for n, b in _REAL.items() if hashlib.sha256(b).hexdigest() != _CAPTURES["files"][n]["sha256"]) -check(f"both captured bodies are the bytes the provenance file recorded " - f"{sorted(_REAL)}", - len(_REAL) == 2 and _bad_sha == []) +check(f"all three captured bodies are the bytes the provenance file " + f"recorded {sorted(_REAL)}", + len(_REAL) == 3 and _bad_sha == []) # The document really is columnar, derived from the bytes rather than asserted: # if it ever grows a `data` key this check says so before any parser does. _shapes = {n: sorted(json.loads(b.decode("utf-8"))) for n, b in _REAL.items()} -check(f"...and both are COLUMNAR - parallel arrays, no 'data' key " +check(f"...and every one is COLUMNAR - parallel arrays, no 'data' key " f"{sorted({tuple(v) for v in _shapes.values()})}", all(v == ["deprecated", "license_info", "price", "unit", "unix_seconds"] for v in _shapes.values())) @@ -707,6 +709,275 @@ check(f"the networked plugins are {_networked}; {_replayed} is replayed from a " "totalenergies_dynamic", "eneco_eddy"])) +# --------------------------------------------------------------------------- # +print() +print("2c. TARIFF-09 - the RANGE the caller asked for, on the Brussels " + "calendar") +# +# ⚠️ WHAT WAS WRONG, AND WHY NO CHECK IN THIS FILE COULD SEE IT. +# `generic_dayahead.fetch()` accepted `since` and `until` and requested the +# bare `https://api.energy-charts.info/price?bzn=BE`, which answers ONE +# Brussels day. Everything past today was then filled from yesterday's profile +# by FCST-04, flagged provisional, and §8.3 dropped all of it from the trade +# set - so the optimiser never saw tomorrow as firm, at any hour of any day. +# Measured on the merged tree at 2026-09-10T09:00+02:00 over a 48 h horizon: +# 15 firm whole hours and 33 carried, both ends of the only permitted cycle +# inside ONE calendar day; and at 23:00, 1 firm hour and 47 carried, where the +# firm spread no longer clears the round-trip cost at all. +# +# The reason section 2b could not catch it is that section 2b fed the parser +# BYTES and never looked at the REQUEST. `canned()` has always recorded the URL +# it was asked for and nothing asserted on it - so the transport knew the +# plugin was ignoring its arguments and no check ever asked. The checks below +# assert on the URL, and the two-day body they replay is a capture of what that +# URL really answers. + +_TD = "energy-charts-be-price-2026-09-09_2026-09-10-twoday.json" + +# ⚠️ NON-DEGENERACY FIRST (habit 1): the two-day capture must really cover TWO +# Brussels days and the one-day capture ONE, or "the range reached the feed" +# is graded against two fixtures that are the same shape. +_bdays = {n: sorted({brussels(datetime.fromtimestamp(t, UTC)).date().isoformat() + for t in json.loads(b.decode("utf-8"))["unix_seconds"]}) + for n, b in _REAL.items()} +check(f"the captures span the local-day counts this section needs - " + f"{ {n: len(v) for n, v in _bdays.items()} } - or the range check has " + f"nothing to distinguish {_bdays[_TD]} from {_bdays[_QH]}", + len(_bdays[_TD]) == 2 and len(_bdays[_QH]) == 1 + and len(_bdays[_H1]) == 1) +_npts = {n: len(json.loads(b.decode("utf-8"))["price"]) + for n, b in _REAL.items()} +check(f"...and the two-day body carries 192 quarter-hours where the one-day " + f"body carries 96, which is the counterfactual this card is built on " + f"{_npts}", + len(json.loads(_REAL[_TD].decode("utf-8"))["price"]) == 192 + and len(json.loads(_REAL[_QH].decode("utf-8"))["price"]) == 96) + +# --- the URL the SHIPPED plugin asks for ----------------------------------- # + + +def asked_url(since, until, extra=None): + """The URL `generic_dayahead` really requests for a window. Behaviour, + not source text: it comes back off the injected transport.""" + t = canned(_REAL[_TD]) + o = opts(price_provider="generic_dayahead", **(extra or {})) + run(load_prices(o, since, until, transport=t)) + return t.calls[0][0] + + +# The expectation is built HERE, from the window, by different arithmetic than +# price.py uses: a literal string with the two dates spelled out. +_w0 = datetime.fromisoformat("2026-09-09T09:17:43+02:00") +check(f"a 48 h horizon from 09:17 local asks the feed for the three Brussels " + f"days it touches -> {asked_url(_w0, _w0 + timedelta(hours=48))!r}", + asked_url(_w0, _w0 + timedelta(hours=48)) + == "https://api.energy-charts.info/price?bzn=BE" + "&start=2026-09-09&end=2026-09-11") +check("...and the shipped default no longer asks the bare URL that answers a " + "single day", + asked_url(_w0, _w0 + timedelta(hours=48)) + != "https://api.energy-charts.info/price?bzn=BE") + +# ⚠️ THE BRUSSELS/UTC BOUNDARY, WHICH IS THE TRAP. `since`/`until` are +# INSTANTS and the API takes DATES. At 22:30 UTC in September Belgium is +# already on the NEXT calendar day, so a range derived from `since.date()` - +# or from `since.astimezone(utc).date()`, which is the same thing - asks for +# the day the controller has just left and drops the day it is standing in. +# This is `_bdate()`'s lesson one layer up: price_history.py MISFILES an hour +# when it gets this wrong; here the hours are never fetched at all. +_edge = datetime.fromisoformat("2026-09-10T22:30:00+00:00") +check(f"the precondition: {_edge.isoformat()} really is a different day in " + f"Brussels ({brussels(_edge).date().isoformat()}) than in UTC " + f"({_edge.date().isoformat()}), or the boundary check grades nothing", + brussels(_edge).date().isoformat() != _edge.date().isoformat()) +check(f"...so the range starts on the BRUSSELS day, 2026-09-11, and not on " + f"the UTC day 2026-09-10 -> " + f"{asked_url(_edge, _edge + timedelta(hours=12))!r}", + asked_url(_edge, _edge + timedelta(hours=12)) + == "https://api.energy-charts.info/price?bzn=BE" + "&start=2026-09-11&end=2026-09-11") + +# ...and the SAME trap on the other end. `until` at 22:30 UTC is already +# 00:30 of the next Brussels day, so a UTC-derived END stops one day short and +# the last hours of the horizon are never fetched. Both ends of the range +# derivation are graded, or a mutation only has to get one of them right. +_edge2_a = datetime.fromisoformat("2026-09-09T12:00:00+00:00") +_edge2_b = datetime.fromisoformat("2026-09-10T22:30:00+00:00") +check(f"the precondition: {_edge2_b.isoformat()} is " + f"{brussels(_edge2_b).date().isoformat()} in Brussels and " + f"{_edge2_b.date().isoformat()} in UTC, so the two answers differ", + brussels(_edge2_b).date().isoformat() != _edge2_b.date().isoformat()) +check(f"...so the range ENDS on the Brussels day 2026-09-11, not on the UTC " + f"day 2026-09-10 -> {asked_url(_edge2_a, _edge2_b)!r}", + asked_url(_edge2_a, _edge2_b) + == "https://api.energy-charts.info/price?bzn=BE" + "&start=2026-09-09&end=2026-09-11") + +# ⚠️ `until` IS EXCLUSIVE, so local midnight belongs to the day BEFORE it. A +# 24 h horizon from local midnight is ONE day, not two. +_mid = datetime.fromisoformat("2026-09-09T00:00:00+02:00") +check(f"a horizon ending exactly at local midnight asks for one day, because " + f"`until` is exclusive -> {asked_url(_mid, _mid + timedelta(hours=24))!r}", + asked_url(_mid, _mid + timedelta(hours=24)) + == "https://api.energy-charts.info/price?bzn=BE" + "&start=2026-09-09&end=2026-09-09") +check(f"...while one second past midnight is two days, so the rule is the " + f"exclusive END and not a fixed subtraction -> " + f"{asked_url(_mid, _mid + timedelta(hours=24, seconds=1))!r}", + asked_url(_mid, _mid + timedelta(hours=24, seconds=1)) + == "https://api.energy-charts.info/price?bzn=BE" + "&start=2026-09-09&end=2026-09-10") +check(f"an empty or inverted window collapses to a single day rather than " + f"asking for a backwards range no feed has a defined answer for -> " + f"{asked_url(_mid, _mid - timedelta(hours=5))!r}", + asked_url(_mid, _mid - timedelta(hours=5)) + == "https://api.energy-charts.info/price?bzn=BE" + "&start=2026-09-09&end=2026-09-09") + +# The helper on its own, over the DST transitions, where a fixed-offset +# assumption and a correct one disagree. +_mar = datetime.fromisoformat("2026-03-29T00:30:00+00:00") # 02:30 CEST +_bdr = derived(lambda: brussels_date_range(_mar, _mar + timedelta(hours=1))) +check(f"brussels_date_range() is right across the spring transition, where " + f"00:30 UTC is already 02:30 local -> {_bdr}", + _bdr == (date(2026, 3, 29), date(2026, 3, 29))) + +# --- the two-day body, end to end through the SHIPPED parser --------------- # + +_two = load("generic_dayahead", transport=canned(_REAL[_TD])) +check(f"the shipped generic_dayahead parses the real two-day body " + f"{_two.words}", _two.kind == "series") +def _series_days(read): + return derived(lambda: sorted({brussels(p.start).date().isoformat() + for p in read.points})) + + +_two_days = _series_days(_two) +check(f"...and the series it produces spans TWO Brussels local days " + f"{_two_days}, where the one-day body's spans one " + f"{_series_days(_REPLAY[_QH])}", + _two_days == ["2026-09-09", "2026-09-10"] + and len(_two_days) == 2) +check(f"...one point per published interval, 192 of them, no decimation and " + f"no day dropped ({derived(lambda: len(_two.points))})", + derived(lambda: len(_two.points)) == 192) + +# --- the SHORT ANSWER, which is normal and must stay a series -------------- # +# +# ⚠️ ASKING FOR TOMORROW BEFORE THE ~13:00-14:00 CET AUCTION RETURNS FEWER DAYS +# THAN REQUESTED, AND THAT IS NOT AN ERROR. Measured: at 2026-09-10T07:46:55Z +# the request `?bzn=BE&start=2026-09-10&end=2026-09-11` came back BYTE-IDENTICAL +# to the bare-URL capture - one day, 96 quarter-hours. A `no_data` there would +# switch a working site's arbitrage off every morning, so the requirement is +# that the short body stays a `series` of what EXISTS and FCST-04's fill covers +# the rest with flagged, provisional hours. The identity is re-derived from the +# bytes rather than restated. +_prepub = _CAPTURES["files"][_QH]["also_returned_verbatim_by"] +check(f"the pre-publication range request {_prepub['url']!r} returned the " + f"SAME BYTES as the bare default URL - one day where two were asked for", + hashlib.sha256(_REAL[_QH]).hexdigest() + == _CAPTURES["files"][_QH]["sha256"] + and "start=2026-09-10&end=2026-09-11" in _prepub["url"]) +check(f"...and the shipped parser answers that short body with a SERIES of " + f"what exists, never no_data - {_REPLAY[_QH].words}", + _REPLAY[_QH].kind == "series" + and derived(lambda: len(_REPLAY[_QH].points)) == 96) +# ⚠️ THE COUNTS ARE TAKEN THROUGH `derived()` AND COMPARED ONLY WHEN BOTH ARE +# NUMBERS. A mutation that turns the short answer into `no_data` makes +# `.points` raise `Absence` by design, and `_Raised < _Raised` is a TypeError - +# which takes the file down with a traceback instead of reporting a FAIL. A +# check whose failure mode is a crash has no name attached (habit 5's shape, +# with an exception in place of a hang). +_n_short = derived(lambda: len(_REPLAY[_QH].points)) +_n_two = derived(lambda: len(_two.points)) +check(f"...which is strictly fewer points than the two-day body " + f"({_n_short} < {_n_two}), so 'short' is a real property of this " + f"stimulus and not a name for it", + isinstance(_n_short, int) and isinstance(_n_two, int) + and _n_short < _n_two) + +# --- price_feed_url: the exact URL, and the cost said out loud ------------- # +# +# ⚠️ THE DECISION. An override is used VERBATIM and this plugin appends +# nothing to it. Merging our `start`/`end` into an operator's query string +# would mean re-serialising a URL they wrote, and `start`/`end` are +# energy-charts' spelling - and this provider also DECLARES a SECOND host, +# `web-api.tp.entsoe.eu`, whose date parameters are not those and are not +# verified here, so appending our vocabulary there yields two ignored +# parameters and the short horizon anyway. The cost is +# that an overriding site silently gets the horizon this card exists to fix, +# so it is NOT silent: a warning names the consequence at fetch time. +_ovr = "https://api.energy-charts.info/price?bzn=BE&somethingelse=1" +check(f"an override is requested EXACTLY as given, with nothing appended -> " + f"{asked_url(_w0, _w0 + timedelta(hours=48), {'price_feed_url': _ovr})!r}", + asked_url(_w0, _w0 + timedelta(hours=48), + {"price_feed_url": _ovr}) == _ovr) + +_obuf = io.StringIO() +_oh = logging.StreamHandler(_obuf) +logging.getLogger("goodwe.price").addHandler(_oh) +asked_url(_w0, _w0 + timedelta(hours=48), {"price_feed_url": _ovr}) +_ovr_log = _obuf.getvalue() +_obuf2 = io.StringIO() +_oh2 = logging.StreamHandler(_obuf2) +logging.getLogger("goodwe.price").removeHandler(_oh) +logging.getLogger("goodwe.price").addHandler(_oh2) +asked_url(_w0, _w0 + timedelta(hours=48)) +_dflt_log = _obuf2.getvalue() +logging.getLogger("goodwe.price").removeHandler(_oh2) +check(f"...and the operator is TOLD what it costs them, at fetch time, naming " + f"the horizon consequence rather than only the substitution", + "price_feed_url" in _ovr_log and _ovr in _ovr_log + and "provisional" in _ovr_log and "2026-09-11" in _ovr_log) +# ⚠️ THE CONTROL. Without it, "warns about everything" passes the line above. +check("...while the DEFAULT path says nothing, so the warning is about the " + "override and not about fetching", + "price_feed_url is set" not in _dflt_log) + +# The builder refuses to append to a base that already has a query string, +# which is the "URL nobody intended" this decision is meant to make +# unreachable. Driven, not read off the source. +check(f"dated_feed_url() refuses a base that already carries a query string, " + f"rather than producing a second '?' - " + f"{_reason(lambda: dated_feed_url('https://h/p?a=1', b='2'))!r}", + raises(lambda: dated_feed_url("https://h/p?a=1", b="2"), + SourceUnavailable)) +check("...and a clean base still builds, so the refusal is the existing query " + "string and not the builder", + dated_feed_url("https://h/p", b="2") == "https://h/p?b=2") + +# --- the plugin contract: who consumes the range, pinned ------------------- # +# +# ⚠️ THIS IS A SOURCE-SHAPE PIN AND IS NAMED AS ONE (habit 4). The claim is +# about the CODE - which plugins reference the range arguments they are handed +# - which is the same kind of claim `code_leaks()` makes, and it is not +# evidence that any feed parses. `generic_dayahead`'s BEHAVIOUR is graded above +# by the URL it really requests. +# +# It exists because `fetch(self, opts, since, until, transport)` promises every +# plugin the range and the promise is NOT enforced. Enforcing it would oblige +# the three plugins whose endpoints nobody has verified (TARIFF-07) to invent a +# date-parameter vocabulary - TARIFF-08's defect moved from the response to the +# request. So the split is pinned instead, and a sixth supplier has to change +# this line rather than inherit the silence. +def _uses_range(cls): + src = dedent(inspect.getsource(cls.fetch)) + tree = ast.parse(src) + names = {n.id for n in ast.walk(tree) if isinstance(n, ast.Name)} + return bool(names & {"since", "until"}) + + +_uses = sorted(pid for pid, cls in PROVIDERS.items() if _uses_range(cls)) +check(f"the plugins that CONSUME their `since`/`until` are {_uses}; the rest " + f"take the range and ignore it, which is what TARIFF-09 fixed for " + f"generic_dayahead and what TARIFF-07 has to decide for the others", + _uses == ["generic_dayahead"]) +check("...and `manual_csv` is honestly among the ones that do not: it reads a " + "whole file and the fill windows it downstream, so a range parameter " + "would have nowhere to go", + "manual_csv" in sorted(set(PROVIDERS) - set(_uses))) + + # --------------------------------------------------------------------------- # print() print("3. all-in: energy + markup + taxes, added exactly once (AC1, trap 1)") diff --git a/goodwe_controller/test_price_history.py b/goodwe_controller/test_price_history.py index 55dac90..97032ad 100644 --- a/goodwe_controller/test_price_history.py +++ b/goodwe_controller/test_price_history.py @@ -315,8 +315,12 @@ def transport_for(body): return _t +# ⚠️ NO `price_feed_url` (TARIFF-09). An override is used VERBATIM and the +# plugin appends no date range to it, so a CFG that sets one - even to the +# feed's own address - would exercise the override path in every check in this +# file and never the shipped default. The override has its own grading in +# test_price.py section 2c. CFG = {"price_provider": "generic_dayahead", - "price_feed_url": "https://api.energy-charts.info/price?bzn=BE", "price_timeout_s": 1, "price_markup_eur_kwh": MARKUP, "price_taxes_eur_kwh": TAXES, @@ -517,6 +521,220 @@ check("...and the plan is still a provider plan, not a §11.4 fallback: the " _after.origin == "provider" and _after.product is PriceProduct.DAY_AHEAD) +# --------------------------------------------------------------------------- # +print() +print("1c. TARIFF-09 - the feed is now ASKED for the horizon, and tomorrow " + "comes back FIRM") +# +# ⚠️ WHAT SECTION 1 COULD NOT SEE, AND WHY. Everything above grades the FILL +# against a feed that published one day, and it was right to: FCST-04 covers +# the window between midnight and the ~13:00-14:00 CET auction. But +# `generic_dayahead.fetch()` accepted `since`/`until` and requested the bare +# `?bzn=BE`, which answers one Brussels day AT EVERY HOUR OF EVERY DAY - so the +# one-day feed was not the pre-publication state, it was the ONLY state, and +# §8.3 dropped every hour past today from the trade set as provisional. The +# optimiser never saw tomorrow as firm. +# +# Measured on the merged tree (4257e38) at 2026-09-10T09:00+02:00 over a 48 h +# horizon, live feed, yesterday seeded: 15 firm whole hours and 33 carried, +# and BOTH ends of the permitted cycle inside 2026-09-10. At 23:00 it was 1 +# firm and 47 carried, and the firm spread no longer cleared the round-trip +# cost at all. The ceiling was structural - at hour H a one-day window gives at +# most 24-H firm hours - which is why no amount of carrying could fix it. +# +# The two plans below differ in ONE thing: what the feed answered. The code +# path, the window, the store and the instant are identical, so the difference +# is the horizon and nothing else (habit 7). + +TOMORROW_D = "2026-01-16" +TOMORROW_H = bday_hours(TOMORROW_D) + +# ⚠️ TOMORROW'S PRICES MUST NOT OVERLAP TODAY'S OR YESTERDAY'S, or a point's +# day of origin is not recoverable from its number and every claim below is +# about a coincidence. Asserted, not assumed. +TOMORROW_MWH = {h: 300.0 + h for h in range(24)} +TOMORROW_MWH[18] = 900.0 # the dear evening TOMORROW: the global peak of + # the 48 h window, and it is on the second day + +PRICE_AT = {u: TODAY_MWH[brussels(u).hour] for u in TODAY_H} +PRICE_AT.update({u: TOMORROW_MWH[brussels(u).hour] for u in TOMORROW_H}) + + +def body_at(instants): + """A columnar body priced PER INSTANT, so two days can differ at the same + local hour. `feed_body()` keys on the local hour alone and cannot.""" + return json.dumps({ + "unix_seconds": [int(u.timestamp()) for u in instants], + "price": [PRICE_AT[u] for u in instants], + "unit": "EUR / MWh", + "license_info": "CC BY 4.0 (creativecommons.org/licenses/by/4.0) " + "from Bundesnetzagentur | SMARD.de", + "deprecated": False}).encode() + + +def recording(body): + """A transport that serves bytes and remembers the URL it was asked for.""" + seen = [] + + async def _t(url, timeout_s): + seen.append(url) + return body + _t.seen = seen + return _t + + +_vals_today = {TODAY_MWH[h] for h in range(24)} +_vals_tmrw = {TOMORROW_MWH[h] for h in range(24)} +_vals_yday = {YESTERDAY_MWH[h] for h in range(24)} +check(f"the three days' price tables share no value at all " + f"({len(_vals_today)}/{len(_vals_tmrw)}/{len(_vals_yday)} distinct), so " + f"which day a point came from is recoverable from the point", + not (_vals_today & _vals_tmrw) and not (_vals_tmrw & _vals_yday) + and not (_vals_today & _vals_yday)) +check(f"...and both fixture days are 24 Brussels hours long " + f"({len(TODAY_H)}/{len(TOMORROW_H)}), found by search rather than by " + f"adding 24 UTC hours", + len(TODAY_H) == 24 and len(TOMORROW_H) == 24) + +# THE TWO FEED STATES, one code path. +_t_two = recording(body_at(TODAY_H + TOMORROW_H)) +_h_two = store() +_ph.PriceHistory._remember(_h_two, series_of(YDAY_H, YESTERDAY_MWH), NOW) +_pub = run(resolve_tariff(CFG, NOW, _until, transport=_t_two, now=NOW, + history=_h_two)) + +_t_one = recording(body_at(TODAY_H)) +_h_one = store() +_ph.PriceHistory._remember(_h_one, series_of(YDAY_H, YESTERDAY_MWH), NOW) +_pre = run(resolve_tariff(CFG, NOW, _until, transport=_t_one, now=NOW, + history=_h_one)) + +# ⚠️ THE REQUEST, WHICH IS THE THING THIS CARD CHANGED. Both calls ask for the +# same three Brussels days; only the answer differs. The expected dates are +# spelled out here rather than derived from the module that builds them. +check(f"the shipped plugin asks the feed for the Brussels days the horizon " + f"touches, on both calls -> {_t_two.seen}", + _t_two.seen == _t_one.seen + == ["https://api.energy-charts.info/price?bzn=BE" + "&start=2026-01-15&end=2026-01-17"]) +check("...and it is no longer the bare URL that answers a single day, which " + "is what made the one-day state permanent rather than pre-publication", + "start=" in _t_two.seen[0] and "end=" in _t_two.seen[0]) + + +def split(plan): + """(firm, carried), and ([], []) for a plan that observed no series. + + ⚠️ THE EMPTY ANSWER IS DELIBERATE. A mutation that turns the short + pre-publication answer into `no_data` makes `plan.read.points` raise + `Absence` by design, and an unguarded read here would take this file down + with a traceback rather than reporting a named FAIL - "the suite is red" + and "the check that watches this is red" are different claims + (MUTATION.md, reporting a mutation, point 3). With this guard the counts + below go red BY NAME, which is what a mutation report can cite. + """ + if not plan.has_prices: + return [], [] + firm = [p for p in plan.read.points if not p.provisional] + carried = [p for p in plan.read.points if p.provisional] + return firm, carried + + +def local_days(points): + return sorted({brussels(p.start).strftime("%Y-%m-%d") for p in points}) + + +_f2, _c2 = split(_pub) +_f1, _c1 = split(_pre) + +# The expected counts are re-derived from the fixture by set arithmetic: the +# horizon hours the feed printed, versus the horizon hours it did not. +_window = [SINCE_HOUR + timedelta(hours=n) for n in range(HORIZON_H)] +_two_pub_utc = {u.astimezone(UTC) for u in TODAY_H + TOMORROW_H} +_one_pub_utc = {u.astimezone(UTC) for u in TODAY_H} +_want_f2 = len([u for u in _window if u.astimezone(UTC) in _two_pub_utc]) +_want_f1 = len([u for u in _window if u.astimezone(UTC) in _one_pub_utc]) +check(f"WITH TOMORROW PUBLISHED the 48 h plan is {len(_f2)} firm hours and " + f"{len(_c2)} carried; with only today, {len(_f1)} firm and {len(_c1)} " + f"carried. Re-derived from the fixture: {_want_f2} and {_want_f1}", + len(_f2) == _want_f2 and len(_f1) == _want_f1 + and len(_f2) + len(_c2) == HORIZON_H + and len(_f1) + len(_c1) == HORIZON_H + and len(_f2) > len(_f1)) +check(f"...and the firm hours now span TWO Brussels local days " + f"{local_days(_f2)}, where the one-day answer spans exactly one " + f"{local_days(_f1)}. This is the acceptance criterion, and the ONE thing " + f"no amount of carry-forward could ever produce: a carried hour is " + f"provisional by construction", + len(local_days(_f2)) == 2 and len(local_days(_f1)) == 1 + and local_days(_f2) == [TODAY_D, TOMORROW_D]) + +# ⚠️ THE TAIL IS STILL CARRIED, WHICH IS FCST-04 STILL DOING ITS JOB. A 48 h +# horizon asked for at 09:17 runs 9 hours into a THIRD Brussels day, and no +# day-ahead market publishes that far. The fix widens what is firm; it does not +# claim the market publishes more than it does. +check(f"FCST-04 still fills the remainder: the {len(_c2)} hours past the " + f"market's own horizon are carried from yesterday and flagged, on a " + f"third Brussels day {local_days(_c2)} - so the plan still covers all " + f"{HORIZON_H} hours", + len(_c2) > 0 and local_days(_c2) == ["2026-01-17"] + and all(p.provisional for p in _c2)) +check("...and the plan is still a provider plan reporting itself provisional, " + "not a §11.4 fallback", + _pub.origin == "provider" and _pub.provisional) + +# ⚠️ THE PRE-PUBLICATION ANSWER IS A SERIES OF WHAT EXISTS, NOT `no_data`. +# Asking for tomorrow before the auction returns fewer days than requested. +# A `no_data` there would switch a working site's arbitrage off every morning. +check(f"the short answer stays a SERIES of what exists - {_pre.read.kind}, " + f"{len(_pre.read.points)} hours, origin {_pre.origin!r} - and is never " + f"no_data and never a truncated series presented as complete " + f"({len(_c1)} of its hours are flagged provisional)", + _pre.read.kind == "series" and _pre.origin == "provider" + and len(_pre.read.points) == HORIZON_H and len(_c1) > 0) + +# --- the trade set, which is where the whole thing lands ------------------- # +_A9 = dict(peak_guard_active=False, min_spread_eur_kwh=0.0, + uncertainty=frozenset()) +_v2 = _pub.arbitrage(**_A9) +_v1 = _pre.arbitrage(**_A9) +check("both plans still ALLOW a trade, so the two verdicts below are compared " + "against each other and not against a refusal", + _v2.kind == "allowed" and _v1.kind == "allowed") +_v2 = _v2 if _v2.kind == "allowed" else None +_v1 = _v1 if _v1.kind == "allowed" else None + +# ⚠️ NON-DEGENERACY, DERIVED. The global cheapest firm hour must be on TODAY +# and the global dearest on TOMORROW, or "the trade set spans two days" is a +# property of the arithmetic rather than of the fix. +def _day_of(v, end): + """The Brussels day of one end of a permitting verdict, or "" for a + refusal - so a mutation that turns the verdict into a refusal is a named + FAIL below rather than an `Absence` traceback.""" + return "" if v is None else brussels(getattr(v, end).start).strftime( + "%Y-%m-%d") + + +_lo2, _hi2 = _day_of(_v2, "cheapest"), _day_of(_v2, "dearest") +_lo1, _hi1 = _day_of(_v1, "cheapest"), _day_of(_v1, "dearest") +check(f"WITH TOMORROW FIRM the trade set contains hours from BOTH days: " + f"buy on {_lo2}, sell on {_hi2}", + sorted({_lo2, _hi2}) == [TODAY_D, TOMORROW_D]) +check(f"...where the one-day answer's buy AND sell are both inside {_lo1} - " + f"a trade set that is a subset of ONE calendar day, structurally, which " + f"is the defect this card is about", + _lo1 == _hi1 == TODAY_D) +_s2 = None if _v2 is None else _v2.spread_eur_kwh +_s1 = None if _v1 is None else _v1.spread_eur_kwh +check(f"...and the cross-day spread is strictly larger ({_s2} vs {_s1} " + f"EUR/kWh), so the second day is not merely visible but PRICED", + _s2 is not None and _s1 is not None and _s2 > _s1) +check("...and both ends of the permitted cycle are FIRM, which " + "`_firm_or_die()` asserts at construction and §8.3 requires", + _v2 is not None and not _v2.cheapest.provisional + and not _v2.dearest.provisional) + + # --------------------------------------------------------------------------- # print() print("2. AC2 - the spread stopped being taken over hours that have gone") diff --git a/goodwe_controller/test_static_tariff.py b/goodwe_controller/test_static_tariff.py index b1a0fa6..163a68a 100644 --- a/goodwe_controller/test_static_tariff.py +++ b/goodwe_controller/test_static_tariff.py @@ -743,8 +743,10 @@ def run(coro): return asyncio.run(coro) +# ⚠️ NO `price_feed_url` (TARIFF-09): an override is used verbatim and the +# plugin appends no date range to it, so setting one - even to the feed's own +# address - exercises the override path rather than the shipped default. _cfg = opts(TWO_PERIOD, price_provider="generic_dayahead", - price_feed_url="https://api.energy-charts.info/price?bzn=BE", price_timeout_s=1, price_markup_eur_kwh=0.03, price_taxes_eur_kwh=0.06) _fb = run(resolve_tariff(_cfg, THU, THU + timedelta(days=1), transport=_dead, diff --git a/goodwe_controller/translations/en.yaml b/goodwe_controller/translations/en.yaml index 3b60ace..5509a9e 100644 --- a/goodwe_controller/translations/en.yaml +++ b/goodwe_controller/translations/en.yaml @@ -161,7 +161,7 @@ configuration: description: "For price source manual_csv only: where in /share the add-on reads your own price file from." price_feed_url: name: "Tariff · Price feed URL override" - description: "Leave empty to use the chosen source's own feed. Only a host that source itself declares is accepted, so this cannot be pointed anywhere." + description: "Leave empty to use the chosen source's own feed. Only a host that source itself declares is accepted, so this cannot be pointed anywhere. WHAT YOU TYPE IS SENT EXACTLY AS WRITTEN: the add-on adds nothing to it, not even the date range it normally asks its own feed for, so an override gets whatever horizon that URL returns - often a single day, with the rest of the plan carried over from yesterday and excluded from price arbitrage. A warning in the log says so on every price refresh. Clear this option to get the full horizon back." price_horizon_h: name: "Tariff · How far ahead to fetch prices (hours)" description: "How many hours of prices to ask the source for." diff --git a/goodwe_controller/translations/nl.yaml b/goodwe_controller/translations/nl.yaml index 34dc627..99343f4 100644 --- a/goodwe_controller/translations/nl.yaml +++ b/goodwe_controller/translations/nl.yaml @@ -141,7 +141,7 @@ configuration: description: "Alleen voor prijsbron manual_csv: waar in /share de add-on uw eigen prijsbestand leest." price_feed_url: name: "Tarief · Afwijkende URL voor de prijsfeed" - description: "Laat leeg om de eigen feed van de gekozen bron te gebruiken. Alleen een host die die bron zelf opgeeft, wordt aanvaard, zodat dit niet zomaar ergens naar kan verwijzen." + description: "Laat leeg om de eigen feed van de gekozen bron te gebruiken. Alleen een host die die bron zelf opgeeft, wordt aanvaard, zodat dit niet zomaar ergens naar kan verwijzen. WAT U HIER TYPT WORDT LETTERLIJK VERSTUURD: de add-on voegt er niets aan toe, ook niet het datumbereik dat ze normaal aan haar eigen feed vraagt. Een afwijkende URL levert dus de horizon op die die URL zelf teruggeeft - vaak één dag, waarna de rest van het plan uit de prijzen van gisteren wordt overgenomen en buiten de prijsarbitrage valt. Bij elke prijsvernieuwing verschijnt daarover een waarschuwing in het logboek. Maak dit veld leeg om de volledige horizon terug te krijgen." price_horizon_h: name: "Tarief · Hoe ver vooruit prijzen ophalen (uren)" description: "Hoeveel uur aan prijzen er bij de bron opgevraagd wordt."