Merge pull request 'TARIFF-14: the window a point covers is the point's, not an assumed hour' (#80) from TARIFF-14 into release/1.0
_Series.at() hardcoded a one-hour coverage window and returned the FIRST point whose start fell inside it. Belgium moved to a 15-minute MTU on 2025-10-01 and the live feed returns 96 points at 900 s, so on the real series at() answered with the wrong point for 45 of every 60 minutes - and then drifted, because from the second hour on the first match was an earlier point still inside its own one-hour window. Reproduced against shipped code: asking for 03:00 returned the 02:15 point. It reached the trade decision directly. arbitrage._act() calls _point_at(plan, now) -> series.at(now), then tests here.start == cheapest.start / == dearest.start to decide whether this interval is the buy or the sell one. With a stale and drifting point that identity test almost never matches the extremum, so the strategy idled through exactly the interval it exists to trade. It also tested here.provisional and the section 17 negative-price guard on the wrong point's price. Nothing caught it because no test drove at() on a sub-hourly series: the DST tests count points on 23- and 25-hour days but never ask for a quarter-hour, and every price fixture before TARIFF-08 was the invented hourly row shape. static_tariff.py emits hourly points, so static sites are immune - the defect was reachable only by the dynamic path TARIFF-08/09 had just made work. Each point's window now runs to the next point's start, CAPPED AT ONE HOUR. The cap is not cosmetic: uncapped, the rule would have stretched the point before a three-hour FCST-04 hole across the whole hole, introducing hole-filling as a side effect of fixing a lookup. The last point takes the span of the point before it, because a flat one-hour tail answers at(00:30) tomorrow with today's 23:45 price - reaching for a neighbour across a day boundary, which section 11.4 forbids. That tail choice was independently validated by TARIFF-12, which merged during this work. A served plan can now be up to TARIFF_MAX_AGE_S (5400 s) old, and price_horizon_h permits a 1 h horizon - so now can walk past the end of a short series. Under the shipped rule at() raises Absence and the site falls to the static profile; under the rejected flat tail the same stale plan would have gone on trading against an expired :45 quarter for up to 45 minutes. Re-read the rule if TARIFF_MAX_AGE_S is ever widened. covers() shared the bug exactly - it has no rule of its own and delegates to at() - so one change fixes both. window() did not: it grades a start against a half-open range and never asks how long a point lasts. at() now depends on sortedness AND ESTABLISHES IT ITSELF, sorting once in _Series.__init__, because nothing else guaranteed it: _validate_raw() covers only the load_prices() path, _parse_points() only the store read, and PriceRead.series() never enforced anything. A series handed its points reversed now answers identically, as a check. Evidence: 29/29 green on the merged tree. Eight mutations, all red. The killing one restores the shipped rule (+68 B) and is red on 24 named checks across both files, including the one named THE PROPERTY THE DEFECT BROKE - an arbitrage-level check driving the shipped decide() over a 96-point day whose cheapest is a :45 and dearest a :15, asserting as its own check that the old and new rules return different points. The tail mutation is a ZERO-BYTE edit and is caught. It is stated honestly that five of the eight leave test_arbitrage.py green, because those are at()-level properties no arbitrage fixture reaches. Named and not fixed: price_forecast._walk() steps section 8.3's gate in whole hours, so on a sub-hourly series it reads one point per hour and never sees the :15/:30/:45 flags. Unreachable today - FCST-04 carries whole hours and the feed publishes whole days - but that is an assumption about producers, not a property of the gate.
This commit was merged in pull request #80.
This commit is contained in:
@@ -19,6 +19,7 @@ if it has, it needs 0.10.0. Whoever deploys decides.
|
||||
| _this branch_ | TARIFF-09 - the day-ahead feed is asked for the horizon it was given, so tomorrow arrives firm instead of carried |
|
||||
| _this branch_ | TARIFF-10 - configuring a price provider is what fetches prices; trading stays its own switch |
|
||||
| _this branch_ | TARIFF-12 - a supplier outage stops deleting the plan on its first failed fetch, and TARIFF_MAX_AGE_S starts governing |
|
||||
| _this branch_ | TARIFF-14 - the price a trade is decided against is the price of the interval being traded |
|
||||
|
||||
### TARIFF-10 - a configured provider is fetched, whether or not you trade
|
||||
|
||||
@@ -184,6 +185,91 @@ driven, and a mutation that makes the guard depend on `arbitrage.enabled()` is
|
||||
red on five named checks.
|
||||
|
||||
|
||||
### TARIFF-14 - the trade was decided against the wrong price
|
||||
|
||||
**On the only price feed that works, `_Series.at()` returned the wrong point,
|
||||
and the error grew with depth into the series.** It asked
|
||||
`s <= u < s + 1 h` of every point **in list order** and returned the **first**
|
||||
match. On the hourly shape every fixture in this repository carried before
|
||||
TARIFF-08 that is right by accident; Belgium moved to a **15-minute MTU on
|
||||
2025-10-01** and the live day-ahead feed returns **96 points at 900 s**, where
|
||||
it is wrong twice over. Measured against the shipped code at `5059e28`:
|
||||
|
||||
```
|
||||
series: 02:00=0.10 02:15=0.11 02:30=0.12 02:45=0.13 03:00=0.14 ...
|
||||
|
||||
ask + 0min -> 02:00 0.10
|
||||
ask +15min -> 02:00 0.10 <-- WRONG (wanted 02:15)
|
||||
ask +45min -> 02:00 0.10 <-- WRONG
|
||||
ask +60min -> 02:15 0.11 <-- WRONG (wanted 03:00)
|
||||
ask +75min -> 02:30 0.12 <-- WRONG (wanted 03:15)
|
||||
```
|
||||
|
||||
Inside the first hour the `:00` point wins because its hour has not run out.
|
||||
**From the second hour on the answer is not even the top of the hour** - the
|
||||
first point still inside its own hour wins - so the error compounds.
|
||||
|
||||
**What it cost is not "a slightly wrong price".** `arbitrage._act()` tests
|
||||
`here.start == v.cheapest.start` / `== v.dearest.start` to decide whether this
|
||||
interval is the buy one or the sell one. That identity almost never held, so
|
||||
**the strategy idled through exactly the interval it exists to trade**,
|
||||
reporting *"this hour is neither the buy hour nor the sell hour"* - and read
|
||||
§8.3's `here.provisional` and §17's `price < 0` off the same wrong point.
|
||||
`static_tariff.py` emits hourly points, so **static sites were never affected**;
|
||||
this was reachable only down the dynamic path TARIFF-08/09 made work.
|
||||
|
||||
**The rule now, in one sentence: a point covers from its own start until the
|
||||
next point's start, and never more than one hour.**
|
||||
|
||||
| | before | after |
|
||||
|---|---|---|
|
||||
| 900 s | `at(02:15)` -> the 02:00 point | -> the 02:15 point |
|
||||
| 3600 s | `at(02:30)` -> the 02:00 point | **unchanged** |
|
||||
| **mixed** `[900, 3600]` | each point one hour wide | **each point takes its own gap** |
|
||||
|
||||
The mixed case is not an edge case: FCST-04's carry-forward emits **hourly
|
||||
carried points beside quarter-hourly firm ones**, and the soak measured spacing
|
||||
`[900, 3600]` in **70 % of samples**. A window derived from one global spacing
|
||||
would be wrong in the normal steady state.
|
||||
|
||||
**The one-hour cap is load-bearing.** FCST-04 leaves an hour *out* of the
|
||||
series when nothing published it and yesterday has no value either, and
|
||||
`at()` raising `Absence` for that hour is what sends the caller to the static
|
||||
profile (§11.4). Uncapped, "until the next point's start" would stretch the
|
||||
point before a three-hour hole across the whole hole and answer every instant
|
||||
in it with a stale price - filling holes, introduced by the fix for a different
|
||||
defect.
|
||||
|
||||
**The last point's window is a decision and it is written down.**
|
||||
`DECISIONS/TARIFF-14-the-window-a-point-covers.md` §2 argues it: the last point
|
||||
is given **the same span as the point before it** (one hour when it is the only
|
||||
point). A flat one-hour tail would answer **00:30 tomorrow with today's 23:45
|
||||
price**, which is reaching for a neighbour across a day boundary. The cost of
|
||||
the choice - a mixed series ending at the firm/carried seam loses its last 45
|
||||
minutes to `Absence` - is stated there rather than buried, and the direction is
|
||||
the safe one.
|
||||
|
||||
**`at()` now reads a point's neighbours, so it would depend on the series being
|
||||
sorted - and it establishes that itself** rather than trusting the three
|
||||
producers that happen to guarantee it today. The lookup index is sorted by UTC
|
||||
start once, at construction; a series handed its 96 points in reverse order
|
||||
answers identically, which is a check. `covers()` shared the defect exactly
|
||||
(it has no rule of its own - it asks `at()`) and is fixed by the same change;
|
||||
`window()` did not (it grades a point's **start** against a range and asks
|
||||
nothing about duration), and its docstring now says so.
|
||||
|
||||
**Nothing about §9.3 or a point's `start` changed.** `PricePoint` still has
|
||||
three fields; the span is derived from the neighbours, not stored.
|
||||
|
||||
Graded by `test_price.py` section 14 - 900 s, 3600 s, **mixed**, holes, the
|
||||
tail, caller order, `covers()`/`window()` - and end to end by
|
||||
`test_arbitrage.py` section 10, which drives `decide()` over a 96-point
|
||||
quarter-hourly day whose cheapest and dearest quarters are deliberately **not**
|
||||
on a whole hour. Eight mutations, all red, including a **zero-byte** one on the
|
||||
tail rule; the table with byte deltas and the checks that were NOT reached is
|
||||
in the decision note §6.
|
||||
|
||||
|
||||
### TARIFF-09 - the feed is asked for the range the caller wanted
|
||||
|
||||
**`generic_dayahead` took `since` and `until` and requested the bare feed URL,
|
||||
|
||||
@@ -0,0 +1,349 @@
|
||||
# TARIFF-14 — how long one price point covers, and what the last one does
|
||||
|
||||
One defect and one genuinely open decision. The defect had no defensible
|
||||
alternative once it was measured; the decision — the **last** point's window —
|
||||
did, and the alternative is argued below rather than dismissed.
|
||||
|
||||
---
|
||||
|
||||
## 0. The measurement this card exists for
|
||||
|
||||
`_Series.at()` asked `s <= u < s + 1 h` of every point **in list order** and
|
||||
returned the **first** match. Run against the shipped code at `5059e28` on a
|
||||
quarter-hourly series:
|
||||
|
||||
```
|
||||
series: 02:00=0.10 02:15=0.11 02:30=0.12 02:45=0.13 03:00=0.14 ...
|
||||
|
||||
ask + 0min -> 02:00 0.10
|
||||
ask +15min -> 02:00 0.10 <-- WRONG (wanted 02:15)
|
||||
ask +30min -> 02:00 0.10 <-- WRONG
|
||||
ask +45min -> 02:00 0.10 <-- WRONG
|
||||
ask +60min -> 02:15 0.11 <-- WRONG (wanted 03:00)
|
||||
ask +75min -> 02:30 0.12 <-- WRONG (wanted 03:15)
|
||||
```
|
||||
|
||||
**Two failure modes, not one.** Inside the first hour the `:00` point wins
|
||||
because its hour has not run out. From the second hour on the answer is not
|
||||
even the top of the hour — the first point still inside its own hour wins — so
|
||||
**the error compounds with depth into the series**.
|
||||
|
||||
Same code, same run, after the fix:
|
||||
|
||||
```
|
||||
ask + 0min -> 02:00 0.10 ask +60min -> 03:00 0.14
|
||||
ask +15min -> 02:15 0.11 ask +75min -> 03:15 0.15
|
||||
ask +30min -> 02:30 0.12 ask +90min -> 03:30 0.16
|
||||
ask +45min -> 02:45 0.13 ask+120min -> 04:00 0.18
|
||||
```
|
||||
|
||||
An hourly series is **unchanged at every instant**, before and after — which is
|
||||
the whole reason nothing caught this. Every price fixture in this repository
|
||||
before TARIFF-08 was the invented hourly row shape, `static_tariff.py` emits
|
||||
hourly points, and the DST checks count points on the 23- and 25-hour days
|
||||
without ever asking `at()` for a quarter-hour. **The suite had been exercising
|
||||
the one resolution on which the broken rule and the right one agree.**
|
||||
|
||||
Belgium moved to a 15-minute MTU on 2025-10-01 and the live day-ahead feed
|
||||
returns **96 points at 900 s**, so this is reachable only down the dynamic path
|
||||
TARIFF-08/09 just made work. Static sites are immune.
|
||||
|
||||
## 0.1 What it cost, which is not "a slightly wrong price"
|
||||
|
||||
`arbitrage._act()` calls `_point_at(plan, now)` → `series.at(now)` and then
|
||||
tests `here.start == v.cheapest.start` / `== v.dearest.start` to decide whether
|
||||
this interval is the buy one or the sell one. That is an **identity test
|
||||
between the matched point and the point the verdict was built from**, and on a
|
||||
quarter-hourly series it almost never matched — so the strategy idled through
|
||||
exactly the interval it exists to trade, reporting *"this hour is neither the
|
||||
buy hour nor the sell hour"*. It also read `here.provisional` (§8.3) and §17's
|
||||
`price < 0` off that same wrong point.
|
||||
|
||||
---
|
||||
|
||||
## 1. The rule
|
||||
|
||||
> **A point covers from its own start until the next point's start, and never
|
||||
> more than one hour.**
|
||||
|
||||
* **900 s** → each point covers its own quarter.
|
||||
* **3600 s** → each point covers its own hour. Identical to what shipped.
|
||||
* **MIXED** → **each point takes its own gap.** FCST-04's carry-forward emits
|
||||
hourly carried points beside quarter-hourly firm ones, and the soak measured
|
||||
spacing `[900, 3600]` in **70 % of samples** — so a window derived from one
|
||||
global spacing would be wrong in the **normal steady state**, not in an edge
|
||||
case.
|
||||
|
||||
### 1.1 Why the one-hour cap is not decoration
|
||||
|
||||
FCST-04 **leaves holes on purpose**: an hour with no published price and no
|
||||
yesterday value is left *out* of the series, and `price_forecast.py`'s module
|
||||
note names `at()` raising `Absence` for it as the behaviour that sends the
|
||||
caller to the static profile (§11.4 for that hour).
|
||||
|
||||
Uncapped, "until the next point's start" stretches the point **before** a
|
||||
three-hour hole across the whole hole and answers every instant in it with a
|
||||
stale price. Filling holes is the defect this module exists to refuse, and it
|
||||
would have been introduced **by the fix for a different one**. Graded in
|
||||
`test_price.py` 14d; the mutation that removes the cap is red there.
|
||||
|
||||
---
|
||||
|
||||
## 2. The decision: what the **last** point covers
|
||||
|
||||
There is no next start, and §9.3 says nothing about how wide the final interval
|
||||
is — a `PricePoint` has **three fields** and must not learn a fourth, because
|
||||
that is the wire format every store, replay, fixture and captured body in this
|
||||
repository is written against. So the tail is a **choice**, and both candidates
|
||||
are defensible.
|
||||
|
||||
**Chosen: the last point is given the same span as the point before it** (one
|
||||
hour when it is the only point).
|
||||
|
||||
| | flat one hour | the span in front of it (**chosen**) |
|
||||
|---|---|---|
|
||||
| 96-point 900 s day, `at(23:50)` | answers 23:45 | answers 23:45 |
|
||||
| the same day, `at(00:30)` **tomorrow** | **answers with today's 23:45 price** | `Absence` |
|
||||
| 24-point hourly day, `at(23:50)` | answers 23:00 | answers 23:00 |
|
||||
| mixed series ending at the firm/carried seam | answers for an hour | `Absence` after 15 min |
|
||||
|
||||
**The second row is why.** A flat one-hour tail reaches across a day boundary
|
||||
and prices tomorrow morning with yesterday's last print — which is
|
||||
"reaching for a neighbouring one", the single thing the `Absence` message has
|
||||
always refused to do. The chosen rule never extends a point past a span the
|
||||
series itself demonstrates.
|
||||
|
||||
### 2.1 The cost of the choice, stated rather than buried
|
||||
|
||||
On a **mixed** series whose last point is a carried **hourly** one sitting
|
||||
behind quarter-hourly firm ones, the tail is **900 s**, and the last 45 minutes
|
||||
of the horizon raise `Absence` for an hour something did carry a value for.
|
||||
|
||||
That is accepted, not tolerated, and the direction is the safe one:
|
||||
|
||||
* the caller falls back to the static profile (§11.4) instead of trading on a
|
||||
span the series never demonstrated;
|
||||
* the point is **carried**, so §8.3 forbids any irreversible action resting on
|
||||
it regardless;
|
||||
* it is at the far end of a 48 h horizon, where nothing is decided at 30 s.
|
||||
|
||||
Under-covering costs a missed decision. Over-covering prices a real interval
|
||||
with a number that does not describe it. Those are not symmetric.
|
||||
|
||||
### 2.2 The degenerate case
|
||||
|
||||
A series of **one** point has no gap in front of it to copy and gets **one
|
||||
hour** — the shipped behaviour, kept deliberately. `test_price.py` 14e states
|
||||
it as its own check so it cannot decay into an accident.
|
||||
|
||||
---
|
||||
|
||||
## 3. Does `at()` now depend on sortedness? Yes — and it establishes it itself
|
||||
|
||||
The old rule read one point at a time and did not care about order. The new one
|
||||
reads a point's **neighbours**, so it *would* depend on the series being
|
||||
ascending.
|
||||
|
||||
It does not, because **the lookup index is sorted by UTC start at construction**
|
||||
(`_cover_table()`, called once from `_Series.__init__`). No caller's list order
|
||||
can reach `at()`.
|
||||
|
||||
That matters because the three things that "guarantee" order today each
|
||||
guarantee it somewhere else:
|
||||
|
||||
| | what it guarantees | where |
|
||||
|---|---|---|
|
||||
| `_validate_raw()` | strictly increasing starts | the `load_prices()` path only |
|
||||
| `price_history._parse_points()` | re-sorts by instant | the store read path only |
|
||||
| FCST-04's fill | emits ascending | that one producer only |
|
||||
| `PriceRead.series()` | **nothing** | every other caller, including tests |
|
||||
|
||||
Three producers agreeing today is not an invariant; it is a coincidence with
|
||||
good references. `test_price.py` 14f drives a series handed its 96 points in
|
||||
**reverse** order and asserts `at()` answers identically at every one of them —
|
||||
red the moment the sort is removed.
|
||||
|
||||
**Duplicated starts** (refused by `_validate_raw()`, constructible by hand) give
|
||||
the earlier list entry a zero-width window and the later one the real span. One
|
||||
of the pair becomes unreachable through `at()`; **nothing raises**, at
|
||||
construction or on the 30 s control path. Stated as a check rather than left to
|
||||
be discovered.
|
||||
|
||||
---
|
||||
|
||||
## 4. `covers()` shared it. `window()` did not
|
||||
|
||||
* **`covers()`** has no window rule of its own — it calls `at()` — so it shared
|
||||
the defect exactly and the same change fixes it. It is kept as a delegation
|
||||
for that reason: a second copy of the rule is a second place for it to be
|
||||
wrong.
|
||||
* **`window(since, until)`** grades a point's **start** against a half-open
|
||||
range and never asks how long a point lasts, so it carries no resolution
|
||||
assumption at all. It is **not** a coverage query — a point that starts before
|
||||
`since` and runs into it is absent from its result by design. That is now
|
||||
said in its docstring and pinned by a check, because the next reader will ask.
|
||||
It has **no callers** in `app/`.
|
||||
|
||||
---
|
||||
|
||||
## 5. What this card does NOT fix, with its shape
|
||||
|
||||
`price_forecast._walk()` — §8.3's irreversible-action gate — iterates
|
||||
`committed_hours()`, which steps in **whole hours**, and asks `series.at(u)` at
|
||||
each. On a quarter-hourly series that now reliably returns the hour's `:00`
|
||||
point (it previously returned something earlier and arbitrary), so the gate
|
||||
reads **one point per hour** and never looks at the `:15`/`:30`/`:45` flags.
|
||||
|
||||
A partially-provisional hour would therefore be gated on its first quarter
|
||||
alone. **Two things keep it off every shipped install, and both are
|
||||
circumstantial rather than structural:** `price_permits()`/`begin_phase()` have
|
||||
**no caller in `app/main.py`** — the §12 price-justified entry point is built
|
||||
and not yet wired — and FCST-04 carries **whole** hours while
|
||||
`generic_dayahead` publishes whole days, so the four quarters of an hour are
|
||||
firm or carried together anyway. Both are facts about the *callers and
|
||||
producers*, not properties of the *gate*, so this is written down here rather
|
||||
than left for the first sub-hourly feed or the first wiring-up to find. Fixing it means teaching
|
||||
`committed_hours()` about resolution, which is a data-model question this card
|
||||
deliberately does not open.
|
||||
|
||||
---
|
||||
|
||||
## 6. Mutations
|
||||
|
||||
Eight, each purged before the run (TEST-15), each reverted with the tree
|
||||
fingerprint checked afterwards (MUTATION.md habit 18), backup at a `mktemp`
|
||||
path per invocation.
|
||||
|
||||
⚠️ **Re-driven in full on the MERGED tree** after TARIFF-10 landed on
|
||||
`release/1.0` (base `88c59a9` -> `559f0eb`). All eight give the same verdicts
|
||||
and the same red-check counts as before the merge, which is the answer that
|
||||
matters: TARIFF-10 added 650 lines of checks to `test_price.py` and none of
|
||||
them moves under any of these edits, because they grade `resolve_tariff()`'s
|
||||
*reachability from a configuration* and not the lookup. Clean `app/price.py` on
|
||||
the merged tree = **`90d8ca31f7c0109d`, 85953 B** (pre-merge it was
|
||||
`30d1a56f0117e571`, 83742 B; the difference is TARIFF-10's `provider_id()` /
|
||||
`provider_configured()`, which this card does not touch).
|
||||
|
||||
| | mutation | Δ | red in |
|
||||
|---|---|---|---|
|
||||
| M1 | `at()` back to the shipped fixed one-hour scan over `self.points` | +68 B | **24 named checks**: `test_price.py` 14b ×4, 14c ×3, 14d, 14e ×3, 14f, 14g; `test_arbitrage.py` 10b ×6, 10c ×3, 10d ×2 |
|
||||
| M2 | drop the one-hour cap | −68 B | `test_price.py` 14d |
|
||||
| M3 | flat one-hour tail instead of the span in front | **+0 B** | `test_price.py` 14e ×3, 14g — 4 checks |
|
||||
| M4 | trust the caller's list order — drop the sort | −61 B | `test_price.py` 14f |
|
||||
| M5 | `covers()` grows its own hour-wide rule | +95 B | `test_price.py` 14e, 14g |
|
||||
| M6 | `window()` becomes a coverage query | −37 B | `test_price.py` 14g |
|
||||
| M7 | the window's end becomes inclusive | +1 B | `test_price.py` 14b ×4, 14c ×4, 14d, 14e ×4, 14f ×2, 14g — 16 checks |
|
||||
| M8 | the window's start becomes exclusive | −1 B | `test_price.py` 14b ×4, 14c ×4, 14d, 14f ×2, 14g, 14h, plus the pre-existing section 6 check — 14 checks |
|
||||
|
||||
**M3 is the zero-byte one and it is the tail decision itself** — the case
|
||||
MUTATION.md names as the risky size, caught by the check written for it.
|
||||
|
||||
**What the arbitrage file does NOT see, with sizes.** M2, M4, M5, M6 and M8
|
||||
leave `test_arbitrage.py` green, and that is honest rather than a gap to round
|
||||
off: they are `at()`-level properties (holes, caller order, `covers()`,
|
||||
`window()`, the start boundary) that no arbitrage fixture reaches, because the
|
||||
probes are seven minutes into a slot and the plans have no holes. The one that
|
||||
matters end to end — M1, the shipped rule — is red in **both** files, on the
|
||||
named check `THE PROPERTY THE DEFECT BROKE`.
|
||||
|
||||
---
|
||||
|
||||
## 7. TARIFF-10 landed on the base while this was open — what it changes here
|
||||
|
||||
Base moved `88c59a9` → `559f0eb` mid-card. The question is not whether the
|
||||
branches merge; it is whether TARIFF-10 makes any sentence above **false**.
|
||||
|
||||
**In `app/price.py` the two do not meet.** TARIFF-10 added `provider_id()` and
|
||||
`provider_configured()` beside `price_option()` and routed `load_prices()`
|
||||
through them. That is the option-reading layer; it touches nothing in
|
||||
`_Series`, `at()`, `covers()`, `window()` or `_cover_table()`. Git merged the
|
||||
file without a conflict, and the merged `at()` is byte-identical to the one the
|
||||
mutations above were driven against.
|
||||
|
||||
**Behaviourally it widens who has a series at all, and that cuts one way only.**
|
||||
`run_tariff_refresh()` now runs on installs with `arbitrage_enabled` **off**, so
|
||||
a provider-configured site has a standing plan — and therefore a `_Series` —
|
||||
where before it had none. **It does not add a new `at()` consumer to the 30 s
|
||||
tick**, which is the claim worth checking rather than assuming:
|
||||
|
||||
| caller of `at()`/`covers()` | reached by a non-trading install? |
|
||||
|---|---|
|
||||
| `arbitrage._point_at()` (`arbitrage.py:300`) | no — trading only |
|
||||
| `static_tariff.TariffPlan.price_at()` / `.words` `here` (`:750`, `:819`) | on the tick these are reached through `static_status()`, which builds a **`static_plan`** — hourly points, immune. `main.py` calls only `plan.words` on the fetched plan, which counts points and never looks one up |
|
||||
| `price_forecast._walk()` (`:562`, `:568`) | **`price_permits()` and `begin_phase()` have no caller in `app/main.py` at all** — §12's price-justified entry point is built and not yet wired |
|
||||
|
||||
So TARIFF-10 neither creates nor closes any exposure this card is about. What it
|
||||
does do is make the defect's reach larger the moment such a site arms trading,
|
||||
because the series it would trade against now exists on every configured
|
||||
install rather than only on armed ones. That strengthens the case for the fix
|
||||
and changes nothing in its reasoning.
|
||||
|
||||
**In `test_price.py` both cards appended a section 13.** Nothing is dropped:
|
||||
TARIFF-10's stays **13**, this card's is renumbered **14** (which is also this
|
||||
card's number), and the temp-directory teardown TARIFF-10 put at the end of the
|
||||
file stays at the end of the file. The two sections share no fixture, no option
|
||||
dict and no helper name; TARIFF-10's grades `resolve_tariff()`'s *reachability
|
||||
from a configuration* and drives real fetches with a counting transport, and
|
||||
**none of its checks moves under any of the eight mutations above** — measured,
|
||||
not assumed.
|
||||
|
||||
---
|
||||
|
||||
## 8. TARIFF-12 landed too — a plan may now be 90 minutes old when it is asked
|
||||
|
||||
Base moved `559f0eb` → `a6e64fb` while this was open. TARIFF-12 stops a
|
||||
returned §11.4 degradation from overwriting a standing plan that is still
|
||||
inside `TARIFF_MAX_AGE_S`, so **a served plan can now be up to
|
||||
`3 × TARIFF_REFRESH_S = 5400 s` old** where before it was at most one refresh
|
||||
interval. `at()` is what answers *"what does it cost right now"* out of that
|
||||
older plan, so the question is fair and the answer is written out rather than
|
||||
assumed.
|
||||
|
||||
**Per-point span: unaffected, structurally.** Every window is derived from the
|
||||
neighbouring points' **absolute `start` instants**. Those are `datetime`s inside
|
||||
a frozen dataclass inside a tuple — they cannot move, and `_cover_table()` runs
|
||||
once at construction over data that cannot change afterwards. A plan's *age* is
|
||||
a property of `fetched_at` / `tariff_at`, never of a window. **The index cannot
|
||||
go stale**, which is a small positive under TARIFF-12 rather than a risk: a kept
|
||||
plan now lives longer, and there is nothing in it that needed recomputing.
|
||||
|
||||
**The one-hour cap: unaffected.** It bites only where a gap exceeds an hour,
|
||||
i.e. at a hole FCST-04 left. Walking further into a series crosses more windows;
|
||||
each is still resolved on its own neighbours.
|
||||
|
||||
**The tail: reachable for the first time, and the choice made in §2 is the one
|
||||
that survives it.** This is the half worth spelling out.
|
||||
|
||||
* At the **shipped 48 h horizon** the tail is unreachable by staleness. A plan
|
||||
is served for at most 90 minutes, so `now` is at most 90 minutes into a
|
||||
~48 h series — the last point is still about 46½ hours away.
|
||||
* But `price_horizon_h` is `int(1,72)` in `config.yaml`, so an operator **can**
|
||||
configure a **1 h** horizon. A 90-minute serve-stale window against a 1 h
|
||||
series puts `now` genuinely past the end. That is not hypothetical; it is a
|
||||
supported configuration meeting a supported outage.
|
||||
* Under the **chosen** rule a quarter-hourly 1 h series ends at `:45` and that
|
||||
point's window closes at the top of the next hour: `at()` raises `Absence`,
|
||||
`_point_at()` returns None, `_act()` idles with its "no price for this
|
||||
instant" reason, and §11.4 sends the site to the static profile.
|
||||
* Under the **rejected** flat one-hour tail the same stale plan would have gone
|
||||
on trading against the `:45` print for **up to 45 minutes after the interval
|
||||
it describes had ended** — pricing an outage with a number for a quarter that
|
||||
is over.
|
||||
|
||||
So TARIFF-12 does not weaken the tail decision. It is the first thing that makes
|
||||
it reachable on a shipped configuration, and it independently selects the same
|
||||
answer. ⚠️ **Note which direction that runs**: the more stale a plan is allowed
|
||||
to be, the more the tail rule is doing real work, so it should be re-read if
|
||||
`TARIFF_MAX_AGE_S` is ever widened again.
|
||||
|
||||
**`TariffPlan.from_feed` cannot produce a series this code has not seen.** It is
|
||||
a pure predicate on `origin` (`== "provider"`), and TARIFF-12's decision keeps or
|
||||
replaces a **whole plan object**. It never splices two reads into one series, so
|
||||
it cannot hand `at()` an unsorted or duplicated point list — the case §3 covers
|
||||
defensively and that nothing shipped produces.
|
||||
|
||||
**Re-graded on this base.** All 29 files green. `app/price.py` is byte-identical
|
||||
to the TARIFF-10 merge (`90d8ca31f7c0109d`, 85953 B) because TARIFF-12 does not
|
||||
touch it. **M1** re-driven: +68 B, red on the same **24 named checks** across
|
||||
both files. **M3** re-driven: **+0 B**, red on the same 4. This branch
|
||||
contributes **zero diff** to `app/main.py` and `app/static_tariff.py`.
|
||||
@@ -541,6 +541,121 @@ class PriceRead(_Outcome):
|
||||
return _NoData(reason)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# How long ONE point covers - TARIFF-14
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
# ⚠️ A POINT DOES NOT CARRY ITS OWN DURATION AND MUST NOT LEARN TO. §9.3 lists
|
||||
# THREE fields and `PricePoint` has three; a fourth would change the wire
|
||||
# format every store, replay, fixture and captured body in this repository is
|
||||
# written against. The span is DERIVED from the neighbours instead - here,
|
||||
# once, at construction.
|
||||
MAX_POINT_SPAN = timedelta(hours=1)
|
||||
|
||||
|
||||
def _cover_table(points) -> tuple:
|
||||
"""`(start_utc, end_utc, point)` per point, ascending. The `at()` index.
|
||||
|
||||
THE DEFECT THIS PREVENTS (TARIFF-14, HIGH). `at()` asked
|
||||
`s <= u < s + 1 h` of every point IN LIST ORDER and returned the FIRST
|
||||
match. On an hourly series - the only resolution any fixture carried
|
||||
before TARIFF-08 - that is right by accident. Belgium moved to a
|
||||
15-minute MTU on 2025-10-01 and the live day-ahead feed returns 96 points
|
||||
at 900 s, where it is wrong twice over. Measured on shipped `5059e28`:
|
||||
|
||||
series: 02:00=0.10 02:15=0.11 02:30=0.12 02:45=0.13 03:00=0.14
|
||||
|
||||
at(02:15) -> 02:00 the :00 point's HOUR has not run out yet
|
||||
at(03:15) -> 02:30 and from the second hour on it is not even the
|
||||
top of the hour: the first point still inside
|
||||
its own hour wins, so the error COMPOUNDS with
|
||||
depth into the series
|
||||
|
||||
`arbitrage._act()` compares `here.start` against `v.cheapest.start` and
|
||||
`v.dearest.start` to decide whether this interval is the buy or the sell
|
||||
one. That identity test almost never matched the extremum, so the
|
||||
strategy idled through exactly the interval it exists to trade - and read
|
||||
`here.provisional` and §17's `price < 0` guard off the wrong point while
|
||||
doing it. `static_tariff.py` emits hourly points, so static sites never
|
||||
saw it; this is reachable only down the dynamic path TARIFF-08/09 made
|
||||
work.
|
||||
|
||||
THE RULE, in one sentence:
|
||||
|
||||
A POINT COVERS FROM ITS OWN START UNTIL THE NEXT POINT'S START, AND
|
||||
NEVER MORE THAN ONE HOUR.
|
||||
|
||||
* 900 s -> each point covers its own quarter, so `at(02:15)` is the
|
||||
02:15 point;
|
||||
* 3600 s -> each point covers its own hour, unchanged from shipped;
|
||||
* MIXED -> EACH POINT TAKES ITS OWN GAP. FCST-04's carry-forward emits
|
||||
HOURLY carried points beside QUARTER-HOURLY firm ones, and the soak
|
||||
measured spacing `[900, 3600]` in 70 % of samples - so mixed is the
|
||||
normal steady state, and a window derived from one global spacing
|
||||
would be wrong THERE rather than in an edge case.
|
||||
|
||||
⚠️ WHY THE ONE-HOUR CAP, WHICH IS NOT DECORATION. FCST-04 LEAVES HOLES ON
|
||||
PURPOSE: an hour with no published price AND no yesterday value is left
|
||||
OUT of the series, and `price_forecast.py`'s module note names `at()`
|
||||
raising `Absence` for it as the behaviour that sends the caller to the
|
||||
static profile (§11.4 for that hour). Uncapped, "until the next point's
|
||||
start" would stretch the point BEFORE a three-hour hole across the whole
|
||||
hole and answer every instant in it with a stale price. Filling holes is
|
||||
the defect this module exists to refuse, and it would have been
|
||||
introduced by the fix for a different one.
|
||||
|
||||
⚠️ THE LAST POINT'S WINDOW IS DECIDED, NOT INHERITED. There is no next
|
||||
start and nothing in §9.3 says how wide the final interval is, so THE
|
||||
LAST POINT IS GIVEN THE SAME SPAN AS THE POINT BEFORE IT - one hour when
|
||||
it is the only point. Three consequences, all intended:
|
||||
|
||||
* on a 96-point 900 s day the 23:45 point covers 23:45 .. 00:00, so
|
||||
`at(23:50)` ANSWERS and `at(00:00)` raises `Absence`. A flat one-hour
|
||||
tail would have answered 00:30 TOMORROW with today's last price,
|
||||
which is reaching for a neighbour across a day boundary;
|
||||
* on an hourly series the tail is an hour, exactly as it shipped, so no
|
||||
static or hourly site changes behaviour;
|
||||
* on a MIXED series whose last point is a carried HOURLY one behind
|
||||
quarter-hourly firm ones, the tail is 900 s and the last 45 minutes
|
||||
of the horizon raise `Absence`. That is the SAFE direction and it is
|
||||
accepted rather than tolerated: the caller falls back to the static
|
||||
profile (§11.4) instead of trading on a span the series never
|
||||
demonstrated, and a carried point is provisional, so §8.3 forbids any
|
||||
irreversible action resting on it regardless.
|
||||
|
||||
⚠️ ORDER IS ESTABLISHED HERE, NOT ASSUMED - and that is the difference
|
||||
between this and the rule it replaces. `at()` now reads neighbours, so it
|
||||
WOULD depend on sortedness; the table is therefore sorted by UTC start at
|
||||
construction, and no caller's list order can reach it. `_validate_raw()`
|
||||
rejects out-of-order and duplicate starts on the `load_prices()` path,
|
||||
`price_history._parse_points()` re-sorts what it reads off disk, and
|
||||
FCST-04's fill emits ascending - but `PriceRead.series()` itself has
|
||||
never enforced any of that, and an assumption that holds today and is
|
||||
unstated is how this class of defect recurs.
|
||||
|
||||
⚠️ NOTHING HERE RAISES ON A MALFORMED SERIES. A duplicated start - not
|
||||
reachable through `load_prices()`, which refuses it - gives the earlier
|
||||
list entry a ZERO-width window and the later one the real span, so one of
|
||||
the pair becomes unreachable through `at()` and neither the 30 s control
|
||||
path nor construction raises.
|
||||
"""
|
||||
rows = sorted(((p.start.astimezone(timezone.utc), i, p)
|
||||
for i, p in enumerate(points)),
|
||||
key=lambda r: (r[0], r[1]))
|
||||
out = []
|
||||
for n, (start, _i, point) in enumerate(rows):
|
||||
if n + 1 < len(rows):
|
||||
span = rows[n + 1][0] - start # -> the next point's start
|
||||
elif n:
|
||||
span = start - rows[n - 1][0] # -> the tail, see above
|
||||
else:
|
||||
span = MAX_POINT_SPAN # -> a series of one point
|
||||
if span > MAX_POINT_SPAN:
|
||||
span = MAX_POINT_SPAN
|
||||
out.append((start, start + span, point))
|
||||
return tuple(out)
|
||||
|
||||
|
||||
class _Series(PriceRead):
|
||||
kind = "series"
|
||||
|
||||
@@ -564,6 +679,10 @@ class _Series(PriceRead):
|
||||
self.provider = provider
|
||||
self.fetched_at = fetched_at
|
||||
self.regime = volatility_regime(product)
|
||||
# ⚠️ TARIFF-14. Built ONCE, here, so that `at()` scans an index whose
|
||||
# order it established itself rather than one it hopes it was handed.
|
||||
# See `_cover_table()` for the window rule and for what the tail does.
|
||||
self._cover = _cover_table(self.points)
|
||||
self.words = (f"{len(self.points)} points from {provider} "
|
||||
f"({product.value}, {self.regime} thresholds)")
|
||||
|
||||
@@ -572,25 +691,47 @@ class _Series(PriceRead):
|
||||
return {"prices": [p.canonical() for p in self.points]}
|
||||
|
||||
def at(self, instant: datetime) -> PricePoint:
|
||||
"""The point covering `instant`. Raises `Absence` if the series does not.
|
||||
"""The point whose OWN interval covers `instant`, at ANY resolution.
|
||||
|
||||
⚠️ RETURNS A POINT OR RAISES; it never returns None. `series.at(t) or
|
||||
fallback` is the same defaulting bug one level down, so the falsy
|
||||
answer does not exist. Ask `covers()` first if you mean to branch.
|
||||
|
||||
⚠️ TARIFF-14: THE WINDOW IS NOT ONE HOUR WIDE ANY MORE, AND WAS NEVER
|
||||
ENTITLED TO BE. It runs to the next point's start, capped at an hour -
|
||||
`_cover_table()` above holds the rule, the tail decision and the
|
||||
reproduction on a quarter-hourly series. A fixed `s + 1 h` here
|
||||
returned the FIRST point still inside its own hour, which on the live
|
||||
900 s feed is neither the covering point nor, past the first hour,
|
||||
even the top of the hour.
|
||||
|
||||
A linear scan over the index, which is 192 entries on the shipped 48 h
|
||||
horizon. This runs on the 30 s control path through
|
||||
`arbitrage._point_at()`; a bisect would be correct and is not worth
|
||||
the second way to be wrong at this size.
|
||||
"""
|
||||
u = instant.astimezone(timezone.utc)
|
||||
for p in self.points:
|
||||
s = p.start.astimezone(timezone.utc)
|
||||
if s <= u < s + timedelta(hours=1):
|
||||
return p
|
||||
for start, end, point in self._cover:
|
||||
if start <= u < end:
|
||||
return point
|
||||
raise Absence(
|
||||
f"no price point covers {u.isoformat()}; the series runs "
|
||||
f"{self.points[0].start.isoformat()} .. "
|
||||
f"{self.points[-1].start.isoformat()}. An hour outside the series "
|
||||
f"is an hour with no price, which is §11.4 for that hour - not a "
|
||||
f"reason to reach for a neighbouring one.")
|
||||
f"{self.points[-1].start.isoformat()}. An instant outside the "
|
||||
f"series - or inside a hole FCST-04 left because nothing "
|
||||
f"published that hour - is an instant with no price, which is "
|
||||
f"§11.4 for it, not a reason to reach for a neighbouring one.")
|
||||
|
||||
def covers(self, instant: datetime) -> bool:
|
||||
"""Whether `at(instant)` would answer.
|
||||
|
||||
⚠️ SHARED TARIFF-14 EXACTLY, AND IS FIXED BY THE SAME CHANGE. It has
|
||||
no window rule of its own - it asks `at()` - so on the 900 s feed it
|
||||
answered True for instants no point covered and False for none of
|
||||
them, because the hour-wide window is never SHORTER than the truth.
|
||||
It is kept as a delegation for that reason: a second copy of the rule
|
||||
is a second place for it to be wrong.
|
||||
"""
|
||||
try:
|
||||
self.at(instant)
|
||||
return True
|
||||
@@ -598,7 +739,17 @@ class _Series(PriceRead):
|
||||
return False
|
||||
|
||||
def window(self, since: datetime, until: datetime) -> tuple:
|
||||
"""Points whose start falls in [since, until). May legitimately be empty."""
|
||||
"""Points whose START falls in [since, until). May legitimately be empty.
|
||||
|
||||
⚠️ DID NOT SHARE TARIFF-14, AND THE REASON IS WORTH STATING RATHER
|
||||
THAN REDISCOVERING. This grades a point's `start` against a half-open
|
||||
range and never asks how long a point lasts, so it carries no
|
||||
resolution assumption to be wrong about - the same call answers the
|
||||
same set at 900 s, at 3600 s and mixed. It is therefore NOT a
|
||||
coverage query: a point that starts before `since` and runs into it
|
||||
is absent from this tuple by design. `at()`/`covers()` are the
|
||||
coverage questions.
|
||||
"""
|
||||
a = since.astimezone(timezone.utc)
|
||||
b = until.astimezone(timezone.utc)
|
||||
return tuple(p for p in self.points
|
||||
|
||||
@@ -1695,6 +1695,312 @@ check("soc_bounds() returns the shipped pair, floor first",
|
||||
float(PRICE_OPTION_DEFAULTS["arbitrage_soc_ceiling"])))
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
print()
|
||||
print("10. TARIFF-14 - the QUARTER-HOURLY day, end to end through decide()")
|
||||
# ⚠️ THIS IS THE PROPERTY THE DEFECT ACTUALLY BROKE, WHICH IS WHY IT IS GRADED
|
||||
# HERE AND NOT ONLY ON `at()`. `_act()` decides whether this interval is the
|
||||
# buy one or the sell one with `here.start == v.cheapest.start` - an IDENTITY
|
||||
# test between the point `_point_at()` matched and the point the verdict was
|
||||
# built from. `_Series.at()` used to return the first point whose start was
|
||||
# within one hour BEFORE the instant, in list order, so on a 900 s series that
|
||||
# identity almost never held: the strategy idled through exactly the interval
|
||||
# it exists to trade, and read `here.provisional` and §17's `price < 0` off a
|
||||
# point that was not the one being traded.
|
||||
#
|
||||
# Belgium moved to a 15-minute MTU on 2025-10-01 and the live day-ahead feed
|
||||
# returns 96 points at 900 s. `static_tariff.py` emits HOURLY points, so every
|
||||
# fixture above this line - and every static site - is immune; this is
|
||||
# reachable only down the dynamic path TARIFF-08/09 made work.
|
||||
|
||||
QD0 = datetime(2026, 1, 15, 0, 0, tzinfo=timezone(timedelta(hours=1)))
|
||||
QUARTER = timedelta(minutes=15)
|
||||
|
||||
|
||||
def qplan_of(prices, *, provisional=False, start=QD0, gaps=None):
|
||||
"""A TariffPlan over `prices` spaced by `gaps` seconds (900 s by default).
|
||||
|
||||
`gaps` is what makes the MIXED fixture below expressible: FCST-04's
|
||||
carry-forward emits HOURLY carried points beside QUARTER-HOURLY firm ones,
|
||||
and the soak measured spacing [900, 3600] in 70 % of samples.
|
||||
"""
|
||||
g = [900] * (len(prices) - 1) if gaps is None else list(gaps)
|
||||
flags = ([bool(provisional)] * len(prices)
|
||||
if isinstance(provisional, bool) else list(provisional))
|
||||
pts, t = [], start
|
||||
for i, (p, f) in enumerate(zip(prices, flags)):
|
||||
pts.append(PricePoint(start=brussels(t), eur_per_kwh=round(float(p), 6),
|
||||
provisional=bool(f)))
|
||||
if i < len(g):
|
||||
t += timedelta(seconds=g[i])
|
||||
return TariffPlan(PriceRead.series(pts, PriceProduct.DAY_AHEAD,
|
||||
"generic_dayahead", start),
|
||||
None, "provider")
|
||||
|
||||
|
||||
def qprobe(i, *, start=QD0, gaps=None):
|
||||
"""An instant 7 minutes into slot `i` - inside every window, never on a
|
||||
boundary, and NOT the midpoint of an hour either, so an implementation that
|
||||
answered per hour and one that answers per quarter differ here."""
|
||||
g = [900] * i if gaps is None else list(gaps)[:i]
|
||||
return start + timedelta(seconds=sum(g)) + timedelta(minutes=7)
|
||||
|
||||
|
||||
def shipped_at(series, u):
|
||||
"""THE RULE TARIFF-14 REPLACED, kept here as the COUNTERFACTUAL ONLY.
|
||||
|
||||
The first point whose start is within one hour before `u`, in list order.
|
||||
Nothing below is graded against it; it exists so that the fixtures can be
|
||||
shown to SEPARATE the two rules. A check whose data cannot reach the
|
||||
failure it names is decoration (MUTATION.md), and on an hourly series the
|
||||
two rules agree at every instant - which is why 29 files stayed green.
|
||||
"""
|
||||
uu = u.astimezone(UTC)
|
||||
for p in series.points:
|
||||
s = p.start.astimezone(UTC)
|
||||
if s <= uu < s + timedelta(hours=1):
|
||||
return p
|
||||
return None
|
||||
|
||||
|
||||
# The hourly day interpolated across its quarters, with the two extremes
|
||||
# planted OFF the hour boundary on purpose.
|
||||
QBASE = []
|
||||
for _i in range(24):
|
||||
_a, _b = DAY[_i], DAY[(_i + 1) % 24]
|
||||
for _q in range(4):
|
||||
QBASE.append(round(_a + (_b - _a) * _q / 4 + 0.000001 * (_i * 4 + _q),
|
||||
6))
|
||||
Q_CHEAP_I = 4 * 5 + 3 # 05:45 - a :45 quarter, not a whole hour
|
||||
Q_DEAR_I = 4 * 19 + 1 # 19:15 - a :15 quarter, not a whole hour
|
||||
QDAY = list(QBASE)
|
||||
QDAY[Q_CHEAP_I] = round(min(QBASE) - 0.030, 6)
|
||||
QDAY[Q_DEAR_I] = round(max(QBASE) + 0.030, 6)
|
||||
QPLAN = qplan_of(QDAY)
|
||||
QPTS = QPLAN.read.points
|
||||
|
||||
print("10a. the fixture is not degenerate, and it SEPARATES the two rules")
|
||||
_qgaps = sorted({int((b.start - a.start).total_seconds())
|
||||
for a, b in zip(QPTS, QPTS[1:])})
|
||||
check(f"the day is 96 points at {_qgaps} s - the shape the live feed has "
|
||||
f"returned since Belgium's 2025-10-01 move to a 15-minute MTU",
|
||||
len(QPTS) == 96 and _qgaps == [900])
|
||||
check(f"the cheapest quarter is slot {Q_CHEAP_I} and the dearest slot "
|
||||
f"{Q_DEAR_I} - neither first, neither last, {abs(Q_CHEAP_I - Q_DEAR_I)} "
|
||||
f"slots apart, and NEITHER ON A WHOLE HOUR, without which this section "
|
||||
f"would grade nothing an hourly fixture does not already grade",
|
||||
QDAY.index(min(QDAY)) == Q_CHEAP_I and QDAY.index(max(QDAY)) == Q_DEAR_I
|
||||
and Q_CHEAP_I % 4 != 0 and Q_DEAR_I % 4 != 0
|
||||
and Q_CHEAP_I not in (0, 95) and Q_DEAR_I not in (0, 95)
|
||||
and abs(Q_CHEAP_I - Q_DEAR_I) > 1)
|
||||
check("...and all 96 prices are distinct, so 'the cheapest quarter' names one "
|
||||
"quarter and no tie-break is what any check below is really grading",
|
||||
len(set(QDAY)) == 96)
|
||||
check(f"...and its spread is {round(max(QDAY) - min(QDAY), 6):.4f} EUR/kWh, "
|
||||
f"clear of the widened threshold, so what follows grades the strategy "
|
||||
f"and not the spread rung",
|
||||
round(max(QDAY) - min(QDAY), 6) > required_spread(_shipped, NO_WEATHER))
|
||||
# ⚠️ HABIT 4, AND THE ONLY REASON THIS SECTION IS WORTH ITS LINES. State, as a
|
||||
# check, that the shipped rule and the fixed one give DIFFERENT points at the
|
||||
# two probes that matter. Without it, every check below could be passing on a
|
||||
# fixture the defect could not have changed.
|
||||
_cf_buy = derived(lambda: shipped_at(QPLAN.read, qprobe(Q_CHEAP_I)))
|
||||
_cf_sell = derived(lambda: shipped_at(QPLAN.read, qprobe(Q_DEAR_I)))
|
||||
check(f"the shipped rule and this one give DIFFERENT points at both probes - "
|
||||
f"buy: {derived(lambda: _cf_buy.eur_per_kwh)} instead of "
|
||||
f"{QDAY[Q_CHEAP_I]}, sell: {derived(lambda: _cf_sell.eur_per_kwh)} "
|
||||
f"instead of {QDAY[Q_DEAR_I]} - so the fixture can reach the failure it "
|
||||
f"names",
|
||||
_cf_buy is not QPTS[Q_CHEAP_I] and _cf_sell is not QPTS[Q_DEAR_I]
|
||||
and _cf_buy is not None and _cf_sell is not None)
|
||||
|
||||
print()
|
||||
print("10b. the buy/sell identity test matches the actual extremum interval")
|
||||
_qv = QPLAN.arbitrage(peak_guard_active=False, min_spread_eur_kwh=_shipped,
|
||||
uncertainty=NO_WEATHER)
|
||||
check("the verdict's cheapest and dearest are the two QUARTERS, not the hours "
|
||||
"containing them - `static_tariff` reduces over every firm point and "
|
||||
"carries no resolution assumption of its own",
|
||||
derived(lambda: _qv.cheapest) is QPTS[Q_CHEAP_I]
|
||||
and derived(lambda: _qv.dearest) is QPTS[Q_DEAR_I])
|
||||
_pa_buy = derived(lambda: _arb._point_at(QPLAN, qprobe(Q_CHEAP_I)))
|
||||
_pa_sell = derived(lambda: _arb._point_at(QPLAN, qprobe(Q_DEAR_I)))
|
||||
check("THE PROPERTY THE DEFECT BROKE: at the buy probe `_point_at()` returns "
|
||||
"the verdict's OWN cheapest point, by identity - which is what makes "
|
||||
"`_act()`'s `here.start == v.cheapest.start` true, and what made it "
|
||||
"false on every quarter-hourly series the add-on has ever fetched",
|
||||
_pa_buy is derived(lambda: _qv.cheapest))
|
||||
check("...and the same at the sell probe, against the verdict's dearest",
|
||||
_pa_sell is derived(lambda: _qv.dearest))
|
||||
_qbuy = decide(QPLAN, qprobe(Q_CHEAP_I), peak_guard_active=False, soc=50.0,
|
||||
min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
|
||||
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
|
||||
soc_ceiling_pct=95.0)
|
||||
_qsell = decide(QPLAN, qprobe(Q_DEAR_I), peak_guard_active=False, soc=50.0,
|
||||
min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
|
||||
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
|
||||
soc_ceiling_pct=95.0)
|
||||
check("...so decide() CHARGES in the cheapest quarter and DISCHARGES in the "
|
||||
"dearest one, which is the trade the strategy exists to make and the "
|
||||
"one it slept through",
|
||||
_qbuy.action == CHARGE and _qsell.action == DISCHARGE)
|
||||
check(f"...at THAT QUARTER's price and not at the price of the hour "
|
||||
f"containing it (buy {_qbuy.price_now_eur_kwh}, sell "
|
||||
f"{_qsell.price_now_eur_kwh})",
|
||||
near(_qbuy.price_now_eur_kwh, QDAY[Q_CHEAP_I])
|
||||
and near(_qsell.price_now_eur_kwh, QDAY[Q_DEAR_I]))
|
||||
check("...and both reasons quote that quarter's price, so the operator's log "
|
||||
"names the interval that was actually traded",
|
||||
f"{QDAY[Q_CHEAP_I]:.4f}" in _qbuy.reason
|
||||
and f"{QDAY[Q_DEAR_I]:.4f}" in _qsell.reason)
|
||||
# The negative arm. Without it "it acted" could be "it acts everywhere".
|
||||
_QOFF_I = Q_CHEAP_I + 2
|
||||
check(f"the off-peak probe's own price differs from its hour's :00 price "
|
||||
f"({QDAY[_QOFF_I]} vs {QDAY[4 * (_QOFF_I // 4)]}), so the check below "
|
||||
f"can tell which point the decision read",
|
||||
QDAY[_QOFF_I] != QDAY[4 * (_QOFF_I // 4)]
|
||||
and _QOFF_I not in (Q_CHEAP_I, Q_DEAR_I))
|
||||
_qoff = decide(QPLAN, qprobe(_QOFF_I), peak_guard_active=False, soc=50.0,
|
||||
min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
|
||||
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
|
||||
soc_ceiling_pct=95.0)
|
||||
check("a quarter that is neither extremum still idles, for the reason that "
|
||||
"names the right rule, and reports ITS OWN price - not the price of the "
|
||||
"quarter the shipped scan would have matched",
|
||||
_qoff.action is None and _qoff.reason == _arb.IDLE_OFF_PEAK_HOUR
|
||||
and near(_qoff.price_now_eur_kwh, QDAY[_QOFF_I]))
|
||||
|
||||
print()
|
||||
print("10c. §17 and §8.3 are evaluated on the QUARTER being traded")
|
||||
# ⚠️ §17 AT 900 s. The whole day is below zero and the dearest QUARTER is the
|
||||
# least negative one - section 2's shape, at the resolution the live feed
|
||||
# actually publishes. Under the shipped scan this probe never reached the
|
||||
# guard at all: the identity test failed first and the decision idled as
|
||||
# "neither the buy hour nor the sell hour", so §17 was not so much wrong as
|
||||
# never consulted.
|
||||
_shift = round(max(QDAY) + 0.020, 6)
|
||||
QNEG = [round(p - _shift, 6) for p in QDAY]
|
||||
QNEG_PLAN = qplan_of(QNEG)
|
||||
check(f"the negative day's dearest quarter is slot {Q_DEAR_I} and it is BELOW "
|
||||
f"ZERO ({max(QNEG):+.4f}), and its spread still clears the widened "
|
||||
f"threshold - so the refusal below is §17's and not the spread rung's",
|
||||
QNEG.index(max(QNEG)) == Q_DEAR_I and max(QNEG) < 0
|
||||
and round(max(QNEG) - min(QNEG), 6) > required_spread(_shipped,
|
||||
NO_WEATHER))
|
||||
_qneg_sell = decide(QNEG_PLAN, qprobe(Q_DEAR_I), peak_guard_active=False,
|
||||
soc=50.0, min_spread_eur_kwh=_shipped,
|
||||
uncertainty=NO_WEATHER, charge_w=MAX_W, discharge_w=MAX_W,
|
||||
soc_floor_pct=20.0, soc_ceiling_pct=95.0)
|
||||
check("selling into a negative-price QUARTER is refused, and the reason names "
|
||||
"§17 and THAT QUARTER's price",
|
||||
_qneg_sell.action is None and "§17" in _qneg_sell.reason
|
||||
and f"{max(QNEG):+.4f}" in _qneg_sell.reason)
|
||||
# ...and the other side, so the refusal is about the sign and nothing else.
|
||||
QPOS = list(QNEG)
|
||||
QPOS[Q_DEAR_I] = round(-max(QNEG), 6)
|
||||
check("...while the SAME quarter-hourly series with that ONE price's SIGN "
|
||||
"FLIPPED discharges instead",
|
||||
decide(qplan_of(QPOS), qprobe(Q_DEAR_I), peak_guard_active=False,
|
||||
soc=50.0, min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
|
||||
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
|
||||
soc_ceiling_pct=95.0).action == DISCHARGE
|
||||
and sum(1 for a, b in zip(QNEG, QPOS) if a != b) == 1)
|
||||
# §8.3's flag, read off the covering point. `_act`'s own provisional guard
|
||||
# stays unreachable here for section 4's reason - the extrema are firm by
|
||||
# `_firm_or_die` and starts are unique - and THAT ARGUMENT IS EXACTLY WHAT THE
|
||||
# DEFECT BROKE at 900 s, because the point `at()` returned was not the
|
||||
# verdict's point at all. It is restated here at the resolution that matters.
|
||||
check("the point matched for the buy quarter IS the verdict's own cheapest "
|
||||
"point and is FIRM, so `_act`'s §8.3 guard is unreachable at 900 s for "
|
||||
"the same construction section 4 states at 3600 s",
|
||||
_pa_buy is derived(lambda: _qv.cheapest)
|
||||
and derived(lambda: _pa_buy.provisional) is False
|
||||
and len({p.start for p in QPTS}) == len(QPTS))
|
||||
|
||||
print()
|
||||
print("10d. the MIXED series FCST-04 actually emits, traded end to end")
|
||||
# ⚠️ 70 % OF SOAK SAMPLES, NOT AN EDGE CASE. The carry-forward emits HOURLY
|
||||
# carried points beside QUARTER-HOURLY firm ones, so a window derived from one
|
||||
# global spacing would be wrong in the NORMAL steady state. Here: 12 carried
|
||||
# hourly points for the morning, then firm quarters for the rest of the day.
|
||||
MIX_N_H, MIX_N_Q = 12, 48
|
||||
MIX_GAPS = [3600] * MIX_N_H + [900] * (MIX_N_Q - 1)
|
||||
MIX_PRICES = [QDAY[4 * i] for i in range(MIX_N_H)] + QDAY[4 * MIX_N_H:]
|
||||
MIX_FLAGS = [True] * MIX_N_H + [False] * MIX_N_Q
|
||||
MIX_PLAN = qplan_of(MIX_PRICES, provisional=MIX_FLAGS, gaps=MIX_GAPS)
|
||||
MIX_PTS = MIX_PLAN.read.points
|
||||
_mix_gaps = sorted({int((b.start - a.start).total_seconds())
|
||||
for a, b in zip(MIX_PTS, MIX_PTS[1:])})
|
||||
check(f"the mixed plan really is mixed - spacing {_mix_gaps} s, "
|
||||
f"{MIX_N_H} carried and {MIX_N_Q} firm points, so neither half is what "
|
||||
f"the other half's window rule would give",
|
||||
_mix_gaps == [900, 3600]
|
||||
and sum(1 for p in MIX_PTS if p.provisional) == MIX_N_H
|
||||
and sum(1 for p in MIX_PTS if not p.provisional) == MIX_N_Q)
|
||||
_mix_cheap_i = MIX_N_H + (MIX_PRICES[MIX_N_H:].index(min(MIX_PRICES[MIX_N_H:])))
|
||||
_mix_dear_i = MIX_N_H + (MIX_PRICES[MIX_N_H:].index(max(MIX_PRICES[MIX_N_H:])))
|
||||
check(f"...and its firm half's extremes are quarters {_mix_cheap_i} and "
|
||||
f"{_mix_dear_i}, neither on a whole hour, so the identity test below is "
|
||||
f"about the quarter and not about the hour",
|
||||
(_mix_cheap_i - MIX_N_H) % 4 != 0 and (_mix_dear_i - MIX_N_H) % 4 != 0)
|
||||
_mv = MIX_PLAN.arbitrage(peak_guard_active=False, min_spread_eur_kwh=_shipped,
|
||||
uncertainty=NO_WEATHER)
|
||||
_mix_buy_t = qprobe(_mix_cheap_i, gaps=MIX_GAPS)
|
||||
_mix_sell_t = qprobe(_mix_dear_i, gaps=MIX_GAPS)
|
||||
check("the verdict over a MIXED series still names firm quarters, and "
|
||||
"`_point_at()` matches them by identity at their own probes",
|
||||
derived(lambda: _mv.cheapest) is MIX_PTS[_mix_cheap_i]
|
||||
and derived(lambda: _mv.dearest) is MIX_PTS[_mix_dear_i]
|
||||
and derived(lambda: _arb._point_at(MIX_PLAN, _mix_buy_t))
|
||||
is MIX_PTS[_mix_cheap_i]
|
||||
and derived(lambda: _arb._point_at(MIX_PLAN, _mix_sell_t))
|
||||
is MIX_PTS[_mix_dear_i])
|
||||
check("...so the mixed day CHARGES and DISCHARGES in its firm quarters, at "
|
||||
"their own prices",
|
||||
decide(MIX_PLAN, _mix_buy_t, peak_guard_active=False, soc=50.0,
|
||||
min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
|
||||
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
|
||||
soc_ceiling_pct=95.0).action == CHARGE
|
||||
and decide(MIX_PLAN, _mix_sell_t, peak_guard_active=False, soc=50.0,
|
||||
min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
|
||||
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
|
||||
soc_ceiling_pct=95.0).action == DISCHARGE)
|
||||
# An instant inside a CARRIED hour of the same series. The covering point is
|
||||
# the carried one; the shipped scan answered a point from the FIRM block
|
||||
# earlier in the list, which is §8.3's flag read off the wrong point.
|
||||
_mix_carried_t = QD0 + timedelta(hours=MIX_N_H - 1, minutes=37)
|
||||
check("an instant inside a CARRIED hour of the mixed series matches the "
|
||||
"CARRIED point, whose provisional flag is what §8.3 reads - and the "
|
||||
"strategy does nothing there, because a carried point is not in the "
|
||||
"firm trade set",
|
||||
derived(lambda: _arb._point_at(MIX_PLAN, _mix_carried_t))
|
||||
is MIX_PTS[MIX_N_H - 1]
|
||||
and MIX_PTS[MIX_N_H - 1].provisional is True
|
||||
and decide(MIX_PLAN, _mix_carried_t, peak_guard_active=False, soc=50.0,
|
||||
min_spread_eur_kwh=_shipped, uncertainty=NO_WEATHER,
|
||||
charge_w=MAX_W, discharge_w=MAX_W, soc_floor_pct=20.0,
|
||||
soc_ceiling_pct=95.0).action is None)
|
||||
|
||||
print()
|
||||
print("10e. the HOURLY day is unchanged, which is the regression half")
|
||||
# ⚠️ A REGRESSION GUARD, NOT A WITNESS. On an hourly series the shipped rule
|
||||
# and this one agree at every instant, which is precisely why 29 files stayed
|
||||
# green through the defect. It is here so that a fix which got 900 s right by
|
||||
# breaking 3600 s cannot pass.
|
||||
_h_buy = act(PLAN, CHEAP_H, thresh=_shipped, unc=NO_WEATHER)
|
||||
_h_sell = act(PLAN, DEAR_H, thresh=_shipped, unc=NO_WEATHER)
|
||||
check("the hourly fixture still charges in its cheapest hour and discharges "
|
||||
"in its dearest, at those hours' prices",
|
||||
_h_buy.action == CHARGE and _h_sell.action == DISCHARGE
|
||||
and near(_h_buy.price_now_eur_kwh, DAY[CHEAP_H])
|
||||
and near(_h_sell.price_now_eur_kwh, DAY[DEAR_H]))
|
||||
check("...and a probe 59 minutes into an hourly slot still matches that slot, "
|
||||
"so the hourly window did not shrink under the fix",
|
||||
derived(lambda: _arb._point_at(
|
||||
PLAN, D0 + timedelta(hours=CHEAP_H, minutes=59)))
|
||||
is derived(lambda: PLAN.read.points[CHEAP_H]))
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
print()
|
||||
if fails:
|
||||
|
||||
@@ -2601,6 +2601,363 @@ check("...and BOTH translations tell an operator which switch does which - "
|
||||
and "ZET DE PRIJSFEED NIET UIT" in _T10_TR["nl"])
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
print()
|
||||
print("14. TARIFF-14 - the point whose OWN interval covers the instant")
|
||||
# ⚠️ THE RESOLUTION IS THE WHOLE SECTION, AND ITS ABSENCE IS WHY THE DEFECT
|
||||
# SHIPPED. Every price fixture in this repository before TARIFF-08 was the
|
||||
# invented HOURLY row shape, and on an hourly series `s <= u < s + 1 h` and
|
||||
# "until the next point's start" AGREE AT EVERY INSTANT - so the suite had
|
||||
# been exercising the one resolution that cannot expose the bug. Section 8
|
||||
# counts points on the 23- and 25-hour days and never asks `at()` for a
|
||||
# quarter-hour. Belgium moved to a 15-minute MTU on 2025-10-01 and the live
|
||||
# day-ahead feed returns 96 points at 900 s.
|
||||
|
||||
# ⚠️ BRUSSELS MIDNIGHT, NOT UTC MIDNIGHT, so that every wall-clock time in
|
||||
# a check name below is the time the series really renders - "00:15" and
|
||||
# "23:45" are what `canonical()` writes, not an offset away from it.
|
||||
Q0 = datetime(2026, 1, 15, 0, 0, tzinfo=timezone(timedelta(hours=1)))
|
||||
HOUR = timedelta(hours=1)
|
||||
|
||||
|
||||
def spaced(gaps, first=0.10, step=0.01, provisional=None, start=Q0):
|
||||
"""A series whose consecutive gaps are exactly `gaps`, in SECONDS.
|
||||
|
||||
`len(points) == len(gaps) + 1`. Prices ascend by `step`, so a point can be
|
||||
named by VALUE as well as by identity and a sweep that compared only
|
||||
prices could still tell two points apart.
|
||||
"""
|
||||
n = len(gaps) + 1
|
||||
flags = [False] * n if provisional is None else list(provisional)
|
||||
pts, t = [], start
|
||||
for i in range(n):
|
||||
pts.append(PricePoint(brussels(t), round(first + step * i, 6),
|
||||
bool(flags[i])))
|
||||
if i < len(gaps):
|
||||
t += timedelta(seconds=gaps[i])
|
||||
return PriceRead.series(pts, PriceProduct.DAY_AHEAD, "tariff14", start)
|
||||
|
||||
|
||||
def gaps_of(s):
|
||||
u = [p.start.astimezone(UTC) for p in s.points]
|
||||
return [int((b - a).total_seconds()) for a, b in zip(u, u[1:])]
|
||||
|
||||
|
||||
# The three shapes the acceptance criteria name, plus the two degenerate ones.
|
||||
Q = spaced([900] * 95) # the shipped 96-point Belgian day
|
||||
H = spaced([3600] * 23) # what every older fixture carried
|
||||
# ⚠️ FCST-04'S STEADY STATE, NOT AN EDGE CASE. Eight firm quarter-hours, then
|
||||
# the carried HOURLY points the fill adds for the hours nobody published. The
|
||||
# seam between them is a 900 s gap and the tail is 3600 s.
|
||||
MIX = spaced([900] * 8 + [3600] * 3,
|
||||
provisional=[False] * 9 + [True] * 3)
|
||||
# The same seam made the END of the series - the last point is the FIRST
|
||||
# carried hourly one, and the gap behind it is 900 s. This is the fixture the
|
||||
# tail decision is graded on.
|
||||
MIX_TAIL = spaced([900] * 8, provisional=[False] * 8 + [True])
|
||||
ONE = spaced([])
|
||||
|
||||
print("14a. the fixtures are not degenerate - asserted before anything else")
|
||||
check(f"the quarter-hourly fixture is the live shape - 96 points, spacing "
|
||||
f"{sorted(set(gaps_of(Q)))} s",
|
||||
len(Q.points) == 96 and set(gaps_of(Q)) == {900})
|
||||
check(f"the hourly fixture is 24 points at {sorted(set(gaps_of(H)))} s - and "
|
||||
f"it is a REGRESSION guard, not a witness: the shipped rule and this "
|
||||
f"one agree at every instant of it, which is exactly why nothing caught "
|
||||
f"this",
|
||||
len(H.points) == 24 and set(gaps_of(H)) == {3600})
|
||||
check(f"...while the MIXED fixture really is mixed, spacing "
|
||||
f"{sorted(set(gaps_of(MIX)))} s - the pair the FCST-04 soak measured in "
|
||||
f"70 % of samples, so a window derived from ONE global spacing would be "
|
||||
f"wrong in the NORMAL steady state",
|
||||
set(gaps_of(MIX)) == {900, 3600}
|
||||
and set(gaps_of(MIX_TAIL)) == {900}
|
||||
and gaps_of(MIX)[-1] == 3600 and gaps_of(MIX_TAIL)[-1] == 900)
|
||||
check("...and every point of every fixture carries a DISTINCT price, so 'the "
|
||||
"covering point' names one point and a wrong answer cannot coincide "
|
||||
"with the right one",
|
||||
all(len({p.eur_per_kwh for p in s.points}) == len(s.points)
|
||||
for s in (Q, H, MIX, MIX_TAIL)))
|
||||
check("...and the mixed fixtures carry BOTH firm and carried points, which is "
|
||||
"what makes them FCST-04's output rather than a spacing exercise",
|
||||
all(any(p.provisional for p in s.points)
|
||||
and any(not p.provisional for p in s.points)
|
||||
for s in (MIX, MIX_TAIL)))
|
||||
|
||||
print()
|
||||
print("14b. the reading the shipped scan got wrong, stated as literals")
|
||||
# ⚠️ ASSERTED AGAINST TYPED-OUT INDICES AND PRICES, not against a window this
|
||||
# file recomputed by price.py's own rule. The sweep below is exhaustive and
|
||||
# the sweep is derived; these four are the anchors that keep it honest.
|
||||
_q15 = derived(lambda: Q.at(Q0 + timedelta(minutes=15)))
|
||||
check(f"at(00:15) on the 900 s series is the 00:15 point at 0.11 EUR/kWh - the "
|
||||
f"shipped scan answered the 00:00 point at 0.10, because the :00 "
|
||||
f"point's HOUR had not run out yet (saw {_q15!r})",
|
||||
_q15 is Q.points[1] and derived(lambda: _q15.eur_per_kwh) == 0.11)
|
||||
_q195 = derived(lambda: Q.at(Q0 + timedelta(minutes=195)))
|
||||
check(f"at(03:15) is the 03:15 point at 0.23 - the shipped scan answered "
|
||||
f"02:30 at 0.20 here, and that is the SECOND failure mode: from the "
|
||||
f"second hour on the first point still inside its own hour wins, so the "
|
||||
f"error COMPOUNDS with depth into the series (saw {_q195!r})",
|
||||
_q195 is Q.points[13] and derived(lambda: _q195.eur_per_kwh) == 0.23)
|
||||
check("at(00:30) and at(00:45) are their own quarters too, so the answer is "
|
||||
"not merely 'one point later'",
|
||||
derived(lambda: Q.at(Q0 + timedelta(minutes=30))) is Q.points[2]
|
||||
and derived(lambda: Q.at(Q0 + timedelta(minutes=45))) is Q.points[3])
|
||||
check("on the MIXED series the quarter-hourly half answers per QUARTER and "
|
||||
"the carried hourly half answers per HOUR, in one series - at(00:15) is "
|
||||
"the 00:15 point and at(02:30) is the 02:00 one",
|
||||
derived(lambda: MIX.at(Q0 + timedelta(minutes=15))) is MIX.points[1]
|
||||
and derived(lambda: MIX.at(Q0 + timedelta(minutes=150))) is MIX.points[8]
|
||||
and derived(lambda: MIX.points[8].start.astimezone(UTC)) ==
|
||||
Q0 + timedelta(hours=2))
|
||||
|
||||
print()
|
||||
print("14c. the exhaustive sweep, at 900 s, 3600 s and mixed")
|
||||
|
||||
|
||||
def windows(s):
|
||||
"""`(start_utc, width)` per point, computed HERE from the gaps.
|
||||
|
||||
Deliberately NOT read off `_Series._cover`: a check built from the table it
|
||||
grades compares a thing against itself. The rule is restated - a point runs
|
||||
to the next start, capped at an hour; the last point copies the width in
|
||||
front of it; a lone point gets an hour.
|
||||
"""
|
||||
u = [p.start.astimezone(UTC) for p in s.points]
|
||||
out = []
|
||||
for i, t in enumerate(u):
|
||||
if i + 1 < len(u):
|
||||
g = u[i + 1] - t
|
||||
elif i:
|
||||
g = t - u[i - 1]
|
||||
else:
|
||||
g = HOUR
|
||||
out.append((t, g if g < HOUR else HOUR))
|
||||
return out
|
||||
|
||||
|
||||
def sweep(s):
|
||||
"""Every probe whose covering point is unambiguous, and what `at()` said."""
|
||||
wrong = []
|
||||
for i, (t, w) in enumerate(windows(s)):
|
||||
for off in (timedelta(0), w / 2, w - timedelta(seconds=1)):
|
||||
got = derived(lambda: s.at(t + off))
|
||||
if got is not s.points[i]:
|
||||
wrong.append(f"{(t + off).isoformat()} -> {got!r}, "
|
||||
f"wanted {s.points[i].eur_per_kwh}")
|
||||
return wrong
|
||||
|
||||
|
||||
for _name, _s in (("900 s", Q), ("3600 s", H), ("mixed 900/3600", MIX),
|
||||
("mixed, 900 s tail", MIX_TAIL)):
|
||||
_w = derived(lambda s=_s: sweep(s))
|
||||
check(f"{_name}: all {3 * len(_s.points)} probes inside the series return "
|
||||
f"the point whose OWN interval covers them "
|
||||
f"({_w if _w != [] else 'no misses'})", _w == [])
|
||||
|
||||
print()
|
||||
print("14d. the one-hour cap, which is what keeps FCST-04's holes holes")
|
||||
# ⚠️ NOT DECORATION. FCST-04 leaves an hour OUT when nothing published it and
|
||||
# yesterday has no value either, and price_forecast.py's module note names
|
||||
# `at()` raising Absence for it as the behaviour that sends the caller to the
|
||||
# static profile (§11.4 for that hour). Uncapped, "until the next point's
|
||||
# start" would stretch the point before a three-hour hole across the whole
|
||||
# hole - filling holes with a stale price, which is the defect this module
|
||||
# exists to refuse, introduced by the fix for a different one.
|
||||
HOLE = PriceRead.series(
|
||||
[PricePoint(brussels(Q0 + timedelta(hours=i)), 0.10 + i / 100.0, False)
|
||||
for i in (0, 1, 4, 5)],
|
||||
PriceProduct.DAY_AHEAD, "tariff14", Q0)
|
||||
_in_hole = [Q0 + timedelta(hours=2), Q0 + timedelta(hours=2, minutes=30),
|
||||
Q0 + timedelta(hours=3), Q0 + timedelta(hours=3, minutes=59)]
|
||||
check(f"a three-hour hole stays a hole: all {len(_in_hole)} instants inside it "
|
||||
f"raise Absence, while the point BEFORE it still covers its own hour and "
|
||||
f"the point after it answers from its own start",
|
||||
all(raises(lambda u=u: HOLE.at(u), Absence) for u in _in_hole)
|
||||
and derived(lambda: HOLE.at(Q0 + timedelta(hours=1, minutes=59)))
|
||||
is HOLE.points[1]
|
||||
and derived(lambda: HOLE.at(Q0 + timedelta(hours=4))) is HOLE.points[2])
|
||||
check(f"...and the cap is stated once, as MAX_POINT_SPAN = "
|
||||
f"{_price.MAX_POINT_SPAN}, rather than spelled into the lookup",
|
||||
_price.MAX_POINT_SPAN == HOUR)
|
||||
# ⚠️ AND THE OTHER HALF OF THE HOLE, WHICH IS THE ONE §8.3 READS. FCST-04
|
||||
# fills a whole hour nobody published with ONE carried point, so a
|
||||
# quarter-hourly day can carry an hourly carried point in the middle of it.
|
||||
# The instant inside that hour is covered by the CARRIED point - and the
|
||||
# shipped scan answered the FIRM quarter 52 minutes earlier, because that was
|
||||
# the first point still inside its own hour. `_act()`'s `here.provisional`
|
||||
# guard and `price_forecast._walk()`'s `series.at(u).provisional` both read the
|
||||
# flag off whatever that lookup returned, so the flag was FIRM on an hour
|
||||
# nobody published.
|
||||
SANDWICH = PriceRead.series(
|
||||
[PricePoint(brussels(Q0 + timedelta(minutes=15 * i)),
|
||||
round(0.10 + i / 1000.0, 6), False) for i in range(44)]
|
||||
+ [PricePoint(brussels(Q0 + timedelta(hours=11)), 0.500, True)]
|
||||
+ [PricePoint(brussels(Q0 + timedelta(hours=12, minutes=15 * i)),
|
||||
round(0.20 + i / 1000.0, 6), False) for i in range(4)],
|
||||
PriceProduct.DAY_AHEAD, "tariff14", Q0)
|
||||
_sand_t = Q0 + timedelta(hours=11, minutes=37)
|
||||
|
||||
|
||||
def shipped_at(s, u):
|
||||
"""The rule TARIFF-14 replaced, as the COUNTERFACTUAL only - nothing is
|
||||
graded against it. It is here so the fixture can be SHOWN to separate the
|
||||
two rules; a check whose data cannot reach the failure it names grades
|
||||
nothing, and on an hourly series these two agree at every instant."""
|
||||
uu = u.astimezone(UTC)
|
||||
for p in s.points:
|
||||
st = p.start.astimezone(UTC)
|
||||
if st <= uu < st + HOUR:
|
||||
return p
|
||||
return None
|
||||
|
||||
|
||||
check("the sandwich fixture is FCST-04's real output - 44 firm quarters, ONE "
|
||||
"carried hourly point filling the hour nobody published, then firm "
|
||||
"quarters again",
|
||||
len(SANDWICH.points) == 49
|
||||
and SANDWICH.points[44].provisional is True
|
||||
and sum(1 for p in SANDWICH.points if p.provisional) == 1)
|
||||
_sand_now = derived(lambda: SANDWICH.at(_sand_t))
|
||||
_sand_old = derived(lambda: shipped_at(SANDWICH, _sand_t))
|
||||
check(f"an instant inside the carried hour is covered by the CARRIED point, "
|
||||
f"so §8.3's flag is read off the hour it belongs to. The shipped scan "
|
||||
f"answered the FIRM quarter 52 minutes earlier "
|
||||
f"({derived(lambda: _sand_old.start.isoformat())}, provisional="
|
||||
f"{derived(lambda: _sand_old.provisional)}), which is a carried hour "
|
||||
f"wearing a published hour's flag",
|
||||
_sand_now is SANDWICH.points[44]
|
||||
and derived(lambda: _sand_now.provisional) is True
|
||||
and _sand_old is SANDWICH.points[43]
|
||||
and derived(lambda: _sand_old.provisional) is False)
|
||||
|
||||
print()
|
||||
print("14e. the LAST point's window - decided and documented, not inherited")
|
||||
# There is no next start and §9.3 says nothing about how wide the final
|
||||
# interval is, so THE LAST POINT IS GIVEN THE SAME SPAN AS THE POINT BEFORE IT
|
||||
# (one hour when it is the only point). The alternative - a flat one-hour tail
|
||||
# - is defensible for an hourly series and wrong for the live one: it answers
|
||||
# 00:30 TOMORROW with today's last price, which is reaching for a neighbour
|
||||
# across a day boundary.
|
||||
_ql = Q.points[-1].start.astimezone(UTC)
|
||||
_hl = H.points[-1].start.astimezone(UTC)
|
||||
_ml = MIX_TAIL.points[-1].start.astimezone(UTC)
|
||||
check("the 900 s day's last point covers its own QUARTER: at(+14:59) answers "
|
||||
"and at(+15:00) raises, so today's 23:45 price cannot be returned for "
|
||||
"00:30 tomorrow",
|
||||
derived(lambda: Q.at(_ql + timedelta(minutes=14, seconds=59)))
|
||||
is Q.points[-1]
|
||||
and raises(lambda: Q.at(_ql + timedelta(minutes=15)), Absence))
|
||||
check("the hourly series' last point still covers an HOUR, exactly as it "
|
||||
"shipped - no static or hourly site changes behaviour",
|
||||
derived(lambda: H.at(_hl + timedelta(minutes=59))) is H.points[-1]
|
||||
and raises(lambda: H.at(_hl + HOUR), Absence))
|
||||
check("the MIXED series ending at the firm/carried seam takes the 900 s in "
|
||||
"front of it, so its last 45 minutes raise Absence. That is the SAFE "
|
||||
"direction and it is CHOSEN: §11.4 sends the caller to the static "
|
||||
"profile rather than trading on a span the series never demonstrated, "
|
||||
"and the point is carried, so §8.3 forbids irreversible action on it "
|
||||
"anyway",
|
||||
derived(lambda: MIX_TAIL.at(_ml + timedelta(minutes=14, seconds=59)))
|
||||
is MIX_TAIL.points[-1]
|
||||
and raises(lambda: MIX_TAIL.at(_ml + timedelta(minutes=15)), Absence)
|
||||
and MIX_TAIL.points[-1].provisional)
|
||||
# ⚠️ HABIT 1. Without this the three checks above would all pass under a
|
||||
# single global tail rule of 15 minutes, or of an hour, depending which two
|
||||
# you read - so the claim that the tail FOLLOWS THE SERIES is asserted as its
|
||||
# own check, at one instant where the three fixtures must disagree.
|
||||
_t30 = [derived(lambda s=s, l=l: s.covers(l + timedelta(minutes=30)))
|
||||
for s, l in ((Q, _ql), (H, _hl), (MIX_TAIL, _ml))]
|
||||
check(f"...and those tails are genuinely three readings of one rule and not "
|
||||
f"one constant: 30 minutes past its own last point the hourly series "
|
||||
f"answers and both 900 s ones do not {_t30}",
|
||||
_t30 == [False, True, False])
|
||||
check("a series of ONE point has no gap in front of it to copy, so it gets an "
|
||||
"hour - the shipped behaviour, kept deliberately for the degenerate case",
|
||||
derived(lambda: ONE.at(Q0 + timedelta(minutes=59, seconds=59)))
|
||||
is ONE.points[0]
|
||||
and raises(lambda: ONE.at(Q0 + HOUR), Absence))
|
||||
|
||||
print()
|
||||
print("14f. order is established by the lookup, not assumed of the caller")
|
||||
# ⚠️ `at()` now reads a point's NEIGHBOURS, so it WOULD depend on sortedness.
|
||||
# The index is sorted at construction so that it cannot. `_validate_raw()`
|
||||
# refuses out-of-order and duplicate starts on the `load_prices()` path
|
||||
# (section 9 above), `price_history._parse_points()` re-sorts what it reads off
|
||||
# disk and FCST-04's fill emits ascending - but `PriceRead.series()` itself has
|
||||
# never enforced any of it, and an assumption that holds today and is unstated
|
||||
# is how this class of defect recurs.
|
||||
_REV = PriceRead.series(tuple(reversed(Q.points)), PriceProduct.DAY_AHEAD,
|
||||
"tariff14", Q0)
|
||||
check("the reversed fixture really is reversed, so the check below is not "
|
||||
"grading two identical series",
|
||||
_REV.points[0] is Q.points[-1] and _REV.points[-1] is Q.points[0]
|
||||
and _REV.points != Q.points)
|
||||
_rev_bad = derived(lambda: [p.start.isoformat() for p in Q.points
|
||||
if _REV.at(p.start) is not p])
|
||||
check(f"...and a series handed its 96 points in REVERSE order answers at() "
|
||||
f"identically at every one of them "
|
||||
f"({_rev_bad if _rev_bad != [] else 'no misses'})",
|
||||
_rev_bad == [])
|
||||
check("...while the load path refuses an unsorted feed outright, so the two "
|
||||
"defences are stated rather than one being quietly relied on",
|
||||
raises(lambda: _price._validate_raw(
|
||||
[RawPoint(T0 + timedelta(minutes=15), 0.1, False),
|
||||
RawPoint(T0, 0.2, False)], _caps), SourceUnavailable))
|
||||
check("a DUPLICATED start does not raise anywhere - not at construction and "
|
||||
"not on the 30 s control path. The later list entry takes the real span "
|
||||
"and the earlier one becomes unreachable through at(); load_prices() "
|
||||
"refuses the shape outright, and this says what happens to anyone who "
|
||||
"builds one by hand",
|
||||
derived(lambda: PriceRead.series(
|
||||
[PricePoint(brussels(Q0), 0.10, False),
|
||||
PricePoint(brussels(Q0), 0.20, False),
|
||||
PricePoint(brussels(Q0 + timedelta(minutes=15)), 0.30, False)],
|
||||
PriceProduct.DAY_AHEAD, "tariff14", Q0).at(Q0).eur_per_kwh) == 0.20)
|
||||
|
||||
print()
|
||||
print("14g. covers() shared it; window() did not")
|
||||
check("covers() shared the defect EXACTLY, because it has no window rule of "
|
||||
"its own - it asks at() - so the one change fixes both. On the 900 s "
|
||||
"day it is now True 14:59 past the last point and False 15:00 past it, "
|
||||
"where an hour-wide window said True for another 45 minutes",
|
||||
derived(lambda: Q.covers(_ql + timedelta(minutes=14, seconds=59))) is True
|
||||
and derived(lambda: Q.covers(_ql + timedelta(minutes=15))) is False
|
||||
and derived(lambda: Q.covers(_ql + timedelta(minutes=30))) is False
|
||||
and derived(lambda: Q.covers(Q0 + timedelta(minutes=15))) is True)
|
||||
check("window() did NOT share it, and the reason is worth a check rather than "
|
||||
"a rediscovery: it grades a point's START against a half-open range and "
|
||||
"never asks how long a point lasts, so it carries no resolution "
|
||||
"assumption at all. 00:07:30 .. 00:22:30 holds exactly the 00:15 point "
|
||||
"- while at(00:07:30) is the 00:00 one, which is what makes it a "
|
||||
"DIFFERENT question and not a coverage query",
|
||||
derived(lambda: Q.window(Q0 + timedelta(minutes=7, seconds=30),
|
||||
Q0 + timedelta(minutes=22, seconds=30)))
|
||||
== (Q.points[1],)
|
||||
and derived(lambda: Q.at(Q0 + timedelta(minutes=7, seconds=30)))
|
||||
is Q.points[0])
|
||||
|
||||
print()
|
||||
print("14h. the 'no data is not free' contract, unchanged")
|
||||
check("at() answers with a PricePoint or raises Absence - never None, at any "
|
||||
"resolution, so `series.at(t) or fallback` cannot become a price of zero",
|
||||
all(isinstance(derived(lambda s=s: s.at(s.points[0].start)), PricePoint)
|
||||
for s in (Q, H, MIX, MIX_TAIL, ONE))
|
||||
and raises(lambda: Q.at(Q0 - timedelta(seconds=1)), Absence)
|
||||
and raises(lambda: Q.at(Q0 + timedelta(days=2)), Absence))
|
||||
_absence_words = _reason(lambda: Q.at(Q0 + timedelta(days=2)))
|
||||
check(f"...and the refusal still names the range the series runs, so an "
|
||||
f"operator can tell 'outside the horizon' from 'into a hole'",
|
||||
Q.points[0].start.isoformat() in _absence_words
|
||||
and Q.points[-1].start.isoformat() in _absence_words)
|
||||
check("...and it never invents a neighbour: the instant one second before the "
|
||||
"series starts is refused, not answered with the first point",
|
||||
raises(lambda: Q.at(Q0 - timedelta(seconds=1)), Absence)
|
||||
and raises(lambda: MIX.at(Q0 - timedelta(seconds=1)), Absence)
|
||||
and raises(lambda: H.at(Q0 - timedelta(seconds=1)), Absence))
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
for _d in _tmpdirs_t10:
|
||||
shutil.rmtree(_d, ignore_errors=True)
|
||||
|
||||
Reference in New Issue
Block a user