admin/goodwe-addon
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

DATA-02: twelve months of real Belgian day-ahead prices, closing OPT-04 AC4 (#24)

Browse Source 
8760 hourly EPEX BE clearings from the Energy-Charts API (Fraunhofer ISE), CC BY 4.0 from Bundesnetzagentur | SMARD.de, committed with attribution and a fetcher that refuses to write if the licence stops saying CC BY 4.0. Raw wholesale with all_in false and the markup/levy/VAT assumption labelled as an assumption; the additive terms cancel out of every daily spread, so trade counts turn on the VAT multiplier alone. DST handled by iterating in UTC and rendering through price.brussels(): 25 points on 2025-10-26, 23 on 2026-03-29. app/ diff is zero lines - nothing tuned. Section 7B asserts that the real year cannot grade section 17, so the synthetic year stays.
This commit was merged in pull request #24.
This commit is contained in:
admin
2026-08-31 11:12:49 +02:00
parent 4601682815 d57122ec74
commit 86c43eba24
4 changed files with 44760 additions and 32 deletions
Download Patch File Download Diff File Expand all files Collapse all files
goodwe_controller/CHANGELOG.md
+76
View File
@@ -2,6 +2,82 @@
## Unreleased
**OPT-04's AC4 is met: twelve months of real EPEX BE day-ahead prices are now in
the repository, and the strategy has been replayed against them (DATA-02).**
AC4 asked for "net-positive across 12 months of replayed prices". OPT-04 shipped
it as NOT MET and replayed a synthetic year instead, because `scenarios/` held
only P1 meter captures. That was the right call at the time and the wrong
diagnosis of why: the criterion was being carried alongside a 30-day *meter*
criterion as though both needed waiting, and EPEX BE day-ahead is **public data,
not a capture**. It needed a fetch, not a month.
`scenarios/epex-be-dayahead-2025-09-01_2026-09-01.json` — **8760 hourly points**,
2025-09-01 to 2026-09-01 Brussels-local, in §9.3's canonical three-field shape.
`scenarios/fetch_epex_be.py` produced it and `--verify` re-derives every claim it
makes about itself without touching the network.
**Licensing was checked before anything was committed.** The source is
Fraunhofer ISE's Energy-Charts API, which publishes the **BE** zone under
**CC BY 4.0 from Bundesnetzagentur | SMARD.de** and other zones under
private-use-only terms — so this zone is redistributable with attribution and the
data is committed rather than left as a script plus a checksum. The fetcher
**refuses to write a file** whose per-response `license_info` stops saying
CC BY 4.0. Cross-checked against the upstream: SMARD's own `4996/DE` series
matched this feed on all 96 quarter-hours of 2026-04-10, maximum difference
0.00 EUR/MWh.
**The figures are raw wholesale and the file says so.** `"all_in": false`, with
the markup/tax assumption carried in `provenance.all_in_assumption` as *numbers
to apply* rather than numbers already applied — TARIFF-02's double-counting trap
in data form. `test_arbitrage.py` section 7B applies it in one visible place and
**asserts arithmetically** that its two additive terms cancel out of every daily
spread, so the trade counts depend on the VAT multiplier alone and not on the
guessed markup. The same replay is run on raw wholesale as well, so how much of
the answer rests on the guess is reported (110 trades all-in, 88 raw).
**DST is handled by construction, not by a special case.** The fetcher iterates
in UTC and renders through `price.brussels()`, so **2025-10-26 comes out 25
points and 2026-03-29 comes out 23** — verified by counting per Brussels day
rather than asserting 24, and by checking that October's 02:00 appears twice at
`+02:00` and `+01:00` with two different prices while March's 02:00 does not
appear at all. The window is bounded at *local* midnight: bounding it at 00:00
UTC produced a 22-hour first day, and a 24-hour day silently missing from the
end, both of which the point-and-gap accounting caught.
**The result, at the shipped default, whatever it is.** At
`cycle_cost_threshold_eur` **0.10** widened by FCST-03's no-weather **1.5×** to a
required 0.1500 EUR/kWh, the real year completes **110 round trips** and is
**net-positive at +24.88 EUR** of benefit net of the cycle cost (2 kWh a trade).
Unwidened it completes 173. **Nothing was tuned:** the shipped default and the
2.5/1.5 factors are untouched, and every figure comes out of the same `decide()`.
**What the product owner's open decision now has under it.** The four
independent reports that a static day/night spread of ~0.0840 does not clear
0.10 **all stand** — a static-tariff site still never trades. What was missing
was the other half: a *dynamic* site's daily spread clears the widened
requirement on **121 of 365 days**, median daily all-in spread **0.1130**. The
threshold at which a median day would clear is **0.0754**, and at that value the
same year completes 156 trips for +35.42 EUR. Both numbers are reported; neither
is applied.
**And the synthetic year stays, because the real one cannot grade §17.** Not one
of the 365 real days has a negative *dearest* hour — 372 negative wholesale hours
and 0 days where the strategy would have sold into one — so "0 discharges into a
negative-price hour" over real prices is a count of something that could not
happen, and deleting the guard would not change it. Section 7B asserts that
degeneracy instead of quietly reporting the zero; section 7's generator is still
the only thing here that puts a negative price where the strategy would sell.
**Size, measured rather than assumed.** Plain JSON: 893 KB in the working tree
but **74.6 KB as a git object**, against 56.5 KB for a committed `.json.gz`. Git
already compresses blobs, so gzipping saves 18 KB and costs a reviewer the
ability to read the provenance block — including the licence claim they are
meant to check — in the diff. Plain, and `load()` still accepts `.gz` for a
future dataset that earns it.
**Nothing is wired into the add-on.** `generic_dayahead` remains the live fetch
path; nothing under `app/` imports the script, and the dataset is for replay.
**The capacity-peak guard's forced discharge now has an EMS-side energy floor
(SAFETY-24).** SAFETY-07 can command a discharge to hold a billed quarter down,
and nothing below it consulted the state of charge — `soc_floor` belongs to the
goodwe_controller/scenarios/epex-be-dayahead-2025-09-01_2026-09-01.json
+43849
View File
File diff suppressed because it is too large Load Diff
goodwe_controller/scenarios/fetch_epex_be.py
+551
View File
@@ -0,0 +1,551 @@
#!/usr/bin/env python3
"""DATA-02: fetch 12 months of EPEX BE day-ahead prices, in §9.3's shape.
python3 scenarios/fetch_epex_be.py --start 2025-09-01 --end 2026-09-01 \
--out scenarios/epex-be-dayahead-2025-09-01_2026-09-01.json
Run from `goodwe_controller/`. Rewrites the output file from scratch; the
committed dataset is exactly what this produces, and `--verify` re-derives every
claim the file makes about itself without going to the network.
WHAT THIS IS NOT
================
NOT wired into the add-on, and it must never be. `price.py`'s `GenericDayAhead`
is the live path and it has its own egress check; this is a one-shot archival
fetch that produces a file for REPLAY, run by hand by a developer. Nothing under
`app/` imports it.
THE SOURCE, AND WHY THIS ONE
============================
`https://api.energy-charts.info/price?bzn=BE` - Fraunhofer ISE's Energy-Charts
API. No key, no registration.
⚠️ THE LICENCE WAS CHECKED BEFORE ANY DATA WAS COMMITTED, because for this
source it is NOT uniform across bidding zones. The API's own OpenAPI document
(`https://api.energy-charts.info/openapi.json`, /price) splits the zones in two:
"The data for the following bidding zones is licensed as CC BY 4.0 from
Bundesnetzagentur | SMARD.de and is published without changes:
AT, BE (Belgium), CH, CZ, DE-LU, ..."
"The data for the other bidding zones is for private and internal use only.
The utilization of any data, whether in its raw or derived form, for
external or commercial purposes is expressly prohibited. Should you require
licensing for market-related data, please direct your inquiries to the
original data providers, including but not limited to EPEX SPOT SE."
BE is in the FIRST list, so redistribution is permitted with attribution and the
dataset is committed rather than left as a fetch script and a checksum. Every
response also carries the licence per-response in `license_info`, and this
script REFUSES to write a file if that field ever stops saying CC BY 4.0 - a
licence checked once by a human and assumed forever is a licence that changes
silently. Had BE been in the second list, the correct output of this card would
have been this script and a checksum and no data.
Attribution, carried in the dataset's own `provenance` block and reproduced
here: data CC BY 4.0, Bundesnetzagentur | SMARD.de, retrieved via Energy-Charts
(Fraunhofer ISE), https://energy-charts.info, published without changes.
Cross-checked against the upstream on 2026-08-31: SMARD's own chart_data series
`4996/DE` over 2026-04-10 matched this API's BE series on all 96 quarter-hours
with a maximum absolute difference of 0.00 EUR/MWh. "Published without changes"
is a claim worth checking once; `--verify` does not repeat it, because it is a
claim about the SOURCE and not about the file.
RAW WHOLESALE, AND THE ASSUMPTION THAT IS NOT BAKED IN
======================================================
⚠️ WHAT COMES BACK IS RAW WHOLESALE, EUR/MWh, DAY-AHEAD AUCTION CLEARING PRICE.
It is NOT all-in: no supplier markup, no distribution or transmission tariff, no
levies, no VAT. §9.3's canonical point is all-in, and TARIFF-02's whole trap is
a figure that already contains a component being handed to something that adds
it again.
So this file does NOT add one. `eur_per_kwh` in the output is the wholesale
figure converted from EUR/MWh, `provenance.all_in` is `false` beside it, and the
markup/tax assumption a consumer needs lives in `provenance.all_in_assumption`
as NUMBERS TO APPLY rather than numbers already applied. test_arbitrage.py
section 7 applies them where a reader can watch it happen.
The assumption's SHAPE is the part that is not arbitrary, and it decides which
conclusions depend on the guessed numbers at all:
all_in(h) = (wholesale(h) + markup + network_and_levies) * (1 + vat)
* The two ADDITIVE terms cancel out of a spread: max - min is unchanged by
them. So how often a day clears `cycle_cost_threshold_eur` does NOT depend
on the markup guess - only on the VAT multiplier.
* They do NOT cancel out of §17's negative-price guard, which compares ONE
hour's all-in price against zero. A wholesale hour at -0.02 EUR/kWh is
comfortably positive all-in once ~0.12 of fixed charges is added.
Both are asserted arithmetically in section 7 rather than believed, and the
trade count is reported on the raw spread as well, so a reader can see exactly
how much of the answer rests on the guess.
The numbers themselves (2026 Belgian residential dynamic contract, order of
magnitude) are a STATED GUESS, labelled as one everywhere they appear. They are
not a measurement and this card had nothing to measure them with.
HOURS, QUARTER-HOURS, AND THE SEAM IN THE MIDDLE OF THE YEAR
============================================================
⚠️ THE SOURCE CHANGES RESOLUTION PART-WAY THROUGH ANY RECENT 12 MONTHS. EPEX
day-ahead moved to a 15-minute market time unit in October 2025 and the feed
follows: 2025-09-10 comes back as 24 hourly points, 2025-12-10 as 96
quarter-hourly ones. An importer that assumed either one silently loses three
quarters of the year, or reads a quarter as an hour.
The canonical format is hourly, so quarter-hours are aggregated: an hour's price
is the unweighted mean of its four quarters. That is a STATED APPROXIMATION of
the hourly product, not the hourly product, and it errs in the direction that
matters here - averaging discards intra-hour variation, so the observed spread
is a LOWER bound on what a 15-minute strategy would see.
`provenance.resolution` records how many hours came from each source shape.
⚠️ AND A PARTIAL HOUR IS DROPPED, NOT AVERAGED OVER WHAT ARRIVED. An hour with
three of four quarters is an hour nobody published a complete price for; the
mean of three would be an invented number carrying no flag. Dropped hours are
listed in `provenance.gaps` with their timestamps, because a missing hour in a
feed is not a zero price, and a zero price is exactly the value that would make
everything look free.
SIZE: WHY THIS IS PLAIN JSON AND NOT A `.gz`, MEASURED
======================================================
The directory's existing captures are gzipped, so the default would have been to
gzip this too. Measured on the committed file instead of assumed:
working tree, plain JSON 893 171 bytes
what GIT actually stores for it (zlib, loose object) 74 590 bytes
what git would store for a committed `.json.gz` 56 504 bytes
⚠️ GIT ALREADY COMPRESSES BLOBS, so gzipping buys 18 KB - 24 % of an object
that is already 75 KB, in a repository whose three existing P1 captures cost
27 KB, 136 KB and 140 KB. Eighteen kilobytes is not worth making the provenance
block unreadable in a pull request, and the licence claim in that block is
precisely the thing a reviewer of this card has to read and check. Plain JSON
also greps, diffs line-by-line when a future refetch changes something, and is
what `--verify` opens. `load()` still handles `.gz` for a future dataset large
enough to want it; this one is not.
The P1 captures are gzipped for the opposite reason and it is not an
inconsistency: they are hundreds of thousands of telegram lines that no human
reads, and there is no metadata in them to review.
TIME
====
Every instant is carried as UTC and rendered with `app.price.brussels()`, the
module that already owns the EU DST rule without a tz database. There is no
second answer to that question here and there must not be: the point of
`brussels()` is that the 02:00-03:00 hour which happens twice in October is two
instants, and a 24-slot local-hour importer collapses them.
So a year is 8760 hours only if it contains no transition, or contains both and
they cancel - 25 in October, 23 in March. `--verify` therefore asserts the
PER-DAY counts, not the total, because the total is exactly the number a broken
importer also produces.
"""
import argparse
import gzip
import hashlib
import json
import os
import sys
import time
import urllib.error
import urllib.request
from collections import Counter, defaultdict
from datetime import datetime, timedelta, timezone
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from app.price import brussels # noqa: E402
API = "https://api.energy-charts.info/price"
BZN = "BE"
LICENCE_MUST_CONTAIN = "CC BY 4.0"
UTC = timezone.utc
ATTRIBUTION = ("Data licensed CC BY 4.0 by Bundesnetzagentur | SMARD.de, "
"retrieved via the Energy-Charts API (Fraunhofer ISE), "
"https://energy-charts.info. Published without changes.")
# ⚠️ A GUESS, LABELLED AS ONE, AND APPLIED BY THE CONSUMER - NOT HERE.
# Order-of-magnitude 2026 Belgian residential dynamic contract. Nothing in this
# repository measured any part of it.
ALL_IN_ASSUMPTION = {
"form": "all_in = (wholesale + markup + network_and_levies) * (1 + vat)",
"markup_eur_per_kwh": 0.020,
"network_and_levies_eur_per_kwh": 0.100,
"vat_fraction": 0.06,
"status": "ASSUMPTION, NOT MEASURED. Stated so a consumer applies it "
"deliberately; the committed eur_per_kwh has none of it applied.",
"note": "The two additive terms cancel out of any spread, so an arbitrage "
"trade count depends on vat_fraction only. They do NOT cancel out "
"of the negative-price guard (17), which compares one hour's "
"all-in price against zero.",
}
PACE_S = 3.0 # between chunks, unconditionally
RETRIES = 6
def _get(url: str) -> dict:
"""One GET, backing off on 429 and 5xx.
⚠️ THIS IS A FREE PUBLIC SERVICE AND IT RATE-LIMITS. The first run of this
script was refused with 429 on its fifth chunk. Backing off is the polite
behaviour; it is also the correct one, because the alternative - catching
the error and carrying on - would produce a file that is silently missing a
month and says nothing about it. A refused request is not an empty month.
"""
req = urllib.request.Request(
url,
headers={"User-Agent": "goodwe-addon/DATA-02 (one-shot archival fetch)"})
wait = 5.0
for attempt in range(1, RETRIES + 1):
try:
with urllib.request.urlopen(req, timeout=90) as r:
return json.loads(r.read().decode("utf-8"))
except urllib.error.HTTPError as e:
if e.code not in (429, 500, 502, 503, 504) or attempt == RETRIES:
raise
print(f" HTTP {e.code}; backing off {wait:.0f}s "
f"(attempt {attempt}/{RETRIES})", file=sys.stderr)
time.sleep(wait)
wait *= 2
raise SystemExit(f"unreachable: {url}")
def local_midnight_utc(day: str) -> datetime:
"""The INSTANT at which Brussels midnight begins on `day` (YYYY-MM-DD).
⚠️ THE WINDOW IS LOCAL, NOT UTC, AND THE FIRST RUN OF THIS SCRIPT PROVED WHY.
Bounding it at 00:00 UTC starts the year at 02:00 CEST, which makes the first
Brussels day 22 hours long - and 22 is not one of the two lengths a
transition produces, so `report()` refused the file. A dataset whose first
day is short is a dataset whose "days per year" arithmetic is quietly wrong
everywhere downstream.
Derived from `brussels()` by trying the only two offsets Belgium has and
keeping the one that round-trips, rather than by restating the EU rule here.
"""
for hours in (1, 2):
guess = datetime.fromisoformat(f"{day}T00:00:00").replace(
tzinfo=timezone(timedelta(hours=hours))).astimezone(UTC)
if brussels(guess).isoformat() == f"{day}T00:00:00+0{hours}:00":
return guess
raise SystemExit(f"no Brussels midnight for {day!r} - brussels() disagrees "
f"with itself, which is a bug in price.py, not here.")
def _months(start: datetime, end: datetime):
"""[start, end) in month-sized chunks, so a bad chunk names its own window."""
a = start
while a < end:
b = min(end, (a.replace(day=1) + timedelta(days=32)).replace(day=1))
yield a, b
a = b
def fetch(start: datetime, end: datetime):
"""Raw {unix_second: EUR/MWh} over [start, end), deduplicated.
⚠️ A `null` price is DROPPED, never coerced. The API returns null for an
interval it holds no result for, and `or 0.0` is the version of this line
that ships a year of free electricity.
"""
seen, licences, nulls = {}, Counter(), 0
for a, b in _months(start, end):
# ⚠️ THE API'S `end` IS AN INCLUSIVE LOCAL DAY, AND THE WINDOW'S IS AN
# EXCLUSIVE INSTANT. Ask for the day containing the LAST hour of the
# chunk, derived from the instant rather than by subtracting a calendar
# day from the bound: `b - 1 day` drops the final local day whenever the
# window ends at local midnight, which is exactly how this window ends.
# The first run lost 2026-08-31 that way, silently - 8736 points instead
# of 8760 - and the points-plus-gaps accounting in report() is what
# noticed. A day nobody asked for is not a day with no prices.
first = brussels(a).date().isoformat()
last = brussels(b - timedelta(hours=1)).date().isoformat()
url = f"{API}?bzn={BZN}&start={first}&end={last}"
d = _get(url)
lic = str(d.get("license_info", ""))
licences[lic] += 1
if LICENCE_MUST_CONTAIN not in lic:
raise SystemExit(
f"REFUSING TO WRITE. {url} came back under licence {lic!r}, "
f"which does not say {LICENCE_MUST_CONTAIN!r}. The BE zone was "
f"redistributable when this card ran; if that has changed, the "
f"right output is this script and a checksum, not the data.")
unit = str(d.get("unit", ""))
if "MWh" not in unit:
raise SystemExit(
f"REFUSING TO WRITE. {url} reports unit {unit!r}, not EUR/MWh. "
f"A silent unit change is a factor of 1000 on every price.")
for t, p in zip(d["unix_seconds"], d["price"]):
if p is None:
nulls += 1
continue
if start.timestamp() <= t < end.timestamp():
seen[int(t)] = float(p)
print(f" {a.date()} .. {b.date()} {len(d['price']):5d} intervals",
file=sys.stderr)
time.sleep(PACE_S)
return seen, licences, nulls
def to_hourly(raw: dict):
"""Hourly means keyed by UTC hour. Returns (hours, gaps, shape_counts).
An hour is emitted only when every interval the source used for it arrived:
1 of 1 where the source was hourly, 4 of 4 where it was quarter-hourly.
Anything else is a gap and is named as one.
"""
buckets = defaultdict(dict)
for t, p in raw.items():
buckets[t - (t % 3600)][t % 3600] = p
hours, gaps, shapes = {}, [], Counter()
for h in sorted(buckets):
got = buckets[h]
if set(got) == {0}:
shapes["hourly"] += 1
hours[h] = got[0]
elif set(got) == {0, 900, 1800, 2700}:
shapes["quarter_hourly_mean"] += 1
hours[h] = sum(got.values()) / 4.0
else:
gaps.append({
"hour_utc": datetime.fromtimestamp(h, UTC).isoformat(),
"intervals_present": sorted(got),
"why": "incomplete hour - dropped rather than averaged over "
"what arrived; a partial read is not an observation",
})
return hours, gaps, shapes
def canonical(hours: dict) -> list:
"""9.3 points. THREE FIELDS, rendered at the Brussels offset for the instant."""
return [{"start": brussels(datetime.fromtimestamp(h, UTC)).isoformat(),
"eur_per_kwh": round(hours[h] / 1000.0, 6) + 0.0,
"provisional": False}
for h in sorted(hours)]
def expected_hours(start: datetime, end: datetime) -> int:
"""Whole hours in the window. Derived, rather than 8760 typed in."""
return int((end - start).total_seconds() // 3600)
def per_local_day(points) -> Counter:
"""Points per BRUSSELS calendar day - the count that exposes a DST bug."""
n = Counter()
for p in points:
n[p["start"][:10]] += 1
return n
def digest(points) -> str:
"""A checksum over the POINTS, so reformatting the envelope does not move it."""
h = hashlib.sha256()
for p in points:
h.update(f"{p['start']}|{p['eur_per_kwh']:.6f}|{int(p['provisional'])}\n"
.encode())
return h.hexdigest()
def load(path: str) -> dict:
op = gzip.open if path.endswith(".gz") else open
with op(path, "rt", encoding="utf-8") as f:
return json.load(f)
def report(doc, *, start=None, end=None) -> int:
"""Re-derive every claim the file makes about itself. Returns failures."""
pts = doc["prices"]
prov = doc["provenance"]
bad = 0
def ck(name, cond):
nonlocal bad
print(f" {'ok ' if cond else 'FAIL'} {name}")
if not cond:
bad += 1
ck(f"the checksum over the {len(pts)} points matches the recorded one "
f"({prov['sha256_of_points'][:16]}...)",
digest(pts) == prov["sha256_of_points"])
ck("every point has exactly the three canonical fields and nothing else",
all(set(p) == {"start", "eur_per_kwh", "provisional"} for p in pts))
ck("every start carries a UTC offset, and it is +01:00 or +02:00 - a naive "
"timestamp here is a wall clock plus an assumption",
bool(pts) and all(p["start"][-6:] in ("+01:00", "+02:00") for p in pts))
ck("the series is strictly increasing in time, so no instant is duplicated",
all(datetime.fromisoformat(a["start"]) < datetime.fromisoformat(b["start"])
for a, b in zip(pts, pts[1:])))
ck("consecutive points are exactly one hour apart AS INSTANTS, so the two "
"02:00s of the October day are two hours rather than one",
all((datetime.fromisoformat(b["start"])
- datetime.fromisoformat(a["start"])) == timedelta(hours=1)
for a, b in zip(pts, pts[1:])))
days = per_local_day(pts)
odd = {d: n for d, n in days.items() if n != 24}
ck(f"exactly two Brussels days are not 24 hours long, and they are the "
f"transitions: {sorted(odd.items())}",
sorted(odd.values()) == [23, 25] and len(odd) == 2)
spring = [d for d, n in odd.items() if n == 23]
autumn = [d for d, n in odd.items() if n == 25]
ck(f"the 23-hour day {spring} is in March and the 25-hour day {autumn} is "
f"in October, which is the direction the EU rule actually runs",
len(spring) == 1 and len(autumn) == 1
and spring[0][5:7] == "03" and autumn[0][5:7] == "10")
if autumn:
twice = [p["start"] for p in pts
if p["start"].startswith(autumn[0] + "T02")]
ck(f"...and 02:00 on {autumn[0]} occurs TWICE at two different offsets "
f"{twice} - exactly the case a 24-slot local-hour array collapses",
len(twice) == 2 and twice[0][-6:] == "+02:00"
and twice[1][-6:] == "+01:00")
ck(f"...and the two carry DIFFERENT prices, so the 25th hour is a real "
f"observation and not the same one written down twice",
len({p["eur_per_kwh"] for p in pts
if p["start"].startswith(autumn[0] + "T02")}) == 2)
if spring:
ck(f"...and 02:00 on {spring[0]} does not occur at all, because that "
f"wall clock reading never happened in Belgium",
not any(p["start"].startswith(spring[0] + "T02") for p in pts))
if start and end:
want = expected_hours(start, end)
ck(f"{len(pts)} points for a window of {want} whole hours, and the "
f"{len(prov['gaps'])} recorded gaps account for the difference "
f"exactly", len(pts) + len(prov["gaps"]) == want)
# ⚠️ A ZERO IS NOT AUTOMATICALLY A DEFECT HERE, AND SAYING IT WAS WOULD BE
# THE WRONG LESSON. The Belgian market really does clear at 0.00 EUR/MWh in
# solar-glut hours; this window contains such hours. What must be impossible
# is a zero that arrived by COERCION - a dropped read turned into a free
# hour. That is excluded structurally rather than by outlawing the value:
# nulls are dropped, incomplete hours become gaps, and the accounting above
# shows points + gaps covering every hour of the window, so every emitted
# point stands for intervals the source actually published.
zeros = [p["start"] for p in pts if p["eur_per_kwh"] == 0.0]
ck(f"the {len(zeros)} hours priced at exactly zero match the count recorded "
f"at build time, so no zero appeared between the fetch and the file",
len(zeros) == prov["exact_zero_hours"])
ck("...and none of them is a coerced read: every emitted hour is backed by "
"published intervals, which is what the points-plus-gaps accounting "
"above establishes",
len(prov["gaps"]) == 0 or all("intervals_present" in g
for g in prov["gaps"]))
ck("the figures are declared RAW WHOLESALE, and the all-in assumption is "
"recorded as an assumption rather than applied",
prov["all_in"] is False
and "ASSUMPTION" in prov["all_in_assumption"]["status"])
return bad
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("--start", default="2025-09-01")
ap.add_argument("--end", default="2026-09-01",
help="exclusive; both bounds are BRUSSELS-local midnights")
ap.add_argument("--out", default=None)
ap.add_argument("--verify", default=None,
help="re-derive the claims of an existing file; no network")
a = ap.parse_args()
if a.verify:
doc = load(a.verify)
print(f"verifying {a.verify}")
s = datetime.fromisoformat(doc["provenance"]["window_utc"]["start"])
e = datetime.fromisoformat(doc["provenance"]["window_utc"]["end"])
bad = report(doc, start=s, end=e)
print("OK" if not bad else f"{bad} FAILED")
return 1 if bad else 0
start = local_midnight_utc(a.start)
end = local_midnight_utc(a.end)
out = a.out or f"scenarios/epex-be-dayahead-{a.start}_{a.end}.json"
print(f"fetching {BZN} day-ahead {start.date()} .. {end.date()}",
file=sys.stderr)
raw, licences, nulls = fetch(start, end)
hours, gaps, shapes = to_hourly(raw)
pts = canonical(hours)
doc = {
"provenance": {
"card": "DATA-02",
"what": "EPEX BE day-ahead auction clearing prices, hourly",
"source": f"{API}?bzn={BZN} (Energy-Charts API, Fraunhofer ISE)",
"upstream": "Bundesnetzagentur | SMARD.de",
"licence": "CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)",
"licence_checked": "before committing anything; the API's OpenAPI "
"document lists BE among the CC BY 4.0 zones and "
"the remaining zones as private/internal use "
"only, so redistribution of THIS zone is "
"permitted with attribution",
"licence_as_returned": sorted(licences),
"attribution": ATTRIBUTION,
"cross_check": "SMARD chart_data 4996/DE over 2026-04-10 matched "
"this feed on all 96 quarter-hours, max abs "
"difference 0.00 EUR/MWh (2026-08-31)",
"retrieved_utc": datetime.now(UTC).replace(microsecond=0).isoformat(),
"fetched_by": "scenarios/fetch_epex_be.py",
"window_utc": {"start": start.isoformat(), "end": end.isoformat()},
"unit": "EUR per kWh (source publishes EUR/MWh; divided by 1000)",
"all_in": False,
"all_in_note": "RAW WHOLESALE. No supplier markup, no network or "
"transmission tariff, no levies, no VAT. 9.3's "
"canonical point is all-in and this is NOT; the "
"consumer applies the assumption below, visibly.",
"all_in_assumption": ALL_IN_ASSUMPTION,
"resolution": {
"note": "EPEX day-ahead moved to a 15-minute market time unit "
"in Oct 2025, part-way through this window.",
"hours_from_hourly_source": shapes["hourly"],
"hours_from_quarter_hourly_mean": shapes["quarter_hourly_mean"],
"aggregation": "unweighted mean of the four quarters; a stated "
"approximation of the hourly product, and a "
"LOWER bound on the spread a 15-minute strategy "
"would have seen",
},
"timezone": "starts rendered at the Brussels offset by "
"app.price.brussels(); instants carried in UTC, so the "
"transition days come out 23 and 25 points by "
"construction rather than by a special case",
"nulls_dropped": nulls,
"gaps": gaps,
"gaps_note": "hours the source did not publish completely. Dropped, "
"never zero-filled: a missing hour is not a free hour.",
"points": len(pts),
"exact_zero_hours": sum(1 for p in pts if p["eur_per_kwh"] == 0.0),
"negative_hours": sum(1 for p in pts if p["eur_per_kwh"] < 0),
"zero_note": "REAL CLEARINGS AT 0.00 EUR/MWh, not dropped reads. "
"Nulls are dropped and incomplete hours become gaps, "
"so points + gaps == hours in the window and no point "
"stands in for something that failed to arrive.",
"sha256_of_points": digest(pts),
},
"prices": pts,
}
with open(out, "w", encoding="utf-8", newline="\n") as f:
json.dump(doc, f, indent=1, ensure_ascii=False)
f.write("\n")
print(f"wrote {out} {len(pts)} points {os.path.getsize(out)} bytes",
file=sys.stderr)
print(f"verifying {out}")
return 1 if report(doc, start=start, end=end) else 0
if __name__ == "__main__":
raise SystemExit(main())
goodwe_controller/test_arbitrage.py
+284 -32
View File
@@ -20,12 +20,15 @@ WHAT THIS FILE IS DEFENDING, in the order the card's acceptance criteria name it
exist - static_tariff's required `peak_guard_active` interlock and
arbiter.py's clause 2 - both driven end to end, plus the third state
(the guard never ran) that neither of them can express on its own.
AC4 NET-POSITIVE ACROSS 12 MONTHS OF REPLAYED PRICES. ⚠️ SEE SECTION 7. THIS
REPOSITORY HOLDS NO PRICE HISTORY AT ALL - `scenarios/` carries two days
of real METER data and nothing else - so the criterion as written cannot
be met, and it is NOT redefined here into something that can be. What
section 7 does instead is stated there in full, and what it cannot say is
stated with it.
AC4 NET-POSITIVE ACROSS 12 MONTHS OF REPLAYED PRICES. ⚠️ MET AS OF DATA-02,
AND SECTIONS 7 AND 7B DO DIFFERENT HALVES OF IT. 7B replays twelve months
of REAL EPEX BE day-ahead prices - `scenarios/epex-be-dayahead-*.json`,
8760 hours, public data under CC BY 4.0 - through the shipped `decide()`,
and reports the euro figure at the shipped threshold. Section 7 keeps the
SYNTHETIC year, because the real one turns out to be unable to grade §17
at all: not one of its 365 days has a negative DEAREST hour, so a count
of zero negative-price discharges over it grades nothing. 7B asserts that
degeneracy rather than quietly reporting the zero.
THREE RULES THIS FILE HOLDS ITSELF TO, because the reviewer will:
@@ -786,34 +789,39 @@ check("...and it is still ARMED when arbitrage is idle and the loop wins, so "
print()
print("7. AC4 - 12 months of replayed prices, and what could NOT be measured")
# ⚠️ READ THIS BEFORE READING THE NUMBERS BELOW.
# ⚠️ READ THIS BEFORE READING THE NUMBERS BELOW. THIS SECTION IS THE SYNTHETIC
# YEAR, AND IT IS NOT AC4's EVIDENCE ANY MORE - SECTION 7B IS.
#
# The acceptance criterion is "net-positive across 12 months of replayed prices
# in simulation". THIS REPOSITORY CONTAINS NO PRICE HISTORY. `scenarios/` holds
# three gzipped captures and all three are P1 METER data - two days of net power
# from the reference house plus one age capture. There is no EPEX archive, no
# supplier export, and no CSV of a past year anywhere in the tree.
# When OPT-04 shipped, this repository held no price history: `scenarios/`
# carried three gzipped P1 METER captures and nothing else. AC4 was therefore
# reported NOT MET rather than redefined into something passable, which is what
# TARIFF-01 and SAFETY-07 had both been upheld for doing.
#
# TARIFF-01 and SAFETY-07 both hit an unmeetable criterion and both DECLINED to
# redefine it into one they could pass, and both were upheld for it. So:
# DATA-02 pointed out that the missing ingredient was never a capture. EPEX BE
# day-ahead is PUBLIC DATA; the 30-day meter criterion it was being carried
# beside genuinely needs waiting, and this one only ever needed a fetch. Section
# 7B replays the real year and AC4 is met there.
#
# AC4 IS NOT MET, AND IT IS NOT RESTATED. What follows is not a replay of the
# Belgian market; it is a replay of a SYNTHETIC year, and a euro figure from a
# synthetic year grades the generator that produced it, not the strategy.
# ⚠️ SO WHY IS THE SYNTHETIC YEAR STILL HERE? Because the real one cannot grade
# §17, and 7B proves that rather than assuming it. Across all 365 real days, the
# number whose DEAREST hour is priced below zero is ZERO - raw wholesale and
# all-in alike - so "no discharges into a negative hour" over the real year is a
# count of something that never had the chance to happen. Deleting the guard
# would not change it. This section's generator drives a summer midday block
# negative ON PURPOSE, and it is the only thing in this file that puts a
# negative price where the strategy would otherwise sell.
#
# What the replay is FOR is the half of AC4 that does not need real prices. Net
# positivity here is not an empirical outcome to be hoped for - it is a property
# of the decision rule, and the replay's job is to demonstrate that the property
# holds on every trade the strategy actually chose, over a year of varied days
# including negative-price ones. Stated as an invariant:
# What this section establishes is the half of AC4 that does not need real
# prices, and it is a property of the decision rule rather than an outcome to be
# hoped for:
#
# every executed round trip has (sell - buy) >= required >= threshold,
# so its benefit net of the threshold is >= 0, on every day, always.
#
# That is asserted per trade below, over 365 days. What it does NOT establish -
# and no synthetic series could - is how OFTEN a real Belgian year clears the
# threshold, which is the actual open question and the reason the shipped
# default is a product decision rather than a number this card picked.
# That is asserted per trade below, over 365 varied days including negative ones.
# HOW OFTEN a real Belgian year clears the threshold is the other question, it
# is the one the product owner's open decision turns on, and section 7B is the
# only place in this file that answers it.
def synth_day(n):
"""A synthetic all-in day-ahead day. Deterministic, seeded by day number.
@@ -920,12 +928,256 @@ check("the shipped threshold is strictly more selective than the loose one on "
"the same year - the parameter does what it says, at whatever value the "
"owner sets it to",
len(_sh_trades) < len(_trades))
check("AC4 IS REPORTED AS NOT MET rather than redefined: this file says so in "
"section 7's preamble, and there is no price history in the repository "
"to meet it with",
not any(f.endswith((".csv", ".json.gz"))
and "price" in f.lower()
for f in os.listdir(os.path.join(HERE, "scenarios"))))
check("...and the shipped default was NOT moved to make any of this look "
"better: app/arbitrage.py reads the threshold through price_option() and "
"names no number of its own",
"0.10" not in open(os.path.join(HERE, "app", "arbitrage.py"),
encoding="utf-8").read().split('"""', 2)[2])
# --------------------------------------------------------------------------- #
print()
print("7B. AC4 - twelve months of REAL EPEX BE prices, replayed")
# THE DATASET, AND WHAT IT IS NOT.
#
# `scenarios/epex-be-dayahead-2025-09-01_2026-09-01.json` - 8760 hourly Belgian
# day-ahead auction prices, 2025-09-01 to 2026-09-01 Brussels-local, fetched by
# `scenarios/fetch_epex_be.py` from Fraunhofer ISE's Energy-Charts API, which
# publishes the BE zone under CC BY 4.0 from Bundesnetzagentur | SMARD.de. The
# licence was checked BEFORE the data was committed, because the same API serves
# other zones under private-use-only terms; the script refuses to write a file
# whose per-response `license_info` stops saying CC BY 4.0. Provenance,
# attribution and the DST accounting live in the file's own `provenance` block
# and `fetch_epex_be.py --verify` re-derives every claim in it.
#
# ⚠️ THE FIGURES IN THAT FILE ARE RAW WHOLESALE, NOT ALL-IN, AND THIS SECTION IS
# WHERE THE DIFFERENCE IS MADE VISIBLE. §9.3's canonical point is all-in; the
# committed file says `"all_in": false` and carries the markup/tax assumption as
# NUMBERS TO APPLY rather than numbers already applied - TARIFF-02's
# double-counting trap, avoided by never baking the guess in. `_all_in()` below
# applies it in one place where a reader can watch it happen, and both replays
# are run, so exactly how much of the answer rests on the guess is reported
# rather than hidden.
import json # noqa: E402
from collections import defaultdict # noqa: E402
_DS_PATH = os.path.join(HERE, "scenarios",
"epex-be-dayahead-2025-09-01_2026-09-01.json")
_DS = derived(lambda: json.load(open(_DS_PATH, encoding="utf-8")))
_ASSUME = _DS["provenance"]["all_in_assumption"]
def _all_in(wholesale):
"""The stated assumption, applied HERE and nowhere in the committed data.
all_in = (wholesale + markup + network_and_levies) * (1 + vat), with the
coefficients read from the dataset's own provenance block rather than
retyped - a second copy is a second thing to disagree with the file.
"""
return round((wholesale + _ASSUME["markup_eur_per_kwh"]
+ _ASSUME["network_and_levies_eur_per_kwh"])
* (1.0 + _ASSUME["vat_fraction"]), 6)
def _days(price_of):
"""The dataset as Brussels calendar days of `PricePoint`s, in order.
⚠️ GROUPED BY THE LOCAL DATE IN THE `start` STRING, NOT BY SLICING 24 AT A
TIME. Two of these days are not 24 hours long, which is the whole DST check,
and a fixed stride would put October's 25th hour into the next day and
misalign every day after it.
"""
out = defaultdict(list)
for p in _DS["prices"]:
out[p["start"][:10]].append(
PricePoint(start=datetime.fromisoformat(p["start"]),
eur_per_kwh=price_of(p["eur_per_kwh"]),
provisional=bool(p["provisional"])))
return [out[k] for k in sorted(out)]
_AI_DAYS = derived(lambda: _days(_all_in))
_RAW_DAYS = derived(lambda: _days(lambda w: w))
# ---- the dataset is not degenerate, asserted BEFORE anything is graded ------
_lens = [len(d) for d in _AI_DAYS]
check(f"the replay has {len(_AI_DAYS)} Brussels days and {sum(_lens)} hourly "
f"points, which is a full year rather than a fortnight repeated",
len(_AI_DAYS) == 365 and sum(_lens) == 8760)
check(f"...and the day lengths are {sorted(set(_lens))}, so BOTH DST days are "
f"in it: an importer that produced 24 everywhere would have lost an hour "
f"in October and invented one in March",
sorted(set(_lens)) == [23, 24, 25]
and _lens.count(23) == 1 and _lens.count(25) == 1)
_dst_day = [d for d in _AI_DAYS if len(d) == 25][0]
check("...and the 25-hour day's two 02:00 points are two different INSTANTS at "
"two different offsets, which is what `at()` has to tell apart",
len({p.start for p in _dst_day}) == 25
and {p.start.utcoffset() for p in _dst_day}
== {timedelta(hours=1), timedelta(hours=2)})
_ai_spreads = sorted(round(max(p.eur_per_kwh for p in d)
- min(p.eur_per_kwh for p in d), 6)
for d in _AI_DAYS)
_raw_spreads = sorted(round(max(p.eur_per_kwh for p in d)
- min(p.eur_per_kwh for p in d), 6)
for d in _RAW_DAYS)
_req_shipped = required_spread(_shipped, NO_WEATHER)
_clears = [s for s in _ai_spreads if s > _req_shipped]
check(f"...and the year's daily spreads STRADDLE the shipped requirement of "
f"{_req_shipped:.4f} EUR/kWh - {len(_clears)} of {len(_ai_spreads)} days "
f"clear it - so the threshold is grading something. A year in which every "
f"day cleared, or none did, would grade nothing at any value",
0 < len(_clears) < len(_ai_spreads))
check(f"...and the spreads are not one repeated number: median "
f"{_ai_spreads[len(_ai_spreads) // 2]:.4f}, min {_ai_spreads[0]:.4f}, "
f"max {_ai_spreads[-1]:.4f}",
len(set(_ai_spreads)) > 300 and _ai_spreads[-1] > 4 * _ai_spreads[0])
# ---- the assumption's arithmetic, asserted rather than asserted-about -------
check("the ADDITIVE half of the all-in assumption cancels out of every daily "
"spread, so the trade counts below do not depend on the markup guess at "
"all - only on the VAT multiplier",
all(near(a, r * (1.0 + _ASSUME["vat_fraction"]), 1e-6)
for a, r in zip(_ai_spreads, _raw_spreads)))
# ---- ⚠️ THE DEGENERACY THAT KEEPS SECTION 7 ALIVE --------------------------
_raw_neg_h = sum(1 for d in _RAW_DAYS for p in d if p.eur_per_kwh < 0)
_ai_neg_h = sum(1 for d in _AI_DAYS for p in d if p.eur_per_kwh < 0)
_raw_neg_dear = sum(1 for d in _RAW_DAYS if max(p.eur_per_kwh for p in d) < 0)
_ai_neg_dear = sum(1 for d in _AI_DAYS if max(p.eur_per_kwh for p in d) < 0)
check(f"the real year DOES contain negative hours - {_raw_neg_h} of them at "
f"wholesale, {_ai_neg_h} still negative once the assumed markup and "
f"charges are added - so §17 is a live case on this market and not a "
f"hypothetical", _raw_neg_h > 100 and _ai_neg_h > 0)
check(f"BUT NOT ONE OF THE 365 DAYS HAS A NEGATIVE DEAREST HOUR "
f"({_raw_neg_dear} raw, {_ai_neg_dear} all-in), and the guard is on the "
f"hour being SOLD INTO. So a count of zero negative-price discharges over "
f"this year would be a count of something that could not happen, and "
f"section 7's synthetic year stays because it is the only thing here that "
f"puts a negative price where the strategy would sell",
_raw_neg_dear == 0 and _ai_neg_dear == 0)
def replay_real(days, threshold, unc, *, soc0=50.0, floor=20.0, ceiling=95.0):
"""The shipped `decide()` over the real year. Returns the ledger, nothing else.
One `TariffPlan` per Brussels day, so "the cheapest hour" means the cheapest
hour of that day - the same shape as section 7's replay, and the same shape
a site with a day-ahead feed actually sees.
"""
soc = soc0
trades, holdings, bad, neg_sells = [], [], [], 0
for day in days:
plan = TariffPlan(
PriceRead.series(day, PriceProduct.DAY_AHEAD, "generic_dayahead",
day[0].start), None, "provider")
open_buy = None
for p in day:
d = decide(plan, p.start + timedelta(minutes=30),
peak_guard_active=False, soc=soc,
min_spread_eur_kwh=threshold, uncertainty=unc,
charge_w=MAX_W, discharge_w=MAX_W,
soc_floor_pct=floor, soc_ceiling_pct=ceiling)
if d.action is None:
continue
if d.spread_eur_kwh is None or d.required_eur_kwh is None:
bad.append(p.start)
if d.action == CHARGE:
soc = min(ceiling, soc + STEP_PCT)
open_buy = (p.eur_per_kwh, d)
else:
if p.eur_per_kwh < 0:
neg_sells += 1
soc = max(floor, soc - STEP_PCT)
if open_buy is not None:
trades.append((open_buy[0], p.eur_per_kwh, d))
open_buy = None
else:
holdings.append(p.start)
return trades, holdings, bad, neg_sells
# ⚠️ AT THE SHIPPED THRESHOLD, UNTOUCHED, AND WIDENED BY FCST-03's NO-WEATHER
# FACTOR - the harsher of the two operating states, so the figure reported is
# the conservative one.
_r_trades, _r_held, _r_bad, _r_neg = derived(
lambda: replay_real(_AI_DAYS, _shipped, NO_WEATHER)) or (None,) * 4
_r_kwh = MAX_W / 1000.0
_r_net = round(sum((s - b - _shipped) * _r_kwh for b, s, _ in _r_trades), 2)
check(f"the real year traded at the SHIPPED threshold: {len(_r_trades)} "
f"completed round trips over 365 days of Belgian day-ahead prices - so "
f"the criterion is being graded on trades that happened, not on an empty "
f"ledger that is net-positive by vacuity", len(_r_trades) > 0)
check("...and every one of them carries the spread and the widened requirement "
"it cleared, so the ledger is auditable rather than a count", _r_bad == [])
check(f"...and it did NOT trade every day ({len(_r_trades)} round trips over "
f"365 days), so the threshold selected rather than waved everything "
f"through", len(_r_trades) < 365)
_r_losses = [(b, s) for b, s, _ in _r_trades if s - b < _shipped]
check(f"AC4: every completed round trip on the REAL year cleared the shipped "
f"{_shipped} EUR/kWh - {len(_r_losses)} exceptions of {len(_r_trades)}",
_r_losses == [])
_r_under = [d.spread_eur_kwh for _b, _s, d in _r_trades
if d.spread_eur_kwh < d.required_eur_kwh]
check(f"...and each cleared its own WIDENED requirement of "
f"{_req_shipped:.4f} EUR/kWh, which is the stricter statement §8.3 asks "
f"for", _r_under == [])
check(f"AC4 IS MET: twelve months of REAL replayed EPEX BE prices are "
f"NET-POSITIVE at the shipped default - {_r_net:+.2f} EUR of benefit net "
f"of the cycle cost over {len(_r_trades)} round trips of {_r_kwh:.1f} kWh "
f"each, accumulated from the trades the strategy actually chose",
_r_net > 0)
# ---- what the owner's open decision actually turns on -----------------------
# ⚠️ REPORTED, NOT SETTLED. `cycle_cost_threshold_eur` and FCST-03's 2.5/1.5 are
# the product owner's decision. DATA-02 supplies the input and changes nothing:
# every number below comes out of the same untouched `decide()`.
_r_calm = derived(lambda: replay_real(_AI_DAYS, _shipped, frozenset())[0])
_r_raw = derived(lambda: replay_real(_RAW_DAYS, _shipped, NO_WEATHER)[0])
# The threshold at which the MEDIAN day of this year clears its widened
# requirement. Derived from the observed spreads, not chosen.
_median_spread = _ai_spreads[len(_ai_spreads) // 2]
_suggested = round(_median_spread / widen_factor(NO_WEATHER), 4)
_r_sugg, _, _, _ = derived(
lambda: replay_real(_AI_DAYS, _suggested, NO_WEATHER)) or (None,) * 4
_sugg_net = round(sum((s - b - _suggested) * _r_kwh for b, s, _ in _r_sugg), 2)
print(f" [report] SHIPPED {_shipped} EUR/kWh, widened {widen_factor(NO_WEATHER)}x "
f"to {_req_shipped:.4f}: {len(_r_trades)} round trips, {_r_net:+.2f} EUR "
f"over the year. Unwidened (weather present): {len(_r_calm)} trips. On "
f"RAW wholesale instead of the assumed all-in: {len(_r_raw)} trips, so "
f"the markup/VAT guess moves the count by "
f"{len(_r_trades) - len(_r_raw):+d}.")
print(f" [report] the year's MEDIAN daily all-in spread is "
f"{_median_spread:.4f} EUR/kWh, so the threshold at which a median day "
f"clears its widened requirement is {_suggested} - and at that value the "
f"same year completes {len(_r_sugg)} round trips for {_sugg_net:+.2f} "
f"EUR. THE SHIPPED DEFAULT IS NOT CHANGED BY THIS CARD; both numbers are "
f"reported so the owner can decide between them.")
print(f" [report] and the four-developer finding stands unchanged: a STATIC "
f"day/night spread of ~0.0840 EUR/kWh still does not clear "
f"{_req_shipped:.4f}, so a static-tariff site never trades at the shipped "
f"default. What this year adds is that a DYNAMIC site does - on "
f"{len(_clears)} of its 365 days the spread was there to be taken.")
check("the loose threshold is strictly less selective than the shipped one on "
"the SAME real year - the parameter does what it says at whatever value "
"the owner sets, and neither run tuned anything",
len(_r_sugg) > len(_r_trades) > 0)
check(f"§17 held over the real year too ({_r_neg} discharges into a negative "
f"hour) - but see the degeneracy check above: on this data that zero is "
f"structural, and section 7's synthetic year is what actually grades the "
f"guard", _r_neg == 0)
check("the dataset carries its own provenance, states it is NOT all-in, and "
"keeps the markup/tax figures as an assumption rather than applying "
"them - the TARIFF-02 trap, in data form",
_DS["provenance"]["all_in"] is False
and "ASSUMPTION" in _ASSUME["status"]
and _DS["provenance"]["licence"].startswith("CC BY 4.0"))
# --------------------------------------------------------------------------- #