Commit Graph
8 Commits
Author SHA1 Message Date
adminandClaude Opus 5 e6c45a61e7 TARIFF-10: configuring a price provider is what fetches prices
checks / unit (pull_request) Successful in 7m19s
amain() started the background tariff refresh only when arbitrage_enabled was
on, and that option ships false. run_tariff_refresh is the only caller of
resolve_tariff(), which is the only caller of price_history.carry() - so on a
shipped install with a price provider configured nothing was ever fetched: no
plan resolved, /data/price_history.json never created, the Price history status
row reading `empty` for the life of the install, and TARIFF-09's short-horizon
warning unable to fire because there was no fetch to warn about. Measured over
a 40 h two-instance soak on one feed: 0 requests and `present: false` across all
802 samples, against a resolved 132-point plan on the instance with arbitrage
armed.

git log -S puts the refresh task, its arbitrage.enabled() guard AND the DOCS.md
sentence saying a configured provider still fetches in the same commit - OPT-04,
bdd7f18; config.yaml has carried the claim since TARIFF-02, 852fc7c. The prose
and the code disagreed from the moment both existed and nothing compared them.
config.yaml and both translations additionally carried a claim that had rotted
the other way - "nothing consumes the price series yet", untrue since OPT-04
shipped the consumer. All four surfaces now say what the add-on does, and a
check pairs DOCS against the DRIVEN behaviour so they cannot drift again.

The gate is tariff_refresh_wanted() = provider_configured(opts) or
arbitrage.enabled(opts), and the `or` is a STRICT WIDENING: every configuration
that started the task before still starts it. The second term is not there for
symmetry. price_provider: off + a static_profile + arbitrage_enabled: true is a
configuration validate.py explicitly ACCEPTS and it trades - measured, at
cycle_cost_threshold_eur 0.05 on a 0.20/0.284 day/night profile,
ARBITRAGE_CHARGE -5000 W. Gating on the provider alone would leave that site's
tariff_plan at None forever and answer "no tariff plan has been resolved yet" on
every cycle, silently disarming a strategy the UI says is running.

Nothing acquires a periodic HTTPS call. With price_provider off, resolve_tariff()
answers out of the static profile without opening a socket (driven and counted:
0 requests), and the true shipped default - no provider, no trading - starts no
task at all. What decides whether a request leaves the machine is price_provider
and only that. The literal reading of the ticket's AC2 - drop the `or` - is one
token and is recorded in DECISIONS §3 with what it costs, for the owner.

AND NOTHING TRADES THAT DID NOT TRADE BEFORE. cycle()'s
`if arbitrage.enabled(self.o):` is untouched, and that is graded directly rather
than inferred from the gate having moved: Controller.cycle() is driven on a
resolved provider plan with the peak guard asserted clear, and arbitrage_enabled
is the only option that moves between the rows - false/false give +180 W
("loop -> 180 W") on both the cheap-now and dear-now plans, true/true give
-5000 W and +5000 W ("arbitrage -> ...").

TariffPlan.cheap_now() is deliberately NOT wired, decided rather than inherited
(DECISIONS §4). A provider-backed plan now exists on every configured site, so
Cheap window could read it - and Cheap window feeds the maintenance cycle's grid
top-up, the one place a price moves energy without trading being armed. On a
provider-only site static_status() returns False on every cycle, so adopting it
would take a paid grid top-up from never to sometimes. self.cheap stays on
static_status(); what adoption would take is listed rather than left implied.

THE CHECK THAT WOULD HAVE CAUGHT THIS is test_price.py section 13, and it grades
REACHABILITY FROM A CONFIGURATION rather than the shape of an `if` - the `if` was
exactly what somebody meant to write. amain()'s guards are EVALUATED against real
option dicts, whatever they are spelled as; the call graph under app/ is walked
to see whether the tasks those guards start arrive at resolve_tariff(), wherever
it has moved to; and then the fetch is DRIVEN with a counting transport, because
a graph is an argument and a counted request is an observation. What the
instrument cannot see is printed on every run, largest hole first.

Evidence: 29/29 green. 13 mutations in 14 runs - M10 re-run as M10b once it was
closed - each applied to a fresh cp -a of the clean worktree rather than
reverted (MUTATION.md habit 18), purged, :ro, -B. The
shipped defect - the gate back at arbitrage.enabled(opts), -4 B - is red on
seven named checks. Dropping the `or` (-27 B) is red on the strict-widening row;
removing cycle()'s trading gate (+8 B) is red on both AC3 rows. The instrument
is mutated too (habit 16): _reaches() always True and _task_sites() returning []
are both +0 B and both red.

Two findings are recorded rather than rounded off. provider_id() dropping
.strip() (-8 B) was a real SURVIVOR across four price suites - nothing fed a
padded provider id - and is closed in this commit by the padded-id pair; and the
first draft of the AC3 fixture fed five minutes of P1 telegrams, so whether the
peak guard was blind depended on where the wall clock sat inside the billing
quarter and the armed baseline failed to trade on some runs. It is 900 s now,
and the guard's clear state is a check rather than an assumption.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-12 20:58:48 +02:00
Claude Opus 5 17b7b447c4 TARIFF-09: ask the day-ahead feed for the horizon it was handed
checks / unit (pull_request) Successful in 7m10s
`generic_dayahead.fetch()` accepted `since` and `until` and requested the
bare `?bzn=BE`, which answers ONE Brussels day. Everything past today was
then filled from yesterday's profile by FCST-04, flagged provisional, and
§8.3 drops provisional hours from the trade set -- so the optimiser never
saw tomorrow as firm, at any hour of any day, and nothing reported it.

Measured on 4257e38 at 2026-09-10T09:00+02:00, 48 h horizon, live feed,
yesterday seeded through the shipped read path:

    60 firm points (15 whole hours) + 33 carried hours of 48 requested
    carried: 24 h on 2026-09-11, 9 h on 2026-09-12
    arbitrage(0.10): buy 11:45 and sell 19:45 -- BOTH on 2026-09-10
    firm local days = [2026-09-10]

The ceiling was structural: with a one-day window, at hour H a 48 h horizon
is at most 24-H firm and 24+H carried. At 23:00 that is 1 and 47, and the
remaining firm spread (0.0464 EUR/kWh) no longer clears the default 0.10
threshold, so arbitrage is refused outright. FCST-04's fill covers the
window until the ~13:00-14:00 CET auction; it was never meant to cover
every hour of every day.

The data exists and the endpoint serves it: verified 2026-09-10T07:45:49Z,
`?bzn=BE&start=2026-09-09&end=2026-09-10` returns 192 quarter-hours across
two local days. Committed unmodified beside TARIFF-08's captures, with the
URL, retrieval time and sha256 recorded and RE-DERIVED by the replay.

THE RANGE IS DERIVED ON THE BRUSSELS CALENDAR, not the UTC one. 22:30 UTC
in summer is already tomorrow in Belgium, so a UTC-derived start asks for
the day the controller has just left and drops the day it is standing in,
for the last one or two hours of every day. That is `_bdate()`'s lesson one
layer up with a worse consequence: `_bdate()` misfiles an hour, this never
fetches it. `until` is exclusive, so local midnight belongs to the day
before it; an inverted window collapses to one day rather than a backwards
range. Both ends are graded and both are driven red by mutation.

After, same code path and window, a feed serving both days: 39 firm hours
and 9 carried (was 15 and 33), firm across TWO Brussels days, and the
permitted cycle buys on one day and sells on the next.

`price_feed_url` IS THE EXACT URL AND THE PLUGIN APPENDS NOTHING. `start`/
`end` are this feed's spelling and the provider declares a second host
whose parameters are different, so merging would produce a request nobody
typed and the short horizon anyway. The cost is real, so it is not silent:
a WARNING on the shipped logger at fetch time names the consequence and the
start..end that would otherwise have been asked for. `dated_feed_url()`
refuses to append to a base that already carries a query string, so the
blind-append shape cannot come back by edit.
DECISIONS/TARIFF-09-the-range-and-the-override.md argues the alternative.

Before publication nothing regresses and it is graded, not assumed: at
09:46 CEST the two-day request came back byte-identical to the one-day
body, and that stays a `series` of what exists with FCST-04 filling the
rest. A `no_data` there would switch a working site's arbitrage off every
morning -- the mutation that adds that rule is red by name.

The plugin contract is still NOT enforced: three shipped plugins point at
endpoints nobody has verified, and a rule obliging them to pass a range
would be satisfied by an invented parameter name -- TARIFF-08's defect
moved from the response to the request. Which plugins consume the range is
pinned as a literal instead.

11 mutations, all red, no survivors; each applied to a fresh copy of the
tree rather than reverted, through tools/mutation_run.sh. One is +0 B (a
single price digit inside the committed capture) and the re-derived sha256
catches it. MUTATION.md habit 20 records why a fixture could never have
reached this: a captured body is the answer to a URL, and no property of a
body says which URL was asked for. All 29 suites green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-10 10:35:12 +02:00
adminandClaude Opus 5 6eba18968c TARIFF-08: the day-ahead feed is read in the shape it actually publishes
checks / unit (pull_request) Failing after 3s
`generic_dayahead` could not read one byte of `api.energy-charts.info`, and
never had been able to. `_points_from_json()` asks for `doc["data"]`, a list of
`{start, price_eur_mwh}` objects; that document does not exist. Fetched live on
2026-09-10 the feed returns COLUMNAR parallel arrays and no `data` key at all -
`unix_seconds` (96 UTC epoch seconds, 900 s apart), `price` (96 floats),
`unit: 'EUR / MWh'`. Driving the shipped parser against that body:
`SourceUnavailable: the feed carried no 'data' list`. It is the only shipped
provider whose endpoint anybody has reached, so dynamic pricing did not work.

AND §11.4 MADE IT SILENT, which is why it survived to release: a parse error
becomes no_data, becomes the static profile, becomes arbitrage off - the
CORRECT degradation, saying nothing about the cause. A site would have sat on
the fallback indefinitely.

AND THE SUITE COULD NOT SEE IT, because every fixture was built from the
parser's own assumption. test_price.py, test_price_history.py and
test_static_tariff.py each built `json.dumps({"data": [...]})`, a shape invented
here and published by nobody, so the checks graded the parser against the
parser's guess - a closed loop no coverage inside it could ever have escaped.
MUTATION.md habit 19 records it as the fourteenth instance of habit 4 and the
first where the fixture and the code under test shared a SHAPE, not a number.

WHAT CHANGED

* `_points_from_columnar_json()` - a SECOND function, not a sniff. A parser that
  decides which document it received is one that can decide wrong, and a
  wrong-but-parseable response then becomes a price. Feeding the row shape to
  `generic_dayahead` is now a graded no_data.
* Unequal arrays are refused in both directions, with the two lengths in the
  reason. `zip()` stops at the shorter list silently, and a truncated `price`
  array would have produced a real series missing the evening peak - traded on
  as complete, where no data at all switches arbitrage off.
* A unit that is not EUR/MWh is refused. The plugin converts with a fixed
  0.001, and `MAX_PLAUSIBLE_EUR_KWH` only bounds prices that are too LARGE.
* `unix_seconds` are read as UTC instants explicitly; §9.3's Brussels rendering
  stays in `assemble()` rather than being copied into the plugin, where a
  mutation could not tell it from a no-op (M4 proved exactly that on a first
  draft, and the line was removed rather than fenced).
* A published null is dropped and logged with a count; an all-null body is
  no_data named by this parser rather than by the boundary validator.

FIXTURES

Two unmodified response bodies under `scenarios/`, captured by hand on
2026-09-10: quarter-hourly (2026-09-10, 96 points at 900 s) and hourly
(2025-09-10, 24 at 3600 s - the same endpoint before Belgium's 15-minute MTU).
`energy-charts-be-price-captures.json` records the URL, retrieval time, unit,
licence-as-returned and a sha256 of each, and the replay RE-DERIVES the
checksums, so a fixture edited to make a check pass stops being a capture. Data
CC BY 4.0, Bundesnetzagentur | SMARD.de, via Energy-Charts (Fraunhofer ISE),
published without changes - the same BE-zone licence check DATA-02 established.

The surviving invented fixtures are renamed `synthetic_row_feed` at every call
site. `engie_dynamic`, `totalenergies_dynamic` and `eneco_eddy` are untouched
and have no capture: their URLs are unverified and withdrawing them is
TARIFF-07's open decision. Nothing should read their green checks as evidence
that those feeds parse.

MUTATIONS - 12, all red, no survivors. Purged tree, mounted `:ro`, `python3 -B`
(TEST-15). M1 drop the equal-length refusal; M2 make it one-sided; M3 drop the
unit check; M4 `assemble()` stops rendering at the Brussels offset; M5 read
`unix_seconds` as a naive local clock; M6 zero-fill a null instead of dropping
it; M7 drop the all-null refusal; M8 sniff for the row shape; M9 drop the
deprecated warning; M10 drop the dropped-nulls warning; M11 edit one price in
the committed capture; M12 point `generic_dayahead` back at the row parser -
red across all three price suites.

END TO END, LIVE, 2026-09-10: `resolve_tariff()` returns a `TariffPlan`,
`origin='provider'`, 96 quarter-hourly points, spacing [900] s, no decimation,
first point 2026-09-10T00:00:00+02:00 at 0.30608 EUR/kWh all-in. FCST-04's H1
fix was never theoretical. All 29 test files green.

STILL OPEN: `generic_dayahead` asks for no date range, so the default URL
returns one published day - a full 48 h horizon is reached only through §8.3's
carry-forward fill.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-10 08:57:32 +02:00
glenn schrooyenandClaude Opus 5 7b4877d56d TEST-15: purge the bytecode cache before a mutation run, not after
checks / unit (pull_request) Successful in 3m35s
FCST-03's developer reverted `self.provisional` to `here.provisional`, re-ran,
and reported the mutation as a survivor. It was not one. CPython validates a
cached `.pyc` on (source size, source mtime truncated to whole seconds), and
that edit moved neither number - same length, same second - so the interpreter
loaded the bytecode it already had. Disassembly settled it at the time: the
source read `here`, the running code loaded `self`. A mutation run had graded
code that was not there, with no error and exit status 0.

This does not make the results already on the record wrong. It makes them
UNVERIFIED, and that distinction is the whole card - so nothing here re-runs a
campaign and nothing here quietly assumes the old numbers hold.

REPRODUCED, THEN CLOSED. test_bytecode.py builds a real package whose two
return values are the same length, imports it in a real subprocess, rewrites it
inside the same second (retrying until both writes land in one whole second -
milliseconds, no sleep) and reads back which value ran. Part 1 shows the
interpreter running SELF off a file that says HERE.

AND THE TWO OBVIOUS MITIGATIONS DO NOT WORK. Measured, not read off
documentation: `python3 -B` and PYTHONDONTWRITEBYTECODE=1 are write-side only.
Neither stops an EXISTING stale .pyc being read, and a mutation run always has
one, because the previous run wrote it. Part 2 grades both still running the old
code. The advice in test_control.py's coverage audit led with `-B`; it is
corrected here - the purge is the half that works, and `-B` earns its place by
keeping THIS run from leaving a cache for the NEXT edit to inherit.

NO SLEEP ANYWHERE. One-second mtime resolution is the mechanism, not the enemy.
Waiting it out would slow every run down and would still only cover the edit
somebody remembered to slow down.

THE HARNESS. tools/mutation_run.sh purges, prints a sha256/byte fingerprint of
every app/*.py it is about to grade - so a mutation report can say WHICH source
produced it, which is the reviewer's addition to this card - runs each suite
under `python3 -B`, refuses a tree with no test_*.py (CI-01's rule, for CI-01's
reason), and fails if a cache appears during the run. It takes the tree as an
argument so the infra repo's replay harnesses can call it without being moved.

WHERE IT IS SAID (AC 2). MUTATION.md carries the mechanism, the measurement,
the reporting rules and the survey, and every one of the 16 suites now points at
it from its docstring. That list is DERIVED FROM THE TREE by test_bytecode.py
part 5, so a new suite that joins this directory without the pointer is red -
the same failure shape CI-01's file glob exists to prevent.

CI WAS ALREADY IMMUNE, AND IS DESCRIBED THAT WAY (AC 4). Verified rather than
reasoned: with the checkout bind mounted `-v "$WS:/w:ro"` the interpreter writes
no __pycache__ into the tree at all, and it fails SILENTLY - no error, exit
status 0 - while the same probe on a writable mount produces one. Each job also
rm -rf's its workspace and clones afresh, so nothing crosses between jobs on the
shared runner. PYTHONDONTWRITEBYTECODE=1 is added as belt and braces only: the
`:ro` is one character wide and the next person debugging a write will be
tempted by it. Part 6 grades both ON THE DOCKER COMMAND LINE rather than on the
file - the first draft searched the whole workflow and stayed green after the
flag was deleted, because the comment explaining it still named it.

THE SHIPPED ADD-ON IS NOT AFFECTED and the question is closed:
goodwe_controller/Dockerfile has set PYTHONDONTWRITEBYTECODE=1 since it was
written. There is no deploy-side finding.

THE SURVEY (AC 3), in MUTATION.md, scoped BY THE PROPERTY. The defect is
one-directional - a mutation that never took leaves the suite green, so stale
bytecode manufactures SURVIVORS and cannot manufacture a red - which puts every
"all red, no survivors" claim in the window out of scope, along with mutations
executed in-process (SAFETY-07's B1/B2/B3, test_peakguard parts 7 and 10, where
no source file is edited) and everything run after the purge discipline landed
on 2026-08-31 (SAFETY-24, SAFETY-26, OPT-04, FCST-03's re-run). What is left is
the recorded SURVIVORS from source-edit runs before that date. Two were re-run
here under the harness and both still stand: TARIFF-01's record_block() ignoring
its policy argument, and SAFETY-03's abs() on the clamp limit - 15 suites green
each, on the tree as it stood. The rest are labelled still-unverified rather
than assumed or re-run; each is one harness run to close, and the moment to
spend it is when somebody next touches that module.

OUT OF SCOPE AND RECORDED: the infra repo's sim/ replay harnesses import the
add-on and write bytecode into both trees (goodwe/sim/__pycache__ exists today),
so SAFETY-02's headline replay carries the same exposure. mutation_run.sh takes
a tree argument for exactly that, but the change belongs in that repo and some
of those harnesses want the rig.

RED AND GREEN. Every new check was broken and watched fail: the purge commented
out of the harness turns part 4 red (and its non-vacuity probe already proves
the fixture would mis-grade without it); one docstring pointer removed turns
part 5 red naming the file; `:ro` and the `-e` flag removed from the workflow
turn part 6 red. Full suite green, 16 files, in the CI image on a read-only
mount - the shape the runner uses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 02:07:09 +02:00
glenn schrooyenandClaude Opus 5 bdd7f18a54 OPT-04: charge cheap, discharge expensive - and only when it pays
checks / unit (pull_request) Successful in 3m16s
Four cards built the price stack and nothing traded on it. app/arbitrage.py is
§10.1's priority 3, and it consumes all four rather than re-deciding anything:
whether a trade is permitted at all, on which two hours, against which widened
threshold is TariffPlan.arbitrage()'s answer, and that answer cannot be
obtained without first saying whether the peak guard is holding. §9.1's "peak
shaving outranks arbitrage" is therefore honoured structurally - there is no
path to a spread that skips the question - and again in the arbiter, where
P_PEAK (50) clamps a claim filed at P_ARBITRAGE (30).

A guard that never ran is not a guard that is clear. On meter_source off the
peak guard is never evaluated; that third state is carried as None and refuses,
because charging at the rail creates the billed quantity while nothing measures
the quarter. validate.py refuses that configuration too, along with arbitrage
armed with no price source and a SoC floor at or above the ceiling.

§17: never export into a negative price. The guard is on the hour being sold
into, not on the spread - a day priced -0.30 .. -0.05 clears any threshold and
its dearest hour is still negative. Buying in a negative hour stays allowed.

resolve_tariff() may open an HTTPS connection and the tick feeds the ESP32
heartbeat, so the fetch runs in its own task at a 30 min cadence and the tick
reads an attribute and an age bound. test_arbitrage.py section 8 asserts that
structurally on the shipped AST: a fetch put back on cycle() is red.

AC4 is NOT met and is not restated - this repository holds no price history.
Section 7 replays a synthetic year and asserts the property that does not need
real prices: every executed round trip cleared its own widened requirement, and
zero discharges landed in the 522 negative-priced hours. At the shipped 0.10
EUR/kWh threshold a realistic Belgian day/night site does not trade at all.
Reported, not tuned: that number and FCST-03's widening factors are one product
decision and it is the owner's.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-31 01:36:09 +02:00
glenn schrooyenandClaude Opus 5 303525cfc5 TARIFF-03: static day/night and fully-fixed tariffs, first-class
checks / unit (pull_request) Successful in 2m43s
§9.2 puts a large share of the Belgian installed base on a static profile, and
an EMS that only works under dynamic pricing is an EMS for a minority. A new
`static_profile` option describes a two-period day/night tariff - or a single
fixed price - and app/static_tariff.py turns it into the same canonical
{start, eur_per_kwh, provisional} series the fetched providers emit.

Not a sixth provider plugin, for two mechanical reasons. capabilities() is a
classmethod, so a plugin's PriceProduct is fixed when the class is written -
but §9.2 makes STATIC and FIXED two different products and which one a site is
on is a property of the operator's data; StaticProfile.product derives it from
the number of distinct prices. And the plugin framework exists to stop a
FETCHED figure being marked up twice, which a number typed as all-in cannot be.
So the profile sits where §11.4 already says it does - under the providers, as
the fallback. Configured product and feed-is-down are one generator differing
in one flag, so the degradation path cannot rot unused.

"No arbitrage" is a type. TariffPlan.arbitrage() returns a variant whose
ALLOWED arm carries the spread and the cheapest/dearest hours and whose REFUSED
arm carries a reason and nothing else - .spread_eur_kwh on a refusal raises
rather than answering 0.0, so an optimiser cannot compute a trade from one.
Four refusals, all derived: peak guard holding, no price data, plan
provisional, or the series is FLAT. The last is AC2: a fixed tariff disables
arbitrage by arithmetic, not by matching on an enum. peak_guard_active is a
required keyword argument, so §10.1's priority 2 over 3 cannot be skipped.

Time-window shifting reaches the one shiftable load that exists: main.py's
cheap-window flag falls back to the static profile, and the UI gained a Tariff
row. Cheap is STRICTLY below the seven-day mean - `<=` would call every hour of
a flat tariff cheap, and 48 h taken on a Friday is two-thirds weekend.

A profile must tile every day type exactly once, checked at startup; a gap is
refused with the range named rather than filled in. `22:00-07:00 mon-fri` does
not cover Saturday morning, and that trap is what the check catches.

validate.py rule 7's stated reason expired exactly as TARIFF-02 predicted: the
test_price.py section 12 tripwire went red naming static_tariff.py. The message
now says what is true instead, and the tripwire is re-aimed rather than removed
- consumer set pinned, expired phrase asserted gone. Rule 7 also grew a second
refusal for static_profile alongside the HA price pair, the same silent-loser
conflict and a live one.

CFG-03 is not widened: static_profile went into price.py's PRICE_OPTION_DEFAULTS
- the one in-code mirror pinned to config.yaml by a check - and the new module
carries no defaults dict of its own.

Measured on the two committed days of this house's real P1 meter. 2026-08-20
(Thursday): 15.220 kWh imported, 9.606 day / 5.614 night, EUR 4.7185 under
day/night vs EUR 4.6725 flat, shifting saves EUR 0.8069 = 9.606 kWh x the
0.0840 spread, derived twice. 2026-08-23 (Sunday): 0.728 kWh, all of it
night-priced, which is the weekend row proving itself. §17 names six Belgian
scenarios; this is the seventh, and it is two days rather than a season - the
file says so rather than implying an annual figure.

All 12 suites green. Eleven mutations of static_tariff.py were each caught by
at least one check.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 23:07:53 +02:00
adminandClaude Opus 5 e102cde841 TARIFF-02 review: close the three other doors to the markup, and stop overclaiming
checks / unit (pull_request) Successful in 2m34s
Review of PR #16 wrote a double-counting plugin four ways. PluginOptions
blocked one. The one that decided it touched nothing but
PRICE_OPTION_DEFAULTS - a public dict in the module every plugin must
already import for register() - so "the number is not in scope" was
simply false, and an absolute guarantee a reviewer trusts is worse than
a stated limit, because they stop checking.

register() now refuses, AT IMPORT, any plugin whose own code mentions a
framework-owned name: FRAMEWORK_KEYS plus PRICE_OPTION_DEFAULTS,
price_option and _o. code_leaks() parses the class and drops every
docstring node before searching, because every plugin is REQUIRED to
name its sourced components in prose - engie_dynamic says "markup" in
its docstring and a raw text scan false-positives on the documentation
the same file mandates. A check that fires on required documentation is
a check somebody deletes. test_price.py asserts that a raw scan really
would have false-positived, so the strip is proven load-bearing rather
than asserted.

It fails OPEN if inspect.getsource() cannot find the .py, which the
add-on image always can. Refusing to start a battery controller because
a static analysis could not run would trade a real outage for a
hypothetical accounting error - so instead the checks assert source was
found for all five shipped plugins, and a build that lost it is a red
check rather than a quietly disarmed guard.

All three reachable attacks are written out as plugins in the suite and
each must be refused at registration. The cost is measured, not
asserted: honest wholesale 0.188, every attack 0.219 - one markup high,
every hour, invisible in every log this add-on writes.

The guarantee is now stated at its true size everywhere it appears: the
framework adds the complement exactly once, GIVEN A CORRECT sources
DECLARATION. A plugin that declares {ENERGY} while its feed is all-in
still doubles while touching no framework name, and no code can catch
that - the declaration is an unverifiable claim about somebody else's
tariff. Measured in the suite (0.276) so the limit is a number rather
than a caveat. The docstring/declaration check verifies that a plugin's
two claims AGREE, not that either is true; that now says so.

The three surviving mutations:

M3 - _NoData's reason taking a default. This is the identical survivor
left open on RIG-01's _Unread.cause: the shape was borrowed from
sim/riglock.py including its blind spot, which is what borrowing a shape
does if nobody re-runs the mutations against the copy. Closed by
SIGNATURE - no constructor or handler of an outcome type may carry a
default - so the next one is red without anyone writing a new case.

M5 - the ENERGY-required guard was untested. Its consequence is that
adds becomes {ENERGY, TAXES} and the emitted price silently omits the
energy it is pricing.

M7 - deleting the TLS refusal went green AND the check named for it
still passed, because the old parser stripped a fixed len("https://")
and mangled http://api.energy-charts.info/p into the host
pi.energy-charts.info, which then failed the ALLOW-LIST. An assertion
that does not test what it names, inside a security check. Fixed on both
sides: the scheme is parsed as a scheme, so an http URL now resolves to
its real and declared host and only the TLS rule can refuse it; and the
check asserts the REASON. Fixing it surfaced M7b - the no-scheme guard
was survivable the same way - found by mutating my own fix, and closed
the same way.

Also carried forward: rule 7's refusal is correct forever but its stated
reason ("the provider decides nothing") expires the moment TARIFF-03 or
OPT-04 lands. test_price.py section 12 now greps app/ by AST for a
load_prices consumer and goes red when one appears, telling whoever
lands it to update the message in the same commit. The first draft of
that tripwire used a substring and matched the comment in validate.py
explaining the tripwire - the same false-positive class as the plugin
docstrings, hit twice in one ticket, and fixed the same way.

Documented rather than coded around: vars(read).get("points", []) and
bool(read) both still get past the absence type, and neither is worth
code - defining __bool__ would turn a line that reads like a guard into
one that IS a guard by accident, which is how .get(k, default) got here
in the first place.

Stated plainly in DOCS/CHANGELOG/PR: every fixture is synthetic, so the
five feed URLs and JSON shapes are verified against a belief about what
those suppliers publish rather than against the feeds. A wrong shape
degrades to no-price-data, not to a wrong price.

Review's own DST verification recorded: 149,040 hourly instants,
2020-2036, zero mismatches against zoneinfo. The residual is political,
not arithmetic, and the docstring now says so.

24 mutations driven through the suite, 0 survivors. Whole add-on suite
green. No deleted lines outside this ticket's own files; main.py still
untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 19:35:36 +02:00
adminandClaude Opus 5 852fc7ced6 TARIFF-02: one canonical price format, and a seam that keeps supplier names out of the optimiser
checks / unit (pull_request) Successful in 2m33s
Five provider plugins - generic_dayahead, engie_dynamic,
totalenergies_dynamic, eneco_eddy, manual_csv - all emitting exactly
{"prices": [{"start", "eur_per_kwh", "provisional"}]}, tz-aware at the
Brussels offset, EUR/kWh, all-in. Nothing consumes it yet; the seam
lands first, per the PM's ordering.

It is built in goodwe_controller/app/ rather than at specs.md 2.2's
ems/ingest/prices/, per the product owner's decision of 2026-08-30. The
behaviour 9.3 / 8.3 / 11.4 specifies is unchanged.

Three things shape the design, and each is the mechanism rather than a
comment saying to be careful:

Double-counting the markup is unreachable. A plugin does not return a
price - it returns the figure its source published and declares which of
{energy, markup, taxes} that figure already contains. assemble() adds
the complement, once, in the framework. A plugin is never handed
price_markup_eur_kwh: PluginOptions raises FrameworkOption on it, so the
line that adds the margin a second time cannot be written. Engie and
TotalEnergies publish a retail figure with their margin inside; adding
the configured 3 c/kWh on top would have made every arbitrage decision
wrong by that amount with nothing reporting it.

The product type is declared, not inferred from the plugin's name.
eneco_eddy is real-time/imbalance and gets 8.3's widened thresholds
through capabilities().product and volatility_regime(), never through a
name comparison - which is AC2. test_price.py asserts mechanically that
no plugin id appears anywhere in the framework's code, and drives a
sixth provider defined inside the test file to the right regime with
price.py untouched.

"No price data" is a type, not an empty list - the same rework
sim/riglock.py took under TEST-04. A fetch returns PriceRead.series(...)
or PriceRead.no_data(reason), and only the first carries fields. An
empty series is not constructible; no_data has no .points, no .at(), and
.get("points", []) raises rather than returning the default. A day of
free electricity and a day with no prices are different instructions -
arbitrage is pointless under one and forbidden under the other - and an
implementation returning [] cannot tell them apart. Every site-level
failure returns no_data; nothing raises into the control tick.

Also: injection stays a separate scalar (export_price_eur_kwh, AC4),
negative prices survive the boundary validator (17's regression case),
egress is allow-listed against each plugin's own declaration (16), and
no tzdata is used - the shipped Alpine image has none, so Brussels
offsets come from the EU rule and instants are carried as UTC.

validate.py grows rule 7: an unknown provider id, manual_csv with no
path, and - the HA overlap the PM told us to decide about -
price_provider set alongside price_now_entity/price_avg_entity, which is
two price sources where the HA pair silently wins. The HA sensors are
kept and still work alone; TARIFF-03 and OPT-04 are what replace them.

CFG-03 is not widened: the price defaults have ONE in-code mirror,
PRICE_OPTION_DEFAULTS, which validate.py imports rather than restating
and which test_price.py pins to config.yaml by reading the file.

test_price.py is runnable, assert-free in the house style, and every
verdict it prints is derived. It grades the markup arithmetic with
0.031/0.057 rather than the shipped 0.03/0.06, because 0.06 is exactly
twice 0.03 and a degenerate pair would make double-counting
indistinguishable from correct addition - and it asserts that
non-degeneracy as its first section. The DST checks run on the 23-hour
and 25-hour days, not on a normal day where a fixed 24-slot
implementation and a correct one agree. Thirteen mutations were driven
through it; all thirteen go red.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 19:05:12 +02:00