5 Commits

Include renames

Author	SHA1	Message	Date
glenn schrooyenandClaude Opus 5	20f9f4969d	TEST-16: pin the two one-sided arbitrage rungs, and un-mask them from each other ... SAFETY-26's AC4 gave the same tolerance mutation to every price and quantity boundary in arbitrage.py and static_tariff.py. Everything in arbitrage.py went red - `price < 0`, `soc <= floor_pct`, `soc >= ceiling_pct` are each red under +/-5. Two survived, both in static_tariff.py, and this card closes them. D `spread == 0.0` (:912) weakened to `spread <= 0.001` SURVIVED at +2 B. F `len(firm) < 2` (:901) weakened to `len(firm) < 1` SURVIVED at +0 B - the delta MUTATION.md names as the risky one. THE DEFECT IS NOT EITHER ROW, IT IS THE INTERACTION. D and F MASK EACH OTHER. A single firm hour also has a spread of 0.0, so with F weakened the flat rung refuses the stimulus anyway; with D weakened the small-but-real spread is still refused, only for a lie. Each rung's failure is absorbed by the other, so BOTH can be weakened together and a suite grading `kind == "refused"` reports nothing. That is structurally invisible to single-mutation testing, which is how this pair came through four rounds of review on TARIFF-03 and FCST-03 - and it is the arrangement in which the next edit to either rung opens a real hole with nothing to report it. D is not cosmetic either. With the tolerance in place a series priced 0.20000 / 0.20050 - a REAL 0.0005 EUR/kWh spread - flips ALLOWED to REFUSED at min_spread_eur_kwh=0.0, a value arbitrage()'s own docstring documents ("pass 0.0 to mean any positive spread, explicitly") and its validator at :861 accepts on purpose. At a shipped threshold the outcome does not move, but the operator reads the false sentence "the tariff is flat - every one of the 2 firm hours costs 0.2000 EUR/kWh" about a tariff that is not flat. WHAT WAS ADDED. test_static_tariff.py section 9, and the shape of it is the argument: * THE REASON IS GRADED, NOT THE VERDICT. The masked case refuses CORRECTLY while reporting the wrong sentence, so a check asserting only "refused" passes on a broken tree. A `rung9()` helper names which refusal answered, from the sentence the operator would read; an unrecognised refusal reports itself as UNATTRIBUTED rather than quietly matching nothing. * EACH RUNG IS GIVEN A STIMULUS THE OTHER CANNOT ABSORB, or the pinning is circular. The flat rung gets a THREE-firm-hour flat series, which no `len(firm) < N` a person would write can reach. The firm-pair rung gets a ONE-FIRM-HOUR series, which the flat rung CAN reach - so its attribution is the whole check, and that is why it is written as a rung name rather than a verdict. * AC1 is a sweep of small-but-real spreads (0.0005 down to 0.000001) all ALLOWED at min_spread_eur_kwh=0.0, with the exactly flat series on the other side still refused BY THE FLAT RUNG - without that mirror the sweep would also pass for a rung deleted outright. 1e-6 is the floor because the shipped line rounds the spread to 6 decimals; that quantisation is graded in the open as its own row rather than left unstated. * AC3 is a five-row table graded on rung attribution, which denies the masking from both sides at once. * The firm-pair count is also pinned from ABOVE (`< 3`) and BELOW (`< 0`), which the criteria do not ask for but which is what stops this section being one-sided in exactly the way it was written to fix. EVIDENCE. Eight rounds through TEST-15's tools/mutation_run.sh, used UNCHANGED: recursive __pycache__ purge before each run, python3 -B, the tree re-copied FROM DISK between rounds, the mutated line grepped out of the file before the run, byte delta printed, test_static_tariff.py and test_arbitrage.py graded each time. Baseline green. baseline, shipped +0 B PASS D1 `spread == 0.0` -> `<= 0.001` +2 B RED, 3 checks D2 `spread == 0.0` -> `<= 0.000001` +5 B RED, AC1 alone D3 `spread == 0.0` -> `< 0.0` (rung gone) +0 B RED (section 2 as well) F1 `len(firm) < 2` -> `< 1` +0 B RED, 3 checks F2 `len(firm) < 2` -> `< 3` +0 B RED, 6 checks F3 `len(firm) < 2` -> `< 0` (rung gone) +0 B RED (ValueError) D1+F1 BOTH AT ONCE - THE TICKET +2 B RED, 5 checks Four of the seven are +0 B, the delta MUTATION.md names as the one stale bytecode can hide. All are REDS, the direction that defect cannot fabricate - but they were run under the purge anyway, because "it went red so the cache cannot have lied" is a conclusion and not a procedure. NO SHIPPED BEHAVIOUR CHANGED. app/ is untouched; both rungs were correct all along, and the card is explicit that a rung found actually wrong is a stop-and- report rather than an edit. Also updated: MUTATION.md gains a short section on masking as a SECOND way a survivor lies, with the two rules that fall out of it (grade the reason, and mutate suspected pairs together); and test_arbitrage.py's SAFETY-26 AC4 note is marked closed rather than left reading as an open finding. UNSURE ABOUT. The 5e-7 row asserts the shipped `round(spread, 6)` quantisation. That is a real property of the line and worth stating, but it does pin a rounding the ticket never named - if a reviewer would rather that stayed unpinned, it is one check to drop and nothing else depends on it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-01 07:37:19 +02:00
glenn schrooyenandClaude Opus 5	7b4877d56d	TEST-15: purge the bytecode cache before a mutation run, not after ... FCST-03's developer reverted `self.provisional` to `here.provisional`, re-ran, and reported the mutation as a survivor. It was not one. CPython validates a cached `.pyc` on (source size, source mtime truncated to whole seconds), and that edit moved neither number - same length, same second - so the interpreter loaded the bytecode it already had. Disassembly settled it at the time: the source read `here`, the running code loaded `self`. A mutation run had graded code that was not there, with no error and exit status 0. This does not make the results already on the record wrong. It makes them UNVERIFIED, and that distinction is the whole card - so nothing here re-runs a campaign and nothing here quietly assumes the old numbers hold. REPRODUCED, THEN CLOSED. test_bytecode.py builds a real package whose two return values are the same length, imports it in a real subprocess, rewrites it inside the same second (retrying until both writes land in one whole second - milliseconds, no sleep) and reads back which value ran. Part 1 shows the interpreter running SELF off a file that says HERE. AND THE TWO OBVIOUS MITIGATIONS DO NOT WORK. Measured, not read off documentation: `python3 -B` and PYTHONDONTWRITEBYTECODE=1 are write-side only. Neither stops an EXISTING stale .pyc being read, and a mutation run always has one, because the previous run wrote it. Part 2 grades both still running the old code. The advice in test_control.py's coverage audit led with `-B`; it is corrected here - the purge is the half that works, and `-B` earns its place by keeping THIS run from leaving a cache for the NEXT edit to inherit. NO SLEEP ANYWHERE. One-second mtime resolution is the mechanism, not the enemy. Waiting it out would slow every run down and would still only cover the edit somebody remembered to slow down. THE HARNESS. tools/mutation_run.sh purges, prints a sha256/byte fingerprint of every app/*.py it is about to grade - so a mutation report can say WHICH source produced it, which is the reviewer's addition to this card - runs each suite under `python3 -B`, refuses a tree with no test_*.py (CI-01's rule, for CI-01's reason), and fails if a cache appears during the run. It takes the tree as an argument so the infra repo's replay harnesses can call it without being moved. WHERE IT IS SAID (AC 2). MUTATION.md carries the mechanism, the measurement, the reporting rules and the survey, and every one of the 16 suites now points at it from its docstring. That list is DERIVED FROM THE TREE by test_bytecode.py part 5, so a new suite that joins this directory without the pointer is red - the same failure shape CI-01's file glob exists to prevent. CI WAS ALREADY IMMUNE, AND IS DESCRIBED THAT WAY (AC 4). Verified rather than reasoned: with the checkout bind mounted `-v "$WS:/w:ro"` the interpreter writes no __pycache__ into the tree at all, and it fails SILENTLY - no error, exit status 0 - while the same probe on a writable mount produces one. Each job also rm -rf's its workspace and clones afresh, so nothing crosses between jobs on the shared runner. PYTHONDONTWRITEBYTECODE=1 is added as belt and braces only: the `:ro` is one character wide and the next person debugging a write will be tempted by it. Part 6 grades both ON THE DOCKER COMMAND LINE rather than on the file - the first draft searched the whole workflow and stayed green after the flag was deleted, because the comment explaining it still named it. THE SHIPPED ADD-ON IS NOT AFFECTED and the question is closed: goodwe_controller/Dockerfile has set PYTHONDONTWRITEBYTECODE=1 since it was written. There is no deploy-side finding. THE SURVEY (AC 3), in MUTATION.md, scoped BY THE PROPERTY. The defect is one-directional - a mutation that never took leaves the suite green, so stale bytecode manufactures SURVIVORS and cannot manufacture a red - which puts every "all red, no survivors" claim in the window out of scope, along with mutations executed in-process (SAFETY-07's B1/B2/B3, test_peakguard parts 7 and 10, where no source file is edited) and everything run after the purge discipline landed on 2026-08-31 (SAFETY-24, SAFETY-26, OPT-04, FCST-03's re-run). What is left is the recorded SURVIVORS from source-edit runs before that date. Two were re-run here under the harness and both still stand: TARIFF-01's record_block() ignoring its policy argument, and SAFETY-03's abs() on the clamp limit - 15 suites green each, on the tree as it stood. The rest are labelled still-unverified rather than assumed or re-run; each is one harness run to close, and the moment to spend it is when somebody next touches that module. OUT OF SCOPE AND RECORDED: the infra repo's sim/ replay harnesses import the add-on and write bytecode into both trees (goodwe/sim/__pycache__ exists today), so SAFETY-02's headline replay carries the same exposure. mutation_run.sh takes a tree argument for exactly that, but the change belongs in that repo and some of those harnesses want the rig. RED AND GREEN. Every new check was broken and watched fail: the purge commented out of the harness turns part 4 red (and its non-vacuity probe already proves the fixture would mis-grade without it); one docstring pointer removed turns part 5 red naming the file; `:ro` and the `-e` flag removed from the workflow turn part 6 red. Full suite green, 16 files, in the CI image on a read-only mount - the shape the runner uses. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-01 02:07:09 +02:00
glenn schrooyenandClaude Opus 5	d57122ec74	DATA-02: a real Belgian price year, so AC4 stops being unmeetable ... OPT-04 reported AC4 - "net-positive across 12 months of replayed prices" - as NOT MET and replayed a synthetic year, because scenarios/ held only P1 meter captures. Right call, wrong diagnosis: the criterion was being carried beside a 30-day METER criterion as though both needed waiting. EPEX BE day-ahead is public data, not a capture. It needed a fetch. 8760 hourly points, 2025-09-01 to 2026-09-01 Brussels-local, canonical three fields. Licence checked BEFORE committing: Energy-Charts publishes the BE zone under CC BY 4.0 from Bundesnetzagentur | SMARD.de and other zones under private-use-only terms, so this one is redistributable with attribution. The fetcher refuses to write a file whose per-response license_info stops saying so. Cross-checked against SMARD's own 4996/DE: 96 of 96 quarter-hours, max difference 0.00 EUR/MWh. Raw wholesale, and the file says so. all_in: false, with the markup/tax assumption carried as numbers to APPLY rather than numbers applied - TARIFF-02's trap in data form. Section 7B applies it in one visible place and asserts arithmetically that its additive terms cancel out of every daily spread, so the trade count turns on the VAT multiplier alone; the raw replay is run too, so how much rests on the guess is reported (110 vs 88). DST by construction: iterate in UTC, render through price.brussels(). 2025-10-26 is 25 points, 2026-03-29 is 23, verified by counting per Brussels day rather than asserting 24, and by checking October's 02:00 appears twice at two offsets with two prices while March's never appears. Both bugs the point-and-gap accounting caught were real: a UTC-bounded window made the first day 22 hours, and an inclusive-end off-by-one dropped the last day silently. A day nobody asked for is not a day with no prices. THE RESULT, AT THE SHIPPED DEFAULT, WHATEVER IT IS. At 0.10 widened 1.5x to 0.1500: 110 round trips, +24.88 EUR net of the cycle cost. Unwidened, 173. Nothing tuned - the threshold and the 2.5/1.5 factors are untouched. And what the owner's decision now has under it: the four reports that a static ~0.0840 spread does not clear 0.10 all STAND. What was missing is that a dynamic site clears the widened requirement on 121 of 365 days, median all-in spread 0.1130, so a median day would clear at 0.0754 - 156 trips, +35.42 EUR. Reported, not applied. The synthetic year stays, because the real one cannot grade section 17: not one of 365 days has a negative DEAREST hour, so a zero there counts something that could not happen. 7B asserts that degeneracy rather than reporting the zero. Size measured, not assumed: 893 KB plain but 74.6 KB as a git object against 56.5 KB gzipped. Git already compresses; 18 KB is not worth making the provenance block - the licence claim a reviewer must check - unreadable in a diff. Nothing wired into the add-on; generic_dayahead is still the live path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC	2026-08-31 11:01:42 +02:00
glenn schrooyenandClaude Opus 5	2b8b8cb97f	SAFETY-26: pin the negative-price refusal from below, not only from above ... OPT-04's reviewer weakened `if price < 0` to `if price < -0.001` and every suite stayed green while a -0.0005 EUR/kWh sell hour discharged. The shipped code was right the whole time; the CHECK could not tell correct from nearly-correct. The only probe near the boundary sat at 0.0000, which catches a `<=` and says nothing about a tolerance invented on the other side - and a tolerance on the other side is §17's named regression walking through with the suite passing. A dearest hour at -0.0005 is an ordinary Belgian day-ahead print, not a fuzz value. `app/` IS NOT TOUCHED. `if price < 0` is already correct and changing it would be the defect; this card closes the check, not the behaviour. The 0.0000 probe stays and is now labelled as the half that pins the boundary from above. The new probe is a SWEEP rather than a point, because one probe at -0.0005 closes `-0.001` and nothing else - a mutation reaching for `-0.01` would be the identical defect one decimal further out. Five prices span four orders of magnitude down to -0.000001, the last place the canonical six-decimal format can express; each is asserted to survive that rounding as strictly negative, to still be the DEAREST hour, and to still clear the widened threshold, so §17 is the only rung that can refuse. The same five magnitudes with the sign flipped all discharge, so this is a boundary at exactly zero rather than a refusal of small numbers. AC4, the look: every price/quantity boundary in arbitrage.py and static_tariff.py was given the same tolerance mutation, both directions where both exist. Two survive, both in static_tariff.py and both NAMED RATHER THAN FIXED - they are TARIFF-03's rungs and want probes in its own file. `spread == 0.0` loosened to `<= 0.001` flips a real 0.0005 spread from allowed to refused at min_spread_eur_kwh=0.0, a documented supported value, and otherwise prints a false reason about a tariff that is not flat. `len(firm) < 2` loosened to `< 1` survives at +0 bytes - TEST-15's risky delta - and is masked only by the flat rung, so the two mask each other and a later edit to either opens a hole nothing reports. Mutation verified under `python -B` with a recursive __pycache__ purge, byte delta printed (+5 B), the mutated line asserted changed, and - per the reviewer's addition to TEST-15 - the tree re-read from disk and asserted to be the tree that was mutated before each run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC	2026-08-31 02:07:56 +02:00
glenn schrooyenandClaude Opus 5	bdd7f18a54	OPT-04: charge cheap, discharge expensive - and only when it pays ... Four cards built the price stack and nothing traded on it. app/arbitrage.py is §10.1's priority 3, and it consumes all four rather than re-deciding anything: whether a trade is permitted at all, on which two hours, against which widened threshold is TariffPlan.arbitrage()'s answer, and that answer cannot be obtained without first saying whether the peak guard is holding. §9.1's "peak shaving outranks arbitrage" is therefore honoured structurally - there is no path to a spread that skips the question - and again in the arbiter, where P_PEAK (50) clamps a claim filed at P_ARBITRAGE (30). A guard that never ran is not a guard that is clear. On meter_source off the peak guard is never evaluated; that third state is carried as None and refuses, because charging at the rail creates the billed quantity while nothing measures the quarter. validate.py refuses that configuration too, along with arbitrage armed with no price source and a SoC floor at or above the ceiling. §17: never export into a negative price. The guard is on the hour being sold into, not on the spread - a day priced -0.30 .. -0.05 clears any threshold and its dearest hour is still negative. Buying in a negative hour stays allowed. resolve_tariff() may open an HTTPS connection and the tick feeds the ESP32 heartbeat, so the fetch runs in its own task at a 30 min cadence and the tick reads an attribute and an age bound. test_arbitrage.py section 8 asserts that structurally on the shipped AST: a fetch put back on cycle() is red. AC4 is NOT met and is not restated - this repository holds no price history. Section 7 replays a synthetic year and asserts the property that does not need real prices: every executed round trip cleared its own widened requirement, and zero discharges landed in the 522 negative-priced hours. At the shipped 0.10 EUR/kWh threshold a realistic Belgian day/night site does not trade at all. Reported, not tuned: that number and FCST-03's widening factors are one product decision and it is the owner's. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC	2026-08-31 01:36:09 +02:00