diff --git a/goodwe_controller/CHANGELOG.md b/goodwe_controller/CHANGELOG.md index 71f1b27..70038c1 100644 --- a/goodwe_controller/CHANGELOG.md +++ b/goodwe_controller/CHANGELOG.md @@ -2,6 +2,33 @@ ## Unreleased +**The two `arbitrage()` rungs SAFETY-26's AC4 sweep found pinned from ONE SIDE +ONLY are now pinned from both - and, the actual finding, are pinned so they stop +MASKING EACH OTHER (TEST-16).** `spread == 0.0` weakened to `<= 0.001` survived +the whole suite at +2 B, and `len(firm) < 2` weakened to `< 1` survived at ++0 B - the delta MUTATION.md names as the risky one. Neither was individually +live, and that was the problem: a single firm hour *also* has a spread of zero, +so each rung's failure was absorbed by the other and both could be weakened +together with nothing reporting it. That is invisible to single-mutation +testing, which is how the pair survived four rounds of review on TARIFF-03 and +FCST-03. `test_static_tariff.py` gains section 9, which grades WHICH RUNG +answered each stimulus rather than only whether it refused - the masked case +refuses correctly while reporting the false sentence "the tariff is flat" about +a series that is not flat, so a check asserting only "refused" passes on a +broken tree. The flat rung is swept with small-but-REAL spreads down to +0.000001 EUR/kWh at `min_spread_eur_kwh=0.0` (a value `arbitrage()`'s docstring +documents and its validator accepts on purpose), the firm-pair rung is probed +with a ONE-FIRM-HOUR series and graded on §8.3's sentence coming back rather +than the flat rung's, and a five-row table denies the masking on both sides. +Verified through TEST-15's `tools/mutation_run.sh` unchanged - recursive +`__pycache__` purge, `python3 -B`, tree re-copied from disk between rounds, +mutated line grepped off disk, byte delta printed: 7 mutations plus a green +baseline, 4 of them at +0 B, ALL RED, including both rungs weakened AT ONCE +(5 checks red). **No shipped behaviour changed - both rungs were correct all +along**; `app/` is untouched. `MUTATION.md` gains a section on masking as a +second way a survivor lies, and `test_arbitrage.py`'s SAFETY-26 AC4 note is +marked closed rather than left reading as an open finding. + **`check_output_clamp()` could be moved above `write_setpoint()` in `cycle()` with test_clamp.py, test_freeze.py and test_loop.py all staying GREEN — the guard's write-path evidence, `self.clamp_last`, had nothing watching it.** diff --git a/goodwe_controller/MUTATION.md b/goodwe_controller/MUTATION.md index 3a31d97..01209ab 100644 --- a/goodwe_controller/MUTATION.md +++ b/goodwe_controller/MUTATION.md @@ -97,6 +97,36 @@ State the things a reviewer cannot reconstruct afterwards: 4. for a **survivor**, that the run was purged. An unpurged survivor is not a result. +## The other thing a mutation sweep cannot see: masking (TEST-16) + +Stale bytecode is not the only way a survivor lies. **Two guards can mask each +other**, and a sweep that mutates one line at a time is structurally blind to it. + +TEST-16 is the worked example. `static_tariff.arbitrage()` refuses a +one-firm-hour series at `len(firm) < 2`, and refuses a flat series at +`spread == 0.0`. A single firm hour *also* has a spread of zero. So: + +* weaken the firm-pair rung alone → the flat rung refuses the stimulus anyway; +* weaken the flat rung alone → the tiny-spread stimulus is still refused, just + for the wrong reason; +* and the suite, grading `kind == "refused"`, reports **nothing either way**. + +Both rungs could then be weakened *together* and still nothing would report — +which is precisely the state in which the next edit to either one opens a real +hole. It survived four rounds of review on TARIFF-03 and FCST-03. + +Two rules come out of it, and both are cheap: + +1. **Grade the reason, not just the verdict.** "Refused" and "refused for the + right reason" are different claims, and a masked rung is the case where the + first is true and the second is false. A refusal sentence is what the + operator reads; it is also the only observable that moves when one rung + quietly absorbs another's work. +2. **When two rungs can answer the same stimulus, mutate them together.** One + extra run per suspected pair. If a sweep finds two survivors in the same + function, that pair is the first thing to try — TEST-16's two survivors were + adjacent lines in one function and neither was individually live. + ## Where each surface stands * **Local and dev-harness runs — the exposed one.** This is where FCST-03 hit @@ -136,7 +166,7 @@ commit and PR bodies. | **FCST-03** M9 `self.provisional`→`here.provisional` (dd40c56, 71cd784) | **re-verified** | this is the original false survivor. Re-run with the cache cleared, M9 is caught. The shipped source was correct throughout. | | **FCST-03** X1, X6 degenerate stimuli (71cd784) | **not at risk** | run after the purge discipline landed in that branch. | | **SAFETY-24** 11 author + 9 reviewer mutations (e709554, 4601682) | **not at risk** | the commit records them run under `python -B` with a purged `__pycache__` and real byte deltas. Its one survivor is an equivalent mutant, argued structurally rather than by a green run. | -| **SAFETY-26** tolerance sweep over `arbitrage.py` and `static_tariff.py`, 2 survivors (2b8b8cb) | **not at risk** | purge, printed `+5 B` delta, mutated line asserted present, tree re-read before each run. One survivor is explicitly recorded at `+0 B` — the risky delta — and was produced *under* the purge. | +| **SAFETY-26** tolerance sweep over `arbitrage.py` and `static_tariff.py`, 2 survivors (2b8b8cb) | **not at risk** | purge, printed `+5 B` delta, mutated line asserted present, tree re-read before each run. One survivor is explicitly recorded at `+0 B` — the risky delta — and was produced *under* the purge. **Both survivors are now closed by TEST-16** (`test_static_tariff.py` section 9), which also found that the two of them *masked each other* — each was individually non-live only because the other rung caught its stimulus, an arrangement single-mutation testing cannot see. Its own ledger is 7 mutations, 4 of them at `+0 B`, all red. | | **OPT-04** mutations reported on PR #22 (bdd7f18) | **not at risk** | `test_arbitrage.py`'s header records the exact command: purge, then `python3 -B`. | | **TEST-17** `checks()` dropping `when` (504e8da) | **not at risk** | a *red* claim, and dropping an argument changes the file length. Neither half of the collision is available. | | **SAFETY-07** wiring mutations B1/B2/B3 and `test_peakguard.py` parts 7 and 10 (2d570d8, e7c95bf) | **not at risk** | those mutations are **executed in-process**, monkeypatched onto the imported module. No source file is edited, so no `.pyc` is consulted. | diff --git a/goodwe_controller/test_arbitrage.py b/goodwe_controller/test_arbitrage.py index f4882c8..5509acf 100644 --- a/goodwe_controller/test_arbitrage.py +++ b/goodwe_controller/test_arbitrage.py @@ -513,6 +513,16 @@ check(f"...while THE SAME MAGNITUDES with the sign flipped all DISCHARGE, so " # Not fixed here on purpose. D and F are TARIFF-03's rungs, they need probes in # test_static_tariff.py rather than in this file, and the card is explicit that # its fourth criterion is a look and not a refactor. +# +# ✅ CLOSED BY TEST-16, in test_static_tariff.py SECTION 9, and the two rows +# above are kept as the diagnosis rather than as an open finding. D is pinned by +# a sweep of small-but-real spreads down to 0.000001 EUR/kWh at +# min_spread_eur_kwh=0.0; F is pinned by a ONE-FIRM-HOUR series graded on WHICH +# RUNG'S SENTENCE came back, which is the only way it goes red without borrowing +# the flat rung; and the masking is denied by a table that grades every stimulus +# on its rung's attribution rather than on `kind`. Both are red on their own and +# five checks are red with BOTH weakened at once. Ledger in that section's +# header. Nothing in app/ changed - these rungs were correct all along. # --------------------------------------------------------------------------- # diff --git a/goodwe_controller/test_static_tariff.py b/goodwe_controller/test_static_tariff.py index f36a185..84f8fa9 100644 --- a/goodwe_controller/test_static_tariff.py +++ b/goodwe_controller/test_static_tariff.py @@ -10,6 +10,11 @@ it: AC2 Fully-fixed pricing DISABLES ARBITRAGE and leaves self-consumption only. AC3 The Belgian scenario suite carries a static-profile regression case. + Section 9 is TEST-16 and is not one of those criteria. It pins the two + `arbitrage()` rungs SAFETY-26's AC4 sweep found pinned from ONE SIDE ONLY - + `spread == 0.0` and `len(firm) < 2` - and, more to the point, pins them so + they stop MASKING EACH OTHER. Read its header before editing either rung. + THREE RULES THIS FILE HOLDS ITSELF TO, because the reviewer will: * EVERY VERDICT IS DERIVED. No `check("the split is right", True)`. Where a @@ -62,7 +67,7 @@ from pathlib import Path from app.price import ( Absence, PricePoint, PriceProduct, PriceRead, arbitrage_allowed, brussels, - plan_is_provisional, + plan_is_provisional, required_spread, uncertainty_of, ) from app.static_tariff import ( MEAN_WINDOW_DAYS, ProfileInvalid, StaticProfile, TariffPlan, @@ -860,6 +865,255 @@ check("...and main.py grew no `o.get(\"static_profile\", )` fallback 'get("static_profile"' not in _MAIN and "static_status(" in _MAIN) +# --------------------------------------------------------------------------- # +print() +print("9. TEST-16 - the two arbitrage rungs, pinned from BOTH sides and " + "un-masked from each other") +# SAFETY-26's AC4 gave the same tolerance mutation to every price and quantity +# boundary in arbitrage.py and static_tariff.py. Everything in arbitrage.py went +# red. The two survivors are the two rungs graded below, and section 2 above is +# why they survived: it probes the flat rung with an EXACTLY FLAT series and +# never probes the firm-pair rung at all. +# +# D `spread == 0.0` weakened to `spread <= 0.001` SURVIVED (+2 B). An +# exactly flat series refuses under both spellings, so a tolerance is +# invisible to a flat probe. It is NOT cosmetic: a real 0.0005 spread +# flips ALLOWED -> REFUSED at min_spread_eur_kwh=0.0, which arbitrage()'s +# own docstring documents as supported ("pass 0.0 to mean any positive +# spread, explicitly") and which its validator at :861 accepts on purpose. +# +# F `len(firm) < 2` weakened to `< 1` SURVIVED, at +0 BYTES - the delta +# MUTATION.md names as the risky one. +# +# ⚠️ AND THE FINDING IS THE INTERACTION, NOT EITHER ROW. D AND F MASK EACH +# OTHER. A single firm hour has a spread of 0.0, so with F weakened the FLAT +# rung catches the stimulus and the verdict is still "refused"; with D weakened +# the tiny-spread stimulus is still refused, just for a lie. Each rung's failure +# is absorbed by the other, so BOTH can be weakened together and a suite that +# grades only `kind` reports nothing. That is invisible to single-mutation +# testing, which is how this pair survived four rounds of review on TARIFF-03 +# and FCST-03. +# +# TWO THINGS FOLLOW, and they shape every check below. +# +# 1. THE REASON IS GRADED, NOT JUST THE REFUSAL. The masking case REFUSES +# CORRECTLY WHILE REPORTING THE WRONG SENTENCE - an operator reading "the +# tariff is flat" about a tariff that is not flat, or reading it about a +# series whose real problem is that §8.3 has no firm pair to trade on. A +# check that asserts only "refused" passes on a broken tree. So each +# stimulus below is graded on WHICH RUNG ANSWERED IT. +# 2. EACH RUNG NEEDS A STIMULUS THE OTHER CANNOT ABSORB, or the pinning is +# circular. The flat rung gets a series with THREE firm hours, where the +# firm-pair rung cannot fire under any weakening down to `< 0`. The +# firm-pair rung gets a ONE-FIRM series, which the flat rung CAN answer - +# so its attribution is the whole check, and that is why row 2 of the table +# is written as a rung name rather than as a verdict. +# +# ⚠️ SHIPPED BEHAVIOUR IS UNCHANGED BY THIS SECTION. Both rungs are correct +# today; nothing in app/ was touched. This is the missing measurement, not a fix. +# +# THE LEDGER. Eight rounds through `tools/mutation_run.sh` (TEST-15): recursive +# `__pycache__` purge before each, `python3 -B`, the tree re-copied FROM DISK +# between rounds, the mutated line grepped out of the file before the run, and +# the byte delta printed. `test_static_tariff.py` and `test_arbitrage.py` graded +# each time. Both suites green on the unmutated tree. +# +# round delta test_static_tariff.py +# --------------------------------------------- ------ --------------------- +# baseline, shipped +0 B PASS +# D1 `spread == 0.0` -> `<= 0.001` +2 B RED, 3 checks +# D2 `spread == 0.0` -> `<= 0.000001` +5 B RED, AC1 alone +# D3 `spread == 0.0` -> `< 0.0` (rung gone) +0 B RED (section 2 too) +# F1 `len(firm) < 2` -> `< 1` +0 B RED, 3 checks +# F2 `len(firm) < 2` -> `< 3` +0 B RED, 6 checks +# F3 `len(firm) < 2` -> `< 0` (rung gone) +0 B RED (and ValueError) +# D1+F1 BOTH AT ONCE - THE TICKET +2 B RED, 5 checks +# +# Read the rows in pairs. D1 and F1 are SAFETY-26's two survivors and each is +# now red on its own; D2 is the tolerance bisected to the rung's own resolution, +# so "pinned at exactly zero" is measured rather than asserted at one epsilon; +# F2 and F3 pin the firm-pair count from ABOVE and BELOW, which the ticket's +# criteria do not ask for but which is what stops this section being one-sided +# in the way it was written to fix. THE LAST ROW IS THE TICKET: with BOTH rungs +# weakened together - the arrangement that reported nothing before this section +# existed - five checks go red, including AC1 and AC2, because neither of them +# is spelled in terms of the other rung's behaviour. +# +# ⚠️ FOUR OF THOSE SEVEN MUTATIONS ARE +0 BYTES, which MUTATION.md names as the +# delta stale bytecode can hide. They are all REDS, the direction the bytecode +# defect cannot fabricate - but they were run under the purge anyway, because +# "it went red so the cache cannot have lied" is a conclusion, not a procedure. + +_A9 = dict(peak_guard_active=False, min_spread_eur_kwh=0.0, + uncertainty=frozenset()) + + +def plan9(prices, provisional=None, product=PriceProduct.DAY_AHEAD): + """A bare hourly series, built here rather than through a profile. + + A `static_profile` cannot express a 0.0005 spread or a part-provisional day, + and going through one would grade the profile generator as well. These are + the smallest series that put exactly one rung in question. + """ + flags = provisional if provisional is not None else [False] * len(prices) + pts = [PricePoint(start=brussels(THU + timedelta(hours=i)), + eur_per_kwh=p, provisional=v) + for i, (p, v) in enumerate(zip(prices, flags))] + return TariffPlan(PriceRead.series(pts, product, "generic_dayahead", THU), + None, "provider") + + +def rung9(verdict): + """WHICH refusal answered, named from the reason the operator would read. + + Deliberately not a code inspection: the sentence in the log IS the artefact + that degrades when these two rungs mask each other, so it is the sentence + that gets graded. An unrecognised refusal is reported as such rather than + quietly matching nothing - "no rung fired" and "a rung I did not list fired" + must not grade the same. + """ + if verdict.kind == "allowed": + return "allowed" + r = verdict.reason + hit = [n for n, needle in (("flat", "the tariff is flat"), + ("firm-pair", "carry a published price"), + ("threshold", "needs"), + ("peak", "peak guard"), + ("no-series", "No series exists")) + if needle in r] + return hit[0] if len(hit) == 1 else f"UNATTRIBUTED<{r[:60]}>" + + +# --- the sweep D survived: a small but REAL spread, at min_spread 0.0 -------- +# ⚠️ NOT AN EXACTLY FLAT SERIES, which is the whole point - a flat probe cannot +# see a tolerance. The threshold is 0.0 so the `spread < need` rung below cannot +# fire either (required_spread(0.0, anything) is 0.0, asserted first), leaving +# the flat rung as the only thing that can refuse these. +check(f"the widened threshold at min_spread_eur_kwh=0.0 is still 0.0, so the " + f"probes below are graded by the FLAT rung and not by the threshold rung " + f"({required_spread(0.0, uncertainty_of(PriceProduct.DAY_AHEAD)):.6f})", + near(required_spread(0.0, uncertainty_of(PriceProduct.DAY_AHEAD)), 0.0)) + +# 1e-6 is the floor of the question, not of the check: app/static_tariff.py +# rounds the spread to 6 decimals before comparing, so a 5e-7 difference IS +# zero to this rung by construction. Graded as its own row below rather than +# left as an unstated edge. +TINY9 = (0.0005, 0.0001, 0.00001, 0.000001) +_tiny9 = [(d, plan9([0.20000, round(0.20000 + d, 6)])) for d in TINY9] +check(f"each tiny-spread probe really carries the spread it is named for, and " + f"two FIRM hours - so the firm-pair rung cannot be what answers them " + f"{[(f'{d:.6f}', f'{p.spread_eur_kwh():.6f}') for d, p in _tiny9]}", + all(near(p.spread_eur_kwh(), d, 1e-12) + and len([q for q in p.read.points if not q.provisional]) == 2 + for d, p in _tiny9)) +_tinyv9 = [(d, rung9(p.arbitrage(**_A9))) for d, p in _tiny9] +check(f"TEST-16 AC1: a spread that is SMALL BUT REAL is ALLOWED at " + f"min_spread_eur_kwh=0.0 - down to 0.000001 EUR/kWh, so the flat rung is " + f"`spread == 0.0` and not `spread <= epsilon` for any epsilon " + f"{_tinyv9}", + all(r == "allowed" for _d, r in _tinyv9)) +check("...and the exactly flat series on the OTHER side of that boundary is " + "still refused, and refused BY THE FLAT RUNG - without this the sweep " + "above would also pass for a rung that had been deleted outright", + rung9(plan9([0.20000, 0.20000]).arbitrage(**_A9)) == "flat") +_q9 = plan9([0.20000, 0.2000005]) +check(f"...and a difference below the rung's own 6-decimal rounding is flat BY " + f"CONSTRUCTION and says so: the raw difference is " + f"{_q9.spread_eur_kwh():.7f} and non-zero, `round(..., 6)` makes it " + f"exactly 0.0, and the rung agrees. That is the QUANTISATION the shipped " + f"line chooses in the open, not a tolerance smuggled into the comparison " + f"- and it is why the sweep above stops at 0.000001 rather than pretending " + f"to bisect further", + not near(_q9.spread_eur_kwh(), 0.0, 1e-9) + and near(round(_q9.spread_eur_kwh(), 6), 0.0, 1e-12) + and rung9(_q9.arbitrage(**_A9)) == "flat") + +# --- and what D's survival actually costs: the sentence, not the verdict ----- +_lie9 = plan9([0.20000, 0.20050]).arbitrage(**_A9) +check(f"TEST-16 AC4: the 0.0005 verdict carries the real spread rather than a " + f"flatness claim about a tariff that is not flat - the false sentence is " + f"what a loosened rung ships to the operator while the outcome at a " + f"shipped threshold looks unchanged ({_lie9.kind})", + _lie9.kind == "allowed" and near(_lie9.spread_eur_kwh, 0.0005) + and near(_lie9.cheapest.eur_per_kwh, 0.20000) + and near(_lie9.dearest.eur_per_kwh, 0.20050)) + +# --- the rung F survived: ON ITS OWN, not through the flat rung -------------- +# ⚠️ THIS ROW IS THE ANTI-MASKING ONE. A one-firm series has a spread of 0.0, +# so with `< 2` weakened to `< 1` the FLAT rung refuses it and `kind` never +# moves. The only thing that moves is the sentence: §8.3's honest "only 1 of 3 +# hours carry a published price" degrades to "the tariff is flat" about a series +# whose firm and provisional prices differ by 0.20 EUR/kWh. Grading the rung +# name is what makes `< 1` red without borrowing anything from the flat rung. +_one9 = plan9([0.20, 0.40, 0.10], [False, True, True]) +check(f"the one-firm probe is well formed: exactly ONE firm hour out of three, " + f"and its full series is anything but flat (spread " + f"{max(p.eur_per_kwh for p in _one9.read.points) - min(p.eur_per_kwh for p in _one9.read.points):.2f} " + f"EUR/kWh) - so 'flat' would be a false sentence about it", + len([p for p in _one9.read.points if not p.provisional]) == 1 + and len(_one9.read.points) == 3) +check("TEST-16 AC2: one firm hour is refused BY §8.3's FIRM-PAIR RUNG, named " + "in the reason - not by the flat rung that would also have refused it. " + "This is what takes `len(firm) < 2` -> `< 1` red on its own", + rung9(_one9.arbitrage(**_A9)) == "firm-pair") +check("...and the sentence is §8.3's in full, so the operator is told the " + "series is half-published rather than told it is fixed-price", + "§8.3" in _one9.arbitrage(**_A9).reason + and "only 1 of 3" in _one9.arbitrage(**_A9).reason + and "flat" not in _one9.arbitrage(**_A9).reason) +# The rung pinned from ABOVE as well. Without this, `len(firm) < 3` - the same +# +0 B edit in the other direction - would pass everything above. +_two9 = plan9([0.20, 0.40, 0.10], [False, False, True]) +check("...and TWO firm hours are ALLOWED on the identical series shape, so the " + "rung is pinned from above too and `< 3` is red as well as `< 1`", + rung9(_two9.arbitrage(**_A9)) == "allowed" + and near(_two9.arbitrage(**_A9).spread_eur_kwh, 0.20)) +# ...and from below, where the failure is not a wrong sentence but a crash: +# `max()` over an empty `firm` raises ValueError, so `< 0` does not degrade the +# reason, it takes the verdict away entirely. +_zero9 = plan9([0.20, 0.40, 0.10], [True, True, True]) +check("...and a FULLY provisional series is refused by the same rung and does " + "not raise - `max()` over an empty firm set is a ValueError, so this row " + "is what makes `len(firm) < 0` red", + rung9(derived(lambda: _zero9.arbitrage(**_A9))) == "firm-pair") + +# --- AC3: the two rungs do not mask each other ------------------------------ +# ⚠️ THE POINT OF THE TICKET, AND IT IS A PROPERTY OF THE TABLE, NOT OF A ROW. +# Masking is possible exactly when every stimulus that one rung answers could +# also be answered by the other with an acceptable-looking result. The table +# below denies that on both sides: +# +# * the flat rung is asked a question the firm-pair rung CANNOT reach - three +# firm hours, where `len(firm) < N` refuses for no N a person would write; +# * the firm-pair rung is asked a question the flat rung CAN reach - one firm +# hour - and the answer is graded on WHICH rung gave it, so the flat rung +# absorbing it is a failure rather than a pass; +# * and the tiny-spread row is allowed, which neither rung may refuse. +# +# Verified by running BOTH mutations at once (`spread <= 0.001` AND +# `len(firm) < 1`, purged harness, +2 B and +0 B): three rows move. Under +# single-mutation testing this arrangement reports nothing at all. +_TABLE9 = ( + ("three firm hours, all 0.2000", plan9([0.2, 0.2, 0.2]), "flat"), + ("two firm hours, 0.0005 apart", plan9([0.20000, 0.20050]), "allowed"), + ("one firm hour of three", _one9, "firm-pair"), + ("two firm hours of three", _two9, "allowed"), + ("no firm hours at all", _zero9, "firm-pair"), +) +_got9 = [(name, want, rung9(derived(lambda p=pl: p.arbitrage(**_A9)))) + for name, pl, want in _TABLE9] +check(f"TEST-16 AC3: each stimulus is answered by the rung that OWNS it, and " + f"by no other - the flat rung is asked a question the firm-pair rung " + f"cannot reach, and the firm-pair rung is asked one the flat rung could " + f"reach but must not be allowed to. Weakening EITHER rung moves a row; " + f"weakening BOTH TOGETHER still moves three " + f"{[(n, w, g) for n, w, g in _got9]}", + all(want == got for _n, want, got in _got9)) +check("...and the table really does exercise both rungs and the allowing path, " + "so 'every row matched' is not a statement about one rung repeated", + {want for _n, _p, want in _TABLE9} == {"flat", "firm-pair", "allowed"}) + + # --------------------------------------------------------------------------- # print() if fails: