TEST-16: pin the two one-sided arbitrage rungs, and un-mask them from each other
checks / unit (pull_request) Successful in 3m51s

SAFETY-26's AC4 gave the same tolerance mutation to every price and quantity
boundary in arbitrage.py and static_tariff.py. Everything in arbitrage.py went
red - `price < 0`, `soc <= floor_pct`, `soc >= ceiling_pct` are each red under
+/-5. Two survived, both in static_tariff.py, and this card closes them.

  D  `spread == 0.0` (:912) weakened to `spread <= 0.001` SURVIVED at +2 B.
  F  `len(firm) < 2` (:901) weakened to `len(firm) < 1` SURVIVED at +0 B - the
     delta MUTATION.md names as the risky one.

THE DEFECT IS NOT EITHER ROW, IT IS THE INTERACTION. D and F MASK EACH OTHER. A
single firm hour also has a spread of 0.0, so with F weakened the flat rung
refuses the stimulus anyway; with D weakened the small-but-real spread is still
refused, only for a lie. Each rung's failure is absorbed by the other, so BOTH
can be weakened together and a suite grading `kind == "refused"` reports
nothing. That is structurally invisible to single-mutation testing, which is how
this pair came through four rounds of review on TARIFF-03 and FCST-03 - and it
is the arrangement in which the next edit to either rung opens a real hole with
nothing to report it.

D is not cosmetic either. With the tolerance in place a series priced 0.20000 /
0.20050 - a REAL 0.0005 EUR/kWh spread - flips ALLOWED to REFUSED at
min_spread_eur_kwh=0.0, a value arbitrage()'s own docstring documents ("pass 0.0
to mean any positive spread, explicitly") and its validator at :861 accepts on
purpose. At a shipped threshold the outcome does not move, but the operator
reads the false sentence "the tariff is flat - every one of the 2 firm hours
costs 0.2000 EUR/kWh" about a tariff that is not flat.

WHAT WAS ADDED. test_static_tariff.py section 9, and the shape of it is the
argument:

  * THE REASON IS GRADED, NOT THE VERDICT. The masked case refuses CORRECTLY
    while reporting the wrong sentence, so a check asserting only "refused"
    passes on a broken tree. A `rung9()` helper names which refusal answered,
    from the sentence the operator would read; an unrecognised refusal reports
    itself as UNATTRIBUTED rather than quietly matching nothing.
  * EACH RUNG IS GIVEN A STIMULUS THE OTHER CANNOT ABSORB, or the pinning is
    circular. The flat rung gets a THREE-firm-hour flat series, which no
    `len(firm) < N` a person would write can reach. The firm-pair rung gets a
    ONE-FIRM-HOUR series, which the flat rung CAN reach - so its attribution is
    the whole check, and that is why it is written as a rung name rather than a
    verdict.
  * AC1 is a sweep of small-but-real spreads (0.0005 down to 0.000001) all
    ALLOWED at min_spread_eur_kwh=0.0, with the exactly flat series on the other
    side still refused BY THE FLAT RUNG - without that mirror the sweep would
    also pass for a rung deleted outright. 1e-6 is the floor because the shipped
    line rounds the spread to 6 decimals; that quantisation is graded in the
    open as its own row rather than left unstated.
  * AC3 is a five-row table graded on rung attribution, which denies the masking
    from both sides at once.
  * The firm-pair count is also pinned from ABOVE (`< 3`) and BELOW (`< 0`),
    which the criteria do not ask for but which is what stops this section being
    one-sided in exactly the way it was written to fix.

EVIDENCE. Eight rounds through TEST-15's tools/mutation_run.sh, used UNCHANGED:
recursive __pycache__ purge before each run, python3 -B, the tree re-copied FROM
DISK between rounds, the mutated line grepped out of the file before the run,
byte delta printed, test_static_tariff.py and test_arbitrage.py graded each
time. Baseline green.

  baseline, shipped                            +0 B  PASS
  D1  `spread == 0.0` -> `<= 0.001`            +2 B  RED, 3 checks
  D2  `spread == 0.0` -> `<= 0.000001`         +5 B  RED, AC1 alone
  D3  `spread == 0.0` -> `< 0.0`  (rung gone)  +0 B  RED (section 2 as well)
  F1  `len(firm) < 2` -> `< 1`                 +0 B  RED, 3 checks
  F2  `len(firm) < 2` -> `< 3`                 +0 B  RED, 6 checks
  F3  `len(firm) < 2` -> `< 0`  (rung gone)    +0 B  RED (ValueError)
  D1+F1  BOTH AT ONCE - THE TICKET             +2 B  RED, 5 checks

Four of the seven are +0 B, the delta MUTATION.md names as the one stale
bytecode can hide. All are REDS, the direction that defect cannot fabricate -
but they were run under the purge anyway, because "it went red so the cache
cannot have lied" is a conclusion and not a procedure.

NO SHIPPED BEHAVIOUR CHANGED. app/ is untouched; both rungs were correct all
along, and the card is explicit that a rung found actually wrong is a stop-and-
report rather than an edit. Also updated: MUTATION.md gains a short section on
masking as a SECOND way a survivor lies, with the two rules that fall out of it
(grade the reason, and mutate suspected pairs together); and test_arbitrage.py's
SAFETY-26 AC4 note is marked closed rather than left reading as an open finding.

UNSURE ABOUT. The 5e-7 row asserts the shipped `round(spread, 6)` quantisation.
That is a real property of the line and worth stating, but it does pin a
rounding the ticket never named - if a reviewer would rather that stayed
unpinned, it is one check to drop and nothing else depends on it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
glenn schrooyen
2026-09-01 07:37:19 +02:00
co-authored by Claude Opus 5
parent e8fb596b99
commit 20f9f4969d
4 changed files with 323 additions and 2 deletions
+31 -1
View File
@@ -97,6 +97,36 @@ State the things a reviewer cannot reconstruct afterwards:
4. for a **survivor**, that the run was purged. An unpurged survivor is not a
result.
## The other thing a mutation sweep cannot see: masking (TEST-16)
Stale bytecode is not the only way a survivor lies. **Two guards can mask each
other**, and a sweep that mutates one line at a time is structurally blind to it.
TEST-16 is the worked example. `static_tariff.arbitrage()` refuses a
one-firm-hour series at `len(firm) < 2`, and refuses a flat series at
`spread == 0.0`. A single firm hour *also* has a spread of zero. So:
* weaken the firm-pair rung alone → the flat rung refuses the stimulus anyway;
* weaken the flat rung alone → the tiny-spread stimulus is still refused, just
for the wrong reason;
* and the suite, grading `kind == "refused"`, reports **nothing either way**.
Both rungs could then be weakened *together* and still nothing would report —
which is precisely the state in which the next edit to either one opens a real
hole. It survived four rounds of review on TARIFF-03 and FCST-03.
Two rules come out of it, and both are cheap:
1. **Grade the reason, not just the verdict.** "Refused" and "refused for the
right reason" are different claims, and a masked rung is the case where the
first is true and the second is false. A refusal sentence is what the
operator reads; it is also the only observable that moves when one rung
quietly absorbs another's work.
2. **When two rungs can answer the same stimulus, mutate them together.** One
extra run per suspected pair. If a sweep finds two survivors in the same
function, that pair is the first thing to try — TEST-16's two survivors were
adjacent lines in one function and neither was individually live.
## Where each surface stands
* **Local and dev-harness runs — the exposed one.** This is where FCST-03 hit
@@ -136,7 +166,7 @@ commit and PR bodies.
| **FCST-03** M9 `self.provisional`→`here.provisional` (dd40c56, 71cd784) | **re-verified** | this is the original false survivor. Re-run with the cache cleared, M9 is caught. The shipped source was correct throughout. |
| **FCST-03** X1, X6 degenerate stimuli (71cd784) | **not at risk** | run after the purge discipline landed in that branch. |
| **SAFETY-24** 11 author + 9 reviewer mutations (e709554, 4601682) | **not at risk** | the commit records them run under `python -B` with a purged `__pycache__` and real byte deltas. Its one survivor is an equivalent mutant, argued structurally rather than by a green run. |
| **SAFETY-26** tolerance sweep over `arbitrage.py` and `static_tariff.py`, 2 survivors (2b8b8cb) | **not at risk** | purge, printed `+5 B` delta, mutated line asserted present, tree re-read before each run. One survivor is explicitly recorded at `+0 B` — the risky delta — and was produced *under* the purge. |
| **SAFETY-26** tolerance sweep over `arbitrage.py` and `static_tariff.py`, 2 survivors (2b8b8cb) | **not at risk** | purge, printed `+5 B` delta, mutated line asserted present, tree re-read before each run. One survivor is explicitly recorded at `+0 B` — the risky delta — and was produced *under* the purge. **Both survivors are now closed by TEST-16** (`test_static_tariff.py` section 9), which also found that the two of them *masked each other* — each was individually non-live only because the other rung caught its stimulus, an arrangement single-mutation testing cannot see. Its own ledger is 7 mutations, 4 of them at `+0 B`, all red. |
| **OPT-04** mutations reported on PR #22 (bdd7f18) | **not at risk** | `test_arbitrage.py`'s header records the exact command: purge, then `python3 -B`. |
| **TEST-17** `checks()` dropping `when` (504e8da) | **not at risk** | a *red* claim, and dropping an argument changes the file length. Neither half of the collision is available. |
| **SAFETY-07** wiring mutations B1/B2/B3 and `test_peakguard.py` parts 7 and 10 (2d570d8, e7c95bf) | **not at risk** | those mutations are **executed in-process**, monkeypatched onto the imported module. No source file is edited, so no `.pyc` is consulted. |