TEST-16: pin the two one-sided arbitrage rungs, and un-mask them from each other
checks / unit (pull_request) Successful in 3m51s
checks / unit (pull_request) Successful in 3m51s
SAFETY-26's AC4 gave the same tolerance mutation to every price and quantity
boundary in arbitrage.py and static_tariff.py. Everything in arbitrage.py went
red - `price < 0`, `soc <= floor_pct`, `soc >= ceiling_pct` are each red under
+/-5. Two survived, both in static_tariff.py, and this card closes them.
D `spread == 0.0` (:912) weakened to `spread <= 0.001` SURVIVED at +2 B.
F `len(firm) < 2` (:901) weakened to `len(firm) < 1` SURVIVED at +0 B - the
delta MUTATION.md names as the risky one.
THE DEFECT IS NOT EITHER ROW, IT IS THE INTERACTION. D and F MASK EACH OTHER. A
single firm hour also has a spread of 0.0, so with F weakened the flat rung
refuses the stimulus anyway; with D weakened the small-but-real spread is still
refused, only for a lie. Each rung's failure is absorbed by the other, so BOTH
can be weakened together and a suite grading `kind == "refused"` reports
nothing. That is structurally invisible to single-mutation testing, which is how
this pair came through four rounds of review on TARIFF-03 and FCST-03 - and it
is the arrangement in which the next edit to either rung opens a real hole with
nothing to report it.
D is not cosmetic either. With the tolerance in place a series priced 0.20000 /
0.20050 - a REAL 0.0005 EUR/kWh spread - flips ALLOWED to REFUSED at
min_spread_eur_kwh=0.0, a value arbitrage()'s own docstring documents ("pass 0.0
to mean any positive spread, explicitly") and its validator at :861 accepts on
purpose. At a shipped threshold the outcome does not move, but the operator
reads the false sentence "the tariff is flat - every one of the 2 firm hours
costs 0.2000 EUR/kWh" about a tariff that is not flat.
WHAT WAS ADDED. test_static_tariff.py section 9, and the shape of it is the
argument:
* THE REASON IS GRADED, NOT THE VERDICT. The masked case refuses CORRECTLY
while reporting the wrong sentence, so a check asserting only "refused"
passes on a broken tree. A `rung9()` helper names which refusal answered,
from the sentence the operator would read; an unrecognised refusal reports
itself as UNATTRIBUTED rather than quietly matching nothing.
* EACH RUNG IS GIVEN A STIMULUS THE OTHER CANNOT ABSORB, or the pinning is
circular. The flat rung gets a THREE-firm-hour flat series, which no
`len(firm) < N` a person would write can reach. The firm-pair rung gets a
ONE-FIRM-HOUR series, which the flat rung CAN reach - so its attribution is
the whole check, and that is why it is written as a rung name rather than a
verdict.
* AC1 is a sweep of small-but-real spreads (0.0005 down to 0.000001) all
ALLOWED at min_spread_eur_kwh=0.0, with the exactly flat series on the other
side still refused BY THE FLAT RUNG - without that mirror the sweep would
also pass for a rung deleted outright. 1e-6 is the floor because the shipped
line rounds the spread to 6 decimals; that quantisation is graded in the
open as its own row rather than left unstated.
* AC3 is a five-row table graded on rung attribution, which denies the masking
from both sides at once.
* The firm-pair count is also pinned from ABOVE (`< 3`) and BELOW (`< 0`),
which the criteria do not ask for but which is what stops this section being
one-sided in exactly the way it was written to fix.
EVIDENCE. Eight rounds through TEST-15's tools/mutation_run.sh, used UNCHANGED:
recursive __pycache__ purge before each run, python3 -B, the tree re-copied FROM
DISK between rounds, the mutated line grepped out of the file before the run,
byte delta printed, test_static_tariff.py and test_arbitrage.py graded each
time. Baseline green.
baseline, shipped +0 B PASS
D1 `spread == 0.0` -> `<= 0.001` +2 B RED, 3 checks
D2 `spread == 0.0` -> `<= 0.000001` +5 B RED, AC1 alone
D3 `spread == 0.0` -> `< 0.0` (rung gone) +0 B RED (section 2 as well)
F1 `len(firm) < 2` -> `< 1` +0 B RED, 3 checks
F2 `len(firm) < 2` -> `< 3` +0 B RED, 6 checks
F3 `len(firm) < 2` -> `< 0` (rung gone) +0 B RED (ValueError)
D1+F1 BOTH AT ONCE - THE TICKET +2 B RED, 5 checks
Four of the seven are +0 B, the delta MUTATION.md names as the one stale
bytecode can hide. All are REDS, the direction that defect cannot fabricate -
but they were run under the purge anyway, because "it went red so the cache
cannot have lied" is a conclusion and not a procedure.
NO SHIPPED BEHAVIOUR CHANGED. app/ is untouched; both rungs were correct all
along, and the card is explicit that a rung found actually wrong is a stop-and-
report rather than an edit. Also updated: MUTATION.md gains a short section on
masking as a SECOND way a survivor lies, with the two rules that fall out of it
(grade the reason, and mutate suspected pairs together); and test_arbitrage.py's
SAFETY-26 AC4 note is marked closed rather than left reading as an open finding.
UNSURE ABOUT. The 5e-7 row asserts the shipped `round(spread, 6)` quantisation.
That is a real property of the line and worth stating, but it does pin a
rounding the ticket never named - if a reviewer would rather that stayed
unpinned, it is one check to drop and nothing else depends on it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
e8fb596b99
commit
20f9f4969d
@@ -97,6 +97,36 @@ State the things a reviewer cannot reconstruct afterwards:
|
||||
4. for a **survivor**, that the run was purged. An unpurged survivor is not a
|
||||
result.
|
||||
|
||||
## The other thing a mutation sweep cannot see: masking (TEST-16)
|
||||
|
||||
Stale bytecode is not the only way a survivor lies. **Two guards can mask each
|
||||
other**, and a sweep that mutates one line at a time is structurally blind to it.
|
||||
|
||||
TEST-16 is the worked example. `static_tariff.arbitrage()` refuses a
|
||||
one-firm-hour series at `len(firm) < 2`, and refuses a flat series at
|
||||
`spread == 0.0`. A single firm hour *also* has a spread of zero. So:
|
||||
|
||||
* weaken the firm-pair rung alone → the flat rung refuses the stimulus anyway;
|
||||
* weaken the flat rung alone → the tiny-spread stimulus is still refused, just
|
||||
for the wrong reason;
|
||||
* and the suite, grading `kind == "refused"`, reports **nothing either way**.
|
||||
|
||||
Both rungs could then be weakened *together* and still nothing would report —
|
||||
which is precisely the state in which the next edit to either one opens a real
|
||||
hole. It survived four rounds of review on TARIFF-03 and FCST-03.
|
||||
|
||||
Two rules come out of it, and both are cheap:
|
||||
|
||||
1. **Grade the reason, not just the verdict.** "Refused" and "refused for the
|
||||
right reason" are different claims, and a masked rung is the case where the
|
||||
first is true and the second is false. A refusal sentence is what the
|
||||
operator reads; it is also the only observable that moves when one rung
|
||||
quietly absorbs another's work.
|
||||
2. **When two rungs can answer the same stimulus, mutate them together.** One
|
||||
extra run per suspected pair. If a sweep finds two survivors in the same
|
||||
function, that pair is the first thing to try — TEST-16's two survivors were
|
||||
adjacent lines in one function and neither was individually live.
|
||||
|
||||
## Where each surface stands
|
||||
|
||||
* **Local and dev-harness runs — the exposed one.** This is where FCST-03 hit
|
||||
@@ -136,7 +166,7 @@ commit and PR bodies.
|
||||
| **FCST-03** M9 `self.provisional`→`here.provisional` (dd40c56, 71cd784) | **re-verified** | this is the original false survivor. Re-run with the cache cleared, M9 is caught. The shipped source was correct throughout. |
|
||||
| **FCST-03** X1, X6 degenerate stimuli (71cd784) | **not at risk** | run after the purge discipline landed in that branch. |
|
||||
| **SAFETY-24** 11 author + 9 reviewer mutations (e709554, 4601682) | **not at risk** | the commit records them run under `python -B` with a purged `__pycache__` and real byte deltas. Its one survivor is an equivalent mutant, argued structurally rather than by a green run. |
|
||||
| **SAFETY-26** tolerance sweep over `arbitrage.py` and `static_tariff.py`, 2 survivors (2b8b8cb) | **not at risk** | purge, printed `+5 B` delta, mutated line asserted present, tree re-read before each run. One survivor is explicitly recorded at `+0 B` — the risky delta — and was produced *under* the purge. |
|
||||
| **SAFETY-26** tolerance sweep over `arbitrage.py` and `static_tariff.py`, 2 survivors (2b8b8cb) | **not at risk** | purge, printed `+5 B` delta, mutated line asserted present, tree re-read before each run. One survivor is explicitly recorded at `+0 B` — the risky delta — and was produced *under* the purge. **Both survivors are now closed by TEST-16** (`test_static_tariff.py` section 9), which also found that the two of them *masked each other* — each was individually non-live only because the other rung caught its stimulus, an arrangement single-mutation testing cannot see. Its own ledger is 7 mutations, 4 of them at `+0 B`, all red. |
|
||||
| **OPT-04** mutations reported on PR #22 (bdd7f18) | **not at risk** | `test_arbitrage.py`'s header records the exact command: purge, then `python3 -B`. |
|
||||
| **TEST-17** `checks()` dropping `when` (504e8da) | **not at risk** | a *red* claim, and dropping an argument changes the file length. Neither half of the collision is available. |
|
||||
| **SAFETY-07** wiring mutations B1/B2/B3 and `test_peakguard.py` parts 7 and 10 (2d570d8, e7c95bf) | **not at risk** | those mutations are **executed in-process**, monkeypatched onto the imported module. No source file is edited, so no `.pyc` is consulted. |
|
||||
|
||||
Reference in New Issue
Block a user