Merge pull request 'TEST-16: pin the two one-sided arbitrage rungs, and un-mask them from each other' (#37) from TEST-16 into release/1.0

This commit was merged in pull request #37.
This commit is contained in:
2026-09-01 08:02:57 +02:00
4 changed files with 323 additions and 2 deletions
+27
View File
@@ -2,6 +2,33 @@
## Unreleased
**The two `arbitrage()` rungs SAFETY-26's AC4 sweep found pinned from ONE SIDE
ONLY are now pinned from both - and, the actual finding, are pinned so they stop
MASKING EACH OTHER (TEST-16).** `spread == 0.0` weakened to `<= 0.001` survived
the whole suite at +2 B, and `len(firm) < 2` weakened to `< 1` survived at
+0 B - the delta MUTATION.md names as the risky one. Neither was individually
live, and that was the problem: a single firm hour *also* has a spread of zero,
so each rung's failure was absorbed by the other and both could be weakened
together with nothing reporting it. That is invisible to single-mutation
testing, which is how the pair survived four rounds of review on TARIFF-03 and
FCST-03. `test_static_tariff.py` gains section 9, which grades WHICH RUNG
answered each stimulus rather than only whether it refused - the masked case
refuses correctly while reporting the false sentence "the tariff is flat" about
a series that is not flat, so a check asserting only "refused" passes on a
broken tree. The flat rung is swept with small-but-REAL spreads down to
0.000001 EUR/kWh at `min_spread_eur_kwh=0.0` (a value `arbitrage()`'s docstring
documents and its validator accepts on purpose), the firm-pair rung is probed
with a ONE-FIRM-HOUR series and graded on §8.3's sentence coming back rather
than the flat rung's, and a five-row table denies the masking on both sides.
Verified through TEST-15's `tools/mutation_run.sh` unchanged - recursive
`__pycache__` purge, `python3 -B`, tree re-copied from disk between rounds,
mutated line grepped off disk, byte delta printed: 7 mutations plus a green
baseline, 4 of them at +0 B, ALL RED, including both rungs weakened AT ONCE
(5 checks red). **No shipped behaviour changed - both rungs were correct all
along**; `app/` is untouched. `MUTATION.md` gains a section on masking as a
second way a survivor lies, and `test_arbitrage.py`'s SAFETY-26 AC4 note is
marked closed rather than left reading as an open finding.
**`check_output_clamp()` could be moved above `write_setpoint()` in `cycle()`
with test_clamp.py, test_freeze.py and test_loop.py all staying GREEN — the
guard's write-path evidence, `self.clamp_last`, had nothing watching it.**
+31 -1
View File
@@ -97,6 +97,36 @@ State the things a reviewer cannot reconstruct afterwards:
4. for a **survivor**, that the run was purged. An unpurged survivor is not a
result.
## The other thing a mutation sweep cannot see: masking (TEST-16)
Stale bytecode is not the only way a survivor lies. **Two guards can mask each
other**, and a sweep that mutates one line at a time is structurally blind to it.
TEST-16 is the worked example. `static_tariff.arbitrage()` refuses a
one-firm-hour series at `len(firm) < 2`, and refuses a flat series at
`spread == 0.0`. A single firm hour *also* has a spread of zero. So:
* weaken the firm-pair rung alone → the flat rung refuses the stimulus anyway;
* weaken the flat rung alone → the tiny-spread stimulus is still refused, just
for the wrong reason;
* and the suite, grading `kind == "refused"`, reports **nothing either way**.
Both rungs could then be weakened *together* and still nothing would report —
which is precisely the state in which the next edit to either one opens a real
hole. It survived four rounds of review on TARIFF-03 and FCST-03.
Two rules come out of it, and both are cheap:
1. **Grade the reason, not just the verdict.** "Refused" and "refused for the
right reason" are different claims, and a masked rung is the case where the
first is true and the second is false. A refusal sentence is what the
operator reads; it is also the only observable that moves when one rung
quietly absorbs another's work.
2. **When two rungs can answer the same stimulus, mutate them together.** One
extra run per suspected pair. If a sweep finds two survivors in the same
function, that pair is the first thing to try — TEST-16's two survivors were
adjacent lines in one function and neither was individually live.
## Where each surface stands
* **Local and dev-harness runs — the exposed one.** This is where FCST-03 hit
@@ -136,7 +166,7 @@ commit and PR bodies.
| **FCST-03** M9 `self.provisional`→`here.provisional` (dd40c56, 71cd784) | **re-verified** | this is the original false survivor. Re-run with the cache cleared, M9 is caught. The shipped source was correct throughout. |
| **FCST-03** X1, X6 degenerate stimuli (71cd784) | **not at risk** | run after the purge discipline landed in that branch. |
| **SAFETY-24** 11 author + 9 reviewer mutations (e709554, 4601682) | **not at risk** | the commit records them run under `python -B` with a purged `__pycache__` and real byte deltas. Its one survivor is an equivalent mutant, argued structurally rather than by a green run. |
| **SAFETY-26** tolerance sweep over `arbitrage.py` and `static_tariff.py`, 2 survivors (2b8b8cb) | **not at risk** | purge, printed `+5 B` delta, mutated line asserted present, tree re-read before each run. One survivor is explicitly recorded at `+0 B` — the risky delta — and was produced *under* the purge. |
| **SAFETY-26** tolerance sweep over `arbitrage.py` and `static_tariff.py`, 2 survivors (2b8b8cb) | **not at risk** | purge, printed `+5 B` delta, mutated line asserted present, tree re-read before each run. One survivor is explicitly recorded at `+0 B` — the risky delta — and was produced *under* the purge. **Both survivors are now closed by TEST-16** (`test_static_tariff.py` section 9), which also found that the two of them *masked each other* — each was individually non-live only because the other rung caught its stimulus, an arrangement single-mutation testing cannot see. Its own ledger is 7 mutations, 4 of them at `+0 B`, all red. |
| **OPT-04** mutations reported on PR #22 (bdd7f18) | **not at risk** | `test_arbitrage.py`'s header records the exact command: purge, then `python3 -B`. |
| **TEST-17** `checks()` dropping `when` (504e8da) | **not at risk** | a *red* claim, and dropping an argument changes the file length. Neither half of the collision is available. |
| **SAFETY-07** wiring mutations B1/B2/B3 and `test_peakguard.py` parts 7 and 10 (2d570d8, e7c95bf) | **not at risk** | those mutations are **executed in-process**, monkeypatched onto the imported module. No source file is edited, so no `.pyc` is consulted. |
+10
View File
@@ -513,6 +513,16 @@ check(f"...while THE SAME MAGNITUDES with the sign flipped all DISCHARGE, so "
# Not fixed here on purpose. D and F are TARIFF-03's rungs, they need probes in
# test_static_tariff.py rather than in this file, and the card is explicit that
# its fourth criterion is a look and not a refactor.
#
# ✅ CLOSED BY TEST-16, in test_static_tariff.py SECTION 9, and the two rows
# above are kept as the diagnosis rather than as an open finding. D is pinned by
# a sweep of small-but-real spreads down to 0.000001 EUR/kWh at
# min_spread_eur_kwh=0.0; F is pinned by a ONE-FIRM-HOUR series graded on WHICH
# RUNG'S SENTENCE came back, which is the only way it goes red without borrowing
# the flat rung; and the masking is denied by a table that grades every stimulus
# on its rung's attribution rather than on `kind`. Both are red on their own and
# five checks are red with BOTH weakened at once. Ledger in that section's
# header. Nothing in app/ changed - these rungs were correct all along.
# --------------------------------------------------------------------------- #
+255 -1
View File
@@ -10,6 +10,11 @@ it:
AC2 Fully-fixed pricing DISABLES ARBITRAGE and leaves self-consumption only.
AC3 The Belgian scenario suite carries a static-profile regression case.
Section 9 is TEST-16 and is not one of those criteria. It pins the two
`arbitrage()` rungs SAFETY-26's AC4 sweep found pinned from ONE SIDE ONLY -
`spread == 0.0` and `len(firm) < 2` - and, more to the point, pins them so
they stop MASKING EACH OTHER. Read its header before editing either rung.
THREE RULES THIS FILE HOLDS ITSELF TO, because the reviewer will:
* EVERY VERDICT IS DERIVED. No `check("the split is right", True)`. Where a
@@ -62,7 +67,7 @@ from pathlib import Path
from app.price import (
Absence, PricePoint, PriceProduct, PriceRead, arbitrage_allowed, brussels,
plan_is_provisional,
plan_is_provisional, required_spread, uncertainty_of,
)
from app.static_tariff import (
MEAN_WINDOW_DAYS, ProfileInvalid, StaticProfile, TariffPlan,
@@ -860,6 +865,255 @@ check("...and main.py grew no `o.get(\"static_profile\", <a default>)` fallback
'get("static_profile"' not in _MAIN and "static_status(" in _MAIN)
# --------------------------------------------------------------------------- #
print()
print("9. TEST-16 - the two arbitrage rungs, pinned from BOTH sides and "
"un-masked from each other")
# SAFETY-26's AC4 gave the same tolerance mutation to every price and quantity
# boundary in arbitrage.py and static_tariff.py. Everything in arbitrage.py went
# red. The two survivors are the two rungs graded below, and section 2 above is
# why they survived: it probes the flat rung with an EXACTLY FLAT series and
# never probes the firm-pair rung at all.
#
# D `spread == 0.0` weakened to `spread <= 0.001` SURVIVED (+2 B). An
# exactly flat series refuses under both spellings, so a tolerance is
# invisible to a flat probe. It is NOT cosmetic: a real 0.0005 spread
# flips ALLOWED -> REFUSED at min_spread_eur_kwh=0.0, which arbitrage()'s
# own docstring documents as supported ("pass 0.0 to mean any positive
# spread, explicitly") and which its validator at :861 accepts on purpose.
#
# F `len(firm) < 2` weakened to `< 1` SURVIVED, at +0 BYTES - the delta
# MUTATION.md names as the risky one.
#
# ⚠️ AND THE FINDING IS THE INTERACTION, NOT EITHER ROW. D AND F MASK EACH
# OTHER. A single firm hour has a spread of 0.0, so with F weakened the FLAT
# rung catches the stimulus and the verdict is still "refused"; with D weakened
# the tiny-spread stimulus is still refused, just for a lie. Each rung's failure
# is absorbed by the other, so BOTH can be weakened together and a suite that
# grades only `kind` reports nothing. That is invisible to single-mutation
# testing, which is how this pair survived four rounds of review on TARIFF-03
# and FCST-03.
#
# TWO THINGS FOLLOW, and they shape every check below.
#
# 1. THE REASON IS GRADED, NOT JUST THE REFUSAL. The masking case REFUSES
# CORRECTLY WHILE REPORTING THE WRONG SENTENCE - an operator reading "the
# tariff is flat" about a tariff that is not flat, or reading it about a
# series whose real problem is that §8.3 has no firm pair to trade on. A
# check that asserts only "refused" passes on a broken tree. So each
# stimulus below is graded on WHICH RUNG ANSWERED IT.
# 2. EACH RUNG NEEDS A STIMULUS THE OTHER CANNOT ABSORB, or the pinning is
# circular. The flat rung gets a series with THREE firm hours, where the
# firm-pair rung cannot fire under any weakening down to `< 0`. The
# firm-pair rung gets a ONE-FIRM series, which the flat rung CAN answer -
# so its attribution is the whole check, and that is why row 2 of the table
# is written as a rung name rather than as a verdict.
#
# ⚠️ SHIPPED BEHAVIOUR IS UNCHANGED BY THIS SECTION. Both rungs are correct
# today; nothing in app/ was touched. This is the missing measurement, not a fix.
#
# THE LEDGER. Eight rounds through `tools/mutation_run.sh` (TEST-15): recursive
# `__pycache__` purge before each, `python3 -B`, the tree re-copied FROM DISK
# between rounds, the mutated line grepped out of the file before the run, and
# the byte delta printed. `test_static_tariff.py` and `test_arbitrage.py` graded
# each time. Both suites green on the unmutated tree.
#
# round delta test_static_tariff.py
# --------------------------------------------- ------ ---------------------
# baseline, shipped +0 B PASS
# D1 `spread == 0.0` -> `<= 0.001` +2 B RED, 3 checks
# D2 `spread == 0.0` -> `<= 0.000001` +5 B RED, AC1 alone
# D3 `spread == 0.0` -> `< 0.0` (rung gone) +0 B RED (section 2 too)
# F1 `len(firm) < 2` -> `< 1` +0 B RED, 3 checks
# F2 `len(firm) < 2` -> `< 3` +0 B RED, 6 checks
# F3 `len(firm) < 2` -> `< 0` (rung gone) +0 B RED (and ValueError)
# D1+F1 BOTH AT ONCE - THE TICKET +2 B RED, 5 checks
#
# Read the rows in pairs. D1 and F1 are SAFETY-26's two survivors and each is
# now red on its own; D2 is the tolerance bisected to the rung's own resolution,
# so "pinned at exactly zero" is measured rather than asserted at one epsilon;
# F2 and F3 pin the firm-pair count from ABOVE and BELOW, which the ticket's
# criteria do not ask for but which is what stops this section being one-sided
# in the way it was written to fix. THE LAST ROW IS THE TICKET: with BOTH rungs
# weakened together - the arrangement that reported nothing before this section
# existed - five checks go red, including AC1 and AC2, because neither of them
# is spelled in terms of the other rung's behaviour.
#
# ⚠️ FOUR OF THOSE SEVEN MUTATIONS ARE +0 BYTES, which MUTATION.md names as the
# delta stale bytecode can hide. They are all REDS, the direction the bytecode
# defect cannot fabricate - but they were run under the purge anyway, because
# "it went red so the cache cannot have lied" is a conclusion, not a procedure.
_A9 = dict(peak_guard_active=False, min_spread_eur_kwh=0.0,
uncertainty=frozenset())
def plan9(prices, provisional=None, product=PriceProduct.DAY_AHEAD):
"""A bare hourly series, built here rather than through a profile.
A `static_profile` cannot express a 0.0005 spread or a part-provisional day,
and going through one would grade the profile generator as well. These are
the smallest series that put exactly one rung in question.
"""
flags = provisional if provisional is not None else [False] * len(prices)
pts = [PricePoint(start=brussels(THU + timedelta(hours=i)),
eur_per_kwh=p, provisional=v)
for i, (p, v) in enumerate(zip(prices, flags))]
return TariffPlan(PriceRead.series(pts, product, "generic_dayahead", THU),
None, "provider")
def rung9(verdict):
"""WHICH refusal answered, named from the reason the operator would read.
Deliberately not a code inspection: the sentence in the log IS the artefact
that degrades when these two rungs mask each other, so it is the sentence
that gets graded. An unrecognised refusal is reported as such rather than
quietly matching nothing - "no rung fired" and "a rung I did not list fired"
must not grade the same.
"""
if verdict.kind == "allowed":
return "allowed"
r = verdict.reason
hit = [n for n, needle in (("flat", "the tariff is flat"),
("firm-pair", "carry a published price"),
("threshold", "needs"),
("peak", "peak guard"),
("no-series", "No series exists"))
if needle in r]
return hit[0] if len(hit) == 1 else f"UNATTRIBUTED<{r[:60]}>"
# --- the sweep D survived: a small but REAL spread, at min_spread 0.0 --------
# ⚠️ NOT AN EXACTLY FLAT SERIES, which is the whole point - a flat probe cannot
# see a tolerance. The threshold is 0.0 so the `spread < need` rung below cannot
# fire either (required_spread(0.0, anything) is 0.0, asserted first), leaving
# the flat rung as the only thing that can refuse these.
check(f"the widened threshold at min_spread_eur_kwh=0.0 is still 0.0, so the "
f"probes below are graded by the FLAT rung and not by the threshold rung "
f"({required_spread(0.0, uncertainty_of(PriceProduct.DAY_AHEAD)):.6f})",
near(required_spread(0.0, uncertainty_of(PriceProduct.DAY_AHEAD)), 0.0))
# 1e-6 is the floor of the question, not of the check: app/static_tariff.py
# rounds the spread to 6 decimals before comparing, so a 5e-7 difference IS
# zero to this rung by construction. Graded as its own row below rather than
# left as an unstated edge.
TINY9 = (0.0005, 0.0001, 0.00001, 0.000001)
_tiny9 = [(d, plan9([0.20000, round(0.20000 + d, 6)])) for d in TINY9]
check(f"each tiny-spread probe really carries the spread it is named for, and "
f"two FIRM hours - so the firm-pair rung cannot be what answers them "
f"{[(f'{d:.6f}', f'{p.spread_eur_kwh():.6f}') for d, p in _tiny9]}",
all(near(p.spread_eur_kwh(), d, 1e-12)
and len([q for q in p.read.points if not q.provisional]) == 2
for d, p in _tiny9))
_tinyv9 = [(d, rung9(p.arbitrage(**_A9))) for d, p in _tiny9]
check(f"TEST-16 AC1: a spread that is SMALL BUT REAL is ALLOWED at "
f"min_spread_eur_kwh=0.0 - down to 0.000001 EUR/kWh, so the flat rung is "
f"`spread == 0.0` and not `spread <= epsilon` for any epsilon "
f"{_tinyv9}",
all(r == "allowed" for _d, r in _tinyv9))
check("...and the exactly flat series on the OTHER side of that boundary is "
"still refused, and refused BY THE FLAT RUNG - without this the sweep "
"above would also pass for a rung that had been deleted outright",
rung9(plan9([0.20000, 0.20000]).arbitrage(**_A9)) == "flat")
_q9 = plan9([0.20000, 0.2000005])
check(f"...and a difference below the rung's own 6-decimal rounding is flat BY "
f"CONSTRUCTION and says so: the raw difference is "
f"{_q9.spread_eur_kwh():.7f} and non-zero, `round(..., 6)` makes it "
f"exactly 0.0, and the rung agrees. That is the QUANTISATION the shipped "
f"line chooses in the open, not a tolerance smuggled into the comparison "
f"- and it is why the sweep above stops at 0.000001 rather than pretending "
f"to bisect further",
not near(_q9.spread_eur_kwh(), 0.0, 1e-9)
and near(round(_q9.spread_eur_kwh(), 6), 0.0, 1e-12)
and rung9(_q9.arbitrage(**_A9)) == "flat")
# --- and what D's survival actually costs: the sentence, not the verdict -----
_lie9 = plan9([0.20000, 0.20050]).arbitrage(**_A9)
check(f"TEST-16 AC4: the 0.0005 verdict carries the real spread rather than a "
f"flatness claim about a tariff that is not flat - the false sentence is "
f"what a loosened rung ships to the operator while the outcome at a "
f"shipped threshold looks unchanged ({_lie9.kind})",
_lie9.kind == "allowed" and near(_lie9.spread_eur_kwh, 0.0005)
and near(_lie9.cheapest.eur_per_kwh, 0.20000)
and near(_lie9.dearest.eur_per_kwh, 0.20050))
# --- the rung F survived: ON ITS OWN, not through the flat rung --------------
# ⚠️ THIS ROW IS THE ANTI-MASKING ONE. A one-firm series has a spread of 0.0,
# so with `< 2` weakened to `< 1` the FLAT rung refuses it and `kind` never
# moves. The only thing that moves is the sentence: §8.3's honest "only 1 of 3
# hours carry a published price" degrades to "the tariff is flat" about a series
# whose firm and provisional prices differ by 0.20 EUR/kWh. Grading the rung
# name is what makes `< 1` red without borrowing anything from the flat rung.
_one9 = plan9([0.20, 0.40, 0.10], [False, True, True])
check(f"the one-firm probe is well formed: exactly ONE firm hour out of three, "
f"and its full series is anything but flat (spread "
f"{max(p.eur_per_kwh for p in _one9.read.points) - min(p.eur_per_kwh for p in _one9.read.points):.2f} "
f"EUR/kWh) - so 'flat' would be a false sentence about it",
len([p for p in _one9.read.points if not p.provisional]) == 1
and len(_one9.read.points) == 3)
check("TEST-16 AC2: one firm hour is refused BY §8.3's FIRM-PAIR RUNG, named "
"in the reason - not by the flat rung that would also have refused it. "
"This is what takes `len(firm) < 2` -> `< 1` red on its own",
rung9(_one9.arbitrage(**_A9)) == "firm-pair")
check("...and the sentence is §8.3's in full, so the operator is told the "
"series is half-published rather than told it is fixed-price",
"§8.3" in _one9.arbitrage(**_A9).reason
and "only 1 of 3" in _one9.arbitrage(**_A9).reason
and "flat" not in _one9.arbitrage(**_A9).reason)
# The rung pinned from ABOVE as well. Without this, `len(firm) < 3` - the same
# +0 B edit in the other direction - would pass everything above.
_two9 = plan9([0.20, 0.40, 0.10], [False, False, True])
check("...and TWO firm hours are ALLOWED on the identical series shape, so the "
"rung is pinned from above too and `< 3` is red as well as `< 1`",
rung9(_two9.arbitrage(**_A9)) == "allowed"
and near(_two9.arbitrage(**_A9).spread_eur_kwh, 0.20))
# ...and from below, where the failure is not a wrong sentence but a crash:
# `max()` over an empty `firm` raises ValueError, so `< 0` does not degrade the
# reason, it takes the verdict away entirely.
_zero9 = plan9([0.20, 0.40, 0.10], [True, True, True])
check("...and a FULLY provisional series is refused by the same rung and does "
"not raise - `max()` over an empty firm set is a ValueError, so this row "
"is what makes `len(firm) < 0` red",
rung9(derived(lambda: _zero9.arbitrage(**_A9))) == "firm-pair")
# --- AC3: the two rungs do not mask each other ------------------------------
# ⚠️ THE POINT OF THE TICKET, AND IT IS A PROPERTY OF THE TABLE, NOT OF A ROW.
# Masking is possible exactly when every stimulus that one rung answers could
# also be answered by the other with an acceptable-looking result. The table
# below denies that on both sides:
#
# * the flat rung is asked a question the firm-pair rung CANNOT reach - three
# firm hours, where `len(firm) < N` refuses for no N a person would write;
# * the firm-pair rung is asked a question the flat rung CAN reach - one firm
# hour - and the answer is graded on WHICH rung gave it, so the flat rung
# absorbing it is a failure rather than a pass;
# * and the tiny-spread row is allowed, which neither rung may refuse.
#
# Verified by running BOTH mutations at once (`spread <= 0.001` AND
# `len(firm) < 1`, purged harness, +2 B and +0 B): three rows move. Under
# single-mutation testing this arrangement reports nothing at all.
_TABLE9 = (
("three firm hours, all 0.2000", plan9([0.2, 0.2, 0.2]), "flat"),
("two firm hours, 0.0005 apart", plan9([0.20000, 0.20050]), "allowed"),
("one firm hour of three", _one9, "firm-pair"),
("two firm hours of three", _two9, "allowed"),
("no firm hours at all", _zero9, "firm-pair"),
)
_got9 = [(name, want, rung9(derived(lambda p=pl: p.arbitrage(**_A9))))
for name, pl, want in _TABLE9]
check(f"TEST-16 AC3: each stimulus is answered by the rung that OWNS it, and "
f"by no other - the flat rung is asked a question the firm-pair rung "
f"cannot reach, and the firm-pair rung is asked one the flat rung could "
f"reach but must not be allowed to. Weakening EITHER rung moves a row; "
f"weakening BOTH TOGETHER still moves three "
f"{[(n, w, g) for n, w, g in _got9]}",
all(want == got for _n, want, got in _got9))
check("...and the table really does exercise both rungs and the allowing path, "
"so 'every row matched' is not a statement about one rung repeated",
{want for _n, _p, want in _TABLE9} == {"flat", "firm-pair", "allowed"})
# --------------------------------------------------------------------------- #
print()
if fails: