SAFETY-38 round 2: assert the ordered pair, and correct the M3 attribution
checks / unit (pull_request) Successful in 5m58s
checks / unit (pull_request) Successful in 5m58s
Two blockers, one of them mine to fix and one a mis-measurement to undo. No
`app/` change in either: the shipped `sum` is correct.
1. A ZERO-BYTE SURVIVOR AT THE CALL SITE ONE LINE BELOW THE PINNED VALUE.
Round 1 pinned `site_standing` and never pinned WHERE IT GOES. Swapping the
detector's first two arguments at main.py:1620 -
`evaluate(site_standing, self.grid, ...)` -> `evaluate(self.grid, site_standing, ...)`
- is a ZERO-BYTE edit and left the round-1 tree fully green. The cost is the
banner: the site stands at -500 W of charge into a +2500 W import and the
LATCHING, SITE-STOPPING alarm reads "commanded +2500 W against a meter
reading -500 W" - which instrument read what, inverted, on the one line
somebody woken by it reads first (watchdog.py:658-665).
Round 1's check could not reach it. It asserted `"-500 W"` appears SOMEWHERE
in the sentence; under the swap it still does, IN THE METER SLOT. A substring
test grades PRESENCE, never ARRANGEMENT. It now asserts the ORDERED PAIR:
`f"commanded {sum:+.0f} W against a meter reading {grid:+.0f} W"`. Measured:
red in test_watchdog.py on that one named check, green in the other twenty.
2. THE M3 ATTRIBUTION WAS FALSE, AND HOW IT BECAME FALSE IS THE USEFUL PART.
Round 1 recorded `list(standing.values())[0]` as closed by SAFETY-32's
package-wide pin and told future authors NOT TO RE-COVER IT. Re-measured on
release/1.0 @ 86eaef2 carrying only that edit: the full 21-file suite is
GREEN and test_peakguard.py is PASS PASS PASS PASS PASS in isolation. THE
MUTATION SURVIVES. It was originally measured once, that run hit CI-06's
~20% flake, and the red was recorded as a kill.
The mechanism, read off 12j's own enumeration rather than argued: 12j taints
names BOUND to roster values and lists ACCUMULATED in a loop. `standing` is a
dict filled by SUBSCRIPT assignment (main.py:1554/1558), so it never enters
the taint set and 1599 is not a roster site. On the mutated tree 12j lists
main.py:1553, 1555 and 1690 - not 1599. Part 9a and 9b close it.
Corrected in all four places it was committed: DECISIONS section 1 and
section 4, CHANGELOG, and the part 9 header that carried the instruction.
MEASUREMENT, all on 86eaef2, fresh copy per mutation, mounted :ro, bytecode
purged by tools/mutation_run.sh, edit re-read from disk, byte delta printed:
* M1-M10 re-run INDEPENDENTLY rather than carried forward - all 10 red, at
the deltas the table records (M1/M2/M4 re-run in the exact wrapped form).
* M11, the argument swap: 0 bytes, red on one check in one file.
* W1 `"sign_site": SCOPE_SITE -> SCOPE_UNIT`: 0 bytes, red; part 9a is one of
the two killers.
* W2 `"circulation": SCOPE_SITE -> SCOPE_UNIT`: 0 bytes, red - but ENTIRELY
on pre-existing checks. This card adds nothing to circulation and does not
claim it.
Control: 21/21 green before and after. The suite is 21 files, not 20 - TEL-07
added test_mqtt.py in the rebase onto 86eaef2.
DECISIONS section 6a enumerates what was NOT closed: SAFETY-31's circ_wd
detector itself, the other eight CLAIM_SCOPE entries, actuator_of()'s body, the
n_units > 1 guard, and the fact that the "green in the other twenty files" half
of the ledger was re-verified on 86eaef2 only for M3 and M11.
MUTATION.md gains habit 14 - assert the ordered pair, not the substring - and
records this as the tenth instance of the family, found INSIDE THE FIX WRITTEN
TO CLOSE THE NINTH, for the second time in that list.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
This commit is contained in:
co-authored by
Claude Opus 5
parent
eb7a1e754e
commit
ce46a4b76d
@@ -33,19 +33,55 @@ Both are P0 paths. **Neither was pinned to the reduction.**
|
||||
## 1. What survived, and what the rescope measured
|
||||
|
||||
The card opened with three demonstrated live mutants, all green across the whole
|
||||
suite. Re-measured on 2026-09-05 against `release/1.0` @ `f72c48d`, after
|
||||
[[SAFETY-32]] merged:
|
||||
suite. Re-measured in round 2 against `release/1.0` @ `86eaef2`:
|
||||
|
||||
| mutation of `site_standing` | before SAFETY-32 | after SAFETY-32 |
|
||||
| mutation of `site_standing` | on release/1.0 @ 86eaef2 | closed by |
|
||||
|---|---|---|
|
||||
| `list(standing.values())[0]` — one unit's share | survives | **CAUGHT** by SAFETY-32's package-wide structural pin |
|
||||
| `sum(...) / max(1, len(standing))` — the mean | survives | **still survives** |
|
||||
| `max(standing.values(), key=abs)` — the largest share | survives | **still survives** |
|
||||
| `list(standing.values())[0]` — one unit's share | **SURVIVES**, full suite 21/21 green | part 9a + 9b (this card) |
|
||||
| `sum(...) / max(1, len(standing))` — the mean | **SURVIVES**, full suite 21/21 green | part 9a + 9b (this card) |
|
||||
| `max(standing.values(), key=abs)` — the largest share | **SURVIVES**, full suite 21/21 green | part 9a + 9b (this card) |
|
||||
|
||||
SAFETY-32's pin catches *a slice or index standing between a roster-derived
|
||||
value and its reduction*, on the line it is written, at every roster length,
|
||||
with no fixture — which is exactly `[0]`. Its own disclosed blind spot says why
|
||||
the other two are out of reach:
|
||||
### ⚠️ ROUND 1 SAID `[0]` WAS ALREADY CLOSED BY SAFETY-32. THAT WAS FALSE.
|
||||
|
||||
Round 1 of this card recorded `[0]` as **CAUGHT by SAFETY-32's package-wide
|
||||
structural pin**, wrote that claim into four places, and told future authors
|
||||
*"DO NOT RE-COVER THAT ONE HERE"*. It is wrong, and **how** it went wrong is the
|
||||
part worth keeping.
|
||||
|
||||
**How the error was made.** The mutation was measured **once**. That single run
|
||||
happened to hit `test_peakguard.py`'s [[CI-06]] flake — a **measured ~20%**
|
||||
false-red rate — and the red was recorded as a **mutation kill**. Nothing else
|
||||
was checked, because a red is the answer you are hoping for and it did not look
|
||||
like it needed a second run.
|
||||
|
||||
**The correct measurement.** On `release/1.0` @ `86eaef2` carrying **only** the
|
||||
`[0]` edit (+4 bytes, re-read from disk, bytecode purged by
|
||||
`tools/mutation_run.sh`): the **full 21-file suite is green**, and
|
||||
`test_peakguard.py` run **five times in isolation** on that same mutated tree is
|
||||
**PASS PASS PASS PASS PASS**. The mutation **survives**. (The reviewer's own
|
||||
independent re-measurement of the same tree hit the flake once —
|
||||
PASS PASS PASS FAIL PASS — which is the same verdict plus a live demonstration
|
||||
of the mechanism that produced the original error.)
|
||||
|
||||
**Why SAFETY-32 cannot see it, read off 12j's own enumeration rather than
|
||||
argued.** Part 12j taints names *bound* to roster-carrying values and lists
|
||||
*accumulated* in a loop over one. `standing` is neither: it is a dict created
|
||||
empty at `main.py:1554` and filled by **subscript assignment**
|
||||
(`standing[inv.name] = stood`) inside the roster loop, and a subscript store is
|
||||
not a binding 12j follows. So `standing` never enters its taint set and
|
||||
`sum(standing.values())` is **not a roster site** at all. Run 12j against a tree
|
||||
carrying the `[0]` mutation and its enumeration lists `main.py:1553`, `1555` and
|
||||
`1690` — the three `self.inverters` sites bracketing this code — and **not
|
||||
`1599`**. No shape pin reaches this line; only the behavioural fixture below
|
||||
does.
|
||||
|
||||
**The rule this cost.** `test_peakguard.py` flakes at a measured ~20%. A single
|
||||
red from it proves nothing. Re-run it in isolation **at least five times** and
|
||||
**name the failing check** before believing any red from it. This card is the
|
||||
**third** measurement CI-06 has corrupted.
|
||||
|
||||
SAFETY-32's own disclosed blind spot says why the other two were always out of
|
||||
reach:
|
||||
|
||||
> *SEMANTICS. `min` → `max` → the mean are edits invisible to any shape pin,
|
||||
> permanently. Only the behavioural sets can see them.*
|
||||
@@ -54,6 +90,13 @@ the other two are out of reach:
|
||||
different meaning. **No shape pin will ever close them.** That disclosure was
|
||||
accurate, and is now confirmed by measurement rather than taken on trust.
|
||||
|
||||
`[0]` is the one that *looks* structural and therefore *looks* covered — and
|
||||
that appearance is exactly what produced the false attribution above. It is
|
||||
out of reach here for a different reason from the other two: not because the
|
||||
shape is invisible, but because **the value never becomes a roster** in the
|
||||
analysis's eyes. All three are closed by the same behavioural fixture, and
|
||||
**all three needed it.**
|
||||
|
||||
## 2. Why the suite could not see it — habit 4, inside the site half of the very card that names it
|
||||
|
||||
`test_watchdog.py`'s `_sign_site` is the **only** N≥2 site fixture in the
|
||||
@@ -174,13 +217,13 @@ Every run: a **fresh full-repo copy**, `__pycache__` and `*.pyc` excluded,
|
||||
mounted `:ro`, graded by `tools/mutation_run.sh` (which purges, prints the tree
|
||||
fingerprint, and verifies no cache appears during the run). The mutated line was
|
||||
**re-read from disk** after the edit and the byte delta printed. Shipped
|
||||
`app/main.py` is 232 520 bytes.
|
||||
`app/main.py` is 234 542 bytes at `86eaef2`.
|
||||
|
||||
| # | mutation of `site_standing` | Δ bytes | verdict |
|
||||
|---|---|---|---|
|
||||
| M1 | `sum(...) / max(1, len(standing))` — the mean | +24 | **RED** |
|
||||
| M2 | `max(standing.values(), key=abs)` — largest share | +9 | **RED** |
|
||||
| M3 | `list(standing.values())[0]` — one share | +4 | **RED** |
|
||||
| M3 | `list(standing.values())[0]` — one share | +4 | **RED** — and **survives on release/1.0**, see §1 |
|
||||
| M4 | `min(standing.values(), key=abs)` — smallest share | +9 | **RED** |
|
||||
| M5 | `sum(abs(v) for v in ...)` — magnitude, sign lost | +16 | **RED** |
|
||||
| M6 | `-sum(standing.values())` — sign flipped | +1 | **RED** |
|
||||
@@ -188,12 +231,15 @@ fingerprint, and verifies no cache appears during the run). The mutated line was
|
||||
| M8 | `sum(standing.values()) / 2.0` | +6 | **RED** |
|
||||
| M9 | `sum(standing.values()) * 2.0` | +6 | **RED** |
|
||||
| M10 | `sorted(standing.values())[-1]` — largest signed | +7 | **RED** |
|
||||
| **M11** | **argument SWAP** at the call site: `evaluate(site_standing, self.grid, …)` → `evaluate(self.grid, site_standing, …)` (main.py:1620) | **0** | **was a SURVIVOR in round 1 — now RED**, see §4a |
|
||||
|
||||
**Every one of the ten is red in `test_watchdog.py` and in nothing else** — the
|
||||
other 19 files are green under all ten, which is the measurement of how
|
||||
unpinned this line was: before part 9 existed, three of these left the whole
|
||||
suite green and the remaining seven were caught by SAFETY-32's structural pin or
|
||||
by SAFETY-30's own N≥2 checks, none of which grade the *quantity*.
|
||||
**Every one of the ten is red in `test_watchdog.py`.** Re-measured in round 2
|
||||
on `release/1.0` @ `86eaef2` (where `app/main.py` is 234 542 bytes, 2 022 more
|
||||
than the `f72c48d` the round-1 figures were taken on): all ten still red, at the
|
||||
byte deltas in the table, each on a fresh copy with the edit re-read from disk.
|
||||
Before part 9 existed, **three** of these left the whole suite green — M1, M2
|
||||
**and M3** — and the remaining seven were caught by SAFETY-30's own N≥2 checks,
|
||||
none of which grade the *quantity*.
|
||||
|
||||
⚠️ **ONE FALSE RED, AND IT IS [[CI-06]]'s.** In the batch run M10 also reported
|
||||
`FAIL test_peakguard.py`. Re-run in isolation **three times** and once more
|
||||
@@ -204,9 +250,76 @@ and that **no claim in this card rests on it**. `test_peakguard.py` is the file
|
||||
CI-06 records as flaky, and this is the behaviour it records. Believe no red
|
||||
from it without three isolated re-runs.
|
||||
|
||||
M3 is listed for completeness only. SAFETY-32 already closes it **structurally**
|
||||
and this card does not claim it; that the behavioural fixture also catches it is
|
||||
a free consequence of the same discriminating numbers, not a second pin.
|
||||
⚠️ **M3 IS NOT "FOR COMPLETENESS", AND ROUND 1 SAID IT WAS.** Round 1 recorded
|
||||
M3 as already closed structurally by SAFETY-32 and disclaimed it. §1 above has
|
||||
the correction and the mechanism: `[0]` **survives** a full green 21-file suite
|
||||
on `release/1.0` @ `86eaef2`, because `standing` is a dict filled by subscript
|
||||
assignment and never enters 12j's taint set. **M3 is a real kill by this card's
|
||||
fixture and by nothing else in the repository.** It is claimed here.
|
||||
|
||||
### 4a. ⚠️ The zero-byte survivor one line below the one this card pinned
|
||||
|
||||
Round 1 pinned the **value** `site_standing` and never pinned **where it goes**.
|
||||
Swapping the detector's first two arguments at `main.py:1620` is a **zero-byte**
|
||||
edit — the same characters, reordered — and it left the round-1 tree **21/21
|
||||
green**.
|
||||
|
||||
What it costs is the banner. `SignWatchdog.evaluate()` formats
|
||||
`commanded_w` first and `against_w` second, so under the swap part 9a's own
|
||||
output reads:
|
||||
|
||||
```
|
||||
SIGN_DISAGREEMENT: commanded +2500 W against a meter reading -500 W for 2 s …
|
||||
```
|
||||
|
||||
The site is standing at **−500 W of charge into a +2500 W import** and the
|
||||
**latching, site-stopping** banner tells the operator **the exact opposite** —
|
||||
which instrument read what, inverted. `watchdog.py:658-665` names this failure
|
||||
in its own words: *"a false statement about which instrument produced the
|
||||
evidence, on the one line somebody woken by this alarm reads first."*
|
||||
|
||||
**Why round 1's check could not see it, and this is the general lesson.** It
|
||||
asserted
|
||||
|
||||
```python
|
||||
f"{sum(_share38):.0f} W" in _c38.site_sign_wd.reason
|
||||
```
|
||||
|
||||
— that `"-500 W"` appears **somewhere** in the sentence. Under the swap it still
|
||||
does: in the **meter slot**. A substring test cannot reach a failure that is
|
||||
purely **positional**, and round 1's own comment claimed the opposite ("*would
|
||||
put a different number in the store, the action log and the banner, and this is
|
||||
where that shows*"). It shows the **number**, never its **position**. This is
|
||||
MUTATION.md's *"a check whose data cannot reach the failure it names"*, in the
|
||||
card that exists to complain about exactly that.
|
||||
|
||||
**The fix is one line and touches no `app/` code** — assert the **ordered
|
||||
pair**:
|
||||
|
||||
```python
|
||||
f"commanded {sum(_share38):+.0f} W against a meter reading "
|
||||
f"{_mid38['grid']:+.0f} W" in _c38.site_sign_wd.reason
|
||||
```
|
||||
|
||||
Measured: with that line, M11 is **red in `test_watchdog.py` and green in the
|
||||
other twenty files** — one file, one check.
|
||||
|
||||
### 4b. Claim wiring, re-measured rather than assumed
|
||||
|
||||
Round 1's reviewer disclosed it had **not** mutated the arbiter claim wiring or
|
||||
[[SAFETY-31]]/circulation. Both were mutated in round 2. Both are **zero-byte**
|
||||
edits to `arbiter.CLAIM_SCOPE`:
|
||||
|
||||
| # | mutation | Δ bytes | verdict |
|
||||
|---|---|---|---|
|
||||
| W1 | `"sign_site": SCOPE_SITE` → `SCOPE_UNIT` | **0** | **RED** — `test_watchdog.py` (parts 5 **and 9a**), `test_arbiter.py`, `test_peakguard.py` |
|
||||
| W2 | `"circulation": SCOPE_SITE` → `SCOPE_UNIT` | **0** | **RED** — `test_watchdog.py`, `test_arbiter.py`, `test_peakguard.py` |
|
||||
|
||||
Part 9a's *"…and it stops the WHOLE site, through the `sign_site` claim by name"*
|
||||
is one of the two checks that kills W1, so this card contributes to that
|
||||
coverage rather than merely inheriting it. **W2 is caught entirely by
|
||||
pre-existing checks** — SAFETY-38 adds nothing to circulation, and does not
|
||||
claim to.
|
||||
|
||||
## 5. The survivor this card went looking for, and closed
|
||||
|
||||
@@ -250,3 +363,49 @@ operator typed.
|
||||
* **Everything about inverter B here is simulated and labelled so.** No number
|
||||
in this document or in the fixture is a measurement of a device that does not
|
||||
yet exist.
|
||||
|
||||
### 6a. Round 2: what was closed, and what was NOT
|
||||
|
||||
Round 1's reviewer disclosed that it had not mutated SAFETY-31/circulation, the
|
||||
arbiter claim wiring, or ledger entries M4/M5/M6/M8/M9/M10. Enumerated, with the
|
||||
gaps left open stated rather than rounded off:
|
||||
|
||||
**Closed in round 2 (measured, on `86eaef2`, bytecode purged, edit re-read from
|
||||
disk, byte delta printed):**
|
||||
|
||||
1. **M11**, the zero-byte argument swap — §4a. Red on one named check.
|
||||
2. **M3**, the `[0]` reduction — re-measured, **survives on release/1.0**, closed
|
||||
by part 9a and 9b. The false attribution corrected in all four places it was
|
||||
committed: this file §1 and §4, `CHANGELOG.md`, and `test_watchdog.py`'s part
|
||||
9 header (which previously *instructed future authors not to re-cover it*).
|
||||
3. **M1, M2, M4, M5, M6, M7, M8, M9, M10** — every ledger entry re-run
|
||||
independently in round 2 rather than carried forward from round 1's report.
|
||||
All ten red; M1/M2/M4 re-run in the exact wrapped form the table records so
|
||||
the byte deltas are the table's own (+24/+9/+9).
|
||||
4. **W1**, `sign_site` claim scope — §4b. Zero bytes, red, part 9a among the
|
||||
killers.
|
||||
5. **W2**, `circulation` claim scope — §4b. Zero bytes, red, but **entirely on
|
||||
pre-existing checks**.
|
||||
|
||||
**NOT closed, and not claimed:**
|
||||
|
||||
* **SAFETY-31's `circ_wd` detector itself** — its threshold, its hold, its
|
||||
`circulating_w()` reduction over the roster. Only its **claim scope** was
|
||||
mutated (W2), and that kill belongs to checks this card did not write.
|
||||
Circulation's own behavioural coverage is `test_watchdog.py` part 4's and is
|
||||
untouched here.
|
||||
* **The other eight `CLAIM_SCOPE` entries** — `loop`, `peak`, `arbitrage`,
|
||||
`safety` (site) and `device`, `supervised`, `failsafe`, `maintenance` (unit).
|
||||
`CLAIM_SCOPE` has ten entries; only the two named above were mutated. `test_peakguard.py` part 10f grades the
|
||||
classification statically; that is a different instrument and this card did
|
||||
not re-measure it.
|
||||
* **`actuator_of()`'s own body** and the `n_units > 1` guard on the `sign_site`
|
||||
append at `main.py:1764` were **not** mutated.
|
||||
* **The ten ledger mutations were re-graded against `test_watchdog.py` only.**
|
||||
The "green in the other twenty files" half of the round-1 claim was
|
||||
re-verified on `86eaef2` for **M3** and **M11** (both full 21-file runs) and is
|
||||
otherwise carried forward from round 1's measurement on `f72c48d`. It is a
|
||||
weaker statement than round 1 made and is written down as such.
|
||||
* **`test_peakguard.py`'s CI-06 flake is not fixed by this card** and is not
|
||||
this card's subject. It is *characterised* here (five isolated runs on the M3
|
||||
tree, all PASS) only far enough to overturn the false kill it produced.
|
||||
|
||||
Reference in New Issue
Block a user