SAFETY-32 round 5: stop closing sites, build an instrument that finds its own scope
checks / unit (pull_request) Successful in 4m59s

Rounds 2, 3 and 4 each closed exactly the survivor they were handed and each time
the reviewer found the same shape one step over: the reduction, the call site, the
sibling argument, and now the sibling MODULE. Round 4's enumeration stopped at the
`Controller` class boundary; round 5's four blockers were reductions in validate.py,
over a list DERIVED from the roster three calls away from it. Closing four more sites
by hand would have produced a fifth rejection, so this round widens the instrument
instead and lets it enumerate.

⚠️ THE BLOCKERS ARE NOT EQUIVALENT MUTANTS - a safety refusal disappears, verified by
calling the shipped validator both ways on the real tree:

  roster max_w 5000 (inverter-a) beside 2000 (inverter-b), sign_disagree_w 3000
    clean                        -> 1 problem, "...max_w (inverter-b) is 2000 W, so the
                                    SAFETY-02 sign-disagreement watchdog can never fire"
    min(caps[:1])          +4 B  -> 0 problems. The site STARTS.
    caps_ix[:1]            +4 B  -> 0 problems.
  the SAME roster with the small cap FIRST (2000, 5000)
    clean                        -> 1 problem, naming inverter-a
    min(caps[-1:])         +5 B  -> 0 problems.

⚠️ AND THE ONE-ROSTER FIXTURE COULD NOT HAVE SEEN THAT. On [5000, 2000] the `[-1:]`
mutant still refuses, because last IS min there. A fixture of N members cannot separate
two reductions that agree on N members - which is why the behavioural half below is a
SET and not a roster.

THE INSTRUMENT (test_peakguard.py part 12j, widened past `Controller`)

Every `.py` under app/ is read off disk and parsed - NO MODULE IS NAMED IN THE CHECK -
and the roster is found by transitive closure from ONE seed, the `inverters` option key:
through returns, through lists ACCUMULATED in a loop over a roster (which is exactly
what `caps`, `roster` and `problems` are, and what round 4's rule could not express),
through attributes and properties, and THROUGH CALL ARGUMENTS, which is how it walks
into peakguard, arbiter, allocator, circulation, maintenance and store without being
told they exist. A slice of a roster stays a roster so a mutation cannot cut the taint;
an index is one unit and does not.

  183 roster reductions across 9 modules, out of 23 files parsed:
  main 96, allocator 24, validate 19, arbiter 14, circulation 10, maintenance 10,
  peakguard 4, inverter 3, store 3.  14 modules reduce over no roster at all.

⚠️ THE HAND-WRITTEN MODULE LIST WOULD HAVE BEEN WRONG IN BOTH DIRECTIONS. It named
watchdog.py, which reduces over no roster at all (SAFETY-30's sign watchdogs are
strictly per-unit), and omitted maintenance.py and store.py, which both do. A vacuous
claim about one module and no claim at all about two others - which is the argument for
computing the scope, made with data.

The pin rediscovers ALL FOUR blockers without being told where to look, and finds a
FIFTH beside them (validate.py:783, rule 5's own message) that no review round named.
Its counterfactual runs at two depths: every enumerated site's expression mutated with
each of the four cuts this card has been defeated by (183 x 4 = 732 spellings, all
reported), and `[:1]` written into app/validate.py at each blocker position with the
WHOLE analysis re-run over the mutated source.

THE SWEEP, AND THE NUMBER THAT MUST NOT BE QUOTED ALONE

All 183 sites mutated to `[:1]`, +4 B each, against the full 19-file suite in
goodwe-addon-ci (:ro, -B, fresh rsync per mutant, 0 .pyc before and after - a read-only
mount cannot write one): 183/183 RED. ⚠️ That is TAUTOLOGICAL on its own, because the
pin makes every slice red. So the 51 whose first red was test_peakguard.py were re-run
with the pin NEUTRALISED:

  145 of 183   a behavioural fixture separates the site from a slice of itself
   38 of 183   ONLY the structural claim. Behaviourally UNGRADED.

All 38 are listed by site with what a `[:1]` there would cost, in the DECISIONS record
and in part 12j itself: 15 are rows and counts on the operator's page, 3 are
`len(targets)` arithmetic beside a write path that IS graded, 6 are allocator internals
ALLOC-01 reaches but not with enough units to separate. NONE of the 38 is closed
behaviourally by this card and none is counted as covered.

THE BEHAVIOURAL HALF, because a shape pin grades an identity and never a meaning

test_validate.py grows a ROSTER SET for the three rules that reduce over
`unit_caps_indexed()`: widths 2, 3, 4, 5 with the one small cap at EVERY position,
14 rosters x 3 rules = 42 runs of the real validator, each asserting the refusal fires
AND names the unit the number came off. Beside it a COMPUTED family table of twelve
candidates - max, first, last, the mean, [:1], [-1:], [:2], [:3], [1:], [:-1], [::2] -
each compared against what the shipped validator actually answered on the same roster.
Only `min` agrees everywhere; a member no roster kills is printed as a NAMED SURVIVOR.

⚠️ A THIRD RULE WAS READING THE FLAT `max_w`, AND IT IS SAFETY-30's

Found by reading the three branches side by side, NOT by the pin - and said that way
because it is the shape of what the pin cannot see. `unit_sign_disagree_w` was graded
against `_number(opts, "max_w", 2000.0)`, three lines under a branch that had already
stopped doing that. Both directions were measured before the fix:

  two 2000 W units + a leftover `max_w: 5000` + unit_sign_disagree_w 3000 -> 0 problems.
  The site started, the status page said `armed`, and the per-inverter watchdog could
  not trip on EITHER unit. Identical sentence to the blocker, one branch over.
  two 5000 W units with NO flat key -> REFUSED against a 2000 W default the operator
  never wrote, for a threshold both units can reach.

It takes the same `weakest` the branch above takes and names the same unit. At a roster
of one `unit_caps()` is `[max_w]`, so the shipped single-inverter refusal is the same
comparison, the same boundary and the same sentence.

MUTATION PROOF (each: bytecode purged and counted 0 before and 0 after, the edit
re-read FROM DISK with the new text present and the old absent, byte delta printed)

  B1  validate min(caps)     -> min(caps[:1])        +4 B  RED test_validate.py
  B2  validate min(caps)     -> min(caps[-1:])       +5 B  RED test_validate.py
  B3  validate caps_ix       -> caps_ix[:1]          +4 B  RED test_validate.py
  B4  validate rule 6 min    -> min(caps_ix[-1:])    +5 B  RED test_validate.py
  B5  validate rule 5 msg    -> min(caps_ix[:1])     +4 B  RED test_validate.py
  B6  validate rule 5b msg   -> min(caps_ix[-1:])    +5 B  RED test_validate.py
  B7  CONTROL: weakest -> max(0.0, weakest)         +10 B  GREEN, as intended
  R3a peakguard max(floors)  -> min(floors)           0 B  RED test_peakguard.py
  R3b peakguard max(floors)  -> max(floors[-2:])     +5 B  RED test_peakguard.py
  R3c main call site packs   -> self.packs[:2]       +4 B  RED test_peakguard.py
  R4a main pack_floor_soc    -> vals[:2]             +4 B  RED test_peakguard.py
  R4b main pack_floor_soc    -> vals[-2:]            +5 B  RED test_peakguard.py
  R4c main pack_floor_soc    -> vals[:3]             +4 B  RED test_peakguard.py
  R4d main cycle() any(...)  -> self.inverters[:1]   +4 B  RED test_watchdog.py

B1-B6 are caught by the BEHAVIOURAL set in test_validate.py, before the pin is reached.
B7 is a deliberate no-op control: `max(0.0, weakest)` is `weakest` for weakest > 0, and
it stays green, so the battery is not merely red for everything.

R3a is a ZERO-byte delta, so the shipped function was called both ways as well:
  packs floors [15.0, 30.0, 20.0], peak_soc_floor 15
  binding_soc_floor(...) = 30.0   the 0-byte mutant would answer 15.0   (+15.0 pp)
  -> the site would discharge to 15 % while pack-b's own BMS cuts out at 30 %.

⚠️ PRODUCTION IS DEGENERATE ON THE FLOOR AXIS. SPIKE-04: pack A and pack B BOTH floor
at 10 %, so a differing floor exists ONLY in the fixtures. That is the argument for
keeping them, not for simplifying them - collapsing them deletes the only data in this
repository that can see the reduction at all. Restated in config.yaml's example, in the
probe and in the fixtures.

⚠️ INVERTER B DOES NOT PHYSICALLY EXIST (~2026-09-12). Every second-unit figure here -
4600 W FROM ITS DATASHEET, 15 % illustrative, the whole roster SET - is SIMULATED and
labelled. Only inverter A's 10 % is read off hardware.

REBASED onto 67e649a. Two conflicts with SPIKE-04a, both in the blocks this card had
rewritten (config.yaml's roster example and the probe's C/D sections). Resolved to carry
BOTH claims, and the probe was RE-RUN rather than hand-merged: `all cases passed` on
ghcr.io/home-assistant/amd64-hassio-supervisor:2026.08.0, transcript updated with the
date. MUTATION.md habit 8 - a committed transcript stitched together out of two runs is
not a measurement.

Suite: 19/19 green in goodwe-addon-ci, mounted :ro, bytecode purged and verified 0
before and after. CI-06: test_peakguard.py run four times in isolation, green each time
(288 ok lines); test_validate.py three times, green each time.

MUTATION.md gains habits 12 and 13: build the instrument once the same shape has been
found twice AND make it discover its own scope, and say what the instrument cannot see
with sizes in the same breath.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
This commit is contained in:
2026-09-05 05:36:16 +02:00
co-authored by Claude Opus 5
parent 8beba0a3e7
commit 19decf433f
7 changed files with 1188 additions and 137 deletions
@@ -470,13 +470,227 @@ quietly or left unmentioned.**
### And the family closed rather than the member
`test_peakguard.py` part 12j enumerates **every** comprehension in `Controller`
that iterates the roster — 26 of them across 23 methods, computed from main.py's
AST, printed, and required to iterate the roster **whole**. A new reduction added
over `self.inverters[:1]` anywhere in the class is red **on the line it is
written**, at every roster length, with no fixture required. That check is what
would have caught rounds 2, 3 and 4's blockers on the round they were written
instead of one review round each; it is also what found rows 18-21.
`test_peakguard.py` part 12j enumerates the family from the AST and requires
every member to take the roster **whole**, so a new reduction over a slice is red
**on the line it is written**, at every roster length, with no fixture required.
That check is what would have caught rounds 2, 3 and 4's blockers on the round
they were written instead of one review round each; it is also what found rows
18-21.
⚠️ **AND IT WAS STILL TOO NARROW — see §4d.** Round 4's version enumerated the
26 roster comprehensions inside the `Controller` class, because that is where
the four known sites were. Round 5's blockers were four reductions in
`validate.py`, one module over, over a list *derived* from the roster three
calls away from it. The pin now parses every `.py` under `app/` and finds the
roster by transitive closure from one option key: **183 reductions across 9
modules.**
## 4d. ⚠️ ROUND 5 — the same family one MODULE over, and an instrument that discovers its own scope
Round 4 enumerated the family and closed it **inside the `Controller` class**,
because that is where the four known sites lived. The review found four more
survivors immediately, in `validate.py`, on a 19/19 green suite:
| site | mutation | delta | suite |
|---|---|---|---|
| `validate.py:779` | `min(caps)` → `min(caps[:1])` | +4 B | **GREEN** |
| `validate.py:779` | `min(caps)` → `min(caps[-1:])` | +5 B | **GREEN** |
| `validate.py:778` | `caps = [cap for _, cap in caps_ix[:1]]` | +4 B | **GREEN** |
| `validate.py:893` | `min(caps_ix[-1:], key=lambda p: p[1])` | +5 B | **GREEN** |
**None of them is an equivalent mutant — a safety refusal disappears.** With
`max_w: 5000` on inverter-a beside `2000` on inverter-b and
`sign_disagree_w: 3000`, the shipped validator answers one problem —
*"sign_disagree_w is 3000 W but max_w (inverter-b) is 2000 W, so the SAFETY-02
sign-disagreement watchdog can never fire"* — and every one of the four answers
**zero**. The site starts, with a watchdog reporting `armed` on the status page
that cannot trip on pack B. That is rule 5's own failure mode reached **through**
rule 5, which is the sentence `test_inverter.py:1657` says the rule exists to
prevent. Rule 6 loses its refusal the same way.
### Why round 4's pin could not see them, and why the fix is not four more checks
`caps` and `caps_ix` are not comprehensions over `self.inverters`; they are lists
**derived** from the roster, three function calls away from it, in a module the
enumeration never opened. Closing them by hand would have produced a fifth
rejection with the same shape one step over — that is the whole record of this
card. **A hand-written scope is the same defect as a hand-written fixture**
(MUTATION.md habit 12): a check whose data cannot reach the failure it names.
### The instrument
`test_peakguard.py` part 12j now parses **every `.py` under `app/`** off disk —
no module is named anywhere in the check — and finds the roster by transitive
closure from **one seed**, the `inverters` option key:
* a function whose returns are roster-carrying is **roster-producing**
(`roster_entries` → `build_roster` → `unit_caps_indexed` → `unit_caps`);
* a name assigned one carries it, **and so does a list accumulated in a loop
over one** — which is exactly what `caps`, `roster` and `problems` are, and
what round 4's rule could not express;
* an attribute or property answering one carries it (`self.inverters`,
`Controller.packs`, `Controller.units`, `live_units()`);
* **an argument at a call site taints that parameter in the callee**, which is
how the pin walks into `peakguard.binding_soc_floor`, `arbiter.resolve`,
`allocator.plan`, `circulation.circulating_w`, `maintenance.arbitrate` and
`store.migrate_maint_keys` without being told they exist;
* and **a slice of a roster is still a roster**, so a mutation cannot cut the
taint and hide every reduction downstream of itself. An **index** is one unit
and is not.
It then requires, at every place a roster-carrying value is reduced or iterated,
that the whole of it arrives: no `[:1]`, `[-1:]`, `[:2]` or `[0]` between the
roster and the reduction.
### What it found — and the hand-written list was wrong in BOTH directions
**183 reductions across 9 modules**, out of 23 `.py` files parsed:
`main.py` 96, `allocator.py` 24, `validate.py` 19, `arbiter.py` 14,
`circulation.py` 10, `maintenance.py` 10, `peakguard.py` 4, `inverter.py` 3,
`store.py` 3 — and 14 modules with no roster reduction at all.
The card's own suggested module list named `watchdog.py`, which reduces over no
roster at all (SAFETY-30's sign watchdogs are strictly per-unit — there is
nothing to slice), and did **not** name `maintenance.py` or `store.py`, which
both do. A pin built from that list would have made a vacuous claim about one
module and no claim at all about two others. That is the argument for computing
the scope, made with data rather than with an assertion.
The pin **independently rediscovers all four blockers** — it is told one option
key and nothing else — and finds a **fifth** beside them at
`validate.py:783`, the `min(caps_ix, key=...)` inside rule 5's message, which no
review round named.
### The counterfactual, at two depths
* **Per site, all of them.** Every enumerated site's own expression, in its own
scope, mutated with each of the four cuts this card has actually been defeated
by. **183 sites × 4 cuts = 732** mutated spellings; every one reported. The same run asserts
the predicate **accepts** the legitimate wrapped forms the package really uses
(`enumerate`, `zip`, `sorted`, `pack_permissions(packs, floor)`) and does not
treat a keyed lookup (`weights[u]` in `allocator._fill`) as a slice — which is
what four false SLICED reports cost before `shortens()` was written.
* **End to end, on the real files.** `[:1]` written into `app/validate.py` at
each blocker position, the **whole** analysis re-run over the mutated source,
and a sliced site required at that exact line — where the clean package
reports none. This is the check that would have taken PR #48 red four rounds
ago, with no fixture reaching it.
### The sweep, and the number that must not be quoted alone
All 183 sites were mutated to `[:1]`, one at a time, +4 bytes each, and each
mutant run against the full 19-file suite in `goodwe-addon-ci` (mounted `:ro`,
`python3 -B`, a fresh rsync per mutant; 0 `.pyc` in the tree before and after —
a read-only mount cannot write one, so TEST-15 is closed by construction here
rather than by discipline). **183 of 183 RED.**
⚠️ **On its own that is tautological**: with the pin in the tree every slice is
red *by the pin*. So all 51 mutants whose first red was `test_peakguard.py` were
re-run with the pin **neutralised** (`_SLICED` blanked), leaving only the
behavioural checks:
| | |
|---|---|
| **145 of 183** | a behavioural fixture in this repository separates the site from a slice of itself — red with the pin turned off |
| **38 of 183** | **nothing but this pin sees it.** Structurally closed by this card; behaviourally **UNGRADED**, and listed rather than counted as covered |
The 38: `main.py` 27, `allocator.py` 6, `validate.py` 2, `arbiter.py` 1,
`circulation.py` 1, `maintenance.py` 1. Grouped by what a `[:1]` there costs:
* **The operator's page — 15, the largest group.** `Controller.checks`
(3310, 3311, 3319, 3328, 3330, 3347, 3368, 3370, 3400, 3458) and
`status_payload` (3556, 3566, 3578). A slice reports one unit's row where two
exist, or "1 of 2 live" reading 1 of 1. Nothing commands differently; what is
lost is the evidence a human uses to notice that it should.
* **The write path's own counts.** `write_setpoint` 1267/1272/1280 — three
`len(targets)` guards. The write itself iterates `targets` at 1287 and *is*
graded (test_inverter.py); these three are its log arithmetic and refusal
count.
* **The allocator's internals.** `_fill`'s zero-weight last-resort pool (365),
`plan`'s `dropped` (526, 653), `free` (584), `frac_order` (591). ALLOC-01
drives two and three units through the *weighted* pass; the last-resort pass
and the largest-remainder tie-break are reached, but not with enough units for
a slice to change the answer.
* **Startup and FSM plumbing.** `__init__` 235/351, `_only` 831,
`_maint_button` 805, `cycle`'s maintenance zip 1505, `send_heartbeats` 3015,
`check_output_clamp` 2445/2452, `check_circulating` 2701,
`maintenance.arbitrate` 547, `arbiter.site_resolve` 517,
`circulation.opposing` 242.
* **Two refusal-TEXT sites.** `validate._roster_problems`'s `len(entries)` (554)
and `refuse()`'s numbering loop (1263) — both shape the words of a refusal
that still happens.
* **`_grade_store_migration` (698)**, already row 21 of §4c, argued there to be
an equivalent mutant at that call site. The argument is unchanged and is still
an argument, not a proof.
⚠️ **None of the 38 is closed behaviourally by this card, and saying so is the
point.** Closing sites one at a time is what cost this card four review rounds.
The pin closes the **family** — a slice at any of them is red on the line it is
written — and this list is what a shape pin can never promise.
### And the behavioural half, because a shape pin cannot grade a meaning
The pin grades an **identity** ("the whole roster arrives"). `min` → `max` is a
zero-byte edit and invisible to it, forever. So `test_validate.py` grows a
**roster SET** for the three rules that reduce over `unit_caps_indexed()`:
rosters of width 2, 3, 4 and 5 with the one small cap at **every position** —
14 rosters × 3 rules = 42 runs of the real validator — plus a **computed family
table** of twelve candidate reductions (`max`, `first`, `last`, the mean, and
`[:1]`, `[-1:]`, `[:2]`, `[:3]`, `[1:]`, `[:-1]`, `[::2]`) each compared against
what the shipped validator actually said on the same roster. Only `min` agrees
everywhere; any member no roster kills is **printed as a named survivor** and
takes the check red.
### ⚠️ A THIRD RULE WAS READING THE FLAT `max_w`, AND IT IS SAFETY-30's
Found by reading the three branches side by side — **not** by the pin, and said
that way because it is the shape of what the pin cannot see. Rule 5's sibling
branch, `unit_sign_disagree_w` (SAFETY-30's per-inverter sign watchdog), was
graded against `_number(opts, "max_w", 2000.0)` — the **flat** option — three
lines under a branch that had already stopped doing that. Both directions were
measured before the fix:
* a roster of two 2000 W units beside a leftover `max_w: 5000` from a
single-inverter install, with `unit_sign_disagree_w: 3000`, produced **zero
problems**. The site started, the status page said `armed`, and the per-unit
watchdog could not trip on **either** unit. Identical sentence to the blocker,
reached through the branch beside it;
* and a roster of two 5000 W units with no flat key was **refused** at 3000 W
against a 2000 W *default the operator never wrote*, for a threshold both
units can reach.
It takes the same `weakest` the branch above takes, from the same list, and
names the same unit. At a roster of one `unit_caps()` is `[max_w]`, so the
shipped single-inverter refusal is the same comparison, the same boundary and
the same sentence. Both directions are asserted in `test_validate.py`.
### ⚠️ WHAT THE PIN CANNOT SEE — open rows, with sizes
1. **Semantics.** `min` vs `max` vs the mean: zero-byte edits, invisible to any
shape pin. Only the behavioural sets above can see them, and they only see
the rules they drive.
2. **Direct index sites that are not reductions.** `self.inverters[0]` in
`_only()` and `actuator_of()`, `self.inverters[self.maint_index]` in
`maint_unit`, `entries[index]` in `_unit_label()`. Single-unit lookups, meant
to be, graded where they live.
3. **A subscript by a NAME.** `weights[u]`, `cap[u]`, `add[u]` in
`allocator._fill` are keyed lookups into a roster-derived mapping and are
accepted deliberately, so `caps[i]` with a variable `i` would pass. Not in
the family this card has been defeated by; buying it costs four false reports.
4. **A slice through an intermediate call.** `sorted(roster)[:1]` subscripts a
call result, not a roster reference. Nothing in `app/` is written that way
today; if something is, it would not be seen.
5. **Anything outside `app/`.** `tools/`, `sim/` and the checks themselves are
not parsed — and `sim/inject.py` reaches `validate_startup_config()`.
6. **The taint's own seam.** It crosses a call boundary only where the callee is
a plain function this package defines and the argument is positional or
keyword. A roster handed through `*args`, stored in a dict and read back, or
passed to a method on an unresolvable object, is lost at that hop — and
everything downstream goes **unenumerated** rather than reported unreachable.
That is the shape of the remaining exposure: not zero, and not measurable
from inside the check.
7. **`cycle()`'s undeliverable-unit log line** (row 22 of §4c) is still green at
+4 bytes, unchanged from round 4 and still filed open.
## 5. ⚠️ THE RESIDUAL — a per-pack EMS floor is NOT this card
@@ -546,7 +760,7 @@ A's measured 10 % while the 35 % cut-out that moves along the roster is invented
---
## 7. The probe transcript, re-run 2026-09-04 on 2026.08.0
## 7. The probe transcript, re-run 2026-09-05 on 2026.08.0
Command (unchanged, in `tools/supervisor_shape_probe.py`'s own docstring):
@@ -561,6 +775,14 @@ It reads `config.yaml` and runs Supervisor's own `AppOptions` validator in a
throwaway container. It touches no live Supervisor, POSTs nothing and installs
nothing.
⚠️ **RE-RUN AGAIN ON 2026-09-05**, because the rebase onto `67e649a` merged
SPIKE-04a's `max_w: 4600` into the same §C/§D blocks this card had rewritten,
and MUTATION.md habit 8 says a committed transcript that a document quotes is a
check. A hand-merged transcript is not a measurement of anything. The block
below is the output of that run, not of the 2026-09-04 one: B's rating reads
**4600** — its datasheet figure, still not a measurement of any unit — and every
case still passes.
```
== C. AC2: a two-inverter blob an operator would POST ==
PASS two inverters
@@ -568,7 +790,7 @@ nothing.
== D. what the schema itself refuses, per entry ==
PASS the per-entry BASELINE is ACCEPTED, so every refusal below has nowhere else to come from
[{"name": "inverter-b", "setpoint_entity": "number.b", "max_w": 3000, "bms_soc_floor": 10}]
[{"name": "inverter-b", "setpoint_entity": "number.b", "max_w": 4600, "bms_soc_floor": 10}]
PASS an entry with no name is refused, and the refusal NAMES it
Missing option 'name' in inverters in GoodWe RS485 Controller (local_goodwe_controller)
PASS an entry with no setpoint_entity is refused, and the refusal NAMES it