Files
goodwe-addon/goodwe_controller/MUTATION.md
T
adminandClaude Opus 5 19decf433f
checks / unit (pull_request) Successful in 4m59s
SAFETY-32 round 5: stop closing sites, build an instrument that finds its own scope
Rounds 2, 3 and 4 each closed exactly the survivor they were handed and each time
the reviewer found the same shape one step over: the reduction, the call site, the
sibling argument, and now the sibling MODULE. Round 4's enumeration stopped at the
`Controller` class boundary; round 5's four blockers were reductions in validate.py,
over a list DERIVED from the roster three calls away from it. Closing four more sites
by hand would have produced a fifth rejection, so this round widens the instrument
instead and lets it enumerate.

⚠️ THE BLOCKERS ARE NOT EQUIVALENT MUTANTS - a safety refusal disappears, verified by
calling the shipped validator both ways on the real tree:

  roster max_w 5000 (inverter-a) beside 2000 (inverter-b), sign_disagree_w 3000
    clean                        -> 1 problem, "...max_w (inverter-b) is 2000 W, so the
                                    SAFETY-02 sign-disagreement watchdog can never fire"
    min(caps[:1])          +4 B  -> 0 problems. The site STARTS.
    caps_ix[:1]            +4 B  -> 0 problems.
  the SAME roster with the small cap FIRST (2000, 5000)
    clean                        -> 1 problem, naming inverter-a
    min(caps[-1:])         +5 B  -> 0 problems.

⚠️ AND THE ONE-ROSTER FIXTURE COULD NOT HAVE SEEN THAT. On [5000, 2000] the `[-1:]`
mutant still refuses, because last IS min there. A fixture of N members cannot separate
two reductions that agree on N members - which is why the behavioural half below is a
SET and not a roster.

THE INSTRUMENT (test_peakguard.py part 12j, widened past `Controller`)

Every `.py` under app/ is read off disk and parsed - NO MODULE IS NAMED IN THE CHECK -
and the roster is found by transitive closure from ONE seed, the `inverters` option key:
through returns, through lists ACCUMULATED in a loop over a roster (which is exactly
what `caps`, `roster` and `problems` are, and what round 4's rule could not express),
through attributes and properties, and THROUGH CALL ARGUMENTS, which is how it walks
into peakguard, arbiter, allocator, circulation, maintenance and store without being
told they exist. A slice of a roster stays a roster so a mutation cannot cut the taint;
an index is one unit and does not.

  183 roster reductions across 9 modules, out of 23 files parsed:
  main 96, allocator 24, validate 19, arbiter 14, circulation 10, maintenance 10,
  peakguard 4, inverter 3, store 3.  14 modules reduce over no roster at all.

⚠️ THE HAND-WRITTEN MODULE LIST WOULD HAVE BEEN WRONG IN BOTH DIRECTIONS. It named
watchdog.py, which reduces over no roster at all (SAFETY-30's sign watchdogs are
strictly per-unit), and omitted maintenance.py and store.py, which both do. A vacuous
claim about one module and no claim at all about two others - which is the argument for
computing the scope, made with data.

The pin rediscovers ALL FOUR blockers without being told where to look, and finds a
FIFTH beside them (validate.py:783, rule 5's own message) that no review round named.
Its counterfactual runs at two depths: every enumerated site's expression mutated with
each of the four cuts this card has been defeated by (183 x 4 = 732 spellings, all
reported), and `[:1]` written into app/validate.py at each blocker position with the
WHOLE analysis re-run over the mutated source.

THE SWEEP, AND THE NUMBER THAT MUST NOT BE QUOTED ALONE

All 183 sites mutated to `[:1]`, +4 B each, against the full 19-file suite in
goodwe-addon-ci (:ro, -B, fresh rsync per mutant, 0 .pyc before and after - a read-only
mount cannot write one): 183/183 RED. ⚠️ That is TAUTOLOGICAL on its own, because the
pin makes every slice red. So the 51 whose first red was test_peakguard.py were re-run
with the pin NEUTRALISED:

  145 of 183   a behavioural fixture separates the site from a slice of itself
   38 of 183   ONLY the structural claim. Behaviourally UNGRADED.

All 38 are listed by site with what a `[:1]` there would cost, in the DECISIONS record
and in part 12j itself: 15 are rows and counts on the operator's page, 3 are
`len(targets)` arithmetic beside a write path that IS graded, 6 are allocator internals
ALLOC-01 reaches but not with enough units to separate. NONE of the 38 is closed
behaviourally by this card and none is counted as covered.

THE BEHAVIOURAL HALF, because a shape pin grades an identity and never a meaning

test_validate.py grows a ROSTER SET for the three rules that reduce over
`unit_caps_indexed()`: widths 2, 3, 4, 5 with the one small cap at EVERY position,
14 rosters x 3 rules = 42 runs of the real validator, each asserting the refusal fires
AND names the unit the number came off. Beside it a COMPUTED family table of twelve
candidates - max, first, last, the mean, [:1], [-1:], [:2], [:3], [1:], [:-1], [::2] -
each compared against what the shipped validator actually answered on the same roster.
Only `min` agrees everywhere; a member no roster kills is printed as a NAMED SURVIVOR.

⚠️ A THIRD RULE WAS READING THE FLAT `max_w`, AND IT IS SAFETY-30's

Found by reading the three branches side by side, NOT by the pin - and said that way
because it is the shape of what the pin cannot see. `unit_sign_disagree_w` was graded
against `_number(opts, "max_w", 2000.0)`, three lines under a branch that had already
stopped doing that. Both directions were measured before the fix:

  two 2000 W units + a leftover `max_w: 5000` + unit_sign_disagree_w 3000 -> 0 problems.
  The site started, the status page said `armed`, and the per-inverter watchdog could
  not trip on EITHER unit. Identical sentence to the blocker, one branch over.
  two 5000 W units with NO flat key -> REFUSED against a 2000 W default the operator
  never wrote, for a threshold both units can reach.

It takes the same `weakest` the branch above takes and names the same unit. At a roster
of one `unit_caps()` is `[max_w]`, so the shipped single-inverter refusal is the same
comparison, the same boundary and the same sentence.

MUTATION PROOF (each: bytecode purged and counted 0 before and 0 after, the edit
re-read FROM DISK with the new text present and the old absent, byte delta printed)

  B1  validate min(caps)     -> min(caps[:1])        +4 B  RED test_validate.py
  B2  validate min(caps)     -> min(caps[-1:])       +5 B  RED test_validate.py
  B3  validate caps_ix       -> caps_ix[:1]          +4 B  RED test_validate.py
  B4  validate rule 6 min    -> min(caps_ix[-1:])    +5 B  RED test_validate.py
  B5  validate rule 5 msg    -> min(caps_ix[:1])     +4 B  RED test_validate.py
  B6  validate rule 5b msg   -> min(caps_ix[-1:])    +5 B  RED test_validate.py
  B7  CONTROL: weakest -> max(0.0, weakest)         +10 B  GREEN, as intended
  R3a peakguard max(floors)  -> min(floors)           0 B  RED test_peakguard.py
  R3b peakguard max(floors)  -> max(floors[-2:])     +5 B  RED test_peakguard.py
  R3c main call site packs   -> self.packs[:2]       +4 B  RED test_peakguard.py
  R4a main pack_floor_soc    -> vals[:2]             +4 B  RED test_peakguard.py
  R4b main pack_floor_soc    -> vals[-2:]            +5 B  RED test_peakguard.py
  R4c main pack_floor_soc    -> vals[:3]             +4 B  RED test_peakguard.py
  R4d main cycle() any(...)  -> self.inverters[:1]   +4 B  RED test_watchdog.py

B1-B6 are caught by the BEHAVIOURAL set in test_validate.py, before the pin is reached.
B7 is a deliberate no-op control: `max(0.0, weakest)` is `weakest` for weakest > 0, and
it stays green, so the battery is not merely red for everything.

R3a is a ZERO-byte delta, so the shipped function was called both ways as well:
  packs floors [15.0, 30.0, 20.0], peak_soc_floor 15
  binding_soc_floor(...) = 30.0   the 0-byte mutant would answer 15.0   (+15.0 pp)
  -> the site would discharge to 15 % while pack-b's own BMS cuts out at 30 %.

⚠️ PRODUCTION IS DEGENERATE ON THE FLOOR AXIS. SPIKE-04: pack A and pack B BOTH floor
at 10 %, so a differing floor exists ONLY in the fixtures. That is the argument for
keeping them, not for simplifying them - collapsing them deletes the only data in this
repository that can see the reduction at all. Restated in config.yaml's example, in the
probe and in the fixtures.

⚠️ INVERTER B DOES NOT PHYSICALLY EXIST (~2026-09-12). Every second-unit figure here -
4600 W FROM ITS DATASHEET, 15 % illustrative, the whole roster SET - is SIMULATED and
labelled. Only inverter A's 10 % is read off hardware.

REBASED onto 67e649a. Two conflicts with SPIKE-04a, both in the blocks this card had
rewritten (config.yaml's roster example and the probe's C/D sections). Resolved to carry
BOTH claims, and the probe was RE-RUN rather than hand-merged: `all cases passed` on
ghcr.io/home-assistant/amd64-hassio-supervisor:2026.08.0, transcript updated with the
date. MUTATION.md habit 8 - a committed transcript stitched together out of two runs is
not a measurement.

Suite: 19/19 green in goodwe-addon-ci, mounted :ro, bytecode purged and verified 0
before and after. CI-06: test_peakguard.py run four times in isolation, green each time
(288 ok lines); test_validate.py three times, green each time.

MUTATION.md gains habits 12 and 13: build the instrument once the same shape has been
found twice AND make it discover its own scope, and say what the instrument cannot see
with sizes in the same breath.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-05 05:36:16 +02:00

430 lines
30 KiB
Markdown

# Mutation runs in this repository — and the one way they lie
Nearly every safety claim in this add-on was established the same way: change
the shipped module, run the suites, watch one go **red**, put the module back,
watch it go **green**. There is no coverage tool here and there is not meant to
be — the check that matters is whether a *check* notices when the behaviour it
names is removed.
That method has exactly one silent failure mode, and TEST-15 exists because we
walked into it.
## The defect
FCST-03's developer reverted `self.provisional` to `here.provisional`, re-ran,
and reported the mutation as a **survivor**. It was not. CPython decides a
cached `.pyc` is still valid from two numbers:
* the source file's **size in bytes**, and
* the source file's **mtime truncated to whole seconds**.
`self` and `here` are the same length, and the revert happened in the same
second as the edit. Neither number moved, so the interpreter loaded the
bytecode it already had. Confirmed at the time by disassembly: **the source read
`here`, the running code loaded `self`.** The suite had graded code that was not
there — no error, no warning, exit status 0.
The same collision, in the other direction, can leave a **red** standing on
correct source after a revert that never took.
### Which way it lies, and why that scopes everything
A mutation that never took effect leaves the suite **green**, and a green run of
a mutated tree is reported as a survivor. So stale bytecode **manufactures
survivors**; it cannot manufacture a red. The false-red case exists but
announces itself, because a developer chases a failure they cannot explain.
Two consequences, and both are used below:
* a recorded **"all red, no survivors"** result is not at risk from this defect;
* a recorded **survivor** — and a recorded *zero*, such as a coverage-audit
count of 0 — is exactly what this defect produces.
`test_control.py`'s coverage-audit table says the same thing in its own terms:
the error is one-directional, so every figure there is a lower bound at worst.
## The rule
**Purge `__pycache__` before every mutation run.** Not afterwards, not usually,
not when the edit looks big enough.
```sh
cd goodwe_controller
tools/mutation_run.sh # every test_*.py in this directory
tools/mutation_run.sh . test_tariff.py # or just the ones you are watching
tools/mutation_run.sh "$GOODWE_ADDON_REPO/goodwe_controller" # from elsewhere
```
The harness purges, prints a `sha256`/byte-count fingerprint of every `app/*.py`
it is about to grade, runs each suite under `python3 -B`, and fails if a cache
appeared during the run. Diff two fingerprints and the file you mutated is the
line that changed — which is also how a mutation report proves *which source*
produced it.
### What does **not** fix this
| | writes bytecode | reads a stale `.pyc` |
|---|---|---|
| `python3 -B` | no | **yes** |
| `PYTHONDONTWRITEBYTECODE=1` | no | **yes** |
| purging `__pycache__` | (unchanged) | **no — nothing to read** |
Both flags are *write-side only*. A mutation run always has a cache already,
because the previous run wrote it, so a harness that only adds `-B` closes
nothing. This is measured, not read off documentation: `test_bytecode.py`
part 2 arranges the collision and shows both flags still running the old code.
`-B` is still worth passing — it stops **this** run leaving a cache for the
**next** edit to inherit, which is the one nobody is thinking about three
mutations into a batch. Purge for this run, `-B` for the next one.
### And not a `sleep`
One-second mtime resolution is the *mechanism*, not the enemy. Waiting out the
clock would slow every run down, and would still only cover the edit somebody
remembered to slow down. Stop trusting mtime instead.
## Reporting a mutation
State the things a reviewer cannot reconstruct afterwards:
1. **the byte delta** of the mutated file (`+5 B`, `+0 B`). `+0 B` is the risky
one — say so;
2. **that the mutated line was on disk** when the run started (the harness
fingerprint, or a `grep` of the line);
3. **which suite went red, and on which named check** — "the suite is red" and
"the check that watches this is red" are different claims;
4. for a **survivor**, that the run was purged. An unpurged survivor is not a
result.
## The other thing a mutation sweep cannot see: masking (TEST-16)
Stale bytecode is not the only way a survivor lies. **Two guards can mask each
other**, and a sweep that mutates one line at a time is structurally blind to it.
TEST-16 is the worked example. `static_tariff.arbitrage()` refuses a
one-firm-hour series at `len(firm) < 2`, and refuses a flat series at
`spread == 0.0`. A single firm hour *also* has a spread of zero. So:
* weaken the firm-pair rung alone → the flat rung refuses the stimulus anyway;
* weaken the flat rung alone → the tiny-spread stimulus is still refused, just
for the wrong reason;
* and the suite, grading `kind == "refused"`, reports **nothing either way**.
Both rungs could then be weakened *together* and still nothing would report —
which is precisely the state in which the next edit to either one opens a real
hole. It survived four rounds of review on TARIFF-03 and FCST-03.
Two rules come out of it, and both are cheap:
1. **Grade the reason, not just the verdict.** "Refused" and "refused for the
right reason" are different claims, and a masked rung is the case where the
first is true and the second is false. A refusal sentence is what the
operator reads; it is also the only observable that moves when one rung
quietly absorbs another's work.
2. **When two rungs can answer the same stimulus, mutate them together.** One
extra run per suspected pair. If a sweep finds two survivors in the same
function, that pair is the first thing to try — TEST-16's two survivors were
adjacent lines in one function and neither was individually live.
## The third way a check lies: its DATA cannot reach the failure it names
Stale bytecode manufactures survivors from a mutation that never ran. Masking
manufactures them from a second guard doing the first one's work. This one
manufactures them from a **fixture**: the check runs, the assertion is
evaluated, the sentence above it is true — and the numbers it compares are
arranged so that the failure it describes could not have changed the answer.
It is not hypothetical and it is not rare. EIGHT instances in two weeks; four
were found by review, two - CFG-04's fixture pair - by an author's own mutation
battery, which is the cheaper place, one by review again, in the one
place nobody had thought to point the rule at: not a check in the suite, but a
COMMITTED EVIDENCE ARTIFACT four documents cite - and the eighth INSIDE THE FIX
WRITTEN TO CLOSE THE SEVENTH.
⚠️ **AND IT HAS NOW APPEARED IN SIX CONSECUTIVE CARDS.** The seventh instance,
MAINT-05's, is the one that says why: the defect is not carelessness about
fixtures, it is that **a check and the thing it checks were built from the same
constant**. Ask of every difference-claim not "is this assertion true" but
"where did each side of this comparison come from" - and if the answer is the
same place twice, the check is decoration.
* **ARB-03** shipped a call-site scan asserting that every `claims.append` in
`cycle()` routes through `actuator_of()`. Its own parse **saw 3 of the 8
appends**; the other five were invisible to it and would have passed however
they were routed.
* **ALLOC-01** shipped a replay generator to hunt for envelope escapes. It found
**0**. A widened generator over the same allocator found **39,991**.
* **DEV-02** shipped `"each unit's supervised bound is ITS OWN max_w — give
inverter B a different rating and the claim filed on inverter B moves with
it"`, comparing inverter B's bound against `_b2.max_w`. The fixture defaulted
B's rating to A's, so it compared `(-5000, 5000)` against `(-5000, 5000)` —
**a number against itself**. Filing both per-unit bounds from
`self.inverters[0]` left **all 19 files green**.
* **CFG-04** wrote *"an entry that omits `soc_entity` has NO SoC sensor and does
not borrow the flat one"* against a fixture whose **flat `soc_entity` was
blank**. Inherited and not-inherited were both `""`, so the mutation that
makes `build_roster()` fall back to the flat key survived. The fixture now
sets a real entity there, and the check asserts that the flat value is present
as well as that the entry's is empty.
* **CFG-04** also wrote *"main.py runs the per-unit range check
UNCONDITIONALLY"* as a **substring test over `app/main.py`'s source**. The
mutation that defeats it puts the call back inside `if entity:` — one level of
indentation — and the substring is still there. That is habit 4 below in a new
costume: the check asserted against the TEXT of the thing under test rather
than against its behaviour. It now drives the shipped `run_control()` on a
two-inverter config, **under a deadline**, because the mutation's real symptom
is that the controller enters its loop and never returns.
* **MAINT-05** shipped `Controller.__init__` calling
`migrate_maint_keys(store, self.inverters[0].name, ...)` under a comment
saying *"it targets `self.inverters[0].name` and NOT `UNIT_A` ... writing
`UNIT_A` here would deposit an install's whole maintenance history under a
name that is not on the roster at all"*. `inverter.UNIT_A` **is** the string
`"inverter-a"`, and every two-pack fixture in `test_maintenance.py` named its
first roster entry `inverter-a` — so the two expressions were the same value
in every check that existed, and swapping them left **all 19 files green**
while losing a real install's whole maintenance history. Two-pack fixtures are
now `["garage", "shed"]`, each behind a precondition check (habit 1) that the
first name is not `UNIT_A`.
⚠️ **AND THE POSTCONDITION HAD THE SAME SHAPE AS THE FIXTURE.**
`store._verify_moves()` read back the keys it had just written, under the
namespace it had just been handed — so it verified clean, and the caller
stamped the schema marker, on a migration nothing would ever read. *A check
that reads back the namespace it was handed cannot detect being handed the
wrong one*: it compared a thing against itself, exactly as DEV-02's fixture
compared `(-5000, 5000)` against `(-5000, 5000)`. The habit-2 fix is the same
one — grade against something the thing under test did not supply, here the
ROSTER the machines read.
⚠️ **AND THE FIX ITSELF CARRIED THE DEFECT, WHICH IS THE SIXTH CARD IN A ROW
AND THE FIRST TIME IT HAS APPEARED *INSIDE* A REMEDY.** The fix round's
recovery check was `migrate_maint_keys(_m5, ROSTER[0], ROSTER)` — the
**corrected** name. It measures recovery along the one path where the operator
has *already fixed the bug*, and nothing in the 19 files called
`migrate_maint_keys()` twice with the **same wrong** `unit`. So the refusal
the card was rejected twice to obtain lasted **exactly one start**: measured
at Controller level over one persisted store, roster `["garage", "shed"]` —
construction 1 `ok=False misdirected=4 moved=4 kept=0 marker=<absent>`,
construction 2 `ok=True misdirected=0 moved=0 kept=4 marker=1`. On start 2
rule 1 routes the wrongly-written keys to `rep.kept`, `rep.moved` is empty,
the roster loop iterates over nothing, and the marker is stamped on a
migration that reached no readable key — after which the early return skips it
forever, *including after the call site is fixed*. The check was true and
passed for the wrong reason. Habit **9** below is what falls out of it.
* **CFG-04's `tools/supervisor_shape_probe.py` §B** — and this one is the rule
reaching outside the suite. The probe is the committed evidence that
`config.yaml`'s `inverters: []` default is load-bearing, and four documents
quoted it as answering *"Missing option 'inverters' in root"*. §B validated
the reconstructed LIVE options blob, which is missing many required keys
unrelated to this card, so its refusal named whichever of THOSE the validator
reached first — `meter_freeze_s`, `p1_phase_export_entities` and
`sign_disagree_w` on three different runs. The quoted string was never
produced by the instrument. And the assertion was a bare **`not ok`**: delete
`inverters: []` from `config.yaml` and §B went on printing **PASS**, blind to
the exact regression it was cited for. §B is now an isolated counterfactual —
the shipped defaults, and the same defaults minus that one key — asserting on
the **key name in the refusal**; the mis-cited blob survives, relabelled, as
§B2. Re-run with the default deleted it prints `2 CASE(S) FAILED` and exits 1.
⚠️ **A committed transcript is a check, and it decays like one.** Nothing in
this document had been pointed at `tools/`, because the artifacts there are
filed as *measurements* rather than as *checks* — and a measurement whose
assertion cannot fail is exactly the thing this section is about. Two habits
follow, and both are habit 5 and 6 below.
Note what these have in common with a masked rung: the mutation *runs*, and the
suite is *honestly* green. Nothing in a purge, a fingerprint or a disassembly
can see it, because the code under test really did behave identically on the
stimulus it was given. The defect is upstream of the assertion, in what the
check was handed.
**The rule, and it is one line:**
> **Any check asserting "X differs from Y" must be run once with X and Y
> actually made different.**
Cheap habits fall out of it, and all of them are now used in this repository:
1. **Write the PRECONDITION as its own check.** `test_peakguard.py` 10g asserts
that A and B have different ratings *before* it asserts that B's bound is
B's. If a later edit makes them equal again, the precondition goes red rather
than the claim going quietly vacuous.
2. **Assert against literals, not against the expression under test.**
`== [(-_b2.max_w, _b2.max_w)]` cannot distinguish "B's number" from "the
number this fixture happens to have twice"; `== (-3000.0, 3000.0)` beside a
printed `A = 5000` can.
3. **Print what the check saw.** Every one of the three above would have been
caught by a human reading one line of output — `supervised bounds by
actuator: {'inverter-a': (-5000, 5000), 'inverter-b': (-5000, 5000)}` is
self-evidently not evidence of a difference.
4. **Assert on the BEHAVIOUR, not on the source text of the thing under test.**
A substring test over a module is a check on how the code is spelled;
whitespace, an alias or a moved line defeats it while the property it names
is gone. `test_inverter.py` part 10 has one of each — a source pin for a
claim that genuinely IS about the code's shape, and a driven `run_control()`
for a claim about behaviour — and the second replaced the first after the
mutation battery walked straight through it.
5. **A check whose failure mode is a HANG needs a deadline.** Otherwise the
mutation that defeats it produces a CI timeout with no name attached, which
reads like flaky infrastructure and gets retried rather than read.
6. **Assert on the CONTENT of a refusal, not on the fact that one happened.**
`not ok` is satisfied by any error at all, including one about a different
key on a different blob. `"Missing option 'inverters' in root" in out` goes
red the moment the refusal stops being about the thing it is cited for. This
is habit 2 in the negative direction: a refusal, like a difference, has to be
asserted against a literal.
7. **Vary ONE thing in a counterfactual, and vary it from an ACCEPTED
baseline.** `supervisor_shape_probe.py` §B validates the shipped defaults
first, precisely so the refusal on the next line has nowhere else to come
from. Without the accepted baseline the second line is a refusal of unknown
provenance, which is what §B used to print.
8. **A committed transcript is a check.** Anything under `tools/` that a
document quotes is subject to every rule above, and to the mutation battery:
break the property, re-run the instrument, and confirm the transcript goes
red. `inverters: []` was deleted from `config.yaml` and the probe re-run
against `ghcr.io/home-assistant/amd64-hassio-supervisor:2026.08.0` for
exactly that reason.
9. **Exercise a refusal a SECOND time with the SAME wrong input — not the
corrected one.** A guard that refuses once and then admits the same bad
caller is not a guard, it is a delay, and the second call is a *different
code path* whenever the first call left state behind. MAINT-05's roster
grading iterated the keys it had just **written**; on the second start the
same wrong caller found them already present, they moved to the
`already namespaced` list, and the loop graded an empty set. Re-running with
the **fixed** argument — which is what the check did — measures the operator's
recovery, never the persistence of the refusal. Ask of every refusal: *what
does this do when the caller has not been fixed and simply runs again?* And
assert the **shape** of that second report, not only its verdict, so the
check cannot silently revert to grading the first path.
10. **A fixture of N members cannot separate two reductions that agree on N
members — so grade the FAMILY, and grade it from the SOURCE.** This is the
only lesson SAFETY-32 cost four review rounds to learn, and it generalises
past packs to every roster, list and window in this repository. Two packs
cannot tell `max` from `min` from first from last. Three cannot tell `max`
from `max(x[-2:])`. **The regress does not terminate in a fixture**: `[:1]`
needs two members, `[:2]` three, `[:3]` four, and adding one more member
moves the hole rather than closing it. Three things follow, and all three
are now in `test_peakguard.py` parts 12c2, 12g, 12i and 12j:
* the fixture is a **SET** — the extreme at every position of the roster,
not one roster with the extreme somewhere;
* the family table is **COMPUTED and PRINTED**, so a member no fixture kills
shows up as a named survivor and takes the check red, instead of being
absent from a list nobody re-derived;
* where the claim is an **IDENTITY** ("this roster", "this list"), pin it
**exactly on the AST** — `ast.unparse(arg) == "self.packs"` is false of
every slice, where `"self.packs" in src` is true of all of them. Where the
claim is **SEMANTIC** ("the maximum"), a shape pin would refuse honest
refactors, so report the unreached width **with its size** instead.
11. **When a check of this shape is found, mutate its SIBLINGS before shipping
the fix.** SAFETY-32 was rejected three times for the same defect one step
over: the reduction, then the roster handed to it, then the reduction handed
to the *other argument of the same call*. Enumerating the family from the
AST and mutating each member took one afternoon and found **four more
survivors in code the card never touched** — three of SAFETY-30's
`any(i.sign_wd.tripped for i in self.inverters)` reductions, invisible
because the only fixture that reached them was a roster of ONE, where
`self.inverters[:1]` **is** `self.inverters`. Ask of every fix: *what is the
sibling of the thing I just hardened, and does any fixture separate it from
a slice of itself?*
12. **When the same shape has been found twice, STOP CLOSING SITES AND BUILD
THE INSTRUMENT — and make it DISCOVER its own scope.** SAFETY-32 was
rejected four times, and each round closed exactly the site it was handed:
the reduction, then the roster handed to it, then the *sibling argument* of
the same call, then the same family one *module* over, in validate.py.
Round 4 did enumerate — and enumerated inside the `Controller` class,
because that is where the four known sites lived. **A hand-written scope is
the same defect as a hand-written fixture**: it is a check whose data cannot
reach the failure it names, and it goes on passing while the thing it was
supposed to cover sits in the file nobody added to the list. Round 5's pin
parses every `.py` under `app/`, seeds itself from ONE option key, and
propagates through returns, accumulator loops, attributes and *call
arguments* to find the roster wherever it went — 183 reductions across 9
modules. It rediscovered all four blockers without being told where to look,
and it named two modules the hand-written list would not have contained
(`maintenance.py`, `store.py`) while *omitting* one the list did contain
(`watchdog.py`, which is strictly per-unit and reduces over no roster at
all). **The list was wrong in both directions**, which is exactly what a
computed scope is for.
13. **Say what the instrument CANNOT see, with sizes, in the same breath.** A
structural pin grades an **identity** ("this whole roster arrives"); it can
never grade a **meaning** (`min` vs `max` is a zero-byte edit and invisible
to it). Ship the pin AND the behavioural set, name the gap, and give every
unreached case its byte delta — `cycle()`'s undeliverable-unit log line
(+4 B, still green), a subscript by a *variable* rather than a constant, a
slice taken through an intermediate call, anything outside `app/`. **An
enumerated list with an honest gap is worth more than a claim of
completeness a reviewer falsifies in ten minutes**, and every round of this
card that claimed the latter was rejected for it.
And the mutation-table consequence: **a variant is not covered because a
near-miss of it is.** DEV-02's table listed the *actuator* variant of the
per-unit claim defect (`device`/`supervised` filed without `inv`), which the
static routing pin catches. The *bound* variant — right actuator, wrong
numbers — is a different defect with a different fix, and it was absent. Both
are in the table now.
## Where each surface stands
* **Local and dev-harness runs — the exposed one.** This is where FCST-03 hit
it. `goodwe_controller/__pycache__` and `app/__pycache__` are gitignored and
are sitting in most working trees right now. Use the harness.
* **CI — structurally immune, and it was before this card.** `.gitea/workflows/
checks.yml` bind mounts the checkout `-v "$WS:/w:ro"`, so the interpreter
cannot write a `__pycache__` into the tree under test at all; measured for
TEST-15, and it fails *silently* — no error, exit status 0. Each job also
`rm -rf`s its workspace and clones afresh, so nothing crosses between jobs on
the shared runner. `PYTHONDONTWRITEBYTECODE=1` is now set there too, as belt
and braces: the `:ro` is one character wide and the next person debugging a
write will be tempted by it.
* **The shipped add-on — not affected, and this is closed.**
`goodwe_controller/Dockerfile` has carried `PYTHONDONTWRITEBYTECODE=1` since
it was written. There is no deploy-side finding here; do not re-open it.
* **The infra repo's `sim/` harnesses — exposed, and NOT fixed by this card.**
`sim/replay_sign.py` and friends import the add-on out of
`$GOODWE_ADDON_REPO` and write `__pycache__` into *both* trees
(`goodwe/sim/__pycache__` exists today). SAFETY-02's headline validation is a
replay, not a suite, so a mutation graded through it carries the same
exposure. `tools/mutation_run.sh` takes the tree to purge as its first
argument precisely so that repo can call it, but the change belongs in that
repo and some of those harnesses want the rig. **Anyone mutating a module
graded by a replay must purge both trees by hand until then.**
## Survey — mutation claims of 2026-08-25 … 2026-09-01
Scoped **by the property, not by re-running everything**. A claim is at risk
only if a mutation could have preserved the file's byte count *and* been
reverted inside the same second, *and* the claim is of the kind stale bytecode
can fabricate (see "which way it lies"). Sources: `git log release/1.0` and the
commit and PR bodies.
| claim | verdict | why |
|---|---|---|
| **FCST-03** M9 `self.provisional`→`here.provisional` (dd40c56, 71cd784) | **re-verified** | this is the original false survivor. Re-run with the cache cleared, M9 is caught. The shipped source was correct throughout. |
| **FCST-03** X1, X6 degenerate stimuli (71cd784) | **not at risk** | run after the purge discipline landed in that branch. |
| **SAFETY-24** 11 author + 9 reviewer mutations (e709554, 4601682) | **not at risk** | the commit records them run under `python -B` with a purged `__pycache__` and real byte deltas. Its one survivor is an equivalent mutant, argued structurally rather than by a green run. |
| **SAFETY-26** tolerance sweep over `arbitrage.py` and `static_tariff.py`, 2 survivors (2b8b8cb) | **not at risk** | purge, printed `+5 B` delta, mutated line asserted present, tree re-read before each run. One survivor is explicitly recorded at `+0 B` — the risky delta — and was produced *under* the purge. **Both survivors are now closed by TEST-16** (`test_static_tariff.py` section 9), which also found that the two of them *masked each other* — each was individually non-live only because the other rung caught its stimulus, an arrangement single-mutation testing cannot see. Its own ledger is 7 mutations, 4 of them at `+0 B`, all red. |
| **OPT-04** mutations reported on PR #22 (bdd7f18) | **not at risk** | `test_arbitrage.py`'s header records the exact command: purge, then `python3 -B`. |
| **TEST-17** `checks()` dropping `when` (504e8da) | **not at risk** | a *red* claim, and dropping an argument changes the file length. Neither half of the collision is available. |
| **SAFETY-07** wiring mutations B1/B2/B3 and `test_peakguard.py` parts 7 and 10 (2d570d8, e7c95bf) | **not at risk** | those mutations are **executed in-process**, monkeypatched onto the imported module. No source file is edited, so no `.pyc` is consulted. |
| every **"all red, no survivors"** claim in the window — TARIFF-02 (24/0), TARIFF-03 (11), MAINT-04 (10), SAFETY-18 (6+4), SAFETY-03 (6), TEL-04 (10), TEL-05 (6) | **not at risk** | direction. Stale bytecode turns a red into a green survivor, never the reverse. A red is evidence that *something* changed under the check. |
| **TEST-06**'s deleted assertion, "no new notification while parked at 0 W" (c34b5a2) | **not at risk / superseded** | it was recorded as undecidable and then *falsified* — `test_freeze.py` records the one-line mutation that takes the file red 9 times. The stronger later claim is a red one. |
| **TARIFF-01** reviewer survivor: `record_block()` ignoring its `policy` argument (0dbd485) | **re-verified — still a survivor** | re-run here through `tools/mutation_run.sh` against the 15 suites that were on the record, all green with `billed_w(block, policy)` mutated to `billed_w(block, POLICY_NET)`. The recorded diagnosis holds: every block reaching the store is single-phase, where NET, SUM and MAX collapse. **Now CLOSED by TEST-13** (`test_tariff.py` section 7), which round-trips section 7's own unbalanced `(3000, 500, -200) W` block THROUGH the store under each policy - three distinct banked figures, so a hardcoded policy takes at least two of them red. Third instance of "a stimulus evaluated where the quantity it grades is degenerate", after WITNESS-02's span normalisation and TARIFF-01's own DST window. |
| **SAFETY-25** 8 mutations against `test_peakguard.py` part 10, then 9 more after review, all red | **not at risk** | each run against a tree copied fresh with `__pycache__` excluded and mounted `:ro`, so no `.pyc` existed to go stale - the same structural immunity recorded for CI above. Byte deltas recorded on the PR; none is `+0 B`. All are **reds**, which is the direction this defect cannot fabricate. ⚠️ The second round exists because the first round's mutations were all aimed at DISARMING the guard, and the claim part 10 actually made was about the guard being FIXED - the reviewer built the fix and the suite stayed green. Bytecode was never the issue there; an unaimed battery was. |
| **SAFETY-03** reviewer survivor: `abs()` dropped from `clamp_to_rating`'s limit (6aabb83) | **re-verified — still a survivor** | re-run the same way, all 15 green. Also as recorded: unreachable under the `int(100, 5000)` schema. |
| **MAINT-05 round 3**, `migrate_maint_keys()` marker gate `if rep.ok:` → `if rep.ok or rep.kept:` (`+12 B`) | **was a survivor — now CLOSED** | measured against head `65cddf3` under the purge, tree mounted `:ro`, `19/19 PASS`. It stamps the schema marker on a migration that LOST state, defeating the older `lost` guard as well as the new `misdirected` one, because every check that reached the marker did so on a store whose `kept` list was empty. Closed by `test_maintenance.py` **9d**; the mutation is now red on three named checks. |
| **MAINT-05 round 3**, `migrate_unit_keys()` marker gate `if rep.ok:` → `if rep.ok or rep.kept:` (`+12 B`) | **found here — SAFETY-02's half, now CLOSED** | the SAME mutation one function up, on the LATCH migration rather than the history one. Measured a survivor against this branch under the purge, `:ro`, `19/19 PASS`, then closed by `test_inverter.py` part 10's new fixture — a store with `unit/inverter-a/sign_reason` already present (so `kept` is non-empty) whose `sign_tripped: true` write is dropped. Shipped: marker withheld. Mutant: marker stamped on a **disarmed latch**, and the next start skips the migration that would have re-armed it. |
| **TARIFF-01**'s other three survivors, **SAFETY-03**'s `describe()` survivor, **TARIFF-02**'s M3/M5/M7, **SAFETY-18**'s three | **still unverified** | recorded before the purge discipline, with no byte delta on the record, so a `+0 B` same-second revert cannot be excluded. All are survivors, which is the direction this defect fabricates. TARIFF-02's and SAFETY-18's were **closed by later commits whose claims are reds**, so what is unverified is the *diagnosis*, not the current state of the code. |
**Nothing here says a past result is wrong.** The two survivors re-run above
both stood. What changed is that they are now on the record as measured rather
than assumed, and the rest are labelled honestly. The still-unverified rows are
cheap to close — one harness run each — and the right moment is when somebody
next touches that module, not in a re-run campaign nobody will read.