adminandClaude Opus 5
7a5281e97f
checks / unit (pull_request) Successful in 7m10s
One conflict, in CHANGELOG.md's `## Unreleased` block, exactly as expected from two branches landing beside each other: both `_this branch_` table rows and both sections are kept, FCST-06's first. Nothing else conflicted - FCST-06's `test_price_history.py` section 5d and this branch's columnar `feed_body()` merged cleanly - and all 29 test files are green on the result. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
801 lines
55 KiB
Markdown
801 lines
55 KiB
Markdown
# Mutation runs in this repository — and the one way they lie
|
||
|
||
Nearly every safety claim in this add-on was established the same way: change
|
||
the shipped module, run the suites, watch one go **red**, put the module back,
|
||
watch it go **green**. There is no coverage tool here and there is not meant to
|
||
be — the check that matters is whether a *check* notices when the behaviour it
|
||
names is removed.
|
||
|
||
That method has exactly one silent failure mode, and TEST-15 exists because we
|
||
walked into it.
|
||
|
||
## The defect
|
||
|
||
FCST-03's developer reverted `self.provisional` to `here.provisional`, re-ran,
|
||
and reported the mutation as a **survivor**. It was not. CPython decides a
|
||
cached `.pyc` is still valid from two numbers:
|
||
|
||
* the source file's **size in bytes**, and
|
||
* the source file's **mtime truncated to whole seconds**.
|
||
|
||
`self` and `here` are the same length, and the revert happened in the same
|
||
second as the edit. Neither number moved, so the interpreter loaded the
|
||
bytecode it already had. Confirmed at the time by disassembly: **the source read
|
||
`here`, the running code loaded `self`.** The suite had graded code that was not
|
||
there — no error, no warning, exit status 0.
|
||
|
||
The same collision, in the other direction, can leave a **red** standing on
|
||
correct source after a revert that never took.
|
||
|
||
### Which way it lies, and why that scopes everything
|
||
|
||
A mutation that never took effect leaves the suite **green**, and a green run of
|
||
a mutated tree is reported as a survivor. So stale bytecode **manufactures
|
||
survivors**; it cannot manufacture a red. The false-red case exists but
|
||
announces itself, because a developer chases a failure they cannot explain.
|
||
|
||
Two consequences, and both are used below:
|
||
|
||
* a recorded **"all red, no survivors"** result is not at risk from this defect;
|
||
* a recorded **survivor** — and a recorded *zero*, such as a coverage-audit
|
||
count of 0 — is exactly what this defect produces.
|
||
|
||
`test_control.py`'s coverage-audit table says the same thing in its own terms:
|
||
the error is one-directional, so every figure there is a lower bound at worst.
|
||
|
||
## The rule
|
||
|
||
**Purge `__pycache__` before every mutation run.** Not afterwards, not usually,
|
||
not when the edit looks big enough.
|
||
|
||
```sh
|
||
cd goodwe_controller
|
||
tools/mutation_run.sh # every test_*.py in this directory
|
||
tools/mutation_run.sh . test_tariff.py # or just the ones you are watching
|
||
tools/mutation_run.sh "$GOODWE_ADDON_REPO/goodwe_controller" # from elsewhere
|
||
```
|
||
|
||
The harness purges, prints a `sha256`/byte-count fingerprint of every `app/*.py`
|
||
it is about to grade, runs each suite under `python3 -B`, and fails if a cache
|
||
appeared during the run. Diff two fingerprints and the file you mutated is the
|
||
line that changed — which is also how a mutation report proves *which source*
|
||
produced it.
|
||
|
||
### What does **not** fix this
|
||
|
||
| | writes bytecode | reads a stale `.pyc` |
|
||
|---|---|---|
|
||
| `python3 -B` | no | **yes** |
|
||
| `PYTHONDONTWRITEBYTECODE=1` | no | **yes** |
|
||
| purging `__pycache__` | (unchanged) | **no — nothing to read** |
|
||
|
||
Both flags are *write-side only*. A mutation run always has a cache already,
|
||
because the previous run wrote it, so a harness that only adds `-B` closes
|
||
nothing. This is measured, not read off documentation: `test_bytecode.py`
|
||
part 2 arranges the collision and shows both flags still running the old code.
|
||
|
||
`-B` is still worth passing — it stops **this** run leaving a cache for the
|
||
**next** edit to inherit, which is the one nobody is thinking about three
|
||
mutations into a batch. Purge for this run, `-B` for the next one.
|
||
|
||
### And not a `sleep`
|
||
|
||
One-second mtime resolution is the *mechanism*, not the enemy. Waiting out the
|
||
clock would slow every run down, and would still only cover the edit somebody
|
||
remembered to slow down. Stop trusting mtime instead.
|
||
|
||
## Reporting a mutation
|
||
|
||
State the things a reviewer cannot reconstruct afterwards:
|
||
|
||
1. **the byte delta** of the mutated file (`+5 B`, `+0 B`). `+0 B` is the risky
|
||
one — say so;
|
||
2. **that the mutated line was on disk** when the run started (the harness
|
||
fingerprint, or a `grep` of the line);
|
||
3. **which suite went red, and on which named check** — "the suite is red" and
|
||
"the check that watches this is red" are different claims;
|
||
4. for a **survivor**, that the run was purged. An unpurged survivor is not a
|
||
result.
|
||
|
||
## The other thing a mutation sweep cannot see: masking (TEST-16)
|
||
|
||
Stale bytecode is not the only way a survivor lies. **Two guards can mask each
|
||
other**, and a sweep that mutates one line at a time is structurally blind to it.
|
||
|
||
TEST-16 is the worked example. `static_tariff.arbitrage()` refuses a
|
||
one-firm-hour series at `len(firm) < 2`, and refuses a flat series at
|
||
`spread == 0.0`. A single firm hour *also* has a spread of zero. So:
|
||
|
||
* weaken the firm-pair rung alone → the flat rung refuses the stimulus anyway;
|
||
* weaken the flat rung alone → the tiny-spread stimulus is still refused, just
|
||
for the wrong reason;
|
||
* and the suite, grading `kind == "refused"`, reports **nothing either way**.
|
||
|
||
Both rungs could then be weakened *together* and still nothing would report —
|
||
which is precisely the state in which the next edit to either one opens a real
|
||
hole. It survived four rounds of review on TARIFF-03 and FCST-03.
|
||
|
||
Two rules come out of it, and both are cheap:
|
||
|
||
1. **Grade the reason, not just the verdict.** "Refused" and "refused for the
|
||
right reason" are different claims, and a masked rung is the case where the
|
||
first is true and the second is false. A refusal sentence is what the
|
||
operator reads; it is also the only observable that moves when one rung
|
||
quietly absorbs another's work.
|
||
2. **When two rungs can answer the same stimulus, mutate them together.** One
|
||
extra run per suspected pair. If a sweep finds two survivors in the same
|
||
function, that pair is the first thing to try — TEST-16's two survivors were
|
||
adjacent lines in one function and neither was individually live.
|
||
|
||
⚠️ **AND FCST-05 WALKED INTO IT AGAIN, IN ITS OWN NEW CHECK, WHICH IS WHY THE
|
||
RULE IS “mutate every check you add”.** `price_history.py` reports a malformed
|
||
point row from **two** places into **one** field: `_usable_days()` records what
|
||
it FOUND when the file was opened, and `_remember()` records what it DROPPED on
|
||
the next refresh. Both land in `last_error`, and `words` prints `last_error`.
|
||
The first draft of the new check asked `"malformed point row" in words` — which
|
||
whichever rung is still standing satisfies — so deleting `_remember()`'s note
|
||
entirely was a **−1 B SURVIVOR of the whole suite**, on the half of the fix that
|
||
tells an operator the cache **self-healed**. It is graded on the TENSE now
|
||
(`… in <day>` at open, `… dropped from <day>` after the refresh), the two are
|
||
asserted to DIFFER, and the paired mutation — both notes deleted, −91 B — is
|
||
red on 15 named checks. Same shape as TEST-16 and the same fix: **grade the
|
||
reason, not the verdict**, and when two rungs answer one stimulus, mutate them
|
||
together.
|
||
|
||
⚠️ **AND FCST-06 FOUND THE COUNT ITSELF LYING — LOW, NOT SILENT.** A mutation
|
||
whose red is a **traceback** does not report one red; it reports however many
|
||
checks ran before the traceback, and grades **none** of the ones after it. The
|
||
new section drove M4 (`self._days = before` → `self._days = {}`) and the run
|
||
printed **1 FAIL**. The true figure is **3**: an f-string in the next check read
|
||
`len(read.points)`, and `.points` on a `no_data` **raises `Absence` by design**,
|
||
so the harness died mid-section with the remaining checks ungraded. That is
|
||
`why()`'s and `derived()`'s rule — *"it answered nothing" and "it broke on the
|
||
way to answering" must not grade the same* — arriving in the **printed operand**
|
||
of a check rather than in its condition, which is the half that is easy to miss
|
||
because it looks like formatting. Two things follow:
|
||
|
||
* **a mutation report's red COUNT is only a lower bound until the run is seen to
|
||
have reached the end of its section.** Read the output, not the count;
|
||
* **the total accessor has to cover what a check PRINTS, not only what it
|
||
asserts.** `test_price_history.py` 5d added `npts()` beside `why()` for
|
||
exactly this, and re-driving M4 against it moved 1 FAIL to 3.
|
||
|
||
Worse, had the traceback landed one check earlier the mutation would have
|
||
produced **no named red at all** — a result that reads like a broken harness and
|
||
gets retried rather than read.
|
||
|
||
## The third way a check lies: its DATA cannot reach the failure it names
|
||
|
||
Stale bytecode manufactures survivors from a mutation that never ran. Masking
|
||
manufactures them from a second guard doing the first one's work. This one
|
||
manufactures them from a **fixture**: the check runs, the assertion is
|
||
evaluated, the sentence above it is true — and the numbers it compares are
|
||
arranged so that the failure it describes could not have changed the answer.
|
||
|
||
It is not hypothetical and it is not rare. ELEVEN instances in two weeks; four
|
||
were found by review, two - CFG-04's fixture pair - by an author's own mutation
|
||
battery, which is the cheaper place, one by review again, in the one
|
||
place nobody had thought to point the rule at: not a check in the suite, but a
|
||
COMMITTED EVIDENCE ARTIFACT four documents cite - the eighth INSIDE THE FIX
|
||
WRITTEN TO CLOSE THE SEVENTH, the ninth inside the SITE HALF of the card
|
||
that names the rule in its own text - and the TENTH INSIDE THE FIX WRITTEN TO
|
||
CLOSE THE NINTH, for the second time in this list. See SAFETY-38 below: the
|
||
check that closed it asserted a NUMBER and not the number's POSITION.
|
||
|
||
⚠️ AND THE ELEVENTH IS IN A MEASURING INSTRUMENT RATHER THAN IN A CHECK, which
|
||
is where the seventh was too. SAFETY-38 round 4's render probe - the thing that
|
||
decides whether a mutation changed the SENTENCE and not merely the bytes -
|
||
numbered each interpolated operand BY ORDER OF APPEARANCE. After a transposition
|
||
the other operand takes the same number, so all thirteen mutant renders came back
|
||
BYTE-IDENTICAL to the clean tree: the probe compared the swap against itself, and
|
||
its output read exactly like "the edit is inert". Keying the value to the
|
||
operand's NAME fixed it. Habit 16.
|
||
|
||
⚠️ **AND IT HAS NOW APPEARED IN SIX CONSECUTIVE CARDS.** The seventh instance,
|
||
MAINT-05's, is the one that says why: the defect is not carelessness about
|
||
fixtures, it is that **a check and the thing it checks were built from the same
|
||
constant**. Ask of every difference-claim not "is this assertion true" but
|
||
"where did each side of this comparison come from" - and if the answer is the
|
||
same place twice, the check is decoration.
|
||
|
||
* **ARB-03** shipped a call-site scan asserting that every `claims.append` in
|
||
`cycle()` routes through `actuator_of()`. Its own parse **saw 3 of the 8
|
||
appends**; the other five were invisible to it and would have passed however
|
||
they were routed.
|
||
* **ALLOC-01** shipped a replay generator to hunt for envelope escapes. It found
|
||
**0**. A widened generator over the same allocator found **39,991**.
|
||
* **DEV-02** shipped `"each unit's supervised bound is ITS OWN max_w — give
|
||
inverter B a different rating and the claim filed on inverter B moves with
|
||
it"`, comparing inverter B's bound against `_b2.max_w`. The fixture defaulted
|
||
B's rating to A's, so it compared `(-5000, 5000)` against `(-5000, 5000)` —
|
||
**a number against itself**. Filing both per-unit bounds from
|
||
`self.inverters[0]` left **all 19 files green**.
|
||
* **CFG-04** wrote *"an entry that omits `soc_entity` has NO SoC sensor and does
|
||
not borrow the flat one"* against a fixture whose **flat `soc_entity` was
|
||
blank**. Inherited and not-inherited were both `""`, so the mutation that
|
||
makes `build_roster()` fall back to the flat key survived. The fixture now
|
||
sets a real entity there, and the check asserts that the flat value is present
|
||
as well as that the entry's is empty.
|
||
* **CFG-04** also wrote *"main.py runs the per-unit range check
|
||
UNCONDITIONALLY"* as a **substring test over `app/main.py`'s source**. The
|
||
mutation that defeats it puts the call back inside `if entity:` — one level of
|
||
indentation — and the substring is still there. That is habit 4 below in a new
|
||
costume: the check asserted against the TEXT of the thing under test rather
|
||
than against its behaviour. It now drives the shipped `run_control()` on a
|
||
two-inverter config, **under a deadline**, because the mutation's real symptom
|
||
is that the controller enters its loop and never returns.
|
||
* **MAINT-05** shipped `Controller.__init__` calling
|
||
`migrate_maint_keys(store, self.inverters[0].name, ...)` under a comment
|
||
saying *"it targets `self.inverters[0].name` and NOT `UNIT_A` ... writing
|
||
`UNIT_A` here would deposit an install's whole maintenance history under a
|
||
name that is not on the roster at all"*. `inverter.UNIT_A` **is** the string
|
||
`"inverter-a"`, and every two-pack fixture in `test_maintenance.py` named its
|
||
first roster entry `inverter-a` — so the two expressions were the same value
|
||
in every check that existed, and swapping them left **all 19 files green**
|
||
while losing a real install's whole maintenance history. Two-pack fixtures are
|
||
now `["garage", "shed"]`, each behind a precondition check (habit 1) that the
|
||
first name is not `UNIT_A`.
|
||
|
||
⚠️ **AND THE POSTCONDITION HAD THE SAME SHAPE AS THE FIXTURE.**
|
||
`store._verify_moves()` read back the keys it had just written, under the
|
||
namespace it had just been handed — so it verified clean, and the caller
|
||
stamped the schema marker, on a migration nothing would ever read. *A check
|
||
that reads back the namespace it was handed cannot detect being handed the
|
||
wrong one*: it compared a thing against itself, exactly as DEV-02's fixture
|
||
compared `(-5000, 5000)` against `(-5000, 5000)`. The habit-2 fix is the same
|
||
one — grade against something the thing under test did not supply, here the
|
||
ROSTER the machines read.
|
||
|
||
⚠️ **AND THE FIX ITSELF CARRIED THE DEFECT, WHICH IS THE SIXTH CARD IN A ROW
|
||
AND THE FIRST TIME IT HAS APPEARED *INSIDE* A REMEDY.** The fix round's
|
||
recovery check was `migrate_maint_keys(_m5, ROSTER[0], ROSTER)` — the
|
||
**corrected** name. It measures recovery along the one path where the operator
|
||
has *already fixed the bug*, and nothing in the 19 files called
|
||
`migrate_maint_keys()` twice with the **same wrong** `unit`. So the refusal
|
||
the card was rejected twice to obtain lasted **exactly one start**: measured
|
||
at Controller level over one persisted store, roster `["garage", "shed"]` —
|
||
construction 1 `ok=False misdirected=4 moved=4 kept=0 marker=<absent>`,
|
||
construction 2 `ok=True misdirected=0 moved=0 kept=4 marker=1`. On start 2
|
||
rule 1 routes the wrongly-written keys to `rep.kept`, `rep.moved` is empty,
|
||
the roster loop iterates over nothing, and the marker is stamped on a
|
||
migration that reached no readable key — after which the early return skips it
|
||
forever, *including after the call site is fixed*. The check was true and
|
||
passed for the wrong reason. Habit **9** below is what falls out of it.
|
||
|
||
* ⚠️ **ARCH-02's OWN REHEARSAL HARNESS, CAUGHT BY ITS OWN MUTATION BATTERY —
|
||
the THIRTEENTH instance, and the third in an INSTRUMENT rather than in a
|
||
check.** `tools/rehearse_roster.py` reports orphaned Home Assistant entities
|
||
by composing the ids the add-on announces **before** the roster switch and
|
||
**after** it, and diffing. Both sides are composed by the same build. So the
|
||
two mutations that change the id composition itself — renaming the MQTT device
|
||
(`"GoodWe"` -> `"GoodWe Controller"`), renaming one site sensor (`"Grid power"`
|
||
-> `"Grid power W"`) — moved **every id on both sides equally**, and the
|
||
harness reported *"no orphans"* about a change that orphans the entire live
|
||
entity list. It read exactly like a clean transition. The first draft's
|
||
M5/M6 came back `exit 0` and `exit 2`, and neither was the failure they name.
|
||
|
||
The fix is habit 2 with a filename: **the reference has to be data that does
|
||
not move with the code.** `--live-entities` takes a list of the ids the
|
||
install actually has, `tools/rehearsal/live-entities-recorded.txt` is the
|
||
recorded stand-in that keeps it live in CI, and a build that announces an id
|
||
the list does not carry is now a `COMPOSITION DRIFT` finding rather than a
|
||
silence. ⚠️ **The anchor moved for the same reason**: it had been
|
||
`compose(mqtt.DEVICE["name"], p1["name"]) == "sensor.goodwe_p1_sample_age"`,
|
||
which any renaming of the device satisfies by moving both operands, and it is
|
||
now composed from two recorded **literals**.
|
||
|
||
Ask of an instrument what habit 4 asks of a check: *where did each side of
|
||
this comparison come from* — and if a mutation of the subject moves both
|
||
sides, the instrument is decoration.
|
||
|
||
⚠️ **AND REVIEW FOUND TWO MORE IN THE SAME CARD, ONE OF THEM INSIDE THE FIX
|
||
FOR THE OTHER — the FOURTEENTH and FIFTEENTH.** Both are the pin, not the
|
||
tool, and both are the same sentence in a new costume: *a check whose data
|
||
cannot reach the failure it names.*
|
||
|
||
* **Fourteenth — a COUNT cannot tell one red leg from two.** The rehearsal
|
||
reports two independent routes to the disarmed latch (`SAME-STEP` and
|
||
`STAGED`) and prints every finding **twice per red leg**: once in that
|
||
leg's own block and once in the closing summary. The check named *"NOT THE
|
||
SAME-STEP PATH ALONE"* was written `out.count("LATCH CAME UP DISARMED")
|
||
>= 2`, so **one** red leg always satisfied it. Measured on three builds:
|
||
shipped code (both legs red) → 4; `migrate_unit_keys()` handed the roster
|
||
name (SAME-STEP fixed) → 2, **passes**; a guard-side fix (both fixed) → 0.
|
||
The check reported `ok` on the build whose only remaining failure *is* the
|
||
path it says is not the only one. The fix is to parse the legs and require
|
||
the finding in **each**, which is habit 2 again: grade the thing, not a
|
||
total over the thing.
|
||
|
||
* **Fifteenth — and it is INSIDE THAT FIX, which is the third time this file
|
||
has had to write that sentence.** The first leg-splitter opened on a
|
||
`LEG <id>` marker and never closed, so every line after the *last* marker
|
||
was attributed to that leg — including the closing summary, which repeats
|
||
every finding from every leg. The last leg therefore read red whenever
|
||
**any** leg was red, and the new per-leg check was the old count with extra
|
||
steps. It was caught by building the one fixture that separates them: a
|
||
tree in which `SAME-STEP` is genuinely clean and `STAGED` is not. The tool
|
||
now prints `END LEG <id>` and the splitter closes on it.
|
||
|
||
The rule that falls out, and it is the cheap one: **a per-thing check that
|
||
has never seen the things DISAGREE is a total.** Part 5b exists to make
|
||
them disagree on every run.
|
||
|
||
⚠️ **AND ONE THAT HAS ONLY SEEN THEM DISAGREE IN ONE ARRANGEMENT IS A TOTAL
|
||
OVER THAT ARRANGEMENT.** Review found this by attacking the fix itself. Part
|
||
5b does make the legs disagree, on every run - and was still blind to a build
|
||
that merely REORDERS them, because it had only ever seen the clean leg
|
||
printed LAST. Three edits, two of them ordinary refactors (reorder the report
|
||
legs, reword a summary line), produced `all checks passed` from a splitter
|
||
that closed ZERO legs.
|
||
|
||
A fixture cannot enumerate the arrangements, so the STRUCTURE is asserted
|
||
instead: a leg may not open while one is open, and every leg that opens must
|
||
be explicitly closed. That fires on the FIRST edit of all three attacks,
|
||
whatever the order or the wording. **When a fixture cannot enumerate the
|
||
arrangements, assert the invariant they share.**
|
||
|
||
* **CFG-04's `tools/supervisor_shape_probe.py` §B** — and this one is the rule
|
||
reaching outside the suite. The probe is the committed evidence that
|
||
`config.yaml`'s `inverters: []` default is load-bearing, and four documents
|
||
quoted it as answering *"Missing option 'inverters' in root"*. §B validated
|
||
the reconstructed LIVE options blob, which is missing many required keys
|
||
unrelated to this card, so its refusal named whichever of THOSE the validator
|
||
reached first — `meter_freeze_s`, `p1_phase_export_entities` and
|
||
`sign_disagree_w` on three different runs. The quoted string was never
|
||
produced by the instrument. And the assertion was a bare **`not ok`**: delete
|
||
`inverters: []` from `config.yaml` and §B went on printing **PASS**, blind to
|
||
the exact regression it was cited for. §B is now an isolated counterfactual —
|
||
the shipped defaults, and the same defaults minus that one key — asserting on
|
||
the **key name in the refusal**; the mis-cited blob survives, relabelled, as
|
||
§B2. Re-run with the default deleted it prints `2 CASE(S) FAILED` and exits 1.
|
||
|
||
⚠️ **A committed transcript is a check, and it decays like one.** Nothing in
|
||
this document had been pointed at `tools/`, because the artifacts there are
|
||
filed as *measurements* rather than as *checks* — and a measurement whose
|
||
assertion cannot fail is exactly the thing this section is about. Two habits
|
||
follow, and both are habit 5 and 6 below.
|
||
|
||
* **SAFETY-38** — `main.py`'s
|
||
`site_standing = float(sum(standing.values()))`, the first argument of
|
||
SAFETY-30's latching site sign detector *and* SAFETY-18's `looks_frozen()`
|
||
input. `test_watchdog.py`'s `_sign_site` is the suite's **only** N≥2 site
|
||
fixture, and it runs shares of about **−2500 W each** — every one
|
||
individually over the 320 W gate it is configured with. On numbers like those
|
||
`sum`, the mean and `max(key=abs)` **all agree**, so the reduction could be
|
||
swapped for either of the other two and the site guard went **blind with the
|
||
whole suite green** — still commanding −500 W of charge into a +2500 W import at
|
||
t = 595 s with `following=True`. At a roster of **one** it is worse and it is
|
||
not a fixture defect at all: `sum == mean == max == the one share`
|
||
*identically*, so every such mutation is genuinely **equivalent** there and no
|
||
N=1 fixture could ever separate them. The fixture that separates them is the
|
||
one the detector exists for — **per-unit shares EACH BELOW the gate whose SUM
|
||
clears it** — and it is now `test_watchdog.py` part 9, at asymmetric ratings
|
||
so that all four candidate reductions give four **distinct** answers.
|
||
|
||
⚠️ **AND THE CARD'S OWN SEARCH FOR ITS NEXT SURVIVOR FOUND ONE, WHICH IS THE
|
||
POINT OF ASKING.** Having built the fixture, the question *what mutation would
|
||
this NOT see?* produced `float(self.target)` — the arbiter's site target
|
||
instead of the sum of what stands on the wire — at **−11 B**, green across the
|
||
whole suite, because every fixture in the repository was built where the two
|
||
are the same number. They separate only when a device's reported range is
|
||
narrower than its `max_w`, since the `device` claim is filed **per actuator**
|
||
and never bounds the site target. Closed in the same commit, by a
|
||
counterfactual that asks for −500 W while −250 W stands.
|
||
|
||
⚠️ **AND THEN THE FIX ITSELF WAS THE TENTH INSTANCE — A SUBSTRING WHERE THE
|
||
FAILURE WAS POSITIONAL.** The check written to pin the banner asserted
|
||
`f"{sum(...):.0f} W" in reason`: that **"−500 W" appears somewhere** in the
|
||
sentence. Swapping the detector's first two arguments
|
||
(`evaluate(site_standing, self.grid, …)` → `evaluate(self.grid, site_standing, …)`)
|
||
is a **ZERO-BYTE** edit that leaves both numbers in the string and moves each
|
||
into the other's slot — so the substring is still found, **in the meter
|
||
slot**, and the whole suite stayed green while the latching site-stopping
|
||
banner told the operator which instrument read what **exactly backwards**.
|
||
The fix is to assert the **ORDERED PAIR**,
|
||
`f"commanded {sum:+.0f} W against a meter reading {grid:+.0f} W"`.
|
||
|
||
⚠️ **AND THE ELEVENTH WAS THE SAME DEFECT ONE CALL SITE OVER — THE ONE A
|
||
SHIPPED INSTALL CAN ACTUALLY SEE.** Round 2 pinned the operand order of
|
||
`SignWatchdog.evaluate()` at `main.py:1620`. That call site is the **site**
|
||
detector, and `test_watchdog.py` part 9c asserts it is **never armed at a
|
||
roster of one** — which is every install this add-on has. The **per-unit**
|
||
call at `main.py:1560` was left unpinned, and swapping its first two
|
||
arguments is the same **ZERO-BYTE** edit: `disagrees()` is symmetric in them
|
||
(`commanded_w * against_w < 0` with an `abs()` gate on both), so **no trip
|
||
decision can change** and the full 21-file suite stays green — measured. What
|
||
moves is five banner lines, including the one persisted to `sign_reason` and
|
||
re-read after a restart: *"commanded −1000 W against its own measured power
|
||
+1000 W"*, operands inverted, on the only sign-disagreement sentence a
|
||
one-inverter install can ever print. Closed in round 3 by the ordered pair on
|
||
**both** per-unit banners — the N≥2 `its own measured power` one and the N=1
|
||
`a meter reading` one — red on four named checks, green in the other twenty
|
||
files. **Habit 15 is what falls out of finding the same shape twice in the
|
||
same card.**
|
||
|
||
The general habits are **14** and **15** below.
|
||
|
||
Note what these have in common with a masked rung: the mutation *runs*, and the
|
||
suite is *honestly* green. Nothing in a purge, a fingerprint or a disassembly
|
||
can see it, because the code under test really did behave identically on the
|
||
stimulus it was given. The defect is upstream of the assertion, in what the
|
||
check was handed.
|
||
|
||
**The rule, and it is one line:**
|
||
|
||
> **Any check asserting "X differs from Y" must be run once with X and Y
|
||
> actually made different.**
|
||
|
||
Cheap habits fall out of it, and all of them are now used in this repository:
|
||
|
||
1. **Write the PRECONDITION as its own check.** `test_peakguard.py` 10g asserts
|
||
that A and B have different ratings *before* it asserts that B's bound is
|
||
B's. If a later edit makes them equal again, the precondition goes red rather
|
||
than the claim going quietly vacuous.
|
||
2. **Assert against literals, not against the expression under test.**
|
||
`== [(-_b2.max_w, _b2.max_w)]` cannot distinguish "B's number" from "the
|
||
number this fixture happens to have twice"; `== (-3000.0, 3000.0)` beside a
|
||
printed `A = 5000` can.
|
||
3. **Print what the check saw.** Every one of the three above would have been
|
||
caught by a human reading one line of output — `supervised bounds by
|
||
actuator: {'inverter-a': (-5000, 5000), 'inverter-b': (-5000, 5000)}` is
|
||
self-evidently not evidence of a difference.
|
||
|
||
4. **Assert on the BEHAVIOUR, not on the source text of the thing under test.**
|
||
A substring test over a module is a check on how the code is spelled;
|
||
whitespace, an alias or a moved line defeats it while the property it names
|
||
is gone. `test_inverter.py` part 10 has one of each — a source pin for a
|
||
claim that genuinely IS about the code's shape, and a driven `run_control()`
|
||
for a claim about behaviour — and the second replaced the first after the
|
||
mutation battery walked straight through it.
|
||
5. **A check whose failure mode is a HANG needs a deadline.** Otherwise the
|
||
mutation that defeats it produces a CI timeout with no name attached, which
|
||
reads like flaky infrastructure and gets retried rather than read.
|
||
6. **Assert on the CONTENT of a refusal, not on the fact that one happened.**
|
||
`not ok` is satisfied by any error at all, including one about a different
|
||
key on a different blob. `"Missing option 'inverters' in root" in out` goes
|
||
red the moment the refusal stops being about the thing it is cited for. This
|
||
is habit 2 in the negative direction: a refusal, like a difference, has to be
|
||
asserted against a literal.
|
||
7. **Vary ONE thing in a counterfactual, and vary it from an ACCEPTED
|
||
baseline.** `supervisor_shape_probe.py` §B validates the shipped defaults
|
||
first, precisely so the refusal on the next line has nowhere else to come
|
||
from. Without the accepted baseline the second line is a refusal of unknown
|
||
provenance, which is what §B used to print.
|
||
8. **A committed transcript is a check.** Anything under `tools/` that a
|
||
document quotes is subject to every rule above, and to the mutation battery:
|
||
break the property, re-run the instrument, and confirm the transcript goes
|
||
red. `inverters: []` was deleted from `config.yaml` and the probe re-run
|
||
against `ghcr.io/home-assistant/amd64-hassio-supervisor:2026.08.0` for
|
||
exactly that reason.
|
||
9. **Exercise a refusal a SECOND time with the SAME wrong input — not the
|
||
corrected one.** A guard that refuses once and then admits the same bad
|
||
caller is not a guard, it is a delay, and the second call is a *different
|
||
code path* whenever the first call left state behind. MAINT-05's roster
|
||
grading iterated the keys it had just **written**; on the second start the
|
||
same wrong caller found them already present, they moved to the
|
||
`already namespaced` list, and the loop graded an empty set. Re-running with
|
||
the **fixed** argument — which is what the check did — measures the operator's
|
||
recovery, never the persistence of the refusal. Ask of every refusal: *what
|
||
does this do when the caller has not been fixed and simply runs again?* And
|
||
assert the **shape** of that second report, not only its verdict, so the
|
||
check cannot silently revert to grading the first path.
|
||
|
||
10. **A fixture of N members cannot separate two reductions that agree on N
|
||
members — so grade the FAMILY, and grade it from the SOURCE.** This is the
|
||
only lesson SAFETY-32 cost four review rounds to learn, and it generalises
|
||
past packs to every roster, list and window in this repository. Two packs
|
||
cannot tell `max` from `min` from first from last. Three cannot tell `max`
|
||
from `max(x[-2:])`. **The regress does not terminate in a fixture**: `[:1]`
|
||
needs two members, `[:2]` three, `[:3]` four, and adding one more member
|
||
moves the hole rather than closing it. Three things follow, and all three
|
||
are now in `test_peakguard.py` parts 12c2, 12g, 12i and 12j:
|
||
* the fixture is a **SET** — the extreme at every position of the roster,
|
||
not one roster with the extreme somewhere;
|
||
* the family table is **COMPUTED and PRINTED**, so a member no fixture kills
|
||
shows up as a named survivor and takes the check red, instead of being
|
||
absent from a list nobody re-derived;
|
||
* where the claim is an **IDENTITY** ("this roster", "this list"), pin it
|
||
**exactly on the AST** — `ast.unparse(arg) == "self.packs"` is false of
|
||
every slice, where `"self.packs" in src` is true of all of them. Where the
|
||
claim is **SEMANTIC** ("the maximum"), a shape pin would refuse honest
|
||
refactors, so report the unreached width **with its size** instead.
|
||
|
||
11. **When a check of this shape is found, mutate its SIBLINGS before shipping
|
||
the fix.** SAFETY-32 was rejected three times for the same defect one step
|
||
over: the reduction, then the roster handed to it, then the reduction handed
|
||
to the *other argument of the same call*. Enumerating the family from the
|
||
AST and mutating each member took one afternoon and found **four more
|
||
survivors in code the card never touched** — three of SAFETY-30's
|
||
`any(i.sign_wd.tripped for i in self.inverters)` reductions, invisible
|
||
because the only fixture that reached them was a roster of ONE, where
|
||
`self.inverters[:1]` **is** `self.inverters`. Ask of every fix: *what is the
|
||
sibling of the thing I just hardened, and does any fixture separate it from
|
||
a slice of itself?*
|
||
|
||
12. **When the same shape has been found twice, STOP CLOSING SITES AND BUILD
|
||
THE INSTRUMENT — and make it DISCOVER its own scope.** SAFETY-32 was
|
||
rejected four times, and each round closed exactly the site it was handed:
|
||
the reduction, then the roster handed to it, then the *sibling argument* of
|
||
the same call, then the same family one *module* over, in validate.py.
|
||
Round 4 did enumerate — and enumerated inside the `Controller` class,
|
||
because that is where the four known sites lived. **A hand-written scope is
|
||
the same defect as a hand-written fixture**: it is a check whose data cannot
|
||
reach the failure it names, and it goes on passing while the thing it was
|
||
supposed to cover sits in the file nobody added to the list. Round 5's pin
|
||
parses every `.py` under `app/`, seeds itself from ONE option key, and
|
||
propagates through returns, accumulator loops, attributes and *call
|
||
arguments* to find the roster wherever it went — 183 reductions across 9
|
||
modules. It rediscovered all four blockers without being told where to look,
|
||
and it named two modules the hand-written list would not have contained
|
||
(`maintenance.py`, `store.py`) while *omitting* one the list did contain
|
||
(`watchdog.py`, which is strictly per-unit and reduces over no roster at
|
||
all). **The list was wrong in both directions**, which is exactly what a
|
||
computed scope is for.
|
||
|
||
13. **Say what the instrument CANNOT see, with sizes, in the same breath.** A
|
||
structural pin grades an **identity** ("this whole roster arrives"); it can
|
||
never grade a **meaning** (`min` vs `max` is a zero-byte edit and invisible
|
||
to it). Ship the pin AND the behavioural set, name the gap, and give every
|
||
unreached case its byte delta — `cycle()`'s undeliverable-unit log line
|
||
(+4 B, still green), a subscript by a *variable* rather than a constant, a
|
||
slice taken through an intermediate call, anything outside `app/`. **An
|
||
enumerated list with an honest gap is worth more than a claim of
|
||
completeness a reviewer falsifies in ten minutes**, and every round of this
|
||
card that claimed the latter was rejected for it.
|
||
|
||
14. **An `in` test against a formatted sentence grades PRESENCE, never
|
||
ARRANGEMENT — so assert the ORDERED PAIR.** SAFETY-38 pinned the banner of a
|
||
latching, site-stopping alarm with
|
||
`f"{sum(_share38):.0f} W" in ...reason` — that `"-500 W"` appears
|
||
*somewhere*. Swapping the detector's first two arguments at the call site
|
||
(`evaluate(site_standing, self.grid, …)` →
|
||
`evaluate(self.grid, site_standing, …)`) is a **ZERO-BYTE** edit: both
|
||
numbers are still in the string, each in the other's slot. The substring was
|
||
still found — **in the meter slot** — the whole suite stayed green, and the
|
||
banner told the operator which instrument read what **exactly backwards**.
|
||
Whenever the property you mean is "**this** value went in **this** slot" — a
|
||
reason string, a log line, a notification, an MQTT payload — assert the
|
||
surrounding **literal text with the value embedded in its own position**
|
||
(`f"commanded {x:+.0f} W against a meter reading {y:+.0f} W"`), or assert
|
||
the fields **before** they are formatted. Two values that both appear in one
|
||
sentence can always be transposed by an edit that costs **nothing**, and
|
||
that no byte delta, no fingerprint and no shape pin can see. ⚠️ Note where
|
||
this one was found: **inside the fix written to close the previous instance
|
||
of the same family**, which is the second time in this list.
|
||
|
||
⚠️ **AND THE FIX FOR IT WAS ONE CALL SITE WIDE, WHICH IS THE HALF WORTH
|
||
KEEPING.** The ordered pair went onto `main.py:1620`, the **site** detector
|
||
— armed only at N≥2, and part 9c asserts it is **absent at a roster of
|
||
one**. The identical swap at `main.py:1560`, the **per-unit** detector, was
|
||
still **ZERO BYTES**, still changed no decision, still left the 21-file
|
||
suite green — and it is the **only** sign-disagreement sentence a shipped
|
||
one-inverter install can ever print: *"commanded −1000 W against its own
|
||
measured power +1000 W"*, backwards, in the log, on the status page and in
|
||
`sign_reason` after the restart the latch is designed to survive. Measured
|
||
size: **main.py:1560, 0 bytes, 21/21 green.** So habit 14 is not "pin the
|
||
banner", it is **"pin every call site of the banner, and say which one the
|
||
shipped roster reaches"** — and habit 15 is how you find them all.
|
||
|
||
15. **A SYMMETRIC PREDICATE MAKES ARGUMENT ORDER A PROPERTY OF THE MESSAGE
|
||
ALONE — so ENUMERATE THAT FAMILY BY MACHINE, and grade the CALL SITES, not
|
||
the function.** Where a callee is symmetric in two arguments, a swap at any
|
||
call site is invisible to every behavioural check **by construction**: not
|
||
"no fixture happens to separate them" but *no fixture can*. There is
|
||
nothing for a mutation battery to find, because the mutant and the shipped
|
||
tree take identical actions on every input. The only surface that moves is
|
||
the sentence, and the only check that can see it is one asserting the
|
||
**ordered pair** (habit 14) at **that** call site.
|
||
Three consequences, and all three are why this is its own habit:
|
||
* **The unit of coverage is the CALL SITE.** SAFETY-38 pinned
|
||
`evaluate()`'s operand order and still shipped the defect, because the
|
||
pin was on the call two lines below the one that matters. Ask of every
|
||
banner: *how many places call this, and which does the shipped roster
|
||
reach?*
|
||
* **The scope must be COMPUTED** — habit 12's argument, and SAFETY-38 is
|
||
its proof: the family was enumerated by hand twice and was short both
|
||
times. `tools/symmetric_message_scan.py` renames each parameter pair,
|
||
erases every formatted string, canonicalises the commutative operators
|
||
and iterates to a **fixpoint** over `app/` (the symmetry is usually one
|
||
call deep — `evaluate()` is symmetric only *through* `disagrees()`), then
|
||
classifies each sentence by where it GOES. A returned string may be a
|
||
**key** — swapping `store.unit_key(unit, key)` renames a store key, which
|
||
is behaviour and is MAINT-05's card — or a **sentence**, which is this
|
||
family.
|
||
* **Say what the instrument cannot see, with shapes** (habit 13). That scan
|
||
under-reports semantic symmetry no canonicaliser on its finite list can
|
||
spell; its second census, for sentences the CALLER builds, pairs only
|
||
expressions **one leaf apart** and never sees `%s`-style logging; it
|
||
covers only `app/`; and it **over-reports in exactly one way** — it
|
||
erases the sentence, so a swap that would make the FORMATTING itself
|
||
raise is scored as changing no decision. It prints all of that on every
|
||
run. The family, every member's call sites, and which of them are pinned
|
||
are in `DECISIONS/SAFETY-38-site-standing-pin.md` §7 — **with two
|
||
measured, still-open survivors named there rather than rounded off**,
|
||
both zero bytes and both 21/21 green.
|
||
* ⚠️ **AND THE SINK IS HALF THE SCOPE, WHICH IS HOW THE FIRST THREE ROUNDS
|
||
OF THIS SCAN MISSED THE PLACE THE CODEBASE ACTUALLY KEEPS ITS SENTENCES.**
|
||
The scan classified a sentence as operator-facing when it was **appended,
|
||
assigned to a `reason`-shaped name, or logged** — and every one of its 16
|
||
reported members was one of those. **None was RETURNED.** A method
|
||
formatting its own **attributes** and handing the string back —
|
||
`ClampResult.describe()`, `PeakDecision.explain()`, every reason-builder
|
||
in the tree — was outside all of it, and so were sentences handed to a
|
||
constructor as `reason=` or placed under a `"text"` dict key. Four
|
||
zero-byte survivors of a green 21-file suite were sitting there, two of
|
||
them on the alarm SAFETY-40 and SAFETY-41 are about. **The general form:
|
||
an enumeration is scoped by a SINK as much as by a PREDICATE, and a sink
|
||
list is a hand-written list like any other (habit 12).** Ask of any such
|
||
scan: *which syntactic forms does this call a message, and what does the
|
||
code under it actually use?*
|
||
|
||
16. ⚠️ **AN INSTRUMENT THAT ASSERTS A COMPLETENESS IT DOES NOT HAVE IS WORSE
|
||
THAN NO INSTRUMENT, AND ITS BLIND-SPOT LIST IS PART OF THE CODE UNDER
|
||
REVIEW.** `tools/symmetric_message_scan.py` printed five honest gaps and a
|
||
census count, and a reader — including its author, twice — took the count
|
||
for the family. The gap that mattered, the RETURN sink, was **in none of the
|
||
five**. A gap list is not a disclaimer; it is a claim, and it is the claim
|
||
most likely to be believed because nothing else in the output argues with
|
||
it. Two consequences:
|
||
* **State what is NOT scanned, largest hole first, as a list of forms** —
|
||
not "this under-reports". The tool now names `%s`-style lazy logging
|
||
(`_LOG.error("… %s … %s", a, b)`, where the operands are *arguments*, so
|
||
no f-string census can ever see them), `.format()`/`%`/`+`/`.join()`
|
||
sentences, sentences that leave through a dict or dataclass, `yield`,
|
||
message attributes outside its own constant set, and everything outside
|
||
`app/`. Where the list is derived from a constant in the file, it is
|
||
**printed from that constant**, so the two cannot drift.
|
||
* **AND THE PROBE THAT GRADES THE MUTATIONS IS SUBJECT TO EVERY HABIT
|
||
ABOVE.** Round 4's render probe gave each interpolated operand a value by
|
||
**order of appearance** — so after a transposition the other operand took
|
||
the same number and **all thirteen mutant renders came back
|
||
byte-identical**, which reads exactly like *the edit changed nothing*.
|
||
Keying the value to the operand's **NAME** fixed it. That is habit 4, in
|
||
the measuring instrument: the fixture's own values were arranged so the
|
||
failure it was built to see could not change the answer. **Eleventh
|
||
instance of habit 4 in this file, and the second one in an instrument
|
||
rather than in a check.**
|
||
|
||
17. ⚠️ **TWO TERMS OF A CONJUNCTION THAT MOVE TOGETHER IN EVERY FIXTURE ARE
|
||
ONE TERM AS FAR AS THE CHECKS ARE CONCERNED - TWELFTH INSTANCE OF HABIT 4,
|
||
and the third found by an author's own battery rather than by review.**
|
||
SAFETY-37's freshness gate is `unchanged >= 35 s AND |standing| >= 250 W
|
||
AND |travel| >= 250 W`. Deleting the MIDDLE term left every check in the
|
||
first draft green, because in all of them a command that had *travelled*
|
||
250 W had also *ended up* above 250 W: the fixture never separated the two,
|
||
so one of them was decoration as far as the suite could tell. The case that
|
||
separates them is a site being brought to REST - the register reaches its
|
||
new value on the same read at which the OLD command is still standing, so
|
||
the travel term alone reads 1300 W of unfollowed command while the site
|
||
sits truthfully at +5 W. Ask of every conjunction: **is there a fixture in
|
||
which each term is the ONLY one that is false?** If not, the suite cannot
|
||
tell an AND from an OR.
|
||
|
||
18. ⚠️ **AN UNVERIFIED REVERT IS THE SAME DEFECT AS STALE BYTECODE, AND TWO
|
||
HARNESSES SHARING ONE BACKUP PATH IS THE WORST VERSION OF IT.** SAFETY-37's
|
||
sweep hit this twice, in ascending order of nastiness.
|
||
* A mutation's container run overran a two-minute command budget and the
|
||
wrapper was killed **between the edit and the `cp` that puts the file
|
||
back**, leaving `app/freshness.py` mutated on disk. Nothing announced it;
|
||
the next mutation's harness cheerfully fingerprinted and graded the mutant
|
||
*plus* a second mutation.
|
||
* Then two copies of the sweep were started - a scheduling mistake, not a
|
||
tool defect - and the harness kept its backup at the FIXED path
|
||
`/tmp/mut.bak`. One copy saved `app/main.py` there; the other restored it
|
||
**over `app/freshness.py`**. The result was a 238 KB `freshness.py` that
|
||
still parsed as Python, plus `main.py` left holding a live mutation. No
|
||
error, no warning, and the only reason it was noticed is that the tree
|
||
fingerprint did not match the recorded clean one.
|
||
|
||
Two rules, and the first is the cheap one:
|
||
* **Fingerprint after the revert, not only before the run.**
|
||
`tools/mutation_run.sh` prints the tree it is about to grade for exactly
|
||
this reason; the revert deserves the same treatment, and a mismatch is a
|
||
stop-everything result rather than a note.
|
||
* **A harness's scratch file must be unique per invocation** (`mktemp`), so
|
||
that two of it cannot silently overwrite each other's work. A fixed path
|
||
in a tool that edits source is a shared mutable global with a filename.
|
||
|
||
19. ⚠️ **A FIXTURE IN A SHAPE NOBODY PUBLISHES GRADES THE PARSER AGAINST THE
|
||
PARSER — FOURTEENTH INSTANCE OF HABIT 4, AND THE FIRST WHERE THE FIXTURE
|
||
AND THE CODE UNDER TEST SHARED A *SHAPE* RATHER THAN A NUMBER.** TARIFF-08.
|
||
`price.py`'s `_points_from_json()` read `doc["data"]` as a list of
|
||
`{start, price_eur_mwh}` objects. `api.energy-charts.info` publishes
|
||
parallel `unix_seconds` / `price` arrays and has no `data` key at all, so
|
||
`generic_dayahead` raised `SourceUnavailable` on **every live fetch it ever
|
||
made** — and the suite was green, because all three price suites built
|
||
their bodies with `json.dumps({"data": [...]})`. The fixture was derived
|
||
from the parser's guess, so no mutation of the parser and no amount of
|
||
coverage inside that loop could ever have reached the failure. §11.4 then
|
||
turned it into a *correct* silent degradation: parse error -> `no_data` ->
|
||
static profile -> arbitrage off, with nothing naming the cause.
|
||
|
||
Habit 2 with a network on the end of it: **the reference has to be data the
|
||
code under test did not shape.** For a parser that means BYTES, captured
|
||
from the source and committed unmodified, with the URL, the retrieval time
|
||
and a checksum beside them - `scenarios/energy-charts-be-price-*.json` and
|
||
its `-captures.json` provenance, re-derived by the replay so an edited
|
||
fixture stops being a capture. Two granularities, because the same endpoint
|
||
answers hourly before 2025-10-01 and quarter-hourly after it and a fixture
|
||
at one spacing cannot grade an assumption about the other.
|
||
|
||
⚠️ **AND THE SURVIVING SYNTHETIC FIXTURES ARE NAMED FOR WHAT THEY ARE.**
|
||
Three shipped plugins point at endpoints nobody has verified (TARIFF-07),
|
||
so no capture exists for them; their fixture builder is called
|
||
`synthetic_row_feed` at every call site, because the next reader's default
|
||
assumption will otherwise be that a green check means the feed parses.
|
||
|
||
⚠️ **ONE OF THIS CARD'S OWN NEW LINES WAS DECORATION AND THE BATTERY SAID
|
||
SO.** The columnar parser first rendered its instants with `brussels()`
|
||
before returning them. `assemble()` already renders every plugin's points,
|
||
so removing the parser's call changed nothing observable - **M4 came back a
|
||
survivor on a `:ro`, purged tree**, which is not the direction stale
|
||
bytecode fabricates. The line was removed rather than fenced with a check:
|
||
a second copy of a rule that already has an owner is indistinguishable from
|
||
a no-op, and the mutation that proves the rendering is real belongs on
|
||
`assemble()`, where it is now aimed.
|
||
|
||
And the mutation-table consequence: **a variant is not covered because a
|
||
near-miss of it is.** DEV-02's table listed the *actuator* variant of the
|
||
per-unit claim defect (`device`/`supervised` filed without `inv`), which the
|
||
static routing pin catches. The *bound* variant — right actuator, wrong
|
||
numbers — is a different defect with a different fix, and it was absent. Both
|
||
are in the table now.
|
||
|
||
## Where each surface stands
|
||
|
||
* **Local and dev-harness runs — the exposed one.** This is where FCST-03 hit
|
||
it. `goodwe_controller/__pycache__` and `app/__pycache__` are gitignored and
|
||
are sitting in most working trees right now. Use the harness.
|
||
* **CI — structurally immune, and it was before this card.** `.gitea/workflows/
|
||
checks.yml` bind mounts the checkout `-v "$WS:/w:ro"`, so the interpreter
|
||
cannot write a `__pycache__` into the tree under test at all; measured for
|
||
TEST-15, and it fails *silently* — no error, exit status 0. Each job also
|
||
`rm -rf`s its workspace and clones afresh, so nothing crosses between jobs on
|
||
the shared runner. `PYTHONDONTWRITEBYTECODE=1` is now set there too, as belt
|
||
and braces: the `:ro` is one character wide and the next person debugging a
|
||
write will be tempted by it.
|
||
* **The shipped add-on — not affected, and this is closed.**
|
||
`goodwe_controller/Dockerfile` has carried `PYTHONDONTWRITEBYTECODE=1` since
|
||
it was written. There is no deploy-side finding here; do not re-open it.
|
||
* **The infra repo's `sim/` harnesses — exposed, and NOT fixed by this card.**
|
||
`sim/replay_sign.py` and friends import the add-on out of
|
||
`$GOODWE_ADDON_REPO` and write `__pycache__` into *both* trees
|
||
(`goodwe/sim/__pycache__` exists today). SAFETY-02's headline validation is a
|
||
replay, not a suite, so a mutation graded through it carries the same
|
||
exposure. `tools/mutation_run.sh` takes the tree to purge as its first
|
||
argument precisely so that repo can call it, but the change belongs in that
|
||
repo and some of those harnesses want the rig. **Anyone mutating a module
|
||
graded by a replay must purge both trees by hand until then.**
|
||
|
||
## Survey — mutation claims of 2026-08-25 … 2026-09-01
|
||
|
||
Scoped **by the property, not by re-running everything**. A claim is at risk
|
||
only if a mutation could have preserved the file's byte count *and* been
|
||
reverted inside the same second, *and* the claim is of the kind stale bytecode
|
||
can fabricate (see "which way it lies"). Sources: `git log release/1.0` and the
|
||
commit and PR bodies.
|
||
|
||
| claim | verdict | why |
|
||
|---|---|---|
|
||
| **FCST-03** M9 `self.provisional`→`here.provisional` (dd40c56, 71cd784) | **re-verified** | this is the original false survivor. Re-run with the cache cleared, M9 is caught. The shipped source was correct throughout. |
|
||
| **FCST-03** X1, X6 degenerate stimuli (71cd784) | **not at risk** | run after the purge discipline landed in that branch. |
|
||
| **SAFETY-24** 11 author + 9 reviewer mutations (e709554, 4601682) | **not at risk** | the commit records them run under `python -B` with a purged `__pycache__` and real byte deltas. Its one survivor is an equivalent mutant, argued structurally rather than by a green run. |
|
||
| **SAFETY-26** tolerance sweep over `arbitrage.py` and `static_tariff.py`, 2 survivors (2b8b8cb) | **not at risk** | purge, printed `+5 B` delta, mutated line asserted present, tree re-read before each run. One survivor is explicitly recorded at `+0 B` — the risky delta — and was produced *under* the purge. **Both survivors are now closed by TEST-16** (`test_static_tariff.py` section 9), which also found that the two of them *masked each other* — each was individually non-live only because the other rung caught its stimulus, an arrangement single-mutation testing cannot see. Its own ledger is 7 mutations, 4 of them at `+0 B`, all red. |
|
||
| **OPT-04** mutations reported on PR #22 (bdd7f18) | **not at risk** | `test_arbitrage.py`'s header records the exact command: purge, then `python3 -B`. |
|
||
| **TEST-17** `checks()` dropping `when` (504e8da) | **not at risk** | a *red* claim, and dropping an argument changes the file length. Neither half of the collision is available. |
|
||
| **SAFETY-07** wiring mutations B1/B2/B3 and `test_peakguard.py` parts 7 and 10 (2d570d8, e7c95bf) | **not at risk** | those mutations are **executed in-process**, monkeypatched onto the imported module. No source file is edited, so no `.pyc` is consulted. |
|
||
| every **"all red, no survivors"** claim in the window — TARIFF-02 (24/0), TARIFF-03 (11), MAINT-04 (10), SAFETY-18 (6+4), SAFETY-03 (6), TEL-04 (10), TEL-05 (6) | **not at risk** | direction. Stale bytecode turns a red into a green survivor, never the reverse. A red is evidence that *something* changed under the check. |
|
||
| **TEST-06**'s deleted assertion, "no new notification while parked at 0 W" (c34b5a2) | **not at risk / superseded** | it was recorded as undecidable and then *falsified* — `test_freeze.py` records the one-line mutation that takes the file red 9 times. The stronger later claim is a red one. |
|
||
| **TARIFF-01** reviewer survivor: `record_block()` ignoring its `policy` argument (0dbd485) | **re-verified — still a survivor** | re-run here through `tools/mutation_run.sh` against the 15 suites that were on the record, all green with `billed_w(block, policy)` mutated to `billed_w(block, POLICY_NET)`. The recorded diagnosis holds: every block reaching the store is single-phase, where NET, SUM and MAX collapse. **Now CLOSED by TEST-13** (`test_tariff.py` section 7), which round-trips section 7's own unbalanced `(3000, 500, -200) W` block THROUGH the store under each policy - three distinct banked figures, so a hardcoded policy takes at least two of them red. Third instance of "a stimulus evaluated where the quantity it grades is degenerate", after WITNESS-02's span normalisation and TARIFF-01's own DST window. |
|
||
| **SAFETY-25** 8 mutations against `test_peakguard.py` part 10, then 9 more after review, all red | **not at risk** | each run against a tree copied fresh with `__pycache__` excluded and mounted `:ro`, so no `.pyc` existed to go stale - the same structural immunity recorded for CI above. Byte deltas recorded on the PR; none is `+0 B`. All are **reds**, which is the direction this defect cannot fabricate. ⚠️ The second round exists because the first round's mutations were all aimed at DISARMING the guard, and the claim part 10 actually made was about the guard being FIXED - the reviewer built the fix and the suite stayed green. Bytecode was never the issue there; an unaimed battery was. |
|
||
| **SAFETY-03** reviewer survivor: `abs()` dropped from `clamp_to_rating`'s limit (6aabb83) | **re-verified — still a survivor** | re-run the same way, all 15 green. Also as recorded: unreachable under the `int(100, 5000)` schema. |
|
||
| **MAINT-05 round 3**, `migrate_maint_keys()` marker gate `if rep.ok:` → `if rep.ok or rep.kept:` (`+12 B`) | **was a survivor — now CLOSED** | measured against head `65cddf3` under the purge, tree mounted `:ro`, `19/19 PASS`. It stamps the schema marker on a migration that LOST state, defeating the older `lost` guard as well as the new `misdirected` one, because every check that reached the marker did so on a store whose `kept` list was empty. Closed by `test_maintenance.py` **9d**; the mutation is now red on three named checks. |
|
||
| **MAINT-05 round 3**, `migrate_unit_keys()` marker gate `if rep.ok:` → `if rep.ok or rep.kept:` (`+12 B`) | **found here — SAFETY-02's half, now CLOSED** | the SAME mutation one function up, on the LATCH migration rather than the history one. Measured a survivor against this branch under the purge, `:ro`, `19/19 PASS`, then closed by `test_inverter.py` part 10's new fixture — a store with `unit/inverter-a/sign_reason` already present (so `kept` is non-empty) whose `sign_tripped: true` write is dropped. Shipped: marker withheld. Mutant: marker stamped on a **disarmed latch**, and the next start skips the migration that would have re-armed it. |
|
||
| **TARIFF-01**'s other three survivors, **SAFETY-03**'s `describe()` survivor, **TARIFF-02**'s M3/M5/M7, **SAFETY-18**'s three | **still unverified** | recorded before the purge discipline, with no byte delta on the record, so a `+0 B` same-second revert cannot be excluded. All are survivors, which is the direction this defect fabricates. TARIFF-02's and SAFETY-18's were **closed by later commits whose claims are reds**, so what is unverified is the *diagnosis*, not the current state of the code. |
|
||
|
||
**Nothing here says a past result is wrong.** The two survivors re-run above
|
||
both stood. What changed is that they are now on the record as measured rather
|
||
than assumed, and the rest are labelled honestly. The still-unverified rows are
|
||
cheap to close — one harness run each — and the right moment is when somebody
|
||
next touches that module, not in a re-run campaign nobody will read.
|