checks / unit (pull_request) Successful in 7m59s
0.11.0 shipped `pv_entity` as a single string. The reference site's owner reported within minutes of it going live that they have several PV inverters and could configure one - which is ordinary for a multi-string roof, not a property of one house. `pv_entities` is the roster and the row carries the SITE total. ⚠️ THE HARD PART IS ABSENCE, NOT THE LIST. Three configured, one `unavailable`, and the sum of the other two is a SMALLER NUMBER INDISTINGUISHABLE FROM A REAL ONE. Read back by OPT-10 that says "there was less PV than there was" - the "no irradiance" conclusion the channel exists to TEST. So `pv.total()` answers `None` the moment any configured entity is absent: the topic carries HA's own `None`, the entity goes to `unknown`, the recorder records a GAP. The alternative (the partial sum beside a contributor count) was implemented and run as mutation N1b; it loses because the MQTT row is read COLD a week later by an attribution that would believe it, while the qualifying count would live in a second entity with its own history. The partial information is not thrown away - the STATUS PAGE names every entity that reported, with its watts, and every one that did not, with its reason, because that surface is read LIVE by a human who wants the diagnostic. ⚠️ AND THE MIRROR: a member honestly reporting 0 W is SUMMED. The test is `is None`, never falsiness - a roster that blanked the site on a zero would report an absence on every clear evening. THE SURFACE IS MEASURED, NOT PREFERRED. `pv_entity` could not become a list: tools/supervisor_shape_probe.py §G shows Supervisor 2026.08.0 answering an 0.11.0 install's stored string with `Invalid list for option 'pv_entity'`, and deepmerge's list strategy is `override` so a `[]` default does not rescue it - the live add-on would not start. `match(.*)?` does accept both shapes and is rejected for a different, measured reason: it validates nothing (a string, a list and the integer 417 all pass) and renders as a plain text box. So `pv_entities` joins `pv_entity` and REPLACES it when set, CFG-04's rule for `inverters:`, with a status caution saying which key won - a union would double-count the inverter the old key still names. STILL NOT A CONTROL INPUT. tools/tel11_pv_trace.py was extended rather than replaced and now grades five claims over three document sets. The control hash is UNMOVED - 846a10a5b89e… on `unset` and on `one`, on both trees - and the roster control document is byte-identical between the two trees. 2/5 is a claim TEL-11 could not make: with 0.11.0 as the baseline, "one entity behaves exactly as 0.11.0 does" is a real cross-tree identity over control AND telemetry. ⚠️ TEL-11's OWN `--compare` REFUSAL HAD TO CHANGE, and that is MUTATION.md habit 24: a discriminator defined as "the baseline lacks this feature" dies on the release that ships the feature, and dies as a false refusal. Twelve mutations, each on a fresh purged `cp -a`, no survivors. N5 is +0 B (`sum` -> `max`, the same three characters) and was run on a fresh copy with `cmp` confirming the edit on disk. The fixture is three distinct NON-ZERO contributions in two units with absence driven at EVERY position - a member contributing 0 W could go absent without moving the sum of the rest, which is habit 23 with a coinciding value in the arithmetic rather than in the plant. 31/31 in the CI image. Not deployable as merged: config.yaml stays at 0.11.0, so predeploy_check.py reports a NO-OP until PKG cuts a version. A CHANGELOG entry is wanted and deliberately not written - 11b forbids `## Unreleased` and the bump is PKG's call; DECISIONS/TEL-12-several-pv-inverters.md is the material it needs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
1136 lines
77 KiB
Markdown
1136 lines
77 KiB
Markdown
# Mutation runs in this repository — and the one way they lie
|
||
|
||
Nearly every safety claim in this add-on was established the same way: change
|
||
the shipped module, run the suites, watch one go **red**, put the module back,
|
||
watch it go **green**. There is no coverage tool here and there is not meant to
|
||
be — the check that matters is whether a *check* notices when the behaviour it
|
||
names is removed.
|
||
|
||
That method has exactly one silent failure mode, and TEST-15 exists because we
|
||
walked into it.
|
||
|
||
## The defect
|
||
|
||
FCST-03's developer reverted `self.provisional` to `here.provisional`, re-ran,
|
||
and reported the mutation as a **survivor**. It was not. CPython decides a
|
||
cached `.pyc` is still valid from two numbers:
|
||
|
||
* the source file's **size in bytes**, and
|
||
* the source file's **mtime truncated to whole seconds**.
|
||
|
||
`self` and `here` are the same length, and the revert happened in the same
|
||
second as the edit. Neither number moved, so the interpreter loaded the
|
||
bytecode it already had. Confirmed at the time by disassembly: **the source read
|
||
`here`, the running code loaded `self`.** The suite had graded code that was not
|
||
there — no error, no warning, exit status 0.
|
||
|
||
The same collision, in the other direction, can leave a **red** standing on
|
||
correct source after a revert that never took.
|
||
|
||
### Which way it lies, and why that scopes everything
|
||
|
||
A mutation that never took effect leaves the suite **green**, and a green run of
|
||
a mutated tree is reported as a survivor. So stale bytecode **manufactures
|
||
survivors**; it cannot manufacture a red. The false-red case exists but
|
||
announces itself, because a developer chases a failure they cannot explain.
|
||
|
||
Two consequences, and both are used below:
|
||
|
||
* a recorded **"all red, no survivors"** result is not at risk from this defect;
|
||
* a recorded **survivor** — and a recorded *zero*, such as a coverage-audit
|
||
count of 0 — is exactly what this defect produces.
|
||
|
||
`test_control.py`'s coverage-audit table says the same thing in its own terms:
|
||
the error is one-directional, so every figure there is a lower bound at worst.
|
||
|
||
## The rule
|
||
|
||
**Purge `__pycache__` before every mutation run.** Not afterwards, not usually,
|
||
not when the edit looks big enough.
|
||
|
||
```sh
|
||
cd goodwe_controller
|
||
tools/mutation_run.sh # every test_*.py in this directory
|
||
tools/mutation_run.sh . test_tariff.py # or just the ones you are watching
|
||
tools/mutation_run.sh "$GOODWE_ADDON_REPO/goodwe_controller" # from elsewhere
|
||
```
|
||
|
||
The harness purges, prints a `sha256`/byte-count fingerprint of every `app/*.py`
|
||
it is about to grade, runs each suite under `python3 -B`, and fails if a cache
|
||
appeared during the run. Diff two fingerprints and the file you mutated is the
|
||
line that changed — which is also how a mutation report proves *which source*
|
||
produced it.
|
||
|
||
### What does **not** fix this
|
||
|
||
| | writes bytecode | reads a stale `.pyc` |
|
||
|---|---|---|
|
||
| `python3 -B` | no | **yes** |
|
||
| `PYTHONDONTWRITEBYTECODE=1` | no | **yes** |
|
||
| purging `__pycache__` | (unchanged) | **no — nothing to read** |
|
||
|
||
Both flags are *write-side only*. A mutation run always has a cache already,
|
||
because the previous run wrote it, so a harness that only adds `-B` closes
|
||
nothing. This is measured, not read off documentation: `test_bytecode.py`
|
||
part 2 arranges the collision and shows both flags still running the old code.
|
||
|
||
`-B` is still worth passing — it stops **this** run leaving a cache for the
|
||
**next** edit to inherit, which is the one nobody is thinking about three
|
||
mutations into a batch. Purge for this run, `-B` for the next one.
|
||
|
||
### And not a `sleep`
|
||
|
||
One-second mtime resolution is the *mechanism*, not the enemy. Waiting out the
|
||
clock would slow every run down, and would still only cover the edit somebody
|
||
remembered to slow down. Stop trusting mtime instead.
|
||
|
||
## Reporting a mutation
|
||
|
||
State the things a reviewer cannot reconstruct afterwards:
|
||
|
||
1. **the byte delta** of the mutated file (`+5 B`, `+0 B`). `+0 B` is the risky
|
||
one — say so;
|
||
2. **that the mutated line was on disk** when the run started (the harness
|
||
fingerprint, or a `grep` of the line);
|
||
3. **which suite went red, and on which named check** — "the suite is red" and
|
||
"the check that watches this is red" are different claims;
|
||
4. for a **survivor**, that the run was purged. An unpurged survivor is not a
|
||
result.
|
||
|
||
## The other thing a mutation sweep cannot see: masking (TEST-16)
|
||
|
||
Stale bytecode is not the only way a survivor lies. **Two guards can mask each
|
||
other**, and a sweep that mutates one line at a time is structurally blind to it.
|
||
|
||
TEST-16 is the worked example. `static_tariff.arbitrage()` refuses a
|
||
one-firm-hour series at `len(firm) < 2`, and refuses a flat series at
|
||
`spread == 0.0`. A single firm hour *also* has a spread of zero. So:
|
||
|
||
* weaken the firm-pair rung alone → the flat rung refuses the stimulus anyway;
|
||
* weaken the flat rung alone → the tiny-spread stimulus is still refused, just
|
||
for the wrong reason;
|
||
* and the suite, grading `kind == "refused"`, reports **nothing either way**.
|
||
|
||
Both rungs could then be weakened *together* and still nothing would report —
|
||
which is precisely the state in which the next edit to either one opens a real
|
||
hole. It survived four rounds of review on TARIFF-03 and FCST-03.
|
||
|
||
Two rules come out of it, and both are cheap:
|
||
|
||
1. **Grade the reason, not just the verdict.** "Refused" and "refused for the
|
||
right reason" are different claims, and a masked rung is the case where the
|
||
first is true and the second is false. A refusal sentence is what the
|
||
operator reads; it is also the only observable that moves when one rung
|
||
quietly absorbs another's work.
|
||
2. **When two rungs can answer the same stimulus, mutate them together.** One
|
||
extra run per suspected pair. If a sweep finds two survivors in the same
|
||
function, that pair is the first thing to try — TEST-16's two survivors were
|
||
adjacent lines in one function and neither was individually live.
|
||
|
||
⚠️ **AND FCST-05 WALKED INTO IT AGAIN, IN ITS OWN NEW CHECK, WHICH IS WHY THE
|
||
RULE IS “mutate every check you add”.** `price_history.py` reports a malformed
|
||
point row from **two** places into **one** field: `_usable_days()` records what
|
||
it FOUND when the file was opened, and `_remember()` records what it DROPPED on
|
||
the next refresh. Both land in `last_error`, and `words` prints `last_error`.
|
||
The first draft of the new check asked `"malformed point row" in words` — which
|
||
whichever rung is still standing satisfies — so deleting `_remember()`'s note
|
||
entirely was a **−1 B SURVIVOR of the whole suite**, on the half of the fix that
|
||
tells an operator the cache **self-healed**. It is graded on the TENSE now
|
||
(`… in <day>` at open, `… dropped from <day>` after the refresh), the two are
|
||
asserted to DIFFER, and the paired mutation — both notes deleted, −91 B — is
|
||
red on 15 named checks. Same shape as TEST-16 and the same fix: **grade the
|
||
reason, not the verdict**, and when two rungs answer one stimulus, mutate them
|
||
together.
|
||
|
||
⚠️ **AND FCST-06 FOUND THE COUNT ITSELF LYING — LOW, NOT SILENT.** A mutation
|
||
whose red is a **traceback** does not report one red; it reports however many
|
||
checks ran before the traceback, and grades **none** of the ones after it. The
|
||
new section drove M4 (`self._days = before` → `self._days = {}`) and the run
|
||
printed **1 FAIL**. The true figure is **3**: an f-string in the next check read
|
||
`len(read.points)`, and `.points` on a `no_data` **raises `Absence` by design**,
|
||
so the harness died mid-section with the remaining checks ungraded. That is
|
||
`why()`'s and `derived()`'s rule — *"it answered nothing" and "it broke on the
|
||
way to answering" must not grade the same* — arriving in the **printed operand**
|
||
of a check rather than in its condition, which is the half that is easy to miss
|
||
because it looks like formatting. Two things follow:
|
||
|
||
* **a mutation report's red COUNT is only a lower bound until the run is seen to
|
||
have reached the end of its section.** Read the output, not the count;
|
||
* **the total accessor has to cover what a check PRINTS, not only what it
|
||
asserts.** `test_price_history.py` 5d added `npts()` beside `why()` for
|
||
exactly this, and re-driving M4 against it moved 1 FAIL to 3.
|
||
|
||
Worse, had the traceback landed one check earlier the mutation would have
|
||
produced **no named red at all** — a result that reads like a broken harness and
|
||
gets retried rather than read.
|
||
|
||
## The third way a check lies: its DATA cannot reach the failure it names
|
||
|
||
Stale bytecode manufactures survivors from a mutation that never ran. Masking
|
||
manufactures them from a second guard doing the first one's work. This one
|
||
manufactures them from a **fixture**: the check runs, the assertion is
|
||
evaluated, the sentence above it is true — and the numbers it compares are
|
||
arranged so that the failure it describes could not have changed the answer.
|
||
|
||
It is not hypothetical and it is not rare. TWELVE instances in two weeks; four
|
||
were found by review, two - CFG-04's fixture pair - by an author's own mutation
|
||
battery, which is the cheaper place, one by review again, in the one
|
||
place nobody had thought to point the rule at: not a check in the suite, but a
|
||
COMMITTED EVIDENCE ARTIFACT four documents cite - the eighth INSIDE THE FIX
|
||
WRITTEN TO CLOSE THE SEVENTH, the ninth inside the SITE HALF of the card
|
||
that names the rule in its own text - and the TENTH INSIDE THE FIX WRITTEN TO
|
||
CLOSE THE NINTH, for the second time in this list. See SAFETY-38 below: the
|
||
check that closed it asserted a NUMBER and not the number's POSITION.
|
||
|
||
⚠️ AND THE ELEVENTH IS IN A MEASURING INSTRUMENT RATHER THAN IN A CHECK, which
|
||
is where the seventh was too. SAFETY-38 round 4's render probe - the thing that
|
||
decides whether a mutation changed the SENTENCE and not merely the bytes -
|
||
numbered each interpolated operand BY ORDER OF APPEARANCE. After a transposition
|
||
the other operand takes the same number, so all thirteen mutant renders came back
|
||
BYTE-IDENTICAL to the clean tree: the probe compared the swap against itself, and
|
||
its output read exactly like "the edit is inert". Keying the value to the
|
||
operand's NAME fixed it. Habit 16.
|
||
|
||
⚠️ **AND IT HAS NOW APPEARED IN SIX CONSECUTIVE CARDS.** The seventh instance,
|
||
MAINT-05's, is the one that says why: the defect is not carelessness about
|
||
fixtures, it is that **a check and the thing it checks were built from the same
|
||
constant**. Ask of every difference-claim not "is this assertion true" but
|
||
"where did each side of this comparison come from" - and if the answer is the
|
||
same place twice, the check is decoration.
|
||
|
||
* **ARB-03** shipped a call-site scan asserting that every `claims.append` in
|
||
`cycle()` routes through `actuator_of()`. Its own parse **saw 3 of the 8
|
||
appends**; the other five were invisible to it and would have passed however
|
||
they were routed.
|
||
* **ALLOC-01** shipped a replay generator to hunt for envelope escapes. It found
|
||
**0**. A widened generator over the same allocator found **39,991**.
|
||
* **DEV-02** shipped `"each unit's supervised bound is ITS OWN max_w — give
|
||
inverter B a different rating and the claim filed on inverter B moves with
|
||
it"`, comparing inverter B's bound against `_b2.max_w`. The fixture defaulted
|
||
B's rating to A's, so it compared `(-5000, 5000)` against `(-5000, 5000)` —
|
||
**a number against itself**. Filing both per-unit bounds from
|
||
`self.inverters[0]` left **all 19 files green**.
|
||
* **CFG-04** wrote *"an entry that omits `soc_entity` has NO SoC sensor and does
|
||
not borrow the flat one"* against a fixture whose **flat `soc_entity` was
|
||
blank**. Inherited and not-inherited were both `""`, so the mutation that
|
||
makes `build_roster()` fall back to the flat key survived. The fixture now
|
||
sets a real entity there, and the check asserts that the flat value is present
|
||
as well as that the entry's is empty.
|
||
* **CFG-04** also wrote *"main.py runs the per-unit range check
|
||
UNCONDITIONALLY"* as a **substring test over `app/main.py`'s source**. The
|
||
mutation that defeats it puts the call back inside `if entity:` — one level of
|
||
indentation — and the substring is still there. That is habit 4 below in a new
|
||
costume: the check asserted against the TEXT of the thing under test rather
|
||
than against its behaviour. It now drives the shipped `run_control()` on a
|
||
two-inverter config, **under a deadline**, because the mutation's real symptom
|
||
is that the controller enters its loop and never returns.
|
||
* **MAINT-05** shipped `Controller.__init__` calling
|
||
`migrate_maint_keys(store, self.inverters[0].name, ...)` under a comment
|
||
saying *"it targets `self.inverters[0].name` and NOT `UNIT_A` ... writing
|
||
`UNIT_A` here would deposit an install's whole maintenance history under a
|
||
name that is not on the roster at all"*. `inverter.UNIT_A` **is** the string
|
||
`"inverter-a"`, and every two-pack fixture in `test_maintenance.py` named its
|
||
first roster entry `inverter-a` — so the two expressions were the same value
|
||
in every check that existed, and swapping them left **all 19 files green**
|
||
while losing a real install's whole maintenance history. Two-pack fixtures are
|
||
now `["garage", "shed"]`, each behind a precondition check (habit 1) that the
|
||
first name is not `UNIT_A`.
|
||
|
||
⚠️ **AND THE POSTCONDITION HAD THE SAME SHAPE AS THE FIXTURE.**
|
||
`store._verify_moves()` read back the keys it had just written, under the
|
||
namespace it had just been handed — so it verified clean, and the caller
|
||
stamped the schema marker, on a migration nothing would ever read. *A check
|
||
that reads back the namespace it was handed cannot detect being handed the
|
||
wrong one*: it compared a thing against itself, exactly as DEV-02's fixture
|
||
compared `(-5000, 5000)` against `(-5000, 5000)`. The habit-2 fix is the same
|
||
one — grade against something the thing under test did not supply, here the
|
||
ROSTER the machines read.
|
||
|
||
⚠️ **AND THE FIX ITSELF CARRIED THE DEFECT, WHICH IS THE SIXTH CARD IN A ROW
|
||
AND THE FIRST TIME IT HAS APPEARED *INSIDE* A REMEDY.** The fix round's
|
||
recovery check was `migrate_maint_keys(_m5, ROSTER[0], ROSTER)` — the
|
||
**corrected** name. It measures recovery along the one path where the operator
|
||
has *already fixed the bug*, and nothing in the 19 files called
|
||
`migrate_maint_keys()` twice with the **same wrong** `unit`. So the refusal
|
||
the card was rejected twice to obtain lasted **exactly one start**: measured
|
||
at Controller level over one persisted store, roster `["garage", "shed"]` —
|
||
construction 1 `ok=False misdirected=4 moved=4 kept=0 marker=<absent>`,
|
||
construction 2 `ok=True misdirected=0 moved=0 kept=4 marker=1`. On start 2
|
||
rule 1 routes the wrongly-written keys to `rep.kept`, `rep.moved` is empty,
|
||
the roster loop iterates over nothing, and the marker is stamped on a
|
||
migration that reached no readable key — after which the early return skips it
|
||
forever, *including after the call site is fixed*. The check was true and
|
||
passed for the wrong reason. Habit **9** below is what falls out of it.
|
||
|
||
* ⚠️ **ARCH-02's OWN REHEARSAL HARNESS, CAUGHT BY ITS OWN MUTATION BATTERY —
|
||
the THIRTEENTH instance, and the third in an INSTRUMENT rather than in a
|
||
check.** `tools/rehearse_roster.py` reports orphaned Home Assistant entities
|
||
by composing the ids the add-on announces **before** the roster switch and
|
||
**after** it, and diffing. Both sides are composed by the same build. So the
|
||
two mutations that change the id composition itself — renaming the MQTT device
|
||
(`"GoodWe"` -> `"GoodWe Controller"`), renaming one site sensor (`"Grid power"`
|
||
-> `"Grid power W"`) — moved **every id on both sides equally**, and the
|
||
harness reported *"no orphans"* about a change that orphans the entire live
|
||
entity list. It read exactly like a clean transition. The first draft's
|
||
M5/M6 came back `exit 0` and `exit 2`, and neither was the failure they name.
|
||
|
||
The fix is habit 2 with a filename: **the reference has to be data that does
|
||
not move with the code.** `--live-entities` takes a list of the ids the
|
||
install actually has, `tools/rehearsal/live-entities-recorded.txt` is the
|
||
recorded stand-in that keeps it live in CI, and a build that announces an id
|
||
the list does not carry is now a `COMPOSITION DRIFT` finding rather than a
|
||
silence. ⚠️ **The anchor moved for the same reason**: it had been
|
||
`compose(mqtt.DEVICE["name"], p1["name"]) == "sensor.goodwe_p1_sample_age"`,
|
||
which any renaming of the device satisfies by moving both operands, and it is
|
||
now composed from two recorded **literals**.
|
||
|
||
Ask of an instrument what habit 4 asks of a check: *where did each side of
|
||
this comparison come from* — and if a mutation of the subject moves both
|
||
sides, the instrument is decoration.
|
||
|
||
⚠️ **AND REVIEW FOUND TWO MORE IN THE SAME CARD, ONE OF THEM INSIDE THE FIX
|
||
FOR THE OTHER — the FOURTEENTH and FIFTEENTH.** Both are the pin, not the
|
||
tool, and both are the same sentence in a new costume: *a check whose data
|
||
cannot reach the failure it names.*
|
||
|
||
* **Fourteenth — a COUNT cannot tell one red leg from two.** The rehearsal
|
||
reports two independent routes to the disarmed latch (`SAME-STEP` and
|
||
`STAGED`) and prints every finding **twice per red leg**: once in that
|
||
leg's own block and once in the closing summary. The check named *"NOT THE
|
||
SAME-STEP PATH ALONE"* was written `out.count("LATCH CAME UP DISARMED")
|
||
>= 2`, so **one** red leg always satisfied it. Measured on three builds:
|
||
shipped code (both legs red) → 4; `migrate_unit_keys()` handed the roster
|
||
name (SAME-STEP fixed) → 2, **passes**; a guard-side fix (both fixed) → 0.
|
||
The check reported `ok` on the build whose only remaining failure *is* the
|
||
path it says is not the only one. The fix is to parse the legs and require
|
||
the finding in **each**, which is habit 2 again: grade the thing, not a
|
||
total over the thing.
|
||
|
||
* **Fifteenth — and it is INSIDE THAT FIX, which is the third time this file
|
||
has had to write that sentence.** The first leg-splitter opened on a
|
||
`LEG <id>` marker and never closed, so every line after the *last* marker
|
||
was attributed to that leg — including the closing summary, which repeats
|
||
every finding from every leg. The last leg therefore read red whenever
|
||
**any** leg was red, and the new per-leg check was the old count with extra
|
||
steps. It was caught by building the one fixture that separates them: a
|
||
tree in which `SAME-STEP` is genuinely clean and `STAGED` is not. The tool
|
||
now prints `END LEG <id>` and the splitter closes on it.
|
||
|
||
The rule that falls out, and it is the cheap one: **a per-thing check that
|
||
has never seen the things DISAGREE is a total.** Part 5b exists to make
|
||
them disagree on every run.
|
||
|
||
⚠️ **AND ONE THAT HAS ONLY SEEN THEM DISAGREE IN ONE ARRANGEMENT IS A TOTAL
|
||
OVER THAT ARRANGEMENT.** Review found this by attacking the fix itself. Part
|
||
5b does make the legs disagree, on every run - and was still blind to a build
|
||
that merely REORDERS them, because it had only ever seen the clean leg
|
||
printed LAST. Three edits, two of them ordinary refactors (reorder the report
|
||
legs, reword a summary line), produced `all checks passed` from a splitter
|
||
that closed ZERO legs.
|
||
|
||
A fixture cannot enumerate the arrangements, so the STRUCTURE is asserted
|
||
instead: a leg may not open while one is open, and every leg that opens must
|
||
be explicitly closed. That fires on the FIRST edit of all three attacks,
|
||
whatever the order or the wording. **When a fixture cannot enumerate the
|
||
arrangements, assert the invariant they share.**
|
||
|
||
* **CFG-04's `tools/supervisor_shape_probe.py` §B** — and this one is the rule
|
||
reaching outside the suite. The probe is the committed evidence that
|
||
`config.yaml`'s `inverters: []` default is load-bearing, and four documents
|
||
quoted it as answering *"Missing option 'inverters' in root"*. §B validated
|
||
the reconstructed LIVE options blob, which is missing many required keys
|
||
unrelated to this card, so its refusal named whichever of THOSE the validator
|
||
reached first — `meter_freeze_s`, `p1_phase_export_entities` and
|
||
`sign_disagree_w` on three different runs. The quoted string was never
|
||
produced by the instrument. And the assertion was a bare **`not ok`**: delete
|
||
`inverters: []` from `config.yaml` and §B went on printing **PASS**, blind to
|
||
the exact regression it was cited for. §B is now an isolated counterfactual —
|
||
the shipped defaults, and the same defaults minus that one key — asserting on
|
||
the **key name in the refusal**; the mis-cited blob survives, relabelled, as
|
||
§B2. Re-run with the default deleted it prints `2 CASE(S) FAILED` and exits 1.
|
||
|
||
⚠️ **A committed transcript is a check, and it decays like one.** Nothing in
|
||
this document had been pointed at `tools/`, because the artifacts there are
|
||
filed as *measurements* rather than as *checks* — and a measurement whose
|
||
assertion cannot fail is exactly the thing this section is about. Two habits
|
||
follow, and both are habit 5 and 6 below.
|
||
|
||
* **SAFETY-38** — `main.py`'s
|
||
`site_standing = float(sum(standing.values()))`, the first argument of
|
||
SAFETY-30's latching site sign detector *and* SAFETY-18's `looks_frozen()`
|
||
input. `test_watchdog.py`'s `_sign_site` is the suite's **only** N≥2 site
|
||
fixture, and it runs shares of about **−2500 W each** — every one
|
||
individually over the 320 W gate it is configured with. On numbers like those
|
||
`sum`, the mean and `max(key=abs)` **all agree**, so the reduction could be
|
||
swapped for either of the other two and the site guard went **blind with the
|
||
whole suite green** — still commanding −500 W of charge into a +2500 W import at
|
||
t = 595 s with `following=True`. At a roster of **one** it is worse and it is
|
||
not a fixture defect at all: `sum == mean == max == the one share`
|
||
*identically*, so every such mutation is genuinely **equivalent** there and no
|
||
N=1 fixture could ever separate them. The fixture that separates them is the
|
||
one the detector exists for — **per-unit shares EACH BELOW the gate whose SUM
|
||
clears it** — and it is now `test_watchdog.py` part 9, at asymmetric ratings
|
||
so that all four candidate reductions give four **distinct** answers.
|
||
|
||
⚠️ **AND THE CARD'S OWN SEARCH FOR ITS NEXT SURVIVOR FOUND ONE, WHICH IS THE
|
||
POINT OF ASKING.** Having built the fixture, the question *what mutation would
|
||
this NOT see?* produced `float(self.target)` — the arbiter's site target
|
||
instead of the sum of what stands on the wire — at **−11 B**, green across the
|
||
whole suite, because every fixture in the repository was built where the two
|
||
are the same number. They separate only when a device's reported range is
|
||
narrower than its `max_w`, since the `device` claim is filed **per actuator**
|
||
and never bounds the site target. Closed in the same commit, by a
|
||
counterfactual that asks for −500 W while −250 W stands.
|
||
|
||
⚠️ **AND THEN THE FIX ITSELF WAS THE TENTH INSTANCE — A SUBSTRING WHERE THE
|
||
FAILURE WAS POSITIONAL.** The check written to pin the banner asserted
|
||
`f"{sum(...):.0f} W" in reason`: that **"−500 W" appears somewhere** in the
|
||
sentence. Swapping the detector's first two arguments
|
||
(`evaluate(site_standing, self.grid, …)` → `evaluate(self.grid, site_standing, …)`)
|
||
is a **ZERO-BYTE** edit that leaves both numbers in the string and moves each
|
||
into the other's slot — so the substring is still found, **in the meter
|
||
slot**, and the whole suite stayed green while the latching site-stopping
|
||
banner told the operator which instrument read what **exactly backwards**.
|
||
The fix is to assert the **ORDERED PAIR**,
|
||
`f"commanded {sum:+.0f} W against a meter reading {grid:+.0f} W"`.
|
||
|
||
⚠️ **AND THE ELEVENTH WAS THE SAME DEFECT ONE CALL SITE OVER — THE ONE A
|
||
SHIPPED INSTALL CAN ACTUALLY SEE.** Round 2 pinned the operand order of
|
||
`SignWatchdog.evaluate()` at `main.py:1620`. That call site is the **site**
|
||
detector, and `test_watchdog.py` part 9c asserts it is **never armed at a
|
||
roster of one** — which is every install this add-on has. The **per-unit**
|
||
call at `main.py:1560` was left unpinned, and swapping its first two
|
||
arguments is the same **ZERO-BYTE** edit: `disagrees()` is symmetric in them
|
||
(`commanded_w * against_w < 0` with an `abs()` gate on both), so **no trip
|
||
decision can change** and the full 21-file suite stays green — measured. What
|
||
moves is five banner lines, including the one persisted to `sign_reason` and
|
||
re-read after a restart: *"commanded −1000 W against its own measured power
|
||
+1000 W"*, operands inverted, on the only sign-disagreement sentence a
|
||
one-inverter install can ever print. Closed in round 3 by the ordered pair on
|
||
**both** per-unit banners — the N≥2 `its own measured power` one and the N=1
|
||
`a meter reading` one — red on four named checks, green in the other twenty
|
||
files. **Habit 15 is what falls out of finding the same shape twice in the
|
||
same card.**
|
||
|
||
The general habits are **14** and **15** below.
|
||
|
||
* ⚠️ **SAFETY-47 IS THE TWELFTH, AND IT IS THE SECOND INSIDE A COMMITTED
|
||
EVIDENCE ARTIFACT.** `DECISIONS/SAFETY-10-shadow-mode.md` §6 tabulated a live
|
||
run and a shadow run agreeing on `arb_power_w`, on the reason **verbatim**
|
||
and on the arbiter's winner, and concluded that shadow mode reaches the same
|
||
decision. **Two independent reasons it could not have gone red, and closing
|
||
either one alone closes nothing:**
|
||
* its fixture is `peak_cap_w: 15000` against a 350 W house, so the capacity
|
||
guard never engages and the thing whose invariance is being claimed —
|
||
SAFETY-45's sizing of the buy leg from the billed-quarter meter — is in the
|
||
run and **inert**. ⚠️ **The harness's own comment says so**: *"at this
|
||
scenario's `peak_cap_w: 15000` it is not binding anyway"*. The rule was
|
||
written down beside the check it disabled and nobody read it as a
|
||
disclosure;
|
||
* and every identity row reads `_sa[0]`/`_la[0]` — **the first charging
|
||
sample**. At `elapsed = 0` the projection **is** the held level, so the two
|
||
runs agree **to the float even at a cap that does bind**. Measured:
|
||
widening the fixture alone — the obvious fix — leaves that comparison
|
||
passing.
|
||
|
||
⚠️ **THE SECOND REASON IS A NEW SHAPE FOR THIS LIST: AN INDEX, NOT A
|
||
FIXTURE.** Ask of every identity claim not only *"could this data differ"*
|
||
but *"**at the point I sampled it**, could it differ yet"*. A comparison
|
||
taken at `t = 0` between two runs that diverge **because one of them acts**
|
||
is a comparison of a plant with itself, and no amount of widening the fixture
|
||
changes that. The replacement, `test_arbitrage.py` part 3d, grades the whole
|
||
run — 716 of 720 charging cycles differ — and grades the `[0]` agreement **as
|
||
an expectation**, so nobody rebuilds a check on it believing it covers
|
||
something.
|
||
|
||
⚠️ **AND THE FIX'S OWN FIRST DRAFT CARRIED A THIRTEENTH.** The new part's
|
||
action-log check could not reach the log line at all: the sentence was an
|
||
f-string inside `Controller._shadow_settle()`, and the only suite that drives
|
||
that method uses a `meter_source: off` fixture where no arbitrage decision —
|
||
and therefore no caveat — exists. Deleting the caveat from it left **all 29
|
||
files green**. It is `shadow.log_line()` now. *A sentence assembled at a call
|
||
site is a sentence no check can grade*, which is CFG-04's substring lesson
|
||
arriving from the other direction.
|
||
|
||
Note what these have in common with a masked rung: the mutation *runs*, and the
|
||
suite is *honestly* green. Nothing in a purge, a fingerprint or a disassembly
|
||
can see it, because the code under test really did behave identically on the
|
||
stimulus it was given. The defect is upstream of the assertion, in what the
|
||
check was handed.
|
||
|
||
**The rule, and it is one line:**
|
||
|
||
> **Any check asserting "X differs from Y" must be run once with X and Y
|
||
> actually made different.**
|
||
|
||
Cheap habits fall out of it, and all of them are now used in this repository:
|
||
|
||
1. **Write the PRECONDITION as its own check.** `test_peakguard.py` 10g asserts
|
||
that A and B have different ratings *before* it asserts that B's bound is
|
||
B's. If a later edit makes them equal again, the precondition goes red rather
|
||
than the claim going quietly vacuous.
|
||
2. **Assert against literals, not against the expression under test.**
|
||
`== [(-_b2.max_w, _b2.max_w)]` cannot distinguish "B's number" from "the
|
||
number this fixture happens to have twice"; `== (-3000.0, 3000.0)` beside a
|
||
printed `A = 5000` can.
|
||
3. **Print what the check saw.** Every one of the three above would have been
|
||
caught by a human reading one line of output — `supervised bounds by
|
||
actuator: {'inverter-a': (-5000, 5000), 'inverter-b': (-5000, 5000)}` is
|
||
self-evidently not evidence of a difference.
|
||
|
||
4. **Assert on the BEHAVIOUR, not on the source text of the thing under test.**
|
||
A substring test over a module is a check on how the code is spelled;
|
||
whitespace, an alias or a moved line defeats it while the property it names
|
||
is gone. `test_inverter.py` part 10 has one of each — a source pin for a
|
||
claim that genuinely IS about the code's shape, and a driven `run_control()`
|
||
for a claim about behaviour — and the second replaced the first after the
|
||
mutation battery walked straight through it.
|
||
5. **A check whose failure mode is a HANG needs a deadline.** Otherwise the
|
||
mutation that defeats it produces a CI timeout with no name attached, which
|
||
reads like flaky infrastructure and gets retried rather than read.
|
||
6. **Assert on the CONTENT of a refusal, not on the fact that one happened.**
|
||
`not ok` is satisfied by any error at all, including one about a different
|
||
key on a different blob. `"Missing option 'inverters' in root" in out` goes
|
||
red the moment the refusal stops being about the thing it is cited for. This
|
||
is habit 2 in the negative direction: a refusal, like a difference, has to be
|
||
asserted against a literal.
|
||
7. **Vary ONE thing in a counterfactual, and vary it from an ACCEPTED
|
||
baseline.** `supervisor_shape_probe.py` §B validates the shipped defaults
|
||
first, precisely so the refusal on the next line has nowhere else to come
|
||
from. Without the accepted baseline the second line is a refusal of unknown
|
||
provenance, which is what §B used to print.
|
||
8. **A committed transcript is a check.** Anything under `tools/` that a
|
||
document quotes is subject to every rule above, and to the mutation battery:
|
||
break the property, re-run the instrument, and confirm the transcript goes
|
||
red. `inverters: []` was deleted from `config.yaml` and the probe re-run
|
||
against `ghcr.io/home-assistant/amd64-hassio-supervisor:2026.08.0` for
|
||
exactly that reason.
|
||
9. **Exercise a refusal a SECOND time with the SAME wrong input — not the
|
||
corrected one.** A guard that refuses once and then admits the same bad
|
||
caller is not a guard, it is a delay, and the second call is a *different
|
||
code path* whenever the first call left state behind. MAINT-05's roster
|
||
grading iterated the keys it had just **written**; on the second start the
|
||
same wrong caller found them already present, they moved to the
|
||
`already namespaced` list, and the loop graded an empty set. Re-running with
|
||
the **fixed** argument — which is what the check did — measures the operator's
|
||
recovery, never the persistence of the refusal. Ask of every refusal: *what
|
||
does this do when the caller has not been fixed and simply runs again?* And
|
||
assert the **shape** of that second report, not only its verdict, so the
|
||
check cannot silently revert to grading the first path.
|
||
|
||
10. **A fixture of N members cannot separate two reductions that agree on N
|
||
members — so grade the FAMILY, and grade it from the SOURCE.** This is the
|
||
only lesson SAFETY-32 cost four review rounds to learn, and it generalises
|
||
past packs to every roster, list and window in this repository. Two packs
|
||
cannot tell `max` from `min` from first from last. Three cannot tell `max`
|
||
from `max(x[-2:])`. **The regress does not terminate in a fixture**: `[:1]`
|
||
needs two members, `[:2]` three, `[:3]` four, and adding one more member
|
||
moves the hole rather than closing it. Three things follow, and all three
|
||
are now in `test_peakguard.py` parts 12c2, 12g, 12i and 12j:
|
||
* the fixture is a **SET** — the extreme at every position of the roster,
|
||
not one roster with the extreme somewhere;
|
||
* the family table is **COMPUTED and PRINTED**, so a member no fixture kills
|
||
shows up as a named survivor and takes the check red, instead of being
|
||
absent from a list nobody re-derived;
|
||
* where the claim is an **IDENTITY** ("this roster", "this list"), pin it
|
||
**exactly on the AST** — `ast.unparse(arg) == "self.packs"` is false of
|
||
every slice, where `"self.packs" in src` is true of all of them. Where the
|
||
claim is **SEMANTIC** ("the maximum"), a shape pin would refuse honest
|
||
refactors, so report the unreached width **with its size** instead.
|
||
|
||
11. **When a check of this shape is found, mutate its SIBLINGS before shipping
|
||
the fix.** SAFETY-32 was rejected three times for the same defect one step
|
||
over: the reduction, then the roster handed to it, then the reduction handed
|
||
to the *other argument of the same call*. Enumerating the family from the
|
||
AST and mutating each member took one afternoon and found **four more
|
||
survivors in code the card never touched** — three of SAFETY-30's
|
||
`any(i.sign_wd.tripped for i in self.inverters)` reductions, invisible
|
||
because the only fixture that reached them was a roster of ONE, where
|
||
`self.inverters[:1]` **is** `self.inverters`. Ask of every fix: *what is the
|
||
sibling of the thing I just hardened, and does any fixture separate it from
|
||
a slice of itself?*
|
||
|
||
12. **When the same shape has been found twice, STOP CLOSING SITES AND BUILD
|
||
THE INSTRUMENT — and make it DISCOVER its own scope.** SAFETY-32 was
|
||
rejected four times, and each round closed exactly the site it was handed:
|
||
the reduction, then the roster handed to it, then the *sibling argument* of
|
||
the same call, then the same family one *module* over, in validate.py.
|
||
Round 4 did enumerate — and enumerated inside the `Controller` class,
|
||
because that is where the four known sites lived. **A hand-written scope is
|
||
the same defect as a hand-written fixture**: it is a check whose data cannot
|
||
reach the failure it names, and it goes on passing while the thing it was
|
||
supposed to cover sits in the file nobody added to the list. Round 5's pin
|
||
parses every `.py` under `app/`, seeds itself from ONE option key, and
|
||
propagates through returns, accumulator loops, attributes and *call
|
||
arguments* to find the roster wherever it went — 183 reductions across 9
|
||
modules. It rediscovered all four blockers without being told where to look,
|
||
and it named two modules the hand-written list would not have contained
|
||
(`maintenance.py`, `store.py`) while *omitting* one the list did contain
|
||
(`watchdog.py`, which is strictly per-unit and reduces over no roster at
|
||
all). **The list was wrong in both directions**, which is exactly what a
|
||
computed scope is for.
|
||
|
||
13. **Say what the instrument CANNOT see, with sizes, in the same breath.** A
|
||
structural pin grades an **identity** ("this whole roster arrives"); it can
|
||
never grade a **meaning** (`min` vs `max` is a zero-byte edit and invisible
|
||
to it). Ship the pin AND the behavioural set, name the gap, and give every
|
||
unreached case its byte delta — `cycle()`'s undeliverable-unit log line
|
||
(+4 B, still green), a subscript by a *variable* rather than a constant, a
|
||
slice taken through an intermediate call, anything outside `app/`. **An
|
||
enumerated list with an honest gap is worth more than a claim of
|
||
completeness a reviewer falsifies in ten minutes**, and every round of this
|
||
card that claimed the latter was rejected for it.
|
||
|
||
14. **An `in` test against a formatted sentence grades PRESENCE, never
|
||
ARRANGEMENT — so assert the ORDERED PAIR.** SAFETY-38 pinned the banner of a
|
||
latching, site-stopping alarm with
|
||
`f"{sum(_share38):.0f} W" in ...reason` — that `"-500 W"` appears
|
||
*somewhere*. Swapping the detector's first two arguments at the call site
|
||
(`evaluate(site_standing, self.grid, …)` →
|
||
`evaluate(self.grid, site_standing, …)`) is a **ZERO-BYTE** edit: both
|
||
numbers are still in the string, each in the other's slot. The substring was
|
||
still found — **in the meter slot** — the whole suite stayed green, and the
|
||
banner told the operator which instrument read what **exactly backwards**.
|
||
Whenever the property you mean is "**this** value went in **this** slot" — a
|
||
reason string, a log line, a notification, an MQTT payload — assert the
|
||
surrounding **literal text with the value embedded in its own position**
|
||
(`f"commanded {x:+.0f} W against a meter reading {y:+.0f} W"`), or assert
|
||
the fields **before** they are formatted. Two values that both appear in one
|
||
sentence can always be transposed by an edit that costs **nothing**, and
|
||
that no byte delta, no fingerprint and no shape pin can see. ⚠️ Note where
|
||
this one was found: **inside the fix written to close the previous instance
|
||
of the same family**, which is the second time in this list.
|
||
|
||
⚠️ **AND THE FIX FOR IT WAS ONE CALL SITE WIDE, WHICH IS THE HALF WORTH
|
||
KEEPING.** The ordered pair went onto `main.py:1620`, the **site** detector
|
||
— armed only at N≥2, and part 9c asserts it is **absent at a roster of
|
||
one**. The identical swap at `main.py:1560`, the **per-unit** detector, was
|
||
still **ZERO BYTES**, still changed no decision, still left the 21-file
|
||
suite green — and it is the **only** sign-disagreement sentence a shipped
|
||
one-inverter install can ever print: *"commanded −1000 W against its own
|
||
measured power +1000 W"*, backwards, in the log, on the status page and in
|
||
`sign_reason` after the restart the latch is designed to survive. Measured
|
||
size: **main.py:1560, 0 bytes, 21/21 green.** So habit 14 is not "pin the
|
||
banner", it is **"pin every call site of the banner, and say which one the
|
||
shipped roster reaches"** — and habit 15 is how you find them all.
|
||
|
||
15. **A SYMMETRIC PREDICATE MAKES ARGUMENT ORDER A PROPERTY OF THE MESSAGE
|
||
ALONE — so ENUMERATE THAT FAMILY BY MACHINE, and grade the CALL SITES, not
|
||
the function.** Where a callee is symmetric in two arguments, a swap at any
|
||
call site is invisible to every behavioural check **by construction**: not
|
||
"no fixture happens to separate them" but *no fixture can*. There is
|
||
nothing for a mutation battery to find, because the mutant and the shipped
|
||
tree take identical actions on every input. The only surface that moves is
|
||
the sentence, and the only check that can see it is one asserting the
|
||
**ordered pair** (habit 14) at **that** call site.
|
||
Three consequences, and all three are why this is its own habit:
|
||
* **The unit of coverage is the CALL SITE.** SAFETY-38 pinned
|
||
`evaluate()`'s operand order and still shipped the defect, because the
|
||
pin was on the call two lines below the one that matters. Ask of every
|
||
banner: *how many places call this, and which does the shipped roster
|
||
reach?*
|
||
* **The scope must be COMPUTED** — habit 12's argument, and SAFETY-38 is
|
||
its proof: the family was enumerated by hand twice and was short both
|
||
times. `tools/symmetric_message_scan.py` renames each parameter pair,
|
||
erases every formatted string, canonicalises the commutative operators
|
||
and iterates to a **fixpoint** over `app/` (the symmetry is usually one
|
||
call deep — `evaluate()` is symmetric only *through* `disagrees()`), then
|
||
classifies each sentence by where it GOES. A returned string may be a
|
||
**key** — swapping `store.unit_key(unit, key)` renames a store key, which
|
||
is behaviour and is MAINT-05's card — or a **sentence**, which is this
|
||
family.
|
||
* **Say what the instrument cannot see, with shapes** (habit 13). That scan
|
||
under-reports semantic symmetry no canonicaliser on its finite list can
|
||
spell; its second census, for sentences the CALLER builds, pairs only
|
||
expressions **one leaf apart** and never sees `%s`-style logging; it
|
||
covers only `app/`; and it **over-reports in exactly one way** — it
|
||
erases the sentence, so a swap that would make the FORMATTING itself
|
||
raise is scored as changing no decision. It prints all of that on every
|
||
run. The family, every member's call sites, and which of them are pinned
|
||
are in `DECISIONS/SAFETY-38-site-standing-pin.md` §7 — **with two
|
||
measured, still-open survivors named there rather than rounded off**,
|
||
both zero bytes and both 21/21 green.
|
||
* ⚠️ **AND THE SINK IS HALF THE SCOPE, WHICH IS HOW THE FIRST THREE ROUNDS
|
||
OF THIS SCAN MISSED THE PLACE THE CODEBASE ACTUALLY KEEPS ITS SENTENCES.**
|
||
The scan classified a sentence as operator-facing when it was **appended,
|
||
assigned to a `reason`-shaped name, or logged** — and every one of its 16
|
||
reported members was one of those. **None was RETURNED.** A method
|
||
formatting its own **attributes** and handing the string back —
|
||
`ClampResult.describe()`, `PeakDecision.explain()`, every reason-builder
|
||
in the tree — was outside all of it, and so were sentences handed to a
|
||
constructor as `reason=` or placed under a `"text"` dict key. Four
|
||
zero-byte survivors of a green 21-file suite were sitting there, two of
|
||
them on the alarm SAFETY-40 and SAFETY-41 are about. **The general form:
|
||
an enumeration is scoped by a SINK as much as by a PREDICATE, and a sink
|
||
list is a hand-written list like any other (habit 12).** Ask of any such
|
||
scan: *which syntactic forms does this call a message, and what does the
|
||
code under it actually use?*
|
||
|
||
16. ⚠️ **AN INSTRUMENT THAT ASSERTS A COMPLETENESS IT DOES NOT HAVE IS WORSE
|
||
THAN NO INSTRUMENT, AND ITS BLIND-SPOT LIST IS PART OF THE CODE UNDER
|
||
REVIEW.** `tools/symmetric_message_scan.py` printed five honest gaps and a
|
||
census count, and a reader — including its author, twice — took the count
|
||
for the family. The gap that mattered, the RETURN sink, was **in none of the
|
||
five**. A gap list is not a disclaimer; it is a claim, and it is the claim
|
||
most likely to be believed because nothing else in the output argues with
|
||
it. Two consequences:
|
||
* **State what is NOT scanned, largest hole first, as a list of forms** —
|
||
not "this under-reports". The tool now names `%s`-style lazy logging
|
||
(`_LOG.error("… %s … %s", a, b)`, where the operands are *arguments*, so
|
||
no f-string census can ever see them), `.format()`/`%`/`+`/`.join()`
|
||
sentences, sentences that leave through a dict or dataclass, `yield`,
|
||
message attributes outside its own constant set, and everything outside
|
||
`app/`. Where the list is derived from a constant in the file, it is
|
||
**printed from that constant**, so the two cannot drift.
|
||
* **AND THE PROBE THAT GRADES THE MUTATIONS IS SUBJECT TO EVERY HABIT
|
||
ABOVE.** Round 4's render probe gave each interpolated operand a value by
|
||
**order of appearance** — so after a transposition the other operand took
|
||
the same number and **all thirteen mutant renders came back
|
||
byte-identical**, which reads exactly like *the edit changed nothing*.
|
||
Keying the value to the operand's **NAME** fixed it. That is habit 4, in
|
||
the measuring instrument: the fixture's own values were arranged so the
|
||
failure it was built to see could not change the answer. **Eleventh
|
||
instance of habit 4 in this file, and the second one in an instrument
|
||
rather than in a check.**
|
||
|
||
17. ⚠️ **TWO TERMS OF A CONJUNCTION THAT MOVE TOGETHER IN EVERY FIXTURE ARE
|
||
ONE TERM AS FAR AS THE CHECKS ARE CONCERNED - TWELFTH INSTANCE OF HABIT 4,
|
||
and the third found by an author's own battery rather than by review.**
|
||
SAFETY-37's freshness gate is `unchanged >= 35 s AND |standing| >= 250 W
|
||
AND |travel| >= 250 W`. Deleting the MIDDLE term left every check in the
|
||
first draft green, because in all of them a command that had *travelled*
|
||
250 W had also *ended up* above 250 W: the fixture never separated the two,
|
||
so one of them was decoration as far as the suite could tell. The case that
|
||
separates them is a site being brought to REST - the register reaches its
|
||
new value on the same read at which the OLD command is still standing, so
|
||
the travel term alone reads 1300 W of unfollowed command while the site
|
||
sits truthfully at +5 W. Ask of every conjunction: **is there a fixture in
|
||
which each term is the ONLY one that is false?** If not, the suite cannot
|
||
tell an AND from an OR.
|
||
|
||
18. ⚠️ **AN UNVERIFIED REVERT IS THE SAME DEFECT AS STALE BYTECODE, AND TWO
|
||
HARNESSES SHARING ONE BACKUP PATH IS THE WORST VERSION OF IT.** SAFETY-37's
|
||
sweep hit this twice, in ascending order of nastiness.
|
||
* A mutation's container run overran a two-minute command budget and the
|
||
wrapper was killed **between the edit and the `cp` that puts the file
|
||
back**, leaving `app/freshness.py` mutated on disk. Nothing announced it;
|
||
the next mutation's harness cheerfully fingerprinted and graded the mutant
|
||
*plus* a second mutation.
|
||
* Then two copies of the sweep were started - a scheduling mistake, not a
|
||
tool defect - and the harness kept its backup at the FIXED path
|
||
`/tmp/mut.bak`. One copy saved `app/main.py` there; the other restored it
|
||
**over `app/freshness.py`**. The result was a 238 KB `freshness.py` that
|
||
still parsed as Python, plus `main.py` left holding a live mutation. No
|
||
error, no warning, and the only reason it was noticed is that the tree
|
||
fingerprint did not match the recorded clean one.
|
||
|
||
Two rules, and the first is the cheap one:
|
||
* **Fingerprint after the revert, not only before the run.**
|
||
`tools/mutation_run.sh` prints the tree it is about to grade for exactly
|
||
this reason; the revert deserves the same treatment, and a mismatch is a
|
||
stop-everything result rather than a note.
|
||
* **A harness's scratch file must be unique per invocation** (`mktemp`), so
|
||
that two of it cannot silently overwrite each other's work. A fixed path
|
||
in a tool that edits source is a shared mutable global with a filename.
|
||
|
||
19. ⚠️ **A FIXTURE IN A SHAPE NOBODY PUBLISHES GRADES THE PARSER AGAINST THE
|
||
PARSER — FOURTEENTH INSTANCE OF HABIT 4, AND THE FIRST WHERE THE FIXTURE
|
||
AND THE CODE UNDER TEST SHARED A *SHAPE* RATHER THAN A NUMBER.** TARIFF-08.
|
||
`price.py`'s `_points_from_json()` read `doc["data"]` as a list of
|
||
`{start, price_eur_mwh}` objects. `api.energy-charts.info` publishes
|
||
parallel `unix_seconds` / `price` arrays and has no `data` key at all, so
|
||
`generic_dayahead` raised `SourceUnavailable` on **every live fetch it ever
|
||
made** — and the suite was green, because all three price suites built
|
||
their bodies with `json.dumps({"data": [...]})`. The fixture was derived
|
||
from the parser's guess, so no mutation of the parser and no amount of
|
||
coverage inside that loop could ever have reached the failure. §11.4 then
|
||
turned it into a *correct* silent degradation: parse error -> `no_data` ->
|
||
static profile -> arbitrage off, with nothing naming the cause.
|
||
|
||
Habit 2 with a network on the end of it: **the reference has to be data the
|
||
code under test did not shape.** For a parser that means BYTES, captured
|
||
from the source and committed unmodified, with the URL, the retrieval time
|
||
and a checksum beside them - `scenarios/energy-charts-be-price-*.json` and
|
||
its `-captures.json` provenance, re-derived by the replay so an edited
|
||
fixture stops being a capture. Two granularities, because the same endpoint
|
||
answers hourly before 2025-10-01 and quarter-hourly after it and a fixture
|
||
at one spacing cannot grade an assumption about the other.
|
||
|
||
⚠️ **AND THE SURVIVING SYNTHETIC FIXTURES ARE NAMED FOR WHAT THEY ARE.**
|
||
Three shipped plugins point at endpoints nobody has verified (TARIFF-07),
|
||
so no capture exists for them; their fixture builder is called
|
||
`synthetic_row_feed` at every call site, because the next reader's default
|
||
assumption will otherwise be that a green check means the feed parses.
|
||
|
||
⚠️ **ONE OF THIS CARD'S OWN NEW LINES WAS DECORATION AND THE BATTERY SAID
|
||
SO.** The columnar parser first rendered its instants with `brussels()`
|
||
before returning them. `assemble()` already renders every plugin's points,
|
||
so removing the parser's call changed nothing observable - **M4 came back a
|
||
survivor on a `:ro`, purged tree**, which is not the direction stale
|
||
bytecode fabricates. The line was removed rather than fenced with a check:
|
||
a second copy of a rule that already has an owner is indistinguishable from
|
||
a no-op, and the mutation that proves the rendering is real belongs on
|
||
`assemble()`, where it is now aimed.
|
||
|
||
20. ⚠️ **A FIXTURE GRADES THE RESPONSE. NOTHING ABOUT THE REQUEST IS REACHABLE
|
||
FROM ONE, AND THE TRANSPORT HAD BEEN RECORDING IT ALL ALONG.** TARIFF-09,
|
||
and it is habit 19's closed loop with the other half still shut.
|
||
`generic_dayahead.fetch()` took `since` and `until` and asked for the bare
|
||
`?bzn=BE`, which answers ONE Brussels day - so §8.3 dropped every hour past
|
||
today from the trade set as provisional and the optimiser never saw
|
||
tomorrow as firm, at any hour of any day. Measured at 09:00: **15 firm
|
||
hours and 33 carried of 48**, both ends of the permitted cycle inside one
|
||
calendar day; at 23:00, **1 and 47**, with the firm spread no longer
|
||
clearing the round-trip cost.
|
||
|
||
TARIFF-08 had just fixed the RESPONSE side of this same plugin by
|
||
committing real bytes. It could not have caught this: a captured body is
|
||
the answer to a URL, and **no property of any body says which URL was
|
||
asked for**. `canned()` in `test_price.py` had recorded `(url, timeout_s)`
|
||
on every call since TARIFF-02 and not one check had ever asserted on the
|
||
first element - so the harness *knew* the plugin was ignoring its arguments
|
||
and nothing asked it. The fix is one line of habit: **where a plugin takes
|
||
arguments that shape a request, assert on the REQUEST off the injected
|
||
transport, not only on what comes back.** Section 2c does, and the
|
||
shipped-defect mutation (restore the bare URL, `-173 B`) is red on six
|
||
named checks in `test_price.py` and two in `test_price_history.py`.
|
||
|
||
⚠️ **AND A MUTATION THAT TURNS A READ INTO `no_data` CRASHES A SUITE
|
||
INSTEAD OF NAMING A CHECK, UNLESS THE CHECK EXPECTS IT.** `_NoData.points`
|
||
raises `Absence` by design, so `derived(...) < derived(...)` over two
|
||
absent reads is a `TypeError` and `[p for p in plan.read.points]` is a
|
||
traceback - habit 5's shape with an exception in place of a hang. The
|
||
mutation that matters here is the "defensive" one somebody writes after
|
||
reading this card: *refuse when the feed returned fewer days than we asked
|
||
for*. It is exactly what a healthy feed does every morning before the
|
||
~13:00-14:00 CET auction, and switching arbitrage off for it would be a
|
||
worse defect than the one being fixed. It is red BY NAME in
|
||
`test_price.py` (33 checks, the AC's own line among them) and takes
|
||
`test_price_history.py` down at **FCST-04's section 1**, whose `_before`
|
||
read is unguarded - a red, but an unnamed one. That is recorded rather than
|
||
retrofitted: guarding another card's section is a change to what its
|
||
checks report, and it belongs to whoever next touches it.
|
||
|
||
21. ⚠️ **A CHECK CAN BE GREEN UNDER ITS OWN CARD'S NAMED MUTATION, AND TWICE
|
||
IN ONE FILE THE REASON WAS THAT THE WRONG QUANTITY EVENTUALLY REACHES THE
|
||
RIGHT VALUE.** TEL-10's acceptance was *"a run that winds the integrator
|
||
into the bound must show the published number reaching the bound"*, and its
|
||
named mutation was *"publish `prev_w` instead of `i_w`"*. The check was
|
||
**GREEN** on the mutant: the command reaches the same bound too, twelve
|
||
cycles later, and a check that only reads the VALUE cannot tell the two
|
||
apart. The same file's *"a stopped controller publishes `not-running`"* was
|
||
green under deleting that branch's assignments outright (−45 B and −67 B),
|
||
because the fixture STARTED stopped and `__init__` writes the same values.
|
||
Both are habit 4 - the data could not reach the failure - and both were
|
||
found by the card's own battery. The fixes are the shape to copy:
|
||
* where two quantities CONVERGE, grade the **order or the interval**, not
|
||
the value: "the integrator reaches the bound BEFORE the command does",
|
||
and a PLATEAU (the lead sits at exactly −800 W for eight consecutive
|
||
cycles) rather than a maximum, because a maximum is one tick and a
|
||
plateau is the shape;
|
||
* where a branch WRITES a value the constructor also writes, the fixture
|
||
has to pass THROUGH the other state first. Stop the controller part-way
|
||
through a run that has already wound the integrator to −5000 W, so a row
|
||
left standing is a WRONG row and not merely an unset one.
|
||
|
||
22. ⚠️ **AN EXPLANATION IN A COMMENT IS A CLAIM, AND SAFETY-48 SHIPPED TWO THAT
|
||
WERE INVENTED.** A gate was written one-sided (`if target > slewed: min(...)
|
||
elif target < slewed: max(...)`) under a comment saying the two-sided band
|
||
`max(lo, min(hi, i_w))` was a BUG that would drag a trailing integrator
|
||
toward the command, with a worked example. It is not a bug and the example
|
||
was wrong: `target` is monotone in `i_w`, so the other clamp cannot reach,
|
||
and the two spellings agree on **205 920 of 205 920** swept states. A second
|
||
bullet said two placements of the same cap *"differ on 118 of 256 252
|
||
ticks"*; that 118 had been measured against a **different pair** and the
|
||
placement actually being compared is bit-identical on the shipped
|
||
configuration - 0 of 256 246 ticks. Both sentences were plausible, both were
|
||
load-bearing for a design choice, and neither had been run.
|
||
**The rule: a comment that says "X would be wrong" is subject to the same
|
||
battery as a check - BUILD X AND MEASURE IT.** Where the answer is
|
||
"equivalent", say so with the sweep size and keep the spelling for what it
|
||
NAMES; where the answer is a number, quote the run it came from. An
|
||
invented justification for a correct decision is still a defect, and it is
|
||
the kind a reviewer cannot see because the code is right.
|
||
|
||
23. ⚠️ **THE MUTATION IS SUBJECT TO EVERY HABIT ABOVE, HABIT 4 INCLUDED - AND
|
||
THE PURGE, THE FINGERPRINT AND THE BYTE DELTA ALL SAY IT TOOK.** SAFETY-57's
|
||
battery reported a **survivor at +21 B** for a mutation of
|
||
`Freshness._open_window()`: `self.graded_from = float(now_mono)` rewritten
|
||
as `float(self.changed_mono or now_mono)`, which is meant to re-anchor the
|
||
verdict's window to the register's own history and so delete the card's
|
||
whole exit. The harness purged, the tree fingerprint showed the edit on
|
||
disk, the delta was non-zero, and the mutant really did run. It was still a
|
||
survivor for a reason that has nothing to do with the property: in the
|
||
fixture the register never moves after `t = 0`, so `changed_mono` is
|
||
**`0.0`** - and `0.0 or x` is `x`. **The mutant was a no-op on that
|
||
stimulus.** Rewritten with an explicit `is None` test (+82 B) it is red on
|
||
four named checks.
|
||
|
||
Stale bytecode is a mutation that never *ran*. This is a mutation that ran
|
||
and could not *change the answer* on the data it was given, which is habit 4
|
||
exactly, one level up - and none of the three things a mutation report is
|
||
required to state (MUTATION.md's own list) can see it, because all three are
|
||
about the file rather than about the stimulus. Two rules:
|
||
* **Ask of every survivor what it would have had to change, and check that
|
||
the fixture reaches that value.** Here: "does any fixture have a non-zero
|
||
`changed_mono` at the moment `_open_window()` is called?" The answer was
|
||
no, and it is one line to ask.
|
||
* **Prefer a mutation that CANNOT be a no-op.** `or`, `and`, truthiness on
|
||
a numeric, a default argument and a `getattr` fallback all have a value at
|
||
which the mutant and the shipped tree coincide. An explicit comparison,
|
||
a constant swap or a deleted statement does not.
|
||
|
||
⚠️ Note where this one was found: in the battery of a card whose whole
|
||
subject is a check that cannot reach the failure it names. **Second
|
||
instance in an instrument rather than in a check since habit 16, and the
|
||
third overall.**
|
||
|
||
⚠️ **AND THE SAME CARD'S BATTERY CAUGHT A HABIT-22 COMMENT IN THE FIX
|
||
ITSELF, WHICH IS WHY THESE TWO BELONG NEXT TO EACH OTHER.**
|
||
`freshness.py` shipped `⚠️ RELEASE BEFORE GRADE, AND THE ORDER IS
|
||
LOAD-BEARING` with a worked reason. Swapping the two calls is **zero bytes**
|
||
and survived the battery: `_grade()` acts only when `|standing| >= move_w`
|
||
and `_release()` only when `|standing| < move_w`, so at most one can act on
|
||
any read and the order is **not observable**. Habit 22 says build X and
|
||
measure it; the battery is what builds X. The comment now states the
|
||
disjointness and the suite pins **that** - at 0, 100, 249, 250, 251 and
|
||
1300 W - rather than an order nothing depends on.
|
||
|
||
⚠️ **SECOND INSTANCE, TEL-11, AND THE NO-OP WAS A SATURATED RAIL RATHER
|
||
THAN A VALUE.** The card's whole subject is that a PV reading must never
|
||
reach the control law, so its central mutation is
|
||
`self.grid = self.grid - self.pv_w` in `read_inputs()` - **+106 B**, on
|
||
disk, purged, genuinely running - and it came back a **SURVIVOR** of a
|
||
six-file battery. The property held in no sense: the fixture drove a 6 kW
|
||
export over 12 cycles, which puts the loop **against its rail**, and a
|
||
command moving `slew_w` every cycle produces the SAME ramp for -6000 W and
|
||
for -14000 W of apparent error. The mutant ran and could not change the
|
||
answer.
|
||
Two things generalise past PV:
|
||
* **A SATURATED ACTUATOR IS A VALUE AT WHICH EVERY INPUT COINCIDES.** The
|
||
habit-23 list above names `or`, `and`, truthiness, default arguments and
|
||
`getattr` fallbacks - all *expressions* with a coinciding value. A rail,
|
||
a clamp, a slew limit, a deadband and a latched alarm are the same hazard
|
||
in the **plant**: they are where a controller's output stops being a
|
||
function of its input. A fixture that parks there cannot separate any two
|
||
inputs, and this repository's control fixtures park there by default,
|
||
because the dramatic stimulus is the saturating one.
|
||
* **SO THE PRECONDITION IS NOT "DOES IT COMMAND SOMETHING", IT IS "DOES IT
|
||
REST SOMEWHERE THE MUTATION COULD MOVE IT FROM".** `test_pv.py` part 6
|
||
now drives a 600 W import the loop SETTLES on, asserts the resting target
|
||
is well inside the rail, and asks habit 23's own question as a check:
|
||
shift the METER by the same 8 kW and confirm the commands really would
|
||
have moved. On the first fixture that question answers NO, in one line,
|
||
and it is the line that turns the identity into evidence.
|
||
|
||
24. **A REFUSAL DEFINED RELATIVE TO "THE BASELINE LACKS THIS FEATURE" DIES ON
|
||
THE RELEASE THAT SHIPS THE FEATURE — AND IT DIES AS A FALSE REFUSAL, WHICH
|
||
IS THE LOUD DIRECTION.** TEL-11 built `tools/tel11_pv_trace.py` to grade a
|
||
cross-tree identity, and because "the documents are identical" was its
|
||
*result* rather than its warning sign, it had to prove instead that the two
|
||
runs came from different trees and that **exactly one side reacts to
|
||
`pv_entity`**. That was correct evidence: `release/1.0` predated the option
|
||
entirely, so a side that reacted to it was the HEAD by construction.
|
||
|
||
TEL-11 then **merged**, `release/1.0` became 0.11.0, and the baseline for
|
||
the next card reacts to `pv_entity` too. `--compare` now answers
|
||
`REFUSED: set differs from unset on ['A', 'B']` on every honest run — the
|
||
instrument refuses its own shipped tree. TEL-12 found it on the first
|
||
comparison it ran.
|
||
|
||
* **A discriminator is a fact about a MOMENT, not about a card.** "The
|
||
baseline does not have X" is true until X ships and then false forever.
|
||
Write the discriminator against something the NEXT baseline still will not
|
||
have — TEL-12's is the roster key, not the channel — and say in the tool
|
||
which fact it rests on, so the next card can see what it has to replace.
|
||
* **Prefer a discriminator that is MEASURED over one the tree REPORTS.**
|
||
`tree.supports_roster` is the tree describing itself and would have been
|
||
one line; the shipped test is "this side's roster cells ARE its unset
|
||
cells", which is the tree being measured. Both are printed, side by side,
|
||
so a disagreement between the self-report and the measurement is visible
|
||
rather than silently trusted.
|
||
* ⚠️ **And this is the SAFE direction of the same defect, which is why it
|
||
was found in ten minutes rather than in a review.** A stale refusal is a
|
||
developer chasing a failure they cannot explain (the false-red case at the
|
||
top of this file). A stale *identity* — an instrument that goes on printing
|
||
OK because its reference drifted into agreement — is the survivor-
|
||
manufacturing direction, and nothing announces it. Ask of every cross-tree
|
||
instrument: *what does this print when the feature it is about has been in
|
||
the baseline for a year?*
|
||
|
||
TEL-12's own battery produced two more instrument defects of the ordinary
|
||
kind, both recorded in `DECISIONS/TEL-12-several-pv-inverters.md` §4: a
|
||
stimulus annotation written into the document being compared (so every cell
|
||
differed for a reason that was not behaviour, and the discriminator fired on
|
||
both sides), and a fixture cell whose TRUE total was labelled as its
|
||
"partial" (so the check refused the shipped tree for publishing a number it
|
||
is required to publish). ⚠️ **Both were caught by running the instrument
|
||
against the shipped tree BEFORE running it against a mutant** — which is the
|
||
one ordering rule that separates "the instrument is wrong" from "the tree is
|
||
wrong", and `test_mqtt.py` part 8's first line ("the shipped module satisfies
|
||
every property this instrument grades") is the same rule as a check.
|
||
|
||
⚠️ **AND TEL-12's SUBJECT WAS HABIT 23 IN THE FIXTURE ITSELF.** The card adds
|
||
a SUM over a roster, and the defect it exists to prevent is publishing the
|
||
partial sum when a member is absent. A fixture whose absent member
|
||
contributes **0 W** cannot see that defect at all: the partial and the full
|
||
total are **the same number**, so "published the absence" and "published the
|
||
partial" produce identical output. That is a rail in the *arithmetic* rather
|
||
than in the plant — the same hazard as TEL-11's saturated actuator, one
|
||
domain over. `test_pv.py`'s roster is three distinct non-zero contributions
|
||
in two different units, absence is driven at **every position** rather than
|
||
at one (habit 10), and each position's partial is **computed from the
|
||
fixture** and asserted to differ from the true total as its own precondition.
|
||
|
||
And the mutation-table consequence: **a variant is not covered because a
|
||
near-miss of it is.** DEV-02's table listed the *actuator* variant of the
|
||
per-unit claim defect (`device`/`supervised` filed without `inv`), which the
|
||
static routing pin catches. The *bound* variant — right actuator, wrong
|
||
numbers — is a different defect with a different fix, and it was absent. Both
|
||
are in the table now.
|
||
|
||
## TARIFF-15 — the card-graded formula, mutation by mutation
|
||
|
||
Twenty-two mutations, one per check the card added, **run** rather than
|
||
reasoned about. Harness: `/home/homelab/.cache/t15-mut/run.py`, backups beside
|
||
it rather than in a shared scratchpad — two agents' harnesses collided in one
|
||
this week. Each run restores from backup, purges `__pycache__` **on the host**
|
||
first, and grades through the CI image with the tree bind-mounted `:ro`, so the
|
||
interpreter cannot write a cache into the tree under test at all (TEST-15's
|
||
structural immunity, borrowed).
|
||
|
||
**No survivors.** Two entries are `+0 B` and are called out as such, per the
|
||
reporting rule.
|
||
|
||
| | mutation | file | Δ | red |
|
||
|---|---|---|---|---|
|
||
| M1 | `CARD_TOLERANCE` widened to 100 — the self-check stops checking | `custom_formula.py` | +1 B | **8** |
|
||
| M2 | `CARD_TOLERANCE` tightened to 0 — the other direction, so the tolerance is a number and not a shape | `custom_formula.py` | −1 B | **16** |
|
||
| M3 | `units` becomes optional | `custom_formula.py` | +10 B | 1 |
|
||
| M4 | the `factor == 0` rung removed | `custom_formula.py` | −8 B | 1 |
|
||
| M5 | the `factor < 0` rung removed | `custom_formula.py` | −7 B | 1 |
|
||
| M6 | the `vat_pct` band widened to ±1000 | `custom_formula.py` | +6 B | 2 |
|
||
| M7 | the non-finite rung removed from `_finite()` | `custom_formula.py` | −26 B | 2 |
|
||
| M8 | the non-numeric rung removed from `_finite()` | `custom_formula.py` | −16 B | 4 |
|
||
| M9 | an unknown top-level field ignored instead of refused | `custom_formula.py` | −2 B | 1 |
|
||
| M10 | the plausibility **floor** applied to the all-in figure instead of the energy one | `custom_formula.py` | **+0 B** | 2 |
|
||
| M11 | `hourly` stops averaging and emits the feed's own points | `custom_formula.py` | +12 B | 3 |
|
||
| M12 | the hourly bucket takes the FIRST quarter instead of the mean | `custom_formula.py` | −11 B | 1 |
|
||
| M13 | the preview drops the index column from each row | `custom_formula.py` | −13 B | 1 |
|
||
| M14 | the preview is emitted on every refresh rather than once | `price.py` | −8 B | 1 |
|
||
| M15 | `custom_formula` declares `ALL_IN` — the double-counting lie | `price.py` | −41 B | **10** |
|
||
| M16 | a stale card never warns | `custom_formula.py` | −8 B | 1 |
|
||
| M17 | a stale card REFUSES instead of warning | `validate.py` | −3 B | 1 |
|
||
| M18 | the empty-formula refusal removed from the startup gate | `validate.py` | −38 B | 1 |
|
||
| M19 | the index scaled to EUR/kWh before the formula — the factor-1000 | `price.py` | +2 B | **7** |
|
||
| M20 | an `eval` call planted in `app/custom_formula.py` | `custom_formula.py` | +14 B | 1 |
|
||
| M21 | `custom_formula` stops passing the caller's range to the feed | `price.py` | **+0 B** | 1 |
|
||
| M22 | **PAIRED**: M1 *and* the plausibility band removed, together | `custom_formula.py` | −60 B | **12** |
|
||
|
||
### The four that are worth reading rather than counting
|
||
|
||
**M13 was a SURVIVOR of the whole suite on the first attempt, at −13 B.** The
|
||
check read `"index EUR/MWh -> all-in EUR/kWh" in log` — which is the preview's
|
||
**heading**. Deleting the index column from every **row** left the heading
|
||
saying it prints something it no longer printed, and nothing went red. That is
|
||
this file's third failure mode arriving in the check written to close it: the
|
||
assertion ran, the sentence was true, and the data could not reach the failure
|
||
it named. It is graded on a rendered ROW now, built here from the captured
|
||
document (`' 216.080 -> 0.30123'`), and M13 is red.
|
||
|
||
**M10 is `+0 B` and it is a real defect, not a synthetic one.** It came out of
|
||
writing the mutation rather than out of review: `implausible_all_in()`'s floor
|
||
was applied to the all-in figure, and `price_taxes_eur_kwh` shipped at 0.06 —
|
||
**three times the 0.02 floor** — so a card read a hundred times too small
|
||
(0.0016 EUR/kWh of energy) is carried over the floor by the flat tax addend
|
||
alone and passes. The hundredfold error is exactly what that band exists for.
|
||
The floor is applied to the **energy** figure now and the ceiling to the all-in
|
||
one, both directions are graded, and the trap is stated as a check in its own
|
||
right so the asymmetry cannot be "tidied up" later. *(Annotated 2026-09-13:
|
||
TARIFF-18 raised that default to 0.11736, **nearly six times** the floor. The
|
||
addend that does the carrying got BIGGER, so the trap is wider than when M10
|
||
was measured and the asymmetry is more load-bearing rather than less.)*
|
||
|
||
**M15 is the one the framework CANNOT catch, and the reds say which half does.**
|
||
Declaring `ALL_IN` makes `assemble()` add nothing, and section 3's all-in check
|
||
is *derived from the declaration* — so it passes on the lie, exactly as
|
||
`price.py`'s trap 1 says no code can catch a misdeclaration. What goes red is
|
||
the **docstring/declaration agreement** check, the card reproductions, and the
|
||
`sources` pin: ten reds, none of them from the arithmetic. That is the
|
||
guarantee at its true size — the two claims are held to agree, and whether
|
||
either is true about somebody else's tariff remains human.
|
||
|
||
**M22 is the paired run (TEST-16).** The card self-check and the plausibility
|
||
band both answer a unit error, so each could be masking the other. Weakened
|
||
**together** they produce twelve reds rather than the eight M1 alone produces,
|
||
and the band's own checks stay red — so neither rung is doing the other's work.
|
||
|
||
### What this battery does NOT establish
|
||
|
||
It does not establish that a card is the right card. Every mutation above moves
|
||
the relationship between the coefficients and the printed price; **none of them
|
||
can move the relationship between the printed price and the operator's actual
|
||
contract**, because nothing in this repository observes that. A
|
||
consistent-but-wrong pair passes every one of these checks, by construction,
|
||
and `DECISIONS/TARIFF-15-the-card-is-the-grader.md` §3 says so in the same
|
||
sentence as the claim.
|
||
|
||
## Where each surface stands
|
||
|
||
* **Local and dev-harness runs — the exposed one.** This is where FCST-03 hit
|
||
it. `goodwe_controller/__pycache__` and `app/__pycache__` are gitignored and
|
||
are sitting in most working trees right now. Use the harness.
|
||
* **CI — structurally immune, and it was before this card.** `.gitea/workflows/
|
||
checks.yml` bind mounts the checkout `-v "$WS:/w:ro"`, so the interpreter
|
||
cannot write a `__pycache__` into the tree under test at all; measured for
|
||
TEST-15, and it fails *silently* — no error, exit status 0. Each job also
|
||
`rm -rf`s its workspace and clones afresh, so nothing crosses between jobs on
|
||
the shared runner. `PYTHONDONTWRITEBYTECODE=1` is now set there too, as belt
|
||
and braces: the `:ro` is one character wide and the next person debugging a
|
||
write will be tempted by it.
|
||
* **The shipped add-on — not affected, and this is closed.**
|
||
`goodwe_controller/Dockerfile` has carried `PYTHONDONTWRITEBYTECODE=1` since
|
||
it was written. There is no deploy-side finding here; do not re-open it.
|
||
* **The infra repo's `sim/` harnesses — exposed, and NOT fixed by this card.**
|
||
`sim/replay_sign.py` and friends import the add-on out of
|
||
`$GOODWE_ADDON_REPO` and write `__pycache__` into *both* trees
|
||
(`goodwe/sim/__pycache__` exists today). SAFETY-02's headline validation is a
|
||
replay, not a suite, so a mutation graded through it carries the same
|
||
exposure. `tools/mutation_run.sh` takes the tree to purge as its first
|
||
argument precisely so that repo can call it, but the change belongs in that
|
||
repo and some of those harnesses want the rig. **Anyone mutating a module
|
||
graded by a replay must purge both trees by hand until then.**
|
||
|
||
## Survey — mutation claims of 2026-08-25 … 2026-09-01
|
||
|
||
Scoped **by the property, not by re-running everything**. A claim is at risk
|
||
only if a mutation could have preserved the file's byte count *and* been
|
||
reverted inside the same second, *and* the claim is of the kind stale bytecode
|
||
can fabricate (see "which way it lies"). Sources: `git log release/1.0` and the
|
||
commit and PR bodies.
|
||
|
||
| claim | verdict | why |
|
||
|---|---|---|
|
||
| **FCST-03** M9 `self.provisional`→`here.provisional` (dd40c56, 71cd784) | **re-verified** | this is the original false survivor. Re-run with the cache cleared, M9 is caught. The shipped source was correct throughout. |
|
||
| **FCST-03** X1, X6 degenerate stimuli (71cd784) | **not at risk** | run after the purge discipline landed in that branch. |
|
||
| **SAFETY-24** 11 author + 9 reviewer mutations (e709554, 4601682) | **not at risk** | the commit records them run under `python -B` with a purged `__pycache__` and real byte deltas. Its one survivor is an equivalent mutant, argued structurally rather than by a green run. |
|
||
| **SAFETY-26** tolerance sweep over `arbitrage.py` and `static_tariff.py`, 2 survivors (2b8b8cb) | **not at risk** | purge, printed `+5 B` delta, mutated line asserted present, tree re-read before each run. One survivor is explicitly recorded at `+0 B` — the risky delta — and was produced *under* the purge. **Both survivors are now closed by TEST-16** (`test_static_tariff.py` section 9), which also found that the two of them *masked each other* — each was individually non-live only because the other rung caught its stimulus, an arrangement single-mutation testing cannot see. Its own ledger is 7 mutations, 4 of them at `+0 B`, all red. |
|
||
| **OPT-04** mutations reported on PR #22 (bdd7f18) | **not at risk** | `test_arbitrage.py`'s header records the exact command: purge, then `python3 -B`. |
|
||
| **TEST-17** `checks()` dropping `when` (504e8da) | **not at risk** | a *red* claim, and dropping an argument changes the file length. Neither half of the collision is available. |
|
||
| **SAFETY-07** wiring mutations B1/B2/B3 and `test_peakguard.py` parts 7 and 10 (2d570d8, e7c95bf) | **not at risk** | those mutations are **executed in-process**, monkeypatched onto the imported module. No source file is edited, so no `.pyc` is consulted. |
|
||
| every **"all red, no survivors"** claim in the window — TARIFF-02 (24/0), TARIFF-03 (11), MAINT-04 (10), SAFETY-18 (6+4), SAFETY-03 (6), TEL-04 (10), TEL-05 (6) | **not at risk** | direction. Stale bytecode turns a red into a green survivor, never the reverse. A red is evidence that *something* changed under the check. |
|
||
| **TEST-06**'s deleted assertion, "no new notification while parked at 0 W" (c34b5a2) | **not at risk / superseded** | it was recorded as undecidable and then *falsified* — `test_freeze.py` records the one-line mutation that takes the file red 9 times. The stronger later claim is a red one. |
|
||
| **TARIFF-01** reviewer survivor: `record_block()` ignoring its `policy` argument (0dbd485) | **re-verified — still a survivor** | re-run here through `tools/mutation_run.sh` against the 15 suites that were on the record, all green with `billed_w(block, policy)` mutated to `billed_w(block, POLICY_NET)`. The recorded diagnosis holds: every block reaching the store is single-phase, where NET, SUM and MAX collapse. **Now CLOSED by TEST-13** (`test_tariff.py` section 7), which round-trips section 7's own unbalanced `(3000, 500, -200) W` block THROUGH the store under each policy - three distinct banked figures, so a hardcoded policy takes at least two of them red. Third instance of "a stimulus evaluated where the quantity it grades is degenerate", after WITNESS-02's span normalisation and TARIFF-01's own DST window. |
|
||
| **SAFETY-25** 8 mutations against `test_peakguard.py` part 10, then 9 more after review, all red | **not at risk** | each run against a tree copied fresh with `__pycache__` excluded and mounted `:ro`, so no `.pyc` existed to go stale - the same structural immunity recorded for CI above. Byte deltas recorded on the PR; none is `+0 B`. All are **reds**, which is the direction this defect cannot fabricate. ⚠️ The second round exists because the first round's mutations were all aimed at DISARMING the guard, and the claim part 10 actually made was about the guard being FIXED - the reviewer built the fix and the suite stayed green. Bytecode was never the issue there; an unaimed battery was. |
|
||
| **SAFETY-03** reviewer survivor: `abs()` dropped from `clamp_to_rating`'s limit (6aabb83) | **re-verified — still a survivor** | re-run the same way, all 15 green. Also as recorded: unreachable under the `int(100, 5000)` schema. |
|
||
| **MAINT-05 round 3**, `migrate_maint_keys()` marker gate `if rep.ok:` → `if rep.ok or rep.kept:` (`+12 B`) | **was a survivor — now CLOSED** | measured against head `65cddf3` under the purge, tree mounted `:ro`, `19/19 PASS`. It stamps the schema marker on a migration that LOST state, defeating the older `lost` guard as well as the new `misdirected` one, because every check that reached the marker did so on a store whose `kept` list was empty. Closed by `test_maintenance.py` **9d**; the mutation is now red on three named checks. |
|
||
| **MAINT-05 round 3**, `migrate_unit_keys()` marker gate `if rep.ok:` → `if rep.ok or rep.kept:` (`+12 B`) | **found here — SAFETY-02's half, now CLOSED** | the SAME mutation one function up, on the LATCH migration rather than the history one. Measured a survivor against this branch under the purge, `:ro`, `19/19 PASS`, then closed by `test_inverter.py` part 10's new fixture — a store with `unit/inverter-a/sign_reason` already present (so `kept` is non-empty) whose `sign_tripped: true` write is dropped. Shipped: marker withheld. Mutant: marker stamped on a **disarmed latch**, and the next start skips the migration that would have re-armed it. |
|
||
| **TARIFF-09** 11 mutations over `app/price.py` and the two-day capture, all red, no survivors | **not at risk** | direction, and method: every mutation was applied to a FRESH `cp -a` of the clean worktree rather than reverted, so there is no revert to verify (habit 18) and no shared scratch path two runs could collide on. Each ran through `tools/mutation_run.sh`, which purges and fingerprints. Byte deltas recorded on the PR; one is **+0 B** - a single price digit edited inside the committed capture, the risky delta - and the re-derived sha256 catches it. |
|
||
| **TARIFF-01**'s other three survivors, **SAFETY-03**'s `describe()` survivor, **TARIFF-02**'s M3/M5/M7, **SAFETY-18**'s three | **still unverified** | recorded before the purge discipline, with no byte delta on the record, so a `+0 B` same-second revert cannot be excluded. All are survivors, which is the direction this defect fabricates. TARIFF-02's and SAFETY-18's were **closed by later commits whose claims are reds**, so what is unverified is the *diagnosis*, not the current state of the code. |
|
||
|
||
**Nothing here says a past result is wrong.** The two survivors re-run above
|
||
both stood. What changed is that they are now on the record as measured rather
|
||
than assumed, and the rest are labelled honestly. The still-unverified rows are
|
||
cheap to close — one harness run each — and the right moment is when somebody
|
||
next touches that module, not in a re-run campaign nobody will read.
|