From 564ba11c11b6e7c7556ef2140309c1c10f2de9cc Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Sat, 5 Sep 2026 17:51:10 +0200 Subject: [PATCH] SAFETY-38 round 2: assert the ordered pair, and correct the M3 attribution Two blockers, one of them mine to fix and one a mis-measurement to undo. No `app/` change in either: the shipped `sum` is correct. 1. A ZERO-BYTE SURVIVOR AT THE CALL SITE ONE LINE BELOW THE PINNED VALUE. Round 1 pinned `site_standing` and never pinned WHERE IT GOES. Swapping the detector's first two arguments at main.py:1620 - `evaluate(site_standing, self.grid, ...)` -> `evaluate(self.grid, site_standing, ...)` - is a ZERO-BYTE edit and left the round-1 tree fully green. The cost is the banner: the site stands at -500 W of charge into a +2500 W import and the LATCHING, SITE-STOPPING alarm reads "commanded +2500 W against a meter reading -500 W" - which instrument read what, inverted, on the one line somebody woken by it reads first (watchdog.py:658-665). Round 1's check could not reach it. It asserted `"-500 W"` appears SOMEWHERE in the sentence; under the swap it still does, IN THE METER SLOT. A substring test grades PRESENCE, never ARRANGEMENT. It now asserts the ORDERED PAIR: `f"commanded {sum:+.0f} W against a meter reading {grid:+.0f} W"`. Measured: red in test_watchdog.py on that one named check, green in the other twenty. 2. THE M3 ATTRIBUTION WAS FALSE, AND HOW IT BECAME FALSE IS THE USEFUL PART. Round 1 recorded `list(standing.values())[0]` as closed by SAFETY-32's package-wide pin and told future authors NOT TO RE-COVER IT. Re-measured on release/1.0 @ 86eaef2 carrying only that edit: the full 21-file suite is GREEN and test_peakguard.py is PASS PASS PASS PASS PASS in isolation. THE MUTATION SURVIVES. It was originally measured once, that run hit CI-06's ~20% flake, and the red was recorded as a kill. The mechanism, read off 12j's own enumeration rather than argued: 12j taints names BOUND to roster values and lists ACCUMULATED in a loop. `standing` is a dict filled by SUBSCRIPT assignment (main.py:1554/1558), so it never enters the taint set and 1599 is not a roster site. On the mutated tree 12j lists main.py:1553, 1555 and 1690 - not 1599. Part 9a and 9b close it. Corrected in all four places it was committed: DECISIONS section 1 and section 4, CHANGELOG, and the part 9 header that carried the instruction. MEASUREMENT, all on 86eaef2, fresh copy per mutation, mounted :ro, bytecode purged by tools/mutation_run.sh, edit re-read from disk, byte delta printed: * M1-M10 re-run INDEPENDENTLY rather than carried forward - all 10 red, at the deltas the table records (M1/M2/M4 re-run in the exact wrapped form). * M11, the argument swap: 0 bytes, red on one check in one file. * W1 `"sign_site": SCOPE_SITE -> SCOPE_UNIT`: 0 bytes, red; part 9a is one of the two killers. * W2 `"circulation": SCOPE_SITE -> SCOPE_UNIT`: 0 bytes, red - but ENTIRELY on pre-existing checks. This card adds nothing to circulation and does not claim it. Control: 21/21 green before and after. The suite is 21 files, not 20 - TEL-07 added test_mqtt.py in the rebase onto 86eaef2. DECISIONS section 6a enumerates what was NOT closed: SAFETY-31's circ_wd detector itself, the other eight CLAIM_SCOPE entries, actuator_of()'s body, the n_units > 1 guard, and the fact that the "green in the other twenty files" half of the ledger was re-verified on 86eaef2 only for M3 and M11. MUTATION.md gains habit 14 - assert the ordered pair, not the substring - and records this as the tenth instance of the family, found INSIDE THE FIX WRITTEN TO CLOSE THE NINTH, for the second time in that list. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU --- goodwe_controller/CHANGELOG.md | 54 +++-- .../DECISIONS/SAFETY-38-site-standing-pin.md | 199 ++++++++++++++++-- goodwe_controller/MUTATION.md | 46 +++- goodwe_controller/test_watchdog.py | 72 +++++-- 4 files changed, 317 insertions(+), 54 deletions(-) diff --git a/goodwe_controller/CHANGELOG.md b/goodwe_controller/CHANGELOG.md index e661657..06c0a9f 100644 --- a/goodwe_controller/CHANGELOG.md +++ b/goodwe_controller/CHANGELOG.md @@ -121,13 +121,35 @@ shipped `sum` is correct and must not be "fixed".** `[0]` are **the same number** — all three mutations are genuinely *equivalent* on a shipped install and no N=1 fixture could ever separate them. Printed and checked, beside the site detector still being absent there. Part 9c. -* **ONE OF THE THREE WAS ALREADY CLOSED, AND THIS CARD DOES NOT RE-COVER IT.** - [[SAFETY-32]]'s package-wide structural pin catches - `list(standing.values())[0]` on the line it is written, at every roster - length, with no fixture. Its own disclosed blind spot — *"SEMANTICS. `min` → - `max` → the mean are edits invisible to any shape pin, permanently"* — is why - the other two needed this card, and it is now confirmed by measurement rather - than taken on trust. +* **⚠️ A CORRECTION, AND IT IS THE MOST USEFUL LINE IN THIS CARD.** An earlier + revision of this entry said one of the three — `list(standing.values())[0]` — + was **already closed** by [[SAFETY-32]]'s package-wide structural pin, and + told future authors not to re-cover it. **That was false.** It had been + measured **once**, that run hit `test_peakguard.py`'s [[CI-06]] flake (a + measured **~20%** false-red rate), and the red was recorded as a mutation + **kill**. Re-measured on `release/1.0` @ `86eaef2` carrying only that edit: + the **full 21-file suite is green** and `test_peakguard.py` passes **five for + five** in isolation — **the mutation survives.** The reason SAFETY-32 cannot + see it is not the shape: part 12j taints names *bound* to roster values and + lists *accumulated* in a loop, and `standing` is a dict filled by **subscript + assignment**, so it never enters the taint set and `sum(standing.values())` is + not a roster site at all. 12j's own enumeration on the mutated tree lists + `main.py:1553`, `1555`, `1690` — **not `1599`**. **All three mutations needed + this card; all three are now covered by it.** *A single red from + `test_peakguard.py` proves nothing — re-run it five times in isolation and + name the failing check.* +* **⚠️ AND A ZERO-BYTE SURVIVOR ONE LINE BELOW THE ONE THIS CARD PINNED.** + Pinning the *value* `site_standing` said nothing about **where it goes**. + Swapping the detector's first two arguments — + `evaluate(site_standing, self.grid, …)` → `evaluate(self.grid, site_standing, …)` + — is **zero bytes** and left the tree fully green, because the check asserted + only that `"-500 W"` appeared **somewhere** in the banner. It still does: + **in the meter slot.** The swapped banner reads *"commanded +2500 W against a + meter reading −500 W"* — a site charging at −500 W into a +2500 W import, told + to the operator **exactly backwards**, on the one line somebody woken by a + latching site-stopping alarm reads first. The check now asserts the **ordered + pair** rather than a substring, and is red on it. **No `app/` change:** the + hole was in the test. * **⭐ A SURVIVOR THIS CARD WENT LOOKING FOR AND CLOSED.** `site_standing = float(self.target)` — the arbiter's site *target* instead of the sum of what actually **stands on the wire** — is **−11 bytes** and @@ -137,13 +159,19 @@ shipped `sum` is correct and must not be "fixed".** unit's share and never the site target. A site asking for −500 W while −250 W stands would then take a **latching site-wide false trip** under its own 300 W gate. Closed by part 9a's device-clamped counterfactual. -* **10 mutations of that one line, each on a fresh full-repo copy mounted `:ro`, +* **13 mutations — 11 of that line and its call site, plus 2 zero-byte edits to + the arbiter claim wiring — each on a fresh full-repo copy mounted `:ro`, bytecode purged by `tools/mutation_run.sh`, the edit re-read from disk and the - byte delta printed. All 10 red — every one of them in `test_watchdog.py` and - in nothing else.** ⚠️ One **false red** from `test_peakguard.py` on M10 did - not reproduce in four subsequent runs; it is CI-06's known flakiness, no claim - here rests on it, and it is recorded rather than quietly dropped. Ledger and - the honest gaps in `DECISIONS/SAFETY-38-site-standing-pin.md`. + byte delta printed. All 13 red.** Every one was **re-run independently** in + this round rather than carried forward from the previous round's report. The + claim-wiring pair (`sign_site` and `circulation` moved from site scope to unit + scope) closes two gaps a reviewer disclosed — though **`circulation`'s kill + belongs entirely to pre-existing checks and is not claimed by this card.** + ⚠️ One **false red** from `test_peakguard.py` on M10 did not reproduce in four + subsequent runs; it is CI-06's known flakiness, no claim here rests on it, and + it is recorded rather than quietly dropped — unlike the one that *was* + believed, two bullets up. Ledger, and an **enumerated list of what was not + mutated**, in `DECISIONS/SAFETY-38-site-standing-pin.md` §4a, §4b and §6a. **Inverter B gets its own MQTT device, and inverter A keeps the one it has (TEL-07).** MQTT discovery announced **one** device, and every entity hung off diff --git a/goodwe_controller/DECISIONS/SAFETY-38-site-standing-pin.md b/goodwe_controller/DECISIONS/SAFETY-38-site-standing-pin.md index 289c789..16d6672 100644 --- a/goodwe_controller/DECISIONS/SAFETY-38-site-standing-pin.md +++ b/goodwe_controller/DECISIONS/SAFETY-38-site-standing-pin.md @@ -33,19 +33,55 @@ Both are P0 paths. **Neither was pinned to the reduction.** ## 1. What survived, and what the rescope measured The card opened with three demonstrated live mutants, all green across the whole -suite. Re-measured on 2026-09-05 against `release/1.0` @ `f72c48d`, after -[[SAFETY-32]] merged: +suite. Re-measured in round 2 against `release/1.0` @ `86eaef2`: -| mutation of `site_standing` | before SAFETY-32 | after SAFETY-32 | +| mutation of `site_standing` | on release/1.0 @ 86eaef2 | closed by | |---|---|---| -| `list(standing.values())[0]` — one unit's share | survives | **CAUGHT** by SAFETY-32's package-wide structural pin | -| `sum(...) / max(1, len(standing))` — the mean | survives | **still survives** | -| `max(standing.values(), key=abs)` — the largest share | survives | **still survives** | +| `list(standing.values())[0]` — one unit's share | **SURVIVES**, full suite 21/21 green | part 9a + 9b (this card) | +| `sum(...) / max(1, len(standing))` — the mean | **SURVIVES**, full suite 21/21 green | part 9a + 9b (this card) | +| `max(standing.values(), key=abs)` — the largest share | **SURVIVES**, full suite 21/21 green | part 9a + 9b (this card) | -SAFETY-32's pin catches *a slice or index standing between a roster-derived -value and its reduction*, on the line it is written, at every roster length, -with no fixture — which is exactly `[0]`. Its own disclosed blind spot says why -the other two are out of reach: +### ⚠️ ROUND 1 SAID `[0]` WAS ALREADY CLOSED BY SAFETY-32. THAT WAS FALSE. + +Round 1 of this card recorded `[0]` as **CAUGHT by SAFETY-32's package-wide +structural pin**, wrote that claim into four places, and told future authors +*"DO NOT RE-COVER THAT ONE HERE"*. It is wrong, and **how** it went wrong is the +part worth keeping. + +**How the error was made.** The mutation was measured **once**. That single run +happened to hit `test_peakguard.py`'s [[CI-06]] flake — a **measured ~20%** +false-red rate — and the red was recorded as a **mutation kill**. Nothing else +was checked, because a red is the answer you are hoping for and it did not look +like it needed a second run. + +**The correct measurement.** On `release/1.0` @ `86eaef2` carrying **only** the +`[0]` edit (+4 bytes, re-read from disk, bytecode purged by +`tools/mutation_run.sh`): the **full 21-file suite is green**, and +`test_peakguard.py` run **five times in isolation** on that same mutated tree is +**PASS PASS PASS PASS PASS**. The mutation **survives**. (The reviewer's own +independent re-measurement of the same tree hit the flake once — +PASS PASS PASS FAIL PASS — which is the same verdict plus a live demonstration +of the mechanism that produced the original error.) + +**Why SAFETY-32 cannot see it, read off 12j's own enumeration rather than +argued.** Part 12j taints names *bound* to roster-carrying values and lists +*accumulated* in a loop over one. `standing` is neither: it is a dict created +empty at `main.py:1554` and filled by **subscript assignment** +(`standing[inv.name] = stood`) inside the roster loop, and a subscript store is +not a binding 12j follows. So `standing` never enters its taint set and +`sum(standing.values())` is **not a roster site** at all. Run 12j against a tree +carrying the `[0]` mutation and its enumeration lists `main.py:1553`, `1555` and +`1690` — the three `self.inverters` sites bracketing this code — and **not +`1599`**. No shape pin reaches this line; only the behavioural fixture below +does. + +**The rule this cost.** `test_peakguard.py` flakes at a measured ~20%. A single +red from it proves nothing. Re-run it in isolation **at least five times** and +**name the failing check** before believing any red from it. This card is the +**third** measurement CI-06 has corrupted. + +SAFETY-32's own disclosed blind spot says why the other two were always out of +reach: > *SEMANTICS. `min` → `max` → the mean are edits invisible to any shape pin, > permanently. Only the behavioural sets can see them.* @@ -54,6 +90,13 @@ the other two are out of reach: different meaning. **No shape pin will ever close them.** That disclosure was accurate, and is now confirmed by measurement rather than taken on trust. +`[0]` is the one that *looks* structural and therefore *looks* covered — and +that appearance is exactly what produced the false attribution above. It is +out of reach here for a different reason from the other two: not because the +shape is invisible, but because **the value never becomes a roster** in the +analysis's eyes. All three are closed by the same behavioural fixture, and +**all three needed it.** + ## 2. Why the suite could not see it — habit 4, inside the site half of the very card that names it `test_watchdog.py`'s `_sign_site` is the **only** N≥2 site fixture in the @@ -174,13 +217,13 @@ Every run: a **fresh full-repo copy**, `__pycache__` and `*.pyc` excluded, mounted `:ro`, graded by `tools/mutation_run.sh` (which purges, prints the tree fingerprint, and verifies no cache appears during the run). The mutated line was **re-read from disk** after the edit and the byte delta printed. Shipped -`app/main.py` is 232 520 bytes. +`app/main.py` is 234 542 bytes at `86eaef2`. | # | mutation of `site_standing` | Δ bytes | verdict | |---|---|---|---| | M1 | `sum(...) / max(1, len(standing))` — the mean | +24 | **RED** | | M2 | `max(standing.values(), key=abs)` — largest share | +9 | **RED** | -| M3 | `list(standing.values())[0]` — one share | +4 | **RED** | +| M3 | `list(standing.values())[0]` — one share | +4 | **RED** — and **survives on release/1.0**, see §1 | | M4 | `min(standing.values(), key=abs)` — smallest share | +9 | **RED** | | M5 | `sum(abs(v) for v in ...)` — magnitude, sign lost | +16 | **RED** | | M6 | `-sum(standing.values())` — sign flipped | +1 | **RED** | @@ -188,12 +231,15 @@ fingerprint, and verifies no cache appears during the run). The mutated line was | M8 | `sum(standing.values()) / 2.0` | +6 | **RED** | | M9 | `sum(standing.values()) * 2.0` | +6 | **RED** | | M10 | `sorted(standing.values())[-1]` — largest signed | +7 | **RED** | +| **M11** | **argument SWAP** at the call site: `evaluate(site_standing, self.grid, …)` → `evaluate(self.grid, site_standing, …)` (main.py:1620) | **0** | **was a SURVIVOR in round 1 — now RED**, see §4a | -**Every one of the ten is red in `test_watchdog.py` and in nothing else** — the -other 19 files are green under all ten, which is the measurement of how -unpinned this line was: before part 9 existed, three of these left the whole -suite green and the remaining seven were caught by SAFETY-32's structural pin or -by SAFETY-30's own N≥2 checks, none of which grade the *quantity*. +**Every one of the ten is red in `test_watchdog.py`.** Re-measured in round 2 +on `release/1.0` @ `86eaef2` (where `app/main.py` is 234 542 bytes, 2 022 more +than the `f72c48d` the round-1 figures were taken on): all ten still red, at the +byte deltas in the table, each on a fresh copy with the edit re-read from disk. +Before part 9 existed, **three** of these left the whole suite green — M1, M2 +**and M3** — and the remaining seven were caught by SAFETY-30's own N≥2 checks, +none of which grade the *quantity*. ⚠️ **ONE FALSE RED, AND IT IS [[CI-06]]'s.** In the batch run M10 also reported `FAIL test_peakguard.py`. Re-run in isolation **three times** and once more @@ -204,9 +250,76 @@ and that **no claim in this card rests on it**. `test_peakguard.py` is the file CI-06 records as flaky, and this is the behaviour it records. Believe no red from it without three isolated re-runs. -M3 is listed for completeness only. SAFETY-32 already closes it **structurally** -and this card does not claim it; that the behavioural fixture also catches it is -a free consequence of the same discriminating numbers, not a second pin. +⚠️ **M3 IS NOT "FOR COMPLETENESS", AND ROUND 1 SAID IT WAS.** Round 1 recorded +M3 as already closed structurally by SAFETY-32 and disclaimed it. §1 above has +the correction and the mechanism: `[0]` **survives** a full green 21-file suite +on `release/1.0` @ `86eaef2`, because `standing` is a dict filled by subscript +assignment and never enters 12j's taint set. **M3 is a real kill by this card's +fixture and by nothing else in the repository.** It is claimed here. + +### 4a. ⚠️ The zero-byte survivor one line below the one this card pinned + +Round 1 pinned the **value** `site_standing` and never pinned **where it goes**. +Swapping the detector's first two arguments at `main.py:1620` is a **zero-byte** +edit — the same characters, reordered — and it left the round-1 tree **21/21 +green**. + +What it costs is the banner. `SignWatchdog.evaluate()` formats +`commanded_w` first and `against_w` second, so under the swap part 9a's own +output reads: + +``` +SIGN_DISAGREEMENT: commanded +2500 W against a meter reading -500 W for 2 s … +``` + +The site is standing at **−500 W of charge into a +2500 W import** and the +**latching, site-stopping** banner tells the operator **the exact opposite** — +which instrument read what, inverted. `watchdog.py:658-665` names this failure +in its own words: *"a false statement about which instrument produced the +evidence, on the one line somebody woken by this alarm reads first."* + +**Why round 1's check could not see it, and this is the general lesson.** It +asserted + +```python +f"{sum(_share38):.0f} W" in _c38.site_sign_wd.reason +``` + +— that `"-500 W"` appears **somewhere** in the sentence. Under the swap it still +does: in the **meter slot**. A substring test cannot reach a failure that is +purely **positional**, and round 1's own comment claimed the opposite ("*would +put a different number in the store, the action log and the banner, and this is +where that shows*"). It shows the **number**, never its **position**. This is +MUTATION.md's *"a check whose data cannot reach the failure it names"*, in the +card that exists to complain about exactly that. + +**The fix is one line and touches no `app/` code** — assert the **ordered +pair**: + +```python +f"commanded {sum(_share38):+.0f} W against a meter reading " +f"{_mid38['grid']:+.0f} W" in _c38.site_sign_wd.reason +``` + +Measured: with that line, M11 is **red in `test_watchdog.py` and green in the +other twenty files** — one file, one check. + +### 4b. Claim wiring, re-measured rather than assumed + +Round 1's reviewer disclosed it had **not** mutated the arbiter claim wiring or +[[SAFETY-31]]/circulation. Both were mutated in round 2. Both are **zero-byte** +edits to `arbiter.CLAIM_SCOPE`: + +| # | mutation | Δ bytes | verdict | +|---|---|---|---| +| W1 | `"sign_site": SCOPE_SITE` → `SCOPE_UNIT` | **0** | **RED** — `test_watchdog.py` (parts 5 **and 9a**), `test_arbiter.py`, `test_peakguard.py` | +| W2 | `"circulation": SCOPE_SITE` → `SCOPE_UNIT` | **0** | **RED** — `test_watchdog.py`, `test_arbiter.py`, `test_peakguard.py` | + +Part 9a's *"…and it stops the WHOLE site, through the `sign_site` claim by name"* +is one of the two checks that kills W1, so this card contributes to that +coverage rather than merely inheriting it. **W2 is caught entirely by +pre-existing checks** — SAFETY-38 adds nothing to circulation, and does not +claim to. ## 5. The survivor this card went looking for, and closed @@ -250,3 +363,49 @@ operator typed. * **Everything about inverter B here is simulated and labelled so.** No number in this document or in the fixture is a measurement of a device that does not yet exist. + +### 6a. Round 2: what was closed, and what was NOT + +Round 1's reviewer disclosed that it had not mutated SAFETY-31/circulation, the +arbiter claim wiring, or ledger entries M4/M5/M6/M8/M9/M10. Enumerated, with the +gaps left open stated rather than rounded off: + +**Closed in round 2 (measured, on `86eaef2`, bytecode purged, edit re-read from +disk, byte delta printed):** + +1. **M11**, the zero-byte argument swap — §4a. Red on one named check. +2. **M3**, the `[0]` reduction — re-measured, **survives on release/1.0**, closed + by part 9a and 9b. The false attribution corrected in all four places it was + committed: this file §1 and §4, `CHANGELOG.md`, and `test_watchdog.py`'s part + 9 header (which previously *instructed future authors not to re-cover it*). +3. **M1, M2, M4, M5, M6, M7, M8, M9, M10** — every ledger entry re-run + independently in round 2 rather than carried forward from round 1's report. + All ten red; M1/M2/M4 re-run in the exact wrapped form the table records so + the byte deltas are the table's own (+24/+9/+9). +4. **W1**, `sign_site` claim scope — §4b. Zero bytes, red, part 9a among the + killers. +5. **W2**, `circulation` claim scope — §4b. Zero bytes, red, but **entirely on + pre-existing checks**. + +**NOT closed, and not claimed:** + +* **SAFETY-31's `circ_wd` detector itself** — its threshold, its hold, its + `circulating_w()` reduction over the roster. Only its **claim scope** was + mutated (W2), and that kill belongs to checks this card did not write. + Circulation's own behavioural coverage is `test_watchdog.py` part 4's and is + untouched here. +* **The other eight `CLAIM_SCOPE` entries** — `loop`, `peak`, `arbitrage`, + `safety` (site) and `device`, `supervised`, `failsafe`, `maintenance` (unit). + `CLAIM_SCOPE` has ten entries; only the two named above were mutated. `test_peakguard.py` part 10f grades the + classification statically; that is a different instrument and this card did + not re-measure it. +* **`actuator_of()`'s own body** and the `n_units > 1` guard on the `sign_site` + append at `main.py:1764` were **not** mutated. +* **The ten ledger mutations were re-graded against `test_watchdog.py` only.** + The "green in the other twenty files" half of the round-1 claim was + re-verified on `86eaef2` for **M3** and **M11** (both full 21-file runs) and is + otherwise carried forward from round 1's measurement on `f72c48d`. It is a + weaker statement than round 1 made and is written down as such. +* **`test_peakguard.py`'s CI-06 flake is not fixed by this card** and is not + this card's subject. It is *characterised* here (five isolated runs on the M3 + tree, all PASS) only far enough to overturn the false kill it produced. diff --git a/goodwe_controller/MUTATION.md b/goodwe_controller/MUTATION.md index 36e863d..c276686 100644 --- a/goodwe_controller/MUTATION.md +++ b/goodwe_controller/MUTATION.md @@ -135,13 +135,15 @@ manufactures them from a **fixture**: the check runs, the assertion is evaluated, the sentence above it is true — and the numbers it compares are arranged so that the failure it describes could not have changed the answer. -It is not hypothetical and it is not rare. NINE instances in two weeks; four +It is not hypothetical and it is not rare. TEN instances in two weeks; four were found by review, two - CFG-04's fixture pair - by an author's own mutation battery, which is the cheaper place, one by review again, in the one place nobody had thought to point the rule at: not a check in the suite, but a COMMITTED EVIDENCE ARTIFACT four documents cite - the eighth INSIDE THE FIX -WRITTEN TO CLOSE THE SEVENTH, and the ninth inside the SITE HALF of the card -that names the rule in its own text. +WRITTEN TO CLOSE THE SEVENTH, the ninth inside the SITE HALF of the card +that names the rule in its own text - and the TENTH INSIDE THE FIX WRITTEN TO +CLOSE THE NINTH, for the second time in this list. See SAFETY-38 below: the +check that closed it asserted a NUMBER and not the number's POSITION. ⚠️ **AND IT HAS NOW APPEARED IN SIX CONSECUTIVE CARDS.** The seventh instance, MAINT-05's, is the one that says why: the defect is not carelessness about @@ -242,8 +244,8 @@ same place twice, the check is decoration. fixture, and it runs shares of about **−2500 W each** — every one individually over the 320 W gate it is configured with. On numbers like those `sum`, the mean and `max(key=abs)` **all agree**, so the reduction could be - swapped for either of the other two and the site guard went **blind with all - 19 files green** — still commanding −500 W of charge into a +2500 W import at + swapped for either of the other two and the site guard went **blind with the + whole suite green** — still commanding −500 W of charge into a +2500 W import at t = 595 s with `following=True`. At a roster of **one** it is worse and it is not a fixture defect at all: `sum == mean == max == the one share` *identically*, so every such mutation is genuinely **equivalent** there and no @@ -262,6 +264,20 @@ same place twice, the check is decoration. and never bounds the site target. Closed in the same commit, by a counterfactual that asks for −500 W while −250 W stands. + ⚠️ **AND THEN THE FIX ITSELF WAS THE TENTH INSTANCE — A SUBSTRING WHERE THE + FAILURE WAS POSITIONAL.** The check written to pin the banner asserted + `f"{sum(...):.0f} W" in reason`: that **"−500 W" appears somewhere** in the + sentence. Swapping the detector's first two arguments + (`evaluate(site_standing, self.grid, …)` → `evaluate(self.grid, site_standing, …)`) + is a **ZERO-BYTE** edit that leaves both numbers in the string and moves each + into the other's slot — so the substring is still found, **in the meter + slot**, and the whole suite stayed green while the latching site-stopping + banner told the operator which instrument read what **exactly backwards**. + The fix is to assert the **ORDERED PAIR**, + `f"commanded {sum:+.0f} W against a meter reading {grid:+.0f} W"`. + + The general habit is **14** below. + Note what these have in common with a masked rung: the mutation *runs*, and the suite is *honestly* green. Nothing in a purge, a fingerprint or a disassembly can see it, because the code under test really did behave identically on the @@ -391,6 +407,26 @@ Cheap habits fall out of it, and all of them are now used in this repository: completeness a reviewer falsifies in ten minutes**, and every round of this card that claimed the latter was rejected for it. +14. **An `in` test against a formatted sentence grades PRESENCE, never + ARRANGEMENT — so assert the ORDERED PAIR.** SAFETY-38 pinned the banner of a + latching, site-stopping alarm with + `f"{sum(_share38):.0f} W" in ...reason` — that `"-500 W"` appears + *somewhere*. Swapping the detector's first two arguments at the call site + (`evaluate(site_standing, self.grid, …)` → + `evaluate(self.grid, site_standing, …)`) is a **ZERO-BYTE** edit: both + numbers are still in the string, each in the other's slot. The substring was + still found — **in the meter slot** — the whole suite stayed green, and the + banner told the operator which instrument read what **exactly backwards**. + Whenever the property you mean is "**this** value went in **this** slot" — a + reason string, a log line, a notification, an MQTT payload — assert the + surrounding **literal text with the value embedded in its own position** + (`f"commanded {x:+.0f} W against a meter reading {y:+.0f} W"`), or assert + the fields **before** they are formatted. Two values that both appear in one + sentence can always be transposed by an edit that costs **nothing**, and + that no byte delta, no fingerprint and no shape pin can see. ⚠️ Note where + this one was found: **inside the fix written to close the previous instance + of the same family**, which is the second time in this list. + And the mutation-table consequence: **a variant is not covered because a near-miss of it is.** DEV-02's table listed the *actuator* variant of the per-unit claim defect (`device`/`supervised` filed without `inv`), which the diff --git a/goodwe_controller/test_watchdog.py b/goodwe_controller/test_watchdog.py index 930564a..10715e4 100644 --- a/goodwe_controller/test_watchdog.py +++ b/goodwe_controller/test_watchdog.py @@ -2000,15 +2000,40 @@ print(" asked about - pinned by shares that CANNOT answer for it") # ⚠️ WHAT THIS SECTION PINS, AND WHY NOTHING ABOVE IT COULD. # `main.py`'s `site_standing = float(sum(standing.values()))` is the first # argument of SAFETY-30's site sign detector AND the `commanded_w` handed to -# SAFETY-18's `looks_frozen()`. It was pinned by NOTHING. Three mutations of it -# were demonstrated live against a full green suite; SAFETY-32's package-wide -# structural pin has since closed `list(standing.values())[0]` - a slice -# standing between a roster-derived value and its reduction, which it sees on -# the line it is written, at every roster length, with no fixture. DO NOT -# RE-COVER THAT ONE HERE. The two that remain are SEMANTIC: +# SAFETY-18's `looks_frozen()`. It was pinned by NOTHING. THREE mutations of it +# were demonstrated live against a full green suite, and ALL THREE ARE COVERED +# HERE: # # sum(...) / max(1, len(standing)) the MEAN # max(standing.values(), key=abs) the LARGEST SHARE +# list(standing.values())[0] the FIRST SHARE +# +# ⚠️ ROUND 1 OF THIS CARD SAID THE THIRD ONE WAS ALREADY CLOSED BY SAFETY-32's +# PACKAGE-WIDE STRUCTURAL PIN AND TOLD FUTURE AUTHORS "DO NOT RE-COVER THAT ONE +# HERE". THAT WAS FALSE, AND HOW IT BECAME FALSE IS THE USEFUL PART. It was +# measured ONCE; that single run happened to hit test_peakguard.py's ~20% +# CI-06 flake, the red was recorded as a mutation KILL, and the claim was then +# written into four places. RE-MEASURED IN ROUND 2, on release/1.0 @ 86eaef2 +# carrying ONLY the `[0]` edit (+4 bytes, re-read from disk): the full suite is +# 21/21 GREEN, and test_peakguard.py run FIVE times in isolation on that same +# mutated tree is PASS PASS PASS PASS PASS. The mutation SURVIVES. (The +# reviewer's own independent re-measurement of the same tree caught the flake +# once - PASS PASS PASS FAIL PASS - which is the same verdict: a survivor, plus +# a demonstration of the flake that produced the original error.) A SINGLE RED +# FROM test_peakguard.py PROVES NOTHING - re-run it at least five times and NAME +# the failing check before believing it. That rule is CI-06's, and this card is +# the third measurement it has corrupted. +# +# THE MECHANISM, confirmed by reading 12j's own enumeration rather than by +# argument. SAFETY-32's part 12j taints names BOUND to roster-carrying values +# and lists ACCUMULATED in a loop over one. `standing` is neither: it is a dict +# created empty at main.py:1554 and filled by SUBSCRIPT assignment +# (`standing[inv.name] = stood`) inside the roster loop, and a subscript store +# is not a binding 12j follows. So `standing` never enters its taint set at all, +# and `sum(standing.values())` is not a roster site. Run 12j on a tree carrying +# the `[0]` mutation and it enumerates main.py:1553, 1555 and 1690 - the three +# `self.inverters` sites around this code - and NOT 1599. A structural pin +# cannot see this line. Only the behavioural fixture below can. # # Identical shape, different meaning - which is precisely the blind spot # SAFETY-32 disclosed in its own words ("SEMANTICS. `min` -> `max` -> the mean @@ -2217,19 +2242,34 @@ check("PRECONDITION: EVERY inverter is obeying - each unit's measured power " and sum(_mid38["stood"]) * _mid38["grid"] < 0 and _mid38["following"] is True) check("⚠️ THE SITE WATCHDOG TRIPS ON THE SUM, and no per-unit watchdog does. " - "Under `sum -> mean` and `sum -> max(key=abs)` NOTHING trips, ever: the " - "reviewer's tree was still commanding -500 W of charge into a +2500 W " - "import at t = 595 s with following=True. This is the check both of " - "those mutations go red on", + "Under ALL THREE semantic mutations - `sum -> mean`, " + "`sum -> max(key=abs)` AND `sum -> list(...)[0]` - NOTHING trips, ever: " + "the reviewer's tree was still commanding -500 W of charge into a " + "+2500 W import at t = 595 s with following=True. This is the check all " + "three go red on. `[0]` is here BECAUSE IT IS NOT COVERED ELSEWHERE - " + "see the section header for the mis-measurement that said it was", _c38.site_sign_wd.tripped and not any(i.sign_wd.tripped for i in _c38.inverters)) print(f" {_c38.site_sign_wd.reason!r}") -check("...and the sentence an operator reads quotes THE SUM, digit for digit - " - "-500 W, not -250, not -280, not -220. A mutation that still tripped but " - "on some other reduction would put a different number in the store, the " - "action log and the banner, and this is where that shows", - f"{sum(_share38):.0f} W" in _c38.site_sign_wd.reason - and "against a meter reading" in _c38.site_sign_wd.reason) +check("⚠️ ...and the sentence an operator reads carries THE SUM IN THE " + "COMMANDED SLOT AND THE METER IN THE METER SLOT - the ORDERED PAIR, not " + "two numbers loose in a sentence. ROUND 1 ASSERTED `f'{sum(...):.0f} W' " + "in reason`, i.e. that '-500 W' appears SOMEWHERE, AND THAT WAS A HOLE " + "ONE LINE WIDE. Swapping the detector's first two arguments - " + "`evaluate(site_standing, self.grid, ...)` -> " + "`evaluate(self.grid, site_standing, ...)`, a ZERO-BYTE edit at " + "main.py:1620 - leaves BOTH numbers in the string and moves each into " + "the other's slot, so the substring test still passed and the whole " + "suite stayed green. The swapped banner reads 'commanded +2500 W " + "against a meter reading -500 W': the site is standing at -500 W of " + "charge INTO a +2500 W import and the latching, site-stopping banner " + "tells the operator the exact opposite, on the one line somebody woken " + "by this alarm reads first. watchdog.py:658-665 calls naming the wrong " + "instrument there 'a false statement about which instrument produced " + "the evidence'. A check whose data cannot reach the failure it names is " + "MUTATION.md's own defect; this is the line that closes it", + f"commanded {sum(_share38):+.0f} W against a meter reading " + f"{_mid38['grid']:+.0f} W" in _c38.site_sign_wd.reason) check("...and it stops the WHOLE site, through the `sign_site` claim by name " "rather than by some other guard happening to stop it", _c38.target == 0.0 and "sign_site" in _c38.reason