watchdog.py's predicate was fed the same second argument for every inverter:
inv.sign_wd.evaluate(stood,self.grid,self.following,now)# every unit
stood is the command this controller put on THIS actuator. self.grid is
what the whole site is exchanging with the network. At a roster of one those
are two views of one thing — §6.2's identity — which is what makes the check
sound and what earned it the record NOTES.md describes: 93 % sign agreement while
healthy against 4 % during runaway, all four observed failure episodes, zero
false positives.
At two inverters they are not, and it is wrong in both directions:
what happens
why no threshold fixes it
false trip
A commanded −1500 W while the meter reads +1500 W, because B is discharging 3000 W exactly as told
the meter answers for a unit A's command says nothing about
missed fault
B commanded +1500 W and measured at −1500 W, beside A at +1500 W — the meter sits where a healthy site leaves it
the summation destroys the evidence
The first row spends a record earned at N=1. The second is the guard being absent
in the case it exists for.
DEV-02 had already shipped the per-instance half — one SignWatchdog per
inverter, namespaced latch, a trip on B no longer latching A — and nothing here
re-tests it. What was left was the comparison.
What this changes
Two questions, each asked of the evidence that can answer it.
PER UNIT — this unit's standing command against its own measured power
(Inverter.batt, holding register 0x0518, per-instance since DEV-02).
Conclusion: the actuator I am driving is not doing what I told it. A fact
about one device, so the remedy stays the per-unit failsafe SET that arbiter.CLAIM_SCOPE has classified SCOPE_UNIT since it was written — that
entry read "the evidence is site-wide and the remedy is not", and after this
card the first half is simply true.
SITE — the sum of the standing commands against the meter, as a new sign_site SET at P_SAFETY on the site actuator, latched under flat site_sign_tripped / site_sign_reason. This is the 2026-08-20 fault day, and no per-unit check can see it: every inverter may be faithfully executing
its share of a number that is itself wrong. It stops every inverter, and the
banner names the site rather than an inverter so nobody is sent to swap the
wrong box.
Controller._arm_sign_watchdogs() is the only place the roster decides anything
here, and it is a pure function of len(self.inverters).
Which fault each guard covers — SAFETY-31 and the freeze, named
Two guards that overlap without saying so is how you get a false-alarm generator,
so both neighbours are written down in watchdog.py, in DECISIONS/SAFETY-30-per-unit-sign.md §2, and asserted in test_watchdog.py.
SAFETY-31 — circulation.py
SAFETY-30 — the per-unit half
compares
the units against each other: Σ|Bᵢ| − |ΣBᵢ| over measured power
each unit against its own command
catches
two inverters pushing through each other; invisible at the meter
a unit not doing what it was told; that unit is faulty or wedged
attributes
nothing — defined over the roster, identically zero for one unit
the unit, by name, and parks it
They genuinely overlap in one scenario — a unit reversing while its sibling
pulls the other way — and part 4 of test_watchdog.py asserts exactly that on
the same fixture, with both readings printed: circulating power is over its
threshold and SAFETY-30 names B. Neither is the other with a different
threshold.
It is also not the saturation freeze: that is a magnitude test
(|command − measured| > saturation_w) that stops the integrator winding, fires
on a merely saturated unit, and neither latches nor parks anything. This is a direction test that latches and parks one. They meet at a number rather than
overlapping in a band nobody drew — see the threshold below.
The four numbers, re-derived at two inverters
Read off admin/goodwe's sim/ two-master rig, used read-only and run
container-free. Real and imported, never transcribed: the control law, the
allocator, the sign predicate, the firmware's own encoder, plantsim.Plant, and
the per-unit reading taken from register 0x0518 — the register the live batt_entity binds.
1890 closed-loop runs in which nothing is faulty — every unit executes the
share the allocator gave it — over 6 gains × 5 slews (the whole range config.yaml accepts), across 1050 load reversals with write staggers and 840
flapping loads. ⚠️Every run is executed twice, once at each roster, so the N=1
arm is the control and this can separate "a property of the check that
shipped" from "a property of the second inverter".
option
value
why
unit_sign_disagree_w
250
saturation_w / 2. Both sides gated and opposed, so the smallest gap it can see is 500 W — exactly where compute()'s anti-windup test already says the inverter is not following. Worst legitimate opposition at 250 W: 10.0 s
unit_sign_disagree_s
85
8.5× that 10.0 s, 2.8× the firmware's own 30 s link veto (inverter.LINK_STALE_S), under half the 180 s site hold
site_sign_disagree_w
300
CARRIED, and justified: the argument behind it is the meter's noise floor and the 15 W deadband, neither of which the roster touches. The gate opens in 994/1050 runs at two inverters and 955/1050 at one
site_sign_disagree_s
325
NOT CARRIED. See below
The number that refused to be carried
Worst continuous legitimate opposition, load-reversal family, at 300 W:
N=1 (the shipped comparison) 110.0 s inside the 180 s hold, 1.6x
N=2 (the sum, same stimuli) 215.0 s OUTSIDE it
Carrying 180 s would have shipped a false trip, and it is reachable: gain: 0.05 and slew_w: 50 are both schema floors, and ±2200 W is an oven and a heat
pump. The mechanism predicts the ratio — the transient the hold exists to reject
is the loop crossing zero, and the command has to travel a rail that is the
roster sum at a slew_w that is not. 325 s is 1.51× the measured worst, inside
the 900 s billing quarter, and inside every episode of the fault day (its four
clusters run 16, 30, 77 and 77 minutes). Cost, stated rather than hidden: about
two and a half extra minutes of wrong-direction site power.
At one inverter, nothing changes — asserted, not assumed
Only the detector that shipped is armed. Same comparison against the meter, same
300 W / 180 s, same reason string byte for byte (asserted against the
literal), same namespaced latch and flat rollback mirror, same failsafe claim,
same banner, same single status row, no sign_site claim — and the site detector
is never called, counted on the object over twelve cycles. The four new
options do nothing at a roster of one.
Verified by breaking it
Thirteen mutations, each on its own freshly copied tree with no __pycache__, mounted :ro, and the battery refuses to report anything unless
the baseline is green. Twelve red on the first pass.
M4 survived and is recorded as a survivor. Arming the site detector at N=1 as
well left the suite green, because the check read disagree_since is None: the
unit detector trips first, parks the inverter, and the site's standing command is
then 0 W — so its window is never opened and "never evaluated" read identically
to "evaluated and found nothing". The assertion ran, the sentence was true, and
the data could not reach the failure it named. Replaced with a call counter on the
object; M4 is red now.
⚠️ The first battery also reported a RED BASELINE — one shared mutation
directory, rebuilt between runs, read by a container mid-copy. That whole run was
discarded rather than reported. Ledger, deltas and both stories: DECISIONS/SAFETY-30-per-unit-sign.md §5.
SAFETY-24's lo-only asymmetry proof re-run unchanged: 1296 combinations, 0 negative floors, hi+inf on every claim, every claim on the SITE
actuator.
Full suite: 19/19 green in goodwe-addon-ci.
What I am unsure about, and what a reviewer should look hardest at
site_sign_disagree_s: 325 is a real slowdown of a P0 check, and it is the
decision I am least sure of. The alternative I rejected was keeping 180 s and
raising the site threshold to 1000 W, where the worst legitimate case is 85 s —
faster to act, better margin, and (I think) wrong, because it raises a noise
gate to reject a transient and leaves the site blind to a 900 W whole-site
reversal at a 4 kW site. Worth arguing with.
The flapping-load blind spot is recorded, not fixed. A square wave whose
half period is about one control cycle holds the site comparison opposed for
~1000 s at both rosters. The N=1 control arm is what says this has nothing
to do with the second inverter, and the healthy captured day's longest opposed
run is 30 s, so I judged it out of scope — but it does mean the site check has
a shape of load it cannot survive, and somebody should decide whether that
wants a card now.
Gate 1 (following) is kept for the per-unit comparison. A unit reversing
during a maintenance charge is therefore still the freeze's to notice, not
this module's. Conservative, and it preserves the whole no-false-positive
argument — but it is a deliberate hole and I have written it down rather than
closed it.
Inverter B does not exist. Its ramp and lag in the rig are inverter A's,
copied (measured=False). unit_sign_disagree_s depends on that most, site_sign_disagree_s less (it is dominated by the rail and slew_w), and
neither threshold depends on it at all. SPIKE-04 is the revisit trigger; §4 of
the decision record says which figure is which.
validate.py deliberately has NO rule pairing site_sign_disagree_w with max_w, unlike the other two thresholds: the bound on the site command is Controller.site_max_w, the roster sum, and validate_startup_config() is
handed options rather than a roster. A rule against max_w would refuse a
legal two-inverter config the day CFG-04 lands. The schema range is the bound
there, and I would rather be told if that is the wrong call.
Both guards fire on the one overlapping scenario, which is correct but
means an operator sees two alarms for one event. Whether the banner should say
so in one sentence is a UI question I did not answer.
## The defect
`watchdog.py`'s predicate was fed the same second argument for every inverter:
```python
inv.sign_wd.evaluate(stood, self.grid, self.following, now) # every unit
```
`stood` is the command **this controller put on THIS actuator**. `self.grid` is
what the **whole site** is exchanging with the network. At a roster of one those
are two views of one thing — §6.2's identity — which is what makes the check
sound and what earned it the record NOTES.md describes: 93 % sign agreement while
healthy against 4 % during runaway, all four observed failure episodes, **zero
false positives**.
At two inverters they are not, and it is wrong in **both** directions:
| | what happens | why no threshold fixes it |
|---|---|---|
| **false trip** | A commanded −1500 W while the meter reads +1500 W, because B is discharging 3000 W exactly as told | the meter answers for a unit A's command says nothing about |
| **missed fault** | B commanded +1500 W and **measured** at −1500 W, beside A at +1500 W — the meter sits where a healthy site leaves it | the summation **destroys** the evidence |
The first row spends a record earned at N=1. The second is the guard being absent
in the case it exists for.
**DEV-02 had already shipped the per-instance half** — one `SignWatchdog` per
inverter, namespaced latch, a trip on B no longer latching A — and nothing here
re-tests it. What was left was the comparison.
## What this changes
**Two questions, each asked of the evidence that can answer it.**
* **PER UNIT** — this unit's standing command against **its own measured power**
(`Inverter.batt`, holding register 0x0518, per-instance since DEV-02).
Conclusion: *the actuator I am driving is not doing what I told it.* A fact
about one device, so the remedy stays the per-unit `failsafe` SET that
`arbiter.CLAIM_SCOPE` has classified `SCOPE_UNIT` since it was written — that
entry read *"the evidence is site-wide and the remedy is not"*, and after this
card the first half is simply true.
* **SITE** — the **sum** of the standing commands against the meter, as a new
`sign_site` SET at `P_SAFETY` on the site actuator, latched under flat
`site_sign_tripped` / `site_sign_reason`. This is the 2026-08-20 fault day, and
**no per-unit check can see it**: every inverter may be faithfully executing
its share of a number that is itself wrong. It stops every inverter, and the
banner names the site rather than an inverter so nobody is sent to swap the
wrong box.
`Controller._arm_sign_watchdogs()` is the only place the roster decides anything
here, and it is a pure function of `len(self.inverters)`.
## Which fault each guard covers — SAFETY-31 and the freeze, named
Two guards that overlap without saying so is how you get a false-alarm generator,
so both neighbours are written down in `watchdog.py`, in
`DECISIONS/SAFETY-30-per-unit-sign.md` §2, and asserted in `test_watchdog.py`.
| | **SAFETY-31 — `circulation.py`** | **SAFETY-30 — the per-unit half** |
|---|---|---|
| compares | the units **against each other**: `Σ\|Bᵢ\| − \|ΣBᵢ\|` over measured power | each unit against **its own command** |
| catches | two inverters pushing **through** each other; invisible at the meter | a unit **not doing what it was told**; that unit is faulty or wedged |
| attributes | nothing — defined over the roster, identically zero for one unit | **the unit**, by name, and parks it |
They genuinely overlap in **one** scenario — a unit reversing while its sibling
pulls the other way — and part 4 of `test_watchdog.py` asserts exactly that on
the same fixture, with both readings printed: circulating power is over its
threshold **and** SAFETY-30 names B. Neither is the other with a different
threshold.
It is also **not the saturation freeze**: that is a *magnitude* test
(`|command − measured| > saturation_w`) that stops the integrator winding, fires
on a merely saturated unit, and neither latches nor parks anything. This is a
*direction* test that latches and parks one. They meet at a number rather than
overlapping in a band nobody drew — see the threshold below.
## The four numbers, re-derived at two inverters
Read off `admin/goodwe`'s `sim/` two-master rig, used read-only and run
container-free. Real and imported, never transcribed: the control law, the
allocator, the sign predicate, the firmware's own encoder, `plantsim.Plant`, and
the per-unit reading taken from **register 0x0518** — the register the live
`batt_entity` binds.
**1890 closed-loop runs in which nothing is faulty** — every unit executes the
share the allocator gave it — over 6 gains × 5 slews (the whole range
`config.yaml` accepts), across 1050 load reversals with write staggers and 840
flapping loads. ⚠️ **Every run is executed twice, once at each roster, so the N=1
arm is the control** and this can separate *"a property of the check that
shipped"* from *"a property of the second inverter"*.
| option | value | why |
|---|---:|---|
| `unit_sign_disagree_w` | 250 | `saturation_w / 2`. Both sides gated and opposed, so the smallest gap it can see is 500 W — exactly where `compute()`'s anti-windup test already says the inverter is not following. Worst legitimate opposition at 250 W: **10.0 s** |
| `unit_sign_disagree_s` | 85 | 8.5× that 10.0 s, 2.8× the firmware's own 30 s link veto (`inverter.LINK_STALE_S`), under half the 180 s site hold |
| `site_sign_disagree_w` | 300 | **CARRIED**, and justified: the argument behind it is the meter's noise floor and the 15 W deadband, neither of which the roster touches. The gate opens in 994/1050 runs at two inverters and 955/1050 at one |
| `site_sign_disagree_s` | 325 | **NOT CARRIED.** See below |
### The number that refused to be carried
Worst continuous legitimate opposition, load-reversal family, at 300 W:
```
N=1 (the shipped comparison) 110.0 s inside the 180 s hold, 1.6x
N=2 (the sum, same stimuli) 215.0 s OUTSIDE it
```
**Carrying 180 s would have shipped a false trip**, and it is reachable: `gain:
0.05` and `slew_w: 50` are both schema floors, and ±2200 W is an oven and a heat
pump. The mechanism predicts the ratio — the transient the hold exists to reject
is the loop crossing zero, and the command has to travel a rail that **is the
roster sum** at a `slew_w` that is not. 325 s is 1.51× the measured worst, inside
the 900 s billing quarter, and inside every episode of the fault day (its four
clusters run 16, 30, 77 and 77 minutes). Cost, stated rather than hidden: about
two and a half extra minutes of wrong-direction **site** power.
## At one inverter, nothing changes — asserted, not assumed
Only the detector that shipped is armed. Same comparison against the meter, same
300 W / 180 s, **same reason string byte for byte** (asserted against the
literal), same namespaced latch and flat rollback mirror, same `failsafe` claim,
same banner, same single status row, no `sign_site` claim — and the site detector
is **never called**, counted on the object over twelve cycles. The four new
options do nothing at a roster of one.
## Verified by breaking it
Thirteen mutations, each on **its own freshly copied tree** with no
`__pycache__`, mounted `:ro`, and the battery refuses to report anything unless
the baseline is green. **Twelve red on the first pass.**
**M4 survived and is recorded as a survivor.** Arming the site detector at N=1 as
well left the suite green, because the check read `disagree_since is None`: the
unit detector trips first, parks the inverter, and the site's standing command is
then 0 W — so its window is never opened and *"never evaluated"* read identically
to *"evaluated and found nothing"*. The assertion ran, the sentence was true, and
the data could not reach the failure it named. Replaced with a call counter on the
object; M4 is red now.
⚠️ The **first** battery also reported a **RED BASELINE** — one shared mutation
directory, rebuilt between runs, read by a container mid-copy. That whole run was
discarded rather than reported. Ledger, deltas and both stories:
`DECISIONS/SAFETY-30-per-unit-sign.md` §5.
`SAFETY-24`'s lo-only asymmetry proof re-run unchanged: **1296** combinations,
**0** negative floors, `hi` `+inf` on every claim, every claim on the SITE
actuator.
Full suite: **19/19 green** in `goodwe-addon-ci`.
## What I am unsure about, and what a reviewer should look hardest at
1. **`site_sign_disagree_s: 325` is a real slowdown of a P0 check**, and it is the
decision I am least sure of. The alternative I rejected was keeping 180 s and
raising the site threshold to 1000 W, where the worst legitimate case is 85 s —
faster to act, better margin, and (I think) wrong, because it raises a *noise*
gate to reject a *transient* and leaves the site blind to a 900 W whole-site
reversal at a 4 kW site. Worth arguing with.
2. **The flapping-load blind spot is recorded, not fixed.** A square wave whose
half period is about one control cycle holds the site comparison opposed for
~1000 s at **both** rosters. The N=1 control arm is what says this has nothing
to do with the second inverter, and the healthy captured day's longest opposed
run is 30 s, so I judged it out of scope — but it does mean the site check has
a shape of load it cannot survive, and somebody should decide whether that
wants a card now.
3. **Gate 1 (`following`) is kept for the per-unit comparison.** A unit reversing
during a maintenance charge is therefore still the freeze's to notice, not
this module's. Conservative, and it preserves the whole no-false-positive
argument — but it is a deliberate hole and I have written it down rather than
closed it.
4. **Inverter B does not exist.** Its ramp and lag in the rig are inverter A's,
copied (`measured=False`). `unit_sign_disagree_s` depends on that most,
`site_sign_disagree_s` less (it is dominated by the rail and `slew_w`), and
neither threshold depends on it at all. SPIKE-04 is the revisit trigger; §4 of
the decision record says which figure is which.
5. **`validate.py` deliberately has NO rule pairing `site_sign_disagree_w` with
`max_w`**, unlike the other two thresholds: the bound on the site command is
`Controller.site_max_w`, the roster sum, and `validate_startup_config()` is
handed options rather than a roster. A rule against `max_w` would refuse a
legal two-inverter config the day CFG-04 lands. The schema range is the bound
there, and I would rather be told if that is the wrong call.
6. **Both guards fire on the one overlapping scenario**, which is correct but
means an operator sees two alarms for one event. Whether the banner should say
so in one sentence is a UI question I did not answer.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
THE DEFECT, in one line. `inv.sign_wd.evaluate(stood, self.grid, ...)` compared
the command THIS controller put on ONE actuator against what the WHOLE SITE is
exchanging with the network. At a roster of one those are two views of the same
thing - specs.md §6.2's identity - and that is what makes the check sound and
what earned it the best record in the safety table (NOTES.md: 93 % agreement
while healthy against 4 % during runaway, four observed episodes, zero false
positives). At two inverters they are not, and it fails in BOTH directions:
FALSE TRIP inverter A commanded -1500 W while the meter reads +1500 W is
evidence of nothing when inverter B is discharging 3000 W exactly
as told. Nothing is faulty and the highest-value check fires.
MISSED the converse, and it is worse. A unit commanded +1500 W and
MEASURED at -1500 W, beside a sibling at +1500 W, leaves P1
exactly where a healthy site leaves it. The summation destroys the
evidence and no threshold recovers it.
DEV-02 had already shipped the per-INSTANCE half - one SignWatchdog per
inverter, namespaced latch, a trip on B no longer latching A. What was left was
the COMPARISON, and it is two questions rather than one:
PER UNIT this unit's standing command against ITS OWN measured power
(`Inverter.batt`, holding register 0x0518, per-instance since
DEV-02). Conclusion: "the actuator I am driving is not doing what I
told it" - a fact about one device, so the remedy stays the per-unit
`failsafe` SET that arbiter.CLAIM_SCOPE has classified SCOPE_UNIT
from the day it was written. That entry read "the evidence is
site-wide and the remedy is not"; the first half is now simply true.
SITE the SUM of the standing commands against the meter, as a new
`sign_site` SET at P_SAFETY on the site actuator, latched under flat
`site_sign_tripped` / `site_sign_reason`. This is the 2026-08-20
fault day and NO per-unit check can see it, because every inverter
may be faithfully executing its share of a number that is itself
wrong. It stops every inverter and the banner names the site.
WHAT IT IS NOT, said here because two guards that overlap without saying so is
how a false-alarm generator gets built. It is not SAFETY-31: circulation.py
compares the units AGAINST EACH OTHER and catches them pushing power THROUGH
each other, which nets to zero at the meter and about which no single command is
evidence. This compares each unit against ITS OWN COMMAND. They overlap in
exactly one scenario - a unit reversing while its sibling pulls the other way -
and test_watchdog.py asserts on that fixture that both fire and what each
concludes. It is also not the saturation freeze, which is a MAGNITUDE test
(|command - measured| > 500 W) that shapes the loop and fires on a merely
saturated unit; this is a DIRECTION test that latches and parks one.
THE FOUR NUMBERS, RE-DERIVED AT TWO INVERTERS - one carried with a justification
and one refused. 1890 closed-loop runs on admin/goodwe's TEST-19 two-master rig
in which NOTHING is faulty, swept over the whole range config.yaml accepts for
`gain` and `slew_w`, across load reversals with write staggers and flapping
loads, and RUN AT BOTH ROSTERS so the N=1 arm is the control:
unit_sign_disagree_w 250 `saturation_w / 2`. The check gates on both sides,
opposed, so the smallest gap it can see is 500 W -
exactly where compute()'s own anti-windup test
already says the inverter is not following. It
cannot fire on a divergence the control law calls
ordinary tracking. Worst legitimate opposition at
250 W: 10.0 s; the curve is nearly flat, because a
unit's own power chases its own command.
unit_sign_disagree_s 85 8.5x that 10.0 s, 2.8x the firmware's own 30 s
link veto (inverter.LINK_STALE_S), under half the
180 s site hold.
site_sign_disagree_w 300 CARRIED. Its argument is the meter's noise floor
and the 15 W deadband; the roster touches neither,
and the campaign agrees - the gate opens in 994 of
1050 runs at two inverters and 955 of 1050 at one.
site_sign_disagree_s 325 NOT CARRIED, and this is the number the card
exists to not assume. The worst legitimate
crossing at 300 W is 110 s with one inverter and
215 s with TWO, because the command must travel a
rail that is the roster sum at a `slew_w` that is
not - reachable at gain 0.05, slew_w 50 and a
2.2 kW load reversal, all schema-legal. 180 s
would have shipped a false trip. 325 s is 1.51x
the measured worst, inside the 900 s billing
quarter and inside every episode of the fault day.
AT ONE INVERTER NOTHING CHANGES, and "nothing" is asserted rather than assumed.
The two questions are the same sentence there, so only the detector that shipped
is armed: same comparison against the meter, same 300 W / 180 s, same reason
string byte for byte, same namespaced latch and flat rollback mirror, same
`failsafe` claim, same banner, same single status row - and the site detector is
NEVER CALLED, counted on the object over twelve cycles.
VERIFIED BY BREAKING IT. Thirteen mutations on fresh per-mutation trees with no
__pycache__, mounted :ro, with the battery refusing to report at all unless the
baseline is green. Twelve red on the first pass. M4 - the site half armed at N=1
as well - SURVIVED, because the check read `disagree_since is None` and the unit
detector trips first, parks the inverter and leaves the site standing at 0 W:
the assertion ran, the sentence was true, and the data could not reach the
failure it named. Replaced with a call counter on the object; M4 is red now.
Recorded as a survivor in DECISIONS/SAFETY-30-per-unit-sign.md §5, with the
first battery's own RED BASELINE - a shared mutation directory read mid-copy -
discarded rather than reported.
RECORDED, NOT FIXED: the site comparison is defeated by a square-wave load whose
half period is about one control cycle, which holds it opposed for ~1000 s. The
N=1 control arm is what makes that interpretable - it is identical at the roster
this add-on ships, so it is a property of comparing a lagged command against a
meter and has nothing to do with the second inverter. It wants an alternation
test rather than a number, and a card of its own.
SAFETY-24's lo-only asymmetry proof re-run unchanged: 1296 combinations, 0
negative floors, `hi` +inf on every claim, every claim on the SITE actuator.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
REVIEW FOUND A MUTATION SURVIVOR THAT UNDOES THE WHOLE PRODUCT OF AC 5, and it
is a coverage defect rather than a behaviour one - the shipped code arms the
detectors correctly today. In `_arm_sign_watchdogs()`, collapsing
inv.sign_wd.arm(self.sign_cfg if one else self.unit_sign_cfg, ...)
to `arm(self.sign_cfg, ...)` throws the re-derived 250 W / 85 s per-unit pair
away and arms every unit with SAFETY-02's meter-facing 300 W / 180 s - and left
ALL 19 FILES GREEN. Reproduced here before fixing anything: fresh per-mutation
tree, no __pycache__, mounted :ro, graded through tools/mutation_run.sh; -31 B;
19 pass, 0 fail.
WHY NOTHING CAUGHT IT. `grep -rn "sign_wd.cfg" test_*.py` found exactly ONE
threshold assertion, part 7's N=1 arm at (300.0, 1.0) - which the mutation
PRESERVES. Nothing at a roster of two read `inv.sign_wd.cfg` at all. The ledger
mutates the VALUES in SIGN_OPTION_DEFAULTS (M10 250->300, M11 325->180, both
correctly red through the config pins) and never the WIRING that delivers them.
Part 3's behavioural checks cannot close the gap either, and the comment now
says why: B is reversed at ~2000 W with short holds, so it trips under EITHER
pair, and a fault big enough and long enough to trip under both thresholds is
evidence about neither.
THE FIX - test_watchdog.py part 3, on the real two-inverter Controller. Every
`inv.sign_wd.cfg` equals the per-unit pair, `site_sign_wd.cfg` equals the site
pair, and each detector's `against` is its own instrument. Graded against
LITERALS, because `== self.unit_sign_cfg` cannot tell "the per-unit pair" from
"whatever this Controller was handed". Both pairs are printed.
⚠️ AND THE FIXTURE HAD TO CHANGE FOR THE PIN TO MEAN ANYTHING - the recursion
worth naming, because it is the same defect class as the one being closed.
`_sign_site` set `sign_disagree_s`, `unit_sign_disagree_s` AND
`site_sign_disagree_s` all to 1 s, with two of the three magnitude gates at
300 W. A bare equality there discriminates on `min_w` alone: a mis-wiring that
delivered the right magnitude with the WRONG HOLD compares EQUAL to the pair it
was supposed to have. A check comparing two things that happen to be equal
cannot see the difference it names (MUTATION.md's third failure mode). The
three pairs are now pairwise distinct IN BOTH COORDINATES - 250 W / 1 s per
unit, 320 W / 2 s site, 300 W / 3 s shipped - a PRECONDITION asserts that
distinctness so a later edit collapsing them goes red rather than quietly
vacuous, and a second PRECONDITION asserts the literals are the pairs this
Controller actually holds.
VERIFIED BY BREAKING IT. Three mutations, each on its own freshly copied tree
with no __pycache__, mounted :ro, through tools/mutation_run.sh:
M14 arm() handed self.sign_cfg at every roster -31 B RED (was the
survivor; the AC 5 WIRING check names the wiring)
M15 right min_w, SHIPPED hold - hold-only mis-wire +78 B RED, same pin
M16 right min_w, SITE hold - hold-only, and it still
trips inside the drive +83 B RED, and the
ONLY red in test_watchdog.py
M16 is the one that says the hold half of the equality bites on its own.
Unmutated tree: 19 pass, 0 fail, purged and fingerprinted.
Ledger rows M14-M16 and the write-up in DECISIONS/SAFETY-30-per-unit-sign.md
section 5; a CHANGELOG bullet.
NOT IN THIS PR, so it gets a card rather than a quiet fix: a stuck per-unit
power sensor would false-trip the new per-unit check at N>=2. Already-merged
SAFETY-31 has the identical exposure on the same fixture. Pre-existing N>=2
class, unreachable at the deployed roster of one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
Blocking finding closed: the WIRING is pinned now, not just the values
Fix pushed as d5c1577. The shipped code was not touched - the survivor was a
coverage gap, not a behaviour defect. Every run below is on its own freshly
copied tree with no __pycache__, mounted :ro, graded through tools/mutation_run.sh (purge + fingerprint + python3 -B), per MUTATION.md.
1. The survivor, reproduced FIRST on the rejected head f000547
arm(self.sign_cfg if one else self.unit_sign_cfg, ...) -> arm(self.sign_cfg, ...), -31 B, the mutated line asserted present on disk before the run:
== mutated app/main.py 213844 B -> 213813 B (delta -31 B)
== the line ON DISK now:
804: inv.sign_wd.arm(self.sign_cfg,
== any bytecode in the tree: 0 entries
== purge
no bytecode cache under /w/goodwe_controller
== run (python3 -B)
PASS test_alloc.py (25s)
PASS test_arbiter.py (0s)
PASS test_arbitrage.py (6s)
PASS test_bytecode.py (1s)
PASS test_ci_gate.py (1s)
PASS test_clamp.py (1s)
PASS test_control.py (0s)
PASS test_freeze.py (34s)
PASS test_inverter.py (5s)
PASS test_loop.py (93s)
PASS test_maintenance.py (0s)
PASS test_p1.py (8s)
PASS test_peakguard.py (60s)
PASS test_price.py (1s)
PASS test_price_forecast.py (0s)
PASS test_static_tariff.py (4s)
PASS test_tariff.py (8s)
PASS test_validate.py (5s)
PASS test_watchdog.py (30s)
19 pass, 0 fail. Confirmed exactly as reported.
2. The same mutation on the fixed head - now RED, and the failure names the wiring
M14, same edit, -31 B, app/main.py fingerprint e8562fcaadde68a0 / 213813 B
against the unmutated 179bf7b57187f301 / 213844 B:
and part 3 of that file, verbatim - note the printed line, which is the pin
showing its working rather than asserting in the dark:
3. AC 1: ONE unit drives the wrong way, and ONLY that unit trips
armed per unit: {'inverter-a': (300.0, 3.0), 'inverter-b': (300.0, 3.0)}
armed site: (320.0, 2.0) shipped pair, wanted by NOBODY here: (300.0, 3.0)
ok PRECONDITION: the three pairs are pairwise distinct IN BOTH COORDINATES, so an equality below can fail on the magnitude alone, on the hold alone, or on both. This fixture originally gave all three the same 1 s hold and two of them the same 300 W gate, and a mis-wiring that delivered the right magnitude with the wrong hold would then have compared EQUAL to the pair it was supposed to have
ok PRECONDITION: and all three are pairs this Controller actually holds, so the literals below are the fixture's own numbers and not three constants that agree with nothing. In particular the shipped pair is still constructed and still a reachable mis-wiring target
FAIL ⚠️ AC 5 WIRING. EVERY per-unit detector on this roster of two is armed with the PER-UNIT pair - 250 W / 1 s here, `unit_sign_disagree_w` and `unit_sign_disagree_s` in the field. Graded against LITERALS rather than against `_c30.unit_sign_cfg`, which cannot tell 'the per-unit pair' from 'whatever this Controller was handed'. This is the line that goes red when `_arm_sign_watchdogs()` stops choosing between the two pairs and hands every unit the site-facing one
ok ...and the SITE detector is armed with the SITE pair - 320 W / 2 s here - which differs from the per-unit pair AND from the shipped pair in both coordinates, so this equality is not carried by a number the fixture happens to hold twice
ok ...and each detector is pointed at its own INSTRUMENT as well as at its own thresholds: the units at their own 0x0518 power, the site at the meter. They are two arguments of one call and a pin on one is not a pin on the other
Six further checks in that file go red as fallout (B no longer trips inside the
drive under a 3 s hold); the pin above is the one that names the defect.
3. THE FIXTURE HAD TO CHANGE, and that is the part worth reading
You were right that a bare equality would not have bitten. _sign_site set sign_disagree_s, unit_sign_disagree_sandsite_sign_disagree_s all to 1, with two of the three magnitude gates at 300 W - so the equality could only
discriminate on min_w, and a hold delivered from the wrong pair would have
compared EQUAL to the pair it was supposed to have. The three pairs are now
pairwise distinct in both coordinates - 250 W / 1 s per unit, 320 W / 2 s site, 300 W / 3 s shipped - and a PRECONDITION check asserts
that distinctness, so a later edit collapsing them again goes red rather than
leaving the claim quietly vacuous.
Proof that the HOLD half bites, not only min_w. Two hold-only mis-wirings,
each delivering the CORRECT 250 W with the wrong hold:
RED - the wiring pin, and the only red in test_watchdog.py
M16 is the isolated case: its 2 s hold still expires inside the drive, so every
behavioural check in part 3 stays green and the pin is the sole discriminator.
armed per unit: {'inverter-a': (250.0, 2.0), 'inverter-b': (250.0, 2.0)}
armed site: (320.0, 2.0) shipped pair, wanted by NOBODY here: (300.0, 3.0)
FAIL ⚠️ AC 5 WIRING. EVERY per-unit detector on this roster of two is armed with the PER-UNIT pair - 250 W / 1 s here ...
1 FAILED: ⚠️ AC 5 WIRING. EVERY per-unit detector on this roster of two is armed with the PER-UNIT pair ...
(M16 also takes test_inverter.py red, because its own two-inverter fixture
borrows inverters[0].sign_wd.cfg and inherits the doubled hold.)
4. The unmutated tree at d5c1577, purged and fingerprinted
19 files, 19 pass, 0 fail. (The last block is reflowed to three columns for
width; every other block on this comment is verbatim.)
5. On the field consequence, corrected
The finding put the survivor's cost at "180 s instead of 85 s - 3.8x". At config.yaml's own defaults the mutation arms the units with sign_disagree_s 180 s against the per-unit 85 s, which is 2.1x, and moves the gate
from 250 W to 300 W. (3.8x would be the site hold, 325 s, which this particular
edit does not deliver.) The direction and the conclusion are unchanged.
6. Not filed here, on purpose - wants a card
A stuck per-unit power sensor would false-trip the new per-unit check at
N>=2. Already-merged SAFETY-31 has the identical exposure on the same
fixture, so this is a pre-existing N>=2 class rather than something this card
introduces, and it is unreachable at the deployed roster of one. Left out of
this PR deliberately; it needs its own ticket.
Ledger rows M14-M16 and the write-up are in DECISIONS/SAFETY-30-per-unit-sign.md section 5, with a CHANGELOG bullet.
## Blocking finding closed: the WIRING is pinned now, not just the values
Fix pushed as `d5c1577`. The shipped code was not touched - the survivor was a
coverage gap, not a behaviour defect. Every run below is on **its own freshly
copied tree with no `__pycache__`, mounted `:ro`, graded through
`tools/mutation_run.sh`** (purge + fingerprint + `python3 -B`), per MUTATION.md.
### 1. The survivor, reproduced FIRST on the rejected head `f000547`
`arm(self.sign_cfg if one else self.unit_sign_cfg, ...)` -> `arm(self.sign_cfg, ...)`,
**-31 B**, the mutated line asserted present on disk before the run:
```
== mutated app/main.py 213844 B -> 213813 B (delta -31 B)
== the line ON DISK now:
804: inv.sign_wd.arm(self.sign_cfg,
== any bytecode in the tree: 0 entries
== purge
no bytecode cache under /w/goodwe_controller
== run (python3 -B)
PASS test_alloc.py (25s)
PASS test_arbiter.py (0s)
PASS test_arbitrage.py (6s)
PASS test_bytecode.py (1s)
PASS test_ci_gate.py (1s)
PASS test_clamp.py (1s)
PASS test_control.py (0s)
PASS test_freeze.py (34s)
PASS test_inverter.py (5s)
PASS test_loop.py (93s)
PASS test_maintenance.py (0s)
PASS test_p1.py (8s)
PASS test_peakguard.py (60s)
PASS test_price.py (1s)
PASS test_price_forecast.py (0s)
PASS test_static_tariff.py (4s)
PASS test_tariff.py (8s)
PASS test_validate.py (5s)
PASS test_watchdog.py (30s)
```
19 pass, 0 fail. Confirmed exactly as reported.
### 2. The same mutation on the fixed head - now RED, and the failure names the wiring
`M14`, same edit, **-31 B**, `app/main.py` fingerprint `e8562fcaadde68a0 / 213813 B`
against the unmutated `179bf7b57187f301 / 213844 B`:
```
PASS test_validate.py (5s)
FAIL test_watchdog.py (30s)
```
and part 3 of that file, verbatim - note the printed line, which is the pin
showing its working rather than asserting in the dark:
```
3. AC 1: ONE unit drives the wrong way, and ONLY that unit trips
armed per unit: {'inverter-a': (300.0, 3.0), 'inverter-b': (300.0, 3.0)}
armed site: (320.0, 2.0) shipped pair, wanted by NOBODY here: (300.0, 3.0)
ok PRECONDITION: the three pairs are pairwise distinct IN BOTH COORDINATES, so an equality below can fail on the magnitude alone, on the hold alone, or on both. This fixture originally gave all three the same 1 s hold and two of them the same 300 W gate, and a mis-wiring that delivered the right magnitude with the wrong hold would then have compared EQUAL to the pair it was supposed to have
ok PRECONDITION: and all three are pairs this Controller actually holds, so the literals below are the fixture's own numbers and not three constants that agree with nothing. In particular the shipped pair is still constructed and still a reachable mis-wiring target
FAIL ⚠️ AC 5 WIRING. EVERY per-unit detector on this roster of two is armed with the PER-UNIT pair - 250 W / 1 s here, `unit_sign_disagree_w` and `unit_sign_disagree_s` in the field. Graded against LITERALS rather than against `_c30.unit_sign_cfg`, which cannot tell 'the per-unit pair' from 'whatever this Controller was handed'. This is the line that goes red when `_arm_sign_watchdogs()` stops choosing between the two pairs and hands every unit the site-facing one
ok ...and the SITE detector is armed with the SITE pair - 320 W / 2 s here - which differs from the per-unit pair AND from the shipped pair in both coordinates, so this equality is not carried by a number the fixture happens to hold twice
ok ...and each detector is pointed at its own INSTRUMENT as well as at its own thresholds: the units at their own 0x0518 power, the site at the meter. They are two arguments of one call and a pin on one is not a pin on the other
```
Six further checks in that file go red as fallout (B no longer trips inside the
drive under a 3 s hold); the pin above is the one that names the defect.
### 3. THE FIXTURE HAD TO CHANGE, and that is the part worth reading
You were right that a bare equality would not have bitten. `_sign_site` set
`sign_disagree_s`, `unit_sign_disagree_s` **and** `site_sign_disagree_s` all to
`1`, with two of the three magnitude gates at 300 W - so the equality could only
discriminate on `min_w`, and a hold delivered from the wrong pair would have
compared EQUAL to the pair it was supposed to have. The three pairs are now
pairwise distinct **in both coordinates** - `250 W / 1 s` per unit,
`320 W / 2 s` site, `300 W / 3 s` shipped - and a PRECONDITION check asserts
that distinctness, so a later edit collapsing them again goes red rather than
leaving the claim quietly vacuous.
**Proof that the HOLD half bites, not only `min_w`.** Two hold-only mis-wirings,
each delivering the CORRECT 250 W with the wrong hold:
| # | wiring mutation | Δ | result |
|---|---|---|---|
| M15 | `arm(SignWatchdogConfig(min_w=self.unit_sign_cfg.min_w, hold_s=self.sign_cfg.hold_s))` | +78 B | **RED** - the AC 5 wiring pin |
| M16 | ...`hold_s=self.site_sign_cfg.hold_s` | +83 B | **RED** - the wiring pin, and the **only** red in `test_watchdog.py` |
M16 is the isolated case: its 2 s hold still expires inside the drive, so every
behavioural check in part 3 stays green and the pin is the sole discriminator.
```
armed per unit: {'inverter-a': (250.0, 2.0), 'inverter-b': (250.0, 2.0)}
armed site: (320.0, 2.0) shipped pair, wanted by NOBODY here: (300.0, 3.0)
FAIL ⚠️ AC 5 WIRING. EVERY per-unit detector on this roster of two is armed with the PER-UNIT pair - 250 W / 1 s here ...
```
```
1 FAILED: ⚠️ AC 5 WIRING. EVERY per-unit detector on this roster of two is armed with the PER-UNIT pair ...
```
(M16 also takes `test_inverter.py` red, because its own two-inverter fixture
borrows `inverters[0].sign_wd.cfg` and inherits the doubled hold.)
### 4. The unmutated tree at `d5c1577`, purged and fingerprinted
```
== purge
no bytecode cache under /w/goodwe_controller
== tree graded (sha256, bytes)
179bf7b57187f301 213844 app/main.py
0ad2408f8935970b 40709 app/watchdog.py
== run (python3 -B)
PASS test_alloc.py PASS test_arbiter.py PASS test_arbitrage.py
PASS test_bytecode.py PASS test_ci_gate.py PASS test_clamp.py
PASS test_control.py PASS test_freeze.py PASS test_inverter.py
PASS test_loop.py PASS test_maintenance.py PASS test_p1.py
PASS test_peakguard.py PASS test_price.py PASS test_price_forecast.py
PASS test_static_tariff.py PASS test_tariff.py PASS test_validate.py
PASS test_watchdog.py
```
**19 files, 19 pass, 0 fail.** (The last block is reflowed to three columns for
width; every other block on this comment is verbatim.)
### 5. On the field consequence, corrected
The finding put the survivor's cost at "180 s instead of 85 s - 3.8x". At
`config.yaml`'s own defaults the mutation arms the units with `sign_disagree_s`
**180 s** against the per-unit **85 s**, which is **2.1x**, and moves the gate
from 250 W to 300 W. (3.8x would be the site hold, 325 s, which this particular
edit does not deliver.) The direction and the conclusion are unchanged.
### 6. Not filed here, on purpose - wants a card
A **stuck per-unit power sensor** would false-trip the new per-unit check at
N>=2. Already-merged **SAFETY-31** has the identical exposure on the same
fixture, so this is a pre-existing N>=2 class rather than something this card
introduces, and it is unreachable at the deployed roster of one. Left out of
this PR deliberately; it needs its own ticket.
Ledger rows M14-M16 and the write-up are in
`DECISIONS/SAFETY-30-per-unit-sign.md` section 5, with a CHANGELOG bullet.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
CFG-04 (#45) merged first and moved the base. Two conflicts, both resolved
as the union of the two sides:
- app/main.py: an import-block collision. CFG-04 added validate_unit_ranges
to the .validate import; SAFETY-30 added AGAINST_METER, AGAINST_OWN_POWER,
SIGN_OPTION_DEFAULTS and SignWatchdog to the .watchdog import. Neither
side dropped anything, so the resolution keeps both lists whole.
- CHANGELOG.md: two adjacent new entries under ## Unreleased. Both kept.
Both sides also independently rewrote large parts of test_peakguard.py
(60 and 47 insertions against the merge base) and git merged them
textually, so the full suite was run on the MERGE RESULT rather than
inferred from the two parents: 19 passed, 0 failed, twice.
One caveat recorded rather than hidden: test_peakguard.py failed once
during the first sequential full-suite run and then passed 10/10 in
isolation and in two further full runs on this same tree. It passes on
both parents too, so this is the pre-existing flakiness CFG-04 already
disclosed, not a merge defect - it is timing- or load-sensitive. It sits
on the required CI context and can therefore block an unrelated merge at
random, which wants a card of its own.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
admin
merged commit 2d025eebf7 into release/1.02026-09-04 12:28:44 +02:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
The defect
watchdog.py's predicate was fed the same second argument for every inverter:stoodis the command this controller put on THIS actuator.self.gridiswhat the whole site is exchanging with the network. At a roster of one those
are two views of one thing — §6.2's identity — which is what makes the check
sound and what earned it the record NOTES.md describes: 93 % sign agreement while
healthy against 4 % during runaway, all four observed failure episodes, zero
false positives.
At two inverters they are not, and it is wrong in both directions:
The first row spends a record earned at N=1. The second is the guard being absent
in the case it exists for.
DEV-02 had already shipped the per-instance half — one
SignWatchdogperinverter, namespaced latch, a trip on B no longer latching A — and nothing here
re-tests it. What was left was the comparison.
What this changes
Two questions, each asked of the evidence that can answer it.
(
Inverter.batt, holding register 0x0518, per-instance since DEV-02).Conclusion: the actuator I am driving is not doing what I told it. A fact
about one device, so the remedy stays the per-unit
failsafeSET thatarbiter.CLAIM_SCOPEhas classifiedSCOPE_UNITsince it was written — thatentry read "the evidence is site-wide and the remedy is not", and after this
card the first half is simply true.
sign_siteSET atP_SAFETYon the site actuator, latched under flatsite_sign_tripped/site_sign_reason. This is the 2026-08-20 fault day, andno per-unit check can see it: every inverter may be faithfully executing
its share of a number that is itself wrong. It stops every inverter, and the
banner names the site rather than an inverter so nobody is sent to swap the
wrong box.
Controller._arm_sign_watchdogs()is the only place the roster decides anythinghere, and it is a pure function of
len(self.inverters).Which fault each guard covers — SAFETY-31 and the freeze, named
Two guards that overlap without saying so is how you get a false-alarm generator,
so both neighbours are written down in
watchdog.py, inDECISIONS/SAFETY-30-per-unit-sign.md§2, and asserted intest_watchdog.py.circulation.pyΣ|Bᵢ| − |ΣBᵢ|over measured powerThey genuinely overlap in one scenario — a unit reversing while its sibling
pulls the other way — and part 4 of
test_watchdog.pyasserts exactly that onthe same fixture, with both readings printed: circulating power is over its
threshold and SAFETY-30 names B. Neither is the other with a different
threshold.
It is also not the saturation freeze: that is a magnitude test
(
|command − measured| > saturation_w) that stops the integrator winding, fireson a merely saturated unit, and neither latches nor parks anything. This is a
direction test that latches and parks one. They meet at a number rather than
overlapping in a band nobody drew — see the threshold below.
The four numbers, re-derived at two inverters
Read off
admin/goodwe'ssim/two-master rig, used read-only and runcontainer-free. Real and imported, never transcribed: the control law, the
allocator, the sign predicate, the firmware's own encoder,
plantsim.Plant, andthe per-unit reading taken from register 0x0518 — the register the live
batt_entitybinds.1890 closed-loop runs in which nothing is faulty — every unit executes the
share the allocator gave it — over 6 gains × 5 slews (the whole range
config.yamlaccepts), across 1050 load reversals with write staggers and 840flapping loads. ⚠️ Every run is executed twice, once at each roster, so the N=1
arm is the control and this can separate "a property of the check that
shipped" from "a property of the second inverter".
unit_sign_disagree_wsaturation_w / 2. Both sides gated and opposed, so the smallest gap it can see is 500 W — exactly wherecompute()'s anti-windup test already says the inverter is not following. Worst legitimate opposition at 250 W: 10.0 sunit_sign_disagree_sinverter.LINK_STALE_S), under half the 180 s site holdsite_sign_disagree_wsite_sign_disagree_sThe number that refused to be carried
Worst continuous legitimate opposition, load-reversal family, at 300 W:
Carrying 180 s would have shipped a false trip, and it is reachable:
gain: 0.05andslew_w: 50are both schema floors, and ±2200 W is an oven and a heatpump. The mechanism predicts the ratio — the transient the hold exists to reject
is the loop crossing zero, and the command has to travel a rail that is the
roster sum at a
slew_wthat is not. 325 s is 1.51× the measured worst, insidethe 900 s billing quarter, and inside every episode of the fault day (its four
clusters run 16, 30, 77 and 77 minutes). Cost, stated rather than hidden: about
two and a half extra minutes of wrong-direction site power.
At one inverter, nothing changes — asserted, not assumed
Only the detector that shipped is armed. Same comparison against the meter, same
300 W / 180 s, same reason string byte for byte (asserted against the
literal), same namespaced latch and flat rollback mirror, same
failsafeclaim,same banner, same single status row, no
sign_siteclaim — and the site detectoris never called, counted on the object over twelve cycles. The four new
options do nothing at a roster of one.
Verified by breaking it
Thirteen mutations, each on its own freshly copied tree with no
__pycache__, mounted:ro, and the battery refuses to report anything unlessthe baseline is green. Twelve red on the first pass.
M4 survived and is recorded as a survivor. Arming the site detector at N=1 as
well left the suite green, because the check read
disagree_since is None: theunit detector trips first, parks the inverter, and the site's standing command is
then 0 W — so its window is never opened and "never evaluated" read identically
to "evaluated and found nothing". The assertion ran, the sentence was true, and
the data could not reach the failure it named. Replaced with a call counter on the
object; M4 is red now.
⚠️ The first battery also reported a RED BASELINE — one shared mutation
directory, rebuilt between runs, read by a container mid-copy. That whole run was
discarded rather than reported. Ledger, deltas and both stories:
DECISIONS/SAFETY-30-per-unit-sign.md§5.SAFETY-24's lo-only asymmetry proof re-run unchanged: 1296 combinations,0 negative floors,
hi+infon every claim, every claim on the SITEactuator.
Full suite: 19/19 green in
goodwe-addon-ci.What I am unsure about, and what a reviewer should look hardest at
site_sign_disagree_s: 325is a real slowdown of a P0 check, and it is thedecision I am least sure of. The alternative I rejected was keeping 180 s and
raising the site threshold to 1000 W, where the worst legitimate case is 85 s —
faster to act, better margin, and (I think) wrong, because it raises a noise
gate to reject a transient and leaves the site blind to a 900 W whole-site
reversal at a 4 kW site. Worth arguing with.
half period is about one control cycle holds the site comparison opposed for
~1000 s at both rosters. The N=1 control arm is what says this has nothing
to do with the second inverter, and the healthy captured day's longest opposed
run is 30 s, so I judged it out of scope — but it does mean the site check has
a shape of load it cannot survive, and somebody should decide whether that
wants a card now.
following) is kept for the per-unit comparison. A unit reversingduring a maintenance charge is therefore still the freeze's to notice, not
this module's. Conservative, and it preserves the whole no-false-positive
argument — but it is a deliberate hole and I have written it down rather than
closed it.
copied (
measured=False).unit_sign_disagree_sdepends on that most,site_sign_disagree_sless (it is dominated by the rail andslew_w), andneither threshold depends on it at all. SPIKE-04 is the revisit trigger; §4 of
the decision record says which figure is which.
validate.pydeliberately has NO rule pairingsite_sign_disagree_wwithmax_w, unlike the other two thresholds: the bound on the site command isController.site_max_w, the roster sum, andvalidate_startup_config()ishanded options rather than a roster. A rule against
max_wwould refuse alegal two-inverter config the day CFG-04 lands. The schema range is the bound
there, and I would rather be told if that is the wrong call.
means an operator sees two alarms for one event. Whether the banner should say
so in one sentence is a UI question I did not answer.
🤖 Generated with Claude Code
https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
THE DEFECT, in one line. `inv.sign_wd.evaluate(stood, self.grid, ...)` compared the command THIS controller put on ONE actuator against what the WHOLE SITE is exchanging with the network. At a roster of one those are two views of the same thing - specs.md §6.2's identity - and that is what makes the check sound and what earned it the best record in the safety table (NOTES.md: 93 % agreement while healthy against 4 % during runaway, four observed episodes, zero false positives). At two inverters they are not, and it fails in BOTH directions: FALSE TRIP inverter A commanded -1500 W while the meter reads +1500 W is evidence of nothing when inverter B is discharging 3000 W exactly as told. Nothing is faulty and the highest-value check fires. MISSED the converse, and it is worse. A unit commanded +1500 W and MEASURED at -1500 W, beside a sibling at +1500 W, leaves P1 exactly where a healthy site leaves it. The summation destroys the evidence and no threshold recovers it. DEV-02 had already shipped the per-INSTANCE half - one SignWatchdog per inverter, namespaced latch, a trip on B no longer latching A. What was left was the COMPARISON, and it is two questions rather than one: PER UNIT this unit's standing command against ITS OWN measured power (`Inverter.batt`, holding register 0x0518, per-instance since DEV-02). Conclusion: "the actuator I am driving is not doing what I told it" - a fact about one device, so the remedy stays the per-unit `failsafe` SET that arbiter.CLAIM_SCOPE has classified SCOPE_UNIT from the day it was written. That entry read "the evidence is site-wide and the remedy is not"; the first half is now simply true. SITE the SUM of the standing commands against the meter, as a new `sign_site` SET at P_SAFETY on the site actuator, latched under flat `site_sign_tripped` / `site_sign_reason`. This is the 2026-08-20 fault day and NO per-unit check can see it, because every inverter may be faithfully executing its share of a number that is itself wrong. It stops every inverter and the banner names the site. WHAT IT IS NOT, said here because two guards that overlap without saying so is how a false-alarm generator gets built. It is not SAFETY-31: circulation.py compares the units AGAINST EACH OTHER and catches them pushing power THROUGH each other, which nets to zero at the meter and about which no single command is evidence. This compares each unit against ITS OWN COMMAND. They overlap in exactly one scenario - a unit reversing while its sibling pulls the other way - and test_watchdog.py asserts on that fixture that both fire and what each concludes. It is also not the saturation freeze, which is a MAGNITUDE test (|command - measured| > 500 W) that shapes the loop and fires on a merely saturated unit; this is a DIRECTION test that latches and parks one. THE FOUR NUMBERS, RE-DERIVED AT TWO INVERTERS - one carried with a justification and one refused. 1890 closed-loop runs on admin/goodwe's TEST-19 two-master rig in which NOTHING is faulty, swept over the whole range config.yaml accepts for `gain` and `slew_w`, across load reversals with write staggers and flapping loads, and RUN AT BOTH ROSTERS so the N=1 arm is the control: unit_sign_disagree_w 250 `saturation_w / 2`. The check gates on both sides, opposed, so the smallest gap it can see is 500 W - exactly where compute()'s own anti-windup test already says the inverter is not following. It cannot fire on a divergence the control law calls ordinary tracking. Worst legitimate opposition at 250 W: 10.0 s; the curve is nearly flat, because a unit's own power chases its own command. unit_sign_disagree_s 85 8.5x that 10.0 s, 2.8x the firmware's own 30 s link veto (inverter.LINK_STALE_S), under half the 180 s site hold. site_sign_disagree_w 300 CARRIED. Its argument is the meter's noise floor and the 15 W deadband; the roster touches neither, and the campaign agrees - the gate opens in 994 of 1050 runs at two inverters and 955 of 1050 at one. site_sign_disagree_s 325 NOT CARRIED, and this is the number the card exists to not assume. The worst legitimate crossing at 300 W is 110 s with one inverter and 215 s with TWO, because the command must travel a rail that is the roster sum at a `slew_w` that is not - reachable at gain 0.05, slew_w 50 and a 2.2 kW load reversal, all schema-legal. 180 s would have shipped a false trip. 325 s is 1.51x the measured worst, inside the 900 s billing quarter and inside every episode of the fault day. AT ONE INVERTER NOTHING CHANGES, and "nothing" is asserted rather than assumed. The two questions are the same sentence there, so only the detector that shipped is armed: same comparison against the meter, same 300 W / 180 s, same reason string byte for byte, same namespaced latch and flat rollback mirror, same `failsafe` claim, same banner, same single status row - and the site detector is NEVER CALLED, counted on the object over twelve cycles. VERIFIED BY BREAKING IT. Thirteen mutations on fresh per-mutation trees with no __pycache__, mounted :ro, with the battery refusing to report at all unless the baseline is green. Twelve red on the first pass. M4 - the site half armed at N=1 as well - SURVIVED, because the check read `disagree_since is None` and the unit detector trips first, parks the inverter and leaves the site standing at 0 W: the assertion ran, the sentence was true, and the data could not reach the failure it named. Replaced with a call counter on the object; M4 is red now. Recorded as a survivor in DECISIONS/SAFETY-30-per-unit-sign.md §5, with the first battery's own RED BASELINE - a shared mutation directory read mid-copy - discarded rather than reported. RECORDED, NOT FIXED: the site comparison is defeated by a square-wave load whose half period is about one control cycle, which holds it opposed for ~1000 s. The N=1 control arm is what makes that interpretable - it is identical at the roster this add-on ships, so it is a property of comparing a lagged command against a meter and has nothing to do with the second inverter. It wants an alternation test rather than a number, and a card of its own. SAFETY-24's lo-only asymmetry proof re-run unchanged: 1296 combinations, 0 negative floors, `hi` +inf on every claim, every claim on the SITE actuator. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjUREVIEW FOUND A MUTATION SURVIVOR THAT UNDOES THE WHOLE PRODUCT OF AC 5, and it is a coverage defect rather than a behaviour one - the shipped code arms the detectors correctly today. In `_arm_sign_watchdogs()`, collapsing inv.sign_wd.arm(self.sign_cfg if one else self.unit_sign_cfg, ...) to `arm(self.sign_cfg, ...)` throws the re-derived 250 W / 85 s per-unit pair away and arms every unit with SAFETY-02's meter-facing 300 W / 180 s - and left ALL 19 FILES GREEN. Reproduced here before fixing anything: fresh per-mutation tree, no __pycache__, mounted :ro, graded through tools/mutation_run.sh; -31 B; 19 pass, 0 fail. WHY NOTHING CAUGHT IT. `grep -rn "sign_wd.cfg" test_*.py` found exactly ONE threshold assertion, part 7's N=1 arm at (300.0, 1.0) - which the mutation PRESERVES. Nothing at a roster of two read `inv.sign_wd.cfg` at all. The ledger mutates the VALUES in SIGN_OPTION_DEFAULTS (M10 250->300, M11 325->180, both correctly red through the config pins) and never the WIRING that delivers them. Part 3's behavioural checks cannot close the gap either, and the comment now says why: B is reversed at ~2000 W with short holds, so it trips under EITHER pair, and a fault big enough and long enough to trip under both thresholds is evidence about neither. THE FIX - test_watchdog.py part 3, on the real two-inverter Controller. Every `inv.sign_wd.cfg` equals the per-unit pair, `site_sign_wd.cfg` equals the site pair, and each detector's `against` is its own instrument. Graded against LITERALS, because `== self.unit_sign_cfg` cannot tell "the per-unit pair" from "whatever this Controller was handed". Both pairs are printed. ⚠️ AND THE FIXTURE HAD TO CHANGE FOR THE PIN TO MEAN ANYTHING - the recursion worth naming, because it is the same defect class as the one being closed. `_sign_site` set `sign_disagree_s`, `unit_sign_disagree_s` AND `site_sign_disagree_s` all to 1 s, with two of the three magnitude gates at 300 W. A bare equality there discriminates on `min_w` alone: a mis-wiring that delivered the right magnitude with the WRONG HOLD compares EQUAL to the pair it was supposed to have. A check comparing two things that happen to be equal cannot see the difference it names (MUTATION.md's third failure mode). The three pairs are now pairwise distinct IN BOTH COORDINATES - 250 W / 1 s per unit, 320 W / 2 s site, 300 W / 3 s shipped - a PRECONDITION asserts that distinctness so a later edit collapsing them goes red rather than quietly vacuous, and a second PRECONDITION asserts the literals are the pairs this Controller actually holds. VERIFIED BY BREAKING IT. Three mutations, each on its own freshly copied tree with no __pycache__, mounted :ro, through tools/mutation_run.sh: M14 arm() handed self.sign_cfg at every roster -31 B RED (was the survivor; the AC 5 WIRING check names the wiring) M15 right min_w, SHIPPED hold - hold-only mis-wire +78 B RED, same pin M16 right min_w, SITE hold - hold-only, and it still trips inside the drive +83 B RED, and the ONLY red in test_watchdog.py M16 is the one that says the hold half of the equality bites on its own. Unmutated tree: 19 pass, 0 fail, purged and fingerprinted. Ledger rows M14-M16 and the write-up in DECISIONS/SAFETY-30-per-unit-sign.md section 5; a CHANGELOG bullet. NOT IN THIS PR, so it gets a card rather than a quiet fix: a stuck per-unit power sensor would false-trip the new per-unit check at N>=2. Already-merged SAFETY-31 has the identical exposure on the same fixture. Pre-existing N>=2 class, unreachable at the deployed roster of one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjUBlocking finding closed: the WIRING is pinned now, not just the values
Fix pushed as
d5c1577. The shipped code was not touched - the survivor was acoverage gap, not a behaviour defect. Every run below is on its own freshly
copied tree with no
__pycache__, mounted:ro, graded throughtools/mutation_run.sh(purge + fingerprint +python3 -B), per MUTATION.md.1. The survivor, reproduced FIRST on the rejected head
f000547arm(self.sign_cfg if one else self.unit_sign_cfg, ...)->arm(self.sign_cfg, ...),-31 B, the mutated line asserted present on disk before the run:
19 pass, 0 fail. Confirmed exactly as reported.
2. The same mutation on the fixed head - now RED, and the failure names the wiring
M14, same edit, -31 B,app/main.pyfingerprinte8562fcaadde68a0 / 213813 Bagainst the unmutated
179bf7b57187f301 / 213844 B:and part 3 of that file, verbatim - note the printed line, which is the pin
showing its working rather than asserting in the dark:
Six further checks in that file go red as fallout (B no longer trips inside the
drive under a 3 s hold); the pin above is the one that names the defect.
3. THE FIXTURE HAD TO CHANGE, and that is the part worth reading
You were right that a bare equality would not have bitten.
_sign_sitesetsign_disagree_s,unit_sign_disagree_sandsite_sign_disagree_sall to1, with two of the three magnitude gates at 300 W - so the equality could onlydiscriminate on
min_w, and a hold delivered from the wrong pair would havecompared EQUAL to the pair it was supposed to have. The three pairs are now
pairwise distinct in both coordinates -
250 W / 1 sper unit,320 W / 2 ssite,300 W / 3 sshipped - and a PRECONDITION check assertsthat distinctness, so a later edit collapsing them again goes red rather than
leaving the claim quietly vacuous.
Proof that the HOLD half bites, not only
min_w. Two hold-only mis-wirings,each delivering the CORRECT 250 W with the wrong hold:
arm(SignWatchdogConfig(min_w=self.unit_sign_cfg.min_w, hold_s=self.sign_cfg.hold_s))hold_s=self.site_sign_cfg.hold_stest_watchdog.pyM16 is the isolated case: its 2 s hold still expires inside the drive, so every
behavioural check in part 3 stays green and the pin is the sole discriminator.
(M16 also takes
test_inverter.pyred, because its own two-inverter fixtureborrows
inverters[0].sign_wd.cfgand inherits the doubled hold.)4. The unmutated tree at
d5c1577, purged and fingerprinted19 files, 19 pass, 0 fail. (The last block is reflowed to three columns for
width; every other block on this comment is verbatim.)
5. On the field consequence, corrected
The finding put the survivor's cost at "180 s instead of 85 s - 3.8x". At
config.yaml's own defaults the mutation arms the units withsign_disagree_s180 s against the per-unit 85 s, which is 2.1x, and moves the gate
from 250 W to 300 W. (3.8x would be the site hold, 325 s, which this particular
edit does not deliver.) The direction and the conclusion are unchanged.
6. Not filed here, on purpose - wants a card
A stuck per-unit power sensor would false-trip the new per-unit check at
N>=2. Already-merged SAFETY-31 has the identical exposure on the same
fixture, so this is a pre-existing N>=2 class rather than something this card
introduces, and it is unreachable at the deployed roster of one. Left out of
this PR deliberately; it needs its own ticket.
Ledger rows M14-M16 and the write-up are in
DECISIONS/SAFETY-30-per-unit-sign.mdsection 5, with a CHANGELOG bullet.🤖 Generated with Claude Code
https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU