Compare commits

..
8 Commits
Author SHA1 Message Date
adminandClaude Opus 5 905b6fd2b3 MAINT-05 fix round 2: the refusal has to survive the NEXT start, and both marker guards have to be visible
checks / unit (pull_request) Successful in 5m14s
REJECTED because the fix round's own load-bearing sentence was FALSE, and the
whole suite was blind to that. `_verify_moves()` produced the red the reviewer
reproduced - that half stood - but DECISIONS §2 claimed "`ok` gates the marker,
so a misdirected migration is not stamped and the next start runs it again ...
a re-run is recoverable". THE REFUSAL LASTED EXACTLY ONE START.

MEASURED, Controller level, ONE persisted store, roster ["garage","shed"], flat
last_completed 2026-08-07, call site still handing `inverter-a` on both starts:

    construction 1: ok=False misdirected=4 moved=4 kept=0   marker=<absent>
    construction 2: ok=True  misdirected=0 moved=0 kept=4   marker=1
    history parked under unit/inverter-a/...; marker stamped after ONE restart.

WHY. `_verify_moves()` graded only `rep.moved`. On start 2 the same wrong call
site finds the four keys already present, so rule 1 routes every one of them to
`rep.kept`, `rep.moved` is EMPTY, the roster loop iterates over nothing,
`misdirected` is empty and `ok` is True. Start 3 then takes the
`from_version >= MAINT_SCHEMA_VERSION` early return FOREVER - including after
the call site is fixed. Loud-and-recoverable became silent-and-permanent after
one restart: the exact state store.py's header says must never happen, reached
by the fix written to prevent it.

WHY THE CHECK MISSED IT - the recurring defect, this time INSIDE the remedy.
Part 9b's recovery check was `migrate_maint_keys(_m5, ROSTER[0], ROSTER)` - the
CORRECTED name. It measures recovery along the one path where the operator has
ALREADY fixed the bug. Nothing in the 19 files called `migrate_maint_keys()`
twice with the SAME WRONG `unit`. The check was true and passed for the wrong
reason.

WHAT CHANGED

1. app/store.py `_verify_moves()` grades `set(rep.moved) | set(rep.kept)`.
   Sound by construction: every entry in both lists is a `unit_key(unit,
   legacy)` built from the SAME `unit` argument, so grading `kept` grades the
   same namespace decision `moved` does. A correct call site's `kept` entries
   sit under a roster name and pass; a wrong one cannot launder its namespace by
   having already written it. `migrate_unit_keys()` passes `roster=None` and
   returns before the loop, so it is untouched.

2. test_maintenance.py part 9c - THE CHECK THAT WAS MISSING. Two starts, SAME
   wrong unit, ONE store: the marker is still withheld on the second, the
   refusal names the same roster, the flat history is still there to recover
   from, and a third start with the CORRECTED name still gets a run (which a
   stamped marker would have made impossible). Its PRECONDITION asserts the
   SHAPE of the second report - `moved` empty, `kept` full - so it cannot
   quietly revert to grading 9b's path. Counterfactual beside it: the same two
   calls with the right name are accepted, then skipped as `already at schema`.

3. test_maintenance.py part 9d - THE OTHER GUARD ON THE SAME LINE, and a
   survivor nothing in the repository could see. `if rep.ok:` is
   `not lost and not misdirected`; every check that reached the marker did so on
   a store whose `kept` was empty, so

       if rep.ok:        ->      if rep.ok or rep.kept:        (+12 B)

   stamped the marker on a migration that LOST A MONTH OF HISTORY - defeating
   the older, settled `lost` guard as well as the new one - and ALL 19 SUITES
   WERE GREEN on it (measured at 65cddf3, purged, :ro). 9d's fixture is the one
   shape that separates the guards: two keys already namespaced (`kept`
   non-empty, the mutant's `or` fires) and two whose write the store drops
   (`lost` non-empty), `misdirected` empty throughout.

4. FOUND HERE AND CLOSED: THE SAME MUTATION ONE FUNCTION UP. `migrate_unit_keys()`
   - SAFETY-02's LATCH migration - gates its own marker the same way, and every
   check that reached it ran on a store with an empty `kept`. `if rep.ok or
   rep.kept:` at store.py:307 SURVIVED ALL 19 FILES. It stamps `schema_version`
   on a migration that DROPPED THE LATCH: the guard is off, the file still reads
   `sign_tripped: true`, and the store now says the migration finished, so the
   next start skips the one thing that would re-arm it. test_inverter.py part 10
   now drops the `sign_tripped` write BESIDE an already-namespaced
   `sign_reason`, with the accepted-baseline counterfactual. No source change -
   the shipped gate was right; nothing could see it. Mutation table 34 -> 35.

5. DECISIONS/MAINT-05 §2 CORRECTED. The paragraph that stated the false
   guarantee now states what was measured, why grading `moved` alone could not
   hold, and why the union is sound. §7 records the marker-gate hole found in
   `migrate_unit_keys()` and that it was closed. MUTATION.md gains habit 9 -
   "exercise a refusal a SECOND time with the SAME wrong input, not the
   corrected one" - and two survey rows.

MUTATION LEDGER - 11 mutations, ALL RED on the final tree, every run on a fresh
copy with the bytecode cache purged (entry count printed before and after) and
the tree mounted :ro so none could be written. Two were survivors BEFORE this
commit and both are recorded as such. Full transcript on the PR.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-04 19:40:51 +02:00
glenn schrooyenandClaude Opus 5 65cddf35f1 MAINT-05 fix round: grade the migration against the ROSTER, not against the name it was handed
checks / unit (pull_request) Successful in 4m47s
REJECTED because a survivor undoes AC 5, and it survived for the recurring
reason: two fixture values that must be distinguishable were EQUAL.

THE SURVIVOR. app/main.py:282 mutated from `self.inverters[0].name` to `UNIT_A`
left the full 19-file suite green. `inverter.UNIT_A == "inverter-a"`, and every
two-pack fixture in test_maintenance.py named its first roster entry
"inverter-a" - so the two expressions were the same string in every check that
existed. main.py's own comment made the distinction load-bearing and nothing in
the repository could see whether that comment was true.

THE FAILURE IS REAL. `build_roster()` names a declared roster's entries from the
config and numbers a nameless one POSITIONALLY (`inverter-1`), so "inverter-a"
is not even the fallback. On ["garage","shed"] with a flat last_completed of
2026-08-07 the mutant migrates into unit/inverter-a/, pack 0 reads back None,
and both packs come up believing they have never been recalibrated.

AND IT FAILED SILENTLY, which was the worse half: `_verify_moves()` read back
the keys it had just WRITTEN, under the namespace it had just been HANDED, was
satisfied, and MAINT_SCHEMA_KEY was stamped - so the next start skipped the
migration too. A postcondition that reads back the namespace it was handed
cannot detect being handed the wrong one.

WHAT CHANGED

1. THE POSTCONDITION IS GRADED AGAINST THE ROSTER. `_verify_moves()` takes the
   names the MACHINES read and reports a namespace that is not among them as
   `misdirected` - a new, differently-named failure from `lost`:
     * `lost` is "the store took the write and dropped it" - a storage fault,
       with a harmless-default case (a dropped `last_completed: None` loses
       nothing, because absent and never-recalibrated are the same statement);
     * `misdirected` is "the store did exactly as it was told, by a caller that
       was wrong". There is no harmless-default version of it - a write nobody
       reads is not information whatever its value - so it fails `ok` alone.
   `ok` gates the marker, so a misdirected migration is NOT stamped and the next
   start re-runs it. Asymmetric on purpose: a re-run is recoverable (rule 1
   makes it a no-op, rule 2 left the flat keys to recover from), a silent skip
   is not. Part 9b MEASURES the recovery rather than asserting it.

2. `migrate_maint_keys()` TAKES THE ROSTER IN PLACE OF AN `n_units` COUNT. The
   count was `len(self.inverters)` at the only call site, so the roster carries
   it, and a count beside the names is a second opinion about how many packs
   there are.

3. THE FIXTURES CAN NOW REACH THE FAILURE THEY NAME. Parts 9 and 11 run on
   ["garage","shed"], each behind a PRECONDITION check that the first name is
   not UNIT_A - so an edit that makes them equal again goes red instead of
   making the claims vacuous. Part 11's fixture date is relative to `now`, not
   a literal, so "history preserved" and "history lost" cannot converge as the
   file ages past `interval_days`.

MUTATION LEDGER - all red, purged tree, none at +0 B, full detail on the PR:
   M1  main.py  inverters[0].name -> UNIT_A            -16 B  4 checks red
   M2  store.py drop `roster=names` from _verify_moves -14 B  5 checks red
   M3  store.py drop the misdirected.append            -37 B  5 checks red
   M4  store.py ok: drop `and not self.misdirected`    -25 B  3 checks red
   M5  fixture  garage -> inverter-a (the trap itself)  +4 B  5 checks red
   M6  store.py parts[1] -> parts[-1]                   +1 B  1 check red

FOUND AND NOT FIXED - `migrate_unit_keys()` HAS THE SAME HOLE AND IT IS A LATCH.
It runs BEFORE `build_roster()` (each Inverter restores its latch in its
constructor), so it is handed the constant UNIT_A with no roster to grade
against. On a declared roster a pre-DEV-02 `sign_tripped: true` lands under
unit/inverter-a/sign_tripped, which no unit reads - and
`_grade_store_migration()`, whose whole job is to re-arm exactly this, is
DEFEATED BY THE KEY THE DEFECT WROTE: its "is anything namespaced latched" test
is satisfied by the misdirected value itself. Measured, with the flat-config
counterfactual, and pinned as a running check (test_maintenance.py part 13)
whose assertions state the WRONG behaviour on purpose. It wants its own card:
the remedy is in inverter.py and SAFETY-02's semantics, and a silent expansion
into a safety latch is how a rejected PR becomes a broken one.

Suite: 19/19 PASS, purged tree, bind-mounted :ro in goodwe-addon-ci.
No hardware was touched; every fixture here is SIMULATED and pack B's 3000 W
remains a marked hypothesis.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-04 16:18:34 +02:00
adminandClaude Opus 5 1aef19e995 MAINT-05: one recalibration per pack, and only one pack moves power at a time
checks / unit (pull_request) Successful in 4m48s
§12's monthly cycle drains a pack to empty and refills it to true its coulomb
counter. There was exactly one `Maintenance` and one un-namespaced history, so
at two packs only the first was ever recalibrated - and nothing prevented the
cycle draining pack A while the loop charged pack B, which is two power-moving
phases on one grid connection pulling against each other.

* ONE `Maintenance` PER ROSTER ENTRY, each with its own schedule, phase and
  history under `unit/<name>/...`. `store.migrate_maint_keys()` moves an
  install's flat history into the first pack's namespace under
  `migrate_unit_keys()`'s three rules - shared in code via `_move_flat_keys()`
  and `_verify_moves()`, not restated - on its own marker, because unlike
  DEV-02's it runs only where the roster has more than one pack. The flat keys
  are kept LIVE as the rollback view.
* ONE MAINTENANCE SLOT PER SITE (`maintenance.arbitrate()`). At most one pack
  in a power-moving phase - drain, charge AND hold. The holder is DERIVED from
  the packs' own stored phases, never persisted, so a restart mid-drain
  re-derives it. Several packs due in the same hour: the slot goes to the one
  that has gone longest without a completed cycle; roster order breaks a
  numeric tie and nothing else. Nothing preempts a cycle in flight.
* THE DEFERRED PACK IS NOT STARVED AND IS NOT A FAULT. A deferral does not
  stamp `last_start_attempt` and does not move the phase, so neither the 12 h
  drain nor the 36 h charge timeout is reachable by waiting.
* The claim is filed on the pack whose state machine produced it, and stays
  SCOPE_UNIT - ARCH-01's rule.
* `validate.py` rule 6 grades `maintenance_charge_w` against the WEAKEST cap on
  the roster now, as its own comment said it must once `maint_unit` stopped
  being `inverters[0]`.
* At a roster of one nothing changes, and it is asserted: no new store key, the
  migration does not run, the four cycle events are the shipped strings, the
  banner is the bare phase.
* RECORDED NOT FIXED: a drain pins its pack while the loop's target is a site
  claim, so below the pin the allocator refuses the split and the site parks at
  0 W. Pre-existing at 2d025ee. Pinned as a running check.

Verified: 19/19 suites green in the CI image; 17 mutations run through
tools/mutation_run.sh (bytecode purged), every one observed red.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-04 13:38:29 +02:00
admin 2d025eebf7 Merge pull request 'SAFETY-30: grade each inverter against its own power, and the site's total against the meter' (#46) from SAFETY-30 into release/1.0 2026-09-04 12:28:44 +02:00
glenn schrooyenandClaude Opus 5 ae21b71824 Merge release/1.0 into SAFETY-30
checks / unit (pull_request) Successful in 4m39s
CFG-04 (#45) merged first and moved the base. Two conflicts, both resolved
as the union of the two sides:

- app/main.py: an import-block collision. CFG-04 added validate_unit_ranges
  to the .validate import; SAFETY-30 added AGAINST_METER, AGAINST_OWN_POWER,
  SIGN_OPTION_DEFAULTS and SignWatchdog to the .watchdog import. Neither
  side dropped anything, so the resolution keeps both lists whole.
- CHANGELOG.md: two adjacent new entries under ## Unreleased. Both kept.

Both sides also independently rewrote large parts of test_peakguard.py
(60 and 47 insertions against the merge base) and git merged them
textually, so the full suite was run on the MERGE RESULT rather than
inferred from the two parents: 19 passed, 0 failed, twice.

One caveat recorded rather than hidden: test_peakguard.py failed once
during the first sequential full-suite run and then passed 10/10 in
isolation and in two further full runs on this same tree. It passes on
both parents too, so this is the pre-existing flakiness CFG-04 already
disclosed, not a merge defect - it is timing- or load-sensitive. It sits
on the required CI context and can therefore block an unrelated merge at
random, which wants a card of its own.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-04 12:23:50 +02:00
admin 18f052a071 Merge pull request 'CFG-04: an operator can declare a second inverter, and the shape was measured against the Supervisor' (#45) from CFG-04 into release/1.0 2026-09-04 11:51:57 +02:00
glenn schrooyenandClaude Opus 5 d5c1577212 SAFETY-30: pin WHICH thresholds each sign detector is armed with, not just what the numbers are
checks / unit (pull_request) Successful in 4m48s
REVIEW FOUND A MUTATION SURVIVOR THAT UNDOES THE WHOLE PRODUCT OF AC 5, and it
is a coverage defect rather than a behaviour one - the shipped code arms the
detectors correctly today. In `_arm_sign_watchdogs()`, collapsing

    inv.sign_wd.arm(self.sign_cfg if one else self.unit_sign_cfg, ...)

to `arm(self.sign_cfg, ...)` throws the re-derived 250 W / 85 s per-unit pair
away and arms every unit with SAFETY-02's meter-facing 300 W / 180 s - and left
ALL 19 FILES GREEN. Reproduced here before fixing anything: fresh per-mutation
tree, no __pycache__, mounted :ro, graded through tools/mutation_run.sh; -31 B;
19 pass, 0 fail.

WHY NOTHING CAUGHT IT. `grep -rn "sign_wd.cfg" test_*.py` found exactly ONE
threshold assertion, part 7's N=1 arm at (300.0, 1.0) - which the mutation
PRESERVES. Nothing at a roster of two read `inv.sign_wd.cfg` at all. The ledger
mutates the VALUES in SIGN_OPTION_DEFAULTS (M10 250->300, M11 325->180, both
correctly red through the config pins) and never the WIRING that delivers them.
Part 3's behavioural checks cannot close the gap either, and the comment now
says why: B is reversed at ~2000 W with short holds, so it trips under EITHER
pair, and a fault big enough and long enough to trip under both thresholds is
evidence about neither.

THE FIX - test_watchdog.py part 3, on the real two-inverter Controller. Every
`inv.sign_wd.cfg` equals the per-unit pair, `site_sign_wd.cfg` equals the site
pair, and each detector's `against` is its own instrument. Graded against
LITERALS, because `== self.unit_sign_cfg` cannot tell "the per-unit pair" from
"whatever this Controller was handed". Both pairs are printed.

⚠️ AND THE FIXTURE HAD TO CHANGE FOR THE PIN TO MEAN ANYTHING - the recursion
worth naming, because it is the same defect class as the one being closed.
`_sign_site` set `sign_disagree_s`, `unit_sign_disagree_s` AND
`site_sign_disagree_s` all to 1 s, with two of the three magnitude gates at
300 W. A bare equality there discriminates on `min_w` alone: a mis-wiring that
delivered the right magnitude with the WRONG HOLD compares EQUAL to the pair it
was supposed to have. A check comparing two things that happen to be equal
cannot see the difference it names (MUTATION.md's third failure mode). The
three pairs are now pairwise distinct IN BOTH COORDINATES - 250 W / 1 s per
unit, 320 W / 2 s site, 300 W / 3 s shipped - a PRECONDITION asserts that
distinctness so a later edit collapsing them goes red rather than quietly
vacuous, and a second PRECONDITION asserts the literals are the pairs this
Controller actually holds.

VERIFIED BY BREAKING IT. Three mutations, each on its own freshly copied tree
with no __pycache__, mounted :ro, through tools/mutation_run.sh:

  M14  arm() handed self.sign_cfg at every roster        -31 B  RED (was the
       survivor; the AC 5 WIRING check names the wiring)
  M15  right min_w, SHIPPED hold - hold-only mis-wire    +78 B  RED, same pin
  M16  right min_w, SITE hold - hold-only, and it still
       trips inside the drive                            +83 B  RED, and the
       ONLY red in test_watchdog.py

M16 is the one that says the hold half of the equality bites on its own.
Unmutated tree: 19 pass, 0 fail, purged and fingerprinted.

Ledger rows M14-M16 and the write-up in DECISIONS/SAFETY-30-per-unit-sign.md
section 5; a CHANGELOG bullet.

NOT IN THIS PR, so it gets a card rather than a quiet fix: a stuck per-unit
power sensor would false-trip the new per-unit check at N>=2. Already-merged
SAFETY-31 has the identical exposure on the same fixture. Pre-existing N>=2
class, unreachable at the deployed roster of one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-04 11:13:13 +02:00
adminandClaude Opus 5 f000547518 SAFETY-30: grade each inverter against its own power, and the site's total against the meter
checks / unit (pull_request) Successful in 4m38s
THE DEFECT, in one line. `inv.sign_wd.evaluate(stood, self.grid, ...)` compared
the command THIS controller put on ONE actuator against what the WHOLE SITE is
exchanging with the network. At a roster of one those are two views of the same
thing - specs.md §6.2's identity - and that is what makes the check sound and
what earned it the best record in the safety table (NOTES.md: 93 % agreement
while healthy against 4 % during runaway, four observed episodes, zero false
positives). At two inverters they are not, and it fails in BOTH directions:

  FALSE TRIP  inverter A commanded -1500 W while the meter reads +1500 W is
              evidence of nothing when inverter B is discharging 3000 W exactly
              as told. Nothing is faulty and the highest-value check fires.
  MISSED      the converse, and it is worse. A unit commanded +1500 W and
              MEASURED at -1500 W, beside a sibling at +1500 W, leaves P1
              exactly where a healthy site leaves it. The summation destroys the
              evidence and no threshold recovers it.

DEV-02 had already shipped the per-INSTANCE half - one SignWatchdog per
inverter, namespaced latch, a trip on B no longer latching A. What was left was
the COMPARISON, and it is two questions rather than one:

  PER UNIT  this unit's standing command against ITS OWN measured power
            (`Inverter.batt`, holding register 0x0518, per-instance since
            DEV-02). Conclusion: "the actuator I am driving is not doing what I
            told it" - a fact about one device, so the remedy stays the per-unit
            `failsafe` SET that arbiter.CLAIM_SCOPE has classified SCOPE_UNIT
            from the day it was written. That entry read "the evidence is
            site-wide and the remedy is not"; the first half is now simply true.
  SITE      the SUM of the standing commands against the meter, as a new
            `sign_site` SET at P_SAFETY on the site actuator, latched under flat
            `site_sign_tripped` / `site_sign_reason`. This is the 2026-08-20
            fault day and NO per-unit check can see it, because every inverter
            may be faithfully executing its share of a number that is itself
            wrong. It stops every inverter and the banner names the site.

WHAT IT IS NOT, said here because two guards that overlap without saying so is
how a false-alarm generator gets built. It is not SAFETY-31: circulation.py
compares the units AGAINST EACH OTHER and catches them pushing power THROUGH
each other, which nets to zero at the meter and about which no single command is
evidence. This compares each unit against ITS OWN COMMAND. They overlap in
exactly one scenario - a unit reversing while its sibling pulls the other way -
and test_watchdog.py asserts on that fixture that both fire and what each
concludes. It is also not the saturation freeze, which is a MAGNITUDE test
(|command - measured| > 500 W) that shapes the loop and fires on a merely
saturated unit; this is a DIRECTION test that latches and parks one.

THE FOUR NUMBERS, RE-DERIVED AT TWO INVERTERS - one carried with a justification
and one refused. 1890 closed-loop runs on admin/goodwe's TEST-19 two-master rig
in which NOTHING is faulty, swept over the whole range config.yaml accepts for
`gain` and `slew_w`, across load reversals with write staggers and flapping
loads, and RUN AT BOTH ROSTERS so the N=1 arm is the control:

  unit_sign_disagree_w  250   `saturation_w / 2`. The check gates on both sides,
                              opposed, so the smallest gap it can see is 500 W -
                              exactly where compute()'s own anti-windup test
                              already says the inverter is not following. It
                              cannot fire on a divergence the control law calls
                              ordinary tracking. Worst legitimate opposition at
                              250 W: 10.0 s; the curve is nearly flat, because a
                              unit's own power chases its own command.
  unit_sign_disagree_s   85   8.5x that 10.0 s, 2.8x the firmware's own 30 s
                              link veto (inverter.LINK_STALE_S), under half the
                              180 s site hold.
  site_sign_disagree_w  300   CARRIED. Its argument is the meter's noise floor
                              and the 15 W deadband; the roster touches neither,
                              and the campaign agrees - the gate opens in 994 of
                              1050 runs at two inverters and 955 of 1050 at one.
  site_sign_disagree_s  325   NOT CARRIED, and this is the number the card
                              exists to not assume. The worst legitimate
                              crossing at 300 W is 110 s with one inverter and
                              215 s with TWO, because the command must travel a
                              rail that is the roster sum at a `slew_w` that is
                              not - reachable at gain 0.05, slew_w 50 and a
                              2.2 kW load reversal, all schema-legal. 180 s
                              would have shipped a false trip. 325 s is 1.51x
                              the measured worst, inside the 900 s billing
                              quarter and inside every episode of the fault day.

AT ONE INVERTER NOTHING CHANGES, and "nothing" is asserted rather than assumed.
The two questions are the same sentence there, so only the detector that shipped
is armed: same comparison against the meter, same 300 W / 180 s, same reason
string byte for byte, same namespaced latch and flat rollback mirror, same
`failsafe` claim, same banner, same single status row - and the site detector is
NEVER CALLED, counted on the object over twelve cycles.

VERIFIED BY BREAKING IT. Thirteen mutations on fresh per-mutation trees with no
__pycache__, mounted :ro, with the battery refusing to report at all unless the
baseline is green. Twelve red on the first pass. M4 - the site half armed at N=1
as well - SURVIVED, because the check read `disagree_since is None` and the unit
detector trips first, parks the inverter and leaves the site standing at 0 W:
the assertion ran, the sentence was true, and the data could not reach the
failure it named. Replaced with a call counter on the object; M4 is red now.
Recorded as a survivor in DECISIONS/SAFETY-30-per-unit-sign.md §5, with the
first battery's own RED BASELINE - a shared mutation directory read mid-copy -
discarded rather than reported.

RECORDED, NOT FIXED: the site comparison is defeated by a square-wave load whose
half period is about one control cycle, which holds it opposed for ~1000 s. The
N=1 control arm is what makes that interpretable - it is identical at the roster
this add-on ships, so it is a property of comparing a lagged command against a
meter and has nothing to do with the second inverter. It wants an alternation
test rather than a number, and a card of its own.

SAFETY-24's lo-only asymmetry proof re-run unchanged: 1296 combinations, 0
negative floors, `hi` +inf on every claim, every claim on the SITE actuator.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-04 09:33:20 +02:00
18 changed files with 5198 additions and 264 deletions
+199
View File
@@ -2,6 +2,205 @@
## Unreleased
**Every pack is recalibrated on its own schedule, and only one moves power at a
time (MAINT-05).** §12's monthly cycle drains a pack to empty and refills it to
true its coulomb counter. There was exactly one `Maintenance` state machine and
one un-namespaced history, so at two packs **only the first was ever
recalibrated** and the second's counter drifted forever — and nothing prevented
the cycle draining pack A while the loop or arbitrage charged pack B, which is
two power-moving phases on one grid connection pulling against each other.
* **ONE `Maintenance` PER ROSTER ENTRY**, each with its own schedule, phase and
history under `unit/<name>/...`. `store.migrate_maint_keys()` moves an
existing install's flat history into the first pack's namespace under
`migrate_unit_keys()`'s three rules — shared in code, not restated — with the
same read-back postcondition, and the flat keys are kept **live** as the
rollback view so a downgrade finds a current history rather than one dated
migration day.
* **ONE MAINTENANCE SLOT PER SITE.** At most one pack may be in a power-moving
phase (`drain`, `charge` **and** `hold` — `charge` is in the set because a
pack pulled full from the grid beside a pack exporting to empty is 4 kW
circulating with the meter at zero). The holder is **derived** from the packs'
own stored phases, not persisted, so a restart mid-drain re-derives it. When
several packs are due in the same hour the slot goes to the one that has gone
**longest without a completed cycle**; roster order breaks a numeric tie and
nothing else. Nothing preempts a cycle in flight.
* **THE DEFERRED PACK IS NOT STARVED AND IS NOT A FAULT.** A deferral does not
stamp `last_start_attempt` (that is the 90 min post-abort cooldown, inside a
60 min start hour), and does not move the phase, so neither the 12 h `drain`
nor the 36 h `charge` timeout — *"the battery is not following commands"* —
can be reached by waiting. It says WAITING, names the pack it waits for, and
recalibrates at the next start hour; a test drives the winner's whole cycle
and observes it start.
* **THE SCHEDULE IS NOT STAGGERED BY CONSTRUCTION.** The interlock defers and
the stagger is what comes out of it: one contested month separates two packs
by one cycle length and they stay separated. `DECISIONS/MAINT-05-per-pack-maintenance.md`
§3 carries the argument.
* **`validate.py` rule 6 now grades `maintenance_charge_w` against the WEAKEST
cap on the roster**, like rule 5. Its own comment said it would have to move
the day `maint_unit` stopped being `inverters[0]`; that day is this card. It
refuses configurations that used to start — a 3000 W charge beside a 2000 W
pack that is now recalibrated too — which is the `lo > hi` contradiction rule
6 exists to refuse, caught when the config is written. Identical at N=1.
* **AT A ROSTER OF ONE, NOTHING CHANGES**, and it is asserted rather than
assumed: the state file keeps the flat keys and gains none, the migration does
not run, the four cycle events are the shipped strings character for
character, and the status banner is the bare phase with no pack name.
* **THE MIGRATION IS GRADED AGAINST THE ROSTER, NOT AGAINST THE NAME IT WAS
HANDED** (fix round). `migrate_maint_keys()` takes the roster rather than a
pack count and `_verify_moves()` checks the namespace it wrote against the
names the machines actually read. A postcondition that reads back the
namespace it was given cannot detect being given the wrong one: handed
`inverter.UNIT_A` on a roster of `["garage", "shed"]`, the first version of
this card wrote all four keys under `unit/inverter-a/...`, read all four back
equal, reported **verified**, and stamped `maint_schema_version` — so both
packs came up believing they had never been recalibrated and the next start
skipped the migration too. That is now `misdirected`: not `ok`, the marker is
**not** written, and the next start re-runs and recovers (rule 1 makes the
re-run a no-op). `build_roster()` numbers a nameless roster entry
`inverter-1`, so `inverter-a` is not even the fallback for a declared roster.
Every two-pack fixture that could see this now names its first pack
`garage`; with them all called `inverter-a`, `self.inverters[0].name` and
`UNIT_A` were the same string and the swap left all 19 suites green.
* **AND THE REFUSAL NOW SURVIVES THE NEXT START** (second fix round). The fix
above graded only the keys it had just *written*, so it refused exactly once:
measured at Controller level over one persisted store, roster
`["garage", "shed"]`, with the call site still wrong on both starts —
construction 1 `ok=False misdirected=4 moved=4 kept=0 marker=<absent>`,
construction 2 `ok=True misdirected=0 moved=0 kept=4` and the marker
**stamped**. On the second start rule 1 finds the wrong keys already there and
routes all four to `kept`, so `moved` is empty and the roster loop grades
nothing — and start 3 skips the migration forever, *including after the call
site is fixed*. Loud-and-recoverable became silent-and-permanent after one
restart. `_verify_moves()` now grades `set(moved) | set(kept)`; both lists are
built from the same `unit` argument, so grading `kept` grades the same
namespace decision. `test_maintenance.py` part **9c** drives two starts with
the same wrong unit over one store — the shape nothing in the suite had, since
9b measured recovery along the path where the operator has *already* fixed the
call site.
* **AND THE MARKER GATE IS PINNED ON BOTH GUARDS.** `if rep.ok:` is
`not lost and not misdirected`, and the mutation `if rep.ok or rep.kept:`
stamped the marker on a migration that **lost a month of history** with all 19
suites green. Part **9d** is the fixture that separates the two guards: two
keys already namespaced (`kept` non-empty) and two whose write the store
drops (`lost` non-empty), `misdirected` empty throughout.
* ⚠️ **AND THE SAME MUTATION ONE FUNCTION UP WAS A SURVIVOR, FOUND HERE AND
CLOSED.** `migrate_unit_keys()` — SAFETY-02's **latch** migration — gates its
own marker on `rep.ok` too, and every check that reached that gate ran on a
store where nothing was namespaced yet, so `rep.kept` was empty in all of
them. `if rep.ok or rep.kept:` there stamps `schema_version` on a migration
that **dropped the latch**: the guard is off, the file still says
`sign_tripped: true`, and the next start skips the migration that would have
re-armed it. Measured a survivor of all 19 files; now red on
`test_inverter.py` part 10, which drops the `sign_tripped` write beside an
already-namespaced `sign_reason`. No source change — the shipped gate was
right; nothing could see it.
* ⚠️ **FOUND, RECORDED, NOT FIXED (SECOND ONE).** `migrate_unit_keys()` —
SAFETY-02's latch migration — has the identical hole and it costs a **guard**
rather than a history. It runs before `build_roster()` (each `Inverter`
restores its latch in its constructor), so it is handed the constant `UNIT_A`
and there is no roster to grade against. On a declared roster a pre-DEV-02
`sign_tripped: true` lands under `unit/inverter-a/sign_tripped`, which no unit
reads — and `_grade_store_migration()`, whose job is to re-arm exactly this,
is **defeated by the key the defect wrote**: its "is anything namespaced
latched" test is satisfied by the misdirected value itself. Fixing it means
naming the migration's target from the config before any `Inverter` exists,
which is `inverter.py` and SAFETY-02's semantics, with its own battery.
Measured and pinned as a running check (`test_maintenance.py` part 13), whose
assertions state the WRONG behaviour on purpose.
* ⚠️ **FOUND, RECORDED, NOT FIXED.** A maintenance drain **pins** its pack while
the loop's target is a **site** claim, so when the site target is below the
pin the only split asks the sibling to charge against the drain and
`allocator.plan()` refuses it — the site parks at 0 W and the pack's 12 h
timeout would eventually report *"drain stuck"* about a healthy battery.
Pre-existing on `release/1.0` at `2d025ee` with the unit names swapped; fixing
it means changing what the loop's target means while a unit is pinned, which
is loop and allocator territory. Pinned as a running check
(`test_maintenance.py` part 12) so it cannot be forgotten.
**At two inverters the sign watchdog stops comparing one unit against the whole
site meter, and becomes two checks instead of one (SAFETY-30).**
`disagrees(commanded_w, grid_w, ...)` compared the command **this controller put
on ONE actuator** against **what the whole site** is exchanging with the network.
At a roster of one those are two views of the same thing - §6.2's identity - and
that is what makes the check sound and what earned it the best record in the
safety table. At two they are not, and it fails **in both directions**: inverter
A commanded −1500 W while the meter reads +1500 W is evidence of nothing when
inverter B is discharging 3000 W exactly as told, and - worse - a unit genuinely
driving the wrong way is **MASKED** by a sibling that covers for it, because the
site figure nets. No threshold recovers that; the summation destroys the
evidence.
* **PER UNIT: each inverter's own share against its OWN measured power**
(`Inverter.batt`, holding register 0x0518, per-instance since DEV-02).
Conclusion: *the actuator I am driving is not doing what I told it* - evidence
about one device, so the remedy stays the per-unit `failsafe` SET that
`arbiter.CLAIM_SCOPE` has classified `SCOPE_UNIT` since it was written. That
entry read *"the evidence is site-wide and the remedy is not"*; after this card
the first half is simply true. Verified by making one unit reverse while its
sibling covers it at the meter: the shipped comparison is false on **0 of 14**
cycles while the new one convicts that unit on every one, and only that unit is
parked.
* **SITE: the SUM of the standing commands against the meter**, as a new
`sign_site` SET at `P_SAFETY` on the site actuator, latched under flat
`site_sign_tripped` / `site_sign_reason`. This is the fault the 2026-08-20
capture contains and **no per-unit check can see it**, because every inverter
is faithfully executing its share - what is wrong is the number they were
given. It stops every inverter and the banner names the site rather than an
inverter, so nobody is sent to swap the wrong box.
* **⚠️ Different questions from SAFETY-31, and the PR and the decision record
both say which.** `circulation.py` compares the units **against each other**
and catches two inverters pushing *through* each other, which nets to zero at
the meter and which no single command is evidence about. This compares each
unit against **its own command**. They overlap in exactly one scenario - a unit
reversing while its sibling pulls the other way - and `test_watchdog.py`
asserts on that fixture that both fire and what each concludes. It is also not
the saturation freeze: that is a magnitude test (`|command − measured| >
500 W`) that shapes the loop and fires on a merely saturated unit; this is a
direction test that latches and parks one.
* **⚠️ All four thresholds RE-DERIVED at two inverters - one carried with a
justification, one refused.** 1890 closed-loop runs in which nothing is faulty,
over the whole range `config.yaml` accepts for `gain` and `slew_w`, across load
reversals with write staggers and flapping loads, **run at both rosters so the
N=1 arm is the control.** `unit_sign_disagree_w: 250` is `saturation_w / 2`, so
the smallest divergence this can see is exactly the gap at which the control
law itself already says the inverter is not following; `unit_sign_disagree_s:
85` is 8.5× the worst legitimate opposition those runs produce (10.0 s), 2.8×
the firmware's own 30 s link veto and under half the site hold.
`site_sign_disagree_w: 300` is **carried** - its argument is the meter's noise
floor and the deadband, and the roster touches neither. **`site_sign_disagree_s`
is NOT: 180 s would have shipped a false trip.** The worst legitimate crossing
at 300 W is 110 s with one inverter and **215 s with two**, because the command
must travel a rail that is the roster sum at a `slew_w` that is not - reachable
at `gain: 0.05`, `slew_w: 50` and a 2.2 kW load reversal, all schema-legal.
325 s is 1.51× that, inside the 900 s billing quarter and inside every episode
of the fault day. See `DECISIONS/SAFETY-30-per-unit-sign.md`.
* **⚠️ At ONE inverter this changes nothing, and "nothing" is asserted rather
than assumed.** The two questions are the same sentence at a roster of one, so
only the detector that shipped is armed: same comparison against the meter,
same 300 W / 180 s, same reason string byte for byte, same namespaced latch and
flat rollback mirror, same `failsafe` claim, same banner, same single status
row - and the site detector's window is never so much as opened.
`unit_sign_disagree_*` and `site_sign_disagree_*` do nothing on a
one-inverter install.
* **The threshold WIRING is pinned, not just the threshold values.** Review
found that `arm(self.sign_cfg if one else self.unit_sign_cfg, ...)` could be
collapsed to `arm(self.sign_cfg, ...)` — arming every unit with the
meter-facing pair instead of the re-derived per-unit one — with all 19 test
files still green, because nothing at a roster of two read `sign_wd.cfg` and
the trips in the fixture happen under either pair. `test_watchdog.py` part 3
now asserts the pair and the instrument on **every** detector against
literals, and the fixture's three threshold pairs were made pairwise distinct
in both coordinates so that a hold delivered from the wrong pair cannot
compare equal to the right one. M14–M16 in
`DECISIONS/SAFETY-30-per-unit-sign.md` §5.
* **Known and recorded rather than fixed:** the site comparison is defeated by a
square-wave load whose half period is about one control cycle, which holds it
opposed for ~1000 s at *both* rosters. The N=1 control arm is what says this is
a property of comparing a lagged command against a meter and has nothing to do
with the second inverter, so it is not acted on here - it wants an alternation
test, not a number, and a card of its own.
**An operator can declare a second inverter, and the shape was measured against
the Supervisor rather than assumed (CFG-04).** DEV-02 made rating, SoC,
watchdogs, heartbeat and store keys per-instance, so the code no longer assumed
@@ -0,0 +1,388 @@
# MAINT-05 — one recalibration per pack, one at a time, and what the card left open
**Status: DECIDED 2026-09-04.** There is one `Maintenance` per roster entry,
each with its own schedule, phase and history. The site holds exactly **one
maintenance slot**: at most one pack may be in a power-moving phase, the slot is
derived from the packs' own stored phases rather than persisted separately, and
when several packs are due in the same hour it goes to the one that has gone
**longest without a completed cycle**. The schedule is **not** staggered by
construction — the interlock defers, and the stagger emerges from it.
**Fix round 2026-09-04.** The store migration is now graded against the
**roster** rather than against the unit name it was handed — see §2's
*"The read-back postcondition is graded against the ROSTER"* — and the same
hole in SAFETY-02's latch migration is recorded in §7.
Filed with MAINT-05. Card `6a9a9e8503872905f298d80a`.
---
## 0. What the card asked to be decided, and the answers
| the card's open question | the decision |
|---|---|
| §2 the arbitration | **one pack at a time; the other stays under the loop.** Nothing preempts a cycle in flight. |
| §2 who wins when several are due | **the longest-waiting pack**, `last_completed` oldest first, `inf` for never-recalibrated. Roster order breaks a numeric tie and nothing else. |
| §4 staggering | **not by construction.** The interlock defers the second pack; the stagger is what comes out of it. §3 below. |
| AC 7 rule 6's `maint_unit` comment | **no, the first entry's cap is no longer right.** It grades against the weakest cap on the roster now, like rule 5. §4 below. |
| the store layout at N=1 | **flat, unchanged.** The namespace appears the day the roster grows. §2 below. |
## 1. Why the slot is derived and not persisted
The holder is *"whichever pack's stored phase is not `idle`"* — a fact already on
disk, written by the packs themselves in the per-pack history this card
introduces. A `maint_slot_holder` key would be a **second opinion about the same
thing**, and the two would disagree the first time the add-on was killed between
writing the phase and writing the holder. Derived, a controller that comes back
up mid-drain reads the drain and re-derives the holder from it; there is no
recovery path to get wrong because there is no second fact.
`CHARGE` is in `POWER_MOVING` even though the control **loop** owns the setpoint
in that phase, and that is the decision most likely to be "simplified" later.
Pack A charging from the grid at 2000 W while pack B drains 2000 W to export is
4 kW going round in a circle with the meter reading zero and both packs
believing they are behaving — the circulating power [[SAFETY-31]] exists to
detect, arriving as something this module can simply not do. `HOLD` is in the
tuple for the other half of the sentence: it moves nothing, and it **owns the
actuator**, so a second cycle starting under it would still be two
recalibrations at once.
## 2. Why the store layout at a roster of one does not change
The maintenance keys are flat — `phase`, `phase_started`, `last_completed`,
`last_start_attempt` — on every install since the add-on shipped. At one pack
there is nothing for a namespace to distinguish that pack **from**: §6.2's
identity, the same one `arbiter.actuator_for()` leans on when it routes every
claim to the site actuator at N=1. So `Maintenance` is constructed with
`unit=None` there, reads and writes the flat keys directly, and
`migrate_maint_keys()` **returns without writing anything**. A one-inverter state
file comes out of this card byte-identical, which is AC 6 and which
`test_maintenance.py` part 10 asserts by driving a whole cycle and comparing the
key set.
### The migration has its own marker, and that is not tidiness
`SCHEMA_VERSION` / `schema_version` is DEV-02's, and it is **monotone**: every
install runs that migration exactly once. This one runs **only where the roster
has more than one pack**, so a single counter could not express *"migrated on
this site, deliberately not migrated on that one"* without lying to the next
version that reads it — a one-inverter store would have to be pinned at the old
number forever, and a genuine future v2 migration could then never be told apart
from a site that simply never grew. `maint_schema_version` answers one question
and answers it honestly.
The **three rules are shared in code**, not restated: `_move_flat_keys()` is
rule 1 (never overwrite an existing namespaced value) and rule 2 (the flat keys
stay in the file), `_verify_moves()` is the read-back postcondition, and the
marker is written last. A second transcription of a migration is a second
opinion about what "it landed" means.
### The read-back postcondition is graded against the ROSTER (fix round)
**A postcondition that reads back the namespace it was handed cannot detect
being handed the wrong one.** That sentence is the whole finding, and this card
shipped it broken.
`_verify_moves()` iterated `rep.moved` — the keys it had just written — and
asked whether each read back equal. On a store where the write lands, it always
does. So a migration handed a `unit` that is on no roster wrote four keys, read
four keys back, set `verified = True`, and the caller stamped
`maint_schema_version`. Every symptom of a clean migration, and an install's
whole maintenance history parked under a name nothing will ever look up. And
because the marker was stamped, **the next start skipped the migration too** —
which is precisely the state `store.py`'s header says must never happen: a
failed migration a later version treats as done.
Reproduced on the reviewer's roster, a house with a garage pack and a shed pack,
carrying a flat `last_completed` of 2026-08-07:
| | `unit` the migration was handed | key written | pack 0 reads back | due now? |
|---|---|---|---|---|
| correct | `self.inverters[0].name` = `garage` | `unit/garage/last_completed` | `2026-08-07` | **False** |
| wrong | `inverter.UNIT_A` = `inverter-a` | `unit/inverter-a/last_completed` | `None` | **True** |
`build_roster()` names a declared roster's entries **from the config**, and its
fallback for a nameless entry is a **positional** `inverter-1` — so `inverter-a`
is not the name of anything on a declared roster, not even by default.
**The remedy, and why this one.** `migrate_maint_keys()` now takes the **roster**
in place of an `n_units` count (the count was `len(self.inverters)` at the only
call site, so the roster carries it, and a count passed beside the names is a
second opinion about how many packs there are). `_verify_moves()` grades the
namespace it wrote against those names, and a unit that is not among them is
reported as `misdirected` — a **different** failure from `lost`:
* `lost` is *the store took the write and dropped it* — a storage fault, and it
has a harmless-default case (a write of `last_completed: None` loses nothing,
because "absent" and "never recalibrated" are the same statement).
* `misdirected` is *the store did exactly as it was told, by a caller that was
wrong*. There is no harmless-default version of it: a write nobody reads is
not information, whatever its value. So anything in it fails `ok` on its own.
And **`ok` gates the marker**, so a misdirected migration is not stamped and the
next start runs it again. The argument for it is asymmetric: a re-run is
recoverable — rule 1 makes it a no-op where keys did land, and rule 2 left the
flat keys in the file to recover *from* — while a silent skip is not recoverable
at all, because the store now claims the migration happened.
### ⚠️ THE PARAGRAPH ABOVE USED TO END HERE, AND AS IT STOOD IT WAS FALSE
**The refusal lasted exactly one start. The second start stamped the marker.**
That is a correction to what the first fix round wrote down, not a new
qualification of it, and it is recorded here because a decision document that
states a guarantee the code does not provide is worse than one that says
nothing.
`_verify_moves()` graded only `rep.moved`. Measured at Controller level over
**one persisted store**, roster `["garage", "shed"]`, flat `last_completed`
2026-08-07, with the call site still handing `inverter-a` on **both** starts:
```
construction 1: ok=False misdirected=4 moved=4 kept=0 marker=<absent> (withheld)
construction 2: ok=True misdirected=0 moved=0 kept=4 marker=1 (STAMPED)
history parked under unit/inverter-a/…; marker stamped after ONE restart.
```
On start 2 the same wrong call site finds the four keys it wrote already
present, so **rule 1** — never overwrite a namespaced value — routes every one
of them to `rep.kept`. `rep.moved` is empty, the roster-grading loop iterates
over nothing, `misdirected` is empty, `ok` is True, the marker is stamped, and
start 3 takes the `from_version >= MAINT_SCHEMA_VERSION` early return **forever
— including after the call site is fixed**. Loud-and-recoverable became
silent-and-permanent after one restart: exactly the state `store.py`'s header
says must never happen, reached by the fix written to prevent it.
**The remedy is one line**, and it is sound by construction rather than by
argument: grade `set(rep.moved) | set(rep.kept)`. Every entry in *both* lists is
a `unit_key(unit, legacy)` built from the **same** `unit` argument, so grading
`kept` grades the same namespace decision `moved` does. A correct call site's
`kept` entries are under a roster name and pass; a wrong one cannot launder its
namespace by having already written it. `migrate_unit_keys()` passes
`roster=None` and returns before the loop, so it is unaffected.
**Why the check missed it, which is the part worth keeping.** Part 9b's recovery
check was `migrate_maint_keys(_m5, ROSTER[0], ROSTER)` — the **corrected** name.
It measures recovery along the one path where the operator has *already fixed
the bug*. The real failure is a call site that is still wrong on the next start,
and nothing in the 19 files called `migrate_maint_keys()` twice with the **same
wrong** `unit`. The check was true, passed, and could not reach the failure its
sentence named — MUTATION.md's third defect, this time sitting inside the fix
written to close it. Part **9c** now drives two starts with the same wrong unit
over one store, asserts the marker is still withheld on the second, and asserts
the *shape* of the second report (`moved` empty, `kept` full) so it cannot
quietly revert to 9b's path.
### The marker gate is `rep.ok` and nothing weaker — both guards, each pinned
`if rep.ok:` is `not lost and not misdirected`, and until this round only the
`misdirected` half was reachable by a check that also had a non-empty `kept`.
The mutation
```
if rep.ok: -> if rep.ok or rep.kept:
```
stamps the marker on a migration that **lost a month of history** — defeating
the older, settled `lost` guard as well as the new one — and **all 19 suites
were green on it**. Part **9d** is the fixture that separates them: two keys
already namespaced (`kept` non-empty, so the mutant's `or` fires) and two keys
whose write the store drops (`lost` non-empty, so `ok` is False), with
`misdirected` empty throughout so the section cannot pass on the other guard by
accident. Measured differential on it: shipped → marker absent; mutant → marker
`1`, both reporting `ok=False lost=2 kept=2`.
**Why the fixtures had to change, and this is the part that got the card
rejected.** Nothing in the repository could see whether any of the above was
true, because `inverter.UNIT_A` **is** the string `"inverter-a"` and every
two-pack fixture in `test_maintenance.py` named its first roster entry
`inverter-a`. `self.inverters[0].name` and `UNIT_A` were therefore the same
value in every check that existed, and swapping one for the other at the call
site left **all 19 suites green**. MUTATION.md's third defect exactly — a check
whose *data* cannot reach the failure it names — and the fifth consecutive card
to carry it. Parts 9 and 11 now run on `["garage", "shed"]`, each behind a
**precondition check** asserting that the first name is not `UNIT_A`, so an edit
that makes them equal again goes red instead of making the claims vacuous.
### And the flat keys are kept LIVE, which is the half DEV-02 had to add later
Rule 2 originally said the flat keys are left in place and go stale. DEV-02
amended it, for the sign latch, to *"left in place is not enough"* — and the
same sentence holds for history. A fossil dated the day the second inverter was
declared would hand a rolled-back single-FSM add-on a `last_completed` months
old (it recalibrates a pack that does not need it) or a stale `phase` with a
`phase_started` old enough to abort on the 12 h timeout the moment it comes up.
So `UnitHistory` **mirrors the first pack's writes back onto the flat keys** for
as long as the roster has more than one entry. Only the first pack: the flat
spelling means `inverters[0]` on every install that has one, and a mirror from
pack B would overwrite pack A's rollback view with pack B's phase.
## 3. Staggering: the interlock defers, and the stagger is the residue
Two packs on one monthly schedule are due in the same hour every month. The
options were a stagger **by construction** — offset each pack's first cycle by
`interval_days / N` — or **deferral**. Deferral, for three reasons:
1. **A constructed stagger cannot remove the need for the interlock.** A forced
start, a commissioning cycle, an abort-and-retry, or a pack added mid-month
desynchronises any static offset immediately, so the interlock has to be
there anyway — and then the stagger is a second mechanism that must also be
correct, for a case the first one already handles.
2. **A deferral is observable and a stagger is not.** The deferred pack says, in
the action log and on the status page, that it is waiting, which pack it is
waiting for, and that it will run at the next start hour. A silent offset
moves an operator's configured start day and explains nothing.
3. **The stagger comes out of the deferral for free and then persists.** Pack A
completes on day 1 and is next due on day 29; pack B, deferred, completes on
day 2 and is next due on day 30. One contested month separates them by one
cycle length and they stay separated.
**Starvation was the risk and it is closed at two points.** A deferral does
**not** stamp `last_start_attempt` — that key is the post-abort cooldown, and at
90 minutes inside a 60-minute start hour, stamping it would lock the deferred
pack out of exactly the hour it is waiting for. And the winner is the
longest-waiting pack rather than the first in the roster, so a pack that keeps
aborting cannot monopolise the slot month after month while its sibling drifts —
which is this card's own defect wearing a different costume.
**The deferral must not read as a fault.** §12's 12 h and 36 h phase timeouts
mean *"the battery is not following commands"*. A deferred pack stays `IDLE`
with `phase_started` untouched, so no timeout is reachable by waiting, and the
line it logs says WAITING, names the pack it waits for and says it will run
next — none of which any failure path in `maintenance.py` says.
## 4. Rule 6 (the card calls it rule 8): the first entry's cap is no longer right
`validate.py`'s rule 6 comment said, in as many words, that
`maintenance_charge_w` was graded against `inverters[0]`'s cap because there was
exactly one `Maintenance` and it could only ever recalibrate the first pack —
and that taking `min()` would be an over-refusal, because it would refuse *"a
5000 W pack being recalibrated beside a 2000 W sibling the cycle never
touches"*. It closed: **"when `maint_unit` stops being `inverters[0]`, this line
moves with it."**
It has stopped. Every pack is recalibrated on its own schedule, so **the sibling
the cycle never touches does not exist** and the over-refusal argument has no
subject. The shaping claim is filed on whichever pack is charging, so the cap
that can contradict it is any cap on the roster and the one that contradicts it
first is the smallest.
**This refuses configurations that used to start**, and that is the point rather
than a side effect: `max_w` 5000 on pack A beside 2000 on pack B with
`maintenance_charge_w` 3000 was correct while only pack A was ever recalibrated,
and is now a cheap-window charge asking the arbiter for `hi = -3000` against a
supervised `lo = -2000` — `lo > hi`, arbiter clause 3, 0 W, on the pack nobody
was watching. Refused when it is **written**, which is rule 6's whole job. At a
roster of one `min()` over one cap is that cap: same comparison, same message,
same boundary.
## 5. The claim stays per-unit, and nothing here files a site claim
`arbiter.CLAIM_SCOPE` has classified `maintenance` as `SCOPE_UNIT` since it was
written. Two FSMs mean two claims on two actuators, which the allocator consumes
as per-unit intervals. [[ARCH-01]] is explicit that this must never become a
site claim, because per-inverter constraints filed site-wide are the
cross-inverter `lo > hi` contradiction at `arbiter.py:114`. The only line this
card changed there is **which** roster entry the claim is filed on: the one whose
state machine produced the result, rather than `inverters[0]`.
## 6. FOUND AND NOT FIXED: the site loop still contradicts a drain at N>=2
Measured while wiring this card, and **pre-existing on `release/1.0` at
`2d025ee`** — the same drain filed on `inverters[0]` produces the identical
refusal with the two unit names swapped.
The maintenance SET **pins** its pack. The loop's target is a **site** claim.
When the site target is *below* the pin — a 2000 W drain while the house imports
less than that, which is most of a night — the only split that sums to the site
target asks the sibling to **charge** while the maintained pack **discharges**,
and `allocator.plan()` refuses it by name as circulating power. The refusal is
correct and fails safe: the whole site goes to 0 W with an `ARBITER: REFUSED`
line in the action log.
**The consequence an operator would see is a misdiagnosis.** The pack is held at
0 W while its state machine believes it is draining, so after 12 h the DRAIN
timeout fires and reports *"drain stuck"* — §12's words for a battery that is not
following commands. The battery is fine; the site refused to allocate.
**Why it is not fixed here.** The fix is to change what the site loop's target
*means* while a unit is pinned — regulate the meter using only the unpinned
packs — which is loop and allocator territory and changes the integrator's
subject. MAINT-05 moves which pack is pinned and nothing else about it. It is
pinned as a running check (`test_maintenance.py` part 12) rather than left as a
comment, so it cannot be forgotten or quietly tuned away, and it wants a card of
its own.
## 7. FOUND AND NOT FIXED: `migrate_unit_keys()` has the same hole, and it is a LATCH
Found while fixing §2's instance. It is the same defect one module over and it
costs a **disarmed safety guard** rather than a lost history.
`Controller.__init__` calls `migrate_unit_keys(store, UNIT_A)` — a hardcoded
constant where the maintenance migration now passes the roster's own first name.
It **has** to run before `build_roster()`, because each `Inverter` restores its
sign latch in its constructor, so there is no roster to name yet and
`_verify_moves()` is passed `roster=None`. On a **declared** roster a pre-DEV-02
`sign_tripped: true` therefore lands under `unit/inverter-a/sign_tripped`, which
no unit on that site reads.
**And the guard that exists for exactly this is defeated by the key the defect
wrote.** `_grade_store_migration()` re-arms the latch when the flat key says
tripped and *nothing namespaced is latched*. The misdirected write **is**
namespaced and **is** latched, so `namespaced_latched` is `True`, the guard
returns, and no inverter on the roster comes up tripped — on an install whose
`/data/state.json` still reads `sign_tripped: true`. That is DEV-02's own
sentence for the worst case: the state file says protected, the banner is green,
and the controller resumes commanding into the fault that tripped it.
Measured, on the same `["garage", "shed"]` roster:
```
pre-DEV-02 flat latch + declared roster ['garage', 'shed'] ->
migrated 2 flat key(s): unit/inverter-a/sign_reason, unit/inverter-a/sign_tripped
latch keys in the store: ['sign_tripped', 'site_sign_tripped', 'unit/inverter-a/sign_tripped']
roster came up tripped: [('garage', False), ('shed', False)]
the same store on the FLAT config -> [('inverter-a', True)]
```
The last line is the counterfactual: on the flat shape, where `UNIT_A` really is
the roster's only name, the same store migrates and comes up latched. So this is
a defect of the **declared roster**, not of the migration as such.
**Why it is not fixed here.** The remedy is to name the migration's target from
the config before any `Inverter` exists, which means factoring `build_roster()`'s
naming rule out of `build_roster()` so there is still only **one** spelling of
it — `inverter.py` and SAFETY-02's latch semantics, with its own mutation battery
over `test_inverter.py` and `test_watchdog.py`. MAINT-05 moves the maintenance
history and nothing about the sign latch, and a silent expansion into a safety
latch is how a rejected PR becomes a broken one.
It is pinned as a running check (`test_maintenance.py` part 13) rather than left
as a comment. ⚠️ **Its assertions state the WRONG behaviour on purpose** — read
them as the specification of a defect; the card that fixes it should expect them
to go red and should replace them.
### And its MARKER GATE had the round-3 hole too — found here, closed here
Separate from the namespace hole above, and **fixed** rather than recorded,
because closing it cost one test fixture and no source change.
`migrate_unit_keys()` gates `schema_version` on `rep.ok` exactly as
`migrate_maint_keys()` gates its own marker, and every check that reached that
gate ran on a store where **nothing** was namespaced yet — so `rep.kept` was
empty in all of them and
```
if rep.ok: -> if rep.ok or rep.kept:
```
**survived all 19 files**. What it does is stamp the schema marker on a
migration that DROPPED THE LATCH: the guard is off, the state file still reads
`sign_tripped: true`, and because the store now says the migration finished, the
next start skips the one thing that would have re-armed it. DEV-02's own worst
case, filed under *done*.
`test_inverter.py` part 10 now drops the `sign_tripped` write **beside an
already-namespaced `sign_reason`** — `lost` non-empty *and* `kept` non-empty,
which is the one shape none of the existing fixtures had — and asserts the
marker is withheld, with the accepted-baseline counterfactual beside it (habit
7). The shipped gate was correct throughout; what changed is that a check can
now see it.
@@ -0,0 +1,474 @@
# SAFETY-30 — what the sign watchdog compares once there is more than one inverter, and the numbers it compares with
**Status: DECIDED 2026-09-04.** The one comparison becomes **two**, asked of the
evidence that can answer each: **each unit's standing command against that
unit's own measured power** (register 0x0518, per-instance since [[DEV-02]]),
and **the sum of the standing commands against the meter**. At a roster of one
nothing changes at all — the two questions are the same sentence there, and the
detector that ships is the only one armed.
This document records the argument for that split, the campaign the four numbers
were read off, **a number that was carried and a number that refused to be**, and
the mutation ledger. Filed with SAFETY-30, card `6a9a1828e844f22fabf7fe9c`.
---
## 0. The half that was already built, and the half this card is
[[DEV-02]] shipped the **per-instance** half and nothing here re-tests it: each
`Inverter` owns its own `SignWatchdog`, the latch is namespaced under
`unit/<name>/sign_tripped`, the flat pre-DEV-02 pair is maintained as a rollback
view, and a trip on B no longer latches A. `test_inverter.py` part 3 grades that.
What was left is **the comparison itself**, and it was one line:
```python
inv.sign_wd.evaluate(stood, self.grid, self.following, now) # every unit
```
`stood` is the command **this controller put on THIS actuator**. `self.grid` is
what the **whole site** is exchanging with the network. At one inverter those are
two views of one thing — §6.2's identity — which is what makes the check sound
and what earned its record. At two they are not, and it fails in **both**
directions:
| | what happens | why no threshold fixes it |
|---|---|---|
| **false trip** | A commanded −1500 W while the meter reads +1500 W, because B is discharging 3000 W exactly as told | the meter is answering for a unit A's command says nothing about |
| **missed fault** | B commanded +1500 W and **measured** at −1500 W, beside A at +1500 W — the meter sits where a healthy site leaves it | the summation **destroys** the evidence |
NOTES.md calls SAFETY-02 the highest-value check in the safety table: 93 %
agreement while healthy against 4 % during runaway, four observed episodes, zero
false positives. **That record was earned at N=1 on the site comparison.** The
first row above spends it; the second row is the guard being absent when it
matters.
## 1. The two questions, and the two remedies
> **PER UNIT** — this unit's standing command against **its own** measured power.
> Conclusion: *the actuator I am driving is not doing what I told it.* Evidence
> about one device, so the remedy is that device: the `failsafe` SET, which
> `arbiter.CLAIM_SCOPE` has classified `SCOPE_UNIT` since it was written.
>
> **SITE** — the **sum** of the standing commands against the meter. Conclusion:
> *the site as a whole is driving the wrong way* — an inverted gain, an
> integrator wound past the rail, a strategy on the wrong side of the meter.
> That is the 2026-08-20 fault day, it is a property of the **connection**, and
> the remedy is a new SITE SET, `sign_site`, that stops every inverter.
### Why `failsafe` stays per-unit and `sign_site` is new
`CLAIM_SCOPE`'s entry for `failsafe` used to read *"the evidence is site-wide and
the remedy is not"* — a site measurement attributed to one actuator on the
strength of the N=1 identity. After this card the first half of that sentence is
simply true: the evidence **is** per-unit. The site-wide half did not disappear;
it moved to a claim that is site-scoped for the reason `circulation` is, one
entry above it: **the offending number is the site target, which no single
inverter owns.** Parking one unit on it would leave the rest of the roster
pushing the same way with the guard satisfied.
### At N=1, exactly one detector is armed
`Controller._arm_sign_watchdogs()` is the only place the roster decides anything
here, and it is a pure function of `len(self.inverters)`:
* **one inverter** — the unit detector is armed against **the meter**, at the
shipped `sign_disagree_w` / `sign_disagree_s`. The site detector is
**constructed and never evaluated**. Same comparison, same thresholds, same
reason string byte for byte, same namespaced keys and flat mirror, same
`failsafe` claim, same banner, same single status row. `test_watchdog.py`
part 7 asserts each of those, including that the site detector's window was
never so much as opened.
* **two or more** — each unit against its own 0x0518 power under the per-unit
pair, and the site detector against the meter under the site pair.
Arming both at N=1 would be two latches and two banners for one fault, and a
false-positive record measured on **one** comparison spent twice.
## 2. What this is NOT — the two neighbours, named
Two guards that overlap without saying so is how a false-alarm generator gets
built, so both are written down.
### It is not [[SAFETY-31]]
| | **SAFETY-31 — `circulation.py`** | **SAFETY-30 — the per-unit half** |
|---|---|---|
| compares | the units **against each other**: `Σ\|Bᵢ\| − \|ΣBᵢ\|` over measured power | each unit against **its own command** |
| catches | two inverters pushing **through** each other; nets to zero at the meter | a unit **not doing what it was told**; that unit is faulty or wedged |
| attributes | nothing — the quantity is defined over the roster and is identically zero for one unit | **the unit**, by name, and parks it |
| scope | SITE claim, flat latch | UNIT claim, namespaced latch |
They genuinely overlap in **one** scenario — a unit reversing while its sibling
pulls the other way — and `test_watchdog.py` part 4 asserts exactly that, on the
same fixture, with both readings printed: circulating power is over its threshold
**and** SAFETY-30 names B. Neither is the other with a different threshold. Take
two units faithfully executing an allocation with two signs: circulating power is
the whole loop and every per-unit sign check reads perfect agreement. Take one
unit doing the opposite of its share while its sibling covers: this reads the
whole disagreement and circulating power can be zero.
### It is not the saturation freeze
`compute()`'s anti-windup test is `|prev_w − actual_w| > saturation_w` (500 W). It
concludes *the inverter cannot follow*, it fires on a merely **saturated** unit as
readily as on a reversed one, it **stops the integrator winding** — an ordinary
control action taken every day — and it neither latches nor parks anything.
This asks for **opposition**: a sign, on both sides of a threshold, held
continuously. A saturated-but-obedient unit cannot produce it. And the two meet
at a number rather than overlapping in a band nobody drew — see §3.
## 3. The numbers, and where they come from
Read off `admin/goodwe`'s `sim/` two-master rig (TEST-19), used **read-only** and
run container-free. Real and imported, never transcribed: the control law
(`control.compute`, `Tuning`), the allocator (`arbiter.allocate_record` →
`allocator.plan`), the sign predicate (`watchdog.disagrees`), the wire
(`frame_check.setpoint_atomic`, the firmware's own encoder), the plant
(`plantsim.Plant`), and the per-unit reading, taken from **holding register
0x0518** — the register the live `batt_entity` binds.
**1890 closed-loop runs in which NOTHING is faulty** — every unit executes the
share the allocator gave it — over **6 gains × 5 slews**, the whole range
`config.yaml` accepts for `gain` `float(0.05,1.0)` and `slew_w` `int(50,5000)`,
across both families of stimulus that can produce legitimate opposition:
* **1050 load reversals** — 7 sizes (±300 W … ±4000 W) × 5 write staggers, the
serialised write path landing unit B's number up to four cycles after A's;
* **840 flapping loads** — 7 periods (10 s … 600 s) × 4 amplitudes.
Every run is executed **twice**: once at a roster of two and once at a roster of
one on the same stimulus. **The N=1 arm is the control**, and it is the reason
this document can separate *"a property of the check that shipped"* from *"a
property of the second inverter"* instead of asserting one and hoping.
Supervised caps are the shipped `max_w` (2000 W per unit), so the site rail is
2000 W at N=1 and 4000 W at N=2 — `Controller.site_max_w`, the roster sum.
### Reproducing it
The harness is **not committed here**, for the reason SAFETY-31's was not: it
imports `plantsim` and `frame_check`, which live in the other repository and are
not present in CI. What it does is small enough to restate: build two
`plantsim.Inverter`s from `INVERTER_A` / `INVERTER_B`, close the loop with the
add-on's own `compute()` and `arbiter.allocate_record()`, put each share on its
own wire through `Inverter.command()`, advance `Site.settle()` one control cycle,
and record `(standing command, inv.reg(0x0518))` per unit and
`(Σ standing, site.grid)` for the site. Then, for each candidate threshold, the
**longest run of consecutive cycles** on which the shipped `disagrees()` is true —
not a count of true samples, because the watchdog's window is cleared by any
disagreeing sample. Point it at a checkout with `GOODWE_ADDON_REPO`.
⚠️ MUTATION.md's note applies: those harnesses write `__pycache__` into **both**
trees.
### A. PER-UNIT — a unit's standing command vs its own 0x0518 power, N=2
| threshold | worst continuous, reversals | worst continuous, flapping | runs crossing (of 1890) |
|---:|---:|---:|---:|
| 50 W | 25.0 s | 20.0 s | 962 |
| 100 W | 20.0 s | 15.0 s | 511 |
| 150 W | 15.0 s | 15.0 s | 379 |
| 200 W | 10.0 s | 15.0 s | 258 |
| **250 W** | **10.0 s** | **10.0 s** | **194** |
| 300 W | 10.0 s | 10.0 s | 124 |
| 500 W | 5.0 s | 5.0 s | 37 |
| 700 W | 5.0 s | 5.0 s | 16 |
| 1000 W | 0.0 s | 0.0 s | 0 |
Largest opposed magnitude anywhere in the 1890 runs: **875 W**.
**The curve is flat, and that is the finding that decided the shape of this
pair.** Unlike the site comparison — where the meter is driven by the LOAD and a
flapping load can hold it opposed for minutes — a unit's own power **chases its
own command** through a bounded ramp and a first-order lag, so the opposition can
only last as long as the crossing does. Dropping the threshold from 300 W to
50 W buys 15 s of extra transient and nothing else. **So the work is done by the
window, and the threshold is put where the control law already draws its line.**
#### `unit_sign_disagree_w: 250` — schema `int(50,5000)`
`saturation_w / 2`. The check gates on **both** sides being over `min_w`, and the
two are **opposed**, so the smallest command-versus-power gap it can ever see is
`2 × min_w`. At 250 W that is exactly the 500 W at which `compute()`'s own
anti-windup test already concludes the inverter is not following. **This watchdog
therefore cannot fire on a divergence the shipped control law itself calls
ordinary tracking**, and the two guards meet at a number.
Rejected: a threshold above the observed 875 W maximum, which would gate nothing
legitimate in — and would also be blind to a unit reversing a 900 W share.
#### `unit_sign_disagree_s: 85` — schema `int(30,1800)`
Three bounds, and the value clears all three.
* **lower, from the rig — 10.0 s.** Two control cycles, the worst continuous
legitimate opposition above 250 W across all 1890 runs. It is a reversing site
whose second write lands a cycle after the first, both units then slewing
through zero behind a 10 s lag. Nothing is wrong.
* **lower, from the firmware — 30 s (`inverter.LINK_STALE_S`).** A unit not
executing its share for less than the firmware's own veto window is a *link*
fault the add-on already answers by dropping the unit from the split. Firing
inside that window would raise a second, louder alarm about a state already
being handled — the SAFETY-15/SAFETY-19 failure exactly.
* **upper, from SAFETY-02 itself — 180 s, the site hold.** A unit that is not
doing what it was told must not be parked *later* than the site would be
parked for the same watts, and this conclusion needs nothing to net out: it is
complete on the first cycle it appears.
**85 s** is seventeen control cycles: **8.5× the worst legitimate episode**,
**2.8× the firmware veto**, and **under half** the site hold. Deliberately not
60, 75, 90, 120 or 180.
### B / C. SITE — Σ standing commands vs the meter, at BOTH rosters
**Load-reversal family** (1050 runs each), worst continuous legitimate opposition:
| threshold | **N=2** | **N=1 (control)** | ratio |
|---:|---:|---:|---:|
| 100 W | 335.0 s | 175.0 s | 1.91× |
| 200 W | 260.0 s | 120.0 s | 2.17× |
| **300 W** | **215.0 s** | **110.0 s** | **1.95×** |
| 500 W | 155.0 s | 90.0 s | 1.72× |
| 700 W | 130.0 s | 70.0 s | 1.86× |
| 1000 W | 85.0 s | 35.0 s | 2.43× |
**The second inverter roughly DOUBLES the worst legitimate crossing, and at the
shipped 300 W threshold it takes it from 110 s — comfortably inside the 180 s
hold — to 215 s, which is outside it.**
The mechanism is not mysterious and is worth stating because it predicts the
ratio: the transient the 180 s hold exists to reject is the loop crossing zero
after a load reversal, and the command has to travel the whole **site rail** at
`slew_w` per cycle. At N≥2 that rail is `Controller.site_max_w`, the roster sum,
while `slew_w` is unchanged. The worst case is the corner of the config space
where that takes longest — `gain: 0.05`, `slew_w: 50`, both **schema floors** —
against a ±2200 W load reversal, which is an oven and a heat pump.
#### `site_sign_disagree_w: 300` — CARRIED, and here is the justification
The argument behind 300 W is about **noise**: *"near zero the sign is noise, and
near zero is where a healthy loop LIVES"* — the deadband holds the command still
while the meter wanders either side of `target_grid_w`. `deadband_w` is 15 W and
there is one P1 whatever is behind it. **Nothing in that argument depends on the
roster**, and the campaign agrees: the fraction of samples gated in at 300 W is
within a few percent at both rosters. Carried, with a measurement behind the
carry rather than an assumption.
#### `site_sign_disagree_s: 325` — NOT carried, and that is the point of the card
180 s is **inside** the 215 s the two-inverter campaign measures, at a
configuration the schema accepts. Carrying it would have shipped a false trip on
the check with the best record in the safety table — which is the exact failure
this card was opened to prevent, arriving from the other direction.
**325 s** is 1.51× the measured worst. It is well inside the **900 s** quarter
that bills the capacity tariff, and it is inside **every** episode the fault day
contains: 2026-08-20's four clusters run 16, 30, 77 and 77 minutes, and its
opposed runs reach 1750 s. What it costs is stated rather than hidden — about two
and a half extra minutes of wrong-direction **site** power before the stop —
and it is bought to remove a false trip a legal configuration produces.
Deliberately not 300 or 360.
**Rejected alternative:** keep 180 s and raise the site threshold to 1000 W,
where the worst is 85 s. It gives a better margin and a faster stop, and it is
wrong: it would be raising a **noise** gate to reject a **transient**, which is
what the window is for, and it would leave the site check blind to a 900 W
whole-site reversal at a 4 kW site.
### A limitation this campaign found, which SAFETY-30 does not introduce and does not fix
The **flapping-load** family behaves completely differently for the SITE
comparison, and the control arm is what makes it interpretable:
| threshold | **N=2** | **N=1 (control)** |
|---:|---:|---:|
| 300 W | 980.0 s | 1000.0 s |
| 700 W | 975.0 s | 1000.0 s |
| 1000 W | 950.0 s | 1000.0 s |
A square wave whose half period is about one control cycle opposes the command on
**every** cycle: the command chases the previous half-cycle's meter while the
meter has already reversed. **No window inside the billing quarter survives it and
no threshold up to 1000 W does either — at EITHER roster.** The N=1 column is
this document's whole reason for existing: this is a property of comparing a
lagged command against a meter, and it has nothing to do with the second
inverter.
It is not acted on here, in order of weight: it is **identical at the roster this
add-on ships**, so acting on it would be changing a live safety path on evidence
that has nothing to do with this card; a ~6 kW square wave at 0.1 Hz sustained
for a quarter of an hour is not a house (the captured healthy day's longest
opposed run is **30 s**); and the fix is not a number — it wants the detector to
notice that the disagreement is **alternating**, which is a different check.
**It is a card, and this section is its evidence.**
### D. The fault arm — because a survey of healthy data is not a derivation
210 runs with inverter B executing the **negation** of its share throughout,
sibling faithful, meter left to net:
| threshold | longest run the per-unit check detects | runs the SITE check also sees |
|---:|---:|---:|
| 250 W | 890.0 s | 210 / 210 |
| 300 W | 890.0 s | 180 / 210 |
| 700 W | 885.0 s | 112 / 210 |
| 1000 W | 885.0 s | 60 / 210 |
| 1500 W | 880.0 s | 45 / 210 |
| 2000 W | 0.0 s | 39 / 210 |
At the shipped 250 W the fault is detected continuously for the whole run in
every one of the 210 — an 85 s hold has ten times the evidence it needs. The
right-hand column is the honest half: on a *reversing* stimulus the site
comparison often does see something too, because the reversal is large enough to
move the meter. **The case it cannot see is the steady one**, and that is the
fixture `test_watchdog.py` part 4 runs: the shipped comparison is false on
**0 of 14** cycles while the per-unit check convicts B on every one.
## 4. Inverter B does not exist
Every two-inverter figure in §3 was produced with **inverter A's ramp and lag
copied onto B** (`plantsim.INVERTER_B`, `measured=False`); its 3000 W rating,
where this repository's fixtures use one, is a **hypothesis marked as one**.
[[SPIKE-04]] measures the real ones and is the revisit trigger for all four
numbers — not a reason to move any of them now.
Which figures depend on it, stated so nobody has to work it out:
* **`unit_sign_disagree_s: 85`** — most. Its lower bound is the 10.0 s crossing,
which is a direct function of B's ramp and lag. A slower second unit lengthens
it; the 8.5× is what buys headroom for that.
* **`site_sign_disagree_s: 325`** — yes, but less directly. Its 215 s is
dominated by the **rail** and by `slew_w`, which are configuration, not
hardware; B's dynamics move it at the margin.
* **`unit_sign_disagree_w: 250`** — **no.** It is `saturation_w / 2`, read off
the add-on's own control law. The rig only confirms that nothing legitimate
sits over it for long.
* **`site_sign_disagree_w: 300`** — **no.** It is the meter's noise floor and the
deadband, neither of which is a property of any inverter.
## 5. Mutation ledger
Every run on a **freshly copied tree of its own, no `__pycache__`, mounted
`:ro`** into the CI image — the structural immunity MUTATION.md records. ⚠️ The
first battery used ONE shared directory rebuilt between mutations and reported a
**RED baseline**; a container mounted it mid-copy and read a missing
`config.yaml` as a failing check. That whole run was discarded: MUTATION.md's
rule is that a run which could have been racing is not a result, and a battery
whose baseline is not green establishes nothing. The harness now copies a
directory per mutation and **refuses to report anything if the baseline is not
green**.
Byte deltas measured, not estimated; the `+0 B` ones are marked, since a
size-preserving edit is what TEST-15 turned into a false survivor once already.
| # | file | mutation | Δ | result |
|---|---|---|---|---|
| M1 | `main.py` | the per-unit comparison put back to `self.grid` at every roster | −30 B | **RED** — `test_watchdog.py`. This is the card |
| M2 | `main.py` | the per-unit comparison forced to `inv.batt` at N=1 too | −31 B | **RED** — AC 8 |
| M3 | `main.py` | the site half never evaluated | −6 B | **RED** — AC 3 |
| **M4** | `main.py` | **the site half armed at N=1 as well** | **+0 B** | **SURVIVOR on the first battery — GAP CLOSED**, then RED. See below |
| M5 | `main.py` | the `sign_site` claim never filed | −6 B | **RED** — `test_watchdog.py`. ⚠️ `test_peakguard.py` stayed **GREEN**, correctly: 10f's AST scan pins which sources CAN be filed and the append is still written; whether it is REACHED is a dynamic question no fixture there gets to |
| M6 | `main.py` | the site latch not persisted to the store | −58 B | **RED** — the flat-keys and restart checks |
| M7 | `main.py` | the operator clear leaves the site latch set | −42 B | **RED** — the clear check |
| M8 | `watchdog.py` | `arm()` clears `disagree_since` | +35 B | **RED** — "re-armed on every one of 38 cycles still trips" |
| M9 | `watchdog.py` | the reason hard-codes "a meter reading" again | +1 B | **RED** — `test_watchdog.py`. `test_inverter.py` stayed **GREEN**; its part-3 row reads the noun off `sign_wd.against` rather than out of the reason string, so the two checks bite in different places |
| M10 | `watchdog.py` | `unit_sign_disagree_w` 250 → 300 | **+0 B** | **RED** — the `saturation_w / 2` pin and the CFG-03 pin |
| M11 | `watchdog.py` | `site_sign_disagree_s` 325 → 180, i.e. **the number carried** | **+0 B** | **RED** — the measured-215 s check and the CFG-03 pin |
| M12 | `arbiter.py` | `sign_site` scoped `SCOPE_UNIT` | **+0 B** | **RED** — `test_arbiter.py` **and** `test_peakguard.py` |
| M13 | `validate.py` | the `unit_sign_disagree_w` rule removed | −126 B | **RED** — `test_validate.py` |
| **M14** | `main.py` | **`arm()` handed `self.sign_cfg` at EVERY roster** — the WIRING, not the values | −31 B | **SURVIVOR at review — GAP CLOSED**, then RED. See below |
| M15 | `main.py` | per-unit `arm()` given the right `min_w` and the SHIPPED hold — a HOLD-ONLY mis-wire | +78 B | **RED** — the same wiring pin |
| M16 | `main.py` | per-unit `arm()` given the right `min_w` and the SITE hold — hold-only again, and behaviour-preserving | +83 B | **RED** — the wiring pin, and the ONLY red in `test_watchdog.py` |
### M4 is recorded as a survivor because it was one
The check it should have gone red on read:
```python
not _c1s.site_sign_wd.tripped and _c1s.site_sign_wd.disagree_since is None
```
Arming the site detector at a roster of one left that **green**, and the reason
is worth reading because the check looked airtight. In the N=1 fixture the unit
detector trips first at its 1 s hold and parks the inverter; the site's standing
command is then **0 W**, so the site predicate is false, its window is never
opened, and `disagree_since is None` is true *whether or not the detector was
evaluated at all*. **The assertion ran, the sentence was true, and the data
could not reach the failure it named** — MUTATION.md's third failure mode,
inside the check written to prevent it.
Replaced with a **call counter installed on the object** before the run: twelve
cycles, zero calls, printed. A counter can only be zero for one reason. The
mutation was re-run against it and is red; the run that found it was discarded
along with its tree, so this is a result and not an artefact.
### M14: the VALUES were pinned and the WIRING was not
M10 and M11 mutate the numbers in `watchdog.SIGN_OPTION_DEFAULTS` and both are
red through the config pins. Neither of them touches the line that DELIVERS
those numbers to a detector:
```python
inv.sign_wd.arm(self.sign_cfg if one else self.unit_sign_cfg,
AGAINST_METER if one else AGAINST_OWN_POWER)
```
Drop the conditional — `arm(self.sign_cfg, ...)` — and every unit is armed with
SAFETY-02's meter-facing pair instead of the re-derived per-unit one. At review
that left **all 19 files green**. `grep -rn "sign_wd.cfg" test_*.py` found
exactly one threshold assertion, and it is §7's N=1 arm — `(300.0, 1.0)` —
which the mutation PRESERVES. Nothing anywhere read `inv.sign_wd.cfg` at a
roster of two.
The behavioural checks in part 3 cannot close it and it is worth saying why: B
is reversed at ~2000 W and every hold in that fixture is short, so B trips under
either pair. **A fault big enough and long enough to trip under both thresholds
is evidence about neither.**
What the survivor costs in the field is the whole product of §3: at
`config.yaml`'s own defaults the per-unit check would run at `sign_disagree_s`
**180 s instead of 85 s** — 2.1× slower to park a reversed inverter — and its
gate would move from 250 W to 300 W, so a unit reversing a share between the two
becomes invisible, while `SIGN_OPTION_DEFAULTS`, `config.yaml` and §3 all go on
describing a pair the running code does not use. With CI green.
**Closed by a wiring pin in `test_watchdog.py` part 3**, on the real
two-inverter Controller: every `inv.sign_wd.cfg` equals the per-unit pair,
`site_sign_wd.cfg` equals the site pair, and each detector's `against` is its
own instrument — all graded **against literals**, since `== self.unit_sign_cfg`
cannot tell *the per-unit pair* from *whatever this Controller was handed*.
⚠️ **And the fixture had to be changed for that pin to mean anything.**
`_sign_site` set `unit_sign_disagree_s: 1` **and** `site_sign_disagree_s: 1`
**and** `sign_disagree_s: 1`, with two of the three magnitude gates at 300 W. A
bare equality there discriminates on `min_w` alone: a mis-wiring that delivered
the right magnitude with the wrong hold would compare EQUAL to the pair it was
supposed to have. That is this document's own §5 lesson and MUTATION.md's third
failure mode — a check comparing two things that happen to be equal cannot see
the difference it names — reappearing inside the fix for it. The three pairs are
now pairwise distinct **in both coordinates** (250 W / 1 s, 320 W / 2 s,
300 W / 3 s), a PRECONDITION check asserts that distinctness so a later edit
collapsing them goes red rather than quietly vacuous, and both pairs are
**printed**. M15 and M16 exist to show the hold half actually bites: right
magnitude, wrong hold, red both times — and M16, whose wrong hold still trips
inside the drive, is red on the wiring pin and on **nothing else** in that file.
## 6. What this card does NOT close
* **The flapping-load blind spot in the SITE comparison.** §3, at length. Present
identically at the shipped roster of one; wants an alternation test, not a
number; a card of its own.
* **A unit misbehaving while it is NOT loop-driven.** Gate 1 (`following`) is
kept for the per-unit comparison, deliberately: the whole no-false-positive
argument is built on it, and the saturation freeze already grades that window.
So between the start of a maintenance cycle and its end, a unit reversing
against its own command is the freeze's to notice. Widening it is a card with a
rig campaign behind it, not an edit.
* **A sign inversion downstream of this add-on** — in the firmware, on the wire,
or in a future driver. Unchanged: our view stays self-consistent and both
comparisons stay silent. That is the witness's problem and it always was.
* **A false-alarm interaction with SAFETY-31 that nobody has to think about.**
They overlap in one scenario and both fire in it. That is correct — they
conclude different things — but an operator will see two alarms for one event.
Whether the banner should say so in one sentence is a UI card.
+41
View File
@@ -363,6 +363,10 @@ behind it would trip the firmware watchdog on a system that is working fine.
| `auto_start` | false | Start controlling on boot (only after commissioning) |
| `sign_disagree_w` | 300 | Both the command and the meter must exceed this before a sign difference counts. Must stay **below `max_w`** or the watchdog can never fire, and the add-on refuses to start |
| `sign_disagree_s` | 180 | How long they must stay opposed, continuously, before the failsafe trips |
| `unit_sign_disagree_w` | 250 | **Two or more inverters only.** Both a unit's own share and its own measured power must exceed this before their sign difference counts. It is `saturation_w / 2`, so the smallest divergence it can see is exactly the gap at which the loop already calls the inverter "not following". Must stay **below `max_w`** or it can never fire, and the add-on refuses to start |
| `unit_sign_disagree_s` | 85 | **Two or more inverters only.** How long one unit's command and its own power must stay opposed before that unit is parked |
| `site_sign_disagree_w` | 300 | **Two or more inverters only.** Threshold for the site-wide check - the sum of the commands against the meter. The same 300 W as `sign_disagree_w`, for the same reason: near zero the sign is noise |
| `site_sign_disagree_s` | 325 | **Two or more inverters only.** How long the site's total command and the meter must stay opposed before every inverter is parked. Longer than `sign_disagree_s` because the command has to cross zero over a rail that is the sum of the roster, at a `slew_w` that is not - measured at 110 s with one inverter and 215 s with two |
#### The sign-disagreement failsafe
@@ -374,6 +378,43 @@ there. The banner reads `FAILSAFE`, the commissioning list says which two
numbers disagreed and for how long, and the `Controller status` entity reports
`failsafe`.
#### ⚠️ At two or more inverters this is TWO checks, not one (SAFETY-30)
The paragraph above describes what a **one-inverter** install does, and it is
unchanged: there is one meter, one actuator, and comparing the command against
the meter is comparing a thing with a view of itself.
With a second inverter that stops being true. The meter answers for **both**
units, so a command put on one of them can be opposed to it with nothing wrong
(inverter A charging while inverter B discharges more), and - the worse
direction - a unit genuinely driving the wrong way can be **hidden**, because its
sibling's watts arrive at the meter and the sum looks healthy. So the check
splits in two, and they are different faults:
- **Per inverter** — that unit's own share against **its own measured power**
(the `batt_entity` reading, register 0x0518). This catches *"this inverter is
not doing what it was told"*. Only that inverter is parked; the site keeps
running on the one that works. Thresholds: `unit_sign_disagree_w` /
`unit_sign_disagree_s`.
- **Site-wide** — the **sum** of what every inverter has standing, against the
meter. This catches *"the site as a whole is pushing the wrong way"*, which no
per-inverter check can see because every unit may be faithfully executing its
share of a number that is itself wrong. **Every** inverter is parked and the
banner names the site rather than an inverter. Thresholds:
`site_sign_disagree_w` / `site_sign_disagree_s`, latched under flat
`site_sign_tripped` in `/data/state.json`.
Both latch, both are cleared by the same *stop, then start*, and both survive a
restart. **On a one-inverter install neither pair does anything at all** - the
two questions are the same question there, and only the check described above is
armed.
This is **not** the circulating-power alarm further down: that one compares the
inverters *against each other* and catches them pushing power through each other,
which nets to zero at the meter and which no command is evidence about. It is
also not the saturation freeze, which is a *magnitude* test that shapes the loop
rather than a *direction* test that parks an inverter.
**It does not clear itself, and restarting the add-on does not clear it either.**
Pressing *Start / stop control* twice — stop, then start — is the only thing that
does. That is deliberate on both counts:
+63 -3
View File
@@ -135,11 +135,19 @@ manufactures them from a **fixture**: the check runs, the assertion is
evaluated, the sentence above it is true — and the numbers it compares are
arranged so that the failure it describes could not have changed the answer.
It is not hypothetical and it is not rare. Six instances in two weeks; three
It is not hypothetical and it is not rare. EIGHT instances in two weeks; four
were found by review, two - CFG-04's fixture pair - by an author's own mutation
battery, which is the cheaper place, and the sixth by review again, in the one
battery, which is the cheaper place, one by review again, in the one
place nobody had thought to point the rule at: not a check in the suite, but a
COMMITTED EVIDENCE ARTIFACT four documents cite.
COMMITTED EVIDENCE ARTIFACT four documents cite - and the eighth INSIDE THE FIX
WRITTEN TO CLOSE THE SEVENTH.
⚠️ **AND IT HAS NOW APPEARED IN SIX CONSECUTIVE CARDS.** The seventh instance,
MAINT-05's, is the one that says why: the defect is not carelessness about
fixtures, it is that **a check and the thing it checks were built from the same
constant**. Ask of every difference-claim not "is this assertion true" but
"where did each side of this comparison come from" - and if the answer is the
same place twice, the check is decoration.
* **ARB-03** shipped a call-site scan asserting that every `claims.append` in
`cycle()` routes through `actuator_of()`. Its own parse **saw 3 of the 8
@@ -167,6 +175,44 @@ COMMITTED EVIDENCE ARTIFACT four documents cite.
than against its behaviour. It now drives the shipped `run_control()` on a
two-inverter config, **under a deadline**, because the mutation's real symptom
is that the controller enters its loop and never returns.
* **MAINT-05** shipped `Controller.__init__` calling
`migrate_maint_keys(store, self.inverters[0].name, ...)` under a comment
saying *"it targets `self.inverters[0].name` and NOT `UNIT_A` ... writing
`UNIT_A` here would deposit an install's whole maintenance history under a
name that is not on the roster at all"*. `inverter.UNIT_A` **is** the string
`"inverter-a"`, and every two-pack fixture in `test_maintenance.py` named its
first roster entry `inverter-a` — so the two expressions were the same value
in every check that existed, and swapping them left **all 19 files green**
while losing a real install's whole maintenance history. Two-pack fixtures are
now `["garage", "shed"]`, each behind a precondition check (habit 1) that the
first name is not `UNIT_A`.
⚠️ **AND THE POSTCONDITION HAD THE SAME SHAPE AS THE FIXTURE.**
`store._verify_moves()` read back the keys it had just written, under the
namespace it had just been handed — so it verified clean, and the caller
stamped the schema marker, on a migration nothing would ever read. *A check
that reads back the namespace it was handed cannot detect being handed the
wrong one*: it compared a thing against itself, exactly as DEV-02's fixture
compared `(-5000, 5000)` against `(-5000, 5000)`. The habit-2 fix is the same
one — grade against something the thing under test did not supply, here the
ROSTER the machines read.
⚠️ **AND THE FIX ITSELF CARRIED THE DEFECT, WHICH IS THE SIXTH CARD IN A ROW
AND THE FIRST TIME IT HAS APPEARED *INSIDE* A REMEDY.** The fix round's
recovery check was `migrate_maint_keys(_m5, ROSTER[0], ROSTER)` — the
**corrected** name. It measures recovery along the one path where the operator
has *already fixed the bug*, and nothing in the 19 files called
`migrate_maint_keys()` twice with the **same wrong** `unit`. So the refusal
the card was rejected twice to obtain lasted **exactly one start**: measured
at Controller level over one persisted store, roster `["garage", "shed"]` —
construction 1 `ok=False misdirected=4 moved=4 kept=0 marker=<absent>`,
construction 2 `ok=True misdirected=0 moved=0 kept=4 marker=1`. On start 2
rule 1 routes the wrongly-written keys to `rep.kept`, `rep.moved` is empty,
the roster loop iterates over nothing, and the marker is stamped on a
migration that reached no readable key — after which the early return skips it
forever, *including after the call site is fixed*. The check was true and
passed for the wrong reason. Habit **9** below is what falls out of it.
* **CFG-04's `tools/supervisor_shape_probe.py` §B** — and this one is the rule
reaching outside the suite. The probe is the committed evidence that
`config.yaml`'s `inverters: []` default is load-bearing, and four documents
@@ -241,6 +287,18 @@ Cheap habits fall out of it, and all of them are now used in this repository:
red. `inverters: []` was deleted from `config.yaml` and the probe re-run
against `ghcr.io/home-assistant/amd64-hassio-supervisor:2026.08.0` for
exactly that reason.
9. **Exercise a refusal a SECOND time with the SAME wrong input — not the
corrected one.** A guard that refuses once and then admits the same bad
caller is not a guard, it is a delay, and the second call is a *different
code path* whenever the first call left state behind. MAINT-05's roster
grading iterated the keys it had just **written**; on the second start the
same wrong caller found them already present, they moved to the
`already namespaced` list, and the loop graded an empty set. Re-running with
the **fixed** argument — which is what the check did — measures the operator's
recovery, never the persistence of the refusal. Ask of every refusal: *what
does this do when the caller has not been fixed and simply runs again?* And
assert the **shape** of that second report, not only its verdict, so the
check cannot silently revert to grading the first path.
And the mutation-table consequence: **a variant is not covered because a
near-miss of it is.** DEV-02's table listed the *actuator* variant of the
@@ -297,6 +355,8 @@ commit and PR bodies.
| **TARIFF-01** reviewer survivor: `record_block()` ignoring its `policy` argument (0dbd485) | **re-verified — still a survivor** | re-run here through `tools/mutation_run.sh` against the 15 suites that were on the record, all green with `billed_w(block, policy)` mutated to `billed_w(block, POLICY_NET)`. The recorded diagnosis holds: every block reaching the store is single-phase, where NET, SUM and MAX collapse. **Now CLOSED by TEST-13** (`test_tariff.py` section 7), which round-trips section 7's own unbalanced `(3000, 500, -200) W` block THROUGH the store under each policy - three distinct banked figures, so a hardcoded policy takes at least two of them red. Third instance of "a stimulus evaluated where the quantity it grades is degenerate", after WITNESS-02's span normalisation and TARIFF-01's own DST window. |
| **SAFETY-25** 8 mutations against `test_peakguard.py` part 10, then 9 more after review, all red | **not at risk** | each run against a tree copied fresh with `__pycache__` excluded and mounted `:ro`, so no `.pyc` existed to go stale - the same structural immunity recorded for CI above. Byte deltas recorded on the PR; none is `+0 B`. All are **reds**, which is the direction this defect cannot fabricate. ⚠️ The second round exists because the first round's mutations were all aimed at DISARMING the guard, and the claim part 10 actually made was about the guard being FIXED - the reviewer built the fix and the suite stayed green. Bytecode was never the issue there; an unaimed battery was. |
| **SAFETY-03** reviewer survivor: `abs()` dropped from `clamp_to_rating`'s limit (6aabb83) | **re-verified — still a survivor** | re-run the same way, all 15 green. Also as recorded: unreachable under the `int(100, 5000)` schema. |
| **MAINT-05 round 3**, `migrate_maint_keys()` marker gate `if rep.ok:` → `if rep.ok or rep.kept:` (`+12 B`) | **was a survivor — now CLOSED** | measured against head `65cddf3` under the purge, tree mounted `:ro`, `19/19 PASS`. It stamps the schema marker on a migration that LOST state, defeating the older `lost` guard as well as the new `misdirected` one, because every check that reached the marker did so on a store whose `kept` list was empty. Closed by `test_maintenance.py` **9d**; the mutation is now red on three named checks. |
| **MAINT-05 round 3**, `migrate_unit_keys()` marker gate `if rep.ok:` → `if rep.ok or rep.kept:` (`+12 B`) | **found here — SAFETY-02's half, now CLOSED** | the SAME mutation one function up, on the LATCH migration rather than the history one. Measured a survivor against this branch under the purge, `:ro`, `19/19 PASS`, then closed by `test_inverter.py` part 10's new fixture — a store with `unit/inverter-a/sign_reason` already present (so `kept` is non-empty) whose `sign_tripped: true` write is dropped. Shipped: marker withheld. Mutant: marker stamped on a **disarmed latch**, and the next start skips the migration that would have re-armed it. |
| **TARIFF-01**'s other three survivors, **SAFETY-03**'s `describe()` survivor, **TARIFF-02**'s M3/M5/M7, **SAFETY-18**'s three | **still unverified** | recorded before the purge discipline, with no byte delta on the record, so a `+0 B` same-second revert cannot be excluded. All are survivors, which is the direction this defect fabricates. TARIFF-02's and SAFETY-18's were **closed by later commits whose claims are reds**, so what is unverified is the *diagnosis*, not the current state of the code. |
**Nothing here says a past result is wrong.** The two survivors re-run above
+40 -7
View File
@@ -151,6 +151,30 @@ CLAIM_SCOPE = {
# other went on feeding the loop. The evidence is site-level and so is the
# remedy: stop commanding, everywhere, and say why.
"circulation": SCOPE_SITE,
# main.py: `Claim.set("sign_site", P_SAFETY, 0.0, "site sign disagreement")`.
# SAFETY-30, and it is the twin of `failsafe` below rather than a duplicate
# of it - the two exist BECAUSE the same words describe two different
# measurements once there is more than one inverter.
#
# `failsafe` grades ONE unit's standing command against THAT UNIT'S OWN
# measured power (register 0x0518). Its conclusion is "the actuator I am
# driving is not doing what I told it", which is a fact about one device,
# so it parks one device.
#
# This one grades the SUM of the standing commands against P1. Its
# conclusion is "the site as a whole is driving the wrong way" - an inverted
# gain, an integrator wound past the rail, a strategy on the wrong side of
# the meter - and there is nothing per-unit about it: the quantity is a sum
# over the roster and the offending number is the site target, which no
# single inverter owns. Parking one unit on that evidence would leave the
# rest of the site pushing the same way with the guard satisfied.
#
# ⚠️ AT N=1 IT IS NEVER FILED, and that is the identity rather than a
# special case: with one inverter the sum of the commands IS that unit's
# command and `failsafe` already carries exactly this trip, under the keys,
# the banner and the claim an install has had since SAFETY-02 shipped.
# `Controller.cycle()` does not even arm the site detector there.
"sign_site": SCOPE_SITE,
# --- UNIT: about one pack, one inverter, one device -------------------
# main.py: `Claim.limit("device", P_SAFETY, self.dev_min, self.dev_max,
@@ -164,13 +188,22 @@ CLAIM_SCOPE = {
# same reason, so it is scoped like `device`.
"supervised": SCOPE_UNIT,
# main.py: `Claim.set("failsafe", P_SAFETY, 0.0, "sign disagreement")`.
# ⚠️ THE EVIDENCE IS SITE-WIDE AND THE REMEDY IS NOT, and that is why this
# entry needs its reasoning written down. SAFETY-02 compares P1 - a site
# measurement - against `standing`, THE COMMAND THIS CONTROLLER PUT ON THIS
# ACTUATOR. What it concludes is "the unit I am driving is pushing power the
# wrong way", and what it does is park that unit. At N>=2 the same evidence
# cannot be attributed to a sibling nobody here commands, so a site-wide
# remedy would be this watchdog stopping an inverter it never graded.
# ⚠️ THE EVIDENCE IS PER-UNIT AND SO IS THE REMEDY - AND SINCE SAFETY-30 THE
# FIRST HALF OF THAT SENTENCE IS TRUE. It used to read "the evidence is
# site-wide and the remedy is not", because the watchdog compared THIS
# unit's standing command against P1: a site measurement, attributed to one
# actuator on the strength of §6.2's N=1 identity. That identity does not
# survive a second inverter - at N>=2 the meter answers for a sibling this
# unit's command says nothing about, in both directions (a false trip when
# the sibling covers the meter, and a missed one when the sibling MASKS a
# unit that really is reversed).
#
# So at N>=2 the comparison behind this claim is THIS unit's command against
# THIS unit's own measured power - register 0x0518, per-instance since
# DEV-02 - and the scope that was already right for the wrong reason is now
# right for the right one. The site-wide half moved to `sign_site` above.
# At N=1 the two are the same comparison and this claim is the one filed,
# exactly as it shipped.
"failsafe": SCOPE_UNIT,
# main.py: `Claim.set("maintenance", P_MAINTENANCE, ...)` when the cycle owns
# the setpoint, and `Claim.limit("maintenance", P_MAINT_SHAPE, hi=...)` when
+634 -121
View File
@@ -54,17 +54,22 @@ from .hass import HomeAssistant
# window an operator would look up here is the one the roster actually uses.
from .inverter import (LINK_STALE_S, UNIT_A, build_roster,
roster_entries)
from .maintenance import IDLE, MaintConfig, Maintenance
from .maintenance import (
IDLE, POWER_MOVING, MaintConfig, Maintenance, arbitrate as maint_arbitrate,
)
from .mqtt import MqttPublisher
from . import peakguard
from .p1 import P1Ingest, build_source, is_enabled
from .price import Uncertainty, price_option
from .static_tariff import resolve_tariff, static_status
from .tariff import PeakStore, PeakTracker
from .store import migrate_unit_keys
from .store import migrate_maint_keys, migrate_unit_keys
from .validate import (ConfigRefused, refuse, validate_startup_config,
validate_unit_ranges)
from .watchdog import SignWatchdogConfig, loop_is_following
from .watchdog import (
AGAINST_METER, AGAINST_OWN_POWER, SIGN_OPTION_DEFAULTS,
SignWatchdog, SignWatchdogConfig, loop_is_following,
)
from . import web
OPTIONS_PATH = "/data/options.json"
@@ -173,10 +178,42 @@ class Controller:
# first or every existing tripped install comes up ARMED, into the
# fault, with a green banner - which is the exact failure the latch was
# persisted to prevent, rebuilt by a key rename.
sign_cfg = SignWatchdogConfig(
# SAFETY-02's pair, and since SAFETY-30 it is THE SITE COMPARISON's
# pair - which at the shipped roster of one is the same thing under a
# more careful name. It is what `Inverter.sign_wd` is armed with at N=1,
# byte for byte, and what the site detector is armed with at N>=2.
self.sign_cfg = sign_cfg = SignWatchdogConfig(
min_w=float(opts.get("sign_disagree_w", 300)),
hold_s=float(opts.get("sign_disagree_s", 180)),
)
# SAFETY-30. THE PER-UNIT COMPARISON'S OWN PAIR, RE-DERIVED AT TWO
# INVERTERS AND NOT CARRIED ACROSS. `sign_disagree_w` / `sign_disagree_s`
# were measured on two SINGLE-INVERTER bus captures against a command
# opposed to THE METER; this comparison is a command against ONE UNIT'S
# OWN POWER, where the magnitudes, the transient and the worst
# legitimate episode are all different quantities. watchdog.py's
# SIGN_OPTION_DEFAULTS carries the campaign that produced these, and
# DECISIONS/SAFETY-30-per-unit-sign.md carries the tables.
self.unit_sign_cfg = SignWatchdogConfig(
min_w=float(opts.get("unit_sign_disagree_w",
SIGN_OPTION_DEFAULTS["unit_sign_disagree_w"])),
hold_s=float(opts.get("unit_sign_disagree_s",
SIGN_OPTION_DEFAULTS["unit_sign_disagree_s"])),
)
# SAFETY-30. THE SITE COMPARISON'S OWN PAIR, USED ONLY AT N>=2. The
# threshold is the shipped 300 W - carried, because the argument behind
# it is about the meter's noise floor and the deadband, neither of which
# the roster touches - and the WINDOW is not, because the transient it
# rejects is the loop crossing zero at `slew_w` per cycle across a rail
# that IS the roster sum. Measured at both rosters on the same stimuli:
# 110 s at one inverter, 215 s at two, against a 180 s hold. See
# watchdog.SIGN_OPTION_DEFAULTS.
self.site_sign_cfg = SignWatchdogConfig(
min_w=float(opts.get("site_sign_disagree_w",
SIGN_OPTION_DEFAULTS["site_sign_disagree_w"])),
hold_s=float(opts.get("site_sign_disagree_s",
SIGN_OPTION_DEFAULTS["site_sign_disagree_s"])),
)
self.store_migration = migrate_unit_keys(store, UNIT_A)
self.inverters = build_roster(opts, store, sign_cfg)
# ⚠️ CFG-04. SAY WHICH SHAPE IS IN FORCE, BECAUSE THE OTHER ONE'S KEYS
@@ -230,7 +267,66 @@ class Controller:
# maintenance.MAINT_OPTION_DEFAULTS before adding a ninth key: it goes
# in config.yaml's `options:` and `schema:` blocks and in that dict, and
# nowhere else.
self.maint = Maintenance(MaintConfig.from_options(opts), store)
# --- MAINT-05: ONE `Maintenance` PER ROSTER ENTRY --------------------
# ⚠️ AND THE MIGRATION RUNS FIRST, FOR THE REASON THE SAFETY-02 ONE
# DOES. These objects READ their history in their constructors' wake -
# `phase`, `last_completed` - so a history still sitting under the flat
# spelling when they are built is a pack that comes up believing it has
# never been recalibrated, on a site whose state file says otherwise.
# It targets `self.inverters[0].name` and NOT `UNIT_A`: the flat keys
# belonged to whichever pack the one FSM was driving, which is the first
# roster entry, and a declared `inverters:` roster names its own entries
# (`inverter.build_roster()` says so, and its fallback for a nameless
# entry is a POSITIONAL `inverter-1`, so `inverter-a` is not even the
# default there). Writing `UNIT_A` here would deposit an install's whole
# maintenance history under a name that is not on the roster at all.
#
# ⚠️ AND THAT SENTENCE USED TO BE A COMMENT NOTHING COULD GRADE, WHICH
# IS WHY THE ROSTER IS PASSED RATHER THAN A COUNT. Every two-pack
# fixture in the suite named its first entry `inverter-a`, so
# `self.inverters[0].name` and `UNIT_A` were the same string in every
# check that existed and swapping them left all 19 files green - while
# on a roster of `["garage", "shed"]` it silently loses the history and
# stamps the marker, so the next start skips the migration too.
# `migrate_maint_keys()` now grades the namespace it wrote against the
# names the machines read; test_maintenance.py sections 9 and 11 drive
# a `["garage", "shed"]` roster, which is what makes the mutation red
# instead of invisible.
self.maint_migration = migrate_maint_keys(
store, self.inverters[0].name, [inv.name for inv in self.inverters])
_LOG.info("maintenance history: %s", self.maint_migration.explain())
if not self.maint_migration.ok:
# Not a disarmed guard - history, not a latch - so it does not
# re-arm anything. It is still the difference between a pack that is
# recalibrated on schedule and one that silently is not, which is
# §12's opening sentence, so it is said where an operator looks.
self.log_event("maintenance history did NOT migrate cleanly: "
+ self.maint_migration.explain())
# ⚠️ ONE CONFIG, N MACHINES. The `maintenance_*` options describe the
# PROCEDURE - how hard to drain, how full to charge, how long to hold -
# and the procedure is the same on every pack in the roster; what is per
# pack is the SCHEDULE and the HISTORY, which live in the store. A
# per-entry config block would be a second shape for CFG-04's roster to
# carry and a fifth place for a `maintenance_charge_w` to disagree with
# config.yaml. If a site ever needs two different procedures, that is a
# roster-schema card and it starts here.
_maint_cfg = MaintConfig.from_options(opts)
_one_pack = len(self.inverters) <= 1
self.maints = [
Maintenance(
_maint_cfg, store,
# ⚠️ `None` AT A ROSTER OF ONE IS THE FLAT LAYOUT, NOT A MISSING
# NAME. §6.2's identity: one pack, one history, and the keys it
# has been written under since the add-on shipped. See
# `maintenance.UnitHistory` and `store.migrate_maint_keys()` -
# this is the line that keeps a one-inverter state file
# byte-identical.
unit=None if _one_pack else inv.name,
name=inv.name,
# The rollback view, and only ever the first pack's: the flat
# keys mean `inverters[0]` on every install that has them.
mirror_flat=(not _one_pack and i == 0))
for i, inv in enumerate(self.inverters)]
# SAFETY-02. ONE INSTANCE PER ACTUATOR - which is what watchdog.py's own
# docstring has always said and what this Controller did not do. Each
@@ -261,6 +357,45 @@ class Controller:
"opposed the meter.", inv.name,
inv.sign_wd.reason or "(reason not recorded)")
# --- SAFETY-30: THE SITE HALF OF THE SIGN CHECK ----------------------
# ⚠️ ONE PER SITE, BESIDE THE ONE PER INVERTER, AND ARMED ONLY AT N>=2.
# The per-unit watchdogs above ask "is THIS actuator doing what I told
# it", graded on that unit's own measured power. This one asks "is the
# SITE driving the wrong way", graded on the SUM of the standing
# commands against P1 - the comparison SAFETY-02 shipped, re-scoped to
# the quantity it was always actually about.
#
# ⚠️ AT A ROSTER OF ONE IT IS CONSTRUCTED AND NEVER EVALUATED, and that
# is the whole compatibility promise of this card rather than an
# oversight. With one inverter the sum of the commands IS that
# inverter's command, so arming this would be arming a duplicate of
# `inverters[0].sign_wd`: two latches for one fault, two banners, and a
# false-positive record measured on ONE comparison being spent twice.
# `cycle()` is where that is enforced and says so at the line.
#
# Restored from the store like SAFETY-02's and SAFETY-31's latches, for
# the reason store.py gives: "restart the add-on" is the first thing an
# operator looking at a FAILSAFE banner tries, and a latch held only in
# memory brings the controller back armed, into the fault, unattended.
# FLAT keys, not namespaced - store.py carries that argument too.
self.site_sign_wd = SignWatchdog(
self.site_sign_cfg,
tripped=bool(store.data.get("site_sign_tripped", False)),
reason=str(store.data.get("site_sign_reason", "") or ""))
if self.site_sign_wd.tripped:
_LOG.error(
"SAFETY-30 (site) was TRIPPED before this restart and stays "
"tripped: %s. Every inverter is held at 0 W. Restarting does "
"not clear this - start control from the UI once you know why "
"the site's total command opposed the meter.",
self.site_sign_wd.reason or "(reason not recorded)")
# And the per-unit detectors are told which question they are asking,
# here as well as in `cycle()`. `checks()` and the status page read
# `inv.sign_wd.cfg` and can be called before the first cycle has run, so
# a roster of two would otherwise advertise the site thresholds on a
# detector that is about to be graded against a unit's own power.
self._arm_sign_watchdogs()
# SAFETY-18. The frozen-but-fresh meter detector - see freeze.py for why
# it is a conjunction and not an `unchanged_s` threshold, and for the
# measurements behind the default.
@@ -379,6 +514,9 @@ class Controller:
self._peak_saturated = False
self._peak_floor_limited = False
self._maint_shape_suspended = False
# [(pack name, why)] for packs that are due and waiting their turn.
# Empty at a roster of one, where nothing can defer anything.
self._maint_deferred = []
# Set by amain() once the transport is built, so the status page can show
# what only the transport knows (homewizard_local's unchanged_s). Stays
# None when P1 is off, or under a transport that has no such counter.
@@ -593,18 +731,79 @@ class Controller:
return tuple(inv.name for inv in self.inverters)
@property
def maint_unit(self):
"""The pack the ONE maintenance state machine is recalibrating.
def maint_index(self) -> int:
"""Which roster slot holds the site's ONE maintenance slot right now.
⚠️ NAMED RATHER THAN ASSUMED. This add-on holds exactly one
`Maintenance`, whose history lives under the store's un-namespaced
`phase` / `last_completed` keys, so the pack it drives is the first in
the roster. Two packs recalibrating on two schedules needs a second FSM
and a namespaced history of its own; what THIS card fixes is only that
the claim is filed on ONE actuator instead of on whichever one happened
to be asked. Read store.LEGACY_UNIT_KEYS on why those keys did not move.
The pack that is mid-cycle if there is one, otherwise the first - which
at a roster of one is always 0, always was, and is the whole of this
property's N=1 behaviour.
"""
return self.inverters[0]
for i, m in enumerate(self.maints):
if m.phase in POWER_MOVING:
return i
return 0
@property
def maint(self):
"""The `Maintenance` the site's one maintenance slot belongs to.
⚠️ MAINT-05 TURNED THIS FROM AN ATTRIBUTE INTO A VIEW, AND THE THING IT
IS A VIEW OF CHANGED WITH IT. There are now `self.maints` - one FSM per
roster entry, each with its own schedule, phase and history - and every
SITE-level surface that used to read `self.maint` (the `phase` MQTT
value, the status banner, the operator's start/abort buttons) wants THE
ONE THAT IS RECALIBRATING, not "the first one". `maintenance.arbitrate()`
guarantees at most one pack is ever out of IDLE, so "the one that is
recalibrating" is well defined; when none is, this is `maints[0]` and
the surfaces read exactly what they read before.
⚠️ AT A ROSTER OF ONE IT IS `maints[0]` ON EVERY TICK, in both branches,
which is the object that used to be assigned here. Nothing downstream
can tell the difference - that is AC 6, and test_maintenance.py part 9
asserts it rather than trusting this paragraph.
"""
return self.maints[self.maint_index]
@property
def maint_unit(self):
"""The pack the site's one maintenance slot is currently on.
⚠️ THIS USED TO BE `self.inverters[0]`, FLATLY, AND THE COMMENT THAT
STOOD HERE EXPLAINED WHY: there was exactly one `Maintenance`, its
history lived under the store's un-namespaced `phase` / `last_completed`
keys, so the pack it drove could only be the first in the roster. That
sentence is what MAINT-05 removed. There is now one FSM and one
namespaced history PER ROSTER ENTRY, so the pack being recalibrated is
whichever one holds the site's maintenance slot, and `maint_index` -
not the roster's order - is what answers it.
⚠️ AND `validate.py` RULE 6 MOVED WITH IT, exactly as its own comment
said it must ("when `maint_unit` stops being `inverters[0]`, this line
moves with it"). It graded `maintenance_charge_w` against the FIRST
entry's cap on the argument that the cycle never touches the others;
that argument is now false, so it grades against the weakest cap on the
roster, like rule 5. See the ⚠️ block there.
"""
return self.inverters[self.maint_index]
def _maint_button(self, now):
"""The pack an operator's one maintenance button acts on.
The pack holding the site's maintenance slot when one holds it, so
"abort" stops the cycle that is running and "start" restarts it -
exactly what both buttons meant while there was one pack, and the reason
this is not simply "the longest-waiting one". When every pack is idle it
IS the longest-waiting one, which is `arbitrate()`'s own clause 2, so a
forced start and a scheduled start choose the same pack rather than two
different ones.
⚠️ AT A ROSTER OF ONE BOTH BRANCHES ARE `maints[0]`. There is no third
case: `max()` over a one-element list is that element.
"""
if self.maints[self.maint_index].phase in POWER_MOVING:
return self.maints[self.maint_index]
return max(enumerate(self.maints),
key=lambda p: (p[1].waited_days(now), -p[0]))[1]
def inverter(self, name: str):
"""The roster entry called `name`, or None."""
@@ -712,6 +911,45 @@ class Controller:
def sign_wd(self):
return self._only().sign_wd
# -- SAFETY-30: which comparison each sign detector is making -------------
def _arm_sign_watchdogs(self) -> None:
"""Tell every sign detector which of SAFETY-30's two questions it asks.
⚠️ A PURE FUNCTION OF `len(self.inverters)`, AND THE ONLY PLACE THE
ROSTER DECIDES ANYTHING ABOUT THIS CHECK. Written as one method rather
than as a conditional at the two call sites for `loop_is_following()`'s
reason: the arrangement that does not keep two things in step is the one
where they are written down twice.
AT ONE INVERTER - §6.2's identity, and the whole of AC 8. The single
unit's detector is armed against THE METER with the shipped
`sign_disagree_w` / `sign_disagree_s`, which is what it has always been
armed against, and the site detector is not armed at all. A one-inverter
install therefore makes the same comparison, at the same thresholds,
writes the same reason string, latches under the same keys and files the
same `failsafe` claim as it did before this card.
AT TWO OR MORE - the identity is gone and the meter stops being a view
of any one actuator. Each unit's detector is armed against ITS OWN
MEASURED POWER (`Inverter.batt`, register 0x0518) under the re-derived
per-unit pair, and the site detector is armed against the meter under
the site pair. Two questions, two remedies, two latches.
⚠️ CALLED AGAIN AT THE TOP OF EVERY `cycle()`, not only from
`__init__`. The roster is re-read every cycle for liveness and for the
allocation, and this repository's two-inverter fixtures append an
`Inverter` AFTER the Controller is constructed - a detector frozen at
construction would then be graded against the meter at a roster of two,
which is the defect this card exists to remove, wearing a fixture's
name. `arm()` touches no timer and no latch, so calling it every cycle
cannot disturb an episode in progress.
"""
one = len(self.inverters) <= 1
for inv in self.inverters:
inv.sign_wd.arm(self.sign_cfg if one else self.unit_sign_cfg,
AGAINST_METER if one else AGAINST_OWN_POWER)
self.site_sign_wd.arm(self.site_sign_cfg, AGAINST_METER)
# -- the site aggregates, each one named for the question it answers ------
# ⚠️ THESE ARE NOT "THE SCALARS WITH A SUM IN FRONT". Each one picks a
# DIFFERENT reduction, because the questions are different, and getting the
@@ -1216,14 +1454,36 @@ class Controller:
"""
now = datetime.now(timezone.utc)
# ⚠️ THE PACK IT IS MAINTAINING, NOT "the SoC". §12's recalibration
# drains and refills ONE pack and grades ONE coulomb counter, so it is
# graded on that pack's own reading - see `maint_unit`. A minimum across
# packs would abort the cycle on a sibling's state, and an average would
# declare a pack full that is not.
result = self.maint.tick(now, self.maint_unit.soc)
for msg in result.events:
self.log_event(msg)
# ⚠️ EACH PACK ON ITS OWN READING, NOT "the SoC". §12's recalibration
# drains and refills ONE pack and grades ONE coulomb counter, so each
# state machine is ticked with the SoC of the inverter it belongs to. A
# minimum across packs would abort a cycle on a sibling's state, and an
# average would declare a pack full that is not. The pairing is
# positional and both lists are built from the roster in `__init__`, so
# a pack cannot be graded on its neighbour's meter.
#
# ⚠️ AND `arbitrate()` RUNS FIRST, ONCE, OVER ALL OF THEM. It is the
# site's one maintenance slot - see maintenance.py - and a pack that
# does not hold it is told so by name rather than left to discover it.
# Calling `tick()` without the answer this produced is the
# two-packs-draining-at-once defect with the interlock present but not
# wired, which is why the plan is built here and not inside the loop.
maint_plan = maint_arbitrate(self.maints, now)
maint_results = []
for inv, machine, blocked in zip(self.inverters, self.maints, maint_plan):
res = machine.tick(now, inv.soc, blocked_by=blocked)
for msg in res.events:
self.log_event(msg)
maint_results.append(res)
# ⚠️ CARRIED TO THE STATUS PAGE, NOT LEFT IN THE 40-LINE ACTION LOG. The
# deferral is edge-logged (it lasts as long as the other pack's cycle),
# so by the time an operator asks "why has pack B not recalibrated" the
# line that said so has scrolled off. §12's whole failure mode is a
# schedule that silently stops firing; a pack that is due, idle and
# WAITING has to be able to say so at the moment it is looked at.
self._maint_deferred = [(m.name, r.deferred)
for m, r in zip(self.maints, maint_results)
if r.deferred]
# --- SAFETY-02, and it runs BEFORE the early return below -------------
# ⚠️ ORDER MATTERS HERE. If this were evaluated after the "inputs missing
@@ -1239,17 +1499,44 @@ class Controller:
# case where the last write was REJECTED - the inverter is then still
# latched on `last_write`, which is exactly what `last_write` holds).
# ⚠️ ONE WATCHDOG PER INVERTER, EVALUATED ON THAT INVERTER'S OWN
# STANDING COMMAND. The meter is site-wide - there is one P1 - but what
# the watchdog concludes is "THE UNIT I AM DRIVING is pushing power the
# wrong way", and that evidence cannot be pinned on a sibling this
# controller commanded differently. `inv.target` is that unit's share
# from the last resolution, which at N=1 is the site target itself.
# STANDING COMMAND. What the watchdog concludes is "THE UNIT I AM
# DRIVING is pushing power the wrong way", and that evidence cannot be
# pinned on a sibling this controller commanded differently.
# `inv.target` is that unit's share from the last resolution, which at
# N=1 is the site target itself.
#
# ⚠️ SAFETY-30: AND WHAT IT IS COMPARED AGAINST IS DECIDED BY THE ROSTER.
# This block used to hand every unit's detector `self.grid`. There is
# ONE P1, so at N>=2 that compares a number this controller put on ONE
# actuator against a measurement that answers for ALL of them, and it is
# wrong in both directions:
#
# FALSE TRIP inverter A commanded -1500 W while the meter reads
# +1500 W is evidence of nothing when inverter B is
# discharging 3000 W exactly as instructed. Nothing is
# faulty and the highest-value check in the safety table
# fires - on a record of zero false positives.
# MISSED the converse, and it is worse. A unit commanded +1500 W
# and MEASURED at -1500 W, beside a sibling at +1500 W,
# leaves P1 exactly where a healthy site leaves it. The
# sum destroys the evidence; no threshold recovers it.
#
# So at N>=2 each unit is graded against ITS OWN measured power -
# `inv.batt`, register 0x0518, per-instance since DEV-02 - which is the
# only instrument that can answer "is THIS actuator doing what I told
# it". At N=1 §6.2's identity holds, the meter IS the view of that one
# unit, and the comparison stays the meter, byte for byte. The site's
# own question has not gone away; it is asked below, once, of the sum.
# `_arm_sign_watchdogs()` states the choice and its docstring argues it.
self._arm_sign_watchdogs()
n_units = len(self.inverters)
standing = {}
for inv in self.inverters:
was_tripped = inv.sign_wd.tripped
stood = inv.last_write if inv.last_write is not None else inv.target
standing[inv.name] = stood
if inv.sign_wd.evaluate(stood, self.grid, self.following, now) \
against = self.grid if n_units <= 1 else inv.batt
if inv.sign_wd.evaluate(stood, against, self.following, now) \
and not was_tripped:
# ⚠️ REASON FIRST, FLAG SECOND, and each `set` writes the file.
# The ordering is the whole atomicity story: a crash between the
@@ -1290,6 +1577,50 @@ class Controller:
# share of it. At the shipped roster of one the two are the same number.
site_standing = float(sum(standing.values()))
# --- SAFETY-30: THE SITE HALF OF THE SIGN CHECK ----------------------
# The comparison SAFETY-02 shipped, asked of the quantity it was always
# about: the TOTAL this controller has standing on the wire, against the
# meter that total is answerable for. What it catches is a fault no
# per-unit check can see, because every unit is faithfully executing its
# share of a site target that is itself pointing the wrong way - an
# inverted gain, an integrator wound past the rail, a strategy on the
# wrong side of the meter. That is the 2026-08-20 fault day, and it is
# the reason the site question survives rather than being replaced.
#
# ⚠️ ONLY AT N>=2, AND THIS BRANCH IS AC 8. At one inverter the sum of
# the commands IS that inverter's command and the loop above has already
# made exactly this comparison, at exactly these thresholds, on the
# detector whose latch, store keys, claim and banner an install has had
# since SAFETY-02 shipped. Arming this as well would be two latches for
# one fault, two banners, and the same false-positive record spent
# twice. A shipped install does not reach this block at all.
if n_units > 1:
was_site = self.site_sign_wd.tripped
if self.site_sign_wd.evaluate(site_standing, self.grid,
self.following, now) and not was_site:
# REASON FIRST, FLAG SECOND - the same atomicity ordering the
# per-unit trip uses above and SAFETY-31 uses below: a crash
# between the two must not persist a latch with no explanation
# for the operator who has to decide about clearing it.
#
# ⚠️ AND THE KEYS ARE FLAT, NOT NAMESPACED, AND NOT THE
# PRE-DEV-02 PAIR EITHER. `sign_tripped` / `sign_reason` are the
# ROLLBACK VIEW of a UNIT latch (store.py rule 2) and mirroring
# a site trip onto them would hand a downgraded add-on - which
# drives exactly one actuator - a latch taken on evidence about
# a roster it cannot have. `site_sign_*` is its own pair, for
# the same reason `circ_*` is.
self.store.set("site_sign_reason", self.site_sign_wd.reason)
self.store.set("site_sign_tripped", True)
self.log_event("FAILSAFE site: " + self.site_sign_wd.reason)
_LOG.error("SAFETY-30 TRIPPED at the SITE - %s. That is the "
"SUM of every standing command opposing the meter, "
"which no single inverter is answerable for. "
"Commanding 0 W on every inverter and holding it "
"until control is started again. This survives an "
"add-on restart; restarting will not clear it.",
self.site_sign_wd.reason)
# --- SAFETY-31, AND IT RUNS BESIDE SAFETY-02 FOR THE SAME REASON -----
# Evaluated EVERY cycle, unconditionally, above the "inputs missing but
# not yet stale" hold below. A run of held cycles would otherwise leave
@@ -1329,8 +1660,14 @@ class Controller:
# list below, because its `failsafe` SET is what parks it. One tripped
# unit therefore takes the whole roster past the hold, which is the
# conservative direction: every other unit's claims are re-filed too.
# ⚠️ AND SAFETY-30's SITE LATCH TAKES THE ROSTER PAST THE HOLD TOO, for
# the same reason: its `sign_site` SET is what parks every inverter, and
# a latched site that returned early here would leave each unit's last
# share standing for the heartbeat to re-emit - i.e. the trip would stop
# acting the moment a poll was missed.
if not self.inputs_ok and not any(i.sign_wd.tripped
for i in self.inverters):
for i in self.inverters) \
and not (n_units > 1 and self.site_sign_wd.tripped):
bad_for = (now - self.inputs_bad_since).total_seconds() if self.inputs_bad_since else 0.0
if bad_for < stale_after:
# A single missed poll is not a fault. Hold, keep the heartbeat
@@ -1384,6 +1721,30 @@ class Controller:
"sign disagreement",
actuator=self.actuator_of("failsafe", inv)))
# --- SAFETY-30: THE SITE SIGN TRIP, AND IT IS A SITE SET ------------
# ⚠️ THE TWIN OF THE LOOP ABOVE, NOT A DUPLICATE OF IT. That loop files
# one `failsafe` per unit on the evidence of THAT unit's own measured
# power, and parks that unit. This files ONE claim on the SITE, because
# what tripped is the SUM of every standing command against the meter -
# a fact about the grid connection, whose offending number is the site
# target and which no single inverter owns. Parking one unit on it would
# leave the rest of the roster pushing the same way with the guard
# satisfied. See arbiter.CLAIM_SCOPE, which classifies `sign_site` and
# `failsafe` side by side and says why they differ.
#
# ⚠️ AND THE `n_units > 1` GUARD IS NOT REDUNDANT, EVEN THOUGH `cycle()`
# never arms this detector at a roster of one. The latch is RESTORED
# FROM THE STORE, so an install that ran with two inverters, tripped,
# and was then reconfigured back to one would come up with
# `site_sign_tripped: true` and would otherwise file a claim on evidence
# about a roster it no longer has. Guarded, that install comes up
# driving - which is right: the fault it latched on is not a statement
# about the site it is now.
if n_units > 1 and self.site_sign_wd.tripped:
claims.append(Claim.set("sign_site", P_SAFETY, 0.0,
"site sign disagreement",
actuator=self.actuator_of("sign_site")))
# --- SAFETY-31: WHAT "ACTED ON" MEANS, AND WHAT IT HONESTLY CANNOT --
# ⚠️ A SITE SET AT P_SAFETY: STOP COMMANDING, EVERYWHERE. Read the next
# paragraph before changing it, because the obvious objection is right
@@ -1624,88 +1985,114 @@ class Controller:
self.log_event(self.arb.explain())
# --- maintenance -----------------------------------------------------
if result.owns_setpoint:
# PER-INVERTER. §12's monthly recalibration is a PACK procedure -
# drain THIS pack to its floor, charge THIS pack full, recalibrate
# THIS pack's coulomb counter. Two packs recalibrate on two
# schedules. See arbiter.CLAIM_SCOPE.
# ⚠️ FILED ON THE UNIT BEING MAINTAINED, WHICH IS THE FIRST ONE.
# §12's monthly recalibration is a PACK procedure and this add-on
# holds exactly ONE `Maintenance` state machine, so the pack it is
# maintaining is `self.maint_unit`. Two packs recalibrating on two
# schedules is a second FSM with its own namespaced history, which
# is a card of its own - what this card must not do is file one
# pack's procedure on every actuator.
claims.append(Claim.set("maintenance", P_MAINTENANCE,
float(result.setpoint_w or 0.0), result.phase,
actuator=self.actuator_of("maintenance",
self.maint_unit)))
self.sat_count = 0
elif result.charge_only:
# Money outranks the maintenance schedule: while the quarter-hour
# projection is over the cap, the charge-only shaping is simply not
# claimed, so the loop can discharge and shave the peak. When a
# dedicated peak-shaving strategy arrives it will claim SET at
# P_PEAK and outrank this limit without any code here changing.
# ⚠️ AND SAFETY-07 IS THE OTHER HALF OF THIS CONDITION, NOT A
# SECOND MECHANISM. The shaping is `hi <= 0`; the peak guard's claim
# is `lo >= 0`. Both are limits, arbiter.py applies EVERY limit at or
# above the winner's priority, and most restrictive wins - so a peak
# event during a maintenance charge window files `lo > hi`, which is
# a CONTRADICTION and resolves to 0 W. The battery would park in the
# exact quarter it should be shaving. Priority does not save it:
# clause 2 has no notion of a higher-priority limit overriding a
# lower one, and it should not - "most restrictive" is what makes the
# safety bounds unconditional.
#
# So the higher-priority claimant SUPPRESSES the lower one rather
# than out-voting it, which is what this branch already did for
# `peak_forecast_entity`. That entity is optional and empty on a
# default install, so until now the suspension could not fire on the
# site that needed it. `self.peak_guard` is measured from the meter
# and is always there.
#
# ⚠️ THE CONJUNCTION LIVES IN peakguard.py, NOT HERE, and that is not
# tidiness. It was inline, and a reviewer mutated it back to its
# historical broken form - re-gated on `peak_forecast_entity` alone -
# and every suite in this repository stayed green while a default
# install parked at 0 W in the quarter that sets the month's bill.
# Read shaping_suspended() before touching this: both terms are
# there, and why neither is sufficient alone.
if peakguard.shaping_suspended(
self.peak_guard, self.peak_fc,
float(self.o.get("peak_cap_w", 3500))):
# Edge-logged: level-logging here would write one line per cycle
# for as long as the event lasts and flush the 40-line action log
# of the very context that explains it.
if not self._maint_shape_suspended:
self._maint_shape_suspended = True
self.log_event("peak at risk - maintenance charge shaping "
"suspended so the peak claim is not "
"contradicted")
else:
self._maint_shape_suspended = False
hi = 0.0
reason = "charge-only"
if self.cheap:
# ⚠️ CFG-03. `self.maint.cfg.charge_w`, NOT A SECOND READ
# OF `self.o`. This line used to re-open the options dict
# with its own literal fallback, so the rate the shaping
# asked for and the rate the state machine drained at could
# be different numbers from the same install - and, with the
# key absent, they were. The Controller already parsed this
# once in __init__; asking the object that owns it cannot
# drift from it.
floor = maintenance_charge_floor(
self.maint.cfg.charge_w,
self.peak_fc, float(self.o.get("peak_cap_w", 3500)))
if floor > 0:
hi, reason = -floor, "cheap-window charge"
# PER-INVERTER, like the SET above and for the same reason -
# the shaping is about the pack being charged.
claims.append(Claim.limit("maintenance", P_MAINT_SHAPE, hi=hi, reason=reason,
actuator=self.actuator_of("maintenance",
self.maint_unit)))
# ⚠️ ONE PASS PER PACK, AND `arbitrate()` IS WHY THE LIST STAYS SHORT.
# At most one machine is out of IDLE on any tick, so at most one of
# these branches fires and the claim list is the shape it has always
# been. The loop is not a widening of what maintenance may ask for; it
# is what makes the ONE claim it does file land on the pack that is
# actually being recalibrated, whichever roster slot that is.
for inv, result in zip(self.inverters, maint_results):
if result.owns_setpoint:
# PER-INVERTER. §12's monthly recalibration is a PACK procedure -
# drain THIS pack to its floor, charge THIS pack full,
# recalibrate THIS pack's coulomb counter. Two packs recalibrate
# on two schedules. See arbiter.CLAIM_SCOPE.
#
# ⚠️ FILED ON THE PACK WHOSE STATE MACHINE PRODUCED IT - `inv`,
# the roster entry this `result` was ticked from - and that is
# the whole of MAINT-05 at this line. It used to be
# `self.maint_unit`, which was `inverters[0]` flatly, because
# there was one FSM and its history had nowhere else to live.
# Pack B's drain filed on pack A is one pack being commanded for
# another's procedure while the pack that is actually draining
# stays under the loop: the site would move the right total and
# recalibrate nothing.
#
# ⚠️ AND IT STAYS `SCOPE_UNIT`. Two FSMs mean two claims on two
# actuators, which the allocator consumes as per-unit intervals;
# ARCH-01 is explicit that this must never become a site claim,
# because per-inverter constraints filed site-wide are the
# cross-inverter `lo > hi` contradiction at arbiter.py:114.
claims.append(Claim.set("maintenance", P_MAINTENANCE,
float(result.setpoint_w or 0.0), result.phase,
actuator=self.actuator_of("maintenance", inv)))
self.sat_count = 0
elif result.charge_only:
# Money outranks the maintenance schedule: while the quarter-hour
# projection is over the cap, the charge-only shaping is simply not
# claimed, so the loop can discharge and shave the peak. When a
# dedicated peak-shaving strategy arrives it will claim SET at
# P_PEAK and outrank this limit without any code here changing.
# ⚠️ AND SAFETY-07 IS THE OTHER HALF OF THIS CONDITION, NOT A
# SECOND MECHANISM. The shaping is `hi <= 0`; the peak guard's
# claim is `lo >= 0`. Both are limits, arbiter.py applies EVERY
# limit at or above the winner's priority, and most restrictive
# wins - so a peak event during a maintenance charge window files
# `lo > hi`, which is a CONTRADICTION and resolves to 0 W. The
# battery would park in the exact quarter it should be shaving.
# Priority does not save it: clause 2 has no notion of a
# higher-priority limit overriding a lower one, and it should not -
# "most restrictive" is what makes the safety bounds unconditional.
#
# So the higher-priority claimant SUPPRESSES the lower one rather
# than out-voting it, which is what this branch already did for
# `peak_forecast_entity`. That entity is optional and empty on a
# default install, so until now the suspension could not fire on the
# site that needed it. `self.peak_guard` is measured from the meter
# and is always there.
#
# ⚠️ THE CONJUNCTION LIVES IN peakguard.py, NOT HERE, and that is
# not tidiness. It was inline, and a reviewer mutated it back to its
# historical broken form - re-gated on `peak_forecast_entity` alone
# - and every suite in this repository stayed green while a default
# install parked at 0 W in the quarter that sets the month's bill.
# Read shaping_suspended() before touching this: both terms are
# there, and why neither is sufficient alone.
#
# ⚠️ ONE SUSPENSION FLAG FOR THE SITE, NOT ONE PER PACK, AND THE
# INTERLOCK IS WHAT MAKES THAT TRUE. The quantity it edge-logs is
# a peak event at the grid connection, and `arbitrate()` allows
# at most one pack to be in CHARGE at a time, so this branch can
# run at most once per tick. Were two packs ever allowed to
# charge together, this flag would have to become per-pack - and
# so would the reasoning above it.
if peakguard.shaping_suspended(
self.peak_guard, self.peak_fc,
float(self.o.get("peak_cap_w", 3500))):
# Edge-logged: level-logging here would write one line per
# cycle for as long as the event lasts and flush the 40-line
# action log of the very context that explains it.
if not self._maint_shape_suspended:
self._maint_shape_suspended = True
self.log_event("peak at risk - maintenance charge shaping "
"suspended so the peak claim is not "
"contradicted")
else:
self._maint_shape_suspended = False
hi = 0.0
reason = "charge-only"
if self.cheap:
# ⚠️ CFG-03. `self.maint.cfg.charge_w`, NOT A SECOND READ
# OF `self.o`. This line used to re-open the options dict
# with its own literal fallback, so the rate the shaping
# asked for and the rate the state machine drained at
# could be different numbers from the same install - and,
# with the key absent, they were. The Controller already
# parsed this once in __init__; asking the object that
# owns it cannot drift from it. Every pack's machine
# holds the same `MaintConfig` instance, so this is the
# rate the pack in front of us is charging at.
floor = maintenance_charge_floor(
self.maint.cfg.charge_w,
self.peak_fc, float(self.o.get("peak_cap_w", 3500)))
if floor > 0:
hi, reason = -floor, "cheap-window charge"
# PER-INVERTER, like the SET above and for the same reason -
# the shaping is about the pack being charged, which is `inv`.
claims.append(Claim.limit("maintenance", P_MAINT_SHAPE, hi=hi,
reason=reason,
actuator=self.actuator_of("maintenance",
inv)))
# --- the grid-following loop -----------------------------------------
if self.auto and self.inputs_ok:
@@ -2608,8 +2995,15 @@ class Controller:
# here would be a surface to migrate later. This sensor is already
# discovered, already on every dashboard, and "failsafe" is
# additive to its vocabulary rather than a change to it.
# SAFETY-30 adds the site sign latch to the same vocabulary, for
# the reason the note above gives: "failsafe" is additive to a
# sensor that already exists and is already on every dashboard, and
# a P_SAFETY stop that did not reach it would be a site at 0 W
# reporting "running".
"status": ("failsafe" if any(i.sign_wd.tripped
for i in self.inverters)
or (self.site_sign_wd.tripped
and len(self.inverters) > 1)
or self.circ_wd.tripped
else "running" if self.auto else "stopped"),
# SAFETY-31. ⚠️ MEASURED, not the allocation record - see
@@ -2936,24 +3330,70 @@ class Controller:
# and a trip on B leaves A commanding. A single row would have to pick
# one of them to describe, and the honest answer at a roster of two is
# both. At a roster of one the wording is unchanged.
# ⚠️ AND SINCE SAFETY-30 THE ROW SAYS WHAT THE UNIT WAS COMPARED
# AGAINST, because at a roster of two that is no longer the meter and a
# row that still said "the command opposed the meter" would send an
# operator to look at P1 for a fault that is visible only on one
# inverter's own power sensor. `sign_wd.against` is the same string the
# trip sentence uses, so the two cannot drift apart.
for inv in self.inverters:
tag = "" if len(self.inverters) == 1 else f" [{inv.name}]"
# ⚠️ ADDED ONLY WHEN THERE IS SOMETHING TO DISAMBIGUATE. At the
# shipped roster of one the two sentences below are the ones that
# have always been on the page, word for word - AC 8 is about what
# an operator sees as much as about what the detector computes, and
# "opposed a meter reading" would be a gratuitous change to a line
# somebody has learned to recognise. At N>=2 the noun is load-
# bearing: the row would otherwise send them to look at P1 for a
# fault that is visible only on one inverter's own power sensor.
vs = ("" if len(self.inverters) == 1
else f" against {inv.sign_wd.against}")
if inv.sign_wd.tripped:
what = ("the meter" if len(self.inverters) == 1
else inv.sign_wd.against)
out.append({"ok": False, "warn": False,
"text": f"SIGN-DISAGREEMENT FAILSAFE{tag} - "
+ inv.sign_wd.reason
+ ". The inverter is held at 0 W. Find out why the "
"command opposed the meter before starting control "
+ f". The inverter is held at 0 W. Find out why the "
f"command opposed {what} before starting control "
"again; starting clears this. Restarting the "
"add-on does NOT clear it."})
else:
held = inv.sign_wd.held_s(datetime.now(timezone.utc))
out.append({"ok": True, "warn": False,
"text": f"sign watchdog{tag}: armed (trips after "
"text": f"sign watchdog{tag}: armed{vs} (trips after "
f"{inv.sign_wd.cfg.hold_s:.0f} s opposed, both over "
f"{inv.sign_wd.cfg.min_w:.0f} W)"
+ (f" - opposed for {held:.0f} s now" if held else "")})
# --- SAFETY-30: the SITE row -----------------------------------------
# ⚠️ ONLY AT N>=2, and the absence at N=1 is the honest report rather
# than a missing row. With one inverter the site comparison and the row
# above are the same sentence about the same actuator; printing it twice
# would advertise two guards where there is one, which is how an
# operator learns to skim the list.
if len(self.inverters) > 1:
if self.site_sign_wd.tripped:
out.append({"ok": False, "warn": False,
"text": "SITE SIGN-DISAGREEMENT FAILSAFE - "
+ self.site_sign_wd.reason
+ ". EVERY inverter is held at 0 W. This is the "
"SUM of the commands against the meter, so it is "
"not one inverter's fault to find - look at the "
"loop, the integrator and whatever won the "
"arbitration. Starting control clears this; "
"restarting the add-on does NOT."})
else:
_sheld = self.site_sign_wd.held_s(datetime.now(timezone.utc))
out.append({"ok": True, "warn": False,
"text": f"site sign watchdog: armed against the sum of "
f"the commands (trips after "
f"{self.site_sign_wd.cfg.hold_s:.0f} s opposed, "
f"both over "
f"{self.site_sign_wd.cfg.min_w:.0f} W)"
+ (f" - opposed for {_sheld:.0f} s now"
if _sheld else "")})
# --- SAFETY-31 -----------------------------------------------------
# ⚠️ SITE-LEVEL, WHERE THE ROWS ABOVE ARE PER-INVERTER, and the row says
# so. There is one circulating figure for the site; naming an inverter
@@ -3010,10 +3450,28 @@ class Controller:
level, banner = "bad", ("CIRCULATING POWER — the inverters were "
"pushing against each other. Site held at "
"0 W.")
elif self.site_sign_wd.tripped and len(self.inverters) > 1:
# SAFETY-30's site half, ABOVE the per-unit rows below and below
# circulating power above, and the ordering is the same argument
# both times: the banner carries one sentence, so it carries the one
# that says the widest true thing. A site sign trip means EVERY
# inverter is parked and the fault is in what the site was asking
# for, not in one device - naming a unit here would send an operator
# to swap the wrong box.
level, banner = "bad", ("FAILSAFE — the site's total command "
"opposed the meter. Every inverter held at "
"0 W.")
elif any(i.sign_wd.tripped for i in self.inverters):
# Outranks "NOT READY": nothing is missing, something went wrong.
level, banner = "bad", ("FAILSAFE — the command opposed the meter. "
"Inverter held at 0 W.")
# ⚠️ THE SENTENCE IS UNCHANGED AT A ROSTER OF ONE, where "the meter"
# is what the unit was graded against. At N>=2 it is that unit's own
# power, so the banner says so - see checks() on why the noun is
# load-bearing rather than cosmetic.
level, banner = "bad", (
"FAILSAFE — the command opposed the meter. "
"Inverter held at 0 W." if len(self.inverters) == 1 else
"FAILSAFE — an inverter's command opposed its own measured "
"power. That inverter held at 0 W.")
elif bad:
# ⚠️ `banner` IF THE ROW OFFERS ONE, `text` OTHERWISE. A row may
# need to say more to an operator reading the checks list than fits
@@ -3025,14 +3483,29 @@ class Controller:
elif not self.auto:
level, banner = "warn", "Stopped — inverter commanded to 0 W"
elif self.maint.phase != IDLE:
# ⚠️ THE PACK IS NAMED ONLY WHEN THERE IS MORE THAN ONE. At a roster
# of one this is the string it has always been, character for
# character; at two, "Maintenance: drain" without a name is a
# banner an operator cannot act on, because the remedy is at one
# device and the site has two.
level, banner = "warn", f"Maintenance: {self.maint.phase}"
if len(self.inverters) > 1:
banner += f" ({self.maint_unit.name})"
else:
level, banner = "ok", "Running — holding grid at zero"
def w(v):
return "—" if v is None else f"{v:,.0f} W".replace(",", " ")
next_due = self.maint.next_due(datetime.now(timezone.utc))
# ⚠️ THE EARLIEST ACROSS THE PACKS, NOT THE FIRST PACK'S. One date on a
# site with two schedules has to be the one that comes first, or the row
# says "2026-10-01" while a pack recalibrates tomorrow. `None` means
# "never run, due now" and has to win over any date, so it is tested for
# rather than sorted with. At a roster of one this is that pack's own
# date, unchanged.
_now = datetime.now(timezone.utc)
_dues = [m.next_due(_now) for m in self.maints]
next_due = None if any(d is None for d in _dues) else min(_dues)
rows = [
("Grid", w(self.grid)),
("Battery", w(self.site_batt_w)),
@@ -3048,7 +3521,14 @@ class Controller:
else f"{self.pack_floor_soc:g} %"),
("Commanded", w(self.target)),
("Why", self.reason),
("Maintenance phase", self.maint.phase),
("Maintenance phase", self.maint.phase if len(self.inverters) <= 1
else f"{self.maint.phase} ({self.maint_unit.name})"),
# ⚠️ ONLY WHEN A PACK IS ACTUALLY WAITING, so a one-inverter status
# page - where `_maint_deferred` is empty on every tick - carries
# exactly the rows it carried before.
*((("Maintenance waiting", "; ".join(f"{n}: {why}"
for n, why in self._maint_deferred)),)
if self._maint_deferred else ()),
("Maintenance due", "now" if next_due is None else next_due.strftime("%Y-%m-%d")),
("Cheap window", "yes" if self.cheap else "no"),
("Tariff", self.tariff_words),
@@ -3118,6 +3598,24 @@ class Controller:
# unit in particular.
self.store.set("sign_tripped", False)
self.store.set("sign_reason", "")
# SAFETY-30's SITE latch, cleared by the SAME press and for the
# same reason the per-unit ones are: the condition always goes
# away on its own, because the trip commands 0 W everywhere and
# a site commanding zero cannot oppose anything. Only a human
# who has read the banner can say the site is clear.
#
# ⚠️ UNCONDITIONAL, LIKE THE FLAT PAIR ABOVE, AND NOT GUARDED ON
# THE ROSTER. An install that ran with two inverters, tripped,
# and was then reconfigured back to one would otherwise carry
# `site_sign_tripped: true` in its file forever, with no roster
# that can reach the code which clears it - and would re-arm the
# moment a second inverter was configured again.
if self.site_sign_wd.tripped:
self.log_event("site sign-disagreement failsafe cleared "
"by operator")
self.site_sign_wd.reset()
self.store.set("site_sign_tripped", False)
self.store.set("site_sign_reason", "")
# SAFETY-31, cleared by the SAME press and for the same reason:
# the condition always goes away on its own, because the trip
# commands 0 W everywhere and two inverters at zero cannot
@@ -3142,11 +3640,26 @@ class Controller:
await self.command_site_zero()
return {"auto": self.auto}
if what == "maint_start":
for msg in self.maint.force_start(now):
# ⚠️ WHICH PACK, BECAUSE THERE ARE NOW SEVERAL AND THE BUTTON HAS
# ONE LABEL. `_maint_button()` answers it: the pack that holds the
# site's maintenance slot if one does - which is what "start"
# already meant at a roster of one, where pressing it mid-drain
# re-entered DRAIN - and otherwise the pack that has gone longest
# without a cycle, which is the same pack `arbitrate()` would have
# picked on its own. At N=1 both branches are `maints[0]` and this
# is the shipped call, byte for byte.
#
# A PER-PACK BUTTON IS A UI CARD, NOT THIS ONE. `web.py` posts a
# bare `maint_start` with no unit, and inventing a unit parameter
# here would ship half a surface: the page would still have one
# button and the API would have two meanings.
for msg in self._maint_button(now).force_start(now):
self.log_event(msg)
return {"phase": self.maint.phase}
if what == "maint_abort":
for msg in self.maint.abort(now, "operator"):
# Aborts the pack that is actually cycling - `_maint_button()` again
# - and then zeroes the SITE, which is what it always did.
for msg in self._maint_button(now).abort(now, "operator"):
self.log_event(msg)
await self.command_site_zero()
return {"phase": self.maint.phase}
+309 -16
View File
@@ -18,16 +18,47 @@ OWNERSHIP, not priority. Exactly one thing writes the setpoint at any moment:
The loop is never "disabled" - it yields. The heartbeat to the inverter keeps
running in every phase, so the hardware watchdog stays fed and a crash here
still ends with the inverter at 0 W rather than latched.
ONE PACK AT A TIME (MAINT-05). There is now one `Maintenance` per roster entry,
each with its own schedule, its own phase and its own history under its own
Store keys - and therefore, for the first time, two of them able to move power
on ONE grid connection at once. Read `arbitrate()` before touching any of it:
the site holds exactly ONE maintenance slot, a pack that is not IDLE holds it,
and every other pack is held at IDLE under the loop until it is released. That
is not a scheduling nicety. A drain of pack A against a charge of pack B is two
power-moving phases pulling opposite ways across the meter - the circulating
power SAFETY-31 exists to DETECT, arriving here as something this module can
simply not do.
"""
import logging
import math
from dataclasses import dataclass, field
from datetime import datetime, timedelta, timezone
from .store import MAINT_UNIT_KEYS, unit_key
_LOG = logging.getLogger("goodwe.maint")
IDLE, DRAIN, CHARGE, HOLD = "idle", "drain", "charge", "hold"
# ⚠️ THE PHASES THAT MOVE POWER, WHICH IS NOT THE SAME LIST AS "owns the
# setpoint". The table above is about WHO WRITES the number; this is about
# whether the pack is being pushed. DRAIN and HOLD own the setpoint. CHARGE does
# NOT - the loop owns it - and CHARGE is in here anyway, because the charge-only
# clamp plus the cheap-window floor is what pulls a second pack full while the
# first is being emptied. Leaving CHARGE out is precisely the interlock hole
# this card is about: pack A charging from the grid at 2000 W while pack B
# drains 2000 W to export is 4 kW going round in a circle with the meter at
# zero, and both packs believing they are behaving.
#
# HOLD commands 0 W and moves nothing, and it is in here for the other half of
# the sentence: it OWNS the actuator - the pack is parked at the top for the BMS
# and is deliberately not available to the loop - so a second cycle starting
# under it would still be two recalibrations at once, which is the thing whose
# value is a clean uninterrupted coulomb count.
POWER_MOVING = (DRAIN, CHARGE, HOLD)
# A drain of a 13.8 kWh usable pack (300 Ah x 51.2 V, DoD 90) at the shipped
# 2000 W is ~6.9 h even with no house load helping - and less in practice, since
# house load drains alongside the export and soc_floor stops short of empty.
@@ -153,17 +184,102 @@ class MaintResult:
setpoint_w: float | None = None
charge_only: bool = False
events: list = field(default_factory=list)
# ⚠️ WHY A DEFERRAL IS A FIELD AND NOT JUST A LOG LINE. A pack that is due
# and is not starting looks, from every surface an operator has, exactly
# like a pack whose schedule has quietly stopped firing - §12's opening
# sentence, the failure this whole module exists because of. It is carried
# so the status page can SAY "waiting for the other pack" rather than
# showing an idle machine and a due date in the past. Empty on every path
# that is not deferring, so a roster of one never sees it.
deferred: str = ""
class UnitHistory:
"""One pack's maintenance history in the Store, under one spelling.
⚠️ THE NAMESPACE IS A CONSTRUCTOR ARGUMENT AND `None` IS A REAL VALUE, NOT
AN OMISSION. `unit=None` means the FLAT keys - `phase`, `phase_started`,
`last_completed`, `last_start_attempt` - which is what every install has
written since the add-on shipped and what a one-inverter site keeps writing
after MAINT-05. §6.2's identity: with one pack there is nothing for a
namespace to distinguish it FROM, and adding one would change the layout of
a live state file to say something it already said. `arbiter.actuator_for()`
makes the same move for the same reason and its docstring argues it.
⚠️ `mirror_flat` IS THE ROLLBACK VIEW AND IT IS ONLY EVER THE FIRST PACK'S.
At N>=2 pack A's history has been migrated into its namespace and is written
there; the flat keys are ALSO kept in step, so an operator who rolls the
add-on back to a single-FSM version finds a CURRENT history rather than one
dated the day the second inverter was declared. store.migrate_maint_keys()
carries the argument, and it is DEV-02's rule 2 applied to history instead
of to a latch: "left in the file" is not the same promise as "still true".
Pack B has no flat spelling to mirror to - the flat keys mean pack A on
every install that has them - so it is constructed with mirroring off, and
a mirror from B would OVERWRITE A's rollback view with B's phase.
"""
def __init__(self, store, unit: str | None = None, mirror_flat: bool = False):
self.store = store
self.unit = unit
self.mirror_flat = bool(mirror_flat)
def key(self, name: str) -> str:
"""The Store key this pack's `name` lives under."""
return name if self.unit is None else unit_key(self.unit, name)
def get(self, name: str, default=None):
return getattr(self.store, "data", {}).get(self.key(name), default)
def set(self, name: str, value) -> None:
self.store.set(self.key(name), value)
if self.mirror_flat:
self.store.set(name, value)
def get_time(self, name: str):
return self.store.get_time(self.key(name))
def set_time(self, name: str, when) -> None:
self.store.set_time(self.key(name), when)
if self.mirror_flat:
self.store.set_time(name, when)
class Maintenance:
def __init__(self, cfg: MaintConfig, store):
"""ONE pack's recalibration cycle: its schedule, its phase, its history.
⚠️ ONE PER ROSTER ENTRY SINCE MAINT-05, AND THE SLOT IS NOT ITS OWN. This
object decides WHEN ITS PACK would like to start; `arbitrate()` decides
which of them may, because the constraint is a property of the grid
connection and no single FSM can see it. Never call `tick()` in a loop
without passing the `blocked_by` that `arbitrate()` returned for it: that is
the two-packs-at-once bug with the interlock present but not wired.
"""
def __init__(self, cfg: MaintConfig, store, unit: str | None = None,
name: str = "", mirror_flat: bool = False):
self.cfg = cfg
self.store = store
# ⚠️ `self.store` IS THE VIEW, NOT THE Store. Everything in this class
# names an UNQUALIFIED key - "phase", "last_completed" - and the view is
# the one place that turns it into the key this pack owns. A raw
# `store.set("phase", ...)` anywhere below would write pack A's history
# from pack B's state machine, which is the shared-state bug this card
# is about, and there would be nothing to see at a roster of one.
self.store = UnitHistory(store, unit, mirror_flat)
self.unit = unit
# What an event or a refusal calls this pack. At a roster of one it is
# never rendered - a single machine cannot be deferred - so the shipped
# one-inverter reason strings are untouched.
self.name = str(name or unit or "the pack")
# Edge-memo for the deferral note. The action log holds 40 lines and a
# deferral lasts as long as the other pack's cycle - hours - so a line
# per cycle would flush the very context that explains it. Same pattern
# as `Controller._arb_action` and `_maint_shape_suspended`.
self._deferred = ""
# -- state ---------------------------------------------------------------
@property
def phase(self) -> str:
return self.store.data.get("phase", IDLE)
return self.store.get("phase", IDLE)
def _enter(self, phase: str, now: datetime, events: list, msg: str) -> None:
self.store.set("phase", phase)
@@ -185,14 +301,60 @@ class Maintenance:
last = self.store.get_time("last_completed")
return None if last is None else last + timedelta(days=self.cfg.interval_days)
def waited_days(self, now: datetime) -> float:
"""Days since this pack last COMPLETED a cycle; `inf` if it never has.
⚠️ THE ARBITRATION KEY, AND IT IS DELIBERATELY NOT `due()`. `due()` is a
yes/no and two packs that are both due are indistinguishable by it -
which is how a roster-order tie-break gets written and how the first
entry then wins every contested month forever while the second drifts.
This is an ORDERING: the pack whose coulomb counter has gone longest
without a recalibration goes first. `inf` for never-recalibrated is the
same statement `due()` makes about an absent `last_completed`, kept as
one so a fresh pack cannot lose to one that ran last month.
"""
last = self.store.get_time("last_completed")
return math.inf if last is None else (now - last).total_seconds() / 86400.0
def _cooled(self, now: datetime) -> bool:
"""Is the post-abort cooldown over?"""
last_try = self.store.get_time("last_start_attempt")
return (last_try is None
or (now - last_try) >= timedelta(minutes=RESTART_COOLDOWN_MIN))
def wants_start(self, now: datetime) -> bool:
"""Would this pack begin a cycle on this tick if the site let it?
⚠️ THE SAME PREDICATE `tick()` USES, NOT A COPY OF IT. `arbitrate()` has
to know who is asking before anybody moves, and a second transcription
of "enabled, in the start hour, due and cooled down" is a second opinion
about who was asking - so the IDLE branch of `tick()` calls this one.
"""
return (self.phase == IDLE and self.cfg.enabled
and now.hour == self.cfg.start_hour and self.due(now)
and self._cooled(now))
# -- operations ----------------------------------------------------------
def force_start(self, now: datetime) -> list:
def force_start(self, now: datetime, blocked_by: str = "") -> list:
"""Start a cycle regardless of the calendar.
Commissioning uses this: a cycle must be proven end to end before a site
is signed off, and nobody waits a month to find out the schedule never
fires.
⚠️ REGARDLESS OF THE CALENDAR, NOT REGARDLESS OF THE INTERLOCK. The
calendar is a preference and an operator may overrule it; "one pack
moves power at a time" is a property of the grid connection and an
operator pressing a button does not add a second one. A forced start
against a busy site is REFUSED, by name, and the phase machine does not
move - the refusal is returned as an event so the operator reads it in
the action log rather than watching nothing happen.
"""
if blocked_by:
msg = (f"maintenance: forced start REFUSED on {self.name} - "
f"{blocked_by}")
_LOG.warning("%s", msg)
return [msg]
events = []
self.store.set_time("last_start_attempt", now)
self._enter(DRAIN, now, events, "maintenance: forced start -> drain")
@@ -203,23 +365,80 @@ class Maintenance:
self._enter(IDLE, now, events, f"maintenance: ABORT ({why})")
return events
def _defer_note(self, why: str, events: list) -> list:
"""Say once, not every second, that this pack is waiting its turn.
⚠️ THE WORDING IS DELIBERATELY NOT AN ABORT AND NOT A TIMEOUT. §12's
notes are explicit that a 12 h phase timeout means "the battery is not
following commands - a FAULT, not a slow day", and the one thing that
must not happen here is an operator reading a deferral as that. This
line says the pack is WAITING, names the pack it is waiting for, and
says it will run next - none of which any failure path in this module
says.
"""
if self._deferred != why:
self._deferred = why
msg = (f"maintenance: {self.name} is due and is WAITING its turn - "
f"{why}. It is idle under the control loop and will "
f"recalibrate at the next start hour once the slot is free; "
f"this is not a stalled phase.")
_LOG.info("%s", msg)
events.append(msg)
return events
# -- the machine ---------------------------------------------------------
def tick(self, now: datetime, soc: float | None) -> MaintResult:
def tick(self, now: datetime, soc: float | None,
blocked_by: str = "") -> MaintResult:
"""One cycle of this pack's state machine.
`blocked_by` is `arbitrate()`'s answer for THIS pack: empty when the
site's one maintenance slot is available to it, otherwise the reason it
is not. It gates two things and nothing else - a start, and a phase that
is already moving power on a pack that does not hold the slot.
"""
events: list = []
phase = self.phase
# ⚠️ THE SLOT IS CHECKED BEFORE THE PHASE, AND THIS BRANCH IS A RECOVERY
# RATHER THAN AN EXPECTED PATH. `arbitrate()` grants the slot to the
# pack that is already mid-cycle, so a pack that is BOTH moving power
# and blocked means two histories on disk say "drain" - a hand-edited
# state file, or a store carried across a version that had no interlock.
# The fail-safe direction is unambiguous: two packs pulling against each
# other on one connection is the fault SAFETY-31 latches the whole site
# for. Stand this one down to 0 W and say why; the holder keeps its
# cycle, which is the outcome that loses no coulomb count at all.
if blocked_by and phase in POWER_MOVING:
events += self.abort(now, f"interlock - {blocked_by}")
return MaintResult(IDLE, True, 0.0, False, events)
if phase == IDLE:
if self.cfg.enabled and now.hour == self.cfg.start_hour and self.due(now):
last_try = self.store.get_time("last_start_attempt")
cooled = (
last_try is None
or (now - last_try) >= timedelta(minutes=RESTART_COOLDOWN_MIN)
)
if cooled:
self.store.set_time("last_start_attempt", now)
self._enter(DRAIN, now, events,
"maintenance: due -> drain (exports the surplus)")
return MaintResult(DRAIN, True, self.cfg.discharge_w, False, events)
if self.wants_start(now):
if blocked_by:
# ⚠️ `last_start_attempt` IS NOT STAMPED HERE, AND THAT IS
# THE STARVATION FIX RATHER THAN AN OMISSION. That key is
# the post-ABORT cooldown: it exists so a cycle that failed
# does not re-trigger inside the same start hour. A deferral
# is not an attempt - nothing was tried and nothing failed -
# and stamping it would put this pack under a 90 minute
# cooldown for a start hour that is 60 minutes long, so the
# pack that lost the arbitration would still be locked out
# at the moment the winner released the slot.
#
# ⚠️ AND THE PHASE DOES NOT MOVE, WHICH IS THE OTHER HALF.
# `phase_started` is untouched, so the 12 h DRAIN and 36 h
# CHARGE timeouts - which mean "the battery is not following
# commands, a fault" - cannot be reached by waiting. A
# deferred pack is IDLE under the loop, not a stuck one.
return MaintResult(IDLE, False, None, False,
self._defer_note(blocked_by, events),
blocked_by)
self._deferred = ""
self.store.set_time("last_start_attempt", now)
self._enter(DRAIN, now, events,
"maintenance: due -> drain (exports the surplus)")
return MaintResult(DRAIN, True, self.cfg.discharge_w, False, events)
self._deferred = ""
return MaintResult(IDLE, False, None, False, events)
# ⚠️ Fail toward inaction. Without a state-of-charge reading none of the
@@ -273,3 +492,77 @@ class Maintenance:
_LOG.error("maintenance: unknown phase %r, forcing idle", phase)
events += self.abort(now, f"unknown phase {phase!r}")
return MaintResult(IDLE, True, 0.0, False, events)
def _days(n: float) -> str:
return "never recalibrated" if n == math.inf else f"{n:.0f} d since its last cycle"
def arbitrate(machines, now: datetime) -> list:
"""Grant the site's ONE maintenance slot. Returns a list parallel to
`machines`: "" for the pack that may move power, otherwise the reason the
pack may not.
⚠️ THE SLOT IS A PROPERTY OF THE GRID CONNECTION, WHICH IS WHY IT IS NOT ON
`Maintenance`. Each FSM can see its own pack's schedule and nothing else; the
quantity being rationed - "how many phases are moving power across one
meter" - is a sum over the roster, exactly like `sum(|B_i|) - |sum(B_i)|` in
SAFETY-31 and for the same reason. A per-pack answer to a site-level
question is the shape of defect this repository keeps finding.
⚠️ IT IS DERIVED, NOT PERSISTED, AND THAT IS DELIBERATE. The holder is
"whichever pack's stored phase is not IDLE" - a fact already on disk, in the
per-pack history the packs themselves write. A separate `maint_slot_holder`
key would be a second opinion about the same thing, and the two would
disagree the first time the add-on was killed between writing the phase and
writing the holder. Restart-safe by construction: a controller that comes
back up mid-drain reads the drain and re-derives the holder from it.
TWO CLAUSES, IN ORDER:
1. A PACK ALREADY MOVING POWER KEEPS THE SLOT. Nothing preempts a cycle in
flight, because a recalibration interrupted halfway is worth less than
nothing - the coulomb counter is graded on ONE uninterrupted drain and
refill, so a cycle stopped at 40 % has spent the energy and bought no
calibration. If more than one pack somehow comes up moving power, the
FIRST in roster order holds and `tick()` stands the others down; see the
recovery branch there.
2. OTHERWISE THE LONGEST-WAITING CANDIDATE STARTS. `wants_start()` is asked
of every pack, and among those that say yes the slot goes to the one with
the largest `waited_days()`. NOT roster order: two packs on one monthly
schedule are due in the same hour every month, and a roster-order tie
would hand the slot to `inverters[0]` every single time while the second
pack's coulomb counter drifted forever - which is, in a different
costume, the exact defect this card was opened for. Roster order breaks a
genuine numeric tie and nothing else.
⚠️ AT A ROSTER OF ONE THIS IS THE IDENTITY FUNCTION. One machine is never
another machine's blocker: clause 1 finds at most itself and clause 2 has at
most one candidate, so it returns `[""]` on every tick and no reason string
it can build is ever rendered. That is what makes the one-inverter behaviour
byte-identical rather than merely similar.
"""
blocks = [""] * len(machines)
busy = [(i, m) for i, m in enumerate(machines) if m.phase in POWER_MOVING]
if busy:
held, holder = busy[0]
for i, m in enumerate(machines):
if i != held:
blocks[i] = (f"{holder.name} is mid-cycle ({holder.phase}) and "
f"holds the site's one maintenance slot")
return blocks
wants = [(i, m) for i, m in enumerate(machines) if m.wants_start(now)]
if len(wants) <= 1:
return blocks
# Largest waited_days wins; `-index` makes a numeric tie fall to roster
# order rather than to whatever `max()` happens to see first.
won, winner = max(wants, key=lambda p: (p[1].waited_days(now), -p[0]))
for i, m in wants:
if i != won:
blocks[i] = (f"{winner.name} has waited longer "
f"({_days(winner.waited_days(now))} against "
f"{_days(m.waited_days(now))}) and takes the site's "
f"one maintenance slot first")
return blocks
+301 -26
View File
@@ -71,6 +71,24 @@ _LOG = logging.getLogger("goodwe.store")
# the card is gone while the fault runs on - exactly how the 2026-09-02 outage
# went unseen for 6 h 25 m with the detector correct throughout. main.py
# re-raises the notification from this key on every start.
#
# ⚠️ AND SAFETY-30's PAIR IS FLAT FOR THE SAME REASON SAFETY-31's IS, WHICH IS
# NOT THE SAME REASON THE ONES ABOVE ARE NAMESPACED. `site_sign_tripped` /
# `site_sign_reason` hold the SITE sign-disagreement latch: the SUM of every
# standing command opposed the meter, continuously. That is a statement about
# the grid connection - the offending number is the site target, which no single
# inverter owns - so there is no unit to hang it on, and hanging it on one would
# let an operator clear "inverter B's half" of a fault that has no halves.
# `unit/<name>/sign_tripped` above stays what it always was and now means what
# it says: THAT unit's own command against THAT unit's own measured power.
#
# ⚠️ AND THEY ARE NOT IN `LEGACY_UNIT_KEYS`, MUST NEVER BE, AND ARE NOT WRITTEN
# AT A ROSTER OF ONE AT ALL. At N=1 the site comparison and the unit comparison
# are the same sentence and `Controller.cycle()` arms only the per-unit
# detector, so a one-inverter install's state file is byte-identical to what it
# was before this card - these two keys sit at their defaults and nothing reads
# them. A rollback therefore finds exactly the flat `sign_tripped` mirror it
# always found.
DEFAULTS = {
"phase": "idle",
"phase_started": None,
@@ -80,6 +98,8 @@ DEFAULTS = {
"sign_reason": "",
"circ_tripped": False,
"circ_reason": "",
"site_sign_tripped": False,
"site_sign_reason": "",
}
# --------------------------------------------------------------------------- #
@@ -106,16 +126,53 @@ SCHEMA_VERSION = 1
SCHEMA_KEY = "schema_version"
# Which flat keys belong to ONE inverter, and therefore move into its namespace.
# ⚠️ THE MAINTENANCE KEYS ARE NOT IN HERE AND THAT IS DELIBERATE. `phase`,
# `phase_started`, `last_completed` and `last_start_attempt` belong to the
# maintenance state machine, of which this add-on still holds exactly ONE - a
# monthly recalibration is a pack procedure and per-pack SCHEDULING is a card of
# its own, not something to infer from a key rename. Moving them here would
# quietly re-home an install's whole maintenance history under a unit name; they
# stay where they are, untouched, which is the other half of "migrates without
# losing its latch OR ITS HISTORY".
# ⚠️ THE MAINTENANCE KEYS ARE STILL NOT IN HERE, AND SINCE MAINT-05 THAT IS A
# DIFFERENT DECISION RATHER THAN THE SAME ONE. They DO move now - one
# `Maintenance` per roster entry needs one history per roster entry - but they
# move on their OWN marker, `MAINT_SCHEMA_KEY`, and only on a site that actually
# has more than one pack. Two conditions the SAFETY-02 keys do not share, so
# folding them into this tuple would either drag them across on a
# single-inverter install (which is the byte-identical store layout MAINT-05
# must not change) or hold the sign latch hostage to a roster count it has
# nothing to do with. `MAINT_UNIT_KEYS` below carries them.
LEGACY_UNIT_KEYS = ("sign_tripped", "sign_reason")
# --------------------------------------------------------------------------- #
# MAINT-05: THE SECOND MIGRATION, AND WHY IT IS A SECOND ONE
#
# 0 = the flat, one-history layout every install has had since the add-on
# shipped: `phase` / `phase_started` / `last_completed` /
# `last_start_attempt` describe THE pack, because there was one FSM.
# 1 = one history per roster entry, under `unit/<name>/...`.
#
# ⚠️ ITS OWN MARKER, NOT A BUMP OF `SCHEMA_VERSION`, AND THE REASON IS THAT THE
# TWO MIGRATIONS DO NOT HAVE THE SAME PRECONDITION. DEV-02's runs on every
# install exactly once and is monotone in the version number. This one runs ONLY
# WHERE THE ROSTER HAS MORE THAN ONE PACK, because at N=1 §6.2's identity holds
# and the flat keys ARE that pack's namespace - moving them would add four keys
# and a version marker to a store MAINT-05's own acceptance criterion requires
# to be byte-identical. A single monotone counter cannot express "migrated on
# this site, deliberately not migrated on that one" without lying to the next
# version that reads it: a one-inverter store would have to be left at the old
# number forever, and then a genuine v2 migration could never be told apart from
# a site that simply never grew.
#
# ⚠️ AND IT IS TWO MARKERS, NOT TWO COPIES OF THE THREE RULES.
# `_move_flat_keys()` and `_verify_moves()` below are shared verbatim by both
# migrations, so rules 1 and 3 and the READ-BACK POSTCONDITION are one
# implementation. A second transcription of a migration is a second opinion
# about what "it landed" means.
MAINT_SCHEMA_VERSION = 1
MAINT_SCHEMA_KEY = "maint_schema_version"
# The flat maintenance-history keys, in the order they are moved.
# ⚠️ THE VALUES ARE THE FLAT SPELLINGS AND THE MODULE THAT READS THEM INDEXES
# THIS SAME TUPLE (`maintenance.UnitHistory`), so a key renamed here cannot
# silently stop being migrated - the same property `UNIT_STORE_KEYS` gives the
# SAFETY-02 pair.
MAINT_UNIT_KEYS = ("phase", "phase_started", "last_completed",
"last_start_attempt")
def unit_key(unit: str, key: str) -> str:
"""The namespaced Store key for `key` on inverter `unit`.
@@ -154,16 +211,35 @@ class MigrationReport:
# to shout: an alarm that fires on the harmless case is an alarm an
# operator learns to dismiss before the real one arrives.
self.lost = []
# ⚠️ A DIFFERENT FAILURE FROM `lost`, AND IT HAS TO BE, BECAUSE THE
# READ-BACK CANNOT SEE IT. `lost` is "the store took the write and
# dropped it". This is "the store took the write, kept it perfectly, and
# NOTHING WILL EVER READ IT" - the migration was handed a unit name that
# is not on the roster, so every machine looks under a different key and
# finds nothing. There is no harmless-default version of this the way
# there is for `lost`: a write nobody reads is not information, whatever
# its value, so anything in here fails `ok` on its own.
self.misdirected = []
# Why nothing ran, when nothing ran. Empty means "already at schema".
self.skipped = ""
@property
def ok(self) -> bool:
"""Nothing that CARRIED STATE failed to land."""
return not self.lost
"""Nothing that CARRIED STATE failed to land, and it landed WHERE THE
MACHINES LOOK."""
return not self.lost and not self.misdirected
def explain(self) -> str:
if not self.ran:
return (f"store already at schema {self.from_version}; no "
f"per-inverter migration needed")
# ⚠️ `skipped` RATHER THAN A SECOND SENTENCE ABOUT VERSIONS. Since
# MAINT-05 there are two reasons a migration does not run - the
# store is already at this schema, or the roster has one pack and
# the flat keys ARE its namespace - and an operator reading the
# startup log needs to be told which, because only one of them
# changes when a second inverter is declared.
return self.skipped or (f"store already at schema "
f"{self.from_version}; no "
f"per-inverter migration needed")
out = (f"migrated {len(self.moved)} flat key(s) to inverter "
f"namespaces: " + ", ".join(sorted(self.moved)))
if self.kept:
@@ -172,6 +248,8 @@ class MigrationReport:
out += " - PROBLEMS: " + "; ".join(self.problems)
if self.lost:
out += " - STATE LOST: " + "; ".join(self.lost)
if self.misdirected:
out += " - WRONG NAMESPACE: " + "; ".join(self.misdirected)
return out
@@ -223,7 +301,30 @@ def migrate_unit_keys(store, unit: str) -> MigrationReport:
return rep
rep.ran = True
for legacy in LEGACY_UNIT_KEYS:
_move_flat_keys(store, unit, LEGACY_UNIT_KEYS, rep)
_verify_moves(store, rep)
if rep.ok:
store.set(SCHEMA_KEY, SCHEMA_VERSION)
else:
# ⚠️ NOT MARKED DONE. A migration that lost state must run again on the
# next start rather than be recorded as finished - the one thing worse
# than a failed migration is a failed migration a later version skips.
_LOG.error("per-inverter store migration LOST STATE and has not been "
"marked done: %s", rep.explain())
return rep
def _move_flat_keys(store, unit: str, keys, rep: MigrationReport) -> None:
"""Rule 1, once, for both migrations.
Copies each flat `key` to `unit/<unit>/<key>` UNLESS the namespaced spelling
is already present - a store that has both has been written by a version
that knew about the namespace, and that value is the live one. Rule 2 is the
absence of a delete: the flat keys are left exactly where they were.
"""
data = getattr(store, "data", {})
for legacy in keys:
target = unit_key(unit, legacy)
if target in data:
rep.kept.append(target)
@@ -232,17 +333,63 @@ def migrate_unit_keys(store, unit: str) -> MigrationReport:
# tripped has no `sign_tripped` in its file at all, and the value it
# then lands on has to be the one `Store.load()` would have supplied -
# otherwise "absent" and "never tripped" stop being the same statement,
# which is the distinction store.py's own header is about.
# which is the distinction store.py's own header is about. The same
# sentence holds for `last_completed`: absent means "never recalibrated"
# and a literal here would be a second opinion about that.
value = data.get(legacy, DEFAULTS.get(legacy))
store.set(target, value)
rep.moved[target] = value
# THE POSTCONDITION, AND IT IS A READ-BACK RATHER THAN AN ASSUMPTION. This
# is the difference between this function and a two-line copy: the caller
# can ask "is the latch that was in the file before now under the key the
# watchdog reads", through the same `unit_key()` the watchdog's constructor
# reads through, and a `False` there is a guard that is off on an install
# whose state file says it is on.
def _verify_moves(store, rep: MigrationReport, roster=None) -> None:
"""THE POSTCONDITION, AND IT IS A READ-BACK RATHER THAN AN ASSUMPTION.
This is the difference between these functions and a two-line copy: the
caller can ask "is the latch that was in the file before now under the key
the watchdog reads" - and, since MAINT-05, "is the history that was in the
file before now under the key THIS PACK's state machine reads" - through the
same `unit_key()` those readers read through. A `False` here is a guard that
is off, or a month of coulomb-count history gone, on an install whose state
file says otherwise.
⚠️ AND `roster` IS THE OTHER HALF, WHICH THE READ-BACK ON ITS OWN CANNOT BE.
A POSTCONDITION THAT READS BACK THE NAMESPACE IT WAS HANDED CANNOT DETECT
BEING HANDED THE WRONG ONE. That is not a hypothetical: MAINT-05 shipped
with `migrate_maint_keys()` taking a bare `unit` string, and a call site that
passed `inverter.UNIT_A` instead of the first roster entry's name migrated an
install's whole maintenance history into `unit/inverter-a/...` on a roster of
`["garage", "shed"]`, read all four keys back under that same name, was
SATISFIED, and stamped the schema marker - so the next start skipped the
migration too and both packs came up believing they had never been
recalibrated. Every symptom of a clean migration, and the history gone.
`build_roster()` names a declared roster's entries from the config and falls
back to a POSITIONAL `inverter-N`, so `inverter-a` is not even the default
there; there is no roster on which that write could have been read.
So the namespace is graded against the one the MACHINES read - the roster -
rather than against the argument this function was handed, which is the
thing under test. A unit that is not on the roster is `misdirected`, which
fails `ok`, which means the caller does NOT stamp the version marker and the
next start runs the migration again. A re-run is recoverable, by rule 1; a
silent skip is the state store.py's header says must never happen.
⚠️ AND THE GRADED SET IS `moved` UNION `kept`, WHICH IS WHAT MAKES THAT
PARAGRAPH TRUE ON THE SECOND START AS WELL AS THE FIRST. Graded over `moved`
alone the refusal lasted exactly one start: the wrong call site is still
wrong next time, finds the keys it wrote already present, and rule 1 routes
every one of them to `kept` - so `moved` is empty, this loop sees nothing,
`ok` comes back True and the marker is stamped on a migration that has
reached no readable key. The measurement is in the comment on the loop
itself. `test_maintenance.py` section 9c calls this twice with the SAME
wrong `unit` over one store, which is the shape nothing in the suite had.
`roster=None` means "not graded", and the one caller that passes it is
`migrate_unit_keys()`, which runs BEFORE `build_roster()` for the reason its
own docstring gives - the `Inverter` objects restore their latch in their
constructors, so the keys have to be in place before there is a roster to
check against. That gap is real and is recorded as a running check rather
than left to a comment: see test_maintenance.py section 13.
"""
for target, value in rep.moved.items():
got = getattr(store, "data", {}).get(target, "<<missing>>")
if got == value:
@@ -254,14 +401,142 @@ def migrate_unit_keys(store, unit: str) -> MigrationReport:
if value != DEFAULTS.get(legacy):
rep.lost.append(note)
if roster is None:
return
names = [str(n) for n in roster]
# `unit_key()` builds `unit/<unit>/<key>`, so the unit is field 1. Split
# through the same spelling the writer used rather than re-deriving it.
#
# ⚠️ `moved` UNION `kept`, AND THE UNION IS THE WHOLE OF THE SECOND FIX
# ROUND. Grading only `moved` made the refusal last EXACTLY ONE START.
# Measured at Controller level over one persisted store, roster
# ["garage", "shed"], flat `last_completed` 2026-08-07, with the call site
# still handing `inverter-a` on both starts:
#
# construction 1: ok=False misdirected=4 moved=4 kept=0 marker=<absent>
# construction 2: ok=True misdirected=0 moved=0 kept=4 marker=1
#
# On start 2 the four keys the WRONG start 1 wrote are already present, so
# rule 1 routes every one of them to `rep.kept`, `rep.moved` is EMPTY, this
# loop iterates over nothing, `misdirected` is empty and `ok` is True - so
# the caller stamps the marker and start 3 takes the `from_version >=
# SCHEMA` early return FOREVER, including after the call site is fixed.
# Loud-and-recoverable became silent-and-permanent after one restart: the
# exact state this module's header says must never happen, reached by the
# fix that was written to prevent it.
#
# The union is sound by construction rather than by hope: every entry in
# both `rep.moved` and `rep.kept` is a `unit_key(unit, legacy)` built from
# the SAME `unit` argument, so grading `kept` grades the same namespace
# decision `moved` does. A correct call site puts its `kept` entries under a
# roster name and they pass; a wrong one cannot launder its namespace by
# having already written it.
for target in sorted(set(rep.moved) | set(rep.kept)):
parts = target.split("/")
unit = parts[1] if len(parts) > 2 else ""
if unit in names:
continue
rep.verified = False
note = (f"{target} was migrated into namespace {unit!r}, which is NOT "
f"on the roster {names!r} - nothing reads that key")
rep.problems.append(note)
rep.misdirected.append(note)
def migrate_maint_keys(store, unit: str, roster) -> MigrationReport:
"""Move the flat maintenance history into the FIRST pack's namespace.
MAINT-05. Until this card there was one `Maintenance` and one history, and
`store.py`'s own header argued the keys should stay flat because per-pack
scheduling was a card of its own. This is that card: there is now one FSM
per roster entry, so there is one history per roster entry, and the flat one
belongs to the pack that was being recalibrated - `inverters[0]`, which is
what `Controller.maint_unit` named and what `validate.py` rule 6 graded
against.
⚠️ `len(roster) <= 1` IS A RETURN, NOT A SPECIAL CASE. At one inverter the
flat keys ARE that pack's namespace (§6.2's identity, the same one
`arbiter.actuator_for()` leans on), `Maintenance` is constructed to read
them directly, and a migration would add four keys plus a marker to a store
that must come out byte-identical. The migration happens on the day the
roster grows, which is the day the flat spelling stops being able to answer
"which pack".
⚠️ `roster` IS REQUIRED, AND IT REPLACED AN `n_units: int`. Two reasons, and
the second is the one MAINT-05 was rejected over.
The small one: `n_units` was `len(self.inverters)` at the only call site, so
the roster carries it - `len(roster)` - and a count passed beside the names
is a second opinion about how many packs there are.
The load-bearing one: THE NAMES ARE THE POSTCONDITION. `unit` says which
pack the flat keys belonged to and that is genuinely a decision - the first
roster entry, because that is the pack the one FSM was driving - but a
decision the caller can get WRONG, and until this signature existed nothing
could tell. `_verify_moves()` read back the namespace it was handed, so a
migration into a name no machine reads verified clean and stamped the
marker. Handing the roster in means the read-back is graded against the
keys the MACHINES read, and a `unit` that is not among these names comes
back `misdirected`: not `ok`, marker NOT written, migration runs again next
start - and again the start after that, because `_verify_moves()` grades
`kept` as well as `moved`. See `_verify_moves()` for the install this was
measured on, and for what grading `moved` alone cost on start 2.
THE THREE RULES ARE `migrate_unit_keys()`'s, SHARED IN CODE rather than
restated: `_move_flat_keys()` is rule 1 (never overwrite a namespaced value)
and rule 2 (the flat keys are left in the file), `_verify_moves()` is the
read-back postcondition, and the marker is written LAST so a crash before it
leaves the migration to run again - which rule 1 makes a no-op.
⚠️ AND RULE 2 IS KEPT LIVE HERE TOO, WHICH IS THE HALF DEV-02 HAD TO ADD
LATER. "Left in place" alone makes the flat history a fossil dated the day
the second inverter was declared: an operator who rolls back to a
single-FSM add-on would find a `last_completed` months old and recalibrate a
pack that does not need it, or - worse - a stale `phase` and a
`phase_started` old enough to abort on the 12 h timeout the moment it comes
up. So `maintenance.UnitHistory` MIRRORS the first pack's writes back onto
the flat keys for as long as the roster has more than one entry. The flat
pair is the ROLLBACK VIEW of the history, maintained, exactly as
`sign_tripped` is the rollback view of the latch.
"""
rep = MigrationReport()
data = getattr(store, "data", {})
names = [str(n) for n in roster]
rep.from_version = int(data.get(MAINT_SCHEMA_KEY, 0) or 0)
if len(names) <= 1:
rep.skipped = ("one inverter: the flat maintenance keys ARE that pack's "
"history and are read in place, so there is nothing to "
"move and the store layout does not change")
return rep
if rep.from_version >= MAINT_SCHEMA_VERSION:
rep.skipped = (f"maintenance history already at schema "
f"{rep.from_version}; written by a multi-inverter "
f"version and not touched")
return rep
rep.ran = True
_move_flat_keys(store, unit, MAINT_UNIT_KEYS, rep)
_verify_moves(store, rep, roster=names)
# ⚠️ `rep.ok` AND NOTHING ELSE - NOT `ok or kept`, NOT `not lost`, NOT
# `not misdirected`. BOTH guards gate this line and each one is pinned by a
# check that the other cannot satisfy: section 9c refuses a store whose
# failure is `misdirected` with `lost` empty, and section 9d refuses one
# whose failure is `lost` with `misdirected` empty AND `kept` non-empty -
# the mutation `if rep.ok or rep.kept:` stamps the marker on a migration
# that lost a month of history, and until section 9d existed all 19 suites
# were green on it.
if rep.ok:
store.set(SCHEMA_KEY, SCHEMA_VERSION)
store.set(MAINT_SCHEMA_KEY, MAINT_SCHEMA_VERSION)
else:
# ⚠️ NOT MARKED DONE. A migration that lost state must run again on the
# next start rather than be recorded as finished - the one thing worse
# than a failed migration is a failed migration a later version skips.
_LOG.error("per-inverter store migration LOST STATE and has not been "
"marked done: %s", rep.explain())
# ⚠️ NOT MARKED DONE, FOR EITHER FAILURE. The same sentence
# `migrate_unit_keys()` carries: the one thing worse than a failed
# migration is a failed migration a later version skips. Rule 1 makes
# the re-run a no-op where the keys did land, so refusing the marker
# costs one repeated pass and buys the difference between a migration
# that can still be fixed and one the store says already happened.
_LOG.error("per-pack maintenance-history migration did NOT land and "
"has not been marked done: %s", rep.explain())
return rep
+74 -17
View File
@@ -74,7 +74,8 @@ NOT READY banner already names the problem, (b) does not apply.
was overruled. That firmware file's own comment says a lower bound on this
"belongs in CFG-01, not here" - so here it is.
5. `sign_disagree_w` at or above `max_w`. [(a)]
5. `sign_disagree_w` - or SAFETY-30's `unit_sign_disagree_w` - at or above
`max_w`. [(a)]
The arbiter's `supervised` limit bounds every command to +-max_w, and
SAFETY-02's predicate needs `abs(command) > sign_disagree_w`. Set the
threshold at or above the ceiling and the command side of the comparison
@@ -234,6 +235,7 @@ from .p1 import is_enabled, validate_meter_options
from .peakguard import PEAK_OPTION_DEFAULTS
from .price import PRICE_OPTION_DEFAULTS, PROVIDERS
from .static_tariff import ProfileInvalid, parse_static_profile
from .watchdog import SIGN_OPTION_DEFAULTS
_LOG = logging.getLogger("goodwe.config")
@@ -741,6 +743,38 @@ def validate_startup_config(opts: dict, dev_min=None, dev_max=None) -> list[str]
f"sign_disagree_w below {weakest:.0f} W (300 W is the measured "
"default), or raise max_w.")
# ⚠️ AND THE SAME RULE FOR SAFETY-30's PER-UNIT THRESHOLD, WHICH IS A
# SEPARATE COMPARISON AND THEREFORE A SEPARATE WAY TO DISARM. It grades a
# unit's own SHARE against that unit's own measured power, and a share is
# bounded by the same `supervised` claim at +-max_w - so a threshold at or
# above the ceiling makes the command side unsatisfiable by construction,
# exactly as above. Written as its own branch rather than folded into the
# one above because the two produce different sentences and an operator
# needs to be told WHICH watchdog they have disarmed.
#
# ⚠️ REFUSED EVEN AT A ROSTER OF ONE, WHERE THIS CHECK IS NOT ARMED. A
# config that cannot work is refused when it is WRITTEN, not on the day a
# second inverter is added - which is the day nobody is watching the startup
# log. This is `meter_max_age_s`'s rule read the same way: the failure it
# prevents is silent.
unit_sign_w = _number(opts, "unit_sign_disagree_w",
float(SIGN_OPTION_DEFAULTS["unit_sign_disagree_w"]))
if unit_sign_w is None:
problems.append(f"unit_sign_disagree_w "
f"{opts.get('unit_sign_disagree_w')!r} is not a number")
elif max_w is not None and max_w > 0 and unit_sign_w >= max_w:
problems.append(
f"unit_sign_disagree_w is {unit_sign_w:.0f} W but max_w is "
f"{max_w:.0f} W, so the SAFETY-30 per-inverter sign-disagreement "
"watchdog can never fire: no inverter is ever commanded past max_w, "
f"and the watchdog needs a command over {unit_sign_w:.0f} W. At two "
"inverters that is the guard which catches a unit doing the "
"OPPOSITE of what it was told while its sibling covers for it at "
"the meter - the case the site-level watchdog is structurally "
f"blind to. Lower unit_sign_disagree_w below {max_w:.0f} W "
f"({SIGN_OPTION_DEFAULTS['unit_sign_disagree_w']:.0f} W is the "
"measured default), or raise max_w.")
# --- 6. a maintenance charge the ceiling can actually accommodate --------
# ⚠️ Strictly `>`, matching the arbiter exactly. The shaping is
# `Claim.limit(hi=-charge_w)` against `supervised`'s `lo=-max_w`, so
@@ -768,24 +802,47 @@ def validate_startup_config(opts: dict, dev_min=None, dev_max=None) -> list[str]
# rule 8 below already makes with `PEAK_OPTION_DEFAULTS` - and
# test_maintenance.py pins it by reading the shipped file.
#
# ⚠️ AND IT IS GRADED AGAINST THE FIRST UNIT'S CAP, NOT THE SMALLEST AND NOT
# THE FLAT ONE. Rule 5 takes the weakest unit because there is a watchdog on
# every one of them; there is exactly ONE `Maintenance` in this add-on and
# `Controller.maint_unit` names the pack it recalibrates - the first in the
# roster, because the cycle's history lives under un-namespaced Store keys.
# The shaping claim is filed on THAT actuator and on no other, so the cap
# that can contradict it is THAT unit's. Taking `min()` here would refuse a
# perfectly workable site - a 5000 W pack being recalibrated beside a 2000 W
# sibling the cycle never touches - which is the over-refusal this file's
# docstring says gets a validator switched off. When `maint_unit` stops
# being `inverters[0]`, this line moves with it.
# ⚠️ MAINT-05: IT IS GRADED AGAINST THE WEAKEST UNIT'S CAP NOW, AND THE
# COMMENT THIS REPLACES SAID SO WOULD HAPPEN. It read: "there is exactly ONE
# `Maintenance` in this add-on and `Controller.maint_unit` names the pack it
# recalibrates - the first in the roster, because the cycle's history lives
# under un-namespaced Store keys. The shaping claim is filed on THAT
# actuator and on no other, so the cap that can contradict it is THAT
# unit's. Taking `min()` here would refuse a perfectly workable site - a
# 5000 W pack being recalibrated beside a 2000 W sibling the cycle never
# touches ... When `maint_unit` stops being `inverters[0]`, this line moves
# with it."
#
# `maint_unit` has stopped being `inverters[0]`. There is one `Maintenance`
# per roster entry, each with its own namespaced history, and EVERY pack is
# recalibrated on its own schedule - so the sibling "the cycle never
# touches" does not exist any more and the over-refusal argument has no
# subject. The shaping claim is filed on whichever pack is charging, so the
# cap that can contradict it is any cap on the roster, and the one that
# contradicts it FIRST is the smallest. This is rule 5's conjunction for
# rule 5's reason: a threshold that is fine against the biggest unit says
# nothing about the smallest, and the smallest is the one that goes to 0 W.
#
# ⚠️ THIS REFUSES CONFIGS THAT USED TO START, AND THAT IS THE POINT RATHER
# THAN A SIDE EFFECT. `max_w` 5000 on pack A beside 2000 on pack B with
# `maintenance_charge_w` 3000 was accepted before this card and was correct
# while only pack A was ever recalibrated. Pack B now recalibrates too, and
# its cheap-window charge asks the arbiter for `hi = -3000` against a
# supervised `lo = -2000`: `lo > hi`, arbiter clause 3, 0 W, contradiction -
# a maintenance charge that silently never happens on that pack, which is
# exactly what rule 6 exists to refuse at the moment it is WRITTEN. At a
# roster of one `min()` over one cap is that cap and the comparison, the
# message and the boundary are identical.
charge_w = _number(opts, "maintenance_charge_w",
float(MAINT_OPTION_DEFAULTS["maintenance_charge_w"]))
# ⚠️ THE SAME PAIRING AS RULE 5, FOR THE SAME REASON. `caps[0]` is the first
# unit WITH A USABLE CAP, which is `inverters[0]` only while every earlier
# entry has one; `_unit_label(opts, 0)` was hard-wired to `inverters[0]`
# regardless. Both come off the one pair now.
maint_index, maint_cap = caps_ix[0] if caps_ix else (0, None)
# ⚠️ THE SAME PAIRING AS RULE 5, FOR THE SAME REASON: `unit_caps_indexed()`
# SKIPS entries with no usable `max_w`, so a caps-list position is not a
# roster position and `_unit_label()` indexes the roster. The index travels
# with the number, so the unit the message names is the unit the number came
# off - which on a roster whose first entry has a junk `max_w` is not the
# same thing at all.
maint_index, maint_cap = (min(caps_ix, key=lambda p: p[1]) if caps_ix
else (0, None))
maint_where = _unit_label(opts, maint_index)
if charge_w is None:
problems.append(
+351 -18
View File
@@ -164,11 +164,106 @@ saturation freeze's question (|command - the battery's actual power| > 500 W,
NOTES.md §4.7 row 4), and the two checks are complementary rather than
overlapping. Neither substitutes for the other.
⚠️ THAT PARAGRAPH IS ABOUT THE COMMAND-VERSUS-METER COMPARISON, AND SINCE
SAFETY-30 THAT IS NOT THE ONLY ONE THIS MODULE DRIVES. At a roster of two or
more each unit is ALSO graded against its own measured power, which is exactly
the comparison the paragraph says nothing here makes - see the SAFETY-30 section
below, which says what that does and does not take away from the freeze. At the
shipped roster of one the paragraph stands unchanged, word for word: the site
comparison is the only detector armed and the freeze is still the whole answer.
Nor does it see a sign inversion introduced DOWNSTREAM of this add-on - in the
firmware, on the wire, or in a future driver. Our view stays self-consistent and
this stays silent. That is a wire-versus-intent problem and it belongs to the
witness, which is the only vantage point that sees both.
------------------------------------------------------------------------------
SAFETY-30 - AT TWO INVERTERS THIS IS TWO QUESTIONS, NOT ONE
Everything above was written at a roster of ONE, and at N=1 it is right for a
reason that stops being true at N=2. `commanded_w` is what THIS controller put
on THIS actuator. `grid_w` is what the WHOLE SITE is exchanging with the
network. At one inverter those are two views of one thing - §6.2's identity -
which is what makes comparing them sound. At two they are not:
* A FALSE TRIP THE COMPARISON CANNOT AVOID. "Inverter A commanded -1500 W
while the meter reads +1500 W" is evidence of nothing at N=2: inverter B may
be discharging 3000 W and doing exactly what it was told. No fault anywhere,
and the check NOTES.md calls the highest-value entry in the safety table -
93 % agreement while healthy against 4 % during runaway, four episodes, zero
false positives - fires.
* THE CONVERSE, WHICH IS WORSE. A unit genuinely driving the wrong way is
MASKED, because the site figure NETS. Commanded +1500 W, measured -1500 W,
beside a sibling measured at +1500 W: the meter sits exactly where a healthy
site leaves it. This is not a threshold that is set too high - the
information is destroyed by the summation, and no tuning recovers it.
So the one comparison becomes two, each asked of the evidence that can answer
it, and they are DIFFERENT FAULTS rather than one fault at two scales:
PER UNIT this unit's standing command against ITS OWN measured power -
`Inverter.batt`, which on the live system is holding register
0x0518, this inverter's own AC power, per-instance since DEV-02.
Conclusion: "THE ACTUATOR I AM DRIVING IS NOT DOING WHAT I TOLD
IT". The evidence is about one unit, so the remedy is that unit:
the `failsafe` SET, which arbiter.CLAIM_SCOPE has called a UNIT
claim from the day it was written, for this exact reason.
SITE the SUM of the standing commands against the meter. Conclusion:
"THE SITE AS A WHOLE IS DRIVING THE WRONG WAY" - an inverted gain,
an integrator wound past the rail, a strategy pushing the wrong
side of the meter. That is the fault the 2026-08-20 capture
contains and it is a property of the CONNECTION, so the remedy is
the connection: a SITE SET that stops every inverter.
⚠️ AND AT N=1 THERE IS EXACTLY ONE ARMED DETECTOR, WHICH IS THE ONE THAT
SHIPPED. The two questions above are the SAME SENTENCE at a roster of one: the
sum of one command is that command, and "the site" and "the unit" name one
actuator. Arming both would be arming a duplicate - two latches, two banners and
two chances to spend a false-positive record that was earned by ONE comparison
on two captured bus days. `Controller.cycle()` therefore hands the single unit's
watchdog THE METER, under the shipped `sign_disagree_w` / `sign_disagree_s`,
exactly as before, and never arms the site one. `arm()` below is where that
choice is written down. Nothing about a one-inverter install changes - not the
comparison, not the thresholds, not the store keys, not the claim, not the
banner.
⚠️ WHAT THE PER-UNIT COMPARISON IS NOT. Two guards that overlap without saying
so is how a false-alarm generator gets built, so both neighbours are named:
* IT IS NOT SAFETY-31. `circulation.py` compares the units AGAINST EACH OTHER
- `sum(|B_i|) - |sum(B_i)|` over MEASURED power - and catches two inverters
pushing THROUGH each other, which nets to zero at the meter and about which
no single command is evidence at all. This compares each unit against ITS
OWN COMMAND. Take two units faithfully executing an allocation that ALLOC-01
would refuse: circulating power is the whole loop, and every per-unit sign
check here reads perfect agreement. Take one unit doing the exact opposite
of its share while its sibling covers for it: this reads the whole
disagreement, and circulating power can be zero, because the two measured
powers may well agree in sign. Neither is the other with a different
threshold.
* IT IS NOT THE SATURATION FREEZE, AND THE OVERLAP IS REAL AND BOUNDED. The
freeze asks `|command - measured| > saturation_w` (500 W) and concludes "the
inverter cannot follow"; it then stops the integrator winding, which is a
control action taken every day, on a unit that is merely SATURATED as
readily as on one that is reversed, and it neither latches nor parks
anything. This asks for OPPOSITION - a sign, on both sides of a threshold,
held continuously - which a saturated-but-obedient unit cannot produce at
all, and it LATCHES and commands that unit to zero. The freeze is a
magnitude test that shapes the loop; this is a direction test that stops an
actuator.
⚠️ GATE 1 STILL APPLIES TO BOTH, AND THAT IS A DELIBERATE CHOICE WITH A COST.
The per-unit comparison could have been armed whenever ANY non-zero command is
standing - a unit should do what it is told whoever told it - and that would
catch a wedged inverter during a maintenance charge. It is not, because the
whole no-false-positive argument above is built on `following`, because the
freeze already grades that window, and because this card's own warning is that
SAFETY-02's record was earned at N=1 and must not be spent on a check that fires
when nothing is wrong. What it costs is written here rather than discovered:
between the start of a maintenance cycle and its end, a unit reversing against
its own command is the freeze's to notice, not this module's. Widening it is a
card, with a rig campaign behind it, not an edit.
------------------------------------------------------------------------------
STATE, TRIP AND LATCH
@@ -198,6 +293,178 @@ from dataclasses import dataclass
from datetime import datetime
# SAFETY-30. CFG-03: THE ONE IN-CODE COPY OF THE FOUR MULTI-INVERTER DEFAULTS.
# `config.yaml`'s `options:` block is the source of truth an operator edits and
# the Supervisor reads, and `schema:` holds the bounds. This dict exists for the
# reason `circulation.CIRC_OPTION_DEFAULTS` and `maintenance.MAINT_OPTION_DEFAULTS`
# do - the Dockerfile does not put config.yaml in the image, so the running
# add-on cannot read it - and `test_watchdog.py` pins the two together by READING
# config.yaml, so a drift goes red in CI rather than shipping.
#
# ⚠️ EVERY KEY HERE IS DEAD AT A ROSTER OF ONE, and that is the point rather
# than a caveat. `Controller._arm_sign_watchdogs()` arms the single unit's
# detector against the meter with the SHIPPED `sign_disagree_w` /
# `sign_disagree_s`, which are read in main.py exactly where they always were
# and are not in this dict. Nothing below is consulted until a second inverter
# exists.
SIGN_OPTION_DEFAULTS = {
# --- THE MAGNITUDE GATE, AND IT IS NOT SAFETY-02's 300 W BY ACCIDENT ---
# ⚠️ DERIVED FROM `control.Tuning.saturation_w`, WHICH IS 500 W, AND THE
# DERIVATION IS THE POINT. This check gates on BOTH sides being over
# `min_w`, so the smallest divergence it can ever see between a command and
# the power that command produced is `2 * min_w` - the two are OPPOSED, so
# the gap is the sum of the magnitudes. At 250 W that is exactly 500 W: the
# gap at which `compute()`'s own anti-windup test already concludes THE
# INVERTER IS NOT FOLLOWING and freezes the integrator. So this watchdog
# cannot fire on a divergence the shipped control law itself calls ordinary
# tracking, and the two guards meet at a number rather than overlapping in a
# band nobody has drawn.
#
# MEASURED, on 1890 closed-loop two-inverter runs in which NOTHING is faulty
# (every unit executes the share the allocator gave it), swept over the
# WHOLE range config.yaml accepts for `gain` float(0.05,1.0) and `slew_w`
# int(50,5000), across both families of stimulus that can produce legitimate
# opposition. Worst CONTINUOUS opposition between a unit's standing command
# and its own 0x0518 power, as a function of this threshold:
#
# 50 W -> 25.0 s 250 W -> 10.0 s 700 W -> 5.0 s
# 150 W -> 15.0 s 300 W -> 10.0 s 1000 W -> 0.0 s
# 200 W -> 15.0 s 500 W -> 5.0 s
#
# ⚠️ AND THE CURVE IS FLAT, WHICH IS THE FINDING THAT DECIDED THE SHAPE OF
# THIS PAIR. Unlike the site comparison - where the meter is driven by the
# LOAD and a flapping load holds it opposed for minutes - a unit's own power
# CHASES ITS OWN COMMAND through a bounded ramp and a first-order lag, so the
# opposition can only last as long as the crossing does. Dropping the
# threshold from 300 W to 50 W buys 15 s of extra transient and nothing else.
# The work is therefore done by the WINDOW below, and this number is set
# where the control law already draws its line rather than being tuned.
#
# The largest opposed magnitude seen anywhere in those 1890 runs is 875 W;
# a threshold above that would gate NOTHING legitimate in at all, and is
# rejected because it would also be blind to a unit reversing a 900 W share.
"unit_sign_disagree_w": 250,
# --- THE WINDOW -------------------------------------------------------
# Three bounds, and the value clears all three.
#
# LOWER, from the rig: 10.0 s - two control cycles, the worst CONTINUOUS
# legitimate opposition above 250 W across all 1890 runs. It is a
# reversing site whose second write lands a cycle after the first, both
# units then slewing through zero behind a 10 s lag; nothing is wrong.
#
# LOWER, from the firmware: `inverter.LINK_STALE_S`, 30 s. A unit not
# executing its share for less than the firmware's own veto window is a
# LINK fault, and the add-on already answers it - liveness drops the unit
# and the survivors carry its share, with the record saying so. Firing
# inside that window would raise a second, louder alarm about a state
# already being handled, which is the SAFETY-15/SAFETY-19 failure exactly.
#
# UPPER, from SAFETY-02 itself: 180 s, the SITE hold. A unit that is not
# doing what it was told must not be parked LATER than the site would be
# parked for the same watts, and unlike the site comparison this one needs
# nothing to net out before it can conclude - the evidence is complete on
# the first cycle it appears.
#
# 85 s: seventeen control cycles - 8.5x the worst legitimate episode this
# rig can produce, 2.8x the firmware's own veto window, and under HALF the
# site hold. ⚠️ IT IS DELIBERATELY NOT 60, 75, 90, 120 OR 180. A round
# number here would be a guess wearing a measurement's clothes, and the
# three bounds above are what the number has to answer to.
#
# ⚠️ AND INVERTER B DOES NOT PHYSICALLY EXIST. Its ramp and lag in the rig
# are inverter A's, COPIED, carrying `measured=False` - SPIKE-04 measures
# the real ones. A slower second unit lengthens every crossing above, which
# is what the 8.5x buys headroom for. SPIKE-04 is the revisit trigger for
# both numbers here, not a reason to move them now.
"unit_sign_disagree_s": 85,
# --- THE SITE COMPARISON AT N>=2, AND IT IS NOT THE SHIPPED PAIR -------
# ⚠️ THE THRESHOLD IS CARRIED AND THE WINDOW IS NOT, WHICH IS A FINDING
# RATHER THAN A COMPROMISE. Only one of the two arguments behind 300 W /
# 180 s depends on how many inverters the site has:
#
# THE THRESHOLD is about NOISE. "Near zero the sign is noise, and near
# zero is where a healthy loop LIVES" - the deadband holds the command
# still while the meter wanders either side of `target_grid_w`. The
# deadband is 15 W and P1 is one meter whatever the roster, so nothing
# about that argument moves. 300 W, carried - and the two-inverter
# campaign says the carry is safe rather than merely convenient: in the
# load-reversal family the site figure ever goes opposed AND over 300 W in
# 994 of 1050 runs at two inverters and 955 of 1050 at one. The gate opens
# about as often either way; what changed is how LONG it stays open, which
# is the window's problem and not the threshold's.
#
# THE WINDOW is about a TRANSIENT, and the transient scales with the site.
# The measured worst legitimate case is a load reversal that the loop
# crosses zero slowly: the command has to travel the whole rail at
# `slew_w` per cycle, and at N>=2 the rail is `Controller.site_max_w` - the
# ROSTER SUM - while `slew_w` is unchanged. So the crossing takes about
# twice as long at two inverters as at one, and that is exactly what the
# rig measures.
#
# 1890 closed-loop runs in which nothing is faulty, the SAME campaign the
# per-unit pair above is read off, run at BOTH rosters as its own control.
# Worst CONTINUOUS opposition between the standing command(s) and the meter,
# LOAD-REVERSAL family, at a 300 W threshold:
#
# N=1 (the shipped comparison) 110.0 s - inside 180 s, 1.6x
# N=2 (the sum, same stimuli) 215.0 s - OVER 180 s
#
# ⚠️ SO CARRYING 180 s WOULD HAVE SHIPPED A FALSE TRIP, and it is a config a
# schema-legal install can reach: `gain` 0.05 and `slew_w` 50 are both the
# floor of what config.yaml accepts, and a +-2200 W load reversal is an oven
# and a heat pump. That is the number this card exists to not assume.
#
# 325 s: 1.51x the measured worst, well inside the 900 s quarter that bills
# the capacity tariff, and still inside every episode the fault day
# contains - 2026-08-20's four clusters run 16, 30, 77 and 77 minutes and
# its opposed runs reach 1750 s, so a 325 s window fires in all of them.
# What it costs is stated rather than hidden: about two and a half extra
# minutes of wrong-direction SITE power before the stop, bought to remove a
# false trip that a legal configuration produces. ⚠️ DELIBERATELY NOT 300 OR
# 360; the measurement is what it answers to.
#
# ⚠️ AND IT IS NOT THE PER-UNIT WINDOW EITHER, IN THE OTHER DIRECTION. The
# per-unit comparison is 85 s because a unit's own power CHASES ITS OWN
# COMMAND and the transient is two cycles; the site comparison is 325 s
# because the meter is driven by the LOAD and the transient is the loop
# crossing zero. Same predicate, two instruments, two windows, both measured.
"site_sign_disagree_w": 300,
"site_sign_disagree_s": 325,
}
# ⚠️ A LIMIT OF THE SITE COMPARISON THAT SAFETY-30 DID NOT INTRODUCE AND DOES
# NOT FIX, RECORDED HERE BECAUSE THE CAMPAIGN FOUND IT AND SILENCE WOULD BE
# WORSE. The rig's other stimulus family is a FLAPPING LOAD, and a square wave
# whose half period is about one control cycle opposes the command on EVERY
# cycle: the command chases the previous half-cycle's meter while the meter has
# already reversed. Across the 840 flapping runs the worst continuous opposition
# at a 300 W threshold is 980 s at two inverters - and 1000 s at ONE, which is
# the whole point. The N=1 arm is the control, and it says this is a property of
# comparing a lagged command against a meter, not a property of the second
# inverter. No window inside the 900 s billing quarter survives it, and no
# threshold up to 1000 W does either.
#
# It is not acted on here for three reasons, in order of weight: it is identical
# at the roster this add-on actually ships, so acting on it would be changing a
# live safety path on evidence that has nothing to do with this card; a
# ~6 kW square wave at 0.1 Hz sustained for a quarter of an hour is not a house
# (the captured healthy day's longest opposed run is 30 s); and the fix is not a
# number - it wants the detector to notice that the disagreement is ALTERNATING,
# which is a different check. It is a card, and this comment is its evidence.
# SAFETY-30. THE TWO NOUNS THE TRIP SENTENCE CAN NAME, WRITTEN ONCE. They are
# constants rather than literals at the call sites because `main.py` arms the
# detectors and `test_watchdog.py` grades what they say, and a string typed in
# two places is a string that drifts in one of them - the same argument
# `store.unit_key()` makes about a key name. `AGAINST_METER` is the wording that
# has always shipped and must not change: it is what a one-inverter install's
# `reason` reads, in the store, in the action log and on the status page.
AGAINST_METER = "a meter reading"
AGAINST_OWN_POWER = "its own measured power"
@dataclass(frozen=True)
class SignWatchdogConfig:
"""Defaults, not constants - config.yaml carries both (SAFETY-02 AC 5)."""
@@ -206,7 +473,7 @@ class SignWatchdogConfig:
hold_s: float = 180.0
def disagrees(commanded_w, grid_w, following: bool,
def disagrees(commanded_w, against_w, following: bool,
cfg: SignWatchdogConfig) -> bool:
"""The predicate, and the ONLY place it is written down.
@@ -215,12 +482,23 @@ def disagrees(commanded_w, grid_w, following: bool,
inline. Two copies of a safety predicate is one copy plus a future
disagreement about what the check means, and the copy that drifts is always
the one in the harness that grades it.
⚠️ `against_w` WAS CALLED `grid_w`, AND THE RENAME IS THE WHOLE OF SAFETY-30
IN ONE PARAMETER. The arithmetic is unchanged - opposed signs, neither side
near zero, and the loop actually driving - but WHAT the command is opposed
TO is now a property of the caller rather than a fact about this function.
The site comparison hands it the meter; each unit's own comparison hands it
that unit's measured power (register 0x0518). One predicate, two callers,
for `loop_is_following()`'s reason: a safety predicate written down twice is
one copy plus a future disagreement about what the check means. The
parameter is positional at every call site in both repositories, so the
rename moves no caller.
"""
return (following
and commanded_w is not None and grid_w is not None
and commanded_w * grid_w < 0
and commanded_w is not None and against_w is not None
and commanded_w * against_w < 0
and abs(commanded_w) > cfg.min_w
and abs(grid_w) > cfg.min_w)
and abs(against_w) > cfg.min_w)
def loop_is_following(resolution, loop_asked) -> bool:
@@ -272,7 +550,17 @@ def loop_is_following(resolution, loop_asked) -> bool:
class SignWatchdog:
"""One instance per actuator, evaluated once per control cycle.
"""One instance per COMPARISON, evaluated once per control cycle.
⚠️ SINCE SAFETY-30 THAT IS NOT THE SAME SENTENCE AS "one per actuator",
which is what this docstring used to say. There is one instance per
inverter - `Inverter.sign_wd`, latched under that unit's own store keys -
AND one for the site, `Controller.site_sign_wd`, latched under flat ones.
The two ask different questions of different evidence and reach different
remedies; `arm()` below is where an instance is told which of them it is.
At a roster of one they would be the same question, so only the per-unit
instance is ever armed there - see the SAFETY-30 section of the module
docstring.
⚠️ NOT SHARED ACROSS CADENCES. `disagree_since` is a wall-clock start time,
so driving one instance from two loops at two rates would have each one
@@ -282,8 +570,14 @@ class SignWatchdog:
"""
def __init__(self, cfg: SignWatchdogConfig | None = None, *,
tripped: bool = False, reason: str = ""):
tripped: bool = False, reason: str = "",
against: str = AGAINST_METER):
self.cfg = cfg or SignWatchdogConfig()
# SAFETY-30. WHAT THE COMMAND IS BEING COMPARED AGAINST, as the operator
# reads it in the trip sentence. The default is the meter, which is what
# every trip this add-on has ever taken was compared against and is what
# keeps a one-inverter install's `reason` string byte-identical.
self.against = str(against)
self.disagree_since: datetime | None = None
# ⚠️ RESTORED FROM THE STORE, NOT DEFAULTED. A latch that lives only in
# memory is not a latch: a container restart clears it, and the add-on
@@ -292,8 +586,31 @@ class SignWatchdog:
self.tripped = bool(tripped)
self.reason = reason if self.tripped else ""
# -- which of SAFETY-30's two questions this instance is asking ----------
def arm(self, cfg: SignWatchdogConfig, against: str) -> None:
"""Set the thresholds and the subject of the comparison. Nothing else.
⚠️ IT IS CONFIGURATION, NOT STATE, AND IT DELIBERATELY DOES NOT TOUCH
`disagree_since`, `tripped` OR `reason`. A detector that cleared its own
window when it was told what it was comparing against would lose the
episode it is in the middle of, every cycle, and would never accumulate
a hold at all - the check would be permanently green and nothing would
say so.
⚠️ AND IT IS RE-STATED EVERY CYCLE RATHER THAN FROZEN AT CONSTRUCTION.
The answer is a property of the ROSTER and of nothing else, and
`Controller.cycle()` already re-reads the roster every cycle for
liveness and for the allocation. Freezing it in `__init__` would also
make it wrong for any caller that appends an inverter afterwards, which
is exactly how a two-inverter fixture is built in this repository - and
a fixture that silently got the one-inverter comparison would grade
nothing while looking green.
"""
self.cfg = cfg
self.against = str(against)
# -- the whole check, and it is a pure function of its arguments ----------
def evaluate(self, commanded_w, grid_w, following: bool,
def evaluate(self, commanded_w, against_w, following: bool,
now: datetime) -> bool:
"""Advance the detector one cycle. Returns True while tripped.
@@ -301,15 +618,23 @@ class SignWatchdog:
after deadband, slew, quantisation and the arbiter. Not
the optimizer's raw wish: the wire carries the emitted
value, so anything else can disagree with the meter for a
reason that is not a fault.
`grid_w` net P1, + = importing. None when the meter is stale or
missing, which CLEARS the timer rather than accumulating
- a frozen figure held against a moving command would
otherwise manufacture a disagreement in about three
minutes, on top of SAFETY-01 having already commanded
zero for the same cause.
reason that is not a fault. For a per-unit instance it is
THAT UNIT'S share of it; for the site instance it is the
roster sum.
`against_w` what the command is graded against, and SAFETY-30 made
this a decision rather than a fact: net P1 for the SITE
comparison (+ = importing), and this unit's own measured
power for the PER-UNIT one (+ = discharging, register
0x0518). Both carry the same sign convention as the
command, which is why one predicate serves both. None
when the reading is stale or missing, which CLEARS the
timer rather than accumulating - a frozen figure held
against a moving command would otherwise manufacture a
disagreement in about three minutes, on top of SAFETY-01
having already commanded zero for the same cause.
`following` did the grid-following loop actually win this cycle? See
gate 1 in the module docstring.
gate 1 in the module docstring, and the SAFETY-30 section
on why the per-unit comparison keeps it.
`now` the caller's clock, so this stays testable and replayable.
"""
if self.tripped:
@@ -317,7 +642,7 @@ class SignWatchdog:
return True
cfg = self.cfg
if not disagrees(commanded_w, grid_w, following, cfg):
if not disagrees(commanded_w, against_w, following, cfg):
self.disagree_since = None
return False
@@ -330,9 +655,17 @@ class SignWatchdog:
return False
self.tripped = True
# ⚠️ `self.against` IS THE ONLY THING SAFETY-30 CHANGED IN THIS STRING,
# and it defaults to the wording that shipped - "a meter reading" - so a
# one-inverter install's `reason` is byte-identical to what an operator
# (and test_watchdog.py) has always read. A per-unit instance names what
# it actually compared instead, because "commanded +1500 W against a
# meter reading -1500 W" would be a false statement about which
# instrument produced the evidence, on the one line somebody woken by
# this alarm reads first.
self.reason = (
f"SIGN_DISAGREEMENT: commanded {commanded_w:+.0f} W against a meter "
f"reading {grid_w:+.0f} W for {held:.0f} s continuously "
f"SIGN_DISAGREEMENT: commanded {commanded_w:+.0f} W against "
f"{self.against} {against_w:+.0f} W for {held:.0f} s continuously "
f"(limits: both over {cfg.min_w:.0f} W, held over {cfg.hold_s:.0f} s)"
)
return True
+83
View File
@@ -167,6 +167,62 @@ options:
sign_disagree_w: 300
sign_disagree_s: 180
# --- SAFETY-30 per-inverter sign-disagreement watchdog ---------------------
# ⚠️ A DIFFERENT COMPARISON, NOT A SECOND COPY OF THE ONE ABOVE, AND IT IS
# ARMED ONLY WHEN THIS SITE HAS MORE THAN ONE INVERTER. The pair above grades
# the SITE's total command against P1 - one meter, one grid connection. This
# pair grades EACH inverter's own share against THAT inverter's own measured
# power (holding register 0x0518), and it catches the fault the meter cannot
# answer for once there are two units: a unit doing the opposite of what it
# was told, while its sibling covers for it and P1 reads exactly what a
# healthy site reads.
#
# At ONE inverter these two keys do nothing at all. §6.2's identity makes the
# site's total command that inverter's command, so the pair above is already
# asking this question of the same actuator, and arming a second detector
# would be two latches and two banners for one fault.
#
# Both defaults are measured on admin/goodwe's two-master rig, over 1890
# closed-loop runs in which nothing is faulty, swept across the whole range
# this file accepts for `gain` and `slew_w`. 250 W is `saturation_w / 2`, so
# the smallest divergence this can see is exactly the gap at which the control
# law itself already says the inverter is not following; 85 s is 8.5x the
# worst legitimate opposition those runs produce (10.0 s), 2.8x the firmware's
# own 30 s link veto, and under half the 180 s the site check takes. See
# app/watchdog.py's SIGN_OPTION_DEFAULTS and
# DECISIONS/SAFETY-30-per-unit-sign.md.
#
# ⚠️ unit_sign_disagree_w must stay BELOW max_w, for sign_disagree_w's reason:
# a share is bounded to +-max_w, so a threshold at or above it can never fire.
# validate.py refuses that config.
unit_sign_disagree_w: 250
unit_sign_disagree_s: 85
# --- SAFETY-30 SITE sign-disagreement watchdog -----------------------------
# ⚠️ THE SAME COMPARISON `sign_disagree_*` MAKES, AT THE ROSTER WHERE IT STOPS
# BEING A STATEMENT ABOUT ONE INVERTER. At two or more units the SUM of the
# standing commands is graded against P1, and the trip stops the whole site -
# what it catches is the site being asked for the wrong thing, which every
# per-inverter check is blind to because each unit is faithfully executing its
# share. At ONE inverter these two keys do nothing: `sign_disagree_*` above is
# already asking this of the same actuator.
#
# The threshold is the shipped 300 W, CARRIED, and the reason is that the
# argument behind it does not depend on the roster: near zero the sign is
# noise, the deadband is 15 W, and there is one P1 whatever is behind it.
#
# ⚠️ THE WINDOW IS NOT CARRIED, AND THAT IS A MEASUREMENT. The transient 180 s
# exists to reject is the loop crossing zero after a load reversal, and the
# command has to travel the whole SITE rail at `slew_w` per cycle - a rail
# that is the roster sum while `slew_w` is not. On 1890 two-inverter runs in
# which nothing is faulty, the worst continuous legitimate opposition at 300 W
# is 110 s at one inverter and 215 s at two: carrying 180 s across would have
# shipped a false trip reachable at `gain: 0.05`, `slew_w: 50` and an ordinary
# 2.2 kW load reversal. 325 s is 1.51x that, inside the 900 s billing quarter,
# and still inside every episode of the 2026-08-20 fault day.
site_sign_disagree_w: 300
site_sign_disagree_s: 325
# --- SAFETY-18 frozen-but-fresh meter detector -----------------------------
# Seconds the meter may keep answering with an UNCHANGED measurement while the
# command stands at the rail before the add-on raises an alarm. Both halves
@@ -487,6 +543,33 @@ schema:
sign_disagree_w: int(50,5000)
sign_disagree_s: int(30,1800)
# SAFETY-30. No "off" value, for SAFETY-02's reason.
#
# ⚠️ THE THRESHOLD'S RANGE IS THE SAME AS THE SITE CHECK'S ON PURPOSE, AND THE
# WINDOW'S FLOOR IS NOT. On the two-inverter rig the per-unit comparison's
# worst continuous legitimate opposition is nearly FLAT in the threshold -
# 25.0 s at 50 W, 15.0 s at 150 W, 10.0 s at 250 W, 5.0 s at 500 W - because a
# unit's own power chases its own command rather than the load. So no value in
# this range puts the guard outside the regime it can be answered for, and the
# floor does not have to be a knee the way `circulation_w`'s does.
unit_sign_disagree_w: int(50,5000)
# The floor of 30 is `inverter.LINK_STALE_S`, the firmware's own veto window:
# below it, a unit not executing its share is a LINK fault the add-on already
# answers by dropping the unit from the split, and firing inside that window
# would be a second, louder alarm about a state that is already being handled.
unit_sign_disagree_s: int(30,1800)
# SAFETY-30's SITE pair. Same ranges as `sign_disagree_*`, because it is the
# same comparison asked of the site total - and no "off" value, for the same
# reason. ⚠️ THERE IS DELIBERATELY NO validate.py RULE PAIRING
# site_sign_disagree_w WITH max_w, unlike the other two thresholds: the bound
# on the SITE command is `Controller.site_max_w`, the ROSTER SUM, and
# `validate_startup_config()` is handed options rather than a roster. A rule
# written against max_w would refuse a perfectly reachable two-inverter
# configuration the day CFG-04 lands. The schema range is the bound here.
site_sign_disagree_w: int(50,5000)
site_sign_disagree_s: int(30,1800)
# No "off" value, for SAFETY-02's reason: disarming a safety alarm is not a
# supported configuration. ⚠️ THE FLOOR OF 60 IS NOT A RECOMMENDATION. At 300 s
# the slowest a HEALTHY meter's 1 Wh register can tick and still look unchanged
+13 -5
View File
@@ -165,21 +165,29 @@ SITE_SOURCES = {
"identically zero for any one unit - there is no fact of "
"the matter about which of two opposing inverters is the "
"offender, so there is nothing to file it per-unit on",
# SAFETY-30, and it is the entry most easily confused with `failsafe`
# below, because the same three words describe both.
"sign_site": "the SUM of every standing command against the meter. The "
"offending number is the SITE target, which no single "
"inverter owns - parking one of them would leave the rest "
"pushing the same way with the guard satisfied",
}
UNIT_SOURCES = {
"device": "one inverter's own reported min/max rating",
"supervised": "max_w caps what may be WRITTEN to one setpoint entity",
"failsafe": "site-wide evidence, per-unit remedy: SAFETY-02 grades the "
"command THIS controller put on THIS actuator",
"failsafe": "per-unit evidence, per-unit remedy: since SAFETY-30 it "
"grades THIS unit's command against THIS unit's OWN "
"measured power (0x0518), so what it learns is about one "
"device. The site-wide half moved to `sign_site`",
"maintenance": "the monthly recalibration is a pack procedure",
}
for _src, _why in sorted(SITE_SOURCES.items()):
print(f" site {_src:<11} {_why}")
for _src, _why in sorted(UNIT_SOURCES.items()):
print(f" unit {_src:<11} {_why}")
check("all NINE claim sources are classified, and only those nine",
check("all TEN claim sources are classified, and only those ten",
set(CLAIM_SCOPE) == set(SITE_SOURCES) | set(UNIT_SOURCES))
check("the five that are about the GRID CONNECTION are site claims",
check("the six that are about the GRID CONNECTION are site claims",
{s for s, v in CLAIM_SCOPE.items() if v == SCOPE_SITE} == set(SITE_SOURCES))
check("the four that are about a PACK, a DEVICE or one write are per-inverter",
{s for s, v in CLAIM_SCOPE.items() if v == SCOPE_UNIT} == set(UNIT_SOURCES))
@@ -200,7 +208,7 @@ check("with one inverter every source routes to the site, per-inverter ones "
check("...and with none configured at all, likewise",
all(actuator_for(s, "inverter-a", ()) == ACTUATOR_SITE for s in CLAIM_SCOPE))
check("with TWO the identity breaks and the split becomes real: the four "
"per-inverter sources route to the unit this controller drives, the five "
"per-inverter sources route to the unit this controller drives, the six "
"site sources stay on the grid connection",
{s: actuator_for(s, "inverter-a", TWO) for s in CLAIM_SCOPE}
== {**{s: "inverter-a" for s in UNIT_SOURCES},
+137 -23
View File
@@ -572,6 +572,58 @@ check("...and a store that only fails to persist a DEFAULT is not reported as "
"dismiss before the real one arrives",
migrate_unit_keys(LosesTheLatch(), UNIT_A).ok is True)
# ⚠️ AND THE MARKER GATE, ON A STORE WHERE SOMETHING WAS ALREADY NAMESPACED.
# FOUND BY MAINT-05's ROUND-3 BATTERY, AND IT WAS A SURVIVOR OF ALL 19 FILES.
# `migrate_unit_keys()` gates `SCHEMA_KEY` on `rep.ok`, and every check above
# reaches that gate on a store where NOTHING was already under `unit/...` - so
# `rep.kept` is empty in all of them, and the mutation
#
# if rep.ok: -> if rep.ok or rep.kept:
#
# stamped the marker on a migration that DROPPED THE LATCH, with the whole suite
# green. That is a disarmed safety guard plus a store that says the migration
# finished, so the next start skips the one thing that would have re-armed it -
# DEV-02's own worst case with the paper trail filed under "done".
#
# The fixture is the one shape that separates them: `sign_reason` ALREADY
# namespaced (so `kept` is non-empty and the mutant's `or` fires) and the
# `sign_tripped: true` write dropped on the floor (so `lost` is non-empty and
# `ok` is False). `migrate_unit_keys()` is passed no roster, so `misdirected`
# cannot be what refuses this - it is the `lost` guard or nothing.
_kept_reason = "SIGN_DISAGREEMENT: already namespaced by a later version"
_broken_kept = LosesTheLatch({
"sign_tripped": True,
"sign_reason": "SIGN_DISAGREEMENT: measured",
unit_key(UNIT_A, "sign_reason"): _kept_reason})
_rep_kept = migrate_unit_keys(_broken_kept, UNIT_A)
print(f" a dropped latch write beside an already-namespaced reason: "
f"{_rep_kept.explain()}")
print(f" lost={len(_rep_kept.lost)} kept={len(_rep_kept.kept)} "
f"ok={_rep_kept.ok}; marker "
f"{_broken_kept.data.get(SCHEMA_KEY, '<<absent>>')!r}")
check("PRECONDITION: this fixture reaches the marker gate through the `lost` "
"guard with a NON-EMPTY `kept`, which is the combination none of the "
"checks above has. One key dropped, one key left alone, `ok` False. If a "
"later edit empties `kept` here the mutation goes invisible again, so the "
"shape is asserted rather than assumed",
_rep_kept.ran and len(_rep_kept.lost) == 1 and len(_rep_kept.kept) == 1
and not _rep_kept.ok
and _broken_kept.data[unit_key(UNIT_A, "sign_reason")] == _kept_reason)
check("⚠️ AND THE STORE IS NOT MARKED MIGRATED. The key that was ALREADY there "
"does not buy a marker for the latch that was not: stamping here would "
"tell the next start that the migration which dropped `sign_tripped` had "
"finished, so it would never run again and the guard would stay off on an "
"install whose file still reads `sign_tripped: true`",
SCHEMA_KEY not in _broken_kept.data)
check("...and the counterfactual, one property different: the SAME store shape "
"on a store that does not drop writes IS marked migrated. So the refusal "
"above is about the latch that was lost, not about `kept` being non-empty",
migrate_unit_keys(
MemStore({"sign_tripped": True,
"sign_reason": "SIGN_DISAGREEMENT: measured",
unit_key(UNIT_A, "sign_reason"): _kept_reason}),
UNIT_A).ok is True)
_broken2 = LosesTheLatch({"sign_tripped": True,
"sign_reason": "SIGN_DISAGREEMENT: measured"})
_c2d, _ = built(store=_broken2)
@@ -609,28 +661,40 @@ print("3. AC 3: a trip on B does not latch A, and it survives a restart")
# inverters get a share of ONE site target, so an ordinary wrong-way loop trips
# BOTH - correctly, because both really are pushing the wrong way. The state
# this part is about is the one the flat store keys made unreachable: ONE
# inverter standing on a command that opposes the meter while the other does
# not. It is reached the way the field would reach it - inverter B's writes are
# being REJECTED, so `inv.last_write` stays where it was while A's tracks the
# loop - and `standing` is what SAFETY-02 grades, per unit.
# inverter standing on a command that opposes what it is graded against while
# the other does not. It is reached the way the field would reach it - inverter
# B's writes are being REJECTED, so `inv.last_write` stays where it was while
# A's tracks the loop - and `standing` is what SAFETY-02 grades, per unit.
#
# ⚠️ SAFETY-30 CHANGED WHAT "OPPOSES" MEANS HERE, AND THE FIXTURE HAD TO MOVE
# WITH IT - WHICH IS THIS CHECK DOING ITS JOB. It used to stand B on a -1500 W
# charge against a +2000 W METER and let the site figure convict it. At a roster
# of two that comparison is exactly the defect SAFETY-30 removes: the meter
# answers for BOTH inverters, so it is evidence about neither. B is now
# convicted by its OWN power sensor - standing on a -1500 W charge while
# `sensor.batt_b` measures it DISCHARGING at +1500 W, which is one inverter
# doing the opposite of what it was told and is a fact about that inverter
# alone. Inverter A's own reading stays at 0 W, so A is never gated in.
#
# ⚠️ AND THE REJECTION WINDOW IS WHY THIS RUNS IN THREE SECONDS AND NOT THIRTY.
# `sign_disagree_s` is 1 s here; `LINK_STALE_S` is 30 s, so B is still LIVE for
# the whole run and the trip is graded before liveness has anything to say.
# `unit_sign_disagree_s` is 1 s here; `LINK_STALE_S` is 30 s, so B is still LIVE
# for the whole run and the trip is graded before liveness has anything to say.
def _tripping_site():
hass = Hass(
{"sensor.grid": 2000.0, "sensor.soc_a": 50.0, "sensor.batt_a": 0.0,
"sensor.soc_b": 50.0, "sensor.batt_b": 0.0},
"sensor.soc_b": 50.0, "sensor.batt_b": 1500.0},
{"number.sp_a": (-A_RATING, A_RATING, 1.0),
"number.sp_b": (-B_RATING_HYPOTHESIS, B_RATING_HYPOTHESIS, 1.0)},
reject={"number.sp_b"})
store = MemStore()
c, _ = built({"sign_disagree_w": 300, "sign_disagree_s": 1,
"unit_sign_disagree_w": 300, "unit_sign_disagree_s": 1,
"auto_start": True}, store=store, hass=hass)
c.inverters.append(inverter_b(store, sign_cfg=c.inverters[0].sign_wd.cfg))
c.auto = True
# B is standing on a charge command while the house imports 2000 W. Its
# writes are rejected, so this is what it goes on standing at.
# B is standing on a charge command while its own sensor says it is
# discharging. Its writes are rejected, so this is what it goes on
# standing at.
c.inverters[1].last_write = -1500.0
return c, hass, store
@@ -681,10 +745,12 @@ check("⚠️ ...AND THE TRIP IS MIRRORED ONTO THE PRE-DEV-02 FLAT PAIR, WHICH I
# THE RESTART, which is the whole reason the latch is persisted at all.
_h3b = Hass({"sensor.grid": 2000.0, "sensor.soc_a": 50.0,
"sensor.batt_a": 0.0, "sensor.soc_b": 50.0, "sensor.batt_b": 0.0},
"sensor.batt_a": 0.0, "sensor.soc_b": 50.0,
"sensor.batt_b": 1500.0},
{"number.sp_a": (-A_RATING, A_RATING, 1.0),
"number.sp_b": (-B_RATING_HYPOTHESIS, B_RATING_HYPOTHESIS, 1.0)})
_c3r, _ = built({"sign_disagree_w": 300, "sign_disagree_s": 1,
"unit_sign_disagree_w": 300, "unit_sign_disagree_s": 1,
"auto_start": True}, store=_s3, hass=_h3b)
_c3r.inverters.append(inverter_b(_s3,
sign_cfg=_c3r.inverters[0].sign_wd.cfg))
@@ -703,12 +769,32 @@ check("a brand new Controller against the same store comes back with B "
check("...and A goes on carrying the site target across the restart too",
_c3r.allocation.shares.get("inverter-a") == _c3r.allocation.sum_w
and _c3r.allocation.sum_w != 0)
check("...and the status page shows BOTH rows - one latched, one armed. A "
"single row would have to pick one inverter to describe, and 'the sign "
"watchdog is armed' printed over a latched sibling is the page lying",
len([r for r in _c3r.checks() if "sign watchdog" in r["text"]]) == 1
check("...and the status page shows BOTH per-inverter rows - one latched, one "
"armed. A single row would have to pick one inverter to describe, and "
"'the sign watchdog is armed' printed over a latched sibling is the page "
"lying",
len([r for r in _c3r.checks()
if r["text"].startswith("sign watchdog [")]) == 1
and len([r for r in _c3r.checks()
if "SIGN-DISAGREEMENT FAILSAFE" in r["text"]]) == 1)
if "SIGN-DISAGREEMENT FAILSAFE [" in r["text"]]) == 1)
check("⚠️ AND SAFETY-30's SITE ROW IS THERE BESIDE THEM, EXACTLY ONCE. At two "
"inverters there are three sign questions on this page and they are three "
"rows: is A doing what it was told, is B, and is the SITE'S TOTAL "
"pointing the right way. The last one is not derivable from the other "
"two - both units can execute their shares perfectly while the number "
"they were given is itself wrong - so a page that showed only the "
"per-unit rows would be advertising two guards where the install has "
"three",
len([r for r in _c3r.checks()
if r["text"].startswith("site sign watchdog:")]) == 1)
check("...and the per-inverter rows now say WHAT the unit was compared "
"against, because at a roster of two it is no longer the meter. A row "
"still reading 'the command opposed the meter' would send an operator to "
"look at P1 for a fault that is visible only on one inverter's own power "
"sensor",
all("its own measured power" in r["text"] for r in _c3r.checks()
if r["text"].startswith(("sign watchdog [",
"SIGN-DISAGREEMENT FAILSAFE ["))))
# The operator's clear, which is one button and has to reach every latch.
run(_c3r.handle_action("auto_toggle")) # -> stopped
@@ -728,8 +814,9 @@ check("...and the same press cleared the flat rollback view, so a downgrade "
"- and neither does the next start, which now reads that key on EVERY "
"boot rather than only on the one that migrates",
_s3.data["sign_tripped"] is False and _s3.data["sign_reason"] == "")
_c3r2, _ = built({"sign_disagree_w": 300, "sign_disagree_s": 1}, store=_s3,
hass=_h3b)
_c3r2, _ = built({"sign_disagree_w": 300, "sign_disagree_s": 1,
"unit_sign_disagree_w": 300, "unit_sign_disagree_s": 1},
store=_s3, hass=_h3b)
check("⚠️ AND THE RESTART AFTER THAT CLEAR COMES UP UNLATCHED. This is the "
"'re-arm forever' hazard the every-start guard would have if the clear "
"did not reach both spellings, driven through the real button and a real "
@@ -1594,11 +1681,28 @@ check("⚠️ RULE 5 NAMES THE UNIT THE NUMBER CAME OFF. The weakest cap is "
"is inverter-B. The refusal says inverter-c",
_why(_p_skew, "max_w (inverter-c) is 1000 W")
and not _why(_p_skew, "max_w (inverter-b) is 1000 W"))
check("⚠️ RULE 6 TOO. `caps[0]` is the first entry WITH a usable cap - "
"inverter-b - while `_unit_label(opts, 0)` was hard-wired to inverter-a, "
"so the maintenance refusal quoted B's rating under A's name",
_why(_p_skew, "max_w (inverter-b) is 2000 W")
and not _why(_p_skew, "max_w (inverter-a) is 2000 W"))
# ⚠️ MAINT-05 MOVED RULE 6's SUBJECT, AND THIS CHECK MOVED WITH IT. It used to
# assert the refusal named inverter-b, because rule 6 graded the FIRST entry
# with a usable cap - `Controller.maint_unit` was `inverters[0]` and the shaping
# claim could be filed nowhere else. There is now one `Maintenance` per roster
# entry, so the shaping claim can be filed on ANY pack and the cap that
# contradicts it first is the SMALLEST - inverter-c's 1000 W. The property this
# check exists for is unchanged and is the reason it is still here: the index
# has to travel with the cap, or the message quotes one unit's rating under
# another unit's name.
check("⚠️ RULE 6 TOO, AND SINCE MAINT-05 IT IS THE WEAKEST PACK IT NAMES. "
"inverter-c's 1000 W sits at caps index 1, and `_unit_label(opts, 1)` is "
"inverter-B - so a caps index leaking into a roster index would say "
"inverter-b here. The refusal says inverter-c, with inverter-c's number",
_why(_p_skew, "maintenance_charge_w is 3000 W but max_w (inverter-c) is "
"1000 W")
and not _why(_p_skew, "maintenance_charge_w is 3000 W but max_w "
"(inverter-b)"))
check("...and it is no longer graded against the FIRST usable cap at all, which "
"is the behaviour change MAINT-05 makes here: inverter-b's 2000 W was the "
"pre-card answer and appears nowhere in the maintenance refusal now",
not _why(_p_skew, "maintenance_charge_w is 3000 W but max_w "
"(inverter-b) is 2000 W"))
check("...and the counterfactual, so this is not a check that would pass on a "
"validator that named nothing: the OLD expressions still evaluate, on "
"this same config, to the wrong names",
@@ -2097,6 +2201,16 @@ MUTATIONS = [
("validate.py allowing two roster entries with the SAME name",
"part 10: 'two entries with the SAME name are refused' - one Store "
"namespace for two units is DEV-02's defect rebuilt in YAML"),
("`migrate_unit_keys()`'s marker gate `if rep.ok:` -> `if rep.ok or "
"rep.kept:` - the schema marker stamped on a migration that DROPPED THE "
"LATCH, because one key happened to be namespaced already",
"part 10: 'AND THE STORE IS NOT MARKED MIGRATED'. ⚠️ THIS ONE SURVIVED "
"ALL 19 FILES and was found by MAINT-05's round-3 battery, not by this "
"card: every migration check here reached the gate on a store where "
"NOTHING was namespaced yet, so `rep.kept` was empty in all of them and "
"the `or` never fired. Stamping there disarms the latch AND tells the "
"next start the migration finished, so it never re-arms - DEV-02's own "
"worst case filed under 'done'"),
]
# ⚠️ FOUR OF THESE SURVIVED THE FIRST TIME AND THE CHECKS WERE CHANGED, NOT THE
# TABLE - two in DEV-02, two more in CFG-04, and the CFG-04 pair are recorded in
@@ -2115,7 +2229,7 @@ for what, caught in MUTATIONS:
check(f"all {len(MUTATIONS)} defeats of this card are caught by a named check "
f"above. A guard whose checks survive its own defeat is not a guard, and "
f"this repository has three separate records of exactly that",
len(MUTATIONS) == 34)
len(MUTATIONS) == 35)
print()
File diff suppressed because it is too large Load Diff
+47 -25
View File
@@ -2410,8 +2410,19 @@ check("...and so does the persistent notification, which outlives the page and "
# loop} at every SoC from 0 to 100 %, because nothing in those fixtures
# circulates. What it does is stop the site when two inverters are measured
# pushing against each other. See DECISIONS/SAFETY-31-circulating-power.md.
# ⚠️ SAFETY-30 ADDED THE TENTH, AND IT NEEDS ITS OWN PARAGRAPH FOR THE REASON
# `circulation` needed one. `sign_site` is a SET at P_SAFETY on the SITE
# actuator, filed only while `site_sign_wd.tripped` AND the roster holds more
# than one inverter. It is not a second copy of `failsafe`: that one grades ONE
# unit's command against THAT unit's own measured power and parks that unit,
# while this grades the SUM of the commands against the meter and stops the
# site. It appears in NO cycle of the sweeps below - the dynamic halves still
# observe exactly {device, supervised, loop} at every SoC, because the shipped
# roster is one inverter and this claim cannot be filed there at all. See
# DECISIONS/SAFETY-30-per-unit-sign.md.
CLAIM_SOURCES = {"device", "supervised", "failsafe", "safety", "peak",
"arbitrage", "maintenance", "loop", "circulation"}
"arbitrage", "maintenance", "loop", "circulation",
"sign_site"}
# The two appends that file a claim BUILT ELSEWHERE in the method. Named here so
# that a THIRD such indirection reads as an unrecognised append rather than as a
# silent gap in the scan.
@@ -2590,13 +2601,17 @@ print(" site: " + ", ".join(sorted(s for s, v in CLAIM_SCOPE.items()
if v == SCOPE_SITE))
+ " | per-inverter: "
+ ", ".join(sorted(s for s, v in CLAIM_SCOPE.items() if v == SCOPE_UNIT)))
check("...and the split is the one ARB-03 argued, with SAFETY-31's addition: "
"the five about the GRID CONNECTION are site claims and the four about a "
"pack, a device or one write are per-inverter. `circulation` is a site "
"claim because its quantity is defined over the roster and is identically "
"zero for any one unit - there is no inverter to attribute it to",
check("...and the split is the one ARB-03 argued, with SAFETY-31's and "
"SAFETY-30's additions: the six about the GRID CONNECTION are site "
"claims and the four about a pack, a device or one write are "
"per-inverter. `circulation` is a site claim because its quantity is "
"defined over the roster and is identically zero for any one unit; "
"`sign_site` is one because the number that tripped it is the SITE "
"target, which no single inverter owns - while `failsafe` beside it is "
"per-inverter because since SAFETY-30 it grades one unit's command "
"against that unit's OWN measured power",
{s for s, v in CLAIM_SCOPE.items() if v == SCOPE_SITE}
== {"loop", "peak", "arbitrage", "safety", "circulation"}
== {"loop", "peak", "arbitrage", "safety", "circulation", "sign_site"}
and {s for s, v in CLAIM_SCOPE.items() if v == SCOPE_UNIT}
== {"device", "supervised", "failsafe", "maintenance"})
@@ -2672,19 +2687,21 @@ _restore_b()
print(" with a second inverter on the roster, main.py routes: "
+ ", ".join(f"{s}->{a}" for s, a in sorted(_routed.items())))
check("⚠️ AND THIS IS THE BREAKING HALF. Give the same Controller a second "
"inverter and its OWN routing method splits the nine: `device`, "
"inverter and its OWN routing method splits the ten: `device`, "
"`supervised`, `failsafe` and `maintenance` bind the unit this controller "
"drives, `loop`, `peak`, `arbitrage`, `safety` and SAFETY-31's "
"`circulation` stay on the grid connection. Swap any one of them - route "
"`peak` to the unit and the site keeps no floor; route `maintenance` to "
"the site and it can cross that floor into `lo > hi`; route `circulation` "
"to one unit and a site-wide loop parks the inverter that may well be the "
"one still listening - and this line goes red",
"drives, `loop`, `peak`, `arbitrage`, `safety`, SAFETY-31's `circulation` "
"and SAFETY-30's `sign_site` stay on the grid connection. Swap any one of "
"them - route `peak` to the unit and the site keeps no floor; route "
"`maintenance` to the site and it can cross that floor into `lo > hi`; "
"route `circulation` to one unit and a site-wide loop parks the inverter "
"that may well be the one still listening; route `sign_site` to one unit "
"and a SITE total pointing the wrong way parks one inverter while the "
"rest of the roster goes on pushing the same way - and this line goes red",
_routed == {"device": "inverter-a", "supervised": "inverter-a",
"failsafe": "inverter-a", "maintenance": "inverter-a",
"loop": ACTUATOR_SITE, "peak": ACTUATOR_SITE,
"arbitrage": ACTUATOR_SITE, "safety": ACTUATOR_SITE,
"circulation": ACTUATOR_SITE})
"circulation": ACTUATOR_SITE, "sign_site": ACTUATOR_SITE})
check("...and the roster it was restored to is the shipped one, so nothing "
"below inherits a two-inverter Controller",
c_src.units == (app_main.UNIT_A,) and len(c_src.inverters) == 1)
@@ -2716,7 +2733,7 @@ check("...and the roster it was restored to is the shipped one, so nothing "
# actuator as a LITERAL - `actuator=self.actuator_of('<source>')` - so the
# literal is simply required to EQUAL the source the claim is filed under. That
# is what makes "a mis-typed `actuator_of(...)` goes red" a true sentence about
# all nine rather than about the three a fixture happens to reach.
# all ten rather than about the three a fixture happens to reach.
def append_actuator(node):
@@ -2765,16 +2782,16 @@ _unit_sources_unrouted = [
if CLAIM_SCOPE.get(r[0]) == SCOPE_UNIT and "," not in r[1]]
print(f" {len(_direct)} direct appends, each read at its own line: "
+ ", ".join(f"main.py:{ln} {s} -> {a}" for s, a, ln in _routing))
check("all NINE of `Controller.cycle()`'s DIRECT appends route through "
check("all TEN of `Controller.cycle()`'s DIRECT appends route through "
"`self.actuator_of(<string literal>)`, so this scan can read the routing "
"of EVERY one of them. An append that computed its actuator, or omitted "
"the argument, or hand-wrote a string, fails HERE rather than being "
"silently skipped - which is how a scan like this goes quietly vacuous, "
"and 10f is in this file because that already happened once",
len(_direct) == 9
len(_direct) == 10
and all(a is not None and a.startswith("self.actuator_of('")
for _s, a, _ln in _routing))
check("⚠️ AND AT EVERY ONE OF THE NINE THE LITERAL IT ROUTES THROUGH IS THE "
check("⚠️ AND AT EVERY ONE OF THE TEN THE LITERAL IT ROUTES THROUGH IS THE "
"SOURCE THE CLAIM IS FILED UNDER. THIS is the line a mis-typed "
"`actuator_of(...)` at a call site goes red on - all nine of them, on "
"the line the typo is written, at every roster, INCLUDING the six "
@@ -2788,8 +2805,11 @@ check("⚠️ AND EVERY PER-INVERTER APPEND NAMES THE INVERTER IT IS ABOUT. "
"site-scoped sources and is the whole defect for a per-inverter one: a "
"`device` limit built inside the roster loop and routed without `inv` "
"files inverter B's rating on inverter A, where it binds the wrong "
"share. Five of the nine appends are SCOPE_UNIT - `device`, "
"`supervised`, `failsafe` and maintenance's two - and all five pass one",
"share. Five of the ten appends are SCOPE_UNIT - `device`, "
"`supervised`, `failsafe` and maintenance's two - and all five pass one. "
"SAFETY-30's `sign_site` is deliberately NOT one of them: it is filed on "
"the site, beside `circulation`, because the number that tripped it is "
"the site target",
_unit_sources_unrouted == []
and len([r for r in _routing
if CLAIM_SCOPE.get(r[0]) == SCOPE_UNIT]) == 5)
@@ -2844,13 +2864,15 @@ check("⚠️ AND THE RATING LIMITS ARE FILED ONCE PER INVERTER, ON ITS OWN "
("loop", ACTUATOR_SITE), ("supervised", "inverter-a"),
("supervised", "inverter-b")])
check("...and naming those three is not an accident of this fixture but the "
"reason the static pin exists - six of the nine sources need a state "
"reason the static pin exists - SEVEN of the ten sources need a state "
"no check in this file reaches. SAFETY-31's `circulation` is one of the "
"six: it is filed only while the site is MEASURED to be circulating, "
"which no fixture here is",
"seven: it is filed only while the site is MEASURED to be circulating, "
"which no fixture here is. SAFETY-30's `sign_site` is another, and it "
"needs TWO states at once - a roster of two AND the site's total command "
"standing opposed to the meter for its whole window",
{c.source for c in _cl2}
== CLAIM_SOURCES - {"failsafe", "safety", "maintenance", "peak",
"arbitrage", "circulation"})
"arbitrage", "circulation", "sign_site"})
# ⚠️ AND NOW THE HALF THAT WAS EVIDENCE-SHAPED AND OBSERVED NOTHING. The three
# checks above grade WHICH ACTUATOR each claim lands on. They say nothing about
# WHOSE NUMBERS the bound is built from - and those are two different defects
+75
View File
@@ -353,6 +353,64 @@ check("...and it is refused with maintenance_enabled off, as shipped",
base(max_w=2000, maintenance_charge_w=2500,
maintenance_enabled=False), -5000, 5000), "THE CEILING WINS"))
# ⚠️ MAINT-05: AND AT TWO PACKS IT IS THE WEAKEST CAP, NOT THE FIRST ENTRY'S.
# The comment this replaces in validate.py said, in as many words, that it took
# the FIRST unit's cap because there was exactly one `Maintenance` and it could
# only ever recalibrate `inverters[0]` - and that "when `maint_unit` stops being
# `inverters[0]`, this line moves with it". It has. Every pack is now
# recalibrated on its own schedule, so the shaping claim can be filed on any of
# them and the cap that contradicts it first is the smallest.
#
# ⚠️ THE FIXTURE'S TWO CAPS ARE DELIBERATELY DIFFERENT AND THE RATE SITS
# BETWEEN THEM. Two equal caps could not tell "grades the first" from "grades
# the smallest", and a rate above both would be refused by either rule - so
# 3000 W against caps of 5000 W and 2000 W is the one arrangement where the old
# answer and the new one disagree. ⚠️ 2000 W for the second pack is a
# HYPOTHESIS: inverter B does not physically exist and nothing here is measured
# from one.
def _roster(a_max, b_max, **over):
o = base(**over)
for k in ("setpoint_entity", "soc_entity", "batt_entity", "max_w"):
o.pop(k, None)
o["inverters"] = [
dict(name="inverter-a", setpoint_entity="number.sp_a",
soc_entity="sensor.soc_a", batt_entity="sensor.batt_a",
max_w=a_max),
dict(name="inverter-b", setpoint_entity="number.sp_b",
soc_entity="sensor.soc_b", batt_entity="sensor.batt_b",
max_w=b_max)]
return o
_two = _roster(5000, 2000, maintenance_charge_w=3000, maintenance_discharge_w=1000,
sign_disagree_w=300, unit_sign_disagree_w=300)
_p_two = [p for p in validate_startup_config(_two, -5000, 5000)
if "maintenance_charge_w" in p]
print(f" 5000 W and 2000 W packs, charge 3000 W -> {_p_two}")
check("⚠️ A CHARGE RATE THE SMALLEST PACK CANNOT TAKE IS REFUSED, even though "
"the FIRST pack could. Before MAINT-05 the second pack was never "
"recalibrated, so its cap could not contradict anything; it is now, and "
"its cheap-window charge would ask for hi=-3000 against lo=-2000 - the "
"`lo > hi` contradiction, 0 W, on the pack nobody was watching",
len(_p_two) == 1 and why(_p_two, "max_w (inverter-b) is 2000 W"))
check("...and the message names the pack the number came off, not the first "
"entry - an operator told to lower a number on the wrong device is being "
"sent to the wrong device by the message that exists to stop that",
not why(_p_two, "max_w (inverter-a)"))
check("...while the same rate against two packs that can both take it starts, "
"so this is a comparison and not a blanket refusal of rosters",
[p for p in validate_startup_config(
_roster(5000, 4000, maintenance_charge_w=3000,
maintenance_discharge_w=1000, sign_disagree_w=300,
unit_sign_disagree_w=300), -5000, 5000)
if "maintenance_charge_w" in p] == [])
check("...and at a ROSTER OF ONE `min()` over one cap is that cap: the same "
"message, the same boundary, the same refusal the flat config always got",
why(validate_startup_config(base(max_w=2000, maintenance_charge_w=2500),
-5000, 5000), "THE CEILING WINS")
and validate_startup_config(base(max_w=2000, maintenance_charge_w=2000),
-5000, 5000) == [])
# ⚠️ THE SIBLING, CHECKED RATHER THAN ASSUMED. The shaping is NOT symmetric:
# drain files Claim.set at P_MAINTENANCE and owns the actuator, so the ceiling
# CLAMPS it (+2000 W) instead of contradicting it, and the arbiter's explain()
@@ -523,9 +581,26 @@ check("...and the message explains it would read as a trip, not a mistake",
# check below.
check("the smallest sane max_w still starts",
validate_startup_config(base(max_w=1, sign_disagree_w=0.5,
unit_sign_disagree_w=0.5,
maintenance_charge_w=1), -5000, 5000) == [])
check("...but that same 1 W ceiling with the DEFAULT sign threshold is refused",
why(validate_startup_config(base(max_w=1), -5000, 5000), "never fire"))
# ⚠️ SAFETY-30 ADDED A SECOND WAY TO DISARM A SIGN WATCHDOG BY CONFIGURATION,
# and the fixture above now has to name BOTH thresholds - which is the check
# doing its job rather than a bookkeeping edit. `unit_sign_disagree_w` bounds a
# unit's own SHARE, which the same `supervised` claim holds at +-max_w, so at or
# above the ceiling it is unsatisfiable by construction exactly as
# `sign_disagree_w` is. Refused even at the shipped roster of one, where this
# watchdog is not armed: a config that cannot work is refused when it is
# written, not on the day somebody adds a second inverter.
check("...and the per-inverter threshold has the same rule, named separately "
"so an operator is told WHICH watchdog they disarmed",
why(validate_startup_config(base(max_w=1, sign_disagree_w=0.5,
maintenance_charge_w=1), -5000, 5000),
"unit_sign_disagree_w"))
check("...and one watt under that ceiling is reachable, so accepted",
validate_startup_config(base(max_w=2000, unit_sign_disagree_w=1999),
-5000, 5000) == [])
# --------------------------------------------------------------------------- #
print("what must NOT be refused - a validator that fires on trivia gets deleted")
+706 -2
View File
@@ -401,10 +401,19 @@ print("controller and the first thing an operator will try")
class _MemStore:
"""A Store that keeps its dict across "restarts", like the real file does."""
"""A Store that keeps its dict across "restarts", like the real file does.
⚠️ SEEDED FROM `store.DEFAULTS`, NOT FROM A LITERAL. `Store.load()` merges
that dict over whatever is on disk, so a fake that hand-listed three keys
answered `None` where the real store answers a default - and a check reading
"this latch was never written" then could not tell an absent key from a
False one. It is also a place to drift: SAFETY-31 and SAFETY-30 both added
flat pairs, and neither would have appeared here.
"""
def __init__(self):
self.data = {"phase": "idle", "sign_tripped": False, "sign_reason": ""}
from app.store import DEFAULTS as _SDEF
self.data = dict(_SDEF)
def set(self, k, v):
self.data[k] = v
@@ -1105,6 +1114,701 @@ check("⚠️ AND THE 'HAS A HUMAN SEEN THIS' FLAG IS CLEARED WITH IT. Left "
_cc._circ_announced is False)
print()
print("=" * 72)
print("SAFETY-30: at two inverters the sign check is TWO questions, not one")
print("=" * 72)
# ⚠️ READ THIS BEFORE ADDING A CHECK HERE. Half of this card was already built
# and it is NOT the half below. DEV-02 shipped the PER-INSTANCE half: each
# `Inverter` owns its own `SignWatchdog`, latched under its own namespaced store
# keys, and a trip on B no longer latches A. test_inverter.py part 3 grades
# that; nothing here re-tests it.
#
# What is graded here is THE COMPARISON ITSELF. Before this card every unit's
# detector was handed `self.grid`. There is ONE P1, so at N>=2 that is a number
# this controller put on ONE actuator, graded against a measurement that answers
# for ALL of them - wrong in both directions: a FALSE trip when a sibling
# accounts for the meter, and a MISSED one when a sibling MASKS a unit that
# really is reversed. The fix is two comparisons, and they are two different
# faults: each unit against ITS OWN measured power, and the SUM of the standing
# commands against the meter.
#
# ⚠️ AND IT IS NOT SAFETY-31, WHICH IS THE SECTION ABOVE. `circulation.py`
# compares the units AGAINST EACH OTHER and catches two inverters pushing
# THROUGH each other, which nets to zero at the meter. This compares each unit
# against ITS OWN COMMAND and catches a unit not doing what it was told. They
# overlap in exactly one scenario - a unit reversing while its sibling pulls the
# other way - and part 4 below asserts what each one concludes there, because
# two guards that overlap without saying so is how a false-alarm generator gets
# built.
from app.control import Tuning as _Tuning # noqa: E402
from app.watchdog import ( # noqa: E402
AGAINST_METER, AGAINST_OWN_POWER, SIGN_OPTION_DEFAULTS, disagrees,
)
print()
print("1. ONE predicate, TWO second terms - and the trip sentence says which")
# ⚠️ THE ARITHMETIC IS UNCHANGED AND THAT IS THE POINT: `disagrees()` was not
# split in two. What moved is a PARAMETER NAME - `grid_w` became `against_w` -
# because what the command is opposed TO is now the caller's decision. A second
# copy of a safety predicate is one copy plus a future disagreement about what
# the check means (see `loop_is_following()`), so there is still exactly one.
_scfg = SignWatchdogConfig(min_w=300.0, hold_s=180.0)
_ucfg = SignWatchdogConfig(
min_w=float(SIGN_OPTION_DEFAULTS["unit_sign_disagree_w"]),
hold_s=float(SIGN_OPTION_DEFAULTS["unit_sign_disagree_s"]))
check("the same call that means 'commanded -1000 W into a +1500 W import' "
"means 'commanded -1000 W while measuring +1500 W of discharge' when the "
"caller hands it a unit's own power - one function, two questions",
disagrees(-1000.0, 1500.0, True, _scfg) is True)
check("...and every gate still binds on the second term whatever it is: a "
"250 W reading does not qualify against a 300 W gate",
disagrees(-1000.0, 250.0, True, _scfg) is False)
check("...nor does a unit sitting at zero, which is a REFUSED command and not "
"a reversed one - the distinction the module docstring makes about a "
"battery declining a discharge, now read of the unit's own sensor",
disagrees(1500.0, 0.0, True, _ucfg) is False)
_wd_site = SignWatchdog(_scfg)
_wd_unit = SignWatchdog(_ucfg, against=AGAINST_OWN_POWER)
for _t in range(0, 400, 5):
_wd_site.evaluate(-1000.0, 1500.0, True, T0 + timedelta(seconds=_t))
_wd_unit.evaluate(-1000.0, 1500.0, True, T0 + timedelta(seconds=_t))
print(f" site: {_wd_site.reason!r}")
print(f" unit: {_wd_unit.reason!r}")
check("⚠️ THE SITE INSTANCE'S SENTENCE IS BYTE-IDENTICAL TO THE ONE THAT "
"SHIPPED. It is what an operator reads on the status page, what goes in "
"the action log and what is persisted to /data/state.json, so it is "
"asserted against the literal rather than against a format string",
_wd_site.reason == (
"SIGN_DISAGREEMENT: commanded -1000 W against a meter reading "
"+1500 W for 180 s continuously (limits: both over 300 W, held over "
"180 s)"))
check("...and the per-unit instance names what it ACTUALLY compared instead. "
"'against a meter reading' there would be a false statement about which "
"instrument produced the evidence, on the one line somebody woken by "
"this alarm reads first",
AGAINST_OWN_POWER in _wd_unit.reason
and AGAINST_METER not in _wd_unit.reason
and "+1500 W" in _wd_unit.reason)
# ⚠️ `arm()` IS CONFIGURATION AND MUST NOT BE STATE. `Controller.cycle()` calls
# it on EVERY cycle, so a version that cleared `disagree_since` there would
# restart the window every cycle, never accumulate a hold, and leave the check
# permanently green with nothing saying so. Driven rather than argued: open a
# window, re-arm mid-episode, and see the window still standing.
_rearm = SignWatchdog(_ucfg, against=AGAINST_OWN_POWER)
_rearm.evaluate(-1000.0, 1500.0, True, T0)
_rearm.evaluate(-1000.0, 1500.0, True, T0 + timedelta(seconds=5))
_rearm.arm(_ucfg, AGAINST_OWN_POWER)
check("re-arming mid-episode leaves the window exactly where it was",
_rearm.disagree_since == T0)
for _t in range(10, 200, 5):
_rearm.arm(_ucfg, AGAINST_OWN_POWER)
_rearm.evaluate(-1000.0, 1500.0, True, T0 + timedelta(seconds=_t))
check("...so a detector re-armed on every one of 38 cycles still trips at its "
"85 s hold, which is what says `arm()` did not quietly disarm it",
_rearm.tripped)
print()
print("2. CFG-03: config.yaml is the source of truth and there is ONE copy")
_sign_from_yaml = {k: int(v) for k, v in
_re.findall(r"^ ((?:unit|site)_sign_disagree_\w+):\s*(\d+)\s*$",
_cfg_opts, _re.M)}
print(f" config.yaml options: {_sign_from_yaml}")
print(f" SIGN_OPTION_DEFAULTS: {SIGN_OPTION_DEFAULTS}")
check("all four new options are declared in config.yaml's options: block and "
"the in-code mirror agrees key for key and value for value - the "
"Dockerfile does not put config.yaml in the image, so the running add-on "
"cannot read it and this is where the two can drift",
_sign_from_yaml == SIGN_OPTION_DEFAULTS and len(_sign_from_yaml) == 4)
_sign_bounds = {m[0]: (int(m[1]), int(m[2])) for m in
_re.findall(r"^ ((?:unit|site)_sign_disagree_\w+):"
r"\s*int\((\d+),(\d+)\)\s*$",
_cfg_schema, _re.M)}
print(f" schema bounds: {_sign_bounds}")
check("...and ALL FOUR have a schema range, so the Supervisor refuses an "
"out-of-range value rather than the add-on discovering it at runtime",
set(_sign_bounds) == set(SIGN_OPTION_DEFAULTS))
check("the shipped defaults sit inside their own schema ranges",
all(_sign_bounds[k][0] <= v <= _sign_bounds[k][1]
for k, v in SIGN_OPTION_DEFAULTS.items()))
check("⚠️ THE THRESHOLD IS `saturation_w / 2`, WHICH IS WHERE THE CONTROL LAW "
"ITSELF ALREADY DRAWS THE LINE. This check gates on BOTH sides, and the "
"two are OPPOSED, so the smallest command-versus-power gap it can ever "
"see is 2 x min_w. At 250 W that is exactly the 500 W at which "
"`compute()`'s own anti-windup test concludes the inverter is not "
"following and freezes the integrator - so this watchdog cannot fire on "
"a divergence the shipped loop calls ordinary tracking, and the two "
"guards meet at a number instead of overlapping in a band nobody drew",
SIGN_OPTION_DEFAULTS["unit_sign_disagree_w"] * 2
== _Tuning().saturation_w)
check("...and the window clears its three bounds: above the firmware's own "
"30 s link veto (`inverter.LINK_STALE_S` - inside it, a unit not "
"executing its share is a LINK fault the add-on already answers by "
"dropping it from the split), and UNDER HALF the 180 s the site check "
"takes, because the per-unit conclusion needs nothing to net out before "
"it is complete",
SIGN_OPTION_DEFAULTS["unit_sign_disagree_s"] > LINK_STALE_S
and SIGN_OPTION_DEFAULTS["unit_sign_disagree_s"] < 180 / 2)
check("⚠️ AND NEITHER IS A ROUND NUMBER. 250 W is read off `saturation_w`; "
"85 s is 8.5x the worst legitimate opposition 1890 two-inverter runs "
"produce (10.0 s) and 2.8x the firmware veto. 60, 75, 90, 100, 120, 180 "
"and 300 are all values a guess would have landed on",
SIGN_OPTION_DEFAULTS["unit_sign_disagree_s"]
not in (30, 60, 75, 90, 100, 120, 180, 300))
# ⚠️ AND THE SITE PAIR IS TWO DECISIONS, NOT ONE, WHICH IS WHY BOTH ARE PINNED.
# The THRESHOLD is carried from the shipped pair because its argument is about
# the meter's noise floor and the deadband - a property of ONE P1 and of
# `Tuning.deadband_w`, neither of which the roster touches. The WINDOW is NOT
# carried, because the transient it rejects is the loop crossing zero across a
# rail that IS the roster sum: measured at 110 s with one inverter and 215 s
# with two, on the same stimuli, against a 180 s hold. That is a carried number
# justified and a carried number refused, in the same pair.
check("the SITE threshold is the shipped one, carried - the noise argument "
"behind 300 W is about the meter and the deadband, and a second inverter "
"changes neither",
SIGN_OPTION_DEFAULTS["site_sign_disagree_w"] == 300)
check("⚠️ ...AND THE SITE WINDOW IS LONGER THAN THE SHIPPED 180 s, WHICH IS THE "
"MEASUREMENT REFUSING TO BE CARRIED. At two inverters the worst "
"legitimate opposition the rig produces is 215.0 s - reachable at "
"`gain: 0.05`, `slew_w: 50` and a 2.2 kW load reversal, all inside what "
"config.yaml accepts - so 180 s would have shipped a false trip on the "
"check with the best record in the safety table. 325 s is 1.51x that",
SIGN_OPTION_DEFAULTS["site_sign_disagree_s"] > 215.0
and SIGN_OPTION_DEFAULTS["site_sign_disagree_s"] < 215.0 * 2)
check("...and it is still inside the 900 s quarter that bills the capacity "
"tariff, and inside every episode the 2026-08-20 fault day contains - "
"its four clusters run 16, 30, 77 and 77 minutes",
SIGN_OPTION_DEFAULTS["site_sign_disagree_s"] < 900
and SIGN_OPTION_DEFAULTS["site_sign_disagree_s"] < 16 * 60)
check("...and it is deliberately not 300 or 360, the two round numbers next to "
"it",
SIGN_OPTION_DEFAULTS["site_sign_disagree_s"] not in (300, 360, 600, 900))
print()
print("3. AC 1: ONE unit drives the wrong way, and ONLY that unit trips")
# ⚠️ THE PLANT IS MODELLED AND SAYS SO. `_SignHass` is two inverters that obey
# INSTANTLY - no ramp, no lag - and one meter that is `load - sum(measured)`.
# That is deliberately not a claim about transients: the worst legitimate
# opposition a real ramp and lag can produce is measured on admin/goodwe's
# two-master rig, over 1890 closed-loop runs, and recorded in
# DECISIONS/SAFETY-30-per-unit-sign.md. What this fixture is for is the
# ATTRIBUTION - which detector concludes what, from which instrument - and for
# that an instant plant is the honest model, because it removes the one
# mechanism that could make a unit look reversed without being reversed.
class _SignHass(_Hass):
"""Two inverters, one meter, and a chosen subset that does the OPPOSITE.
`reversed_units` are roster indices whose measured power is the NEGATION of
what this controller told them - HW-01's wedged power stage, an inverted
driver, a unit executing a stale share backwards. Everything else obeys.
"""
def __init__(self, load_w=2000.0, reversed_units=()):
super().__init__()
self.load_w = float(load_w)
self.reversed_units = set(reversed_units)
self.ctrl = None
def measured(self):
if self.ctrl is None:
return [0.0, 0.0]
out = []
for i, inv in enumerate(self.ctrl.inverters):
w = inv.last_write if inv.last_write is not None else inv.target
w = float(w or 0.0)
out.append(-w if i in self.reversed_units else w)
return out
async def number(self, entity, invert=False):
m = self.measured() + [0.0, 0.0]
return {"sensor.grid": self.load_w - sum(m[:2]),
"sensor.soc": 50.0, "sensor.soc_b": 50.0,
"sensor.batt": m[0], "sensor.batt_b": m[1]}.get(entity)
def _sign_site(hass, store=None, **over):
"""A two-inverter Controller, with SAFETY-31 deliberately given a long window.
⚠️ THE LONG `circulation_hold_s` IS ISOLATION, NOT A CLAIM THAT SAFETY-31 IS
SILENT HERE. A unit reversing while its sibling pulls the other way IS
circulating power and SAFETY-31 sees it - part 4 asserts exactly that, and
what each guard concludes about it. What must not happen is the SITE being
stopped by circulation while this section is trying to grade a PER-UNIT
attribution, because then every "only B is parked" check below would be
measuring the wrong guard.
⚠️ THE THREE THRESHOLD PAIRS ARE PAIRWISE DISTINCT IN BOTH COORDINATES,
AND THAT IS LOAD-BEARING RATHER THAN COSMETIC. This fixture is the only
place in the suite where `_arm_sign_watchdogs()` runs at a roster of two, so
it is the only place that can see WHICH pair reaches WHICH detector. The
first version of it set every hold to 1 s and two of the three magnitude
gates to 300 W; a detector armed with the shipped pair, or with the site
pair, or with the right magnitude and the wrong hold, then compared EQUAL
to the pair it was supposed to have, and the wiring check below would have
passed on three of those four arrangements. That is MUTATION.md's third
failure mode written into a fixture - a check comparing two things that
happen to be equal cannot see the difference it names - and the
precondition beside the check is what stops a later edit collapsing them
again without a red line.
Anyone changing a hold here: the PER-UNIT hold is what part 3's trip timing
is read against, and the SITE hold must still expire inside `_drive()`'s
window (14 cycles x 0.2 s), which part 5 depends on.
"""
opts = {"setpoint_entity": "number.sp", "meter_entity": "sensor.grid",
"soc_entity": "sensor.soc", "batt_entity": "sensor.batt",
"meter_source": "off", "max_w": 2000, "gain": 0.6,
"sign_disagree_w": 300, "sign_disagree_s": 3,
"unit_sign_disagree_w": 250, "unit_sign_disagree_s": 1,
"site_sign_disagree_w": 320, "site_sign_disagree_s": 2,
"circulation_hold_s": 3600, "auto_start": True}
opts.update(over)
c = Controller(opts, hass, store if store is not None else _MemStore(),
_P(), stop=asyncio.Event())
b = Inverter("inverter-b", setpoint_entity="number.sp_b",
soc_entity="sensor.soc_b", batt_entity="sensor.batt_b",
max_w=B_RATING_HYPOTHESIS, store=c.store)
b.dev_min, b.dev_max = -B_RATING_HYPOTHESIS, B_RATING_HYPOTHESIS
c.inverters.append(b)
c.auto = True
for inv in c.inverters:
if inv.dev_min is None:
inv.dev_min, inv.dev_max = -5000.0, 5000.0
hass.ctrl = c
return c
async def _drive(c, n=14, dt=0.2, watch=None):
seen = []
for _ in range(n):
await c.read_inputs()
await c.cycle()
if watch is not None:
seen.append(watch(c))
await asyncio.sleep(dt)
return seen
async def _one_reversed():
hass = _SignHass(load_w=2000.0, reversed_units={1})
c = _sign_site(hass)
seen = await _drive(c, watch=lambda c: dict(
grid=c.grid,
stood=[i.last_write if i.last_write is not None else i.target
for i in c.inverters],
meas=[i.batt for i in c.inverters],
a=c.inverters[0].sign_wd.tripped, b=c.inverters[1].sign_wd.tripped,
site=c.site_sign_wd.tripped))
return c, hass, seen
_c30, _h30, _s30 = asyncio.run(_one_reversed())
_a30, _b30 = _c30.inverters
# ⚠️ THE WIRING PIN FOR AC 5, AND IT IS A DIFFERENT CLAIM FROM EVERY
# BEHAVIOURAL CHECK IN THIS SECTION. The rest of part 3 grades what the
# detectors CONCLUDED; this grades WHICH THRESHOLDS THEY WERE HANDED, and the
# gap between the two was live. Replacing
# `arm(self.sign_cfg if one else self.unit_sign_cfg, ...)` with
# `arm(self.sign_cfg, ...)` in `Controller._arm_sign_watchdogs()` throws the
# re-derived per-unit pair away and arms every unit with SAFETY-02's
# meter-facing one - and left ALL 19 FILES GREEN. Nothing at a roster of two
# read `inv.sign_wd.cfg` at all; the one place that does is part 7's N=1 arm,
# which that edit PRESERVES.
#
# WHY THE TRIPS BELOW CANNOT CATCH IT, said here because "the suite is green"
# and "nothing was wrong" are the two readings this comment exists to separate.
# B is reversed at ~2000 W - far over 250 W and far over 300 W - and every hold
# in this fixture is short enough to expire inside `_drive()`. A fault that is
# big enough and long enough to trip under BOTH pairs is evidence about
# neither. Only an equality on the pair itself can tell them apart.
#
# WHAT THE SURVIVOR COSTS IN THE FIELD, at config.yaml's own defaults: the
# per-unit check runs at `sign_disagree_s` 180 s instead of the re-derived 85 s
# - 2.1x slower to park a reversed inverter - and its gate moves from 250 W to
# 300 W, so a unit reversing a share between the two becomes invisible, while
# watchdog.SIGN_OPTION_DEFAULTS, config.yaml and DECISIONS section 3 all go on
# describing a pair the running code does not use.
_UNIT_PAIR = (250.0, 1.0) # `unit_sign_disagree_w` / `_s` - one per unit
_SITE_PAIR = (320.0, 2.0) # `site_sign_disagree_w` / `_s` - one per site
_SHIPPED_PAIR = (300.0, 3.0) # `sign_disagree_w` / `_s` - N=1 only
_armed = {inv.name: (inv.sign_wd.cfg.min_w, inv.sign_wd.cfg.hold_s)
for inv in _c30.inverters}
_armed_site = (_c30.site_sign_wd.cfg.min_w, _c30.site_sign_wd.cfg.hold_s)
print(f" armed per unit: {_armed}")
print(f" armed site: {_armed_site} "
f"shipped pair, wanted by NOBODY here: {_SHIPPED_PAIR}")
check("PRECONDITION: the three pairs are pairwise distinct IN BOTH "
"COORDINATES, so an equality below can fail on the magnitude alone, on "
"the hold alone, or on both. This fixture originally gave all three the "
"same 1 s hold and two of them the same 300 W gate, and a mis-wiring "
"that delivered the right magnitude with the wrong hold would then have "
"compared EQUAL to the pair it was supposed to have",
len({_UNIT_PAIR, _SITE_PAIR, _SHIPPED_PAIR}) == 3
and len({p[0] for p in (_UNIT_PAIR, _SITE_PAIR, _SHIPPED_PAIR)}) == 3
and len({p[1] for p in (_UNIT_PAIR, _SITE_PAIR, _SHIPPED_PAIR)}) == 3)
check("PRECONDITION: and all three are pairs this Controller actually holds, "
"so the literals below are the fixture's own numbers and not three "
"constants that agree with nothing. In particular the shipped pair is "
"still constructed and still a reachable mis-wiring target",
(_c30.unit_sign_cfg.min_w, _c30.unit_sign_cfg.hold_s) == _UNIT_PAIR
and (_c30.site_sign_cfg.min_w, _c30.site_sign_cfg.hold_s) == _SITE_PAIR
and (_c30.sign_cfg.min_w, _c30.sign_cfg.hold_s) == _SHIPPED_PAIR)
check("⚠️ AC 5 WIRING. EVERY per-unit detector on this roster of two is armed "
"with the PER-UNIT pair - 250 W / 1 s here, `unit_sign_disagree_w` and "
"`unit_sign_disagree_s` in the field. Graded against LITERALS rather "
"than against `_c30.unit_sign_cfg`, which cannot tell 'the per-unit "
"pair' from 'whatever this Controller was handed'. This is the line that "
"goes red when `_arm_sign_watchdogs()` stops choosing between the two "
"pairs and hands every unit the site-facing one",
len(_armed) == 2 and all(p == _UNIT_PAIR for p in _armed.values()))
check("...and the SITE detector is armed with the SITE pair - 320 W / 2 s "
"here - which differs from the per-unit pair AND from the shipped pair "
"in both coordinates, so this equality is not carried by a number the "
"fixture happens to hold twice",
_armed_site == _SITE_PAIR)
check("...and each detector is pointed at its own INSTRUMENT as well as at its "
"own thresholds: the units at their own 0x0518 power, the site at the "
"meter. They are two arguments of one call and a pin on one is not a pin "
"on the other",
all(inv.sign_wd.against == AGAINST_OWN_POWER for inv in _c30.inverters)
and _c30.site_sign_wd.against == AGAINST_METER)
_mid = _s30[3]
print(f" cycle 3: stood {[round(x) for x in _mid['stood']]} "
f"measured {[round(x) for x in _mid['meas']]} "
f"meter {_mid['grid']:+.0f} W")
check("PRECONDITION: the stimulus is not vacuous - inverter B really is doing "
"the OPPOSITE of what it was told, and inverter A really is obeying. "
"Without this line every claim below could be true of a fixture in which "
"nothing was reversed",
_mid["stood"][1] * _mid["meas"][1] < 0
and abs(_mid["meas"][1]) > 250
and _mid["stood"][0] * _mid["meas"][0] > 0)
check("⚠️ AC 1. THE REVERSED UNIT TRIPS AND ITS SIBLING DOES NOT. The evidence "
"is B's own 0x0518 power against B's own share, so it is a fact about B "
"and nothing about A - which is what makes parking one of them the right "
"remedy rather than an attribution nobody measured",
_b30.sign_wd.tripped and not _a30.sign_wd.tripped)
check("...and the reason names the instrument, so the operator is sent to the "
"right one", AGAINST_OWN_POWER in _b30.sign_wd.reason)
print(f" {_b30.sign_wd.reason!r}")
# ⚠️ READ OFF THE ALLOCATION RECORD, NOT OFF `Controller.reason`. At a roster
# of two `reason` is the SITE resolution's line - "loop -> 2000 W" - because
# B's failsafe is a UNIT claim resolved on B's own actuator and never competes
# for the site winner. That is the DEV-02 split working, and a check that
# grepped the site line for "failsafe" would go green only while the roster was
# one. The per-unit record is where the pin is stated, and it names its source.
_bplan = [pl for pl in (_c30.allocation.plans if _c30.allocation else ())
if pl.unit == "inverter-b"]
print(f" allocation: {_c30.allocation.explain()}")
print(f" inverter-b plan: {_bplan[0].why if _bplan else '(none)'}")
check("...B is commanded 0 W by its OWN `failsafe` claim, named in the "
"allocation record, rather than by some other guard happening to stop it",
_b30.target == 0.0 and _bplan and "failsafe" in _bplan[0].why)
check("...and A IS STILL COMMANDING. One inverter in trouble stops that "
"inverter; the site goes on running on the one that works, which is what "
"a UNIT-scoped claim buys and what a site-wide stop would have thrown "
"away", _a30.target != 0.0)
check("...the latch is under B's OWN namespaced key, and A's is untouched",
_c30.store.data.get(unit_key("inverter-b", "sign_tripped")) is True
and _c30.store.data.get(TRIPPED_KEY) is False)
print()
print("4. AC 2: and it is caught EVEN THOUGH THE METER SAYS EVERYTHING IS FINE")
# ⚠️ THIS IS THE CHECK THE WHOLE CARD IS FOR, so it is built the way MUTATION.md
# says to build one: the claim is "the new comparison sees what the old one
# structurally cannot", and the OLD comparison is RUN, on the same samples, and
# its blindness is asserted as its own precondition and PRINTED - not argued.
_old_view = [disagrees(s["stood"][1], s["grid"], True, _scfg) for s in _s30]
_site_view = [disagrees(sum(s["stood"]), s["grid"], True, _scfg) for s in _s30]
print(f" the SHIPPED comparison (B's command vs the meter) was true on "
f"{sum(_old_view)} of {len(_old_view)} cycles")
print(f" the SITE comparison (sum of commands vs the meter) on "
f"{sum(_site_view)} of {len(_site_view)}")
check("⚠️ THE COMPARISON THAT SHIPPED IS FALSE ON EVERY SINGLE CYCLE HERE. B is "
"commanded to discharge and the house is importing, so B's command and "
"the meter AGREE all the way through - the meter cannot tell that the "
"watts arriving are inverter A's while B does the opposite. This is not "
"a threshold set too high: the summation destroys the evidence and no "
"tuning recovers it",
not any(_old_view))
check("...and the SITE comparison is blind for the same reason, which is why "
"it is kept as a SEPARATE question rather than offered as the fix",
not any(_site_view))
check("...while the shipped SITE watchdog stayed correctly silent, so nothing "
"here is a site-level trip wearing a per-unit label",
not _c30.site_sign_wd.tripped
and _c30.store.data.get("site_sign_tripped") is not True)
check("⚠️ AND SAFETY-31 SEES SOMETHING TOO, WHICH IS NOT A DUPLICATE AND IS "
"SAID HERE RATHER THAN LEFT FOR SOMEBODY TO DISCOVER. A unit reversing "
"while its sibling pulls the other way IS power going round a loop, so "
"`circulating_w()` is over its threshold on this fixture - it is only "
"the long window this fixture gives it that keeps it from stopping the "
"site. The two guards CONCLUDE different things: circulating power is "
"defined over the roster and names no offender, while this one names B "
"and parks B. Neither is the other with a different threshold",
max(circulating_w(s["meas"]) for s in _s30) > _c30.circ_wd.cfg.min_w
and not _c30.circ_wd.tripped
and _c30.site_circulating_w == 0.0)
print(f" circulating power over the episode: "
f"{[round(circulating_w(s['meas'])) for s in _s30]}")
check("...and it falls back to ZERO the moment B is parked, which is the "
"remedy working rather than the sensor going quiet: a unit held at 0 W "
"cannot be half of a loop. It is asserted on the EPISODE rather than on "
"the end state for exactly that reason",
_s30[-1]["meas"][1] == 0.0)
print()
print("5. AC 3: the SITE question survives, separately, and catches whole-site")
print(" reversal - which no per-unit check can see")
# ⚠️ THE FAULT HERE IS THE ONE THE 2026-08-20 CAPTURE CONTAINS, and it is
# INVISIBLE to every per-unit comparison by construction: both inverters are
# doing EXACTLY what they were told. What is wrong is the number they were told.
# A negative gain is the minimal faithful model of it - an inverted polarity, a
# wound-up integrator, a strategy on the wrong side of the meter - and it is not
# a reachable configuration (`gain` is float(0.05,1.0)), which is the point: it
# stands in for a code regression, which is the case a check can hold the line on.
async def _site_reversed():
hass = _SignHass(load_w=2000.0, reversed_units=())
c = _sign_site(hass, gain=-0.6)
seen = await _drive(c, watch=lambda c: dict(
grid=c.grid, site=c.site_sign_wd.tripped,
stood=[i.last_write if i.last_write is not None else i.target
for i in c.inverters],
meas=[i.batt for i in c.inverters]))
return c, hass, seen
_c31, _h31, _s31 = asyncio.run(_site_reversed())
# ⚠️ THE LAST SAMPLE BEFORE THE TRIP, NOT A FIXED INDEX. After the trip every
# command is 0 W by design, so a fixed index reads the REMEDY rather than the
# fault and the precondition below would be asserting that a stopped site is
# stopped. This is `Run.p2p_between`'s lesson from the rig, in a unit test: name
# the window rather than inferring it.
_pre31 = [x for x in _s31 if not x["site"]]
_mid31 = _pre31[-1]
print(f" last cycle before the trip: stood "
f"{[round(x) for x in _mid31['stood']]} "
f"measured {[round(x) for x in _mid31['meas']]} "
f"meter {_mid31['grid']:+.0f} W")
check("PRECONDITION: EVERY inverter is obeying - each unit's measured power "
"agrees in sign with its own share - and the site is still charging into "
"an importing house. Without this the site trip below could be a "
"per-unit fault in disguise",
all(s * m > 0 for s, m in zip(_mid31["stood"], _mid31["meas"]))
and sum(_mid31["stood"]) * _mid31["grid"] < 0)
check("⚠️ AC 3. THE SITE WATCHDOG TRIPS AND NOT ONE PER-UNIT WATCHDOG DOES. "
"That asymmetry is the card: the fault is in the number the site was "
"asking for, which no single inverter owns, and every unit is innocent "
"of it",
_c31.site_sign_wd.tripped
and not any(i.sign_wd.tripped for i in _c31.inverters))
print(f" {_c31.site_sign_wd.reason!r}")
check("...and it stops the WHOLE site, through the `sign_site` claim by name "
"rather than by some other guard happening to stop it",
_c31.target == 0.0 and "sign_site" in _c31.reason
and all(i.target == 0.0 for i in _c31.inverters))
check("...the latch is FLAT - `site_sign_tripped`, beside SAFETY-31's pair - "
"and NOT mirrored onto the pre-DEV-02 `sign_tripped`, which is the "
"rollback view of a UNIT latch. A downgraded add-on drives exactly one "
"actuator; handing it a latch taken on evidence about a roster it cannot "
"have would be a trip it can never explain",
_c31.store.data.get("site_sign_tripped") is True
and "SIGN_DISAGREEMENT" in str(_c31.store.data.get("site_sign_reason"))
and _c31.store.data.get("sign_tripped") is not True)
check("the status page carries a SITE row of its own, and the banner says the "
"site's TOTAL command opposed the meter rather than naming an inverter - "
"naming one would send an operator to swap the wrong box",
any("SITE SIGN-DISAGREEMENT FAILSAFE" in ck["text"]
for ck in _c31.checks())
and "total command" in _c31.status_payload()["banner"])
_pub30 = {}
_c31.mqtt = type("P", (), {"enabled": True,
"publish": lambda _s, v: _pub30.update(v),
"close": lambda _s: None})()
_c31.publish()
check("...and the controller-status sensor reports the failsafe state, so a "
"site parked at 0 W cannot publish 'running'",
_pub30.get("status") == "failsafe")
# The latch, and the one way out of it.
_was31 = _c31.site_sign_wd.tripped
asyncio.run(_c31.handle_action("auto_toggle")) # stop
check("pressing STOP clears nothing - the condition always goes away on its "
"own, because the trip commands 0 W and a site at zero cannot oppose "
"anything", _c31.site_sign_wd.tripped and _was31)
asyncio.run(_c31.handle_action("auto_toggle")) # the deliberate start
check("pressing START clears the latch in memory and on disk together, which "
"is the same deliberate human act that clears every other latch here",
not _c31.site_sign_wd.tripped
and _c31.store.data.get("site_sign_tripped") is False
and _c31.store.data.get("site_sign_reason") == "")
async def _site_restart(store):
hass = _SignHass(load_w=2000.0)
c = _sign_site(hass, store=store, gain=-0.6)
await _drive(c, n=3, dt=0.05)
return c
_store31 = _MemStore()
_store31.data["site_sign_tripped"] = True
_store31.data["site_sign_reason"] = "SIGN_DISAGREEMENT: (from before the restart)"
_c31r = asyncio.run(_site_restart(_store31))
check("⚠️ AND IT SURVIVES A RESTART, which is the only thing that makes it a "
"latch rather than a pause. 'Restart the add-on' is the first thing an "
"operator staring at a FAILSAFE banner tries, and a site that came back "
"armed would drive straight into the fault it had just stopped for",
_c31r.site_sign_wd.tripped and _c31r.target == 0.0
and "sign_site" in _c31r.reason)
print()
print("6. AC 4: two inverters commanded OPPOSITE SIGNS by a legitimate")
print(" allocation, and nothing trips")
# ⚠️ THIS IS THE FALSE TRIP THE CARD OPENS WITH, AND IT IS DRIVEN THROUGH THE
# SHIPPED PREDICATE RATHER THAN DESCRIBED. Inverter A charging 1500 W while
# inverter B discharges 1500 W into a 1500 W import is an ordinary two-inverter
# arrangement - a per-unit maintenance charge on A while the loop drives B, or
# SAFETY-25's floor pinning one unit. Nothing is faulty. Before this card the
# meter convicted A on it.
_A_CMD, _B_CMD, _METER = -1500.0, 1500.0, 1500.0
_wd_a = SignWatchdog(_ucfg, against=AGAINST_OWN_POWER)
_wd_b = SignWatchdog(_ucfg, against=AGAINST_OWN_POWER)
_sitecfg = SignWatchdogConfig(
min_w=float(SIGN_OPTION_DEFAULTS["site_sign_disagree_w"]),
hold_s=float(SIGN_OPTION_DEFAULTS["site_sign_disagree_s"]))
_wd_s = SignWatchdog(_sitecfg)
for _t in range(0, 1200, 5):
_wd_a.evaluate(_A_CMD, _A_CMD, True, T0 + timedelta(seconds=_t))
_wd_b.evaluate(_B_CMD, _B_CMD, True, T0 + timedelta(seconds=_t))
_wd_s.evaluate(_A_CMD + _B_CMD, _METER, True, T0 + timedelta(seconds=_t))
print(f" A commanded {_A_CMD:+.0f} W and measured {_A_CMD:+.0f} W; "
f"B {_B_CMD:+.0f} / {_B_CMD:+.0f}; meter {_METER:+.0f} W")
check("PRECONDITION: THE COMPARISON THAT SHIPPED CONVICTS INVERTER A HERE, on "
"every one of these samples. A is commanded to CHARGE while the meter "
"reads IMPORT - opposed, both far over 300 W, and it would have held for "
"20 minutes. That is the false trip, and it is asserted rather than "
"asserted about",
disagrees(_A_CMD, _METER, True, _scfg) is True)
check("⚠️ AC 4. NEITHER UNIT TRIPS UNDER THE NEW COMPARISON, over 20 minutes at "
"an 85 s hold. Each one is doing exactly what it was told, and its own "
"power sensor is the instrument that can say so",
not _wd_a.tripped and not _wd_b.tripped)
check("...and the SITE watchdog does not trip either, because the shares SUM "
"to zero and a 0 W total is not a wrong-direction command - the "
"magnitude gate that has always been there, now applied to the quantity "
"it belongs on", not _wd_s.tripped and _wd_s.disagree_since is None)
# ⚠️ AND THE COUNTERFACTUAL, BECAUSE "nothing tripped" IS ALSO WHAT A DISARMED
# GUARD REPORTS. Same site, same meter, but the shares no longer cancel: a NET
# -1600 W of charge into a +1500 W import is the site genuinely driving the
# wrong way, and it must trip. Without this line the check above passes
# identically against a site watchdog that was never wired to anything.
_wd_s_net = SignWatchdog(_sitecfg)
for _t in range(0, 600, 5):
_wd_s_net.evaluate(-1600.0, _METER, True, T0 + timedelta(seconds=_t))
check("...and the site question is not thereby disarmed: the same site, "
"commanded a NET -1600 W into the same import, DOES trip",
_wd_s_net.tripped)
print()
print("7. AC 8: at ONE inverter this card changes NOTHING, and 'nothing' is")
print(" asserted rather than assumed")
async def _n1_sign():
hass = _SignHass(load_w=2000.0)
opts = {"setpoint_entity": "number.sp", "meter_entity": "sensor.grid",
"soc_entity": "sensor.soc", "batt_entity": "sensor.batt",
"meter_source": "off", "max_w": 2000, "gain": -0.6,
"sign_disagree_w": 300, "sign_disagree_s": 1,
"auto_start": True}
c = Controller(opts, hass, _MemStore(), _P(), stop=asyncio.Event())
c.auto = True
c.dev_min, c.dev_max = -5000.0, 5000.0
hass.ctrl = c
# ⚠️ A CALL COUNTER, NOT AN OBSERVATION OF THE DETECTOR'S STATE, AND A
# MUTATION IS WHY. The first version of the check below read
# `site_sign_wd.disagree_since is None` - and arming the site detector at
# N=1 as well left this suite GREEN, because the unit detector trips first
# at its 1 s hold, parks the inverter, and the site's standing command is
# then 0 W: the site predicate is false, so its window is never opened and
# "never evaluated" reads identically to "evaluated and found nothing". That
# is MUTATION.md's third failure mode - the assertion ran, the sentence was
# true, and the data could not reach the failure it names. A counter can
# only be zero for the one reason.
_n1_calls = []
_orig_eval = c.site_sign_wd.evaluate
def _spy(*a, **kw):
_n1_calls.append(a)
return _orig_eval(*a, **kw)
c.site_sign_wd.evaluate = _spy
await _drive(c, n=12, dt=0.2)
return c, _n1_calls
_c1s, _n1_site_calls = asyncio.run(_n1_sign())
_inv1 = _c1s.inverters[0]
print(f" roster={_c1s.units} armed against {_inv1.sign_wd.against!r} "
f"at {_inv1.sign_wd.cfg.min_w:.0f} W / {_inv1.sign_wd.cfg.hold_s:.0f} s")
check("the single unit is graded against THE METER, at the SHIPPED thresholds "
"- §6.2's identity, and the whole compatibility promise of this card. "
"The unit's own power is not consulted, and `unit_sign_disagree_*` does "
"nothing on a one-inverter install",
_inv1.sign_wd.against == AGAINST_METER
and (_inv1.sign_wd.cfg.min_w, _inv1.sign_wd.cfg.hold_s) == (300.0, 1.0))
check("...it trips, on the same evidence and through the same UNIT `failsafe` "
"claim as before, commanding 0 W",
_inv1.sign_wd.tripped and _c1s.target == 0.0
and "failsafe" in _c1s.reason and "sign_site" not in _c1s.reason)
check("...the reason sentence still says 'a meter reading', word for word - it "
"is what an operator reads, what the action log carries and what is "
"persisted, so it is graded on the literal",
"against a meter reading" in _inv1.sign_wd.reason)
check("...the latch is under the same namespaced key with the same flat "
"rollback mirror, and SAFETY-30's site pair is untouched at its default. "
"A one-inverter install's state file is byte-identical to what it was",
_c1s.store.data.get(TRIPPED_KEY) is True
and _c1s.store.data.get("sign_tripped") is True
and _c1s.store.data.get("site_sign_tripped") is False
and _c1s.store.data.get("site_sign_reason") == "")
print(f" site detector evaluated {len(_n1_site_calls)} times over "
f"12 cycles")
check("⚠️ ...AND THE SITE DETECTOR IS NEVER ARMED. Not 'did not trip', and not "
"'its window stayed closed' - NEVER CALLED, counted on the object over "
"twelve cycles. That is what says the second detector is ABSENT at a "
"roster of one rather than merely quiet, and arming it would be two "
"latches and two banners for one fault with a false-positive record "
"measured on ONE comparison spent twice",
_n1_site_calls == []
and not _c1s.site_sign_wd.tripped
and _c1s.site_sign_wd.disagree_since is None)
check("...the banner is the sentence that shipped",
_c1s.status_payload()["banner"]
== "FAILSAFE — the command opposed the meter. Inverter held at 0 W.")
check("...the status page carries ONE sign row, with no unit tag and no site "
"row - an extra row for an identically-duplicated question is a page "
"advertising two guards where the install has one",
len([ck for ck in _c1s.checks()
if "sign watchdog" in ck["text"] or "SIGN-DISAGREEMENT" in ck["text"]
]) == 1
and not any("site sign" in ck["text"].lower() for ck in _c1s.checks()))
check("...and no `sign_site` claim can be filed at a roster of one, so the "
"arbitration is byte-identical to what shipped",
"sign_site" not in _c1s.reason)
print()
if fails:
print(f"{len(fails)} FAILED: {', '.join(fails)}")