Files
goodwe-addon/goodwe_controller/DECISIONS/SAFETY-31-circulating-power.md
T
adminandClaude Opus 5 7ef3d23245
checks / unit (pull_request) Successful in 4m28s
SAFETY-31: measure circulating power, which nets to zero at the meter
Inverter A charging at 3 kW while inverter B discharges at 3 kW nets ~0 W at
P1. The site reads idle while both packs are cycled through two round-trip
conversion losses continuously, for no benefit, and the wear is real.

NOTHING EXISTING CAN SEE IT, AND THAT IS STRUCTURAL. Every guard in this
add-on is a consumer of the NET figure at the grid connection: SAFETY-02's
sign watchdog gates on abs(grid_w) > 300 and the gate never opens at a netted
zero; the peak guard and the capacity tariff both bill grid exchange.
Circulating power is precisely the component of per-inverter power that the
sum destroys, so no threshold on any existing detector could reach it.

WHAT ALLOC-01 ALREADY COVERED, AND IS NOT RE-IMPLEMENTED HERE. Its AC 7
shipped the sign-coherence gate (allocator.REFUSE_SIGN, graded by
test_alloc.py part 6): an allocation whose shares carry two signs is REFUSED,
both units named, no shares returned. That is the COMMAND half - the allocator
can no longer command circulating power. This commit is the MEASUREMENT half:
circulation the allocator NEVER COMMANDED.

  app/circulation.py   the sensor and the trip
  main.py              site_circulating_w, the evaluation, the claim, the alarm
  arbiter.py           `circulation` classified SCOPE_SITE, with the argument
  store.py             a flat, site-level durable latch
  mqtt.py              sensor.goodwe_circulating_power
  config.yaml          two options, two schema ranges, CFG-03 discipline

THE SENSOR: sum(|Bi|) - |sum(Bi)|. Zero when the units pull the same way,
2*min(|Ba|,|Bb|) when they oppose. DERIVED FROM MEASURED PER-INVERTER POWER -
Inverter.batt, register 0x0518 - and never from the allocation record. The
cheap version would be simpler and worth nothing: ALLOC-01's gate already
guarantees the record is sign-coherent, so a record-derived sensor reads zero
BY CONSTRUCTION and can only confirm that the allocator agrees with itself,
which is the third failure mode MUTATION.md names. What this catches is a unit
ignoring or lagging its share, a stale share still being executed, a
maintenance or SAFETY-25 pin holding one unit while the loop drives the other,
or HW-01's power stage running its last instruction for 6 h 25 m with the
comms dead and the allocation record impeccable throughout. batt became
per-instance in DEV-02; before that this could not have been written.

VERIFIED BY MAKING ONE UNIT IGNORE ITS SHARE, on the TEST-19 two-master rig:

  S = -2500 W   shares a = -1250 W  b = -1250 W   <- record ok, ONE sign, sums
  MEASURED      A = -1250 W  B = +1000 W          <- hardware
  sensor over the RECORD     :    0 W
  sensor over MEASURED power : 2000 W  -> trips at 190 s

and the card's own scenario, A -3000 W against B +3000 W at zero house load:
the meter reads +0 W, the sensor reads 6000 W, and the shipped SignWatchdog's
predicate is FALSE on 30 of 30 cycles.

WHAT "ACTED ON" MEANS - decided, and recorded in DECISIONS/. The trip files
Claim.set("circulation", P_SAFETY, 0.0) on the SITE actuator: every inverter
to 0 W, latched. HW-01 established that commanding a wedged inverter achieves
nothing, so the remedy is not aimed at it. A circulating loop needs TWO
participants and one of them is THIS CONTROLLER, faithfully driving the
listening sibling into the one that stopped listening. Stopping (1) halves the
loop, which needs no cooperation from the wedged unit, and (2) - the more
important half - makes the residue VISIBLE at the meter instead of cancelled
at it, where every ordinary guard can finally see it. What it cannot do is
stop the wedged unit; the notification says so and names the isolator as the
last actuator. SITE-scoped because the quantity is defined over the roster and
is identically zero for any one unit - there is no fact of the matter about
which of two opposing inverters is the offender, so filing it per-unit would
invent an attribution and could park the only unit still listening.

THE WINDOW IS FROM THE RIG, NOT A ROUND NUMBER (AC 3). 924 closed-loop runs in
which NOTHING is faulty, over both families of legitimate opposition (7
reversals x 5 write staggers; 7 flap periods x 4 amplitudes), swept across the
whole range the schema accepts for gain and slew_w. Worst CONTINUOUS
legitimate opposition by threshold: 505.0 s at 100 W, 40.0 s at 250 W, 10.0 s
at 300 W, 5.0 s at 400 W and above. The knee is 400 W and it is the schema
floor - below it, ordinary operation holds the sensor over the line for
minutes and no window inside the 900 s billing quarter could tell that from a
fault. circulation_w 700 is 1.75x the knee, the headroom bought for inverter
B's ramp and lag being PLACEHOLDERS until SPIKE-04 measures them.
circulation_hold_s 75 clears three bounds: 15x the worst legitimate episode,
2.5x the firmware's own LINK_STALE_S veto, 12x inside the quarter. A momentary
opposition while one unit slews through zero is LEGITIMATE - writes are
serialised, so B's number lands up to a cycle after A's - and 0 of the 924
runs trip.

THE ALARM SURVIVES AN HA RESTART (SAFETY-34). persistent_notification is in
memory, which is exactly how the 2026-09-02 outage went unseen for 6 h 25 m
with the detector correct throughout. The latch is persisted to /data as flat
site-level keys and the notification is RE-RAISED on the first cycle after
every start, through the same code path as a fresh trip - two paths would be
two chances for the restart one to rot unnoticed.

AT N=1 NOTHING CHANGES (AC 5). |B| - |B| is 0 for every reading including one
nobody took, no claim is ever filed, the arbitration is byte-identical, and
the status page says n/a rather than claiming to watch something that cannot
happen.

VERIFIED BY BREAKING IT (AC 4). 18 mutations, all RED on named checks, each on
a freshly copied tree with no __pycache__ mounted :ro. M10 - the sensor
rewritten to read self.allocation.shares - is the headline: it takes AC 2 red.
Three mutations are +0 B, the risky delta, and are marked as such. One
equivalent mutant (the max(0.0,...) guard) is ARGUED rather than claimed:
searched over 700 000 rosters without a negative. One genuine SURVIVOR was
found and closed - publish() turning an unknown into a published 0 W, the
sensor's own None rule defeated one layer out. Full ledger in DECISIONS/.

arbiter.CLAIM_SCOPE gains its ninth source, so test_peakguard.py's 10f and 10g
tripwires and test_arbiter.py's scope table went red on the line the append
was written. Updated deliberately, with the argument, as their own comments
require. Full 19-file suite green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-04 01:56:23 +02:00

317 lines
18 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# SAFETY-31 — what "acted on" means for circulating power, and what it cannot mean
**Status: DECIDED 2026-09-04.** The trip **stops the site**: a `SET 0 W` claim
at `P_SAFETY` on the **site** actuator, latched, persisted, and re-announced
after a restart. This document records why that is an action rather than
theatre, given that [[HW-01]] established a wedged inverter cannot be commanded
at all — and it records the numbers the threshold and the window were read off,
because "establish the window from the rig, do not pick a round number" is an
acceptance criterion and a number without its measurement is a guess.
Filed with SAFETY-31. Card `6a99e0022b6a6c8ca15dbe3e`.
---
## 0. The two halves, and which one this card built
Half of this card was already shipped, and it is not the half below.
| | **ALLOC-01, AC 7 — the COMMAND half** | **SAFETY-31 — the MEASUREMENT half** |
|---|---|---|
| what it is | `allocator.REFUSE_SIGN` | `circulation.circulating_w()` + `CirculationWatchdog` |
| what it sees | the shares the allocator is about to hand out | each inverter's **own measured power** (`Inverter.batt`, register 0x0518) |
| what it does | **refuses** the allocation, names both units, returns no shares | **stops the site**, latches, alarms durably |
| what it covers | circulating power the allocator would have **commanded** | circulating power **nobody commanded** |
| graded by | `test_alloc.py` part 6 | `test_watchdog.py`, the SAFETY-31 section |
**They are complements, not duplicates.** The gate refuses an allocation before
it is written and cannot see one millisecond past that. This card reads the
hardware and does not care what was written. Neither replaces the other, and
**a sensor built on the allocation record would collapse the second into the
first and detect nothing** — see §2.
## 1. What "acted on" means
> **The trip files `Claim.set("circulation", P_SAFETY, 0.0)` on the SITE
> actuator. Every inverter is commanded 0 W and held there until an operator
> starts control again.**
### The objection, which is correct
[[HW-01]] is on the record: on 2026-09-02 the comms subsystem failed and the
power stage went on executing its last instruction for **6 h 25 m**, through
every command this add-on sent it. If the cause of the circulation is a unit
that has stopped listening, **a command is not a remedy**, and shipping one as
though it were is theatre.
### The answer
**A circulating loop needs two participants.** In every cause on this card's
list — a unit ignoring its share, a unit lagging it, a stale share still being
executed, a maintenance or [[SAFETY-25]] pin holding one unit while the loop
drives the other, or HW-01's wedged power stage — **exactly one participant is
the unit that stopped listening, and the other one is this controller**,
faithfully commanding the sibling to a share computed to sum with a number the
wedged unit is not actually producing.
Stopping the commanding half does two things a notification cannot:
1. **It halves the loop.** The unit that *is* still listening stops being driven
into the one that is not. That is real energy and real cycles off a real
pack, and it needs no cooperation from the wedged unit to take effect.
2. **It makes the residue visible — and this is the more important half.**
Circulating power hides *because it nets to zero at the meter*. With this
controller's contribution removed, whatever the wedged unit is still doing
stops being cancelled and appears at P1 as a plain net flow, where
SAFETY-02's sign watchdog, the peak guard and the capacity tariff can all see
it. **The fault stops being invisible.**
Measured on the [[TEST-19]] rig, and this is the number that makes the case:
in the card's own scenario — A at −3000 W, B at +3000 W, house load 0 W — the
meter reads **+0 W** and the shipped `SignWatchdog`'s predicate is **false on
every one of 30 cycles**, because it gates on `abs(grid_w) > 300` and the gate
never opens.
### What it does not do, said plainly
**It does not stop the wedged inverter. Nothing this add-on can send will.** If
the loop persists at the meter after this trip, the remaining actuator is a
human at the isolator, and the notification says so in those words.
### Why the claim is SITE-scoped
`arbiter.CLAIM_SCOPE` classifies `circulation` as `SCOPE_SITE`, beside `loop`,
`peak`, `arbitrage` and `safety`. The neighbouring `failsafe` entry looks like
it and is not: SAFETY-02 has site-wide *evidence* and a per-unit *conclusion*,
because it compares the meter against the command **this controller put on one
actuator**.
Circulating power has no such conclusion available. It is
`Σ|Bᵢ| − |ΣBᵢ|` — defined over the whole roster, **identically zero for any one
unit** — and there is no fact of the matter about which of two opposing
inverters is the offender: *a unit ignoring its share* and *a sibling faithfully
executing a share computed to sum with it* produce **identical readings**.
Filing this per-unit would require inventing an attribution the measurement
cannot support, and would park one inverter while the other went on feeding the
loop — quite possibly parking the only one still listening.
The same argument is why the latch is **flat** in the store (`circ_tripped`,
`circ_reason`) and not namespaced like SAFETY-02's. One site, one latch.
## 2. Why the sensor is MEASURED, and why the cheap version is worthless
The obvious cheap version computes the same formula over `Allocation.shares`.
It would be simpler, would never be `None`, and would be **worth nothing**, for
two independent reasons:
1. **It cannot fire.** ALLOC-01's sign-coherence gate already refuses any
allocation whose shares carry two signs. A record-derived sensor is therefore
a sensor over a set that is **guaranteed sign-coherent**. It reads zero *by
construction*, forever.
2. **It is [[MUTATION.md]]'s third failure mode.** It could only ever confirm
that the allocator agrees with itself — a check whose data cannot reach the
failure it names.
Verified rather than argued. On the rig, with inverter B wedged on a stale
discharge while PV drives the site to charge:
```
S = -2500 W shares a = -1250 W b = -1250 W <- record: ok, ONE sign, sums to S
MEASURED A = -1250 W B = +1000 W <- hardware
sensor over the RECORD : 0 W
sensor over MEASURED power : 2000 W -> trips at 190 s
```
And the card's own headline scenario, driven on the same rig — A at −3000 W,
B at +3000 W, house load 0 W:
```
METER (P1) : +0 W <- the site looks IDLE
sensor over MEASURED power : 6000 W -> trips at 100 s
SAFETY-02 SignWatchdog : predicate false on 30 of 30 cycles
```
The same pair is asserted in `test_watchdog.py` **with the precondition written
as its own check and printed**, per MUTATION.md's rule, and mutation **M10**
(the property rewritten to read `self.allocation.shares`) takes it **red**.
## 3. The numbers, and where they come from
Both were read off a curve produced on `admin/goodwe`'s `sim/` two-master rig,
used read-only: the add-on's own `compute()`, its own `allocator.plan()` and its
own `circulating_w()`, driving `plantsim.Plant` through the firmware's own
encoder, with the sensor input taken from each unit's holding register 0x0518.
**924 closed-loop runs in which nothing is faulty** — every unit executes the
share it was given — over both families of stimulus that can produce legitimate
opposition (7 reversals × 5 write staggers, and 7 flap periods × 4 amplitudes),
each swept across the **whole range `config.yaml` accepts** for the two settings
that decide how fast the site can turn around: `gain` `float(0.05,1.0)` and
`slew_w` `int(50,5000)`.
### Reproducing it
The rig lives in `admin/goodwe` (`sim/`) and was used **read-only**; the harness
is not committed here because it imports that repo's `plantsim`/`frame_check`,
which are not present in this one or in CI. What it does is small enough to
restate: build `Site([Inverter.from_spec("A", INVERTER_A),
Inverter.from_spec("B", INVERTER_B)])`, close the loop with the add-on's own
`compute()` and `allocator.plan()`, put each share on its unit's wire through
`Inverter.command()` (which goes out through the firmware's real encoder), and
read the sensor from `inv.reg(0x0518)` — **the same register the live add-on's
`batt_entity` binds**, so the input is the one the shipped code gets. Point it
at a checkout with `GOODWE_ADDON_REPO`. ⚠️ MUTATION.md's note applies: those
harnesses write `__pycache__` into **both** trees, so purge both by hand before
grading a mutation through one.
Every figure below was re-run against the final shipped `CirculationConfig`
(`min_w=700`, `hold_s=75`) rather than against the draft values they were first
measured at.
### Worst CONTINUOUS legitimate opposition, by threshold
| threshold | worst continuous | runs that ever cross it |
|---:|---:|---:|
| 100 W | **505.0 s** | 281 / 924 |
| 200 W | 445.0 s | 174 / 924 |
| 250 W | 40.0 s | 132 / 924 |
| 300 W | 10.0 s | 115 / 924 |
| **400 W** | **5.0 s** | 74 / 924 |
| 700 W | 5.0 s | 33 / 924 |
| 1000 W | 5.0 s | 16 / 924 |
| 1700 W | 0.0 s | 0 / 924 |
****With the shipped `min_w=700 W` and `hold_s=75 s`, 0 of those 924 runs trip.**
The knee is at 400 W and it is sharp.** Below ~250 W a flapping load holds the
sensor above the threshold for *minutes* at a time, and no window shorter than
the 900 s billing quarter could tell that from a fault. At 400 W and above the
worst legitimate episode collapses to **5.0 s — one control cycle — and stays
there**. The worst peak on any legitimate run is **1700 W**.
That 5.0 s is the serialised write path doing exactly what it should: on a
reversing cycle inverter B's new number lands up to a whole control cycle after
A's, both then slew through zero at 800 W/s behind a 10 s lag, and **between the
two zero crossings they genuinely oppose**. Nothing is wrong. A guard that
tripped on it would be the fourth false-alarm generator of the week, and this
project's doctrine on crying wolf is explicit ([[SAFETY-15]], [[SAFETY-19]]).
### `circulation_w: 700` — schema floor `400`
`700` is **1.75× the measured knee**. The headroom is bought for one named
reason: **inverter B does not physically exist.** Its ramp and lag in the rig
are inverter A's, *copied*, carrying `measured=False`; [[SPIKE-04]] measures the
real ones. A slower or faster second unit moves the transient's shape, and the
margin above the knee is what absorbs that without re-opening this card.
**Revisit when SPIKE-04 lands, not before.**
The **schema floor is the knee itself**, 400 W, so a legally configured install
cannot land in the 445–505 s regime. What the threshold costs: a circulating
loop smaller than 350 W per unit is not reported, which is deliberate — at that
scale the round-trip loss is a few watts.
### `circulation_hold_s: 75` — schema range `30..900`
Three bounds, and the value clears all three:
* **lower, from the rig — 5.0 s.** The worst continuous legitimate opposition
above 700 W across all 924 runs.
* **lower, from the firmware — 30 s (`inverter.LINK_STALE_S`).** A unit still
executing a stale share for less than the firmware's own veto window is a
*link* fault, and the add-on already answers it: liveness drops the unit and
the survivors carry its share, with the record saying so. Firing inside that
window would raise a second, louder alarm about a state already being handled.
* **upper, from the bill — 900 s**, the quarter that sets the capacity tariff.
**75 s** is fifteen control cycles: **15× the worst legitimate episode**, **2.5×
the firmware veto**, and **12× inside the quarter**. It is deliberately not 60
or 120 — a round number here would be a guess wearing a measurement's clothes,
and a check in `test_watchdog.py` asserts it is none of the values a guess
lands on.
## 4. Why the window cannot be defeated the way SAFETY-02's was
`allocator.py` records the measurement that makes SAFETY-02 blind to the
broadcast fight: the sign disagreement **flips on alternate ticks**, so
`disagree_since` is cleared between every pair of them and the 180 s hold never
accumulates.
Circulating power cannot do that. It is a **magnitude**, non-negative by
construction, and a sustained loop holds it high on *every* cycle rather than
every other one. An oscillation that genuinely alternated between opposing and
agreeing is not a circulating loop and should not be reported as one — which is
exactly the flapping-load case in §3, and it is why the window clears.
## 5. SAFETY-34: the latch is not the alarm
`persistent_notification` is **in memory**. Home Assistant restarts and the card
is gone while the fault runs on — which is precisely how the 2026-09-02 outage
went unseen for **6 h 25 m with the detector correct throughout**.
So the durable half and the visible half are joined, and the join is
`Controller._circ_announced`: a **per-process** flag, deliberately *not*
persisted. The latch in `/data/state.json` says *"this site is tripped"*; the
flag says *"this process has put that in front of a human"*. A restart clears
the flag and keeps the latch, so **the notification is re-raised on the first
cycle after every start** — which is the first cycle after a Home Assistant
restart, because the add-on restarts with it.
**One code path serves both** the fresh trip and the restart, on purpose: two
paths would be two chances for the restart one to rot unnoticed, and the restart
one is the case nobody exercises by hand. Mutations **M13** (flag initialised
from the latch) and **M18** (the call removed) both take it red.
## 6. Mutation ledger
Every run on a **freshly copied tree with no `__pycache__`, mounted `:ro`** into
the CI image — the structural immunity MUTATION.md records for CI and for
SAFETY-25 — so no stale `.pyc` could exist to be read. Byte deltas recorded;
**three are `+0 B`, the risky delta, and are marked**.
| # | file | mutation | Δ | result |
|---|---|---|---|---|
| M1 | `circulation.py` | `sum(abs(v))` → `sum(v)` in the sensor | −5 B | **RED** — the 6000 W, 2000 W and three-unit checks |
| M2 | `circulation.py` | N≤1 returns `None` instead of `0.0` | +1 B | **RED** — AC 5's nine-reading sweep |
| M3 | `circulation.py` | a missing reading at N≥2 returns `0.0` | −1 B | **RED** — "health from an absence" |
| M4 | `circulation.py` | drop `max(0.0, ...)` | — | **EQUIVALENT, not run as a claim.** For a same-sign roster the two sums accumulate the same magnitudes in the same order and are bit-identical; 700 000 random rosters produced no negative. Argued at the line and in the check's own text. |
| M5 | `circulation.py` | threshold `<=` → `<` | −1 B | **RED** — "exactly 700 W forever does not trip" |
| M6 | `circulation.py` | window `<` → `<=` | +1 B | **RED** — "trips at 75 s, not before" |
| M7 | `circulation.py` | the window removed entirely | −85 B | **RED** — both slew-transient checks |
| M8 | `circulation.py` | `None` no longer clears the window | +34 B | **RED** — the stale-reading gap check |
| M9 | `circulation.py` | the latch removed | −123 B | **RED** — "restored as tripped stays tripped" |
| **M10** | `main.py` | **the sensor derived from `self.allocation.shares`** | +60 B | **RED** — AC 2's headline check. This is the card. |
| M11 | `main.py` | the `circulation` claim never filed | −235 B | **RED** — `test_watchdog.py` **and** `test_peakguard.py` 10f/10g |
| M12 | `main.py` | the latch not persisted to the store | −112 B | **RED** — all three durability checks |
| M13 | `main.py` | `_circ_announced` initialised from the latch | +21 B | **RED** — SAFETY-34's re-raise |
| M14 | `main.py` | the operator clear leaves `_circ_announced` set | −45 B | **RED** — "the next trip would raise nothing" |
| M15 | `arbiter.py` | `circulation` scoped `SCOPE_UNIT` | **+0 B** | **RED** — `test_arbiter.py` ×3 and `test_peakguard.py` ×3 |
| M16 | `config.yaml` | schema floor `400` → `100` | **+0 B** | **RED** — the measured-knee check |
| M17 | `circulation.py` | default `700` → `500` | **+0 B** | **RED** — the CFG-03 pin and the not-a-round-number check |
| M18 | `main.py` | `await self.check_circulating()` removed | −39 B | **RED** — all four notification checks |
| M19 | `main.py` | publish `round(x or 0.0)` instead of `None` | −68 B | **survivor on the first battery — GAP CLOSED**, then RED |
**M19 is recorded as a survivor because it was one.** The first battery found no
check grading what `publish()` does when a per-unit reading is missing, so
turning *"I do not know"* into *"0 W"* on the one surface an operator actually
watches went unnoticed — the sensor's own `None` rule, defeated one layer out.
A check was added and the mutation re-run; it is red now. The run that found it
was purged, so it is a result and not an artefact.
## 7. What this card does NOT close
* **Inverter B's ramp and lag are placeholders.** Every transient number in §3
was produced with inverter A's parameters copied onto B (`measured=False`).
SPIKE-04 measures the real ones and is the revisit trigger for the 1.75×
margin. **Do not present any figure here as a measurement of inverter B.**
* **A wedged inverter still cannot be stopped by this add-on.** §1 says so; the
notification says so; there is no software fix and this card does not pretend
to one.
* **A loop below ~350 W per unit is not reported.** Deliberate, §3.
* **The saturation freeze fires under a wedged unit and leaves a standing grid
error.** Observed on the rig while producing §2's numbers: with B wedged, the
loop's anti-windup test compares a site target against a measured site power
that cannot track it, freezes the integrator and leaves the meter parked at
−3750 W. Nothing alarms on that today — a freeze is an ordinary control action
by design. It is a real finding, it is **not** in scope here, and it wants a
card of its own.