SAFETY-42: a unit that stops delivering, and the discriminator the card asked for does not exist
A unit standing at a non-zero share and producing nothing is seen by NOTHING. SAFETY-30's per-unit sign gate needs OPPOSITION on both sides of 250 W and a unit at 0 W is not opposing, it is ABSENT. The site sum still agrees with the meter in direction. Circulating power is exactly zero when one unit does nothing. The only trace left is `reason == "saturated-freeze"`, which clamp.py:93-96 documents as the alarm an operator learns to ignore. It is HW-01's shape, which ran 6 h 25 m with all three layers spectators. ⚠️ THE CARD'S OWN DISCRIMINATOR WAS BUILT FIRST AND FALSIFIED, AND THAT IS THE MAIN CONTENT. SAFETY-42 asks for "saturation persisting past N while |grid error| > X, both derived". Run on the two-master rig it fails in BOTH directions, and no (N, X) fixes either: FALSE POSITIVE `plantsim.Plant.limit()` returns 0 W for a discharge command into a pack at its own floor, and a wedged unit produces 0 W too. Load 2000 W, equal weights, 2000 W per-unit cap, 120 cycles: THE TWO ARMS ARE IDENTICAL ON ALL 120 in grid, site target, every share and every measured watt. Both saturated-freeze on 116 of them, both hold |grid error| over 300 W on all 120 and settle at 1200 W, both read circulating 0.0 W. The predicate's inputs do not differ, so no threshold on them can differ. And "a cold pack at its floor must not alarm" is the card's own AC 2. FALSE NEGATIVE the standing grid error is what a HEALTHY SIBLING ERASES. Load 800 W, same config, B wedged: over the last 30 of 200 cycles the loop is frozen on 30 of 30 while the worst |grid error| is 30 W, with B standing at +770 W and delivering 0 W. Any X above 30 W is silent for the whole run on the exact fault the card describes. WHAT REPLACES IT. Per unit, for a discharge share, `share - measured`; readable because of a reduction that only exists at N>=2: a unit is named while it is short by more than `saturation_w` with NO ACCOUNT for it, and the units in that state are a PROPER SUBSET of the units asked to discharge. THE ACCOUNT keeps the cold pack out and is the only thing that can: the two cases are identical in the power domain, so the discriminator has to be a non-power fact, and there is exactly one - the pack's own SoC against its own `bms_soc_floor`, asked through `peakguard.discharge_permitted()` rather than transcribed. ⚠️ `bms_soc_floor` ALONE, not `pack_floor_pct()`'s conjunction with `peak_soc_floor`: that floor bounds a money-saving discharge only and using it would explain away a wedged unit anywhere between the two. THE PROPER SUBSET keeps a whole-site cause out - a derate this add-on cannot see hits every unit; one box that stopped listening does not - and it is what makes this INERT AT N=1, by algebra rather than by a roster guard. IT REPORTS. IT DOES NOT ACT, and the strongest argument is for acting. Dropping the wedged unit from `live` genuinely recovers a site the shipped path does not: at load 1500 W, kept in the split the site settles at 800 W with a 700 W error and NEVER comes inside 100 W in 150 cycles (the freeze pins the target, so the sibling's SHARE never grows); dropped, it delivers 1500 W from cycle 14. What refuses it is that the only re-allocation this add-on can express is all-or-nothing: a unit derated to 50 % is short by more than saturation_w with nothing to explain it, so this detector names it too, and at load 3000 W dropping it leaves the site delivering 2000 W against 2400 W kept - 400 W WORSE. A partial re-allocation is what would make acting safe and the allocator cannot express it. That is a card. control.py is NOT MODIFIED. The freeze shapes the loop; this reports on it. NUMBERS, MEASURED. The watts side is `Tuning.saturation_w`, read at runtime, so there is no fourth CFG-03 copy and it cannot drift from the number the control law itself calls "not following". `delivery_hold_s: 185` comes off 2640 closed-loop two-inverter runs in which nothing is faulty (6 gains x 5 slews over the whole schema range, across load reversals, a flapping load and ASYMMETRIC SPLITS - a family SAFETY-30's campaign did not have and the one that dominates here) plus 1890 runs at a roster of ONE as the control: threshold reversals flapping weighted N=1 control * 500 W 20.0 s 0.0 s 30.0 s 0.0 s The N=1 column is 0.0 s at EVERY threshold in all 1890 runs, and the flapping column - the family that defeated SAFETY-30's site window - is zero too, because a common stimulus makes both units short at once and the subset stops being proper. ⚠️ The curve is NOT monotone (55.0 s at 250 W against 40.0 s at 100 W): lowering the threshold can turn a proper subset into the whole roster and BREAK the run. ⚠️ THE MARGIN IS MEASURED, NOT FELT, BECAUSE INVERTER B DOES NOT PHYSICALLY EXIST. `plantsim.INVERTER_B` is A COPIED, `measured=False`, so every two-inverter figure above is two copies of A. Sweeping a different second unit: the worst legitimate episode is insensitive to B's RAMP (30.0 s at half A's) and scales with its LAG - 45.0 s at twice A's measured 10 s, 90.0 s at four times it. 185 s is 6.2x the measured worst, 2.06x the four-times-slower case, 6.2x LINK_STALE_S and 4.9x inside the 900 s quarter. Deliberately not 180, which is SAFETY-02's site hold. SAFETY-34. `unit/<name>/delivery_tripped` in the store, one notification id per unit, re-raised on the first cycle of every process, cleared by the Start press. No flat mirror: a rollback lands on a version that drives one actuator, where this check is inert. ⚠️ SCHEMA CHANGE - `delivery_hold_s` joins options: and schema:, so this needs a version bump before it ships. Additive and defaulted, so an existing options blob stays valid. STATED GAPS: charge shares are not graded (no configured charge ceiling exists, so a constant-voltage taper cannot be explained and arming it would fire on every maintenance charge); every unit wedged at once is not caught (the subset is not proper and no instrument here separates that from a plant that cannot). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
This commit is contained in:
glenn schrooyen
co-authored by
Claude Opus 5
Claude Opus 5
parent
0581648c73
commit
706e625d99
@@ -97,6 +97,58 @@ the bound from 10 to 13. That ceiling change also removes a degenerate setting:
|
||||
the release bound, so **no accepted configuration can stop the block engaging**.
|
||||
At the old bound of 9 a setting of 9 or 10 deleted it silently.
|
||||
|
||||
⚠️ **AN INVERTER THAT STOPS DELIVERING ITS SHARE IS NOW REPORTED BY NAME
|
||||
([[SAFETY-42]]) - AND THIS ADDS AN OPTION, SO IT IS A SCHEMA CHANGE.**
|
||||
`delivery_hold_s: 185` joins `options:` and `schema:` (`int(30,900)`). The
|
||||
Supervisor keys the built image by `version:`, so this needs a bump before it
|
||||
ships - see the 0.7.0 heading above, which makes the same point about
|
||||
`bms_soc_floor`. The key is ADDITIVE and defaulted, so an existing options blob
|
||||
stays valid; what a missing bump costs is that nobody's Supervisor re-reads the
|
||||
schema.
|
||||
|
||||
**Two or more inverters only, and at one inverter it is inert by construction.**
|
||||
A unit standing at a discharge share it is not producing - short by more than
|
||||
`saturation_w`, with nothing this add-on can read to explain it, while at least
|
||||
one sibling is NOT in that state - is named after `delivery_hold_s`. That fault
|
||||
was invisible to everything: the per-unit sign check needs OPPOSITION and a unit
|
||||
at 0 W is not opposing but ABSENT, the site sum still agrees with the meter in
|
||||
direction, circulating power is exactly zero when one unit does nothing, and the
|
||||
only remaining trace is the saturation freeze - which `clamp.py` itself calls an
|
||||
alarm operators learn to ignore. It is [[HW-01]]'s shape, which ran 6 h 25 m.
|
||||
|
||||
⚠️ **IT REPORTS. IT DOES NOT ACT.** Nothing is commanded differently, the loop is
|
||||
untouched, `control.py` is not modified, and the site goes on running short by
|
||||
that unit's share. Commanding an inverter that has stopped listening achieves
|
||||
nothing (HW-01), and the only re-allocation this add-on can express is
|
||||
all-or-nothing - measured on the rig, dropping a unit derated to 50 % from the
|
||||
split leaves the site **400 W worse** than keeping it, and this detector cannot
|
||||
tell that unit from a wedged one. The status row is a CAUTION, not a failure,
|
||||
because the controller has not stopped.
|
||||
|
||||
⚠️ **THE CARD'S OWN DISCRIMINATOR WAS FALSIFIED, AND THAT IS THE MAIN CONTENT.**
|
||||
SAFETY-42 asked for "saturation persisting past N while |grid error| > X, both
|
||||
derived". Built and run on the two-master rig first, it fails in both
|
||||
directions, and no (N, X) fixes either: a pack at its own floor and a wedged unit
|
||||
are **identical on all 120 cycles** of a run in grid, site target, every share
|
||||
and every measured watt - so the benign case fires - and with a sibling that has
|
||||
headroom the standing grid error collapses to **30 W** while the wedged unit
|
||||
stands at a +770 W share delivering nothing, so the real case does not. What
|
||||
replaces it is a per-unit shortfall plus two reductions the card did not ask for:
|
||||
an ACCOUNT (the pack's own `bms_soc_floor`, the only non-power fact that
|
||||
separates an empty pack from a wedged one) and a PROPER SUBSET of the units asked
|
||||
to discharge (which is what keeps a whole-site derate out, and what makes this
|
||||
inert at N=1 - measured zero in all 1890 one-inverter runs).
|
||||
|
||||
The report **survives a Home Assistant restart** ([[SAFETY-34]]) under
|
||||
`unit/<name>/delivery_tripped`, with one notification id per unit, re-raised on
|
||||
the first cycle of every process and cleared by the same **Start** press as the
|
||||
other latches. **Charge shares are deliberately not graded** - there is no
|
||||
configured charge ceiling to explain a constant-voltage taper with, and arming
|
||||
that direction would fire on every maintenance charge.
|
||||
`DECISIONS/SAFETY-42-unit-delivery.md` carries the campaign (2640 healthy
|
||||
two-inverter runs plus 1890 at one inverter as the control), the
|
||||
inverter-B sensitivity sweep the margin is sized on, and the mutation ledger.
|
||||
|
||||
⚠️ **A PER-INVERTER POWER READING THAT HAS STOPPED FOLLOWING ITS OWN COMMAND IS
|
||||
NOW `None`, NOT A HELD NUMBER ([[SAFETY-37]]).** `Inverter.batt` (register
|
||||
`0x0518`) goes through a freshness gate: unchanged for >= 35 s *while* the
|
||||
|
||||
@@ -0,0 +1,427 @@
|
||||
# SAFETY-42 — a unit that stops delivering its share, the discriminator the card asked for, and why that discriminator does not exist
|
||||
|
||||
**Status: DECIDED 2026-09-06.** A new per-inverter check, `app/delivery.py`,
|
||||
names an inverter that is **standing at a discharge share it is not producing**
|
||||
while at least one sibling is not in that state and nothing this add-on can read
|
||||
explains it. It **reports and does not act**. At a roster of one it is inert by
|
||||
construction.
|
||||
|
||||
⚠️ **This record also FALSIFIES the discriminator the card proposed.** SAFETY-42
|
||||
asked for *"saturation persisting past N while |grid error| stays above X, with N
|
||||
and X derived"*. That predicate was built and run on the rig before anything was
|
||||
written, and it fails in **both** directions — not for want of better N and X,
|
||||
but because of what the quantity is. §2 is the measurement. This is the
|
||||
SAFETY-39 shape: saying the criterion is wrong was worth more than five
|
||||
near-misses at satisfying it.
|
||||
|
||||
Filed with SAFETY-42, card `6a9d498a49082c0086e525c1`.
|
||||
|
||||
---
|
||||
|
||||
## 0. What is uncovered, and why it is structural rather than a threshold
|
||||
|
||||
A unit delivering **0 W against a non-zero share** is seen by nothing:
|
||||
|
||||
| guard | why it is silent |
|
||||
|---|---|
|
||||
| [[SAFETY-30]] per-unit sign | gates on **opposition ≥ 250 W on both sides**. A unit at 0 W never opens it — it is not opposing, it is **absent** |
|
||||
| SAFETY-30 site sign | the sum still **agrees with the meter in direction**; only the magnitude is short |
|
||||
| [[SAFETY-31]] circulation | `Σ\|Bᵢ\| − \|ΣBᵢ\|` is **zero** when one unit does nothing — circulation needs two units pulling *against* each other |
|
||||
| the saturation freeze | **fires** — and `clamp.py`:93-96 says of exactly this: *"A loop can be legally saturated all day — a cold pack, a curtailed inverter, an export limit — so a saturation-shaped alarm is one an operator learns to ignore"* |
|
||||
|
||||
So the one signal that does fire is the one this codebase documents as
|
||||
unreadable, and that documentation is **right**. This card therefore needed a
|
||||
different signal, not a louder one.
|
||||
|
||||
It is [[HW-01]]'s shape. On 2026-09-02 the comms subsystem failed and the power
|
||||
stage went on executing its last instruction for **6 h 25 m** with all three
|
||||
layers spectators. At N=2 the same fault leaves a healthy sibling regulating
|
||||
alone, the site short by up to one unit's share, and nothing saying so.
|
||||
|
||||
## 1. The rig
|
||||
|
||||
Everything in §2 and §3 was measured on `admin/goodwe`'s two-master rig
|
||||
(TEST-19), used **read-only** and run container-free, from a harness in a
|
||||
scratch tree. Real and imported, never transcribed: the control law
|
||||
(`control.compute`, `Tuning`), the allocator (`arbiter.allocate_record` →
|
||||
`allocator.plan`), the per-unit envelope the shipped `supervised` and `device`
|
||||
claims build, the sign predicate (`watchdog.disagrees`), the circulating sensor
|
||||
(`circulation.circulating_w`), the check under test (`delivery.delinquent`,
|
||||
`short_by`, `account`), the wire (`frame_check.setpoint_atomic`, the firmware's
|
||||
own encoder, in and back out through `plantsim.parse`/`decode`), and the plant
|
||||
(`plantsim.Plant`). The per-unit reading is **holding register 0x0518**, the one
|
||||
the live `batt_entity` binds.
|
||||
|
||||
Modelled, and therefore not evidence about the thing modelled: the master's
|
||||
failsafe machine (absent), and the two fault injections —
|
||||
|
||||
* **wedged** — the write never reaches the plant, so its latched command stands.
|
||||
Started at 0 W, which is *"delivering 0 W against a non-zero share"*.
|
||||
* **at its floor** — the write arrives and `Plant.limit()` zeroes it, because
|
||||
`soc <= soc_min`. This is the benign case the acceptance criterion names.
|
||||
|
||||
### ⚠️ Inverter B does not physically exist
|
||||
|
||||
`plantsim.INVERTER_B` is **inverter A copied**, `measured=False` (NOTES.md
|
||||
§1.1b; the datasheet-and-serial figures 4600 W / 20.48 kWh / floor 10 % are
|
||||
deliberately *not* in the model). **Every two-inverter figure in this document
|
||||
is two copies of inverter A.** §3.3 is the sensitivity sweep that sizes the
|
||||
margin that costs.
|
||||
|
||||
### Reproducing it
|
||||
|
||||
The harness is **not committed here**, for the reason SAFETY-30's and
|
||||
SAFETY-31's were not: it imports `plantsim` and `frame_check`, which live in the
|
||||
other repository and are not present in this one's CI image. What it does is
|
||||
small enough to restate: build two `plantsim.Inverter`s from `INVERTER_A` /
|
||||
`INVERTER_B`; close the loop with the add-on's own `compute()` on the SITE sum
|
||||
(`Controller.site_batt_w`) and `arbiter.allocate_record()` over per-unit
|
||||
envelopes of `±min(max_w, rating)`; put each share on its own wire through
|
||||
`Inverter.command()`, honouring a configurable **write stagger** so B's number
|
||||
can land up to four cycles after A's (the serialised write path); advance
|
||||
`Site.settle()` one control cycle; and record, per unit, the command standing
|
||||
*before* the write and that unit's 0x0518 power. Then grade each cycle with the
|
||||
**shipped** `delivery.delinquent()` and take the **longest run of consecutive
|
||||
cycles** on which it names anybody — not a count of naming samples, because the
|
||||
window is cleared by any cycle that does not.
|
||||
⚠️ MUTATION.md's note applies: this harness writes `__pycache__` into **both**
|
||||
trees. Run it with `python3 -B` and purge.
|
||||
|
||||
## 2. The card's discriminator, measured — and it does not work
|
||||
|
||||
### 2.1 It fires on the benign case, and no (N, X) can stop it
|
||||
|
||||
`Plant.limit()` returns 0 W for a discharge command into a pack at its own
|
||||
floor. A wedged unit that never receives its command also produces 0 W. The two
|
||||
are not merely similar:
|
||||
|
||||
> **Load 2000 W, equal weights, 2000 W per-unit cap, 120 control cycles.
|
||||
> The two arms are IDENTICAL on all 120 cycles** in (grid, site target, each
|
||||
> unit's share, each unit's measured power), to the last emitted watt. Both
|
||||
> report `reason == "saturated-freeze"` on **116 of the 120** cycles (the other
|
||||
> four are `slew-limited`, before it engages on cycle 4), both hold |grid error|
|
||||
> over 300 W on **all 120** and settle at **1200 W**, and `circulating_w` is
|
||||
> **0.0 W** in both.
|
||||
|
||||
A predicate whose inputs are *saturation duration* and *|grid error|* reads the
|
||||
same number on both arms at every cycle. There is no N and no X. And *"a
|
||||
legitimately saturated site — cold pack at its floor — does **not** alarm"* is
|
||||
this card's own acceptance criterion.
|
||||
|
||||
`test_delivery.py` part 4 asserts that identity on the add-on's own end-to-end
|
||||
fixture, so it goes red if it ever stops being true.
|
||||
|
||||
### 2.2 It is silent on the headline case, because a healthy sibling erases it
|
||||
|
||||
The standing grid error is exactly the thing a working sibling removes.
|
||||
|
||||
> **Load 800 W, equal weights, 2000 W per-unit cap, B wedged, 200 cycles.**
|
||||
> Over the last 30 cycles the loop is saturation-frozen on **30 of 30** — so
|
||||
> "saturation persisting past N" is true for every N — while the worst
|
||||
> **|grid error| in that window is 30 W**. B is standing at a **+770 W**
|
||||
> discharge share and delivering **0 W**.
|
||||
|
||||
Any X above 30 W leaves the card's predicate silent for the whole run, on the
|
||||
exact fault it was written for. The card's own sentence — *"a healthy sibling
|
||||
regulating alone, the site short by up to one unit's rating"* — describes the
|
||||
case, and the grid error is the one number that cannot report it.
|
||||
|
||||
### 2.3 What that leaves
|
||||
|
||||
The derivation instruction stands and is honoured in §3. What could not be
|
||||
derived was a threshold on the wrong quantity. Both failures are written into
|
||||
`delivery.py`'s docstring as well, because the next person to read the card will
|
||||
propose the same predicate.
|
||||
|
||||
## 3. The quantity that does work
|
||||
|
||||
Per unit, for a share that asks it to **discharge**:
|
||||
|
||||
```
|
||||
short_by(i) = share_i − measured_i # watts that did not arrive
|
||||
```
|
||||
|
||||
and the reduction that makes it readable:
|
||||
|
||||
> a unit is **delinquent** while it is short by more than `saturation_w` with
|
||||
> **no account for it**, and the set of units in that state is a **proper
|
||||
> subset** of the units that were asked to discharge.
|
||||
|
||||
### 3.1 The account — the only thing that separates a wedged unit from a cold pack
|
||||
|
||||
§2.1 says there is nothing in the power domain. There is exactly one non-power
|
||||
fact this add-on holds about a pack: **its own state of charge against its own
|
||||
`bms_soc_floor`**, which SAFETY-32 made a required per-entry key. `account()`
|
||||
asks it through **`peakguard.discharge_permitted()`** — public since SAFETY-32
|
||||
precisely so that this question has one answer — and writes its own sentence,
|
||||
because that function's wording names the *peak guard's* floor and would be a
|
||||
false statement here.
|
||||
|
||||
Three things account for a shortfall, and each is a state already being handled:
|
||||
|
||||
| account | why it is not this fault |
|
||||
|---|---|
|
||||
| the unit is not live | `live_units()` has already dropped it from the split and the action log says so — firing inside that is the SAFETY-15/19 failure |
|
||||
| the unit is parked by a sign trip | it is commanded 0 W on purpose; not delivering is compliance |
|
||||
| `discharge_permitted(soc, bms_soc_floor)` | the pack is at or below its own cut-out and physically cannot |
|
||||
|
||||
⚠️ **The floor passed in is `bms_soc_floor` alone, NOT
|
||||
`peakguard.pack_floor_pct()`'s conjunction with `peak_soc_floor`.** That
|
||||
conjunction is right for the peak guard and wrong here: `peak_soc_floor` bounds
|
||||
a money-saving discharge only (SAFETY-25), ordinary self-consumption discharges
|
||||
straight past it, and using it would **explain away** a wedged unit sitting
|
||||
anywhere between the two floors.
|
||||
|
||||
⚠️ **Every unknown accounts for the shortfall**, i.e. fails toward silence —
|
||||
the opposite direction to `discharge_permitted()`'s own caller, and the right
|
||||
one here: there it withholds a discharge, here it withholds an **alarm**. The
|
||||
cost is bounded, because the same absence stops the site through `inputs_ok`
|
||||
(an unreadable SoC) or is refused by the roster schema (an unstated floor).
|
||||
|
||||
### 3.2 The proper subset — what only exists at N ≥ 2
|
||||
|
||||
A derate this add-on cannot see — thermal, grid-code, a DC-side restriction —
|
||||
hits **every** unit that was asked. One box that has stopped listening does not.
|
||||
So if every asked unit is short and unexplained, this module says **nothing**:
|
||||
it has no instrument that tells that from a plant which simply cannot deliver,
|
||||
and reporting it anyway would be the saturation-shaped alarm produced by the
|
||||
guard written to replace it.
|
||||
|
||||
**At N = 1 the subset is never proper, for every input**, so the check is inert
|
||||
there — not by a roster guard, but by the algebra, the same way
|
||||
`circulation.circulating_w()` is identically zero at N ≤ 1. That is AC 5, and
|
||||
§3.3's control arm measures it rather than asserting it.
|
||||
|
||||
### 3.3 The window — the one number this card chose
|
||||
|
||||
**2640 closed-loop runs at a roster of two in which NOTHING is faulty** — every
|
||||
unit executes the share the allocator gave it — over 6 gains × 5 slews, the
|
||||
whole range `config.yaml` accepts for `gain` `float(0.05,1.0)` and `slew_w`
|
||||
`int(0,5000)` (0 = follow the rail), across three families:
|
||||
|
||||
* **1050 load reversals** — 7 sizes (±300 W … ±5000 W) × 5 write staggers;
|
||||
* **840 flapping loads** — 7 periods (10 s … 600 s) × 4 amplitudes;
|
||||
* **750 weighted reversals** — 5 splits (0.5, 0.7, 0.9, 0.99, 0.1) × 5 staggers
|
||||
on a ±2200 W reversal. ⚠️ **This family did not exist in SAFETY-30's campaign
|
||||
and it is the one that dominates here**, for a mechanical reason: an
|
||||
asymmetric split gives one unit a much larger step than its sibling, so the
|
||||
big one is still ramping while the small one has settled — which is precisely
|
||||
the *proper subset* this check keys on.
|
||||
|
||||
Every run is executed a second time at a **roster of one** on the same stimulus.
|
||||
**The N=1 arm is the control**, and it is what separates *"a property of the
|
||||
check"* from *"a property of the second inverter"*.
|
||||
|
||||
Worst **CONTINUOUS** legitimate episode, as a function of the shortfall
|
||||
threshold:
|
||||
|
||||
| threshold | reversals (of 1050) | flapping (of 840) | weighted (of 750) | **N=1 control (1890 runs)** |
|
||||
|---:|---:|---:|---:|---:|
|
||||
| 100 W | 40.0 s (608) | 0.0 s (0) | 75.0 s (576) | **0.0 s (0)** |
|
||||
| 250 W | 55.0 s (490) | 0.0 s (0) | 60.0 s (480) | **0.0 s (0)** |
|
||||
| **500 W** | **20.0 s (206)** | **0.0 s (0)** | **30.0 s (368)** | **0.0 s (0)** |
|
||||
| 700 W | 20.0 s (125) | 0.0 s (0) | 20.0 s (174) | **0.0 s (0)** |
|
||||
| 1000 W | 15.0 s (70) | 0.0 s (0) | 15.0 s (111) | **0.0 s (0)** |
|
||||
| 1500 W | 10.0 s (51) | 0.0 s (0) | 10.0 s (50) | **0.0 s (0)** |
|
||||
| 2000 W | 10.0 s (24) | 0.0 s (0) | 5.0 s (23) | **0.0 s (0)** |
|
||||
|
||||
Three findings worth stating rather than leaving in the table:
|
||||
|
||||
1. **The N=1 column is zero everywhere, in all 1890 runs and at every
|
||||
threshold.** That is the compatibility promise measured, not argued.
|
||||
2. **The flapping family is zero everywhere too** — and it is the family that
|
||||
defeated SAFETY-30's site window (980 s at two inverters, 1000 s at one, no
|
||||
window inside the billing quarter surviving it). It cannot touch this check,
|
||||
because a common stimulus makes both units short at the same time and the
|
||||
subset stops being proper. That is the proper-subset rule earning its place
|
||||
on a case it was not designed for.
|
||||
3. **The curve is NOT monotone** — 55.0 s at 250 W against 40.0 s at 100 W —
|
||||
and that is a property of the reduction rather than noise: lowering the
|
||||
threshold puts MORE units into the short set, which can turn a proper subset
|
||||
into the whole roster and **break** the run. Both the numerator and the
|
||||
denominator move. Do not read this table as a sensitivity curve.
|
||||
|
||||
#### The watts side is not a new option — it is `Tuning.saturation_w`
|
||||
|
||||
`main.py` hands the check `self.tuning.saturation_w` (500 W) and there is no
|
||||
`delivery_w` key anywhere. That is a derivation, not a saving:
|
||||
|
||||
* it is already **the gap at which `compute()` itself concludes the inverter is
|
||||
not following** and freezes the integrator. Below it the shipped control law
|
||||
calls the divergence ordinary tracking, and an alarm under a threshold the
|
||||
control law calls normal is an alarm about nothing;
|
||||
* `saturation_cycles` (3) consecutive cycles over that same gap are needed
|
||||
before the freeze engages, so **every cycle this counts is a cycle the freeze
|
||||
has already seen**. This is a reduction over the freeze's input, not a second
|
||||
opinion about it;
|
||||
* CFG-03 already tracks three places a default has to be kept in step. A fourth
|
||||
copy could drift, and the drift that matters has a direction: an operator who
|
||||
raises `saturation_w` has told the loop that a bigger gap is ordinary
|
||||
tracking, and a check that stayed put would then alarm on it.
|
||||
|
||||
**Rejected:** a `delivery_w` option, for the three reasons above. A margin above
|
||||
`saturation_w` (the way `circulation_w` sits 1.75× above its knee) was also
|
||||
rejected: there is no knee here — the table above is not monotone — so a margin
|
||||
would be a preference, and all the margin this check needs is in the window.
|
||||
|
||||
#### `delivery_hold_s: 185` — schema `int(30,900)`
|
||||
|
||||
Four bounds, and the value clears all four.
|
||||
|
||||
* **lower, from the rig — 30.0 s.** The worst continuous legitimate episode at
|
||||
the shipped threshold across all 2640 runs (weighted family; 20.0 s on
|
||||
reversals, 0.0 s on flapping).
|
||||
* **lower, from the firmware — 30 s (`inverter.LINK_STALE_S`).** A unit not
|
||||
executing its share for less than the firmware's own veto window is a *link*
|
||||
fault the add-on already answers by dropping it from the split. Reporting
|
||||
inside that window would be a second, louder alarm about a state already
|
||||
being handled — the SAFETY-15/SAFETY-19 failure exactly.
|
||||
* **margin, from inverter B not existing — and it is MEASURED, not felt.** The
|
||||
campaign's B is A copied, so 30.0 s is two identical machines. Sweeping a
|
||||
*different* second unit through the weighted family (750 runs each):
|
||||
|
||||
| B lag | B ramp | worst legitimate | runs naming |
|
||||
|---:|---:|---:|---:|
|
||||
| 10.0 s (A) | 800 W/s (A) | **30.0 s** | 368/750 |
|
||||
| 20.0 s | 800 W/s | 45.0 s | 469/750 |
|
||||
| 10.0 s | 400 W/s | **30.0 s** | 370/750 |
|
||||
| 20.0 s | 400 W/s | 45.0 s | 469/750 |
|
||||
| 40.0 s | 200 W/s | **90.0 s** | 500/750 |
|
||||
|
||||
**The RAMP does not move it at all and the LAG moves it roughly linearly.**
|
||||
The shortfall lasts as long as the unit's first-order lag holds it more than
|
||||
`saturation_w` below its share; at these step sizes the ramp is never the
|
||||
binding term. So the number to carry margin against is B's lag, and a unit
|
||||
**four times slower than A** produces 90.0 s.
|
||||
* **upper, from the bill — 900 s**, the quarter that sets the capacity tariff.
|
||||
|
||||
**185 s** is thirty-seven control cycles: **6.2×** the measured worst, **2.06×**
|
||||
the four-times-slower case, **6.2×** the firmware veto and **4.9×** inside the
|
||||
quarter. ⚠️ Deliberately **not 180** — that is SAFETY-02's site hold, and a
|
||||
reader must not be able to think this was carried across — and not 150 or 200.
|
||||
|
||||
**What it costs, stated rather than hidden:** up to 185 s of a site running
|
||||
short by one unit's share before anybody is told. Against a fault that ran
|
||||
6 h 25 m unnoticed, and against a guard that takes **no action**, the window
|
||||
buys its margin with reporting latency and nothing else.
|
||||
|
||||
## 4. It reports. It does not act — and the strongest argument is for acting
|
||||
|
||||
Card requirement 3 asks which, and why. The answer is recorded with the
|
||||
evidence **against** it first, because "alarm-only" is the weaker-looking answer.
|
||||
|
||||
**Command it harder — no.** HW-01 settles it: the power stage executed its last
|
||||
instruction for 6 h 25 m through every command this add-on sent. A command is
|
||||
not a remedy for a box that is not listening.
|
||||
|
||||
**Stop the site, as SAFETY-31 does — no**, and the asymmetry with SAFETY-31 is
|
||||
exact. Circulating power is stopped because *this controller is one of the two
|
||||
participants*: removing our half halves the loop and un-hides the residue at a
|
||||
meter that was netting it to zero. Here this controller is **not** a participant,
|
||||
nothing is netting, and the sibling is the only thing still delivering. Parking
|
||||
it takes the site from "short by one unit" to "short by all of them".
|
||||
|
||||
**Re-allocate its share to the sibling — no, and NOT because the loop already
|
||||
does it.** That comfortable answer is false, and the rig says so. Dropping the
|
||||
wedged unit from `live` — the mechanism `live_units()` already uses for a dead
|
||||
link — is a real improvement at loads the site cannot cover on one unit's
|
||||
*share*:
|
||||
|
||||
| B wedged, per-unit cap 2000 W | kept in the split | dropped from the split |
|
||||
|---|---|---|
|
||||
| load 800 W | settles inside 100 W from cycle 6 | from cycle 8 |
|
||||
| load 1500 W | **800 W delivered, 700 W error, NEVER inside 100 W in 150 cycles** | 1500 W delivered, inside 100 W from cycle 14 |
|
||||
| load 2000 W | **800 W delivered, 1200 W error, never settles** | 2000 W delivered, inside 100 W from cycle 8 |
|
||||
|
||||
The mechanism is the saturation freeze doing its job on a site comparison that a
|
||||
non-delivering unit has poisoned: `|site target − site batt|` stays over
|
||||
`saturation_w` forever, so the target cannot rise and the sibling's *share*
|
||||
never grows. That is not three cycles of latency; it is the difference between a
|
||||
site that recovers and one that does not.
|
||||
|
||||
**What refuses it anyway** is that the only re-allocation this add-on can
|
||||
express is **all or nothing**. There is no way to say *"give the sibling the
|
||||
watts that did not arrive and leave this unit the ones that did"*. And a unit
|
||||
delivering **part** of its share is short by more than `saturation_w` too, with
|
||||
nothing here to explain it — so this detector names it, and cannot tell a 50 %
|
||||
derate from a wedged box, because no instrument this add-on has can:
|
||||
|
||||
| B derated to 50 %, load 3000 W, cap 2000 W | site delivers | B contributes | grid error |
|
||||
|---|---:|---:|---:|
|
||||
| kept in the split | 2400 W | 800 W | 600 W |
|
||||
| **dropped** | **2000 W** | **0 W** | **1000 W — 400 W worse** |
|
||||
|
||||
**SAFETY-42 names B in that run.** So the remedy would make the site worse, in a
|
||||
case the detector cannot rule out, on a live control law, driven by a brand-new
|
||||
alarm. A **partial** re-allocation is what would make acting safe, and the
|
||||
allocator has no way to express it today. That is a card, and this section is
|
||||
its evidence.
|
||||
|
||||
`test_delivery.py` part 5 asserts the decision as a **counterfactual** rather
|
||||
than by reading the code: the same fixture run with the report tripping and with
|
||||
it never tripping produces byte-identical grid, site target, shares and measured
|
||||
power on every cycle.
|
||||
|
||||
## 5. Requirement 4: the saturation freeze is untouched
|
||||
|
||||
`control.py` is not modified by this card, at all. The freeze still shapes the
|
||||
loop; this reports on it. `test_delivery.py` part 6 asserts what each
|
||||
neighbouring guard concludes on the same fixture — SAFETY-30 per-unit silent,
|
||||
SAFETY-30 site silent, SAFETY-31 identically zero, the freeze firing at the
|
||||
same counter on **both** arms — which is the honest version of "every existing
|
||||
guard is silent", including the one that is not.
|
||||
|
||||
## 6. Surviving a restart (SAFETY-34)
|
||||
|
||||
`persistent_notification` is **in memory**. Home Assistant restarts, the card is
|
||||
gone, and the fault runs on — precisely how the 2026-09-02 outage went unseen
|
||||
for 6 h 25 m with the detector correct throughout. It matters more here than
|
||||
anywhere else in this add-on, because SAFETY-02 and SAFETY-31 at least leave a
|
||||
**stopped** site for somebody to notice; this one leaves a site that looks like
|
||||
it is working.
|
||||
|
||||
* durable half: `unit/<name>/delivery_tripped` / `delivery_reason`, namespaced
|
||||
per unit for `store.unit_key()`'s reason, written **reason first**.
|
||||
* visible half: `_delivery_announced`, a per-**process** set of unit names,
|
||||
deliberately not persisted. A restart empties the set and keeps the latch, so
|
||||
the alarm is re-raised on the first cycle of every process — **one code path**
|
||||
for a fresh trip and for a restart into an existing one, `check_circulating()`'s
|
||||
argument.
|
||||
* **no flat mirror**, where SAFETY-02 has one. `store.py`'s rule 2 keeps the flat
|
||||
keys so a rollback still finds its latch, and the version being rolled back to
|
||||
drives exactly one actuator — a roster at which this check is inert.
|
||||
* one **notification id per unit**, built from the same `unit_slugs()` TEL-07
|
||||
announces the MQTT entities under. Two wedged inverters must produce two cards,
|
||||
not one replacing the other.
|
||||
* it **latches**, and not for SAFETY-02's reason. Nothing here removes its own
|
||||
evidence, because nothing here changes what is commanded. It latches so that a
|
||||
unit which stopped delivering for minutes and then resumed cannot erase its own
|
||||
record — the intermittent fault is the one an operator most needs told about.
|
||||
Cleared by the same **Start** press as the others.
|
||||
* the status row is `ok: False, warn: True` — this list's **caution**. Every
|
||||
other latched row means the site is at 0 W and a red cross is the truth; this
|
||||
one means the site is still running and quietly short, and a red cross would
|
||||
say the controller had stopped.
|
||||
|
||||
## 7. What this does NOT catch, said plainly
|
||||
|
||||
* **Charge shares are not graded.** The account gate needs a bound this
|
||||
controller reads in the commanded direction. There is one for discharge
|
||||
(`bms_soc_floor`, required of every roster entry since SAFETY-32) and **none**
|
||||
for charge. A pack in constant-voltage taper legitimately takes hundreds of
|
||||
watts less than it is told for tens of minutes, and arming that direction would
|
||||
fire on the taper of every §12 maintenance charge. The same fault in the other
|
||||
direction wants the same treatment; what it needs first is a per-pack charge
|
||||
ceiling this add-on does not have. **That is a card.**
|
||||
* **Every unit wedged at once is not caught.** The subset is not proper, and
|
||||
there is no instrument here that separates that from a plant which simply
|
||||
cannot deliver. The site is short at the meter in that case, which is where the
|
||||
existing guards can see it.
|
||||
* **A pack that refuses above its configured floor is reported.** That is
|
||||
deliberate: either the unit has stopped delivering, or `bms_soc_floor` does not
|
||||
describe that battery. The notification says both, in those words.
|
||||
* **A unit with a share of 0 W is not graded**, and neither is it a peer. With
|
||||
only one unit asked to discharge there is nothing to compare against.
|
||||
|
||||
## 8. Mutation ledger
|
||||
|
||||
@@ -485,6 +485,7 @@ behind it would trip the firmware watchdog on a system that is working fine.
|
||||
| `unit_sign_disagree_s` | 85 | **Two or more inverters only.** How long one unit's command and its own power must stay opposed before that unit is parked |
|
||||
| `site_sign_disagree_w` | 300 | **Two or more inverters only.** Threshold for the site-wide check - the sum of the commands against the meter. The same 300 W as `sign_disagree_w`, for the same reason: near zero the sign is noise |
|
||||
| `site_sign_disagree_s` | 325 | **Two or more inverters only.** How long the site's total command and the meter must stay opposed before every inverter is parked. Longer than `sign_disagree_s` because the command has to cross zero over a rail that is the sum of the roster, at a `slew_w` that is not - measured at 110 s with one inverter and 215 s with two |
|
||||
| `delivery_hold_s` | 185 | **Two or more inverters only.** How long one inverter may stand at a discharge share it is not producing - short by more than `saturation_w`, with no reason this add-on can read - while at least one sibling is **not** in that state, before it is named. **It reports; it does not act.** Nothing is commanded differently and the site goes on running short by that unit's share |
|
||||
|
||||
#### The sign-disagreement failsafe
|
||||
|
||||
@@ -584,6 +585,70 @@ agreement is 93.4 %; on the four captured fault clusters it is 0.3 %, 0.9 %,
|
||||
10.5 % and 0.0 %. There is no reason to move either value without a capture that
|
||||
says so.
|
||||
|
||||
|
||||
#### ⚠️ A unit that stops DELIVERING is a third fault again (SAFETY-42)
|
||||
|
||||
The two checks above both key on **direction**. A unit sitting at **0 W against
|
||||
a non-zero share** has no direction: it is not pushing the wrong way, it is
|
||||
**absent**. So the per-unit sign gate never opens (it needs opposition on both
|
||||
sides of `unit_sign_disagree_w`), the site sum still agrees with the meter in
|
||||
direction, and circulating power is exactly **zero** — that quantity needs two
|
||||
units pulling *against* each other, and one of these is pulling nowhere.
|
||||
|
||||
The only thing that does fire is the **saturation freeze**, and this add-on's own
|
||||
code calls that an alarm operators learn to ignore, because a loop can be legally
|
||||
saturated all day: a cold pack, a curtailed inverter, an export limit.
|
||||
|
||||
So `delivery_hold_s` grades a different quantity — **how many watts of a
|
||||
discharge share did not arrive** — and makes it readable with a reduction that
|
||||
only exists once there is more than one inverter:
|
||||
|
||||
> a unit is named while it is short of its share by more than `saturation_w`,
|
||||
> **with nothing this add-on can read to explain it**, and the units in that
|
||||
> state are **fewer than the units that were asked to discharge**.
|
||||
|
||||
Both halves matter, and the second is why this cannot fire on a
|
||||
one-inverter install at all:
|
||||
|
||||
- **"nothing explains it"** is what keeps a legitimately empty pack out. A pack
|
||||
at or below its own `bms_soc_floor` cannot discharge, and that is the *only*
|
||||
fact this add-on holds that distinguishes an empty pack from a wedged one —
|
||||
the two are otherwise identical at the unit and at the meter, watt for watt.
|
||||
A unit the add-on has already dropped from the split, or parked by a sign
|
||||
trip, is likewise accounted for.
|
||||
- **"fewer than were asked"** is what keeps a whole-site cause out. A derate this
|
||||
add-on cannot see — thermal, grid-code, DC-side — hits every unit. One box that
|
||||
has stopped listening does not. If *every* unit that was asked is short and
|
||||
unexplained, nothing is reported: there is no instrument here that tells that
|
||||
from a plant which simply cannot deliver.
|
||||
|
||||
**It reports. It does not act.** Commanding an inverter that has stopped
|
||||
listening achieves nothing — that happened on this hardware once for 6 h 25 m —
|
||||
and the only re-allocation this add-on can express is all-or-nothing, which
|
||||
would throw away the output of a unit delivering *part* of its share. So the
|
||||
loop is untouched, the sibling keeps regulating, and the site keeps running
|
||||
**short by that unit's share** until somebody looks. The status page shows it as
|
||||
a caution rather than a failure for exactly that reason: the controller has not
|
||||
stopped.
|
||||
|
||||
The report **survives a Home Assistant restart** (`unit/<name>/delivery_tripped`
|
||||
in the add-on's own state file) and the notification is re-raised on the next
|
||||
start, because a `persistent_notification` is in memory and that is precisely how
|
||||
the 2026-09-02 outage went unseen. It is cleared by pressing **Start**, like the
|
||||
other latches.
|
||||
|
||||
The banner reads **`NOT DELIVERING — <unit> …`** as a *warning*, not a failure,
|
||||
and it **displaces** `Running — holding grid at zero`. That last sentence is a
|
||||
claim the site cannot make while a unit is absent, and a page that goes on
|
||||
reading healthy while the fault runs is the whole shape this check exists to
|
||||
remove.
|
||||
|
||||
**Charge shares are not graded**, and that is a stated gap rather than an
|
||||
oversight: there is no configured charge ceiling anywhere in this add-on, so a
|
||||
pack in constant-voltage taper — which legitimately takes hundreds of watts less
|
||||
than it is told, for tens of minutes — could not be explained, and arming that
|
||||
direction would fire on the taper of every maintenance charge.
|
||||
|
||||
#### Saturation is counted in cycles, not seconds
|
||||
|
||||
The specification states the saturation window as **"> 10 s"**. This add-on counts
|
||||
|
||||
@@ -0,0 +1,611 @@
|
||||
"""SAFETY-42 - a unit that stops delivering its share, which nothing else can see.
|
||||
|
||||
At one inverter this file is inert by construction (see `delinquent()`), and
|
||||
that is deliberate rather than a limitation: the whole signal it reads is a
|
||||
DIFFERENCE BETWEEN SIBLINGS, and a roster of one has none.
|
||||
|
||||
WHAT IT CATCHES, in one sentence: an inverter that accepts its share and does
|
||||
not produce it, beside a sibling that does - HW-01's shape at N=2, where the
|
||||
2026-09-02 fault wedged inverter A for 6 h 25 m with every layer a spectator.
|
||||
|
||||
WHY EVERY EXISTING GUARD IS SILENT, AND IT IS STRUCTURAL
|
||||
=======================================================
|
||||
SAFETY-30 per unit gates on OPPOSITION: both the command and the unit's own
|
||||
power over `unit_sign_disagree_w` (250 W) with opposite
|
||||
signs. A unit sitting at 0 W never opens that gate. It is
|
||||
not opposing, it is ABSENT.
|
||||
SAFETY-30 site the sum of the commands still AGREES with the meter in
|
||||
direction. Only the magnitude is short, and the site
|
||||
comparison is a sign test.
|
||||
SAFETY-31 `sum(|Bi|) - |sum(Bi)|` is ZERO when one unit does
|
||||
nothing: circulation needs two units pulling AGAINST each
|
||||
other, and one of these is pulling nowhere at all.
|
||||
the saturation freeze fires - and `clamp.py`:93-96 says of exactly this: "A
|
||||
loop can be legally saturated all day - a cold pack, a
|
||||
curtailed inverter, an export limit - so a
|
||||
saturation-shaped alarm is one an operator learns to
|
||||
ignore."
|
||||
|
||||
⚠️ SO THE ONE SIGNAL THAT DOES FIRE IS THE ONE THIS CODEBASE DOCUMENTS AS
|
||||
UNREADABLE, and that documentation is right. This module therefore does not make
|
||||
that signal louder; it computes a different one.
|
||||
|
||||
⚠️ THE ONE PLACE IT OVERLAPS SAFETY-30, AND THE OVERLAP RESOLVES ITSELF. A unit
|
||||
measured at -2000 W against a +2000 W share is OPPOSING - SAFETY-30's fault -
|
||||
and is ALSO short by 4000 W with nothing here to explain it. Both detectors have
|
||||
evidence, and only one alarm results, because SAFETY-30's per-unit hold is the
|
||||
shorter of the two (85 s against 185 s at the shipped defaults): it trips first,
|
||||
files its `failsafe` SET, the share goes to 0 W, and this check's gate closes on
|
||||
`share > 0` while `account()` answers "parked" as well. That is asserted on a
|
||||
reversed-unit fixture in `test_delivery.py` part 6 rather than left to the
|
||||
reader, because two alarms for one fault is how an operator learns to skim.
|
||||
|
||||
------------------------------------------------------------------------------
|
||||
THE CARD'S OWN DISCRIMINATOR WAS MEASURED AND IT DOES NOT WORK - READ THIS FIRST
|
||||
==============================================================================
|
||||
SAFETY-42 proposed: alarm on `saturation persisting past N` while
|
||||
`|grid error| > X`, with N and X derived rather than chosen. That predicate was
|
||||
built and run on the two-master rig before this module was written, and it
|
||||
fails in BOTH directions. Neither failure is a matter of picking better N and X;
|
||||
both are properties of the quantity.
|
||||
|
||||
1. IT FIRES ON EVERY BENIGN SATURATION, and the two cases are not merely
|
||||
similar - they are THE SAME TRACE. `plantsim.Plant.limit()` returns 0 W for
|
||||
a discharge command into a pack at its own floor; a wedged unit that never
|
||||
receives its command also produces 0 W. Run both against the same stimulus
|
||||
(load 2000 W, equal weights, 2000 W per-unit cap) and every emitted watt
|
||||
agrees: over 120 control cycles the two arms are IDENTICAL on all 120 in
|
||||
(grid, site target, each unit's share, each unit's measured power). Both
|
||||
report `reason == "saturated-freeze"` on 116 of those 120 cycles - the
|
||||
other four are `slew-limited`, before the freeze engages on cycle 4 - and
|
||||
both hold |grid error| over 300 W on all 120, settling at 1200 W. A
|
||||
predicate reading only those
|
||||
numbers cannot separate them at ANY (N, X), because nothing in its inputs
|
||||
differs. And "a cold pack at its floor must not alarm" is this card's own
|
||||
acceptance criterion. `test_delivery.py` part 4 asserts that identity on
|
||||
the add-on's own end-to-end fixture, so it goes red if it ever stops being
|
||||
true.
|
||||
|
||||
2. IT IS SILENT ON THE HEADLINE CASE, because the standing grid error is
|
||||
exactly what a HEALTHY SIBLING ERASES. Same rig, load 800 W, equal weights,
|
||||
2000 W per-unit cap, inverter B wedged: over the last 30 of 200 cycles the
|
||||
loop is saturation-frozen on every one of them - so "saturation persisting
|
||||
past N" is true for any N - while the worst |grid error| in that window is
|
||||
30 W. B is standing at a +770 W discharge share and delivering 0 W. Any X
|
||||
above 30 W leaves the card's predicate silent for the whole run, on the
|
||||
exact fault it was written for: a healthy sibling regulating alone with the
|
||||
site short by one unit's share.
|
||||
|
||||
The card's derivation instruction stands and is honoured; what could not be
|
||||
derived was a threshold on the wrong quantity. This is written down rather than
|
||||
quietly replaced, because the next person to read the card will propose it again.
|
||||
|
||||
------------------------------------------------------------------------------
|
||||
THE QUANTITY THAT DOES WORK, AND WHY IT NEEDS TWO INVERTERS
|
||||
===========================================================
|
||||
Per unit, for a share that asks it to DISCHARGE:
|
||||
|
||||
short_by(i) = share_i - measured_i (watts that did not arrive)
|
||||
|
||||
`share_i` is the command STANDING ON THAT UNIT'S WIRE - `Inverter.standing_w()`,
|
||||
the same operand SAFETY-30's per-unit comparison uses - and `measured_i` is that
|
||||
unit's own power, holding register 0x0518, per-instance since DEV-02.
|
||||
|
||||
That number alone is the saturation signal again, per unit. What makes it
|
||||
readable is the reduction over the roster:
|
||||
|
||||
a unit is DELINQUENT while it is short with NO ACCOUNT FOR IT, and the set
|
||||
of units in that state is a PROPER SUBSET of the units that were asked to
|
||||
discharge.
|
||||
|
||||
Both halves are load-bearing:
|
||||
|
||||
* THE ACCOUNT is what keeps the cold pack out, and it is the only thing that
|
||||
can. The two cases are identical in the power domain (see 1 above), so the
|
||||
discriminator has to be a fact this controller reads that is NOT power - and
|
||||
there is exactly one: the pack's own state of charge against its own
|
||||
`bms_soc_floor`. `account()` below asks it through `peakguard.
|
||||
discharge_permitted()`, the predicate SAFETY-32 already made public, rather
|
||||
than transcribing the comparison a second time.
|
||||
* THE PROPER SUBSET is what keeps a site-wide cause out, and it is the half
|
||||
that only exists at N>=2. A derate this add-on cannot see - a thermal
|
||||
limit, a grid-code curtailment, an inverter-side DC restriction - hits the
|
||||
whole roster. One box that has stopped listening does not. If EVERY unit
|
||||
that was asked to deliver is short and unexplained, this module says
|
||||
nothing: it cannot tell that from a plant that simply cannot do it, and
|
||||
saying so anyway is the saturation-shaped alarm all over again.
|
||||
|
||||
⚠️ AT N=1 THE SUBSET IS NEVER PROPER AND THIS IS INERT. One asked unit is either
|
||||
accounted for (nothing to say) or unaccounted for (the whole roster, so nothing
|
||||
to compare it against). `delinquent()` returns `{}` for every input at a roster
|
||||
of one - not by a guard on `len`, but because a proper subset of a one-element
|
||||
set that contains the element is impossible. That is AC 5, and it is asserted
|
||||
rather than assumed in `test_delivery.py` part 1.
|
||||
|
||||
------------------------------------------------------------------------------
|
||||
DISCHARGE SHARES ONLY, AND THE COST OF THAT IS STATED RATHER THAN HIDDEN
|
||||
=======================================================================
|
||||
`short_by()` answers None for a charge share. The account gate needs a bound
|
||||
this controller actually reads in the commanded direction, and there is one for
|
||||
discharge - `bms_soc_floor`, required of every roster entry since SAFETY-32 -
|
||||
and NONE for charge. A pack in constant-voltage taper legitimately takes
|
||||
hundreds of watts less than it is told for tens of minutes, and there is no
|
||||
configured ceiling anywhere in this add-on to explain that with. Armed on charge
|
||||
shares this module would alarm on the taper of every §12 maintenance charge,
|
||||
which is the crying-wolf failure it exists to avoid.
|
||||
|
||||
What that costs, said plainly: a unit that stops ABSORBING its share is not
|
||||
caught here. It is the same fault in the other direction and it wants the same
|
||||
treatment; what it needs first is a per-pack charge ceiling this add-on does not
|
||||
have. That is a card, and this paragraph is its evidence.
|
||||
|
||||
------------------------------------------------------------------------------
|
||||
IT REPORTS. IT DOES NOT ACT - AND THAT IS A DECISION, NOT AN OMISSION
|
||||
====================================================================
|
||||
Card requirement 3 asks which, and why. Three candidates, and the reasoning is
|
||||
recorded because "alarm-only" is the weaker-looking answer and is still right:
|
||||
|
||||
COMMAND IT HARDER - no. HW-01 settles it: on 2026-09-02 the power stage went
|
||||
on executing its last instruction for 6 h 25 m through every command this
|
||||
add-on sent it. A command is not a remedy for a box that is not listening,
|
||||
and pretending otherwise is theatre.
|
||||
|
||||
RE-ALLOCATE ITS SHARE TO THE SIBLING - no, AND NOT FOR THE COMFORTABLE
|
||||
REASON. The comfortable reason would be that the loop already does it, and
|
||||
the rig says otherwise: with B wedged and equal weights, dropping B from
|
||||
`live` - the mechanism `live_units()` already uses for a dead link - is a
|
||||
real improvement at loads the site cannot cover on one unit's SHARE.
|
||||
Measured, per-unit cap 2000 W, B wedged:
|
||||
|
||||
load 1500 W kept in the split: the site settles at 800 W with a 700 W
|
||||
grid error and NEVER comes inside 100 W in 150 cycles -
|
||||
the saturation freeze pins the site target, so A's share
|
||||
never rises. Dropped: 1500 W delivered, inside 100 W from
|
||||
cycle 14.
|
||||
load 2000 W kept: 800 W delivered, 1200 W error, never settles.
|
||||
Dropped: 2000 W delivered, inside 100 W from cycle 8.
|
||||
|
||||
That is not three cycles of latency; it is the difference between a site
|
||||
that recovers and one that does not. It is the strongest argument on this
|
||||
card and it is stated first, because the answer still has to be no.
|
||||
|
||||
WHAT REFUSES IT is that the only re-allocation this add-on can express is
|
||||
ALL OR NOTHING. `live_units()` drops a unit entirely and the survivors carry
|
||||
the whole target; there is no way to say "give the sibling the watts that
|
||||
did not arrive and leave this unit the ones that did". And a unit delivering
|
||||
PART of its share is short by more than `saturation_w` too, with nothing
|
||||
here to explain it, so this detector names it - it cannot tell a 50 %
|
||||
derate from a wedged box, because neither can any instrument this add-on
|
||||
has. Measured, B derated to 50 %, load 3000 W, per-unit cap 2000 W, and
|
||||
SAFETY-42 names B in that run:
|
||||
|
||||
kept in the split site delivers 2400 W (B contributing 800 W),
|
||||
grid error 600 W
|
||||
dropped site delivers 2000 W (B contributing 0 W),
|
||||
grid error 1000 W - 400 W WORSE
|
||||
|
||||
So the remedy would make the site worse in a case this detector cannot rule
|
||||
out, on a live control law, driven by a brand-new alarm. A partial
|
||||
re-allocation - the sibling gets the undelivered watts without the short
|
||||
unit being commanded to zero - is what would make acting safe, and the
|
||||
allocator has no way to express it today. That is a card, and this block is
|
||||
its evidence.
|
||||
|
||||
STOP THE SITE, as SAFETY-31 does - no, and the asymmetry with SAFETY-31 is
|
||||
exact. Circulating power is stopped because THIS CONTROLLER IS ONE OF THE
|
||||
TWO PARTICIPANTS: removing our half halves the loop and un-hides the residue
|
||||
at a meter that was netting it to zero. Here this controller is not a
|
||||
participant in the fault, nothing is netting, and the sibling is the only
|
||||
thing still delivering. Parking it would take the site from "short by one
|
||||
unit" to "short by all of them" and would remove the evidence.
|
||||
|
||||
So: a durable latch, a named unit, an action-log row, and an operator
|
||||
notification - and the loop is left exactly as it was. Requirement 4 of the card
|
||||
("do not weaken the saturation freeze to make room") is met by not touching
|
||||
`control.py` at all.
|
||||
|
||||
------------------------------------------------------------------------------
|
||||
IT SURVIVES A RESTART, WHICH IS SAFETY-34 AND IS WHY THE LATCH EXISTS AT ALL
|
||||
===========================================================================
|
||||
`persistent_notification` is IN MEMORY. Home Assistant restarts and the card is
|
||||
gone with the fault still running, which is precisely how the 2026-09-02 outage
|
||||
went unseen for 6 h 25 m. So the durable half is `unit/<name>/delivery_tripped`
|
||||
and `unit/<name>/delivery_reason` in `/data/state.json` - namespaced per unit,
|
||||
for `store.unit_key()`'s reason - and `main.py` re-raises the notification from
|
||||
it on the first cycle of every process.
|
||||
|
||||
⚠️ AND THAT IS WHY THIS LATCHES THOUGH IT DOES NOT ACT. A detector that cleared
|
||||
itself when the unit resumed would erase its own record of an intermittent
|
||||
fault, which is the one shape an operator most needs to be told about. It is
|
||||
cleared by the same operator press that clears SAFETY-02's and SAFETY-31's -
|
||||
a deliberate human act, after looking at why.
|
||||
"""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
|
||||
from .peakguard import discharge_permitted
|
||||
|
||||
# The reason code. Distinct from every other one in this add-on by construction:
|
||||
# nothing else uses it, and it deliberately does not contain the word
|
||||
# "saturation" - see the module docstring on why a saturation-shaped alarm is
|
||||
# the thing this module must not produce.
|
||||
REASON_CODE = "UNIT_NOT_DELIVERING"
|
||||
|
||||
# CFG-03. THE ONE IN-CODE COPY OF THIS DEFAULT. `config.yaml`'s `options:` block
|
||||
# is the source of truth an operator edits and the Supervisor reads; `schema:`
|
||||
# holds the bounds. This dict exists for the reason
|
||||
# `circulation.CIRC_OPTION_DEFAULTS` and `watchdog.SIGN_OPTION_DEFAULTS` do -
|
||||
# the Dockerfile does not put config.yaml in the image, so the running add-on
|
||||
# cannot read it - and `test_delivery.py` pins the two together by READING
|
||||
# config.yaml, so a drift goes red in CI rather than shipping.
|
||||
#
|
||||
# ⚠️ THERE IS ONLY ONE KEY HERE, AND THE MISSING SECOND ONE IS THE POINT. The
|
||||
# other half of this check is a WATTS threshold, and it is not an option and not
|
||||
# a constant: it is `control.Tuning.saturation_w`, read at runtime. See
|
||||
# `MIN_W_IS_SATURATION_W` below.
|
||||
DELIVERY_OPTION_DEFAULTS = {
|
||||
# --- THE WINDOW, AND IT IS THE ONLY NUMBER THIS CARD CHOSE ------------
|
||||
# ⚠️ MEASURED ON THE TEST-19 TWO-MASTER RIG, NOT PICKED. 2640 closed-loop
|
||||
# runs at a roster of two in which NOTHING is faulty - every unit executes
|
||||
# the share the allocator gave it - over 6 gains x 5 slews, the whole range
|
||||
# config.yaml accepts for `gain` float(0.05,1.0) and `slew_w` int(0,5000),
|
||||
# across three families of stimulus that can produce a legitimate shortfall:
|
||||
#
|
||||
# 1050 load reversals 7 sizes x 5 write staggers (the serialised write
|
||||
# path landing B's number up to four cycles late)
|
||||
# 840 flapping loads 7 periods x 4 amplitudes
|
||||
# 750 weighted splits 5 weightings x 5 staggers on a +-2200 W reversal
|
||||
#
|
||||
# ⚠️ THE WEIGHTED FAMILY IS NEW HERE AND IT IS THE ONE THAT DOMINATES. An
|
||||
# asymmetric split hands one unit a much larger step than its sibling, so
|
||||
# the big one is still ramping while the small one has settled - which is
|
||||
# exactly the PROPER SUBSET this check keys on. SAFETY-30's campaign had no
|
||||
# such family, so reading its numbers across would have understated this.
|
||||
#
|
||||
# Every run was executed a second time at a roster of ONE, on the same
|
||||
# stimulus, as the control. Worst CONTINUOUS legitimate episode:
|
||||
#
|
||||
# threshold reversals flapping weighted N=1 control (1890 runs)
|
||||
# 100 W 40.0 s 0.0 s 75.0 s 0.0 s
|
||||
# 250 W 55.0 s 0.0 s 60.0 s 0.0 s
|
||||
# * 500 W 20.0 s 0.0 s 30.0 s 0.0 s
|
||||
# 700 W 20.0 s 0.0 s 20.0 s 0.0 s
|
||||
# 1000 W 15.0 s 0.0 s 15.0 s 0.0 s
|
||||
# 2000 W 10.0 s 0.0 s 5.0 s 0.0 s
|
||||
#
|
||||
# * is the shipped threshold, `Tuning.saturation_w`.
|
||||
#
|
||||
# ⚠️ THREE THINGS IN THAT TABLE ARE WORTH MORE THAN THE ROW THEY SIT ON.
|
||||
# (a) The N=1 column is ZERO EVERYWHERE, in all 1890 runs at every
|
||||
# threshold. The compatibility promise is measured, not argued.
|
||||
# (b) The FLAPPING column is zero everywhere too - and that is the family
|
||||
# that defeated SAFETY-30's site window (980 s at N=2, 1000 s at N=1, no
|
||||
# window inside the billing quarter surviving it). It cannot touch this
|
||||
# check, because a common stimulus makes BOTH units short at once and
|
||||
# the subset stops being proper.
|
||||
# (c) THE CURVE IS NOT MONOTONE - 55.0 s at 250 W against 40.0 s at 100 W -
|
||||
# and that is a property of the reduction, not noise: a lower threshold
|
||||
# puts MORE units into the short set, which can turn a proper subset
|
||||
# into the whole roster and BREAK the run. Do not read it as a
|
||||
# sensitivity curve.
|
||||
#
|
||||
# THE BOUNDS THE VALUE ANSWERS TO:
|
||||
#
|
||||
# LOWER, from the rig - 30.0 s, the worst continuous legitimate episode at
|
||||
# the shipped threshold across all 2640 runs.
|
||||
#
|
||||
# LOWER, from the firmware - `inverter.LINK_STALE_S`, 30 s. A unit not
|
||||
# executing its share for less than the firmware's own veto window is a
|
||||
# LINK fault the add-on already answers by dropping it from the split with
|
||||
# a line in the action log. Reporting inside that window would be a
|
||||
# second, louder alarm about a state already being handled - the
|
||||
# SAFETY-15/SAFETY-19 failure exactly.
|
||||
#
|
||||
# MARGIN, from INVERTER B NOT EXISTING, AND IT IS MEASURED RATHER THAN
|
||||
# GUESSED. The campaign's B is A COPIED (`plantsim.INVERTER_B`,
|
||||
# `measured=False`), so the 30.0 s above is two identical machines.
|
||||
# Sweeping a DIFFERENT second unit through the weighted family: the worst
|
||||
# legitimate episode is insensitive to B's RAMP (30.0 s at half A's) and
|
||||
# scales with its LAG - 45.0 s at twice A's measured 10 s lag, 90.0 s at
|
||||
# four times it. That is the sensitivity the margin has to carry until
|
||||
# SPIKE-04 measures the real unit.
|
||||
#
|
||||
# UPPER, from the bill - 900 s, the quarter that sets the capacity tariff.
|
||||
# A site running short of one unit's share must be reported well inside
|
||||
# one.
|
||||
#
|
||||
# 185 s: thirty-seven control cycles. 6.2x the worst legitimate episode this
|
||||
# rig produces, 2.06x the worst a second unit FOUR TIMES slower than A
|
||||
# produces, 6.2x the firmware's own veto window, and 4.9x inside the
|
||||
# quarter. ⚠️ DELIBERATELY NOT 180 - that is SAFETY-02's site hold and this
|
||||
# is a different check on a different quantity, so a reader must not be
|
||||
# able to think it was carried across - and not 150 or 200.
|
||||
#
|
||||
# WHAT IT COSTS, stated rather than hidden: up to 185 s of a site running
|
||||
# short by one unit's share before anybody is told. Against a fault that ran
|
||||
# 6 h 25 m unnoticed, and against a guard that TAKES NO ACTION, this window
|
||||
# buys its margin with reporting latency and with nothing else.
|
||||
"delivery_hold_s": 185,
|
||||
}
|
||||
|
||||
|
||||
# ⚠️ WHY THERE IS NO `delivery_w` OPTION, AND WHY THAT IS A DERIVATION RATHER
|
||||
# THAN A SAVING. The watts side of this check asks "how far short is far
|
||||
# enough to be worth saying anything about", and the shipped control law
|
||||
# already answers that question in one number: `Tuning.saturation_w`, 500 W,
|
||||
# the gap at which `compute()` itself concludes THE INVERTER IS NOT FOLLOWING
|
||||
# and freezes the integrator. Below it the loop calls the divergence ordinary
|
||||
# tracking, and an alarm under a threshold the control law calls normal is an
|
||||
# alarm about nothing.
|
||||
#
|
||||
# So `main.py` hands this check `self.tuning.saturation_w` and there is no
|
||||
# fourth copy of a default to keep in step (CFG-03's complaint) and no way for
|
||||
# the two to drift: an operator who raises `saturation_w` has told the loop
|
||||
# that a bigger gap is ordinary tracking, and this check moves with them by
|
||||
# construction rather than by somebody remembering.
|
||||
#
|
||||
# ⚠️ IT IS ALSO WHY THIS CANNOT DOUBLE-REPORT THE SATURATION FREEZE'S OWN
|
||||
# EVENT. `compute()` needs `saturation_cycles` (3) CONSECUTIVE cycles over
|
||||
# that same gap before it freezes anything, so every cycle this module counts
|
||||
# is a cycle the freeze has already seen. This is a reduction over the freeze's
|
||||
# input, not a second opinion about it.
|
||||
MIN_W_IS_SATURATION_W = "control.Tuning.saturation_w"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DeliveryConfig:
|
||||
"""Defaults, not constants - config.yaml carries both. See CFG-03 above."""
|
||||
|
||||
hold_s: float = float(DELIVERY_OPTION_DEFAULTS["delivery_hold_s"])
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class UnitDelivery:
|
||||
"""ONE unit's four facts for this cycle, and nothing else.
|
||||
|
||||
`name` the roster name. An alarm that cannot say which box to look at
|
||||
is the whole reason SAFETY-30 split per-unit from site.
|
||||
`share_w` what is STANDING on this unit's wire - `Inverter.standing_w()`,
|
||||
the same operand SAFETY-30's per-unit comparison takes, after
|
||||
the deadband, the slew limit, quantisation and the arbiter.
|
||||
Not the optimizer's wish: the wire carries the emitted value.
|
||||
`measured_w` this unit's OWN power, holding register 0x0518, + = discharge.
|
||||
None when the reading did not arrive.
|
||||
`accounted` why a shortfall on this unit is already explained, or None if
|
||||
nothing this controller reads explains one. Built by
|
||||
`account()`; it is a field rather than a method because the
|
||||
facts behind it (liveness, the sign latch, SoC) belong to
|
||||
main.py's `Inverter` and this module imports none of it.
|
||||
"""
|
||||
|
||||
name: str
|
||||
share_w: float | None
|
||||
measured_w: float | None
|
||||
accounted: str | None = None
|
||||
|
||||
|
||||
def short_by(share_w, measured_w):
|
||||
"""Watts of a DISCHARGE share that did not arrive, or None if unanswerable.
|
||||
|
||||
Positive means short. Zero and negative are folded into None rather than
|
||||
reported, because "delivered its share" and "delivered more than its share"
|
||||
are both not-this-fault and a caller comparing a float against a threshold
|
||||
should not have to know which.
|
||||
|
||||
⚠️ CHARGE SHARES ANSWER None, AND THAT IS THE STATED LIMIT AT THE TOP OF
|
||||
THIS FILE, not an oversight to be tidied away. There is no configured charge
|
||||
ceiling in this add-on to explain a taper with.
|
||||
|
||||
⚠️ AND A MISSING READING ANSWERS None RATHER THAN "the whole share". Zero
|
||||
would be this check inventing a fault from an absence, which is the shape of
|
||||
the 2026-09-02 outage read the other way round; and the site is already
|
||||
stopped for a missing per-unit reading anyway (`inputs_ok` requires every
|
||||
unit's `batt`, and §11.1 rule 1 commands 0 W without consulting this).
|
||||
"""
|
||||
if share_w is None or measured_w is None:
|
||||
return None
|
||||
share = float(share_w)
|
||||
if share <= 0.0:
|
||||
return None
|
||||
gap = share - float(measured_w)
|
||||
return gap if gap > 0.0 else None
|
||||
|
||||
|
||||
def account(*, live: bool, parked: bool, soc, bms_floor) -> str | None:
|
||||
"""Why a shortfall on this unit is ALREADY EXPLAINED, or None if nothing is.
|
||||
|
||||
Every branch here is a state this add-on can read and already has an answer
|
||||
for, and naming one of them a fault would be raising a second, louder alarm
|
||||
about something already being handled - the SAFETY-15/SAFETY-19 failure that
|
||||
`circulation.py` and `watchdog.py` both take their 30 s lower bound from.
|
||||
|
||||
`live` `Inverter.is_live()`. A unit whose link is stale or whose
|
||||
writes are being rejected has ALREADY been dropped from the
|
||||
split with a line in the action log saying so.
|
||||
`parked` that unit's sign watchdog has latched. It is being commanded
|
||||
0 W on purpose; not delivering is compliance, not a fault.
|
||||
`soc` that pack's measured state of charge, percent. None is not a
|
||||
full battery and is not permission - see below.
|
||||
`bms_floor` that pack's OWN cut-out, `bms_soc_floor`. Required of every
|
||||
roster entry since SAFETY-32, so at the only roster this
|
||||
module is live at it is always configured.
|
||||
|
||||
⚠️ THE SoC BRANCH IS THE WHOLE DISCRIMINATOR AND IT IS ASKED THROUGH
|
||||
`peakguard.discharge_permitted()`, WHICH IS PUBLIC FOR EXACTLY THIS REASON:
|
||||
"a safety predicate transcribed into its own check is one copy plus a future
|
||||
disagreement about what the check means". What is NOT reused is its wording,
|
||||
which is peak-guard flavoured ("the peak guard's SoC floor") and would be a
|
||||
false statement here - this module is not the peak guard and its floor is
|
||||
the pack's own cut-out, not `peak_soc_floor`. One predicate, two sentences.
|
||||
⚠️ AND THE FLOOR PASSED IN IS `bms_soc_floor` ALONE, NOT
|
||||
`peakguard.pack_floor_pct()`'s conjunction with `peak_soc_floor`. That
|
||||
conjunction is right for the peak guard and wrong here: `peak_soc_floor`
|
||||
bounds a money-saving discharge only (SAFETY-25), ordinary self-consumption
|
||||
discharges straight past it, and using it would EXPLAIN AWAY a wedged unit
|
||||
sitting anywhere between the two floors.
|
||||
|
||||
⚠️ EVERY UNKNOWN ACCOUNTS FOR THE SHORTFALL, i.e. FAILS TOWARD SILENCE, and
|
||||
that direction is the opposite of the one `discharge_permitted()` was
|
||||
written for. There it withholds a discharge; here it withholds an ALARM. An
|
||||
unreadable SoC or an unstated floor means this module cannot say whether the
|
||||
pack is empty, and an alarm nobody can justify is the crying-wolf failure
|
||||
this whole file is written against. The cost is bounded: the same absence
|
||||
stops the site through `inputs_ok` (SoC) or is refused at startup by the
|
||||
roster schema (the floor).
|
||||
"""
|
||||
if not live:
|
||||
return ("its link is stale or its writes are being rejected, and the "
|
||||
"allocator has already dropped it from the split")
|
||||
if parked:
|
||||
return "it is latched at 0 W by a sign trip, so it is doing as it is told"
|
||||
why = discharge_permitted(soc, bms_floor)
|
||||
if why is not None:
|
||||
if soc is None:
|
||||
return "its state of charge did not read, so its pack cannot be graded"
|
||||
if bms_floor is None:
|
||||
return ("no bms_soc_floor is configured for it, so there is nothing "
|
||||
"to say whether its pack is empty")
|
||||
return (f"its pack is at {float(soc):.0f} %, at or below its own "
|
||||
f"{float(bms_floor):.0f} % cut-out (bms_soc_floor)")
|
||||
return None
|
||||
|
||||
|
||||
def delinquent(units, min_w: float) -> dict:
|
||||
"""Which units are short with NO ACCOUNT, when the roster disagrees with them.
|
||||
|
||||
`units` one `UnitDelivery` per inverter, in roster order.
|
||||
`min_w` the shortfall that counts - `control.Tuning.saturation_w`. See
|
||||
`MIN_W_IS_SATURATION_W`.
|
||||
|
||||
Returns `{name: sentence}`, empty when there is nothing to say. The sentence
|
||||
is the operator's first line and it carries the peer clause, because "B is
|
||||
not delivering" and "B is not delivering while A is" are different facts and
|
||||
only the second one is this module's conclusion.
|
||||
|
||||
⚠️ THE PROPER-SUBSET RULE IS THE CHECK, NOT A FILTER ON IT. A unit is named
|
||||
only while the units in its state are FEWER than the units that were asked
|
||||
to discharge. Everything asked and nothing explained is a SITE condition -
|
||||
a derate, a curtailment, a grid-code limit, a plant that simply cannot -
|
||||
and this module has no instrument that can tell that from every box being
|
||||
wedged at once. Reporting it anyway would be the saturation-shaped alarm
|
||||
`clamp.py` warns about, produced by the guard written to replace it.
|
||||
|
||||
⚠️ AND THAT IS WHY N=1 IS INERT WITHOUT A ROSTER GUARD. `asked` has at most
|
||||
one element there, `len(asked) < 2` short-circuits, and every input answers
|
||||
`{}`. A `if n_units > 1` in the caller would be a second place the same
|
||||
decision lives; the algebra is the guard, the same way
|
||||
`circulation.circulating_w()` is identically zero at N<=1.
|
||||
"""
|
||||
asked = [u for u in units
|
||||
if u.share_w is not None and float(u.share_w) > 0.0]
|
||||
if len(asked) < 2:
|
||||
return {}
|
||||
|
||||
gaps = {}
|
||||
for u in asked:
|
||||
gap = short_by(u.share_w, u.measured_w)
|
||||
if gap is not None and gap > float(min_w):
|
||||
gaps[u.name] = gap
|
||||
|
||||
unaccounted = {u.name: gaps[u.name] for u in asked
|
||||
if u.name in gaps and u.accounted is None}
|
||||
if not unaccounted or len(unaccounted) == len(asked):
|
||||
return {}
|
||||
|
||||
peers = [u.name for u in asked if u.name not in unaccounted]
|
||||
out = {}
|
||||
for u in asked:
|
||||
if u.name not in unaccounted:
|
||||
continue
|
||||
out[u.name] = (
|
||||
f"{REASON_CODE}: {u.name} is standing at {float(u.share_w):+.0f} W "
|
||||
f"and delivering {float(u.measured_w):+.0f} W - "
|
||||
f"{unaccounted[u.name]:.0f} W of its share is not arriving - while "
|
||||
f"{', '.join(peers)} {'is' if len(peers) == 1 else 'are'} not in "
|
||||
f"that state. Nothing this add-on reads explains it")
|
||||
return out
|
||||
|
||||
|
||||
class DeliveryWatchdog:
|
||||
"""ONE PER INVERTER, evaluated once per control cycle.
|
||||
|
||||
Per unit rather than per site, and that is card requirement 2: the site
|
||||
figure cannot say which box to look at, and naming the box is the whole
|
||||
reason SAFETY-30 split per-unit from site in the first place. It sits beside
|
||||
`Inverter.sign_wd` on the same object, under the same namespaced store keys,
|
||||
for the same reason.
|
||||
|
||||
⚠️ NOT SHARED ACROSS CADENCES, for `SignWatchdog`'s reason: `since` is a
|
||||
wall-clock start time, so two loops at two rates would each reset the
|
||||
other's window.
|
||||
"""
|
||||
|
||||
def __init__(self, cfg: DeliveryConfig | None = None, *,
|
||||
tripped: bool = False, reason: str = ""):
|
||||
self.cfg = cfg or DeliveryConfig()
|
||||
self.since: datetime | None = None
|
||||
# ⚠️ RESTORED FROM THE STORE, NOT DEFAULTED - SAFETY-34, and it is the
|
||||
# requirement the card states in its own words. See the module note.
|
||||
self.tripped = bool(tripped)
|
||||
self.reason = reason if self.tripped else ""
|
||||
|
||||
def arm(self, cfg: DeliveryConfig) -> None:
|
||||
"""Set the window. Nothing else.
|
||||
|
||||
⚠️ IT IS CONFIGURATION, NOT STATE, AND IT DELIBERATELY DOES NOT TOUCH
|
||||
`since`, `tripped` OR `reason` - `SignWatchdog.arm()`'s argument,
|
||||
which was found the expensive way: a detector that cleared its own
|
||||
window when it was told what it was measuring would lose the episode it
|
||||
is in the middle of, every cycle, and would never accumulate a hold.
|
||||
"""
|
||||
self.cfg = cfg
|
||||
|
||||
def evaluate(self, sentence, now: datetime) -> bool:
|
||||
"""Advance one cycle. `sentence` is this unit's entry from `delinquent()`
|
||||
(None when it has none). Returns True while tripped.
|
||||
|
||||
`now` is the caller's clock - `SignWatchdog.evaluate()`'s and
|
||||
`CirculationWatchdog.evaluate()`'s discipline, so this is testable and
|
||||
replayable rather than reading a clock of its own.
|
||||
"""
|
||||
if self.tripped:
|
||||
# Latched. It does not re-evaluate, and unlike SAFETY-02 and
|
||||
# SAFETY-31 that is NOT because the trip removed the evidence -
|
||||
# nothing here changes what is commanded, so the evidence is still
|
||||
# standing. It is because an intermittent unit that resumed must not
|
||||
# erase its own record. See the module note.
|
||||
return True
|
||||
|
||||
if sentence is None:
|
||||
self.since = None
|
||||
return False
|
||||
|
||||
if self.since is None:
|
||||
self.since = now
|
||||
return False
|
||||
|
||||
held = (now - self.since).total_seconds()
|
||||
if held < self.cfg.hold_s:
|
||||
return False
|
||||
|
||||
self.tripped = True
|
||||
self.reason = (f"{sentence} - held for {held:.0f} s continuously "
|
||||
f"(limit: over {self.cfg.hold_s:.0f} s)")
|
||||
return True
|
||||
|
||||
def held_s(self, now: datetime) -> float:
|
||||
"""How long the current shortfall has stood, for the status page.
|
||||
|
||||
Takes the clock rather than reading one, for `SignWatchdog.held_s()`'s
|
||||
reason: an unclocked function is one nothing can test, which is exactly
|
||||
what happened to that one.
|
||||
"""
|
||||
if self.since is None:
|
||||
return 0.0
|
||||
return (now - self.since).total_seconds()
|
||||
|
||||
def reset(self) -> None:
|
||||
self.since = None
|
||||
self.tripped = False
|
||||
self.reason = ""
|
||||
@@ -122,6 +122,7 @@ import math
|
||||
from datetime import datetime
|
||||
|
||||
from .clamp import ClampResult
|
||||
from .delivery import DeliveryConfig, DeliveryWatchdog
|
||||
# SAFETY-37. The freshness policy, and the ONE place its bounds are written.
|
||||
# freshness.py imports nothing from this package, so the direction is one-way.
|
||||
from .freshness import BOUNDS, Freshness
|
||||
@@ -166,6 +167,13 @@ UNIT_A = "inverter-a"
|
||||
# migration mechanical rather than transcribed: `store.migrate_unit_keys()`
|
||||
# reads this same mapping, so a key renamed here cannot silently stop being
|
||||
# migrated. See store.py.
|
||||
# ⚠️ SAFETY-42's PAIR IS DELIBERATELY NOT IN THIS TUPLE, AND THE ABSENCE IS THE
|
||||
# STATEMENT. The values here are the PRE-DEV-02 FLAT key names that
|
||||
# `store.migrate_unit_keys()` moves under a unit; `delivery_tripped` /
|
||||
# `delivery_reason` never existed flat, because the check that writes them
|
||||
# cannot fire at a roster of one (see delivery.py) and a one-inverter install is
|
||||
# the only install that ever had flat keys. Adding them here would make the
|
||||
# migration look for keys that cannot exist and would report them as absent.
|
||||
UNIT_STORE_KEYS = ("sign_tripped", "sign_reason")
|
||||
|
||||
|
||||
@@ -184,7 +192,8 @@ class Inverter:
|
||||
|
||||
def __init__(self, name, *, setpoint_entity="", soc_entity="",
|
||||
batt_entity="", batt_invert=False, max_w=2000.0,
|
||||
bms_soc_floor=None, store=None, sign_cfg=None):
|
||||
bms_soc_floor=None, store=None, sign_cfg=None,
|
||||
delivery_cfg=None):
|
||||
self.name = str(name)
|
||||
self.setpoint_entity = str(setpoint_entity or "")
|
||||
self.soc_entity = str(soc_entity or "")
|
||||
@@ -274,6 +283,25 @@ class Inverter:
|
||||
tripped=bool(data.get(self.key("sign_tripped"), False)),
|
||||
reason=str(data.get(self.key("sign_reason"), "") or ""))
|
||||
|
||||
# --- SAFETY-42, one instance per unit, under NAMESPACED keys ---------
|
||||
# ⚠️ PER UNIT BECAUSE THE CONCLUSION IS ABOUT A UNIT. "This box is not
|
||||
# producing what it was told, while its sibling is" names a device, and
|
||||
# naming the device is card requirement 2 - the site figure cannot say
|
||||
# which one to look at, which is the whole reason SAFETY-30 split
|
||||
# per-unit from site. The site-level REDUCTION that decides whether a
|
||||
# unit qualifies lives in `delivery.delinquent()` and is computed once
|
||||
# per cycle in `Controller.cycle()`; what is per-unit here is the
|
||||
# WINDOW and the LATCH.
|
||||
#
|
||||
# ⚠️ AND IT IS RESTORED FROM THIS UNIT'S OWN KEYS, LIKE `sign_wd`. A
|
||||
# latch held only in memory is not a latch: Home Assistant restarts, the
|
||||
# `persistent_notification` goes with it, and the fault runs on unseen.
|
||||
# That is SAFETY-34, and it is how the 2026-09-02 outage lasted 6 h 25 m.
|
||||
self.delivery_wd = DeliveryWatchdog(
|
||||
delivery_cfg or DeliveryConfig(),
|
||||
tripped=bool(data.get(self.key("delivery_tripped"), False)),
|
||||
reason=str(data.get(self.key("delivery_reason"), "") or ""))
|
||||
|
||||
# -- store ---------------------------------------------------------------
|
||||
def key(self, name: str) -> str:
|
||||
"""This unit's namespaced Store key for `name`. See store.unit_key()."""
|
||||
@@ -436,7 +464,8 @@ def roster_names(opts: dict) -> list:
|
||||
return names
|
||||
|
||||
|
||||
def build_roster(opts: dict, store, sign_cfg=None) -> list:
|
||||
def build_roster(opts: dict, store, sign_cfg=None,
|
||||
delivery_cfg=None) -> list:
|
||||
"""The site's inverters, in roster order.
|
||||
|
||||
TWO SHAPES, AND WHICH ONE IS IN FORCE IS DECIDED BY ONE TEST: is
|
||||
@@ -490,7 +519,7 @@ def build_roster(opts: dict, store, sign_cfg=None) -> list:
|
||||
# mirror exactly as validate.py rule 8 defaults it, so an install
|
||||
# that never set the key reads the same 10 % in both places.
|
||||
bms_soc_floor=_flat_bms_floor(opts),
|
||||
store=store, sign_cfg=sign_cfg)]
|
||||
store=store, sign_cfg=sign_cfg, delivery_cfg=delivery_cfg)]
|
||||
|
||||
roster = []
|
||||
names = roster_names(opts)
|
||||
@@ -517,7 +546,7 @@ def build_roster(opts: dict, store, sign_cfg=None) -> list:
|
||||
# SAFETY-32's own defect written in three lines - the shallower pack
|
||||
# graded against the deeper pack's BMS.
|
||||
bms_soc_floor=_entry_bms_floor(entry),
|
||||
store=store, sign_cfg=sign_cfg))
|
||||
store=store, sign_cfg=sign_cfg, delivery_cfg=delivery_cfg))
|
||||
return roster
|
||||
|
||||
|
||||
|
||||
@@ -38,6 +38,10 @@ from .arbiter import (
|
||||
Claim, Resolution, actuator_for, resolve,
|
||||
)
|
||||
from .control import Tuning, compute, maintenance_charge_floor, peak_at_risk
|
||||
from .delivery import (
|
||||
DELIVERY_OPTION_DEFAULTS, DeliveryConfig, UnitDelivery,
|
||||
account as delivery_account, delinquent,
|
||||
)
|
||||
from .circulation import (
|
||||
CIRC_OPTION_DEFAULTS, CirculationConfig, CirculationWatchdog,
|
||||
circulating_w, opposing,
|
||||
@@ -216,8 +220,17 @@ class Controller:
|
||||
hold_s=float(opts.get("site_sign_disagree_s",
|
||||
SIGN_OPTION_DEFAULTS["site_sign_disagree_s"])),
|
||||
)
|
||||
# SAFETY-42. THE WINDOW A UNIT MAY BE SHORT OF ITS SHARE BEFORE IT IS
|
||||
# NAMED. There is no second key: the WATTS half of that check is
|
||||
# `Tuning.saturation_w`, read at runtime rather than copied - see
|
||||
# `delivery.MIN_W_IS_SATURATION_W`, and CFG-03 on why a fourth copy of a
|
||||
# default is worth avoiding.
|
||||
self.delivery_cfg = delivery_cfg = DeliveryConfig(
|
||||
hold_s=float(opts.get("delivery_hold_s",
|
||||
DELIVERY_OPTION_DEFAULTS["delivery_hold_s"])),
|
||||
)
|
||||
self.store_migration = migrate_unit_keys(store, UNIT_A)
|
||||
self.inverters = build_roster(opts, store, sign_cfg)
|
||||
self.inverters = build_roster(opts, store, sign_cfg, delivery_cfg)
|
||||
# ⚠️ CFG-04. SAY WHICH SHAPE IS IN FORCE, BECAUSE THE OTHER ONE'S KEYS
|
||||
# ARE STILL SITTING IN THE UI. A declared `inverters:` roster reads none
|
||||
# of `soc_entity`, `batt_entity`, `batt_invert` or `max_w`, and an
|
||||
@@ -365,6 +378,32 @@ class Controller:
|
||||
"opposed the meter.", inv.name,
|
||||
inv.sign_wd.reason or "(reason not recorded)")
|
||||
|
||||
# --- SAFETY-42 + SAFETY-34 -------------------------------------------
|
||||
# ⚠️ A LATCH THAT DOES NOT ACT, WHICH IS THE ONE ASYMMETRY WITH EVERY
|
||||
# OTHER RESTORED LATCH ON THIS OBJECT, AND IT IS DELIBERATE. SAFETY-02's
|
||||
# and SAFETY-31's latches survive a restart because they command 0 W and
|
||||
# an add-on that came back armed would drive straight into the fault
|
||||
# again. This one commands nothing at all: it is restored so that the
|
||||
# ALARM survives, which is the whole of SAFETY-34 and is exactly how the
|
||||
# 2026-09-02 outage went unseen for 6 h 25 m with the detector correct
|
||||
# the entire time. `delivery.py` carries the argument for reporting
|
||||
# rather than acting, with the rig measurements on both sides of it.
|
||||
#
|
||||
# `_delivery_announced` is the per-PROCESS half - see `_circ_announced`,
|
||||
# which is the same pattern for the same reason. The store says "this
|
||||
# unit is tripped"; the flag says "this process has put that in front of
|
||||
# a human". A restart clears the flag and keeps the latch.
|
||||
self._delivery_announced = set()
|
||||
for inv in self.inverters:
|
||||
if inv.delivery_wd.tripped:
|
||||
_LOG.error(
|
||||
"SAFETY-42 was TRIPPED on %s before this restart and stays "
|
||||
"tripped: %s. NOTHING IS BEING COMMANDED DIFFERENTLY - this "
|
||||
"guard reports and does not act - so if that inverter is "
|
||||
"still not delivering, the site is still short by its "
|
||||
"share. Restarting does not clear this.",
|
||||
inv.name, inv.delivery_wd.reason or "(reason not recorded)")
|
||||
|
||||
# --- SAFETY-30: THE SITE HALF OF THE SIGN CHECK ----------------------
|
||||
# ⚠️ ONE PER SITE, BESIDE THE ONE PER INVERTER, AND ARMED ONLY AT N>=2.
|
||||
# The per-unit watchdogs above ask "is THIS actuator doing what I told
|
||||
@@ -1738,6 +1777,79 @@ class Controller:
|
||||
"add-on restart AND a Home Assistant restart; restarting will "
|
||||
"not clear it.", self.circ_wd.reason)
|
||||
|
||||
# --- SAFETY-42, AND IT RUNS HERE FOR SAFETY-02's AND SAFETY-31's REASON
|
||||
# Evaluated EVERY cycle, unconditionally, ABOVE the "inputs missing but
|
||||
# not yet stale" hold below. A run of held cycles would otherwise leave
|
||||
# each unit's `since` untouched across the gap and its window would
|
||||
# resume from a start time that predates it - the ordering bug the sign
|
||||
# watchdog's own note describes, in a third detector.
|
||||
#
|
||||
# ⚠️ THE OPERANDS ARE `standing` AND `inv.batt`, THE SAME PAIR SAFETY-30
|
||||
# GRADES, AND THE CONCLUSION IS A DIFFERENT ONE. SAFETY-30 asks whether
|
||||
# they OPPOSE; this asks whether the second one ARRIVED. A unit sitting
|
||||
# at 0 W against a +2000 W share opposes nothing - it is not pushing the
|
||||
# wrong way, it is absent - so the sign gate never opens, the site sum
|
||||
# still agrees with the meter in direction, circulating power is
|
||||
# identically zero, and the only trace left is `reason ==
|
||||
# "saturated-freeze"`, which clamp.py:93-96 documents as the alarm an
|
||||
# operator learns to ignore. That is this card.
|
||||
#
|
||||
# ⚠️ AND `delinquent()` IS CALLED ONCE, OVER THE WHOLE ROSTER, BECAUSE
|
||||
# THE CHECK IS A REDUCTION ACROSS UNITS AND NOT A PER-UNIT PREDICATE.
|
||||
# What separates a wedged box from a legitimately saturated site is that
|
||||
# the site's shortfall is a PROPER SUBSET of the units asked to deliver.
|
||||
# Called per unit it would be a shortfall test, which is the saturation
|
||||
# signal again. delivery.py carries the measurement: a pack at its own
|
||||
# cut-out and a wedged unit produce IDENTICAL traces at the unit and at
|
||||
# the meter, so nothing computed from one unit alone can tell them
|
||||
# apart. At a roster of one the reduction is empty for every input.
|
||||
for inv in self.inverters:
|
||||
inv.delivery_wd.arm(self.delivery_cfg)
|
||||
_delinquent = delinquent(
|
||||
[UnitDelivery(
|
||||
inv.name, standing[inv.name], inv.batt,
|
||||
delivery_account(
|
||||
# ⚠️ `is_live()` AT ITS OWN DEFAULT WINDOW, which is
|
||||
# `LINK_STALE_S` - the SAME call `live_units()` makes for
|
||||
# the allocator. It has to be: what this branch means is
|
||||
# "the allocator has already dropped this unit and said so",
|
||||
# and a different window here would be this check answering
|
||||
# a question about a roster the allocator did not build.
|
||||
live=inv.is_live(now)[0],
|
||||
parked=inv.sign_wd.tripped,
|
||||
soc=inv.soc, bms_floor=inv.bms_soc_floor))
|
||||
for inv in self.inverters],
|
||||
self.tuning.saturation_w)
|
||||
for inv in self.inverters:
|
||||
was_short = inv.delivery_wd.tripped
|
||||
if inv.delivery_wd.evaluate(_delinquent.get(inv.name), now) \
|
||||
and not was_short:
|
||||
# REASON FIRST, FLAG SECOND, and each `set` writes the file -
|
||||
# the same atomicity ordering the three trips above use: a crash
|
||||
# between the two must not persist a latch with no explanation
|
||||
# for the operator who has to decide about clearing it.
|
||||
#
|
||||
# ⚠️ AND THERE IS NO FLAT MIRROR OF THIS PAIR, WHERE SAFETY-02
|
||||
# HAS ONE. store.py's rule 2 keeps the flat keys "so a rollback
|
||||
# still finds its latch", and the version being rolled back TO
|
||||
# drives exactly ONE actuator - a roster at which this check is
|
||||
# inert by construction. A flat mirror would hand a downgraded
|
||||
# add-on a latch taken on evidence about a roster it cannot have.
|
||||
inv.store_set("delivery_reason", inv.delivery_wd.reason)
|
||||
inv.store_set("delivery_tripped", True)
|
||||
self._delivery_announced.discard(inv.name)
|
||||
self.log_event(f"NOT DELIVERING {inv.name}: "
|
||||
+ inv.delivery_wd.reason)
|
||||
_LOG.error(
|
||||
"SAFETY-42 TRIPPED on %s - %s. NOTHING IS COMMANDED "
|
||||
"DIFFERENTLY: this guard reports and does not act, because "
|
||||
"commanding a unit that is not listening achieves nothing "
|
||||
"(HW-01) and the only re-allocation this add-on can express "
|
||||
"would throw away a partially-delivering unit's remaining "
|
||||
"output. The site is short by that unit's share until "
|
||||
"somebody looks at it. This survives an add-on AND a Home "
|
||||
"Assistant restart.", inv.name, inv.delivery_wd.reason)
|
||||
|
||||
stale_after = float(self.o.get("stale_input_s", 15))
|
||||
# ⚠️ `any`, NOT the sole unit's. The hold exists so a single missed poll
|
||||
# does not become a fault; a LATCHED inverter must still reach the claim
|
||||
@@ -2458,6 +2570,15 @@ class Controller:
|
||||
# fresh trip instead of getting its own.
|
||||
await self.check_circulating()
|
||||
|
||||
# --- SAFETY-42's alarm, and its boundary is the LATCH, like SAFETY-31's
|
||||
# It grades `inv.delivery_wd.tripped`, which was advanced near the top
|
||||
# of this method beside SAFETY-02 and SAFETY-31 and which nothing since
|
||||
# has touched. Called on EVERY cycle rather than only on the cycle that
|
||||
# trips, because its second job is re-raising the alarm after a restart
|
||||
# into an existing trip - see its docstring on why that shares one code
|
||||
# path with the fresh trip instead of getting its own.
|
||||
await self.check_not_delivering()
|
||||
|
||||
# --- SAFETY-07's alarm, and it is NOT the engagement ------------------
|
||||
# Its boundary is unconstrained: it grades `self.peak_guard`, which this
|
||||
# method assigned above and nothing since has touched.
|
||||
@@ -2795,6 +2916,85 @@ class Controller:
|
||||
f"know why two inverters were opposing."),
|
||||
})
|
||||
|
||||
async def check_not_delivering(self) -> None:
|
||||
"""SAFETY-42 + SAFETY-34. The alarm, and it must survive a restart.
|
||||
|
||||
⚠️ THE LATCH IS NOT THE ALARM, and this guard has nothing BUT the alarm.
|
||||
`unit/<name>/delivery_tripped` in `/data/state.json` survives everything,
|
||||
but nothing looks at a JSON file at 3 a.m. What the operator sees is a
|
||||
`persistent_notification`, and THAT IS IN MEMORY: a Home Assistant
|
||||
restart deletes it while the fault runs on. That is precisely how the
|
||||
2026-09-02 outage went unseen for 6 h 25 m with the detector correct the
|
||||
whole time (SAFETY-34), and it matters more here than anywhere else in
|
||||
this file, because SAFETY-02 and SAFETY-31 at least leave a stopped site
|
||||
for somebody to notice. This one leaves a site that looks like it is
|
||||
working and is quietly delivering one unit less than it has.
|
||||
|
||||
So the durable half and the visible half are joined HERE, and the join
|
||||
is `_delivery_announced`: a per-PROCESS set of unit names, deliberately
|
||||
not persisted. The latch says "this unit is tripped"; membership says
|
||||
"this process has put that in front of a human". A restart empties the
|
||||
set and keeps the latch, so the notification is RE-RAISED on the first
|
||||
cycle after every add-on start - which is also the first cycle after a
|
||||
Home Assistant restart, because the add-on is restarted with it.
|
||||
|
||||
⚠️ ONE CODE PATH FOR BOTH, ON PURPOSE - `check_circulating()`'s
|
||||
argument. A fresh trip and a restart into an existing trip raise the
|
||||
alarm through these same lines; two paths would be two chances for the
|
||||
restart one to rot unnoticed, and the restart one is the case nobody
|
||||
exercises by hand.
|
||||
|
||||
⚠️ AND ONE NOTIFICATION ID PER UNIT. Every other alarm in this file has
|
||||
a single fixed id because its subject is the site; this one's subject is
|
||||
a named box, and two wedged inverters must produce two cards rather than
|
||||
one replacing the other. The id is built from the same `unit_slugs()`
|
||||
TEL-07 announces the MQTT entities under, so the card and the entity an
|
||||
operator correlates it with carry the same token.
|
||||
"""
|
||||
names = self.units
|
||||
slugs = unit_slugs(names)
|
||||
for inv, slug in zip(self.inverters, slugs):
|
||||
if not inv.delivery_wd.tripped or inv.name in self._delivery_announced:
|
||||
continue
|
||||
# Added BEFORE the await, for `check_circulating()`'s reason:
|
||||
# `hass.call()` can raise or hang, and a retry every cycle for a
|
||||
# broker that is down would be an alarm storm on the event bus. The
|
||||
# latch is already durable and the log line already fired; this
|
||||
# surface is best-effort by design.
|
||||
self._delivery_announced.add(inv.name)
|
||||
peers = ", ".join(n for n in names if n != inv.name) or "no sibling"
|
||||
await self.hass.call("persistent_notification", "create", {
|
||||
"notification_id": f"goodwe_not_delivering_{slug}",
|
||||
"title": f"GoodWe: {inv.name} is not delivering its share",
|
||||
"message": (
|
||||
f"{inv.delivery_wd.reason}\n\nWHY NOTHING ELSE SAID SO. A "
|
||||
f"unit sitting at 0 W against a non-zero share is not "
|
||||
f"pushing the wrong way, it is ABSENT: the per-unit sign "
|
||||
f"check needs opposition on both sides of "
|
||||
f"{self.unit_sign_cfg.min_w:.0f} W and never opens, the "
|
||||
f"site sum still agrees with the meter in "
|
||||
f"direction, and circulating power is exactly zero when one "
|
||||
f"unit does nothing. The only other trace is a saturation "
|
||||
f"freeze, which this add-on's own code calls an alarm "
|
||||
f"operators learn to ignore.\n\nNOTHING HAS BEEN COMMANDED "
|
||||
f"DIFFERENTLY. This guard reports and does not act. "
|
||||
f"Commanding an inverter that has stopped listening "
|
||||
f"achieves nothing - that has happened on this hardware, "
|
||||
f"once for 6 h 25 m - and the only re-allocation this "
|
||||
f"add-on can express is all-or-nothing, which would throw "
|
||||
f"away the output of a unit that is delivering PART of its "
|
||||
f"share. So {peers} is still regulating and the site is "
|
||||
f"short by up to {inv.rating_w():.0f} W until somebody "
|
||||
f"looks at {inv.name}.\n\nWHAT TO CHECK, in order: is "
|
||||
f"{inv.name} answering on RS485 at all; is its pack colder "
|
||||
f"or emptier than its configured bms_soc_floor says (if it "
|
||||
f"is, that setting does not describe this battery and this "
|
||||
f"alarm is telling you so); and is anything else writing to "
|
||||
f"it.\n\nThis alarm survives a Home Assistant restart. "
|
||||
f"Restarting will not clear it; start control from the "
|
||||
f"add-on UI once you know why."),
|
||||
})
|
||||
|
||||
async def check_meter_frozen(self) -> None:
|
||||
"""SAFETY-18. Alarm on a meter that answers forever without changing.
|
||||
|
||||
@@ -3557,6 +3757,49 @@ class Controller:
|
||||
+ (f" - now {_circ:.0f} W" if _circ else "")
|
||||
+ (f", opposed for {held:.0f} s" if held else "")})
|
||||
|
||||
# --- SAFETY-42 -------------------------------------------------------
|
||||
# ⚠️ PER INVERTER, AND WARN RATHER THAN BAD, WHICH IS THE ONE PLACE THE
|
||||
# "reports, does not act" DECISION IS VISIBLE TO AN OPERATOR. Every
|
||||
# other latched row in this list means the site is at 0 W and a red
|
||||
# cross is the truth. This one means the site is STILL RUNNING and is
|
||||
# quietly short by one unit's share; rendering it as a hard failure
|
||||
# would say the controller had stopped, which it has not. It is not "ok"
|
||||
# either - `ok: False` with `warn: True` is this list's caution, and a
|
||||
# caution is what a fault nobody is acting on actually is.
|
||||
if len(self.inverters) > 1:
|
||||
for inv in self.inverters:
|
||||
tag = f" ({inv.name})"
|
||||
if inv.delivery_wd.tripped:
|
||||
out.append({"ok": False, "warn": True,
|
||||
"text": f"NOT DELIVERING{tag} - "
|
||||
+ inv.delivery_wd.reason
|
||||
+ ". Nothing has been commanded differently and "
|
||||
"the rest of the roster is still regulating, "
|
||||
"so the site is short by this unit's share. "
|
||||
"Starting control clears this row; restarting "
|
||||
"the add-on does NOT, and neither does "
|
||||
"restarting Home Assistant."})
|
||||
else:
|
||||
_dheld = inv.delivery_wd.held_s(datetime.now(timezone.utc))
|
||||
out.append({"ok": True, "warn": False,
|
||||
"text": f"delivery watchdog{tag}: armed (reports after "
|
||||
f"{inv.delivery_wd.cfg.hold_s:.0f} s short of "
|
||||
f"its share by over "
|
||||
f"{self.tuning.saturation_w:.0f} W, with a "
|
||||
f"sibling that is not)"
|
||||
+ (f" - short for {_dheld:.0f} s now"
|
||||
if _dheld else "")})
|
||||
else:
|
||||
# AC 5, and the same honesty `circulating power: n/a` uses one block
|
||||
# up. At one inverter the reduction this check is has no proper
|
||||
# subset to find, for EVERY input, so "armed" would be a claim to be
|
||||
# watching something that cannot happen.
|
||||
out.append({"ok": True, "warn": False,
|
||||
"text": "delivery watchdog: n/a at one inverter - a unit "
|
||||
"short of its share is indistinguishable from a "
|
||||
"plant that cannot deliver until there is a "
|
||||
"sibling to compare it with"})
|
||||
|
||||
if not o.get("estop_fitted", False):
|
||||
out.append({"ok": False, "warn": True,
|
||||
"text": "no e-stop fitted: if this host dies the battery stays "
|
||||
@@ -3616,6 +3859,26 @@ class Controller:
|
||||
level, banner = "bad", f"NOT READY — {bad[0].get('banner') or bad[0]['text']}"
|
||||
elif not self.auto:
|
||||
level, banner = "warn", "Stopped — inverter commanded to 0 W"
|
||||
elif any(i.delivery_wd.tripped for i in self.inverters):
|
||||
# ⚠️ SAFETY-42, AND IT IS A `warn` BANNER RATHER THAN A `bad` ONE,
|
||||
# WHICH IS THE WHOLE "reports, does not act" DECISION IN ONE LINE.
|
||||
# Every `bad` banner above means the site is at 0 W; this one means
|
||||
# the site is STILL RUNNING and quietly short by one unit's share,
|
||||
# and a red headline would say the controller had stopped.
|
||||
#
|
||||
# ⚠️ BUT IT MUST DISPLACE "Running — holding grid at zero", AND THAT
|
||||
# IS NOT COSMETIC. That sentence is a claim the site cannot make
|
||||
# while a unit is absent - the loop is saturated and the meter is
|
||||
# short - and "the page reads healthy while the fault runs" is
|
||||
# precisely the shape this whole card exists to remove. It sits
|
||||
# BELOW "Stopped" because a stopped site is the more urgent fact and
|
||||
# the banner carries one sentence.
|
||||
_nd = ", ".join(i.name for i in self.inverters
|
||||
if i.delivery_wd.tripped)
|
||||
level, banner = "warn", (
|
||||
f"NOT DELIVERING — {_nd} is standing at a share it is not "
|
||||
f"producing. The site is still running and short by it; "
|
||||
f"nothing has been commanded differently.")
|
||||
elif self.maint.phase != IDLE:
|
||||
# ⚠️ THE PACK IS NAMED ONLY WHEN THERE IS MORE THAN ONE. At a roster
|
||||
# of one this is the string it has always been, character for
|
||||
@@ -3766,6 +4029,27 @@ class Controller:
|
||||
self.store.set("circ_reason", "")
|
||||
self.log_event("circulating-power trip cleared by operator")
|
||||
self._circ_announced = False
|
||||
# SAFETY-42, cleared by the SAME press, and for a reason that is
|
||||
# NOT the one above. The other three latches are cleared here
|
||||
# because their trip removed its own evidence - a site commanding
|
||||
# 0 W cannot oppose anything - so only a human can say the fault
|
||||
# is gone. This one commands nothing, so its evidence is still
|
||||
# standing and it could in principle clear itself. It does not:
|
||||
# a unit that stopped delivering for minutes and then resumed is
|
||||
# an INTERMITTENT fault, and a detector that erased its own
|
||||
# record of that would be the least useful thing in this file.
|
||||
# ⚠️ AND `_delivery_announced` GOES WITH IT, like
|
||||
# `_circ_announced`: leaving a name in it through a clear would
|
||||
# mean the NEXT trip on that unit latched and raised no
|
||||
# notification at all.
|
||||
for inv in self.inverters:
|
||||
if inv.delivery_wd.tripped:
|
||||
self.log_event(f"not-delivering report on {inv.name} "
|
||||
f"cleared by operator")
|
||||
inv.delivery_wd.reset()
|
||||
inv.store_set("delivery_tripped", False)
|
||||
inv.store_set("delivery_reason", "")
|
||||
self._delivery_announced = set()
|
||||
if not self.auto:
|
||||
# ⚠️ THE WHOLE SITE, TARGETS INCLUDED - see command_site_zero().
|
||||
# Zeroing `self.target` alone would leave each unit's last share
|
||||
|
||||
@@ -511,6 +511,48 @@ options:
|
||||
circulation_w: 700
|
||||
circulation_hold_s: 75
|
||||
|
||||
# ⚠️ CFG-03. THIS LINE IS THE SOURCE OF TRUTH FOR THIS DEFAULT, and
|
||||
# `app/delivery.py`'s `DELIVERY_OPTION_DEFAULTS` is the ONE in-code copy -
|
||||
# the Dockerfile does not put this file in the image, so the running add-on
|
||||
# cannot read it. test_delivery.py pins the copy by READING this line, so a
|
||||
# drift goes red in CI rather than shipping. Two places - options:, schema: -
|
||||
# plus that one dict, and no fourth.
|
||||
#
|
||||
# ⚠️ THERE IS NO `delivery_w` TO GO WITH IT, AND THAT IS DELIBERATE. The watts
|
||||
# half of this check is `saturation_w`, the option two blocks down, read
|
||||
# directly: it is already the gap at which the control law itself concludes
|
||||
# the inverter is not following, and a second key would be a fourth place a
|
||||
# threshold has to be kept in step and a way for the two to disagree.
|
||||
#
|
||||
# How long ONE inverter may stand at a discharge share it is not producing -
|
||||
# short by more than `saturation_w` - while at least one sibling is NOT in
|
||||
# that state, before the add-on names it. It REPORTS: nothing is commanded
|
||||
# differently, the loop is untouched, and the site goes on running short by
|
||||
# that unit's share until somebody looks. app/delivery.py carries why, with
|
||||
# the rig measurements on both sides of the decision.
|
||||
#
|
||||
# Ignored at one inverter, where the check is inert by construction: with no
|
||||
# sibling, "this box cannot deliver" and "this plant cannot deliver" are the
|
||||
# same observation.
|
||||
# ⚠️ MEASURED ON THE TEST-19 TWO-MASTER RIG, NOT CHOSEN. 2640 closed-loop runs
|
||||
# at a roster of two in which nothing was faulty, over 6 gains x 5 slews - the
|
||||
# whole range this schema accepts - across load reversals, a flapping load and
|
||||
# ASYMMETRIC SPLITS, plus 1890 runs at a roster of ONE as the control. At the
|
||||
# shipped threshold the worst CONTINUOUS legitimate episode is 30.0 s (the
|
||||
# weighted family; 20.0 s on reversals, 0.0 s on flapping, and 0.0 s at every
|
||||
# threshold in all 1890 one-inverter runs). 185 s is 6.2x that.
|
||||
#
|
||||
# ⚠️ AND THE MARGIN IS SIZED ON A MEASUREMENT, NOT ON A FEELING, BECAUSE
|
||||
# INVERTER B DOES NOT PHYSICALLY EXIST. The rig's B is A copied. Sweeping a
|
||||
# DIFFERENT second unit through that campaign: the worst legitimate episode is
|
||||
# insensitive to B's ramp (30.0 s at half A's ramp) and scales with its LAG -
|
||||
# 45.0 s at twice A's measured 10 s lag, 90.0 s at four times it. 185 s is
|
||||
# 2.06x even the four-times-slower case.
|
||||
#
|
||||
# DELIBERATELY NOT 180 - that is SAFETY-02's site hold and this is not it -
|
||||
# and not 150 or 200. 37 control cycles.
|
||||
delivery_hold_s: 185
|
||||
|
||||
# ==========================================================================
|
||||
# ADVANCED - the control law and diagnostics
|
||||
# ==========================================================================
|
||||
@@ -772,6 +814,16 @@ schema:
|
||||
# loop must be caught and acted on well inside one.
|
||||
circulation_hold_s: int(30,900)
|
||||
|
||||
# SAFETY-42. The floor of 30 is `inverter.LINK_STALE_S`, for
|
||||
# `circulation_hold_s`'s reason and the same one SAFETY-30's per-unit window
|
||||
# takes it from: below the firmware's own veto window a unit not executing its
|
||||
# share is a LINK fault the add-on already answers by dropping it from the
|
||||
# split and saying so in the action log, and reporting inside that window
|
||||
# would be a second, louder alarm about a state already being handled. The
|
||||
# ceiling of 900 is the quarter that bills the capacity tariff - a site
|
||||
# running short of one unit's share must be reported well inside one.
|
||||
delivery_hold_s: int(30,900)
|
||||
|
||||
# ==========================================================================
|
||||
# ADVANCED - the control law and diagnostics
|
||||
# ==========================================================================
|
||||
|
||||
@@ -0,0 +1,787 @@
|
||||
"""Runnable check for SAFETY-42 - a unit that stops delivering its share.
|
||||
`python3 test_delivery.py`
|
||||
|
||||
⚠️ THE FIXTURE IS THE CARD, AND IT IS PART 4. What this guard has to do is tell
|
||||
a WEDGED unit from a COLD PACK AT ITS FLOOR, and those two are IDENTICAL in the
|
||||
power domain - the same shares, the same measured watts, the same meter, the
|
||||
same `saturated-freeze`. Measured on admin/goodwe's two-master rig over a
|
||||
120-cycle run: the two arms agree on every one of 120 cycles in (grid, site
|
||||
target, per-unit share, per-unit measured power), to the last emitted watt. If
|
||||
this file's two fixtures ever stop differing, the check has stopped being
|
||||
graded on the only thing that separates the two cases.
|
||||
|
||||
⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run
|
||||
`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it
|
||||
stops bytecode being WRITTEN, not a stale one being READ. That is TEST-15, and
|
||||
it has already turned one live mutation into a false survivor. Read MUTATION.md
|
||||
before you report a survivor.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import io
|
||||
import re
|
||||
import sys
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from app.control import Tuning
|
||||
from app.delivery import (
|
||||
DELIVERY_OPTION_DEFAULTS, REASON_CODE, DeliveryConfig, DeliveryWatchdog,
|
||||
UnitDelivery, account, delinquent, short_by,
|
||||
)
|
||||
from app.inverter import Inverter
|
||||
from app.main import Controller
|
||||
from app.peakguard import discharge_permitted
|
||||
from app.store import unit_key
|
||||
|
||||
fails = []
|
||||
T0 = datetime(2026, 9, 2, 12, 0, 0, tzinfo=timezone.utc)
|
||||
|
||||
|
||||
def check(name, cond):
|
||||
print(f" {'ok ' if cond else 'FAIL'} {name}")
|
||||
if not cond:
|
||||
fails.append(name)
|
||||
|
||||
|
||||
def U(name, share, meas, why=None):
|
||||
return UnitDelivery(name, share, meas, why)
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
print("=" * 72)
|
||||
print("1. the quantity, and the reduction that makes it readable")
|
||||
print("=" * 72)
|
||||
|
||||
check("short_by is the watts of a DISCHARGE share that did not arrive",
|
||||
short_by(2000, 0) == 2000.0 and short_by(2000, 1400) == 600.0)
|
||||
check("...delivering the share, or more than it, is not a shortfall - both "
|
||||
"answer None rather than a number a caller would have to sign-test",
|
||||
short_by(2000, 2000) is None and short_by(2000, 2600) is None)
|
||||
check("...a CHARGE share answers None. There is no configured charge ceiling "
|
||||
"in this add-on, so a pack in constant-voltage taper cannot be explained "
|
||||
"and arming it would fire on every maintenance charge",
|
||||
short_by(-2000, 0) is None and short_by(-2000, -100) is None)
|
||||
check("...a missing reading is None, never 'the whole share'. Zero would be "
|
||||
"this check inventing a fault out of an absence",
|
||||
short_by(2000, None) is None and short_by(None, 0) is None)
|
||||
|
||||
print()
|
||||
print(" `account()` - the ONLY thing that separates a wedged unit from a")
|
||||
print(" cold pack, because nothing in the power domain does")
|
||||
check("a pack at its own cut-out accounts for its shortfall",
|
||||
account(live=True, parked=False, soc=10.0, bms_floor=10.0) is not None)
|
||||
check("...and a pack above it does not",
|
||||
account(live=True, parked=False, soc=10.1, bms_floor=10.0) is None)
|
||||
check("...the boundary is `<=`, the same one discharge_permitted() draws, "
|
||||
"BECAUSE IT IS discharge_permitted() - not a transcription of it",
|
||||
(account(live=True, parked=False, soc=10.0, bms_floor=10.0) is None)
|
||||
== (discharge_permitted(10.0, 10.0) is None)
|
||||
and (account(live=True, parked=False, soc=10.1, bms_floor=10.0) is None)
|
||||
== (discharge_permitted(10.1, 10.0) is None))
|
||||
check("a unit the allocator has already dropped accounts for itself - firing "
|
||||
"inside a state already being handled is the SAFETY-15/19 failure",
|
||||
account(live=False, parked=False, soc=50.0, bms_floor=10.0) is not None)
|
||||
check("a unit parked at 0 W by a sign trip accounts for itself: not "
|
||||
"delivering is compliance",
|
||||
account(live=True, parked=True, soc=50.0, bms_floor=10.0) is not None)
|
||||
check("EVERY UNKNOWN FAILS TOWARD SILENCE - an unreadable SoC and an unstated "
|
||||
"floor both account for the shortfall, which is the OPPOSITE direction "
|
||||
"to discharge_permitted()'s own caller and is the right one here: an "
|
||||
"alarm nobody can justify is the crying-wolf failure this file exists "
|
||||
"to avoid",
|
||||
account(live=True, parked=False, soc=None, bms_floor=10.0) is not None
|
||||
and account(live=True, parked=False, soc=50.0, bms_floor=None) is not None)
|
||||
print(f" e.g. {account(live=True, parked=False, soc=9.0, bms_floor=10.0)!r}")
|
||||
|
||||
print()
|
||||
print(" the PROPER-SUBSET rule - what keeps a site-wide cause out")
|
||||
_wedged = delinquent([U("a", 2000, 2000), U("b", 2000, 0)], 500)
|
||||
check("one unit short and unexplained beside a sibling that is not: NAMED",
|
||||
list(_wedged) == ["b"])
|
||||
check("...and the sentence names the unit, the two watt figures and the peer, "
|
||||
"because 'B is not delivering' and 'B is not delivering while A is' are "
|
||||
"different facts and only the second one is this check's conclusion",
|
||||
REASON_CODE in _wedged["b"] and "+2000 W" in _wedged["b"]
|
||||
and "+0 W" in _wedged["b"] and "a" in _wedged["b"])
|
||||
print(f" {_wedged['b']}")
|
||||
_cold = delinquent([U("a", 2000, 2000), U("b", 2000, 0, "its pack is empty")], 500)
|
||||
check("THE CARD: the same numbers, with the shortfall ACCOUNTED FOR, name "
|
||||
"nobody. This pair of assertions is the whole guard",
|
||||
_cold == {})
|
||||
_both = delinquent([U("a", 2000, 0), U("b", 2000, 0)], 500)
|
||||
check("EVERY asked unit short and unexplained: NOBODY is named. That is a "
|
||||
"site condition - a derate, a curtailment, a plant that cannot - and "
|
||||
"this check has no instrument that tells it from every box being wedged "
|
||||
"at once. Reporting it would be the saturation-shaped alarm again",
|
||||
_both == {})
|
||||
_three = delinquent([U("a", 2000, 2000), U("b", 2000, 0), U("c", 2000, 0)], 500)
|
||||
check("...but a proper subset of THREE names both of its members",
|
||||
sorted(_three) == ["b", "c"])
|
||||
_mixed = delinquent([U("a", 2000, 0, "at its floor"), U("b", 2000, 0)], 500)
|
||||
check("a wedged unit beside a pack that is legitimately empty still names the "
|
||||
"wedged one: `asked` is the denominator, not `short`",
|
||||
list(_mixed) == ["b"])
|
||||
check("a shortfall AT the threshold is not one - the bound is strict, so a "
|
||||
"unit exactly on the gap the control law calls ordinary tracking is not "
|
||||
"reported",
|
||||
delinquent([U("a", 2000, 2000), U("b", 2000, 1500)], 500) == {}
|
||||
and list(delinquent([U("a", 2000, 2000), U("b", 2000, 1499)], 500)) == ["b"])
|
||||
check("a unit given no share is not asked and cannot be named, and it is not "
|
||||
"a peer either - with only one unit asked there is nothing to compare",
|
||||
delinquent([U("a", 0, 0), U("b", 2000, 0)], 500) == {})
|
||||
|
||||
print()
|
||||
print(" AC 5: AT ONE INVERTER IT IS INERT, AND BY ALGEBRA RATHER THAN BY A")
|
||||
print(" ROSTER GUARD - `len(asked) < 2` is unreachable-by-construction there")
|
||||
_n1 = [delinquent([U("a", s, m)], 500)
|
||||
for s in (0, 500, 2000, 5000, -2000, None)
|
||||
for m in (0, 250, 2000, -2000, None)]
|
||||
check(f"every one of {len(_n1)} single-unit rosters over the whole "
|
||||
f"(share, measured) grid answers {{}}", all(r == {} for r in _n1))
|
||||
check("...including a single unit that IS short and unexplained, which is the "
|
||||
"case a caller might expect to fire. It must not: with no sibling, "
|
||||
"'this box cannot' and 'this plant cannot' are the same observation",
|
||||
delinquent([U("a", 2000, 0)], 500) == {})
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
print()
|
||||
print("=" * 72)
|
||||
print("2. CFG-03: config.yaml is the source of truth and there is ONE copy")
|
||||
print("=" * 72)
|
||||
_CFG_TEXT = io.open("config.yaml", encoding="utf-8").read()
|
||||
|
||||
|
||||
def read_option(key, block="options"):
|
||||
"""The value of `key` inside one top-level block of config.yaml, as text.
|
||||
|
||||
⚠️ READ OUT OF THE FILE, NOT TRANSCRIBED. CFG-03's whole complaint is that a
|
||||
default lives in more than one place; a check that hard-coded the number
|
||||
here would be a THIRD copy, and the one that never goes red.
|
||||
|
||||
⚠️ ANCHORED AT EXACTLY TWO SPACES, WHICH IS CFG-04's LESSON AND NOT A STYLE
|
||||
CHOICE. a leading-whitespace pattern matches a PER-INVERTER `max_w` nested under
|
||||
`inverters:` as readily as the site one, first hit wins, and a reader that
|
||||
quietly answers inverter #1's number is the defect `test_inverter.py`'s
|
||||
config.yaml read-closure audit exists to hunt. This function is in that
|
||||
audit's table by signature - first argument `key` - so it is fed a
|
||||
two-inverter config and graded on what it answers.
|
||||
"""
|
||||
body = _CFG_TEXT.split(f"\n{block}:", 1)[1]
|
||||
if block == "options":
|
||||
body = body.split("\nschema:", 1)[0]
|
||||
m = re.search(rf"^ {re.escape(key)}:\s*(\S+)\s*$", body, re.M)
|
||||
return m.group(1) if m else None
|
||||
|
||||
|
||||
check("`delivery_hold_s` is in config.yaml's options: block",
|
||||
read_option("delivery_hold_s") is not None)
|
||||
check("...and DELIVERY_OPTION_DEFAULTS agrees with it, so the running add-on - "
|
||||
"which cannot read config.yaml, the Dockerfile does not ship it - uses "
|
||||
"the number an operator sees on the Configuration page",
|
||||
int(read_option("delivery_hold_s"))
|
||||
== int(DELIVERY_OPTION_DEFAULTS["delivery_hold_s"]))
|
||||
check("...and it has a schema: entry, so the Supervisor refuses a value "
|
||||
"outside the range the derivation answers for",
|
||||
read_option("delivery_hold_s", "schema") is not None)
|
||||
_schema = read_option("delivery_hold_s", "schema")
|
||||
_lo, _hi = (int(x) for x in re.match(r"int\((\d+),(\d+)\)", _schema).groups())
|
||||
check(f"the schema floor is inverter.LINK_STALE_S = 30 s ({_schema}): below "
|
||||
f"the firmware's own veto window a unit not executing its share is a "
|
||||
f"LINK fault the add-on already answers by dropping it from the split",
|
||||
_lo == 30)
|
||||
check("the schema ceiling is the 900 s quarter that bills the capacity tariff "
|
||||
"- a site running short of one unit's share must be reported well "
|
||||
"inside one", _hi == 900)
|
||||
check("the shipped default is inside its own schema",
|
||||
_lo <= int(DELIVERY_OPTION_DEFAULTS["delivery_hold_s"]) <= _hi)
|
||||
check("THERE IS NO SECOND KEY. The watts half is `Tuning.saturation_w`, read "
|
||||
"at runtime, so it cannot drift from the number the control law itself "
|
||||
"calls 'not following' - CFG-03 already tracks three places a default "
|
||||
"has to be kept in step and this card does not earn a fourth",
|
||||
list(DELIVERY_OPTION_DEFAULTS) == ["delivery_hold_s"]
|
||||
and not re.search(r"^\s+delivery_w:", _CFG_TEXT, re.M))
|
||||
print(f" delivery_hold_s: {DELIVERY_OPTION_DEFAULTS['delivery_hold_s']} s, "
|
||||
f"schema {_schema}; watts = Tuning.saturation_w = {Tuning().saturation_w:.0f} W")
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
print()
|
||||
print("=" * 72)
|
||||
print("3. the window and the latch")
|
||||
print("=" * 72)
|
||||
|
||||
|
||||
def drive(seconds, cfg=None, sentence="short", step=5):
|
||||
wd = DeliveryWatchdog(cfg or DeliveryConfig(hold_s=60.0))
|
||||
at = None
|
||||
for t in range(0, int(seconds) + 1, step):
|
||||
if wd.evaluate(sentence, T0 + timedelta(seconds=t)) and at is None:
|
||||
at = t
|
||||
return wd, at
|
||||
|
||||
|
||||
_wd, _at = drive(300)
|
||||
check("a shortfall held past the window reports", _wd.tripped)
|
||||
check("...at the window, not before", _at == 60)
|
||||
check("...and the reason carries the sentence AND how long it stood",
|
||||
"short" in _wd.reason and "held for" in _wd.reason and "60 s" in _wd.reason)
|
||||
print(f" {_wd.reason!r}")
|
||||
_wd2, _at2 = drive(55)
|
||||
check("a shortfall shorter than the window reports nothing",
|
||||
not _wd2.tripped and _at2 is None)
|
||||
|
||||
_wd3 = DeliveryWatchdog(DeliveryConfig(hold_s=60.0))
|
||||
for _t in range(0, 300, 5):
|
||||
_wd3.evaluate("short" if (_t // 5) % 2 == 0 else None,
|
||||
T0 + timedelta(seconds=_t))
|
||||
check("an INTERMITTENT shortfall never accumulates: any clean cycle clears the "
|
||||
"window, so this needs a CONTINUOUS episode the way SAFETY-02 and "
|
||||
"SAFETY-31 do", not _wd3.tripped and _wd3.since is None)
|
||||
|
||||
_wd4, _ = drive(300)
|
||||
_before = _wd4.reason
|
||||
_wd4.evaluate(None, T0 + timedelta(seconds=999))
|
||||
check("it LATCHES - and not for SAFETY-02's reason. Nothing here removes the "
|
||||
"evidence, because nothing here changes what is commanded; it latches so "
|
||||
"that a unit which stopped delivering for minutes and then resumed "
|
||||
"cannot erase its own record",
|
||||
_wd4.tripped and _wd4.reason == _before)
|
||||
_wd4.reset()
|
||||
check("...and reset() is the operator's press, which clears everything",
|
||||
not _wd4.tripped and _wd4.reason == "" and _wd4.since is None)
|
||||
|
||||
_wd5 = DeliveryWatchdog(DeliveryConfig(hold_s=60.0))
|
||||
_wd5.evaluate("short", T0)
|
||||
_wd5.evaluate("short", T0 + timedelta(seconds=5))
|
||||
_since = _wd5.since
|
||||
_wd5.arm(DeliveryConfig(hold_s=120.0))
|
||||
check("arm() is CONFIGURATION, NOT STATE: it does not touch the window, the "
|
||||
"latch or the reason. A detector that cleared its own window every cycle "
|
||||
"when it was told what it was measuring would never accumulate a hold - "
|
||||
"that is SignWatchdog.arm()'s lesson, found the expensive way",
|
||||
_wd5.since == _since and _wd5.cfg.hold_s == 120.0 and not _wd5.tripped)
|
||||
check("held_s takes the caller's clock rather than reading one",
|
||||
_wd5.held_s(T0 + timedelta(seconds=65)) == 65.0)
|
||||
_wd6 = DeliveryWatchdog(DeliveryConfig(hold_s=60.0), tripped=True, reason="from disk")
|
||||
check("a latch RESTORED from the store comes up tripped, with its reason",
|
||||
_wd6.tripped and _wd6.reason == "from disk")
|
||||
check("...and a reason with no latch behind it is dropped, not carried",
|
||||
DeliveryWatchdog(tripped=False, reason="stale").reason == "")
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
print()
|
||||
print("=" * 72)
|
||||
print("4. END TO END, AND THIS IS THE CARD: a wedged unit and a cold pack at")
|
||||
print(" its floor are IDENTICAL in the power domain. Only the SoC differs.")
|
||||
print("=" * 72)
|
||||
# ⚠️ THE PLANT IS MODELLED AND SAYS SO. `_DeliveryHass` is two inverters that
|
||||
# obey INSTANTLY - no ramp, no lag - and one meter that is `load - sum(measured)`.
|
||||
# That is deliberately not a claim about transients: the worst legitimate
|
||||
# episode a real ramp and lag can produce is measured on admin/goodwe's
|
||||
# two-master rig and recorded in DECISIONS/SAFETY-42-unit-delivery.md. What this
|
||||
# fixture is for is the DISCRIMINATION - which of two identical power traces is
|
||||
# reported - and for that an instant plant is the honest model, because it
|
||||
# removes the one mechanism that could make an obedient unit look absent.
|
||||
#
|
||||
# ⚠️ AND INVERTER B DOES NOT PHYSICALLY EXIST (NOTES.md §1.1b). Every
|
||||
# two-inverter figure in this file, and every one in the decision record, is
|
||||
# TWO COPIES OF INVERTER A - `plantsim.INVERTER_B`, `measured=False`.
|
||||
|
||||
_LOAD_W = 3000.0
|
||||
_FLOOR_PCT = 10
|
||||
|
||||
|
||||
class _DeliveryHass:
|
||||
"""Two inverters, one meter, and unit B that produces nothing.
|
||||
|
||||
`b_soc` is the ONLY thing that differs between the two arms of the
|
||||
experiment: 50 % is a wedged unit with a healthy pack, `_FLOOR_PCT` is a
|
||||
pack sitting exactly on its own cut-out. Every watt on every wire is the
|
||||
same in both.
|
||||
"""
|
||||
|
||||
def __init__(self, load_w=_LOAD_W, b_soc=50.0):
|
||||
self.load_w = float(load_w)
|
||||
self.b_soc = float(b_soc)
|
||||
self.writes = []
|
||||
self.calls = []
|
||||
self.ctrl = None
|
||||
|
||||
async def call(self, domain, service, data):
|
||||
self.calls.append((domain, service, data))
|
||||
|
||||
async def limits(self, entity):
|
||||
return None
|
||||
|
||||
def measured(self):
|
||||
if self.ctrl is None:
|
||||
return [0.0, 0.0]
|
||||
a = self.ctrl.inverters[0]
|
||||
# A obeys. B produces NOTHING, whatever it is told - and whether that is
|
||||
# a wedged power stage or an empty pack is not a question its power
|
||||
# register can answer.
|
||||
return [float((a.last_write if a.last_write is not None else a.target)
|
||||
or 0.0), 0.0]
|
||||
|
||||
async def number(self, entity, invert=False):
|
||||
m = self.measured()
|
||||
return {"sensor.grid": self.load_w - sum(m),
|
||||
"sensor.soc_a": 50.0, "sensor.soc_b": self.b_soc,
|
||||
"sensor.batt_a": m[0], "sensor.batt_b": m[1]}.get(entity)
|
||||
|
||||
async def set_number(self, entity, value):
|
||||
self.writes.append((entity, float(value)))
|
||||
return True
|
||||
|
||||
|
||||
def _roster():
|
||||
return [{"name": n, "setpoint_entity": f"number.sp_{t}",
|
||||
"soc_entity": f"sensor.soc_{t}", "batt_entity": f"sensor.batt_{t}",
|
||||
"max_w": 2000, "bms_soc_floor": _FLOOR_PCT}
|
||||
for n, t in (("inverter-a", "a"), ("inverter-b", "b"))]
|
||||
|
||||
|
||||
def _ctrl(hass, store=None, **over):
|
||||
"""A two-inverter Controller, with the NEIGHBOURING guards given long windows.
|
||||
|
||||
⚠️ THE LONG `circulation_hold_s` AND SIGN HOLDS ARE ISOLATION, NOT A CLAIM
|
||||
THAT THOSE GUARDS ARE SILENT HERE - part 7 asserts what each of them
|
||||
actually concludes on this same fixture, which is the honest version of the
|
||||
claim. What must not happen is the SITE being stopped by another guard while
|
||||
this section is trying to grade an attribution, because then every check
|
||||
below would be measuring the wrong thing.
|
||||
"""
|
||||
opts = {"meter_entity": "sensor.grid", "meter_source": "off",
|
||||
"inverters": _roster(), "gain": 0.6, "slew_w": 1000,
|
||||
# ⚠️ 1 s, NOT THE SHIPPED 95. This window is WALL CLOCK - the
|
||||
# detector takes the caller's `now`, and `_drive()` runs fourteen
|
||||
# real cycles 0.2 s apart, so a shipped-length hold would make every
|
||||
# check below pass for the wrong reason: nothing would have tripped
|
||||
# yet and "no alarm" would read the same as "no fault". The SHIPPED
|
||||
# value is graded in part 2 against config.yaml, which is where a
|
||||
# number belongs; what is graded here is the attribution.
|
||||
"delivery_hold_s": 1,
|
||||
"unit_sign_disagree_s": 3600, "site_sign_disagree_s": 3600,
|
||||
"circulation_hold_s": 3600, "auto_start": True}
|
||||
opts.update(over)
|
||||
c = Controller(opts, hass, store if store is not None else _MemStore(),
|
||||
_P(), stop=asyncio.Event())
|
||||
for inv in c.inverters:
|
||||
inv.dev_min, inv.dev_max = -inv.max_w, inv.max_w
|
||||
c.auto = True
|
||||
hass.ctrl = c
|
||||
return c
|
||||
|
||||
|
||||
class _MemStore:
|
||||
"""A Store that keeps its dict across "restarts", like the real file does.
|
||||
|
||||
Seeded from `store.DEFAULTS` for test_watchdog.py's reason: a fake that
|
||||
hand-listed keys answers None where the real store answers a default, and a
|
||||
check reading "this latch was never written" then cannot tell an absent key
|
||||
from a False one.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
from app.store import DEFAULTS as _SDEF
|
||||
self.data = dict(_SDEF)
|
||||
|
||||
def set(self, k, v):
|
||||
self.data[k] = v
|
||||
|
||||
def get_time(self, *a):
|
||||
return None
|
||||
|
||||
|
||||
class _P:
|
||||
enabled = False
|
||||
|
||||
def publish(self, *a, **k):
|
||||
pass
|
||||
|
||||
def close(self):
|
||||
pass
|
||||
|
||||
|
||||
async def _drive(c, n=14, dt=0.2, watch=None):
|
||||
seen = []
|
||||
for _ in range(n):
|
||||
await c.read_inputs()
|
||||
await c.cycle()
|
||||
if watch is not None:
|
||||
seen.append(watch(c))
|
||||
await asyncio.sleep(dt)
|
||||
return seen
|
||||
|
||||
|
||||
def _sample(c):
|
||||
return dict(
|
||||
grid=c.grid, target=round(c.target, 6),
|
||||
shares=tuple(round(float(i.last_write if i.last_write is not None
|
||||
else i.target), 6) for i in c.inverters),
|
||||
meas=tuple(None if i.batt is None else round(float(i.batt), 6)
|
||||
for i in c.inverters),
|
||||
soc=tuple(i.soc for i in c.inverters),
|
||||
named=tuple(sorted(i.name for i in c.inverters
|
||||
if i.delivery_wd.tripped)),
|
||||
reason=c.reason)
|
||||
|
||||
|
||||
async def _arm(b_soc):
|
||||
hass = _DeliveryHass(b_soc=b_soc)
|
||||
c = _ctrl(hass)
|
||||
return c, hass, await _drive(c, watch=_sample)
|
||||
|
||||
|
||||
_cw, _hw, _sw = asyncio.run(_arm(50.0)) # WEDGED: healthy pack, no output
|
||||
_cf, _hf, _sf = asyncio.run(_arm(float(_FLOOR_PCT))) # COLD: pack at its floor
|
||||
|
||||
|
||||
def _power_domain(rows):
|
||||
return [(r["grid"], r["target"], r["shares"], r["meas"]) for r in rows]
|
||||
|
||||
|
||||
_pw, _pf = _power_domain(_sw), _power_domain(_sf)
|
||||
check(f"THE TWO ARMS ARE BYTE-IDENTICAL IN THE POWER DOMAIN on all "
|
||||
f"{len(_pw)} cycles - grid, site target, every share, every measured "
|
||||
f"watt. No threshold on any of those numbers, at any N and any X, can "
|
||||
f"separate a wedged unit from a pack at its floor, and the card's own "
|
||||
f"proposed discriminator reads only those numbers",
|
||||
len(_pw) == len(_pf) and _pw == _pf)
|
||||
check("...and the ONLY thing that differs is the state of charge",
|
||||
[r["soc"] for r in _sw] != [r["soc"] for r in _sf])
|
||||
print(f" both arms, last cycle: grid {_sw[-1]['grid']:.0f} W, shares "
|
||||
f"{tuple(round(v) for v in _sw[-1]['shares'])}, measured "
|
||||
f"{tuple(round(v) for v in _sw[-1]['meas'])}, reason {_sw[-1]['reason']!r}")
|
||||
print(f" SoC: wedged arm {_sw[-1]['soc']} cold arm {_sf[-1]['soc']}")
|
||||
|
||||
check("AC 1: the WEDGED unit is NAMED, and only it",
|
||||
_sw[-1]["named"] == ("inverter-b",))
|
||||
check("AC 2: the COLD PACK AT ITS FLOOR IS NOT - on the identical trace",
|
||||
_sf[-1]["named"] == ())
|
||||
print(f" {_cw.inverters[1].delivery_wd.reason}")
|
||||
|
||||
# ⚠️ THE VACUOUS-PASS GUARD, AND IT IS NOT DECORATION. MUTATION.md's third
|
||||
# failure mode is a check whose data cannot reach the failure it names: "the
|
||||
# cold arm named nobody" reads exactly the same against a fixture where nothing
|
||||
# was short, nothing was asked, or the loop never ran. So the cold arm is
|
||||
# asserted to have REACHED the branch and been turned away by the account, not
|
||||
# to have missed it.
|
||||
_cold_b = _cf.inverters[1]
|
||||
check("...and the cold arm REACHED that decision rather than falling short of "
|
||||
"it: B was asked for a discharge share, was short of it by more than "
|
||||
"saturation_w, and was turned away by `account()` alone",
|
||||
float(_cold_b.standing_w()) > 0
|
||||
and short_by(_cold_b.standing_w(), _cold_b.batt) > _cf.tuning.saturation_w
|
||||
and account(live=True, parked=False, soc=_cold_b.soc,
|
||||
bms_floor=_cold_b.bms_soc_floor) is not None)
|
||||
check("...and the WEDGED arm's B differs from it in exactly that one place",
|
||||
account(live=True, parked=False, soc=_cw.inverters[1].soc,
|
||||
bms_floor=_cw.inverters[1].bms_soc_floor) is None)
|
||||
|
||||
# ⚠️ WHICH FLOOR, AND THIS IS A WIRING CHECK THAT THE PURE ONES ABOVE CANNOT
|
||||
# MAKE. `peakguard.pack_floor_pct()` conjoins a pack's own cut-out with the
|
||||
# SITE's `peak_soc_floor`, and handing THAT to `account()` would look correct
|
||||
# and would be wrong: `peak_soc_floor` bounds a money-saving discharge only
|
||||
# (SAFETY-25), ordinary self-consumption discharges straight past it, and a
|
||||
# wedged unit sitting anywhere between the two floors would be EXPLAINED AWAY.
|
||||
# So this arm puts B exactly there - 15 %, above its own 10 % cut-out and below
|
||||
# a 20 % `peak_soc_floor` - and it must still be named.
|
||||
|
||||
|
||||
async def _between_floors():
|
||||
hass = _DeliveryHass(b_soc=15.0)
|
||||
c = _ctrl(hass, peak_soc_floor=20)
|
||||
await _drive(c)
|
||||
return c
|
||||
|
||||
|
||||
_cbf = asyncio.run(_between_floors())
|
||||
check("a wedged unit BETWEEN its own cut-out and the site's peak_soc_floor is "
|
||||
"still named: the floor this check asks about is the pack's own, not the "
|
||||
"peak guard's, and swapping them would hide the fault in a band an "
|
||||
"ordinary install has 10 points of",
|
||||
_cbf.inverters[1].delivery_wd.tripped
|
||||
and _cbf.inverters[1].soc == 15.0
|
||||
and float(_cbf.o.get("peak_soc_floor")) == 20.0)
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
print()
|
||||
print("=" * 72)
|
||||
print("5. IT REPORTS AND DOES NOT ACT - asserted by counterfactual, not by")
|
||||
print(" reading the code")
|
||||
print("=" * 72)
|
||||
|
||||
|
||||
async def _never_trips():
|
||||
hass = _DeliveryHass(b_soc=50.0)
|
||||
c = _ctrl(hass, delivery_hold_s=900) # the schema ceiling: never, here
|
||||
return c, await _drive(c, watch=_sample)
|
||||
|
||||
|
||||
_cn, _sn = asyncio.run(_never_trips())
|
||||
check("the guard TRIPPED in one run and not in the other, so the comparison "
|
||||
"below is between two different states of this detector",
|
||||
_sw[-1]["named"] == ("inverter-b",) and _sn[-1]["named"] == ()
|
||||
and _cw.inverters[1].delivery_wd.tripped
|
||||
and not _cn.inverters[1].delivery_wd.tripped)
|
||||
check("...and EVERY WATT IS THE SAME. Grid, site target, both shares and both "
|
||||
"measured powers are identical cycle for cycle whether this guard "
|
||||
"reported or not - which is what 'it does not act' means, stated as a "
|
||||
"measurement instead of as an intention",
|
||||
_power_domain(_sw) == _power_domain(_sn))
|
||||
check("...the wedged unit is still being commanded its share after the trip, "
|
||||
"not parked: this is NOT SAFETY-02's remedy and must not become it",
|
||||
abs(float(_cw.inverters[1].target)) > 0)
|
||||
check("...and nothing named `delivery` reaches the arbiter - the resolution's "
|
||||
"own sentence is unchanged by the trip",
|
||||
"deliver" not in _cw.reason.lower() and _cw.reason == _cn.reason)
|
||||
_pay = _cw.status_payload()
|
||||
check("...and control is still running, which is the whole difference from "
|
||||
"every other latched guard in this add-on",
|
||||
_cw.auto and _pay["level"] != "bad")
|
||||
check("...but the banner does NOT go on saying 'Running - holding grid at "
|
||||
"zero'. That sentence is a claim the site cannot make while a unit is "
|
||||
"absent, and 'the page reads healthy while the fault runs' is the shape "
|
||||
"this whole card exists to remove. It names the unit and stays a WARNING",
|
||||
_pay["level"] == "warn" and "NOT DELIVERING" in _pay["banner"]
|
||||
and "inverter-b" in _pay["banner"]
|
||||
and "holding grid at zero" not in _pay["banner"])
|
||||
print(f" banner: {_pay['level']!r} / {_pay['banner']!r}")
|
||||
_row = [r for r in _cw.checks() if "NOT DELIVERING" in r["text"]]
|
||||
check("the status page carries it as a CAUTION, not a failure: the site is "
|
||||
"still running and short, which a red cross would misdescribe",
|
||||
len(_row) == 1 and _row[0]["ok"] is False and _row[0]["warn"] is True)
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
print()
|
||||
print("=" * 72)
|
||||
print("6. what the NEIGHBOURING guards conclude on this same fixture - the")
|
||||
print(" honest version of 'every existing guard is silent'")
|
||||
print("=" * 72)
|
||||
_b = _cw.inverters[1]
|
||||
check("SAFETY-30 per unit: SILENT. It needs OPPOSITION on both sides of "
|
||||
f"{_cw.unit_sign_cfg.min_w:.0f} W, and a unit at 0 W is not opposing - "
|
||||
"it is absent, so the gate never opens",
|
||||
not _b.sign_wd.tripped and _b.sign_wd.disagree_since is None)
|
||||
check("SAFETY-30 site: SILENT. The sum of the standing commands still agrees "
|
||||
"with the meter in DIRECTION; only the magnitude is short",
|
||||
not _cw.site_sign_wd.tripped and _cw.site_sign_wd.disagree_since is None)
|
||||
check("SAFETY-31: SILENT, and identically so. sum(|Bi|) - |sum(Bi)| is ZERO "
|
||||
"when one unit does nothing - circulation needs two units pulling "
|
||||
"AGAINST each other",
|
||||
_cw.site_circulating_w == 0.0 and not _cw.circ_wd.tripped)
|
||||
check("the saturation freeze: FIRES - and that is the point. It is the one "
|
||||
"signal that does, and clamp.py:93-96 documents it as the alarm an "
|
||||
"operator learns to ignore, which is why this card needed a different "
|
||||
"signal rather than a louder one",
|
||||
_cw.sat_count >= _cw.tuning.saturation_cycles)
|
||||
print(f" saturation counter {_cw.sat_count} (freezes at "
|
||||
f"{_cw.tuning.saturation_cycles}); site commanded "
|
||||
f"{_cw.target:.0f} W against {_cw.site_batt_w:.0f} W measured")
|
||||
check("...and it fires IDENTICALLY on the benign arm, on the identical trace, "
|
||||
"at the same counter. A saturation-shaped alarm cannot tell these two "
|
||||
"apart at any threshold and any duration",
|
||||
_cf.sat_count == _cw.sat_count
|
||||
and _cf.site_batt_w == _cw.site_batt_w
|
||||
and _cf.target == _cw.target)
|
||||
|
||||
print()
|
||||
print(" ...and the one case where BOTH have something to say: a unit that is")
|
||||
print(" REVERSED is short too, and SAFETY-30 gets there first")
|
||||
# ⚠️ THE OVERLAP IS REAL AND IT RESOLVES ITSELF, WHICH IS WORTH ASSERTING
|
||||
# RATHER THAN ASSUMING. A unit measured at -2000 W against a +2000 W share is
|
||||
# OPPOSING (SAFETY-30's fault) and is also short by 4000 W with nothing to
|
||||
# explain it (this one). SAFETY-30's hold is shorter, it parks the unit, the
|
||||
# share goes to 0 W - and this check's gate closes on `share > 0` while
|
||||
# `account()` answers "parked". Two alarms for one fault is how an operator
|
||||
# learns to skim, and this is the arrangement that prevents it.
|
||||
|
||||
|
||||
class _ReversedHass(_DeliveryHass):
|
||||
"""Unit B does the OPPOSITE of what it is told - SAFETY-30's fault."""
|
||||
|
||||
def measured(self):
|
||||
a, b = self.ctrl.inverters
|
||||
w = lambda i: float((i.last_write if i.last_write is not None
|
||||
else i.target) or 0.0)
|
||||
return [w(a), -w(b)]
|
||||
|
||||
|
||||
async def _reversed_run():
|
||||
hass = _ReversedHass(b_soc=50.0)
|
||||
# SAFETY-30's per-unit hold SHORTER than this check's, which is the shipped
|
||||
# ordering too: 85 s against 185 s.
|
||||
c = _ctrl(hass, unit_sign_disagree_s=1, delivery_hold_s=2)
|
||||
seen = await _drive(c, n=24, dt=0.15, watch=lambda c: dict(
|
||||
sign=c.inverters[1].sign_wd.tripped,
|
||||
deliv=c.inverters[1].delivery_wd.tripped,
|
||||
share=float(c.inverters[1].target),
|
||||
acct=account(live=True, parked=c.inverters[1].sign_wd.tripped,
|
||||
soc=c.inverters[1].soc,
|
||||
bms_floor=c.inverters[1].bms_soc_floor)))
|
||||
return c, seen
|
||||
|
||||
|
||||
_crev, _srev = asyncio.run(_reversed_run())
|
||||
check("a REVERSED unit is short by more than saturation_w too, so both guards "
|
||||
"have evidence - the fixture reaches the overlap rather than dodging it",
|
||||
any(r["deliv"] or r["sign"] for r in _srev)
|
||||
and any(abs(r["share"]) > _crev.tuning.saturation_w for r in _srev))
|
||||
check("SAFETY-30 gets there first and PARKS it - its hold is shorter, which is "
|
||||
"also the shipped ordering (85 s against 185 s)",
|
||||
_crev.inverters[1].sign_wd.tripped)
|
||||
check("...and once parked, this check goes quiet on its own: the share is 0 W "
|
||||
"so it is no longer ASKED, and `account()` answers 'parked' anyway. ONE "
|
||||
"alarm for one fault",
|
||||
not _crev.inverters[1].delivery_wd.tripped
|
||||
and _crev.inverters[1].delivery_wd.since is None
|
||||
and float(_crev.inverters[1].target) == 0.0
|
||||
and _srev[-1]["acct"] is not None)
|
||||
print(f" shipped ordering: unit sign hold "
|
||||
f"{_crev.unit_sign_cfg.hold_s:.0f} s in this fixture, delivery hold "
|
||||
f"{_crev.delivery_cfg.hold_s:.0f} s; account now {_srev[-1]['acct']!r}")
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
print()
|
||||
print("=" * 72)
|
||||
print("7. AC 5: AT ONE INVERTER NOTHING CHANGES - asserted, not assumed")
|
||||
print("=" * 72)
|
||||
|
||||
|
||||
class _OneHass(_DeliveryHass):
|
||||
"""One inverter that produces nothing against a non-zero share."""
|
||||
|
||||
def measured(self):
|
||||
return [0.0]
|
||||
|
||||
async def number(self, entity, invert=False):
|
||||
return {"sensor.grid": self.load_w, "sensor.soc": 50.0,
|
||||
"sensor.batt": 0.0}.get(entity)
|
||||
|
||||
|
||||
async def _one():
|
||||
hass = _OneHass()
|
||||
opts = {"setpoint_entity": "number.sp", "meter_entity": "sensor.grid",
|
||||
"soc_entity": "sensor.soc", "batt_entity": "sensor.batt",
|
||||
"meter_source": "off", "max_w": 2000, "gain": 0.6,
|
||||
# ⚠️ THE SAME 1 s AS THE TWO-INVERTER FIXTURE, AND THAT IS WHAT
|
||||
# MAKES THIS SECTION NON-VACUOUS. The run below is 2 s of wall clock
|
||||
# against a 1 s window, so a hold is not what stops this firing -
|
||||
# the reduction is. With a shipped-length hold "nothing was
|
||||
# reported" would be true of a fixture that simply ran out of time.
|
||||
"bms_soc_floor": _FLOOR_PCT, "delivery_hold_s": 1,
|
||||
"sign_disagree_s": 3600, "auto_start": True}
|
||||
store = _MemStore()
|
||||
c = Controller(opts, hass, store, _P(), stop=asyncio.Event())
|
||||
c.inverters[0].dev_min, c.inverters[0].dev_max = -2000.0, 2000.0
|
||||
c.auto = True
|
||||
hass.ctrl = c
|
||||
await _drive(c, n=20, dt=0.1)
|
||||
return c, hass, store
|
||||
|
||||
|
||||
_c1, _h1, _st1 = asyncio.run(_one())
|
||||
_only = _c1.inverters[0]
|
||||
check("a roster of ONE, whose only unit is standing at a discharge share and "
|
||||
"delivering nothing for the whole run - the exact fault, at N=1",
|
||||
len(_c1.inverters) == 1 and float(_only.standing_w()) > 0
|
||||
and _only.batt == 0.0
|
||||
and short_by(_only.standing_w(), _only.batt) > _c1.tuning.saturation_w)
|
||||
check("...and NOTHING is reported. With no sibling, 'this box cannot' and "
|
||||
"'this plant cannot' are the same observation, and the saturation freeze "
|
||||
"is still the whole answer there", not _only.delivery_wd.tripped)
|
||||
check("...the window was never even opened", _only.delivery_wd.since is None)
|
||||
check("...and no store key was written, so a one-inverter install's state file "
|
||||
"is byte-identical to what it was before this card",
|
||||
unit_key(_only.name, "delivery_tripped") not in _st1.data
|
||||
and unit_key(_only.name, "delivery_reason") not in _st1.data)
|
||||
_n1row = [r for r in _c1.checks() if "delivery watchdog" in r["text"]]
|
||||
check("...and the status page says n/a rather than 'armed', which would be a "
|
||||
"claim to be watching something that cannot happen",
|
||||
len(_n1row) == 1 and _n1row[0]["ok"] and "n/a at one inverter" in _n1row[0]["text"])
|
||||
check("...no persistent notification of this kind was raised either",
|
||||
not [c for c in _h1.calls
|
||||
if "not_delivering" in str(c[2].get("notification_id", ""))])
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
print()
|
||||
print("=" * 72)
|
||||
print("8. SAFETY-34: the report survives a Home Assistant restart, which is")
|
||||
print(" how the 2026-09-02 outage went unseen for 6 h 25 m")
|
||||
print("=" * 72)
|
||||
|
||||
|
||||
async def _restart():
|
||||
shared = _MemStore()
|
||||
h1 = _DeliveryHass(b_soc=50.0)
|
||||
c1 = _ctrl(h1, store=shared)
|
||||
await _drive(c1)
|
||||
# A NEW process, a NEW HomeAssistant object (so the in-memory notification
|
||||
# is gone with it), the SAME durable file. That is what a Home Assistant
|
||||
# restart is from this add-on's point of view: the add-on goes with it.
|
||||
h2 = _DeliveryHass(b_soc=50.0)
|
||||
c2 = _ctrl(h2, store=shared)
|
||||
await c2.check_not_delivering()
|
||||
return c1, h1, c2, h2, shared
|
||||
|
||||
|
||||
_r1, _rh1, _r2, _rh2, _rs = asyncio.run(_restart())
|
||||
_key = unit_key("inverter-b", "delivery_tripped")
|
||||
check("the latch reached the durable file, namespaced under the unit it is "
|
||||
"about", _rs.data.get(_key) is True)
|
||||
check("...with its reason, and the reason was written FIRST - a crash between "
|
||||
"the two must not persist a latch nobody can explain",
|
||||
bool(_rs.data.get(unit_key("inverter-b", "delivery_reason"))))
|
||||
check("...and there is NO flat mirror of it, where SAFETY-02 has one: the "
|
||||
"version a rollback would land on drives exactly one actuator, a roster "
|
||||
"at which this check is inert by construction",
|
||||
"delivery_tripped" not in _rs.data and "delivery_reason" not in _rs.data)
|
||||
check("a fresh process comes up with the report STILL STANDING",
|
||||
_r2.inverters[1].delivery_wd.tripped)
|
||||
check("...and RE-RAISES the notification, because a persistent_notification is "
|
||||
"in memory and did not survive - that is the whole of SAFETY-34",
|
||||
any(c[0] == "persistent_notification"
|
||||
and "not_delivering" in str(c[2].get("notification_id", ""))
|
||||
for c in _rh2.calls))
|
||||
_note = [c for c in _rh2.calls
|
||||
if "not_delivering" in str(c[2].get("notification_id", ""))][0]
|
||||
check("the notification NAMES THE UNIT - in its id, its title and its body - "
|
||||
"which is card requirement 2 and the whole reason SAFETY-30 split "
|
||||
"per-unit from site",
|
||||
"inverter-b" in _note[2]["notification_id"].replace("_", "-")
|
||||
and "inverter-b" in _note[2]["title"]
|
||||
and "inverter-b" in _note[2]["message"])
|
||||
check("...and it says, in the operator's own sentence, that nothing has been "
|
||||
"commanded differently. An alarm that let somebody believe the add-on "
|
||||
"had handled it would be worse than no alarm",
|
||||
"NOTHING HAS BEEN COMMANDED DIFFERENTLY" in _note[2]["message"])
|
||||
print(f" id={_note[2]['notification_id']!r} title={_note[2]['title']!r}")
|
||||
_before_calls = len(_rh2.calls)
|
||||
await_again = asyncio.run(_r2.check_not_delivering())
|
||||
check("...and it is raised ONCE per process, not once per cycle: the alarm "
|
||||
"storm this add-on has argued against twice already",
|
||||
len(_rh2.calls) == _before_calls)
|
||||
|
||||
_r2.auto = False
|
||||
asyncio.run(_r2.handle_action("auto_toggle")) # off
|
||||
asyncio.run(_r2.handle_action("auto_toggle")) # and on again: the press
|
||||
check("the operator's START press is the only way out, and it clears the "
|
||||
"latch, the reason and the 'a human has seen this' flag together",
|
||||
not _r2.inverters[1].delivery_wd.tripped
|
||||
and _rs.data.get(_key) is False
|
||||
and _rs.data.get(unit_key("inverter-b", "delivery_reason")) == ""
|
||||
and _r2._delivery_announced == set())
|
||||
|
||||
|
||||
print()
|
||||
if fails:
|
||||
print(f"{len(fails)} FAILED: {', '.join(fails)}")
|
||||
sys.exit(1)
|
||||
print("all checks passed")
|
||||
@@ -1917,6 +1917,12 @@ _AUDITED = {
|
||||
("test_freshness.py", "option_default"),
|
||||
("test_price.py", "shipped"),
|
||||
("test_tariff.py", "shipped_option"),
|
||||
# SAFETY-42. `read_option()` there takes a key and reads config.yaml, so the
|
||||
# signature puts it in this table whether anybody remembered it or not -
|
||||
# which is what this table is for. It answers the SITE value because it is
|
||||
# anchored at exactly two spaces; a loose-whitespace pattern would have
|
||||
# answered a per-inverter `max_w`, which is the defect below.
|
||||
("test_delivery.py", "read_option"),
|
||||
}
|
||||
# Helpers the audited readers call. Not called with a key themselves, so there
|
||||
# is nothing to feed them; they are exec'd so their callers work.
|
||||
|
||||
@@ -130,6 +130,11 @@ DERIVED = {
|
||||
"meter_freeze_s": ("SAFETY-18", "MEASURED"),
|
||||
"circulation_w": ("SAFETY-31", "924"),
|
||||
"circulation_hold_s": ("SAFETY-31", "DERIVED"),
|
||||
# SAFETY-42. Two needles rather than one: the campaign it was derived from,
|
||||
# and the word that says what it does NOT do. An operator who reads
|
||||
# "not-delivering report" and assumes the add-on has handled it is the
|
||||
# failure this alarm's whole wording is written against.
|
||||
"delivery_hold_s": ("SAFETY-42", "DERIVED FROM", "REPORTS ONLY"),
|
||||
"peak_soc_floor": ("SAFETY-24", "DERIVED"),
|
||||
"bms_soc_floor": ("0x0560", "MEASURED"),
|
||||
"integrator_max_w": ("MEASURED", "Do not raise it"),
|
||||
|
||||
@@ -234,6 +234,9 @@ configuration:
|
||||
circulation_hold_s:
|
||||
name: "Safety · Circulating-power hold (s) — derived, do not change"
|
||||
description: "TWO OR MORE INVERTERS ONLY. DERIVED (SAFETY-31): 75 s is fifteen control cycles - 15 times the worst legitimate episode the rig can produce, 2.5 times the firmware's own 30 s link veto, and twelve times inside the 900 s billing quarter. It is deliberately not a round 60 or 120, which would be a guess wearing a measurement's clothes. Do not change it without re-deriving it."
|
||||
delivery_hold_s:
|
||||
name: "Safety · Not-delivering report after (s) — derived, do not change"
|
||||
description: "TWO OR MORE INVERTERS ONLY; with one inverter this check cannot fire at all, and that is deliberate rather than a limitation. How long ONE inverter may stand at a discharge share it is not producing - short by more than the saturation threshold, with nothing this add-on can read to explain it - while at least one sibling is NOT in that state, before it is named. REPORTS ONLY (SAFETY-42): nothing is commanded differently, the loop is untouched, and the site goes on running short by that unit's share until you look. DERIVED FROM 2640 CLOSED-LOOP TEST RUNS in which nothing was faulty (SAFETY-42, plus 1890 runs at one inverter as the control, which report nothing at any threshold): the worst legitimate episode those runs produce is 30.0 s, the floor of 30 s is the firmware's own link veto below which a unit is already being dropped from the split, and the ceiling is the 900 s quarter that bills the capacity tariff. 185 s is 6.2 times that, and the margin is sized on a measurement rather than a feeling: the second inverter in those tests is a copy of the first, and a second unit four times slower produces 90.0 s. Do not change it without re-deriving it. A pack at or below its own bms_soc_floor is never reported: an empty pack and a wedged one are identical watt for watt, and that setting is the only thing that tells them apart - so if this fires on a pack you believe is empty, that setting does not describe your battery."
|
||||
auto_start:
|
||||
name: "Advanced · Start controlling automatically on boot"
|
||||
description: "Off by default. Leave it off until the site has been commissioned: with it off, somebody has to press Start after every restart of the add-on, which is what you want while you are still checking signs and entities."
|
||||
|
||||
@@ -214,6 +214,9 @@ configuration:
|
||||
circulation_hold_s:
|
||||
name: "Veiligheid · Wachttijd rondpompend vermogen (s) — afgeleid, niet wijzigen"
|
||||
description: "ALLEEN BIJ TWEE OF MEER OMVORMERS. AFGELEID (SAFETY-31): 75 s zijn vijftien regelcycli - 15 keer de langste terechte episode die de testopstelling kan produceren, 2,5 keer het eigen veto van 30 s in de firmware, en twaalf keer binnen het factuurkwartier van 900 s. Het is bewust geen ronde 60 of 120, want dat zou een gok zijn in het jasje van een meting. Verander dit niet zonder de afleiding opnieuw te maken."
|
||||
delivery_hold_s:
|
||||
name: "Veiligheid · Melding niet-leverende omvormer na (s) — afgeleid, niet wijzigen"
|
||||
description: "ALLEEN BIJ TWEE OF MEER OMVORMERS; bij één omvormer kan deze controle helemaal niet aanslaan, en dat is een bewuste keuze en geen tekortkoming. Hoe lang één omvormer mag blijven staan op een ontlaadaandeel dat hij niet levert - meer dan de verzadigingsdrempel te weinig, zonder dat deze add-on er een verklaring voor kan uitlezen - terwijl minstens één andere omvormer dat NIET doet, voordat hij bij naam wordt gemeld. ALLEEN MELDEN (SAFETY-42): er wordt niets anders gestuurd, de regeling blijft ongemoeid, en de installatie blijft draaien met het aandeel van die omvormer te weinig tot u gaat kijken. AFGELEID UIT 2640 GESLOTEN-LUSTESTS waarbij niets defect was (SAFETY-42, plus 1890 tests met een omvormer als controlegroep, die bij geen enkele drempel iets melden): de langste terechte episode in die tests duurt 30.0 s, de ondergrens van 30 s is het eigen veto van de firmware waaronder een omvormer al uit de verdeling wordt gehaald, en de bovengrens is het factuurkwartier van 900 s. 185 s is 6,2 keer zoveel, en die marge is op een meting gebaseerd en niet op een gevoel: de tweede omvormer in die tests is een kopie van de eerste, en een tweede eenheid die vier keer trager is levert 90,0 s op. Verander dit niet zonder de afleiding opnieuw te maken. Een accu op of onder zijn eigen bms_soc_floor wordt nooit gemeld: een lege accu en een vastgelopen omvormer zijn watt voor watt identiek, en die instelling is het enige wat ze uit elkaar houdt - slaat dit toch aan bij een accu die volgens u leeg is, dan beschrijft die instelling uw batterij niet."
|
||||
auto_start:
|
||||
name: "Geavanceerd · Automatisch beginnen te regelen bij opstarten"
|
||||
description: "Staat standaard uit. Laat het uit tot de installatie in dienst is genomen: zolang het uit staat, moet iemand na elke herstart van de add-on op Start drukken, en dat is precies wat u wilt terwijl u tekens en entiteiten nog aan het nakijken bent."
|
||||
|
||||
Reference in New Issue
Block a user