admin/goodwe-addon
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

SAFETY-42: a unit that stops delivering, and the discriminator the card asked for does not exist
checks / unit (pull_request) Successful in 5m54s
Details

Browse Source 
A unit standing at a non-zero share and producing nothing is seen by NOTHING.
SAFETY-30's per-unit sign gate needs OPPOSITION on both sides of 250 W and a
unit at 0 W is not opposing, it is ABSENT. The site sum still agrees with the
meter in direction. Circulating power is exactly zero when one unit does
nothing. The only trace left is `reason == "saturated-freeze"`, which
clamp.py:93-96 documents as the alarm an operator learns to ignore. It is
HW-01's shape, which ran 6 h 25 m with all three layers spectators.

⚠️ THE CARD'S OWN DISCRIMINATOR WAS BUILT FIRST AND FALSIFIED, AND THAT IS THE
MAIN CONTENT. SAFETY-42 asks for "saturation persisting past N while
|grid error| > X, both derived". Run on the two-master rig it fails in BOTH
directions, and no (N, X) fixes either:

  FALSE POSITIVE  `plantsim.Plant.limit()` returns 0 W for a discharge command
      into a pack at its own floor, and a wedged unit produces 0 W too. Load
      2000 W, equal weights, 2000 W per-unit cap, 120 cycles: THE TWO ARMS ARE
      IDENTICAL ON ALL 120 in grid, site target, every share and every measured
      watt. Both saturated-freeze on 116 of them, both hold |grid error| over
      300 W on all 120 and settle at 1200 W, both read circulating 0.0 W. The
      predicate's inputs do not differ, so no threshold on them can differ. And
      "a cold pack at its floor must not alarm" is the card's own AC 2.

  FALSE NEGATIVE  the standing grid error is what a HEALTHY SIBLING ERASES.
      Load 800 W, same config, B wedged: over the last 30 of 200 cycles the loop
      is frozen on 30 of 30 while the worst |grid error| is 30 W, with B standing
      at +770 W and delivering 0 W. Any X above 30 W is silent for the whole run
      on the exact fault the card describes.

WHAT REPLACES IT. Per unit, for a discharge share, `share - measured`; readable
because of a reduction that only exists at N>=2:

    a unit is named while it is short by more than `saturation_w` with NO
    ACCOUNT for it, and the units in that state are a PROPER SUBSET of the units
    asked to discharge.

THE ACCOUNT keeps the cold pack out and is the only thing that can: the two
cases are identical in the power domain, so the discriminator has to be a
non-power fact, and there is exactly one - the pack's own SoC against its own
`bms_soc_floor`, asked through `peakguard.discharge_permitted()` rather than
transcribed. ⚠️ `bms_soc_floor` ALONE, not `pack_floor_pct()`'s conjunction with
`peak_soc_floor`: that floor bounds a money-saving discharge only and using it
would explain away a wedged unit anywhere between the two.

THE PROPER SUBSET keeps a whole-site cause out - a derate this add-on cannot see
hits every unit; one box that stopped listening does not - and it is what makes
this INERT AT N=1, by algebra rather than by a roster guard.

IT REPORTS. IT DOES NOT ACT, and the strongest argument is for acting. Dropping
the wedged unit from `live` genuinely recovers a site the shipped path does not:
at load 1500 W, kept in the split the site settles at 800 W with a 700 W error
and NEVER comes inside 100 W in 150 cycles (the freeze pins the target, so the
sibling's SHARE never grows); dropped, it delivers 1500 W from cycle 14. What
refuses it is that the only re-allocation this add-on can express is
all-or-nothing: a unit derated to 50 % is short by more than saturation_w with
nothing to explain it, so this detector names it too, and at load 3000 W
dropping it leaves the site delivering 2000 W against 2400 W kept - 400 W WORSE.
A partial re-allocation is what would make acting safe and the allocator cannot
express it. That is a card.

control.py is NOT MODIFIED. The freeze shapes the loop; this reports on it.

NUMBERS, MEASURED. The watts side is `Tuning.saturation_w`, read at runtime, so
there is no fourth CFG-03 copy and it cannot drift from the number the control
law itself calls "not following". `delivery_hold_s: 185` comes off 2640
closed-loop two-inverter runs in which nothing is faulty (6 gains x 5 slews over
the whole schema range, across load reversals, a flapping load and ASYMMETRIC
SPLITS - a family SAFETY-30's campaign did not have and the one that dominates
here) plus 1890 runs at a roster of ONE as the control:

    threshold   reversals   flapping   weighted   N=1 control
     * 500 W      20.0 s      0.0 s      30.0 s      0.0 s

The N=1 column is 0.0 s at EVERY threshold in all 1890 runs, and the flapping
column - the family that defeated SAFETY-30's site window - is zero too, because
a common stimulus makes both units short at once and the subset stops being
proper. ⚠️ The curve is NOT monotone (55.0 s at 250 W against 40.0 s at 100 W):
lowering the threshold can turn a proper subset into the whole roster and BREAK
the run.

⚠️ THE MARGIN IS MEASURED, NOT FELT, BECAUSE INVERTER B DOES NOT PHYSICALLY
EXIST. `plantsim.INVERTER_B` is A COPIED, `measured=False`, so every two-inverter
figure above is two copies of A. Sweeping a different second unit: the worst
legitimate episode is insensitive to B's RAMP (30.0 s at half A's) and scales
with its LAG - 45.0 s at twice A's measured 10 s, 90.0 s at four times it.
185 s is 6.2x the measured worst, 2.06x the four-times-slower case, 6.2x
LINK_STALE_S and 4.9x inside the 900 s quarter. Deliberately not 180, which is
SAFETY-02's site hold.

SAFETY-34. `unit/<name>/delivery_tripped` in the store, one notification id per
unit, re-raised on the first cycle of every process, cleared by the Start press.
No flat mirror: a rollback lands on a version that drives one actuator, where
this check is inert.

⚠️ SCHEMA CHANGE - `delivery_hold_s` joins options: and schema:, so this needs a
version bump before it ships. Additive and defaulted, so an existing options blob
stays valid.

STATED GAPS: charge shares are not graded (no configured charge ceiling exists,
so a constant-voltage taper cannot be explained and arming it would fire on every
maintenance charge); every unit wedged at once is not caught (the subset is not
proper and no instrument here separates that from a plant that cannot).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
This commit is contained in:
glenn schrooyen
2026-09-06 15:46:00 +02:00
co-authored by Claude Opus 5
parent c8dcb38a95
commit ccc4bf100c
12 changed files with 2329 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
goodwe_controller/app/delivery.py
+611
View File
@@ -0,0 +1,611 @@
"""SAFETY-42 - a unit that stops delivering its share, which nothing else can see.
At one inverter this file is inert by construction (see `delinquent()`), and
that is deliberate rather than a limitation: the whole signal it reads is a
DIFFERENCE BETWEEN SIBLINGS, and a roster of one has none.
WHAT IT CATCHES, in one sentence: an inverter that accepts its share and does
not produce it, beside a sibling that does - HW-01's shape at N=2, where the
2026-09-02 fault wedged inverter A for 6 h 25 m with every layer a spectator.
WHY EVERY EXISTING GUARD IS SILENT, AND IT IS STRUCTURAL
=======================================================
SAFETY-30 per unit gates on OPPOSITION: both the command and the unit's own
power over `unit_sign_disagree_w` (250 W) with opposite
signs. A unit sitting at 0 W never opens that gate. It is
not opposing, it is ABSENT.
SAFETY-30 site the sum of the commands still AGREES with the meter in
direction. Only the magnitude is short, and the site
comparison is a sign test.
SAFETY-31 `sum(|Bi|) - |sum(Bi)|` is ZERO when one unit does
nothing: circulation needs two units pulling AGAINST each
other, and one of these is pulling nowhere at all.
the saturation freeze fires - and `clamp.py`:93-96 says of exactly this: "A
loop can be legally saturated all day - a cold pack, a
curtailed inverter, an export limit - so a
saturation-shaped alarm is one an operator learns to
ignore."
⚠️ SO THE ONE SIGNAL THAT DOES FIRE IS THE ONE THIS CODEBASE DOCUMENTS AS
UNREADABLE, and that documentation is right. This module therefore does not make
that signal louder; it computes a different one.
⚠️ THE ONE PLACE IT OVERLAPS SAFETY-30, AND THE OVERLAP RESOLVES ITSELF. A unit
measured at -2000 W against a +2000 W share is OPPOSING - SAFETY-30's fault -
and is ALSO short by 4000 W with nothing here to explain it. Both detectors have
evidence, and only one alarm results, because SAFETY-30's per-unit hold is the
shorter of the two (85 s against 185 s at the shipped defaults): it trips first,
files its `failsafe` SET, the share goes to 0 W, and this check's gate closes on
`share > 0` while `account()` answers "parked" as well. That is asserted on a
reversed-unit fixture in `test_delivery.py` part 6 rather than left to the
reader, because two alarms for one fault is how an operator learns to skim.
------------------------------------------------------------------------------
THE CARD'S OWN DISCRIMINATOR WAS MEASURED AND IT DOES NOT WORK - READ THIS FIRST
==============================================================================
SAFETY-42 proposed: alarm on `saturation persisting past N` while
`|grid error| > X`, with N and X derived rather than chosen. That predicate was
built and run on the two-master rig before this module was written, and it
fails in BOTH directions. Neither failure is a matter of picking better N and X;
both are properties of the quantity.
1. IT FIRES ON EVERY BENIGN SATURATION, and the two cases are not merely
similar - they are THE SAME TRACE. `plantsim.Plant.limit()` returns 0 W for
a discharge command into a pack at its own floor; a wedged unit that never
receives its command also produces 0 W. Run both against the same stimulus
(load 2000 W, equal weights, 2000 W per-unit cap) and every emitted watt
agrees: over 120 control cycles the two arms are IDENTICAL on all 120 in
(grid, site target, each unit's share, each unit's measured power). Both
report `reason == "saturated-freeze"` on 116 of those 120 cycles - the
other four are `slew-limited`, before the freeze engages on cycle 4 - and
both hold |grid error| over 300 W on all 120, settling at 1200 W. A
predicate reading only those
numbers cannot separate them at ANY (N, X), because nothing in its inputs
differs. And "a cold pack at its floor must not alarm" is this card's own
acceptance criterion. `test_delivery.py` part 4 asserts that identity on
the add-on's own end-to-end fixture, so it goes red if it ever stops being
true.
2. IT IS SILENT ON THE HEADLINE CASE, because the standing grid error is
exactly what a HEALTHY SIBLING ERASES. Same rig, load 800 W, equal weights,
2000 W per-unit cap, inverter B wedged: over the last 30 of 200 cycles the
loop is saturation-frozen on every one of them - so "saturation persisting
past N" is true for any N - while the worst |grid error| in that window is
30 W. B is standing at a +770 W discharge share and delivering 0 W. Any X
above 30 W leaves the card's predicate silent for the whole run, on the
exact fault it was written for: a healthy sibling regulating alone with the
site short by one unit's share.
The card's derivation instruction stands and is honoured; what could not be
derived was a threshold on the wrong quantity. This is written down rather than
quietly replaced, because the next person to read the card will propose it again.
------------------------------------------------------------------------------
THE QUANTITY THAT DOES WORK, AND WHY IT NEEDS TWO INVERTERS
===========================================================
Per unit, for a share that asks it to DISCHARGE:
short_by(i) = share_i - measured_i (watts that did not arrive)
`share_i` is the command STANDING ON THAT UNIT'S WIRE - `Inverter.standing_w()`,
the same operand SAFETY-30's per-unit comparison uses - and `measured_i` is that
unit's own power, holding register 0x0518, per-instance since DEV-02.
That number alone is the saturation signal again, per unit. What makes it
readable is the reduction over the roster:
a unit is DELINQUENT while it is short with NO ACCOUNT FOR IT, and the set
of units in that state is a PROPER SUBSET of the units that were asked to
discharge.
Both halves are load-bearing:
* THE ACCOUNT is what keeps the cold pack out, and it is the only thing that
can. The two cases are identical in the power domain (see 1 above), so the
discriminator has to be a fact this controller reads that is NOT power - and
there is exactly one: the pack's own state of charge against its own
`bms_soc_floor`. `account()` below asks it through `peakguard.
discharge_permitted()`, the predicate SAFETY-32 already made public, rather
than transcribing the comparison a second time.
* THE PROPER SUBSET is what keeps a site-wide cause out, and it is the half
that only exists at N>=2. A derate this add-on cannot see - a thermal
limit, a grid-code curtailment, an inverter-side DC restriction - hits the
whole roster. One box that has stopped listening does not. If EVERY unit
that was asked to deliver is short and unexplained, this module says
nothing: it cannot tell that from a plant that simply cannot do it, and
saying so anyway is the saturation-shaped alarm all over again.
⚠️ AT N=1 THE SUBSET IS NEVER PROPER AND THIS IS INERT. One asked unit is either
accounted for (nothing to say) or unaccounted for (the whole roster, so nothing
to compare it against). `delinquent()` returns `{}` for every input at a roster
of one - not by a guard on `len`, but because a proper subset of a one-element
set that contains the element is impossible. That is AC 5, and it is asserted
rather than assumed in `test_delivery.py` part 1.
------------------------------------------------------------------------------
DISCHARGE SHARES ONLY, AND THE COST OF THAT IS STATED RATHER THAN HIDDEN
=======================================================================
`short_by()` answers None for a charge share. The account gate needs a bound
this controller actually reads in the commanded direction, and there is one for
discharge - `bms_soc_floor`, required of every roster entry since SAFETY-32 -
and NONE for charge. A pack in constant-voltage taper legitimately takes
hundreds of watts less than it is told for tens of minutes, and there is no
configured ceiling anywhere in this add-on to explain that with. Armed on charge
shares this module would alarm on the taper of every §12 maintenance charge,
which is the crying-wolf failure it exists to avoid.
What that costs, said plainly: a unit that stops ABSORBING its share is not
caught here. It is the same fault in the other direction and it wants the same
treatment; what it needs first is a per-pack charge ceiling this add-on does not
have. That is a card, and this paragraph is its evidence.
------------------------------------------------------------------------------
IT REPORTS. IT DOES NOT ACT - AND THAT IS A DECISION, NOT AN OMISSION
====================================================================
Card requirement 3 asks which, and why. Three candidates, and the reasoning is
recorded because "alarm-only" is the weaker-looking answer and is still right:
COMMAND IT HARDER - no. HW-01 settles it: on 2026-09-02 the power stage went
on executing its last instruction for 6 h 25 m through every command this
add-on sent it. A command is not a remedy for a box that is not listening,
and pretending otherwise is theatre.
RE-ALLOCATE ITS SHARE TO THE SIBLING - no, AND NOT FOR THE COMFORTABLE
REASON. The comfortable reason would be that the loop already does it, and
the rig says otherwise: with B wedged and equal weights, dropping B from
`live` - the mechanism `live_units()` already uses for a dead link - is a
real improvement at loads the site cannot cover on one unit's SHARE.
Measured, per-unit cap 2000 W, B wedged:
load 1500 W kept in the split: the site settles at 800 W with a 700 W
grid error and NEVER comes inside 100 W in 150 cycles -
the saturation freeze pins the site target, so A's share
never rises. Dropped: 1500 W delivered, inside 100 W from
cycle 14.
load 2000 W kept: 800 W delivered, 1200 W error, never settles.
Dropped: 2000 W delivered, inside 100 W from cycle 8.
That is not three cycles of latency; it is the difference between a site
that recovers and one that does not. It is the strongest argument on this
card and it is stated first, because the answer still has to be no.
WHAT REFUSES IT is that the only re-allocation this add-on can express is
ALL OR NOTHING. `live_units()` drops a unit entirely and the survivors carry
the whole target; there is no way to say "give the sibling the watts that
did not arrive and leave this unit the ones that did". And a unit delivering
PART of its share is short by more than `saturation_w` too, with nothing
here to explain it, so this detector names it - it cannot tell a 50 %
derate from a wedged box, because neither can any instrument this add-on
has. Measured, B derated to 50 %, load 3000 W, per-unit cap 2000 W, and
SAFETY-42 names B in that run:
kept in the split site delivers 2400 W (B contributing 800 W),
grid error 600 W
dropped site delivers 2000 W (B contributing 0 W),
grid error 1000 W - 400 W WORSE
So the remedy would make the site worse in a case this detector cannot rule
out, on a live control law, driven by a brand-new alarm. A partial
re-allocation - the sibling gets the undelivered watts without the short
unit being commanded to zero - is what would make acting safe, and the
allocator has no way to express it today. That is a card, and this block is
its evidence.
STOP THE SITE, as SAFETY-31 does - no, and the asymmetry with SAFETY-31 is
exact. Circulating power is stopped because THIS CONTROLLER IS ONE OF THE
TWO PARTICIPANTS: removing our half halves the loop and un-hides the residue
at a meter that was netting it to zero. Here this controller is not a
participant in the fault, nothing is netting, and the sibling is the only
thing still delivering. Parking it would take the site from "short by one
unit" to "short by all of them" and would remove the evidence.
So: a durable latch, a named unit, an action-log row, and an operator
notification - and the loop is left exactly as it was. Requirement 4 of the card
("do not weaken the saturation freeze to make room") is met by not touching
`control.py` at all.
------------------------------------------------------------------------------
IT SURVIVES A RESTART, WHICH IS SAFETY-34 AND IS WHY THE LATCH EXISTS AT ALL
===========================================================================
`persistent_notification` is IN MEMORY. Home Assistant restarts and the card is
gone with the fault still running, which is precisely how the 2026-09-02 outage
went unseen for 6 h 25 m. So the durable half is `unit/<name>/delivery_tripped`
and `unit/<name>/delivery_reason` in `/data/state.json` - namespaced per unit,
for `store.unit_key()`'s reason - and `main.py` re-raises the notification from
it on the first cycle of every process.
⚠️ AND THAT IS WHY THIS LATCHES THOUGH IT DOES NOT ACT. A detector that cleared
itself when the unit resumed would erase its own record of an intermittent
fault, which is the one shape an operator most needs to be told about. It is
cleared by the same operator press that clears SAFETY-02's and SAFETY-31's -
a deliberate human act, after looking at why.
"""
from dataclasses import dataclass
from datetime import datetime
from .peakguard import discharge_permitted
# The reason code. Distinct from every other one in this add-on by construction:
# nothing else uses it, and it deliberately does not contain the word
# "saturation" - see the module docstring on why a saturation-shaped alarm is
# the thing this module must not produce.
REASON_CODE = "UNIT_NOT_DELIVERING"
# CFG-03. THE ONE IN-CODE COPY OF THIS DEFAULT. `config.yaml`'s `options:` block
# is the source of truth an operator edits and the Supervisor reads; `schema:`
# holds the bounds. This dict exists for the reason
# `circulation.CIRC_OPTION_DEFAULTS` and `watchdog.SIGN_OPTION_DEFAULTS` do -
# the Dockerfile does not put config.yaml in the image, so the running add-on
# cannot read it - and `test_delivery.py` pins the two together by READING
# config.yaml, so a drift goes red in CI rather than shipping.
#
# ⚠️ THERE IS ONLY ONE KEY HERE, AND THE MISSING SECOND ONE IS THE POINT. The
# other half of this check is a WATTS threshold, and it is not an option and not
# a constant: it is `control.Tuning.saturation_w`, read at runtime. See
# `MIN_W_IS_SATURATION_W` below.
DELIVERY_OPTION_DEFAULTS = {
# --- THE WINDOW, AND IT IS THE ONLY NUMBER THIS CARD CHOSE ------------
# ⚠️ MEASURED ON THE TEST-19 TWO-MASTER RIG, NOT PICKED. 2640 closed-loop
# runs at a roster of two in which NOTHING is faulty - every unit executes
# the share the allocator gave it - over 6 gains x 5 slews, the whole range
# config.yaml accepts for `gain` float(0.05,1.0) and `slew_w` int(0,5000),
# across three families of stimulus that can produce a legitimate shortfall:
#
# 1050 load reversals 7 sizes x 5 write staggers (the serialised write
# path landing B's number up to four cycles late)
# 840 flapping loads 7 periods x 4 amplitudes
# 750 weighted splits 5 weightings x 5 staggers on a +-2200 W reversal
#
# ⚠️ THE WEIGHTED FAMILY IS NEW HERE AND IT IS THE ONE THAT DOMINATES. An
# asymmetric split hands one unit a much larger step than its sibling, so
# the big one is still ramping while the small one has settled - which is
# exactly the PROPER SUBSET this check keys on. SAFETY-30's campaign had no
# such family, so reading its numbers across would have understated this.
#
# Every run was executed a second time at a roster of ONE, on the same
# stimulus, as the control. Worst CONTINUOUS legitimate episode:
#
# threshold reversals flapping weighted N=1 control (1890 runs)
# 100 W 40.0 s 0.0 s 75.0 s 0.0 s
# 250 W 55.0 s 0.0 s 60.0 s 0.0 s
# * 500 W 20.0 s 0.0 s 30.0 s 0.0 s
# 700 W 20.0 s 0.0 s 20.0 s 0.0 s
# 1000 W 15.0 s 0.0 s 15.0 s 0.0 s
# 2000 W 10.0 s 0.0 s 5.0 s 0.0 s
#
# * is the shipped threshold, `Tuning.saturation_w`.
#
# ⚠️ THREE THINGS IN THAT TABLE ARE WORTH MORE THAN THE ROW THEY SIT ON.
# (a) The N=1 column is ZERO EVERYWHERE, in all 1890 runs at every
# threshold. The compatibility promise is measured, not argued.
# (b) The FLAPPING column is zero everywhere too - and that is the family
# that defeated SAFETY-30's site window (980 s at N=2, 1000 s at N=1, no
# window inside the billing quarter surviving it). It cannot touch this
# check, because a common stimulus makes BOTH units short at once and
# the subset stops being proper.
# (c) THE CURVE IS NOT MONOTONE - 55.0 s at 250 W against 40.0 s at 100 W -
# and that is a property of the reduction, not noise: a lower threshold
# puts MORE units into the short set, which can turn a proper subset
# into the whole roster and BREAK the run. Do not read it as a
# sensitivity curve.
#
# THE BOUNDS THE VALUE ANSWERS TO:
#
# LOWER, from the rig - 30.0 s, the worst continuous legitimate episode at
# the shipped threshold across all 2640 runs.
#
# LOWER, from the firmware - `inverter.LINK_STALE_S`, 30 s. A unit not
# executing its share for less than the firmware's own veto window is a
# LINK fault the add-on already answers by dropping it from the split with
# a line in the action log. Reporting inside that window would be a
# second, louder alarm about a state already being handled - the
# SAFETY-15/SAFETY-19 failure exactly.
#
# MARGIN, from INVERTER B NOT EXISTING, AND IT IS MEASURED RATHER THAN
# GUESSED. The campaign's B is A COPIED (`plantsim.INVERTER_B`,
# `measured=False`), so the 30.0 s above is two identical machines.
# Sweeping a DIFFERENT second unit through the weighted family: the worst
# legitimate episode is insensitive to B's RAMP (30.0 s at half A's) and
# scales with its LAG - 45.0 s at twice A's measured 10 s lag, 90.0 s at
# four times it. That is the sensitivity the margin has to carry until
# SPIKE-04 measures the real unit.
#
# UPPER, from the bill - 900 s, the quarter that sets the capacity tariff.
# A site running short of one unit's share must be reported well inside
# one.
#
# 185 s: thirty-seven control cycles. 6.2x the worst legitimate episode this
# rig produces, 2.06x the worst a second unit FOUR TIMES slower than A
# produces, 6.2x the firmware's own veto window, and 4.9x inside the
# quarter. ⚠️ DELIBERATELY NOT 180 - that is SAFETY-02's site hold and this
# is a different check on a different quantity, so a reader must not be
# able to think it was carried across - and not 150 or 200.
#
# WHAT IT COSTS, stated rather than hidden: up to 185 s of a site running
# short by one unit's share before anybody is told. Against a fault that ran
# 6 h 25 m unnoticed, and against a guard that TAKES NO ACTION, this window
# buys its margin with reporting latency and with nothing else.
"delivery_hold_s": 185,
}
# ⚠️ WHY THERE IS NO `delivery_w` OPTION, AND WHY THAT IS A DERIVATION RATHER
# THAN A SAVING. The watts side of this check asks "how far short is far
# enough to be worth saying anything about", and the shipped control law
# already answers that question in one number: `Tuning.saturation_w`, 500 W,
# the gap at which `compute()` itself concludes THE INVERTER IS NOT FOLLOWING
# and freezes the integrator. Below it the loop calls the divergence ordinary
# tracking, and an alarm under a threshold the control law calls normal is an
# alarm about nothing.
#
# So `main.py` hands this check `self.tuning.saturation_w` and there is no
# fourth copy of a default to keep in step (CFG-03's complaint) and no way for
# the two to drift: an operator who raises `saturation_w` has told the loop
# that a bigger gap is ordinary tracking, and this check moves with them by
# construction rather than by somebody remembering.
#
# ⚠️ IT IS ALSO WHY THIS CANNOT DOUBLE-REPORT THE SATURATION FREEZE'S OWN
# EVENT. `compute()` needs `saturation_cycles` (3) CONSECUTIVE cycles over
# that same gap before it freezes anything, so every cycle this module counts
# is a cycle the freeze has already seen. This is a reduction over the freeze's
# input, not a second opinion about it.
MIN_W_IS_SATURATION_W = "control.Tuning.saturation_w"
@dataclass(frozen=True)
class DeliveryConfig:
"""Defaults, not constants - config.yaml carries both. See CFG-03 above."""
hold_s: float = float(DELIVERY_OPTION_DEFAULTS["delivery_hold_s"])
@dataclass(frozen=True)
class UnitDelivery:
"""ONE unit's four facts for this cycle, and nothing else.
`name` the roster name. An alarm that cannot say which box to look at
is the whole reason SAFETY-30 split per-unit from site.
`share_w` what is STANDING on this unit's wire - `Inverter.standing_w()`,
the same operand SAFETY-30's per-unit comparison takes, after
the deadband, the slew limit, quantisation and the arbiter.
Not the optimizer's wish: the wire carries the emitted value.
`measured_w` this unit's OWN power, holding register 0x0518, + = discharge.
None when the reading did not arrive.
`accounted` why a shortfall on this unit is already explained, or None if
nothing this controller reads explains one. Built by
`account()`; it is a field rather than a method because the
facts behind it (liveness, the sign latch, SoC) belong to
main.py's `Inverter` and this module imports none of it.
"""
name: str
share_w: float | None
measured_w: float | None
accounted: str | None = None
def short_by(share_w, measured_w):
"""Watts of a DISCHARGE share that did not arrive, or None if unanswerable.
Positive means short. Zero and negative are folded into None rather than
reported, because "delivered its share" and "delivered more than its share"
are both not-this-fault and a caller comparing a float against a threshold
should not have to know which.
⚠️ CHARGE SHARES ANSWER None, AND THAT IS THE STATED LIMIT AT THE TOP OF
THIS FILE, not an oversight to be tidied away. There is no configured charge
ceiling in this add-on to explain a taper with.
⚠️ AND A MISSING READING ANSWERS None RATHER THAN "the whole share". Zero
would be this check inventing a fault from an absence, which is the shape of
the 2026-09-02 outage read the other way round; and the site is already
stopped for a missing per-unit reading anyway (`inputs_ok` requires every
unit's `batt`, and §11.1 rule 1 commands 0 W without consulting this).
"""
if share_w is None or measured_w is None:
return None
share = float(share_w)
if share <= 0.0:
return None
gap = share - float(measured_w)
return gap if gap > 0.0 else None
def account(*, live: bool, parked: bool, soc, bms_floor) -> str | None:
"""Why a shortfall on this unit is ALREADY EXPLAINED, or None if nothing is.
Every branch here is a state this add-on can read and already has an answer
for, and naming one of them a fault would be raising a second, louder alarm
about something already being handled - the SAFETY-15/SAFETY-19 failure that
`circulation.py` and `watchdog.py` both take their 30 s lower bound from.
`live` `Inverter.is_live()`. A unit whose link is stale or whose
writes are being rejected has ALREADY been dropped from the
split with a line in the action log saying so.
`parked` that unit's sign watchdog has latched. It is being commanded
0 W on purpose; not delivering is compliance, not a fault.
`soc` that pack's measured state of charge, percent. None is not a
full battery and is not permission - see below.
`bms_floor` that pack's OWN cut-out, `bms_soc_floor`. Required of every
roster entry since SAFETY-32, so at the only roster this
module is live at it is always configured.
⚠️ THE SoC BRANCH IS THE WHOLE DISCRIMINATOR AND IT IS ASKED THROUGH
`peakguard.discharge_permitted()`, WHICH IS PUBLIC FOR EXACTLY THIS REASON:
"a safety predicate transcribed into its own check is one copy plus a future
disagreement about what the check means". What is NOT reused is its wording,
which is peak-guard flavoured ("the peak guard's SoC floor") and would be a
false statement here - this module is not the peak guard and its floor is
the pack's own cut-out, not `peak_soc_floor`. One predicate, two sentences.
⚠️ AND THE FLOOR PASSED IN IS `bms_soc_floor` ALONE, NOT
`peakguard.pack_floor_pct()`'s conjunction with `peak_soc_floor`. That
conjunction is right for the peak guard and wrong here: `peak_soc_floor`
bounds a money-saving discharge only (SAFETY-25), ordinary self-consumption
discharges straight past it, and using it would EXPLAIN AWAY a wedged unit
sitting anywhere between the two floors.
⚠️ EVERY UNKNOWN ACCOUNTS FOR THE SHORTFALL, i.e. FAILS TOWARD SILENCE, and
that direction is the opposite of the one `discharge_permitted()` was
written for. There it withholds a discharge; here it withholds an ALARM. An
unreadable SoC or an unstated floor means this module cannot say whether the
pack is empty, and an alarm nobody can justify is the crying-wolf failure
this whole file is written against. The cost is bounded: the same absence
stops the site through `inputs_ok` (SoC) or is refused at startup by the
roster schema (the floor).
"""
if not live:
return ("its link is stale or its writes are being rejected, and the "
"allocator has already dropped it from the split")
if parked:
return "it is latched at 0 W by a sign trip, so it is doing as it is told"
why = discharge_permitted(soc, bms_floor)
if why is not None:
if soc is None:
return "its state of charge did not read, so its pack cannot be graded"
if bms_floor is None:
return ("no bms_soc_floor is configured for it, so there is nothing "
"to say whether its pack is empty")
return (f"its pack is at {float(soc):.0f} %, at or below its own "
f"{float(bms_floor):.0f} % cut-out (bms_soc_floor)")
return None
def delinquent(units, min_w: float) -> dict:
"""Which units are short with NO ACCOUNT, when the roster disagrees with them.
`units` one `UnitDelivery` per inverter, in roster order.
`min_w` the shortfall that counts - `control.Tuning.saturation_w`. See
`MIN_W_IS_SATURATION_W`.
Returns `{name: sentence}`, empty when there is nothing to say. The sentence
is the operator's first line and it carries the peer clause, because "B is
not delivering" and "B is not delivering while A is" are different facts and
only the second one is this module's conclusion.
⚠️ THE PROPER-SUBSET RULE IS THE CHECK, NOT A FILTER ON IT. A unit is named
only while the units in its state are FEWER than the units that were asked
to discharge. Everything asked and nothing explained is a SITE condition -
a derate, a curtailment, a grid-code limit, a plant that simply cannot -
and this module has no instrument that can tell that from every box being
wedged at once. Reporting it anyway would be the saturation-shaped alarm
`clamp.py` warns about, produced by the guard written to replace it.
⚠️ AND THAT IS WHY N=1 IS INERT WITHOUT A ROSTER GUARD. `asked` has at most
one element there, `len(asked) < 2` short-circuits, and every input answers
`{}`. A `if n_units > 1` in the caller would be a second place the same
decision lives; the algebra is the guard, the same way
`circulation.circulating_w()` is identically zero at N<=1.
"""
asked = [u for u in units
if u.share_w is not None and float(u.share_w) > 0.0]
if len(asked) < 2:
return {}
gaps = {}
for u in asked:
gap = short_by(u.share_w, u.measured_w)
if gap is not None and gap > float(min_w):
gaps[u.name] = gap
unaccounted = {u.name: gaps[u.name] for u in asked
if u.name in gaps and u.accounted is None}
if not unaccounted or len(unaccounted) == len(asked):
return {}
peers = [u.name for u in asked if u.name not in unaccounted]
out = {}
for u in asked:
if u.name not in unaccounted:
continue
out[u.name] = (
f"{REASON_CODE}: {u.name} is standing at {float(u.share_w):+.0f} W "
f"and delivering {float(u.measured_w):+.0f} W - "
f"{unaccounted[u.name]:.0f} W of its share is not arriving - while "
f"{', '.join(peers)} {'is' if len(peers) == 1 else 'are'} not in "
f"that state. Nothing this add-on reads explains it")
return out
class DeliveryWatchdog:
"""ONE PER INVERTER, evaluated once per control cycle.
Per unit rather than per site, and that is card requirement 2: the site
figure cannot say which box to look at, and naming the box is the whole
reason SAFETY-30 split per-unit from site in the first place. It sits beside
`Inverter.sign_wd` on the same object, under the same namespaced store keys,
for the same reason.
⚠️ NOT SHARED ACROSS CADENCES, for `SignWatchdog`'s reason: `since` is a
wall-clock start time, so two loops at two rates would each reset the
other's window.
"""
def __init__(self, cfg: DeliveryConfig | None = None, *,
tripped: bool = False, reason: str = ""):
self.cfg = cfg or DeliveryConfig()
self.since: datetime | None = None
# ⚠️ RESTORED FROM THE STORE, NOT DEFAULTED - SAFETY-34, and it is the
# requirement the card states in its own words. See the module note.
self.tripped = bool(tripped)
self.reason = reason if self.tripped else ""
def arm(self, cfg: DeliveryConfig) -> None:
"""Set the window. Nothing else.
⚠️ IT IS CONFIGURATION, NOT STATE, AND IT DELIBERATELY DOES NOT TOUCH
`since`, `tripped` OR `reason` - `SignWatchdog.arm()`'s argument,
which was found the expensive way: a detector that cleared its own
window when it was told what it was measuring would lose the episode it
is in the middle of, every cycle, and would never accumulate a hold.
"""
self.cfg = cfg
def evaluate(self, sentence, now: datetime) -> bool:
"""Advance one cycle. `sentence` is this unit's entry from `delinquent()`
(None when it has none). Returns True while tripped.
`now` is the caller's clock - `SignWatchdog.evaluate()`'s and
`CirculationWatchdog.evaluate()`'s discipline, so this is testable and
replayable rather than reading a clock of its own.
"""
if self.tripped:
# Latched. It does not re-evaluate, and unlike SAFETY-02 and
# SAFETY-31 that is NOT because the trip removed the evidence -
# nothing here changes what is commanded, so the evidence is still
# standing. It is because an intermittent unit that resumed must not
# erase its own record. See the module note.
return True
if sentence is None:
self.since = None
return False
if self.since is None:
self.since = now
return False
held = (now - self.since).total_seconds()
if held < self.cfg.hold_s:
return False
self.tripped = True
self.reason = (f"{sentence} - held for {held:.0f} s continuously "
f"(limit: over {self.cfg.hold_s:.0f} s)")
return True
def held_s(self, now: datetime) -> float:
"""How long the current shortfall has stood, for the status page.
Takes the clock rather than reading one, for `SignWatchdog.held_s()`'s
reason: an unclocked function is one nothing can test, which is exactly
what happened to that one.
"""
if self.since is None:
return 0.0
return (now - self.since).total_seconds()
def reset(self) -> None:
self.since = None
self.tripped = False
self.reason = ""
goodwe_controller/app/inverter.py
+33 -4
View File
@@ -122,6 +122,7 @@ import math
from datetime import datetime
from .clamp import ClampResult
from .delivery import DeliveryConfig, DeliveryWatchdog
# SAFETY-37. The freshness policy, and the ONE place its bounds are written.
# freshness.py imports nothing from this package, so the direction is one-way.
from .freshness import BOUNDS, Freshness
@@ -166,6 +167,13 @@ UNIT_A = "inverter-a"
# migration mechanical rather than transcribed: `store.migrate_unit_keys()`
# reads this same mapping, so a key renamed here cannot silently stop being
# migrated. See store.py.
# ⚠️ SAFETY-42's PAIR IS DELIBERATELY NOT IN THIS TUPLE, AND THE ABSENCE IS THE
# STATEMENT. The values here are the PRE-DEV-02 FLAT key names that
# `store.migrate_unit_keys()` moves under a unit; `delivery_tripped` /
# `delivery_reason` never existed flat, because the check that writes them
# cannot fire at a roster of one (see delivery.py) and a one-inverter install is
# the only install that ever had flat keys. Adding them here would make the
# migration look for keys that cannot exist and would report them as absent.
UNIT_STORE_KEYS = ("sign_tripped", "sign_reason")
@@ -184,7 +192,8 @@ class Inverter:
def __init__(self, name, *, setpoint_entity="", soc_entity="",
batt_entity="", batt_invert=False, max_w=2000.0,
bms_soc_floor=None, store=None, sign_cfg=None):
bms_soc_floor=None, store=None, sign_cfg=None,
delivery_cfg=None):
self.name = str(name)
self.setpoint_entity = str(setpoint_entity or "")
self.soc_entity = str(soc_entity or "")
@@ -274,6 +283,25 @@ class Inverter:
tripped=bool(data.get(self.key("sign_tripped"), False)),
reason=str(data.get(self.key("sign_reason"), "") or ""))
# --- SAFETY-42, one instance per unit, under NAMESPACED keys ---------
# ⚠️ PER UNIT BECAUSE THE CONCLUSION IS ABOUT A UNIT. "This box is not
# producing what it was told, while its sibling is" names a device, and
# naming the device is card requirement 2 - the site figure cannot say
# which one to look at, which is the whole reason SAFETY-30 split
# per-unit from site. The site-level REDUCTION that decides whether a
# unit qualifies lives in `delivery.delinquent()` and is computed once
# per cycle in `Controller.cycle()`; what is per-unit here is the
# WINDOW and the LATCH.
#
# ⚠️ AND IT IS RESTORED FROM THIS UNIT'S OWN KEYS, LIKE `sign_wd`. A
# latch held only in memory is not a latch: Home Assistant restarts, the
# `persistent_notification` goes with it, and the fault runs on unseen.
# That is SAFETY-34, and it is how the 2026-09-02 outage lasted 6 h 25 m.
self.delivery_wd = DeliveryWatchdog(
delivery_cfg or DeliveryConfig(),
tripped=bool(data.get(self.key("delivery_tripped"), False)),
reason=str(data.get(self.key("delivery_reason"), "") or ""))
# -- store ---------------------------------------------------------------
def key(self, name: str) -> str:
"""This unit's namespaced Store key for `name`. See store.unit_key()."""
@@ -436,7 +464,8 @@ def roster_names(opts: dict) -> list:
return names
def build_roster(opts: dict, store, sign_cfg=None) -> list:
def build_roster(opts: dict, store, sign_cfg=None,
delivery_cfg=None) -> list:
"""The site's inverters, in roster order.
TWO SHAPES, AND WHICH ONE IS IN FORCE IS DECIDED BY ONE TEST: is
@@ -490,7 +519,7 @@ def build_roster(opts: dict, store, sign_cfg=None) -> list:
# mirror exactly as validate.py rule 8 defaults it, so an install
# that never set the key reads the same 10 % in both places.
bms_soc_floor=_flat_bms_floor(opts),
store=store, sign_cfg=sign_cfg)]
store=store, sign_cfg=sign_cfg, delivery_cfg=delivery_cfg)]
roster = []
names = roster_names(opts)
@@ -517,7 +546,7 @@ def build_roster(opts: dict, store, sign_cfg=None) -> list:
# SAFETY-32's own defect written in three lines - the shallower pack
# graded against the deeper pack's BMS.
bms_soc_floor=_entry_bms_floor(entry),
store=store, sign_cfg=sign_cfg))
store=store, sign_cfg=sign_cfg, delivery_cfg=delivery_cfg))
return roster
goodwe_controller/app/main.py
+285 -1
View File
@@ -38,6 +38,10 @@ from .arbiter import (
Claim, Resolution, actuator_for, resolve,
)
from .control import Tuning, compute, maintenance_charge_floor, peak_at_risk
from .delivery import (
DELIVERY_OPTION_DEFAULTS, DeliveryConfig, UnitDelivery,
account as delivery_account, delinquent,
)
from .circulation import (
CIRC_OPTION_DEFAULTS, CirculationConfig, CirculationWatchdog,
circulating_w, opposing,
@@ -216,8 +220,17 @@ class Controller:
hold_s=float(opts.get("site_sign_disagree_s",
SIGN_OPTION_DEFAULTS["site_sign_disagree_s"])),
)
# SAFETY-42. THE WINDOW A UNIT MAY BE SHORT OF ITS SHARE BEFORE IT IS
# NAMED. There is no second key: the WATTS half of that check is
# `Tuning.saturation_w`, read at runtime rather than copied - see
# `delivery.MIN_W_IS_SATURATION_W`, and CFG-03 on why a fourth copy of a
# default is worth avoiding.
self.delivery_cfg = delivery_cfg = DeliveryConfig(
hold_s=float(opts.get("delivery_hold_s",
DELIVERY_OPTION_DEFAULTS["delivery_hold_s"])),
)
self.store_migration = migrate_unit_keys(store, UNIT_A)
self.inverters = build_roster(opts, store, sign_cfg)
self.inverters = build_roster(opts, store, sign_cfg, delivery_cfg)
# ⚠️ CFG-04. SAY WHICH SHAPE IS IN FORCE, BECAUSE THE OTHER ONE'S KEYS
# ARE STILL SITTING IN THE UI. A declared `inverters:` roster reads none
# of `soc_entity`, `batt_entity`, `batt_invert` or `max_w`, and an
@@ -365,6 +378,32 @@ class Controller:
"opposed the meter.", inv.name,
inv.sign_wd.reason or "(reason not recorded)")
# --- SAFETY-42 + SAFETY-34 -------------------------------------------
# ⚠️ A LATCH THAT DOES NOT ACT, WHICH IS THE ONE ASYMMETRY WITH EVERY
# OTHER RESTORED LATCH ON THIS OBJECT, AND IT IS DELIBERATE. SAFETY-02's
# and SAFETY-31's latches survive a restart because they command 0 W and
# an add-on that came back armed would drive straight into the fault
# again. This one commands nothing at all: it is restored so that the
# ALARM survives, which is the whole of SAFETY-34 and is exactly how the
# 2026-09-02 outage went unseen for 6 h 25 m with the detector correct
# the entire time. `delivery.py` carries the argument for reporting
# rather than acting, with the rig measurements on both sides of it.
#
# `_delivery_announced` is the per-PROCESS half - see `_circ_announced`,
# which is the same pattern for the same reason. The store says "this
# unit is tripped"; the flag says "this process has put that in front of
# a human". A restart clears the flag and keeps the latch.
self._delivery_announced = set()
for inv in self.inverters:
if inv.delivery_wd.tripped:
_LOG.error(
"SAFETY-42 was TRIPPED on %s before this restart and stays "
"tripped: %s. NOTHING IS BEING COMMANDED DIFFERENTLY - this "
"guard reports and does not act - so if that inverter is "
"still not delivering, the site is still short by its "
"share. Restarting does not clear this.",
inv.name, inv.delivery_wd.reason or "(reason not recorded)")
# --- SAFETY-30: THE SITE HALF OF THE SIGN CHECK ----------------------
# ⚠️ ONE PER SITE, BESIDE THE ONE PER INVERTER, AND ARMED ONLY AT N>=2.
# The per-unit watchdogs above ask "is THIS actuator doing what I told
@@ -1738,6 +1777,79 @@ class Controller:
"add-on restart AND a Home Assistant restart; restarting will "
"not clear it.", self.circ_wd.reason)
# --- SAFETY-42, AND IT RUNS HERE FOR SAFETY-02's AND SAFETY-31's REASON
# Evaluated EVERY cycle, unconditionally, ABOVE the "inputs missing but
# not yet stale" hold below. A run of held cycles would otherwise leave
# each unit's `since` untouched across the gap and its window would
# resume from a start time that predates it - the ordering bug the sign
# watchdog's own note describes, in a third detector.
#
# ⚠️ THE OPERANDS ARE `standing` AND `inv.batt`, THE SAME PAIR SAFETY-30
# GRADES, AND THE CONCLUSION IS A DIFFERENT ONE. SAFETY-30 asks whether
# they OPPOSE; this asks whether the second one ARRIVED. A unit sitting
# at 0 W against a +2000 W share opposes nothing - it is not pushing the
# wrong way, it is absent - so the sign gate never opens, the site sum
# still agrees with the meter in direction, circulating power is
# identically zero, and the only trace left is `reason ==
# "saturated-freeze"`, which clamp.py:93-96 documents as the alarm an
# operator learns to ignore. That is this card.
#
# ⚠️ AND `delinquent()` IS CALLED ONCE, OVER THE WHOLE ROSTER, BECAUSE
# THE CHECK IS A REDUCTION ACROSS UNITS AND NOT A PER-UNIT PREDICATE.
# What separates a wedged box from a legitimately saturated site is that
# the site's shortfall is a PROPER SUBSET of the units asked to deliver.
# Called per unit it would be a shortfall test, which is the saturation
# signal again. delivery.py carries the measurement: a pack at its own
# cut-out and a wedged unit produce IDENTICAL traces at the unit and at
# the meter, so nothing computed from one unit alone can tell them
# apart. At a roster of one the reduction is empty for every input.
for inv in self.inverters:
inv.delivery_wd.arm(self.delivery_cfg)
_delinquent = delinquent(
[UnitDelivery(
inv.name, standing[inv.name], inv.batt,
delivery_account(
# ⚠️ `is_live()` AT ITS OWN DEFAULT WINDOW, which is
# `LINK_STALE_S` - the SAME call `live_units()` makes for
# the allocator. It has to be: what this branch means is
# "the allocator has already dropped this unit and said so",
# and a different window here would be this check answering
# a question about a roster the allocator did not build.
live=inv.is_live(now)[0],
parked=inv.sign_wd.tripped,
soc=inv.soc, bms_floor=inv.bms_soc_floor))
for inv in self.inverters],
self.tuning.saturation_w)
for inv in self.inverters:
was_short = inv.delivery_wd.tripped
if inv.delivery_wd.evaluate(_delinquent.get(inv.name), now) \
and not was_short:
# REASON FIRST, FLAG SECOND, and each `set` writes the file -
# the same atomicity ordering the three trips above use: a crash
# between the two must not persist a latch with no explanation
# for the operator who has to decide about clearing it.
#
# ⚠️ AND THERE IS NO FLAT MIRROR OF THIS PAIR, WHERE SAFETY-02
# HAS ONE. store.py's rule 2 keeps the flat keys "so a rollback
# still finds its latch", and the version being rolled back TO
# drives exactly ONE actuator - a roster at which this check is
# inert by construction. A flat mirror would hand a downgraded
# add-on a latch taken on evidence about a roster it cannot have.
inv.store_set("delivery_reason", inv.delivery_wd.reason)
inv.store_set("delivery_tripped", True)
self._delivery_announced.discard(inv.name)
self.log_event(f"NOT DELIVERING {inv.name}: "
+ inv.delivery_wd.reason)
_LOG.error(
"SAFETY-42 TRIPPED on %s - %s. NOTHING IS COMMANDED "
"DIFFERENTLY: this guard reports and does not act, because "
"commanding a unit that is not listening achieves nothing "
"(HW-01) and the only re-allocation this add-on can express "
"would throw away a partially-delivering unit's remaining "
"output. The site is short by that unit's share until "
"somebody looks at it. This survives an add-on AND a Home "
"Assistant restart.", inv.name, inv.delivery_wd.reason)
stale_after = float(self.o.get("stale_input_s", 15))
# ⚠️ `any`, NOT the sole unit's. The hold exists so a single missed poll
# does not become a fault; a LATCHED inverter must still reach the claim
@@ -2458,6 +2570,15 @@ class Controller:
# fresh trip instead of getting its own.
await self.check_circulating()
# --- SAFETY-42's alarm, and its boundary is the LATCH, like SAFETY-31's
# It grades `inv.delivery_wd.tripped`, which was advanced near the top
# of this method beside SAFETY-02 and SAFETY-31 and which nothing since
# has touched. Called on EVERY cycle rather than only on the cycle that
# trips, because its second job is re-raising the alarm after a restart
# into an existing trip - see its docstring on why that shares one code
# path with the fresh trip instead of getting its own.
await self.check_not_delivering()
# --- SAFETY-07's alarm, and it is NOT the engagement ------------------
# Its boundary is unconstrained: it grades `self.peak_guard`, which this
# method assigned above and nothing since has touched.
@@ -2795,6 +2916,85 @@ class Controller:
f"know why two inverters were opposing."),
})
async def check_not_delivering(self) -> None:
"""SAFETY-42 + SAFETY-34. The alarm, and it must survive a restart.
⚠️ THE LATCH IS NOT THE ALARM, and this guard has nothing BUT the alarm.
`unit/<name>/delivery_tripped` in `/data/state.json` survives everything,
but nothing looks at a JSON file at 3 a.m. What the operator sees is a
`persistent_notification`, and THAT IS IN MEMORY: a Home Assistant
restart deletes it while the fault runs on. That is precisely how the
2026-09-02 outage went unseen for 6 h 25 m with the detector correct the
whole time (SAFETY-34), and it matters more here than anywhere else in
this file, because SAFETY-02 and SAFETY-31 at least leave a stopped site
for somebody to notice. This one leaves a site that looks like it is
working and is quietly delivering one unit less than it has.
So the durable half and the visible half are joined HERE, and the join
is `_delivery_announced`: a per-PROCESS set of unit names, deliberately
not persisted. The latch says "this unit is tripped"; membership says
"this process has put that in front of a human". A restart empties the
set and keeps the latch, so the notification is RE-RAISED on the first
cycle after every add-on start - which is also the first cycle after a
Home Assistant restart, because the add-on is restarted with it.
⚠️ ONE CODE PATH FOR BOTH, ON PURPOSE - `check_circulating()`'s
argument. A fresh trip and a restart into an existing trip raise the
alarm through these same lines; two paths would be two chances for the
restart one to rot unnoticed, and the restart one is the case nobody
exercises by hand.
⚠️ AND ONE NOTIFICATION ID PER UNIT. Every other alarm in this file has
a single fixed id because its subject is the site; this one's subject is
a named box, and two wedged inverters must produce two cards rather than
one replacing the other. The id is built from the same `unit_slugs()`
TEL-07 announces the MQTT entities under, so the card and the entity an
operator correlates it with carry the same token.
"""
names = self.units
slugs = unit_slugs(names)
for inv, slug in zip(self.inverters, slugs):
if not inv.delivery_wd.tripped or inv.name in self._delivery_announced:
continue
# Added BEFORE the await, for `check_circulating()`'s reason:
# `hass.call()` can raise or hang, and a retry every cycle for a
# broker that is down would be an alarm storm on the event bus. The
# latch is already durable and the log line already fired; this
# surface is best-effort by design.
self._delivery_announced.add(inv.name)
peers = ", ".join(n for n in names if n != inv.name) or "no sibling"
await self.hass.call("persistent_notification", "create", {
"notification_id": f"goodwe_not_delivering_{slug}",
"title": f"GoodWe: {inv.name} is not delivering its share",
"message": (
f"{inv.delivery_wd.reason}\n\nWHY NOTHING ELSE SAID SO. A "
f"unit sitting at 0 W against a non-zero share is not "
f"pushing the wrong way, it is ABSENT: the per-unit sign "
f"check needs opposition on both sides of "
f"{self.unit_sign_cfg.min_w:.0f} W and never opens, the "
f"site sum still agrees with the meter in "
f"direction, and circulating power is exactly zero when one "
f"unit does nothing. The only other trace is a saturation "
f"freeze, which this add-on's own code calls an alarm "
f"operators learn to ignore.\n\nNOTHING HAS BEEN COMMANDED "
f"DIFFERENTLY. This guard reports and does not act. "
f"Commanding an inverter that has stopped listening "
f"achieves nothing - that has happened on this hardware, "
f"once for 6 h 25 m - and the only re-allocation this "
f"add-on can express is all-or-nothing, which would throw "
f"away the output of a unit that is delivering PART of its "
f"share. So {peers} is still regulating and the site is "
f"short by up to {inv.rating_w():.0f} W until somebody "
f"looks at {inv.name}.\n\nWHAT TO CHECK, in order: is "
f"{inv.name} answering on RS485 at all; is its pack colder "
f"or emptier than its configured bms_soc_floor says (if it "
f"is, that setting does not describe this battery and this "
f"alarm is telling you so); and is anything else writing to "
f"it.\n\nThis alarm survives a Home Assistant restart. "
f"Restarting will not clear it; start control from the "
f"add-on UI once you know why."),
})
async def check_meter_frozen(self) -> None:
"""SAFETY-18. Alarm on a meter that answers forever without changing.
@@ -3557,6 +3757,49 @@ class Controller:
+ (f" - now {_circ:.0f} W" if _circ else "")
+ (f", opposed for {held:.0f} s" if held else "")})
# --- SAFETY-42 -------------------------------------------------------
# ⚠️ PER INVERTER, AND WARN RATHER THAN BAD, WHICH IS THE ONE PLACE THE
# "reports, does not act" DECISION IS VISIBLE TO AN OPERATOR. Every
# other latched row in this list means the site is at 0 W and a red
# cross is the truth. This one means the site is STILL RUNNING and is
# quietly short by one unit's share; rendering it as a hard failure
# would say the controller had stopped, which it has not. It is not "ok"
# either - `ok: False` with `warn: True` is this list's caution, and a
# caution is what a fault nobody is acting on actually is.
if len(self.inverters) > 1:
for inv in self.inverters:
tag = f" ({inv.name})"
if inv.delivery_wd.tripped:
out.append({"ok": False, "warn": True,
"text": f"NOT DELIVERING{tag} - "
+ inv.delivery_wd.reason
+ ". Nothing has been commanded differently and "
"the rest of the roster is still regulating, "
"so the site is short by this unit's share. "
"Starting control clears this row; restarting "
"the add-on does NOT, and neither does "
"restarting Home Assistant."})
else:
_dheld = inv.delivery_wd.held_s(datetime.now(timezone.utc))
out.append({"ok": True, "warn": False,
"text": f"delivery watchdog{tag}: armed (reports after "
f"{inv.delivery_wd.cfg.hold_s:.0f} s short of "
f"its share by over "
f"{self.tuning.saturation_w:.0f} W, with a "
f"sibling that is not)"
+ (f" - short for {_dheld:.0f} s now"
if _dheld else "")})
else:
# AC 5, and the same honesty `circulating power: n/a` uses one block
# up. At one inverter the reduction this check is has no proper
# subset to find, for EVERY input, so "armed" would be a claim to be
# watching something that cannot happen.
out.append({"ok": True, "warn": False,
"text": "delivery watchdog: n/a at one inverter - a unit "
"short of its share is indistinguishable from a "
"plant that cannot deliver until there is a "
"sibling to compare it with"})
if not o.get("estop_fitted", False):
out.append({"ok": False, "warn": True,
"text": "no e-stop fitted: if this host dies the battery stays "
@@ -3616,6 +3859,26 @@ class Controller:
level, banner = "bad", f"NOT READY — {bad[0].get('banner') or bad[0]['text']}"
elif not self.auto:
level, banner = "warn", "Stopped — inverter commanded to 0 W"
elif any(i.delivery_wd.tripped for i in self.inverters):
# ⚠️ SAFETY-42, AND IT IS A `warn` BANNER RATHER THAN A `bad` ONE,
# WHICH IS THE WHOLE "reports, does not act" DECISION IN ONE LINE.
# Every `bad` banner above means the site is at 0 W; this one means
# the site is STILL RUNNING and quietly short by one unit's share,
# and a red headline would say the controller had stopped.
#
# ⚠️ BUT IT MUST DISPLACE "Running — holding grid at zero", AND THAT
# IS NOT COSMETIC. That sentence is a claim the site cannot make
# while a unit is absent - the loop is saturated and the meter is
# short - and "the page reads healthy while the fault runs" is
# precisely the shape this whole card exists to remove. It sits
# BELOW "Stopped" because a stopped site is the more urgent fact and
# the banner carries one sentence.
_nd = ", ".join(i.name for i in self.inverters
if i.delivery_wd.tripped)
level, banner = "warn", (
f"NOT DELIVERING — {_nd} is standing at a share it is not "
f"producing. The site is still running and short by it; "
f"nothing has been commanded differently.")
elif self.maint.phase != IDLE:
# ⚠️ THE PACK IS NAMED ONLY WHEN THERE IS MORE THAN ONE. At a roster
# of one this is the string it has always been, character for
@@ -3766,6 +4029,27 @@ class Controller:
self.store.set("circ_reason", "")
self.log_event("circulating-power trip cleared by operator")
self._circ_announced = False
# SAFETY-42, cleared by the SAME press, and for a reason that is
# NOT the one above. The other three latches are cleared here
# because their trip removed its own evidence - a site commanding
# 0 W cannot oppose anything - so only a human can say the fault
# is gone. This one commands nothing, so its evidence is still
# standing and it could in principle clear itself. It does not:
# a unit that stopped delivering for minutes and then resumed is
# an INTERMITTENT fault, and a detector that erased its own
# record of that would be the least useful thing in this file.
# ⚠️ AND `_delivery_announced` GOES WITH IT, like
# `_circ_announced`: leaving a name in it through a clear would
# mean the NEXT trip on that unit latched and raised no
# notification at all.
for inv in self.inverters:
if inv.delivery_wd.tripped:
self.log_event(f"not-delivering report on {inv.name} "
f"cleared by operator")
inv.delivery_wd.reset()
inv.store_set("delivery_tripped", False)
inv.store_set("delivery_reason", "")
self._delivery_announced = set()
if not self.auto:
# ⚠️ THE WHOLE SITE, TARGETS INCLUDED - see command_site_zero().
# Zeroing `self.target` alone would leave each unit's last share