SAFETY-02: a loop can be fresh, tracking and inside its rating and still push the wrong way
The stale-input watchdog asks whether the meter is still talking. The saturation freeze asks whether the inverter can follow. The output clamp asks whether the number is too big. A loop can pass all three while charging the battery out of the grid, because the only thing wrong is the sign. On 2026-08-20 a commercial controller did exactly that four times in one day, for up to 77 minutes at a stretch, reaching a commanded -14 768 W against a ±5000 W inverter. app/watchdog.py compares the emitted setpoint against the authoritative P1 figure and, when they oppose each other with both over 300 W for 180 s continuously, files a failsafe SET claim at P_SAFETY, commands 0 W and latches there until an operator starts control again. Validated against that day rather than against an argument: sim/replay_sign.py in the infra repo drives this module over the committed captures of both 2026-08-20 and the healthy 2026-08-23 and reaches 18 trips on the fault day - every one inside a window NOTES.md §4.7 accounts for - and 0 on the healthy day. The four episodes do share one signature: 0.3 / 0.9 / 10.5 / 0.0 % sign agreement inside them against 92.9 % healthy. Three decisions the ticket could not make: - It is DISARMED whenever the grid-following loop is not what is driving. Sign agreement is a property of a loop that follows the meter, and a maintenance grid charge opposes the meter by design - 2500 W of charge into a house drawing 500 W is +3000 W on the meter against a -2500 W command, for hours. The gate tests the emitted number, not the winning claim's name, because a limit claim can let the loop "win" and still replace its value. - The trip LATCHES. Commanding 0 W removes the command side of the comparison, so the disagreement always clears the moment the trip works. - sign_disagree_w at or above max_w is refused at startup. The arbiter never lets a command exceed max_w, so such a threshold leaves the watchdog incapable of firing while the status page still reports it armed. What it does not catch, said out loud: an inverter that ACTS opposite to its command moves the meter further the same way, leaving command and meter in agreement. That is the saturation freeze's job. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Du77usMj8XNKNFZGmUiWDa
This commit is contained in:
co-authored by
Claude Opus 5
parent
35856a73ae
commit
9aed43990b
@@ -24,6 +24,9 @@ The inverter **holds its last command forever** — it has no meter-timeout. So:
|
||||
commanding it.
|
||||
- This add-on commands 0 W when its inputs go missing, when you stop control,
|
||||
and when it shuts down.
|
||||
- This add-on commands 0 W **and latches there** if what it is commanding and
|
||||
what the meter reads point opposite ways for three minutes — see "The
|
||||
sign-disagreement failsafe" below.
|
||||
- The **optional RS485 e-stop** is the only thing that covers this machine
|
||||
dying. Without it, a failed host leaves the battery latched at its last
|
||||
command until someone intervenes.
|
||||
@@ -278,6 +281,54 @@ behind it would trip the firmware watchdog on a system that is working fine.
|
||||
| `cycle_floor_s` | 15 | The longest the loop may go without evaluating. A deadline on the gap between cycles, reset by every cycle — on a healthy meter it never elapses and changes nothing. Raise it only if your meter is slower than ~10 s; it must stay under `meter_max_age_s`, past which there is no reading to evaluate anyway |
|
||||
| `stale_input_s` | 15 | How long inputs may be missing before commanding 0 W. In P1 mode this clock starts only *after* `meter_max_age_s` has already expired — the two stack, see "How long a dead meter takes to reach 0 W" |
|
||||
| `auto_start` | false | Start controlling on boot (only after commissioning) |
|
||||
| `sign_disagree_w` | 300 | Both the command and the meter must exceed this before a sign difference counts. Must stay **below `max_w`** or the watchdog can never fire, and the add-on refuses to start |
|
||||
| `sign_disagree_s` | 180 | How long they must stay opposed, continuously, before the failsafe trips |
|
||||
|
||||
#### The sign-disagreement failsafe
|
||||
|
||||
If the setpoint this add-on is emitting and the grid figure it is regulating on
|
||||
point **opposite ways** — charging the battery while the house imports, or
|
||||
discharging it while the house exports — with both over `sign_disagree_w`, for
|
||||
`sign_disagree_s` **continuously**, the add-on commands **0 W** and holds it
|
||||
there. The banner reads `FAILSAFE`, the commissioning list says which two
|
||||
numbers disagreed and for how long, and the `Controller status` entity reports
|
||||
`failsafe`.
|
||||
|
||||
**It does not clear itself.** Pressing *Start / stop control* twice clears it.
|
||||
That is deliberate: commanding 0 W removes the command side of the comparison,
|
||||
so the disagreement always vanishes the instant the trip works, and a watchdog
|
||||
that took that as proof the fault was over would hand the inverter straight back
|
||||
to the loop that was driving the wrong way.
|
||||
|
||||
**What it is for.** This is a loop-integrity check: it asks whether the loop's
|
||||
output direction still matches the input state it was regulating. It catches a
|
||||
polarity error anywhere between the meter and the wire, and an integrator so far
|
||||
wound up that unwinding takes minutes. It is the check that would have caught all
|
||||
four episodes of 2026-08-20, when a commercial controller charged the battery out
|
||||
of the grid, four times, for up to 77 minutes at a time.
|
||||
|
||||
**What it is not for.** It does not detect an inverter that acts opposite to what
|
||||
it was told — that moves the meter *further* the same way, so command and meter
|
||||
agree while everything runs away. That is the saturation freeze's job
|
||||
(`saturation_w` / `saturation_cycles`), which compares the command against the
|
||||
battery's actual power. The two checks are complementary and neither replaces the
|
||||
other.
|
||||
|
||||
**Why it will not false-trip on ordinary operation.** A battery that refuses a
|
||||
command (empty, full, standby) leaves the meter where it was, so command and
|
||||
meter agree. A frozen meter drives the loop into agreeing with it. Around the
|
||||
resting point both signals are inside the 300 W gate, so the deadband can never
|
||||
produce one. A load step does flip the meter's sign before the loop catches up —
|
||||
measured on a full captured day, the longest such crossing is 30 s, against a
|
||||
180 s window. And the watchdog is **disarmed whenever the grid-following loop is
|
||||
not what is driving**: a maintenance grid charge opposes the meter by design and
|
||||
must not be mistaken for a fault.
|
||||
|
||||
Both numbers are measured rather than chosen, and are exposed only because the
|
||||
ticket requires defaults rather than constants. On the captured healthy day sign
|
||||
agreement is 92.9 %; on the four captured fault episodes it is 0.3 %, 0.9 %,
|
||||
10.5 % and 0.0 %. There is no reason to move either value without a capture that
|
||||
says so.
|
||||
|
||||
#### Saturation is counted in cycles, not seconds
|
||||
|
||||
|
||||
Reference in New Issue
Block a user