SAFETY-02: a loop can be fresh, tracking and inside its rating and still push the wrong way

The stale-input watchdog asks whether the meter is still talking. The saturation
freeze asks whether the inverter can follow. The output clamp asks whether the
number is too big. A loop can pass all three while charging the battery out of
the grid, because the only thing wrong is the sign.

On 2026-08-20 a commercial controller did exactly that four times in one day,
for up to 77 minutes at a stretch, reaching a commanded -14 768 W against a
±5000 W inverter. app/watchdog.py compares the emitted setpoint against the
authoritative P1 figure and, when they oppose each other with both over 300 W
for 180 s continuously, files a failsafe SET claim at P_SAFETY, commands 0 W and
latches there until an operator starts control again.

Validated against that day rather than against an argument: sim/replay_sign.py
in the infra repo drives this module over the committed captures of both
2026-08-20 and the healthy 2026-08-23 and reaches 18 trips on the fault day -
every one inside a window NOTES.md §4.7 accounts for - and 0 on the healthy day.
The four episodes do share one signature: 0.3 / 0.9 / 10.5 / 0.0 % sign
agreement inside them against 92.9 % healthy.

Three decisions the ticket could not make:

- It is DISARMED whenever the grid-following loop is not what is driving. Sign
  agreement is a property of a loop that follows the meter, and a maintenance
  grid charge opposes the meter by design - 2500 W of charge into a house
  drawing 500 W is +3000 W on the meter against a -2500 W command, for hours.
  The gate tests the emitted number, not the winning claim's name, because a
  limit claim can let the loop "win" and still replace its value.
- The trip LATCHES. Commanding 0 W removes the command side of the comparison,
  so the disagreement always clears the moment the trip works.
- sign_disagree_w at or above max_w is refused at startup. The arbiter never
  lets a command exceed max_w, so such a threshold leaves the watchdog incapable
  of firing while the status page still reports it armed.

What it does not catch, said out loud: an inverter that ACTS opposite to its
command moves the meter further the same way, leaving command and meter in
agreement. That is the saturation freeze's job.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Du77usMj8XNKNFZGmUiWDa
This commit is contained in:
glenn schrooyen
2026-08-28 21:51:35 +02:00
co-authored by Claude Opus 5
parent 35856a73ae
commit 9aed43990b
9 changed files with 755 additions and 8 deletions
+31
View File
@@ -74,6 +74,17 @@ NOT READY banner already names the problem, (b) does not apply.
was overruled. That firmware file's own comment says a lower bound on this
"belongs in CFG-01, not here" - so here it is.
5. `sign_disagree_w` at or above `max_w`. [(a)]
The arbiter's `supervised` limit bounds every command to +-max_w, and
SAFETY-02's predicate needs `abs(command) > sign_disagree_w`. Set the
threshold at or above the ceiling and the command side of the comparison
is unsatisfiable BY CONSTRUCTION: the watchdog can never fire, on any
input, ever. Same shape as 1 and 4 - the operator set a number, a layer
they cannot see makes it mean something else, and nothing says so. Worse
here, because what the status page reports in that state is "sign
watchdog: armed". A disarmed guard that advertises itself as armed is the
one thing this project has decided repeatedly it will not ship.
WHAT IS DELIBERATELY *NOT* REFUSED, because it is untidy rather than unsafe:
- `integrator_max_w` above `max_w`. DOCS.md is right that the only thing the
@@ -238,6 +249,26 @@ def validate_startup_config(opts: dict, dev_min=None, dev_max=None) -> list[str]
"or rebuild the firmware with a matching max_age_s and update "
"validate.FIRMWARE_INGEST_VETO_S.")
# --- 5. a sign watchdog that can actually fire ---------------------------
# ⚠️ Strictly `>=`, matching the predicate exactly: watchdog.py tests
# `abs(commanded_w) > min_w`, and the arbiter holds `abs(commanded_w)` at or
# below max_w, so threshold == max_w is already unsatisfiable. Anything
# BELOW max_w is reachable and therefore fine - this refuses the disarmed
# case only, not "close to the rail".
sign_w = _number(opts, "sign_disagree_w", 300.0)
if sign_w is None:
problems.append(f"sign_disagree_w {opts.get('sign_disagree_w')!r} is not a number")
elif max_w is not None and max_w > 0 and sign_w >= max_w:
problems.append(
f"sign_disagree_w is {sign_w:.0f} W but max_w is {max_w:.0f} W, so "
"the SAFETY-02 sign-disagreement watchdog can never fire: the "
"arbiter never lets a command exceed max_w, and the watchdog needs "
f"a command over {sign_w:.0f} W. It would sit there reporting "
"'armed' on the status page while being incapable of tripping - the "
f"one runaway failure mode this system has actually observed. Lower "
f"sign_disagree_w below {max_w:.0f} W (300 W is the measured "
"default), or raise max_w.")
# --- warnings: real, but not worth refusing over -------------------------
integ = opts.get("integrator_max_w")
integ_w = None if not integ else _number(opts, "integrator_max_w", None)