SAFETY-02 review: a latch that a restart clears is not a latch
BLOCKER. `auto` was persisted and `sign_wd.tripped` was not, so reconstructing the Controller against the same store - which is what a container restart does - brought it back ARMED, into the same fault, unattended: tripped=False, banner back to NOT READY, and it resumed commanding -1000 then -2000 W before re-tripping 180 s later. "Restart the add-on" is the first thing an operator looking at a FAILSAFE banner and a 0 W inverter will try, and under a supervisor that restarts a crashed add-on it is a loop of three-minute wrong-direction bursts. DOCS.md's "it does not clear itself" was false across a restart. `sign_tripped` and `sign_reason` are Store keys now, beside the maintenance state and for the reason store.py's own docstring gives. Written the moment the trip happens - reason first, flag second, so a crash between the two can never persist a latch with no explanation - restored in Controller.__init__, cleared only by the operator's start. The regression is covered: reverting the restore fails seven checks in test_watchdog.py. Review follow-ups, no behaviour change: - The arming gate is tested against the REAL arbiter. It is the decision the ticket turns on and it was asserted in three comment blocks and verified by nothing - every case passed `following` in as a literal. Twelve claim configurations now derive it from resolve(), including the redirect case `winner == "loop"` alone would have got wrong and the `lo` limit that raises magnitude without flipping sign. - The MQTT `status: "failsafe"` value is tested. The check that claimed to cover it restated the line above it and never touched the payload. - The worked example described a config that does not ship: at the defaults maintenance_charge_w 2500 against max_w 2000 is an arbiter contradiction and 0 W, not a -2500 W charge. Rewritten at 1500 W. The contradiction is a real pre-existing bug - a default cheap-window maintenance charge silently never happens - filed separately and deliberately not fixed here. - Forward hazard recorded at the gate: `winner == "loop"` disarms the watchdog under any higher-priority SET, and a peak-shaving strategy at P_PEAK IS grid-following in sign, so it would blind this for every peak event. - disagrees() is public so the replay harness stops carrying its own copy of the predicate it grades. - held_s takes the clock instead of reading one. - The two limits are now each stated in both places rather than one each. - The no-false-positive argument is tied to control.py's freeze permitting the command to cross zero; under the shrink-only encoding control.py records as rejected, this would false-trip on a load step. - Measured figures corrected: 93.4 % / 4212 of 17280, per-cluster 0.3/0.9/10.5/0.0, and the near-circularity of those figures is now stated rather than leaned on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Du77usMj8XNKNFZGmUiWDa
This commit is contained in:
co-authored by
Claude Opus 5
parent
2513ba1f12
commit
cc4e88490b
@@ -105,10 +105,27 @@ class Controller:
|
||||
)
|
||||
|
||||
# SAFETY-02. One instance, one actuator, one cadence - see watchdog.py.
|
||||
self.sign_wd = SignWatchdog(SignWatchdogConfig(
|
||||
min_w=float(opts.get("sign_disagree_w", 300)),
|
||||
hold_s=float(opts.get("sign_disagree_s", 180)),
|
||||
))
|
||||
#
|
||||
# ⚠️ RESTORED FROM THE STORE, and that is the difference between a latch
|
||||
# and a pause. A trip held only in memory is cleared by the one action an
|
||||
# operator staring at a FAILSAFE banner is most likely to take - restart
|
||||
# the add-on - and the controller then comes back armed, into the fault,
|
||||
# with nobody involved. Restored here, the banner and the 0 W survive the
|
||||
# restart and only `auto_toggle` clears them.
|
||||
self.sign_wd = SignWatchdog(
|
||||
SignWatchdogConfig(
|
||||
min_w=float(opts.get("sign_disagree_w", 300)),
|
||||
hold_s=float(opts.get("sign_disagree_s", 180)),
|
||||
),
|
||||
tripped=bool(store.data.get("sign_tripped", False)),
|
||||
reason=str(store.data.get("sign_reason", "") or ""),
|
||||
)
|
||||
if self.sign_wd.tripped:
|
||||
_LOG.error("SAFETY-02 was TRIPPED before this restart and stays "
|
||||
"tripped: %s. The inverter is held at 0 W. Restarting the "
|
||||
"add-on does not clear this - start control from the UI "
|
||||
"once you know why the command opposed the meter.",
|
||||
self.sign_wd.reason or "(reason not recorded)")
|
||||
|
||||
# P1 ingestion (TEL-01). `meter_source: off` keeps the original
|
||||
# single-entity meter_entity path, so an existing install is unchanged
|
||||
@@ -243,9 +260,18 @@ class Controller:
|
||||
standing = self.last_write if self.last_write is not None else self.target
|
||||
if self.sign_wd.evaluate(standing, self.grid, self.following, now) \
|
||||
and not was_tripped:
|
||||
# ⚠️ REASON FIRST, FLAG SECOND, and each `set` writes the file. The
|
||||
# ordering is the whole atomicity story: a crash between the two
|
||||
# persists a reason nothing reads, which is harmless, where the other
|
||||
# order persists a latch with no explanation for the operator who has
|
||||
# to decide whether to clear it.
|
||||
self.store.set("sign_reason", self.sign_wd.reason)
|
||||
self.store.set("sign_tripped", True)
|
||||
self.log_event("FAILSAFE " + self.sign_wd.reason)
|
||||
_LOG.error("SAFETY-02 TRIPPED - %s. Commanding 0 W and holding it "
|
||||
"until control is started again.", self.sign_wd.reason)
|
||||
"until control is started again. This survives an add-on "
|
||||
"restart; restarting will not clear it.",
|
||||
self.sign_wd.reason)
|
||||
|
||||
stale_after = float(self.o.get("stale_input_s", 15))
|
||||
if not self.inputs_ok and not self.sign_wd.tripped:
|
||||
@@ -358,15 +384,33 @@ class Controller:
|
||||
#
|
||||
# ⚠️ "THE LOOP WON" IS NOT ENOUGH, and the difference is a whole class of
|
||||
# false trip. A LIMIT claim can survive the loop winning and still
|
||||
# replace its number: maintenance's charge-only shaping is
|
||||
# `hi = -charge_w`, so the loop can ask for +500 W of discharge, win the
|
||||
# arbitration, and have the emitted value come out at -2500 W of grid
|
||||
# charge. Winner says "loop"; the wire carries maintenance's intent. So
|
||||
# the test is on the NUMBER, not on the name: the loop is following only
|
||||
# while the emitted value points the way the loop asked and is no larger
|
||||
# than it asked - i.e. limits reduced it, they did not redirect it.
|
||||
# Zero on either side falls out as not-following, which costs nothing:
|
||||
# a 0 W command cannot be a wrong-direction command.
|
||||
# replace its number: maintenance's cheap-window shaping is
|
||||
# `hi = -maintenance_charge_w`, so with `maintenance_charge_w: 1500` the
|
||||
# loop can ask for +500 W of discharge, win the arbitration, and have the
|
||||
# emitted value come out at -1500 W of grid charge. Winner says "loop";
|
||||
# the wire carries maintenance's intent. So the test is on the NUMBER,
|
||||
# not on the name: the loop is following only while the emitted value
|
||||
# points the way the loop asked and is no larger than it asked - i.e.
|
||||
# limits reduced it, they did not redirect it. Zero on either side falls
|
||||
# out as not-following, which costs nothing: a 0 W command cannot be a
|
||||
# wrong-direction command. The `abs(...) <= abs(asked)` half is
|
||||
# load-bearing on its own: a `lo` limit can raise the magnitude without
|
||||
# flipping the sign, and that is not the loop following either.
|
||||
#
|
||||
# (1500 W because at the SHIPPED defaults - maintenance_charge_w 2500 vs
|
||||
# max_w 2000 - that claim is `lo=-2000 > hi=-2500`, which the arbiter
|
||||
# calls a contradiction and resolves to 0 W. Filed separately; the
|
||||
# redirect hazard here is real at any charge rate inside max_w.)
|
||||
#
|
||||
# ⚠️ FORWARD HAZARD, AND IT IS NOT HYPOTHETICAL - the claim block above
|
||||
# already anticipates it. `winner == "loop"` disarms this watchdog under
|
||||
# ANY higher-priority SET claim. Today only maintenance and the safety
|
||||
# stops emit one, and neither is grid-following, so the gate costs
|
||||
# nothing. A peak-shaving strategy claiming SET at P_PEAK **is**
|
||||
# grid-following in sign, and the moment it lands this watchdog goes
|
||||
# blind for the duration of every peak event. Whoever adds it must widen
|
||||
# the gate to "the winning strategy follows the meter" rather than
|
||||
# "the winner is the loop", not just add another name to a comparison.
|
||||
asked = self.loop_w
|
||||
self.following = (resolution.winner == "loop" and asked is not None
|
||||
and resolution.target_w * asked > 0
|
||||
@@ -600,9 +644,10 @@ class Controller:
|
||||
"text": "SIGN-DISAGREEMENT FAILSAFE - " + self.sign_wd.reason
|
||||
+ ". The inverter is held at 0 W. Find out why the "
|
||||
"command opposed the meter before starting control "
|
||||
"again; starting clears this."})
|
||||
"again; starting clears this. Restarting the "
|
||||
"add-on does NOT clear it."})
|
||||
else:
|
||||
held = self.sign_wd.held_s
|
||||
held = self.sign_wd.held_s(datetime.now(timezone.utc))
|
||||
out.append({"ok": True, "warn": False,
|
||||
"text": f"sign watchdog: armed (trips after "
|
||||
f"{self.sign_wd.cfg.hold_s:.0f} s opposed, both over "
|
||||
@@ -672,6 +717,8 @@ class Controller:
|
||||
# An operator pressing start has, at minimum, looked at the
|
||||
# banner that says why.
|
||||
self.sign_wd.reset()
|
||||
self.store.set("sign_tripped", False)
|
||||
self.store.set("sign_reason", "")
|
||||
self.log_event("sign-disagreement failsafe cleared by operator")
|
||||
if not self.auto:
|
||||
self.target = 0.0
|
||||
|
||||
Reference in New Issue
Block a user