Compare commits

..
Author SHA1 Message Date
glenn schrooyenandClaude Opus 5 ffb8ec6373 MAINT-04 review: pin the default divergence with a check, not a comment
validate.py's comment said "if config.yaml's value moves, move this and
main.py's together" and nothing enforced it - a comment where a check
belonged, in a PR that elsewhere proves it knows the difference.

test_validate.py now reads main.py and maintenance.py directly and asserts
the divergence EXACTLY AS IT STANDS: config.yaml/validate.py at 2000,
main.py's two fallbacks and MaintConfig's dataclass default at 2500. It
records the mismatch rather than demanding it be closed, so it passes today
and goes red if either side moves - including when somebody closes it in
main.py, at which point the check and validate.py's block are deleted
together. Nothing in a file another ticket owns is touched.

The consequence is asserted through the real code as well as the literals:
with the key absent the validator accepts, MaintConfig builds 2500 W against
a 2000 W ceiling, and the arbiter resolves it to 0 W as a contradiction -
the bug rule 6 exists to catch, passing rule 6. Production-unreachable (the
Supervisor always supplies every key) but fully reachable from sim/inject.py
and the tests, which is where anyone debugging maintenance works.

Also: a CHANGELOG section for the upgrade path, covering both the case where
the Supervisor re-derives the default and the case where it persisted the
old one, since which applies was not verifiable without probing the live
install. And maintenance.py's timeout comment said 13.5 kWh where everything
else says 13.8 (300 Ah x 51.2 V, DoD 90); SAFETY-18 does not touch that file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 02:51:40 +02:00
glenn schrooyenandClaude Opus 5 878a6ce452 MAINT-04: refuse a maintenance charge the ceiling cannot deliver
config.yaml shipped max_w 2000 against maintenance_charge_w 2500. The
cheap-window shaping files Claim.limit(hi = -maintenance_charge_w) against
the supervised SAFETY limit's lo = -max_w, so lo > hi - the arbiter's
clause 3, correctly resolved to 0 W and logged as a CONTRADICTION.
Maintenance silently never charged in a cheap window on a default install,
and the only evidence read like a bug in the arbiter rather than like the
two numbers the operator set.

validate_startup_config() now refuses maintenance_charge_w > max_w, the
same shape as CFG-01's sign_disagree_w >= max_w. Equality is reachable
(-max_w is a single valid point) and is not refused. Demonstrated by
starting the add-on on the pre-fix pair: exit 78 and the refusal banner.

maintenance_discharge_w was checked rather than assumed and is NOT the same
case: the drain phase OWNS the actuator (Claim.set at P_MAINTENANCE), so
the ceiling clamps it to +2000 W and the arbiter's explain() names the
clamp every cycle. Loud, and it still moves energy - warned, not refused.

The shipped maintenance rates drop to 2000 so the defaults pass the rule;
max_w was deliberately not raised, because it is the supervised safety
ceiling. test_validate.py now reads config.yaml itself, so a default the
validator refuses cannot ship again.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 02:28:09 +02:00
admin bb64f8fdd7 SAFETY-02 follow-up: persist the latch, and make the gate assertion bite (#9) 2026-08-28 23:09:27 +02:00
glenn schrooyenandClaude Opus 5 e40eb0768d SAFETY-02: the P_PEAK assertion was pinning the test's copy of the gate
Review finding 2, and it is the same duplication `disagrees()` just removed, so
it gets the same fix rather than a follow-up card.

`following_from()` in test_watchdog.py was a TRANSCRIPTION of the gate in
main.py, carrying "keep the two in step" as an instruction - which is exactly the
arrangement that does not keep two things in step. Measured: editing the shipped
gate alone to `winner in ("loop", "peak")` left every suite green while the
watchdog went blind for the duration of every peak event. The row whose whole job
is to force whoever lands peak shaving to come and change an expectation did not
do that.

The expression is now `watchdog.loop_is_following()`, called by main.py and by
the test. One definition, two callers, same reasoning as `disagrees()`: the copy
that drifts is always the one in the harness doing the grading.

⚠️ AND EXTRACTING IT WAS NOT SUFFICIENT - the drift proof still came out green
after the extraction, which is why it was worth running rather than assuming. The
realistic P_PEAK row has peak shaving asking for MORE than the loop, so the
magnitude term rejects it whatever `winner` says and masks the term under test. A
second row with peak asking for LESS makes `winner` the only deciding term. With
that row, widening the shipped gate fails two checks; without it, none.

Also enforcing the write ordering that was documented and not asserted: swapping
the two `store.set` calls passed everything. A store that fails the second write
now proves the property the ordering exists for - a crash mid-trip leaves a
reason with no latch, never a latch with no explanation. Swapping them now turns
that check red.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Du77usMj8XNKNFZGmUiWDa
2026-08-28 22:59:46 +02:00
glenn schrooyenandClaude Opus 5 cc4e88490b SAFETY-02 review: a latch that a restart clears is not a latch
BLOCKER. `auto` was persisted and `sign_wd.tripped` was not, so reconstructing
the Controller against the same store - which is what a container restart does -
brought it back ARMED, into the same fault, unattended: tripped=False, banner
back to NOT READY, and it resumed commanding -1000 then -2000 W before
re-tripping 180 s later. "Restart the add-on" is the first thing an operator
looking at a FAILSAFE banner and a 0 W inverter will try, and under a supervisor
that restarts a crashed add-on it is a loop of three-minute wrong-direction
bursts. DOCS.md's "it does not clear itself" was false across a restart.

`sign_tripped` and `sign_reason` are Store keys now, beside the maintenance
state and for the reason store.py's own docstring gives. Written the moment the
trip happens - reason first, flag second, so a crash between the two can never
persist a latch with no explanation - restored in Controller.__init__, cleared
only by the operator's start. The regression is covered: reverting the restore
fails seven checks in test_watchdog.py.

Review follow-ups, no behaviour change:

- The arming gate is tested against the REAL arbiter. It is the decision the
  ticket turns on and it was asserted in three comment blocks and verified by
  nothing - every case passed `following` in as a literal. Twelve claim
  configurations now derive it from resolve(), including the redirect case
  `winner == "loop"` alone would have got wrong and the `lo` limit that raises
  magnitude without flipping sign.
- The MQTT `status: "failsafe"` value is tested. The check that claimed to cover
  it restated the line above it and never touched the payload.
- The worked example described a config that does not ship: at the defaults
  maintenance_charge_w 2500 against max_w 2000 is an arbiter contradiction and
  0 W, not a -2500 W charge. Rewritten at 1500 W. The contradiction is a real
  pre-existing bug - a default cheap-window maintenance charge silently never
  happens - filed separately and deliberately not fixed here.
- Forward hazard recorded at the gate: `winner == "loop"` disarms the watchdog
  under any higher-priority SET, and a peak-shaving strategy at P_PEAK IS
  grid-following in sign, so it would blind this for every peak event.
- disagrees() is public so the replay harness stops carrying its own copy of the
  predicate it grades.
- held_s takes the clock instead of reading one.
- The two limits are now each stated in both places rather than one each.
- The no-false-positive argument is tied to control.py's freeze permitting the
  command to cross zero; under the shrink-only encoding control.py records as
  rejected, this would false-trip on a load step.
- Measured figures corrected: 93.4 % / 4212 of 17280, per-cluster
  0.3/0.9/10.5/0.0, and the near-circularity of those figures is now stated
  rather than leaned on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Du77usMj8XNKNFZGmUiWDa
2026-08-28 22:33:15 +02:00
admin 2513ba1f12 SAFETY-02: the sign-disagreement watchdog (#8) 2026-08-28 21:55:52 +02:00
glenn schrooyenandClaude Opus 5 9aed43990b SAFETY-02: a loop can be fresh, tracking and inside its rating and still push the wrong way
The stale-input watchdog asks whether the meter is still talking. The saturation
freeze asks whether the inverter can follow. The output clamp asks whether the
number is too big. A loop can pass all three while charging the battery out of
the grid, because the only thing wrong is the sign.

On 2026-08-20 a commercial controller did exactly that four times in one day,
for up to 77 minutes at a stretch, reaching a commanded -14 768 W against a
±5000 W inverter. app/watchdog.py compares the emitted setpoint against the
authoritative P1 figure and, when they oppose each other with both over 300 W
for 180 s continuously, files a failsafe SET claim at P_SAFETY, commands 0 W and
latches there until an operator starts control again.

Validated against that day rather than against an argument: sim/replay_sign.py
in the infra repo drives this module over the committed captures of both
2026-08-20 and the healthy 2026-08-23 and reaches 18 trips on the fault day -
every one inside a window NOTES.md §4.7 accounts for - and 0 on the healthy day.
The four episodes do share one signature: 0.3 / 0.9 / 10.5 / 0.0 % sign
agreement inside them against 92.9 % healthy.

Three decisions the ticket could not make:

- It is DISARMED whenever the grid-following loop is not what is driving. Sign
  agreement is a property of a loop that follows the meter, and a maintenance
  grid charge opposes the meter by design - 2500 W of charge into a house
  drawing 500 W is +3000 W on the meter against a -2500 W command, for hours.
  The gate tests the emitted number, not the winning claim's name, because a
  limit claim can let the loop "win" and still replace its value.
- The trip LATCHES. Commanding 0 W removes the command side of the comparison,
  so the disagreement always clears the moment the trip works.
- sign_disagree_w at or above max_w is refused at startup. The arbiter never
  lets a command exceed max_w, so such a threshold leaves the watchdog incapable
  of firing while the status page still reports it armed.

What it does not catch, said out loud: an inverter that ACTS opposite to its
command moves the meter further the same way, leaving command and meter in
agreement. That is the saturation freeze's job.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Du77usMj8XNKNFZGmUiWDa
2026-08-28 21:51:35 +02:00
admin 35856a73ae SAFETY-12: a loop that only thinks when the number changes cannot fail safe (#7) 2026-08-28 21:18:20 +02:00
11 changed files with 1642 additions and 18 deletions
+25 -3
View File
@@ -53,14 +53,36 @@ Run the checks before deploying. No framework needed:
```bash
cd goodwe_controller
python3 test_control.py && python3 test_maintenance.py && python3 test_arbiter.py \
&& python3 test_validate.py && python3 test_p1.py && python3 test_loop.py
&& python3 test_validate.py && python3 test_p1.py && python3 test_watchdog.py \
&& python3 test_loop.py
```
`test_watchdog.py` covers SAFETY-02's logic, its arming gate (derived from the
real arbiter, not passed in), its trip path, and the fact that the latch survives
an add-on restart. Its headline validation is not here at all: `sim/replay_sign.py`
in the infra repo drives the same module over the two captured RS485 days and must
fire on all four of the 2026-08-20 episodes and stay silent on 2026-08-23. That
needs both checkouts:
```bash
GOODWE_ADDON_REPO=/path/to/goodwe-addon python3 sim/replay_sign.py
GOODWE_ADDON_REPO=/path/to/goodwe-addon python3 sim/replay_sign.py --stub
```
The second run checks the grader still *discriminates* rather than merely still
passing — a watchdog that never trips and one that always trips must both be
rejected.
⚠️ **If you are testing the failsafe by hand, note that stopping and starting the
add-on will not clear a trip.** That is the point of it; use the *Start / stop
control* button twice, or delete `sign_tripped` from `/data/state.json`.
⚠️ `test_loop.py` takes about **95 seconds** (measured 93 s) and that is not a bug. It
drives the real control loop, which sleeps 1 s a tick, eight times over, to prove a
wall-clock property about it (SAFETY-12); mocking the clock there would leave the
assertion testing the mock. The other five suites together are ~395 checks and finish in
seconds. If you shorten `test_loop.py`'s windows, re-measure and correct this figure —
assertion testing the mock. The other six suites together are ~440 checks and finish in
seconds — `test_watchdog.py` spends ~3 s of that driving a real `Controller` through a
trip. If you shorten `test_loop.py`'s windows, re-measure and correct this figure —
an understated runtime is what makes a slow test look like a hung one, and a slow test
that looks hung gets deleted.
+198
View File
@@ -1,5 +1,203 @@
# Changelog
## Unreleased
**Cheap-window maintenance charging was unreachable at the shipped defaults, and
now the config that makes it unreachable is refused (MAINT-04).** `config.yaml`
shipped `max_w: 2000` against `maintenance_charge_w: 2500`. The cheap-window
shaping files `Claim.limit(hi = -maintenance_charge_w)` against the supervised
SAFETY limit's `lo = -max_w`, so `lo > hi` — the arbiter's clause 3, correctly
resolved to **0 W** and logged as a CONTRADICTION. **Maintenance silently never
charged in a cheap window on a default install**, and the only evidence read
like a bug in the arbiter rather than like the two numbers an operator set.
Measured against the real arbiter, loop asking +500 W:
```
max_w=2000 hi=-2500 -> +0 W contradiction=True
max_w=2500 hi=-2500 -> -2500 W limited by supervised(max_w)
max_w=3000 hi=-2500 -> -2500 W limited by maintenance(cheap-window charge)
```
`validate_startup_config()` now refuses `maintenance_charge_w > max_w` (rule 6),
the same shape as CFG-01's `sign_disagree_w >= max_w`: a number the operator set,
overridden by a layer they cannot see. Equality is reachable (`-max_w` is a
single valid point) and deliberately not refused. Verified by starting the
add-on on the pre-fix pair: exit 78, the refusal banner, no traceback.
**`maintenance_discharge_w` was checked, not assumed, and it is NOT the same
case.** The shaping is not symmetric. The drain phase does not shape the loop,
it OWNS the actuator — `Claim.set(P_MAINTENANCE, +discharge_w)` — and a SET is
*clamped* by the limits above it rather than contradicted by them. 2500 against
a 2000 W ceiling emits **+2000 W**, and the arbiter's own `explain()` prints
`maintenance -> 2000 W, limited by supervised(max_w)` every cycle. Nothing is
silent and energy still moves; the cycle completes slower (13.8 kWh usable at
2000 W is ~6.9 h against the 720 min drain timeout). **Warned about, not
refused.**
**The shipped defaults changed so the validator does not reject them.**
`maintenance_charge_w` and `maintenance_discharge_w` are now **2000**, at the
shipped `max_w`. `max_w` was deliberately NOT raised instead: it is the
supervised safety ceiling and raising a default upward ships more power to every
new install. Raising `max_w` (the reference install runs 5000) is what unlocks a
faster maintenance cycle — raise it first, then these. `test_validate.py` now
reads `config.yaml` itself and asserts the shipped numbers pass the rule, so a
default that the validator refuses cannot ship again.
### ⚠️ Upgrading an existing default install: the add-on may refuse to start
An install still carrying `maintenance_charge_w: 2500` against `max_w: 2000`
now meets rule 6 and **exits 78 with the refusal banner** instead of starting.
That is the intended behaviour — the configuration was already broken, it was
just silent about it — but an add-on that will not come back after an update
needs the fix stated here rather than only in the log.
**The fix, either way: set `maintenance_charge_w` to `2000` or lower, OR raise
`max_w` to `2500` or higher** (Settings → Add-ons → GoodWe RS485 Controller →
Configuration), then start it. The banner in the log says the same thing and
names both numbers.
Whether you hit this at all depends on whether the Supervisor kept the old
value, and **both cases are covered above because we did not verify which one
applies here.** The Supervisor merges an add-on's *stored* options over the
defaults in `config.yaml`, so:
- **A key the operator never edited** re-derives from the new `config.yaml` and
becomes 2000 on upgrade. Non-event, nothing to do.
- **A key the operator did set** persists at 2500 and the add-on refuses to
start until it is changed. Note this includes anyone who used the
Configuration page's YAML editor, which writes the whole options block back —
so "I never touched that field" is not a guarantee the old value is gone.
⚠️ **This is reasoned from the Supervisor's documented merge behaviour, not
measured on the live install.** The reference install's stored options could not
be read (`/api/hassio/` refuses a long-lived token) and probing it was out of
scope — it regulates a real battery. It runs `max_w: 5000` and is unaffected
either way.
⚠️ **Known, deliberately left, and worth a follow-up:** `main.py` and
`maintenance.py` still fall back to **2500** when the key is absent from the
options dict, while `config.yaml` and `validate.py` now say 2000. Unreachable on
a Supervisor install (every key is always supplied) and reachable only from
tests/`sim`. The real fix is one source of truth for option defaults, which is
bigger than this ticket and touches files another ticket is in.
## 0.5.1
**SAFETY-02's latch now survives an add-on restart, which is the difference
between a latch and a pause.** `auto` was persisted and `sign_wd.tripped` was
not, so reconstructing the `Controller` against the same store — exactly what a
container restart does — brought it back **armed, into the same fault, with no
operator involved**: `tripped=False`, banner back to `NOT READY`, and it resumed
commanding (measured: −1000 W, −2000 W) before re-tripping 180 s later. Under a
Supervisor that restarts a crashed add-on that is a loop of three-minute
wrong-direction bursts — and "restart the add-on" is the first thing an operator
looking at a `FAILSAFE` banner and a 0 W inverter will try.
`sign_tripped` and `sign_reason` are now `Store` keys beside the maintenance
state, written the moment the trip happens (reason first, flag second, so a crash
between the two can never persist a latch with no explanation) and restored in
`Controller.__init__`. Cleared only by the operator's start. `DOCS.md` said "it
does not clear itself"; that was false across a restart and now is not.
Also in this release, all of it review follow-up rather than new behaviour:
- **The arming gate is tested against the real arbiter.** It was the decision the
whole ticket turns on and it was asserted in three comment blocks and verified
by nothing — every case passed `following` in as a literal. `test_watchdog.py`
now derives it from `resolve()` across twelve claim configurations, including
the redirect case `winner == "loop"` alone would have got wrong and the `lo`
limit that raises magnitude without flipping sign.
- **The MQTT `status: "failsafe"` value is tested.** The check that claimed to
cover it restated the assertion three lines above it and never touched the
payload.
- **A forward hazard written down where it will be read, and made to bite.**
`winner == "loop"` disarms the watchdog under any higher-priority SET claim.
Today only maintenance and the safety stops emit one and neither follows the
meter; a peak-shaving strategy at `P_PEAK` **does**, and would blind the
watchdog for every peak event.
The gate is now `watchdog.loop_is_following()`, called by both `main.py` and
the test, for the same reason `disagrees()` was extracted: the test held a
*transcription* of the shipped expression, so editing the shipped gate alone to
`winner in ("loop", "peak")` left every suite green while the watchdog went
blind. Two copies, and the one that drifts is the one doing the grading.
⚠️ Extracting it was not sufficient on its own. The realistic P_PEAK row —
peak shaving asking for *more* than the loop — still could not detect the
widened gate, because the magnitude term rejects it regardless and masks the
winner term. A second row with peak asking for *less* isolates `winner` as the
only deciding term, and that is the one that turns red.
- **The worked example described a configuration that does not ship.** At the
defaults `maintenance_charge_w` (2500) exceeds `max_w` (2000), so that claim is
an arbiter *contradiction* resolving to 0 W, not a −2500 W charge. The example
is now written at 1500 W. (The contradiction is a real pre-existing bug —
a default cheap-window maintenance charge silently never happens — filed
separately, deliberately not fixed here.)
- **`disagrees()` is public**, so `sim/replay_sign.py` stops carrying its own copy
of the predicate it grades.
- `held_s` takes the clock instead of reading one, restoring the pure-function
property the module advertises.
- **The `sign_reason`-before-`sign_tripped` write ordering is enforced, not just
documented.** Swapping the two lines used to pass every check. A store that
fails the second write now proves the property the ordering exists for: what
survives a crash mid-trip is a reason with no latch, never a latch with no
explanation.
- Measured figures corrected throughout: 93.4 % / 4212 of 17280 (was 92.9 % /
4127 of 16349), per-cluster 0.3 / 0.9 / 10.5 / 0.0 %.
## 0.5.0
**SAFETY-02 — the sign-disagreement watchdog.** The add-on now watches whether
the setpoint it is emitting and the grid figure it is regulating on point the
same way. If they oppose each other, both over `sign_disagree_w` (300 W), for
`sign_disagree_s` (180 s) **continuously**, it files a failsafe claim at
`P_SAFETY`, commands **0 W**, and latches there until an operator starts control
again. New module `app/watchdog.py`; new options `sign_disagree_w` and
`sign_disagree_s`.
This is the gap the existing failsafes leave open. The stale-input watchdog asks
whether the meter is still talking, the saturation freeze asks whether the
inverter can follow, the output clamp asks whether the number is too big — a
loop can pass all three while charging the battery out of the grid, because the
only thing wrong is the sign. On 2026-08-20 a commercial controller did exactly
that four times in one day, for up to 77 minutes at a stretch, and reached a
commanded −14 768 W against a ±5000 W inverter.
Validated against that day. `sim/replay_sign.py` in the infra repo drives this
module over the committed RS485 captures of both 2026-08-20 (the four episodes)
and 2026-08-23 (healthy): **18 trips on the fault day, every one inside a window
`NOTES.md` §4.7 accounts for, and 0 on the healthy day.** The four episodes do
share one signature rather than merely resembling each other — sign agreement
runs 0.3 / 0.9 / 10.5 / 0.0 % inside them against 93.4 % on the healthy day.
Three decisions worth knowing about:
- **It is disarmed whenever the grid-following loop is not what is driving.**
Sign agreement is a property of a loop that follows the meter. A maintenance
grid charge opposes the meter *by design* — 2500 W of charge into a house
drawing 500 W is +3000 W on the meter against a −2500 W command, for hours —
and armed unconditionally this would trip on every maintenance cycle. The gate
tests the emitted number, not the winning claim's name: a limit claim can let
the loop "win" and still replace its value with one pointing the other way.
- **The trip latches.** Commanding 0 W removes the command side of the
comparison, so the disagreement always clears the moment the trip works.
Un-latched, this would trip, clear, and hand the actuator back to the loop that
was driving the wrong way, every three minutes forever.
- **`sign_disagree_w` at or above `max_w` is refused at startup** (validator rule
5). The arbiter never lets a command exceed `max_w`, so such a threshold makes
the watchdog incapable of firing while the status page still reports it armed.
A disarmed guard that advertises itself as armed is the failure mode this
project keeps refusing to ship.
What it does **not** catch, stated so nobody assumes otherwise: an inverter that
acts opposite to what it was told moves the meter *further* the same way, so
command and meter agree while everything runs away. That is the saturation
freeze's job, which compares the command against the battery's actual power.
`test_validate.py`'s "the smallest sane max_w still starts" case now names a
reachable sign threshold, because a 1 W ceiling genuinely does disarm the 300 W
default and is now correctly refused.
## 0.4.0
**CFG-01.** The add-on now refuses to start on a configuration it cannot honour,
+62 -2
View File
@@ -24,6 +24,9 @@ The inverter **holds its last command forever** — it has no meter-timeout. So:
commanding it.
- This add-on commands 0 W when its inputs go missing, when you stop control,
and when it shuts down.
- This add-on commands 0 W **and latches there** if what it is commanding and
what the meter reads point opposite ways for three minutes — see "The
sign-disagreement failsafe" below.
- The **optional RS485 e-stop** is the only thing that covers this machine
dying. Without it, a failed host leaves the battery latched at its last
command until someone intervenes.
@@ -278,6 +281,63 @@ behind it would trip the firmware watchdog on a system that is working fine.
| `cycle_floor_s` | 15 | The longest the loop may go without evaluating. A deadline on the gap between cycles, reset by every cycle — on a healthy meter it never elapses and changes nothing. Raise it only if your meter is slower than ~10 s; it must stay under `meter_max_age_s`, past which there is no reading to evaluate anyway |
| `stale_input_s` | 15 | How long inputs may be missing before commanding 0 W. In P1 mode this clock starts only *after* `meter_max_age_s` has already expired — the two stack, see "How long a dead meter takes to reach 0 W" |
| `auto_start` | false | Start controlling on boot (only after commissioning) |
| `sign_disagree_w` | 300 | Both the command and the meter must exceed this before a sign difference counts. Must stay **below `max_w`** or the watchdog can never fire, and the add-on refuses to start |
| `sign_disagree_s` | 180 | How long they must stay opposed, continuously, before the failsafe trips |
#### The sign-disagreement failsafe
If the setpoint this add-on is emitting and the grid figure it is regulating on
point **opposite ways** — charging the battery while the house imports, or
discharging it while the house exports — with both over `sign_disagree_w`, for
`sign_disagree_s` **continuously**, the add-on commands **0 W** and holds it
there. The banner reads `FAILSAFE`, the commissioning list says which two
numbers disagreed and for how long, and the `Controller status` entity reports
`failsafe`.
**It does not clear itself, and restarting the add-on does not clear it either.**
Pressing *Start / stop control* twice — stop, then start — is the only thing that
does. That is deliberate on both counts:
- Commanding 0 W removes the command side of the comparison, so the disagreement
always vanishes the instant the trip works. A watchdog that took that as proof
the fault was over would hand the inverter straight back to the loop that was
driving the wrong way, three minutes later, forever.
- The trip is written to `/data/state.json` and restored on startup. Restarting
the add-on is the first thing most people try when they see a red banner, and
a latch that a restart clears would bring the controller back **armed, into the
same fault, unattended** — and under a Supervisor that restarts a crashed add-on
automatically, would do so in a loop. After a restart the banner, the 0 W and
the recorded reason are all still there.
**What it is for.** This is a loop-integrity check: it asks whether the loop's
output direction still matches the input state it was regulating. It catches a
polarity error anywhere between the meter and the wire, and an integrator so far
wound up that unwinding takes minutes. It is the check that would have caught all
four episodes of 2026-08-20, when a commercial controller charged the battery out
of the grid, four times, for up to 77 minutes at a time.
**What it is not for.** It does not detect an inverter that acts opposite to what
it was told — that moves the meter *further* the same way, so command and meter
agree while everything runs away. That is the saturation freeze's job
(`saturation_w` / `saturation_cycles`), which compares the command against the
battery's actual power. The two checks are complementary and neither replaces the
other.
**Why it will not false-trip on ordinary operation.** A battery that refuses a
command (empty, full, standby) leaves the meter where it was, so command and
meter agree. A frozen meter drives the loop into agreeing with it. Around the
resting point both signals are inside the 300 W gate, so the deadband can never
produce one. A load step does flip the meter's sign before the loop catches up —
measured on a full captured day, the longest such crossing is 30 s, against a
180 s window. And the watchdog is **disarmed whenever the grid-following loop is
not what is driving**: a maintenance grid charge opposes the meter by design and
must not be mistaken for a fault.
Both numbers are measured rather than chosen, and are exposed only because the
ticket requires defaults rather than constants. On the captured healthy day sign
agreement is 93.4 %; on the four captured fault clusters it is 0.3 %, 0.9 %,
10.5 % and 0.0 %. There is no reason to move either value without a capture that
says so.
#### Saturation is counted in cycles, not seconds
@@ -354,8 +414,8 @@ help - see "Why the tuning is what it is".
| `maintenance_enabled` | false | Enable the monthly cycle |
| `maintenance_interval_days` | 28 | Minimum gap between cycles |
| `maintenance_start_hour` | 10 | Hour of day a due cycle begins |
| `maintenance_discharge_w` | 2500 | Drain rate (exports the surplus) |
| `maintenance_charge_w` | 2500 | Charge ceiling, capped again by peak headroom |
| `maintenance_discharge_w` | 2000 | Drain rate (exports the surplus). Above `max_w` it is **clamped** to `max_w` — the drain phase SETs the setpoint and the arbiter bounds it, saying so on the status page (`limited by supervised(max_w)`). Warned about, not refused: the cycle still completes, just slower |
| `maintenance_charge_w` | 2000 | Charge ceiling, capped again by peak headroom. Must stay **at or below `max_w`** or the cheap-window shaping (`hi = -maintenance_charge_w`) contradicts the supervised limit (`lo = -max_w`), the arbiter resolves it to **0 W**, and maintenance silently never charges in a cheap window — **the add-on refuses to start** |
| `maintenance_soc_floor` | 11 | Drain target — stay just above the inverter's own floor |
| `maintenance_soc_target` | 99 | Charge target |
| `maintenance_hold_min` | 120 | Hold at full so the BMS can balance |
+125 -3
View File
@@ -41,6 +41,7 @@ from .maintenance import IDLE, MaintConfig, Maintenance
from .mqtt import MqttPublisher
from .p1 import P1Ingest, build_source, is_enabled
from .validate import ConfigRefused, refuse, validate_startup_config
from .watchdog import SignWatchdog, SignWatchdogConfig, loop_is_following
from . import web
OPTIONS_PATH = "/data/options.json"
@@ -103,6 +104,29 @@ class Controller:
store,
)
# SAFETY-02. One instance, one actuator, one cadence - see watchdog.py.
#
# ⚠️ RESTORED FROM THE STORE, and that is the difference between a latch
# and a pause. A trip held only in memory is cleared by the one action an
# operator staring at a FAILSAFE banner is most likely to take - restart
# the add-on - and the controller then comes back armed, into the fault,
# with nobody involved. Restored here, the banner and the 0 W survive the
# restart and only `auto_toggle` clears them.
self.sign_wd = SignWatchdog(
SignWatchdogConfig(
min_w=float(opts.get("sign_disagree_w", 300)),
hold_s=float(opts.get("sign_disagree_s", 180)),
),
tripped=bool(store.data.get("sign_tripped", False)),
reason=str(store.data.get("sign_reason", "") or ""),
)
if self.sign_wd.tripped:
_LOG.error("SAFETY-02 was TRIPPED before this restart and stays "
"tripped: %s. The inverter is held at 0 W. Restarting the "
"add-on does not clear this - start control from the UI "
"once you know why the command opposed the meter.",
self.sign_wd.reason or "(reason not recorded)")
# P1 ingestion (TEL-01). `meter_source: off` keeps the original
# single-entity meter_entity path, so an existing install is unchanged
# until it opts in.
@@ -120,6 +144,11 @@ class Controller:
self.sat_count = 0
self.i_w = 0.0 # the loop's integrator, carried between cycles
self.loop_w = None # what the loop asked for last cycle, or None
# Did the grid-following loop actually win the LAST arbitration? That is
# the question SAFETY-02 has to ask about the command standing on the
# wire right now, so it is recorded when the arbitration happens rather
# than re-derived a cycle later. Starts False: nothing is on the wire.
self.following = False
self.reason = "starting"
self.grid = self.soc = self.batt = None
self.peak_fc = None
@@ -214,8 +243,38 @@ class Controller:
for msg in result.events:
self.log_event(msg)
# --- SAFETY-02, and it runs BEFORE the early return below -------------
# ⚠️ ORDER MATTERS HERE. If this were evaluated after the "inputs missing
# but not yet stale" hold, a run of held cycles would leave
# `disagree_since` untouched across the gap and the timer would resume
# from a start time that predates it. It is evaluated every cycle,
# unconditionally, so a missing meter CLEARS the window - see watchdog.py.
#
# The value compared is the command STANDING ON THE WIRE, which is what
# this cycle's fresh meter reading is a response to. `last_write` is that
# value after the device clamp and confirmed accepted; `target` covers
# the case where nothing has been written yet (and, deliberately, the
# case where the last write was REJECTED - the inverter is then still
# latched on `last_write`, which is exactly what `last_write` holds).
was_tripped = self.sign_wd.tripped
standing = self.last_write if self.last_write is not None else self.target
if self.sign_wd.evaluate(standing, self.grid, self.following, now) \
and not was_tripped:
# ⚠️ REASON FIRST, FLAG SECOND, and each `set` writes the file. The
# ordering is the whole atomicity story: a crash between the two
# persists a reason nothing reads, which is harmless, where the other
# order persists a latch with no explanation for the operator who has
# to decide whether to clear it.
self.store.set("sign_reason", self.sign_wd.reason)
self.store.set("sign_tripped", True)
self.log_event("FAILSAFE " + self.sign_wd.reason)
_LOG.error("SAFETY-02 TRIPPED - %s. Commanding 0 W and holding it "
"until control is started again. This survives an add-on "
"restart; restarting will not clear it.",
self.sign_wd.reason)
stale_after = float(self.o.get("stale_input_s", 15))
if not self.inputs_ok:
if not self.inputs_ok and not self.sign_wd.tripped:
bad_for = (now - self.inputs_bad_since).total_seconds() if self.inputs_bad_since else 0.0
if bad_for < stale_after:
# A single missed poll is not a fault. Hold, keep the heartbeat
@@ -235,6 +294,16 @@ class Controller:
claims.append(Claim.limit("supervised", P_SAFETY, -max_w, max_w, "max_w"))
# --- safety stops ----------------------------------------------------
# ⚠️ SAFETY-02 is claimed FIRST and unconditionally while latched. It is
# a SET at P_SAFETY, so it outranks maintenance, peak shaving and the
# loop alike - a controller pushing power the wrong way is wrong
# whatever it thinks it is doing. Re-filed EVERY cycle: claims are not
# sticky, and one filed once would lapse the next cycle and hand the
# actuator silently back to the loop that was driving the wrong way.
if self.sign_wd.tripped:
claims.append(Claim.set("failsafe", P_SAFETY, 0.0,
"sign disagreement"))
if not self.auto:
claims.append(Claim.set("safety", P_SAFETY, 0.0, "stopped"))
elif not self.inputs_ok:
@@ -306,6 +375,20 @@ class Controller:
resolution = resolve(claims)
if resolution.contradiction:
self.log_event("ARBITER CONTRADICTION - commanding 0 W, see the log")
# --- SAFETY-02's arming gate ------------------------------------------
# Was the number about to go on the wire chosen by the grid-following
# loop, or by a strategy that commands against the meter on purpose?
# Only the former is expected to agree in sign with P1 - without this,
# every maintenance grid charge trips the failsafe within three minutes.
#
# ⚠️ THE EXPRESSION LIVES IN watchdog.py, NOT HERE, and that is not
# tidiness. It was inline, test_watchdog.py held a transcription of it,
# and the P_PEAK case meant to force whoever lands peak shaving to come
# and change an expectation did not: editing this line alone to
# `winner in ("loop", "peak")` left every suite green while the watchdog
# went blind for every peak event. Read loop_is_following() before
# touching the gate - the forward hazard and the reasoning are there.
self.following = loop_is_following(resolution, self.loop_w)
self.target = resolution.target_w
self.reason = resolution.explain()
await self.write_setpoint(self.target)
@@ -439,7 +522,14 @@ class Controller:
"battery": self.batt,
"soc": self.soc,
"phase": self.maint.phase,
"status": "running" if self.auto else "stopped",
# ⚠️ SAFETY-02's alarm surface, and it is deliberately an existing
# one. IF-01/IF-02 own `ems/event/failsafe` and the `failsafe active`
# binary sensor and neither is built yet; inventing a bespoke topic
# here would be a surface to migrate later. This sensor is already
# discovered, already on every dashboard, and "failsafe" is
# additive to its vocabulary rather than a change to it.
"status": ("failsafe" if self.sign_wd.tripped
else "running" if self.auto else "stopped"),
}
# ⚠️ ONLY when P1 ingestion is actually running. The ESP32's stale-input
# watchdog subscribes to sensor.goodwe_p1_sample_age - see the entity-id
@@ -521,6 +611,23 @@ class Controller:
out.append({"ok": False, "warn": False,
"text": "last write to the inverter was REJECTED - check the log"})
# SAFETY-02. Reported in both states: a watchdog nobody can see is armed
# is a watchdog nobody trusts when it fires.
if self.sign_wd.tripped:
out.append({"ok": False, "warn": False,
"text": "SIGN-DISAGREEMENT FAILSAFE - " + self.sign_wd.reason
+ ". The inverter is held at 0 W. Find out why the "
"command opposed the meter before starting control "
"again; starting clears this. Restarting the "
"add-on does NOT clear it."})
else:
held = self.sign_wd.held_s(datetime.now(timezone.utc))
out.append({"ok": True, "warn": False,
"text": f"sign watchdog: armed (trips after "
f"{self.sign_wd.cfg.hold_s:.0f} s opposed, both over "
f"{self.sign_wd.cfg.min_w:.0f} W)"
+ (f" - opposed for {held:.0f} s now" if held else "")})
if not o.get("estop_fitted", False):
out.append({"ok": False, "warn": True,
"text": "no e-stop fitted: if this host dies the battery stays "
@@ -538,7 +645,11 @@ class Controller:
def status_payload(self) -> dict:
checks = self.checks()
bad = [c for c in checks if not c["ok"] and not c["warn"]]
if bad:
if self.sign_wd.tripped:
# Outranks "NOT READY": nothing is missing, something went wrong.
level, banner = "bad", ("FAILSAFE — the command opposed the meter. "
"Inverter held at 0 W.")
elif bad:
level, banner = "bad", f"NOT READY — {bad[0]['text']}"
elif not self.auto:
level, banner = "warn", "Stopped — inverter commanded to 0 W"
@@ -572,6 +683,17 @@ class Controller:
self.auto = not self.auto
self.store.set("auto", self.auto)
self.log_event("control started" if self.auto else "control stopped")
if self.auto and self.sign_wd.tripped:
# ⚠️ THE ONLY WAY OUT OF A SIGN-DISAGREEMENT TRIP, on purpose.
# It clears on a deliberate human start, not on the condition
# going away - the condition ALWAYS goes away, because the trip
# commands 0 W and a 0 W command cannot disagree with anything.
# An operator pressing start has, at minimum, looked at the
# banner that says why.
self.sign_wd.reset()
self.store.set("sign_tripped", False)
self.store.set("sign_reason", "")
self.log_event("sign-disagreement failsafe cleared by operator")
if not self.auto:
self.target = 0.0
await self.write_setpoint(0.0, force=True)
+4 -3
View File
@@ -28,9 +28,10 @@ _LOG = logging.getLogger("goodwe.maint")
IDLE, DRAIN, CHARGE, HOLD = "idle", "drain", "charge", "hold"
# A drain of a 13.5 kWh usable pack at 2500 W is ~5 h even with no house load
# helping. 12 h means the battery is not following commands - a fault, not a
# slow day. The charge phase deliberately waits for sun and for cheap hours, so
# A drain of a 13.8 kWh usable pack (300 Ah x 51.2 V, DoD 90) at the shipped
# 2000 W is ~6.9 h even with no house load helping - and less in practice, since
# house load drains alongside the export and soc_floor stops short of empty.
# 12 h means the battery is not following commands - a fault, not a slow day. The charge phase deliberately waits for sun and for cheap hours, so
# it is expected to span a night; 36 h still bounds it.
DRAIN_TIMEOUT_MIN = 720
CHARGE_TIMEOUT_MIN = 2160
+23 -2
View File
@@ -22,11 +22,30 @@ _LOG = logging.getLogger("goodwe.store")
# `auto_start` option only when the key is absent - if a default supplied False,
# the fallback could never fire and auto_start would silently do nothing on
# every fresh install.
#
# ⚠️ `sign_tripped` IS in here, and the difference from `auto` is the whole
# reason both decisions are written down. `auto` has an option to fall back to
# (`auto_start`), so a stored default would mask a fresh install. A latched
# failsafe has no such fallback: absent means "never tripped", which is exactly
# what a default of False says.
#
# It is persisted for the reason this module's docstring gives for maintenance
# state, applied to something with a much shorter fuse. SAFETY-02's latch is the
# only thing standing between a controller that has been caught driving the wrong
# way and its own restart, and "restart the add-on" is the FIRST thing an operator
# looking at a FAILSAFE banner will try. Held only in memory it came back armed,
# into the fault, unattended - measured: after reconstructing Controller against
# the same store, `tripped` was False, the banner read NOT READY, and it resumed
# commanding -1000, -2000 W before re-tripping 180 s later. Under a supervisor
# that restarts a crashed add-on that is a loop, three minutes of wrong-direction
# power at a time, forever.
DEFAULTS = {
"phase": "idle",
"phase_started": None,
"last_completed": None,
"last_start_attempt": None,
"sign_tripped": False,
"sign_reason": "",
}
@@ -42,8 +61,10 @@ class Store:
loaded = json.load(fh)
if isinstance(loaded, dict):
self.data = {**DEFAULTS, **loaded}
_LOG.info("state restored: phase=%s last_completed=%s",
self.data.get("phase"), self.data.get("last_completed"))
_LOG.info("state restored: phase=%s last_completed=%s "
"sign_tripped=%s",
self.data.get("phase"), self.data.get("last_completed"),
self.data.get("sign_tripped"))
except FileNotFoundError:
_LOG.info("no saved state, starting fresh")
except (json.JSONDecodeError, OSError) as err:
+130
View File
@@ -74,6 +74,48 @@ NOT READY banner already names the problem, (b) does not apply.
was overruled. That firmware file's own comment says a lower bound on this
"belongs in CFG-01, not here" - so here it is.
5. `sign_disagree_w` at or above `max_w`. [(a)]
The arbiter's `supervised` limit bounds every command to +-max_w, and
SAFETY-02's predicate needs `abs(command) > sign_disagree_w`. Set the
threshold at or above the ceiling and the command side of the comparison
is unsatisfiable BY CONSTRUCTION: the watchdog can never fire, on any
input, ever. Same shape as 1 and 4 - the operator set a number, a layer
they cannot see makes it mean something else, and nothing says so. Worse
here, because what the status page reports in that state is "sign
watchdog: armed". A disarmed guard that advertises itself as armed is the
one thing this project has decided repeatedly it will not ship.
6. `maintenance_charge_w` above `max_w`. [(a)]
The same shape as 5, and found the same way - by checking a worked example
against what actually ships. maintenance's cheap-window shaping files
`Claim.limit(hi=-maintenance_charge_w)` at P_MAINT_SHAPE, against the
`supervised` SAFETY limit's `lo=-max_w`. Both bind (P_SAFETY and
P_MAINT_SHAPE both outrank the loop that wins the SET), so a charge rate
above the ceiling gives `lo > hi` - the arbiter's clause 3, which it
correctly resolves to 0 W and logs as a CONTRADICTION. Measured against
the arbiter, loop asking +500 W:
max_w=2000 hi=-2500 -> +0 W contradiction=True
max_w=2500 hi=-2500 -> -2500 W limited by supervised(max_w)
max_w=3000 hi=-2500 -> -2500 W limited by maintenance(...)
So MAINTENANCE SILENTLY NEVER CHARGES IN A CHEAP WINDOW until max_w is at
or above the charge rate, and the only evidence is a contradiction line
that reads like a bug in the arbiter rather than like the two numbers the
operator set. Equality is fine and deliberately not refused - at
max_w == charge_w the window is a single reachable point, -max_w.
⚠️ `maintenance_discharge_w` IS NOT THE SAME CASE, and this was checked
rather than assumed. THE SHAPING IS NOT SYMMETRIC: the drain phase does
not shape the loop, it OWNS the actuator - `Claim.set(P_MAINTENANCE,
+discharge_w)`. A SET is clamped by the limits above it, not contradicted
by them, so 2500 against a 2000 W ceiling emits +2000 W and the arbiter's
own explain() says `maintenance -> 2000 W, limited by supervised(max_w)`
on the status page every cycle. That is neither (a) - nothing is silent -
nor (b) - it moves energy and the cycle still completes, just slower
(13.8 kWh usable at 2000 W is ~6.9 h against the 720 min DRAIN_TIMEOUT).
It is warned about below, not refused.
WHAT IS DELIBERATELY *NOT* REFUSED, because it is untidy rather than unsafe:
- `integrator_max_w` above `max_w`. DOCS.md is right that the only thing the
@@ -91,6 +133,9 @@ WHAT IS DELIBERATELY *NOT* REFUSED, because it is untidy rather than unsafe:
NOT READY banner.
- `meter_poll_s` above half `meter_max_age_s`. One missed poll costs a stale
reading and a 0 W cycle. Warned about in p1.py, not refused.
- `maintenance_discharge_w` above `max_w`. Clamped, not contradicted, and the
clamp names itself in the arbiter's explain() - see rule 6. Warned about,
not refused.
"""
import logging
@@ -238,6 +283,72 @@ def validate_startup_config(opts: dict, dev_min=None, dev_max=None) -> list[str]
"or rebuild the firmware with a matching max_age_s and update "
"validate.FIRMWARE_INGEST_VETO_S.")
# --- 5. a sign watchdog that can actually fire ---------------------------
# ⚠️ Strictly `>=`, matching the predicate exactly: watchdog.py tests
# `abs(commanded_w) > min_w`, and the arbiter holds `abs(commanded_w)` at or
# below max_w, so threshold == max_w is already unsatisfiable. Anything
# BELOW max_w is reachable and therefore fine - this refuses the disarmed
# case only, not "close to the rail".
sign_w = _number(opts, "sign_disagree_w", 300.0)
if sign_w is None:
problems.append(f"sign_disagree_w {opts.get('sign_disagree_w')!r} is not a number")
elif max_w is not None and max_w > 0 and sign_w >= max_w:
problems.append(
f"sign_disagree_w is {sign_w:.0f} W but max_w is {max_w:.0f} W, so "
"the SAFETY-02 sign-disagreement watchdog can never fire: the "
"arbiter never lets a command exceed max_w, and the watchdog needs "
f"a command over {sign_w:.0f} W. It would sit there reporting "
"'armed' on the status page while being incapable of tripping - the "
f"one runaway failure mode this system has actually observed. Lower "
f"sign_disagree_w below {max_w:.0f} W (300 W is the measured "
"default), or raise max_w.")
# --- 6. a maintenance charge the ceiling can actually accommodate --------
# ⚠️ Strictly `>`, matching the arbiter exactly. The shaping is
# `Claim.limit(hi=-charge_w)` against `supervised`'s `lo=-max_w`, so
# charge_w == max_w leaves the single reachable point -max_w and works;
# only ABOVE is `lo > hi`, arbiter clause 3, 0 W and a contradiction.
#
# Not scoped to maintenance_enabled, deliberately. The pair is
# contradictory whether or not the schedule is on, and scoping it would
# leave the trap armed for the one operator action - ticking
# maintenance_enabled - that cannot possibly be expected to change what a
# charge rate means.
# ⚠️ THE FALLBACK IS config.yaml's SHIPPED VALUE, and it has to be kept
# equal to it - that is the whole point of AC3. It is reachable only when a
# caller hands over a dict with the key missing (sim/inject.py, a test); the
# Supervisor always supplies every key from config.yaml, so on a real
# install the argument is always present.
#
# ⚠️ KNOWN MISMATCH, LEFT DELIBERATELY AND FLAGGED RATHER THAN FIXED HERE:
# `main.py` still falls back to 2500 in two places
# (`opts.get("maintenance_charge_w", 2500)`), as does maintenance.py's
# MaintConfig dataclass. So a dict with the key ABSENT is accepted by this
# rule at 2000 and then built by main.py at 2500 - the contradiction this
# rule exists to catch, in the one case it cannot see. It is unreachable on
# a Supervisor install and those files are another ticket's right now.
# THE REAL FIX is one source of truth for option defaults; until then the
# divergence is PINNED BY A CHECK rather than by this comment - see
# test_validate.py, "the KNOWN cross-file default divergence". It asserts
# the mismatch as it stands and goes red if EITHER side moves, including
# when somebody closes it in main.py, at which point this block and that
# check both get deleted.
charge_w = _number(opts, "maintenance_charge_w", 2000.0)
if charge_w is None:
problems.append(
f"maintenance_charge_w {opts.get('maintenance_charge_w')!r} is not a number")
elif max_w is not None and max_w > 0 and charge_w > max_w:
problems.append(
f"maintenance_charge_w is {charge_w:.0f} W but max_w is "
f"{max_w:.0f} W, so a maintenance charge in a cheap window can "
"never happen: the shaping asks the arbiter for at most "
f"{-charge_w:.0f} W while the supervised limit allows no less than "
f"{-max_w:.0f} W, which is a contradiction it resolves to 0 W - "
"THE CEILING WINS, and what you would see is an ARBITER "
"CONTRADICTION line that reads like a bug rather than like these "
f"two numbers. Lower maintenance_charge_w to {max_w:.0f} W or less, "
f"or raise max_w to at least {charge_w:.0f} W.")
# --- warnings: real, but not worth refusing over -------------------------
integ = opts.get("integrator_max_w")
integ_w = None if not integ else _number(opts, "integrator_max_w", None)
@@ -249,6 +360,25 @@ def validate_startup_config(opts: dict, dev_min=None, dev_max=None) -> list[str]
"saturation. 0 (follow max_w) is the recommended value.",
integ_w, max_w)
# ⚠️ THE SIBLING OF RULE 6, AND NOT THE SAME CASE - see the docstring. The
# drain phase OWNS the actuator (`Claim.set` at P_MAINTENANCE), and a SET is
# clamped by the limits above it rather than contradicted by them. So this
# drains at max_w instead of the configured rate, the arbiter says so in
# words on the status page every cycle, and the cycle still completes -
# slower, and eating into maintenance.DRAIN_TIMEOUT_MIN. Loud and it moves
# energy, so it is neither (a) nor (b): warned, not refused.
disch_w = _number(opts, "maintenance_discharge_w", 2000.0)
if disch_w is not None and max_w is not None and max_w > 0 and disch_w > max_w:
_LOG.warning(
"maintenance_discharge_w %.0f W is above max_w %.0f W. Not refused - "
"the drain phase SETS the setpoint and the arbiter clamps it to "
"%.0f W and names the clamp ('limited by supervised(max_w)'), so "
"nothing is silent and the cycle still completes. But it drains at "
"%.0f W, not the %.0f W you configured, and the drain phase has a "
"720 min timeout. Lower it to %.0f W or raise max_w to remove the "
"surprise.",
disch_w, max_w, max_w, max_w, disch_w, max_w)
return problems
+354
View File
@@ -0,0 +1,354 @@
"""SAFETY-02 - the sign-disagreement watchdog.
WHAT IT CATCHES, in one sentence: a grid-following loop that is pushing power in
the wrong direction, which every other guard in this system is blind to.
The stale-input watchdog (SAFETY-01) asks "is the meter still talking?". The
saturation freeze asks "can the inverter follow?". The output clamp asks "is the
number too big?". A loop can pass all three and still be charging the battery
out of the grid while the house imports - the meter is fresh, the inverter is
tracking beautifully, and the magnitude is inside the rating. The only thing
that separates that from healthy operation is the SIGN.
⚠️ THIS IS NOT A HYPOTHETICAL. On 2026-08-20 the vendor controller (Illusmart)
did exactly this four times in one day (NOTES.md §4.7). Both captures of that
bus are committed - `sim/scenarios/bus-2026-08-20.log.gz` and the healthy
`bus-2026-08-23.log.gz` in the infra repo - and `sim/replay_sign.py` drives this
module over both of them. Every number in this file was measured on those two
days, not chosen:
day samples gated in sign agreement
2026-08-23 4 212 / 17 280 93.4 % (healthy)
2026-08-20 7 182 / 17 280 56.4 % (the whole fault day)
cluster A 15:26-16:43 0.3 %
cluster B 17:11-18:28 0.9 %
cluster C 20:01-20:31 10.5 %
cluster D 20:55-21:11 0.0 %
⚠️ THOSE AGREEMENT FIGURES ARE THE WEAKEST EVIDENCE ON THIS PAGE, and an earlier
version of this comment leaned on them as though they were the strongest. The
clusters are found by looking for sustained opposed samples, so measuring "how
opposed is this cluster" over them is close to a tautology - and "the healthy day
contains no such cluster" is the same sentence as "the healthy day produces no
trip", not independent corroboration of it.
What actually carries the argument is two things neither the detector nor the
clustering assumes:
1. The RUN-LENGTH DISTRIBUTION. The longest unbroken opposed run on the healthy
day is 30 s. On the fault day there are 18 runs over 180 s and the longest is
1750 s. Nothing about that gap was chosen - it is what the two days measure.
2. The clusters land where a human independently wrote down onsets. NOTES.md
§4.7 records 15:01, 16:56, 19:57 and 20:23 from watching it happen, before
any of this existed, and cluster D coincides with the separately documented
inverter-off runaway of §4.7 item 2.
Be precise about (2), because the convenient overstatement is available and wrong:
the clustering does NOT reproduce four episodes. It yields THREE clusters over the
four documented onsets - C spans both 19:57 and 20:23 - plus a fourth for §4.7
item 2, which is not one of the four. Every trip does fall inside a documented
window, which is the claim worth making.
------------------------------------------------------------------------------
WHY IT LIVES HERE AND NOT IN THE FIRMWARE OR THE WITNESS
Three vantage points see three different things. The firmware knows what it
WROTE, the witness Pi knows what was ON THE WIRE and whether the inverter ACKed
it, and the add-on knows what it INTENDED and holds the authoritative meter.
This check is not command-versus-wire; it is command-versus-physics. It needs
the emitted setpoint and the P1 net figure in the same place, and it needs the
authority to command zero when they fight:
- the witness is read-only by charter and cannot command anything, so it could
raise the alarm but never take the trip action the AC requires;
- the firmware has the wire but not the meter. §5.1 makes P1 the only
authoritative grid figure and the board only ever receives its AGE
(`sensor.goodwe_p1_sample_age`), never its value. Putting the check there
would mean shipping the meter to the ESP32 for no other reason, and it would
be C++ per install rather than one generic guard;
- the add-on has both halves already, plus an arbiter that can outrank every
strategy with a P_SAFETY claim.
The cost of standing here is real and worth stating: this watchdog compares what
we MEANT to put on the wire against the meter. If the firmware or the driver
inverts the sign on the way out, the add-on's own view still agrees with itself
and this check stays silent. That gap is a wire-versus-intent problem and it
belongs to the witness, not here.
------------------------------------------------------------------------------
WHAT "DISAGREEMENT" MEANS, PRECISELY
disagree = following
and commanded_w * grid_w < 0
and abs(commanded_w) > min_w
and abs(grid_w) > min_w
Four gates, and each one exists to exclude a specific thing that is NOT a fault:
1. `following`. ⚠️ THE GATE THAT IS EASIEST TO FORGET AND MOST EXPENSIVE TO
OMIT. Sign agreement is a property of a GRID-FOLLOWING loop. It is not
expected of a strategy that deliberately commands against the meter -
maintenance is exactly that. A maintenance grid charge of 1500 W into a house
drawing 500 W puts +2000 W on the meter against a -1500 W command: opposed,
both far over any sane threshold, and sustained for HOURS. Armed
unconditionally this watchdog would trip on every single maintenance cycle,
which is not a tuning problem, it is the wrong question being asked. So the
caller says whether the grid-following loop actually won the arbitration this
cycle, and the timer only accumulates when it did.
⚠️ 1500 W, NOT THE DEFAULT 2500 W, AND THE REASON IS NOT COSMETIC. At the
shipped defaults `maintenance_charge_w` is 2500 and `max_w` is 2000, so the
cheap-window claim is `hi = -2500` against a supervised `lo = -2000`: lo > hi,
which arbiter.py resolves as a CONTRADICTION and 0 W, not as a -2500 W charge.
The example above is written at 1500 W because that is a combination the
schema and the arbiter both actually produce. (The contradiction at the
defaults is a real pre-existing bug - a default maintenance charge silently
never happens in a cheap window - but it is maintenance's, not this module's,
and it is filed separately. Do not "fix" it here.)
2. The strict `< 0` product. "Opposed" and "neither side is zero" in one
expression.
3. The 300 W magnitude gate ON BOTH SIDES. Near zero the sign is noise, and near
zero is where a healthy loop LIVES - the deadband holds the command still
while the meter wanders either side of `target_grid_w`. On the healthy day
only 4 212 of 17 280 samples (24 %) are even gated in; the other 76 % are the
converged band where sign carries no information. This is also why the
deadband cannot produce a false trip: inside it the meter is by definition
under 300 W.
4. The 180 s continuity requirement. This is what excludes the legitimate
transient. A load step reverses the meter's sign instantly while the command
is still where the last cycle left it; slew limiting then needs a few cycles
to cross over. Measured on the healthy day, the LONGEST such crossing is 30 s
- and that is with a vendor controller far cruder than ours. 180 s is six
times that. It is not a fitted threshold: at 60 s the healthy day still
produces zero trips, and at 300 s the fault day still produces 13.
⚠️ A BATTERY THAT REFUSES A COMMAND CANNOT TRIP THIS, and that is structural
rather than lucky. At SoC 10 % in Standby, a 1350 W discharge command into a
1350 W import is DECLINED: the battery does nothing, so the meter stays at
+1350 W and the command stays at +1350 W. Same sign. Agreement. A refusal leaves
the meter where it was; only the inverter ACTING moves it, and acting in the
commanded direction moves it toward zero, not through it. The failure mode this
check keys on is the opposite of a refusal - it is the inverter obediently doing
a wrong thing.
⚠️ A FROZEN METER cannot trip it either, for the same reason read the other way:
a stuck reading of +800 W makes the loop command discharge and keep commanding
discharge. Command and meter converge on AGREEING. Freezes are SAFETY-12's and
SAFETY-01's problem, and this watchdog neither helps nor hinders there.
⚠️ ALL OF THAT DEPENDS ON ONE PROPERTY OF THE CONTROL LAW, so it is an invariant
and not a happy accident: the saturation freeze in control.py must keep letting
the command FALL, CROSS ZERO AND REVERSE - "may not wind further in the direction
it is already pushing", not "only corrections that shrink |i_w|". control.py
records the shrink-only encoding as tried and rejected because it pins the command
at its last value indefinitely; a pinned command against a meter that has moved on
is precisely a sustained sign disagreement, so under that encoding this watchdog
WOULD false-trip on a plain load step. If anyone ever revisits that freeze, this
module's no-false-positive argument is downstream of the answer.
------------------------------------------------------------------------------
WHAT IT DOES NOT CATCH
An inverter that ACTS opposite to what it was told. Tell it to discharge 2000 W
and have it charge instead, and the meter moves FURTHER the way it was already
going - so the command and the meter agree, all the way into the rail. This
watchdog compares the loop's output direction against the input state it was
regulating; it is blind to the plant lying about which way it went. That is the
saturation freeze's question (|command - the battery's actual power| > 500 W,
NOTES.md §4.7 row 4), and the two checks are complementary rather than
overlapping. Neither substitutes for the other.
Nor does it see a sign inversion introduced DOWNSTREAM of this add-on - in the
firmware, on the wire, or in a future driver. Our view stays self-consistent and
this stays silent. That is a wire-versus-intent problem and it belongs to the
witness, which is the only vantage point that sees both.
------------------------------------------------------------------------------
STATE, TRIP AND LATCH
One timestamp, not a sample counter. The cycle rate is not fixed - a meter
arrival, or `cycle_floor_s` elapsing, whichever comes first - so N samples is a
different wall-clock window on every path, and the AC is written in minutes.
The latch is not optional and not a nicety. Commanding 0 W removes the command
side of the comparison (|0| is not > 300), so the disagreement disappears the
instant the trip works. Un-latched, this watchdog would trip, clear itself, and
hand the actuator straight back to the loop that was driving the wrong way -
three minutes later, forever. `tripped` therefore stays true until an operator
`reset()`s it by starting control again, which is a deliberate human act after
looking at why.
⚠️ AND THE LATCH HAS TO OUTLIVE THE PROCESS. Held only in memory it is not a
latch at all - a container restart clears it and the add-on comes back armed,
into the same fault, unattended. Worse than merely losing state: "restart the
add-on" is the first thing an operator confronted with a FAILSAFE banner and a
0 W inverter will try, and under a supervisor that restarts a crashed add-on it
is a loop of three-minute wrong-direction bursts. `tripped` and `reason` are
therefore constructor arguments, restored from `Store` in `Controller.__init__`
and written back the moment the trip happens. See store.py.
"""
from dataclasses import dataclass
from datetime import datetime
@dataclass(frozen=True)
class SignWatchdogConfig:
"""Defaults, not constants - config.yaml carries both (SAFETY-02 AC 5)."""
min_w: float = 300.0
hold_s: float = 180.0
def disagrees(commanded_w, grid_w, following: bool,
cfg: SignWatchdogConfig) -> bool:
"""The predicate, and the ONLY place it is written down.
⚠️ Public because `sim/replay_sign.py` needs it too - to describe episodes, to
measure agreement, to sweep the clock-skew guard - and it had reimplemented it
inline. Two copies of a safety predicate is one copy plus a future
disagreement about what the check means, and the copy that drifts is always
the one in the harness that grades it.
"""
return (following
and commanded_w is not None and grid_w is not None
and commanded_w * grid_w < 0
and abs(commanded_w) > cfg.min_w
and abs(grid_w) > cfg.min_w)
def loop_is_following(resolution, loop_asked) -> bool:
"""Did the grid-following loop actually choose the number going on the wire?
Gate 1 of the predicate above, and it lives here rather than inline in
`cycle()` for the reason `disagrees()` does: the test suite has to be able to
assert on THE SHIPPED EXPRESSION, not on a transcription of it that drifts
apart from it in silence.
⚠️ IT WAS A TRANSCRIPTION, AND THAT WAS MEASURED, NOT FEARED. The P_PEAK case
in test_watchdog.py exists to make whoever lands peak shaving come here and
change an expectation. With the expression copied into the test it did not do
that: editing the shipped gate alone to `winner in ("loop", "peak")` left every
suite green while the watchdog went blind for the whole of every peak event.
One definition, two callers, and that test now bites.
`resolution` the arbiter's Resolution. Duck-typed on `.winner` and
`.target_w` so this module still imports nothing from arbiter.
`loop_asked` what the grid-following loop claimed this cycle, or None if it
did not claim at all.
"The loop won" is NOT the question, and the difference is a whole class of
false trip. A LIMIT claim can survive the loop winning and still replace its
number: maintenance's cheap-window shaping is `hi = -maintenance_charge_w`, so
at 1500 W the loop can ask for +500 W of discharge, win the arbitration, and
have the emitted value come out at -1500 W of grid charge. Winner says "loop";
the wire carries maintenance's intent.
So the test is on the NUMBER: the loop is following only while the emitted
value points the way the loop asked AND is no larger than it asked - limits
reduced it, they did not redirect it. The magnitude half is load-bearing on
its own, because a `lo` limit can raise magnitude without flipping sign, and
that is not the loop following either. Zero on either side falls out as
not-following, which costs nothing: a 0 W command cannot be a
wrong-direction command.
⚠️ FORWARD HAZARD, and the claim block in cycle() already anticipates it.
`winner == "loop"` disarms the watchdog under ANY higher-priority SET claim.
Today only maintenance and the safety stops emit one and neither follows the
meter, so the gate costs nothing. A peak-shaving strategy claiming SET at
P_PEAK **is** grid-following in sign, and the moment it lands this goes blind
for the duration of every peak event. Widening it means asking "does the
winning strategy follow the meter", not adding another name to a comparison.
"""
return (resolution.winner == "loop" and loop_asked is not None
and resolution.target_w * loop_asked > 0
and abs(resolution.target_w) <= abs(loop_asked))
class SignWatchdog:
"""One instance per actuator, evaluated once per control cycle.
⚠️ NOT SHARED ACROSS CADENCES. `disagree_since` is a wall-clock start time,
so driving one instance from two loops at two rates would have each one
resetting the other's window. If a fast on-device path ever wants this
check, it gets its own instance - the same exactly-one-writer discipline the
arbiter applies to the actuator, applied to the detector's own state.
"""
def __init__(self, cfg: SignWatchdogConfig | None = None, *,
tripped: bool = False, reason: str = ""):
self.cfg = cfg or SignWatchdogConfig()
self.disagree_since: datetime | None = None
# ⚠️ RESTORED FROM THE STORE, NOT DEFAULTED. A latch that lives only in
# memory is not a latch: a container restart clears it, and the add-on
# comes back armed, into the same fault, with no operator involved. See
# store.py's note on `sign_tripped` for the measured behaviour.
self.tripped = bool(tripped)
self.reason = reason if self.tripped else ""
# -- the whole check, and it is a pure function of its arguments ----------
def evaluate(self, commanded_w, grid_w, following: bool,
now: datetime) -> bool:
"""Advance the detector one cycle. Returns True while tripped.
`commanded_w` the EMITTED setpoint - what is standing on the wire now,
after deadband, slew, quantisation and the arbiter. Not
the optimizer's raw wish: the wire carries the emitted
value, so anything else can disagree with the meter for a
reason that is not a fault.
`grid_w` net P1, + = importing. None when the meter is stale or
missing, which CLEARS the timer rather than accumulating
- a frozen figure held against a moving command would
otherwise manufacture a disagreement in about three
minutes, on top of SAFETY-01 having already commanded
zero for the same cause.
`following` did the grid-following loop actually win this cycle? See
gate 1 in the module docstring.
`now` the caller's clock, so this stays testable and replayable.
"""
if self.tripped:
# Latched. Do not re-evaluate: the trip removed the evidence.
return True
cfg = self.cfg
if not disagrees(commanded_w, grid_w, following, cfg):
self.disagree_since = None
return False
if self.disagree_since is None:
self.disagree_since = now
return False
held = (now - self.disagree_since).total_seconds()
if held < cfg.hold_s:
return False
self.tripped = True
self.reason = (
f"SIGN_DISAGREEMENT: commanded {commanded_w:+.0f} W against a meter "
f"reading {grid_w:+.0f} W for {held:.0f} s continuously "
f"(limits: both over {cfg.min_w:.0f} W, held over {cfg.hold_s:.0f} s)"
)
return True
def held_s(self, now: datetime) -> float:
"""How long the current disagreement has been standing, for the UI.
⚠️ Takes the clock rather than reading one. It was a property calling
`datetime.now()`, which quietly broke the property this module's header
advertises - every other observable here is a pure function of what it
was handed - and an unclocked function is one nothing can test, which is
exactly what happened to it.
"""
if self.disagree_since is None:
return 0.0
return (now - self.disagree_since).total_seconds()
def reset(self) -> None:
self.disagree_since = None
self.tripped = False
self.reason = ""
+44 -3
View File
@@ -1,5 +1,5 @@
name: GoodWe RS485 Controller
version: "0.4.0"
version: "0.5.1"
slug: goodwe_controller
description: >-
Drives a GoodWe ES/BP battery inverter over RS485 by emulating its smart
@@ -93,12 +93,48 @@ options:
cycle_floor_s: 15
auto_start: false
# --- SAFETY-02 sign-disagreement watchdog ----------------------------------
# Trips when the emitted setpoint and the P1 net figure point OPPOSITE ways,
# both over sign_disagree_w, continuously for sign_disagree_s. Action: 0 W and
# a failsafe that latches until an operator starts control again.
#
# Both defaults are measured, not chosen. On the captured healthy day
# (2026-08-23) the longest legitimate opposed run - a load step the loop has
# not caught up with - is 30 s, and sign agreement over the gated samples is
# 93.4 %. On the captured fault day (2026-08-20) the clusters run 0.0-10.5 %
# agreement with opposed runs up to 1752 s. 300 W / 180 s sits in the middle of
# a very wide gap: at 60 s the healthy day still trips nothing, at 300 s the
# fault day still trips 13 times. There is no reason to move either number
# without a capture that says so. They are options because the AC says
# thresholds are defaults rather than constants - not because they are knobs
# anyone is expected to turn.
#
# ⚠️ sign_disagree_w must stay BELOW max_w or the watchdog can never fire (the
# arbiter bounds the command to +-max_w). validate.py refuses that config.
sign_disagree_w: 300
sign_disagree_s: 180
# --- maintenance -----------------------------------------------------------
maintenance_enabled: false
maintenance_interval_days: 28
maintenance_start_hour: 10
maintenance_discharge_w: 2500
maintenance_charge_w: 2500
# ⚠️ BOTH MUST STAY AT OR BELOW max_w, and that is why they are 2000 rather
# than the 2500 they shipped as. The cheap-window charge shaping asks the
# arbiter for `hi = -maintenance_charge_w` against the supervised `lo =
# -max_w`: above the ceiling that is `lo > hi`, a contradiction resolved to
# 0 W, so maintenance would SILENTLY NEVER CHARGE in a cheap window on a
# default install. validate.py refuses that config (rule 6), so these
# defaults had to become self-consistent with the max_w above rather than
# ship a configuration the validator rejects.
#
# The discharge side is clamped rather than contradicted - it drains at max_w
# and the arbiter says so - so it is only warned about; it is 2000 here for
# symmetry and to keep the shipped config warning-free.
#
# Raising max_w (the live reference install runs 5000) is what unlocks a
# faster maintenance cycle. Raise max_w FIRST, then these.
maintenance_discharge_w: 2000
maintenance_charge_w: 2000
maintenance_soc_floor: 11
maintenance_soc_target: 99
maintenance_hold_min: 120
@@ -155,6 +191,11 @@ schema:
cycle_floor_s: int(5,60)
auto_start: bool
# No "off" value, deliberately: this is a P0 safety check and the schema is
# not the place to make disarming it a supported configuration.
sign_disagree_w: int(50,5000)
sign_disagree_s: int(30,1800)
maintenance_enabled: bool
maintenance_interval_days: int(1,90)
maintenance_start_hour: int(0,23)
+182 -2
View File
@@ -26,9 +26,15 @@ def check(name, cond):
def base(**over):
"""A configuration that must start. Everything else is this, spoilt."""
# ⚠️ The maintenance rates are named EXPLICITLY, at the values config.yaml
# ships (MAINT-04). They are not decoration: with the key absent, rule 6
# falls back to main.py's 2500 W, which against this 2000 W ceiling is a
# genuine refusal - so a base() without them would be a config that does
# not start, and every check below would be measuring the wrong thing.
o = {"setpoint_entity": "number.goodwe_setpoint", "max_w": 2000,
"soc_entity": "sensor.soc", "batt_entity": "sensor.batt",
"meter_source": "off"}
"meter_source": "off",
"maintenance_charge_w": 2000, "maintenance_discharge_w": 2000}
o.update(over)
return o
@@ -151,6 +157,170 @@ check("with meter_source off the number governs nothing and is not refused",
check("the constant is the firmware's, not a rounded guess",
FIRMWARE_INGEST_VETO_S == 30.0)
# --------------------------------------------------------------------------- #
print("6. a maintenance charge the ceiling can actually accommodate (MAINT-04)")
# ⚠️ THE SHIPPED BUG. config.yaml shipped max_w 2000 against
# maintenance_charge_w 2500, so the cheap-window shaping asked for hi=-2500
# against the supervised lo=-2000 - lo > hi, arbiter clause 3, 0 W. Maintenance
# silently never charged in a cheap window on a default install.
p = validate_startup_config(base(max_w=2000, maintenance_charge_w=2500),
-5000, 5000)
check("maintenance_charge_w above max_w is refused", len(p) == 1)
check("...and the message names the charge rate", why(p, "2500"))
check("...and the ceiling", why(p, "2000"))
check("...and says which one wins", why(p, "THE CEILING WINS"))
# ⚠️ Equality is REACHABLE, not a near miss: lo=-2000, hi=-2000 is the single
# point -2000 W. Refusing it would be exactly the trivia that gets a validator
# deleted, and it is the boundary the rule is written `>` for.
check("maintenance_charge_w exactly at max_w starts",
validate_startup_config(base(max_w=2000, maintenance_charge_w=2000),
-5000, 5000) == [])
check("below it starts",
validate_startup_config(base(max_w=2000, maintenance_charge_w=1500),
-5000, 5000) == [])
# Not scoped to maintenance_enabled: the pair is contradictory either way, and
# scoping would leave the trap armed for the one action - ticking the box - that
# cannot be expected to change what a charge rate means.
check("...and it is refused with maintenance_enabled off, as shipped",
why(validate_startup_config(
base(max_w=2000, maintenance_charge_w=2500,
maintenance_enabled=False), -5000, 5000), "THE CEILING WINS"))
# ⚠️ THE SIBLING, CHECKED RATHER THAN ASSUMED. The shaping is NOT symmetric:
# drain files Claim.set at P_MAINTENANCE and owns the actuator, so the ceiling
# CLAMPS it (+2000 W) instead of contradicting it, and the arbiter's explain()
# names the clamp every cycle. Loud, and it still moves energy - so warned, not
# refused. If someone later "fixes the asymmetry" by making this a refusal,
# this check goes red and they have to justify it.
check("maintenance_discharge_w above max_w WARNS but starts",
validate_startup_config(base(max_w=2000, maintenance_discharge_w=2500),
-5000, 5000) == [])
# The demonstration the rule is derived from, asserted rather than described.
from app.arbiter import ( # noqa: E402
Claim, P_LOOP, P_MAINTENANCE, P_MAINT_SHAPE, P_SAFETY, resolve,
)
def _charge(max_w, charge_w):
return resolve([Claim.limit("supervised", P_SAFETY, -max_w, max_w, "max_w"),
Claim.limit("maintenance", P_MAINT_SHAPE, hi=-charge_w,
reason="cheap-window charge"),
Claim.set("loop", P_LOOP, 500.0, "grid")])
def _drain(max_w, disch_w):
return resolve([Claim.limit("supervised", P_SAFETY, -max_w, max_w, "max_w"),
Claim.set("maintenance", P_MAINTENANCE, disch_w, "drain"),
Claim.set("loop", P_LOOP, 500.0, "grid")])
r = _charge(2000, 2500)
check("arbiter: the refused pair really does emit 0 W",
r.target_w == 0.0 and r.contradiction)
r = _charge(2000, 2000)
check("arbiter: the permitted equality really does charge",
r.target_w == -2000.0 and not r.contradiction)
r = _charge(3000, 2500)
check("arbiter: raising max_w above the rate charges at the rate",
r.target_w == -2500.0 and not r.contradiction)
r = _drain(2000, 2500)
check("arbiter: the drain SET is clamped, not contradicted",
r.target_w == 2000.0 and not r.contradiction)
check("...and the clamp names itself, which is why it is not refused",
"supervised" in (r.bound_by or "") and "limited by" in r.explain())
# --------------------------------------------------------------------------- #
print("the SHIPPED defaults must pass the rule they are governed by (AC3)")
# ⚠️ Shipping a default the validator refuses would be worse than the bug it
# refuses. Read config.yaml itself rather than a transcription of it - a
# transcription is what lets the two drift apart silently. Regex, not PyYAML:
# the add-on image has no PyYAML (Dockerfile installs py3-aiohttp/py3-paho-mqtt
# only) and a check that skips on ImportError is not a check.
import os # noqa: E402
import re # noqa: E402
_CFG = os.path.join(os.path.dirname(os.path.abspath(__file__)), "config.yaml")
with open(_CFG, encoding="utf-8") as _fh:
_txt = _fh.read()
_shipped = {k: float(v) for k, v in re.findall(
r"^ (max_w|maintenance_charge_w|maintenance_discharge_w|sign_disagree_w): "
r"(-?\d+(?:\.\d+)?)\s*$", _txt, re.M)}
check("config.yaml really was read (all four defaults found)",
set(_shipped) == {"max_w", "maintenance_charge_w",
"maintenance_discharge_w", "sign_disagree_w"})
check(f"the shipped defaults are accepted {_shipped}",
validate_startup_config(base(**_shipped), -5000, 5000) == [])
check("...and the shipped charge rate is at or under the shipped ceiling",
_shipped["maintenance_charge_w"] <= _shipped["max_w"])
check("...and the shipped discharge rate too, so it warns about nothing",
_shipped["maintenance_discharge_w"] <= _shipped["max_w"])
# --------------------------------------------------------------------------- #
print("the KNOWN cross-file default divergence, pinned rather than commented")
# ⚠️ WHAT THIS GUARDS, AND WHY IT ASSERTS THE BUG RATHER THAN THE FIX.
# config.yaml and validate.py now default maintenance_charge_w to 2000, while
# main.py's two `opts.get(..., 2500)` fallbacks and MaintConfig's dataclass
# default still say 2500. So an options dict with the key ABSENT is ACCEPTED by
# rule 6 at 2000 and then built by main.py at 2500 - which is a contradiction:
#
# validate_startup_config -> [] (accepted)
# MaintConfig().charge_w -> 2500.0
# arbiter -> 0.0 W, contradiction=True
#
# That is the bug rule 6 exists to catch, passing rule 6. It is unreachable on a
# Supervisor install (every key is always supplied from config.yaml) but fully
# reachable from sim/inject.py and from tests - which is exactly where anyone
# debugging maintenance works.
#
# The fix is ONE SOURCE OF TRUTH for option defaults. That is a bigger change
# than this ticket and it lands in main.py, which another ticket is live in - so
# the divergence is RECORDED here instead of closed. The check therefore passes
# on the mismatch and FAILS when either side moves, in EITHER direction:
#
# - config.yaml moves -> FAIL: re-decide what the fallback should be
# - main.py is fixed to match -> FAIL: the divergence is gone, DELETE this
# check and the ⚠️ block in validate.py
#
# A comment cannot do that, and "a comment where a check belonged" is this
# repo's most repeated defect.
_APP = os.path.join(os.path.dirname(os.path.abspath(__file__)), "app")
def _read(*parts):
with open(os.path.join(*parts), encoding="utf-8") as fh:
return fh.read()
# Every literal fallback for this key in main.py, whatever the call shape.
_main_falls = [float(m) for m in re.findall(
r'\.get\(\s*"maintenance_charge_w"\s*,\s*(-?\d+(?:\.\d+)?)\s*\)',
_read(_APP, "main.py"))]
# ...and MaintConfig's own dataclass default, which is the one that bites when
# main.py is bypassed entirely (a Maintenance built straight from defaults).
_dc_falls = [float(m) for m in re.findall(
r"^\s*charge_w:\s*float\s*=\s*(-?\d+(?:\.\d+)?)\s*$",
_read(_APP, "maintenance.py"), re.M)]
check(f"main.py's fallbacks are still found and still 2500 {_main_falls}",
len(_main_falls) == 2 and set(_main_falls) == {2500.0})
check(f"MaintConfig's default is still found and still 2500 {_dc_falls}",
len(_dc_falls) == 1 and set(_dc_falls) == {2500.0})
check(f"...and config.yaml still ships 2000, so the divergence is UNCHANGED "
f"(shipped {_shipped['maintenance_charge_w']:.0f}, code 2500)",
_shipped["maintenance_charge_w"] == 2000.0)
# ⚠️ And the consequence itself, through the REAL code rather than through the
# numbers above - so that if someone "fixes" this by changing a literal without
# understanding it, the thing that goes red is the behaviour, not the transcript.
from app.maintenance import MaintConfig # noqa: E402
_absent = {k: v for k, v in base().items() if k != "maintenance_charge_w"}
_built = MaintConfig().charge_w
check("with the key ABSENT the validator accepts...",
validate_startup_config(_absent, -5000, 5000) == [])
check(f"...while the code would build {_built:.0f} W against a "
f"{_absent['max_w']:.0f} W ceiling...", _built > _absent["max_w"])
_r = _charge(_absent["max_w"], _built)
check("...which the arbiter resolves to 0 W as a contradiction - the hole, "
"asserted so it cannot widen unnoticed",
_r.target_w == 0.0 and _r.contradiction)
# --------------------------------------------------------------------------- #
print("max_w must be a positive magnitude")
# Schema-blocked on a real install, but validate_startup_config() is also
@@ -160,8 +330,18 @@ for bad in (0, -1, -2000):
check(f"max_w {bad} is refused", len(p) == 1 and why(p, "must be positive"))
check("...and the message explains it would read as a trip, not a mistake",
why(validate_startup_config(base(max_w=0), -5000, 5000), "contradiction"))
# ⚠️ SAFETY-02 changed what this case has to say, and the change is real rather
# than bookkeeping: with a 1 W ceiling the default 300 W sign-disagreement
# threshold is unreachable, so rule 5 refuses - correctly, because the watchdog
# would report itself armed while being incapable of firing. What this check is
# FOR is "a positive max_w is not itself a refusal", so it now names a sign
# threshold the ceiling can actually reach, and the interaction gets its own
# check below.
check("the smallest sane max_w still starts",
validate_startup_config(base(max_w=1), -5000, 5000) == [])
validate_startup_config(base(max_w=1, sign_disagree_w=0.5,
maintenance_charge_w=1), -5000, 5000) == [])
check("...but that same 1 W ceiling with the DEFAULT sign threshold is refused",
why(validate_startup_config(base(max_w=1), -5000, 5000), "never fire"))
# --------------------------------------------------------------------------- #
print("what must NOT be refused - a validator that fires on trivia gets deleted")
+495
View File
@@ -0,0 +1,495 @@
"""Runnable check for the sign-disagreement watchdog. `python3 test_watchdog.py`
Every case here is either a fault this must catch or a piece of NORMAL behaviour
it must not mistake for one. The second list is the longer one on purpose: an
alarm that fires on ordinary operation is worse than no alarm, because the next
person switches it off.
The headline validation is not here - it is `sim/replay_sign.py` in the infra
repo, which drives this module over the two captured bus days. This file is the
table-driven half: boundaries, timers, the latch, and the arming gate.
"""
import sys
from datetime import datetime, timedelta, timezone
from app.watchdog import (
SignWatchdog, SignWatchdogConfig, loop_is_following,
)
fails = []
T0 = datetime(2026, 8, 20, 15, 0, 0, tzinfo=timezone.utc)
def check(name, cond):
print(f" {'ok ' if cond else 'FAIL'} {name}")
if not cond:
fails.append(name)
def drive(samples, cfg=None, following=True):
"""Feed (offset_s, commanded_w, grid_w) tuples. Returns (wd, trip_offset)."""
wd = SignWatchdog(cfg)
trip_at = None
for item in samples:
offset, cmd, grid = item[0], item[1], item[2]
f = item[3] if len(item) > 3 else following
if wd.evaluate(cmd, grid, f, T0 + timedelta(seconds=offset)) and trip_at is None:
trip_at = offset
return wd, trip_at
def opposed(duration_s, step=5, cmd=-1000, grid=1500, following=True, start=0):
return [(start + t, cmd, grid, following)
for t in range(0, int(duration_s) + 1, step)]
print("the fault it exists to catch")
wd, at = drive(opposed(300))
check("charging into an importing house trips", wd.tripped)
check("and trips at 180 s, not before", at == 180)
check("the reason names both signals", "-1000" in wd.reason and "+1500" in wd.reason)
print(f" e.g. {wd.reason!r}")
wd, at = drive(opposed(300, cmd=2000, grid=-2000))
check("discharging into an exporting house trips too", wd.tripped and at == 180)
print()
print("the boundaries")
wd, _ = drive(opposed(600, cmd=-300, grid=1500))
check("300 W exactly does NOT qualify (strictly greater)", not wd.tripped)
wd, _ = drive(opposed(600, cmd=-301, grid=1500))
check("301 W does", wd.tripped)
wd, _ = drive(opposed(600, cmd=-1000, grid=300))
check("a 300 W meter does not qualify either", not wd.tripped)
wd, _ = drive(opposed(175))
check("175 s of disagreement does not trip", not wd.tripped)
wd, _ = drive(opposed(180))
check("180 s does (>= hold_s)", wd.tripped)
print()
print("continuity - the rule that produced zero false positives on real data")
broken = opposed(175) + [(180, -1000, -1500)] + opposed(175, start=185)
wd, _ = drive(broken)
check("179 s, one agreeing sample, then 175 s more: no trip", not wd.tripped)
check("and the window really was reset", wd.disagree_since is not None)
long_enough = broken + opposed(200, start=365)
wd, _ = drive(long_enough)
check("but a full 180 s after the reset does trip", wd.tripped)
print()
print("things that are NOT faults")
# The load-step transient. Measured on the healthy capture: longest 30 s.
wd, _ = drive(opposed(30) + [(35 + t, 800, 900) for t in range(0, 300, 5)])
check("a 30 s load-step crossover does not trip", not wd.tripped)
# SoC 10 %, Standby, declining a 1350 W discharge. The battery does nothing, so
# the meter stays where it was and the command agrees with it.
wd, _ = drive([(t, 1350, 1350) for t in range(0, 900, 5)])
check("a battery refusing a discharge command cannot trip it", not wd.tripped)
# A frozen meter drives command and reading into agreement, not disagreement.
wd, _ = drive([(t, 800, 800) for t in range(0, 900, 5)])
check("a frozen meter cannot trip it", not wd.tripped)
# Maintenance grid charge: 2500 W of charge against an importing house, for
# hours. Opposed, both huge - and entirely correct.
wd, _ = drive(opposed(7200, cmd=-2500, grid=3000, following=False))
check("two hours of maintenance grid charge does not trip", not wd.tripped)
check("and the timer never even started", wd.disagree_since is None)
wd, _ = drive([(t, -1000, None) for t in range(0, 900, 5)])
check("a missing meter reading does not trip", not wd.tripped)
wd, _ = drive(opposed(175) + [(180, -1000, None)] + opposed(600, start=185))
check("a missing reading mid-window RESETS it rather than accumulating",
wd.tripped and wd.disagree_since is not None)
wd, at = drive(opposed(175) + [(180, -1000, None)] + opposed(600, start=185))
check("...so the trip lands 180 s after the gap, not after 180 s total",
at == 365)
wd, _ = drive([(t, 0.0, 3000) for t in range(0, 900, 5)])
check("a 0 W command is not a wrong-direction command", not wd.tripped)
print()
print("the latch")
wd, _ = drive(opposed(200))
check("tripped after the fault", wd.tripped)
# The trip commands 0 W, which removes the command side of the comparison.
for t in range(200, 900, 5):
wd.evaluate(0.0, 3000, True, T0 + timedelta(seconds=t))
check("stays tripped once the command is zeroed", wd.tripped)
for t in range(900, 1500, 5):
wd.evaluate(1000, 1000, True, T0 + timedelta(seconds=t))
check("stays tripped even once everything looks healthy again", wd.tripped)
wd.reset()
check("reset() is the only way out", not wd.tripped and wd.disagree_since is None)
check("and the reason is cleared with it", wd.reason == "")
print()
print("the thresholds are configuration, not constants")
cfg = SignWatchdogConfig(min_w=800.0, hold_s=60.0)
wd, at = drive(opposed(300, cmd=-1000, grid=1500), cfg=cfg)
check("a 60 s hold trips at 60 s", wd.tripped and at == 60)
wd, _ = drive(opposed(600, cmd=-700, grid=1500), cfg=cfg)
check("an 800 W gate ignores a 700 W command", not wd.tripped)
print()
print("the configuration that would disarm it is refused")
from app.validate import validate_startup_config # noqa: E402
base = {"setpoint_entity": "number.sp", "meter_source": "off",
"meter_phases": 1, "meter_max_age_s": 30}
p = validate_startup_config({**base, "max_w": 2000, "sign_disagree_w": 300},
-5000, 5000)
check("300 W under a 2000 W ceiling is accepted", p == [])
p = validate_startup_config({**base, "max_w": 2000, "sign_disagree_w": 2000},
-5000, 5000)
check("a threshold AT max_w is refused", len(p) == 1 and "never fire" in p[0])
# ⚠️ maintenance_charge_w is named here only to keep this case about the SIGN
# rule. MAINT-04 added a second rule against the same ceiling, and a 500 W
# max_w is also below the 2000 W shipped maintenance charge - so without this
# the dict is refused twice, for two correct but different reasons, and the
# `len(p) == 1` below would be measuring the interaction rather than this rule.
p = validate_startup_config({**base, "max_w": 500, "sign_disagree_w": 3000,
"maintenance_charge_w": 500}, -5000, 5000)
check("a threshold above max_w is refused", len(p) == 1 and "never fire" in p[0])
p = validate_startup_config({**base, "max_w": 2000, "sign_disagree_w": 1999},
-5000, 5000)
check("one watt under the ceiling is reachable, so accepted", p == [])
print()
print("the arming gate, derived from the REAL arbiter")
# ⚠️ THIS IS THE DECISION THE WHOLE TICKET TURNS ON, and until now it was
# asserted in three comment blocks and verified by nothing: every other case in
# this file passes `following` in as a literal. What has to hold is that the
# expression in main.py - loop won AND the emitted value points the way the loop
# asked AND is no larger - is never True for a resolution the loop did not
# actually get. So run real claims through the real resolve() and derive it.
from app.arbiter import ( # noqa: E402
P_LOOP, P_MAINT_SHAPE, P_MAINTENANCE, P_PEAK, P_SAFETY, Claim, resolve,
)
def following_from(claims, asked):
"""THE SHIPPED gate, not a copy of it.
⚠️ This used to transcribe main.py's expression with a comment saying
"keep the two in step", which is exactly the arrangement that does not keep
two things in step. Measured: editing the shipped gate alone to
`winner in ("loop", "peak")` left this suite - including the P_PEAK row below,
whose entire job is to catch that edit - completely green, while the watchdog
went blind for the duration of every peak event. Import it or the P_PEAK
assertion is decoration.
"""
r = resolve(claims)
return r, loop_is_following(r, asked)
MAXW = Claim.limit("supervised", P_SAFETY, -2000, 2000, "max_w")
DEV = Claim.limit("device", P_SAFETY, -5000, 5000, "rating")
# name, claims, what the loop asked, expected `following`
TABLE = [
("the loop alone, unopposed",
[Claim.set("loop", P_LOOP, 900)], 900, True),
("the loop bound by max_w - reduced, not redirected",
[Claim.set("loop", P_LOOP, 4000), MAXW], 4000, True),
("charging, bound by max_w on the other side",
[Claim.set("loop", P_LOOP, -4000), MAXW], -4000, True),
("maintenance owns the setpoint outright",
[Claim.set("loop", P_LOOP, 900),
Claim.set("maintenance", P_MAINTENANCE, -2500, "charge"), MAXW], 900, False),
("maintenance charge-only clamps the loop to 0 W",
[Claim.set("loop", P_LOOP, 900),
Claim.limit("maintenance", P_MAINT_SHAPE, hi=0.0, reason="charge-only"),
MAXW], 900, False),
# ⚠️ THE CASE `winner == \"loop\"` ALONE WOULD HAVE GOT WRONG. 1500 W, not
# the default 2500 W: at max_w 2000 a -2500 hi against a -2000 lo is an
# arbiter contradiction, not a charge - see main.py.
("cheap-window charge REDIRECTS the loop while it still 'wins'",
[Claim.set("loop", P_LOOP, 500),
Claim.limit("maintenance", P_MAINT_SHAPE, hi=-1500, reason="cheap-window"),
MAXW], 500, False),
("...and the contradiction at the shipped defaults is not following either",
[Claim.set("loop", P_LOOP, 500),
Claim.limit("maintenance", P_MAINT_SHAPE, hi=-2500, reason="cheap-window"),
MAXW], 500, False),
("a lo limit RAISES the magnitude without flipping the sign",
[Claim.set("loop", P_LOOP, 200),
Claim.limit("floor", P_SAFETY, lo=800), MAXW], 200, False),
("a safety stop wins outright",
[Claim.set("loop", P_LOOP, 900),
Claim.set("safety", P_SAFETY, 0.0, "stopped"), MAXW], 900, False),
("no claims at all - the arbiter's own failsafe",
[MAXW, DEV], None, False),
("the loop asked for zero",
[Claim.set("loop", P_LOOP, 0.0), MAXW], 0.0, False),
# The forward hazard, asserted as it stands TODAY rather than as it should
# be. When peak shaving lands, these expectations are what have to change.
("peak shaving at P_PEAK disarms the gate (loop_is_following's forward hazard)",
[Claim.set("loop", P_LOOP, 900),
Claim.set("peak", P_PEAK, 1800, "shaving"), MAXW], 900, False),
# ⚠️ AND THE ROW ABOVE CANNOT DETECT A WIDENED GATE ON ITS OWN, which was
# found by trying it: with the shipped gate edited to
# `winner in ("loop", "peak")` that case STILL comes out False, because peak
# shaving asked for MORE than the loop and the magnitude term rejects it
# anyway. The winner term is masked. This row isolates it - a peak claim
# smaller than the loop's ask and pointing the same way, so `winner` is the
# only term that can decide - and it is what actually turns red if the gate
# is widened without anyone revisiting the hazard.
("...and with peak BELOW the loop's ask, only the winner term separates them",
[Claim.set("loop", P_LOOP, 1800),
Claim.set("peak", P_PEAK, 900, "shaving"), MAXW], 1800, False),
]
for name, claims, asked, expect in TABLE:
r, got = following_from(claims, asked)
check(f"{name} -> following={expect}", got is expect)
if got is not expect:
print(f" resolution was {r.explain()!r}")
# And the gate is never True when the loop did not win, whatever else is going on.
check("no configuration above arms the gate without the loop winning",
all(not following_from(c, a)[1]
for _, c, a, _ in TABLE if following_from(c, a)[0].winner != "loop"))
print()
print("end to end: the trip is COMMAND ZERO, not alarm-only")
# ⚠️ THE FAULT INJECTED HERE IS A NEGATIVE GAIN, and that is not a contrivance -
# it is the minimal faithful model of what this watchdog is for. The check
# compares the loop's OUTPUT DIRECTION against the INPUT STATE it was regulating,
# so what it detects is a loop whose two ends have become decoupled in sign: a
# gain, a claim or a strategy with its polarity inverted, or an integrator so far
# wound up that unwinding takes minutes. (Illusmart's was the last one: it
# reached -14 768 W against a +-5000 W rail while writing once every 20-60 s, and
# that arithmetic alone is 5-15 minutes of wrong-direction command.)
#
# `gain` is float(0.05,1.0) in the schema, so this is not a reachable
# configuration - it is a stand-in for a code regression, which is the case a
# unit test can actually hold the line on.
import asyncio # noqa: E402
from app.main import Controller # noqa: E402
class _Hass:
def __init__(self):
self.writes = []
async def limits(self, entity):
return (-5000.0, 5000.0, 1.0)
async def number(self, entity, invert=False):
# A house importing 2000 W, steadily, whatever the battery does.
return {"sensor.grid": 2000.0, "sensor.soc": 50.0,
"sensor.batt": 0.0}.get(entity)
async def set_number(self, entity, value):
self.writes.append(float(value))
return True
class _S:
data = {}
def set(self, *a):
pass
def get_time(self, *a):
return None
class _P:
enabled = False
def publish(self, *a, **k):
pass
def close(self):
pass
async def _e2e():
hass = _Hass()
opts = {"setpoint_entity": "number.sp", "meter_entity": "sensor.grid",
"soc_entity": "sensor.soc", "batt_entity": "sensor.batt",
"meter_source": "off", "max_w": 2000, "gain": -0.6,
"sign_disagree_w": 300, "sign_disagree_s": 1, "auto_start": True}
c = Controller(opts, hass, _S(), _P(), stop=asyncio.Event())
c.auto = True
c.dev_min, c.dev_max = -5000.0, 5000.0
seen_wrong_way = False
for _ in range(12):
await c.read_inputs()
await c.cycle()
if c.target < -300 and not c.sign_wd.tripped:
seen_wrong_way = True
await asyncio.sleep(0.25)
return c, hass, seen_wrong_way
ctrl, hass, wrong_way = asyncio.run(_e2e())
check("the broken loop really did command charge into an import first", wrong_way)
check("the watchdog tripped", ctrl.sign_wd.tripped)
check("the commanded value is 0 W", ctrl.target == 0.0)
check("0 W actually reached the device", 0.0 in hass.writes)
check("and it got there through the failsafe claim, not by accident",
"failsafe" in ctrl.reason)
print(f" status line: {ctrl.reason!r}")
check("the status page says so in words",
any("SIGN-DISAGREEMENT FAILSAFE" in c["text"] for c in ctrl.checks()))
check("the banner leads with it",
"FAILSAFE" in ctrl.status_payload()["banner"])
check("it stays at 0 W on later cycles too",
hass.writes[-1] == 0.0 and hass.writes.count(0.0) >= 1)
# ⚠️ THIS USED TO BE `check("MQTT reports the failsafe state", ctrl.sign_wd.tripped)`
# which restates the assertion three lines above and never touches the payload.
# The `status: "failsafe"` value was shipped untested. Capture what publish()
# actually hands the broker.
published = {}
ctrl.mqtt = type("P", (), {"enabled": True,
"publish": lambda _s, v: published.update(v),
"close": lambda _s: None})()
ctrl.publish()
check("MQTT's controller-status sensor reports 'failsafe'",
published.get("status") == "failsafe")
ctrl.sign_wd.reset()
ctrl.publish()
check("...and goes back to 'running' once cleared",
published.get("status") == "running")
print()
print("the latch survives a restart - it is the only thing between a tripped")
print("controller and the first thing an operator will try")
# ⚠️ THE BLOCKER THIS FILE MISSED FIRST TIME ROUND. `auto` is persisted and
# `tripped` was not, so reconstructing a Controller against the same Store -
# which is exactly what a container restart does - came back armed, into the
# fault, with no operator involved: tripped=False, banner NOT READY, and it
# resumed commanding -1000 then -2000 W before re-tripping 180 s later.
class _MemStore:
"""A Store that keeps its dict across "restarts", like the real file does."""
def __init__(self):
self.data = {"phase": "idle", "sign_tripped": False, "sign_reason": ""}
def set(self, k, v):
self.data[k] = v
def get_time(self, *a):
return None
async def _restart():
shared = _MemStore()
hass1 = _Hass()
opts = {"setpoint_entity": "number.sp", "meter_entity": "sensor.grid",
"soc_entity": "sensor.soc", "batt_entity": "sensor.batt",
"meter_source": "off", "max_w": 2000, "gain": -0.6,
"sign_disagree_w": 300, "sign_disagree_s": 1, "auto_start": True}
a = Controller(opts, hass1, shared, _P(), stop=asyncio.Event())
a.auto = True
a.dev_min, a.dev_max = -5000.0, 5000.0
for _ in range(12):
await a.read_inputs()
await a.cycle()
await asyncio.sleep(0.25)
# The restart. Same options, same store, brand new Controller and Hass.
hass2 = _Hass()
b = Controller(opts, hass2, shared, _P(), stop=asyncio.Event())
b.auto = True
b.dev_min, b.dev_max = -5000.0, 5000.0
for _ in range(6):
await b.read_inputs()
await b.cycle()
await asyncio.sleep(0.05)
return a, b, shared, hass2
pre, post, store, hass2 = asyncio.run(_restart())
check("it tripped before the restart", pre.sign_wd.tripped)
check("the trip reached the store", store.data["sign_tripped"] is True)
check("...with the reason beside it, so the operator is told why",
"SIGN_DISAGREEMENT" in store.data["sign_reason"])
check("the new Controller comes back TRIPPED", post.sign_wd.tripped)
check("...and says why, restored rather than re-detected",
"SIGN_DISAGREEMENT" in post.sign_wd.reason)
check("...commands 0 W from its very first cycle", post.target == 0.0)
check("...and never commanded anything else after the restart",
hass2.writes != [] and set(hass2.writes) == {0.0})
check("the banner still leads with the failsafe",
"FAILSAFE" in post.status_payload()["banner"])
# ⚠️ TWO PRESSES, and the first one must NOT clear it. `auto` is True while
# tripped, so the first press stops control; only the transition back to True
# clears. That is the documented behaviour and it is worth an assertion, because
# a one-press clear would be reachable by an operator who never read the banner.
asyncio.run(post.handle_action("auto_toggle")) # -> stopped
check("pressing stop does not clear the failsafe",
post.sign_wd.tripped and store.data["sign_tripped"] is True)
asyncio.run(post.handle_action("auto_toggle")) # -> started
check("pressing start DOES clear it, and clears the store with it",
not post.sign_wd.tripped and store.data["sign_tripped"] is False
and store.data["sign_reason"] == "")
# One more restart from the cleared store, to prove the clear persisted too.
fresh = Controller({"setpoint_entity": "number.sp", "meter_source": "off"},
_Hass(), store, _P(), stop=asyncio.Event())
check("...so the next restart comes back armed and untripped, as it should",
not fresh.sign_wd.tripped)
# ⚠️ THE WRITE ORDERING IS AN INVARIANT, SO IT GETS AN ASSERTION. main.py writes
# `sign_reason` and THEN `sign_tripped`, each one its own `save()`, so that a
# process that dies between the two leaves a reason nothing reads - harmless -
# rather than a latch with no explanation for the operator who has to decide
# whether to clear it. Documented reasoning is not enforcement: swapping those two
# lines used to pass every check in this file.
#
# The death is modelled as the second write raising. In the field it is a power
# cut or a container kill (Store.save() swallows OSError rather than raising), so
# what is asserted here is the PROPERTY - what survives on disk - not the
# mechanism.
class _CrashOnFlag(_MemStore):
def set(self, k, v):
if k == "sign_tripped" and v:
raise RuntimeError("power cut, mid-trip")
super().set(k, v)
async def _crash_mid_write():
store = _CrashOnFlag()
hass = _Hass()
opts = {"setpoint_entity": "number.sp", "meter_entity": "sensor.grid",
"soc_entity": "sensor.soc", "batt_entity": "sensor.batt",
"meter_source": "off", "max_w": 2000, "gain": -0.6,
"sign_disagree_w": 300, "sign_disagree_s": 1, "auto_start": True}
c = Controller(opts, hass, store, _P(), stop=asyncio.Event())
c.auto = True
c.dev_min, c.dev_max = -5000.0, 5000.0
died = False
for _ in range(12):
try:
await c.read_inputs()
await c.cycle()
except RuntimeError:
died = True
break
await asyncio.sleep(0.25)
return store, died
crashed_store, died = asyncio.run(_crash_mid_write())
check("the second write is the one that sets the flag", died)
check("a crash between the two writes leaves the REASON on disk",
"SIGN_DISAGREEMENT" in crashed_store.data.get("sign_reason", ""))
check("...and never a latch with nothing to explain it",
not crashed_store.data.get("sign_tripped"))
print()
if fails:
print(f"{len(fails)} FAILED: {', '.join(fails)}")
sys.exit(1)
print("all checks passed")