Compare commits

...
30 Commits
Author SHA1 Message Date
glenn schrooyenandClaude Opus 5 4b521b2df0 ci: trigger a run on the rebased head
checks / unit (pull_request) Successful in 2m33s
No content change. The gate never created a run for 0942228d - the newest
run stayed on d31a12b7, the pre-rebase head - so `checks / unit` had
nothing to report and sat pending. Why no run was created is CI-02 and
belongs to CI-01's owner; this commit only asks for one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 18:11:21 +02:00
glenn schrooyenandClaude Opus 5 0942228d65 Merge release/1.0 (SAFETY-03) into TEST-06
checks / unit (pull_request) Canceled after 0s
Two conflicts, both predicted, both mine as the later merger.

app/main.py - #12 inserted `await self.check_output_clamp()` on the line
immediately above `await self.check_meter_frozen()`, so its hunk abuts my
comment correction. Resolved so EACH CALL CARRIES ITS OWN BOUNDARY,
because the two guards do not share one:

  - check_output_clamp() reads `self.clamp_last`, which is None until
    write_setpoint() has run, so its boundary IS the write;
  - check_meter_frozen() grades `self.target`, assigned further up, so
    its boundary is the assignment.

Letting my correction generalise across both would have been worse than
the vague comment it replaces, because it would read as deliberate. The
AST of app/main.py is identical to merged release/1.0 (+1251 bytes, all
comment).

test_freeze.py - #12 independently added `frozen_notes()` keyed on the
same id. `notes_of()` subsumes it: same id, plus the title fallback that
keeps the id's own check independently falsifiable, plus `all_notes_of()`
for claims about the whole surface.

THE THREE SURFACE-WIDE ASSERTIONS, DECIDED ONE AT A TIME. Measured on the
merged tree:

  scenario                              surface  ids             freeze
  frozen meter, railed command             2     frozen + clamp     1
  TEL-05 converged, -14.2 W                0     -                  0
  moving meter, a real 8 kW load           1     clamp              0
  uncovered transport, no unchanged_s      1     clamp              0

  - the wiring scenario STAYS SURFACE-WIDE and now asserts exactly TWO
    alarms with BOTH IDS NAMED. The answer to "is this still meant to
    raise one alarm" is no, it raises two, and a third guard has to come
    back to this line. That is the one assertion in the file that
    accounts for the controller's entire alarm output;
  - the CONVERGED case stays surface-wide and stays silent: at the
    operating point the whole add-on should have nothing to say, that is
    still true, and narrowing it would give away a real property;
  - the other two NARROW to notes_of(). The clamp alarm there is correct
    and is SAFETY-03's subject: a real 8 kW load does drive the loop past
    the rating, and an uncovered transport does not stop it. Asserting
    whole-surface silence would make this file fail whenever another
    guard correctly does its job. Each still owns the load-bearing half -
    THE FREEZE DETECTOR SAID NOTHING - and the surface is PRINTED so a
    reader sees what else fired without this file grading it.

Re-verified on the merged tree: M1 RED 4, M2 RED 5, M3 RED 3 + RED 2,
N6 RED 1, N3 green (correctly), N8 RED 4, R1 RED 2, R2 RED 1. All nine
suites green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 17:45:09 +02:00
admin 6aabb832fc SAFETY-03: alarm when a computed setpoint exceeds the rating, do not just clip it (#12)
The clamp already existed; the alarm did not, and specs 10.3 records a silently clipped integrator that reached 14,768 W while every dashboard read a legal 5000 W. OUTPUT_CLAMP now alarms on every engagement, with its own reason code, notification_id, status row and counter, distinct from ordinary saturation. Clamp lives in write_setpoint(), the one path every write takes; the alarm runs after it, out of the write path. Not a trip: the clamped value is written and the loop keeps running.

Reviewed over two rounds. Round one held the PR on three findings: clamp_to_rating() returned early from the max_w stage so the device bound was never applied when the first stage engaged (the bigger the overshoot, the less the guard did), and two checks that could not fail - range(CLEAR_CYCLES - 1) issued zero clean cycles at CLEAR_CYCLES=1, and REASSERT_S + 9 passed for any value including an hour of silence. All three fixed in 4e66583 and verified independently: both clamp stages now run on symmetric, asymmetric and wider-device ratings with no escapes; the 492-point sweep was shown to bite where the witness values cannot (a positive-side-only device bound gives 181 escapes with both witnesses passing, and a single-grid-point hole gives 1); and the rewritten checks pin the requirement rather than the constant - CLEAR_CYCLES red at 1 and at 51, REASSERT_S red at <=5 and at >=300.

Author's six mutations all reproduced red, including the cross-suite one where removing the event-log coalescing turns test_freeze red - this guard's log had been evicting SAFETY-18's METER FROZEN? row. Headline claim reproduced: keying the alarm on Decision.reason instead of the i_clamped/clamped booleans takes the 10.3 replay from 2 engagements to 0. AC2 verified with an independent saturation-without-clamping scenario: sat_count 10 at 21.6% of the rating, zero clamp engagements.

Two reviewer mutations survive, both recorded and neither blocking: describe() dropping the bound name (the bound field itself is pinned), and abs() on limit_w (unreachable under the int(100,5000) schema).

First merge in this repository to pass through CI-01's gate rather than around it: 46383c5 merged release/1.0 in so the required context could run at all, and checks / unit (pull_request) reports success in 2m32s.

Note for TEST-06 (#14), which rebases behind this: it conflicts in BOTH app/main.py and test_freeze.py, and its 8 red assertions are its own service-keyed notification counting - keying those four call sites on notification_id turns the whole suite green against this app/ with no change here.
2026-08-30 17:22:30 +02:00
glenn schrooyenandClaude Opus 5 d31a12b7c6 TEST-06: the id-selection note said 'the only assertion'; there are two
checks / unit (pull_request) Successful in 2m32s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 17:10:42 +02:00
glenn schrooyen 2983d7c3b0 Merge remote-tracking branch 'origin/release/1.0' into TEST-06
checks / unit (pull_request) Canceled after 0s
2026-08-30 17:05:35 +02:00
glenn schrooyenandClaude Opus 5 611d3dbcdf TEST-06 review follow-ups: two notification surfaces, the real ordering boundary, the schema bounds
1. notes_of() could not tell this guard's alarm from anybody else's. It
   selected by SERVICE, so against #12's app/ (which adds
   goodwe_output_clamp) eight assertions went red, AC1's core check among
   them. Blanket id-keying is not the repair and was measured, not
   guessed:

       selector                    vs #12's app/   id renamed
       by service (as committed)     8 red           1 red
       by id alone                   3 red           6 red
       by id or title (shipped)      3 red           1 red

   all_notes_of() is the whole surface; notes_of() is this guard's,
   recognised by the id it should carry OR the title it writes - the `or`
   is what keeps the one assertion whose subject IS the id able to fail on
   its own. The three surface-wide claims stay surface-wide, each with a
   comment naming which surface it is about and why the next guard's
   author must decide rather than narrow them. They are the three that go
   red against #12, which is the correct signal.

2. The ordering comment named the wrong boundary. N3 - moving the call to
   immediately before write_setpoint() - is green and correctly so:
   self.target is assigned two lines earlier, write_setpoint() clamps only
   a local and can skip the write entirely. So the invariant is "after
   `self.target = resolution.target_w`", and the detector grades pre-clamp
   INTENT, not the wire. Corrected in test_freeze.py and in main.py's own
   note above the call (comment-only; I merge second and own the
   conflict). N8 - moving the call above the assignment - is RED 4.

3. The schema range's own bounds are now pinned. int(60,3600) ->
   int(1,86400) survived green: an operator could set meter_freeze_s:
   86400 and the guard is dead with nothing red. Both bounds are
   re-derived rather than restated.

4. find_hwsim() crashed the whole suite on a checkout one level below the
   root (HERE.parents[1] -> IndexError, reproduced at C:/goodwe_controller).
   Guarded. And part 3 now names its transport in the summary: the rig
   hwsim and the stub are not equivalent - the rig path failed 1 run in 3
   - so a green log has to say which one backed it.

Also: the "revived transport re-rails" check is labelled a PRECONDITION
like its sibling; freeze_rows() records that flipping p1_enabled exercises
the row's content and never its gate; and the note about the deleted
assertion is corrected - it was REDUNDANT, not undecidable. The reviewer
falsified it in one line (move the hass.call above `if not fired: return`,
RED 9). "I could not construct a falsifier, therefore it cannot fail" was
exactly the inference this file exists to distrust.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 17:05:28 +02:00
glenn schrooyenandClaude Opus 5 46383c5ae5 Merge release/1.0 (CI-01) into SAFETY-03 so the required check can run
checks / unit (pull_request) Successful in 2m32s
Branch protection on release/1.0 now requires the context
`checks / unit (pull_request)`, and CI-01's workflow arrived after this
branch was cut - so no run had ever been triggered for this PR and the
required context would have sat pending forever. Clean merge, no
conflicts: .gitattributes, .gitea/ci/Dockerfile and
.gitea/workflows/checks.yml only. No add-on source or check is touched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 16:03:32 +02:00
admin 5e966ca59f CI-01: this repository's first automated check (#13) 2026-08-30 16:00:42 +02:00
glenn schrooyenandClaude Opus 5 4e66583a52 SAFETY-03 review fixes: both clamp stages always run, and two checks that could not fail
1. BEHAVIOURAL, and a regression this PR introduced. clamp_to_rating() returned
early from the max_w stage, so the device bound was applied only when the rating
bound had NOT fired - the bigger the overshoot, the less the guard did. At max_w
5000 against a +-4000 device, +9999 emitted +5000 W, outside the device range,
while +4500 was correctly held to +4000. It undid the pre-PR guarantee "never
send a value the device will reject outright" and undercut this module's own
case for max_w-not-min(), which rests on that stage catching a validation that
did not run. Both stages now always run; the result names both bounds when both
bite (max_w+device). Asserted over a 492-point sweep, because the defect was
invisible to any single-value check sitting inside one stage.

2. The hysteresis check issued range(CLEAR_CYCLES - 1) clean cycles, which at
CLEAR_CYCLES = 1 issues zero, passes, and prints "0 clean cycles do NOT clear
it" - green for the chatter setting the constant exists to prevent. Now: ONE
clean cycle must not clear the alarm, a literal that does not move with the
constant, and the de-assert count is measured by looping until it clears.

3. The re-assert check stimulated at REASSERT_S + 9, green for any value
including 3600 - the hour of silence the module promises not to produce. Now
stated in wall-clock terms: a loop pinned across the 900 s capacity-tariff
quarter must speak at least 5 times and on no more than 20% of cycles. Pinned in
both directions.

Red confirmed for each: the restored early return (3 checks), CLEAR_CYCLES = 1
(1), REASSERT_S = 3600 (1) and REASSERT_S = 0 (3).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 15:52:09 +02:00
glenn schrooyenandClaude Opus 5 c34b5a2922 TEST-06: close the three gaps that survived SAFETY-18's suite green
None of the three is a bug in the shipped code. They are gaps in what the
suite could detect, and the next person to touch freeze.py had less
protection than the passing count implied.

1. WHERE check_meter_frozen() runs in cycle(). Moving it to the top left
   the whole file green. Pinned by CONSEQUENCE, not by source order, so a
   refactor that keeps the semantics stays green:
     - the alarm must land on the SAME cycle the command reaches the rail
       (at the top it grades the previous command against this cycle's
       meter and arrives a cycle late);
     - the dead-poll-task composition: poll task dies, stale_input_s is
       crossed, the arbiter commands 0 W, and on THAT cycle the alarm is
       already clear - because the detector ran after the 0 W write.
   wired() now returns a per-cycle trace, which is what makes "on which
   cycle" an observation rather than a hope.

2. The NOT COVERED status row. The old check only asserted that an
   uncovered transport is SILENT, which a page claiming `armed` on a guard
   that cannot run also satisfies. The string is now asserted in both
   directions: the uncovered row must say NOT COVERED, name the
   meter_source and homewizard_local, and warn; the covered row must still
   say armed, or "does not say armed" would be satisfied by deleting the
   armed branch.

3. meter_freeze_s. Pinned in test_validate.py by MAINT-04's regex
   technique (no PyYAML in the image), and in test_freeze.py by its
   consequences: the shipped value drives the replays, must stay inside
   the 900 s billing quarter, and 3600/hold_s must stay inside the shipped
   deadband. A Controller built with the key ABSENT must land on the same
   number, which pins main.py's fallback without touching CFG-03.

All three demonstrated red before being accepted green, and every new
check was mutated to confirm it can fail; one candidate assertion ("no new
notification while parked at 0 W") was DELETED because no mutation could
break it, and replaced by the exact notification count across a revived
transport, which can.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 15:50:08 +02:00
glenn schrooyenandClaude Opus 5 9c1a23328a CI-01: this repository's first automated check
checks / unit (pull_request) Successful in 2m39s
Before this commit `admin/goodwe-addon` had no `.gitea/workflows/` directory at
all. Eight runnable test files - `test_loop.py` alone drives the real
`run_control()` for 93 s of wall clock - and not one of them had ever been run
by a machine. Every merge here was a human reading a diff, and the product
owner's requirement that a green CI gate a merge could not be met on a
repository with no CI to be green.

One job, `unit`, with no `paths:` filter, so it reports on every PR and can
therefore be a REQUIRED status check: Gitea leaves a required context it never
receives pending forever, which is the failure mode on the other side of this
ticket. Concurrency is scoped per PR from the start rather than to a bare group
name - the bare name is what let the infra repo's run 521 be superseded while
still pending, so its four jobs concluded `cancelled` with `started_at` at the
Unix epoch and its PR merged on no evidence at all.

The tests run in `.gitea/ci/Dockerfile`: `alpine:3` plus `py3-aiohttp` and
`py3-paho-mqtt` from apk, matching goodwe_controller/Dockerfile exactly, because
pip would install glibc wheels of a library the shipped image gets from Alpine.
The runner itself cannot host them - bare python3, no aiohttp, no pip - and
mutating a shared long-lived runner with `apk add` is invisible to anyone
reading this repo.

Two things measured while building it and recorded in the file rather than
learned again: the checkout must sit two levels below the mount root or
test_freeze.py's `find_hwsim()` dies on `IndexError: 1` before its first
assertion, and test_freeze's hwsim section is silently absent here because
admin/goodwe is private - so its PASS is a weaker claim than it looks, and the
file says so.

Verified green on release/1.0 content before this workflow was written: 8 of 8
files pass, ~4 minutes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 15:46:30 +02:00
glenn schrooyenandClaude Opus 5 c040aac4b8 SAFETY-03: alarm when a computed setpoint exceeds the rating, do not just clip it
The clamp was never the missing half. specs.md 10.3 records an unclamped
integrator climbing ~130 W per 4 s past 10 kW and reporting 14 768 W pinned at
the 5 kW rail; clipped silently that is a flat, legal-looking 5000 W hold and
nothing says the loop asked for three times it.

write_setpoint() - the one path every write takes - now clamps to the VALIDATED
max_w and reports whether the raw value differed; check_output_clamp() raises
OUTPUT_CLAMP as a notification, an ERROR, an event row and a red status banner.
Not a trip: the clamped value is written and the loop keeps running.

The rating is max_w and nothing is min()ed at runtime - CFG-01 already refuses a
max_w outside the device range at startup. The device range is applied second as
defence in depth; it cannot bite after a successful validation, so if it ever
does it alarms too and names the bound. No new config key.

Decision grew i_clamped / clamped booleans, and that is the load-bearing part:
`reason` is one string that slew-limiting and the saturation freeze overwrite,
so an alarm keyed on it is silent on exactly the cycles a winding loop produces.
Mutating main.py back onto the string drops the 10.3 replay from 2 engagements
to 0.

Distinct from saturation by construction - no shared reason code, notification
id, status row or counter - and checked in the negative, after first asserting
the saturation freeze really did engage.

Two things found by running it rather than by reading it: a row per engagement
flushed the 40-row event log and evicted SAFETY-18's own METER FROZEN? row (now
coalesced per episode with an exact count), and a naive clamp turns a nan into a
command at the full rating because every comparison against nan is False.

test_clamp.py is new: assert-based, no framework. Six author mutations
reproduced red, including "clamp kept, alarm deleted". Two of the check's own
hypotheses about the freeze/clamp interaction were refused by its sweep and
rewritten to what it measured.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 15:22:22 +02:00
admin 0cdaa3bd3e SAFETY-18: alarm on a meter frozen at a wrong-but-fresh value (#11)
A meter answering 200 OK with a document that never changes reads healthy to every freshness mechanism in the stack while the loop rails. Detected by a conjunction - unchanged_s >= meter_freeze_s AND |command| within 5% of max_w - which fails in both directions for independent reasons. Alarm only: no claim, no latch, no 0 W.

Reviewed: all six author mutations reproduced red (redder than reported), four more added, three survivors recorded for follow-up. Capture figures re-derived independently. Attack point 4 (dead poll task) measured, not just reasoned. Known follow-up: a device rating that clips max_w by >5% silently disarms the rail term while the status page says armed - does not affect the live install, where dev_max == max_w == 5000.
2026-08-30 03:20:38 +02:00
glenn schrooyenandClaude Opus 5 7cde5d862a Merge release/1.0 into SAFETY-18 - CHANGELOG only
MAINT-04 (#10) landed on release/1.0 mid-review, so both branches had
prepended a stanza. config.yaml and DOCS.md auto-merged clean; only the
CHANGELOG needed a hand. Both stanzas are kept verbatim, MAINT-04's
Unreleased above the new 0.5.2 - no text from either author edited.
MAINT-04 is unreleased and 0.5.2 is being cut here, so a release manager
may want to fold the two; that is an editorial call, not a reviewer's.

Full suite green on the merged tree: arbiter, control, loop, maintenance,
p1, validate, watchdog, freeze.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 03:20:26 +02:00
admin b0a17474a3 MAINT-04: refuse a maintenance charge the ceiling cannot deliver (#10)
Rule 6 and the default change verified; four claimed mutations plus six of my own all reproduce red. The cross-file default divergence is now pinned by a check that fails in both directions, including when someone closes it - verified against MaintConfig-only, one-call-site-refactored, both-refactored and one-literal-silently-changed.
2026-08-30 03:01:14 +02:00
glenn schrooyenandClaude Opus 5 ffb8ec6373 MAINT-04 review: pin the default divergence with a check, not a comment
validate.py's comment said "if config.yaml's value moves, move this and
main.py's together" and nothing enforced it - a comment where a check
belonged, in a PR that elsewhere proves it knows the difference.

test_validate.py now reads main.py and maintenance.py directly and asserts
the divergence EXACTLY AS IT STANDS: config.yaml/validate.py at 2000,
main.py's two fallbacks and MaintConfig's dataclass default at 2500. It
records the mismatch rather than demanding it be closed, so it passes today
and goes red if either side moves - including when somebody closes it in
main.py, at which point the check and validate.py's block are deleted
together. Nothing in a file another ticket owns is touched.

The consequence is asserted through the real code as well as the literals:
with the key absent the validator accepts, MaintConfig builds 2500 W against
a 2000 W ceiling, and the arbiter resolves it to 0 W as a contradiction -
the bug rule 6 exists to catch, passing rule 6. Production-unreachable (the
Supervisor always supplies every key) but fully reachable from sim/inject.py
and the tests, which is where anyone debugging maintenance works.

Also: a CHANGELOG section for the upgrade path, covering both the case where
the Supervisor re-derives the default and the case where it persisted the
old one, since which applies was not verifiable without probing the live
install. And maintenance.py's timeout comment said 13.5 kWh where everything
else says 13.8 (300 Ah x 51.2 V, DoD 90); SAFETY-18 does not touch that file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 02:51:40 +02:00
glenn schrooyenandClaude Opus 5 0ba3a200fe SAFETY-18: verify on the real rig, and record what it showed
AC3 says "verified on the rig". The previous commit drove hwsim in-process
because `riglock status` reported UNKNOWN - which is the tool declining to
observe, not an observation. riglock shells out to the local docker CLI; on a
Windows host without DOCKER_HOST that resolves to a Docker Desktop npipe which
is not running, and the result is indistinguishable from a genuinely absent rig.
The rig was up the whole time. With DOCKER_HOST=ssh://homelab@192.168.2.147 it
reads FREE.

Re-run under sim/riglock.py against the real goodwe-sim compose project, with
the shipped modules staged into goodwe-sim-plant so they poll that container's
own hwsim while plantsim drives it. Both phases passed, lock released cleanly.

The rig produced a result the in-process stub could not, and it strengthens the
design rather than merely confirming it: in the NEGATIVE twin - +8000 W of house
load, healthy meter, correctly railed command - `active_power_w` did not move at
all, sitting at exactly 10130.0 W for 26 consecutive polls. The only thing
keeping `unchanged_s` at 0.0 s was the kWh registers advancing.

So the register terms in measurement_fingerprint() are not belt-and-braces for
this detector, they ARE the negative case: fingerprinted on the power figure
alone it would have raised a false alarm on a healthy meter under a real load,
which is precisely how a safety guard gets switched off. Recorded in freeze.py
so nobody simplifies the fingerprint without meeting that run first.

No behaviour change - documentation and the harness note only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 02:49:55 +02:00
glenn schrooyenandClaude Opus 5 7aa1bd052e SAFETY-18: a meter frozen at a wrong-but-fresh value now raises an alarm
A meter that answers 200 OK with a stale document forever is ARRIVING on every
poll, so every freshness mechanism in the stack reads it as healthy while the
loop integrates a constant error and rails: 360/135/75 s at the live install's
max_w: 5000 for a meter frozen at 350/1000/>=3000 W. Fifteen minutes of a 5 kW
unattended import sets the month's billed capaciteitstarief peak.

The detector is a conjunction, not the bare unchanged_s threshold TEL-05
measured and rejected:

    unchanged_s >= meter_freeze_s   AND   |command| within 5 % of max_w

Both terms already existed and were already on the status page, and they fail
for independent reasons: at the converged point the command is nowhere near the
rail, and at the rail against a real load the kWh registers move in seconds. At
the shipped 300 s the relation is structural rather than a margin - the slowest
a healthy 1 Wh register can tick and still look unchanged is 12 W, which is
inside the 15 W deadband, so the command there is parked at 0 W.

Notification only, deliberately: an HA persistent notification with a fixed
notification_id, an ERROR log line, a UI event and a red status row. No claim,
no latch, no 0 W. A false trip parks the battery; a false notification costs an
alert. Only meter_source: homewizard_local is covered - the status page says
NOT COVERED elsewhere rather than pretending the guard is armed.

DOCS.md states the capacity-tariff exposure this does NOT close: peak_cap_w caps
maintenance charging and nothing caps the loop. That cap is SAFETY-07.

test_freeze.py grades both directions and was verified by breaking what it
watches (six mutations, all red): silent over the committed 40-minute converged
capture, silent over an hour of TEL-05's low-power case where the rejected bare
threshold is shown to fire, alarming on a genuinely frozen meter driven over
real HTTP through the real HomeWizardLocalSource (the rig's own hwsim freeze
fault when reachable), silent on the same railed command against a meter that is
still moving, and asserting on the service call that leaves Controller.cycle()
rather than on the watchdog's own flag.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 02:39:54 +02:00
glenn schrooyenandClaude Opus 5 878a6ce452 MAINT-04: refuse a maintenance charge the ceiling cannot deliver
config.yaml shipped max_w 2000 against maintenance_charge_w 2500. The
cheap-window shaping files Claim.limit(hi = -maintenance_charge_w) against
the supervised SAFETY limit's lo = -max_w, so lo > hi - the arbiter's
clause 3, correctly resolved to 0 W and logged as a CONTRADICTION.
Maintenance silently never charged in a cheap window on a default install,
and the only evidence read like a bug in the arbiter rather than like the
two numbers the operator set.

validate_startup_config() now refuses maintenance_charge_w > max_w, the
same shape as CFG-01's sign_disagree_w >= max_w. Equality is reachable
(-max_w is a single valid point) and is not refused. Demonstrated by
starting the add-on on the pre-fix pair: exit 78 and the refusal banner.

maintenance_discharge_w was checked rather than assumed and is NOT the same
case: the drain phase OWNS the actuator (Claim.set at P_MAINTENANCE), so
the ceiling clamps it to +2000 W and the arbiter's explain() names the
clamp every cycle. Loud, and it still moves energy - warned, not refused.

The shipped maintenance rates drop to 2000 so the defaults pass the rule;
max_w was deliberately not raised, because it is the supervised safety
ceiling. test_validate.py now reads config.yaml itself, so a default the
validator refuses cannot ship again.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-30 02:28:09 +02:00
admin bb64f8fdd7 SAFETY-02 follow-up: persist the latch, and make the gate assertion bite (#9) 2026-08-28 23:09:27 +02:00
glenn schrooyenandClaude Opus 5 e40eb0768d SAFETY-02: the P_PEAK assertion was pinning the test's copy of the gate
Review finding 2, and it is the same duplication `disagrees()` just removed, so
it gets the same fix rather than a follow-up card.

`following_from()` in test_watchdog.py was a TRANSCRIPTION of the gate in
main.py, carrying "keep the two in step" as an instruction - which is exactly the
arrangement that does not keep two things in step. Measured: editing the shipped
gate alone to `winner in ("loop", "peak")` left every suite green while the
watchdog went blind for the duration of every peak event. The row whose whole job
is to force whoever lands peak shaving to come and change an expectation did not
do that.

The expression is now `watchdog.loop_is_following()`, called by main.py and by
the test. One definition, two callers, same reasoning as `disagrees()`: the copy
that drifts is always the one in the harness doing the grading.

⚠️ AND EXTRACTING IT WAS NOT SUFFICIENT - the drift proof still came out green
after the extraction, which is why it was worth running rather than assuming. The
realistic P_PEAK row has peak shaving asking for MORE than the loop, so the
magnitude term rejects it whatever `winner` says and masks the term under test. A
second row with peak asking for LESS makes `winner` the only deciding term. With
that row, widening the shipped gate fails two checks; without it, none.

Also enforcing the write ordering that was documented and not asserted: swapping
the two `store.set` calls passed everything. A store that fails the second write
now proves the property the ordering exists for - a crash mid-trip leaves a
reason with no latch, never a latch with no explanation. Swapping them now turns
that check red.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Du77usMj8XNKNFZGmUiWDa
2026-08-28 22:59:46 +02:00
glenn schrooyenandClaude Opus 5 cc4e88490b SAFETY-02 review: a latch that a restart clears is not a latch
BLOCKER. `auto` was persisted and `sign_wd.tripped` was not, so reconstructing
the Controller against the same store - which is what a container restart does -
brought it back ARMED, into the same fault, unattended: tripped=False, banner
back to NOT READY, and it resumed commanding -1000 then -2000 W before
re-tripping 180 s later. "Restart the add-on" is the first thing an operator
looking at a FAILSAFE banner and a 0 W inverter will try, and under a supervisor
that restarts a crashed add-on it is a loop of three-minute wrong-direction
bursts. DOCS.md's "it does not clear itself" was false across a restart.

`sign_tripped` and `sign_reason` are Store keys now, beside the maintenance
state and for the reason store.py's own docstring gives. Written the moment the
trip happens - reason first, flag second, so a crash between the two can never
persist a latch with no explanation - restored in Controller.__init__, cleared
only by the operator's start. The regression is covered: reverting the restore
fails seven checks in test_watchdog.py.

Review follow-ups, no behaviour change:

- The arming gate is tested against the REAL arbiter. It is the decision the
  ticket turns on and it was asserted in three comment blocks and verified by
  nothing - every case passed `following` in as a literal. Twelve claim
  configurations now derive it from resolve(), including the redirect case
  `winner == "loop"` alone would have got wrong and the `lo` limit that raises
  magnitude without flipping sign.
- The MQTT `status: "failsafe"` value is tested. The check that claimed to cover
  it restated the line above it and never touched the payload.
- The worked example described a config that does not ship: at the defaults
  maintenance_charge_w 2500 against max_w 2000 is an arbiter contradiction and
  0 W, not a -2500 W charge. Rewritten at 1500 W. The contradiction is a real
  pre-existing bug - a default cheap-window maintenance charge silently never
  happens - filed separately and deliberately not fixed here.
- Forward hazard recorded at the gate: `winner == "loop"` disarms the watchdog
  under any higher-priority SET, and a peak-shaving strategy at P_PEAK IS
  grid-following in sign, so it would blind this for every peak event.
- disagrees() is public so the replay harness stops carrying its own copy of the
  predicate it grades.
- held_s takes the clock instead of reading one.
- The two limits are now each stated in both places rather than one each.
- The no-false-positive argument is tied to control.py's freeze permitting the
  command to cross zero; under the shrink-only encoding control.py records as
  rejected, this would false-trip on a load step.
- Measured figures corrected: 93.4 % / 4212 of 17280, per-cluster
  0.3/0.9/10.5/0.0, and the near-circularity of those figures is now stated
  rather than leaned on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Du77usMj8XNKNFZGmUiWDa
2026-08-28 22:33:15 +02:00
admin 2513ba1f12 SAFETY-02: the sign-disagreement watchdog (#8) 2026-08-28 21:55:52 +02:00
glenn schrooyenandClaude Opus 5 9aed43990b SAFETY-02: a loop can be fresh, tracking and inside its rating and still push the wrong way
The stale-input watchdog asks whether the meter is still talking. The saturation
freeze asks whether the inverter can follow. The output clamp asks whether the
number is too big. A loop can pass all three while charging the battery out of
the grid, because the only thing wrong is the sign.

On 2026-08-20 a commercial controller did exactly that four times in one day,
for up to 77 minutes at a stretch, reaching a commanded -14 768 W against a
±5000 W inverter. app/watchdog.py compares the emitted setpoint against the
authoritative P1 figure and, when they oppose each other with both over 300 W
for 180 s continuously, files a failsafe SET claim at P_SAFETY, commands 0 W and
latches there until an operator starts control again.

Validated against that day rather than against an argument: sim/replay_sign.py
in the infra repo drives this module over the committed captures of both
2026-08-20 and the healthy 2026-08-23 and reaches 18 trips on the fault day -
every one inside a window NOTES.md §4.7 accounts for - and 0 on the healthy day.
The four episodes do share one signature: 0.3 / 0.9 / 10.5 / 0.0 % sign
agreement inside them against 92.9 % healthy.

Three decisions the ticket could not make:

- It is DISARMED whenever the grid-following loop is not what is driving. Sign
  agreement is a property of a loop that follows the meter, and a maintenance
  grid charge opposes the meter by design - 2500 W of charge into a house
  drawing 500 W is +3000 W on the meter against a -2500 W command, for hours.
  The gate tests the emitted number, not the winning claim's name, because a
  limit claim can let the loop "win" and still replace its value.
- The trip LATCHES. Commanding 0 W removes the command side of the comparison,
  so the disagreement always clears the moment the trip works.
- sign_disagree_w at or above max_w is refused at startup. The arbiter never
  lets a command exceed max_w, so such a threshold leaves the watchdog incapable
  of firing while the status page still reports it armed.

What it does not catch, said out loud: an inverter that ACTS opposite to its
command moves the meter further the same way, leaving command and meter in
agreement. That is the saturation freeze's job.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Du77usMj8XNKNFZGmUiWDa
2026-08-28 21:51:35 +02:00
admin 35856a73ae SAFETY-12: a loop that only thinks when the number changes cannot fail safe (#7) 2026-08-28 21:18:20 +02:00
glenn schrooyenandClaude Opus 5 aae74d7571 SAFETY-12 review: name the safety claim that could not fire, and fix two figures
Review fixes.

A. test_loop.py takes 73 s, not the ~35 s claimed in three places. That number
is the whole defence against a future developer deleting a slow test, so it was
understated by 2x in exactly the place it matters. Corrected - and it is now
~95 s (measured 93 s), because this commit adds the scenario below rather than
trimming windows to make an inconvenient number smaller. Both call sites carry
an instruction to re-measure if the windows change.

B. "242 checks" was test_p1's own count quoted as the total for all five
suites. The real figure is 395 across the five, plus 14 here.

And the finding this PR should have led with. stale_input_s and the
`inputs missing -> 0 W` claim both live INSIDE cycle(), so pre-fix, on a meter
whose value happened to repeat, that claim could not fire at all. Constant
meter, SoC entity dying at t=4 s, one binary:

  pre-fix   final target 220.0 W, reason 'loop -> 220 W'
  post-fix  final target   0.0 W, reason 'safety -> 0 W'

The old controller held 220 W through the whole input outage with its status
line still reading loop -> 220 W, and layer 1 does not cover that on a
`meter_source: off` install - no P1 age sensor is published, so the firmware has
nothing to veto on, while the heartbeat keeps its watchdog fed. The pre-fix
behaviour was not "holds a stale command safely"; it was a failsafe a repeating
meter switched off. Asserted in test_loop.py, both halves.

Three points of reasoning tightened, none of them wrong before:

- the re-tune is unreachable by construction, not merely by the reset: the
  combined cycle rate is max(arrival rate, 1/floor), never the sum, and 1/15 s
  is strictly slower than the ~1/5 s arrival rate. That holds only while the
  floor exceeds the meter's update interval - and the schema minimum of 5 s is
  exactly the reference meter's interval. Flagged for validate.py, not fixed
  here.
- the ~11 s lower bracket is not only a dropped poll: a 5.0 s poller against a
  4.97 s meter beats, so about every 14 minutes the poller steps over an update
  entirely. Reaching 15 s takes a beat-skip AND a failed poll.
- the "freshness is decided upstream" guarantee is transport-conditional. On
  `meter_source: off` the grid figure comes straight from hass.number() with no
  age gate. Running the full law is still right; the claim is narrower than it
  was stated.

The 15 s default is not independently justified - it rests on the firmware's
30 s ingest veto and moves if that ever does. Said beside the constant.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Du77usMj8XNKNFZGmUiWDa
2026-08-28 21:14:58 +02:00
glenn schrooyenandClaude Opus 5 8cc9438aec SAFETY-12: record the six-minute observation, not just the unit check
test_loop.py proves the property in 35 s. This is the same three scenarios run
for six and three real minutes each, on one binary, and it is what AC2 and AC3
were actually asked for: 1 cycle before vs 24 after on a constant meter with the
command stuck at 220 W against a 350 W import, and byte-identical write
sequences on both the converged and the normally-changing meter.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Du77usMj8XNKNFZGmUiWDa
2026-08-28 20:48:49 +02:00
glenn schrooyenandClaude Opus 5 96a808a057 HANDOVER: run every suite before deploying, and say why one is slow
The pre-deploy command listed three of the five suites; test_validate and
test_p1 were never added when they arrived. Adds those and test_loop, with a
note that test_loop's ~35 s is the cost of proving a wall-clock property about a
loop that sleeps 1 s a tick rather than a sign something is wrong.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Du77usMj8XNKNFZGmUiWDa
2026-08-28 20:32:00 +02:00
glenn schrooyenandClaude Opus 5 88cd97cc7a SAFETY-12: the loop thinks on a clock, not only when the number moves
run_control() ran cycle() only when the meter value differed from the previous
poll. A meter that repeats a value therefore ran no control cycles at all -
while the heartbeat kept re-emitting the last command and the firmware kept
putting it on the wire every 2 s. Layer 1 stayed fed and every indicator read
healthy, so a command computed from an old reading went on being executed
through whatever the load did next. TEL-05 measured it: 70 s at a literally
constant 350.0 W, 14 arrivals, zero cycles.

cycle_floor_s (default 15 s) is a deadline on the GAP between cycles, reset by
every cycle whichever trigger fired - not a second scheduler. On a healthy meter
it never elapses, so the cycle rate and the commanded sequence are unchanged;
test_loop.py asserts that by driving the real loop twice on one binary (a huge
cycle_floor_s reproduces the old behaviour) and diffing the writes.

15 s is bracketed: above the ~11 s worst gap on a merely-unlucky healthy meter
(~5 s arrivals, 1 s detection, one dropped poll retried on the next tick), below
meter_max_age_s and the firmware's 30 s ingest veto, and equal to stale_input_s.

A floor cycle runs the full law. Freshness is already decided upstream - past
meter_max_age_s the reading is None, which is a P_SAFETY 0 W claim - and the
integrator advances per cycle, not per second, so bounding the gap bounds the
accumulation.

Not a freeze detector: nothing here decides whether a repeated reading is true.
DOCS.md's "accepted deviation" passage is qualified rather than deleted - the
window is now bounded at 3 x cycle_floor_s instead of open-ended, and its
"nothing accumulates while it is stalled" argument is called out as having
covered windup and not staleness. Same correction in control.py's comment.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Du77usMj8XNKNFZGmUiWDa
2026-08-28 20:29:16 +02:00
admin e2e331ea54 CFG-01 AC2: refuse to start on a config the lower layer will silently override (#6) 2026-08-28 20:17:55 +02:00
21 changed files with 5115 additions and 52 deletions
+8
View File
@@ -8,3 +8,11 @@
* text=auto eol=lf
*.png binary
*.gz binary
# `.gitea/workflows/*.yml` steps embed shell heredocs; a `\r` on the terminator
# line means the heredoc never closes and the job fails on a syntax error that is
# invisible in the diff. A Windows checkout with core.autocrlf=true would
# reintroduce it on the next edit. CI-01.
*.yml text eol=lf
*.yaml text eol=lf
Dockerfile text eol=lf
+22
View File
@@ -0,0 +1,22 @@
# The image the add-on's tests run in (CI-01). This repo had NO CI at all before
# that ticket - eight runnable test files, none of them ever run by a machine.
#
# ⚠️ THE DEPENDENCIES ARE THE SHIPPED ONES, FROM APK, AND THAT IS THE POINT.
# goodwe_controller/Dockerfile installs `py3-aiohttp py3-paho-mqtt` from Alpine
# packages rather than pip, because Alpine ships musl and there are no musl
# wheels for aiohttp - pip would compile it from source on a client's Raspberry
# Pi. Testing against pip-installed glibc wheels would be testing a different
# add-on from the one that ships. Same base, same package manager, same names.
#
# ⚠️ NOT `FROM ghcr.io/home-assistant/...-base`. The real Dockerfile takes its
# base from the Supervisor's `BUILD_FROM` build-arg, which is per-architecture
# and not resolvable outside a Supervisor build. `alpine:3` is what those images
# are, minus the s6 supervision the tests do not use - and it is already cached
# on this runner's host, so the gate does not depend on a registry pull.
#
# ⚠️ The runner container itself CANNOT run these tests: it is Alpine with a bare
# python3, no aiohttp, no pip (`No module named pip`; checked on the box, not
# assumed). An image is not gold-plating here, it is the only option that does
# not mutate a shared long-lived runner.
FROM alpine:3
RUN apk add --no-cache python3 py3-aiohttp py3-paho-mqtt
+136
View File
@@ -0,0 +1,136 @@
# CI-01: this repository's FIRST automated check. Before this file, admin/goodwe-addon
# had no `.gitea/workflows/` directory at all - eight runnable test files, ~100 s
# of real coverage including a wall-clock cadence test, and not one machine ever
# ran them. Every merge here was a human reading a diff.
#
# ⚠️ NO `paths:` FILTER, AND ITS ABSENCE IS LOAD-BEARING. Branch protection on
# release/1.0 requires the context `checks / unit (pull_request)`. Gitea leaves a
# required context it never receives PENDING forever, so the moment this workflow
# learns to sit a PR out, that PR becomes unmergeable. Condition a STEP if you
# must; never the workflow.
#
# ⚠️ THE RUNNER IS SHARED WITH THE OTHER REPO'S RIG. `goodwe-runner` on
# 192.168.2.147 carries the labels `self-hosted` and `goodwe-rig` and is
# registered `:host`, and it is the only runner there is - so this job wears the
# rig label without wanting a rig. Nothing below starts, stops, or reads the
# `goodwe-sim` compose project or takes `sim/riglock.py`; it runs one short
# container and exits. It does occupy the runner's single slot (`capacity: 1`),
# so it QUEUES behind a rig campaign - a delayed verdict, never a destroyed one.
#
# NO JS ACTIONS: the runner image has no node, so `actions/checkout` cannot run.
# Checkout is seven lines of git below.
name: checks
on:
pull_request:
branches: [release/1.0, master]
workflow_dispatch:
concurrency:
# ⚠️ PER PULL REQUEST, NOT A BARE NAME. A shared group holds one running and
# one pending run, so a bare name lets a second PR supersede a first PR's run
# before any runner has picked it up - which is exactly what happened to the
# other repo's run 521 (four jobs, `started_at` at the Unix epoch,
# `runner_id: 0`, concluded `cancelled`, and the PR merged anyway). Do not
# replace this with a constant.
group: addon-checks-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
unit:
runs-on: [self-hosted, goodwe-rig]
timeout-minutes: 70 # ~4 to run; the rest is queueing for the single slot
env:
# ⚠️ UNDER /home/homelab, NOT $GITHUB_WORKSPACE (ENV-01, and re-measured
# during CI-01). act's workspace is `/root/.cache/act/<id>/hostexecutor`, a
# path that exists only INSIDE the runner container. Hand it to the host's
# snap-confined dockerd and the build fails with "failed to read
# dockerfile", or - worse, and this is the documented case - a bind mount
# resolves to an EMPTY DIRECTORY with no error at all. /home/homelab is
# mirrored 1:1 into the runner and is under $HOME, where snap confinement
# accepts it. The same probe under /tmp failed and under $HOME succeeded.
WS: /home/homelab/ci/addon-checks-${{ github.run_number }}
steps:
# ⚠️ THE CHECKOUT IS TWO LEVELS BELOW THE MOUNT ROOT, DELIBERATELY.
# test_freeze.py's find_hwsim() reads `HERE.parents[1] / "goodwe" / "sim"`,
# looking for the infra repo as a SIBLING of this one. Check out flat -
# goodwe_controller directly at the mount root - and `parents[1]` does not
# exist, so the test dies on `IndexError: 1` before its first assertion.
# Measured during CI-01; the flat layout was tried first and did exactly
# that. $WS/goodwe-addon/goodwe_controller gives parents[1] == $WS.
- name: checkout into a path the host shares (git; no node for actions/checkout)
env:
URL: ${{ github.server_url }}/${{ github.repository }}.git
TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -eu
rm -rf "$WS" && mkdir -p "$WS/goodwe-addon" && cd "$WS/goodwe-addon"
git init -q .
git remote add origin "$(echo "$URL" | sed "s#://#://x-access-token:${TOKEN}@#")"
git fetch -q --depth 1 origin "$GITHUB_REF" || git fetch -q --depth 1 origin "$GITHUB_SHA"
git checkout -qf FETCH_HEAD
echo "checked out $(git log --oneline -1)"
- name: build the test image (shipped deps, from apk - see .gitea/ci/Dockerfile)
run: |
set -eu
docker build -q -t goodwe-addon-checks:1 "$WS/goodwe-addon/.gitea/ci" >/dev/null
docker run --rm goodwe-addon-checks:1 python3 -c \
'import aiohttp, paho.mqtt.client as m, sys; print("python", sys.version.split()[0], "aiohttp", aiohttp.__version__)'
# ⚠️ THE FILE LIST IS DERIVED FROM THE TREE, NOT TYPED HERE. A hand-written
# list is how a new test file joins the repo and is never run - the exact
# shape of failure this ticket exists to close. The step globs `test_*.py`,
# prints the count, and REFUSES A COUNT OF ZERO: a loop over an empty glob
# exits 0 and would report a green gate that examined nothing.
#
# ⚠️ test_loop.py TAKES ~93 SECONDS OF WALL CLOCK ON PURPOSE (its own
# docstring says so, and says why: it drives the real `run_control()`
# against a real wall-clock floor, and mocking the clock would leave the
# assertion testing the mock). It is not hung. Do not "fix" it with a
# shorter timeout - a 60 s limit was tried during CI-01 and reported
# exactly the false hang the docstring predicts.
#
# ⚠️ WHAT THIS RUN DOES NOT COVER, stated because a pass here would
# otherwise over-claim. test_freeze.py's third section wants the infra
# repo's `sim/hwsim.py` to drive a real HTTP meter through the real
# transport. admin/goodwe is PRIVATE (checked via the API) and this
# workflow's GITHUB_TOKEN is minted for THIS repository only, so hwsim is
# absent here and `find_hwsim()` returns None. The rest of the file runs
# and passes. It does so SILENTLY - the file prints nothing to say the
# hwsim section was skipped - so "test_freeze.py: PASS" below is a weaker
# claim than it looks. That belongs to whoever owns test_freeze.py next;
# naming it here is the least this gate can do about it.
- name: the add-on's own tests
run: |
set -eu
docker run --rm -v "$WS:/w:ro" goodwe-addon-checks:1 sh -eu -c '
cd /w/goodwe-addon/goodwe_controller
set -- test_*.py
if [ "$1" = "test_*.py" ]; then
echo "NO test_*.py FOUND - the mount is empty or the layout moved. NOTHING has been checked."
exit 1
fi
echo "$# test file(s) to run"
rc=0
for f in "$@"; do
start=$(date +%s)
if python3 "$f" > /tmp/out 2>&1; then
echo "PASS $f ($(( $(date +%s) - start ))s)"
else
echo "FAIL $f ($(( $(date +%s) - start ))s)"
tail -40 /tmp/out
rc=1
fi
done
exit $rc
'
- name: clean up
if: always()
run: |
rm -rf "$WS" || true
# Root-owned by this runner inside a directory the `homelab` user owns,
# and sudo on that box is interactive-only, so the user cannot clean
# this up and the job that made it has to.
ls -1dt /home/homelab/ci/addon-checks-*/ 2>/dev/null | tail -n +6 | xargs -r rm -rf || true
+32 -1
View File
@@ -52,9 +52,40 @@ Run the checks before deploying. No framework needed:
```bash
cd goodwe_controller
python3 test_control.py && python3 test_maintenance.py && python3 test_arbiter.py
python3 test_control.py && python3 test_maintenance.py && python3 test_arbiter.py \
&& python3 test_validate.py && python3 test_p1.py && python3 test_watchdog.py \
&& python3 test_loop.py
```
`test_watchdog.py` covers SAFETY-02's logic, its arming gate (derived from the
real arbiter, not passed in), its trip path, and the fact that the latch survives
an add-on restart. Its headline validation is not here at all: `sim/replay_sign.py`
in the infra repo drives the same module over the two captured RS485 days and must
fire on all four of the 2026-08-20 episodes and stay silent on 2026-08-23. That
needs both checkouts:
```bash
GOODWE_ADDON_REPO=/path/to/goodwe-addon python3 sim/replay_sign.py
GOODWE_ADDON_REPO=/path/to/goodwe-addon python3 sim/replay_sign.py --stub
```
The second run checks the grader still *discriminates* rather than merely still
passing — a watchdog that never trips and one that always trips must both be
rejected.
⚠️ **If you are testing the failsafe by hand, note that stopping and starting the
add-on will not clear a trip.** That is the point of it; use the *Start / stop
control* button twice, or delete `sign_tripped` from `/data/state.json`.
⚠️ `test_loop.py` takes about **95 seconds** (measured 93 s) and that is not a bug. It
drives the real control loop, which sleeps 1 s a tick, eight times over, to prove a
wall-clock property about it (SAFETY-12); mocking the clock there would leave the
assertion testing the mock. The other six suites together are ~440 checks and finish in
seconds — `test_watchdog.py` spends ~3 s of that driving a real `Controller` through a
trip. If you shorten `test_loop.py`'s windows, re-measure and correct this figure —
an understated runtime is what makes a slow test look like a hung one, and a slow test
that looks hung gets deleted.
---
## 3. Known broken / unfinished
+461
View File
@@ -1,5 +1,351 @@
# Changelog
## Unreleased
**A computed setpoint over the rating now alarms instead of being clipped in
silence (SAFETY-03).** The clamp was never the missing half - specs.md 10.3
records an unclamped integrator climbing ~130 W per 4 s past 10 kW and reporting
**14 768 W** while pinned at the 5 kW rail, and a silent clamp turns exactly that
into a flat, legal-looking 5000 W hold. `write_setpoint()` - the one path every
write takes, cycle, heartbeat, operator button and shutdown alike - now clamps to
the **validated** `max_w` and reports whether the raw value differed, and
`Controller.check_output_clamp()` raises `OUTPUT_CLAMP`: a persistent
notification (`goodwe_output_clamp`), an `ERROR`, an event row and a red status
banner. **Not a trip**: the clamped value is written and the loop keeps running,
per 11.2's "clamp *and alarm*".
**The bound is `max_w` and nothing is `min()`ed at runtime.** Section 14 requires
the loop clamp and the driver's `setpoint_limit_w` to agree, validated at startup
with a refusal on mismatch - which CFG-01 already ships. The device range is
still applied second, as defence in depth; after a successful validation it
cannot engage, so if it ever does, that is louder than the first stage and it
alarms too, naming the bound. No new config key: the guard's limit is the one
the operator already set, and its cadence is a module constant.
**`Decision` grew `i_clamped` / `clamped` booleans, and that is the load-bearing
part.** `reason` is one string that later stages overwrite - a cycle that clamps
the integrator and is then slew limited reports `slew-limited`, and one that
clamps while saturated reports `saturated-freeze`. Both *are* the winding case,
so an alarm keyed on the string is silent on exactly the cycles it exists for:
mutating main.py back onto `"clamp" in decision.reason` drops the 10.3 replay
from 2 engagements to **0**. The flags are set where the clamp happens and are
never overwritten. On a default install (`integrator_max_w` following `max_w`)
the integrator bound is the one that bites and the output clamp is unreachable,
so the integrator flag is the whole signal.
**Distinct from the saturation freeze, and checked in the negative.** A clamp is
a *software* bug signal; saturation is the *hardware* failing to follow a legal
command, and its action is to freeze the integrator, not to alarm. They share no
reason code, notification id, status row or counter. `test_clamp.py` drives a
loop that saturates for 10 consecutive cycles at 1.4 % of the rating and asserts
zero clamp notifications, zero engagements, no log row and no status row - after
first asserting that the saturation freeze *really did* engage, so the silence is
an observation rather than a scenario that did nothing.
**The action log is coalesced per episode, and that was a real bug.**
`log_event()` keeps 40 rows; one row per engagement flushed the entire ring
within 40 cycles of a winding loop, and the first thing it evicted was
SAFETY-18's own `METER FROZEN?` row - this guard silencing another guard's log.
Caught by `test_freeze.py` going red, not by inspection. One row per episode now,
carrying the exact engagement count read off the counter.
**A non-finite setpoint is refused, and it was worse than passing one through.**
`nan` is loss of input (11.1 rule 1: command zero, keep commanding), not an
out-of-range value. Measured: because every comparison against `nan` is False, a
naive `max(-limit, min(limit, nan))` keeps its *first* argument and turns a `nan`
into a command **at the full rating**, then reports it as a clamp engagement.
**Review fixes (PR #12).** Three, and the first was a behavioural regression
this PR introduced: `clamp_to_rating()` **returned early** from the `max_w`
stage, so the device bound - the "applied second, defence in depth" half - was
reached only when the rating bound had NOT fired. The bigger the overshoot, the
less the guard did. At `max_w` 5000 against a +-4000 device, +9999 emitted
**+5000 W, outside the device range**, while a mere +4500 was correctly held to
+4000. That undid the pre-PR guarantee "never send a value the device will
reject outright" and undercut this module's own case for clamping to `max_w`
rather than a runtime `min()`, which rests entirely on that stage catching a
validation that did not run. Both stages now always run, the result names both
bounds when both bite (`max_w+device`), and the property is asserted over a
492-point sweep rather than on witness values - the defect was invisible to any
single-value check that happened to sit inside one stage. No reachable
production path was found for it (two validation gates and the arbiter bound sit
in front), but latent is not harmless.
Two harness defects, both of the kind where a check cannot fail for the value it
forbids: the hysteresis check issued `range(CLEAR_CYCLES - 1)` clean cycles,
which at `CLEAR_CYCLES = 1` issues **zero** and passes while printing
"0 clean cycles do NOT clear it" - green for the chatter setting the constant
exists to prevent. And the re-assert check stimulated at `REASSERT_S + 9`, green
for **any** value including 3600, i.e. the hour of silence the module promises
not to produce. Both are now written in wall-clock terms that do not move when
the constant does: **one** clean cycle must not clear the alarm, and a loop
pinned across the 900 s capacity-tariff quarter must speak at least 5 times and
on no more than 20 % of cycles. `CLEAR_CYCLES = 1`, `REASSERT_S = 3600` and
`REASSERT_S = 0` each go red.
**`test_clamp.py`** (new, assert-based, no framework) covers the clamp
arithmetic and its boundaries - exactly +-`max_w` is *not* an engagement - the
alarm's edge/re-assert/hysteresis cadence, the 10.3 runaway replayed through the
real `Controller.cycle()` against an unclamped transcription, the saturation
negative, and the direct-write path. Six author mutations reproduced red,
including "clamp kept, alarm deleted" (the historical build) and "alarm keyed on
`reason`". Two of the check's own hypotheses about how the freeze and the clamp
interact were **refused by the sweep** and rewritten to what it measured: a
frozen cycle *can* clamp, and does so exactly when the integrator reverses.
`test_freeze.py`'s notification assertions now filter by `notification_id`
rather than by service - the add-on raises more than one kind of notification,
and its railing scenarios legitimately raise this one too.
**Cheap-window maintenance charging was unreachable at the shipped defaults, and
now the config that makes it unreachable is refused (MAINT-04).** `config.yaml`
shipped `max_w: 2000` against `maintenance_charge_w: 2500`. The cheap-window
shaping files `Claim.limit(hi = -maintenance_charge_w)` against the supervised
SAFETY limit's `lo = -max_w`, so `lo > hi` — the arbiter's clause 3, correctly
resolved to **0 W** and logged as a CONTRADICTION. **Maintenance silently never
charged in a cheap window on a default install**, and the only evidence read
like a bug in the arbiter rather than like the two numbers an operator set.
Measured against the real arbiter, loop asking +500 W:
```
max_w=2000 hi=-2500 -> +0 W contradiction=True
max_w=2500 hi=-2500 -> -2500 W limited by supervised(max_w)
max_w=3000 hi=-2500 -> -2500 W limited by maintenance(cheap-window charge)
```
`validate_startup_config()` now refuses `maintenance_charge_w > max_w` (rule 6),
the same shape as CFG-01's `sign_disagree_w >= max_w`: a number the operator set,
overridden by a layer they cannot see. Equality is reachable (`-max_w` is a
single valid point) and deliberately not refused. Verified by starting the
add-on on the pre-fix pair: exit 78, the refusal banner, no traceback.
**`maintenance_discharge_w` was checked, not assumed, and it is NOT the same
case.** The shaping is not symmetric. The drain phase does not shape the loop,
it OWNS the actuator — `Claim.set(P_MAINTENANCE, +discharge_w)` — and a SET is
*clamped* by the limits above it rather than contradicted by them. 2500 against
a 2000 W ceiling emits **+2000 W**, and the arbiter's own `explain()` prints
`maintenance -> 2000 W, limited by supervised(max_w)` every cycle. Nothing is
silent and energy still moves; the cycle completes slower (13.8 kWh usable at
2000 W is ~6.9 h against the 720 min drain timeout). **Warned about, not
refused.**
**The shipped defaults changed so the validator does not reject them.**
`maintenance_charge_w` and `maintenance_discharge_w` are now **2000**, at the
shipped `max_w`. `max_w` was deliberately NOT raised instead: it is the
supervised safety ceiling and raising a default upward ships more power to every
new install. Raising `max_w` (the reference install runs 5000) is what unlocks a
faster maintenance cycle — raise it first, then these. `test_validate.py` now
reads `config.yaml` itself and asserts the shipped numbers pass the rule, so a
default that the validator refuses cannot ship again.
### ⚠️ Upgrading an existing default install: the add-on may refuse to start
An install still carrying `maintenance_charge_w: 2500` against `max_w: 2000`
now meets rule 6 and **exits 78 with the refusal banner** instead of starting.
That is the intended behaviour — the configuration was already broken, it was
just silent about it — but an add-on that will not come back after an update
needs the fix stated here rather than only in the log.
**The fix, either way: set `maintenance_charge_w` to `2000` or lower, OR raise
`max_w` to `2500` or higher** (Settings → Add-ons → GoodWe RS485 Controller →
Configuration), then start it. The banner in the log says the same thing and
names both numbers.
Whether you hit this at all depends on whether the Supervisor kept the old
value, and **both cases are covered above because we did not verify which one
applies here.** The Supervisor merges an add-on's *stored* options over the
defaults in `config.yaml`, so:
- **A key the operator never edited** re-derives from the new `config.yaml` and
becomes 2000 on upgrade. Non-event, nothing to do.
- **A key the operator did set** persists at 2500 and the add-on refuses to
start until it is changed. Note this includes anyone who used the
Configuration page's YAML editor, which writes the whole options block back —
so "I never touched that field" is not a guarantee the old value is gone.
⚠️ **This is reasoned from the Supervisor's documented merge behaviour, not
measured on the live install.** The reference install's stored options could not
be read (`/api/hassio/` refuses a long-lived token) and probing it was out of
scope — it regulates a real battery. It runs `max_w: 5000` and is unaffected
either way.
⚠️ **Known, deliberately left, and worth a follow-up:** `main.py` and
`maintenance.py` still fall back to **2500** when the key is absent from the
options dict, while `config.yaml` and `validate.py` now say 2000. Unreachable on
a Supervisor install (every key is always supplied) and reachable only from
tests/`sim`. The real fix is one source of truth for option defaults, which is
bigger than this ticket and touches files another ticket is in.
## 0.5.2
**A meter frozen at a wrong-but-fresh value now raises an alarm. It still is not
stopped.** (SAFETY-18)
A meter that answers `200 OK` with a stale document forever is *arriving* on
every poll, so every freshness mechanism in the stack reads it as healthy:
`meter_max_age_s` is reset by each response, the firmware watchdog is fed by the
heartbeat and by a sample age that never climbs, and the arbiter contributes only
±`max_w`. The loop integrates a constant error and rails — replaying the real
`compute()` at 15 s cycles, in 150/60/30 s at `max_w: 2000` for a meter frozen at
350/1000/≥3000 W, and 360/135/75 s at `max_w: 5000`. **The live install runs
`max_w: 5000`**, which is both the schema maximum and the device rating.
The new detector (`app/freeze.py`) is a **conjunction**, not the bare
`unchanged_s` threshold TEL-05 measured and rejected:
unchanged_s ≥ meter_freeze_s AND |command| within 5 % of max_w
Both terms already existed and were already on the status page. They fail for
independent reasons: at the converged point the command is nowhere near the rail,
and at the rail against a real load the kWh registers move within seconds. At the
shipped 300 s there is also a structural relation, not just a margin — the
slowest a healthy 1 Wh register can tick and still look unchanged is 12 W, which
is inside the 15 W deadband, so the command there is parked at 0 W and the two
halves cannot both be true.
**Notification only, deliberately.** An HA persistent notification (fixed
`notification_id`, so a repeat replaces rather than stacks), an `ERROR` log line,
a UI event and a red status banner. No claim, no latch, no 0 W: a false trip
parks the battery and the operator's next move is to switch the guard off; a
false notification costs an alert. The trip is a separate decision that needs
data from a real install.
- New option `meter_freeze_s` (default 300, range 60–3600, no "off").
- **Only `meter_source: homewizard_local` is covered** — it is the only transport
that sees the meter's own document. Everywhere else the status page says
**NOT COVERED** rather than "armed".
- `DOCS.md` now states the **capacity-tariff exposure** this does *not* close:
`peak_cap_w` caps maintenance charging and nothing caps the loop, so a lying
meter can drive a 5 kW unattended import, and ≥15 minutes of that sets the
month's billed peak (measured: one 4998 W quarter moved a monthly peak from
4832 → 5013 W). That cap is SAFETY-07.
- `test_freeze.py` grades it in both directions: silent over 40 minutes of the
live house's real converged capture (committed as
`scenarios/p1-age-2026-08-26.jsonl.gz`), silent over an hour of TEL-05's
low-power case where a bare threshold is shown to fire, and alarming on a
genuinely frozen meter driven over real HTTP through the real
`HomeWizardLocalSource` — against the rig's own `hwsim` `freeze` fault when the
infra repo is reachable, an equivalent stub otherwise. Its negative twin is the
load-bearing one: the same railed command against a meter that is still moving
must stay silent at the same threshold.
## 0.5.1
**SAFETY-02's latch now survives an add-on restart, which is the difference
between a latch and a pause.** `auto` was persisted and `sign_wd.tripped` was
not, so reconstructing the `Controller` against the same store — exactly what a
container restart does — brought it back **armed, into the same fault, with no
operator involved**: `tripped=False`, banner back to `NOT READY`, and it resumed
commanding (measured: −1000 W, −2000 W) before re-tripping 180 s later. Under a
Supervisor that restarts a crashed add-on that is a loop of three-minute
wrong-direction bursts — and "restart the add-on" is the first thing an operator
looking at a `FAILSAFE` banner and a 0 W inverter will try.
`sign_tripped` and `sign_reason` are now `Store` keys beside the maintenance
state, written the moment the trip happens (reason first, flag second, so a crash
between the two can never persist a latch with no explanation) and restored in
`Controller.__init__`. Cleared only by the operator's start. `DOCS.md` said "it
does not clear itself"; that was false across a restart and now is not.
Also in this release, all of it review follow-up rather than new behaviour:
- **The arming gate is tested against the real arbiter.** It was the decision the
whole ticket turns on and it was asserted in three comment blocks and verified
by nothing — every case passed `following` in as a literal. `test_watchdog.py`
now derives it from `resolve()` across twelve claim configurations, including
the redirect case `winner == "loop"` alone would have got wrong and the `lo`
limit that raises magnitude without flipping sign.
- **The MQTT `status: "failsafe"` value is tested.** The check that claimed to
cover it restated the assertion three lines above it and never touched the
payload.
- **A forward hazard written down where it will be read, and made to bite.**
`winner == "loop"` disarms the watchdog under any higher-priority SET claim.
Today only maintenance and the safety stops emit one and neither follows the
meter; a peak-shaving strategy at `P_PEAK` **does**, and would blind the
watchdog for every peak event.
The gate is now `watchdog.loop_is_following()`, called by both `main.py` and
the test, for the same reason `disagrees()` was extracted: the test held a
*transcription* of the shipped expression, so editing the shipped gate alone to
`winner in ("loop", "peak")` left every suite green while the watchdog went
blind. Two copies, and the one that drifts is the one doing the grading.
⚠️ Extracting it was not sufficient on its own. The realistic P_PEAK row —
peak shaving asking for *more* than the loop — still could not detect the
widened gate, because the magnitude term rejects it regardless and masks the
winner term. A second row with peak asking for *less* isolates `winner` as the
only deciding term, and that is the one that turns red.
- **The worked example described a configuration that does not ship.** At the
defaults `maintenance_charge_w` (2500) exceeds `max_w` (2000), so that claim is
an arbiter *contradiction* resolving to 0 W, not a −2500 W charge. The example
is now written at 1500 W. (The contradiction is a real pre-existing bug —
a default cheap-window maintenance charge silently never happens — filed
separately, deliberately not fixed here.)
- **`disagrees()` is public**, so `sim/replay_sign.py` stops carrying its own copy
of the predicate it grades.
- `held_s` takes the clock instead of reading one, restoring the pure-function
property the module advertises.
- **The `sign_reason`-before-`sign_tripped` write ordering is enforced, not just
documented.** Swapping the two lines used to pass every check. A store that
fails the second write now proves the property the ordering exists for: what
survives a crash mid-trip is a reason with no latch, never a latch with no
explanation.
- Measured figures corrected throughout: 93.4 % / 4212 of 17280 (was 92.9 % /
4127 of 16349), per-cluster 0.3 / 0.9 / 10.5 / 0.0 %.
## 0.5.0
**SAFETY-02 — the sign-disagreement watchdog.** The add-on now watches whether
the setpoint it is emitting and the grid figure it is regulating on point the
same way. If they oppose each other, both over `sign_disagree_w` (300 W), for
`sign_disagree_s` (180 s) **continuously**, it files a failsafe claim at
`P_SAFETY`, commands **0 W**, and latches there until an operator starts control
again. New module `app/watchdog.py`; new options `sign_disagree_w` and
`sign_disagree_s`.
This is the gap the existing failsafes leave open. The stale-input watchdog asks
whether the meter is still talking, the saturation freeze asks whether the
inverter can follow, the output clamp asks whether the number is too big — a
loop can pass all three while charging the battery out of the grid, because the
only thing wrong is the sign. On 2026-08-20 a commercial controller did exactly
that four times in one day, for up to 77 minutes at a stretch, and reached a
commanded −14 768 W against a ±5000 W inverter.
Validated against that day. `sim/replay_sign.py` in the infra repo drives this
module over the committed RS485 captures of both 2026-08-20 (the four episodes)
and 2026-08-23 (healthy): **18 trips on the fault day, every one inside a window
`NOTES.md` §4.7 accounts for, and 0 on the healthy day.** The four episodes do
share one signature rather than merely resembling each other — sign agreement
runs 0.3 / 0.9 / 10.5 / 0.0 % inside them against 93.4 % on the healthy day.
Three decisions worth knowing about:
- **It is disarmed whenever the grid-following loop is not what is driving.**
Sign agreement is a property of a loop that follows the meter. A maintenance
grid charge opposes the meter *by design* — 2500 W of charge into a house
drawing 500 W is +3000 W on the meter against a −2500 W command, for hours —
and armed unconditionally this would trip on every maintenance cycle. The gate
tests the emitted number, not the winning claim's name: a limit claim can let
the loop "win" and still replace its value with one pointing the other way.
- **The trip latches.** Commanding 0 W removes the command side of the
comparison, so the disagreement always clears the moment the trip works.
Un-latched, this would trip, clear, and hand the actuator back to the loop that
was driving the wrong way, every three minutes forever.
- **`sign_disagree_w` at or above `max_w` is refused at startup** (validator rule
5). The arbiter never lets a command exceed `max_w`, so such a threshold makes
the watchdog incapable of firing while the status page still reports it armed.
A disarmed guard that advertises itself as armed is the failure mode this
project keeps refusing to ship.
What it does **not** catch, stated so nobody assumes otherwise: an inverter that
acts opposite to what it was told moves the meter *further* the same way, so
command and meter agree while everything runs away. That is the saturation
freeze's job, which compares the command against the battery's actual power.
`test_validate.py`'s "the smallest sane max_w still starts" case now names a
reachable sign threshold, because a 1 W ceiling genuinely does disarm the 300 W
default and is now correctly refused.
## 0.4.0
**CFG-01.** The add-on now refuses to start on a configuration it cannot honour,
@@ -67,6 +413,121 @@ A refusal is legible as a refusal: a banner that says in words that this is a
configuration problem and nothing is broken, the numbered problems, no traceback,
and exit status 78 (`EX_CONFIG`).
**SAFETY-12.** The control loop now evaluates on a clock as well as on a change.
`run_control()` ran `cycle()` only when the meter value differed from the
previous poll, so a meter that repeated a value ran **no control cycles at
all** - while the heartbeat went on re-emitting the last command and the
firmware went on putting it on the wire every 2 s. Layer 1 stayed fed, every
indicator read healthy, and a command computed from an old reading kept being
executed through whatever the load did next. Measured on the rig during TEL-05:
70 s at a literally constant 350.0 W with 14 arrivals and zero cycles.
**It was worse than a latency limit: it was a safety claim that could not
fire.** `stale_input_s` and the `inputs missing -> 0 W` claim both live *inside*
`cycle()`. Pre-fix, if the meter value happened to be constant, `cycle()` was
never called again - so that claim had no way to run. Constant meter, SoC entity
dying at t=4 s, same binary:
```
pre-fix writes: [(0.0, 220.0), (0.0, 220.0), (10.1, 220.0)]
final target: 220.0 W reason: 'loop -> 220 W'
post-fix writes: [(0.0, 220.0), (0.0, 220.0), (3.0, 430.0), (9.1, 0.0), (10.1, 0.0)]
final target: 0.0 W reason: 'safety -> 0 W'
```
The pre-fix controller held 220 W through the entire input outage with its
status line still reading `loop -> 220 W`, and layer 1 does not cover that case
on a `meter_source: off` install: no P1 age sensor is published, so the firmware
has nothing to veto on, while the heartbeat keeps its watchdog fed. So the old
behaviour was not "holds a stale command safely" - it was a failsafe that a
repeating meter switched off. `test_loop.py` asserts both halves.
`cycle_floor_s` (default **15 s**) is a deadline on the gap between cycles, not
a second scheduler: every cycle resets it, whichever trigger fired. On a healthy
meter it never elapses, so the cycle rate, the write sequence and therefore the
per-cycle gain the tuning was measured against are unchanged - asserted in
`test_loop.py` by driving the real loop twice on one binary (a huge
`cycle_floor_s` reproduces the old behaviour exactly) and comparing the
commanded values.
The reset is not the only thing holding that up, and the other argument is the
sturdier one: the combined cycle rate is `max(arrival rate, 1/floor)`, never the
sum. At 1/15 s the floor is strictly slower than the ~1/5 s arrival rate, so
even a hypothetical floor racing the arrivals could not exceed the nominal
healthy rate. The re-tune is unreachable by construction. ⚠️ That holds only
while `cycle_floor_s` stays above the meter's update interval, and the schema
minimum is 5 s - which is exactly the reference meter's interval. Nothing
enforces the relationship; `validate.py` is the obvious home for it and does not
have it yet.
Observed over six real minutes on the same rig, one binary, meter held at a
literally constant 350.0 W: **before, 1 cycle** (at startup) and then nothing,
the command stuck at 220 W against a 350 W import for the rest of the window.
**After, 24 cycles**, worst gap 15.14 s, and the ordinary PI ramp doing its job -
220, 430, 650, 860, 1080, 1300, 1510, 1730, 1940, 2000 W, one step per cycle,
rail at t=136 s and flat after. Three minutes with the meter parked at the
target instead: **1 cycle before, 12 after, and 19 writes either way - the same
19**. Three minutes on a normally-changing meter: **36 cycles and 49 writes on
both sides, the same values in the same order.** The heartbeat write count never
moves; the only writes the fix adds are the ten the loop was supposed to be
making all along.
15 s is bracketed, not round.
**Below it, ~11 s, and it arises more routinely than a dropped poll.**
`HomeWizardLocalSource` polls at 5.0 s against a meter that updates every 4.97 s
(NOTES.md:640). Two near-equal periods beat, so roughly once every ~14 minutes
the poller steps over an update entirely - a ~10 s gap between changed values on
a perfectly healthy meter, plus up to 1 s for the 1 s tick to notice. A failed
HTTP poll (retried on the next tick, no backoff) adds another 5 s. The floor
sits above the beat-skip so it does not fire on that alone; what it takes to
reach 15 s is a beat-skip *and* a failed poll coinciding.
**Above it, 30 s.** `meter_max_age_s` is 30 s, past which there is no reading to
evaluate, and the firmware's ingest veto is also 30 s - so the loop is
guaranteed to have re-decided at least twice inside any window in which layer 1
might otherwise be the only thing deciding. ⚠️ The default is therefore not
independently justified at 15: if the firmware's veto ever moves, this moves
with it. Noted beside the constant in `main.py`.
It is also exactly `stale_input_s`, this system's existing answer to "how long
may we hold without re-deciding".
A floor cycle runs the **full** law on the same inputs, integrator included. In
P1 mode the freshness question is already answered upstream and answered harder
- past `meter_max_age_s` the reading is None, which is "inputs missing", which
is a `P_SAFETY` 0 W claim - so a floor cycle there can only ever run on a reading
this system already calls fresh. ⚠️ That is transport-conditional: with
`meter_source: off`, `read_inputs()` takes the grid figure straight from
`hass.number()` with **no age gate at all**, so a Home Assistant entity that has
gone quiet reads as current on that path, with or without this change. Running
the full law is still right there - the alternative is a degraded second law
reasoning about an age nobody measured - but the floor does not make that path
fresh. The integrator advances per *cycle*, not per second,
so bounding the gap bounds the accumulation; `integrator_max_w` bounds it after
that. A gentler second law for floor cycles would be a second set of dynamics
nobody can test on this hardware.
⚠️ **This is not a freeze detector**, and TEL-05 established one cannot be built
here: at the converged -10 W target a 1 Wh register needs ~6 minutes to move
while the power figure legitimately repeats, so any threshold on `unchanged_s`
false-trips at the exact operating point we run at. Nothing in this change
decides whether a repeated reading is *true*. It stops "the number is the same"
from meaning "do not think". The consequence, stated plainly: against a meter
genuinely frozen at a wrong value the loop will now correct toward that wrong
value rather than hold a stale command, bounded by `max_w`, the arbiter and the
device rating. That was already what happened to a lying meter whose value
merely jittered; bit-exact repetition was never a safety property.
`DOCS.md`'s "Saturation is counted in cycles, not seconds" section documented
the old behaviour as an accepted deviation. It is **qualified, not deleted**:
the deviation survives (the window is still counted in cycles, and 3 x 15 s is
still not the specification's 10 s) but it is now bounded at 45 s instead of
open-ended, and the passage's second justification - "nothing accumulates while
it is stalled" - is called out as having covered integrator windup and not
staleness, which was the actual hazard. The same wrong argument in
`control.py`'s saturation comment is corrected too.
**TEL-04.** `meter_source: ha_signed` reads a single *signed* power entity, which
is the shape a HomeWizard P1 publishes. Both transports 0.3.0 shipped require two
unsigned registers and refuse negatives, so neither fits the meter installed here.
+246 -16
View File
@@ -24,9 +24,16 @@ The inverter **holds its last command forever** — it has no meter-timeout. So:
commanding it.
- This add-on commands 0 W when its inputs go missing, when you stop control,
and when it shuts down.
- This add-on commands 0 W **and latches there** if what it is commanding and
what the meter reads point opposite ways for three minutes — see "The
sign-disagreement failsafe" below.
- The **optional RS485 e-stop** is the only thing that covers this machine
dying. Without it, a failed host leaves the battery latched at its last
command until someone intervenes.
- This add-on **raises an alarm — and nothing more —** when the meter keeps
answering with a reading that never changes while the command sits at the
rail. See "The frozen-meter alarm" and, before you rely on it, "Capacity-tariff
exposure of the grid-following loop".
**If anything looks wrong: stop the add-on.** That commands 0 W and the
hardware holds it there.
@@ -272,11 +279,69 @@ behind it would trip the firmware watchdog on a system that is working fine.
| `target_grid_w` | -10 | What the meter should rest at. Negative = a slight export |
| `step_w` | 10 | Quantisation |
| `saturation_w` | 500 | Divergence that counts as "the inverter is at a limit" |
| `saturation_cycles` | 3 | How many consecutive cycles before freezing. A cycle is one *changed* meter reading, not a fixed period - see the note below. **Do not set to 1** |
| `saturation_cycles` | 3 | How many consecutive cycles before freezing. A cycle is one meter update or `cycle_floor_s`, whichever comes first - see the note below. **Do not set to 1** |
| `integrator_max_w` | 0 | Bound on the loop's accumulator, and 0 means "same as `max_w`". Caps how much stale error can be waiting to unwind when the sign flips. **Do not raise it above `max_w`** - the output clamp already bounds what is commanded, so the only thing extra headroom buys is more cycles of wrong-direction power after every saturation event. Lowering it below `max_w` is the useful direction |
| `heartbeat_s` | 10 | Refresh interval; must stay well under the firmware watchdog |
| `cycle_floor_s` | 15 | The longest the loop may go without evaluating. A deadline on the gap between cycles, reset by every cycle — on a healthy meter it never elapses and changes nothing. Raise it only if your meter is slower than ~10 s; it must stay under `meter_max_age_s`, past which there is no reading to evaluate anyway |
| `stale_input_s` | 15 | How long inputs may be missing before commanding 0 W. In P1 mode this clock starts only *after* `meter_max_age_s` has already expired — the two stack, see "How long a dead meter takes to reach 0 W" |
| `auto_start` | false | Start controlling on boot (only after commissioning) |
| `sign_disagree_w` | 300 | Both the command and the meter must exceed this before a sign difference counts. Must stay **below `max_w`** or the watchdog can never fire, and the add-on refuses to start |
| `sign_disagree_s` | 180 | How long they must stay opposed, continuously, before the failsafe trips |
#### The sign-disagreement failsafe
If the setpoint this add-on is emitting and the grid figure it is regulating on
point **opposite ways** — charging the battery while the house imports, or
discharging it while the house exports — with both over `sign_disagree_w`, for
`sign_disagree_s` **continuously**, the add-on commands **0 W** and holds it
there. The banner reads `FAILSAFE`, the commissioning list says which two
numbers disagreed and for how long, and the `Controller status` entity reports
`failsafe`.
**It does not clear itself, and restarting the add-on does not clear it either.**
Pressing *Start / stop control* twice — stop, then start — is the only thing that
does. That is deliberate on both counts:
- Commanding 0 W removes the command side of the comparison, so the disagreement
always vanishes the instant the trip works. A watchdog that took that as proof
the fault was over would hand the inverter straight back to the loop that was
driving the wrong way, three minutes later, forever.
- The trip is written to `/data/state.json` and restored on startup. Restarting
the add-on is the first thing most people try when they see a red banner, and
a latch that a restart clears would bring the controller back **armed, into the
same fault, unattended** — and under a Supervisor that restarts a crashed add-on
automatically, would do so in a loop. After a restart the banner, the 0 W and
the recorded reason are all still there.
**What it is for.** This is a loop-integrity check: it asks whether the loop's
output direction still matches the input state it was regulating. It catches a
polarity error anywhere between the meter and the wire, and an integrator so far
wound up that unwinding takes minutes. It is the check that would have caught all
four episodes of 2026-08-20, when a commercial controller charged the battery out
of the grid, four times, for up to 77 minutes at a time.
**What it is not for.** It does not detect an inverter that acts opposite to what
it was told — that moves the meter *further* the same way, so command and meter
agree while everything runs away. That is the saturation freeze's job
(`saturation_w` / `saturation_cycles`), which compares the command against the
battery's actual power. The two checks are complementary and neither replaces the
other.
**Why it will not false-trip on ordinary operation.** A battery that refuses a
command (empty, full, standby) leaves the meter where it was, so command and
meter agree. A frozen meter drives the loop into agreeing with it. Around the
resting point both signals are inside the 300 W gate, so the deadband can never
produce one. A load step does flip the meter's sign before the loop catches up —
measured on a full captured day, the longest such crossing is 30 s, against a
180 s window. And the watchdog is **disarmed whenever the grid-following loop is
not what is driving**: a maintenance grid charge opposes the meter by design and
must not be mistaken for a fault.
Both numbers are measured rather than chosen, and are exposed only because the
ticket requires defaults rather than constants. On the captured healthy day sign
agreement is 93.4 %; on the four captured fault clusters it is 0.3 %, 0.9 %,
10.5 % and 0.0 %. There is no reason to move either value without a capture that
says so.
#### Saturation is counted in cycles, not seconds
@@ -284,22 +349,43 @@ The specification states the saturation window as **"> 10 s"**. This add-on coun
**cycles** instead, and that is a deliberate, accepted deviation rather than an
oversight - the acceptance criterion is not met as literally written.
A cycle here is one *changed* meter reading: the controller only runs the loop when the
meter value differs from the previous poll. At the reference P1's ~5 s update rate the
default of 3 cycles is usually around 15 s, but there is **no guaranteed wall-clock
window** - a meter that repeats the same value stalls the counter for as long as it
repeats.
A cycle here is one meter update **or `cycle_floor_s` elapsing, whichever comes
first**. At the reference P1's ~5 s update rate the default of 3 cycles is usually
around 15 s, and the worst case is bounded at `3 x cycle_floor_s` = 45 s. It is still
not the 10 s the specification asks for, so the deviation stands - but it is now a
bounded number rather than an open one.
Two reasons that is acceptable:
⚠️ **This passage used to say something different, and the second half of it was
wrong.** Until SAFETY-12 the loop ran only when the meter value *changed*, so a meter
that repeated a value stalled the counter for as long as it repeated, with no upper
bound at all. That was defended here on the grounds that "a stalled counter is a
detection-latency limit and not a runaway risk - the condition that stalls it stops the
whole loop, so nothing accumulates while it is stalled". That argument covers
**integrator windup and nothing else**. The hazard it misses is **staleness**: while the
loop was stopped the heartbeat kept re-emitting the last command and the firmware kept
putting it on the wire every 2 s, so a command computed from an old reading went on
being executed through a real load change, with every indicator reading healthy.
Measured on the rig during TEL-05: 70 s at a literally constant 350.0 W, 14 arrivals,
zero control cycles.
- the control law is a pure function with no clock, which is what makes it testable
without hardware, and a seconds-based window would have to live in the controller;
- a stalled counter is a detection-latency limit and not a runaway risk. The condition
that stalls it - an unchanging meter - stops the whole loop, so nothing accumulates
while it is stalled.
The half that survives is the first one: the control law is still a pure function with
no clock, and the wall-clock window still lives in `Controller`, which has one - see
`run_control()`. What changed is that the controller now enforces a floor on the gap
between cycles, so `saturation_cycles` finally means a bounded number of seconds.
If a guaranteed window matters on your site, raise `saturation_cycles` for a fast meter,
and treat the figure as "N meter updates" rather than "N seconds".
If a tighter window matters on your site, lower `cycle_floor_s` (not below your meter's
update interval, or the floor starts firing on a healthy meter) or raise
`saturation_cycles` for a fast meter.
**This is not a freeze detector, and it deliberately cannot be one.** Nothing here
decides whether a repeated reading is true — see `unchanged_s` in the meter section for
why no threshold on that is safe at the converged operating point. The floor only stops
"the number is the same" from meaning "do not think". The consequence is worth stating
plainly: against a meter that is genuinely frozen at a wrong value, the loop will now
correct toward that wrong value instead of holding a stale command. It is bounded by
`max_w`, the arbiter and the device rating, and it is the same thing that already
happened to a lying meter whose value merely jittered — bit-exact repetition was never a
safety property.
#### Why `target_grid_w` is not zero
@@ -332,8 +418,8 @@ help - see "Why the tuning is what it is".
| `maintenance_enabled` | false | Enable the monthly cycle |
| `maintenance_interval_days` | 28 | Minimum gap between cycles |
| `maintenance_start_hour` | 10 | Hour of day a due cycle begins |
| `maintenance_discharge_w` | 2500 | Drain rate (exports the surplus) |
| `maintenance_charge_w` | 2500 | Charge ceiling, capped again by peak headroom |
| `maintenance_discharge_w` | 2000 | Drain rate (exports the surplus). Above `max_w` it is **clamped** to `max_w` — the drain phase SETs the setpoint and the arbiter bounds it, saying so on the status page (`limited by supervised(max_w)`). Warned about, not refused: the cycle still completes, just slower |
| `maintenance_charge_w` | 2000 | Charge ceiling, capped again by peak headroom. Must stay **at or below `max_w`** or the cheap-window shaping (`hi = -maintenance_charge_w`) contradicts the supervised limit (`lo = -max_w`), the arbiter resolves it to **0 W**, and maintenance silently never charges in a cheap window — **the add-on refuses to start** |
| `maintenance_soc_floor` | 11 | Drain target — stay just above the inverter's own floor |
| `maintenance_soc_target` | 99 | Charge target |
| `maintenance_hold_min` | 120 | Hold at full so the BMS can balance |
@@ -351,6 +437,150 @@ under `peak_cap_w`, and if the forecast goes over the cap the charge-only clamp
is dropped so the battery can shave the peak instead. Money outranks the
maintenance schedule.
#### Capacity-tariff exposure of the grid-following loop
⚠️ **`peak_cap_w` caps MAINTENANCE charging. It does not cap the loop.** The
grid-following loop is bounded by `max_w` and by the device rating, and by
nothing else. That is deliberate — the loop exists to *reduce* what crosses the
meter, so under every healthy condition it lowers the peak rather than raising
it — but it means one thing is true and worth writing down:
**If the meter lies, the loop will happily buy `max_w` from the grid for as long
as the lie lasts, and no cap in this add-on stops it.** A meter frozen at a large
export reads as "the house is exporting hard", the loop charges the battery to
correct it, and the correction is a grid import. At the schema maximum
`max_w: 5000` — which is what this house runs, and is also the device rating —
that is a 5 kW unattended import.
The bill is not the energy. In the Belgian capaciteitstarief the month is billed
on its highest quarter-hour average, so **fifteen minutes of this sets the whole
month's peak.** A measured case: one 4998 W quarter moved a monthly peak from
4832 W to 5013 W, and every day of that month was billed at the higher figure.
What exists today, and what does not:
| | |
|---|---|
| `max_w` bounds the magnitude | ✅ — but the exposure IS the magnitude |
| Maintenance charging capped under `peak_cap_w` | ✅ |
| **Loop capped under `peak_cap_w`** | ❌ — not built. This is SAFETY-07 |
| Frozen meter detected | ✅ on `homewizard_local` only, **alarm only** |
| Frozen meter *stopped* | ❌ — the alarm does not command anything |
| SoC guard on the loop | ❌ — the BMS is the only energy backstop |
So the frozen-meter alarm below **narrows the window; it does not close it.** On
a capacity-tariff site where an unattended runaway is unacceptable, the mitigation
today is operational: set `max_w` no higher than the loop actually needs (the
tuning was measured at 2000 W and a 1.78 kW load step is tracked fine there), fit
the e-stop, and watch for the alarm.
### The winding-loop alarm (the output clamp)
Every setpoint this add-on writes is clamped to `max_w` before it leaves the
process, and **when that clamp actually engages, it alarms.** The clamp on its
own is not the feature - this project already clipped, and the clipping is what
hid the defect the guard exists for: with the inverter switched off entirely an
unclamped integrator climbed ~130 W per 4 s past 10 kW and, pinned at the 5 kW
rail, reported **14 768 W**. Clipped silently, that is a flat, legal-looking
5000 W hold on every dashboard, and nothing anywhere says the loop asked for
three times it.
**What "the rating" is.** The bound is `max_w`, and only `max_w`. It is already
validated against the setpoint device's own range at startup - the add-on
*refuses to start* if `max_w` exceeds what the device accepts (and again once
ESPHome is up and the real rating is known), so by the time the loop runs the two
numbers agree. The device range is applied a second time inside the write as
defence in depth; after a successful validation it cannot bite, and if it ever
does, that engagement alarms too and names the bound.
**Clamping is not saturation, and the two never share a channel.**
| | what it means | what happens |
|---|---|---|
| **clamp** | the *computed* value exceeded the rating | a software bug signal - something upstream is winding. **Alarm.** |
| **saturation** | the inverter is not following a perfectly *legal* command | a hardware condition - freeze the integrator. **No alarm.** |
A loop can be legally saturated all day: a cold pack, a curtailed inverter, an
export limit. Routing the clamp through a saturation-shaped alarm puts the bug
signal inside the noise, which is the same silence as clipping quietly. They
share no reason code, no notification id, no status row and no counter.
**It is an alarm, not a trip.** The clamped value is written and the loop keeps
running - this is not the sign-disagreement failsafe, which commands 0 W. You
get a persistent notification (`goodwe_output_clamp`, replaced rather than
stacked), an `ERROR` in the log with the reason code `OUTPUT_CLAMP`, a row in
the event list, and a red status banner while it is engaged. Once it clears, a
warning row keeps the engagement count so the evidence does not vanish with the
condition. Consecutive engagements share one event row carrying the exact count,
because the event list keeps only 40 rows and a row per cycle would flush every
other guard's entries out of it.
**There is nothing to configure**, deliberately. The bound is `max_w`, which you
already set; giving the guard a limit of its own would be a second number that
can disagree with the first, which is exactly the failure the startup validation
exists to prevent. The alarm's cadence - first engagement, then re-asserted every
30 s while it persists, cleared after 3 clean cycles - is fixed for the same
reason `RAIL_FRAC` is: it describes the notification bus, not the plant.
WARNING: **on a default install the engagement you will see is the integrator's,
not the output's.** The control law bounds the integrator to `max_w` and *then*
derives the command from it, so the command itself can never come out over the
rating unless `integrator_max_w` has been deliberately raised above `max_w`.
Both are watched. A guard that watched only the emitted value would be silent on
every standard configuration.
### The frozen-meter alarm
A meter that has died is easy — it stops answering, the sample age climbs past
`meter_max_age_s`, the reading becomes missing, and the add-on and the ESP32 both
command 0 W. A meter that keeps answering `200 OK` with a document that never
changes is the hard one: **every freshness mechanism in the stack reads it as
healthy**, because it really is arriving on every poll. The loop then integrates
a constant error and winds to the rail, in 75–360 s at `max_w: 5000`.
The add-on alarms when **both** of these are true at once:
- the meter's own document (power figure *and* both kWh registers) has not
changed for `meter_freeze_s` — 300 s by default; **and**
- the command standing on the wire is at the rail, within 5 % of `max_w`.
Neither half is usable alone. A bare "unchanged for N seconds" rule false-trips
at the exact operating point this controller aims for — at a converged −10 W a
1 Wh register legitimately takes about six minutes to move — and a bare "at the
rail" rule fires on any genuinely large load. Together they are safe in both
directions, because a real railed condition means kilowatts still crossing the
meter, and a kilowatt advances a 1 Wh register every 3.6 s.
**It is an alarm and nothing else.** It does not command 0 W, does not latch, and
does not claim the actuator. A false trip here parks the battery; a false
notification costs an alert. Turning it into a trip is a separate decision that
needs data from a real install — and an answer to "what should the loop do
instead", given that 0 W is itself an import of whatever the house is drawing.
When it fires you get a Home Assistant persistent notification
(`goodwe_meter_frozen`, replaced rather than stacked on a repeat), an `ERROR` in
the add-on log, an entry in the UI's event list, and a red banner on the status
page. **The loop keeps running.** If the meter really is stuck, stop control from
the add-on UI.
⚠️ **Only `meter_source: homewizard_local` is covered.** It is the only transport
that reads the meter's own document; on `ha_dsmr`, `mqtt_p1`, `ha_signed` and
`off` there is no signal to threshold and the status page says **NOT COVERED**
rather than pretending the guard is armed.
| option | default | meaning |
|---|---|---|
| `meter_freeze_s` | 300 | Seconds of an unchanged measurement, *while the command is at the rail*, before the alarm |
Why 300 s: over 2399 s of this house at the converged operating point (mean grid
−14.2 W, 1194 polls, 735 of them repeating the previous reading) the largest
`unchanged_s` any sample sees is **8.0 s**, and the longest interval serving no
distinct reading at all is **15.4 s**. There is also a structural reason not to
lower it much: at 300 s the slowest a *healthy* register can tick and still look
unchanged is 12 W, which is inside the 15 W deadband — so the command there is
parked at 0 W and the two halves of the rule cannot both be true. At the schema
floor of 60 s that figure is 60 W and only the margin is left.
### Site
| option | meaning |
+284
View File
@@ -0,0 +1,284 @@
"""SAFETY-03 - the output clamp, and the alarm that makes it audible.
WHAT IT CATCHES, in one sentence: a control law that computes a setpoint the
actuator cannot deliver, which every layer below it will happily clip to
something legal-looking while the thing that produced it goes on winding.
THE CLAMP IS NOT THE DELIVERABLE. specs.md 10.3 is explicit that this project
already clipped, and that the clipping is what hid the defect:
with the inverter switched off entirely, an unclamped integrator climbed
~130 W per 4 s to over 10 kW and, pinned at the 5 kW rail, reported
14 768 W
A clamp alone turns that into a silent, perfectly flat 5000 W hold. Every
dashboard reads "commanded 5000 W", which is a legal number, and nothing
anywhere says the loop asked for three times it. 11.2's watchdog row says
"Clamp AND ALARM - the loop is winding"; the second verb is this module.
------------------------------------------------------------------------------
WHAT "THE RATING" IS HERE, AND WHY IT IS ONE NUMBER AND NOT A MIN()
The card names two bounds and warns against choosing between them:
- `DriverCapabilities.setpoint_limit_w` (3.2), the inverter's own magnitude
rating. In this add-on that arrives as the setpoint entity's `min`/`max`
from `hass.limits()` - `Controller.dev_min` / `dev_max`.
- `optimizer.loop.clamp_w` (14), the configured loop clamp. In this add-on
that is the `max_w` option, which `Tuning.max_w` and the arbiter's
`supervised` limit both already use.
14 says these two must AGREE, validated at startup with a refusal on
mismatch - and that gate already ships: `validate_startup_config()` refuses
`max_w > dev_max` and `-max_w < dev_min` (validate.py, problem 1), and
run_control() re-runs it once ESPHome is up and the real rating is known.
So this guard clamps to `max_w`, THE VALIDATED VALUE, and does not `min()` it
with the device rating at runtime. That is the card's instruction and it is also
the only reading that keeps `max_w` meaningful: a silent runtime `min()` is
precisely the "helper bounded to +-500 W under a loop clamped to 1500 W" failure
the 14 warning box is about.
WARNING - THE DEVICE BOUND IS STILL APPLIED, SECOND, AND IT IS NOT REDUNDANT.
After a successful validation `+-max_w` lies inside `[dev_min, dev_max]`, so the
second stage is arithmetically a no-op and can never engage. If it DOES engage,
the validation did not run or no longer holds - `hass.limits()` returns
(None, None) whenever ESPHome is not up yet, and the range checks are skipped
rather than guessed at - and that is a louder fault than the first stage, not a
quieter one. It therefore alarms too, through the same channel, with the bound
named. This is NOT SAFETY-20; nothing here changes what `freeze.py` measures its
rail against.
------------------------------------------------------------------------------
WHERE THE WIND ACTUALLY SHOWS UP - READ THIS BEFORE "SIMPLIFYING" THE CALLER
`control.compute()` bounds the integrator to `integrator_max_w or max_w` and
THEN derives the command from it, so on a default install (`integrator_max_w`
unset, i.e. following `max_w`) the command can never exceed `max_w` and the
actuator-side clamp below IS UNREACHABLE. control.py says so in as many words:
SAFETY-03 ... must watch for THIS, not for "clamped" below.
An alarm wired only to this module's `ClampResult.engaged` would therefore be
silent on the exact configuration the whole ticket is about. The winding is
visible one layer up, at the integrator bound, and that is why `Decision` now
carries `i_clamped` / `clamped` BOOLEANS and the caller ORs them in here.
AND WHY BOOLEANS RATHER THAN `Decision.reason`. `reason` is a single string that
later stages overwrite: a cycle that clamps the integrator and is then slew
limited reports "slew-limited", and one that clamps while saturated reports
"saturated-freeze". Both are exactly the winding-into-a-rail case, and on the
string the clamp signal is destroyed before anything can alarm on it. The two
flags are set where the clamp happens and are never overwritten. A check in
test_clamp.py drives that precise combination.
------------------------------------------------------------------------------
DISTINCT FROM SATURATION - THIS IS ACCEPTANCE CRITERION 2, NOT A NICETY
clamp the COMPUTED value exceeded the rating. A software bug signal:
something upstream is winding. Alarm, every engagement.
saturation |command - inverter AC power| > `saturation_w` for
`saturation_cycles`. The HARDWARE cannot follow a perfectly legal
command. Action is "freeze the integrator" (11.2), not an alarm.
A loop can be legally saturated all day - a cold pack, a curtailed inverter, an
export limit - so a saturation-shaped alarm is one an operator learns to ignore.
Routing the clamp through it puts the bug signal inside the noise, which is the
same silence 10.3 is complaining about. They share no reason code, no
notification id, no status row and no counter.
------------------------------------------------------------------------------
ANTI-FLOOD, WITHOUT GOING SILENT
A genuinely winding loop clamps on EVERY cycle for as long as it winds. One HA
notification per cycle is how an operator learns to dismiss notifications
unread, and this add-on already made that argument once (see the fixed
`notification_id` note in main.check_meter_frozen).
So the two surfaces are split, deliberately:
- EVERY engagement is recorded, never deduplicated, and that record is what
says how hard the loop is pushing. 15's action log is this add-on's
`Controller.events`, and it gets a row from `write_setpoint()` on every
engaged WRITE; `engagements` here counts every engaged CYCLE. The two layers
are counted separately on purpose - the heartbeat re-writes an
already-clamped value between cycles, so a per-write count would inflate and
a per-cycle log would miss an out-of-cycle write.
- THE NOTIFICATION fires on the rising edge and then RE-ASSERTS every
`REASSERT_S`, so a loop pinned on the rail for an hour does not go quiet.
- It DE-ASSERTS only after `CLEAR_CYCLES` consecutive clean cycles, which is
the 10.2 hysteresis: a command sitting exactly on the rail dithers in and
out of engagement by one quantisation step, and without this the alarm
chatters.
`REASSERT_S` and `CLEAR_CYCLES` are module constants, not options, for the same
reason `freeze.RAIL_FRAC` is: they describe the event bus, not the plant, and
every knob 10.3 asked to keep exposed is a LOOP TUNING knob and is untouched by
this module. No new config key is added anywhere - CFG-03 already has three
places a default has to be kept in step and this ticket does not earn a fourth.
------------------------------------------------------------------------------
NOT A TRIP
Clamping does not stop the loop and does not command 0 W. The clamped value is
written and control continues, which is what 11.2 asks for ("Clamp and alarm")
and what distinguishes this from SAFETY-01/02, whose action is 0 W. It follows
that this must NOT drive the failsafe status - `publish()` keeps reporting
"running", because the controller is.
"""
import math
from dataclasses import dataclass
from datetime import datetime
# The reason code, distinct from any saturation-freeze code by construction:
# nothing else in this add-on uses it. See "DISTINCT FROM SATURATION".
REASON_CODE = "OUTPUT_CLAMP"
# Re-assert the notification this often while the clamp stays engaged. 30 s is
# the planning cadence (2.3), i.e. slow enough not to flood and fast enough
# that a pinned loop is never quiet for a whole billing quarter.
REASSERT_S = 30.0
# Consecutive clean cycles before the alarm de-asserts (10.2 hysteresis).
CLEAR_CYCLES = 3
@dataclass(frozen=True)
class ClampResult:
"""What the guard did, with the RAW value preserved.
AC 2 turns on "the raw computed value differed from the clamped one", so the
raw value has to survive the call - a guard that returns only the applied
number cannot be asked the question the acceptance criterion asks.
"""
requested_w: float
applied_w: float
limit_w: float
engaged: bool
bound: str = "" # "" | "max_w" | "device" - which stage bit
def describe(self) -> str:
return (f"{REASON_CODE}: the computed setpoint {self.requested_w:+.0f} W "
f"was clamped to {self.applied_w:+.0f} W by the {self.bound} "
f"bound (rating {self.limit_w:.0f} W) - something upstream is "
f"winding")
def is_commandable(value) -> bool:
"""False for None, NaN and the infinities.
A non-finite setpoint is LOSS OF INPUT, not an out-of-range value, and
11.1 rule 1 says loss of input means command zero and keep commanding. It
must not reach the clamp arithmetic: `min(limit, nan)` returns nan on
CPython, so a nan would sail straight through a naive clamp and then report
`engaged` True on the `applied != requested` test - because nan != nan - i.e.
a transport bug filed in the winding-loop counter, with a nan on the wire.
"""
if value is None:
return False
try:
return math.isfinite(float(value))
except (TypeError, ValueError):
return False
def clamp_to_rating(requested_w, limit_w, dev_min=None, dev_max=None) -> ClampResult:
"""Bound a computed setpoint to the validated rating. The ONLY clamp.
`limit_w` the validated magnitude bound - `max_w`. Symmetric, because
3.2 calls `setpoint_limit_w` a MAGNITUDE clamp: it binds import
(+) and export (-) identically.
`dev_min` / `dev_max` the device's own asymmetric range, applied second and
expected never to bite. See the module note.
Public, and the predicate lives here rather than in the test, for the reason
`freeze.looks_frozen()` gives: a safety expression transcribed into its own
check is one copy plus a future disagreement about what the check means.
EXACTLY +-`limit_w` IS NOT AN ENGAGEMENT. The bound is inclusive; a loop
resting on the rail is doing what it was told, and alarming there would fire
on every healthy full-power hour.
"""
requested = float(requested_w)
limit = abs(float(limit_w))
# ⚠️ BOTH STAGES ALWAYS RUN. THIS USED TO `return` FROM THE FIRST ONE, AND
# THAT WAS A REGRESSION AGAINST THE CODE IT REPLACED. `write_setpoint()`
# previously clamped to the device range unconditionally, under the comment
# "never send a value the device will reject outright"; short-circuiting here
# meant the device bound was applied only when the rating bound had NOT
# fired, i.e. THE BIGGER THE OVERSHOOT THE LESS THE GUARD DID. Measured on
# the shipped function at max_w 5000 against a device of +-4000:
#
# +9999 -> +5000 bound='max_w' <- outside the device range
# +4500 -> +4000 bound='device'
#
# It also undercut this module's own argument for clamping to `max_w` rather
# than a runtime `min()`, which rests entirely on this stage catching the
# case where the startup validation did not run.
applied = max(-limit, min(limit, requested))
bound = "max_w" if applied != requested else ""
# Second stage. Arithmetically dead after a successful startup validation -
# and therefore worth alarming on precisely BECAUSE it fired.
if dev_min is not None and dev_max is not None:
bounded = max(float(dev_min), min(float(dev_max), applied))
if bounded != applied:
bound = "device" if not bound else "max_w+device"
applied = bounded
return ClampResult(requested, applied, limit, applied != requested, bound)
class ClampWatchdog:
"""One instance per controller. Counts every engagement, alarms sparsely.
`note()` is called once per write with whether the clamp bit; the caller ORs
in the loop's own `i_clamped` / `clamped` flags, which is where the winding
is actually visible on a default install (see the module note).
"""
def __init__(self):
self.active = False
self.engagements = 0 # every engagement, never deduplicated
self.episodes = 0 # rising edges
self.since: datetime | None = None
self.last_alarm: datetime | None = None
self.reason = ""
self.clean_cycles = 0
def note(self, engaged: bool, reason: str, now: datetime) -> bool:
"""Advance one cycle. True when the caller should raise the alarm.
`now` is the caller's clock, the same discipline as
`SignWatchdog.evaluate()` and `FreezeWatchdog.evaluate()`.
"""
if engaged:
self.engagements += 1
self.clean_cycles = 0
self.reason = reason
if not self.active:
self.active = True
self.since = now
self.episodes += 1
self.last_alarm = now
return True
# Still engaged: re-assert on the slow cadence, not every cycle.
if (self.last_alarm is None
or (now - self.last_alarm).total_seconds() >= REASSERT_S):
self.last_alarm = now
return True
return False
if not self.active:
return False
self.clean_cycles += 1
if self.clean_cycles >= CLEAR_CYCLES:
self.active = False
self.since = None
self.last_alarm = None
self.reason = ""
self.clean_cycles = 0
return False
+39 -15
View File
@@ -64,6 +64,23 @@ class Decision:
# and quantisation. Carry it back in as `i_w` next cycle; that is what keeps
# it a separate quantity from the command.
i_w: float = 0.0
# SAFETY-03. Did a clamp actually engage this cycle - i.e. did the RAW
# computed value differ from the bounded one?
#
# ⚠️ THESE EXIST BECAUSE `reason` CANNOT ANSWER THAT QUESTION. `reason` is
# one string and every later stage overwrites it: a cycle that clamps the
# integrator and is then slew limited reports "slew-limited", and one that
# clamps while saturated reports "saturated-freeze". Both ARE the winding
# case, and on the string the clamp signal is gone before anything can alarm
# on it. These two are written where the clamp happens and are never
# overwritten, so they mean exactly what they say and nothing else.
#
# `i_clamped` is the one that fires on a default install: the integrator
# bound is reached first, so `clamped` is unreachable unless
# `integrator_max_w` is deliberately set above `max_w`. Alarm on BOTH - see
# the comment at the bound below, and clamp.py.
i_clamped: bool = False
clamped: bool = False
def compute(
@@ -95,19 +112,23 @@ def compute(
# lets slew be larger than saturation_w.
#
# The spec states this window twice and differently: "> 10 s" (§11.2) and
# "3 samples" (§10.3). This counts CYCLES, and a cycle is not a unit of
# time: run_control() calls cycle() only when the meter value CHANGES
# (`if self.grid != last_grid`), so three cycles is three distinct meter
# readings and nothing more. At the reference P1's ~5 s update rate that is
# usually ~15 s, but there is no upper bound on it - a meter that repeats a
# value stalls the counter.
# "3 samples" (§10.3). This counts CYCLES, and a cycle is still not a unit
# of time - but it is now a BOUNDED one. run_control() runs a cycle on a
# changed meter value OR on `cycle_floor_s` elapsing, whichever comes first
# (SAFETY-12), so three cycles is at most 3 x cycle_floor_s = 45 s at the
# defaults, and usually ~15 s at the reference P1's ~5 s update rate.
#
# That is a detection-latency limit, not a windup hazard: the same
# condition that stalls the counter stalls the whole loop, so nothing
# accumulates in the meantime either. If a wall-clock window is ever
# required, it belongs in Controller (which has a clock) and not here.
# ponytail: this function is worth keeping clockless; the ceiling is that
# saturation_cycles cannot express a guaranteed number of seconds.
# ⚠️ THIS COMMENT USED TO CLAIM there was no upper bound and that it did not
# matter, "because the same condition that stalls the counter stalls the
# whole loop, so nothing accumulates in the meantime". The first clause was
# true and the second was the bug: a stalled loop does not stop commanding,
# it goes on emitting its last command through the heartbeat. Windup was
# never the hazard; staleness was.
#
# The wall-clock window lives in Controller, which has a clock, exactly as
# this comment always said it should. ponytail: this function stays
# clockless; the remaining ceiling is that saturation_cycles expresses a
# bound in seconds rather than an exact number of them.
saturated_now = abs(prev_w - actual_w) > tuning.saturation_w
sat_count = min(sat_count + 1, 10) if saturated_now else 0
frozen = sat_count >= tuning.saturation_cycles
@@ -184,7 +205,8 @@ def compute(
# bound is not. It is what makes the worst-case unwind time finite and
# knowable instead of a function of how long the error happened to stand.
bounded = max(-limit, min(limit, i_w))
if bounded != i_w:
i_clamped = bounded != i_w
if i_clamped:
# ⚠️ SAFETY-03 (alarm whenever the loop winds into a rail) must watch
# for THIS, not for "clamped" below. At the default limit == max_w the
# integrator bound is reached first and the command derived from it can
@@ -205,7 +227,8 @@ def compute(
# --- ORDER MATTERS: clamp -> slew -> freeze ----------------------------
# An early draft applied a floor after the clamp and let demand escape it.
target = max(-tuning.max_w, min(tuning.max_w, want))
if target != want:
clamped = target != want
if clamped:
reason = "clamped"
slewed = max(prev_w - tuning.slew_w, min(prev_w + tuning.slew_w, target))
@@ -231,7 +254,8 @@ def compute(
step = max(1, int(tuning.step_w))
target = round(target / step) * step
return Decision(float(target), sat_count, frozen, reason, float(i_w))
return Decision(float(target), sat_count, frozen, reason, float(i_w),
i_clamped, clamped)
def maintenance_charge_floor(
+275
View File
@@ -0,0 +1,275 @@
"""SAFETY-18 - the frozen-but-fresh meter detector.
WHAT IT CATCHES, in one sentence: a meter that keeps answering 200 OK with a
document that never changes, which every freshness mechanism in this stack reads
as perfectly healthy while the loop winds to the rail and stays there.
THE FAILURE NEEDS NO FAULT ANYWHERE ELSE. A frozen-but-talking meter is
*arriving* on every poll, so:
- `meter_max_age_s` is structurally blind. Every response resets the arrival
stamp (p1.py, `HomeWizardLocalSource`), so the age never climbs. Age catches
a meter that stops TALKING; this one stops CHANGING.
- firmware layer 1 is fed. Its watchdog keys on setpoint staleness, which the
heartbeat satisfies, and on `sensor.goodwe_p1_sample_age`, which a talking
meter satisfies.
- the arbiter contributes only the device rating and +-`max_w` - it bounds the
magnitude, which is the number this fault drives us to, not away from.
- there is no SoC guard on the grid-following loop at all. `soc_floor` belongs
to the maintenance cycle; the BMS is the sole energy backstop.
The loop reads a constant error and integrates it. Time from freeze to the rail,
replaying the real `control.compute()` at 15 s cycles:
meter frozen at to rail, max_w 2000 to rail, max_w 5000
350 W 150 s 360 s
1000 W 60 s 135 s
>= 3000 W 30 s 75 s
⚠️ THE LIVE INSTALL RUNS `max_w: 5000`, which is both the schema maximum and the
device rating, so the right-hand column is the one that describes this house.
------------------------------------------------------------------------------
THIS HAS A EURO FIGURE, AND IT IS THE REASON THE TICKET IS P1
`maintenance_charge_floor()` caps MAINTENANCE charging against `peak_cap_w`. The
grid-following loop is not capped by anything but `max_w`. A meter frozen
reading a large export drives a 5 kW unattended grid IMPORT, and >= 15 minutes
of that sets the month's billed capaciteitstarief peak: specs.md:583 records a
measured case where a single 4998 W quarter moved a monthly peak from 4832 W to
5013 W and billed the whole month at it.
⚠️ THIS CUT DOES NOT CAP THAT. It notifies. See DOCS.md, "Capacity-tariff
exposure of the grid-following loop", and SAFETY-07.
------------------------------------------------------------------------------
WHY IT IS A CONJUNCTION AND NOT AN `unchanged_s` THRESHOLD
`unchanged_s` (p1.py) already measures exactly the right thing: seconds since
the meter served a DIFFERENT measurement, fingerprinted over the power figure
and both energy registers. It is computed, it is on the status page, and TEL-05
deliberately refused to threshold it - correctly:
this controller regulates grid power toward ~0 W, and at a converged -10 W
the export register takes about six minutes to advance by its 1 Wh
resolution while the power figure legitimately repeats.
So a bare `unchanged_s > N` false-trips at the exact operating point we aim for,
and `p1.py` says as much in a note that ends "whoever wires a freeze detector
must handle the low-power case first". This is that handling, and it is not a
smaller number:
unchanged_s >= hold_s AND |commanded_w| is at the rail
The second term is what TEL-05's case cannot satisfy. At the converged point the
command is nowhere near +-`max_w`; on the 40 minutes of live converged behaviour
in `sim/scenarios/p1-age-2026-08-26.jsonl.gz` the meter never once leaves
+-500 W, so the loop never even approaches a 5000 W rail. And the reverse case -
a command genuinely pinned at the rail by a real 8 kW load - cannot satisfy the
FIRST term, because a real railed condition means kilowatts still crossing the
meter, and a kilowatt advances a 1 Wh register every 3.6 s.
The two terms fail for independent reasons. That is the whole design.
⚠️ AND THE SECOND HALF OF THAT WAS MEASURED ON THE RIG, WITH A RESULT WORTH
READING CAREFULLY. Driving the real transport against the rig's own hwsim under
+8000 W of house load - a genuinely railed command, meter perfectly healthy - the
POWER FIGURE DID NOT MOVE AT ALL: `active_power_w` sat at exactly 10130.0 W for
26 consecutive polls. What kept `unchanged_s` pinned at 0.0 s was the kWh
registers advancing, and nothing else.
So the register terms in `measurement_fingerprint()` are not belt-and-braces
here, they are the entire negative case. A freeze detector fingerprinted on the
power figure alone would have raised a false alarm on that rig run - a healthy
meter, a real load, a correctly railed command - which is the exact failure mode
that gets a safety guard switched off. Anyone tempted to simplify the fingerprint
should read that run first.
------------------------------------------------------------------------------
WHERE `hold_s` COMES FROM
Measured on `sim/scenarios/p1-age-2026-08-26.jsonl.gz` - 2399 s (40 min) of the
live house at the converged operating point, mean grid -14.2 W, 1194 polls:
polls that repeated the previous reading 735 of 1193 (61.6 %)
distinct-reading transitions 458
median gap between distinct readings 6.0 s
p95 gap 6.0 s
LARGEST `unchanged_s` any sample observes 8.0 s
longest interval serving no distinct reading 15.4 s
samples outside +-500 W 0
The default 300 s is 37x the largest `unchanged_s` this house has been recorded
producing while converged, and 19.5x the longest interval in which it served no
distinct reading at all. The two figures differ because the capture has an 11.4 s
hole in its OWN polling; the second is quoted because it is the pessimistic one.
300 s is not fitted to either margin - it is picked as "clearly past any
plausible register-tick interval, and clearly inside the 900 s quarter that bills
the capacity tariff". test_freeze.py re-measures both every run, so this
paragraph cannot quietly stop being true.
⚠️ BE PRECISE ABOUT WHAT THAT CAPTURE PROVES. It is POWER-ONLY - the shipped
fingerprint also covers both kWh registers, so a real `unchanged_s` on that data
can only be shorter, never longer, which makes it a conservative measurement of
the converged case. But it does NOT on its own refute a bare `unchanged_s`
threshold, and saying it did would be exactly the overstatement this project
keeps having to walk back: at 8 s worst, a bare 300 s threshold is silent on it
too. The case that refutes the bare threshold is the one below, which no capture
of THIS meter contains because this meter's power figure jitters.
The floor on the other side: at the rail, the smallest residual grid flow that
still ticks a 1 Wh register inside 300 s is 12 W. Any condition that genuinely
pins this loop at +-5000 W has three orders of magnitude more than that crossing
the meter.
⚠️ AND THOSE TWO NUMBERS DO NOT MERELY HAVE MARGIN - THEY CANNOT BOTH BE TRUE.
The worry the conjunction has to answer is "a HEALTHY meter whose register is
slower than `hold_s`". At 300 s that is |grid| < 12 W. The shipped `deadband_w`
is 15 W, so anywhere the register is that slow the control law is already inside
its deadband and the command is parked - the rail term is not merely unlikely to
be true there, it is 0 W. The two terms are mutually exclusive by construction
at the defaults, not just far apart, and the check re-derives that every run.
That relationship is the one to protect. `hold_s` may be lowered only while
3600/hold_s stays under `deadband_w`; at the schema floor of 60 s that is 60 W
against a 15 W deadband, and the mutual exclusion is gone. A 60 s `meter_freeze_s`
is a supported configuration that gives up this argument and keeps only the
margin - do not set one without a capture that says the margin is enough.
⚠️ WHAT `hold_s` DOES NOT BUY IS DETECTION SPEED. The alert lands at
max(hold_s, time-to-rail) after the freeze, so at the shipped default and
`max_w: 5000` that is 300-360 s. The rail is REACHED first, from 75 s. A
notification is not a brake; see the DOCS.md note.
------------------------------------------------------------------------------
THE RAIL TERM, PRECISELY
abs(commanded_w) >= RAIL_FRAC * max_w
`commanded_w` is the EMITTED setpoint - what the arbiter resolved and
`write_setpoint()` put on the wire - not the loop's raw wish, for the same
reason SAFETY-02 uses the emitted value: the wire carries the emitted number.
RAIL_FRAC is 0.95 and is a constant rather than an option. The command is
quantised to `step_w` (max 100) and clamped to `max_w` (min 100), so at the rail
the emitted value IS `max_w` on every schema-legal combination; the 5 % is slack
for a device rating that clips `max_w` slightly in `write_setpoint()`, not a
tuning knob. Making it configurable would offer a way to widen the one term that
carries the whole no-false-positive argument.
⚠️ NOT GATED ON `following`, and that is deliberate. A maintenance charge
commands against the meter on purpose and can sit at the rail for hours - but it
does so while pushing kilowatts through the meter, so the FIRST term is what
excludes it, and it excludes it on physics rather than on bookkeeping. Gating on
`following` as well would additionally blind the detector to a maintenance phase
running against a meter that really has frozen, which is the same fault with a
worse actuator behind it.
------------------------------------------------------------------------------
NOTIFY, DO NOT TRIP
First cut is an alarm and nothing else: an HA persistent notification, an ERROR
in the log, an event on the status page, and a failed check row. It does NOT
claim the actuator, does NOT file a P_SAFETY claim and does NOT latch.
That is a deliberate ordering of costs, not timidity. A false trip here parks
the battery - on a house that is running this loop precisely to avoid buying
power at the wrong moment - and the operator's response to a parked battery is
to switch the guard off. A false notification costs one alert. The trip is a
separate decision that has to be earned with observed data from a real install,
and it needs an answer to a question this cut does not have one for: what should
the loop do instead, given that 0 W is itself a grid import of whatever the
house is drawing.
⚠️ TRANSPORT-CONDITIONAL. `unchanged_s` exists ONLY on `homewizard_local`,
because it is the only transport that sees the meter's own document. Under
`ha_dsmr`, `mqtt_p1`, `ha_signed` and `meter_source: off` this detector is
silent and the exposure above is uncovered. The caller passes None and gets
False; there is no half-measure available on those paths, and pretending
otherwise would be worse than the gap.
"""
from dataclasses import dataclass
from datetime import datetime
# See "THE RAIL TERM, PRECISELY" above. A constant, on purpose.
RAIL_FRAC = 0.95
@dataclass(frozen=True)
class FreezeWatchdogConfig:
"""Default, not a constant - config.yaml carries `meter_freeze_s`."""
hold_s: float = 300.0
def rail_w(max_w) -> float:
"""The magnitude at or above which a command counts as railed."""
return abs(float(max_w)) * RAIL_FRAC
def looks_frozen(unchanged_s, commanded_w, max_w,
cfg: FreezeWatchdogConfig) -> bool:
"""The predicate, and the ONLY place the conjunction is written down.
Public for the same reason `watchdog.disagrees()` is: the checks that grade
it must assert on THE SHIPPED EXPRESSION. A safety predicate transcribed
into its own test is one copy plus a future disagreement about what the
check means, and the copy that drifts is always the one doing the grading.
`unchanged_s` seconds since the meter served a different measurement, or
None on a transport that cannot know (see the module note).
`commanded_w` the EMITTED setpoint standing on the wire.
`max_w` the configured output bound the rail is measured against.
"""
if unchanged_s is None or commanded_w is None or not max_w:
return False
return (float(unchanged_s) >= cfg.hold_s
and abs(float(commanded_w)) >= rail_w(max_w))
class FreezeWatchdog:
"""One instance per controller, evaluated once per control cycle.
Level-triggered state with an EDGE for the alarm: `evaluate()` returns True
on the cycle the conjunction first becomes true and False on every cycle
after, so the caller raises one notification per episode rather than one
every second. `active` is the level, for the status page.
No latch, deliberately - see "NOTIFY, DO NOT TRIP". A latch exists so a trip
cannot erase its own evidence by working; nothing here acts, so there is
nothing to erase. When the meter starts moving again the alert clears
itself, which is the honest report of the state.
"""
def __init__(self, cfg: FreezeWatchdogConfig | None = None):
self.cfg = cfg or FreezeWatchdogConfig()
self.active = False
self.since: datetime | None = None
self.reason = ""
self.episodes = 0
def evaluate(self, unchanged_s, commanded_w, max_w, now: datetime) -> bool:
"""Advance one cycle. Returns True ONLY on the rising edge.
`now` is the caller's clock so this stays testable and replayable - the
same discipline as `SignWatchdog.evaluate()`.
"""
if not looks_frozen(unchanged_s, commanded_w, max_w, self.cfg):
self.active = False
self.since = None
self.reason = ""
return False
self.reason = (
f"METER_FROZEN: the meter has answered {float(unchanged_s):.0f} s "
f"without its measurement changing while the command stands at "
f"{float(commanded_w):+.0f} W, at the {float(max_w):.0f} W rail "
f"(limits: unchanged over {self.cfg.hold_s:.0f} s, command over "
f"{rail_w(max_w):.0f} W)")
if self.active:
return False
self.active = True
self.since = now
self.episodes += 1
return True
+479 -10
View File
@@ -36,11 +36,17 @@ from .arbiter import (
P_LOOP, P_MAINT_SHAPE, P_MAINTENANCE, P_SAFETY, Claim, resolve,
)
from .control import Tuning, compute, maintenance_charge_floor, peak_at_risk
from .clamp import (
REASON_CODE as CLAMP_REASON, ClampResult, ClampWatchdog, clamp_to_rating,
is_commandable,
)
from .freeze import FreezeWatchdog, FreezeWatchdogConfig, rail_w
from .hass import HomeAssistant
from .maintenance import IDLE, MaintConfig, Maintenance
from .mqtt import MqttPublisher
from .p1 import P1Ingest, build_source, is_enabled
from .validate import ConfigRefused, refuse, validate_startup_config
from .watchdog import SignWatchdog, SignWatchdogConfig, loop_is_following
from . import web
OPTIONS_PATH = "/data/options.json"
@@ -103,6 +109,54 @@ class Controller:
store,
)
# SAFETY-02. One instance, one actuator, one cadence - see watchdog.py.
#
# ⚠️ RESTORED FROM THE STORE, and that is the difference between a latch
# and a pause. A trip held only in memory is cleared by the one action an
# operator staring at a FAILSAFE banner is most likely to take - restart
# the add-on - and the controller then comes back armed, into the fault,
# with nobody involved. Restored here, the banner and the 0 W survive the
# restart and only `auto_toggle` clears them.
self.sign_wd = SignWatchdog(
SignWatchdogConfig(
min_w=float(opts.get("sign_disagree_w", 300)),
hold_s=float(opts.get("sign_disagree_s", 180)),
),
tripped=bool(store.data.get("sign_tripped", False)),
reason=str(store.data.get("sign_reason", "") or ""),
)
if self.sign_wd.tripped:
_LOG.error("SAFETY-02 was TRIPPED before this restart and stays "
"tripped: %s. The inverter is held at 0 W. Restarting the "
"add-on does not clear this - start control from the UI "
"once you know why the command opposed the meter.",
self.sign_wd.reason or "(reason not recorded)")
# SAFETY-18. The frozen-but-fresh meter detector - see freeze.py for why
# it is a conjunction and not an `unchanged_s` threshold, and for the
# measurements behind the default.
#
# ⚠️ NOT restored from the store, unlike SAFETY-02's latch, and the
# asymmetry is the point: this one does not latch because it does not
# act. A restart re-derives its state from the meter within one
# `meter_freeze_s` window, and until then the honest report is "not
# alarming yet" rather than an alarm nobody can clear.
self.freeze_wd = FreezeWatchdog(
FreezeWatchdogConfig(hold_s=float(opts.get("meter_freeze_s", 300))))
# SAFETY-03. The output clamp's alarm - see clamp.py for why the clamp
# itself is the cheap half.
#
# ⚠️ NO CONFIG OF ITS OWN, on purpose. The bound it clamps to is `max_w`,
# which is already an option, already validated against the device's
# rating at startup, and already used by Tuning and the arbiter. Giving
# this a second key would be a fourth place a limit has to be kept in
# step (CFG-03 already tracks three) and, worse, would recreate exactly
# the two-numbers-that-disagree failure the §14 warning box describes.
# The anti-flood cadence is a module constant for the same reason
# freeze.RAIL_FRAC is: it describes the event bus, not the plant.
self.clamp_wd = ClampWatchdog()
# P1 ingestion (TEL-01). `meter_source: off` keeps the original
# single-entity meter_entity path, so an existing install is unchanged
# until it opts in.
@@ -120,6 +174,11 @@ class Controller:
self.sat_count = 0
self.i_w = 0.0 # the loop's integrator, carried between cycles
self.loop_w = None # what the loop asked for last cycle, or None
# Did the grid-following loop actually win the LAST arbitration? That is
# the question SAFETY-02 has to ask about the command standing on the
# wire right now, so it is recorded when the arbitration happens rather
# than re-derived a cycle later. Starts False: nothing is on the wire.
self.following = False
self.reason = "starting"
self.grid = self.soc = self.batt = None
self.peak_fc = None
@@ -129,6 +188,11 @@ class Controller:
self.last_write_ok = None
self.inputs_bad_since = None
self.dev_min = self.dev_max = None
# SAFETY-03. What the last write's clamp did, and what the loop's own
# bounds did last cycle. `loop_clamped` is a (engaged, note) pair so the
# alarm can say WHICH bound bit without re-deriving it from a string.
self.clamp_last: ClampResult | None = None
self.loop_clamped: tuple[bool, str] = (False, "")
self.events: list[str] = []
self.stopping = False
@@ -187,11 +251,37 @@ class Controller:
entity = self.o.get("setpoint_entity", "")
if not entity:
return
# Never send a value the device will reject outright. A rejected write is
# silent from the controller's point of view, and the inverter then keeps
# doing whatever it was already doing.
if self.dev_min is not None and self.dev_max is not None:
value = max(self.dev_min, min(self.dev_max, value))
# --- SAFETY-03: THE CLAMP, AND IT IS ON THE ONLY PATH TO THE WIRE -----
# Every write in this add-on comes through here - the cycle, the
# heartbeat, the operator's stop button and shutdown() - so no optimizer,
# maintenance or manual path can route around it. That placement is the
# first acceptance criterion, and it is why the guard is not in
# control.py: a guard the control law owns is a guard every other claimer
# bypasses.
#
# ⚠️ A NON-FINITE VALUE IS LOSS OF INPUT, NOT AN OUT-OF-RANGE ONE. §11.1
# rule 1: command zero and keep commanding. It must not reach the clamp
# arithmetic - see is_commandable() for what a nan does to it - and it
# must not be counted as a clamp engagement, or a transport bug lands in
# the winding-loop counter.
if not is_commandable(value):
self.log_event(f"non-finite setpoint {value!r} - commanding 0 W")
_LOG.error("refusing to write a non-finite setpoint %r - commanding "
"0 W and continuing to command. This is a bug upstream, "
"not a limit.", value)
value = 0.0
result = clamp_to_rating(value, float(self.o.get("max_w", 2000)),
self.dev_min, self.dev_max)
self.clamp_last = result
if result.engaged:
# ⚠️ EVERY engagement, never deduplicated: this is the action log
# (§15), and it is a different surface from the notification, which
# IS deduplicated. The count is what says how hard the loop pushed;
# the notification only says that it did.
self.log_event(result.describe())
value = result.applied_w
changed = (self.last_write is None or value != self.last_write)
if not (changed or force):
return
@@ -214,8 +304,38 @@ class Controller:
for msg in result.events:
self.log_event(msg)
# --- SAFETY-02, and it runs BEFORE the early return below -------------
# ⚠️ ORDER MATTERS HERE. If this were evaluated after the "inputs missing
# but not yet stale" hold, a run of held cycles would leave
# `disagree_since` untouched across the gap and the timer would resume
# from a start time that predates it. It is evaluated every cycle,
# unconditionally, so a missing meter CLEARS the window - see watchdog.py.
#
# The value compared is the command STANDING ON THE WIRE, which is what
# this cycle's fresh meter reading is a response to. `last_write` is that
# value after the device clamp and confirmed accepted; `target` covers
# the case where nothing has been written yet (and, deliberately, the
# case where the last write was REJECTED - the inverter is then still
# latched on `last_write`, which is exactly what `last_write` holds).
was_tripped = self.sign_wd.tripped
standing = self.last_write if self.last_write is not None else self.target
if self.sign_wd.evaluate(standing, self.grid, self.following, now) \
and not was_tripped:
# ⚠️ REASON FIRST, FLAG SECOND, and each `set` writes the file. The
# ordering is the whole atomicity story: a crash between the two
# persists a reason nothing reads, which is harmless, where the other
# order persists a latch with no explanation for the operator who has
# to decide whether to clear it.
self.store.set("sign_reason", self.sign_wd.reason)
self.store.set("sign_tripped", True)
self.log_event("FAILSAFE " + self.sign_wd.reason)
_LOG.error("SAFETY-02 TRIPPED - %s. Commanding 0 W and holding it "
"until control is started again. This survives an add-on "
"restart; restarting will not clear it.",
self.sign_wd.reason)
stale_after = float(self.o.get("stale_input_s", 15))
if not self.inputs_ok:
if not self.inputs_ok and not self.sign_wd.tripped:
bad_for = (now - self.inputs_bad_since).total_seconds() if self.inputs_bad_since else 0.0
if bad_for < stale_after:
# A single missed poll is not a fault. Hold, keep the heartbeat
@@ -235,6 +355,16 @@ class Controller:
claims.append(Claim.limit("supervised", P_SAFETY, -max_w, max_w, "max_w"))
# --- safety stops ----------------------------------------------------
# ⚠️ SAFETY-02 is claimed FIRST and unconditionally while latched. It is
# a SET at P_SAFETY, so it outranks maintenance, peak shaving and the
# loop alike - a controller pushing power the wrong way is wrong
# whatever it thinks it is doing. Re-filed EVERY cycle: claims are not
# sticky, and one filed once would lapse the next cycle and hand the
# actuator silently back to the loop that was driving the wrong way.
if self.sign_wd.tripped:
claims.append(Claim.set("failsafe", P_SAFETY, 0.0,
"sign disagreement"))
if not self.auto:
claims.append(Claim.set("safety", P_SAFETY, 0.0, "stopped"))
elif not self.inputs_ok:
@@ -297,19 +427,201 @@ class Controller:
self.sat_count = decision.sat_count
self.i_w = decision.i_w
self.loop_w = decision.target_w
# --- SAFETY-03 ---------------------------------------------------
# ⚠️ READ THE BOOLEANS, NOT `decision.reason`. On a default install
# the integrator bound is what bites and the output clamp is
# unreachable (control.py says so), and `reason` is then overwritten
# by whichever of slew-limiting or the saturation freeze ran after
# it - so a string test is silent on the two cases a winding loop
# most often presents as. These flags are set where the clamp
# happens and nothing downstream touches them.
self.loop_clamped = (
decision.i_clamped or decision.clamped,
(f"the loop asked for {decision.i_w:+.0f} W"
+ (" (integrator bound)" if decision.i_clamped else "")
+ (" (output bound)" if decision.clamped else "")),
)
claims.append(Claim.set("loop", P_LOOP, decision.target_w, decision.reason))
else:
# Stopped or blind: no accumulation may survive the outage.
self.i_w = 0.0
self.loop_w = None
# A loop that is not running cannot be winding. Clearing this here
# rather than letting the last value stand is what makes the alarm
# de-assert when control is stopped.
self.loop_clamped = (False, "")
resolution = resolve(claims)
if resolution.contradiction:
self.log_event("ARBITER CONTRADICTION - commanding 0 W, see the log")
# --- SAFETY-02's arming gate ------------------------------------------
# Was the number about to go on the wire chosen by the grid-following
# loop, or by a strategy that commands against the meter on purpose?
# Only the former is expected to agree in sign with P1 - without this,
# every maintenance grid charge trips the failsafe within three minutes.
#
# ⚠️ THE EXPRESSION LIVES IN watchdog.py, NOT HERE, and that is not
# tidiness. It was inline, test_watchdog.py held a transcription of it,
# and the P_PEAK case meant to force whoever lands peak shaving to come
# and change an expectation did not: editing this line alone to
# `winner in ("loop", "peak")` left every suite green while the watchdog
# went blind for every peak event. Read loop_is_following() before
# touching the gate - the forward hazard and the reasoning are there.
self.following = loop_is_following(resolution, self.loop_w)
self.target = resolution.target_w
self.reason = resolution.explain()
await self.write_setpoint(self.target)
# --- SAFETY-03, and its boundary is the WRITE ------------------------
# ⚠️ NOT THE SAME BOUNDARY AS SAFETY-18's BELOW, THOUGH THE CALLS ABUT.
# This one reads `self.clamp_last`, which is None until write_setpoint()
# has actually run, so it has to follow the write. The one below grades
# `self.target`, which is assigned further up - see check_output_clamp()
# and check_meter_frozen() for each. Two guards, two orderings, adjacent
# lines: do not "tidy" them into one rule.
await self.check_output_clamp()
# --- SAFETY-18, and its boundary is the ASSIGNMENT --------------------
# ⚠️ THE BOUNDARY IS `self.target = resolution.target_w` ABOVE, NOT
# write_setpoint(). This comment used to say "the command that has just
# gone on the wire", which is wrong twice over and was corrected by
# TEST-06's reviewer: `check_meter_frozen()` grades `self.target` - the
# arbiter's resolved intent - and `write_setpoint()` clamps only a local
# `value` and can skip the write entirely, so that number may never have
# gone anywhere.
#
# What both terms of the conjunction must describe is the SAME moment:
# the command this cycle resolved, against how long the meter that chose
# it has been standing still. Evaluated before
# `self.target = resolution.target_w` it would grade the PREVIOUS
# cycle's command against THIS cycle's meter - the one combination that
# was true of neither moment. Moving it between the assignment and
# write_setpoint() is therefore harmless; moving it above the assignment
# is not, and test_freeze.py part 5 fails on exactly that.
await self.check_meter_frozen()
async def check_output_clamp(self) -> None:
"""SAFETY-03. Alarm whenever a clamp actually engaged this cycle.
Runs AFTER write_setpoint() for the same reason SAFETY-18 does: the
clamp is arithmetic done inside the write, so before the write there is
nothing yet to report.
⚠️ PUBLISHING IS NOT IN THE WRITE PATH. The clamp itself is synchronous
inside write_setpoint(); this - the awaits, the notification, anything
that can block on Home Assistant - is out here. A stalled supervisor must
never be able to delay or drop a frame.
⚠️ THIS IS NOT A TRIP. No claim is filed, nothing is latched, 0 W is not
commanded and `publish()` still reports "running". §11.2's action for
this row is "clamp and alarm", and the clamped value is written and the
loop keeps running. Turning this into a trip means coming here and
changing that deliberately, not discovering it in the field.
"""
engaged_write = bool(self.clamp_last and self.clamp_last.engaged)
engaged_loop, loop_note = self.loop_clamped
engaged = engaged_write or engaged_loop
if engaged:
where = (self.clamp_last.describe() if engaged_write
else f"{CLAMP_REASON}: {loop_note}, above the "
f"{float(self.o.get('max_w', 2000)):.0f} W rating, and "
f"was clamped to {self.target:+.0f} W")
reason = (f"{where}. This is a BUG SIGNAL, not a limit being used: "
f"something upstream computed a setpoint the actuator "
f"cannot deliver.")
else:
reason = ""
fired = self.clamp_wd.note(engaged, reason, datetime.now(timezone.utc))
# --- THE ACTION LOG, AND IT IS COALESCED PER EPISODE ------------------
# Every engagement is recorded, but consecutive ones are recorded by
# UPDATING one row's count rather than inserting a row each.
#
# ⚠️ THAT IS NOT COSMETIC, AND IT WAS FOUND THE HARD WAY. `log_event()`
# keeps 40 rows; a winding loop clamps on every cycle, so a row per
# engagement flushes the whole ring within 40 cycles - and the first
# thing it evicted, in test_freeze.py, was SAFETY-18's "METER FROZEN?"
# row. A guard whose log drowns out the other guards' logs has moved the
# flood rather than removed it. The count here is `clamp_wd.engagements`,
# so nothing is lost: the row says exactly how many, and the number is
# read off the counter rather than off the number of rows.
#
# This lives here rather than only in write_setpoint() because on a
# default install the actuator-side clamp is unreachable - compute() has
# already bounded the integrator - so the only engagements that exist are
# loop-side ones. Measured: 60 cycles of the §10.3 runaway produce 0
# write-path engagements and 2 loop-side ones.
if engaged and not engaged_write:
tag = f"{CLAMP_REASON} episode {self.clamp_wd.episodes}"
row = (f"{tag}: {self.clamp_wd.engagements} engagement(s) - "
f"{reason}")
if self.events and tag in self.events[0]:
stamp = self.events[0].split(" ", 1)[0]
self.events[0] = f"{stamp} {row}"
else:
self.log_event(row)
if not fired:
return
_LOG.error(
"SAFETY-03 - %s Silently clipping this is how a winding integrator "
"reached 14 768 W on this project before: the clamp makes every "
"dashboard read a legal number while the thing producing it goes on "
"climbing. The command IS clamped and the loop is STILL RUNNING - "
"this is an alarm, not a trip. Engagements so far: %d.",
self.clamp_wd.reason, self.clamp_wd.engagements)
# A fixed notification_id, for the reason check_meter_frozen() gives -
# and a DIFFERENT one from every other alarm this add-on raises, because
# a clamp is not a frozen meter and neither is a saturation freeze.
await self.hass.call("persistent_notification", "create", {
"notification_id": "goodwe_output_clamp",
"title": "GoodWe: a computed setpoint exceeded the rating",
"message": (
f"{self.clamp_wd.reason}\n\nThe command has been clamped and "
f"the controller is STILL RUNNING. This is NOT the inverter "
f"failing to follow a legal command (that is the saturation "
f"freeze, which is normal and does not alarm) - the value asked "
f"for was itself out of range."),
})
async def check_meter_frozen(self) -> None:
"""SAFETY-18. Alarm on a meter that answers forever without changing.
Notification only - no claim, no latch, no 0 W. freeze.py carries the
reasoning and the euro figure; DOCS.md carries the exposure this does
NOT close.
⚠️ getattr, not an attribute access: `unchanged_s` exists only on
`homewizard_local`, and `p1_source` is None until amain() builds the
transport. Every other meter_source hands the predicate None and gets
False, which is the honest answer - see the transport note in freeze.py.
"""
unchanged = getattr(self.p1_source, "unchanged_s", None)
fired = self.freeze_wd.evaluate(unchanged, self.target,
float(self.o.get("max_w", 2000)),
datetime.now(timezone.utc))
if not fired:
return
self.log_event("METER FROZEN? " + self.freeze_wd.reason)
_LOG.error(
"SAFETY-18 - %s. The grid-following loop is regulating against a "
"number that has stopped moving, and nothing in this stack stops "
"it: the meter is still answering, so the age is fresh, the "
"firmware watchdog is fed and the arbiter only bounds the "
"magnitude. CHECK THE METER. Sustained for a quarter of an hour "
"this sets the month's billed capacity-tariff peak. This is an "
"ALARM ONLY - the loop is still running.", self.freeze_wd.reason)
# ⚠️ A fixed notification_id, so a second episode REPLACES the first
# rather than stacking. An operator who has to dismiss eleven identical
# notifications learns to dismiss the twelfth without reading it.
await self.hass.call("persistent_notification", "create", {
"notification_id": "goodwe_meter_frozen",
"title": "GoodWe: the P1 meter may be frozen",
"message": (
f"{self.freeze_wd.reason}.\n\nThe controller is STILL RUNNING "
f"and still following this reading. Check the meter; if it is "
f"stuck, stop control from the add-on UI."),
})
# -- tasks ---------------------------------------------------------------
async def run_control(self) -> None:
entity = self.o.get("setpoint_entity", "")
@@ -342,11 +654,85 @@ class Controller:
last_grid = object()
heartbeat = float(self.o.get("heartbeat_s", 10))
last_beat = 0.0
# ⚠️ SAFETY-12. The floor is a DEADLINE ON THE GAP BETWEEN CYCLES, not a
# second scheduler running alongside the arrival trigger. Every cycle
# resets it, whichever trigger fired, so on a healthy changing meter it
# never elapses and not one existing cycle moves - and therefore no
# change to the per-cycle gain the tuning was measured against.
#
# That is true twice over, and the second reason is the sturdier one:
# the combined cycle rate is max(arrival rate, 1/floor), never the sum.
# At the defaults 1/15 s is strictly slower than the ~1/5 s arrival
# rate, so even a floor that raced the arrivals could not push the loop
# past its nominal healthy rate. The re-tune is unreachable by
# construction, not merely by the reset. ⚠️ That argument holds only
# while floor > the meter's update interval - which is why the schema
# minimum of 5 s deserves the second look validate.py has not given it
# yet: 5 s IS the reference meter's interval, i.e. the one configurable
# value that would double the cycle rate.
#
# Why it has to exist, and it is not only about latency: `self.grid !=
# last_grid` alone means a meter that repeats a value runs NO control
# cycles at all. The heartbeat below keeps re-emitting the last command
# and the firmware keeps putting it on the wire, so every indicator
# reads healthy - and, worse, the `inputs missing -> 0 W` claim lives
# INSIDE cycle(), so on a repeating meter THAT SAFETY CLAIM CANNOT FIRE.
# Demonstrated in test_loop.py: constant meter, SoC entity dies at t=4 s,
# and the pre-fix controller holds 220 W through the whole outage with
# the status line still reading "loop -> 220 W". Layer 1 does not cover
# it on a `meter_source: off` install either - no P1 age sensor is
# published, so the firmware has nothing to veto on. Measured on the rig
# during TEL-05: 70 s at a literally constant 350.0 W, 14 arrivals, zero
# cycles.
#
# This is NOT a freeze detector, and deliberately so - see
# HomeWizardLocalSource.unchanged_s for why one cannot be built here (at
# the converged -10 W target a 1 Wh register needs ~6 minutes to move,
# so any freeze threshold false-trips at the exact operating point we
# run at). Nothing here decides whether a repeated reading is true. It
# only stops "the number is the same" from meaning "do not think".
#
# ⚠️ THE 15 s DEFAULT HAS A CEILING IT DOES NOT OWN. It is under
# validate.FIRMWARE_INGEST_VETO_S (30 s) and under meter_max_age_s on
# purpose, so the loop re-decides at least twice inside any window in
# which layer 1 might otherwise be the only thing deciding. If the
# firmware's veto ever moves, this default moves with it - it is not
# independently justified at 15.
floor = float(self.o.get("cycle_floor_s", 15))
last_cycle = None
while not self.stopping:
await self.read_inputs()
# A cycle is one meter update, exactly as on the reference install.
if self.grid != last_grid:
now_mono = asyncio.get_running_loop().time()
# A cycle is one meter update, exactly as on the reference install -
# or the floor elapsing, whichever comes first.
overdue = last_cycle is None or (now_mono - last_cycle) >= floor
if self.grid != last_grid or overdue:
last_grid = self.grid
last_cycle = now_mono
# ⚠️ The FULL cycle, not a degraded one. A floor cycle runs the
# same law on the same inputs as an arrival cycle, because in P1
# mode the freshness question is already answered upstream and
# answered harder: past `meter_max_age_s` (30 s) p1.net_w is
# None, which is "inputs missing", which becomes a P_SAFETY 0 W
# claim. So a floor cycle cannot run the PI on a 60 s old
# figure, because at 60 s there is no figure.
#
# ⚠️ THAT GUARANTEE IS TRANSPORT-CONDITIONAL, and saying so
# matters more than the guarantee: with `meter_source: off`,
# read_inputs() takes self.grid straight from hass.number() with
# NO age gate at all, so a Home Assistant entity that has gone
# quiet reads as a current value on that path with or without
# this change. Running the full law is still the right choice
# there - the alternative is a degraded second law reasoning
# about an age nobody measured - but the floor does not make
# that path fresh, and nothing here should be read as claiming
# it does. And the integrator
# advances per CYCLE, not per second (control.py), so bounding
# the gap bounds the accumulation: at most one extra
# gain x error per floor interval, under the same
# integrator_max_w as every other cycle. A second, gentler
# control law for floor cycles would be a second set of
# dynamics nobody can test on this hardware.
await self.cycle()
# ⚠️ The heartbeat is not an optimisation. The ESP32 treats silence
# longer than ~30 s as "the controller is gone" and zeroes the
@@ -365,7 +751,14 @@ class Controller:
"battery": self.batt,
"soc": self.soc,
"phase": self.maint.phase,
"status": "running" if self.auto else "stopped",
# ⚠️ SAFETY-02's alarm surface, and it is deliberately an existing
# one. IF-01/IF-02 own `ems/event/failsafe` and the `failsafe active`
# binary sensor and neither is built yet; inventing a bespoke topic
# here would be a surface to migrate later. This sensor is already
# discovered, already on every dashboard, and "failsafe" is
# additive to its vocabulary rather than a change to it.
"status": ("failsafe" if self.sign_wd.tripped
else "running" if self.auto else "stopped"),
}
# ⚠️ ONLY when P1 ingestion is actually running. The ESP32's stale-input
# watchdog subscribes to sensor.goodwe_p1_sample_age - see the entity-id
@@ -417,6 +810,33 @@ class Controller:
f"{self.p1.parse_errors} rejected"
+ (f", measurement unchanged for {unchanged:.0f} s"
if unchanged is not None else "")})
# SAFETY-18. Reported in both states, for the reason SAFETY-02 is:
# a detector nobody can see is armed is a detector nobody trusts
# when it fires. ok=False/warn=False so the banner goes red - this
# is an alarm, and the banner is the loudest surface we own. It
# changes NOTHING about control; see check_meter_frozen().
if self.freeze_wd.active:
out.append({"ok": False, "warn": False,
"text": "METER MAY BE FROZEN - " + self.freeze_wd.reason
+ ". The loop is STILL FOLLOWING it and is "
"still running. Check the meter. Held here "
"for a quarter of an hour this sets the "
"month's billed capacity peak."})
elif getattr(self.p1_source, "unchanged_s", None) is None:
# Not "armed". Every transport but homewizard_local is invisible
# to this check, and saying "armed" there would be a lie the
# status page tells on behalf of a guard that cannot run.
out.append({"ok": False, "warn": True,
"text": f"freeze detector: NOT COVERED on "
f"meter_source {o.get('meter_source')} - only "
f"homewizard_local sees whether the meter's own "
f"document changes"})
else:
out.append({"ok": True, "warn": False,
"text": f"freeze detector: armed (alerts after "
f"{self.freeze_wd.cfg.hold_s:.0f} s unchanged "
f"while the command is over "
f"{rail_w(self.o.get('max_w', 2000)):.0f} W)"})
rows = [("battery SoC", self.soc, o.get("soc_entity")),
("battery power", self.batt, o.get("batt_entity"))]
if not self.p1_enabled:
@@ -443,10 +863,44 @@ class Controller:
out.append({"ok": True, "warn": False,
"text": f"setpoint {sp} (range {self.dev_min:g}…{self.dev_max:g} W)"})
# SAFETY-03. Red while a clamp is engaged, and the engagement COUNT is
# shown even after it clears - a loop that wound once and stopped is
# still a bug, and a row that vanishes with the condition takes the
# evidence with it. Reported only when there is something to report:
# unlike the freeze detector this guard has no "not covered" state, it
# runs on every write on every transport.
if self.clamp_wd.active:
out.append({"ok": False, "warn": False,
"text": "OUTPUT CLAMP ENGAGED - " + self.clamp_wd.reason
+ f" Engagements: {self.clamp_wd.engagements}."})
elif self.clamp_wd.engagements:
out.append({"ok": False, "warn": True,
"text": f"output clamp: not engaged now, but it has "
f"engaged {self.clamp_wd.engagements} time(s) in "
f"{self.clamp_wd.episodes} episode(s) since "
f"start - see the event log"})
if self.last_write_ok is False:
out.append({"ok": False, "warn": False,
"text": "last write to the inverter was REJECTED - check the log"})
# SAFETY-02. Reported in both states: a watchdog nobody can see is armed
# is a watchdog nobody trusts when it fires.
if self.sign_wd.tripped:
out.append({"ok": False, "warn": False,
"text": "SIGN-DISAGREEMENT FAILSAFE - " + self.sign_wd.reason
+ ". The inverter is held at 0 W. Find out why the "
"command opposed the meter before starting control "
"again; starting clears this. Restarting the "
"add-on does NOT clear it."})
else:
held = self.sign_wd.held_s(datetime.now(timezone.utc))
out.append({"ok": True, "warn": False,
"text": f"sign watchdog: armed (trips after "
f"{self.sign_wd.cfg.hold_s:.0f} s opposed, both over "
f"{self.sign_wd.cfg.min_w:.0f} W)"
+ (f" - opposed for {held:.0f} s now" if held else "")})
if not o.get("estop_fitted", False):
out.append({"ok": False, "warn": True,
"text": "no e-stop fitted: if this host dies the battery stays "
@@ -464,7 +918,11 @@ class Controller:
def status_payload(self) -> dict:
checks = self.checks()
bad = [c for c in checks if not c["ok"] and not c["warn"]]
if bad:
if self.sign_wd.tripped:
# Outranks "NOT READY": nothing is missing, something went wrong.
level, banner = "bad", ("FAILSAFE — the command opposed the meter. "
"Inverter held at 0 W.")
elif bad:
level, banner = "bad", f"NOT READY — {bad[0]['text']}"
elif not self.auto:
level, banner = "warn", "Stopped — inverter commanded to 0 W"
@@ -498,6 +956,17 @@ class Controller:
self.auto = not self.auto
self.store.set("auto", self.auto)
self.log_event("control started" if self.auto else "control stopped")
if self.auto and self.sign_wd.tripped:
# ⚠️ THE ONLY WAY OUT OF A SIGN-DISAGREEMENT TRIP, on purpose.
# It clears on a deliberate human start, not on the condition
# going away - the condition ALWAYS goes away, because the trip
# commands 0 W and a 0 W command cannot disagree with anything.
# An operator pressing start has, at minimum, looked at the
# banner that says why.
self.sign_wd.reset()
self.store.set("sign_tripped", False)
self.store.set("sign_reason", "")
self.log_event("sign-disagreement failsafe cleared by operator")
if not self.auto:
self.target = 0.0
await self.write_setpoint(0.0, force=True)
+4 -3
View File
@@ -28,9 +28,10 @@ _LOG = logging.getLogger("goodwe.maint")
IDLE, DRAIN, CHARGE, HOLD = "idle", "drain", "charge", "hold"
# A drain of a 13.5 kWh usable pack at 2500 W is ~5 h even with no house load
# helping. 12 h means the battery is not following commands - a fault, not a
# slow day. The charge phase deliberately waits for sun and for cheap hours, so
# A drain of a 13.8 kWh usable pack (300 Ah x 51.2 V, DoD 90) at the shipped
# 2000 W is ~6.9 h even with no house load helping - and less in practice, since
# house load drains alongside the export and soc_floor stops short of empty.
# 12 h means the battery is not following commands - a fault, not a slow day. The charge phase deliberately waits for sun and for cheap hours, so
# it is expected to span a night; 36 h still bounds it.
DRAIN_TIMEOUT_MIN = 720
CHARGE_TIMEOUT_MIN = 2160
+23 -2
View File
@@ -22,11 +22,30 @@ _LOG = logging.getLogger("goodwe.store")
# `auto_start` option only when the key is absent - if a default supplied False,
# the fallback could never fire and auto_start would silently do nothing on
# every fresh install.
#
# ⚠️ `sign_tripped` IS in here, and the difference from `auto` is the whole
# reason both decisions are written down. `auto` has an option to fall back to
# (`auto_start`), so a stored default would mask a fresh install. A latched
# failsafe has no such fallback: absent means "never tripped", which is exactly
# what a default of False says.
#
# It is persisted for the reason this module's docstring gives for maintenance
# state, applied to something with a much shorter fuse. SAFETY-02's latch is the
# only thing standing between a controller that has been caught driving the wrong
# way and its own restart, and "restart the add-on" is the FIRST thing an operator
# looking at a FAILSAFE banner will try. Held only in memory it came back armed,
# into the fault, unattended - measured: after reconstructing Controller against
# the same store, `tripped` was False, the banner read NOT READY, and it resumed
# commanding -1000, -2000 W before re-tripping 180 s later. Under a supervisor
# that restarts a crashed add-on that is a loop, three minutes of wrong-direction
# power at a time, forever.
DEFAULTS = {
"phase": "idle",
"phase_started": None,
"last_completed": None,
"last_start_attempt": None,
"sign_tripped": False,
"sign_reason": "",
}
@@ -42,8 +61,10 @@ class Store:
loaded = json.load(fh)
if isinstance(loaded, dict):
self.data = {**DEFAULTS, **loaded}
_LOG.info("state restored: phase=%s last_completed=%s",
self.data.get("phase"), self.data.get("last_completed"))
_LOG.info("state restored: phase=%s last_completed=%s "
"sign_tripped=%s",
self.data.get("phase"), self.data.get("last_completed"),
self.data.get("sign_tripped"))
except FileNotFoundError:
_LOG.info("no saved state, starting fresh")
except (json.JSONDecodeError, OSError) as err:
+130
View File
@@ -74,6 +74,48 @@ NOT READY banner already names the problem, (b) does not apply.
was overruled. That firmware file's own comment says a lower bound on this
"belongs in CFG-01, not here" - so here it is.
5. `sign_disagree_w` at or above `max_w`. [(a)]
The arbiter's `supervised` limit bounds every command to +-max_w, and
SAFETY-02's predicate needs `abs(command) > sign_disagree_w`. Set the
threshold at or above the ceiling and the command side of the comparison
is unsatisfiable BY CONSTRUCTION: the watchdog can never fire, on any
input, ever. Same shape as 1 and 4 - the operator set a number, a layer
they cannot see makes it mean something else, and nothing says so. Worse
here, because what the status page reports in that state is "sign
watchdog: armed". A disarmed guard that advertises itself as armed is the
one thing this project has decided repeatedly it will not ship.
6. `maintenance_charge_w` above `max_w`. [(a)]
The same shape as 5, and found the same way - by checking a worked example
against what actually ships. maintenance's cheap-window shaping files
`Claim.limit(hi=-maintenance_charge_w)` at P_MAINT_SHAPE, against the
`supervised` SAFETY limit's `lo=-max_w`. Both bind (P_SAFETY and
P_MAINT_SHAPE both outrank the loop that wins the SET), so a charge rate
above the ceiling gives `lo > hi` - the arbiter's clause 3, which it
correctly resolves to 0 W and logs as a CONTRADICTION. Measured against
the arbiter, loop asking +500 W:
max_w=2000 hi=-2500 -> +0 W contradiction=True
max_w=2500 hi=-2500 -> -2500 W limited by supervised(max_w)
max_w=3000 hi=-2500 -> -2500 W limited by maintenance(...)
So MAINTENANCE SILENTLY NEVER CHARGES IN A CHEAP WINDOW until max_w is at
or above the charge rate, and the only evidence is a contradiction line
that reads like a bug in the arbiter rather than like the two numbers the
operator set. Equality is fine and deliberately not refused - at
max_w == charge_w the window is a single reachable point, -max_w.
⚠️ `maintenance_discharge_w` IS NOT THE SAME CASE, and this was checked
rather than assumed. THE SHAPING IS NOT SYMMETRIC: the drain phase does
not shape the loop, it OWNS the actuator - `Claim.set(P_MAINTENANCE,
+discharge_w)`. A SET is clamped by the limits above it, not contradicted
by them, so 2500 against a 2000 W ceiling emits +2000 W and the arbiter's
own explain() says `maintenance -> 2000 W, limited by supervised(max_w)`
on the status page every cycle. That is neither (a) - nothing is silent -
nor (b) - it moves energy and the cycle still completes, just slower
(13.8 kWh usable at 2000 W is ~6.9 h against the 720 min DRAIN_TIMEOUT).
It is warned about below, not refused.
WHAT IS DELIBERATELY *NOT* REFUSED, because it is untidy rather than unsafe:
- `integrator_max_w` above `max_w`. DOCS.md is right that the only thing the
@@ -91,6 +133,9 @@ WHAT IS DELIBERATELY *NOT* REFUSED, because it is untidy rather than unsafe:
NOT READY banner.
- `meter_poll_s` above half `meter_max_age_s`. One missed poll costs a stale
reading and a 0 W cycle. Warned about in p1.py, not refused.
- `maintenance_discharge_w` above `max_w`. Clamped, not contradicted, and the
clamp names itself in the arbiter's explain() - see rule 6. Warned about,
not refused.
"""
import logging
@@ -238,6 +283,72 @@ def validate_startup_config(opts: dict, dev_min=None, dev_max=None) -> list[str]
"or rebuild the firmware with a matching max_age_s and update "
"validate.FIRMWARE_INGEST_VETO_S.")
# --- 5. a sign watchdog that can actually fire ---------------------------
# ⚠️ Strictly `>=`, matching the predicate exactly: watchdog.py tests
# `abs(commanded_w) > min_w`, and the arbiter holds `abs(commanded_w)` at or
# below max_w, so threshold == max_w is already unsatisfiable. Anything
# BELOW max_w is reachable and therefore fine - this refuses the disarmed
# case only, not "close to the rail".
sign_w = _number(opts, "sign_disagree_w", 300.0)
if sign_w is None:
problems.append(f"sign_disagree_w {opts.get('sign_disagree_w')!r} is not a number")
elif max_w is not None and max_w > 0 and sign_w >= max_w:
problems.append(
f"sign_disagree_w is {sign_w:.0f} W but max_w is {max_w:.0f} W, so "
"the SAFETY-02 sign-disagreement watchdog can never fire: the "
"arbiter never lets a command exceed max_w, and the watchdog needs "
f"a command over {sign_w:.0f} W. It would sit there reporting "
"'armed' on the status page while being incapable of tripping - the "
f"one runaway failure mode this system has actually observed. Lower "
f"sign_disagree_w below {max_w:.0f} W (300 W is the measured "
"default), or raise max_w.")
# --- 6. a maintenance charge the ceiling can actually accommodate --------
# ⚠️ Strictly `>`, matching the arbiter exactly. The shaping is
# `Claim.limit(hi=-charge_w)` against `supervised`'s `lo=-max_w`, so
# charge_w == max_w leaves the single reachable point -max_w and works;
# only ABOVE is `lo > hi`, arbiter clause 3, 0 W and a contradiction.
#
# Not scoped to maintenance_enabled, deliberately. The pair is
# contradictory whether or not the schedule is on, and scoping it would
# leave the trap armed for the one operator action - ticking
# maintenance_enabled - that cannot possibly be expected to change what a
# charge rate means.
# ⚠️ THE FALLBACK IS config.yaml's SHIPPED VALUE, and it has to be kept
# equal to it - that is the whole point of AC3. It is reachable only when a
# caller hands over a dict with the key missing (sim/inject.py, a test); the
# Supervisor always supplies every key from config.yaml, so on a real
# install the argument is always present.
#
# ⚠️ KNOWN MISMATCH, LEFT DELIBERATELY AND FLAGGED RATHER THAN FIXED HERE:
# `main.py` still falls back to 2500 in two places
# (`opts.get("maintenance_charge_w", 2500)`), as does maintenance.py's
# MaintConfig dataclass. So a dict with the key ABSENT is accepted by this
# rule at 2000 and then built by main.py at 2500 - the contradiction this
# rule exists to catch, in the one case it cannot see. It is unreachable on
# a Supervisor install and those files are another ticket's right now.
# THE REAL FIX is one source of truth for option defaults; until then the
# divergence is PINNED BY A CHECK rather than by this comment - see
# test_validate.py, "the KNOWN cross-file default divergence". It asserts
# the mismatch as it stands and goes red if EITHER side moves, including
# when somebody closes it in main.py, at which point this block and that
# check both get deleted.
charge_w = _number(opts, "maintenance_charge_w", 2000.0)
if charge_w is None:
problems.append(
f"maintenance_charge_w {opts.get('maintenance_charge_w')!r} is not a number")
elif max_w is not None and max_w > 0 and charge_w > max_w:
problems.append(
f"maintenance_charge_w is {charge_w:.0f} W but max_w is "
f"{max_w:.0f} W, so a maintenance charge in a cheap window can "
"never happen: the shaping asks the arbiter for at most "
f"{-charge_w:.0f} W while the supervised limit allows no less than "
f"{-max_w:.0f} W, which is a contradiction it resolves to 0 W - "
"THE CEILING WINS, and what you would see is an ARBITER "
"CONTRADICTION line that reads like a bug rather than like these "
f"two numbers. Lower maintenance_charge_w to {max_w:.0f} W or less, "
f"or raise max_w to at least {charge_w:.0f} W.")
# --- warnings: real, but not worth refusing over -------------------------
integ = opts.get("integrator_max_w")
integ_w = None if not integ else _number(opts, "integrator_max_w", None)
@@ -249,6 +360,25 @@ def validate_startup_config(opts: dict, dev_min=None, dev_max=None) -> list[str]
"saturation. 0 (follow max_w) is the recommended value.",
integ_w, max_w)
# ⚠️ THE SIBLING OF RULE 6, AND NOT THE SAME CASE - see the docstring. The
# drain phase OWNS the actuator (`Claim.set` at P_MAINTENANCE), and a SET is
# clamped by the limits above it rather than contradicted by them. So this
# drains at max_w instead of the configured rate, the arbiter says so in
# words on the status page every cycle, and the cycle still completes -
# slower, and eating into maintenance.DRAIN_TIMEOUT_MIN. Loud and it moves
# energy, so it is neither (a) nor (b): warned, not refused.
disch_w = _number(opts, "maintenance_discharge_w", 2000.0)
if disch_w is not None and max_w is not None and max_w > 0 and disch_w > max_w:
_LOG.warning(
"maintenance_discharge_w %.0f W is above max_w %.0f W. Not refused - "
"the drain phase SETS the setpoint and the arbiter clamps it to "
"%.0f W and names the clamp ('limited by supervised(max_w)'), so "
"nothing is silent and the cycle still completes. But it drains at "
"%.0f W, not the %.0f W you configured, and the drain phase has a "
"720 min timeout. Lower it to %.0f W or raise max_w to remove the "
"surprise.",
disch_w, max_w, max_w, max_w, disch_w, max_w)
return problems
+354
View File
@@ -0,0 +1,354 @@
"""SAFETY-02 - the sign-disagreement watchdog.
WHAT IT CATCHES, in one sentence: a grid-following loop that is pushing power in
the wrong direction, which every other guard in this system is blind to.
The stale-input watchdog (SAFETY-01) asks "is the meter still talking?". The
saturation freeze asks "can the inverter follow?". The output clamp asks "is the
number too big?". A loop can pass all three and still be charging the battery
out of the grid while the house imports - the meter is fresh, the inverter is
tracking beautifully, and the magnitude is inside the rating. The only thing
that separates that from healthy operation is the SIGN.
⚠️ THIS IS NOT A HYPOTHETICAL. On 2026-08-20 the vendor controller (Illusmart)
did exactly this four times in one day (NOTES.md §4.7). Both captures of that
bus are committed - `sim/scenarios/bus-2026-08-20.log.gz` and the healthy
`bus-2026-08-23.log.gz` in the infra repo - and `sim/replay_sign.py` drives this
module over both of them. Every number in this file was measured on those two
days, not chosen:
day samples gated in sign agreement
2026-08-23 4 212 / 17 280 93.4 % (healthy)
2026-08-20 7 182 / 17 280 56.4 % (the whole fault day)
cluster A 15:26-16:43 0.3 %
cluster B 17:11-18:28 0.9 %
cluster C 20:01-20:31 10.5 %
cluster D 20:55-21:11 0.0 %
⚠️ THOSE AGREEMENT FIGURES ARE THE WEAKEST EVIDENCE ON THIS PAGE, and an earlier
version of this comment leaned on them as though they were the strongest. The
clusters are found by looking for sustained opposed samples, so measuring "how
opposed is this cluster" over them is close to a tautology - and "the healthy day
contains no such cluster" is the same sentence as "the healthy day produces no
trip", not independent corroboration of it.
What actually carries the argument is two things neither the detector nor the
clustering assumes:
1. The RUN-LENGTH DISTRIBUTION. The longest unbroken opposed run on the healthy
day is 30 s. On the fault day there are 18 runs over 180 s and the longest is
1750 s. Nothing about that gap was chosen - it is what the two days measure.
2. The clusters land where a human independently wrote down onsets. NOTES.md
§4.7 records 15:01, 16:56, 19:57 and 20:23 from watching it happen, before
any of this existed, and cluster D coincides with the separately documented
inverter-off runaway of §4.7 item 2.
Be precise about (2), because the convenient overstatement is available and wrong:
the clustering does NOT reproduce four episodes. It yields THREE clusters over the
four documented onsets - C spans both 19:57 and 20:23 - plus a fourth for §4.7
item 2, which is not one of the four. Every trip does fall inside a documented
window, which is the claim worth making.
------------------------------------------------------------------------------
WHY IT LIVES HERE AND NOT IN THE FIRMWARE OR THE WITNESS
Three vantage points see three different things. The firmware knows what it
WROTE, the witness Pi knows what was ON THE WIRE and whether the inverter ACKed
it, and the add-on knows what it INTENDED and holds the authoritative meter.
This check is not command-versus-wire; it is command-versus-physics. It needs
the emitted setpoint and the P1 net figure in the same place, and it needs the
authority to command zero when they fight:
- the witness is read-only by charter and cannot command anything, so it could
raise the alarm but never take the trip action the AC requires;
- the firmware has the wire but not the meter. §5.1 makes P1 the only
authoritative grid figure and the board only ever receives its AGE
(`sensor.goodwe_p1_sample_age`), never its value. Putting the check there
would mean shipping the meter to the ESP32 for no other reason, and it would
be C++ per install rather than one generic guard;
- the add-on has both halves already, plus an arbiter that can outrank every
strategy with a P_SAFETY claim.
The cost of standing here is real and worth stating: this watchdog compares what
we MEANT to put on the wire against the meter. If the firmware or the driver
inverts the sign on the way out, the add-on's own view still agrees with itself
and this check stays silent. That gap is a wire-versus-intent problem and it
belongs to the witness, not here.
------------------------------------------------------------------------------
WHAT "DISAGREEMENT" MEANS, PRECISELY
disagree = following
and commanded_w * grid_w < 0
and abs(commanded_w) > min_w
and abs(grid_w) > min_w
Four gates, and each one exists to exclude a specific thing that is NOT a fault:
1. `following`. ⚠️ THE GATE THAT IS EASIEST TO FORGET AND MOST EXPENSIVE TO
OMIT. Sign agreement is a property of a GRID-FOLLOWING loop. It is not
expected of a strategy that deliberately commands against the meter -
maintenance is exactly that. A maintenance grid charge of 1500 W into a house
drawing 500 W puts +2000 W on the meter against a -1500 W command: opposed,
both far over any sane threshold, and sustained for HOURS. Armed
unconditionally this watchdog would trip on every single maintenance cycle,
which is not a tuning problem, it is the wrong question being asked. So the
caller says whether the grid-following loop actually won the arbitration this
cycle, and the timer only accumulates when it did.
⚠️ 1500 W, NOT THE DEFAULT 2500 W, AND THE REASON IS NOT COSMETIC. At the
shipped defaults `maintenance_charge_w` is 2500 and `max_w` is 2000, so the
cheap-window claim is `hi = -2500` against a supervised `lo = -2000`: lo > hi,
which arbiter.py resolves as a CONTRADICTION and 0 W, not as a -2500 W charge.
The example above is written at 1500 W because that is a combination the
schema and the arbiter both actually produce. (The contradiction at the
defaults is a real pre-existing bug - a default maintenance charge silently
never happens in a cheap window - but it is maintenance's, not this module's,
and it is filed separately. Do not "fix" it here.)
2. The strict `< 0` product. "Opposed" and "neither side is zero" in one
expression.
3. The 300 W magnitude gate ON BOTH SIDES. Near zero the sign is noise, and near
zero is where a healthy loop LIVES - the deadband holds the command still
while the meter wanders either side of `target_grid_w`. On the healthy day
only 4 212 of 17 280 samples (24 %) are even gated in; the other 76 % are the
converged band where sign carries no information. This is also why the
deadband cannot produce a false trip: inside it the meter is by definition
under 300 W.
4. The 180 s continuity requirement. This is what excludes the legitimate
transient. A load step reverses the meter's sign instantly while the command
is still where the last cycle left it; slew limiting then needs a few cycles
to cross over. Measured on the healthy day, the LONGEST such crossing is 30 s
- and that is with a vendor controller far cruder than ours. 180 s is six
times that. It is not a fitted threshold: at 60 s the healthy day still
produces zero trips, and at 300 s the fault day still produces 13.
⚠️ A BATTERY THAT REFUSES A COMMAND CANNOT TRIP THIS, and that is structural
rather than lucky. At SoC 10 % in Standby, a 1350 W discharge command into a
1350 W import is DECLINED: the battery does nothing, so the meter stays at
+1350 W and the command stays at +1350 W. Same sign. Agreement. A refusal leaves
the meter where it was; only the inverter ACTING moves it, and acting in the
commanded direction moves it toward zero, not through it. The failure mode this
check keys on is the opposite of a refusal - it is the inverter obediently doing
a wrong thing.
⚠️ A FROZEN METER cannot trip it either, for the same reason read the other way:
a stuck reading of +800 W makes the loop command discharge and keep commanding
discharge. Command and meter converge on AGREEING. Freezes are SAFETY-12's and
SAFETY-01's problem, and this watchdog neither helps nor hinders there.
⚠️ ALL OF THAT DEPENDS ON ONE PROPERTY OF THE CONTROL LAW, so it is an invariant
and not a happy accident: the saturation freeze in control.py must keep letting
the command FALL, CROSS ZERO AND REVERSE - "may not wind further in the direction
it is already pushing", not "only corrections that shrink |i_w|". control.py
records the shrink-only encoding as tried and rejected because it pins the command
at its last value indefinitely; a pinned command against a meter that has moved on
is precisely a sustained sign disagreement, so under that encoding this watchdog
WOULD false-trip on a plain load step. If anyone ever revisits that freeze, this
module's no-false-positive argument is downstream of the answer.
------------------------------------------------------------------------------
WHAT IT DOES NOT CATCH
An inverter that ACTS opposite to what it was told. Tell it to discharge 2000 W
and have it charge instead, and the meter moves FURTHER the way it was already
going - so the command and the meter agree, all the way into the rail. This
watchdog compares the loop's output direction against the input state it was
regulating; it is blind to the plant lying about which way it went. That is the
saturation freeze's question (|command - the battery's actual power| > 500 W,
NOTES.md §4.7 row 4), and the two checks are complementary rather than
overlapping. Neither substitutes for the other.
Nor does it see a sign inversion introduced DOWNSTREAM of this add-on - in the
firmware, on the wire, or in a future driver. Our view stays self-consistent and
this stays silent. That is a wire-versus-intent problem and it belongs to the
witness, which is the only vantage point that sees both.
------------------------------------------------------------------------------
STATE, TRIP AND LATCH
One timestamp, not a sample counter. The cycle rate is not fixed - a meter
arrival, or `cycle_floor_s` elapsing, whichever comes first - so N samples is a
different wall-clock window on every path, and the AC is written in minutes.
The latch is not optional and not a nicety. Commanding 0 W removes the command
side of the comparison (|0| is not > 300), so the disagreement disappears the
instant the trip works. Un-latched, this watchdog would trip, clear itself, and
hand the actuator straight back to the loop that was driving the wrong way -
three minutes later, forever. `tripped` therefore stays true until an operator
`reset()`s it by starting control again, which is a deliberate human act after
looking at why.
⚠️ AND THE LATCH HAS TO OUTLIVE THE PROCESS. Held only in memory it is not a
latch at all - a container restart clears it and the add-on comes back armed,
into the same fault, unattended. Worse than merely losing state: "restart the
add-on" is the first thing an operator confronted with a FAILSAFE banner and a
0 W inverter will try, and under a supervisor that restarts a crashed add-on it
is a loop of three-minute wrong-direction bursts. `tripped` and `reason` are
therefore constructor arguments, restored from `Store` in `Controller.__init__`
and written back the moment the trip happens. See store.py.
"""
from dataclasses import dataclass
from datetime import datetime
@dataclass(frozen=True)
class SignWatchdogConfig:
"""Defaults, not constants - config.yaml carries both (SAFETY-02 AC 5)."""
min_w: float = 300.0
hold_s: float = 180.0
def disagrees(commanded_w, grid_w, following: bool,
cfg: SignWatchdogConfig) -> bool:
"""The predicate, and the ONLY place it is written down.
⚠️ Public because `sim/replay_sign.py` needs it too - to describe episodes, to
measure agreement, to sweep the clock-skew guard - and it had reimplemented it
inline. Two copies of a safety predicate is one copy plus a future
disagreement about what the check means, and the copy that drifts is always
the one in the harness that grades it.
"""
return (following
and commanded_w is not None and grid_w is not None
and commanded_w * grid_w < 0
and abs(commanded_w) > cfg.min_w
and abs(grid_w) > cfg.min_w)
def loop_is_following(resolution, loop_asked) -> bool:
"""Did the grid-following loop actually choose the number going on the wire?
Gate 1 of the predicate above, and it lives here rather than inline in
`cycle()` for the reason `disagrees()` does: the test suite has to be able to
assert on THE SHIPPED EXPRESSION, not on a transcription of it that drifts
apart from it in silence.
⚠️ IT WAS A TRANSCRIPTION, AND THAT WAS MEASURED, NOT FEARED. The P_PEAK case
in test_watchdog.py exists to make whoever lands peak shaving come here and
change an expectation. With the expression copied into the test it did not do
that: editing the shipped gate alone to `winner in ("loop", "peak")` left every
suite green while the watchdog went blind for the whole of every peak event.
One definition, two callers, and that test now bites.
`resolution` the arbiter's Resolution. Duck-typed on `.winner` and
`.target_w` so this module still imports nothing from arbiter.
`loop_asked` what the grid-following loop claimed this cycle, or None if it
did not claim at all.
"The loop won" is NOT the question, and the difference is a whole class of
false trip. A LIMIT claim can survive the loop winning and still replace its
number: maintenance's cheap-window shaping is `hi = -maintenance_charge_w`, so
at 1500 W the loop can ask for +500 W of discharge, win the arbitration, and
have the emitted value come out at -1500 W of grid charge. Winner says "loop";
the wire carries maintenance's intent.
So the test is on the NUMBER: the loop is following only while the emitted
value points the way the loop asked AND is no larger than it asked - limits
reduced it, they did not redirect it. The magnitude half is load-bearing on
its own, because a `lo` limit can raise magnitude without flipping sign, and
that is not the loop following either. Zero on either side falls out as
not-following, which costs nothing: a 0 W command cannot be a
wrong-direction command.
⚠️ FORWARD HAZARD, and the claim block in cycle() already anticipates it.
`winner == "loop"` disarms the watchdog under ANY higher-priority SET claim.
Today only maintenance and the safety stops emit one and neither follows the
meter, so the gate costs nothing. A peak-shaving strategy claiming SET at
P_PEAK **is** grid-following in sign, and the moment it lands this goes blind
for the duration of every peak event. Widening it means asking "does the
winning strategy follow the meter", not adding another name to a comparison.
"""
return (resolution.winner == "loop" and loop_asked is not None
and resolution.target_w * loop_asked > 0
and abs(resolution.target_w) <= abs(loop_asked))
class SignWatchdog:
"""One instance per actuator, evaluated once per control cycle.
⚠️ NOT SHARED ACROSS CADENCES. `disagree_since` is a wall-clock start time,
so driving one instance from two loops at two rates would have each one
resetting the other's window. If a fast on-device path ever wants this
check, it gets its own instance - the same exactly-one-writer discipline the
arbiter applies to the actuator, applied to the detector's own state.
"""
def __init__(self, cfg: SignWatchdogConfig | None = None, *,
tripped: bool = False, reason: str = ""):
self.cfg = cfg or SignWatchdogConfig()
self.disagree_since: datetime | None = None
# ⚠️ RESTORED FROM THE STORE, NOT DEFAULTED. A latch that lives only in
# memory is not a latch: a container restart clears it, and the add-on
# comes back armed, into the same fault, with no operator involved. See
# store.py's note on `sign_tripped` for the measured behaviour.
self.tripped = bool(tripped)
self.reason = reason if self.tripped else ""
# -- the whole check, and it is a pure function of its arguments ----------
def evaluate(self, commanded_w, grid_w, following: bool,
now: datetime) -> bool:
"""Advance the detector one cycle. Returns True while tripped.
`commanded_w` the EMITTED setpoint - what is standing on the wire now,
after deadband, slew, quantisation and the arbiter. Not
the optimizer's raw wish: the wire carries the emitted
value, so anything else can disagree with the meter for a
reason that is not a fault.
`grid_w` net P1, + = importing. None when the meter is stale or
missing, which CLEARS the timer rather than accumulating
- a frozen figure held against a moving command would
otherwise manufacture a disagreement in about three
minutes, on top of SAFETY-01 having already commanded
zero for the same cause.
`following` did the grid-following loop actually win this cycle? See
gate 1 in the module docstring.
`now` the caller's clock, so this stays testable and replayable.
"""
if self.tripped:
# Latched. Do not re-evaluate: the trip removed the evidence.
return True
cfg = self.cfg
if not disagrees(commanded_w, grid_w, following, cfg):
self.disagree_since = None
return False
if self.disagree_since is None:
self.disagree_since = now
return False
held = (now - self.disagree_since).total_seconds()
if held < cfg.hold_s:
return False
self.tripped = True
self.reason = (
f"SIGN_DISAGREEMENT: commanded {commanded_w:+.0f} W against a meter "
f"reading {grid_w:+.0f} W for {held:.0f} s continuously "
f"(limits: both over {cfg.min_w:.0f} W, held over {cfg.hold_s:.0f} s)"
)
return True
def held_s(self, now: datetime) -> float:
"""How long the current disagreement has been standing, for the UI.
⚠️ Takes the clock rather than reading one. It was a property calling
`datetime.now()`, which quietly broke the property this module's header
advertises - every other observable here is a pure function of what it
was handed - and an unclocked function is one nothing can test, which is
exactly what happened to it.
"""
if self.disagree_since is None:
return 0.0
return (now - self.disagree_since).total_seconds()
def reset(self) -> None:
self.disagree_since = None
self.tripped = False
self.reason = ""
+75 -3
View File
@@ -1,5 +1,5 @@
name: GoodWe RS485 Controller
version: "0.4.0"
version: "0.5.2"
slug: goodwe_controller
description: >-
Drives a GoodWe ES/BP battery inverter over RS485 by emulating its smart
@@ -90,14 +90,71 @@ options:
integrator_max_w: 0
heartbeat_s: 10
stale_input_s: 15
cycle_floor_s: 15
auto_start: false
# --- SAFETY-02 sign-disagreement watchdog ----------------------------------
# Trips when the emitted setpoint and the P1 net figure point OPPOSITE ways,
# both over sign_disagree_w, continuously for sign_disagree_s. Action: 0 W and
# a failsafe that latches until an operator starts control again.
#
# Both defaults are measured, not chosen. On the captured healthy day
# (2026-08-23) the longest legitimate opposed run - a load step the loop has
# not caught up with - is 30 s, and sign agreement over the gated samples is
# 93.4 %. On the captured fault day (2026-08-20) the clusters run 0.0-10.5 %
# agreement with opposed runs up to 1752 s. 300 W / 180 s sits in the middle of
# a very wide gap: at 60 s the healthy day still trips nothing, at 300 s the
# fault day still trips 13 times. There is no reason to move either number
# without a capture that says so. They are options because the AC says
# thresholds are defaults rather than constants - not because they are knobs
# anyone is expected to turn.
#
# ⚠️ sign_disagree_w must stay BELOW max_w or the watchdog can never fire (the
# arbiter bounds the command to +-max_w). validate.py refuses that config.
sign_disagree_w: 300
sign_disagree_s: 180
# --- SAFETY-18 frozen-but-fresh meter detector -----------------------------
# Seconds the meter may keep answering with an UNCHANGED measurement while the
# command stands at the rail before the add-on raises an alarm. Both halves
# are required: neither one alone is safe, and app/freeze.py carries why.
#
# ALARM ONLY. It does not command 0 W and does not latch. A false trip parks
# the battery; a false notification costs an alert.
#
# ⚠️ Only `meter_source: homewizard_local` can see this at all - it is the
# only transport that reads the meter's own document. On every other source
# the status page says NOT COVERED rather than "armed".
#
# 300 s is measured, not chosen. Over 2399 s of the live house at the
# converged operating point (sim/scenarios/p1-age-2026-08-26.jsonl.gz, mean
# grid -14.2 W, 1194 polls, 735 of them repeating the previous reading) the
# LONGEST run without a distinct reading is 15.4 s (the largest unchanged_s any
# sample observes is 8.0 s). 300 s is 19.5x the pessimistic figure, and
# still inside the 900 s quarter that bills the capacity tariff.
meter_freeze_s: 300
# --- maintenance -----------------------------------------------------------
maintenance_enabled: false
maintenance_interval_days: 28
maintenance_start_hour: 10
maintenance_discharge_w: 2500
maintenance_charge_w: 2500
# ⚠️ BOTH MUST STAY AT OR BELOW max_w, and that is why they are 2000 rather
# than the 2500 they shipped as. The cheap-window charge shaping asks the
# arbiter for `hi = -maintenance_charge_w` against the supervised `lo =
# -max_w`: above the ceiling that is `lo > hi`, a contradiction resolved to
# 0 W, so maintenance would SILENTLY NEVER CHARGE in a cheap window on a
# default install. validate.py refuses that config (rule 6), so these
# defaults had to become self-consistent with the max_w above rather than
# ship a configuration the validator rejects.
#
# The discharge side is clamped rather than contradicted - it drains at max_w
# and the arbiter says so - so it is only warned about; it is 2000 here for
# symmetry and to keep the shipped config warning-free.
#
# Raising max_w (the live reference install runs 5000) is what unlocks a
# faster maintenance cycle. Raise max_w FIRST, then these.
maintenance_discharge_w: 2000
maintenance_charge_w: 2000
maintenance_soc_floor: 11
maintenance_soc_target: 99
maintenance_hold_min: 120
@@ -151,8 +208,23 @@ schema:
integrator_max_w: int(0,15000)
heartbeat_s: int(2,25)
stale_input_s: int(5,120)
cycle_floor_s: int(5,60)
auto_start: bool
# No "off" value, deliberately: this is a P0 safety check and the schema is
# not the place to make disarming it a supported configuration.
sign_disagree_w: int(50,5000)
sign_disagree_s: int(30,1800)
# No "off" value, for SAFETY-02's reason: disarming a safety alarm is not a
# supported configuration. ⚠️ THE FLOOR OF 60 IS NOT A RECOMMENDATION. At 300 s
# the slowest a HEALTHY meter's 1 Wh register can tick and still look unchanged
# is 12 W, which is inside the 15 W deadband, so the two terms of the
# conjunction are mutually exclusive rather than merely far apart. At 60 s that
# figure is 60 W and the argument is gone - only the margin is left. See
# app/freeze.py.
meter_freeze_s: int(60,3600)
maintenance_enabled: bool
maintenance_interval_days: int(1,90)
maintenance_start_hour: int(0,23)
+550
View File
@@ -0,0 +1,550 @@
"""Runnable check for SAFETY-03, the output clamp and its winding-loop alarm.
python3 test_clamp.py
No framework, asserts only, same house style as test_freeze.py.
Five parts, and parts 3 and 4 are the ones that carry the acceptance criteria:
1. THE CLAMP ARITHMETIC, on the shipped `clamp_to_rating()` - boundaries,
symmetry, the device second stage, and the non-finite cases. Table driven.
2. THE ALARM'S CADENCE - rising edge, re-assert, hysteresis - on the shipped
`ClampWatchdog`. Every engagement counted, notifications not flooded.
3. AC 1 + AC 3, WIRED. A computed setpoint that exceeds the rating, driven
through the REAL `Controller.cycle()`: nothing over the rating reaches the
device, and the alarm fires. Includes the historical runaway from
specs.md 10.3 replayed against an unclamped transcription, so the check
says what the clamp is actually worth in watts rather than asserting a
constant.
4. AC 2, AND IT IS THE LOAD-BEARING HALF. A loop that SATURATES WITHOUT
CLAMPING - the inverter refusing to follow a perfectly legal command - must
raise NO clamp alarm. If it does, the bug signal is inside the noise again
and the whole ticket has bought nothing.
5. NO PATH BYPASSES IT. write_setpoint() is the only route to the wire, so it
is called directly with an out-of-range value the way the heartbeat and the
operator's buttons call it.
------------------------------------------------------------------------------
WHY PART 4 ALSO ASSERTS ON `Decision`'s BOOLEANS AND NOT ON `reason`
`Decision.reason` is one string and the stages after the clamp overwrite it. The
check below drives a cycle that clamps the integrator AND is then slew limited,
and asserts that `reason` does NOT say so while `i_clamped` does. That is not a
style preference - an alarm keyed on the string is silent on exactly the cycles a
winding loop produces, and this is the check that would go red if someone
"simplified" main.py back onto it.
------------------------------------------------------------------------------
WHAT THIS CHECK DOES NOT COVER
- The firmware-side magnitude clamp (the card's "two clamps, not one"). This
add-on's `write_setpoint()` is the add-on half; the ESP32 half is DRIVER-07's
and is not in this repo's Python.
- A rig run. Nothing here needs one: the clamp is arithmetic on the write path
and part 3 drives the shipped `Controller.cycle()` against a stub Home
Assistant, which is the same surface test_freeze.py part 1-2 use. The wire-
level invariant the card asks for (`|emitted| <= rating` on every frame) is
asserted here over every value that reached `set_number()`, which is the last
thing this process controls; proving it at the inverter needs the witness log
and belongs with DRIVER-07.
"""
import asyncio
import math
import sys
from datetime import datetime, timedelta, timezone
from app.clamp import (
CLEAR_CYCLES, REASON_CODE, REASSERT_S, ClampWatchdog, clamp_to_rating,
is_commandable,
)
from app.control import Tuning, compute
from app.main import Controller
fails = []
T0 = datetime(2026, 8, 30, 9, 0, 0, tzinfo=timezone.utc)
# The live install's numbers. NOTES.md and freeze.py both say max_w: 5000 there,
# which is also the device rating, so this is the bound the alarm is written at.
LIVE_MAX_W = 5000.0
CLAMP_ID = "goodwe_output_clamp"
def check(name, cond):
print(f" {'ok ' if cond else 'FAIL'} {name}")
if not cond:
fails.append(name)
# ==============================================================================
print("1. the clamp arithmetic, on the shipped expression")
# (requested, limit, expected applied, expected engaged, expected bound)
TABLE = [
(5001.0, 5000.0, 5000.0, True, "max_w"), # +limit+1
(-5001.0, 5000.0, -5000.0, True, "max_w"), # -limit-1, the export side
(5000.0, 5000.0, 5000.0, False, ""), # exactly +limit: NOT engaged
(-5000.0, 5000.0, -5000.0, False, ""), # exactly -limit: NOT engaged
(0.0, 5000.0, 0.0, False, ""), # the failsafe value
(1234.0, 5000.0, 1234.0, False, ""),
(14768.0, 5000.0, 5000.0, True, "max_w"), # 10.3's reported runaway
]
for requested, limit, want_applied, want_engaged, want_bound in TABLE:
r = clamp_to_rating(requested, limit)
check(f"{requested:+.0f} W against a {limit:.0f} W rating -> "
f"{r.applied_w:+.0f} W, engaged={r.engaged}, bound={r.bound!r}",
r.applied_w == want_applied and r.engaged is want_engaged
and r.bound == want_bound)
# It is a MAGNITUDE clamp (3.2), so import and export must be bound identically.
# Derived over a sweep rather than asserted on two hand-picked values.
asym = [m for m in range(0, 12001, 37)
if clamp_to_rating(m, LIVE_MAX_W).applied_w
!= -clamp_to_rating(-m, LIVE_MAX_W).applied_w]
check(f"symmetric over {len(range(0, 12001, 37))} magnitudes "
f"({len(asym)} asymmetric)", not asym)
# The engagement flag has to mean "the RAW value differed", which is the words
# AC 2 uses. Re-derived from the result's own fields, not from the table.
mismatch = [m for m in range(0, 12001, 37)
for r in [clamp_to_rating(m, LIVE_MAX_W)]
if r.engaged != (r.requested_w != r.applied_w)]
check("engaged is exactly 'raw differs from applied', over the same sweep",
not mismatch)
# The device second stage. Only reachable when max_w is ABOVE the device range,
# i.e. when validate_startup_config() did not run or no longer holds.
r = clamp_to_rating(3000.0, 5000.0, dev_min=-2000.0, dev_max=2000.0)
check(f"a device narrower than max_w still bites, and says which bound "
f"({r.applied_w:+.0f} W, {r.bound!r})",
r.applied_w == 2000.0 and r.engaged and r.bound == "device")
r = clamp_to_rating(3000.0, 5000.0, dev_min=-5000.0, dev_max=5000.0)
check("...and is a no-op once the startup validation holds",
r.applied_w == 3000.0 and not r.engaged)
# ⚠️ REGRESSION, REVIEW-FOUND. THE FIRST STAGE USED TO `return` AND THE DEVICE
# STAGE WAS THEN NEVER REACHED, so the harder the overshoot the less the guard
# did: at max_w 5000 against a +-4000 device, +9999 emitted +5000 - OUTSIDE THE
# DEVICE RANGE - while a mere +4500 was correctly held to +4000. That inverts
# the guard and it was a regression against the pre-PR `write_setpoint()`, whose
# comment read "never send a value the device will reject outright".
#
# The property is asserted over a sweep, not on the two witness values, because
# the defect was invisible to every single-value check that happened to sit
# inside one stage.
DEV_LO, DEV_HI = -4000.0, 4000.0
escapes = [(m, clamp_to_rating(m, 5000.0, DEV_LO, DEV_HI).applied_w)
for m in range(-15000, 15001, 61)
for a in [clamp_to_rating(m, 5000.0, DEV_LO, DEV_HI).applied_w]
if not (DEV_LO <= a <= DEV_HI)]
big = clamp_to_rating(9999.0, 5000.0, DEV_LO, DEV_HI)
small = clamp_to_rating(4500.0, 5000.0, DEV_LO, DEV_HI)
print(f" max_w 5000 vs a device of {DEV_LO:+.0f}..{DEV_HI:+.0f}: "
f"+9999 -> {big.applied_w:+.0f} W ({big.bound}), "
f"+4500 -> {small.applied_w:+.0f} W ({small.bound})")
check(f"BOTH stages always run - {big.requested_w:+.0f} W emits "
f"{big.applied_w:+.0f} W, inside the device's {DEV_HI:+.0f} W, and names "
f"both bounds ({big.bound!r})",
big.applied_w == DEV_HI and big.bound == "max_w+device")
check(f"...and it holds over the whole range, so no overshoot escapes the "
f"device ({len(escapes)} escapes in {len(range(-15000, 15001, 61))})",
not escapes)
check("...a big overshoot is not clamped LESS than a small one",
abs(big.applied_w) <= abs(small.applied_w) + 0.0)
print(" non-finite input is loss of input, not an out-of-range value")
for bad in (None, float("nan"), float("inf"), float("-inf"), "abc", object()):
check(f"is_commandable({bad!r:.30}) is False", is_commandable(bad) is False)
for good in (0, -5000.0, 5000, 12.5):
check(f"is_commandable({good!r}) is True", is_commandable(good) is True)
# ⚠️ WHY THAT GUARD EXISTS, AND THE ANSWER IS WORSE THAN "it passes a nan
# through". MEASURED HERE, not recalled: every comparison against a nan is
# False, so `min(limit, nan)` keeps its FIRST argument and a naive clamp turns a
# nan into a command at the FULL RATING - and then reports engaged=True, because
# limit != nan, so the winding-loop counter gets a transport bug filed in it.
# The number below is computed by the same expression the guard replaces.
nan = float("nan")
naive = max(-LIVE_MAX_W, min(LIVE_MAX_W, nan))
check(f"a naive clamp turns a nan into {naive:+,.0f} W and calls it "
f"engaged={naive != nan} - hence the guard",
naive == LIVE_MAX_W and (naive != nan) is True)
# ==============================================================================
print()
print("2. the alarm's cadence: every engagement counted, no flood")
wd = ClampWatchdog()
t = T0
fired = [wd.note(True, "winding", t + timedelta(seconds=i)) for i in range(10)]
check(f"the rising edge fires ({fired[0]})", fired[0] is True)
check(f"the next 9 one-second cycles fire {sum(fired[1:])} more times "
f"(re-assert is {REASSERT_S:.0f} s)", sum(fired[1:]) == 0)
check(f"but every one of them is counted: engagements={wd.engagements}",
wd.engagements == 10)
check(f"and it is one episode, not ten: episodes={wd.episodes}",
wd.episodes == 1)
# --- THE RE-ASSERT CADENCE, PINNED IN BOTH DIRECTIONS ------------------------
# ⚠️ REVIEW-FOUND. This used to read `wd.note(True, ..., T0 + REASSERT_S + 9)`,
# which is green for ANY value of the constant - REASSERT_S = 3600 passed, and an
# hour of silence is precisely what the module promises not to do. A check whose
# stimulus is written in terms of the constant it is grading cannot grade it.
#
# The requirements are stated in wall-clock seconds and derived from what the
# alarm is FOR, so they do not move when the constant does:
#
# floor - a loop pinned on the rail must speak at least MIN_PER_QUARTER times
# inside the 900 s quarter that bills the capacity tariff. That window
# is the same one freeze.py and DOCS.md reason about.
# ceiling - it must not speak on more than MAX_FRACTION of cycles, or it is
# the per-cycle flood the split exists to remove.
QUARTER_S = 900.0
MIN_PER_QUARTER = 5
MAX_FRACTION = 0.20
pinned = ClampWatchdog()
alarms = sum(1 for i in range(int(QUARTER_S))
if pinned.note(True, "pinned", T0 + timedelta(seconds=i)))
print(f" pinned on the rail for {QUARTER_S:.0f} s at 1 s cycles: "
f"{pinned.engagements} engagements, {alarms} alarm(s) "
f"({alarms / QUARTER_S:.1%} of cycles)")
check(f"a loop pinned for a whole billing quarter speaks at least "
f"{MIN_PER_QUARTER} times (it spoke {alarms})",
alarms >= MIN_PER_QUARTER)
check(f"...and on no more than {MAX_FRACTION:.0%} of cycles (it was "
f"{alarms / QUARTER_S:.1%})", alarms <= MAX_FRACTION * QUARTER_S)
check("...while every single cycle is still counted",
pinned.engagements == int(QUARTER_S))
# --- THE HYSTERESIS, PINNED AGAINST THE CHATTER DIRECTION --------------------
# ⚠️ REVIEW-FOUND. This used to issue `range(CLEAR_CYCLES - 1)` clean cycles,
# which at CLEAR_CYCLES = 1 issues ZERO of them and passes while printing
# "0 clean cycles do NOT clear it". Green for every value >= 1 - including 1,
# which IS the chatter the constant exists to prevent. The stimulus is a literal
# now: ONE clean cycle must not clear the alarm, whatever the constant says.
wd2 = ClampWatchdog()
wd2.note(True, "winding", T0)
wd2.note(False, "", T0 + timedelta(seconds=1))
check(f"ONE clean cycle does NOT clear the alarm (active={wd2.active}) - "
f"a command dithering across the rail must not chatter",
wd2.active is True)
survived = 1
while wd2.active and survived < 50:
wd2.note(False, "", T0 + timedelta(seconds=1 + survived))
survived += 1
print(f" it took {survived} consecutive clean cycles to de-assert "
f"(CLEAR_CYCLES = {CLEAR_CYCLES})")
check(f"...and it does de-assert, after {survived} of them, rather than "
f"latching", wd2.active is False and survived == CLEAR_CYCLES)
check("...and the engagement count survives the clear, so the evidence does",
wd2.engagements == 1)
check("a clean cycle on an idle watchdog says nothing",
wd2.note(False, "", T0 + timedelta(seconds=300)) is False)
# ==============================================================================
print()
print("3. AC 1 + AC 3, wired: over the rating both clamps AND alarms")
class _Store:
data = {}
def set(self, *a):
pass
def get_time(self, *a):
return None
class _Pub:
enabled = False
def publish(self, *a, **k):
pass
def close(self):
pass
class _Hass:
"""Stub HA. `track` False models an inverter that is switched off entirely,
which is the 10.3 runaway's own setup."""
def __init__(self, grid, batt=0.0, track=True):
self.grid = grid
self.batt = batt
self.track = track
self.calls = []
self.writes = []
async def limits(self, entity):
return (-LIVE_MAX_W, LIVE_MAX_W, 1.0)
async def number(self, entity, invert=False):
return {"sensor.grid": self.grid, "sensor.soc": 50.0,
"sensor.batt": self.batt}.get(entity)
async def set_number(self, entity, value):
self.writes.append(float(value))
if self.track:
self.batt = float(value)
return True
async def call(self, domain, service, data):
self.calls.append((domain, service, data))
return True
def wired(grid, cycles=40, batt=0.0, track=True, **over):
async def go():
hass = _Hass(grid, batt=batt, track=track)
c = Controller(dict(
{"setpoint_entity": "number.sp", "meter_entity": "sensor.grid",
"soc_entity": "sensor.soc", "batt_entity": "sensor.batt",
"meter_source": "off", "max_w": LIVE_MAX_W, "gain": 0.6,
"slew_w": 1000, "deadband_w": 15, "target_grid_w": 0,
"step_w": 10, "heartbeat_s": 10, "stale_input_s": 15},
**over), hass, _Store(), _Pub(), stop=asyncio.Event())
c.auto = True
c.dev_min, c.dev_max = -LIVE_MAX_W, LIVE_MAX_W
# The red status row exists only WHILE the clamp is engaged, and the
# hysteresis clears it a few cycles later - so it is captured live
# rather than looked for after the run, when it is correctly gone.
c.red_snapshot = []
for _ in range(cycles):
await c.read_inputs()
await c.cycle()
if c.clamp_wd.active and not c.red_snapshot:
c.red_snapshot = c.checks()
return hass, c
return asyncio.run(go())
def clamp_notes(hass):
return [d for dom, svc, d in hass.calls
if (dom, svc) == ("persistent_notification", "create")
and d.get("notification_id") == CLAMP_ID]
# THE 10.3 RUNAWAY, replayed. "With the inverter switched off entirely, an
# unclamped integrator climbed ~130 W per 4 s to over 10 kW and, pinned at the
# 5 kW rail, reported 14 768 W." `track=False` is the inverter switched off: it
# reports 0 W however hard it is commanded, so the error never closes.
hass_r, ctl_r = wired(6000.0, cycles=60, track=False)
emitted = [abs(w) for w in hass_r.writes]
# What the SAME error would have done to an accumulator with no bound - the
# transcription is deliberately the unclamped velocity form 10.3 describes, so
# the figure below is computed here and not quoted from the spec.
runaway, err = 0.0, 6000.0
for _ in range(60):
runaway += 0.6 * err
print(f" unclamped, the same 60 cycles reach {runaway:,.0f} W; "
f"the largest value that reached the device is {max(emitted):,.0f} W")
check(f"an unbounded accumulator really does run away here "
f"({runaway:,.0f} W >> {LIVE_MAX_W:.0f} W)", runaway > 3 * LIVE_MAX_W)
check(f"AC 1: NOT ONE of the {len(emitted)} values that reached the device "
f"exceeds the {LIVE_MAX_W:.0f} W rating",
emitted and max(emitted) <= LIVE_MAX_W)
# ⚠️ AND THE CLAMP IS NOT THE ONLY THING HOLDING THAT LINE HERE, which is worth
# saying out loud rather than letting the number above imply otherwise. Against
# an inverter reporting 0 W the saturation freeze engages after
# `saturation_cycles` and pins the command well BELOW the rail - that is why the
# largest emitted value is what it is and not 5000 W. The clamp is what makes it
# bounded; the freeze is what makes it small. A check that read
# `max(emitted) == max_w` would have been asserting the freeze, not the clamp.
check(f"...and the saturation freeze is what pins it at {max(emitted):,.0f} W "
f"rather than the rail", ctl_r.sat_count >= ctl_r.tuning.saturation_cycles)
notes_r = clamp_notes(hass_r)
print(f" the loop clamped on {ctl_r.clamp_wd.engagements} of {60} cycles, "
f"in {ctl_r.clamp_wd.episodes} episode(s), raising {len(notes_r)} "
f"notification(s)")
check("AC 3: and it ALARMED - silently clipping this is the defect",
ctl_r.clamp_wd.engagements > 0 and len(notes_r) >= 1)
check("...with its own notification_id, not shared with any other guard",
all(d["notification_id"] == CLAMP_ID for d in notes_r))
check("...one episode, re-asserted rather than one per cycle",
ctl_r.clamp_wd.episodes == 1 and len(notes_r) < ctl_r.clamp_wd.engagements)
check("...and it names itself a bug signal, not a limit in use",
bool(notes_r) and "out of range" in notes_r[0]["message"])
rows_r = [e for e in ctl_r.events if REASON_CODE in e]
check(f"the action log records every engagement, coalesced into one row per "
f"episode ({len(rows_r)} row(s), {ctl_r.clamp_wd.engagements} "
f"engagement(s))",
len(rows_r) == ctl_r.clamp_wd.episodes
and f"{ctl_r.clamp_wd.engagements} engagement(s)" in rows_r[0])
# ⚠️ REGRESSION, AND IT WAS A REAL BUG BEFORE IT WAS A CHECK. `log_event()` keeps
# 40 rows. A row per engagement flushes that ring inside 40 cycles of a winding
# loop, and the first thing it evicted was SAFETY-18's own "METER FROZEN?" row -
# i.e. this guard silencing another guard's log. Driven here with a plant that
# TRACKS the command, so the saturation freeze never engages and the integrator
# clamps on essentially every cycle.
hass_l, ctl_l = wired(9000.0, cycles=80, track=True)
rows_l = [e for e in ctl_l.events if REASON_CODE in e]
other_l = [e for e in ctl_l.events if REASON_CODE not in e]
print(f" 80 winding cycles: {ctl_l.clamp_wd.engagements} engagement(s) -> "
f"{len(rows_l)} log row(s), leaving room for {len(other_l)} other event(s)")
check(f"a long winding episode does NOT flood the 40-row event log "
f"({len(rows_l)} rows for {ctl_l.clamp_wd.engagements} engagements)",
ctl_l.clamp_wd.engagements > 40 and len(rows_l) <= ctl_l.clamp_wd.episodes)
check("...and the count in the row is the counter's, so nothing is lost",
f"{ctl_l.clamp_wd.engagements} engagement(s)" in rows_l[-1])
# The RED row exists only while engaged, and the episode above is short, so it
# is captured mid-run rather than asserted after the hysteresis has cleared it.
check("the status page goes red WHILE it is engaged",
any(not c["ok"] and not c["warn"] and "OUTPUT CLAMP ENGAGED" in c["text"]
for c in ctl_r.red_snapshot))
check("...and once clear, a warn row still says how many times it engaged",
any(c["warn"] and f"{ctl_r.clamp_wd.engagements} time(s)" in c["text"]
for c in ctl_r.checks()))
# ⚠️ NOT A TRIP. 11.2's action is "clamp and alarm"; the loop keeps running and
# keeps writing the clamped value. If a later cut changes that it must come here.
check(f"it is NOT a trip - the command stands at {ctl_r.target:+.0f} W, not 0 W",
ctl_r.target != 0.0 and ctl_r.target == hass_r.writes[-1])
check("...and the controller still reports itself running, not failsafe",
ctl_r.auto and not ctl_r.sign_wd.tripped)
# The export side, because the clamp is a magnitude clamp and a sign bug in it
# would be invisible on the import case alone.
hass_x, ctl_x = wired(-6000.0, cycles=60, track=False)
check(f"the export side clamps and alarms too "
f"({min(hass_x.writes):+,.0f} W, {len(clamp_notes(hass_x))} note(s), "
f"{ctl_x.clamp_wd.engagements} engagement(s))",
min(hass_x.writes) < 0 and abs(min(hass_x.writes)) <= LIVE_MAX_W
and clamp_notes(hass_x) and ctl_x.clamp_wd.engagements > 0)
# ==============================================================================
print()
print("4. AC 2: saturation WITHOUT clamping must NOT raise a clamp alarm")
# A legal command the inverter will not follow. The meter sits just outside the
# deadband so the loop moves a little and then rests; `batt` is pinned far from
# whatever is commanded, which is the saturation condition (|cmd - actual| >
# saturation_w) and nothing else. The integrator never approaches the rating.
hass_s, ctl_s = wired(60.0, cycles=40, batt=2500.0, track=False)
peak = max(abs(w) for w in hass_s.writes)
print(f" saturated for {ctl_s.sat_count} consecutive cycles; the command "
f"peaked at {peak:.0f} W, {peak / LIVE_MAX_W:.1%} of the rating")
# ⚠️ THE POSITIVE HALF FIRST. A negative check that saw nothing happen is not an
# observation - if the saturation freeze never engaged, "no clamp alarm" is
# vacuous and this check is worthless.
check(f"the saturation freeze REALLY DID engage (sat_count="
f"{ctl_s.sat_count} >= {ctl_s.tuning.saturation_cycles})",
ctl_s.sat_count >= ctl_s.tuning.saturation_cycles)
check("...and the event log says so, in the saturation's own words",
any("saturation freeze" in e for e in ctl_s.events))
check(f"...while the command stayed well inside the rating "
f"({peak:.0f} W < {LIVE_MAX_W:.0f} W)", peak < LIVE_MAX_W)
# Now the negative, which is only worth anything because of the three above.
check("AC 2: NO clamp notification was raised", not clamp_notes(hass_s))
check("AC 2: the clamp counted ZERO engagements", ctl_s.clamp_wd.engagements == 0)
check("AC 2: no OUTPUT_CLAMP row in the action log",
not [e for e in ctl_s.events if REASON_CODE in e])
check("AC 2: and no clamp row on the status page",
not [c for c in ctl_s.checks() if "output clamp" in c["text"].lower()])
print(" ...and the two alarms do not share a notification_id")
ids_clamp = {d["notification_id"] for d in clamp_notes(hass_r)}
ids_other = {d.get("notification_id") for dom, svc, d in hass_r.calls
if svc == "create"} - ids_clamp
check(f"clamp ids {sorted(ids_clamp)} are disjoint from every other alarm's "
f"{sorted(x for x in ids_other if x)}", not (ids_clamp & ids_other))
print(" ...and the alarm cannot be keyed on Decision.reason")
# A cycle that clamps the integrator AND is then slew limited. `reason` reports
# the LAST stage, so a string-keyed alarm is silent on it; the boolean is not.
t = Tuning(max_w=2000.0, slew_w=100.0, gain=0.6)
d = compute(prev_w=1000.0, grid_w=6000.0, actual_w=1000.0, tuning=t, i_w=1900.0)
print(f" reason={d.reason!r} i_clamped={d.i_clamped} clamped={d.clamped}")
check("the integrator DID clamp on that cycle", d.i_clamped is True)
check("...but `reason` says 'slew-limited' - the clamp was overwritten "
"downstream, so a string-keyed alarm is SILENT here",
d.reason == "slew-limited" and "clamp" not in d.reason)
# And the same for the OUTPUT clamp, on a config that lets the integrator run
# above the rail - then both flags fire and `reason` still reports neither.
t2 = Tuning(max_w=2000.0, integrator_max_w=3000.0, slew_w=100.0, gain=0.6)
d2 = compute(prev_w=1000.0, grid_w=6000.0, actual_w=1000.0, tuning=t2, i_w=2900.0)
print(f" reason={d2.reason!r} i_clamped={d2.i_clamped} clamped={d2.clamped}")
check("both clamps engaged, and `reason` reports neither",
d2.i_clamped and d2.clamped and "clamp" not in d2.reason)
# ⚠️ HOW THE FREEZE AND THE CLAMP INTERACT, MEASURED RATHER THAN ASSUMED, AND
# THE MEASUREMENT REFUSED TWO GUESSES BEFORE THIS ONE. Written first as "a
# frozen cycle never clamps" (false), then as "only the i_w == 0 reseed
# exception clamps" (also false - at i_w == 0 with a matching actual the cycle
# is not saturated at all, so it is never frozen). What the sweep actually
# shows: a frozen cycle clamps exactly when the integrator REVERSES. control.py
# permits that on purpose - "it may fall, cross zero, or reverse outright" -
# because the alternative deadlocks the loop, and a reversal is a full swing
# that can clear the bound on the far side in one step.
#
# That matters for this ticket rather than being trivia: it is the one shape
# where the saturation freeze and a clamp engagement coincide, so it is exactly
# where an alarm that shared a channel with saturation would lose the clamp.
grid_sweep = (-9000.0, -3000.0, 3000.0, 9000.0)
grid = [(pp, g) for pp in (-1900.0, -500.0, 0.0, 500.0, 1900.0)
for g in grid_sweep]
runs = {(pp, g): compute(pp, g, 0.0, t, sat_count=5, i_w=pp) for pp, g in grid}
frozen = [k for k, d in runs.items() if d.frozen]
frozen_clamps = [k for k in frozen if runs[k].i_clamped]
# The property, stated independently of the control law's own branches: the
# correction pushes AGAINST the standing integrator, and the resulting swing
# clears the bound.
reversals = [(pp, g) for pp, g in frozen
if pp * g < 0 and abs(pp + t.gain * g) > t.max_w]
print(f" {len(frozen)} of {len(grid)} sweep cells are frozen; "
f"those that also clamp: {frozen_clamps}")
check(f"a frozen cycle CAN clamp, and only by reversing "
f"({len(frozen_clamps)} of {len(frozen)} frozen cells)",
frozen_clamps and frozen_clamps == reversals)
check("...and a frozen cycle pushing the way it already leans never clamps - "
"that is the freeze doing its job",
not [k for k in frozen if runs[k].i_clamped and k[0] * k[1] > 0])
check("a healthy cycle sets neither flag",
not any([compute(0.0, 100.0, 0.0, t).i_clamped,
compute(0.0, 100.0, 0.0, t).clamped]))
# ==============================================================================
print()
print("5. no path reaches the wire around the clamp")
async def _direct():
hass = _Hass(0.0)
c = Controller({"setpoint_entity": "number.sp", "max_w": 2000,
"meter_source": "off"},
hass, _Store(), _Pub(), stop=asyncio.Event())
c.dev_min, c.dev_max = -LIVE_MAX_W, LIVE_MAX_W
# The shape the heartbeat, shutdown() and handle_action() all use.
await c.write_setpoint(9999.0, force=True)
await c.write_setpoint(-9999.0, force=True)
await c.write_setpoint(float("nan"), force=True)
await c.write_setpoint(1500.0, force=True)
return hass, c
hass_d, ctl_d = asyncio.run(_direct())
print(f" asked for 9999 / -9999 / nan / 1500 W; the device saw "
f"{hass_d.writes}")
check("a direct out-of-range write is clamped, not passed through",
hass_d.writes[:2] == [2000.0, -2000.0])
check("a nan is refused and becomes 0 W, per 11.1 rule 1",
hass_d.writes[2] == 0.0)
check("...and is NOT counted as a clamp engagement - it is a transport bug",
sum(1 for e in ctl_d.events if REASON_CODE in e) == 2)
check("an in-range value is untouched", hass_d.writes[3] == 1500.0)
check("every emitted frame satisfies |emitted| <= the rating",
all(abs(w) <= 2000.0 for w in hass_d.writes))
print()
if fails:
print(f"{len(fails)} FAILED: {', '.join(fails)}")
sys.exit(1)
print("all checks passed")
File diff suppressed because it is too large Load Diff
+213
View File
@@ -0,0 +1,213 @@
"""Runnable check for the control loop's cadence. `python3 test_loop.py`
SAFETY-12: the loop must evaluate on a clock as well as on a change. Before the
fix, `run_control()` ran `cycle()` only when the meter value differed from the
previous poll, so a meter that repeats a value ran NO control cycles while the
heartbeat kept re-emitting the last command - measured on the rig during TEL-05
as 70 s at a literally constant 350.0 W with 14 arrivals and zero cycles.
⚠️ THIS TEST TAKES ~95 SECONDS OF WALL CLOCK (measured 93 s), on purpose, and
that figure is the only thing standing between it and a future developer who
decides a slow test is a broken one. The thing under test is a wall-clock floor
inside a loop that sleeps 1 s; mocking the clock would leave the assertion
testing the mock. It drives the real `run_control()` and counts what actually
happened. Eight runs of the real loop is what the eight scenarios cost.
⚠️ If you shorten the windows, re-measure and update this number and the one in
HANDOVER.md. An understated runtime reads as a hang.
`cycle_floor_s` is read from options with no lower bound in code, so a huge
value reproduces the pre-fix behaviour exactly. Every claim below is therefore
an A/B on one binary, not a comparison against a remembered old one.
"""
import asyncio
import sys
import time
from app.main import Controller
fails = []
NEVER = 1e9 # cycle_floor_s so large the floor can never elapse: pre-fix
def check(name, cond):
print(f" {'ok ' if cond else 'FAIL'} {name}")
if not cond:
fails.append(name)
class FakeHass:
"""A scripted meter and a plant that tracks the command instantly."""
def __init__(self, meter, soc_dies_at=None):
self.meter = meter
self.soc_dies_at = soc_dies_at
self.batt = 0.0
self.writes = []
self.t0 = time.monotonic()
async def limits(self, entity):
return (-5000.0, 5000.0, 1.0)
async def number(self, entity, invert=False):
now = time.monotonic() - self.t0
if entity == "sensor.grid":
return self.meter(now)
if entity == "sensor.batt":
return self.batt
if entity == "sensor.soc":
if self.soc_dies_at is not None and now >= self.soc_dies_at:
return None
return 50.0
return None
async def set_number(self, entity, value):
self.writes.append(float(value))
self.batt = float(value)
return True
class _Store:
data = {}
def set(self, *a):
pass
class _Pub:
enabled = False
def publish(self, *a, **k):
pass
def close(self):
pass
BASE = {"setpoint_entity": "number.sp", "meter_entity": "sensor.grid",
"soc_entity": "sensor.soc", "batt_entity": "sensor.batt",
"meter_source": "off", "max_w": 2000, "gain": 0.6, "slew_w": 1000,
"deadband_w": 15, "target_grid_w": -10, "step_w": 10,
"heartbeat_s": 10, "stale_input_s": 15}
def drive(meter, floor, seconds, soc_dies_at=None, **over):
"""Run the real control loop for `seconds`; return (cycle times, writes)."""
async def go():
hass = FakeHass(meter, soc_dies_at)
c = Controller(dict(BASE, cycle_floor_s=floor, **over), hass, _Store(),
_Pub(), stop=asyncio.Event())
c.auto = True
stamps = []
real = c.cycle
async def counted():
stamps.append(time.monotonic() - hass.t0)
await real()
c.cycle = counted
task = asyncio.create_task(c.run_control())
await asyncio.sleep(seconds)
c.stopping = True
await asyncio.wait_for(task, timeout=10)
return stamps, hass.writes, c.target, c.reason
return asyncio.run(go())
def worst_gap(stamps, seconds):
"""Longest stretch with no cycle, including the tail after the last one."""
edges = [0.0] + list(stamps) + [seconds]
return max(b - a for a, b in zip(edges, edges[1:]))
# --------------------------------------------------------------------------- #
print("a meter that repeats the same value")
FLAT = lambda t: 350.0 # noqa: E731
FLOOR, SECONDS = 3.0, 12.0
pre_stamps, pre_writes, _, _ = drive(FLAT, NEVER, SECONDS)
# The defect itself, demonstrated rather than described: one cycle at startup
# and then nothing for the rest of the window, while writes keep happening.
check("pre-fix: a constant meter runs exactly one cycle, at startup",
len(pre_stamps) == 1)
check("...while the heartbeat keeps writing anyway, so nothing looks wrong",
len(pre_writes) >= 2)
post_stamps, post_writes, _, _ = drive(FLAT, FLOOR, SECONDS)
check("post-fix: the same constant meter runs cycles throughout",
len(post_stamps) >= int(SECONDS / FLOOR) - 1)
# The actual acceptance criterion: a BOUND on the gap, not a count.
gap = worst_gap(post_stamps, SECONDS)
check(f"...with no gap longer than the floor ({gap:.1f}s <= {FLOOR + 1.5:.1f}s)",
gap <= FLOOR + 1.5)
check("...and the loop actually corrected, instead of holding a stale command",
len(post_writes) > len(pre_writes))
# --------------------------------------------------------------------------- #
print("a meter that changes normally - nothing may move")
# ⚠️ THE REGRESSION GUARD, and the more important half. The floor is a deadline
# on the gap between cycles, reset by every cycle whichever trigger fired - not
# a second scheduler. On a healthy meter it must never elapse, so the cycle
# count, the write count and the commanded values must be IDENTICAL either side
# of the fix. Anything else is a new write cadence, which SAFETY-12 forbids:
# the deadband, step_w and the 10 s force-write decide what reaches the wire,
# and this change is only allowed to alter when the loop thinks.
STEPS = [0.0, 300.0, 850.0, -400.0, 120.0, -900.0, 60.0]
LIVE = lambda t: STEPS[int(t // 2) % len(STEPS)] # noqa: E731
a_stamps, a_writes, _, _ = drive(LIVE, NEVER, 12.0)
b_stamps, b_writes, _, _ = drive(LIVE, FLOOR, 12.0)
check("a changing meter runs the same number of cycles as before the fix",
len(a_stamps) == len(b_stamps))
check("...commanding exactly the same sequence of values", a_writes == b_writes)
# --------------------------------------------------------------------------- #
print("a meter parked at the target - the converged operating point")
# The case TEL-05 warned about: at the converged target the power figure
# legitimately repeats for minutes. The loop must still run, and must still
# write nothing but the heartbeat, because the deadband - not the arrival
# trigger - is what holds the command still.
AT_TARGET = lambda t: -10.0 # noqa: E731
c_stamps, c_writes, _, _ = drive(AT_TARGET, NEVER, 12.0)
d_stamps, d_writes, _, _ = drive(AT_TARGET, FLOOR, 12.0)
check("the loop evaluates at the converged point too", len(d_stamps) > len(c_stamps))
check("...and the deadband still holds the command, so writes are unchanged",
d_writes == c_writes)
# --------------------------------------------------------------------------- #
print("an input dies while the meter repeats - the safety claim must still fire")
# ⚠️ THE STRONGEST PROPERTY IN SAFETY-12, and the one that makes it a safety fix
# rather than a latency fix. The `inputs missing -> 0 W` claim lives INSIDE
# cycle() (main.py, the stale_input_s branch and the P_SAFETY claim below it).
# Pre-fix, on a meter whose value happened to repeat, cycle() was never called
# again - so that claim could not fire AT ALL. The controller held its last
# command through an input outage of unbounded length with the status line still
# reading "loop -> N W".
#
# Layer 1 does not cover this either on a `meter_source: off` install: no P1 age
# sensor is published (see Controller.publish), so the firmware has nothing to
# veto on, and the heartbeat keeps its watchdog fed the whole time.
#
# stale_input_s is shortened to 3 s here so the outage resolves inside a test
# rather than inside a quarter of a minute. The mechanism is the default one.
c2_stamps, c2_writes, pre_target, pre_reason = drive(
FLAT, NEVER, 10.0, soc_dies_at=4.0, stale_input_s=3)
check("pre-fix: a dead input during a constant meter never reaches the safety claim",
pre_target != 0.0 and "safety" not in pre_reason)
check(f"...so it holds its last command instead ({pre_target:g} W, {pre_reason!r})",
pre_target == 220.0)
d2_stamps, d2_writes, post_target, post_reason = drive(
FLAT, FLOOR, 10.0, soc_dies_at=4.0, stale_input_s=3)
check("post-fix: the same outage commands 0 W", post_target == 0.0)
check(f"...through the safety claim, not by accident ({post_reason!r})",
"safety" in post_reason)
check("...and the 0 W actually reached the device", 0.0 in d2_writes)
# --------------------------------------------------------------------------- #
print()
if fails:
print(f"{len(fails)} FAILED: " + "; ".join(fails))
sys.exit(1)
print("all checks passed")
+267 -2
View File
@@ -26,9 +26,15 @@ def check(name, cond):
def base(**over):
"""A configuration that must start. Everything else is this, spoilt."""
# ⚠️ The maintenance rates are named EXPLICITLY, at the values config.yaml
# ships (MAINT-04). They are not decoration: with the key absent, rule 6
# falls back to main.py's 2500 W, which against this 2000 W ceiling is a
# genuine refusal - so a base() without them would be a config that does
# not start, and every check below would be measuring the wrong thing.
o = {"setpoint_entity": "number.goodwe_setpoint", "max_w": 2000,
"soc_entity": "sensor.soc", "batt_entity": "sensor.batt",
"meter_source": "off"}
"meter_source": "off",
"maintenance_charge_w": 2000, "maintenance_discharge_w": 2000}
o.update(over)
return o
@@ -151,6 +157,255 @@ check("with meter_source off the number governs nothing and is not refused",
check("the constant is the firmware's, not a rounded guess",
FIRMWARE_INGEST_VETO_S == 30.0)
# --------------------------------------------------------------------------- #
print("6. a maintenance charge the ceiling can actually accommodate (MAINT-04)")
# ⚠️ THE SHIPPED BUG. config.yaml shipped max_w 2000 against
# maintenance_charge_w 2500, so the cheap-window shaping asked for hi=-2500
# against the supervised lo=-2000 - lo > hi, arbiter clause 3, 0 W. Maintenance
# silently never charged in a cheap window on a default install.
p = validate_startup_config(base(max_w=2000, maintenance_charge_w=2500),
-5000, 5000)
check("maintenance_charge_w above max_w is refused", len(p) == 1)
check("...and the message names the charge rate", why(p, "2500"))
check("...and the ceiling", why(p, "2000"))
check("...and says which one wins", why(p, "THE CEILING WINS"))
# ⚠️ Equality is REACHABLE, not a near miss: lo=-2000, hi=-2000 is the single
# point -2000 W. Refusing it would be exactly the trivia that gets a validator
# deleted, and it is the boundary the rule is written `>` for.
check("maintenance_charge_w exactly at max_w starts",
validate_startup_config(base(max_w=2000, maintenance_charge_w=2000),
-5000, 5000) == [])
check("below it starts",
validate_startup_config(base(max_w=2000, maintenance_charge_w=1500),
-5000, 5000) == [])
# Not scoped to maintenance_enabled: the pair is contradictory either way, and
# scoping would leave the trap armed for the one action - ticking the box - that
# cannot be expected to change what a charge rate means.
check("...and it is refused with maintenance_enabled off, as shipped",
why(validate_startup_config(
base(max_w=2000, maintenance_charge_w=2500,
maintenance_enabled=False), -5000, 5000), "THE CEILING WINS"))
# ⚠️ THE SIBLING, CHECKED RATHER THAN ASSUMED. The shaping is NOT symmetric:
# drain files Claim.set at P_MAINTENANCE and owns the actuator, so the ceiling
# CLAMPS it (+2000 W) instead of contradicting it, and the arbiter's explain()
# names the clamp every cycle. Loud, and it still moves energy - so warned, not
# refused. If someone later "fixes the asymmetry" by making this a refusal,
# this check goes red and they have to justify it.
check("maintenance_discharge_w above max_w WARNS but starts",
validate_startup_config(base(max_w=2000, maintenance_discharge_w=2500),
-5000, 5000) == [])
# The demonstration the rule is derived from, asserted rather than described.
from app.arbiter import ( # noqa: E402
Claim, P_LOOP, P_MAINTENANCE, P_MAINT_SHAPE, P_SAFETY, resolve,
)
def _charge(max_w, charge_w):
return resolve([Claim.limit("supervised", P_SAFETY, -max_w, max_w, "max_w"),
Claim.limit("maintenance", P_MAINT_SHAPE, hi=-charge_w,
reason="cheap-window charge"),
Claim.set("loop", P_LOOP, 500.0, "grid")])
def _drain(max_w, disch_w):
return resolve([Claim.limit("supervised", P_SAFETY, -max_w, max_w, "max_w"),
Claim.set("maintenance", P_MAINTENANCE, disch_w, "drain"),
Claim.set("loop", P_LOOP, 500.0, "grid")])
r = _charge(2000, 2500)
check("arbiter: the refused pair really does emit 0 W",
r.target_w == 0.0 and r.contradiction)
r = _charge(2000, 2000)
check("arbiter: the permitted equality really does charge",
r.target_w == -2000.0 and not r.contradiction)
r = _charge(3000, 2500)
check("arbiter: raising max_w above the rate charges at the rate",
r.target_w == -2500.0 and not r.contradiction)
r = _drain(2000, 2500)
check("arbiter: the drain SET is clamped, not contradicted",
r.target_w == 2000.0 and not r.contradiction)
check("...and the clamp names itself, which is why it is not refused",
"supervised" in (r.bound_by or "") and "limited by" in r.explain())
# --------------------------------------------------------------------------- #
print("the SHIPPED defaults must pass the rule they are governed by (AC3)")
# ⚠️ Shipping a default the validator refuses would be worse than the bug it
# refuses. Read config.yaml itself rather than a transcription of it - a
# transcription is what lets the two drift apart silently. Regex, not PyYAML:
# the add-on image has no PyYAML (Dockerfile installs py3-aiohttp/py3-paho-mqtt
# only) and a check that skips on ImportError is not a check.
import os # noqa: E402
import re # noqa: E402
_CFG = os.path.join(os.path.dirname(os.path.abspath(__file__)), "config.yaml")
with open(_CFG, encoding="utf-8") as _fh:
_txt = _fh.read()
_shipped = {k: float(v) for k, v in re.findall(
r"^ (max_w|maintenance_charge_w|maintenance_discharge_w|sign_disagree_w): "
r"(-?\d+(?:\.\d+)?)\s*$", _txt, re.M)}
check("config.yaml really was read (all four defaults found)",
set(_shipped) == {"max_w", "maintenance_charge_w",
"maintenance_discharge_w", "sign_disagree_w"})
check(f"the shipped defaults are accepted {_shipped}",
validate_startup_config(base(**_shipped), -5000, 5000) == [])
check("...and the shipped charge rate is at or under the shipped ceiling",
_shipped["maintenance_charge_w"] <= _shipped["max_w"])
check("...and the shipped discharge rate too, so it warns about nothing",
_shipped["maintenance_discharge_w"] <= _shipped["max_w"])
# --------------------------------------------------------------------------- #
print("the shipped meter_freeze_s, pinned (TEST-06 / AC3)")
# ⚠️ WHY A NUMBER GETS ITS OWN CHECK. `meter_freeze_s: 300` is the threshold
# SAFETY-18's entire argument is about, and before this it was pinned NOWHERE:
# 300 -> 3600 left test_freeze.py AND this file green, because the schema
# accepts int(60,3600) and no check ever read the shipped value. The schema
# bounds what is CONFIGURABLE; this pins what is SHIPPED, which is a different
# question and the one the docs answer.
#
# Read by regex like _shipped above, for the same reason: no PyYAML in the
# image, and a check that skips on ImportError is not a check.
_freeze_s = [float(v) for v in re.findall(
r"^ meter_freeze_s: (\d+(?:\.\d+)?)\s*$", _txt, re.M)]
_freeze_schema = re.findall(r"^ meter_freeze_s: int\((\d+),(\d+)\)\s*$",
_txt, re.M)
check(f"config.yaml carries exactly one meter_freeze_s default {_freeze_s}",
len(_freeze_s) == 1)
check("...and exactly one schema range for it", len(_freeze_schema) == 1)
_hold_s = _freeze_s[0] if _freeze_s else None
_lo, _hi = ((float(_freeze_schema[0][0]), float(_freeze_schema[0][1]))
if _freeze_schema else (None, None))
check("the shipped meter_freeze_s is still the measured 300 s that freeze.py, "
"config.yaml's own comment and test_freeze.py are all written at",
_hold_s == 300.0)
check(f"...and it is inside its own schema range ({_lo}-{_hi} s)",
None not in (_lo, _hi, _hold_s) and _lo <= _hold_s <= _hi)
# The two properties the number has to have, DERIVED rather than restated - so
# they go red on a value that is merely schema-legal. Both come straight from
# config.yaml's own comment on the key.
#
# upper: the alert must land inside the 900 s quarter that bills the capacity
# tariff, or the notification arrives after the money is spent;
# lower: the slowest a HEALTHY meter can tick a 1 Wh register and still look
# unchanged for hold_s is 3600/hold_s W, and that must stay inside the
# deadband or the conjunction's two terms stop being mutually
# exclusive (freeze.py, "WHERE hold_s COMES FROM").
_QUARTER_S = 900.0
_shipped_deadband = [float(v) for v in re.findall(
r"^ deadband_w: (\d+(?:\.\d+)?)\s*$", _txt, re.M)]
check(f"config.yaml carries exactly one deadband_w default {_shipped_deadband}",
len(_shipped_deadband) == 1)
_db = _shipped_deadband[0] if _shipped_deadband else None
_slowest_healthy_w = 3600.0 / _hold_s if _hold_s else None
print(f" shipped meter_freeze_s {_hold_s} s: alert inside the "
f"{_QUARTER_S:.0f} s billing quarter, and a healthy meter slow enough to "
f"look unchanged for it is {_slowest_healthy_w} W against a "
f"{_db} W deadband")
check(f"...the alarm still lands inside the {_QUARTER_S:.0f} s billing quarter",
_hold_s is not None and _hold_s < _QUARTER_S)
check("...and a healthy meter that looks unchanged for it is still inside the "
"deadband, so the two terms stay mutually exclusive",
_slowest_healthy_w is not None and _db is not None
and _slowest_healthy_w < _db)
# ⚠️ AND THE RANGE'S OWN BOUNDS. "300 sits inside the range" is worth nothing
# while the range itself is unpinned: widened to `int(1,86400)` an operator can
# configure `meter_freeze_s: 86400` and the guard is dead for a day at a time,
# with every check in this repo still green. That mutation survived the first
# cut of this PR and is what this block closes. Both numbers are argued for in
# config.yaml's own comment and in freeze.py, and each is re-derived here:
#
# 60 s the floor freeze.py calls NOT A RECOMMENDATION - there a healthy
# meter can look unchanged at 60 W, four times the shipped deadband, so
# the conjunction's two terms stop being mutually exclusive and only
# the margin is left;
# 3600 s the ceiling - already four times the billing quarter, so the top of
# the range has given up detecting a freeze inside the quarter that
# bills it. It is the last bound between a supported configuration and
# a disabled guard, and moving it is a decision rather than a tidy-up.
print(f" schema range {_lo}-{_hi} s: at the floor a healthy meter that looks "
f"unchanged is "
+ (f"{3600.0 / _lo:.0f} W" if _lo else "?")
+ f" against a {_db} W deadband; the ceiling is "
+ (f"{_hi / _QUARTER_S:.1f}x" if _hi else "?")
+ " the billing quarter")
check(f"the schema range itself is still the argued 60-3600 s (found "
f"{_lo}-{_hi})", (_lo, _hi) == (60.0, 3600.0))
check("...a floor that gives up the mutual exclusion and keeps only the "
"margin, which is why config.yaml calls it not a recommendation",
None not in (_lo, _db) and 3600.0 / _lo > _db)
check("...and a ceiling already past the billing quarter, so no value the "
"schema still accepts can alarm inside the quarter that bills it",
_hi is not None and _hi > _QUARTER_S)
# --------------------------------------------------------------------------- #
print("the KNOWN cross-file default divergence, pinned rather than commented")
# ⚠️ WHAT THIS GUARDS, AND WHY IT ASSERTS THE BUG RATHER THAN THE FIX.
# config.yaml and validate.py now default maintenance_charge_w to 2000, while
# main.py's two `opts.get(..., 2500)` fallbacks and MaintConfig's dataclass
# default still say 2500. So an options dict with the key ABSENT is ACCEPTED by
# rule 6 at 2000 and then built by main.py at 2500 - which is a contradiction:
#
# validate_startup_config -> [] (accepted)
# MaintConfig().charge_w -> 2500.0
# arbiter -> 0.0 W, contradiction=True
#
# That is the bug rule 6 exists to catch, passing rule 6. It is unreachable on a
# Supervisor install (every key is always supplied from config.yaml) but fully
# reachable from sim/inject.py and from tests - which is exactly where anyone
# debugging maintenance works.
#
# The fix is ONE SOURCE OF TRUTH for option defaults. That is a bigger change
# than this ticket and it lands in main.py, which another ticket is live in - so
# the divergence is RECORDED here instead of closed. The check therefore passes
# on the mismatch and FAILS when either side moves, in EITHER direction:
#
# - config.yaml moves -> FAIL: re-decide what the fallback should be
# - main.py is fixed to match -> FAIL: the divergence is gone, DELETE this
# check and the ⚠️ block in validate.py
#
# A comment cannot do that, and "a comment where a check belonged" is this
# repo's most repeated defect.
_APP = os.path.join(os.path.dirname(os.path.abspath(__file__)), "app")
def _read(*parts):
with open(os.path.join(*parts), encoding="utf-8") as fh:
return fh.read()
# Every literal fallback for this key in main.py, whatever the call shape.
_main_falls = [float(m) for m in re.findall(
r'\.get\(\s*"maintenance_charge_w"\s*,\s*(-?\d+(?:\.\d+)?)\s*\)',
_read(_APP, "main.py"))]
# ...and MaintConfig's own dataclass default, which is the one that bites when
# main.py is bypassed entirely (a Maintenance built straight from defaults).
_dc_falls = [float(m) for m in re.findall(
r"^\s*charge_w:\s*float\s*=\s*(-?\d+(?:\.\d+)?)\s*$",
_read(_APP, "maintenance.py"), re.M)]
check(f"main.py's fallbacks are still found and still 2500 {_main_falls}",
len(_main_falls) == 2 and set(_main_falls) == {2500.0})
check(f"MaintConfig's default is still found and still 2500 {_dc_falls}",
len(_dc_falls) == 1 and set(_dc_falls) == {2500.0})
check(f"...and config.yaml still ships 2000, so the divergence is UNCHANGED "
f"(shipped {_shipped['maintenance_charge_w']:.0f}, code 2500)",
_shipped["maintenance_charge_w"] == 2000.0)
# ⚠️ And the consequence itself, through the REAL code rather than through the
# numbers above - so that if someone "fixes" this by changing a literal without
# understanding it, the thing that goes red is the behaviour, not the transcript.
from app.maintenance import MaintConfig # noqa: E402
_absent = {k: v for k, v in base().items() if k != "maintenance_charge_w"}
_built = MaintConfig().charge_w
check("with the key ABSENT the validator accepts...",
validate_startup_config(_absent, -5000, 5000) == [])
check(f"...while the code would build {_built:.0f} W against a "
f"{_absent['max_w']:.0f} W ceiling...", _built > _absent["max_w"])
_r = _charge(_absent["max_w"], _built)
check("...which the arbiter resolves to 0 W as a contradiction - the hole, "
"asserted so it cannot widen unnoticed",
_r.target_w == 0.0 and _r.contradiction)
# --------------------------------------------------------------------------- #
print("max_w must be a positive magnitude")
# Schema-blocked on a real install, but validate_startup_config() is also
@@ -160,8 +415,18 @@ for bad in (0, -1, -2000):
check(f"max_w {bad} is refused", len(p) == 1 and why(p, "must be positive"))
check("...and the message explains it would read as a trip, not a mistake",
why(validate_startup_config(base(max_w=0), -5000, 5000), "contradiction"))
# ⚠️ SAFETY-02 changed what this case has to say, and the change is real rather
# than bookkeeping: with a 1 W ceiling the default 300 W sign-disagreement
# threshold is unreachable, so rule 5 refuses - correctly, because the watchdog
# would report itself armed while being incapable of firing. What this check is
# FOR is "a positive max_w is not itself a refusal", so it now names a sign
# threshold the ceiling can actually reach, and the interaction gets its own
# check below.
check("the smallest sane max_w still starts",
validate_startup_config(base(max_w=1), -5000, 5000) == [])
validate_startup_config(base(max_w=1, sign_disagree_w=0.5,
maintenance_charge_w=1), -5000, 5000) == [])
check("...but that same 1 W ceiling with the DEFAULT sign threshold is refused",
why(validate_startup_config(base(max_w=1), -5000, 5000), "never fire"))
# --------------------------------------------------------------------------- #
print("what must NOT be refused - a validator that fires on trivia gets deleted")
+505
View File
@@ -0,0 +1,505 @@
"""Runnable check for the sign-disagreement watchdog. `python3 test_watchdog.py`
Every case here is either a fault this must catch or a piece of NORMAL behaviour
it must not mistake for one. The second list is the longer one on purpose: an
alarm that fires on ordinary operation is worse than no alarm, because the next
person switches it off.
The headline validation is not here - it is `sim/replay_sign.py` in the infra
repo, which drives this module over the two captured bus days. This file is the
table-driven half: boundaries, timers, the latch, and the arming gate.
"""
import sys
from datetime import datetime, timedelta, timezone
from app.watchdog import (
SignWatchdog, SignWatchdogConfig, loop_is_following,
)
fails = []
T0 = datetime(2026, 8, 20, 15, 0, 0, tzinfo=timezone.utc)
def check(name, cond):
print(f" {'ok ' if cond else 'FAIL'} {name}")
if not cond:
fails.append(name)
def drive(samples, cfg=None, following=True):
"""Feed (offset_s, commanded_w, grid_w) tuples. Returns (wd, trip_offset)."""
wd = SignWatchdog(cfg)
trip_at = None
for item in samples:
offset, cmd, grid = item[0], item[1], item[2]
f = item[3] if len(item) > 3 else following
if wd.evaluate(cmd, grid, f, T0 + timedelta(seconds=offset)) and trip_at is None:
trip_at = offset
return wd, trip_at
def opposed(duration_s, step=5, cmd=-1000, grid=1500, following=True, start=0):
return [(start + t, cmd, grid, following)
for t in range(0, int(duration_s) + 1, step)]
print("the fault it exists to catch")
wd, at = drive(opposed(300))
check("charging into an importing house trips", wd.tripped)
check("and trips at 180 s, not before", at == 180)
check("the reason names both signals", "-1000" in wd.reason and "+1500" in wd.reason)
print(f" e.g. {wd.reason!r}")
wd, at = drive(opposed(300, cmd=2000, grid=-2000))
check("discharging into an exporting house trips too", wd.tripped and at == 180)
print()
print("the boundaries")
wd, _ = drive(opposed(600, cmd=-300, grid=1500))
check("300 W exactly does NOT qualify (strictly greater)", not wd.tripped)
wd, _ = drive(opposed(600, cmd=-301, grid=1500))
check("301 W does", wd.tripped)
wd, _ = drive(opposed(600, cmd=-1000, grid=300))
check("a 300 W meter does not qualify either", not wd.tripped)
wd, _ = drive(opposed(175))
check("175 s of disagreement does not trip", not wd.tripped)
wd, _ = drive(opposed(180))
check("180 s does (>= hold_s)", wd.tripped)
print()
print("continuity - the rule that produced zero false positives on real data")
broken = opposed(175) + [(180, -1000, -1500)] + opposed(175, start=185)
wd, _ = drive(broken)
check("179 s, one agreeing sample, then 175 s more: no trip", not wd.tripped)
check("and the window really was reset", wd.disagree_since is not None)
long_enough = broken + opposed(200, start=365)
wd, _ = drive(long_enough)
check("but a full 180 s after the reset does trip", wd.tripped)
print()
print("things that are NOT faults")
# The load-step transient. Measured on the healthy capture: longest 30 s.
wd, _ = drive(opposed(30) + [(35 + t, 800, 900) for t in range(0, 300, 5)])
check("a 30 s load-step crossover does not trip", not wd.tripped)
# SoC 10 %, Standby, declining a 1350 W discharge. The battery does nothing, so
# the meter stays where it was and the command agrees with it.
wd, _ = drive([(t, 1350, 1350) for t in range(0, 900, 5)])
check("a battery refusing a discharge command cannot trip it", not wd.tripped)
# A frozen meter drives command and reading into agreement, not disagreement.
wd, _ = drive([(t, 800, 800) for t in range(0, 900, 5)])
check("a frozen meter cannot trip it", not wd.tripped)
# Maintenance grid charge: 2500 W of charge against an importing house, for
# hours. Opposed, both huge - and entirely correct.
wd, _ = drive(opposed(7200, cmd=-2500, grid=3000, following=False))
check("two hours of maintenance grid charge does not trip", not wd.tripped)
check("and the timer never even started", wd.disagree_since is None)
wd, _ = drive([(t, -1000, None) for t in range(0, 900, 5)])
check("a missing meter reading does not trip", not wd.tripped)
wd, _ = drive(opposed(175) + [(180, -1000, None)] + opposed(600, start=185))
check("a missing reading mid-window RESETS it rather than accumulating",
wd.tripped and wd.disagree_since is not None)
wd, at = drive(opposed(175) + [(180, -1000, None)] + opposed(600, start=185))
check("...so the trip lands 180 s after the gap, not after 180 s total",
at == 365)
wd, _ = drive([(t, 0.0, 3000) for t in range(0, 900, 5)])
check("a 0 W command is not a wrong-direction command", not wd.tripped)
print()
print("the latch")
wd, _ = drive(opposed(200))
check("tripped after the fault", wd.tripped)
# The trip commands 0 W, which removes the command side of the comparison.
for t in range(200, 900, 5):
wd.evaluate(0.0, 3000, True, T0 + timedelta(seconds=t))
check("stays tripped once the command is zeroed", wd.tripped)
for t in range(900, 1500, 5):
wd.evaluate(1000, 1000, True, T0 + timedelta(seconds=t))
check("stays tripped even once everything looks healthy again", wd.tripped)
wd.reset()
check("reset() is the only way out", not wd.tripped and wd.disagree_since is None)
check("and the reason is cleared with it", wd.reason == "")
print()
print("the thresholds are configuration, not constants")
cfg = SignWatchdogConfig(min_w=800.0, hold_s=60.0)
wd, at = drive(opposed(300, cmd=-1000, grid=1500), cfg=cfg)
check("a 60 s hold trips at 60 s", wd.tripped and at == 60)
wd, _ = drive(opposed(600, cmd=-700, grid=1500), cfg=cfg)
check("an 800 W gate ignores a 700 W command", not wd.tripped)
print()
print("the configuration that would disarm it is refused")
from app.validate import validate_startup_config # noqa: E402
base = {"setpoint_entity": "number.sp", "meter_source": "off",
"meter_phases": 1, "meter_max_age_s": 30}
p = validate_startup_config({**base, "max_w": 2000, "sign_disagree_w": 300},
-5000, 5000)
check("300 W under a 2000 W ceiling is accepted", p == [])
p = validate_startup_config({**base, "max_w": 2000, "sign_disagree_w": 2000},
-5000, 5000)
check("a threshold AT max_w is refused", len(p) == 1 and "never fire" in p[0])
# ⚠️ maintenance_charge_w is named here only to keep this case about the SIGN
# rule. MAINT-04 added a second rule against the same ceiling, and a 500 W
# max_w is also below the 2000 W shipped maintenance charge - so without this
# the dict is refused twice, for two correct but different reasons, and the
# `len(p) == 1` below would be measuring the interaction rather than this rule.
p = validate_startup_config({**base, "max_w": 500, "sign_disagree_w": 3000,
"maintenance_charge_w": 500}, -5000, 5000)
check("a threshold above max_w is refused", len(p) == 1 and "never fire" in p[0])
p = validate_startup_config({**base, "max_w": 2000, "sign_disagree_w": 1999},
-5000, 5000)
check("one watt under the ceiling is reachable, so accepted", p == [])
print()
print("the arming gate, derived from the REAL arbiter")
# ⚠️ THIS IS THE DECISION THE WHOLE TICKET TURNS ON, and until now it was
# asserted in three comment blocks and verified by nothing: every other case in
# this file passes `following` in as a literal. What has to hold is that the
# expression in main.py - loop won AND the emitted value points the way the loop
# asked AND is no larger - is never True for a resolution the loop did not
# actually get. So run real claims through the real resolve() and derive it.
from app.arbiter import ( # noqa: E402
P_LOOP, P_MAINT_SHAPE, P_MAINTENANCE, P_PEAK, P_SAFETY, Claim, resolve,
)
def following_from(claims, asked):
"""THE SHIPPED gate, not a copy of it.
⚠️ This used to transcribe main.py's expression with a comment saying
"keep the two in step", which is exactly the arrangement that does not keep
two things in step. Measured: editing the shipped gate alone to
`winner in ("loop", "peak")` left this suite - including the P_PEAK row below,
whose entire job is to catch that edit - completely green, while the watchdog
went blind for the duration of every peak event. Import it or the P_PEAK
assertion is decoration.
"""
r = resolve(claims)
return r, loop_is_following(r, asked)
MAXW = Claim.limit("supervised", P_SAFETY, -2000, 2000, "max_w")
DEV = Claim.limit("device", P_SAFETY, -5000, 5000, "rating")
# name, claims, what the loop asked, expected `following`
TABLE = [
("the loop alone, unopposed",
[Claim.set("loop", P_LOOP, 900)], 900, True),
("the loop bound by max_w - reduced, not redirected",
[Claim.set("loop", P_LOOP, 4000), MAXW], 4000, True),
("charging, bound by max_w on the other side",
[Claim.set("loop", P_LOOP, -4000), MAXW], -4000, True),
("maintenance owns the setpoint outright",
[Claim.set("loop", P_LOOP, 900),
Claim.set("maintenance", P_MAINTENANCE, -2500, "charge"), MAXW], 900, False),
("maintenance charge-only clamps the loop to 0 W",
[Claim.set("loop", P_LOOP, 900),
Claim.limit("maintenance", P_MAINT_SHAPE, hi=0.0, reason="charge-only"),
MAXW], 900, False),
# ⚠️ THE CASE `winner == \"loop\"` ALONE WOULD HAVE GOT WRONG. 1500 W, not
# the default 2500 W: at max_w 2000 a -2500 hi against a -2000 lo is an
# arbiter contradiction, not a charge - see main.py.
("cheap-window charge REDIRECTS the loop while it still 'wins'",
[Claim.set("loop", P_LOOP, 500),
Claim.limit("maintenance", P_MAINT_SHAPE, hi=-1500, reason="cheap-window"),
MAXW], 500, False),
("...and the contradiction at the shipped defaults is not following either",
[Claim.set("loop", P_LOOP, 500),
Claim.limit("maintenance", P_MAINT_SHAPE, hi=-2500, reason="cheap-window"),
MAXW], 500, False),
("a lo limit RAISES the magnitude without flipping the sign",
[Claim.set("loop", P_LOOP, 200),
Claim.limit("floor", P_SAFETY, lo=800), MAXW], 200, False),
("a safety stop wins outright",
[Claim.set("loop", P_LOOP, 900),
Claim.set("safety", P_SAFETY, 0.0, "stopped"), MAXW], 900, False),
("no claims at all - the arbiter's own failsafe",
[MAXW, DEV], None, False),
("the loop asked for zero",
[Claim.set("loop", P_LOOP, 0.0), MAXW], 0.0, False),
# The forward hazard, asserted as it stands TODAY rather than as it should
# be. When peak shaving lands, these expectations are what have to change.
("peak shaving at P_PEAK disarms the gate (loop_is_following's forward hazard)",
[Claim.set("loop", P_LOOP, 900),
Claim.set("peak", P_PEAK, 1800, "shaving"), MAXW], 900, False),
# ⚠️ AND THE ROW ABOVE CANNOT DETECT A WIDENED GATE ON ITS OWN, which was
# found by trying it: with the shipped gate edited to
# `winner in ("loop", "peak")` that case STILL comes out False, because peak
# shaving asked for MORE than the loop and the magnitude term rejects it
# anyway. The winner term is masked. This row isolates it - a peak claim
# smaller than the loop's ask and pointing the same way, so `winner` is the
# only term that can decide - and it is what actually turns red if the gate
# is widened without anyone revisiting the hazard.
("...and with peak BELOW the loop's ask, only the winner term separates them",
[Claim.set("loop", P_LOOP, 1800),
Claim.set("peak", P_PEAK, 900, "shaving"), MAXW], 1800, False),
]
for name, claims, asked, expect in TABLE:
r, got = following_from(claims, asked)
check(f"{name} -> following={expect}", got is expect)
if got is not expect:
print(f" resolution was {r.explain()!r}")
# And the gate is never True when the loop did not win, whatever else is going on.
check("no configuration above arms the gate without the loop winning",
all(not following_from(c, a)[1]
for _, c, a, _ in TABLE if following_from(c, a)[0].winner != "loop"))
print()
print("end to end: the trip is COMMAND ZERO, not alarm-only")
# ⚠️ THE FAULT INJECTED HERE IS A NEGATIVE GAIN, and that is not a contrivance -
# it is the minimal faithful model of what this watchdog is for. The check
# compares the loop's OUTPUT DIRECTION against the INPUT STATE it was regulating,
# so what it detects is a loop whose two ends have become decoupled in sign: a
# gain, a claim or a strategy with its polarity inverted, or an integrator so far
# wound up that unwinding takes minutes. (Illusmart's was the last one: it
# reached -14 768 W against a +-5000 W rail while writing once every 20-60 s, and
# that arithmetic alone is 5-15 minutes of wrong-direction command.)
#
# `gain` is float(0.05,1.0) in the schema, so this is not a reachable
# configuration - it is a stand-in for a code regression, which is the case a
# unit test can actually hold the line on.
import asyncio # noqa: E402
from app.main import Controller # noqa: E402
class _Hass:
def __init__(self):
self.writes = []
self.calls = []
# ⚠️ ADDED FOR SAFETY-03, AND THE REASON IS WORTH READING. The e2e below
# runs a DELIBERATELY INVERTED loop (gain -0.6) into a steady 2000 W import,
# which is a loop that winds straight into the rail - so the output-clamp
# alarm fires here, and the stub not having `call` is how that was
# discovered. This scenario is now a second, independent witness that the
# clamp alarm sees a real winding loop; test_clamp.py asserts it explicitly.
async def call(self, domain, service, data):
self.calls.append((domain, service, data))
async def limits(self, entity):
return (-5000.0, 5000.0, 1.0)
async def number(self, entity, invert=False):
# A house importing 2000 W, steadily, whatever the battery does.
return {"sensor.grid": 2000.0, "sensor.soc": 50.0,
"sensor.batt": 0.0}.get(entity)
async def set_number(self, entity, value):
self.writes.append(float(value))
return True
class _S:
data = {}
def set(self, *a):
pass
def get_time(self, *a):
return None
class _P:
enabled = False
def publish(self, *a, **k):
pass
def close(self):
pass
async def _e2e():
hass = _Hass()
opts = {"setpoint_entity": "number.sp", "meter_entity": "sensor.grid",
"soc_entity": "sensor.soc", "batt_entity": "sensor.batt",
"meter_source": "off", "max_w": 2000, "gain": -0.6,
"sign_disagree_w": 300, "sign_disagree_s": 1, "auto_start": True}
c = Controller(opts, hass, _S(), _P(), stop=asyncio.Event())
c.auto = True
c.dev_min, c.dev_max = -5000.0, 5000.0
seen_wrong_way = False
for _ in range(12):
await c.read_inputs()
await c.cycle()
if c.target < -300 and not c.sign_wd.tripped:
seen_wrong_way = True
await asyncio.sleep(0.25)
return c, hass, seen_wrong_way
ctrl, hass, wrong_way = asyncio.run(_e2e())
check("the broken loop really did command charge into an import first", wrong_way)
check("the watchdog tripped", ctrl.sign_wd.tripped)
check("the commanded value is 0 W", ctrl.target == 0.0)
check("0 W actually reached the device", 0.0 in hass.writes)
check("and it got there through the failsafe claim, not by accident",
"failsafe" in ctrl.reason)
print(f" status line: {ctrl.reason!r}")
check("the status page says so in words",
any("SIGN-DISAGREEMENT FAILSAFE" in c["text"] for c in ctrl.checks()))
check("the banner leads with it",
"FAILSAFE" in ctrl.status_payload()["banner"])
check("it stays at 0 W on later cycles too",
hass.writes[-1] == 0.0 and hass.writes.count(0.0) >= 1)
# ⚠️ THIS USED TO BE `check("MQTT reports the failsafe state", ctrl.sign_wd.tripped)`
# which restates the assertion three lines above and never touches the payload.
# The `status: "failsafe"` value was shipped untested. Capture what publish()
# actually hands the broker.
published = {}
ctrl.mqtt = type("P", (), {"enabled": True,
"publish": lambda _s, v: published.update(v),
"close": lambda _s: None})()
ctrl.publish()
check("MQTT's controller-status sensor reports 'failsafe'",
published.get("status") == "failsafe")
ctrl.sign_wd.reset()
ctrl.publish()
check("...and goes back to 'running' once cleared",
published.get("status") == "running")
print()
print("the latch survives a restart - it is the only thing between a tripped")
print("controller and the first thing an operator will try")
# ⚠️ THE BLOCKER THIS FILE MISSED FIRST TIME ROUND. `auto` is persisted and
# `tripped` was not, so reconstructing a Controller against the same Store -
# which is exactly what a container restart does - came back armed, into the
# fault, with no operator involved: tripped=False, banner NOT READY, and it
# resumed commanding -1000 then -2000 W before re-tripping 180 s later.
class _MemStore:
"""A Store that keeps its dict across "restarts", like the real file does."""
def __init__(self):
self.data = {"phase": "idle", "sign_tripped": False, "sign_reason": ""}
def set(self, k, v):
self.data[k] = v
def get_time(self, *a):
return None
async def _restart():
shared = _MemStore()
hass1 = _Hass()
opts = {"setpoint_entity": "number.sp", "meter_entity": "sensor.grid",
"soc_entity": "sensor.soc", "batt_entity": "sensor.batt",
"meter_source": "off", "max_w": 2000, "gain": -0.6,
"sign_disagree_w": 300, "sign_disagree_s": 1, "auto_start": True}
a = Controller(opts, hass1, shared, _P(), stop=asyncio.Event())
a.auto = True
a.dev_min, a.dev_max = -5000.0, 5000.0
for _ in range(12):
await a.read_inputs()
await a.cycle()
await asyncio.sleep(0.25)
# The restart. Same options, same store, brand new Controller and Hass.
hass2 = _Hass()
b = Controller(opts, hass2, shared, _P(), stop=asyncio.Event())
b.auto = True
b.dev_min, b.dev_max = -5000.0, 5000.0
for _ in range(6):
await b.read_inputs()
await b.cycle()
await asyncio.sleep(0.05)
return a, b, shared, hass2
pre, post, store, hass2 = asyncio.run(_restart())
check("it tripped before the restart", pre.sign_wd.tripped)
check("the trip reached the store", store.data["sign_tripped"] is True)
check("...with the reason beside it, so the operator is told why",
"SIGN_DISAGREEMENT" in store.data["sign_reason"])
check("the new Controller comes back TRIPPED", post.sign_wd.tripped)
check("...and says why, restored rather than re-detected",
"SIGN_DISAGREEMENT" in post.sign_wd.reason)
check("...commands 0 W from its very first cycle", post.target == 0.0)
check("...and never commanded anything else after the restart",
hass2.writes != [] and set(hass2.writes) == {0.0})
check("the banner still leads with the failsafe",
"FAILSAFE" in post.status_payload()["banner"])
# ⚠️ TWO PRESSES, and the first one must NOT clear it. `auto` is True while
# tripped, so the first press stops control; only the transition back to True
# clears. That is the documented behaviour and it is worth an assertion, because
# a one-press clear would be reachable by an operator who never read the banner.
asyncio.run(post.handle_action("auto_toggle")) # -> stopped
check("pressing stop does not clear the failsafe",
post.sign_wd.tripped and store.data["sign_tripped"] is True)
asyncio.run(post.handle_action("auto_toggle")) # -> started
check("pressing start DOES clear it, and clears the store with it",
not post.sign_wd.tripped and store.data["sign_tripped"] is False
and store.data["sign_reason"] == "")
# One more restart from the cleared store, to prove the clear persisted too.
fresh = Controller({"setpoint_entity": "number.sp", "meter_source": "off"},
_Hass(), store, _P(), stop=asyncio.Event())
check("...so the next restart comes back armed and untripped, as it should",
not fresh.sign_wd.tripped)
# ⚠️ THE WRITE ORDERING IS AN INVARIANT, SO IT GETS AN ASSERTION. main.py writes
# `sign_reason` and THEN `sign_tripped`, each one its own `save()`, so that a
# process that dies between the two leaves a reason nothing reads - harmless -
# rather than a latch with no explanation for the operator who has to decide
# whether to clear it. Documented reasoning is not enforcement: swapping those two
# lines used to pass every check in this file.
#
# The death is modelled as the second write raising. In the field it is a power
# cut or a container kill (Store.save() swallows OSError rather than raising), so
# what is asserted here is the PROPERTY - what survives on disk - not the
# mechanism.
class _CrashOnFlag(_MemStore):
def set(self, k, v):
if k == "sign_tripped" and v:
raise RuntimeError("power cut, mid-trip")
super().set(k, v)
async def _crash_mid_write():
store = _CrashOnFlag()
hass = _Hass()
opts = {"setpoint_entity": "number.sp", "meter_entity": "sensor.grid",
"soc_entity": "sensor.soc", "batt_entity": "sensor.batt",
"meter_source": "off", "max_w": 2000, "gain": -0.6,
"sign_disagree_w": 300, "sign_disagree_s": 1, "auto_start": True}
c = Controller(opts, hass, store, _P(), stop=asyncio.Event())
c.auto = True
c.dev_min, c.dev_max = -5000.0, 5000.0
died = False
for _ in range(12):
try:
await c.read_inputs()
await c.cycle()
except RuntimeError:
died = True
break
await asyncio.sleep(0.25)
return store, died
crashed_store, died = asyncio.run(_crash_mid_write())
check("the second write is the one that sets the flag", died)
check("a crash between the two writes leaves the REASON on disk",
"SIGN_DISAGREEMENT" in crashed_store.data.get("sign_reason", ""))
check("...and never a latch with nothing to explain it",
not crashed_store.data.get("sign_tripped"))
print()
if fails:
print(f"{len(fails)} FAILED: {', '.join(fails)}")
sys.exit(1)
print("all checks passed")