Files

T

glenn schrooyenandClaude Opus 5 9aed43990b SAFETY-02: a loop can be fresh, tracking and inside its rating and still push the wrong way

The stale-input watchdog asks whether the meter is still talking. The saturation
freeze asks whether the inverter can follow. The output clamp asks whether the
number is too big. A loop can pass all three while charging the battery out of
the grid, because the only thing wrong is the sign.

On 2026-08-20 a commercial controller did exactly that four times in one day,
for up to 77 minutes at a stretch, reaching a commanded -14 768 W against a
±5000 W inverter. app/watchdog.py compares the emitted setpoint against the
authoritative P1 figure and, when they oppose each other with both over 300 W
for 180 s continuously, files a failsafe SET claim at P_SAFETY, commands 0 W and
latches there until an operator starts control again.

Validated against that day rather than against an argument: sim/replay_sign.py
in the infra repo drives this module over the committed captures of both
2026-08-20 and the healthy 2026-08-23 and reaches 18 trips on the fault day -
every one inside a window NOTES.md §4.7 accounts for - and 0 on the healthy day.
The four episodes do share one signature: 0.3 / 0.9 / 10.5 / 0.0 % sign
agreement inside them against 92.9 % healthy.

Three decisions the ticket could not make:

- It is DISARMED whenever the grid-following loop is not what is driving. Sign
  agreement is a property of a loop that follows the meter, and a maintenance
  grid charge opposes the meter by design - 2500 W of charge into a house
  drawing 500 W is +3000 W on the meter against a -2500 W command, for hours.
  The gate tests the emitted number, not the winning claim's name, because a
  limit claim can let the loop "win" and still replace its value.
- The trip LATCHES. Commanding 0 W removes the command side of the comparison,
  so the disagreement always clears the moment the trip works.
- sign_disagree_w at or above max_w is refused at startup. The arbiter never
  lets a command exceed max_w, so such a threshold leaves the watchdog incapable
  of firing while the status page still reports it armed.

What it does not catch, said out loud: an inverter that ACTS opposite to its
command moves the meter further the same way, leaving command and meter in
agreement. That is the saturation freeze's job.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Du77usMj8XNKNFZGmUiWDa

2026-08-28 21:51:35 +02:00

7.6 KiB

Raw Permalink Blame History

Handover — 2026-08-23

Start here. README.md is what the thing is; this is where it stands, what is known to be broken, and what to do next.

1. State of the world

The add-on is controlling a real battery right now — the author's own house, a GoodWe GW5000S-BP.


running	`local_goodwe_controller` 0.4.0, `boot: auto`, holding grid at zero
installed as	a Local add-on: `/addons/goodwe_controller` on HA OS at 192.168.2.158
repo	Gitea `admin/goodwe-addon` (ssh://git@192.168.2.147:2222) — private
tuning	gain 0.6, `max_w` 5000, slew 1000, deadband 15, step 10, saturation 500/3, heartbeat 10 s
maintenance	disabled, and with no completion history — first cycle is "due" the moment it is enabled
e-stop	fitted at this site, Pi tap armed with `--panic`

The thing it replaced — a working Home Assistant YAML implementation — is parked at /config/packages_disabled_addon_migration/ on that machine. Rollback is: stop the add-on, move those four files back into /config/packages/, reload automations and helpers, switch input_boolean.goodwe_auto on.

⚠️ Never run both. They write the same setpoint. Two controllers on one actuator is the failure the whole design exists to prevent.

The research record — protocol, register map, why every tuning value is what it is, and a long "ruled out, do not re-investigate" list — lives in the other repo, Documents/goodwe, in NOTES.md. Read it before changing a constant.

2. Deploying a change

# 1. edit under goodwe_controller/
# 2. BUMP version: in goodwe_controller/config.yaml        <-- not optional
scp -r goodwe_controller root@192.168.2.158:/addons/
ssh root@192.168.2.158 "ha store reload && ha addons update local_goodwe_controller"
ssh root@192.168.2.158 "ha addons logs local_goodwe_controller | tail -20"

⚠️ Supervisor keys the built image by version:. Without a bump it silently reuses the old image and your change appears not to work. This cost two rounds of "the fix doesn't work" in one evening. It applies to client updates too — a client only gets a change when the version increments.

Run the checks before deploying. No framework needed:

cd goodwe_controller
python3 test_control.py && python3 test_maintenance.py && python3 test_arbiter.py \
  && python3 test_validate.py && python3 test_p1.py && python3 test_watchdog.py \
  && python3 test_loop.py

test_watchdog.py covers SAFETY-02's logic and its trip path. Its headline validation is not here at all: sim/replay_sign.py in the infra repo drives the same module over the two captured RS485 days and must fire on all four of the 2026-08-20 episodes and stay silent on 2026-08-23. That needs both checkouts — GOODWE_ADDON_REPO=/path/to/goodwe-addon python3 sim/replay_sign.py.

⚠️ test_loop.py takes about 95 seconds (measured 93 s) and that is not a bug. It drives the real control loop, which sleeps 1 s a tick, eight times over, to prove a wall-clock property about it (SAFETY-12); mocking the clock there would leave the assertion testing the mock. The other six suites together are ~440 checks and finish in seconds — test_watchdog.py spends ~3 s of that driving a real Controller through a trip. If you shorten test_loop.py's windows, re-measure and correct this figure — an understated runtime is what makes a slow test look like a hung one, and a slow test that looks hung gets deleted.

3. Known broken / unfinished

MQTT status entities are not adopted by Home Assistant. The add-on publishes correct discovery and live state — verified on the broker itself (goodwe_ctl/setpoint => 940.0). After the entities were deleted from the registry during development, HA would not re-adopt them, even after clearing the retained topics and reconnecting. Affects observability only; control is untouched. Try again after an HA restart, which is the cheapest next test.
The repository is private, so it cannot be used as an add-on repository URL — HA clones anonymously. Either make it public, or keep installing locally as at this site. Decide before the first client.
No real maintenance cycle has ever run. The mechanism is proven end-to-end in simulated time and partially on hardware, but a genuine low→full→hold cycle takes hours and has not happened. Do not decommission a client's vendor controller before one has.
The HA-side offline alarm is unproven. The Pi's 30 s panic write is the real cover; treat the HA alert as "within a few minutes".
Only platform 105 is supported — see §5.

4. What to do next, in order

Peak shaving as a real strategy. This is why the arbiter exists, and it is now purely additive: a claim producer emitting Claim.set("peak", P_PEAK, …) from the quarter-hour forecast, plus a priority. Nothing existing needs to change — the maintenance charge-only limit already cannot bind it. Highest value per unit of work in a capacity-tariff market.
Run a full maintenance cycle at the reference site and record what it actually costs in energy and time.
Fleet health before more features. This system's signature failure is silent: an entity-id prefix killed two safety alarms for a day, a shebang made every call 401 while looking healthy. One site with the author watching found those. Twenty sites will not.
Decide the repository question (§3.2).

5. The commercial question nobody has answered

A GoodWe "-20" / G2 inverter cannot be driven by this.

Platform comes from characters 6–8 of the serial number. Only ESU EMU ESA BPS BPU EMJ IJL — platform 105, the AA55-era meter bus — are supported. SBP G2 (SPB/SPN) and ES G2 (ESN/ESC) are platform 745, a different protocol. The reference unit is BPS.

That matters more than it first sounds: the SBP G2 is the model GoodWe currently sells to replace a GW5000S-BP. So today's product serves the installed base, not the base being installed. Everything above the firmware — control law, arbiter, maintenance, add-on, field guide — is protocol-agnostic and would carry over; only the register map is platform-specific.

Answering it is an afternoon: put the Pi tap passively on a G2 unit's meter bus with its vendor controller running, and capture. That either hands over the new register map or proves the approach does not transfer. Do this before building more features on top of a shrinking base.

6. Things that will bite you

Each of these was found the expensive way. They are in the code comments too.

run.sh must start #!/usr/bin/with-contenv sh. s6-overlay sanitises the environment, so a plain shebang means no SUPERVISOR_TOKEN and every HA call returns 401 — while homeassistant_api: true makes permissions look granted.
A wrong entity id is not an error anywhere in HA. It just never produces a value, and the automation's own state still reads on.
HA composes entity ids from the device area and device name. The same firmware yields sensor.goodwe_master_… at one site and sensor.cellar_goodwe_master_… at another.
input_datetime states parse timezone-naive; subtracting now() raises and kills the whole automation before it records itself as triggered. Use the timestamp attribute.
Alpine ships paho-mqtt 1.x, which has no CallbackAPIVersion, and musl has no aiohttp wheel — install from apk, not pip.
The inverter holds its last command forever. It has no meter-timeout. Every failsafe here exists because of that one fact, and anything in doubt commands 0 W rather than holding.

7.6 KiB Raw Permalink Blame History Unescape Escape