checks / unit (pull_request) Successful in 8m22s
A done-callback on run_control()'s task logs the traceback at the death and sets `stop`; amain() runs the ordinary wind-down (shutdown() -> command_site_zero()) and main() exits 70 (EX_SOFTWARE). Normal stops and run_control()'s refusal path are ignored by the callback (still 0 / 78). hass.py: a malformed JSON body, a JSON non-object, and an undecodable error body are an absence / a failed write, not an exception into the loop. checks() gains a first "control loop" row, red past FIRMWARE_INGEST_VETO_S. Why exit rather than restart in-process, and what the Supervisor actually does on exit (restarts only with the per-install Watchdog toggle on, default off): DECISIONS/SAFETY-61-control-task-death.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
107 lines
5.4 KiB
Markdown
107 lines
5.4 KiB
Markdown
# SAFETY-61 — the control task's death ends the process
|
|
|
|
## 0. The defect
|
|
|
|
`amain()` started `run_control()` as a bare task and waited on `stop`. Nothing
|
|
watched the task. An uncaught exception in `read_inputs()`, `cycle()` or
|
|
`publish()` ended the task silently, and `stop` was never set. The web UI, P1
|
|
and tariff tasks kept running, and the Supervisor saw a healthy container. The
|
|
traceback appeared only at shutdown. The ESP32's layer-1 veto zeroed the
|
|
inverter about 30 s later, so the hardware was safe. The pack then sat at 0 W
|
|
indefinitely while the add-on showed green.
|
|
|
|
`test_control_task.py` reproduces this on the pre-fix tree. The child process
|
|
reaches its 40 s deadline, has written no 0 W, and has logged nothing.
|
|
|
|
## 1. Decision: exit with status 70 after the ordinary wind-down
|
|
|
|
A done-callback on the control task (`control_task_done()`) handles the death.
|
|
It logs the traceback at CRITICAL at the moment of death, then sets `stop`.
|
|
`amain()` then runs the same wind-down that SIGTERM runs (`shutdown()` →
|
|
`command_site_zero()`, then cancel, then clean up). After that it raises
|
|
`ControlTaskDied`, and `main()` turns that into `SystemExit(70)` (sysexits
|
|
EX_SOFTWARE). A normal stop and `run_control()`'s own refusal path both set
|
|
`stop` before the task ends, so the callback ignores them. The refusal still
|
|
exits with 78.
|
|
|
|
**Why not restart the loop in the same process:**
|
|
|
|
* `cycle()` writes controller state before its awaits: `cycle_at`,
|
|
`peak_guard`, `allocation`, then `target`/`reason`, then the writes. An
|
|
exception between those steps leaves the controller half-updated. A loop
|
|
restarted in the same process would continue from a state that no boot ever
|
|
produces. A process restart rebuilds the controller from the Store, which is
|
|
the path every boot already runs.
|
|
* An unbounded retry is the invisible spin the card forbids. A bounded retry
|
|
would reimplement the Supervisor watchdog's own throttle: 10 restarts per
|
|
30 min, and 5 attempts with backoff (`apps/const.py`).
|
|
|
|
## 2. What happens after the exit, verified rather than assumed
|
|
|
|
* **Exit status reaches Docker.** The add-on image (s6-overlay, `init: false`,
|
|
`CMD /run.sh`) was run with `exec python3 -c 'raise SystemExit(70)'`.
|
|
`docker inspect` gave ExitCode 70. The same test with exit 0 gave 0, and an
|
|
uncaught exception gave 1.
|
|
* **Supervisor 2026.08.0** (`ghcr.io/home-assistant/amd64-hassio-supervisor`,
|
|
source read from the image):
|
|
* A non-zero container exit is recorded as `ContainerState.FAILED`
|
|
(`docker/monitor.py`). The Supervisor logs *"App … exited with non-zero
|
|
exit code 70"* at ERROR (`apps/app.py container_state_changed`).
|
|
* The Supervisor restarts the add-on **only if the per-install Watchdog
|
|
toggle is on**. `watchdog_container()` returns when `not self.watchdog`.
|
|
The toggle defaults to **off** (`apps/validate.py`:
|
|
`vol.Optional(ATTR_WATCHDOG, default=False)`).
|
|
* Add-on containers get **no Docker restart policy**. The only
|
|
`restart_policy=` in the Supervisor belongs to the observer.
|
|
* `boot: auto` in `config.yaml` only controls starting at host boot. The
|
|
`watchdog:` key in `config.yaml` is a health-check URL, not a restart
|
|
policy, and this add-on does not set it.
|
|
* **So on a default install the add-on stays stopped after this exit.** The UI
|
|
shows it as stopped, its log holds the traceback, and the Supervisor log holds
|
|
an ERROR line. The pack is at 0 W, which is the same physical state as before
|
|
this card. The difference is that 0 W is written immediately instead of after
|
|
the ~30 s veto, and the add-on no longer shows green. With the Watchdog toggle
|
|
on, the Supervisor restarts the add-on.
|
|
* **Not verified:** an actual restart by a running Supervisor. That would need a
|
|
live Supervisor, and the only one available is the live house.
|
|
|
|
## 3. Left open for the owner
|
|
|
|
* Whether `DOCS.md` should tell installers to enable Watchdog. DOCS.md is not
|
|
this card's file.
|
|
* Setting the Watchdog toggle from inside the add-on (`/addons/self/options`)
|
|
was rejected here, because it would change a user setting without the user
|
|
knowing.
|
|
|
|
## 4. Scope: the siblings, and why they are not supervised
|
|
|
|
* `run_tariff_refresh()` and the P1 HTTP and HA transports already catch
|
|
`Exception` inside their loops by design, and the values they feed are
|
|
age-bounded downstream.
|
|
* The P1 MQTT source *returns* on purpose when it is misconfigured. Making every
|
|
long-lived task's end exit the process would turn that deliberate fail-safe
|
|
into a restart loop.
|
|
* Only the control task is supervised.
|
|
|
|
## 5. `hass.py`: an undecodable body is an absence
|
|
|
|
A non-JSON `200` served as `text/html` was **already** treated as an absence,
|
|
because aiohttp's `ContentTypeError` is a `ClientError`. The reads that actually
|
|
escaped were these three:
|
|
|
|
* a malformed body labelled `application/json`, which raises `JSONDecodeError`
|
|
(a `ValueError`);
|
|
* a JSON body that is not an object, which raises `AttributeError` in
|
|
`number()` and `limits()`;
|
|
* an undecodable error body in `call()`, which raises `UnicodeDecodeError`.
|
|
|
|
All three were measured against aiohttp 3.13.5. The first two now read as
|
|
`None`, and the third is now a failed write (`False`).
|
|
|
|
## 6. Status row
|
|
|
|
`checks()` now starts with a *control loop* row showing how long ago the loop
|
|
last completed a pass. The row turns red once that age exceeds
|
|
`FIRMWARE_INGEST_VETO_S`. The heartbeat is sent from this loop, so past that age
|
|
layer 1 has zeroed the inverter.
|