SAFETY-62/SAFETY-60: operator tools refuse input they cannot read #120

Merged
admin merged 2 commits from SAFETY-62-refuse-unread-input into release/1.0 2026-09-25 00:25:23 +02:00
Owner

SAFETY-62 + SAFETY-60. These operator tools used to print a pass, or exit 0, on input they could not read. Now they refuse it, exit non-zero and name what is wrong with the shape.

What changed

  • predeploy_check.py (SAFETY-60): --options must be a JSON object, and at least half of its top-level keys must be option names from the shipped config.yaml. The ha apps info --raw-json wrapper is refused, and the message tells the operator to pass .data.options. Also removed a false "--options was not given" skip message that printed when the file was given but unreadable.
  • rehearse_roster.py (SAFETY-62): --state is parsed and shape-checked before Store.load() sees it. Store keys are DEFAULTS, the schema markers, the maintenance keys and unit/.... An empty or comment-only store is still accepted, because that is what a fresh install looks like. --before/--after get the same option-key check. A clean --no-live-entities run now exits 3, not 0.
  • alloc_sim.py: exits 1 when split-invariance FAILS. Before, it always exited 0.
  • decode_bus_capture.py: takes several paths, as the docstring's glob does. Exits 1 on a file where frames are not the majority of lines, or where there are no frames.
  • DOCS.md: one sentence at the rehearsal owner steps and one at the predeploy export: pass the bare object (jq .data.options). The rehearsal paragraph also says a clean degraded run exits 3.
  • test_tool_inputs.py (new): drives each tool on good input, then on the shapes that used to pass. It builds a realistic ha apps info capture, including per-language translations (the shape that produced the four bogus "actuators"). No live config is committed.
  • test_rehearse.py: mutant_tree also copies predeploy_check.py + config.yaml, because the gate is imported from there. The exit-code tuple and the degraded-run expectation are updated to 3.

False-pass cases, before -> after (base a45ed06 vs this branch)

case before after
predeploy, wrapped capture CLEARED, exit 0 REFUSED, exit 1 (0 of 2 top-level key(s) recognised ... pass its .data.options)
predeploy, wrapped with slew_w 5000 CLEARED, exit 0 REFUSED, exit 1 (shape, not a validator rule)
predeploy, list-wrapped exit 1 (wrong reason) exit 1, is a JSON list
rehearse, wrapped state NO FINDINGS, exit 0 exit 2
rehearse, state truncated by 20 bytes NO FINDINGS, exit 0 exit 2
rehearse, list-wrapped state NO FINDINGS, exit 0 exit 2
rehearse, --after wrapped NO FINDINGS, exit 0 exit 2
rehearse, --before wrapped exit 1 (spurious findings) exit 2
rehearse, clean --no-live-entities exit 0 exit 3
alloc_sim, split-invariance FAILS exit 0 exit 1
decode, junk / empty file exit 0 exit 1

Good input is unchanged. A bare options dict is CLEARED with exit 0. The rehearse control (legacy state, flat -> roster.yaml) still gives 2 FINDINGS and exit 1. All 8 fixture state x roster combinations give the same exit codes and findings counts as on the base. The committed capture still exits 0.

Mutations (purged tree, :ro, goodwe-addon-checks:1, --network none): all 10 go RED

M1 shape_problem always None · M2 majority threshold dropped · M3 predeploy ignores the gate result · M4 load_state not called · M5 load_options gate off · M6 degraded exits 0 (red in test_tool_inputs and test_rehearse) · M7 alloc_sim returns 0 · M8 decode never refuses · M9 decode reads only the first path · M10 over-refusal of an empty store (red in both test files via the fresh-state control).

Suite

checks.yml's exact command, goodwe-addon-checks:1, :ro, from a clean archive: 32/32 PASS.

Not done, and why

app/validate.py's actuator_writers() still walks nested structures, so the validator alone, handed a wrapped capture, finds 5 "actuators" and no problems. That is outside this card's file scope (app/). The tool-side gate means the tool never hands it that shape again, and at runtime the add-on reads the bare /data/options.json.

🤖 Generated with Claude Code

https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU

SAFETY-62 + SAFETY-60. These operator tools used to print a pass, or exit 0, on input they could not read. Now they refuse it, exit non-zero and name what is wrong with the shape. ## What changed - **predeploy_check.py** (SAFETY-60): `--options` must be a JSON object, and at least half of its top-level keys must be option names from the shipped `config.yaml`. The `ha apps info --raw-json` wrapper is refused, and the message tells the operator to pass `.data.options`. Also removed a false "--options was not given" skip message that printed when the file was given but unreadable. - **rehearse_roster.py** (SAFETY-62): `--state` is parsed and shape-checked before `Store.load()` sees it. Store keys are `DEFAULTS`, the schema markers, the maintenance keys and `unit/...`. An empty or comment-only store is still accepted, because that is what a fresh install looks like. `--before`/`--after` get the same option-key check. A clean `--no-live-entities` run now exits **3**, not 0. - **alloc_sim.py**: exits 1 when split-invariance FAILS. Before, it always exited 0. - **decode_bus_capture.py**: takes several paths, as the docstring's glob does. Exits 1 on a file where frames are not the majority of lines, or where there are no frames. - **DOCS.md**: one sentence at the rehearsal owner steps and one at the predeploy export: pass the bare object (`jq .data.options`). The rehearsal paragraph also says a clean degraded run exits 3. - **test_tool_inputs.py** (new): drives each tool on good input, then on the shapes that used to pass. It builds a realistic `ha apps info` capture, including per-language `translations` (the shape that produced the four bogus "actuators"). No live config is committed. - **test_rehearse.py**: `mutant_tree` also copies `predeploy_check.py` + `config.yaml`, because the gate is imported from there. The exit-code tuple and the degraded-run expectation are updated to 3. ## False-pass cases, before -> after (base a45ed06 vs this branch) | case | before | after | |---|---|---| | predeploy, wrapped capture | CLEARED, exit 0 | REFUSED, exit 1 (`0 of 2 top-level key(s) recognised ... pass its .data.options`) | | predeploy, wrapped with slew_w 5000 | CLEARED, exit 0 | REFUSED, exit 1 (shape, not a validator rule) | | predeploy, list-wrapped | exit 1 (wrong reason) | exit 1, `is a JSON list` | | rehearse, wrapped state | NO FINDINGS, exit 0 | exit 2 | | rehearse, state truncated by 20 bytes | NO FINDINGS, exit 0 | exit 2 | | rehearse, list-wrapped state | NO FINDINGS, exit 0 | exit 2 | | rehearse, --after wrapped | NO FINDINGS, exit 0 | exit 2 | | rehearse, --before wrapped | exit 1 (spurious findings) | exit 2 | | rehearse, clean --no-live-entities | exit 0 | exit 3 | | alloc_sim, split-invariance FAILS | exit 0 | exit 1 | | decode, junk / empty file | exit 0 | exit 1 | Good input is unchanged. A bare options dict is CLEARED with exit 0. The rehearse control (legacy state, flat -> roster.yaml) still gives 2 FINDINGS and exit 1. All 8 fixture state x roster combinations give the same exit codes and findings counts as on the base. The committed capture still exits 0. ## Mutations (purged tree, `:ro`, goodwe-addon-checks:1, --network none): all 10 go RED M1 shape_problem always None · M2 majority threshold dropped · M3 predeploy ignores the gate result · M4 load_state not called · M5 load_options gate off · M6 degraded exits 0 (red in test_tool_inputs **and** test_rehearse) · M7 alloc_sim returns 0 · M8 decode never refuses · M9 decode reads only the first path · M10 over-refusal of an empty store (red in both test files via the fresh-state control). ## Suite checks.yml's exact command, `goodwe-addon-checks:1`, `:ro`, from a clean archive: 32/32 PASS. ## Not done, and why `app/validate.py`'s `actuator_writers()` still walks nested structures, so the validator alone, handed a wrapped capture, finds 5 "actuators" and no problems. That is outside this card's file scope (`app/`). The tool-side gate means the tool never hands it that shape again, and at runtime the add-on reads the bare `/data/options.json`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
admin added 1 commit 2026-09-24 23:50:13 +02:00
SAFETY-60/62: operator tools refuse input they cannot read
checks / unit (pull_request) Successful in 8m31s
f4f72211a9
predeploy_check.py and rehearse_roster.py now assert a positive property of
their input (a JSON object, most of whose top-level keys are known option or
store keys) and refuse anything else with the shape named. The `ha apps info
--raw-json` wrapper, a truncated or list-wrapped state file and a wrapped
config no longer produce "CLEARED" / "NO FINDINGS" with exit 0.

Also: a clean --no-live-entities rehearsal exits 3, not 0; alloc_sim.py
exits 1 when split-invariance FAILS; decode_bus_capture.py takes several
paths and exits 1 on a file that is mostly not frames.

Graded by the new test_tool_inputs.py; DOCS.md says to pass .data.options.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
Author
Owner

[BLOCKING] R1: DOCS.md still lists three exit codes for rehearse_roster.

goodwe_controller/DOCS.md:2823 (the command block for tools/rehearse_roster.py) still reads: "Exit 0 clean · 1 at least one finding · 2 could not run." That is the summary line an operator reads. The new 3 only appears in the paragraph about 35 lines further down. Please add 3 to that line, e.g. "· 3 clean but degraded (--no-live-entities). ⚠️ Neither 2 nor 3 is a pass." I found no other consumer of the exit code: no CI step, no script, no RELEASE doc. So this line is the only thing still out of date.

**[BLOCKING] R1: DOCS.md still lists three exit codes for rehearse_roster.** `goodwe_controller/DOCS.md:2823` (the command block for `tools/rehearse_roster.py`) still reads: "Exit `0` clean · `1` at least one finding · `2` could not run." That is the summary line an operator reads. The new `3` only appears in the paragraph about 35 lines further down. Please add `3` to that line, e.g. "· `3` clean but degraded (`--no-live-entities`). ⚠️ Neither `2` nor `3` is a pass." I found no other consumer of the exit code: no CI step, no script, no RELEASE doc. So this line is the only thing still out of date.
Author
Owner

[non-blocking] R2: auto is a real Store key but is missing from _STORE_KEYS.

app/main.py:5312 does store.set("auto", ...), but rehearse_roster._STORE_KEYS doesn't list it. I measured this. A real file always passes, because Store.save() writes {**DEFAULTS, **loaded}, which is 11 known keys against 1 unknown. Every fixture passes too. The gap only shows up in a hand-trimmed state: {"auto": true} is REFUSED with exit 2 ("0 of 1 top-level key(s) recognised ['auto']"). One word in the set fixes it. Or leave it and say why.

**[non-blocking] R2: `auto` is a real Store key but is missing from `_STORE_KEYS`.** `app/main.py:5312` does `store.set("auto", ...)`, but `rehearse_roster._STORE_KEYS` doesn't list it. I measured this. A real file always passes, because `Store.save()` writes `{**DEFAULTS, **loaded}`, which is 11 known keys against 1 unknown. Every fixture passes too. The gap only shows up in a hand-trimmed state: `{"auto": true}` is REFUSED with exit 2 ("0 of 1 top-level key(s) recognised ['auto']"). One word in the set fixes it. Or leave it and say why.
Author
Owner

Review notes (no action needed): what I measured.

  • Merge: clean onto the current base fa3d237 (DOC-1 is included). CI run 833 is this PR's own run: PRNUM 120, base release/1.0 at fa3d237, head f4f7221, GRADED: refs/pull/120/head merged into release/1.0, run_attempt 1, 33/33 PASS, test_tool_inputs.py included.
  • Good input is still CLEARED (exit 0) by predeploy_check.py in goodwe-addon-checks:1 with --network none. I tried: the real .data.options of two live captures (76 keys and 70 keys, 100% recognised); the fresh-install defaults from config.yaml (with setpoint_entity set); a 3-key sparse dict with pv_entities and a _comment; and a 2-unit roster with a pv_entities list. The two wrapped captures are REFUSED with the .data.options hint.
  • Older options versus the threshold: across all 55 historical config.yaml revisions, no options key has ever been removed. Every old default set is 100% recognised by the current key set, so "at least half" has plenty of margin.
  • Mutations (source mounted :ro, --network none), all red: M1 gate threshold -> True (wrapped CLEARED); M2 load_state() call removed; M3 degraded -> EXIT_OK (red in both suites); M4 alloc_sim return 0; M5 decode unreadable always None; M6 threshold tightened to "all keys known" (red, because the fixtures carry auto, so the tolerance is pinned).
  • Length: proportionate. The comments are one sentence plus a ticket pointer. Nothing to trim.
**Review notes (no action needed): what I measured.** - **Merge:** clean onto the current base fa3d237 (DOC-1 is included). CI run **833** is this PR's own run: PRNUM 120, base release/1.0 at fa3d237, head f4f7221, `GRADED: refs/pull/120/head merged into release/1.0`, run_attempt 1, 33/33 PASS, `test_tool_inputs.py` included. - **Good input is still CLEARED (exit 0)** by `predeploy_check.py` in `goodwe-addon-checks:1` with `--network none`. I tried: the real `.data.options` of two live captures (76 keys and 70 keys, 100% recognised); the fresh-install defaults from config.yaml (with setpoint_entity set); a 3-key sparse dict with `pv_entities` and a `_comment`; and a 2-unit roster with a `pv_entities` list. The two wrapped captures are REFUSED with the `.data.options` hint. - **Older options versus the threshold:** across all 55 historical `config.yaml` revisions, no options key has ever been removed. Every old default set is 100% recognised by the current key set, so "at least half" has plenty of margin. - **Mutations** (source mounted `:ro`, `--network none`), all red: M1 gate threshold -> `True` (wrapped CLEARED); M2 `load_state()` call removed; M3 degraded -> `EXIT_OK` (red in both suites); M4 alloc_sim `return 0`; M5 decode `unreadable` always None; M6 threshold tightened to "all keys known" (red, because the fixtures carry `auto`, so the tolerance is pinned). - **Length:** proportionate. The comments are one sentence plus a ticket pointer. Nothing to trim.
Author
Owner

#issuecomment-1606

R1 fixed in b06f1fa: DOCS.md's exit-code line under the rehearse_roster command block now lists 3 clean but degraded (--no-live-entities), and says 2 and 3 are not a pass.

> #issuecomment-1606 R1 fixed in b06f1fa: DOCS.md's exit-code line under the rehearse_roster command block now lists `3` clean but degraded (`--no-live-entities`), and says `2` and `3` are not a pass.
admin added 1 commit 2026-09-25 00:12:00 +02:00
Author
Owner

#issuecomment-1607

R2 accepted and fixed in b06f1fa: auto (written at app/main.py:5312) is now in rehearse_roster._STORE_KEYS. New test case in test_tool_inputs.py: a state holding only {"auto": true} is recognised as a store. Mutation (drop auto from the set) turns that case RED. Suite 32/32 green in goodwe-addon-checks:1, :ro.

> #issuecomment-1607 R2 accepted and fixed in b06f1fa: `auto` (written at app/main.py:5312) is now in `rehearse_roster._STORE_KEYS`. New test case in test_tool_inputs.py: a state holding only `{"auto": true}` is recognised as a store. Mutation (drop `auto` from the set) turns that case RED. Suite 32/32 green in goodwe-addon-checks:1, `:ro`.
Author
Owner

Re-review of b06f1fa: approved.

  • R1 (1612): the DOCS exit-code line now includes 3, and says neither 2 nor 3 is a pass. Resolved.
  • R2 (1614): auto is now in _STORE_KEYS. The new check turns red when I revert the fix (mutation run in goodwe-addon-checks:1, --network none, source mounted :ro). Resolved.
  • CI-10: run 836, job 1961. PRNUM 120, base release/1.0 at 6e65272 (the current base, SAFETY-61 included), head b06f1fa, GRADED: refs/pull/120/head merged into release/1.0, run_attempt 1, 34/34 PASS.

Merging with head_commit_id pinned to b06f1fa.

**Re-review of b06f1fa: approved.** - R1 (1612): the DOCS exit-code line now includes `3`, and says neither `2` nor `3` is a pass. Resolved. - R2 (1614): `auto` is now in `_STORE_KEYS`. The new check turns red when I revert the fix (mutation run in `goodwe-addon-checks:1`, `--network none`, source mounted `:ro`). Resolved. - CI-10: run **836**, job 1961. PRNUM 120, base release/1.0 at 6e65272 (the current base, SAFETY-61 included), head b06f1fa, `GRADED: refs/pull/120/head merged into release/1.0`, run_attempt 1, 34/34 PASS. Merging with head_commit_id pinned to b06f1fa.
admin merged commit 3e38bafcee into release/1.0 2026-09-25 00:25:23 +02:00
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: admin/goodwe-addon#120