Two ways a deploy of this add-on has gone wrong, and the Supervisor reports both as success.
The silent no-op. The Supervisor keys the built image by version:, so ha apps update against an unchanged version installs nothing and says it worked. PKG-06 recorded this on 2026-09-05 with four merges behind an unbumped version; PKG-07 recorded it again on 2026-09-06 with six. Both times a human noticed. Nothing else was going to.
The refused config. A config the shipped validator rejects is SystemExit(78) — the add-on does not start. ⚠️And the ingress panel goes with it, because the add-on serving it is not running, so recovery means the Supervisor UI. SAFETY-39's rule 10 made this reachable from a config that started fine on the previous version.
⚠️ Why this is NOT a CI check
"A pull request that touches runtime code must bump the version" is the obvious implementation and the wrong shape for this project. It deliberately batches many merges into one deploy, so a per-PR bump would put every open branch in conflict over one line of config.yaml — and would still not prevent the actual failure, which happens at deploy, not at merge. The check belongs where the mistake is made.
Exit codes
0 cleared · 1 REFUSED, do not deploy · 2 NOT CLEARED — a check did not run. ⚠️2 is not a pass.
⚠️ The test's first draft reported four silent skips as passes
It copied the gate into each throwaway fixture, so the script's sibling app/ was the stub the fixture built, the validator import failed, and four assertions written to expect a clearance passed on a skip instead. Fixed by invoking the gate where it ships and pointing --tree/--repo at the fixture — which is also the behaviour that matters: the gate must check a config against the validator it ships beside.
That is MUTATION.md habit 4 again — a check whose data cannot reach the failure it names — caught this time by the harness positive control rather than in review.
Every refusal in test_version_gate.py is driven: a repository is built in which the failure is really present and the check is required to say so. Plus a positive control that the no-op detector does not fire on a docs-only change, and a pin that omitting --options can never exit 0.
Demonstrated against the real situation
Box on 0.7.0, tree on 0.8.0, a config with slew_w: 1000 on a 5000 W rail → exit 1, REFUSED, surfacing the validator's own message (ceiling 556 W).
No version bump, and the gate agrees: tools/ and test_*.py are outside its runtime set, because demanding a bump for a typo fix trains people to bump without reading.
Two ways a deploy of this add-on has gone wrong, and **the Supervisor reports both as success**.
**The silent no-op.** The Supervisor keys the built image by `version:`, so `ha apps update` against an unchanged version installs nothing and says it worked. PKG-06 recorded this on 2026-09-05 with four merges behind an unbumped version; PKG-07 recorded it again on 2026-09-06 with six. Both times a human noticed. Nothing else was going to.
**The refused config.** A config the shipped validator rejects is `SystemExit(78)` — the add-on does not start. ⚠️ **And the ingress panel goes with it**, because the add-on serving it is not running, so recovery means the Supervisor UI. SAFETY-39's rule 10 made this reachable from a config that started fine on the previous version.
## ⚠️ Why this is NOT a CI check
"A pull request that touches runtime code must bump the version" is the obvious implementation and the wrong shape for this project. It deliberately batches many merges into one deploy, so a per-PR bump would put every open branch in conflict over one line of `config.yaml` — and would still not prevent the actual failure, which happens at **deploy**, not at merge. The check belongs where the mistake is made.
## Exit codes
`0` cleared · `1` REFUSED, do not deploy · `2` NOT CLEARED — a check did not run. ⚠️ **2 is not a pass.**
## ⚠️ The test's first draft reported four silent skips as passes
It copied the gate into each throwaway fixture, so the script's sibling `app/` was the stub the fixture built, the validator import failed, and four assertions written to expect a clearance passed on a skip instead. Fixed by invoking the gate **where it ships** and pointing `--tree`/`--repo` at the fixture — which is also the behaviour that matters: the gate must check a config against the validator it ships beside.
That is `MUTATION.md` habit 4 again — *a check whose data cannot reach the failure it names* — caught this time by the harness positive control rather than in review.
Every refusal in `test_version_gate.py` is **driven**: a repository is built in which the failure is really present and the check is required to say so. Plus a positive control that the no-op detector does *not* fire on a docs-only change, and a pin that omitting `--options` can never exit 0.
## Demonstrated against the real situation
Box on 0.7.0, tree on 0.8.0, a config with `slew_w: 1000` on a 5000 W rail → **exit 1, REFUSED**, surfacing the validator's own message (ceiling 556 W).
No version bump, and the gate agrees: `tools/` and `test_*.py` are outside its runtime set, because demanding a bump for a typo fix trains people to bump without reading.
Suite: **24 files, all pass**.
Card: PKG-09.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
Two ways a deploy of this add-on has gone wrong, and the Supervisor reports
both as success.
THE SILENT NO-OP. The Supervisor keys the built image by `version:`, so
`ha apps update` against an unchanged version installs nothing and says it
worked. PKG-06 recorded this on 2026-09-05 with six merges behind an unbumped
version; PKG-07 recorded it again on 2026-09-06 with six. Both times a human
noticed. Nothing else was going to.
THE REFUSED CONFIG. A config the shipped validator rejects is SystemExit(78) -
the add-on does not start. ⚠️ AND THE INGRESS PANEL GOES WITH IT, because the
add-on serving it is not running, so recovery means the Supervisor UI.
⚠️ IT CAUGHT A LIVE ONE ON ITS OWN PULL REQUEST. Run against release/1.0 while
this branch was in review: version 0.8.0 on both sides with UX-02's and
OPT-09's runtime commits already merged behind it. The class this exists to
catch had recurred, the same day, unnoticed - which is the argument for PKG-10,
a non-required post-merge check, since this script only helps someone who
remembers to run it.
⚠️ THE FIRST VERSION WAS BROKEN ON EVERY REAL REPOSITORY AND ITS TESTS COULD NOT
SEE IT. The "when did the version last change" lookup was `git log -S`, the
pickaxe, which finds commits that change the COUNT of a literal string. A bump
edits the line IN PLACE, so the count never moves and it returned only the
commit that first INTRODUCED the line. Measured here:
-S -> 0a1e61d 2026-08-23 Packaged as a Home Assistant add-on
-G -> 06661c5 2026-09-06 PKG-07: bump 0.7.0 -> 0.8.0
so a correctly bumped tree was reported as 87 COMMITS BEHIND AN UNBUMPED
VERSION, PKG-06's and PKG-07's own bump commits among the offenders. A tool
whose whole purpose is to be trusted cried wolf on every run.
⚠️ AND `-G` IS NOT THE FIX EITHER, ONLY A BETTER FILTER - it matches a reformat,
a quote change, a comment moved onto the line. `-G` narrows the candidates and
the VALUE is then compared against the parent's.
⚠️ THREE DISTINCT SHAPES OF HABIT 4 IN ONE FILE, each found by a different
means, which is the honest record of how this landed:
1. Fixtures built fresh repositories whose version line is introduced once and
never edited - the one shape in which `-S` looks correct. Found by review.
Answered with a three-state history; the old lookup goes red on 4 checks.
2. That three-state history still could not tell "value-compared" from
"filtered-only": review swapped in `-G`'s newest raw hit and THE WHOLE
SUITE STAYED GREEN, so the docstring's central claim was pinned by nothing.
Answered with a cosmetic touch of the version line - `-G` matches it, the
value comparison does not. That mutant now goes red on 3 checks.
3. The counts below were claimed fixed and two of them had not landed at all.
Two git shapes it still gets wrong are recorded in the docstring rather than
fixed - a bump resolved inside a merge commit (false refusal) and a bump
reverted while the work is kept (false clearance, the dangerous direction).
Neither has ever occurred here; both were constructed in review. Card PKG-11.
⚠️ AND THE PROSE COUNTS, WHICH I REPORTED AS FIXED WHEN THEY WERE NOT. Round 1
found "PKG-06 ... four merges" (its own commit message says six) and "recorded
eleven times" (MUTATION.md:550 says TWELFTH). I replaced them without asserting
the replacement matched, said "all applied", and round 2 found both still in the
tree - one fixed in DOCS.md but not in the script, one not fixed anywhere. A
claim that a correction landed is itself a claim that needs checking, and this
is PROC-01's tenth instance.
WHAT IT DOES NOT DO. It does not read the live box: the options blob needs the
Supervisor API. Without --options the validator half SKIPS - loudly, exit 2 -
and never pretends. On a bare host `app/validate.py` needs aiohttp, so that half
skips too and names the container to re-run in. A missing or malformed --options
file is a REFUSAL with a sentence, not a traceback.
EXIT CODES: 0 cleared, 1 REFUSED, 2 NOT CLEARED. ⚠️ 2 IS NOT A PASS.
Suite: 26 files, all pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Two ways a deploy of this add-on has gone wrong, and the Supervisor reports both as success.
The silent no-op. The Supervisor keys the built image by
version:, soha apps updateagainst an unchanged version installs nothing and says it worked. PKG-06 recorded this on 2026-09-05 with four merges behind an unbumped version; PKG-07 recorded it again on 2026-09-06 with six. Both times a human noticed. Nothing else was going to.The refused config. A config the shipped validator rejects is
SystemExit(78)— the add-on does not start. ⚠️ And the ingress panel goes with it, because the add-on serving it is not running, so recovery means the Supervisor UI. SAFETY-39's rule 10 made this reachable from a config that started fine on the previous version.⚠️ Why this is NOT a CI check
"A pull request that touches runtime code must bump the version" is the obvious implementation and the wrong shape for this project. It deliberately batches many merges into one deploy, so a per-PR bump would put every open branch in conflict over one line of
config.yaml— and would still not prevent the actual failure, which happens at deploy, not at merge. The check belongs where the mistake is made.Exit codes
0cleared ·1REFUSED, do not deploy ·2NOT CLEARED — a check did not run. ⚠️ 2 is not a pass.⚠️ The test's first draft reported four silent skips as passes
It copied the gate into each throwaway fixture, so the script's sibling
app/was the stub the fixture built, the validator import failed, and four assertions written to expect a clearance passed on a skip instead. Fixed by invoking the gate where it ships and pointing--tree/--repoat the fixture — which is also the behaviour that matters: the gate must check a config against the validator it ships beside.That is
MUTATION.mdhabit 4 again — a check whose data cannot reach the failure it names — caught this time by the harness positive control rather than in review.Every refusal in
test_version_gate.pyis driven: a repository is built in which the failure is really present and the check is required to say so. Plus a positive control that the no-op detector does not fire on a docs-only change, and a pin that omitting--optionscan never exit 0.Demonstrated against the real situation
Box on 0.7.0, tree on 0.8.0, a config with
slew_w: 1000on a 5000 W rail → exit 1, REFUSED, surfacing the validator's own message (ceiling 556 W).No version bump, and the gate agrees:
tools/andtest_*.pyare outside its runtime set, because demanding a bump for a typo fix trains people to bump without reading.Suite: 24 files, all pass.
Card: PKG-09.
🤖 Generated with Claude Code
https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
09b84e30b0to6c348974a36c348974a3to04b3330def