checks / unit (pull_request) Successful in 6m14s
Two ways a deploy of this add-on has gone wrong, and the Supervisor reports both as success. THE SILENT NO-OP. The Supervisor keys the built image by `version:`, so `ha apps update` against an unchanged version installs nothing and says it worked. PKG-06 recorded this on 2026-09-05 with six merges behind an unbumped version. PKG-07 recorded it again on 2026-09-06 with six. Both times a human noticed. Nothing else was going to. THE REFUSED CONFIG. A config the shipped validator rejects is SystemExit(78) - the add-on does not start. ⚠️ AND THE INGRESS PANEL GOES WITH IT, because the add-on serving it is not running, so recovery means the Supervisor UI. ⚠️ THE FIRST VERSION OF THIS TOOL WAS BROKEN ON EVERY REAL REPOSITORY, AND ITS OWN TESTS COULD NOT SEE IT. The "when did the version last change" lookup was `git log -S 'version: "'`. `-S` is the pickaxe: it finds commits that change the COUNT of a literal string. A bump edits the line IN PLACE - `version: "0.7.0"` -> `version: "0.8.0"` - so the substring still occurs exactly once before and after, and `-S` never sees it. It returned only the commit that first INTRODUCED the line. MEASURED on admin/goodwe-addon, in its current correctly-bumped state: -S ->0a1e61d2026-08-23 Packaged as a Home Assistant add-on -G ->06661c52026-09-06 PKG-07: bump 0.7.0 -> 0.8.0 so the shipped script reported a correctly bumped tree as 87 COMMITS BEHIND AN UNBUMPED VERSION, listing PKG-06's and PKG-07's own bump commits among the offenders. A tool whose entire purpose is to be trusted before a deploy cried wolf on every run. ⚠️ AND `-G` IS NOT THE FIX EITHER, ONLY A BETTER FILTER. `-G'^version: "'` matches any commit whose diff touches a line of that shape - a reformat, a quote change, a comment moved onto the line. So `-G` narrows the candidates and then the VALUE is compared against the parent's. The question is "did the version change", and only comparing the value answers it. ⚠️ WHY THE TESTS MISSED IT, WHICH IS THE FINDING BEHIND THE FINDING. Every fixture built a fresh repository whose `version:` line is introduced once and never edited - the one shape in which `-S` looks correct, because introducing the line does change the count from 0 to 1. A test that builds its own one-bump history is a test asserting against a structure it built itself: MUTATION.md habit 4, in the file written to prevent a recurrence, and the thirteenth recorded instance. There is now a fixture with THREE version states and runtime commits between them, and it goes red on four checks against the old `-S` lookup. ⚠️ WHY THIS IS NOT A CI CHECK. "A pull request that touches runtime code must bump the version" would put every open branch in conflict over one line of config.yaml and still not prevent the failure, which happens at DEPLOY and not at merge. A cheaper form review proposed - a NON-REQUIRED check on push to release/1.0 that raises a visible alarm when commits accumulate behind an unbumped version - is a good idea and is filed as PKG-10 rather than built here. WHAT IT DOES NOT DO. It does not read the live box: the options blob needs the Supervisor API and a supervisor token. Without --options the validator half SKIPS - loudly, exit 2 - and never pretends. `app/validate.py` imports its siblings and those need aiohttp, so on a bare host that half skips too and says which container to re-run it in. A missing or malformed --options file is now a REFUSAL with a sentence rather than a traceback. EXIT CODES: 0 cleared, 1 REFUSED, 2 NOT CLEARED. ⚠️ 2 IS NOT A PASS. Also corrected: this file said PKG-06 was "four merges" when its own commit message says six, and the test docstring said habit 4 had eleven instances when MUTATION.md already said TWELFTH. Both were caught in review. `requirements.txt` stays in RUNTIME although the Dockerfile installs via apk - over-inclusive is the safe direction and costs only an unnecessary bump. Suite: 25 files, all pass (25 with UX-02's test_selfconfig.py now in the base). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
214 lines
9.2 KiB
Python
214 lines
9.2 KiB
Python
"""PKG-09 - grade the check that refuses a no-op deploy. `python3 test_version_gate.py`
|
|
|
|
⚠️ THE POINT OF THIS FILE IS THE REFUSALS, NOT THE CLEARANCES. A pre-deploy
|
|
check that only ever says "CLEARED" is the defect `MUTATION.md` habit 4 names -
|
|
"a check whose DATA cannot reach the failure it names" - which this repository
|
|
has recorded eleven times. So every refusal below is DRIVEN: a repository is
|
|
built in which the failure is really present, and the check is required to say
|
|
so. A clearance is only meaningful next to a refusal from the same harness.
|
|
|
|
The third property is the one that is easy to lose: a SKIPPED check must not
|
|
look like a PASSED one. Omitting `--options` cannot exit 0.
|
|
"""
|
|
import json
|
|
import os
|
|
import shutil
|
|
import subprocess
|
|
import sys
|
|
import tempfile
|
|
|
|
HERE = os.path.dirname(os.path.abspath(__file__))
|
|
GATE = os.path.join(HERE, "tools", "predeploy_check.py")
|
|
sys.path.insert(0, os.path.join(HERE, "tools"))
|
|
|
|
fails = []
|
|
tmpdirs = []
|
|
|
|
|
|
def check(name, ok):
|
|
print(f" {'ok ' if ok else 'FAIL'} {name}")
|
|
if not ok:
|
|
fails.append(name)
|
|
|
|
|
|
def git(repo, *args):
|
|
return subprocess.run(
|
|
["git", "-C", repo, "-c", "user.email=t@t", "-c", "user.name=t", *args],
|
|
capture_output=True, text=True)
|
|
|
|
|
|
def make_repo(version="0.7.0"):
|
|
"""A repo shaped like this one: <root>/goodwe_controller/{config.yaml,app,tools}."""
|
|
root = tempfile.mkdtemp(prefix="test_version_gate.")
|
|
tmpdirs.append(root)
|
|
addon = os.path.join(root, "goodwe_controller")
|
|
os.makedirs(os.path.join(addon, "app"))
|
|
write(addon, "config.yaml", f'name: t\nversion: "{version}"\n')
|
|
write(addon, "app/control.py", "X = 1\n")
|
|
write(addon, "CHANGELOG.md", "# Changelog\n")
|
|
git(root, "init", "-q", ".")
|
|
git(root, "add", "-A")
|
|
git(root, "commit", "-q", "-m", f"initial at {version}")
|
|
return root, addon
|
|
|
|
|
|
def write(addon, rel, text):
|
|
p = os.path.join(addon, rel)
|
|
with open(p, "w", encoding="utf-8") as fh:
|
|
fh.write(text)
|
|
|
|
|
|
def run_gate(addon, root, deployed, options=None):
|
|
"""Run the REAL gate against a throwaway tree.
|
|
|
|
⚠️ The script is invoked where it SHIPS, not copied into the fixture. It
|
|
imports the validator from its own sibling `app/`, which is the behaviour
|
|
that matters: the gate must check a config against the validator it ships
|
|
beside, not against whatever happens to sit next to a copy. Copying it into
|
|
the fixture made every validator assertion silently SKIP - the first draft
|
|
of this file did exactly that and reported four green skips as passes.
|
|
"""
|
|
cmd = [sys.executable, GATE,
|
|
"--deployed-version", deployed, "--tree", addon, "--repo", root]
|
|
if options:
|
|
cmd += ["--options", options]
|
|
return subprocess.run(cmd, capture_output=True, text=True)
|
|
|
|
|
|
VALID = {"max_w": 2000, "slew_w": 0, "target_grid_w": -10, "gain": 0.6,
|
|
"inverters": [], "setpoint_entity": "number.goodwe_setpoint_w"}
|
|
|
|
|
|
def opts_file(d):
|
|
fd, p = tempfile.mkstemp(suffix=".json")
|
|
with os.fdopen(fd, "w") as fh:
|
|
json.dump(d, fh)
|
|
return p
|
|
|
|
|
|
print("classification - what forces a rebuild and what does not")
|
|
from predeploy_check import is_runtime, shipped_version # noqa: E402
|
|
|
|
for p in ("goodwe_controller/app/control.py", "goodwe_controller/config.yaml",
|
|
"goodwe_controller/translations/en.yaml", "goodwe_controller/run.sh"):
|
|
check(f"runtime: {p}", is_runtime(p))
|
|
for p in ("goodwe_controller/CHANGELOG.md", "goodwe_controller/DOCS.md",
|
|
"goodwe_controller/test_control.py", "goodwe_controller/tools/x.py",
|
|
"goodwe_controller/DECISIONS/a.md", "README.md"):
|
|
check(f"NOT runtime: {p}", not is_runtime(p))
|
|
|
|
print("\nTHE REFUSAL - a runtime change behind an unchanged version")
|
|
root, addon = make_repo("0.7.0")
|
|
write(addon, "app/control.py", "X = 2\n")
|
|
git(root, "commit", "-qam", "SAFETY-99: change behaviour, bump nothing")
|
|
r = run_gate(addon, root, "0.7.0", opts_file(VALID))
|
|
check("refuses (exit 1)", r.returncode == 1)
|
|
check("...and says the deploy would be a NO-OP", "NO-OP" in r.stdout)
|
|
check("...and names the offending commit", "SAFETY-99" in r.stdout)
|
|
check("...and does NOT say cleared", "CLEARED for deploy" not in r.stdout)
|
|
|
|
print("\nPOSITIVE CONTROL - the same check must NOT fire on a docs-only change")
|
|
root2, addon2 = make_repo("0.7.0")
|
|
write(addon2, "CHANGELOG.md", "# Changelog\n\n## notes\n")
|
|
git(root2, "commit", "-qam", "PKG-99: documentation only")
|
|
r2 = run_gate(addon2, root2, "0.7.0", opts_file(VALID))
|
|
check("does not refuse a docs-only tree", r2.returncode == 0)
|
|
check("...and says there is nothing to deploy",
|
|
"nothing to deploy" in r2.stdout)
|
|
|
|
print("\nTHE CLEARANCE - a bumped version with a valid config")
|
|
root3, addon3 = make_repo("0.7.0")
|
|
write(addon3, "app/control.py", "X = 3\n")
|
|
write(addon3, "config.yaml", 'name: t\nversion: "0.8.0"\n')
|
|
git(root3, "commit", "-qam", "PKG-99: change and bump together")
|
|
r3 = run_gate(addon3, root3, "0.7.0", opts_file(VALID))
|
|
check("clears (exit 0)", r3.returncode == 0)
|
|
check("...and says so", "CLEARED for deploy" in r3.stdout)
|
|
|
|
print("\n⭐ A SKIPPED CHECK MUST NOT LOOK LIKE A PASSED ONE")
|
|
r4 = run_gate(addon3, root3, "0.7.0") # no --options
|
|
check("omitting --options cannot exit 0", r4.returncode != 0)
|
|
check("...says SKIPPED", "SKIPPED" in r4.stdout)
|
|
check("...says the config was NEVER CHECKED", "NEVER CHECKED" in r4.stdout)
|
|
check("...and does NOT claim clearance", "CLEARED for deploy" not in r4.stdout)
|
|
|
|
print("\nTHE OTHER REFUSAL - a config this version's validator rejects")
|
|
bad = dict(VALID, slew_w=1000) # rule 10: > 2/18 x 2000 = 222 W
|
|
r5 = run_gate(addon3, root3, "0.7.0", opts_file(bad))
|
|
check("refuses a config the validator rejects", r5.returncode == 1)
|
|
check("...and warns it is exit 78", "78" in r5.stdout)
|
|
check("...and warns the ingress panel goes too", "ingress panel" in r5.stdout)
|
|
check("...and prints the validator's own words", "slew_w is 1000 W" in r5.stdout)
|
|
|
|
print()
|
|
print("⭐ HISTORY WITH MORE THAN ONE BUMP - THE FIXTURE THE FIRST DRAFT LACKED")
|
|
# ⚠️ EVERY FIXTURE ABOVE BUILDS A FRESH REPOSITORY WHOSE `version:` LINE IS
|
|
# INTRODUCED ONCE AND NEVER EDITED. That is precisely the shape in which the
|
|
# original `git log -S 'version: "'` looked correct - the pickaxe finds commits
|
|
# that change a string's COUNT, and introducing the line changes it from 0 to 1.
|
|
# A real repository edits the line IN PLACE, the count never moves again, and
|
|
# the lookup silently returns the commit that first added it.
|
|
#
|
|
# MEASURED on admin/goodwe-addon before the fix: `-S` named `0a1e61d`
|
|
# (2026-08-23, the original packaging commit) as "the last bump", so a correctly
|
|
# bumped tree was reported as 87 commits behind an unbumped version - with
|
|
# PKG-06's and PKG-07's own bump commits listed among the offenders. The tool
|
|
# cried wolf on every run, and none of the fixtures above could see it, because
|
|
# a test that builds its own one-bump history is a test asserting against a
|
|
# structure it built itself. MUTATION.md habit 4, in the file written to
|
|
# prevent a recurrence.
|
|
from predeploy_check import last_version_change # noqa: E402
|
|
|
|
root4 = tempfile.mkdtemp(prefix="test_version_gate.hist.")
|
|
tmpdirs.append(root4)
|
|
addon4 = os.path.join(root4, "goodwe_controller")
|
|
os.makedirs(os.path.join(addon4, "app"))
|
|
write(addon4, "config.yaml", 'name: t\nversion: "0.5.0"\n')
|
|
write(addon4, "app/control.py", "X = 1\n")
|
|
git(root4, "init", "-q", ".")
|
|
git(root4, "add", "-A")
|
|
git(root4, "commit", "-q", "-m", "initial at 0.5.0")
|
|
_first = git(root4, "rev-parse", "HEAD").stdout.strip()
|
|
|
|
write(addon4, "app/control.py", "X = 2\n")
|
|
git(root4, "commit", "-qam", "runtime change between bumps")
|
|
write(addon4, "config.yaml", 'name: t\nversion: "0.6.0"\n')
|
|
git(root4, "commit", "-qam", "bump to 0.6.0")
|
|
write(addon4, "app/control.py", "X = 3\n")
|
|
git(root4, "commit", "-qam", "another runtime change")
|
|
write(addon4, "config.yaml", 'name: t\nversion: "0.7.0"\n')
|
|
git(root4, "commit", "-qam", "bump to 0.7.0")
|
|
_last_bump = git(root4, "rev-parse", "HEAD").stdout.strip()
|
|
|
|
check("⭐ THE LAST BUMP IS FOUND, NOT THE FIRST. An in-place edit of the "
|
|
"version line does not change how many times the string occurs, so a "
|
|
"pickaxe search returns the commit that introduced it and every bump "
|
|
"after that is invisible",
|
|
last_version_change(root4) == _last_bump)
|
|
check("...and it is emphatically not the commit that introduced the line",
|
|
last_version_change(root4) != _first)
|
|
|
|
r6 = run_gate(addon4, root4, "0.7.0", opts_file(VALID))
|
|
check("a tree bumped with nothing after it clears, even with bumps in history",
|
|
r6.returncode == 0 and "CLEARED for deploy" in r6.stdout)
|
|
|
|
write(addon4, "app/control.py", "X = 4\n")
|
|
git(root4, "commit", "-qam", "SAFETY-98: runtime change after the last bump")
|
|
r7 = run_gate(addon4, root4, "0.7.0", opts_file(VALID))
|
|
check("and one runtime commit after that last bump is refused", r7.returncode == 1)
|
|
check("...naming only the commit since the LAST bump, not since the first",
|
|
"SAFETY-98" in r7.stdout and "runtime change between bumps" not in r7.stdout)
|
|
|
|
|
|
print("\nthe shipped tree's own version is readable")
|
|
check("shipped_version() finds a version", bool(shipped_version(HERE)))
|
|
|
|
for d in tmpdirs:
|
|
shutil.rmtree(d, ignore_errors=True)
|
|
|
|
print()
|
|
if fails:
|
|
print(f"{len(fails)} FAILED: " + "; ".join(fails))
|
|
sys.exit(1)
|
|
print("all checks passed")
|