Files
goodwe-addon/goodwe_controller/tools/supervisor_shape_probe.py
glenn schrooyenandClaude Opus 5 22c016fadd
checks / unit (pull_request) Successful in 4m33s
CFG-04 fix round: the probe now measures the key it is cited for
Review blocked this on ONE finding, and it was in the EVIDENCE, not the code.
The design and the shipped behaviour are unchanged.

THE INSTRUMENT DID NOT MEASURE WHAT FOUR DOCUMENTS SAID IT MEASURED

tools/supervisor_shape_probe.py section B validated the RECONSTRUCTED LIVE
options blob - which is missing many required keys that have nothing to do with
this card. Its refusal therefore named whichever of THOSE the validator reached
first: the committed transcript recorded `meter_freeze_s`, the reviewer's run
printed `p1_phase_export_entities`, and a run today printed `sign_disagree_w`.
The string those documents quoted - `Missing option 'inverters' in root` - was
never produced by the instrument.

And section B asserted a bare `not ok`. Any refusal at all satisfied it, so
deleting `inverters: []` from config.yaml left it printing PASS: the probe could
not detect the one regression it is the committed evidence for.

THE UNDERLYING CLAIM IS TRUE AND IS NOW MEASURED PROPERLY

Section B is an ISOLATED COUNTERFACTUAL now: the full shipped `options:` block
(expect accept), and the SAME block with `inverters` removed and nothing else
varying (expect refusal). One key differs, so the refusal has nowhere else to
come from - and the assertion is on the KEY NAME IN THE REFUSAL STRING, so the
line goes red if the refusal is ever about a different key. The probe also exits
non-zero when any case fails; without that a transcript is only evidence to
someone who reads it.

The old live-blob case survives as section B2, relabelled for what it is: an
incomplete reconstruction whose RAW half is refused for unrelated missing keys,
kept because its MERGED half genuinely demonstrates what App.options is.

VERIFIED BY BREAKING IT, WHICH IS THE WHOLE POINT

`inverters: []` deleted from config.yaml, probe re-run against
ghcr.io/home-assistant/amd64-hassio-supervisor:2026.08.0:

  == B. THE LOAD-BEARING ONE: an options blob with no `inverters` key ==
    FAIL  the shipped `options:` block validates AS IT STANDS - ...
          Missing option 'inverters' in root in GoodWe RS485 Controller
  ...
  == 2 CASE(S) FAILED ==

exit 1. Key restored, same image, same command: `== all cases passed ==`, exit
0. The transcript in DECISIONS is that restored run, re-run today, not the old
one.

THE FOUR MIS-CITATIONS

DECISIONS/CFG-04-inverter-roster.md section 4's "section B shows both halves"
sentence and its line-103 transcript; CHANGELOG.md; test_inverter.py's part-9
comment and its part-12 mutation row; and the PR description. All now attribute
the string to section B and say plainly what section B2 does not show. The
part-12 row additionally records that the probe was blind to that mutation until
review, because a mutation table that lists a defeat nothing catches is the
failure this repository keeps finding.

MUTATION.md gains it as the sixth instance of "the check's data cannot reach the
failure it names" - the first found OUTSIDE the suite, in a committed evidence
artifact, because nothing in that document had ever been pointed at tools/. Three
habits with it: assert on the CONTENT of a refusal rather than that one happened;
vary ONE thing, from an ACCEPTED baseline; and treat a committed transcript as a
check subject to the mutation battery.

TWO NON-BLOCKING FINDINGS, ALSO FIXED

  validate.py rules 5 and 6 NAMED THE WRONG INVERTER. `unit_caps()` SKIPS an
  entry with no usable `max_w`, so caps position i stops being roster position i
  the moment one entry is malformed - and rule 5 fed `caps.index(weakest)` to
  `_unit_label()`, which indexes the roster, while rule 6 paired `caps[0]` with a
  hard-wired `_unit_label(opts, 0)`. Right number, another inverter's name: an
  operator sent to the wrong device by the message that exists to stop that. New
  `unit_caps_indexed()` carries the roster index with the cap. It is latent -
  rule 9(d) refuses such a config in the same pass - but that was a property of
  today's rule ordering, not of the code, and validate_startup_config() is also
  sim/inject.py's entry point. Part 10 gains a check that asserts the fixture
  really does desynchronise the two lists, then that both messages name the right
  unit, then the counterfactual: the OLD expressions, on that same config, still
  evaluate to the wrong names.

  DOCS.md called a downgrade "safe". True of the SCHEMA - the older schema drops
  the key with a warning - and incomplete about the ADD-ON: rule 9(e) requires the
  flat `setpoint_entity` be cleared when a roster is declared, so a config that
  downgrades cleanly is by construction one whose flat actuator is blank, and the
  older version has nothing to write to. Confirmed by running release/1.0's own
  validate_startup_config() on that blob: one refusal, naming setpoint_entity.
  Recoverable and loud, but an outage until a field is refilled. The note says so
  now, and tells the operator to write the entity down before downgrading.

19/19 green in the goodwe-addon-ci image.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
2026-09-04 10:33:51 +02:00

211 lines
11 KiB
Python

"""CFG-04: does Supervisor accept a list-of-objects option? MEASURED, not assumed.
⚠️ WHY THIS SCRIPT EXISTS AND WHY IT IS COMMITTED. config.yaml had no
list-of-objects option anywhere before CFG-04, and `static_profile` carries a
structured value as a JSON STRING in a `str?` precisely because nobody knew
whether the Supervisor's options validator supported a nested one. That is a
question about a specific version of a specific piece of somebody else's
software, and the honest way to answer it is to run that software.
WHAT IT RUNS. Supervisor's OWN `AppOptions` validator and its OWN
`SCHEMA_APP_CONFIG`, out of the release image, against THIS repository's
config.yaml. Nothing here is a re-implementation of the validation rules; if
Supervisor changes them, this script changes its answer.
HOW TO RUN IT (the version is the one `ha info` reports on the target install -
2026.08.0 is what 192.168.2.158 ran when CFG-04 landed):
docker run --rm \
-v "$PWD:/w:ro" -v "$PWD/tools:/probe:ro" -v /tmp/out:/out \
--entrypoint python3 \
ghcr.io/home-assistant/amd64-hassio-supervisor:2026.08.0 \
/probe/supervisor_shape_probe.py
⚠️ IT TOUCHES NOTHING. It reads config.yaml and runs a validator in a throwaway
container. It does not talk to the live Supervisor, does not POST options and
does not install anything - the schema conflict in docs/deploy-rollback.md §4 is
exactly what you do NOT want to discover on a running site.
WHAT IT ANSWERED, 2026-09-04, on 2026.08.0 - the full output is in
DECISIONS/CFG-04-inverter-roster.md:
A. The manifest, list-of-objects schema and all, is ACCEPTED.
B. The shipped `options:` block validates as it stands, and the SAME block
with ONLY `inverters` removed is refused BY NAME - `Missing option
'inverters' in root`. That is the one measurement the `inverters: []`
default rests on, and it is an ISOLATED counterfactual: exactly one key
varies between the two runs, so the refusal cannot be about anything else.
B2. A reconstructed pre-0.6.0 options blob, merged with those defaults,
validates. Its RAW form is refused - but for whichever required key the
reconstruction happens to lack FIRST, which is an artefact of the
reconstruction and NOT evidence about `inverters`. B is that evidence.
C. A two-inverter blob validates, per-entry types enforced.
D. An entry missing `name`, `setpoint_entity` or `max_w` is refused BY NAME,
and a JSON string where the list belongs is refused outright.
"""
import sys, json, copy
sys.path.insert(0, "/usr/src/supervisor")
import voluptuous as vol
from voluptuous.humanize import humanize_error
from deepmerge import Merger
from supervisor.apps.validate import SCHEMA_APP_CONFIG
from supervisor.apps.options import AppOptions
from pathlib import Path
from supervisor.utils.yaml import read_yaml_file
MERGER = Merger(type_strategies=[(dict, ["merge"])],
fallback_strategies=["override"], type_conflict_strategies=["override"])
raw = read_yaml_file(Path("/w/config.yaml"))
print("== A. the whole config.yaml through SCHEMA_APP_CONFIG ==")
try:
cfg = SCHEMA_APP_CONFIG(raw)
print(" PASS Supervisor accepts the add-on manifest, version", cfg["version"])
except vol.Invalid as e:
print(" FAIL ", humanize_error(raw, e)); sys.exit(1)
schema, defaults = cfg["schema"], cfg["options"]
print(" schema declares", len(schema), "keys; inverters ->", json.dumps(schema["inverters"]))
# .get, not [], deliberately: if someone deletes the `inverters: []` default
# this line must not crash the probe before section B gets to REFUSE it.
print(" options ship inverters ->", json.dumps(defaults.get("inverters")))
def validate(options):
v = vol.Schema(vol.All(dict, AppOptions(None, schema, cfg["name"], "local_goodwe_controller")))
try:
return True, v(copy.deepcopy(options))
except vol.Invalid as e:
return False, str(e)
FAILED = []
def show(t, ok, d=""):
print(f" {'PASS' if ok else 'FAIL'} {t}")
if d: print(" " + str(d)[:300])
if not ok: FAILED.append(t)
# The live install's persisted options, reconstructed from HANDOVER.md §1 and
# docs/deploy-rollback.md §0/§B.5. No `inverters` key: it did not exist.
LIVE = {
"meter_entity": "sensor.p1_meter_active_power", "meter_invert": False,
"soc_entity": "sensor.goodwe_battery_soc", "batt_entity": "sensor.goodwe_battery_power",
"batt_invert": False,
"setpoint_entity": "number.kelder_goodwe_master_stage1_goodwe_setpoint_w",
"meter_source": "homewizard_local", "meter_phases": 1, "meter_max_age_s": 30,
"meter_poll_s": 5, "p1_host": "192.168.2.250",
"max_w": 5000, "gain": 0.6, "slew_w": 1000, "deadband_w": 15, "target_grid_w": -10,
"step_w": 10, "saturation_w": 500, "saturation_cycles": 3, "integrator_max_w": 0,
"heartbeat_s": 10, "stale_input_s": 15, "cycle_floor_s": 15, "auto_start": False,
"maintenance_enabled": False, "estop_fitted": True, "log_level": "info",
}
print("\n== B. THE LOAD-BEARING ONE: an options blob with no `inverters` key ==")
# ⚠️ THIS IS AN ISOLATED COUNTERFACTUAL AND IT HAS TO STAY ISOLATED. The claim
# the `inverters: []` default rests on is about ONE key, so exactly ONE key
# varies between these two runs: the full shipped `options:` block, and the same
# block with `inverters` removed and nothing else touched. Feeding the validator
# some other, INCOMPLETE blob would also produce a refusal - but about whichever
# required key that blob lacks first, which is a different measurement wearing
# this one's label. §B2 below is that other blob, and it is labelled as such.
ok, out = validate(dict(defaults))
show("the shipped `options:` block validates AS IT STANDS - so anything refused "
"on the next line is refused for the one key that was removed", ok,
"" if ok else out)
_no_inv = {k: v for k, v in defaults.items() if k != "inverters"}
ok_ni, out_ni = validate(_no_inv)
# ⚠️ THE ASSERTION IS ON THE KEY NAME, NOT ON `not ok`, AND THAT IS THE POINT.
# A refusal is only evidence for THIS key if it NAMES this key. Asserted as a
# bare `not ok`, this line would go on printing PASS after someone deleted
# `inverters: []` from config.yaml - blind to the exact regression it is the
# committed evidence for. Asserted on the string, it goes red: with the default
# gone the line above fails too, because then nothing distinguishes the two runs.
_names_it = (not ok_ni) and "Missing option 'inverters' in root" in str(out_ni)
show("...and the SAME block with ONLY `inverters` removed is REFUSED, BY NAME - "
"which is every options blob written before 0.6.0, since the key did not "
"exist when they were written", _names_it, out_ni)
print("\n== B2. AC3: a reconstructed pre-0.6.0 blob upgrades into this version ==")
# ⚠️ WHAT THIS SECTION IS AND IS NOT. LIVE is an INCOMPLETE reconstruction - it
# carries the keys HANDOVER.md and deploy-rollback.md record and no others, so
# it is missing many required keys that have nothing to do with this card. Its
# raw refusal is therefore about whichever of THOSE the validator reaches first
# (it has printed `meter_freeze_s` and `p1_phase_export_entities` on different
# runs), NOT about `inverters`. It is kept because the MERGED half is a real
# demonstration that `deepmerge(options:, stored)` is what App.options is; the
# raw half is printed only so the merge has something to be contrasted with.
# The `inverters` measurement is §B, where one key and only one key varies.
ok, out = validate(LIVE)
show("the reconstructed raw blob is REFUSED - for an unrelated missing key, "
"because the reconstruction is incomplete; see the refusal printed below "
"and §B for the `inverters` measurement", not ok, out)
merged = MERGER.merge(copy.deepcopy(defaults), copy.deepcopy(LIVE))
ok, out = validate(merged)
show("the same blob MERGED with these defaults - what App.options actually is", ok,
f"inverters={out.get('inverters')!r} setpoint_entity={out.get('setpoint_entity')!r} "
f"max_w={out.get('max_w')!r}" if ok else out)
print("\n== C. AC2: a two-inverter blob an operator would POST ==")
TWO = dict(LIVE, setpoint_entity="", inverters=[
{"name": "inverter-a", "setpoint_entity": "number.goodwe_a_setpoint_w",
"soc_entity": "sensor.goodwe_a_soc", "batt_entity": "sensor.goodwe_a_battery_power",
"max_w": 5000},
{"name": "inverter-b", "setpoint_entity": "number.goodwe_b_setpoint_w",
"soc_entity": "sensor.goodwe_b_soc", "batt_entity": "sensor.goodwe_b_battery_power",
"max_w": 3000}])
ok, out = validate(MERGER.merge(copy.deepcopy(defaults), copy.deepcopy(TWO)))
show("two inverters", ok, json.dumps(out.get("inverters")) if ok else out)
# The validated blob, for anyone who wants to POST it. Optional: the probe's
# whole job is the verdicts above, and a missing /out must not fail it.
try:
json.dump(out if ok else {}, open("/out/two_inverter_options.json", "w"), indent=1)
except OSError:
pass
print("\n== D. what the schema itself refuses, per entry ==")
for title, entry in (
("no name", {"setpoint_entity": "number.b", "max_w": 3000}),
("no setpoint_entity", {"name": "inverter-b", "max_w": 3000}),
("no max_w", {"name": "inverter-b", "setpoint_entity": "number.b"}),
("max_w over the range", {"name": "inverter-b", "setpoint_entity": "number.b", "max_w": 9999})):
ok, out = validate(dict(TWO, inverters=[entry]))
show(f"an entry with {title} is refused", not ok, out)
ok, out = validate(dict(TWO, inverters="[]"))
show("a JSON STRING where the list belongs is refused", not ok, out)
print("\n== E. the ROLLBACK direction, and the operator's UI ==")
# The old (flat) schema, i.e. what an operator would be running after a
# downgrade, meeting options that carry an `inverters` list.
_flat_schema = {k: v for k, v in schema.items() if k != "inverters"}
_v = vol.Schema(vol.All(dict, AppOptions(None, _flat_schema, cfg["name"],
"local_goodwe_controller")))
try:
_rolled = _v(MERGER.merge(copy.deepcopy(defaults), copy.deepcopy(TWO)))
show("a schema WITHOUT `inverters` drops the key with a warning rather "
"than refusing - so a downgrade is not the §B trap in reverse",
"inverters" not in _rolled, sorted(_rolled)[:6])
except vol.Invalid as e:
show("a schema WITHOUT `inverters` drops the key", False, str(e))
from supervisor.apps.options import UiOptions # noqa: E402
try:
_ui = [e for e in UiOptions(None)(schema) if e.get("name") == "inverters"]
show("Supervisor's UI renderer turns it into a repeatable section rather "
"than choking - an operator can add the second inverter in the "
"Configuration tab, which the JSON-string alternative could not offer",
bool(_ui) and _ui[0].get("multiple") is True, json.dumps(_ui))
except Exception as e: # noqa: BLE001
show("Supervisor's UI renderer survives the shape", False, f"{type(e).__name__}: {e}")
# ⚠️ THE EXIT CODE IS PART OF THE INSTRUMENT. Without it the probe prints FAIL
# and still succeeds, and a transcript is only evidence if a wrong answer is
# distinguishable from a right one by something other than reading it.
print()
if FAILED:
print(f"== {len(FAILED)} CASE(S) FAILED ==")
for t in FAILED:
print(" - " + t)
sys.exit(1)
print("== all cases passed ==")