CFG-04 fix round: the probe now measures the key it is cited for
checks / unit (pull_request) Successful in 4m33s
checks / unit (pull_request) Successful in 4m33s
Review blocked this on ONE finding, and it was in the EVIDENCE, not the code.
The design and the shipped behaviour are unchanged.
THE INSTRUMENT DID NOT MEASURE WHAT FOUR DOCUMENTS SAID IT MEASURED
tools/supervisor_shape_probe.py section B validated the RECONSTRUCTED LIVE
options blob - which is missing many required keys that have nothing to do with
this card. Its refusal therefore named whichever of THOSE the validator reached
first: the committed transcript recorded `meter_freeze_s`, the reviewer's run
printed `p1_phase_export_entities`, and a run today printed `sign_disagree_w`.
The string those documents quoted - `Missing option 'inverters' in root` - was
never produced by the instrument.
And section B asserted a bare `not ok`. Any refusal at all satisfied it, so
deleting `inverters: []` from config.yaml left it printing PASS: the probe could
not detect the one regression it is the committed evidence for.
THE UNDERLYING CLAIM IS TRUE AND IS NOW MEASURED PROPERLY
Section B is an ISOLATED COUNTERFACTUAL now: the full shipped `options:` block
(expect accept), and the SAME block with `inverters` removed and nothing else
varying (expect refusal). One key differs, so the refusal has nowhere else to
come from - and the assertion is on the KEY NAME IN THE REFUSAL STRING, so the
line goes red if the refusal is ever about a different key. The probe also exits
non-zero when any case fails; without that a transcript is only evidence to
someone who reads it.
The old live-blob case survives as section B2, relabelled for what it is: an
incomplete reconstruction whose RAW half is refused for unrelated missing keys,
kept because its MERGED half genuinely demonstrates what App.options is.
VERIFIED BY BREAKING IT, WHICH IS THE WHOLE POINT
`inverters: []` deleted from config.yaml, probe re-run against
ghcr.io/home-assistant/amd64-hassio-supervisor:2026.08.0:
== B. THE LOAD-BEARING ONE: an options blob with no `inverters` key ==
FAIL the shipped `options:` block validates AS IT STANDS - ...
Missing option 'inverters' in root in GoodWe RS485 Controller
...
== 2 CASE(S) FAILED ==
exit 1. Key restored, same image, same command: `== all cases passed ==`, exit
0. The transcript in DECISIONS is that restored run, re-run today, not the old
one.
THE FOUR MIS-CITATIONS
DECISIONS/CFG-04-inverter-roster.md section 4's "section B shows both halves"
sentence and its line-103 transcript; CHANGELOG.md; test_inverter.py's part-9
comment and its part-12 mutation row; and the PR description. All now attribute
the string to section B and say plainly what section B2 does not show. The
part-12 row additionally records that the probe was blind to that mutation until
review, because a mutation table that lists a defeat nothing catches is the
failure this repository keeps finding.
MUTATION.md gains it as the sixth instance of "the check's data cannot reach the
failure it names" - the first found OUTSIDE the suite, in a committed evidence
artifact, because nothing in that document had ever been pointed at tools/. Three
habits with it: assert on the CONTENT of a refusal rather than that one happened;
vary ONE thing, from an ACCEPTED baseline; and treat a committed transcript as a
check subject to the mutation battery.
TWO NON-BLOCKING FINDINGS, ALSO FIXED
validate.py rules 5 and 6 NAMED THE WRONG INVERTER. `unit_caps()` SKIPS an
entry with no usable `max_w`, so caps position i stops being roster position i
the moment one entry is malformed - and rule 5 fed `caps.index(weakest)` to
`_unit_label()`, which indexes the roster, while rule 6 paired `caps[0]` with a
hard-wired `_unit_label(opts, 0)`. Right number, another inverter's name: an
operator sent to the wrong device by the message that exists to stop that. New
`unit_caps_indexed()` carries the roster index with the cap. It is latent -
rule 9(d) refuses such a config in the same pass - but that was a property of
today's rule ordering, not of the code, and validate_startup_config() is also
sim/inject.py's entry point. Part 10 gains a check that asserts the fixture
really does desynchronise the two lists, then that both messages name the right
unit, then the counterfactual: the OLD expressions, on that same config, still
evaluate to the wrong names.
DOCS.md called a downgrade "safe". True of the SCHEMA - the older schema drops
the key with a warning - and incomplete about the ADD-ON: rule 9(e) requires the
flat `setpoint_entity` be cleared when a roster is declared, so a config that
downgrades cleanly is by construction one whose flat actuator is blank, and the
older version has nothing to write to. Confirmed by running release/1.0's own
validate_startup_config() on that blob: one refusal, naming setpoint_entity.
Recoverable and loud, but an outage until a field is refilled. The note says so
now, and tells the operator to write the entity down before downgrading.
19/19 green in the goodwe-addon-ci image.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
This commit is contained in:
co-authored by
Claude Opus 5
parent
6b85acb0f1
commit
22c016fadd
@@ -135,9 +135,11 @@ manufactures them from a **fixture**: the check runs, the assertion is
|
||||
evaluated, the sentence above it is true — and the numbers it compares are
|
||||
arranged so that the failure it describes could not have changed the answer.
|
||||
|
||||
It is not hypothetical and it is not rare. Five instances in two weeks; the
|
||||
first three were found by review rather than by a red line, and the last two -
|
||||
CFG-04's - by an author's own mutation battery, which is the cheaper place:
|
||||
It is not hypothetical and it is not rare. Six instances in two weeks; three
|
||||
were found by review, two - CFG-04's fixture pair - by an author's own mutation
|
||||
battery, which is the cheaper place, and the sixth by review again, in the one
|
||||
place nobody had thought to point the rule at: not a check in the suite, but a
|
||||
COMMITTED EVIDENCE ARTIFACT four documents cite.
|
||||
|
||||
* **ARB-03** shipped a call-site scan asserting that every `claims.append` in
|
||||
`cycle()` routes through `actuator_of()`. Its own parse **saw 3 of the 8
|
||||
@@ -165,6 +167,26 @@ CFG-04's - by an author's own mutation battery, which is the cheaper place:
|
||||
than against its behaviour. It now drives the shipped `run_control()` on a
|
||||
two-inverter config, **under a deadline**, because the mutation's real symptom
|
||||
is that the controller enters its loop and never returns.
|
||||
* **CFG-04's `tools/supervisor_shape_probe.py` §B** — and this one is the rule
|
||||
reaching outside the suite. The probe is the committed evidence that
|
||||
`config.yaml`'s `inverters: []` default is load-bearing, and four documents
|
||||
quoted it as answering *"Missing option 'inverters' in root"*. §B validated
|
||||
the reconstructed LIVE options blob, which is missing many required keys
|
||||
unrelated to this card, so its refusal named whichever of THOSE the validator
|
||||
reached first — `meter_freeze_s`, `p1_phase_export_entities` and
|
||||
`sign_disagree_w` on three different runs. The quoted string was never
|
||||
produced by the instrument. And the assertion was a bare **`not ok`**: delete
|
||||
`inverters: []` from `config.yaml` and §B went on printing **PASS**, blind to
|
||||
the exact regression it was cited for. §B is now an isolated counterfactual —
|
||||
the shipped defaults, and the same defaults minus that one key — asserting on
|
||||
the **key name in the refusal**; the mis-cited blob survives, relabelled, as
|
||||
§B2. Re-run with the default deleted it prints `2 CASE(S) FAILED` and exits 1.
|
||||
|
||||
⚠️ **A committed transcript is a check, and it decays like one.** Nothing in
|
||||
this document had been pointed at `tools/`, because the artifacts there are
|
||||
filed as *measurements* rather than as *checks* — and a measurement whose
|
||||
assertion cannot fail is exactly the thing this section is about. Two habits
|
||||
follow, and both are habit 5 and 6 below.
|
||||
|
||||
Note what these have in common with a masked rung: the mutation *runs*, and the
|
||||
suite is *honestly* green. Nothing in a purge, a fingerprint or a disassembly
|
||||
@@ -177,7 +199,7 @@ check was handed.
|
||||
> **Any check asserting "X differs from Y" must be run once with X and Y
|
||||
> actually made different.**
|
||||
|
||||
Three cheap habits fall out of it, and all three are now used in this repository:
|
||||
Cheap habits fall out of it, and all of them are now used in this repository:
|
||||
|
||||
1. **Write the PRECONDITION as its own check.** `test_peakguard.py` 10g asserts
|
||||
that A and B have different ratings *before* it asserts that B's bound is
|
||||
@@ -202,6 +224,23 @@ Three cheap habits fall out of it, and all three are now used in this repository
|
||||
5. **A check whose failure mode is a HANG needs a deadline.** Otherwise the
|
||||
mutation that defeats it produces a CI timeout with no name attached, which
|
||||
reads like flaky infrastructure and gets retried rather than read.
|
||||
6. **Assert on the CONTENT of a refusal, not on the fact that one happened.**
|
||||
`not ok` is satisfied by any error at all, including one about a different
|
||||
key on a different blob. `"Missing option 'inverters' in root" in out` goes
|
||||
red the moment the refusal stops being about the thing it is cited for. This
|
||||
is habit 2 in the negative direction: a refusal, like a difference, has to be
|
||||
asserted against a literal.
|
||||
7. **Vary ONE thing in a counterfactual, and vary it from an ACCEPTED
|
||||
baseline.** `supervisor_shape_probe.py` §B validates the shipped defaults
|
||||
first, precisely so the refusal on the next line has nowhere else to come
|
||||
from. Without the accepted baseline the second line is a refusal of unknown
|
||||
provenance, which is what §B used to print.
|
||||
8. **A committed transcript is a check.** Anything under `tools/` that a
|
||||
document quotes is subject to every rule above, and to the mutation battery:
|
||||
break the property, re-run the instrument, and confirm the transcript goes
|
||||
red. `inverters: []` was deleted from `config.yaml` and the probe re-run
|
||||
against `ghcr.io/home-assistant/amd64-hassio-supervisor:2026.08.0` for
|
||||
exactly that reason.
|
||||
|
||||
And the mutation-table consequence: **a variant is not covered because a
|
||||
near-miss of it is.** DEV-02's table listed the *actuator* variant of the
|
||||
|
||||
Reference in New Issue
Block a user