adminandClaude Opus 5
8a6acd9fa2
checks / unit (pull_request) Successful in 8m0s
Text only. No behaviour change - same conjunction, same limits, same MISSING, same 0 W, same NOT READY. `tools/safety56_behaviour_trace.py` says so: 2390 capture reads, a 66-cell swept grid, and six driven controller worlds produce the byte-identical document `6030175e...` on origin/release/1.0 and on this branch, while the sentences differ. THE DEFECT. SAFETY-37's conjunction is satisfied by two disjoint worlds - the register has stopped reporting reality, or the SOURCE cannot comply with the command - and nothing `freshness.py` is handed separates them. Three sentences asserted the first. On the live 2026-09-21 episode it was the second: the pack read 11 % for the whole 850 s hold, every command in that hold was a discharge, and the register answered the first CHARGE command three seconds later. The readiness row was worse still, because it asserted a false fact about a NAMED entity that was present, numeric and updating - so the operator opened Developer Tools, found it working, and had every reason to conclude the diagnostics were broken. THE FIX. Each message states what was OBSERVED and names BOTH explanations. Three wordings, no shared helper: two are a log line about a behaviour, the third is a banner about an entity, and a helper serving both would drop whichever half the other caller does not need. SoC stays STATED, NOT WIRED - part 11b drives the two worlds at 50 % and 11 % and the sentence does not move one byte. The row does not branch on `batt_fresh.stale` either: `stale_note()` already rides `liveness_notes()` a few rows below, so the discriminator is "is that row there?", and part 11d asserts it is present in one world and absent in the other. THE CHECKS. test_freshness.py part 11 builds the pair of worlds with ground truth written down rather than derived from the published stream, asserts they are really two (40 of 40 reads differ) while the published stream is bit-identical, and only then grades the sentences. Eleven mutations, all red: each of the three sentences reverted (3, 2 and 2 named reds), the two explanations transposed at ZERO bytes (1), the second explanation deleted (1), the admission of ambiguity replaced by a ranking (1), all three reverted at once (5), and two aimed at this card's own check - collapsing the world pair (2) and emptying the accusation list, which a positive control against the three sentences as 0.10.1 shipped them catches (1). AND TWO OF THE ELEVEN FOUND DEFECTS IN THE CHECK RATHER THAN IN THE CODE. Transposing `label` and `entity` in the row made it unfindable and the run died on an IndexError - a red with no name and every later check ungraded, FCST-06's shape exactly; `_s56_row()` returns a named absence now, 0 named reds before, 3 after. And the first invariance run compared a tree with itself, because it archived a branch whose edits were not committed yet; the tool now REFUSES to report "behaviour unchanged" unless the sentences halves differ, and both of its refusals are driven. SAFETY-57's part 10h check was the handoff and is inverted rather than deleted: it required the accusation to still be present so that removing it had to be a deliberate act. This is that act. Its escape clause is untouched. Suite: 30/30 in the CI image, checks.yml's exact command, purged tree, :ro. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
182 lines
10 KiB
Markdown
182 lines
10 KiB
Markdown
# SAFETY-56 — what the gate cannot tell apart, said in the sentence
|
||
|
||
**Text only. No behaviour changes in this card**, and §5 is the measurement that
|
||
says so: the same stimuli through the tree before and after produce a
|
||
**byte-identical** trace of every verdict, every `reading()`, every `inputs_ok`
|
||
and every readiness row's `(ok, warn)`.
|
||
|
||
---
|
||
|
||
## 1. The defect
|
||
|
||
0.10.1 is regulating a real GW5000S-BP. Two messages tell the operator that
|
||
healthy hardware has failed.
|
||
|
||
**The log line**, as emitted on the reference site on 2026-09-21:
|
||
|
||
> SAFETY-37: inverter-a battery-power register **is not following its own
|
||
> command** - STALE_READING: -3 unchanged for 36 s while the standing command
|
||
> moved 800 W to +1600 W - **the reading is not following**, so it is reported
|
||
> as MISSING rather than as a measurement … Treating it as MISSING, which
|
||
> commands 0 W.
|
||
|
||
**The readiness row**, at the same moment:
|
||
|
||
> NOT READY — battery power: `sensor.kelder_goodwe_master_stage1_goodwe_inverter_ac_power`
|
||
> **is missing or not numeric**
|
||
|
||
That entity was present, numeric and updating.
|
||
|
||
**SAFETY-37's conjunction is correct and is not touched by this card** —
|
||
`unchanged > 35 s AND |command| > 250 W AND |travel| > 250 W`. A register
|
||
genuinely stuck against a real command is exactly what it should catch, and a
|
||
pack that cannot comply is a state the site must stop in too. What was wrong is
|
||
what the sentences **concluded** from it.
|
||
|
||
## 2. Two worlds satisfy the conjunction
|
||
|
||
| | the register | the pack |
|
||
|---|---|---|
|
||
| **DEAD** | has stopped reporting reality | is doing what it was told |
|
||
| **REFUSED** | is reporting reality correctly | cannot comply — the BMS is refusing at its floor |
|
||
|
||
A refused pack holds still under any setpoint, so an honest register publishes
|
||
the same unchanged number a dead one does. **Nothing `freshness.py` is handed
|
||
separates them.** All three sentences asserted DEAD.
|
||
|
||
## 3. The live episode was REFUSED, and it is measured, not argued
|
||
|
||
Off the committed capture `scenarios/safety57-2026-09-21/bus-a-2026-09-21-1030-1110.log.gz`,
|
||
whose bytes and decoder are already graded by `test_freshness.py` part 10a:
|
||
|
||
| | |
|
||
|---|---|
|
||
| the register | answered **-3 W for 850 s**, 10:51:24 → 11:05:34 |
|
||
| the commands during that hold | **+260, +660, +1060 W** — discharge only, never a charge |
|
||
| pack SoC throughout | **11 %** |
|
||
| first **charge** command after the restart | 11:05:31 at **-400 W** |
|
||
| the register moves | **11:05:34** — **3 s**, inside one 17.0 s obligation window |
|
||
|
||
**A register that answers a reversed command inside one obligation window had
|
||
not stopped reporting.** What would not move was the pack. The sentence the
|
||
operator read named the one thing in the loop that was telling the truth.
|
||
|
||
⚠️ What this does **not** establish: that no register is ever intermittently
|
||
wrong. It excludes a register that had simply stopped; it cannot exclude one
|
||
that was wrong and then right. That is the point — the new sentence claims
|
||
neither.
|
||
|
||
## 4. The fix, and why it is three sentences and not one helper
|
||
|
||
Each message states **what was observed** and names **both** explanations.
|
||
|
||
| where | before | after |
|
||
|---|---|---|
|
||
| `app/freshness.py` `_grade()` | *"…+1600 W - the reading is not following, so it is reported as MISSING…"* | *"…+1600 W. **TWO THINGS LOOK LIKE THIS AND THIS GATE CANNOT TELL THEM APART**: the register may have stopped reporting reality, or the source may be unable to comply with the command - a pack the BMS is refusing holds still under any setpoint, and an honest register then reads exactly like a frozen one. Either way the number cannot be read as this command's effect, so it is reported as MISSING…"* |
|
||
| `app/main.py` `_read_unit()` | *"SAFETY-37: %s battery-power **register is not following its own command** - %s."* | *"SAFETY-37: %s battery-power reading **did not move under a command that did, WHICH IS NOT BY ITSELF A VERDICT ON THE REGISTER** - %s."* |
|
||
| `app/main.py` `checks()` | *"{label}: {entity} **is missing or not numeric**"* | *"{label}: **no usable value this cycle from** {entity} - **and this row cannot tell which of two things that is**: the entity is missing or not numeric, or it is publishing a number that this controller is withholding (a freshness gate does exactly that, and names itself in a row of its own). Confirm the entity in Developer Tools before replacing it"* |
|
||
|
||
**Three sentences, three wordings, no shared helper.** The first two are a log
|
||
line about a **behaviour**; the third is a banner about a named **entity**, and
|
||
the two answer different questions ("what did the register do" against "can I
|
||
trust this entity"). A helper serving both would have to drop whichever half
|
||
the other caller does not need, and the half it dropped would be the
|
||
operator's next step. The log line names the pair **once**, inside the `%s` it
|
||
already interpolates, rather than in both of its own halves.
|
||
|
||
**No new input, and SoC is not wired.** `BOUNDS["soc"]` stays *STATED, NOT
|
||
WIRED*; telling the two worlds apart needs the pack floor (SAFETY-32's number)
|
||
and SoC (TEL-08's wiring), and a message honest about its own ambiguity needs
|
||
neither. Part 11b drives the two worlds at **50 %** and **11 %** SoC and
|
||
asserts the sentence does not move one byte.
|
||
|
||
**The readiness row does not branch.** `inv.batt_fresh.stale` is on the same
|
||
object at the same moment, and the row deliberately does not ask it: the same
|
||
loop renders the grid and SoC rows, which have no gate, and a branch per row
|
||
would be a second place where "why is this None" is decided. The gate already
|
||
has a surface — `stale_note()` rides `liveness_notes()` a few rows below with
|
||
the whole STALE_READING sentence in it — so the operator's discriminator is
|
||
*"is that row there?"*, and part 11d asserts that row is **present in REFUSED
|
||
and absent in ABSENT**.
|
||
|
||
## 5. Behaviour is byte-identical, measured
|
||
|
||
`tools/safety56_behaviour_trace.py` prints two documents from one stimulus set.
|
||
The `behaviour` half contains **no sentence**: only `stale`, `reading()`,
|
||
`releases`, `held`, the window bookkeeping, `inputs_ok`, and every readiness
|
||
row's `(ok, warn)`. Run in the CI image, purged, `:ro`, against
|
||
`origin/release/1.0` **archived out of git** and against this branch:
|
||
|
||
| | |
|
||
|---|---|
|
||
| the committed live capture | **2390 reads**, 7 verdict edges |
|
||
| a swept grid of command × register schedules | **66 cells × 120 reads** — 1060 stale reads, 1309 `MISSING` readings, 6 releases |
|
||
| six driven controller worlds through the shipped read path | DEAD, REFUSED, ABSENT, HEALTHY, RELEASE, RETRIP — every read's `(batt, inputs_ok, stale)` and every `checks()` row's `(ok, warn)` |
|
||
| **result** | `6030175ed2ab9a4143e86e89a71d45c8ab041ac1c00121a3e3e023cd2b08996c` **on both trees** |
|
||
|
||
⚠️ **AND THE CONTROL IS INSIDE THE TOOL, BECAUSE AN IDENTITY BETWEEN TWO TREES
|
||
THAT ARE THE SAME TREE IS HABIT 4.** `--compare` refuses to report "behaviour
|
||
unchanged" unless the `sentences` halves **differ**. This card's first
|
||
invariance run was exactly that mistake — it archived a branch whose edits were
|
||
not committed yet, so `before` and `after` were the same code and the identity
|
||
was perfect and worthless.
|
||
|
||
Both of the tool's refusals are **driven**, not asserted:
|
||
|
||
| | |
|
||
|---|---|
|
||
| the same file compared with itself | `REFUSED: the sentences halves are IDENTICAL…`, exit **2** |
|
||
| `MOVE_W` 250.0 → 240.0 in the after tree | `BEHAVIOUR MOVED at 'sweep_stale_reads' / 'sweep_missing_reads' / 'sweep_trace'`, exit **1** |
|
||
| this card | `OK: sentences differ, behaviour is byte-identical.`, exit **0** |
|
||
|
||
## 6. The mutation battery
|
||
|
||
Each in its own `mktemp`'d copy (habit 18), purged on the host, mounted `:ro`,
|
||
graded by `tools/mutation_run.sh` in the CI image.
|
||
|
||
| | mutation | Δ bytes | named reds |
|
||
|---|---|---|---|
|
||
| M1 | `_grade()` reverted to *"the reading is not following"* | -402 | **3** |
|
||
| M2 | the log line reverted to *"register is not following its own command"* | -80 | **2** |
|
||
| M3 | the row reverted to *"{entity} is missing or not numeric"* | -619 | **2** |
|
||
| M4 | *register* and *source* transposed between the two explanations | **0** | **1** |
|
||
| M5 | the second explanation deleted, the first kept | -216 | **1** |
|
||
| M6 | *"CANNOT TELL THEM APART"* replaced by *"IN ORDER OF LIKELIHOOD"* | -13 | **1** |
|
||
| M7 | the REFUSED world's ground truth set to the DEAD world's | 0 | **2** |
|
||
| M8 | the row's second cause deleted | -309 | **1** |
|
||
| M9 | `label` and `entity` transposed in the row | **0** | **3** |
|
||
| M10 | M1 + M2 + M3 together, i.e. exactly what 0.10.1 ships | -1101 | **5** |
|
||
| M11 | the accusation literals replaced by one that matches nothing | 0 | **1** |
|
||
|
||
⚠️ **M9's first run was a defect in this card's own check, and it is the one
|
||
worth reading.** The row was fetched with `[...][0]`. Transposing `label` and
|
||
`entity` makes the row unfindable, so the run died on an `IndexError` — a red
|
||
with **no name attached**, with every check after that line **ungraded**. That
|
||
is FCST-06's shape exactly. `_s56_row()` now returns a named absence instead of
|
||
raising, and a precondition grades it: **0 named reds before, 3 after.**
|
||
|
||
⚠️ **M11 is the other one aimed inward.** `not any(...)` over an empty list is
|
||
True, and so is one over three mistyped literals. The positive control runs the
|
||
same predicate against the three sentences **as 0.10.1 shipped them** and
|
||
requires it to find an accusation in every one, and every literal to match at
|
||
least one of them (habit 7: vary one thing from an accepted baseline).
|
||
|
||
## 7. What part 11 does not grade
|
||
|
||
* It does not show the live episode was a BMS refusal beyond doubt — see §3.
|
||
* It does not grade English. A sentence that named both explanations and then
|
||
said something useless about them would pass.
|
||
* The `no entity configured` branch beside the rewritten row is untouched and
|
||
ungraded: a blank entity really is the only thing it can be.
|
||
* The **grid-power and battery-SoC rows** go through the same rewritten line and
|
||
have no freshness gate today, so for them the second cause is a hypothesis
|
||
about this controller rather than a live possibility. That is weaker than the
|
||
battery-power case and is the price of not branching per row. A gate on either
|
||
signal — TEL-08 would be one — makes it exact.
|
||
* **SAFETY-58 is not touched** (the first read after a long rest trips before a
|
||
healthy register could answer), and neither is the 710/790 disagreement
|
||
between `test_freshness.py` and the changelog. Both are separate cards; the
|
||
second has a **fourth site** at
|
||
`DECISIONS/SAFETY-57-the-exit-from-a-stale-verdict.md:274` that a fix written
|
||
off the PR table alone would miss.
|