Files
goodwe-addon/goodwe_controller/DECISIONS/MAINT-05-per-pack-maintenance.md
T
Claude 5d27f8496f
checks / unit (pull_request) Failing after 17m13s
infra-pin / fresh (pull_request) Successful in 2s
SAFETY-78: maintenance drain at N=2 no longer stalls
At two inverters the drain pins its pack and the site target is split
around the pin. Below the pin the other pack had to charge, the allocator
refused the opposite directions, and the site sat at 0 W with an ARBITER
line every cycle (house below or above the drain rate, or a grid charge).

main.py now files a site limit `drain` (lo = what the pin places, at
P_MAINTENANCE) at N>=2 while a pack drains, so the other pack only adds
power in the same direction. A drain that cannot give power waits with
one log line. With battery modes on, no grid charge while a pack drains,
and the other battery's card says why. N=1 is byte-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:52:51 +02:00

368 lines
21 KiB
Markdown

# MAINT-05 — one recalibration per pack, one at a time, and what the card left open
**Status: DECIDED 2026-09-04.** There is one `Maintenance` per roster entry,
each with its own schedule, phase and history. The site holds exactly **one
maintenance slot**: at most one pack may be in a power-moving phase, the slot is
derived from the packs' own stored phases rather than persisted separately, and
when several packs are due in the same hour it goes to the one that has gone
**longest without a completed cycle**. The schedule is **not** staggered by
construction — the interlock defers, and the stagger emerges from it.
**Fix round 2026-09-04.** The store migration is now graded against the
**roster** rather than against the unit name it was handed — see §2's
*"The read-back postcondition is graded against the ROSTER"* — and the same
hole in SAFETY-02's latch migration is recorded in §7.
Filed with MAINT-05. Card `6a9a9e8503872905f298d80a`.
---
## 0. What the card asked to be decided, and the answers
| the card's open question | the decision |
|---|---|
| §2 the arbitration | **one pack at a time; the other stays under the loop.** Nothing preempts a cycle in flight. |
| §2 who wins when several are due | **the longest-waiting pack**, `last_completed` oldest first, `inf` for never-recalibrated. Roster order breaks a numeric tie and nothing else. |
| §4 staggering | **not by construction.** The interlock defers the second pack; the stagger is what comes out of it. §3 below. |
| AC 7 rule 6's `maint_unit` comment | **no, the first entry's cap is no longer right.** It grades against the weakest cap on the roster now, like rule 5. §4 below. |
| the store layout at N=1 | **flat, unchanged.** The namespace appears the day the roster grows. §2 below. |
## 1. Why the slot is derived and not persisted
The holder is *"whichever pack's stored phase is not `idle`"* — a fact already on
disk, written by the packs themselves in the per-pack history this card
introduces. A `maint_slot_holder` key would be a **second opinion about the same
thing**, and the two would disagree the first time the add-on was killed between
writing the phase and writing the holder. Derived, a controller that comes back
up mid-drain reads the drain and re-derives the holder from it; there is no
recovery path to get wrong because there is no second fact.
`CHARGE` is in `POWER_MOVING` even though the control **loop** owns the setpoint
in that phase, and that is the decision most likely to be "simplified" later.
Pack A charging from the grid at 2000 W while pack B drains 2000 W to export is
4 kW going round in a circle with the meter reading zero and both packs
believing they are behaving — the circulating power [[SAFETY-31]] exists to
detect, arriving as something this module can simply not do. `HOLD` is in the
tuple for the other half of the sentence: it moves nothing, and it **owns the
actuator**, so a second cycle starting under it would still be two
recalibrations at once.
## 2. Why the store layout at a roster of one does not change
The maintenance keys are flat — `phase`, `phase_started`, `last_completed`,
`last_start_attempt` — on every install since the add-on shipped. At one pack
there is nothing for a namespace to distinguish that pack **from**: §6.2's
identity, the same one `arbiter.actuator_for()` leans on when it routes every
claim to the site actuator at N=1. So `Maintenance` is constructed with
`unit=None` there, reads and writes the flat keys directly, and
`migrate_maint_keys()` **returns without writing anything**. A one-inverter state
file comes out of this card byte-identical, which is AC 6 and which
`test_maintenance.py` part 10 asserts by driving a whole cycle and comparing the
key set.
### The migration has its own marker, and that is not tidiness
`SCHEMA_VERSION` / `schema_version` is DEV-02's, and it is **monotone**: every
install runs that migration exactly once. This one runs **only where the roster
has more than one pack**, so a single counter could not express *"migrated on
this site, deliberately not migrated on that one"* without lying to the next
version that reads it — a one-inverter store would have to be pinned at the old
number forever, and a genuine future v2 migration could then never be told apart
from a site that simply never grew. `maint_schema_version` answers one question
and answers it honestly.
The **three rules are shared in code**, not restated: `_move_flat_keys()` is
rule 1 (never overwrite an existing namespaced value) and rule 2 (the flat keys
stay in the file), `_verify_moves()` is the read-back postcondition, and the
marker is written last. A second transcription of a migration is a second
opinion about what "it landed" means.
### The read-back postcondition is graded against the ROSTER (fix round)
**A postcondition that reads back the namespace it was handed cannot detect
being handed the wrong one.** That sentence is the whole finding, and this card
shipped it broken.
`_verify_moves()` iterated `rep.moved` — the keys it had just written — and
asked whether each read back equal. On a store where the write lands, it always
does. So a migration handed a `unit` that is on no roster wrote four keys, read
four keys back, set `verified = True`, and the caller stamped
`maint_schema_version`. Every symptom of a clean migration, and an install's
whole maintenance history parked under a name nothing will ever look up. And
because the marker was stamped, **the next start skipped the migration too** —
which is precisely the state `store.py`'s header says must never happen: a
failed migration a later version treats as done.
Reproduced on the reviewer's roster, a house with a garage pack and a shed pack,
carrying a flat `last_completed` of 2026-08-07:
| | `unit` the migration was handed | key written | pack 0 reads back | due now? |
|---|---|---|---|---|
| correct | `self.inverters[0].name` = `garage` | `unit/garage/last_completed` | `2026-08-07` | **False** |
| wrong | `inverter.UNIT_A` = `inverter-a` | `unit/inverter-a/last_completed` | `None` | **True** |
`build_roster()` names a declared roster's entries **from the config**, and its
fallback for a nameless entry is a **positional** `inverter-1` — so `inverter-a`
is not the name of anything on a declared roster, not even by default.
**The remedy, and why this one.** `migrate_maint_keys()` now takes the **roster**
in place of an `n_units` count (the count was `len(self.inverters)` at the only
call site, so the roster carries it, and a count passed beside the names is a
second opinion about how many packs there are). `_verify_moves()` grades the
namespace it wrote against those names, and a unit that is not among them is
reported as `misdirected` — a **different** failure from `lost`:
* `lost` is *the store took the write and dropped it* — a storage fault, and it
has a harmless-default case (a write of `last_completed: None` loses nothing,
because "absent" and "never recalibrated" are the same statement).
* `misdirected` is *the store did exactly as it was told, by a caller that was
wrong*. There is no harmless-default version of it: a write nobody reads is
not information, whatever its value. So anything in it fails `ok` on its own.
And **`ok` gates the marker**, so a misdirected migration is not stamped and the
next start runs it again. The argument for it is asymmetric: a re-run is
recoverable — rule 1 makes it a no-op where keys did land, and rule 2 left the
flat keys in the file to recover *from* — while a silent skip is not recoverable
at all, because the store now claims the migration happened.
### ⚠️ THE PARAGRAPH ABOVE USED TO END HERE, AND AS IT STOOD IT WAS FALSE
**The refusal lasted exactly one start. The second start stamped the marker.**
That is a correction to what the first fix round wrote down, not a new
qualification of it, and it is recorded here because a decision document that
states a guarantee the code does not provide is worse than one that says
nothing.
`_verify_moves()` graded only `rep.moved`. Measured at Controller level over
**one persisted store**, roster `["garage", "shed"]`, flat `last_completed`
2026-08-07, with the call site still handing `inverter-a` on **both** starts:
```
construction 1: ok=False misdirected=4 moved=4 kept=0 marker=<absent> (withheld)
construction 2: ok=True misdirected=0 moved=0 kept=4 marker=1 (STAMPED)
history parked under unit/inverter-a/…; marker stamped after ONE restart.
```
On start 2 the same wrong call site finds the four keys it wrote already
present, so **rule 1** — never overwrite a namespaced value — routes every one
of them to `rep.kept`. `rep.moved` is empty, the roster-grading loop iterates
over nothing, `misdirected` is empty, `ok` is True, the marker is stamped, and
start 3 takes the `from_version >= MAINT_SCHEMA_VERSION` early return **forever
— including after the call site is fixed**. Loud-and-recoverable became
silent-and-permanent after one restart: exactly the state `store.py`'s header
says must never happen, reached by the fix written to prevent it.
**The remedy is one line**, and it is sound by construction rather than by
argument: grade `set(rep.moved) | set(rep.kept)`. Every entry in *both* lists is
a `unit_key(unit, legacy)` built from the **same** `unit` argument, so grading
`kept` grades the same namespace decision `moved` does. A correct call site's
`kept` entries are under a roster name and pass; a wrong one cannot launder its
namespace by having already written it. `migrate_unit_keys()` passes
`roster=None` and returns before the loop, so it is unaffected.
**Why the check missed it, which is the part worth keeping.** Part 9b's recovery
check was `migrate_maint_keys(_m5, ROSTER[0], ROSTER)` — the **corrected** name.
It measures recovery along the one path where the operator has *already fixed
the bug*. The real failure is a call site that is still wrong on the next start,
and nothing in the 19 files called `migrate_maint_keys()` twice with the **same
wrong** `unit`. The check was true, passed, and could not reach the failure its
sentence named — MUTATION.md's third defect, this time sitting inside the fix
written to close it. Part **9c** now drives two starts with the same wrong unit
over one store, asserts the marker is still withheld on the second, and asserts
the *shape* of the second report (`moved` empty, `kept` full) so it cannot
quietly revert to 9b's path.
### The marker gate is `rep.ok` and nothing weaker — both guards, each pinned
`if rep.ok:` is `not lost and not misdirected`, and until this round only the
`misdirected` half was reachable by a check that also had a non-empty `kept`.
The mutation
```
if rep.ok: -> if rep.ok or rep.kept:
```
stamps the marker on a migration that **lost a month of history** — defeating
the older, settled `lost` guard as well as the new one — and **all 19 suites
were green on it**. Part **9d** is the fixture that separates them: two keys
already namespaced (`kept` non-empty, so the mutant's `or` fires) and two keys
whose write the store drops (`lost` non-empty, so `ok` is False), with
`misdirected` empty throughout so the section cannot pass on the other guard by
accident. Measured differential on it: shipped → marker absent; mutant → marker
`1`, both reporting `ok=False lost=2 kept=2`.
**Why the fixtures had to change, and this is the part that got the card
rejected.** Nothing in the repository could see whether any of the above was
true, because `inverter.UNIT_A` **is** the string `"inverter-a"` and every
two-pack fixture in `test_maintenance.py` named its first roster entry
`inverter-a`. `self.inverters[0].name` and `UNIT_A` were therefore the same
value in every check that existed, and swapping one for the other at the call
site left **all 19 suites green**. MUTATION.md's third defect exactly — a check
whose *data* cannot reach the failure it names — and the fifth consecutive card
to carry it. Parts 9 and 11 now run on `["garage", "shed"]`, each behind a
**precondition check** asserting that the first name is not `UNIT_A`, so an edit
that makes them equal again goes red instead of making the claims vacuous.
### And the flat keys are kept LIVE, which is the half DEV-02 had to add later
Rule 2 originally said the flat keys are left in place and go stale. DEV-02
amended it, for the sign latch, to *"left in place is not enough"* — and the
same sentence holds for history. A fossil dated the day the second inverter was
declared would hand a rolled-back single-FSM add-on a `last_completed` months
old (it recalibrates a pack that does not need it) or a stale `phase` with a
`phase_started` old enough to abort on the 12 h timeout the moment it comes up.
So `UnitHistory` **mirrors the first pack's writes back onto the flat keys** for
as long as the roster has more than one entry. Only the first pack: the flat
spelling means `inverters[0]` on every install that has one, and a mirror from
pack B would overwrite pack A's rollback view with pack B's phase.
## 3. Staggering: the interlock defers, and the stagger is the residue
Two packs on one monthly schedule are due in the same hour every month. The
options were a stagger **by construction** — offset each pack's first cycle by
`interval_days / N` — or **deferral**. Deferral, for three reasons:
1. **A constructed stagger cannot remove the need for the interlock.** A forced
start, a commissioning cycle, an abort-and-retry, or a pack added mid-month
desynchronises any static offset immediately, so the interlock has to be
there anyway — and then the stagger is a second mechanism that must also be
correct, for a case the first one already handles.
2. **A deferral is observable and a stagger is not.** The deferred pack says, in
the action log and on the status page, that it is waiting, which pack it is
waiting for, and that it will run at the next start hour. A silent offset
moves an operator's configured start day and explains nothing.
3. **The stagger comes out of the deferral for free and then persists.** Pack A
completes on day 1 and is next due on day 29; pack B, deferred, completes on
day 2 and is next due on day 30. One contested month separates them by one
cycle length and they stay separated.
**Starvation was the risk and it is closed at two points.** A deferral does
**not** stamp `last_start_attempt` — that key is the post-abort cooldown, and at
90 minutes inside a 60-minute start hour, stamping it would lock the deferred
pack out of exactly the hour it is waiting for. And the winner is the
longest-waiting pack rather than the first in the roster, so a pack that keeps
aborting cannot monopolise the slot month after month while its sibling drifts —
which is this card's own defect wearing a different costume.
**The deferral must not read as a fault.** §12's 12 h and 36 h phase timeouts
mean *"the battery is not following commands"*. A deferred pack stays `IDLE`
with `phase_started` untouched, so no timeout is reachable by waiting, and the
line it logs says WAITING, names the pack it waits for and says it will run
next — none of which any failure path in `maintenance.py` says.
## 4. Rule 6 (the card calls it rule 8): the first entry's cap is no longer right
`validate.py`'s rule 6 comment said, in as many words, that
`maintenance_charge_w` was graded against `inverters[0]`'s cap because there was
exactly one `Maintenance` and it could only ever recalibrate the first pack —
and that taking `min()` would be an over-refusal, because it would refuse *"a
5000 W pack being recalibrated beside a 2000 W sibling the cycle never
touches"*. It closed: **"when `maint_unit` stops being `inverters[0]`, this line
moves with it."**
It has stopped. Every pack is recalibrated on its own schedule, so **the sibling
the cycle never touches does not exist** and the over-refusal argument has no
subject. The shaping claim is filed on whichever pack is charging, so the cap
that can contradict it is any cap on the roster and the one that contradicts it
first is the smallest.
**This refuses configurations that used to start**, and that is the point rather
than a side effect: `max_w` 5000 on pack A beside 2000 on pack B with
`maintenance_charge_w` 3000 was correct while only pack A was ever recalibrated,
and is now a cheap-window charge asking the arbiter for `hi = -3000` against a
supervised `lo = -2000` — `lo > hi`, arbiter clause 3, 0 W, on the pack nobody
was watching. Refused when it is **written**, which is rule 6's whole job. At a
roster of one `min()` over one cap is that cap: same comparison, same message,
same boundary.
## 5. The claim stays per-unit, and nothing here files a site claim
`arbiter.CLAIM_SCOPE` has classified `maintenance` as `SCOPE_UNIT` since it was
written. Two FSMs mean two claims on two actuators, which the allocator consumes
as per-unit intervals. [[ARCH-01]] is explicit that this must never become a
site claim, because per-inverter constraints filed site-wide are the
cross-inverter `lo > hi` contradiction at `arbiter.py:114`. The only line this
card changed there is **which** roster entry the claim is filed on: the one whose
state machine produced the result, rather than `inverters[0]`.
## 6. FIXED BY SAFETY-78: the site loop contradicted a drain at N>=2
Found here and pre-existing: below the pin the other pack had to charge, the
allocator refused it and the site sat at 0 W. Fixed by a site `drain` floor;
see [[SAFETY-78]] (`DECISIONS/SAFETY-78-drain-at-two.md`).
## 7. FOUND AND NOT FIXED: `migrate_unit_keys()` has the same hole, and it is a LATCH
Found while fixing §2's instance. It is the same defect one module over and it
costs a **disarmed safety guard** rather than a lost history.
`Controller.__init__` calls `migrate_unit_keys(store, UNIT_A)` — a hardcoded
constant where the maintenance migration now passes the roster's own first name.
It **has** to run before `build_roster()`, because each `Inverter` restores its
sign latch in its constructor, so there is no roster to name yet and
`_verify_moves()` is passed `roster=None`. On a **declared** roster a pre-DEV-02
`sign_tripped: true` therefore lands under `unit/inverter-a/sign_tripped`, which
no unit on that site reads.
**And the guard that exists for exactly this is defeated by the key the defect
wrote.** `_grade_store_migration()` re-arms the latch when the flat key says
tripped and *nothing namespaced is latched*. The misdirected write **is**
namespaced and **is** latched, so `namespaced_latched` is `True`, the guard
returns, and no inverter on the roster comes up tripped — on an install whose
`/data/state.json` still reads `sign_tripped: true`. That is DEV-02's own
sentence for the worst case: the state file says protected, the banner is green,
and the controller resumes commanding into the fault that tripped it.
Measured, on the same `["garage", "shed"]` roster:
```
pre-DEV-02 flat latch + declared roster ['garage', 'shed'] ->
migrated 2 flat key(s): unit/inverter-a/sign_reason, unit/inverter-a/sign_tripped
latch keys in the store: ['sign_tripped', 'site_sign_tripped', 'unit/inverter-a/sign_tripped']
roster came up tripped: [('garage', False), ('shed', False)]
the same store on the FLAT config -> [('inverter-a', True)]
```
The last line is the counterfactual: on the flat shape, where `UNIT_A` really is
the roster's only name, the same store migrates and comes up latched. So this is
a defect of the **declared roster**, not of the migration as such.
**Why it is not fixed here.** The remedy is to name the migration's target from
the config before any `Inverter` exists, which means factoring `build_roster()`'s
naming rule out of `build_roster()` so there is still only **one** spelling of
it — `inverter.py` and SAFETY-02's latch semantics, with its own mutation battery
over `test_inverter.py` and `test_watchdog.py`. MAINT-05 moves the maintenance
history and nothing about the sign latch, and a silent expansion into a safety
latch is how a rejected PR becomes a broken one.
It is pinned as a running check (`test_maintenance.py` part 13) rather than left
as a comment. ⚠️ **Its assertions state the WRONG behaviour on purpose** — read
them as the specification of a defect; the card that fixes it should expect them
to go red and should replace them.
### And its MARKER GATE had the round-3 hole too — found here, closed here
Separate from the namespace hole above, and **fixed** rather than recorded,
because closing it cost one test fixture and no source change.
`migrate_unit_keys()` gates `schema_version` on `rep.ok` exactly as
`migrate_maint_keys()` gates its own marker, and every check that reached that
gate ran on a store where **nothing** was namespaced yet — so `rep.kept` was
empty in all of them and
```
if rep.ok: -> if rep.ok or rep.kept:
```
**survived all 19 files**. What it does is stamp the schema marker on a
migration that DROPPED THE LATCH: the guard is off, the state file still reads
`sign_tripped: true`, and because the store now says the migration finished, the
next start skips the one thing that would have re-armed it. DEV-02's own worst
case, filed under *done*.
`test_inverter.py` part 10 now drops the `sign_tripped` write **beside an
already-namespaced `sign_reason`** — `lost` non-empty *and* `kept` non-empty,
which is the one shape none of the existing fixtures had — and asserts the
marker is withheld, with the accepted-baseline counterfactual beside it (habit
7). The shipped gate was correct throughout; what changed is that a check can
now see it.