Files
goodwe-addon/goodwe_controller/DECISIONS/MAINT-05-per-pack-maintenance.md
T
Claude 5d27f8496f
checks / unit (pull_request) Failing after 17m13s
infra-pin / fresh (pull_request) Successful in 2s
SAFETY-78: maintenance drain at N=2 no longer stalls
At two inverters the drain pins its pack and the site target is split
around the pin. Below the pin the other pack had to charge, the allocator
refused the opposite directions, and the site sat at 0 W with an ARBITER
line every cycle (house below or above the drain rate, or a grid charge).

main.py now files a site limit `drain` (lo = what the pin places, at
P_MAINTENANCE) at N>=2 while a pack drains, so the other pack only adds
power in the same direction. A drain that cannot give power waits with
one log line. With battery modes on, no grid charge while a pack drains,
and the other battery's card says why. N=1 is byte-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:52:51 +02:00

21 KiB

MAINT-05 — one recalibration per pack, one at a time, and what the card left open

Status: DECIDED 2026-09-04. There is one Maintenance per roster entry, each with its own schedule, phase and history. The site holds exactly one maintenance slot: at most one pack may be in a power-moving phase, the slot is derived from the packs' own stored phases rather than persisted separately, and when several packs are due in the same hour it goes to the one that has gone longest without a completed cycle. The schedule is not staggered by construction — the interlock defers, and the stagger emerges from it.

Fix round 2026-09-04. The store migration is now graded against the roster rather than against the unit name it was handed — see §2's "The read-back postcondition is graded against the ROSTER" — and the same hole in SAFETY-02's latch migration is recorded in §7.

Filed with MAINT-05. Card 6a9a9e8503872905f298d80a.


0. What the card asked to be decided, and the answers

the card's open question the decision
§2 the arbitration one pack at a time; the other stays under the loop. Nothing preempts a cycle in flight.
§2 who wins when several are due the longest-waiting pack, last_completed oldest first, inf for never-recalibrated. Roster order breaks a numeric tie and nothing else.
§4 staggering not by construction. The interlock defers the second pack; the stagger is what comes out of it. §3 below.
AC 7 rule 6's maint_unit comment no, the first entry's cap is no longer right. It grades against the weakest cap on the roster now, like rule 5. §4 below.
the store layout at N=1 flat, unchanged. The namespace appears the day the roster grows. §2 below.

1. Why the slot is derived and not persisted

The holder is "whichever pack's stored phase is not idle" — a fact already on disk, written by the packs themselves in the per-pack history this card introduces. A maint_slot_holder key would be a second opinion about the same thing, and the two would disagree the first time the add-on was killed between writing the phase and writing the holder. Derived, a controller that comes back up mid-drain reads the drain and re-derives the holder from it; there is no recovery path to get wrong because there is no second fact.

CHARGE is in POWER_MOVING even though the control loop owns the setpoint in that phase, and that is the decision most likely to be "simplified" later. Pack A charging from the grid at 2000 W while pack B drains 2000 W to export is 4 kW going round in a circle with the meter reading zero and both packs believing they are behaving — the circulating power SAFETY-31 exists to detect, arriving as something this module can simply not do. HOLD is in the tuple for the other half of the sentence: it moves nothing, and it owns the actuator, so a second cycle starting under it would still be two recalibrations at once.

2. Why the store layout at a roster of one does not change

The maintenance keys are flat — phase, phase_started, last_completed, last_start_attempt — on every install since the add-on shipped. At one pack there is nothing for a namespace to distinguish that pack from: §6.2's identity, the same one arbiter.actuator_for() leans on when it routes every claim to the site actuator at N=1. So Maintenance is constructed with unit=None there, reads and writes the flat keys directly, and migrate_maint_keys() returns without writing anything. A one-inverter state file comes out of this card byte-identical, which is AC 6 and which test_maintenance.py part 10 asserts by driving a whole cycle and comparing the key set.

The migration has its own marker, and that is not tidiness

SCHEMA_VERSION / schema_version is DEV-02's, and it is monotone: every install runs that migration exactly once. This one runs only where the roster has more than one pack, so a single counter could not express "migrated on this site, deliberately not migrated on that one" without lying to the next version that reads it — a one-inverter store would have to be pinned at the old number forever, and a genuine future v2 migration could then never be told apart from a site that simply never grew. maint_schema_version answers one question and answers it honestly.

The three rules are shared in code, not restated: _move_flat_keys() is rule 1 (never overwrite an existing namespaced value) and rule 2 (the flat keys stay in the file), _verify_moves() is the read-back postcondition, and the marker is written last. A second transcription of a migration is a second opinion about what "it landed" means.

The read-back postcondition is graded against the ROSTER (fix round)

A postcondition that reads back the namespace it was handed cannot detect being handed the wrong one. That sentence is the whole finding, and this card shipped it broken.

_verify_moves() iterated rep.moved — the keys it had just written — and asked whether each read back equal. On a store where the write lands, it always does. So a migration handed a unit that is on no roster wrote four keys, read four keys back, set verified = True, and the caller stamped maint_schema_version. Every symptom of a clean migration, and an install's whole maintenance history parked under a name nothing will ever look up. And because the marker was stamped, the next start skipped the migration too — which is precisely the state store.py's header says must never happen: a failed migration a later version treats as done.

Reproduced on the reviewer's roster, a house with a garage pack and a shed pack, carrying a flat last_completed of 2026-08-07:

unit the migration was handed key written pack 0 reads back due now?
correct self.inverters[0].name = garage unit/garage/last_completed 2026-08-07 False
wrong inverter.UNIT_A = inverter-a unit/inverter-a/last_completed None True

build_roster() names a declared roster's entries from the config, and its fallback for a nameless entry is a positional inverter-1 — so inverter-a is not the name of anything on a declared roster, not even by default.

The remedy, and why this one. migrate_maint_keys() now takes the roster in place of an n_units count (the count was len(self.inverters) at the only call site, so the roster carries it, and a count passed beside the names is a second opinion about how many packs there are). _verify_moves() grades the namespace it wrote against those names, and a unit that is not among them is reported as misdirected — a different failure from lost:

  • lost is the store took the write and dropped it — a storage fault, and it has a harmless-default case (a write of last_completed: None loses nothing, because "absent" and "never recalibrated" are the same statement).
  • misdirected is the store did exactly as it was told, by a caller that was wrong. There is no harmless-default version of it: a write nobody reads is not information, whatever its value. So anything in it fails ok on its own.

And ok gates the marker, so a misdirected migration is not stamped and the next start runs it again. The argument for it is asymmetric: a re-run is recoverable — rule 1 makes it a no-op where keys did land, and rule 2 left the flat keys in the file to recover from — while a silent skip is not recoverable at all, because the store now claims the migration happened.

⚠️ THE PARAGRAPH ABOVE USED TO END HERE, AND AS IT STOOD IT WAS FALSE

The refusal lasted exactly one start. The second start stamped the marker. That is a correction to what the first fix round wrote down, not a new qualification of it, and it is recorded here because a decision document that states a guarantee the code does not provide is worse than one that says nothing.

_verify_moves() graded only rep.moved. Measured at Controller level over one persisted store, roster ["garage", "shed"], flat last_completed 2026-08-07, with the call site still handing inverter-a on both starts:

construction 1: ok=False misdirected=4 moved=4 kept=0   marker=<absent>   (withheld)
construction 2: ok=True  misdirected=0 moved=0 kept=4   marker=1          (STAMPED)
history parked under unit/inverter-a/…; marker stamped after ONE restart.

On start 2 the same wrong call site finds the four keys it wrote already present, so rule 1 — never overwrite a namespaced value — routes every one of them to rep.kept. rep.moved is empty, the roster-grading loop iterates over nothing, misdirected is empty, ok is True, the marker is stamped, and start 3 takes the from_version >= MAINT_SCHEMA_VERSION early return forever — including after the call site is fixed. Loud-and-recoverable became silent-and-permanent after one restart: exactly the state store.py's header says must never happen, reached by the fix written to prevent it.

The remedy is one line, and it is sound by construction rather than by argument: grade set(rep.moved) | set(rep.kept). Every entry in both lists is a unit_key(unit, legacy) built from the same unit argument, so grading kept grades the same namespace decision moved does. A correct call site's kept entries are under a roster name and pass; a wrong one cannot launder its namespace by having already written it. migrate_unit_keys() passes roster=None and returns before the loop, so it is unaffected.

Why the check missed it, which is the part worth keeping. Part 9b's recovery check was migrate_maint_keys(_m5, ROSTER[0], ROSTER) — the corrected name. It measures recovery along the one path where the operator has already fixed the bug. The real failure is a call site that is still wrong on the next start, and nothing in the 19 files called migrate_maint_keys() twice with the same wrong unit. The check was true, passed, and could not reach the failure its sentence named — MUTATION.md's third defect, this time sitting inside the fix written to close it. Part 9c now drives two starts with the same wrong unit over one store, asserts the marker is still withheld on the second, and asserts the shape of the second report (moved empty, kept full) so it cannot quietly revert to 9b's path.

The marker gate is rep.ok and nothing weaker — both guards, each pinned

if rep.ok: is not lost and not misdirected, and until this round only the misdirected half was reachable by a check that also had a non-empty kept. The mutation

if rep.ok:      ->      if rep.ok or rep.kept:

stamps the marker on a migration that lost a month of history — defeating the older, settled lost guard as well as the new one — and all 19 suites were green on it. Part 9d is the fixture that separates them: two keys already namespaced (kept non-empty, so the mutant's or fires) and two keys whose write the store drops (lost non-empty, so ok is False), with misdirected empty throughout so the section cannot pass on the other guard by accident. Measured differential on it: shipped → marker absent; mutant → marker 1, both reporting ok=False lost=2 kept=2.

Why the fixtures had to change, and this is the part that got the card rejected. Nothing in the repository could see whether any of the above was true, because inverter.UNIT_A is the string "inverter-a" and every two-pack fixture in test_maintenance.py named its first roster entry inverter-a. self.inverters[0].name and UNIT_A were therefore the same value in every check that existed, and swapping one for the other at the call site left all 19 suites green. MUTATION.md's third defect exactly — a check whose data cannot reach the failure it names — and the fifth consecutive card to carry it. Parts 9 and 11 now run on ["garage", "shed"], each behind a precondition check asserting that the first name is not UNIT_A, so an edit that makes them equal again goes red instead of making the claims vacuous.

And the flat keys are kept LIVE, which is the half DEV-02 had to add later

Rule 2 originally said the flat keys are left in place and go stale. DEV-02 amended it, for the sign latch, to "left in place is not enough" — and the same sentence holds for history. A fossil dated the day the second inverter was declared would hand a rolled-back single-FSM add-on a last_completed months old (it recalibrates a pack that does not need it) or a stale phase with a phase_started old enough to abort on the 12 h timeout the moment it comes up. So UnitHistory mirrors the first pack's writes back onto the flat keys for as long as the roster has more than one entry. Only the first pack: the flat spelling means inverters[0] on every install that has one, and a mirror from pack B would overwrite pack A's rollback view with pack B's phase.

3. Staggering: the interlock defers, and the stagger is the residue

Two packs on one monthly schedule are due in the same hour every month. The options were a stagger by construction — offset each pack's first cycle by interval_days / N — or deferral. Deferral, for three reasons:

  1. A constructed stagger cannot remove the need for the interlock. A forced start, a commissioning cycle, an abort-and-retry, or a pack added mid-month desynchronises any static offset immediately, so the interlock has to be there anyway — and then the stagger is a second mechanism that must also be correct, for a case the first one already handles.
  2. A deferral is observable and a stagger is not. The deferred pack says, in the action log and on the status page, that it is waiting, which pack it is waiting for, and that it will run at the next start hour. A silent offset moves an operator's configured start day and explains nothing.
  3. The stagger comes out of the deferral for free and then persists. Pack A completes on day 1 and is next due on day 29; pack B, deferred, completes on day 2 and is next due on day 30. One contested month separates them by one cycle length and they stay separated.

Starvation was the risk and it is closed at two points. A deferral does not stamp last_start_attempt — that key is the post-abort cooldown, and at 90 minutes inside a 60-minute start hour, stamping it would lock the deferred pack out of exactly the hour it is waiting for. And the winner is the longest-waiting pack rather than the first in the roster, so a pack that keeps aborting cannot monopolise the slot month after month while its sibling drifts — which is this card's own defect wearing a different costume.

The deferral must not read as a fault. §12's 12 h and 36 h phase timeouts mean "the battery is not following commands". A deferred pack stays IDLE with phase_started untouched, so no timeout is reachable by waiting, and the line it logs says WAITING, names the pack it waits for and says it will run next — none of which any failure path in maintenance.py says.

4. Rule 6 (the card calls it rule 8): the first entry's cap is no longer right

validate.py's rule 6 comment said, in as many words, that maintenance_charge_w was graded against inverters[0]'s cap because there was exactly one Maintenance and it could only ever recalibrate the first pack — and that taking min() would be an over-refusal, because it would refuse "a 5000 W pack being recalibrated beside a 2000 W sibling the cycle never touches". It closed: "when maint_unit stops being inverters[0], this line moves with it."

It has stopped. Every pack is recalibrated on its own schedule, so the sibling the cycle never touches does not exist and the over-refusal argument has no subject. The shaping claim is filed on whichever pack is charging, so the cap that can contradict it is any cap on the roster and the one that contradicts it first is the smallest.

This refuses configurations that used to start, and that is the point rather than a side effect: max_w 5000 on pack A beside 2000 on pack B with maintenance_charge_w 3000 was correct while only pack A was ever recalibrated, and is now a cheap-window charge asking the arbiter for hi = -3000 against a supervised lo = -2000 — lo > hi, arbiter clause 3, 0 W, on the pack nobody was watching. Refused when it is written, which is rule 6's whole job. At a roster of one min() over one cap is that cap: same comparison, same message, same boundary.

5. The claim stays per-unit, and nothing here files a site claim

arbiter.CLAIM_SCOPE has classified maintenance as SCOPE_UNIT since it was written. Two FSMs mean two claims on two actuators, which the allocator consumes as per-unit intervals. ARCH-01 is explicit that this must never become a site claim, because per-inverter constraints filed site-wide are the cross-inverter lo > hi contradiction at arbiter.py:114. The only line this card changed there is which roster entry the claim is filed on: the one whose state machine produced the result, rather than inverters[0].

6. FIXED BY SAFETY-78: the site loop contradicted a drain at N>=2

Found here and pre-existing: below the pin the other pack had to charge, the allocator refused it and the site sat at 0 W. Fixed by a site drain floor; see SAFETY-78 (DECISIONS/SAFETY-78-drain-at-two.md).

7. FOUND AND NOT FIXED: migrate_unit_keys() has the same hole, and it is a LATCH

Found while fixing §2's instance. It is the same defect one module over and it costs a disarmed safety guard rather than a lost history.

Controller.__init__ calls migrate_unit_keys(store, UNIT_A) — a hardcoded constant where the maintenance migration now passes the roster's own first name. It has to run before build_roster(), because each Inverter restores its sign latch in its constructor, so there is no roster to name yet and _verify_moves() is passed roster=None. On a declared roster a pre-DEV-02 sign_tripped: true therefore lands under unit/inverter-a/sign_tripped, which no unit on that site reads.

And the guard that exists for exactly this is defeated by the key the defect wrote. _grade_store_migration() re-arms the latch when the flat key says tripped and nothing namespaced is latched. The misdirected write is namespaced and is latched, so namespaced_latched is True, the guard returns, and no inverter on the roster comes up tripped — on an install whose /data/state.json still reads sign_tripped: true. That is DEV-02's own sentence for the worst case: the state file says protected, the banner is green, and the controller resumes commanding into the fault that tripped it.

Measured, on the same ["garage", "shed"] roster:

pre-DEV-02 flat latch + declared roster ['garage', 'shed'] ->
    migrated 2 flat key(s): unit/inverter-a/sign_reason, unit/inverter-a/sign_tripped
latch keys in the store: ['sign_tripped', 'site_sign_tripped', 'unit/inverter-a/sign_tripped']
roster came up tripped:  [('garage', False), ('shed', False)]
the same store on the FLAT config -> [('inverter-a', True)]

The last line is the counterfactual: on the flat shape, where UNIT_A really is the roster's only name, the same store migrates and comes up latched. So this is a defect of the declared roster, not of the migration as such.

Why it is not fixed here. The remedy is to name the migration's target from the config before any Inverter exists, which means factoring build_roster()'s naming rule out of build_roster() so there is still only one spelling of it — inverter.py and SAFETY-02's latch semantics, with its own mutation battery over test_inverter.py and test_watchdog.py. MAINT-05 moves the maintenance history and nothing about the sign latch, and a silent expansion into a safety latch is how a rejected PR becomes a broken one.

It is pinned as a running check (test_maintenance.py part 13) rather than left as a comment. ⚠️ Its assertions state the WRONG behaviour on purpose — read them as the specification of a defect; the card that fixes it should expect them to go red and should replace them.

And its MARKER GATE had the round-3 hole too — found here, closed here

Separate from the namespace hole above, and fixed rather than recorded, because closing it cost one test fixture and no source change. migrate_unit_keys() gates schema_version on rep.ok exactly as migrate_maint_keys() gates its own marker, and every check that reached that gate ran on a store where nothing was namespaced yet — so rep.kept was empty in all of them and

if rep.ok:      ->      if rep.ok or rep.kept:

survived all 19 files. What it does is stamp the schema marker on a migration that DROPPED THE LATCH: the guard is off, the state file still reads sign_tripped: true, and because the store now says the migration finished, the next start skips the one thing that would have re-armed it. DEV-02's own worst case, filed under done.

test_inverter.py part 10 now drops the sign_tripped write beside an already-namespaced sign_reason — lost non-empty and kept non-empty, which is the one shape none of the existing fixtures had — and asserts the marker is withheld, with the accepted-baseline counterfactual beside it (habit 7). The shipped gate was correct throughout; what changed is that a check can now see it.