goodwe-addon/goodwe_controller/DECISIONS/SAFETY-10-shadow-mode.md

SAFETY-10 — shadow mode, and why "default-on" is not true

Status: decided and implemented on branch SAFETY-10. ⚠️ AMENDED BY SAFETY-47, and three of the claims below were false when this was merged. §5's "the strategies are unaffected" exemption, §4's peak-guard row and §6's evidence table are corrected in place and each correction is marked. The amendments are marked rather than rewritten silently, because what this card believed is part of why the defect shipped. See DECISIONS/SAFETY-47-what-a-shadow-run-can-show.md. Card: SAFETY-10 — "Shadow mode (default-on, log-only decisions)". Nothing here reached the inverter. No deployment, no ha addons update, no contact with 192.168.2.158 or 192.168.2.221. Everything below was measured in the CI image and on TEST-23 stage 3's closed-loop rig.

---

0. The four things a reader should take away

1. "Default-on" is delivered as auto, not as true, and the difference is an install that is regulating a real battery today. The Supervisor composes an add-on's options as its defaults, overlaid with the stored user options, so a new key with a new default reaches every existing install on the next update. A boolean shadow_mode: true would silently stop the reference site. §1.
2. Shadow mode WRITES. It writes 0 W, on the ordinary heartbeat, through the ordinary single writer. "Commands nothing" implemented as "stops writing" would leave the inverter latched on whatever was standing — it has no meter timeout — and would put the ESP32 into a tripped stale-input watchdog for the whole run. §2.
3. The sign watchdogs are inert, and that was the decision that had to be got right. Fed the shadow decision they would trip within sign_disagree_s and latch a trip that survives into live operation and a restart. §4.
4. The loop leg's shadow decision is one slew step, not a trajectory, and that is a real limitation of this feature. §5.
5. ⚠️ SAFETY-47: and neither is anything else that depends on the plant answering back. A shadow run cannot surface a closed-loop defect at all, by the same construction that makes it safe — the battery is held at 0 W, so every measured input describes a plant that did not respond. Of the three defects §"WHAT DEFECT THIS PREVENTS" was written for, exactly one — TARIFF-14's wrong quarter-hour — is visible from a shadow run. §5, §6.

---

1. The default

The literal reading, and what it costs

config.yaml's options: block is the default. There is exactly one of it, and HA's Supervisor resolves an add-on's effective options as

{**config.yaml options, **the stored user options}

A key that has never been stored therefore takes the file's value — on a fresh install and on an install that has been running for months, identically. There is no mechanism in the add-on options system by which one key can default differently for the two. So the card's title, taken at face value, is:

the next ha addons update stops the battery at the reference site, and the only symptom is a battery that has stopped doing anything.

That is strictly worse than every defect shadow mode exists to catch. It is refused.

What was shipped instead

shadow_mode is a three-valued list(auto|on|off) defaulting to "auto":

value	resolves to
off	live
on	shadow, always
auto	shadow while control is stopped; live once it is started

auto reads Controller.auto — the persisted start/stop state, which a fresh install takes from auto_start and which the Start / stop control button moves. No new persisted state, no migration, no guess:

• a fresh install ships auto_start: false, has no stored auto key, and therefore comes up in shadow mode. It observes a day of decisions before it is ever allowed to command anything. That is the card's intent, delivered where it is safe;
• an install that is controlling is started, so auto resolves to live and not one watt changes;
• an install that is stopped keeps its actuator at 0 W exactly as before — the same number, from the same writer, on the same heartbeat. What changes is what the status page and the action log say, and that the decision path now runs and is recorded.

⚠️ Under auto, pressing Start leaves shadow mode. That is the point of auto: the button means "control the battery". An operator who wants the decisions recorded while the loop runs — the configuration this card exists to make possible — sets on explicitly, and cannot reach it by accident.

⚠️ An unrecognised value resolves to SHADOW, and is REFUSED at startup. shadow.mode() fails toward inaction; validate.py rule 11 refuses the value by name so the reinterpretation is never silent. A typo is not a decision, and an install silently left in shadow mode commands nothing indefinitely while its status page calmly explains that this is intentional.

---

2. "Commands nothing" is a number, not a silence

The inverter holds its last command forever — HANDOVER.md §6 says so, and firmware/goodwe-master.yaml's header records what it cost when a controller stopped refreshing: "Illusmart went silent mid-command and the inverter held +5000 W for 113 s". firmware/goodwe-master.yaml states the contract that follows — "HA must now REWRITE the setpoint every ~10 s even when the value has not changed" — because without a refresh, "stale" and "steady" are the same thing on the wire.

So shadow mode holds the actuator at shadow.WIRE_W = 0.0 W, refreshed on the ordinary heartbeat. Two consequences were weighed and are recorded:

• it is not silence. A shadow mode that stopped writing would leave the standing command up until the ESP32's 30 s watchdog noticed — and at a site with no such board, forever;
• it does not use an alarm as a mode. Letting that watchdog trip for the duration of a shadow run teaches the operator that wd_tripped is normal, which is exactly the training that makes it useless when the add-on has actually died.

It is not a second writer

SAFETY-28's rule is one actuator, one writer. Shadow mode files no claim, owns no transport, and calls nothing in hass.py. It is a substitution inside Controller._write_unit() — the single choke point every write in this add-on already reaches, including the heartbeat, the stop button, shutdown() and maint_abort() — in exactly the form of the three substitutions above it (non-finite → 0 W, unauthorised → 0 W, clamped → the rating).

It is the last of them. Below SAFETY-28's gate and below SAFETY-03's clamp, so both still grade the number the decision path actually produced, and a shadow run can report "it would have hit the rating here".

---

3. What the run records

surface	what it carries
status banner	SHADOW MODE - the battery is NOT being controlled..., first in the cascade, amber — red, with the latch named, if one is standing
status row Shadow mode	which rung decided, the watts it would have commanded on this cycle, where the inverter actually is, and Resolution.explain() — plus, when a measured billed quarter sized those watts, SAFETY-47's caveat saying so (shadow.SIZED_FROM_METER)
status row Commanded	0 W — the truth, which the row above reconciles with the Why row below it
action log	one line when shadow mode comes on; one line per change of decision (winner, rounded watts, or SAFETY-47's caveat state), carrying the arbiter's own sentence and the same caveat the row carries — shadow.log_line() is the one sentence both read
MQTT	status: shadow, its own value, below failsafe

The action log is edge-logged, not level-logged: the log holds 40 rows, and a line per cycle would flush the context of the run's own findings inside four minutes at the 15 s cycle floor.

---

4. The failsafes, one by one

failsafe	in shadow mode	why
SAFETY-02/30 sign watchdogs	INERT	They grade Inverter.standing_w() — what this controller has standing on the wire — and shadow mode holds that at 0 W, so they cannot arm. ⚠️ Fed the shadow decision instead they would disagree with a meter that never saw it and latch a trip that survives into live operation and a restart: shadow mode would manufacture the failsafe it exists to help debug. ⚠️ Two rungs hold it there — last_write (the wire gate) and inv.target (the re-seed) — which is MUTATION.md's masking shape, so the mutation that reproduces the hazard defeats both as a pair (M6). Driven singly, dropping the re-seed does not reproduce it at all — measured
SAFETY-03 clamp	OBSERVES	Runs on the decision, sets clamp_last, logs every engagement. Changes no command
SAFETY-28 write gate	OBSERVES	Same placement, same reason: the question is who chose the number
SAFETY-18 meter freeze	OBSERVES, unchanged	It takes no action live either; both halves of its conjunction are as true in shadow
SAFETY-07/24/25 peak guard	OBSERVES, AND SEES ONLY THE HOUSE	Level-triggered, unlatched, files its claim as usual, so the recorded decision is the real one for this meter. ⚠️ AMENDED BY SAFETY-47. This row used to end "SAFETY-45's limit cycle is visible from a shadow run, which is the point". It is not: every term of wanted = batt_w + excess is measured, and in shadow mode the battery contributes nothing to either. Measured on the rig, same hour, same unsized code — 410 of 720 cycles engaged live, 0 of 720 in shadow
SAFETY-31 circulation	OBSERVES AND MAY LATCH	Its evidence is measured power. Packs pushing against each other while all are told 0 W is a real fault
SAFETY-42 delivery	OBSERVES AND MAY LATCH	Same, against a share that is 0 W
operator stop	SUPPRESSED AS A CLAIM, ENFORCED AT THE WIRE	The one deliberate change. Its P_SAFETY 0 W claim out-ranks everything, so leaving it in makes the recorded decision "stopped" on every cycle and the run answers the wrong question. The actuator ends at the same number either way, and the wire substitution is strictly stronger: a claim can be out-ranked, and it cannot
inputs missing, non-finite, allocator refusal, arbiter contradiction	UNCHANGED	All resolve to 0 W, which is what the wire gets anyway

⚠️ A latch that is already up stays up. Nothing here clears one. The shadow banner names any standing latch and goes red for it, because a latch that is up while the controller is inert is up the moment it stops being inert.

⚠️ Maintenance is not in that table because shadow mode does not touch it. The state machines are ticked on every cycle whether control is started or not, so a stopped install already runs them against a battery that does not move. Shadow mode inherits that, unchanged and unimproved. Recorded here rather than quietly fixed under another card's name.

---

5. The limitation, stated rather than discovered later

Controller._shadow_settle() re-seeds the carried command (self.target and each inv.target) to WIRE_W at the end of every shadow cycle — after every in-cycle guard has read the decision. Three machines read that value as a statement about the hardware: the loop's prev_w and integrator, standing_w(), and SAFETY-42's share comparison. Re-seeded, they track reality, and the exit is safe by construction — the carried command already equals the standing one, so the first live cycle slews from 0 W like any other.

⚠️ The cost is real. The loop's prev_w is therefore 0 W on every shadow cycle, so the loop leg's recorded decision is one slew step from the standing command and stays there. A shadow run answers "what would this controller command NOW, from where the plant really is" — a true statement about the present — and not "what trajectory would it have followed". The trajectory is a closed-loop property, it did not happen, and reporting it would be a simulation of a plant that does not exist; three cycles in, such a number describes nothing.

⚠️ AMENDED BY SAFETY-47 — THE PARAGRAPH THAT STOOD HERE WAS THE DEFECT. It read:

The strategies are unaffected: arbitrage, the peak guard and maintenance all file SET claims at a fixed power from level-triggered inputs. That is why the three defects this card was written for — SAFETY-45's 28 onsets an hour, TARIFF-14's wrong quarter-hour, SAFETY-46's ceiling churn — are all visible from a shadow run, and the loop's ramp is not.

It was true until SAFETY-45 merged, one commit before this card. That change makes the buy leg min(asked, headroom) with

headroom = (900 / remaining) x (ceiling - projected)

and projected reads the open quarter's accumulator — the meter — which is a function of what the battery did earlier in that same quarter. The strategy is closed-loop, and the loop's one-slew-step limitation above is not a special case of anything: it is the general case, and the limitation is the whole feature's, not the loop leg's.

The correct statement, and it is now graded rather than asserted:

A shadow run cannot surface a closed-loop defect — one whose stimulus is the plant's response to the controller — because the battery is held at 0 W and there is no response to be the stimulus. Which hour a strategy picks and why are unaffected, because the clock and the price series are not functions of the battery. How hard it trades is not.

Of the three defects this card was written for:

	visible from a shadow run?	measured
SAFETY-45's limit cycle	no	28 charge onsets live, 1 in shadow; grid −455.5 … +3200.0 W live, flat at the 1200 W house in shadow; guard engaged 410/720 live, 0/720 in shadow
SAFETY-46's SoC churn	no	the SoC moves 0.000000000 pp in a shadow run. No SoC limit can be reached in one
TARIFF-14's wrong quarter-hour	yes, and it is the one	which hour, at what price, clearing what spread: identical live and shadow on 720 of 720 charging cycles, at a binding cap

Graded in test_arbitrage.py part 3d — the same closed loop part 3c uses, run once live and once in shadow — and the loop leg's own limitation stays named in test_watchdog.py part 10 as RECORDED LIMITATION.

---

6. How "nothing reaches the inverter" is graded

Not on a flag. A check that asserted Controller.shadow would pass on a build in which the flag is set and the substitution is deleted — which is exactly mutation M1.

test_clamp.py part 8 drives the shipped Controller.cycle() and the shipped heartbeat against a recording Home Assistant stub, and asserts on what reached set_number():

• the precondition first — the same stimulus with shadow_mode: off puts 30 non-zero writes on the wire, the largest 5000 W. MUTATION.md habit 4: a check whose data cannot reach the failure it names is decoration;
• then set(writes) == {0.0}, and that there were 26 of them — a check that only said "no non-zero write" would pass on the silent build too;
• entry and exit are measured on inv.last_write — what the inverter is latched on, updated only on an accepted write — either side of the boundary, against the same run that never entered shadow mode.

The closed-loop half is /home/homelab/s10-mut/shadowloop.py on TEST-23 stage 3's rig: the same hour, twice, against plantsim.Plant through the firmware's own frame encoder.

⚠️ SUPERSEDED BY SAFETY-47 — THE TABLE THAT STOOD HERE COULD NEVER HAVE GONE RED, FOR TWO INDEPENDENT REASONS. It read arb_power_w −2000 W "identical", arb_reason "identical, verbatim", arbiter winner "identical", and concluded "the decision happened, at the same instant, for the same reason, at the same signed power — and the battery did not move." The battery half is true and is unchanged. The decision half was decoration:

1. the fixture is peak_cap_w: 15000 against a 350 W house, so the capacity guard never engages and SAFETY-45's sizing is in the run and inert. The harness's own comment says so — "at this scenario's peak_cap_w: 15000 it is not binding anyway" — which is MUTATION.md habit 4 written down beside the check it disables;
2. every identity row reads the FIRST charging sample. At elapsed = 0 the projection is the held level, so the two runs agree to the float there even at a cap that does bind. Measured: closing reason (1) alone leaves that comparison passing.

Both had to be closed. Re-taken below at peak_cap_w 1500 / house 1200 / max_w 2000, one hour at a 5 s cycle, across the whole run — harness /home/homelab/s47-mut/shadowloop47.py, same rig, same three transcribed lines.

	live	shadow
grid	1224.1 … 1506.0 W — held up at the ceiling	1200.0 W flat = the house, exactly
SoC	50.000019 → 51.856789 %	50.000000 → 50.000000 %
ARBITRAGE_CHARGE cycles	720 / 720	720 / 720
first charging cycle, arb_power_w	−300.0000 W	identical — and this is why the old table passed
arb_power_w, whole run	−307.799 … −299.930 W	−2000.000 … −300.000 W
…cycles where the two differ	—	716 of 720
arb_reason, whole run	the strategy's own sentence	differs on 719 of 720
measured charging headroom	299.93 … 307.80 W	300.0 … 54 000.0 W
final recorded command	−302.4 W	−2000.0 W — 6.61× the live figure
price / spread / which hour		identical on 720 of 720
arbiter winner	arbitrage	identical

What the re-taken table says. The decision still happens, at the same instant, on the same hour, at the same price, for the same kind of reason, and the battery still does not move — those were the card's real claims and they hold. What does not hold is that the decision is the same decision: its size is a function of a meter the battery never moved, and at this cap the recorded figure is 6.61× what a live controller commands. 54 000 W of "headroom" under a 1500 W ceiling is 900/remaining multiplying a gap the battery never closed.

⚠️ And the SoC row is the second half of it. 0.000000000 pp of movement is not a curiosity: it is the proof that no SoC-limit behaviour is reachable from a shadow run at all, which §5 used to promise.

SAFETY-47 puts the caveat on both surfaces that carry the number and narrows the three documents' claims; test_arbitrage.py part 3d is the CI-side grader so this class cannot recur.

⚠️ Three lines of each harness are a TRANSCRIPTION and are marked as one. priceloop.Rig has no _write_unit(), no clamp and no failsafe machine (divergence 2 of its README), so the substitution's placement is transcribed there. The shipped placement is graded by test_clamp.py part 8, which drives the real Controller. ⚠️ And the rig's transcribed cycle predates SAFETY-45: arbitrage.decide() gained a required charge_headroom_w after 478ba93, and the harness supplies it from the shipped peakguard.charge_headroom_w() reading the shipped peak decision rather than from a constant, so SAFETY-45 stays in the run.

---

7. The mutation battery — 22 mutations, no survivors

Harness: /home/homelab/s10-mut/run.py, backups beside it. Each run restores from backup, purges __pycache__ on the host, and grades through the CI image with the tree bind-mounted :ro, so the interpreter cannot write a cache into the tree under test (TEST-15). The full table is in the branch report; one per new behaviour, including M1 (the substitution deleted — a write escapes), M2 (it returns instead of substituting — the inverter is left latched), M3 (it moves above SAFETY-28 and SAFETY-03 — the observers go blind), M8 (auto resolves to shadow unconditionally — the literal default-on that stops the live install), M12 (the stop claim is filed in shadow — every recorded decision reads stopped), M21 (the substitution and the re-seed together) and M22 (the re-seed moved one line earlier — SAFETY-18 grades a 0 W no decision produced and goes permanently blind).

⚠️ Two findings from driving it, both of the kind MUTATION.md says to expect.

• M2 first came back a SURVIVOR and was not one. The "stop writing altogether" build left inv.last_write as None, the check did float() on it, and the section died with a TypeError — grading none of the checks after it and reporting zero reds, which reads exactly like a survivor. That is FCST-06's shape. The fixture now hands None back and names it as its own check; M2 is red on three.
• M6 had to become a PAIR, and only the battery said so. standing_w() prefers last_write — the 0.0 W the wire gate accepted — and falls back to inv.target, which the re-seed also holds at 0.0. Either rung alone keeps the detector inert, so dropping the re-seed singly came back red on the limitation check and on nothing about the latch: the hazard it names was not reproduced. TEST-16's rule with a new name on it.
• M7/M8/M9/M19 had to MOVE SUITES. They defeat shadow.engaged(), and a defeated default stops test_clamp.py part 3's wired run reaching the wire at all — aborting the file with an IndexError before part 8 is graded. The red count there was a lower bound naming the saturation freeze. The mode table now lives in test_validate.py, which drives no Controller.
• shadow_would_w = float(self.target) is an EQUIVALENT mutation and was measured as one. _shadow_settle() runs before the re-seed, so at that line self.target is resolution.target_w. It is recorded as equivalent rather than reported as a survivor, and the non-equivalent form of the same defect — the record becoming the wire — is driven instead.

The table

Each row: the mutation, the file, the byte delta, the suite it was graded in, and how many named checks went red. ⚠️ marks a row whose run did not reach the end of its file, so its count is a lower bound (MUTATION.md).

#	mutation	file	Δ	suite	reds
M1	a write escapes — the substitution deleted	main.py	−6 B	clamp	5
M2	silence instead of 0 W — it returns rather than substituting	main.py	−15 B	clamp	3
M3	the observers go blind — it moves above SAFETY-28 and SAFETY-03	main.py	+58 B	clamp	1
M4	the site re-seed dropped — the exit resumes from a fiction	main.py	−36 B	clamp	3
M5	the per-unit re-seed dropped	main.py	−22 B	clamp	1
M6	PAIRED: standing_w() reads the target and the re-seed dropped — the sign latch that survives into live operation	inverter.py + main.py	−170 B	watchdog	3
M7	auto inverted — shadow while started, live while stopped	shadow.py	+2 B	validate	4
M8	default-on for everyone — auto always shadow; the literal reading that stops the live install	shadow.py	−15 B	validate	2
M9	an unreadable value resolves to LIVE instead of shadow	shadow.py	+1 B	validate	1
M10	validate.py stops refusing an unknown value	shadow.py	−9 B	validate	9
M11	the loop leg gated on auto alone again — the decision path stops running	main.py	−17 B	clamp	1
M12	the stop claim filed in shadow — every decision reads stopped	main.py	−20 B	clamp	1
M13	the banner goes green while nothing is commanded	shadow.py	−2 B	clamp	1
M14	the banner loses the standing latch	shadow.py	−2 B	clamp	1
M15	the row renders the standing command instead of the decision	shadow.py	−8 B	clamp	1
M16 ⚠️	the row disappears when live	main.py	+43 B	clamp	1
M17	the record becomes the wire	main.py	−6 B	clamp	7
M18	level-logged — the 40-row action log floods	main.py	−23 B	clamp	1
M19	the two rungs render identically (masking)	shadow.py	+22 B	validate	1
M20	the shipped default moves to off in config.yaml alone	config.yaml	−1 B	defaults_pin	1
M21	PAIRED: the substitution and the re-seed deleted together	main.py	−103 B	clamp	10
M22	the re-seed moved one line earlier — SAFETY-18 grades a 0 W no decision produced and goes permanently blind	main.py	+40 B	clamp	1

Survivors: none. ⚠️ M17's +0 B-adjacent sibling — shadow_would_w = float(self.target) — is recorded in §7 as equivalent rather than counted here.