goodwe-addon/goodwe_controller/DECISIONS/SAFETY-10-shadow-mode.md

# SAFETY-10 — shadow mode, and why "default-on" is not `true`

**Status:** decided and implemented on branch `SAFETY-10`.
**Card:** SAFETY-10 — *"Shadow mode (default-on, log-only decisions)"*.
**Nothing here reached the inverter.** No deployment, no `ha addons update`, no
contact with 192.168.2.158 or 192.168.2.221. Everything below was measured in
the CI image and on TEST-23 stage 3's closed-loop rig.

---

## 0. The four things a reader should take away

1. **"Default-on" is delivered as `auto`, not as `true`, and the difference is
   an install that is regulating a real battery today.** The Supervisor
   composes an add-on's options as *its defaults, overlaid with the stored user
   options*, so a new key with a new default reaches **every existing install**
   on the next update. A boolean `shadow_mode: true` would silently stop the
   reference site. §1.
2. **Shadow mode WRITES. It writes 0 W, on the ordinary heartbeat, through the
   ordinary single writer.** "Commands nothing" implemented as "stops writing"
   would leave the inverter latched on whatever was standing — it has no meter
   timeout — and would put the ESP32 into a tripped stale-input watchdog for
   the whole run. §2.
3. **The sign watchdogs are inert, and that was the decision that had to be got
   right.** Fed the shadow decision they would trip within `sign_disagree_s`
   and **latch a trip that survives into live operation and a restart**. §4.
4. **The loop leg's shadow decision is one slew step, not a trajectory, and
   that is a real limitation of this feature.** §5.

---

## 1. The default

### The literal reading, and what it costs

`config.yaml`'s `options:` block **is** the default. There is exactly one of it,
and HA's Supervisor resolves an add-on's effective options as

```
{**config.yaml options, **the stored user options}
```

A key that has never been stored therefore takes the file's value — on a fresh
install and on an install that has been running for months, identically. There
is **no mechanism in the add-on options system** by which one key can default
differently for the two. So the card's title, taken at face value, is:

> the next `ha addons update` stops the battery at the reference site, and the
> only symptom is a battery that has stopped doing anything.

That is strictly worse than every defect shadow mode exists to catch. It is
refused.

### What was shipped instead

`shadow_mode` is a three-valued `list(auto|on|off)` defaulting to `"auto"`:

| value | resolves to |
|---|---|
| `off` | live |
| `on` | shadow, always |
| `auto` | shadow **while control is stopped**; live once it is started |

`auto` reads `Controller.auto` — the persisted start/stop state, which a fresh
install takes from `auto_start` and which the *Start / stop control* button
moves. No new persisted state, no migration, no guess:

* a **fresh** install ships `auto_start: false`, has no stored `auto` key, and
  therefore comes up **in shadow mode**. It observes a day of decisions before
  it is ever allowed to command anything. That is the card's intent, delivered
  where it is safe;
* an install that is **controlling** is started, so `auto` resolves to live and
  **not one watt changes**;
* an install that is **stopped** keeps its actuator at 0 W exactly as before —
  the same number, from the same writer, on the same heartbeat. What changes is
  what the status page and the action log **say**, and that the decision path
  now runs and is recorded.

⚠️ **Under `auto`, pressing Start leaves shadow mode.** That is the point of
`auto`: the button means "control the battery". An operator who wants the
decisions recorded *while the loop runs* — the configuration this card exists
to make possible — sets `on` explicitly, and cannot reach it by accident.

⚠️ **An unrecognised value resolves to SHADOW, and is REFUSED at startup.**
`shadow.mode()` fails toward inaction; `validate.py` rule 11 refuses the value
by name so the reinterpretation is never silent. A typo is not a decision, and
an install silently left in shadow mode commands nothing indefinitely while its
status page calmly explains that this is intentional.

---

## 2. "Commands nothing" is a number, not a silence

The inverter holds its last command **forever** — `HANDOVER.md` §6 says so, and
`firmware/goodwe-master.yaml`'s header records what it cost when a controller
stopped refreshing: *"Illusmart went silent mid-command and the inverter held
+5000 W for 113 s"*. `firmware/goodwe-master.yaml` states the contract that follows — *"HA
must now REWRITE the setpoint every ~10 s even when the value has not
changed"* — because without a refresh, **"stale" and "steady" are the same
thing on the wire**.

So shadow mode holds the actuator at `shadow.WIRE_W` = **0.0 W**, refreshed on
the ordinary heartbeat. Two consequences were weighed and are recorded:

* **it is not silence.** A shadow mode that stopped writing would leave the
  standing command up until the ESP32's 30 s watchdog noticed — and at a site
  with no such board, forever;
* **it does not use an alarm as a mode.** Letting that watchdog trip for the
  duration of a shadow run teaches the operator that `wd_tripped` is normal,
  which is exactly the training that makes it useless when the add-on has
  actually died.

### It is not a second writer

SAFETY-28's rule is one actuator, one writer. Shadow mode files **no claim**,
owns **no transport**, and calls nothing in `hass.py`. It is a **substitution**
inside `Controller._write_unit()` — the single choke point every write in this
add-on already reaches, including the heartbeat, the stop button, `shutdown()`
and `maint_abort()` — in exactly the form of the three substitutions above it
(non-finite → 0 W, unauthorised → 0 W, clamped → the rating).

It is the **last** of them. Below SAFETY-28's gate and below SAFETY-03's clamp,
so both still grade the number the decision path actually produced, and a
shadow run can report *"it would have hit the rating here"*.

---

## 3. What the run records

| surface | what it carries |
|---|---|
| status banner | `SHADOW MODE - the battery is NOT being controlled...`, **first in the cascade**, amber — red, with the latch named, if one is standing |
| status row `Shadow mode` | which rung decided, the watts it **would have commanded**, where the inverter actually is, and `Resolution.explain()` |
| status row `Commanded` | `0 W` — the truth, which the row above reconciles with the `Why` row below it |
| action log | one line when shadow mode comes on; one line per **change of decision** (winner or rounded watts), carrying the arbiter's own sentence |
| MQTT | `status: shadow`, its own value, below `failsafe` |

The action log is **edge**-logged, not level-logged: the log holds 40 rows, and
a line per cycle would flush the context of the run's own findings inside four
minutes at the 15 s cycle floor.

---

## 4. The failsafes, one by one

| failsafe | in shadow mode | why |
|---|---|---|
| SAFETY-02/30 sign watchdogs | **INERT** | They grade `Inverter.standing_w()` — what this controller has standing on the wire — and shadow mode holds that at 0 W, so they cannot arm. ⚠️ Fed the shadow decision instead they would disagree with a meter that never saw it and **latch a trip that survives into live operation and a restart**: shadow mode would manufacture the failsafe it exists to help debug. ⚠️ **Two rungs hold it there** — `last_write` (the wire gate) and `inv.target` (the re-seed) — which is MUTATION.md's masking shape, so the mutation that reproduces the hazard defeats both as a pair (M6). Driven singly, dropping the re-seed does not reproduce it at all — measured |
| SAFETY-03 clamp | **OBSERVES** | Runs on the decision, sets `clamp_last`, logs every engagement. Changes no command |
| SAFETY-28 write gate | **OBSERVES** | Same placement, same reason: the question is who chose the number |
| SAFETY-18 meter freeze | **OBSERVES**, unchanged | It takes no action live either; both halves of its conjunction are as true in shadow |
| SAFETY-07/24/25 peak guard | **OBSERVES** | Level-triggered, unlatched, files its claim as usual — SAFETY-45's limit cycle is visible from a shadow run, which is the point |
| SAFETY-31 circulation | **OBSERVES AND MAY LATCH** | Its evidence is *measured* power. Packs pushing against each other while all are told 0 W is a real fault |
| SAFETY-42 delivery | **OBSERVES AND MAY LATCH** | Same, against a share that is 0 W |
| operator stop | **SUPPRESSED AS A CLAIM, ENFORCED AT THE WIRE** | The one deliberate change. Its `P_SAFETY` 0 W claim out-ranks everything, so leaving it in makes the recorded decision `"stopped"` on every cycle and the run answers the wrong question. The actuator ends at the same number either way, and the wire substitution is **strictly stronger**: a claim can be out-ranked, and it cannot |
| inputs missing, non-finite, allocator refusal, arbiter contradiction | **UNCHANGED** | All resolve to 0 W, which is what the wire gets anyway |

⚠️ **A latch that is already up stays up.** Nothing here clears one. The shadow
banner **names** any standing latch and goes red for it, because a latch that is
up while the controller is inert is up the moment it stops being inert.

⚠️ **Maintenance is not in that table because shadow mode does not touch it.**
The state machines are ticked on every cycle whether control is started or not,
so a stopped install already runs them against a battery that does not move.
Shadow mode inherits that, unchanged and unimproved. Recorded here rather than
quietly fixed under another card's name.

---

## 5. The limitation, stated rather than discovered later

`Controller._shadow_settle()` re-seeds the carried command (`self.target` and
each `inv.target`) to `WIRE_W` at the **end** of every shadow cycle — after
every in-cycle guard has read the decision. Three machines read that value as a
statement about the **hardware**: the loop's `prev_w` and integrator,
`standing_w()`, and SAFETY-42's share comparison. Re-seeded, they track reality,
and **the exit is safe by construction** — the carried command already equals
the standing one, so the first live cycle slews from 0 W like any other.

⚠️ **The cost is real.** The loop's `prev_w` is therefore 0 W on every shadow
cycle, so the loop leg's recorded decision is **one slew step from the standing
command** and stays there. A shadow run answers *"what would this controller
command NOW, from where the plant really is"* — a true statement about the
present — and **not** *"what trajectory would it have followed"*. The trajectory
is a closed-loop property, it did not happen, and reporting it would be a
simulation of a plant that does not exist; three cycles in, such a number
describes nothing.

The strategies are unaffected: arbitrage, the peak guard and maintenance all
file `SET` claims at a fixed power from level-triggered inputs. That is why the
three defects this card was written for — SAFETY-45's 28 onsets an hour,
TARIFF-14's wrong quarter-hour, SAFETY-46's ceiling churn — are all visible from
a shadow run, and the loop's ramp is not. Graded, named, in
`test_watchdog.py` part 10 as `RECORDED LIMITATION`.

---

## 6. How "nothing reaches the inverter" is graded

**Not on a flag.** A check that asserted `Controller.shadow` would pass on a
build in which the flag is set and the substitution is deleted — which is
exactly mutation **M1**.

`test_clamp.py` part 8 drives the **shipped** `Controller.cycle()` and the
shipped heartbeat against a recording Home Assistant stub, and asserts on what
reached `set_number()`:

* **the precondition first** — the same stimulus with `shadow_mode: off` puts
  30 non-zero writes on the wire, the largest 5000 W. MUTATION.md habit 4: a
  check whose data cannot reach the failure it names is decoration;
* then `set(writes) == {0.0}`, **and** that there were 26 of them — a check
  that only said "no non-zero write" would pass on the silent build too;
* entry and exit are measured on `inv.last_write` — what the inverter is
  latched on, updated only on an **accepted** write — either side of the
  boundary, against the same run that never entered shadow mode.

The closed-loop half is `/home/homelab/s10-mut/shadowloop.py` on TEST-23 stage
3's rig: the same hour, twice, against `plantsim.Plant` through the firmware's
own frame encoder.

| | live | shadow |
|---|---|---|
| `plant_cmd_w` | −2000 W throughout | **0 W throughout** |
| `batt_w` | −2000 .. −40.6 W | **0.0 W, exactly** |
| SoC | 50.000 → **62.331 %** | 50.0 → **50.0 %** |
| grid | 390.6 .. 2350 W | **350 W = the house, exactly** |
| `ARBITRAGE_CHARGE` samples | 3600 | **3600** |
| first charge instant | 2026-01-15T00:00:00+00:00 | **identical** |
| `arb_power_w` | −2000 W | **identical** |
| `arb_reason` | the strategy's own sentence | **identical, verbatim** |
| arbiter winner | `arbitrage` | **identical** |

The decision happened, at the same instant, for the same reason, at the same
signed power — and the battery did not move.

⚠️ **Three lines of that harness are a TRANSCRIPTION and are marked as one.**
`priceloop.Rig` has no `_write_unit()`, no clamp and no failsafe machine
(divergence 2 of its README), so the substitution's **placement** is
transcribed there. The shipped placement is graded by `test_clamp.py` part 8,
which drives the real `Controller`. ⚠️ **And the rig's transcribed cycle
predates SAFETY-45**: `arbitrage.decide()` gained a required
`charge_headroom_w` after `478ba93`, and the harness supplies it from the
**shipped** `peakguard.charge_headroom_w()` reading the **shipped** peak
decision rather than from a constant, so SAFETY-45 stays in the run.

---

## 7. The mutation battery — 22 mutations, no survivors

Harness: `/home/homelab/s10-mut/run.py`, backups beside it. Each run restores
from backup, purges `__pycache__` **on the host**, and grades through the CI
image with the tree bind-mounted `:ro`, so the interpreter cannot write a cache
into the tree under test (TEST-15). The full table is in the branch report;
one per new behaviour, including **M1** (the substitution deleted — a write
escapes), **M2** (it returns instead of substituting — the inverter is left
latched), **M3** (it moves above SAFETY-28 and SAFETY-03 — the observers go
blind), **M8** (`auto` resolves to shadow unconditionally — the literal
default-on that stops the live install), **M12** (the stop claim is filed in
shadow — every recorded decision reads `stopped`), **M21** (the substitution
and the re-seed together) and **M22** (the re-seed moved one line earlier —
SAFETY-18 grades a 0 W no decision produced and goes permanently blind).

⚠️ **Two findings from driving it, both of the kind MUTATION.md says to expect.**

* **M2 first came back a SURVIVOR and was not one.** The "stop writing
  altogether" build left `inv.last_write` as `None`, the check did `float()` on
  it, and the section died with a `TypeError` — grading none of the checks
  after it and reporting **zero reds**, which reads exactly like a survivor.
  That is FCST-06's shape. The fixture now hands `None` back and names it as
  its own check; M2 is red on three.
* **M6 had to become a PAIR, and only the battery said so.** `standing_w()`
  prefers `last_write` — the 0.0 W the wire gate accepted — and falls back to
  `inv.target`, which the re-seed also holds at 0.0. Either rung alone keeps
  the detector inert, so dropping the re-seed **singly** came back red on the
  limitation check and on **nothing about the latch**: the hazard it names was
  not reproduced. TEST-16's rule with a new name on it.
* **M7/M8/M9/M19 had to MOVE SUITES.** They defeat `shadow.engaged()`, and a
  defeated default stops `test_clamp.py` part 3's wired run reaching the wire
  at all — aborting the file with an `IndexError` before part 8 is graded. The
  red count there was a lower bound naming the saturation freeze. The mode
  table now lives in `test_validate.py`, which drives no Controller.
* **`shadow_would_w = float(self.target)` is an EQUIVALENT mutation and was
  measured as one.** `_shadow_settle()` runs before the re-seed, so at that
  line `self.target` **is** `resolution.target_w`. It is recorded as equivalent
  rather than reported as a survivor, and the non-equivalent form of the same
  defect — the record becoming the **wire** — is driven instead.

### The table

Each row: the mutation, the file, the byte delta, the suite it was graded in,
and how many **named** checks went red. `⚠️` marks a row whose run did not
reach the end of its file, so its count is a **lower bound** (MUTATION.md).

| # | mutation | file | Δ | suite | reds |
|---|---|---|---|---|---|
| M1 | **a write escapes** — the substitution deleted | `main.py` | −6 B | clamp | 5 |
| M2 | **silence instead of 0 W** — it returns rather than substituting | `main.py` | −15 B | clamp | 3 |
| M3 | **the observers go blind** — it moves above SAFETY-28 and SAFETY-03 | `main.py` | +58 B | clamp | 1 |
| M4 | the site re-seed dropped — the exit resumes from a fiction | `main.py` | −36 B | clamp | 3 |
| M5 | the per-unit re-seed dropped | `main.py` | −22 B | clamp | 1 |
| M6 | **PAIRED**: `standing_w()` reads the target **and** the re-seed dropped — the sign latch that survives into live operation | `inverter.py` + `main.py` | −170 B | watchdog | 3 |
| M7 | `auto` inverted — shadow while started, live while stopped | `shadow.py` | +2 B | validate | 4 |
| M8 | **default-on for everyone** — `auto` always shadow; the literal reading that stops the live install | `shadow.py` | −15 B | validate | 2 |
| M9 | an unreadable value resolves to LIVE instead of shadow | `shadow.py` | +1 B | validate | 1 |
| M10 | `validate.py` stops refusing an unknown value | `shadow.py` | −9 B | validate | 9 |
| M11 | the loop leg gated on `auto` alone again — the decision path stops running | `main.py` | −17 B | clamp | 1 |
| M12 | the stop claim filed in shadow — every decision reads `stopped` | `main.py` | −20 B | clamp | 1 |
| M13 | the banner goes green while nothing is commanded | `shadow.py` | −2 B | clamp | 1 |
| M14 | the banner loses the standing latch | `shadow.py` | −2 B | clamp | 1 |
| M15 | the row renders the standing command instead of the decision | `shadow.py` | −8 B | clamp | 1 |
| M16 ⚠️ | the row disappears when live | `main.py` | +43 B | clamp | 1 |
| M17 | the record becomes the wire | `main.py` | −6 B | clamp | 7 |
| M18 | level-logged — the 40-row action log floods | `main.py` | −23 B | clamp | 1 |
| M19 | the two rungs render identically (masking) | `shadow.py` | +22 B | validate | 1 |
| M20 | the shipped default moves to `off` in `config.yaml` alone | `config.yaml` | −1 B | defaults_pin | 1 |
| M21 | **PAIRED**: the substitution **and** the re-seed deleted together | `main.py` | −103 B | clamp | 10 |
| M22 | the re-seed moved one line earlier — SAFETY-18 grades a 0 W no decision produced and goes permanently blind | `main.py` | +40 B | clamp | 1 |

**Survivors: none.** ⚠️ **M17's `+0 B`-adjacent sibling** —
`shadow_would_w = float(self.target)` — is recorded in §7 as **equivalent**
rather than counted here.