goodwe-addon/goodwe_controller/app/shadow.py

"""SAFETY-10 - shadow mode: the whole decision path, and no command.

WHAT DEFECT THIS PREVENTS
=========================
Not one defect - the CLASS of them. Every optimiser fault this repository has
found was invisible from outside while it was happening, and each one was paid
for at the meter before anybody could see it:

  * SAFETY-45  arbitrage against a binding `peak_cap_w` ran 28 charge onsets an
               hour, the grid swinging -455 .. +3200 W, and put a billed
               quarter 59 W over the cap;
  * TARIFF-14  the price lookup answered for the wrong quarter-hour 45 minutes
               in every 60, and traded on it;
  * SAFETY-46  the SoC ceiling churns 598 Wh to gain 310 Wh, no hysteresis.

There was no way to WATCH what the optimiser would do without letting it do it.
A commissioning engineer could arm arbitrage and find out, or not arm it and
learn nothing. Shadow mode is the third option: strategies, arbiter, clamps and
guards all run, the decision is recorded with its reason, and NOTHING THE
CONTROLLER DECIDES REACHES THE INVERTER.

⚠️ "REACHES NOTHING" IS NOT "WRITES NOTHING", AND THE DIFFERENCE IS THE WHOLE
   SAFETY ARGUMENT
=======================================================================
The inverter holds its last command FOREVER - it has no meter timeout
(HANDOVER.md §6), and firmware/goodwe-master.yaml's header records what that
cost when a controller stopped refreshing: Illusmart went silent mid-command
and the inverter held +5000 W for 113 s. main.py's own header carries the same
measurement. So a shadow mode that
simply stopped writing would leave whatever was standing when it was switched
on standing for as long as it was left on. That is the exact failure this
repository keeps finding - LOOKS HEALTHY BUT IS INERT - with the status page
calmly reporting that nothing is being commanded.

So shadow mode does write, and it writes ONE number:

    the actuator is held at WIRE_W (0.0 W), refreshed on the ordinary
    heartbeat, by the ordinary single writer.

That is a DEFINED, SAFE actuator state, it is the same state the operator's
stop button reaches, and it honours the contract firmware/goodwe-master.yaml
§"CONTRACT CHANGE" imposes on us - HA must rewrite the setpoint every ~10 s or
the ESP32's 30 s stale-input watchdog cannot tell "steady" from "dead". Going
silent would put that board into a tripped-watchdog state for the whole shadow
run, which is an ALARM being used as a MODE: the operator who learns that
`wd_tripped` is normal during shadow runs is the operator who ignores it when
the add-on has actually died.

⚠️ AND IT IS NOT A SECOND WRITE PATH. SAFETY-28's rule is one actuator, one
writer. Shadow mode files no claim, owns no transport and calls nothing in
hass.py. It is a SUBSTITUTION inside `Controller._write_unit()` - the single
choke point every write in this add-on already goes through - in exactly the
form the three substitutions above it take (non-finite -> 0 W, unauthorised ->
0 W, clamped -> the rating). It is the LAST of them, so SAFETY-28's
authorisation gate and SAFETY-03's clamp still grade the number the decision
path actually produced, and a shadow run therefore records that the clamp WOULD
have engaged.

THE DEFAULT, AND WHY IT IS NOT A PLAIN `true`  (§ the ticket's own warning)
==========================================================================
The card is titled "default-on". Taken literally - `shadow_mode: true` in
`config.yaml`'s `options:` block - it is the worst change in this repository's
history. Home Assistant's Supervisor composes an add-on's effective options as
`{**config.yaml defaults, **the stored user options}`, so a NEW key with a new
default reaches EVERY install on the next update, including the one regulating
a real battery in the author's house today. Literal default-on silently stops
an existing install from controlling, and the only symptom is a battery that
has stopped doing anything.

There is no mechanism in the add-on options system that lets one key default
differently for a new install and an existing one: `options:` IS the default,
and there is exactly one of it. So "default-on for new" has to be expressed as
a value the add-on RESOLVES, not as a constant the Supervisor merges.

Hence three values rather than two, defaulting to `auto`:

    off    never shadow. The controller commands the battery.
    on     always shadow, whatever the start/stop button says.
    auto   shadow WHILE CONTROL IS STOPPED; live once it is started.

`auto` is default-on for a new install and a no-op for an existing one, and it
is that without any persisted marker, any migration, and any guess:

  * a FRESH install ships `auto_start: false` and has no stored `auto` key, so
    it comes up stopped -> shadow. It observes a day of decisions before it is
    ever allowed to command anything. That is the card's intent, delivered.
  * an install that is CONTROLLING - `auto_start: true`, or the operator has
    pressed *Start / stop control* and the store carries `auto: true` - is
    started, so `auto` resolves to LIVE and its behaviour does not change by
    one watt.
  * an install that is STOPPED keeps its actuator at 0 W exactly as before -
    the same number, from the same writer, on the same heartbeat - and gains
    the recording. The only thing that changes for it is what the status page
    and the action log SAY.

⚠️ AND `auto` MEANS THE START BUTTON LEAVES SHADOW MODE. That is deliberate:
under `auto` the button reads "Start / stop control" and pressing Start is the
operator saying "control the battery". An operator who wants the decisions
recorded WHILE the loop runs sets `shadow_mode: on` explicitly - that is the
configuration this card exists to make possible, and it is the one that cannot
be reached by accident.

⚠️ AN UNRECOGNISED VALUE RESOLVES TO SHADOW, NOT TO LIVE. `validate.py` refuses
it at startup so this is not meant to be reachable, but the fail-safe direction
for "the operator's intent is unreadable" is the one that does not move energy.

THE FAILSAFES, ONE BY ONE (the card asks for each to be decided and recorded)
============================================================================
| failsafe                       | in shadow mode                            |
|--------------------------------|-------------------------------------------|
| SAFETY-02/30 sign watchdogs    | INERT, and by construction rather than by |
| (per-unit and site)            | a special case. They grade                |
|                                | `Inverter.standing_w()` - what this       |
|                                | controller has standing on the wire - and |
|                                | shadow mode holds that at 0 W. A 0 W      |
|                                | command cannot oppose a meter, so the     |
|                                | detector never arms. ⚠️ THIS IS THE ONE   |
|                                | THAT HAD TO BE GOT RIGHT: fed the SHADOW  |
|                                | decision instead, every shadow run would  |
|                                | latch a trip within `sign_disagree_s` -   |
|                                | the decision opposes a meter that never   |
|                                | saw it - and that latch SURVIVES INTO     |
|                                | LIVE OPERATION and a restart. Shadow mode |
|                                | would manufacture the failsafe it is      |
|                                | meant to help debug.                      |
| SAFETY-03 clamp                | OBSERVES. It runs on the number the       |
|                                | decision path produced, `clamp_last` is   |
|                                | set and every engagement goes to the      |
|                                | action log, because "it would have hit    |
|                                | the rating" is part of what a shadow run  |
|                                | is for. It changes no command: the value  |
|                                | it narrows is replaced by 0 W below it.   |
| SAFETY-28 write authorisation  | OBSERVES, for the same reason and in the  |
|                                | same place - above the substitution, so   |
|                                | it still asks who chose the number.       |
| SAFETY-18 meter freeze         | OBSERVES, unchanged. It takes no action   |
|                                | in live operation either; it grades the   |
|                                | resolved intent against a meter that has  |
|                                | stopped moving, and both halves are as    |
|                                | true in shadow as out of it.              |
| SAFETY-07/24/25 peak guard     | OBSERVES. Level-triggered, unlatched, and |
|                                | it files its claim as usual so the        |
|                                | recorded decision is the real one.        |
|                                | SAFETY-45's limit cycle is visible from a |
|                                | shadow run; that is the point.            |
| SAFETY-31 circulation          | OBSERVES AND MAY LATCH, legitimately. Its |
| SAFETY-42 delivery             | evidence is MEASURED power, not a         |
|                                | command: packs pushing against each other |
|                                | while every one of them is told 0 W is a  |
|                                | real fault and a latch is the right       |
|                                | answer to it.                             |
| operator stop (`auto` False)   | SUPPRESSED AS A CLAIM, ENFORCED AT THE    |
|                                | WIRE. It is the one deliberate change.    |
|                                | Its `P_SAFETY` 0 W claim out-ranks        |
|                                | everything, so leaving it in would make   |
|                                | the recorded decision "stopped" on every  |
|                                | cycle - the shadow run would answer the   |
|                                | wrong question. The actuator state is     |
|                                | identical either way, and the wire        |
|                                | substitution is STRICTLY STRONGER than    |
|                                | the claim: a claim can be out-ranked, and |
|                                | this cannot.                              |
| inputs missing, non-finite,    | UNCHANGED. All of them resolve to 0 W,    |
| allocator refusal, arbiter     | which is what the wire gets in shadow     |
| contradiction                  | mode anyway, so they act redundantly and  |
|                                | are recorded as they always were.         |

⚠️ MAINTENANCE IS NOT IN THAT TABLE BECAUSE SHADOW MODE DOES NOT TOUCH IT. The
state machines are ticked on every cycle today whether control is started or
not, so a stopped install already runs them against a battery that does not
move. Shadow mode inherits that, unchanged and unimproved; it is a pre-existing
property of `auto_start: false` and it is recorded here rather than quietly
fixed under another card's name.

LEAVING SHADOW MODE
===================
`Controller.cycle()` re-seeds its carried command (`self.target` and each
`inv.target`) to WIRE_W at the END of every shadow cycle, AFTER every in-cycle
guard has read the decision. Two things follow, and both are safety properties:

  * the loop's `prev_w` and its integrator track WHAT IS ACTUALLY STANDING ON
    THE WIRE, so every shadow cycle answers "from where the plant really is,
    what would this controller do NOW" - a true statement about the present,
    rather than a simulation of a counterfactual plant that drifts further from
    reality every tick;
  * the exit is safe BY CONSTRUCTION. The carried command already equals the
    standing command, so the first live cycle slews from 0 W like any other
    cycle. Carrying the shadow decision instead would let the slew limiter
    depart from a base that never existed - up to a full `slew_w` step onto a
    battery sitting at zero.

The entry is safe for the ordinary reason: the substitution happens on the very
first write of the very first shadow cycle, and `last_write` was not 0 W, so
that write is issued rather than skipped as unchanged.
"""
from __future__ import annotations

# ⚠️ CFG-03's mirror. `test_defaults_pin.py` part 2 discovers every
# module-level name ending in OPTION_DEFAULTS and asserts each value
# reproduces config.yaml EXACTLY, so this table cannot drift from the shipped
# default. Nothing in the add-on may carry a literal fallback for a key a
# mirror answers for - part 3 scans for that - which is why `main.py` asks the
# functions below and never reads `shadow_mode` itself.
SHADOW_OPTION_DEFAULTS = {
    "shadow_mode": "auto",
}

#: The three values `shadow_mode` may take, in the order config.yaml's schema
#: lists them. `validate.py` refuses anything else by name.
MODES = ("auto", "on", "off")

#: What stands on the wire while shadow mode is engaged. ⚠️ A NUMBER, NOT A
#: SILENCE - see this module's header. It is 0.0 W for the same reason every
#: other refusal in this add-on is 0.0 W: fail toward inaction, and keep
#: commanding it so the inverter cannot be left latched on anything else.
WIRE_W = 0.0


def mode(opts: dict) -> str:
    """The configured `shadow_mode`, normalised to one of `MODES`.

    ⚠️ AN UNRECOGNISED VALUE ANSWERS `"on"`, NOT `"auto"` AND NOT `"off"`.
    `validate.py` refuses such a value at startup, so this branch is not meant
    to be reachable at all; it exists because the fail-safe direction for "the
    operator's intent cannot be read" is the one that does not move energy. A
    typo that silently resolved to `off` would put a battery under a controller
    whose configuration nobody could parse.
    """
    raw = opts.get("shadow_mode", SHADOW_OPTION_DEFAULTS["shadow_mode"])
    text = str("" if raw is None else raw).strip().lower()
    if text in MODES:
        return text
    return "on"


def engaged(opts: dict, *, control_started: bool) -> bool:
    """Is the battery being left alone right now?

    `control_started` is `Controller.auto` - the persisted start/stop state,
    which a fresh install gets from `auto_start` and an operator moves with the
    *Start / stop control* button. See the header on why `auto` reads it.
    """
    m = mode(opts)
    if m == "off":
        return False
    if m == "on":
        return True
    return not control_started


def why(opts: dict, *, control_started: bool) -> str:
    """Which rung decided, in the words an operator can act on.

    ⚠️ THE REASON, NOT ONLY THE VERDICT - MUTATION.md's masking rule. `on` and
    `auto`-while-stopped produce the same boolean and want different operator
    actions: one is turned off by editing an option, the other by pressing a
    button. A surface that showed only "shadow" would make a mutation that
    swapped the two rungs invisible.
    """
    # ⚠️ THE LIVE SENTENCES SAY "NOT IN SHADOW MODE", NOT "THE BATTERY IS
    # BEING CONTROLLED". The second is FALSE on a stopped install with
    # `shadow_mode: off` - nothing is regulating there either - and a row that
    # is false in one of the four live states is a row an operator learns to
    # discount. What is true in every live state is that what this add-on
    # decides reaches the inverter; whether control is RUNNING is the
    # `Stopped` banner's question and it is answered there.
    m = mode(opts)
    if m == "off":
        return ("off (shadow_mode: off) - NOT in shadow mode: what this "
                "add-on decides DOES reach the inverter")
    if m == "on":
        return ("on (shadow_mode: on) - decisions are recorded and nothing "
                "is commanded")
    if control_started:
        return ("off (shadow_mode: auto, control is started) - NOT in shadow "
                "mode: what this add-on decides DOES reach the inverter")
    return ("on (shadow_mode: auto, control is stopped) - decisions are "
            "recorded and nothing is commanded")


def words(opts: dict, *, control_started: bool, would_w: float | None,
          reason: str = "") -> str:
    """The status page's `Shadow mode` row.

    ⚠️ THE ROW IS RENDERED IN EVERY STATE, INCLUDING `off`. A surface whose
    ABSENCE is how you learn which mode you are in is a surface an operator
    has to already know about; TARIFF-11's rule is that the rendered text is
    what gets graded, so there has to be text in both states.

    `would_w` is the decision the path reached this cycle - what a live
    controller would have put on the wire - and `reason` is
    `Resolution.explain()`, the same sentence the `Why` row carries. Together
    they are the whole of "what would it have done, and why"; without them the
    row would say a shadow run happened and nothing about what it found.
    """
    head = why(opts, control_started=control_started)
    if not engaged(opts, control_started=control_started):
        return head
    if would_w is None:
        return (f"{head}; no decision has been reached yet - the inverter is "
                f"held at {WIRE_W:.0f} W")
    # ⚠️ THE THOUSANDS SEPARATOR IS SWAPPED ON THE NUMBER ALONE. Applied to
    # the whole sentence it ate the comma out of `head` - and `head` is the
    # half that says WHICH RUNG decided, which is the claim MUTATION.md's
    # masking rule says must survive.
    num = f"{float(would_w):+,.0f}".replace(",", " ")
    out = (f"{head}; WOULD HAVE COMMANDED {num} W, inverter held at "
           f"{WIRE_W:.0f} W")
    if reason:
        out += f" - {reason}"
    return out


def banner(opts: dict, *, control_started: bool, latched: tuple = ()) -> tuple:
    """`(level, text)` for the top of the status page, or `None` when live.

    ⚠️ IT NAMES A STANDING LATCH AND GOES RED FOR ONE. The shadow banner sits
    ABOVE every other branch of the cascade, because "the battery is not being
    controlled" is the one sentence that may never be displaced - but a
    failsafe latch that is up while the controller is inert is up when it
    stops being inert, and an operator who turns shadow mode off without
    clearing it gets a battery that will not move and a green page that
    never mentioned why. So the latch is carried INTO the shadow banner
    rather than losing its own.
    """
    if not engaged(opts, control_started=control_started):
        return None
    text = ("SHADOW MODE - the battery is NOT being controlled. Every "
            f"inverter is held at {WIRE_W:.0f} W and nothing this add-on "
            "decides reaches the inverter.")
    if latched:
        return ("bad", text + " ⚠️ A FAILSAFE LATCH IS STILL UP ("
                + ", ".join(latched) + ") and it will still be up when shadow "
                "mode is turned off.")
    return ("warn", text)


def problems(opts: dict) -> list:
    """`validate.py`'s rule: `shadow_mode` must be one of the three values.

    ⚠️ THE SENTENCE SPELLS `shadow_mode`. `configui.problems_by_field()` files
    a refusal under the field whose key its text contains; a rule that never
    names a key lands in the page's top-level block instead, where the operator
    cannot see which control is wrong. UX-02's whole point is that the shipped
    validator is the only rule surface, so the rule has to be readable there.

    ⚠️ AND IT IS A REFUSAL RATHER THAN A COERCION. `mode()` resolves an
    unreadable value to shadow, which is the safe direction but is also a
    SILENT reinterpretation of what the operator wrote - and an install left in
    shadow mode by a typo commands nothing at all, indefinitely.
    """
    raw = opts.get("shadow_mode", SHADOW_OPTION_DEFAULTS["shadow_mode"])
    text = str("" if raw is None else raw).strip().lower()
    if text in MODES:
        return []
    return [f"shadow_mode is {raw!r}, which is not one of "
            + ", ".join(f"'{m}'" for m in MODES)
            + ". 'auto' leaves the battery alone while control is stopped and "
              "commands it once control is started; 'on' records every "
              "decision and commands nothing at all; 'off' is ordinary "
              "control."]