goodwe-addon/goodwe_controller/app/freshness.py

"""SAFETY-37 - a reading that holds its last value and reads healthy.

THE DEFECT, in one sentence: `hass.number()` returns a held value as a value,
and nothing anywhere in this add-on distinguishes "the inverter reports 40 %"
from "the entity still says 40 % because nothing has updated it since Tuesday".

The architect pass of 2026-09-06 put three cards on one mechanism. This module
is that mechanism; the other two are its consumers.

  LAYER 1     `goodwe/master/goodwe-master-stage4-failsafe.yaml:690` - the
  (TEL-06)    ingest-age veto tests the VALUE of `sensor.goodwe_p1_sample_age`.
              A dead publisher leaves the entity frozen at a healthy number and
              the veto can never fire. NOT FIXED HERE - it is firmware, it is on
              the other side of a flash, and nothing in this module can reach
              it. What it owes is written at the bottom of this docstring.
  THIS ADD-ON `app/hass.py:69-82`, consumed by SAFETY-30's per-unit sign check
  (SAFETY-37) and SAFETY-31's circulating-power sensor. Both read `Inverter.batt`
              (register 0x0518) and NEITHER had a freshness gate of any kind.
  THE EVIDENCE HW-01, 2026-09-02: SoC held 40 % for 6 h 25 m while the pack
              drained to its floor. Every detector read healthy throughout.

⚠️ AT N>=2 EVERY ONE OF THOSE READINGS DOUBLES, which is why this card is the
one that had to build it: the per-unit sign check and the circulation sensor
only exist at N>=2, and both are fed straight from `Inverter.batt`.


WHAT CANNOT BE THE SOURCE OF TRUTH, AND IT IS NOT A DETAIL
==========================================================
Home Assistant's `last_updated` DOES NOT MOVE when a value is rewritten
unchanged. Neither does `last_changed`, and this project measured that
`last_reported` does not advance on a repeat on either serialiser either - see
`p1.HomeWizardLocalSource`'s docstring, which records the ENV-01 rig measurement
over 70 s of a frozen meter and an independent one against the live house.

So a value-side timestamp out of Home Assistant is a "time since the number
changed" sensor wearing an arrival sensor's name. Thresholding it is how a
detector ends up unable to fire, which is TEL-06's defect exactly. That trap
produced two false diagnoses on 2026-09-05, both by a reader who already knew
about it, which is why it is written here in full rather than referenced.

⚠️ AND THE CONVERSE TRAP IS THE ONE THIS MODULE HAD TO SOLVE. The add-on polls
Home Assistant's state cache roughly once a second (`run_control()`'s
`await asyncio.sleep(1)` bounds the loop body and `read_inputs()` runs every
pass). While the entity exists, HA answers 200 with a number whether or not the
publisher behind it is alive. So AN ARRIVAL AT THE ADD-ON IS NOT AN ARRIVAL FROM
THE INVERTER, and an arrival age alone cannot see HW-01 at all.

What IS arrival-stamped here, and what makes this module honest, is the clock
the verdict is measured on: `changed_mono` is the MONOTONIC INSTANT OF THE
ARRIVAL THAT LAST CARRIED A DIFFERENT NUMBER. Every term below is a fact about
arrivals this process observed, on its own clock. Nothing is read out of a Home
Assistant timestamp, and nothing would change if Home Assistant stopped
publishing timestamps altogether.


THE VERDICT, AND WHY IT IS A CONJUNCTION
========================================
A bare "unchanged for N seconds" threshold is not available, and that is
MEASURED rather than feared. Over the committed 30-day capture of this house's
own 1 W-resolution power entity - `scenarios/p1-net-30d-2026-08/`, 485,841
samples straight out of Home Assistant's recorder, where a row exists only when
the state string CHANGED, so the gap between two rows IS the unchanged duration
every consumer saw:

    longest unchanged run, 30 days   263.2 s   (held -899.0 W, 2026-08-11 17:04)
    runs >= 35 s                     10 of 485,840   (0.0021 %)
    runs >= 17 s                    386 of 485,840   (0.0794 %)

Measured on 2026-09-06 by replaying that capture. A real power figure at 1 W
resolution held ONE value for four and a half minutes with nothing whatever
wrong. ⚠️ That is the METER's register, not 0x0518, and it is named as a
surrogate rather than passed off as the thing: no 30-day capture of 0x0518
exists, because that register is not in the recorder export. What it does
establish - and this is all it is used for - is that the LEGITIMATE unchanged
duration of a power entity in this house is minutes, so an unchanged threshold
alone must be minutes, and minutes is three times too slow to protect consumers
that trip at 75 s and 85 s. The conjunction is not a refinement; without it
there is no admissible threshold at all.

⚠️ AND EVERY ONE OF THOSE LONG RUNS HAPPENED UNDER AN UNMOVING LOAD. That is
the way out, and it is the same shape `freeze.looks_frozen()` already uses for
the meter (unchanged AND the command pinned at the rail). A reading is stale
when it has not moved THROUGH A CHANGE IT WAS OBLIGED TO FOLLOW:

    STALE  <=>  unchanged for >= UNCHANGED_S
                AND |standing command now| >= MOVE_W
                AND |standing command now - standing command when the reading
                     last changed| >= MOVE_W

0x0518 is the inverter reporting ITSELF and it tracks its own command: r = 0.998
with battery power against r = 0.09 with the real meter (goodwe/NOTES.md §3.3),
command and actual agreeing within 4 W once settled (§5.10, 2026-08-22), and the
register dithering +/-1 W while holding (§5.11, same day). A unit told to move
250 W and still answering the same integer watt cannot be tracking.


THE BOUNDS - ONE PER SIGNAL, EACH WITH ITS DERIVATION
=====================================================
⚠️ ONE CONSTANT FOR BOTH SIGNALS WOULD BE USELESS ON SoC OR TRIGGER-HAPPY ON
POWER, which is the third clause of the policy. They are separated here, and
only one of them is WIRED - see `BOUNDS` below and the note on SoC.

`batt` / 0x0518, S16, 1 W, published every 10 s
----------------------------------------------
  LOWER BOUND, 17.0 s, the fastest a healthy register can be OBLIGED to have
  answered. Three terms, each read out of this repository on 2026-09-06:

     6.0 s  the plant. goodwe/NOTES.md §5.9, measured 2026-08-22 with the poll
            temporarily at 1 s and a 0 -> +500 W step: 26 W at +1.43 s, 469 W
            (94 %) at +3.34 s, settled 507 W at +6.05 s.
    10.0 s  the publisher. `firmware/goodwe-master.yaml:181`,
            `update_interval: 10s` on the `modbus_controller` that carries both
            0x0518 and 0x050E. A command that moves right after a poll is not
            visible until the next one.
     1.0 s  this add-on's own read period - `run_control()`'s
            `await asyncio.sleep(1)`, which bounds the loop body that calls
            `read_inputs()`.

  UPPER BOUND, 75.0 s, the tighter of the two consumers this card must protect,
  both read from `config.yaml` on 2026-09-06:

    75 s   `circulation_hold_s` (config.yaml:512) - SAFETY-31 trips here.
    85 s   `unit_sign_disagree_s` (config.yaml:451) - SAFETY-30's per-unit sign
           check trips here.

  A gate slower than either is a gate that hands its consumer the frozen number
  and then explains afterwards. ⚠️ AND THAT INEQUALITY IS GRADED, NOT ASSERTED -
  `test_freshness.py` part 6 reads BOTH ends out of the shipped `config.yaml`
  and goes red if an operator's default moves under this bound.

  35.0 s IS THE CHOSEN VALUE, and the admissible band is narrow enough that it
  is nearly forced: at least 2x the 17.0 s floor (34.0) and at most half the
  75.0 s consumer window (37.5) leaves [34.0, 37.5]. 35 is the integer in it.
  ⚠️ DELIBERATELY NOT 30 AND NOT 60. 30 is `validate.FIRMWARE_INGEST_VETO_S` and
  would read as a borrowed constant nobody re-derived; 60 is round, and a round
  number here would be a guess wearing a measurement's clothes.

  WHAT IT COSTS, stated rather than absorbed: a frozen 0x0518 on a unit that is
  never asked to move 250 W is not detected by this module at all.

  ⚠️ THAT COST IS BOUNDED FOR ONE CONSUMER AND NOT FOR THE OTHER, and an earlier
  version of this note claimed both. TRUE for the per-unit sign check: no such
  check can speak about that unit either, since `unit_sign_disagree_w` is 250 W.
  FALSE for circulation. "Contributing at most 250 W" bounds the COMMAND - but a
  frozen register's REPORTED value is unrelated to its command, and that is
  precisely HW-01's shape: commanded ~0 W, register stuck on the last large
  number. Measured in review: a unit frozen at -1200 W beside a healthy +900 W
  sibling gives `circulating_w` = 1800 W against a 700 W floor, 2.6x over, with
  no gate armed. So this gate is scoped to where the SIGN consumer can be misled;
  the circulation consumer keeps an unbounded exposure below MOVE_W and that is
  a known, stated gap rather than a covered one.

`soc` / 0x050E, U16, 1 %, published every 10 s - STATED, NOT WIRED
-----------------------------------------------------------------
  ⚠️ THIS ROW IS THE POLICY'S ANSWER FOR TEL-08 AND IT IS NOT CONNECTED TO
  ANYTHING. It is here because the third clause of the policy is "state each
  bound and its derivation", and because a reader who finds only `batt` wired
  would reasonably conclude SoC was forgotten. It was not; it needs a
  corroborator this card cannot derive.

  SoC CANNOT USE THE CONJUNCTION ABOVE. A pack resting at its floor holds one
  percent for hours with nothing wrong - the card's own example - and the
  standing command says nothing about it, because a command of 0 W is exactly
  what a rested pack is given. The physically obliged movement for SoC is
  ENERGY, not command travel: one percent of pack A is about 153.6 Wh (15.36 kWh
  nominal, goodwe/NOTES.md §1.1b's table - ⚠️ NOMINAL and FROM DATASHEET, not a
  measured usable capacity), so at pack A's 5000 W ceiling one percent takes at
  least 110 s and at a realistic 1000 W about 553 s. The bound TEL-08 needs is
  therefore "SoC unchanged while the integral of |battery power| since its last
  change exceeded K x 153.6 Wh", and K cannot be chosen from anything written
  down today: nobody has measured this pack's usable capacity, its BMS reporting
  granularity, or how far its reported percent lags true state of charge.

  So TEL-08 owes: that capacity measurement (or a defensible bound on it), the
  integrator, and K. It does NOT owe a second freshness mechanism - `Freshness`
  below takes its obligation term from the caller precisely so that an energy
  integral can be handed in where command travel is handed in here.
"""

import time
from dataclasses import dataclass

# --- the three terms of the lower bound, each read out of this repository ----
# See the module docstring for where each one is measured. They are separate
# names rather than a literal 17.0 so that a reader who changes the firmware's
# `update_interval` can find this file by grepping for the number they moved.
PLANT_SETTLE_S = 6.0        # goodwe/NOTES.md §5.9, measured 2026-08-22
PUBLISH_PERIOD_S = 10.0     # firmware/goodwe-master.yaml:181
READ_PERIOD_S = 1.0         # main.run_control()'s `await asyncio.sleep(1)`
BATT_OBLIGED_S = PLANT_SETTLE_S + PUBLISH_PERIOD_S + READ_PERIOD_S   # 17.0

# The unchanged bound for register 0x0518. See the derivation above; the band is
# [2 * BATT_OBLIGED_S, 0.5 * circulation_hold_s] = [34.0, 37.5].
BATT_UNCHANGED_S = 35.0

# How far the standing command must have travelled for the register to be
# OBLIGED to have followed it.
#
# ⚠️ IT IS THE LARGEST ADMISSIBLE VALUE, NOT A SENSITIVITY PREFERENCE, and that
# is what makes it a derivation. The gate has to arm everywhere a stale reading
# could mislead a consumer; the earliest either consumer speaks is
# `unit_sign_disagree_w`, 250 W (config.yaml:450), because below that
# `watchdog.disagrees()` is False whatever the measurement says. Taking the
# ceiling of the admissible range is what minimises spurious arming. A smaller
# number would arm the gate in states where nothing downstream can be misled,
# and every such arming is a spurious site stop bought for nothing.
#
# ⚠️ NOT A CONFIG KEY. CFG-03 tracks the places a default has to be kept in
# step and this does not earn another one - it is derived from a default
# `config.yaml` already carries, and `test_freshness.py` part 6 READS that file
# and goes red if the two drift.
MOVE_W = 250.0


@dataclass(frozen=True)
class Bound:
    """One signal's freshness bound, and whether anything consumes it."""

    signal: str
    unchanged_s: float
    move_w: float
    wired: bool
    derivation: str


# ⚠️ THE POLICY AS DATA, AND THE `wired` COLUMN IS LOAD-BEARING. A reader must be
# able to tell, without reading main.py, which of these bounds is actually
# gating a consumer today. `test_freshness.py` part 7 greps the shipped app/ for
# each row and refuses a row that claims to be wired and is not - and a row that
# claims not to be and is.
BOUNDS = {
    "batt": Bound(
        signal="batt (0x0518, AC power, S16, 1 W, 10 s publish)",
        unchanged_s=BATT_UNCHANGED_S,
        move_w=MOVE_W,
        wired=True,
        derivation="settle 6.0 + publish 10.0 + read 1.0 = 17.0 s floor; "
                   "circulation_hold_s 75 s ceiling; band [34.0, 37.5]"),
    "soc": Bound(
        signal="soc (0x050E, pack SoC, U16, 1 %, 10 s publish)",
        unchanged_s=float("inf"),
        move_w=float("inf"),
        wired=False,
        derivation="TEL-08. Command travel is the WRONG obligation term for "
                   "SoC - a rested pack holds a percent for hours under a 0 W "
                   "command. The term is energy: >= K x 153.6 Wh through the "
                   "pack since the reading last changed. K needs a measured "
                   "usable capacity nobody has taken."),
}


class Freshness:
    """One signal's arrival record, and the stale verdict it supports.

    ⚠️ ARRIVAL-DRIVEN THROUGHOUT. Every clock reading this object ever makes is
    handed to `submit()` by the caller at the moment a value arrived; nothing
    here reads a clock of its own and nothing here reads a Home Assistant
    timestamp. That is what makes the verdict replayable - `test_freshness.py`
    drives real seconds through it without sleeping - and it is the difference
    between this and the value-stamped veto TEL-06 is blocked on.

    ⚠️ THE VERDICT LATCHES, AND WITHOUT THAT IT OSCILLATES AT ABOUT 1 Hz.
    Declaring the reading stale makes `Controller.inputs_ok` False, which
    commands 0 W (§11.1 rule 1), which drops the standing command below
    `MOVE_W`, which un-arms the conjunction, which would return the held number
    to the consumers - and round again every read. So the ONLY thing that clears
    the latch is an arrival carrying a DIFFERENT number, which is exactly the
    evidence the verdict was about. A register that is working clears it on its
    next 10 s publish; a register that is not, does not.
    """

    def __init__(self, bound: Bound):
        self.bound = bound
        # The last number that ARRIVED, or None if the most recent read did not
        # produce one. This is what `reading()` hands back when the gate is open,
        # so a read failure is still an immediate None exactly as it is today.
        self.last = None
        # The number the reading is currently HOLDING, and the monotonic instant
        # of the arrival that last carried a different one. `held` survives a
        # failed read; the unchanged clock is about the value, not about our
        # ability to fetch it.
        self.held = None
        self.changed_mono = None
        # The standing command at that same instant. The conjunction compares
        # against THIS, not against the previous cycle's command: what matters
        # is whether the register has followed anything since it last moved.
        self.standing_at_change = None
        self.stale = False
        self.reason = ""
        self.arrivals = 0

    def submit(self, value, standing_w, now_mono) -> None:
        """One READ of this signal. `value` None means nothing arrived.

        `value`       the number `hass.number()` produced, or None.
        `standing_w`  what this controller currently has standing on this unit's
                      wire, signed, + = discharge. None when nothing is known.
        `now_mono`    the caller's monotonic clock, in seconds.

        ⚠️ A FAILED READ IS NOT AN ARRIVAL AND IS NOT AN AGE EITHER. It clears
        `last` immediately - which is byte-for-byte what shipping code does
        today, a missing reading being a missing reading - and it leaves the
        unchanged clock running, because the value has not changed and our
        inability to fetch it is not evidence that it has.
        """
        if value is None:
            self.last = None
            return
        value = float(value)
        self.arrivals += 1
        if self.changed_mono is None or value != self.held:
            self.held = value
            self.changed_mono = float(now_mono)
            self.standing_at_change = (None if standing_w is None
                                       else float(standing_w))
            # Movement is the ONLY thing that clears the latch. See the class
            # docstring on the 1 Hz oscillation this prevents.
            self.stale = False
            self.reason = ""
        self.last = value
        self._grade(standing_w, now_mono)

    def _grade(self, standing_w, now_mono) -> None:
        """The conjunction. Sets the latch; never clears it."""
        if self.stale or self.changed_mono is None:
            return
        held_s = self.unchanged_s(now_mono)
        if held_s < self.bound.unchanged_s:
            return
        # ⚠️ BOTH None GUARDS ARE FOR A CALLER THAT DOES NOT KNOW ITS OWN
        # OBLIGATION YET, WHICH IS NOT `_read_unit`. `Inverter.standing_w()`
        # always answers a float, so neither branch is reachable from the wired
        # consumer today. They are here for TEL-08, whose obligation term is an
        # energy integral that has nothing to say until it has a window - and
        # "nothing to say" must withhold the verdict, not manufacture one.
        if standing_w is None or self.standing_at_change is None:
            return
        standing_w = float(standing_w)
        if abs(standing_w) < self.bound.move_w:
            return
        moved_w = abs(standing_w - self.standing_at_change)
        if moved_w < self.bound.move_w:
            return
        self.stale = True
        self.reason = (
            f"STALE_READING: {self.held:+.0f} unchanged for {held_s:.0f} s "
            f"while the standing command moved {moved_w:.0f} W to "
            f"{standing_w:+.0f} W - the reading is not following, so it is "
            f"reported as MISSING rather than as a measurement (limits: "
            f"unchanged over {self.bound.unchanged_s:.0f} s, command over "
            f"{self.bound.move_w:.0f} W and moved by over "
            f"{self.bound.move_w:.0f} W)")

    def unchanged_s(self, now_mono) -> float | None:
        """Seconds since an arrival carried a different number. Status page."""
        if self.changed_mono is None:
            return None
        return max(0.0, float(now_mono) - self.changed_mono)

    def reading(self):
        """What the consumers get: the value, or None when it cannot be trusted.

        ⚠️ `None`, NEVER A HELD NUMBER, and never a zero. Both consumers already
        do the right thing with None and neither could ever receive one:
        `watchdog.SignWatchdog.evaluate` CLEARS its window on a None measurement
        rather than accumulating it, and `circulation.circulating_w` returns None
        for a roster with a missing term, which clears the circulation window
        too. This module is the thing that was missing between them.
        """
        return None if self.stale else self.last

    def note(self, unit_name: str) -> str:
        """One line for the status page. Empty when there is nothing to say."""
        return f"{unit_name}: {self.reason}" if self.stale else ""


def monotonic() -> float:
    """The clock this module's callers use, in one place so tests can find it.

    ⚠️ `time.monotonic()`, not the wall clock, for `p1.P1Sample.age_s`'s reason:
    an NTP step on a Raspberry Pi that has just found the network is routinely
    minutes, and a freshness verdict computed across one would either fake a
    stale reading or, worse, hide a real one.
    """
    return time.monotonic()