From 09e1fbb898403efe39d97053fbf6df89028633e7 Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Sun, 1 Nov 2026 23:45:03 +0100 Subject: [PATCH 01/22] SAFETY-128: control does not act on late Home Assistant readings Readings that Home Assistant delivers late but still changing passed every staleness gate, so the loop corrected again each cycle an error its last command had already fixed (10 s late: the give run swung and exported to -1.9 kW), and the peak guard pushed to the inverter limit on the same late figures (OPT-34 F-2). app/lateness.py fits how long after a power request the meter shows it. Over 8 s the readings are late: a request the battery reading has not answered holds the loop (it may not lead the reading by more than 100 W) and caps the peak guard at what stands; unanswered for 23 s it holds, then 0 W after stale_input_s. An amber row and one log line per episode say so. With readings on time nothing changes (digests equal). DECISIONS/SAFETY-128-late-readings.md; test_safety128.py. Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_01Bdqr7PiaHcBnzPituu2yTC --- goodwe_controller/CHANGELOG.md | 4 + .../DECISIONS/SAFETY-128-late-readings.md | 73 ++++ goodwe_controller/app/control.py | 13 + goodwe_controller/app/lateness.py | 218 ++++++++++ goodwe_controller/app/main.py | 52 +++ goodwe_controller/app/messages.py | 23 + goodwe_controller/app/peakguard.py | 13 + goodwe_controller/app/ui_text/en.json | 1 + goodwe_controller/app/ui_text/nl.json | 1 + goodwe_controller/test_peakguard.py | 4 +- goodwe_controller/test_safety128.py | 408 ++++++++++++++++++ goodwe_controller/test_safety94.py | 4 + goodwe_controller/test_safety96.py | 4 + goodwe_controller/tools/safety128_rig.py | 284 ++++++++++++ 14 files changed, 1100 insertions(+), 2 deletions(-) create mode 100644 goodwe_controller/DECISIONS/SAFETY-128-late-readings.md create mode 100644 goodwe_controller/app/lateness.py create mode 100644 goodwe_controller/test_safety128.py create mode 100644 goodwe_controller/tools/safety128_rig.py diff --git a/goodwe_controller/CHANGELOG.md b/goodwe_controller/CHANGELOG.md index 08eb53f..e0d1eee 100644 --- a/goodwe_controller/CHANGELOG.md +++ b/goodwe_controller/CHANGELOG.md @@ -2,6 +2,10 @@ ## 0.20.0 +### Not in a release yet + +- When Home Assistant passes readings on late, control no longer swings on them: it waits until the battery shows its last power request, and the peak limit no longer asks for full power on them. The page says so, and if the readings stay too late every inverter goes to 0 W ([[SAFETY-128]]). + **A minor bump: a Forecast tab, a night grid charge sized to what the sun will not give, a kept evening forecast, and no option changes.** No option was added, removed or renamed, and no default changed (86 options). It is a minor bump because Leftover solar now buys different amounts from the grid and the night charge keeps its level when the forecast goes quiet; every saved setting still starts. **Existing installs need no option change:** options that start 0.19.2 also diff --git a/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md b/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md new file mode 100644 index 0000000..e03642f --- /dev/null +++ b/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md @@ -0,0 +1,73 @@ +# SAFETY-128 - a reading that does not answer the last command is not current + +**Status: FIXED on release/1.0 (after 0.20.0).** Code: `app/lateness.py`, +`control.compute(hold=, hold_w=)`, `peakguard.cap_floor()`, +`Controller._readings_late()`. Tests: `test_safety128.py` +(`tools/safety128_rig.py`, TEST-23's plant). + +**Cause** (SAFETY-127 report). Home Assistant can deliver readings late but +still changing. Every gate judges arrival (SAFETY-01) or a value that stops +changing (SAFETY-18, SAFETY-37), so a late reading passes all of them. The loop +then corrects, each cycle, an error its last command has already fixed: a loop +with a dead time far longer than its cycle, which swings and grows. The peak +guard projects the quarter from the same late figures and pushes to the +inverter limit in its last seconds (OPT-34 rig F-2). + +**Rule.** +1. *Find late readings.* `Delay` fits, every 4 s over the last 40 s of new + meter values, how long after a power request the meter shows it: the meter + is the house less the batteries, so `meter(t) + request(t - d)` is flattest + at the right `d`. Over `DELAY_S` the readings are late (`late_seen`). The + fit counts only if the requests explain half the meter's swing; a late `d` + only if it fits 30 % better than any fresh one and better than a slow + battery answering at once (10 or 20 s lag, the SAFETY-85/94 rigs). A + silent meter adds no values (SAFETY-99/125). A battery that gives nothing + under its ask, or refuses, spoils the fit and is left out for 40 s. With no + fit for 120 s (a steady site shows nothing) the verdict is dropped. + Nothing below acts unless `late_seen`, so with readings on time control is + unchanged. +2. *Hold.* While `late_seen`, a request more than `ANSWER_W` from the battery + reading is pending until that reading comes within `ANSWER_W` of it, or half + way. Meanwhile the loop may not lead the reading by more than `ANSWER_W` + that way (`compute(hold=, hold_w=)`; it may come back), and the peak + guard's floor is capped at the request that stands (`cap_floor()`). The + loop is not held while a battery gives nothing under its ask or refuses: + SAFETY-52/94 need it to go on asking. +3. *Late.* Pending for `LATE_S` with no reading moving `ANSWER_W` towards it is + the late verdict: control holds, then after `stale_input_s` goes to 0 W + ("readings late"), as for a missing input. It ends when a reading starts + towards the request. +4. *Seen.* While `late_seen` an amber row says "Home Assistant readings arrive + about N seconds late, so control holds the batteries." (NL too), and + "every inverter is at 0 W" past the hold. One WARNING (`READINGS_LATE`) + and one action-log line per episode, and one when it ends. + +**Constants** (`app/lateness.py`). + +| | Value | Why | +|---|---|---| +| `ANSWER_W` | 100 W | `control.PROBE_ANSWER_W`: a reading this far towards the ask has answered it (SAFETY-52's probe). | +| `DELAY_S` | 8 s | `freshness.PLANT_SETTLE_S` 6 s + one `LOOP_TICK_S` 2 s: a fresh meter shows a request within the plant's settle (SAFETY-75's 6 s dead time) and one meter tick. The fit reads 0 s at 1/2/5 s meters, 5-7 s with a 6 s dead time, 9 s at 10 s late. | +| `LATE_S` | 23 s | 2 x `PLANT_SETTLE_S` + `PUBLISH_PERIOD_S` 10 s + `READ_PERIOD_S` 1 s: SAFETY-75's 6 s dead time, the 6 s settle, one publish and one read, the slowest healthy battery answer. | +| `FIT_S`, `FIT_EVERY_S` | 40 s, 4 s | Short enough to find 10 s and 30 s late within 50-60 s of the start (the swing has not grown yet); one 2 s cycle in two. | +| `MAX_DELAY_S` | 90 s | F-2 measured 17-78 s. | +| `FIT_SHARE`, `FIT_BETTER` | 0.5, 0.7 | Below these a battery that stops following (a refusal at N=2) was read as 23 s late. | +| `SLOW_TAUS_S` | 10 s, 20 s | The SAFETY-85/94 lagging battery: without this check 16 of 36 of its reversals read as 11-56 s late. | +| `FIT_GONE_S` | 120 s | The verdict outlives the lag by at most `FIT_S` + this when the site goes steady. | + +**What it costs.** +- Late readings are found only after the requests have moved 100 W and the + meter has shown it late: 48-68 s after the lag starts in the tests. Until + then control is as before. +- The row can stay up to 160 s after the readings are on time again, while + the site is steady. +- A battery that cannot follow while readings are late (at a limit, not + reading 0 W) is held, then set to 0 W after 38 s, until it moves. +- The fit costs about 93 x 20 sums every 4 s. +- With readings late, a large step is followed in steps of the lag: the loop + waits for each request to show before the next. + +**Left out.** The round trip setpoint -> Home Assistant: the fitted delay gives +the same figure from readings the add-on already has. The arbitrage charge's +own sizing on late readings (`charge_room_w()`) is not capped here; the test +shows it swinging while the peak guard holds (see the pull request). diff --git a/goodwe_controller/app/control.py b/goodwe_controller/app/control.py index 3484292..5dbe91d 100644 --- a/goodwe_controller/app/control.py +++ b/goodwe_controller/app/control.py @@ -298,6 +298,9 @@ PROBE_S = 34.0 # command: every battery that may move this way is at its mode's limit. MODE_BOUND = "mode-bound" +# SAFETY-128. The loop's reason while it waits for the readings to answer. +WAITING = "waiting for readings" + @dataclass(frozen=True) class Tuning: @@ -486,6 +489,8 @@ def compute( bound: tuple | None = None, gave_nothing: bool | None = None, idle_w: float = 0.0, + hold: int = 0, + hold_w: float | None = None, ) -> Decision: """One control cycle. A cycle is one meter update (~5 s on a HomeWizard P1). @@ -506,6 +511,9 @@ def compute( `idle_w` SAFETY-97: what `actual_w` reads while the batteries give nothing (app/idle.py). The refusal's band and its end are centred on it. 0 is the law as it was. + `hold` SAFETY-128: +1 / -1, the batteries have not answered the last + command that way (app/lateness.py), so the loop may not go + past `hold_w` (None: `prev_w`) that way. 0 is the law as it was. """ reason = "tracking" # OPT-11: a bound that leaves out 0 W would FORCE a direction, which no @@ -622,6 +630,11 @@ def compute( i_w = min(moved, i_w) else: i_w = max(moved, i_w) + # SAFETY-128: no further than `hold_w` until the readings answer the ask. + held_at = float(prev_w if hold_w is None else hold_w) + if hold and hold * (i_w - held_at) > 0: + i_w = held_at + reason = WAITING # OPT-11: the integrator does not wind past what the modes let the # batteries place. Only a side tighter than the rail is left in `bound` diff --git a/goodwe_controller/app/lateness.py b/goodwe_controller/app/lateness.py new file mode 100644 index 0000000..c8b8f51 --- /dev/null +++ b/goodwe_controller/app/lateness.py @@ -0,0 +1,218 @@ +"""SAFETY-128: a reading that does not answer the last command is not current. + +When Home Assistant delivers readings late but still changing, every gate that +judges arrival (SAFETY-01) or an unchanged value (SAFETY-18, SAFETY-37) passes +them, and the loop corrects again, each cycle, an error its last command has +already fixed. The readings themselves show it: after a command the meter moves +the other way and the battery reading follows (r = 0.998, freshness.py), and +late readings do both late. + +`Answer` watches the site command on the wire, the meter and the site battery +reading (less the batteries' idle reading, SAFETY-97): +- `Delay` fits how long after a command the meter shows it. Over DELAY_S the + readings run late (`late_seen`); within it they do not. A battery that does + not follow gives no fit, so no evidence either way; +- a new command more than ANSWER_W from the battery reading is PENDING until + that reading comes within ANSWER_W of it, or half way. The first move of + either reading towards it (ANSWER_W) is its START; +- while `late_seen` and pending, the loop may not lead the battery reading by + more than ANSWER_W that way (`hold`, `hold_w`; not while `exempt`: a + battery that gives nothing under its ask or refuses is SAFETY-52/94's case) + and the peak guard may not push past what stands (`peak_cap`). Pending for LATE_S with no start is LATE: control + holds, then 0 W after `stale_input_s`, as for a missing input. + +With readings on time nothing here acts. DECISIONS/SAFETY-128-late-readings.md. +""" + +import bisect +import math +from collections import deque + +from . import freshness +from .control import LOOP_TICK_S, PROBE_ANSWER_W + +# Within this of the command the reading has answered it (a probe's answer). +ANSWER_W = PROBE_ANSWER_W # 100 +# A fresh meter shows a command within the plant's settle (SAFETY-75's 6 s dead +# time) and one meter tick; later than that the readings run late. +DELAY_S = freshness.PLANT_SETTLE_S + LOOP_TICK_S # 8.0 +# The slowest healthy battery answer: SAFETY-75's 6 s dead time, the 6 s plant +# settle, one 10 s publish and one 1 s read. +LATE_S = (2 * freshness.PLANT_SETTLE_S + freshness.PUBLISH_PERIOD_S + + freshness.READ_PERIOD_S) # 23.0 + + +# The meter's delay is fitted over this much history, up to this delay. +FIT_S = 40.0 +MAX_DELAY_S = 90 +FIT_EVERY_S = 4.0 +# A slow battery answers at once but takes this long (SAFETY-85/94 rigs). +SLOW_TAUS_S = (10.0, 20.0) +# With no fit for this long the evidence is dropped (a steady site shows none). +FIT_GONE_S = 120.0 +# A fit counts only if the commands explain this share of the meter's swing. +FIT_SHARE = 0.5 +# ...and a late delay only if it fits this much better than any fresh one. +FIT_BETTER = 0.7 + + +class Delay: + """How long after a command the meter shows it, fitted from history. + + The meter is the house less the batteries, and the batteries follow the + commands, so `grid(t) + command(t - d)` is the house alone at the right + `d`: the `d` that leaves it flattest is the delay. Only new meter values + count, so a silent meter (SAFETY-99) is not a late one. None while the + commands have not moved enough to tell, or explain too little of the meter + (a battery that does not follow); not late if a slow battery that answers + at once (SLOW_TAUS_S) fits as well.""" + + def __init__(self): + self.cmds = deque() # (t, command) when it changed + self.grid = deque() # (t, meter) + + def add(self, now, cmd, grid): + if not self.cmds or self.cmds[-1][1] != cmd: + self.cmds.append((now, cmd)) + if grid is not None and (not self.grid or self.grid[-1][1] != grid): + self.grid.append((now, grid)) # new meter values only + while self.grid and self.grid[0][0] < now - FIT_S: + self.grid.popleft() + while (len(self.cmds) > 1 + and self.cmds[1][0] <= now - FIT_S - MAX_DELAY_S): + self.cmds.popleft() + + def estimate(self): + if len(self.grid) < 10 or len(self.cmds) < 2: + return None + ts = [t for t, _c in self.cmds] + t0 = self.cmds[0][0] + rows = [(t, g) for t, g in self.grid if t >= t0 + MAX_DELAY_S] + if len(rows) < 10: + return None + cs = [c for _t, c in self.cmds] + if max(cs) - min(cs) < ANSWER_W: + return None + g_mean = sum(g for _t, g in rows) / len(rows) + g_var = sum((g - g_mean) ** 2 for _t, g in rows) / len(rows) + + def var(f, d): + e = [g + f(t - d) for t, g in rows] + m = sum(e) / len(e) + return sum((x - m) ** 2 for x in e) / len(e) + step = lambda t: cs[bisect.bisect_right(ts, t) - 1] # noqa: E731 + pure = [var(step, d) for d in range(0, MAX_DELAY_S + 1)] + best = min(range(len(pure)), key=lambda d: (pure[d], d)) + if g_var <= 0 or pure[best] > (1 - FIT_SHARE) * g_var: + return None + if best <= DELAY_S: + return float(best) + # Late only if it fits clearly better than any fresh delay, and better + # than a slow battery answering at once (SAFETY-85/94's 10-20 s lag). + if pure[best] > FIT_BETTER * min(pure[:int(DELAY_S) + 1]): + return None + first, last = int(t0), int(rows[-1][0]) + for tau in SLOW_TAUS_S: + k, f, slow = 1 - math.exp(-1.0 / tau), None, [] + for sec in range(first, last + 1): # the battery, each second + f = step(sec) if f is None else f + (step(sec) - f) * k + slow.append(f) + if var(lambda t: slow[min(int(t), last) - first], 0) <= pure[best]: + return None + return float(best) + + +class Answer: + """The answer gate. `observe()` once per cycle, before the loop.""" + + def __init__(self): + self.ask = None # the pending command, W (site, + = give) + self.from_w = None # the battery reading when it was asked + self.from_grid = None # the meter when it was asked + self.since = None # when it was asked + self.started = False # a reading has moved towards it + self.late_seen = False # the last fitted delay was over DELAY_S + self.late_at = None # when the late verdict started + self.exempt = False # a battery that cannot follow explains it + self._was = None # (command, reading, meter) at the last call + self.delay = Delay() + self.delay_s = None # the meter's last fitted delay, s + self._fit_at = self._fit_ok = self._exempt_at = None + + @property + def active(self) -> bool: + return self.ask is not None and self.late_seen + + @property + def hold(self) -> int: + """+1 / -1: the loop may not push further that way. 0: no hold.""" + if not self.active or self.exempt: + return 0 + return 1 if self.ask > self.from_w else -1 + + @property + def hold_w(self): + """How far the loop may go while held: ANSWER_W past the reading.""" + if not self.hold: + return None + return self._was[1] + self.hold * ANSWER_W + + def peak_cap(self, standing_w): + """The most the peak guard may ask, or None.""" + return max(float(standing_w), 0.0) if self.active else None + + def late_s(self, now): + """Seconds the late verdict has stood, or None.""" + return None if self.late_at is None else max(0.0, now - self.late_at) + + def observe(self, now, standing_w, batt_w, *, grid_w=None, exempt=False, + idle_w=0.0): + """`standing_w` the site command on the wire, `batt_w` the site + battery reading (None: unknown, nothing changes), `grid_w` the meter + (+ = import), `idle_w` what the batteries read while giving nothing. + `exempt`: a battery gives nothing under its ask, or refuses.""" + if batt_w is None or standing_w is None: + return + cmd, r = float(standing_w), float(batt_w) - float(idle_w) + g = None if grid_w is None else float(grid_w) + self.exempt = bool(exempt) + if self.exempt: + self._exempt_at = now + was, self._was = self._was, (cmd, r, g) + self.delay.add(now, cmd, g) + if self._fit_at is None or now - self._fit_at >= FIT_EVERY_S: + self._fit_at = now + d = self.delay.estimate() + if self._exempt_at is not None and now - self._exempt_at < FIT_S: + d = None # a battery that does not follow spoils it + if d is not None: + self.delay_s, self.late_seen, self._fit_ok = d, d > DELAY_S, now + elif self._fit_ok is None or now - self._fit_ok > FIT_GONE_S: + self.late_seen = False # nothing to tell it by + if self.ask is None: + if was is None or cmd == was[0] or abs(cmd - was[1]) < ANSWER_W: + return + self.ask, self.from_w, self.from_grid = cmd, was[1], was[2] + self.since, self.started = now, False + self.ask = cmd + age = now - self.since + gap = self.ask - self.from_w + way = 1.0 if gap > 0 else -1.0 + batt_to = (r - self.from_w) * way + grid_to = (-1e9 if g is None or self.from_grid is None + else (self.from_grid - g) * way) + if not self.started and max(batt_to, grid_to) >= ANSWER_W: + self.started, self.late_at = True, None + if (self.ask - r) * way < ANSWER_W or batt_to >= abs(gap) / 2: + self._clear() # answered + elif abs(gap) < ANSWER_W: + self._clear() # withdrawn + elif exempt: + self.late_at = None + elif (self.late_at is None and self.late_seen and not self.started + and age >= LATE_S): + self.late_at = now + + def _clear(self): + self.ask = self.from_w = self.from_grid = self.since = None + self.started, self.late_at = False, None diff --git a/goodwe_controller/app/main.py b/goodwe_controller/app/main.py index 35fced1..b6ce466 100644 --- a/goodwe_controller/app/main.py +++ b/goodwe_controller/app/main.py @@ -64,6 +64,7 @@ from .clamp import ( shortfall_note, write_clamp_note, ) from . import freshness +from . import lateness from . import dayview, homeview from .daycurve import SolarToday from . import messages @@ -1281,6 +1282,8 @@ class Controller: self.cycle_at = None self._arb_action = None self.inputs_bad_since = None + self.answer = lateness.Answer() # SAFETY-128 + self._late_logged = self._late_zero = False # SAFETY-03. What the loop's OWN bounds did last cycle - a # (engaged, note) pair so the alarm can say WHICH bound bit without # re-deriving it from a string. ⚠️ THIS ONE STAYS SITE-LEVEL, and it is @@ -3116,6 +3119,36 @@ class Controller: return (decision if self._peak_needed else dataclasses.replace(decision, within_aim=True)) + def _readings_late(self, standing_w): + """SAFETY-128: update the answer gate; seconds the late verdict has + stood, or None. Readings found late are logged once per episode. + DECISIONS/SAFETY-128-late-readings.md.""" + a = self.answer + if self._now_mono is None: + return None + a.observe(self._now_mono, standing_w, self.site_batt_w, + grid_w=self.grid, exempt=bool(self.batt_refused) or any( + i.refused or ( + i.batt is not None + and abs(i.standing_w()) >= lateness.ANSWER_W + and abs(i.batt - i.idle_w) < lateness.ANSWER_W) + for i in self.inverters), + idle_w=self._site_idle_w()) + if a.late_seen != self._late_logged: + self._late_logged = a.late_seen + if a.late_seen: + self.log_event(f"readings late ({a.delay_s:.0f} s) - holding", + hint=messages.hint("readings_late")) + _LOG.warning("Home Assistant readings arrive late, so control " + "holds the battery: READINGS_LATE.") + _LOG.debug("READINGS_LATE: the meter shows a power request " + "%.0f s after it is sent.", a.delay_s) + else: + self.log_event("readings on time", hint=messages.hint( + "readings_back")) + _LOG.info("Home Assistant readings are on time again.") + return a.late_s(self._now_mono) + def _peak_reserve(self) -> frozenset: """SAFETY-87: the batteries that give nothing for the peak this cycle: under their reserve while the others carry a peak floor.""" @@ -4647,6 +4680,8 @@ class Controller: "away a partly delivering unit's output.") stale_after = float(self.o.get("stale_input_s", 15)) + late_s = self._readings_late(site_standing) # SAFETY-128 + self._late_zero = late_s is not None and late_s >= stale_after # ⚠️ `any`, NOT the sole unit's. The hold exists so a single missed poll # does not become a fault; a LATCHED inverter must still reach the claim # list below, because its `failsafe` SET is what parks it. One tripped @@ -4672,6 +4707,13 @@ class Controller: self.why_words = messages.why_waiting(bad_for) self._why_waiting_since = now - timedelta(seconds=bad_for) return + if late_s is not None and late_s < stale_after and self.inputs_ok \ + and (self.auto or self.shadow) \ + and not any(i.sign_wd.tripped for i in self.inverters) \ + and not (n_units > 1 and self.site_sign_wd.tripped): + self.reason = f"readings late {late_s:.0f}s" # SAFETY-128: hold + self.why_words = messages.why_late() + return claims = [] @@ -4850,6 +4892,9 @@ class Controller: hint=messages.hint("inputs_missing")) claims.append(Claim.set("safety", P_SAFETY, 0.0, "inputs missing", actuator=self.actuator_of("safety"))) + elif late_s is not None: + claims.append(Claim.set("safety", P_SAFETY, 0.0, "readings late", + actuator=self.actuator_of("safety"))) # --- SAFETY-07: the quarter-peak projection guard -------------------- # ONE `lo` limit at P_PEAK, and peakguard.py carries the whole argument. @@ -4973,6 +5018,8 @@ class Controller: self.peak_guard = self._peak_on_able(self.peak_guard, maint_results) self.peak_guard = self._peak_landing(self.peak_guard) + self.peak_guard = peakguard.cap_floor( # SAFETY-128 + self.peak_guard, self.answer.peak_cap(site_standing)) # SITE, and peakguard.claim() says so on the object it builds. The # quantity is the 15-minute average OFFTAKE AT THE GRID CONNECTION, # which is what the capacity tariff bills; there is one quarter-hour @@ -5437,6 +5484,7 @@ class Controller: if len(self.inverters) == 1 else None), # SAFETY-97: "gives nothing" is centred on the idle reading. idle_w=self._site_idle_w(), + hold=self.answer.hold, hold_w=self.answer.hold_w, # SAFETY-128 # OPT-11: None unless battery modes are on. SAFETY-85: or while # maintenance holds a battery at N>=2, so the loop does not # wind past what the others can give and jump when it ends. @@ -7333,6 +7381,10 @@ class Controller: (inv.refusal_note(), False)): if note: out.append({"ok": False, "warn": not red, "text": note}) + if self.answer.late_seen: # SAFETY-128 + out.append({"ok": False, "warn": True, "id": "late", + "text": messages.row_readings_late( + self.answer.delay_s, self._late_zero)}) # SAFETY-03. Red while an over-rating clamp is engaged (amber for the # integrator bound, SAFETY-40), and the engagement COUNT is shown even diff --git a/goodwe_controller/app/messages.py b/goodwe_controller/app/messages.py index 9ae3607..b375b0a 100644 --- a/goodwe_controller/app/messages.py +++ b/goodwe_controller/app/messages.py @@ -644,6 +644,19 @@ def row_controller_stalled(age_s, veto_s) -> str: f"stuurmodule zet de omvormer na {seconden(veto_s)} op 0 W.") +def row_readings_late(delay_s, zero=False) -> str: + """SAFETY-128: Home Assistant readings arrive late; past the hold, 0 W.""" + if zero: + return Bi(f"Home Assistant readings arrive about {seconds(delay_s)} " + f"late, so every inverter is at 0 W.", + f"Metingen uit Home Assistant komen zo'n " + f"{seconden(delay_s)} te laat, omvormers op 0 W.") + return Bi(f"Home Assistant readings arrive about {seconds(delay_s)} late, " + f"so control holds the batteries.", + f"Metingen uit Home Assistant komen zo'n {seconden(delay_s)} te " + f"laat, dus de batterijen wachten.") + + def row_meter_waiting(limit_s) -> str: """UX-10: no reading yet since the start, and still inside the limit.""" return Bi(f"Meter: waiting for the first reading, limit " @@ -1270,6 +1283,7 @@ WHY_WINNER = { "circulation": "Safety stop, inverters worked against each other", "stopped": "Control is stopped", "inputs missing": "A reading is missing", + "readings late": "Home Assistant readings arrive late", "no claim was made": "Nothing asked for power", # UX-10: the claim reasons main.why_key() can return for a safety or # failsafe winner. test_messages.py part 7 finds every one in the code. @@ -1320,6 +1334,11 @@ def why_missing(what, limit_s) -> str: return f"No usable {what} reading for over {seconds(limit_s)}: 0 W" +def why_late() -> str: + """SAFETY-128: readings arrive late, so control holds the batteries.""" + return "Home Assistant readings arrive late, control holds" + + def why_waiting(missing_s) -> str: """A reading went missing a moment ago: control holds and waits.""" return f"A reading is missing for {seconds(missing_s)}, control waits" @@ -1337,6 +1356,8 @@ HINTS = { "roster": "Inverter list read: {n} inverters", "maint_history": "Maintenance history did not move over cleanly", "stop_restored": "{unit}: an earlier safety stop is back, it stays at 0 W", + "readings_late": "Home Assistant readings arrive late, control holds", + "readings_back": "Home Assistant readings are on time again", "battery_stuck": "{unit}: battery power reading stopped changing", "battery_back": "{unit}: battery power reading is used again", "battery_refuses": "The battery refuses to give power, the add-on asks 0 W", @@ -1488,6 +1509,8 @@ def status_samples() -> list: ("row_controller_stalled", RED, row_controller_stalled(45, 30)), ("row_meter_stale", RED, row_meter_stale(40, 10)), ("row_meter_waiting", AMBER, row_meter_waiting(30)), + ("row_readings_late", AMBER, row_readings_late(30)), + ("row_readings_late", AMBER, row_readings_late(30, True)), ("row_meter", GREEN, row_meter(-350, 2, 3, 340)), ("row_meter_frozen", RED, row_meter_frozen()), *(("row_freeze_off", AMBER, row_freeze_off(k)) for k in FREEZE_OFF), diff --git a/goodwe_controller/app/peakguard.py b/goodwe_controller/app/peakguard.py index b189970..356569a 100644 --- a/goodwe_controller/app/peakguard.py +++ b/goodwe_controller/app/peakguard.py @@ -446,6 +446,19 @@ def shaping_suspended(decision, peak_forecast_w, cap_w) -> bool: return decision is not None and decision.engaged +def cap_floor(decision, cap_w): + """SAFETY-128: the floor no higher than `cap_w` (the command standing while + the readings have not answered it), or `decision` unchanged when None. + DECISIONS/SAFETY-128-late-readings.md.""" + if (cap_w is None or decision is None or decision.floor_w is None + or decision.floor_w <= cap_w): + return decision + return replace( + decision, floor_w=float(cap_w), saturated=False, + reason=decision.reason + f", held at {float(cap_w):+.0f} W until the " + "readings answer the last power request") + + def claim(decision: PeakDecision) -> Claim | None: """The decision as one arbiter claim (`lo` only, `hi` stays `+inf`), or None.""" if not decision.engaged or decision.floor_w is None: diff --git a/goodwe_controller/app/ui_text/en.json b/goodwe_controller/app/ui_text/en.json index 969cc99..f60971c 100644 --- a/goodwe_controller/app/ui_text/en.json +++ b/goodwe_controller/app/ui_text/en.json @@ -476,6 +476,7 @@ "stopped": "Control is stopped. Every inverter is at 0 W." }, "attn": { + "late": "Readings arrive late", "home_assistant": "Home Assistant is not answering", "controller": "Control is not running", "meter": "Meter reading", diff --git a/goodwe_controller/app/ui_text/nl.json b/goodwe_controller/app/ui_text/nl.json index ab96551..7f250f3 100644 --- a/goodwe_controller/app/ui_text/nl.json +++ b/goodwe_controller/app/ui_text/nl.json @@ -397,6 +397,7 @@ "peak_not_covered": "Geen piekgrens: deze meter meet geen kwartieren.", "control": {"shadow": "Schaduwmodus: elke omvormer blijft op 0 W.", "running": "De regeling is actief.", "stopped": "De regeling is gestopt. Elke omvormer staat op 0 W."}, "attn": { + "late": "Metingen komen te laat", "home_assistant": "Home Assistant antwoordt niet", "controller": "Regeling is niet actief", "meter": "Meting van de meter", diff --git a/goodwe_controller/test_peakguard.py b/goodwe_controller/test_peakguard.py index bc237db..42e2f52 100644 --- a/goodwe_controller/test_peakguard.py +++ b/goodwe_controller/test_peakguard.py @@ -3080,13 +3080,13 @@ _unit_sources_unrouted = [ if CLAIM_SCOPE.get(r[0]) == SCOPE_UNIT and "," not in r[1]] print(f" {len(_direct)} direct appends, each read at its own line: " + ", ".join(f"main.py:{ln} {s} -> {a}" for s, a, ln in _routing)) -check("all SEVENTEEN of `Controller.cycle()`'s DIRECT appends route through " +check("all EIGHTEEN of `Controller.cycle()`'s DIRECT appends route through " "`self.actuator_of()`, so this scan can read the routing " "of EVERY one of them. An append that computed its actuator, or omitted " "the argument, or hand-wrote a string, fails HERE rather than being " "silently skipped - which is how a scan like this goes quietly vacuous, " "and 10f is in this file because that already happened once", - len(_direct) == 17 + len(_direct) == 18 and all(a is not None and a.startswith("self.actuator_of('") for _s, a, _ln in _routing)) check("⚠️ AND AT EVERY ONE OF THE ELEVEN THE LITERAL IT ROUTES THROUGH IS THE " diff --git a/goodwe_controller/test_safety128.py b/goodwe_controller/test_safety128.py new file mode 100644 index 0000000..a960979 --- /dev/null +++ b/goodwe_controller/test_safety128.py @@ -0,0 +1,408 @@ +"""Runnable check for SAFETY-128: control does not act on late readings. +`python3 test_safety128.py` + +When Home Assistant runs behind, its readings arrive late but still change, so +every staleness gate passes them. The loop corrected again, each cycle, an +error its last command had already fixed, and the evening give run swung with +growing export (SAFETY-127 rig: 10 s late, commands -450..+3 790 W, grid to +-1 947 W). The peak guard pushed to the inverter limit on the same late figures +(OPT-34 rig F-2). Now a command the battery reading has not answered holds the +loop and caps the peak guard (app/lateness.py), and readings found late hold, +then go to 0 W. DECISIONS/SAFETY-128-late-readings.md. + +The REAL Controller on TEST-23's plant (tools/safety128_rig.py). "Without the +gate" is the same run with `Controller._readings_late` returning None, which +is the tree before this card (the rig runs unchanged on 77fe31b). + +1. The give run (two batteries, house 1 800 W), readings 10 s late for 150 s + and 30 s late for 300 s: without the gate the swing grows to export below + -1 kW; with it the command stays near the last good one, no export below + -200 W, and it settles once the lag stops. N=1 too. +2. The peak guard in a quarter's last 30 s (a charge at the peak aim, readings + 30 s late, the house steps): it does not go to the inverter limit. +3. Seen: the late hold says so on the page (EN and NL) and logs once per + episode. +4. Nothing else moves: no-lag runs at 1, 2 and 5 s meters, a load step on a + battery with a 6 s dead time (SAFETY-75), start-up, a battery at its + cut-out level: within the stated tolerance of the run without the gate. +5. `Answer` and `compute(hold=)` at their edges. + +Mutations (MUTATION.md, purged copies) are listed in the pull request. +""" + +import logging +import os +import sys +from datetime import timedelta + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, HERE) +sys.path.insert(0, os.path.join(HERE, "tools")) +logging.disable(logging.WARNING) + +import opt11p3_replay as rp # noqa: E402 +import safety128_rig as R # noqa: E402 +import app.main as app_main # noqa: E402 +from app import control, lateness, messages # noqa: E402 + +fails = [] + + +def check(name, cond): + print(f" {'ok ' if cond else 'FAIL'} {name}") + if not cond: + fails.append(name) + + +def no_gate(fn): + """`fn()` with the answer gate off: the tree before SAFETY-128.""" + real = app_main.Controller._readings_late + app_main.Controller._readings_late = lambda self, standing_w: None + try: + return fn() + finally: + app_main.Controller._readings_late = real + + +def window(rows, t0, t1): + return [r for r in rows if t0 <= r[0] < t1] + + +def last_good(rows, t): + """The site command standing just before `t`.""" + return [r for r in rows if r[0] < t][-1][2] + + +def settled_at(rows, t0, band=150.0): + """First second from `t0` after which the grid stays within `band` W.""" + out = None + for r in rows: + if r[0] < t0: + continue + if abs(r[1]) > band: + out = None + elif out is None: + out = r[0] + return out + + +def swing(rows, t0, t1): + w = window(rows, t0, t1) + return max(r[1] for r in w) - min(r[1] for r in w) + + +# --------------------------------------------------------------------------- # +print("1. the give run with readings late") +ON = 300.0 +CASES = ((10.0, 150.0), (30.0, 300.0)) +for units in (("a", "b"), ("a",)): + house = 1800.0 if len(units) == 2 else 1500.0 + for lag, span in CASES: + off = ON + span + new = R.Rig(house=house, lag_s=lag, lag_on=ON, lag_off=off, + units=units) + new.run(off + 120.0) + old = R.Rig(house=house, lag_s=lag, lag_on=ON, lag_off=off, + units=units) + no_gate(lambda: old.run(off + 120.0)) + n = f"N={len(units)}, {lag:.0f} s late for {span:.0f} s" + wo, wn = window(old.rows, ON, off), window(new.rows, ON, off) + good = last_good(new.rows, ON) + half = ON + span / 2 + print(f" {n}: without the gate cmd {min(r[2] for r in wo):.0f}.." + f"{max(r[2] for r in wo):.0f} W, grid {min(r[1] for r in wo):.0f}" + f"..{max(r[1] for r in wo):.0f} W (swing {swing(old.rows, ON, half):.0f}" + f" then {swing(old.rows, half, off):.0f}); with it cmd " + f"{min(r[2] for r in wn):.0f}..{max(r[2] for r in wn):.0f} W " + f"(last good {good:.0f}), grid {min(r[1] for r in wn):.0f}.." + f"{max(r[1] for r in wn):.0f} W, settled " + f"{settled_at(new.rows, off) - off:.0f} s after") + if len(units) == 2: + check(f"{n}: without the gate the export goes below -1 kW and the " + f"swing grows (red on 77fe31b)", + min(r[1] for r in wo) < -1000.0 + and swing(old.rows, half, off) > swing(old.rows, ON, half)) + check(f"{n}: with it the command stays within 250 W of the last good " + f"one and no export below -200 W", + all(abs(r[2] - good) <= 250.0 for r in wn) + and min(r[1] for r in wn) >= -200.0) + check(f"{n}: settled within 30 s after the lag stops", + settled_at(new.rows, off) is not None + and settled_at(new.rows, off) - off <= 30.0) + +# --------------------------------------------------------------------------- # +print("\n2. the peak guard in a quarter's last 30 s") +# A charge sized to the peak aim (arbitrage on a cheap hour, the shape of the +# owner's night charge), readings 30 s late from 300 s; the house steps +# +2 000 W 100 s before the quarter ends (900 s). +PTS = [(R.BASE + timedelta(hours=h), 3600, 0.02 if h < 2 else 0.40) + for h in range(-3, 30)] + + +def peak_run(lag, gate=True, step_at=800.0): + seen = [] + + def sample(rig, c, t): + now = rig.start + timedelta(seconds=t) + if int(t) % 300 == 0: + c.tariff_plan = rp.plan_at(now, PTS) + c.tariff_at = now + g = c.peak_guard + if g is not None and 870.0 <= t < 900.0 and g.engaged: + seen.append((t, g.saturated, float(g.floor_w or 0.0), + float(c.target))) + r = R.Rig(house=lambda t: 1000.0 + (2000.0 if t >= step_at else 0.0), + lag_s=lag, lag_on=ON, lag_off=2000.0, soc=30.0, + extra={"arbitrage_enabled": True, + "cycle_cost_threshold_eur": 0.03}) + (lambda f: f() if gate else no_gate(f))(lambda: r.run(900.0, sample)) + return seen, r + + +RATING = sum(R.MAXW.values()) +for lag in (10.0, 30.0): + pg_new, _r = peak_run(lag) + pg_old, _r = peak_run(lag, gate=False) + top_new = max([s[2] for s in pg_new] or [0.0]) + top_old = max([s[2] for s in pg_old] or [0.0]) + print(f" {lag:.0f} s late: the guard's largest ask in the last 30 s " + f"without the gate {top_old:.0f} W ({sum(s[1] for s in pg_old)} " + f"cycles at the limit), with it {top_new:.0f} W " + f"({sum(s[1] for s in pg_new)})") + if lag == 30.0: + check("without the gate the guard goes to the inverter limit on the " + "late projection (red on 77fe31b)", + top_old >= RATING and any(s[1] for s in pg_old)) + check(f"{lag:.0f} s late: with it the guard never goes to the limit and " + f"never says it is saturated", + top_new < RATING and not any(s[1] for s in pg_new)) + +# --------------------------------------------------------------------------- # +print("\n3. the page and the log say so") +rows_seen = [] + + +def watch(rig, c, t): + rows_seen.extend((t, r) for r in c.checks() if r.get("id") == "late") + + +late = R.Rig(lag_s=30.0, lag_on=ON, lag_off=ON + 300.0) +c = late.run(ON + 300.0 + lateness.FIT_GONE_S + 60.0, watch) +log = [e for e in c.events if "readings late" in e or "readings on time" in e] +n_late = sum(1 for e in log if "readings late" in e) +n_back = sum(1 for e in log if "readings on time" in e) +first = min([t for t, _r in rows_seen] or [None]) +lastr = max([t for t, _r in rows_seen] or [None]) +print(f" 30 s late for 300 s: {n_late} late episode(s), {n_back} end(s), " + f"the row from {first} to {lastr} s ({len(rows_seen)} s); " + f"{rows_seen[0][1]['text'] if rows_seen else ''}") +_d = c.answer.delay_s +check("found late within a minute of the lag starting, and an amber row says " + "so with the delay it measured, in plain words, EN and NL", + first is not None and first - ON <= 60.0 + and all(r["warn"] and not r["ok"] for _t, r in rows_seen) + and str(rows_seen[-1][1]["text"]) in { + str(messages.row_readings_late(d)) for d in range(20, 33)} + and rows_seen[-1][1]["text"].nl.startswith( + "Metingen uit Home Assistant komen zo'n ")) +check("logged once for the episode, and once when it ends, not per cycle", + n_late == 1 and n_back == 1) +check("the row goes once a fit shows the readings on time, or 2 minutes " + "after the last fit, at most 160 s after the lag (a steady site shows " + "nothing to fit)", + lastr is not None + and lastr - (ON + 300.0) <= lateness.FIT_S + lateness.FIT_GONE_S + and not [r for r in c.checks() if r.get("id") == "late"]) +# The late verdict through the real cycle(): readings found late (30 s), then +# the gate is frozen with its verdict 5 s old (held) or 20 s old (past +# stale_input_s, 15 s). +seen_zero = {} + + +def verdict(after_s): + def at(rig, c, t): + if t == 500.0: + c.answer.observe = lambda *a, **k: None + c.answer.late_seen, c.answer.late_at = True, c._now_mono - after_s + seen_zero[after_s] = [c.target] + if t == 506.0: + seen_zero[after_s] += [ + c.target, c.reason, [r["text"] for r in c.checks() + if r.get("id") == "late"], c.why_words] + v = R.Rig(lag_s=30.0, lag_on=ON, lag_off=2000.0) + v.run(507.0, at) + + +verdict(5.0) +verdict(20.0) +held, zero = seen_zero[5.0], seen_zero[20.0] +print(f" late verdict 5 s old: {held[:3]}; 20 s old: {zero[:3]}") +check("the late verdict holds the request until stale_input_s, then every " + "inverter goes to 0 W, and the row says so (EN and NL)", + held[1] == held[0] and held[2].startswith("readings late") + and zero[1] == 0.0 and zero[4].startswith( + "Home Assistant readings arrive late: 0 W") + and len(zero[3]) == 1 and "every inverter is at 0 W" in zero[3][0] + and "omvormers op 0 W" in zero[3][0].nl) +on_time = R.Rig(house=lambda t: 600.0 + (2500.0 if t >= 200.0 else 0.0)) +c = on_time.run(600.0) +check("readings on time, a load step: no late episode, no row", + not [e for e in c.events if "readings late" in e] + and not c.answer.late_seen) + +# --------------------------------------------------------------------------- # +print("\n4. nothing else moves") +STEP = dict(house=lambda t: 600.0 + (2500.0 if t >= 200.0 else 0.0)) + + +def compare(name, tol_w, **kw): + a = R.Rig(**kw) + a.run(500.0) + b = R.Rig(**kw) + no_gate(lambda: b.run(500.0)) + diff = max(abs(x[2] - y[2]) for x, y in zip(a.rows, b.rows)) + sa, sb = settled_at(a.rows, 200.0), settled_at(b.rows, 200.0) + print(f" {name}: largest command difference {diff:.0f} W, settled " + f"{sa - 200.0 if sa else None} s after the step (without the gate " + f"{sb - 200.0 if sb else None} s)") + return diff, sa, sb, a + + +for p1 in (1.0, 2.0, 5.0): + for units in (("a", "b"), ("a",)): + d, sa, sb, _a = compare(f"no lag, {p1:.0f} s meter, N={len(units)}, " + f"house 600 -> 3 100 W", 0.0, p1_s=p1, + units=units, **STEP) + check(f"{p1:.0f} s meter, N={len(units)}: settles within 10 s of the " + f"run without the gate", + sa is not None and sb is not None and sa - sb <= 10.0) +d, sa, sb, a75 = compare("SAFETY-75 shape, 6 s dead time", 0.0, dead_s=6.0, + **STEP) +check("6 s dead time (SAFETY-75): the same commands as without the gate, and " + "no late episode (the fit says 5-7 s, under 8 s)", + d == 0.0 and not any("readings late" in e for e in a75.c.events)) +d, sa, sb, a0 = compare("start-up, the house at 2 000 W from the first second", + 0.0, house=2000.0) +check("start-up: the first 120 s reach the house within 30 s, as without the " + "gate, and no late episode", + settled_at(a0.rows, 0.0) is not None and settled_at(a0.rows, 0.0) <= 30.0 + and not any("readings late" in e for e in a0.c.events)) +import safety94_rig as s94 # noqa: E402 +slow = [(n, tau, tick) for n in (1, 2) for tau in (10.0, 20.0) + for tick in (1.0, 2.0) + if any("readings late" in e for e in s94.run( + n, s94.reversal(300.0, -2500.0, 800.0), 600.0, tau=tau, + tick=tick)["c"].events)] +check("a slow battery (10 or 20 s lag, SAFETY-85/94's rig) reversing out of a " + "charge, readings on time: never taken for late readings, N=1 and N=2, " + f"1 and 2 s meters (taken: {slow})", not slow) +for name, kw in (("at their cut-out level", dict(soc=10.5)), + ("A refusing from 200 s", + dict(cut=lambda u, t: u == "a" and t >= 200.0))): + x = R.Rig(house=1800.0, **kw) + x.run(600.0) + y = R.Rig(house=1800.0, **kw) + no_gate(lambda: y.run(600.0)) + check(f"readings on time, batteries {name}: not taken for late " + f"readings, the same commands as without the gate", + not any("readings late" in e for e in x.c.events) + and [r[2] for r in x.rows] == [r[2] for r in y.rows]) + +# --------------------------------------------------------------------------- # +print("\n5. the rules at their edges") +check("LATE_S is SAFETY-75's slowest healthy answer: 6 + 6 + 10 + 1 = 23 s", + lateness.LATE_S == 23.0) +check("ANSWER_W is a probe's answer (100 W)", + lateness.ANSWER_W == control.PROBE_ANSWER_W == 100.0) + + +check("DELAY_S is the plant's 6 s settle and one 2 s meter tick", + lateness.DELAY_S == 8.0) + + +def fit(delay, *, follows=True, step_every=20, noise=40.0, dur=300): + """`Delay` on a meter that shows each command `delay` s later (a house + of 2 000 W with +-`noise`), or never (`follows` False).""" + import random + rng, d = random.Random(7), lateness.Delay() + edges = [0] + while edges[-1] < dur: + edges.append(edges[-1] + step_every * rng.uniform(0.5, 1.5)) + cmd = lambda t: 1500.0 + (400.0 if sum(e <= t for e in edges) % 2 else 0.0) + for t in range(0, dur, 2): + shown = cmd(t - delay) if follows and t >= delay else 1500.0 + d.add(float(t), cmd(t), 2000.0 - shown + rng.uniform(-noise, noise)) + return d.estimate() + + +got = {k: fit(k) for k in (0, 6, 10, 30, 60)} +print(f" fitted delays: {got}") +check("the fit finds the meter's delay (0, 6, 10, 30, 60 s) to within 1 s", + all(v is not None and abs(v - k) <= 1.0 for k, v in got.items())) +check("a battery that does not follow gives no fit, and so does a command " + "that does not move", + fit(10, follows=False) is None and fit(10, step_every=10_000) is None) + + +def gate(steps, exempt=(), late_seen=True): + """[(t, command, battery reading, meter)] through one Answer, its fit + replaced by `late_seen`; after each: (pending, hold, late s, peak cap).""" + a = lateness.Answer() + a.delay.estimate = lambda: (30.0 if late_seen else 0.0) + + def one(t, cmd, batt, grid): + a.observe(float(t), cmd, batt, grid_w=grid, exempt=t in exempt) + return (a.ask is not None, a.hold, a.late_s(float(t)), + a.peak_cap(cmd)) + return a, [one(*x) for x in steps] + + +STEP = [(0, 1000, 1000, 500), (2, 1500, 1000, 500)] +HELD = STEP + [(t, 1500, 1000, 500) for t in range(4, 40, 2)] +a, out = gate(HELD, late_seen=False) +check("readings on time: an unanswered ask holds nothing, caps nothing and " + "is never late (a battery at its limit)", + out[-1][0] and all(o[1] == 0 and o[2] is None and o[3] is None + for o in out)) +a, out = gate(STEP + [(4, 1500, 1000, 500), (6, 1500, 1450, 60)]) +check("readings late: the ask holds the loop up and caps the peak guard at " + "what stands until the battery reading answers", + out[1][1:] == (1, None, 1500.0) and out[2][1:] == (1, None, 1500.0) + and out[3] == (False, 0, None, None)) +a, out = gate(HELD) +check("...and with no move towards it for 23 s it is late (control holds, " + "then 0 W)", + [o[2] for o in out].index(0.0) == len([t for t, *_ in HELD + if t - 2 < 23])) +a, out = gate(HELD + [(40, 1500, 1000, 380)]) +check("...which ends when a reading starts towards the ask", + out[-2][2] is not None and out[-1][2] is None and out[-1][0]) +a, out = gate(HELD, exempt=set(range(2, 40, 2))) +check("exempt (a battery gives nothing under its ask, or refuses): no hold " + "and no late verdict, the cap stays", + all(o[1] == 0 and o[2] is None for o in out) and out[-1][3] == 1500.0) +a, out = gate([(0, 1000, 1000, 500), (2, 1090, 1000, 500), + (4, 1000, 1000, 500)]) +check("a move under 100 W is not an ask", not any(o[0] for o in out)) +a, out = gate(STEP + [(4, 1050, 1000, 500)]) +check("an ask taken back within 100 W of the reading is withdrawn", + out[2][0] is False) +T = control.Tuning(max_w=5000.0) +base = dict(prev_w=1500.0, grid_w=800.0, actual_w=1000.0, tuning=T, + i_w=1500.0) +check("compute(hold=+1): the loop may not push past the standing command, " + "but may come back", + control.compute(**base).target_w > 1500.0 + and control.compute(**base, hold=1).target_w == 1500.0 + and control.compute(**base, hold=1).reason == control.WAITING + and control.compute(**dict(base, grid_w=-800.0), hold=1).target_w + < 1500.0) +check("compute(hold=-1) mirrors it, and hold=0 is the law as it was", + control.compute(**dict(base, grid_w=-800.0), hold=-1).target_w == 1500.0 + and control.compute(**base, hold=0) == control.compute(**base)) + +print() +if fails: + print(f"{len(fails)} FAILED: " + ", ".join(fails)) + sys.exit(1) +print("all checks passed") diff --git a/goodwe_controller/test_safety94.py b/goodwe_controller/test_safety94.py index c7dfc51..512834c 100644 --- a/goodwe_controller/test_safety94.py +++ b/goodwe_controller/test_safety94.py @@ -61,6 +61,9 @@ def without_fix(fn): """Run `fn` with the gate's old clock and the old refusal evidence.""" real_f, real_c = app_inverter.Freshness, app_main.compute real_p = app_main.REFUSED_PROGRESS_W + real_l = app_main.Controller._readings_late + # SAFETY-128: that tree had no late-readings gate either. + app_main.Controller._readings_late = lambda self, standing_w: None app_inverter.Freshness = Before94 app_main.compute = lambda *a, **k: real_c(*a, **dict(k, gave_nothing=None)) app_main.REFUSED_PROGRESS_W = float("inf") @@ -69,6 +72,7 @@ def without_fix(fn): finally: app_inverter.Freshness, app_main.compute = real_f, real_c app_main.REFUSED_PROGRESS_W = real_p + app_main.Controller._readings_late = real_l # ============================================================================ # diff --git a/goodwe_controller/test_safety96.py b/goodwe_controller/test_safety96.py index af33b81..85985d4 100644 --- a/goodwe_controller/test_safety96.py +++ b/goodwe_controller/test_safety96.py @@ -68,10 +68,14 @@ def without(fn): Controller._held_out_w = lambda self, live: 0.0 Controller._reseed_drop = lambda self, live: setattr( self, "_live_was", frozenset(live)) + real_late = Controller._readings_late + # SAFETY-128: that tree had no late-readings gate either. + Controller._readings_late = lambda self, standing_w: None try: return fn() finally: Controller._held_out_w, Controller._reseed_drop = fixed, real_drop + Controller._readings_late = real_late def run(n=2, *, house=lambda t: 2400.0, dur=600.0, store=None, opt=None, diff --git a/goodwe_controller/tools/safety128_rig.py b/goodwe_controller/tools/safety128_rig.py new file mode 100644 index 0000000..3d97eec --- /dev/null +++ b/goodwe_controller/tools/safety128_rig.py @@ -0,0 +1,284 @@ +#!/usr/bin/env python3 +"""SAFETY-128: the REAL Controller with Home Assistant readings arriving late. + + python3 tools/safety128_rig.py # prints a few runs' summaries + +The plant is TEST-23's (SAFETY-127 rig, t127_plant.py): two batteries +(A 5 000 W, B 4 600 W) that ramp at 5 000 W/s and then follow a 1 s lag; each +battery reading is published every 10 s (A at :x6, B at :x1.6); the meter every +2 s; the house is `house` W plus seeded +-`noise` W, new each second. + +`p1_s` sets the meter's period, `dead_s` a command dead time (SAFETY-75's +laggy battery), `units` one battery ("a") or two. + +Home Assistant runs `lag_s` seconds behind (a number, or a function of t) between `lag_on` and `lag_off`: +every reading the add-on gets (meter, battery power, battery level) is the +one published `lag_s` seconds earlier. When the lag starts, nothing new comes +for `lag_s` seconds, then the backlog; when it stops, the readings jump to +now. Commands reach the plant at once (t127_shim.py's lag mode). + +The add-on reads every 1 s and runs a cycle on a new meter value or after +`cycle_floor_s`, as `run_control()` does. Uses only the Controller API that +release/1.0 77fe31b has, so the same file runs on the tree before SAFETY-128. +""" + +import asyncio +import math +import os +import random +import sys +import tempfile +import time +from datetime import datetime, timedelta, timezone + +sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) + +import app.main as m # noqa: E402 +from app.main import Controller # noqa: E402 +from app.p1 import make_sample, split_signed # noqa: E402 + +# 18:00 local (CET) on a weekday: the evening give run. +BASE = datetime(2026, 11, 4, 17, 0, tzinfo=timezone.utc) +UNITS = ("a", "b") +MAXW = {"a": 5000.0, "b": 4600.0} +CAPWH = {"a": 15360.0, "b": 20480.0} +PHASE = {"a": 6.0, "b": 1.6} +BPER = 10.0 +P1PER = 2.0 +RAMP = 5000.0 +LAG = 1.0 + +def options(units=UNITS): + o = dict(OPTIONS) + o["inverters"] = [ + {"name": f"inverter-{u}", "setpoint_entity": f"number.sp_{u}", + "soc_entity": f"sensor.soc_{u}", "batt_entity": f"sensor.batt_{u}", + "max_w": int(MAXW[u]), "bms_soc_floor": 10, + "capacity_kwh": CAPWH[u] / 1000.0} for u in units] + return o + + +OPTIONS = { + "meter_entity": "sensor.grid", "meter_source": "ha_signed", + "p1_net_entity": "sensor.grid", "meter_phases": 1, "meter_max_age_s": 30, + "peak_cap_w": 3500, "peak_margin_w": 50, "peak_soc_floor": 20, + "gain": 0.6, "slew_w": 0, "target_grid_w": -10, "deadband_w": 15, + "step_w": 10, "heartbeat_s": 10, "stale_input_s": 15, + "cycle_floor_s": 15, "saturation_w": 500, "saturation_cycles": 3, + "auto_start": True, "battery_modes_enabled": False, + "arbitrage_enabled": False, "bms_soc_floor": 10, +} + + +class _Store: + def __init__(self): + from app.store import DEFAULTS + self.data = dict(DEFAULTS) + + def set(self, k, v): + self.data[k] = v + + def get_time(self, k): + v = self.data.get(k) + return datetime.fromisoformat(v) if v else None + + def set_time(self, k, when): + self.data[k] = when.isoformat() if when else None + + +class _Pub: + enabled = False + + def publish(self, *a, **k): + pass + + def close(self): + pass + + +class Plant: + """The batteries, their registers and the meter, in 0.1 s steps.""" + + def __init__(self, house, noise, seed, soc, cut=None, units=UNITS, + p1_s=P1PER, dead_s=0.0): + self.house_fn, self.noise, self.seed = house, noise, seed + self.units, self.p1_s, self.dead_s = units, p1_s, dead_s + self.cut = cut or (lambda u, t: False) + self.cmd = {u: 0.0 for u in units} + self.queue = [] # (t, unit, command), dead time + self.tgt = {u: 0.0 for u in units} + self.act = {u: 0.0 for u in units} + self.soc = {u: float(soc) for u in units} + self.reg = {u: 0.0 for u in units} + self.slot = {u: None for u in units} + self.p1 = None + self.p1_n = 0 + self.t = 0.0 + self.house = float(house(0.0)) + self.hist = [] # (t, p1 value, p1 index, {reg}, {soc}) + + def grid(self): + return self.house - sum(self.act.values()) + + def advance(self, t): + while self.t < t - 1e-9: + self.t = round(self.t + 0.1, 6) + sec = int(math.floor(self.t + 1e-9)) + rs = random.Random(self.seed * 1000003 + sec) + self.house = (float(self.house_fn(self.t)) + + rs.uniform(-self.noise, self.noise)) + while self.queue and self.queue[0][0] <= self.t + 1e-9: + _t, u, v = self.queue.pop(0) + self.cmd[u] = v + for u in self.units: + want = max(-MAXW[u], min(MAXW[u], self.cmd[u])) + if self.cut(u, self.t): + want = 0.0 + if self.soc[u] <= 10.0 and want > 0: + want = 0.0 + if self.soc[u] >= 100.0 and want < 0: + want = 0.0 + d = want - self.tgt[u] + self.tgt[u] += math.copysign(min(abs(d), RAMP * 0.1), d) + self.act[u] += ((self.tgt[u] - self.act[u]) + * (1 - math.exp(-0.1 / LAG))) + self.soc[u] = max(0.0, min(100.0, self.soc[u] - self.act[u] + * 0.1 / 3600.0 / CAPWH[u] * 100.0)) + slot = math.floor((self.t - PHASE[u]) / BPER + 1e-9) + if slot != self.slot[u]: + self.slot[u] = slot + self.reg[u] = float(round(self.act[u])) + if abs(self.t / self.p1_s - round(self.t / self.p1_s)) < 1e-6: + self.p1 = round(self.grid(), 1) + self.p1_n += 1 + self.hist.append((self.t, self.p1, self.p1_n, dict(self.reg), + {u: round(v, 2) for u, v in self.soc.items()})) + del self.hist[:-3000] + + def view(self, t, lag_s, lag_on, lag_off): + """What Home Assistant shows at `t`: (p1, p1 index, regs, socs).""" + lag_s = lag_s(t) if callable(lag_s) else lag_s + if lag_s and lag_on <= t < lag_off: + want = max(lag_on, t - lag_s) + for row in reversed(self.hist): + if row[0] <= want + 1e-9: + return row[1:] + return self.hist[0][1:] + return (self.p1, self.p1_n, dict(self.reg), + {u: round(v, 2) for u, v in self.soc.items()}) + + +class _Hass: + def __init__(self, rig): + self.rig = rig + + async def limits(self, e): + return (-MAXW[e[-1]], MAXW[e[-1]], 1.0) + + async def call(self, *a): + return True + + async def state(self, e): + return None + + async def number(self, e, invert=False): + _p1, _n, regs, socs = self.rig.seen + u = e[-1] + if e.startswith("sensor.soc"): + return socs[u] + if e.startswith("sensor.batt"): + return regs[u] + return None + + async def set_number(self, e, v): + p = self.rig.plant + p.queue.append((p.t + p.dead_s, e[-1], float(v))) + self.rig.writes.append((self.rig.plant.t, e[-1], float(v))) + return True + + +class Rig: + def __init__(self, house=1800.0, noise=50.0, seed=90, soc=60.0, + lag_s=0.0, lag_on=0.0, lag_off=0.0, extra=None, cut=None, + start=BASE, units=UNITS, p1_s=P1PER, dead_s=0.0): + house_fn = house if callable(house) else (lambda t, h=house: h) + self.plant = Plant(house_fn, noise, seed, soc, cut, units, p1_s, + dead_s) + self.lag = (lag_s, lag_on, lag_off) + self.o = dict(options(units), **(extra or {})) + self.start = start + self.writes = [] + self.rows = [] + self.seen = None + + def run(self, dur, sample=None): + """Run `dur` s. Rows: (t, true grid, site command, site actual, + loop reason, controller reason). `sample(rig, c, t)` per second.""" + async def go(): + db = os.path.join(tempfile.mkdtemp(prefix="s128-"), "tariff.db") + c = Controller(self.o, _Hass(self), _Store(), _Pub(), + stop=asyncio.Event(), peak_db_path=db) + for i in c.inverters: + i.dev_min, i.dev_max = -MAXW[i.name[-1]], MAXW[i.name[-1]] + c.auto = True + self.c = c + real_dt = m.datetime + clock = [self.start] + + class _DT(real_dt): + @classmethod + def now(cls, tz=None): + return clock[0] if tz is None else clock[0].astimezone(tz) + + m.datetime = _DT + floor = float(self.o.get("cycle_floor_s", 15)) + last_grid, last_cycle, last_n = object(), None, None + try: + t = 0.0 + while t < dur: + self.plant.advance(t) + now = self.start + timedelta(seconds=t) + clock[0] = now + self.seen = self.plant.view(t, *self.lag) + p1, n = self.seen[0], self.seen[1] + if p1 is not None and n != last_n: + last_n = n + imp, exp = split_signed(p1) + c.p1.submit(make_sample( + "ha_signed", imp, exp, phases=1, ingest_ts=now, + ingest_mono=time.monotonic())) + await c.read_inputs(now_mono=t) + if (c.grid != last_grid or last_cycle is None + or t - last_cycle >= floor): + last_grid, last_cycle = c.grid, t + await c.cycle() + self.rows.append(( + t, self.plant.grid(), sum(self.plant.cmd.values()), + sum(self.plant.act.values()), c.loop_reason, + c.reason)) + if sample is not None: + sample(self, c, t) + t += 1.0 + finally: + m.datetime = real_dt + return c + return asyncio.run(go()) + + +def summary(rows, t0, t1): + """Over [t0, t1): (min cmd, max cmd, min grid, max grid, s of export + below -200 W, s below -1 000 W).""" + sel = [r for r in rows if t0 <= r[0] < t1] + return (min(r[2] for r in sel), max(r[2] for r in sel), + min(r[1] for r in sel), max(r[1] for r in sel), + sum(1 for r in sel if r[1] < -200), sum(1 for r in sel + if r[1] < -1000)) + + +if __name__ == "__main__": + for lag in (0.0, 10.0, 30.0): + r = Rig(lag_s=lag, lag_on=300.0, lag_off=450.0) + r.run(600.0) + print(f"lag {lag:4.0f} s: before {summary(r.rows, 200, 300)}") + print(f" during {summary(r.rows, 300, 450)}") + print(f" after {summary(r.rows, 480, 600)}") -- 2.54.0 From d3f4f9f48ba81e6e25f6223e5a8d979a63ac06a2 Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Mon, 2 Nov 2026 04:32:04 +0100 Subject: [PATCH 02/22] SAFETY-128 round 2: find late readings from the batteries, not the meter alone Review B1: the meter fit took repeating house loads (ovens, a heat pump) for late readings on healthy 1-2 s sites, about 9 % of the recorded house. B2: at a 5 s meter it never ran. M1: no hysteresis. M2: an unbounded check. lateness.Delay now fits how long each battery's reading takes to show its own request, which the house does not move, from the 1 s reads (read_inputs -> Answer.watch). The meter fit stays for speed at 1-3 s meters, but counts only when an ask was slow to start and no battery fit said on time. Late from 9.5 s after 2 fits, on time under 6.5 s, dropped after 120 s without a fit. Fits are bounded (a few ms). Readings on time: no verdict in any reviewer scenario (22 shapes at 1/2/5 s meters, 7 harm runs) nor in 2 days of the recorded house at 1, 2 and 5 s ticks; digests equal to 77fe31b. Late: found 36-68 s after the start at 1-2 s, about 3 min at 5 s. The row gives the batteries' delay, or no figure while only the meter has shown it. Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_01Bdqr7PiaHcBnzPituu2yTC --- .../DECISIONS/SAFETY-128-late-readings.md | 126 +++---- goodwe_controller/app/lateness.py | 333 +++++++++++++----- goodwe_controller/app/main.py | 12 +- goodwe_controller/app/messages.py | 28 +- goodwe_controller/test_safety128.py | 159 +++++++-- 5 files changed, 462 insertions(+), 196 deletions(-) diff --git a/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md b/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md index e03642f..41fd33a 100644 --- a/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md +++ b/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md @@ -2,72 +2,80 @@ **Status: FIXED on release/1.0 (after 0.20.0).** Code: `app/lateness.py`, `control.compute(hold=, hold_w=)`, `peakguard.cap_floor()`, -`Controller._readings_late()`. Tests: `test_safety128.py` -(`tools/safety128_rig.py`, TEST-23's plant). +`Controller._readings_late()`, `Controller.read_inputs()` (feeds the fits). +Tests: `test_safety128.py` (`tools/safety128_rig.py`, TEST-23's plant). **Cause** (SAFETY-127 report). Home Assistant can deliver readings late but still changing. Every gate judges arrival (SAFETY-01) or a value that stops -changing (SAFETY-18, SAFETY-37), so a late reading passes all of them. The loop -then corrects, each cycle, an error its last command has already fixed: a loop -with a dead time far longer than its cycle, which swings and grows. The peak -guard projects the quarter from the same late figures and pushes to the +changing (SAFETY-18, SAFETY-37), so a late reading passes them all. The loop +then corrects, each cycle, an error its last command has already fixed: a +loop with a dead time far longer than its cycle, which swings and grows. The +peak guard projects the quarter from the same late figures and pushes to the inverter limit in its last seconds (OPT-34 rig F-2). **Rule.** -1. *Find late readings.* `Delay` fits, every 4 s over the last 40 s of new - meter values, how long after a power request the meter shows it: the meter - is the house less the batteries, so `meter(t) + request(t - d)` is flattest - at the right `d`. Over `DELAY_S` the readings are late (`late_seen`). The - fit counts only if the requests explain half the meter's swing; a late `d` - only if it fits 30 % better than any fresh one and better than a slow - battery answering at once (10 or 20 s lag, the SAFETY-85/94 rigs). A - silent meter adds no values (SAFETY-99/125). A battery that gives nothing - under its ask, or refuses, spoils the fit and is left out for 40 s. With no - fit for 120 s (a steady site shows nothing) the verdict is dropped. - Nothing below acts unless `late_seen`, so with readings on time control is - unchanged. -2. *Hold.* While `late_seen`, a request more than `ANSWER_W` from the battery - reading is pending until that reading comes within `ANSWER_W` of it, or half - way. Meanwhile the loop may not lead the reading by more than `ANSWER_W` - that way (`compute(hold=, hold_w=)`; it may come back), and the peak - guard's floor is capped at the request that stands (`cap_floor()`). The - loop is not held while a battery gives nothing under its ask or refuses: - SAFETY-52/94 need it to go on asking. -3. *Late.* Pending for `LATE_S` with no reading moving `ANSWER_W` towards it is - the late verdict: control holds, then after `stale_input_s` goes to 0 W - ("readings late"), as for a missing input. It ends when a reading starts - towards the request. -4. *Seen.* While `late_seen` an amber row says "Home Assistant readings arrive - about N seconds late, so control holds the batteries." (NL too), and - "every inverter is at 0 W" past the hold. One WARNING (`READINGS_LATE`) - and one action-log line per episode, and one when it ends. +1. *Find late readings* (`Answer.watch()`, every 2 s, from the 1 s reads). + - **Batteries** (`Delay`): fit one `d` so that each battery's reading minus + its own request at `t - d` (averaged over 2 s for the battery's lag and + the read) is flattest. The house does not move that reading, so a + repeating load cannot line up with it (review round 1, B1). Needs 5 new + readings in 60 s and the request and the reading each to move 300 W. + Late from 9.5 s, after 2 late fits in a row. A late fit must explain 75 % + and beat every fit up to 6.5 s by 2x; the shortest delay that fits about + as well (within 2x, or within 2 500 W^2) is taken instead, since a swing's + period fits again. A slow battery (3-20 s lag) answering at once gives no + late fit (checked in the test). No late battery fit counts for 60 s + after a battery gave nothing under its ask: an emptied battery lined its + old movement up as 85 s late. + - **Meter** (`MeterFit`, round 1's fit): `meter(t) + request(t - d)` is the + house alone at the right `d`. Quick, but confounded by the house. Needs + 16 new values in 60 s, so only a meter of about 3.5 s or faster: at 5 s + it gave false delays. Late over 8 s. + - A meter fit says late only with evidence from the batteries' side too: + a request more than 100 W from the battery reading took 3 s or more to + move a reading towards it (`SLOW_START_S`; on time they start within + 0-1 s in the recorded house at 1 and 2 s ticks), and no battery fit in + the last 60 s said on time (under 9.5 s). An ask during which a battery + gave nothing under its ask is no slow evidence. A repeating load fools + the meter fit but not these. No meter verdict in the first 150 s. + - On time again on a battery fit under 6.5 s, or a meter fit within 8 s + unless a battery fit said late (hysteresis). With no fit for 120 s the + verdict is dropped. +2. *Hold.* While late, a request more than 100 W from the battery reading is + pending until that reading comes within 100 W of it, or half way. Meanwhile + the loop may not lead the reading by more than 100 W that way, and the peak + guard's floor is capped at the request that stands. The loop is not held + while a battery gives nothing under its ask or refuses (SAFETY-52/94). +3. *Late verdict.* Pending for `LATE_S` (23 s) with no reading moving 100 W + towards it: control holds, then after `stale_input_s` goes to 0 W + ("readings late"), as for a missing input. +4. *Seen.* While late an amber row says "Home Assistant readings show requests + about N seconds late, so control holds the batteries." (NL too). N is the + batteries' fitted delay: Home Assistant's lag plus the battery's own + answer. While only the meter has shown it the row gives no figure. One + WARNING (`READINGS_LATE`) per episode and one line when it ends. -**Constants** (`app/lateness.py`). - -| | Value | Why | -|---|---|---| -| `ANSWER_W` | 100 W | `control.PROBE_ANSWER_W`: a reading this far towards the ask has answered it (SAFETY-52's probe). | -| `DELAY_S` | 8 s | `freshness.PLANT_SETTLE_S` 6 s + one `LOOP_TICK_S` 2 s: a fresh meter shows a request within the plant's settle (SAFETY-75's 6 s dead time) and one meter tick. The fit reads 0 s at 1/2/5 s meters, 5-7 s with a 6 s dead time, 9 s at 10 s late. | -| `LATE_S` | 23 s | 2 x `PLANT_SETTLE_S` + `PUBLISH_PERIOD_S` 10 s + `READ_PERIOD_S` 1 s: SAFETY-75's 6 s dead time, the 6 s settle, one publish and one read, the slowest healthy battery answer. | -| `FIT_S`, `FIT_EVERY_S` | 40 s, 4 s | Short enough to find 10 s and 30 s late within 50-60 s of the start (the swing has not grown yet); one 2 s cycle in two. | -| `MAX_DELAY_S` | 90 s | F-2 measured 17-78 s. | -| `FIT_SHARE`, `FIT_BETTER` | 0.5, 0.7 | Below these a battery that stops following (a refusal at N=2) was read as 23 s late. | -| `SLOW_TAUS_S` | 10 s, 20 s | The SAFETY-85/94 lagging battery: without this check 16 of 36 of its reversals read as 11-56 s late. | -| `FIT_GONE_S` | 120 s | The verdict outlives the lag by at most `FIT_S` + this when the site goes steady. | +**With readings on time** no verdict was reached in any reviewer scenario +(owner-shaped house, ovens 20/40 to 60/60 s, clouds, a 300 W/s battery, 6-10 s +dead times, HA jitter 0-4 s, an EV following the meter, capped batteries, HA +restarts, meter pauses; 1, 2 and 5 s meters) nor in 2 days of the recorded +house at 1, 2 and 5 s ticks, so the commands are the same as without the gate. +The fits run all the time; the pull request has the counts. **What it costs.** -- Late readings are found only after the requests have moved 100 W and the - meter has shown it late: 48-68 s after the lag starts in the tests. Until - then control is as before. -- The row can stay up to 160 s after the readings are on time again, while - the site is steady. -- A battery that cannot follow while readings are late (at a limit, not - reading 0 W) is held, then set to 0 W after 38 s, until it moves. -- The fit costs about 93 x 20 sums every 4 s. -- With readings late, a large step is followed in steps of the lag: the loop - waits for each request to show before the next. +- Found 36-68 s after a lag starts at 1-2 s meters, about 3 minutes at a 5 s + meter (battery fit only; the meter fit needs a faster meter). Until then control is as before. At 5 s and 30 s + late the export is cut to about a third, not removed. +- A battery whose own answer takes about 10 s is close to the threshold: an + 8 s dead time reads 7, 10 s reads 9, HA 10 s late reads 10. +- The row can stay up to 180 s after the readings are on time on a steady + site. While only the meter has shown the delay it gives no figure. +- A battery that cannot follow while the readings are late (at a limit, not + reading 0 W) is held, then set to 0 W after 38 s. +- The fits cost a few ms every 2 s (x86, measured after 3 idle days); the + history they read is bounded (60 s of readings, 150 s of requests). -**Left out.** The round trip setpoint -> Home Assistant: the fitted delay gives -the same figure from readings the add-on already has. The arbitrage charge's -own sizing on late readings (`charge_room_w()`) is not capped here; the test -shows it swinging while the peak guard holds (see the pull request). +**Left out.** The setpoint -> Home Assistant round trip: the fitted delay gives +the figure from readings the add-on already has. The hold's step-and-pull-back +at extreme lag and the arbitrage charge's sizing on late readings +(`charge_room_w()`) are follow-up cards. diff --git a/goodwe_controller/app/lateness.py b/goodwe_controller/app/lateness.py index c8b8f51..d673c0b 100644 --- a/goodwe_controller/app/lateness.py +++ b/goodwe_controller/app/lateness.py @@ -3,122 +3,221 @@ When Home Assistant delivers readings late but still changing, every gate that judges arrival (SAFETY-01) or an unchanged value (SAFETY-18, SAFETY-37) passes them, and the loop corrects again, each cycle, an error its last command has -already fixed. The readings themselves show it: after a command the meter moves -the other way and the battery reading follows (r = 0.998, freshness.py), and -late readings do both late. +already fixed. The readings show it: they answer each power request late. -`Answer` watches the site command on the wire, the meter and the site battery -reading (less the batteries' idle reading, SAFETY-97): -- `Delay` fits how long after a command the meter shows it. Over DELAY_S the - readings run late (`late_seen`); within it they do not. A battery that does - not follow gives no fit, so no evidence either way; +`Answer.watch()` (each 1 s read) fits two delays every FIT_EVERY_S: +- `Delay`: how long after its request each battery's reading shows it. The + house does not move that reading, so a repeating load cannot fool it. It + needs the request to move FIT_MOVE_W; one reading per 10 s makes it slow. +- `MeterFit`: how long after the site request the meter shows it. Quick at a + 1-3 s meter, none at a slower one, and a house load that repeats can fool it. +The meter counts only when an ask was slow to start and no battery fit said +"on time"; with hysteresis (ON_S / OFF_S) the result is `late_seen`. Then (`observe()`, each cycle): - a new command more than ANSWER_W from the battery reading is PENDING until - that reading comes within ANSWER_W of it, or half way. The first move of - either reading towards it (ANSWER_W) is its START; -- while `late_seen` and pending, the loop may not lead the battery reading by - more than ANSWER_W that way (`hold`, `hold_w`; not while `exempt`: a - battery that gives nothing under its ask or refuses is SAFETY-52/94's case) - and the peak guard may not push past what stands (`peak_cap`). Pending for LATE_S with no start is LATE: control - holds, then 0 W after `stale_input_s`, as for a missing input. + that reading comes within ANSWER_W of it, or half way; +- while pending the loop may not lead the battery reading by more than + ANSWER_W that way (`hold`, `hold_w`; not while `exempt`: a battery that + gives nothing under its ask or refuses is SAFETY-52/94's case) and the peak + guard may not push past what stands (`peak_cap`). Pending for LATE_S with + no reading moving towards it is LATE: control holds, then 0 W after + `stale_input_s`, as for a missing input. -With readings on time nothing here acts. DECISIONS/SAFETY-128-late-readings.md. +Not `late_seen`, nothing here acts. DECISIONS/SAFETY-128-late-readings.md. """ import bisect -import math from collections import deque from . import freshness -from .control import LOOP_TICK_S, PROBE_ANSWER_W +from .control import PROBE_ANSWER_W # Within this of the command the reading has answered it (a probe's answer). ANSWER_W = PROBE_ANSWER_W # 100 -# A fresh meter shows a command within the plant's settle (SAFETY-75's 6 s dead -# time) and one meter tick; later than that the readings run late. -DELAY_S = freshness.PLANT_SETTLE_S + LOOP_TICK_S # 8.0 # The slowest healthy battery answer: SAFETY-75's 6 s dead time, the 6 s plant # settle, one 10 s publish and one 1 s read. LATE_S = (2 * freshness.PLANT_SETTLE_S + freshness.PUBLISH_PERIOD_S + freshness.READ_PERIOD_S) # 23.0 -# The meter's delay is fitted over this much history, up to this delay. -FIT_S = 40.0 +# Each battery reading's delay is fitted over this much history, up to this. +FIT_S = 60.0 MAX_DELAY_S = 90 -FIT_EVERY_S = 4.0 -# A slow battery answers at once but takes this long (SAFETY-85/94 rigs). -SLOW_TAUS_S = (10.0, 20.0) +FIT_EVERY_S = 2.0 +# A fit needs this many new readings (all batteries) and this much movement +# of the reading and the request. +FIT_ROWS = 5 +FIT_MOVE_W = 3 * ANSWER_W # 300 +# Late from ON_S, on time again under OFF_S (hysteresis). +ON_S = 9.5 +OFF_S = 6.5 +# ...after this many late fits in a row. +ON_FITS = 2 # With no fit for this long the evidence is dropped (a steady site shows none). FIT_GONE_S = 120.0 -# A fit counts only if the commands explain this share of the meter's swing. +# A fit counts only if the commands explain this share of the reading's swing. FIT_SHARE = 0.5 +# A late delay must explain this share. +FIT_SHARE_LATE = 0.75 # ...and a late delay only if it fits this much better than any fresh one. -FIT_BETTER = 0.7 +FIT_BETTER = 0.5 +# A shorter delay within this factor of the best fit is taken instead. +FIT_NEAR = 2.0 +# ...or within this of it (W^2): two fits this good are both exact. +FIT_FLOOR_W2 = (ANSWER_W / 2) ** 2 # 2 500 -class Delay: - """How long after a command the meter shows it, fitted from history. +def per_second(cmds, first, last): + """The request standing at each whole second from `first` to `last`.""" + ts = [t for t, _c in cmds] + return [cmds[max(0, bisect.bisect_right(ts, k) - 1)][1] + for k in range(first, last + 1)] - The meter is the house less the batteries, and the batteries follow the - commands, so `grid(t) + command(t - d)` is the house alone at the right - `d`: the `d` that leaves it flattest is the delay. Only new meter values - count, so a silent meter (SAFETY-99) is not a late one. None while the - commands have not moved enough to tell, or explain too little of the meter - (a battery that does not follow); not late if a slow battery that answers - at once (SLOW_TAUS_S) fits as well.""" + +def pooled(sets, d): + """Pooled variance over [(rows, c, first, sign)] of value + sign * + c[t - d], each set about its own mean.""" + tot, n = 0.0, 0 + for rows, c, first, sign in sets: + e = [v + sign * c[int(t) - d - first] for t, v in rows] + m = sum(e) / len(e) + tot += sum((x - m) ** 2 for x in e) + n += len(e) + return tot / n + + +class UnitFit: + """One battery's power requests and readings, as they were seen.""" def __init__(self): - self.cmds = deque() # (t, command) when it changed - self.grid = deque() # (t, meter) + self.cmds = deque() # (t, request) when it changed + self.reads = deque() # (t, reading) when it changed - def add(self, now, cmd, grid): - if not self.cmds or self.cmds[-1][1] != cmd: - self.cmds.append((now, cmd)) - if grid is not None and (not self.grid or self.grid[-1][1] != grid): - self.grid.append((now, grid)) # new meter values only - while self.grid and self.grid[0][0] < now - FIT_S: - self.grid.popleft() + def add(self, now, cmd, reading): + if cmd is not None and (not self.cmds or self.cmds[-1][1] != cmd): + self.cmds.append((now, float(cmd))) + if reading is not None and (not self.reads + or self.reads[-1][1] != reading): + self.reads.append((now, float(reading))) + while self.reads and self.reads[0][0] < now - FIT_S: + self.reads.popleft() while (len(self.cmds) > 1 and self.cmds[1][0] <= now - FIT_S - MAX_DELAY_S): self.cmds.popleft() - def estimate(self): - if len(self.grid) < 10 or len(self.cmds) < 2: - return None - ts = [t for t, _c in self.cmds] - t0 = self.cmds[0][0] - rows = [(t, g) for t, g in self.grid if t >= t0 + MAX_DELAY_S] - if len(rows) < 10: + def series(self): + """(rows, request each second, first second) for the fit, or None: + too few new readings, or a reading or a request that has not moved + FIT_MOVE_W. The request is averaged over the 2 s before each second: + the battery's own lag and the read.""" + rows = [r for r in self.reads + if self.cmds and r[0] >= self.cmds[0][0] + MAX_DELAY_S] + vals = [v for _t, v in rows] + if len(rows) < 3 or max(vals) - min(vals) < FIT_MOVE_W: return None cs = [c for _t, c in self.cmds] - if max(cs) - min(cs) < ANSWER_W: + if max(cs) - min(cs) < FIT_MOVE_W: return None - g_mean = sum(g for _t, g in rows) / len(rows) - g_var = sum((g - g_mean) ** 2 for _t, g in rows) / len(rows) + first, last = int(rows[0][0]) - MAX_DELAY_S - 2, int(rows[-1][0]) + c = per_second(self.cmds, first, last) + avg = [c[0], c[0]] + [(c[k - 2] + c[k - 1] + c[k]) / 3 + for k in range(2, len(c))] + return rows, avg, first - def var(f, d): - e = [g + f(t - d) for t, g in rows] - m = sum(e) / len(e) - return sum((x - m) ** 2 for x in e) / len(e) - step = lambda t: cs[bisect.bisect_right(ts, t) - 1] # noqa: E731 - pure = [var(step, d) for d in range(0, MAX_DELAY_S + 1)] - best = min(range(len(pure)), key=lambda d: (pure[d], d)) - if g_var <= 0 or pure[best] > (1 - FIT_SHARE) * g_var: + +class Delay: + """How long after a power request the batteries' readings show it. + + A reading follows its battery's own request and the house does not move + it, so `reading(t) - request(t - d)` is flattest at the right `d`. Each new + reading counts once, at the second it was seen, so the 10 s publish adds no + delay. One `d` for every battery (Home Assistant delays them all). None + with too few readings, or when the requests explain too little of them (a + battery that does not follow); not late unless it fits clearly better than + any fresh delay, and then the shortest delay that fits about as well.""" + + def __init__(self): + self.units = {} + + def add(self, now, unit, cmd, reading): + self.units.setdefault(unit, UnitFit()).add(now, cmd, reading) + + def estimate(self): + ser = [x for x in (u.series() for u in self.units.values()) if x] + if sum(len(rows) for rows, _c, _f in ser) < FIT_ROWS: return None - if best <= DELAY_S: + sets = [(rows, c, first, -1.0) for rows, c, first in ser] + r_var = pooled([(rows, [0.0] * len(c), first, -1.0) + for rows, c, first in ser], 0) + pure = [pooled(sets, d) for d in range(0, MAX_DELAY_S + 1)] + # The shortest that fits about as well: a swing's period fits again. + best = next(d for d, v in enumerate(pure) + if v <= FIT_NEAR * min(pure) + FIT_FLOOR_W2) + if r_var <= 0 or pure[best] > (1 - FIT_SHARE) * r_var: + return None + if best < ON_S: return float(best) - # Late only if it fits clearly better than any fresh delay, and better - # than a slow battery answering at once (SAFETY-85/94's 10-20 s lag). - if pure[best] > FIT_BETTER * min(pure[:int(DELAY_S) + 1]): + if (pure[best] > (1 - FIT_SHARE_LATE) * r_var + or pure[best] > FIT_BETTER * min(pure[:int(OFF_S) + 1])): + return None + return float(best) + + +# The meter's delay: fitted over this much history, from this many new +# values (a meter of about 3.5 s or faster: a slower one gives too few to +# fit without false delays); late over M_LATE_S (the plant's 6 s settle and one 2 s meter tick), +# but only when the batteries' own readings have not just shown them on time. +M_FIT_S = 60.0 +M_ROWS = 16 +M_LATE_S = freshness.PLANT_SETTLE_S + 2.0 # 8.0 +M_SHARE, M_BETTER = 0.4, 0.7 +# A meter fit counts only after an ask that took this long to start moving a +# reading: on time they start within 1-2 s (the recorded house, at 1-5 s). +SLOW_START_S = 3.0 + + +class MeterFit: + """How long after a site request the meter shows it (round 1's fit). + + `meter(t) + request(t - d)` is the house alone at the right `d`. Quick, + as the meter reads every 1-5 s, but a house load that repeats can line + up with an earlier request, so it only counts while the batteries' own + fit is not on time (`Answer.watch`).""" + + def __init__(self): + self.cmds = deque() # (t, request) when it changed + self.grid = deque() # (t, meter) new values only + + def add(self, now, cmd, grid): + if cmd is not None and (not self.cmds or self.cmds[-1][1] != cmd): + self.cmds.append((now, float(cmd))) + if grid is not None and (not self.grid or self.grid[-1][1] != grid): + self.grid.append((now, float(grid))) + while self.grid and self.grid[0][0] < now - M_FIT_S: + self.grid.popleft() + while (len(self.cmds) > 1 + and self.cmds[1][0] <= now - M_FIT_S - MAX_DELAY_S): + self.cmds.popleft() + + def estimate(self): + if len(self.cmds) < 2: + return None + rows = [(t, g) for t, g in self.grid + if t >= self.cmds[0][0] + MAX_DELAY_S] + cs = [c for _t, c in self.cmds] + if len(rows) < M_ROWS or max(cs) - min(cs) < ANSWER_W: + return None + first, last = int(rows[0][0]) - MAX_DELAY_S, int(rows[-1][0]) + c = per_second(self.cmds, first, last) + g_var = pooled([(rows, [0.0] * len(c), first, 1.0)], 0) + pure = [pooled([(rows, c, first, 1.0)], d) + for d in range(0, MAX_DELAY_S + 1)] + best = min(range(len(pure)), key=lambda d: (pure[d], d)) + if g_var <= 0 or pure[best] > (1 - M_SHARE) * g_var: + return None + if best <= M_LATE_S: + return float(best) + if pure[best] > M_BETTER * min(pure[:int(M_LATE_S) + 1]): return None - first, last = int(t0), int(rows[-1][0]) - for tau in SLOW_TAUS_S: - k, f, slow = 1 - math.exp(-1.0 / tau), None, [] - for sec in range(first, last + 1): # the battery, each second - f = step(sec) if f is None else f + (step(sec) - f) * k - slow.append(f) - if var(lambda t: slow[min(int(t), last) - first], 0) <= pure[best]: - return None return float(best) @@ -131,13 +230,22 @@ class Answer: self.from_grid = None # the meter when it was asked self.since = None # when it was asked self.started = False # a reading has moved towards it - self.late_seen = False # the last fitted delay was over DELAY_S + self.late_seen = False # the batteries' readings run late self.late_at = None # when the late verdict started self.exempt = False # a battery that cannot follow explains it self._was = None # (command, reading, meter) at the last call self.delay = Delay() - self.delay_s = None # the meter's last fitted delay, s - self._fit_at = self._fit_ok = self._exempt_at = None + self.meter = MeterFit() + self._batt_fit = None # (when, d): the batteries' last fit + self._b_run = 0 # late battery fits in a row + self._start_age = None # (when, how long the last ask took to start) + self._exempt_at = None # when a battery last gave nothing under its ask + self._ask_exempt = False + self._start = None + self.delay_s = None # the last fitted delay (either fit), s + self.shown_s = None # the batteries' last late delay, for the page + self.fits = 0 # fits made (instrumentation) + self._fit_at = self._fit_ok = None @property def active(self) -> bool: @@ -165,6 +273,56 @@ class Answer: """Seconds the late verdict has stood, or None.""" return None if self.late_at is None else max(0.0, now - self.late_at) + def watch(self, now, units, standing_w=None, grid_w=None): + """Each read (1 s): `units` = [(name, request on its wire, its battery + reading)], the site request and the meter. Every FIT_EVERY_S, fit both + delays. Late on ON_FITS battery fits from ON_S, or on a meter fit + over M_LATE_S when an ask was slow to start (SLOW_START_S) and no + battery fit within FIT_S said on time: a house load that repeats can + fool the meter fit, not these. On time again on a battery fit under + OFF_S, or a meter fit within M_LATE_S unless a battery fit said late. + """ + if self._start is None: + self._start = now + for name, cmd, reading in units: + self.delay.add(now, name, cmd, reading) + self.meter.add(now, standing_w, grid_w) + if self._fit_at is not None and now - self._fit_at < FIT_EVERY_S: + return + self._fit_at = now + b = self.delay.estimate() + if (b or 0.0) >= ON_S and self._exempt_at is not None \ + and now - self._exempt_at <= FIT_S: + b = None # a battery that stopped following: no late fit + m = self.meter.estimate() + if b is not None: + self._batt_fit = (now, b) + self.shown_s = b if b >= ON_S else None + recent = (self._batt_fit is not None + and now - self._batt_fit[0] <= FIT_S) + b_fresh = recent and self._batt_fit[1] < ON_S + b_late = recent and self._batt_fit[1] >= ON_S + + self._b_run = self._b_run + 1 if (b or 0.0) >= ON_S else 0 + d = b if b is not None else m + if d is None: + if self._fit_ok is None or now - self._fit_ok > FIT_GONE_S: + self.late_seen = False # nothing to tell it by + return + self.fits += 1 + self.delay_s, self._fit_ok = d, now + warm = now - self._start >= MAX_DELAY_S + FIT_S + m_late = (m or 0.0) > M_LATE_S + slow = (self._start_age is not None + and now - self._start_age[0] <= M_FIT_S + and self._start_age[1] >= SLOW_START_S) + if self._b_run >= ON_FITS or ( + m_late and warm and slow and not b_fresh): + self.late_seen = True + elif (b is not None and b < OFF_S) or ( + m is not None and m <= M_LATE_S and not b_late): + self.late_seen = False + def observe(self, now, standing_w, batt_w, *, grid_w=None, exempt=False, idle_w=0.0): """`standing_w` the site command on the wire, `batt_w` the site @@ -179,22 +337,15 @@ class Answer: if self.exempt: self._exempt_at = now was, self._was = self._was, (cmd, r, g) - self.delay.add(now, cmd, g) - if self._fit_at is None or now - self._fit_at >= FIT_EVERY_S: - self._fit_at = now - d = self.delay.estimate() - if self._exempt_at is not None and now - self._exempt_at < FIT_S: - d = None # a battery that does not follow spoils it - if d is not None: - self.delay_s, self.late_seen, self._fit_ok = d, d > DELAY_S, now - elif self._fit_ok is None or now - self._fit_ok > FIT_GONE_S: - self.late_seen = False # nothing to tell it by if self.ask is None: if was is None or cmd == was[0] or abs(cmd - was[1]) < ANSWER_W: return self.ask, self.from_w, self.from_grid = cmd, was[1], was[2] self.since, self.started = now, False + self._ask_exempt = False self.ask = cmd + # A battery that gives nothing under its ask explains a slow start. + self._ask_exempt = self._ask_exempt or self.exempt age = now - self.since gap = self.ask - self.from_w way = 1.0 if gap > 0 else -1.0 @@ -203,6 +354,10 @@ class Answer: else (self.from_grid - g) * way) if not self.started and max(batt_to, grid_to) >= ANSWER_W: self.started, self.late_at = True, None + self._start_age = (now, 0.0 if self._ask_exempt else age) + elif (not self.started and age >= SLOW_START_S + and not self._ask_exempt): + self._start_age = (now, age) if (self.ask - r) * way < ANSWER_W or batt_to >= abs(gap) / 2: self._clear() # answered elif abs(gap) < ANSWER_W: diff --git a/goodwe_controller/app/main.py b/goodwe_controller/app/main.py index b6ce466..52305dc 100644 --- a/goodwe_controller/app/main.py +++ b/goodwe_controller/app/main.py @@ -3137,11 +3137,11 @@ class Controller: if a.late_seen != self._late_logged: self._late_logged = a.late_seen if a.late_seen: - self.log_event(f"readings late ({a.delay_s:.0f} s) - holding", + self.log_event("readings late - holding", hint=messages.hint("readings_late")) _LOG.warning("Home Assistant readings arrive late, so control " - "holds the battery: READINGS_LATE.") - _LOG.debug("READINGS_LATE: the meter shows a power request " + "holds the batteries: READINGS_LATE.") + _LOG.debug("READINGS_LATE: the readings show a power request " "%.0f s after it is sent.", a.delay_s) else: self.log_event("readings on time", hint=messages.hint( @@ -3760,6 +3760,10 @@ class Controller: else: _LOG.debug("Meter readings are back.") + self.answer.watch(now_mono, [(inv.name, inv.standing_w(), inv.batt) + for inv in self.inverters], # SAFETY-128 + sum(inv.standing_w() for inv in self.inverters), + self.grid) if self.inputs_ok: if self.inputs_bad_since is not None: self.log_event("inputs recovered", @@ -7384,7 +7388,7 @@ class Controller: if self.answer.late_seen: # SAFETY-128 out.append({"ok": False, "warn": True, "id": "late", "text": messages.row_readings_late( - self.answer.delay_s, self._late_zero)}) + self.answer.shown_s, self._late_zero)}) # SAFETY-03. Red while an over-rating clamp is engaged (amber for the # integrator bound, SAFETY-40), and the engagement COUNT is shown even diff --git a/goodwe_controller/app/messages.py b/goodwe_controller/app/messages.py index b375b0a..831da93 100644 --- a/goodwe_controller/app/messages.py +++ b/goodwe_controller/app/messages.py @@ -645,16 +645,28 @@ def row_controller_stalled(age_s, veto_s) -> str: def row_readings_late(delay_s, zero=False) -> str: - """SAFETY-128: Home Assistant readings arrive late; past the hold, 0 W.""" + """SAFETY-128: readings show the power request late; `delay_s` the + batteries' fitted delay (their own answer included), or None when only + the meter has shown it; past the hold, 0 W.""" + if delay_s is None: + if zero: + return Bi("Home Assistant readings show requests late, so every " + "inverter is at 0 W.", + "Metingen uit Home Assistant tonen vermogensvraag te " + "laat, omvormers op 0 W.") + return Bi("Home Assistant readings show requests late, so control " + "holds the batteries.", + "Metingen uit Home Assistant tonen vermogensvraag te laat, " + "dus de batterijen wachten.") if zero: - return Bi(f"Home Assistant readings arrive about {seconds(delay_s)} " + return Bi(f"Home Assistant readings show requests {seconds(delay_s)} " f"late, so every inverter is at 0 W.", - f"Metingen uit Home Assistant komen zo'n " + f"Metingen uit Home Assistant tonen vermogensvraag " f"{seconden(delay_s)} te laat, omvormers op 0 W.") - return Bi(f"Home Assistant readings arrive about {seconds(delay_s)} late, " - f"so control holds the batteries.", - f"Metingen uit Home Assistant komen zo'n {seconden(delay_s)} te " - f"laat, dus de batterijen wachten.") + return Bi(f"Home Assistant readings show requests about {seconds(delay_s)} " + f"late, so control holds the batteries.", + f"Metingen uit Home Assistant tonen vermogensvraag zo'n " + f"{seconden(delay_s)} te laat, dus de batterijen wachten.") def row_meter_waiting(limit_s) -> str: @@ -1511,6 +1523,8 @@ def status_samples() -> list: ("row_meter_waiting", AMBER, row_meter_waiting(30)), ("row_readings_late", AMBER, row_readings_late(30)), ("row_readings_late", AMBER, row_readings_late(30, True)), + ("row_readings_late", AMBER, row_readings_late(None)), + ("row_readings_late", AMBER, row_readings_late(None, True)), ("row_meter", GREEN, row_meter(-350, 2, 3, 340)), ("row_meter_frozen", RED, row_meter_frozen()), *(("row_freeze_off", AMBER, row_freeze_off(k)) for k in FREEZE_OFF), diff --git a/goodwe_controller/test_safety128.py b/goodwe_controller/test_safety128.py index a960979..b96d249 100644 --- a/goodwe_controller/test_safety128.py +++ b/goodwe_controller/test_safety128.py @@ -95,37 +95,55 @@ def swing(rows, t0, t1): print("1. the give run with readings late") ON = 300.0 CASES = ((10.0, 150.0), (30.0, 300.0)) -for units in (("a", "b"), ("a",)): +for p1, units in ((2.0, ("a", "b")), (2.0, ("a",)), (1.0, ("a", "b")), + (5.0, ("a", "b"))): house = 1800.0 if len(units) == 2 else 1500.0 for lag, span in CASES: off = ON + span + seen = [] new = R.Rig(house=house, lag_s=lag, lag_on=ON, lag_off=off, - units=units) - new.run(off + 120.0) + units=units, p1_s=p1) + new.run(off + 120.0, lambda rig, c, t: seen.append( + (t, c.answer.late_seen)) if c.answer.late_seen else None) old = R.Rig(house=house, lag_s=lag, lag_on=ON, lag_off=off, - units=units) + units=units, p1_s=p1) no_gate(lambda: old.run(off + 120.0)) - n = f"N={len(units)}, {lag:.0f} s late for {span:.0f} s" + n = f"N={len(units)}, {p1:.0f} s meter, {lag:.0f} s late for {span:.0f} s" wo, wn = window(old.rows, ON, off), window(new.rows, ON, off) good = last_good(new.rows, ON) half = ON + span / 2 + red = min(r[1] for r in wo) < -1000.0 + found = min([t for t, _l in seen] or [None]) print(f" {n}: without the gate cmd {min(r[2] for r in wo):.0f}.." f"{max(r[2] for r in wo):.0f} W, grid {min(r[1] for r in wo):.0f}" f"..{max(r[1] for r in wo):.0f} W (swing {swing(old.rows, ON, half):.0f}" f" then {swing(old.rows, half, off):.0f}); with it cmd " f"{min(r[2] for r in wn):.0f}..{max(r[2] for r in wn):.0f} W " f"(last good {good:.0f}), grid {min(r[1] for r in wn):.0f}.." - f"{max(r[1] for r in wn):.0f} W, settled " - f"{settled_at(new.rows, off) - off:.0f} s after") - if len(units) == 2: + f"{max(r[1] for r in wn):.0f} W, found late " + f"{None if found is None else found - ON} s after the start, " + f"settled {settled_at(new.rows, off) - off:.0f} s after") + if len(units) == 2 and p1 == 2.0: check(f"{n}: without the gate the export goes below -1 kW and the " f"swing grows (red on 77fe31b)", - min(r[1] for r in wo) < -1000.0 - and swing(old.rows, half, off) > swing(old.rows, ON, half)) - check(f"{n}: with it the command stays within 250 W of the last good " - f"one and no export below -200 W", - all(abs(r[2] - good) <= 250.0 for r in wn) - and min(r[1] for r in wn) >= -200.0) + red and swing(old.rows, half, off) > swing(old.rows, ON, half)) + if red: + check(f"{n}: found late while the lag lasts", found is not None + and found < off) + if p1 < 5.0: + # Found within about a minute: at 2 s the brief's bounds; at 1 s + # the swing grows faster before it is found, so twice them. + lim = 250.0 if p1 == 2.0 else 500.0 + check(f"{n}: with it the command stays within {lim:.0f} W of the " + f"last good one and no export below -{lim - 50:.0f} W", + all(abs(r[2] - good) <= lim for r in wn) + and min(r[1] for r in wn) >= -(lim - 50.0)) + elif red: + # 5 s meter: only the batteries' fit runs (too few meter values), + # so it is found later; the export is cut to under a third. + check(f"{n}: with it the export is under a third of the run " + f"without the gate", + min(r[1] for r in wn) >= min(r[1] for r in wo) / 3.0) check(f"{n}: settled within 30 s after the lag stops", settled_at(new.rows, off) is not None and settled_at(new.rows, off) - off <= 30.0) @@ -196,15 +214,24 @@ lastr = max([t for t, _r in rows_seen] or [None]) print(f" 30 s late for 300 s: {n_late} late episode(s), {n_back} end(s), " f"the row from {first} to {lastr} s ({len(rows_seen)} s); " f"{rows_seen[0][1]['text'] if rows_seen else ''}") -_d = c.answer.delay_s -check("found late within a minute of the lag starting, and an amber row says " - "so with the delay it measured, in plain words, EN and NL", - first is not None and first - ON <= 60.0 +_ok_texts = {str(messages.row_readings_late(None))} | { + str(messages.row_readings_late(d)) for d in range(20, 33)} +check("found late within 70 s of the lag starting, and an amber row says " + "so in plain words, EN and NL, with the batteries' delay once they have " + "shown it (20-32 s for 30 s late; never a figure from the meter alone)", + first is not None and first - ON <= 70.0 and all(r["warn"] and not r["ok"] for _t, r in rows_seen) - and str(rows_seen[-1][1]["text"]) in { - str(messages.row_readings_late(d)) for d in range(20, 33)} + and {str(r["text"]) for _t, r in rows_seen} <= _ok_texts and rows_seen[-1][1]["text"].nl.startswith( - "Metingen uit Home Assistant komen zo'n ")) + "Metingen uit Home Assistant tonen vermogensvraag")) +rows5 = [] +late5 = R.Rig(lag_s=30.0, lag_on=ON, lag_off=ON + 300.0, p1_s=5.0) +late5.run(ON + 300.0, lambda rig, c, t: rows5.extend( + str(r["text"]) for r in c.checks() if r.get("id") == "late")) +print(f" 5 s meter, 30 s late: {sorted(set(rows5))}") +check("at a 5 s meter (the batteries' fit) the row names the delay: about " + "20-32 s for 30 s late", + rows5 and set(rows5) <= _ok_texts and any("about" in x for x in rows5)) check("logged once for the episode, and once when it ends, not per cycle", n_late == 1 and n_back == 1) check("the row goes once a fit shows the readings on time, or 2 minutes " @@ -287,6 +314,51 @@ check("start-up: the first 120 s reach the house within 30 s, as without the " "gate, and no late episode", settled_at(a0.rows, 0.0) is not None and settled_at(a0.rows, 0.0) <= 30.0 and not any("readings late" in e for e in a0.c.events)) +for on, off in ((45, 60), (30, 30)): + house = (lambda t, on=on, off=off: + 400.0 + (2000.0 if t % (on + off) < on else 0.0)) + oven = R.Rig(house=house, noise=80.0) + oven.run(1500.0) + oven0 = R.Rig(house=house, noise=80.0) + no_gate(lambda: oven0.run(1500.0)) + check(f"an oven cycling 2 kW {on} s on, {off} s off (2 s meter, readings " + f"on time): never taken for late readings ({oven.c.answer.fits} " + "fits), the same commands as without the gate", + oven.c.answer.fits > 100 and not any( + "readings late" in e for e in oven.c.events) + and [r[2] for r in oven.rows] == [r[2] for r in oven0.rows]) +# One battery that empties to its cut-out level under an oven (1 s meter): +# its fit lined the old movement up as 85 s late (round 2, before the fix). +emp = R.Rig(house=lambda t: 400.0 + (2000.0 if t % 105 < 45 else 0.0), + noise=150.0, soc=10.6, units=("a",), p1_s=1.0) +emp.run(1800.0) +check("one battery emptying to its cut-out level under a cycling oven (1 s " + "meter): not taken for late readings", + not any("readings late" in e for e in emp.c.events)) +emp2 = R.Rig(house=lambda t: 400.0 + (2000.0 if t % 60 < 30 else 0.0), + noise=150.0, soc=10.2, units=("a",), p1_s=2.0) +emp2.run(1800.0) +check("one empty battery under an oven cycling 30/30 s (2 s meter): its " + "requests do not start, but it gives nothing, so that is no evidence " + "for the meter fit", + not any("readings late" in e for e in emp2.c.events)) +# The recorded house (2026-09-02, its first 2 hours, modes off) at a 2 s +# meter: the meter fit alone read it as up to 65 s late here (review, B1). +from datetime import datetime as _dt, timezone as _tz # noqa: E402 +_tick, rp.TICK = rp.TICK, 2.0 +_late = [] +try: + _st, _rc = rp.run("modes_off", rp.load_house(["2026-09-02"]), + rp.epex_points(), + _dt.fromisoformat("2026-09-02T00:00:00+02:00") + .astimezone(_tz.utc), 7200.0, + check=lambda c, h, now, pr: _late.append( + c.answer.late_seen)) +finally: + rp.TICK = _tick +check(f"the recorded house, 2 hours at a 2 s meter: {_rc.answer.fits} fits, " + "never taken for late readings", + _rc.answer.fits > 100 and not any(_late)) import safety94_rig as s94 # noqa: E402 slow = [(n, tau, tick) for n in (1, 2) for tau in (10.0, 20.0) for tick in (1.0, 2.0) @@ -316,39 +388,52 @@ check("ANSWER_W is a probe's answer (100 W)", lateness.ANSWER_W == control.PROBE_ANSWER_W == 100.0) -check("DELAY_S is the plant's 6 s settle and one 2 s meter tick", - lateness.DELAY_S == 8.0) +check("late from ON_S 9.5 s, on time again under OFF_S 6.5 s", + (lateness.ON_S, lateness.OFF_S) == (9.5, 6.5)) -def fit(delay, *, follows=True, step_every=20, noise=40.0, dur=300): - """`Delay` on a meter that shows each command `delay` s later (a house - of 2 000 W with +-`noise`), or never (`follows` False).""" +def fit(delay, *, follows=True, step_every=20, dur=300, tau=0.0, units="ab"): + """`Delay` on batteries whose readings, published every 10 s at their own + phase, show each request `delay` s late (through a `tau` s lag), or never + (`follows` False).""" + import math import random rng, d = random.Random(7), lateness.Delay() edges = [0] while edges[-1] < dur: edges.append(edges[-1] + step_every * rng.uniform(0.5, 1.5)) - cmd = lambda t: 1500.0 + (400.0 if sum(e <= t for e in edges) % 2 else 0.0) - for t in range(0, dur, 2): - shown = cmd(t - delay) if follows and t >= delay else 1500.0 - d.add(float(t), cmd(t), 2000.0 - shown + rng.uniform(-noise, noise)) + cmd = lambda t: 1500.0 + (600.0 if sum(e <= t for e in edges) % 2 else 0.0) + act, shown = 1500.0, {} + for t in range(0, dur): + want = cmd(t - delay) if t >= delay else 1500.0 + act = want if not tau else act + (want - act) * (1 - math.exp(-1 / tau)) + for k, u in enumerate(units): + if (t + 4 * k) % 10 == 0: + shown[u] = round(act + rng.uniform(-3, 3)) if follows else 900 + d.add(float(t), u, cmd(t), shown.get(u)) return d.estimate() got = {k: fit(k) for k in (0, 6, 10, 30, 60)} -print(f" fitted delays: {got}") -check("the fit finds the meter's delay (0, 6, 10, 30, 60 s) to within 1 s", - all(v is not None and abs(v - k) <= 1.0 for k, v in got.items())) -check("a battery that does not follow gives no fit, and so does a command " - "that does not move", - fit(10, follows=False) is None and fit(10, step_every=10_000) is None) +print(f" fitted delays: {got}; one battery {fit(10, units='a')}") +check("the fit finds the readings' delay (0, 6, 10, 30, 60 s) to within 2 s " + "(it reads the request over the 2 s before), also with one battery", + all(v is not None and abs(v - k) <= 2.0 for k, v in got.items()) + and abs((fit(10, units="a") or 0.0) - 10.0) <= 2.0) +slow = [fit(0, tau=tau, step_every=s) for tau in (3.0, 10.0, 20.0) + for s in (10, 20, 40)] +check("a battery that does not follow gives no fit, nor does a request that " + f"does not move, and a slow battery (3, 10, 20 s lag) is never late " + f"({slow})", + fit(10, follows=False) is None and fit(10, step_every=10_000) is None + and all((d or 0.0) < lateness.ON_S for d in slow)) def gate(steps, exempt=(), late_seen=True): """[(t, command, battery reading, meter)] through one Answer, its fit replaced by `late_seen`; after each: (pending, hold, late s, peak cap).""" a = lateness.Answer() - a.delay.estimate = lambda: (30.0 if late_seen else 0.0) + a.late_seen = late_seen def one(t, cmd, batt, grid): a.observe(float(t), cmd, batt, grid_w=grid, exempt=t in exempt) -- 2.54.0 From 7de77a5a097a2ccfd2ed925a7196e0e45ee880df Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Mon, 2 Nov 2026 05:16:28 +0100 Subject: [PATCH 03/22] SAFETY-128 review: DECISIONS states the slow-battery limit as measured Round 2 review: a 10 s dead time reads late at 1-2 s meters, and a 15-20 s lag gives late fits; the note said 3-20 s and 6-10 s gave none. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Bdqr7PiaHcBnzPituu2yTC --- .../DECISIONS/SAFETY-128-late-readings.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md b/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md index 41fd33a..13888d7 100644 --- a/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md +++ b/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md @@ -23,8 +23,9 @@ inverter limit in its last seconds (OPT-34 rig F-2). Late from 9.5 s, after 2 late fits in a row. A late fit must explain 75 % and beat every fit up to 6.5 s by 2x; the shortest delay that fits about as well (within 2x, or within 2 500 W^2) is taken instead, since a swing's - period fits again. A slow battery (3-20 s lag) answering at once gives no - late fit (checked in the test). No late battery fit counts for 60 s + period fits again. A slow battery (3-10 s lag) answering at once gives no + late fit; at a 15-20 s lag some fits read late (see "What it costs"). + No late battery fit counts for 60 s after a battery gave nothing under its ask: an emptied battery lined its old movement up as 85 s late. - **Meter** (`MeterFit`, round 1's fit): `meter(t) + request(t - d)` is the @@ -56,7 +57,7 @@ inverter limit in its last seconds (OPT-34 rig F-2). WARNING (`READINGS_LATE`) per episode and one line when it ends. **With readings on time** no verdict was reached in any reviewer scenario -(owner-shaped house, ovens 20/40 to 60/60 s, clouds, a 300 W/s battery, 6-10 s +(owner-shaped house, ovens 20/40 to 60/60 s, clouds, a 300 W/s battery, 6-8 s dead times, HA jitter 0-4 s, an EV following the meter, capped batteries, HA restarts, meter pauses; 1, 2 and 5 s meters) nor in 2 days of the recorded house at 1, 2 and 5 s ticks, so the commands are the same as without the gate. @@ -66,8 +67,10 @@ The fits run all the time; the pull request has the counts. - Found 36-68 s after a lag starts at 1-2 s meters, about 3 minutes at a 5 s meter (battery fit only; the meter fit needs a faster meter). Until then control is as before. At 5 s and 30 s late the export is cut to about a third, not removed. -- A battery whose own answer takes about 10 s is close to the threshold: an - 8 s dead time reads 7, 10 s reads 9, HA 10 s late reads 10. +- A battery that answers slowly is read as late readings and held: a dead + time of 10 s or more (8 s reads 7-8; 10 s reads 10 at 1-2 s meters, 9 at + 5 s), or a 15-20 s lag (review round 2). The GoodWe measured settles in 6 s + (`freshness.PLANT_SETTLE_S`); the row then blames Home Assistant wrongly. - The row can stay up to 180 s after the readings are on time on a steady site. While only the meter has shown the delay it gives no figure. - A battery that cannot follow while the readings are late (at a limit, not -- 2.54.0 From 45535e19d78bea08faed5ade2a856db016a04c53 Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Thu, 5 Nov 2026 21:48:56 +0100 Subject: [PATCH 04/22] UX-77: Home draws today's grid charge for the hours control will really charge The cheap run is sized at the full rating, so under the peak limit a battery is still short when it ends, and control plans the next run then. Home drew only the run under way. `cheaproll` rolls the run on the way control does (`modetrade.energy()` + `priceruns.plan()` at each run end, lifted from `dayview._levels()`), reading the levels at the charge power the peak limit leaves (peak aim less the house, else the rating). `homeview.mode_runs()` and `price_windows()` use it for today: the grid block, the Today line and the chart's cheap bars run to where control stops, each battery's block to its own stop. Display only: no control change. test_ux54: today's grid block of the night-before scenario now follows the roll. Co-Authored-By: Claude --- goodwe_controller/CHANGELOG.md | 1 + .../DECISIONS/UX-54-forecast-tab.md | 10 +- goodwe_controller/app/cheaproll.py | 170 ++++++++++ goodwe_controller/app/dayview.py | 32 +- goodwe_controller/app/homeview.py | 25 +- goodwe_controller/test_ux54.py | 35 +- goodwe_controller/test_ux77.py | 307 ++++++++++++++++++ 7 files changed, 541 insertions(+), 39 deletions(-) create mode 100644 goodwe_controller/app/cheaproll.py create mode 100644 goodwe_controller/test_ux77.py diff --git a/goodwe_controller/CHANGELOG.md b/goodwe_controller/CHANGELOG.md index 695196b..915b037 100644 --- a/goodwe_controller/CHANGELOG.md +++ b/goodwe_controller/CHANGELOG.md @@ -2,6 +2,7 @@ ### Not in a release yet +- Home now draws today's charge from the grid for every hour the add-on will really charge, not only the run under way ([[UX-77]]). - When Home Assistant passes readings on late, control no longer swings on them: it waits until the battery shows its last power request, and the peak limit no longer asks for full power on them. The page says so, and if the readings stay too late every inverter goes to 0 W ([[SAFETY-128]]). ## 0.20.1 diff --git a/goodwe_controller/DECISIONS/UX-54-forecast-tab.md b/goodwe_controller/DECISIONS/UX-54-forecast-tab.md index e0ff423..577a681 100644 --- a/goodwe_controller/DECISIONS/UX-54-forecast-tab.md +++ b/goodwe_controller/DECISIONS/UX-54-forecast-tab.md @@ -97,9 +97,13 @@ is `SolarToday.add()` after the solar read, in a `try`, in memory. - Prices are planned about a day ahead (`priceruns.HORIZON`), so a block of tomorrow later than 24 h from now, or one that needs prices not yet published, appears as the day comes near. The tab says so. -- Today's blocks are Home's, and are not extended or shortened: control may - plan more once they end, and Home draws it then. The curve stops charging - at its level (`charge_to()`) and giving at the floor. +- Today's blocks are Home's. UX-77: Home draws today's grid block to where + control will stop charging, as the tab does for tomorrow's: the cheap run + rolls on at its end while a battery is short (`cheaproll.roll_end()`, the + same loop as `dayview._levels()`), the level read at the charge power the + peak limit leaves (peak aim less the house, else the rating), and each + battery's block ends at its own stop. The curve stops charging at its level + (`charge_to()`) and giving at the floor. - Today, a give run in the plan on a battery that is empty now has no block on Home (no energy to give). The tab equals Home, and says so: "The add-on plans to give power. No block shows while a battery has no energy to give." The diff --git a/goodwe_controller/app/cheaproll.py b/goodwe_controller/app/cheaproll.py new file mode 100644 index 0000000..e9eb515 --- /dev/null +++ b/goodwe_controller/app/cheaproll.py @@ -0,0 +1,170 @@ +"""UX-77: where the cheap run rolls on. Display only; control is unchanged. + +A cheap run is sized for the batteries charging together, at their full +rating, and starts at the first of the cheapest hours. The peak limit leaves +less power than that, so a battery is still short when the run ends, and +control plans the next run then (`Controller._price_runs()`). On a flat night +that goes on until the batteries are at their level or the cheap hours end. + +`roll_end()` asks what control asks at each run end, of the same functions +(`modetrade.energy()`, `priceruns.plan()`), until nothing is short or the next +run does not follow on. `for_today()` reads the batteries at the level they +will have by then: the charge power is the peak limit's room (peak aim minus +the house, as `dayplan` counts it), shared by rating. Where that room is not +known (no house profile or no aim), the rating is used. +DECISIONS/UX-54-forecast-tab.md has the record. +""" + +import dataclasses +import math +from dataclasses import dataclass +from datetime import datetime, timedelta, timezone + +from . import arbitrage, dayplan, modetrade, priceruns +from .dayplan import EFF +from .price import required_spread, uncertainty_of + +SLOT_S = 900 # a price slot +STEP = timedelta(minutes=5) # how finely the level is followed +LIMIT = 24 # rolls at most (a day of one-hour runs) + + +def roll_end(run, points, packs_at, *, required, floor_pct, ceiling_pct, + targets=None, limit=LIMIT): + """Where the cheap run `run` ends once control has rolled it on. + + `packs_at(end)` gives the `modetrade.Pack`s as they stand when a run + ends at `end`. At each end the charge still short is asked of + `modetrade.energy()` and the next run of `priceruns.plan()`, as control + asks them. It stops when nothing is short or the next run does not follow + on, and is `run.end` when control would not roll it.""" + end_t = run.end + for _ in range(limit): + e = modetrade.energy(packs_at(end_t), floor_pct, ceiling_pct, + targets=targets) + if e[0] <= 1e-6: + break + new = priceruns.plan(points, end_t, required_eur_kwh=required, + charge_kwh=e[0], charge_kw=e[1], sell_kwh=0.0, + sell_kw=0.0) + nxt = new.cheap + if nxt is None or not new.charge_useful() or nxt.start > end_t \ + or nxt.end <= end_t: + break + end_t = nxt.end + return end_t + + +@dataclass(frozen=True) +class Roll: + """`end`: where control stops charging, `stops`: name -> when that battery + reaches its level (rounded up to a price slot), None while it still + charges at `end`. A battery not in `stops` does not charge in the run.""" + end: datetime + stops: dict + + def end_for(self, name, base_end): + """The end of `name`'s block: its stop, no later than `end`.""" + if name not in self.stops: + return base_end + stop = self.stops[name] + return self.end if stop is None else min(self.end, stop) + + def last(self, base_end): + """The last end of any battery's block (the chart's cheap bar).""" + return max([base_end] + [self.end_for(n, base_end) + for n in self.stops]) + + +def _ceil_slot(t): + return datetime.fromtimestamp( + math.ceil(t.timestamp() / SLOT_S) * SLOT_S, tz=t.tzinfo + or timezone.utc) + + +class _Charge: + """The batteries' levels followed from `t0` while the chargers charge at + the room the peak limit leaves: `min(sum of ratings, aim - house)`, shared + by rating, each stopping at `modetrade.charge_to()`.""" + + def __init__(self, packs, t0, ceiling_pct, targets, aim_w, house): + self.packs = {p.name: p for p in packs} + self.t = t0 + self.ceil = ceiling_pct + self.targets = targets + self.aim = aim_w + self.house = house + self.stops = {} + + def at(self, end): + """The packs when a run ends at `end`.""" + self._to(end) + return list(self.packs.values()) + + def _room_w(self, hour, total): + if self.aim is None or self.house is None: + return total + house_w, base_w = self.house + return min(total, max(0.0, float(self.aim) + - max(house_w(hour), base_w, 0.0))) + + def _to(self, end): + while self.t < end: + nxt = min(end, self.t + STEP) + for s, e, hour in dayplan._hour_slices(self.t, nxt): + self._charge(s, e, hour) + self.t = nxt + + def _charge(self, s, e, hour): + ch = modetrade.chargers(list(self.packs.values()), self.ceil, + targets=self.targets) + total = sum(abs(p.rating_w) for p in ch) + if not ch or total <= 0.0: + return + w = self._room_w(hour, total) + hrs = (e - s).total_seconds() / 3600.0 + for p in ch: + self.stops.setdefault(p.name, None) + if not p.capacity_kwh: + continue + to = modetrade.charge_to(p, self.ceil, self.targets) + kwh = w * abs(p.rating_w) / total * hrs / 1000.0 * EFF + room = (to - p.soc) / 100.0 * float(p.capacity_kwh) + if kwh <= 0.0: + continue + if kwh >= room - 1e-9: + self.stops[p.name] = _ceil_slot( + s + (e - s) * (room / kwh)) + kwh = room + self.packs[p.name] = dataclasses.replace( + p, soc=p.soc + kwh / float(p.capacity_kwh) * 100.0) + + +def for_today(c, rp, now): + """The `Roll` of the cheap run `rp.cheap` now under way or to come today, + or None when control does not roll it on (no run left, charging not + useful, the trade gate shut, nothing short at its end). Display only: + nothing here is fed back to control.""" + if rp is None or not rp.on or rp.cheap is None or rp.cheap.end <= now \ + or not rp.charge_useful() \ + or getattr(c, "mode_gate", None) is not None: + return None + plan = c.fresh_tariff_plan(now) + if plan is None: + return None + floor, ceil = arbitrage.soc_bounds(c.o) + hp = getattr(c, "house_profile", None) + sim = _Charge(c._mode_packs(), max(now, rp.cheap.start), ceil, + c.plan_targets(), c.peak_aim_w(), + None if hp is None else hp.estimator()) + required = required_spread( + arbitrage.threshold_eur_kwh(c.o), + uncertainty_of(plan.read.product) + | arbitrage.weather_uncertainty(True, c.solar_fc, + bool(c.weather_held(now)))) + end = roll_end(rp.cheap, plan.read.points, sim.at, required=required, + floor_pct=floor, ceiling_pct=ceil, + targets=c.plan_targets()) + if end <= rp.cheap.end: + return None + return Roll(end, dict(sim.stops)) diff --git a/goodwe_controller/app/dayview.py b/goodwe_controller/app/dayview.py index 1664a13..acf3d58 100644 --- a/goodwe_controller/app/dayview.py +++ b/goodwe_controller/app/dayview.py @@ -19,7 +19,8 @@ empty chart. DECISIONS/UX-54-forecast-tab.md has the record. import dataclasses from datetime import datetime, timedelta -from . import arbitrage, dayplan, homeview as hv, modetrade, priceruns +from . import arbitrage, cheaproll, dayplan, homeview as hv, modetrade +from . import priceruns from . import solarfc from .daycurve import Batt, curve from .packmode import SOLAR_FIRST @@ -210,25 +211,18 @@ def _levels(c, est, fc, trust, now, end, runs, floor, ceil, notes_b, shadow, arbitrage.threshold_eur_kwh(c.o), uncertainty_of(plan.read.product) | arbitrage.weather_uncertainty(True, c.solar_fc)) - end_t = cheap.end - for _ in range(24): - ext = dataclasses.replace(rp, cheap=priceruns.Run( + + def packs_at(end_t): + """The packs at the curve's level when a run ends at `end_t`.""" + at = dataclasses.replace(rp, cheap=priceruns.Run( cheap.start, end_t, cheap.mean_eur_kwh)) - rt, lv = settle(ext, lv0) - at_end = [dataclasses.replace(packs[n], soc=v) - for n, v in ((n, _level_at(lv.get(n, ()), end_t)) - for n in lv) if v is not None] - e = modetrade.energy(at_end, floor, ceil, targets=targets) - if e[0] <= 1e-6: - break - new = priceruns.plan( - plan.read.points, end_t, required_eur_kwh=required, - charge_kwh=e[0], charge_kw=e[1], sell_kwh=0.0, sell_kw=0.0) - nxt = new.cheap - if nxt is None or not new.charge_useful() or nxt.start > end_t \ - or nxt.end <= end_t: - break - end_t = nxt.end + _rt, lv = settle(at, lv0) + return [dataclasses.replace(packs[n], soc=v) + for n, v in ((n, _level_at(lv.get(n, ()), end_t)) + for n in lv) if v is not None] + end_t = cheaproll.roll_end( + cheap, plan.read.points, packs_at, required=required, + floor_pct=floor, ceiling_pct=ceil, targets=targets) ext = dataclasses.replace(rp, cheap=priceruns.Run( cheap.start, end_t, cheap.mean_eur_kwh)) rt, lv = settle(ext, lv0) diff --git a/goodwe_controller/app/homeview.py b/goodwe_controller/app/homeview.py index 80c2318..e2812d3 100644 --- a/goodwe_controller/app/homeview.py +++ b/goodwe_controller/app/homeview.py @@ -19,7 +19,7 @@ section at the bottom and is held to STYLE.md by test_messages.py. import contextlib from datetime import datetime, timedelta, timezone -from . import arbitrage, messages, peakguard, uitext +from . import arbitrage, cheaproll, messages, peakguard, priceruns, uitext from .dayplan import EFF from .maintenance import CHARGE as M_CHARGE, DRAIN as M_DRAIN, HOLD as M_HOLD from .maintenance import IDLE as M_IDLE, next_slot_start @@ -218,14 +218,20 @@ def price_windows(c, plan, verdict, now=None): """`[(kind, start, end)]`: where the add-on itself buys ("cheap") and gives ("dear"). With battery modes on that is the kept `priceruns.RunPlan` (`c.mode_runs`): the cheap run when charging in it pays and the trade gate - allows it (UX-53), and every give run. With them off it is the arbitrage - verdict's legs the add-on will take, the ones `trade_runs()` draws - (`trade_legs()`, given `now`). Nothing is classified here.""" + allows it (UX-53), and every give run. UX-77: the cheap run is drawn to + where control stops charging as it rolls on (`cheaproll`). With them off + it is the arbitrage verdict's legs the add-on will take, the ones + `trade_runs()` draws (`trade_legs()`, given `now`). Nothing is classified + here.""" if getattr(c, "modes_on", False): rp = getattr(c, "mode_runs", None) if rp is None or not rp.on or not _price_mode_in_use(c): return [] - out = [("cheap", rp.cheap.start, rp.cheap.end)] \ + end = rp.cheap.end + roll = None if now is None else cheaproll.for_today(c, rp, now) + if roll is not None: + end = roll.last(end) # UX-77: to where control stops + out = [("cheap", rp.cheap.start, end)] \ if rp.charge_useful() and getattr(c, "mode_gate", None) is None \ else [] return out + [("dear", g.start, g.end) for g in rp.give] @@ -422,6 +428,7 @@ def mode_runs(c, day0, day1, now, shadow, ahead=False, rp=None): `ahead=True`) a charge from the floor and a give from the ceiling. The same filter, the same `MIN_KWH`. `rp`: a run plan to read instead of `c.mode_runs`. Home never passes either.""" + today = rp is None and not ahead rp = rp if rp is not None else getattr(c, "mode_runs", None) if getattr(c, "mode_tick", None) is None: return [], None # no cycle yet: nothing is known @@ -431,6 +438,9 @@ def mode_runs(c, day0, day1, now, shadow, ahead=False, rp=None): return [], "not_worth" floor, ceiling = arbitrage.soc_bounds(c.o) gate = getattr(c, "mode_gate", None) + # UX-77: today's grid block runs to where control stops charging (the run + # rolls on when it ends), each battery's to its own stop. + roll = cheaproll.for_today(c, rp, now) if today else None out = [] for i, inv in enumerate(c.inverters): pm = c.pack_mode(inv) @@ -465,7 +475,10 @@ def mode_runs(c, day0, day1, now, shadow, ahead=False, rp=None): if gate is None and rp.charge_useful() and ( need > 0.0 if need is not None else pm.mode in GRID_CHARGE_MODES): - blocks.append(("grid", rp.cheap)) + cheap = rp.cheap if roll is None else priceruns.Run( + rp.cheap.start, roll.end_for(inv.name, rp.cheap.end), + rp.cheap.mean_eur_kwh) + blocks.append(("grid", cheap)) sell = rp.sell if pm.sell and pm.can_sell else None for g in rp.give: if sell is not None and g.start <= sell.start < g.end: diff --git a/goodwe_controller/test_ux54.py b/goodwe_controller/test_ux54.py index 968888d..2a03992 100644 --- a/goodwe_controller/test_ux54.py +++ b/goodwe_controller/test_ux54.py @@ -39,7 +39,8 @@ sys.path.insert(0, os.path.join(HERE, "tools")) logging.disable(logging.WARNING) import ux54_scenarios as S # noqa: E402 -from app import arbitrage, dayplan, dayview, homeview, modetrade # noqa: E402 +from app import arbitrage, cheaproll, dayplan, dayview, homeview # noqa: E402 +from app import modetrade # noqa: E402 from app import packmode, packshare, peakguard, solarfc, uitext # noqa: E402 from app.main import pack_energy_of # noqa: E402 from app.price import Uncertainty # noqa: E402 @@ -232,11 +233,19 @@ END_S = S.brussels(END).strftime("%H:%M") check(f"the cheap run is 01:00 to {END_S} (02:30 for this house)", c_n.mode_runs.cheap.start == at(1, 0, 0) and END == at(2, 30, 0)) nb = tab(c_n, at(0, 18))["days"][0]["batteries"] -check("in the night before the cheap run both charge from the grid in it " - f"(01:00 to {END_S}), then give at 17:00", - all([r["kind"] for r in b["blocks"]] == ["grid", "give"] for b in nb) - and all(b["blocks"][0]["from"] == "01:00" - and b["blocks"][0]["to"] == END_S for b in nb)) +# UX-77: today's block runs to where control stops charging: the 1.5 h run is +# sized at the full rating, the peak limit leaves 3.1 kW, so control plans the +# next run when it ends (`cheaproll`), and each battery's block ends at its stop. +RL = cheaproll.for_today(c_n, c_n.mode_runs, at(0, 18)) +ENDS = [S.brussels(RL.end_for(inv.name, END)).strftime("%H:%M") + for inv in c_n.inverters] +check("in the night before the cheap run both charge from the grid from " + f"01:00 to where control stops ({ENDS}, the run rolls on after " + f"{END_S}), then give at 17:00", + RL is not None and RL.end > END and ENDS[0] < ENDS[1] + and all([r["kind"] for r in b["blocks"]] == ["grid", "give"] for b in nb) + and all(b["blocks"][0]["from"] == "01:00" for b in nb) + and [b["blocks"][0]["to"] for b in nb] == ENDS) # UX-53: no grid block while the gate refuses. for why in ("peak", "not_worth"): c_n.mode_gate = why @@ -255,12 +264,16 @@ def at_h(b, h): LVL_A = c_n.plan_targets()[c_n.inverters[0].name] +END_HS = [(S.brussels(RL.end_for(inv.name, END)).replace(tzinfo=None) + - S.brussels(DAY0).replace(tzinfo=None)).total_seconds() / 3600.0 + for inv in c_n.inverters] check(f"the curve charges in a grid block: A (planned level {LVL_A:.0f} %) " - "and B (Cheap hours, to the end of its run) are well above where they " - "were at 01:00, and stay put after the run", - all(at_h(b, END_H) - at_h(b, 1.0) > 8.0 - and abs(at_h(b, END_H + 0.5) - at_h(b, END_H)) < 1e-6 - for b in nb) and nb[0]["levels"][0][1] == 30.0) + "and B (Cheap hours, to the end of its block) are well above where " + "they were at 01:00, and stay put after the block", + all(at_h(b, eh) - at_h(b, 1.0) > 8.0 + and all(abs(v - at_h(b, eh)) < 1e-6 for x, v in b["levels"] + if eh + 1e-6 < x < eh + 3.0) + for b, eh in zip(nb, END_HS)) and nb[0]["levels"][0][1] == 30.0) # Modes off: the verdict's legs. c_f = S.make("off", at(10, 7)) fb = tab(c_f, at(10, 7)) diff --git a/goodwe_controller/test_ux77.py b/goodwe_controller/test_ux77.py new file mode 100644 index 0000000..55b9900 --- /dev/null +++ b/goodwe_controller/test_ux77.py @@ -0,0 +1,307 @@ +"""UX-77: Home draws today's grid charge for the hours control will charge. + +The real `Controller._mode_roles()` and `homeview.build()`, stepped each +minute over a flat night (the walk of the investigation): at five times, every +minute control charges battery X in that local day lies inside X's drawn grid +block, the block does not run past the hour control stops charging X by more +than one slot, and the chart's cheap bars cover every hour with a charge. +The peak-bound night is the one that needs the peak limit's room (a mutant that +rolls at the full rating fails it, part 3); dynamic prices draw what they drew +before (part 4). Mutations (MUTATION.md, purged copies): listed in the pull +request. Run: python3 test_ux77.py +""" +import json +import logging +import os +import sys +from datetime import datetime, timedelta, timezone + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, HERE) +sys.path.insert(0, os.path.join(HERE, "tools")) +logging.disable(logging.WARNING) + +import ux54_scenarios as S # noqa: E402 +from app import dayplan, homeview, modetrade, packmode # noqa: E402 +from app.main import Controller # noqa: E402 +from app.peakguard import PeakDecision # noqa: E402 +from app.static_tariff import static_plan # noqa: E402 +from app.store import unit_key # noqa: E402 +from app.tariff import BILLING_TZ # noqa: E402 + +try: # not on the base, where it is red + from app import cheaproll +except ImportError: + cheaproll = None + +UTC = timezone.utc +EFF = 0.95 +AIM = 3450.0 +SLOT = timedelta(hours=1) # the price slot of the day/night tariff +SAMPLES = ("23:30", "00:30", "01:30", "02:30", "04:30") +OLD = ("leftover_solar", "cheap_hours") # the owner's old shape +NOW = ("leftover_solar", "leftover_solar") # the current one +PASS = FAIL = 0 + + +def check(name, ok, detail=""): + global PASS, FAIL + if ok: + PASS += 1 + else: + FAIL += 1 + print(f"FAIL {name} {detail}") + + +def no_ties(h): + """Two dips, no two slots at the same price: nothing rolls on.""" + return (0.05 if 1.0 <= h < 2.0 else 0.08 if 5.0 <= h < 6.0 + else 0.30) + 0.0001 * h + + +def shape(night=None): + d = {h: 0.973 * (600.0 if h < 6 or h >= 22 else 1200.0 if h < 10 + else 900.0 if h < 17 else 1800.0) for h in range(24)} + if night is not None: + for h in list(range(0, 7)) + [22, 23]: + d[h] = night + return d + + +def profile(sh, now): + d = now.astimezone(BILLING_TZ).date() + return dayplan.HouseProfile({(d - timedelta(days=k)).isoformat(): + {str(h): w for h, w in sh.items()} for k in (1, 2, 3)}) + + +def loc(t): + return t.astimezone(BILLING_TZ) + + +def hm(s, day): + h, m = map(int, s.split(":")) + return day.replace(hour=h, minute=m) + + +def walk(modes, soc0, house_w, night=None, dynamic=False, samples=SAMPLES, + start_h=22, end_h=7, plain=None): + """Walk 22:00-07:15. Returns (charging, homes, commands): + charging[minute] = set of batteries control charges (power > 0), + homes[sample] = Home's `/status` plan at that minute, commands = the + (minute, unit, role, size) control gave.""" + opts = dict(S.BASE) + opts.update(battery_modes_enabled=True, + pv_forecast_entities=[f"{S.FC1}_{i}" for i in range(3)], + pv_forecast_tomorrow_entities=[f"{S.FCT1}_{i}" + for i in range(3)], + static_profile=json.dumps(S.DN), arbitrage_enabled=False, + arbitrage_soc_floor=20, arbitrage_soc_ceiling=95, + cycle_cost_threshold_eur=0.04) + c = Controller(opts, S.Hass(), S.Store(), S.Pub(), stop=S.asyncio.Event()) + c.auto = True + soc = list(soc0) + caps = [inv.capacity_kwh for inv in c.inverters] + rating = [inv.max_w for inv in c.inverters] + for i, inv in enumerate(c.inverters): + inv.batt, inv.last_write = 0.0, 0.0 + inv.dev_min, inv.dev_max = -5000.0, 5000.0 + c.store.data[unit_key(inv.name, packmode.MODE_KEY)] = modes[i] + c.pv_w = 0.0 + c.peak_guard = PeakDecision(engaged=False, floor_w=None, ceiling_w=7000.0, + ceiling_from="cap", projected_w=800.0) + start = datetime(2026, 11, 1, start_h, 0, tzinfo=BILLING_TZ).astimezone(UTC) + end = datetime(2026, 11, 2, end_h, 15, tzinfo=BILLING_TZ).astimezone(UTC) + c.house_profile = profile(shape(night), start) + now, plan_at = start, None + charging, homes, cmds = {}, {}, [] + while now < end: + lt = loc(now) + today, tomorrow = (10.2, 10.0) if lt.hour < 12 else (10.0, 10.2) + c.solar_fc = S.forecast(now, today, tomorrow, members=3) + c.solar_fc_why = None + if plan_at is None or (now - plan_at).total_seconds() >= 1800: + c.tariff_plan = (S.quarters(now, dynamic) if dynamic else + static_plan(opts, now, now + timedelta(hours=48), + now=now)) + plan_at = now + c.tariff_at = plan_at + for i, inv in enumerate(c.inverters): + inv.soc = round(soc[i], 1) + room = AIM - house_w + c.arb_headroom_w = room + c.grid = house_w + tick = c._mode_roles(now, None) + size = modetrade.size(tick, (-sum(rating), sum(rating)), room) \ + if tick.action else 0.0 + ch = [i for i, inv in enumerate(c.inverters) + if tick.roles.get(inv.name) == modetrade.CHARGE] + p = [0.0, 0.0] + if size < 0 and ch: + tot = sum(rating[i] for i in ch) + for i in ch: + p[i] = -size * rating[i] / tot + for i in range(2): + soc[i] = min(100.0, soc[i] + p[i] / 1000.0 / 60.0 * EFF + / caps[i] * 100.0) + charging[lt.replace(second=0, microsecond=0)] = { + i for i in range(2) if p[i] > 0.0} + cmds.append((lt.strftime("%H:%M"), tick.action, + tuple(sorted(tick.roles.items())), round(size))) + if lt.strftime("%H:%M") in samples: + homes[lt.replace(second=0, microsecond=0)] = homeview.build( + c, level="ok", checks=[], now=now, lang="en") + if plain is not None: # the same, with no roll + real = cheaproll.for_today + cheaproll.for_today = lambda *_a: None + try: + plain[lt.replace(second=0, microsecond=0)] = \ + homeview.build(c, level="ok", checks=[], now=now, + lang="en") + finally: + cheaproll.for_today = real + now += timedelta(minutes=1) + return charging, homes, cmds + + +def blocks_of(st, i): + """Battery i's drawn grid blocks as (start, end) local datetimes.""" + return [(loc(datetime.fromisoformat(r["start"])), + loc(datetime.fromisoformat(r["end"]))) + for r in st["plan"]["runs"] + if r["kind"] == "grid" and r["battery"] == i] + + +def today_of(st, i): + return [(t["from"], t["to"]) for t in st["batteries"][i]["today"]] + + +def violations(charging, homes): + """What is wrong with Home's drawing at each sample.""" + bad = [] + for t, st in sorted(homes.items()): + day_end = (t.replace(hour=0, minute=0) + timedelta(days=1)) + mins = {m: who for m, who in charging.items() if t <= m < day_end} + for i in range(2): + mine = sorted(m for m, who in mins.items() if i in who) + bl = blocks_of(st, i) + for m in mine: + if not any(a <= m < z for a, z in bl): + bad.append(f"{t:%H:%M} battery {i}: charges at " + f"{m:%H:%M}, not in a block {bl}") + break + if mine: + stop = mine[-1] + timedelta(minutes=1) + for a, z in bl: + if z > stop + SLOT: + bad.append(f"{t:%H:%M} battery {i}: block to " + f"{z:%H:%M}, stops {stop:%H:%M}") + # the Today line says the same as the block + tl = today_of(st, i) + for m in mine: + if not any(f <= m.strftime("%H:%M") < (z if z != "24:00" + else "99") for f, z in tl): + bad.append(f"{t:%H:%M} battery {i}: Today line {tl} " + f"has no {m:%H:%M}") + break + cheap = {b["label"][:2] for b in st["plan"]["prices"] + if b["kind"] == "cheap"} + for m in mins: + if mins[m] and m.strftime("%H") not in cheap: + bad.append(f"{t:%H:%M}: charge at {m:%H:%M}, hour not cheap") + break + return bad + + +# --------------------------------------------------------------------------- # +# Run +# --------------------------------------------------------------------------- # +def show(tag, ch, homes, want_bad): + bad = violations(ch, homes) + if want_bad: + check(f"{tag}: red", bool(bad), "no violation found") + else: + check(f"{tag}: every charging minute is drawn, no block runs on", + not bad, "\n " + "\n ".join(bad)) + return bad + + +def charged(ch, i, after): + return [m for m, who in ch.items() if i in who and m >= after] + + +def main(): + global cheaproll + if cheaproll is None: + print("no app/cheaproll.py: the base; the walk is red there") + cheaproll = type("Off", (), {"for_today": staticmethod(lambda *a: None)}) + + print("1. the walk: no-bind (0.6 kW house) and peak-bound (2.85 kW house)") + CASES = (("old shape, no bind", OLD, (20.0, 20.0), 584.0, None), + ("current shape, no bind", NOW, (20.0, 20.0), 584.0, None), + ("old shape, peak-bound", OLD, (40.0, 90.0), 2850.0, 2850.0), + ("current shape, peak-bound", NOW, (40.0, 90.0), 2850.0, 2850.0)) + walked = {} + for tag, modes, soc, hw, night in CASES: + ch, homes, _cmds = walk(modes, soc, hw, night=night) + walked[tag] = (ch, homes) + show(tag, ch, homes, False) + # the check has something to check: charging goes on past 00:30 + day = datetime(2026, 11, 2, 0, 30, tzinfo=BILLING_TZ) + check(f"{tag}: control charges after 00:30", bool( + charged(ch, 0, day) or charged(ch, 1, day))) + + print("2. a battery with nothing to charge gets no block") + ch, homes = walked["old shape, peak-bound"] + st = homes[datetime(2026, 11, 2, 2, 30, tzinfo=BILLING_TZ)] + check("B is at its level at 02:30: no block", blocks_of(st, 1) == [], + str(blocks_of(st, 1))) + check("A still has one at 02:30", bool(blocks_of(st, 0))) + + print("3. a roll at the full rating fails the peak-bound night") + if hasattr(cheaproll, "_Charge"): + real = cheaproll._Charge._room_w + cheaproll._Charge._room_w = lambda self, hour, total: total + try: + for tag, modes, soc, hw, night in CASES[2:]: + ch, homes, _c = walk(modes, soc, hw, night=night) + show(f"mutant, {tag}", ch, homes, True) + finally: + cheaproll._Charge._room_w = real + + print("4. dynamic prices: Home is what it was") + for tag, modes, soc, hw, night in (CASES[2], CASES[3], CASES[1]): + plain = {} + ch, homes, _c = walk(modes, soc, hw, night=night, dynamic=no_ties, + plain=plain) + same = all(json.dumps(homes[t], sort_keys=True, default=str) + == json.dumps(plain[t], sort_keys=True, default=str) + for t in homes) + check(f"dynamic, {tag}: same as with no roll", same) + check(f"dynamic, {tag}: and there are grid blocks to compare", + any(r["kind"] == "grid" for st in homes.values() + for r in st["plan"]["runs"])) + + print("5. the pieces") + if hasattr(cheaproll, "Roll"): + t = datetime(2026, 11, 2, 1, 46, tzinfo=UTC) + check("a stop is rounded up to the slot", cheaproll._ceil_slot(t) + == datetime(2026, 11, 2, 2, 0, tzinfo=UTC)) + t = datetime(2026, 11, 2, 1, 45, tzinfo=UTC) + check("a stop on a slot stays", cheaproll._ceil_slot(t) == t) + e0 = datetime(2026, 11, 2, 0, 0, tzinfo=UTC) + e1 = datetime(2026, 11, 2, 6, 0, tzinfo=UTC) + r = cheaproll.Roll(e1, {"a": None, "b": datetime(2026, 11, 2, 2, 0, + tzinfo=UTC)}) + check("a battery still charging ends at the roll", r.end_for("a", e0) == e1) + check("a battery at its level ends at its stop", + r.end_for("b", e0) == datetime(2026, 11, 2, 2, 0, tzinfo=UTC)) + check("a battery that does not charge keeps the run's end", + r.end_for("c", e0) == e0) + check("the chart's bar runs to the last block", r.last(e0) == e1) + + print(f"\n{PASS} passed, {FAIL} failed") + return FAIL + + +if __name__ == "__main__": + sys.exit(1 if main() else 0) -- 2.54.0 From 6d2b9b53355607d7e923f6082d6b4b47cb08c188 Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Fri, 6 Nov 2026 09:11:00 +0100 Subject: [PATCH 05/22] UX-77: Home's roll shares the peak room by room left to full, as control does The cheap-run roll (cheaproll._Charge) shared the peak room by rating, control shares it by room left to full (packshare.weights). With unequal levels one battery's drawn block ended too early (rig F-1). The roll now uses packshare.weights over the simulated levels, each battery capped at its rating, a full one dropping out. test_ux77 gains two unequal-level walks and a by-rating mutant; its walks model control's split by room too. Co-Authored-By: Claude --- .../DECISIONS/UX-54-forecast-tab.md | 5 +- goodwe_controller/app/cheaproll.py | 41 ++++++++-- goodwe_controller/test_ux77.py | 75 ++++++++++++++++--- 3 files changed, 102 insertions(+), 19 deletions(-) diff --git a/goodwe_controller/DECISIONS/UX-54-forecast-tab.md b/goodwe_controller/DECISIONS/UX-54-forecast-tab.md index 577a681..54b66d7 100644 --- a/goodwe_controller/DECISIONS/UX-54-forecast-tab.md +++ b/goodwe_controller/DECISIONS/UX-54-forecast-tab.md @@ -101,8 +101,9 @@ is `SolarToday.add()` after the solar read, in a `try`, in memory. control will stop charging, as the tab does for tomorrow's: the cheap run rolls on at its end while a battery is short (`cheaproll.roll_end()`, the same loop as `dayview._levels()`), the level read at the charge power the - peak limit leaves (peak aim less the house, else the rating), and each - battery's block ends at its own stop. The curve stops charging at its level + peak limit leaves (peak aim less the house, else the rating) and shared by + room left to full as control shares it (`packshare.weights()`, re-read as + the levels move), and each battery's block ends at its own stop. The curve stops charging at its level (`charge_to()`) and giving at the floor. - Today, a give run in the plan on a battery that is empty now has no block on Home (no energy to give). The tab equals Home, and says so: "The add-on plans diff --git a/goodwe_controller/app/cheaproll.py b/goodwe_controller/app/cheaproll.py index e9eb515..d3d0346 100644 --- a/goodwe_controller/app/cheaproll.py +++ b/goodwe_controller/app/cheaproll.py @@ -10,8 +10,10 @@ that goes on until the batteries are at their level or the cheap hours end. (`modetrade.energy()`, `priceruns.plan()`), until nothing is short or the next run does not follow on. `for_today()` reads the batteries at the level they will have by then: the charge power is the peak limit's room (peak aim minus -the house, as `dayplan` counts it), shared by rating. Where that room is not -known (no house profile or no aim), the rating is used. +the house, as `dayplan` counts it), shared as control shares it: by room +left to full (`packshare.weights()`), read again as the levels move, so a +battery that fills drops out and its share goes to the others. Where that +room is not known (no house profile or no aim), the rating is used. DECISIONS/UX-54-forecast-tab.md has the record. """ @@ -20,7 +22,7 @@ import math from dataclasses import dataclass from datetime import datetime, timedelta, timezone -from . import arbitrage, dayplan, modetrade, priceruns +from . import arbitrage, dayplan, modetrade, packshare, priceruns from .dayplan import EFF from .price import required_spread, uncertainty_of @@ -85,7 +87,7 @@ def _ceil_slot(t): class _Charge: """The batteries' levels followed from `t0` while the chargers charge at the room the peak limit leaves: `min(sum of ratings, aim - house)`, shared - by rating, each stopping at `modetrade.charge_to()`.""" + by room left to full as control does (`_shares()`), each stopping at `modetrade.charge_to()`.""" def __init__(self, packs, t0, ceiling_pct, targets, aim_w, house): self.packs = {p.name: p for p in packs} @@ -115,6 +117,34 @@ class _Charge: self._charge(s, e, hour) self.t = nxt + def _shares(self, ch, w): + """Watts per charger: `w` shared by `packshare.weights()` over all + packs (room left to full, as control passes it), no charger above its + rating, the surplus going to the others. By rating where the room left + is not known (no capacity).""" + wt = packshare.weights(list(self.packs.values()), -1.0) + if wt is None: + wt = {p.name: abs(p.rating_w) for p in ch} + wt = {p.name: wt.get(p.name, 0.0) for p in ch} + cap = {p.name: abs(p.rating_w) for p in ch} + out, open_, left = {}, set(wt), float(w) + while open_: + tot = sum(wt[n] for n in open_) + if tot <= 0.0: + for n in open_: + out[n] = 0.0 + break + over = [n for n in open_ if left * wt[n] / tot > cap[n] + 1e-9] + if not over: + for n in open_: + out[n] = left * wt[n] / tot + break + for n in over: + out[n] = cap[n] + left -= cap[n] + open_.discard(n) + return out + def _charge(self, s, e, hour): ch = modetrade.chargers(list(self.packs.values()), self.ceil, targets=self.targets) @@ -123,12 +153,13 @@ class _Charge: return w = self._room_w(hour, total) hrs = (e - s).total_seconds() / 3600.0 + share = self._shares(ch, w) for p in ch: self.stops.setdefault(p.name, None) if not p.capacity_kwh: continue to = modetrade.charge_to(p, self.ceil, self.targets) - kwh = w * abs(p.rating_w) / total * hrs / 1000.0 * EFF + kwh = share[p.name] * hrs / 1000.0 * EFF room = (to - p.soc) / 100.0 * float(p.capacity_kwh) if kwh <= 0.0: continue diff --git a/goodwe_controller/test_ux77.py b/goodwe_controller/test_ux77.py index 55b9900..8ebd730 100644 --- a/goodwe_controller/test_ux77.py +++ b/goodwe_controller/test_ux77.py @@ -22,7 +22,7 @@ sys.path.insert(0, os.path.join(HERE, "tools")) logging.disable(logging.WARNING) import ux54_scenarios as S # noqa: E402 -from app import dayplan, homeview, modetrade, packmode # noqa: E402 +from app import dayplan, homeview, modetrade, packmode, packshare # noqa: E402 from app.main import Controller # noqa: E402 from app.peakguard import PeakDecision # noqa: E402 from app.static_tariff import static_plan # noqa: E402 @@ -38,6 +38,7 @@ UTC = timezone.utc EFF = 0.95 AIM = 3450.0 SLOT = timedelta(hours=1) # the price slot of the day/night tariff +QUARTER = timedelta(minutes=15) # the run's own slot SAMPLES = ("23:30", "00:30", "01:30", "02:30", "04:30") OLD = ("leftover_solar", "cheap_hours") # the owner's old shape NOW = ("leftover_solar", "leftover_solar") # the current one @@ -83,8 +84,27 @@ def hm(s, day): return day.replace(hour=h, minute=m) +def split_by_room(w, ch, rating, wt, invs): + """Control's split of `w` over the chargers `ch`: by `packshare.weights()`, + no battery over its rating, the surplus to the others.""" + p, open_, left = [0.0, 0.0], set(ch), w + while open_: + tot = sum(wt[invs[i].name] for i in open_) + over = [i for i in open_ + if left * wt[invs[i].name] / tot > rating[i] + 1e-9] + if not over: + for i in open_: + p[i] = left * wt[invs[i].name] / tot + break + for i in over: + p[i] = rating[i] + left -= rating[i] + open_.discard(i) + return p + + def walk(modes, soc0, house_w, night=None, dynamic=False, samples=SAMPLES, - start_h=22, end_h=7, plain=None): + start_h=22, end_h=7, plain=None, by_room=True, fc=(10.2, 10.0)): """Walk 22:00-07:15. Returns (charging, homes, commands): charging[minute] = set of batteries control charges (power > 0), homes[sample] = Home's `/status` plan at that minute, commands = the @@ -116,7 +136,7 @@ def walk(modes, soc0, house_w, night=None, dynamic=False, samples=SAMPLES, charging, homes, cmds = {}, {}, [] while now < end: lt = loc(now) - today, tomorrow = (10.2, 10.0) if lt.hour < 12 else (10.0, 10.2) + today, tomorrow = fc if lt.hour < 12 else fc[::-1] c.solar_fc = S.forecast(now, today, tomorrow, members=3) c.solar_fc_why = None if plan_at is None or (now - plan_at).total_seconds() >= 1800: @@ -137,9 +157,13 @@ def walk(modes, soc0, house_w, night=None, dynamic=False, samples=SAMPLES, if tick.roles.get(inv.name) == modetrade.CHARGE] p = [0.0, 0.0] if size < 0 and ch: - tot = sum(rating[i] for i in ch) - for i in ch: - p[i] = -size * rating[i] / tot + if by_room: # as control: room left to full, rating caps + wt = packshare.weights(c.pack_energy, -1.0) + p = split_by_room(-size, ch, rating, wt, c.inverters) + else: + tot = sum(rating[i] for i in ch) + for i in ch: + p[i] = -size * rating[i] / tot for i in range(2): soc[i] = min(100.0, soc[i] + p[i] / 1000.0 / 60.0 * EFF / caps[i] * 100.0) @@ -175,7 +199,7 @@ def today_of(st, i): return [(t["from"], t["to"]) for t in st["batteries"][i]["today"]] -def violations(charging, homes): +def violations(charging, homes, slot=SLOT): """What is wrong with Home's drawing at each sample.""" bad = [] for t, st in sorted(homes.items()): @@ -191,8 +215,10 @@ def violations(charging, homes): break if mine: stop = mine[-1] + timedelta(minutes=1) + if slot == QUARTER: # the last charging quarter's end + stop += timedelta(minutes=-stop.minute % 15) for a, z in bl: - if z > stop + SLOT: + if z > stop + slot: bad.append(f"{t:%H:%M} battery {i}: block to " f"{z:%H:%M}, stops {stop:%H:%M}") # the Today line says the same as the block @@ -215,8 +241,8 @@ def violations(charging, homes): # --------------------------------------------------------------------------- # # Run # --------------------------------------------------------------------------- # -def show(tag, ch, homes, want_bad): - bad = violations(ch, homes) +def show(tag, ch, homes, want_bad, slot=SLOT): + bad = violations(ch, homes, slot) if want_bad: check(f"{tag}: red", bool(bad), "no violation found") else: @@ -240,6 +266,11 @@ def main(): ("current shape, no bind", NOW, (20.0, 20.0), 584.0, None), ("old shape, peak-bound", OLD, (40.0, 90.0), 2850.0, 2850.0), ("current shape, peak-bound", NOW, (40.0, 90.0), 2850.0, 2850.0)) + # the rig's checks 2 and 3 (UX-77 round 2): room left and rating disagree + UNEQUAL = (("unequal levels, A+B Leftover solar, no bind", NOW, + (30.0, 20.0), 584.0, None, (5.0, 5.0)), + ("unequal levels, A Leftover solar + B Cheap hours", OLD, + (40.0, 88.0), 2850.0, 2850.0, (10.2, 10.0))) walked = {} for tag, modes, soc, hw, night in CASES: ch, homes, _cmds = walk(modes, soc, hw, night=night) @@ -251,7 +282,7 @@ def main(): charged(ch, 0, day) or charged(ch, 1, day))) print("2. a battery with nothing to charge gets no block") - ch, homes = walked["old shape, peak-bound"] + ch, homes, _c = walk(OLD, (40.0, 94.0), 2850.0, night=2850.0) st = homes[datetime(2026, 11, 2, 2, 30, tzinfo=BILLING_TZ)] check("B is at its level at 02:30: no block", blocks_of(st, 1) == [], str(blocks_of(st, 1))) @@ -281,7 +312,27 @@ def main(): any(r["kind"] == "grid" for st in homes.values() for r in st["plan"]["runs"])) - print("5. the pieces") + print("5. unequal levels: the roll shares the room as control does") + for tag, modes, soc, hw, night, fc in UNEQUAL: + ch, homes, _c = walk(modes, soc, hw, night=night, fc=fc) + show(tag, ch, homes, False, QUARTER) + day = datetime(2026, 11, 2, 0, 30, tzinfo=BILLING_TZ) + check(f"{tag}: control charges after 00:30", bool( + charged(ch, 0, day) or charged(ch, 1, day))) + if hasattr(cheaproll, "_Charge") \ + and hasattr(cheaproll._Charge, "_shares"): + real = cheaproll._Charge._shares + cheaproll._Charge._shares = lambda self, ch, w: { + p.name: w * abs(p.rating_w) / sum(abs(q.rating_w) for q in ch) + for p in ch} + try: + for tag, modes, soc, hw, night, fc in UNEQUAL: + ch, homes, _c = walk(modes, soc, hw, night=night, fc=fc) + show(f"mutant by rating, {tag}", ch, homes, True, QUARTER) + finally: + cheaproll._Charge._shares = real + + print("6. the pieces") if hasattr(cheaproll, "Roll"): t = datetime(2026, 11, 2, 1, 46, tzinfo=UTC) check("a stop is rounded up to the slot", cheaproll._ceil_slot(t) -- 2.54.0 From 5b6cb0bf084b82046b0d44da97562cc7cf2d0ec7 Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Fri, 6 Nov 2026 19:38:21 +0100 Subject: [PATCH 06/22] SAFETY-129: find readings 30 s late within about 90 s at a 5 s meter The meter's delay fit needed 16 new values in 60 s, so at the owner's 5 s P1 only the batteries' fit ran and 30 s late was found after about 3 minutes (export to -420 W). A meter that slow is now also fitted over 120 s; it calls a delay late only over 16 s, on two fits in a row, with the same corroboration as before, and never ends a verdict, so a slow battery (R2-1) runs as before. Found about 90 s after the start, export above -200 W. 10 s late at a 5 s meter is not found (nothing swings; it cannot be told from a 10 s battery). Co-Authored-By: Claude --- goodwe_controller/CHANGELOG.md | 1 + .../DECISIONS/SAFETY-128-late-readings.md | 13 +- goodwe_controller/app/lateness.py | 65 ++++-- goodwe_controller/test_safety129.py | 211 ++++++++++++++++++ 4 files changed, 271 insertions(+), 19 deletions(-) create mode 100644 goodwe_controller/test_safety129.py diff --git a/goodwe_controller/CHANGELOG.md b/goodwe_controller/CHANGELOG.md index 915b037..9def348 100644 --- a/goodwe_controller/CHANGELOG.md +++ b/goodwe_controller/CHANGELOG.md @@ -4,6 +4,7 @@ - Home now draws today's charge from the grid for every hour the add-on will really charge, not only the run under way ([[UX-77]]). - When Home Assistant passes readings on late, control no longer swings on them: it waits until the battery shows its last power request, and the peak limit no longer asks for full power on them. The page says so, and if the readings stay too late every inverter goes to 0 W ([[SAFETY-128]]). +- With a 5 s meter, readings 30 s late are now noticed after about 90 seconds, not 3 minutes ([[SAFETY-129]]). ## 0.20.1 diff --git a/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md b/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md index 13888d7..d1b280c 100644 --- a/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md +++ b/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md @@ -31,7 +31,11 @@ inverter limit in its last seconds (OPT-34 rig F-2). - **Meter** (`MeterFit`, round 1's fit): `meter(t) + request(t - d)` is the house alone at the right `d`. Quick, but confounded by the house. Needs 16 new values in 60 s, so only a meter of about 3.5 s or faster: at 5 s - it gave false delays. Late over 8 s. + it gave false delays. Late over 8 s. At a slower meter (a 5 s P1) the + same fit runs over 120 s and calls only a delay over 16 s late, on 2 fits + in a row, and never ends a verdict (SAFETY-129): a 10-12 s slow battery + reads up to 15 s there, so 10 s late is not told from it, and it does not + swing at 5 s. - A meter fit says late only with evidence from the batteries' side too: a request more than 100 W from the battery reading took 3 s or more to move a reading towards it (`SLOW_START_S`; on time they start within @@ -64,9 +68,10 @@ house at 1, 2 and 5 s ticks, so the commands are the same as without the gate. The fits run all the time; the pull request has the counts. **What it costs.** -- Found 36-68 s after a lag starts at 1-2 s meters, about 3 minutes at a 5 s - meter (battery fit only; the meter fit needs a faster meter). Until then control is as before. At 5 s and 30 s - late the export is cut to about a third, not removed. +- Found 36-68 s after a lag starts at 1-2 s meters; at a 5 s meter 30 s late + is found about 90 s after it starts (the loop first moves 100 W after 60 s, + SAFETY-129) with the export above -200 W, and 10 s late is not found. Until + then control is as before. - A battery that answers slowly is read as late readings and held: a dead time of 10 s or more (8 s reads 7-8; 10 s reads 10 at 1-2 s meters, 9 at 5 s), or a 15-20 s lag (review round 2). The GoodWe measured settles in 6 s diff --git a/goodwe_controller/app/lateness.py b/goodwe_controller/app/lateness.py index d673c0b..423342b 100644 --- a/goodwe_controller/app/lateness.py +++ b/goodwe_controller/app/lateness.py @@ -10,7 +10,8 @@ already fixed. The readings show it: they answer each power request late. house does not move that reading, so a repeating load cannot fool it. It needs the request to move FIT_MOVE_W; one reading per 10 s makes it slow. - `MeterFit`: how long after the site request the meter shows it. Quick at a - 1-3 s meter, none at a slower one, and a house load that repeats can fool it. + 1-3 s meter, slower and for long lags only at a 5 s one (SAFETY-129), and a + house load that repeats can fool it. The meter counts only when an ask was slow to start and no battery fit said "on time"; with hysteresis (ON_S / OFF_S) the result is `late_seen`. Then (`observe()`, each cycle): - a new command more than ANSWER_W from the battery reading is PENDING until @@ -163,13 +164,20 @@ class Delay: # The meter's delay: fitted over this much history, from this many new -# values (a meter of about 3.5 s or faster: a slower one gives too few to -# fit without false delays); late over M_LATE_S (the plant's 6 s settle and one 2 s meter tick), -# but only when the batteries' own readings have not just shown them on time. +# values (a meter of about 3.5 s or faster; a slower one see M_SLOW_FIT_S); +# late over M_LATE_S (the plant's 6 s settle and one 2 s meter tick), but only +# when the batteries' own readings have not just shown them on time. M_FIT_S = 60.0 M_ROWS = 16 M_LATE_S = freshness.PLANT_SETTLE_S + 2.0 # 8.0 M_SHARE, M_BETTER = 0.4, 0.7 +# SAFETY-129: a meter slower than that (a 5 s P1) has fewer than M_ROWS values +# in M_FIT_S, so it is fitted over M_SLOW_FIT_S instead, from the same M_ROWS +# values. It calls a delay late only over M_SLOW_LATE_S and on ON_FITS fits in +# a row: the slow-battery runs (10-12 s dead time, R2-1) read up to 15 s at a +# 5 s meter, and a 10 s lag cannot be told from them, nor does it swing. +M_SLOW_FIT_S = 120.0 +M_SLOW_LATE_S = 16.0 # A meter fit counts only after an ask that took this long to start moving a # reading: on time they start within 1-2 s (the recorded house, at 1-5 s). SLOW_START_S = 3.0 @@ -183,19 +191,30 @@ class MeterFit: up with an earlier request, so it only counts while the batteries' own fit is not on time (`Answer.watch`).""" - def __init__(self): + def __init__(self, fit_s=M_FIT_S, late_s=M_LATE_S): + self.fit_s, self.late_s = fit_s, late_s self.cmds = deque() # (t, request) when it changed self.grid = deque() # (t, meter) new values only + def slow(self): + """The meter is too slow for M_ROWS values in `fit_s`: the usual gap + between its new values (median, so a pause does not count) is over + fit_s / M_ROWS.""" + ts = [t for t, _g in self.grid] + if len(ts) < 5: + return False + gaps = sorted(b - a for a, b in zip(ts, ts[1:])) + return gaps[len(gaps) // 2] * M_ROWS > self.fit_s + def add(self, now, cmd, grid): if cmd is not None and (not self.cmds or self.cmds[-1][1] != cmd): self.cmds.append((now, float(cmd))) if grid is not None and (not self.grid or self.grid[-1][1] != grid): self.grid.append((now, float(grid))) - while self.grid and self.grid[0][0] < now - M_FIT_S: + while self.grid and self.grid[0][0] < now - self.fit_s: self.grid.popleft() while (len(self.cmds) > 1 - and self.cmds[1][0] <= now - M_FIT_S - MAX_DELAY_S): + and self.cmds[1][0] <= now - self.fit_s - MAX_DELAY_S): self.cmds.popleft() def estimate(self): @@ -214,9 +233,9 @@ class MeterFit: best = min(range(len(pure)), key=lambda d: (pure[d], d)) if g_var <= 0 or pure[best] > (1 - M_SHARE) * g_var: return None - if best <= M_LATE_S: + if best <= self.late_s: return float(best) - if pure[best] > M_BETTER * min(pure[:int(M_LATE_S) + 1]): + if pure[best] > M_BETTER * min(pure[:int(self.late_s) + 1]): return None return float(best) @@ -236,6 +255,8 @@ class Answer: self._was = None # (command, reading, meter) at the last call self.delay = Delay() self.meter = MeterFit() + self.meter_slow = MeterFit(M_SLOW_FIT_S, M_SLOW_LATE_S) # SAFETY-129 + self._ms_run = 0 # late slow-meter fits in a row self._batt_fit = None # (when, d): the batteries' last fit self._b_run = 0 # late battery fits in a row self._start_age = None # (when, how long the last ask took to start) @@ -279,14 +300,18 @@ class Answer: delays. Late on ON_FITS battery fits from ON_S, or on a meter fit over M_LATE_S when an ask was slow to start (SLOW_START_S) and no battery fit within FIT_S said on time: a house load that repeats can - fool the meter fit, not these. On time again on a battery fit under - OFF_S, or a meter fit within M_LATE_S unless a battery fit said late. + fool the meter fit, not these. A meter too slow for M_ROWS values in + M_FIT_S (a 5 s P1) is fitted over M_SLOW_FIT_S and counts only over + M_SLOW_LATE_S, on ON_FITS fits in a row. On time again on a battery fit + under OFF_S, or a meter fit within M_LATE_S unless a battery fit said + late. """ if self._start is None: self._start = now for name, cmd, reading in units: self.delay.add(now, name, cmd, reading) self.meter.add(now, standing_w, grid_w) + self.meter_slow.add(now, standing_w, grid_w) if self._fit_at is not None and now - self._fit_at < FIT_EVERY_S: return self._fit_at = now @@ -295,6 +320,9 @@ class Answer: and now - self._exempt_at <= FIT_S: b = None # a battery that stopped following: no late fit m = self.meter.estimate() + ms = None + if self.meter.slow(): + ms = self.meter_slow.estimate() if b is not None: self._batt_fit = (now, b) self.shown_s = b if b >= ON_S else None @@ -304,7 +332,13 @@ class Answer: b_late = recent and self._batt_fit[1] >= ON_S self._b_run = self._b_run + 1 if (b or 0.0) >= ON_S else 0 - d = b if b is not None else m + self._ms_run = self._ms_run + 1 if (ms or 0.0) > M_SLOW_LATE_S else 0 + # A slow-meter fit only ever adds a late verdict: it counts as a fit + # when it is late on ON_FITS in a row, and never ends one, so a slow + # battery (R2-1) runs exactly as before. + if self._ms_run < ON_FITS: + ms = None + d = b if b is not None else m if m is not None else ms if d is None: if self._fit_ok is None or now - self._fit_ok > FIT_GONE_S: self.late_seen = False # nothing to tell it by @@ -312,12 +346,13 @@ class Answer: self.fits += 1 self.delay_s, self._fit_ok = d, now warm = now - self._start >= MAX_DELAY_S + FIT_S - m_late = (m or 0.0) > M_LATE_S + m_late = ((m or 0.0) > M_LATE_S and warm) or ( + self._ms_run >= ON_FITS + and now - self._start >= MAX_DELAY_S + M_SLOW_FIT_S) slow = (self._start_age is not None and now - self._start_age[0] <= M_FIT_S and self._start_age[1] >= SLOW_START_S) - if self._b_run >= ON_FITS or ( - m_late and warm and slow and not b_fresh): + if self._b_run >= ON_FITS or (m_late and slow and not b_fresh): self.late_seen = True elif (b is not None and b < OFF_S) or ( m is not None and m <= M_LATE_S and not b_late): diff --git a/goodwe_controller/test_safety129.py b/goodwe_controller/test_safety129.py new file mode 100644 index 0000000..2d4d44b --- /dev/null +++ b/goodwe_controller/test_safety129.py @@ -0,0 +1,211 @@ +"""Runnable check for SAFETY-129: late Home Assistant readings found at a 5 s meter. +`python3 test_safety129.py` + +SAFETY-128 fitted the meter's delay only at a meter of about 3.5 s or faster +(16 new values in 60 s). The owner's P1 reports every 5 s, so there only the +batteries' fit ran: with readings 30 s late the loop swung to -420 W export +and was found after about 3 minutes (-1 325 W before the hold). +`lateness.MeterFit` now also runs at a slow meter over 120 s, and calls a +delay late only over `M_SLOW_LATE_S` on two fits in a row, with the same +corroboration as before (an ask slow to start, no battery fit on time). +DECISIONS/SAFETY-128-late-readings.md. + +The REAL Controller on TEST-23's plant (tools/safety128_rig.py). + +1. 5 s meter, readings 30 s late (N=2 house 1 800 W, N=1 house 1 500 W): found + within 120 s, and the export never goes below -200 W (red on c66aa7f: found + after 190 s, export to -420 W). +2. 5 s meter, readings 10 s late: no swing, nothing to hold; and the limit that + goes with it: a battery that answers in 10 s (R2-1) is still not read as + late at a 5 s meter, which a lower slow-meter threshold would break. +3. No false holds at a slow meter: the owner's shape, HA restarts and meter + pauses at 1, 2 and 5 s, a 4 s and a 10 s meter, 6-8 s dead times: never + late, so the commands are those of the run without the gate. +4. The slow-meter fit at its edges. + +Mutations (MUTATION.md, purged copies) are listed in the pull request. +""" + +import logging +import os +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, HERE) +sys.path.insert(0, os.path.join(HERE, "tools")) +logging.disable(logging.WARNING) + +import safety128_rig as R # noqa: E402 +import app.main as app_main # noqa: E402 +from app import lateness # noqa: E402 + +fails = [] + + +def check(name, cond): + print(f" {'ok ' if cond else 'FAIL'} {name}") + if not cond: + fails.append(name) + + +def no_gate(fn): + """`fn()` with the answer gate off: the tree before SAFETY-128.""" + real = app_main.Controller._readings_late + app_main.Controller._readings_late = lambda self, standing_w: None + try: + return fn() + finally: + app_main.Controller._readings_late = real + + +def run_late(**kw): + """Run the rig; (rig, seconds the verdict stood, first second it stood).""" + dur = kw.pop("dur") + seen = [] + r = R.Rig(**kw) + r.run(dur, lambda rig, c, t: seen.append(t) if c.answer.late_seen + else None) + return r, len(seen), min(seen) if seen else None + + +def grid_min(rows, t0, t1): + return min(r[1] for r in rows if t0 <= r[0] < t1) + + +AB, A = ("a", "b"), ("a",) +ON = 300.0 + +# --------------------------------------------------------------------------- # +print("1. 5 s meter, readings 30 s late for 300 s") +for units, house in ((AB, 1800.0), (A, 1500.0)): + n = f"N={len(units)}" + r, secs, first = run_late(house=house, lag_s=30.0, lag_on=ON, + lag_off=ON + 300.0, units=units, p1_s=5.0, + dur=ON + 420.0) + low = grid_min(r.rows, ON, ON + 300.0) + print(f" {n}: found {None if first is None else first - ON:.0f} s " + f"after the start, export to {low:.0f} W") + check(f"{n}: found within 120 s of the lag starting", + first is not None and first - ON <= 120.0) + check(f"{n}: the export never goes below -200 W", low >= -200.0) + old = R.Rig(house=house, lag_s=30.0, lag_on=ON, lag_off=ON + 300.0, + units=units, p1_s=5.0) + no_gate(lambda: old.run(ON + 300.0)) + check(f"{n}: without the gate the export goes below -1 kW", + grid_min(old.rows, ON, ON + 300.0) < -1000.0) + +# --------------------------------------------------------------------------- # +print("\n2. 5 s meter, readings 10 s late; a battery that answers in 10 s") +r, secs, first = run_late(house=1800.0, lag_s=10.0, lag_on=ON, + lag_off=ON + 150.0, units=AB, p1_s=5.0, + dur=ON + 270.0) +low = grid_min(r.rows, ON, ON + 150.0) +print(f" 10 s late: export to {low:.0f} W, verdict stood {secs} s") +check("10 s late: no swing, the export never goes below -200 W", low >= -200.0) + + +def owner(t): + w = 400.0 + if 600 <= t < 780: + w += 2200 + if 1200 <= t < 2400 and ((t - 1200) % 105) < 45: + w += 2000 + return w + + +for dead in (8.0, 10.0): + r, secs, first = run_late(house=owner, noise=80.0, units=AB, p1_s=5.0, + dead_s=dead, dur=1500.0) + print(f" {dead:.0f} s dead time, 5 s meter: late {secs} s") + check(f"{dead:.0f} s dead time at a 5 s meter is not read as late " + f"(R2-1 does not get worse)", secs == 0) + +# A 12 s dead time is already read as late (R2-1, SAFETY-128); the slow-meter +# fit must not make that longer. The same run with the slow fit off is the +# tree before this card. +_slow = lateness.MeterFit.slow +secs = {} +for label in ("with", "without"): + lateness.MeterFit.slow = _slow if label == "with" else (lambda self: False) + try: + secs[label] = run_late(house=owner, noise=80.0, units=A, p1_s=5.0, + dead_s=12.0, dur=3300.0)[1] + finally: + lateness.MeterFit.slow = _slow +print(f" 12 s dead time, N=1, 5 s meter: late {secs['with']} s, with the " + f"slow fit off {secs['without']} s") +check("a 12 s dead time at a 5 s meter is late no longer than before " + "(R2-1 does not get worse)", secs["with"] <= secs["without"]) + +# --------------------------------------------------------------------------- # +print("\n3. no false holds") +_view = R.Plant.view + + +def frozen(windows, meter_only): + def view(self, t, lag_s, lag_on, lag_off): + cur = _view(self, t, lag_s, lag_on, lag_off) + for a, b in windows: + if a <= t < b: + for row in reversed(self.hist): + if row[0] <= a + 1e-9: + old = row[1:] + return (old[0], old[1], cur[2], cur[3]) \ + if meter_only else old + return cur + return view + + +for p1 in (1.0, 2.0, 4.0, 5.0, 10.0): + kw = dict(house=owner, noise=80.0, units=AB, p1_s=p1) + r, secs, first = run_late(dur=1500.0, **kw) + new = r + old = R.Rig(**kw) + no_gate(lambda: old.run(1500.0)) + same = max(abs(x[2] - y[2]) for x, y in zip(new.rows, old.rows)) + print(f" owner shape, {p1:.0f} s meter: late {secs} s, command " + f"difference {same:.0f} W") + check(f"{p1:.0f} s meter, owner shape: never late, same commands", + secs == 0 and same == 0) +for label, windows, meter_only in ( + ("HA restart", ((650.0, 710.0), (1300.0, 1345.0)), False), + ("meter pause", ((640.0, 700.0), (1250.0, 1300.0)), True)): + for p1 in (1.0, 2.0, 5.0): + R.Plant.view = frozen(windows, meter_only) + try: + r, secs, first = run_late(house=owner, noise=80.0, units=AB, + p1_s=p1, dur=1400.0) + finally: + R.Plant.view = _view + check(f"{label} at a {p1:.0f} s meter: never late", secs == 0) + +# --------------------------------------------------------------------------- # +print("\n4. the slow-meter fit at its edges") + + +def meter_with(period, pause_after=None, pause_s=40.0, n=24): + f = lateness.MeterFit() + f.add(0.0, 100.0, 0.0) + for k in range(1, n): + gap = pause_s if pause_after is not None and k > pause_after else 0.0 + f.add(period * k + gap, None, float(k)) + return f + + +check("a 5 s and a 10 s meter are slow", + meter_with(5.0).slow() and meter_with(10.0).slow()) +check("a 1 s, a 2 s and a 3 s meter are not slow", + not any(meter_with(p, n=60).slow() for p in (1.0, 2.0, 3.0))) +check("a pause in a 2 s meter does not make it slow", + not meter_with(2.0, pause_after=10, n=40).slow()) +check("the slow fit asks for a longer window and a later delay than the " + "fast one", lateness.M_SLOW_FIT_S > lateness.M_FIT_S + and lateness.M_SLOW_LATE_S > lateness.M_LATE_S) +check("the slow fit's delay is above what a 10-12 s slow battery reads " + "(R2-1: up to 15 s at a 5 s meter)", lateness.M_SLOW_LATE_S >= 16.0) + +print() +if fails: + print(f"{len(fails)} FAILED: " + ", ".join(fails)) + sys.exit(1) +print("all checks passed") -- 2.54.0 From 2667cf0ddafba9993e12b9f3366ee4adaf0a7e28 Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Fri, 6 Nov 2026 20:04:08 +0100 Subject: [PATCH 07/22] SAFETY-129: the 5 s row no longer waits for the batteries' figure At a 5 s meter the slow meter fit now finds 30 s late first, and the hold leaves the batteries' fit nothing to measure, so the row says it without a figure. Co-Authored-By: Claude --- goodwe_controller/test_safety128.py | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/goodwe_controller/test_safety128.py b/goodwe_controller/test_safety128.py index b96d249..92d884b 100644 --- a/goodwe_controller/test_safety128.py +++ b/goodwe_controller/test_safety128.py @@ -229,9 +229,11 @@ late5 = R.Rig(lag_s=30.0, lag_on=ON, lag_off=ON + 300.0, p1_s=5.0) late5.run(ON + 300.0, lambda rig, c, t: rows5.extend( str(r["text"]) for r in c.checks() if r.get("id") == "late")) print(f" 5 s meter, 30 s late: {sorted(set(rows5))}") -check("at a 5 s meter (the batteries' fit) the row names the delay: about " - "20-32 s for 30 s late", - rows5 and set(rows5) <= _ok_texts and any("about" in x for x in rows5)) +# SAFETY-129: the 5 s meter's fit finds it first (no figure); the hold then +# leaves the batteries' fit nothing to measure, so the row names no delay. +check("at a 5 s meter the row says so, and names a delay only from the " + "batteries' fit: about 20-32 s for 30 s late", + rows5 and set(rows5) <= _ok_texts) check("logged once for the episode, and once when it ends, not per cycle", n_late == 1 and n_back == 1) check("the row goes once a fit shows the readings on time, or 2 minutes " -- 2.54.0 From 85433edac68daea00849a5d30e8600ce9164a867 Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Fri, 6 Nov 2026 21:24:37 +0100 Subject: [PATCH 08/22] SAFETY-129 round 2: shorter test_safety129, pinned 5 s row, 100 s bound Co-Authored-By: Claude --- goodwe_controller/CHANGELOG.md | 2 +- .../DECISIONS/SAFETY-128-late-readings.md | 2 +- goodwe_controller/test_safety128.py | 6 +- goodwe_controller/test_safety129.py | 60 ++++++++++++------- 4 files changed, 44 insertions(+), 26 deletions(-) diff --git a/goodwe_controller/CHANGELOG.md b/goodwe_controller/CHANGELOG.md index 9def348..f5c1b6e 100644 --- a/goodwe_controller/CHANGELOG.md +++ b/goodwe_controller/CHANGELOG.md @@ -4,7 +4,7 @@ - Home now draws today's charge from the grid for every hour the add-on will really charge, not only the run under way ([[UX-77]]). - When Home Assistant passes readings on late, control no longer swings on them: it waits until the battery shows its last power request, and the peak limit no longer asks for full power on them. The page says so, and if the readings stay too late every inverter goes to 0 W ([[SAFETY-128]]). -- With a 5 s meter, readings 30 s late are now noticed after about 90 seconds, not 3 minutes ([[SAFETY-129]]). +- With a 5 s meter, readings 30 s late are now noticed within 100 seconds, not 3 minutes ([[SAFETY-129]]). ## 0.20.1 diff --git a/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md b/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md index d1b280c..4d9a8f3 100644 --- a/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md +++ b/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md @@ -69,7 +69,7 @@ The fits run all the time; the pull request has the counts. **What it costs.** - Found 36-68 s after a lag starts at 1-2 s meters; at a 5 s meter 30 s late - is found about 90 s after it starts (the loop first moves 100 W after 60 s, + is found within 100 s after it starts (88 s measured) (the loop first moves 100 W after 60 s, SAFETY-129) with the export above -200 W, and 10 s late is not found. Until then control is as before. - A battery that answers slowly is read as late readings and held: a dead diff --git a/goodwe_controller/test_safety128.py b/goodwe_controller/test_safety128.py index 92d884b..1ddeb43 100644 --- a/goodwe_controller/test_safety128.py +++ b/goodwe_controller/test_safety128.py @@ -231,9 +231,9 @@ late5.run(ON + 300.0, lambda rig, c, t: rows5.extend( print(f" 5 s meter, 30 s late: {sorted(set(rows5))}") # SAFETY-129: the 5 s meter's fit finds it first (no figure); the hold then # leaves the batteries' fit nothing to measure, so the row names no delay. -check("at a 5 s meter the row says so, and names a delay only from the " - "batteries' fit: about 20-32 s for 30 s late", - rows5 and set(rows5) <= _ok_texts) +check("at a 5 s meter the row says the readings are late, with no figure " + "(the meter alone found it)", + rows5 and set(rows5) == {str(messages.row_readings_late(None))}) check("logged once for the episode, and once when it ends, not per cycle", n_late == 1 and n_back == 1) check("the row goes once a fit shows the readings on time, or 2 minutes " diff --git a/goodwe_controller/test_safety129.py b/goodwe_controller/test_safety129.py index 2d4d44b..7ef4a23 100644 --- a/goodwe_controller/test_safety129.py +++ b/goodwe_controller/test_safety129.py @@ -13,13 +13,13 @@ DECISIONS/SAFETY-128-late-readings.md. The REAL Controller on TEST-23's plant (tools/safety128_rig.py). 1. 5 s meter, readings 30 s late (N=2 house 1 800 W, N=1 house 1 500 W): found - within 120 s, and the export never goes below -200 W (red on c66aa7f: found + within 100 s, and the export never goes below -200 W (red on c66aa7f: found after 190 s, export to -420 W). 2. 5 s meter, readings 10 s late: no swing, nothing to hold; and the limit that goes with it: a battery that answers in 10 s (R2-1) is still not read as late at a 5 s meter, which a lower slow-meter threshold would break. -3. No false holds at a slow meter: the owner's shape, HA restarts and meter - pauses at 1, 2 and 5 s, a 4 s and a 10 s meter, 6-8 s dead times: never +3. No false holds at a slow meter: the owner's shape at 2, 5 and 10 s, HA + restarts and meter pauses at 2 and 5 s, 8-10 s dead times: never late, so the commands are those of the run without the gate. 4. The slow-meter fit at its edges. @@ -85,20 +85,21 @@ for units, house in ((AB, 1800.0), (A, 1500.0)): low = grid_min(r.rows, ON, ON + 300.0) print(f" {n}: found {None if first is None else first - ON:.0f} s " f"after the start, export to {low:.0f} W") - check(f"{n}: found within 120 s of the lag starting", - first is not None and first - ON <= 120.0) + check(f"{n}: found within 100 s of the lag starting", + first is not None and first - ON <= 100.0) check(f"{n}: the export never goes below -200 W", low >= -200.0) - old = R.Rig(house=house, lag_s=30.0, lag_on=ON, lag_off=ON + 300.0, - units=units, p1_s=5.0) - no_gate(lambda: old.run(ON + 300.0)) - check(f"{n}: without the gate the export goes below -1 kW", - grid_min(old.rows, ON, ON + 300.0) < -1000.0) + if len(units) == 2: + old = R.Rig(house=house, lag_s=30.0, lag_on=ON, lag_off=ON + 300.0, + units=units, p1_s=5.0) + no_gate(lambda: old.run(ON + 300.0)) + check(f"{n}: without the gate the export goes below -1 kW", + grid_min(old.rows, ON, ON + 300.0) < -1000.0) # --------------------------------------------------------------------------- # print("\n2. 5 s meter, readings 10 s late; a battery that answers in 10 s") r, secs, first = run_late(house=1800.0, lag_s=10.0, lag_on=ON, lag_off=ON + 150.0, units=AB, p1_s=5.0, - dur=ON + 270.0) + dur=ON + 200.0) low = grid_min(r.rows, ON, ON + 150.0) print(f" 10 s late: export to {low:.0f} W, verdict stood {secs} s") check("10 s late: no swing, the export never goes below -200 W", low >= -200.0) @@ -115,7 +116,7 @@ def owner(t): for dead in (8.0, 10.0): r, secs, first = run_late(house=owner, noise=80.0, units=AB, p1_s=5.0, - dead_s=dead, dur=1500.0) + dead_s=dead, dur=800.0) print(f" {dead:.0f} s dead time, 5 s meter: late {secs} s") check(f"{dead:.0f} s dead time at a 5 s meter is not read as late " f"(R2-1 does not get worse)", secs == 0) @@ -125,11 +126,11 @@ for dead in (8.0, 10.0): # tree before this card. _slow = lateness.MeterFit.slow secs = {} -for label in ("with", "without"): +for label in ("with", "without"): # 12 s dead time: late from about 140 s lateness.MeterFit.slow = _slow if label == "with" else (lambda self: False) try: secs[label] = run_late(house=owner, noise=80.0, units=A, p1_s=5.0, - dead_s=12.0, dur=3300.0)[1] + dead_s=12.0, dur=900.0)[1] finally: lateness.MeterFit.slow = _slow print(f" 12 s dead time, N=1, 5 s meter: late {secs['with']} s, with the " @@ -156,25 +157,25 @@ def frozen(windows, meter_only): return view -for p1 in (1.0, 2.0, 4.0, 5.0, 10.0): +for p1 in (2.0, 5.0, 10.0): kw = dict(house=owner, noise=80.0, units=AB, p1_s=p1) - r, secs, first = run_late(dur=1500.0, **kw) + r, secs, first = run_late(dur=800.0, **kw) new = r old = R.Rig(**kw) - no_gate(lambda: old.run(1500.0)) + no_gate(lambda: old.run(800.0)) same = max(abs(x[2] - y[2]) for x, y in zip(new.rows, old.rows)) print(f" owner shape, {p1:.0f} s meter: late {secs} s, command " f"difference {same:.0f} W") check(f"{p1:.0f} s meter, owner shape: never late, same commands", secs == 0 and same == 0) for label, windows, meter_only in ( - ("HA restart", ((650.0, 710.0), (1300.0, 1345.0)), False), - ("meter pause", ((640.0, 700.0), (1250.0, 1300.0)), True)): - for p1 in (1.0, 2.0, 5.0): + ("HA restart", ((650.0, 710.0),), False), + ("meter pause", ((640.0, 700.0),), True)): + for p1 in (2.0, 5.0): R.Plant.view = frozen(windows, meter_only) try: r, secs, first = run_late(house=owner, noise=80.0, units=AB, - p1_s=p1, dur=1400.0) + p1_s=p1, dur=800.0) finally: R.Plant.view = _view check(f"{label} at a {p1:.0f} s meter: never late", secs == 0) @@ -198,6 +199,23 @@ check("a 1 s, a 2 s and a 3 s meter are not slow", not any(meter_with(p, n=60).slow() for p in (1.0, 2.0, 3.0))) check("a pause in a 2 s meter does not make it slow", not meter_with(2.0, pause_after=10, n=40).slow()) +def fed(values): + """Answer.watch() with the slow fit reading `values`, one per fit.""" + a = lateness.Answer() + a.meter.slow = lambda: True + a.meter.estimate = a.delay.estimate = lambda: None + it = iter(values) + a.meter_slow.estimate = lambda: next(it) + for k in range(len(values)): + a.watch(2.0 * k, [], 0.0, 0.0) + return a + + +check("one late slow-meter fit is no fit; two in a row are one, and " + "one on time or in between is none", + fed([30.0]).fits == 0 and fed([30.0, 30.0]).fits == 1 + and fed([30.0, None, 30.0]).fits == 0 and fed([30.0, 12.0]).fits == 0 + and fed([3.0, 3.0]).fits == 0 and not fed([30.0, 30.0]).late_seen) check("the slow fit asks for a longer window and a later delay than the " "fast one", lateness.M_SLOW_FIT_S > lateness.M_FIT_S and lateness.M_SLOW_LATE_S > lateness.M_LATE_S) -- 2.54.0 From 03541a1eee88d81ed10efb84a0659ab8aee0ec18 Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Fri, 6 Nov 2026 22:12:39 +0100 Subject: [PATCH 09/22] SAFETY-129 round 2: next() with a default in test_safety129 Co-Authored-By: Claude --- goodwe_controller/test_safety129.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/goodwe_controller/test_safety129.py b/goodwe_controller/test_safety129.py index 7ef4a23..11c1c45 100644 --- a/goodwe_controller/test_safety129.py +++ b/goodwe_controller/test_safety129.py @@ -205,7 +205,7 @@ def fed(values): a.meter.slow = lambda: True a.meter.estimate = a.delay.estimate = lambda: None it = iter(values) - a.meter_slow.estimate = lambda: next(it) + a.meter_slow.estimate = lambda: next(it, None) for k in range(len(values)): a.watch(2.0 * k, [], 0.0, 0.0) return a -- 2.54.0 From c70eeb691650c7e7cffbc7eb8087ec193f64bb37 Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Sat, 7 Nov 2026 00:12:33 +0100 Subject: [PATCH 10/22] SAFETY-129 round 3: the real reason for the 16 s limit, a slow-start check, the row's tail Co-Authored-By: Claude --- .../DECISIONS/SAFETY-128-late-readings.md | 11 +++++++---- goodwe_controller/app/lateness.py | 6 ++++-- goodwe_controller/test_safety129.py | 13 +++++++++++-- 3 files changed, 22 insertions(+), 8 deletions(-) diff --git a/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md b/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md index 4d9a8f3..6b9e644 100644 --- a/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md +++ b/goodwe_controller/DECISIONS/SAFETY-128-late-readings.md @@ -33,9 +33,10 @@ inverter limit in its last seconds (OPT-34 rig F-2). 16 new values in 60 s, so only a meter of about 3.5 s or faster: at 5 s it gave false delays. Late over 8 s. At a slower meter (a 5 s P1) the same fit runs over 120 s and calls only a delay over 16 s late, on 2 fits - in a row, and never ends a verdict (SAFETY-129): a 10-12 s slow battery - reads up to 15 s there, so 10 s late is not told from it, and it does not - swing at 5 s. + in a row, and never ends a verdict (SAFETY-129). Alone it reads 35-90 s on healthy + runs, a few times in a row; only the slow-start and battery-fit check + below keeps those from a verdict. 10 s late is not found: it cannot be + told from a 10 s slow battery, and it does not swing at 5 s. - A meter fit says late only with evidence from the batteries' side too: a request more than 100 W from the battery reading took 3 s or more to move a reading towards it (`SLOW_START_S`; on time they start within @@ -71,7 +72,9 @@ The fits run all the time; the pull request has the counts. - Found 36-68 s after a lag starts at 1-2 s meters; at a 5 s meter 30 s late is found within 100 s after it starts (88 s measured) (the loop first moves 100 W after 60 s, SAFETY-129) with the export above -200 W, and 10 s late is not found. Until - then control is as before. + then control is as before. In a quiet house the "readings late" row stays + about 2 minutes longer after the lag ends (126 s against 32 s on base); no + harm measured, and it clears as early as on base when the load changes. - A battery that answers slowly is read as late readings and held: a dead time of 10 s or more (8 s reads 7-8; 10 s reads 10 at 1-2 s meters, 9 at 5 s), or a 15-20 s lag (review round 2). The GoodWe measured settles in 6 s diff --git a/goodwe_controller/app/lateness.py b/goodwe_controller/app/lateness.py index 423342b..84480ef 100644 --- a/goodwe_controller/app/lateness.py +++ b/goodwe_controller/app/lateness.py @@ -174,8 +174,10 @@ M_SHARE, M_BETTER = 0.4, 0.7 # SAFETY-129: a meter slower than that (a 5 s P1) has fewer than M_ROWS values # in M_FIT_S, so it is fitted over M_SLOW_FIT_S instead, from the same M_ROWS # values. It calls a delay late only over M_SLOW_LATE_S and on ON_FITS fits in -# a row: the slow-battery runs (10-12 s dead time, R2-1) read up to 15 s at a -# 5 s meter, and a 10 s lag cannot be told from them, nor does it swing. +# a row. Alone it reads 35-90 s on healthy runs, a few times in a row; what +# keeps those from a verdict is the slow-start and battery-fit check in +# watch(), not the limit. A 10 s lag is not found: it cannot be told from a +# slow battery (R2-1), nor does it swing at 5 s. M_SLOW_FIT_S = 120.0 M_SLOW_LATE_S = 16.0 # A meter fit counts only after an ask that took this long to start moving a diff --git a/goodwe_controller/test_safety129.py b/goodwe_controller/test_safety129.py index 11c1c45..08c96df 100644 --- a/goodwe_controller/test_safety129.py +++ b/goodwe_controller/test_safety129.py @@ -121,6 +121,15 @@ for dead in (8.0, 10.0): check(f"{dead:.0f} s dead time at a 5 s meter is not read as late " f"(R2-1 does not get worse)", secs == 0) +# At a 6 s meter an 8 s dead time reads late on the slow fit alone; only the +# slow-start and battery-fit check keeps it from a verdict (122 s late +# without the check, from 1 084 s). +r, secs, first = run_late(house=owner, noise=80.0, units=AB, p1_s=6.0, + dead_s=8.0, dur=1500.0) +print(f" 8 s dead time, 6 s meter: late {secs} s") +check("an 8 s dead time at a 6 s meter is never late (the slow fit needs an " + "ask slow to start and no battery fit on time)", secs == 0) + # A 12 s dead time is already read as late (R2-1, SAFETY-128); the slow-meter # fit must not make that longer. The same run with the slow fit off is the # tree before this card. @@ -219,8 +228,8 @@ check("one late slow-meter fit is no fit; two in a row are one, and " check("the slow fit asks for a longer window and a later delay than the " "fast one", lateness.M_SLOW_FIT_S > lateness.M_FIT_S and lateness.M_SLOW_LATE_S > lateness.M_LATE_S) -check("the slow fit's delay is above what a 10-12 s slow battery reads " - "(R2-1: up to 15 s at a 5 s meter)", lateness.M_SLOW_LATE_S >= 16.0) +check("the slow fit's late limit is above the fast one's and above a " + "10 s lag", lateness.M_SLOW_LATE_S >= 16.0) print() if fails: -- 2.54.0 From c43873b142530b0da30286b5583e556fd569cd11 Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Sat, 7 Nov 2026 04:24:01 +0100 Subject: [PATCH 11/22] OPT-38: a kept tomorrow forecast reading stands in at any hour One stale Forecast.Solar tomorrow sensor dropped the whole OPT-17 sum from 17:54 to 20:00. Today readings and the OPT-34 morning rules are unchanged. Co-Authored-By: Claude --- goodwe_controller/CHANGELOG.md | 1 + ...PT-27-last-fresh-forecast-for-the-night.md | 2 + goodwe_controller/app/solarfc.py | 8 +- goodwe_controller/test_opt27.py | 5 +- goodwe_controller/test_opt34.py | 5 +- goodwe_controller/test_opt38.py | 126 ++++++++++++++++++ 6 files changed, 140 insertions(+), 7 deletions(-) create mode 100644 goodwe_controller/test_opt38.py diff --git a/goodwe_controller/CHANGELOG.md b/goodwe_controller/CHANGELOG.md index f5c1b6e..6ebdc4f 100644 --- a/goodwe_controller/CHANGELOG.md +++ b/goodwe_controller/CHANGELOG.md @@ -5,6 +5,7 @@ - Home now draws today's charge from the grid for every hour the add-on will really charge, not only the run under way ([[UX-77]]). - When Home Assistant passes readings on late, control no longer swings on them: it waits until the battery shows its last power request, and the peak limit no longer asks for full power on them. The page says so, and if the readings stay too late every inverter goes to 0 W ([[SAFETY-128]]). - With a 5 s meter, readings 30 s late are now noticed within 100 seconds, not 3 minutes ([[SAFETY-129]]). +- If one of the tomorrow solar forecast sensors stops updating in the afternoon, tomorrow's forecast now stays on the Forecast tab instead of vanishing until 20:00 ([[OPT-38]]). ## 0.20.1 diff --git a/goodwe_controller/DECISIONS/OPT-27-last-fresh-forecast-for-the-night.md b/goodwe_controller/DECISIONS/OPT-27-last-fresh-forecast-for-the-night.md index ae9d68a..11f6932 100644 --- a/goodwe_controller/DECISIONS/OPT-27-last-fresh-forecast-for-the-night.md +++ b/goodwe_controller/DECISIONS/OPT-27-last-fresh-forecast-for-the-night.md @@ -44,3 +44,5 @@ tonight." (`home_words.solar.kept`, EN and NL). The event log says **Not changed.** Control with modes off: it does not use the forecast. The screens still show the line, so it says "kept", not "used for the plan". A forecast that stays fresh. The hold does not look at the weather, only at the clock and the day. + +**OPT-38 (2026-11-07).** A kept TOMORROW reading (at most `KEPT_MAX_AGE_S` old, same `day0`) now stands in at any hour, not only from `BELL_TO_H`: one sensor that stopped at 14:54 had dropped the whole sum (OPT-17) from 17:54 to 20:00. Today readings and the OPT-34 morning rules are unchanged; `test_opt38.py`. diff --git a/goodwe_controller/app/solarfc.py b/goodwe_controller/app/solarfc.py index 3928d71..3cda6dc 100644 --- a/goodwe_controller/app/solarfc.py +++ b/goodwe_controller/app/solarfc.py @@ -34,7 +34,9 @@ evening and api.forecast.solar is sometimes out for a poll. A kept reading stands in for its sensor (a) for up to `FRESH_S` when the sensor is missing or unavailable, as a sensor that merely stopped updating is already tolerated, and (b) from the end of the bell (`BELL_TO_H`) until local -midnight, if it is at most `KEPT_MAX_AGE_S` old when used. +midnight, if it is at most `KEPT_MAX_AGE_S` old when used. A TOMORROW +reading needs no evening for (b), at any hour (OPT-38: one Forecast.Solar +sensor stopped at 14:54 and tomorrow vanished from 17:54 to 20:00). Never beyond that, and never for another day: a today figure is only today's, a tomorrow figure is tomorrow's until midnight and then, until the bell starts (`BELL_FROM_H`, 06:00 local), today's (OPT-34: at most `KEPT_MORNING_MAX_AGE_S` @@ -347,8 +349,8 @@ class Hold: return (day, at, True) if ok else None if age > KEPT_MAX_AGE_S: return None - # A tomorrow sensor is asked with day0 = the next midnight, which is - # in the future all day, so the evening ends at that day's end. + if kind == "tomorrow": + return day, at, True # OPT-38: no evening needed evening = _local_hour(now) >= BELL_TO_H and now < _bounds(day0)[1] return (day, at, True) if evening else None diff --git a/goodwe_controller/test_opt27.py b/goodwe_controller/test_opt27.py index 0967f3e..1d0e7a8 100644 --- a/goodwe_controller/test_opt27.py +++ b/goodwe_controller/test_opt27.py @@ -144,8 +144,9 @@ rs(st(5.0, at(4, 9, 5)), st(12.0, at(4, 9, 5)), at(4, 9, 6), h9) for label, tm in (("stale", st(12.0, at(4, 9, 5))), ("unavailable", GONE)): fd = rs(st(3.0, at(4, 13, 55)), tm, at(4, 14, 0), h9)[0] check(f"14:00, today fresh, tomorrow {label} since 09:05 (over 3 h): " - "no tomorrow, nothing kept", - fd is not None and fd.tomorrow is None and not fd.kept) + "the 09:05 figure covers it, kept (OPT-38: at any hour)", + fd is not None and fd.tomorrow is not None + and fd.tomorrow.kwh == 12.0 and fd.kept) fn = rs(st(0.1, at(4, 20, 55)), tm, at(4, 21, 0), h9)[0] check(f"21:00, tomorrow {label}: the 09:05 figure covers it, kept", fn is not None and fn.tomorrow is not None diff --git a/goodwe_controller/test_opt34.py b/goodwe_controller/test_opt34.py index 869e7fc..6165010 100644 --- a/goodwe_controller/test_opt34.py +++ b/goodwe_controller/test_opt34.py @@ -186,8 +186,9 @@ check("a fresh figure replaces the kept one at once (04:00: 5.0 today, " and fy.today.kwh == 5.0 and fy.tomorrow.kwh == 9.0) fy2 = rs(st(5.0, at(4, 7, 55)), GONE, at(4, 8, 0), hd)[0] check("... and at 08:00, today fresh and tomorrow's sensor down for over 3 h: " - "no tomorrow (the 3rd's figure is not the 5th's)", - fy2 is not None and fy2.tomorrow is None and fy2.today.kwh == 5.0) + "tomorrow is the 03:50 figure for the 5th (9.0, kept: OPT-38), never " + "the 3rd's 12.0", fy2 is not None and fy2.today.kwh == 5.0 + and fy2.tomorrow is not None and fy2.tomorrow.kwh == 9.0 and fy2.kept) # Restart: the saved hold keeps the figure. for label, now in (("01:00", at(4, 1, 0)), ("04:00", at(4, 4, 0))): diff --git a/goodwe_controller/test_opt38.py b/goodwe_controller/test_opt38.py new file mode 100644 index 0000000..d4ef310 --- /dev/null +++ b/goodwe_controller/test_opt38.py @@ -0,0 +1,126 @@ +"""Runnable check for OPT-38. `python3 test_opt38.py` + +Live 0.20.1, 2026-11-06: one of three Forecast.Solar tomorrow sensors last +reported 14:54. From 17:54 (over FRESH_S) to 20:00 (BELL_TO_H) nothing stood in +for it, the OPT-17 sum was dropped and the Forecast tab had no tomorrow. A kept +tomorrow reading now stands in at any hour (`Hold.stand_in()`). Today members, +and the OPT-34 morning rules, are as before. + +Mutations (MUTATION.md, purged copies): see the pull request. +""" + +import os +import sys +from datetime import datetime, timedelta, timezone + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, HERE) + +from app import solarfc # noqa: E402 + +fails = [] + + +def check(name, cond): + print(f" {'ok ' if cond else 'FAIL'} {name}") + if not cond: + fails.append(name) + + +L = timezone(timedelta(hours=1)) # Brussels in winter (CET) + + +def at(day, hh, mm=0): + return datetime(2026, 11, day, hh, mm, tzinfo=L).astimezone(timezone.utc) + + +def st(kwh, when): + return {"state": f"{kwh}", "attributes": {"unit_of_measurement": "kWh"}, + "last_updated": when.isoformat(), "last_reported": when.isoformat()} + + +TODAY = [("sensor.t", None)] +M = ["sensor.m1", "sensor.m2", "sensor.m3"] + + +def run(hold, readings, now, today_at=None): + t = [("sensor.t", st(0.5, today_at or now - timedelta(minutes=5)))] + return solarfc.read_sum( + t, [(e, st(k, w)) for e, (k, w) in zip(M, readings)], now, hold) + + +# The live day: m2 last reported 14:54, the others keep reporting. +h = solarfc.Hold() +run(h, [(4.0, at(6, 14, 0)), (5.0, at(6, 14, 54)), (6.0, at(6, 14, 0))], + at(6, 14, 55)) +fresh = lambda w: [(4.0, w), (5.0, at(6, 14, 54)), (6.0, w)] # noqa: E731 +f, why, gaps = run(h, fresh(at(6, 18, 20)), at(6, 18, 30)) +check("18:30, one member last reported 14:54: tomorrow sum is there, kept", + f is not None and f.tomorrow is not None and f.tomorrow.kwh == 15.0 + and f.kept and gaps == ()) +f, _w, _g = run(h, fresh(at(6, 17, 40)), at(6, 17, 50)) +check("17:50 (just under 3 h): fresh, not kept", + f is not None and f.tomorrow.kwh == 15.0 and not f.kept) +f, _w, _g = run(h, fresh(at(6, 17, 55)), at(6, 18, 0)) +check("18:00 (stale since 17:54): kept, tomorrow there", + f is not None and f.tomorrow is not None and f.kept) +f, _w, _g = run(h, fresh(at(6, 19, 50)), at(6, 19, 55)) +check("19:55 (before the bell ends): tomorrow there", f.tomorrow is not None) + +# Too old: over KEPT_MAX_AGE_S (12 h) when used. +h2 = solarfc.Hold() +run(h2, [(4.0, at(6, 5, 0)), (5.0, at(6, 5, 0)), (6.0, at(6, 5, 0))], + at(6, 5, 5)) +w = at(6, 16, 50) +f, why, gaps = run(h2, [(4.0, w), (5.0, at(6, 5, 0)), (6.0, w)], + at(6, 17, 10)) +check("member 12 h 10 min old: no tomorrow sum (OPT-17 rule), today stays", + f is not None and f.tomorrow is None + and [g[0] for g in gaps] == ["tomorrow"]) +f, _w, _g = run(h2, [(4.0, w), (5.0, at(6, 5, 0)), (6.0, w)], + at(6, 16, 50)) +check("... and 11 h 50 min old is still kept", + f is not None and f.tomorrow is not None and f.kept) + +# Not a partial sum: a member never read has nothing to stand in. +f, _w, gaps = run(solarfc.Hold(), [(4.0, at(6, 18, 0)), + (5.0, at(6, 14, 54)), + (6.0, at(6, 18, 0))], at(6, 18, 30)) +check("a member with no kept reading: no tomorrow sum, no partial sum", + f is not None and f.tomorrow is None and len(gaps) == 1) + +# Today members are as before: a stale today member at 18:30 is not kept. +h3 = solarfc.Hold() +solarfc.read_sum([("sensor.t", st(1.0, at(6, 14, 54)))], + [(M[0], st(9.0, at(6, 14, 54)))], at(6, 14, 55), h3) +f, why, _g = solarfc.read_sum([("sensor.t", st(1.0, at(6, 14, 54)))], + [(M[0], st(9.0, at(6, 18, 20)))], + at(6, 18, 30), h3) +check("stale today member at 18:30: not kept, no forecast", + f is None and why == solarfc.STALE) +f, _w, _g = solarfc.read_sum([("sensor.t", st(1.0, at(6, 14, 54)))], + [(M[0], st(9.0, at(6, 20, 0)))], + at(6, 20, 30), h3) +check("... and at 20:30 it is kept, as before (the bell has ended)", + f is not None and f.kept) + +# After midnight: a tomorrow reading from before midnight is never tomorrow's. +h4 = solarfc.Hold() +run(h4, [(4.0, at(6, 14, 0)), (5.0, at(6, 14, 0)), (6.0, at(6, 14, 0))], + at(6, 14, 5)) +for label, now in (("00:30", at(7, 0, 30)), ("10:00", at(7, 10, 0)), + ("15:00", at(7, 15, 0))): + f, _w, _g = solarfc.read_sum( + [("sensor.t", st(2.0, now - timedelta(minutes=5)))], + [(e, st(k, at(6, 14, 0))) for e, k in zip(M, (4.0, 5.0, 6.0))], + now, h4) + check(f"{label} on the 7th: the 6th's tomorrow figure is not the " + "7th's tomorrow", f is not None and f.tomorrow is None) + +print() +if fails: + print(f"{len(fails)} FAILED:") + for f in fails: + print(" -", f) + sys.exit(1) +print("all checks passed") -- 2.54.0 From 0bacafadf04f814efbae7702d9b3fa3750a1121d Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Sat, 7 Nov 2026 06:02:00 +0100 Subject: [PATCH 12/22] OPT-38 round 2: a kept tomorrow reading is flagged kept only from 20:00 Before the bell ends it replaced today's Home line and the event log with the kept-for-tonight wording. stand_in() docstring covers the tomorrow case. Co-Authored-By: Claude --- goodwe_controller/app/solarfc.py | 10 ++++--- goodwe_controller/test_opt27.py | 2 +- goodwe_controller/test_opt34.py | 4 +-- goodwe_controller/test_opt38.py | 45 +++++++++++++++++++++++++++----- 4 files changed, 49 insertions(+), 12 deletions(-) diff --git a/goodwe_controller/app/solarfc.py b/goodwe_controller/app/solarfc.py index 3cda6dc..6a4939e 100644 --- a/goodwe_controller/app/solarfc.py +++ b/goodwe_controller/app/solarfc.py @@ -332,8 +332,9 @@ class Hold: The kept reading must be for `day0`, and either (a) at most `FRESH_S` old, or (b) used at night: from the end of the bell to local midnight, - at most `KEPT_MAX_AGE_S` old. `kept` is true when (b) was needed, - or `as_today` (OPT-34: a tomorrow figure standing in for today, after + at most `KEPT_MAX_AGE_S` old; a "tomorrow" reading at any hour + (OPT-38; `read_sum()` flags it kept only from the end of the bell). + `kept` is true when (b) was needed, or `as_today` (OPT-34: a tomorrow figure standing in for today, after midnight and until the bell starts, `KEPT_MORNING_MAX_AGE_S`).""" rec = self._r.get((kind, entity, day0)) if rec is None: @@ -558,6 +559,9 @@ def read_sum(today, tomorrow, now: datetime, hold: Hold | None = None): if why0 is not None: return None, why0, gaps tday = _sum(tdays) if tomorrow and len(tdays) == len(tomorrow) else None - kept = kept or (tday is not None and tkept) + # OPT-38: before the bell ends a kept tomorrow reading is no "kept for + # tonight": it must not replace today's line or the log. + kept = kept or (tday is not None and tkept + and _local_hour(now) >= BELL_TO_H) # The oldest member's report is when the sum was last fully known. return Forecast(_sum(days), tday, min(ats), kept), None, gaps diff --git a/goodwe_controller/test_opt27.py b/goodwe_controller/test_opt27.py index 1d0e7a8..7f3809d 100644 --- a/goodwe_controller/test_opt27.py +++ b/goodwe_controller/test_opt27.py @@ -146,7 +146,7 @@ for label, tm in (("stale", st(12.0, at(4, 9, 5))), ("unavailable", GONE)): check(f"14:00, today fresh, tomorrow {label} since 09:05 (over 3 h): " "the 09:05 figure covers it, kept (OPT-38: at any hour)", fd is not None and fd.tomorrow is not None - and fd.tomorrow.kwh == 12.0 and fd.kept) + and fd.tomorrow.kwh == 12.0 and not fd.kept) fn = rs(st(0.1, at(4, 20, 55)), tm, at(4, 21, 0), h9)[0] check(f"21:00, tomorrow {label}: the 09:05 figure covers it, kept", fn is not None and fn.tomorrow is not None diff --git a/goodwe_controller/test_opt34.py b/goodwe_controller/test_opt34.py index 6165010..fdb4698 100644 --- a/goodwe_controller/test_opt34.py +++ b/goodwe_controller/test_opt34.py @@ -186,9 +186,9 @@ check("a fresh figure replaces the kept one at once (04:00: 5.0 today, " and fy.today.kwh == 5.0 and fy.tomorrow.kwh == 9.0) fy2 = rs(st(5.0, at(4, 7, 55)), GONE, at(4, 8, 0), hd)[0] check("... and at 08:00, today fresh and tomorrow's sensor down for over 3 h: " - "tomorrow is the 03:50 figure for the 5th (9.0, kept: OPT-38), never " + "tomorrow is the 03:50 figure for the 5th (9.0, not flagged kept before 20:00: OPT-38), never " "the 3rd's 12.0", fy2 is not None and fy2.today.kwh == 5.0 - and fy2.tomorrow is not None and fy2.tomorrow.kwh == 9.0 and fy2.kept) + and fy2.tomorrow is not None and fy2.tomorrow.kwh == 9.0 and not fy2.kept) # Restart: the saved hold keeps the figure. for label, now in (("01:00", at(4, 1, 0)), ("04:00", at(4, 4, 0))): diff --git a/goodwe_controller/test_opt38.py b/goodwe_controller/test_opt38.py index d4ef310..11ce381 100644 --- a/goodwe_controller/test_opt38.py +++ b/goodwe_controller/test_opt38.py @@ -16,7 +16,7 @@ from datetime import datetime, timedelta, timezone HERE = os.path.dirname(os.path.abspath(__file__)) sys.path.insert(0, HERE) -from app import solarfc # noqa: E402 +from app import homeview, solarfc # noqa: E402 fails = [] @@ -55,15 +55,16 @@ run(h, [(4.0, at(6, 14, 0)), (5.0, at(6, 14, 54)), (6.0, at(6, 14, 0))], at(6, 14, 55)) fresh = lambda w: [(4.0, w), (5.0, at(6, 14, 54)), (6.0, w)] # noqa: E731 f, why, gaps = run(h, fresh(at(6, 18, 20)), at(6, 18, 30)) -check("18:30, one member last reported 14:54: tomorrow sum is there, kept", +check("18:30, one member last reported 14:54: tomorrow sum is there (not " + "flagged kept before 20:00)", f is not None and f.tomorrow is not None and f.tomorrow.kwh == 15.0 - and f.kept and gaps == ()) + and not f.kept and gaps == ()) f, _w, _g = run(h, fresh(at(6, 17, 40)), at(6, 17, 50)) check("17:50 (just under 3 h): fresh, not kept", f is not None and f.tomorrow.kwh == 15.0 and not f.kept) f, _w, _g = run(h, fresh(at(6, 17, 55)), at(6, 18, 0)) -check("18:00 (stale since 17:54): kept, tomorrow there", - f is not None and f.tomorrow is not None and f.kept) +check("18:00 (stale since 17:54): tomorrow there", + f is not None and f.tomorrow is not None and not f.kept) f, _w, _g = run(h, fresh(at(6, 19, 50)), at(6, 19, 55)) check("19:55 (before the bell ends): tomorrow there", f.tomorrow is not None) @@ -80,7 +81,7 @@ check("member 12 h 10 min old: no tomorrow sum (OPT-17 rule), today stays", f, _w, _g = run(h2, [(4.0, w), (5.0, at(6, 5, 0)), (6.0, w)], at(6, 16, 50)) check("... and 11 h 50 min old is still kept", - f is not None and f.tomorrow is not None and f.kept) + f is not None and f.tomorrow is not None and not f.kept) # Not a partial sum: a member never read has nothing to stand in. f, _w, gaps = run(solarfc.Hold(), [(4.0, at(6, 18, 0)), @@ -117,6 +118,38 @@ for label, now in (("00:30", at(7, 0, 30)), ("10:00", at(7, 10, 0)), check(f"{label} on the 7th: the 6th's tomorrow figure is not the " "7th's tomorrow", f is not None and f.tomorrow is None) +# Before 20:00 a kept tomorrow reading is not "kept for tonight": today's +# line stays the normal one (EN and NL) and the event log (main.py, from +# `fc.kept`) has no "kept". +h5 = solarfc.Hold() +solarfc.read_sum([("t", st(3.0, at(6, 9, 0)))], [("m", st(12.0, at(6, 9, 5)))], + at(6, 9, 6), h5) +for hh in (14, 18, 19): + f, _w, _g = solarfc.read_sum([("t", st(2.5, at(6, hh - 1, 55)))], + [("m", st(12.0, at(6, 9, 5)))], + at(6, hh, 0), h5) + check(f"{hh}:00, today fresh, tomorrow kept from 09:05: tomorrow there, " + "not flagged kept", f is not None and f.tomorrow is not None + and f.tomorrow.kwh == 12.0 and f.kept is False) +f, _w, _g = solarfc.read_sum([("t", st(2.5, at(6, 13, 55)))], + [("m", st(12.0, at(6, 9, 5)))], at(6, 14, 0), h5) +for lang in ("en", "nl"): + with homeview.in_lang(lang): + line = homeview.line_solar(f.today.kwh, f.today.kwh * 0.7, + homeview.hhmm(f.at), kept=f.kept) + kept_line = homeview.line_solar(1, 1, homeview.hhmm(f.at), kept=True) + print(f" {lang}: {line!r}") + if lang == "en": + check("... EN says 'Solar still to come today'", + line.startswith("Solar still to come today")) + check(f"14:00 Home line ({lang}) is the normal line, not the kept one", + line != kept_line and "2,5" in line.replace(".", ",") + and "13:55" in line) +f, _w, _g = solarfc.read_sum([("t", st(0.2, at(6, 20, 55)))], + [("m", st(12.0, at(6, 9, 5)))], at(6, 21, 0), h5) +check("21:00 as before: tomorrow kept from 09:05 is flagged kept", + f is not None and f.kept and f.tomorrow.kwh == 12.0) + print() if fails: print(f"{len(fails)} FAILED:") -- 2.54.0 From e56eeef8f0dc4dc1ace10d89aca2f5507b3b1f7a Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Sat, 7 Nov 2026 11:28:47 +0100 Subject: [PATCH 13/22] SMA-2: each inverter is reached through a driver; GoodWe behaviour unchanged app/drivers.py holds one Driver per roster entry (configured, actuator_id, limits, read, write, release, probe, link_stale_s, batt_bound, settle_s, brand, words). GoodWeEsphomeDriver makes the calls the code it replaced made, in the same order. Inverter gets driver; has_writer, is_live and link_note read it. The SoC/power read, the write, the device range (run_control, amain, the settings panel), rule 2 per unit and the SAFETY-28 census go through it. setpoint_entity is optional per entry beside driver, host, port, unit_id and maintenance; only goodwe_esphome is selectable, the rest are checked and unused. Co-Authored-By: Claude --- goodwe_controller/CHANGELOG.md | 1 + .../DECISIONS/SMA-2-driver-seam.md | 39 ++ goodwe_controller/app/drivers.py | 299 +++++++++ goodwe_controller/app/inverter.py | 71 +- goodwe_controller/app/main.py | 43 +- goodwe_controller/app/messages.py | 10 + goodwe_controller/app/selfconfig.py | 12 +- goodwe_controller/app/uilayout.py | 13 + goodwe_controller/app/validate.py | 76 ++- goodwe_controller/config.yaml | 21 +- goodwe_controller/test_freshness.py | 2 +- goodwe_controller/test_inverter.py | 16 +- goodwe_controller/test_sma2_driver.py | 626 ++++++++++++++++++ goodwe_controller/test_translations.py | 4 +- goodwe_controller/test_uilayout.py | 4 +- .../tools/supervisor_shape_probe.py | 36 +- goodwe_controller/translations/en.yaml | 15 + goodwe_controller/translations/nl.yaml | 15 + 18 files changed, 1238 insertions(+), 65 deletions(-) create mode 100644 goodwe_controller/DECISIONS/SMA-2-driver-seam.md create mode 100644 goodwe_controller/app/drivers.py create mode 100644 goodwe_controller/test_sma2_driver.py diff --git a/goodwe_controller/CHANGELOG.md b/goodwe_controller/CHANGELOG.md index 6ebdc4f..ef51a31 100644 --- a/goodwe_controller/CHANGELOG.md +++ b/goodwe_controller/CHANGELOG.md @@ -2,6 +2,7 @@ ### Not in a release yet +- Each inverter is now reached through a driver, so another make can be added later. Nothing changes for a GoodWe site, and the inverter list accepts five new optional settings that nothing reads yet ([[SMA-2]]). - Home now draws today's charge from the grid for every hour the add-on will really charge, not only the run under way ([[UX-77]]). - When Home Assistant passes readings on late, control no longer swings on them: it waits until the battery shows its last power request, and the peak limit no longer asks for full power on them. The page says so, and if the readings stay too late every inverter goes to 0 W ([[SAFETY-128]]). - With a 5 s meter, readings 30 s late are now noticed within 100 seconds, not 3 minutes ([[SAFETY-129]]). diff --git a/goodwe_controller/DECISIONS/SMA-2-driver-seam.md b/goodwe_controller/DECISIONS/SMA-2-driver-seam.md new file mode 100644 index 0000000..8981c42 --- /dev/null +++ b/goodwe_controller/DECISIONS/SMA-2-driver-seam.md @@ -0,0 +1,39 @@ +# SMA-2 - each inverter is reached through a driver + +**Status: for review.** Tests: `test_sma2_driver.py`. Source: SMA-9 report +section 1.1. A pure refactor: no behaviour change for GoodWe, no change to a +command, a log line or a sentence the add-on shows. + +Every place that read or steered an inverter used the GoodWe ESPHome entities +directly, so a second make meant touching all of them. `app/drivers.py` now +holds one `Driver` per roster entry (`Inverter.driver`): `configured()`, +`actuator_id()`, `limits()`, `read()`, `write()`, `release()`, `probe()`, and +`link_stale_s`, `batt_bound`, `settle_s`, `brand`, `words`. `GoodWeEsphomeDriver` +makes the same calls in the same order as the code it replaced; its +`release()` is `write(0)`. + +**Moved to the driver:** the SoC and power read (`_read_unit`), the write +(`_write_unit`, still under the SAFETY-28 gate, the clamp, shadow mode and the +write budget), the device range (`run_control`, `amain`, the settings panel), +liveness and its window (`has_writer`, `is_live`, `link_note`), the freshness +bound, the status rows' "is a source set" test, rule 2's per-unit range check +and SAFETY-28's census. **Not moved:** the clamp, the gate, the heartbeat and +`command_site_zero()`; stop is still 0 W for every unit, and nothing calls +`release()` yet (SMA-10). `settle_s` and the control constants are not read +from the driver yet either: with only GoodWe they would change nothing. + +**The seam is not TCP-shaped.** The census asks each entry's driver for its +actuator id: the setpoint entity for GoodWe, `host:port:unit_id` for a driver +reached over a socket, so two units behind one gateway are two actuators and +one unit named twice is the SAFETY-28 refusal. The driver binds its Home +Assistant on first use by the Controller, so a unit added later is wired too. + +**Schema.** `setpoint_entity` is optional per entry (rule 9(d) still requires +it for GoodWe, with the same sentence), beside `driver` (only `goodwe_esphome`), +`host`, `port`, `unit_id`, `maintenance`. The last four are checked for shape +and not read. The Supervisor shape probe was re-run (section K). + +**Checked against 23ff9d0:** replay digests and a log of every Home Assistant +call are byte-identical at N=1 and N=2; the TEL-07 announce diff is 0/0/0; a +fake driver beside a GoodWe unit allocates, grades, drops and publishes per +unit; a call site that goes round the seam is caught. diff --git a/goodwe_controller/app/drivers.py b/goodwe_controller/app/drivers.py new file mode 100644 index 0000000..98d616e --- /dev/null +++ b/goodwe_controller/app/drivers.py @@ -0,0 +1,299 @@ +"""The inverter driver seam (SMA-2): how this add-on reads and steers ONE inverter. + +Before this, every place that read or steered an inverter talked to the GoodWe +ESPHome entities itself (`hass.limits`, `hass.number`, `hass.set_number`, the +SAFETY-28 census, selfconfig, validate), so a second brand meant touching all of +them. Now each `Inverter` holds a `Driver`, and those places ask it. This card +is a pure refactor: `GoodWeEsphomeDriver` does exactly what the code it replaced +did, in the same order, with the same arguments. DECISIONS/SMA-2-driver-seam.md. + +THE SEAM IS NOT TCP-SHAPED. Nothing here assumes a connection, or one connection +per host: a later FoxESS unit is RTU framing over a TCP socket, and its +`actuator_id()` is host:port:unit_id, so two units behind one gateway are two +actuators and one unit named twice is the SAFETY-28 refusal. + +WHAT STAYS OUTSIDE THE DRIVER, on purpose. The gate (SAFETY-28), the clamp +(SAFETY-03), shadow mode and the write budget all sit above `write()` in +`Controller._write_unit`, and the freshness gate sits above `read()`; a driver +never sees a value those did not pass. A driver moves one number to or from one +inverter and says whether it worked. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field + +from .control import PLANT_SETTLE_S +from .freshness import BOUNDS + +# How long every write to one GoodWe inverter must have been REJECTED before +# this controller stops claiming it can command it. Layer 1's own window: the +# ESP32 commands 0 W after ~30 s without a refresh (`validate. +# FIRMWARE_INGEST_VETO_S`). `inverter.LINK_STALE_S` is this number, re-exported. +GOODWE_LINK_STALE_S = 30.0 + +# `driver` when an entry does not say: every install that exists today. +DEFAULT_DRIVER = "goodwe_esphome" + +# The `driver` values the options accept. Only the GoodWe one for now; the +# registry below may hold more (a test registers a fake), but this tuple is +# what validate.py rule 9 lets through and what config.yaml's schema lists. +SELECTABLE = (DEFAULT_DRIVER,) + + +class DriverNotBound(RuntimeError): + """A driver was asked to talk to the world before it was given a Home + Assistant. A wiring bug, so it is loud rather than a quiet "no answer".""" + + +@dataclass(frozen=True) +class Probe: + """What a read-only look at one inverter found. None means "did not answer".""" + + dev_min: float | None = None + dev_max: float | None = None + soc: float | None = None + batt_w: float | None = None + extra: dict = field(default_factory=dict) + + +def census_key(actuator_id: str) -> str: + """The form SAFETY-28 compares: stripped and lowercase, so a config cannot + name one actuator twice by changing its case (`Number.X` vs `number.x`).""" + return str(actuator_id or "").strip().lower() + + +class Driver: + """One inverter's I/O. Subclasses set the class attributes and the methods. + + Constructed once per roster entry, with no Home Assistant: `bind()` gives + it one (the Controller does, and so does the settings panel, which has no + Controller). Everything that talks to the world is a coroutine. + """ + + #: the `driver` option value that selects this class + name = "" + #: the option key that names the actuator in an entry (the census reads it) + writer_key = "" + #: the brand word the per-driver sentences and device names use + brand = "" + #: per-driver sentences (SMA-10 and later). Empty until a caller needs one. + words: dict = {} + #: how long all-rejected writes last before the unit stops being live + link_stale_s = GOODWE_LINK_STALE_S + #: this driver's `freshness.Bound` for the battery power reading + batt_bound = BOUNDS["batt"] + #: the plant's settling time after a step, in s (control.PLANT_SETTLE_S) + settle_s = PLANT_SETTLE_S + + # -- from the options ---------------------------------------------------- + @classmethod + def from_entry(cls, entry: dict) -> "Driver": + raise NotImplementedError + + @classmethod + def actuator_id_of(cls, entry: dict) -> str: + """The actuator an options entry names, or "" for none. Pure: the + SAFETY-28 census runs on options, before any driver exists.""" + raise NotImplementedError + + # -- wiring -------------------------------------------------------------- + def bind(self, hass) -> "Driver": + """Give this driver its Home Assistant (or whatever it talks through).""" + return self + + def bind_getter(self, getter) -> "Driver": + """Like `bind()`, but looked up at each use: the Controller passes + `lambda: self.hass`, so replacing `controller.hass` is seen at once.""" + return self + + def bound(self) -> bool: + """Has it been given what it talks through? A driver that talks + through nothing the Controller owns is always bound.""" + return True + + # -- what it is ---------------------------------------------------------- + def configured(self) -> bool: + """Is there any way to reach this inverter at all?""" + raise NotImplementedError + + def actuator_id(self) -> str: + """What this driver writes to. Two units with one id are one actuator.""" + raise NotImplementedError + + def soc_source(self) -> str: + """Where the pack level is read from; "" when nowhere. Status rows only.""" + return "" + + def batt_source(self) -> str: + """Where the battery power is read from; "" when nowhere.""" + return "" + + # -- what it does -------------------------------------------------------- + async def limits(self) -> tuple: + """`(dev_min, dev_max)` in W the device accepts; (None, None) unread.""" + raise NotImplementedError + + async def read(self) -> tuple: + """`(soc, batt_w, extra)`. A value that did not arrive is None. `batt_w` + is the RAW power, in the sign the controller uses; the freshness gate + is the caller's.""" + raise NotImplementedError + + async def write(self, w: float) -> bool: + """Put `w` on the inverter. True only when it took it.""" + raise NotImplementedError + + async def release(self) -> bool: + """Stop steering this inverter. For a driver with nothing better, 0 W.""" + return await self.write(0.0) + + async def probe(self) -> Probe: + """A read-only look, usable without a Controller (panel, wizard).""" + raise NotImplementedError + + +class GoodWeEsphomeDriver(Driver): + """The GoodWe behind the ESP32: three Home Assistant entities. + + A transcription of what `main.py` and `selfconfig.py` did inline: `limits()` + is `hass.limits(setpoint)`, `read()` is `hass.number(soc)` then + `hass.number(batt, invert)`, `write()` is `hass.set_number(setpoint, w)`. + `release()` is `write(0)`: the board's own watchdog is what hands back. + """ + + name = DEFAULT_DRIVER + writer_key = "setpoint_entity" + brand = "GoodWe" + + def __init__(self, setpoint_entity="", soc_entity="", batt_entity="", + batt_invert=False): + self.setpoint_entity = str(setpoint_entity or "") + self.soc_entity = str(soc_entity or "") + self.batt_entity = str(batt_entity or "") + self.batt_invert = bool(batt_invert) + self._hass = None + self._getter = None + + @classmethod + def from_entry(cls, entry: dict) -> "GoodWeEsphomeDriver": + return cls(setpoint_entity=entry.get("setpoint_entity", ""), + soc_entity=entry.get("soc_entity", ""), + batt_entity=entry.get("batt_entity", ""), + batt_invert=bool(entry.get("batt_invert"))) + + @classmethod + def actuator_id_of(cls, entry: dict) -> str: + return str(entry.get("setpoint_entity") or "").strip() + + def bind(self, hass) -> "GoodWeEsphomeDriver": + self._hass, self._getter = hass, None + return self + + def bind_getter(self, getter) -> "GoodWeEsphomeDriver": + self._hass, self._getter = None, getter + return self + + def bound(self) -> bool: + return self._hass is not None or self._getter is not None + + def _h(self): + hass = self._getter() if self._getter is not None else self._hass + if hass is None: + raise DriverNotBound(f"{self.brand} driver for " + f"{self.setpoint_entity!r} has no Home Assistant") + return hass + + def configured(self) -> bool: + return bool(self.setpoint_entity) + + def actuator_id(self) -> str: + return self.setpoint_entity + + def soc_source(self) -> str: + return self.soc_entity + + def batt_source(self) -> str: + return self.batt_entity + + async def limits(self) -> tuple: + dev_min, dev_max, _step = await self._h().limits(self.setpoint_entity) + return dev_min, dev_max + + async def read(self) -> tuple: + hass = self._h() + soc = await hass.number(self.soc_entity) + raw = await hass.number(self.batt_entity, self.batt_invert) + return soc, raw, {} + + async def write(self, w: float) -> bool: + return await self._h().set_number(self.setpoint_entity, w) + + async def probe(self) -> Probe: + dev_min, dev_max = await self.limits() + soc, raw, extra = await self.read() + return Probe(dev_min, dev_max, soc, raw, extra) + + +class UnavailableDriver(Driver): + """What an entry naming a driver this build does not have gets. Validation + refuses such an entry before a Controller exists; this is only what + `build_roster()` (which refuses nothing) builds meanwhile: a unit that is + not configured, so the refusal branch names it and commands 0 W.""" + + name = "" + # The census still counts a `setpoint_entity` on such an entry: refusing + # too much is the safe direction for SAFETY-28. + writer_key = "setpoint_entity" + brand = "" + + def __init__(self, wanted=""): + self.wanted = str(wanted or "") + + @classmethod + def from_entry(cls, entry: dict) -> "UnavailableDriver": + return cls(entry.get("driver")) + + @classmethod + def actuator_id_of(cls, entry: dict) -> str: + return str(entry.get("setpoint_entity") or "").strip() + + def configured(self) -> bool: + return False + + def actuator_id(self) -> str: + return "" + + async def limits(self) -> tuple: + return None, None + + async def read(self) -> tuple: + return None, None, {} + + async def write(self, w: float) -> bool: + return False + + async def probe(self) -> Probe: + return Probe() + + +# name -> class. The options' `driver` value picks one. A test may add a fake; +# `SELECTABLE` (not this) is what the options accept. +REGISTRY = {GoodWeEsphomeDriver.name: GoodWeEsphomeDriver} + + +def driver_name(entry) -> str: + """The `driver` an options entry selects; blank or absent is the default.""" + raw = entry.get("driver") if isinstance(entry, dict) else None + return str(raw).strip() if raw not in (None, "") else DEFAULT_DRIVER + + +def driver_class(entry): + """The class an options entry selects, `UnavailableDriver` if none.""" + return REGISTRY.get(driver_name(entry), UnavailableDriver) + + +def build_driver(entry) -> Driver: + """The driver for ONE options entry (or the flat options, for one inverter).""" + entry = entry if isinstance(entry, dict) else {} + return driver_class(entry).from_entry(entry) diff --git a/goodwe_controller/app/inverter.py b/goodwe_controller/app/inverter.py index 1816811..64a7003 100644 --- a/goodwe_controller/app/inverter.py +++ b/goodwe_controller/app/inverter.py @@ -124,9 +124,12 @@ from datetime import datetime from .clamp import ClampResult from .delivery import DeliveryConfig, DeliveryWatchdog from . import messages +# SMA-2. The driver seam: how this unit's inverter is read and steered. +from .drivers import (GOODWE_LINK_STALE_S, GoodWeEsphomeDriver, + build_driver) # SAFETY-37. The freshness policy, and the ONE place its bounds are written. # freshness.py imports nothing from this package, so the direction is one-way. -from .freshness import BOUNDS, Freshness +from .freshness import Freshness from .idle import IdleReading # CFG-03's ONE in-code mirror of config.yaml's SAFETY-24 defaults. Imported # rather than retyped: the flat-keys path below needs `bms_soc_floor`'s @@ -149,7 +152,11 @@ from .watchdog import SignWatchdog, SignWatchdogConfig # decided it will not. No config key: CFG-03 tracks three places a default has # to be kept in step and this does not earn a fourth - it is derived from a # constant the firmware and validate.py already share. -LINK_STALE_S = 30.0 +# +# SMA-2. This is the GoodWe driver's `link_stale_s` (drivers.py), kept under its +# old name for the checks and for main.py. `is_live()` and `link_note()` now ask +# each unit's DRIVER for its window; a driver for another make states its own. +LINK_STALE_S = GOODWE_LINK_STALE_S # The name of the inverter a SINGLE-INVERTER config describes. ⚠️ A NAME, NOT A # CONFIGURATION - it exists so that a per-inverter claim has an actuator to be @@ -195,12 +202,14 @@ class Inverter: def __init__(self, name, *, setpoint_entity="", soc_entity="", batt_entity="", batt_invert=False, max_w=2000.0, bms_soc_floor=None, capacity_kwh=None, store=None, - sign_cfg=None, delivery_cfg=None): + sign_cfg=None, delivery_cfg=None, driver=None): self.name = str(name) - self.setpoint_entity = str(setpoint_entity or "") - self.soc_entity = str(soc_entity or "") - self.batt_entity = str(batt_entity or "") - self.batt_invert = bool(batt_invert) + # SMA-2. HOW THIS UNIT IS READ AND STEERED. The entity keywords above + # are the GoodWe driver's, kept so every existing caller builds the + # same unit; a roster entry that selects another driver passes + # `driver` and the entity keywords stay empty. + self.driver = driver if driver is not None else GoodWeEsphomeDriver( + setpoint_entity, soc_entity, batt_entity, batt_invert) # The supervised cap on what may be written to THIS setpoint entity. # See the module docstring on why this is a number and not a `Tuning`. self.max_w = float(max_w) @@ -256,7 +265,7 @@ class Inverter: # constructed here: SoC's obligation term is energy through the pack, not # command travel, and the constant it needs has not been measured. That # is TEL-08's, and freshness.py's docstring says exactly what it owes. - self.batt_fresh = Freshness(BOUNDS["batt"]) + self.batt_fresh = Freshness(self.driver.batt_bound) # --- what this unit was last told, and what it last accepted --------- # `target` is THIS unit's share of the site target from the last @@ -338,6 +347,25 @@ class Inverter: def idle_w(self) -> float: return self.idle.w + # -- the GoodWe driver's entities, read through for the callers that name + # them (the panel's range check, the status rows, the checks). A driver + # without such an entity answers "" - the same as "not set". + @property + def setpoint_entity(self) -> str: + return getattr(self.driver, "setpoint_entity", "") + + @property + def soc_entity(self) -> str: + return getattr(self.driver, "soc_entity", "") + + @property + def batt_entity(self) -> str: + return getattr(self.driver, "batt_entity", "") + + @property + def batt_invert(self) -> bool: + return bool(getattr(self.driver, "batt_invert", False)) + # -- store --------------------------------------------------------------- def key(self, name: str) -> str: """This unit's namespaced Store key for `name`. See store.unit_key().""" @@ -387,7 +415,7 @@ class Inverter: return out def has_writer(self) -> bool: - return bool(self.setpoint_entity) + return self.driver.configured() # -- liveness ------------------------------------------------------------ def note_write(self, ok: bool, now: datetime) -> None: @@ -398,7 +426,7 @@ class Inverter: elif self.rejected_since is None: self.rejected_since = now - def is_live(self, now: datetime, stale_s: float = LINK_STALE_S): + def is_live(self, now: datetime, stale_s: float | None = None): """`(live, why_not)` - may this controller command this unit THIS TICK? Read the module docstring before widening this - in particular on why a @@ -411,7 +439,11 @@ class Inverter: ⚠️ ANSWERED PER TICK, NEVER CACHED, for `allocator.plan()`'s reason: a link goes stale between one cycle and the next. + + `stale_s` is the driver's `link_stale_s` unless a caller passes one. """ + if stale_s is None: + stale_s = self.driver.link_stale_s if self.rejected_since is not None: held = (now - self.rejected_since).total_seconds() if held >= float(stale_s): @@ -451,9 +483,11 @@ class Inverter: held = (0.0 if self.rejected_since is None else (now - self.rejected_since).total_seconds()) if not live: - return messages.row_not_controllable(self.name, held, LINK_STALE_S) + return messages.row_not_controllable(self.name, held, + self.driver.link_stale_s) if self.rejected_since is not None: - return messages.row_commands_refused(self.name, held, LINK_STALE_S) + return messages.row_commands_refused(self.name, held, + self.driver.link_stale_s) return "" def __repr__(self) -> str: @@ -556,10 +590,8 @@ def build_roster(opts: dict, store, sign_cfg=None, # identically, which is AC 3. return [Inverter( UNIT_A, - setpoint_entity=opts.get("setpoint_entity", ""), - soc_entity=opts.get("soc_entity", ""), - batt_entity=opts.get("batt_entity", ""), - batt_invert=bool(opts.get("batt_invert")), + # SMA-2. The flat keys are the GoodWe shape and only that. + driver=GoodWeEsphomeDriver.from_entry(opts), max_w=_flat_max_w(opts), # SAFETY-32. THE FLAT KEY, because this IS the single-inverter shape # - the one pack the flat options describe is the one pack whose @@ -588,10 +620,9 @@ def build_roster(opts: dict, store, sign_cfg=None, entry = entry if isinstance(entry, dict) else {} roster.append(Inverter( name, - setpoint_entity=entry.get("setpoint_entity", ""), - soc_entity=entry.get("soc_entity", ""), - batt_entity=entry.get("batt_entity", ""), - batt_invert=bool(entry.get("batt_invert")), + # SMA-2. The entry's `driver` (default goodwe_esphome) picks the + # class; the entry itself says where its entities are. + driver=build_driver(entry), # ⚠️ NOT `opts["max_w"]`. See the docstring: a rating an entry did # not state is not the flat one, it is a rating rule 9 refuses. max_w=_entry_max_w(entry), diff --git a/goodwe_controller/app/main.py b/goodwe_controller/app/main.py index 0fc87fb..eb14dd7 100644 --- a/goodwe_controller/app/main.py +++ b/goodwe_controller/app/main.py @@ -71,6 +71,7 @@ from . import messages from .freeze import ( FreezeWatchdog, FreezeWatchdogConfig, looks_frozen, rail_unreachable, rail_w, ) +from .drivers import GoodWeEsphomeDriver from .hass import CORE_API, OUTAGE, HomeAssistant # ⚠️ `LINK_STALE_S` IS RE-EXPORTED DELIBERATELY. `live_units()` reasons about it # and two checks read `app.main.LINK_STALE_S` to build a stale link, so the @@ -3774,6 +3775,18 @@ class Controller: elif self.inputs_bad_since is None: self.inputs_bad_since = datetime.now(timezone.utc) + def _driver(self, inv): + """`inv`'s driver, wired to this Controller's Home Assistant on first use. + + SMA-2. Bound HERE rather than when the roster is built, so a unit added + to `self.inverters` afterwards is wired too, and the driver follows + whatever `self.hass` is at the time (a check may swap it). + """ + drv = inv.driver + if not drv.bound(): + drv.bind_getter(lambda: self.hass) + return drv + async def _read_unit(self, inv, now_mono: float) -> None: """One inverter's own SoC and battery power, from ITS OWN entities. @@ -3793,8 +3806,10 @@ class Controller: second spelling of "what is standing" would let the two disagree about the number they are both reasoning about. """ - inv.soc = await self.hass.number(inv.soc_entity) - raw = await self.hass.number(inv.batt_entity, inv.batt_invert) + # SMA-2. The driver reads, in the order this method always did: SoC, + # then power. `extra` is whatever else the driver learned; nothing + # consumes it yet. + inv.soc, raw, _extra = await self._driver(inv).read() was_stale = inv.batt_fresh.stale was_releases = inv.batt_fresh.releases inv.batt_fresh.submit(raw, inv.standing_w(), now_mono) @@ -4029,9 +4044,9 @@ class Controller: for inv in targets)) async def _write_unit(self, inv, value: float, *, force: bool = False) -> None: - entity = inv.setpoint_entity - if not entity: + if not inv.driver.configured(): return + entity = inv.driver.actuator_id() # for the log line below # --- SAFETY-03: THE CLAMP, AND IT IS ON THE ONLY PATH TO THE WIRE ----- # Every write in this add-on comes through here - the cycle, the @@ -4154,7 +4169,7 @@ class Controller: # tick by itself: `last_write` is updated only on an ACCEPTED write, so # `changed` is still true. try: - ok = await asyncio.wait_for(self.hass.set_number(entity, value), + ok = await asyncio.wait_for(self._driver(inv).write(value), timeout=WRITE_BUDGET_S) except (asyncio.TimeoutError, TimeoutError): ok = False @@ -6730,9 +6745,8 @@ class Controller: # Concurrently, for `read_inputs()`'s reason: a slow ESPHome on one unit # must not delay the controller's start on every other. async def _limits(inv): - if inv.setpoint_entity: - inv.dev_min, inv.dev_max, _ = await self.hass.limits( - inv.setpoint_entity) + if inv.driver.configured(): + inv.dev_min, inv.dev_max = await self._driver(inv).limits() await asyncio.gather(*(_limits(inv) for inv in self.inverters)) entity = self.o.get("setpoint_entity", "") # ⚠️ THIS USED TO WARN AND CARRY ON, and that was the bug CFG-01 @@ -7316,9 +7330,10 @@ class Controller: rows = [] for inv in self.inverters: unit = None if len(self.inverters) == 1 else inv.name - rows.append((messages.LABEL_SOC, unit, inv.soc, inv.soc_entity)) + rows.append((messages.LABEL_SOC, unit, inv.soc, + inv.driver.soc_source())) rows.append((messages.LABEL_BATTERY, unit, inv.batt, - inv.batt_entity)) + inv.driver.batt_source())) if not self.p1_enabled: # In P1 mode the check above replaces this one; leaving both in # would report "no entity configured" for a meter_entity that is @@ -7360,9 +7375,8 @@ class Controller: "text": messages.row_sensor(label, value, unit)}) for inv in self.inverters: - sp = inv.setpoint_entity unit = None if len(self.inverters) == 1 else inv.name - if not sp: + if not inv.driver.configured(): out.append({"ok": False, "warn": False, "id": "control", "text": messages.row_control_unset(unit)}) elif inv.dev_max is None: @@ -8205,10 +8219,13 @@ async def amain() -> None: # validator cannot do for itself; (None, None) when ESPHome has not come # up yet is not a refusal - run_control() re-checks once the real rating # is known. See validate.py for where the line is drawn and why. + # SMA-2. The flat options are the one-inverter shape, so it is the + # GoodWe driver's, asked without a Controller (none exists yet). _sp = str(opts.get("setpoint_entity", "") or "").strip() + _drv = GoodWeEsphomeDriver(setpoint_entity=_sp).bind(hass) _dev_min = _dev_max = None if _sp: - _dev_min, _dev_max, _ = await hass.limits(_sp) + _dev_min, _dev_max = await _drv.limits() _LOG.info("setpoint device %s reports range %s…%s W", _sp, _dev_min, _dev_max) _problems = validate_startup_config(opts, _dev_min, _dev_max) # UX-15: a new install names no inverter. When that is the ONLY diff --git a/goodwe_controller/app/messages.py b/goodwe_controller/app/messages.py index 831da93..d6d1bbe 100644 --- a/goodwe_controller/app/messages.py +++ b/goodwe_controller/app/messages.py @@ -1683,6 +1683,11 @@ LABELS = { "capacity_kwh": "Usable battery capacity", "inverters": "Inverter list", "name": "Name", + "driver": "Inverter type", + "host": "Inverter address", + "port": "Inverter port", + "unit_id": "Inverter unit number", + "maintenance": "Battery maintenance", "meter_source": "Where the meter reading comes from", "meter_phases": "Number of phases", "p1_host": "HomeWizard P1 address", @@ -1764,6 +1769,11 @@ LABELS_NL = { "capacity_kwh": "Bruikbare capaciteit", "inverters": "Lijst met omvormers", "name": "Naam", + "driver": "Type omvormer", + "host": "Adres van de omvormer", + "port": "Poort van de omvormer", + "unit_id": "Eenheidsnummer van de omvormer", + "maintenance": "Onderhoud van de batterij", "meter_source": "Waar de meterstand vandaan komt", "meter_phases": "Aantal fasen", "p1_host": "Adres van de HomeWizard P1", diff --git a/goodwe_controller/app/selfconfig.py b/goodwe_controller/app/selfconfig.py index 5458253..d5b0498 100644 --- a/goodwe_controller/app/selfconfig.py +++ b/goodwe_controller/app/selfconfig.py @@ -343,10 +343,12 @@ async def problems_and_ranges(opts: dict, hass=None, notes=None): f"disagreement between the two and should be reported."] return found, [] if hass is not None: + # SMA-2. Each unit's driver, bound to the panel's own (read-only) + # Home Assistant: there is no Controller here. async def _limits(unit): - if unit.setpoint_entity: - unit.dev_min, unit.dev_max, _ = await hass.limits( - unit.setpoint_entity) + if unit.driver.configured(): + unit.dev_min, unit.dev_max = await unit.driver.bind( + hass).limits() await asyncio.gather(*(_limits(unit) for unit in units)) dev_min = dev_max = None @@ -357,8 +359,8 @@ async def problems_and_ranges(opts: dict, hass=None, notes=None): problems = list(validate_startup_config(opts, dev_min, dev_max, notes=notes)) problems += validate_unit_ranges(units, skip_entity=flat) - unread = [unit.setpoint_entity for unit in units - if unit.setpoint_entity and unit.dev_min is None + unread = [unit.driver.actuator_id() for unit in units + if unit.driver.configured() and unit.dev_min is None and unit.dev_max is None] return problems, unread diff --git a/goodwe_controller/app/uilayout.py b/goodwe_controller/app/uilayout.py index 421dac7..d19bf9f 100644 --- a/goodwe_controller/app/uilayout.py +++ b/goodwe_controller/app/uilayout.py @@ -302,6 +302,19 @@ DERIVED = { HIDDEN: dict = {"price_service_token": "a secret, set on the Configuration tab"} +# SMA-2. Per-entry keys of the inverter list the page never shows. They pick and +# address an inverter driver, only `goodwe_esphome` is selectable, and nothing +# reads the other four yet. An entry's keys the page does not draw travel +# through a save as stored (uimodel._roster_decode keeps the whole entry). +ROSTER_HIDDEN: dict = { + "driver": "only goodwe_esphome for now; empty means that", + "host": "not read yet", + "port": "not read yet", + "unit_id": "not read yet", + "maintenance": "not read yet", +} + + def fields(): """Every field once, in page order, roster fields after `inverters`.""" for group in GROUPS: diff --git a/goodwe_controller/app/validate.py b/goodwe_controller/app/validate.py index 659bc05..d9e1371 100644 --- a/goodwe_controller/app/validate.py +++ b/goodwe_controller/app/validate.py @@ -334,6 +334,7 @@ from typing import NamedTuple from . import dayplan, messages, pv, solarfc from .arbitrage import enabled as arbitrage_on from .control import SLEW_RAIL_FRAC, LOOP_TICK_S, PLANT_SETTLE_S +from . import drivers from .custom_formula import ( EXPORT_FORMULA_OPTION, FormulaInvalid, implausible_all_in, parse_export_formula, parse_price_formula, stale_card) @@ -457,12 +458,18 @@ def _collect_writers(node, path: str, out: list) -> None: however deeply nested and under whatever container. """ if isinstance(node, dict): + # SMA-2. THE DRIVER NAMES THE ACTUATOR. An entry's `driver` (default + # goodwe_esphome) says which key declares its writer - `setpoint_entity` + # for GoodWe, `host` for a driver reached over a socket - and what that + # entry's `actuator_id()` is. The walk is as structural as before: it + # still finds an entry wherever the options keep it. + cls = drivers.driver_class(node) for k, v in node.items(): here = f"{path}.{k}" if path else str(k) - if k in ACTUATOR_WRITE_KEYS: - entity = str(v or "").strip() - if entity: - out.append((here, entity)) + if k == cls.writer_key: + actuator = cls.actuator_id_of(node) + if actuator: + out.append((here, actuator)) continue _collect_writers(v, here, out) elif isinstance(node, (list, tuple)): @@ -498,7 +505,7 @@ def actuator_writers(opts: dict) -> dict: _collect_writers(opts, "", found) census: dict = {} for where, entity in found: - census.setdefault(entity.lower(), []).append(where) + census.setdefault(drivers.census_key(entity), []).append(where) return census @@ -729,11 +736,16 @@ def _roster_problems(opts: dict) -> list[str]: else: seen[name] = where who = f"{where} ({name})" if name else where - if not str(entry.get("setpoint_entity") or "").strip(): + # SMA-2. WHETHER THE ENTRY REACHES ITS INVERTER IS THE DRIVER'S TO SAY. + # For goodwe_esphome that is a setpoint entity, and the sentence is the + # one it always was. An unknown driver is refused by (h) below. + _cls = drivers.driver_class(entry) + if _cls is not drivers.UnavailableDriver and not _cls.actuator_id_of(entry): problems.append( f"{who} has no {S('setpoint_entity')}, so the add-on cannot " "control it. Set its ESPHome number entity, or remove the " "entry.") + problems += _driver_problems(entry, who) if _number(entry, "max_w", None) is None or (_number(entry, "max_w", None) or 0) <= 0: problems.append( f"{who} has no usable {S('max_w')}: it is " @@ -775,6 +787,47 @@ def _roster_problems(opts: dict) -> list[str]: return problems +def _whole(raw, lo: int, hi: int): + """`raw` as an int in [lo, hi], or None. A bool or a fraction is not one.""" + if isinstance(raw, bool): + return None + try: + val = float(raw) + except (TypeError, ValueError): + return None + return int(val) if val == int(val) and lo <= val <= hi else None + + +def _driver_problems(entry: dict, who: str) -> list[str]: + """Rule 9(h) (SMA-2): the keys that choose and address an inverter driver. + + Only `goodwe_esphome` is selectable for now. `host`, `port`, `unit_id` and + `maintenance` are accepted and checked for shape, and nothing reads them + yet: a later driver's entry must not be refused for carrying them. + """ + out: list[str] = [] + raw = entry.get("driver") + if raw not in (None, "") and str(raw).strip() not in drivers.SELECTABLE: + out.append( + f"{who} has {S('driver')} {messages.shown(raw)}. Use " + f"{', '.join(drivers.SELECTABLE)}, or leave it empty.") + host = entry.get("host") + if host not in (None, "") and (not isinstance(host, str) + or re.search(r"\s", host.strip())): + out.append(f"{who} has {S('host')} {messages.shown(host)}, which is " + "not an address. Use a name or an IP address.") + for key, lo, hi in (("port", 1, 65535), ("unit_id", 0, 247)): + val = entry.get(key) + if val not in (None, "") and _whole(val, lo, hi) is None: + out.append(f"{who} has {S(key)} {messages.shown(val)}. Use a " + f"whole number from {lo} to {hi}.") + maint = entry.get("maintenance") + if maint is not None and not isinstance(maint, bool): + out.append(f"{who} has {S('maintenance')} {messages.shown(maint)}. " + "Use on or off.") + return out + + def validate_unit_ranges(units, *, skip_entity: str = "") -> list[str]: """Rule 2, per inverter, against each unit's OWN reported device range. @@ -787,8 +840,8 @@ def validate_unit_ranges(units, *, skip_entity: str = "") -> list[str]: 2's own failure mode restored by a config shape: the operator's stated ceiling is not the ceiling and nothing says which won. - `units` is the roster (anything with `.name`, `.setpoint_entity`, `.max_w`, - `.dev_min`, `.dev_max`). A unit whose device did not answer is SKIPPED, not + `units` is the roster (anything with `.name`, `.driver` or `.setpoint_entity`, + `.max_w`, `.dev_min`, `.dev_max`). A unit whose device did not answer is SKIPPED, not guessed at, for the reason in this module's docstring: ESPHome is often not up when the add-on boots and (None, None) means "not read", not "zero". @@ -800,7 +853,12 @@ def validate_unit_ranges(units, *, skip_entity: str = "") -> list[str]: problems: list[str] = [] skip = str(skip_entity or "").strip().lower() for unit in units: - entity = str(getattr(unit, "setpoint_entity", "") or "").strip() + # SMA-2. THE DRIVER'S ACTUATOR, not a GoodWe key: a unit reached some + # other way is graded too. A bare object with only `.setpoint_entity` + # (the checks pass some) is read as it always was. + drv = getattr(unit, "driver", None) + entity = str((drv.actuator_id() if drv is not None + else getattr(unit, "setpoint_entity", "")) or "").strip() if not entity or entity.lower() == skip: continue cap = float(getattr(unit, "max_w", 0.0) or 0.0) diff --git a/goodwe_controller/config.yaml b/goodwe_controller/config.yaml index b02e072..1754991 100644 --- a/goodwe_controller/config.yaml +++ b/goodwe_controller/config.yaml @@ -942,11 +942,17 @@ schema: # batteries, whereas an absent capacity has an honest answer - no opinion # about the split, every unit weighs 1.0, which is what shipped. # - # ⚠️ FOUR KEYS ARE REQUIRED PER ENTRY AND THAT IS DELIBERATE. An entry with + # SMA-2. `driver` empty means goodwe_esphome, the only choice for now. + # `host`, `port`, `unit_id` and `maintenance` are accepted and checked for + # shape; nothing reads them yet (DECISIONS/SMA-2-driver-seam.md). + # + # ⚠️ THREE KEYS ARE REQUIRED PER ENTRY AND THAT IS DELIBERATE. An entry with # no `name` has no Store namespace and would share another unit's SAFETY-02 - # latch; one with no `setpoint_entity` is a unit nothing can command; one with - # no `max_w` would have to borrow a rating, which is the sibling-grading - # defect DEV-02 removed; `bms_soc_floor` is SAFETY-32's (options block above). + # latch; one with no `max_w` would have to borrow a rating, which is the + # sibling-grading defect DEV-02 removed; `bms_soc_floor` is SAFETY-32's + # (options block above). `setpoint_entity` WAS A FOURTH; since SMA-2 it is the + # GoodWe driver's own key, so a unit nothing can command is refused by + # validate.py rule 9(d) per driver rather than by this schema. # The optional keys are the sensors, whose absence honestly means "this unit # reports none", and `capacity_kwh`. # @@ -958,13 +964,18 @@ schema: # test_inverter.py part 12 asserts the ordering is still hostile. inverters: - name: str - setpoint_entity: str + setpoint_entity: str? max_w: int(100,5000) bms_soc_floor: int(0,50) soc_entity: str? batt_entity: str? batt_invert: bool? capacity_kwh: float(0.1,200)? + driver: list(goodwe_esphome)? + host: str? + port: port? + unit_id: int(0,247)? + maintenance: bool? estop_fitted: bool battery_modes_enabled: bool diff --git a/goodwe_controller/test_freshness.py b/goodwe_controller/test_freshness.py index 620da86..ab3e301 100644 --- a/goodwe_controller/test_freshness.py +++ b/goodwe_controller/test_freshness.py @@ -790,7 +790,7 @@ print(f" BOUNDS['soc'] subscripted in CODE by: {_uses_soc or 'nothing'}") check("⚠️ `wired=True` MEANS SOMETHING CONSUMES IT. A policy table nobody " "checks is a comment; this is the row `batt` claims and the module that " "claims it", - BOUNDS["batt"].wired is True and _uses_batt == ["inverter.py"]) + BOUNDS["batt"].wired is True and _uses_batt == ["drivers.py"]) check("⚠️ AND `wired=False` MEANS NOTHING DOES. SoC is TEL-08's and it is " "STATED here rather than wired, because its obligation term is energy " "through the pack and the constant that needs has never been measured on " diff --git a/goodwe_controller/test_inverter.py b/goodwe_controller/test_inverter.py index 4b7281e..1fe17cb 100644 --- a/goodwe_controller/test_inverter.py +++ b/goodwe_controller/test_inverter.py @@ -1536,8 +1536,9 @@ _decl9 = dict(ln.strip().lstrip("- ").split(": ", 1) for ln in _entry9.splitlines()[1:] if ": " in ln) print(f" the shipped per-entry schema: {_decl9}") check("...and the schema half declares it as a LIST OF OBJECTS whose `name`, " - "`setpoint_entity`, `max_w` and `bms_soc_floor` are REQUIRED - no " - "trailing `?` - so the Supervisor refuses a half-written entry before the " + "`max_w` and `bms_soc_floor` are REQUIRED - no " + "trailing `?` (SMA-2: `setpoint_entity` is the GoodWe driver's own key " + "and optional here; rule 9(d) requires it per driver) - so the Supervisor refuses a half-written entry before the " "add-on sees it, and an entry can never fall back on a sibling's rating " "or on a sibling's BMS cut-out because there is no entry without one. " "⚠️ SAFETY-32 ADDED THE FOURTH ONE AS REQUIRED RATHER THAN AS " @@ -1553,13 +1554,14 @@ check("...and the schema half declares it as a LIST OF OBJECTS whose `name`, " "refuses every options blob written before it existed", set(_decl9) == {"name", "setpoint_entity", "max_w", "bms_soc_floor", "soc_entity", "batt_entity", "batt_invert", - "capacity_kwh"} + "capacity_kwh", "driver", "host", "port", "unit_id", + "maintenance"} and not any(_decl9[k].endswith("?") - for k in ("name", "setpoint_entity", "max_w", - "bms_soc_floor")) + for k in ("name", "max_w", "bms_soc_floor")) and all(_decl9[k].endswith("?") - for k in ("soc_entity", "batt_entity", "batt_invert", - "capacity_kwh"))) + for k in ("setpoint_entity", "soc_entity", "batt_entity", + "batt_invert", "capacity_kwh", "driver", "host", + "port", "unit_id", "maintenance"))) # ⚠️ NOTHING IS INHERITED, AND THIS IS DEV-02 RESTATED AS A CONFIG RULE. # ⚠️ THE FLAT KEYS ARE SET TO REAL ENTITIES HERE, AND THAT IS THE WHOLE CHECK. diff --git a/goodwe_controller/test_sma2_driver.py b/goodwe_controller/test_sma2_driver.py new file mode 100644 index 0000000..7629308 --- /dev/null +++ b/goodwe_controller/test_sma2_driver.py @@ -0,0 +1,626 @@ +"""SMA-2: the inverter driver seam. `python3 test_sma2_driver.py` + +Part 1 grades the GoodWe driver against a Home Assistant that logs every call: +it must make exactly the calls the code it replaced made, in the same order. +Part 2 grades what `Inverter` reads from its driver. Part 3 drives a REAL +Controller with a roster of one fake-driver unit and one GoodWe unit and +checks that it writes, reads, limits, grades, drops and publishes each by its +own driver, with a Home Assistant that records every entity it is asked about +so a call site that goes round the seam is seen. Part 4 is the SAFETY-28 +census and rule 9 on the new keys. Part 5 is the schema. + +Mutating a module this file grades? Read MUTATION.md first (purge __pycache__). +The seam mutants this file was run against are listed at the end. +""" + +import asyncio +import io +import sys +from datetime import datetime, timedelta, timezone +from pathlib import Path + +from app import drivers +from app.drivers import (DEFAULT_DRIVER, REGISTRY, SELECTABLE, Driver, + DriverNotBound, GoodWeEsphomeDriver, Probe, + UnavailableDriver, build_driver, census_key) +from app.freshness import BOUNDS, Bound +from app.inverter import LINK_STALE_S, Inverter, build_roster +from app.main import Controller +from app.store import DEFAULTS +from app.validate import (actuator_writers, validate_startup_config, + validate_unit_ranges) + +fails = [] +HERE = Path(__file__).resolve().parent + + +def check(name, cond): + print(f" {'ok ' if cond else 'FAIL'} {name}") + if not cond: + fails.append(name) + + +class MemStore: + def __init__(self): + self.data = dict(DEFAULTS) + + def set(self, k, v): + self.data[k] = v + + def get_time(self, *a): + return None + + def set_time(self, *a): + pass + + +class Pub: + """Keeps the last published values, like test_mqtt.py's.""" + enabled = False + + def __init__(self): + self.last = {} + + def publish(self, values): + self.last = dict(values) + + def close(self): + pass + + +class LogHass: + """Home Assistant for the GoodWe unit. EVERY call is logged with its entity, + so a path that talks to Home Assistant about anything else is on record.""" + + def __init__(self, numbers=None, limits=None, accept=True): + self.numbers = dict(numbers or {}) + self.limits_by = dict(limits or {}) + self.accept = accept + self.log = [] # (method, entity, extra) + + async def number(self, entity, invert=False): + self.log.append(("number", entity, invert)) + v = self.numbers.get(entity) + return None if v is None else (-v if invert else v) + + async def limits(self, entity): + self.log.append(("limits", entity, None)) + return self.limits_by.get(entity, (None, None, None)) + + async def set_number(self, entity, value): + self.log.append(("set_number", entity, value)) + if self.accept: + self.numbers[entity.replace("number.sp", "sensor.batt")] = value + return self.accept + + async def state(self, entity): + self.log.append(("state", entity, None)) + return None + + async def call(self, *a): + self.log.append(("call", repr(a), None)) + return True + + def entities(self): + """Every entity name Home Assistant was asked about (service calls + like notifications name none).""" + return {e for m, e, _x in self.log if m != "call"} + + +def run(coro): + return asyncio.run(coro) + + +def raises(exc, make): + """Does `run(make())` raise `exc`? `make` builds the coroutine.""" + try: + run(make()) + except exc: + return True + except Exception: + return False + return False + + +# --------------------------------------------------------------------------- +# a fake driver: a unit reached by host:port:unit, no Home Assistant at all +# --------------------------------------------------------------------------- +FAKE_BOUND = Bound(signal="fake", unchanged_s=11.0, move_w=100.0, + release_s=5.0, wired=True, derivation="test") + + +class FakeDriver(Driver): + name = "fake" + writer_key = "host" + brand = "Fake" + link_stale_s = 7.0 + batt_bound = FAKE_BOUND + settle_s = 3.0 + words = {"idle": "handed back"} + + def __init__(self, host="", port=502, unit_id=1, rating=1000.0, soc=60.0): + self.host, self.port, self.unit_id = host, port, unit_id + self.rating, self.soc, self.power = rating, soc, 0.0 + self.accept = True + self.calls = [] # (what, arg) + + @classmethod + def from_entry(cls, entry): + return cls(entry.get("host", ""), entry.get("port", 502), + entry.get("unit_id", 1), rating=entry.get("_rating", 1000.0)) + + @classmethod + def actuator_id_of(cls, entry): + host = str(entry.get("host") or "").strip() + return (f"{host}:{entry.get('port', 502)}:{entry.get('unit_id', 1)}" + if host else "") + + def configured(self): + return bool(self.host) + + def actuator_id(self): + return f"{self.host}:{self.port}:{self.unit_id}" + + def soc_source(self): + return "fake-soc" + + def batt_source(self): + return "fake-power" + + async def limits(self): + self.calls.append(("limits", None)) + return -self.rating, self.rating + + async def read(self): + self.calls.append(("read", None)) + return self.soc, self.power, {"state": "ok"} + + async def write(self, w): + self.calls.append(("write", w)) + if self.accept: + self.power = w # the battery follows its command + return self.accept + + async def release(self): + self.calls.append(("release", None)) + return True + + async def probe(self): + self.calls.append(("probe", None)) + return Probe(-self.rating, self.rating, self.soc, self.power, {}) + + +def calls(drv, what): + return [a for w, a in drv.calls if w == what] + + +print("SMA-2 - the inverter driver seam") +print() +# ============================================================================== +print("1. the GoodWe driver makes the calls the code it replaced made") + +GW = dict(setpoint_entity="number.sp_a", soc_entity="sensor.soc_a", + batt_entity="sensor.batt_a") +h1 = LogHass({"sensor.soc_a": 55.0, "sensor.batt_a": 700.0}, + {"number.sp_a": (-5000.0, 4000.0, 1.0)}) +d1 = GoodWeEsphomeDriver(**GW).bind(h1) +check("a driver built from an entry names its three entities and is configured", + GoodWeEsphomeDriver.from_entry(dict(GW)).setpoint_entity == "number.sp_a" + and d1.configured() and d1.actuator_id() == "number.sp_a" + and (d1.soc_source(), d1.batt_source()) + == ("sensor.soc_a", "sensor.batt_a")) +check("...and one with no setpoint entity is not", + not GoodWeEsphomeDriver().configured() + and not GoodWeEsphomeDriver(soc_entity="sensor.soc_a").configured()) +check("limits() is hass.limits(setpoint) and hands back (min, max)", + run(d1.limits()) == (-5000.0, 4000.0) + and h1.log == [("limits", "number.sp_a", None)]) +h1.log.clear() +check("read() asks for the SoC first, then the power, with the invert flag, " + "and nothing else", + run(d1.read()) == (55.0, 700.0, {}) + and h1.log == [("number", "sensor.soc_a", False), + ("number", "sensor.batt_a", False)]) +h1.log.clear() +d1i = GoodWeEsphomeDriver(batt_invert=True, **GW).bind(h1) +check("...and a reversed power sensor is reversed by hass.number, as before", + run(d1i.read()) == (55.0, -700.0, {}) + and h1.log[1] == ("number", "sensor.batt_a", True)) +h1.log.clear() +check("write(w) is hass.set_number(setpoint, w) and says whether it took it", + run(d1.write(-250.0)) is True and h1.log == [("set_number", "number.sp_a", + -250.0)]) +h1.log.clear() +h1.accept = False +check("...and False when Home Assistant refuses", + run(d1.write(10.0)) is False) +h1.accept = True +h1.log.clear() +check("release() is write(0): GoodWe is handed back by its board's own watchdog", + run(d1.release()) is True + and h1.log == [("set_number", "number.sp_a", 0.0)]) +h1.log.clear() +h1.numbers.update({"sensor.soc_a": 55.0, "sensor.batt_a": 700.0}) +pr = run(d1.probe()) +check("probe() reads the range, the SoC and the power and WRITES NOTHING", + pr == Probe(-5000.0, 4000.0, 55.0, 700.0, {}) + and all(m != "set_number" for m, _e, _x in h1.log)) +check("a driver with no Home Assistant is loud, not a silent 'no answer'", + raises(DriverNotBound, GoodWeEsphomeDriver(**GW).limits) + and raises(DriverNotBound, GoodWeEsphomeDriver(**GW).read) + and raises(DriverNotBound, lambda: GoodWeEsphomeDriver(**GW).write(1.0)) + and not GoodWeEsphomeDriver(**GW).bound()) +_swap = {"h": h1} +d1g = GoodWeEsphomeDriver(**GW).bind_getter(lambda: _swap["h"]) +h1b = LogHass({"sensor.soc_a": 1.0, "sensor.batt_a": 2.0}) +_swap["h"] = h1b +check("bind_getter() looks Home Assistant up at each use, so a Controller whose " + ".hass is replaced afterwards is followed", + d1g.bound() and run(d1g.read()) == (1.0, 2.0, {}) and h1b.log) +check("its numbers are the ones the code it replaced used", + (d1.link_stale_s, d1.batt_bound, d1.brand) + == (LINK_STALE_S, BOUNDS["batt"], "GoodWe") and LINK_STALE_S == 30.0 + and d1.settle_s == 18.0 and d1.words == {}) + +# ============================================================================== +print() +print("2. what Inverter reads from its driver") + +NOW = datetime(2026, 11, 7, 12, 0, tzinfo=timezone.utc) +fk = FakeDriver(host="10.9.9.9", unit_id=3) +inv_f = Inverter("fake-a", driver=fk, max_w=2000.0) +inv_g = Inverter("inverter-b", max_w=3000.0, **{ + "setpoint_entity": "number.sp_b", "soc_entity": "sensor.soc_b", + "batt_entity": "sensor.batt_b"}) +check("an Inverter built from the old keywords gets the GoodWe driver, with " + "the same entities on the same attributes", + isinstance(inv_g.driver, GoodWeEsphomeDriver) + and (inv_g.setpoint_entity, inv_g.soc_entity, inv_g.batt_entity, + inv_g.batt_invert) == ("number.sp_b", "sensor.soc_b", + "sensor.batt_b", False)) +check("...and one given another driver has none of those entities", + (inv_f.setpoint_entity, inv_f.soc_entity, inv_f.batt_entity) == ("", "", "")) +check("has_writer() is the driver's configured()", + inv_f.has_writer() and inv_g.has_writer() + and not Inverter("x", driver=FakeDriver(host="")).has_writer() + and not Inverter("y").has_writer()) +check("the freshness gate is built on the DRIVER's bound", + inv_f.batt_fresh.bound is FAKE_BOUND + and inv_g.batt_fresh.bound is BOUNDS["batt"]) +for u in (inv_f, inv_g): + u.note_write(False, NOW - timedelta(seconds=10)) +check("is_live() takes its window from the driver: 10 s of rejected writes " + "drops a 7 s unit and not a GoodWe one", + inv_f.is_live(NOW)[0] is False and inv_g.is_live(NOW)[0] is True) +check("...and an explicit window still wins", + inv_f.is_live(NOW, stale_s=60.0)[0] is True + and inv_g.is_live(NOW, stale_s=5.0)[0] is False) +inv_f5 = Inverter("fake-5", driver=FakeDriver(host="h"), max_w=2000.0) +inv_f5.note_write(False, NOW - timedelta(seconds=5)) +check("link_note() says the driver's window: 'after 7 seconds' for the fake " + "unit, 'after 30 seconds' for the GoodWe one, and nothing for a healthy unit", + "after 7 seconds" in inv_f5.link_note(NOW) + and "after 30 seconds" in inv_g.link_note(NOW) + and "cannot be controlled" in inv_f.link_note(NOW) + and Inverter("z", driver=FakeDriver(host="h")).link_note(NOW) == "") +check("an unconfigured unit gets the no-setpoint row, as before", + Inverter("x", driver=FakeDriver(host="")).link_note(NOW).startswith("x: ") + and Inverter("x", driver=FakeDriver(host="")).link_note(NOW) + == Inverter("x").link_note(NOW) != "") +check("an entry that names a driver this build lacks is an unconfigured unit, " + "not a crash and not GoodWe", + isinstance(build_driver({"driver": "nope", "setpoint_entity": "number.x"}), + UnavailableDriver) + and not build_driver({"driver": "nope"}).configured() + and build_roster({"inverters": [{"name": "n", "driver": "nope", + "setpoint_entity": "number.x", + "max_w": 1000}]}, None)[0].has_writer() + is False) + +# ============================================================================== +print() +print("3. a roster of one fake unit and one GoodWe unit, in a real Controller") + +REGISTRY["fake"] = FakeDriver +OPTS = dict(meter_source="off", meter_entity="sensor.grid", max_w=5000, + gain=0.6, slew_w=0, deadband_w=15, target_grid_w=0, step_w=10, + heartbeat_s=10, stale_input_s=15, shadow_mode="off", + inverters=[ + dict(name="fake-a", driver="fake", host="10.9.9.9", port=502, + unit_id=3, max_w=2000, bms_soc_floor=10), + dict(name="inverter-b", setpoint_entity="number.sp_b", + soc_entity="sensor.soc_b", batt_entity="sensor.batt_b", + max_w=3000, bms_soc_floor=15)]) +B_ENT = {"number.sp_b", "sensor.soc_b", "sensor.batt_b"} + + +def site(rating_f=1000.0, **over): + hass = LogHass({"sensor.grid": 3000.0, "sensor.soc_b": 50.0, + "sensor.batt_b": 0.0}, + {"number.sp_b": (-3000.0, 3000.0, 1.0)}) + pub = Pub() + c = Controller(dict(OPTS, **over), hass, MemStore(), pub, + stop=asyncio.Event()) + f = c.inverters[0].driver + f.rating = rating_f + c.inverters[0].dev_min, c.inverters[0].dev_max = -rating_f, rating_f + c.inverters[1].dev_min, c.inverters[1].dev_max = -3000.0, 3000.0 + c.auto = True + return c, hass, pub, f + + +async def cycles(c, n=6): + for _ in range(n): + await c.read_inputs() + await c.cycle() + + +c3, h3, p3, f3 = site() +check("the roster is built through the registry: a fake unit and a GoodWe unit", + isinstance(c3.inverters[0].driver, FakeDriver) + and isinstance(c3.inverters[1].driver, GoodWeEsphomeDriver) + and f3.actuator_id() == "10.9.9.9:502:3") +run(cycles(c3)) +fa, fb = c3.inverters +print(f" after 6 cycles: fake-a wrote {calls(f3, 'write')}, GoodWe wrote " + f"{[x for m, e, x in h3.log if m == 'set_number']}, shares " + f"{c3.allocation.shares if c3.allocation else None}") +check("READ: the unit's SoC and power come from its driver, the GoodWe unit's " + "from its entities", + calls(f3, "read") and fa.soc == 60.0 and fb.soc == 50.0 + and ("number", "sensor.soc_b", False) in h3.log) +check("WRITE: the fake unit's share went to ITS driver and the GoodWe unit's to " + "ITS entity, each non-zero, the same direction", + calls(f3, "write") and calls(f3, "write")[-1] > 0 + and [x for m, e, x in h3.log if m == "set_number" and e == "number.sp_b"][-1] > 0) +check("ALLOCATE: each unit holds a share inside its own rating, and the shares " + "are the site's (3000 W of grid, fake capped at 1000 W by its driver)", + c3.allocation is not None + and set(c3.allocation.shares) == {"fake-a", "inverter-b"} + and 0 < c3.allocation.shares["fake-a"] <= 1000.0 + and 0 < c3.allocation.shares["inverter-b"] <= 3000.0 + and calls(f3, "write")[-1] == c3.allocation.shares["fake-a"]) +check("GRADE: the fake unit's rating is the driver's range (1000 W), the " + "GoodWe unit's is its entry's (3000 W), each from its own source", + (fa.rating_w(), fb.rating_w()) == (1000.0, 3000.0)) +check("the fake's power follows its command, so the freshness gate (the fake's " + "own bound) lets the reading through", + fa.batt == calls(f3, "write")[-1] and fa.batt_fresh.bound is FAKE_BOUND) +run(c3.send_heartbeats(10_000.0, 10.0)) +n_before = len(calls(f3, "write")) +check("HEARTBEAT: it re-emits the fake unit's standing command through its driver", + n_before >= 2 and calls(f3, "write")[-1] == calls(f3, "write")[-2]) +run(c3.command_site_zero()) +check("STOP: command_site_zero() reaches the fake unit through its driver, as " + "0 W (the hand-back is SMA-10's, and release() has no caller yet)", + calls(f3, "write")[-1] == 0.0 and not calls(f3, "release")) +c3.publish() +print(f" published unit rows: " + f"{sorted(k for k in p3.last if k.startswith('unit/'))}") +check("PUBLISH: one set of rows per unit, with that unit's own numbers", + p3.last.get("unit/fake_a/soc") == 60.0 + and p3.last.get("unit/inverter_b/soc") == 50.0 + and "unit/fake_a/battery" in p3.last and "unit/inverter_b/battery" in p3.last) +rows = c3.checks() +check("STATUS: the fake unit's sensor rows are not 'unset' (its driver has a " + "source), and it has the control row", + not [r for r in rows if r.get("id") == "sensor" and not r["ok"] + and "fake" in r["text"]]) +# liveness: the fake's 7 s window, the GoodWe unit's 30 s +c4, h4, p4, f4 = site() +run(cycles(c4, 2)) +f4.accept = False # the fake's link is down; GoodWe's is fine +_now = datetime.now(timezone.utc) +for u in c4.inverters: + u.note_write(False, _now - timedelta(seconds=10)) +check("LIVENESS: after 10 s of rejected writes the fake unit is left out of the " + "split and the GoodWe unit stays in it, per its driver's window", + c4.live_units() == ("inverter-b",)) +run(cycles(c4, 4)) +check("...and the survivor carries the site: the allocation names only it", + c4.allocation is not None and set(c4.allocation.shares) == {"inverter-b"} + and c4.allocation.shares["inverter-b"] > 1000.0) +# the tripwire: nothing in the world was asked about the fake unit +asked = h3.entities() | h4.entities() +print(f" Home Assistant was asked about: {sorted(asked)}") +check("TRIPWIRE: Home Assistant was only ever asked about the GoodWe unit's " + "entities and the meter - no empty name, no host - on read, write, " + "heartbeat, stop and limits alike", + asked <= B_ENT | {"sensor.grid"} and "" not in asked + and not any("10.9.9.9" in e or "fake" in e for e in asked)) +check("...and the GoodWe unit was never touched through the fake", + not [1 for w, a in f3.calls + f4.calls + if isinstance(a, str) and "number.sp_b" in a]) + +# limits and the per-unit range check, through run_control() +async def drive(c, secs): + try: + await asyncio.wait_for(c.run_control(), timeout=secs) + except asyncio.TimeoutError: + pass + + +c5, h5, p5, f5 = site(rating_f=2500.0) +f5.rating = 2500.0 +c5.inverters[0].dev_min = c5.inverters[0].dev_max = None +c5.inverters[1].dev_min = c5.inverters[1].dev_max = None +run(drive(c5, 3.0)) +check("LIMITS: run_control() reads the fake unit's range from its driver and " + "the GoodWe unit's from its entity", + calls(f5, "limits") + and (c5.inverters[0].dev_min, c5.inverters[0].dev_max) == (-2500.0, 2500.0) + and (c5.inverters[1].dev_min, c5.inverters[1].dev_max) == (-3000.0, 3000.0) + and [e for m, e, x in h5.log if m == "limits"] == ["number.sp_b"]) +c6, h6, p6, f6 = site() +f6.rating = 1500.0 # the entry says 2000 W +c6.inverters[0].dev_min = c6.inverters[0].dev_max = None +run(drive(c6, 10.0)) +check("RULE 2 PER UNIT: a fake unit rated above what its driver says it accepts " + "is refused by name, as a GoodWe one is", + c6.refused and any("fake-a" in p and "10.9.9.9:502:3" in p + for p in c6.refused) + and not calls(f6, "write")) + +# the settings panel has no Controller: it binds each unit's driver itself +from app import selfconfig # noqa: E402 +hs = LogHass({}, {"number.sp_b": (-3000.0, 3000.0, 1.0)}) +_ps, _unread = run(selfconfig.problems_and_ranges( + dict(OPTS, inverters=[dict(OPTS["inverters"][0], _rating=1500.0), + OPTS["inverters"][1]]), hs)) +check("PANEL: the settings panel reads the fake unit's range from its driver " + "and the GoodWe unit's from its entity, and grades the fake unit by name", + any("fake-a" in p and "10.9.9.9:502:3 accepts at most 1500" in p + for p in _ps) + and [e for m, e, x in hs.log if m == "limits"] == ["number.sp_b"] + and _unread == []) + +# the seam is the only way in: a fake-only roster touches Home Assistant for +# nothing but the meter +hz = LogHass({"sensor.grid": 800.0}) +cz = Controller(dict(OPTS, inverters=[OPTS["inverters"][0]]), hz, MemStore(), + Pub(), stop=asyncio.Event()) +cz.inverters[0].dev_min, cz.inverters[0].dev_max = -1000.0, 1000.0 +cz.auto = True +run(cycles(cz, 3)) +check("A FAKE-ONLY ROSTER never asks Home Assistant for anything but the " + "meter, and still reads and writes (nothing on the control path is " + "keyed on a GoodWe entity)", + hz.entities() <= {"sensor.grid"} and calls(cz.inverters[0].driver, "read") + and calls(cz.inverters[0].driver, "write")) + +# ============================================================================== +print() +print("4. SAFETY-28's census counts actuator ids; rule 9 on the new keys") + +def entry(**kw): + return dict(name="u", max_w=3000, bms_soc_floor=10, **kw) + + +check("GoodWe entries are counted as before: one entity named twice is one " + "actuator with two writers, whatever its case", + {k: len(v) for k, v in actuator_writers({"inverters": [ + entry(setpoint_entity="number.X"), + entry(setpoint_entity=" number.x ")]}).items()} == {"number.x": 2} + and actuator_writers({"setpoint_entity": "number.a", "inverters": [ + entry(setpoint_entity="number.a")]})["number.a"] + == ["setpoint_entity", "inverters[0].setpoint_entity"]) +cen = actuator_writers({"inverters": [ + entry(driver="fake", host="gw", port=502, unit_id=1), + entry(driver="fake", host="gw", port=502, unit_id=2), + entry(driver="fake", host="gw", port=503, unit_id=1), + entry(driver="fake", host="GW", port=502, unit_id=1)]}) +print(f" census {cen}") +check("another driver's actuator is host:port:unit: two units behind one " + "gateway are two actuators, and one unit named twice is one", + set(cen) == {"gw:502:1", "gw:502:2", "gw:503:1"} + and cen["gw:502:1"] == ["inverters[0].host", "inverters[3].host"]) +check("a GoodWe key on a fake entry declares no writer there, and a `host` on " + "a GoodWe entry declares none", + actuator_writers({"inverters": [entry(driver="fake", host="gw", + setpoint_entity="number.x")]}) + == {"gw:502:1": ["inverters[0].host"]} + and actuator_writers({"inverters": [entry(setpoint_entity="number.x", + host="gw")]}) + == {"number.x": ["inverters[0].setpoint_entity"]}) +check("an entry naming a driver this build lacks still counts its setpoint " + "entity - refusing too much is the safe direction", + actuator_writers({"inverters": [entry(driver="nope", + setpoint_entity="number.q")]}) + == {"number.q": ["inverters[0].setpoint_entity"]}) +check("census_key() folds case and space", + census_key(" Number.X ") == "number.x") + +GOOD = dict(entry(setpoint_entity="number.sp_a", soc_entity="sensor.soc_a", + batt_entity="sensor.batt_a")) +BASEOPTS = {"meter_source": "off", "meter_entity": "sensor.grid"} +base_problems = validate_startup_config(dict(BASEOPTS, inverters=[dict(GOOD)])) +print(f" baseline problems for a plain GoodWe roster: {base_problems}") + + +def prob(**kw): + return validate_startup_config(dict(BASEOPTS, inverters=[dict(GOOD, **kw)])) + + +check("an existing entry (no new key) is graded exactly as before", + prob() == base_problems) +check("...and so is one that says driver goodwe_esphome, or carries every new " + "key in a valid shape", + prob(driver="goodwe_esphome") == base_problems + and prob(driver="", host="", maintenance=False) == base_problems + and prob(host="inv.local", port=502, unit_id=3, maintenance=True) + == base_problems) +def named(problems, key): + return [p for p in problems if f"inverters.{key}" in p or f"({key})" in p] + + +check("only goodwe_esphome can be selected for now", + named(prob(driver="sma_modbus"), "driver") + and named(prob(driver="fake"), "driver") + and SELECTABLE == (DEFAULT_DRIVER,)) +check("a host with a space, a bad port or unit number and a non-boolean " + "maintenance are each refused by key; the right shapes are not", + named(prob(host="a b"), "host") + and named(prob(port=0), "port") and named(prob(port=70000), "port") + and named(prob(port="x"), "port") and named(prob(port=True), "port") + and named(prob(unit_id=248), "unit_id") + and named(prob(unit_id=1.5), "unit_id") + and named(prob(maintenance="yes"), "maintenance") + and not named(prob(port=65535, unit_id=0, host="10.0.0.1"), "port")) +check("a GoodWe entry with no setpoint entity is refused with the same sentence " + "as before", + any("has no" in p and "setpoint entity" in p.lower() + for p in validate_startup_config(dict(BASEOPTS, inverters=[ + dict(GOOD, setpoint_entity="")])))) +fu = FakeDriver(host="gw") +check("rule 2 grades any unit by its driver's actuator id, so a unit with no " + "setpoint entity is no longer skipped", + validate_unit_ranges([Inverter("a", driver=fu, max_w=2000.0)]) + == [] and validate_unit_ranges([_u := Inverter("a", driver=fu, + max_w=2000.0)]) == [] + and (setattr(_u, "dev_max", 1000.0) or True) + and len(validate_unit_ranges([_u])) == 1 + and "gw:502:1" in validate_unit_ranges([_u])[0]) + +# ============================================================================== +print() +print("5. the schema") +cfg = io.open(HERE / "config.yaml", encoding="utf-8").read() +sch = cfg[cfg.index("\nschema:"):] +ent = sch[sch.index("\n inverters:"):] +ent = ent[:ent.index("\n\n")] +decl = dict(ln.strip().lstrip("- ").split(": ", 1) + for ln in ent.splitlines()[1:] if ": " in ln) +check("setpoint_entity is optional per entry, beside driver, host, port, " + "unit_id and maintenance; name, max_w and bms_soc_floor stay required", + decl["setpoint_entity"] == "str?" and decl["driver"] == "list(goodwe_esphome)?" + and decl["host"] == "str?" and decl["port"] == "port?" + and decl["unit_id"] == "int(0,247)?" and decl["maintenance"] == "bool?" + and not any(decl[k].endswith("?") for k in ("name", "max_w", + "bms_soc_floor"))) +check("config.yaml lists exactly the selectable drivers", + decl["driver"] == f"list({'|'.join(SELECTABLE)})?") + +del REGISTRY["fake"] + +# THE SEAM MUTANTS THIS FILE WAS RUN AGAINST (twelve, each one edit to a copy of +# the tree; all twelve turn it red): +# write_unit calls hass.set_number itself -> part 3 WRITE (and the tripwire) +# _read_unit calls hass.number(soc/batt entity) -> part 3 READ +# run_control's limits keyed on setpoint_entity -> part 3 LIMITS +# the panel's range check keyed on setpoint_entity -> part 3 PANEL +# has_writer() back to bool(setpoint_entity) -> part 2 has_writer +# is_live() window back to LINK_STALE_S -> part 2 is_live +# link_note() window back to LINK_STALE_S -> part 2 link_note +# Freshness built on BOUNDS["batt"] -> part 2 freshness bound +# _write_unit's guard keyed on setpoint_entity -> part 3 WRITE +# the census back to the GoodWe key only -> part 4 census +# rule 2 per unit back to .setpoint_entity -> part 3 RULE 2 PER UNIT +# the status rows' source test back to the entities -> part 3 STATUS +# NOT COVERED: the call in amain() that reads the flat setpoint entity's range +# before any Controller exists. It runs only inside amain(), which no check +# starts; it is two lines and the GoodWe driver's own limits(), graded in part 1. +print() +if fails: + print(f"{len(fails)} FAILED") + sys.exit(1) +print("all passed") diff --git a/goodwe_controller/test_translations.py b/goodwe_controller/test_translations.py index 47e1db8..a05f169 100644 --- a/goodwe_controller/test_translations.py +++ b/goodwe_controller/test_translations.py @@ -75,7 +75,7 @@ check(f"the option surface was actually read - {len(ORDER)} options, not zero", check("...and the repeatable `inverters:` section's per-entry keys were read " "too, which is the half a top-level-only reader would miss", NESTED.get("inverters", [])[:2] == ["name", "setpoint_entity"] - and len(NESTED["inverters"]) == 8) + and len(NESTED["inverters"]) == 13) # SMA-2: +5 driver keys print("\n2. AC: every option and every per-entry key is translated, in BOTH " "languages") @@ -281,7 +281,7 @@ check("...and the accepted subset really does round-trip the shipped files - " f"per-entry keys came back out of each, which is what says the " f"strictness above is not just refusing everything", all(len(TR[lg]["configuration"]) == len(ORDER) - and len(TR[lg]["configuration"]["inverters"]["fields"]) == 8 + and len(TR[lg]["configuration"]["inverters"]["fields"]) == 13 for lg in LANGS)) print(" ⚠️ the licence for this hand parser is NOT this file. " "tools/supervisor_shape_probe.py section F loads the same two files with " diff --git a/goodwe_controller/test_uilayout.py b/goodwe_controller/test_uilayout.py index 70c4bdb..ab68084 100644 --- a/goodwe_controller/test_uilayout.py +++ b/goodwe_controller/test_uilayout.py @@ -51,8 +51,10 @@ def coverage_problems(order, nested): out += [f"{k} is not in config.yaml" for k in set(placed) | set(L.DERIVED) | set(L.HIDDEN) if k not in order] sub = [k.split(".", 1)[1] for k in KEYS if k.startswith("inverters.")] + # SMA-2: a per-entry key may instead be listed in L.ROSTER_HIDDEN (the + # driver keys the page does not draw), and then it must not also be a field. out += [f"inverters.{k} has no card field" for k in nested.get("inverters", []) - if sub.count(k) != 1] + if sub.count(k) + (k in L.ROSTER_HIDDEN) != 1] out += [f"inverters.{k} is not in config.yaml" for k in sub if k not in nested.get("inverters", [])] return out diff --git a/goodwe_controller/tools/supervisor_shape_probe.py b/goodwe_controller/tools/supervisor_shape_probe.py index 426d2a2..c370844 100644 --- a/goodwe_controller/tools/supervisor_shape_probe.py +++ b/goodwe_controller/tools/supervisor_shape_probe.py @@ -243,8 +243,6 @@ show("the per-entry BASELINE is ACCEPTED, so every refusal below has nowhere " "else to come from", ok, json.dumps(out.get("inverters")) if ok else out) for title, entry, expect in ( ("no name", {k: v for k, v in D_FULL.items() if k != "name"}, "name"), - ("no setpoint_entity", - {k: v for k, v in D_FULL.items() if k != "setpoint_entity"}, "setpoint_entity"), ("no max_w", {k: v for k, v in D_FULL.items() if k != "max_w"}, "max_w"), ("no bms_soc_floor (SAFETY-32)", {k: v for k, v in D_FULL.items() if k != "bms_soc_floor"}, "bms_soc_floor"), @@ -257,6 +255,40 @@ for title, entry, expect in ( ok, out = validate(dict(D_BASE, inverters="[]")) show("a JSON STRING where the list belongs is refused", not ok, out) +print("\n== K. SMA-2: the driver keys on an inverter entry ==") +# `setpoint_entity` is OPTIONAL per entry since SMA-2 (it is the GoodWe driver's +# own key; validate.py rule 9(d) requires it for GoodWe), beside `driver`, +# `host`, `port`, `unit_id` and `maintenance`. Same method as §D: an ACCEPTED +# baseline, ONE value varied, the refusal asserted on the KEY'S NAME. +K_FULL = dict(D_FULL, driver="goodwe_esphome", host="inverter.local", port=502, + unit_id=1, maintenance=False) +ok, out = validate(dict(D_BASE, inverters=[K_FULL])) +show("an entry carrying every new key is ACCEPTED, and the keys survive", + ok and all(out["inverters"][0].get(k) == K_FULL[k] + for k in ("driver", "host", "port", "unit_id", "maintenance")), + json.dumps(out.get("inverters")) if ok else out) +ok, out = validate(dict(D_BASE, inverters=[D_FULL])) +show("an entry from before SMA-2 (none of the new keys) is ACCEPTED unchanged", + ok and out["inverters"] == [D_FULL], out) +ok, out = validate(dict(D_BASE, inverters=[ + {k: v for k, v in D_FULL.items() if k != "setpoint_entity"}])) +show("an entry with NO setpoint_entity is now accepted by the schema " + "(rule 9(d) refuses it for GoodWe, at startup)", ok, out) +# ⚠️ THE SUPERVISOR'S REFUSAL NAMES THE VALUE'S RULE, NOT THE KEY, FOR A NESTED +# ENTRY (§D's "at most 5000"), so the expectation is that rule's own words. +for title, entry, expect in ( + ("driver other than goodwe_esphome", dict(K_FULL, driver="sma"), + "one of ['goodwe_esphome']"), + ("port 0", dict(K_FULL, port=0), "at least 1"), + ("port 70000", dict(K_FULL, port=70000), "at most 65535"), + ("unit_id 248", dict(K_FULL, unit_id=248), "at most 247"), + ("host a number", dict(K_FULL, host=5), "expected str"), + ("maintenance a word that is not a boolean", + dict(K_FULL, maintenance="banana"), "boolean")): + ok, out = validate(dict(D_BASE, inverters=[entry])) + show(f"an entry with {title} is refused, with the rule's own words", + (not ok) and expect in str(out), out) + print("\n== E. the ROLLBACK direction, and the operator's UI ==") # The old (flat) schema, i.e. what an operator would be running after a diff --git a/goodwe_controller/translations/en.yaml b/goodwe_controller/translations/en.yaml index d390d74..a23daba 100644 --- a/goodwe_controller/translations/en.yaml +++ b/goodwe_controller/translations/en.yaml @@ -79,6 +79,21 @@ configuration: capacity_kwh: name: "Usable battery capacity (kWh) (optional)" description: "This battery's usable capacity from its datasheet. Fill it in on every inverter or on none. The add-on then shares the work by the energy each battery has left." + driver: + name: "Inverter type (optional)" + description: "The kind of inverter. Leave it empty for a GoodWe." + host: + name: "Inverter address (optional)" + description: "Not used yet. Leave it empty." + port: + name: "Inverter port (optional)" + description: "Not used yet. Leave it empty." + unit_id: + name: "Inverter unit number (optional)" + description: "Not used yet. Leave it empty." + maintenance: + name: "Battery maintenance (optional)" + description: "Not used yet. Leave it empty." estop_fitted: name: "Hardware · RS485 emergency stop is fitted" description: "Switch this on if the emergency stop is installed. It only changes a warning on the status page." diff --git a/goodwe_controller/translations/nl.yaml b/goodwe_controller/translations/nl.yaml index 3db9bd5..0eecb3b 100644 --- a/goodwe_controller/translations/nl.yaml +++ b/goodwe_controller/translations/nl.yaml @@ -60,6 +60,21 @@ configuration: capacity_kwh: name: "Bruikbare capaciteit (kWh) (optioneel)" description: "De bruikbare capaciteit van deze batterij, van het datablad. Vul ze bij elke omvormer in of bij geen. Dan verdeelt de add-on het werk volgens de energie in elke batterij." + driver: + name: "Type omvormer (optioneel)" + description: "Het soort omvormer. Laat leeg voor een GoodWe." + host: + name: "Adres van de omvormer (optioneel)" + description: "Nog niet in gebruik. Laat leeg." + port: + name: "Poort van de omvormer (optioneel)" + description: "Nog niet in gebruik. Laat leeg." + unit_id: + name: "Eenheidsnummer van de omvormer (optioneel)" + description: "Nog niet in gebruik. Laat leeg." + maintenance: + name: "Onderhoud van de batterij (optioneel)" + description: "Nog niet in gebruik. Laat leeg." estop_fitted: name: "Hardware · RS485-noodstop is geplaatst" description: "Zet dit aan als de noodstop geïnstalleerd is. Het verandert alleen een waarschuwing op de statuspagina." -- 2.54.0 From 0b20f53cf8fe1b757f925e17eb23d2649dfb9c0b Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Sat, 7 Nov 2026 12:20:56 +0100 Subject: [PATCH 14/22] SMA-2 round 2: census keeps ACTUATOR_WRITE_KEYS, setters, NaN/inf port refused, tests Co-Authored-By: Claude --- goodwe_controller/CHANGELOG.md | 2 +- .../DECISIONS/SMA-2-driver-seam.md | 15 +- goodwe_controller/app/inverter.py | 25 ++++ goodwe_controller/app/validate.py | 17 ++- goodwe_controller/test_sma2_driver.py | 135 +++++++++++++++++- 5 files changed, 185 insertions(+), 9 deletions(-) diff --git a/goodwe_controller/CHANGELOG.md b/goodwe_controller/CHANGELOG.md index ef51a31..63bf561 100644 --- a/goodwe_controller/CHANGELOG.md +++ b/goodwe_controller/CHANGELOG.md @@ -2,7 +2,7 @@ ### Not in a release yet -- Each inverter is now reached through a driver, so another make can be added later. Nothing changes for a GoodWe site, and the inverter list accepts five new optional settings that nothing reads yet ([[SMA-2]]). +- Each inverter is now reached through a driver, so another make can be added later. Nothing changes for a GoodWe site. The inverter list also accepts five new optional settings that nothing reads yet, and the schema no longer requires a setpoint entity per entry: a GoodWe entry without one is still refused at start ([[SMA-2]]). - Home now draws today's charge from the grid for every hour the add-on will really charge, not only the run under way ([[UX-77]]). - When Home Assistant passes readings on late, control no longer swings on them: it waits until the battery shows its last power request, and the peak limit no longer asks for full power on them. The page says so, and if the readings stay too late every inverter goes to 0 W ([[SAFETY-128]]). - With a 5 s meter, readings 30 s late are now noticed within 100 seconds, not 3 minutes ([[SAFETY-129]]). diff --git a/goodwe_controller/DECISIONS/SMA-2-driver-seam.md b/goodwe_controller/DECISIONS/SMA-2-driver-seam.md index 8981c42..f904173 100644 --- a/goodwe_controller/DECISIONS/SMA-2-driver-seam.md +++ b/goodwe_controller/DECISIONS/SMA-2-driver-seam.md @@ -25,11 +25,16 @@ from the driver yet either: with only GoodWe they would change nothing. **The seam is not TCP-shaped.** The census asks each entry's driver for its actuator id: the setpoint entity for GoodWe, `host:port:unit_id` for a driver reached over a socket, so two units behind one gateway are two actuators and -one unit named twice is the SAFETY-28 refusal. The driver binds its Home +one unit named twice is the SAFETY-28 refusal. The census still counts every +`setpoint_entity` key as before and adds the driver's own actuator to it (parity +with 23ff9d0 on 349 option shapes is a test). The driver binds its Home Assistant on first use by the Controller, so a unit added later is wired too. -**Schema.** `setpoint_entity` is optional per entry (rule 9(d) still requires -it for GoodWe, with the same sentence), beside `driver` (only `goodwe_esphome`), +**Schema.** The brief says "`setpoint_entity` becomes optional" (SMA-2, "Schema per +inverter entry"; SMA-9 section 1.1): it is the GoodWe driver's own key, so a +driver reached by host and port has none. `setpoint_entity` is optional per entry +in config.yaml; rule 9(d) still refuses a GoodWe entry without it, with the same +sentence (`test_sma2_driver.py` part 6), beside `driver` (only `goodwe_esphome`), `host`, `port`, `unit_id`, `maintenance`. The last four are checked for shape and not read. The Supervisor shape probe was re-run (section K). @@ -37,3 +42,7 @@ and not read. The Supervisor shape probe was re-run (section K). call are byte-identical at N=1 and N=2; the TEL-07 announce diff is 0/0/0; a fake driver beside a GoodWe unit allocates, grades, drops and publishes per unit; a call site that goes round the seam is caught. + +**Per-entry keys the page does not draw** are listed in `uilayout.ROSTER_HIDDEN`; +test_uilayout and test_sma2_driver fail for a key that is neither a card field +nor hidden, and a save through the page keeps them (a test). diff --git a/goodwe_controller/app/inverter.py b/goodwe_controller/app/inverter.py index 64a7003..d5fa66a 100644 --- a/goodwe_controller/app/inverter.py +++ b/goodwe_controller/app/inverter.py @@ -366,6 +366,31 @@ class Inverter: def batt_invert(self) -> bool: return bool(getattr(self.driver, "batt_invert", False)) + # Assigning one writes through to the driver, as the attributes did before + # SMA-2 (nothing in the repo assigns them today; this keeps it working). + # A driver that has no such field refuses, rather than losing the value. + def _set_driver_field(self, name, value): + if not hasattr(self.driver, name): + raise AttributeError( + f"{type(self.driver).__name__} has no {name}") + setattr(self.driver, name, value) + + @setpoint_entity.setter + def setpoint_entity(self, v): + self._set_driver_field("setpoint_entity", str(v or "")) + + @soc_entity.setter + def soc_entity(self, v): + self._set_driver_field("soc_entity", str(v or "")) + + @batt_entity.setter + def batt_entity(self, v): + self._set_driver_field("batt_entity", str(v or "")) + + @batt_invert.setter + def batt_invert(self, v): + self._set_driver_field("batt_invert", bool(v)) + # -- store --------------------------------------------------------------- def key(self, name: str) -> str: """This unit's namespaced Store key for `name`. See store.unit_key().""" diff --git a/goodwe_controller/app/validate.py b/goodwe_controller/app/validate.py index d9e1371..9d9dec4 100644 --- a/goodwe_controller/app/validate.py +++ b/goodwe_controller/app/validate.py @@ -328,6 +328,7 @@ WHAT IS DELIBERATELY *NOT* REFUSED, because it is untidy rather than unsafe: """ import logging +import math import re from typing import NamedTuple @@ -466,6 +467,15 @@ def _collect_writers(node, path: str, out: list) -> None: cls = drivers.driver_class(node) for k, v in node.items(): here = f"{path}.{k}" if path else str(k) + # Every key in ACTUATOR_WRITE_KEYS declares a writer, as before + # SMA-2 and whatever the entry's driver; PLUS the entry driver's + # own writer key. The second is added to the first, never in place + # of it (census parity with 23ff9d0 is graded in test_sma2_driver). + if k in ACTUATOR_WRITE_KEYS: + entity = str(v or "").strip() + if entity: + out.append((here, entity)) + continue if k == cls.writer_key: actuator = cls.actuator_id_of(node) if actuator: @@ -793,9 +803,12 @@ def _whole(raw, lo: int, hi: int): return None try: val = float(raw) - except (TypeError, ValueError): + # NaN and infinity: int() raises on them, so refuse them first. + if not math.isfinite(val) or val != int(val): + return None + except (TypeError, ValueError, OverflowError): return None - return int(val) if val == int(val) and lo <= val <= hi else None + return int(val) if lo <= val <= hi else None def _driver_problems(entry: dict, who: str) -> list[str]: diff --git a/goodwe_controller/test_sma2_driver.py b/goodwe_controller/test_sma2_driver.py index 7629308..e98b296 100644 --- a/goodwe_controller/test_sma2_driver.py +++ b/goodwe_controller/test_sma2_driver.py @@ -514,11 +514,13 @@ check("another driver's actuator is host:port:unit: two units behind one " "gateway are two actuators, and one unit named twice is one", set(cen) == {"gw:502:1", "gw:502:2", "gw:503:1"} and cen["gw:502:1"] == ["inverters[0].host", "inverters[3].host"]) -check("a GoodWe key on a fake entry declares no writer there, and a `host` on " - "a GoodWe entry declares none", +check("a GoodWe key on a fake entry STILL declares a writer (the pre-SMA-2 " + "census kept, the driver's actuator added to it), and a `host` on a " + "GoodWe entry declares none", actuator_writers({"inverters": [entry(driver="fake", host="gw", setpoint_entity="number.x")]}) - == {"gw:502:1": ["inverters[0].host"]} + == {"number.x": ["inverters[0].setpoint_entity"], + "gw:502:1": ["inverters[0].host"]} and actuator_writers({"inverters": [entry(setpoint_entity="number.x", host="gw")]}) == {"number.x": ["inverters[0].setpoint_entity"]}) @@ -600,6 +602,133 @@ check("setpoint_entity is optional per entry, beside driver, host, port, " check("config.yaml lists exactly the selectable drivers", decl["driver"] == f"list({'|'.join(SELECTABLE)})?") +# ============================================================================== +print() +print("6. round 2: census parity, setters, rule 9(d), NaN ports, hidden keys") + +import copy, itertools, math # noqa: E402 +from app import uilayout as L, uimodel # noqa: E402 +from app.validate import ACTUATOR_WRITE_KEYS # noqa: E402 + + +def old_census(opts): + """23ff9d0's census, copied: every ACTUATOR_WRITE_KEYS key, wherever it is.""" + found = [] + + def walk(node, path): + if isinstance(node, dict): + for k, v in node.items(): + here = f"{path}.{k}" if path else str(k) + if k in ACTUATOR_WRITE_KEYS: + e = str(v or "").strip() + if e: + found.append((here, e)) + continue + walk(v, here) + elif isinstance(node, (list, tuple)): + for i, v in enumerate(node): + walk(v, f"{path}[{i}]") + walk(opts, "") + cen = {} + for where, e in found: + cen.setdefault(e.lower(), []).append(where) + return cen + + +check("ACTUATOR_WRITE_KEYS is exactly ('setpoint_entity',): the parity below " + "is about that key", tuple(ACTUATOR_WRITE_KEYS) == ("setpoint_entity",)) +SP = ["number.a", "Number.A", " number.b ", "", None, 5, ["x"]] +shapes = [] +for flat, e0, e1 in itertools.product(SP, SP, SP): + shapes.append({"setpoint_entity": flat, "inverters": [ + {"name": "a", "setpoint_entity": e0}, {"name": "b", "setpoint_entity": e1}]}) +shapes += [{"x": {"y": [{"setpoint_entity": "number.z"}]}, "setpoint_entity": "n.q"}, + {"inverters": "junk"}, {"inverters": [{"driver": "nope", + "setpoint_entity": "n.k"}]}, + {"inverters": [{"driver": "goodwe_esphome", "setpoint_entity": "n.k"}]}, + {}, {"inverters": []}] +bad = [i for i, o in enumerate(shapes) if actuator_writers(o) != old_census(o)] +check(f"CENSUS PARITY: on {len(shapes)} flat/roster/odd option shapes the " + "census equals 23ff9d0's, keys, order and all", not bad) +mixed = {"setpoint_entity": "number.a", "inverters": [ + {"name": "a", "setpoint_entity": "number.a"}, + {"name": "f", "driver": "fake", "host": "gw", "port": 502, "unit_id": 1, + "setpoint_entity": "number.ignored"}, + {"name": "g", "setpoint_entity": "number.g"}]} +new_c, old_c = actuator_writers(mixed), old_census(mixed) +check("...and on a mixed config (a fake-driver entry too) the census is a " + "SUPERSET of the pre-SMA-2 one: every old writer is still counted, " + "plus the driver's own actuator", + all(new_c.get(k) and set(v) <= set(new_c[k]) for k, v in old_c.items()) + and "gw:502:1" in new_c) + +gi = Inverter("s", setpoint_entity="number.s", soc_entity="sensor.a", + batt_entity="sensor.b") +gi.setpoint_entity, gi.soc_entity, gi.batt_entity, gi.batt_invert = ( + "number.t", "sensor.c", "sensor.d", True) +check("SETTERS: assigning setpoint_entity/soc_entity/batt_entity/batt_invert " + "on an Inverter writes through to its GoodWe driver, as it did before", + (gi.driver.setpoint_entity, gi.driver.soc_entity, gi.driver.batt_entity, + gi.driver.batt_invert) == ("number.t", "sensor.c", "sensor.d", True) + and gi.has_writer() and gi.driver.actuator_id() == "number.t") + + +def _assign(): + inv_f.setpoint_entity = "x" + + +try: + _assign() + _refused = False +except AttributeError: + _refused = True +check("...and on a driver that has no such field it refuses, not loses the value", + _refused) + +nogood = validate_startup_config(dict(BASEOPTS, inverters=[ + {k: v for k, v in GOOD.items() if k != "setpoint_entity"}])) +check("RULE 9(d): a GoodWe entry with no setpoint_entity is refused, by name " + "and by position, with the sentence it always had", + any(p.startswith("inverters[0] (u) has no") and "setpoint entity" in p.lower() + and "Set its ESPHome number entity, or remove the entry." in p + for p in nogood) + and nogood != base_problems) +check("...also with `driver: goodwe_esphome` said out loud", + any("has no" in p for p in validate_startup_config(dict( + BASEOPTS, inverters=[{k: v for k, v in dict(GOOD, driver="goodwe_esphome").items() + if k != "setpoint_entity"}])))) + +nan, inf = float("nan"), float("inf") +for key in ("port", "unit_id"): + for bad_v, nm in ((nan, "NaN"), (inf, "inf"), (-inf, "-inf")): + try: + got = prob(**{key: bad_v}) + ok_ = bool(named(got, key)) + except Exception as e: # noqa: BLE001 + got, ok_ = repr(e), False + check(f"{key} = {nm} is refused by key and does not raise", ok_) + +keys_cfg = set(decl) +placed = {k.split(".", 1)[1] for _g, _p, f in L.fields() + for k in [f.key] if k.startswith("inverters.")} +check("ROSTER_HIDDEN: every per-entry key in config.yaml is either a card " + "field or hidden, never both and never neither", + all((k in placed) != (k in L.ROSTER_HIDDEN) for k in keys_cfg) + and not (placed & set(L.ROSTER_HIDDEN)) + and set(L.ROSTER_HIDDEN) <= keys_cfg) +check("...and a per-entry key that is neither would be flagged (test_uilayout " + "grades the same rule with a made-up key)", + not all((k in placed) != (k in L.ROSTER_HIDDEN) + for k in keys_cfg | {"brand_new_key"})) +ent = dict(name="a", setpoint_entity="number.a", max_w=3000, bms_soc_floor=10, + soc_entity="sensor.s", batt_entity="sensor.b", driver="goodwe_esphome", + host="h", port=502, unit_id=3, maintenance=False) +opts_rt = {"inverters": [ent, dict(ent, name="b", setpoint_entity="number.b")]} +back = uimodel.ui_to_options(uimodel.options_to_ui(copy.deepcopy(opts_rt))) +check("the page model carries the hidden keys through a save untouched " + "(this is how ROSTER_HIDDEN keys survive the settings page)", + back.get("inverters") == opts_rt["inverters"]) + del REGISTRY["fake"] # THE SEAM MUTANTS THIS FILE WAS RUN AGAINST (twelve, each one edit to a copy of -- 2.54.0 From c0152304524f45cea74d02bc800720a180454ae9 Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Sat, 7 Nov 2026 13:09:31 +0100 Subject: [PATCH 15/22] SMA-2 round 2: keep the driver keys out of the schema, translations and labels Owner decision 2026-11-07: clients see nothing new until SMA-4. config.yaml, translations, labels and the page layout are back to 23ff9d0's. The validation of the keys stays for callers that skip the schema. Co-Authored-By: Claude --- goodwe_controller/CHANGELOG.md | 2 +- .../DECISIONS/SMA-2-driver-seam.md | 27 ++++++---- goodwe_controller/app/messages.py | 10 ---- goodwe_controller/app/uilayout.py | 13 ----- goodwe_controller/app/validate.py | 14 ++++-- goodwe_controller/config.yaml | 21 ++------ goodwe_controller/test_inverter.py | 16 +++--- goodwe_controller/test_sma2_driver.py | 43 +++++++--------- goodwe_controller/test_translations.py | 4 +- goodwe_controller/test_uilayout.py | 4 +- .../tools/supervisor_shape_probe.py | 49 +++++++------------ goodwe_controller/translations/en.yaml | 15 ------ goodwe_controller/translations/nl.yaml | 15 ------ 13 files changed, 79 insertions(+), 154 deletions(-) diff --git a/goodwe_controller/CHANGELOG.md b/goodwe_controller/CHANGELOG.md index 63bf561..0af8a9f 100644 --- a/goodwe_controller/CHANGELOG.md +++ b/goodwe_controller/CHANGELOG.md @@ -2,7 +2,7 @@ ### Not in a release yet -- Each inverter is now reached through a driver, so another make can be added later. Nothing changes for a GoodWe site. The inverter list also accepts five new optional settings that nothing reads yet, and the schema no longer requires a setpoint entity per entry: a GoodWe entry without one is still refused at start ([[SMA-2]]). +- Each inverter is now reached through a driver, so another make can be added later. Nothing changes for a GoodWe site ([[SMA-2]]). - Home now draws today's charge from the grid for every hour the add-on will really charge, not only the run under way ([[UX-77]]). - When Home Assistant passes readings on late, control no longer swings on them: it waits until the battery shows its last power request, and the peak limit no longer asks for full power on them. The page says so, and if the readings stay too late every inverter goes to 0 W ([[SAFETY-128]]). - With a 5 s meter, readings 30 s late are now noticed within 100 seconds, not 3 minutes ([[SAFETY-129]]). diff --git a/goodwe_controller/DECISIONS/SMA-2-driver-seam.md b/goodwe_controller/DECISIONS/SMA-2-driver-seam.md index f904173..a6777c0 100644 --- a/goodwe_controller/DECISIONS/SMA-2-driver-seam.md +++ b/goodwe_controller/DECISIONS/SMA-2-driver-seam.md @@ -30,19 +30,26 @@ one unit named twice is the SAFETY-28 refusal. The census still counts every with 23ff9d0 on 349 option shapes is a test). The driver binds its Home Assistant on first use by the Controller, so a unit added later is wired too. -**Schema.** The brief says "`setpoint_entity` becomes optional" (SMA-2, "Schema per -inverter entry"; SMA-9 section 1.1): it is the GoodWe driver's own key, so a -driver reached by host and port has none. `setpoint_entity` is optional per entry -in config.yaml; rule 9(d) still refuses a GoodWe entry without it, with the same -sentence (`test_sma2_driver.py` part 6), beside `driver` (only `goodwe_esphome`), -`host`, `port`, `unit_id`, `maintenance`. The last four are checked for shape -and not read. The Supervisor shape probe was re-run (section K). +**Schema, and why the new keys are not in it.** The brief asks for +`driver`, `host`, `port`, `unit_id`, `maintenance` per entry and for +`setpoint_entity` to become optional. Owner decision 2026-11-07: clients see +nothing new until SMA-4/SMA-7 use the keys. The Supervisor form shows every +schema key (and marks required ones), and a nested entry with a key the schema +lacks has that key silently dropped (measured, shape probe section K), so the +only way to hide them is to keep them out of config.yaml. So for now config.yaml, the +translations and the labels are exactly 23ff9d0's, and `setpoint_entity` stays +required there: until a driver without one is selectable, no entry can lack it. +The code underneath is the brief's: the driver defaults to `goodwe_esphome`, +rule 9(d) is per driver (a GoodWe entry without a setpoint entity is refused, +same sentence), and rule 9(h) checks `driver`, `host`, `port`, `unit_id`, +`maintenance` when a caller that skips the schema (the panel's save path, +sim/inject, the checks) supplies them; those sentences name the key, having no +label. When SMA-4 needs them: add them to the schema as optional with +translations, make `setpoint_entity` `str?`, and add them to `uilayout` as card +fields or a hidden list. **Checked against 23ff9d0:** replay digests and a log of every Home Assistant call are byte-identical at N=1 and N=2; the TEL-07 announce diff is 0/0/0; a fake driver beside a GoodWe unit allocates, grades, drops and publishes per unit; a call site that goes round the seam is caught. -**Per-entry keys the page does not draw** are listed in `uilayout.ROSTER_HIDDEN`; -test_uilayout and test_sma2_driver fail for a key that is neither a card field -nor hidden, and a save through the page keeps them (a test). diff --git a/goodwe_controller/app/messages.py b/goodwe_controller/app/messages.py index d6d1bbe..831da93 100644 --- a/goodwe_controller/app/messages.py +++ b/goodwe_controller/app/messages.py @@ -1683,11 +1683,6 @@ LABELS = { "capacity_kwh": "Usable battery capacity", "inverters": "Inverter list", "name": "Name", - "driver": "Inverter type", - "host": "Inverter address", - "port": "Inverter port", - "unit_id": "Inverter unit number", - "maintenance": "Battery maintenance", "meter_source": "Where the meter reading comes from", "meter_phases": "Number of phases", "p1_host": "HomeWizard P1 address", @@ -1769,11 +1764,6 @@ LABELS_NL = { "capacity_kwh": "Bruikbare capaciteit", "inverters": "Lijst met omvormers", "name": "Naam", - "driver": "Type omvormer", - "host": "Adres van de omvormer", - "port": "Poort van de omvormer", - "unit_id": "Eenheidsnummer van de omvormer", - "maintenance": "Onderhoud van de batterij", "meter_source": "Waar de meterstand vandaan komt", "meter_phases": "Aantal fasen", "p1_host": "Adres van de HomeWizard P1", diff --git a/goodwe_controller/app/uilayout.py b/goodwe_controller/app/uilayout.py index d19bf9f..421dac7 100644 --- a/goodwe_controller/app/uilayout.py +++ b/goodwe_controller/app/uilayout.py @@ -302,19 +302,6 @@ DERIVED = { HIDDEN: dict = {"price_service_token": "a secret, set on the Configuration tab"} -# SMA-2. Per-entry keys of the inverter list the page never shows. They pick and -# address an inverter driver, only `goodwe_esphome` is selectable, and nothing -# reads the other four yet. An entry's keys the page does not draw travel -# through a save as stored (uimodel._roster_decode keeps the whole entry). -ROSTER_HIDDEN: dict = { - "driver": "only goodwe_esphome for now; empty means that", - "host": "not read yet", - "port": "not read yet", - "unit_id": "not read yet", - "maintenance": "not read yet", -} - - def fields(): """Every field once, in page order, roster fields after `inverters`.""" for group in GROUPS: diff --git a/goodwe_controller/app/validate.py b/goodwe_controller/app/validate.py index 9d9dec4..1993187 100644 --- a/goodwe_controller/app/validate.py +++ b/goodwe_controller/app/validate.py @@ -814,6 +814,12 @@ def _whole(raw, lo: int, hi: int): def _driver_problems(entry: dict, who: str) -> list[str]: """Rule 9(h) (SMA-2): the keys that choose and address an inverter driver. + NOT IN config.yaml's SCHEMA, so the Supervisor form and its validator never + see them (owner decision 2026-11-07: clients see nothing new until SMA-4). + They can still reach this rule from callers that skip the schema (the + panel's save path, sim/inject, the checks), so they are graded here, and + named by key because they have no label. + Only `goodwe_esphome` is selectable for now. `host`, `port`, `unit_id` and `maintenance` are accepted and checked for shape, and nothing reads them yet: a later driver's entry must not be refused for carrying them. @@ -822,21 +828,21 @@ def _driver_problems(entry: dict, who: str) -> list[str]: raw = entry.get("driver") if raw not in (None, "") and str(raw).strip() not in drivers.SELECTABLE: out.append( - f"{who} has {S('driver')} {messages.shown(raw)}. Use " + f"{who} has `inverters.driver` {messages.shown(raw)}. Use " f"{', '.join(drivers.SELECTABLE)}, or leave it empty.") host = entry.get("host") if host not in (None, "") and (not isinstance(host, str) or re.search(r"\s", host.strip())): - out.append(f"{who} has {S('host')} {messages.shown(host)}, which is " + out.append(f"{who} has `inverters.host` {messages.shown(host)}, which is " "not an address. Use a name or an IP address.") for key, lo, hi in (("port", 1, 65535), ("unit_id", 0, 247)): val = entry.get(key) if val not in (None, "") and _whole(val, lo, hi) is None: - out.append(f"{who} has {S(key)} {messages.shown(val)}. Use a " + out.append(f"{who} has `inverters.{key}` {messages.shown(val)}. Use a " f"whole number from {lo} to {hi}.") maint = entry.get("maintenance") if maint is not None and not isinstance(maint, bool): - out.append(f"{who} has {S('maintenance')} {messages.shown(maint)}. " + out.append(f"{who} has `inverters.maintenance` {messages.shown(maint)}. " "Use on or off.") return out diff --git a/goodwe_controller/config.yaml b/goodwe_controller/config.yaml index 1754991..b02e072 100644 --- a/goodwe_controller/config.yaml +++ b/goodwe_controller/config.yaml @@ -942,17 +942,11 @@ schema: # batteries, whereas an absent capacity has an honest answer - no opinion # about the split, every unit weighs 1.0, which is what shipped. # - # SMA-2. `driver` empty means goodwe_esphome, the only choice for now. - # `host`, `port`, `unit_id` and `maintenance` are accepted and checked for - # shape; nothing reads them yet (DECISIONS/SMA-2-driver-seam.md). - # - # ⚠️ THREE KEYS ARE REQUIRED PER ENTRY AND THAT IS DELIBERATE. An entry with + # ⚠️ FOUR KEYS ARE REQUIRED PER ENTRY AND THAT IS DELIBERATE. An entry with # no `name` has no Store namespace and would share another unit's SAFETY-02 - # latch; one with no `max_w` would have to borrow a rating, which is the - # sibling-grading defect DEV-02 removed; `bms_soc_floor` is SAFETY-32's - # (options block above). `setpoint_entity` WAS A FOURTH; since SMA-2 it is the - # GoodWe driver's own key, so a unit nothing can command is refused by - # validate.py rule 9(d) per driver rather than by this schema. + # latch; one with no `setpoint_entity` is a unit nothing can command; one with + # no `max_w` would have to borrow a rating, which is the sibling-grading + # defect DEV-02 removed; `bms_soc_floor` is SAFETY-32's (options block above). # The optional keys are the sensors, whose absence honestly means "this unit # reports none", and `capacity_kwh`. # @@ -964,18 +958,13 @@ schema: # test_inverter.py part 12 asserts the ordering is still hostile. inverters: - name: str - setpoint_entity: str? + setpoint_entity: str max_w: int(100,5000) bms_soc_floor: int(0,50) soc_entity: str? batt_entity: str? batt_invert: bool? capacity_kwh: float(0.1,200)? - driver: list(goodwe_esphome)? - host: str? - port: port? - unit_id: int(0,247)? - maintenance: bool? estop_fitted: bool battery_modes_enabled: bool diff --git a/goodwe_controller/test_inverter.py b/goodwe_controller/test_inverter.py index 1fe17cb..4b7281e 100644 --- a/goodwe_controller/test_inverter.py +++ b/goodwe_controller/test_inverter.py @@ -1536,9 +1536,8 @@ _decl9 = dict(ln.strip().lstrip("- ").split(": ", 1) for ln in _entry9.splitlines()[1:] if ": " in ln) print(f" the shipped per-entry schema: {_decl9}") check("...and the schema half declares it as a LIST OF OBJECTS whose `name`, " - "`max_w` and `bms_soc_floor` are REQUIRED - no " - "trailing `?` (SMA-2: `setpoint_entity` is the GoodWe driver's own key " - "and optional here; rule 9(d) requires it per driver) - so the Supervisor refuses a half-written entry before the " + "`setpoint_entity`, `max_w` and `bms_soc_floor` are REQUIRED - no " + "trailing `?` - so the Supervisor refuses a half-written entry before the " "add-on sees it, and an entry can never fall back on a sibling's rating " "or on a sibling's BMS cut-out because there is no entry without one. " "⚠️ SAFETY-32 ADDED THE FOURTH ONE AS REQUIRED RATHER THAN AS " @@ -1554,14 +1553,13 @@ check("...and the schema half declares it as a LIST OF OBJECTS whose `name`, " "refuses every options blob written before it existed", set(_decl9) == {"name", "setpoint_entity", "max_w", "bms_soc_floor", "soc_entity", "batt_entity", "batt_invert", - "capacity_kwh", "driver", "host", "port", "unit_id", - "maintenance"} + "capacity_kwh"} and not any(_decl9[k].endswith("?") - for k in ("name", "max_w", "bms_soc_floor")) + for k in ("name", "setpoint_entity", "max_w", + "bms_soc_floor")) and all(_decl9[k].endswith("?") - for k in ("setpoint_entity", "soc_entity", "batt_entity", - "batt_invert", "capacity_kwh", "driver", "host", - "port", "unit_id", "maintenance"))) + for k in ("soc_entity", "batt_entity", "batt_invert", + "capacity_kwh"))) # ⚠️ NOTHING IS INHERITED, AND THIS IS DEV-02 RESTATED AS A CONFIG RULE. # ⚠️ THE FLAT KEYS ARE SET TO REAL ENTITIES HERE, AND THAT IS THE WHOLE CHECK. diff --git a/goodwe_controller/test_sma2_driver.py b/goodwe_controller/test_sma2_driver.py index e98b296..42b57fa 100644 --- a/goodwe_controller/test_sma2_driver.py +++ b/goodwe_controller/test_sma2_driver.py @@ -592,22 +592,29 @@ ent = sch[sch.index("\n inverters:"):] ent = ent[:ent.index("\n\n")] decl = dict(ln.strip().lstrip("- ").split(": ", 1) for ln in ent.splitlines()[1:] if ": " in ln) -check("setpoint_entity is optional per entry, beside driver, host, port, " - "unit_id and maintenance; name, max_w and bms_soc_floor stay required", - decl["setpoint_entity"] == "str?" and decl["driver"] == "list(goodwe_esphome)?" - and decl["host"] == "str?" and decl["port"] == "port?" - and decl["unit_id"] == "int(0,247)?" and decl["maintenance"] == "bool?" - and not any(decl[k].endswith("?") for k in ("name", "max_w", - "bms_soc_floor"))) -check("config.yaml lists exactly the selectable drivers", - decl["driver"] == f"list({'|'.join(SELECTABLE)})?") +NEW_KEYS = ("driver", "host", "port", "unit_id", "maintenance") +check("HIDDEN FROM CLIENTS (owner, 2026-11-07): config.yaml's per-entry schema " + "is what it was on 23ff9d0 - the five driver keys are not in it, and " + "setpoint_entity is still required - so the Supervisor form shows " + "nothing new", + not set(NEW_KEYS) & set(decl) and not decl["setpoint_entity"].endswith("?") + and set(decl) == {"name", "setpoint_entity", "max_w", "bms_soc_floor", + "soc_entity", "batt_entity", "batt_invert", + "capacity_kwh"}) +_tr = {lg: io.open(HERE / "translations" / f"{lg}.yaml", encoding="utf-8").read() + for lg in ("en", "nl")} +from app import messages as _msgs # noqa: E402 +check("...and no translation, label or description was added for them", + not any(f" {k}:\n" in t.split(" estop_fitted:")[0].split(" inverters:")[1] + for t in _tr.values() for k in NEW_KEYS) + and not set(NEW_KEYS) & (set(_msgs.LABELS) | set(_msgs.LABELS_NL))) # ============================================================================== print() print("6. round 2: census parity, setters, rule 9(d), NaN ports, hidden keys") import copy, itertools, math # noqa: E402 -from app import uilayout as L, uimodel # noqa: E402 +from app import uimodel # noqa: E402 from app.validate import ACTUATOR_WRITE_KEYS # noqa: E402 @@ -708,25 +715,13 @@ for key in ("port", "unit_id"): got, ok_ = repr(e), False check(f"{key} = {nm} is refused by key and does not raise", ok_) -keys_cfg = set(decl) -placed = {k.split(".", 1)[1] for _g, _p, f in L.fields() - for k in [f.key] if k.startswith("inverters.")} -check("ROSTER_HIDDEN: every per-entry key in config.yaml is either a card " - "field or hidden, never both and never neither", - all((k in placed) != (k in L.ROSTER_HIDDEN) for k in keys_cfg) - and not (placed & set(L.ROSTER_HIDDEN)) - and set(L.ROSTER_HIDDEN) <= keys_cfg) -check("...and a per-entry key that is neither would be flagged (test_uilayout " - "grades the same rule with a made-up key)", - not all((k in placed) != (k in L.ROSTER_HIDDEN) - for k in keys_cfg | {"brand_new_key"})) ent = dict(name="a", setpoint_entity="number.a", max_w=3000, bms_soc_floor=10, soc_entity="sensor.s", batt_entity="sensor.b", driver="goodwe_esphome", host="h", port=502, unit_id=3, maintenance=False) opts_rt = {"inverters": [ent, dict(ent, name="b", setpoint_entity="number.b")]} back = uimodel.ui_to_options(uimodel.options_to_ui(copy.deepcopy(opts_rt))) -check("the page model carries the hidden keys through a save untouched " - "(this is how ROSTER_HIDDEN keys survive the settings page)", +check("an entry that carries the driver keys (from a caller that skips the " + "schema) comes back from a save through the page model unchanged", back.get("inverters") == opts_rt["inverters"]) del REGISTRY["fake"] diff --git a/goodwe_controller/test_translations.py b/goodwe_controller/test_translations.py index a05f169..47e1db8 100644 --- a/goodwe_controller/test_translations.py +++ b/goodwe_controller/test_translations.py @@ -75,7 +75,7 @@ check(f"the option surface was actually read - {len(ORDER)} options, not zero", check("...and the repeatable `inverters:` section's per-entry keys were read " "too, which is the half a top-level-only reader would miss", NESTED.get("inverters", [])[:2] == ["name", "setpoint_entity"] - and len(NESTED["inverters"]) == 13) # SMA-2: +5 driver keys + and len(NESTED["inverters"]) == 8) print("\n2. AC: every option and every per-entry key is translated, in BOTH " "languages") @@ -281,7 +281,7 @@ check("...and the accepted subset really does round-trip the shipped files - " f"per-entry keys came back out of each, which is what says the " f"strictness above is not just refusing everything", all(len(TR[lg]["configuration"]) == len(ORDER) - and len(TR[lg]["configuration"]["inverters"]["fields"]) == 13 + and len(TR[lg]["configuration"]["inverters"]["fields"]) == 8 for lg in LANGS)) print(" ⚠️ the licence for this hand parser is NOT this file. " "tools/supervisor_shape_probe.py section F loads the same two files with " diff --git a/goodwe_controller/test_uilayout.py b/goodwe_controller/test_uilayout.py index ab68084..70c4bdb 100644 --- a/goodwe_controller/test_uilayout.py +++ b/goodwe_controller/test_uilayout.py @@ -51,10 +51,8 @@ def coverage_problems(order, nested): out += [f"{k} is not in config.yaml" for k in set(placed) | set(L.DERIVED) | set(L.HIDDEN) if k not in order] sub = [k.split(".", 1)[1] for k in KEYS if k.startswith("inverters.")] - # SMA-2: a per-entry key may instead be listed in L.ROSTER_HIDDEN (the - # driver keys the page does not draw), and then it must not also be a field. out += [f"inverters.{k} has no card field" for k in nested.get("inverters", []) - if sub.count(k) + (k in L.ROSTER_HIDDEN) != 1] + if sub.count(k) != 1] out += [f"inverters.{k} is not in config.yaml" for k in sub if k not in nested.get("inverters", [])] return out diff --git a/goodwe_controller/tools/supervisor_shape_probe.py b/goodwe_controller/tools/supervisor_shape_probe.py index c370844..6e1b1f4 100644 --- a/goodwe_controller/tools/supervisor_shape_probe.py +++ b/goodwe_controller/tools/supervisor_shape_probe.py @@ -243,6 +243,8 @@ show("the per-entry BASELINE is ACCEPTED, so every refusal below has nowhere " "else to come from", ok, json.dumps(out.get("inverters")) if ok else out) for title, entry, expect in ( ("no name", {k: v for k, v in D_FULL.items() if k != "name"}, "name"), + ("no setpoint_entity", + {k: v for k, v in D_FULL.items() if k != "setpoint_entity"}, "setpoint_entity"), ("no max_w", {k: v for k, v in D_FULL.items() if k != "max_w"}, "max_w"), ("no bms_soc_floor (SAFETY-32)", {k: v for k, v in D_FULL.items() if k != "bms_soc_floor"}, "bms_soc_floor"), @@ -255,39 +257,22 @@ for title, entry, expect in ( ok, out = validate(dict(D_BASE, inverters="[]")) show("a JSON STRING where the list belongs is refused", not ok, out) -print("\n== K. SMA-2: the driver keys on an inverter entry ==") -# `setpoint_entity` is OPTIONAL per entry since SMA-2 (it is the GoodWe driver's -# own key; validate.py rule 9(d) requires it for GoodWe), beside `driver`, -# `host`, `port`, `unit_id` and `maintenance`. Same method as §D: an ACCEPTED -# baseline, ONE value varied, the refusal asserted on the KEY'S NAME. -K_FULL = dict(D_FULL, driver="goodwe_esphome", host="inverter.local", port=502, - unit_id=1, maintenance=False) -ok, out = validate(dict(D_BASE, inverters=[K_FULL])) -show("an entry carrying every new key is ACCEPTED, and the keys survive", - ok and all(out["inverters"][0].get(k) == K_FULL[k] - for k in ("driver", "host", "port", "unit_id", "maintenance")), - json.dumps(out.get("inverters")) if ok else out) +print("\n== K. SMA-2: the driver keys are NOT in the schema (owner, 2026-11-07) ==") +# `driver`, `host`, `port`, `unit_id` and `maintenance` are graded by validate.py +# rule 9(h) but kept out of config.yaml so the Configuration tab shows nothing +# new. This pins both halves: the shipped entry is the one it always was, and the +# Supervisor (measured) silently DROPS an unknown key from an entry, so they +# cannot arrive by the form or by YAML. ok, out = validate(dict(D_BASE, inverters=[D_FULL])) -show("an entry from before SMA-2 (none of the new keys) is ACCEPTED unchanged", - ok and out["inverters"] == [D_FULL], out) -ok, out = validate(dict(D_BASE, inverters=[ - {k: v for k, v in D_FULL.items() if k != "setpoint_entity"}])) -show("an entry with NO setpoint_entity is now accepted by the schema " - "(rule 9(d) refuses it for GoodWe, at startup)", ok, out) -# ⚠️ THE SUPERVISOR'S REFUSAL NAMES THE VALUE'S RULE, NOT THE KEY, FOR A NESTED -# ENTRY (§D's "at most 5000"), so the expectation is that rule's own words. -for title, entry, expect in ( - ("driver other than goodwe_esphome", dict(K_FULL, driver="sma"), - "one of ['goodwe_esphome']"), - ("port 0", dict(K_FULL, port=0), "at least 1"), - ("port 70000", dict(K_FULL, port=70000), "at most 65535"), - ("unit_id 248", dict(K_FULL, unit_id=248), "at most 247"), - ("host a number", dict(K_FULL, host=5), "expected str"), - ("maintenance a word that is not a boolean", - dict(K_FULL, maintenance="banana"), "boolean")): - ok, out = validate(dict(D_BASE, inverters=[entry])) - show(f"an entry with {title} is refused, with the rule's own words", - (not ok) and expect in str(out), out) +show("the pre-SMA-2 entry is ACCEPTED unchanged", ok and out["inverters"] == [D_FULL], out) +for _k in ("driver", "host", "port", "unit_id", "maintenance"): + ok, out = validate(dict(D_BASE, inverters=[dict(D_FULL, **{_k: "x"})])) + show(f"an entry carrying `{_k}` is accepted and the key is dropped", + ok and out["inverters"] == [D_FULL], out) +show("the per-entry schema has exactly the eight keys it had", + sorted(schema["inverters"][0]) == sorted(["name", "setpoint_entity", "max_w", + "bms_soc_floor", "soc_entity", "batt_entity", "batt_invert", "capacity_kwh"]), + json.dumps(schema["inverters"])) print("\n== E. the ROLLBACK direction, and the operator's UI ==") diff --git a/goodwe_controller/translations/en.yaml b/goodwe_controller/translations/en.yaml index a23daba..d390d74 100644 --- a/goodwe_controller/translations/en.yaml +++ b/goodwe_controller/translations/en.yaml @@ -79,21 +79,6 @@ configuration: capacity_kwh: name: "Usable battery capacity (kWh) (optional)" description: "This battery's usable capacity from its datasheet. Fill it in on every inverter or on none. The add-on then shares the work by the energy each battery has left." - driver: - name: "Inverter type (optional)" - description: "The kind of inverter. Leave it empty for a GoodWe." - host: - name: "Inverter address (optional)" - description: "Not used yet. Leave it empty." - port: - name: "Inverter port (optional)" - description: "Not used yet. Leave it empty." - unit_id: - name: "Inverter unit number (optional)" - description: "Not used yet. Leave it empty." - maintenance: - name: "Battery maintenance (optional)" - description: "Not used yet. Leave it empty." estop_fitted: name: "Hardware · RS485 emergency stop is fitted" description: "Switch this on if the emergency stop is installed. It only changes a warning on the status page." diff --git a/goodwe_controller/translations/nl.yaml b/goodwe_controller/translations/nl.yaml index 0eecb3b..3db9bd5 100644 --- a/goodwe_controller/translations/nl.yaml +++ b/goodwe_controller/translations/nl.yaml @@ -60,21 +60,6 @@ configuration: capacity_kwh: name: "Bruikbare capaciteit (kWh) (optioneel)" description: "De bruikbare capaciteit van deze batterij, van het datablad. Vul ze bij elke omvormer in of bij geen. Dan verdeelt de add-on het werk volgens de energie in elke batterij." - driver: - name: "Type omvormer (optioneel)" - description: "Het soort omvormer. Laat leeg voor een GoodWe." - host: - name: "Adres van de omvormer (optioneel)" - description: "Nog niet in gebruik. Laat leeg." - port: - name: "Poort van de omvormer (optioneel)" - description: "Nog niet in gebruik. Laat leeg." - unit_id: - name: "Eenheidsnummer van de omvormer (optioneel)" - description: "Nog niet in gebruik. Laat leeg." - maintenance: - name: "Onderhoud van de batterij (optioneel)" - description: "Nog niet in gebruik. Laat leeg." estop_fitted: name: "Hardware · RS485-noodstop is geplaatst" description: "Zet dit aan als de noodstop geïnstalleerd is. Het verandert alleen een waarschuwing op de statuspagina." -- 2.54.0 From a1be9c48b3eda06fe4ed8b8d6143999090bec1d3 Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Sat, 7 Nov 2026 18:15:05 +0100 Subject: [PATCH 16/22] SMA-10: two idle states, hold 0 W or hand the inverter back A driver now says whether its release() hands the inverter back (Driver.hands_back; GoodWe is False, so release() stays write(0) and the Controller never calls it). command_site_release() sits next to command_site_zero(): the Stop button and shutdown() (so SIGTERM, restart, control task death and a late config refusal) hand a hand-back unit back, once, and zero every other unit as before. Safety stops and refusals still hold 0 W for every unit. Shadow mode releases a hand-back unit. After a release nothing is written to the unit (heartbeat and cycle included) until control starts again. release() is bounded by RELEASE_BUDGET_S (= the write budget), runs concurrently per unit, and a raising or hanging release on one unit does not stop the others. The "set to 0 W" sentences for Stop, restart, shadow mode and the stopped banner are built from the roster (app/handback.py, EN and NL, words from the driver); a GoodWe-only roster keeps every old literal. A handed-back unit shows "Back under " on Home. Existing test edited: test_messages.py (two classification entries for the CRITICAL line that moved into a roster builder, and the new release bound). Co-Authored-By: Claude --- goodwe_controller/CHANGELOG.md | 1 + .../DECISIONS/SMA-10-two-idle-states.md | 51 + goodwe_controller/app/drivers.py | 19 +- goodwe_controller/app/handback.py | 386 +++++++ goodwe_controller/app/homeview.py | 12 +- goodwe_controller/app/inverter.py | 9 + goodwe_controller/app/main.py | 165 ++- goodwe_controller/app/messages.py | 33 +- goodwe_controller/app/uitext.py | 31 +- goodwe_controller/app/web.py | 3 +- goodwe_controller/test_messages.py | 9 +- .../test_sma10_two_idle_states.py | 1023 +++++++++++++++++ 12 files changed, 1720 insertions(+), 22 deletions(-) create mode 100644 goodwe_controller/DECISIONS/SMA-10-two-idle-states.md create mode 100644 goodwe_controller/app/handback.py create mode 100644 goodwe_controller/test_sma10_two_idle_states.py diff --git a/goodwe_controller/CHANGELOG.md b/goodwe_controller/CHANGELOG.md index 0af8a9f..5d02a9f 100644 --- a/goodwe_controller/CHANGELOG.md +++ b/goodwe_controller/CHANGELOG.md @@ -2,6 +2,7 @@ ### Not in a release yet +- When the add-on stops, an inverter that can steer itself is handed back instead of held at 0 W. Safety stops still hold 0 W, and a GoodWe site is not affected ([[SMA-10]]). - Each inverter is now reached through a driver, so another make can be added later. Nothing changes for a GoodWe site ([[SMA-2]]). - Home now draws today's charge from the grid for every hour the add-on will really charge, not only the run under way ([[UX-77]]). - When Home Assistant passes readings on late, control no longer swings on them: it waits until the battery shows its last power request, and the peak limit no longer asks for full power on them. The page says so, and if the readings stay too late every inverter goes to 0 W ([[SAFETY-128]]). diff --git a/goodwe_controller/DECISIONS/SMA-10-two-idle-states.md b/goodwe_controller/DECISIONS/SMA-10-two-idle-states.md new file mode 100644 index 0000000..778b1b6 --- /dev/null +++ b/goodwe_controller/DECISIONS/SMA-10-two-idle-states.md @@ -0,0 +1,51 @@ +# SMA-10 - two idle states: hold 0 W, or hand the inverter back + +**Status: for review.** Tests: `test_sma10_two_idle_states.py`. Source: SMA-9 +report sections 1.4, 3.2; owner decision 2026-10-20 (Q1, "as proposed"). + +A unit is idle in one of two ways, and the driver says which: +`Driver.hands_back`. **False (GoodWe):** the add-on holds 0 W, `release()` is +`write(0)` and the Controller never calls it, so Stop keeps today's 0 W write and +today's 0 W heartbeat. **True:** `release()` gives the unit back to whatever +steers it by itself (SMA energy management), once, and then nothing is written +to it until control starts again. + +**Why two.** When the add-on goes away (Stop, shutdown, SIGTERM, restart, task +death, a config refusal) nobody is left to hold a number, so the unit goes back +to its own controller. When the add-on itself decides to stop (sign trip, site +sign, circulating power, a refused or non-finite write, missing readings, +another writer, an inverter fault) it has just said it cannot trust that +battery, and handing it to something else would let that something do what it +likes with it. Those hold 0 W through `write(0)` for every unit, whatever its +driver. Shadow mode on a unit that hands back uses `release()`: it must leave +the unit alone, and 0 W would be a command. + +**Where.** `Controller.command_site_release()` sits beside `command_site_zero()` +(unchanged). Stop and `shutdown()` call it; the two maintenance-abort paths keep +zero (the add-on runs on, the sentence says 0 W for a moment). With no +hand-back unit it is `command_site_zero()`. `_write_unit()` also refuses to write +a hand-back unit while the add-on is idle (stopping, stopped, or shadow mode): it +releases once and returns, so the heartbeat and the cycle's own "stopped" 0 W +cannot reach it. A safety stop runs while control runs, so it falls through to +the write. Start re-arms: `released` drops and `last_write` clears. + +**Bounds.** `RELEASE_BUDGET_S` equals `WRITE_BUDGET_S` (4 s): units are released +together, so one slow unit costs the kill window no more than a hung GoodWe +write does. A release at the bound is abandoned, not awaited. A release that +raises, hangs or returns false leaves that unit un-handed (the next idle write +tries again) and does not touch the others. Exit 78 before the Controller +exists has no driver to release and keeps today's path. + +**Words.** Every sentence that said "set to 0 W" for the add-on going away or +for shadow mode is built from the roster in `app/handback.py` (EN and NL): a +roster with no hand-back unit gets None and keeps its old literal, so a GoodWe +site sees no changed byte. A hand-back unit's words (`Driver.words`: `back`, +`back_nl`) come from its driver. MQTT `status` is unchanged; the Home battery +row of a handed-back unit says "Back under SMA control" and carries +`handed_back`. + +**Not done here (SMA-4, SMA-5, SMA-11):** any SMA/FoxESS driver, the read-back +after `release()`, per-driver timing, DOCS.md's safety model and the +"control stalled" sentence, the site-level "Held at 0 W" tiles in shadow mode on +a mixed roster, and the MQTT per-unit setpoint of a handed-back unit (it still +publishes 0). diff --git a/goodwe_controller/app/drivers.py b/goodwe_controller/app/drivers.py index 98d616e..915a891 100644 --- a/goodwe_controller/app/drivers.py +++ b/goodwe_controller/app/drivers.py @@ -77,7 +77,13 @@ class Driver: writer_key = "" #: the brand word the per-driver sentences and device names use brand = "" - #: per-driver sentences (SMA-10 and later). Empty until a caller needs one. + #: SMA-10. Does `release()` give the inverter BACK to something that steers + #: it by itself (SMA energy management)? False: `release()` is `write(0)`, + #: the add-on goes on holding 0 W, and the Controller never calls it. + hands_back = False + #: per-driver sentences. A driver that hands back says whose control the + #: unit returns to: `{"back": "SMA control", "back_nl": "de sturing van SMA"}` + #: ("goes back to SMA control"). Missing words fall back to a plain phrase. words: dict = {} #: how long all-rejected writes last before the unit stops being live link_stale_s = GOODWE_LINK_STALE_S @@ -145,7 +151,13 @@ class Driver: raise NotImplementedError async def release(self) -> bool: - """Stop steering this inverter. For a driver with nothing better, 0 W.""" + """Stop steering this inverter. For a driver with nothing better, 0 W. + + SMA-10. Only a driver with `hands_back = True` has this called by the + Controller, and only when the add-on goes away (Stop, shutdown, restart, + task death) or sits in shadow mode. After it the Controller writes + nothing more to the unit until control starts again. A safety stop is + never a release: that is `write(0)`.""" return await self.write(0.0) async def probe(self) -> Probe: @@ -159,7 +171,8 @@ class GoodWeEsphomeDriver(Driver): A transcription of what `main.py` and `selfconfig.py` did inline: `limits()` is `hass.limits(setpoint)`, `read()` is `hass.number(soc)` then `hass.number(batt, invert)`, `write()` is `hass.set_number(setpoint, w)`. - `release()` is `write(0)`: the board's own watchdog is what hands back. + `release()` is `write(0)` and `hands_back` is False: the board's own watchdog + is what hands back, so after Stop the unit keeps its 0 W write and heartbeat. """ name = DEFAULT_DRIVER diff --git a/goodwe_controller/app/handback.py b/goodwe_controller/app/handback.py new file mode 100644 index 0000000..b0c5a74 --- /dev/null +++ b/goodwe_controller/app/handback.py @@ -0,0 +1,386 @@ +"""SMA-10: the two idle states, in words. + +A unit is either HELD at 0 W (GoodWe: the add-on goes on writing 0 W) or +HANDED BACK (its driver has `hands_back = True`: `release()` gives it back to +whatever steers it by itself, and the add-on then writes nothing to it). So every +sentence that said "every inverter is set to 0 W" for the add-on going away or +for shadow mode has to be built from the roster. DECISIONS/SMA-10-two-idle- +states.md. + +THE RULE FOR EVERY BUILDER HERE: a roster with no hand-back unit gets None, and +the caller then uses the sentence it always used, untouched. So a GoodWe-only +site cannot see a changed byte; test_sma10_two_idle_states.py renders each one +and compares it with the literal from before this card. + +The words for a hand-back unit come from its driver (`Driver.words`), not from +here: `back` is "SMA control" in "goes back to SMA control", `back_nl` is the +Dutch noun phrase after "terug naar". A driver without them gets a plain, +brand-free phrase. + +Safety stops and refusals are not in this file on purpose: they hold 0 W for +every unit, and their sentences say so. +""" + +from __future__ import annotations + +import re +from dataclasses import dataclass + +_BACK_EN = "its own control" +_BACK_NL = "eigen sturing" + + +@dataclass(frozen=True) +class Unit: + name: str + hands_back: bool = False + back_en: str = _BACK_EN + back_nl: str = _BACK_NL + + +def unit_of(inv) -> Unit: + """One roster entry's view: its name, and (for a hand-back driver) the + words of whose control it returns to.""" + drv = inv.driver + back = bool(getattr(drv, "hands_back", False)) + words = getattr(drv, "words", None) or {} + return Unit(str(inv.name), back, + str(words.get("back") or _BACK_EN), + str(words.get("back_nl") or _BACK_NL)) + + +def units_of(inverters) -> tuple: + return tuple(unit_of(i) for i in inverters) + + +# The roster of the Controller that is running, for the callers that have no +# Controller in hand (the page, the banner). One process runs one Controller. +_CURRENT: list = [()] + + +def set_roster(units) -> None: + _CURRENT[0] = tuple(units) + + +def current() -> tuple: + return _CURRENT[0] + + +def active(units) -> bool: + """Is any unit handed back? False is every GoodWe-only roster.""" + return any(u.hands_back for u in units) + + +# --------------------------------------------------------------------------- # +# The grammar. English and Dutch, a few verb forms each. +# --------------------------------------------------------------------------- # +def _join(names, lang) -> str: + names = list(names) + word = " en " if lang == "nl" else " and " + if len(names) <= 1: + return "".join(names) + return ", ".join(names[:-1]) + word + names[-1] + + +def _forms(lang): + """kind -> (zero template, back template); `{n}` the names, `{b}` the words. + `one`/`many` pick the verb by how many units the clause is about.""" + if lang == "nl": + return { + "set": (("{n} gaat naar 0 W", "{n} gaan naar 0 W"), + ("{n} gaat terug naar {b}", "{n} gaan terug naar {b}")), + "at": (("{n} op 0 W",) * 2, ("{n} onder {b}",) * 2), + "is": (("{n} staat op 0 W", "{n} staan op 0 W"), + ("{n} staat onder {b}", "{n} staan onder {b}")), + "stays": (("{n} blijft op 0 W", "{n} blijven op 0 W"), + ("{n} blijft onder {b}", "{n} blijven onder {b}")), + } + return { + "set": (("{n} is set to 0 W", "{n} are set to 0 W"), + ("{n} goes back to {b}", "{n} go back to {b}")), + "at": (("{n} at 0 W",) * 2, ("{n} under {b}",) * 2), + "is": (("{n} is at 0 W", "{n} are at 0 W"), + ("{n} is under {b}", "{n} are under {b}")), + "stays": (("{n} stays at 0 W", "{n} stay at 0 W"), + ("{n} stays under {b}", "{n} stay under {b}")), + } + + +def _groups(units, lang): + """`(zero names, [(names, back words)])`: the held units, and the handed-back + ones grouped by whose control they return to, in roster order.""" + zeros = [u.name for u in units if not u.hands_back] + groups: dict = {} + for u in units: + if u.hands_back: + groups.setdefault(u.back_nl if lang == "nl" else u.back_en, + []).append(u.name) + return zeros, [(names, back) for back, names in groups.items()] + + +# A clause may not eat the sentence it sits in: STYLE.md allows 25 words, and +# the longest tail after a clause is about 11. +CLAUSE_MAX_WORDS = 14 + + +def _count_txt(n, lang) -> str: + if lang == "nl": + return f"{n} omvormer" if n == 1 else f"{n} omvormers" + return f"{n} inverter" if n == 1 else f"{n} inverters" + + +def _clause(units, lang, kind, sep, generic, counted) -> str: + forms = _forms(lang)[kind] + zeros, groups = _groups(units, lang) + if generic: # one group, and no make named + names = [n for g, _b in groups for n in g] + own = ("de eigen sturing" if lang == "nl" else + "their own control" if len(names) > 1 else "its own control") + groups = [(names, own)] if names else [] + subject = ("de omvormers", "de omvormer") if lang == "nl" else ( + "the inverters", "the inverter") + parts = [] + if zeros: + parts.append(forms[0][1 if len(zeros) > 1 else 0].format( + n=(str(len(zeros)) if groups else _count_txt(len(zeros), lang)) + if counted else _join(zeros, lang))) + for names, back in groups: + if not zeros and len(groups) == 1: + names_txt = subject[0 if len(names) > 1 else 1] + elif counted: + # the bare number after a counted first clause: "1 inverter is set + # to 0 W and 2 go back to their own control" + names_txt = (str(len(names)) if zeros + else _count_txt(len(names), lang)) + else: + names_txt = _join(names, lang) + parts.append(forms[1][1 if len(names) > 1 else 0].format( + n=names_txt, b=back)) + if sep: + return sep.join(parts) + word = " en " if lang == "nl" else " and " + return (", ".join(parts[:-1]) + word + parts[-1]) if len(parts) > 1 \ + else "".join(parts) + + +def clause(units, lang, kind, sep=None) -> str: + """One clause about the whole roster, `kind` one of set/at/is/stays. + + Held units first, then the handed-back ones, joined by "and" (or `sep`). + When EVERY unit is handed back to the same control the subject is "the + inverters", not their names, as the card's table has it. + + A long roster would break the 25-word rule, so a clause over + CLAUSE_MAX_WORDS is said again without naming the make (one "own control" + for all), and then by counting units instead of naming them.""" + text = _clause(units, lang, kind, sep, False, False) + if sep or _words(text) <= CLAUSE_MAX_WORDS: + return text + text = _clause(units, lang, kind, sep, True, False) + if _words(text) <= CLAUSE_MAX_WORDS: + return text + return _clause(units, lang, kind, sep, True, True) + + +def _cap(text) -> str: + return text[:1].upper() + text[1:] + + +def _bi(en, nl): + from .messages import Bi + return Bi(en, nl) + + +def _words(text) -> int: + return len([w for w in re.findall(r"\S+", text) + if re.search(r"[A-Za-z0-9]", w)]) + + +# --------------------------------------------------------------------------- # +# The sentences. Each returns None for a roster with no hand-back unit. +# --------------------------------------------------------------------------- # +def stop_dialog(units, lang): + """Stop control? (was "Every inverter is set to 0 W until you start + control again.")""" + if not active(units): + return None + c = clause(units, lang, "set") + return (f"{_cap(c)} tot u de regeling opnieuw start." if lang == "nl" + else f"{_cap(c)} until you start control again.") + + +def restart_dialog(units, lang): + """Restart now? / Save and restart? (live)""" + if not active(units): + return None + c = clause(units, lang, "set") + return (f"{_cap(c)} terwijl de add-on herstart, daarna gelden de nieuwe " + f"instellingen." if lang == "nl" else + f"{_cap(c)} while the add-on restarts, and then the new settings " + f"apply.") + + +def restart_dialog_shadow(units, lang): + """Save and restart? when shadow mode keeps the units idle afterwards.""" + if not active(units): + return None + c = clause(units, lang, "set") + return (f"{_cap(c)} terwijl de add-on herstart, en de schaduwmodus houdt " + f"het zo." if lang == "nl" else + f"{_cap(c)} while the add-on restarts, and shadow mode keeps it " + f"that way.") + + +def setup_dialog_shadow(units, lang): + """Save and restart? in setup, shadow mode on.""" + if not active(units): + return None + c = clause(units, lang, "at") + return (f"De add-on herstart met deze instellingen, en de schaduwmodus " + f"houdt {c}." if lang == "nl" else + f"The add-on restarts with these settings, and shadow mode keeps " + f"{c}.") + + +def control_stopped(units, lang): + """The page's control line when stopped.""" + if not active(units): + return None + c = clause(units, lang, "is") + return (f"De regeling is gestopt. {_cap(c)}." if lang == "nl" + else f"Control is stopped. {_cap(c)}.") + + +def control_shadow(units, lang): + """The page's control line in shadow mode.""" + if not active(units): + return None + c = clause(units, lang, "stays") + return (f"Schaduwmodus: {c}." if lang == "nl" else f"Shadow mode: {c}.") + + +def attn_shadow_body(units, lang): + """The attention card's body in shadow mode.""" + if not active(units): + return None + c = clause(units, lang, "stays") + return (f"De add-on beslist alles, maar {c}." if lang == "nl" else + f"The add-on decides everything, but {c}.") + + +def shadow_state(units, lang, many=True): + """The tail of a notification: "Shadow mode is on, so ." For the + three output-clamp sentences that said every inverter stays at 0 W.""" + if not active(units): + return None + return clause(units, lang, "stays") + + +def banner_stopped(units): + """Stopped. 12 words at most (STYLE.md).""" + if not active(units): + return None + out = {} + for lang, stopped in (("en", "Stopped"), ("nl", "Gestopt")): + text = f"{stopped} — {clause(units, lang, 'at', sep=', ')}" + if _words(text) > 12: + text = _banner_counts(units, lang, stopped) + out[lang] = text + return _bi(out["en"], out["nl"]) + + +def _banner_counts(units, lang, stopped) -> str: + """The long-roster form: how many, not which.""" + zeros, groups = _groups(units, lang) + if len(groups) == 1: + back = groups[0][1] + n_back = len(groups[0][0]) + if lang == "nl": + held = f"{len(zeros)} op 0 W, " if zeros else "" + return f"{stopped} — {held}{n_back} onder {back}" + held = f"{len(zeros)} at 0 W, " if zeros else "" + return f"{stopped} — {held}{n_back} under {back}" + return (f"{stopped} — omvormers op 0 W of teruggegeven" if lang == "nl" + else f"{stopped} — inverters at 0 W or handed back") + + +def banner_shadow(units): + """Shadow mode, nothing standing. The GoodWe sentence adds "Inverters stay + at 0 W." which is false for a unit that hands back, and a banner is 12 + words, so the unit-by-unit state is left to the control line.""" + if not active(units): + return None + return _bi("Shadow mode: the battery is not controlled.", + "Schaduwmodus: de batterij wordt niet geregeld.") + + +def hint_shadow_on(units): + """"Shadow mode on, inverters stay at 0 W" for the Last event line.""" + if not active(units): + return None + return "Shadow mode on, " + clause(units, "en", "at", sep=", ") + + +def unit_state(unit, lang): + """One handed-back unit's own state, in place of "Held at 0 W".""" + if lang == "nl": + return f"Terug onder {unit.back_nl}" + return f"Back under {unit.back_en}" + + +def log_shutdown(units): + """The INFO line at shutdown (was "shutting down - commanding 0 W on every + inverter").""" + if not active(units): + return None + zeros, groups = _groups(units, "en") + held = f"commanding 0 W on {_join(zeros, 'en')}" if zeros else "" + back = " and ".join(f"handing back {_join(n, 'en')} to {b}" + for n, b in groups) + return "shutting down - " + " and ".join(x for x in (held, back) if x) + + +def task_died_state(units) -> str: + """The part of the CRITICAL line after "so" when the control task dies. + For a roster with no hand-back unit it is the sentence it always was.""" + if not active(units): + return "every inverter is set to 0 W" + zeros, groups = _groups(units, "en") + backs = [n for names, _b in groups for n in names] + parts = [] + if zeros: + parts.append(f"{_join(zeros, 'en')} " + f"{'are' if len(zeros) > 1 else 'is'} set to 0 W") + parts.append(f"{_join(backs, 'en')} " + f"{'are' if len(backs) > 1 else 'is'} handed back") + return " and ".join(parts) + + +# --------------------------------------------------------------------------- # +# The page words: the ui_text sentences that move with the roster, by language. +# --------------------------------------------------------------------------- # +def overlay(units) -> dict: + """`{lang: nested dict}` of ui_text paths to replace; {} when no unit is + handed back, so the page is the file as it always was.""" + if not active(units): + return {} + out = {} + for lang in ("en", "nl"): + out[lang] = { + "home": {"dialog": {"stop": { + "description": stop_dialog(units, lang)}}}, + "settings": { + "dialog": {"description": restart_dialog(units, lang)}, + "setup": { + "dialog": {"description": + restart_dialog_shadow(units, lang)}, + "dialog_live": {"description": + restart_dialog(units, lang)}, + "dialog_setup": {"description": + setup_dialog_shadow(units, lang)}}}, + "home_words": { + "attn_shadow_body": attn_shadow_body(units, lang), + "control": {"shadow": control_shadow(units, lang), + "stopped": control_stopped(units, lang)}}, + } + return out diff --git a/goodwe_controller/app/homeview.py b/goodwe_controller/app/homeview.py index e2812d3..ce644f1 100644 --- a/goodwe_controller/app/homeview.py +++ b/goodwe_controller/app/homeview.py @@ -19,7 +19,7 @@ section at the bottom and is held to STYLE.md by test_messages.py. import contextlib from datetime import datetime, timedelta, timezone -from . import arbitrage, cheaproll, messages, peakguard, priceruns, uitext +from . import arbitrage, cheaproll, handback, messages, peakguard, priceruns, uitext from .dayplan import EFF from .maintenance import CHARGE as M_CHARGE, DRAIN as M_DRAIN, HOLD as M_HOLD from .maintenance import IDLE as M_IDLE, next_slot_start @@ -936,6 +936,13 @@ def battery_rows(c, shadow, running, held=False): else other, at=at, **plan_words(c, inv.name, why)), }) + if getattr(inv, "released", False): + # SMA-10. A unit handed back says so, in its driver's words, not + # "Held at 0 W": nothing is written to it. The key exists only for + # such a unit, so a GoodWe-only page is unchanged. + rows[-1]["now"] = handback.unit_state(handback.unit_of(inv), + _LANG[0]) + rows[-1]["handed_back"] = True if modes_on: # OPT-11 P1: the battery's switch and mode, for the Batteries # screen. Only with battery modes on, so the page is unchanged @@ -1211,6 +1218,9 @@ _WORDS: dict = {} def _hw() -> dict: lang = _LANG[0] + if _WORDS.get("") != uitext.generation(): # SMA-10: the roster's words moved + _WORDS.clear() + _WORDS[""] = uitext.generation() if lang not in _WORDS: _WORDS[lang] = uitext.words(lang) return _WORDS[lang] diff --git a/goodwe_controller/app/inverter.py b/goodwe_controller/app/inverter.py index d5fa66a..69161d0 100644 --- a/goodwe_controller/app/inverter.py +++ b/goodwe_controller/app/inverter.py @@ -275,6 +275,15 @@ class Inverter: self.target = 0.0 self.last_write = None self.last_write_at = None + # SMA-10. True from the moment this unit is handed back (`release()`) + # until control starts again. While it is True nothing is written to the + # unit - not the heartbeat, not the cycle - so the unit's own + # controller keeps what it was given. Only a driver with + # `hands_back = True` ever sets it. + self.released = False + # SMA-10. Writes to a hand-back driver that are on their way (see + # Controller._write_unit), so the release can wait for them and be LAST. + self.inflight: set = set() self.last_write_ok = None # ⚠️ TWO TIMESTAMPS, NOT ONE FLAG. `last_write_ok` is the last write's # verdict and it flips on a single transient 400; liveness needs a RUN diff --git a/goodwe_controller/app/main.py b/goodwe_controller/app/main.py index eb14dd7..a34cc7f 100644 --- a/goodwe_controller/app/main.py +++ b/goodwe_controller/app/main.py @@ -67,6 +67,7 @@ from . import freshness from . import lateness from . import dayview, homeview from .daycurve import SolarToday +from . import handback from . import messages from .freeze import ( FreezeWatchdog, FreezeWatchdogConfig, looks_frozen, rail_unreachable, rail_w, @@ -89,6 +90,7 @@ from . import peakguard from . import packshare from . import pv as pv_read from . import shadow +from . import uitext from .p1 import P1Ingest, build_source, is_enabled from .price import Absence, Uncertainty, brussels, price_option, provider_configured from .price import required_spread as price_required_spread @@ -138,6 +140,13 @@ __all__ = ["UNIT_A", "Controller", "amain", "main"] # the re-issue happens by itself) than hold the control loop for ten seconds. WRITE_BUDGET_S = 4.0 +# SMA-10. The same bound for handing a unit back (`driver.release()`). It is the +# time the shutdown path already allows one 0 W write, and the units are +# released CONCURRENTLY, so one slow or hung unit costs the wind-down no more +# than a hung GoodWe write does today: the Supervisor's kill window is not lost +# to it. A release still running at the bound is abandoned, not awaited. +RELEASE_BUDGET_S = WRITE_BUDGET_S + # SAFETY-34: a Core-only restart wipes persistent_notification and does NOT # restart this add-on (measured on TEST-23), so a latched alarm is re-asserted # this often. Why: goodwe docs/DECISIONS.md, SAFETY-34. @@ -561,6 +570,10 @@ class Controller: self.store_migration = migrate_unit_keys( store, _roster_names[0], _roster_names) self.inverters = build_roster(opts, store, sign_cfg, delivery_cfg) + # SMA-10. Which units hand back decides what the page says about Stop, + # restart and shadow mode. A GoodWe-only roster adds nothing. + handback.set_roster(handback.units_of(self.inverters)) + uitext.set_idle(handback.overlay(handback.current())) # ⚠️ CFG-04. SAY WHICH SHAPE IS IN FORCE, BECAUSE THE OTHER ONE'S KEYS # ARE STILL SITTING IN THE UI. A declared `inverters:` roster reads none # of `soc_entity`, `batt_entity`, `batt_invert` or `max_w`, and an @@ -4043,9 +4056,82 @@ class Controller: await asyncio.gather(*(self._write_unit(inv, value, force=force) for inv in targets)) + async def _release_last(self, inv) -> bool: + """`release()`, after a write already on its way to `inv` has finished + (at most half the budget is spent waiting for it), so the release is the + LAST thing the unit hears and a late write cannot undo it.""" + pending = [t for t in inv.inflight if not t.done()] + if pending: + await asyncio.wait(pending, timeout=RELEASE_BUDGET_S / 2) + return await self._driver(inv).release() + + async def _hand_back(self, inv) -> bool: + """SMA-10. Give ONE unit back: exactly one `release()`, then nothing. + + True when the unit is handed back (now, or already). Never raises: a + driver that raises or hangs is logged and the unit is tried again on the + next idle write, and it cannot stop a sibling being released or zeroed. + + ⚠️ `released` IS SET BEFORE THE FIRST AWAIT. A second caller (the + heartbeat while the Stop button is mid-release) then sees it and does + nothing, so a unit gets one `release()` and not two. If the release + did not take, the flag goes back down and the unit stays un-handed + until a later idle write tries again. + + ⚠️ IT IS NOT A WRITE: no `note_write`, no `last_write`, so a hand-back + cannot be read as a link that answered. `last_write` is cleared so the + first write after control starts again is always sent. + """ + if inv.released: + return True + if not inv.driver.configured(): + return False + inv.released = True + try: + ok = bool(await _bounded(self._release_last(inv), + RELEASE_BUDGET_S)) + except (asyncio.TimeoutError, TimeoutError): + ok = False + _LOG.error("Handing back %s took over %.0f s and was dropped. It " + "is tried again.", inv.name, RELEASE_BUDGET_S) + except Exception as err: # noqa: BLE001 - one unit never stops the rest + ok = False + _LOG.error("Handing back %s failed: %s. It is tried again.", + inv.name, err) + if ok: + inv.last_write = None + self.log_event(f"{inv.name}: handed back", + hint=messages.hint("handed_back", unit=inv.name)) + else: + inv.released = False + self.log_event(f"{inv.name}: hand-back failed", + hint=messages.hint("hand_back_failed", unit=inv.name)) + return ok + async def _write_unit(self, inv, value: float, *, force: bool = False) -> None: if not inv.driver.configured(): return + # --- SMA-10: A UNIT THAT HANDS BACK IS NOT WRITTEN WHILE THE ADD-ON IS + # IDLE. Idle is: shutting down, control stopped, or shadow mode. The + # unit is released once and then nothing is written to it - not the + # heartbeat, not the cycle's "stopped" 0 W - so its own controller keeps + # it. This is the passive half; `command_site_release()` is the active + # one, for the moment the add-on goes away. + # + # ⚠️ A SAFETY STOP IS NOT IDLE. It happens while control runs (`auto`, + # not shadow), so it falls through to the 0 W write below: the add-on's + # own decision to hold 0 W is never turned into a hand-back. + # + # ⚠️ AND CONTROL STARTING AGAIN RE-ARMS THE UNIT: the flag drops and + # `last_write` is cleared, so the first write goes out whatever it is. + # `stopping` never re-arms: a shutdown in progress stays handed back. + if inv.driver.hands_back: + if self.stopping or self.shadow or not self.auto: + await self._hand_back(inv) + return + if inv.released: + inv.released = False + inv.last_write = None entity = inv.driver.actuator_id() # for the log line below # --- SAFETY-03: THE CLAMP, AND IT IS ON THE ONLY PATH TO THE WIRE ----- @@ -4169,8 +4255,13 @@ class Controller: # tick by itself: `last_write` is updated only on an ACCEPTED write, so # `changed` is still true. try: - ok = await asyncio.wait_for(self._driver(inv).write(value), - timeout=WRITE_BUDGET_S) + _w = self._driver(inv).write(value) + if inv.driver.hands_back: + # SMA-10: kept, so a release can wait for it (`_release_last`) + _w = asyncio.ensure_future(_w) + inv.inflight.add(_w) + _w.add_done_callback(inv.inflight.discard) + ok = await asyncio.wait_for(_w, timeout=WRITE_BUDGET_S) except (asyncio.TimeoutError, TimeoutError): ok = False _LOG.error("Sending %+.0f W to %s (%s) took over %.0f s and was " @@ -6786,9 +6877,10 @@ class Controller: self.log_event("REFUSED: " + problems[0], hint=messages.hint("refused")) # Signal, then return WITHOUT entering the loop. amain() wakes - # on this, runs the normal wind-down (which commands 0 W) and - # re-raises ConfigRefused afterwards, so the exit code is 78 and - # the inverter is left at zero rather than latched. + # on this, runs the normal wind-down (which commands 0 W, or hands + # back a unit whose driver does: SMA-10) and re-raises + # ConfigRefused afterwards, so the exit code is 78 and the + # inverter is left at zero rather than latched. self.stop_event.set() return @@ -7052,10 +7144,42 @@ class Controller: async def shutdown(self) -> None: """Deterministic wind-down. Do not skip this.""" self.stopping = True - _LOG.info("shutting down - commanding 0 W on every inverter") - await self.command_site_zero() + _LOG.info(handback.log_shutdown(handback.current()) + or "shutting down - commanding 0 W on every inverter") + await self.command_site_release() self.mqtt.close() + async def command_site_release(self) -> None: + """SMA-10. The add-on is going away (Stop, shutdown, restart, task + death, a config refusal): every unit that hands back is given back, and + every other unit is held at 0 W exactly as `command_site_zero()` does. + + ⚠️ THE SAFETY STOPS AND REFUSALS DO NOT COME HERE. A sign trip, a + refused write, a missing reading, another writer: those are the add-on's + own decisions and hold 0 W through `write(0)` for EVERY unit, handed-back + drivers included. `command_site_zero()` is that, and it is unchanged. A + unit the add-on has just declared it cannot trust must not be handed to + something else that will then do what it likes with the battery. + + ⚠️ A ROSTER WITH NO HAND-BACK UNIT IS `command_site_zero()`, line for + line: GoodWe's release is its 0 W write, so Stop keeps today's write + and today's heartbeat. + + ⚠️ ONE UNIT'S FAILURE IS ITS OWN. `_hand_back()` never raises and is + bounded by RELEASE_BUDGET_S; the units run concurrently, so the whole + call takes no longer than its slowest bound. + """ + if not any(inv.driver.hands_back for inv in self.inverters): + await self.command_site_zero() + return + self.target = 0.0 + for inv in self.inverters: + inv.target = 0.0 + await asyncio.gather(*( + self._hand_back(inv) if inv.driver.hands_back + else self._write_unit(inv, 0.0, force=True) + for inv in self.inverters)) + async def command_site_zero(self) -> None: """Every inverter to 0 W, and every inverter's TARGET to 0 W with it. @@ -8011,7 +8135,9 @@ class Controller: # Zeroing `self.target` alone would leave each unit's last share # standing for its next heartbeat to re-emit, which an operator # would read as the stop button not working. - await self.command_site_zero() + # SMA-10: the Stop button is the add-on going quiet, so a unit + # that hands back is given back; every other unit is zeroed. + await self.command_site_release() return {"auto": self.auto} if what in ("maint_start", "maint_abort") and unit is not None: # MAINT-11: the operator named the battery. A request with no unit @@ -8055,6 +8181,21 @@ MAINT_CHARGING_W = 50.0 EXIT_CONTROL_DIED = 70 +async def _bounded(awaitable, budget_s: float): + """SMA-10. `await awaitable`, but give up after `budget_s` and do NOT wait + for it to wind down. `asyncio.wait_for` waits for the cancelled task to + finish, so a driver that ignores its cancellation would hold the whole + shutdown; this abandons it. Raises `asyncio.TimeoutError` at the bound.""" + task = asyncio.ensure_future(awaitable) + done, _ = await asyncio.wait({task}, timeout=budget_s) + if not done: + task.cancel() + # a result or error that arrives later has nobody to read it + task.add_done_callback(lambda t: t.cancelled() or t.exception()) + raise asyncio.TimeoutError + return task.result() + + class ControlTaskDied(RuntimeError): """The control task ended while nothing had asked it to stop.""" @@ -8081,10 +8222,12 @@ def control_task_done(task: asyncio.Task, *, controller, stop: asyncio.Event, if stop.is_set() or controller.stopping: return # the normal wind-down, or run_control()'s refusal path exc = None if task.cancelled() else task.exception() - _LOG.critical("Control stopped unexpectedly (%s), so every inverter is " - "set to 0 W and the add-on exits with code %d.", + _LOG.critical("Control stopped unexpectedly (%s), so %s and the add-on " + "exits with code %d.", "cancelled" if task.cancelled() else - (repr(exc) if exc else "returned"), EXIT_CONTROL_DIED, + (repr(exc) if exc else "returned"), + handback.task_died_state(handback.current()), + EXIT_CONTROL_DIED, exc_info=exc) died.append(exc) stop.set() diff --git a/goodwe_controller/app/messages.py b/goodwe_controller/app/messages.py index 831da93..3e24bc1 100644 --- a/goodwe_controller/app/messages.py +++ b/goodwe_controller/app/messages.py @@ -12,6 +12,8 @@ import re from dataclasses import dataclass from datetime import datetime +from . import handback + # Severity (STYLE.md section 3). An ALARM may use up to three sentences so # that no action a person must take is lost; everything else gets one. ALARM, WARNING, INFO, RESOLVED = "alarm", "warning", "info", "resolved" @@ -228,6 +230,19 @@ class Clamp: limit: str # a key of LIMIT_WORDS +def _shadow_so(held: str) -> str: + """"Shadow mode is on, so ." SMA-10: with a unit handed back, the + roster's own words say where each unit stays; otherwise `held`, as ever.""" + moved = handback.shadow_state(handback.current(), "en") + return f"Shadow mode is on, so {moved or held}." + + +def _shadow_keeps(held: str) -> str: + moved = handback.clause(handback.current(), "en", "at") \ + if handback.active(handback.current()) else None + return moved or held + + def output_clamp(clamps, *, shadow: bool = False) -> Notice: """SAFETY-03: a computed command above the limit. A software fault. @@ -238,7 +253,7 @@ def output_clamp(clamps, *, shadow: bool = False) -> Notice: fault = "This is a software fault, so please report it to your installer." if not clamps: cut = ("would have been cut back" if shadow else "was cut back") - state = ("Shadow mode is on, so every inverter stays at 0 W." if shadow + state = ((_shadow_so("every inverter stays at 0 W")) if shadow else "Control is still running.") return Notice(ID_OUTPUT_CLAMP, ALARM, T_OUTPUT_CLAMP, f"The add-on asked for more power than {RATING} allows " @@ -250,11 +265,11 @@ def output_clamp(clamps, *, shadow: bool = False) -> Notice: if others: verb = "was" if len(others) == 1 else "were" state = (f"{names(others)} would have been cut too, and shadow mode " - f"keeps every inverter at 0 W." if shadow else + f"keeps {_shadow_keeps('every inverter at 0 W')}." if shadow else f"{names(others)} {verb} cut too, and control is still " f"running.") else: - state = ("Shadow mode is on, so the inverter stays at 0 W." if shadow + state = (_shadow_so("the inverter stays at 0 W") if shadow else "Control is still running.") return Notice(ID_OUTPUT_CLAMP, ALARM, T_OUTPUT_CLAMP, f"{who} add-on asked for {watts(c.asked_w)}, above " @@ -576,6 +591,9 @@ def banner_running() -> str: def banner_stopped() -> str: + moved = handback.banner_stopped(handback.current()) + if moved is not None: # SMA-10: a unit is handed back, not held + return moved return Bi("Stopped — the inverters are set to 0 W", "Gestopt — de omvormers staan op 0 W") @@ -601,6 +619,9 @@ def banner_shadow(latched: bool) -> str: mode. Each stop has its own red row below, which names it and says how it clears. """ + moved = None if latched else handback.banner_shadow(handback.current()) + if moved is not None: # SMA-10: a unit is handed back, not held + return moved if latched: return Bi("Shadow mode: battery not controlled. A safety stop stays " "on, see below.", @@ -1367,6 +1388,8 @@ HINTS = { "setup_only": "In setup, nothing is sent to any inverter", "roster": "Inverter list read: {n} inverters", "maint_history": "Maintenance history did not move over cleanly", + "handed_back": "{unit}: handed back, the add-on writes nothing to it", + "hand_back_failed": "{unit}: could not be handed back, trying again", "stop_restored": "{unit}: an earlier safety stop is back, it stays at 0 W", "readings_late": "Home Assistant readings arrive late, control holds", "readings_back": "Home Assistant readings are on time again", @@ -1470,6 +1493,10 @@ def hint(key: str, **kw) -> str: kw["w"] = watts(kw["w"]) if "sw" in kw: kw["sw"] = signed_watts(kw["sw"]) + if key == "shadow_on": # SMA-10: a unit handed back is not at 0 W + moved = handback.hint_shadow_on(handback.current()) + if moved is not None: + return moved return HINTS[key].format(**kw) diff --git a/goodwe_controller/app/uitext.py b/goodwe_controller/app/uitext.py index d6caf53..85b0487 100644 --- a/goodwe_controller/app/uitext.py +++ b/goodwe_controller/app/uitext.py @@ -13,6 +13,11 @@ import os DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "ui_text") LANGS = ("en", "nl") _CACHE: dict = {} +# SMA-10. The sentences that move with the roster (handback.overlay): empty for +# every GoodWe-only roster, and then `raw()` is the file as it always was. +_IDLE: dict = {} +_GEN = [0] +_MERGED: dict = {} def lang_of(value) -> str: @@ -21,14 +26,36 @@ def lang_of(value) -> str: return head if head in LANGS else "en" -def raw(lang: str) -> dict: - """One language file as written, without the English fallback.""" +def _file(lang: str) -> dict: if lang not in _CACHE: with io.open(os.path.join(DIR, f"{lang}.json"), encoding="utf-8") as f: _CACHE[lang] = json.load(f) return _CACHE[lang] +def set_idle(overlay: dict) -> None: + """SMA-10: the roster-built sentences, `{lang: nested dict}` (or {}).""" + _IDLE.clear() + _IDLE.update(overlay or {}) + _MERGED.clear() + _GEN[0] += 1 + + +def generation() -> int: + """Changes whenever `set_idle()` does, so a cache of words can tell.""" + return _GEN[0] + + +def raw(lang: str) -> dict: + """One language file as written, without the English fallback (and with the + roster's own sentences laid over it when a unit is handed back).""" + if lang not in _IDLE: + return _file(lang) + if lang not in _MERGED: + _MERGED[lang] = _merge(_file(lang), _IDLE[lang]) + return _MERGED[lang] + + def _merge(base, over): if not (isinstance(base, dict) and isinstance(over, dict)): return copy.deepcopy(over) diff --git a/goodwe_controller/app/web.py b/goodwe_controller/app/web.py index d6a0551..f3f5b62 100644 --- a/goodwe_controller/app/web.py +++ b/goodwe_controller/app/web.py @@ -442,7 +442,8 @@ def build_app(controller, selfcfg=None) -> web.Application: """Restart the add-on, at the operator's explicit request. ⚠️ THIS IS THE ONE ROUTE ON THIS SURFACE THAT COSTS ANYTHING. The - wind-down commands 0 W on every inverter (`Controller.shutdown()`), so + wind-down commands 0 W on every inverter, or hands a unit back when its + driver does (`Controller.shutdown()`, SMA-10), so the request must carry the intent token; a reload, a retry or a stray POST cannot spend a control interruption. It restarts THIS ADD-ON and nothing else: no setpoint is written from here, and a latched failsafe diff --git a/goodwe_controller/test_messages.py b/goodwe_controller/test_messages.py index c0fd4ae..3636a2a 100644 --- a/goodwe_controller/test_messages.py +++ b/goodwe_controller/test_messages.py @@ -793,6 +793,7 @@ from datetime import datetime as _dt, timedelta as _td, timezone as _tz # noqa: from app import clamp as _clamp, freshness as _fresh, peakguard as _pg # noqa: E402 from app import store as _store # noqa: E402 +from app import handback # noqa: E402 from app.circulation import CirculationConfig, CirculationWatchdog # noqa: E402 from app.control import Tuning, compute # noqa: E402 from app.delivery import (DeliveryConfig, DeliveryWatchdog, UnitDelivery, # noqa: E402 @@ -850,7 +851,7 @@ DATA = { "main.py:inv.batt_fresh.bound.release_s", "main.py:inv.batt_fresh.bound.move_w", "main.py:inv.batt_fresh.bound.unchanged_s", "main.py:inv.batt_fresh.held", "main.py:value", "main.py:float(value)", - "main.py:entity", "main.py:WRITE_BUDGET_S", + "main.py:entity", "main.py:WRITE_BUDGET_S", "main.py:RELEASE_BUDGET_S", "main.py:_e", "main.py:messages.times(self.clamp_wd.engagements)", "main.py:messages.when(brussels(self.tariff_at))", @@ -1058,6 +1059,12 @@ TEXT = { "main.py:messages.reason_code(inv.batt_fresh.reason)": _coded(_stale_texts), "main.py:rep.explain()": _migration_texts, "store.py:rep.explain()": _migration_texts, + # SMA-10: the control-task CRITICAL line's state, for a GoodWe-only roster + # (the sentence it always was) and a mixed one. + "main.py:handback.task_died_state(handback.current())": lambda: [ + handback.task_died_state(()), + handback.task_died_state((handback.Unit("inverter-a"), + handback.Unit("inverter-b", True)))], "main.py:plan.words": _tariff_texts, # OPT-11 P2: why an MQTT command was ignored. "main.py:why": lambda: list(messages.PACK_REFUSED.values()), diff --git a/goodwe_controller/test_sma10_two_idle_states.py b/goodwe_controller/test_sma10_two_idle_states.py new file mode 100644 index 0000000..7bad08b --- /dev/null +++ b/goodwe_controller/test_sma10_two_idle_states.py @@ -0,0 +1,1023 @@ +"""SMA-10: two idle states, hold 0 W or hand the inverter back. `python3 test_sma10_two_idle_states.py` + +A unit whose driver says `hands_back` is given back (`release()`) when the add-on +goes away and in shadow mode; every safety stop and refusal still holds 0 W. A +GoodWe unit never sees a changed byte. + +1. GoodWe only: every sentence this card builds from the roster is the literal + it was before the card, in English and Dutch, and the page words are the + files as they were. +2. The sentences for a mixed and a hand-back-only roster, English and Dutch, + printed, held to STYLE.md, and taken from the DRIVER'S words. +3. A real Controller with a fake hand-back unit (and a GoodWe unit): Stop, + shutdown, a safety stop, Start, shadow mode, the per-unit state, the call + sites, one unit's release raising or hanging. +4. The real main() in a child process: SIGTERM, the control task's death, a late + config refusal (exit 78) and a startup refusal (exit 78, no driver exists). + +Mutating a module this file grades? Read MUTATION.md first (purge __pycache__). +The mutants this file was run against are listed at the end. +""" + +import asyncio +import json +import logging +import os +import signal +import subprocess +import sys +import time +from datetime import datetime, timezone +from pathlib import Path + +HERE = Path(__file__).resolve().parent +sys.path.insert(0, str(HERE / "tools")) + +fails = [] + + +def check(name, cond): + print(f" {'ok ' if cond else 'FAIL'} {name}") + if not cond: + fails.append(name) + + +# --------------------------------------------------------------------------- # +# the child: the real main() with the outside world faked (as test_control_task) +# --------------------------------------------------------------------------- # +CHILD_BASE = { + "meter_entity": "sensor.grid", "meter_source": "off", "gain": 0.6, + "slew_w": 0, "deadband_w": 15, "target_grid_w": -10, "step_w": 10, + "heartbeat_s": 10, "stale_input_s": 15, "cycle_floor_s": 1, + "auto_start": True, "shadow_mode": "off", + "inverters": [ + dict(name="inverter-a", setpoint_entity="number.sp_a", + soc_entity="sensor.soc_a", batt_entity="sensor.batt_a", + max_w=2000, bms_soc_floor=10), + dict(name="inverter-b", driver="hbfake", host="10.9.9.9", port=502, + unit_id=3, max_w=2000, bms_soc_floor=10)]} +DEADLINE_S = 40 + + +def make_hb_driver(event=None): + from app.drivers import Driver, Probe + + class HBDriver(Driver): + """A unit that hands back. Calls are kept; `mode` decides how its + release() behaves: ok, false, raise or hang.""" + name = "hbfake" + writer_key = "host" + brand = "Fake" + hands_back = True + words = {"back": "SMA control", "back_nl": "de sturing van SMA"} + + def __init__(self, host="", port=502, unit_id=1, rating=2000.0): + self.host, self.port, self.unit_id = host, port, unit_id + self.rating, self.power = rating, 0.0 + self.calls = [] + self.mode = "ok" + self.write_ok = True + self.write_delay = 0.0 + + @classmethod + def from_entry(cls, entry): + return cls(entry.get("host", ""), entry.get("port", 502), + entry.get("unit_id", 1)) + + @classmethod + def actuator_id_of(cls, entry): + host = str(entry.get("host") or "").strip() + return f"{host}:{entry.get('port', 502)}:{entry.get('unit_id', 1)}" \ + if host else "" + + def configured(self): + return bool(self.host) + + def actuator_id(self): + return f"{self.host}:{self.port}:{self.unit_id}" + + def soc_source(self): + return "fake-soc" + + def batt_source(self): + return "fake-power" + + async def limits(self): + return -self.rating, self.rating + + async def read(self): + return 60.0, self.power, {} + + async def write(self, w): + self.calls.append(("write", w)) + if event: + event("fake", "write", float(w)) + if self.write_delay: + await asyncio.sleep(self.write_delay) + self.calls.append(("write_done", w)) + if self.write_ok: + self.power = w + return self.write_ok + + async def release(self): + self.calls.append(("release", None)) + if event: + event("fake", "release") + if self.mode == "raise": + raise RuntimeError("release exploded") + if self.mode == "false": + return False + if self.mode == "hang": + await asyncio.sleep(3600) + return True + + async def probe(self): + return Probe(-self.rating, self.rating, 60.0, self.power, {}) + + return HBDriver + + +def child(scenario): + import aiohttp + import app.main as M + import app.store + import app.web + from app import drivers + + def event(*e): + print("EV " + json.dumps(list(e)), flush=True) + + drivers.REGISTRY["hbfake"] = make_hb_driver(event) + drivers.SELECTABLE = drivers.SELECTABLE + ("hbfake",) + t0 = time.monotonic() + + class FakeHass: + def __init__(self): + self.limit_calls = 0 + + async def limits(self, entity): + self.limit_calls += 1 + if scenario == "refuse": # a rating below max_w, once read + return -1000.0, 1000.0, 1.0 + return -5000.0, 5000.0, 1.0 + + async def number(self, entity, invert=False): + if entity == "sensor.grid": + return 600.0 + (int(time.monotonic() - t0) % 2) + if entity == "sensor.soc_a": + return 50.0 + if entity == "sensor.batt_a": + return 0.0 + return None + + async def state(self, entity): + return None + + async def set_number(self, entity, value): + event("goodwe", "write", float(value)) + return True + + async def call(self, *a, **k): + return True + + async def mqtt_service(self): + raise RuntimeError("no broker in this test") + + class _Store: + data = {} + + def get(self, key, default=None): + return default + + def set(self, *a, **k): + pass + + class _Runner: + async def cleanup(self): + event("cleanup") + + async def _web_start(controller, port=8099, selfcfg=None): + return _Runner() + + class _NoProbeSession: + async def __aenter__(self): + return self + + async def __aexit__(self, *a): + return False + + def get(self, *a, **k): + raise OSError("no network in this test") + + opts = json.loads(json.dumps(CHILD_BASE)) + if scenario == "badcfg": + opts["slew_w"] = -5 # refused at the startup gate + M.load_options = lambda: dict(opts) + M.HomeAssistant = lambda session: FakeHass() + app.store.Store = _Store + app.web.start = _web_start + aiohttp.ClientSession = _NoProbeSession + + real_cycle = M.Controller.cycle + n = {"cycles": 0} + + async def cycle(self): + n["cycles"] += 1 + if scenario == "fault" and n["cycles"] == 4: + event("fault") + raise RuntimeError("injected SMA-10 fault") + await real_cycle(self) + if scenario == "sigterm" and n["cycles"] == 4: + event("sigterm") + os.kill(os.getpid(), signal.SIGTERM) + + M.Controller.cycle = cycle + M.main() + + +def run_child(scenario): + env = dict(os.environ, PYTHONDONTWRITEBYTECODE="1") + try: + p = subprocess.run([sys.executable, "-B", __file__, "--child", scenario], + cwd=str(HERE), env=env, capture_output=True, + text=True, timeout=DEADLINE_S) + except subprocess.TimeoutExpired as e: + rc, out, err = None, e.stdout or "", e.stderr or "" + else: + rc, out, err = p.returncode, p.stdout, p.stderr + out = out.decode() if isinstance(out, bytes) else out + err = err.decode() if isinstance(err, bytes) else err + return rc, err, [json.loads(ln[3:]) for ln in out.splitlines() + if ln.startswith("EV ")] + + +if len(sys.argv) == 3 and sys.argv[1] == "--child": + child(sys.argv[2]) + sys.exit(0) + +# --------------------------------------------------------------------------- # +from app import drivers, handback, homeview, messages, uitext # noqa: E402 +from app import main as M # noqa: E402 +from app.drivers import GoodWeEsphomeDriver, REGISTRY # noqa: E402 +from app.main import Controller # noqa: E402 +from app.store import DEFAULTS # noqa: E402 +from style_rules import BANNED, SHOUT, sentences, words # noqa: E402 + +HB = make_hb_driver() +REGISTRY["hbfake"] = HB + + +class OtherHB(HB): + """A second make that hands back, with other words: the sentences must + come from the driver, not from a hard-coded brand.""" + name = "otherhb" + words = {"back": "Zeta control", "back_nl": "de sturing van Zeta"} + + +REGISTRY["otherhb"] = OtherHB + + +class MemStore: + def __init__(self): + self.data = dict(DEFAULTS) + + def set(self, k, v): + self.data[k] = v + + def get_time(self, *a): + return None + + def set_time(self, *a): + pass + + +class Pub: + enabled = False + + def __init__(self): + self.last = {} + + def publish(self, values): + self.last = dict(values) + + def close(self): + pass + + +class LogHass: + def __init__(self): + self.log = [] + self.numbers = {"sensor.grid": 3000.0, "sensor.soc_a": 50.0, + "sensor.batt_a": 0.0} + + async def number(self, entity, invert=False): + self.log.append(("number", entity, invert)) + return self.numbers.get(entity) + + async def limits(self, entity): + self.log.append(("limits", entity, None)) + return (-3000.0, 3000.0, 1.0) + + async def set_number(self, entity, value): + self.log.append(("set_number", entity, value)) + return True + + async def state(self, entity): + return None + + async def call(self, *a): + return True + + def writes(self): + return [v for m, e, v in self.log if m == "set_number"] + + +def run(coro): + return asyncio.run(coro) + + +def gw(name="inverter-a", suffix="a"): + return dict(name=name, setpoint_entity=f"number.sp_{suffix}", + soc_entity=f"sensor.soc_{suffix}", + batt_entity=f"sensor.batt_{suffix}", max_w=3000, + bms_soc_floor=10) + + +def fk(name="inverter-b", driver="hbfake", host="10.9.9.9"): + return dict(name=name, driver=driver, host=host, port=502, unit_id=3, + max_w=2000, bms_soc_floor=10) + + +def site(inverters, auto=True, **over): + opts = dict(meter_source="off", meter_entity="sensor.grid", max_w=5000, + gain=0.6, slew_w=0, deadband_w=15, target_grid_w=0, step_w=10, + heartbeat_s=10, stale_input_s=15, shadow_mode="off", + inverters=inverters) + opts.update(over) + hass = LogHass() + pub = Pub() + c = Controller(opts, hass, MemStore(), pub, stop=asyncio.Event()) + for inv in c.inverters: + inv.dev_min, inv.dev_max = -3000.0, 3000.0 + c.auto = auto + return c, hass, pub + + +def fakes(c): + return [i.driver for i in c.inverters if isinstance(i.driver, HB)] + + +def n_calls(drv, what): + return len([1 for w, _a in drv.calls if w == what]) + + +def writes_of(drv): + return [a for w, a in drv.calls if w == "write"] + + +async def cycles(c, n=4): + for _ in range(n): + await c.read_inputs() + await c.cycle() + + +# ============================================================================= +print("SMA-10 - two idle states") +print() +print("1. a GoodWe-only roster: every sentence is what it was before this card") + +BASE = { + "en": { + "stop": "Every inverter is set to 0 W until you start control again.", + "restart": "Every inverter is set to 0 W while the add-on restarts, and " + "then the new settings apply.", + "shadow_restart": "Every inverter is set to 0 W while the add-on " + "restarts, and shadow mode keeps them at 0 W after " + "that.", + "setup_shadow": "The add-on restarts with these settings, and shadow " + "mode keeps every inverter at 0 W.", + "attn": "The add-on decides everything, but every inverter stays at " + "0 W.", + "ctl_shadow": "Shadow mode: every inverter stays at 0 W.", + "ctl_stopped": "Control is stopped. Every inverter is at 0 W.", + "banner_stopped": "Stopped — the inverters are set to 0 W", + "banner_shadow": "Shadow mode: the battery is not controlled. " + "Inverters stay at 0 W.", + "banner_shadow_latched": "Shadow mode: battery not controlled. A " + "safety stop stays on, see below.", + "hint": "Shadow mode on, inverters stay at 0 W", + }, + "nl": { + "stop": "Elke omvormer gaat naar 0 W tot u de regeling opnieuw start.", + "restart": "Elke omvormer gaat naar 0 W terwijl de add-on herstart, " + "daarna gelden de nieuwe instellingen.", + "shadow_restart": "Elke omvormer gaat naar 0 W terwijl de add-on " + "herstart, en de schaduwmodus houdt ze daarna op " + "0 W.", + "setup_shadow": "De add-on herstart met deze instellingen, en de " + "schaduwmodus houdt elke omvormer op 0 W.", + "attn": "De add-on beslist alles, maar elke omvormer blijft op 0 W.", + "ctl_shadow": "Schaduwmodus: elke omvormer blijft op 0 W.", + "ctl_stopped": "De regeling is gestopt. Elke omvormer staat op 0 W.", + "banner_stopped": "Gestopt — de omvormers staan op 0 W", + "banner_shadow": "Schaduwmodus: de batterij wordt niet geregeld. " + "Omvormers blijven op 0 W.", + "banner_shadow_latched": "Schaduwmodus: batterij niet geregeld. Een " + "veiligheidsstop blijft aan, zie hieronder.", + }, +} +CLAMP_SHADOW = [ + "Shadow mode is on, so every inverter stays at 0 W.", + "Shadow mode is on, so the inverter stays at 0 W.", + "and shadow mode keeps every inverter at 0 W.", +] + + +def page_sentences(lang): + """The nine page sentences this card moves, as the page gets them.""" + r = uitext.raw(lang) + s, hw = r["settings"], r["home_words"] + return {"stop": r["home"]["dialog"]["stop"]["description"], + "restart": s["dialog"]["description"], + "shadow_restart": s["setup"]["dialog"]["description"], + "setup_shadow": s["setup"]["dialog_setup"]["description"], + "attn": hw["attn_shadow_body"], "ctl_shadow": hw["control"]["shadow"], + "ctl_stopped": hw["control"]["stopped"]} + + +cg, _hg, _pg = site([gw("inverter-a", "a"), gw("inverter-b", "b")]) +check("a GoodWe-only Controller leaves the roster with no hand-back unit", + not handback.active(handback.current()) and uitext.generation() > 0) +for lang in ("en", "nl"): + got = page_sentences(lang) + check(f"[{lang}] the seven page sentences are the literals from before", + all(got[k] == BASE[lang][k] for k in got)) + on_disk = json.loads((HERE / "app" / "ui_text" / f"{lang}.json") + .read_text(encoding="utf-8")) + check(f"[{lang}] the whole page word file is the file on disk, byte for " + f"byte as data", uitext.raw(lang) == on_disk) +homeview._LANG[0] = "en" +check("[en] homeview reads the same control line", + homeview.control_text(False, False) == BASE["en"]["ctl_stopped"] + and homeview.control_text(True, True) == BASE["en"]["ctl_shadow"] + and homeview.attn_shadow_body() == BASE["en"]["attn"]) +homeview._LANG[0] = "nl" +check("[nl] homeview reads the same control line", + homeview.control_text(False, False) == BASE["nl"]["ctl_stopped"] + and homeview.control_text(True, True) == BASE["nl"]["ctl_shadow"] + and homeview.attn_shadow_body() == BASE["nl"]["attn"]) +homeview._LANG[0] = "en" +for key, fn in (("banner_stopped", messages.banner_stopped), + ("banner_shadow", lambda: messages.banner_shadow(False)), + ("banner_shadow_latched", lambda: messages.banner_shadow(True))): + t = fn() + check(f"{key}: English and Dutch are the literals from before", + str(t) == BASE["en"][key] and t.nl == BASE["nl"][key]) +check("hint shadow_on is the literal from before", + messages.hint("shadow_on") == BASE["en"]["hint"]) +bodies = " | ".join([messages.output_clamp((), shadow=True).body, + messages.output_clamp( + [messages.Clamp("a", 5, 4, "rating")], + shadow=True).body, + messages.output_clamp( + [messages.Clamp("a", 5, 4, "rating"), + messages.Clamp("b", 5, 4, "rating")], + shadow=True).body]) +check("the three shadow-mode output-clamp sentences are the literals from " + "before", all(x in bodies for x in CLAMP_SHADOW)) +_logs = [] + + +class _Cap(logging.Handler): + def emit(self, rec): + _logs.append(rec.getMessage()) + + +_h = _Cap() +logging.getLogger("goodwe").addHandler(_h) +logging.getLogger("goodwe").setLevel(logging.DEBUG) +_LOG_NAMES = [n for n in logging.root.manager.loggerDict + if n.startswith("goodwe")] +run(cg.shutdown()) +check("the shutdown INFO line is the literal from before", + "shutting down - commanding 0 W on every inverter" in _logs) +_ev = asyncio.Event() +_t = type("T", (), {"cancelled": lambda s: False, + "exception": lambda s: RuntimeError("x")})() +M.control_task_done(_t, controller=type("C", (), {"stopping": False})(), + stop=_ev, died=[]) +check("the control-task CRITICAL line is the literal from before", + any(m == "Control stopped unexpectedly (RuntimeError('x')), so every " + "inverter is set to 0 W and the add-on exits with code 70." + for m in _logs)) +logging.getLogger("goodwe").removeHandler(_h) + +# ============================================================================= +print() +print("2. the sentences for a mixed and a hand-back-only roster") + + +def units(*inv): + return handback.units_of(site(list(inv))[0].inverters) + + +MIXED = units(gw("inverter-a", "a"), fk("inverter-b")) +ALLHB = units(fk("inverter-a"), fk("inverter-b", host="10.9.9.8")) +ONEHB = units(fk("inverter-b")) +TWO_MAKES = units(gw("inverter-a", "a"), fk("inverter-b"), + fk("inverter-c", "otherhb", "10.9.9.7")) +SHAPES = (("mixed (GoodWe + hand-back)", MIXED), + ("hand-back only (two)", ALLHB), ("hand-back only (one)", ONEHB), + ("GoodWe + two makes", TWO_MAKES)) +ROWS = (("stop dialog", handback.stop_dialog), + ("restart dialog", handback.restart_dialog), + ("restart dialog, shadow", handback.restart_dialog_shadow), + ("setup dialog, shadow", handback.setup_dialog_shadow), + ("control line, stopped", handback.control_stopped), + ("control line, shadow", handback.control_shadow), + ("attention body, shadow", handback.attn_shadow_body)) +problems = [] +for label, us in SHAPES: + print(f" --- {label}") + for what, fn in ROWS: + for lang in ("en", "nl"): + t = fn(us, lang) + print(f" {lang} {what}: {t}") + if len(words(t)) > 25 or sentences(t) > 2 or BANNED.search(t) \ + or SHOUT.search(t.replace("SMA", "S")): + if not (what.startswith("control line") and sentences(t) == 2): + problems.append((label, lang, what, t)) + b = handback.banner_stopped(us) + print(f" en banner stopped: {b}") + print(f" nl banner stopped: {b.nl}") + if len(words(str(b))) > 12 or len(words(b.nl)) > 12: + problems.append((label, "banner", str(b), b.nl)) + print(f" en hint shadow on: {handback.hint_shadow_on(us)}") +check("every sentence is plain and short (STYLE.md: 25 words, banner 12, no " + "banned words)", not problems) +if problems: + print(" " + repr(problems)) +m_en = handback.stop_dialog(MIXED, "en") +m_nl = handback.stop_dialog(MIXED, "nl") +check("EN stop dialog, mixed roster, is the card's sentence", + m_en == "Inverter-a is set to 0 W and inverter-b goes back to SMA " + "control until you start control again.") +check("NL stop dialog, mixed roster, is the card's sentence", + m_nl == "Inverter-a gaat naar 0 W en inverter-b gaat terug naar de " + "sturing van SMA tot u de regeling opnieuw start.") +check("hand-back only: 'The inverters go back to SMA control ...'", + handback.stop_dialog(ALLHB, "en") == + "The inverters go back to SMA control until you start control again." + and handback.stop_dialog(ONEHB, "en") == + "The inverter goes back to SMA control until you start control again." + and handback.stop_dialog(ALLHB, "nl") == + "De omvormers gaan terug naar de sturing van SMA tot u de regeling " + "opnieuw start.") +check("banner (mixed) is the card's: 'Stopped — inverter-a at 0 W, inverter-b " + "under SMA control'", + str(handback.banner_stopped(MIXED)) == + "Stopped — inverter-a at 0 W, inverter-b under SMA control" + and handback.banner_stopped(MIXED).nl == + "Gestopt — inverter-a op 0 W, inverter-b onder de sturing van SMA") +ZETA = units(gw(), fk("inverter-c", "otherhb")) +check("the words come from the DRIVER: another make's words, never 'SMA'", + handback.stop_dialog(ZETA, "en") == "Inverter-a is set to 0 W and " + "inverter-c goes back to Zeta control until you start control again." + and "de sturing van Zeta" in handback.stop_dialog(ZETA, "nl") + and "SMA" not in handback.stop_dialog(ZETA, "en") + and "SMA" not in str(handback.banner_stopped(ZETA))) +check("two makes in one roster: the sentence says 'its own control' rather " + "than run past 25 words", + "go back to their own control" in handback.restart_dialog(TWO_MAKES, "en") + and "SMA" not in handback.restart_dialog(TWO_MAKES, "en") + and "Zeta" not in handback.restart_dialog(TWO_MAKES, "en")) +bare = type("D", (), {"hands_back": True, "words": {}})() +bare_inv = type("I", (), {"name": "x", "driver": bare})() +check("a hand-back driver with no words gets a plain brand-free phrase", + handback.stop_dialog((handback.unit_of(bare_inv),), "en") == + "The inverter goes back to its own control until you start control " + "again.") +long_roster = tuple(handback.Unit(f"inverter-{c}", i % 2 == 1, "SMA control", + "de sturing van SMA") + for i, c in enumerate("abcdefg")) +lb = handback.banner_stopped(long_roster) +check("a long roster falls back to counts and the banner stays 12 words", + len(words(str(lb))) <= 12 and len(words(lb.nl)) <= 12 + and "3 under SMA control" in str(lb)) +print(f" long roster banner: {lb} / {lb.nl}") + +# the Controller builds the page from its own roster +cm, hm, pm = site([gw("inverter-a", "a"), fk("inverter-b")]) +check("a Controller with a hand-back unit puts the sentences in the page words", + page_sentences("en")["stop"] == m_en + and page_sentences("nl")["stop"] == m_nl + and "SMA control" in page_sentences("en")["ctl_stopped"] + and messages.banner_stopped() == handback.banner_stopped(MIXED) + and messages.hint("shadow_on") == handback.hint_shadow_on(MIXED)) +from app import web # noqa: E402 +html = web.page() +check("...and the assembled page carries them (and not the old stop sentence)", + m_en in html and BASE["en"]["stop"] not in html and m_nl in html) +homeview._LANG[0] = "nl" +check("homeview (nl) says the same in its control line", + homeview.control_text(False, False) == page_sentences("nl")["ctl_stopped"]) +homeview._LANG[0] = "en" +check("the shadow banner and the clamp notice no longer say every inverter " + "stays at 0 W", + "0 W" not in str(messages.banner_shadow(False)) + and "stays at 0 W and inverter-b stays under SMA control" in + messages.output_clamp((), shadow=True).body) +print(f" shadow banner: {messages.banner_shadow(False)}") +print(f" shadow clamp: {messages.output_clamp((), shadow=True).body}") + +# ============================================================================= +print() +print("3. a real Controller with a fake hand-back unit") + + +async def hb_pause(): + await asyncio.sleep(0) + + +# --- 3a. Stop: one release; GoodWe 0 W + heartbeat; no later write ----------- +c, h, p = site([gw("inverter-a", "a"), fk("inverter-b")]) +f = fakes(c)[0] +spy = {"zero": 0, "release": 0} +_z, _r = c.command_site_zero, c.command_site_release + + +async def spy_zero(): + spy["zero"] += 1 + await _z() + + +async def spy_release(): + spy["release"] += 1 + await _r() + + +c.command_site_zero, c.command_site_release = spy_zero, spy_release +run(cycles(c, 5)) +check("precondition: both units were being written a non-zero share while " + "control ran", any(v for v in writes_of(f)) and any(h.writes())) +n_f = len(writes_of(f)) +n_g = len(h.writes()) + + +async def stop_then_beat(): + await c.handle_action("auto_toggle") # auto True -> False: Stop + out = {} + out["after_stop_f"] = len(f.calls) + for k in range(4): # heartbeats well apart + await c.send_heartbeats(10_000.0 + 100 * k, 10.0) + await cycles(c, 3) # and the cycle's own writes + return out + + +r = run(stop_then_beat()) +check("STOP: the button calls command_site_release(), not command_site_zero()", + spy["release"] == 1 and spy["zero"] == 0) +check("STOP: the fake unit got exactly one release() and nothing after it", + n_calls(f, "release") == 1 and f.calls[-1] == ("release", None) + and len(writes_of(f)) == n_f) +check("STOP: ...no write at all from the heartbeat or the cycle, after it", + len(f.calls) == r["after_stop_f"] and c.inverters[1].released) +gw_after = h.writes()[n_g:] +check("STOP: the GoodWe unit got 0 W, and its heartbeat and cycle go on " + "writing 0 W (today's behaviour)", + gw_after and set(gw_after) == {0.0} and len(gw_after) >= 2) +check("STOP: the MQTT status word is unchanged: 'stopped'", + (c.publish(), p.last.get("status"))[1] == "stopped") + +# --- 3b. shutdown after Stop: still one release ------------------------------ +run(c.shutdown()) +check("SHUTDOWN after Stop: still exactly one release(); GoodWe got its 0 W", + n_calls(f, "release") == 1 and h.writes()[-1] == 0.0 + and spy["release"] == 2 and spy["zero"] == 0) + +# --- 3c. shutdown from running, then a late heartbeat ------------------------ +c, h, p = site([gw("inverter-a", "a"), fk("inverter-b")]) +f = fakes(c)[0] +run(cycles(c, 5)) +n_f = len(f.calls) + + +async def down_then_beat(): + await c.shutdown() + last = h.writes()[-1] + await c.send_heartbeats(50_000.0, 10.0) + await cycles(c, 2) # (the real loop has ended by now) + return last + + +gw_last = run(down_then_beat()) +check("SHUTDOWN while running: one release() for the fake unit, no write " + "after it (heartbeat and cycle included)", + n_calls(f, "release") == 1 and f.calls[n_f:] == [("release", None)]) +check("SHUTDOWN while running: the GoodWe unit is at 0 W", + gw_last == 0.0) + +# --- 3d. a safety stop on the same unit: 0 W written, never released --------- +c, h, p = site([gw("inverter-a", "a"), fk("inverter-b")]) +f = fakes(c)[0] +run(cycles(c, 5)) +fb = c.inverters[1] + + +async def safety(): + c.circ_wd.tripped, c.circ_wd.reason = True, "test" # circulating power + await c.cycle() + await c.cycle() + + +n0 = len(f.calls) +precondition = writes_of(f)[-1] != 0.0 +run(safety()) +check("SAFETY STOP (circulating power): the fake unit, which was carrying " + "power, is written 0 W and its release() is never called", + precondition and writes_of(f)[-1] == 0.0 and n_calls(f, "release") == 0 + and not fb.released and len(f.calls) > n0) + + +async def nonfinite(): + await c._write_unit(fb, float("nan"), force=True) + + +c.allocation = None +run(nonfinite()) +check("SAFETY STOP (non-finite value): 0 W on the fake unit, no release()", + writes_of(f)[-1] == 0.0 and n_calls(f, "release") == 0) + + +async def refused_write(): + await c._write_unit(fb, 777.0, force=True) # nothing authorised this + + +c.allocation = None +run(refused_write()) +check("SAFETY STOP (write refused): an unauthorised 777 W becomes 0 W on the " + "fake unit; release() is never called", + writes_of(f)[-1] == 0.0 and n_calls(f, "release") == 0) +run(c.command_site_zero()) +check("the maintenance abort path (command_site_zero) holds 0 W on the fake " + "unit too while control runs, with no release()", + writes_of(f)[-1] == 0.0 and n_calls(f, "release") == 0) + +# --- 3e. Start writes again --------------------------------------------------- +c, h, p = site([gw("inverter-a", "a"), fk("inverter-b")]) +f = fakes(c)[0] +run(cycles(c, 4)) +run(c.handle_action("auto_toggle")) # Stop +check("precondition: handed back after Stop", + c.inverters[1].released and n_calls(f, "release") == 1) +n0 = len(writes_of(f)) +run(c.handle_action("auto_toggle")) # Start +run(cycles(c, 5)) +check("START: control writes to the unit again, and it is no longer " + "'released'", len(writes_of(f)) > n0 and not c.inverters[1].released + and any(v != 0.0 for v in writes_of(f)[n0:])) +run(c.handle_action("auto_toggle")) # Stop again +check("...and a second Stop hands it back again (one release() per Stop)", + n_calls(f, "release") == 2) + +# --- 3e2. a write already on its way is finished BEFORE the release ---------- +c, h, p = site([gw("inverter-a", "a"), fk("inverter-b")]) +f = fakes(c)[0] +f.write_delay = 0.6 + + +async def race(): + fb = c.inverters[1] + t = asyncio.ensure_future(c._write_unit(fb, 0.0, force=True)) + await asyncio.sleep(0.1) # the write is on its way + c.auto = False # Stop + await c.command_site_release() + await t + + +run(race()) +print(f" calls on the unit: {[w for w, _a in f.calls]}") +check("RACE: a write on its way when Stop is pressed finishes first and the " + "release() is the LAST thing the unit hears", + [w for w, _a in f.calls] == ["write", "write_done", "release"] + and c.inverters[1].released) + +# --- 3f. shadow mode ---------------------------------------------------------- +c, h, p = site([gw("inverter-a", "a"), fk("inverter-b")], shadow_mode="on") +f = fakes(c)[0] +run(cycles(c, 5)) +run(c.send_heartbeats(20_000.0, 10.0)) +check("SHADOW: the fake hand-back unit is released once and never written", + n_calls(f, "release") == 1 and n_calls(f, "write") == 0 + and c.shadow) +check("SHADOW: the GoodWe unit is held at 0 W as today", + h.writes() and set(h.writes()) == {0.0}) +c2, h2, p2 = site([gw("inverter-a", "a"), gw("inverter-b", "b")], + shadow_mode="on") +run(cycles(c2, 5)) +run(c2.send_heartbeats(20_000.0, 10.0)) +check("SHADOW, GoodWe only: unchanged, 0 W on both units", + h2.writes() and set(h2.writes()) == {0.0}) +c, h, p = site([gw("inverter-a", "a"), fk("inverter-b")], auto=False, + shadow_mode="auto") +f = fakes(c)[0] +run(cycles(c, 3)) +check("SHADOW (auto, control stopped): released once, no writes", + n_calls(f, "release") == 1 and n_calls(f, "write") == 0) + +# --- 3g. mixed/hand-back-only rosters at the edges --------------------------- +c, h, p = site([fk("inverter-a"), fk("inverter-b", host="10.9.9.8")]) +fa, fb2 = fakes(c) +run(cycles(c, 4)) +run(c.handle_action("auto_toggle")) +run(c.send_heartbeats(30_000.0, 10.0)) +check("HAND-BACK ONLY roster: each unit released once, no write after, and " + "Home Assistant was asked about nothing but the meter", + n_calls(fa, "release") == 1 and n_calls(fb2, "release") == 1 + and fa.calls[-1] == ("release", None) + and fb2.calls[-1] == ("release", None) + and not h.writes()) + +# --- 3h. one unit's release() raising, failing or hanging -------------------- +print(" (the next checks wait for the release bound)") +c, h, p = site([gw("inverter-a", "a"), fk("inverter-b", host="10.1.1.1"), + fk("inverter-c", host="10.1.1.2"), + fk("inverter-d", host="10.1.1.3"), + fk("inverter-e", host="10.1.1.4")]) +fb_raise, fc_hang, fd_ok, ff_false = fakes(c) +fb_raise.mode, fc_hang.mode, ff_false.mode = "raise", "hang", "false" +run(cycles(c, 3)) +c.auto = False # what the Stop button does +t0 = time.monotonic() +run(c.command_site_release()) +took = time.monotonic() - t0 +print(f" command_site_release() with one raising, one hanging and one " + f"failing unit: {took:.2f} s " + f"(bound {M.RELEASE_BUDGET_S:.1f} s)") +check("the bound: it returns within RELEASE_BUDGET_S plus a margin, however " + "many units hang (they run together)", + took < M.RELEASE_BUDGET_S + 1.5) +check("the healthy unit was released (once) and the GoodWe unit zeroed, " + "though three others raised, failed or hung", + n_calls(fd_ok, "release") == 1 and c.inverters[3].released + and h.writes()[-1] == 0.0) +check("the units that raised, failed or hung are NOT marked handed back, so " + "they are tried again", not any(c.inverters[i].released + for i in (1, 2, 4))) +fb_raise.mode = "ok" +run(c.send_heartbeats(60_000.0, 10.0)) +check("...and the next idle write retries the one that raised (now answers): " + "released, and the healthy unit is not released a second time", + n_calls(fb_raise, "release") == 2 and c.inverters[1].released + and n_calls(fd_ok, "release") == 1) +c, h, p = site([gw("inverter-a", "a"), fk("inverter-b"), + fk("inverter-c", host="10.1.1.2")]) +fb_h, fc_ok = fakes(c) +fb_h.mode = "hang" +t0 = time.monotonic() +run(c.shutdown()) +took = time.monotonic() - t0 +check("shutdown() with one unit hanging finishes inside the bound, the other " + "unit is released and the GoodWe unit at 0 W", + took < M.RELEASE_BUDGET_S + 1.5 and n_calls(fc_ok, "release") == 1 + and h.writes()[-1] == 0.0) + +# --- 3i. per-unit state and the other counts --------------------------------- +c, h, p = site([gw("inverter-a", "a"), fk("inverter-b")]) +run(cycles(c, 4)) +rows = {r["name"]: r for r in homeview.battery_rows(c, False, True, False)} +check("before Stop neither unit says it is handed back", + "handed_back" not in rows["inverter-a"] + and "handed_back" not in rows["inverter-b"]) +run(c.handle_action("auto_toggle")) +homeview._LANG[0] = "en" +rows = {r["name"]: r for r in homeview.battery_rows(c, False, False, False)} +print(f" per-unit state after Stop: inverter-a: {rows['inverter-a']['now']!r}" + f"; inverter-b: {rows['inverter-b']['now']!r}") +check("PER-UNIT STATE: the handed-back unit says so in its driver's words " + "(not '0 W'); the GoodWe unit does not change", + rows["inverter-b"].get("handed_back") is True + and rows["inverter-b"]["now"] == "Back under SMA control" + and "handed_back" not in rows["inverter-a"] + and "0 W" in rows["inverter-a"]["now"]) +homeview._LANG[0] = "nl" +rows = {r["name"]: r for r in homeview.battery_rows(c, False, False, False)} +check("PER-UNIT STATE (nl)", + rows["inverter-b"]["now"] == "Terug onder de sturing van SMA") +homeview._LANG[0] = "en" +check("the MQTT status word is 'stopped' as before", + (c.publish(), p.last.get("status"))[1] == "stopped") +check("the log says which unit was handed back", + any("inverter-b: handed back" in str(e) for e in c.events)) +check("the two new Last-event hints exist and are plain", + messages.hint("handed_back", unit="inverter-b").endswith("to it") + and messages.hint("hand_back_failed", unit="inverter-b") + .endswith("trying again")) + +# --- 3j. the call sites ------------------------------------------------------- +src = (HERE / "app" / "main.py").read_text(encoding="utf-8") +ss = src.splitlines() + + +def calls_in(name): + """Line numbers of every `await self.()` in main.py.""" + return [i + 1 for i, ln in enumerate(ss) if f"await self.{name}()" in ln] + + +rel, zer = calls_in("command_site_release"), calls_in("command_site_zero") +print(f" main.py call sites: command_site_release at lines {rel}; " + f"command_site_zero at lines {zer}") +check("main.py: the Stop button and shutdown() call command_site_release(); " + "the two maintenance-abort paths keep command_site_zero() (a third call " + "is the all-GoodWe branch inside command_site_release itself)", + len(rel) == 2 and len(zer) == 3 + and "async def command_site_release" in "\n".join(ss[zer[1] - 30:zer[1]])) + +# ============================================================================= +print() +print("4. the real main() in a child process") + + +def events_of(ev, who, what): + return [e for e in ev if e[0] == who and e[1] == what] + + +rc, err, ev = run_child("sigterm") +print(f" SIGTERM: exit status {rc}; fake events " + f"{[e[1:] for e in ev if e[0] == 'fake']}") +i_sig = ev.index(["sigterm"]) if ["sigterm"] in ev else None +after = ev[i_sig + 1:] if i_sig is not None else [] +check("SIGTERM: the process ends cleanly (0)", rc == 0) +check("SIGTERM: the fake unit is released exactly once, after the signal, and " + "written nothing after that", + len(events_of(after, "fake", "release")) == 1 + and [e for e in after if e[0] == "fake"][-1] == ["fake", "release"] + and ["cleanup"] in after) +check("SIGTERM: the GoodWe unit gets 0 W on the way out", + [e for e in after if e[0] == "goodwe"][-1] == ["goodwe", "write", 0.0]) +check("SIGTERM: the fake unit was written before the signal (the test is not " + "vacuous)", len(events_of(ev[:i_sig], "fake", "write")) >= 1) +check("SIGTERM: the log says what happened to each unit", + "shutting down - commanding 0 W on inverter-a and handing back " + "inverter-b to SMA control" in err) + +rc, err, ev = run_child("fault") +i_f = ev.index(["fault"]) if ["fault"] in ev else None +after = ev[i_f + 1:] if i_f is not None else [] +print(f" task death: exit status {rc}; after the fault " + f"{[e[1:] for e in after if e[0] in ('fake', 'goodwe')]}") +check("TASK DEATH: exit status 70", rc == 70) +check("TASK DEATH: the fake unit is released exactly once and nothing is " + "written to it after; the GoodWe unit is at 0 W", + len(events_of(after, "fake", "release")) == 1 + and [e for e in after if e[0] == "fake"][-1] == ["fake", "release"] + and [e for e in after if e[0] == "goodwe"][-1] == ["goodwe", "write", 0.0]) +check("TASK DEATH: the CRITICAL line says 0 W for one and hand-back for the " + "other", + "so inverter-a is set to 0 W and inverter-b is handed back and the " + "add-on exits with code 70" in err) + +rc, err, ev = run_child("refuse") +print(f" late config refusal: exit status {rc}; events " + f"{[e[1:] for e in ev if e[0] in ('fake', 'goodwe')]}") +check("CONFIG REFUSAL after the drivers exist: exit status 78, the fake unit " + "released exactly once, never written", + rc == 78 and len(events_of(ev, "fake", "release")) == 1 + and not events_of(ev, "fake", "write")) +check("CONFIG REFUSAL: the GoodWe unit is at 0 W", + events_of(ev, "goodwe", "write") + and events_of(ev, "goodwe", "write")[-1] == ["goodwe", "write", 0.0]) + +rc, err, ev = run_child("badcfg") +print(f" startup refusal: exit status {rc}; events {ev}") +check("CONFIG REFUSAL before any driver exists: exit 78 as today, and nothing " + "is released or written", rc == 78 and not ev) + +# ============================================================================= +print() +if fails: + print(f"{len(fails)} FAILED:") + for n in fails: + print(" -", n) + sys.exit(1) +print("all passed") + +# --------------------------------------------------------------------------- +# MUTANTS this file was run against (one edit each to a copy of the tree; every +# one turns it red). The two the card names are the first two. +# a. a safety stop (a 0 W write while control runs) calls release(): the +# idle test in Controller._write_unit gains `or value == 0.0`. +# b. the heartbeat (and the cycle) write to a released unit: the `return` +# after the idle hand-back in _write_unit is removed. +# c. the unit is released twice (Stop, then shutdown): the `if inv.released: +# return True` guard in _hand_back is removed. +# d. the Stop button calls command_site_zero() instead of the release. +# e. shutdown() calls command_site_zero() instead of the release. +# f. shadow mode is not idle for a hand-back unit. +# g. Start does not re-arm a released unit. +# h. the release is awaited without a bound (a hanging unit holds shutdown). +# i. a raising release() is not caught (it stops the others). +# j. a GoodWe unit is "released" instead of zeroed on Stop. +# k. the release does not wait for a write on its way (it is not the last). +# l. the hand-back words are hard-coded "SMA control", not the driver's. +# m. a GoodWe-only roster gets a changed banner. -- 2.54.0 From d26cc573fe91cdf8cda636d1d53e44457187c18b Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Sat, 7 Nov 2026 20:17:01 +0100 Subject: [PATCH 17/22] SMA-10 round 2: a failed release sends 0 W once, retries at heartbeat pace When release() fails (False, raises or hangs past RELEASE_BUDGET_S) the unit gets one bounded write(0) through the new Driver.release_fallback() hook (default write(0)), then nothing more, so the inverter's own fallback timer is not reset. The release is retried once per heartbeat, not on every idle write. One error line and one action-log event per failure streak, one line on recovery. Dutch default "terug naar de eigen sturing". DECISIONS says what a unit is left at. test_messages.py: the RELEASE_BUDGET_S data entry is gone again (the line that needed it is gone). Co-Authored-By: Claude --- .../DECISIONS/SMA-10-two-idle-states.md | 9 ++ goodwe_controller/app/drivers.py | 8 + goodwe_controller/app/handback.py | 2 +- goodwe_controller/app/inverter.py | 6 + goodwe_controller/app/main.py | 55 +++++-- goodwe_controller/app/messages.py | 2 +- goodwe_controller/test_messages.py | 2 +- .../test_sma10_two_idle_states.py | 142 +++++++++++++++++- 8 files changed, 205 insertions(+), 21 deletions(-) diff --git a/goodwe_controller/DECISIONS/SMA-10-two-idle-states.md b/goodwe_controller/DECISIONS/SMA-10-two-idle-states.md index 778b1b6..4f8d701 100644 --- a/goodwe_controller/DECISIONS/SMA-10-two-idle-states.md +++ b/goodwe_controller/DECISIONS/SMA-10-two-idle-states.md @@ -49,3 +49,12 @@ after `release()`, per-driver timing, DOCS.md's safety model and the "control stalled" sentence, the site-level "Held at 0 W" tiles in shadow mode on a mixed roster, and the MQTT per-unit setpoint of a handed-back unit (it still publishes 0). + +**After a failed release** (returns False, raises or hangs past the bound) the +unit gets one `write(0)` through `Driver.release_fallback()` (default +`write(0)`; a driver where 0 W is not always safe overrides it), then nothing: +a repeated 0 W would keep resetting the inverter's own fallback timer, so after +that the unit is left to its own fallback. The release is retried once per +heartbeat; one error line and one action-log event per failure streak, one line +on recovery. Worst case at shutdown is `RELEASE_BUDGET_S` + `FALLBACK_BUDGET_S` +(6 s). diff --git a/goodwe_controller/app/drivers.py b/goodwe_controller/app/drivers.py index 915a891..8d607d8 100644 --- a/goodwe_controller/app/drivers.py +++ b/goodwe_controller/app/drivers.py @@ -160,6 +160,14 @@ class Driver: never a release: that is `write(0)`.""" return await self.write(0.0) + async def release_fallback(self) -> bool: + """SMA-10 round 2. What to send, ONCE, when `release()` has failed + (False, raised, or hung past the bound), so the unit does not carry its + last command while nothing else steers it. Default: `write(0)`. It is + not repeated: a repeat would keep resetting the inverter's own fallback + timer. A driver for which 0 W is not always a safe hold overrides this.""" + return await self.write(0.0) + async def probe(self) -> Probe: """A read-only look, usable without a Controller (panel, wizard).""" raise NotImplementedError diff --git a/goodwe_controller/app/handback.py b/goodwe_controller/app/handback.py index b0c5a74..8f2c819 100644 --- a/goodwe_controller/app/handback.py +++ b/goodwe_controller/app/handback.py @@ -27,7 +27,7 @@ import re from dataclasses import dataclass _BACK_EN = "its own control" -_BACK_NL = "eigen sturing" +_BACK_NL = "de eigen sturing" @dataclass(frozen=True) diff --git a/goodwe_controller/app/inverter.py b/goodwe_controller/app/inverter.py index 69161d0..7af5188 100644 --- a/goodwe_controller/app/inverter.py +++ b/goodwe_controller/app/inverter.py @@ -284,6 +284,12 @@ class Inverter: # SMA-10. Writes to a hand-back driver that are on their way (see # Controller._write_unit), so the release can wait for them and be LAST. self.inflight: set = set() + # SMA-10 round 2. A failed hand-back: how many tries in a row failed + # (the log is written at the first and on recovery only), when the next + # try is due (monotonic s), and whether the one fallback 0 W was sent. + self.release_fails = 0 + self.release_retry_at = 0.0 + self.release_zero_sent = False self.last_write_ok = None # ⚠️ TWO TIMESTAMPS, NOT ONE FLAG. `last_write_ok` is the last write's # verdict and it flips on a single transient 400; liveness needs a RUN diff --git a/goodwe_controller/app/main.py b/goodwe_controller/app/main.py index a34cc7f..afeb1a3 100644 --- a/goodwe_controller/app/main.py +++ b/goodwe_controller/app/main.py @@ -147,6 +147,12 @@ WRITE_BUDGET_S = 4.0 # to it. A release still running at the bound is abandoned, not awaited. RELEASE_BUDGET_S = WRITE_BUDGET_S +# SMA-10 round 2. The one fallback write after a failed release (default 0 W, +# `Driver.release_fallback()`). Worst case for a unit whose release hangs: +# RELEASE_BUDGET_S + FALLBACK_BUDGET_S = 6 s, against the Supervisor's 10 s. +FALLBACK_BUDGET_S = 2.0 +HAND_BACK_WORST_S = RELEASE_BUDGET_S + FALLBACK_BUDGET_S + # SAFETY-34: a Core-only restart wipes persistent_notification and does NOT # restart this add-on (measured on TEST-23), so a latched alarm is re-asserted # this often. Why: goodwe docs/DECISIONS.md, SAFETY-34. @@ -4065,7 +4071,7 @@ class Controller: await asyncio.wait(pending, timeout=RELEASE_BUDGET_S / 2) return await self._driver(inv).release() - async def _hand_back(self, inv) -> bool: + async def _hand_back(self, inv, *, force: bool = False) -> bool: """SMA-10. Give ONE unit back: exactly one `release()`, then nothing. True when the unit is handed back (now, or already). Never raises: a @@ -4086,27 +4092,54 @@ class Controller: return True if not inv.driver.configured(): return False + # After a failure the next try waits one heartbeat, so a hung release + # costs a cycle its 4 s once per heartbeat, not on every cycle. The + # Stop button and shutdown (`force`) always try. + now_m = time.monotonic() + if not force and now_m < inv.release_retry_at: + return False inv.released = True + why = "" try: ok = bool(await _bounded(self._release_last(inv), RELEASE_BUDGET_S)) except (asyncio.TimeoutError, TimeoutError): - ok = False - _LOG.error("Handing back %s took over %.0f s and was dropped. It " - "is tried again.", inv.name, RELEASE_BUDGET_S) + ok, why = False, f"took over {RELEASE_BUDGET_S:.0f} s" except Exception as err: # noqa: BLE001 - one unit never stops the rest - ok = False - _LOG.error("Handing back %s failed: %s. It is tried again.", - inv.name, err) + ok, why = False, f"{type(err).__name__}: {err}" if ok: inv.last_write = None + if inv.release_fails: + _LOG.info("Handing back %s worked again after %d failed " + "tries.", inv.name, inv.release_fails) + inv.release_fails = 0 + inv.release_retry_at = 0.0 + inv.release_zero_sent = False self.log_event(f"{inv.name}: handed back", hint=messages.hint("handed_back", unit=inv.name)) - else: - inv.released = False + return True + inv.released = False + inv.release_fails += 1 + inv.release_retry_at = now_m + float(self.o.get("heartbeat_s", 10)) + _LOG.debug("Handing back %s failed (try %d): %s", inv.name, + inv.release_fails, why) + if inv.release_fails == 1: # one line per failure streak + _LOG.error("Handing back %s failed, so it is sent 0 W once and " + "handing back is tried again.", inv.name) self.log_event(f"{inv.name}: hand-back failed", hint=messages.hint("hand_back_failed", unit=inv.name)) - return ok + if not inv.release_zero_sent: + # ⚠️ ONCE. The unit must not keep carrying its last command, but a + # 0 W re-sent on every try would reset the inverter's own fallback + # timer for ever. The driver says what the one write is. + inv.release_zero_sent = True + try: + if await _bounded(self._driver(inv).release_fallback(), + FALLBACK_BUDGET_S): + inv.last_write = 0.0 + except Exception as err: # noqa: BLE001 + _LOG.debug("The fallback write to %s failed: %r", inv.name, err) + return False async def _write_unit(self, inv, value: float, *, force: bool = False) -> None: if not inv.driver.configured(): @@ -7176,7 +7209,7 @@ class Controller: for inv in self.inverters: inv.target = 0.0 await asyncio.gather(*( - self._hand_back(inv) if inv.driver.hands_back + self._hand_back(inv, force=True) if inv.driver.hands_back else self._write_unit(inv, 0.0, force=True) for inv in self.inverters)) diff --git a/goodwe_controller/app/messages.py b/goodwe_controller/app/messages.py index 3e24bc1..0c234e1 100644 --- a/goodwe_controller/app/messages.py +++ b/goodwe_controller/app/messages.py @@ -1389,7 +1389,7 @@ HINTS = { "roster": "Inverter list read: {n} inverters", "maint_history": "Maintenance history did not move over cleanly", "handed_back": "{unit}: handed back, the add-on writes nothing to it", - "hand_back_failed": "{unit}: could not be handed back, trying again", + "hand_back_failed": "{unit}: could not be handed back, set to 0 W once, trying again", "stop_restored": "{unit}: an earlier safety stop is back, it stays at 0 W", "readings_late": "Home Assistant readings arrive late, control holds", "readings_back": "Home Assistant readings are on time again", diff --git a/goodwe_controller/test_messages.py b/goodwe_controller/test_messages.py index 3636a2a..c007134 100644 --- a/goodwe_controller/test_messages.py +++ b/goodwe_controller/test_messages.py @@ -851,7 +851,7 @@ DATA = { "main.py:inv.batt_fresh.bound.release_s", "main.py:inv.batt_fresh.bound.move_w", "main.py:inv.batt_fresh.bound.unchanged_s", "main.py:inv.batt_fresh.held", "main.py:value", "main.py:float(value)", - "main.py:entity", "main.py:WRITE_BUDGET_S", "main.py:RELEASE_BUDGET_S", + "main.py:entity", "main.py:WRITE_BUDGET_S", "main.py:_e", "main.py:messages.times(self.clamp_wd.engagements)", "main.py:messages.when(brussels(self.tariff_at))", diff --git a/goodwe_controller/test_sma10_two_idle_states.py b/goodwe_controller/test_sma10_two_idle_states.py index 7bad08b..2d4d614 100644 --- a/goodwe_controller/test_sma10_two_idle_states.py +++ b/goodwe_controller/test_sma10_two_idle_states.py @@ -336,6 +336,23 @@ def run(coro): return asyncio.run(coro) +_real_time = M.time + + +class _FakeTime: + """A clock the test moves; everything else is the real `time`.""" + t = 1000.0 + + def monotonic(self): + return self.t + + def __getattr__(self, name): + return getattr(_real_time, name) + + +FT = _FakeTime() + + def gw(name="inverter-a", suffix="a"): return dict(name=name, setpoint_entity=f"number.sp_{suffix}", soc_entity=f"sensor.soc_{suffix}", @@ -853,10 +870,11 @@ run(c.command_site_release()) took = time.monotonic() - t0 print(f" command_site_release() with one raising, one hanging and one " f"failing unit: {took:.2f} s " - f"(bound {M.RELEASE_BUDGET_S:.1f} s)") -check("the bound: it returns within RELEASE_BUDGET_S plus a margin, however " - "many units hang (they run together)", - took < M.RELEASE_BUDGET_S + 1.5) + f"(release {M.RELEASE_BUDGET_S:.1f} s + fallback {M.FALLBACK_BUDGET_S:.1f}" + f" s = {M.HAND_BACK_WORST_S:.1f} s at most)") +check("the bound: it returns within RELEASE_BUDGET_S + FALLBACK_BUDGET_S plus " + "a margin, however many units hang (they run together)", + took < M.HAND_BACK_WORST_S + 1.5) check("the healthy unit was released (once) and the GoodWe unit zeroed, " "though three others raised, failed or hung", n_calls(fd_ok, "release") == 1 and c.inverters[3].released @@ -865,9 +883,13 @@ check("the units that raised, failed or hung are NOT marked handed back, so " "they are tried again", not any(c.inverters[i].released for i in (1, 2, 4))) fb_raise.mode = "ok" +FT.t = _real_time.monotonic() + 11.0 # one heartbeat later +M.time = FT run(c.send_heartbeats(60_000.0, 10.0)) -check("...and the next idle write retries the one that raised (now answers): " - "released, and the healthy unit is not released a second time", +M.time = _real_time +check("...and the next idle write (a heartbeat later) retries the one that " + "raised (now answers): released, and the healthy unit is not released a " + "second time", n_calls(fb_raise, "release") == 2 and c.inverters[1].released and n_calls(fd_ok, "release") == 1) c, h, p = site([gw("inverter-a", "a"), fk("inverter-b"), @@ -879,9 +901,111 @@ run(c.shutdown()) took = time.monotonic() - t0 check("shutdown() with one unit hanging finishes inside the bound, the other " "unit is released and the GoodWe unit at 0 W", - took < M.RELEASE_BUDGET_S + 1.5 and n_calls(fc_ok, "release") == 1 + took < M.HAND_BACK_WORST_S + 1.5 and n_calls(fc_ok, "release") == 1 and h.writes()[-1] == 0.0) +# --- 3h2. a failed release: one 0 W, retries at heartbeat pace, logged once --- +print(" (a failed release: the unit carries 1120 W)") +_cap = [] + + +class _Cap2(logging.Handler): + def emit(self, rec): + _cap.append((rec.levelno, rec.getMessage())) + + +_h2 = _Cap2() +logging.getLogger("goodwe").addHandler(_h2) + + +def failing_site(mode): + c, h, p = site([gw("inverter-a", "a"), fk("inverter-b")]) + f = fakes(c)[0] + run(cycles(c, 5)) + f.power = 1120.0 # the unit carries 1120 W + f.mode = mode + return c, h, f + + +for mode in ("false", "raise", "hang"): + c, h, f = failing_site(mode) + n_w = len(writes_of(f)) + t0 = time.monotonic() + run(c.handle_action("auto_toggle")) # Stop + took = time.monotonic() - t0 + print(f" Stop, release {mode}: {took:.2f} s; calls after: " + f"{[x for x in f.calls[-4:]]}") + check(f"FAILED RELEASE ({mode}): the unit that carried 1120 W gets exactly " + f"one 0 W write after the release, inside the bound", + writes_of(f)[n_w:] == [0.0] and n_calls(f, "release") == 1 + and took < M.HAND_BACK_WORST_S + 1.5 + and not c.inverters[1].released) + run(c.send_heartbeats(10_000.0, 10.0)) + run(cycles(c, 3)) + check(f"FAILED RELEASE ({mode}): heartbeats and cycles in the same moment " + f"add no write and no retry", + writes_of(f)[n_w:] == [0.0] and n_calls(f, "release") == 1) + +# the cadence and the log, over N failed retries on a clock we move +c, h, f = failing_site("false") +M.time = FT +FT.t = 5000.0 +_cap.clear() +run(c.handle_action("auto_toggle")) # Stop: try 1 (forced) +n_w = len(writes_of(f)) +N = 6 +for k in range(N): + for step in range(5): # idle writes inside a heartbeat + FT.t += 1.9 + run(c.send_heartbeats(20_000.0 + FT.t, 10.0)) + run(cycles(c, 1)) + FT.t += 1.0 # the heartbeat is due + run(c.send_heartbeats(40_000.0 + FT.t, 10.0)) +print(f" {N} heartbeats: release() called {n_calls(f, 'release')} times, " + f"0 W writes after the first: {writes_of(f)[n_w:]}") +check("CADENCE: one try at Stop plus one per heartbeat, not one per idle " + "write", n_calls(f, "release") == 1 + N) +check("ONE 0 W only, however many retries fail: the inverter's fallback timer " + "is not reset", writes_of(f)[n_w - 1:] == [0.0]) +errs = [m for lv, m in _cap if lv >= logging.ERROR and "inverter-b" in m] +evs = [e for e in c.events if "hand-back failed" in str(e)] +check(f"LOG: {N + 1} failed tries make ONE error line and ONE action-log event", + len(errs) == 1 and len(evs) == 1) +f.mode = "ok" +FT.t += 11.0 +run(c.send_heartbeats(90_000.0, 10.0)) +for _ in range(3): + FT.t += 11.0 + run(c.send_heartbeats(95_000.0 + FT.t, 10.0)) +inf = [m for lv, m in _cap if "worked again" in m] +check("RECOVERY: logged once ('worked again after 7 failed tries'), released, " + "and no further release() calls", + len(inf) == 1 and "7 failed" in inf[0] and c.inverters[1].released + and n_calls(f, "release") == 2 + N + and len([e for e in c.events if "inverter-b: handed back" in str(e)]) == 1) +run(c.handle_action("auto_toggle")) # Start +run(cycles(c, 3)) +f.mode = "false" +_cap.clear() +run(c.handle_action("auto_toggle")) # Stop, fails again +check("a NEW failure streak is logged again, once", + len([1 for lv, m in _cap if lv >= logging.ERROR]) == 1) +M.time = _real_time + +# shutdown with a failed release: the one 0 W still goes out inside the bound +c, h, f = failing_site("hang") +n_w = len(writes_of(f)) +t0 = time.monotonic() +run(c.shutdown()) +took = time.monotonic() - t0 +print(f" shutdown, release hangs: {took:.2f} s (worst {M.HAND_BACK_WORST_S:.1f} s)") +check("SHUTDOWN with a failed release: the one 0 W is sent, inside the bound", + writes_of(f)[n_w:] == [0.0] and took < M.HAND_BACK_WORST_S + 1.5) +check("a driver's own fallback replaces the 0 W (the hook), not the add-on", + "release_fallback" in dir(HB) and HB.release_fallback is + drivers.Driver.release_fallback) +logging.getLogger("goodwe").removeHandler(_h2) + # --- 3i. per-unit state and the other counts --------------------------------- c, h, p = site([gw("inverter-a", "a"), fk("inverter-b")]) run(cycles(c, 4)) @@ -1021,3 +1145,7 @@ print("all passed") # k. the release does not wait for a write on its way (it is not the last). # l. the hand-back words are hard-coded "SMA control", not the driver's. # m. a GoodWe-only roster gets a changed banner. +# n. (round 2) the fallback 0 W is sent on every failed try. +# o. (round 2) a failed release is retried on every idle write. +# p. (round 2) every failed try is logged. +# q. (round 2) no fallback 0 W after a failed release. -- 2.54.0 From 313503724d7ad84469ace48511e93e7c5cb41548 Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Sat, 7 Nov 2026 21:31:28 +0100 Subject: [PATCH 18/22] SMA-10 round 3: Start clears a failed hand-back streak; the hook is tested When control takes the unit again, release_fails, release_retry_at and release_zero_sent are reset, so the next Stop is a new streak with its own one 0 W, error line and event. Tests: a streak that Start interrupts without a recovery; a driver that overrides release_fallback(). _BACK_NL "de eigen sturing" (round 2) is kept. Co-Authored-By: Claude --- goodwe_controller/app/main.py | 8 ++- .../test_sma10_two_idle_states.py | 54 +++++++++++++++++++ 2 files changed, 61 insertions(+), 1 deletion(-) diff --git a/goodwe_controller/app/main.py b/goodwe_controller/app/main.py index afeb1a3..6b2e6fe 100644 --- a/goodwe_controller/app/main.py +++ b/goodwe_controller/app/main.py @@ -4162,9 +4162,15 @@ class Controller: if self.stopping or self.shadow or not self.auto: await self._hand_back(inv) return - if inv.released: + if inv.released or inv.release_fails or inv.release_zero_sent: + # Control has the unit again: forget the hand-back, and any + # failed streak with it, so the next Stop starts clean (its own + # one 0 W, its own error line and event). inv.released = False inv.last_write = None + inv.release_fails = 0 + inv.release_retry_at = 0.0 + inv.release_zero_sent = False entity = inv.driver.actuator_id() # for the log line below # --- SAFETY-03: THE CLAMP, AND IT IS ON THE ONLY PATH TO THE WIRE ----- diff --git a/goodwe_controller/test_sma10_two_idle_states.py b/goodwe_controller/test_sma10_two_idle_states.py index 2d4d614..4990941 100644 --- a/goodwe_controller/test_sma10_two_idle_states.py +++ b/goodwe_controller/test_sma10_two_idle_states.py @@ -1004,6 +1004,58 @@ check("SHUTDOWN with a failed release: the one 0 W is sent, inside the bound", check("a driver's own fallback replaces the 0 W (the hook), not the add-on", "release_fallback" in dir(HB) and HB.release_fallback is drivers.Driver.release_fallback) + +# a failed streak that Start interrupts WITHOUT a recovery: the next Stop is a +# new streak with its own one 0 W, its own error line and its own event +c, h, f = failing_site("false") +_cap.clear() +run(c.handle_action("auto_toggle")) # Stop, release fails +w1 = writes_of(f).count(0.0) +run(c.handle_action("auto_toggle")) # Start, no recovery +iv = c.inverters[1] +run(cycles(c, 5)) +nz = [w for w in writes_of(f) if w] +check("START after a failed release: control writes non-zero again and the " + "failed-streak state is cleared", + nz and writes_of(f)[-1] != 0.0 and iv.release_fails == 0 + and not iv.release_zero_sent and iv.release_retry_at == 0.0) +n_before = len(writes_of(f)) +run(c.handle_action("auto_toggle")) # Stop, release fails again +after = writes_of(f)[n_before:] +errs = [m for lv, m in _cap if lv >= logging.ERROR and "inverter-b" in m] +evs = [e for e in c.events if "hand-back failed" in str(e)] +check("SECOND STREAK (Start came between, no recovery): the unit gets its own " + "one 0 W write", + after == [0.0] and w1 == 1) +check("SECOND STREAK: a second error line and a second action-log event", + len(errs) == 2 and len(evs) == 2) +run(c.send_heartbeats(70_000.0, 10.0)) +check("...and still only the one 0 W in that streak", + writes_of(f)[n_before:] == [0.0]) + +# the hook: a driver's own fallback replaces the 0 W, and _hand_back does not +# call write(0) itself +class HookHB(HB): + name = "hookhb" + + async def release_fallback(self): + self.calls.append(("fallback", None)) + return await self.write(5.0) # a non-zero safe value + + +REGISTRY["hookhb"] = HookHB +c, h, p = site([gw("inverter-a", "a"), fk("inverter-b", "hookhb")]) +f = fakes(c)[0] +run(cycles(c, 4)) +f.mode = "false" +n0 = len(f.calls) +run(c.handle_action("auto_toggle")) +tail = [x for x in f.calls[n0:] if x[0] in ("release", "fallback", "write")] +print(f" a driver with its own fallback: {tail}") +check("HOOK: after a failed release the driver's release_fallback() is called " + "once, its own value (5 W) is the only write, and the add-on does not " + "write 0 W itself", + tail == [("release", None), ("fallback", None), ("write", 5.0)]) logging.getLogger("goodwe").removeHandler(_h2) # --- 3i. per-unit state and the other counts --------------------------------- @@ -1149,3 +1201,5 @@ print("all passed") # o. (round 2) a failed release is retried on every idle write. # p. (round 2) every failed try is logged. # q. (round 2) no fallback 0 W after a failed release. +# r. (round 3) Start does not clear a failed streak. +# s. (round 3) _hand_back writes 0 W itself instead of calling the hook. -- 2.54.0 From 6cae5a2f8f71e04ad3a670c533c3e098c43a7f19 Mon Sep 17 00:00:00 2001 From: glenn schrooyen Date: Sat, 7 Nov 2026 22:45:08 +0100 Subject: [PATCH 19/22] SMA-10 round 4: pin the Dutch default "de eigen sturing" The NL default for a hand-back driver without words (handback._BACK_NL) was changed to "de eigen sturing" in round 2 at the PR #239 review's request (comment 2495: the article as in the generic clause). The brief does not name it, so DECISIONS now says where it came from and a test checks the exact Dutch sentence. No app code change. Co-Authored-By: Claude --- goodwe_controller/DECISIONS/SMA-10-two-idle-states.md | 4 +++- goodwe_controller/test_sma10_two_idle_states.py | 7 +++++++ 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/goodwe_controller/DECISIONS/SMA-10-two-idle-states.md b/goodwe_controller/DECISIONS/SMA-10-two-idle-states.md index 4f8d701..519800e 100644 --- a/goodwe_controller/DECISIONS/SMA-10-two-idle-states.md +++ b/goodwe_controller/DECISIONS/SMA-10-two-idle-states.md @@ -40,7 +40,9 @@ exists has no driver to release and keeps today's path. for shadow mode is built from the roster in `app/handback.py` (EN and NL): a roster with no hand-back unit gets None and keeps its old literal, so a GoodWe site sees no changed byte. A hand-back unit's words (`Driver.words`: `back`, -`back_nl`) come from its driver. MQTT `status` is unchanged; the Home battery +`back_nl`) come from its driver; one without them gets "its own control" / +"de eigen sturing" (the article as in the generic clause; PR #239 review +comment 2495, not the brief). MQTT `status` is unchanged; the Home battery row of a handed-back unit says "Back under SMA control" and carries `handed_back`. diff --git a/goodwe_controller/test_sma10_two_idle_states.py b/goodwe_controller/test_sma10_two_idle_states.py index 4990941..d88d2f1 100644 --- a/goodwe_controller/test_sma10_two_idle_states.py +++ b/goodwe_controller/test_sma10_two_idle_states.py @@ -614,6 +614,13 @@ check("a hand-back driver with no words gets a plain brand-free phrase", handback.stop_dialog((handback.unit_of(bare_inv),), "en") == "The inverter goes back to its own control until you start control " "again.") +# NL default "de eigen sturing" (PR #239 review comment 2495): same article as +# the generic clause in handback._clause. +check("NL: a hand-back driver with no words goes back to 'de eigen sturing'", + handback._BACK_NL == "de eigen sturing" + and handback.stop_dialog((handback.unit_of(bare_inv),), "nl") == + "De omvormer gaat terug naar de eigen sturing tot u de regeling " + "opnieuw start.") long_roster = tuple(handback.Unit(f"inverter-{c}", i % 2 == 1, "SMA control", "de sturing van SMA") for i, c in enumerate("abcdefg")) -- 2.54.0 From 8953be03626234ee5c31f53466bb7e0c5c6d1bab Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 8 Nov 2026 02:39:38 +0100 Subject: [PATCH 20/22] SAFETY-137: the shutdown and Stop 0 W is the last write every unit hears The 0 W of shutdown and of the Stop button now waits (bounded, 2 s) for a write already on its way to the same unit, for every driver, GoodWe included. Once shutdown has begun, cycle, heartbeat and safety-stop writes are dropped right before the write is created. One shared wait (_await_inflight) serves the 0 W and SMA-10's _release_last. Co-Authored-By: Claude --- goodwe_controller/CHANGELOG.md | 1 + .../SAFETY-137-last-write-is-zero.md | 21 ++ goodwe_controller/app/main.py | 60 +++- .../test_safety137_last_write.py | 332 ++++++++++++++++++ .../test_sma10_two_idle_states.py | 5 +- 5 files changed, 402 insertions(+), 17 deletions(-) create mode 100644 goodwe_controller/DECISIONS/SAFETY-137-last-write-is-zero.md create mode 100644 goodwe_controller/test_safety137_last_write.py diff --git a/goodwe_controller/CHANGELOG.md b/goodwe_controller/CHANGELOG.md index 5d02a9f..67ecef3 100644 --- a/goodwe_controller/CHANGELOG.md +++ b/goodwe_controller/CHANGELOG.md @@ -2,6 +2,7 @@ ### Not in a release yet +- When the add-on stops, a command that was already on its way can no longer reach an inverter after the 0 W. The 0 W is now the last thing every inverter gets, also when Stop is pressed ([[SAFETY-137]]). - When the add-on stops, an inverter that can steer itself is handed back instead of held at 0 W. Safety stops still hold 0 W, and a GoodWe site is not affected ([[SMA-10]]). - Each inverter is now reached through a driver, so another make can be added later. Nothing changes for a GoodWe site ([[SMA-2]]). - Home now draws today's charge from the grid for every hour the add-on will really charge, not only the run under way ([[UX-77]]). diff --git a/goodwe_controller/DECISIONS/SAFETY-137-last-write-is-zero.md b/goodwe_controller/DECISIONS/SAFETY-137-last-write-is-zero.md new file mode 100644 index 0000000..c558f07 --- /dev/null +++ b/goodwe_controller/DECISIONS/SAFETY-137-last-write-is-zero.md @@ -0,0 +1,21 @@ +# SAFETY-137 - the shutdown 0 W is the last write every unit hears + +SMA-10 made a hand-back release wait for a write already on its way +(`_release_last`, `Inverter.inflight`). A GoodWe unit skipped that, and rig 5b +saw a cycle write (-2 823 W) land after the shutdown 0 W and stay until relaunch. + +Now the 0 W of shutdown and of the Stop button is a *last* write +(`_write_unit(last=True)`), for every driver: + +- It first waits, at most `SETTLE_BUDGET_S` (2 s), for the writes already on + its way to that unit (`_await_inflight`, shared with `_release_last`). A hung + write is not waited out and not cancelled; the 0 W goes anyway. +- Once shutdown has begun (`stopping`), every other write (cycle, heartbeat, + safety stop) is dropped at the same place, right before the write is created. + That is the gate that keeps the control cycle quiet; the cycle itself needs + no cancelling. +- Worst case for a GoodWe unit is 2 s + `WRITE_BUDGET_S` (4 s) = 6 s, the same + as `HAND_BACK_WORST_S`; the units are handled together. + +Not touched: the heartbeat's own timing, safety-stop paths, the maintenance +abort (still a plain 0 W). diff --git a/goodwe_controller/app/main.py b/goodwe_controller/app/main.py index 6b2e6fe..25def79 100644 --- a/goodwe_controller/app/main.py +++ b/goodwe_controller/app/main.py @@ -153,6 +153,13 @@ RELEASE_BUDGET_S = WRITE_BUDGET_S FALLBACK_BUDGET_S = 2.0 HAND_BACK_WORST_S = RELEASE_BUDGET_S + FALLBACK_BUDGET_S +# SAFETY-137. How long a LAST write (the Stop 0 W, the shutdown 0 W, a release) +# waits for a write already on its way to the same unit. Half the release +# budget, as `_release_last` always waited. For a GoodWe unit that is +# SETTLE_BUDGET_S + WRITE_BUDGET_S = 6 s at worst, which is HAND_BACK_WORST_S: +# the wind-down is no slower than SMA-10's worst case. +SETTLE_BUDGET_S = RELEASE_BUDGET_S / 2 + # SAFETY-34: a Core-only restart wipes persistent_notification and does NOT # restart this add-on (measured on TEST-23), so a latched alarm is re-asserted # this often. Why: goodwe docs/DECISIONS.md, SAFETY-34. @@ -4020,7 +4027,7 @@ class Controller: return rec.authorises(inv.name, value, self.alloc_tick) async def write_setpoint(self, value: float, *, force: bool = False, - unit=None) -> None: + unit=None, last: bool = False) -> None: """Put `value` on ONE inverter, or - with `unit=None` - on all of them. ⚠️ `unit=None` IS FOR A NUMBER THAT IS NOT A SHARE. The stop button, @@ -4059,16 +4066,23 @@ class Controller: targets = [unit] if not targets: return - await asyncio.gather(*(self._write_unit(inv, value, force=force) + await asyncio.gather(*(self._write_unit(inv, value, force=force, + last=last) for inv in targets)) + async def _await_inflight(self, inv, budget_s: float) -> None: + """Wait, at most `budget_s`, for the writes already on their way to + `inv` (SAFETY-137: every driver, GoodWe too). A write still running at + the bound is left alone, not cancelled: its own WRITE_BUDGET_S ends it.""" + pending = [t for t in inv.inflight if not t.done()] + if pending: + await asyncio.wait(pending, timeout=budget_s) + async def _release_last(self, inv) -> bool: """`release()`, after a write already on its way to `inv` has finished (at most half the budget is spent waiting for it), so the release is the LAST thing the unit hears and a late write cannot undo it.""" - pending = [t for t in inv.inflight if not t.done()] - if pending: - await asyncio.wait(pending, timeout=RELEASE_BUDGET_S / 2) + await self._await_inflight(inv, SETTLE_BUDGET_S) return await self._driver(inv).release() async def _hand_back(self, inv, *, force: bool = False) -> bool: @@ -4141,7 +4155,8 @@ class Controller: _LOG.debug("The fallback write to %s failed: %r", inv.name, err) return False - async def _write_unit(self, inv, value: float, *, force: bool = False) -> None: + async def _write_unit(self, inv, value: float, *, force: bool = False, + last: bool = False) -> None: if not inv.driver.configured(): return # --- SMA-10: A UNIT THAT HANDS BACK IS NOT WRITTEN WHILE THE ADD-ON IS @@ -4284,6 +4299,21 @@ class Controller: value, inv.name, shadow.WIRE_W) value = shadow.WIRE_W + # --- SAFETY-137: THE 0 W THAT ENDS THINGS IS THE LAST WRITE THE UNIT + # HEARS. `last` is the Stop 0 W and the shutdown 0 W (and nothing else). + # It first waits, bounded, for a write already on its way to this unit, + # so that write cannot land after it. And once shutdown has begun every + # other write (cycle, heartbeat, a safety stop) is dropped here: the + # check sits right before the write is created with no await between, + # so a write that starts after it cannot exist, and one that started + # before is in `inflight` for the wait above. The idle gate for a + # hand-back unit is higher up and does the same for those. + if last: + await self._await_inflight(inv, SETTLE_BUDGET_S) + elif self.stopping: + _LOG.debug("Shutting down, so %+.0f W to %s is not sent.", + value, inv.name) + return changed = (inv.last_write is None or value != inv.last_write) if not (changed or force): return @@ -4295,11 +4325,11 @@ class Controller: # `changed` is still true. try: _w = self._driver(inv).write(value) - if inv.driver.hands_back: - # SMA-10: kept, so a release can wait for it (`_release_last`) - _w = asyncio.ensure_future(_w) - inv.inflight.add(_w) - _w.add_done_callback(inv.inflight.discard) + # SMA-10 / SAFETY-137: kept for every driver, so a release or a last + # 0 W can wait for it (`_await_inflight`). + _w = asyncio.ensure_future(_w) + inv.inflight.add(_w) + _w.add_done_callback(inv.inflight.discard) ok = await asyncio.wait_for(_w, timeout=WRITE_BUDGET_S) except (asyncio.TimeoutError, TimeoutError): ok = False @@ -7209,17 +7239,17 @@ class Controller: call takes no longer than its slowest bound. """ if not any(inv.driver.hands_back for inv in self.inverters): - await self.command_site_zero() + await self.command_site_zero(last=True) return self.target = 0.0 for inv in self.inverters: inv.target = 0.0 await asyncio.gather(*( self._hand_back(inv, force=True) if inv.driver.hands_back - else self._write_unit(inv, 0.0, force=True) + else self._write_unit(inv, 0.0, force=True, last=True) for inv in self.inverters)) - async def command_site_zero(self) -> None: + async def command_site_zero(self, *, last: bool = False) -> None: """Every inverter to 0 W, and every inverter's TARGET to 0 W with it. ⚠️ THE SECOND HALF IS NOT BOOKKEEPING, AND IT IS A HAZARD DEV-02 @@ -7241,7 +7271,7 @@ class Controller: self.target = 0.0 for inv in self.inverters: inv.target = 0.0 - await self.write_setpoint(0.0, force=True) + await self.write_setpoint(0.0, force=True, last=last) # -- UI ------------------------------------------------------------------ def checks(self, when: datetime | None = None) -> list: diff --git a/goodwe_controller/test_safety137_last_write.py b/goodwe_controller/test_safety137_last_write.py new file mode 100644 index 0000000..483109f --- /dev/null +++ b/goodwe_controller/test_safety137_last_write.py @@ -0,0 +1,332 @@ +"""SAFETY-137: the 0 W of shutdown and of the Stop button is the last write every +unit hears. `python3 test_safety137_last_write.py` + +A cycle write already on its way (0.3-0.6 s, or hung) must not land after the +0 W, for a GoodWe unit and for a fake unit that does not hand back, at N=1 and +N=2, at shutdown and at Stop. The control cycle (and the heartbeat) issue no +new write once shutdown has begun. The whole wind-down stays inside +HAND_BACK_WORST_S (6 s) when a write hangs. + +Mutating a module this file grades? Read MUTATION.md first (purge __pycache__). +The mutants this file was run against are listed at the end. +""" + +import asyncio +import sys +import time +from pathlib import Path + +HERE = Path(__file__).resolve().parent +sys.path.insert(0, str(HERE)) + +from app import main as M # noqa: E402 +from app.drivers import Driver, Probe, REGISTRY # noqa: E402 +from app.main import Controller # noqa: E402 +from app.store import DEFAULTS # noqa: E402 + +fails = [] + + +def check(name, cond): + print(f" {'ok ' if cond else 'FAIL'} {name}") + if not cond: + fails.append(name) + + +class PlainFake(Driver): + """A unit that does not hand back (so, like GoodWe, no release()). A write + of anything but 0 W takes `delay` s to land; every write is logged when it + LANDS (`landed`) and when it STARTS (`started`).""" + name = "plainfake" + writer_key = "host" + brand = "Fake" + hands_back = False + delay = 0.0 + + def __init__(self, host="", port=502, unit_id=1, rating=2000.0): + self.host, self.port, self.unit_id = host, port, unit_id + self.rating, self.power = rating, 0.0 + self.landed, self.started = [], [] + + @classmethod + def from_entry(cls, entry): + return cls(entry.get("host", ""), entry.get("port", 502), + entry.get("unit_id", 1)) + + @classmethod + def actuator_id_of(cls, entry): + host = str(entry.get("host") or "").strip() + return f"{host}:{entry.get('port', 502)}:{entry.get('unit_id', 1)}" \ + if host else "" + + def configured(self): + return bool(self.host) + + def actuator_id(self): + return f"{self.host}:{self.port}:{self.unit_id}" + + def soc_source(self): + return "fake-soc" + + def batt_source(self): + return "fake-power" + + async def limits(self): + return -self.rating, self.rating + + async def read(self): + return 60.0, self.power, {} + + async def write(self, w): + self.started.append((time.monotonic(), float(w))) + if w != 0 and self.delay: + await asyncio.sleep(self.delay) + self.landed.append((time.monotonic(), float(w))) + self.power = w + return True + + async def probe(self): + return Probe(-self.rating, self.rating, 60.0, self.power, {}) + + +REGISTRY["plainfake"] = PlainFake + + +class MemStore: + def __init__(self): + self.data = dict(DEFAULTS) + + def set(self, k, v): + self.data[k] = v + + def get_time(self, *a): + return None + + def set_time(self, *a): + pass + + +class Pub: + enabled = False + + def publish(self, values): + pass + + def close(self): + pass + + +class SlowHass: + """Home Assistant for the GoodWe units. A set_number of anything but 0 W + takes `delay` s to land; the log gets the write when it lands.""" + + def __init__(self): + self.delay = 0.0 + self.landed, self.started = [], [] # (t, entity, value) + self.numbers = {"sensor.grid": 3000.0, "sensor.soc_a": 50.0, + "sensor.batt_a": 0.0, "sensor.soc_b": 50.0, + "sensor.batt_b": 0.0} + + async def number(self, entity, invert=False): + return self.numbers.get(entity) + + async def limits(self, entity): + return (-3000.0, 3000.0, 1.0) + + async def set_number(self, entity, value): + self.started.append((time.monotonic(), entity, value)) + if value != 0 and self.delay: + await asyncio.sleep(self.delay) + self.landed.append((time.monotonic(), entity, value)) + return True + + async def state(self, entity): + return None + + async def call(self, *a): + return True + + +def gw(name, suffix): + return dict(name=name, setpoint_entity=f"number.sp_{suffix}", + soc_entity=f"sensor.soc_{suffix}", + batt_entity=f"sensor.batt_{suffix}", max_w=3000, + bms_soc_floor=10) + + +def fk(name): + return dict(name=name, driver="plainfake", host="10.9.9.9", port=502, + unit_id=3, max_w=2000, bms_soc_floor=10) + + +ROSTERS = { + "N=1 GoodWe": [gw("inverter-a", "a")], + "N=1 fake": [fk("inverter-a")], + "N=2 GoodWe+GoodWe": [gw("inverter-a", "a"), gw("inverter-b", "b")], + "N=2 GoodWe+fake": [gw("inverter-a", "a"), fk("inverter-b")], +} + + +def site(inverters, delay): + opts = dict(meter_source="off", meter_entity="sensor.grid", max_w=5000, + gain=0.6, slew_w=0, deadband_w=15, target_grid_w=0, step_w=10, + heartbeat_s=10, stale_input_s=15, shadow_mode="off", + inverters=inverters) + hass = SlowHass() + hass.delay = delay + c = Controller(opts, hass, MemStore(), Pub(), stop=asyncio.Event()) + for inv in c.inverters: + inv.dev_min, inv.dev_max = -3000.0, 3000.0 + if isinstance(inv.driver, PlainFake): + inv.driver.delay = delay + c.auto = True + return c, hass + + +def per_unit(c, hass, kind="landed"): + """name -> [(t, value)] in the order the writes LANDED (or started).""" + out = {} + for inv in c.inverters: + d = inv.driver + if isinstance(d, PlainFake): + out[inv.name] = list(getattr(d, kind)) + else: + ent = d.setpoint_entity + out[inv.name] = [(t, v) for t, e, v in getattr(hass, kind) + if e == ent] + return out + + +async def cycles(c, n=2): + for _ in range(n): + await c.read_inputs() + await c.cycle() + + +async def in_flight(c): + """A cycle write on its way to every unit (what rig 5b had): the real + cycle, whose writes are authorised and take `delay` s to land.""" + tasks = [asyncio.ensure_future(cycles(c, 1))] + await asyncio.sleep(0.1) + assert all(not t.done() for t in tasks) + return tasks + + +async def finish(tasks): + for t in tasks: + t.cancel() + for t in tasks: + try: + await t + except BaseException: # noqa: BLE001 - cancelled or timed out + pass + + +def last_is_zero(c, hass): + got = per_unit(c, hass) + return all(w and w[-1][1] == 0.0 for w in got.values()), got + + +def scenario(label, inverters, how, delay): + """how: 'shutdown' or 'stop'. Returns (seconds, ok, detail).""" + c, hass = site(inverters, delay) + + async def go(): + tasks = await in_flight(c) + t0 = time.monotonic() + if how == "shutdown": + await c.shutdown() + else: + await c.handle_action("auto_toggle") # Stop + dt = time.monotonic() - t0 + # let a write that is still on its way land, to see if it overtakes + await asyncio.sleep(0.8 if delay < 5 else 0.0) + await finish(tasks) + return dt + + dt = asyncio.run(go()) + ok, got = last_is_zero(c, hass) + n_sent = {n: len(w) for n, w in got.items()} + check(f"{label}: the cycle write was on its way at {how} (test sees it)", + all(len(w) >= 2 for w in per_unit(c, hass, "started").values())) + print(f" {label}: {how}, delay {delay} s: {dt:.2f} s, " + f"landed per unit {n_sent}, last " + f"{ {n: (w[-1][1] if w else None) for n, w in got.items()} }") + return dt, ok, got + + +print("SAFETY-137 - the 0 W is the last write") + +print("1. a slow write on its way (0.3 s, 0.6 s) when shutdown / Stop starts") +for label, roster in ROSTERS.items(): + for how in ("shutdown", "stop"): + for delay in (0.3, 0.6): + dt, ok, got = scenario(label, roster, how, delay) + check(f"{label}, {how}, {delay} s in flight: the last write per " + f"unit is 0 W", ok) + check(f"{label}, {how}, {delay} s in flight: the slow write " + f"landed first, then the 0 W (not dropped, not overtaken)", + all(len(w) >= 2 and w[-2][1] != 0.0 + for w in got.values())) + check(f"{label}, {how}: took about the write, not the budget " + f"({dt:.2f} s)", dt < delay + 1.0) + +print("2. a hung write on its way: the 0 W still reaches every unit, in time") +for label, roster in ROSTERS.items(): + for how in ("shutdown", "stop"): + dt, ok, got = scenario(label, roster, how, 3600.0) + check(f"{label}, {how}, hung write: 0 W landed on every unit and is " + f"its last write", ok) + check(f"{label}, {how}, hung write: inside HAND_BACK_WORST_S " + f"({dt:.2f} s <= {M.HAND_BACK_WORST_S:.0f} s)", + dt <= M.HAND_BACK_WORST_S) + check(f"{label}, {how}, hung write: the 0 W goes after the settle " + f"wait ({M.SETTLE_BUDGET_S:.0f} s), not after the write's own " + f"budget ({dt:.2f} s)", dt <= M.SETTLE_BUDGET_S + 0.5) + +print("3. no new write once shutdown has begun (cycle, heartbeat)") +for label, roster in ROSTERS.items(): + c, hass = site(roster, 0.4) + + async def go3(): + tasks = await in_flight(c) + s = asyncio.ensure_future(c.shutdown()) + await asyncio.sleep(0) # shutdown has begun + begun = time.monotonic() + await cycles(c, 3) # would write: grid 3000 W + await c.send_heartbeats(10 ** 9, 10) # every unit is due + await asyncio.wait_for(s, 10) + await asyncio.sleep(0.8) + await finish(tasks) + return begun + + begun = asyncio.run(go3()) + starts = per_unit(c, hass, "started") + late = {n: [v for t, v in w if t >= begun and v != 0.0] + for n, w in starts.items()} + check(f"{label}: no non-zero write started after shutdown began " + f"({late})", all(not v for v in late.values())) + ok, got = last_is_zero(c, hass) + check(f"{label}: and the last write per unit is 0 W", ok) + # sanity: the same cycles DO write before shutdown + c2, hass2 = site(roster, 0.0) + asyncio.run(cycles(c2, 3)) + before = per_unit(c2, hass2, "started") + check(f"{label}: control, same cycles before shutdown, write non-zero " + f"(the test can see a write)", + all(any(v != 0.0 for _t, v in w) for w in before.values())) + +print() +if fails: + print(f"FAILED {len(fails)}:") + for f in fails: + print(" -", f) + sys.exit(1) +print("all ok") + +# MUTANTS this file was run against (each must turn it red): +# (a) _await_inflight does nothing (no wait for in-flight writes), GoodWe too +# (b) the `elif self.stopping: return` gate removed (cycle not stopped) +# (c) `last=True` not passed by command_site_release (Stop/shutdown 0 W is a +# plain write) +# (d) the wait unbounded (no timeout): red on the settle-bound check diff --git a/goodwe_controller/test_sma10_two_idle_states.py b/goodwe_controller/test_sma10_two_idle_states.py index d88d2f1..b395333 100644 --- a/goodwe_controller/test_sma10_two_idle_states.py +++ b/goodwe_controller/test_sma10_two_idle_states.py @@ -1103,8 +1103,9 @@ ss = src.splitlines() def calls_in(name): - """Line numbers of every `await self.()` in main.py.""" - return [i + 1 for i, ln in enumerate(ss) if f"await self.{name}()" in ln] + """Line numbers of every `await self.(...)` in main.py (SAFETY-137: + the all-GoodWe branch passes `last=True`).""" + return [i + 1 for i, ln in enumerate(ss) if f"await self.{name}(" in ln] rel, zer = calls_in("command_site_release"), calls_in("command_site_zero") -- 2.54.0 From 0f208a0500b677e5f1d013e4af38ebc30e997e50 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 8 Nov 2026 09:08:59 +0100 Subject: [PATCH 21/22] Release 0.21.0: version bump and CHANGELOG Minor bump, no option changes. Folds "Not in a release yet" into a new 0.21.0 section, adds the contents table for the nine merges on release/1.0 since master 0.20.1, updates the known issues, and rewrites the 0.20.1 SAFETY-128 known issue to say it ships in 0.21.0. Co-Authored-By: Claude --- goodwe_controller/CHANGELOG.md | 127 ++++++++++++++++++++++++++++++--- goodwe_controller/config.yaml | 2 +- 2 files changed, 119 insertions(+), 10 deletions(-) diff --git a/goodwe_controller/CHANGELOG.md b/goodwe_controller/CHANGELOG.md index 67ecef3..53be9ee 100644 --- a/goodwe_controller/CHANGELOG.md +++ b/goodwe_controller/CHANGELOG.md @@ -1,14 +1,123 @@ # Changelog -### Not in a release yet +## 0.21.0 + +**A minor bump: control now holds the batteries when Home Assistant readings come late, Home draws the whole night's grid charge, and the shutdown 0 W is sent last. No option changes.** +No option was added, removed or renamed, and no default changed (86 options). **Existing installs need no option change:** options that start 0.20.1 also start 0.21.0, and the Supervisor form is the same. It is a minor bump because control acts differently when readings come late and when the add-on stops; every saved setting still starts. + +**Late Home Assistant readings are held, not followed.** When Home Assistant passes readings on late, control used to correct a mistake its last command had already fixed, so the give run could swing and send power to the grid. It now waits until the battery shows its last power request, and the peak limit no longer asks for full power on late readings. A row on the page says so, and if the readings stay too late every inverter goes to 0 W. With readings on time, the commands are the same as before. +- With a 2 s meter, readings 10 s late were found in 43 s in a rig test. +- With a 5 s meter, readings 30 s late are now found in about 90 s (88 s in a simulation, 75 s in a rig test), where they took about 3 minutes. Readings 10 s late at a 5 s meter are not found ([[SAFETY-136]]). + +**Home draws the whole night's grid charge.** Home used to draw a battery's grid charge only for the cheap run under way. It now draws every hour the add-on will charge it tonight, and shares the peak room between batteries as control does. It shows the plan and changes no command. + +**Tomorrow's forecast stays in the afternoon.** If one of the tomorrow solar forecast sensors stops updating, tomorrow's forecast now stays instead of vanishing until 20:00. + +**The shutdown 0 W is sent last.** When the add-on stops, the 0 W now waits up to 2 s for a command already on its way, so it is the last thing each inverter hears. A command slower than that can still land after it ([[SAFETY-138]]). + +Contents of this release, in merge order since 0.20.1 (`d53f70f`): + +| | | +|---|---| +| `371961e` | When Home Assistant passes readings on late, control no longer corrects a mistake its last command has already fixed. It waits until the battery shows its last power request, and the peak limit no longer asks for full power on late readings. The page says so, and if the readings stay too late every inverter goes to 0 W ([[SAFETY-128]], #228) | +| `af2d0d2` | History only: brings 0.20.1, already live, into this branch. Nothing else is new (#233) | +| `47c89a1`, `c66aa7f` | Home draws a battery's grid charge for every hour the add-on will charge it tonight, not only the cheap run under way, and shares the peak room between batteries as control does. It shows the plan and changes no command ([[UX-77]], #234, with its second round #235) | +| `b291262` | With a 5 s meter, readings 30 s late are found in about 90 s, not about 3 minutes. Readings 10 s late are not found at a 5 s meter; the owner accepted this and it has its own card ([[SAFETY-129]], #236, [[SAFETY-136]]) | +| `23ff9d0` | If one of the tomorrow solar forecast sensors stops updating in the afternoon, tomorrow's forecast now stays on the Forecast tab instead of vanishing until 20:00. The night plan can use it in that time ([[OPT-38]], #237) | +| `57c8d1b` | Each inverter is now reached through a driver. This prepares support for other inverter makes; there is no change for GoodWe sites ([[SMA-2]], #238) | +| `dbdc37b` | When the add-on stops, an inverter that can steer itself can be handed back instead of held at 0 W. This prepares support for other inverter makes; there is no change for GoodWe sites ([[SMA-10]], #239) | +| `b3cfcf2` | When the add-on stops, the 0 W now waits up to 2 s for a command already on its way, so it is the last thing each inverter hears. In a rig test, 0 of 111 stops ended on a value other than 0 W, against 4 of 110 before. A slower command can still land after it ([[SAFETY-137]], #240, [[SAFETY-138]]) | + +### Known issues + +- A false "Battery is not keeping up" can still show when the peak guard pauses a maintenance charge ([[SAFETY-92]]). +- A battery capped at a low limit with a perfectly flat reading can still trip the stale check ([[SAFETY-74]]). +- A battery whose reading at rest is beyond about ±175 W is still not caught when it refuses ([[SAFETY-97]]). +- The peak guard can still back off near the end of a 15-minute period when a meter reading is late, by less than 30 s. This is about the guard's discharge only; a grid charge is cut back since 0.19.2 ([[SAFETY-99]]). +- Two inverters: the maintenance charge can switch on and off while the house is above the peak aim ([[SAFETY-88]]). +- Two inverters: the "circulating" figure counts a battery's reading at rest ([[SAFETY-98]]). +- Two inverters: a drain that starts during a big step in house use can send the other battery to full power, and about 1 kW goes to the grid for about 10 s ([[SAFETY-104]]). +- Two inverters: when a battery comes back during a maintenance charge, the grid can step by several kW at once ([[SAFETY-101]]). +- Two inverters: when a battery drops out during a solar surplus, the others can swing by up to 1.5 kW in a simulation ([[SAFETY-105]]). +- Two inverters: after a drain ends, a change in house use can go unanswered for about 9 s, because some readings are old ([[SAFETY-106]]). +- Two inverters: when one battery's level is missing, both batteries are held at 0 W, and for about 15 s the last command stays on ([[SAFETY-108]]). +- Two inverters: if one battery's level stays missing for 12 h, the other battery's cycle can be marked failed, with a 7-day wait and a notice, though it is healthy. This comes from reading the code; it has not been seen on a site ([[MAINT-17]]). +- When the 15-minute demand forecast sensor counts the battery's own charge, the maintenance charge runs at about half the power the peak limit leaves, so it takes about twice as long. Seen on a site: about 1.2 kW instead of 2.5 kW, and the peak limit was kept ([[SAFETY-111]]). +- One inverter: after a link drop, the battery's own safety stop to 0 W can be taken as a refusal, and the site stays at 0 W for up to 15 min ([[SAFETY-102]]). +- After a Home Assistant restart, control still waits until Home Assistant has finished starting, which can take several minutes, though the readings are already there ([[SAFETY-115]]). +- While Home Assistant is not answering, the page can list inverters as refusing commands and sensors as missing, and send you to Developer Tools, though they are fine ([[UX-37]]). +- If Home Assistant hangs or stops, the page keeps saying "Waiting for Home Assistant" and never turns into an alarm ([[SAFETY-113]]). +- Right after Home Assistant has started, the page can say for about 4 s that an inverter cannot be controlled, and the first command can come late ([[SAFETY-114]]). +- After a Home Assistant restart, the controller status sensor can still say "running" while the add-on cannot control. It was seen once, and the cause is not known ([[SAFETY-116]]). +- Battery modes on: after a planned top-up from the grid, the Batteries screen can say "Enough sun is forecast" ([[UX-32]]). +- Battery modes on: the forecast line "we count on" can show more sun than the plan counts, because the plan takes house use off first ([[UX-33]]). +- Battery modes on: before a planned top-up, the Today list shows a smaller "gives" amount than the plan will use; it grows as the charge happens ([[UX-34]]). +- Two inverters, battery modes on: while the other battery is in its maintenance charge, a battery's card can say it charges from the grid while it stays at 0 W ([[OPT-18]]). +- Two inverters, battery modes on: while a battery refuses to charge in a cheap run, the action log on the page gets about 8 lines an hour for its retries, and the page hint says "waiting for a better price" ([[UX-38]]). +- A battery in a safety stop, or one the add-on cannot reach, can still be started for a cycle, and its drain then waits about 12 h and ends in a failed-cycle notice. This comes from reading the code; it has not been seen on a site ([[MAINT-19]]). +- On the Settings page, the rows of the solar forecast lists are labelled "Phase 1", "Phase 2" instead of "Sensor 1", "Sensor 2" ([[UX-27]]). +- In the solar forecast list, a blank or repeated row makes "source 2 of 2" count differently from the rows on the Settings page ([[UX-28]]). +- The solar forecast message says "has no reading" also when the sensor shows a number that is too old or in the wrong unit ([[UX-29]]). +- If you use only the forecast lists, two Settings notes sit on the single-sensor field, and one of them tells you to set that field ([[UX-30]]). +- After a restart during a price outage, the "No prices" message can say "yet" when prices did come in earlier ([[UX-39]]). +- On the new settings page, you cannot see whether a price service token is set. Check the Configuration tab ([[UX-41]]). +- On the classic settings page, the first click on Save after editing a field can do nothing. Click again ([[UX-42]]). +- A steady overload on a 1 s meter can warn, and send the Home Assistant notification, 2 or 3 times at the start of each 15-minute period instead of once. The peak limit is still kept ([[SAFETY-122]]). +- A fixed tariff is priced once per hour, so a price window that starts or ends at :15, :30 or :45 gets the wrong price for part of that hour. Windows that start and end on the hour are right ([[TARIFF-23]]). +- Step 6 of the setup wizard stays empty because of an error in the page, in English and in Dutch ([[UX-51]]). +- On Dutch pages, refusal texts, the last-event line under the Home buttons, the installer details and the trading status sentences are still English, and a few Dutch texts read badly, such as "1 metingen geweigerd" ([[UX-52]]). +- Some price texts say "hour" where the add-on acts on a 15-minute period, such as "cheapest hour" ([[UX-47]]). +- After the add-on restarts, the Home price chart shows no price for the earlier part of today, and the setup check can say "12 of 24 hours", though the prices are known ([[UX-48]]). +- With battery modes on and only batteries on "Leftover solar", the Home price chart can mark a cheap period where no battery charges ([[UX-49]]). +- On a phone, the bars of the Home price chart are too thin to tap one 15-minute period. The arrow keys work ([[UX-50]]). +- In the last second of a 15-minute period, being a few watts over the aim can make the peak guard ask a battery for full power for one second. When a charge ends on a period edge, the last-second give can fade out over about 10 s, and a rig test exported up to 7.4 kW for about 8 s ([[SAFETY-124]]). +- During the night grid charge, a 15-minute period can end 6 to 14 W over the peak aim, because the battery charges harder than asked. This was seen in simulation; a rig test saw 2 to 8 W ([[OPT-35]]). +- The test rig could not bring back the false peak alarm that [[SAFETY-126]] fixes, so that fix is shown in simulation only. Watch the notifications for a few nights ([[TEST-35]]). +- The night cheap run is sized for the batteries' full charging power, not for what the peak limit leaves. Under a tight limit the run can end before the batteries reach their level, and Home can show a level they do not reach (79.9 % against a shown 95 % in a rig test) ([[OPT-25]]). +- By day, a Leftover solar plan that has only a daily forecast figure counts the sun from now, not from when the figure was read. It counts about 22 % too much for a figure one hour old; the night plan is not affected ([[OPT-30]]). +- The night charge can buy too much in a house that uses more at night than by day, such as one with a car charger or heat pump, and on a sunny day, because spare sun in one expensive hour does not count for the later ones ([[OPT-31]]). +- A cheap run that has already started can overlap a new give run after a re-plan, so a battery can charge from the grid at the dearest price for part of an hour. This was seen in a test case, not on a site. Also in 0.19.2 ([[OPT-32]]). +- If the solar forecast sensors stop updating before 17:00, as in winter or further north, there is no forecast from three hours after their last update until 20:00 ([[OPT-33]]). +- With a forecast that gives only a daily total, the sun is spread an hour off on the two clock-change days after the change. The day's total is right ([[OPT-24]]). +- The add-on learns the house's use only once a solar forecast is set, so a site that adds a forecast later waits one more day for its first plan ([[OPT-26]]). +- Home says "No trading on price yet: no prices have arrived" when every battery is on Solar first, though the prices are shown ([[UX-55]]). +- The Home plan header can say "Charges in the cheapest hour, gives power in the dearest" when Home draws no charge, and it says "hour" for 15-minute prices ([[UX-56]]). +- While the peak guard holds back charging, Home hides the rest of the cheap run and gives no reason. The charge starts again when the guard lets go ([[UX-57]]). +- With battery modes on and selling on, Home can draw a "Sells up to" block when the add-on judges the trade not worth it, and then it will not sell ([[UX-58]]). +- Home, the plan and the Forecast tab stop drawing a battery's give at the trading floor, though control goes on giving down to the battery's cut-out ([[UX-59]]). +- During a price outage, the Batteries screen says "Prices differ too little" and Home says "The price gap is too small" ([[UX-60]]). +- With battery modes on and no meter, or a meter that is a Home Assistant entity, a grid charge is not possible and nothing says so ([[UX-61]]). +- On the Batteries screen, a charge the add-on refused in a cheap run reads as "Saves its power for ...", for up to 15 minutes ([[UX-62]]). +- A Leftover solar battery with no capacity set never charges from the grid when a solar forecast is set, and nothing warns about it ([[UX-63]]). +- The "How the modes work" box on the Batteries screen does not describe Cheap hours only ([[UX-64]]). +- With battery modes on, Settings still shows "Trade on the price spread", which the modes ignore. A site with modes on and no price source starts, and the Batteries screen says "No prices, so it follows the house" without pointing to Settings ([[UX-65]]). +- With battery modes off and several equal prices in a row, Home and the Forecast tab draw one price quarter, while the add-on trades every quarter of the run ([[UX-67]]). +- Late in the day, the Forecast tab's "now" line and last bars sit over the axis labels at the right edge ([[UX-68]]). +- The Forecast tab's level curve charges at each battery's full rating. Under a peak limit the levels it draws are too high and a charge block can end early (54.6 % shown against 39.7 % in a rig test) ([[UX-70]]). +- The peak forecast check says maintenance charging is not kept under the peak limit when no peak forecast is set, but since 0.19.2 it is ([[UX-73]]). +- The peak guard row can show the same figure twice, such as "at this rate 3 500 W, over 3 500 W". Also in 0.19.2 ([[UX-74]]). +- If the inverter's setpoint entity appears in Home Assistant after the add-on has started, the page keeps saying "Inverter control: not found in Home Assistant" and the batteries say "An alarm stopped control" until a restart, though control runs ([[UX-75]]). +- In the last minute of a 15-minute period, with Home Assistant readings 30 s late, the period can end about 80 W over the peak aim and over the limit: 3 530 W against a 3 450 W aim and a 3 500 W limit in a rig test with a 2 kW load added 45 s before the end. The peak guard sees the load too late to hold it back ([[SAFETY-133]]). +- With Home Assistant readings 10 s late, the grid charge in a cheap run can swing by several kW. In a test the add-on asked for an 8.7 kW charge (9.6 kW on 0.20.0) ([[SAFETY-132]]). +- A plan rebuilt while the battery levels are missing for a moment can drop a cheap run that has not started yet, so the battery does not charge that day. Seen in a replay of real prices, where it cost about EUR 0.40 of extra grid use; the Leftover solar change in 0.20.1 no longer triggers it in those replays, but the cause stays ([[OPT-37]]). +- On a 5 s meter, Home Assistant readings 10 to 15 s late are not caught. With a changing load such as an oven or a kettle, 21 to 142 Wh can go to the grid in a simulation, against 2 to 27 Wh with readings on time. At 15 s late they are caught only after about 4 minutes ([[SAFETY-136]]). +- With readings 10 to 15 s late, the hold can end in the middle of the delay and the swing can come back. Seen in rig tests with a 2 s and a 5 s meter ([[SAFETY-134]]). +- With readings 10 s late on a 1 s meter, the grid can still move between -422 and +340 W while the batteries are held. With a very long delay (30 to 90 s) the request can swing by about 900 W every few seconds. Seen in review tests ([[SAFETY-130]]). +- A battery that answers very slowly (10 s or more) can be taken for late Home Assistant readings and held, and the page then blames Home Assistant. A GoodWe battery settles in about 6 s and this has not been seen on one ([[SAFETY-131]]). +- The page row for late readings reads awkwardly in English ("Home Assistant readings show requests about N seconds late, so control holds the batteries"), and the Dutch text misses a word ([[UX-78]]). +- On a night grid charge held to the peak limit, the peak guard can stop the charge 1 to 3 s into a 15-minute period and keep it paused for that whole period. In a rig test this cost up to about 0.27 kWh of grid charge per run; it may come from the sped-up rig clock ([[SAFETY-135]]). +- When the add-on stops, a write that takes more than 2 s can still land after the 0 W, and the inverter's own 30 s timer then zeroes it. After the Stop button, a cycle that had already decided can send its share (560 to 670 W) just after the 0 W; the next cycle sends 0 W again ([[SAFETY-138]]). +- In the last hour of a cheap run, Home can draw a battery's grid charge up to an hour past the time control stops charging it. It only draws too much, never too little ([[UX-79]]). +- A tomorrow solar forecast sensor that reports just after midnight with yesterday's value can be used as the new day's forecast for up to 12 hours. This has not been seen on a site ([[OPT-39]]). +- During a cheap-hours charge, at some 15-minute boundaries the add-on sends 0 W for one tick ("no trade (headroom)") and then goes on. Seen in a rig test, in 5 of 8 cases; once, with a frozen battery reading, it left a battery at 0 W until 23:45 ([[OPT-40]]). + +### If you are crossing from a version older than 0.10.0 + +`## 0.10.0` below can refuse an old config: `slew_w` above its limit, and the +withdrawn `price_provider` ids `engie_dynamic`, `totalenergies_dynamic` and +`eneco_eddy`. Read it before you update. Run `tools/predeploy_check.py` with the +bare options (`jq '.data.options'`) first. -- When the add-on stops, a command that was already on its way can no longer reach an inverter after the 0 W. The 0 W is now the last thing every inverter gets, also when Stop is pressed ([[SAFETY-137]]). -- When the add-on stops, an inverter that can steer itself is handed back instead of held at 0 W. Safety stops still hold 0 W, and a GoodWe site is not affected ([[SMA-10]]). -- Each inverter is now reached through a driver, so another make can be added later. Nothing changes for a GoodWe site ([[SMA-2]]). -- Home now draws today's charge from the grid for every hour the add-on will really charge, not only the run under way ([[UX-77]]). -- When Home Assistant passes readings on late, control no longer swings on them: it waits until the battery shows its last power request, and the peak limit no longer asks for full power on them. The page says so, and if the readings stay too late every inverter goes to 0 W ([[SAFETY-128]]). -- With a 5 s meter, readings 30 s late are now noticed within 100 seconds, not 3 minutes ([[SAFETY-129]]). -- If one of the tomorrow solar forecast sensors stops updating in the afternoon, tomorrow's forecast now stays on the Forecast tab instead of vanishing until 20:00 ([[OPT-38]]). ## 0.20.1 @@ -65,7 +174,7 @@ Contents of this release, in merge order since 0.20.0 (`01e88a2`): - After the add-on restarts, the Home price chart shows no price for the earlier part of today, and the setup check can say "12 of 24 hours", though the prices are known ([[UX-48]]). - With battery modes on and only batteries on "Leftover solar", the Home price chart can mark a cheap period where no battery charges ([[UX-49]]). - On a phone, the bars of the Home price chart are too thin to tap one 15-minute period. The arrow keys work ([[UX-50]]). -- Control acts on Home Assistant readings that arrive late. With readings 10 s behind, the give run can swing and send up to 1.9 kW to the grid for about 50 s in a rig test; with readings 30 s behind, a battery can charge from the grid in an expensive period. Also in 0.19.2. A fix is made for the next release and is not in 0.20.1 ([[SAFETY-128]]). +- Control acts on Home Assistant readings that arrive late. With readings 10 s behind, the give run can swing and send up to 1.9 kW to the grid for about 50 s in a rig test; with readings 30 s behind, a battery can charge from the grid in an expensive period. Also in 0.19.2. The fix is not in 0.20.1; it ships in 0.21.0 ([[SAFETY-128]]). - In the last second of a 15-minute period, being a few watts over the aim can make the peak guard ask a battery for full power for one second. When a charge ends on a period edge, the last-second give can fade out over about 10 s, and a rig test exported up to 7.4 kW for about 8 s ([[SAFETY-124]]). - During the night grid charge, a 15-minute period can end 6 to 14 W over the peak aim, because the battery charges harder than asked. This was seen in simulation; a rig test saw 2 to 8 W ([[OPT-35]]). - The test rig could not bring back the false peak alarm that [[SAFETY-126]] fixes, so that fix is shown in simulation only. Watch the notifications for a few nights ([[TEST-35]]). diff --git a/goodwe_controller/config.yaml b/goodwe_controller/config.yaml index b02e072..6df5b74 100644 --- a/goodwe_controller/config.yaml +++ b/goodwe_controller/config.yaml @@ -1,5 +1,5 @@ name: GoodWe RS485 Controller -version: "0.20.1" +version: "0.21.0" slug: goodwe_controller description: >- Drives a GoodWe ES/BP battery inverter over its RS485 EMS port by sending it a -- 2.54.0 From 8e26574b03acaddc8cbcde597943bfe1735fc79a Mon Sep 17 00:00:00 2001 From: Claude review Date: Sun, 8 Nov 2026 10:20:46 +0100 Subject: [PATCH 22/22] Release 0.21.0: CHANGELOG wording (SAFETY-136, SAFETY-138 known issues) SAFETY-136: the line said 10 to 15 s late is not caught, then that 15 s late is caught after about 4 minutes. It now says 10 s is not caught and 15 s only after about 4 minutes, as the card says. SAFETY-138: the 30 s timer that zeroes a late write is the ESPHome board's watchdog (README), not the inverter's own. Co-Authored-By: Claude Opus 5.5 --- goodwe_controller/CHANGELOG.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/goodwe_controller/CHANGELOG.md b/goodwe_controller/CHANGELOG.md index 53be9ee..d094d02 100644 --- a/goodwe_controller/CHANGELOG.md +++ b/goodwe_controller/CHANGELOG.md @@ -100,13 +100,13 @@ Contents of this release, in merge order since 0.20.1 (`d53f70f`): - In the last minute of a 15-minute period, with Home Assistant readings 30 s late, the period can end about 80 W over the peak aim and over the limit: 3 530 W against a 3 450 W aim and a 3 500 W limit in a rig test with a 2 kW load added 45 s before the end. The peak guard sees the load too late to hold it back ([[SAFETY-133]]). - With Home Assistant readings 10 s late, the grid charge in a cheap run can swing by several kW. In a test the add-on asked for an 8.7 kW charge (9.6 kW on 0.20.0) ([[SAFETY-132]]). - A plan rebuilt while the battery levels are missing for a moment can drop a cheap run that has not started yet, so the battery does not charge that day. Seen in a replay of real prices, where it cost about EUR 0.40 of extra grid use; the Leftover solar change in 0.20.1 no longer triggers it in those replays, but the cause stays ([[OPT-37]]). -- On a 5 s meter, Home Assistant readings 10 to 15 s late are not caught. With a changing load such as an oven or a kettle, 21 to 142 Wh can go to the grid in a simulation, against 2 to 27 Wh with readings on time. At 15 s late they are caught only after about 4 minutes ([[SAFETY-136]]). +- On a 5 s meter, Home Assistant readings 10 s late are not caught, and readings 15 s late only after about 4 minutes. With a changing load such as an oven or a kettle, 21 to 142 Wh can go to the grid in a simulation, against 2 to 27 Wh with readings on time ([[SAFETY-136]]). - With readings 10 to 15 s late, the hold can end in the middle of the delay and the swing can come back. Seen in rig tests with a 2 s and a 5 s meter ([[SAFETY-134]]). - With readings 10 s late on a 1 s meter, the grid can still move between -422 and +340 W while the batteries are held. With a very long delay (30 to 90 s) the request can swing by about 900 W every few seconds. Seen in review tests ([[SAFETY-130]]). - A battery that answers very slowly (10 s or more) can be taken for late Home Assistant readings and held, and the page then blames Home Assistant. A GoodWe battery settles in about 6 s and this has not been seen on one ([[SAFETY-131]]). - The page row for late readings reads awkwardly in English ("Home Assistant readings show requests about N seconds late, so control holds the batteries"), and the Dutch text misses a word ([[UX-78]]). - On a night grid charge held to the peak limit, the peak guard can stop the charge 1 to 3 s into a 15-minute period and keep it paused for that whole period. In a rig test this cost up to about 0.27 kWh of grid charge per run; it may come from the sped-up rig clock ([[SAFETY-135]]). -- When the add-on stops, a write that takes more than 2 s can still land after the 0 W, and the inverter's own 30 s timer then zeroes it. After the Stop button, a cycle that had already decided can send its share (560 to 670 W) just after the 0 W; the next cycle sends 0 W again ([[SAFETY-138]]). +- When the add-on stops, a write that takes more than 2 s can still land after the 0 W, and the 30 s safety timer on the ESPHome board then sets 0 W. After the Stop button, a cycle that had already decided can send its share (560 to 670 W) just after the 0 W; the next cycle sends 0 W again ([[SAFETY-138]]). - In the last hour of a cheap run, Home can draw a battery's grid charge up to an hour past the time control stops charging it. It only draws too much, never too little ([[UX-79]]). - A tomorrow solar forecast sensor that reports just after midnight with yesterday's value can be used as the new day's forecast for up to 12 hours. This has not been seen on a site ([[OPT-39]]). - During a cheap-hours charge, at some 15-minute boundaries the add-on sends 0 W for one tick ("no trade (headroom)") and then goes on. Seen in a rig test, in 5 of 8 cases; once, with a frozen battery reading, it left a battery at 0 W until 23:45 ([[OPT-40]]). -- 2.54.0