SAFETY-135: a young 15-minute period does not cost the night charge the whole period

With battery modes on, a grid charge the peak guard (or a measured headroom) stops in the first 60 s of a period no longer latches off for the period: it may start again once the period is 60 s old, if the guard is clear and the headroom is over START_W. A stop after 60 s latches as before. In a young period no charge starts while the batteries' give is still falling (over START_W within 15 s), since the meter may not show that drop yet. The guard itself is unchanged, so a real over-aim load is cut on the same cycle. DECISIONS/SAFETY-135-young-period.md, test_safety135.py. Co-Authored-By: Claude <noreply@anthropic.com>
Merge pull request #244 : SAFETY-136 a gentler loop on a slow meter until the last move shows
2026-11-10 17:43:28 +01:00 · 2026-11-09 22:35:21 +01:00
5 changed files with 320 additions and 4 deletions
@@ -0,0 +1,74 @@
+# SAFETY-135 - a young 15-minute period does not cost the night charge the whole period
+
+**Status: for review.** Code: `Controller._mode_roles()` and
+`Controller._give_falling()` (`app/ctl_trade.py`), `modetrade.decide(young=)`,
+`YOUNG_S`, `GAVE_S` (`app/modetrade.py`). Tests: `test_safety135.py`. Battery
+modes only. No option, no text change, no change to the peak guard.
+
+**Problem.** In the first seconds of a period the guard's projection is almost
+only the last reading (`held_w`). Two cases seen on TEST-23:
+- *A spike at a period's start* (UX-77 I-1, x10): a 1-3 s spike, or a charge
+  still at full power from the end of the last period, makes the guard project
+  4.3-6.6 kW and cut the night charge. The cut is right. But a cut charge
+  stays off to the period's end (the OPT-11 latch), so one 2 s spike cost 15
+  minutes of charge, about 0.16 kWh per trip on the rig.
+- *A start at the tariff edge* (SAFETY-136 rig F-4, x1): a give ends at 01:00
+  and the cheap run starts. The battery reading drops at once, the meter
+  reading a few seconds later. In between, grid + batteries reads the house
+  too low, so the charge room is too big. The charge starts and the guard cuts
+  it 4-61 s later, and the latch then holds it off for the period.
+
+**Decision.** Two small rules, both only in a young period (its first
+`YOUNG_S` = 60 s):
+1. *A stop does not latch.* A charge the guard or a measured headroom stops in
+   a young period may not start again until the period is 60 s old; then the
+   normal start rule (guard clear, over `START_W` of headroom) decides. A stop
+   after 60 s latches to the period's end as before, so there is at most one
+   restart per period and the OPT-11 flip guard keeps its job.
+2. *No start while a give is still falling.* While the batteries give over
+   `START_W` less than they did in the last `GAVE_S` = 15 s, no charge starts:
+   the meter may not show that drop yet. 15 s covers a 5 s meter twice plus
+   Home Assistant's own delay. A give that falls by `START_W` or less does not
+   hold the start: the error it leaves is under the smallest charge worth
+   starting.
+
+The guard itself is not touched: its claim, floor and projection are the same
+code, so a real over-aim load is cut on the same cycle as before. A running
+charge is never held by rule 2.
+
+**Options weighed.**
+- *A minimum data window before the latch* (no latch in the first N s): this
+  is rule 1. Alone it lets the edge start flap once more.
+- *No start in a young period within 60 s of a give or a guard action*: also
+  stops the edge flap, but delays every run start that follows a give. In the owner-shape night sim it
+  charged 0.018 kWh (N=1) and 0.003 kWh (N=2) less than base. Rule 2 holds the
+  start only while the give is still falling, and charges the same as base.
+- *Cutting the young charge room by the recent give* (instead of holding the
+  start): it double-counts once the meter catches up, cut a running charge 3 s
+  after its start in the 1 s meter edge run, and so flapped where base did not.
+- *Re-evaluation each cycle with no latch*: brings back the 13 flips in 4
+  minutes the latch was made for (2026-09-09 replay).
+- *A hysteresis band on the start*: does not help at the edge. The error there
+  is the readings' age, not a small room.
+
+**Numbers** (`scratch/SAFETY-135/dev/s135_sim.py`, real `cycle()`, TEST-23-like
+lag, 1 s cycle, two 2.5 kW batteries, cap 3 500 W, aim 3 450 W; grid
+average over each period as the sim saw it):
+
+| Scenario | Base: paused s, kWh, flaps, top period | Head |
+|---|---|---|
+| (a) 4 s spike at 01:30 and 01:45, house 2.8 kW | 1 197 s, 0.330 kWh, 1, 3 458 W | 135 s, 0.537 kWh, 0, 3 458 W |
+| (b) 01:00 edge after a give, 5 s meter, house 3.0 kW | 882 s, 0.006 kWh, 1, 3 009 W | 21 s, 0.119 kWh, 0, 3 464 W |
+| (b) the same, house 2.4 kW | 0 s, 0.266 kWh, 0, 3 461 W | 11 s, 0.266 kWh, 0, 3 461 W |
+| (c) +4 kW for 5 min from 01:30:20 | cut 01:30:22, 0.387 kWh | cut 01:30:22, 0.492 kWh |
+| Owner-shape night, N=1 / N=2 | 5.510 / 6.048 kWh, top 3 454 / 3 453 W | 5.510 / 6.048 kWh, 3 454 / 3 453 W |
+| Owner-shape day, N=1 / N=2 | top 331 / 145 W | the same |
+
+The add-on's own period averages in `test_safety135.py`: 3 450.0 W in the
+spike period (base 2 807.9 W, the charge lost), 3 450.0 W in the 01:00 period
+(base 2 996.5 W at 3.0 kW). Over the aim by a few watts in the sim's grid
+average is the landing error base has too (SAFETY-126).
+
+**Not changed.** With battery modes off, `arbitrage.decide()` has no period
+latch and sizes the charge to the headroom each cycle (SAFETY-45). Nothing
+here touches it.
@@ -401,6 +401,10 @@ class TradeMixin:
        quarter = int(now.timestamp() // 900)
        was = self.mode_tick is not None and \
            self.mode_tick.action == modetrade.CHARGE
+        # SAFETY-135: a stop in a young period holds only until it is not
+        # young (no latch), and no charge starts there while a give falls.
+        young = now.timestamp() - quarter * 900 < modetrade.YOUNG_S
+        falling = self._give_falling(now)
        self.mode_tick = modetrade.decide(
            runs, now, packs, charge_ok=charge_ok, sell_ok=sell_ok,
            trade_why=trade_why, price_now=price_now,
@@ -408,7 +412,8 @@ class TradeMixin:
            ceiling_pct=ceiling_pct, house_w=house, charging_now=was,
            blocked=self._charge_blocked_q == quarter,
            done_charge=done_c, done_sell=done_s, targets=targets,
-            draining=any(r.phase == DRAIN for r in maint_results or ()))
+            draining=any(r.phase == DRAIN for r in maint_results or ()),
+            young=falling or (young and self._charge_young_q == quarter))
        self._latch_charge(now, runs, targets, packs)
        # A charge the guard or the headroom stopped stays off this period.
        # OPT-15: only a MEASURED stop latches; the first unmeasured headroom
@@ -419,11 +424,31 @@ class TradeMixin:
            gap = why == "headroom" and self.arb_headroom_w is None
            if why == "peak" or (why == "headroom" and not gap) or (
                    gap and self._charge_gap_q == quarter):
-                self._charge_blocked_q = quarter
+                if young and not gap:
+                    self._charge_young_q = quarter      # SAFETY-135
+                else:
+                    self._charge_blocked_q = quarter
            elif gap:
                self._charge_gap_q = quarter
        return self.mode_tick

+    def _give_falling(self, now) -> bool:
+        """SAFETY-135: True in a young period while the batteries give over
+        START_W less than they did in the last GAVE_S s: the meter may not
+        show that yet, so a charge room read now is too big.
+        DECISIONS/SAFETY-135-young-period.md"""
+        t = now.timestamp()
+        gave = self._gave
+        if self.site_batt_w is not None:
+            gave.append((t, max(0.0, float(self.site_batt_w))))
+        while gave and not 0.0 <= t - gave[0][0] < modetrade.GAVE_S:
+            gave.popleft()
+        if self.site_batt_w is None or not gave \
+                or t % 900 >= modetrade.YOUNG_S:
+            return False
+        drop = max(w for _t, w in gave) - max(0.0, float(self.site_batt_w))
+        return drop > modetrade.START_W
+
    def _run_mark(self, kind, run, packs, out, extra=None) -> set:
        """The batteries that left run `run` (full, or at the floor); they
        stay out until it ends, so a level edge does not re-arm them.
@@ -22,6 +22,7 @@ ctl_inputs, ctl_writes, ctl_trade, ctl_cycle, ctl_alarms, ctl_status and
 ctl_tasks (MAINT-21), the status-row functions in rows."""

 import asyncio
+import collections
 import contextlib
 import dataclasses
 import functools
@@ -528,6 +529,8 @@ class Controller(InputsMixin, WritesMixin, TradeMixin, CycleMixin, AlarmsMixin,
        self._mode_trade_was = False
        self._charge_blocked_q = None
        self._charge_gap_q = None       # OPT-15: an unmeasured stop's quarter
+        self._charge_young_q = None     # SAFETY-135: a young stop's quarter
+        self._gave = collections.deque()  # SAFETY-135: (t, W given), GAVE_S
        self._ref_cheap = None
        # OPT-15: a run edge the loop holds at E: (E, step sign, until, the
        # last battery reading).
@@ -36,6 +36,9 @@ Rules that keep this safe (tests: test_modes_p3.py, test_price_run_fixes.py):
    makes the other batteries stop giving power, so a few watts of charge
    flipping on and off would swing the house between them and the grid
    (measured on the 2026-09-09 replay: 13 flips in 4 minutes at 3.5 kW);
+    SAFETY-135: a stop in a period's first YOUNG_S s holds only until then,
+    not to the period's end, and no charge starts then while the batteries'
+    give is still falling (`young`);
  * selling needs selling on for that battery, the sell run, a price >= 0
    now, and the battery above `arbitrage_soc_floor`; it stops there. The
    peak guard never stops a sell (section 9.1: discharge is never capped);
@@ -69,6 +72,12 @@ PRICE_MODES = (LEFTOVER_SOLAR, CHEAP_HOURS)
 # would stop covering the house for less than they give.
 START_W = 500.0

+# SAFETY-135: seconds a 15-minute period is young, and how far back a
+# battery's give counts as maybe not yet on the meter.
+# DECISIONS/SAFETY-135-young-period.md
+YOUNG_S = 60.0
+GAVE_S = 15.0
+

@dataclass(frozen=True)
 class Pack:
@@ -164,7 +173,7 @@ def energy(packs, floor_pct, ceiling_pct, done_charge=(), done_sell=(),
 def decide(runs, now, packs, *, charge_ok, sell_ok, trade_why, price_now,
           headroom_w, floor_pct, ceiling_pct, house_w, charging_now=False,
           blocked=False, done_charge=(), done_sell=(),
-           targets=None, draining=False) -> Tick:
+           targets=None, draining=False, young=False) -> Tick:
    """This tick's ranges, roles and site ask.

    `runs` is the kept `priceruns.RunPlan` or None (no fresh prices).
@@ -177,7 +186,8 @@ def decide(runs, now, packs, *, charge_ok, sell_ok, trade_why, price_now,
    it going); `blocked` that the guard or the headroom stopped a charge in
    this 15-minute period. `targets` is the day plan (`charge_to()`).
    `draining` says a battery drains for its monthly cycle, so none may
-    charge from the grid (why "cycle").
+    charge from the grid (why "cycle"). `young` says no charge may start
+    yet in this young period (SAFETY-135, why "young").
    """
    plan_on = runs is not None and runs.on
    in_cheap = plan_on and runs.cheap.holds(now) and runs.charge_useful()
@@ -198,6 +208,8 @@ def decide(runs, now, packs, *, charge_ok, sell_ok, trade_why, price_now,
            why = trade_why
        elif headroom_w is None or not headroom_w > need:
            why = "headroom"
+        elif young and not charging_now:
+            why = "young"
        else:
            charging = True
    se = sellers(packs, floor_pct, done_sell) if in_sell and not charging \
@@ -0,0 +1,202 @@
+"""Runnable check for SAFETY-135: a young 15-minute period does not cost the
+night charge the whole period.  `python3 test_safety135.py`
+
+The REAL Controller (tools/opt11p3_replay.py, read_inputs + cycle, battery
+modes, the peak guard on a P1 meter) with TEST-23-like lag: 1 s cycles,
+readings 1 s old, a 5 s old meter in part 2. "Off" is the same run with
+`modetrade.YOUNG_S = 0`: the tree before this card.
+DECISIONS/SAFETY-135-young-period.md.
+
+1. A peak-bound night charge; a period starts with a 4 s kettle spike. The
+   guard cuts at once, and the charge resumes in that period (off: it stays
+   off to the period's end). The period stays at or under the aim.
+2. A cheap run starts at 01:00 right after a give run, with a 5 s meter: no
+   start-then-cut within 60 s (off: one), and no period over the aim.
+3. A real over-aim load: the guard and the charge stop on the same cycle as
+   off, and the charge stays off while the load is on.
+
+Mutations (MUTATION.md, purged copies) are listed in the pull request.
+"""
+
+import logging
+import os
+import sys
+from datetime import datetime, timedelta, timezone
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, HERE)
+sys.path.insert(0, os.path.join(HERE, "tools"))
+logging.disable(logging.WARNING)
+
+import opt11p3_replay as rp                                    # noqa: E402
+from app import arbitrage, modetrade, packmode                 # noqa: E402
+
+fails = []
+
+
+def check(name, cond):
+    print(f"  {'ok  ' if cond else 'FAIL'} {name}")
+    if not cond:
+        fails.append(name)
+
+
+L = timezone(timedelta(hours=1))
+D0 = datetime(2026, 11, 3, tzinfo=L)
+UTC0 = D0.astimezone(timezone.utc)
+# Local quarters: dear 00:00-01:00 (a give run), cheap 01:00-07:00, 0.30 else.
+PRICES = [0.40 if q < 4 else 0.10 if q < 28 else 0.30 for q in range(96)] * 2
+WPTS = [(UTC0 + timedelta(minutes=15 * i), 900, p)
+        for i, p in enumerate(PRICES)]
+AIM = rp.PEAK_CAP - 50
+CH = {"a": (True, packmode.CHEAP_HOURS, False),
+      "b": (True, packmode.CHEAP_HOURS, False)}
+T1 = D0 + timedelta(hours=1)                    # 01:00 local, the cheap edge
+
+
+def off(fn):
+    """`fn()` with SAFETY-135 off: no period is young."""
+    real = getattr(modetrade, "YOUNG_S", None)
+    if real is None:
+        return fn()
+    modetrade.YOUNG_S = 0.0
+    try:
+        return fn()
+    finally:
+        modetrade.YOUNG_S = real
+
+
+def run(house_w, start, secs, meter_s=1.0):
+    """Rows per cycle and the Controller, for `house_w(seconds since 01:00)`."""
+    t0 = start.timestamp()
+    house = [(t0 + k, house_w(t0 + k - T1.timestamp()))
+             for k in range(int(secs) + 5)]
+    rows = []
+
+    def cb(c, h, now, pr):
+        g = c.peak_guard
+        rows.append(dict(now=now, s=now.timestamp() - T1.timestamp(),
+                         grid=h.grid(),
+                         charge=getattr(c.arb, "action", None)
+                         == arbitrage.CHARGE,
+                         engaged=g is not None and g.engaged))
+    _st, c = rp.run("p3_ch_sell", house, WPTS, start.astimezone(timezone.utc),
+                    secs, check=cb, soc0={"a": 30.0, "b": 30.0}, choice=CH,
+                    extra={"slew_w": 0, "cycle_cost_threshold_eur": 0.03},
+                    lag=rp.Lag(meter_s=meter_s))
+    return rows, c
+
+
+def banked(c, frm):
+    """The add-on's own P1 period averages from `frm` (seconds after 01:00)."""
+    return [b.offtake_avg_w for b in c.p1.blocks
+            if b.start.timestamp() - T1.timestamp() >= frm]
+
+
+def flaps(rows, frm, to):
+    """Charge starts in [frm, to) s that stopped again within 60 s."""
+    out = []
+    for i in range(1, len(rows)):
+        r = rows[i]
+        if frm <= r["s"] < to and r["charge"] and not rows[i - 1]["charge"]:
+            if any(not x["charge"] for x in rows[i:]
+                   if x["s"] - r["s"] <= 60):
+                out.append(r["s"])
+    return out
+
+
+# ============================================================================ #
+print("1. a 4 s spike at the start of a peak-bound night charge's period")
+
+
+def spike(s):
+    return 2800.0 + (2500.0 if s >= 1800 and (s - 1800) % 900 < 4 else 0.0)
+
+
+R1, C1 = run(spike, T1 + timedelta(minutes=20), 1560)
+R1o, C1o = off(lambda: run(spike, T1 + timedelta(minutes=20), 1560))
+
+
+def period(rows, frm):
+    return [r for r in rows if frm <= r["s"] < frm + 900]
+
+
+for tag, rows, c in (("on", R1, C1), ("off", R1o, C1o)):
+    p = period(rows, 1800)
+    cut = next((r["s"] - 1800 for r in p if r["engaged"]), None)
+    on = sum(r["charge"] for r in p)
+    print(f"    {tag}: guard cut at +{cut} s, charging {on} of {len(p)} s, "
+          f"period {banked(c, 1800)[:1]}")
+_p = period(R1, 1800)
+_cut = next((i for i, r in enumerate(_p) if r["engaged"]), None)
+check("the spike trips the guard in the period's first seconds and the "
+      "charge stops there",
+      _cut is not None and _p[_cut]["s"] - 1800 <= 5
+      and not _p[_cut]["charge"])
+_back = [r["s"] - 1800 for r in _p[(_cut or 0):] if r["charge"]]
+check("the charge resumes in the same period, within 90 s of its start",
+      _back and _back[0] <= 90)
+_po = period(R1o, 1800)
+check("off, the same spike keeps the charge off to the period's end (the "
+      "defect this card fixes)",
+      any(r["engaged"] for r in _po[:5])
+      and not any(r["charge"] for r in _po[10:]))
+check("every period with the charge is at or under the aim (lag allows 2 W)",
+      banked(C1, 0) and max(banked(C1, 0)) <= AIM + 2.0)
+
+# ============================================================================ #
+print("\n2. a cheap run at 01:00 right after a give run, 5 s meter")
+for house in (2400.0, 3000.0):
+    def flat(s, w=house):
+        return w
+    R2, C2 = run(flat, T1 - timedelta(minutes=5), 1260, meter_s=5.0)
+    R2o, C2o = off(lambda: run(flat, T1 - timedelta(minutes=5), 1260,
+                               meter_s=5.0))
+    f2, f2o = flaps(R2, 0, 900), flaps(R2o, 0, 900)
+    p2, p2o = banked(C2, 0)[:1], banked(C2o, 0)[:1]
+    on2 = sum(r["charge"] for r in period(R2, 0))
+    on2o = sum(r["charge"] for r in period(R2o, 0))
+    print(f"    house {house:.0f} W: on flaps {f2} charging {on2} s period "
+          f"{p2}; off flaps {f2o} charging {on2o} s period {p2o}")
+    check(f"house {house:.0f} W: no charge start is cut within 60 s in the "
+          f"01:00 period", not f2)
+    check(f"house {house:.0f} W: the 01:00 period is at or under the aim "
+          f"(lag allows 2 W)", p2 and p2[0] <= AIM + 2.0)
+    check(f"house {house:.0f} W: the charge still runs in the 01:00 period",
+          on2 >= 800)
+    if house == 3000.0:
+        check("off, the 3 000 W house starts on the young period and is cut "
+              "within 60 s (the defect this card fixes)", bool(f2o))
+
+# ============================================================================ #
+print("\n3. a real over-aim load still cuts on the same cycle")
+
+
+def load(s):
+    return 2800.0 + (4000.0 if 1820 <= s < 2120 else 0.0)
+
+
+R3, C3 = run(load, T1 + timedelta(minutes=20), 1500)
+R3o, C3o = off(lambda: run(load, T1 + timedelta(minutes=20), 1500))
+
+
+def first(rows, key, val, frm):
+    return next((r["s"] for r in rows if r["s"] >= frm and r[key] == val),
+                None)
+
+
+_g, _go = first(R3, "engaged", True, 1800), first(R3o, "engaged", True, 1800)
+_s, _so = first(R3, "charge", False, 1800), first(R3o, "charge", False, 1800)
+print(f"    guard cut at {_g} s (off {_go} s), charge stop at {_s} s "
+      f"(off {_so} s)")
+check("the guard cuts on the same cycle as off, and the charge stops on it",
+      _g is not None and _g == _go and _s == _so == _g)
+check("the charge stays off while the load is on",
+      not any(r["charge"] for r in R3 if _g <= r["s"] < 2120))
+check("every period is at or under the aim (lag allows 2 W)",
+      banked(C3, 0) and max(banked(C3, 0)) <= AIM + 2.0)
+
+print()
+if fails:
+    print(f"FAILED: {len(fails)}")
+    sys.exit(1)
+print("all SAFETY-135 checks passed")