OPT-34: the night charge keeps its level when the kept forecast ends

With the forecast sensors silent from 17:05 the kept evening figure ended at
midnight + 3 h, in the middle of the night grid charge, and Leftover solar's
planned level went with it (rig: A 49.8 % against 79.9 %).

- A night grid charge that has started keeps its planned level when the forecast
  goes (Controller._latch_charge, store key night_charge_latch), and the narrow
  price gap with it (weather_held). A fresh figure still moves the level.
- The evening's tomorrow figure is today's until the bell starts (06:00 local),
  up to 15 h old but only if it was usable at midnight (solarfc.Hold.stand_in).
- Screen: "No solar forecast now. The charge under way goes on to its planned
  level." (EN/NL) while a latched battery is below its level.
- test_opt34.py (whole nights through the real control cycle, mutants), test_opt27
  and test_modes_p4 adjusted to the new window and the kept level.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bdqr7PiaHcBnzPituu2yTC

goodwe_controller/app/homeview.py

@@ -535,15 +535,24 @@ def plan_cover(c, name):
     return targets[name], max(0.0, top - floor) / 100.0 * cap * EFF
 
 
+def _latched_below(c) -> bool:
+    """OPT-34: no forecast, but a battery is still below the level its night
+    charge keeps (`Controller.plan_targets()`), so control goes on charging."""
+    targets = getattr(c, "plan_targets", lambda: None)() or {}
+    return any(inv.soc is not None and inv.name in targets
+               and inv.soc < targets[inv.name] for inv in c.inverters)
+
+
 def plan_state(c):
     """P4: `"plan"` (a day plan is made), `"learning"` (no house profile
-    yet), `"no_forecast"` (set but missing or stale), or None (not set, or
-    battery modes off)."""
+    yet), `"no_forecast"` (set but missing or stale), `"latched"` (OPT-34:
+    the same, but a night charge under way is still going on to its level),
+    or None (not set, or battery modes off)."""
     if not getattr(c, "modes_on", False) or \
             getattr(c, "fc_entity", None) is None:
         return None
     if getattr(c, "solar_fc", None) is None:
-        return "no_forecast"
+        return "latched" if _latched_below(c) else "no_forecast"
     hp = getattr(c, "house_profile", None)
     if hp is not None and not hp.ready():
         return "learning"
@@ -564,7 +573,7 @@ def solar_block(c, now):
         out.update(fc_kwh=round(fc.today.kwh, 1),
                    count_kwh=round(fc.today.kwh * trust, 1),
                    updated=hhmm(fc.at))
-    if st in ("no_forecast", "learning"):
+    if st in ("no_forecast", "learning", "latched"):
         out["line"] = _t("solar." + st)
     elif fc is not None:
         out["line"] = line_solar(fc.today.kwh, out["count_kwh"],
@@ -1642,7 +1651,8 @@ def _samples() -> list:
                                kwh="9.4 kWh")) for k in words["why"]]
     out += [("plan", today_words(k)) for k in words["today"]]
     out += [("plan", line_solar(9.2, 6.4, "10:15")),
-            ("plan", _t("solar.no_forecast")), ("plan", _t("solar.learning"))]
+            ("plan", _t("solar.no_forecast")), ("plan", _t("solar.learning")),
+            ("plan", _t("solar.latched"))]
     out += [("plan", day_note(k, pct=70, name="Battery A", at="08:15",
                               kwh="9.2 kWh", count="6.4 kWh"))
             for k in words["day"]]

goodwe_controller/app/main.py

@@ -211,9 +211,30 @@ HOUSE_PROFILE_KEY = "house_profile"
 # OPT-27: the forecast sensors' last fresh readings (`solarfc.Hold`), so an
 # evening restart keeps the figure the night plan uses.
 FC_HOLD_KEY = "solar_fc_hold"
+# OPT-34: the level of a night grid charge that has started, so it does not
+# drop when the forecast goes (`Controller._latch_charge()`).
+CHARGE_LATCH_KEY = "night_charge_latch"
+# OPT-34: a latch outlives the cheap run it was last used in by one price
+# slot, because the next run is planned at the slot edge, in the cycle after
+# the old one ended.
+CHARGE_LATCH_GRACE_S = 900.0
 FORECAST_READ_S = 60.0
 
 
+def _load_latch(data):
+    """`(until, {name: level %})` from the store, or None if it is not a
+    well-formed latch (OPT-34)."""
+    try:
+        until = datetime.fromisoformat(data["until"])
+        levels = {str(n): float(v) for n, v in data["levels"].items()}
+    except (TypeError, KeyError, ValueError, AttributeError):
+        return None
+    if until.tzinfo is None or not levels or not all(
+            math.isfinite(v) and 0.0 <= v <= 100.0 for v in levels.values()):
+        return None
+    return until, levels
+
+
 def why_raised(resolution, claims) -> bool:
     """F2 (OPT-11 P1 test): did the limit that bound the site target push it
     ABOVE what the winner asked (the peak floor asking for more power)? Then
@@ -1005,6 +1026,9 @@ class Controller:
             (getattr(store, "data", None) or {}).get(FC_HOLD_KEY))
         self.day_plan = None
         self._plan_targets = None
+        # OPT-34: `(until, {name: level %})` of a night charge under way.
+        self._charge_latch = _load_latch(
+            (getattr(store, "data", None) or {}).get(CHARGE_LATCH_KEY))
         self.house_profile = None
         if self.fc_entity is not None:
             self.house_profile = dayplan.HouseProfile(
@@ -2265,9 +2289,13 @@ class Controller:
     def weather_held(self, now) -> bool:
         """TARIFF-24: no forecast now, but a run planned with one is under
         way: it keeps its gap to its end. The wider gap applies from the next
-        run. Pure; Home asks the same (`homeview.trade_verdict()`)."""
-        return (self.solar_fc is None and self._wx_free_to is not None
-                and now < self._wx_free_to)
+        run. OPT-34: a night charge that keeps its level (`_latch_charge()`)
+        keeps the gap too, run after run, or the next slice of it would be
+        refused. Pure; Home asks the same (`homeview.trade_verdict()`)."""
+        return self.solar_fc is None and (
+            (self._wx_free_to is not None and now < self._wx_free_to)
+            or (self._charge_latch is not None
+                and now < self._charge_latch[0]))
 
     def _price_runs(self, now, packs, floor_pct, ceiling_pct):
         """The kept run plan, rebuilt at most once per price slot (15 min) or
@@ -2352,11 +2380,48 @@ class Controller:
 
     def plan_targets(self):
         """P4: each battery's planned level, or None (no day plan). Dropped
-        at once when the forecast goes (stale or missing)."""
+        at once when the forecast goes (stale or missing), except the level
+        of a night charge that has started (OPT-34): it keeps that level."""
         if self.solar_fc is None:
-            return None
+            lt = self._charge_latch
+            return None if lt is None else dict(lt[1])
         return self._plan_targets
 
+    def _latch_charge(self, now, runs, targets) -> None:
+        """OPT-34, once per cycle after `modetrade.decide()`. A night grid
+        charge that has started keeps its planned level when the forecast goes
+        mid-charge (the kept figure ends and the sensors are still silent).
+
+        While there is a forecast, fresh or kept, each battery that charges
+        from the grid in the cheap run takes its planned level into the latch;
+        a fresh figure moves it up or down, as it moves the plan. While there
+        is none, `plan_targets()` gives the latched levels, so the charge goes
+        on to them, and `weather_held()` keeps the narrow price gap. The latch
+        lasts through every cheap run on and one price slot after the last
+        (`CHARGE_LATCH_GRACE_S`); a charge that never started has none."""
+        old = lt = self._charge_latch
+        tick = self.mode_tick
+        run = None if runs is None else runs.cheap
+        in_run = run is not None and run.holds(now)
+        if lt is not None:
+            if in_run:
+                lt = (max(lt[0], run.end + timedelta(
+                    seconds=CHARGE_LATCH_GRACE_S)), lt[1])
+            elif now >= lt[0]:
+                lt = None
+        if (in_run and self.solar_fc is not None and targets
+                and tick is not None and tick.action == modetrade.CHARGE):
+            lv = dict(lt[1]) if lt is not None else {}
+            lv.update({n: float(targets[n]) for n, r in tick.roles.items()
+                       if r == modetrade.CHARGE and n in targets})
+            if lv:
+                until = run.end + timedelta(seconds=CHARGE_LATCH_GRACE_S)
+                lt = (max(until, lt[0]) if lt is not None else until, lv)
+        if lt != old:
+            self._charge_latch = lt
+            self.store.set(CHARGE_LATCH_KEY, None if lt is None else {
+                "until": lt[0].isoformat(), "levels": lt[1]})
+
     def _mode_roles(self, now, maint_results):
         """OPT-11 P3: each battery's role and range this tick. Grid charging
         and selling need a fresh plan that `TariffPlan.arbitrage()` allows
@@ -2429,6 +2494,7 @@ class Controller:
             blocked=self._charge_blocked_q == quarter,
             done_charge=done_c, done_sell=done_s, targets=targets,
             draining=any(r.phase == DRAIN for r in maint_results or ()))
+        self._latch_charge(now, runs, targets)
         # A charge the guard or the headroom stopped stays off this period.
         # OPT-15: only a MEASURED stop latches; the first unmeasured headroom
         # (None) in a quarter stops this tick only, so a gap cannot cost the

goodwe_controller/app/solarfc.py

@@ -34,11 +34,13 @@ evening and api.forecast.solar is sometimes out for a poll. A kept reading
 stands in for its sensor (a) for up to `FRESH_S` when the sensor is missing
 or unavailable, as a sensor that merely stopped updating is already
 tolerated, and (b) from the end of the bell (`BELL_TO_H`) until local
-midnight + `KEPT_UNTIL_H`, if it is at most `KEPT_MAX_AGE_S` old when used.
-Never beyond that, and never
-for another day: a today figure is only today's, a tomorrow figure is
-tomorrow's until midnight and then, until midnight + 3 h, today's.
-DECISIONS/OPT-27-last-fresh-forecast-for-the-night.md.
+midnight, if it is at most `KEPT_MAX_AGE_S` old when used.
+Never beyond that, and never for another day: a today figure is only today's,
+a tomorrow figure is tomorrow's until midnight and then, until the bell starts
+(`BELL_FROM_H`, 06:00 local), today's (OPT-34: at most `KEPT_MORNING_MAX_AGE_S`
+old, and it must have been usable at midnight).
+DECISIONS/OPT-27-last-fresh-forecast-for-the-night.md,
+DECISIONS/OPT-34-the-night-charge-keeps-its-level.md.
 
 OPT-17: each day may instead name a list (`pv_forecast_entities`,
 `pv_forecast_tomorrow_entities`), summed by `read_sum()`. One member that is
@@ -63,11 +65,16 @@ FRESH_S = 3 * 3600.0
 BELL_FROM_H, BELL_TO_H = 6.0, 20.0
 
 # OPT-27: a kept reading serves the night plan from the end of the bell until
-# this many hours after local midnight (the card: "local midnight + 3 h").
-KEPT_UNTIL_H = 3.0
-# OPT-27: and a reading older than this when used is not kept (a morning
-# figure must not drive the night).
+# local midnight, and a reading older than this when used is not kept (a
+# morning figure must not drive the night).
 KEPT_MAX_AGE_S = 12 * 3600.0
+# OPT-34: the evening's "tomorrow" figure, once midnight has passed, is today's
+# until the bell starts (`BELL_FROM_H`): nothing has been produced yet, so it
+# is still the whole day. A 17:05 reading is 12 h 55 min old at 06:00, so this
+# case may be older than `KEPT_MAX_AGE_S`, up to this, but only a figure that
+# was usable at midnight (at most `KEPT_MAX_AGE_S` old then), so one that has
+# not been used all evening does not come back at 00:00.
+KEPT_MORNING_MAX_AGE_S = 15 * 3600.0
 SAVE_EVERY_S = 900.0
 
 # Why there is no forecast (the page words are in ui_text `home_words`).
@@ -279,7 +286,7 @@ class Hold:
     `day0`) or "tomorrow" (all of `day0`), and `day0` is the day the figure is
     FOR. A today figure therefore never answers for another day, and a
     tomorrow figure answers for its own day only: tomorrow until midnight,
-    then today until midnight + `KEPT_UNTIL_H`. Only the newest reading of a
+    then today until the bell starts (OPT-34). Only the newest reading of a
     key is kept, so the next fresh reading ends the hold at once.
 
     `dump()` / `load()` are what the controller keeps in its store, so an
@@ -322,25 +329,27 @@ class Hold:
 
         The kept reading must be for `day0`, and either (a) at most `FRESH_S`
         old, or (b) used at night: from the end of the bell to local midnight,
-        or within `KEPT_UNTIL_H` after it, and then at most `KEPT_MAX_AGE_S`
-        old. `kept` is true when (b) was needed,
-        or `as_today` (a tomorrow figure standing in for today, after
-        midnight)."""
+        at most `KEPT_MAX_AGE_S` old. `kept` is true when (b) was needed,
+        or `as_today` (OPT-34: a tomorrow figure standing in for today, after
+        midnight and until the bell starts, `KEPT_MORNING_MAX_AGE_S`)."""
         rec = self._r.get((kind, entity, day0))
         if rec is None:
             return None
         day, at = rec
-        if (now - at).total_seconds() <= FRESH_S and not as_today:
+        age = (now - at).total_seconds()
+        if age <= FRESH_S and not as_today:
             return day, at, False
-        if (now - at).total_seconds() > KEPT_MAX_AGE_S:
-            return None
-        # Both sides: a tomorrow sensor is asked with day0 = the next
-        # midnight, which is in the future all day (round 2).
-        after_midnight = day0 <= now < day0 + timedelta(hours=KEPT_UNTIL_H)
-        evening = _local_hour(now) >= BELL_TO_H and now < _bounds(day0)[1]
         if as_today:
-            return (day, at, True) if after_midnight else None
-        return (day, at, True) if (evening or after_midnight) else None
+            ok = (day0 <= now < _bell_start(day0)
+                  and age <= KEPT_MORNING_MAX_AGE_S
+                  and (day0 - at).total_seconds() <= KEPT_MAX_AGE_S)
+            return (day, at, True) if ok else None
+        if age > KEPT_MAX_AGE_S:
+            return None
+        # A tomorrow sensor is asked with day0 = the next midnight, which is
+        # in the future all day, so the evening ends at that day's end.
+        evening = _local_hour(now) >= BELL_TO_H and now < _bounds(day0)[1]
+        return (day, at, True) if evening else None
 
     def dump(self) -> dict:
         return {"v": 1, "r": [
@@ -395,6 +404,15 @@ def _bounds(now):
     return d0.astimezone(timezone.utc), d1.astimezone(timezone.utc)
 
 
+def _bell_start(day0):
+    """The moment the bell starts on the local day that begins at `day0`
+    (`BELL_FROM_H` local, so 06:00 on a 23 h or 25 h day too)."""
+    here = day0.astimezone(BILLING_TZ)
+    t = datetime(here.year, here.month, here.day, tzinfo=BILLING_TZ) \
+        + timedelta(hours=BELL_FROM_H)
+    return t.astimezone(timezone.utc)
+
+
 def _today(state, day0, now):
     """`(Day, at, None)` or `(None, None, why)` for one today sensor."""
     today, why = _day(state, day0, now, today=True)
@@ -474,7 +492,7 @@ def read_sum(today, tomorrow, now: datetime, hold: Hold | None = None):
     where `Hold.stand_in()` allows it (the forecast then has `kept` set when
     that took a reading older than `FRESH_S`). After midnight, if the today
     side is still short, the tomorrow figures read before midnight are
-    today's, until midnight + `KEPT_UNTIL_H`. Without a `hold` this is
+    today's, until the bell starts (`stand_in()`). Without a `hold` this is
     exactly as before."""
     if not today:
         return None, NOT_SET, ()

goodwe_controller/app/ui_text/en.json

@@ -360,7 +360,7 @@
       "tops_up": "It charges from the grid to {pct}. That covers {kwh} of expensive hours the sun will not.",
       "no_forecast": "No solar forecast, so it charges from solar only."
     },
-    "solar": {"line": "Solar still to come today: {kwh}, we count on {count}. Updated {at}.", "uncovered": "The expensive hours need more than Leftover solar can hold. The rest, {kwh}, comes from another battery or the grid.", "kept": "Solar forecast read {at}, kept for tonight.", "no_forecast": "No solar forecast now, so Leftover solar charges from solar only.", "learning": "The add-on is learning your house's use. The plan starts after one full day."},
+    "solar": {"line": "Solar still to come today: {kwh}, we count on {count}. Updated {at}.", "uncovered": "The expensive hours need more than Leftover solar can hold. The rest, {kwh}, comes from another battery or the grid.", "kept": "Solar forecast read {at}, kept for tonight.", "no_forecast": "No solar forecast now, so Leftover solar charges from solar only.", "latched": "No solar forecast now. The charge under way goes on to its planned level.", "learning": "The add-on is learning your house's use. The plan starts after one full day."},
     "today": {
       "not_planned": "Not planned yet.",
       "no_runs": "No cheap or expensive hours left today.",

goodwe_controller/app/ui_text/nl.json

@@ -300,7 +300,7 @@
       "tops_up": "Ze laadt van het net tot {pct}. Dat dekt {kwh} dat de zon in dure uren niet levert.",
       "no_forecast": "Geen zonneprognose, dus ze laadt alleen met zonne-energie."
     },
-    "solar": {"line": "Nog verwachte zon vandaag: {kwh}, we rekenen op {count}. Bijgewerkt om {at}.", "uncovered": "De dure uren vragen meer dan Overschot zonne-energie kan opslaan. De rest, {kwh}, komt van een andere batterij of het net.", "kept": "Zonneprognose van {at}, bewaard voor vannacht.", "no_forecast": "Nu geen zonneprognose, dus Overschot zonne-energie laadt alleen met zon.", "learning": "De add-on leert eerst het verbruik van uw huis. Het plan start na één volle dag."},
+    "solar": {"line": "Nog verwachte zon vandaag: {kwh}, we rekenen op {count}. Bijgewerkt om {at}.", "uncovered": "De dure uren vragen meer dan Overschot zonne-energie kan opslaan. De rest, {kwh}, komt van een andere batterij of het net.", "kept": "Zonneprognose van {at}, bewaard voor vannacht.", "no_forecast": "Nu geen zonneprognose, dus Overschot zonne-energie laadt alleen met zon.", "latched": "Nu geen zonneprognose. De lopende lading gaat door tot het geplande niveau.", "learning": "De add-on leert eerst het verbruik van uw huis. Het plan start na één volle dag."},
     "today": {
       "not_planned": "Nog niet gepland.",
       "no_runs": "Vandaag geen goedkope of dure uren meer.",