Merge pull request 'OPT-34 fix (rig F-1): a replaced evening forecast does not come back' (#225) from OPT-34-fix into release/1.0

goodwe_controller/CHANGELOG.md

@@ -5,7 +5,7 @@
 ### Not in a release yet
 
 - The night grid charge no longer raises "Battery stopped limiting the peak" when the 15-minute period stays on the aim. A steadily rising load now alarms a little later, and with a peak margin of 0 a period can end up to 10 W over the limit without this alarm ([[SAFETY-126]]).
-- A night grid charge for *Leftover solar* no longer stops at 03:00 when the solar forecast sensors went quiet in the evening. The evening's figure is used until 06:00, and a charge that has started goes on to its level ([[OPT-34]]).
+- A night grid charge for *Leftover solar* no longer stops at 03:00 when the solar forecast sensors went quiet in the evening. The evening's figure is used until 06:00, unless a newer reading has replaced it that night, and a charge that has started goes on to its level ([[OPT-34]]).
 
 **A patch bump: two fixes, one for charging from the grid under the peak limit and one for the night charge of Leftover solar, and no option changes.**
 No option was added, removed or renamed, and no default changed (86 options).

goodwe_controller/DECISIONS/OPT-34-the-night-charge-keeps-its-level.md

@@ -34,6 +34,13 @@ known, so the add-on does not depend on it.
    midnight, so a 09:05 reading, unusable since 21:05, does not come back for five
    minutes after midnight. Same-day figures and the daytime rule keep 12 h and 3 h.
 4. A fresh figure replaces the kept one at once, as before.
+   *Rig F-1 fix:* once every today sensor has given a fresh reading after
+   midnight, the evening figure is no stand-in for that day any more
+   (`Hold.replaced()`). When that reading goes stale (3 h), it is kept or not
+   as any today reading (OPT-27), so it is "no forecast" at night and the latch
+   keeps the level it set; the older 17:05 figure never comes back. Rig: A
+   stopped at 69.5 % after a brighter 02:00 figure, then charged back to 94.4 %
+   from 05:15 when the 17:05 figure returned. Test: `test_opt34.py` F-1.
 
 **Step 1 alone is enough for the owner's shape, steps 2-3 are still needed.**
 With the window put back to midnight + 3 h the owner's night still ends as a
@@ -59,4 +66,4 @@ A fresh figure that LOWERS the level during a charge still lowers it, as before.
 **Mutants** (all red in `test_opt34.py`): no latch; window back to midnight + 3 h;
 15 h without a limit; no "usable at midnight" rule; the latch without the narrow
 gap, without its save, or not kept through the next run; the window 6 h after
-midnight instead of 06:00 local (DST); 15 h cut to 12 h; the new screen line off or shown without a cheap run; no expiry check; no pruning of a changed mode; no night bound; a left-the-plan battery keeping its level.
+midnight instead of 06:00 local (DST); 15 h cut to 12 h; the new screen line off or shown without a cheap run; no expiry check; no pruning of a changed mode; no night bound; a left-the-plan battery keeping its level; no `replaced()` check (F-1); `replaced()` with one of several today sensors (any, not all).

goodwe_controller/app/solarfc.py

@@ -38,7 +38,8 @@ midnight, if it is at most `KEPT_MAX_AGE_S` old when used.
 Never beyond that, and never for another day: a today figure is only today's,
 a tomorrow figure is tomorrow's until midnight and then, until the bell starts
 (`BELL_FROM_H`, 06:00 local), today's (OPT-34: at most `KEPT_MORNING_MAX_AGE_S`
-old, and it must have been usable at midnight).
+old, it must have been usable at midnight, and no fresh today reading may
+have replaced it since).
 DECISIONS/OPT-27-last-fresh-forecast-for-the-night.md,
 DECISIONS/OPT-34-the-night-charge-keeps-its-level.md.
 
@@ -351,6 +352,13 @@ class Hold:
         evening = _local_hour(now) >= BELL_TO_H and now < _bounds(day0)[1]
         return (day, at, True) if evening else None
 
+    def replaced(self, entities, day0) -> bool:
+        """OPT-34 F-1: every today sensor has given a fresh reading for
+        `day0` (so after midnight, newer than any evening figure). The
+        evening's tomorrow figure is then no stand-in for today any more,
+        also once those readings are stale."""
+        return all(("today", e, day0) in self._r for e in entities)
+
     def dump(self) -> dict:
         return {"v": 1, "r": [
             [kind, ent, d.day0.isoformat(), d.kwh, at.isoformat(),
@@ -492,7 +500,8 @@ def read_sum(today, tomorrow, now: datetime, hold: Hold | None = None):
     where `Hold.stand_in()` allows it (the forecast then has `kept` set when
     that took a reading older than `FRESH_S`). After midnight, if the today
     side is still short, the tomorrow figures read before midnight are
-    today's, until the bell starts (`stand_in()`). Without a `hold` this is
+    today's, until the bell starts (`stand_in()`), unless fresh today
+    readings have replaced them (`Hold.replaced()`). Without a `hold` this is
     exactly as before."""
     if not today:
         return None, NOT_SET, ()
@@ -537,7 +546,8 @@ def read_sum(today, tomorrow, now: datetime, hold: Hold | None = None):
         # day; all of them are needed, as for any sum.
         subs = [hold.stand_in("tomorrow", ent, day0, now, as_today=True)
                 for ent, _st in tomorrow]
-        if all(r is not None for r in subs):
+        if all(r is not None for r in subs) and not hold.replaced(
+                [ent for ent, _st in today], day0):
             days = [r[0] for r in subs]
             ats = [r[1] for r in subs]
             gaps = [g for g in gaps if g[0] != "today"]

goodwe_controller/test_opt34.py

@@ -36,6 +36,8 @@ DECISIONS note and the dev report.
       and before a charge has started, up and down;
    e. the last hour, after the kept figure has ended at 06:00: the charge under
       way goes on to its level, also over a restart; the screens say so.
+   f. F-1 (rig): one fresh reading at 02:00, then silent again: when it is
+      stale (05:00) the 17:05 figure it replaced does not come back.
 3. The text, EN and NL.
 4. `_latch_charge()` alone: a battery that leaves the plan leaves the latch.
 """
@@ -193,6 +195,47 @@ for label, now in (("01:00", at(4, 1, 0)), ("04:00", at(4, 4, 0))):
           "17:05 figure again", fr is not None and fr.kept
           and fr.today.kwh == 12.0 and fr.at == at(3, 17, 5))
 
+# F-1 (rig): a fresh today reading at 02:00 replaces the 17:05 figure; once
+# that reading is stale (05:00) the 17:05 figure must not come back.
+for label, kwh in (("brighter", 40.0), ("darker", 1.0)):
+    hn = hold_of(at(3, 17, 5))
+    f2 = rs(st(kwh, at(4, 2, 0)), st(kwh, at(4, 2, 0)), at(4, 2, 1), hn)[0]
+    f4 = rs(GONE, GONE, at(4, 4, 59), hn)[0]
+    snap = json.loads(json.dumps(hn.dump()))
+    f5 = [rs(GONE, GONE, t, hn)[0] for t in (at(4, 5, 1), at(4, 5, 59))]
+    f5s = [rs(st(kwh, at(4, 2, 0)), st(12.0, at(3, 17, 5)), t, hn)[0]
+           for t in (at(4, 5, 1), at(4, 5, 30))]
+    check(f"F-1 {label} 02:00 reading: used fresh at 02:01 and kept for "
+          "a missing sensor to 04:59 (3 h)",
+          f2 is not None and not f2.kept and f2.today.kwh == kwh
+          and f4 is not None and f4.today.kwh == kwh and not f4.kept)
+    check("... and from 05:01 (stale) to 05:59: no forecast, never the "
+          "17:05 figure (sensors gone, or showing their old readings)",
+          all(x is None for x in f5 + f5s))
+    back = solarfc.Hold(snap)
+    check("... also after a restart at 03:00 (the saved hold keeps the "
+          "02:00 reading)", rs(GONE, GONE, at(4, 3, 0), back)[0] is not None
+          and rs(GONE, GONE, at(4, 5, 1), back)[0] is None)
+# Two today sensors, only one read fresh: the sum never had a fresh figure,
+# so the evening one still stands in (as before).
+h2 = solarfc.Hold()
+td2 = [("sensor.t1", st(0.2, at(3, 17, 5))),
+       ("sensor.t2", st(0.2, at(3, 17, 5)))]
+tm2 = [("sensor.m1", st(6.0, at(3, 17, 5))),
+       ("sensor.m2", st(6.0, at(3, 17, 5)))]
+gone2 = [(e, GONE) for e, _s in td2], [(e, GONE) for e, _s in tm2]
+solarfc.read_sum(td2, tm2, at(3, 17, 10), h2)
+solarfc.read_sum([("sensor.t1", st(9.0, at(4, 2, 0))), td2[1]], tm2,
+                 at(4, 2, 1), h2)
+g5 = solarfc.read_sum(*gone2, at(4, 5, 1), h2)[0]
+check("two today sensors, only one read fresh at 02:00: the evening sum "
+      "(12.0) still stands in at 05:01", g5 is not None and g5.kept
+      and g5.today.kwh == 12.0)
+solarfc.read_sum([td2[0], ("sensor.t2", st(9.0, at(4, 2, 10)))], tm2,
+                 at(4, 2, 11), h2)
+check("... and once both have read fresh, it does not",
+      solarfc.read_sum(*gone2, at(4, 5, 20), h2)[0] is None)
+
 # DST: the 25 h autumn day and the 23 h spring day. The window ends at 06:00
 # LOCAL, not 6 h after midnight.
 for (y, mo, d), label in (((2026, 10, 25), "25 h day 2026-10-25"),
@@ -431,6 +474,19 @@ SCEN = {
                        soc={"a": 65.0, "b": 95.0},
                        marks=[at(4, 5, 50), at(4, 6, 10), at(4, 6, 45)]),
 }
+# F-1 (rig 4v): silent from 17:05, one fresh reading at 02:00, silent again.
+# It is stale at 05:00, before the 06:00 bell: the 17:05 figure it replaced
+# must not come back. Brighter (A's level drops), with a restart at 03:00,
+# and darker (the level rises).
+F1_MARKS = [at(4, 1, 55)] + [at(4, 2, 15) + timedelta(minutes=15 * i)
+                             for i in range(19)]
+for k, kwh, rst, hw in (("f1_bright", 30.0, (), 700.0),
+                        ("f1_bright_rs", 30.0, (at(4, 3, 0),), 700.0),
+                        ("f1_dark", 1.0, (), 500.0)):
+    SCEN[k] = dict(start=at(4, 1, 30), hours=5.5, house_w=hw,
+                   soc={"a": 50.0, "b": 50.0}, fresh_from=at(4, 2, 0),
+                   fresh_until=at(4, 2, 0), fresh=(kwh, kwh),
+                   restarts=list(rst), marks=F1_MARKS)
 t0 = time.time()
 with ProcessPoolExecutor(max_workers=min(6, os.cpu_count() or 2),
                          mp_context=get_context("fork")) as pool:
@@ -563,6 +619,45 @@ check("(control: the same latch, still valid until tomorrow 07:15, does "
       f"charge both: A {sl_['soc'][0]:.1f}, B {sl_['soc'][1]:.1f} %)",
       sl_["soc"][0] > 11.0 and sl_["soc"][1] > 11.0
       and sl_["marks"]["22:20"]["state"] == "latched")
+for k in ("f1_bright", "f1_bright_rs"):
+    fb = R[k]
+    mk = fb["marks"]
+    lo_t = mk["02:15"]["target_a"]
+    after = [mk[h] for h in sorted(mk) if h >= "02:15"]
+    print(f"    {k}: A level 01:55 {mk['01:55']['target_a']} -> 02:15 "
+          f"{lo_t}; A {fb['soc_at']['02:15'][0]:.1f} % at 02:15, "
+          f"{fb['soc_at']['06:55'][0]:.1f} % at 06:55; lines "
+          + " | ".join(sorted({x['line'] for x in after})))
+    check(f"F-1 {k}: a brighter fresh figure at 02:00 lowers A's level "
+          "below its SoC and A stops",
+          lo_t is not None and mk["01:55"]["target_a"] is not None
+          and lo_t < mk["01:55"]["target_a"] - 10.0
+          and lo_t < mk["02:15"]["soc_a"])
+    check(f"... after 02:00 the screen never says 'read 17:05' and from "
+          "05:15 there is no forecast, with the LOWERED level",
+          all("17:05" not in x["line"] and not (x["kept"] and
+                                                 not x["fc_none"])
+              for x in after)
+          and all(mk[h]["fc_none"] and near(mk[h]["target_a"], lo_t, 0.5)
+                  for h in mk if "05:15" <= h < "07:00"))
+    check(f"... so A stays at its lowered level to 07:00 "
+          f"({fb['soc_at']['02:30'][0]:.1f} -> "
+          f"{fb['soc_at']['06:55'][0]:.1f} %)",
+          fb["soc_at"]["06:55"][0] < fb["soc_at"]["02:30"][0] + 1.0)
+fd = R["f1_dark"]
+md = fd["marks"]
+print(f"    f1_dark: A level 01:55 {md['01:55']['target_a']} -> 02:15 "
+      f"{md['02:15']['target_a']}; A {fd['soc_at']['06:55'][0]:.1f} % "
+      "at 06:55")
+check("F-1 darker fresh figure at 02:00: the level rises, is held after "
+      "05:00 with no forecast (never 'read 17:05'), and A charges to it",
+      md["02:15"]["target_a"] is not None
+      and md["01:55"]["target_a"] is not None
+      and md["02:15"]["target_a"] > md["01:55"]["target_a"] + 3.0
+      and all("17:05" not in md[h]["line"] for h in md if h >= "02:15")
+      and all(md[h]["fc_none"] for h in md if "05:15" <= h < "07:00")
+      and near(fd["soc_at"]["06:55"][0], md["02:15"]["target_a"], 1.5))
+
 tb = R["tail_bfull"]
 check("B full: the run ends at 06:33 before A is at its level, so the next "
       "slice is only A's, planned after the figure has ended with A's "