Merge release/1.0 (16029d7) into MAINT-20-2

Nine conflicts in app/main.py, all in comments around new code: every code
line is release/1.0's, every comment follows the part 2 rule.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
glenn schrooyen
2026-11-08 17:28:03 +01:00
co-authored by Claude
51 changed files with 7745 additions and 189 deletions
@@ -6,6 +6,7 @@
- **Shortfall.** `sum_h max(0, house_h - trust x sun_h)` over every expensive hour, `/ EFF`, from the floor. It replaces the use over the run with no sun counted, which on a day and night tariff bought for the whole day (G1, `reports/UX-69-design-report.md`). The sun before the run is unchanged. A run with its dear hours after sunset gives the old number.
- **House use.** An hour with sun is read at no less than the night base, as the sun sum reads it (the profile is net of the sun). An hour without is read as measured. An hour with no forecast counts no sun.
- **Several batteries.** One total, shared by capacity with the ceiling passed on (`dayplan._spread()`). With the same floor and ceiling for all, they reach the ceiling together, so a shortfall that one battery cannot hold fills both to the same level, not one first. More than all hold: all go to the ceiling, `Plan.uncovered_kwh` is the rest, carried by another battery (Cheap hours, Solar first) or the grid, as the screen says.
- **Cheap hours** still fills up and is not counted in the share; **Solar first** still never charges from the grid. The mode help says so.
- **Not done:** the run length (OPT-25); netting what a Cheap hours or Solar first battery will hold against the shortfall.
- **Several batteries.** One total, shared by capacity with the ceiling passed on (`dayplan._spread()`). With the same floor and ceiling for all, they reach the ceiling together, so a shortfall that one battery cannot hold fills both to the same level, not one first. More than all hold: all go to the ceiling, `Plan.uncovered_kwh` is the rest, carried by the grid (OPT-36 counts the other batteries first), as the screen says.
- **Cheap hours** still fills up; **Solar first** still never charges from the grid. The mode help says so.
- **OPT-36 (0.20.1):** Cheap hours and Solar first batteries give first; Leftover solar shares only the rest of the shortfall (`dayplan._cheap_give_kwh()`, `_solar_first_give_kwh()`). A Cheap hours battery counts the level its charge reaches: the cheap run, then later slots still worth charging in and no dearer than any slot from the expensive hours on, under the peak aim, x EFF stored. A Solar first battery counts what is left after the house's use until the hours start, each hour at no less than the night base. Live, 2026-11-02: A at 60 % went to 95 % while B at 94 % covered the hours. Owner, 2026-11-02: Solar first counts too. Test: `test_opt36.py`.
- **Not done:** the run length (OPT-25). On a dynamic tariff the Cheap hours count guesses where control will charge; a counted slot can still go unused, and that gap comes from the grid.
@@ -44,3 +44,5 @@ tonight." (`home_words.solar.kept`, EN and NL). The event log says
**Not changed.** Control with modes off: it does not use the forecast. The
screens still show the line, so it says "kept", not "used for the plan". A
forecast that stays fresh. The hold does not look at the weather, only at the clock and the day.
**OPT-38 (2026-11-07).** A kept TOMORROW reading (at most `KEPT_MAX_AGE_S` old, same `day0`) now stands in at any hour, not only from `BELL_TO_H`: one sensor that stopped at 14:54 had dropped the whole sum (OPT-17) from 17:54 to 20:00. Today readings and the OPT-34 morning rules are unchanged; `test_opt38.py`.
@@ -34,6 +34,13 @@ known, so the add-on does not depend on it.
midnight, so a 09:05 reading, unusable since 21:05, does not come back for five
minutes after midnight. Same-day figures and the daytime rule keep 12 h and 3 h.
4. A fresh figure replaces the kept one at once, as before.
*Rig F-1 fix:* once every today sensor has given a fresh reading after
midnight, the evening figure is no stand-in for that day any more
(`Hold.replaced()`). When that reading goes stale (3 h), it is kept or not
as any today reading (OPT-27), so it is "no forecast" at night and the latch
keeps the level it set; the older 17:05 figure never comes back. Rig: A
stopped at 69.5 % after a brighter 02:00 figure, then charged back to 94.4 %
from 05:15 when the 17:05 figure returned. Test: `test_opt34.py` F-1.
**Step 1 alone is enough for the owner's shape, steps 2-3 are still needed.**
With the window put back to midnight + 3 h the owner's night still ends as a
@@ -59,4 +66,4 @@ A fresh figure that LOWERS the level during a charge still lowers it, as before.
**Mutants** (all red in `test_opt34.py`): no latch; window back to midnight + 3 h;
15 h without a limit; no "usable at midnight" rule; the latch without the narrow
gap, without its save, or not kept through the next run; the window 6 h after
midnight instead of 06:00 local (DST); 15 h cut to 12 h; the new screen line off or shown without a cheap run; no expiry check; no pruning of a changed mode; no night bound; a left-the-plan battery keeping its level.
midnight instead of 06:00 local (DST); 15 h cut to 12 h; the new screen line off or shown without a cheap run; no expiry check; no pruning of a changed mode; no night bound; a left-the-plan battery keeping its level; no `replaced()` check (F-1); `replaced()` with one of several today sensors (any, not all).
@@ -0,0 +1,92 @@
# SAFETY-128 - a reading that does not answer the last command is not current
**Status: FIXED on release/1.0 (after 0.20.0).** Code: `app/lateness.py`,
`control.compute(hold=, hold_w=)`, `peakguard.cap_floor()`,
`Controller._readings_late()`, `Controller.read_inputs()` (feeds the fits).
Tests: `test_safety128.py` (`tools/safety128_rig.py`, TEST-23's plant).
**Cause** (SAFETY-127 report). Home Assistant can deliver readings late but
still changing. Every gate judges arrival (SAFETY-01) or a value that stops
changing (SAFETY-18, SAFETY-37), so a late reading passes them all. The loop
then corrects, each cycle, an error its last command has already fixed: a
loop with a dead time far longer than its cycle, which swings and grows. The
peak guard projects the quarter from the same late figures and pushes to the
inverter limit in its last seconds (OPT-34 rig F-2).
**Rule.**
1. *Find late readings* (`Answer.watch()`, every 2 s, from the 1 s reads).
- **Batteries** (`Delay`): fit one `d` so that each battery's reading minus
its own request at `t - d` (averaged over 2 s for the battery's lag and
the read) is flattest. The house does not move that reading, so a
repeating load cannot line up with it (review round 1, B1). Needs 5 new
readings in 60 s and the request and the reading each to move 300 W.
Late from 9.5 s, after 2 late fits in a row. A late fit must explain 75 %
and beat every fit up to 6.5 s by 2x; the shortest delay that fits about
as well (within 2x, or within 2 500 W^2) is taken instead, since a swing's
period fits again. A slow battery (3-10 s lag) answering at once gives no
late fit; at a 15-20 s lag some fits read late (see "What it costs").
No late battery fit counts for 60 s
after a battery gave nothing under its ask: an emptied battery lined its
old movement up as 85 s late.
- **Meter** (`MeterFit`, round 1's fit): `meter(t) + request(t - d)` is the
house alone at the right `d`. Quick, but confounded by the house. Needs
16 new values in 60 s, so only a meter of about 3.5 s or faster: at 5 s
it gave false delays. Late over 8 s. At a slower meter (a 5 s P1) the
same fit runs over 120 s and calls only a delay over 16 s late, on 2 fits
in a row, and never ends a verdict (SAFETY-129). Alone it reads 35-90 s on healthy
runs, a few times in a row; only the slow-start and battery-fit check
below keeps those from a verdict. 10 s late is not found: it cannot be
told from a 10 s slow battery, and it does not swing at 5 s.
- A meter fit says late only with evidence from the batteries' side too:
a request more than 100 W from the battery reading took 3 s or more to
move a reading towards it (`SLOW_START_S`; on time they start within
0-1 s in the recorded house at 1 and 2 s ticks), and no battery fit in
the last 60 s said on time (under 9.5 s). An ask during which a battery
gave nothing under its ask is no slow evidence. A repeating load fools
the meter fit but not these. No meter verdict in the first 150 s.
- On time again on a battery fit under 6.5 s, or a meter fit within 8 s
unless a battery fit said late (hysteresis). With no fit for 120 s the
verdict is dropped.
2. *Hold.* While late, a request more than 100 W from the battery reading is
pending until that reading comes within 100 W of it, or half way. Meanwhile
the loop may not lead the reading by more than 100 W that way, and the peak
guard's floor is capped at the request that stands. The loop is not held
while a battery gives nothing under its ask or refuses (SAFETY-52/94).
3. *Late verdict.* Pending for `LATE_S` (23 s) with no reading moving 100 W
towards it: control holds, then after `stale_input_s` goes to 0 W
("readings late"), as for a missing input.
4. *Seen.* While late an amber row says "Home Assistant readings show requests
about N seconds late, so control holds the batteries." (NL too). N is the
batteries' fitted delay: Home Assistant's lag plus the battery's own
answer. While only the meter has shown it the row gives no figure. One
WARNING (`READINGS_LATE`) per episode and one line when it ends.
**With readings on time** no verdict was reached in any reviewer scenario
(owner-shaped house, ovens 20/40 to 60/60 s, clouds, a 300 W/s battery, 6-8 s
dead times, HA jitter 0-4 s, an EV following the meter, capped batteries, HA
restarts, meter pauses; 1, 2 and 5 s meters) nor in 2 days of the recorded
house at 1, 2 and 5 s ticks, so the commands are the same as without the gate.
The fits run all the time; the pull request has the counts.
**What it costs.**
- Found 36-68 s after a lag starts at 1-2 s meters; at a 5 s meter 30 s late
is found within 100 s after it starts (88 s measured) (the loop first moves 100 W after 60 s,
SAFETY-129) with the export above -200 W, and 10 s late is not found. Until
then control is as before. In a quiet house the "readings late" row stays
about 2 minutes longer after the lag ends (126 s against 32 s on base); no
harm measured, and it clears as early as on base when the load changes.
- A battery that answers slowly is read as late readings and held: a dead
time of 10 s or more (8 s reads 7-8; 10 s reads 10 at 1-2 s meters, 9 at
5 s), or a 15-20 s lag (review round 2). The GoodWe measured settles in 6 s
(`freshness.PLANT_SETTLE_S`); the row then blames Home Assistant wrongly.
- The row can stay up to 180 s after the readings are on time on a steady
site. While only the meter has shown the delay it gives no figure.
- A battery that cannot follow while the readings are late (at a limit, not
reading 0 W) is held, then set to 0 W after 38 s.
- The fits cost a few ms every 2 s (x86, measured after 3 idle days); the
history they read is bounded (60 s of readings, 150 s of requests).
**Left out.** The setpoint -> Home Assistant round trip: the fitted delay gives
the figure from readings the add-on already has. The hold's step-and-pull-back
at extreme lag and the arbitrage charge's sizing on late readings
(`charge_room_w()`) are follow-up cards.
@@ -0,0 +1,21 @@
# SAFETY-137 - the shutdown 0 W is the last write every unit hears
SMA-10 made a hand-back release wait for a write already on its way
(`_release_last`, `Inverter.inflight`). A GoodWe unit skipped that, and rig 5b
saw a cycle write (-2 823 W) land after the shutdown 0 W and stay until relaunch.
Now the 0 W of shutdown and of the Stop button is a *last* write
(`_write_unit(last=True)`), for every driver:
- It first waits, at most `SETTLE_BUDGET_S` (2 s), for the writes already on
its way to that unit (`_await_inflight`, shared with `_release_last`). A hung
write is not waited out and not cancelled; the 0 W goes anyway.
- Once shutdown has begun (`stopping`), every other write (cycle, heartbeat,
safety stop) is dropped at the same place, right before the write is created.
That is the gate that keeps the control cycle quiet; the cycle itself needs
no cancelling.
- Worst case for a GoodWe unit is 2 s + `WRITE_BUDGET_S` (4 s) = 6 s, the same
as `HAND_BACK_WORST_S`; the units are handled together.
Not touched: the heartbeat's own timing, safety-stop paths, the maintenance
abort (still a plain 0 W).
@@ -0,0 +1,62 @@
# SMA-10 - two idle states: hold 0 W, or hand the inverter back
**Status: for review.** Tests: `test_sma10_two_idle_states.py`. Source: SMA-9
report sections 1.4, 3.2; owner decision 2026-10-20 (Q1, "as proposed").
A unit is idle in one of two ways, and the driver says which:
`Driver.hands_back`. **False (GoodWe):** the add-on holds 0 W, `release()` is
`write(0)` and the Controller never calls it, so Stop keeps today's 0 W write and
today's 0 W heartbeat. **True:** `release()` gives the unit back to whatever
steers it by itself (SMA energy management), once, and then nothing is written
to it until control starts again.
**Why two.** When the add-on goes away (Stop, shutdown, SIGTERM, restart, task
death, a config refusal) nobody is left to hold a number, so the unit goes back
to its own controller. When the add-on itself decides to stop (sign trip, site
sign, circulating power, a refused or non-finite write, missing readings,
another writer, an inverter fault) it has just said it cannot trust that
battery, and handing it to something else would let that something do what it
likes with it. Those hold 0 W through `write(0)` for every unit, whatever its
driver. Shadow mode on a unit that hands back uses `release()`: it must leave
the unit alone, and 0 W would be a command.
**Where.** `Controller.command_site_release()` sits beside `command_site_zero()`
(unchanged). Stop and `shutdown()` call it; the two maintenance-abort paths keep
zero (the add-on runs on, the sentence says 0 W for a moment). With no
hand-back unit it is `command_site_zero()`. `_write_unit()` also refuses to write
a hand-back unit while the add-on is idle (stopping, stopped, or shadow mode): it
releases once and returns, so the heartbeat and the cycle's own "stopped" 0 W
cannot reach it. A safety stop runs while control runs, so it falls through to
the write. Start re-arms: `released` drops and `last_write` clears.
**Bounds.** `RELEASE_BUDGET_S` equals `WRITE_BUDGET_S` (4 s): units are released
together, so one slow unit costs the kill window no more than a hung GoodWe
write does. A release at the bound is abandoned, not awaited. A release that
raises, hangs or returns false leaves that unit un-handed (the next idle write
tries again) and does not touch the others. Exit 78 before the Controller
exists has no driver to release and keeps today's path.
**Words.** Every sentence that said "set to 0 W" for the add-on going away or
for shadow mode is built from the roster in `app/handback.py` (EN and NL): a
roster with no hand-back unit gets None and keeps its old literal, so a GoodWe
site sees no changed byte. A hand-back unit's words (`Driver.words`: `back`,
`back_nl`) come from its driver; one without them gets "its own control" /
"de eigen sturing" (the article as in the generic clause; PR #239 review
comment 2495, not the brief). MQTT `status` is unchanged; the Home battery
row of a handed-back unit says "Back under SMA control" and carries
`handed_back`.
**Not done here (SMA-4, SMA-5, SMA-11):** any SMA/FoxESS driver, the read-back
after `release()`, per-driver timing, DOCS.md's safety model and the
"control stalled" sentence, the site-level "Held at 0 W" tiles in shadow mode on
a mixed roster, and the MQTT per-unit setpoint of a handed-back unit (it still
publishes 0).
**After a failed release** (returns False, raises or hangs past the bound) the
unit gets one `write(0)` through `Driver.release_fallback()` (default
`write(0)`; a driver where 0 W is not always safe overrides it), then nothing:
a repeated 0 W would keep resetting the inverter's own fallback timer, so after
that the unit is left to its own fallback. The release is retried once per
heartbeat; one error line and one action-log event per failure streak, one line
on recovery. Worst case at shutdown is `RELEASE_BUDGET_S` + `FALLBACK_BUDGET_S`
(6 s).
@@ -0,0 +1,55 @@
# SMA-2 - each inverter is reached through a driver
**Status: for review.** Tests: `test_sma2_driver.py`. Source: SMA-9 report
section 1.1. A pure refactor: no behaviour change for GoodWe, no change to a
command, a log line or a sentence the add-on shows.
Every place that read or steered an inverter used the GoodWe ESPHome entities
directly, so a second make meant touching all of them. `app/drivers.py` now
holds one `Driver` per roster entry (`Inverter.driver`): `configured()`,
`actuator_id()`, `limits()`, `read()`, `write()`, `release()`, `probe()`, and
`link_stale_s`, `batt_bound`, `settle_s`, `brand`, `words`. `GoodWeEsphomeDriver`
makes the same calls in the same order as the code it replaced; its
`release()` is `write(0)`.
**Moved to the driver:** the SoC and power read (`_read_unit`), the write
(`_write_unit`, still under the SAFETY-28 gate, the clamp, shadow mode and the
write budget), the device range (`run_control`, `amain`, the settings panel),
liveness and its window (`has_writer`, `is_live`, `link_note`), the freshness
bound, the status rows' "is a source set" test, rule 2's per-unit range check
and SAFETY-28's census. **Not moved:** the clamp, the gate, the heartbeat and
`command_site_zero()`; stop is still 0 W for every unit, and nothing calls
`release()` yet (SMA-10). `settle_s` and the control constants are not read
from the driver yet either: with only GoodWe they would change nothing.
**The seam is not TCP-shaped.** The census asks each entry's driver for its
actuator id: the setpoint entity for GoodWe, `host:port:unit_id` for a driver
reached over a socket, so two units behind one gateway are two actuators and
one unit named twice is the SAFETY-28 refusal. The census still counts every
`setpoint_entity` key as before and adds the driver's own actuator to it (parity
with 23ff9d0 on 349 option shapes is a test). The driver binds its Home
Assistant on first use by the Controller, so a unit added later is wired too.
**Schema, and why the new keys are not in it.** The brief asks for
`driver`, `host`, `port`, `unit_id`, `maintenance` per entry and for
`setpoint_entity` to become optional. Owner decision 2026-11-07: clients see
nothing new until SMA-4/SMA-7 use the keys. The Supervisor form shows every
schema key (and marks required ones), and a nested entry with a key the schema
lacks has that key silently dropped (measured, shape probe section K), so the
only way to hide them is to keep them out of config.yaml. So for now config.yaml, the
translations and the labels are exactly 23ff9d0's, and `setpoint_entity` stays
required there: until a driver without one is selectable, no entry can lack it.
The code underneath is the brief's: the driver defaults to `goodwe_esphome`,
rule 9(d) is per driver (a GoodWe entry without a setpoint entity is refused,
same sentence), and rule 9(h) checks `driver`, `host`, `port`, `unit_id`,
`maintenance` when a caller that skips the schema (the panel's save path,
sim/inject, the checks) supplies them; those sentences name the key, having no
label. When SMA-4 needs them: add them to the schema as optional with
translations, make `setpoint_entity` `str?`, and add them to `uilayout` as card
fields or a hidden list.
**Checked against 23ff9d0:** replay digests and a log of every Home Assistant
call are byte-identical at N=1 and N=2; the TEL-07 announce diff is 0/0/0; a
fake driver beside a GoodWe unit allocates, grades, drops and publishes per
unit; a call site that goes round the seam is caught.
@@ -97,9 +97,14 @@ is `SolarToday.add()` after the solar read, in a `try`, in memory.
- Prices are planned about a day ahead (`priceruns.HORIZON`), so a block of
tomorrow later than 24 h from now, or one that needs prices not yet
published, appears as the day comes near. The tab says so.
- Today's blocks are Home's, and are not extended or shortened: control may
plan more once they end, and Home draws it then. The curve stops charging
at its level (`charge_to()`) and giving at the floor.
- Today's blocks are Home's. UX-77: Home draws today's grid block to where
control will stop charging, as the tab does for tomorrow's: the cheap run
rolls on at its end while a battery is short (`cheaproll.roll_end()`, the
same loop as `dayview._levels()`), the level read at the charge power the
peak limit leaves (peak aim less the house, else the rating) and shared by
room left to full as control shares it (`packshare.weights()`, re-read as
the levels move), and each battery's block ends at its own stop. The curve stops charging at its level
(`charge_to()`) and giving at the floor.
- Today, a give run in the plan on a battery that is empty now has no block on
Home (no energy to give). The tab equals Home, and says so: "The add-on plans
to give power. No block shows while a battery has no energy to give." The