Files
glenn schrooyenandClaude Opus 5 4bd659c499 TEL-01 review fixes: stop the age sensor tripping installs that have no meter
T-1, and it was a fleet-wide trip to zero. publish() emitted p1_age
unconditionally, and published_age_s counts from P1Ingest.__init__ when no
sample has ever arrived. With meter_source defaulting to off, every existing
install would have published sensor.p1_sample_age_s climbing without bound; the
ESP32 does `has_state() && state >= max_age_s` and forces the layer-1 failsafe,
so each of them would have pinned its inverter at 0 W within 30 s. Exactly the
opposite of the zero-regression the off default was for. The key is now omitted
from the payload AND from MQTT discovery when P1 is off, so the entity does not
exist at all - which is the status quo, and what has_state() is testing for.
The predicate is one function, is_enabled(), because the grid reading, the task
start and the discovery announcement have to agree or this comes back.

T-2, connect no longer manufactures a sample. get_states returns whatever HA
currently holds, which after a Core restart is a RestoreEntity value of unknown
age; stamping it with ingest_ts=now reset the age and reported a fresh meter
that could have been dead for an hour. run()'s own docstring already said a
reconnect must emit nothing - the code disagreed with it, and a test asserted
the violation. The cache is still primed, so the first real state_changed
builds a complete sample; the age just stays honest until one arrives.

T-3, gaps are no longer filled with the last held value. The averager held a
sample forward across any interval, so a meter dying at 5 kW and returning ten
minutes later credited 5 kW x 600 s to the capacity-tariff accumulator - a
fabricated peak on a permanent record. The hold is capped at max_age_s: past
that the stretch is walked so block boundaries still land correctly, but
nothing accumulates and elapsed does not grow, which is what finally makes the
comment about a gap dragging the billed average down true. Same threshold for
control and billing: a reading too old to steer by is too old to bill by.

T-4, the out-of-order/duplicate guard is covered. It was untested, and the
reason is worth recording: the obvious assertion passes without the guard,
because the negative interval is separately refused by the covered > 0 test.
What the guard prevents is the timestamp REWIND, which only shows up one sample
later as a re-integrated window. The test now goes one sample later.

T-6, DOCS was wrong about latency. meter_max_age_s and stale_input_s stack, so
meter death to 0 W is 45 s and not 30. Documented as a table with both clocks.

Also documented the T-5 asymmetry rather than papering over it: the age
measures arrival, not change, so a stuck MQTT bridge republishing its last
telegram still looks fresh. Correct on ha_dsmr, not detectable on mqtt_p1
without a change-detector. Written up as a known limit.

Writing the T-1 test caught a second defect in the test itself: it recorded
only MQTT topics, and object_id lives in the payload, so "the age sensor is not
announced" had been passing for the wrong reason.

test_p1.py: 99 -> 122 checks. 14 mutations run, all 14 red, files restored
byte-identical - including one per fix above.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Du77usMj8XNKNFZGmUiWDa
2026-08-24 23:16:31 +02:00

566 lines
25 KiB
Python

"""GoodWe RS485 Controller add-on - entry point and orchestration.
WHAT THIS THING DOES, in one paragraph: it reads the house's net grid power from
Home Assistant, decides how hard the battery should charge or discharge to hold
that at zero, and writes that figure to an ESP32 which puts it on the inverter's
RS485 meter bus. Once a month it runs a full low->full battery cycle so the BMS
can balance cells and recalibrate its coulomb counter.
⚠️ THE ONE THING TO UNDERSTAND BEFORE CHANGING ANY OF THIS: the inverter holds
the last command it understood FOREVER. It has no meter-timeout of its own.
Measured on real hardware: a controller died mid-command and the inverter held
5 kW of discharge for 113 s until a human noticed. Every failsafe in this system
exists because of that one fact:
layer 1 the ESP32's own watchdog - if we stop refreshing for ~30 s it
commands 0 W and KEEPS WRITING it. Stopping is the failure, not the
fix. This add-on's heartbeat is what feeds it.
layer 2 wind-down before a firmware update, in the ESP32.
layer 3 the optional RS485 e-stop, which writes 0 W after 30 s of total bus
silence. It is the ONLY thing that covers this add-on's host dying.
So: when in doubt, this process stops writing, and the hardware takes the
battery to zero on its own. Never "hold the last value to be safe".
"""
import asyncio
import contextlib
import json
import logging
import signal
from datetime import datetime, timezone
import aiohttp
from .arbiter import (
P_LOOP, P_MAINT_SHAPE, P_MAINTENANCE, P_SAFETY, Claim, resolve,
)
from .control import Tuning, compute, maintenance_charge_floor, peak_at_risk
from .hass import HomeAssistant
from .maintenance import IDLE, MaintConfig, Maintenance
from .mqtt import MqttPublisher
from .p1 import P1Ingest, build_source, is_enabled
from . import web
OPTIONS_PATH = "/data/options.json"
_LOG = logging.getLogger("goodwe")
def load_options() -> dict:
try:
with open(OPTIONS_PATH, "r", encoding="utf-8") as fh:
return json.load(fh)
except (OSError, json.JSONDecodeError) as err:
_LOG.error("cannot read %s (%s) - using defaults", OPTIONS_PATH, err)
return {}
class Controller:
def __init__(self, opts: dict, hass: HomeAssistant, store, mqtt_pub):
self.o = opts
self.hass = hass
self.store = store
self.mqtt = mqtt_pub
self.tuning = Tuning(
gain=float(opts.get("gain", 0.6)),
max_w=float(opts.get("max_w", 2000)),
slew_w=float(opts.get("slew_w", 1000)),
deadband_w=float(opts.get("deadband_w", 15)),
target_grid_w=float(opts.get("target_grid_w", 0)),
step_w=int(opts.get("step_w", 10)),
saturation_w=float(opts.get("saturation_w", 500)),
saturation_cycles=int(opts.get("saturation_cycles", 3)),
# 0 / unset means "follow max_w", which is the recommended
# value. Read the note in control.py before raising it above
# max_w: every watt above the rail is unwind latency.
integrator_max_w=(float(opts["integrator_max_w"])
if opts.get("integrator_max_w") else None),
)
self.maint = Maintenance(
MaintConfig(
enabled=bool(opts.get("maintenance_enabled", False)),
interval_days=int(opts.get("maintenance_interval_days", 28)),
start_hour=int(opts.get("maintenance_start_hour", 10)),
discharge_w=float(opts.get("maintenance_discharge_w", 2500)),
charge_w=float(opts.get("maintenance_charge_w", 2500)),
soc_floor=float(opts.get("maintenance_soc_floor", 11)),
soc_target=float(opts.get("maintenance_soc_target", 99)),
hold_min=int(opts.get("maintenance_hold_min", 120)),
),
store,
)
# P1 ingestion (TEL-01). `meter_source: off` keeps the original
# single-entity meter_entity path, so an existing install is unchanged
# until it opts in.
self.p1 = P1Ingest(phases=int(opts.get("meter_phases", 1)),
max_age_s=float(opts.get("meter_max_age_s", 30)))
self.p1_enabled = is_enabled(opts)
# live state
self.auto = bool(store.data.get("auto", opts.get("auto_start", False)))
self.target = 0.0
self.sat_count = 0
self.i_w = 0.0 # the loop's integrator, carried between cycles
self.loop_w = None # what the loop asked for last cycle, or None
self.reason = "starting"
self.grid = self.soc = self.batt = None
self.peak_fc = None
self.cheap = False
self.last_write = None
self.last_write_at = None
self.last_write_ok = None
self.inputs_bad_since = None
self.dev_min = self.dev_max = None
self.events: list[str] = []
self.stopping = False
# -- helpers -------------------------------------------------------------
def log_event(self, msg: str) -> None:
stamp = datetime.now(timezone.utc).strftime("%H:%M:%S")
self.events.insert(0, f"{stamp} {msg}")
del self.events[40:]
@property
def inputs_ok(self) -> bool:
return None not in (self.grid, self.soc, self.batt)
# -- io ------------------------------------------------------------------
async def read_inputs(self) -> None:
o = self.o
if self.p1_enabled:
# ⚠️ P1 is the only authoritative measurement of what the utility
# sees (§5.1). When it is stale this is None, which falls into the
# existing "inputs missing -> command 0 W" path below. There is
# deliberately NO fallback to an inverter-side figure: the
# inverter's own AC power correlates 0.998 with battery power and
# 0.09 with the real meter, so a controller that failed over to it
# would be regulating against its own output.
self.grid = self.p1.net_w
else:
self.grid = await self.hass.number(o.get("meter_entity", ""),
bool(o.get("meter_invert")))
self.soc = await self.hass.number(o.get("soc_entity", ""))
self.batt = await self.hass.number(o.get("batt_entity", ""),
bool(o.get("batt_invert")))
if o.get("peak_forecast_entity"):
raw = await self.hass.number(o["peak_forecast_entity"])
# Accept kW or W - nobody's quarter-hour forecast is 50 W.
self.peak_fc = None if raw is None else (raw * 1000 if abs(raw) < 50 else raw)
else:
self.peak_fc = None
now_e, avg_e = o.get("price_now_entity"), o.get("price_avg_entity")
if now_e and avg_e:
now_p = await self.hass.number(now_e)
avg_p = await self.hass.number(avg_e)
self.cheap = (now_p is not None and avg_p is not None and now_p <= avg_p)
else:
# Fixed-tariff site: never force a paid grid top-up, wait for sun.
self.cheap = False
if self.inputs_ok:
if self.inputs_bad_since is not None:
self.log_event("inputs recovered")
self.inputs_bad_since = None
elif self.inputs_bad_since is None:
self.inputs_bad_since = datetime.now(timezone.utc)
async def write_setpoint(self, value: float, *, force: bool = False) -> None:
entity = self.o.get("setpoint_entity", "")
if not entity:
return
# Never send a value the device will reject outright. A rejected write is
# silent from the controller's point of view, and the inverter then keeps
# doing whatever it was already doing.
if self.dev_min is not None and self.dev_max is not None:
value = max(self.dev_min, min(self.dev_max, value))
changed = (self.last_write is None or value != self.last_write)
if not (changed or force):
return
ok = await self.hass.set_number(entity, value)
self.last_write_ok = ok
if ok:
self.last_write = value
self.last_write_at = datetime.now(timezone.utc)
# -- the cycle -----------------------------------------------------------
async def cycle(self) -> None:
"""Gather claims, resolve precedence, write the winner.
Nothing here decides who wins - arbiter.py does, by one rule. This
method's only job is to state honestly what each strategy wants.
"""
now = datetime.now(timezone.utc)
result = self.maint.tick(now, self.soc)
for msg in result.events:
self.log_event(msg)
stale_after = float(self.o.get("stale_input_s", 15))
if not self.inputs_ok:
bad_for = (now - self.inputs_bad_since).total_seconds() if self.inputs_bad_since else 0.0
if bad_for < stale_after:
# A single missed poll is not a fault. Hold, keep the heartbeat
# going, and give it a few seconds to come back.
self.reason = f"inputs missing {bad_for:.0f}s"
return
claims = []
# --- safety limits: these bind EVERY strategy, always ----------------
# Defence in depth: write_setpoint() clamps to the device range too. The
# duplication is deliberate - one of them is the policy, the other is the
# last thing between a bug and the hardware.
if self.dev_min is not None and self.dev_max is not None:
claims.append(Claim.limit("device", P_SAFETY, self.dev_min, self.dev_max, "rating"))
max_w = float(self.o.get("max_w", 2000))
claims.append(Claim.limit("supervised", P_SAFETY, -max_w, max_w, "max_w"))
# --- safety stops ----------------------------------------------------
if not self.auto:
claims.append(Claim.set("safety", P_SAFETY, 0.0, "stopped"))
elif not self.inputs_ok:
if self.target != 0.0:
self.log_event("inputs missing - commanding 0 W")
claims.append(Claim.set("safety", P_SAFETY, 0.0, "inputs missing"))
# --- maintenance -----------------------------------------------------
if result.owns_setpoint:
claims.append(Claim.set("maintenance", P_MAINTENANCE,
float(result.setpoint_w or 0.0), result.phase))
self.sat_count = 0
elif result.charge_only:
# Money outranks the maintenance schedule: while the quarter-hour
# projection is over the cap, the charge-only shaping is simply not
# claimed, so the loop can discharge and shave the peak. When a
# dedicated peak-shaving strategy arrives it will claim SET at
# P_PEAK and outrank this limit without any code here changing.
if peak_at_risk(self.peak_fc, float(self.o.get("peak_cap_w", 3500))):
self.log_event("peak at risk - maintenance charge shaping suspended")
else:
hi = 0.0
reason = "charge-only"
if self.cheap:
floor = maintenance_charge_floor(
float(self.o.get("maintenance_charge_w", 2500)),
self.peak_fc, float(self.o.get("peak_cap_w", 3500)))
if floor > 0:
hi, reason = -floor, "cheap-window charge"
claims.append(Claim.limit("maintenance", P_MAINT_SHAPE, hi=hi, reason=reason))
# --- the grid-following loop -----------------------------------------
if self.auto and self.inputs_ok:
# ⚠️ prev is the ARBITER's last output, not the loop's own last
# wish. If something outranked the loop, that is what the hardware
# actually did, and the controller must track reality or it jumps
# the moment it regains control.
#
# ⚠️ The integrator has to track the same reality, and it is no
# longer prev_w, so it needs saying out loud: if the arbiter did not
# give the loop what it asked for last cycle, the loop's
# accumulated error belongs to a command that never happened.
# Re-seed from what the hardware was actually told. This is the
# failsafe case too - every layer-1 stop resolves to 0 W, so
# entering failsafe re-seeds the integrator to zero and the first
# cycle after release starts from zero instead of dumping the whole
# stale period as power.
if self.loop_w is None or self.target != self.loop_w:
self.i_w = self.target
decision = compute(
prev_w=self.target,
grid_w=self.grid,
actual_w=self.batt,
tuning=self.tuning,
sat_count=self.sat_count,
i_w=self.i_w,
)
if decision.frozen and self.sat_count < self.tuning.saturation_cycles:
self.log_event(f"saturation freeze ({self.target:.0f} W vs {self.batt:.0f} W)")
self.sat_count = decision.sat_count
self.i_w = decision.i_w
self.loop_w = decision.target_w
claims.append(Claim.set("loop", P_LOOP, decision.target_w, decision.reason))
else:
# Stopped or blind: no accumulation may survive the outage.
self.i_w = 0.0
self.loop_w = None
resolution = resolve(claims)
if resolution.contradiction:
self.log_event("ARBITER CONTRADICTION - commanding 0 W, see the log")
self.target = resolution.target_w
self.reason = resolution.explain()
await self.write_setpoint(self.target)
# -- tasks ---------------------------------------------------------------
async def run_control(self) -> None:
entity = self.o.get("setpoint_entity", "")
if entity:
self.dev_min, self.dev_max, _ = await self.hass.limits(entity)
if self.dev_max is not None and self.tuning.max_w > self.dev_max:
self.log_event(
f"clamp {self.tuning.max_w:.0f} W exceeds the device maximum "
f"{self.dev_max:.0f} W - the device wins")
last_grid = object()
heartbeat = float(self.o.get("heartbeat_s", 10))
last_beat = 0.0
while not self.stopping:
await self.read_inputs()
# A cycle is one meter update, exactly as on the reference install.
if self.grid != last_grid:
last_grid = self.grid
await self.cycle()
# ⚠️ The heartbeat is not an optimisation. The ESP32 treats silence
# longer than ~30 s as "the controller is gone" and zeroes the
# inverter. Refreshing the SAME value is what proves we are alive.
loop_now = asyncio.get_running_loop().time()
if loop_now - last_beat >= heartbeat:
last_beat = loop_now
await self.write_setpoint(self.target, force=True)
self.publish()
await asyncio.sleep(1)
def publish(self) -> None:
values = {
"setpoint": self.target,
"grid": self.grid,
"battery": self.batt,
"soc": self.soc,
"phase": self.maint.phase,
"status": "running" if self.auto else "stopped",
}
# ⚠️ ONLY when P1 ingestion is actually running. The ESP32's stale-input
# watchdog subscribes to sensor.p1_sample_age_s and forces the layer-1
# failsafe once it reaches max_age_s. With meter_source off there is no
# ingester feeding it, so published_age_s would be time-since-startup
# climbing without bound - i.e. every existing install would cross the
# threshold within 30 s and pin its inverter at 0 W forever. Publishing
# nothing leaves the entity non-existent, which is the status quo and
# what has_state() in the firmware is checking for.
if self.p1_enabled:
# Recomputed here, once a second, on purpose - see P1Ingest.
values["p1_age"] = round(self.p1.published_age_s, 1)
self.mqtt.publish(values)
async def shutdown(self) -> None:
"""Deterministic wind-down. Do not skip this."""
self.stopping = True
_LOG.info("shutting down - commanding 0 W")
await self.write_setpoint(0.0, force=True)
self.mqtt.close()
# -- UI ------------------------------------------------------------------
def checks(self) -> list:
o = self.o
out = []
if self.p1_enabled:
age = self.p1.published_age_s
if self.p1.stale:
out.append({"ok": False, "warn": False,
"text": f"P1 meter ({o.get('meter_source')}): no reading for "
f"{age:.0f} s (limit {self.p1.max_age_s:.0f} s)"
+ (f" - last error: {self.p1.last_error}"
if self.p1.last_error else "")})
else:
out.append({"ok": True, "warn": False,
"text": f"P1 meter ({o.get('meter_source')}): {self.p1.net_w:g} W, "
f"{age:.0f} s old, {self.p1.samples} telegrams, "
f"{self.p1.parse_errors} rejected"})
rows = [("battery SoC", self.soc, o.get("soc_entity")),
("battery power", self.batt, o.get("batt_entity"))]
if not self.p1_enabled:
# In P1 mode the check above replaces this one; leaving both in
# would report "no entity configured" for a meter_entity that is
# correctly unused, i.e. a permanent false NOT READY.
rows.insert(0, ("grid power", self.grid, o.get("meter_entity")))
for label, value, entity in rows:
if not entity:
out.append({"ok": False, "warn": False, "text": f"{label}: no entity configured"})
elif value is None:
out.append({"ok": False, "warn": False,
"text": f"{label}: {entity} is missing or not numeric"})
else:
out.append({"ok": True, "warn": False, "text": f"{label}: {entity} = {value:g}"})
sp = o.get("setpoint_entity")
if not sp:
out.append({"ok": False, "warn": False, "text": "setpoint entity not configured"})
elif self.dev_max is None:
out.append({"ok": False, "warn": False,
"text": f"setpoint entity {sp} not found"})
else:
out.append({"ok": True, "warn": False,
"text": f"setpoint {sp} (range {self.dev_min:g}{self.dev_max:g} W)"})
if self.last_write_ok is False:
out.append({"ok": False, "warn": False,
"text": "last write to the inverter was REJECTED - check the log"})
if not o.get("estop_fitted", False):
out.append({"ok": False, "warn": True,
"text": "no e-stop fitted: if this host dies the battery stays "
"latched at its last command"})
else:
out.append({"ok": True, "warn": False, "text": "e-stop fitted"})
if not o.get("peak_forecast_entity"):
# ok=False + warn=True renders as a caution, not a pass. A check that
# is both is a check nobody reads.
out.append({"ok": False, "warn": True,
"text": "no peak forecast: maintenance charging is not capacity-capped"})
return out
def status_payload(self) -> dict:
checks = self.checks()
bad = [c for c in checks if not c["ok"] and not c["warn"]]
if bad:
level, banner = "bad", f"NOT READY — {bad[0]['text']}"
elif not self.auto:
level, banner = "warn", "Stopped — inverter commanded to 0 W"
elif self.maint.phase != IDLE:
level, banner = "warn", f"Maintenance: {self.maint.phase}"
else:
level, banner = "ok", "Running — holding grid at zero"
def w(v):
return "—" if v is None else f"{v:,.0f} W".replace(",", " ")
next_due = self.maint.next_due(datetime.now(timezone.utc))
rows = [
("Grid", w(self.grid)),
("Battery", w(self.batt)),
("State of charge", "—" if self.soc is None else f"{self.soc:g} %"),
("Commanded", w(self.target)),
("Why", self.reason),
("Maintenance phase", self.maint.phase),
("Maintenance due", "now" if next_due is None else next_due.strftime("%Y-%m-%d")),
("Cheap window", "yes" if self.cheap else "no"),
]
return {
"banner": banner, "level": level, "rows": rows, "checks": checks,
"hint": self.events[0] if self.events else "",
}
async def handle_action(self, what: str):
now = datetime.now(timezone.utc)
if what == "auto_toggle":
self.auto = not self.auto
self.store.set("auto", self.auto)
self.log_event("control started" if self.auto else "control stopped")
if not self.auto:
self.target = 0.0
await self.write_setpoint(0.0, force=True)
return {"auto": self.auto}
if what == "maint_start":
for msg in self.maint.force_start(now):
self.log_event(msg)
return {"phase": self.maint.phase}
if what == "maint_abort":
for msg in self.maint.abort(now, "operator"):
self.log_event(msg)
self.target = 0.0
await self.write_setpoint(0.0, force=True)
return {"phase": self.maint.phase}
return None
async def amain() -> None:
opts = load_options()
logging.basicConfig(
level=getattr(logging, str(opts.get("log_level", "info")).upper(), logging.INFO),
format="%(asctime)s %(levelname)-7s %(name)s: %(message)s",
)
_LOG.info("GoodWe RS485 Controller starting")
from .store import Store
store = Store()
async with aiohttp.ClientSession() as session:
hass = HomeAssistant(session)
# Startup self-check: prove we can actually reach the Core API before
# anything tries to control an inverter with it. A 401 here is a
# permissions/token problem, not a configuration mistake, and saying so
# explicitly saves an installer from re-checking entity ids for an hour.
import os as _os
_tok = _os.environ.get("SUPERVISOR_TOKEN", "")
_LOG.info("supervisor token: %s (%d chars); env has: %s",
"present" if _tok else "MISSING", len(_tok),
",".join(sorted(k for k in _os.environ if "TOKEN" in k.upper())) or "none")
try:
async with session.get("http://supervisor/core/api/",
headers={"Authorization": f"Bearer {_tok}"},
timeout=10) as _r:
_LOG.info("core api probe: HTTP %s %s", _r.status, (await _r.text())[:80])
except Exception as _e: # noqa: BLE001
_LOG.error("core api probe failed: %s", _e)
# ⚠️ Status publishing must NEVER be able to stop the controller. A
# broken broker, a missing library, an API change in paho - all of it is
# observability, and the battery does not care. Caught broadly and on
# purpose: this crashed the add-on once already (paho 1.x vs 2.x) and
# took the control loop down with it.
broker = None
try:
broker = await hass.mqtt_service()
pub = MqttPublisher(
broker.get("host") if broker else None,
broker.get("port", 1883) if broker else 1883,
broker.get("username") if broker else None,
broker.get("password") if broker else None,
omit=() if is_enabled(opts) else ("p1_age",),
)
except Exception as err: # noqa: BLE001
_LOG.warning("MQTT unavailable (%s) - continuing without status entities", err)
class _NoMqtt:
enabled = False
def publish(self, *a, **k): pass
def close(self): pass
pub = _NoMqtt()
controller = Controller(opts, hass, store, pub)
runner = await web.start(controller, port=8099)
stop = asyncio.Event()
loop = asyncio.get_running_loop()
for sig in (signal.SIGTERM, signal.SIGINT):
with contextlib.suppress(NotImplementedError):
loop.add_signal_handler(sig, stop.set)
tasks = [asyncio.create_task(controller.run_control())]
# P1 ingestion runs as its own long-lived task. ⚠️ It must not be driven
# off the control loop: telegrams arrive every ~5 s and the loop would
# decimate them, so the 15-minute average - the capacity-tariff billing
# unit - would be computed from a fraction of the data.
p1_source = build_source(opts, controller.p1, session, broker)
if p1_source is not None:
tasks.append(asyncio.create_task(p1_source.run()))
await stop.wait()
await controller.shutdown()
for task in tasks:
task.cancel()
for task in tasks:
with contextlib.suppress(asyncio.CancelledError):
await task
await runner.cleanup()
_LOG.info("stopped")
def main() -> None:
try:
asyncio.run(amain())
except KeyboardInterrupt:
pass
if __name__ == "__main__":
main()