glenn schrooyenandClaude
065792aad3
Conflicts (5 files, 9 hunks) resolved keeping both sides:
- p1.py: held_t (OPT-15) beside held_since (SAFETY-90).
- modetrade.py: chargers() takes targets (P4) and probe (OPT-15); decide()
clears the chargers, a probing refuser included, while a battery drains
(SAFETY-78); the docstring lists both rules.
- main.py: the roll goes into the SAFETY-90/91 projection chain
(`_peak_proj`); the SAFETY-95 drain-tail grid_w keeps the run-edge hold on
its other branch; the P4 forecast state and the hold state both stay.
- DOCS.md, DECISIONS: both lines kept, OPT-15 item refreshed.
Adaptations the merge needs:
- SAFETY-81 x F1: a rolled projection is not graded by the saturation alarm
until the quarter's first sample, as the old edge tick was not, so a
standing overload is still notified once per quarter.
- test_safety81, test_modes_p4: modes-on pins re-pinned (with a note), modes
off pinned to e3e4ba6; test_price_run_fixes: identity pins on e3e4ba6,
F1 reproduced at one inverter (SAFETY-91 already holds two), tests for the
drain and the day plan against the probe.
Co-Authored-By: Claude <noreply@anthropic.com>
1311 lines
62 KiB
Python
1311 lines
62 KiB
Python
"""P1 meter ingestion - the only authoritative measurement of real grid exchange.
|
|
|
|
Everything downstream trusts this module: the safety checks, the capacity-tariff
|
|
peak, the optimizer, the control loop's sign. So three things happen here and
|
|
nowhere else.
|
|
|
|
1. The IMPORT/EXPORT DERIVATION. A Belgian P1 read over DSMR exposes two
|
|
UNSIGNED registers - consumption and injection. Net power is
|
|
`import_w - export_w`, positive = import, and that subtraction is done
|
|
exactly once, here (spec §5.2: "the derivation is the EMS's job, not a
|
|
template the user has to write"). A second copy of it somewhere else is a
|
|
second chance to invert the control loop.
|
|
|
|
Some P1 readers - the HomeWizard P1 among them - publish the OTHER shape:
|
|
one SIGNED figure, positive = import, and no unsigned registers at all.
|
|
`split_signed()` fans that back out into the same two magnitudes, so there
|
|
is still exactly one internal representation and one sign convention. ⚠️ It
|
|
lives here, next to the subtraction, for the same reason the subtraction
|
|
does: the moment a user is asked to write two template sensors that split a
|
|
signed value, the sign convention is back in unreviewed YAML underneath a
|
|
safety input, which is precisely what §5.2 moved into the EMS.
|
|
|
|
2. THE INGEST TIMESTAMP. Every accepted sample is stamped on arrival. A value
|
|
with no age is a value that cannot be trusted (§5.2), and staleness is the
|
|
failsafe trigger (§11.2).
|
|
|
|
3. VALIDATION. This is untrusted external data at the edge of a safety chain.
|
|
A malformed telegram must not become a plausible-looking number, and it
|
|
must never resolve to 0 W - a fabricated zero is indistinguishable from a
|
|
balanced house and defeats the very staleness trigger this module feeds.
|
|
|
|
⚠️ There is deliberately NO fallback to an inverter-side power figure. The
|
|
inverter's own AC power correlates 0.998 with battery power and 0.09 with the
|
|
real meter (§5.1) - regulating on it means regulating against your own output.
|
|
When the transport dies the correct behaviour is a gap: no sample, a growing
|
|
age, and the existing "inputs missing -> command 0 W" path in main.py.
|
|
"""
|
|
|
|
import asyncio
|
|
import json
|
|
import logging
|
|
import math
|
|
import os
|
|
import time
|
|
from dataclasses import dataclass
|
|
from datetime import datetime, timezone
|
|
|
|
import aiohttp
|
|
|
|
from . import messages
|
|
from .hass import OUTAGE
|
|
|
|
_S = messages.setting # a refusal names the setting by its label (UX-08)
|
|
|
|
try:
|
|
import paho.mqtt.client as mqtt
|
|
except ImportError: # pragma: no cover - container always has it
|
|
mqtt = None
|
|
|
|
_LOG = logging.getLogger("goodwe.p1")
|
|
|
|
SOURCE_HA = "ha_dsmr"
|
|
SOURCE_MQTT = "mqtt_p1"
|
|
SOURCE_HA_SIGNED = "ha_signed"
|
|
SOURCE_HOMEWIZARD = "homewizard_local"
|
|
|
|
QUARTER_S = 900
|
|
|
|
# ⚠️ Plausibility ceiling, not a clamp - anything above it is rejected as an
|
|
# anomaly rather than averaged in. Chosen to sit above the largest Belgian
|
|
# residential connection (3x63 A ~ 43 kW) and BELOW 65535: §20 open question 5
|
|
# records an HA sensor reporting 64954 for -582 W, i.e. an unsigned 16-bit
|
|
# register decoded without its sign. That corruption reads as a perfectly
|
|
# plausible 65 kW if you only bound it at "some big number".
|
|
PLAUSIBLE_MAX_W = 50_000.0
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# the sample
|
|
# --------------------------------------------------------------------------- #
|
|
class P1Error(ValueError):
|
|
"""A telegram that must be rejected rather than believed."""
|
|
|
|
|
|
@dataclass(frozen=True)
|
|
class P1Sample:
|
|
"""One telegram, validated, derived and stamped.
|
|
|
|
Frozen on purpose: this object is handed to readers on other tasks (and,
|
|
for the MQTT transport, produced on paho's network thread). Immutability is
|
|
what makes "read the latest sample" safe without a lock.
|
|
"""
|
|
|
|
ingest_ts: datetime # tz-aware UTC, set at ingest
|
|
ingest_mono: float # time.monotonic() at ingest - see age_s()
|
|
telegram_ts: datetime | None # from the telegram, where the source has one
|
|
source: str # one of the SOURCE_* constants above
|
|
import_w: float # unsigned magnitude, as the meter reports it
|
|
export_w: float # unsigned magnitude
|
|
net_w: float # import_w - export_w (+ import, - export)
|
|
per_phase_w: tuple[float, ...] | None # signed net, len == phases
|
|
per_phase_import_w: tuple[float, ...] | None # offtake only, for the tariff
|
|
|
|
def age_s(self, now_mono: float | None = None,
|
|
now_utc: datetime | None = None) -> float:
|
|
"""Seconds since this sample was ingested, never negative.
|
|
|
|
⚠️ Measured with time.monotonic(), not the wall clock. An NTP step on a
|
|
Pi that just booted moves the wall clock by minutes; using it here would
|
|
either fake a stale meter or, worse, hide a real one.
|
|
|
|
Where the telegram carries its own timestamp we take the WORSE of the
|
|
two ages. That is what stops an MQTT retained message - replayed on
|
|
reconnect with a fresh receive time - from presenting a ten-minute-old
|
|
reading as brand new.
|
|
"""
|
|
now_mono = time.monotonic() if now_mono is None else now_mono
|
|
age = max(0.0, now_mono - self.ingest_mono)
|
|
if self.telegram_ts is not None:
|
|
now_utc = datetime.now(timezone.utc) if now_utc is None else now_utc
|
|
age = max(age, (now_utc - self.telegram_ts).total_seconds())
|
|
return max(0.0, age)
|
|
|
|
|
|
def _watts(value, what: str) -> float:
|
|
"""Parse one power figure, or raise. Never returns a substituted default.
|
|
|
|
⚠️ Strings are refused even when float() would happily take them. A JSON
|
|
telegram carrying "1200" where a number belongs is a payload from a source
|
|
that is not the one we validated against, and the next surprise it has may
|
|
not be a benign one. Transports that legitimately deal in text (HA entity
|
|
states are always strings) convert before they get here, so this stays the
|
|
strict edge for structured payloads.
|
|
"""
|
|
if isinstance(value, (bool, str, bytes)) or value is None:
|
|
raise P1Error(f"{what}: not a number ({value!r})")
|
|
try:
|
|
out = float(value)
|
|
except (TypeError, ValueError):
|
|
raise P1Error(f"{what}: not a number ({value!r})") from None
|
|
if not math.isfinite(out):
|
|
raise P1Error(f"{what}: not finite ({value!r})")
|
|
if abs(out) > PLAUSIBLE_MAX_W:
|
|
raise P1Error(f"{what}: {out:g} W is outside plausible meter range")
|
|
return out
|
|
|
|
|
|
def split_signed(net_w) -> tuple[float, float]:
|
|
"""One signed figure -> the (import, export) magnitudes the module speaks.
|
|
|
|
The inverse of make_sample's subtraction, and the easy direction: no second
|
|
register to disagree with, so there is nothing to mix a fresh reading with a
|
|
stale one. `+` is import, `-` is export - verified in test_p1.py against real
|
|
captured readings from this house's own meter, not against a datasheet.
|
|
|
|
⚠️ Exactly one of the two comes out non-zero. Splitting into `(max(v,0),
|
|
max(-v,0))` rather than clamping keeps `import_w - export_w == v` exactly, so
|
|
the signed value the meter published survives the round trip bit for bit -
|
|
a control loop must not be steered by a number that changed on the way in.
|
|
|
|
⚠️ Validation is `_watts`, the same gate the unsigned path uses: NaN,
|
|
infinity, non-numbers and the §20 open-question-5 unsigned-decode
|
|
contamination (64954 for -582 W) are all refused here rather than believed.
|
|
A signed source makes that check MORE important, not less - on this path
|
|
64954 is not obviously wrong the way a negative "unsigned" register is.
|
|
"""
|
|
v = _watts(net_w, "net")
|
|
return (v, 0.0) if v >= 0 else (0.0, -v)
|
|
|
|
|
|
def make_sample(source: str, import_w, export_w, *, phases: int,
|
|
phase_import_w=None, phase_export_w=None,
|
|
telegram_ts: datetime | None = None,
|
|
ingest_ts: datetime | None = None,
|
|
ingest_mono: float | None = None) -> P1Sample:
|
|
"""Validate, derive net power, stamp. Raises P1Error on anything doubtful.
|
|
|
|
`import_w`/`export_w` are the two unsigned Belgian registers. Per-phase
|
|
figures are equally unsigned and equally split, so each phase gets the same
|
|
derivation.
|
|
"""
|
|
imp = _watts(import_w, "import")
|
|
exp = _watts(export_w, "export")
|
|
# ⚠️ Both registers are magnitudes. A negative one means the upstream
|
|
# already applied a sign we are about to apply again - reject it rather
|
|
# than silently double-signing the control loop.
|
|
if imp < 0 or exp < 0:
|
|
raise P1Error(f"import and export readings cannot be negative "
|
|
f"(import={imp:g} export={exp:g})")
|
|
|
|
per_phase = per_phase_import = None
|
|
if phase_import_w is not None or phase_export_w is not None:
|
|
pi = list(phase_import_w or [])
|
|
pe = list(phase_export_w or [0.0] * len(pi))
|
|
if len(pi) != phases or len(pe) != phases:
|
|
raise P1Error(
|
|
f"the meter sent {len(pi)} import and {len(pe)} export phases, "
|
|
f"but {_S('meter_phases')} is {phases}")
|
|
vals = [_watts(a, f"L{i + 1} import") - _watts(b, f"L{i + 1} export")
|
|
for i, (a, b) in enumerate(zip(pi, pe))]
|
|
per_phase = tuple(vals)
|
|
per_phase_import = tuple(max(v, 0.0) for v in vals)
|
|
|
|
if telegram_ts is not None and telegram_ts.tzinfo is None:
|
|
raise P1Error("telegram timestamp has no timezone")
|
|
|
|
return P1Sample(
|
|
ingest_ts=ingest_ts or datetime.now(timezone.utc),
|
|
ingest_mono=time.monotonic() if ingest_mono is None else ingest_mono,
|
|
telegram_ts=telegram_ts,
|
|
source=source,
|
|
import_w=imp,
|
|
export_w=exp,
|
|
net_w=imp - exp,
|
|
per_phase_w=per_phase,
|
|
per_phase_import_w=per_phase_import,
|
|
)
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# the 15-minute average
|
|
# --------------------------------------------------------------------------- #
|
|
@dataclass(frozen=True)
|
|
class QuarterBlock:
|
|
start: datetime # UTC, aligned to :00/:15/:30/:45
|
|
offtake_avg_w: float # billed figure: net offtake only
|
|
per_phase_offtake_avg_w: tuple[float, ...] | None
|
|
|
|
|
|
class QuarterAverager:
|
|
"""Time-weighted average of net offtake over clock-aligned 15-min blocks.
|
|
|
|
Samples arrive irregularly (~1-10 s), so a plain mean over samples would
|
|
weight a burst of fast telegrams the same as a slow one and produce a figure
|
|
that is not the billed quantity. Each sample's value is therefore HELD until
|
|
the next arrives and integrated over that interval: sum(value * dt) / dt.
|
|
|
|
⚠️ Only OFFTAKE is accumulated (§9.1) - the capacity tariff bills the highest
|
|
quarter-hour average offtake, and a quarter of pure export averages to 0 kW,
|
|
not to a negative one. The signed series stays available for control; this
|
|
accumulator is for the meter's bill.
|
|
|
|
⚠️ Blocks are found by flooring epoch seconds to 900. That IS clock-aligned
|
|
and DST-proof for Belgium, because every offset in that tz is a whole number
|
|
of hours, so a 900 s grid in UTC lands on :00/:15/:30/:45 local before and
|
|
after a transition - no tz database, no DST special case.
|
|
ponytail: the ceiling is a timezone with a sub-hour offset (India +05:30,
|
|
Nepal, Chatham). Those need real tz-aware boundary maths; upgrade path is to
|
|
compute the boundary with zoneinfo instead of the modulo, everything else
|
|
here is unchanged.
|
|
|
|
A sample that straddles a boundary is split at the boundary and its two
|
|
halves credited to the two blocks, never attributed wholly to either.
|
|
"""
|
|
|
|
def __init__(self, phases: int = 1, max_hold_s: float = 30.0):
|
|
self.phases = phases
|
|
# ⚠️ How long one sample may be held forward before the series is
|
|
# treated as a gap rather than a plateau. Without this the meter can die
|
|
# while importing 5 kW, come back ten minutes later, and the hold-forward
|
|
# credits 5 kW x 600 s to the capacity-tariff accumulator - a fabricated
|
|
# peak, on a permanent record, from data that was never measured. Set
|
|
# from meter_max_age_s: the point past which the reading is not trusted
|
|
# for control is the point past which it must not be billed either.
|
|
self.max_hold_s = float(max_hold_s)
|
|
self._block: int | None = None # epoch seconds of the block start
|
|
self._acc = 0.0 # W*s of offtake in the open block
|
|
self._pp_acc = [0.0] * phases
|
|
self._elapsed = 0.0 # seconds integrated in the open block
|
|
self._last_t: float | None = None # epoch seconds of the held sample
|
|
self._last_net = 0.0
|
|
self._last_pp: tuple[float, ...] | None = None
|
|
|
|
# -- reading ------------------------------------------------------------
|
|
@property
|
|
def block_start(self) -> datetime | None:
|
|
if self._block is None:
|
|
return None
|
|
return datetime.fromtimestamp(self._block, timezone.utc)
|
|
|
|
@property
|
|
def elapsed_s(self) -> float:
|
|
"""Seconds already integrated into the open block.
|
|
|
|
Exposed alongside the partial accumulator because SAFETY-07 projects the
|
|
end-of-quarter average and cannot do that from a finished average.
|
|
"""
|
|
return self._elapsed
|
|
|
|
@property
|
|
def partial_ws(self) -> float:
|
|
"""Offtake watt-seconds accumulated in the open block so far."""
|
|
return self._acc
|
|
|
|
@property
|
|
def held_net_w(self) -> float:
|
|
"""The signed net power currently being held forward, in W.
|
|
|
|
PUBLIC BECAUSE SAFETY-07 DEPENDS ON IT, AND THAT IS THE WHOLE REASON.
|
|
`PeakTracker.project()` extrapolates the rest of the quarter from this
|
|
value and used to reach it as `getattr(averager, "_last_net", 0.0)`. A
|
|
default on a private name fails QUIET and fails in the DISARMING
|
|
direction: rename `_last_net` and every projection silently credits 0 W
|
|
held, which reads LOW, which puts the peak guard under the ceiling
|
|
exactly when the meter says it is over. Filed as TEST-13 item 4 and
|
|
fixed here rather than inherited - a money guard may not rest on a read
|
|
that can go wrong without a symptom.
|
|
|
|
Positive is offtake, matching `P1Sample.net_w`. It is the raw held
|
|
value, NOT floored at zero: the flooring is the accumulator's rule (only
|
|
offtake is billed) and belongs where it is applied, not here.
|
|
"""
|
|
return self._last_net
|
|
|
|
@property
|
|
def held_t(self) -> float | None:
|
|
"""Epoch seconds of the held sample. OPT-15: `PeakTracker.project()`
|
|
with `roll=True` credits it into the next block as `add()` will."""
|
|
return self._last_t
|
|
|
|
@property
|
|
def held_since(self) -> datetime | None:
|
|
"""SAFETY-90: when the held sample arrived (its `ingest_ts`), or None."""
|
|
if self._last_t is None:
|
|
return None
|
|
return datetime.fromtimestamp(self._last_t, timezone.utc)
|
|
|
|
@property
|
|
def offtake_avg_w(self) -> float:
|
|
"""Average offtake over the part of the open block seen so far."""
|
|
return self._acc / self._elapsed if self._elapsed > 0 else 0.0
|
|
|
|
@property
|
|
def per_phase_offtake_avg_w(self) -> tuple[float, ...] | None:
|
|
if self._last_pp is None or self._elapsed <= 0:
|
|
return None
|
|
return tuple(a / self._elapsed for a in self._pp_acc)
|
|
|
|
# -- writing ------------------------------------------------------------
|
|
def add(self, sample: P1Sample) -> list[QuarterBlock]:
|
|
"""Integrate up to this sample, then hold its value. Returns any blocks
|
|
that closed in the process (usually none, occasionally one)."""
|
|
t = sample.ingest_ts.timestamp()
|
|
closed: list[QuarterBlock] = []
|
|
|
|
if self._last_t is None:
|
|
self._block = int(t // QUARTER_S) * QUARTER_S
|
|
self._last_t, self._last_net = t, sample.net_w
|
|
self._last_pp = sample.per_phase_w
|
|
return closed
|
|
if t <= self._last_t:
|
|
# Out-of-order or duplicate arrival: integrating a negative dt would
|
|
# subtract energy that really happened. Drop it, keep the held value.
|
|
return closed
|
|
|
|
cursor = self._last_t
|
|
# Beyond this instant the held value stops being evidence of anything.
|
|
# The stretch from here to `t` is walked so the block boundaries are
|
|
# still crossed correctly, but nothing is accumulated and `_elapsed`
|
|
# does not grow - which is what makes a closed block, always divided by
|
|
# the full 900 s, actually get dragged down by the missing coverage.
|
|
hold_end = self._last_t + self.max_hold_s
|
|
while True:
|
|
end = self._block + QUARTER_S
|
|
stop = min(t, end)
|
|
covered = max(0.0, min(stop, hold_end) - cursor)
|
|
if covered > 0:
|
|
self._acc += max(self._last_net, 0.0) * covered
|
|
if self._last_pp is not None:
|
|
for i, v in enumerate(self._last_pp[: self.phases]):
|
|
self._pp_acc[i] += max(v, 0.0) * covered
|
|
self._elapsed += covered
|
|
cursor = stop
|
|
if stop < end:
|
|
break
|
|
closed.append(QuarterBlock(
|
|
start=datetime.fromtimestamp(self._block, timezone.utc),
|
|
# A closed block is always divided by the full 900 s, never by
|
|
# the seconds we happened to observe - a gap in coverage must
|
|
# drag the billed average down, not be averaged away.
|
|
offtake_avg_w=self._acc / QUARTER_S,
|
|
per_phase_offtake_avg_w=(
|
|
tuple(a / QUARTER_S for a in self._pp_acc)
|
|
if self._last_pp is not None else None),
|
|
))
|
|
self._block = end
|
|
self._acc = 0.0
|
|
self._pp_acc = [0.0] * self.phases
|
|
self._elapsed = 0.0
|
|
|
|
self._last_t, self._last_net = t, sample.net_w
|
|
self._last_pp = sample.per_phase_w
|
|
return closed
|
|
|
|
# -- surviving a restart (TARIFF-01) ------------------------------------
|
|
def snapshot(self) -> dict:
|
|
"""The open block's whole state, as JSON-safe primitives.
|
|
|
|
⚠️ `last_t` is the HELD SAMPLE'S OWN timestamp and is stored as it is,
|
|
not refreshed to "now". That is what makes a restart integrate the gap
|
|
under exactly the `max_hold_s` rule a running process would have used:
|
|
a short restart is a plateau, a long one leaves uncovered seconds that
|
|
drag the closed block's average down. Rewriting it to the restore time
|
|
would silently credit the whole outage at the last value seen before it.
|
|
"""
|
|
return {
|
|
"block": self._block,
|
|
"acc": self._acc,
|
|
"pp_acc": list(self._pp_acc),
|
|
"elapsed": self._elapsed,
|
|
"last_t": self._last_t,
|
|
"last_net": self._last_net,
|
|
"last_pp": list(self._last_pp) if self._last_pp is not None else None,
|
|
"phases": self.phases,
|
|
}
|
|
|
|
def restore(self, state: dict) -> bool:
|
|
"""Put a `snapshot()` back. False if it does not fit this averager.
|
|
|
|
A phase-count change is a reconfiguration, not a resumption: the
|
|
per-phase accumulator would be reshaped mid-block and the two halves
|
|
would not be the same measurement. Refusing is the honest answer, and
|
|
the caller reports starting fresh rather than pretending it resumed.
|
|
"""
|
|
try:
|
|
if int(state.get("phases", self.phases)) != self.phases:
|
|
return False
|
|
pp_acc = list(state.get("pp_acc") or [])
|
|
if len(pp_acc) != self.phases:
|
|
return False
|
|
self._block = None if state["block"] is None else int(state["block"])
|
|
self._acc = float(state["acc"])
|
|
self._pp_acc = [float(v) for v in pp_acc]
|
|
self._elapsed = float(state["elapsed"])
|
|
self._last_t = None if state["last_t"] is None else float(state["last_t"])
|
|
self._last_net = float(state["last_net"])
|
|
lp = state.get("last_pp")
|
|
self._last_pp = tuple(float(v) for v in lp) if lp is not None else None
|
|
return True
|
|
except (KeyError, TypeError, ValueError):
|
|
return False
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# what the rest of the add-on talks to
|
|
# --------------------------------------------------------------------------- #
|
|
class P1Ingest:
|
|
"""Holds the latest sample and the rolling quarter-hour average.
|
|
|
|
⚠️ Staleness is DERIVED from the stored sample, not carried as a separate
|
|
flag. That is what makes "flag before the value is visible" free: there is
|
|
one immutable object and a single attribute rebind to publish it, so a
|
|
reader can never see a fresh value with a stale flag or the reverse.
|
|
"""
|
|
|
|
def __init__(self, phases: int = 1, max_age_s: float = 30.0):
|
|
self.phases = phases
|
|
self.max_age_s = float(max_age_s)
|
|
# The same threshold governs control and billing: a reading too old to
|
|
# steer by is too old to bill by. See QuarterAverager.max_hold_s.
|
|
self.averager = QuarterAverager(phases, max_hold_s=self.max_age_s)
|
|
self.blocks: list[QuarterBlock] = []
|
|
self.samples = 0
|
|
self.parse_errors = 0
|
|
self.last_error: str | None = None
|
|
self.started_mono = time.monotonic()
|
|
self._last: P1Sample | None = None
|
|
|
|
@property
|
|
def last(self) -> P1Sample | None:
|
|
return self._last
|
|
|
|
def submit(self, sample: P1Sample) -> None:
|
|
self._last = sample
|
|
self.samples += 1
|
|
for block in self.averager.add(sample):
|
|
self.blocks.append(block)
|
|
del self.blocks[:-96] # a day of quarters; STATE-01 owns real retention
|
|
|
|
def reject(self, err: Exception | str) -> None:
|
|
"""A malformed telegram or an unavailable entity.
|
|
|
|
⚠️ The last good sample and ITS timestamp are left untouched. The reading
|
|
does not become 0 W and it does not become fresh - the age keeps growing,
|
|
which is precisely the signal a rejected telegram should produce.
|
|
"""
|
|
self.parse_errors += 1
|
|
self.last_error = str(err)
|
|
_LOG.warning("A meter message could not be read: %s", err)
|
|
|
|
# -- what consumers read -------------------------------------------------
|
|
def age_s(self) -> float | None:
|
|
"""Age of the newest accepted sample, or None if there has never been one."""
|
|
return None if self._last is None else self._last.age_s()
|
|
|
|
@property
|
|
def published_age_s(self) -> float:
|
|
"""The figure behind `sensor.p1_sample_age_s`.
|
|
|
|
Seconds since the newest accepted telegram, or since this ingester
|
|
started when none has ever arrived.
|
|
|
|
⚠️ Always a number and never `unknown`, because SAFETY-01's firmware
|
|
watchdog subscribes to it: an entity that simply stops existing is
|
|
indistinguishable, from the firmware's side, from a meter that is fine.
|
|
And ⚠️ it is recomputed against the clock on every publish rather than
|
|
stamped once per telegram, so a meter that freezes at a constant reading
|
|
still produces a visibly climbing age. That is the whole point of this
|
|
entity - HA pushes state changes, so a genuinely constant P1 value emits
|
|
nothing at all, and a watchdog watching the value would sit there
|
|
believing the last update was recent.
|
|
"""
|
|
age = self.age_s()
|
|
return max(0.0, time.monotonic() - self.started_mono) if age is None else age
|
|
|
|
@property
|
|
def stale(self) -> bool:
|
|
"""True when there is no sample, or the newest one is past max_age_s."""
|
|
age = self.age_s()
|
|
return age is None or age > self.max_age_s
|
|
|
|
@property
|
|
def net_w(self) -> float | None:
|
|
"""Signed net grid power, or None when stale. Never a substituted zero."""
|
|
return None if self.stale else self._last.net_w
|
|
|
|
@property
|
|
def per_phase_import_w(self) -> tuple[float, ...] | None:
|
|
if self.stale or self._last is None:
|
|
return None
|
|
return self._last.per_phase_import_w
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# transport 1: Home Assistant WebSocket (the DSMR integration's entities)
|
|
# --------------------------------------------------------------------------- #
|
|
WS_URL = "ws://supervisor/core/websocket"
|
|
BAD_STATES = ("unknown", "unavailable", "none", "")
|
|
|
|
|
|
async def _receive_json(ws):
|
|
"""The next JSON message. A socket Home Assistant closed (a restart) is a
|
|
ConnectionResetError, not aiohttp's "Received message 8:1000 is not str"."""
|
|
msg = await ws.receive()
|
|
if msg.type is not aiohttp.WSMsgType.TEXT:
|
|
raise ConnectionResetError(
|
|
f"Home Assistant closed the connection ({msg.type.name})")
|
|
return json.loads(msg.data)
|
|
|
|
|
|
class HaDsmrSource:
|
|
"""Subscribes to state_changed for the configured DSMR entities.
|
|
|
|
⚠️ WebSocket, not REST polling. REST returns states, but polling at the
|
|
30 s planning tick decimates a 5 s telegram stream and the quarter-hour
|
|
average would then be computed from a sixth of the data (§5.3). "Consume
|
|
every telegram" means event-driven.
|
|
|
|
⚠️ One telegram updates several entities, and HA emits one state_changed per
|
|
entity. Building a sample on each event would mix a new import reading with
|
|
a stale export one for a few milliseconds every 5 s. A short debounce
|
|
coalesces the burst back into the single telegram it came from.
|
|
"""
|
|
|
|
DEBOUNCE_S = 0.35
|
|
|
|
def __init__(self, session: aiohttp.ClientSession, ingest: P1Ingest,
|
|
entities: dict, token: str | None = None):
|
|
self.session = session
|
|
self.ingest = ingest
|
|
self.entities = entities # {"import": id, "export": id, "phase_import": [...], ...}
|
|
self.token = token or os.environ.get("SUPERVISOR_TOKEN", "")
|
|
self.ids = self._wanted()
|
|
self.cache: dict[str, float] = {}
|
|
self.connected = False
|
|
self._pending: asyncio.Task | None = None
|
|
|
|
def _wanted(self) -> set[str]:
|
|
out = set()
|
|
for key in ("import", "export"):
|
|
if self.entities.get(key):
|
|
out.add(self.entities[key])
|
|
for key in ("phase_import", "phase_export"):
|
|
out.update(e for e in self.entities.get(key) or [] if e)
|
|
return out
|
|
|
|
async def run(self) -> None:
|
|
"""Long-lived task: connect, subscribe, reconnect with backoff, forever.
|
|
|
|
⚠️ A reconnect emits nothing. A gap must stay a gap - a synthetic sample
|
|
on reconnect would reset the age and hide the outage from the very
|
|
watchdog that exists to catch it.
|
|
"""
|
|
backoff = 1.0
|
|
while True:
|
|
try:
|
|
await self._session_once()
|
|
backoff = 1.0
|
|
except asyncio.CancelledError:
|
|
raise
|
|
except (aiohttp.ClientError, ConnectionError) as err:
|
|
# UX-10: Home Assistant restarting, said once (hass.Outage).
|
|
OUTAGE.down("websocket", "the meter connection closed", err)
|
|
except Exception as err: # noqa: BLE001 - any transport fault retries
|
|
_LOG.warning("The meter connection to Home Assistant failed "
|
|
"(%s), trying again in %.0f s.", err, backoff)
|
|
finally:
|
|
self.connected = False
|
|
await asyncio.sleep(backoff)
|
|
backoff = min(backoff * 2, 30.0)
|
|
|
|
async def _session_once(self) -> None:
|
|
async with self.session.ws_connect(WS_URL, heartbeat=30) as ws:
|
|
hello = await _receive_json(ws)
|
|
if hello.get("type") == "auth_required":
|
|
await ws.send_json({"type": "auth", "access_token": self.token})
|
|
reply = await _receive_json(ws)
|
|
if reply.get("type") != "auth_ok":
|
|
raise RuntimeError(f"auth rejected: {reply.get('message', reply)}")
|
|
await ws.send_json({"id": 1, "type": "subscribe_events",
|
|
"event_type": "state_changed"})
|
|
await ws.send_json({"id": 2, "type": "get_states"})
|
|
self.connected = True
|
|
OUTAGE.up("websocket")
|
|
_LOG.info("P1 ingest: subscribed to %s", ", ".join(sorted(self.ids)))
|
|
|
|
async for msg in ws:
|
|
if msg.type is not aiohttp.WSMsgType.TEXT:
|
|
continue
|
|
payload = json.loads(msg.data)
|
|
if payload.get("id") == 2 and payload.get("type") == "result":
|
|
# ⚠️ Prime the cache, but do NOT build a sample from it.
|
|
# get_states returns whatever HA currently holds, which
|
|
# after a Core restart is a RestoreEntity value of unknown
|
|
# age. Stamping that with ingest_ts=now resets the age to
|
|
# zero and reports a fresh meter that may have been dead for
|
|
# an hour - a synthetic sample hiding the outage from the
|
|
# watchdog that exists to catch it. The cache is what lets
|
|
# the FIRST real state_changed build a complete sample; the
|
|
# age stays honest until one arrives.
|
|
for obj in payload.get("result") or []:
|
|
self._absorb(obj.get("entity_id"), obj.get("state"))
|
|
elif payload.get("type") == "event":
|
|
data = (payload.get("event") or {}).get("data") or {}
|
|
if data.get("entity_id") not in self.ids:
|
|
continue
|
|
new = data.get("new_state") or {}
|
|
self._absorb(data.get("entity_id"), new.get("state"))
|
|
self._schedule()
|
|
raise ConnectionResetError("Home Assistant closed the connection")
|
|
|
|
def _absorb(self, entity_id: str | None, state) -> None:
|
|
if not entity_id or entity_id not in self.ids:
|
|
return
|
|
raw = str(state).strip().lower()
|
|
if raw in BAD_STATES:
|
|
# ⚠️ An `unavailable` DSMR entity is a missing reading, not 0 W.
|
|
# Forget the cached value so no sample can be built from a mixture
|
|
# of a live register and one that stopped reporting.
|
|
self.cache.pop(entity_id, None)
|
|
self.ingest.reject(f"{entity_id} is {raw}")
|
|
return
|
|
try:
|
|
self.cache[entity_id] = float(raw)
|
|
except ValueError:
|
|
self.cache.pop(entity_id, None)
|
|
self.ingest.reject(f"{entity_id} is not numeric: {raw!r}")
|
|
|
|
def _schedule(self) -> None:
|
|
if self._pending and not self._pending.done():
|
|
return
|
|
self._pending = asyncio.get_running_loop().create_task(self._after_debounce())
|
|
|
|
async def _after_debounce(self) -> None:
|
|
await asyncio.sleep(self.DEBOUNCE_S)
|
|
self.build()
|
|
|
|
def build(self) -> bool:
|
|
"""Assemble one sample from the cache. Returns True if one was accepted."""
|
|
imp_id, exp_id = self.entities.get("import"), self.entities.get("export")
|
|
if imp_id not in self.cache or exp_id not in self.cache:
|
|
return False
|
|
pi = [self.cache.get(e) for e in self.entities.get("phase_import") or []]
|
|
pe = [self.cache.get(e) for e in self.entities.get("phase_export") or []]
|
|
if pi and (None in pi or (pe and None in pe)):
|
|
return False # incomplete phase set: wait, do not guess
|
|
try:
|
|
self.ingest.submit(make_sample(
|
|
SOURCE_HA, self.cache[imp_id], self.cache[exp_id],
|
|
phases=self.ingest.phases,
|
|
phase_import_w=pi or None,
|
|
phase_export_w=pe or None,
|
|
# ⚠️ No telegram_ts: HA's last_changed is when the STATE changed,
|
|
# which for a constant reading is minutes ago even though the
|
|
# telegram is current. Using it as a telegram time would fake
|
|
# staleness on a genuinely steady meter.
|
|
))
|
|
return True
|
|
except P1Error as err:
|
|
self.ingest.reject(err)
|
|
return False
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# transport 2: MQTT
|
|
# --------------------------------------------------------------------------- #
|
|
def parse_mqtt_payload(raw: bytes | str, phases: int, *,
|
|
now: datetime | None = None) -> P1Sample:
|
|
"""One JSON telegram from the configured topic. Raises P1Error.
|
|
|
|
The accepted document, documented in DOCS.md:
|
|
|
|
{"import_w": 1234.0, "export_w": 0.0,
|
|
"phases": [{"import_w": 500, "export_w": 0}, ...], # optional
|
|
"timestamp": "2026-08-24T18:00:05+02:00"} # optional
|
|
|
|
ponytail: one strict schema rather than sniffing the half-dozen P1-bridge
|
|
dialects in the wild. The upgrade path is a `meter_mqtt_format` option
|
|
selecting a parser; a lenient parser is the wrong default at a safety
|
|
boundary, where guessing a key means guessing a kilowatt.
|
|
"""
|
|
try:
|
|
doc = json.loads(raw)
|
|
except (ValueError, TypeError) as err:
|
|
raise P1Error(f"payload is not JSON: {err}") from None
|
|
if not isinstance(doc, dict):
|
|
raise P1Error(f"payload is not a JSON object ({type(doc).__name__})")
|
|
|
|
ts = None
|
|
if doc.get("timestamp"):
|
|
try:
|
|
ts = datetime.fromisoformat(str(doc["timestamp"]))
|
|
except ValueError:
|
|
raise P1Error(f"unparseable timestamp {doc['timestamp']!r}") from None
|
|
if ts.tzinfo is None:
|
|
raise P1Error("timestamp has no UTC offset")
|
|
|
|
pi = pe = None
|
|
if "phases" in doc:
|
|
rows = doc["phases"]
|
|
if not isinstance(rows, list) or not all(isinstance(r, dict) for r in rows):
|
|
raise P1Error("'phases' must be a list of objects")
|
|
pi = [r.get("import_w") for r in rows]
|
|
pe = [r.get("export_w", 0.0) for r in rows]
|
|
|
|
return make_sample(SOURCE_MQTT, doc.get("import_w"), doc.get("export_w"),
|
|
phases=phases, phase_import_w=pi, phase_export_w=pe,
|
|
telegram_ts=ts, ingest_ts=now)
|
|
|
|
|
|
class MqttP1Source:
|
|
"""Subscribes to one topic and submits every message that parses.
|
|
|
|
Uses paho's own reconnect loop on its own thread, then hops back onto the
|
|
event loop with call_soon_threadsafe so the ingest state is only ever
|
|
mutated from one thread.
|
|
"""
|
|
|
|
def __init__(self, ingest: P1Ingest, topic: str, host, port=1883,
|
|
username=None, password=None):
|
|
self.ingest = ingest
|
|
self.topic = topic
|
|
self.host, self.port = host, int(port or 1883)
|
|
self.username, self.password = username, password
|
|
self.client = None
|
|
self.loop = None
|
|
|
|
async def run(self) -> None:
|
|
if mqtt is None or not self.host or not self.topic:
|
|
_LOG.error("P1 MQTT source not usable (broker=%s topic=%r) - no meter data",
|
|
self.host, self.topic)
|
|
return
|
|
self.loop = asyncio.get_running_loop()
|
|
try:
|
|
self.client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2,
|
|
client_id="goodwe_p1_ingest")
|
|
except AttributeError: # paho 1.x, which is what Alpine ships
|
|
self.client = mqtt.Client(client_id="goodwe_p1_ingest")
|
|
if self.username:
|
|
self.client.username_pw_set(self.username, self.password or "")
|
|
self.client.on_connect = lambda *_a, **_k: self.client.subscribe(self.topic, qos=0)
|
|
self.client.on_message = self._on_message
|
|
self.client.reconnect_delay_set(min_delay=1, max_delay=30)
|
|
self.client.connect_async(self.host, self.port, keepalive=60)
|
|
self.client.loop_start()
|
|
_LOG.info("P1 ingest: MQTT %s:%s topic %s", self.host, self.port, self.topic)
|
|
try:
|
|
while True:
|
|
await asyncio.sleep(3600)
|
|
finally:
|
|
self.client.loop_stop()
|
|
self.client.disconnect()
|
|
|
|
def _on_message(self, _client, _userdata, msg) -> None:
|
|
# Runs on paho's network thread.
|
|
if self.loop is None:
|
|
return
|
|
self.loop.call_soon_threadsafe(self._handle, msg.payload)
|
|
|
|
def _handle(self, payload) -> None:
|
|
try:
|
|
self.ingest.submit(parse_mqtt_payload(payload, self.ingest.phases))
|
|
except P1Error as err:
|
|
self.ingest.reject(err)
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# transport 3: Home Assistant WebSocket, one signed entity
|
|
# --------------------------------------------------------------------------- #
|
|
class HaSignedSource(HaDsmrSource):
|
|
"""The same websocket, subscribed to ONE signed power entity.
|
|
|
|
For readers that publish net power as a single signed figure - a HomeWizard
|
|
P1's `sensor.p1_meter_active_power`, positive = import - rather than the two
|
|
unsigned DSMR registers. This is the meter actually installed at the house,
|
|
and `ha_dsmr` cannot read it: it needs two registers and refuses a negative
|
|
one outright, which is every exporting telegram.
|
|
|
|
⚠️ A subclass, not a copy. The connect / auth / subscribe / reconnect /
|
|
`_absorb` machinery above is transport, not shape, and it has already been
|
|
debugged once - notably "prime the cache from get_states but never build a
|
|
sample out of it" and "a reconnect emits nothing". Only `_wanted` (which
|
|
entity ids) and `build` (how they become a sample) differ, so only those two
|
|
are overridden. Everything TEL-01 established therefore applies unchanged:
|
|
ingest timestamping, meter_max_age_s, the clock-recomputed age sensor, and
|
|
`unavailable` treated as a missing reading rather than 0 W.
|
|
|
|
⚠️ THE AGE ON THIS TRANSPORT MEASURES TIME SINCE THE VALUE CHANGED, not time
|
|
since the meter reported, and on one entity those are very different things.
|
|
Home Assistant offers no arrival signal for a repeated reading: it emits no
|
|
`state_changed`, it does not advance `last_reported` on either serialiser,
|
|
and `state_reported` cannot be subscribed to over the websocket at all
|
|
("Event filter is required for event state_reported"). All three measured on
|
|
the ENV-01 rig against the real HomeWizard integration with the meter frozen
|
|
- 0 state_changed in 70 s, no timestamp movement anywhere.
|
|
|
|
`ha_dsmr` mostly escapes it because a DSMR telegram moves several entities at
|
|
once. This transport has ONE, so a healthy meter under a flat load looks
|
|
exactly like a dead one - and our own capture has the real meter going 42.2 s
|
|
and 97.0 s between changes, both past the default max_age_s of 30. Hence
|
|
DOCS.md: sensor.p1_sample_age_s is correct while the value moves and must not
|
|
yet be thresholded by the firmware watchdog on this transport. Raising
|
|
meter_max_age_s does not fix it, it only chooses which of the two errors you
|
|
get. The fix is an arrival stamp from the meter itself - reading the
|
|
HomeWizard local API rather than an HA entity - which is a separate ticket.
|
|
|
|
⚠️ The debounce is inherited but does nothing useful here, and that is fine:
|
|
one telegram is one entity, so there is no burst of per-entity events to
|
|
coalesce and no window in which a new reading sits beside a stale one. It
|
|
costs one scheduled sleep per telegram at ~0.2 Hz. Left in place rather than
|
|
special-cased, because a second code path through build() is a second place
|
|
for the sign to go wrong.
|
|
"""
|
|
|
|
def _wanted(self) -> set[str]:
|
|
out = set()
|
|
if self.entities.get("net"):
|
|
out.add(self.entities["net"])
|
|
out.update(e for e in self.entities.get("phase_net") or [] if e)
|
|
return out
|
|
|
|
def build(self) -> bool:
|
|
"""Assemble one sample from the cache. Returns True if one was accepted."""
|
|
net_id = self.entities.get("net")
|
|
if net_id not in self.cache:
|
|
return False
|
|
pn = [self.cache.get(e) for e in self.entities.get("phase_net") or []]
|
|
if pn and None in pn:
|
|
return False # incomplete phase set: wait, do not guess
|
|
try:
|
|
imp, exp = split_signed(self.cache[net_id])
|
|
pi = pe = None
|
|
if pn:
|
|
# ponytail: this split is arithmetically redundant today -
|
|
# make_sample subtracts the two lists again and does not
|
|
# sign-check per-phase figures, so handing it the signed values
|
|
# with a zero export list produces the identical tuple. Verified:
|
|
# mutating it that way leaves all 174 checks green, i.e. no test
|
|
# can tell the difference, and it is recorded here rather than
|
|
# left as a silent equivalent mutant for the next reviewer to
|
|
# rediscover. Kept because `phase_import_w` means a MAGNITUDE:
|
|
# a negative in it is the double-signing that make_sample refuses
|
|
# outright for the connection-level registers, and the day that
|
|
# check is extended per-phase the shortcut breaks the meter, not
|
|
# the test.
|
|
pairs = [split_signed(v) for v in pn]
|
|
pi = [a for a, _ in pairs]
|
|
pe = [b for _, b in pairs]
|
|
self.ingest.submit(make_sample(
|
|
SOURCE_HA_SIGNED, imp, exp,
|
|
phases=self.ingest.phases,
|
|
phase_import_w=pi, phase_export_w=pe,
|
|
# ⚠️ No telegram_ts, for the same reason as ha_dsmr: HA's
|
|
# last_changed is when the VALUE changed, which on a steady meter
|
|
# is minutes ago while the telegram is current. sensor.
|
|
# p1_sample_age_s is what covers a genuinely frozen meter.
|
|
))
|
|
return True
|
|
except P1Error as err:
|
|
self.ingest.reject(err)
|
|
return False
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# transport 4: the HomeWizard P1's own local API
|
|
# --------------------------------------------------------------------------- #
|
|
HOMEWIZARD_PATH = "/api/v1/data"
|
|
HOMEWIZARD_PORT = 80
|
|
|
|
|
|
def parse_homewizard(doc, phases: int, *, now: datetime | None = None,
|
|
ingest_mono: float | None = None) -> P1Sample:
|
|
"""One `GET /api/v1/data` document -> a sample. Raises P1Error.
|
|
|
|
The meter's own JSON, of which we read four keys:
|
|
|
|
{"active_power_w": -5710.0, # SIGNED, + import - export
|
|
"active_power_l1_w": ..., "active_power_l2_w": ..., "..._l3_w": ...,
|
|
"total_power_import_kwh": 1234.567, "total_power_export_kwh": 890.123}
|
|
|
|
Same shape as `ha_signed` - one signed connection figure plus signed
|
|
per-phase figures - so it goes through the same `split_signed` and the same
|
|
`make_sample`. Nothing about the sign convention is re-derived here.
|
|
|
|
⚠️ No telegram timestamp, because the API carries none, and that is correct
|
|
rather than a gap: the document was produced by the meter in the moment it
|
|
answered, so the ingest stamp IS the measurement time. This is the whole
|
|
reason the transport exists - see HomeWizardLocalSource.
|
|
|
|
⚠️ Per-phase figures are taken only when the meter serves ALL `phases` of
|
|
them. A single-phase HomeWizard returns `active_power_l2_w: null`, and a
|
|
partial set must not become a fabricated tuple. Unlike the HA transports
|
|
there is no "wait for the rest" case to worry about: every field here came
|
|
out of ONE response, so a missing phase cannot be a stale phase, and the
|
|
connection-level reading is still a genuine, complete measurement. Dropping
|
|
the whole sample over an absent per-phase field would turn a healthy meter
|
|
into a climbing age, which is the exact false-trip this ticket removes.
|
|
"""
|
|
if not isinstance(doc, dict):
|
|
raise P1Error(f"meter response is not a JSON object ({type(doc).__name__})")
|
|
|
|
imp, exp = split_signed(doc.get("active_power_w"))
|
|
|
|
pi = pe = None
|
|
# ⚠️ range(phases), not the legs the meter served: a 3-phase meter configured
|
|
# as meter_phases: 1 yields a per-phase tuple covering ONE leg of three.
|
|
# Control is unaffected (it uses the connection figure), but the
|
|
# capacity-tariff peak is understated. Filed separately - not fixed here.
|
|
legs = [doc.get(f"active_power_l{i + 1}_w") for i in range(phases)]
|
|
if all(v is not None for v in legs):
|
|
pairs = [split_signed(v) for v in legs]
|
|
pi = [a for a, _ in pairs]
|
|
pe = [b for _, b in pairs]
|
|
|
|
return make_sample(SOURCE_HOMEWIZARD, imp, exp, phases=phases,
|
|
phase_import_w=pi, phase_export_w=pe,
|
|
ingest_ts=now, ingest_mono=ingest_mono)
|
|
|
|
|
|
def measurement_fingerprint(doc) -> tuple:
|
|
"""The part of a response that a genuinely new measurement has to move.
|
|
|
|
Power plus both energy registers. A meter under any real load advances a
|
|
kWh counter (1 Wh resolution: ~10 s at 350 W), so this changes even when the
|
|
power figure happens to repeat. Used ONLY for `unchanged_s` - see there for
|
|
why it must not be allowed anywhere near the age.
|
|
"""
|
|
if not isinstance(doc, dict):
|
|
return ()
|
|
return (doc.get("active_power_w"),
|
|
doc.get("total_power_import_kwh"),
|
|
doc.get("total_power_export_kwh"))
|
|
|
|
|
|
class HomeWizardLocalSource:
|
|
"""Polls the meter's own local HTTP API instead of a Home Assistant entity.
|
|
|
|
⚠️ THIS IS THE ONLY TRANSPORT WHOSE AGE MEASURES ARRIVAL. That is the entire
|
|
reason it exists, and it is not an optimisation - it is the difference
|
|
between an age sensor a firmware watchdog may threshold and one it may not.
|
|
|
|
Every HTTP response is a genuine arrival: the meter answered, now, with its
|
|
current reading. Whether the NUMBER moved is irrelevant, so a healthy meter
|
|
under a flat load resets the age exactly like a busy one. Home Assistant
|
|
cannot provide this at all - it emits `state_changed` only on a change, does
|
|
not advance `last_reported` on either serialiser for a repeat, and refuses a
|
|
`state_reported` subscription outright ("Event filter is required"). Measured
|
|
on the ENV-01 rig over 70 s of a frozen meter, and independently against the
|
|
live house over ten repeated readings. So on `ha_signed` the age means "time
|
|
since the value changed", and our own capture of this house's meter has
|
|
42.2 s and 97.0 s between changes - both past the default `meter_max_age_s`
|
|
of 30, i.e. a false trip to 0 W on a perfectly healthy meter.
|
|
|
|
Everything else is TEL-01's pipeline unchanged: ingest timestamping,
|
|
`meter_max_age_s`, the clock-recomputed age, the plausibility bounds, and a
|
|
failed read treated as a MISSING reading rather than 0 W.
|
|
|
|
⚠️ A failed or timed-out poll submits nothing. It is a rejection, so the
|
|
last good sample and its stamp stay exactly where they were and the age goes
|
|
on climbing - which is precisely the signal a dead meter should produce.
|
|
|
|
⚠️ POLL CADENCE vs `meter_max_age_s`. The age can only be as fresh as the
|
|
poll interval, so `meter_poll_s` must sit well under `meter_max_age_s` or the
|
|
reading flaps in and out of staleness on a healthy meter. The real meter
|
|
updates every ~5.0 s (NOTES.md:640 measures 4.97 s), so polling faster than
|
|
that buys nothing but re-reads. Default 5 s against a 30 s max age; the
|
|
startup check in build_source refuses `meter_poll_s >= meter_max_age_s` and
|
|
warns past half of it.
|
|
|
|
ponytail: a plain `asyncio.sleep` loop rather than a scheduler, and no
|
|
backoff. A poll that fails costs one rejection and is retried on the next
|
|
tick; a meter that is down stays down and the age reports it. The ceiling is
|
|
a meter so slow that polls overlap - the request timeout is held under the
|
|
poll interval so they cannot.
|
|
"""
|
|
|
|
def __init__(self, session, ingest: P1Ingest, host: str, *,
|
|
port: int = HOMEWIZARD_PORT, poll_s: float = 5.0,
|
|
timeout_s: float | None = None):
|
|
self.session = session
|
|
self.ingest = ingest
|
|
self.host, self.port = host, int(port)
|
|
self.poll_s = float(poll_s)
|
|
# ⚠️ Held under the poll interval on purpose: a timeout longer than the
|
|
# cadence queues polls behind a hung meter, and a backlog of requests
|
|
# all landing at once would stamp several arrivals for one measurement.
|
|
self.timeout_s = float(timeout_s) if timeout_s else max(1.0, self.poll_s * 0.8)
|
|
self.url = f"http://{self.host}:{self.port}{HOMEWIZARD_PATH}"
|
|
self.connected = False
|
|
self.polls = 0
|
|
self.poll_errors = 0
|
|
self._fingerprint: tuple | None = None
|
|
self._changed_mono: float | None = None
|
|
|
|
@property
|
|
def unchanged_s(self) -> float | None:
|
|
"""Seconds since the meter last served a DIFFERENT measurement.
|
|
|
|
⚠️ Diagnostic only, and deliberately NOT folded into
|
|
`sensor.p1_sample_age_s`. A frozen meter (SAFETY-01's six-minute
|
|
runaway) answers 200 OK with a stale document forever, so no arrival
|
|
detector can see it - only the document standing still can, and this is
|
|
that signal. It is exposed rather than acted on because it is NOT safe
|
|
as a 30 s watchdog input: this controller regulates grid power toward
|
|
~0 W, and at a converged -10 W the export register takes six minutes to
|
|
advance by its 1 Wh resolution while the power figure legitimately
|
|
repeats. Thresholding that would rebuild the very limit cycle the
|
|
arrival stamp just removed, at the exact operating point we aim for.
|
|
Whoever wires a freeze detector must handle the low-power case first.
|
|
"""
|
|
if self._changed_mono is None:
|
|
return None
|
|
return max(0.0, time.monotonic() - self._changed_mono)
|
|
|
|
async def run(self) -> None:
|
|
"""Long-lived task: poll, submit, sleep, forever.
|
|
|
|
⚠️ Nothing but cancellation may end this loop. An escaping exception
|
|
would kill the poll task for the lifetime of the add-on, and it would do
|
|
it QUIETLY: the failure mode is safe (no submissions, the age climbs, the
|
|
watchdog holds the battery at 0 W) but it looks identical to a dead
|
|
meter, so the operator goes hunting the wrong device. Retry on the next
|
|
tick instead - poll_s is already the retry cadence, so no backoff.
|
|
"""
|
|
_LOG.info("P1 ingest: polling %s every %.1fs", self.url, self.poll_s)
|
|
while True:
|
|
try:
|
|
await self.poll_once()
|
|
except asyncio.CancelledError:
|
|
raise
|
|
except Exception as err: # noqa: BLE001 - the task must outlive it
|
|
_LOG.warning("P1 meter poll %s: %s - retrying in %.1fs", self.url,
|
|
str(err) or type(err).__name__, self.poll_s)
|
|
await asyncio.sleep(self.poll_s)
|
|
|
|
async def poll_once(self) -> bool:
|
|
"""One GET. Returns True if a sample was accepted."""
|
|
self.polls += 1
|
|
try:
|
|
async with self.session.get(
|
|
self.url,
|
|
timeout=aiohttp.ClientTimeout(total=self.timeout_s)) as resp:
|
|
if resp.status != 200:
|
|
raise P1Error(f"meter returned HTTP {resp.status}")
|
|
# content_type=None: the meter's own firmware is the authority on
|
|
# what it serves, and refusing a reading over a Content-Type
|
|
# header would be a fabricated outage.
|
|
# ⚠️ Kept because a real HomeWizard firmware that ever answers
|
|
# text/plain would otherwise read as a dead meter and hold the
|
|
# battery at 0 W. (This was once recorded here as an equivalent
|
|
# mutant - it is not. aiohttp's json_response always sets
|
|
# application/json, but web.Response(text=...) defaults to
|
|
# text/plain, so the fake meter CAN serve valid JSON under the
|
|
# wrong header, and test_p1.py now does.)
|
|
doc = await resp.json(content_type=None)
|
|
except asyncio.CancelledError:
|
|
raise
|
|
except Exception as err: # noqa: BLE001 - any read failure is a gap
|
|
self.connected = False
|
|
self.poll_errors += 1
|
|
# ⚠️ str() of a bare asyncio.TimeoutError is the EMPTY STRING, so
|
|
# f"...: {err}" renders "last error:" and then nothing - on the
|
|
# status page, on a hung meter, at the moment the battery has just
|
|
# dropped to 0 W and someone is reading that line to find out why.
|
|
# The class name is the only thing that says "it timed out".
|
|
# ⚠️ str(err), not `err or ...`: an exception object is ALWAYS truthy,
|
|
# empty message or not, so the `or` would never reach the fallback.
|
|
self.ingest.reject(
|
|
f"meter poll {self.url}: {str(err) or type(err).__name__}")
|
|
return False
|
|
|
|
self.connected = True
|
|
try:
|
|
sample = parse_homewizard(doc, self.ingest.phases)
|
|
# ⚠️ Submitted unconditionally, INCLUDING a document identical to the
|
|
# last one. The response is the arrival; the number is not.
|
|
# Inside the try on purpose: submit() outside it would put a raise on
|
|
# a path with no handler at all, killing the poll task permanently -
|
|
# safely (the age climbs) but silently. run() backstops the rest.
|
|
self.ingest.submit(sample)
|
|
self._note(doc)
|
|
except P1Error as err:
|
|
self.ingest.reject(err)
|
|
return False
|
|
return True
|
|
|
|
def _note(self, doc) -> None:
|
|
fp = measurement_fingerprint(doc)
|
|
if self._changed_mono is None or fp != self._fingerprint:
|
|
self._fingerprint = fp
|
|
self._changed_mono = time.monotonic()
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# selection
|
|
# --------------------------------------------------------------------------- #
|
|
def is_enabled(opts: dict) -> bool:
|
|
"""Whether P1 ingestion is switched on at all.
|
|
|
|
⚠️ One definition, because three places depend on it and they MUST agree:
|
|
where the grid reading comes from, whether the ingest task is started, and
|
|
whether sensor.p1_sample_age_s is announced over MQTT discovery. An age
|
|
sensor announced with no ingester behind it is a watchdog input nobody is
|
|
feeding, and the ESP32 trips on it.
|
|
"""
|
|
return str(opts.get("meter_source", "off") or "off").strip() not in ("off", "")
|
|
|
|
|
|
def homewizard_host_port(opts: dict) -> tuple[str, str]:
|
|
"""`p1_host` split into host and (still unparsed) port text.
|
|
|
|
One definition because validate_meter_options() and build_source() must
|
|
agree byte for byte about what counts as a usable address - a pasted browser
|
|
URL is the obvious way to fill this in wrong.
|
|
"""
|
|
host = str(opts.get("p1_host", "") or "").strip()
|
|
host = host.removeprefix("http://").removeprefix("https://").rstrip("/")
|
|
host, _, port_s = host.partition(":")
|
|
return host, port_s
|
|
|
|
|
|
def validate_meter_options(opts: dict, phases: int, max_age_s: float) -> list[str]:
|
|
"""Every way `meter_source` can name a meter that will never produce a reading.
|
|
|
|
⚠️ THIS IS THE ONLY PLACE THAT KNOWS THESE RULES. It used to be a run of
|
|
inline `return None`s inside build_source(); CFG-01 lifted them out because
|
|
the same predicates now have two callers with two different consequences:
|
|
|
|
validate.validate_startup_config() refuses to start the add-on
|
|
build_source() returns None, as it always did
|
|
|
|
They are one function and not two on purpose. A second copy of "what makes a
|
|
meter config wrong" would drift, and the half that drifted would be the one
|
|
that decides whether the process is allowed to run.
|
|
|
|
Checked ONCE at startup rather than per telegram, because a misconfigured
|
|
source otherwise fails in the one way indistinguishable from a healthy source
|
|
nobody has sent anything to yet: no samples, a climbing age, the firmware
|
|
watchdog holding the battery at 0 W, and nothing in the log saying why.
|
|
|
|
Returns a list of human-readable problems; empty means the config is usable.
|
|
`phases` and `max_age_s` are passed in rather than read from opts so the
|
|
caller decides whether it is validating the options or an already-built
|
|
P1Ingest - build_source() has an ingest whose values may not have come from
|
|
this dict.
|
|
"""
|
|
source = str(opts.get("meter_source", "off") or "off").strip()
|
|
if source in ("off", ""):
|
|
return []
|
|
problems: list[str] = []
|
|
|
|
if source == SOURCE_HA:
|
|
imp = str(opts.get("p1_import_entity", "") or "").strip()
|
|
exp = str(opts.get("p1_export_entity", "") or "").strip()
|
|
if not imp or not exp:
|
|
problems.append(
|
|
f"{_S('meter_source')} is {SOURCE_HA!r}, so it needs "
|
|
f"{_S('p1_import_entity')} and {_S('p1_export_entity')}. They "
|
|
f"are {messages.shown(imp)} and {messages.shown(exp)}.")
|
|
elif source == SOURCE_HA_SIGNED:
|
|
net = str(opts.get("p1_net_entity", "") or "").strip()
|
|
phase_net = [str(e).strip() for e in
|
|
(opts.get("p1_phase_net_entities") or []) if str(e).strip()]
|
|
if not net:
|
|
problems.append(
|
|
f"{_S('meter_source')} is {SOURCE_HA_SIGNED!r}, so it needs "
|
|
f"{_S('p1_net_entity')}.")
|
|
if phase_net and len(phase_net) != phases:
|
|
problems.append(
|
|
f"{_S('p1_phase_net_entities')} has {len(phase_net)} entities, "
|
|
f"but {_S('meter_phases')} is {phases}. Make them match.")
|
|
elif source == SOURCE_MQTT:
|
|
if not str(opts.get("meter_mqtt_topic", "") or "").strip():
|
|
problems.append(f"{_S('meter_source')} is {SOURCE_MQTT!r}, so it "
|
|
f"needs {_S('meter_mqtt_topic')}.")
|
|
elif source == SOURCE_HOMEWIZARD:
|
|
host, port_s = homewizard_host_port(opts)
|
|
if not host:
|
|
problems.append(
|
|
f"{_S('meter_source')} is {SOURCE_HOMEWIZARD!r}, so it needs "
|
|
f"{_S('p1_host')}, the meter's address on your network.")
|
|
if port_s:
|
|
try:
|
|
int(port_s)
|
|
except ValueError:
|
|
problems.append(f"{_S('p1_host')} is {messages.shown(opts.get('p1_host'))}, and "
|
|
"its port is not a number.")
|
|
try:
|
|
# ⚠️ Not `or 5`: that turns an explicit 0 into the default, and a
|
|
# cadence of 0 is a misconfiguration that must be reported, not
|
|
# quietly corrected into something that looks like it was asked for.
|
|
raw = opts.get("meter_poll_s", 5)
|
|
poll_s = float(5 if raw is None or raw == "" else raw)
|
|
except (TypeError, ValueError):
|
|
problems.append(f"{_S('meter_poll_s')} is "
|
|
f"{messages.shown(opts.get('meter_poll_s'))}, which is not a "
|
|
"number.")
|
|
else:
|
|
if poll_s <= 0:
|
|
problems.append(f"{_S('meter_poll_s')} must be above 0.")
|
|
# ⚠️ The age can never be fresher than the poll interval. At or past
|
|
# max_age_s every reading is stale before its successor arrives, so
|
|
# the controller would sit permanently on missing inputs while the
|
|
# meter is perfectly healthy - refuse it rather than ship that.
|
|
elif poll_s >= max_age_s:
|
|
problems.append(
|
|
f"{_S('meter_poll_s')} is {poll_s:.1f}, not below "
|
|
f"{_S('meter_max_age_s')} at {max_age_s:.1f}. Poll more "
|
|
"often, or every reading goes stale.")
|
|
else:
|
|
problems.append(
|
|
f"{_S('meter_source')} is {source!r}. Choose one of: off, "
|
|
+ ", ".join((SOURCE_HA, SOURCE_MQTT, SOURCE_HA_SIGNED, SOURCE_HOMEWIZARD))
|
|
+ ".")
|
|
return problems
|
|
|
|
|
|
def build_source(opts: dict, ingest: P1Ingest, session, broker: dict | None):
|
|
"""Return the transport named by `meter_source`, or None if disabled.
|
|
|
|
This is the whole of AC 1's "switchable": every consumer reads P1Ingest, so
|
|
changing transport is a config edit, never a code path.
|
|
|
|
⚠️ In the add-on this is now unreachable with a bad config -
|
|
validate_startup_config() refuses to start on anything
|
|
validate_meter_options() rejects. The guard stays because this function is
|
|
also called directly by tests and by anything embedding P1Ingest, and
|
|
"returns None" is a safer default for those callers than a transport that
|
|
can never read.
|
|
"""
|
|
source = str(opts.get("meter_source", "off") or "off").strip()
|
|
if not is_enabled(opts):
|
|
return None
|
|
problems = validate_meter_options(opts, ingest.phases, ingest.max_age_s)
|
|
if problems:
|
|
for problem in problems:
|
|
_LOG.error("%s - P1 ingestion disabled", problem)
|
|
return None
|
|
if source == SOURCE_HA:
|
|
return HaDsmrSource(session, ingest, {
|
|
"import": opts.get("p1_import_entity", ""),
|
|
"export": opts.get("p1_export_entity", ""),
|
|
"phase_import": opts.get("p1_phase_import_entities") or [],
|
|
"phase_export": opts.get("p1_phase_export_entities") or [],
|
|
})
|
|
if source == SOURCE_HA_SIGNED:
|
|
return HaSignedSource(session, ingest, {
|
|
"net": str(opts.get("p1_net_entity", "") or "").strip(),
|
|
"phase_net": [str(e).strip() for e in
|
|
(opts.get("p1_phase_net_entities") or []) if str(e).strip()],
|
|
})
|
|
if source == SOURCE_HOMEWIZARD:
|
|
host, port_s = homewizard_host_port(opts)
|
|
port = int(port_s) if port_s else HOMEWIZARD_PORT
|
|
raw = opts.get("meter_poll_s", 5)
|
|
poll_s = float(5 if raw is None or raw == "" else raw)
|
|
# A warning, not a refusal: one missed poll costs a stale reading and a
|
|
# 0 W cycle, which is fail-safe. The refusal above is for cadences that
|
|
# can NEVER be fresh.
|
|
if poll_s > ingest.max_age_s / 2:
|
|
_LOG.warning("“Time between meter reads” (%.1f s) is too close to "
|
|
"“Reading too old after” (%.1f s). Use 5 s and 30 s.",
|
|
poll_s, ingest.max_age_s)
|
|
return HomeWizardLocalSource(session, ingest, host, port=port, poll_s=poll_s)
|
|
if source == SOURCE_MQTT:
|
|
broker = broker or {}
|
|
return MqttP1Source(ingest, str(opts.get("meter_mqtt_topic", "")),
|
|
broker.get("host"), broker.get("port", 1883),
|
|
broker.get("username"), broker.get("password"))
|
|
return None
|