TEL-01 review fixes: stop the age sensor tripping installs that have no meter
T-1, and it was a fleet-wide trip to zero. publish() emitted p1_age unconditionally, and published_age_s counts from P1Ingest.__init__ when no sample has ever arrived. With meter_source defaulting to off, every existing install would have published sensor.p1_sample_age_s climbing without bound; the ESP32 does `has_state() && state >= max_age_s` and forces the layer-1 failsafe, so each of them would have pinned its inverter at 0 W within 30 s. Exactly the opposite of the zero-regression the off default was for. The key is now omitted from the payload AND from MQTT discovery when P1 is off, so the entity does not exist at all - which is the status quo, and what has_state() is testing for. The predicate is one function, is_enabled(), because the grid reading, the task start and the discovery announcement have to agree or this comes back. T-2, connect no longer manufactures a sample. get_states returns whatever HA currently holds, which after a Core restart is a RestoreEntity value of unknown age; stamping it with ingest_ts=now reset the age and reported a fresh meter that could have been dead for an hour. run()'s own docstring already said a reconnect must emit nothing - the code disagreed with it, and a test asserted the violation. The cache is still primed, so the first real state_changed builds a complete sample; the age just stays honest until one arrives. T-3, gaps are no longer filled with the last held value. The averager held a sample forward across any interval, so a meter dying at 5 kW and returning ten minutes later credited 5 kW x 600 s to the capacity-tariff accumulator - a fabricated peak on a permanent record. The hold is capped at max_age_s: past that the stretch is walked so block boundaries still land correctly, but nothing accumulates and elapsed does not grow, which is what finally makes the comment about a gap dragging the billed average down true. Same threshold for control and billing: a reading too old to steer by is too old to bill by. T-4, the out-of-order/duplicate guard is covered. It was untested, and the reason is worth recording: the obvious assertion passes without the guard, because the negative interval is separately refused by the covered > 0 test. What the guard prevents is the timestamp REWIND, which only shows up one sample later as a re-integrated window. The test now goes one sample later. T-6, DOCS was wrong about latency. meter_max_age_s and stale_input_s stack, so meter death to 0 W is 45 s and not 30. Documented as a table with both clocks. Also documented the T-5 asymmetry rather than papering over it: the age measures arrival, not change, so a stuck MQTT bridge republishing its last telegram still looks fresh. Correct on ha_dsmr, not detectable on mqtt_p1 without a change-detector. Written up as a known limit. Writing the T-1 test caught a second defect in the test itself: it recorded only MQTT topics, and object_id lives in the payload, so "the age sensor is not announced" had been passing for the wrong reason. test_p1.py: 99 -> 122 checks. 14 mutations run, all 14 red, files restored byte-identical - including one per fix above. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Du77usMj8XNKNFZGmUiWDa
This commit is contained in:
co-authored by
Claude Opus 5
parent
2097b7aaf6
commit
0a617c5482
@@ -229,6 +229,47 @@ before = a.partial_ws
|
||||
a.add(sample(1000.0, at=BASE + timedelta(seconds=5)))
|
||||
check("an out-of-order telegram is dropped, not integrated backwards",
|
||||
a.partial_ws == before and a.elapsed_s == 10.0)
|
||||
# ⚠️ The assertion above is NOT sufficient on its own, and that is the whole
|
||||
# lesson: deleting the guard still passes it, because the negative interval is
|
||||
# separately refused by the `covered > 0` test. What the guard actually prevents
|
||||
# is the REWIND - without it the held timestamp moves back to +5 s and the next
|
||||
# telegram re-integrates the 5..10 s window that was already counted. The damage
|
||||
# only becomes visible one sample later, so the test has to go one sample later.
|
||||
a.add(sample(1000.0, at=BASE + timedelta(seconds=20)))
|
||||
check("...and the held timestamp is not rewound, so the next telegram "
|
||||
"cannot double-count", a.elapsed_s == 20.0 and a.partial_ws == 20000.0)
|
||||
|
||||
# A duplicate telegram (identical timestamp) is the same rule.
|
||||
a = QuarterAverager(1)
|
||||
a.add(sample(1000.0, at=BASE))
|
||||
a.add(sample(1000.0, at=BASE + timedelta(seconds=10)))
|
||||
a.add(sample(4000.0, at=BASE + timedelta(seconds=10)))
|
||||
a.add(sample(1000.0, at=BASE + timedelta(seconds=20)))
|
||||
check("a duplicate timestamp neither re-integrates nor replaces the held value",
|
||||
a.elapsed_s == 20.0 and a.partial_ws == 20000.0)
|
||||
|
||||
# A gap must not be filled with the last held value. The meter dies at 5 kW and
|
||||
# returns ten minutes later; hold-forward would credit 5 kW x 600 s to the
|
||||
# capacity-tariff accumulator - a fabricated peak, on a permanent record, from
|
||||
# data nobody measured.
|
||||
a = QuarterAverager(1, max_hold_s=30.0)
|
||||
a.add(sample(5000.0, at=BASE))
|
||||
a.add(sample(5000.0, at=BASE + timedelta(seconds=600)))
|
||||
check("a 600 s gap is held for at most max_hold_s, not for the whole gap",
|
||||
a.partial_ws == 5000.0 * 30.0)
|
||||
check("the unobserved stretch does not count as elapsed time", a.elapsed_s == 30.0)
|
||||
closed = a.add(sample(5000.0, at=BASE + timedelta(seconds=900)))
|
||||
check("the outage drags the billed quarter down instead of inventing a peak",
|
||||
len(closed) == 1 and abs(closed[0].offtake_avg_w - 300000.0 / 900.0) < 1e-9)
|
||||
check("...nowhere near the 5000 W a hold-forward would have billed",
|
||||
closed[0].offtake_avg_w < 400.0)
|
||||
|
||||
# The cap must not disturb a normally-spaced stream.
|
||||
a = QuarterAverager(1, max_hold_s=30.0)
|
||||
for i in range(0, 121, 5): # a healthy 5 s telegram cadence
|
||||
a.add(sample(2000.0, at=BASE + timedelta(seconds=i)))
|
||||
check("a healthy 5 s cadence is untouched by the hold cap",
|
||||
a.elapsed_s == 120.0 and abs(a.offtake_avg_w - 2000.0) < 1e-9)
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
print("ingest timestamp, age and staleness")
|
||||
@@ -482,9 +523,15 @@ async def _e2e():
|
||||
|
||||
live, wire = asyncio.run(_e2e())
|
||||
check("the websocket handshake and subscription complete", live.samples >= 1)
|
||||
# Six state_changed events arrived (two per telegram). The debounce is what
|
||||
# makes that three consistent samples instead of six half-updated ones.
|
||||
check("three telegrams produce three samples, not six", live.samples == 3)
|
||||
# ⚠️ TWO, not three. get_states primes the cache but must NOT build a sample:
|
||||
# HA returns whatever it currently holds, which after a Core restart is a
|
||||
# RestoreEntity value of unknown age, and stamping that with ingest_ts=now
|
||||
# resets the age and reports a fresh meter that may have been dead for an hour.
|
||||
# Only the two real state_changed telegrams become samples. Four state_changed
|
||||
# events arrived (two per telegram); the debounce is what makes those two
|
||||
# consistent samples rather than four half-updated ones.
|
||||
check("connecting does not manufacture a sample from cached HA state",
|
||||
live.samples == 2)
|
||||
check("the final export-dominant telegram nets negative",
|
||||
live.last.net_w == -800.0)
|
||||
check("the sample was built over the wire, tagged with its transport",
|
||||
@@ -492,9 +539,102 @@ check("the sample was built over the wire, tagged with its transport",
|
||||
check("an entity we did not subscribe to is never cached",
|
||||
"sensor.something_else" not in wire.cache and len(wire.cache) == 1)
|
||||
check("a mid-stream unavailable is a parse error, not a sample",
|
||||
live.parse_errors == 1 and live.samples == 3)
|
||||
live.parse_errors == 1 and live.samples == 2)
|
||||
check("the last good reading survives the unavailable", live.net_w == -800.0)
|
||||
check("the averager integrated the live stream", live.averager.elapsed_s > 0.5)
|
||||
check("the averager integrated the live stream", live.averager.elapsed_s > 0.2)
|
||||
|
||||
# The reason get_states still matters: it is what lets the FIRST real telegram
|
||||
# build a complete sample instead of waiting for every entity to change once.
|
||||
check("the primed cache let the first telegram build immediately",
|
||||
live.samples == 2 and live.last.import_w == 0.0)
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
print("the age sensor must not exist when P1 is off")
|
||||
# ⚠️ This is a fleet-wide regression guard, not a nicety. The ESP32 watchdog
|
||||
# does `id(p1_age_s).has_state() && id(p1_age_s).state >= max_age_s` and forces
|
||||
# the layer-1 failsafe. published_age_s counts from P1Ingest.__init__, so if the
|
||||
# age were published with meter_source off it would climb past 30 s on every
|
||||
# existing install within half a minute and pin the inverter at 0 W forever.
|
||||
|
||||
from app.p1 import is_enabled # noqa: E402
|
||||
from app.mqtt import SENSORS, MqttPublisher # noqa: E402
|
||||
|
||||
check("meter_source off is disabled", is_enabled({"meter_source": "off"}) is False)
|
||||
check("a missing meter_source is disabled", is_enabled({}) is False)
|
||||
check("an empty meter_source is disabled", is_enabled({"meter_source": ""}) is False)
|
||||
check("ha_dsmr is enabled", is_enabled({"meter_source": SOURCE_HA}) is True)
|
||||
check("mqtt_p1 is enabled", is_enabled({"meter_source": SOURCE_MQTT}) is True)
|
||||
|
||||
# The entity id SAFETY-01's firmware subscribes to, pinned by object_id.
|
||||
row = [s for s in SENSORS if s[0] == "p1_age"]
|
||||
check("the age sensor is declared exactly once", len(row) == 1)
|
||||
check("its object_id pins entity_id to sensor.p1_sample_age_s",
|
||||
row[0][1] == "p1_sample_age_s")
|
||||
check("it is published in seconds", row[0][3] == "s")
|
||||
|
||||
|
||||
class _RecordingClient:
|
||||
def __init__(self):
|
||||
self.sent = []
|
||||
|
||||
def publish(self, topic, payload=None, retain=False):
|
||||
# Topic AND payload: object_id, the thing that actually pins the entity
|
||||
# id, only appears in the discovery payload. Recording topics alone made
|
||||
# the "is not announced" check pass for the wrong reason.
|
||||
self.sent.append(f"{topic} {payload}")
|
||||
|
||||
|
||||
def _announced(omit):
|
||||
pub = MqttPublisher(None, 1883, omit=omit) # host None -> never connects
|
||||
pub.client = _RecordingClient()
|
||||
pub._announce()
|
||||
return " ".join(pub.client.sent)
|
||||
|
||||
|
||||
check("with P1 off the age sensor is never announced",
|
||||
"p1_sample_age_s" not in _announced(("p1_age",)))
|
||||
check("the other status entities are still announced with P1 off",
|
||||
"goodwe_grid_power" in _announced(("p1_age",)))
|
||||
check("with P1 on the age sensor IS announced",
|
||||
"p1_sample_age_s" in _announced(()))
|
||||
|
||||
# And the publish dict itself, through the real Controller.
|
||||
from app.main import Controller # noqa: E402
|
||||
|
||||
|
||||
class _Store:
|
||||
data = {}
|
||||
|
||||
def set(self, *a):
|
||||
pass
|
||||
|
||||
def get_time(self, *a):
|
||||
return None
|
||||
|
||||
|
||||
class _Pub:
|
||||
def __init__(self):
|
||||
self.last = {}
|
||||
|
||||
def publish(self, values):
|
||||
self.last = values
|
||||
|
||||
def close(self):
|
||||
pass
|
||||
|
||||
|
||||
pub_off = _Pub()
|
||||
Controller({"meter_source": "off"}, None, _Store(), pub_off).publish()
|
||||
check("with P1 off, p1_age is absent from the published payload",
|
||||
"p1_age" not in pub_off.last)
|
||||
check("...while the normal status keys are still published",
|
||||
"setpoint" in pub_off.last and "grid" in pub_off.last)
|
||||
|
||||
pub_on = _Pub()
|
||||
Controller({"meter_source": SOURCE_HA}, None, _Store(), pub_on).publish()
|
||||
check("with P1 on, p1_age is published", "p1_age" in pub_on.last)
|
||||
check("...as a number, so has_state() becomes true only once we feed it",
|
||||
isinstance(pub_on.last["p1_age"], float))
|
||||
|
||||
print()
|
||||
if fails:
|
||||
|
||||
Reference in New Issue
Block a user