Files
glenn schrooyenandClaude Opus 5 04292c4a08
checks / unit (pull_request) Successful in 3m36s
DATA-01: thirty days of the real meter, and what AC 3 actually measures
TARIFF-01 and SAFETY-07 both merged with AC 3 - "monthly peak held under
the soft limit in >= 95 % of intervals over a 30-day simulation replay" -
honestly unmet, because two days of capture existed and it was believed
nothing longer could be produced. That belief was a true statement about
the past read as a claim about the future: HA's recorder still held the
whole month, and the reason nobody found out is that the 2026-08-25
export was a one-off script that left no artefact.

So the exporter is committed and runnable now.
scenarios/p1-net-30d-2026-08/ is 2026-08-01 to 2026-08-30, 485,841
samples, 30/30 days read, 0 failed, 4.13 MB gzipped, in the schema the
two committed days already fixed.

VERIFIED AGAINST THE DAYS THAT EXIST. `--verify` re-fetches 2026-08-20
and 2026-08-23 by this exact route and reproduces both committed files
BYTE FOR BYTE after decompression (865,024 B / 15,600 samples and
894,387 B / 16,351 samples). The gzip containers differ by design: this
exporter writes mtime 0 so its output is reproducible, where the one-off
wrote a wall clock.

A FAILED READ IS NOT AN OBSERVATION. A day whose read fails writes no
file, is recorded "read": "FAILED", and the run exits non-zero; the
entity is proved to exist through /api/states before any empty day is
believed, because a typo would otherwise answer 200 [] thirty times and
produce a consistent dataset of nothing. Nothing is interpolated, so the
month's 13 gaps (1291 s, 99.95 % covered) and its 3 `unavailable`
samples stay visible as holes.

SIZE: COMMITTED, DELIBERATELY, AND MEASURED. A git bundle of the whole
history goes from 1.58 MB (release/1.0 @ 86c43eb) to 5.28 MB - +3.70 MB,
x3.3 - and a Home Assistant supervisor clones that on every install.
Downsampling was not on the table: meter_max_age_s is 30 s, so any
coarser resample turns real coverage into modelled meter-gap and moves
the number being measured. "Outside the repo with a pointer" is what
already happened and is why this card exists. The files sit in a
SUBDIRECTORY so the non-recursive globs in test_tariff.py part 9 and
test_peakguard.py part 6 do not pick them up - CI keeps replaying the two
small days instead of growing ~50 s per PR.

AC 3, MEASURED OVER 30 DAYS (tools/replay_ac3.py, which must reproduce
test_peakguard part 6's 189/190 to be believed):

  uncontrolled  2847/2879 = 98.89 % of quarters at or under 3500 W
  guarded       2850/2879 = 98.99 %
  month peak    5029.8 W -> 4107.3 W

Met on the ">= 95 % of intervals" reading - and the uncontrolled figure
is the finding, because a month with no controller at all passes it. Not
met on the other reading of the same sentence: 4107.3 W is still over the
soft limit. The guard pulls 31 of the 32 over-cap quarters down but lands
only 3 under the cap, because CEILING_MONTH_PEAK defends the running
month peak rather than the cap once one is set. Both suites' honesty
notes are corrected to point at the month instead of at its absence.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015VXjA3LWccN1rJsk3okyPC
2026-08-31 11:15:02 +02:00
..

30 days of P1 net power — 2026-08-01 … 2026-08-30

sensor.p1_meter_active_power out of Home Assistant's recorder, one gzipped JSONL file per local day (Europe/Brussels), same schema as the two files one directory up:

{"t": "2026-08-22T22:00:00+00:00", "w": -25.0}

485,841 samples, 30/30 days read, 0 days failed, 4.13 MB gzipped. Produced by tools/export_p1_history.py; manifest.json here carries the per-day counts, sha256 of every file, and every gap.

python3 tools/export_p1_history.py --days 30 --end-date 2026-08-30 \
    --out-dir scenarios/p1-net-30d-2026-08

Why this exists

TARIFF-01 AC 3 and SAFETY-07 AC 3 both ask for "monthly peak held under the soft limit in >= 95 % of intervals over a 30-day simulation replay", and both were merged with the criterion honestly unmet because only two days of capture existed. test_tariff.py's docstring still recorded the reason as "nothing longer was ever captured" — which was true about the past and was quietly read as a statement about the future. It was not one. Probed on 2026-08-31, HA's recorder answered a 20-minute window with 230 samples at 30 days back and 0 at 45, so the whole month was still there and was exported the same day. The window rolls: 2026-08-01 leaves the recorder around 2026-09-15.

The size decision — COMMITTED, deliberately, at 4.13 MB

Three options were on the table. This one was taken and the cost is stated rather than absorbed, and it is measured rather than estimated - a git bundle of the whole history before and after this commit:

release/1.0 @ 86c43eb   1.58 MB
DATA-01                 5.28 MB      +3.70 MB, x3.3

A full clone of this repository therefore more than triples, and a Home Assistant supervisor clones it on every add-on install and update. That is the price; the paragraphs below are why it was judged worth paying rather than avoided by making the data worse or making it disappear.

  • Downsampling is not available. It is not a matter of taste: the shipped meter_max_age_s is 30 s, which is the age past which the controller's own ingest stops counting a held reading as coverage. Any resample coarser than 30 s therefore converts real coverage into modelled meter-gap and changes the very number AC 3 measures. Resampling at or under 30 s saves almost nothing against a 5 s cadence.
  • "Outside the repo with a pointer" is what already happened, and it is why this card exists. The two committed days came from the infra repo's private sim/scenarios/, the one-off export script was thrown away, and three cards then deferred an acceptance criterion on the belief that the data could not be produced again. test_tariff.py states the general rule verbatim: a regression fixture CI cannot see is not a regression fixture. CI's token is minted for this repository only.
  • So: committed — but in a SUBDIRECTORY, which is load-bearing. Both test_tariff.py part 9 and test_peakguard.py part 6 do (HERE / "scenarios").glob("p1-net-*.jsonl.gz"), which does not recurse. Drop these thirty files beside the two and CI grows sixty full-day replays - about 50 s of pure compute - on every PR forever, test_tariff.py's check(... replay_days < 30) starts failing, and both files' honesty notes become false. In p1-net-30d-2026-08/ they are visible to CI, invisible to the globs, and the two committed days keep their job as the small fixtures the suites grade against.

They are also written once and never edited, so git stores one already-deflated blob per day with no delta churn.

Gaps are visible, never interpolated

Nothing is filled, carried forward or resampled. A hole in the recorder is a hole here — consecutive t values further apart than the cadence — which is exactly what the controller's freshness rule already reads as no coverage, so a missing hour replays as a missing hour rather than as an hour of whatever the last sample said.

13 gaps over 30 s, 1291 s in total: 99.950 % of the month is covered. All of them are listed in manifest.json and by python3 tools/replay_ac3.py <dir> --report-gaps. The longest is 263 s on 2026-08-11.

Three samples in the month carried the state unavailable rather than a number (2026-08-18 02:03:54Z and 04:10:41Z, 2026-08-25 01:52:02Z). There is no float to write and the committed schema has no slot for a non-number, so they are absent from the series and recorded individually in manifest.json under non_numeric_states. Each one sits inside a gap that is already listed, which is the honest rendering: the meter was not reporting a power then.

A failed read is never an empty hour. If a day's read fails, the exporter writes NO FILE, records "read": "FAILED" with the error, and exits non-zero — so a missing day is unreadable-and-known rather than silently zero. The entity is also proved to exist through /api/states before any day is trusted, because a typo would otherwise answer 200 [] thirty times and produce a beautifully consistent dataset of nothing. On this run: 30 read, 0 failed, 0 empty.

AC 3, measured

cd goodwe_controller
python3 tools/replay_ac3.py scenarios/p1-net-30d-2026-08

485841 samples, 2879 closed quarters (30.0 days' worth)
month 2026-08: peak 5029.8 W uncontrolled -> 4107.3 W guarded

TARIFF-01 AC 3 (UNCONTROLLED): 2847/2879 = 98.89 % of quarters at or under 3500 W
SAFETY-07 AC 3 (GUARDED):      2850/2879 = 98.99 %
  guard cycles 143810, engaged on the measurement 1542, blind 195,
  saturated 0, modelled discharge 8060 Wh

2879 rather than 2880 quarters: the last quarter of 2026-08-30 closes only when a telegram past its boundary arrives, and that telegram is 2026-08-31's.

What the numbers actually say — three findings, not a pass

  1. AC 3 is met on the ">= 95 % of intervals" reading, and the criterion turns out to be nearly insensitive. The UNCONTROLLED month — no controller, no peak shaving, nothing running — already scores 98.89 %. A criterion that a house passes by doing nothing does not discriminate between a working guard and an absent one. That is a finding about the acceptance criterion, and it is reported rather than filed as a win.
  2. On the other reading of the same sentence, AC 3 is NOT met. "Monthly peak held under the soft limit" taken as the peak itself fails: the guard pulls August's peak from 5029.8 W to 4107.3 W (-922.5 W, -18.3 %), which is real money on a Belgian capacity tariff, but 4107.3 W is still over the 3500 W soft limit. Both readings are printed by the tool; neither is hidden.
  3. The guard stops defending the soft limit once the month's peak is set, and that is by design. quarter_ceiling_w uses CEILING_MONTH_PEAK: the billed quantity is the month's maximum, so a quarter under an already-recorded peak costs nothing extra and the guard does not spend the battery on it. The month's 32 over-cap quarters fall on four nights (2026-08-17, -18, -20, -21); 31 of them are pulled down — 7494 W of quarter-average removed in total, 8060 Wh of modelled discharge — but only 3 land under 3500 W, because by then the ceiling being defended is the running month peak (~4107 W), not the soft limit. The single untouched quarter, 2026-08-18T08:15Z at 4039.7 W, is the same rule seen from the other side: the running peak was already 4833 W, so 4039.7 W added nothing to the bill and the guard correctly stayed out. AC 3's wording asks the guard for something its own tariff logic deliberately does not optimise.

Also worth an eye: 195 blind engagements over 143,810 cycles — the guard acting on a projection it could not fully cover. On the two committed days that count was 5. It scales with the gaps above.

Verification of the exporter

tools/export_p1_history.py --verify re-fetches a committed day by the exact route used here and diffs it against the file:

python3 tools/export_p1_history.py \
    --verify scenarios/p1-net-2026-08-20.jsonl.gz \
             scenarios/p1-net-2026-08-23.jsonl.gz

Both days come back byte-identical after decompression — 865,024 B / 15,600 samples and 894,387 B / 16,351 samples — and the copies of those two days inside this directory are byte-identical to the committed ones too. The gzip CONTAINERS differ by design: a gzip header carries an mtime and a stored filename, and this exporter writes mtime 0 so its own output is reproducible, where the 2026-08-25 one-off wrote a wall-clock mtime. Content equality is the claim; container equality would only assert that two files were written at the same second.

tools/replay_ac3.py is checked the same way rather than trusted: pointed at scenarios/ it must reproduce test_peakguard.py part 6's published totals, and --expect makes it a hard check.

python3 tools/replay_ac3.py scenarios --expect 189/190

reproduces 189/190 quarters, the 5029.8 W billed quarter, the 3485.3 W guarded result and the 878 Wh of modelled discharge that part 6 prints.