admin/goodwe-addon
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

DATA-01: thirty days of the real meter, and what AC 3 actually measures #25

Merged
admin merged 1 commits from DATA-01 into release/1.0 2026-08-31 11:35:46 +02:00
Conversation 1 Commits 1 Files Changed 37
+1779 -16
37 changed files with 1779 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
goodwe_controller/CHANGELOG.md
+39
View File
@@ -2,6 +2,45 @@
## Unreleased
**Thirty days of the real meter, and the acceptance criterion three cards
deferred is now a number (DATA-01).** TARIFF-01 and SAFETY-07 both merged with
AC 3 - *"monthly peak held under the soft limit in >= 95 % of intervals over a
30-day simulation replay"* - honestly unmet, on the belief that only two days of
capture existed and nothing longer could be produced. That belief came from a
true statement about the past being read as one about the future: HA's recorder
still held the whole month. `scenarios/p1-net-30d-2026-08/` is 2026-08-01 to
2026-08-30, **485,841 samples, 30/30 days read, 0 failed, 4.13 MB gzipped**, in
the same schema as the two days that were already there.
`tools/export_p1_history.py` is the exporter, committed and runnable - the
2026-08-25 one-off was not, and that is exactly why the data was thought
unobtainable. It reads day-at-a-time, retries, and **never lets a failed read
look like an empty hour**: a day whose read fails writes no file, is recorded as
`"read": "FAILED"`, and the run exits non-zero; the entity is proved to exist
before any empty day is believed. Nothing is interpolated, so the month's 13
gaps (1291 s, 99.95 % covered) and its 3 `unavailable` samples stay visible.
`--verify` re-fetches the two committed days by the same route and reproduces
them **byte for byte** after decompression.
`tools/replay_ac3.py` runs the month. Over 30 days the criterion measures
**98.89 % uncontrolled and 98.99 % guarded**, so it is met on that reading - and
the uncontrolled figure is the finding: a month with no controller at all passes
it. On the other reading of the same sentence it is not met - the guard takes
August's peak from **5029.8 W to 4107.3 W**, still over the 3500 W soft limit,
because once a month peak is set `CEILING_MONTH_PEAK` defends that peak rather
than the cap. Neither number is hidden; both are in that directory's README.
The size decision is committed and measured, not absorbed: a `git bundle` of
the whole history goes from **1.58 MB to 5.28 MB** (+3.70 MB, x3.3), which a
Home Assistant supervisor clones on every install. Downsampling was not
available - `meter_max_age_s` is 30 s, so any coarser resample converts real
coverage into modelled meter-gap and moves the number being measured.
The thirty files sit in a SUBDIRECTORY on purpose: `test_tariff.py` part 9 and
`test_peakguard.py` part 6 glob `scenarios/p1-net-*.jsonl.gz` non-recursively,
so CI keeps replaying the two small days and does not grow ~50 s per PR. Both
files' honesty notes are corrected to point at the month.
**OPT-04's AC4 is met: twelve months of real EPEX BE day-ahead prices are now in
the repository, and the strategy has been replayed against them (DATA-02).**
AC4 asked for "net-positive across 12 months of replayed prices". OPT-04 shipped
goodwe_controller/scenarios/p1-net-30d-2026-08/README.md
+167
View File
@@ -0,0 +1,167 @@
# 30 days of P1 net power — 2026-08-01 … 2026-08-30
`sensor.p1_meter_active_power` out of Home Assistant's recorder, one gzipped
JSONL file per **local** day (Europe/Brussels), same schema as the two files one
directory up:
{"t": "2026-08-22T22:00:00+00:00", "w": -25.0}
**485,841 samples, 30/30 days read, 0 days failed, 4.13 MB gzipped.**
Produced by `tools/export_p1_history.py`; `manifest.json` here carries the
per-day counts, sha256 of every file, and every gap.
python3 tools/export_p1_history.py --days 30 --end-date 2026-08-30 \
--out-dir scenarios/p1-net-30d-2026-08
## Why this exists
[[TARIFF-01]] AC 3 and [[SAFETY-07]] AC 3 both ask for *"monthly peak held under
the soft limit in >= 95 % of intervals over a 30-day simulation replay"*, and
both were merged with the criterion honestly unmet because only two days of
capture existed. `test_tariff.py`'s docstring still recorded the reason as
"nothing longer was ever captured" — which was true about the past and was
quietly read as a statement about the future. It was not one. Probed on
2026-08-31, HA's recorder answered a 20-minute window with 230 samples at 30
days back and 0 at 45, so the whole month was still there and was exported the
same day. **The window rolls: 2026-08-01 leaves the recorder around 2026-09-15.**
## The size decision — COMMITTED, deliberately, at 4.13 MB
Three options were on the table. This one was taken and the cost is stated
rather than absorbed, and it is measured rather than estimated - a
`git bundle` of the whole history before and after this commit:
release/1.0 @ 86c43eb 1.58 MB
DATA-01 5.28 MB +3.70 MB, x3.3
A full clone of this repository therefore more than triples, and a Home
Assistant supervisor clones it on every add-on install and update. That is the
price; the paragraphs below are why it was judged worth paying rather than
avoided by making the data worse or making it disappear.
* **Downsampling is not available.** It is not a matter of taste: the shipped
`meter_max_age_s` is **30 s**, which is the age past which the controller's
own ingest stops counting a held reading as coverage. Any resample coarser
than 30 s therefore converts real coverage into modelled meter-gap and changes
the very number AC 3 measures. Resampling at or under 30 s saves almost
nothing against a 5 s cadence.
* **"Outside the repo with a pointer" is what already happened, and it is why
this card exists.** The two committed days came from the infra repo's private
`sim/scenarios/`, the one-off export script was thrown away, and three cards
then deferred an acceptance criterion on the belief that the data could not be
produced again. `test_tariff.py` states the general rule verbatim: *a
regression fixture CI cannot see is not a regression fixture.* CI's token is
minted for this repository only.
* **So: committed** — but in a SUBDIRECTORY, which is load-bearing. Both
`test_tariff.py` part 9 and `test_peakguard.py` part 6 do
`(HERE / "scenarios").glob("p1-net-*.jsonl.gz")`, which does not recurse. Drop
these thirty files beside the two and CI grows sixty full-day
replays - about 50 s of pure compute - on every PR forever, `test_tariff.py`'s
`check(... replay_days < 30)` starts failing, and both files' honesty notes
become false. In `p1-net-30d-2026-08/` they are visible to CI, invisible to
the globs, and the two committed days keep their job as the small fixtures the
suites grade against.
They are also written once and never edited, so git stores one already-deflated
blob per day with no delta churn.
## Gaps are visible, never interpolated
Nothing is filled, carried forward or resampled. A hole in the recorder is a
hole here — consecutive `t` values further apart than the cadence — which is
exactly what the controller's freshness rule already reads as *no coverage*, so
a missing hour replays as a missing hour rather than as an hour of whatever the
last sample said.
**13 gaps over 30 s, 1291 s in total: 99.950 % of the month is covered.**
All of them are listed in `manifest.json` and by
`python3 tools/replay_ac3.py <dir> --report-gaps`. The longest is 263 s on
2026-08-11.
Three samples in the month carried the state `unavailable` rather than a number
(2026-08-18 02:03:54Z and 04:10:41Z, 2026-08-25 01:52:02Z). There is no float to
write and the committed schema has no slot for a non-number, so they are absent
from the series and recorded individually in `manifest.json` under
`non_numeric_states`. Each one sits inside a gap that is already listed, which
is the honest rendering: the meter was not reporting a power then.
**A failed read is never an empty hour.** If a day's read fails, the exporter
writes NO FILE, records `"read": "FAILED"` with the error, and exits non-zero —
so a missing day is unreadable-and-known rather than silently zero. The entity
is also proved to exist through `/api/states` before any day is trusted, because
a typo would otherwise answer `200 []` thirty times and produce a beautifully
consistent dataset of nothing. On this run: 30 read, 0 failed, 0 empty.
## AC 3, measured
cd goodwe_controller
python3 tools/replay_ac3.py scenarios/p1-net-30d-2026-08
485841 samples, 2879 closed quarters (30.0 days' worth)
month 2026-08: peak 5029.8 W uncontrolled -> 4107.3 W guarded
TARIFF-01 AC 3 (UNCONTROLLED): 2847/2879 = 98.89 % of quarters at or under 3500 W
SAFETY-07 AC 3 (GUARDED): 2850/2879 = 98.99 %
guard cycles 143810, engaged on the measurement 1542, blind 195,
saturated 0, modelled discharge 8060 Wh
2879 rather than 2880 quarters: the last quarter of 2026-08-30 closes only when
a telegram past its boundary arrives, and that telegram is 2026-08-31's.
### What the numbers actually say — three findings, not a pass
1. **AC 3 is met on the ">= 95 % of intervals" reading, and the criterion turns
out to be nearly insensitive.** The UNCONTROLLED month — no controller, no
peak shaving, nothing running — already scores 98.89 %. A criterion that a
house passes by doing nothing does not discriminate between a working guard
and an absent one. That is a finding about the acceptance criterion, and it
is reported rather than filed as a win.
2. **On the other reading of the same sentence, AC 3 is NOT met.** "Monthly peak
held under the soft limit" taken as *the peak itself* fails: the guard pulls
August's peak from **5029.8 W to 4107.3 W** (-922.5 W, -18.3 %), which is
real money on a Belgian capacity tariff, but 4107.3 W is still over the
3500 W soft limit. Both readings are printed by the tool; neither is hidden.
3. **The guard stops defending the soft limit once the month's peak is set, and
that is by design.** `quarter_ceiling_w` uses `CEILING_MONTH_PEAK`: the
billed quantity is the month's maximum, so a quarter under an
already-recorded peak costs nothing extra and the guard does not spend the
battery on it. The month's 32 over-cap quarters fall on four nights
(2026-08-17, -18, -20, -21); **31 of them are pulled down** — 7494 W of
quarter-average removed in total, 8060 Wh of modelled discharge — but only
**3 land under 3500 W**, because by then the ceiling being defended is the
running month peak (~4107 W), not the soft limit. The single untouched
quarter, 2026-08-18T08:15Z at 4039.7 W, is the same rule seen from the other
side: the running peak was already 4833 W, so 4039.7 W added nothing to the
bill and the guard correctly stayed out. AC 3's wording asks the guard for
something its own tariff logic deliberately does not optimise.
Also worth an eye: **195 blind engagements** over 143,810 cycles — the guard
acting on a projection it could not fully cover. On the two committed days that
count was 5. It scales with the gaps above.
## Verification of the exporter
`tools/export_p1_history.py --verify` re-fetches a committed day by the exact
route used here and diffs it against the file:
python3 tools/export_p1_history.py \
--verify scenarios/p1-net-2026-08-20.jsonl.gz \
scenarios/p1-net-2026-08-23.jsonl.gz
Both days come back **byte-identical after decompression** — 865,024 B /
15,600 samples and 894,387 B / 16,351 samples — and the copies of those two days
inside this directory are byte-identical to the committed ones too. The gzip
CONTAINERS differ by design: a gzip header carries an mtime and a stored
filename, and this exporter writes mtime 0 so its own output is reproducible,
where the 2026-08-25 one-off wrote a wall-clock mtime. Content equality is the
claim; container equality would only assert that two files were written at the
same second.
`tools/replay_ac3.py` is checked the same way rather than trusted: pointed at
`scenarios/` it must reproduce `test_peakguard.py` part 6's published totals,
and `--expect` makes it a hard check.
python3 tools/replay_ac3.py scenarios --expect 189/190
reproduces 189/190 quarters, the 5029.8 W billed quarter, the 3485.3 W guarded
result and the 878 Wh of modelled discharge that part 6 prints.
goodwe_controller/scenarios/p1-net-30d-2026-08/manifest.json
+767
View File
@@ -0,0 +1,767 @@
{
"generated": "2026-08-31T08:42:40.614548+00:00",
"generator": "tools/export_p1_history.py",
"source": "http://192.168.2.158:8123/api/history/period",
"entity": "sensor.p1_meter_active_power",
"timezone": "Europe/Brussels",
"gap_threshold_s": 30.0,
"format": "{\"t\": <ISO-8601 UTC>, \"w\": <float watts>} per line, gzip",
"days": [
{
"date": "2026-08-01",
"window": [
"2026-08-01T00:00:00+02:00",
"2026-08-02T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1128060,
"attempts": 1,
"samples": 16511,
"non_numeric_states": [],
"first": "2026-07-31T22:00:00+00:00",
"last": "2026-08-01T21:59:56.268499+00:00",
"gaps": [],
"covered_span_s": 86396.3,
"mean_cadence_s": 5.233,
"file": "p1-net-2026-08-01.jsonl.gz",
"bytes_gz": 140531,
"bytes_raw": 913203,
"sha256_raw": "120de2a0b7eb552281dac4bc1f8fd76d947ba72d1f5b426d9dcdd85ec103bd06",
"sha256_gz": "e09bdee349dd3668a073b7fd77974e639c269ec37e73e81a9706cadde083f064"
},
{
"date": "2026-08-02",
"window": [
"2026-08-02T00:00:00+02:00",
"2026-08-03T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1115387,
"attempts": 1,
"samples": 16303,
"non_numeric_states": [],
"first": "2026-08-01T22:00:00+00:00",
"last": "2026-08-02T21:59:55.191360+00:00",
"gaps": [],
"covered_span_s": 86395.2,
"mean_cadence_s": 5.3,
"file": "p1-net-2026-08-02.jsonl.gz",
"bytes_gz": 141404,
"bytes_raw": 903234,
"sha256_raw": "9ab4891c0abf9d621fa528848fc3a874687843804768e8aeab6a88f7bb04b679",
"sha256_gz": "94774139b3e19d04bcda3ac5ea988dd4d645e99004f90b917c6b0bf3fbac21cb"
},
{
"date": "2026-08-03",
"window": [
"2026-08-03T00:00:00+02:00",
"2026-08-04T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1123668,
"attempts": 1,
"samples": 16424,
"non_numeric_states": [],
"first": "2026-08-02T22:00:00+00:00",
"last": "2026-08-03T21:59:55.255808+00:00",
"gaps": [],
"covered_span_s": 86395.3,
"mean_cadence_s": 5.261,
"file": "p1-net-2026-08-03.jsonl.gz",
"bytes_gz": 144202,
"bytes_raw": 909942,
"sha256_raw": "13f8dd904a31b5b67fcfde04f6afc49c34c3a9504617278d79d87516fcd6441f",
"sha256_gz": "e1bc0cb0e271cbc8d896aee97aa0f5c18e49e78905d1aaf137904a768047c7bc"
},
{
"date": "2026-08-04",
"window": [
"2026-08-04T00:00:00+02:00",
"2026-08-05T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1108582,
"attempts": 1,
"samples": 16286,
"non_numeric_states": [],
"first": "2026-08-03T22:00:00+00:00",
"last": "2026-08-04T21:59:58.729138+00:00",
"gaps": [],
"covered_span_s": 86398.7,
"mean_cadence_s": 5.305,
"file": "p1-net-2026-08-04.jsonl.gz",
"bytes_gz": 136505,
"bytes_raw": 896650,
"sha256_raw": "a65cd9d1ceda61e5a40043edb401c2154feea2b36bd3221710df808d778eabce",
"sha256_gz": "c19b19ac890f4caad4140cb27d2fe6f3e972545f664127e5d9f457e711ad8c88"
},
{
"date": "2026-08-05",
"window": [
"2026-08-05T00:00:00+02:00",
"2026-08-06T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1136556,
"attempts": 1,
"samples": 16655,
"non_numeric_states": [],
"first": "2026-08-04T22:00:00+00:00",
"last": "2026-08-05T21:59:56.189152+00:00",
"gaps": [],
"covered_span_s": 86396.2,
"mean_cadence_s": 5.188,
"file": "p1-net-2026-08-05.jsonl.gz",
"bytes_gz": 142561,
"bytes_raw": 919827,
"sha256_raw": "dd327685c6f322bfaa49bc5bf228301e790e5a96fd7c23c487b845e44a962db4",
"sha256_gz": "5ff7a1336c49ba38b8058820181dd8ed9c2b8db41b5a3a52412a5a3d8c515332"
},
{
"date": "2026-08-06",
"window": [
"2026-08-06T00:00:00+02:00",
"2026-08-07T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1124419,
"attempts": 1,
"samples": 16412,
"non_numeric_states": [],
"first": "2026-08-05T22:00:00+00:00",
"last": "2026-08-06T21:59:57.192711+00:00",
"gaps": [],
"covered_span_s": 86397.2,
"mean_cadence_s": 5.265,
"file": "p1-net-2026-08-06.jsonl.gz",
"bytes_gz": 142313,
"bytes_raw": 910849,
"sha256_raw": "7cd1fe8315e24dc6391693c49d6306b0c0cf9b6c3a478e457da921001cea930c",
"sha256_gz": "9108f169d645b9da07d4ae310666a1365ea07275ff311cd60adad1b702edbf11"
},
{
"date": "2026-08-07",
"window": [
"2026-08-07T00:00:00+02:00",
"2026-08-08T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1087531,
"attempts": 1,
"samples": 15952,
"non_numeric_states": [],
"first": "2026-08-06T22:00:00+00:00",
"last": "2026-08-07T21:59:58.195053+00:00",
"gaps": [],
"covered_span_s": 86398.2,
"mean_cadence_s": 5.416,
"file": "p1-net-2026-08-07.jsonl.gz",
"bytes_gz": 135979,
"bytes_raw": 879941,
"sha256_raw": "e0f67a9214980e2fd4893208cb402165065dadce2105ce1581c6b1d49952f39a",
"sha256_gz": "b70b84f3743d1d84055c911bcd5589cb8f0371e8916bde741308c5bb50702c29"
},
{
"date": "2026-08-08",
"window": [
"2026-08-08T00:00:00+02:00",
"2026-08-09T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1058569,
"attempts": 1,
"samples": 15468,
"non_numeric_states": [],
"first": "2026-08-07T22:00:00+00:00",
"last": "2026-08-08T21:59:52.805887+00:00",
"gaps": [],
"covered_span_s": 86392.8,
"mean_cadence_s": 5.586,
"file": "p1-net-2026-08-08.jsonl.gz",
"bytes_gz": 134025,
"bytes_raw": 857271,
"sha256_raw": "0be14ca7593b4c7d68570115cc26e24f6a26ab87fe3c168a8b942b167e8d4f68",
"sha256_gz": "d93dcc8ca3eae03af13b9ae36acd5ab1eaa1e7ee93c237d386e8ded1c61f5906"
},
{
"date": "2026-08-09",
"window": [
"2026-08-09T00:00:00+02:00",
"2026-08-10T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1059349,
"attempts": 1,
"samples": 15508,
"non_numeric_states": [],
"first": "2026-08-08T22:00:00+00:00",
"last": "2026-08-09T21:59:58.178998+00:00",
"gaps": [],
"covered_span_s": 86398.2,
"mean_cadence_s": 5.572,
"file": "p1-net-2026-08-09.jsonl.gz",
"bytes_gz": 135697,
"bytes_raw": 857531,
"sha256_raw": "f4793799173c502fcee5f78e2d9b0dcc984b5d2d8bd2fc0db0cd0f8d403733b3",
"sha256_gz": "2fea6c5c8b1452d72ad5aa3b0ae9755bb2ea08071514c08b1a3b1155ef457cf1"
},
{
"date": "2026-08-10",
"window": [
"2026-08-10T00:00:00+02:00",
"2026-08-11T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1066721,
"attempts": 1,
"samples": 15594,
"non_numeric_states": [],
"first": "2026-08-09T22:00:00+00:00",
"last": "2026-08-10T21:59:59.170056+00:00",
"gaps": [],
"covered_span_s": 86399.2,
"mean_cadence_s": 5.541,
"file": "p1-net-2026-08-10.jsonl.gz",
"bytes_gz": 136682,
"bytes_raw": 863785,
"sha256_raw": "ff5c3fa419505127a51acc56dfc13e8437e4ec2c13a11c8d8b2d6c3e20e2410e",
"sha256_gz": "4fd30e495f4e6d1c8501d05c23f9891ac34364d86a9c55ac825d52b2ebb31023"
},
{
"date": "2026-08-11",
"window": [
"2026-08-11T00:00:00+02:00",
"2026-08-12T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1056281,
"attempts": 1,
"samples": 15371,
"non_numeric_states": [],
"first": "2026-08-10T22:00:00+00:00",
"last": "2026-08-11T21:59:58.608150+00:00",
"gaps": [
{
"from": "2026-08-11T08:00:14.121653+00:00",
"to": "2026-08-11T08:02:54.427007+00:00",
"seconds": 160.305
},
{
"from": "2026-08-11T17:04:18.517592+00:00",
"to": "2026-08-11T17:08:41.759777+00:00",
"seconds": 263.242
}
],
"covered_span_s": 86398.6,
"mean_cadence_s": 5.621,
"file": "p1-net-2026-08-11.jsonl.gz",
"bytes_gz": 133711,
"bytes_raw": 856244,
"sha256_raw": "67c2a740019f8908405b96b3436cd953af27baead2957acc7c1dcf10b37ac94e",
"sha256_gz": "4f8c553edc9051644bd7baa659f6f92a8b5f5ce12a0410c8b5235d1e6c3084c9"
},
{
"date": "2026-08-12",
"window": [
"2026-08-12T00:00:00+02:00",
"2026-08-13T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1124945,
"attempts": 1,
"samples": 16432,
"non_numeric_states": [],
"first": "2026-08-11T22:00:00+00:00",
"last": "2026-08-12T21:59:58.648966+00:00",
"gaps": [],
"covered_span_s": 86398.6,
"mean_cadence_s": 5.258,
"file": "p1-net-2026-08-12.jsonl.gz",
"bytes_gz": 136903,
"bytes_raw": 911115,
"sha256_raw": "73c8a1626640226ddffb32d564557d1cbe57a15367ea1ba3632934e15a1413dc",
"sha256_gz": "acec87065b7de2c605db00504ec5a9b3c99584620dfb3b67194e2cf9464d6a5f"
},
{
"date": "2026-08-13",
"window": [
"2026-08-13T00:00:00+02:00",
"2026-08-14T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1126742,
"attempts": 1,
"samples": 16473,
"non_numeric_states": [],
"first": "2026-08-12T22:00:00+00:00",
"last": "2026-08-13T21:59:57.562043+00:00",
"gaps": [],
"covered_span_s": 86397.6,
"mean_cadence_s": 5.245,
"file": "p1-net-2026-08-13.jsonl.gz",
"bytes_gz": 136221,
"bytes_raw": 912379,
"sha256_raw": "1d9a16c529174bd8c74625a742c46d98de4acf7a045f83df8bdbac22fa86e8e7",
"sha256_gz": "adfef36f629decf54c625f2dbfe6b4dab9441218c42557c90ac7f6d331943a59"
},
{
"date": "2026-08-14",
"window": [
"2026-08-14T00:00:00+02:00",
"2026-08-15T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1124413,
"attempts": 1,
"samples": 16427,
"non_numeric_states": [],
"first": "2026-08-13T22:00:00+00:00",
"last": "2026-08-14T21:59:55.611487+00:00",
"gaps": [],
"covered_span_s": 86395.6,
"mean_cadence_s": 5.26,
"file": "p1-net-2026-08-14.jsonl.gz",
"bytes_gz": 135570,
"bytes_raw": 910648,
"sha256_raw": "dc1417881e54fe215e9fcd4ed643d2482a277d61f8f27f25e2ff1e2cba29f7a8",
"sha256_gz": "d4eff781cd16a83fd1be9e643fee5bb5d5897e7d45bb9865972a7304d92c84a4"
},
{
"date": "2026-08-15",
"window": [
"2026-08-15T00:00:00+02:00",
"2026-08-16T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1124461,
"attempts": 1,
"samples": 16516,
"non_numeric_states": [],
"first": "2026-08-14T22:00:00+00:00",
"last": "2026-08-15T21:59:56.581991+00:00",
"gaps": [],
"covered_span_s": 86396.6,
"mean_cadence_s": 5.231,
"file": "p1-net-2026-08-15.jsonl.gz",
"bytes_gz": 137670,
"bytes_raw": 909539,
"sha256_raw": "f07a242faed68313512e0363d914846522ff839315ad80b55689072ac74e8a69",
"sha256_gz": "082314c6003c2208752acc4614b9809f06735d01e216b935ea94faf578c5598e"
},
{
"date": "2026-08-16",
"window": [
"2026-08-16T00:00:00+02:00",
"2026-08-17T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1127871,
"attempts": 1,
"samples": 16552,
"non_numeric_states": [],
"first": "2026-08-15T22:00:00+00:00",
"last": "2026-08-16T21:59:59.527710+00:00",
"gaps": [],
"covered_span_s": 86399.5,
"mean_cadence_s": 5.22,
"file": "p1-net-2026-08-16.jsonl.gz",
"bytes_gz": 138347,
"bytes_raw": 912481,
"sha256_raw": "a44e5d961600a539ddf824d5d211ef319833666b7598e5939a6cdf2cb1e3581f",
"sha256_gz": "8f6511e52f5be9187f422021abc93a5fa8b698b4c785877d69d15dd6de106fd8"
},
{
"date": "2026-08-17",
"window": [
"2026-08-17T00:00:00+02:00",
"2026-08-18T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1133721,
"attempts": 1,
"samples": 16627,
"non_numeric_states": [],
"first": "2026-08-16T22:00:00+00:00",
"last": "2026-08-17T21:59:59.685699+00:00",
"gaps": [
{
"from": "2026-08-17T17:27:00.599126+00:00",
"to": "2026-08-17T17:28:20.389628+00:00",
"seconds": 79.791
}
],
"covered_span_s": 86399.7,
"mean_cadence_s": 5.197,
"file": "p1-net-2026-08-17.jsonl.gz",
"bytes_gz": 140214,
"bytes_raw": 917356,
"sha256_raw": "70cce79b3732f292f7f58f0b2655bdb4a00a20a6530f1f50dac62ef5f039788b",
"sha256_gz": "e2f9b461c4db7902e1a3d30b439ebc11ebcd9618a72d12f10e5131f38fb437bf"
},
{
"date": "2026-08-18",
"window": [
"2026-08-18T00:00:00+02:00",
"2026-08-19T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1074019,
"attempts": 1,
"samples": 15699,
"non_numeric_states": [
{
"t": "2026-08-18T02:03:54.417106+00:00",
"state": "unavailable"
},
{
"t": "2026-08-18T04:10:41.786299+00:00",
"state": "unavailable"
}
],
"first": "2026-08-17T22:00:00+00:00",
"last": "2026-08-18T21:59:54.501766+00:00",
"gaps": [
{
"from": "2026-08-18T02:02:59.633845+00:00",
"to": "2026-08-18T02:04:38.694366+00:00",
"seconds": 99.061
},
{
"from": "2026-08-18T04:09:54.292049+00:00",
"to": "2026-08-18T04:11:38.500238+00:00",
"seconds": 104.208
}
],
"covered_span_s": 86394.5,
"mean_cadence_s": 5.504,
"file": "p1-net-2026-08-18.jsonl.gz",
"bytes_gz": 135894,
"bytes_raw": 869570,
"sha256_raw": "68cd44ac56099ee38f0bba4c32d7d17d33b92d28a99d27419f53c22374491ca6",
"sha256_gz": "d6c97508f805aff040d2b4ed4e06c0e7373db2ea6227c7d79ef24c8c79074115"
},
{
"date": "2026-08-19",
"window": [
"2026-08-19T00:00:00+02:00",
"2026-08-20T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 989720,
"attempts": 1,
"samples": 14549,
"non_numeric_states": [],
"first": "2026-08-18T22:00:00+00:00",
"last": "2026-08-19T21:59:56.556610+00:00",
"gaps": [
{
"from": "2026-08-19T01:33:29.599224+00:00",
"to": "2026-08-19T01:34:01.653383+00:00",
"seconds": 32.054
}
],
"covered_span_s": 86396.6,
"mean_cadence_s": 5.939,
"file": "p1-net-2026-08-19.jsonl.gz",
"bytes_gz": 126345,
"bytes_raw": 800369,
"sha256_raw": "0f11ce0cee43cbd7bdb08dc48b202e58c442157ac08b2cda5b64bbab9bd56300",
"sha256_gz": "13d61b7da903877c8ae7813b833cb5836ec04a1cf1efd0e1fa87e916bef95fd5"
},
{
"date": "2026-08-20",
"window": [
"2026-08-20T00:00:00+02:00",
"2026-08-21T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1068038,
"attempts": 1,
"samples": 15600,
"non_numeric_states": [],
"first": "2026-08-19T22:00:00+00:00",
"last": "2026-08-20T21:59:56.742604+00:00",
"gaps": [
{
"from": "2026-08-20T02:11:56.607373+00:00",
"to": "2026-08-20T02:12:38.840458+00:00",
"seconds": 42.233
},
{
"from": "2026-08-20T20:42:04.623102+00:00",
"to": "2026-08-20T20:43:41.666543+00:00",
"seconds": 97.043
}
],
"covered_span_s": 86396.7,
"mean_cadence_s": 5.539,
"file": "p1-net-2026-08-20.jsonl.gz",
"bytes_gz": 139693,
"bytes_raw": 865024,
"sha256_raw": "2612153963951dd13fe310ad7fbb5bec2e195db56fd99fd963134c8badae25ba",
"sha256_gz": "362f2ee3d9d7f23f9051ba78993c5af7443dabb3b5f16339efc673444720c892"
},
{
"date": "2026-08-21",
"window": [
"2026-08-21T00:00:00+02:00",
"2026-08-22T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1118002,
"attempts": 1,
"samples": 16383,
"non_numeric_states": [],
"first": "2026-08-20T22:00:00+00:00",
"last": "2026-08-21T21:59:57.771588+00:00",
"gaps": [],
"covered_span_s": 86397.8,
"mean_cadence_s": 5.274,
"file": "p1-net-2026-08-21.jsonl.gz",
"bytes_gz": 139105,
"bytes_raw": 904809,
"sha256_raw": "933d9a6899f59c128d0d9b84d6944cea63b1ad58dc57ac52e3e476f9c616f07b",
"sha256_gz": "a563581a9a3338af173abb99a200db32881ddaf893d2458f5d8de59b5ec6041f"
},
{
"date": "2026-08-22",
"window": [
"2026-08-22T00:00:00+02:00",
"2026-08-23T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1129094,
"attempts": 1,
"samples": 16520,
"non_numeric_states": [],
"first": "2026-08-21T22:00:00+00:00",
"last": "2026-08-22T21:59:56.913786+00:00",
"gaps": [
{
"from": "2026-08-21T23:05:20.035692+00:00",
"to": "2026-08-21T23:07:51.147719+00:00",
"seconds": 151.112
},
{
"from": "2026-08-21T23:07:56.966604+00:00",
"to": "2026-08-21T23:08:27.695101+00:00",
"seconds": 30.728
}
],
"covered_span_s": 86396.9,
"mean_cadence_s": 5.23,
"file": "p1-net-2026-08-22.jsonl.gz",
"bytes_gz": 141564,
"bytes_raw": 914120,
"sha256_raw": "39ccdf67c806344490f791648822cd2cd14d73ea7cc0b6e8572a1e15e8f74641",
"sha256_gz": "5c46f1879a723f9986d521124200cab839af2a4722b76bc97c6f6f59d0df65a2"
},
{
"date": "2026-08-23",
"window": [
"2026-08-23T00:00:00+02:00",
"2026-08-24T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1107164,
"attempts": 1,
"samples": 16351,
"non_numeric_states": [],
"first": "2026-08-22T22:00:00+00:00",
"last": "2026-08-23T21:59:56.921115+00:00",
"gaps": [],
"covered_span_s": 86396.9,
"mean_cadence_s": 5.284,
"file": "p1-net-2026-08-23.jsonl.gz",
"bytes_gz": 136277,
"bytes_raw": 894387,
"sha256_raw": "ffb15367bf3c98eff8a5600e4858ac99778b597c03d0a4f6154e6971b8f98a44",
"sha256_gz": "3fa76981758a468ba0aa68b50fe68d5bdb1ef38cd7faec2b6ffdb61c9ddb2db2"
},
{
"date": "2026-08-24",
"window": [
"2026-08-24T00:00:00+02:00",
"2026-08-25T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1122175,
"attempts": 1,
"samples": 16584,
"non_numeric_states": [],
"first": "2026-08-23T22:00:00+00:00",
"last": "2026-08-24T21:59:57.940474+00:00",
"gaps": [],
"covered_span_s": 86397.9,
"mean_cadence_s": 5.21,
"file": "p1-net-2026-08-24.jsonl.gz",
"bytes_gz": 136834,
"bytes_raw": 906369,
"sha256_raw": "30465d5ff51220201b4a79341e5da7bfc3d88290465b1c9d886ba94eee8dd234",
"sha256_gz": "5103088e047bfcbd9b3f55521f431b01cd27de10c2642e00281a13faa762bdd9"
},
{
"date": "2026-08-25",
"window": [
"2026-08-25T00:00:00+02:00",
"2026-08-26T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1110567,
"attempts": 1,
"samples": 16451,
"non_numeric_states": [
{
"t": "2026-08-25T01:52:02.104225+00:00",
"state": "unavailable"
}
],
"first": "2026-08-24T22:00:00+00:00",
"last": "2026-08-25T21:59:56.883744+00:00",
"gaps": [
{
"from": "2026-08-25T01:51:25.942719+00:00",
"to": "2026-08-25T01:52:53.203963+00:00",
"seconds": 87.261
}
],
"covered_span_s": 86396.9,
"mean_cadence_s": 5.252,
"file": "p1-net-2026-08-25.jsonl.gz",
"bytes_gz": 135890,
"bytes_raw": 896416,
"sha256_raw": "0e7c86b2b74f2108bc7f5f5d2de2627caaafe473ca4ae18515534dff10526e26",
"sha256_gz": "2a5411c10d8d3b271f133c62c04bd722279b48b08d7376084fbad6cc6dd66d76"
},
{
"date": "2026-08-26",
"window": [
"2026-08-26T00:00:00+02:00",
"2026-08-27T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1123610,
"attempts": 1,
"samples": 16542,
"non_numeric_states": [],
"first": "2026-08-25T22:00:00+00:00",
"last": "2026-08-26T21:59:58.915457+00:00",
"gaps": [],
"covered_span_s": 86398.9,
"mean_cadence_s": 5.223,
"file": "p1-net-2026-08-26.jsonl.gz",
"bytes_gz": 139301,
"bytes_raw": 908350,
"sha256_raw": "88d72c1b9f51707d62e744f40402de4260a04617875b85c65d2df6079c6c84ca",
"sha256_gz": "0661588fbc0c6c42825b67d483c8818c070405c1002424b3f1d61d8fdadb6370"
},
{
"date": "2026-08-27",
"window": [
"2026-08-27T00:00:00+02:00",
"2026-08-28T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1122433,
"attempts": 1,
"samples": 16573,
"non_numeric_states": [],
"first": "2026-08-26T22:00:00+00:00",
"last": "2026-08-27T21:59:58.538446+00:00",
"gaps": [
{
"from": "2026-08-27T18:50:29.030586+00:00",
"to": "2026-08-27T18:52:22.938470+00:00",
"seconds": 113.908
}
],
"covered_span_s": 86398.5,
"mean_cadence_s": 5.214,
"file": "p1-net-2026-08-27.jsonl.gz",
"bytes_gz": 138009,
"bytes_raw": 906770,
"sha256_raw": "528ab8ee10a46275d2e91d56ef2ed847e5932fae76a1752e6f2e9828dbfa9865",
"sha256_gz": "5cacc8c5788f43d6e6e2f569445fa82d9edc5a63b1f082f9a1d6a3bb0b404260"
},
{
"date": "2026-08-28",
"window": [
"2026-08-28T00:00:00+02:00",
"2026-08-29T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1114796,
"attempts": 1,
"samples": 16456,
"non_numeric_states": [],
"first": "2026-08-27T22:00:00+00:00",
"last": "2026-08-28T21:59:58.515555+00:00",
"gaps": [],
"covered_span_s": 86398.5,
"mean_cadence_s": 5.251,
"file": "p1-net-2026-08-28.jsonl.gz",
"bytes_gz": 135917,
"bytes_raw": 900654,
"sha256_raw": "f89838bc9f61af074c17817e4a6eab037318f38fa3c65dad3edfa2106abbc9fb",
"sha256_gz": "97090b605f67dcc1f84dea2c51d4832dfe1c2af497a4f4cfe50ecb51024d9b4c"
},
{
"date": "2026-08-29",
"window": [
"2026-08-29T00:00:00+02:00",
"2026-08-30T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1092983,
"attempts": 1,
"samples": 16132,
"non_numeric_states": [],
"first": "2026-08-28T22:00:00+00:00",
"last": "2026-08-29T21:59:57.574524+00:00",
"gaps": [
{
"from": "2026-08-29T00:03:20.539844+00:00",
"to": "2026-08-29T00:03:50.569581+00:00",
"seconds": 30.03
}
],
"covered_span_s": 86397.6,
"mean_cadence_s": 5.356,
"file": "p1-net-2026-08-29.jsonl.gz",
"bytes_gz": 134210,
"bytes_raw": 883053,
"sha256_raw": "896d4c9d660be4faabc5a088323e246e257877d5ff6bd0ec084b785480d76e12",
"sha256_gz": "b6f264cd3afba63209123a27490e7e1c3ae70a86731d7ea05ae67f88917f136f"
},
{
"date": "2026-08-30",
"window": [
"2026-08-30T00:00:00+02:00",
"2026-08-31T00:00:00+02:00"
],
"read": "OK",
"http_bytes": 1118895,
"attempts": 1,
"samples": 16490,
"non_numeric_states": [],
"first": "2026-08-29T22:00:00+00:00",
"last": "2026-08-30T21:59:58.533323+00:00",
"gaps": [],
"covered_span_s": 86398.5,
"mean_cadence_s": 5.24,
"file": "p1-net-2026-08-30.jsonl.gz",
"bytes_gz": 138429,
"bytes_raw": 904311,
"sha256_raw": "e704818efb58bb6c505791bc9eab2e9476bf48ff8911d0e79b401fd35b529fa0",
"sha256_gz": "9052fead2418bff3083bec3ae8ce64d6b9f6224b668f4bb510c79775f66c5b79"
}
],
"summary": {
"days_requested": 30,
"days_read": 30,
"days_failed": 0,
"samples": 485841,
"bytes_gz": 4126003,
"empty_days": []
}
}
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-01.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-02.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-03.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-04.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-05.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-06.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-07.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-08.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-09.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-10.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-11.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-12.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-13.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-14.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-15.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-16.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-17.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-18.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-19.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-20.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-21.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-22.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-23.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-24.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-25.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-26.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-27.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-28.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-29.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-30.jsonl.gz
BIN
View File
Binary file not shown.
goodwe_controller/test_peakguard.py
+10 -3
View File
@@ -810,9 +810,16 @@ print()
print(f" BASELINE {totals['u_under']}/{totals['u_q']} = {U_PCT:.1f} % of "
f"quarters under {CAP_W:.0f} W, uncontrolled")
print(f" GUARDED {totals['g_under']}/{totals['g_q']} = {G_PCT:.1f} %")
print(f" Two days, not thirty. AC 3 is TARIFF-01's and is still not met - "
f"see test_tariff.py part 9. What this compares is the same quarters with "
f"and without the guard, which is what two days can support.")
print(f" Two days, not thirty - by choice: the thirty exist now "
f"(scenarios/p1-net-30d-2026-08/, DATA-01) and live one directory down so "
f"this glob does not pick them up and put ~50 s of replay on every PR. "
f"tools/replay_ac3.py runs the month and must reproduce the totals above "
f"to be believed: `python3 tools/replay_ac3.py scenarios --expect "
f"{totals['u_under']}/{totals['u_q']}`. Over the thirty days the guard "
f"takes August's peak from 5029.8 W to 4107.3 W - still over the "
f"{CAP_W:.0f} W soft limit - and moves the interval count from 98.89 % to "
f"98.99 %, because once a month peak is set the ceiling it defends is "
f"that peak and not the cap. See that directory's README.")
check("the guard improves on the uncontrolled baseline rather than merely "
"matching it", G_PCT > U_PCT)
check("...and it does not degrade a single quarter anywhere in the two days",
goodwe_controller/test_tariff.py
+38 -13
View File
@@ -35,16 +35,39 @@ NOT allowed to be tested at:
10. THE WIRING, through the real `Controller` and its real status page.
------------------------------------------------------------------------------
⚠️ AC 3 IS NOT MET, AND PART 9 DOES NOT PRETEND OTHERWISE.
⚠️ PART 9 IS A TWO-DAY MEASUREMENT AND SAYS SO. THE 30-DAY ANSWER IS ELSEWHERE
(DATA-01), AND IT DOES NOT LAND WHERE ANYONE EXPECTED.
AC 3 asks for "the monthly peak held under the soft limit in >= 95 % of
intervals over a 30-day simulation replay". THERE IS NO 30-DAY DATASET. The
infra repo's `sim/scenarios/` holds exactly two days of this house's real meter
- 2026-08-20 (the Illusmart failure day) and 2026-08-23 (the healthy reference)
- exported on 2026-08-25 because they were about to age out of HA's recorder,
and nothing longer was ever captured. `sim/scenarios/README.md` lists the whole
inventory; the rest is RS485 bus frames, OTA witness logs and a 578-row rollback
poll, none of which carries grid power over calendar time.
intervals over a 30-day simulation replay". When this file was written there
were two days of this house's real meter - 2026-08-20 (the Illusmart failure
day) and 2026-08-23 (the healthy reference) - exported on 2026-08-25 because
they were about to age out of HA's recorder.
⚠️ AND THE SENTENCE THAT USED TO STAND HERE, "nothing longer was ever
captured", WAS TRUE ABOUT THE PAST AND WAS READ AS A CLAIM ABOUT THE FUTURE.
It was not one. HA's recorder still held the whole month; DATA-01 probed it on
2026-08-31 (230 samples in a 20-minute window at 30 days back, 0 at 45) and
exported thirty days the same day, into `scenarios/p1-net-30d-2026-08/` with
`tools/export_p1_history.py` - which is committed now precisely because the
2026-08-25 one-off was not, and its absence is what let the belief survive.
Over those thirty days the criterion measures 98.89 % UNCONTROLLED and 98.99 %
guarded, so it is met - and that is a finding about the CRITERION as much as
about the guard, because a month with no controller at all passes it. Read the
other way - the monthly PEAK itself under the soft limit - it is not met: the
guard takes August's peak from 5029.8 W to 4107.3 W, still over 3500 W. Both
numbers, and why they part company, are in
`scenarios/p1-net-30d-2026-08/README.md`.
PART 9 IS DELIBERATELY LEFT GRADING THE TWO COMMITTED DAYS. It is a per-PR
regression check and the thirty days cost about fifty seconds of replay; they
also live one directory down precisely so this file's glob does not pick them
up. `tools/replay_ac3.py` is the thing that runs the month, and it is checked
against part 6 of test_peakguard.py rather than trusted:
python3 tools/replay_ac3.py scenarios --expect 189/190
python3 tools/replay_ac3.py scenarios/p1-net-30d-2026-08
So part 9 replays the two days that exist, reports the percentage over them, and
states the sample size in days next to it. It is a MEASUREMENT, not a verdict:
@@ -60,8 +83,8 @@ export) for the reason `scenarios/p1-age-2026-08-26.jsonl.gz` already is: the
infra repo is private, CI's token is minted for this repository only, and a
regression fixture that CI cannot see is not a regression fixture.
Closing AC 3 needs 30 days of capture. That is TEST-01's replay harness, and it
cannot start before somebody starts recording.
The recorder's window rolls: 2026-08-01 leaves it around 2026-09-15, which is
why the committed copy is the artefact and not the API call.
"""
import asyncio
@@ -748,9 +771,11 @@ print()
print(f" AC 3 MEASURED: {under}/{total_q} quarters = {PCT:.1f} % under the "
f"{SHIPPED_CAP_W:.0f} W soft limit, over {replay_days:.1f} DAYS of "
f"replay, UNCONTROLLED (no peak shaving was running on either day).")
print(f" AC 3 ASKS FOR 30 DAYS. {replay_days:.1f} days exist. "
f"AC 3 IS NOT MET and this run does not claim it is - see the module "
f"docstring. The figure above is the BASELINE SAFETY-07 has to improve.")
print(f" AC 3 ASKS FOR 30 DAYS and this run is {replay_days:.1f}, so it "
f"does not settle it either way. The 30 days DO now exist - "
f"scenarios/p1-net-30d-2026-08/, DATA-01 - and "
f"tools/replay_ac3.py is what runs them; the figure above is the "
f"BASELINE SAFETY-07 has to improve, on the two days CI replays.")
check("the replay produced a measurable number over a stated sample size, "
"rather than a criterion quietly redefined to fit the data",
total_q > 100 and replay_days < 30)
goodwe_controller/tools/export_p1_history.py
+455
View File
@@ -0,0 +1,455 @@
#!/usr/bin/env python3
"""Export a P1 power signal out of Home Assistant's recorder into the committed
scenario format - one gzipped JSON object per line, one file per LOCAL day.
WHY THIS FILE EXISTS AT ALL
===========================
`scenarios/p1-net-2026-08-20.jsonl.gz` and `p1-net-2026-08-23.jsonl.gz` were
produced once, by hand, and the script was thrown away. The consequence was not
theoretical: TARIFF-01's and SAFETY-07's AC 3 ("monthly peak held under the soft
limit in >= 95 % of intervals over a 30-day simulation replay") were both merged
UNMET on the belief that thirty days could not be produced retroactively - a
belief that was simply wrong, and that survived because nobody could re-run the
export to find out. So the export is a committed, runnable artefact now, and the
next person does not have to re-derive it under time pressure while a retention
window rolls.
python3 tools/export_p1_history.py --days 30 --out-dir /some/where
THE TWO RULES THIS FILE IS BUILT AROUND
=======================================
1. A FAILED READ IS NOT AN OBSERVATION. An HTTP error, a timeout, a truncated
body, a non-JSON body - none of those are "the meter reported nothing that
hour". They are "we do not know what the meter reported that hour", and the
two must never end up looking the same in the output. So:
* every request is retried with backoff;
* a day whose read ultimately fails writes NO FILE and is recorded in the
manifest as `"read": "FAILED"` with the error text, and the process exits
non-zero;
* a day that reads back successfully but EMPTY writes an empty file and is
recorded as `"read": "OK", "samples": 0`. Empty-and-known and
unknown are therefore distinguishable both in the manifest and by the
presence or absence of the file;
* before any day is fetched, the entity is proved to exist through
/api/states/<entity>. Without that check a typo in --entity would return
200 + `[]` for all thirty days and produce a beautifully consistent
dataset of nothing.
2. GAPS ARE VISIBLE, NEVER INTERPOLATED. Nothing here carries a value forward,
fills, resamples or averages. A recorder outage or an HA restart shows up as
what it is: consecutive `t` values further apart than the meter's cadence.
The controller's own ingest already reads that correctly - `meter_max_age_s`
(30 s shipped) is exactly the age past which a held reading stops counting as
coverage - so an hour with no telegrams replays as an hour of NO COVERAGE
rather than as an hour of whatever the last sample said. The manifest lists
every gap over --gap-s so a reader does not have to diff timestamps to find
them.
DAY AT A TIME, AND WHY NOT SMALLER
==================================
One request per local day. That is gentle on a live HA (~1 MB of JSON per day)
and it is also the only chunking that reproduces the committed files: the
history API prepends the state as it stood AT the period start
(`include_start_time_state`), stamped with the start instant. Splitting a day
into six-hour chunks would therefore inject three synthetic boundary samples per
day that no meter ever sent. Days are the natural boundary because the committed
files are local days.
VERIFYING THE EXPORTER
======================
python3 tools/export_p1_history.py --verify scenarios/p1-net-2026-08-20.jsonl.gz
re-fetches that day by this exact route and compares the DECOMPRESSED bytes with
the committed file. Byte equality of the .gz containers is not expected and not
required: a gzip header carries an mtime, and this file writes mtime 0 so its
own output is reproducible. Content equality is the claim that matters.
"""
from __future__ import annotations
import argparse
import gzip
import hashlib
import json
import sys
import time
import urllib.error
import urllib.parse
import urllib.request
from datetime import datetime, timedelta, timezone
from pathlib import Path
try: # py3.9+; the add-on image has it
from zoneinfo import ZoneInfo
except ImportError: # pragma: no cover
ZoneInfo = None # type: ignore
DEFAULT_URL = "http://192.168.2.158:8123"
DEFAULT_ENTITY = "sensor.p1_meter_active_power"
DEFAULT_TZ = "Europe/Brussels"
# The shipped meter_max_age_s. A hole longer than this is a hole the controller
# itself would refuse to see through, which makes it the honest gap threshold.
DEFAULT_GAP_S = 30.0
class ReadFailed(RuntimeError):
"""The read did not happen. NOT the same thing as an empty result."""
# ---------------------------------------------------------------------------
# transport
def _get(url: str, token: str, timeout: float):
req = urllib.request.Request(url, headers={
"Authorization": f"Bearer {token}",
"Content-Type": "application/json",
})
with urllib.request.urlopen(req, timeout=timeout) as resp:
if resp.status != 200:
raise ReadFailed(f"HTTP {resp.status}")
return resp.read()
def fetch_json(url: str, token: str, *, timeout: float, retries: int,
backoff: float, log=print):
"""GET and parse, or raise ReadFailed. Never returns a partial answer."""
last = None
for attempt in range(1, retries + 1):
try:
raw = _get(url, token, timeout)
except urllib.error.HTTPError as exc:
last = f"HTTP {exc.code} {exc.reason}"
except Exception as exc: # URLError, timeout, ...
last = f"{type(exc).__name__}: {exc}"
else:
try:
return json.loads(raw.decode("utf-8")), len(raw), attempt
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
# A truncated body looks exactly like this. It is a failure.
last = f"unparseable body ({len(raw)} bytes): {exc}"
if attempt < retries:
wait = backoff * attempt
log(f" read failed ({last}); retry {attempt + 1}/{retries} "
f"in {wait:.0f} s")
time.sleep(wait)
raise ReadFailed(last or "unknown")
def assert_entity_exists(base: str, entity: str, token: str, *, timeout: float,
retries: int, backoff: float, log=print):
"""Prove the entity is real BEFORE trusting any empty day.
Without this, `--entity sensor.typo` answers 200 + [] for every day and the
exporter cheerfully writes thirty empty files.
"""
url = f"{base}/api/states/{urllib.parse.quote(entity)}"
doc, _, _ = fetch_json(url, token, timeout=timeout, retries=retries,
backoff=backoff, log=log)
if not isinstance(doc, dict) or doc.get("entity_id") != entity:
raise ReadFailed(f"{entity} is not a known entity")
return doc
# ---------------------------------------------------------------------------
# one day
def day_bounds(day: datetime, tz):
start = datetime(day.year, day.month, day.day, tzinfo=tz)
return start, start + timedelta(days=1)
def history_url(base: str, entity: str, start: datetime, end: datetime) -> str:
q = urllib.parse.urlencode({
"filter_entity_id": entity,
"end_time": end.isoformat(),
"minimal_response": "",
})
return (f"{base}/api/history/period/"
f"{urllib.parse.quote(start.isoformat())}?{q}")
def rows_from(doc, entity: str):
"""HA's shape -> [(iso_t, state_str)], plus the states that are not numbers.
`minimal_response` returns the first record in full and the rest as
{state, last_changed}. An entity that went unavailable reports the string
"unavailable" (or "unknown") where a number belongs. Those are dropped from
the numeric series - there is no float to write and the committed schema has
no slot for a non-number - but every one is returned here so it lands in the
manifest. Dropping them is what MAKES the outage visible: the series simply
has no samples across it, which is the same thing the meter falling silent
looks like, and is what the controller's freshness rule already handles.
"""
if not isinstance(doc, list):
raise ReadFailed(f"expected a list, got {type(doc).__name__}")
if not doc:
return [], []
series = doc[0]
if not isinstance(series, list):
raise ReadFailed("history series is not a list")
good, bad = [], []
for rec in series:
eid = rec.get("entity_id")
if eid is not None and eid != entity:
raise ReadFailed(f"unexpected entity {eid!r} in the response")
t = rec.get("last_changed") or rec.get("last_updated")
s = rec.get("state")
if t is None:
raise ReadFailed("a record carries no timestamp")
try:
w = float(s)
except (TypeError, ValueError):
bad.append({"t": t, "state": s})
continue
good.append((t, w))
return good, bad
def render(rows, *, newline: str) -> bytes:
"""The committed format, exactly: `{"t": <iso>, "w": <float>}` per line.
json.dumps' default separators (", " / ": ") and repr of a float are what
the two committed files contain; so is the CRLF, an artefact of the original
one-off having been run on Windows in text mode. It is reproduced rather
than tidied because reproducing the committed bytes is this exporter's
verification, and gzip.open(..., 'rt') strips it either way.
"""
out = bytearray()
for t, w in rows:
out += json.dumps({"t": t, "w": w}).encode("utf-8")
out += newline.encode("ascii")
return bytes(out)
def gz(payload: bytes) -> bytes:
"""gzip level 9, mtime 0 - so the same input always gives the same bytes."""
import io
buf = io.BytesIO()
with gzip.GzipFile(fileobj=buf, mode="wb", compresslevel=9, mtime=0) as f:
f.write(payload)
return buf.getvalue()
def gaps_in(rows, gap_s: float):
out = []
prev = None
for t, _ in rows:
cur = datetime.fromisoformat(t)
if prev is not None:
d = (cur - prev).total_seconds()
if d > gap_s:
out.append({"from": prev.isoformat(), "to": cur.isoformat(),
"seconds": round(d, 3)})
prev = cur
return out
def export_day(base, entity, token, day, tz, *, timeout, retries, backoff,
gap_s, newline, log=print):
"""Read one local day. Raises ReadFailed rather than returning a guess."""
start, end = day_bounds(day, tz)
doc, nbytes, attempts = fetch_json(
history_url(base, entity, start, end), token,
timeout=timeout, retries=retries, backoff=backoff, log=log)
rows, bad = rows_from(doc, entity)
entry = {
"date": day.date().isoformat(),
"window": [start.isoformat(), end.isoformat()],
"read": "OK",
"http_bytes": nbytes,
"attempts": attempts,
"samples": len(rows),
"non_numeric_states": bad,
"first": rows[0][0] if rows else None,
"last": rows[-1][0] if rows else None,
"gaps": gaps_in(rows, gap_s),
}
if rows:
span = (datetime.fromisoformat(rows[-1][0])
- datetime.fromisoformat(rows[0][0])).total_seconds()
entry["covered_span_s"] = round(span, 1)
entry["mean_cadence_s"] = round(span / max(1, len(rows) - 1), 3)
return rows, entry
# ---------------------------------------------------------------------------
# modes
def do_export(args, token, tz, log=print):
base = args.url.rstrip("/")
out_dir = Path(args.out_dir)
out_dir.mkdir(parents=True, exist_ok=True)
newline = "\r\n" if args.newline == "crlf" else "\n"
log(f" proving {args.entity} exists before trusting any empty day")
st = assert_entity_exists(base, args.entity, token, timeout=args.timeout,
retries=args.retries, backoff=args.backoff,
log=log)
log(f" {args.entity} = {st.get('state')} "
f"{st.get('attributes', {}).get('unit_of_measurement', '')}")
if args.end_date:
last = datetime.fromisoformat(args.end_date).replace(tzinfo=tz)
else: # yesterday: today is still being logged
last = datetime.now(tz).replace(hour=0, minute=0, second=0,
microsecond=0) - timedelta(days=1)
days = [last - timedelta(days=i) for i in range(args.days - 1, -1, -1)]
manifest = {
"generated": datetime.now(timezone.utc).isoformat(),
"generator": "tools/export_p1_history.py",
"source": f"{base}/api/history/period",
"entity": args.entity,
"timezone": args.tz,
"gap_threshold_s": args.gap_s,
"format": '{"t": <ISO-8601 UTC>, "w": <float watts>} per line, gzip',
"days": [],
}
failures = []
for day in days:
name = f"{args.prefix}{day.date().isoformat()}.jsonl.gz"
log(f" {name}")
try:
rows, entry = export_day(
base, args.entity, token, day, tz, timeout=args.timeout,
retries=args.retries, backoff=args.backoff, gap_s=args.gap_s,
newline=newline, log=log)
except ReadFailed as exc:
# NO FILE. A missing file is the honest record of a read that did
# not happen; an empty one would claim the meter was silent.
entry = {"date": day.date().isoformat(), "read": "FAILED",
"error": str(exc), "file": None}
manifest["days"].append(entry)
failures.append(entry)
log(f" READ FAILED: {exc} - no file written")
continue
payload = render(rows, newline=newline)
blob = gz(payload)
(out_dir / name).write_bytes(blob)
entry["file"] = name
entry["bytes_gz"] = len(blob)
entry["bytes_raw"] = len(payload)
entry["sha256_raw"] = hashlib.sha256(payload).hexdigest()
entry["sha256_gz"] = hashlib.sha256(blob).hexdigest()
manifest["days"].append(entry)
ng = len(entry.get("gaps", []))
log(f" {entry['samples']} samples, {len(blob)} B gz, "
f"{ng} gap(s) over {args.gap_s:g} s"
+ (f", {len(entry['non_numeric_states'])} non-numeric state(s)"
if entry["non_numeric_states"] else ""))
if day is not days[-1]:
time.sleep(args.delay)
ok = [d for d in manifest["days"] if d["read"] == "OK"]
manifest["summary"] = {
"days_requested": len(days),
"days_read": len(ok),
"days_failed": len(failures),
"samples": sum(d["samples"] for d in ok),
"bytes_gz": sum(d["bytes_gz"] for d in ok),
"empty_days": [d["date"] for d in ok if d["samples"] == 0],
}
# newline="\n" explicitly: write_text() on Windows emits CRLF, and
# the manifest would then differ from a Linux run of the same export
# for no reason visible in a diff.
with open(args.manifest or (out_dir / "manifest.json"), "w",
encoding="utf-8", newline="\n") as fh:
fh.write(json.dumps(manifest, indent=2) + "\n")
log(f" {manifest['summary']['days_read']}/{len(days)} days, "
f"{manifest['summary']['samples']} samples, "
f"{manifest['summary']['bytes_gz'] / 1e6:.2f} MB gz")
if failures:
log(f" {len(failures)} DAY(S) NOT READ - the dataset is incomplete "
f"and says so: " + ", ".join(d["date"] for d in failures))
return 1
return 0
def do_verify(args, token, tz, log=print):
"""Re-export a committed day by this route and diff it against the file."""
base = args.url.rstrip("/")
newline = "\r\n" if args.newline == "crlf" else "\n"
rc = 0
for target in args.verify:
path = Path(target)
stamp = path.name.split(".")[0].rsplit("-", 3)[-3:]
day = datetime(int(stamp[0]), int(stamp[1]), int(stamp[2]), tzinfo=tz)
log(f" {path.name} (local day {day.date().isoformat()})")
committed = gzip.decompress(path.read_bytes())
rows, entry = export_day(base, args.entity, token, day, tz,
timeout=args.timeout, retries=args.retries,
backoff=args.backoff, gap_s=args.gap_s,
newline=newline, log=log)
fresh = render(rows, newline=newline)
if fresh == committed:
log(f" IDENTICAL: {len(fresh)} bytes, {entry['samples']} "
f"samples, sha256 {hashlib.sha256(fresh).hexdigest()[:16]}")
continue
rc = 1
cl = committed.split(newline.encode())
fl = fresh.split(newline.encode())
log(f" DIFFERS: committed {len(committed)} B / {len(cl)} lines, "
f"re-export {len(fresh)} B / {len(fl)} lines")
shown = 0
for i in range(max(len(cl), len(fl))):
a = cl[i] if i < len(cl) else b"<absent>"
b = fl[i] if i < len(fl) else b"<absent>"
if a != b:
log(f" line {i}: committed {a!r}")
log(f" re-export {b!r}")
shown += 1
if shown >= args.max_diff:
log(f" ... stopping after {shown} differing lines")
break
return rc
def main(argv=None):
p = argparse.ArgumentParser(description=__doc__.split("\n")[0])
p.add_argument("--url", default=DEFAULT_URL)
p.add_argument("--entity", default=DEFAULT_ENTITY)
p.add_argument("--token", help="bearer token; else $HASS_TOKEN")
p.add_argument("--tz", default=DEFAULT_TZ,
help="local timezone that defines a day boundary")
p.add_argument("--days", type=int, default=30)
p.add_argument("--end-date", help="last local day to export (YYYY-MM-DD); "
"default yesterday")
p.add_argument("--out-dir", default=".")
p.add_argument("--prefix", default="p1-net-")
p.add_argument("--manifest", help="path for the manifest JSON")
p.add_argument("--gap-s", type=float, default=DEFAULT_GAP_S)
p.add_argument("--newline", choices=("crlf", "lf"), default="crlf",
help="line terminator; crlf reproduces the committed files")
p.add_argument("--timeout", type=float, default=120.0)
p.add_argument("--retries", type=int, default=4)
p.add_argument("--backoff", type=float, default=5.0)
p.add_argument("--delay", type=float, default=2.0,
help="pause between days - this is a live HA")
p.add_argument("--verify", nargs="+", metavar="FILE",
help="re-export these committed days and diff them")
p.add_argument("--max-diff", type=int, default=20)
args = p.parse_args(argv)
token = args.token or __import__("os").environ.get("HASS_TOKEN")
if not token:
print("no token: pass --token or set HASS_TOKEN", file=sys.stderr)
return 2
if ZoneInfo is None:
print("zoneinfo unavailable", file=sys.stderr)
return 2
tz = ZoneInfo(args.tz)
try:
if args.verify:
return do_verify(args, token, tz)
return do_export(args, token, tz)
except ReadFailed as exc:
print(f"read failed: {exc}", file=sys.stderr)
return 1
if __name__ == "__main__":
raise SystemExit(main())
goodwe_controller/tools/replay_ac3.py
+303
View File
@@ -0,0 +1,303 @@
#!/usr/bin/env python3
"""Run TARIFF-01's and SAFETY-07's AC 3 over a directory of `p1-net-*.jsonl.gz`.
AC 3, both cards, same words: *"monthly peak held under the soft limit in
>= 95 % of intervals over a 30-day simulation replay."* Both cards merged with
it UNMET because two days of capture existed. DATA-01 exported thirty. This is
the thing that turns those thirty days into the number.
cd goodwe_controller
python3 tools/replay_ac3.py scenarios/p1-net-30d-2026-08
WHAT IT MEASURES
================
Two runs over the same telegrams, one file at a time in date order into a
SHARED peak store, so the 15-minute blocks accumulate into a real MONTH rather
than into 30 disconnected days:
UNCONTROLLED the recorded meter as it happened. Nothing was shaving peaks on
this house during August 2026, so this is not a simulation of
anything - it is what the month did. It is TARIFF-01's AC 3.
GUARDED the same month with `peakguard.decide()` in the loop and the
battery it asks for modelled, closed-loop: the meter the guard
sees is the recorded net MINUS the discharge it obtained. That
is SAFETY-07's AC 3.
Both numbers are `quarters <= peak_cap_w` over `quarters closed`, with the cap
read out of config.yaml rather than typed. Nothing here exits non-zero on 95 %:
a measurement below the criterion is a finding about the house and the guard,
and this tool's job is to report it, not to be graded by it.
FAITHFULNESS TO THE SUITE
=========================
The replay loop is `test_peakguard.py` part 6's, with `pack=None` (SoC pinned
above the floor, so SAFETY-24's energy floor is not the thing deciding the
answer). It is not imported from there because that file runs its whole suite on
import. The duplication is checked instead of asserted: pointed at `scenarios/`
- the two committed days - this tool must reproduce part 6's published totals,
and `--expect` makes that a hard check.
python3 tools/replay_ac3.py scenarios --expect 189/190
GAPS
====
Nothing is interpolated. A hole in the export is a hole here: the shipped
`meter_max_age_s` is what decides whether a held reading still counts as
coverage, so an hour with no telegrams closes no quarters and contributes to
neither numerator nor denominator. `--report-gaps` lists them.
"""
from __future__ import annotations
import argparse
import gzip
import json
import sys
import tempfile
from datetime import datetime, timedelta
from pathlib import Path
HERE = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(HERE))
from app.p1 import P1Ingest, make_sample, split_signed # noqa: E402
from app.peakguard import decide # noqa: E402
from app.tariff import PeakStore, PeakTracker, month_key # noqa: E402
def read_option(key: str):
"""The shipped default for `key`, out of config.yaml.
⚠️ `peak_cap_w:` appears TWICE in config.yaml - once under `options:` with
the default, once under `schema:` as `int(500,15000)`. The FIRST match is
the options block, which is the number an install actually lands on; taking
the last would try to float() the schema line. Same rule as the suites.
"""
for line in (HERE / "config.yaml").read_text(encoding="utf-8").splitlines():
if line.strip().startswith(f"{key}:"):
try:
return float(line.split(":", 1)[1].strip())
except ValueError:
continue
return None
def read_schema_max(key: str):
for line in (HERE / "config.yaml").read_text(encoding="utf-8").splitlines():
s = line.strip()
if s.startswith(f"{key}:") and "(" in s and ")" in s:
try:
return float(s[s.index("(") + 1:s.index(")")].split(",")[-1])
except ValueError:
continue
return None
def sample(t: datetime, net_w: float):
"""One telegram through the SHIPPED sign derivation - the suites' helper."""
imp, exp = split_signed(net_w)
return make_sample("ha_signed", imp, exp, phases=1,
ingest_ts=t, ingest_mono=t.timestamp())
def rows(path):
"""Every sample in one file, in order, with the gaps left as gaps."""
with gzip.open(path, "rt", encoding="utf-8") as fh:
for line in fh:
line = line.strip()
if not line:
continue
row = json.loads(line)
yield datetime.fromisoformat(row["t"]), float(row["w"])
def replay(paths, *, guarded: bool, cap_w, max_w, max_hold_s, cycle_s,
soc, soc_floor, tmp: Path, gap_s: float):
"""test_peakguard.py part 6's loop, over an arbitrary number of days.
⚠️ ONE STORE, AND AN INGEST THAT IS RESET ONLY AT A REAL DISCONTINUITY.
Neither half of that is a detail.
The STORE is shared across the files because "monthly peak" is what AC 3 is
about and a per-file store cannot express it.
The INGEST is carried across a file boundary when the next file's first
telegram follows the previous file's last one within `gap_s`, and rebuilt
when it does not. Both failure modes are real and were measured here:
* carried across a genuine hole, one averager manufactures quarters out of
nothing - streamed straight through the three-day gap between the two
committed days this reported 285/286 where the suite reports 189/190;
* rebuilt at every file, each day's LAST quarter (21:45-22:00 UTC) never
closes, because it closes only when a telegram past the boundary
arrives and that telegram is in the next file. That is why the suite's
two days give 95 quarters each rather than 96, and it silently discards
one quarter per day - 30 of them over a month.
So contiguous days are joined and non-contiguous ones are not, which gives
189/190 on the committed pair and a full 96 quarters a day on a run of
consecutive days.
"""
store = PeakStore(str(tmp / f"ac3-{'g' if guarded else 'u'}.db"))
tracker = PeakTracker(store, cap_w=cap_w)
engaged = blind = saturated = cycles = n = joins = 0
wh = 0.0
gaps = []
months = set()
ing = None
batt = 0.0 # delivered discharge, W (positive)
next_cycle = None
prev_t = None
for path in paths:
for i, (t, w) in enumerate(rows(path)):
if i == 0:
contiguous = (prev_t is not None
and 0 <= (t - prev_t).total_seconds() <= gap_s)
if contiguous:
joins += 1
else:
if ing is not None:
months |= {month_key(b.start) for b in ing.blocks}
ing = P1Ingest(phases=1, max_age_s=max_hold_s)
batt = 0.0
next_cycle = None
prev_t = None
if prev_t is not None:
dt = (t - prev_t).total_seconds()
if dt > gap_s:
gaps.append((prev_t, t, dt))
wh += batt * dt / 3600.0
prev_t = t
n += 1
ing.submit(sample(t, w - batt))
tracker.sync(ing)
if next_cycle is not None and t < next_cycle:
continue
next_cycle = t + timedelta(seconds=cycle_s)
cycles += 1
d = decide(PeakTracker.project(ing.averager, t), batt_w=batt,
cap_w=cap_w, reading=tracker.peak(t),
achievable_w=max_w, max_hold_s=max_hold_s, soc=soc,
soc_floor=soc_floor, frozen=False)
if d.engaged:
blind += 1 if d.blind else 0
engaged += 0 if d.blind else 1
saturated += 1 if d.saturated else 0
if not guarded:
continue
engaged_floor = d.engaged and d.floor_w is not None
batt = max(0.0, d.floor_w) if engaged_floor else 0.0
if ing is not None:
months |= {month_key(b.start) for b in ing.blocks}
out = {m: dict(store.blocks_in(m)) for m in sorted(months)}
return {"months": out, "samples": n, "cycles": cycles, "joins": joins,
"engaged": engaged, "blind": blind, "saturated": saturated,
"wh": wh, "gaps": gaps}
def main(argv=None):
p = argparse.ArgumentParser(description=__doc__.split("\n")[0])
p.add_argument("directory", help="a directory of p1-net-*.jsonl.gz")
p.add_argument("--glob", default="p1-net-*.jsonl.gz")
p.add_argument("--expect", help="UNDER/TOTAL the uncontrolled run must "
"reproduce, e.g. 189/190")
p.add_argument("--report-gaps", action="store_true")
args = p.parse_args(argv)
paths = sorted(Path(args.directory).glob(args.glob))
if not paths:
print(f"no {args.glob} under {args.directory}", file=sys.stderr)
return 2
CAP_W = read_option("peak_cap_w")
MAX_HOLD_S = read_option("meter_max_age_s")
CYCLE_S = read_option("cycle_floor_s")
SOC_FLOOR = read_option("peak_soc_floor")
LIVE_MAX_W = read_schema_max("max_w")
if None in (CAP_W, MAX_HOLD_S, CYCLE_S, SOC_FLOOR, LIVE_MAX_W):
print("could not read the shipped options out of config.yaml",
file=sys.stderr)
return 2
# The suites' fixture SoC: comfortably above the floor, so SAFETY-24's
# energy floor is not what decides an AC 3 that is not about it.
SOC_OK = SOC_FLOOR + 30.0
days = len(paths)
span_first = span_last = None
print(f"AC 3 - {days} file(s) from {args.directory}")
print(f" shipped: peak_cap_w {CAP_W:.0f} W, meter_max_age_s "
f"{MAX_HOLD_S:.0f} s, cycle_floor_s {CYCLE_S:.0f} s, "
f"max_w schema ceiling {LIVE_MAX_W:.0f} W")
tmp = Path(tempfile.mkdtemp(prefix="goodwe-ac3-"))
kw = dict(cap_w=CAP_W, max_w=LIVE_MAX_W, max_hold_s=MAX_HOLD_S,
cycle_s=CYCLE_S, soc=SOC_OK, soc_floor=SOC_FLOOR, tmp=tmp,
gap_s=MAX_HOLD_S)
u = replay(paths, guarded=False, **kw)
g = replay(paths, guarded=True, **kw)
def flat(run):
return [w for m in run["months"].values() for w in m.values()]
uq, gq = flat(u), flat(g)
u_under = sum(1 for w in uq if w <= CAP_W)
g_under = sum(1 for w in gq if w <= CAP_W)
u_pct = 100.0 * u_under / max(len(uq), 1)
g_pct = 100.0 * g_under / max(len(gq), 1)
print(f" {u['samples']} samples, {len(uq)} closed quarters "
f"({len(uq) / 96.0:.1f} days' worth), {days - u['joins']} contiguous "
f"run(s) of days")
for m in sorted(u["months"]):
mu = u["months"][m]
mg = g["months"].get(m, {})
print(f" month {m}: {len(mu)} quarters, peak "
f"{max(mu.values()):.1f} W uncontrolled -> "
f"{max(mg.values()) if mg else 0.0:.1f} W guarded")
print()
print(f" TARIFF-01 AC 3 (UNCONTROLLED, what the month did): "
f"{u_under}/{len(uq)} = {u_pct:.2f} % of quarters at or under "
f"{CAP_W:.0f} W")
print(f" SAFETY-07 AC 3 (GUARDED, peakguard.decide in the loop): "
f"{g_under}/{len(gq)} = {g_pct:.2f} %")
print(f" guard cycles {g['cycles']}, engaged on the measurement "
f"{g['engaged']}, blind {g['blind']}, saturated {g['saturated']}, "
f"modelled discharge {g['wh']:.0f} Wh")
# AC 3'S WORDS SUPPORT TWO READINGS AND BOTH ARE REPORTED, because they
# disagree on this data and picking the flattering one is the failure mode
# this whole card exists to correct.
u_peak = max(uq) if uq else 0.0
g_peak = max(gq) if gq else 0.0
print(f" reading A - PERCENTAGE OF INTERVALS under the cap (the reading "
f"test_tariff.py part 9 already uses): uncontrolled {u_pct:.2f} %, "
f"guarded {g_pct:.2f} %")
print(f" reading B - THE MONTHLY PEAK ITSELF under the cap: "
f"{u_peak:.1f} W uncontrolled -> {g_peak:.1f} W guarded, against a "
f"{CAP_W:.0f} W soft limit "
f"({'under' if g_peak <= CAP_W else 'STILL OVER'})")
print(f" verdict at the 95 % criterion: uncontrolled "
f"{'MEETS' if u_pct >= 95.0 else 'DOES NOT MEET'}, guarded "
f"{'MEETS' if g_pct >= 95.0 else 'DOES NOT MEET'} "
f"({len(uq) / 96.0:.1f} days of replay)")
if u["gaps"]:
tot = sum(d for _, _, d in u["gaps"])
print(f" gaps over {MAX_HOLD_S:.0f} s: {len(u['gaps'])}, "
f"{tot:.0f} s total - NOT interpolated; the quarters they fall in "
f"simply carry less coverage")
if args.report_gaps:
for a, b, d in u["gaps"]:
print(f" {a.isoformat()} -> {b.isoformat()} {d:.0f} s")
if args.expect:
want_u, want_t = (int(x) for x in args.expect.split("/"))
ok = (u_under, len(uq)) == (want_u, want_t)
print(f" reproduces test_peakguard.py part 6's published "
f"{want_u}/{want_t}: {'yes' if ok else 'NO'} "
f"(got {u_under}/{len(uq)})")
return 0 if ok else 1
return 0
if __name__ == "__main__":
raise SystemExit(main())