Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 glenn schrooyenandClaude Opus 5
|
04292c4a08 | ||
 admin
|
86c43eba24 |
@@ -2,6 +2,45 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
**Thirty days of the real meter, and the acceptance criterion three cards
|
||||
deferred is now a number (DATA-01).** TARIFF-01 and SAFETY-07 both merged with
|
||||
AC 3 - *"monthly peak held under the soft limit in >= 95 % of intervals over a
|
||||
30-day simulation replay"* - honestly unmet, on the belief that only two days of
|
||||
capture existed and nothing longer could be produced. That belief came from a
|
||||
true statement about the past being read as one about the future: HA's recorder
|
||||
still held the whole month. `scenarios/p1-net-30d-2026-08/` is 2026-08-01 to
|
||||
2026-08-30, **485,841 samples, 30/30 days read, 0 failed, 4.13 MB gzipped**, in
|
||||
the same schema as the two days that were already there.
|
||||
|
||||
`tools/export_p1_history.py` is the exporter, committed and runnable - the
|
||||
2026-08-25 one-off was not, and that is exactly why the data was thought
|
||||
unobtainable. It reads day-at-a-time, retries, and **never lets a failed read
|
||||
look like an empty hour**: a day whose read fails writes no file, is recorded as
|
||||
`"read": "FAILED"`, and the run exits non-zero; the entity is proved to exist
|
||||
before any empty day is believed. Nothing is interpolated, so the month's 13
|
||||
gaps (1291 s, 99.95 % covered) and its 3 `unavailable` samples stay visible.
|
||||
`--verify` re-fetches the two committed days by the same route and reproduces
|
||||
them **byte for byte** after decompression.
|
||||
|
||||
`tools/replay_ac3.py` runs the month. Over 30 days the criterion measures
|
||||
**98.89 % uncontrolled and 98.99 % guarded**, so it is met on that reading - and
|
||||
the uncontrolled figure is the finding: a month with no controller at all passes
|
||||
it. On the other reading of the same sentence it is not met - the guard takes
|
||||
August's peak from **5029.8 W to 4107.3 W**, still over the 3500 W soft limit,
|
||||
because once a month peak is set `CEILING_MONTH_PEAK` defends that peak rather
|
||||
than the cap. Neither number is hidden; both are in that directory's README.
|
||||
|
||||
The size decision is committed and measured, not absorbed: a `git bundle` of
|
||||
the whole history goes from **1.58 MB to 5.28 MB** (+3.70 MB, x3.3), which a
|
||||
Home Assistant supervisor clones on every install. Downsampling was not
|
||||
available - `meter_max_age_s` is 30 s, so any coarser resample converts real
|
||||
coverage into modelled meter-gap and moves the number being measured.
|
||||
|
||||
The thirty files sit in a SUBDIRECTORY on purpose: `test_tariff.py` part 9 and
|
||||
`test_peakguard.py` part 6 glob `scenarios/p1-net-*.jsonl.gz` non-recursively,
|
||||
so CI keeps replaying the two small days and does not grow ~50 s per PR. Both
|
||||
files' honesty notes are corrected to point at the month.
|
||||
|
||||
**OPT-04's AC4 is met: twelve months of real EPEX BE day-ahead prices are now in
|
||||
the repository, and the strategy has been replayed against them (DATA-02).**
|
||||
AC4 asked for "net-positive across 12 months of replayed prices". OPT-04 shipped
|
||||
|
||||
@@ -0,0 +1,167 @@
|
||||
# 30 days of P1 net power — 2026-08-01 … 2026-08-30
|
||||
|
||||
`sensor.p1_meter_active_power` out of Home Assistant's recorder, one gzipped
|
||||
JSONL file per **local** day (Europe/Brussels), same schema as the two files one
|
||||
directory up:
|
||||
|
||||
{"t": "2026-08-22T22:00:00+00:00", "w": -25.0}
|
||||
|
||||
**485,841 samples, 30/30 days read, 0 days failed, 4.13 MB gzipped.**
|
||||
Produced by `tools/export_p1_history.py`; `manifest.json` here carries the
|
||||
per-day counts, sha256 of every file, and every gap.
|
||||
|
||||
python3 tools/export_p1_history.py --days 30 --end-date 2026-08-30 \
|
||||
--out-dir scenarios/p1-net-30d-2026-08
|
||||
|
||||
## Why this exists
|
||||
|
||||
[[TARIFF-01]] AC 3 and [[SAFETY-07]] AC 3 both ask for *"monthly peak held under
|
||||
the soft limit in >= 95 % of intervals over a 30-day simulation replay"*, and
|
||||
both were merged with the criterion honestly unmet because only two days of
|
||||
capture existed. `test_tariff.py`'s docstring still recorded the reason as
|
||||
"nothing longer was ever captured" — which was true about the past and was
|
||||
quietly read as a statement about the future. It was not one. Probed on
|
||||
2026-08-31, HA's recorder answered a 20-minute window with 230 samples at 30
|
||||
days back and 0 at 45, so the whole month was still there and was exported the
|
||||
same day. **The window rolls: 2026-08-01 leaves the recorder around 2026-09-15.**
|
||||
|
||||
## The size decision — COMMITTED, deliberately, at 4.13 MB
|
||||
|
||||
Three options were on the table. This one was taken and the cost is stated
|
||||
rather than absorbed, and it is measured rather than estimated - a
|
||||
`git bundle` of the whole history before and after this commit:
|
||||
|
||||
release/1.0 @ 86c43eb 1.58 MB
|
||||
DATA-01 5.28 MB +3.70 MB, x3.3
|
||||
|
||||
A full clone of this repository therefore more than triples, and a Home
|
||||
Assistant supervisor clones it on every add-on install and update. That is the
|
||||
price; the paragraphs below are why it was judged worth paying rather than
|
||||
avoided by making the data worse or making it disappear.
|
||||
|
||||
* **Downsampling is not available.** It is not a matter of taste: the shipped
|
||||
`meter_max_age_s` is **30 s**, which is the age past which the controller's
|
||||
own ingest stops counting a held reading as coverage. Any resample coarser
|
||||
than 30 s therefore converts real coverage into modelled meter-gap and changes
|
||||
the very number AC 3 measures. Resampling at or under 30 s saves almost
|
||||
nothing against a 5 s cadence.
|
||||
* **"Outside the repo with a pointer" is what already happened, and it is why
|
||||
this card exists.** The two committed days came from the infra repo's private
|
||||
`sim/scenarios/`, the one-off export script was thrown away, and three cards
|
||||
then deferred an acceptance criterion on the belief that the data could not be
|
||||
produced again. `test_tariff.py` states the general rule verbatim: *a
|
||||
regression fixture CI cannot see is not a regression fixture.* CI's token is
|
||||
minted for this repository only.
|
||||
* **So: committed** — but in a SUBDIRECTORY, which is load-bearing. Both
|
||||
`test_tariff.py` part 9 and `test_peakguard.py` part 6 do
|
||||
`(HERE / "scenarios").glob("p1-net-*.jsonl.gz")`, which does not recurse. Drop
|
||||
these thirty files beside the two and CI grows sixty full-day
|
||||
replays - about 50 s of pure compute - on every PR forever, `test_tariff.py`'s
|
||||
`check(... replay_days < 30)` starts failing, and both files' honesty notes
|
||||
become false. In `p1-net-30d-2026-08/` they are visible to CI, invisible to
|
||||
the globs, and the two committed days keep their job as the small fixtures the
|
||||
suites grade against.
|
||||
|
||||
They are also written once and never edited, so git stores one already-deflated
|
||||
blob per day with no delta churn.
|
||||
|
||||
## Gaps are visible, never interpolated
|
||||
|
||||
Nothing is filled, carried forward or resampled. A hole in the recorder is a
|
||||
hole here — consecutive `t` values further apart than the cadence — which is
|
||||
exactly what the controller's freshness rule already reads as *no coverage*, so
|
||||
a missing hour replays as a missing hour rather than as an hour of whatever the
|
||||
last sample said.
|
||||
|
||||
**13 gaps over 30 s, 1291 s in total: 99.950 % of the month is covered.**
|
||||
All of them are listed in `manifest.json` and by
|
||||
`python3 tools/replay_ac3.py <dir> --report-gaps`. The longest is 263 s on
|
||||
2026-08-11.
|
||||
|
||||
Three samples in the month carried the state `unavailable` rather than a number
|
||||
(2026-08-18 02:03:54Z and 04:10:41Z, 2026-08-25 01:52:02Z). There is no float to
|
||||
write and the committed schema has no slot for a non-number, so they are absent
|
||||
from the series and recorded individually in `manifest.json` under
|
||||
`non_numeric_states`. Each one sits inside a gap that is already listed, which
|
||||
is the honest rendering: the meter was not reporting a power then.
|
||||
|
||||
**A failed read is never an empty hour.** If a day's read fails, the exporter
|
||||
writes NO FILE, records `"read": "FAILED"` with the error, and exits non-zero —
|
||||
so a missing day is unreadable-and-known rather than silently zero. The entity
|
||||
is also proved to exist through `/api/states` before any day is trusted, because
|
||||
a typo would otherwise answer `200 []` thirty times and produce a beautifully
|
||||
consistent dataset of nothing. On this run: 30 read, 0 failed, 0 empty.
|
||||
|
||||
## AC 3, measured
|
||||
|
||||
cd goodwe_controller
|
||||
python3 tools/replay_ac3.py scenarios/p1-net-30d-2026-08
|
||||
|
||||
485841 samples, 2879 closed quarters (30.0 days' worth)
|
||||
month 2026-08: peak 5029.8 W uncontrolled -> 4107.3 W guarded
|
||||
|
||||
TARIFF-01 AC 3 (UNCONTROLLED): 2847/2879 = 98.89 % of quarters at or under 3500 W
|
||||
SAFETY-07 AC 3 (GUARDED): 2850/2879 = 98.99 %
|
||||
guard cycles 143810, engaged on the measurement 1542, blind 195,
|
||||
saturated 0, modelled discharge 8060 Wh
|
||||
|
||||
2879 rather than 2880 quarters: the last quarter of 2026-08-30 closes only when
|
||||
a telegram past its boundary arrives, and that telegram is 2026-08-31's.
|
||||
|
||||
### What the numbers actually say — three findings, not a pass
|
||||
|
||||
1. **AC 3 is met on the ">= 95 % of intervals" reading, and the criterion turns
|
||||
out to be nearly insensitive.** The UNCONTROLLED month — no controller, no
|
||||
peak shaving, nothing running — already scores 98.89 %. A criterion that a
|
||||
house passes by doing nothing does not discriminate between a working guard
|
||||
and an absent one. That is a finding about the acceptance criterion, and it
|
||||
is reported rather than filed as a win.
|
||||
2. **On the other reading of the same sentence, AC 3 is NOT met.** "Monthly peak
|
||||
held under the soft limit" taken as *the peak itself* fails: the guard pulls
|
||||
August's peak from **5029.8 W to 4107.3 W** (-922.5 W, -18.3 %), which is
|
||||
real money on a Belgian capacity tariff, but 4107.3 W is still over the
|
||||
3500 W soft limit. Both readings are printed by the tool; neither is hidden.
|
||||
3. **The guard stops defending the soft limit once the month's peak is set, and
|
||||
that is by design.** `quarter_ceiling_w` uses `CEILING_MONTH_PEAK`: the
|
||||
billed quantity is the month's maximum, so a quarter under an
|
||||
already-recorded peak costs nothing extra and the guard does not spend the
|
||||
battery on it. The month's 32 over-cap quarters fall on four nights
|
||||
(2026-08-17, -18, -20, -21); **31 of them are pulled down** — 7494 W of
|
||||
quarter-average removed in total, 8060 Wh of modelled discharge — but only
|
||||
**3 land under 3500 W**, because by then the ceiling being defended is the
|
||||
running month peak (~4107 W), not the soft limit. The single untouched
|
||||
quarter, 2026-08-18T08:15Z at 4039.7 W, is the same rule seen from the other
|
||||
side: the running peak was already 4833 W, so 4039.7 W added nothing to the
|
||||
bill and the guard correctly stayed out. AC 3's wording asks the guard for
|
||||
something its own tariff logic deliberately does not optimise.
|
||||
|
||||
Also worth an eye: **195 blind engagements** over 143,810 cycles — the guard
|
||||
acting on a projection it could not fully cover. On the two committed days that
|
||||
count was 5. It scales with the gaps above.
|
||||
|
||||
## Verification of the exporter
|
||||
|
||||
`tools/export_p1_history.py --verify` re-fetches a committed day by the exact
|
||||
route used here and diffs it against the file:
|
||||
|
||||
python3 tools/export_p1_history.py \
|
||||
--verify scenarios/p1-net-2026-08-20.jsonl.gz \
|
||||
scenarios/p1-net-2026-08-23.jsonl.gz
|
||||
|
||||
Both days come back **byte-identical after decompression** — 865,024 B /
|
||||
15,600 samples and 894,387 B / 16,351 samples — and the copies of those two days
|
||||
inside this directory are byte-identical to the committed ones too. The gzip
|
||||
CONTAINERS differ by design: a gzip header carries an mtime and a stored
|
||||
filename, and this exporter writes mtime 0 so its own output is reproducible,
|
||||
where the 2026-08-25 one-off wrote a wall-clock mtime. Content equality is the
|
||||
claim; container equality would only assert that two files were written at the
|
||||
same second.
|
||||
|
||||
`tools/replay_ac3.py` is checked the same way rather than trusted: pointed at
|
||||
`scenarios/` it must reproduce `test_peakguard.py` part 6's published totals,
|
||||
and `--expect` makes it a hard check.
|
||||
|
||||
python3 tools/replay_ac3.py scenarios --expect 189/190
|
||||
|
||||
reproduces 189/190 quarters, the 5029.8 W billed quarter, the 3485.3 W guarded
|
||||
result and the 878 Wh of modelled discharge that part 6 prints.
|
||||
@@ -0,0 +1,767 @@
|
||||
{
|
||||
"generated": "2026-08-31T08:42:40.614548+00:00",
|
||||
"generator": "tools/export_p1_history.py",
|
||||
"source": "http://192.168.2.158:8123/api/history/period",
|
||||
"entity": "sensor.p1_meter_active_power",
|
||||
"timezone": "Europe/Brussels",
|
||||
"gap_threshold_s": 30.0,
|
||||
"format": "{\"t\": <ISO-8601 UTC>, \"w\": <float watts>} per line, gzip",
|
||||
"days": [
|
||||
{
|
||||
"date": "2026-08-01",
|
||||
"window": [
|
||||
"2026-08-01T00:00:00+02:00",
|
||||
"2026-08-02T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1128060,
|
||||
"attempts": 1,
|
||||
"samples": 16511,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-07-31T22:00:00+00:00",
|
||||
"last": "2026-08-01T21:59:56.268499+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86396.3,
|
||||
"mean_cadence_s": 5.233,
|
||||
"file": "p1-net-2026-08-01.jsonl.gz",
|
||||
"bytes_gz": 140531,
|
||||
"bytes_raw": 913203,
|
||||
"sha256_raw": "120de2a0b7eb552281dac4bc1f8fd76d947ba72d1f5b426d9dcdd85ec103bd06",
|
||||
"sha256_gz": "e09bdee349dd3668a073b7fd77974e639c269ec37e73e81a9706cadde083f064"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-02",
|
||||
"window": [
|
||||
"2026-08-02T00:00:00+02:00",
|
||||
"2026-08-03T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1115387,
|
||||
"attempts": 1,
|
||||
"samples": 16303,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-01T22:00:00+00:00",
|
||||
"last": "2026-08-02T21:59:55.191360+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86395.2,
|
||||
"mean_cadence_s": 5.3,
|
||||
"file": "p1-net-2026-08-02.jsonl.gz",
|
||||
"bytes_gz": 141404,
|
||||
"bytes_raw": 903234,
|
||||
"sha256_raw": "9ab4891c0abf9d621fa528848fc3a874687843804768e8aeab6a88f7bb04b679",
|
||||
"sha256_gz": "94774139b3e19d04bcda3ac5ea988dd4d645e99004f90b917c6b0bf3fbac21cb"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-03",
|
||||
"window": [
|
||||
"2026-08-03T00:00:00+02:00",
|
||||
"2026-08-04T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1123668,
|
||||
"attempts": 1,
|
||||
"samples": 16424,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-02T22:00:00+00:00",
|
||||
"last": "2026-08-03T21:59:55.255808+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86395.3,
|
||||
"mean_cadence_s": 5.261,
|
||||
"file": "p1-net-2026-08-03.jsonl.gz",
|
||||
"bytes_gz": 144202,
|
||||
"bytes_raw": 909942,
|
||||
"sha256_raw": "13f8dd904a31b5b67fcfde04f6afc49c34c3a9504617278d79d87516fcd6441f",
|
||||
"sha256_gz": "e1bc0cb0e271cbc8d896aee97aa0f5c18e49e78905d1aaf137904a768047c7bc"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-04",
|
||||
"window": [
|
||||
"2026-08-04T00:00:00+02:00",
|
||||
"2026-08-05T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1108582,
|
||||
"attempts": 1,
|
||||
"samples": 16286,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-03T22:00:00+00:00",
|
||||
"last": "2026-08-04T21:59:58.729138+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86398.7,
|
||||
"mean_cadence_s": 5.305,
|
||||
"file": "p1-net-2026-08-04.jsonl.gz",
|
||||
"bytes_gz": 136505,
|
||||
"bytes_raw": 896650,
|
||||
"sha256_raw": "a65cd9d1ceda61e5a40043edb401c2154feea2b36bd3221710df808d778eabce",
|
||||
"sha256_gz": "c19b19ac890f4caad4140cb27d2fe6f3e972545f664127e5d9f457e711ad8c88"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-05",
|
||||
"window": [
|
||||
"2026-08-05T00:00:00+02:00",
|
||||
"2026-08-06T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1136556,
|
||||
"attempts": 1,
|
||||
"samples": 16655,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-04T22:00:00+00:00",
|
||||
"last": "2026-08-05T21:59:56.189152+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86396.2,
|
||||
"mean_cadence_s": 5.188,
|
||||
"file": "p1-net-2026-08-05.jsonl.gz",
|
||||
"bytes_gz": 142561,
|
||||
"bytes_raw": 919827,
|
||||
"sha256_raw": "dd327685c6f322bfaa49bc5bf228301e790e5a96fd7c23c487b845e44a962db4",
|
||||
"sha256_gz": "5ff7a1336c49ba38b8058820181dd8ed9c2b8db41b5a3a52412a5a3d8c515332"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-06",
|
||||
"window": [
|
||||
"2026-08-06T00:00:00+02:00",
|
||||
"2026-08-07T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1124419,
|
||||
"attempts": 1,
|
||||
"samples": 16412,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-05T22:00:00+00:00",
|
||||
"last": "2026-08-06T21:59:57.192711+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86397.2,
|
||||
"mean_cadence_s": 5.265,
|
||||
"file": "p1-net-2026-08-06.jsonl.gz",
|
||||
"bytes_gz": 142313,
|
||||
"bytes_raw": 910849,
|
||||
"sha256_raw": "7cd1fe8315e24dc6391693c49d6306b0c0cf9b6c3a478e457da921001cea930c",
|
||||
"sha256_gz": "9108f169d645b9da07d4ae310666a1365ea07275ff311cd60adad1b702edbf11"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-07",
|
||||
"window": [
|
||||
"2026-08-07T00:00:00+02:00",
|
||||
"2026-08-08T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1087531,
|
||||
"attempts": 1,
|
||||
"samples": 15952,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-06T22:00:00+00:00",
|
||||
"last": "2026-08-07T21:59:58.195053+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86398.2,
|
||||
"mean_cadence_s": 5.416,
|
||||
"file": "p1-net-2026-08-07.jsonl.gz",
|
||||
"bytes_gz": 135979,
|
||||
"bytes_raw": 879941,
|
||||
"sha256_raw": "e0f67a9214980e2fd4893208cb402165065dadce2105ce1581c6b1d49952f39a",
|
||||
"sha256_gz": "b70b84f3743d1d84055c911bcd5589cb8f0371e8916bde741308c5bb50702c29"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-08",
|
||||
"window": [
|
||||
"2026-08-08T00:00:00+02:00",
|
||||
"2026-08-09T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1058569,
|
||||
"attempts": 1,
|
||||
"samples": 15468,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-07T22:00:00+00:00",
|
||||
"last": "2026-08-08T21:59:52.805887+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86392.8,
|
||||
"mean_cadence_s": 5.586,
|
||||
"file": "p1-net-2026-08-08.jsonl.gz",
|
||||
"bytes_gz": 134025,
|
||||
"bytes_raw": 857271,
|
||||
"sha256_raw": "0be14ca7593b4c7d68570115cc26e24f6a26ab87fe3c168a8b942b167e8d4f68",
|
||||
"sha256_gz": "d93dcc8ca3eae03af13b9ae36acd5ab1eaa1e7ee93c237d386e8ded1c61f5906"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-09",
|
||||
"window": [
|
||||
"2026-08-09T00:00:00+02:00",
|
||||
"2026-08-10T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1059349,
|
||||
"attempts": 1,
|
||||
"samples": 15508,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-08T22:00:00+00:00",
|
||||
"last": "2026-08-09T21:59:58.178998+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86398.2,
|
||||
"mean_cadence_s": 5.572,
|
||||
"file": "p1-net-2026-08-09.jsonl.gz",
|
||||
"bytes_gz": 135697,
|
||||
"bytes_raw": 857531,
|
||||
"sha256_raw": "f4793799173c502fcee5f78e2d9b0dcc984b5d2d8bd2fc0db0cd0f8d403733b3",
|
||||
"sha256_gz": "2fea6c5c8b1452d72ad5aa3b0ae9755bb2ea08071514c08b1a3b1155ef457cf1"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-10",
|
||||
"window": [
|
||||
"2026-08-10T00:00:00+02:00",
|
||||
"2026-08-11T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1066721,
|
||||
"attempts": 1,
|
||||
"samples": 15594,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-09T22:00:00+00:00",
|
||||
"last": "2026-08-10T21:59:59.170056+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86399.2,
|
||||
"mean_cadence_s": 5.541,
|
||||
"file": "p1-net-2026-08-10.jsonl.gz",
|
||||
"bytes_gz": 136682,
|
||||
"bytes_raw": 863785,
|
||||
"sha256_raw": "ff5c3fa419505127a51acc56dfc13e8437e4ec2c13a11c8d8b2d6c3e20e2410e",
|
||||
"sha256_gz": "4fd30e495f4e6d1c8501d05c23f9891ac34364d86a9c55ac825d52b2ebb31023"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-11",
|
||||
"window": [
|
||||
"2026-08-11T00:00:00+02:00",
|
||||
"2026-08-12T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1056281,
|
||||
"attempts": 1,
|
||||
"samples": 15371,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-10T22:00:00+00:00",
|
||||
"last": "2026-08-11T21:59:58.608150+00:00",
|
||||
"gaps": [
|
||||
{
|
||||
"from": "2026-08-11T08:00:14.121653+00:00",
|
||||
"to": "2026-08-11T08:02:54.427007+00:00",
|
||||
"seconds": 160.305
|
||||
},
|
||||
{
|
||||
"from": "2026-08-11T17:04:18.517592+00:00",
|
||||
"to": "2026-08-11T17:08:41.759777+00:00",
|
||||
"seconds": 263.242
|
||||
}
|
||||
],
|
||||
"covered_span_s": 86398.6,
|
||||
"mean_cadence_s": 5.621,
|
||||
"file": "p1-net-2026-08-11.jsonl.gz",
|
||||
"bytes_gz": 133711,
|
||||
"bytes_raw": 856244,
|
||||
"sha256_raw": "67c2a740019f8908405b96b3436cd953af27baead2957acc7c1dcf10b37ac94e",
|
||||
"sha256_gz": "4f8c553edc9051644bd7baa659f6f92a8b5f5ce12a0410c8b5235d1e6c3084c9"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-12",
|
||||
"window": [
|
||||
"2026-08-12T00:00:00+02:00",
|
||||
"2026-08-13T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1124945,
|
||||
"attempts": 1,
|
||||
"samples": 16432,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-11T22:00:00+00:00",
|
||||
"last": "2026-08-12T21:59:58.648966+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86398.6,
|
||||
"mean_cadence_s": 5.258,
|
||||
"file": "p1-net-2026-08-12.jsonl.gz",
|
||||
"bytes_gz": 136903,
|
||||
"bytes_raw": 911115,
|
||||
"sha256_raw": "73c8a1626640226ddffb32d564557d1cbe57a15367ea1ba3632934e15a1413dc",
|
||||
"sha256_gz": "acec87065b7de2c605db00504ec5a9b3c99584620dfb3b67194e2cf9464d6a5f"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-13",
|
||||
"window": [
|
||||
"2026-08-13T00:00:00+02:00",
|
||||
"2026-08-14T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1126742,
|
||||
"attempts": 1,
|
||||
"samples": 16473,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-12T22:00:00+00:00",
|
||||
"last": "2026-08-13T21:59:57.562043+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86397.6,
|
||||
"mean_cadence_s": 5.245,
|
||||
"file": "p1-net-2026-08-13.jsonl.gz",
|
||||
"bytes_gz": 136221,
|
||||
"bytes_raw": 912379,
|
||||
"sha256_raw": "1d9a16c529174bd8c74625a742c46d98de4acf7a045f83df8bdbac22fa86e8e7",
|
||||
"sha256_gz": "adfef36f629decf54c625f2dbfe6b4dab9441218c42557c90ac7f6d331943a59"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-14",
|
||||
"window": [
|
||||
"2026-08-14T00:00:00+02:00",
|
||||
"2026-08-15T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1124413,
|
||||
"attempts": 1,
|
||||
"samples": 16427,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-13T22:00:00+00:00",
|
||||
"last": "2026-08-14T21:59:55.611487+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86395.6,
|
||||
"mean_cadence_s": 5.26,
|
||||
"file": "p1-net-2026-08-14.jsonl.gz",
|
||||
"bytes_gz": 135570,
|
||||
"bytes_raw": 910648,
|
||||
"sha256_raw": "dc1417881e54fe215e9fcd4ed643d2482a277d61f8f27f25e2ff1e2cba29f7a8",
|
||||
"sha256_gz": "d4eff781cd16a83fd1be9e643fee5bb5d5897e7d45bb9865972a7304d92c84a4"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-15",
|
||||
"window": [
|
||||
"2026-08-15T00:00:00+02:00",
|
||||
"2026-08-16T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1124461,
|
||||
"attempts": 1,
|
||||
"samples": 16516,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-14T22:00:00+00:00",
|
||||
"last": "2026-08-15T21:59:56.581991+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86396.6,
|
||||
"mean_cadence_s": 5.231,
|
||||
"file": "p1-net-2026-08-15.jsonl.gz",
|
||||
"bytes_gz": 137670,
|
||||
"bytes_raw": 909539,
|
||||
"sha256_raw": "f07a242faed68313512e0363d914846522ff839315ad80b55689072ac74e8a69",
|
||||
"sha256_gz": "082314c6003c2208752acc4614b9809f06735d01e216b935ea94faf578c5598e"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-16",
|
||||
"window": [
|
||||
"2026-08-16T00:00:00+02:00",
|
||||
"2026-08-17T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1127871,
|
||||
"attempts": 1,
|
||||
"samples": 16552,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-15T22:00:00+00:00",
|
||||
"last": "2026-08-16T21:59:59.527710+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86399.5,
|
||||
"mean_cadence_s": 5.22,
|
||||
"file": "p1-net-2026-08-16.jsonl.gz",
|
||||
"bytes_gz": 138347,
|
||||
"bytes_raw": 912481,
|
||||
"sha256_raw": "a44e5d961600a539ddf824d5d211ef319833666b7598e5939a6cdf2cb1e3581f",
|
||||
"sha256_gz": "8f6511e52f5be9187f422021abc93a5fa8b698b4c785877d69d15dd6de106fd8"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-17",
|
||||
"window": [
|
||||
"2026-08-17T00:00:00+02:00",
|
||||
"2026-08-18T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1133721,
|
||||
"attempts": 1,
|
||||
"samples": 16627,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-16T22:00:00+00:00",
|
||||
"last": "2026-08-17T21:59:59.685699+00:00",
|
||||
"gaps": [
|
||||
{
|
||||
"from": "2026-08-17T17:27:00.599126+00:00",
|
||||
"to": "2026-08-17T17:28:20.389628+00:00",
|
||||
"seconds": 79.791
|
||||
}
|
||||
],
|
||||
"covered_span_s": 86399.7,
|
||||
"mean_cadence_s": 5.197,
|
||||
"file": "p1-net-2026-08-17.jsonl.gz",
|
||||
"bytes_gz": 140214,
|
||||
"bytes_raw": 917356,
|
||||
"sha256_raw": "70cce79b3732f292f7f58f0b2655bdb4a00a20a6530f1f50dac62ef5f039788b",
|
||||
"sha256_gz": "e2f9b461c4db7902e1a3d30b439ebc11ebcd9618a72d12f10e5131f38fb437bf"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-18",
|
||||
"window": [
|
||||
"2026-08-18T00:00:00+02:00",
|
||||
"2026-08-19T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1074019,
|
||||
"attempts": 1,
|
||||
"samples": 15699,
|
||||
"non_numeric_states": [
|
||||
{
|
||||
"t": "2026-08-18T02:03:54.417106+00:00",
|
||||
"state": "unavailable"
|
||||
},
|
||||
{
|
||||
"t": "2026-08-18T04:10:41.786299+00:00",
|
||||
"state": "unavailable"
|
||||
}
|
||||
],
|
||||
"first": "2026-08-17T22:00:00+00:00",
|
||||
"last": "2026-08-18T21:59:54.501766+00:00",
|
||||
"gaps": [
|
||||
{
|
||||
"from": "2026-08-18T02:02:59.633845+00:00",
|
||||
"to": "2026-08-18T02:04:38.694366+00:00",
|
||||
"seconds": 99.061
|
||||
},
|
||||
{
|
||||
"from": "2026-08-18T04:09:54.292049+00:00",
|
||||
"to": "2026-08-18T04:11:38.500238+00:00",
|
||||
"seconds": 104.208
|
||||
}
|
||||
],
|
||||
"covered_span_s": 86394.5,
|
||||
"mean_cadence_s": 5.504,
|
||||
"file": "p1-net-2026-08-18.jsonl.gz",
|
||||
"bytes_gz": 135894,
|
||||
"bytes_raw": 869570,
|
||||
"sha256_raw": "68cd44ac56099ee38f0bba4c32d7d17d33b92d28a99d27419f53c22374491ca6",
|
||||
"sha256_gz": "d6c97508f805aff040d2b4ed4e06c0e7373db2ea6227c7d79ef24c8c79074115"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-19",
|
||||
"window": [
|
||||
"2026-08-19T00:00:00+02:00",
|
||||
"2026-08-20T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 989720,
|
||||
"attempts": 1,
|
||||
"samples": 14549,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-18T22:00:00+00:00",
|
||||
"last": "2026-08-19T21:59:56.556610+00:00",
|
||||
"gaps": [
|
||||
{
|
||||
"from": "2026-08-19T01:33:29.599224+00:00",
|
||||
"to": "2026-08-19T01:34:01.653383+00:00",
|
||||
"seconds": 32.054
|
||||
}
|
||||
],
|
||||
"covered_span_s": 86396.6,
|
||||
"mean_cadence_s": 5.939,
|
||||
"file": "p1-net-2026-08-19.jsonl.gz",
|
||||
"bytes_gz": 126345,
|
||||
"bytes_raw": 800369,
|
||||
"sha256_raw": "0f11ce0cee43cbd7bdb08dc48b202e58c442157ac08b2cda5b64bbab9bd56300",
|
||||
"sha256_gz": "13d61b7da903877c8ae7813b833cb5836ec04a1cf1efd0e1fa87e916bef95fd5"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-20",
|
||||
"window": [
|
||||
"2026-08-20T00:00:00+02:00",
|
||||
"2026-08-21T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1068038,
|
||||
"attempts": 1,
|
||||
"samples": 15600,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-19T22:00:00+00:00",
|
||||
"last": "2026-08-20T21:59:56.742604+00:00",
|
||||
"gaps": [
|
||||
{
|
||||
"from": "2026-08-20T02:11:56.607373+00:00",
|
||||
"to": "2026-08-20T02:12:38.840458+00:00",
|
||||
"seconds": 42.233
|
||||
},
|
||||
{
|
||||
"from": "2026-08-20T20:42:04.623102+00:00",
|
||||
"to": "2026-08-20T20:43:41.666543+00:00",
|
||||
"seconds": 97.043
|
||||
}
|
||||
],
|
||||
"covered_span_s": 86396.7,
|
||||
"mean_cadence_s": 5.539,
|
||||
"file": "p1-net-2026-08-20.jsonl.gz",
|
||||
"bytes_gz": 139693,
|
||||
"bytes_raw": 865024,
|
||||
"sha256_raw": "2612153963951dd13fe310ad7fbb5bec2e195db56fd99fd963134c8badae25ba",
|
||||
"sha256_gz": "362f2ee3d9d7f23f9051ba78993c5af7443dabb3b5f16339efc673444720c892"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-21",
|
||||
"window": [
|
||||
"2026-08-21T00:00:00+02:00",
|
||||
"2026-08-22T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1118002,
|
||||
"attempts": 1,
|
||||
"samples": 16383,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-20T22:00:00+00:00",
|
||||
"last": "2026-08-21T21:59:57.771588+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86397.8,
|
||||
"mean_cadence_s": 5.274,
|
||||
"file": "p1-net-2026-08-21.jsonl.gz",
|
||||
"bytes_gz": 139105,
|
||||
"bytes_raw": 904809,
|
||||
"sha256_raw": "933d9a6899f59c128d0d9b84d6944cea63b1ad58dc57ac52e3e476f9c616f07b",
|
||||
"sha256_gz": "a563581a9a3338af173abb99a200db32881ddaf893d2458f5d8de59b5ec6041f"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-22",
|
||||
"window": [
|
||||
"2026-08-22T00:00:00+02:00",
|
||||
"2026-08-23T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1129094,
|
||||
"attempts": 1,
|
||||
"samples": 16520,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-21T22:00:00+00:00",
|
||||
"last": "2026-08-22T21:59:56.913786+00:00",
|
||||
"gaps": [
|
||||
{
|
||||
"from": "2026-08-21T23:05:20.035692+00:00",
|
||||
"to": "2026-08-21T23:07:51.147719+00:00",
|
||||
"seconds": 151.112
|
||||
},
|
||||
{
|
||||
"from": "2026-08-21T23:07:56.966604+00:00",
|
||||
"to": "2026-08-21T23:08:27.695101+00:00",
|
||||
"seconds": 30.728
|
||||
}
|
||||
],
|
||||
"covered_span_s": 86396.9,
|
||||
"mean_cadence_s": 5.23,
|
||||
"file": "p1-net-2026-08-22.jsonl.gz",
|
||||
"bytes_gz": 141564,
|
||||
"bytes_raw": 914120,
|
||||
"sha256_raw": "39ccdf67c806344490f791648822cd2cd14d73ea7cc0b6e8572a1e15e8f74641",
|
||||
"sha256_gz": "5c46f1879a723f9986d521124200cab839af2a4722b76bc97c6f6f59d0df65a2"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-23",
|
||||
"window": [
|
||||
"2026-08-23T00:00:00+02:00",
|
||||
"2026-08-24T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1107164,
|
||||
"attempts": 1,
|
||||
"samples": 16351,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-22T22:00:00+00:00",
|
||||
"last": "2026-08-23T21:59:56.921115+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86396.9,
|
||||
"mean_cadence_s": 5.284,
|
||||
"file": "p1-net-2026-08-23.jsonl.gz",
|
||||
"bytes_gz": 136277,
|
||||
"bytes_raw": 894387,
|
||||
"sha256_raw": "ffb15367bf3c98eff8a5600e4858ac99778b597c03d0a4f6154e6971b8f98a44",
|
||||
"sha256_gz": "3fa76981758a468ba0aa68b50fe68d5bdb1ef38cd7faec2b6ffdb61c9ddb2db2"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-24",
|
||||
"window": [
|
||||
"2026-08-24T00:00:00+02:00",
|
||||
"2026-08-25T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1122175,
|
||||
"attempts": 1,
|
||||
"samples": 16584,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-23T22:00:00+00:00",
|
||||
"last": "2026-08-24T21:59:57.940474+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86397.9,
|
||||
"mean_cadence_s": 5.21,
|
||||
"file": "p1-net-2026-08-24.jsonl.gz",
|
||||
"bytes_gz": 136834,
|
||||
"bytes_raw": 906369,
|
||||
"sha256_raw": "30465d5ff51220201b4a79341e5da7bfc3d88290465b1c9d886ba94eee8dd234",
|
||||
"sha256_gz": "5103088e047bfcbd9b3f55521f431b01cd27de10c2642e00281a13faa762bdd9"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-25",
|
||||
"window": [
|
||||
"2026-08-25T00:00:00+02:00",
|
||||
"2026-08-26T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1110567,
|
||||
"attempts": 1,
|
||||
"samples": 16451,
|
||||
"non_numeric_states": [
|
||||
{
|
||||
"t": "2026-08-25T01:52:02.104225+00:00",
|
||||
"state": "unavailable"
|
||||
}
|
||||
],
|
||||
"first": "2026-08-24T22:00:00+00:00",
|
||||
"last": "2026-08-25T21:59:56.883744+00:00",
|
||||
"gaps": [
|
||||
{
|
||||
"from": "2026-08-25T01:51:25.942719+00:00",
|
||||
"to": "2026-08-25T01:52:53.203963+00:00",
|
||||
"seconds": 87.261
|
||||
}
|
||||
],
|
||||
"covered_span_s": 86396.9,
|
||||
"mean_cadence_s": 5.252,
|
||||
"file": "p1-net-2026-08-25.jsonl.gz",
|
||||
"bytes_gz": 135890,
|
||||
"bytes_raw": 896416,
|
||||
"sha256_raw": "0e7c86b2b74f2108bc7f5f5d2de2627caaafe473ca4ae18515534dff10526e26",
|
||||
"sha256_gz": "2a5411c10d8d3b271f133c62c04bd722279b48b08d7376084fbad6cc6dd66d76"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-26",
|
||||
"window": [
|
||||
"2026-08-26T00:00:00+02:00",
|
||||
"2026-08-27T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1123610,
|
||||
"attempts": 1,
|
||||
"samples": 16542,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-25T22:00:00+00:00",
|
||||
"last": "2026-08-26T21:59:58.915457+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86398.9,
|
||||
"mean_cadence_s": 5.223,
|
||||
"file": "p1-net-2026-08-26.jsonl.gz",
|
||||
"bytes_gz": 139301,
|
||||
"bytes_raw": 908350,
|
||||
"sha256_raw": "88d72c1b9f51707d62e744f40402de4260a04617875b85c65d2df6079c6c84ca",
|
||||
"sha256_gz": "0661588fbc0c6c42825b67d483c8818c070405c1002424b3f1d61d8fdadb6370"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-27",
|
||||
"window": [
|
||||
"2026-08-27T00:00:00+02:00",
|
||||
"2026-08-28T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1122433,
|
||||
"attempts": 1,
|
||||
"samples": 16573,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-26T22:00:00+00:00",
|
||||
"last": "2026-08-27T21:59:58.538446+00:00",
|
||||
"gaps": [
|
||||
{
|
||||
"from": "2026-08-27T18:50:29.030586+00:00",
|
||||
"to": "2026-08-27T18:52:22.938470+00:00",
|
||||
"seconds": 113.908
|
||||
}
|
||||
],
|
||||
"covered_span_s": 86398.5,
|
||||
"mean_cadence_s": 5.214,
|
||||
"file": "p1-net-2026-08-27.jsonl.gz",
|
||||
"bytes_gz": 138009,
|
||||
"bytes_raw": 906770,
|
||||
"sha256_raw": "528ab8ee10a46275d2e91d56ef2ed847e5932fae76a1752e6f2e9828dbfa9865",
|
||||
"sha256_gz": "5cacc8c5788f43d6e6e2f569445fa82d9edc5a63b1f082f9a1d6a3bb0b404260"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-28",
|
||||
"window": [
|
||||
"2026-08-28T00:00:00+02:00",
|
||||
"2026-08-29T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1114796,
|
||||
"attempts": 1,
|
||||
"samples": 16456,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-27T22:00:00+00:00",
|
||||
"last": "2026-08-28T21:59:58.515555+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86398.5,
|
||||
"mean_cadence_s": 5.251,
|
||||
"file": "p1-net-2026-08-28.jsonl.gz",
|
||||
"bytes_gz": 135917,
|
||||
"bytes_raw": 900654,
|
||||
"sha256_raw": "f89838bc9f61af074c17817e4a6eab037318f38fa3c65dad3edfa2106abbc9fb",
|
||||
"sha256_gz": "97090b605f67dcc1f84dea2c51d4832dfe1c2af497a4f4cfe50ecb51024d9b4c"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-29",
|
||||
"window": [
|
||||
"2026-08-29T00:00:00+02:00",
|
||||
"2026-08-30T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1092983,
|
||||
"attempts": 1,
|
||||
"samples": 16132,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-28T22:00:00+00:00",
|
||||
"last": "2026-08-29T21:59:57.574524+00:00",
|
||||
"gaps": [
|
||||
{
|
||||
"from": "2026-08-29T00:03:20.539844+00:00",
|
||||
"to": "2026-08-29T00:03:50.569581+00:00",
|
||||
"seconds": 30.03
|
||||
}
|
||||
],
|
||||
"covered_span_s": 86397.6,
|
||||
"mean_cadence_s": 5.356,
|
||||
"file": "p1-net-2026-08-29.jsonl.gz",
|
||||
"bytes_gz": 134210,
|
||||
"bytes_raw": 883053,
|
||||
"sha256_raw": "896d4c9d660be4faabc5a088323e246e257877d5ff6bd0ec084b785480d76e12",
|
||||
"sha256_gz": "b6f264cd3afba63209123a27490e7e1c3ae70a86731d7ea05ae67f88917f136f"
|
||||
},
|
||||
{
|
||||
"date": "2026-08-30",
|
||||
"window": [
|
||||
"2026-08-30T00:00:00+02:00",
|
||||
"2026-08-31T00:00:00+02:00"
|
||||
],
|
||||
"read": "OK",
|
||||
"http_bytes": 1118895,
|
||||
"attempts": 1,
|
||||
"samples": 16490,
|
||||
"non_numeric_states": [],
|
||||
"first": "2026-08-29T22:00:00+00:00",
|
||||
"last": "2026-08-30T21:59:58.533323+00:00",
|
||||
"gaps": [],
|
||||
"covered_span_s": 86398.5,
|
||||
"mean_cadence_s": 5.24,
|
||||
"file": "p1-net-2026-08-30.jsonl.gz",
|
||||
"bytes_gz": 138429,
|
||||
"bytes_raw": 904311,
|
||||
"sha256_raw": "e704818efb58bb6c505791bc9eab2e9476bf48ff8911d0e79b401fd35b529fa0",
|
||||
"sha256_gz": "9052fead2418bff3083bec3ae8ce64d6b9f6224b668f4bb510c79775f66c5b79"
|
||||
}
|
||||
],
|
||||
"summary": {
|
||||
"days_requested": 30,
|
||||
"days_read": 30,
|
||||
"days_failed": 0,
|
||||
"samples": 485841,
|
||||
"bytes_gz": 4126003,
|
||||
"empty_days": []
|
||||
}
|
||||
}
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -810,9 +810,16 @@ print()
|
||||
print(f" BASELINE {totals['u_under']}/{totals['u_q']} = {U_PCT:.1f} % of "
|
||||
f"quarters under {CAP_W:.0f} W, uncontrolled")
|
||||
print(f" GUARDED {totals['g_under']}/{totals['g_q']} = {G_PCT:.1f} %")
|
||||
print(f" Two days, not thirty. AC 3 is TARIFF-01's and is still not met - "
|
||||
f"see test_tariff.py part 9. What this compares is the same quarters with "
|
||||
f"and without the guard, which is what two days can support.")
|
||||
print(f" Two days, not thirty - by choice: the thirty exist now "
|
||||
f"(scenarios/p1-net-30d-2026-08/, DATA-01) and live one directory down so "
|
||||
f"this glob does not pick them up and put ~50 s of replay on every PR. "
|
||||
f"tools/replay_ac3.py runs the month and must reproduce the totals above "
|
||||
f"to be believed: `python3 tools/replay_ac3.py scenarios --expect "
|
||||
f"{totals['u_under']}/{totals['u_q']}`. Over the thirty days the guard "
|
||||
f"takes August's peak from 5029.8 W to 4107.3 W - still over the "
|
||||
f"{CAP_W:.0f} W soft limit - and moves the interval count from 98.89 % to "
|
||||
f"98.99 %, because once a month peak is set the ceiling it defends is "
|
||||
f"that peak and not the cap. See that directory's README.")
|
||||
check("the guard improves on the uncontrolled baseline rather than merely "
|
||||
"matching it", G_PCT > U_PCT)
|
||||
check("...and it does not degrade a single quarter anywhere in the two days",
|
||||
|
||||
@@ -35,16 +35,39 @@ NOT allowed to be tested at:
|
||||
10. THE WIRING, through the real `Controller` and its real status page.
|
||||
|
||||
------------------------------------------------------------------------------
|
||||
⚠️ AC 3 IS NOT MET, AND PART 9 DOES NOT PRETEND OTHERWISE.
|
||||
⚠️ PART 9 IS A TWO-DAY MEASUREMENT AND SAYS SO. THE 30-DAY ANSWER IS ELSEWHERE
|
||||
(DATA-01), AND IT DOES NOT LAND WHERE ANYONE EXPECTED.
|
||||
|
||||
AC 3 asks for "the monthly peak held under the soft limit in >= 95 % of
|
||||
intervals over a 30-day simulation replay". THERE IS NO 30-DAY DATASET. The
|
||||
infra repo's `sim/scenarios/` holds exactly two days of this house's real meter
|
||||
- 2026-08-20 (the Illusmart failure day) and 2026-08-23 (the healthy reference)
|
||||
- exported on 2026-08-25 because they were about to age out of HA's recorder,
|
||||
and nothing longer was ever captured. `sim/scenarios/README.md` lists the whole
|
||||
inventory; the rest is RS485 bus frames, OTA witness logs and a 578-row rollback
|
||||
poll, none of which carries grid power over calendar time.
|
||||
intervals over a 30-day simulation replay". When this file was written there
|
||||
were two days of this house's real meter - 2026-08-20 (the Illusmart failure
|
||||
day) and 2026-08-23 (the healthy reference) - exported on 2026-08-25 because
|
||||
they were about to age out of HA's recorder.
|
||||
|
||||
⚠️ AND THE SENTENCE THAT USED TO STAND HERE, "nothing longer was ever
|
||||
captured", WAS TRUE ABOUT THE PAST AND WAS READ AS A CLAIM ABOUT THE FUTURE.
|
||||
It was not one. HA's recorder still held the whole month; DATA-01 probed it on
|
||||
2026-08-31 (230 samples in a 20-minute window at 30 days back, 0 at 45) and
|
||||
exported thirty days the same day, into `scenarios/p1-net-30d-2026-08/` with
|
||||
`tools/export_p1_history.py` - which is committed now precisely because the
|
||||
2026-08-25 one-off was not, and its absence is what let the belief survive.
|
||||
|
||||
Over those thirty days the criterion measures 98.89 % UNCONTROLLED and 98.99 %
|
||||
guarded, so it is met - and that is a finding about the CRITERION as much as
|
||||
about the guard, because a month with no controller at all passes it. Read the
|
||||
other way - the monthly PEAK itself under the soft limit - it is not met: the
|
||||
guard takes August's peak from 5029.8 W to 4107.3 W, still over 3500 W. Both
|
||||
numbers, and why they part company, are in
|
||||
`scenarios/p1-net-30d-2026-08/README.md`.
|
||||
|
||||
PART 9 IS DELIBERATELY LEFT GRADING THE TWO COMMITTED DAYS. It is a per-PR
|
||||
regression check and the thirty days cost about fifty seconds of replay; they
|
||||
also live one directory down precisely so this file's glob does not pick them
|
||||
up. `tools/replay_ac3.py` is the thing that runs the month, and it is checked
|
||||
against part 6 of test_peakguard.py rather than trusted:
|
||||
|
||||
python3 tools/replay_ac3.py scenarios --expect 189/190
|
||||
python3 tools/replay_ac3.py scenarios/p1-net-30d-2026-08
|
||||
|
||||
So part 9 replays the two days that exist, reports the percentage over them, and
|
||||
states the sample size in days next to it. It is a MEASUREMENT, not a verdict:
|
||||
@@ -60,8 +83,8 @@ export) for the reason `scenarios/p1-age-2026-08-26.jsonl.gz` already is: the
|
||||
infra repo is private, CI's token is minted for this repository only, and a
|
||||
regression fixture that CI cannot see is not a regression fixture.
|
||||
|
||||
Closing AC 3 needs 30 days of capture. That is TEST-01's replay harness, and it
|
||||
cannot start before somebody starts recording.
|
||||
The recorder's window rolls: 2026-08-01 leaves it around 2026-09-15, which is
|
||||
why the committed copy is the artefact and not the API call.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
@@ -748,9 +771,11 @@ print()
|
||||
print(f" AC 3 MEASURED: {under}/{total_q} quarters = {PCT:.1f} % under the "
|
||||
f"{SHIPPED_CAP_W:.0f} W soft limit, over {replay_days:.1f} DAYS of "
|
||||
f"replay, UNCONTROLLED (no peak shaving was running on either day).")
|
||||
print(f" AC 3 ASKS FOR 30 DAYS. {replay_days:.1f} days exist. "
|
||||
f"AC 3 IS NOT MET and this run does not claim it is - see the module "
|
||||
f"docstring. The figure above is the BASELINE SAFETY-07 has to improve.")
|
||||
print(f" AC 3 ASKS FOR 30 DAYS and this run is {replay_days:.1f}, so it "
|
||||
f"does not settle it either way. The 30 days DO now exist - "
|
||||
f"scenarios/p1-net-30d-2026-08/, DATA-01 - and "
|
||||
f"tools/replay_ac3.py is what runs them; the figure above is the "
|
||||
f"BASELINE SAFETY-07 has to improve, on the two days CI replays.")
|
||||
check("the replay produced a measurable number over a stated sample size, "
|
||||
"rather than a criterion quietly redefined to fit the data",
|
||||
total_q > 100 and replay_days < 30)
|
||||
|
||||
@@ -0,0 +1,455 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Export a P1 power signal out of Home Assistant's recorder into the committed
|
||||
scenario format - one gzipped JSON object per line, one file per LOCAL day.
|
||||
|
||||
WHY THIS FILE EXISTS AT ALL
|
||||
===========================
|
||||
`scenarios/p1-net-2026-08-20.jsonl.gz` and `p1-net-2026-08-23.jsonl.gz` were
|
||||
produced once, by hand, and the script was thrown away. The consequence was not
|
||||
theoretical: TARIFF-01's and SAFETY-07's AC 3 ("monthly peak held under the soft
|
||||
limit in >= 95 % of intervals over a 30-day simulation replay") were both merged
|
||||
UNMET on the belief that thirty days could not be produced retroactively - a
|
||||
belief that was simply wrong, and that survived because nobody could re-run the
|
||||
export to find out. So the export is a committed, runnable artefact now, and the
|
||||
next person does not have to re-derive it under time pressure while a retention
|
||||
window rolls.
|
||||
|
||||
python3 tools/export_p1_history.py --days 30 --out-dir /some/where
|
||||
|
||||
THE TWO RULES THIS FILE IS BUILT AROUND
|
||||
=======================================
|
||||
1. A FAILED READ IS NOT AN OBSERVATION. An HTTP error, a timeout, a truncated
|
||||
body, a non-JSON body - none of those are "the meter reported nothing that
|
||||
hour". They are "we do not know what the meter reported that hour", and the
|
||||
two must never end up looking the same in the output. So:
|
||||
|
||||
* every request is retried with backoff;
|
||||
* a day whose read ultimately fails writes NO FILE and is recorded in the
|
||||
manifest as `"read": "FAILED"` with the error text, and the process exits
|
||||
non-zero;
|
||||
* a day that reads back successfully but EMPTY writes an empty file and is
|
||||
recorded as `"read": "OK", "samples": 0`. Empty-and-known and
|
||||
unknown are therefore distinguishable both in the manifest and by the
|
||||
presence or absence of the file;
|
||||
* before any day is fetched, the entity is proved to exist through
|
||||
/api/states/<entity>. Without that check a typo in --entity would return
|
||||
200 + `[]` for all thirty days and produce a beautifully consistent
|
||||
dataset of nothing.
|
||||
|
||||
2. GAPS ARE VISIBLE, NEVER INTERPOLATED. Nothing here carries a value forward,
|
||||
fills, resamples or averages. A recorder outage or an HA restart shows up as
|
||||
what it is: consecutive `t` values further apart than the meter's cadence.
|
||||
The controller's own ingest already reads that correctly - `meter_max_age_s`
|
||||
(30 s shipped) is exactly the age past which a held reading stops counting as
|
||||
coverage - so an hour with no telegrams replays as an hour of NO COVERAGE
|
||||
rather than as an hour of whatever the last sample said. The manifest lists
|
||||
every gap over --gap-s so a reader does not have to diff timestamps to find
|
||||
them.
|
||||
|
||||
DAY AT A TIME, AND WHY NOT SMALLER
|
||||
==================================
|
||||
One request per local day. That is gentle on a live HA (~1 MB of JSON per day)
|
||||
and it is also the only chunking that reproduces the committed files: the
|
||||
history API prepends the state as it stood AT the period start
|
||||
(`include_start_time_state`), stamped with the start instant. Splitting a day
|
||||
into six-hour chunks would therefore inject three synthetic boundary samples per
|
||||
day that no meter ever sent. Days are the natural boundary because the committed
|
||||
files are local days.
|
||||
|
||||
VERIFYING THE EXPORTER
|
||||
======================
|
||||
python3 tools/export_p1_history.py --verify scenarios/p1-net-2026-08-20.jsonl.gz
|
||||
|
||||
re-fetches that day by this exact route and compares the DECOMPRESSED bytes with
|
||||
the committed file. Byte equality of the .gz containers is not expected and not
|
||||
required: a gzip header carries an mtime, and this file writes mtime 0 so its
|
||||
own output is reproducible. Content equality is the claim that matters.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import gzip
|
||||
import hashlib
|
||||
import json
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
|
||||
try: # py3.9+; the add-on image has it
|
||||
from zoneinfo import ZoneInfo
|
||||
except ImportError: # pragma: no cover
|
||||
ZoneInfo = None # type: ignore
|
||||
|
||||
DEFAULT_URL = "http://192.168.2.158:8123"
|
||||
DEFAULT_ENTITY = "sensor.p1_meter_active_power"
|
||||
DEFAULT_TZ = "Europe/Brussels"
|
||||
# The shipped meter_max_age_s. A hole longer than this is a hole the controller
|
||||
# itself would refuse to see through, which makes it the honest gap threshold.
|
||||
DEFAULT_GAP_S = 30.0
|
||||
|
||||
|
||||
class ReadFailed(RuntimeError):
|
||||
"""The read did not happen. NOT the same thing as an empty result."""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# transport
|
||||
|
||||
|
||||
def _get(url: str, token: str, timeout: float):
|
||||
req = urllib.request.Request(url, headers={
|
||||
"Authorization": f"Bearer {token}",
|
||||
"Content-Type": "application/json",
|
||||
})
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
if resp.status != 200:
|
||||
raise ReadFailed(f"HTTP {resp.status}")
|
||||
return resp.read()
|
||||
|
||||
|
||||
def fetch_json(url: str, token: str, *, timeout: float, retries: int,
|
||||
backoff: float, log=print):
|
||||
"""GET and parse, or raise ReadFailed. Never returns a partial answer."""
|
||||
last = None
|
||||
for attempt in range(1, retries + 1):
|
||||
try:
|
||||
raw = _get(url, token, timeout)
|
||||
except urllib.error.HTTPError as exc:
|
||||
last = f"HTTP {exc.code} {exc.reason}"
|
||||
except Exception as exc: # URLError, timeout, ...
|
||||
last = f"{type(exc).__name__}: {exc}"
|
||||
else:
|
||||
try:
|
||||
return json.loads(raw.decode("utf-8")), len(raw), attempt
|
||||
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
|
||||
# A truncated body looks exactly like this. It is a failure.
|
||||
last = f"unparseable body ({len(raw)} bytes): {exc}"
|
||||
if attempt < retries:
|
||||
wait = backoff * attempt
|
||||
log(f" read failed ({last}); retry {attempt + 1}/{retries} "
|
||||
f"in {wait:.0f} s")
|
||||
time.sleep(wait)
|
||||
raise ReadFailed(last or "unknown")
|
||||
|
||||
|
||||
def assert_entity_exists(base: str, entity: str, token: str, *, timeout: float,
|
||||
retries: int, backoff: float, log=print):
|
||||
"""Prove the entity is real BEFORE trusting any empty day.
|
||||
|
||||
Without this, `--entity sensor.typo` answers 200 + [] for every day and the
|
||||
exporter cheerfully writes thirty empty files.
|
||||
"""
|
||||
url = f"{base}/api/states/{urllib.parse.quote(entity)}"
|
||||
doc, _, _ = fetch_json(url, token, timeout=timeout, retries=retries,
|
||||
backoff=backoff, log=log)
|
||||
if not isinstance(doc, dict) or doc.get("entity_id") != entity:
|
||||
raise ReadFailed(f"{entity} is not a known entity")
|
||||
return doc
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# one day
|
||||
|
||||
|
||||
def day_bounds(day: datetime, tz):
|
||||
start = datetime(day.year, day.month, day.day, tzinfo=tz)
|
||||
return start, start + timedelta(days=1)
|
||||
|
||||
|
||||
def history_url(base: str, entity: str, start: datetime, end: datetime) -> str:
|
||||
q = urllib.parse.urlencode({
|
||||
"filter_entity_id": entity,
|
||||
"end_time": end.isoformat(),
|
||||
"minimal_response": "",
|
||||
})
|
||||
return (f"{base}/api/history/period/"
|
||||
f"{urllib.parse.quote(start.isoformat())}?{q}")
|
||||
|
||||
|
||||
def rows_from(doc, entity: str):
|
||||
"""HA's shape -> [(iso_t, state_str)], plus the states that are not numbers.
|
||||
|
||||
`minimal_response` returns the first record in full and the rest as
|
||||
{state, last_changed}. An entity that went unavailable reports the string
|
||||
"unavailable" (or "unknown") where a number belongs. Those are dropped from
|
||||
the numeric series - there is no float to write and the committed schema has
|
||||
no slot for a non-number - but every one is returned here so it lands in the
|
||||
manifest. Dropping them is what MAKES the outage visible: the series simply
|
||||
has no samples across it, which is the same thing the meter falling silent
|
||||
looks like, and is what the controller's freshness rule already handles.
|
||||
"""
|
||||
if not isinstance(doc, list):
|
||||
raise ReadFailed(f"expected a list, got {type(doc).__name__}")
|
||||
if not doc:
|
||||
return [], []
|
||||
series = doc[0]
|
||||
if not isinstance(series, list):
|
||||
raise ReadFailed("history series is not a list")
|
||||
good, bad = [], []
|
||||
for rec in series:
|
||||
eid = rec.get("entity_id")
|
||||
if eid is not None and eid != entity:
|
||||
raise ReadFailed(f"unexpected entity {eid!r} in the response")
|
||||
t = rec.get("last_changed") or rec.get("last_updated")
|
||||
s = rec.get("state")
|
||||
if t is None:
|
||||
raise ReadFailed("a record carries no timestamp")
|
||||
try:
|
||||
w = float(s)
|
||||
except (TypeError, ValueError):
|
||||
bad.append({"t": t, "state": s})
|
||||
continue
|
||||
good.append((t, w))
|
||||
return good, bad
|
||||
|
||||
|
||||
def render(rows, *, newline: str) -> bytes:
|
||||
"""The committed format, exactly: `{"t": <iso>, "w": <float>}` per line.
|
||||
|
||||
json.dumps' default separators (", " / ": ") and repr of a float are what
|
||||
the two committed files contain; so is the CRLF, an artefact of the original
|
||||
one-off having been run on Windows in text mode. It is reproduced rather
|
||||
than tidied because reproducing the committed bytes is this exporter's
|
||||
verification, and gzip.open(..., 'rt') strips it either way.
|
||||
"""
|
||||
out = bytearray()
|
||||
for t, w in rows:
|
||||
out += json.dumps({"t": t, "w": w}).encode("utf-8")
|
||||
out += newline.encode("ascii")
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def gz(payload: bytes) -> bytes:
|
||||
"""gzip level 9, mtime 0 - so the same input always gives the same bytes."""
|
||||
import io
|
||||
buf = io.BytesIO()
|
||||
with gzip.GzipFile(fileobj=buf, mode="wb", compresslevel=9, mtime=0) as f:
|
||||
f.write(payload)
|
||||
return buf.getvalue()
|
||||
|
||||
|
||||
def gaps_in(rows, gap_s: float):
|
||||
out = []
|
||||
prev = None
|
||||
for t, _ in rows:
|
||||
cur = datetime.fromisoformat(t)
|
||||
if prev is not None:
|
||||
d = (cur - prev).total_seconds()
|
||||
if d > gap_s:
|
||||
out.append({"from": prev.isoformat(), "to": cur.isoformat(),
|
||||
"seconds": round(d, 3)})
|
||||
prev = cur
|
||||
return out
|
||||
|
||||
|
||||
def export_day(base, entity, token, day, tz, *, timeout, retries, backoff,
|
||||
gap_s, newline, log=print):
|
||||
"""Read one local day. Raises ReadFailed rather than returning a guess."""
|
||||
start, end = day_bounds(day, tz)
|
||||
doc, nbytes, attempts = fetch_json(
|
||||
history_url(base, entity, start, end), token,
|
||||
timeout=timeout, retries=retries, backoff=backoff, log=log)
|
||||
rows, bad = rows_from(doc, entity)
|
||||
entry = {
|
||||
"date": day.date().isoformat(),
|
||||
"window": [start.isoformat(), end.isoformat()],
|
||||
"read": "OK",
|
||||
"http_bytes": nbytes,
|
||||
"attempts": attempts,
|
||||
"samples": len(rows),
|
||||
"non_numeric_states": bad,
|
||||
"first": rows[0][0] if rows else None,
|
||||
"last": rows[-1][0] if rows else None,
|
||||
"gaps": gaps_in(rows, gap_s),
|
||||
}
|
||||
if rows:
|
||||
span = (datetime.fromisoformat(rows[-1][0])
|
||||
- datetime.fromisoformat(rows[0][0])).total_seconds()
|
||||
entry["covered_span_s"] = round(span, 1)
|
||||
entry["mean_cadence_s"] = round(span / max(1, len(rows) - 1), 3)
|
||||
return rows, entry
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# modes
|
||||
|
||||
|
||||
def do_export(args, token, tz, log=print):
|
||||
base = args.url.rstrip("/")
|
||||
out_dir = Path(args.out_dir)
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
newline = "\r\n" if args.newline == "crlf" else "\n"
|
||||
|
||||
log(f" proving {args.entity} exists before trusting any empty day")
|
||||
st = assert_entity_exists(base, args.entity, token, timeout=args.timeout,
|
||||
retries=args.retries, backoff=args.backoff,
|
||||
log=log)
|
||||
log(f" {args.entity} = {st.get('state')} "
|
||||
f"{st.get('attributes', {}).get('unit_of_measurement', '')}")
|
||||
|
||||
if args.end_date:
|
||||
last = datetime.fromisoformat(args.end_date).replace(tzinfo=tz)
|
||||
else: # yesterday: today is still being logged
|
||||
last = datetime.now(tz).replace(hour=0, minute=0, second=0,
|
||||
microsecond=0) - timedelta(days=1)
|
||||
days = [last - timedelta(days=i) for i in range(args.days - 1, -1, -1)]
|
||||
|
||||
manifest = {
|
||||
"generated": datetime.now(timezone.utc).isoformat(),
|
||||
"generator": "tools/export_p1_history.py",
|
||||
"source": f"{base}/api/history/period",
|
||||
"entity": args.entity,
|
||||
"timezone": args.tz,
|
||||
"gap_threshold_s": args.gap_s,
|
||||
"format": '{"t": <ISO-8601 UTC>, "w": <float watts>} per line, gzip',
|
||||
"days": [],
|
||||
}
|
||||
failures = []
|
||||
for day in days:
|
||||
name = f"{args.prefix}{day.date().isoformat()}.jsonl.gz"
|
||||
log(f" {name}")
|
||||
try:
|
||||
rows, entry = export_day(
|
||||
base, args.entity, token, day, tz, timeout=args.timeout,
|
||||
retries=args.retries, backoff=args.backoff, gap_s=args.gap_s,
|
||||
newline=newline, log=log)
|
||||
except ReadFailed as exc:
|
||||
# NO FILE. A missing file is the honest record of a read that did
|
||||
# not happen; an empty one would claim the meter was silent.
|
||||
entry = {"date": day.date().isoformat(), "read": "FAILED",
|
||||
"error": str(exc), "file": None}
|
||||
manifest["days"].append(entry)
|
||||
failures.append(entry)
|
||||
log(f" READ FAILED: {exc} - no file written")
|
||||
continue
|
||||
payload = render(rows, newline=newline)
|
||||
blob = gz(payload)
|
||||
(out_dir / name).write_bytes(blob)
|
||||
entry["file"] = name
|
||||
entry["bytes_gz"] = len(blob)
|
||||
entry["bytes_raw"] = len(payload)
|
||||
entry["sha256_raw"] = hashlib.sha256(payload).hexdigest()
|
||||
entry["sha256_gz"] = hashlib.sha256(blob).hexdigest()
|
||||
manifest["days"].append(entry)
|
||||
ng = len(entry.get("gaps", []))
|
||||
log(f" {entry['samples']} samples, {len(blob)} B gz, "
|
||||
f"{ng} gap(s) over {args.gap_s:g} s"
|
||||
+ (f", {len(entry['non_numeric_states'])} non-numeric state(s)"
|
||||
if entry["non_numeric_states"] else ""))
|
||||
if day is not days[-1]:
|
||||
time.sleep(args.delay)
|
||||
|
||||
ok = [d for d in manifest["days"] if d["read"] == "OK"]
|
||||
manifest["summary"] = {
|
||||
"days_requested": len(days),
|
||||
"days_read": len(ok),
|
||||
"days_failed": len(failures),
|
||||
"samples": sum(d["samples"] for d in ok),
|
||||
"bytes_gz": sum(d["bytes_gz"] for d in ok),
|
||||
"empty_days": [d["date"] for d in ok if d["samples"] == 0],
|
||||
}
|
||||
# newline="\n" explicitly: write_text() on Windows emits CRLF, and
|
||||
# the manifest would then differ from a Linux run of the same export
|
||||
# for no reason visible in a diff.
|
||||
with open(args.manifest or (out_dir / "manifest.json"), "w",
|
||||
encoding="utf-8", newline="\n") as fh:
|
||||
fh.write(json.dumps(manifest, indent=2) + "\n")
|
||||
log(f" {manifest['summary']['days_read']}/{len(days)} days, "
|
||||
f"{manifest['summary']['samples']} samples, "
|
||||
f"{manifest['summary']['bytes_gz'] / 1e6:.2f} MB gz")
|
||||
if failures:
|
||||
log(f" {len(failures)} DAY(S) NOT READ - the dataset is incomplete "
|
||||
f"and says so: " + ", ".join(d["date"] for d in failures))
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
def do_verify(args, token, tz, log=print):
|
||||
"""Re-export a committed day by this route and diff it against the file."""
|
||||
base = args.url.rstrip("/")
|
||||
newline = "\r\n" if args.newline == "crlf" else "\n"
|
||||
rc = 0
|
||||
for target in args.verify:
|
||||
path = Path(target)
|
||||
stamp = path.name.split(".")[0].rsplit("-", 3)[-3:]
|
||||
day = datetime(int(stamp[0]), int(stamp[1]), int(stamp[2]), tzinfo=tz)
|
||||
log(f" {path.name} (local day {day.date().isoformat()})")
|
||||
committed = gzip.decompress(path.read_bytes())
|
||||
rows, entry = export_day(base, args.entity, token, day, tz,
|
||||
timeout=args.timeout, retries=args.retries,
|
||||
backoff=args.backoff, gap_s=args.gap_s,
|
||||
newline=newline, log=log)
|
||||
fresh = render(rows, newline=newline)
|
||||
if fresh == committed:
|
||||
log(f" IDENTICAL: {len(fresh)} bytes, {entry['samples']} "
|
||||
f"samples, sha256 {hashlib.sha256(fresh).hexdigest()[:16]}")
|
||||
continue
|
||||
rc = 1
|
||||
cl = committed.split(newline.encode())
|
||||
fl = fresh.split(newline.encode())
|
||||
log(f" DIFFERS: committed {len(committed)} B / {len(cl)} lines, "
|
||||
f"re-export {len(fresh)} B / {len(fl)} lines")
|
||||
shown = 0
|
||||
for i in range(max(len(cl), len(fl))):
|
||||
a = cl[i] if i < len(cl) else b"<absent>"
|
||||
b = fl[i] if i < len(fl) else b"<absent>"
|
||||
if a != b:
|
||||
log(f" line {i}: committed {a!r}")
|
||||
log(f" re-export {b!r}")
|
||||
shown += 1
|
||||
if shown >= args.max_diff:
|
||||
log(f" ... stopping after {shown} differing lines")
|
||||
break
|
||||
return rc
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
p = argparse.ArgumentParser(description=__doc__.split("\n")[0])
|
||||
p.add_argument("--url", default=DEFAULT_URL)
|
||||
p.add_argument("--entity", default=DEFAULT_ENTITY)
|
||||
p.add_argument("--token", help="bearer token; else $HASS_TOKEN")
|
||||
p.add_argument("--tz", default=DEFAULT_TZ,
|
||||
help="local timezone that defines a day boundary")
|
||||
p.add_argument("--days", type=int, default=30)
|
||||
p.add_argument("--end-date", help="last local day to export (YYYY-MM-DD); "
|
||||
"default yesterday")
|
||||
p.add_argument("--out-dir", default=".")
|
||||
p.add_argument("--prefix", default="p1-net-")
|
||||
p.add_argument("--manifest", help="path for the manifest JSON")
|
||||
p.add_argument("--gap-s", type=float, default=DEFAULT_GAP_S)
|
||||
p.add_argument("--newline", choices=("crlf", "lf"), default="crlf",
|
||||
help="line terminator; crlf reproduces the committed files")
|
||||
p.add_argument("--timeout", type=float, default=120.0)
|
||||
p.add_argument("--retries", type=int, default=4)
|
||||
p.add_argument("--backoff", type=float, default=5.0)
|
||||
p.add_argument("--delay", type=float, default=2.0,
|
||||
help="pause between days - this is a live HA")
|
||||
p.add_argument("--verify", nargs="+", metavar="FILE",
|
||||
help="re-export these committed days and diff them")
|
||||
p.add_argument("--max-diff", type=int, default=20)
|
||||
args = p.parse_args(argv)
|
||||
|
||||
token = args.token or __import__("os").environ.get("HASS_TOKEN")
|
||||
if not token:
|
||||
print("no token: pass --token or set HASS_TOKEN", file=sys.stderr)
|
||||
return 2
|
||||
if ZoneInfo is None:
|
||||
print("zoneinfo unavailable", file=sys.stderr)
|
||||
return 2
|
||||
tz = ZoneInfo(args.tz)
|
||||
try:
|
||||
if args.verify:
|
||||
return do_verify(args, token, tz)
|
||||
return do_export(args, token, tz)
|
||||
except ReadFailed as exc:
|
||||
print(f"read failed: {exc}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,303 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run TARIFF-01's and SAFETY-07's AC 3 over a directory of `p1-net-*.jsonl.gz`.
|
||||
|
||||
AC 3, both cards, same words: *"monthly peak held under the soft limit in
|
||||
>= 95 % of intervals over a 30-day simulation replay."* Both cards merged with
|
||||
it UNMET because two days of capture existed. DATA-01 exported thirty. This is
|
||||
the thing that turns those thirty days into the number.
|
||||
|
||||
cd goodwe_controller
|
||||
python3 tools/replay_ac3.py scenarios/p1-net-30d-2026-08
|
||||
|
||||
WHAT IT MEASURES
|
||||
================
|
||||
Two runs over the same telegrams, one file at a time in date order into a
|
||||
SHARED peak store, so the 15-minute blocks accumulate into a real MONTH rather
|
||||
than into 30 disconnected days:
|
||||
|
||||
UNCONTROLLED the recorded meter as it happened. Nothing was shaving peaks on
|
||||
this house during August 2026, so this is not a simulation of
|
||||
anything - it is what the month did. It is TARIFF-01's AC 3.
|
||||
GUARDED the same month with `peakguard.decide()` in the loop and the
|
||||
battery it asks for modelled, closed-loop: the meter the guard
|
||||
sees is the recorded net MINUS the discharge it obtained. That
|
||||
is SAFETY-07's AC 3.
|
||||
|
||||
Both numbers are `quarters <= peak_cap_w` over `quarters closed`, with the cap
|
||||
read out of config.yaml rather than typed. Nothing here exits non-zero on 95 %:
|
||||
a measurement below the criterion is a finding about the house and the guard,
|
||||
and this tool's job is to report it, not to be graded by it.
|
||||
|
||||
FAITHFULNESS TO THE SUITE
|
||||
=========================
|
||||
The replay loop is `test_peakguard.py` part 6's, with `pack=None` (SoC pinned
|
||||
above the floor, so SAFETY-24's energy floor is not the thing deciding the
|
||||
answer). It is not imported from there because that file runs its whole suite on
|
||||
import. The duplication is checked instead of asserted: pointed at `scenarios/`
|
||||
- the two committed days - this tool must reproduce part 6's published totals,
|
||||
and `--expect` makes that a hard check.
|
||||
|
||||
python3 tools/replay_ac3.py scenarios --expect 189/190
|
||||
|
||||
GAPS
|
||||
====
|
||||
Nothing is interpolated. A hole in the export is a hole here: the shipped
|
||||
`meter_max_age_s` is what decides whether a held reading still counts as
|
||||
coverage, so an hour with no telegrams closes no quarters and contributes to
|
||||
neither numerator nor denominator. `--report-gaps` lists them.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import gzip
|
||||
import json
|
||||
import sys
|
||||
import tempfile
|
||||
from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
|
||||
HERE = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(HERE))
|
||||
|
||||
from app.p1 import P1Ingest, make_sample, split_signed # noqa: E402
|
||||
from app.peakguard import decide # noqa: E402
|
||||
from app.tariff import PeakStore, PeakTracker, month_key # noqa: E402
|
||||
|
||||
|
||||
def read_option(key: str):
|
||||
"""The shipped default for `key`, out of config.yaml.
|
||||
|
||||
⚠️ `peak_cap_w:` appears TWICE in config.yaml - once under `options:` with
|
||||
the default, once under `schema:` as `int(500,15000)`. The FIRST match is
|
||||
the options block, which is the number an install actually lands on; taking
|
||||
the last would try to float() the schema line. Same rule as the suites.
|
||||
"""
|
||||
for line in (HERE / "config.yaml").read_text(encoding="utf-8").splitlines():
|
||||
if line.strip().startswith(f"{key}:"):
|
||||
try:
|
||||
return float(line.split(":", 1)[1].strip())
|
||||
except ValueError:
|
||||
continue
|
||||
return None
|
||||
|
||||
|
||||
def read_schema_max(key: str):
|
||||
for line in (HERE / "config.yaml").read_text(encoding="utf-8").splitlines():
|
||||
s = line.strip()
|
||||
if s.startswith(f"{key}:") and "(" in s and ")" in s:
|
||||
try:
|
||||
return float(s[s.index("(") + 1:s.index(")")].split(",")[-1])
|
||||
except ValueError:
|
||||
continue
|
||||
return None
|
||||
|
||||
|
||||
def sample(t: datetime, net_w: float):
|
||||
"""One telegram through the SHIPPED sign derivation - the suites' helper."""
|
||||
imp, exp = split_signed(net_w)
|
||||
return make_sample("ha_signed", imp, exp, phases=1,
|
||||
ingest_ts=t, ingest_mono=t.timestamp())
|
||||
|
||||
|
||||
def rows(path):
|
||||
"""Every sample in one file, in order, with the gaps left as gaps."""
|
||||
with gzip.open(path, "rt", encoding="utf-8") as fh:
|
||||
for line in fh:
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
row = json.loads(line)
|
||||
yield datetime.fromisoformat(row["t"]), float(row["w"])
|
||||
|
||||
|
||||
def replay(paths, *, guarded: bool, cap_w, max_w, max_hold_s, cycle_s,
|
||||
soc, soc_floor, tmp: Path, gap_s: float):
|
||||
"""test_peakguard.py part 6's loop, over an arbitrary number of days.
|
||||
|
||||
⚠️ ONE STORE, AND AN INGEST THAT IS RESET ONLY AT A REAL DISCONTINUITY.
|
||||
Neither half of that is a detail.
|
||||
|
||||
The STORE is shared across the files because "monthly peak" is what AC 3 is
|
||||
about and a per-file store cannot express it.
|
||||
|
||||
The INGEST is carried across a file boundary when the next file's first
|
||||
telegram follows the previous file's last one within `gap_s`, and rebuilt
|
||||
when it does not. Both failure modes are real and were measured here:
|
||||
|
||||
* carried across a genuine hole, one averager manufactures quarters out of
|
||||
nothing - streamed straight through the three-day gap between the two
|
||||
committed days this reported 285/286 where the suite reports 189/190;
|
||||
* rebuilt at every file, each day's LAST quarter (21:45-22:00 UTC) never
|
||||
closes, because it closes only when a telegram past the boundary
|
||||
arrives and that telegram is in the next file. That is why the suite's
|
||||
two days give 95 quarters each rather than 96, and it silently discards
|
||||
one quarter per day - 30 of them over a month.
|
||||
|
||||
So contiguous days are joined and non-contiguous ones are not, which gives
|
||||
189/190 on the committed pair and a full 96 quarters a day on a run of
|
||||
consecutive days.
|
||||
"""
|
||||
store = PeakStore(str(tmp / f"ac3-{'g' if guarded else 'u'}.db"))
|
||||
tracker = PeakTracker(store, cap_w=cap_w)
|
||||
engaged = blind = saturated = cycles = n = joins = 0
|
||||
wh = 0.0
|
||||
gaps = []
|
||||
months = set()
|
||||
ing = None
|
||||
batt = 0.0 # delivered discharge, W (positive)
|
||||
next_cycle = None
|
||||
prev_t = None
|
||||
for path in paths:
|
||||
for i, (t, w) in enumerate(rows(path)):
|
||||
if i == 0:
|
||||
contiguous = (prev_t is not None
|
||||
and 0 <= (t - prev_t).total_seconds() <= gap_s)
|
||||
if contiguous:
|
||||
joins += 1
|
||||
else:
|
||||
if ing is not None:
|
||||
months |= {month_key(b.start) for b in ing.blocks}
|
||||
ing = P1Ingest(phases=1, max_age_s=max_hold_s)
|
||||
batt = 0.0
|
||||
next_cycle = None
|
||||
prev_t = None
|
||||
if prev_t is not None:
|
||||
dt = (t - prev_t).total_seconds()
|
||||
if dt > gap_s:
|
||||
gaps.append((prev_t, t, dt))
|
||||
wh += batt * dt / 3600.0
|
||||
prev_t = t
|
||||
n += 1
|
||||
ing.submit(sample(t, w - batt))
|
||||
tracker.sync(ing)
|
||||
if next_cycle is not None and t < next_cycle:
|
||||
continue
|
||||
next_cycle = t + timedelta(seconds=cycle_s)
|
||||
cycles += 1
|
||||
d = decide(PeakTracker.project(ing.averager, t), batt_w=batt,
|
||||
cap_w=cap_w, reading=tracker.peak(t),
|
||||
achievable_w=max_w, max_hold_s=max_hold_s, soc=soc,
|
||||
soc_floor=soc_floor, frozen=False)
|
||||
if d.engaged:
|
||||
blind += 1 if d.blind else 0
|
||||
engaged += 0 if d.blind else 1
|
||||
saturated += 1 if d.saturated else 0
|
||||
if not guarded:
|
||||
continue
|
||||
engaged_floor = d.engaged and d.floor_w is not None
|
||||
batt = max(0.0, d.floor_w) if engaged_floor else 0.0
|
||||
if ing is not None:
|
||||
months |= {month_key(b.start) for b in ing.blocks}
|
||||
out = {m: dict(store.blocks_in(m)) for m in sorted(months)}
|
||||
return {"months": out, "samples": n, "cycles": cycles, "joins": joins,
|
||||
"engaged": engaged, "blind": blind, "saturated": saturated,
|
||||
"wh": wh, "gaps": gaps}
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
p = argparse.ArgumentParser(description=__doc__.split("\n")[0])
|
||||
p.add_argument("directory", help="a directory of p1-net-*.jsonl.gz")
|
||||
p.add_argument("--glob", default="p1-net-*.jsonl.gz")
|
||||
p.add_argument("--expect", help="UNDER/TOTAL the uncontrolled run must "
|
||||
"reproduce, e.g. 189/190")
|
||||
p.add_argument("--report-gaps", action="store_true")
|
||||
args = p.parse_args(argv)
|
||||
|
||||
paths = sorted(Path(args.directory).glob(args.glob))
|
||||
if not paths:
|
||||
print(f"no {args.glob} under {args.directory}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
CAP_W = read_option("peak_cap_w")
|
||||
MAX_HOLD_S = read_option("meter_max_age_s")
|
||||
CYCLE_S = read_option("cycle_floor_s")
|
||||
SOC_FLOOR = read_option("peak_soc_floor")
|
||||
LIVE_MAX_W = read_schema_max("max_w")
|
||||
if None in (CAP_W, MAX_HOLD_S, CYCLE_S, SOC_FLOOR, LIVE_MAX_W):
|
||||
print("could not read the shipped options out of config.yaml",
|
||||
file=sys.stderr)
|
||||
return 2
|
||||
# The suites' fixture SoC: comfortably above the floor, so SAFETY-24's
|
||||
# energy floor is not what decides an AC 3 that is not about it.
|
||||
SOC_OK = SOC_FLOOR + 30.0
|
||||
|
||||
days = len(paths)
|
||||
span_first = span_last = None
|
||||
print(f"AC 3 - {days} file(s) from {args.directory}")
|
||||
print(f" shipped: peak_cap_w {CAP_W:.0f} W, meter_max_age_s "
|
||||
f"{MAX_HOLD_S:.0f} s, cycle_floor_s {CYCLE_S:.0f} s, "
|
||||
f"max_w schema ceiling {LIVE_MAX_W:.0f} W")
|
||||
|
||||
tmp = Path(tempfile.mkdtemp(prefix="goodwe-ac3-"))
|
||||
kw = dict(cap_w=CAP_W, max_w=LIVE_MAX_W, max_hold_s=MAX_HOLD_S,
|
||||
cycle_s=CYCLE_S, soc=SOC_OK, soc_floor=SOC_FLOOR, tmp=tmp,
|
||||
gap_s=MAX_HOLD_S)
|
||||
u = replay(paths, guarded=False, **kw)
|
||||
g = replay(paths, guarded=True, **kw)
|
||||
|
||||
def flat(run):
|
||||
return [w for m in run["months"].values() for w in m.values()]
|
||||
|
||||
uq, gq = flat(u), flat(g)
|
||||
u_under = sum(1 for w in uq if w <= CAP_W)
|
||||
g_under = sum(1 for w in gq if w <= CAP_W)
|
||||
u_pct = 100.0 * u_under / max(len(uq), 1)
|
||||
g_pct = 100.0 * g_under / max(len(gq), 1)
|
||||
|
||||
print(f" {u['samples']} samples, {len(uq)} closed quarters "
|
||||
f"({len(uq) / 96.0:.1f} days' worth), {days - u['joins']} contiguous "
|
||||
f"run(s) of days")
|
||||
for m in sorted(u["months"]):
|
||||
mu = u["months"][m]
|
||||
mg = g["months"].get(m, {})
|
||||
print(f" month {m}: {len(mu)} quarters, peak "
|
||||
f"{max(mu.values()):.1f} W uncontrolled -> "
|
||||
f"{max(mg.values()) if mg else 0.0:.1f} W guarded")
|
||||
print()
|
||||
print(f" TARIFF-01 AC 3 (UNCONTROLLED, what the month did): "
|
||||
f"{u_under}/{len(uq)} = {u_pct:.2f} % of quarters at or under "
|
||||
f"{CAP_W:.0f} W")
|
||||
print(f" SAFETY-07 AC 3 (GUARDED, peakguard.decide in the loop): "
|
||||
f"{g_under}/{len(gq)} = {g_pct:.2f} %")
|
||||
print(f" guard cycles {g['cycles']}, engaged on the measurement "
|
||||
f"{g['engaged']}, blind {g['blind']}, saturated {g['saturated']}, "
|
||||
f"modelled discharge {g['wh']:.0f} Wh")
|
||||
# AC 3'S WORDS SUPPORT TWO READINGS AND BOTH ARE REPORTED, because they
|
||||
# disagree on this data and picking the flattering one is the failure mode
|
||||
# this whole card exists to correct.
|
||||
u_peak = max(uq) if uq else 0.0
|
||||
g_peak = max(gq) if gq else 0.0
|
||||
print(f" reading A - PERCENTAGE OF INTERVALS under the cap (the reading "
|
||||
f"test_tariff.py part 9 already uses): uncontrolled {u_pct:.2f} %, "
|
||||
f"guarded {g_pct:.2f} %")
|
||||
print(f" reading B - THE MONTHLY PEAK ITSELF under the cap: "
|
||||
f"{u_peak:.1f} W uncontrolled -> {g_peak:.1f} W guarded, against a "
|
||||
f"{CAP_W:.0f} W soft limit "
|
||||
f"({'under' if g_peak <= CAP_W else 'STILL OVER'})")
|
||||
print(f" verdict at the 95 % criterion: uncontrolled "
|
||||
f"{'MEETS' if u_pct >= 95.0 else 'DOES NOT MEET'}, guarded "
|
||||
f"{'MEETS' if g_pct >= 95.0 else 'DOES NOT MEET'} "
|
||||
f"({len(uq) / 96.0:.1f} days of replay)")
|
||||
|
||||
if u["gaps"]:
|
||||
tot = sum(d for _, _, d in u["gaps"])
|
||||
print(f" gaps over {MAX_HOLD_S:.0f} s: {len(u['gaps'])}, "
|
||||
f"{tot:.0f} s total - NOT interpolated; the quarters they fall in "
|
||||
f"simply carry less coverage")
|
||||
if args.report_gaps:
|
||||
for a, b, d in u["gaps"]:
|
||||
print(f" {a.isoformat()} -> {b.isoformat()} {d:.0f} s")
|
||||
|
||||
if args.expect:
|
||||
want_u, want_t = (int(x) for x in args.expect.split("/"))
|
||||
ok = (u_under, len(uq)) == (want_u, want_t)
|
||||
print(f" reproduces test_peakguard.py part 6's published "
|
||||
f"{want_u}/{want_t}: {'yes' if ok else 'NO'} "
|
||||
f"(got {u_under}/{len(uq)})")
|
||||
return 0 if ok else 1
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Reference in New Issue
Block a user