diff --git a/goodwe_controller/CHANGELOG.md b/goodwe_controller/CHANGELOG.md index 5aff1a4..ef47f8b 100644 --- a/goodwe_controller/CHANGELOG.md +++ b/goodwe_controller/CHANGELOG.md @@ -2,6 +2,45 @@ ## Unreleased +**Thirty days of the real meter, and the acceptance criterion three cards +deferred is now a number (DATA-01).** TARIFF-01 and SAFETY-07 both merged with +AC 3 - *"monthly peak held under the soft limit in >= 95 % of intervals over a +30-day simulation replay"* - honestly unmet, on the belief that only two days of +capture existed and nothing longer could be produced. That belief came from a +true statement about the past being read as one about the future: HA's recorder +still held the whole month. `scenarios/p1-net-30d-2026-08/` is 2026-08-01 to +2026-08-30, **485,841 samples, 30/30 days read, 0 failed, 4.13 MB gzipped**, in +the same schema as the two days that were already there. + +`tools/export_p1_history.py` is the exporter, committed and runnable - the +2026-08-25 one-off was not, and that is exactly why the data was thought +unobtainable. It reads day-at-a-time, retries, and **never lets a failed read +look like an empty hour**: a day whose read fails writes no file, is recorded as +`"read": "FAILED"`, and the run exits non-zero; the entity is proved to exist +before any empty day is believed. Nothing is interpolated, so the month's 13 +gaps (1291 s, 99.95 % covered) and its 3 `unavailable` samples stay visible. +`--verify` re-fetches the two committed days by the same route and reproduces +them **byte for byte** after decompression. + +`tools/replay_ac3.py` runs the month. Over 30 days the criterion measures +**98.89 % uncontrolled and 98.99 % guarded**, so it is met on that reading - and +the uncontrolled figure is the finding: a month with no controller at all passes +it. On the other reading of the same sentence it is not met - the guard takes +August's peak from **5029.8 W to 4107.3 W**, still over the 3500 W soft limit, +because once a month peak is set `CEILING_MONTH_PEAK` defends that peak rather +than the cap. Neither number is hidden; both are in that directory's README. + +The size decision is committed and measured, not absorbed: a `git bundle` of +the whole history goes from **1.58 MB to 5.28 MB** (+3.70 MB, x3.3), which a +Home Assistant supervisor clones on every install. Downsampling was not +available - `meter_max_age_s` is 30 s, so any coarser resample converts real +coverage into modelled meter-gap and moves the number being measured. + +The thirty files sit in a SUBDIRECTORY on purpose: `test_tariff.py` part 9 and +`test_peakguard.py` part 6 glob `scenarios/p1-net-*.jsonl.gz` non-recursively, +so CI keeps replaying the two small days and does not grow ~50 s per PR. Both +files' honesty notes are corrected to point at the month. + **OPT-04's AC4 is met: twelve months of real EPEX BE day-ahead prices are now in the repository, and the strategy has been replayed against them (DATA-02).** AC4 asked for "net-positive across 12 months of replayed prices". OPT-04 shipped diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/README.md b/goodwe_controller/scenarios/p1-net-30d-2026-08/README.md new file mode 100644 index 0000000..a612d53 --- /dev/null +++ b/goodwe_controller/scenarios/p1-net-30d-2026-08/README.md @@ -0,0 +1,167 @@ +# 30 days of P1 net power — 2026-08-01 … 2026-08-30 + +`sensor.p1_meter_active_power` out of Home Assistant's recorder, one gzipped +JSONL file per **local** day (Europe/Brussels), same schema as the two files one +directory up: + + {"t": "2026-08-22T22:00:00+00:00", "w": -25.0} + +**485,841 samples, 30/30 days read, 0 days failed, 4.13 MB gzipped.** +Produced by `tools/export_p1_history.py`; `manifest.json` here carries the +per-day counts, sha256 of every file, and every gap. + + python3 tools/export_p1_history.py --days 30 --end-date 2026-08-30 \ + --out-dir scenarios/p1-net-30d-2026-08 + +## Why this exists + +[[TARIFF-01]] AC 3 and [[SAFETY-07]] AC 3 both ask for *"monthly peak held under +the soft limit in >= 95 % of intervals over a 30-day simulation replay"*, and +both were merged with the criterion honestly unmet because only two days of +capture existed. `test_tariff.py`'s docstring still recorded the reason as +"nothing longer was ever captured" — which was true about the past and was +quietly read as a statement about the future. It was not one. Probed on +2026-08-31, HA's recorder answered a 20-minute window with 230 samples at 30 +days back and 0 at 45, so the whole month was still there and was exported the +same day. **The window rolls: 2026-08-01 leaves the recorder around 2026-09-15.** + +## The size decision — COMMITTED, deliberately, at 4.13 MB + +Three options were on the table. This one was taken and the cost is stated +rather than absorbed, and it is measured rather than estimated - a +`git bundle` of the whole history before and after this commit: + + release/1.0 @ 86c43eb 1.58 MB + DATA-01 5.28 MB +3.70 MB, x3.3 + +A full clone of this repository therefore more than triples, and a Home +Assistant supervisor clones it on every add-on install and update. That is the +price; the paragraphs below are why it was judged worth paying rather than +avoided by making the data worse or making it disappear. + +* **Downsampling is not available.** It is not a matter of taste: the shipped + `meter_max_age_s` is **30 s**, which is the age past which the controller's + own ingest stops counting a held reading as coverage. Any resample coarser + than 30 s therefore converts real coverage into modelled meter-gap and changes + the very number AC 3 measures. Resampling at or under 30 s saves almost + nothing against a 5 s cadence. +* **"Outside the repo with a pointer" is what already happened, and it is why + this card exists.** The two committed days came from the infra repo's private + `sim/scenarios/`, the one-off export script was thrown away, and three cards + then deferred an acceptance criterion on the belief that the data could not be + produced again. `test_tariff.py` states the general rule verbatim: *a + regression fixture CI cannot see is not a regression fixture.* CI's token is + minted for this repository only. +* **So: committed** — but in a SUBDIRECTORY, which is load-bearing. Both + `test_tariff.py` part 9 and `test_peakguard.py` part 6 do + `(HERE / "scenarios").glob("p1-net-*.jsonl.gz")`, which does not recurse. Drop + these thirty files beside the two and CI grows sixty full-day + replays - about 50 s of pure compute - on every PR forever, `test_tariff.py`'s + `check(... replay_days < 30)` starts failing, and both files' honesty notes + become false. In `p1-net-30d-2026-08/` they are visible to CI, invisible to + the globs, and the two committed days keep their job as the small fixtures the + suites grade against. + +They are also written once and never edited, so git stores one already-deflated +blob per day with no delta churn. + +## Gaps are visible, never interpolated + +Nothing is filled, carried forward or resampled. A hole in the recorder is a +hole here — consecutive `t` values further apart than the cadence — which is +exactly what the controller's freshness rule already reads as *no coverage*, so +a missing hour replays as a missing hour rather than as an hour of whatever the +last sample said. + +**13 gaps over 30 s, 1291 s in total: 99.950 % of the month is covered.** +All of them are listed in `manifest.json` and by +`python3 tools/replay_ac3.py --report-gaps`. The longest is 263 s on +2026-08-11. + +Three samples in the month carried the state `unavailable` rather than a number +(2026-08-18 02:03:54Z and 04:10:41Z, 2026-08-25 01:52:02Z). There is no float to +write and the committed schema has no slot for a non-number, so they are absent +from the series and recorded individually in `manifest.json` under +`non_numeric_states`. Each one sits inside a gap that is already listed, which +is the honest rendering: the meter was not reporting a power then. + +**A failed read is never an empty hour.** If a day's read fails, the exporter +writes NO FILE, records `"read": "FAILED"` with the error, and exits non-zero — +so a missing day is unreadable-and-known rather than silently zero. The entity +is also proved to exist through `/api/states` before any day is trusted, because +a typo would otherwise answer `200 []` thirty times and produce a beautifully +consistent dataset of nothing. On this run: 30 read, 0 failed, 0 empty. + +## AC 3, measured + + cd goodwe_controller + python3 tools/replay_ac3.py scenarios/p1-net-30d-2026-08 + + 485841 samples, 2879 closed quarters (30.0 days' worth) + month 2026-08: peak 5029.8 W uncontrolled -> 4107.3 W guarded + + TARIFF-01 AC 3 (UNCONTROLLED): 2847/2879 = 98.89 % of quarters at or under 3500 W + SAFETY-07 AC 3 (GUARDED): 2850/2879 = 98.99 % + guard cycles 143810, engaged on the measurement 1542, blind 195, + saturated 0, modelled discharge 8060 Wh + +2879 rather than 2880 quarters: the last quarter of 2026-08-30 closes only when +a telegram past its boundary arrives, and that telegram is 2026-08-31's. + +### What the numbers actually say — three findings, not a pass + +1. **AC 3 is met on the ">= 95 % of intervals" reading, and the criterion turns + out to be nearly insensitive.** The UNCONTROLLED month — no controller, no + peak shaving, nothing running — already scores 98.89 %. A criterion that a + house passes by doing nothing does not discriminate between a working guard + and an absent one. That is a finding about the acceptance criterion, and it + is reported rather than filed as a win. +2. **On the other reading of the same sentence, AC 3 is NOT met.** "Monthly peak + held under the soft limit" taken as *the peak itself* fails: the guard pulls + August's peak from **5029.8 W to 4107.3 W** (-922.5 W, -18.3 %), which is + real money on a Belgian capacity tariff, but 4107.3 W is still over the + 3500 W soft limit. Both readings are printed by the tool; neither is hidden. +3. **The guard stops defending the soft limit once the month's peak is set, and + that is by design.** `quarter_ceiling_w` uses `CEILING_MONTH_PEAK`: the + billed quantity is the month's maximum, so a quarter under an + already-recorded peak costs nothing extra and the guard does not spend the + battery on it. The month's 32 over-cap quarters fall on four nights + (2026-08-17, -18, -20, -21); **31 of them are pulled down** — 7494 W of + quarter-average removed in total, 8060 Wh of modelled discharge — but only + **3 land under 3500 W**, because by then the ceiling being defended is the + running month peak (~4107 W), not the soft limit. The single untouched + quarter, 2026-08-18T08:15Z at 4039.7 W, is the same rule seen from the other + side: the running peak was already 4833 W, so 4039.7 W added nothing to the + bill and the guard correctly stayed out. AC 3's wording asks the guard for + something its own tariff logic deliberately does not optimise. + +Also worth an eye: **195 blind engagements** over 143,810 cycles — the guard +acting on a projection it could not fully cover. On the two committed days that +count was 5. It scales with the gaps above. + +## Verification of the exporter + +`tools/export_p1_history.py --verify` re-fetches a committed day by the exact +route used here and diffs it against the file: + + python3 tools/export_p1_history.py \ + --verify scenarios/p1-net-2026-08-20.jsonl.gz \ + scenarios/p1-net-2026-08-23.jsonl.gz + +Both days come back **byte-identical after decompression** — 865,024 B / +15,600 samples and 894,387 B / 16,351 samples — and the copies of those two days +inside this directory are byte-identical to the committed ones too. The gzip +CONTAINERS differ by design: a gzip header carries an mtime and a stored +filename, and this exporter writes mtime 0 so its own output is reproducible, +where the 2026-08-25 one-off wrote a wall-clock mtime. Content equality is the +claim; container equality would only assert that two files were written at the +same second. + +`tools/replay_ac3.py` is checked the same way rather than trusted: pointed at +`scenarios/` it must reproduce `test_peakguard.py` part 6's published totals, +and `--expect` makes it a hard check. + + python3 tools/replay_ac3.py scenarios --expect 189/190 + +reproduces 189/190 quarters, the 5029.8 W billed quarter, the 3485.3 W guarded +result and the 878 Wh of modelled discharge that part 6 prints. diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/manifest.json b/goodwe_controller/scenarios/p1-net-30d-2026-08/manifest.json new file mode 100644 index 0000000..26cb94f --- /dev/null +++ b/goodwe_controller/scenarios/p1-net-30d-2026-08/manifest.json @@ -0,0 +1,767 @@ +{ + "generated": "2026-08-31T08:42:40.614548+00:00", + "generator": "tools/export_p1_history.py", + "source": "http://192.168.2.158:8123/api/history/period", + "entity": "sensor.p1_meter_active_power", + "timezone": "Europe/Brussels", + "gap_threshold_s": 30.0, + "format": "{\"t\": , \"w\": } per line, gzip", + "days": [ + { + "date": "2026-08-01", + "window": [ + "2026-08-01T00:00:00+02:00", + "2026-08-02T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1128060, + "attempts": 1, + "samples": 16511, + "non_numeric_states": [], + "first": "2026-07-31T22:00:00+00:00", + "last": "2026-08-01T21:59:56.268499+00:00", + "gaps": [], + "covered_span_s": 86396.3, + "mean_cadence_s": 5.233, + "file": "p1-net-2026-08-01.jsonl.gz", + "bytes_gz": 140531, + "bytes_raw": 913203, + "sha256_raw": "120de2a0b7eb552281dac4bc1f8fd76d947ba72d1f5b426d9dcdd85ec103bd06", + "sha256_gz": "e09bdee349dd3668a073b7fd77974e639c269ec37e73e81a9706cadde083f064" + }, + { + "date": "2026-08-02", + "window": [ + "2026-08-02T00:00:00+02:00", + "2026-08-03T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1115387, + "attempts": 1, + "samples": 16303, + "non_numeric_states": [], + "first": "2026-08-01T22:00:00+00:00", + "last": "2026-08-02T21:59:55.191360+00:00", + "gaps": [], + "covered_span_s": 86395.2, + "mean_cadence_s": 5.3, + "file": "p1-net-2026-08-02.jsonl.gz", + "bytes_gz": 141404, + "bytes_raw": 903234, + "sha256_raw": "9ab4891c0abf9d621fa528848fc3a874687843804768e8aeab6a88f7bb04b679", + "sha256_gz": "94774139b3e19d04bcda3ac5ea988dd4d645e99004f90b917c6b0bf3fbac21cb" + }, + { + "date": "2026-08-03", + "window": [ + "2026-08-03T00:00:00+02:00", + "2026-08-04T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1123668, + "attempts": 1, + "samples": 16424, + "non_numeric_states": [], + "first": "2026-08-02T22:00:00+00:00", + "last": "2026-08-03T21:59:55.255808+00:00", + "gaps": [], + "covered_span_s": 86395.3, + "mean_cadence_s": 5.261, + "file": "p1-net-2026-08-03.jsonl.gz", + "bytes_gz": 144202, + "bytes_raw": 909942, + "sha256_raw": "13f8dd904a31b5b67fcfde04f6afc49c34c3a9504617278d79d87516fcd6441f", + "sha256_gz": "e1bc0cb0e271cbc8d896aee97aa0f5c18e49e78905d1aaf137904a768047c7bc" + }, + { + "date": "2026-08-04", + "window": [ + "2026-08-04T00:00:00+02:00", + "2026-08-05T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1108582, + "attempts": 1, + "samples": 16286, + "non_numeric_states": [], + "first": "2026-08-03T22:00:00+00:00", + "last": "2026-08-04T21:59:58.729138+00:00", + "gaps": [], + "covered_span_s": 86398.7, + "mean_cadence_s": 5.305, + "file": "p1-net-2026-08-04.jsonl.gz", + "bytes_gz": 136505, + "bytes_raw": 896650, + "sha256_raw": "a65cd9d1ceda61e5a40043edb401c2154feea2b36bd3221710df808d778eabce", + "sha256_gz": "c19b19ac890f4caad4140cb27d2fe6f3e972545f664127e5d9f457e711ad8c88" + }, + { + "date": "2026-08-05", + "window": [ + "2026-08-05T00:00:00+02:00", + "2026-08-06T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1136556, + "attempts": 1, + "samples": 16655, + "non_numeric_states": [], + "first": "2026-08-04T22:00:00+00:00", + "last": "2026-08-05T21:59:56.189152+00:00", + "gaps": [], + "covered_span_s": 86396.2, + "mean_cadence_s": 5.188, + "file": "p1-net-2026-08-05.jsonl.gz", + "bytes_gz": 142561, + "bytes_raw": 919827, + "sha256_raw": "dd327685c6f322bfaa49bc5bf228301e790e5a96fd7c23c487b845e44a962db4", + "sha256_gz": "5ff7a1336c49ba38b8058820181dd8ed9c2b8db41b5a3a52412a5a3d8c515332" + }, + { + "date": "2026-08-06", + "window": [ + "2026-08-06T00:00:00+02:00", + "2026-08-07T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1124419, + "attempts": 1, + "samples": 16412, + "non_numeric_states": [], + "first": "2026-08-05T22:00:00+00:00", + "last": "2026-08-06T21:59:57.192711+00:00", + "gaps": [], + "covered_span_s": 86397.2, + "mean_cadence_s": 5.265, + "file": "p1-net-2026-08-06.jsonl.gz", + "bytes_gz": 142313, + "bytes_raw": 910849, + "sha256_raw": "7cd1fe8315e24dc6391693c49d6306b0c0cf9b6c3a478e457da921001cea930c", + "sha256_gz": "9108f169d645b9da07d4ae310666a1365ea07275ff311cd60adad1b702edbf11" + }, + { + "date": "2026-08-07", + "window": [ + "2026-08-07T00:00:00+02:00", + "2026-08-08T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1087531, + "attempts": 1, + "samples": 15952, + "non_numeric_states": [], + "first": "2026-08-06T22:00:00+00:00", + "last": "2026-08-07T21:59:58.195053+00:00", + "gaps": [], + "covered_span_s": 86398.2, + "mean_cadence_s": 5.416, + "file": "p1-net-2026-08-07.jsonl.gz", + "bytes_gz": 135979, + "bytes_raw": 879941, + "sha256_raw": "e0f67a9214980e2fd4893208cb402165065dadce2105ce1581c6b1d49952f39a", + "sha256_gz": "b70b84f3743d1d84055c911bcd5589cb8f0371e8916bde741308c5bb50702c29" + }, + { + "date": "2026-08-08", + "window": [ + "2026-08-08T00:00:00+02:00", + "2026-08-09T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1058569, + "attempts": 1, + "samples": 15468, + "non_numeric_states": [], + "first": "2026-08-07T22:00:00+00:00", + "last": "2026-08-08T21:59:52.805887+00:00", + "gaps": [], + "covered_span_s": 86392.8, + "mean_cadence_s": 5.586, + "file": "p1-net-2026-08-08.jsonl.gz", + "bytes_gz": 134025, + "bytes_raw": 857271, + "sha256_raw": "0be14ca7593b4c7d68570115cc26e24f6a26ab87fe3c168a8b942b167e8d4f68", + "sha256_gz": "d93dcc8ca3eae03af13b9ae36acd5ab1eaa1e7ee93c237d386e8ded1c61f5906" + }, + { + "date": "2026-08-09", + "window": [ + "2026-08-09T00:00:00+02:00", + "2026-08-10T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1059349, + "attempts": 1, + "samples": 15508, + "non_numeric_states": [], + "first": "2026-08-08T22:00:00+00:00", + "last": "2026-08-09T21:59:58.178998+00:00", + "gaps": [], + "covered_span_s": 86398.2, + "mean_cadence_s": 5.572, + "file": "p1-net-2026-08-09.jsonl.gz", + "bytes_gz": 135697, + "bytes_raw": 857531, + "sha256_raw": "f4793799173c502fcee5f78e2d9b0dcc984b5d2d8bd2fc0db0cd0f8d403733b3", + "sha256_gz": "2fea6c5c8b1452d72ad5aa3b0ae9755bb2ea08071514c08b1a3b1155ef457cf1" + }, + { + "date": "2026-08-10", + "window": [ + "2026-08-10T00:00:00+02:00", + "2026-08-11T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1066721, + "attempts": 1, + "samples": 15594, + "non_numeric_states": [], + "first": "2026-08-09T22:00:00+00:00", + "last": "2026-08-10T21:59:59.170056+00:00", + "gaps": [], + "covered_span_s": 86399.2, + "mean_cadence_s": 5.541, + "file": "p1-net-2026-08-10.jsonl.gz", + "bytes_gz": 136682, + "bytes_raw": 863785, + "sha256_raw": "ff5c3fa419505127a51acc56dfc13e8437e4ec2c13a11c8d8b2d6c3e20e2410e", + "sha256_gz": "4fd30e495f4e6d1c8501d05c23f9891ac34364d86a9c55ac825d52b2ebb31023" + }, + { + "date": "2026-08-11", + "window": [ + "2026-08-11T00:00:00+02:00", + "2026-08-12T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1056281, + "attempts": 1, + "samples": 15371, + "non_numeric_states": [], + "first": "2026-08-10T22:00:00+00:00", + "last": "2026-08-11T21:59:58.608150+00:00", + "gaps": [ + { + "from": "2026-08-11T08:00:14.121653+00:00", + "to": "2026-08-11T08:02:54.427007+00:00", + "seconds": 160.305 + }, + { + "from": "2026-08-11T17:04:18.517592+00:00", + "to": "2026-08-11T17:08:41.759777+00:00", + "seconds": 263.242 + } + ], + "covered_span_s": 86398.6, + "mean_cadence_s": 5.621, + "file": "p1-net-2026-08-11.jsonl.gz", + "bytes_gz": 133711, + "bytes_raw": 856244, + "sha256_raw": "67c2a740019f8908405b96b3436cd953af27baead2957acc7c1dcf10b37ac94e", + "sha256_gz": "4f8c553edc9051644bd7baa659f6f92a8b5f5ce12a0410c8b5235d1e6c3084c9" + }, + { + "date": "2026-08-12", + "window": [ + "2026-08-12T00:00:00+02:00", + "2026-08-13T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1124945, + "attempts": 1, + "samples": 16432, + "non_numeric_states": [], + "first": "2026-08-11T22:00:00+00:00", + "last": "2026-08-12T21:59:58.648966+00:00", + "gaps": [], + "covered_span_s": 86398.6, + "mean_cadence_s": 5.258, + "file": "p1-net-2026-08-12.jsonl.gz", + "bytes_gz": 136903, + "bytes_raw": 911115, + "sha256_raw": "73c8a1626640226ddffb32d564557d1cbe57a15367ea1ba3632934e15a1413dc", + "sha256_gz": "acec87065b7de2c605db00504ec5a9b3c99584620dfb3b67194e2cf9464d6a5f" + }, + { + "date": "2026-08-13", + "window": [ + "2026-08-13T00:00:00+02:00", + "2026-08-14T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1126742, + "attempts": 1, + "samples": 16473, + "non_numeric_states": [], + "first": "2026-08-12T22:00:00+00:00", + "last": "2026-08-13T21:59:57.562043+00:00", + "gaps": [], + "covered_span_s": 86397.6, + "mean_cadence_s": 5.245, + "file": "p1-net-2026-08-13.jsonl.gz", + "bytes_gz": 136221, + "bytes_raw": 912379, + "sha256_raw": "1d9a16c529174bd8c74625a742c46d98de4acf7a045f83df8bdbac22fa86e8e7", + "sha256_gz": "adfef36f629decf54c625f2dbfe6b4dab9441218c42557c90ac7f6d331943a59" + }, + { + "date": "2026-08-14", + "window": [ + "2026-08-14T00:00:00+02:00", + "2026-08-15T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1124413, + "attempts": 1, + "samples": 16427, + "non_numeric_states": [], + "first": "2026-08-13T22:00:00+00:00", + "last": "2026-08-14T21:59:55.611487+00:00", + "gaps": [], + "covered_span_s": 86395.6, + "mean_cadence_s": 5.26, + "file": "p1-net-2026-08-14.jsonl.gz", + "bytes_gz": 135570, + "bytes_raw": 910648, + "sha256_raw": "dc1417881e54fe215e9fcd4ed643d2482a277d61f8f27f25e2ff1e2cba29f7a8", + "sha256_gz": "d4eff781cd16a83fd1be9e643fee5bb5d5897e7d45bb9865972a7304d92c84a4" + }, + { + "date": "2026-08-15", + "window": [ + "2026-08-15T00:00:00+02:00", + "2026-08-16T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1124461, + "attempts": 1, + "samples": 16516, + "non_numeric_states": [], + "first": "2026-08-14T22:00:00+00:00", + "last": "2026-08-15T21:59:56.581991+00:00", + "gaps": [], + "covered_span_s": 86396.6, + "mean_cadence_s": 5.231, + "file": "p1-net-2026-08-15.jsonl.gz", + "bytes_gz": 137670, + "bytes_raw": 909539, + "sha256_raw": "f07a242faed68313512e0363d914846522ff839315ad80b55689072ac74e8a69", + "sha256_gz": "082314c6003c2208752acc4614b9809f06735d01e216b935ea94faf578c5598e" + }, + { + "date": "2026-08-16", + "window": [ + "2026-08-16T00:00:00+02:00", + "2026-08-17T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1127871, + "attempts": 1, + "samples": 16552, + "non_numeric_states": [], + "first": "2026-08-15T22:00:00+00:00", + "last": "2026-08-16T21:59:59.527710+00:00", + "gaps": [], + "covered_span_s": 86399.5, + "mean_cadence_s": 5.22, + "file": "p1-net-2026-08-16.jsonl.gz", + "bytes_gz": 138347, + "bytes_raw": 912481, + "sha256_raw": "a44e5d961600a539ddf824d5d211ef319833666b7598e5939a6cdf2cb1e3581f", + "sha256_gz": "8f6511e52f5be9187f422021abc93a5fa8b698b4c785877d69d15dd6de106fd8" + }, + { + "date": "2026-08-17", + "window": [ + "2026-08-17T00:00:00+02:00", + "2026-08-18T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1133721, + "attempts": 1, + "samples": 16627, + "non_numeric_states": [], + "first": "2026-08-16T22:00:00+00:00", + "last": "2026-08-17T21:59:59.685699+00:00", + "gaps": [ + { + "from": "2026-08-17T17:27:00.599126+00:00", + "to": "2026-08-17T17:28:20.389628+00:00", + "seconds": 79.791 + } + ], + "covered_span_s": 86399.7, + "mean_cadence_s": 5.197, + "file": "p1-net-2026-08-17.jsonl.gz", + "bytes_gz": 140214, + "bytes_raw": 917356, + "sha256_raw": "70cce79b3732f292f7f58f0b2655bdb4a00a20a6530f1f50dac62ef5f039788b", + "sha256_gz": "e2f9b461c4db7902e1a3d30b439ebc11ebcd9618a72d12f10e5131f38fb437bf" + }, + { + "date": "2026-08-18", + "window": [ + "2026-08-18T00:00:00+02:00", + "2026-08-19T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1074019, + "attempts": 1, + "samples": 15699, + "non_numeric_states": [ + { + "t": "2026-08-18T02:03:54.417106+00:00", + "state": "unavailable" + }, + { + "t": "2026-08-18T04:10:41.786299+00:00", + "state": "unavailable" + } + ], + "first": "2026-08-17T22:00:00+00:00", + "last": "2026-08-18T21:59:54.501766+00:00", + "gaps": [ + { + "from": "2026-08-18T02:02:59.633845+00:00", + "to": "2026-08-18T02:04:38.694366+00:00", + "seconds": 99.061 + }, + { + "from": "2026-08-18T04:09:54.292049+00:00", + "to": "2026-08-18T04:11:38.500238+00:00", + "seconds": 104.208 + } + ], + "covered_span_s": 86394.5, + "mean_cadence_s": 5.504, + "file": "p1-net-2026-08-18.jsonl.gz", + "bytes_gz": 135894, + "bytes_raw": 869570, + "sha256_raw": "68cd44ac56099ee38f0bba4c32d7d17d33b92d28a99d27419f53c22374491ca6", + "sha256_gz": "d6c97508f805aff040d2b4ed4e06c0e7373db2ea6227c7d79ef24c8c79074115" + }, + { + "date": "2026-08-19", + "window": [ + "2026-08-19T00:00:00+02:00", + "2026-08-20T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 989720, + "attempts": 1, + "samples": 14549, + "non_numeric_states": [], + "first": "2026-08-18T22:00:00+00:00", + "last": "2026-08-19T21:59:56.556610+00:00", + "gaps": [ + { + "from": "2026-08-19T01:33:29.599224+00:00", + "to": "2026-08-19T01:34:01.653383+00:00", + "seconds": 32.054 + } + ], + "covered_span_s": 86396.6, + "mean_cadence_s": 5.939, + "file": "p1-net-2026-08-19.jsonl.gz", + "bytes_gz": 126345, + "bytes_raw": 800369, + "sha256_raw": "0f11ce0cee43cbd7bdb08dc48b202e58c442157ac08b2cda5b64bbab9bd56300", + "sha256_gz": "13d61b7da903877c8ae7813b833cb5836ec04a1cf1efd0e1fa87e916bef95fd5" + }, + { + "date": "2026-08-20", + "window": [ + "2026-08-20T00:00:00+02:00", + "2026-08-21T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1068038, + "attempts": 1, + "samples": 15600, + "non_numeric_states": [], + "first": "2026-08-19T22:00:00+00:00", + "last": "2026-08-20T21:59:56.742604+00:00", + "gaps": [ + { + "from": "2026-08-20T02:11:56.607373+00:00", + "to": "2026-08-20T02:12:38.840458+00:00", + "seconds": 42.233 + }, + { + "from": "2026-08-20T20:42:04.623102+00:00", + "to": "2026-08-20T20:43:41.666543+00:00", + "seconds": 97.043 + } + ], + "covered_span_s": 86396.7, + "mean_cadence_s": 5.539, + "file": "p1-net-2026-08-20.jsonl.gz", + "bytes_gz": 139693, + "bytes_raw": 865024, + "sha256_raw": "2612153963951dd13fe310ad7fbb5bec2e195db56fd99fd963134c8badae25ba", + "sha256_gz": "362f2ee3d9d7f23f9051ba78993c5af7443dabb3b5f16339efc673444720c892" + }, + { + "date": "2026-08-21", + "window": [ + "2026-08-21T00:00:00+02:00", + "2026-08-22T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1118002, + "attempts": 1, + "samples": 16383, + "non_numeric_states": [], + "first": "2026-08-20T22:00:00+00:00", + "last": "2026-08-21T21:59:57.771588+00:00", + "gaps": [], + "covered_span_s": 86397.8, + "mean_cadence_s": 5.274, + "file": "p1-net-2026-08-21.jsonl.gz", + "bytes_gz": 139105, + "bytes_raw": 904809, + "sha256_raw": "933d9a6899f59c128d0d9b84d6944cea63b1ad58dc57ac52e3e476f9c616f07b", + "sha256_gz": "a563581a9a3338af173abb99a200db32881ddaf893d2458f5d8de59b5ec6041f" + }, + { + "date": "2026-08-22", + "window": [ + "2026-08-22T00:00:00+02:00", + "2026-08-23T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1129094, + "attempts": 1, + "samples": 16520, + "non_numeric_states": [], + "first": "2026-08-21T22:00:00+00:00", + "last": "2026-08-22T21:59:56.913786+00:00", + "gaps": [ + { + "from": "2026-08-21T23:05:20.035692+00:00", + "to": "2026-08-21T23:07:51.147719+00:00", + "seconds": 151.112 + }, + { + "from": "2026-08-21T23:07:56.966604+00:00", + "to": "2026-08-21T23:08:27.695101+00:00", + "seconds": 30.728 + } + ], + "covered_span_s": 86396.9, + "mean_cadence_s": 5.23, + "file": "p1-net-2026-08-22.jsonl.gz", + "bytes_gz": 141564, + "bytes_raw": 914120, + "sha256_raw": "39ccdf67c806344490f791648822cd2cd14d73ea7cc0b6e8572a1e15e8f74641", + "sha256_gz": "5c46f1879a723f9986d521124200cab839af2a4722b76bc97c6f6f59d0df65a2" + }, + { + "date": "2026-08-23", + "window": [ + "2026-08-23T00:00:00+02:00", + "2026-08-24T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1107164, + "attempts": 1, + "samples": 16351, + "non_numeric_states": [], + "first": "2026-08-22T22:00:00+00:00", + "last": "2026-08-23T21:59:56.921115+00:00", + "gaps": [], + "covered_span_s": 86396.9, + "mean_cadence_s": 5.284, + "file": "p1-net-2026-08-23.jsonl.gz", + "bytes_gz": 136277, + "bytes_raw": 894387, + "sha256_raw": "ffb15367bf3c98eff8a5600e4858ac99778b597c03d0a4f6154e6971b8f98a44", + "sha256_gz": "3fa76981758a468ba0aa68b50fe68d5bdb1ef38cd7faec2b6ffdb61c9ddb2db2" + }, + { + "date": "2026-08-24", + "window": [ + "2026-08-24T00:00:00+02:00", + "2026-08-25T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1122175, + "attempts": 1, + "samples": 16584, + "non_numeric_states": [], + "first": "2026-08-23T22:00:00+00:00", + "last": "2026-08-24T21:59:57.940474+00:00", + "gaps": [], + "covered_span_s": 86397.9, + "mean_cadence_s": 5.21, + "file": "p1-net-2026-08-24.jsonl.gz", + "bytes_gz": 136834, + "bytes_raw": 906369, + "sha256_raw": "30465d5ff51220201b4a79341e5da7bfc3d88290465b1c9d886ba94eee8dd234", + "sha256_gz": "5103088e047bfcbd9b3f55521f431b01cd27de10c2642e00281a13faa762bdd9" + }, + { + "date": "2026-08-25", + "window": [ + "2026-08-25T00:00:00+02:00", + "2026-08-26T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1110567, + "attempts": 1, + "samples": 16451, + "non_numeric_states": [ + { + "t": "2026-08-25T01:52:02.104225+00:00", + "state": "unavailable" + } + ], + "first": "2026-08-24T22:00:00+00:00", + "last": "2026-08-25T21:59:56.883744+00:00", + "gaps": [ + { + "from": "2026-08-25T01:51:25.942719+00:00", + "to": "2026-08-25T01:52:53.203963+00:00", + "seconds": 87.261 + } + ], + "covered_span_s": 86396.9, + "mean_cadence_s": 5.252, + "file": "p1-net-2026-08-25.jsonl.gz", + "bytes_gz": 135890, + "bytes_raw": 896416, + "sha256_raw": "0e7c86b2b74f2108bc7f5f5d2de2627caaafe473ca4ae18515534dff10526e26", + "sha256_gz": "2a5411c10d8d3b271f133c62c04bd722279b48b08d7376084fbad6cc6dd66d76" + }, + { + "date": "2026-08-26", + "window": [ + "2026-08-26T00:00:00+02:00", + "2026-08-27T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1123610, + "attempts": 1, + "samples": 16542, + "non_numeric_states": [], + "first": "2026-08-25T22:00:00+00:00", + "last": "2026-08-26T21:59:58.915457+00:00", + "gaps": [], + "covered_span_s": 86398.9, + "mean_cadence_s": 5.223, + "file": "p1-net-2026-08-26.jsonl.gz", + "bytes_gz": 139301, + "bytes_raw": 908350, + "sha256_raw": "88d72c1b9f51707d62e744f40402de4260a04617875b85c65d2df6079c6c84ca", + "sha256_gz": "0661588fbc0c6c42825b67d483c8818c070405c1002424b3f1d61d8fdadb6370" + }, + { + "date": "2026-08-27", + "window": [ + "2026-08-27T00:00:00+02:00", + "2026-08-28T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1122433, + "attempts": 1, + "samples": 16573, + "non_numeric_states": [], + "first": "2026-08-26T22:00:00+00:00", + "last": "2026-08-27T21:59:58.538446+00:00", + "gaps": [ + { + "from": "2026-08-27T18:50:29.030586+00:00", + "to": "2026-08-27T18:52:22.938470+00:00", + "seconds": 113.908 + } + ], + "covered_span_s": 86398.5, + "mean_cadence_s": 5.214, + "file": "p1-net-2026-08-27.jsonl.gz", + "bytes_gz": 138009, + "bytes_raw": 906770, + "sha256_raw": "528ab8ee10a46275d2e91d56ef2ed847e5932fae76a1752e6f2e9828dbfa9865", + "sha256_gz": "5cacc8c5788f43d6e6e2f569445fa82d9edc5a63b1f082f9a1d6a3bb0b404260" + }, + { + "date": "2026-08-28", + "window": [ + "2026-08-28T00:00:00+02:00", + "2026-08-29T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1114796, + "attempts": 1, + "samples": 16456, + "non_numeric_states": [], + "first": "2026-08-27T22:00:00+00:00", + "last": "2026-08-28T21:59:58.515555+00:00", + "gaps": [], + "covered_span_s": 86398.5, + "mean_cadence_s": 5.251, + "file": "p1-net-2026-08-28.jsonl.gz", + "bytes_gz": 135917, + "bytes_raw": 900654, + "sha256_raw": "f89838bc9f61af074c17817e4a6eab037318f38fa3c65dad3edfa2106abbc9fb", + "sha256_gz": "97090b605f67dcc1f84dea2c51d4832dfe1c2af497a4f4cfe50ecb51024d9b4c" + }, + { + "date": "2026-08-29", + "window": [ + "2026-08-29T00:00:00+02:00", + "2026-08-30T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1092983, + "attempts": 1, + "samples": 16132, + "non_numeric_states": [], + "first": "2026-08-28T22:00:00+00:00", + "last": "2026-08-29T21:59:57.574524+00:00", + "gaps": [ + { + "from": "2026-08-29T00:03:20.539844+00:00", + "to": "2026-08-29T00:03:50.569581+00:00", + "seconds": 30.03 + } + ], + "covered_span_s": 86397.6, + "mean_cadence_s": 5.356, + "file": "p1-net-2026-08-29.jsonl.gz", + "bytes_gz": 134210, + "bytes_raw": 883053, + "sha256_raw": "896d4c9d660be4faabc5a088323e246e257877d5ff6bd0ec084b785480d76e12", + "sha256_gz": "b6f264cd3afba63209123a27490e7e1c3ae70a86731d7ea05ae67f88917f136f" + }, + { + "date": "2026-08-30", + "window": [ + "2026-08-30T00:00:00+02:00", + "2026-08-31T00:00:00+02:00" + ], + "read": "OK", + "http_bytes": 1118895, + "attempts": 1, + "samples": 16490, + "non_numeric_states": [], + "first": "2026-08-29T22:00:00+00:00", + "last": "2026-08-30T21:59:58.533323+00:00", + "gaps": [], + "covered_span_s": 86398.5, + "mean_cadence_s": 5.24, + "file": "p1-net-2026-08-30.jsonl.gz", + "bytes_gz": 138429, + "bytes_raw": 904311, + "sha256_raw": "e704818efb58bb6c505791bc9eab2e9476bf48ff8911d0e79b401fd35b529fa0", + "sha256_gz": "9052fead2418bff3083bec3ae8ce64d6b9f6224b668f4bb510c79775f66c5b79" + } + ], + "summary": { + "days_requested": 30, + "days_read": 30, + "days_failed": 0, + "samples": 485841, + "bytes_gz": 4126003, + "empty_days": [] + } +} diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-01.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-01.jsonl.gz new file mode 100644 index 0000000..3c1bcb5 Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-01.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-02.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-02.jsonl.gz new file mode 100644 index 0000000..24ed378 Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-02.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-03.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-03.jsonl.gz new file mode 100644 index 0000000..4297f4f Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-03.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-04.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-04.jsonl.gz new file mode 100644 index 0000000..61da3d6 Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-04.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-05.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-05.jsonl.gz new file mode 100644 index 0000000..364fdaf Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-05.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-06.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-06.jsonl.gz new file mode 100644 index 0000000..0fd3d19 Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-06.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-07.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-07.jsonl.gz new file mode 100644 index 0000000..38db739 Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-07.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-08.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-08.jsonl.gz new file mode 100644 index 0000000..8af0b54 Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-08.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-09.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-09.jsonl.gz new file mode 100644 index 0000000..9b28a98 Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-09.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-10.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-10.jsonl.gz new file mode 100644 index 0000000..debd332 Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-10.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-11.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-11.jsonl.gz new file mode 100644 index 0000000..de0360a Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-11.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-12.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-12.jsonl.gz new file mode 100644 index 0000000..a4da140 Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-12.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-13.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-13.jsonl.gz new file mode 100644 index 0000000..30728d1 Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-13.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-14.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-14.jsonl.gz new file mode 100644 index 0000000..0598ebc Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-14.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-15.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-15.jsonl.gz new file mode 100644 index 0000000..9c32e4b Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-15.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-16.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-16.jsonl.gz new file mode 100644 index 0000000..7d6b0c7 Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-16.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-17.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-17.jsonl.gz new file mode 100644 index 0000000..75e5131 Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-17.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-18.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-18.jsonl.gz new file mode 100644 index 0000000..80d6356 Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-18.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-19.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-19.jsonl.gz new file mode 100644 index 0000000..7ec26a6 Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-19.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-20.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-20.jsonl.gz new file mode 100644 index 0000000..0c4ef77 Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-20.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-21.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-21.jsonl.gz new file mode 100644 index 0000000..6287fdf Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-21.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-22.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-22.jsonl.gz new file mode 100644 index 0000000..bbf730d Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-22.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-23.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-23.jsonl.gz new file mode 100644 index 0000000..e48cb66 Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-23.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-24.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-24.jsonl.gz new file mode 100644 index 0000000..7acb8d1 Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-24.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-25.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-25.jsonl.gz new file mode 100644 index 0000000..1c4b992 Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-25.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-26.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-26.jsonl.gz new file mode 100644 index 0000000..475a4ac Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-26.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-27.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-27.jsonl.gz new file mode 100644 index 0000000..37bfd31 Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-27.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-28.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-28.jsonl.gz new file mode 100644 index 0000000..1cf2a7f Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-28.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-29.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-29.jsonl.gz new file mode 100644 index 0000000..d475c3e Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-29.jsonl.gz differ diff --git a/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-30.jsonl.gz b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-30.jsonl.gz new file mode 100644 index 0000000..759c6fc Binary files /dev/null and b/goodwe_controller/scenarios/p1-net-30d-2026-08/p1-net-2026-08-30.jsonl.gz differ diff --git a/goodwe_controller/test_peakguard.py b/goodwe_controller/test_peakguard.py index 959d576..0485991 100644 --- a/goodwe_controller/test_peakguard.py +++ b/goodwe_controller/test_peakguard.py @@ -810,9 +810,16 @@ print() print(f" BASELINE {totals['u_under']}/{totals['u_q']} = {U_PCT:.1f} % of " f"quarters under {CAP_W:.0f} W, uncontrolled") print(f" GUARDED {totals['g_under']}/{totals['g_q']} = {G_PCT:.1f} %") -print(f" Two days, not thirty. AC 3 is TARIFF-01's and is still not met - " - f"see test_tariff.py part 9. What this compares is the same quarters with " - f"and without the guard, which is what two days can support.") +print(f" Two days, not thirty - by choice: the thirty exist now " + f"(scenarios/p1-net-30d-2026-08/, DATA-01) and live one directory down so " + f"this glob does not pick them up and put ~50 s of replay on every PR. " + f"tools/replay_ac3.py runs the month and must reproduce the totals above " + f"to be believed: `python3 tools/replay_ac3.py scenarios --expect " + f"{totals['u_under']}/{totals['u_q']}`. Over the thirty days the guard " + f"takes August's peak from 5029.8 W to 4107.3 W - still over the " + f"{CAP_W:.0f} W soft limit - and moves the interval count from 98.89 % to " + f"98.99 %, because once a month peak is set the ceiling it defends is " + f"that peak and not the cap. See that directory's README.") check("the guard improves on the uncontrolled baseline rather than merely " "matching it", G_PCT > U_PCT) check("...and it does not degrade a single quarter anywhere in the two days", diff --git a/goodwe_controller/test_tariff.py b/goodwe_controller/test_tariff.py index 709ed8a..2060317 100644 --- a/goodwe_controller/test_tariff.py +++ b/goodwe_controller/test_tariff.py @@ -35,16 +35,39 @@ NOT allowed to be tested at: 10. THE WIRING, through the real `Controller` and its real status page. ------------------------------------------------------------------------------ -⚠️ AC 3 IS NOT MET, AND PART 9 DOES NOT PRETEND OTHERWISE. +⚠️ PART 9 IS A TWO-DAY MEASUREMENT AND SAYS SO. THE 30-DAY ANSWER IS ELSEWHERE +(DATA-01), AND IT DOES NOT LAND WHERE ANYONE EXPECTED. AC 3 asks for "the monthly peak held under the soft limit in >= 95 % of -intervals over a 30-day simulation replay". THERE IS NO 30-DAY DATASET. The -infra repo's `sim/scenarios/` holds exactly two days of this house's real meter -- 2026-08-20 (the Illusmart failure day) and 2026-08-23 (the healthy reference) -- exported on 2026-08-25 because they were about to age out of HA's recorder, -and nothing longer was ever captured. `sim/scenarios/README.md` lists the whole -inventory; the rest is RS485 bus frames, OTA witness logs and a 578-row rollback -poll, none of which carries grid power over calendar time. +intervals over a 30-day simulation replay". When this file was written there +were two days of this house's real meter - 2026-08-20 (the Illusmart failure +day) and 2026-08-23 (the healthy reference) - exported on 2026-08-25 because +they were about to age out of HA's recorder. + +⚠️ AND THE SENTENCE THAT USED TO STAND HERE, "nothing longer was ever +captured", WAS TRUE ABOUT THE PAST AND WAS READ AS A CLAIM ABOUT THE FUTURE. +It was not one. HA's recorder still held the whole month; DATA-01 probed it on +2026-08-31 (230 samples in a 20-minute window at 30 days back, 0 at 45) and +exported thirty days the same day, into `scenarios/p1-net-30d-2026-08/` with +`tools/export_p1_history.py` - which is committed now precisely because the +2026-08-25 one-off was not, and its absence is what let the belief survive. + +Over those thirty days the criterion measures 98.89 % UNCONTROLLED and 98.99 % +guarded, so it is met - and that is a finding about the CRITERION as much as +about the guard, because a month with no controller at all passes it. Read the +other way - the monthly PEAK itself under the soft limit - it is not met: the +guard takes August's peak from 5029.8 W to 4107.3 W, still over 3500 W. Both +numbers, and why they part company, are in +`scenarios/p1-net-30d-2026-08/README.md`. + +PART 9 IS DELIBERATELY LEFT GRADING THE TWO COMMITTED DAYS. It is a per-PR +regression check and the thirty days cost about fifty seconds of replay; they +also live one directory down precisely so this file's glob does not pick them +up. `tools/replay_ac3.py` is the thing that runs the month, and it is checked +against part 6 of test_peakguard.py rather than trusted: + + python3 tools/replay_ac3.py scenarios --expect 189/190 + python3 tools/replay_ac3.py scenarios/p1-net-30d-2026-08 So part 9 replays the two days that exist, reports the percentage over them, and states the sample size in days next to it. It is a MEASUREMENT, not a verdict: @@ -60,8 +83,8 @@ export) for the reason `scenarios/p1-age-2026-08-26.jsonl.gz` already is: the infra repo is private, CI's token is minted for this repository only, and a regression fixture that CI cannot see is not a regression fixture. -Closing AC 3 needs 30 days of capture. That is TEST-01's replay harness, and it -cannot start before somebody starts recording. +The recorder's window rolls: 2026-08-01 leaves it around 2026-09-15, which is +why the committed copy is the artefact and not the API call. """ import asyncio @@ -748,9 +771,11 @@ print() print(f" AC 3 MEASURED: {under}/{total_q} quarters = {PCT:.1f} % under the " f"{SHIPPED_CAP_W:.0f} W soft limit, over {replay_days:.1f} DAYS of " f"replay, UNCONTROLLED (no peak shaving was running on either day).") -print(f" AC 3 ASKS FOR 30 DAYS. {replay_days:.1f} days exist. " - f"AC 3 IS NOT MET and this run does not claim it is - see the module " - f"docstring. The figure above is the BASELINE SAFETY-07 has to improve.") +print(f" AC 3 ASKS FOR 30 DAYS and this run is {replay_days:.1f}, so it " + f"does not settle it either way. The 30 days DO now exist - " + f"scenarios/p1-net-30d-2026-08/, DATA-01 - and " + f"tools/replay_ac3.py is what runs them; the figure above is the " + f"BASELINE SAFETY-07 has to improve, on the two days CI replays.") check("the replay produced a measurable number over a stated sample size, " "rather than a criterion quietly redefined to fit the data", total_q > 100 and replay_days < 30) diff --git a/goodwe_controller/tools/export_p1_history.py b/goodwe_controller/tools/export_p1_history.py new file mode 100644 index 0000000..bd636c8 --- /dev/null +++ b/goodwe_controller/tools/export_p1_history.py @@ -0,0 +1,455 @@ +#!/usr/bin/env python3 +"""Export a P1 power signal out of Home Assistant's recorder into the committed +scenario format - one gzipped JSON object per line, one file per LOCAL day. + +WHY THIS FILE EXISTS AT ALL +=========================== +`scenarios/p1-net-2026-08-20.jsonl.gz` and `p1-net-2026-08-23.jsonl.gz` were +produced once, by hand, and the script was thrown away. The consequence was not +theoretical: TARIFF-01's and SAFETY-07's AC 3 ("monthly peak held under the soft +limit in >= 95 % of intervals over a 30-day simulation replay") were both merged +UNMET on the belief that thirty days could not be produced retroactively - a +belief that was simply wrong, and that survived because nobody could re-run the +export to find out. So the export is a committed, runnable artefact now, and the +next person does not have to re-derive it under time pressure while a retention +window rolls. + + python3 tools/export_p1_history.py --days 30 --out-dir /some/where + +THE TWO RULES THIS FILE IS BUILT AROUND +======================================= +1. A FAILED READ IS NOT AN OBSERVATION. An HTTP error, a timeout, a truncated + body, a non-JSON body - none of those are "the meter reported nothing that + hour". They are "we do not know what the meter reported that hour", and the + two must never end up looking the same in the output. So: + + * every request is retried with backoff; + * a day whose read ultimately fails writes NO FILE and is recorded in the + manifest as `"read": "FAILED"` with the error text, and the process exits + non-zero; + * a day that reads back successfully but EMPTY writes an empty file and is + recorded as `"read": "OK", "samples": 0`. Empty-and-known and + unknown are therefore distinguishable both in the manifest and by the + presence or absence of the file; + * before any day is fetched, the entity is proved to exist through + /api/states/. Without that check a typo in --entity would return + 200 + `[]` for all thirty days and produce a beautifully consistent + dataset of nothing. + +2. GAPS ARE VISIBLE, NEVER INTERPOLATED. Nothing here carries a value forward, + fills, resamples or averages. A recorder outage or an HA restart shows up as + what it is: consecutive `t` values further apart than the meter's cadence. + The controller's own ingest already reads that correctly - `meter_max_age_s` + (30 s shipped) is exactly the age past which a held reading stops counting as + coverage - so an hour with no telegrams replays as an hour of NO COVERAGE + rather than as an hour of whatever the last sample said. The manifest lists + every gap over --gap-s so a reader does not have to diff timestamps to find + them. + +DAY AT A TIME, AND WHY NOT SMALLER +================================== +One request per local day. That is gentle on a live HA (~1 MB of JSON per day) +and it is also the only chunking that reproduces the committed files: the +history API prepends the state as it stood AT the period start +(`include_start_time_state`), stamped with the start instant. Splitting a day +into six-hour chunks would therefore inject three synthetic boundary samples per +day that no meter ever sent. Days are the natural boundary because the committed +files are local days. + +VERIFYING THE EXPORTER +====================== + python3 tools/export_p1_history.py --verify scenarios/p1-net-2026-08-20.jsonl.gz + +re-fetches that day by this exact route and compares the DECOMPRESSED bytes with +the committed file. Byte equality of the .gz containers is not expected and not +required: a gzip header carries an mtime, and this file writes mtime 0 so its +own output is reproducible. Content equality is the claim that matters. +""" + +from __future__ import annotations + +import argparse +import gzip +import hashlib +import json +import sys +import time +import urllib.error +import urllib.parse +import urllib.request +from datetime import datetime, timedelta, timezone +from pathlib import Path + +try: # py3.9+; the add-on image has it + from zoneinfo import ZoneInfo +except ImportError: # pragma: no cover + ZoneInfo = None # type: ignore + +DEFAULT_URL = "http://192.168.2.158:8123" +DEFAULT_ENTITY = "sensor.p1_meter_active_power" +DEFAULT_TZ = "Europe/Brussels" +# The shipped meter_max_age_s. A hole longer than this is a hole the controller +# itself would refuse to see through, which makes it the honest gap threshold. +DEFAULT_GAP_S = 30.0 + + +class ReadFailed(RuntimeError): + """The read did not happen. NOT the same thing as an empty result.""" + + +# --------------------------------------------------------------------------- +# transport + + +def _get(url: str, token: str, timeout: float): + req = urllib.request.Request(url, headers={ + "Authorization": f"Bearer {token}", + "Content-Type": "application/json", + }) + with urllib.request.urlopen(req, timeout=timeout) as resp: + if resp.status != 200: + raise ReadFailed(f"HTTP {resp.status}") + return resp.read() + + +def fetch_json(url: str, token: str, *, timeout: float, retries: int, + backoff: float, log=print): + """GET and parse, or raise ReadFailed. Never returns a partial answer.""" + last = None + for attempt in range(1, retries + 1): + try: + raw = _get(url, token, timeout) + except urllib.error.HTTPError as exc: + last = f"HTTP {exc.code} {exc.reason}" + except Exception as exc: # URLError, timeout, ... + last = f"{type(exc).__name__}: {exc}" + else: + try: + return json.loads(raw.decode("utf-8")), len(raw), attempt + except (UnicodeDecodeError, json.JSONDecodeError) as exc: + # A truncated body looks exactly like this. It is a failure. + last = f"unparseable body ({len(raw)} bytes): {exc}" + if attempt < retries: + wait = backoff * attempt + log(f" read failed ({last}); retry {attempt + 1}/{retries} " + f"in {wait:.0f} s") + time.sleep(wait) + raise ReadFailed(last or "unknown") + + +def assert_entity_exists(base: str, entity: str, token: str, *, timeout: float, + retries: int, backoff: float, log=print): + """Prove the entity is real BEFORE trusting any empty day. + + Without this, `--entity sensor.typo` answers 200 + [] for every day and the + exporter cheerfully writes thirty empty files. + """ + url = f"{base}/api/states/{urllib.parse.quote(entity)}" + doc, _, _ = fetch_json(url, token, timeout=timeout, retries=retries, + backoff=backoff, log=log) + if not isinstance(doc, dict) or doc.get("entity_id") != entity: + raise ReadFailed(f"{entity} is not a known entity") + return doc + + +# --------------------------------------------------------------------------- +# one day + + +def day_bounds(day: datetime, tz): + start = datetime(day.year, day.month, day.day, tzinfo=tz) + return start, start + timedelta(days=1) + + +def history_url(base: str, entity: str, start: datetime, end: datetime) -> str: + q = urllib.parse.urlencode({ + "filter_entity_id": entity, + "end_time": end.isoformat(), + "minimal_response": "", + }) + return (f"{base}/api/history/period/" + f"{urllib.parse.quote(start.isoformat())}?{q}") + + +def rows_from(doc, entity: str): + """HA's shape -> [(iso_t, state_str)], plus the states that are not numbers. + + `minimal_response` returns the first record in full and the rest as + {state, last_changed}. An entity that went unavailable reports the string + "unavailable" (or "unknown") where a number belongs. Those are dropped from + the numeric series - there is no float to write and the committed schema has + no slot for a non-number - but every one is returned here so it lands in the + manifest. Dropping them is what MAKES the outage visible: the series simply + has no samples across it, which is the same thing the meter falling silent + looks like, and is what the controller's freshness rule already handles. + """ + if not isinstance(doc, list): + raise ReadFailed(f"expected a list, got {type(doc).__name__}") + if not doc: + return [], [] + series = doc[0] + if not isinstance(series, list): + raise ReadFailed("history series is not a list") + good, bad = [], [] + for rec in series: + eid = rec.get("entity_id") + if eid is not None and eid != entity: + raise ReadFailed(f"unexpected entity {eid!r} in the response") + t = rec.get("last_changed") or rec.get("last_updated") + s = rec.get("state") + if t is None: + raise ReadFailed("a record carries no timestamp") + try: + w = float(s) + except (TypeError, ValueError): + bad.append({"t": t, "state": s}) + continue + good.append((t, w)) + return good, bad + + +def render(rows, *, newline: str) -> bytes: + """The committed format, exactly: `{"t": , "w": }` per line. + + json.dumps' default separators (", " / ": ") and repr of a float are what + the two committed files contain; so is the CRLF, an artefact of the original + one-off having been run on Windows in text mode. It is reproduced rather + than tidied because reproducing the committed bytes is this exporter's + verification, and gzip.open(..., 'rt') strips it either way. + """ + out = bytearray() + for t, w in rows: + out += json.dumps({"t": t, "w": w}).encode("utf-8") + out += newline.encode("ascii") + return bytes(out) + + +def gz(payload: bytes) -> bytes: + """gzip level 9, mtime 0 - so the same input always gives the same bytes.""" + import io + buf = io.BytesIO() + with gzip.GzipFile(fileobj=buf, mode="wb", compresslevel=9, mtime=0) as f: + f.write(payload) + return buf.getvalue() + + +def gaps_in(rows, gap_s: float): + out = [] + prev = None + for t, _ in rows: + cur = datetime.fromisoformat(t) + if prev is not None: + d = (cur - prev).total_seconds() + if d > gap_s: + out.append({"from": prev.isoformat(), "to": cur.isoformat(), + "seconds": round(d, 3)}) + prev = cur + return out + + +def export_day(base, entity, token, day, tz, *, timeout, retries, backoff, + gap_s, newline, log=print): + """Read one local day. Raises ReadFailed rather than returning a guess.""" + start, end = day_bounds(day, tz) + doc, nbytes, attempts = fetch_json( + history_url(base, entity, start, end), token, + timeout=timeout, retries=retries, backoff=backoff, log=log) + rows, bad = rows_from(doc, entity) + entry = { + "date": day.date().isoformat(), + "window": [start.isoformat(), end.isoformat()], + "read": "OK", + "http_bytes": nbytes, + "attempts": attempts, + "samples": len(rows), + "non_numeric_states": bad, + "first": rows[0][0] if rows else None, + "last": rows[-1][0] if rows else None, + "gaps": gaps_in(rows, gap_s), + } + if rows: + span = (datetime.fromisoformat(rows[-1][0]) + - datetime.fromisoformat(rows[0][0])).total_seconds() + entry["covered_span_s"] = round(span, 1) + entry["mean_cadence_s"] = round(span / max(1, len(rows) - 1), 3) + return rows, entry + + +# --------------------------------------------------------------------------- +# modes + + +def do_export(args, token, tz, log=print): + base = args.url.rstrip("/") + out_dir = Path(args.out_dir) + out_dir.mkdir(parents=True, exist_ok=True) + newline = "\r\n" if args.newline == "crlf" else "\n" + + log(f" proving {args.entity} exists before trusting any empty day") + st = assert_entity_exists(base, args.entity, token, timeout=args.timeout, + retries=args.retries, backoff=args.backoff, + log=log) + log(f" {args.entity} = {st.get('state')} " + f"{st.get('attributes', {}).get('unit_of_measurement', '')}") + + if args.end_date: + last = datetime.fromisoformat(args.end_date).replace(tzinfo=tz) + else: # yesterday: today is still being logged + last = datetime.now(tz).replace(hour=0, minute=0, second=0, + microsecond=0) - timedelta(days=1) + days = [last - timedelta(days=i) for i in range(args.days - 1, -1, -1)] + + manifest = { + "generated": datetime.now(timezone.utc).isoformat(), + "generator": "tools/export_p1_history.py", + "source": f"{base}/api/history/period", + "entity": args.entity, + "timezone": args.tz, + "gap_threshold_s": args.gap_s, + "format": '{"t": , "w": } per line, gzip', + "days": [], + } + failures = [] + for day in days: + name = f"{args.prefix}{day.date().isoformat()}.jsonl.gz" + log(f" {name}") + try: + rows, entry = export_day( + base, args.entity, token, day, tz, timeout=args.timeout, + retries=args.retries, backoff=args.backoff, gap_s=args.gap_s, + newline=newline, log=log) + except ReadFailed as exc: + # NO FILE. A missing file is the honest record of a read that did + # not happen; an empty one would claim the meter was silent. + entry = {"date": day.date().isoformat(), "read": "FAILED", + "error": str(exc), "file": None} + manifest["days"].append(entry) + failures.append(entry) + log(f" READ FAILED: {exc} - no file written") + continue + payload = render(rows, newline=newline) + blob = gz(payload) + (out_dir / name).write_bytes(blob) + entry["file"] = name + entry["bytes_gz"] = len(blob) + entry["bytes_raw"] = len(payload) + entry["sha256_raw"] = hashlib.sha256(payload).hexdigest() + entry["sha256_gz"] = hashlib.sha256(blob).hexdigest() + manifest["days"].append(entry) + ng = len(entry.get("gaps", [])) + log(f" {entry['samples']} samples, {len(blob)} B gz, " + f"{ng} gap(s) over {args.gap_s:g} s" + + (f", {len(entry['non_numeric_states'])} non-numeric state(s)" + if entry["non_numeric_states"] else "")) + if day is not days[-1]: + time.sleep(args.delay) + + ok = [d for d in manifest["days"] if d["read"] == "OK"] + manifest["summary"] = { + "days_requested": len(days), + "days_read": len(ok), + "days_failed": len(failures), + "samples": sum(d["samples"] for d in ok), + "bytes_gz": sum(d["bytes_gz"] for d in ok), + "empty_days": [d["date"] for d in ok if d["samples"] == 0], + } + # newline="\n" explicitly: write_text() on Windows emits CRLF, and + # the manifest would then differ from a Linux run of the same export + # for no reason visible in a diff. + with open(args.manifest or (out_dir / "manifest.json"), "w", + encoding="utf-8", newline="\n") as fh: + fh.write(json.dumps(manifest, indent=2) + "\n") + log(f" {manifest['summary']['days_read']}/{len(days)} days, " + f"{manifest['summary']['samples']} samples, " + f"{manifest['summary']['bytes_gz'] / 1e6:.2f} MB gz") + if failures: + log(f" {len(failures)} DAY(S) NOT READ - the dataset is incomplete " + f"and says so: " + ", ".join(d["date"] for d in failures)) + return 1 + return 0 + + +def do_verify(args, token, tz, log=print): + """Re-export a committed day by this route and diff it against the file.""" + base = args.url.rstrip("/") + newline = "\r\n" if args.newline == "crlf" else "\n" + rc = 0 + for target in args.verify: + path = Path(target) + stamp = path.name.split(".")[0].rsplit("-", 3)[-3:] + day = datetime(int(stamp[0]), int(stamp[1]), int(stamp[2]), tzinfo=tz) + log(f" {path.name} (local day {day.date().isoformat()})") + committed = gzip.decompress(path.read_bytes()) + rows, entry = export_day(base, args.entity, token, day, tz, + timeout=args.timeout, retries=args.retries, + backoff=args.backoff, gap_s=args.gap_s, + newline=newline, log=log) + fresh = render(rows, newline=newline) + if fresh == committed: + log(f" IDENTICAL: {len(fresh)} bytes, {entry['samples']} " + f"samples, sha256 {hashlib.sha256(fresh).hexdigest()[:16]}") + continue + rc = 1 + cl = committed.split(newline.encode()) + fl = fresh.split(newline.encode()) + log(f" DIFFERS: committed {len(committed)} B / {len(cl)} lines, " + f"re-export {len(fresh)} B / {len(fl)} lines") + shown = 0 + for i in range(max(len(cl), len(fl))): + a = cl[i] if i < len(cl) else b"" + b = fl[i] if i < len(fl) else b"" + if a != b: + log(f" line {i}: committed {a!r}") + log(f" re-export {b!r}") + shown += 1 + if shown >= args.max_diff: + log(f" ... stopping after {shown} differing lines") + break + return rc + + +def main(argv=None): + p = argparse.ArgumentParser(description=__doc__.split("\n")[0]) + p.add_argument("--url", default=DEFAULT_URL) + p.add_argument("--entity", default=DEFAULT_ENTITY) + p.add_argument("--token", help="bearer token; else $HASS_TOKEN") + p.add_argument("--tz", default=DEFAULT_TZ, + help="local timezone that defines a day boundary") + p.add_argument("--days", type=int, default=30) + p.add_argument("--end-date", help="last local day to export (YYYY-MM-DD); " + "default yesterday") + p.add_argument("--out-dir", default=".") + p.add_argument("--prefix", default="p1-net-") + p.add_argument("--manifest", help="path for the manifest JSON") + p.add_argument("--gap-s", type=float, default=DEFAULT_GAP_S) + p.add_argument("--newline", choices=("crlf", "lf"), default="crlf", + help="line terminator; crlf reproduces the committed files") + p.add_argument("--timeout", type=float, default=120.0) + p.add_argument("--retries", type=int, default=4) + p.add_argument("--backoff", type=float, default=5.0) + p.add_argument("--delay", type=float, default=2.0, + help="pause between days - this is a live HA") + p.add_argument("--verify", nargs="+", metavar="FILE", + help="re-export these committed days and diff them") + p.add_argument("--max-diff", type=int, default=20) + args = p.parse_args(argv) + + token = args.token or __import__("os").environ.get("HASS_TOKEN") + if not token: + print("no token: pass --token or set HASS_TOKEN", file=sys.stderr) + return 2 + if ZoneInfo is None: + print("zoneinfo unavailable", file=sys.stderr) + return 2 + tz = ZoneInfo(args.tz) + try: + if args.verify: + return do_verify(args, token, tz) + return do_export(args, token, tz) + except ReadFailed as exc: + print(f"read failed: {exc}", file=sys.stderr) + return 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/goodwe_controller/tools/replay_ac3.py b/goodwe_controller/tools/replay_ac3.py new file mode 100644 index 0000000..f11de9a --- /dev/null +++ b/goodwe_controller/tools/replay_ac3.py @@ -0,0 +1,303 @@ +#!/usr/bin/env python3 +"""Run TARIFF-01's and SAFETY-07's AC 3 over a directory of `p1-net-*.jsonl.gz`. + +AC 3, both cards, same words: *"monthly peak held under the soft limit in +>= 95 % of intervals over a 30-day simulation replay."* Both cards merged with +it UNMET because two days of capture existed. DATA-01 exported thirty. This is +the thing that turns those thirty days into the number. + + cd goodwe_controller + python3 tools/replay_ac3.py scenarios/p1-net-30d-2026-08 + +WHAT IT MEASURES +================ +Two runs over the same telegrams, one file at a time in date order into a +SHARED peak store, so the 15-minute blocks accumulate into a real MONTH rather +than into 30 disconnected days: + + UNCONTROLLED the recorded meter as it happened. Nothing was shaving peaks on + this house during August 2026, so this is not a simulation of + anything - it is what the month did. It is TARIFF-01's AC 3. + GUARDED the same month with `peakguard.decide()` in the loop and the + battery it asks for modelled, closed-loop: the meter the guard + sees is the recorded net MINUS the discharge it obtained. That + is SAFETY-07's AC 3. + +Both numbers are `quarters <= peak_cap_w` over `quarters closed`, with the cap +read out of config.yaml rather than typed. Nothing here exits non-zero on 95 %: +a measurement below the criterion is a finding about the house and the guard, +and this tool's job is to report it, not to be graded by it. + +FAITHFULNESS TO THE SUITE +========================= +The replay loop is `test_peakguard.py` part 6's, with `pack=None` (SoC pinned +above the floor, so SAFETY-24's energy floor is not the thing deciding the +answer). It is not imported from there because that file runs its whole suite on +import. The duplication is checked instead of asserted: pointed at `scenarios/` +- the two committed days - this tool must reproduce part 6's published totals, +and `--expect` makes that a hard check. + + python3 tools/replay_ac3.py scenarios --expect 189/190 + +GAPS +==== +Nothing is interpolated. A hole in the export is a hole here: the shipped +`meter_max_age_s` is what decides whether a held reading still counts as +coverage, so an hour with no telegrams closes no quarters and contributes to +neither numerator nor denominator. `--report-gaps` lists them. +""" + +from __future__ import annotations + +import argparse +import gzip +import json +import sys +import tempfile +from datetime import datetime, timedelta +from pathlib import Path + +HERE = Path(__file__).resolve().parent.parent +sys.path.insert(0, str(HERE)) + +from app.p1 import P1Ingest, make_sample, split_signed # noqa: E402 +from app.peakguard import decide # noqa: E402 +from app.tariff import PeakStore, PeakTracker, month_key # noqa: E402 + + +def read_option(key: str): + """The shipped default for `key`, out of config.yaml. + + ⚠️ `peak_cap_w:` appears TWICE in config.yaml - once under `options:` with + the default, once under `schema:` as `int(500,15000)`. The FIRST match is + the options block, which is the number an install actually lands on; taking + the last would try to float() the schema line. Same rule as the suites. + """ + for line in (HERE / "config.yaml").read_text(encoding="utf-8").splitlines(): + if line.strip().startswith(f"{key}:"): + try: + return float(line.split(":", 1)[1].strip()) + except ValueError: + continue + return None + + +def read_schema_max(key: str): + for line in (HERE / "config.yaml").read_text(encoding="utf-8").splitlines(): + s = line.strip() + if s.startswith(f"{key}:") and "(" in s and ")" in s: + try: + return float(s[s.index("(") + 1:s.index(")")].split(",")[-1]) + except ValueError: + continue + return None + + +def sample(t: datetime, net_w: float): + """One telegram through the SHIPPED sign derivation - the suites' helper.""" + imp, exp = split_signed(net_w) + return make_sample("ha_signed", imp, exp, phases=1, + ingest_ts=t, ingest_mono=t.timestamp()) + + +def rows(path): + """Every sample in one file, in order, with the gaps left as gaps.""" + with gzip.open(path, "rt", encoding="utf-8") as fh: + for line in fh: + line = line.strip() + if not line: + continue + row = json.loads(line) + yield datetime.fromisoformat(row["t"]), float(row["w"]) + + +def replay(paths, *, guarded: bool, cap_w, max_w, max_hold_s, cycle_s, + soc, soc_floor, tmp: Path, gap_s: float): + """test_peakguard.py part 6's loop, over an arbitrary number of days. + + ⚠️ ONE STORE, AND AN INGEST THAT IS RESET ONLY AT A REAL DISCONTINUITY. + Neither half of that is a detail. + + The STORE is shared across the files because "monthly peak" is what AC 3 is + about and a per-file store cannot express it. + + The INGEST is carried across a file boundary when the next file's first + telegram follows the previous file's last one within `gap_s`, and rebuilt + when it does not. Both failure modes are real and were measured here: + + * carried across a genuine hole, one averager manufactures quarters out of + nothing - streamed straight through the three-day gap between the two + committed days this reported 285/286 where the suite reports 189/190; + * rebuilt at every file, each day's LAST quarter (21:45-22:00 UTC) never + closes, because it closes only when a telegram past the boundary + arrives and that telegram is in the next file. That is why the suite's + two days give 95 quarters each rather than 96, and it silently discards + one quarter per day - 30 of them over a month. + + So contiguous days are joined and non-contiguous ones are not, which gives + 189/190 on the committed pair and a full 96 quarters a day on a run of + consecutive days. + """ + store = PeakStore(str(tmp / f"ac3-{'g' if guarded else 'u'}.db")) + tracker = PeakTracker(store, cap_w=cap_w) + engaged = blind = saturated = cycles = n = joins = 0 + wh = 0.0 + gaps = [] + months = set() + ing = None + batt = 0.0 # delivered discharge, W (positive) + next_cycle = None + prev_t = None + for path in paths: + for i, (t, w) in enumerate(rows(path)): + if i == 0: + contiguous = (prev_t is not None + and 0 <= (t - prev_t).total_seconds() <= gap_s) + if contiguous: + joins += 1 + else: + if ing is not None: + months |= {month_key(b.start) for b in ing.blocks} + ing = P1Ingest(phases=1, max_age_s=max_hold_s) + batt = 0.0 + next_cycle = None + prev_t = None + if prev_t is not None: + dt = (t - prev_t).total_seconds() + if dt > gap_s: + gaps.append((prev_t, t, dt)) + wh += batt * dt / 3600.0 + prev_t = t + n += 1 + ing.submit(sample(t, w - batt)) + tracker.sync(ing) + if next_cycle is not None and t < next_cycle: + continue + next_cycle = t + timedelta(seconds=cycle_s) + cycles += 1 + d = decide(PeakTracker.project(ing.averager, t), batt_w=batt, + cap_w=cap_w, reading=tracker.peak(t), + achievable_w=max_w, max_hold_s=max_hold_s, soc=soc, + soc_floor=soc_floor, frozen=False) + if d.engaged: + blind += 1 if d.blind else 0 + engaged += 0 if d.blind else 1 + saturated += 1 if d.saturated else 0 + if not guarded: + continue + engaged_floor = d.engaged and d.floor_w is not None + batt = max(0.0, d.floor_w) if engaged_floor else 0.0 + if ing is not None: + months |= {month_key(b.start) for b in ing.blocks} + out = {m: dict(store.blocks_in(m)) for m in sorted(months)} + return {"months": out, "samples": n, "cycles": cycles, "joins": joins, + "engaged": engaged, "blind": blind, "saturated": saturated, + "wh": wh, "gaps": gaps} + + +def main(argv=None): + p = argparse.ArgumentParser(description=__doc__.split("\n")[0]) + p.add_argument("directory", help="a directory of p1-net-*.jsonl.gz") + p.add_argument("--glob", default="p1-net-*.jsonl.gz") + p.add_argument("--expect", help="UNDER/TOTAL the uncontrolled run must " + "reproduce, e.g. 189/190") + p.add_argument("--report-gaps", action="store_true") + args = p.parse_args(argv) + + paths = sorted(Path(args.directory).glob(args.glob)) + if not paths: + print(f"no {args.glob} under {args.directory}", file=sys.stderr) + return 2 + + CAP_W = read_option("peak_cap_w") + MAX_HOLD_S = read_option("meter_max_age_s") + CYCLE_S = read_option("cycle_floor_s") + SOC_FLOOR = read_option("peak_soc_floor") + LIVE_MAX_W = read_schema_max("max_w") + if None in (CAP_W, MAX_HOLD_S, CYCLE_S, SOC_FLOOR, LIVE_MAX_W): + print("could not read the shipped options out of config.yaml", + file=sys.stderr) + return 2 + # The suites' fixture SoC: comfortably above the floor, so SAFETY-24's + # energy floor is not what decides an AC 3 that is not about it. + SOC_OK = SOC_FLOOR + 30.0 + + days = len(paths) + span_first = span_last = None + print(f"AC 3 - {days} file(s) from {args.directory}") + print(f" shipped: peak_cap_w {CAP_W:.0f} W, meter_max_age_s " + f"{MAX_HOLD_S:.0f} s, cycle_floor_s {CYCLE_S:.0f} s, " + f"max_w schema ceiling {LIVE_MAX_W:.0f} W") + + tmp = Path(tempfile.mkdtemp(prefix="goodwe-ac3-")) + kw = dict(cap_w=CAP_W, max_w=LIVE_MAX_W, max_hold_s=MAX_HOLD_S, + cycle_s=CYCLE_S, soc=SOC_OK, soc_floor=SOC_FLOOR, tmp=tmp, + gap_s=MAX_HOLD_S) + u = replay(paths, guarded=False, **kw) + g = replay(paths, guarded=True, **kw) + + def flat(run): + return [w for m in run["months"].values() for w in m.values()] + + uq, gq = flat(u), flat(g) + u_under = sum(1 for w in uq if w <= CAP_W) + g_under = sum(1 for w in gq if w <= CAP_W) + u_pct = 100.0 * u_under / max(len(uq), 1) + g_pct = 100.0 * g_under / max(len(gq), 1) + + print(f" {u['samples']} samples, {len(uq)} closed quarters " + f"({len(uq) / 96.0:.1f} days' worth), {days - u['joins']} contiguous " + f"run(s) of days") + for m in sorted(u["months"]): + mu = u["months"][m] + mg = g["months"].get(m, {}) + print(f" month {m}: {len(mu)} quarters, peak " + f"{max(mu.values()):.1f} W uncontrolled -> " + f"{max(mg.values()) if mg else 0.0:.1f} W guarded") + print() + print(f" TARIFF-01 AC 3 (UNCONTROLLED, what the month did): " + f"{u_under}/{len(uq)} = {u_pct:.2f} % of quarters at or under " + f"{CAP_W:.0f} W") + print(f" SAFETY-07 AC 3 (GUARDED, peakguard.decide in the loop): " + f"{g_under}/{len(gq)} = {g_pct:.2f} %") + print(f" guard cycles {g['cycles']}, engaged on the measurement " + f"{g['engaged']}, blind {g['blind']}, saturated {g['saturated']}, " + f"modelled discharge {g['wh']:.0f} Wh") + # AC 3'S WORDS SUPPORT TWO READINGS AND BOTH ARE REPORTED, because they + # disagree on this data and picking the flattering one is the failure mode + # this whole card exists to correct. + u_peak = max(uq) if uq else 0.0 + g_peak = max(gq) if gq else 0.0 + print(f" reading A - PERCENTAGE OF INTERVALS under the cap (the reading " + f"test_tariff.py part 9 already uses): uncontrolled {u_pct:.2f} %, " + f"guarded {g_pct:.2f} %") + print(f" reading B - THE MONTHLY PEAK ITSELF under the cap: " + f"{u_peak:.1f} W uncontrolled -> {g_peak:.1f} W guarded, against a " + f"{CAP_W:.0f} W soft limit " + f"({'under' if g_peak <= CAP_W else 'STILL OVER'})") + print(f" verdict at the 95 % criterion: uncontrolled " + f"{'MEETS' if u_pct >= 95.0 else 'DOES NOT MEET'}, guarded " + f"{'MEETS' if g_pct >= 95.0 else 'DOES NOT MEET'} " + f"({len(uq) / 96.0:.1f} days of replay)") + + if u["gaps"]: + tot = sum(d for _, _, d in u["gaps"]) + print(f" gaps over {MAX_HOLD_S:.0f} s: {len(u['gaps'])}, " + f"{tot:.0f} s total - NOT interpolated; the quarters they fall in " + f"simply carry less coverage") + if args.report_gaps: + for a, b, d in u["gaps"]: + print(f" {a.isoformat()} -> {b.isoformat()} {d:.0f} s") + + if args.expect: + want_u, want_t = (int(x) for x in args.expect.split("/")) + ok = (u_under, len(uq)) == (want_u, want_t) + print(f" reproduces test_peakguard.py part 6's published " + f"{want_u}/{want_t}: {'yes' if ok else 'NO'} " + f"(got {u_under}/{len(uq)})") + return 0 if ok else 1 + return 0 + + +if __name__ == "__main__": + raise SystemExit(main())