Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c45893ffad | ||
|
|
5059e28860 | ||
|
|
375a1e29d2 | ||
|
|
c2787da539 | ||
|
|
b59f5ce31b | ||
|
|
0618194d9c | ||
|
|
587751b115 | ||
|
|
3de4453956 | ||
|
|
f6f3f227c1 | ||
|
|
91b0d5e4dc | ||
|
|
806390d8b1 | ||
|
|
775f3a2e82 | ||
|
|
f109153216 | ||
|
|
f6d7bfcb04 | ||
|
|
d1b7e3f113 | ||
|
|
5e382f52f6 | ||
|
|
46c022c926 | ||
|
|
22ca341459 | ||
|
|
9c1a9e0f17 | ||
|
|
638c213944 | ||
|
|
1841416aa4 | ||
|
|
cb8f800180 | ||
|
|
018da00b74 | ||
|
|
e06906d65f | ||
|
|
37dd0f00cc | ||
|
|
a0d981731c | ||
|
|
9099ceb1af | ||
|
|
295a2aedb6 |
@@ -1,5 +1,246 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## Unreleased
|
||||||
|
|
||||||
|
⚠️ **THIS HAS NO VERSION AND CANNOT REACH AN INSTALL UNTIL IT GETS ONE.**
|
||||||
|
PKG-06's and PKG-07's rule: the Supervisor keys the built image by `version:`,
|
||||||
|
so `ha apps update` is a silent no-op on an unbumped change. This branch
|
||||||
|
deliberately does **not** pick the number, because the right one depends on a
|
||||||
|
fact the branch cannot observe - whether 0.9.0's image has already been built
|
||||||
|
and pulled. If it has not, this belongs inside 0.9.0's section before it ships;
|
||||||
|
if it has, it needs 0.10.0. Whoever deploys decides.
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| _this branch_ | FCST-04 - the carry-forward fill reaches the plan, and the day it carries survives a restart |
|
||||||
|
| _this branch_ | FCST-05 - one byte stops stopping the add-on, and the write path gets the read path's standard |
|
||||||
|
| _this branch_ | FCST-06 - "cannot persist" stops meaning "has no /data" and starts meaning what it says |
|
||||||
|
|
||||||
|
### FCST-06 - the second way to be a store that cannot persist
|
||||||
|
|
||||||
|
**A price cache that could not write went on answering a yesterday the file did
|
||||||
|
not contain.** `price_history.py`'s docstring states its rule unconditionally -
|
||||||
|
*"A store that cannot persist now records nothing, and both of its answers say
|
||||||
|
the same thing"* - and FCST-04 enforced it against `not self.ok`, which is one
|
||||||
|
**symptom**: the durable volume is missing. `ok` is about the DIRECTORY, and a
|
||||||
|
read-only remount and a full disk both leave it `True`. Reproduced on `5059e28`
|
||||||
|
against two real volumes - a `/data` bind-mounted read-only and a full tmpfs:
|
||||||
|
|
||||||
|
| | with a `/data` that cannot be written |
|
||||||
|
|---|---|
|
||||||
|
| `last_error` | `could not write ([Errno 30] Read-only file system)` |
|
||||||
|
| `words` | `2026-01-14 (24 points over 24 h); 2026-01-15 (24 points over 24 h) [could not write (...)]` |
|
||||||
|
| `yesterday()` | **series, 24 points** |
|
||||||
|
| on disk | `{}` |
|
||||||
|
|
||||||
|
`words` and `yesterday()` were describing two different stores, and the 24
|
||||||
|
points were gone on the next restart with nothing saying so - the
|
||||||
|
silently-forgetting cache that rule exists to refuse, reached by the condition
|
||||||
|
it did not name. **No shipped path reaches it**: `/data` is the Supervisor's
|
||||||
|
durable volume and this module is its only writer. The defect is that a
|
||||||
|
guarantee the module states about itself did not hold in one condition.
|
||||||
|
|
||||||
|
**"Cannot persist" is now ONE derived predicate, asked once, about the
|
||||||
|
outcome.** It cannot be answered *before* the attempt - a directory that existed
|
||||||
|
and was writable a moment ago is not a write that completed - so `_save()`
|
||||||
|
returns whether the file now holds what memory holds: `False` with no durable
|
||||||
|
volume, `False` on an `OSError` from the write, `True` otherwise. `_remember()`
|
||||||
|
keeps only what that one answer says was kept, and **the `not self.ok` test in
|
||||||
|
`_remember()` is gone** rather than joined by a second one. FCST-04's M4 and
|
||||||
|
FCST-05's HIGH were the same shape - a rule enforced on the conditions somebody
|
||||||
|
thought of - so a third cause now needs no third branch.
|
||||||
|
|
||||||
|
**What is undone is the MERGE, not the store.** The state restored is the last
|
||||||
|
one that *is on disk*, so a site whose volume fills at 14:00 goes on answering
|
||||||
|
the yesterday it wrote at 13:30 - that record survives a restart, so answering
|
||||||
|
it is not a promise the store cannot keep. It simply stops adding to it. The
|
||||||
|
snapshot is a `copy.deepcopy`: a one-level copy shares the day record the merge
|
||||||
|
rebinds `points` on, and measured with one, a store holding 12 persisted points
|
||||||
|
answered **24** after a failed merge.
|
||||||
|
|
||||||
|
⚠️ **A failed write is still not fatal, and that was a constraint rather than an
|
||||||
|
oversight.** §6 and this module's opening: a broken cache must not be a broken
|
||||||
|
controller. An add-on that will not start because `/data` filled up is a worse
|
||||||
|
defect than the one being fixed, so `_save()` reports and returns - the failure
|
||||||
|
stays in the log, in `last_error`, in `words` and in `yesterday()`'s reason -
|
||||||
|
and the shipped `resolve_tariff()` path still returns a provider plan with no
|
||||||
|
§11.4 degradation. What changed is that it is now *acted on*.
|
||||||
|
|
||||||
|
`test_price_history.py` section 5d drives it on two real failures with different
|
||||||
|
syscalls and different errnos - a directory where the file goes (`os.replace`,
|
||||||
|
EISDIR) and a volume that moves out from under an open store (`mkstemp`,
|
||||||
|
ENOENT) - because the rule is enforced on the outcome and one cause would grade
|
||||||
|
a coincidence. See `DECISIONS/FCST-06-cannot-persist-is-one-question.md`.
|
||||||
|
|
||||||
|
### FCST-05 - what FCST-04's own contract still did not hold
|
||||||
|
|
||||||
|
**A single non-UTF-8 byte in `/data/price_history.json` stopped the add-on from
|
||||||
|
starting.** `_load()` caught `(json.JSONDecodeError, OSError)`.
|
||||||
|
`UnicodeDecodeError` is `ValueError -> UnicodeError -> UnicodeDecodeError` -
|
||||||
|
neither of those, and `issubclass(UnicodeDecodeError, OSError)` is **False** -
|
||||||
|
so it escaped `_load()` -> `_open()` -> `PriceHistory.__init__`, which
|
||||||
|
`Controller.__init__` does not guard. Reproduced: `UnicodeDecodeError: 'utf-8'
|
||||||
|
codec can't decode byte 0xff`, and **the controller did not come up**. That is the same class as the day-record defect the store was reworked
|
||||||
|
for, in the one byte class the rework left uncovered: valid-UTF-8-invalid-JSON,
|
||||||
|
a UTF-8 BOM and a non-dict day record all degraded correctly. The suite could
|
||||||
|
not see it because its "unreadable file" fixture writes `"{ this is not json"`,
|
||||||
|
which is **valid UTF-8**. The handler now names the `ValueError` family - the
|
||||||
|
file's CONTENT - beside `OSError`, the file's BYTES, and a fixture holding a
|
||||||
|
genuinely undecodable byte is in `test_price_history.py` section 5.
|
||||||
|
|
||||||
|
**A malformed `start` under the day the feed is writing to killed the tariff
|
||||||
|
plan permanently.** `_remember()` merged a day's points with
|
||||||
|
`{q["start"]: q ...}` and sorted on `q["start"]`, trusting a value read off
|
||||||
|
`/data` to be a hashable, sortable string. An `int`, `null`, a `dict` or a
|
||||||
|
`list` there raised `TypeError`; `run_tariff_refresh` catches `Exception`, so
|
||||||
|
the add-on stayed **up** while **every refresh raised, forever** - the plan
|
||||||
|
retiring after `TARIFF_MAX_AGE_S` and arbitrage and the cheap window off for
|
||||||
|
good, with no self-heal because `_remember()` raised before reaching `_save()`.
|
||||||
|
The read path (`_parse_points()`) was properly defensive throughout, which is
|
||||||
|
why `yesterday()` survived and the existing fixture passed: it files its
|
||||||
|
malformed rows under a day the feed is **not** writing to, so `_remember()`
|
||||||
|
never opened that record. **The two paths now share one validator**,
|
||||||
|
`_point_of()`, and neither has a standard of its own. A row that is not a point
|
||||||
|
is dropped and reported, and `_save()` rewrites the record clean.
|
||||||
|
|
||||||
|
⚠️ **AND THE STATUS ROW SAID THE STORE WAS HEALTHY WHILE THE PLAN WAS DEAD.**
|
||||||
|
`words` read `'2026-01-15 (1 h)'` about the record that was killing every
|
||||||
|
refresh. `_load()` now counts the rows inside a kept record as well as the
|
||||||
|
record itself, so the store stops reporting plain health from the moment it is
|
||||||
|
opened; the note names what was **found** at open and what was **dropped** by
|
||||||
|
the refresh, and clears itself once the file has been rewritten.
|
||||||
|
|
||||||
|
⚠️ **AND THE ROW COUNTED POINTS AND CALLED THEM HOURS.** `f"{d} ({_n_points(d)}
|
||||||
|
h)"` on the quarter-hourly feed this add-on already supports reads
|
||||||
|
`'2026-01-13 (96 h)'` for a 24-hour day - the same points/hours conflation the
|
||||||
|
sub-hourly defect above was about, on the row that defect's own card shipped.
|
||||||
|
The cell is now `2026-01-13 (96 points over 24 h)`: two facts, each under its
|
||||||
|
own unit, and the hour figure is the one that says whether tomorrow morning's
|
||||||
|
fill has a whole day to carry.
|
||||||
|
|
||||||
|
⚠️ **AND THE MERGE KEY IS THE INSTANT RATHER THAN ITS SPELLING**, which is what
|
||||||
|
rule 4 already said. Aware datetimes hash and compare by instant. The string
|
||||||
|
key merged by spelling, and the lexicographic sort beside it put the second
|
||||||
|
02:00 of the 25-hour October day (`+01:00`) **before** the first (`+02:00`);
|
||||||
|
only the stored order was affected, because `_parse_points()` re-sorts by
|
||||||
|
instant on the way out.
|
||||||
|
|
||||||
|
⚠️ **TWO STATED REASONS WERE FALSE, AND THIS REPOSITORY TREATS THAT AS A
|
||||||
|
DEFECT.** `price_forecast._audit()`'s "WHY IT IS STRICTER" section claimed the
|
||||||
|
rework refused two things FCST-03's audit did not. A differential over the
|
||||||
|
whole domain the audit can express - every published mask over 3 hours crossed
|
||||||
|
with every output series of length 0-3 - found **zero divergence in both
|
||||||
|
directions**: the reworked audit is exactly **EQUAL** to FCST-03's, not
|
||||||
|
stronger. The old `seen` count already refused the same published point emitted
|
||||||
|
twice (one point twice is one hour twice), and the old **identity** rule
|
||||||
|
already refused a carried point sharing a published hour. The only behavioural
|
||||||
|
change is the intended **relaxation** - several distinct published points in
|
||||||
|
one hour - and that stays. The docstring and the two check texts now say what
|
||||||
|
is true, and the differential is committed as a check in
|
||||||
|
`test_price_forecast.py` rather than left as prose.
|
||||||
|
|
||||||
|
⚠️ **AND `_audit` IS NOT WHAT STOPS THE DECIMATION DEFECT RECURRING.** It does
|
||||||
|
not check **completeness**: fed the decimation directly - four published points
|
||||||
|
in an hour, output emitting only the last - it **accepts**, and it accepts an
|
||||||
|
empty output. That is a limit, not a regression, and the module docstring said
|
||||||
|
otherwise by presenting the audit as the post-condition guarding provenance.
|
||||||
|
What actually guards it is the point **count** and the pass-through **identity**
|
||||||
|
checks in `test_price_forecast.py` and `test_price_history.py`, and the text now
|
||||||
|
points at them. It is **not** made complete: `published` is the whole read while
|
||||||
|
the fill emits only `[since, until)`, so "every published point appears" is not
|
||||||
|
a true post-condition of those two arguments. Measured - the rule as stated
|
||||||
|
takes the 1 h, 12 h and 48 h horizon checks red on legitimate series.
|
||||||
|
|
||||||
|
**No option was added, `config.yaml` is untouched, and nothing was deployed.**
|
||||||
|
|
||||||
|
|
||||||
|
**FCST-03 shipped the fill and no caller.** `price_forecast.fill_from_yesterday()`
|
||||||
|
was complete and checked, and so were the consumers written for it -
|
||||||
|
`TariffPlan.firm_points()`, `cheap_now()`'s refusal of a provisional hour, and
|
||||||
|
`arbitrage()`'s refusal 3, which drops provisional hours from the trade set
|
||||||
|
rather than refusing the whole series. Nothing in `app/` called the fill, so no
|
||||||
|
series ever contained a carried point and every one of those consumers was
|
||||||
|
exercising its all-firm branch.
|
||||||
|
|
||||||
|
⚠️ **THE DEFECT THAT PRODUCED WAS NOT AN ERROR ANYWHERE.** `load_prices()`
|
||||||
|
returns what the feed published and nothing widened it to the requested horizon,
|
||||||
|
so on a dynamic site between midnight and the 14-16 h CET day-ahead publication
|
||||||
|
the plan covered the rest of today and **nothing more** - typically 10-24 of the
|
||||||
|
48 hours `price_horizon_h` asked for. `arbitrage()` then took `max - min` over
|
||||||
|
that truncated window and reported a spread. The number was real, the window was
|
||||||
|
half the one configured, and **nothing said so**: tomorrow evening's peak, the
|
||||||
|
hour a morning charge would be sold into, was not in the series at all, so the
|
||||||
|
spread it would have created was never compared against anything.
|
||||||
|
|
||||||
|
**What changes:**
|
||||||
|
|
||||||
|
* `app/price_history.py` (new) keeps the **published** prices of the last few
|
||||||
|
Brussels days in `/data/price_history.json`, and `resolve_tariff()` now fills
|
||||||
|
an unpublished hour of the horizon from yesterday's same-wall-clock hour,
|
||||||
|
flagged provisional. ⚠️ **"An" unpublished hour, not "every" one:** the fill
|
||||||
|
can carry only an hour yesterday actually published, so a partial yesterday
|
||||||
|
leaves the local hours it never held **out of the series** - §11.4 for those
|
||||||
|
hours, and asking for one raises rather than answering. Correct behaviour,
|
||||||
|
and the wording is now the behaviour.
|
||||||
|
* ⚠️ **A sub-hourly feed keeps every print.** `eneco_eddy` is the imbalance
|
||||||
|
product and settles per **quarter**; the fill walks whole hours to decide
|
||||||
|
what needs carrying, and that is not a rule about what a published hour may
|
||||||
|
contain. Every published point inside the horizon passes through by identity,
|
||||||
|
and a carried point is added only for a whole hour with no published coverage
|
||||||
|
at all. The first draft keyed the published set by hour - last point wins -
|
||||||
|
and a 96-point quarter-hourly feed reached the optimiser as 15 points with
|
||||||
|
its spread down from 0.143 to 0.014 EUR/kWh, on an all-firm, fully-published
|
||||||
|
series.
|
||||||
|
* The horizon is **whole hours at both ends**: `since` is floored to the hour
|
||||||
|
that contains it, and so is `until`. Asked for 48 hours at 08:17, the plan is
|
||||||
|
48 hours long and not 49.
|
||||||
|
* `resolve_tariff(history=...)` is **required and keyword-only**. A caller with
|
||||||
|
no durable volume passes the explicit `NoHistory()`. A default of `None`
|
||||||
|
meaning "skip the fill" would hand a caller who forgot the argument exactly
|
||||||
|
the truncated horizon this card removes, silently.
|
||||||
|
* A new status row, **Price history**, names the days held, how many points
|
||||||
|
each holds and how many whole hours they cover (see FCST-05: it shipped
|
||||||
|
counting points and printing them under the hour's name).
|
||||||
|
|
||||||
|
⚠️ **ONE CHANGE HERE MOVES ENERGY, AND IT IS NOT THE ONE THE CARD IS NAMED FOR.**
|
||||||
|
Carried hours are provisional, so §8.3 keeps them out of the trade set: the fill
|
||||||
|
widens what the optimiser can **see**, never what it may **do**. What does
|
||||||
|
change a trade is that the filled series is windowed to `[since, until)`, so
|
||||||
|
**hours that have already elapsed are no longer in the spread**. Before, a cheap
|
||||||
|
night that had already gone sat in the series as the day's cheapest hour - and a
|
||||||
|
cycle cannot buy in an hour that is past, so the trade was quietly never taken.
|
||||||
|
`test_price_history.py` section 2 grades this by finding the cheapest firm hour
|
||||||
|
with and without the fill and showing it **moves**.
|
||||||
|
|
||||||
|
⚠️ **A DEAD FEED IS STILL §11.4, NOT YESTERDAY.** The fill runs only on a
|
||||||
|
`series` read; a feed that published nothing still degrades to the configured
|
||||||
|
`static_profile`. The two are indistinguishable to the optimiser - both are
|
||||||
|
100 % provisional, so arbitrage refuses on the firm-pair rung and `cheap_now()`
|
||||||
|
is False - so preferring yesterday would change a documented degradation for a
|
||||||
|
difference only the status page can see.
|
||||||
|
`DECISIONS/FCST-04-the-day-that-is-carried.md` carries that and two more.
|
||||||
|
|
||||||
|
⚠️ **A BROKEN CACHE IS NOT A BROKEN CONTROLLER.** No `/data`, an unreadable
|
||||||
|
file, a schema this build does not know (including a `version` of `1.0`, which
|
||||||
|
`== 1`), a **day record that is not a record**, a malformed point - each its own
|
||||||
|
state with its own reason, none of them a price and none of them an empty
|
||||||
|
series. In every one the plan is still a real provider series and the controller
|
||||||
|
still regulates; the horizon is just as short as it was before this card.
|
||||||
|
`.words` never raises, because `Controller.__init__` reads it unguarded and a
|
||||||
|
status line that throws is an add-on that does not start.
|
||||||
|
|
||||||
|
⚠️ **AND WHERE THERE IS NO `/data`, NOTHING IS RECORDED AT ALL** - not even in
|
||||||
|
memory. An in-memory yesterday answers for one run and is gone on the next
|
||||||
|
restart, which is a cache that forgets without saying so; the honest pair is
|
||||||
|
`unavailable` on the status row and no prices carried.
|
||||||
|
|
||||||
|
⚠️ **"YESTERDAY" IS A LOCAL DATE, AND THE ARITHMETIC IS ON THE DATE.**
|
||||||
|
`brussels()` renders at a fixed offset, so subtracting `timedelta(days=1)` from
|
||||||
|
it subtracts 24 **absolute** hours - and the Brussels day before a
|
||||||
|
spring-forward is 23 hours long. Between 00:00 and 00:59 on the morning after
|
||||||
|
the March transition that lands on the day *before* yesterday. **No option was
|
||||||
|
added and `config.yaml` is untouched.**
|
||||||
|
|
||||||
## 0.9.0
|
## 0.9.0
|
||||||
|
|
||||||
⚠️ **MINOR FOR NEW FUNCTIONALITY, NOT FOR A BREAK - AND THAT DISTINCTION IS THE
|
⚠️ **MINOR FOR NEW FUNCTIONALITY, NOT FOR A BREAK - AND THAT DISTINCTION IS THE
|
||||||
|
|||||||
@@ -0,0 +1,181 @@
|
|||||||
|
# FCST-04 — what "yesterday" is, and what the fill is allowed to reach
|
||||||
|
|
||||||
|
Four decisions, recorded because each had a defensible alternative that a later
|
||||||
|
reader would otherwise re-open from scratch.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. The fill runs on a `series` read only. A dead feed still goes to §11.4.
|
||||||
|
|
||||||
|
`fill_from_yesterday()` accepts a `no_data` read and will happily build a
|
||||||
|
100 %-carried series from yesterday alone. `resolve_tariff()` does not let it:
|
||||||
|
a feed that published nothing still falls to `_degrade()` and §11.4's own row —
|
||||||
|
*"No price data → fall back to the static profile, mark the plan provisional,
|
||||||
|
no arbitrage."*
|
||||||
|
|
||||||
|
**Why not prefer yesterday's real prices to a coarse day/night profile?**
|
||||||
|
Because it buys nothing the optimiser can use, and it silently replaces a
|
||||||
|
fallback the operator configured.
|
||||||
|
|
||||||
|
Measured, not assumed — and stated at the reach it actually has:
|
||||||
|
|
||||||
|
- **This one reaches control.** A fully-carried series is 100 % provisional, so
|
||||||
|
`TariffPlan.arbitrage()` finds fewer than two firm hours and refuses on
|
||||||
|
§8.3's firm-pair rung. That is the verdict the optimiser reads: `arbitrage.py`
|
||||||
|
calls `plan.arbitrage(...)` on `fresh_tariff_plan()`, and it is the only path
|
||||||
|
by which a provider plan moves energy. The §11.4 static fallback refuses on
|
||||||
|
the same rung, for the same reason.
|
||||||
|
- **This one does not, and saying so is the point.** `TariffPlan.cheap_now()`
|
||||||
|
also returns False on a fully-carried series, because `firm_points()` is
|
||||||
|
empty — but it has **no production caller on the provider plan**.
|
||||||
|
`self.cheap` comes from `static_status()`, which builds its *own* static plan
|
||||||
|
and calls `cheap_now()` on that. So it is a property of the type, checked in
|
||||||
|
`test_static_tariff.py`, and it is not evidence about the shipped control
|
||||||
|
path. The conclusion below holds on the first bullet alone.
|
||||||
|
|
||||||
|
The two paths therefore differ only in which numbers the status page shows.
|
||||||
|
|
||||||
|
So the choice is between two identical control behaviours, and the tie-breaker
|
||||||
|
is that `static_profile` is a documented option an operator fills in **for this
|
||||||
|
case**. Overriding it for a cosmetic difference is a behaviour change nobody
|
||||||
|
asked for.
|
||||||
|
|
||||||
|
⚠️ **The consequence, stated so it is not discovered later as a bug:**
|
||||||
|
`fill_from_yesterday()`'s `no_data` branch is not reached from production.
|
||||||
|
It stays checked in `test_price_forecast.py`. Turning it on is a product
|
||||||
|
decision — it would want a rule about how old a carried profile may be before it
|
||||||
|
stops being offered at all — and not a wiring one.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. "Yesterday" is a date, not the newest record.
|
||||||
|
|
||||||
|
`newest()` is the obvious accessor and it is the one that decays. An add-on down
|
||||||
|
from Friday to Monday would answer Friday's prices to a fill whose only
|
||||||
|
vocabulary is `provisional: true` — a flag that says "this is a guess", never
|
||||||
|
"this guess is three days old". Nothing downstream can tell the difference and
|
||||||
|
nothing reports it.
|
||||||
|
|
||||||
|
So `yesterday(stamp)` computes the Brussels date before `stamp`'s Brussels date
|
||||||
|
and looks up **that** record. Anything else is `no_data` with a reason naming
|
||||||
|
both the date wanted and the dates held.
|
||||||
|
|
||||||
|
The cost is real and accepted: after an outage of more than a day, the first
|
||||||
|
morning back has no fill. That is the honest state — there is no yesterday —
|
||||||
|
and the horizon is short for one morning rather than wrong for several.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. What is recorded is the provider's read, and the mechanism is ordering.
|
||||||
|
|
||||||
|
Feeding the filled series back into the store is how this rots: day 2 carries
|
||||||
|
day 1's guesses, day 3 carries day 2's carry of them, and after a week of a dead
|
||||||
|
feed the "yesterday" being carried is a fossil of the last real publication.
|
||||||
|
Every point still carries `provisional=True`, so the **flag never lies** — what
|
||||||
|
decays is the number, behind a flag that is telling the truth.
|
||||||
|
|
||||||
|
⚠️ **No value-level check can catch that**, and claiming one would be worse than
|
||||||
|
claiming none. A carried point and a published-provisional point (`eneco_eddy`'s
|
||||||
|
current quarter, §9.2) are the same three fields carrying the same flag. The
|
||||||
|
difference is provenance, and provenance is not in the value.
|
||||||
|
|
||||||
|
The defence is therefore an **ordering**, not a validation:
|
||||||
|
|
||||||
|
- `carry()` is the only public entry point. It records its argument, then
|
||||||
|
returns the fill.
|
||||||
|
- The filled series exists only as that return value. The object never sees it
|
||||||
|
again.
|
||||||
|
- `_remember()` is private and called from exactly one place.
|
||||||
|
- `test_price_history.py` parses every shipped module under `app/`, `tools/`,
|
||||||
|
`scenarios/` and `estop/` and fails if any of them **uses the name**
|
||||||
|
`_remember` — an attribute, a bare name, a `def`, or the string constant a
|
||||||
|
`getattr` would need. By name rather than by substring, so a comment that
|
||||||
|
merely mentions the rule does not fail it and a call in `tools/` cannot pass
|
||||||
|
unseen.
|
||||||
|
- and it pins the order two ways: the **line numbers** of the two calls in
|
||||||
|
`carry()`, and — the one that matters — the **behaviour**. A feed carrying
|
||||||
|
yesterday alongside today, into an empty store, must fill the whole horizon;
|
||||||
|
reversed, the lookup finds nothing and the plan is only the hours the feed
|
||||||
|
published. Driven, so a reversal is red.
|
||||||
|
|
||||||
|
⚠️ **The first version of that ordering check graded nothing, and this file
|
||||||
|
said it did.** It compared positions in `ast.walk`, which is breadth-first:
|
||||||
|
`self._remember(...)` is a statement and `self.yesterday(...)` is an *argument*
|
||||||
|
of `fill_from_yesterday(...)`, one level deeper, so `_remember` came out first
|
||||||
|
whichever statement was written first. Measured — shipped and reversed both
|
||||||
|
scored the same. A decorative check that a document cites as a guarantee is
|
||||||
|
worse than no check, which is why the correction is recorded here rather than
|
||||||
|
quietly made.
|
||||||
|
|
||||||
|
That ordering has a second benefit worth keeping: a feed that returns yesterday
|
||||||
|
alongside today refreshes yesterday's record from live data *before* it is
|
||||||
|
consumed as the profile.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What this ticket does not claim
|
||||||
|
|
||||||
|
It does not make a trade possible that yesterday's prices justify. Carried hours
|
||||||
|
are dropped from the trade set by §8.3, and that is not a limitation to be fixed
|
||||||
|
later — it is the rule.
|
||||||
|
|
||||||
|
The one change that **does** move energy came from the windowing, not the
|
||||||
|
carrying: the filled series covers `[since, until)`, so hours that have already
|
||||||
|
elapsed are no longer in the spread. A cheap night that has gone used to be the
|
||||||
|
series' cheapest hour, and a cycle cannot buy in an hour that is past — so the
|
||||||
|
trade was silently never taken. `test_price_history.py` section 2 grades that by
|
||||||
|
finding the cheapest firm hour with and without the fill and showing it moves.
|
||||||
|
|
||||||
|
⚠️ **And section 1's identity check now grades identity.** It used to compare
|
||||||
|
`eur_per_kwh` between `_before` and `_after`, which come from two *separate*
|
||||||
|
`resolve_tariff()` calls — each parsing the feed body into its own objects, so
|
||||||
|
`is` could never have held between them however the fill were written. FCST-03's
|
||||||
|
whole anti-copy argument is `filled.points[i] is fetched.points[j]`, and floats
|
||||||
|
cannot grade it. The value comparison is kept and now says what it is (the fill
|
||||||
|
moved the window, it did not touch a price); identity is graded on a read the
|
||||||
|
file constructs and still holds a reference to.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. A sub-hourly feed passes through whole; the fill only decides which hours are missing.
|
||||||
|
|
||||||
|
`eneco_eddy` is the Belgian imbalance product and settles **per quarter**, and
|
||||||
|
`price.py`'s `_validate_raw()` accepts any strictly-increasing spacing — so a
|
||||||
|
published hour may legitimately hold four points.
|
||||||
|
|
||||||
|
FCST-04's first caller keyed the published set as `{_utc_hour(p.start): p for p
|
||||||
|
in r.points}`. It reads perfectly ordinary and it keeps the **last** point of
|
||||||
|
each hour. Measured on a 96-point quarter-hourly feed — all firm, fully
|
||||||
|
published, nothing to carry: the plan reached the optimiser as **15 points**,
|
||||||
|
the arbitrage spread fell from **0.143 to 0.014 EUR/kWh**, and the cheapest
|
||||||
|
"hour" became a `:45` point whose `at()` window runs into the next hour. That is
|
||||||
|
not the windowing effect above; it is a feed losing 84 % of its own prints, and
|
||||||
|
no `provisional` flag would ever have reported it.
|
||||||
|
|
||||||
|
**The alternative was rejected:** refusing sub-hourly feeds at `_validate_raw()`
|
||||||
|
would have made the count correct by breaking the one supported product that
|
||||||
|
legitimately publishes per quarter.
|
||||||
|
|
||||||
|
**What was done instead** keeps FCST-03's existing contract rather than weakening
|
||||||
|
it: *every* published point inside the window passes through **by identity**,
|
||||||
|
and a carried point is added only for a whole hour with **no** published
|
||||||
|
coverage at all. A carried hour is still a single hourly point whatever the
|
||||||
|
feed's own spacing is, because yesterday's profile is only known per wall-clock
|
||||||
|
hour — and carried hours are provisional, so they are out of the trade set
|
||||||
|
anyway.
|
||||||
|
|
||||||
|
⚠️ **`_audit()` got stricter, not looser, and that was the condition on doing
|
||||||
|
this at all.** The old post-condition refused any repeated `_utc_hour` — a count
|
||||||
|
that cannot express four points in one hour. It is replaced by two rules that
|
||||||
|
**subsume** it:
|
||||||
|
|
||||||
|
- a **published** point must *be* one of that hour's published points, by
|
||||||
|
identity, and may be emitted only once;
|
||||||
|
- a **carried** point may stand only in an hour that does not appear in the
|
||||||
|
published set at all, must be flagged `True`, and may not repeat.
|
||||||
|
|
||||||
|
So a duplicated carried hour — the DST mispricing the count existed to catch —
|
||||||
|
still raises; so does the same published point emitted twice; and a carried
|
||||||
|
point sharing an hour with a real print now raises too, which the count never
|
||||||
|
looked at. Each of those three is driven by its own mutation in
|
||||||
|
`test_price_forecast.py`.
|
||||||
@@ -0,0 +1,125 @@
|
|||||||
|
# FCST-06 — "cannot persist" is one question, and it is asked about the outcome
|
||||||
|
|
||||||
|
Two decisions. The first is the one the ticket put up for judgement; the second
|
||||||
|
is the one that decides what a store answers *after* it has failed to write.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. One derived predicate, not two places testing two symptoms — yes
|
||||||
|
|
||||||
|
`price_history.py`'s module docstring states, unconditionally:
|
||||||
|
|
||||||
|
> ⚠️ AND THAT RULE IS CARRIED OUT IN `_remember()`, NOT ONLY IN `_save()`. …
|
||||||
|
> A store that cannot persist now records nothing, and both of its answers say
|
||||||
|
> the same thing.
|
||||||
|
|
||||||
|
It was enforced against `not self.ok`, and `ok` is **about the directory**: it
|
||||||
|
says `/data` was there when the store opened. A read-only remount and a full
|
||||||
|
volume both leave it `True`. So the second way to be a store that cannot
|
||||||
|
persist — `ok=True`, the directory present, `_save()` failing — was outside the
|
||||||
|
one condition the rule named, and in that state `words` printed two full days
|
||||||
|
beside the write error while `yesterday()` answered 24 real points, from memory,
|
||||||
|
until a restart dropped them without a word.
|
||||||
|
|
||||||
|
**The ticket's question was whether "cannot persist" should be one derived
|
||||||
|
predicate rather than two places each testing a different symptom. It should,
|
||||||
|
and the reason is not tidiness — it is that the enumeration cannot be
|
||||||
|
completed.** FCST-04's M4 enforced the rule on the symptom that was in front of
|
||||||
|
it. FCST-05's HIGH was the same shape — an `except` clause naming the one
|
||||||
|
content-failure member somebody had met (`json.JSONDecodeError`) instead of the
|
||||||
|
`ValueError` family — and its fix was to name the family. This is the third instance in
|
||||||
|
one module, and it is the last place in this file where the rule can be spelt
|
||||||
|
twice. A fourth cause — an `OSError` subclass, a filesystem that accepts the
|
||||||
|
`mkstemp` and refuses the `os.replace`, a container whose volume is unmounted
|
||||||
|
mid-refresh — needs no fourth branch now, because none of them is asked about.
|
||||||
|
|
||||||
|
### Where the one question can be asked, which is the whole design
|
||||||
|
|
||||||
|
**"Can this store persist?" cannot be answered before the attempt.** A
|
||||||
|
directory that exists and was writable a moment ago is not a write that
|
||||||
|
completed; there is no pre-check that is not itself a guess, and adding one
|
||||||
|
(`os.access`, a probe write) would be a *third* symptom test — precisely what
|
||||||
|
the ticket rules out. So the predicate is an **outcome**:
|
||||||
|
|
||||||
|
_save() -> bool # True only if the file now holds what memory holds
|
||||||
|
|
||||||
|
Both ways of not persisting are the same answer from the same call: no durable
|
||||||
|
volume returns `False` without touching the disk, an `OSError` out of the write
|
||||||
|
returns `False` after recording why. `_remember()` reads that one answer and
|
||||||
|
keeps only what it says was kept. **The `not self.ok` test in `_remember()` is
|
||||||
|
gone** — not moved, gone — because there is nothing left for a second test to
|
||||||
|
add, and a check in `test_price_history.py` 5d asserts its absence from the
|
||||||
|
source.
|
||||||
|
|
||||||
|
⚠️ **That last check is deliberately a source check, and habit 4 is why it is
|
||||||
|
allowed to be.** The claim "there are not two places testing two symptoms" is a
|
||||||
|
claim about the *shape of the code*, not about a behaviour — the behaviour is
|
||||||
|
identical whether the rule is enforced once or twice. Every other check in 5d
|
||||||
|
is driven.
|
||||||
|
|
||||||
|
### What was NOT done, and why
|
||||||
|
|
||||||
|
- **Not a `can_persist()` helper.** It would have to answer before the attempt,
|
||||||
|
which is the thing that cannot be done honestly, and it would be a second
|
||||||
|
place to keep in step with the first.
|
||||||
|
- **Not a probe write at `_open()`.** It would make `ok` mean something new, it
|
||||||
|
would write to `/data` on every start, and a volume that is writable at 08:00
|
||||||
|
and full at 14:00 would still defeat it. A predicate that can go stale is a
|
||||||
|
symptom test wearing a predicate's name.
|
||||||
|
- **Not making a failed write fatal.** §6, and this module's own opening: a
|
||||||
|
broken cache must not be a broken controller. An add-on that will not start
|
||||||
|
because `/data` filled up is a worse defect than the one being fixed. The
|
||||||
|
store degrades; it does not raise. `_save()` reports and returns.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. The MERGE is undone, not the store
|
||||||
|
|
||||||
|
When `_save()` answers no, `_remember()` restores the state the read started
|
||||||
|
from — **not** an empty one.
|
||||||
|
|
||||||
|
The alternative considered was to clear `self._days` outright, on the argument
|
||||||
|
that a store which cannot write is a store that should stop answering. It is
|
||||||
|
wrong, and measurably so: the state being restored is the last one that **is on
|
||||||
|
disk**, so answering it is not a promise the store cannot keep. A site whose
|
||||||
|
volume fills at 14:00 goes on answering the yesterday it wrote at 13:30 —
|
||||||
|
through the restart, because that record is in the file. What it stops doing is
|
||||||
|
*adding* to it.
|
||||||
|
|
||||||
|
Clearing would throw away a good yesterday that survives a restart in order to
|
||||||
|
punish a write that failed, and it would make the morning fill worse on exactly
|
||||||
|
the day the operator is already dealing with a full disk.
|
||||||
|
|
||||||
|
That gives the file the invariant its rule was always describing, stated once
|
||||||
|
in the class docstring:
|
||||||
|
|
||||||
|
> `self._days` **never holds a point the file does not.**
|
||||||
|
|
||||||
|
⚠️ Not *"mirrors the file"* — `_usable_days()` drops records the file still
|
||||||
|
holds, and that asymmetry is deliberate. The only direction that can **invent a
|
||||||
|
yesterday** is the one named.
|
||||||
|
|
||||||
|
⚠️ **And the snapshot is `copy.deepcopy`, not `dict(self._days)`.** A one-level
|
||||||
|
copy shares the day *records*, and the merge rebinds `points` on the ones it
|
||||||
|
touches — so the "restored" store would answer the merge that never reached the
|
||||||
|
file. Measured: with a shallow copy, a store holding 12 persisted points
|
||||||
|
answered **24** after a failed merge. It would also be correct only for as long
|
||||||
|
as nobody makes the merge mutate a `points` list or a row in place, which is the
|
||||||
|
same "correct for the conditions someone thought of" this ticket is about. The
|
||||||
|
store holds at most `KEEP_DAYS + 2` records and writes 48 times a day; a copy
|
||||||
|
that is right by construction is worth more than the allocations.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What this does not change
|
||||||
|
|
||||||
|
- **No shipped path reaches the defect.** `/data` is the Supervisor's durable
|
||||||
|
volume and this module is its only writer, so on a real install `_save()`
|
||||||
|
succeeds and nothing above is on the hot path. The fix is that a guarantee the
|
||||||
|
module states about itself now holds in the condition it did not.
|
||||||
|
- **The failure was already visible and still is** — `_LOG.error("could not
|
||||||
|
persist price history: %s")` and `last_error`, reported by both `words` and
|
||||||
|
`yesterday()`'s reason. What changed is that it is now *acted on*.
|
||||||
|
- **Nothing about the prices.** The series served is the feed's own, every
|
||||||
|
carried point is still flagged provisional, and a broken cache is still a
|
||||||
|
short horizon rather than a §11.4 degradation.
|
||||||
@@ -149,8 +149,70 @@ each swept across the **whole range `config.yaml` accepts** for the two settings
|
|||||||
that decide how fast the site can turn around: `gain` `float(0.05,1.0)` and
|
that decide how fast the site can turn around: `gain` `float(0.05,1.0)` and
|
||||||
`slew_w` `int(50,5000)`.
|
`slew_w` `int(50,5000)`.
|
||||||
|
|
||||||
|
### ⚠️ THE HARNESS BEHIND EVERY FIGURE ABOVE IS NOT COMMITTED, AND UNTIL 2026-09-07 THAT MADE THEM UNCHECKABLE
|
||||||
|
|
||||||
|
**Read this before quoting any number in this section.** The paragraph below
|
||||||
|
said the campaign harness "is not committed here" and treated that as a
|
||||||
|
packaging detail. It is not: a figure produced by a script that exists in
|
||||||
|
nobody's tree cannot be re-run, cannot be reviewed, and cannot be defended when
|
||||||
|
it is contradicted. It has now been contradicted twice.
|
||||||
|
|
||||||
|
| sweep | worst peak | worst continuous | trips |
|
||||||
|
|---|---:|---:|---:|
|
||||||
|
| the 924-run campaign below — **harness uncommitted** | 1700 W | 505.0 s at 100 W; 5.0 s above 700 W | 0 / 924 |
|
||||||
|
| independent reviewer sweep, 2026-09-07, 48 honest reversals | **2750 W** | **15.0 s above 700 W** | 0 / 48 |
|
||||||
|
| `admin/goodwe` `two_master_circulation`, 840 runs — **committed** | 1048 W | 10.0 s at 100 W; 5.0 s above 700 W | **0 / 840** |
|
||||||
|
|
||||||
|
**This is not proof that anything was fabricated, and it should not be reported
|
||||||
|
as one.** The three swept different grids. What the third one adds is a
|
||||||
|
measurement of *why* the spread is that wide — on one grid, changing nothing but
|
||||||
|
the model:
|
||||||
|
|
||||||
|
| plant / stagger | worst legitimate peak |
|
||||||
|
|---|---:|
|
||||||
|
| modelled plant (lag 10 s, ramp 800 W/s), staggers up to a whole cycle | 1048 W |
|
||||||
|
| `INSTANT` (lag 0, unbounded ramp), staggers up to a whole cycle | 2250 W |
|
||||||
|
| `INSTANT`, staggers strictly under a whole cycle | **0 W** |
|
||||||
|
|
||||||
|
A factor of 2.1×, and a factor of infinity, between three defensible
|
||||||
|
descriptions of the same rig. **The plant model and the write-stagger set are
|
||||||
|
terms of the number**, and a worst case quoted without them is not a
|
||||||
|
measurement. That is enough to explain the whole spread above without anybody
|
||||||
|
having invented anything — and it is exactly what cannot be established when the
|
||||||
|
instrument is missing.
|
||||||
|
|
||||||
|
**What survives, and what does not.**
|
||||||
|
|
||||||
|
* **The window argument survives, and is now committed.** The claim that
|
||||||
|
decides `circulation_hold_s: 75` is *"the worst continuous legitimate episode
|
||||||
|
above 700 W is about one control cycle"*. All three sweeps agree on the shape
|
||||||
|
of that (5.0 s, 15.0 s, 5.0 s — one to three cycles against a fifteen-cycle
|
||||||
|
window), and **all three trip zero times**. `two_master_circulation` re-runs
|
||||||
|
it on every pull request in `admin/goodwe`.
|
||||||
|
* **The threshold curve does NOT survive as stated.** `505.0 s at 100 W` is the
|
||||||
|
number the "knee at 400 W" argument for `circulation_w: 700` rests on, and the
|
||||||
|
committed sweep measures **10.0 s** at 100 W on its grid — 50× smaller. The
|
||||||
|
curve still *falls* with the threshold (126 of 840 runs cross 100 W against 1
|
||||||
|
that crosses 700 W), so the knee is real in direction; its depth is not
|
||||||
|
established by anything anybody can run. **Do not quote 505.0 s.**
|
||||||
|
* **Nothing about the guard is compromised.** 0 trips in 924 + 48 + 840
|
||||||
|
non-faulty runs, from three independent grids.
|
||||||
|
|
||||||
|
**What would settle it:** the original harness, committed. Until then the
|
||||||
|
committed sweep is the only figure in this section anybody can check, and it is
|
||||||
|
labelled as its own grid's rather than as a reproduction.
|
||||||
|
|
||||||
### Reproducing it
|
### Reproducing it
|
||||||
|
|
||||||
|
⚠️ **THE COMMITTED VERSION IS `admin/goodwe`'s `sim/inject.py
|
||||||
|
two_master_circulation`** (`sim/sitesim.py:circulation_findings`, runnable on its
|
||||||
|
own as `python3 sim/sitesim.py --circulation`, `norig`, ~36 s, graded by
|
||||||
|
`checks.yml` on every pull request there). It produces the fault with
|
||||||
|
`plantsim.Inverter.wedge()` — a unit answering its bus and no longer steering its
|
||||||
|
power stage, which is HW-01 — and sweeps the legitimate family with the
|
||||||
|
serialised write path staggered up to a whole control cycle. The paragraph below
|
||||||
|
describes the ORIGINAL campaign, which is the one that is not committed.
|
||||||
|
|
||||||
The rig lives in `admin/goodwe` (`sim/`) and was used **read-only**; the harness
|
The rig lives in `admin/goodwe` (`sim/`) and was used **read-only**; the harness
|
||||||
is not committed here because it imports that repo's `plantsim`/`frame_check`,
|
is not committed here because it imports that repo's `plantsim`/`frame_check`,
|
||||||
which are not present in this one or in CI. What it does is small enough to
|
which are not present in this one or in CI. What it does is small enough to
|
||||||
@@ -206,7 +268,11 @@ margin above the knee is what absorbs that without re-opening this card.
|
|||||||
**Revisit when SPIKE-04 lands, not before.**
|
**Revisit when SPIKE-04 lands, not before.**
|
||||||
|
|
||||||
The **schema floor is the knee itself**, 400 W, so a legally configured install
|
The **schema floor is the knee itself**, 400 W, so a legally configured install
|
||||||
cannot land in the 445–505 s regime. What the threshold costs: a circulating
|
cannot land in the 445–505 s regime — ⚠️ and those two figures are from the
|
||||||
|
uncommitted campaign; the committed sweep measures 10.0 s at 100 W on its own
|
||||||
|
grid. See the disclosure at the head of this section. The floor is still the
|
||||||
|
right shape (fewer runs cross a higher threshold, measured 126/840 at 100 W
|
||||||
|
against 1/840 at 700 W); its DEPTH is not established by anything re-runnable. What the threshold costs: a circulating
|
||||||
loop smaller than 350 W per unit is not reported, which is deliberate — at that
|
loop smaller than 350 W per unit is not reported, which is deliberate — at that
|
||||||
scale the round-trip loss is a few watts.
|
scale the round-trip loss is a few watts.
|
||||||
|
|
||||||
@@ -289,6 +355,32 @@ SAFETY-25 — so no stale `.pyc` could exist to be read. Byte deltas recorded;
|
|||||||
| M17 | `circulation.py` | default `700` → `500` | **+0 B** | **RED** — the CFG-03 pin and the not-a-round-number check |
|
| M17 | `circulation.py` | default `700` → `500` | **+0 B** | **RED** — the CFG-03 pin and the not-a-round-number check |
|
||||||
| M18 | `main.py` | `await self.check_circulating()` removed | −39 B | **RED** — all four notification checks |
|
| M18 | `main.py` | `await self.check_circulating()` removed | −39 B | **RED** — all four notification checks |
|
||||||
| M19 | `main.py` | publish `round(x or 0.0)` instead of `None` | −68 B | **survivor on the first battery — GAP CLOSED**, then RED |
|
| M19 | `main.py` | publish `round(x or 0.0)` instead of `None` | −68 B | **survivor on the first battery — GAP CLOSED**, then RED |
|
||||||
|
| **M20** | `main.py` | **`pos, neg = opposing(...)` → `([], [])`** — the attribution clause deleted from the alarm | −102 B | **SURVIVED ALL 26 FILES on the shipped battery — GAP CLOSED**, then RED on §6a |
|
||||||
|
| M21 | `main.py` | `pos` and `neg` exchanged — the alarm names both packs, both the wrong way round | **+0 B** | **RED** — §6a ×3 (the mirrored run, and "only its own sentence") |
|
||||||
|
| M22 | `main.py` | `opposing(..., min_w=cfg.min_w / 2.0)` → `min_w=0.0` | −25 B | **RED** — §6a's resting-pack check |
|
||||||
|
|
||||||
|
**And four in `admin/goodwe`**, graded through `python3 sim/sitesim.py
|
||||||
|
--circulation` rather than through the injector — deliberately, because
|
||||||
|
`inject.py` is the file that takes the rig lock, and "run the injector to grade a
|
||||||
|
mutation" is not a sentence anybody should have to type. Byte deltas over
|
||||||
|
`plantsim.py` + `sitesim.py` together.
|
||||||
|
|
||||||
|
| # | file | mutation | Δ | result |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| S1 | `plantsim.py` | `wedge()` no longer stops the latch reaching the power stage | −20 B | **RED** — the opposing-signs precondition **and** "the watchdog did not trip" |
|
||||||
|
| S2 | `sitesim.py` | the sensor computed over the **allocation record** instead of register 0x0518 | **+0 B** | **RED** ×3 — this is M10 restated on the rig |
|
||||||
|
| S3 | `sitesim.py` | the campaign run on `plant=INSTANT` | −51 B | **RED** — the plant is a *term* of the figures; it was a SURVIVOR until that check was added, and finding out why produced §3's 1048/2250/0 W table |
|
||||||
|
| S4 | `sitesim.py` | the write stagger removed from the legitimate campaign | −19 B | **RED** — "the legitimate campaign produced no opposition at all" |
|
||||||
|
|
||||||
|
**S3 is recorded as a survivor because it was one, and it is the most useful
|
||||||
|
result in this table.** The scenario first shipped with a vacuity check whose
|
||||||
|
text claimed `plant=INSTANT` would make the sweep measure 0 W. S3 was run and
|
||||||
|
**passed**: `INSTANT` with a stagger of exactly one control cycle samples the
|
||||||
|
plant before the second unit has stepped, so it produces opposition anyway — and
|
||||||
|
a *larger* worst peak (2250 W) than the modelled plant does (1048 W). The claim
|
||||||
|
in the check's own text was wrong. It was corrected against the measurement, the
|
||||||
|
plant was pinned as an explicit term, and S3 is red now for a stated reason
|
||||||
|
rather than by accident. §3's spread table is that measurement.
|
||||||
|
|
||||||
**M19 is recorded as a survivor because it was one.** The first battery found no
|
**M19 is recorded as a survivor because it was one.** The first battery found no
|
||||||
check grading what `publish()` does when a per-unit reading is missing, so
|
check grading what `publish()` does when a per-unit reading is missing, so
|
||||||
@@ -297,6 +389,30 @@ watches went unnoticed — the sensor's own `None` rule, defeated one layer out.
|
|||||||
A check was added and the mutation re-run; it is red now. The run that found it
|
A check was added and the mutation re-run; it is red now. The run that found it
|
||||||
was purged, so it is a result and not an artefact.
|
was purged, so it is a result and not an artefact.
|
||||||
|
|
||||||
|
**M20 IS RECORDED AS A SURVIVOR BECAUSE IT WAS ONE, AND IT IS THE MORE
|
||||||
|
UNCOMFORTABLE OF THE TWO.** The commit message for this card claimed *"18
|
||||||
|
mutations, all RED"*, and M1–M19 above have no entry for the notification's
|
||||||
|
attribution clause. Verification on 2026-09-07 ran it and it **survived the
|
||||||
|
whole 26-file suite**: the alarm still fires, still stops the site, still says
|
||||||
|
the meter cannot see it — and no longer says *which two packs* or *which way
|
||||||
|
round*. `circulation.py`'s own docstring calls that sentence the first thing
|
||||||
|
somebody woken by this alarm needs and *"impossible to reconstruct from the
|
||||||
|
number alone"*.
|
||||||
|
|
||||||
|
It survived for one reason, and it is not reachability: **M18 (the whole call
|
||||||
|
removed) is red**, so `check_circulating()` is exercised. The three checks on
|
||||||
|
that notification asserted `METER CANNOT SEE`, `CONTROL HAS BEEN STOPPED` and
|
||||||
|
`isolator` — and **never an inverter name**. Nothing graded the attribution.
|
||||||
|
|
||||||
|
`test_watchdog.py` §6a closes it, and closes it on a fixture that can tell the
|
||||||
|
three failures apart. It runs a roster of **`["garage", "shed", "cellar"]`** —
|
||||||
|
operator names, none of them `UNIT_A`, for test_maintenance.py §11's reason:
|
||||||
|
on a fixture whose names are the module's own constants a check cannot tell the
|
||||||
|
roster from the constant. It runs the site **twice, mirrored**, so the sentence
|
||||||
|
is not symmetric and a swap (M21, `+0 B`) is a different red from a deletion.
|
||||||
|
And it carries a **third pack resting at 5 W**, so the floor that stops a pack
|
||||||
|
in standby being named as a participant in a 6 kW loop (M22) is graded too.
|
||||||
|
|
||||||
## 7. What this card does NOT close
|
## 7. What this card does NOT close
|
||||||
|
|
||||||
* **Inverter B's ramp and lag are placeholders.** Every transient number in §3
|
* **Inverter B's ramp and lag are placeholders.** Every transient number in §3
|
||||||
@@ -305,7 +421,9 @@ was purged, so it is a result and not an artefact.
|
|||||||
margin. **Do not present any figure here as a measurement of inverter B.**
|
margin. **Do not present any figure here as a measurement of inverter B.**
|
||||||
* **A wedged inverter still cannot be stopped by this add-on.** §1 says so; the
|
* **A wedged inverter still cannot be stopped by this add-on.** §1 says so; the
|
||||||
notification says so; there is no software fix and this card does not pretend
|
notification says so; there is no software fix and this card does not pretend
|
||||||
to one.
|
to one. It *can* now be produced on demand: `plantsim.Inverter.wedge()`.
|
||||||
|
* **The depth of the threshold curve is not established by anything committed.**
|
||||||
|
§3's disclosure. The window is; the knee's magnitude is not.
|
||||||
* **A loop below ~350 W per unit is not reported.** Deliberate, §3.
|
* **A loop below ~350 W per unit is not reported.** Deliberate, §3.
|
||||||
* **The saturation freeze fires under a wedged unit and leaves a standing grid
|
* **The saturation freeze fires under a wedged unit and leaves a standing grid
|
||||||
error.** Observed on the rig while producing §2's numbers: with B wedged, the
|
error.** Observed on the rig while producing §2's numbers: with B wedged, the
|
||||||
|
|||||||
@@ -1424,6 +1424,85 @@ distinct state from "prices, all of them zero": the plan is marked provisional
|
|||||||
and arbitrage is disabled, per the degradation matrix. It never falls back to a
|
and arbitrage is disabled, per the degradation matrix. It never falls back to a
|
||||||
price of zero, which would read as free electricity.
|
price of zero, which would read as free electricity.
|
||||||
|
|
||||||
|
#### The hours the feed has not published yet (FCST-04)
|
||||||
|
|
||||||
|
A day-ahead auction publishes tomorrow's prices once a day, at **14–16 h CET**.
|
||||||
|
Before that, the feed is serving today and nothing else — so a `price_horizon_h`
|
||||||
|
of 48 asks for 48 hours and gets back the ten or fifteen that are left in the
|
||||||
|
day. Until FCST-04 those other hours were simply **absent** from the plan: not
|
||||||
|
priced badly, not flagged, just missing, and nothing said so.
|
||||||
|
|
||||||
|
Now the add-on keeps the **published prices of the last few days** in
|
||||||
|
`/data/price_history.json` and fills an unpublished hour of the horizon with
|
||||||
|
yesterday's price for the same Brussels wall-clock hour, flagged
|
||||||
|
`provisional: true`. Two things follow, and the second one matters more:
|
||||||
|
|
||||||
|
- **The plan says it is guessing** — `provisional: true` on every carried hour,
|
||||||
|
which is what the optimiser reads. Before, a half-length plan looked
|
||||||
|
completely settled.
|
||||||
|
- **Nothing may trade on a carried hour.** §8.3 forbids acting on a provisional
|
||||||
|
price, so arbitrage drops those hours from the trade set entirely — it needs a
|
||||||
|
buy hour and a sell hour that were *both* published. Carrying a price forward
|
||||||
|
widens what the optimiser can **see**; it never widens what it may **do**.
|
||||||
|
|
||||||
|
**"An unpublished hour", not "every unpublished hour"** — the fill can only
|
||||||
|
carry an hour yesterday actually published. If yesterday's record is itself
|
||||||
|
partial (a first day, an outage, a feed that only ever serves the rest of the
|
||||||
|
day), the local hours it never held are **left out of the series**: §11.4
|
||||||
|
applies to those hours, and asking the plan for one raises rather than
|
||||||
|
answering. An hour with no price is not a free hour, and the fill does not
|
||||||
|
invent a neighbour, a mean or a zero for it.
|
||||||
|
|
||||||
|
**A quarter-hourly feed keeps all four of its prints.** `eneco_eddy` settles
|
||||||
|
per quarter, and the fill decides *which whole hours need carrying* — never what
|
||||||
|
a published hour may contain. Every point the provider published inside the
|
||||||
|
horizon is passed through untouched; a carried point is added only for a whole
|
||||||
|
hour the feed did not print at all.
|
||||||
|
|
||||||
|
**What you can actually see on the status page.** *Price history* (below) says
|
||||||
|
what is held. *Arbitrage* carries the strategy's own sentence, which names the
|
||||||
|
firm/provisional split whenever that split is what refused a trade. *Tariff*
|
||||||
|
does **not** report the fill: it is the static-profile or Home Assistant
|
||||||
|
price-entity sentence, regenerated on the 30 s tick. The provider plan's own
|
||||||
|
line — `dynamic_dayahead from generic_dayahead, 48 h, provisional` — goes to the
|
||||||
|
**add-on log** at every refresh, alongside `§8.3: 33 of 48 hours carried forward
|
||||||
|
from yesterday's profile and flagged provisional; 15 published`.
|
||||||
|
|
||||||
|
The plan is also now cut to the horizon you asked for, which means **hours that
|
||||||
|
have already elapsed are no longer in it**. That is a real change to arbitrage:
|
||||||
|
a cheap night that has already gone used to sit in the series as the cheapest
|
||||||
|
hour of the day, and a cycle can never buy in an hour that is past — so the
|
||||||
|
trade was quietly never taken. The spread is now taken over hours still ahead.
|
||||||
|
|
||||||
|
A new status row, **Price history**, says which days are held, how many price
|
||||||
|
points each holds and how many whole hours those cover — `2026-01-13 (96 points
|
||||||
|
over 24 h)`. ⚠️ **Those are two different numbers and the row used to print one
|
||||||
|
of them under the other's name**: it counted points and labelled them `h`, so a
|
||||||
|
quarter-hourly feed read `96 h` for a 24-hour day. The figure that says whether
|
||||||
|
tomorrow morning's fill has a whole day to carry is the **hour** one. If it
|
||||||
|
reads `empty` or `unavailable`, nothing is carried, the horizon
|
||||||
|
is as short as the feed makes it, and every other guarantee is unchanged — a
|
||||||
|
cache that cannot answer is not allowed to be a controller that cannot regulate.
|
||||||
|
The store is rebuilt by the next successful fetch; on a brand-new install the
|
||||||
|
first fill is possible the day after the first refresh.
|
||||||
|
|
||||||
|
The same applies to a file that is *partly* broken: a day record this build
|
||||||
|
cannot read is dropped, the row says how many were dropped and why, and the rest
|
||||||
|
of the file still loads. And a store that **cannot persist records nothing** —
|
||||||
|
not even in memory. That covers a missing `/data`, and it also covers a `/data`
|
||||||
|
that is there and cannot be **written**: a full volume, a read-only remount. An
|
||||||
|
in-memory yesterday would answer for one run and vanish on the next restart,
|
||||||
|
which is a cache that forgets without saying so; the row naming the reason and
|
||||||
|
*no prices carried* are the honest pair.
|
||||||
|
|
||||||
|
⚠️ **What a failed write does not do is throw away what is already in the
|
||||||
|
file.** A volume that fills at 14:00 leaves the yesterday written at 13:30 still
|
||||||
|
answering, because that record survives a restart — the store stops *adding* to
|
||||||
|
it, nothing more. And it never stops the add-on: a write that cannot complete is
|
||||||
|
logged and reported on the status row, never raised. A cache that cannot answer
|
||||||
|
is not allowed to be a controller that cannot regulate, and that applies to the
|
||||||
|
volume being full exactly as it applies to the volume being absent.
|
||||||
|
|
||||||
**Injection is priced separately** (`export_price_eur_kwh`). In Belgium it is
|
**Injection is priced separately** (`export_price_eur_kwh`). In Belgium it is
|
||||||
often near zero, which by itself strongly favours self-consumption over export.
|
often near zero, which by itself strongly favours self-consumption over export.
|
||||||
It is a single number rather than a curve because no supported supplier
|
It is a single number rather than a curve because no supported supplier
|
||||||
@@ -1574,6 +1653,185 @@ battery power, circulating power, state of charge, maintenance phase and
|
|||||||
controller status by MQTT discovery. This is observability only — the controller works fine without a
|
controller status by MQTT discovery. This is observability only — the controller works fine without a
|
||||||
broker, and MQTT problems can never affect control.
|
broker, and MQTT problems can never affect control.
|
||||||
|
|
||||||
|
## Before you commission inverter B: `tools/rehearse_roster.py` (ARCH-02)
|
||||||
|
|
||||||
|
The day the `inverters:` roster is declared, two things change that are
|
||||||
|
**one-way on a live box and invisible until after the restart that applies
|
||||||
|
them**: the store migration renames the keys that hold the SAFETY-02 latch and
|
||||||
|
the maintenance history, and the entity ids Home Assistant composes change
|
||||||
|
shape. This tool runs the **real `Controller`** through that transition against
|
||||||
|
a **copy** of the state file and reports the difference.
|
||||||
|
|
||||||
|
docker run --rm -e PYTHONDONTWRITEBYTECODE=1 -v <repo>:/w:ro \
|
||||||
|
goodwe-addon-checks:1 sh -c 'cd /w/goodwe_controller && \
|
||||||
|
python3 tools/rehearse_roster.py \
|
||||||
|
--state /w/state-copy.json \
|
||||||
|
--before /w/live-options.json \
|
||||||
|
--after /w/roster-options.json \
|
||||||
|
--live-entities /w/live-entities.txt \
|
||||||
|
--reverse --round-trip'
|
||||||
|
|
||||||
|
Exit `0` clean · `1` at least one finding · `2` could not run. ⚠️ **`2` is not a
|
||||||
|
pass.**
|
||||||
|
|
||||||
|
### The owner steps, because this session could not take them
|
||||||
|
|
||||||
|
⚠️ **The rehearsal is only as good as the state it is pointed at, and nothing in
|
||||||
|
this repository can fetch that.** SSH to the live box is blocked by policy and
|
||||||
|
that is correct — `/data/state.json` is the file the running add-on is writing.
|
||||||
|
So three things have to be exported by hand, onto the machine that runs the
|
||||||
|
check, **before** the roster is declared:
|
||||||
|
|
||||||
|
1. **The state file.** From the Supervisor host:
|
||||||
|
|
||||||
|
ha addons info local_goodwe_controller # confirm the slug first
|
||||||
|
docker cp addon_local_goodwe_controller:/data/state.json ./state-copy.json
|
||||||
|
|
||||||
|
or take it out of a full add-on backup, which is the same bytes. ⚠️ **Copy
|
||||||
|
it, never move it, and never write to the original.** The tool opens the path
|
||||||
|
it is given exactly once and does all of its writing on a scratch copy — but
|
||||||
|
the safe habit is to hand it a file the add-on has never heard of.
|
||||||
|
|
||||||
|
2. **The options blob**, as `--before`. It is not reachable through the Home
|
||||||
|
Assistant REST API and no entity carries it; export it from the Supervisor,
|
||||||
|
the same way `tools/predeploy_check.py` needs it. Write the roster you intend
|
||||||
|
to deploy into a second file and pass it as `--after`.
|
||||||
|
|
||||||
|
3. **The entity list**, as `--live-entities` — one entity id per line:
|
||||||
|
|
||||||
|
Developer tools -> Template:
|
||||||
|
{{ states | map(attribute='entity_id') | select('search','goodwe') | join('\n') }}
|
||||||
|
|
||||||
|
⚠️ **Omitting it is a REFUSAL (exit 2), not a quieter run.** The orphan check
|
||||||
|
is graded against the ids the install actually has, and without that file it
|
||||||
|
is not graded against reality at all — so the tool will not run rather than
|
||||||
|
hand back a clean bill of health for a half of the transition it did not
|
||||||
|
check. If you genuinely have to run without it, pass `--no-live-entities`:
|
||||||
|
the run then carries a standing banner in the report header **and** on the
|
||||||
|
exit line saying the orphan check was not graded. The asymmetry used to be
|
||||||
|
the other way round — an unreadable file refused, a missing flag passed with
|
||||||
|
exit 0 and no disclosure at all — and the owner runs this once, at
|
||||||
|
commissioning, under time pressure.
|
||||||
|
|
||||||
|
The install carries roughly 49 `goodwe*` entities and this add-on announces
|
||||||
|
eight of them. The tool separates the ids it owns from the ids it does not,
|
||||||
|
rather than claiming orphans it cannot cause.
|
||||||
|
|
||||||
|
### The reference, which is loaded on every run
|
||||||
|
|
||||||
|
`--reference` defaults to `tools/rehearsal/announced-reference.txt` and is read
|
||||||
|
**whether or not** you pass an entity list. It records what the build is
|
||||||
|
supposed to announce, three columns per row:
|
||||||
|
|
||||||
|
<entity_id> <unique_id> <device identifier>
|
||||||
|
|
||||||
|
⚠️ **All three, and the second and third are the ones an entity_id-only list
|
||||||
|
cannot see.** The tool's own before/after diff is composed twice by the same
|
||||||
|
build, so a change to the composition moves both sides equally and the diff
|
||||||
|
reports nothing. Worse, two changes orphan an install *without moving a single
|
||||||
|
composed entity_id*: renaming `mqtt.BASE` (every `unique_id` changes) and
|
||||||
|
changing `DEVICE["identifiers"]` (HA sees a new device). Home Assistant keys its
|
||||||
|
entity registry on `unique_id` and its device registry on the identifier, so
|
||||||
|
either one creates a **new** entity beside the old — and because discovery is
|
||||||
|
published `retain=True` with no removal payload, the old config stays on the
|
||||||
|
broker and the old entity stays in the registry, still named by every automation
|
||||||
|
that referenced it, never updating again. Both were measured coming back
|
||||||
|
**exit 0** before the reference carried three columns.
|
||||||
|
|
||||||
|
`object_id` is deliberately **not** a column: `app/mqtt.py` records that HA
|
||||||
|
2026.8 does not honour it in a discovery payload, so renaming one is a true
|
||||||
|
no-op, and `test_rehearse.py` requires that mutation to **survive** — otherwise
|
||||||
|
"the reference notices any edit to `mqtt.py`" would be indistinguishable from
|
||||||
|
"the reference notices what HA keys on".
|
||||||
|
|
||||||
|
⚠️ **No result in this repository was produced against real state.** Everything
|
||||||
|
recorded here and in `test_rehearse.py` was measured on the synthetic fixtures
|
||||||
|
in `tools/rehearsal/`.
|
||||||
|
|
||||||
|
### What it confirms on the synthetic fixtures
|
||||||
|
|
||||||
|
⚠️ **This is a SECOND INSTRUMENT reaching a conclusion this repository already
|
||||||
|
holds, not a discovery.** `test_maintenance.py` section 13 — *"RECORDED, NOT
|
||||||
|
FIXED: SAFETY-02's migration has the same wrong-namespace hole, and it is a
|
||||||
|
LATCH"* — already states it, already names `namespaced_latched`, and already
|
||||||
|
carries the sentence *"the guard is defeated by the very key the defect wrote"*.
|
||||||
|
That file is on `release/1.0` and this card does not touch it. What is new here
|
||||||
|
is that the finding is now reachable **from a state file the owner supplies**
|
||||||
|
rather than only from a fixture built in a test, and that it is reached by a
|
||||||
|
different route: a full `Controller` start diffed against its predecessor,
|
||||||
|
rather than assertions over a `MemStore`. Two instruments, one answer.
|
||||||
|
|
||||||
|
**The finding.** A pre-DEV-02 flat store that declares a multi-inverter roster
|
||||||
|
comes up with its SAFETY-02 latch **DISARMED**, unless the first roster entry
|
||||||
|
happens to be named `inverter-a`. `Controller.__init__` calls
|
||||||
|
`migrate_unit_keys(store, UNIT_A)` with a hardcoded constant, so the latch lands
|
||||||
|
in `unit/inverter-a/sign_tripped`; a roster of `["garage", "shed"]` reads
|
||||||
|
`unit/garage/sign_tripped`, which is absent. And `_grade_store_migration()` —
|
||||||
|
the guard written for exactly this — sees the migration's own misdirected key
|
||||||
|
satisfy its `namespaced_latched` test and returns without re-arming. The file
|
||||||
|
says `sign_tripped: true`, the banner is green, and nothing is held at 0 W.
|
||||||
|
|
||||||
|
**Both routes reach it, and they are NOT the same defect.** The rehearsal
|
||||||
|
reports them as separate legs because they need separate fixes:
|
||||||
|
|
||||||
|
| leg | how it gets there |
|
||||||
|
|---|---|
|
||||||
|
| `SAME-STEP` | the roster is declared in the same step as the upgrade. The migration runs, and is handed `UNIT_A`. |
|
||||||
|
| `STAGED` | the flat config runs once under 0.7.0+ first. That start stamps `schema_version = 1`, so the roster start's migration takes its `from_version >= SCHEMA_VERSION` early return and **the latch never moves at all**. |
|
||||||
|
|
||||||
|
⚠️ **So "hand `migrate_unit_keys()` the roster's own first name" is NOT the
|
||||||
|
whole fix, and the rehearsal measures that rather than asserting it.**
|
||||||
|
`test_rehearse.py` part 5b builds exactly that change and reports:
|
||||||
|
|
||||||
|
LEG STAGED: DISARMED
|
||||||
|
LEG SAME-STEP: clean
|
||||||
|
|
||||||
|
The staged path never reaches the line that was changed. A fix that stops there
|
||||||
|
closes the half an operator is least likely to take and leaves the half that
|
||||||
|
"upgrade first, declare the roster later" produces. The guard side —
|
||||||
|
`namespaced_latched` counting a namespace that is not on the roster — is what
|
||||||
|
covers both, and a change there turns every SAFETY-44 check in part 5 red, which
|
||||||
|
is what a fix should do to a pin that specifies a defect.
|
||||||
|
|
||||||
|
The fix belongs to its own card, shown red first. Part 5 pins the current
|
||||||
|
behaviour as a requirement so it cannot drift green by accident.
|
||||||
|
|
||||||
|
### The reverse direction: the retreat is supported, the round trip is not
|
||||||
|
|
||||||
|
If commissioning B goes wrong, **backing out to the flat single-inverter config
|
||||||
|
is supported and does not corrupt state.** Measured on the fixtures:
|
||||||
|
|
||||||
|
* the SAFETY-02 latch and the first pack's maintenance history both survive,
|
||||||
|
because `store.py` rule 2 keeps the flat spellings **live** rather than merely
|
||||||
|
present — `maintenance.UnitHistory.mirror_flat` writes them on every change,
|
||||||
|
so a single-FSM add-on finds a current history rather than one dated the day
|
||||||
|
the roster was declared;
|
||||||
|
* **it orphans eight Home Assistant entities** — the four per-unit rows for each
|
||||||
|
of the two units. **Measured:** the add-on stops announcing them and stops
|
||||||
|
publishing their state, so their history gaps and any automation whose
|
||||||
|
trigger names them stops firing, silently and without an error.
|
||||||
|
|
||||||
|
⚠️ **What is NOT measured, and was previously overstated here:** whether
|
||||||
|
advancing again restores them. An earlier draft of this section said the
|
||||||
|
orphaning was "not recoverable by advancing again: HA keeps the id it composed
|
||||||
|
at first discovery" — that reasoning cuts the other way, since keeping the
|
||||||
|
composed id is precisely what would let a re-advertised entity land back on
|
||||||
|
it, and discovery is published `retain=True` with no removal payload. Nothing
|
||||||
|
in this repository has tested a re-advance against a real Home Assistant, so
|
||||||
|
the claim is withdrawn rather than restated in the other direction. Plan the
|
||||||
|
retreat around the measured part: **the eight rows stop updating for as long
|
||||||
|
as you are retreated.**
|
||||||
|
|
||||||
|
⚠️ **What is NOT supported is the round trip.** A maintenance cycle that
|
||||||
|
completes *while the install is rolled back* is **discarded** when it
|
||||||
|
re-advances: `maint_schema_version` is already stamped, so `migrate_maint_keys()`
|
||||||
|
takes its early return and the pack reads the namespaced history from before the
|
||||||
|
retreat. The pack is then recalibrated on a schedule that does not know it
|
||||||
|
already was. **So: retreat freely, but treat a retreat as ending the current
|
||||||
|
maintenance bookkeeping — and if a cycle ran during it, correct
|
||||||
|
`unit/<first>/last_completed` by hand before advancing again.** This is a stated
|
||||||
|
decision, not a discovery made at the inverter.
|
||||||
|
|
||||||
## Before you deploy: `tools/predeploy_check.py` (PKG-09)
|
## Before you deploy: `tools/predeploy_check.py` (PKG-09)
|
||||||
|
|
||||||
Two ways a deploy has gone wrong on this system, both of which the Supervisor
|
Two ways a deploy has gone wrong on this system, both of which the Supervisor
|
||||||
|
|||||||
@@ -127,6 +127,43 @@ Two rules come out of it, and both are cheap:
|
|||||||
function, that pair is the first thing to try — TEST-16's two survivors were
|
function, that pair is the first thing to try — TEST-16's two survivors were
|
||||||
adjacent lines in one function and neither was individually live.
|
adjacent lines in one function and neither was individually live.
|
||||||
|
|
||||||
|
⚠️ **AND FCST-05 WALKED INTO IT AGAIN, IN ITS OWN NEW CHECK, WHICH IS WHY THE
|
||||||
|
RULE IS “mutate every check you add”.** `price_history.py` reports a malformed
|
||||||
|
point row from **two** places into **one** field: `_usable_days()` records what
|
||||||
|
it FOUND when the file was opened, and `_remember()` records what it DROPPED on
|
||||||
|
the next refresh. Both land in `last_error`, and `words` prints `last_error`.
|
||||||
|
The first draft of the new check asked `"malformed point row" in words` — which
|
||||||
|
whichever rung is still standing satisfies — so deleting `_remember()`'s note
|
||||||
|
entirely was a **−1 B SURVIVOR of the whole suite**, on the half of the fix that
|
||||||
|
tells an operator the cache **self-healed**. It is graded on the TENSE now
|
||||||
|
(`… in <day>` at open, `… dropped from <day>` after the refresh), the two are
|
||||||
|
asserted to DIFFER, and the paired mutation — both notes deleted, −91 B — is
|
||||||
|
red on 15 named checks. Same shape as TEST-16 and the same fix: **grade the
|
||||||
|
reason, not the verdict**, and when two rungs answer one stimulus, mutate them
|
||||||
|
together.
|
||||||
|
|
||||||
|
⚠️ **AND FCST-06 FOUND THE COUNT ITSELF LYING — LOW, NOT SILENT.** A mutation
|
||||||
|
whose red is a **traceback** does not report one red; it reports however many
|
||||||
|
checks ran before the traceback, and grades **none** of the ones after it. The
|
||||||
|
new section drove M4 (`self._days = before` → `self._days = {}`) and the run
|
||||||
|
printed **1 FAIL**. The true figure is **3**: an f-string in the next check read
|
||||||
|
`len(read.points)`, and `.points` on a `no_data` **raises `Absence` by design**,
|
||||||
|
so the harness died mid-section with the remaining checks ungraded. That is
|
||||||
|
`why()`'s and `derived()`'s rule — *"it answered nothing" and "it broke on the
|
||||||
|
way to answering" must not grade the same* — arriving in the **printed operand**
|
||||||
|
of a check rather than in its condition, which is the half that is easy to miss
|
||||||
|
because it looks like formatting. Two things follow:
|
||||||
|
|
||||||
|
* **a mutation report's red COUNT is only a lower bound until the run is seen to
|
||||||
|
have reached the end of its section.** Read the output, not the count;
|
||||||
|
* **the total accessor has to cover what a check PRINTS, not only what it
|
||||||
|
asserts.** `test_price_history.py` 5d added `npts()` beside `why()` for
|
||||||
|
exactly this, and re-driving M4 against it moved 1 FAIL to 3.
|
||||||
|
|
||||||
|
Worse, had the traceback landed one check earlier the mutation would have
|
||||||
|
produced **no named red at all** — a result that reads like a broken harness and
|
||||||
|
gets retried rather than read.
|
||||||
|
|
||||||
## The third way a check lies: its DATA cannot reach the failure it names
|
## The third way a check lies: its DATA cannot reach the failure it names
|
||||||
|
|
||||||
Stale bytecode manufactures survivors from a mutation that never ran. Masking
|
Stale bytecode manufactures survivors from a mutation that never ran. Masking
|
||||||
@@ -225,6 +262,78 @@ same place twice, the check is decoration.
|
|||||||
forever, *including after the call site is fixed*. The check was true and
|
forever, *including after the call site is fixed*. The check was true and
|
||||||
passed for the wrong reason. Habit **9** below is what falls out of it.
|
passed for the wrong reason. Habit **9** below is what falls out of it.
|
||||||
|
|
||||||
|
* ⚠️ **ARCH-02's OWN REHEARSAL HARNESS, CAUGHT BY ITS OWN MUTATION BATTERY —
|
||||||
|
the THIRTEENTH instance, and the third in an INSTRUMENT rather than in a
|
||||||
|
check.** `tools/rehearse_roster.py` reports orphaned Home Assistant entities
|
||||||
|
by composing the ids the add-on announces **before** the roster switch and
|
||||||
|
**after** it, and diffing. Both sides are composed by the same build. So the
|
||||||
|
two mutations that change the id composition itself — renaming the MQTT device
|
||||||
|
(`"GoodWe"` -> `"GoodWe Controller"`), renaming one site sensor (`"Grid power"`
|
||||||
|
-> `"Grid power W"`) — moved **every id on both sides equally**, and the
|
||||||
|
harness reported *"no orphans"* about a change that orphans the entire live
|
||||||
|
entity list. It read exactly like a clean transition. The first draft's
|
||||||
|
M5/M6 came back `exit 0` and `exit 2`, and neither was the failure they name.
|
||||||
|
|
||||||
|
The fix is habit 2 with a filename: **the reference has to be data that does
|
||||||
|
not move with the code.** `--live-entities` takes a list of the ids the
|
||||||
|
install actually has, `tools/rehearsal/live-entities-recorded.txt` is the
|
||||||
|
recorded stand-in that keeps it live in CI, and a build that announces an id
|
||||||
|
the list does not carry is now a `COMPOSITION DRIFT` finding rather than a
|
||||||
|
silence. ⚠️ **The anchor moved for the same reason**: it had been
|
||||||
|
`compose(mqtt.DEVICE["name"], p1["name"]) == "sensor.goodwe_p1_sample_age"`,
|
||||||
|
which any renaming of the device satisfies by moving both operands, and it is
|
||||||
|
now composed from two recorded **literals**.
|
||||||
|
|
||||||
|
Ask of an instrument what habit 4 asks of a check: *where did each side of
|
||||||
|
this comparison come from* — and if a mutation of the subject moves both
|
||||||
|
sides, the instrument is decoration.
|
||||||
|
|
||||||
|
⚠️ **AND REVIEW FOUND TWO MORE IN THE SAME CARD, ONE OF THEM INSIDE THE FIX
|
||||||
|
FOR THE OTHER — the FOURTEENTH and FIFTEENTH.** Both are the pin, not the
|
||||||
|
tool, and both are the same sentence in a new costume: *a check whose data
|
||||||
|
cannot reach the failure it names.*
|
||||||
|
|
||||||
|
* **Fourteenth — a COUNT cannot tell one red leg from two.** The rehearsal
|
||||||
|
reports two independent routes to the disarmed latch (`SAME-STEP` and
|
||||||
|
`STAGED`) and prints every finding **twice per red leg**: once in that
|
||||||
|
leg's own block and once in the closing summary. The check named *"NOT THE
|
||||||
|
SAME-STEP PATH ALONE"* was written `out.count("LATCH CAME UP DISARMED")
|
||||||
|
>= 2`, so **one** red leg always satisfied it. Measured on three builds:
|
||||||
|
shipped code (both legs red) → 4; `migrate_unit_keys()` handed the roster
|
||||||
|
name (SAME-STEP fixed) → 2, **passes**; a guard-side fix (both fixed) → 0.
|
||||||
|
The check reported `ok` on the build whose only remaining failure *is* the
|
||||||
|
path it says is not the only one. The fix is to parse the legs and require
|
||||||
|
the finding in **each**, which is habit 2 again: grade the thing, not a
|
||||||
|
total over the thing.
|
||||||
|
|
||||||
|
* **Fifteenth — and it is INSIDE THAT FIX, which is the third time this file
|
||||||
|
has had to write that sentence.** The first leg-splitter opened on a
|
||||||
|
`LEG <id>` marker and never closed, so every line after the *last* marker
|
||||||
|
was attributed to that leg — including the closing summary, which repeats
|
||||||
|
every finding from every leg. The last leg therefore read red whenever
|
||||||
|
**any** leg was red, and the new per-leg check was the old count with extra
|
||||||
|
steps. It was caught by building the one fixture that separates them: a
|
||||||
|
tree in which `SAME-STEP` is genuinely clean and `STAGED` is not. The tool
|
||||||
|
now prints `END LEG <id>` and the splitter closes on it.
|
||||||
|
|
||||||
|
The rule that falls out, and it is the cheap one: **a per-thing check that
|
||||||
|
has never seen the things DISAGREE is a total.** Part 5b exists to make
|
||||||
|
them disagree on every run.
|
||||||
|
|
||||||
|
⚠️ **AND ONE THAT HAS ONLY SEEN THEM DISAGREE IN ONE ARRANGEMENT IS A TOTAL
|
||||||
|
OVER THAT ARRANGEMENT.** Review found this by attacking the fix itself. Part
|
||||||
|
5b does make the legs disagree, on every run - and was still blind to a build
|
||||||
|
that merely REORDERS them, because it had only ever seen the clean leg
|
||||||
|
printed LAST. Three edits, two of them ordinary refactors (reorder the report
|
||||||
|
legs, reword a summary line), produced `all checks passed` from a splitter
|
||||||
|
that closed ZERO legs.
|
||||||
|
|
||||||
|
A fixture cannot enumerate the arrangements, so the STRUCTURE is asserted
|
||||||
|
instead: a leg may not open while one is open, and every leg that opens must
|
||||||
|
be explicitly closed. That fires on the FIRST edit of all three attacks,
|
||||||
|
whatever the order or the wording. **When a fixture cannot enumerate the
|
||||||
|
arrangements, assert the invariant they share.**
|
||||||
|
|
||||||
* **CFG-04's `tools/supervisor_shape_probe.py` §B** — and this one is the rule
|
* **CFG-04's `tools/supervisor_shape_probe.py` §B** — and this one is the rule
|
||||||
reaching outside the suite. The probe is the committed evidence that
|
reaching outside the suite. The probe is the committed evidence that
|
||||||
`config.yaml`'s `inverters: []` default is load-bearing, and four documents
|
`config.yaml`'s `inverters: []` default is load-bearing, and four documents
|
||||||
|
|||||||
+207
-19
@@ -68,6 +68,7 @@ from . import peakguard
|
|||||||
from . import packshare
|
from . import packshare
|
||||||
from .p1 import P1Ingest, build_source, is_enabled
|
from .p1 import P1Ingest, build_source, is_enabled
|
||||||
from .price import Uncertainty, price_option
|
from .price import Uncertainty, price_option
|
||||||
|
from .price_history import PriceHistory
|
||||||
from .static_tariff import resolve_tariff, static_status
|
from .static_tariff import resolve_tariff, static_status
|
||||||
from .tariff import PeakStore, PeakTracker
|
from .tariff import PeakStore, PeakTracker
|
||||||
from .store import migrate_maint_keys, migrate_unit_keys
|
from .store import migrate_maint_keys, migrate_unit_keys
|
||||||
@@ -165,7 +166,8 @@ def pack_energy_of(inv):
|
|||||||
|
|
||||||
class Controller:
|
class Controller:
|
||||||
def __init__(self, opts: dict, hass: HomeAssistant, store, mqtt_pub, *, stop,
|
def __init__(self, opts: dict, hass: HomeAssistant, store, mqtt_pub, *, stop,
|
||||||
peak_db_path: str = "/data/tariff.db"):
|
peak_db_path: str = "/data/tariff.db",
|
||||||
|
price_history_path: str = "/data/price_history.json"):
|
||||||
self.o = opts
|
self.o = opts
|
||||||
self.hass = hass
|
self.hass = hass
|
||||||
self.store = store
|
self.store = store
|
||||||
@@ -247,7 +249,40 @@ class Controller:
|
|||||||
hold_s=float(opts.get("delivery_hold_s",
|
hold_s=float(opts.get("delivery_hold_s",
|
||||||
DELIVERY_OPTION_DEFAULTS["delivery_hold_s"])),
|
DELIVERY_OPTION_DEFAULTS["delivery_hold_s"])),
|
||||||
)
|
)
|
||||||
self.store_migration = migrate_unit_keys(store, UNIT_A)
|
# SAFETY-44. THE NAMES THE MACHINES READ, HANDED TO THE MIGRATION THAT
|
||||||
|
# WRITES FOR THEM. This line used to read `migrate_unit_keys(store,
|
||||||
|
# UNIT_A)`, and the constant was wrong in exactly the way the
|
||||||
|
# `maint_migration` call below says the SAME constant would be wrong -
|
||||||
|
# ⚠️ this comment used to cite "line 315" and "two hundred lines
|
||||||
|
# below", and both were wrong. The correction I wrote for them was
|
||||||
|
# wrong too: it said "~54 lines down", which was review's distance to
|
||||||
|
# the WRONG line, carried to a new referent without re-measuring -
|
||||||
|
# inside the sentence apologising for a wrong number. There is no
|
||||||
|
# distance here now, because a distance is a coordinate and the
|
||||||
|
# sentence above already names the thing. On a declared roster the flat
|
||||||
|
# latch landed under
|
||||||
|
# `unit/inverter-a/`, which is on no roster, was read back under the
|
||||||
|
# name it was handed, and stamped the marker. `roster_names()` is the
|
||||||
|
# naming rule itself and returns `[UNIT_A]` on the flat shape, so the
|
||||||
|
# upgrade path every install runs today is byte-identical.
|
||||||
|
#
|
||||||
|
# ⚠️ AND `misdirected` IS UNREACHABLE FROM **THIS** CALL SITE, WHICH THE
|
||||||
|
# COMMIT MESSAGE ORIGINALLY CLAIMED AS A BENEFIT. `unit` is
|
||||||
|
# `roster_names(opts)[0]` and `build_roster()` names its entries from
|
||||||
|
# the same `roster_names()` list, so `unit` is a member of `roster` by
|
||||||
|
# construction and the refusal can never fire here. Review measured it:
|
||||||
|
# dropping the `roster` argument from the `_verify_moves()` call inside
|
||||||
|
# `migrate_unit_keys()` survives 28/28.
|
||||||
|
#
|
||||||
|
# THE ARGUMENT STAYS ANYWAY, and the reason is the one `_verify_moves()`
|
||||||
|
# already gives for keeping its own `roster=None` branch: it is what a
|
||||||
|
# NEW caller gets, and the grading it enables is measured at the
|
||||||
|
# function's own level in test_maintenance.py rather than through here.
|
||||||
|
# What is NOT true is that this call site buys the refusal, so that
|
||||||
|
# claim is gone rather than restated.
|
||||||
|
_roster_names = roster_names(opts)
|
||||||
|
self.store_migration = migrate_unit_keys(
|
||||||
|
store, _roster_names[0], _roster_names)
|
||||||
self.inverters = build_roster(opts, store, sign_cfg, delivery_cfg)
|
self.inverters = build_roster(opts, store, sign_cfg, delivery_cfg)
|
||||||
# ⚠️ CFG-04. SAY WHICH SHAPE IS IN FORCE, BECAUSE THE OTHER ONE'S KEYS
|
# ⚠️ CFG-04. SAY WHICH SHAPE IS IN FORCE, BECAUSE THE OTHER ONE'S KEYS
|
||||||
# ARE STILL SITTING IN THE UI. A declared `inverters:` roster reads none
|
# ARE STILL SITTING IN THE UI. A declared `inverters:` roster reads none
|
||||||
@@ -549,6 +584,42 @@ class Controller:
|
|||||||
self.peak.restore(self.p1.averager)
|
self.peak.restore(self.p1.averager)
|
||||||
_LOG.info("capacity-tariff peak store: %s", self.peak.restore_state)
|
_LOG.info("capacity-tariff peak store: %s", self.peak.restore_state)
|
||||||
|
|
||||||
|
# FCST-04. The §8.3 carry-forward store: the published prices of the
|
||||||
|
# last few Brussels days, so the hours a day-ahead feed has not printed
|
||||||
|
# yet can be filled from yesterday's profile and FLAGGED, instead of
|
||||||
|
# being absent from the series entirely.
|
||||||
|
#
|
||||||
|
# ⚠️ IT IS AN INSTRUMENT'S BACKING FILE, NOT A STRATEGY. It commands
|
||||||
|
# nothing and claims nothing; what it changes is how many hours the
|
||||||
|
# plan covers. Before this, a dynamic site between midnight and the
|
||||||
|
# 14-16 h CET publication had a series covering only the rest of today
|
||||||
|
# - typically 10-24 of the 48 hours `price_horizon_h` asked for - and
|
||||||
|
# `TariffPlan.arbitrage()` took its spread over that truncated window
|
||||||
|
# without anything reporting that it had. Every carried hour is
|
||||||
|
# provisional, so refusal 3 drops it from the trade set: CARRYING a
|
||||||
|
# price widens what the optimiser can SEE, never what it may act on.
|
||||||
|
#
|
||||||
|
# ⚠️ AND THE CAVEAT THAT BELONGS WITH THAT SENTENCE, because this is
|
||||||
|
# the one place a maintainer reads it on its own. THE OTHER HALF OF
|
||||||
|
# THIS CHANGE DOES MOVE ENERGY, and it is the windowing rather than the
|
||||||
|
# carrying: the filled series covers `[since, until)`, so hours that
|
||||||
|
# have ALREADY ELAPSED are no longer in the spread. A cheap night that
|
||||||
|
# has gone used to sit in the series as the day's cheapest hour, and no
|
||||||
|
# cycle can buy in an hour that is past - so the trade was quietly
|
||||||
|
# never taken. `resolve_tariff()`'s docstring, DOCS.md and the CHANGELOG
|
||||||
|
# all carry this beside the guarantee; it may not appear here without it.
|
||||||
|
#
|
||||||
|
# ⚠️ A BROKEN OR ABSENT STORE IS NOT AN ERROR HERE. It answers `no_data`,
|
||||||
|
# the fill carries nothing, and the plan is the pre-FCST-04 one minus
|
||||||
|
# the elapsed hours - short, but honest and still refusing on the same
|
||||||
|
# rules. It must not be an error in the other sense either: a day
|
||||||
|
# record this build cannot read is DROPPED by `PriceHistory._load()`
|
||||||
|
# and reported in `.words`, because both the constructor above and the
|
||||||
|
# `.words` read below are unguarded, and a cache that raises here is an
|
||||||
|
# add-on that does not start.
|
||||||
|
self.price_history = PriceHistory(price_history_path)
|
||||||
|
_LOG.info("§8.3 price history: %s", self.price_history.words)
|
||||||
|
|
||||||
# SAFETY-07. The guard TARIFF-01 was built for. It is stateless by
|
# SAFETY-07. The guard TARIFF-01 was built for. It is stateless by
|
||||||
# design (peakguard.py holds no history between calls), so the only
|
# design (peakguard.py holds no history between calls), so the only
|
||||||
# state here is what an EDGE needs: what the last cycle concluded, so
|
# state here is what an EDGE needs: what the last cycle concluded, so
|
||||||
@@ -751,23 +822,132 @@ class Controller:
|
|||||||
legacy_tripped = bool(data.get("sign_tripped", False))
|
legacy_tripped = bool(data.get("sign_tripped", False))
|
||||||
if not legacy_tripped or not self.inverters:
|
if not legacy_tripped or not self.inverters:
|
||||||
return
|
return
|
||||||
# ⚠️ THE QUESTION IS "IS ANY NAMESPACED LATCH SET", AND IT IS ASKED OF
|
# ⚠️ THE QUESTION IS "IS ANY NAMESPACED LATCH SET **THAT A MACHINE
|
||||||
# THE STORE RATHER THAN OF THE ROSTER. At a roster of two a trip on B
|
# READS**", AND THE LAST CLAUSE IS SAFETY-44's SECOND HALF. At a roster
|
||||||
# leaves A untripped, so re-arming A off a flat key that B's own trip
|
# of two a trip on B leaves A untripped, so re-arming A off a flat key
|
||||||
# mirrored would latch a healthy inverter on every restart - the
|
# that B's own trip mirrored would latch a healthy inverter on every
|
||||||
# flat-key defect this card removed, coming back through its own guard.
|
# restart - the flat-key defect DEV-02 removed, coming back through its
|
||||||
# And the STORE is the right place to ask: it holds every
|
# own guard. That much stands, and it is why the question is asked of
|
||||||
# `unit/*/sign_tripped` this install has ever written, including one
|
# the STORE's keys rather than of the roster's objects.
|
||||||
# belonging to a unit that is not in TODAY's roster (a roster is
|
#
|
||||||
# configuration and it changes), whereas the roster only knows what is
|
# ⚠️ BUT IT USED TO ACCEPT **ANY** `unit/*/sign_tripped`, and the
|
||||||
# configured right now. The flat key carries new information only when
|
# paragraph here defended that: the store "holds every one this install
|
||||||
# nothing namespaced is latched at all.
|
# has ever written, including one belonging to a unit that is not in
|
||||||
namespaced_latched = any(
|
# TODAY's roster ... whereas the roster only knows what is configured
|
||||||
bool(v) for k, v in data.items()
|
# right now". THE PREMISE IS TRUE AND IT ARGUES THE WRONG WAY. A latch
|
||||||
if k.startswith("unit/") and k.endswith("/sign_tripped"))
|
# under a name no unit answers to is not protection an operator has - it
|
||||||
if namespaced_latched or any(inv.sign_wd.tripped
|
# is a key nothing reads, which is the definition of the state this
|
||||||
for inv in self.inverters):
|
# method exists to repair. Counting it as protection made the guard
|
||||||
|
# return early on precisely the install that needed it, and that is not
|
||||||
|
# hypothetical: it is where the migration above USED to put the latch
|
||||||
|
# on every declared roster, so the guard was blinded by its own upgrade.
|
||||||
|
#
|
||||||
|
# THE MIGRATION FIX DOES NOT REACH THOSE INSTALLS. A store stamped by
|
||||||
|
# the shipped version already has the marker, so `migrate_unit_keys()`
|
||||||
|
# returns at the version check and never runs again. This guard is the
|
||||||
|
# ONLY repair path for a site that upgraded before this card, and it
|
||||||
|
# re-arms onto `inverters[0]` - which lands the latch back under a name
|
||||||
|
# the roster reads.
|
||||||
|
# ⚠️ AND "OFF THE ROSTER" IS NOT ENOUGH ON ITS OWN - IT HAS TO BE THE
|
||||||
|
# MIGRATION'S OWN FINGERPRINT, WHICH IS `UNIT_A`. The first draft of
|
||||||
|
# this treated EVERY off-roster latch as litter, and test_inverter.py's
|
||||||
|
# DEV-02 restart case caught it: there the store carries
|
||||||
|
# `unit/inverter-b/sign_tripped = True` from B's own trip while the
|
||||||
|
# roster being graded holds only A, and the blunt rule re-armed a
|
||||||
|
# HEALTHY A off the flat key that B's trip had mirrored - DEV-02's
|
||||||
|
# defect rebuilt inside its own guard, which is exactly what the
|
||||||
|
# paragraph above was right to be afraid of.
|
||||||
|
#
|
||||||
|
# `UNIT_A` is the discriminator because it is the ONLY name the broken
|
||||||
|
# migration ever wrote. `roster_names()` falls back to a POSITIONAL
|
||||||
|
# `inverter-N`, so `inverter-a` is not a name this code hands to anyone
|
||||||
|
# else. A latch stranded under any OTHER name is a real unit that is
|
||||||
|
# out of today's config - a roster is configuration and it changes -
|
||||||
|
# and it still counts as protection.
|
||||||
|
#
|
||||||
|
# ⚠️ THE RESIDUAL, STATED RATHER THAN LEFT TO BE FOUND: a site whose
|
||||||
|
# own first unit is NAMED `inverter-a`, latched, and then taken out of
|
||||||
|
# the roster gets re-armed on `inverters[0]`. That is a healthy unit
|
||||||
|
# held at 0 W until an operator clears it, which is the conservative
|
||||||
|
# direction; SAFETY-44's is a site DRIVING on a state file that says it
|
||||||
|
# is latched.
|
||||||
|
#
|
||||||
|
# ⚠️ AND THE SECOND HALF OF THAT RESIDUAL IS THE **SENTENCE**, NOT THE
|
||||||
|
# BOOLEAN. The reason re-armed from is the FLAT mirror, which carries
|
||||||
|
# whatever tripped LAST - so on a multi-unit site this path can print
|
||||||
|
# one inverter's evidence under another inverter's name. The latch is
|
||||||
|
# right and the explanation may not be. It is disclosed here because the
|
||||||
|
# banner is what an operator acts on, and `test_maintenance.py`'s check
|
||||||
|
# that the reason is not the removed unit's is guaranteed by that
|
||||||
|
# fixture's choice of flat string, NOT by this code.
|
||||||
|
# ⚠️ AND THE THREE CASES ARE THREE VARIABLES, NOT ONE SHARED FLAG.
|
||||||
|
# THIS IS WHERE THE FIRST VERSION OF THIS FIX WAS DEFEATED, BY ONE KEY.
|
||||||
|
# It wrote `namespaced_latched = True` for BOTH "on the roster" and "off
|
||||||
|
# the roster but not UNIT_A", so a single latched key belonging to a
|
||||||
|
# unit the operator had removed from `inverters:` vetoed the repair of
|
||||||
|
# the misdirected `inverter-a` latch sitting beside it. Measured on the
|
||||||
|
# shipped build, roster ['garage', 'shed'], store stamped so the
|
||||||
|
# migration cannot reach it:
|
||||||
|
#
|
||||||
|
# sign_tripped + unit/inverter-a/sign_tripped
|
||||||
|
# -> [('garage', True), ('shed', False)] repaired
|
||||||
|
# ...AND unit/old-shed/sign_tripped
|
||||||
|
# -> [('garage', False), ('shed', False)] ⚠️ DRIVING
|
||||||
|
#
|
||||||
|
# A site DRIVING on a state file that reads `sign_tripped: true`, which
|
||||||
|
# is the sentence this whole card exists to make false - and
|
||||||
|
# `tools/rehearse_roster.py` said so, on the unmodified tree, exit 1.
|
||||||
|
#
|
||||||
|
# ⚠️ IT IS ALSO INSTANCE #17 OF HABIT 4, IN THE EXTENSION MUTATION.md
|
||||||
|
# ADDED TWO DAYS AGO. The two fixtures written for this card each hold
|
||||||
|
# ONE arrangement - `_staged_store` has only `inverter-a`, `_other_store`
|
||||||
|
# has only `inverter-b` - and the flag is written twice with opposite
|
||||||
|
# intents only when BOTH are present. A per-key check whose keys have
|
||||||
|
# never disagreed WITHIN ONE STORE is a check over whichever key came
|
||||||
|
# last. The fixture below now holds both.
|
||||||
|
#
|
||||||
|
# THE THREE CASES, AND WHY THE ANSWER DIFFERS FOR EACH:
|
||||||
|
# on the roster a machine reads it: real protection, stop.
|
||||||
|
# off-roster, `UNIT_A` the broken migration's fingerprint - this
|
||||||
|
# IS the flat latch, misdirected. Repair.
|
||||||
|
# off-roster, other name a real unit out of today's config. Its
|
||||||
|
# latch is unread too, but it is NOT the flat
|
||||||
|
# key's misdirection, and re-arming a healthy
|
||||||
|
# sibling off it is DEV-02's defect. Stop -
|
||||||
|
# UNLESS a stranded `UNIT_A` latch is also
|
||||||
|
# present, because then the flat key IS
|
||||||
|
# accounted for and the repair is the point.
|
||||||
|
on_roster = {inv.name for inv in self.inverters}
|
||||||
|
stranded, on_roster_latched, other_off_roster = [], False, False
|
||||||
|
for k, v in data.items():
|
||||||
|
if not (k.startswith("unit/") and k.endswith("/sign_tripped")):
|
||||||
|
continue
|
||||||
|
if not bool(v):
|
||||||
|
continue
|
||||||
|
# `unit_key()` builds `unit/<unit>/<key>`, so the unit is field 1 -
|
||||||
|
# split through the writer's own spelling rather than re-deriving.
|
||||||
|
unit = k.split("/")[1]
|
||||||
|
if unit in on_roster:
|
||||||
|
on_roster_latched = True
|
||||||
|
elif unit == UNIT_A:
|
||||||
|
stranded.append(k)
|
||||||
|
else:
|
||||||
|
other_off_roster = True
|
||||||
|
if on_roster_latched or any(inv.sign_wd.tripped
|
||||||
|
for inv in self.inverters):
|
||||||
return
|
return
|
||||||
|
if other_off_roster and not stranded:
|
||||||
|
# A removed unit's own latch, and no misdirected one. The flat key
|
||||||
|
# is that unit's mirror, so it carries no new information and
|
||||||
|
# re-arming `inverters[0]` from it would latch a healthy pack.
|
||||||
|
return
|
||||||
|
if stranded:
|
||||||
|
_LOG.error(
|
||||||
|
"SAFETY-44: this store carries a LATCHED sign-disagreement "
|
||||||
|
"failsafe under %s, and that unit is NOT on today's roster "
|
||||||
|
"%s - so no machine reads it and the site came up DRIVING on "
|
||||||
|
"a state file that says it is latched. Re-arming below.",
|
||||||
|
", ".join(sorted(stranded)), sorted(on_roster))
|
||||||
first = self.inverters[0]
|
first = self.inverters[0]
|
||||||
# The one state this method exists for.
|
# The one state this method exists for.
|
||||||
reason = str(data.get("sign_reason", "") or "")
|
reason = str(data.get("sign_reason", "") or "")
|
||||||
@@ -3313,7 +3493,8 @@ class Controller:
|
|||||||
now = datetime.now(timezone.utc)
|
now = datetime.now(timezone.utc)
|
||||||
try:
|
try:
|
||||||
plan = await resolve_tariff(
|
plan = await resolve_tariff(
|
||||||
self.o, now, now + timedelta(hours=horizon), now=now)
|
self.o, now, now + timedelta(hours=horizon), now=now,
|
||||||
|
history=self.price_history)
|
||||||
self.tariff_plan, self.tariff_at = plan, now
|
self.tariff_plan, self.tariff_at = plan, now
|
||||||
_LOG.info("tariff refreshed: %s", plan.words)
|
_LOG.info("tariff refreshed: %s", plan.words)
|
||||||
except Exception as err: # noqa: BLE001
|
except Exception as err: # noqa: BLE001
|
||||||
@@ -4162,6 +4343,13 @@ class Controller:
|
|||||||
("Maintenance due", "now" if next_due is None else next_due.strftime("%Y-%m-%d")),
|
("Maintenance due", "now" if next_due is None else next_due.strftime("%Y-%m-%d")),
|
||||||
("Cheap window", "yes" if self.cheap else "no"),
|
("Cheap window", "yes" if self.cheap else "no"),
|
||||||
("Tariff", self.tariff_words),
|
("Tariff", self.tariff_words),
|
||||||
|
# FCST-04. WHAT §8.3 HAS TO CARRY FORWARD, which is the difference
|
||||||
|
# between a 48-hour plan and a 12-hour one on a weekday morning.
|
||||||
|
# It is its own row rather than a clause of "Tariff" because it
|
||||||
|
# answers a different question: "Tariff" says what the plan IS, and
|
||||||
|
# this says what the plan could still be filled from if the feed is
|
||||||
|
# late again tomorrow.
|
||||||
|
("Price history", self.price_history.words),
|
||||||
# OPT-04. The strategy's own sentence, not a re-derivation of it -
|
# OPT-04. The strategy's own sentence, not a re-derivation of it -
|
||||||
# every refusal it can make already carries a reason a human can
|
# every refusal it can make already carries a reason a human can
|
||||||
# act on, and "Arbitrage: no" would throw all of them away.
|
# act on, and "Arbitrage: no" would throw all of them away.
|
||||||
|
|||||||
@@ -48,12 +48,24 @@ So the flag is not passed through anywhere in this file:
|
|||||||
the object the provider produced is the object in the output series - so
|
the object the provider produced is the object in the output series - so
|
||||||
there is no copy that could lose or gain a flag. `filled.points[i] is
|
there is no copy that could lose or gain a flag. `filled.points[i] is
|
||||||
fetched.points[j]` is a checkable property, and test_price_forecast.py
|
fetched.points[j]` is a checkable property, and test_price_forecast.py
|
||||||
checks it.
|
checks it. EVERY published point in the window is passed through, not one
|
||||||
|
per hour: see the sub-hourly note below.
|
||||||
* `_audit()` RE-DERIVES the expected flag for every output point from the set
|
* `_audit()` RE-DERIVES the expected flag for every output point from the set
|
||||||
of hours the feed actually published, and RAISES if any point disagrees.
|
of hours the feed actually published, and RAISES if any point disagrees.
|
||||||
It is a post-condition, not a filter: it never fixes a flag, because a
|
It is a post-condition, not a filter: it never fixes a flag, because a
|
||||||
module that quietly corrects its own provenance is one whose provenance
|
module that quietly corrects its own provenance is one whose provenance
|
||||||
cannot be trusted.
|
cannot be trusted.
|
||||||
|
|
||||||
|
⚠️ IT GRADES THE POINTS THAT ARE EMITTED AND NOTHING ABOUT THE ONES THAT
|
||||||
|
ARE NOT (FCST-05). `_audit()` is a provenance check, NOT a completeness
|
||||||
|
check: fed FCST-04's own decimation defect - four published points in an
|
||||||
|
hour, only the last emitted - it ACCEPTS, and it accepts an empty output
|
||||||
|
too. Measured, and the FCST-03 audit it replaced was equally blind, so
|
||||||
|
this is a limit rather than a regression. What actually guards against a
|
||||||
|
decimated series is the point COUNT and the pass-through IDENTITY checks
|
||||||
|
in `test_price_forecast.py` (section on the quarter-hourly feed) and
|
||||||
|
`test_price_history.py`; see `_audit()`'s own docstring for why the
|
||||||
|
completeness rule is not expressible from its two arguments.
|
||||||
* and test_price_forecast.py walks THIS FILE'S AST and fails if any
|
* and test_price_forecast.py walks THIS FILE'S AST and fails if any
|
||||||
`PricePoint(...)` in it passes anything but a literal for `provisional`, so
|
`PricePoint(...)` in it passes anything but a literal for `provisional`, so
|
||||||
the pass-through idiom cannot come back in a later edit and pass review by
|
the pass-through idiom cannot come back in a later edit and pass review by
|
||||||
@@ -90,6 +102,24 @@ which is the row the spec actually writes for "no price data". If NOTHING can
|
|||||||
be produced, the result is `PriceRead.no_data(reason)` and not an empty series,
|
be produced, the result is `PriceRead.no_data(reason)` and not an empty series,
|
||||||
because an empty series is not constructible on purpose (price.py trap 3).
|
because an empty series is not constructible on purpose (price.py trap 3).
|
||||||
|
|
||||||
|
⚠️ A SUB-HOURLY FEED IS NOT DECIMATED TO ONE POINT AN HOUR, and the hourly walk
|
||||||
|
below is about WHICH HOURS NEED FILLING - never about what a published hour is
|
||||||
|
allowed to contain. `eneco_eddy` is the Belgian imbalance product and settles
|
||||||
|
per QUARTER, `_validate_raw()` accepts any strictly-increasing spacing, and a
|
||||||
|
published hour may therefore hold four points. So EVERY published point inside
|
||||||
|
the window is passed through, in order, and a carried point is added only for a
|
||||||
|
whole hour with NO published coverage at all.
|
||||||
|
|
||||||
|
The alternative was measured rather than reasoned about, because FCST-04's
|
||||||
|
first caller shipped it: keying the published set as `{_utc_hour(p.start): p
|
||||||
|
for p in r.points}` reads perfectly ordinary and keeps the LAST point of each
|
||||||
|
hour, dropping the other three. On a 96-point quarter-hourly feed - all firm,
|
||||||
|
fully published, nothing to carry - the series came out 15 points long, the
|
||||||
|
arbitrage spread collapsed from 0.143 to 0.014 EUR/kWh, and the cheapest
|
||||||
|
"hour" became a `:45` point whose `at()` window runs into the next hour. A feed
|
||||||
|
losing 84 % of its own prints is not a windowing effect and is not something a
|
||||||
|
provisional flag would ever have reported.
|
||||||
|
|
||||||
|
|
||||||
AC2: THE IRREVERSIBLE-ACTION GATE
|
AC2: THE IRREVERSIBLE-ACTION GATE
|
||||||
=================================
|
=================================
|
||||||
@@ -156,6 +186,25 @@ def _utc_hour(instant: datetime) -> datetime:
|
|||||||
return u.replace(minute=0, second=0, microsecond=0)
|
return u.replace(minute=0, second=0, microsecond=0)
|
||||||
|
|
||||||
|
|
||||||
|
def _published_by_hour(read: PriceRead) -> dict:
|
||||||
|
"""UTC hour -> the provider's OWN points that start inside it, in order.
|
||||||
|
|
||||||
|
⚠️ A TUPLE PER HOUR, NOT A POINT, AND THAT IS THE WHOLE OF THE SUB-HOURLY
|
||||||
|
RULE. `{_utc_hour(p.start): p for p in r.points}` reads identically, is one
|
||||||
|
character shorter, and silently keeps one point per hour - the last one
|
||||||
|
written. The mapping answers two DIFFERENT questions here: "does this hour
|
||||||
|
need filling at all?" is `u in published`, and "what does the output carry
|
||||||
|
for it?" is all of `published[u]`. Collapsing them is how a quarter-hourly
|
||||||
|
feed loses three quarters of its prints.
|
||||||
|
"""
|
||||||
|
out: dict = {}
|
||||||
|
read.match(
|
||||||
|
series=lambda r: [out.setdefault(_utc_hour(p.start), []).append(p)
|
||||||
|
for p in r.points],
|
||||||
|
no_data=lambda _r: [])
|
||||||
|
return {u: tuple(ps) for u, ps in out.items()}
|
||||||
|
|
||||||
|
|
||||||
def _carried(start_utc: datetime, eur_per_kwh: float) -> PricePoint:
|
def _carried(start_utc: datetime, eur_per_kwh: float) -> PricePoint:
|
||||||
"""A point filled from yesterday's profile. §8.3: ALWAYS provisional.
|
"""A point filled from yesterday's profile. §8.3: ALWAYS provisional.
|
||||||
|
|
||||||
@@ -172,37 +221,122 @@ def _carried(start_utc: datetime, eur_per_kwh: float) -> PricePoint:
|
|||||||
def _audit(points, published) -> None:
|
def _audit(points, published) -> None:
|
||||||
"""Post-condition: every flag matches the point's TRUE provenance (AC1).
|
"""Post-condition: every flag matches the point's TRUE provenance (AC1).
|
||||||
|
|
||||||
`published` maps a UTC hour to the point the provider produced for it.
|
`published` maps a UTC hour to the TUPLE of points the provider produced
|
||||||
|
for it (`_published_by_hour()`). An hour ABSENT from that mapping is an hour
|
||||||
|
the feed did not print, and a carried point is the only thing that may
|
||||||
|
stand in it.
|
||||||
|
|
||||||
|
The rule, in the two forms an output point can take:
|
||||||
|
|
||||||
|
* PUBLISHED - the point must BE one of `published[u]`, by identity, and
|
||||||
|
may be emitted only once. Its flag is whatever the feed said, never
|
||||||
|
forced to False: an imbalance print is published AND provisional (§9.2,
|
||||||
|
TARIFF-02), and a check that "corrected" it would be a bug wearing a
|
||||||
|
guarantee.
|
||||||
|
* CARRIED - the hour must not appear in `published` AT ALL, the flag must
|
||||||
|
be exactly True, and that hour may hold no second point.
|
||||||
|
|
||||||
|
⚠️ WHAT REPLACED "AT MOST ONE POINT PER HOUR", AND WHAT THAT DID AND DID
|
||||||
|
NOT CHANGE. This audit began as FCST-03's `seen` set: any repeated
|
||||||
|
`_utc_hour` raised, beside an identity rule `p is not src`. A bare hour
|
||||||
|
count cannot express a quarter-hourly feed, where four distinct published
|
||||||
|
points share one hour legitimately - so the count was replaced by the two
|
||||||
|
rules above, which subsume it.
|
||||||
|
|
||||||
|
⚠️ AND IT IS NOT STRICTER. FCST-04's version of this paragraph claimed two
|
||||||
|
strengthenings and BOTH WERE FALSE; FCST-05 measured it. A differential
|
||||||
|
over the whole domain this audit can express - every published mask over 3
|
||||||
|
hours crossed with every output series of length 0-3 drawn from
|
||||||
|
{published, published-provisional, carried, carried-unflagged,
|
||||||
|
copy-of-published} - found ZERO divergence in EITHER direction:
|
||||||
|
|
||||||
|
OLD raised, NEW accepted (WEAKER): 0
|
||||||
|
OLD accepted, NEW raised (STRONGER): 0
|
||||||
|
|
||||||
|
The two specific claims, one at a time:
|
||||||
|
|
||||||
|
* "so does the same published point emitted twice" - the OLD `seen`
|
||||||
|
count already raised on that, because emitting one point twice is one
|
||||||
|
HOUR twice. Preserved by the identity/`emitted` rules, not added.
|
||||||
|
* "a carried point may no longer share an hour with a published one,
|
||||||
|
which the count never checked at all" - true of the COUNT, and the old
|
||||||
|
IDENTITY rule raised on it anyway: a carried point in a published hour
|
||||||
|
took the `else` branch and failed `p is not src`. The message was about
|
||||||
|
a copy rather than about a carry, which is the only thing that moved.
|
||||||
|
|
||||||
|
The one real behavioural change is the intended RELAXATION - several
|
||||||
|
distinct published points in one hour, which the old audit could not
|
||||||
|
express at all and refused. That relaxation is why this was rewritten, and
|
||||||
|
it stays. A right decision with a fabricated reason is still a defect here.
|
||||||
|
|
||||||
|
⚠️ AND THE REAL LIMIT, WHICH IS NOT A REGRESSION AND IS NOT SMALL: THIS
|
||||||
|
AUDIT DOES NOT CHECK COMPLETENESS. It grades every point that IS emitted
|
||||||
|
and says nothing about the ones that are not. Fed the H1 decimation
|
||||||
|
directly - four published points in an hour, output emitting only the last
|
||||||
|
- it ACCEPTS; so does two of four, and so does an EMPTY output. So `_audit`
|
||||||
|
is NOT what stops H1 recurring. What stops it is the point COUNT and the
|
||||||
|
IDENTITY checks in `test_price_history.py` and `test_price_forecast.py`,
|
||||||
|
which compare the emitted series against the feed's own prints.
|
||||||
|
|
||||||
|
It is not made complete here, and the reason is structural rather than
|
||||||
|
effort: `published` is `_published_by_hour(read)` over the WHOLE read,
|
||||||
|
unwindowed, while `fill_from_yesterday()` emits only [since, until). A
|
||||||
|
published point outside the window is legitimately absent, so "every
|
||||||
|
published point appears" is not a true post-condition of these two
|
||||||
|
arguments - it would need the window passed in, which is a different
|
||||||
|
signature and a different check.
|
||||||
|
|
||||||
|
⚠️ AND THAT IS MEASURED, NOT ASSUMED. FCST-05 built the rule - raise unless
|
||||||
|
every point of every `published[u]` is emitted, +162 B - and ran it. It
|
||||||
|
takes the 1 h, 12 h and 48 h HORIZON checks in test_price_forecast.py RED,
|
||||||
|
on the shipped fill and a healthy feed: at an off-the-hour instant the
|
||||||
|
feed's already-elapsed prints are outside [since, until) and are correctly
|
||||||
|
dropped, and the rule calls that a provenance failure. (The differential
|
||||||
|
beside those checks scores the same rule as 49 STRONGER cases, so it is a
|
||||||
|
real change of behaviour and not a no-op.) The rule as stated is therefore
|
||||||
|
not merely unimplemented, it is WRONG - and stating a false post-condition
|
||||||
|
is MUTATION.md habit 16's defect: an instrument that asserts a completeness
|
||||||
|
it does not have is worse than no instrument.
|
||||||
|
|
||||||
⚠️ RAISES, NEVER REPAIRS. A module that silently corrects its own
|
⚠️ RAISES, NEVER REPAIRS. A module that silently corrects its own
|
||||||
provenance is one whose provenance means nothing; the only safe response to
|
provenance is one whose provenance means nothing; the only safe response to
|
||||||
"this point's flag disagrees with where it came from" is to refuse to emit
|
"this point's flag disagrees with where it came from" is to refuse to emit
|
||||||
the series at all.
|
the series at all.
|
||||||
"""
|
"""
|
||||||
seen = set()
|
carried_hours = set()
|
||||||
|
emitted = set()
|
||||||
for p in points:
|
for p in points:
|
||||||
u = _utc_hour(p.start)
|
u = _utc_hour(p.start)
|
||||||
if u in seen:
|
|
||||||
raise ValueError(f"the filled series repeats {u.isoformat()}; a "
|
|
||||||
f"duplicated hour is how a DST day gets mispriced")
|
|
||||||
seen.add(u)
|
|
||||||
src = published.get(u)
|
src = published.get(u)
|
||||||
prov = Provenance.CARRIED if src is None else Provenance.PUBLISHED
|
|
||||||
if src is None:
|
if src is None:
|
||||||
|
if u in carried_hours:
|
||||||
|
raise ValueError(
|
||||||
|
f"the filled series repeats the carried hour "
|
||||||
|
f"{u.isoformat()}; a duplicated hour is how a DST day gets "
|
||||||
|
f"mispriced")
|
||||||
|
carried_hours.add(u)
|
||||||
if p.provisional is not True:
|
if p.provisional is not True:
|
||||||
raise ValueError(
|
raise ValueError(
|
||||||
f"{u.isoformat()} is {prov.value} and is flagged "
|
f"{u.isoformat()} is {Provenance.CARRIED.value} and is "
|
||||||
f"provisional={p.provisional!r}. §8.3: an hour filled from "
|
f"flagged provisional={p.provisional!r}. §8.3: an hour "
|
||||||
f"yesterday's profile is provisional whether or not the "
|
f"filled from yesterday's profile is provisional whether "
|
||||||
f"number turns out to be right - the flag is a claim about "
|
f"or not the number turns out to be right - the flag is a "
|
||||||
f"PROVENANCE, not about accuracy.")
|
f"claim about PROVENANCE, not about accuracy.")
|
||||||
else:
|
else:
|
||||||
if p is not src:
|
if not any(p is q for q in src):
|
||||||
raise ValueError(
|
raise ValueError(
|
||||||
f"{u.isoformat()} is {prov.value} but the series carries a "
|
f"{u.isoformat()} is {Provenance.PUBLISHED.value} but the "
|
||||||
f"COPY of that point rather than the point itself. A copy "
|
f"series carries a COPY of that point rather than the "
|
||||||
f"is where the provisional flag gets lost; published points "
|
f"point itself. A copy is where the provisional flag gets "
|
||||||
f"pass through by identity.")
|
f"lost; published points pass through by identity.")
|
||||||
|
if id(p) in emitted:
|
||||||
|
raise ValueError(
|
||||||
|
f"the filled series emits the published point at "
|
||||||
|
f"{p.start.isoformat()} TWICE. The provider's own points "
|
||||||
|
f"are strictly increasing (`_validate_raw`), so a repeat "
|
||||||
|
f"is never a second print - it is a duplicated hour, which "
|
||||||
|
f"is how a DST day gets mispriced")
|
||||||
|
emitted.add(id(p))
|
||||||
|
|
||||||
|
|
||||||
def fill_from_yesterday(read: PriceRead, *, yesterday: PriceRead,
|
def fill_from_yesterday(read: PriceRead, *, yesterday: PriceRead,
|
||||||
@@ -217,11 +351,21 @@ def fill_from_yesterday(read: PriceRead, *, yesterday: PriceRead,
|
|||||||
|
|
||||||
Returns a `PriceRead`:
|
Returns a `PriceRead`:
|
||||||
|
|
||||||
* a series, whose points are the provider's own objects for the hours it
|
* a series, carrying the provider's own objects - ALL of them, which on
|
||||||
published and `_carried()` points for the rest, ordered by instant;
|
a quarter-hourly feed is four inside one hour - for the hours it
|
||||||
|
published, and one `_carried()` point for each remaining whole hour,
|
||||||
|
ordered by instant;
|
||||||
* `no_data` when neither source can produce a single hour - never an
|
* `no_data` when neither source can produce a single hour - never an
|
||||||
empty series, which is not constructible (price.py trap 3).
|
empty series, which is not constructible (price.py trap 3).
|
||||||
|
|
||||||
|
⚠️ THE WINDOW IS WHOLE HOURS AT BOTH ENDS. `since` is floored to the hour
|
||||||
|
that CONTAINS it, because the current hour is still a tradeable hour; and
|
||||||
|
so is `until`, which is the half FCST-04's first draft left out. With
|
||||||
|
`since` floored and `until` compared raw, a 48 h horizon asked for at 08:17
|
||||||
|
emitted 49 hours - the hour of `until` was still `< until` - and the count
|
||||||
|
was one too many for every `now` that is not exactly on the hour, which is
|
||||||
|
every `now` a running controller has.
|
||||||
|
|
||||||
⚠️ IDEMPOTENT AND RE-RUNNABLE, WHICH IS THE §5.4 LATE-PUBLICATION CONTRACT.
|
⚠️ IDEMPOTENT AND RE-RUNNABLE, WHICH IS THE §5.4 LATE-PUBLICATION CONTRACT.
|
||||||
Day-ahead publication at 14-16 h CET can be late or fail, so the fetcher
|
Day-ahead publication at 14-16 h CET can be late or fail, so the fetcher
|
||||||
retries and this runs again against the fresher read. Published hours always
|
retries and this runs again against the fresher read. Published hours always
|
||||||
@@ -234,9 +378,7 @@ def fill_from_yesterday(read: PriceRead, *, yesterday: PriceRead,
|
|||||||
raise ValueError(f"{name} must be tz-aware; a naive instant is "
|
raise ValueError(f"{name} must be tz-aware; a naive instant is "
|
||||||
f"ambiguous for one hour every October")
|
f"ambiguous for one hour every October")
|
||||||
|
|
||||||
published = read.match(
|
published = _published_by_hour(read)
|
||||||
series=lambda r: {_utc_hour(p.start): p for p in r.points},
|
|
||||||
no_data=lambda _r: {})
|
|
||||||
|
|
||||||
# Yesterday's profile, keyed by BRUSSELS WALL-CLOCK HOUR. First point wins
|
# Yesterday's profile, keyed by BRUSSELS WALL-CLOCK HOUR. First point wins
|
||||||
# when yesterday itself was a 25-hour day and printed local hour 2 twice.
|
# when yesterday itself was a 25-hour day and printed local hour 2 twice.
|
||||||
@@ -248,11 +390,12 @@ def fill_from_yesterday(read: PriceRead, *, yesterday: PriceRead,
|
|||||||
|
|
||||||
out = []
|
out = []
|
||||||
u = _utc_hour(since)
|
u = _utc_hour(since)
|
||||||
end = until.astimezone(timezone.utc)
|
end = _utc_hour(until)
|
||||||
while u < end:
|
while u < end:
|
||||||
if u in published:
|
got = published.get(u)
|
||||||
out.append(published[u]) # BY IDENTITY. No copy.
|
if got:
|
||||||
else:
|
out.extend(got) # BY IDENTITY, and ALL of them. No copy,
|
||||||
|
else: # and no decimation to one an hour.
|
||||||
src = by_local_hour.get(brussels(u).hour)
|
src = by_local_hour.get(brussels(u).hour)
|
||||||
if src is not None:
|
if src is not None:
|
||||||
out.append(_carried(u, src.eur_per_kwh))
|
out.append(_carried(u, src.eur_per_kwh))
|
||||||
|
|||||||
@@ -0,0 +1,864 @@
|
|||||||
|
"""FCST-04 - the day the carry-forward carries, kept somewhere a restart cannot reach.
|
||||||
|
|
||||||
|
WHAT WAS MISSING, AND WHY IT WAS INVISIBLE. FCST-03 shipped
|
||||||
|
`price_forecast.fill_from_yesterday()` complete and checked, and shipped the
|
||||||
|
consumers that were written to expect it - `TariffPlan.firm_points()`,
|
||||||
|
`cheap_now()`'s refusal of a provisional hour, and `arbitrage()`'s refusal 3,
|
||||||
|
which drops provisional hours from the trade set instead of refusing the whole
|
||||||
|
series. Every one of those was live. The FILL ITSELF HAD NO CALLER: nothing in
|
||||||
|
`app/` referenced `fill_from_yesterday`, so no series ever contained a carried
|
||||||
|
point and every one of those consumers was exercising its all-firm branch.
|
||||||
|
|
||||||
|
The defect that produced is not an error anywhere. `load_prices()` returns
|
||||||
|
whatever the feed published and nothing widens it to the requested horizon, so
|
||||||
|
on a dynamic site between midnight and the 14-16 h CET day-ahead publication
|
||||||
|
the series covers the rest of TODAY and nothing more - typically 10-24 of the
|
||||||
|
48 hours `price_horizon_h` asked for. `TariffPlan.arbitrage()` then computes
|
||||||
|
`max - min` over that truncated window and reports a spread. The number is
|
||||||
|
real, the window is half the one the operator configured, and NOTHING SAYS SO:
|
||||||
|
tomorrow evening's peak - the hour a morning charge would be sold into - is
|
||||||
|
simply not in the series, so the spread it would have created cannot clear a
|
||||||
|
threshold it was never compared against. A missing hour is not a wrong price;
|
||||||
|
it is an hour the optimiser never learns exists.
|
||||||
|
|
||||||
|
So this module is the half FCST-03 left out, and it is deliberately the SMALL
|
||||||
|
half: the fill's arithmetic, its DST rule and its provenance audit all stay in
|
||||||
|
`price_forecast.py`, untouched. What lives here is the one thing that
|
||||||
|
arithmetic needs and could not have: A YESTERDAY THAT SURVIVES A RESTART.
|
||||||
|
|
||||||
|
|
||||||
|
WHY PERSISTENCE, AND NOT A VARIABLE ON THE REFRESH TASK
|
||||||
|
======================================================
|
||||||
|
An in-memory "the last full day I saw" costs nothing and is wrong exactly when
|
||||||
|
it matters. The window the fill exists for is midnight -> publication; an
|
||||||
|
add-on restarted at 08:00 - a config change, an HA update, a power cut - would
|
||||||
|
come up with no yesterday at all and spend that whole window on the truncated
|
||||||
|
series, which is the pre-FCST-03 behaviour arriving silently on the mornings
|
||||||
|
after an operator touched something. The fill's own value is a function of
|
||||||
|
surviving a restart, so the day it carries has to.
|
||||||
|
|
||||||
|
⚠️ AND NOT IN `/data/state.json`. That file holds the maintenance history and
|
||||||
|
SAFETY-02's sign latch, and `Store.set()` rewrites the whole of it on every
|
||||||
|
write. The tariff refresh runs every `TARIFF_REFRESH_S` (1800 s), so folding a
|
||||||
|
price cache into it would put 48 extra whole-file rewrites a day through the
|
||||||
|
file whose corruption reads as "no maintenance ever ran" and "never tripped".
|
||||||
|
The cache is worth having; it is not worth putting anywhere near that. Separate
|
||||||
|
file, separate blast radius.
|
||||||
|
|
||||||
|
⚠️ AND THE DIRECTORY IS NOT CREATED, which is `tariff.PeakStore`'s rule and its
|
||||||
|
reason applies unchanged: `/data` is mounted by the Supervisor and always
|
||||||
|
exists on a real install, so `makedirs` here could only ever succeed where the
|
||||||
|
durable volume is MISSING - manufacturing a cache on the container's ephemeral
|
||||||
|
layer that writes fine, reads fine, and is thrown away on the next restart.
|
||||||
|
That is a yesterday that silently forgets, which is worse than one that says it
|
||||||
|
cannot answer.
|
||||||
|
|
||||||
|
⚠️ AND THAT RULE IS CARRIED OUT IN `_remember()`, NOT ONLY IN `_save()`. FCST-04
|
||||||
|
first shipped it half-done: `_save()` short-circuited on `not self.ok` while
|
||||||
|
`_remember()` filled `self._days` anyway, so a store with no durable volume
|
||||||
|
answered `words` = "unavailable" and `yesterday()` = A SERIES - an in-memory
|
||||||
|
yesterday that works all day and is gone on the next restart, which is exactly
|
||||||
|
the silently-forgetting cache this rule exists to refuse. A store that cannot
|
||||||
|
persist now records nothing, and both of its answers say the same thing.
|
||||||
|
|
||||||
|
⚠️ AND "CANNOT PERSIST" IS ONE QUESTION, ASKED ONCE, ABOUT THE OUTCOME -
|
||||||
|
FCST-06. The paragraph above enforced that rule against `not self.ok`, which is
|
||||||
|
one SYMPTOM of not persisting: the durable volume is missing. There is a second,
|
||||||
|
and FCST-06 measured it. With `ok=True` and the directory present, `_save()` can
|
||||||
|
still fail - a read-only remount (`[Errno 30] Read-only file system`), a full
|
||||||
|
volume (`[Errno 28] No space left on device`) - and it logs, records
|
||||||
|
`last_error` and returns. `_remember()` had already populated `self._days`, so
|
||||||
|
`words` read `'2026-01-14 (24 points over 24 h); 2026-01-15 (24 points over
|
||||||
|
24 h) [could not write (...)]'` while the file on disk held `{}`, and
|
||||||
|
`yesterday()` answered 24 REAL POINTS for the rest of the process's life. Gone
|
||||||
|
on the next restart, and nothing said it would be - the same silently-forgetting
|
||||||
|
cache, reached by the condition the rule did not name.
|
||||||
|
|
||||||
|
⚠️ SO IT IS NOT TWO RULES, AND IT IS DELIBERATELY NOT TWO TESTS. "Can this
|
||||||
|
store persist?" CANNOT BE ANSWERED BEFORE THE ATTEMPT - a directory that existed
|
||||||
|
and was writable a moment ago is not a write that completed - so it is answered
|
||||||
|
AFTER it, once, by the only thing holding the evidence: `_save()` returns
|
||||||
|
whether the file now holds what memory holds. A missing volume is one of its
|
||||||
|
False cases and an `OSError` from the write is another, and `_remember()` KEEPS
|
||||||
|
ONLY WHAT THAT ONE ANSWER SAYS WAS KEPT. There is no `not self.ok` test in
|
||||||
|
`_remember()` any more, because there is nothing left for a second test to add -
|
||||||
|
and a third symptom nobody has thought of is how this defect arrived both times.
|
||||||
|
|
||||||
|
⚠️ AND WHAT IS KEPT IS THE LAST STATE THAT IS ON DISK, NOT AN EMPTY ONE. The
|
||||||
|
MERGE is undone, not the store. A site whose volume fills at 14:00 goes on
|
||||||
|
answering the yesterday it wrote at 13:30 - that record survives a restart, so
|
||||||
|
answering it is not a promise the store cannot keep. What it stops doing is
|
||||||
|
adding to it. The invariant the rule was always describing, stated once:
|
||||||
|
`self._days` NEVER HOLDS A POINT THE FILE DOES NOT. (Not "mirrors the file" -
|
||||||
|
`_usable_days()` drops records the file still holds. The only direction that can
|
||||||
|
invent a yesterday is the one named.)
|
||||||
|
|
||||||
|
|
||||||
|
THE FOUR RULES THIS FILE IS SHAPED BY
|
||||||
|
=====================================
|
||||||
|
|
||||||
|
1. IT RECORDS WHAT THE PROVIDER PUBLISHED, NEVER WHAT THE FILL PRODUCED.
|
||||||
|
Feeding a filled series back in is the defect that makes the whole mechanism
|
||||||
|
rot: day 2 carries day 1's guesses, day 3 carries day 2's carry of them, and
|
||||||
|
after a week of a dead feed the "yesterday" being carried forward is a
|
||||||
|
fossil of the last real publication with nothing anywhere reporting its age.
|
||||||
|
Every point still carries `provisional=True`, so the flag never lies - what
|
||||||
|
decays is the NUMBER, silently, behind a flag that is telling the truth.
|
||||||
|
|
||||||
|
⚠️ NO VALUE-LEVEL CHECK CAN CATCH THAT, and saying so is more useful than a
|
||||||
|
guarantee that would not hold. A carried point and a published-provisional
|
||||||
|
point (`eneco_eddy`'s current quarter, §9.2) are the same three fields with
|
||||||
|
the same flag; the difference is provenance, and provenance is not in the
|
||||||
|
value. So the mechanism is ORDERING, not validation: `carry()` is the only
|
||||||
|
public entry point, it remembers the argument and returns the fill, and the
|
||||||
|
filled series exists only as a return value that `carry()` itself never
|
||||||
|
sees again. `_remember()` is private, is called from exactly one place, and
|
||||||
|
`test_price_history.py` PARSES every shipped module under `app/`, `tools/`,
|
||||||
|
`scenarios/` and `estop/` and fails if any of them uses the NAME - an
|
||||||
|
attribute, a bare name, a def, or the string constant a `getattr` needs. By
|
||||||
|
name rather than by substring, so a comment about the rule does not fail it
|
||||||
|
and a call from `tools/` cannot pass unseen.
|
||||||
|
|
||||||
|
⚠️ AND THE ORDER IS GRADED BY DRIVING IT, NOT BY READING THE SOURCE. The
|
||||||
|
first version of that check compared positions in `ast.walk`, which is
|
||||||
|
breadth-first: `self._remember(...)` is a statement and `self.yesterday(...)`
|
||||||
|
is an ARGUMENT of the fill, one level deeper, so `_remember` came out first
|
||||||
|
whichever statement was written first - shipped and reversed scored the
|
||||||
|
same. What grades it now is the behaviour: a feed carrying yesterday
|
||||||
|
alongside today, into an EMPTY store, must fill the whole horizon, and
|
||||||
|
reversed it fills nothing.
|
||||||
|
|
||||||
|
2. "YESTERDAY" MEANS YESTERDAY, NOT "THE NEWEST DAY I HAVE".
|
||||||
|
`newest()` is the natural accessor and it is the one that decays: an add-on
|
||||||
|
down from Friday to Monday would answer Friday's prices to a fill that flags
|
||||||
|
them as one day old. The lookup is BY DATE - the Brussels day before the
|
||||||
|
Brussels day of the instant asked about - and a record for any other date is
|
||||||
|
NO_RECORD with a reason naming both dates. A three-day-old profile is not a
|
||||||
|
worse yesterday; it is not a yesterday.
|
||||||
|
|
||||||
|
3. A STORE THAT CANNOT ANSWER SAYS SO, IN THE TYPE.
|
||||||
|
`yesterday()` returns a `PriceRead`: a `series` when a full record exists,
|
||||||
|
`no_data(reason)` for every other outcome - no file, no record for that
|
||||||
|
date, an unreadable file, a record that parses to nothing. There is no
|
||||||
|
empty-series answer and no `.get(..., [])`-shaped accessor, for
|
||||||
|
`price.py`'s reason: an empty series is how "the cache is broken" gets to
|
||||||
|
wear the shape of "yesterday was free". `fill_from_yesterday()` takes a
|
||||||
|
`PriceRead` and handles `no_data` on its own, so the degraded case needs no
|
||||||
|
branch at the call site - it fills nothing and the horizon stays short,
|
||||||
|
which is exactly the behaviour that shipped before this ticket.
|
||||||
|
|
||||||
|
4. HOURS ACCUMULATE WITHIN A DAY; THEY DO NOT REPLACE IT.
|
||||||
|
A feed that serves only the REMAINING hours of the day (a legitimate shape -
|
||||||
|
nothing in the plugin contract promises a whole day) would, under
|
||||||
|
replace-the-record, overwrite a complete 24-hour profile with the two points
|
||||||
|
left at 22:00 and hand the next morning a yesterday with two hours in it. So
|
||||||
|
a record is MERGED by instant: a later read updates the hours it carries and
|
||||||
|
leaves the rest, and a revised price for an hour already held wins because
|
||||||
|
it is the newer statement about the same hour.
|
||||||
|
|
||||||
|
⚠️ KEYED BY INSTANT, NOT BY LOCAL HOUR, AND THAT IS THE DST DECISION. Storing
|
||||||
|
`hour -> price` would collapse the two 02:00s of a 25-hour October day into
|
||||||
|
one, inventing a second and different collapsing rule beside the one
|
||||||
|
`fill_from_yesterday()` already documents ("the first of its two 02:00 points
|
||||||
|
is the one used, deterministically"). Storing the instants keeps both, in
|
||||||
|
order, and lets the fill's own rule be the only one in the system.
|
||||||
|
|
||||||
|
WHAT THIS DOES NOT DO, ON PURPOSE
|
||||||
|
=================================
|
||||||
|
It does not extend the fill to a feed that published NOTHING. `resolve_tariff()`
|
||||||
|
applies `carry()` only on a `series` read; a `no_data` read still falls to
|
||||||
|
`_degrade()` and §11.4's row - "No price data -> fall back to the static
|
||||||
|
profile, mark the plan provisional, no arbitrage" - which is the fallback the
|
||||||
|
operator configured `static_profile` FOR. Preferring yesterday's feed over it
|
||||||
|
would change a documented degradation without being asked, and it would buy
|
||||||
|
nothing measurable: a fully-carried series is 100 % provisional, so
|
||||||
|
`arbitrage()` finds fewer than two firm hours and refuses, and `cheap_now()` is
|
||||||
|
False, exactly as on the §11.4 fallback. The two differ only in which numbers
|
||||||
|
the status page shows. `fill_from_yesterday()` handles a `no_data` read anyway,
|
||||||
|
and that branch is therefore not reached from production - it stays checked,
|
||||||
|
and turning it on is a product decision, not a wiring one. See
|
||||||
|
DECISIONS/FCST-04-the-day-that-is-carried.md.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import copy
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import tempfile
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
|
||||||
|
from .price import PricePoint, PriceProduct, PriceRead, brussels
|
||||||
|
from .price_forecast import fill_from_yesterday
|
||||||
|
|
||||||
|
_LOG = logging.getLogger("goodwe.price.history")
|
||||||
|
|
||||||
|
# How many Brussels days of published prices the SORT-ORDER cap keeps. The fill
|
||||||
|
# needs exactly one - yesterday - and rule 2 refuses anything older, so this is
|
||||||
|
# not a horizon: it is the number of days a record may sit unused before
|
||||||
|
# pruning removes it, and it is small because a bigger cache would only hold
|
||||||
|
# days no lookup can ask for. Three covers today, yesterday, and the day a
|
||||||
|
# fetch spanning midnight writes ahead of itself.
|
||||||
|
#
|
||||||
|
# ⚠️ IT IS A CAP, NOT THE RETENTION RULE, AND THREE HAS NO SLACK. The steady
|
||||||
|
# state is already {yesterday, today, tomorrow}; a feed that publishes to D+2,
|
||||||
|
# or ONE stray point in the wrong year - which sorts LAST and is therefore
|
||||||
|
# immortal under a newest-N cap - pushes yesterday out of a three-slot store
|
||||||
|
# and keeps it out, permanently and silently. So `_prune()` PINS the two days a
|
||||||
|
# lookup can actually ask for on top of the cap; see its docstring. The store
|
||||||
|
# is then bounded at KEEP_DAYS + 2 records rather than at KEEP_DAYS.
|
||||||
|
KEEP_DAYS = 3
|
||||||
|
|
||||||
|
SCHEMA = 1
|
||||||
|
|
||||||
|
|
||||||
|
def _bdate(instant: datetime) -> str:
|
||||||
|
"""The Brussels calendar date of an instant, as `YYYY-MM-DD`.
|
||||||
|
|
||||||
|
⚠️ BRUSSELS, NOT UTC. 23:30 UTC in July is already tomorrow in Belgium, and
|
||||||
|
a cache keyed on the UTC date would file the last 90-120 minutes of every
|
||||||
|
day under the wrong one - so a lookup on the following morning would find a
|
||||||
|
"yesterday" missing its own evening hours, which are the expensive ones.
|
||||||
|
"""
|
||||||
|
return brussels(instant).strftime("%Y-%m-%d")
|
||||||
|
|
||||||
|
|
||||||
|
def _yesterday_of(instant: datetime) -> str:
|
||||||
|
"""The Brussels date of the day BEFORE `instant`'s Brussels day.
|
||||||
|
|
||||||
|
⚠️ THE ARITHMETIC IS ON THE LOCAL DATE, NOT ON THE INSTANT, AND THAT IS A
|
||||||
|
DST DEFECT THIS FILE SHIPPED. `brussels()` renders at the FIXED offset in
|
||||||
|
force at that instant, so `brussels(stamp) - timedelta(days=1)` subtracts 24
|
||||||
|
ABSOLUTE hours from a wall clock - and the Brussels day before a
|
||||||
|
spring-forward is 23 hours long. Measured at 2026-03-30T00:30+02:00, the
|
||||||
|
first hour of the morning after the March transition: it returned
|
||||||
|
2026-03-28, so the fill carried a TWO-DAY-OLD profile flagged as yesterday's
|
||||||
|
(or, with only the last three days held, answered `no_data` naming a date
|
||||||
|
that is not yesterday).
|
||||||
|
|
||||||
|
A calendar day minus one day is `date` arithmetic, and a `date` has no
|
||||||
|
offset to lose. The 25-hour October day is right for the same reason.
|
||||||
|
"""
|
||||||
|
return (brussels(instant).date() - timedelta(days=1)).strftime("%Y-%m-%d")
|
||||||
|
|
||||||
|
|
||||||
|
def _point_of(row):
|
||||||
|
"""One stored row -> a `PricePoint`, or `None` if the row is not one.
|
||||||
|
|
||||||
|
THE ONE VALIDATOR, AND THE ASYMMETRY IT CLOSES (FCST-05's MEDIUM). This
|
||||||
|
file used to hold TWO standards for what a stored point is. The READ path
|
||||||
|
(`_parse_points()`) was properly defensive - every field checked, a
|
||||||
|
malformed row dropped and reported. The WRITE path was
|
||||||
|
`{q["start"]: q for q in raw if isinstance(q, dict) and "start" in q}`
|
||||||
|
followed by `sorted(..., key=lambda q: q["start"])`, which trusts
|
||||||
|
`q["start"]` to be a hashable, sortable string it never checked. Measured,
|
||||||
|
with the malformed row under the day the feed is currently writing to:
|
||||||
|
|
||||||
|
start is an int -> TypeError: '<' not supported between instances
|
||||||
|
of 'str' and 'int'
|
||||||
|
start is a dict -> TypeError: cannot use 'dict' as a dict key
|
||||||
|
start is null -> TypeError: '<' not supported between 'str' and
|
||||||
|
'NoneType'
|
||||||
|
start is a list -> TypeError: cannot use 'list' as a dict key
|
||||||
|
|
||||||
|
⚠️ AND THE BLAST RADIUS IS PERMANENT, WHICH IS WHY IT IS NOT MERELY UNTIDY.
|
||||||
|
`_remember()` is reached through `carry()` -> `resolve_tariff()` ->
|
||||||
|
`run_tariff_refresh`, which catches `Exception` - so the add-on stays UP
|
||||||
|
and EVERY refresh raises forever. The plan retires after
|
||||||
|
`TARIFF_MAX_AGE_S`, and arbitrage and the cheap window are off from then
|
||||||
|
on. There is no self-heal, because `_remember()` raises before it reaches
|
||||||
|
`_save()`. `_usable_days()` only required `rec["points"]` to be a LIST; the
|
||||||
|
elements were unchecked on the write path, and the existing fixture put its
|
||||||
|
malformed row under a day the feed was NOT writing to, so `_remember()`
|
||||||
|
never touched it.
|
||||||
|
|
||||||
|
So both paths call this, and neither of them has a standard of its own.
|
||||||
|
Returning `None` rather than raising is the contract: a caller decides
|
||||||
|
whether a bad row is dropped (both of them do) and how it is reported.
|
||||||
|
"""
|
||||||
|
if not isinstance(row, dict):
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
start = datetime.fromisoformat(row["start"])
|
||||||
|
value = float(row["eur_per_kwh"])
|
||||||
|
flag = row["provisional"]
|
||||||
|
except (KeyError, TypeError, ValueError):
|
||||||
|
return None
|
||||||
|
if start.tzinfo is None or start.utcoffset() is None \
|
||||||
|
or not isinstance(flag, bool):
|
||||||
|
return None
|
||||||
|
return PricePoint(start=start, eur_per_kwh=value, provisional=flag)
|
||||||
|
|
||||||
|
|
||||||
|
def _held(rec) -> tuple:
|
||||||
|
"""`(points, whole hours they cover)` for a day record. `(0, 0)` otherwise.
|
||||||
|
|
||||||
|
⚠️ POINTS AND HOURS ARE TWO NUMBERS, AND FCST-04's STATUS ROW PRINTED THE
|
||||||
|
FIRST UNDER THE SECOND'S NAME. The row was `f"{d} ({_n_points(rec)} h)"`,
|
||||||
|
which counts POINTS: on the quarter-hourly feed this add-on already
|
||||||
|
supports it read `'2026-01-13 (96 h)'` - 96 "h" for a 24-hour day. That is
|
||||||
|
the points/hours conflation H1 was raised for, on the row H1's own ticket
|
||||||
|
shipped. Both numbers are reported now, so an operator can see a day that
|
||||||
|
is 24 points over 24 h and a day that is 96 points over 24 h and tell them
|
||||||
|
apart, and neither number is wearing the other's unit.
|
||||||
|
|
||||||
|
⚠️ TOLERANT, AND IT IS THE SECOND LINE OF DEFENCE, NOT THE FIRST. `_load()`
|
||||||
|
drops a malformed day record and says so, so this should never see one. It
|
||||||
|
is written this way because `words` is read from `Controller.__init__` and
|
||||||
|
from every status render: a status LINE that raises takes the add-on down,
|
||||||
|
and "the cache is broken" must never be able to mean "the controller does
|
||||||
|
not start". That is exactly what FCST-04 shipped - `{"days": {"2026-01-14":
|
||||||
|
"oops"}}` gave `ok=True`, `last_error=None`, and an `AttributeError` out of
|
||||||
|
`words`, `yesterday()` and `resolve_tariff()` alike, with no self-heal
|
||||||
|
because `_remember()` raised before it could rewrite the file.
|
||||||
|
|
||||||
|
⚠️ AND IT COUNTS ONLY ROWS THAT ARE POINTS. A row `_point_of()` refuses is
|
||||||
|
not held prices, and a row count that included it would report a store as
|
||||||
|
healthier than it is - which is the second half of FCST-05's MEDIUM: the
|
||||||
|
status row said `'2026-01-15 (1 h)'` about a record whose single row was
|
||||||
|
killing every tariff refresh. What is dropped is named in `last_error`, and
|
||||||
|
`words` prints that beside the counts.
|
||||||
|
"""
|
||||||
|
if not isinstance(rec, dict):
|
||||||
|
return (0, 0)
|
||||||
|
raw = rec.get("points")
|
||||||
|
if not isinstance(raw, (list, tuple)):
|
||||||
|
return (0, 0)
|
||||||
|
n, hours = 0, set()
|
||||||
|
for row in raw:
|
||||||
|
pt = _point_of(row)
|
||||||
|
if pt is None:
|
||||||
|
continue
|
||||||
|
n += 1
|
||||||
|
hours.add(pt.start.astimezone(timezone.utc)
|
||||||
|
.replace(minute=0, second=0, microsecond=0))
|
||||||
|
return (n, len(hours))
|
||||||
|
|
||||||
|
|
||||||
|
def _row_of(day: str, points: int, hours: int) -> str:
|
||||||
|
"""One day's cell of the status row: what is held, in BOTH of its units.
|
||||||
|
|
||||||
|
⚠️ THE UNIT IS SPELT OUT ON EACH NUMBER. FCST-04's cell was
|
||||||
|
`f"{day} ({points} h)"` - the point count wearing the hour's name, which on
|
||||||
|
a quarter-hourly feed reads `'2026-01-13 (96 h)'` for a 24-hour day. What
|
||||||
|
an operator needs from this row is whether tomorrow morning's fill has a
|
||||||
|
whole day to carry, and that is the HOUR figure; the point count says how
|
||||||
|
finely it is priced. Neither answers for the other.
|
||||||
|
"""
|
||||||
|
return (f"{day} ({points} point{'' if points == 1 else 's'} "
|
||||||
|
f"over {hours} h)")
|
||||||
|
|
||||||
|
|
||||||
|
def _bad_rows(rec) -> int:
|
||||||
|
"""How many rows of a day record are not points. Total; never raises."""
|
||||||
|
if not isinstance(rec, dict):
|
||||||
|
return 0
|
||||||
|
raw = rec.get("points")
|
||||||
|
if not isinstance(raw, (list, tuple)):
|
||||||
|
return 0
|
||||||
|
return sum(1 for row in raw if _point_of(row) is None)
|
||||||
|
|
||||||
|
|
||||||
|
class PriceHistory:
|
||||||
|
"""The published price series of the last few Brussels days, in /data.
|
||||||
|
|
||||||
|
⚠️ `ok` IS ABOUT THE FILE, NOT ABOUT THE DATA. A healthy store with no
|
||||||
|
record for yesterday is `ok` and answers `no_data` - a normal state on a
|
||||||
|
site's first day, and a different thing from a store that could not be
|
||||||
|
opened. Both are visible: `ok`/`last_error` describe the file, and
|
||||||
|
`yesterday()`'s reason describes the answer.
|
||||||
|
|
||||||
|
⚠️ AND `ok` IS NOT "CAN PERSIST" (FCST-06). It says the directory was
|
||||||
|
there when the store opened, which a full volume and a read-only remount
|
||||||
|
both leave true. The one thing that answers "can this store persist" is
|
||||||
|
`_save()`'s return value, and `self._days` NEVER HOLDS A POINT THE FILE DOES
|
||||||
|
NOT, because `_remember()` undoes a merge that answer refused. That is what
|
||||||
|
makes `words` and `yesterday()` describe the same store in EVERY degraded
|
||||||
|
state rather than only in the ones somebody enumerated.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, path: str = "/data/price_history.json"):
|
||||||
|
self.path = path
|
||||||
|
self.ok = False
|
||||||
|
self.last_error: str | None = None
|
||||||
|
self._days: dict = {}
|
||||||
|
self._open()
|
||||||
|
|
||||||
|
# -- the file ------------------------------------------------------------
|
||||||
|
def _open(self) -> None:
|
||||||
|
parent = os.path.dirname(self.path)
|
||||||
|
# See the module note: NOT created. A cache on the ephemeral layer is
|
||||||
|
# one that forgets without saying so.
|
||||||
|
if parent and not os.path.isdir(parent):
|
||||||
|
self.last_error = f"no durable volume at {parent}"
|
||||||
|
# INFO, not ERROR: on a dev host or in a test container there is no
|
||||||
|
# /data and nothing is wrong.
|
||||||
|
_LOG.info("no price history: %s - §8.3 carry-forward has nothing "
|
||||||
|
"to carry and the horizon stays as short as the feed",
|
||||||
|
self.last_error)
|
||||||
|
return
|
||||||
|
self.ok = True
|
||||||
|
self._load()
|
||||||
|
|
||||||
|
def _load(self) -> None:
|
||||||
|
try:
|
||||||
|
with open(self.path, "r", encoding="utf-8") as fh:
|
||||||
|
blob = json.load(fh)
|
||||||
|
except FileNotFoundError:
|
||||||
|
_LOG.info("no price history yet at %s - the first refresh writes "
|
||||||
|
"one, and the fill has nothing to carry until the day "
|
||||||
|
"after that", self.path)
|
||||||
|
return
|
||||||
|
except (ValueError, OSError) as err:
|
||||||
|
# A corrupt cache must not stop the controller, and must not be
|
||||||
|
# silently re-created either: the record is discarded, the reason
|
||||||
|
# is kept, and `yesterday()` reports it rather than answering
|
||||||
|
# "no record", which would read as a normal first day.
|
||||||
|
#
|
||||||
|
# ⚠️ `ValueError`, NOT `json.JSONDecodeError`, AND THAT IS
|
||||||
|
# FCST-05's HIGH. Reading this file has two failure families: the
|
||||||
|
# BYTES (`OSError`) and the CONTENT - and the content family has at
|
||||||
|
# least two members, because `open(..., encoding="utf-8")` decodes
|
||||||
|
# before `json` ever sees a character. `UnicodeDecodeError` is
|
||||||
|
# `ValueError -> UnicodeError -> UnicodeDecodeError`; it is NOT a
|
||||||
|
# `json.JSONDecodeError` and it is NOT an `OSError`
|
||||||
|
# (`issubclass(UnicodeDecodeError, OSError)` is False). So ONE
|
||||||
|
# non-UTF-8 byte in /data/price_history.json escaped `_load()` ->
|
||||||
|
# `_open()` -> `PriceHistory.__init__`, which is unguarded at
|
||||||
|
# main.py's `Controller.__init__`, AND THE ADD-ON DID NOT START.
|
||||||
|
# Measured: `UnicodeDecodeError: 'utf-8' codec can't decode byte
|
||||||
|
# 0xff in position 39`. Valid-UTF-8-invalid-JSON, a UTF-8 BOM and a
|
||||||
|
# non-dict day record all degraded correctly - it was the one byte
|
||||||
|
# class left uncovered, which is the same class as the defect the
|
||||||
|
# `_usable_days()` rework was raised for.
|
||||||
|
#
|
||||||
|
# The tuple names the FAMILY rather than the two members known
|
||||||
|
# today, because "the file's content is not readable as JSON" is
|
||||||
|
# exactly what this handler says, and a third `ValueError` subclass
|
||||||
|
# nobody has thought of is how this defect arrived the first time.
|
||||||
|
self.last_error = f"unreadable ({err})"
|
||||||
|
_LOG.warning("price history at %s is %s - starting from empty. "
|
||||||
|
"§8.3 carry-forward has nothing to carry until the "
|
||||||
|
"next refresh has written a full day",
|
||||||
|
self.path, self.last_error)
|
||||||
|
return
|
||||||
|
found = blob.get("version") if isinstance(blob, dict) else None
|
||||||
|
# ⚠️ THE TYPE IS PART OF THE VERSION. `1.0 == 1` in Python, so a float
|
||||||
|
# written by some other tool would pass a bare `!=` and be read as
|
||||||
|
# schema 1 - a version check that cannot tell 1 from 1.0 is one that
|
||||||
|
# would not notice a writer this build has never met.
|
||||||
|
if not isinstance(blob, dict) or isinstance(found, bool) \
|
||||||
|
or not isinstance(found, int) or found != SCHEMA:
|
||||||
|
self.last_error = f"schema {found!r}, not {SCHEMA}"
|
||||||
|
_LOG.warning("price history at %s has %s - discarding it rather "
|
||||||
|
"than guessing what its points mean",
|
||||||
|
self.path, self.last_error)
|
||||||
|
return
|
||||||
|
days = blob.get("days")
|
||||||
|
if not isinstance(days, dict):
|
||||||
|
self.last_error = "no `days` object"
|
||||||
|
_LOG.warning("price history at %s has %s", self.path, self.last_error)
|
||||||
|
return
|
||||||
|
self._days = self._usable_days(days)
|
||||||
|
|
||||||
|
def _usable_days(self, days: dict) -> dict:
|
||||||
|
"""The day records that are shaped like day records. The rest are dropped.
|
||||||
|
|
||||||
|
⚠️ THE ASYMMETRY THAT PRODUCED THE DEFECT: `_parse_points()` below is
|
||||||
|
carefully defensive about every field of a POINT, and `_load()` was not
|
||||||
|
defensive about the record holding them at all. `{"version": 1, "days":
|
||||||
|
{"2026-01-14": "oops"}}` therefore loaded as `ok=True`,
|
||||||
|
`last_error=None` - and then `rec.get(...)` raised `AttributeError` out
|
||||||
|
of `words`, `yesterday()` and `resolve_tariff()`. `PriceHistory(...)`
|
||||||
|
and `.words` are both unguarded in `Controller.__init__`, so THE ADD-ON
|
||||||
|
DID NOT START; and it never self-healed, because `_remember()` raised on
|
||||||
|
the same record before it could reach `_save()`.
|
||||||
|
|
||||||
|
A broken cache is a short horizon. It is not a controller that will not
|
||||||
|
regulate a battery, and §6's rule - absent machinery must not disable
|
||||||
|
the loop - does not stop applying because the machinery is a JSON file.
|
||||||
|
|
||||||
|
Dropped, not repaired: an unusable record has nothing in it to keep, and
|
||||||
|
the next `_remember()` writes a fresh one under that date.
|
||||||
|
|
||||||
|
⚠️ AND THE ROWS INSIDE A KEPT RECORD ARE COUNTED, NOT ONLY THE RECORD
|
||||||
|
(FCST-05). This checked that `rec["points"]` is a LIST and stopped
|
||||||
|
there, so a record holding one row with a `start` of `7` loaded as
|
||||||
|
healthy: `last_error=None`, and `words` said `'2026-01-15 (1 h)'` about
|
||||||
|
a store whose next `_remember()` would raise on that row and go on
|
||||||
|
raising forever. The rows are NOT dropped here - `_parse_points()` is
|
||||||
|
the read path's one filter and stays so - but they are COUNTED, so the
|
||||||
|
store stops reporting health it does not have from the moment it is
|
||||||
|
opened. `_remember()` drops them on the next refresh and `_save()`
|
||||||
|
rewrites the file, after which a restart clears the note.
|
||||||
|
"""
|
||||||
|
kept, dropped, malformed = {}, [], {}
|
||||||
|
for key, rec in days.items():
|
||||||
|
if not isinstance(key, str) or not isinstance(rec, dict) \
|
||||||
|
or not isinstance(rec.get("points"), list):
|
||||||
|
dropped.append(str(key))
|
||||||
|
continue
|
||||||
|
bad = _bad_rows(rec)
|
||||||
|
if bad:
|
||||||
|
malformed[key] = bad
|
||||||
|
kept[key] = rec
|
||||||
|
notes = []
|
||||||
|
if dropped:
|
||||||
|
notes.append(f"{len(dropped)} unusable day record(s) dropped "
|
||||||
|
f"({', '.join(sorted(dropped))})")
|
||||||
|
if malformed:
|
||||||
|
notes.append(
|
||||||
|
f"{sum(malformed.values())} malformed point row(s) in "
|
||||||
|
+ ", ".join(f"{d} ({n})" for d, n in sorted(malformed.items())))
|
||||||
|
if notes:
|
||||||
|
# Recorded, not silent: `yesterday()` and `words` both report
|
||||||
|
# `last_error`, and a cache that quietly discards half a file reads
|
||||||
|
# to an operator exactly like a normal first day.
|
||||||
|
self.last_error = "; ".join(notes)
|
||||||
|
_LOG.warning("price history at %s: %s - the controller starts "
|
||||||
|
"anyway and the horizon is as short as the feed makes "
|
||||||
|
"it", self.path, self.last_error)
|
||||||
|
return kept
|
||||||
|
|
||||||
|
def _save(self) -> bool:
|
||||||
|
"""Write `self._days` out. True only if the file now holds it.
|
||||||
|
|
||||||
|
⚠️ THE RETURN VALUE IS THE "CAN THIS STORE PERSIST?" PREDICATE, AND
|
||||||
|
IT IS THE ONLY ONE (FCST-06). Both ways of not persisting end here and
|
||||||
|
are the same answer: no durable volume returns False without touching
|
||||||
|
the disk, an `OSError` out of the write returns False after recording
|
||||||
|
why. `_remember()` acts on that one answer, so the second place that
|
||||||
|
used to test the first symptom on its own - and missed the second - has
|
||||||
|
nothing left to test. See the module docstring's fifth warning.
|
||||||
|
|
||||||
|
⚠️ AND IT REPORTS, IT DOES NOT RAISE. §6 and this module's opening: a
|
||||||
|
broken cache must not be a broken controller. An add-on that will not
|
||||||
|
start because /data filled up is a worse defect than the one FCST-06
|
||||||
|
fixed, so a failed write is logged, kept in `last_error` - where `words`
|
||||||
|
and `yesterday()` both read it - and RETURNED, never thrown.
|
||||||
|
|
||||||
|
⚠️ THE TEMP FILE IS REMOVED ON EVERY FAILING PATH. A write that dies
|
||||||
|
between `mkstemp` and `os.replace` used to leave its temp behind, and
|
||||||
|
the refresh task runs 48 times a day: a persistent write failure - a
|
||||||
|
full volume being the likeliest cause - leaked 48 inodes a day onto the
|
||||||
|
volume that was already the problem.
|
||||||
|
"""
|
||||||
|
if not self.ok:
|
||||||
|
return False
|
||||||
|
tmp = None
|
||||||
|
try:
|
||||||
|
fd, tmp = tempfile.mkstemp(dir=os.path.dirname(self.path))
|
||||||
|
with os.fdopen(fd, "w", encoding="utf-8") as fh:
|
||||||
|
json.dump({"version": SCHEMA, "days": self._days}, fh, indent=1)
|
||||||
|
# Atomic: a power cut mid-write must not leave a truncated file
|
||||||
|
# that parses as an empty day.
|
||||||
|
os.replace(tmp, self.path)
|
||||||
|
tmp = None # it IS the file now; do not unlink
|
||||||
|
except OSError as err:
|
||||||
|
self.last_error = f"could not write ({err})"
|
||||||
|
_LOG.error("could not persist price history: %s", err)
|
||||||
|
return False
|
||||||
|
finally:
|
||||||
|
if tmp is not None:
|
||||||
|
try:
|
||||||
|
os.unlink(tmp)
|
||||||
|
except OSError: # nothing further to try, and a
|
||||||
|
pass # failed cleanup is not a fault
|
||||||
|
return True
|
||||||
|
|
||||||
|
# -- the one public entry point -----------------------------------------
|
||||||
|
def carry(self, read: PriceRead, *, since: datetime, until: datetime,
|
||||||
|
stamp: datetime) -> PriceRead:
|
||||||
|
"""Remember what this read published, then §8.3-fill the missing hours.
|
||||||
|
|
||||||
|
THE ORDER IS DELIBERATE AND IS THE POINT OF THE METHOD. The read is
|
||||||
|
recorded FIRST, so a feed that hands back yesterday alongside today
|
||||||
|
refreshes yesterday's record from live data before it is consumed as
|
||||||
|
the profile; and the filled series - the one thing that must never be
|
||||||
|
recorded - exists only as this method's return value, which this object
|
||||||
|
never sees again. See rule 1, including how that order is graded.
|
||||||
|
|
||||||
|
Returns whatever `fill_from_yesterday()` returns: a series over the
|
||||||
|
whole hours of [since, until), carrying EVERY point the provider
|
||||||
|
published inside it - by identity, and four of them in one hour on a
|
||||||
|
quarter-hourly feed - plus one carried point for each remaining whole
|
||||||
|
hour FOR WHICH YESTERDAY HELD THAT WALL-CLOCK HOUR. An hour neither
|
||||||
|
source can price is left out (§11.4 for that hour), and `no_data` comes
|
||||||
|
back when that is every hour.
|
||||||
|
"""
|
||||||
|
self._remember(read, stamp)
|
||||||
|
return fill_from_yesterday(read, yesterday=self.yesterday(stamp),
|
||||||
|
since=since, until=until, stamp=stamp)
|
||||||
|
|
||||||
|
# -- the halves, private because rule 1 is an ordering ------------------
|
||||||
|
def _remember(self, read: PriceRead, stamp: datetime) -> None:
|
||||||
|
"""File a provider read's points under their own Brussels days.
|
||||||
|
|
||||||
|
⚠️ PRIVATE, AND CALLED FROM EXACTLY ONE PLACE. A caller that can reach
|
||||||
|
this can record a filled series, which is the decay in rule 1. There is
|
||||||
|
no value-level defence against that - a carried point and a published
|
||||||
|
provisional point are the same three fields - so the defence is that
|
||||||
|
nothing outside this file names it, and `test_price_history.py` checks
|
||||||
|
the tree for that rather than trusting the underscore.
|
||||||
|
"""
|
||||||
|
points = read.match(series=lambda r: r.points, no_data=lambda _r: ())
|
||||||
|
if not points:
|
||||||
|
return
|
||||||
|
product = read.match(series=lambda r: r.product.value,
|
||||||
|
no_data=lambda _r: None)
|
||||||
|
provider = read.match(series=lambda r: r.provider,
|
||||||
|
no_data=lambda _r: None)
|
||||||
|
|
||||||
|
# ⚠️ THE STATE THIS READ STARTED FROM, KEPT SO IT CAN BE PUT BACK
|
||||||
|
# (FCST-06). Everything below merges into `self._days` IN PLACE, and
|
||||||
|
# whether any of it may be KEPT is not knowable until `_save()` has
|
||||||
|
# tried - so the merge runs, and it is undone if the file did not take
|
||||||
|
# it. There is no `not self.ok` branch above any more: a store with no
|
||||||
|
# durable volume is one of the answers `_save()` gives, not a separate
|
||||||
|
# symptom with a test of its own. Module docstring, fifth warning.
|
||||||
|
#
|
||||||
|
# ⚠️ DEEP, NOT `dict(self._days)`. A one-level copy shares the day
|
||||||
|
# RECORDS, and the loop below rebinds keys on the ones it touches - so
|
||||||
|
# the "previous" state would hand back records the merge had already
|
||||||
|
# edited, and a store with a good yesterday on disk would go on
|
||||||
|
# answering the merge of it that never reached the file. It would also
|
||||||
|
# be right only for as long as nobody makes this loop mutate a `points`
|
||||||
|
# list or a row in place. A copy that is right by construction outlives
|
||||||
|
# that, and the store holds at most KEEP_DAYS + 2 records, 48 times a
|
||||||
|
# day.
|
||||||
|
before = copy.deepcopy(self._days)
|
||||||
|
|
||||||
|
touched = set()
|
||||||
|
junked = {}
|
||||||
|
for p in points:
|
||||||
|
day = _bdate(p.start)
|
||||||
|
rec = self._days.setdefault(
|
||||||
|
day, {"product": product, "provider": provider, "points": []})
|
||||||
|
# Rule 4: MERGE by instant. A later read updates the hours it
|
||||||
|
# carries and leaves the rest; a revised price for an hour already
|
||||||
|
# held wins, because it is the newer statement about that hour.
|
||||||
|
#
|
||||||
|
# ⚠️ EVERY HELD ROW GOES THROUGH `_point_of()` (FCST-05's MEDIUM).
|
||||||
|
# This was `{q["start"]: q for q in raw if isinstance(q, dict) and
|
||||||
|
# "start" in q}` sorted by `q["start"]` - a key and a sort key that
|
||||||
|
# trusted a value read off disk to be a hashable, sortable string.
|
||||||
|
# An `int`, `null`, a `dict` or a `list` there raised `TypeError`
|
||||||
|
# out of `carry()` on the day the feed was writing to, and
|
||||||
|
# `run_tariff_refresh` caught it, so the add-on stayed up while
|
||||||
|
# EVERY refresh failed for good. A row that is not a point is now
|
||||||
|
# dropped and reported here, exactly as the read path drops it, and
|
||||||
|
# `_save()` below writes the record back clean - which is the
|
||||||
|
# self-heal the raising version could never reach.
|
||||||
|
#
|
||||||
|
# ⚠️ AND THE MERGE KEY IS THE VALIDATED INSTANT, NOT ITS SPELLING.
|
||||||
|
# Aware datetimes hash and compare by instant, so this is rule 4 as
|
||||||
|
# rule 4 is written. The old string key merged by SPELLING, and the
|
||||||
|
# old lexicographic sort put the second 02:00 of the 25-hour
|
||||||
|
# October day (`+01:00`) BEFORE the first (`+02:00`). Only the
|
||||||
|
# stored order was affected - `_parse_points()` re-sorts by instant
|
||||||
|
# on the way out - but the store now holds what it says it holds.
|
||||||
|
raw = rec.get("points")
|
||||||
|
held = {}
|
||||||
|
for q in (raw if isinstance(raw, list) else ()):
|
||||||
|
kept = _point_of(q)
|
||||||
|
if kept is None:
|
||||||
|
junked[day] = junked.get(day, 0) + 1
|
||||||
|
continue
|
||||||
|
held[kept.start] = q
|
||||||
|
held[p.start] = p.canonical()
|
||||||
|
order = sorted(held, key=lambda t: t.astimezone(timezone.utc))
|
||||||
|
rec["points"] = [held[t] for t in order]
|
||||||
|
rec["product"], rec["provider"] = product, provider
|
||||||
|
touched.add(day)
|
||||||
|
|
||||||
|
if junked:
|
||||||
|
# Said, not swallowed. `words` prints `last_error`, so a store that
|
||||||
|
# has just discarded rows off disk stops reporting plain health -
|
||||||
|
# and after `_save()` below the file is clean, so a restart clears
|
||||||
|
# this. `_save()` overwrites it if the write itself fails, which is
|
||||||
|
# the more urgent fact of the two - and on that path the drop is
|
||||||
|
# undone with the rest of the merge, so those rows are still in the
|
||||||
|
# file and the next write that completes drops them again.
|
||||||
|
self.last_error = (
|
||||||
|
f"{sum(junked.values())} malformed point row(s) dropped from "
|
||||||
|
+ ", ".join(f"{d} ({n})" for d, n in sorted(junked.items())))
|
||||||
|
_LOG.warning("price history at %s: %s - the record is rewritten "
|
||||||
|
"without them and the refresh continues",
|
||||||
|
self.path, self.last_error)
|
||||||
|
self._prune(stamp)
|
||||||
|
if not self._save():
|
||||||
|
# ⚠️ A STORE THAT CANNOT PERSIST RECORDS NOTHING - the rule, asked
|
||||||
|
# ONCE as a question about the outcome instead of tested twice
|
||||||
|
# against two symptoms of it. Keeping the merge here is exactly the
|
||||||
|
# cache the module docstring refuses: `yesterday()` would answer a
|
||||||
|
# series the file does not contain, all day, until a restart
|
||||||
|
# dropped it without a word - and `words` and `yesterday()` would
|
||||||
|
# be describing two different stores. The reason is already in
|
||||||
|
# `last_error` (from `_save()`, or from `_open()` when there is no
|
||||||
|
# volume at all), so both answers now describe what is on disk.
|
||||||
|
#
|
||||||
|
# It does NOT raise, and §6 is why: a broken cache must not be a
|
||||||
|
# broken controller. `carry()` still returns the fill, the horizon
|
||||||
|
# is as short as the feed makes it, and the add-on regulates.
|
||||||
|
self._days = before
|
||||||
|
return
|
||||||
|
_LOG.debug("price history: %d point(s) filed under %s; %d day(s) held",
|
||||||
|
len(points), ", ".join(sorted(touched)), len(self._days))
|
||||||
|
|
||||||
|
def _prune(self, stamp: datetime) -> None:
|
||||||
|
"""Drop records no lookup can reach. A sort-order cap, plus two pins.
|
||||||
|
|
||||||
|
⚠️ NOTHING IS DELETED BY ARITHMETIC AGAINST THE CLOCK. A clock that
|
||||||
|
jumped - the failure mode WITNESS-03 spent a session on - would
|
||||||
|
otherwise delete every record in one pass and leave the fill with
|
||||||
|
nothing on the morning after, which is the one morning it is for.
|
||||||
|
Deleting on a bad clock is unrecoverable; holding a few days too many
|
||||||
|
is not, so the cap is `sorted(...)[-KEEP_DAYS:]` and stays that way.
|
||||||
|
|
||||||
|
⚠️ AND A CAP ALONE IS DEFEATED BY ONE POINT IN THE WRONG YEAR. Sorting
|
||||||
|
is lexicographic on the date string, so a stray far-future record sorts
|
||||||
|
LAST and is immortal - it consumes a slot for good, and with the steady
|
||||||
|
state already {yesterday, today, tomorrow} filling all three, yesterday
|
||||||
|
is what falls out. A feed that publishes to D+2 does the same thing
|
||||||
|
without any stray at all. So the two days `yesterday()` can ACTUALLY be
|
||||||
|
asked for are pinned on top of the cap.
|
||||||
|
|
||||||
|
`stamp` is READ here, which it was not before: the pin is the only use
|
||||||
|
of it, and it is additive. Pinning can only RETAIN a record, never
|
||||||
|
delete one, so a jumped `stamp` pins two dates that are not held and
|
||||||
|
changes nothing - the WITNESS-03 property is preserved by construction
|
||||||
|
rather than by remembering to check it.
|
||||||
|
"""
|
||||||
|
if len(self._days) <= KEEP_DAYS:
|
||||||
|
return
|
||||||
|
keep = set(sorted(self._days)[-KEEP_DAYS:])
|
||||||
|
keep |= {_bdate(stamp), _yesterday_of(stamp)}
|
||||||
|
for day in [d for d in self._days if d not in keep]:
|
||||||
|
del self._days[day]
|
||||||
|
|
||||||
|
def yesterday(self, stamp: datetime) -> PriceRead:
|
||||||
|
"""The published series of the Brussels day BEFORE `stamp`'s, or no_data.
|
||||||
|
|
||||||
|
⚠️ THE DATE IS COMPUTED, NOT LOOKED UP. See rule 2: `newest()` would
|
||||||
|
answer Friday's prices to a Monday fill, and the fill would flag them as
|
||||||
|
one day old because that is the only thing its flag can say.
|
||||||
|
"""
|
||||||
|
if stamp.tzinfo is None or stamp.utcoffset() is None:
|
||||||
|
raise ValueError(
|
||||||
|
f"yesterday({stamp!r}): a naive datetime is not an instant, "
|
||||||
|
f"and 'which Brussels day was yesterday' is exactly the "
|
||||||
|
f"question a wall-clock reading cannot answer.")
|
||||||
|
want = _yesterday_of(stamp)
|
||||||
|
if self.last_error and not self._days:
|
||||||
|
return PriceRead.no_data(
|
||||||
|
f"the price history at {self.path} is {self.last_error}, so "
|
||||||
|
f"§8.3 has no profile to carry forward for {want}; the plan "
|
||||||
|
f"covers only the hours the feed published")
|
||||||
|
rec = self._days.get(want)
|
||||||
|
if rec is None:
|
||||||
|
have = ", ".join(sorted(self._days)) or "no days at all"
|
||||||
|
return PriceRead.no_data(
|
||||||
|
f"no published prices recorded for {want} (held: {have}), so "
|
||||||
|
f"§8.3 has no profile to carry forward. A record for another "
|
||||||
|
f"day is not a yesterday")
|
||||||
|
points, bad = _parse_points(rec.get("points"))
|
||||||
|
if not points:
|
||||||
|
why = bad or "it is empty"
|
||||||
|
return PriceRead.no_data(
|
||||||
|
f"the {want} record holds no usable point ({why}), and an "
|
||||||
|
f"empty series is not a free day")
|
||||||
|
try:
|
||||||
|
product = PriceProduct(rec.get("product"))
|
||||||
|
except ValueError:
|
||||||
|
return PriceRead.no_data(
|
||||||
|
f"the {want} record names product {rec.get('product')!r}, "
|
||||||
|
f"which is not a PriceProduct. The optimiser switches on that "
|
||||||
|
f"enum, so a record that cannot name one cannot be a series")
|
||||||
|
if bad:
|
||||||
|
_LOG.warning("price history: %s in the %s record were dropped; "
|
||||||
|
"%d point(s) carried forward", bad, want, len(points))
|
||||||
|
return PriceRead.series(points, product,
|
||||||
|
str(rec.get("provider") or "price_history"),
|
||||||
|
stamp)
|
||||||
|
|
||||||
|
# -- observability -------------------------------------------------------
|
||||||
|
@property
|
||||||
|
def words(self) -> str:
|
||||||
|
"""One line for the status page. Says what is held, or why nothing is.
|
||||||
|
|
||||||
|
⚠️ THIS MUST NOT RAISE, EVER. It is read once from `Controller.__init__`
|
||||||
|
and again on every status render, and neither call site guards it - so
|
||||||
|
an exception here is an add-on that does not start. It went out that
|
||||||
|
way: a day record that was a string instead of a dict raised
|
||||||
|
`AttributeError` from the `len(...)` below. `_load()` now drops such a
|
||||||
|
record and `_held()` is total, so this has two independent reasons not
|
||||||
|
to fail on a broken file.
|
||||||
|
|
||||||
|
⚠️ AND IT COUNTED POINTS AND CALLED THEM HOURS (FCST-05's LOW). The row
|
||||||
|
was `f"{d} ({_n_points(rec)} h)"`; on the quarter-hourly feed this
|
||||||
|
add-on already supports that reads `'2026-01-13 (96 h)'` for a 24-hour
|
||||||
|
day. Both numbers are printed now - see `_held()` - because they are
|
||||||
|
two different facts and only one of them was ever being shown.
|
||||||
|
"""
|
||||||
|
if not self.ok:
|
||||||
|
return f"unavailable - {self.last_error}"
|
||||||
|
if not self._days:
|
||||||
|
why = f" - {self.last_error}" if self.last_error else ""
|
||||||
|
return f"empty{why} - nothing to carry forward yet"
|
||||||
|
held = "; ".join(_row_of(d, *_held(self._days[d]))
|
||||||
|
for d in sorted(self._days, key=str))
|
||||||
|
# A file that PARTLY loaded says so, or the dropped days read as days
|
||||||
|
# that were simply never written.
|
||||||
|
return held + (f" [{self.last_error}]" if self.last_error else "")
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_points(raw) -> tuple:
|
||||||
|
"""Canonical dicts -> PricePoints, and a phrase naming what was dropped.
|
||||||
|
|
||||||
|
⚠️ A MALFORMED POINT IS DROPPED, NOT DEFAULTED, and the drop is reported.
|
||||||
|
`float(row.get("eur_per_kwh", 0))` is the shape of the whole family of
|
||||||
|
defects this repo has been removing: a cache that lost a field would answer
|
||||||
|
"that hour was free" to a fill that flags it as an ordinary guess.
|
||||||
|
"""
|
||||||
|
if not isinstance(raw, list):
|
||||||
|
return (), "the `points` field is not a list"
|
||||||
|
out, dropped = [], 0
|
||||||
|
for row in raw:
|
||||||
|
# ⚠️ `_point_of()` IS THE ONE VALIDATOR, and this loop used to BE it.
|
||||||
|
# The write path had its own, weaker standard; see `_point_of()` for
|
||||||
|
# what that cost. One standard, two callers, no asymmetry to re-open.
|
||||||
|
pt = _point_of(row)
|
||||||
|
if pt is None:
|
||||||
|
dropped += 1
|
||||||
|
continue
|
||||||
|
out.append(pt)
|
||||||
|
out.sort(key=lambda p: p.start.astimezone(timezone.utc))
|
||||||
|
return tuple(out), (f"{dropped} malformed point(s)" if dropped else "")
|
||||||
|
|
||||||
|
|
||||||
|
class NoHistory:
|
||||||
|
"""The explicit "this caller keeps no history" - a NAME, not a `None`.
|
||||||
|
|
||||||
|
⚠️ IT EXISTS SO THE PARAMETER CAN BE REQUIRED. `history=None` meaning "skip
|
||||||
|
the fill" is the shape this repo keeps removing: a caller who forgets the
|
||||||
|
argument gets the truncated-horizon behaviour of the bug this ticket is
|
||||||
|
about, silently, and the code reads as though the fill is happening. With
|
||||||
|
no default there is nothing to forget, and a caller with no durable volume
|
||||||
|
- a check, a replay harness - has to say `NoHistory()` and be visible in
|
||||||
|
the diff.
|
||||||
|
|
||||||
|
It carries out the ordering faithfully: it records nothing and carries
|
||||||
|
nothing, so `carry()` returns the provider's read unchanged and the plan is
|
||||||
|
exactly what it was before FCST-04.
|
||||||
|
"""
|
||||||
|
|
||||||
|
ok = False
|
||||||
|
last_error = "no history is kept by this caller"
|
||||||
|
words = "not kept"
|
||||||
|
|
||||||
|
def carry(self, read: PriceRead, *, since, until, stamp) -> PriceRead:
|
||||||
|
return read
|
||||||
|
|
||||||
|
def yesterday(self, stamp: datetime) -> PriceRead:
|
||||||
|
return PriceRead.no_data(
|
||||||
|
"this caller keeps no price history (NoHistory), so §8.3 has no "
|
||||||
|
"profile to carry forward")
|
||||||
@@ -1019,7 +1019,7 @@ def static_status(opts: dict, now: datetime):
|
|||||||
|
|
||||||
|
|
||||||
async def resolve_tariff(opts: dict, since: datetime, until: datetime,
|
async def resolve_tariff(opts: dict, since: datetime, until: datetime,
|
||||||
transport=None, now=None) -> TariffPlan:
|
transport=None, now=None, *, history) -> TariffPlan:
|
||||||
"""The full §9.2 + §11.4 resolution. MAY PERFORM I/O - see `static_plan`.
|
"""The full §9.2 + §11.4 resolution. MAY PERFORM I/O - see `static_plan`.
|
||||||
|
|
||||||
⚠️ NOT ON THE 30 s TICK, and that is a deliberate boundary rather than an
|
⚠️ NOT ON THE 30 s TICK, and that is a deliberate boundary rather than an
|
||||||
@@ -1033,9 +1033,42 @@ async def resolve_tariff(opts: dict, since: datetime, until: datetime,
|
|||||||
price_provider off + profile -> the CONFIGURED product (§9.2 rows 3-4)
|
price_provider off + profile -> the CONFIGURED product (§9.2 rows 3-4)
|
||||||
price_provider off + none -> no prices at all (§11.4, nothing to
|
price_provider off + none -> no prices at all (§11.4, nothing to
|
||||||
fall back to)
|
fall back to)
|
||||||
provider series -> the feed (§9.2 rows 1-2)
|
provider series -> the feed, §8.3-FILLED (§9.2 rows 1-2)
|
||||||
provider no_data + profile -> §11.4 FALLBACK, provisional, no arbitrage
|
provider no_data + profile -> §11.4 FALLBACK, provisional, no arbitrage
|
||||||
provider no_data + none -> no prices at all
|
provider no_data + none -> no prices at all
|
||||||
|
|
||||||
|
⚠️ `history` IS REQUIRED, KEYWORD-ONLY, AND HAS NO DEFAULT (FCST-04). It is
|
||||||
|
the §8.3 carry-forward store - `price_history.PriceHistory`, or the explicit
|
||||||
|
`price_history.NoHistory()` for a caller that keeps none. A default of
|
||||||
|
`None` meaning "skip the fill" would give a caller who forgot the argument
|
||||||
|
the TRUNCATED HORIZON this ticket exists to remove, silently, while the code
|
||||||
|
read as though the fill were happening. There is nothing to forget now.
|
||||||
|
|
||||||
|
⚠️ AND THE FILL RUNS ONLY ON A `series` READ. A feed that published nothing
|
||||||
|
still falls to `_degrade()` and §11.4's own row - the static profile the
|
||||||
|
operator configured FOR that case - rather than to yesterday's feed. The two
|
||||||
|
are indistinguishable to the optimiser (both are 100 % provisional, so
|
||||||
|
`arbitrage()` finds fewer than two firm hours and refuses, and `cheap_now()`
|
||||||
|
is False), so preferring yesterday would change a documented degradation for
|
||||||
|
a difference only the status page can see. See price_history.py's closing
|
||||||
|
note and DECISIONS/FCST-04-the-day-that-is-carried.md.
|
||||||
|
|
||||||
|
⚠️ THE FILL IS PER WHOLE HOUR AND ITS COVERAGE IS CONDITIONAL. Every point
|
||||||
|
the provider published inside the window passes through BY IDENTITY - four
|
||||||
|
of them in one hour on a quarter-hourly product like `eneco_eddy` - and a
|
||||||
|
carried point is added only for a whole hour the feed did not print at all
|
||||||
|
AND yesterday did. An hour neither source can price is left out, which is
|
||||||
|
§11.4 for that hour: `plan.price_at()` raises rather than answering, and an
|
||||||
|
hour with no price is not a free one.
|
||||||
|
|
||||||
|
⚠️ THE FILLED SERIES IS WINDOWED TO [since, until), AND THAT IS A CHANGE.
|
||||||
|
`load_prices()` returns whatever the feed sent, past hours included, and
|
||||||
|
`TariffPlan.arbitrage()` took its spread over all of it - so an hour that
|
||||||
|
had already gone could be the cheapest hour of a cycle nothing can now buy
|
||||||
|
in. `fill_from_yesterday()` walks the requested horizon and emits exactly
|
||||||
|
that, so the spread is over hours that are still ahead. Both directions of
|
||||||
|
the difference are real: the window gains the unpublished hours it was
|
||||||
|
missing and loses the elapsed ones it should never have had.
|
||||||
"""
|
"""
|
||||||
stamp = now or datetime.now(timezone.utc)
|
stamp = now or datetime.now(timezone.utc)
|
||||||
pid = str(price_option(opts, "price_provider") or "off").strip()
|
pid = str(price_option(opts, "price_provider") or "off").strip()
|
||||||
@@ -1044,6 +1077,20 @@ async def resolve_tariff(opts: dict, since: datetime, until: datetime,
|
|||||||
|
|
||||||
read = await load_prices(opts, since, until, transport)
|
read = await load_prices(opts, since, until, transport)
|
||||||
return read.match(
|
return read.match(
|
||||||
|
series=lambda r: _provider_plan(opts, r, since, until, stamp, history),
|
||||||
|
no_data=lambda r: _degrade(opts, r, since, until, stamp))
|
||||||
|
|
||||||
|
|
||||||
|
def _provider_plan(opts, read, since, until, stamp, history) -> TariffPlan:
|
||||||
|
"""§8.3: record what the feed published, carry yesterday into the gaps.
|
||||||
|
|
||||||
|
The fill can itself return `no_data` - a feed whose points all fall outside
|
||||||
|
the horizon, with nothing in the history to fill it from - and that is the
|
||||||
|
§11.4 row arrived at one step later, so it degrades exactly as a failed
|
||||||
|
fetch does rather than being special-cased here.
|
||||||
|
"""
|
||||||
|
filled = history.carry(read, since=since, until=until, stamp=stamp)
|
||||||
|
return filled.match(
|
||||||
series=lambda r: TariffPlan(r, _quiet_profile(opts), "provider"),
|
series=lambda r: TariffPlan(r, _quiet_profile(opts), "provider"),
|
||||||
no_data=lambda r: _degrade(opts, r, since, until, stamp))
|
no_data=lambda r: _degrade(opts, r, since, until, stamp))
|
||||||
|
|
||||||
|
|||||||
@@ -253,7 +253,7 @@ class MigrationReport:
|
|||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
def migrate_unit_keys(store, unit: str) -> MigrationReport:
|
def migrate_unit_keys(store, unit: str, roster=None) -> MigrationReport:
|
||||||
"""Move the pre-DEV-02 flat per-inverter keys into `unit`'s namespace.
|
"""Move the pre-DEV-02 flat per-inverter keys into `unit`'s namespace.
|
||||||
|
|
||||||
`unit` is the FIRST inverter in the roster, which on every install that can
|
`unit` is the FIRST inverter in the roster, which on every install that can
|
||||||
@@ -261,6 +261,24 @@ def migrate_unit_keys(store, unit: str) -> MigrationReport:
|
|||||||
version that knew about more than one inverter is already at
|
version that knew about more than one inverter is already at
|
||||||
SCHEMA_VERSION and is not touched.
|
SCHEMA_VERSION and is not touched.
|
||||||
|
|
||||||
|
⚠️ `roster` IS THE NAMES THE MACHINES READ, AND IT IS SAFETY-44. This
|
||||||
|
function shipped WITHOUT it, taking a hardcoded `inverter.UNIT_A` at its one
|
||||||
|
call site - which is, exactly, the MAINT-05 defect `_verify_moves()` spends
|
||||||
|
a paragraph describing, still live in the sibling that paragraph was written
|
||||||
|
beside. On a store that ran pre-DEV-02 flat and declares
|
||||||
|
`inverters: [garage, shed]`, the latch moved to `unit/inverter-a/
|
||||||
|
sign_tripped`, was read back under that same name, was SATISFIED, and
|
||||||
|
stamped the marker - so the next start skipped the migration too and NO
|
||||||
|
inverter came up latched on an install whose state file says `sign_tripped:
|
||||||
|
true`. A disarmed SAFETY-02 guard, every symptom of a clean migration.
|
||||||
|
|
||||||
|
`roster_names(opts)` answers this BEFORE any `Inverter` exists - it is
|
||||||
|
defined to, for `MqttPublisher`, which is constructed before the roster for
|
||||||
|
the same reason - so the "there is no roster yet" that kept this argument
|
||||||
|
out was never true. On the flat shape it returns `[UNIT_A]`, so the upgrade
|
||||||
|
every install runs today passes `unit == roster[0] == UNIT_A` and is
|
||||||
|
byte-identical.
|
||||||
|
|
||||||
THE THREE RULES, each one a case that would otherwise be a wrong answer:
|
THE THREE RULES, each one a case that would otherwise be a wrong answer:
|
||||||
|
|
||||||
1. IT NEVER OVERWRITES AN EXISTING NAMESPACED VALUE. A store that has both
|
1. IT NEVER OVERWRITES AN EXISTING NAMESPACED VALUE. A store that has both
|
||||||
@@ -302,7 +320,7 @@ def migrate_unit_keys(store, unit: str) -> MigrationReport:
|
|||||||
|
|
||||||
rep.ran = True
|
rep.ran = True
|
||||||
_move_flat_keys(store, unit, LEGACY_UNIT_KEYS, rep)
|
_move_flat_keys(store, unit, LEGACY_UNIT_KEYS, rep)
|
||||||
_verify_moves(store, rep)
|
_verify_moves(store, rep, roster)
|
||||||
|
|
||||||
if rep.ok:
|
if rep.ok:
|
||||||
store.set(SCHEMA_KEY, SCHEMA_VERSION)
|
store.set(SCHEMA_KEY, SCHEMA_VERSION)
|
||||||
@@ -383,12 +401,26 @@ def _verify_moves(store, rep: MigrationReport, roster=None) -> None:
|
|||||||
itself. `test_maintenance.py` section 9c calls this twice with the SAME
|
itself. `test_maintenance.py` section 9c calls this twice with the SAME
|
||||||
wrong `unit` over one store, which is the shape nothing in the suite had.
|
wrong `unit` over one store, which is the shape nothing in the suite had.
|
||||||
|
|
||||||
`roster=None` means "not graded", and the one caller that passes it is
|
`roster=None` still means "not graded", and SAFETY-44 is what it cost. This
|
||||||
`migrate_unit_keys()`, which runs BEFORE `build_roster()` for the reason its
|
paragraph used to end here, saying that the one caller passing it was
|
||||||
own docstring gives - the `Inverter` objects restore their latch in their
|
`migrate_unit_keys()`, "which runs BEFORE `build_roster()` ... so the keys
|
||||||
constructors, so the keys have to be in place before there is a roster to
|
have to be in place before there is a roster to check against", and calling
|
||||||
check against. That gap is real and is recorded as a running check rather
|
that gap real. THE FIRST HALF IS TRUE AND THE CONCLUSION IS NOT. The keys do
|
||||||
than left to a comment: see test_maintenance.py section 13.
|
have to be written before the `Inverter`s restore from them - but the NAMES
|
||||||
|
do not come from the `Inverter`s. `roster_names(opts)` reads them straight
|
||||||
|
off the options, before anything is constructed, which is the whole reason
|
||||||
|
it exists (`MqttPublisher` is built before the roster too). So the argument
|
||||||
|
that could have graded that write was available at the call site the entire
|
||||||
|
time, and the sibling function three screens down had already been fixed by
|
||||||
|
passing it.
|
||||||
|
|
||||||
|
⚠️ SO NO CALLER PASSES `None` TODAY, and this branch is kept for one reason:
|
||||||
|
it is what a NEW caller gets by DEFAULT. A future migration that forgets the
|
||||||
|
roster gets the ungraded read-back and stamps its marker over a namespace no
|
||||||
|
machine reads - silently, exactly as this one did. That is the shape of both
|
||||||
|
defects, and the gap is recorded as a running check rather than left to a
|
||||||
|
comment: test_maintenance.py section 13, and test_rehearse.py part 5, which
|
||||||
|
drives it end-to-end through the harness an operator actually runs.
|
||||||
"""
|
"""
|
||||||
for target, value in rep.moved.items():
|
for target, value in rep.moved.items():
|
||||||
got = getattr(store, "data", {}).get(target, "<<missing>>")
|
got = getattr(store, "data", {}).get(target, "<<missing>>")
|
||||||
|
|||||||
@@ -21,6 +21,7 @@ Read MUTATION.md before you report a survivor.
|
|||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
import dataclasses
|
import dataclasses
|
||||||
|
import logging
|
||||||
import re
|
import re
|
||||||
import sys
|
import sys
|
||||||
from datetime import datetime, timedelta, timezone
|
from datetime import datetime, timedelta, timezone
|
||||||
@@ -34,7 +35,8 @@ from app.inverter import UNIT_A
|
|||||||
from app.main import Controller
|
from app.main import Controller
|
||||||
from app.store import (
|
from app.store import (
|
||||||
DEFAULTS, MAINT_SCHEMA_KEY, MAINT_SCHEMA_VERSION, MAINT_UNIT_KEYS,
|
DEFAULTS, MAINT_SCHEMA_KEY, MAINT_SCHEMA_VERSION, MAINT_UNIT_KEYS,
|
||||||
migrate_maint_keys, unit_key,
|
SCHEMA_KEY, SCHEMA_VERSION, migrate_maint_keys, migrate_unit_keys,
|
||||||
|
unit_key,
|
||||||
)
|
)
|
||||||
|
|
||||||
fails = []
|
fails = []
|
||||||
@@ -1411,45 +1413,39 @@ check("...while the SAME site with the house importing more than the drain "
|
|||||||
|
|
||||||
# --------------------------------------------------------------------------- #
|
# --------------------------------------------------------------------------- #
|
||||||
print()
|
print()
|
||||||
print("13. RECORDED, NOT FIXED: SAFETY-02's migration has the same "
|
print("13. FIXED (SAFETY-44): SAFETY-02's migration takes the roster too, and "
|
||||||
"wrong-namespace hole, and it is a LATCH")
|
"the guard is no longer defeated by the key the defect wrote")
|
||||||
|
|
||||||
# ⚠️ FOUND WHILE FIXING THIS CARD'S OWN INSTANCE, AND WRITTEN DOWN AS A RUNNING
|
# ⚠️ THIS SECTION USED TO BE THE SPECIFICATION OF A DEFECT. It was written while
|
||||||
# CHECK RATHER THAN A COMMENT SO IT CANNOT BE FORGOTTEN OR QUIETLY "FIXED" BY AN
|
# fixing MAINT-05's own instance, as a RUNNING check rather than a comment, with
|
||||||
# UNRELATED EDIT. It is the SAME defect one module over, and it costs more.
|
# the instruction that "the card that fixes it should expect them to go red and
|
||||||
|
# should replace them". SAFETY-44 is that card and this is that replacement.
|
||||||
#
|
#
|
||||||
# WHAT IT IS. `Controller.__init__` calls `migrate_unit_keys(store, UNIT_A)` -
|
# WHAT IT WAS. `Controller.__init__` called `migrate_unit_keys(store, UNIT_A)` -
|
||||||
# a hardcoded constant, where the maintenance migration now passes the roster's
|
# a hardcoded constant, where the maintenance migration already passed the
|
||||||
# own first name. It has to run BEFORE `build_roster()`, because each
|
# roster's own first name. On a DECLARED roster the pre-DEV-02 flat
|
||||||
# `Inverter` restores its sign latch in its constructor, so there is no roster
|
# `sign_tripped: true` landed under `unit/inverter-a/sign_tripped`, which no
|
||||||
# to name yet and `_verify_moves()` is passed `roster=None`. On a DECLARED
|
# unit on the site reads, and `_verify_moves()` read it back under the name it
|
||||||
# roster the pre-DEV-02 flat `sign_tripped: true` therefore lands under
|
# had been handed and was satisfied.
|
||||||
# `unit/inverter-a/sign_tripped`, which no unit on this site reads.
|
|
||||||
#
|
#
|
||||||
# WHY THE EXISTING GUARD DOES NOT CATCH IT, WHICH IS THE PART WORTH PINNING.
|
# AND THE GUARD DID NOT CATCH IT, WHICH WAS THE PART WORTH PINNING.
|
||||||
# `_grade_store_migration()` re-arms the latch when the flat key says tripped
|
# `_grade_store_migration()` re-arms when the flat key says tripped and NOTHING
|
||||||
# and NOTHING NAMESPACED IS LATCHED. The misdirected write IS namespaced and IS
|
# NAMESPACED IS LATCHED - and the misdirected write IS namespaced and IS
|
||||||
# latched, so `namespaced_latched` is True, the guard returns, and no inverter
|
# latched. The guard was defeated by the very key the defect wrote.
|
||||||
# on the roster comes up tripped. The guard is defeated by the very key the
|
|
||||||
# defect wrote.
|
|
||||||
#
|
#
|
||||||
# WHY IT IS NOT FIXED HERE. The remedy is to name the migration's target from
|
# ⚠️ THE REASON IT WAS LEFT - "there is no roster to name yet, because each
|
||||||
# the config before any `Inverter` exists, which means factoring
|
# `Inverter` restores its latch in its constructor" - WAS THE HALF-TRUTH THAT
|
||||||
# `build_roster()`'s naming rule out of `build_roster()` so there is still only
|
# KEPT IT ALIVE. The keys must be written before the `Inverter`s read them; the
|
||||||
# ONE spelling of it - inverter.py and SAFETY-02's latch semantics, with its own
|
# NAMES do not come from the `Inverter`s. `roster_names(opts)` reads them off
|
||||||
# mutation battery over test_inverter.py and test_watchdog.py. MAINT-05 moves
|
# the options before anything is constructed, which is what it already existed
|
||||||
# the maintenance history and nothing about the sign latch.
|
# to do for `MqttPublisher`. The factoring this section said the fix would need
|
||||||
#
|
# had been done a card earlier.
|
||||||
# ⚠️ AND IT IS A DISARMED SAFETY GUARD, NOT LOST HISTORY. Read the checks below
|
|
||||||
# as the SPECIFICATION OF A DEFECT: every one of them is asserting the WRONG
|
|
||||||
# behaviour, on purpose, and the card that fixes it should expect them to go
|
|
||||||
# red and should replace them.
|
|
||||||
#
|
#
|
||||||
# ⚠️ NO `schema_version` IN THIS STORE, AND THAT IS THE PRE-DEV-02 SHAPE
|
# ⚠️ NO `schema_version` IN THIS STORE, AND THAT IS THE PRE-DEV-02 SHAPE
|
||||||
# ITSELF: an install that has never run a version with namespaced keys, which is
|
# ITSELF: an install that has never run a version with namespaced keys, which is
|
||||||
# what makes `migrate_unit_keys()` run at all.
|
# what makes `migrate_unit_keys()` run at all.
|
||||||
_lat_store = MemStore({"sign_tripped": True,
|
_lat_store = MemStore({"sign_tripped": True,
|
||||||
"sign_reason": "SIGN_DISAGREEMENT: measured pre-DEV-02"})
|
"sign_reason": "SIGN_DISAGREEMENT: SYNTHETIC FIXTURE, not measured"})
|
||||||
_lat, _ = controller(NAMED_OPTS, store=_lat_store)
|
_lat, _ = controller(NAMED_OPTS, store=_lat_store)
|
||||||
print(f" pre-DEV-02 flat latch + declared roster {_C2_NAMES} -> "
|
print(f" pre-DEV-02 flat latch + declared roster {_C2_NAMES} -> "
|
||||||
f"{_lat.store_migration.explain()}")
|
f"{_lat.store_migration.explain()}")
|
||||||
@@ -1457,29 +1453,179 @@ print(f" latch keys in the store: "
|
|||||||
f"{sorted(k for k in _lat_store.data if k.endswith('sign_tripped'))}")
|
f"{sorted(k for k in _lat_store.data if k.endswith('sign_tripped'))}")
|
||||||
print(f" roster came up tripped: "
|
print(f" roster came up tripped: "
|
||||||
f"{[(i.name, i.sign_wd.tripped) for i in _lat.inverters]}")
|
f"{[(i.name, i.sign_wd.tripped) for i in _lat.inverters]}")
|
||||||
check("the flat pre-DEV-02 latch is migrated into `unit/inverter-a/` - a name "
|
check("⭐ the flat pre-DEV-02 latch is migrated into the ROSTER'S OWN first "
|
||||||
"that is NOT on this roster, so no unit reads it. This is the same "
|
"namespace and NOT into `unit/inverter-a/` - the same wrong-namespace "
|
||||||
"wrong-namespace write `migrate_maint_keys()` now refuses, in the "
|
"write `migrate_maint_keys()` refuses, now refused in the migration that "
|
||||||
"migration that runs before there is a roster to grade against",
|
"runs before the `Inverter` objects exist",
|
||||||
_lat_store.data.get(unit_key(UNIT_A, "sign_tripped")) is True
|
_lat_store.data.get(unit_key(_C2_NAMES[0], "sign_tripped")) is True
|
||||||
and unit_key(_C2_NAMES[0], "sign_tripped") not in _lat_store.data)
|
and unit_key(UNIT_A, "sign_tripped") not in _lat_store.data)
|
||||||
check("⚠️ AND NO INVERTER ON THE ROSTER COMES UP LATCHED, on a store whose file "
|
check("⭐ AND AN INVERTER ON THE ROSTER COMES UP LATCHED, which is the whole "
|
||||||
"still reads `sign_tripped: true`. That is DEV-02's own sentence for the "
|
"card: a store whose file reads `sign_tripped: true` no longer produces "
|
||||||
"worst case - a disarmed safety guard on an install that believes it is "
|
"a site that drives",
|
||||||
"protected",
|
_lat.inverters[0].sign_wd.tripped
|
||||||
not any(i.sign_wd.tripped for i in _lat.inverters)
|
|
||||||
and _lat_store.data["sign_tripped"] is True)
|
and _lat_store.data["sign_tripped"] is True)
|
||||||
check("...and `_grade_store_migration()` is DEFEATED BY THE KEY THE DEFECT "
|
check("...and the latch that came back carries the REASON, not just the "
|
||||||
"WROTE, which is why this needs a card rather than a tweak: its "
|
"boolean - an operator meeting a FAILSAFE banner with no sentence under "
|
||||||
"'is anything namespaced latched' test is satisfied by the misdirected "
|
"it has been told the site stopped and not why",
|
||||||
"value itself, so it returns instead of re-arming",
|
"SIGN_DISAGREEMENT" in _lat.inverters[0].sign_wd.reason)
|
||||||
any(k.startswith("unit/") and k.endswith("/sign_tripped") and v
|
check("⚠️ ...and the OTHER units on the roster are NOT latched by it. The flat "
|
||||||
for k, v in _lat_store.data.items())
|
"key is one site-level boolean and re-arming every unit off it would be "
|
||||||
and not any(i.sign_wd.tripped for i in _lat.inverters))
|
"DEV-02's defect in the opposite direction - a healthy pack parked "
|
||||||
|
"because its sibling once disagreed",
|
||||||
|
not any(i.sign_wd.tripped for i in _lat.inverters[1:]))
|
||||||
|
|
||||||
|
# ⚠️ AND THE OTHER PATH, WHICH THE MIGRATION FIX ALONE DOES NOT REACH: a store
|
||||||
|
# ALREADY STAMPED by the shipped version. `migrate_unit_keys()` returns at its
|
||||||
|
# `from_version >= SCHEMA_VERSION` check and never moves anything, so the
|
||||||
|
# stranded `unit/inverter-a/sign_tripped` just sits there. This is the STAGED
|
||||||
|
# shape - "upgrade now, declare the roster later" - and it is the one an
|
||||||
|
# operator actually produces. Only the GUARD repairs it.
|
||||||
|
_staged_store = MemStore({
|
||||||
|
"sign_tripped": True,
|
||||||
|
"sign_reason": "SIGN_DISAGREEMENT: SYNTHETIC FIXTURE, not measured",
|
||||||
|
unit_key(UNIT_A, "sign_tripped"): True,
|
||||||
|
unit_key(UNIT_A, "sign_reason"): "SIGN_DISAGREEMENT: SYNTHETIC FIXTURE, not measured",
|
||||||
|
SCHEMA_KEY: SCHEMA_VERSION})
|
||||||
|
_staged, _ = controller(NAMED_OPTS, store=_staged_store)
|
||||||
|
print(f" already-stamped store + declared roster -> ran="
|
||||||
|
f"{_staged.store_migration.ran}, came up tripped: "
|
||||||
|
f"{[(i.name, i.sign_wd.tripped) for i in _staged.inverters]}")
|
||||||
|
check("⚠️ PRECONDITION: the migration really does NOT run on this store - the "
|
||||||
|
"marker is already there - so anything below is the guard's doing and "
|
||||||
|
"not the migration's",
|
||||||
|
not _staged.store_migration.ran)
|
||||||
|
check("⭐ and the roster's first unit STILL comes up latched: the guard no "
|
||||||
|
"longer counts a latch stranded under `UNIT_A`, which is not on this "
|
||||||
|
"roster, as protection an operator has",
|
||||||
|
_staged.inverters[0].sign_wd.tripped)
|
||||||
|
check("...and the re-armed latch is written where a machine reads it, so the "
|
||||||
|
"NEXT start finds it without needing the guard again",
|
||||||
|
_staged_store.data.get(unit_key(_C2_NAMES[0], "sign_tripped")) is True)
|
||||||
|
|
||||||
|
# ⚠️ THE NEGATIVE CONTROL FOR THAT NARROWING, because "off the roster" on its
|
||||||
|
# own was too blunt and test_inverter.py's DEV-02 restart case caught it: a
|
||||||
|
# latch stranded under a name that is NOT `UNIT_A` is a real unit out of
|
||||||
|
# today's config, and it MUST still count as protection.
|
||||||
|
_other_store = MemStore({
|
||||||
|
"sign_tripped": True,
|
||||||
|
"sign_reason": "SIGN_DISAGREEMENT: B disagreed, mirrored flat",
|
||||||
|
unit_key("inverter-b", "sign_tripped"): True,
|
||||||
|
unit_key("inverter-b", "sign_reason"): "SIGN_DISAGREEMENT: B disagreed",
|
||||||
|
SCHEMA_KEY: SCHEMA_VERSION})
|
||||||
|
_other, _ = controller(NAMED_OPTS, store=_other_store)
|
||||||
|
print(f" stranded latch under `inverter-b` -> came up tripped: "
|
||||||
|
f"{[(i.name, i.sign_wd.tripped) for i in _other.inverters]}")
|
||||||
|
check("⚠️ a latch stranded under a name that is not `UNIT_A` is NOT re-armed "
|
||||||
|
"onto the roster: it belongs to a unit the operator has taken out of "
|
||||||
|
"the config, the flat key is its own mirror, and re-arming a healthy "
|
||||||
|
"sibling off it is the defect DEV-02 removed",
|
||||||
|
not any(i.sign_wd.tripped for i in _other.inverters))
|
||||||
|
|
||||||
|
# ⭐ AND THE ARRANGEMENT NEITHER FIXTURE ABOVE HOLDS: **BOTH** KEYS AT ONCE.
|
||||||
|
# ⚠️ THIS IS INSTANCE #17 OF HABIT 4 AND IT WAS FOUND IN THIS CARD'S OWN FIX.
|
||||||
|
# The first version of the guard classified each key and then wrote ONE shared
|
||||||
|
# `namespaced_latched` flag for both "on the roster" and "off-roster but not
|
||||||
|
# UNIT_A". `_staged_store` above holds only an `inverter-a` latch and
|
||||||
|
# `_other_store` below holds only an `inverter-b` one, so the flag was never
|
||||||
|
# written twice with opposite intents in a single store - and a per-key check
|
||||||
|
# whose keys have never disagreed WITHIN ONE STORE is a check over whichever
|
||||||
|
# key happened to come last. Measured on that build, one added key:
|
||||||
|
#
|
||||||
|
# sign_tripped + unit/inverter-a/sign_tripped -> garage latched, repaired
|
||||||
|
# ...AND unit/old-shed/sign_tripped -> ⚠️ NOTHING latched
|
||||||
|
#
|
||||||
|
# A site DRIVING on a state file that reads `sign_tripped: true` - the sentence
|
||||||
|
# this card exists to make false - reached by an operator taking one unit out of
|
||||||
|
# `inverters:` after it had tripped. "A roster is configuration and it changes"
|
||||||
|
# is this module's own phrase, and the second inverter goes in this month.
|
||||||
|
_both_store = MemStore({
|
||||||
|
"sign_tripped": True,
|
||||||
|
"sign_reason": "SIGN_DISAGREEMENT: SYNTHETIC FIXTURE, not measured",
|
||||||
|
unit_key(UNIT_A, "sign_tripped"): True,
|
||||||
|
unit_key(UNIT_A, "sign_reason"): "SIGN_DISAGREEMENT: SYNTHETIC FIXTURE, not measured",
|
||||||
|
unit_key("old-shed", "sign_tripped"): True,
|
||||||
|
unit_key("old-shed", "sign_reason"): "SIGN_DISAGREEMENT: old-shed, since removed",
|
||||||
|
SCHEMA_KEY: SCHEMA_VERSION})
|
||||||
|
_both, _ = controller(NAMED_OPTS, store=_both_store)
|
||||||
|
print(f" stranded `inverter-a` AND an unrelated off-roster `old-shed` -> "
|
||||||
|
f"{[(i.name, i.sign_wd.tripped) for i in _both.inverters]}")
|
||||||
|
check("⭐ A STRANDED `inverter-a` LATCH IS STILL REPAIRED WHEN AN UNRELATED "
|
||||||
|
"OFF-ROSTER LATCH SITS BESIDE IT. One shared flag let `old-shed` veto the "
|
||||||
|
"repair, and the site came up driving on a file that says it is latched",
|
||||||
|
_both.inverters[0].sign_wd.tripped)
|
||||||
|
check("...and the repair still lands on the roster's first unit, not on the "
|
||||||
|
"removed one",
|
||||||
|
_both_store.data.get(unit_key(_C2_NAMES[0], "sign_tripped")) is True)
|
||||||
|
check("⚠️ ...and the reason carried across is the FLAT one - the misdirected "
|
||||||
|
"latch's own - and not `old-shed`'s, which belongs to a different "
|
||||||
|
"physical inverter",
|
||||||
|
"old-shed" not in _both.inverters[0].sign_wd.reason)
|
||||||
|
|
||||||
|
# ⚠️ AND THE OPERATOR MESSAGE IS ASSERTED, because review deleted it whole
|
||||||
|
# (`if stranded:` -> `if False:`) and 28/28 stayed green. `stranded` has no
|
||||||
|
# other consumer, so the list existed only to print a line nothing checked -
|
||||||
|
# and the line is the entire operator-facing half of this repair: it names the
|
||||||
|
# key that was stranded and the roster it was measured against. A repair an
|
||||||
|
# operator cannot see is a repair they cannot confirm.
|
||||||
|
_log_store = MemStore({
|
||||||
|
"sign_tripped": True,
|
||||||
|
"sign_reason": "SIGN_DISAGREEMENT: SYNTHETIC FIXTURE, not measured",
|
||||||
|
unit_key(UNIT_A, "sign_tripped"): True,
|
||||||
|
unit_key(UNIT_A, "sign_reason"): "SIGN_DISAGREEMENT: SYNTHETIC FIXTURE, not measured",
|
||||||
|
SCHEMA_KEY: SCHEMA_VERSION})
|
||||||
|
_records = []
|
||||||
|
_h = logging.Handler()
|
||||||
|
_h.emit = lambda r: _records.append(r.getMessage())
|
||||||
|
logging.getLogger("goodwe").addHandler(_h)
|
||||||
|
try:
|
||||||
|
_logged, _ = controller(NAMED_OPTS, store=_log_store)
|
||||||
|
finally:
|
||||||
|
logging.getLogger("goodwe").removeHandler(_h)
|
||||||
|
_s44_lines = [m for m in _records if "SAFETY-44" in m]
|
||||||
|
print(f" SAFETY-44 operator lines logged: {len(_s44_lines)}")
|
||||||
|
check("⭐ the repair SAYS SO, at ERROR, naming SAFETY-44 - deleting this line "
|
||||||
|
"left the whole suite green, and it is how an operator learns their "
|
||||||
|
"latch was re-armed rather than never lost",
|
||||||
|
len(_s44_lines) == 1)
|
||||||
|
check("...and it names the STRANDED KEY, so the operator can find it in the "
|
||||||
|
"state file rather than being told something happened",
|
||||||
|
any(unit_key(UNIT_A, "sign_tripped") in m for m in _s44_lines))
|
||||||
|
check("...and the ROSTER it was measured against, which is the half that says "
|
||||||
|
"WHY no machine read it",
|
||||||
|
any("garage" in m and "shed" in m for m in _s44_lines))
|
||||||
|
|
||||||
|
# ⭐ AND `misdirected` IS MEASURED AT THE FUNCTION'S OWN LEVEL, BECAUSE IT IS
|
||||||
|
# UNREACHABLE FROM THE CALL SITE. `Controller.__init__` passes
|
||||||
|
# `roster_names(opts)[0]`, which `build_roster()` also names its first entry
|
||||||
|
# from, so `unit` is a member of `roster` by construction and the refusal can
|
||||||
|
# never fire there. Review measured that: dropping the `roster` argument from
|
||||||
|
# `_verify_moves()` inside `migrate_unit_keys()` survives 28/28. The argument is
|
||||||
|
# kept for the reason `_verify_moves()` gives for its own `roster=None` branch -
|
||||||
|
# it is what a NEW caller gets by default - and a guard nothing exercises is a
|
||||||
|
# guard nobody knows works, so it is exercised HERE rather than claimed.
|
||||||
|
_mis_store = MemStore({"sign_tripped": True,
|
||||||
|
"sign_reason": "SIGN_DISAGREEMENT: SYNTHETIC FIXTURE, not measured"})
|
||||||
|
_mis = migrate_unit_keys(_mis_store, UNIT_A, ["garage", "shed"])
|
||||||
|
print(f" migrate_unit_keys(store, 'inverter-a', ['garage','shed']) -> "
|
||||||
|
f"ok={_mis.ok} misdirected={sorted(_mis.misdirected)}")
|
||||||
|
check("⭐ a WRONG `unit` - one that is not on the roster it was handed - comes "
|
||||||
|
"back `misdirected`, which is the refusal `migrate_maint_keys()` already "
|
||||||
|
"had and this one did not",
|
||||||
|
bool(_mis.misdirected) and not _mis.ok)
|
||||||
|
check("...and the schema marker is NOT stamped, so the migration runs again on "
|
||||||
|
"the next start rather than being recorded as finished - the one thing "
|
||||||
|
"store.py's header says must never happen",
|
||||||
|
_mis_store.data.get(SCHEMA_KEY) != SCHEMA_VERSION)
|
||||||
|
check("⚠️ ...while the SAME call with a unit that IS on the roster stamps the "
|
||||||
|
"marker - so the refusal is caused by the name and not by the fixture",
|
||||||
|
migrate_unit_keys(
|
||||||
|
MemStore({"sign_tripped": True, "sign_reason": "x"}),
|
||||||
|
"garage", ["garage", "shed"]).ok)
|
||||||
|
|
||||||
# The counterfactual, and it is the accepted baseline habit 7 asks for: the
|
# The counterfactual, and it is the accepted baseline habit 7 asks for: the
|
||||||
# same store, the same migration, one thing different - the roster shape.
|
# same store, the same migration, one thing different - the roster shape.
|
||||||
_flat_lat_store = MemStore({"sign_tripped": True,
|
_flat_lat_store = MemStore({"sign_tripped": True,
|
||||||
"sign_reason": "SIGN_DISAGREEMENT: measured pre-DEV-02"})
|
"sign_reason": "SIGN_DISAGREEMENT: SYNTHETIC FIXTURE, not measured"})
|
||||||
_flat_lat, _ = controller(FLAT_OPTS, store=_flat_lat_store)
|
_flat_lat, _ = controller(FLAT_OPTS, store=_flat_lat_store)
|
||||||
print(f" the same store on the FLAT config -> roster "
|
print(f" the same store on the FLAT config -> roster "
|
||||||
f"{[(i.name, i.sign_wd.tripped) for i in _flat_lat.inverters]}")
|
f"{[(i.name, i.sign_wd.tripped) for i in _flat_lat.inverters]}")
|
||||||
|
|||||||
@@ -59,6 +59,7 @@ Read MUTATION.md before you report a survivor.
|
|||||||
|
|
||||||
import ast
|
import ast
|
||||||
import inspect
|
import inspect
|
||||||
|
import itertools
|
||||||
import os
|
import os
|
||||||
import sys
|
import sys
|
||||||
from datetime import datetime, timedelta, timezone
|
from datetime import datetime, timedelta, timezone
|
||||||
@@ -272,16 +273,19 @@ check("the provenance audit REFUSES a carried hour flagged non-provisional - "
|
|||||||
"the 'stale but plausible' point, which is SAFETY-18's frozen-but-fresh "
|
"the 'stale but plausible' point, which is SAFETY-18's frozen-but-fresh "
|
||||||
"meter one layer up",
|
"meter one layer up",
|
||||||
raises(lambda: _fc._audit([_stale_copy], {}), ValueError))
|
raises(lambda: _fc._audit([_stale_copy], {}), ValueError))
|
||||||
|
# ⚠️ `published` MAPS AN HOUR TO A TUPLE, not to a point (FCST-04 H1). An hour
|
||||||
|
# may legitimately hold four published points - see the quarter-hourly block
|
||||||
|
# below - so the audit takes the whole set and checks membership by identity.
|
||||||
check("...and refuses a published hour that arrived as a COPY rather than as "
|
check("...and refuses a published hour that arrived as a COPY rather than as "
|
||||||
"the published object, which is where a flag gets lost",
|
"the published object, which is where a flag gets lost",
|
||||||
raises(lambda: _fc._audit(
|
raises(lambda: _fc._audit(
|
||||||
[PricePoint(start=TODAY_PART.points[0].start,
|
[PricePoint(start=TODAY_PART.points[0].start,
|
||||||
eur_per_kwh=TODAY_PART.points[0].eur_per_kwh,
|
eur_per_kwh=TODAY_PART.points[0].eur_per_kwh,
|
||||||
provisional=TODAY_PART.points[0].provisional)],
|
provisional=TODAY_PART.points[0].provisional)],
|
||||||
{D0: TODAY_PART.points[0]}), ValueError))
|
{D0: (TODAY_PART.points[0],)}), ValueError))
|
||||||
check("...and never repairs a flag: the audit's only outcomes are silence and "
|
check("...and never repairs a flag: the audit's only outcomes are silence and "
|
||||||
"an exception, so a series it passed was already correct",
|
"an exception, so a series it passed was already correct",
|
||||||
_fc._audit(list(FILLED.points), {u: TODAY_PART.points[i]
|
_fc._audit(list(FILLED.points), {u: (TODAY_PART.points[i],)
|
||||||
for i, u in enumerate(DAY[:PUBLISHED_N])})
|
for i, u in enumerate(DAY[:PUBLISHED_N])})
|
||||||
is None)
|
is None)
|
||||||
|
|
||||||
@@ -298,6 +302,231 @@ check("a PUBLISHED point that the feed itself flagged provisional keeps that "
|
|||||||
and all(p.provisional for p in _eddy_filled.points[:2])
|
and all(p.provisional for p in _eddy_filled.points[:2])
|
||||||
and _eddy_filled.points[0] is _eddy.points[0])
|
and _eddy_filled.points[0] is _eddy.points[0])
|
||||||
|
|
||||||
|
# ⚠️ AND A SUB-HOURLY FEED IS NOT DECIMATED TO ONE POINT AN HOUR (FCST-04 H1).
|
||||||
|
# `eneco_eddy` settles per QUARTER and `_validate_raw()` accepts any
|
||||||
|
# strictly-increasing spacing, so a published hour can hold four points. The
|
||||||
|
# fill walks WHOLE HOURS to decide what needs carrying; that must not become a
|
||||||
|
# rule about what a published hour may contain. Measured on FCST-04's first
|
||||||
|
# draft, which keyed the published set `{_utc_hour(p.start): p ...}`: a
|
||||||
|
# 96-point feed came out 15 points long and its arbitrage spread fell from
|
||||||
|
# 0.143 to 0.014 EUR/kWh, on an ALL-FIRM, fully-published series.
|
||||||
|
_QN = 16 # four hours of quarters
|
||||||
|
_QUARTERS = PriceRead.series(
|
||||||
|
[PricePoint(start=brussels(D0 + timedelta(minutes=15 * n)),
|
||||||
|
eur_per_kwh=round(0.100000 + 0.011300 * n, 6),
|
||||||
|
provisional=False)
|
||||||
|
for n in range(_QN)],
|
||||||
|
PriceProduct.REALTIME, "eneco_eddy", D0)
|
||||||
|
_QHOURS = {_fc._utc_hour(p.start) for p in _QUARTERS.points}
|
||||||
|
check(f"the quarter-hourly fixture really is sub-hourly - {_QN} points across "
|
||||||
|
f"{len(_QHOURS)} hours - or every check below passes for an hourly feed "
|
||||||
|
f"too and grades nothing",
|
||||||
|
len(_QUARTERS.points) == _QN and len(_QHOURS) == _QN // 4)
|
||||||
|
_qfill = derived(lambda: fill_from_yesterday(_QUARTERS, yesterday=YESTERDAY,
|
||||||
|
since=D0, until=D1, stamp=D0))
|
||||||
|
_qfirm = derived(lambda: [p for p in _qfill.points if not p.provisional])
|
||||||
|
check(f"every one of the {_QN} published points survives the fill BY IDENTITY "
|
||||||
|
f"({len(_qfirm)} kept). One-per-hour keying would leave {_QN // 4}, and "
|
||||||
|
f"the spread over the hours the feed actually published would collapse "
|
||||||
|
f"with them",
|
||||||
|
len(_qfirm) == _QN
|
||||||
|
and all(any(p is q for q in _QUARTERS.points) for p in _qfirm))
|
||||||
|
check(f"...and the {len(DAY) - len(_QHOURS)} whole hours it published NOTHING "
|
||||||
|
f"for are carried, ONE point each: a carried hour is hourly whatever the "
|
||||||
|
f"feed's spacing is, because yesterday's profile is only known per "
|
||||||
|
f"wall-clock hour",
|
||||||
|
_qfill.kind == "series"
|
||||||
|
and len([p for p in _qfill.points if p.provisional])
|
||||||
|
== len(DAY) - len(_QHOURS))
|
||||||
|
check("...and the filled series is still ordered by instant across the two "
|
||||||
|
"kinds, so `at()` and `window()` are not reading a shuffled list",
|
||||||
|
_qfill.kind == "series"
|
||||||
|
and [p.start.astimezone(UTC) for p in _qfill.points]
|
||||||
|
== sorted(p.start.astimezone(UTC) for p in _qfill.points))
|
||||||
|
|
||||||
|
_PUB_Q = _fc._published_by_hour(_QUARTERS)
|
||||||
|
# ⚠️ `derived`, NOT A RAW CALL. Under the decimating mutation this check
|
||||||
|
# exists for, `_audit` RAISES here - and a harness that dies on the mutation it
|
||||||
|
# is grading reports a traceback instead of a FAIL, which MUTATION.md forbids.
|
||||||
|
check("the audit ACCEPTS four published points inside one hour, which is the "
|
||||||
|
"case the repeated-hour COUNT it replaced could not express",
|
||||||
|
derived(lambda: _fc._audit(list(_QUARTERS.points), _PUB_Q)) is None)
|
||||||
|
check("...and still REFUSES a repeated CARRIED hour, which is the DST "
|
||||||
|
"mispricing that count existed to catch",
|
||||||
|
raises(lambda: _fc._audit([_fc._carried(D0, 0.11),
|
||||||
|
_fc._carried(D0, 0.12)], {}), ValueError))
|
||||||
|
check("...and STILL refuses the SAME published point emitted twice - the "
|
||||||
|
"provider's points are strictly increasing, so a repeat is never a "
|
||||||
|
"second print. The old hour COUNT already refused this (one point twice "
|
||||||
|
"is one hour twice); what is graded here is that the relaxation did not "
|
||||||
|
"lose it, not that it added it",
|
||||||
|
raises(lambda: _fc._audit([_QUARTERS.points[0], _QUARTERS.points[0]],
|
||||||
|
_PUB_Q), ValueError))
|
||||||
|
check("...and STILL refuses a CARRIED point standing in an hour the feed DID "
|
||||||
|
"publish. The old COUNT never looked at it; the old IDENTITY rule did, "
|
||||||
|
"and raised - so this is preserved behaviour, not new strictness. "
|
||||||
|
"FCST-04's text called it new here and in `_audit`'s docstring, and a "
|
||||||
|
"measured differential says otherwise",
|
||||||
|
raises(lambda: _fc._audit([_fc._carried(D0, 0.11)], _PUB_Q), ValueError))
|
||||||
|
|
||||||
|
# ⚠️ AND WHAT THE AUDIT DOES NOT DO, STATED WHERE A READER OF THESE THREE
|
||||||
|
# CHECKS WOULD OTHERWISE ASSUME IT (FCST-05, MUTATION.md habit 16). `_audit` is
|
||||||
|
# a PROVENANCE post-condition, not a COMPLETENESS one: it grades every point
|
||||||
|
# that IS emitted and says nothing about points that are not. So it accepts the
|
||||||
|
# very decimation FCST-04 was raised for. That is not a regression - FCST-03's
|
||||||
|
# audit was equally blind - but it means the audit is NOT the thing that stops
|
||||||
|
# H1 coming back. The count and identity checks above this line are.
|
||||||
|
check("the audit ACCEPTS the H1 decimation itself - four published points in "
|
||||||
|
"one hour, only the last emitted - and accepts an EMPTY output. It is a "
|
||||||
|
"provenance check, not a completeness one, and the checks that guard "
|
||||||
|
"provenance completeness are the COUNT and IDENTITY ones above",
|
||||||
|
derived(lambda: _fc._audit([_QUARTERS.points[3]], _PUB_Q)) is None
|
||||||
|
and derived(lambda: _fc._audit([], _PUB_Q)) is None)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# ⚠️ AND THE CLAIM "STRICTER" IS ITSELF A CHECK NOW (FCST-05).
|
||||||
|
#
|
||||||
|
# FCST-04 rewrote `_audit` and wrote, in the module and in the three checks
|
||||||
|
# above, that the result was STRICTER than FCST-03's. It is not. Two of the
|
||||||
|
# stated reasons were false - the old `seen` count already refused the same
|
||||||
|
# point emitted twice, and the old IDENTITY rule already refused a carried
|
||||||
|
# point in a published hour - and a right decision with a fabricated reason is
|
||||||
|
# a defect in this repository (MUTATION.md, and the reason DECISIONS files
|
||||||
|
# exist). So the claim is measured here rather than asserted in prose:
|
||||||
|
# FCST-03's audit is reproduced below from dd40c56 - its RULES verbatim, its
|
||||||
|
# message TEXT shortened, and nothing here reads a message - and the two are
|
||||||
|
# run over the whole domain `_audit` can express, in BOTH directions.
|
||||||
|
#
|
||||||
|
# THE DOMAIN, AND ITS SIZE. Every published mask over 3 hours (2**3) crossed
|
||||||
|
# with every output series of length 0-3 drawn from 3 hours x 5 point kinds.
|
||||||
|
# The nominal cross-product is 28 928; three of those kinds - published,
|
||||||
|
# published-provisional, copy-of-published - DO NOT EXIST for an hour the feed
|
||||||
|
# did not publish, and those combinations are SKIPPED rather than faked, which
|
||||||
|
# leaves 11 990 realisable comparisons. A fabricated "published point" for an
|
||||||
|
# unpublished hour is not in either audit's domain, and inventing one would
|
||||||
|
# grade the fixture rather than the code.
|
||||||
|
#
|
||||||
|
# ⚠️ THE OLD COPY IS A LITERAL, NOT A CALL. MUTATION.md habit 2: a differential
|
||||||
|
# whose two sides both come from the shipped module compares a thing against
|
||||||
|
# itself and moves with every edit. This side is frozen text.
|
||||||
|
def _audit_fcst03(points, published):
|
||||||
|
"""FCST-03's `_audit`, verbatim from dd40c56. Its `published` maps a UTC
|
||||||
|
hour to ONE point, which is exactly the expressiveness this ticket relaxed.
|
||||||
|
"""
|
||||||
|
seen = set()
|
||||||
|
for p in points:
|
||||||
|
u = _fc._utc_hour(p.start)
|
||||||
|
if u in seen:
|
||||||
|
raise ValueError("the filled series repeats " + u.isoformat())
|
||||||
|
seen.add(u)
|
||||||
|
src = published.get(u)
|
||||||
|
if src is None:
|
||||||
|
if p.provisional is not True:
|
||||||
|
raise ValueError(u.isoformat() + " is carried and is flagged "
|
||||||
|
"provisional=" + repr(p.provisional))
|
||||||
|
else:
|
||||||
|
if p is not src:
|
||||||
|
raise ValueError(u.isoformat() + " is published but the series "
|
||||||
|
"carries a COPY of that point")
|
||||||
|
|
||||||
|
|
||||||
|
def _verdict(fn, pts, pub):
|
||||||
|
try:
|
||||||
|
fn(list(pts), pub)
|
||||||
|
return "accept"
|
||||||
|
except ValueError:
|
||||||
|
return "raise"
|
||||||
|
|
||||||
|
|
||||||
|
_DH = [D0 + timedelta(hours=6 + i) for i in range(3)]
|
||||||
|
_weaker, _stronger, _compared = [], [], 0
|
||||||
|
for _mask in itertools.product((0, 1), repeat=3):
|
||||||
|
_pub = {u: PricePoint(start=u, eur_per_kwh=0.10 + i, provisional=False)
|
||||||
|
for i, (u, _on) in enumerate(zip(_DH, _mask)) if _on}
|
||||||
|
_old_pub = dict(_pub)
|
||||||
|
_new_pub = {u: (q,) for u, q in _pub.items()}
|
||||||
|
# The five shapes an output point can take at a given hour. Three of them
|
||||||
|
# do not exist for an hour the feed did not publish, and those combinations
|
||||||
|
# are skipped rather than faked - a fabricated "published point" for an
|
||||||
|
# unpublished hour is not in either audit's domain.
|
||||||
|
_kinds = []
|
||||||
|
for _u in _DH:
|
||||||
|
_src = _pub.get(_u)
|
||||||
|
_kinds.append(("published", _src))
|
||||||
|
_kinds.append(("published-provisional",
|
||||||
|
None if _src is None else
|
||||||
|
PricePoint(start=_src.start,
|
||||||
|
eur_per_kwh=_src.eur_per_kwh,
|
||||||
|
provisional=True)))
|
||||||
|
_kinds.append(("carried", _fc._carried(_u, 0.5)))
|
||||||
|
_kinds.append(("carried-unflagged",
|
||||||
|
PricePoint(start=_u, eur_per_kwh=0.5,
|
||||||
|
provisional=False)))
|
||||||
|
_kinds.append(("copy-of-published",
|
||||||
|
None if _src is None else
|
||||||
|
PricePoint(start=_src.start,
|
||||||
|
eur_per_kwh=_src.eur_per_kwh,
|
||||||
|
provisional=_src.provisional)))
|
||||||
|
for _n in range(4):
|
||||||
|
for _combo in itertools.product(_kinds, repeat=_n):
|
||||||
|
if any(k[1] is None for k in _combo):
|
||||||
|
continue
|
||||||
|
_pts = [k[1] for k in _combo]
|
||||||
|
_compared += 1
|
||||||
|
_o = _verdict(_audit_fcst03, _pts, _old_pub)
|
||||||
|
_nw = _verdict(_fc._audit, _pts, _new_pub)
|
||||||
|
if _o == "raise" and _nw == "accept":
|
||||||
|
_weaker.append((_mask, [k[0] for k in _combo]))
|
||||||
|
if _o == "accept" and _nw == "raise":
|
||||||
|
_stronger.append((_mask, [k[0] for k in _combo]))
|
||||||
|
|
||||||
|
# ⚠️ THE PRECONDITION FIRST (MUTATION.md habit 1). A differential over a
|
||||||
|
# domain that cannot reach the two disputed cases would score 0-0 for the wrong
|
||||||
|
# reason and read exactly like this one, so the two cases are driven against
|
||||||
|
# the OLD audit by name before the totals are graded.
|
||||||
|
_probe = PricePoint(start=_DH[0], eur_per_kwh=0.1, provisional=False)
|
||||||
|
check(f"the differential really covers the two claims it is here to settle, "
|
||||||
|
f"or it is {_compared} runs of nothing: FCST-03's own audit raises on "
|
||||||
|
f"BOTH 'the same published point twice' and 'a carried point in a "
|
||||||
|
f"published hour', which is what FCST-04 said it did not",
|
||||||
|
_compared > 3000
|
||||||
|
and _verdict(_audit_fcst03, [_probe, _probe], {_DH[0]: _probe}) == "raise"
|
||||||
|
and _verdict(_audit_fcst03, [_fc._carried(_DH[0], 0.5)],
|
||||||
|
{_DH[0]: _probe}) == "raise")
|
||||||
|
check(f"...and over all {_compared} of them the reworked audit is NOT WEAKER "
|
||||||
|
f"than FCST-03's: {len(_weaker)} case(s) the old one raised and this one "
|
||||||
|
f"accepts {_weaker[:2]}",
|
||||||
|
_weaker == [])
|
||||||
|
check(f"...and NOT STRONGER either: {len(_stronger)} case(s) the old one "
|
||||||
|
f"accepted and this one raises {_stronger[:2]}. FCST-04's docstring and "
|
||||||
|
f"two of these checks said it was stricter and named two reasons, and "
|
||||||
|
f"BOTH were already refused by the old count and the old identity rule. "
|
||||||
|
f"The only behavioural change is the RELAXATION below, which is what the "
|
||||||
|
f"rework was for",
|
||||||
|
_stronger == [])
|
||||||
|
_QP = [PricePoint(start=_DH[0] + timedelta(minutes=15 * i),
|
||||||
|
eur_per_kwh=0.1 + i, provisional=False) for i in range(4)]
|
||||||
|
check("...and the relaxation is real and is NOT expressible to the old audit: "
|
||||||
|
"four distinct published points inside one hour, which FCST-03's count "
|
||||||
|
"refused as a repeated hour and this one accepts",
|
||||||
|
_verdict(_fc._audit, _QP, {_DH[0]: tuple(_QP)}) == "accept"
|
||||||
|
and _verdict(_audit_fcst03, _QP, {_DH[0]: _QP[0]}) == "raise")
|
||||||
|
|
||||||
|
# ⚠️ THE HORIZON IS WHOLE HOURS AT BOTH ENDS (FCST-04 L8). `since` is floored
|
||||||
|
# and `until` was not, so a 48 h request made at any instant off the hour came
|
||||||
|
# back 49 hours long. Graded at an OFF-THE-HOUR instant, because on the hour
|
||||||
|
# the two implementations agree and the check would grade nothing.
|
||||||
|
_off = D0 + timedelta(hours=8, minutes=17, seconds=43)
|
||||||
|
for _h in (1, 12, 48):
|
||||||
|
_n = derived(lambda h=_h: len(fill_from_yesterday(
|
||||||
|
TODAY_PART, yesterday=YESTERDAY, since=_off,
|
||||||
|
until=_off + timedelta(hours=h), stamp=_off).points))
|
||||||
|
check(f"a {_h} h horizon asked for at {_off.isoformat()} - 17 min 43 s "
|
||||||
|
f"past the hour, the shape a running controller always has - is "
|
||||||
|
f"{_h} points long, not {_h + 1}",
|
||||||
|
_n == _h)
|
||||||
|
|
||||||
# The structural half: the constructor has nothing to pass, and the file is
|
# The structural half: the constructor has nothing to pass, and the file is
|
||||||
# scanned so the pass-through idiom cannot come back looking ordinary.
|
# scanned so the pass-through idiom cannot come back looking ordinary.
|
||||||
check("_carried() has NO `provisional` parameter, so a caller has nothing to "
|
check("_carried() has NO `provisional` parameter, so a caller has nothing to "
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -69,6 +69,7 @@ from app.price import (
|
|||||||
Absence, PricePoint, PriceProduct, PriceRead, arbitrage_allowed, brussels,
|
Absence, PricePoint, PriceProduct, PriceRead, arbitrage_allowed, brussels,
|
||||||
plan_is_provisional, required_spread, uncertainty_of,
|
plan_is_provisional, required_spread, uncertainty_of,
|
||||||
)
|
)
|
||||||
|
from app.price_history import NoHistory
|
||||||
from app.static_tariff import (
|
from app.static_tariff import (
|
||||||
MEAN_WINDOW_DAYS, ProfileInvalid, StaticProfile, TariffPlan,
|
MEAN_WINDOW_DAYS, ProfileInvalid, StaticProfile, TariffPlan,
|
||||||
parse_static_profile, resolve_tariff, static_plan, static_status,
|
parse_static_profile, resolve_tariff, static_plan, static_status,
|
||||||
@@ -746,7 +747,8 @@ _cfg = opts(TWO_PERIOD, price_provider="generic_dayahead",
|
|||||||
price_feed_url="https://api.energy-charts.info/price?bzn=BE",
|
price_feed_url="https://api.energy-charts.info/price?bzn=BE",
|
||||||
price_timeout_s=1, price_markup_eur_kwh=0.03,
|
price_timeout_s=1, price_markup_eur_kwh=0.03,
|
||||||
price_taxes_eur_kwh=0.06)
|
price_taxes_eur_kwh=0.06)
|
||||||
_fb = run(resolve_tariff(_cfg, THU, THU + timedelta(days=1), transport=_dead))
|
_fb = run(resolve_tariff(_cfg, THU, THU + timedelta(days=1), transport=_dead,
|
||||||
|
history=NoHistory()))
|
||||||
check("a dead feed falls back to the static profile rather than to nothing",
|
check("a dead feed falls back to the static profile rather than to nothing",
|
||||||
_fb.origin == "fallback" and _fb.has_prices)
|
_fb.origin == "fallback" and _fb.has_prices)
|
||||||
check("...with the SAME prices the configured-product path produces, because "
|
check("...with the SAME prices the configured-product path produces, because "
|
||||||
@@ -769,7 +771,8 @@ check("...and the provisional plan has no cheap window either: §8.3 forbids an
|
|||||||
not any(_fb.cheap_now(at_local(THU, h)) for h in range(24)))
|
not any(_fb.cheap_now(at_local(THU, h)) for h in range(24)))
|
||||||
_fb_none = run(resolve_tariff(opts(None, price_provider="generic_dayahead",
|
_fb_none = run(resolve_tariff(opts(None, price_provider="generic_dayahead",
|
||||||
price_timeout_s=1),
|
price_timeout_s=1),
|
||||||
THU, THU + timedelta(days=1), transport=_dead))
|
THU, THU + timedelta(days=1), transport=_dead,
|
||||||
|
history=NoHistory()))
|
||||||
check("a dead feed with NO static_profile has nothing to fall back to, and "
|
check("a dead feed with NO static_profile has nothing to fall back to, and "
|
||||||
"says that rather than inventing a tariff",
|
"says that rather than inventing a tariff",
|
||||||
_fb_none.origin == "none" and not _fb_none.has_prices)
|
_fb_none.origin == "none" and not _fb_none.has_prices)
|
||||||
@@ -785,7 +788,7 @@ async def _live(url, timeout_s):
|
|||||||
|
|
||||||
|
|
||||||
_live_plan = run(resolve_tariff(_cfg, THU, THU + timedelta(days=1),
|
_live_plan = run(resolve_tariff(_cfg, THU, THU + timedelta(days=1),
|
||||||
transport=_live))
|
transport=_live, history=NoHistory()))
|
||||||
check("a live feed is used in preference to the static profile",
|
check("a live feed is used in preference to the static profile",
|
||||||
_live_plan.origin == "provider"
|
_live_plan.origin == "provider"
|
||||||
and _live_plan.product is PriceProduct.DAY_AHEAD)
|
and _live_plan.product is PriceProduct.DAY_AHEAD)
|
||||||
|
|||||||
@@ -722,18 +722,41 @@ print(f" e.g. {_sustained.reason!r}")
|
|||||||
|
|
||||||
print()
|
print()
|
||||||
print(" ⚠️ AND NOT ON THE TRANSIENT, WHICH IS THE FALSE-ALARM CASE")
|
print(" ⚠️ AND NOT ON THE TRANSIENT, WHICH IS THE FALSE-ALARM CASE")
|
||||||
# ⚠️ THE SHAPES BELOW ARE THE RIG'S, NOT INVENTED. Measured on TEST-19 over 924
|
# THE SHAPE BELOW IS THE RIG'S: the serialised write path means that on a
|
||||||
# closed-loop runs in which every unit executed the share it was given: the
|
# reversing cycle B's new number lands up to a whole control cycle after A's,
|
||||||
# worst continuous opposition above 700 W is 5.0 s - ONE control cycle - at a
|
# both slew through zero behind a real ramp and a real lag, and between the two
|
||||||
# peak of 1700 W. That is the serialised write path: on a reversing cycle B's
|
# crossings they genuinely oppose. A guard that trips on that is a false-alarm
|
||||||
# new number lands up to a whole cycle after A's, both slew through zero at
|
# generator, and this project's doctrine on crying wolf is explicit (SAFETY-15,
|
||||||
# 800 W/s behind a 10 s lag, and between the two crossings they genuinely
|
# SAFETY-19).
|
||||||
# oppose. A guard that trips on that is a false-alarm generator, and this
|
#
|
||||||
# project's doctrine on crying wolf is explicit (SAFETY-15, SAFETY-19).
|
# ⚠️ THE MAGNITUDE IS A FIXTURE INPUT, NOT A MEASUREMENT, AND THIS COMMENT USED
|
||||||
|
# TO SAY OTHERWISE. It read "the worst single transient the rig can produce -
|
||||||
|
# 1700 W", sourced to a 924-run campaign whose harness IS NOT COMMITTED IN
|
||||||
|
# EITHER REPOSITORY (DECISIONS/SAFETY-31-circulating-power.md §3 says so in its
|
||||||
|
# own words). Three sweeps have now been run against that claim and no two agree:
|
||||||
|
#
|
||||||
|
# 924-run campaign, harness uncommitted peak 1700 W, 505.0 s at 100 W
|
||||||
|
# independent reviewer sweep, 2026-09-07 peak 2750 W, 15.0 s above 700 W
|
||||||
|
# goodwe `two_master_circulation`, 840 runs peak 1048 W, 5.0 s above 700 W
|
||||||
|
#
|
||||||
|
# ⚠️ THAT IS NOT EVIDENCE THAT ANY OF THEM IS WRONG. The three swept different
|
||||||
|
# grids, and the last one measured WHY the spread is so wide: on its own grid the
|
||||||
|
# worst legitimate peak is 1048 W on the modelled plant, 2250 W on `INSTANT`, and
|
||||||
|
# 0 W on `INSTANT` with no write stagger reaching a whole cycle. The plant model
|
||||||
|
# and the stagger set are TERMS of the number, and a worst case quoted without
|
||||||
|
# them is not a measurement. It is evidence that the claim cannot be adjudicated
|
||||||
|
# while the instrument is missing.
|
||||||
|
#
|
||||||
|
# So what this check now says is what it can support: 1700 W is LARGER than
|
||||||
|
# anything any of the three sweeps produced legitimately, and one control cycle
|
||||||
|
# of it does not trip. The committed, re-runnable version of the claim is
|
||||||
|
# `admin/goodwe`'s `sim/inject.py two_master_circulation`, which runs on every
|
||||||
|
# pull request there and requires ZERO of ~840 non-faulty runs to trip.
|
||||||
_one_cycle = [0.0] * 5 + [1700.0] + [0.0] * 30
|
_one_cycle = [0.0] * 5 + [1700.0] + [0.0] * 30
|
||||||
_t1, _ = circ_drive(_one_cycle)
|
_t1, _ = circ_drive(_one_cycle)
|
||||||
check("the worst single transient the rig can produce - 1700 W for ONE control "
|
check("a single transient of 1700 W - larger than the worst peak any committed "
|
||||||
"cycle - does not trip", not _t1.tripped)
|
"sweep of the legitimate family has produced - lasting ONE control cycle "
|
||||||
|
"does not trip", not _t1.tripped)
|
||||||
_repeated = ([0.0] * 3 + [1700.0]) * 20
|
_repeated = ([0.0] * 3 + [1700.0]) * 20
|
||||||
_t2, _ = circ_drive(_repeated)
|
_t2, _ = circ_drive(_repeated)
|
||||||
check("...and neither does it REPEATED every fourth cycle for 400 s, which is "
|
check("...and neither does it REPEATED every fourth cycle for 400 s, which is "
|
||||||
@@ -1091,6 +1114,158 @@ check("...and it does not raise the same card on every cycle - a per-PROCESS "
|
|||||||
"this' a different question from 'is this site tripped'",
|
"this' a different question from 'is this site tripped'",
|
||||||
len(_notes2) == 1)
|
len(_notes2) == 1)
|
||||||
|
|
||||||
|
|
||||||
|
print()
|
||||||
|
print("6a. AC 4: THE ALARM NAMES WHICH PACKS, AND WHICH WAY ROUND")
|
||||||
|
# ⚠️ THIS BLOCK EXISTS BECAUSE A MUTATION SURVIVED ALL 26 FILES. Replacing
|
||||||
|
#
|
||||||
|
# pos, neg = opposing([inv.batt for inv in self.inverters],
|
||||||
|
# min_w=self.circ_wd.cfg.min_w / 2.0)
|
||||||
|
#
|
||||||
|
# in `check_circulating()` with `pos, neg = ([], [])` (-102 B) silently deletes
|
||||||
|
# the "Right now X is discharging while Y charges" clause from the notification,
|
||||||
|
# and the whole suite stayed GREEN. The block above asserts three tokens -
|
||||||
|
# METER CANNOT SEE, CONTROL HAS BEEN STOPPED, isolator - and NEVER an inverter
|
||||||
|
# name, so the one sentence `circulation.py` calls "the first thing somebody
|
||||||
|
# woken by this alarm needs" and "impossible to reconstruct from the number
|
||||||
|
# alone" was graded by nothing at all.
|
||||||
|
#
|
||||||
|
# ⚠️ IT IS NOT DEAD CODE, WHICH IS THE PART THAT MAKES IT A CHECK GAP RATHER
|
||||||
|
# THAN A REACHABILITY ONE. Ledger M18 - the `await self.check_circulating()`
|
||||||
|
# call removed - goes red on four checks above, so this function runs. What
|
||||||
|
# nothing asserted was the ATTRIBUTION.
|
||||||
|
#
|
||||||
|
# ⚠️ AND THE ROSTER IS ["garage", "shed", "cellar"], WHICH IS THE OTHER HALF OF
|
||||||
|
# THE FIX. Every fixture above runs on `inverter-a`/`inverter-b`;
|
||||||
|
# test_maintenance.py section 11 records what that costs - on a fixture whose
|
||||||
|
# names are the module's own constants, a check cannot tell the roster from the
|
||||||
|
# constant, and `UNIT_A` is not even what a nameless entry falls back to. Here
|
||||||
|
# the names are an operator's, the two participants pull OPPOSITE ways so the
|
||||||
|
# sentence is not symmetric, and a third pack rests at 5 W. That makes three
|
||||||
|
# DIFFERENT mutations - the clause dropped, the two directions swapped, and the
|
||||||
|
# `min_w` floor dropped - into three DIFFERENT red checks. A fixture naming both
|
||||||
|
# units the same, or one in which the direction did not matter, would see the
|
||||||
|
# first and neither of the others.
|
||||||
|
|
||||||
|
_CIRC_ROSTER = [
|
||||||
|
dict(name="garage", setpoint_entity="number.sp_a", soc_entity="sensor.soc_a",
|
||||||
|
batt_entity="sensor.batt_a", max_w=5000),
|
||||||
|
# ⚠️ B_RATING_HYPOTHESIS, and part 4's note above says why it is not 4600:
|
||||||
|
# inverter B does not physically exist and a datasheet is not a measurement.
|
||||||
|
dict(name="shed", setpoint_entity="number.sp_b", soc_entity="sensor.soc_b",
|
||||||
|
batt_entity="sensor.batt_b", max_w=B_RATING_HYPOTHESIS),
|
||||||
|
# THE BYSTANDER. A third pack resting at standby draw, on the roster, read
|
||||||
|
# every cycle, and NOT a participant in anybody's loop.
|
||||||
|
dict(name="cellar", setpoint_entity="number.sp_c", soc_entity="sensor.soc_c",
|
||||||
|
batt_entity="sensor.batt_c", max_w=B_RATING_HYPOTHESIS),
|
||||||
|
]
|
||||||
|
_CIRC_NAMES = [e["name"] for e in _CIRC_ROSTER]
|
||||||
|
_REST_W = 5.0
|
||||||
|
|
||||||
|
|
||||||
|
class _NamedCircHass(_Hass):
|
||||||
|
"""Three named packs; the meter still reads exactly zero.
|
||||||
|
|
||||||
|
Same house as `_CircHass` - grid = load - sum(B) with two units cancelling -
|
||||||
|
with one extra pack idling at standby draw so that "who is named" and "who
|
||||||
|
is merely on the roster" are different questions.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, a_w, b_w, c_w=_REST_W):
|
||||||
|
super().__init__()
|
||||||
|
self.vals = {"sensor.batt_a": a_w, "sensor.batt_b": b_w,
|
||||||
|
"sensor.batt_c": c_w}
|
||||||
|
|
||||||
|
async def number(self, entity, invert=False):
|
||||||
|
return {"sensor.grid": 0.0, "sensor.soc_a": 50.0,
|
||||||
|
"sensor.soc_b": 50.0, "sensor.soc_c": 50.0,
|
||||||
|
**self.vals}.get(entity)
|
||||||
|
|
||||||
|
|
||||||
|
def _named_trio(hass, store, *, hold_s=1, min_w=700):
|
||||||
|
opts = {"meter_entity": "sensor.grid", "meter_source": "off",
|
||||||
|
"max_w": 5000, "gain": 0.6, "auto_start": True,
|
||||||
|
"circulation_w": min_w, "circulation_hold_s": hold_s,
|
||||||
|
"inverters": _CIRC_ROSTER}
|
||||||
|
c = Controller(opts, hass, store, _P(), stop=asyncio.Event())
|
||||||
|
c.auto = True
|
||||||
|
for inv in c.inverters:
|
||||||
|
inv.dev_min, inv.dev_max = -5000.0, 5000.0
|
||||||
|
return c
|
||||||
|
|
||||||
|
|
||||||
|
async def _circ_named(a_w, b_w):
|
||||||
|
hass = _NamedCircHass(a_w, b_w)
|
||||||
|
c = _named_trio(hass, _MemStore())
|
||||||
|
for _ in range(14):
|
||||||
|
await c.read_inputs()
|
||||||
|
await c.cycle()
|
||||||
|
await asyncio.sleep(0.25)
|
||||||
|
notes = [x for x in hass.calls
|
||||||
|
if x[0] == "persistent_notification"
|
||||||
|
and x[2].get("notification_id") == "goodwe_circulating_power"]
|
||||||
|
return c, notes
|
||||||
|
|
||||||
|
|
||||||
|
_cg, _notes_g = asyncio.run(_circ_named(+3000.0, -3000.0))
|
||||||
|
_cs, _notes_s = asyncio.run(_circ_named(-3000.0, +3000.0))
|
||||||
|
_msg_g = _notes_g[0][2]["message"] if _notes_g else ""
|
||||||
|
_msg_s = _notes_s[0][2]["message"] if _notes_s else ""
|
||||||
|
print(f" roster : {_cg.units}")
|
||||||
|
print(f" garage discharging, measured: "
|
||||||
|
f"{dict(zip(_cg.units, [i.batt for i in _cg.inverters]))}")
|
||||||
|
print(f" -> {_msg_g.split(chr(10))[2] if _msg_g.count(chr(10)) > 2 else _msg_g!r}")
|
||||||
|
print(f" shed discharging, measured: "
|
||||||
|
f"{dict(zip(_cs.units, [i.batt for i in _cs.inverters]))}")
|
||||||
|
print(f" -> {_msg_s.split(chr(10))[2] if _msg_s.count(chr(10)) > 2 else _msg_s!r}")
|
||||||
|
|
||||||
|
# --- THE PRECONDITIONS, AS THEIR OWN CHECKS (MUTATION.md rule 1) ------------
|
||||||
|
check("PRECONDITION: the roster carries three DISTINCT operator-chosen names, "
|
||||||
|
f"none of them `UNIT_A` ({UNIT_A!r}) - so every claim below is a claim "
|
||||||
|
"about the roster and not about a constant that happens to equal it",
|
||||||
|
_cg.units == tuple(_CIRC_NAMES) and len(set(_CIRC_NAMES)) == 3
|
||||||
|
and UNIT_A not in _CIRC_NAMES)
|
||||||
|
check("PRECONDITION: the two runs are MIRROR IMAGES - same house, same meter at "
|
||||||
|
"0 W, same magnitudes, and the two participants swapped. Without this the "
|
||||||
|
"direction claim below could be satisfied by a sentence that ignores the "
|
||||||
|
"measurement entirely",
|
||||||
|
[i.batt for i in _cg.inverters] == [+3000.0, -3000.0, _REST_W]
|
||||||
|
and [i.batt for i in _cs.inverters] == [-3000.0, +3000.0, _REST_W]
|
||||||
|
and _cg.grid == 0.0 and _cs.grid == 0.0)
|
||||||
|
check("PRECONDITION: both runs actually tripped and actually raised the card, "
|
||||||
|
"so an absent sentence below is an absent sentence and not an absent alarm",
|
||||||
|
_cg.circ_wd.tripped and _cs.circ_wd.tripped
|
||||||
|
and len(_notes_g) == 1 and len(_notes_s) == 1)
|
||||||
|
|
||||||
|
# --- AC 4: THE ATTRIBUTION ITSELF -------------------------------------------
|
||||||
|
_WHO_G = "Right now garage is discharging while shed charges."
|
||||||
|
_WHO_S = "Right now shed is discharging while garage charges."
|
||||||
|
check("⚠️ THE ALARM NAMES THE TWO PACKS AND SAYS WHICH WAY EACH ONE IS PULLING. "
|
||||||
|
"`circulation.py` calls this the first thing somebody woken at 3 a.m. "
|
||||||
|
"needs and impossible to reconstruct from the watts alone, and until this "
|
||||||
|
"line the sentence could be deleted with all 26 files green",
|
||||||
|
_WHO_G in _msg_g)
|
||||||
|
check("⚠️ AND WITH THE TWO SWAPPED, THE SENTENCE SWAPS. This is the half a "
|
||||||
|
"single fixture cannot have: `pos` and `neg` exchanged, or a hard-coded "
|
||||||
|
"sentence, satisfies the check above and fails this one. The operator is "
|
||||||
|
"being told which pack to walk to",
|
||||||
|
_WHO_S in _msg_s)
|
||||||
|
check("...and each run carries ONLY its own sentence - the mirrored alarm does "
|
||||||
|
"not also contain the other direction, so the clause is derived from this "
|
||||||
|
"cycle's measurement rather than assembled from the roster",
|
||||||
|
_WHO_S not in _msg_g and _WHO_G not in _msg_s)
|
||||||
|
check("...and the RESTING pack is not named. `cellar` is on the roster, is read "
|
||||||
|
"every cycle and is measured at 5 W of standby draw - `opposing()` is "
|
||||||
|
"called with a floor of half the trip threshold precisely so a pack idling "
|
||||||
|
"beside a 6 kW loop is not reported as one of its two participants",
|
||||||
|
"cellar" not in _msg_g and "cellar" not in _msg_s
|
||||||
|
and _cg.inverters[2].batt == _REST_W)
|
||||||
|
check("...and the sentence sits with the figure, not instead of it: the same "
|
||||||
|
"card still carries the watts, the window and the three things only this "
|
||||||
|
"alarm can say",
|
||||||
|
all("CIRCULATING_POWER" in m and "METER CANNOT SEE" in m
|
||||||
|
and "CONTROL HAS BEEN STOPPED" in m and "isolator" in m
|
||||||
|
for m in (_msg_g, _msg_s)))
|
||||||
print()
|
print()
|
||||||
print(" the operator's press is the only way out, and it clears BOTH halves")
|
print(" the operator's press is the only way out, and it clears BOTH halves")
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,48 @@
|
|||||||
|
# WHAT THIS BUILD IS SUPPOSED TO ANNOUNCE, recorded so that a change to the
|
||||||
|
# composition has something to disagree with. THREE COLUMNS:
|
||||||
|
#
|
||||||
|
# <entity_id> <unique_id> <device identifier>
|
||||||
|
#
|
||||||
|
# ⚠️ ALL THREE, AND THE SECOND AND THIRD ARE THE HALF THE FIRST DRAFT MISSED.
|
||||||
|
# A reference that carries only entity_ids cannot see the two changes that
|
||||||
|
# orphan an install without changing a single composed id:
|
||||||
|
#
|
||||||
|
# `BASE` renamed (`goodwe_ctl` -> anything) every unique_id changes
|
||||||
|
# `DEVICE["identifiers"]` changed HA sees a NEW device
|
||||||
|
#
|
||||||
|
# Home Assistant's MQTT integration keys its entity registry on `unique_id` and
|
||||||
|
# its device registry on the device identifier. Change either and HA creates a
|
||||||
|
# NEW entity beside the old one - and because discovery is published with
|
||||||
|
# `retain=True` and nothing publishes a removal payload, the OLD config stays on
|
||||||
|
# the broker and the dead entity stays in the registry, still named by every
|
||||||
|
# automation that referenced it, never updating again. The composed entity_id is
|
||||||
|
# identical on both, so an entity_id-only comparison reports a clean transition.
|
||||||
|
# That is the same shape as the habit-4 this file's own card already caught
|
||||||
|
# once: a reference that does not carry the thing that moved.
|
||||||
|
#
|
||||||
|
# ⚠️ AND `object_id` IS DELIBERATELY NOT A COLUMN. `app/mqtt.py` lines 36-42 and
|
||||||
|
# 61-76 record that HA 2026.8 does NOT honour `object_id` in a discovery
|
||||||
|
# payload - the id is composed from device name plus entity name - so renaming
|
||||||
|
# one is a true no-op and a column for it would make this file refuse a change
|
||||||
|
# that costs nothing. The check is graded on what HA actually keys on.
|
||||||
|
#
|
||||||
|
# PROVENANCE, stated exactly:
|
||||||
|
# - recorded 2026-09-08 from `release/1.0` at `meter_source: firmware`,
|
||||||
|
# roster of one, by reading `mqtt.discovery_payloads(units=[])`. It is what
|
||||||
|
# the code announces.
|
||||||
|
# - `sensor.goodwe_p1_sample_age` is additionally recorded in `app/mqtt.py` as
|
||||||
|
# VERIFIED AGAINST THE LIVE SYSTEM on 2026-08-26, and the ESP32's ingest
|
||||||
|
# guard subscribes to that exact string.
|
||||||
|
# - ⚠️ IT IS NOT A DUMP OF THE LIVE INSTALL'S ENTITY REGISTRY. That is what
|
||||||
|
# `--live-entities` is for. This file is about THE BUILD.
|
||||||
|
#
|
||||||
|
# Editing this file is how you say "the announcement changed on purpose". Do not
|
||||||
|
# edit it to make a check pass.
|
||||||
|
sensor.goodwe_setpoint goodwe_ctl_setpoint goodwe_rs485_controller
|
||||||
|
sensor.goodwe_grid_power goodwe_ctl_grid goodwe_rs485_controller
|
||||||
|
sensor.goodwe_battery_power goodwe_ctl_battery goodwe_rs485_controller
|
||||||
|
sensor.goodwe_battery_soc goodwe_ctl_soc goodwe_rs485_controller
|
||||||
|
sensor.goodwe_circulating_power goodwe_ctl_circulation goodwe_rs485_controller
|
||||||
|
sensor.goodwe_maintenance_phase goodwe_ctl_phase goodwe_rs485_controller
|
||||||
|
sensor.goodwe_controller_status goodwe_ctl_status goodwe_rs485_controller
|
||||||
|
sensor.goodwe_p1_sample_age goodwe_ctl_p1_age goodwe_rs485_controller
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
# ARCH-02 fixture: THE SHAPE THE LIVE BOX IS IN TODAY (a flat, single-inverter
|
||||||
|
# config). ⚠️ ILLUSTRATIVE ENTITY NAMES, NOT THE LIVE ONES - the owner runs the
|
||||||
|
# rehearsal against a copy of the real options blob, and this file exists so
|
||||||
|
# that the harness has a synthetic transition it can be graded on.
|
||||||
|
setpoint_entity: number.goodwe_setpoint
|
||||||
|
soc_entity: sensor.goodwe_battery_soc_pct
|
||||||
|
batt_entity: sensor.goodwe_battery_power_w
|
||||||
|
batt_invert: false
|
||||||
|
max_w: 5000
|
||||||
|
bms_soc_floor: 10
|
||||||
|
meter_source: 'off'
|
||||||
|
meter_entity: sensor.p1_active_power
|
||||||
|
inverters: []
|
||||||
|
gain: 0.6
|
||||||
|
slew_w: 0
|
||||||
|
deadband_w: 15
|
||||||
|
target_grid_w: 0
|
||||||
|
step_w: 10
|
||||||
|
heartbeat_s: 10
|
||||||
|
stale_input_s: 15
|
||||||
|
maintenance_discharge_w: 2000
|
||||||
|
maintenance_charge_w: 2000
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
# THE ENTITY IDS THE INSTALL HAS — a stand-in for the owner's real export.
|
||||||
|
#
|
||||||
|
# ⚠️ THIS FILE AND `announced-reference.txt` ANSWER DIFFERENT QUESTIONS AND ARE
|
||||||
|
# NOT INTERCHANGEABLE:
|
||||||
|
#
|
||||||
|
# announced-reference.txt what the BUILD announces. Loaded on EVERY run.
|
||||||
|
# Three columns, because HA keys on `unique_id` and
|
||||||
|
# on the device identifier as well as on the
|
||||||
|
# composed entity_id.
|
||||||
|
# this file (--live-entities)
|
||||||
|
# what the INSTALL has. One id per line. Orphans
|
||||||
|
# are graded against it: an id the install has and
|
||||||
|
# the AFTER shape does not announce is an orphan
|
||||||
|
# whatever the build's own history says.
|
||||||
|
#
|
||||||
|
# ⚠️ OMITTING `--live-entities` IS A REFUSAL (exit 2), NOT A QUIETER RUN. It
|
||||||
|
# used to be the other way round - an unreadable file refused, a missing flag
|
||||||
|
# passed with exit 0 and said nothing about the half it had not checked - and
|
||||||
|
# the owner runs this ONCE, at commissioning, under time pressure.
|
||||||
|
# `--no-live-entities` is the deliberate opt-out and it prints a standing
|
||||||
|
# banner in the report header and again on the exit line.
|
||||||
|
#
|
||||||
|
# PROVENANCE: these eight are what `release/1.0` announces at
|
||||||
|
# `meter_source: firmware`, roster of one, recorded 2026-09-08. They stand in
|
||||||
|
# for the export so the checks run in CI. ⚠️ THE REAL INSTALL CARRIES ROUGHLY 49
|
||||||
|
# `goodwe*` ENTITIES (card ARCH-02, read 2026-09-06); the other ~41 belong to
|
||||||
|
# the ESPHome firmware, to template sensors and to helpers, and this add-on
|
||||||
|
# announces none of them. The tool separates the ids it owns from the ids it
|
||||||
|
# does not rather than claiming orphans it cannot cause - which is the half of
|
||||||
|
# this file the eight-line stand-in cannot exercise, and the reason DOCS.md's
|
||||||
|
# owner step asks for the real export.
|
||||||
|
sensor.goodwe_battery_power
|
||||||
|
sensor.goodwe_battery_soc
|
||||||
|
sensor.goodwe_circulating_power
|
||||||
|
sensor.goodwe_controller_status
|
||||||
|
sensor.goodwe_grid_power
|
||||||
|
sensor.goodwe_maintenance_phase
|
||||||
|
sensor.goodwe_p1_sample_age
|
||||||
|
sensor.goodwe_setpoint
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
# ARCH-02 fixture: THE SHAPE AFTER INVERTER B IS COMMISSIONED.
|
||||||
|
#
|
||||||
|
# ⚠️ THE FIRST ROSTER ENTRY IS **NOT** CALLED `inverter-a`, AND THAT IS THE
|
||||||
|
# WHOLE FIXTURE. `MUTATION.md` habit 4's seventh instance is MAINT-05 shipping
|
||||||
|
# `migrate_maint_keys(store, UNIT_A, ...)` under a comment saying it must not:
|
||||||
|
# every two-pack fixture in the suite named its first entry `inverter-a`, so
|
||||||
|
# `UNIT_A` and `inverters[0].name` were the same string in every check that
|
||||||
|
# existed and the defect was invisible. Names that are not `inverter-a` are what
|
||||||
|
# make a misdirected namespace something a rehearsal can SEE.
|
||||||
|
#
|
||||||
|
# `roster-inverter-a.yaml` beside this file is the same roster with the first
|
||||||
|
# entry named `inverter-a`, so the two can be compared - see DOCS.md.
|
||||||
|
setpoint_entity: number.goodwe_setpoint
|
||||||
|
soc_entity: sensor.goodwe_battery_soc_pct
|
||||||
|
batt_entity: sensor.goodwe_battery_power_w
|
||||||
|
batt_invert: false
|
||||||
|
max_w: 5000
|
||||||
|
bms_soc_floor: 10
|
||||||
|
meter_source: 'off'
|
||||||
|
meter_entity: sensor.p1_active_power
|
||||||
|
gain: 0.6
|
||||||
|
slew_w: 0
|
||||||
|
deadband_w: 15
|
||||||
|
target_grid_w: 0
|
||||||
|
step_w: 10
|
||||||
|
heartbeat_s: 10
|
||||||
|
stale_input_s: 15
|
||||||
|
maintenance_discharge_w: 2000
|
||||||
|
maintenance_charge_w: 2000
|
||||||
|
inverters:
|
||||||
|
- name: inverter-a
|
||||||
|
setpoint_entity: number.goodwe_setpoint
|
||||||
|
soc_entity: sensor.goodwe_battery_soc_pct
|
||||||
|
batt_entity: sensor.goodwe_battery_power_w
|
||||||
|
max_w: 5000
|
||||||
|
bms_soc_floor: 10
|
||||||
|
- name: inverter-b
|
||||||
|
setpoint_entity: number.goodwe_b_setpoint
|
||||||
|
soc_entity: sensor.goodwe_b_battery_soc_pct
|
||||||
|
batt_entity: sensor.goodwe_b_battery_power_w
|
||||||
|
max_w: 4600
|
||||||
|
bms_soc_floor: 10
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
# ARCH-02 fixture: THE SHAPE AFTER INVERTER B IS COMMISSIONED.
|
||||||
|
#
|
||||||
|
# ⚠️ THE FIRST ROSTER ENTRY IS **NOT** CALLED `inverter-a`, AND THAT IS THE
|
||||||
|
# WHOLE FIXTURE. `MUTATION.md` habit 4's seventh instance is MAINT-05 shipping
|
||||||
|
# `migrate_maint_keys(store, UNIT_A, ...)` under a comment saying it must not:
|
||||||
|
# every two-pack fixture in the suite named its first entry `inverter-a`, so
|
||||||
|
# `UNIT_A` and `inverters[0].name` were the same string in every check that
|
||||||
|
# existed and the defect was invisible. Names that are not `inverter-a` are what
|
||||||
|
# make a misdirected namespace something a rehearsal can SEE.
|
||||||
|
#
|
||||||
|
# `roster-inverter-a.yaml` beside this file is the same roster with the first
|
||||||
|
# entry named `inverter-a`, so the two can be compared - see DOCS.md.
|
||||||
|
setpoint_entity: number.goodwe_setpoint
|
||||||
|
soc_entity: sensor.goodwe_battery_soc_pct
|
||||||
|
batt_entity: sensor.goodwe_battery_power_w
|
||||||
|
batt_invert: false
|
||||||
|
max_w: 5000
|
||||||
|
bms_soc_floor: 10
|
||||||
|
meter_source: 'off'
|
||||||
|
meter_entity: sensor.p1_active_power
|
||||||
|
gain: 0.6
|
||||||
|
slew_w: 0
|
||||||
|
deadband_w: 15
|
||||||
|
target_grid_w: 0
|
||||||
|
step_w: 10
|
||||||
|
heartbeat_s: 10
|
||||||
|
stale_input_s: 15
|
||||||
|
maintenance_discharge_w: 2000
|
||||||
|
maintenance_charge_w: 2000
|
||||||
|
inverters:
|
||||||
|
- name: garage
|
||||||
|
setpoint_entity: number.goodwe_setpoint
|
||||||
|
soc_entity: sensor.goodwe_battery_soc_pct
|
||||||
|
batt_entity: sensor.goodwe_battery_power_w
|
||||||
|
max_w: 5000
|
||||||
|
bms_soc_floor: 10
|
||||||
|
- name: shed
|
||||||
|
setpoint_entity: number.goodwe_b_setpoint
|
||||||
|
soc_entity: sensor.goodwe_b_battery_soc_pct
|
||||||
|
batt_entity: sensor.goodwe_b_battery_power_w
|
||||||
|
max_w: 4600
|
||||||
|
bms_soc_floor: 10
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"_comment": "ARCH-02 SYNTHETIC FIXTURE - a store that has ALREADY run the flat config under 0.7.0+, so DEV-02's `schema_version` is stamped and the SAFETY-02 latch is not tripped. NOT a copy of the live box's /data/state.json. This is the CLEAN CONTROL for the MAINT-05 half of the rehearsal: the flat->roster transition over it must come out with NO findings, which is what makes each deliberate mutation's RED mean something. A harness that is red on every input has not measured anything.",
|
||||||
|
"schema_version": 1,
|
||||||
|
"phase": "idle",
|
||||||
|
"phase_started": "2026-08-07T10:00:00+00:00",
|
||||||
|
"last_completed": "2026-08-07T18:42:00+00:00",
|
||||||
|
"last_start_attempt": "2026-08-07T10:00:00+00:00",
|
||||||
|
"sign_tripped": false,
|
||||||
|
"sign_reason": "",
|
||||||
|
"auto": true
|
||||||
|
}
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
{
|
||||||
|
"_comment": "ARCH-02 SYNTHETIC FIXTURE - a FRESH INSTALL. No maintenance history, no latch, no schema markers: what /data/state.json looks like before the add-on has ever completed a cycle. It is a clean control, and it earned its place: the first draft of the harness reported this transition RED, because the DEV-02 migration writes both SAFETY-02 defaults under `inverter-a` before a roster exists and the tool called them stranded. Losing a default loses nothing."
|
||||||
|
}
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
{
|
||||||
|
"_comment": "INSTR-01 SYNTHETIC FIXTURE - `state-legacy.json` with a SECOND roster unit already latched. Not a copy of any live state file. It exists to restore M12 END-TO-END on the STAGED leg, which is the arrangement SAFETY-44 and INSTR-01's first draft both missed: `snapshot_over()` starts the after-leg over the file the BEFORE-leg wrote, so a transition-made destination is available there and the two questions the provenance clause and the narrowing ask are NOT incompatible. Every same-step arrangement makes them incompatible; that is what was measured and over-generalised to 'no fixture can'.",
|
||||||
|
"phase": "idle",
|
||||||
|
"phase_started": "2026-08-07T10:00:00+00:00",
|
||||||
|
"last_completed": "2026-08-07T18:42:00+00:00",
|
||||||
|
"last_start_attempt": "2026-08-07T10:00:00+00:00",
|
||||||
|
"sign_tripped": true,
|
||||||
|
"sign_reason": "SIGN_DISAGREEMENT: commanded -1800 W against a meter reading -1450 W for 190 s",
|
||||||
|
"auto": true,
|
||||||
|
"unit/shed/sign_tripped": true,
|
||||||
|
"unit/shed/sign_reason": "SIGN_DISAGREEMENT: shed, latched before the switch"
|
||||||
|
}
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
{
|
||||||
|
"_comment": "ARCH-02 SYNTHETIC FIXTURE - a PRE-DEV-02 flat state file. NOT a copy of the live box's /data/state.json; this session cannot read that one and did not try. It is built to carry, deliberately, the two shapes that state built by the happy path does not have: a LATCHED SAFETY-02 failsafe under the flat spelling, and a real maintenance history under the flat spelling. Both are LEGACY-SHAPED: there is no schema_version and no maint_schema_version, so this file was last written by a controller that had never heard of unit namespaces. Losing either one is silent.",
|
||||||
|
"phase": "idle",
|
||||||
|
"phase_started": "2026-08-07T10:00:00+00:00",
|
||||||
|
"last_completed": "2026-08-07T18:42:00+00:00",
|
||||||
|
"last_start_attempt": "2026-08-07T10:00:00+00:00",
|
||||||
|
"sign_tripped": true,
|
||||||
|
"sign_reason": "SIGN_DISAGREEMENT: commanded -1800 W against a meter reading -1450 W for 190 s",
|
||||||
|
"auto": true
|
||||||
|
}
|
||||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user