diff --git a/goodwe_controller/CHANGELOG.md b/goodwe_controller/CHANGELOG.md index 7358949..8c47031 100644 --- a/goodwe_controller/CHANGELOG.md +++ b/goodwe_controller/CHANGELOG.md @@ -22,6 +22,7 @@ if it has, it needs 0.10.0. Whoever deploys decides. | _this branch_ | TARIFF-14 - the price a trade is decided against is the price of the interval being traded | | _this branch_ | TARIFF-07 - three plugins that never fetched a price are withdrawn, and a site configured on one is told so **⚠️ BACKWARD-INCOMPATIBLE** | | _this branch_ | TARIFF-15 - `custom_formula`: your own tariff card's two coefficients, graded against the card's own printed price | +| _this branch_ | TARIFF-13 - the capacity-tariff row's "quarter(s) banked" stops resetting to 0 at every restart, and a tied peak's quarter stops moving | ⚠️ **A CONFIG THAT STARTS ON THE CURRENT VERSION CAN BE REFUSED ON THE NEXT ONE.** TARIFF-07 withdraws `engie_dynamic`, `totalenergies_dynamic` and @@ -57,6 +58,57 @@ separate card. *(Annotated 2026-09-13: that card is **TARIFF-15**, below, and it shipped on this branch as `custom_formula`. It is still not an alias — the operator types their own coefficients; nothing is invented on their behalf.)* +### TARIFF-13 - the coverage figure beside the peak, and which quarter gets it + +**The capacity-tariff status row's `N quarter(s) banked` is read from the +database instead of from process memory.** It was an `int` on `PeakTracker`, +starting at 0 and counted up inside `sync()`, printed next to a peak read from +SQLite. A 40-hour soak caught what that means at a restart, on both instances: + +``` +before: 2026-09: peak 365 W, set by the quarter at 2026-09-11T18:45:00+00:00 + - 10 % of the 3500 W soft limit; 120 quarter(s) banked +after : 2026-09: peak 365 W, set by the quarter at 2026-09-11T18:45:00+00:00 + - 10 % of the 3500 W soft limit; 0 quarter(s) banked +``` + +The peak is durable and was never wrong. The **count** is what says how much of +the billing month that peak covers, and 365 W over 120 quarters and 365 W over +0 quarters are different claims about the same number — so mid-month, after +every restart, the page made the weaker claim while showing the stronger one's +value. TARIFF-01's rule that an unreadable store must not read as a zero peak, +one column over. Two more defects in the same variable, neither in the ticket: +it was **never scoped to a month** (on 1 October the row still showed +September's tally beside October's empty month), and it **missed the block +`restore()` banks** from the checkpoint at startup. + +⚠️ **AN UNREADABLE STORE NOW SAYS SO IN THAT CLAUSE.** Where the database will +not answer a count, the row reads `an unreadable number of quarter(s) banked`. +`0` was never available; `None` was rejected too, because `None quarter(s) +banked` reads as the same zero. `0 quarter(s) banked` stays true and reachable — +it is what a month nothing has been written to says. + +⚠️ **AND A TIED PEAK'S ATTRIBUTED QUARTER IS NOW DECIDED: the EARLIEST wins.** +The wattage is what the utility bills, so this changes no bill and no guard — +`peak_w` is still only ever raised and SAFETY-07 reads the same watts. What it +changes is the answer to "when did my monthly peak happen", which an operator +goes and investigates; it now stops moving. The two readers of that record +**did** disagree: the `monthly_peak` upsert's strict `>` kept whichever tied +block was *written* first, while `recompute_month()` has always kept the +earliest — the same by accident only while blocks arrive in time order, and +measured to diverge (09:45 against 09:00) when they do not. + +⚠️ **THE SOAK'S DRIFTING QUARTER WAS NOT A TIE, AND IS LEFT ALONE.** Driven, +exactly equal quarters never re-attributed at all. What moved the timestamp was +a genuine new maximum **below the row's whole-watt resolution** — 365.0000001 W +is a new peak to the database and the same `365 W` to the reader. That is the +instrument being right, and printing more decimals to explain it would put +noise in the one row an operator reads when the guard engages. + +Reasoning in `DECISIONS/TARIFF-13-the-coverage-of-a-peak.md`; parts 11 and 12 +of `test_tariff.py` drive a real restart — a second `Controller` against the +same file — and pin both decisions. + ### TARIFF-15 - your own tariff card, as two coefficients **New price provider `custom_formula`, and one new option, `price_formula`.** diff --git a/goodwe_controller/DECISIONS/TARIFF-13-the-coverage-of-a-peak.md b/goodwe_controller/DECISIONS/TARIFF-13-the-coverage-of-a-peak.md new file mode 100644 index 0000000..f343e3a --- /dev/null +++ b/goodwe_controller/DECISIONS/TARIFF-13-the-coverage-of-a-peak.md @@ -0,0 +1,225 @@ +# TARIFF-13 — the coverage of a peak, and which quarter gets the credit + +Two decisions. §1 is the defect the ticket was raised on; §2 is the smaller +question it asked to be settled deliberately rather than left as an accident. +§3 is what this branch deliberately did **not** do, and why. + +--- + +## 0. What was actually wrong, stated once + +`main.py`'s capacity-tariff status row prints a peak read from SQLite and a +coverage figure read from **process memory**: + +```python +_pk = self.peak.peak(when) # SQLite, durable +... ++ f"; {self.peak.recorded} quarter(s) banked" # an int on the tracker +``` + +`PeakTracker.recorded` was `self.recorded = 0` in `__init__`, incremented once +per block inside `sync()`. A 40-hour soak caught what that means at a driven +restart, on both instances independently: + +``` +before: 2026-09: peak 365 W, set by the quarter at 2026-09-11T18:45:00+00:00 + - 10 % of the 3500 W soft limit; 120 quarter(s) banked +after : 2026-09: peak 365 W, set by the quarter at 2026-09-11T18:45:00+00:00 + - 10 % of the 3500 W soft limit; 0 quarter(s) banked +``` + +Reproduced here before anything was changed, through the real `Controller`, the +real `P1Ingest`, and a **real restart** — a second `Controller` object against +the same file on disk, never a re-read of the first: + +``` +before: 2026-09: peak 365 W, set by the quarter at 2026-09-10T09:00:00+00:00 + - 10 % of the 3500 W soft limit; 12 quarter(s) banked +after : 2026-09: peak 365 W, set by the quarter at 2026-09-10T09:00:00+00:00 + - 10 % of the 3500 W soft limit; 0 quarter(s) banked + + peak survives the restart: True + count survives the restart: False (12 -> 0, store held 12 rows) +``` + +**TARIFF-01's argument, one column over.** That module's whole design is that +an unreadable store must not read as a zero peak, *because a zero peak is +disarming*. A coverage figure of zero is disarming in exactly the same way and +about the same number: the count is what says how much of the billing month the +watt figure actually covers, and 365 W over 120 quarters and 365 W over 0 +quarters are different claims about the same 365 W. After a restart the page +made the weaker claim while displaying the stronger one's value. It did not +under-report the peak — it under-reported **its own evidence**. + +Two further defects fell out of the same variable once it was driven, neither +of them in the ticket: + +* **it was never scoped to a month.** It counted every block the *process* had + banked, so on 1 October the row read September's tally beside October's "no + completed quarter recorded yet". Measured, on the shipped tree: + `2026-10: no completed quarter recorded yet - 0 % of the 3500 W soft limit; + 1 quarter(s) banked`. +* **it missed the block `restore()` banks.** A stale block closed from the + checkpoint goes into `quarter_blocks` without passing through `sync()`, so + the counter never saw it. + +--- + +## 1. Decision: the count is READ FROM THE TIER THE PEAK IS READ FROM + +The acceptance criterion allowed either route — make the count durable, or stop +printing a number that silently resets. **Durable**, because the count is the +useful half of the row and removing it would have cost the operator the one +thing that says whether the peak means anything yet. + +`PeakStore.banked_count(month)` asks the database `SELECT COUNT(*) FROM +quarter_blocks WHERE month=?`, and `PeakTracker.recorded` is a property over it +rather than an attribute. Three consequences, and all three are the defects +above: + +* a restart re-reads the same rows, so the figure survives it; +* the count is per **month**, so it agrees with the peak it is printed beside; +* a block banked by `restore()` is a row like any other and is counted. + +Three things this route deliberately does **not** do: + +* **it does not count `len(blocks_in(month))`.** `blocks_in()` returns `[]` on a + read error — safe there only because no caller reads its emptiness as a peak, + which its own docstring says — so taking its length would rebuild the + disarming zero one call further out. `banked_count()` returns **`None`** when + the database will not answer. +* **it does not count `monthly_peak`.** That tier holds one row per *month* and + would answer `1` for every month ever measured — wrong by exactly the amount + that makes it look plausible. The two tables are proven equal on the **peak** + by `recompute_month()`; they were never equal on the **count**. +* **it does not remove the dedupe.** `_seen` is still what stops `cycle()`'s + once-a-second `sync()` re-banking a block that `P1Ingest.blocks` keeps for a + day, and `sync()`'s return value is still the number newly banked. What moved + is only where the *status page* reads its figure. ⚠️ The check that graded the + dedupe used to assert `recorded == 1` — the dedupe and the figure reporting it + were the same variable. Counting rows in `quarter_blocks` cannot see a + re-bank at all (it is a PRIMARY KEY no-op), so that check is now graded on + `sync()`'s **return**, with the store's tally as the second half. + +### ⚠️ The unreadable case renders a WORD, not a number + +`banked_count()` returns `int | None`; `recorded_in()` renders the `None` as +the module constant `COUNT_UNREADABLE = "an unreadable number of"`, so the +sentence reads + +``` +... ; an unreadable number of quarter(s) banked +``` + +`0` was never available. `None` was rejected too: `f"{None} quarter(s) banked"` +renders **"None quarter(s) banked"**, which an operator reads as the same zero. +This is the `PeakReading` trade — no `peak_w_or(default)`, ever — applied to the +coverage figure: a caller that formats it gets a true sentence in both states, +and a caller that does arithmetic on it gets a `TypeError`, which is loud. + +"0 quarter(s) banked" remains a **true and reachable** sentence: it is what a +month nothing has been written to says, and part 11 asserts that the two states +are spelled differently. + +--- + +## 2. Decision: at EQUAL wattage, the EARLIEST quarter keeps the attribution + +**First, not most recent.** The reasoning, in the order it matters: + +1. **It is not a money question.** The utility bills the wattage; the timestamp + is not part of the bill. So the decision is settled entirely on what the + status row is *for*. +2. **The row answers "when did my monthly peak happen".** An operator reading it + goes and looks at one event — what was running, what the weather was. A + stable answer is the one that can be investigated; "most recent wins" + guarantees the answer moves every time the wattage recurs, and an operator + who looked yesterday is sent somewhere else today for no reason. +3. **It is what one reader already did unconditionally.** `recompute_month()` + is `ORDER BY offtake_avg_w DESC, quarter_start ASC LIMIT 1` — earliest wins. + Choosing "most recent" would have meant changing both readers; choosing + "earliest" meant making the other one say out loud what it was already doing + by accident. + +### ⚠️ And the two readers DID diverge, which is why this is a code change + +The `monthly_peak` upsert was `WHERE excluded.peak_w > monthly_peak.peak_w` — +strictly greater, so an exact tie never wrote and the row kept whichever tied +block was **written first**. That coincides with *earliest* only while blocks +arrive in time order. Fed four equal 365 W blocks newest-first, measured on the +shipped tree: + +``` +ascending arrival monthly_peak -> 09:00:00 recompute_month -> 09:00:00 agree=True +DESCENDING arrival monthly_peak -> 09:45:00 recompute_month -> 09:00:00 agree=False +``` + +That is a divergence in the derived record that part 8 of `test_tariff.py` +exists to prove cannot happen — latent, because nothing in the shipped path +banks out of order today. The upsert now carries the tie term explicitly: + +```sql +WHERE excluded.peak_w > monthly_peak.peak_w + OR (excluded.peak_w = monthly_peak.peak_w + AND excluded.quarter_start < monthly_peak.quarter_start) +``` + +**`peak_w` is untouched.** The `>` term is exactly as it was, no tie can move +the billed number, and SAFETY-07's guard reads the same watt figure it read +before. The second term moves only the timestamp. + +### ⚠️ AND THE SOAK'S DRIFT WAS NOT A TIE AT ALL + +The ticket's reading — "ties at the same wattage are re-attributed to the most +recent matching quarter" — is **false**, and driving it is what showed that. +Six exactly-equal 365.0 W quarters on the shipped tree moved the attribution +**zero** times, because `>` is strict. + +What the soak saw was a genuine new maximum **smaller than the row's own +resolution**. The row renders `{peak_w:.0f}` — whole watts — so 365.0000001 W is +a new peak to the database and the same `365 W` to the reader: + +``` +q0 fed 365.0 -> renders 365 W, attributed 09:00:00 +q1 fed 365.0000001 -> renders 365 W, attributed 09:15:00 +q2 fed 365.0000002 -> renders 365 W, attributed 09:30:00 +``` + +**Decision: left alone, deliberately.** It is the instrument behaving correctly +— the peak really did rise — and the alternative, printing more decimal places, +puts noise in the one row an operator reads at the moment the guard engages, to +explain a movement that is real. Both cases are pinned in part 12 so that +neither can be changed by accident, and the distinction is recorded here so the +next reader does not re-diagnose it as a tie. + +--- + +## 3. What this branch did NOT do, and who owns it + +**`main.py`'s status row was not touched.** TARIFF-11 holds `main.py`'s status +rows on a concurrent branch, and the standing instruction is to stop rather than +race there. Everything above is in `app/tariff.py`, and the row picks the fix up +without editing, because `self.peak.recorded` is an attribute access and is now +a property. + +**The residual, stated with its scope.** `recorded` answers for +`datetime.now()`. The peak beside it follows `checks(when=...)`. The shipped +install has **exactly one** call site — `self.checks()` with no argument, at +`main.py:4359` — where the two are the same month, so **the live page is correct +and durable**. They part company only under the `when=` affordance TEST-17 added +for the suite, and the mismatch there is strictly smaller than the one it +replaces (this month's tally, rather than a process-wide total across all +months). + +`PeakTracker.recorded_in(when)` is the scoped form and is shipped. Moving the +row onto it is: + +```python +- + f"; {self.peak.recorded} quarter(s) banked" ++ + f"; {self.peak.recorded_in(when)} quarter(s) banked" +``` + +one token, on one line, inside the block TARIFF-11 owns. `test_tariff.py` +part 10 **pins the present behaviour** — graded against a row count taken +straight from `quarter_blocks`, not against the property that produces it — so +the gap goes red if it changes and cannot be quietly forgotten. diff --git a/goodwe_controller/app/tariff.py b/goodwe_controller/app/tariff.py index 59d4a78..ee7283f 100644 --- a/goodwe_controller/app/tariff.py +++ b/goodwe_controller/app/tariff.py @@ -132,6 +132,18 @@ OBSERVED = "observed" # the store answered and there is a peak NO_RECORD = "no_record" # the store answered and this month has none yet NOT_OBSERVED = "not_observed" # the store did not answer. NOT a zero peak. +# TARIFF-13. What the COVERAGE figure reads when the store cannot count. +# +# ⚠️ NOT 0, AND NOT `None`. This is the reading-state argument of TARIFF-01 +# applied to the number printed BESIDE the peak rather than to the peak. A +# coverage figure of 0 says "this month has been measured over no quarters at +# all", which is the strongest possible claim that the peak next to it means +# nothing - and `f"{None} quarter(s) banked"` renders "None quarter(s) banked", +# which an operator reads as that same zero. The one honest answer to "how many +# quarters have I banked" when the table will not answer is that it is not +# known, and it has to SAY so in the sentence the operator reads. +COUNT_UNREADABLE = "an unreadable number of" + # --------------------------------------------------------------------------- # # local time, without a hard dependency on the tz database @@ -388,6 +400,23 @@ class PeakStore: " per_phase_json=excluded.per_phase_json" " WHERE excluded.offtake_avg_w > quarter_blocks.offtake_avg_w", (start, month, w, pp_json)) + # TARIFF-13. THE TIE RULE, DECIDED RATHER THAN EMERGENT: + # at EQUAL wattage the EARLIEST quarter keeps the attribution. + # See DECISIONS/TARIFF-13-the-coverage-of-a-peak.md §2. + # + # `peak_w` itself is still only ever RAISED - the `>` term is + # untouched and no tie can move the billed number - so SAFETY-07 + # reads exactly the watt figure it read before. The second term + # moves only the TIMESTAMP the status page attributes it to. + # + # ⚠️ WITHOUT THE SECOND TERM THE TWO READERS DISAGREE, measured: + # `>` alone keeps whichever tied block was WRITTEN FIRST, while + # `recompute_month()`'s `ORDER BY offtake_avg_w DESC, + # quarter_start ASC` keeps the EARLIEST. Those coincide only + # while blocks arrive in time order. Fed four equal 365 W blocks + # newest-first, `monthly_peak` answered 09:45 and the recompute + # answered 09:00 - a divergence in the derived record part 8 of + # test_tariff.py exists to prove cannot happen. self._db.execute( "INSERT INTO monthly_peak(month, peak_w, quarter_start, per_phase_json)" " VALUES(?,?,?,?)" @@ -395,7 +424,9 @@ class PeakStore: " peak_w=excluded.peak_w," " quarter_start=excluded.quarter_start," " per_phase_json=excluded.per_phase_json" - " WHERE excluded.peak_w > monthly_peak.peak_w", + " WHERE excluded.peak_w > monthly_peak.peak_w" + " OR (excluded.peak_w = monthly_peak.peak_w" + " AND excluded.quarter_start < monthly_peak.quarter_start)", (month, w, start, pp_json)) return True except sqlite3.Error as err: @@ -449,6 +480,49 @@ class PeakStore: quarter_start=datetime.fromisoformat(row[1]), per_phase_w=pp) + def banked_count(self, month: str) -> int | None: + """How many quarters this month has banked. `None` when unknowable. + + TARIFF-13. THE DEFECT THIS PREVENTS: the status page printed a coverage + figure held in PROCESS MEMORY - `PeakTracker.recorded`, incremented once + per block banked by THIS process - beside a peak read from SQLite. The + peak outlives a restart and the counter does not, so a driven restart + mid-month took a real reading of + + 2026-09: peak 365 W ... ; 120 quarter(s) banked + + to the same peak, the same attributed quarter and `0 quarter(s) banked`, + on both instances independently. The count is what tells an operator how + much of the billing month the number beside it actually covers: 365 W + over 120 quarters and 365 W over 0 quarters are different claims about + the same watt figure, and after a restart the page made the weaker claim + while displaying the stronger one's value. Reading the count from the + same durable tier the peak comes from is what makes the two agree. + + ⚠️ `None`, NOT 0, and NOT `len(blocks_in(month))`. `blocks_in()` returns + `[]` on a read error - safe there only because no caller reads its + emptiness as a peak (§ its own docstring) - so counting its length would + rebuild the disarming zero one call further out. This asks the database + for a COUNT and refuses to answer when the database refuses. + + ⚠️ Counts `quarter_blocks`, the tier with one row per closed block, and + NOT `monthly_peak`, which has one row per MONTH and would answer 1 for + every month that has ever been measured. The two tables are proven equal + on the PEAK by `recompute_month()`; they were never equal on the COUNT + and are not meant to be. + """ + if self._db is None: + return None + try: + row = self._db.execute( + "SELECT COUNT(*) FROM quarter_blocks WHERE month=?", + (month,)).fetchone() + except sqlite3.Error as err: + self.last_error = str(err) + _LOG.error("could not count quarters for %s (%s)", month, err) + return None + return int(row[0]) + def blocks_in(self, month: str) -> list: """(quarter_start, offtake_avg_w) for a month, oldest first. [] on error is safe here ONLY because no caller reads emptiness as a peak - AC 3's @@ -539,11 +613,59 @@ class PeakTracker: self.cap_w = float(cap_w) self.policy = policy self.checkpoint_s = float(checkpoint_s) - self.recorded = 0 self.restore_state = "not attempted" self._seen: set = set() self._last_ckpt = 0.0 + # -- coverage ----------------------------------------------------------- + # + # TARIFF-13. `recorded` USED TO BE AN `int` ATTRIBUTE ON THIS OBJECT, + # starting at 0 and incremented in `sync()`. It is a property over the store + # now, and the three things that changes are all defects it was carrying: + # + # * it RESET TO 0 ON EVERY RESTART, beside a peak that did not - see + # `PeakStore.banked_count()` for the driven reading; + # * it was NOT SCOPED TO A MONTH. It counted every block this process had + # banked, so on 1 October the row still read September's tally beside + # October's "no completed quarter recorded yet"; + # * it MISSED the block `restore()` banks. A stale block closed from the + # checkpoint goes into `quarter_blocks` and never went through `sync()`, + # so the counter never saw it. Counting the table cannot miss it. + # + # ⚠️ REMOVING THE COUNTER DOES NOT REMOVE THE DEDUPE. `_seen` is still what + # stops `cycle()`'s once-a-second `sync()` re-banking a closed quarter that + # `P1Ingest.blocks` keeps for a DAY, and `sync()`'s RETURN value is still + # the count of blocks newly banked on that call. What changed is only where + # the STATUS PAGE's figure is read from. The check that grades the dedupe + # must therefore grade `sync()`'s return, not this property, which now + # answers 1 from the database however many times the block was offered. + + def recorded_in(self, when: datetime | None = None): + """Quarters banked in `when`'s billing month - `int`, or the sentinel. + + ⚠️ The return type is `int | str` on purpose, and the `str` is + `COUNT_UNREADABLE`. A caller that formats this into a sentence gets a + true sentence in both cases; a caller that wants to do arithmetic on it + gets a TypeError, which is loud. That is the same trade `PeakReading` + makes by having no `peak_w_or(default)`. + """ + n = self.store.banked_count(month_key(when or datetime.now(timezone.utc))) + return COUNT_UNREADABLE if n is None else n + + @property + def recorded(self): + """Quarters banked in the month being billed TODAY. + + ⚠️ This is the live status page's reading, where the peak beside it is + also read for `datetime.now()`. `checks(when=X)` for an X in another + month is a test affordance (TEST-17); the row scoped that way gets this + month's count beside that month's peak, which is strictly better than + the process-wide counter it replaces but is still a mismatch. + `recorded_in(when)` is the scoped form the row should read once + `main.py`'s status rows are free to change - TARIFF-11 holds them. + """ + return self.recorded_in() + # -- startup ------------------------------------------------------------ def restore(self, averager, now: datetime | None = None) -> str: """Put the open block back. Returns which of the three cases happened.""" @@ -600,7 +722,6 @@ class PeakTracker: continue if self.store.record_block(block, self.policy): self._seen.add(key) - self.recorded += 1 n += 1 if n: # A block just closed, so the checkpoint's block is gone: clearing it diff --git a/goodwe_controller/test_tariff.py b/goodwe_controller/test_tariff.py index 35f0ca4..e18a076 100644 --- a/goodwe_controller/test_tariff.py +++ b/goodwe_controller/test_tariff.py @@ -2,8 +2,8 @@ python3 test_tariff.py -Ten parts. What each is for, and - where it matters - what it is deliberately -NOT allowed to be tested at: +Twelve parts. What each is for, and - where it matters - what it is +deliberately NOT allowed to be tested at: 1. THE BLOCK ARITHMETIC, against a worked example written out segment by segment. The fixture is chosen to be non-degenerate on three axes at once: @@ -33,6 +33,17 @@ NOT allowed to be tested at: proven equal to the tier it derives from, and proven never to be lowered. 9. AC 3, ON THE REPLAY DATA THAT ACTUALLY EXISTS. See the honesty note below. 10. THE WIRING, through the real `Controller` and its real status page. + 11. TARIFF-13: THE COVERAGE FIGURE PRINTED BESIDE THE PEAK, across a REAL + restart - a second `Controller` against the same file. The peak was + always durable; the count of banked quarters was an `int` on the + tracker and reset to 0, so the page asserted "0 quarter(s) banked" + for a month it had measured 120 of. An unreadable store must not + produce a coverage figure of 0 either - part 5's argument, one + column over. + 12. TARIFF-13: WHICH QUARTER A TIED PEAK IS ATTRIBUTED TO, decided + (earliest wins) rather than left emergent, and pinned on BOTH + readers with a newest-first fixture - ascending arrival cannot tell + "the earliest quarter" from "the first row written". ------------------------------------------------------------------------------ ⚠️ PART 9 IS A TWO-DAY MEASUREMENT AND SAYS SO. THE 30-DAY ANSWER IS ELSEWHERE @@ -106,9 +117,9 @@ from pathlib import Path from app.main import Controller from app.p1 import P1Ingest, QuarterAverager, QUARTER_S, make_sample, split_signed from app.tariff import ( - BILLING_TZ, CHECKPOINT_S, NO_RECORD, NOT_OBSERVED, POLICY_MAX, - POLICY_NET, POLICY_SUM, PeakStore, PeakTracker, _EUTimezone, billed_w, - month_key, + BILLING_TZ, CHECKPOINT_S, COUNT_UNREADABLE, NO_RECORD, NOT_OBSERVED, + POLICY_MAX, POLICY_NET, POLICY_SUM, PeakStore, PeakTracker, _ClosedBlock, + _EUTimezone, billed_w, month_key, ) fails = [] @@ -932,18 +943,28 @@ check("the closed quarter reached SQLite through the tracker the cycle calls", # ⚠️ `cycle()` calls sync() ONCE A SECOND against a `blocks` list that keeps a # closed quarter for a DAY. Without a dedupe that is ~86 000 re-banks of the -# same row per block, and `recorded` - the number the status page prints as -# "quarters banked" - would be a count of loop iterations rather than of -# quarters. The database stays correct either way, which is exactly why this -# needs its own check instead of being caught by the peak. +# same row per block, and a coverage figure derived from the number of times a +# block was OFFERED would be a count of loop iterations rather than of quarters. +# The database stays correct either way, which is exactly why this needs its own +# check instead of being caught by the peak. +# +# ⚠️ GRADED ON `sync()`'s RETURN, and on the STORE's tally for the month the +# quarter is in - TARIFF-13. It used to be graded on `PeakTracker.recorded`, +# which was then an `int` incremented inside this very loop, so the dedupe and +# the figure that reported it were the same variable. They are not any more: +# `recorded_in()` counts rows in `quarter_blocks`, where re-banking the same +# `quarter_start` is a PRIMARY KEY no-op and could never have shown up as a +# second quarter. A check written against it alone would now pass however many +# times sync() re-banked, which is why the return value is the load-bearing +# half of this assertion and is asserted first. again = c.peak.sync(c.p1) third = c.peak.sync(c.p1) print(f" sync() called three times over the same closed quarter: " - f"{n}, {again}, {third} newly banked; tracker reports " - f"{c.peak.recorded} quarter(s)") + f"{n}, {again}, {third} newly banked; the store holds " + f"{c.peak.recorded_in(B)} quarter(s) for {month_key(B)}") check("...and syncing the same quarter again banks nothing new, so the count " "the status page shows is quarters and not cycles", - again == 0 and third == 0 and c.peak.recorded == 1) + again == 0 and third == 0 and c.peak.recorded_in(B) == 1) check("...and the peak is unchanged by the repeat", near(c.peak.peak(B).peak_w, 4998.0, 1e-6)) @@ -957,11 +978,33 @@ peak_rows = [r for r in rows if "peak" in r["text"] and "quarter(s) banked" in r print(" status row: " + (peak_rows[0]["text"] if peak_rows else "(none)")) check("the status page carries exactly one capacity-tariff peak row", len(peak_rows) == 1) -check("...naming the peak, the soft limit and the quarters banked, all derived " +check("...naming the peak, the soft limit and a coverage figure, all derived " "from the controller that was actually built", bool(peak_rows) and "4998 W" in peak_rows[0]["text"] and f"{SHIPPED_CAP_W:.0f} W soft limit" in peak_rows[0]["text"] - and "1 quarter(s) banked" in peak_rows[0]["text"]) + and "quarter(s) banked" in peak_rows[0]["text"]) + +# ⚠️ AND THE COVERAGE FIGURE IN THAT ROW IS SCOPED TO THE LIVE BILLING MONTH, +# NOT TO `when` - TARIFF-13, and it is a KNOWN GAP stated rather than rounded +# off. `main.py`'s row reads `self.peak.recorded`, which answers for +# `datetime.now()`; the peak beside it follows `when`. The two agree on the +# ONLY call site a shipped install has - `checks()` with no argument, at +# main.py:4359 - which is why the live page is correct and durable; they part +# company only under this suite's `when=` affordance. `PeakTracker` already +# carries the scoped form, `recorded_in(when)`, and moving the row onto it is a +# one-token edit inside main.py's status-row block, which TARIFF-11 holds. This +# check PINS the present behaviour so the gap cannot be forgotten, and part 11 +# is the authority on the figure itself. +_live_month = month_key(datetime.now(timezone.utc)) +_live_rows = len(c.peak.store.blocks_in(_live_month)) +print(f" the row read at {month_key(B)} carries the tally for " + f"{_live_month} ({_live_rows} quarter(s)), which is this fixture's " + f"known scoping gap") +check("the row's coverage figure is the LIVE month's tally - graded against a " + "row count taken straight from quarter_blocks, so this pins the gap " + "rather than restating the property that produces it", + bool(peak_rows) + and f"; {_live_rows} quarter(s) banked" in peak_rows[0]["text"]) check("...and it is FLAGGED, because 4998 W is over §9.1's 80 % warning point " f"on a {SHIPPED_CAP_W:.0f} W limit", bool(peak_rows) and not peak_rows[0]["ok"] and peak_rows[0]["warn"]) @@ -1149,6 +1192,308 @@ check(f"...and the same IMPORT grader over main.py finds {_main_imports}, so " bool(_main_imports)) +# ============================================================================== +print() +print("11. TARIFF-13: what the instrument says about its OWN COVERAGE, " + "across a restart") + +# THE DEFECT THIS PART PREVENTS. The status row prints a peak read from SQLite +# beside a count of banked quarters, and the count used to be an `int` attribute +# on `PeakTracker` that started at 0 in `__init__`. A 40-hour soak caught what +# that means at a driven restart, on both instances independently: +# +# before: 2026-09: peak 365 W, set by the quarter at 2026-09-11T18:45:00+00:00 +# - 10 % of the 3500 W soft limit; 120 quarter(s) banked +# after : 2026-09: peak 365 W, set by the quarter at 2026-09-11T18:45:00+00:00 +# - 10 % of the 3500 W soft limit; 0 quarter(s) banked +# +# The peak and its attributed quarter persist; the coverage beside them does +# not. §9.1's peak is what the utility bills, but the COUNT is what says how +# much of the billing month the figure covers - 365 W over 120 quarters and +# 365 W over 0 quarters are different claims about the same watt figure, and +# after a restart the page made the weaker claim while showing the stronger +# one's value. TARIFF-01's argument is that an unreadable store must not read as +# a zero peak because a zero peak is DISARMING; a zero COVERAGE beside a live +# peak is the same error one column over. +# +# ⚠️ THE RESTART HERE IS A REAL ONE - a second `Controller` against the same +# file on disk, never a re-read of the first object. 11b asserts that the second +# object has banked nothing itself, so the number it prints cannot have come +# from process memory. Without that precondition this whole part is satisfied by +# a counter that was simply never reset. + +_T13 = TMP / "t13-restart.db" + +# The fixture lives in the month being billed TODAY, and the row is read with NO +# `when` argument - the LIVE status page, the surface that misinformed. That +# also makes the check stable on every day of the year, which a fixed calendar +# date is not (TEST-17, one part up, is the same lesson). +_now13 = datetime.now(timezone.utc) +_mon13 = month_key(_now13) +# Anchor on the first 900 s boundary of the billing month, so all N quarters are +# inside `_mon13` however close to a month edge the suite happens to run. +_ms = datetime(int(_mon13[:4]), int(_mon13[5:]), 1, tzinfo=timezone.utc) +_q0 = datetime.fromtimestamp( + (int(_ms.timestamp()) // QUARTER_S + 1) * QUARTER_S, timezone.utc) + +_N13 = 6 # > 1, so "count == rows" cannot be satisfied by 0 == 0 +_W13 = 1234.0 + +c13 = built(_T13) +for _q in range(_N13): + _base = _q0 + timedelta(seconds=QUARTER_S * _q) + for _i in range(0, QUARTER_S + 1, 5): + c13.p1.submit(sample(_base + timedelta(seconds=_i), _W13)) + c13.peak.sync(c13.p1) + +# Ground truth from the OTHER tier, which the tracker did not supply - the count +# under test must be graded against something that is not itself (habit 2). +_truth_before = len(c13.peak.store.blocks_in(_mon13)) +_row_before = _peak_rows(c13.checks())[0]["text"] +print(" before restart: " + _row_before) +check(f"precondition: the fixture banked {_N13} quarters in {_mon13}, so a " + f"count of 0 and a count of the truth are different numbers", + _truth_before == _N13 and _N13 > 1) +check(f"...and the row says so before the restart", + f"{_N13} quarter(s) banked" in _row_before) + +c13.peak.store.close() + +# THE RESTART. +c13b = built(_T13) +_row_after = _peak_rows(c13b.checks())[0]["text"] +_truth_after = len(c13b.peak.store.blocks_in(_mon13)) +print(" after restart: " + _row_after) +print(f" rows in quarter_blocks: {_truth_before} before, " + f"{_truth_after} after; the restarted tracker has banked " + f"{len(c13b.peak._seen)} block(s) of its own through sync()") + +check("precondition: the second Controller is a REAL restart - it shares no " + "object with the first and has banked nothing through sync(), so any " + "count it prints came off the disk and not out of memory", + c13b.peak is not c13.peak and len(c13b.peak._seen) == 0) +check("the peak and its attributed quarter survive the restart - the durable " + "half, asserted so that the coverage claim below is not graded against a " + "reading that had already collapsed", + _row_before.split(" - ")[0] == _row_after.split(" - ")[0] + and f"{_W13:.0f} W" in _row_after) +check("...and the COUNT BESIDE IT DOES NOT COLLAPSE TO ZERO, which is the " + "defect: a coverage figure that resets while the value beside it " + "persists actively misinforms", + "; 0 quarter(s) banked" not in _row_after) +check(f"...and it is the store's own tally, {_truth_after}, not a " + f"process-lifetime counter - graded against a COUNT OF ROWS in the " + f"other tier rather than against the property under test", + f"{_truth_after} quarter(s) banked" in _row_after and _truth_after >= _N13) + +# ⚠️ The restarted run banks one MORE than it drove, and that is correct: the +# checkpointed open block was stale, so `restore()` closed and banked it. The +# in-memory counter never saw that block at all - it only ever counted what went +# through `sync()` - so this is a third way the old figure under-read, and it is +# asserted rather than tolerated. Driven at a chosen `now` so it cannot depend +# on when the suite runs. +_t13s = fresh_store("t13-stale") +_tr13 = PeakTracker(_t13s, cap_w=SHIPPED_CAP_W) +_av13 = QuarterAverager(phases=1) +_sb = _q0 + timedelta(seconds=QUARTER_S * 40) +for _i in range(0, 600, 5): + list(_av13.add(sample(_sb + timedelta(seconds=_i), 2000.0))) +_t13s.save_partial(_av13.snapshot()) +_before_stale = _tr13.recorded_in(_sb) +_state = _tr13.restore(QuarterAverager(phases=1), + now=_sb + timedelta(seconds=QUARTER_S * 3)) +print(f" stale-block restore: {_state}") +print(f" quarters banked in {month_key(_sb)}: {_before_stale} -> " + f"{_tr13.recorded_in(_sb)}") +check("a block closed by restore() rather than by sync() is counted too - the " + "old counter incremented only in sync() and missed it entirely", + _before_stale == 0 and _tr13.recorded_in(_sb) == 1 + and "banked" in _state) + +# --- the count is scoped to the MONTH it is printed beside -------------------- +# ⚠️ TWO MONTHS AT DIFFERENT COUNTS. Equal counts would be satisfied by a +# cross-month total, by a constant, and by reading either month for the other. +_t13m = fresh_store("t13-months") +_MA = datetime(2026, 4, 6, 9, 0, tzinfo=timezone.utc) # 2026-04 +_MB = datetime(2026, 5, 6, 9, 0, tzinfo=timezone.utc) # 2026-05 +_trm = PeakTracker(_t13m, cap_w=SHIPPED_CAP_W) +for _m, _k in ((_MA, 3), (_MB, 5)): + for _q in range(_k): + _t13m.record_block(_ClosedBlock( + start=_m + timedelta(seconds=QUARTER_S * _q), + offtake_avg_w=900.0 + _q, per_phase_offtake_avg_w=None)) +print(f" banked by month: {month_key(_MA)}={_trm.recorded_in(_MA)}, " + f"{month_key(_MB)}={_trm.recorded_in(_MB)}, " + f"a month never written={_trm.recorded_in(datetime(2026, 7, 6, 9, 0, tzinfo=timezone.utc))}") +check("precondition: the two months were given DIFFERENT numbers of quarters, " + "so a count that answered either one for the other would be visible here", + month_key(_MA) != month_key(_MB)) +check("each month reports ITS OWN quarters and not the pair's total - the old " + "counter was per PROCESS, so on 1 October the row still showed " + "September's tally beside October's empty month", + _trm.recorded_in(_MA) == 3 and _trm.recorded_in(_MB) == 5) +check("...and a month with no quarters reports 0, which is TRUE of a month " + "nothing was written to - the sentinel below is for a store that will " + "not answer, and the two states must not be spelled the same way", + _trm.recorded_in(datetime(2026, 7, 6, 9, 0, tzinfo=timezone.utc)) == 0) + +# --- and a store that cannot answer does NOT report zero ---------------------- +# The other half of part 5's argument, applied to the coverage figure. `0 +# quarter(s) banked` and `an unreadable number of quarter(s) banked` are +# opposite states: the first says the month has been measured over nothing, the +# second says the instrument cannot say. Spelling them the same way is exactly +# what this card was raised about. +_t13u = PeakStore(str(TMP / "no-such-dir" / "nope13.db")) +_tru = PeakTracker(_t13u, cap_w=SHIPPED_CAP_W) +print(f" unreadable store: banked_count -> {_t13u.banked_count(_mon13)!r}, " + f"rendered -> {_tru.recorded_in(_now13)!r}") +check("an unreadable store answers None to a COUNT, never 0 - the store-level " + "half", + _t13u.banked_count(_mon13) is None) +check("...and the rendered figure says it is unreadable instead of naming a " + "number, so the sentence cannot read as a measured zero", + _tru.recorded_in(_now13) == COUNT_UNREADABLE + and f"{_tru.recorded_in(_now13)} quarter(s) banked" + == "an unreadable number of quarter(s) banked") +check("...and that sentence is not merely NON-zero by accident: the string a " + "count of 0 would produce is different text", + f"{_tru.recorded_in(_now13)} quarter(s) banked" + != f"{0} quarter(s) banked") + +# ⚠️ A STORE THAT IS OPEN AND WHOSE COUNT QUERY FAILS IS A SECOND CODE PATH, and +# the checks above cannot reach it - a handle that never opened short-circuits +# on `self._db is None` before any SQL runs, so `banked_count()`'s +# `except sqlite3.Error` arm was executed by nothing. Found by this card's own +# mutation battery, not by review: returning 0 instead of None from that arm +# survived the whole suite at -3 B. It is the same third-path argument part 5 +# makes for `peak()`, and it is the arm a half-corrupt database on a real +# install actually takes. +_t13b = fresh_store("t13-broken") +for _q in range(4): + _t13b.record_block(_ClosedBlock(start=_q0 + timedelta(seconds=QUARTER_S * _q), + offtake_avg_w=800.0 + _q, + per_phase_offtake_avg_w=None)) +_trb = PeakTracker(_t13b, cap_w=SHIPPED_CAP_W) +_count_ok = _t13b.banked_count(_mon13) +_t13b._db.execute("DROP TABLE quarter_blocks") +_t13b._db.commit() +_count_broken = _t13b.banked_count(_mon13) +print(f" open store, count query FAILS: {_count_ok} -> {_count_broken!r}, " + f"rendered -> {_trb.recorded_in(_now13)!r}") +check(f"precondition: before the table was dropped the same call answered " + f"{_count_ok}, so 'it returns nothing now' is a change and not the " + f"store's resting state", + _count_ok == 4) +check("a store whose COUNT QUERY fails answers None too - a different code " + "path from a handle that never opened, and the one a half-corrupt " + "database takes", + _count_broken is None) +check("...and it renders as unreadable rather than as 0, so a half-corrupt " + "database cannot print a coverage figure that reads like a measured " + "month with nothing in it", + _trb.recorded_in(_now13) == COUNT_UNREADABLE) + +# --- the count reads the BLOCK tier, not the month tier ----------------------- +# `monthly_peak` holds one row per MONTH, so counting it answers 1 for every +# month that has ever been measured - a figure that is wrong by exactly the +# amount that makes it look plausible. +check("the coverage figure counts quarter_blocks and not monthly_peak, which " + "would answer 1 for every measured month however many quarters it holds", + _trm.recorded_in(_MB) == 5 + and _t13m.peak(_MB).is_observed and _t13m.recompute_month(month_key(_MB)).is_observed) + + +# ============================================================================== +print() +print("12. TARIFF-13: which quarter a TIED peak is attributed to") + +# THE DECISION, recorded in DECISIONS/TARIFF-13-the-coverage-of-a-peak.md §2: +# at equal wattage the EARLIEST quarter keeps the attribution. The wattage is +# what is billed, so this is not a money question - it is the answer to "when +# did my monthly peak happen", and an operator investigating that goes and looks +# at one event. A stable answer is the useful one; "most recent wins" guarantees +# the answer moves every time the wattage recurs. +# +# ⚠️ AND THE SOAK'S DRIFT WAS NOT A TIE. The 40-hour run showed 365 W attributed +# to three different quarters with no restart and no new peak, and the obvious +# reading - "ties re-attribute to the most recent" - is false, measured: the +# upsert's term is strictly `>`, so an exact tie never wrote. What moved was a +# genuine new maximum SMALLER THAN THE ROW'S OWN RESOLUTION. The row renders +# `{peak_w:.0f}`, whole watts; 365.0000001 W is a new peak to the database and +# the same "365 W" to the reader. Both cases are pinned below, because the fix +# for one is not the fix for the other. + +_t13t = fresh_store("t13-ties") +_TB = datetime(2026, 3, 9, 9, 0, tzinfo=timezone.utc) + + +def _bank(store, q, w): + store.record_block(_ClosedBlock(start=_TB + timedelta(seconds=QUARTER_S * q), + offtake_avg_w=w, per_phase_offtake_avg_w=None)) + + +def _qt(n): + return _TB + timedelta(seconds=QUARTER_S * n) + + +for _q in (0, 1, 2, 3): + _bank(_t13t, _q, 365.0) +_asc = _t13t.peak(_TB) +print(f" four EXACTLY equal 365 W quarters, oldest first -> attributed " + f"{_asc.quarter_start.isoformat()}") +check("an exact tie keeps the EARLIEST quarter: the attribution does not move " + "when the same wattage recurs", + _asc.quarter_start == _qt(0) and near(_asc.peak_w, 365.0)) + +# ⚠️ FED NEWEST-FIRST, which is what separates "the earliest quarter" from +# "whichever row was written first". Ascending arrival cannot tell those apart - +# they are the same row - so an ascending-only fixture grades neither. +_t13r = fresh_store("t13-ties-rev") +for _q in (3, 2, 1, 0): + _bank(_t13r, _q, 365.0) +_desc = _t13r.peak(_TB) +_desc_rc = _t13r.recompute_month(month_key(_TB)) +print(f" the same four fed NEWEST first -> monthly_peak attributed " + f"{_desc.quarter_start.isoformat()}, recompute_month attributed " + f"{_desc_rc.quarter_start.isoformat()}") +check("...and it is the earliest QUARTER, not the first row WRITTEN - fed " + "newest-first the answer is still the oldest quarter", + _desc.quarter_start == _qt(0)) +check("...and BOTH readers give it, so the derived record has not diverged " + "from the tier it derives from on the tie rule either (part 8's " + "invariant, at equal wattage)", + _desc_rc.quarter_start == _desc.quarter_start + and near(_desc_rc.peak_w, _desc.peak_w)) + +# The other direction, and the reason the tie rule may never touch `peak_w`: a +# STRICTLY HIGHER quarter must still take the attribution, whenever it arrives. +_t13h = fresh_store("t13-higher") +for _q, _w in ((0, 365.0), (1, 366.0), (2, 365.0)): + _bank(_t13h, _q, _w) +_hi = _t13h.peak(_TB) +print(f" 365, then 366, then 365 -> peak {_hi.peak_w:.0f} W at " + f"{_hi.quarter_start.isoformat()}") +check("a strictly higher quarter still takes both the peak and the " + "attribution - the tie rule moves the TIMESTAMP and never the watts", + near(_hi.peak_w, 366.0) and _hi.quarter_start == _qt(1)) + +# And the mechanism the soak actually saw: an increase below the row's own +# resolution. Not a tie, and no tie rule can address it. +_t13d = fresh_store("t13-subdisplay") +_seen13 = [] +for _q, _w in enumerate((365.0, 365.0000001, 365.0000002)): + _bank(_t13d, _q, _w) + _seen13.append(_t13d.peak(_TB).quarter_start) +print(f" 365.0, 365.0000001, 365.0000002 W all render " + f"{_t13d.peak(_TB).peak_w:.0f} W and attribute to " + f"{[s.time().isoformat() for s in _seen13]}") +check("an increase BELOW the row's whole-watt resolution is a real new maximum " + "and does move the attribution - so the soak's drifting quarter was a " + "rounded display, not a tie, and is left alone deliberately", + len(set(_seen13)) == 3 + and len({f"{w:.0f}" for w in (365.0, 365.0000001, 365.0000002)}) == 1) + + # ============================================================================== print() shutil.rmtree(TMP, ignore_errors=True)