TARIFF-13: the coverage figure beside the peak stops resetting at a restart
checks / unit (pull_request) Successful in 7m27s

The capacity-tariff status row printed a peak read from SQLite next to a count
of banked quarters read from PROCESS MEMORY - `PeakTracker.recorded`, an `int`
set to 0 in `__init__` and counted up inside `sync()`. A 40-hour soak caught
what that means at a driven restart, on both instances independently:

  before: 2026-09: peak 365 W, set by the quarter at 2026-09-11T18:45:00+00:00
          - 10 % of the 3500 W soft limit; 120 quarter(s) banked
  after : 2026-09: peak 365 W, set by the quarter at 2026-09-11T18:45:00+00:00
          - 10 % of the 3500 W soft limit;   0 quarter(s) banked

Reproduced here before anything changed, through the real Controller and a real
restart - a second Controller object against the same file on disk - at 12 -> 0
with 12 rows standing in `quarter_blocks`.

The peak is durable and was never wrong. The COUNT is what says how much of the
billing month that peak covers, and 365 W over 120 quarters and 365 W over 0
quarters are different claims about the same number. Mid-month, after every
restart, the page made the weaker claim while displaying the stronger one's
value. That is TARIFF-01's own argument - an unreadable store must not read as a
zero peak, because a zero peak is disarming - one column over.

`PeakStore.banked_count(month)` asks the database, and `recorded` is a property
over it. Two more defects in the same variable fell out once it was driven, and
neither was in the ticket: it was never scoped to a MONTH (on 1 October the row
still showed September's tally beside October's empty month - measured), and it
missed the block `restore()` banks from the checkpoint, which never passes
through `sync()`.

⚠️ NOT 0, AND NOT None, WHEN THE STORE WILL NOT ANSWER. `banked_count()` returns
None and the row renders `an unreadable number of quarter(s) banked`.
`f"{None} quarter(s) banked"` was rejected because it reads as the same zero.
`0 quarter(s) banked` stays true and reachable - it is what a month nothing has
been written to says, and the two are asserted to be different text.

⚠️ THE DEDUPE IS NOT REMOVED WITH THE COUNTER. `_seen` still stops `cycle()`'s
once-a-second `sync()` re-banking a block `P1Ingest.blocks` keeps for a day. The
check that graded it asserted `recorded == 1` - the dedupe and the figure
reporting it were the same variable - and now grades `sync()`'s RETURN, because
counting rows cannot see a re-bank at all (PRIMARY KEY no-op).

THE TIE RULE, DECIDED RATHER THAN EMERGENT: at equal wattage the EARLIEST
quarter keeps the attribution. No bill and no guard moves - `peak_w` is still
only ever raised, the `>` term is untouched, SAFETY-07 reads the same watts.
The two readers DID disagree: the upsert's strict `>` kept whichever tied block
was WRITTEN first while `recompute_month()` has always kept the earliest, the
same only while blocks arrive in time order, and measured to diverge (09:45
against 09:00) fed newest-first.

⚠️ AND THE SOAK'S DRIFTING QUARTER WAS NOT A TIE. Six exactly equal 365.0 W
quarters re-attributed zero times on the shipped tree - `>` is strict. What
moved it was a real new maximum BELOW THE ROW'S OWN RESOLUTION: the row renders
whole watts, so 365.0000001 W is a new peak to the database and the same 365 W
to the reader. Left alone deliberately, and pinned so it is not re-diagnosed.

test_tariff.py parts 11 and 12. 12 mutations, all red, listed on the PR; M2 -
`banked_count()`'s sqlite3.Error arm returning 0 - was a SURVIVOR at -3 B on the
first battery, because a handle that never opened short-circuits before any SQL
runs and nothing reached the except arm. Closed with an open store whose table
is dropped under it, with a precondition that the same call answered 4 first.

main.py is NOT touched - TARIFF-11 holds its status rows. The row picks this up
without editing because `recorded` is an attribute access and is now a property.
The residual is stated in DECISIONS §3: `recorded` answers for now() while the
peak follows `checks(when=...)`, and the shipped install has exactly one call
site - `self.checks()` with no argument, main.py:4359 - where they are the same
month, so the live page is correct and durable. `recorded_in(when)` is shipped
and the row's move onto it is one token, for whoever holds main.py next; part 10
pins the present behaviour against a row count taken from `quarter_blocks` so
the gap cannot be quietly forgotten.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BwRLw7eKYj2byVdwB6kbjU
This commit is contained in:
glenn schrooyen
2026-09-13 12:54:15 +02:00
co-authored by Claude Opus 5
parent 068b83f57c
commit a991438e40
4 changed files with 760 additions and 17 deletions
+52
View File
@@ -22,6 +22,7 @@ if it has, it needs 0.10.0. Whoever deploys decides.
| _this branch_ | TARIFF-14 - the price a trade is decided against is the price of the interval being traded |
| _this branch_ | TARIFF-07 - three plugins that never fetched a price are withdrawn, and a site configured on one is told so **⚠️ BACKWARD-INCOMPATIBLE** |
| _this branch_ | TARIFF-15 - `custom_formula`: your own tariff card's two coefficients, graded against the card's own printed price |
| _this branch_ | TARIFF-13 - the capacity-tariff row's "quarter(s) banked" stops resetting to 0 at every restart, and a tied peak's quarter stops moving |
⚠️ **A CONFIG THAT STARTS ON THE CURRENT VERSION CAN BE REFUSED ON THE NEXT
ONE.** TARIFF-07 withdraws `engie_dynamic`, `totalenergies_dynamic` and
@@ -57,6 +58,57 @@ separate card. *(Annotated 2026-09-13: that card is **TARIFF-15**, below, and it
shipped on this branch as `custom_formula`. It is still not an alias — the
operator types their own coefficients; nothing is invented on their behalf.)*
### TARIFF-13 - the coverage figure beside the peak, and which quarter gets it
**The capacity-tariff status row's `N quarter(s) banked` is read from the
database instead of from process memory.** It was an `int` on `PeakTracker`,
starting at 0 and counted up inside `sync()`, printed next to a peak read from
SQLite. A 40-hour soak caught what that means at a restart, on both instances:
```
before: 2026-09: peak 365 W, set by the quarter at 2026-09-11T18:45:00+00:00
- 10 % of the 3500 W soft limit; 120 quarter(s) banked
after : 2026-09: peak 365 W, set by the quarter at 2026-09-11T18:45:00+00:00
- 10 % of the 3500 W soft limit; 0 quarter(s) banked
```
The peak is durable and was never wrong. The **count** is what says how much of
the billing month that peak covers, and 365 W over 120 quarters and 365 W over
0 quarters are different claims about the same number — so mid-month, after
every restart, the page made the weaker claim while showing the stronger one's
value. TARIFF-01's rule that an unreadable store must not read as a zero peak,
one column over. Two more defects in the same variable, neither in the ticket:
it was **never scoped to a month** (on 1 October the row still showed
September's tally beside October's empty month), and it **missed the block
`restore()` banks** from the checkpoint at startup.
⚠️ **AN UNREADABLE STORE NOW SAYS SO IN THAT CLAUSE.** Where the database will
not answer a count, the row reads `an unreadable number of quarter(s) banked`.
`0` was never available; `None` was rejected too, because `None quarter(s)
banked` reads as the same zero. `0 quarter(s) banked` stays true and reachable —
it is what a month nothing has been written to says.
⚠️ **AND A TIED PEAK'S ATTRIBUTED QUARTER IS NOW DECIDED: the EARLIEST wins.**
The wattage is what the utility bills, so this changes no bill and no guard —
`peak_w` is still only ever raised and SAFETY-07 reads the same watts. What it
changes is the answer to "when did my monthly peak happen", which an operator
goes and investigates; it now stops moving. The two readers of that record
**did** disagree: the `monthly_peak` upsert's strict `>` kept whichever tied
block was *written* first, while `recompute_month()` has always kept the
earliest — the same by accident only while blocks arrive in time order, and
measured to diverge (09:45 against 09:00) when they do not.
⚠️ **THE SOAK'S DRIFTING QUARTER WAS NOT A TIE, AND IS LEFT ALONE.** Driven,
exactly equal quarters never re-attributed at all. What moved the timestamp was
a genuine new maximum **below the row's whole-watt resolution** — 365.0000001 W
is a new peak to the database and the same `365 W` to the reader. That is the
instrument being right, and printing more decimals to explain it would put
noise in the one row an operator reads when the guard engages.
Reasoning in `DECISIONS/TARIFF-13-the-coverage-of-a-peak.md`; parts 11 and 12
of `test_tariff.py` drive a real restart — a second `Controller` against the
same file — and pin both decisions.
### TARIFF-15 - your own tariff card, as two coefficients
**New price provider `custom_formula`, and one new option, `price_formula`.**
@@ -0,0 +1,225 @@
# TARIFF-13 — the coverage of a peak, and which quarter gets the credit
Two decisions. §1 is the defect the ticket was raised on; §2 is the smaller
question it asked to be settled deliberately rather than left as an accident.
§3 is what this branch deliberately did **not** do, and why.
---
## 0. What was actually wrong, stated once
`main.py`'s capacity-tariff status row prints a peak read from SQLite and a
coverage figure read from **process memory**:
```python
_pk = self.peak.peak(when) # SQLite, durable
...
+ f"; {self.peak.recorded} quarter(s) banked" # an int on the tracker
```
`PeakTracker.recorded` was `self.recorded = 0` in `__init__`, incremented once
per block inside `sync()`. A 40-hour soak caught what that means at a driven
restart, on both instances independently:
```
before: 2026-09: peak 365 W, set by the quarter at 2026-09-11T18:45:00+00:00
- 10 % of the 3500 W soft limit; 120 quarter(s) banked
after : 2026-09: peak 365 W, set by the quarter at 2026-09-11T18:45:00+00:00
- 10 % of the 3500 W soft limit; 0 quarter(s) banked
```
Reproduced here before anything was changed, through the real `Controller`, the
real `P1Ingest`, and a **real restart** — a second `Controller` object against
the same file on disk, never a re-read of the first:
```
before: 2026-09: peak 365 W, set by the quarter at 2026-09-10T09:00:00+00:00
- 10 % of the 3500 W soft limit; 12 quarter(s) banked
after : 2026-09: peak 365 W, set by the quarter at 2026-09-10T09:00:00+00:00
- 10 % of the 3500 W soft limit; 0 quarter(s) banked
peak survives the restart: True
count survives the restart: False (12 -> 0, store held 12 rows)
```
**TARIFF-01's argument, one column over.** That module's whole design is that
an unreadable store must not read as a zero peak, *because a zero peak is
disarming*. A coverage figure of zero is disarming in exactly the same way and
about the same number: the count is what says how much of the billing month the
watt figure actually covers, and 365 W over 120 quarters and 365 W over 0
quarters are different claims about the same 365 W. After a restart the page
made the weaker claim while displaying the stronger one's value. It did not
under-report the peak — it under-reported **its own evidence**.
Two further defects fell out of the same variable once it was driven, neither
of them in the ticket:
* **it was never scoped to a month.** It counted every block the *process* had
banked, so on 1 October the row read September's tally beside October's "no
completed quarter recorded yet". Measured, on the shipped tree:
`2026-10: no completed quarter recorded yet - 0 % of the 3500 W soft limit;
1 quarter(s) banked`.
* **it missed the block `restore()` banks.** A stale block closed from the
checkpoint goes into `quarter_blocks` without passing through `sync()`, so
the counter never saw it.
---
## 1. Decision: the count is READ FROM THE TIER THE PEAK IS READ FROM
The acceptance criterion allowed either route — make the count durable, or stop
printing a number that silently resets. **Durable**, because the count is the
useful half of the row and removing it would have cost the operator the one
thing that says whether the peak means anything yet.
`PeakStore.banked_count(month)` asks the database `SELECT COUNT(*) FROM
quarter_blocks WHERE month=?`, and `PeakTracker.recorded` is a property over it
rather than an attribute. Three consequences, and all three are the defects
above:
* a restart re-reads the same rows, so the figure survives it;
* the count is per **month**, so it agrees with the peak it is printed beside;
* a block banked by `restore()` is a row like any other and is counted.
Three things this route deliberately does **not** do:
* **it does not count `len(blocks_in(month))`.** `blocks_in()` returns `[]` on a
read error — safe there only because no caller reads its emptiness as a peak,
which its own docstring says — so taking its length would rebuild the
disarming zero one call further out. `banked_count()` returns **`None`** when
the database will not answer.
* **it does not count `monthly_peak`.** That tier holds one row per *month* and
would answer `1` for every month ever measured — wrong by exactly the amount
that makes it look plausible. The two tables are proven equal on the **peak**
by `recompute_month()`; they were never equal on the **count**.
* **it does not remove the dedupe.** `_seen` is still what stops `cycle()`'s
once-a-second `sync()` re-banking a block that `P1Ingest.blocks` keeps for a
day, and `sync()`'s return value is still the number newly banked. What moved
is only where the *status page* reads its figure. ⚠️ The check that graded the
dedupe used to assert `recorded == 1` — the dedupe and the figure reporting it
were the same variable. Counting rows in `quarter_blocks` cannot see a
re-bank at all (it is a PRIMARY KEY no-op), so that check is now graded on
`sync()`'s **return**, with the store's tally as the second half.
### ⚠️ The unreadable case renders a WORD, not a number
`banked_count()` returns `int | None`; `recorded_in()` renders the `None` as
the module constant `COUNT_UNREADABLE = "an unreadable number of"`, so the
sentence reads
```
... ; an unreadable number of quarter(s) banked
```
`0` was never available. `None` was rejected too: `f"{None} quarter(s) banked"`
renders **"None quarter(s) banked"**, which an operator reads as the same zero.
This is the `PeakReading` trade — no `peak_w_or(default)`, ever — applied to the
coverage figure: a caller that formats it gets a true sentence in both states,
and a caller that does arithmetic on it gets a `TypeError`, which is loud.
"0 quarter(s) banked" remains a **true and reachable** sentence: it is what a
month nothing has been written to says, and part 11 asserts that the two states
are spelled differently.
---
## 2. Decision: at EQUAL wattage, the EARLIEST quarter keeps the attribution
**First, not most recent.** The reasoning, in the order it matters:
1. **It is not a money question.** The utility bills the wattage; the timestamp
is not part of the bill. So the decision is settled entirely on what the
status row is *for*.
2. **The row answers "when did my monthly peak happen".** An operator reading it
goes and looks at one event — what was running, what the weather was. A
stable answer is the one that can be investigated; "most recent wins"
guarantees the answer moves every time the wattage recurs, and an operator
who looked yesterday is sent somewhere else today for no reason.
3. **It is what one reader already did unconditionally.** `recompute_month()`
is `ORDER BY offtake_avg_w DESC, quarter_start ASC LIMIT 1` — earliest wins.
Choosing "most recent" would have meant changing both readers; choosing
"earliest" meant making the other one say out loud what it was already doing
by accident.
### ⚠️ And the two readers DID diverge, which is why this is a code change
The `monthly_peak` upsert was `WHERE excluded.peak_w > monthly_peak.peak_w` —
strictly greater, so an exact tie never wrote and the row kept whichever tied
block was **written first**. That coincides with *earliest* only while blocks
arrive in time order. Fed four equal 365 W blocks newest-first, measured on the
shipped tree:
```
ascending arrival monthly_peak -> 09:00:00 recompute_month -> 09:00:00 agree=True
DESCENDING arrival monthly_peak -> 09:45:00 recompute_month -> 09:00:00 agree=False
```
That is a divergence in the derived record that part 8 of `test_tariff.py`
exists to prove cannot happen — latent, because nothing in the shipped path
banks out of order today. The upsert now carries the tie term explicitly:
```sql
WHERE excluded.peak_w > monthly_peak.peak_w
OR (excluded.peak_w = monthly_peak.peak_w
AND excluded.quarter_start < monthly_peak.quarter_start)
```
**`peak_w` is untouched.** The `>` term is exactly as it was, no tie can move
the billed number, and SAFETY-07's guard reads the same watt figure it read
before. The second term moves only the timestamp.
### ⚠️ AND THE SOAK'S DRIFT WAS NOT A TIE AT ALL
The ticket's reading — "ties at the same wattage are re-attributed to the most
recent matching quarter" — is **false**, and driving it is what showed that.
Six exactly-equal 365.0 W quarters on the shipped tree moved the attribution
**zero** times, because `>` is strict.
What the soak saw was a genuine new maximum **smaller than the row's own
resolution**. The row renders `{peak_w:.0f}` — whole watts — so 365.0000001 W is
a new peak to the database and the same `365 W` to the reader:
```
q0 fed 365.0 -> renders 365 W, attributed 09:00:00
q1 fed 365.0000001 -> renders 365 W, attributed 09:15:00
q2 fed 365.0000002 -> renders 365 W, attributed 09:30:00
```
**Decision: left alone, deliberately.** It is the instrument behaving correctly
— the peak really did rise — and the alternative, printing more decimal places,
puts noise in the one row an operator reads at the moment the guard engages, to
explain a movement that is real. Both cases are pinned in part 12 so that
neither can be changed by accident, and the distinction is recorded here so the
next reader does not re-diagnose it as a tie.
---
## 3. What this branch did NOT do, and who owns it
**`main.py`'s status row was not touched.** TARIFF-11 holds `main.py`'s status
rows on a concurrent branch, and the standing instruction is to stop rather than
race there. Everything above is in `app/tariff.py`, and the row picks the fix up
without editing, because `self.peak.recorded` is an attribute access and is now
a property.
**The residual, stated with its scope.** `recorded` answers for
`datetime.now()`. The peak beside it follows `checks(when=...)`. The shipped
install has **exactly one** call site — `self.checks()` with no argument, at
`main.py:4359` — where the two are the same month, so **the live page is correct
and durable**. They part company only under the `when=` affordance TEST-17 added
for the suite, and the mismatch there is strictly smaller than the one it
replaces (this month's tally, rather than a process-wide total across all
months).
`PeakTracker.recorded_in(when)` is the scoped form and is shipped. Moving the
row onto it is:
```python
- + f"; {self.peak.recorded} quarter(s) banked"
+ + f"; {self.peak.recorded_in(when)} quarter(s) banked"
```
one token, on one line, inside the block TARIFF-11 owns. `test_tariff.py`
part 10 **pins the present behaviour** — graded against a row count taken
straight from `quarter_blocks`, not against the property that produces it — so
the gap goes red if it changes and cannot be quietly forgotten.
+124 -3
View File
@@ -132,6 +132,18 @@ OBSERVED = "observed" # the store answered and there is a peak
NO_RECORD = "no_record" # the store answered and this month has none yet
NOT_OBSERVED = "not_observed" # the store did not answer. NOT a zero peak.
# TARIFF-13. What the COVERAGE figure reads when the store cannot count.
#
# ⚠️ NOT 0, AND NOT `None`. This is the reading-state argument of TARIFF-01
# applied to the number printed BESIDE the peak rather than to the peak. A
# coverage figure of 0 says "this month has been measured over no quarters at
# all", which is the strongest possible claim that the peak next to it means
# nothing - and `f"{None} quarter(s) banked"` renders "None quarter(s) banked",
# which an operator reads as that same zero. The one honest answer to "how many
# quarters have I banked" when the table will not answer is that it is not
# known, and it has to SAY so in the sentence the operator reads.
COUNT_UNREADABLE = "an unreadable number of"
# --------------------------------------------------------------------------- #
# local time, without a hard dependency on the tz database
@@ -388,6 +400,23 @@ class PeakStore:
" per_phase_json=excluded.per_phase_json"
" WHERE excluded.offtake_avg_w > quarter_blocks.offtake_avg_w",
(start, month, w, pp_json))
# TARIFF-13. THE TIE RULE, DECIDED RATHER THAN EMERGENT:
# at EQUAL wattage the EARLIEST quarter keeps the attribution.
# See DECISIONS/TARIFF-13-the-coverage-of-a-peak.md §2.
#
# `peak_w` itself is still only ever RAISED - the `>` term is
# untouched and no tie can move the billed number - so SAFETY-07
# reads exactly the watt figure it read before. The second term
# moves only the TIMESTAMP the status page attributes it to.
#
# ⚠️ WITHOUT THE SECOND TERM THE TWO READERS DISAGREE, measured:
# `>` alone keeps whichever tied block was WRITTEN FIRST, while
# `recompute_month()`'s `ORDER BY offtake_avg_w DESC,
# quarter_start ASC` keeps the EARLIEST. Those coincide only
# while blocks arrive in time order. Fed four equal 365 W blocks
# newest-first, `monthly_peak` answered 09:45 and the recompute
# answered 09:00 - a divergence in the derived record part 8 of
# test_tariff.py exists to prove cannot happen.
self._db.execute(
"INSERT INTO monthly_peak(month, peak_w, quarter_start, per_phase_json)"
" VALUES(?,?,?,?)"
@@ -395,7 +424,9 @@ class PeakStore:
" peak_w=excluded.peak_w,"
" quarter_start=excluded.quarter_start,"
" per_phase_json=excluded.per_phase_json"
" WHERE excluded.peak_w > monthly_peak.peak_w",
" WHERE excluded.peak_w > monthly_peak.peak_w"
" OR (excluded.peak_w = monthly_peak.peak_w"
" AND excluded.quarter_start < monthly_peak.quarter_start)",
(month, w, start, pp_json))
return True
except sqlite3.Error as err:
@@ -449,6 +480,49 @@ class PeakStore:
quarter_start=datetime.fromisoformat(row[1]),
per_phase_w=pp)
def banked_count(self, month: str) -> int | None:
"""How many quarters this month has banked. `None` when unknowable.
TARIFF-13. THE DEFECT THIS PREVENTS: the status page printed a coverage
figure held in PROCESS MEMORY - `PeakTracker.recorded`, incremented once
per block banked by THIS process - beside a peak read from SQLite. The
peak outlives a restart and the counter does not, so a driven restart
mid-month took a real reading of
2026-09: peak 365 W ... ; 120 quarter(s) banked
to the same peak, the same attributed quarter and `0 quarter(s) banked`,
on both instances independently. The count is what tells an operator how
much of the billing month the number beside it actually covers: 365 W
over 120 quarters and 365 W over 0 quarters are different claims about
the same watt figure, and after a restart the page made the weaker claim
while displaying the stronger one's value. Reading the count from the
same durable tier the peak comes from is what makes the two agree.
⚠️ `None`, NOT 0, and NOT `len(blocks_in(month))`. `blocks_in()` returns
`[]` on a read error - safe there only because no caller reads its
emptiness as a peak (§ its own docstring) - so counting its length would
rebuild the disarming zero one call further out. This asks the database
for a COUNT and refuses to answer when the database refuses.
⚠️ Counts `quarter_blocks`, the tier with one row per closed block, and
NOT `monthly_peak`, which has one row per MONTH and would answer 1 for
every month that has ever been measured. The two tables are proven equal
on the PEAK by `recompute_month()`; they were never equal on the COUNT
and are not meant to be.
"""
if self._db is None:
return None
try:
row = self._db.execute(
"SELECT COUNT(*) FROM quarter_blocks WHERE month=?",
(month,)).fetchone()
except sqlite3.Error as err:
self.last_error = str(err)
_LOG.error("could not count quarters for %s (%s)", month, err)
return None
return int(row[0])
def blocks_in(self, month: str) -> list:
"""(quarter_start, offtake_avg_w) for a month, oldest first. [] on error
is safe here ONLY because no caller reads emptiness as a peak - AC 3's
@@ -539,11 +613,59 @@ class PeakTracker:
self.cap_w = float(cap_w)
self.policy = policy
self.checkpoint_s = float(checkpoint_s)
self.recorded = 0
self.restore_state = "not attempted"
self._seen: set = set()
self._last_ckpt = 0.0
# -- coverage -----------------------------------------------------------
#
# TARIFF-13. `recorded` USED TO BE AN `int` ATTRIBUTE ON THIS OBJECT,
# starting at 0 and incremented in `sync()`. It is a property over the store
# now, and the three things that changes are all defects it was carrying:
#
# * it RESET TO 0 ON EVERY RESTART, beside a peak that did not - see
# `PeakStore.banked_count()` for the driven reading;
# * it was NOT SCOPED TO A MONTH. It counted every block this process had
# banked, so on 1 October the row still read September's tally beside
# October's "no completed quarter recorded yet";
# * it MISSED the block `restore()` banks. A stale block closed from the
# checkpoint goes into `quarter_blocks` and never went through `sync()`,
# so the counter never saw it. Counting the table cannot miss it.
#
# ⚠️ REMOVING THE COUNTER DOES NOT REMOVE THE DEDUPE. `_seen` is still what
# stops `cycle()`'s once-a-second `sync()` re-banking a closed quarter that
# `P1Ingest.blocks` keeps for a DAY, and `sync()`'s RETURN value is still
# the count of blocks newly banked on that call. What changed is only where
# the STATUS PAGE's figure is read from. The check that grades the dedupe
# must therefore grade `sync()`'s return, not this property, which now
# answers 1 from the database however many times the block was offered.
def recorded_in(self, when: datetime | None = None):
"""Quarters banked in `when`'s billing month - `int`, or the sentinel.
⚠️ The return type is `int | str` on purpose, and the `str` is
`COUNT_UNREADABLE`. A caller that formats this into a sentence gets a
true sentence in both cases; a caller that wants to do arithmetic on it
gets a TypeError, which is loud. That is the same trade `PeakReading`
makes by having no `peak_w_or(default)`.
"""
n = self.store.banked_count(month_key(when or datetime.now(timezone.utc)))
return COUNT_UNREADABLE if n is None else n
@property
def recorded(self):
"""Quarters banked in the month being billed TODAY.
⚠️ This is the live status page's reading, where the peak beside it is
also read for `datetime.now()`. `checks(when=X)` for an X in another
month is a test affordance (TEST-17); the row scoped that way gets this
month's count beside that month's peak, which is strictly better than
the process-wide counter it replaces but is still a mismatch.
`recorded_in(when)` is the scoped form the row should read once
`main.py`'s status rows are free to change - TARIFF-11 holds them.
"""
return self.recorded_in()
# -- startup ------------------------------------------------------------
def restore(self, averager, now: datetime | None = None) -> str:
"""Put the open block back. Returns which of the three cases happened."""
@@ -600,7 +722,6 @@ class PeakTracker:
continue
if self.store.record_block(block, self.policy):
self._seen.add(key)
self.recorded += 1
n += 1
if n:
# A block just closed, so the checkpoint's block is gone: clearing it
+359 -14
View File
@@ -2,8 +2,8 @@
python3 test_tariff.py
Ten parts. What each is for, and - where it matters - what it is deliberately
NOT allowed to be tested at:
Twelve parts. What each is for, and - where it matters - what it is
deliberately NOT allowed to be tested at:
1. THE BLOCK ARITHMETIC, against a worked example written out segment by
segment. The fixture is chosen to be non-degenerate on three axes at once:
@@ -33,6 +33,17 @@ NOT allowed to be tested at:
proven equal to the tier it derives from, and proven never to be lowered.
9. AC 3, ON THE REPLAY DATA THAT ACTUALLY EXISTS. See the honesty note below.
10. THE WIRING, through the real `Controller` and its real status page.
11. TARIFF-13: THE COVERAGE FIGURE PRINTED BESIDE THE PEAK, across a REAL
restart - a second `Controller` against the same file. The peak was
always durable; the count of banked quarters was an `int` on the
tracker and reset to 0, so the page asserted "0 quarter(s) banked"
for a month it had measured 120 of. An unreadable store must not
produce a coverage figure of 0 either - part 5's argument, one
column over.
12. TARIFF-13: WHICH QUARTER A TIED PEAK IS ATTRIBUTED TO, decided
(earliest wins) rather than left emergent, and pinned on BOTH
readers with a newest-first fixture - ascending arrival cannot tell
"the earliest quarter" from "the first row written".
------------------------------------------------------------------------------
⚠️ PART 9 IS A TWO-DAY MEASUREMENT AND SAYS SO. THE 30-DAY ANSWER IS ELSEWHERE
@@ -106,9 +117,9 @@ from pathlib import Path
from app.main import Controller
from app.p1 import P1Ingest, QuarterAverager, QUARTER_S, make_sample, split_signed
from app.tariff import (
BILLING_TZ, CHECKPOINT_S, NO_RECORD, NOT_OBSERVED, POLICY_MAX,
POLICY_NET, POLICY_SUM, PeakStore, PeakTracker, _EUTimezone, billed_w,
month_key,
BILLING_TZ, CHECKPOINT_S, COUNT_UNREADABLE, NO_RECORD, NOT_OBSERVED,
POLICY_MAX, POLICY_NET, POLICY_SUM, PeakStore, PeakTracker, _ClosedBlock,
_EUTimezone, billed_w, month_key,
)
fails = []
@@ -932,18 +943,28 @@ check("the closed quarter reached SQLite through the tracker the cycle calls",
# ⚠️ `cycle()` calls sync() ONCE A SECOND against a `blocks` list that keeps a
# closed quarter for a DAY. Without a dedupe that is ~86 000 re-banks of the
# same row per block, and `recorded` - the number the status page prints as
# "quarters banked" - would be a count of loop iterations rather than of
# quarters. The database stays correct either way, which is exactly why this
# needs its own check instead of being caught by the peak.
# same row per block, and a coverage figure derived from the number of times a
# block was OFFERED would be a count of loop iterations rather than of quarters.
# The database stays correct either way, which is exactly why this needs its own
# check instead of being caught by the peak.
#
# ⚠️ GRADED ON `sync()`'s RETURN, and on the STORE's tally for the month the
# quarter is in - TARIFF-13. It used to be graded on `PeakTracker.recorded`,
# which was then an `int` incremented inside this very loop, so the dedupe and
# the figure that reported it were the same variable. They are not any more:
# `recorded_in()` counts rows in `quarter_blocks`, where re-banking the same
# `quarter_start` is a PRIMARY KEY no-op and could never have shown up as a
# second quarter. A check written against it alone would now pass however many
# times sync() re-banked, which is why the return value is the load-bearing
# half of this assertion and is asserted first.
again = c.peak.sync(c.p1)
third = c.peak.sync(c.p1)
print(f" sync() called three times over the same closed quarter: "
f"{n}, {again}, {third} newly banked; tracker reports "
f"{c.peak.recorded} quarter(s)")
f"{n}, {again}, {third} newly banked; the store holds "
f"{c.peak.recorded_in(B)} quarter(s) for {month_key(B)}")
check("...and syncing the same quarter again banks nothing new, so the count "
"the status page shows is quarters and not cycles",
again == 0 and third == 0 and c.peak.recorded == 1)
again == 0 and third == 0 and c.peak.recorded_in(B) == 1)
check("...and the peak is unchanged by the repeat",
near(c.peak.peak(B).peak_w, 4998.0, 1e-6))
@@ -957,11 +978,33 @@ peak_rows = [r for r in rows if "peak" in r["text"] and "quarter(s) banked" in r
print(" status row: " + (peak_rows[0]["text"] if peak_rows else "(none)"))
check("the status page carries exactly one capacity-tariff peak row",
len(peak_rows) == 1)
check("...naming the peak, the soft limit and the quarters banked, all derived "
check("...naming the peak, the soft limit and a coverage figure, all derived "
"from the controller that was actually built",
bool(peak_rows) and "4998 W" in peak_rows[0]["text"]
and f"{SHIPPED_CAP_W:.0f} W soft limit" in peak_rows[0]["text"]
and "1 quarter(s) banked" in peak_rows[0]["text"])
and "quarter(s) banked" in peak_rows[0]["text"])
# ⚠️ AND THE COVERAGE FIGURE IN THAT ROW IS SCOPED TO THE LIVE BILLING MONTH,
# NOT TO `when` - TARIFF-13, and it is a KNOWN GAP stated rather than rounded
# off. `main.py`'s row reads `self.peak.recorded`, which answers for
# `datetime.now()`; the peak beside it follows `when`. The two agree on the
# ONLY call site a shipped install has - `checks()` with no argument, at
# main.py:4359 - which is why the live page is correct and durable; they part
# company only under this suite's `when=` affordance. `PeakTracker` already
# carries the scoped form, `recorded_in(when)`, and moving the row onto it is a
# one-token edit inside main.py's status-row block, which TARIFF-11 holds. This
# check PINS the present behaviour so the gap cannot be forgotten, and part 11
# is the authority on the figure itself.
_live_month = month_key(datetime.now(timezone.utc))
_live_rows = len(c.peak.store.blocks_in(_live_month))
print(f" the row read at {month_key(B)} carries the tally for "
f"{_live_month} ({_live_rows} quarter(s)), which is this fixture's "
f"known scoping gap")
check("the row's coverage figure is the LIVE month's tally - graded against a "
"row count taken straight from quarter_blocks, so this pins the gap "
"rather than restating the property that produces it",
bool(peak_rows)
and f"; {_live_rows} quarter(s) banked" in peak_rows[0]["text"])
check("...and it is FLAGGED, because 4998 W is over §9.1's 80 % warning point "
f"on a {SHIPPED_CAP_W:.0f} W limit",
bool(peak_rows) and not peak_rows[0]["ok"] and peak_rows[0]["warn"])
@@ -1149,6 +1192,308 @@ check(f"...and the same IMPORT grader over main.py finds {_main_imports}, so "
bool(_main_imports))
# ==============================================================================
print()
print("11. TARIFF-13: what the instrument says about its OWN COVERAGE, "
"across a restart")
# THE DEFECT THIS PART PREVENTS. The status row prints a peak read from SQLite
# beside a count of banked quarters, and the count used to be an `int` attribute
# on `PeakTracker` that started at 0 in `__init__`. A 40-hour soak caught what
# that means at a driven restart, on both instances independently:
#
# before: 2026-09: peak 365 W, set by the quarter at 2026-09-11T18:45:00+00:00
# - 10 % of the 3500 W soft limit; 120 quarter(s) banked
# after : 2026-09: peak 365 W, set by the quarter at 2026-09-11T18:45:00+00:00
# - 10 % of the 3500 W soft limit; 0 quarter(s) banked
#
# The peak and its attributed quarter persist; the coverage beside them does
# not. §9.1's peak is what the utility bills, but the COUNT is what says how
# much of the billing month the figure covers - 365 W over 120 quarters and
# 365 W over 0 quarters are different claims about the same watt figure, and
# after a restart the page made the weaker claim while showing the stronger
# one's value. TARIFF-01's argument is that an unreadable store must not read as
# a zero peak because a zero peak is DISARMING; a zero COVERAGE beside a live
# peak is the same error one column over.
#
# ⚠️ THE RESTART HERE IS A REAL ONE - a second `Controller` against the same
# file on disk, never a re-read of the first object. 11b asserts that the second
# object has banked nothing itself, so the number it prints cannot have come
# from process memory. Without that precondition this whole part is satisfied by
# a counter that was simply never reset.
_T13 = TMP / "t13-restart.db"
# The fixture lives in the month being billed TODAY, and the row is read with NO
# `when` argument - the LIVE status page, the surface that misinformed. That
# also makes the check stable on every day of the year, which a fixed calendar
# date is not (TEST-17, one part up, is the same lesson).
_now13 = datetime.now(timezone.utc)
_mon13 = month_key(_now13)
# Anchor on the first 900 s boundary of the billing month, so all N quarters are
# inside `_mon13` however close to a month edge the suite happens to run.
_ms = datetime(int(_mon13[:4]), int(_mon13[5:]), 1, tzinfo=timezone.utc)
_q0 = datetime.fromtimestamp(
(int(_ms.timestamp()) // QUARTER_S + 1) * QUARTER_S, timezone.utc)
_N13 = 6 # > 1, so "count == rows" cannot be satisfied by 0 == 0
_W13 = 1234.0
c13 = built(_T13)
for _q in range(_N13):
_base = _q0 + timedelta(seconds=QUARTER_S * _q)
for _i in range(0, QUARTER_S + 1, 5):
c13.p1.submit(sample(_base + timedelta(seconds=_i), _W13))
c13.peak.sync(c13.p1)
# Ground truth from the OTHER tier, which the tracker did not supply - the count
# under test must be graded against something that is not itself (habit 2).
_truth_before = len(c13.peak.store.blocks_in(_mon13))
_row_before = _peak_rows(c13.checks())[0]["text"]
print(" before restart: " + _row_before)
check(f"precondition: the fixture banked {_N13} quarters in {_mon13}, so a "
f"count of 0 and a count of the truth are different numbers",
_truth_before == _N13 and _N13 > 1)
check(f"...and the row says so before the restart",
f"{_N13} quarter(s) banked" in _row_before)
c13.peak.store.close()
# THE RESTART.
c13b = built(_T13)
_row_after = _peak_rows(c13b.checks())[0]["text"]
_truth_after = len(c13b.peak.store.blocks_in(_mon13))
print(" after restart: " + _row_after)
print(f" rows in quarter_blocks: {_truth_before} before, "
f"{_truth_after} after; the restarted tracker has banked "
f"{len(c13b.peak._seen)} block(s) of its own through sync()")
check("precondition: the second Controller is a REAL restart - it shares no "
"object with the first and has banked nothing through sync(), so any "
"count it prints came off the disk and not out of memory",
c13b.peak is not c13.peak and len(c13b.peak._seen) == 0)
check("the peak and its attributed quarter survive the restart - the durable "
"half, asserted so that the coverage claim below is not graded against a "
"reading that had already collapsed",
_row_before.split(" - ")[0] == _row_after.split(" - ")[0]
and f"{_W13:.0f} W" in _row_after)
check("...and the COUNT BESIDE IT DOES NOT COLLAPSE TO ZERO, which is the "
"defect: a coverage figure that resets while the value beside it "
"persists actively misinforms",
"; 0 quarter(s) banked" not in _row_after)
check(f"...and it is the store's own tally, {_truth_after}, not a "
f"process-lifetime counter - graded against a COUNT OF ROWS in the "
f"other tier rather than against the property under test",
f"{_truth_after} quarter(s) banked" in _row_after and _truth_after >= _N13)
# ⚠️ The restarted run banks one MORE than it drove, and that is correct: the
# checkpointed open block was stale, so `restore()` closed and banked it. The
# in-memory counter never saw that block at all - it only ever counted what went
# through `sync()` - so this is a third way the old figure under-read, and it is
# asserted rather than tolerated. Driven at a chosen `now` so it cannot depend
# on when the suite runs.
_t13s = fresh_store("t13-stale")
_tr13 = PeakTracker(_t13s, cap_w=SHIPPED_CAP_W)
_av13 = QuarterAverager(phases=1)
_sb = _q0 + timedelta(seconds=QUARTER_S * 40)
for _i in range(0, 600, 5):
list(_av13.add(sample(_sb + timedelta(seconds=_i), 2000.0)))
_t13s.save_partial(_av13.snapshot())
_before_stale = _tr13.recorded_in(_sb)
_state = _tr13.restore(QuarterAverager(phases=1),
now=_sb + timedelta(seconds=QUARTER_S * 3))
print(f" stale-block restore: {_state}")
print(f" quarters banked in {month_key(_sb)}: {_before_stale} -> "
f"{_tr13.recorded_in(_sb)}")
check("a block closed by restore() rather than by sync() is counted too - the "
"old counter incremented only in sync() and missed it entirely",
_before_stale == 0 and _tr13.recorded_in(_sb) == 1
and "banked" in _state)
# --- the count is scoped to the MONTH it is printed beside --------------------
# ⚠️ TWO MONTHS AT DIFFERENT COUNTS. Equal counts would be satisfied by a
# cross-month total, by a constant, and by reading either month for the other.
_t13m = fresh_store("t13-months")
_MA = datetime(2026, 4, 6, 9, 0, tzinfo=timezone.utc) # 2026-04
_MB = datetime(2026, 5, 6, 9, 0, tzinfo=timezone.utc) # 2026-05
_trm = PeakTracker(_t13m, cap_w=SHIPPED_CAP_W)
for _m, _k in ((_MA, 3), (_MB, 5)):
for _q in range(_k):
_t13m.record_block(_ClosedBlock(
start=_m + timedelta(seconds=QUARTER_S * _q),
offtake_avg_w=900.0 + _q, per_phase_offtake_avg_w=None))
print(f" banked by month: {month_key(_MA)}={_trm.recorded_in(_MA)}, "
f"{month_key(_MB)}={_trm.recorded_in(_MB)}, "
f"a month never written={_trm.recorded_in(datetime(2026, 7, 6, 9, 0, tzinfo=timezone.utc))}")
check("precondition: the two months were given DIFFERENT numbers of quarters, "
"so a count that answered either one for the other would be visible here",
month_key(_MA) != month_key(_MB))
check("each month reports ITS OWN quarters and not the pair's total - the old "
"counter was per PROCESS, so on 1 October the row still showed "
"September's tally beside October's empty month",
_trm.recorded_in(_MA) == 3 and _trm.recorded_in(_MB) == 5)
check("...and a month with no quarters reports 0, which is TRUE of a month "
"nothing was written to - the sentinel below is for a store that will "
"not answer, and the two states must not be spelled the same way",
_trm.recorded_in(datetime(2026, 7, 6, 9, 0, tzinfo=timezone.utc)) == 0)
# --- and a store that cannot answer does NOT report zero ----------------------
# The other half of part 5's argument, applied to the coverage figure. `0
# quarter(s) banked` and `an unreadable number of quarter(s) banked` are
# opposite states: the first says the month has been measured over nothing, the
# second says the instrument cannot say. Spelling them the same way is exactly
# what this card was raised about.
_t13u = PeakStore(str(TMP / "no-such-dir" / "nope13.db"))
_tru = PeakTracker(_t13u, cap_w=SHIPPED_CAP_W)
print(f" unreadable store: banked_count -> {_t13u.banked_count(_mon13)!r}, "
f"rendered -> {_tru.recorded_in(_now13)!r}")
check("an unreadable store answers None to a COUNT, never 0 - the store-level "
"half",
_t13u.banked_count(_mon13) is None)
check("...and the rendered figure says it is unreadable instead of naming a "
"number, so the sentence cannot read as a measured zero",
_tru.recorded_in(_now13) == COUNT_UNREADABLE
and f"{_tru.recorded_in(_now13)} quarter(s) banked"
== "an unreadable number of quarter(s) banked")
check("...and that sentence is not merely NON-zero by accident: the string a "
"count of 0 would produce is different text",
f"{_tru.recorded_in(_now13)} quarter(s) banked"
!= f"{0} quarter(s) banked")
# ⚠️ A STORE THAT IS OPEN AND WHOSE COUNT QUERY FAILS IS A SECOND CODE PATH, and
# the checks above cannot reach it - a handle that never opened short-circuits
# on `self._db is None` before any SQL runs, so `banked_count()`'s
# `except sqlite3.Error` arm was executed by nothing. Found by this card's own
# mutation battery, not by review: returning 0 instead of None from that arm
# survived the whole suite at -3 B. It is the same third-path argument part 5
# makes for `peak()`, and it is the arm a half-corrupt database on a real
# install actually takes.
_t13b = fresh_store("t13-broken")
for _q in range(4):
_t13b.record_block(_ClosedBlock(start=_q0 + timedelta(seconds=QUARTER_S * _q),
offtake_avg_w=800.0 + _q,
per_phase_offtake_avg_w=None))
_trb = PeakTracker(_t13b, cap_w=SHIPPED_CAP_W)
_count_ok = _t13b.banked_count(_mon13)
_t13b._db.execute("DROP TABLE quarter_blocks")
_t13b._db.commit()
_count_broken = _t13b.banked_count(_mon13)
print(f" open store, count query FAILS: {_count_ok} -> {_count_broken!r}, "
f"rendered -> {_trb.recorded_in(_now13)!r}")
check(f"precondition: before the table was dropped the same call answered "
f"{_count_ok}, so 'it returns nothing now' is a change and not the "
f"store's resting state",
_count_ok == 4)
check("a store whose COUNT QUERY fails answers None too - a different code "
"path from a handle that never opened, and the one a half-corrupt "
"database takes",
_count_broken is None)
check("...and it renders as unreadable rather than as 0, so a half-corrupt "
"database cannot print a coverage figure that reads like a measured "
"month with nothing in it",
_trb.recorded_in(_now13) == COUNT_UNREADABLE)
# --- the count reads the BLOCK tier, not the month tier -----------------------
# `monthly_peak` holds one row per MONTH, so counting it answers 1 for every
# month that has ever been measured - a figure that is wrong by exactly the
# amount that makes it look plausible.
check("the coverage figure counts quarter_blocks and not monthly_peak, which "
"would answer 1 for every measured month however many quarters it holds",
_trm.recorded_in(_MB) == 5
and _t13m.peak(_MB).is_observed and _t13m.recompute_month(month_key(_MB)).is_observed)
# ==============================================================================
print()
print("12. TARIFF-13: which quarter a TIED peak is attributed to")
# THE DECISION, recorded in DECISIONS/TARIFF-13-the-coverage-of-a-peak.md §2:
# at equal wattage the EARLIEST quarter keeps the attribution. The wattage is
# what is billed, so this is not a money question - it is the answer to "when
# did my monthly peak happen", and an operator investigating that goes and looks
# at one event. A stable answer is the useful one; "most recent wins" guarantees
# the answer moves every time the wattage recurs.
#
# ⚠️ AND THE SOAK'S DRIFT WAS NOT A TIE. The 40-hour run showed 365 W attributed
# to three different quarters with no restart and no new peak, and the obvious
# reading - "ties re-attribute to the most recent" - is false, measured: the
# upsert's term is strictly `>`, so an exact tie never wrote. What moved was a
# genuine new maximum SMALLER THAN THE ROW'S OWN RESOLUTION. The row renders
# `{peak_w:.0f}`, whole watts; 365.0000001 W is a new peak to the database and
# the same "365 W" to the reader. Both cases are pinned below, because the fix
# for one is not the fix for the other.
_t13t = fresh_store("t13-ties")
_TB = datetime(2026, 3, 9, 9, 0, tzinfo=timezone.utc)
def _bank(store, q, w):
store.record_block(_ClosedBlock(start=_TB + timedelta(seconds=QUARTER_S * q),
offtake_avg_w=w, per_phase_offtake_avg_w=None))
def _qt(n):
return _TB + timedelta(seconds=QUARTER_S * n)
for _q in (0, 1, 2, 3):
_bank(_t13t, _q, 365.0)
_asc = _t13t.peak(_TB)
print(f" four EXACTLY equal 365 W quarters, oldest first -> attributed "
f"{_asc.quarter_start.isoformat()}")
check("an exact tie keeps the EARLIEST quarter: the attribution does not move "
"when the same wattage recurs",
_asc.quarter_start == _qt(0) and near(_asc.peak_w, 365.0))
# ⚠️ FED NEWEST-FIRST, which is what separates "the earliest quarter" from
# "whichever row was written first". Ascending arrival cannot tell those apart -
# they are the same row - so an ascending-only fixture grades neither.
_t13r = fresh_store("t13-ties-rev")
for _q in (3, 2, 1, 0):
_bank(_t13r, _q, 365.0)
_desc = _t13r.peak(_TB)
_desc_rc = _t13r.recompute_month(month_key(_TB))
print(f" the same four fed NEWEST first -> monthly_peak attributed "
f"{_desc.quarter_start.isoformat()}, recompute_month attributed "
f"{_desc_rc.quarter_start.isoformat()}")
check("...and it is the earliest QUARTER, not the first row WRITTEN - fed "
"newest-first the answer is still the oldest quarter",
_desc.quarter_start == _qt(0))
check("...and BOTH readers give it, so the derived record has not diverged "
"from the tier it derives from on the tie rule either (part 8's "
"invariant, at equal wattage)",
_desc_rc.quarter_start == _desc.quarter_start
and near(_desc_rc.peak_w, _desc.peak_w))
# The other direction, and the reason the tie rule may never touch `peak_w`: a
# STRICTLY HIGHER quarter must still take the attribution, whenever it arrives.
_t13h = fresh_store("t13-higher")
for _q, _w in ((0, 365.0), (1, 366.0), (2, 365.0)):
_bank(_t13h, _q, _w)
_hi = _t13h.peak(_TB)
print(f" 365, then 366, then 365 -> peak {_hi.peak_w:.0f} W at "
f"{_hi.quarter_start.isoformat()}")
check("a strictly higher quarter still takes both the peak and the "
"attribution - the tie rule moves the TIMESTAMP and never the watts",
near(_hi.peak_w, 366.0) and _hi.quarter_start == _qt(1))
# And the mechanism the soak actually saw: an increase below the row's own
# resolution. Not a tie, and no tie rule can address it.
_t13d = fresh_store("t13-subdisplay")
_seen13 = []
for _q, _w in enumerate((365.0, 365.0000001, 365.0000002)):
_bank(_t13d, _q, _w)
_seen13.append(_t13d.peak(_TB).quarter_start)
print(f" 365.0, 365.0000001, 365.0000002 W all render "
f"{_t13d.peak(_TB).peak_w:.0f} W and attribute to "
f"{[s.time().isoformat() for s in _seen13]}")
check("an increase BELOW the row's whole-watt resolution is a real new maximum "
"and does move the attribution - so the soak's drifting quarter was a "
"rounded display, not a tie, and is left alone deliberately",
len(set(_seen13)) == 3
and len({f"{w:.0f}" for w in (365.0, 365.0000001, 365.0000002)}) == 1)
# ==============================================================================
print()
shutil.rmtree(TMP, ignore_errors=True)