Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d57122ec74 | ||
|
|
4601682815 | ||
|
|
9234016419 | ||
|
|
b7272169b1 | ||
|
|
2b8b8cb97f | ||
|
|
8e589fee3f | ||
|
|
bdd7f18a54 | ||
|
|
a1376e44e4 | ||
|
|
e709554857 | ||
|
|
a30bde1ab1 |
@@ -2,6 +2,175 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
**OPT-04's AC4 is met: twelve months of real EPEX BE day-ahead prices are now in
|
||||
the repository, and the strategy has been replayed against them (DATA-02).**
|
||||
AC4 asked for "net-positive across 12 months of replayed prices". OPT-04 shipped
|
||||
it as NOT MET and replayed a synthetic year instead, because `scenarios/` held
|
||||
only P1 meter captures. That was the right call at the time and the wrong
|
||||
diagnosis of why: the criterion was being carried alongside a 30-day *meter*
|
||||
criterion as though both needed waiting, and EPEX BE day-ahead is **public data,
|
||||
not a capture**. It needed a fetch, not a month.
|
||||
|
||||
`scenarios/epex-be-dayahead-2025-09-01_2026-09-01.json` — **8760 hourly points**,
|
||||
2025-09-01 to 2026-09-01 Brussels-local, in §9.3's canonical three-field shape.
|
||||
`scenarios/fetch_epex_be.py` produced it and `--verify` re-derives every claim it
|
||||
makes about itself without touching the network.
|
||||
|
||||
**Licensing was checked before anything was committed.** The source is
|
||||
Fraunhofer ISE's Energy-Charts API, which publishes the **BE** zone under
|
||||
**CC BY 4.0 from Bundesnetzagentur | SMARD.de** and other zones under
|
||||
private-use-only terms — so this zone is redistributable with attribution and the
|
||||
data is committed rather than left as a script plus a checksum. The fetcher
|
||||
**refuses to write a file** whose per-response `license_info` stops saying
|
||||
CC BY 4.0. Cross-checked against the upstream: SMARD's own `4996/DE` series
|
||||
matched this feed on all 96 quarter-hours of 2026-04-10, maximum difference
|
||||
0.00 EUR/MWh.
|
||||
|
||||
**The figures are raw wholesale and the file says so.** `"all_in": false`, with
|
||||
the markup/tax assumption carried in `provenance.all_in_assumption` as *numbers
|
||||
to apply* rather than numbers already applied — TARIFF-02's double-counting trap
|
||||
in data form. `test_arbitrage.py` section 7B applies it in one visible place and
|
||||
**asserts arithmetically** that its two additive terms cancel out of every daily
|
||||
spread, so the trade counts depend on the VAT multiplier alone and not on the
|
||||
guessed markup. The same replay is run on raw wholesale as well, so how much of
|
||||
the answer rests on the guess is reported (110 trades all-in, 88 raw).
|
||||
|
||||
**DST is handled by construction, not by a special case.** The fetcher iterates
|
||||
in UTC and renders through `price.brussels()`, so **2025-10-26 comes out 25
|
||||
points and 2026-03-29 comes out 23** — verified by counting per Brussels day
|
||||
rather than asserting 24, and by checking that October's 02:00 appears twice at
|
||||
`+02:00` and `+01:00` with two different prices while March's 02:00 does not
|
||||
appear at all. The window is bounded at *local* midnight: bounding it at 00:00
|
||||
UTC produced a 22-hour first day, and a 24-hour day silently missing from the
|
||||
end, both of which the point-and-gap accounting caught.
|
||||
|
||||
**The result, at the shipped default, whatever it is.** At
|
||||
`cycle_cost_threshold_eur` **0.10** widened by FCST-03's no-weather **1.5×** to a
|
||||
required 0.1500 EUR/kWh, the real year completes **110 round trips** and is
|
||||
**net-positive at +24.88 EUR** of benefit net of the cycle cost (2 kWh a trade).
|
||||
Unwidened it completes 173. **Nothing was tuned:** the shipped default and the
|
||||
2.5/1.5 factors are untouched, and every figure comes out of the same `decide()`.
|
||||
|
||||
**What the product owner's open decision now has under it.** The four
|
||||
independent reports that a static day/night spread of ~0.0840 does not clear
|
||||
0.10 **all stand** — a static-tariff site still never trades. What was missing
|
||||
was the other half: a *dynamic* site's daily spread clears the widened
|
||||
requirement on **121 of 365 days**, median daily all-in spread **0.1130**. The
|
||||
threshold at which a median day would clear is **0.0754**, and at that value the
|
||||
same year completes 156 trips for +35.42 EUR. Both numbers are reported; neither
|
||||
is applied.
|
||||
|
||||
**And the synthetic year stays, because the real one cannot grade §17.** Not one
|
||||
of the 365 real days has a negative *dearest* hour — 372 negative wholesale hours
|
||||
and 0 days where the strategy would have sold into one — so "0 discharges into a
|
||||
negative-price hour" over real prices is a count of something that could not
|
||||
happen, and deleting the guard would not change it. Section 7B asserts that
|
||||
degeneracy instead of quietly reporting the zero; section 7's generator is still
|
||||
the only thing here that puts a negative price where the strategy would sell.
|
||||
|
||||
**Size, measured rather than assumed.** Plain JSON: 893 KB in the working tree
|
||||
but **74.6 KB as a git object**, against 56.5 KB for a committed `.json.gz`. Git
|
||||
already compresses blobs, so gzipping saves 18 KB and costs a reviewer the
|
||||
ability to read the provenance block — including the licence claim they are
|
||||
meant to check — in the diff. Plain, and `load()` still accepts `.gz` for a
|
||||
future dataset that earns it.
|
||||
|
||||
**Nothing is wired into the add-on.** `generic_dayahead` remains the live fetch
|
||||
path; nothing under `app/` imports the script, and the dataset is for replay.
|
||||
|
||||
**The capacity-peak guard's forced discharge now has an EMS-side energy floor
|
||||
(SAFETY-24).** SAFETY-07 can command a discharge to hold a billed quarter down,
|
||||
and nothing below it consulted the state of charge — `soc_floor` belongs to the
|
||||
maintenance cycle, and freeze.py already recorded that for the grid-following
|
||||
loop "the BMS is the sole energy backstop". On the 2026-08-20 replay the guard
|
||||
asks for **878 Wh in a day**. Relying on the BMS is not absurd — it is a real
|
||||
floor and it works — but it is the pack's protection rather than the operator's,
|
||||
and a month of aggressive quarters leaves the battery repeatedly at that cut-out
|
||||
with no EMS-side record of why.
|
||||
|
||||
`decide()` now takes `soc` and `soc_floor`, both **required** and neither
|
||||
defaulted. Above the floor nothing changes. At or below it the guard files a
|
||||
**0 W** floor instead of a sized one — one-directional, so charging stays blocked
|
||||
and no discharge is capped — and reports `floor_limited` with the reason it
|
||||
bound. It is not a trip: every stop in §11.2's table still outranks the claim,
|
||||
and the floor can never be negative, so §9.1's "discharge and export are never
|
||||
capped by the peak guard" survives as a property of the object (swept: 0 negative
|
||||
floors across 1296 combinations of load, elapsed, battery, SoC and floor).
|
||||
|
||||
**Failing to shave is not silent.** A floor-limited cycle raises one notification
|
||||
per episode (`goodwe_peak_soc_floor`), distinct from the saturation alarm because
|
||||
the two ask the operator for opposite things, and renders its own status row.
|
||||
An **unreadable SoC is floor-limited, not a full battery** — a full battery is
|
||||
exactly the value that would disarm the floor, and this project has shipped that
|
||||
defaulting once before (`| float(100)` in the stage-5 loop).
|
||||
|
||||
**Verified by replay, not by argument.** The 2026-08-20 capture is driven against
|
||||
a modelled pack sized from the day's own measured appetite, so it genuinely
|
||||
exhausts: without the floor it reaches the BMS cut-out and sits there for 1227
|
||||
guard cycles; with it the pack stops at 20.0 % and never touches the BMS. The
|
||||
energy withheld equals the energy between the two floors to within one cycle at
|
||||
the rail. The honest half is reported too — the worst quarter goes from 4338 W to
|
||||
5030 W, because a floor-limited guard shaves less on purpose.
|
||||
|
||||
**Two new options, mirrored once.** `peak_soc_floor` (20) and `bms_soc_floor`
|
||||
(10), whose defaults live in `peakguard.PEAK_OPTION_DEFAULTS` — TARIFF-02's
|
||||
pattern — imported by `validate.py` and `main.py` rather than typed as literals,
|
||||
and pinned to `config.yaml` by a check that reads the file. CFG-03 still tracks
|
||||
three places. `validate.py` rule 8 refuses `peak_soc_floor <= bms_soc_floor` at
|
||||
startup, in CFG-01/MAINT-04's manner. `maintenance_soc_floor` is deliberately
|
||||
**not** reused and deliberately **not** renamed: it is a drain target, not a
|
||||
spend limit, and the rename is recorded as a decision to revisit if a third
|
||||
floor ever appears.
|
||||
|
||||
**Price arbitrage - the first strategy that trades on a price (OPT-04).**
|
||||
Four cards built the price stack and nothing traded on it. `app/arbitrage.py`
|
||||
is §10.1's priority 3: charge in the cheapest published hour, discharge in the
|
||||
dearest, **and only when the spread covers `cycle_cost_threshold_eur`**. It is
|
||||
off by default (`arbitrage_enabled: false`) - it is the first strategy in this
|
||||
add-on that commands the battery against the meter on purpose.
|
||||
|
||||
**It consumes the four cards rather than re-deciding anything.** Whether a
|
||||
trade is permitted at all, on which two hours, and against which widened
|
||||
threshold is `TariffPlan.arbitrage()`'s answer (TARIFF-03), and that answer
|
||||
cannot be obtained without first stating whether the peak guard is holding. So
|
||||
§9.1's "give peak shaving priority over arbitrage" is honoured
|
||||
**structurally**: there is no code path to a tradeable spread that skips the
|
||||
question. And because the claim is filed at `P_ARBITRAGE` (30) while peakguard
|
||||
files its floor at `P_PEAK` (50), the arbiter clamps an arbitrage charge even
|
||||
if that interlock were bypassed. Two independent mechanisms, neither
|
||||
re-implemented here.
|
||||
|
||||
**A guard that never ran is not a guard that is clear.** On `meter_source: off`
|
||||
there is no telegram stream, so the peak guard is never evaluated. That third
|
||||
state is carried as `None` and it **refuses**: charging at the rail creates the
|
||||
billed quantity while nothing measures the quarter it lands in. `validate.py`
|
||||
refuses that configuration at startup too, along with arbitrage armed on a site
|
||||
with no price source at all, and a SoC floor at or above the ceiling.
|
||||
|
||||
**§17: never export into a negative price.** The guard is on the hour being
|
||||
sold into, not on the spread - a day priced -0.30 ... -0.05 has a 0.25 EUR/kWh
|
||||
spread that clears any shipped threshold and a dearest hour that is still
|
||||
negative. Buying in a negative hour stays allowed; being paid to charge is the
|
||||
right trade.
|
||||
|
||||
**Prices are fetched in the background, never on the tick.** `resolve_tariff()`
|
||||
may open an HTTPS connection, and the control tick is what feeds the ESP32
|
||||
heartbeat - a supplier's TLS timeout there is a supplier able to push the
|
||||
add-on towards layer 1's 30 s veto. A separate task refreshes every 30 minutes;
|
||||
the tick reads an attribute and an age bound, and a plan whose refresh has
|
||||
stopped answering retires rather than standing as if it were current.
|
||||
|
||||
**What is honestly not measured.** The acceptance criterion asked for
|
||||
net-positivity across 12 months of *replayed* prices. This repository holds no
|
||||
price history - `scenarios/` is P1 meter data - so that criterion is **not met,
|
||||
and is not restated**. What `test_arbitrage.py` section 7 does instead is
|
||||
replay a synthetic year and assert the property that does not need real prices:
|
||||
every executed round trip cleared its own widened requirement, on all of them,
|
||||
across a year containing 522 hours priced below zero and zero discharges into
|
||||
one. And at the shipped 0.10 EUR/kWh threshold a realistic Belgian day/night
|
||||
site **does not trade at all** - reported, not tuned. That number, and the
|
||||
widening factors beside it, are one product decision and it is open.
|
||||
|
||||
**Provisional prices, and what may not rest on one (FCST-03).**
|
||||
Day-ahead prices arrive once a day, around 14–16 h CET. Until they do, §8.3 says
|
||||
the unknown hours use yesterday's profile and are flagged `provisional: true` —
|
||||
|
||||
+92
-10
@@ -420,7 +420,7 @@ help - see "Why the tuning is what it is".
|
||||
| `maintenance_start_hour` | 10 | Hour of day a due cycle begins |
|
||||
| `maintenance_discharge_w` | 2000 | Drain rate (exports the surplus). Above `max_w` it is **clamped** to `max_w` — the drain phase SETs the setpoint and the arbiter bounds it, saying so on the status page (`limited by supervised(max_w)`). Warned about, not refused: the cycle still completes, just slower |
|
||||
| `maintenance_charge_w` | 2000 | Charge ceiling, capped again by peak headroom. Must stay **at or below `max_w`** or the cheap-window shaping (`hi = -maintenance_charge_w`) contradicts the supervised limit (`lo = -max_w`), the arbiter resolves it to **0 W**, and maintenance silently never charges in a cheap window — **the add-on refuses to start** |
|
||||
| `maintenance_soc_floor` | 11 | Drain target — stay just above the inverter's own floor |
|
||||
| `maintenance_soc_floor` | 11 | Drain target — stay just above the inverter's own floor. ⚠️ **This is not `peak_soc_floor`.** It is the depth the monthly recalibration deliberately goes to, once, on purpose; `peak_soc_floor` is the depth a money-saving discharge may never reach on any day. The right values differ (11 vs 20 as shipped), which is why they are two keys |
|
||||
| `maintenance_soc_target` | 99 | Charge target |
|
||||
| `maintenance_hold_min` | 120 | Hold at full so the BMS can balance |
|
||||
|
||||
@@ -430,6 +430,8 @@ help - see "Why the tuning is what it is".
|
||||
|---|---|
|
||||
| `peak_forecast_entity` | Quarter-hour demand forecast, for capacity-tariff markets. Empty = no cap |
|
||||
| `peak_cap_w` | The site's capacity-tariff target |
|
||||
| `peak_soc_floor` (20) | **The percent below which the capacity-peak guard will not discharge the battery.** Not optional and not the same thing as `maintenance_soc_floor` — see below |
|
||||
| `bms_soc_floor` (10) | The pack's **own** cut-out percent, the number the BMS enforces whatever you set above. `peak_soc_floor` must be **strictly above** it or the add-on refuses to start |
|
||||
| `price_now_entity`, `price_avg_entity` | Dynamic tariff. Empty = never force a paid grid top-up |
|
||||
|
||||
On a capacity-tariff site the maintenance charge is capped by the headroom left
|
||||
@@ -530,11 +532,46 @@ projected over — raises one notification (`goodwe_peak_saturated`), because th
|
||||
is the one state where the money will be spent and shedding load is the only
|
||||
thing left that changes the bill.
|
||||
|
||||
**No new option.** The ceiling is `peak_cap_w`, which already shipped. The
|
||||
**No new option for the CEILING.** It is `peak_cap_w`, which already shipped. The
|
||||
coverage threshold is `meter_max_age_s`, the freeze predicate is
|
||||
`meter_freeze_s`, and the achievable discharge comes from `max_w` and the device
|
||||
rating — every one of them a key that already exists.
|
||||
|
||||
#### The energy floor under the forced discharge (`peak_soc_floor`)
|
||||
|
||||
The guard above can **command a discharge**, and until SAFETY-24 nothing in the
|
||||
add-on bounded how far down it took the pack — the BMS was the only backstop.
|
||||
On the 2026-08-20 replay the guard asks for **878 Wh in a day**.
|
||||
|
||||
That is not absurd: the BMS floor is real and it works. But it is *the pack's*
|
||||
protection, not *yours*. A month of aggressive quarters can leave the battery
|
||||
repeatedly parked at that cut-out with no record on this side of why, and the
|
||||
first symptom — a battery that is never available when you want it — looks
|
||||
exactly like the SoC drift the monthly maintenance cycle exists to correct.
|
||||
|
||||
- **`peak_soc_floor` (20 %)** is where the forced discharge stops. Below it the
|
||||
guard files a **0 W** floor instead of a sized one: charging stays blocked
|
||||
(that can only ever lower your offtake), and no further discharge is asked for.
|
||||
- **It does not stop the ordinary self-consumption loop.** That loop still has no
|
||||
SoC guard of its own. This bounds what the *peak guard* spends for money.
|
||||
- **When it binds you are told**, once per episode, with notification
|
||||
`goodwe_peak_soc_floor` and a status-page row reading `PEAK GUARD
|
||||
FLOOR-LIMITED`: the quarter **will** go over the ceiling, so shedding load is
|
||||
the only thing left that changes the bill. Failing to shave is never silent —
|
||||
the euro figure is the whole point of the guard.
|
||||
- **An unreadable SoC is treated as floor-limited, not as a full battery.** A
|
||||
full battery is exactly the value that would disarm this floor.
|
||||
- **`bms_soc_floor` (10 %)** is the pack's own cut-out — register `0x0560` on the
|
||||
reference GW5000S-BP, consistent with DoD 90. It exists so the add-on can
|
||||
**refuse to start** when `peak_soc_floor` is at or below it: such a floor can
|
||||
never be the thing that stops the discharge, and a guard that advertises a
|
||||
backstop it does not have is the one thing this project will not ship. Equal is
|
||||
refused too — equal buys zero margin.
|
||||
|
||||
Why 20 %: the reference pack is 13.8 kWh usable, so 10 points of margin over the
|
||||
BMS cut-out is about 1.6× the guard's worst measured day. If your pack is
|
||||
smaller, or your quarters are worse, raise it.
|
||||
|
||||
### The winding-loop alarm (the output clamp)
|
||||
|
||||
Every setpoint this add-on writes is clamped to `max_w` before it leaves the
|
||||
@@ -644,15 +681,16 @@ floor of 60 s that figure is 60 W and only the margin is left.
|
||||
|
||||
### Energy price providers
|
||||
|
||||
> **No *fetched* price moves energy in 0.5.x.** This release ships the
|
||||
> *plumbing*: the five provider plugins, the canonical all-in format they emit,
|
||||
> and the config that selects one. Turning a provider on fetches, validates and
|
||||
> publishes prices and changes no charging decision; the arbitrage strategy that
|
||||
> would use it is a later ticket.
|
||||
> **A fetched price moves energy only when you switch arbitrage on.** This
|
||||
> release ships the five provider plugins, the canonical all-in format they
|
||||
> emit, the config that selects one, and — new in OPT-04 — a price-arbitrage
|
||||
> strategy that trades on them. With `arbitrage_enabled: false`, which is the
|
||||
> shipped default, turning a provider on still fetches, validates and publishes
|
||||
> prices and changes no charging decision.
|
||||
>
|
||||
> The one exception is `static_profile` below. A static day/night tariff needs
|
||||
> no network, so its cheap window is computed on every cycle and does reach the
|
||||
> maintenance cycle's grid top-up.
|
||||
> `static_profile` below needs no network at all, so its cheap window is
|
||||
> computed on every cycle and reaches the maintenance cycle's grid top-up
|
||||
> whether or not arbitrage is armed.
|
||||
|
||||
| option | meaning |
|
||||
|---|---|
|
||||
@@ -804,6 +842,50 @@ to, and the site simply has no tariff until the feed returns. It is not required
|
||||
— the add-on warns rather than refusing — but filling it in is what makes that
|
||||
fallback work.
|
||||
|
||||
### Price arbitrage (`arbitrage_enabled`)
|
||||
|
||||
§10.1 priority 3: charge in the cheapest published hour, discharge in the
|
||||
dearest, **and only when the spread covers what the round trip costs.** Off by
|
||||
default — this is the first strategy in the add-on that commands the battery
|
||||
*against* the meter on purpose, so it is armed deliberately or not at all.
|
||||
|
||||
| option | meaning |
|
||||
|---|---|
|
||||
| `arbitrage_enabled` | `false` by default. Arms the strategy |
|
||||
| `cycle_cost_threshold_eur` | What one full round trip costs per kWh — pack wear plus conversion losses. A spread below this is not traded |
|
||||
| `arbitrage_soc_floor` | The reserve arbitrage may not spend, % |
|
||||
| `arbitrage_soc_ceiling` | The headroom it may not overfill, % |
|
||||
|
||||
**What it needs, and what the add-on refuses.** Arbitrage requires a price
|
||||
source (`price_provider` or `static_profile`) and a P1 meter (`meter_source`).
|
||||
Without a price source there is nothing to tell cheap from expensive; without
|
||||
P1 the capacity-tariff peak guard never runs, and §9.1 puts peak shaving above
|
||||
arbitrage — a charge that nothing measures is exactly what that guard exists to
|
||||
prevent. Both combinations are **refused at startup** rather than left looking
|
||||
armed.
|
||||
|
||||
**Peak shaving always wins.** The strategy cannot obtain a spread at all
|
||||
without first answering whether the peak guard is holding, and even if it
|
||||
could, the guard's floor is filed at a higher arbiter priority and clamps an
|
||||
arbitrage charge. Discharging is never capped by it — only offtake is billed.
|
||||
|
||||
**It never exports into a negative price** (§17). A negative hour is the best
|
||||
possible hour to *charge* in and that is still allowed; selling into one gives
|
||||
energy away and pays for the privilege.
|
||||
|
||||
**At the shipped threshold a static day/night site does not trade.** The
|
||||
realistic Belgian day/night spread is about 0.08 EUR/kWh and the default
|
||||
threshold is 0.10, so the honest answer on such a site is "no opportunity", and
|
||||
the status page says so with the numbers. Whether 0.10 is the right default is
|
||||
a product decision, not a bug: lower it only if you know what a cycle costs
|
||||
your pack.
|
||||
|
||||
**The prices are fetched in the background**, never on the control tick — a
|
||||
supplier's TLS timeout on that path would be a supplier able to push the add-on
|
||||
towards the firmware's 30 s failsafe. The plan refreshes every 30 minutes and
|
||||
retires itself if the refresh stops answering, at which point arbitrage stops
|
||||
with it.
|
||||
|
||||
### Site
|
||||
|
||||
| option | meaning |
|
||||
|
||||
@@ -36,6 +36,15 @@ P_SAFETY = 100 # stopped, inputs missing, shutting down, device limits
|
||||
P_MAINTENANCE = 60 # maintenance owns the actuator outright (drain, hold)
|
||||
P_PEAK = 50 # capacity-tariff peak shaving - costs real money
|
||||
P_MAINT_SHAPE = 40 # maintenance shaping the loop (charge-only)
|
||||
# OPT-04. §10.1 priority 3 - price arbitrage. ⚠️ ITS POSITION IS THE SPEC, NOT A
|
||||
# preference: P_PEAK (50) is ABOVE it so clause 2 lets the capacity-tariff floor
|
||||
# clamp an arbitrage charge, which is §9.1's "give peak shaving priority over
|
||||
# arbitrage whenever the two conflict" expressed as arithmetic rather than as a
|
||||
# rule somebody has to remember. P_MAINT_SHAPE (40) is above it so a speculative
|
||||
# spread cannot override the pack's balancing cycle. P_LOOP (10) is below it
|
||||
# because trading against the meter is the entire point - the grid-following
|
||||
# loop would otherwise cancel every charge the moment it created import.
|
||||
P_ARBITRAGE = 30 # price arbitrage - trades against the meter on purpose
|
||||
P_LOOP = 10 # ordinary grid-following
|
||||
|
||||
SET = "set"
|
||||
|
||||
@@ -0,0 +1,429 @@
|
||||
"""OPT-04 - §10.1 priority 3: charge cheap, discharge expensive, and only when
|
||||
the round trip is worth more than it costs.
|
||||
|
||||
WHERE THIS LIVES, AND WHY IT IS NOT WHERE THE CARD SAYS. The functional notes
|
||||
on the Trello card reference specs.md §2.2's `ems/optimizer/...` layout. That
|
||||
module layout does not exist and is not being built; the product owner's
|
||||
decision of 2026-08-30 is that everything ships in `goodwe_controller/app/`,
|
||||
beside price.py, static_tariff.py, peakguard.py and arbiter.py. The BEHAVIOUR
|
||||
§10.1 / §9.1 / §8.3 / §17 describes is unchanged and still authoritative.
|
||||
|
||||
|
||||
WHAT THIS MODULE IS, AND - MORE IMPORTANTLY - WHAT IT IS NOT
|
||||
============================================================
|
||||
Four cards landed before this one and this one CONSUMES all four. Almost every
|
||||
question an arbitrage strategy has to answer is already answered somewhere
|
||||
else, by an object built for it, and the single biggest way to get this ticket
|
||||
wrong is to answer any of them again here:
|
||||
|
||||
price.py (TARIFF-02) what a price IS, all-in, and what a product is.
|
||||
`Uncertainty`, `WIDEN_FACTOR`, `required_spread`.
|
||||
static_tariff.py (TARIFF-03) `TariffPlan.arbitrage()` - THE VERDICT. Whether
|
||||
a trade is permitted at all, on which two hours,
|
||||
against which widened threshold. Five refusal
|
||||
reasons live there, not here.
|
||||
price_forecast.py (FCST-03) provisional carrying, and the rule that no
|
||||
irreversible action rests on a provisional
|
||||
price (§8.3).
|
||||
peakguard.py (SAFETY-07) the capacity-tariff floor, filed as a `lo`
|
||||
LIMIT at P_PEAK on the arbiter.
|
||||
|
||||
So what is left for this file is genuinely small, and that is the point:
|
||||
|
||||
* WHICH HOUR IS IT NOW - is this instant the buy hour, the sell hour, or
|
||||
neither.
|
||||
* MAY THE BATTERY DO IT - state of charge, and §17's negative-price rule.
|
||||
* HOW HARD - one number, filed as one arbiter claim.
|
||||
|
||||
Everything else is delegated, and delegated by CONSTRUCTION rather than by
|
||||
convention: `TariffPlan.arbitrage()` will not hand over a spread at all unless
|
||||
it is told whether the peak guard is holding, what a cycle costs, and what is
|
||||
uncertain. There is no code path here that can reach a tradeable number without
|
||||
having answered §10.1's priority ordering first.
|
||||
|
||||
|
||||
AC3 - PEAK SHAVING WINS, AND IT WINS TWICE
|
||||
==========================================
|
||||
§9.1: "Give peak shaving priority over arbitrage whenever the two conflict."
|
||||
That is honoured through the two mechanisms that already exist, and NEITHER of
|
||||
them is re-implemented here:
|
||||
|
||||
1. THE INTERLOCK. `plan.arbitrage(peak_guard_active=...)` is keyword-only and
|
||||
REQUIRED. When the guard is engaged the verdict is a refusal carrying the
|
||||
§10.1 wording, so `decide()` below never sees a spread at all. It is not
|
||||
possible to write a caller here that skips the question - the argument has
|
||||
no default and the spread is unreachable without it.
|
||||
|
||||
2. THE ARBITER. Even if (1) were somehow bypassed, `claim()` files a SET at
|
||||
P_ARBITRAGE (30) and peakguard files a LIMIT at P_PEAK (50). arbiter.py
|
||||
clause 2 applies every limit whose priority is >= the winning set's, most
|
||||
restrictive wins - so the peak floor CLAMPS an arbitrage charge whatever
|
||||
this module thinks. 50 >= 30 is what makes that true, and it is why the
|
||||
priority below sits under P_PEAK rather than beside it.
|
||||
|
||||
Two independent mechanisms for one rule is deliberate. The first is the one
|
||||
that reports honestly; the second is the one that holds when the first is
|
||||
wrong.
|
||||
|
||||
⚠️ AND THE GUARD MUST BE ABLE TO SEE. On `meter_source: off` there is no
|
||||
telegram stream, so peakguard.decide() is never run and `peak_guard_active` is
|
||||
not FALSE - it is UNKNOWN. Charging a battery at max_w creates offtake, offtake
|
||||
is the billed quantity (§9.1), and doing it deliberately while nothing measures
|
||||
the quarter is the exact shape of failure §9.1 exists to prevent. So the third
|
||||
state is carried explicitly as `None` and it REFUSES. A failed read is not an
|
||||
observation, and "the guard did not run" is not "the guard is clear".
|
||||
|
||||
|
||||
§17's NEGATIVE-PRICE CASE, STATED ONCE
|
||||
======================================
|
||||
"Negative-price hours - arbitrage must not export." A negative price means the
|
||||
market pays the site to consume. Discharging the battery into such an hour
|
||||
gives away stored energy AND forgoes being paid to take more in; if the house
|
||||
load is covered it becomes export, which at a negative price is paid-for
|
||||
disposal. The buy side is untouched - a negative hour is the best possible hour
|
||||
to charge in, and this file's guard is deliberately asymmetric for that reason.
|
||||
|
||||
⚠️ THE GUARD IS ON THE HOUR BEING SOLD INTO, NOT ON THE SPREAD. A series whose
|
||||
prices are -0.30 and -0.05 has a 0.25 EUR/kWh spread that clears any shipped
|
||||
threshold, and its DEAREST hour is still negative. A guard written as
|
||||
"spread > 0" or "the prices are not all negative" passes that case and exports.
|
||||
|
||||
|
||||
WHAT `cycle_cost_threshold_eur` IS, AND WHAT THIS FILE REFUSES TO DO WITH IT
|
||||
===========================================================================
|
||||
§10.1 gates arbitrage on "expected round-trip benefit exceeds
|
||||
`cycle_cost_threshold_eur`". It ships at 0.10 EUR/kWh. On realistic Belgian
|
||||
day/night numbers the static spread is about 0.08 EUR/kWh, and FCST-03's
|
||||
widening factors (2.5 real-time, 1.5 no-weather) are honest policy defaults
|
||||
with nothing to derive them from.
|
||||
|
||||
⚠️ SO A STATIC-TARIFF SITE DOES NOT TRADE AT SHIPPED DEFAULTS, AND THAT IS
|
||||
REPORTED RATHER THAN TUNED AWAY. The threshold is a parameter here, the
|
||||
behaviour is correct at any value of it, and no number in this file was chosen
|
||||
to make a trade happen. Whether 0.10 is the right shipped default - and whether
|
||||
2.5/1.5 are - is one product decision and it belongs to the owner, not to this
|
||||
card. test_arbitrage.py section 7 reports what the shipped default does over a
|
||||
replayed year rather than adjusting anything so that it looks better.
|
||||
|
||||
|
||||
THE ACCOUNTING BASIS, AND THE TERM THAT IS DELIBERATELY ABSENT
|
||||
=============================================================
|
||||
The card's technical notes ask for `benefit = value_out(d) - cost_in(c)/η`, and
|
||||
then say of the round-trip efficiency: "η is NOT a §14 key - it needs one, or
|
||||
must be derived from measured charge/discharge energy in the state store. DO
|
||||
NOT HARDCODE IT." Both halves are honoured by not having the term at all:
|
||||
|
||||
* THE SELL SIDE IS PRICED AS DISPLACED IMPORT, NOT AS EXPORT. The two hours
|
||||
compared are both all-in CONSUMPTION prices, which is the basis the notes
|
||||
require ("credit displacement at all-in import prices, not at export price,
|
||||
or the number flatters itself"). `export_price_eur_kwh` is therefore not
|
||||
read here at all - §9.3's near-zero Belgian injection price is precisely
|
||||
why the economically sound trade on this market is charge-cheap-to-
|
||||
displace-expensive-import, and §17's guard below is the same fact stated as
|
||||
a rule.
|
||||
|
||||
* AND THERE IS NO η, BECAUSE THERE IS NOTHING HONEST TO PUT IN IT. No config
|
||||
key exists, the state store carries no measured charge/discharge energy
|
||||
pair to derive one from, and a hardcoded 0.85 is exactly what the notes
|
||||
forbid - it would also DOUBLE-COUNT, because `cycle_cost_threshold_eur` is
|
||||
already defined as what a round trip costs per kWh and conversion loss is
|
||||
part of that cost. When a measured η exists it belongs beside the threshold
|
||||
as a second term, and this file's arithmetic is one subtraction, in one
|
||||
place, ready for it.
|
||||
|
||||
|
||||
HYSTERESIS (§10.2)
|
||||
==================
|
||||
The switching decision is "which hour is it", and an hour is 60 minutes wide,
|
||||
so the action cannot chatter within one. The other two terms are monotone
|
||||
inside an hour in the only direction that matters: a charge raises SoC towards
|
||||
the ceiling and a discharge lowers it towards the floor, so each can stop once
|
||||
and cannot restart in the same hour. No timer is needed and none is invented.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
|
||||
from .arbiter import P_ARBITRAGE, Claim
|
||||
from .price import Absence, price_option
|
||||
|
||||
_LOG = logging.getLogger("goodwe.arbitrage")
|
||||
|
||||
SOURCE = "arbitrage"
|
||||
|
||||
# §10.2 reason codes. Written out because the action log is a user-facing
|
||||
# artefact and a code assembled from an f-string cannot be grepped for.
|
||||
CHARGE = "ARBITRAGE_CHARGE"
|
||||
DISCHARGE = "ARBITRAGE_DISCHARGE"
|
||||
|
||||
# Why nothing was done. These are the reasons this MODULE owns; the five
|
||||
# refusals that belong to the verdict itself are carried through verbatim from
|
||||
# static_tariff.py rather than restated, so a refusal's wording has exactly one
|
||||
# author.
|
||||
IDLE_NO_PLAN = "no tariff plan has been resolved yet"
|
||||
IDLE_NOT_COVERED = (
|
||||
"NOT COVERED: meter_source is off, so the capacity-tariff peak guard never "
|
||||
"runs and nothing measures the quarter this charge would land in. §9.1 puts "
|
||||
"peak shaving above arbitrage, and an unmeasured peak cannot be shaved - "
|
||||
"so the honest answer is that arbitrage is not covered on this install, not "
|
||||
"that the guard is clear")
|
||||
IDLE_NO_SOC = (
|
||||
"state of charge is not being read, and a battery whose fill nobody knows "
|
||||
"cannot be told to empty or to fill")
|
||||
IDLE_OFF_PEAK_HOUR = "this hour is neither the buy hour nor the sell hour"
|
||||
NEGATIVE_PRICE = (
|
||||
"§17: the sell hour is priced BELOW ZERO, so discharging into it would give "
|
||||
"energy away and pay for the privilege. Arbitrage must not export at a "
|
||||
"negative price")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ArbitrageDecision:
|
||||
"""What the strategy concluded this cycle, with what it concluded it from.
|
||||
|
||||
⚠️ NO FIELD HAS A DEFAULT, and that is TEST-14's rule rather than a style
|
||||
choice: an outcome type's required field quietly gaining a default is a
|
||||
mutation that survived review in three files in this repository. The
|
||||
signature is scanned in test_price_forecast.py alongside `Gate`'s and
|
||||
`Arbitrage`'s constructors, so a default added here later goes red without
|
||||
anybody remembering to write a case for it.
|
||||
|
||||
`action` is None whenever nothing is to be done, and `power_w` is then None
|
||||
too - not 0.0. Zero watts is a COMMAND (it parks the battery); "no claim"
|
||||
is the absence of one, and the arbiter must be free to let the loop win.
|
||||
"""
|
||||
|
||||
action: str | None # CHARGE | DISCHARGE | None
|
||||
power_w: float | None # signed: negative charges, positive discharges
|
||||
reason: str
|
||||
spread_eur_kwh: float | None
|
||||
required_eur_kwh: float | None
|
||||
price_now_eur_kwh: float | None
|
||||
|
||||
@property
|
||||
def trading(self) -> bool:
|
||||
return self.action is not None
|
||||
|
||||
def explain(self) -> str:
|
||||
if not self.trading:
|
||||
return f"arbitrage idle - {self.reason}"
|
||||
return self.reason
|
||||
|
||||
|
||||
def _idle(reason, *, spread=None, required=None, price=None) -> ArbitrageDecision:
|
||||
if not reason or not isinstance(reason, str):
|
||||
raise ValueError(
|
||||
"an idle arbitrage decision needs a reason a human can act on - "
|
||||
"§10.2 requires every plan change to carry one, and 'arbitrage did "
|
||||
"nothing' with no reason is indistinguishable from a bug.")
|
||||
return ArbitrageDecision(action=None, power_w=None, reason=reason,
|
||||
spread_eur_kwh=spread, required_eur_kwh=required,
|
||||
price_now_eur_kwh=price)
|
||||
|
||||
|
||||
def enabled(opts: dict) -> bool:
|
||||
"""Is the strategy switched on at all? Off in the shipped configuration.
|
||||
|
||||
⚠️ OFF BY DEFAULT ON PURPOSE. Every other card in this repository ships its
|
||||
machinery inert and lets an operator arm it; this one MOVES ENERGY against
|
||||
the meter on a real battery, so it is the last one that should be an
|
||||
exception to that.
|
||||
"""
|
||||
v = price_option(opts, "arbitrage_enabled")
|
||||
return v is True or str(v).strip().lower() in ("true", "yes", "on", "1")
|
||||
|
||||
|
||||
def threshold_eur_kwh(opts: dict) -> float:
|
||||
"""§10.1's `cycle_cost_threshold_eur`, from the one place option defaults live."""
|
||||
v = float(price_option(opts, "cycle_cost_threshold_eur"))
|
||||
if not math.isfinite(v) or v < 0:
|
||||
raise ValueError(
|
||||
f"cycle_cost_threshold_eur is {v!r}. It is what a round trip COSTS, "
|
||||
f"in EUR/kWh, so it is non-negative and finite - a negative "
|
||||
f"threshold is one every spread clears, which is arbitrage with the "
|
||||
f"brake removed.")
|
||||
return v
|
||||
|
||||
|
||||
def soc_bounds(opts: dict) -> tuple:
|
||||
"""(floor, ceiling) in percent. The reserve arbitrage may not spend."""
|
||||
return (float(price_option(opts, "arbitrage_soc_floor")),
|
||||
float(price_option(opts, "arbitrage_soc_ceiling")))
|
||||
|
||||
|
||||
def guard_state(peak_decision):
|
||||
"""The peak guard's answer as the three-state `decide()` takes: T / F / None.
|
||||
|
||||
THE ONE PLACE THIS CONJUNCTION IS WRITTEN, and it is public for the reason
|
||||
`peakguard.shaping_suspended()`, `watchdog.loop_is_following()` and
|
||||
`freeze.looks_frozen()` are: the check that grades it has to assert on THE
|
||||
SHIPPED EXPRESSION. A predicate transcribed into its own test is one copy
|
||||
plus a future disagreement about what the check means, and the copy that
|
||||
drifts is always the grading one - this repository has three separate
|
||||
records of exactly that, one of which left every suite green while a
|
||||
default install parked at 0 W in the quarter that sets the month's bill.
|
||||
|
||||
⚠️ `None` IS NOT `False` AND COLLAPSING THE TWO IS THE WHOLE HAZARD. On
|
||||
`meter_source: off` main.py never runs peakguard.decide() - there is no
|
||||
telegram stream, so there is no open quarter to project - and the decision
|
||||
it hands here is None. That is "the guard never ran", not "the guard is
|
||||
clear", and answering False would arm a strategy whose charging creates the
|
||||
billed quantity (§9.1) while nothing measures the quarter it lands in.
|
||||
"""
|
||||
if peak_decision is None:
|
||||
return None
|
||||
return bool(peak_decision.engaged)
|
||||
|
||||
|
||||
def _point_at(plan, when: datetime):
|
||||
"""The price point covering `when`, or None when the series does not cover it.
|
||||
|
||||
⚠️ None IS NOT A PRICE AND IS NOT ZERO. `TariffPlan` raises `Absence` here
|
||||
rather than answering 0.0, for price.py's reason - "no data" becoming "free"
|
||||
is the defect the whole outcome-type family exists to remove - and this
|
||||
turns that into "do nothing", never into "it is cheap".
|
||||
"""
|
||||
try:
|
||||
return plan.read.match(series=lambda r: r.at(when),
|
||||
no_data=lambda _r: None)
|
||||
except Absence:
|
||||
return None
|
||||
|
||||
|
||||
def decide(plan, now: datetime, *, peak_guard_active, soc,
|
||||
min_spread_eur_kwh, uncertainty, charge_w, discharge_w,
|
||||
soc_floor_pct, soc_ceiling_pct) -> ArbitrageDecision:
|
||||
"""Should the battery trade this cycle, and how hard.
|
||||
|
||||
⚠️ `peak_guard_active` IS A THREE-STATE ARGUMENT AND ALL THREE ARE REAL.
|
||||
True (the guard is holding), False (it ran and is clear), and None (it never
|
||||
ran, because nothing measures this install's quarter). See the module note:
|
||||
None refuses, because the alternative is a strategy that creates the billed
|
||||
quantity while the thing that would have limited it is switched off.
|
||||
|
||||
⚠️ EVERY OTHER ARGUMENT IS KEYWORD-ONLY AND HAS NO DEFAULT, for
|
||||
`TariffPlan.arbitrage()`'s reason. A defaulted `soc_floor_pct` is a reserve
|
||||
floor this module asserted rather than one the operator configured, and the
|
||||
first time it is wrong the battery is at 0 %.
|
||||
"""
|
||||
if peak_guard_active is None:
|
||||
return _idle(IDLE_NOT_COVERED)
|
||||
if plan is None:
|
||||
return _idle(IDLE_NO_PLAN)
|
||||
|
||||
verdict = plan.arbitrage(peak_guard_active=peak_guard_active,
|
||||
min_spread_eur_kwh=min_spread_eur_kwh,
|
||||
uncertainty=uncertainty)
|
||||
return verdict.match(
|
||||
refused=lambda v: _idle(v.reason),
|
||||
allowed=lambda v: _act(plan, now, v, soc, charge_w, discharge_w,
|
||||
soc_floor_pct, soc_ceiling_pct))
|
||||
|
||||
|
||||
def _act(plan, now, v, soc, charge_w, discharge_w, floor_pct, ceiling_pct):
|
||||
"""The permitted case: which hour is this, and may the pack do it."""
|
||||
here = _point_at(plan, now)
|
||||
if here is None:
|
||||
return _idle(
|
||||
f"the series carries a tradeable spread but no price for this "
|
||||
f"instant ({now.isoformat()}), which is §11.4 for this hour rather "
|
||||
f"than a reason to reach for a neighbouring one",
|
||||
spread=v.spread_eur_kwh, required=v.required_spread_eur_kwh)
|
||||
|
||||
price = here.eur_per_kwh
|
||||
buying = here.start == v.cheapest.start
|
||||
selling = here.start == v.dearest.start
|
||||
if not (buying or selling):
|
||||
return _idle(IDLE_OFF_PEAK_HOUR, spread=v.spread_eur_kwh,
|
||||
required=v.required_spread_eur_kwh, price=price)
|
||||
|
||||
# ⚠️ BELT AND BRACES ON §8.3, AND IT IS NOT DEAD CODE. `_firm_or_die` makes
|
||||
# the two points on the verdict firm at construction, but the point matched
|
||||
# HERE is whatever `at()` returned for this instant, and the identity of the
|
||||
# two rests on the series being the one the verdict was built from. An hour
|
||||
# that is provisional is one nobody published a price for, and no
|
||||
# irreversible action may rest on it.
|
||||
if here.provisional:
|
||||
return _idle(
|
||||
f"§8.3: the price for this hour ({here.start.isoformat()}) is "
|
||||
f"CARRIED FORWARD, not published, and no irreversible action may be "
|
||||
f"taken on a provisional price",
|
||||
spread=v.spread_eur_kwh, required=v.required_spread_eur_kwh,
|
||||
price=price)
|
||||
|
||||
if soc is None:
|
||||
return _idle(IDLE_NO_SOC, spread=v.spread_eur_kwh,
|
||||
required=v.required_spread_eur_kwh, price=price)
|
||||
|
||||
common = dict(spread=v.spread_eur_kwh, required=v.required_spread_eur_kwh,
|
||||
price=price)
|
||||
|
||||
if selling:
|
||||
# §17. THE NAMED REGRESSION CASE. Checked BEFORE the SoC question, so a
|
||||
# negative price reports the reason that actually forbids the trade
|
||||
# rather than one that happens to fire first on some other site.
|
||||
if price < 0:
|
||||
return _idle(f"{NEGATIVE_PRICE} - this hour is "
|
||||
f"{price:+.4f} EUR/kWh", **common)
|
||||
if soc <= floor_pct:
|
||||
return _idle(
|
||||
f"the sell hour is here at {price:.4f} EUR/kWh but the pack is "
|
||||
f"at {soc:g} %, at or under the {floor_pct:g} % reserve floor "
|
||||
f"arbitrage may not spend (§10.1 priority 4)", **common)
|
||||
return ArbitrageDecision(
|
||||
action=DISCHARGE, power_w=abs(float(discharge_w)),
|
||||
reason=(f"{DISCHARGE}: this is the dearest firm hour at "
|
||||
f"{price:.4f} EUR/kWh, {v.spread_eur_kwh:.4f} above the "
|
||||
f"cheapest and clear of the {v.required_spread_eur_kwh:.4f} "
|
||||
f"this cycle had to cover; discharging "
|
||||
f"{abs(float(discharge_w)):.0f} W from {soc:g} %"),
|
||||
spread_eur_kwh=v.spread_eur_kwh,
|
||||
required_eur_kwh=v.required_spread_eur_kwh,
|
||||
price_now_eur_kwh=price)
|
||||
|
||||
if soc >= ceiling_pct:
|
||||
return _idle(
|
||||
f"the buy hour is here at {price:.4f} EUR/kWh but the pack is at "
|
||||
f"{soc:g} %, at or over the {ceiling_pct:g} % ceiling - there is "
|
||||
f"nowhere to put the cheap energy", **common)
|
||||
return ArbitrageDecision(
|
||||
action=CHARGE, power_w=-abs(float(charge_w)),
|
||||
reason=(f"{CHARGE}: this is the cheapest firm hour at "
|
||||
f"{price:.4f} EUR/kWh, {v.spread_eur_kwh:.4f} below the dearest "
|
||||
f"and clear of the {v.required_spread_eur_kwh:.4f} this cycle "
|
||||
f"had to cover; charging {abs(float(charge_w)):.0f} W from "
|
||||
f"{soc:g} %"),
|
||||
spread_eur_kwh=v.spread_eur_kwh,
|
||||
required_eur_kwh=v.required_spread_eur_kwh,
|
||||
price_now_eur_kwh=price)
|
||||
|
||||
|
||||
def claim(decision: ArbitrageDecision) -> Claim | None:
|
||||
"""The decision as one arbiter claim, or None when nothing is to be traded.
|
||||
|
||||
A SET at P_ARBITRAGE, and the priority is the whole §10.1 ordering:
|
||||
|
||||
P_PEAK 50 > this, so peakguard's `lo` floor CLAMPS an arbitrage
|
||||
charge (arbiter clause 2 applies every limit at or
|
||||
above the winner's priority). AC3, mechanically.
|
||||
P_MAINT_SHAPE 40 > this, so a maintenance charge window's `hi` shaping is
|
||||
not overridden by a price opportunity. §12.4's "money
|
||||
outranks the schedule" is about the PEAK guard, which
|
||||
is measured and billed; a speculative spread is not
|
||||
that, and letting it out-rank the pack's balancing
|
||||
cycle would be this card quietly rewriting §12.
|
||||
P_LOOP 10 < this, because trading against the meter is the point:
|
||||
the grid-following loop would otherwise cancel every
|
||||
charge the moment it created import.
|
||||
|
||||
⚠️ NONE MEANS NO CLAIM, NOT A 0 W CLAIM. Filing `Claim.set(..., 0.0)` here
|
||||
would out-rank the loop with a park order every cycle the strategy had
|
||||
nothing to say, i.e. it would silently disable self-consumption on any site
|
||||
that armed arbitrage.
|
||||
"""
|
||||
if not decision.trading or decision.power_w is None:
|
||||
return None
|
||||
return Claim.set(SOURCE, P_ARBITRAGE, float(decision.power_w),
|
||||
decision.action)
|
||||
@@ -28,10 +28,11 @@ import contextlib
|
||||
import json
|
||||
import logging
|
||||
import signal
|
||||
from datetime import datetime, timezone
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
import aiohttp
|
||||
|
||||
from . import arbitrage
|
||||
from .arbiter import (
|
||||
P_LOOP, P_MAINT_SHAPE, P_MAINTENANCE, P_PEAK, P_SAFETY, Claim, resolve,
|
||||
)
|
||||
@@ -48,7 +49,8 @@ from .maintenance import IDLE, MaintConfig, Maintenance
|
||||
from .mqtt import MqttPublisher
|
||||
from . import peakguard
|
||||
from .p1 import P1Ingest, build_source, is_enabled
|
||||
from .static_tariff import static_status
|
||||
from .price import Uncertainty, price_option
|
||||
from .static_tariff import resolve_tariff, static_status
|
||||
from .tariff import PeakStore, PeakTracker
|
||||
from .validate import ConfigRefused, refuse, validate_startup_config
|
||||
from .watchdog import SignWatchdog, SignWatchdogConfig, loop_is_following
|
||||
@@ -57,6 +59,34 @@ from . import web
|
||||
OPTIONS_PATH = "/data/options.json"
|
||||
_LOG = logging.getLogger("goodwe")
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# OPT-04: THE BACKGROUND TARIFF REFRESH, AND WHY IT IS NOT OPTIONAL
|
||||
#
|
||||
# `static_tariff.resolve_tariff()` may perform an HTTPS fetch. The control tick
|
||||
# is what feeds the ESP32 heartbeat, and layer 1 commands 0 W if that heartbeat
|
||||
# stops for ~30 s - so a supplier's TLS timeout on the tick is a SUPPLIER able
|
||||
# to push this add-on into the failsafe. Every module in the price stack says
|
||||
# so in its own words (static_tariff.static_plan, resolve_tariff, main's
|
||||
# read_inputs) and this is the task those notes were waiting for.
|
||||
#
|
||||
# The tick therefore never awaits a fetch. It reads `self.tariff_plan`, an
|
||||
# attribute this task assigns, and nothing else.
|
||||
#
|
||||
# ⚠️ THE CADENCE IS NOT A GUESS AT HOW OFTEN PRICES CHANGE. A day-ahead feed
|
||||
# publishes ONCE, at 14-16 h CET, and §8.3's carried hours become firm at that
|
||||
# moment; 30 minutes is short enough that the switch is picked up inside the
|
||||
# hour it happens and long enough to be a polite 48 fetches a day. Nothing in
|
||||
# the control law depends on it.
|
||||
TARIFF_REFRESH_S = 1800.0
|
||||
# ⚠️ AND THE AGE BOUND IS NOT THE SAME QUESTION. A refresh that FAILS still
|
||||
# stores a plan - resolve_tariff() degrades to the §11.4 static fallback, which
|
||||
# is provisional and refuses arbitrage on its own. This bound covers the other
|
||||
# case: the refresh task itself dying, leaving a plan that was true hours ago
|
||||
# standing as if it were current. Three missed refreshes is the threshold, for
|
||||
# `p1.max_age_s`'s reason - one missed poll is not a fault, a run of them is.
|
||||
# A failed read is not an observation, and neither is a stale one.
|
||||
TARIFF_MAX_AGE_S = 3 * TARIFF_REFRESH_S
|
||||
|
||||
|
||||
def load_options() -> dict:
|
||||
try:
|
||||
@@ -199,14 +229,25 @@ class Controller:
|
||||
# the action log and the notification fire on transitions rather than
|
||||
# once per cycle for a quarter of an hour.
|
||||
#
|
||||
# ⚠️ NO CONFIG OF ITS OWN, for the reason the clamp watchdog gives ~90
|
||||
# lines up and TARIFF-01 repeats: the ceiling is `peak_cap_w`, already
|
||||
# shipped and already validated, and every other term is read off
|
||||
# something the arbiter or the averager already owns. CFG-03 still
|
||||
# tracks three places, not four.
|
||||
# ⚠️ THE CEILING HAS NO CONFIG OF ITS OWN, for the reason the clamp
|
||||
# watchdog gives ~90 lines up and TARIFF-01 repeats: it is `peak_cap_w`,
|
||||
# already shipped and already validated, and every other term of the
|
||||
# projection is read off something the arbiter or the averager already
|
||||
# owns.
|
||||
#
|
||||
# ⚠️ SAFETY-24'S FLOOR DOES HAVE ONE, AND IT COULD NOT NOT HAVE. How far
|
||||
# down the guard may spend the pack for money is a number only the site
|
||||
# knows, and refusing one set below the pack's own cut-out means knowing
|
||||
# that cut-out too. So this card adds exactly two keys and reads their
|
||||
# defaults from `peakguard.PEAK_OPTION_DEFAULTS` - TARIFF-02's mirror
|
||||
# pattern - rather than typing literals here. THAT is what keeps CFG-03
|
||||
# at three places: the divergence validate.py's rule 6 documents
|
||||
# (`maintenance_charge_w` 2000 there, 2500 here) is exactly what a
|
||||
# literal in this file would repeat.
|
||||
self.peak_guard = None
|
||||
self._peak_engaged = False
|
||||
self._peak_saturated = False
|
||||
self._peak_floor_limited = False
|
||||
self._maint_shape_suspended = False
|
||||
# Set by amain() once the transport is built, so the status page can show
|
||||
# what only the transport knows (homewizard_local's unchanged_s). Stays
|
||||
@@ -232,6 +273,16 @@ class Controller:
|
||||
# window we are in and why it is or is not cheap - so "Cheap window:
|
||||
# no" is an evidenced answer rather than a bare no.
|
||||
self.tariff_words = "no static profile"
|
||||
|
||||
# OPT-04. The tariff plan the BACKGROUND task last resolved, and when.
|
||||
# cycle() reads these two attributes and never awaits a fetch - see
|
||||
# TARIFF_REFRESH_S. `None` means no refresh has completed yet, which is
|
||||
# not "no prices" and certainly not "free": arbitrage.decide() reports
|
||||
# it as its own reason and does nothing.
|
||||
self.tariff_plan = None
|
||||
self.tariff_at = None
|
||||
self.arb = None
|
||||
self._arb_action = None
|
||||
self.last_write = None
|
||||
self.last_write_at = None
|
||||
self.last_write_ok = None
|
||||
@@ -494,6 +545,17 @@ class Controller:
|
||||
cap_w=float(self.o.get("peak_cap_w", 3500)),
|
||||
reading=self.peak.peak(now),
|
||||
achievable_w=min(max_w, abs(float(dev_top))),
|
||||
# ⚠️ SAFETY-24. `self.soc` IS THE MEASURED READING AND MAY BE None,
|
||||
# and it is handed over as None rather than defaulted here. An early
|
||||
# draft of the stage-5 loop read SoC as `| float(100)` and thereby
|
||||
# invented a full battery (NOTES.md 4.4b); a full battery is
|
||||
# precisely the value that disarms this floor. decide() treats None
|
||||
# as floor-limited, which is the direction that costs a shave rather
|
||||
# than the pack.
|
||||
soc=self.soc,
|
||||
soc_floor=float(self.o.get(
|
||||
"peak_soc_floor",
|
||||
peakguard.PEAK_OPTION_DEFAULTS["peak_soc_floor"])),
|
||||
max_hold_s=self.p1.averager.max_hold_s,
|
||||
frozen=looks_frozen(getattr(self.p1_source, "unchanged_s", None),
|
||||
standing, max_w, self.freeze_wd.cfg),
|
||||
@@ -514,6 +576,64 @@ class Controller:
|
||||
if self.peak_guard is not None:
|
||||
self.log_event(self.peak_guard.explain())
|
||||
|
||||
# --- OPT-04: price arbitrage (§10.1 priority 3) ----------------------
|
||||
# ONE `set` claim at P_ARBITRAGE, and app/arbitrage.py carries the whole
|
||||
# argument. What is decided HERE and nowhere else is what the strategy
|
||||
# is allowed to see:
|
||||
#
|
||||
# ⚠️ IT IS PLACED AFTER THE PEAK BLOCK BECAUSE IT READS ITS CONCLUSION.
|
||||
# `engaged_now` is this cycle's peak verdict, computed ~20 lines up.
|
||||
# Evaluated before it, this would hand the strategy the PREVIOUS cycle's
|
||||
# answer - one cycle of lag on the interlock that implements §9.1's
|
||||
# "peak shaving outranks arbitrage", inside a 900 s quarter.
|
||||
#
|
||||
# ⚠️ AND THE THIRD STATE IS NOT A TIDINESS. Without P1 the guard is not
|
||||
# CLEAR, it never RAN, and `None` says so. arbitrage.decide() refuses on
|
||||
# it, because charging at max_w creates the billed quantity while
|
||||
# nothing is measuring the quarter it lands in. `False` here would be
|
||||
# this line asserting a measurement that was never taken - the defect
|
||||
# `PeakReading` and `quarter_ceiling_w()` both exist to remove.
|
||||
#
|
||||
# ⚠️ THE THREE-STATE COLLAPSE LIVES IN arbitrage.guard_state(), NOT
|
||||
# HERE, and that is not tidiness. It is the same lesson
|
||||
# `shaping_suspended()` records: inline, this expression was reachable
|
||||
# by no check, and mutating it back to a bare `engaged_now` left every
|
||||
# suite in this repository green while a `meter_source: off` install
|
||||
# traded blind. Read guard_state() before touching this line.
|
||||
#
|
||||
# ⚠️ NO FETCH ON THIS PATH. `fresh_tariff_plan()` is an attribute read
|
||||
# and an age comparison; the HTTPS call lives in run_tariff_refresh().
|
||||
# See TARIFF_REFRESH_S: a supplier's TLS timeout here would be a
|
||||
# supplier able to push this add-on towards layer 1's 30 s veto.
|
||||
#
|
||||
# ⚠️ AND THE UNCERTAINTY SET IS NOT EMPTY BY DEFAULT. §11.4's
|
||||
# no-weather row widens the threshold, and this add-on carries no
|
||||
# weather forecast at all - §8.1's PV forecast is not built. Passing
|
||||
# `frozenset()` would be main.py asserting a forecast is healthy when
|
||||
# there is not one, which is exactly what static_tariff's `uncertainty`
|
||||
# argument has no default in order to prevent.
|
||||
self.arb = None
|
||||
if arbitrage.enabled(self.o):
|
||||
floor_pct, ceiling_pct = arbitrage.soc_bounds(self.o)
|
||||
self.arb = arbitrage.decide(
|
||||
self.fresh_tariff_plan(now), now,
|
||||
peak_guard_active=arbitrage.guard_state(self.peak_guard),
|
||||
soc=self.soc,
|
||||
min_spread_eur_kwh=arbitrage.threshold_eur_kwh(self.o),
|
||||
uncertainty=frozenset({Uncertainty.NO_WEATHER}),
|
||||
charge_w=max_w, discharge_w=max_w,
|
||||
soc_floor_pct=floor_pct, soc_ceiling_pct=ceiling_pct)
|
||||
arb_claim = arbitrage.claim(self.arb)
|
||||
if arb_claim is not None:
|
||||
claims.append(arb_claim)
|
||||
# Edge-logged, for the peak guard's reason: the action log holds 40
|
||||
# lines and a trade lasts a whole hour, so a line per cycle would
|
||||
# flush the context that explains it. The transition is what an
|
||||
# operator needs - "it started buying at 03:00", not 240 copies.
|
||||
if self.arb.action != self._arb_action:
|
||||
self._arb_action = self.arb.action
|
||||
self.log_event(self.arb.explain())
|
||||
|
||||
# --- maintenance -----------------------------------------------------
|
||||
if result.owns_setpoint:
|
||||
claims.append(Claim.set("maintenance", P_MAINTENANCE,
|
||||
@@ -682,6 +802,16 @@ class Controller:
|
||||
# method assigned above and nothing since has touched.
|
||||
await self.check_peak_saturated()
|
||||
|
||||
# --- SAFETY-24's alarm, and it is the OTHER way the money gets spent --
|
||||
# Same boundary argument as check_peak_saturated(): it grades
|
||||
# `self.peak_guard`, which cycle() assigned above and nothing since has
|
||||
# touched. Kept as its own method rather than folded into that one,
|
||||
# because the two states are mutually exclusive and want OPPOSITE
|
||||
# operator actions - saturation says "shed load, the inverter is already
|
||||
# flat out"; this says "the pack is at its floor and this quarter is not
|
||||
# being shaved at all".
|
||||
await self.check_peak_floor_limited()
|
||||
|
||||
# --- TARIFF-01, and it is LAST on purpose -----------------------------
|
||||
# Banking a quarter is bookkeeping about a moment that has already
|
||||
# happened; a slow SD card must never sit between the meter and the
|
||||
@@ -826,6 +956,70 @@ class Controller:
|
||||
f"thing that changes the bill."),
|
||||
})
|
||||
|
||||
async def check_peak_floor_limited(self) -> None:
|
||||
"""SAFETY-24. Alarm when the EMS SoC floor stops the guard shaving.
|
||||
|
||||
⚠️ FAILING TO SHAVE MUST NOT BE SILENT, AND THAT IS THE WHOLE POINT.
|
||||
The guard exists for one euro figure - the capaciteitstarief bills the
|
||||
month's single worst quarter - so a cycle in which it CHOSE not to shave
|
||||
is a cycle in which money is being spent on purpose. Under-shaving
|
||||
quietly would leave the operator with a bill and a status page full of
|
||||
green.
|
||||
|
||||
The distinction from check_peak_saturated() is the operator's action,
|
||||
which is why these are two notifications and not one:
|
||||
|
||||
saturated the guard asked for everything the inverter has and it
|
||||
was not enough. SHED LOAD - the controller has already
|
||||
done all it can, and it is still discharging.
|
||||
floor-limited the guard asked for NOTHING, because the pack is at or
|
||||
under the EMS floor, or its SoC could not be read.
|
||||
Nothing is being discharged for this quarter at all.
|
||||
|
||||
They cannot both be true: peakguard.decide()'s floor-limited branch
|
||||
returns before the `achievable_w` clamp that sets `saturated`.
|
||||
|
||||
Edge-triggered, and de-asserting arms it again - FreezeWatchdog's shape,
|
||||
for FreezeWatchdog's reason: one notification per episode, not one per
|
||||
cycle for a quarter of an hour.
|
||||
"""
|
||||
g = self.peak_guard
|
||||
limited = bool(g is not None and g.floor_limited)
|
||||
was, self._peak_floor_limited = self._peak_floor_limited, limited
|
||||
if not limited or was:
|
||||
return
|
||||
# No prefix: `reason` already opens with PEAK GUARD FLOOR-LIMITED, and
|
||||
# a second copy is the kind of stutter that makes a 40-line action log
|
||||
# read like boilerplate. The saturated path needs its prefix because its
|
||||
# reason carries the word in the middle of a sentence, not at the front.
|
||||
self.log_event(g.reason)
|
||||
_LOG.error(
|
||||
"SAFETY-24 - %s. The controller is STILL RUNNING and charging is "
|
||||
"still blocked, but it is not discharging to hold this quarter, so "
|
||||
"the quarter will set or raise the month's billed capacity peak. "
|
||||
"Shed load, or lower peak_soc_floor if this pack can afford it.",
|
||||
g.reason)
|
||||
# A fixed notification_id, and a DIFFERENT one from every other alarm
|
||||
# here - check_meter_frozen() gives the reasoning. A battery at its
|
||||
# energy floor is not a saturated inverter and is not a frozen meter,
|
||||
# and an operator who has to dismiss three identical-looking cards
|
||||
# learns to dismiss the one that mattered.
|
||||
await self.hass.call("persistent_notification", "create", {
|
||||
"notification_id": "goodwe_peak_soc_floor",
|
||||
"title": "GoodWe: the capacity-peak guard has run out of battery",
|
||||
"message": (
|
||||
f"{g.reason}.\n\nTHIS QUARTER WILL GO OVER THE "
|
||||
f"{g.ceiling_w:.0f} W ceiling and will set or raise the "
|
||||
f"month's billed capaciteitstarief peak, because the peak guard "
|
||||
f"is not allowed to discharge any further: {g.soc_floor_reason}."
|
||||
f" This is not a controller fault - it is the EMS-side energy "
|
||||
f"floor (peak_soc_floor) doing what it is set to do, and it sits "
|
||||
f"deliberately above the pack's own BMS cut-out. Charging stays "
|
||||
f"blocked. Shedding load is the only thing that changes the bill "
|
||||
f"now; if this pack can afford a deeper discharge, lower "
|
||||
f"peak_soc_floor."),
|
||||
})
|
||||
|
||||
async def check_meter_frozen(self) -> None:
|
||||
"""SAFETY-18. Alarm on a meter that answers forever without changing.
|
||||
|
||||
@@ -866,6 +1060,57 @@ class Controller:
|
||||
})
|
||||
|
||||
# -- tasks ---------------------------------------------------------------
|
||||
# -- OPT-04: the background tariff refresh -------------------------------
|
||||
def fresh_tariff_plan(self, now):
|
||||
"""The stored plan if it is still current, else None. NEVER a stale one.
|
||||
|
||||
⚠️ THIS IS THE ONE PLACE THE AGE IS JUDGED, and it is public for
|
||||
`loop_is_following()`'s reason: a check has to be able to assert on the
|
||||
shipped expression rather than on a transcription that drifts. Both
|
||||
terms are required - a plan with no timestamp is one nothing stored, and
|
||||
a timestamped plan older than TARIFF_MAX_AGE_S is a reading from a task
|
||||
that has stopped answering.
|
||||
"""
|
||||
if self.tariff_plan is None or self.tariff_at is None:
|
||||
return None
|
||||
if (now - self.tariff_at).total_seconds() > TARIFF_MAX_AGE_S:
|
||||
return None
|
||||
return self.tariff_plan
|
||||
|
||||
async def run_tariff_refresh(self) -> None:
|
||||
"""Resolve the tariff off the control path, forever. NEVER on the tick.
|
||||
|
||||
⚠️ IT CATCHES EVERYTHING AND KEEPS GOING, for MqttPublisher's reason:
|
||||
a supplier's outage, a certificate expiry, a malformed feed - all of it
|
||||
is pricing, and the battery does not care. What it must never do is
|
||||
raise out of here and take the task down, because then the plan freezes
|
||||
at its last value and looks current forever. That is what
|
||||
`fresh_tariff_plan()`'s age bound is the backstop for, and this is the
|
||||
half that keeps the task alive so the bound rarely matters.
|
||||
|
||||
⚠️ AND A FAILURE DOES NOT CLEAR THE PLAN. `resolve_tariff()` already has
|
||||
a §11.4 answer for a dead feed - degrade to the static profile, mark it
|
||||
provisional, arbitrage refuses - and that answer is better than the
|
||||
absence of one. Zeroing the plan here would replace a coarse tariff with
|
||||
no tariff, which is the opposite of what §11.4 asks for.
|
||||
"""
|
||||
horizon = float(price_option(self.o, "price_horizon_h"))
|
||||
while not self.stopping:
|
||||
now = datetime.now(timezone.utc)
|
||||
try:
|
||||
plan = await resolve_tariff(
|
||||
self.o, now, now + timedelta(hours=horizon), now=now)
|
||||
self.tariff_plan, self.tariff_at = plan, now
|
||||
_LOG.info("tariff refreshed: %s", plan.words)
|
||||
except Exception as err: # noqa: BLE001
|
||||
_LOG.warning(
|
||||
"tariff refresh failed (%s); keeping the plan resolved at "
|
||||
"%s. It retires on its own after %.0f s and arbitrage "
|
||||
"stops with it.", err, self.tariff_at, TARIFF_MAX_AGE_S)
|
||||
with contextlib.suppress(asyncio.TimeoutError):
|
||||
await asyncio.wait_for(self.stop_event.wait(),
|
||||
timeout=TARIFF_REFRESH_S)
|
||||
|
||||
async def run_control(self) -> None:
|
||||
entity = self.o.get("setpoint_entity", "")
|
||||
if entity:
|
||||
@@ -1116,6 +1361,13 @@ class Controller:
|
||||
elif _pg.saturated:
|
||||
out.append({"ok": False, "warn": False,
|
||||
"text": "PEAK GUARD SATURATED - " + _pg.reason})
|
||||
elif _pg.floor_limited:
|
||||
# SAFETY-24. NOT ok, like saturation: in both states the money
|
||||
# is going to be spent and a human is the only actuator left. A
|
||||
# separate row from saturation because the two ask for opposite
|
||||
# things - see check_peak_floor_limited().
|
||||
out.append({"ok": False, "warn": False,
|
||||
"text": _pg.explain()})
|
||||
elif _pg.blind:
|
||||
out.append({"ok": False, "warn": True, "text": _pg.explain()})
|
||||
else:
|
||||
@@ -1243,6 +1495,11 @@ class Controller:
|
||||
("Maintenance due", "now" if next_due is None else next_due.strftime("%Y-%m-%d")),
|
||||
("Cheap window", "yes" if self.cheap else "no"),
|
||||
("Tariff", self.tariff_words),
|
||||
# OPT-04. The strategy's own sentence, not a re-derivation of it -
|
||||
# every refusal it can make already carries a reason a human can
|
||||
# act on, and "Arbitrage: no" would throw all of them away.
|
||||
("Arbitrage", "off (arbitrage_enabled)" if self.arb is None
|
||||
else self.arb.explain()),
|
||||
]
|
||||
return {
|
||||
"banner": banner, "level": level, "rows": rows, "checks": checks,
|
||||
@@ -1370,6 +1627,19 @@ async def amain() -> None:
|
||||
|
||||
tasks = [asyncio.create_task(controller.run_control())]
|
||||
|
||||
# OPT-04. The tariff refresh runs as its OWN long-lived task, for the
|
||||
# reason P1 ingestion does and then some: `resolve_tariff()` may open an
|
||||
# HTTPS connection to a supplier, and the control tick is what feeds the
|
||||
# ESP32 heartbeat. A fetch on that path is a supplier's TLS timeout able
|
||||
# to push this add-on towards layer 1's 30 s veto. See TARIFF_REFRESH_S.
|
||||
#
|
||||
# ⚠️ STARTED ONLY WHEN ARBITRAGE IS ARMED. Nothing else in the add-on
|
||||
# reads the resolved plan - `cheap` still comes from static_status() on
|
||||
# the tick, which is pure - so on a default install this task would be
|
||||
# network traffic in aid of an attribute nobody reads.
|
||||
if arbitrage.enabled(opts):
|
||||
tasks.append(asyncio.create_task(controller.run_tariff_refresh()))
|
||||
|
||||
# P1 ingestion runs as its own long-lived task. ⚠️ It must not be driven
|
||||
# off the control loop: telegrams arrive every ~5 s and the loop would
|
||||
# decimate them, so the 15-minute average - the capacity-tariff billing
|
||||
|
||||
@@ -246,19 +246,100 @@ validate.py are untouched by this ticket, exactly as TARIFF-01 left them, and
|
||||
mismatch trap 14's own warning box describes.
|
||||
|
||||
------------------------------------------------------------------------------
|
||||
⚠️ WHAT THIS GUARD DOES NOT HAVE: AN ENERGY BACKSTOP. SEE SAFETY-24.
|
||||
SAFETY-24: THE ENERGY BACKSTOP THIS GUARD SHIPPED WITHOUT
|
||||
|
||||
The graded branch can FORCE DISCHARGE, and nothing in this file or in the loop
|
||||
below it consults the state of charge - `soc_floor` belongs to the maintenance
|
||||
cycle, and freeze.py already records that for the grid-following loop "the BMS
|
||||
is the sole energy backstop". On the 2026-08-20 replay the guard asks for 878 Wh
|
||||
in a day. Against a pack sitting near 9 % that is not nothing, and the guard will
|
||||
keep asking as long as the meter says the quarter is over the ceiling.
|
||||
SAFETY-07 left this file saying, in its own words, that the graded branch can
|
||||
FORCE DISCHARGE and that nothing below it consults the state of charge - "the
|
||||
BMS is the sole energy backstop" (freeze.py). On the 2026-08-20 replay the guard
|
||||
asks for 878 Wh in a day, and it keeps asking for as long as the meter says the
|
||||
quarter is over the ceiling. That is the behaviour working as designed, and it
|
||||
is unbounded from below by anything in the EMS.
|
||||
|
||||
That is deliberate for THIS card and it is filed as SAFETY-24. It is also the
|
||||
reason this must not go to a client install before SAFETY-24 lands: on the
|
||||
reference site the BMS is a known quantity and an operator is present, and on a
|
||||
client site neither is true.
|
||||
⚠️ STATE THE RISK FAIRLY, because overstating it is how a real one gets ignored.
|
||||
Relying on the BMS is not absurd: it is a genuine floor and it works. But it is
|
||||
THE PACK's protection, not the OPERATOR's. A month of aggressive quarters can
|
||||
leave the battery repeatedly parked at that floor with no EMS-side record of
|
||||
why, and the first symptom - a pack that is never available when it is wanted -
|
||||
is indistinguishable from the SoC drift MAINT-01 exists to correct.
|
||||
|
||||
So `decide()` now takes `soc` and `soc_floor`, and `discharge_permitted()` is
|
||||
the one place the question is answered. Four things about the shape of it:
|
||||
|
||||
1. IT IS NOT A TRIP, AND IT MUST NOT BECOME ONE. SAFETY-07's claim is a LIMIT
|
||||
and every stop in 11.2's table still outranks it. This floor is about
|
||||
ENERGY - how far down the guard may spend the pack for money - not about
|
||||
last-resort safety. The action when it binds is therefore to stop ASKING
|
||||
for more discharge, not to command anything.
|
||||
|
||||
2. THE ACTION IS `floor_w = 0.0`, WHICH IS THE BLIND BRANCH'S ACTION FOR THE
|
||||
BLIND BRANCH'S REASON. Zero is one-directional: it forbids CHARGING, which
|
||||
can only ever lower delivered offtake, and it forbids nothing else. It
|
||||
cannot pull the loop off a discharge it is already doing, because a `lo`
|
||||
limit structurally cannot cap discharge - the property this module's whole
|
||||
shape rests on, and one a NEGATIVE floor would destroy. Nothing here can
|
||||
produce a negative floor: the only two values `floor_w` ever takes are 0.0
|
||||
and `min(wanted, achievable)` with `wanted > 0` already established.
|
||||
|
||||
⚠️ AND IT DOES NOT BOUND THE LOOP. Self-consumption may still drain the
|
||||
pack past this floor, exactly as SAFETY-18 recorded. That is not fixed
|
||||
here and must not be claimed: this card bounds THE PEAK GUARD's OWN forced
|
||||
discharge, which is the thing SAFETY-07 introduced. A loop-side SoC guard
|
||||
is a different lever with a different right value.
|
||||
|
||||
3. IT IS GRADED ONLY WHERE IT IS NOT DEGENERATE. The SoC question is asked
|
||||
AFTER `wanted > 0` is established - i.e. only where a forced discharge is
|
||||
actually being asked for. Asked earlier it would grade a quantity that
|
||||
governs nothing, and would report "floor-limited" on a quiet afternoon
|
||||
where the guard was never going to discharge anyway.
|
||||
|
||||
4. AN UNREADABLE SoC IS NOT A FULL BATTERY. `soc=None` is floor-limited, not
|
||||
permitted. This is `PeakReading`'s lesson in the one place it bites
|
||||
hardest: a full battery is exactly the value that would disarm this floor,
|
||||
so a failed read defaulting to one would silently restore the unbounded
|
||||
discharge this card exists to remove. Both new arguments are REQUIRED
|
||||
keyword-only and neither has a default, which is TEST-14 read as a rule
|
||||
rather than as a story: a required field that gains a default is how three
|
||||
separate files in this repo shipped a quiet fallback.
|
||||
|
||||
AND FAILING TO SHAVE MUST NOT BE SILENT. The whole point of the guard is the
|
||||
euro figure, so a floor-limited cycle carries `floor_limited=True` and main.py
|
||||
raises ONE notification on the edge, saying the quarter will be exceeded - the
|
||||
same shape, and the same argument, as `saturated`. The two are mutually
|
||||
exclusive by construction: a floor-limited decision asks for nothing, so it
|
||||
cannot have run out of inverter.
|
||||
|
||||
------------------------------------------------------------------------------
|
||||
CFG-03: TWO NEW OPTIONS, MIRRORED ONCE, AND WHY BOTH ARE NEEDED
|
||||
|
||||
SAFETY-07 added none and said so. This card cannot: an EMS-side floor is a
|
||||
number only the site knows, and refusing one set below the pack's own cut-out
|
||||
means knowing that cut-out too.
|
||||
|
||||
* `peak_soc_floor` - the EMS floor. ⚠️ NOT `maintenance_soc_floor`. That is
|
||||
the maintenance cycle's DRAIN TARGET: the depth a monthly recalibration
|
||||
deliberately goes to, once, under supervision. This is the depth a
|
||||
money-saving discharge may never reach, on any day of the month. Sharing
|
||||
one key would couple two features whose right values differ and would widen
|
||||
CFG-03 rather than narrow it.
|
||||
* `bms_soc_floor` - the pack's own cut-out (register 0x0560 on the reference
|
||||
unit, 10 %). It is here so validate.py rule 8 can REFUSE a `peak_soc_floor`
|
||||
at or below it. That configuration is CFG-01 rule (a): the operator states
|
||||
a floor, the BMS enforces a different one, and the status page reports an
|
||||
energy backstop that can never bind.
|
||||
|
||||
Both defaults are mirrored ONCE, in `PEAK_OPTION_DEFAULTS` below, which is
|
||||
TARIFF-02's `PRICE_OPTION_DEFAULTS` pattern - imported by validate.py and by
|
||||
main.py, pinned to config.yaml by test_peakguard.py rather than transcribed.
|
||||
This module still reads no option itself: `decide()` takes both as ARGUMENTS,
|
||||
makes no `.get()` call, and the mirror is a default table, not a read.
|
||||
|
||||
⚠️ A DECISION RECORDED, NOT A RENAME DONE. Now that a second SoC floor exists,
|
||||
`maintenance_soc_floor` is the ambiguous name of the two - it reads like "the
|
||||
SoC floor" and is not. Renaming it is an option migration on a live install for
|
||||
a clarity gain, so the decision recorded here is: KEEP IT, and carry the
|
||||
distinction in DOCS.md and in this comment. Revisit if a THIRD floor appears -
|
||||
a loop-side guard would be the third - at which point the migration buys enough
|
||||
to be worth its cost.
|
||||
|
||||
------------------------------------------------------------------------------
|
||||
AND IT READS NO PER-PHASE QUANTITY, ON PURPOSE
|
||||
@@ -287,8 +368,44 @@ BLIND_FROZEN = "SAFETY-18: the meter is answering without changing"
|
||||
CEILING_CAP = "peak_cap_w"
|
||||
CEILING_MONTH_PEAK = "the month's banked peak"
|
||||
|
||||
# SAFETY-24. Why the forced discharge may not be sized. Reasons, not booleans,
|
||||
# for BLIND_*'s reason: the status page, the action log and the operator
|
||||
# notification all have to say WHICH, and "floor-limited" alone is unactionable.
|
||||
FLOOR_SOC_UNREADABLE = ("the battery state of charge was not measured this "
|
||||
"cycle, and an unreadable SoC is not a full battery")
|
||||
FLOOR_AT_SOC_FLOOR = "state of charge is at or below the peak guard's SoC floor"
|
||||
|
||||
SOURCE = "peak"
|
||||
|
||||
# ⚠️ CFG-03. THE ONE IN-CODE MIRROR OF config.yaml FOR THIS CARD'S OPTIONS, and
|
||||
# it is TARIFF-02's `PRICE_OPTION_DEFAULTS` pattern for TARIFF-02's reason: the
|
||||
# add-on already carries option defaults in three places (config.yaml's
|
||||
# `options:`, config.yaml's `schema:`, and main.py's literal `.get()`
|
||||
# fallbacks), and a fourth is the open ticket. validate.py and main.py both
|
||||
# import THIS dict rather than typing a literal, and test_peakguard.py pins it
|
||||
# to config.yaml by READING the file - so a drift goes red instead of shipping.
|
||||
#
|
||||
# ⚠️ NOT A READ. `decide()` takes both values as arguments and this module makes
|
||||
# no `.get()` call at all; the dict is a default table for its two callers.
|
||||
PEAK_OPTION_DEFAULTS = {
|
||||
# The EMS-side floor on SAFETY-07's forced discharge, in percent.
|
||||
#
|
||||
# 20 is DERIVED from the two numbers this repository already has, not
|
||||
# chosen. The reference pack is 300 Ah at 51.2 V with DoD 90 - 13.8 kWh
|
||||
# usable, the figure validate.py's rule 6 note already quotes - and the
|
||||
# 2026-08-20 replay models 878 Wh of guard discharge in a day, which is
|
||||
# 6.4 % of that. A floor 10 points above the pack's own 10 % cut-out
|
||||
# therefore leaves the guard roughly 1.6 of its worst observed days of room
|
||||
# before it stops, which is the smallest margin that lets the guard do its
|
||||
# job on a bad day and still never reach the BMS.
|
||||
"peak_soc_floor": 20,
|
||||
# The pack's OWN cut-out, in percent: what the BMS enforces regardless of
|
||||
# anything here. On the reference unit this is register 0x0560, read as 10,
|
||||
# and it is consistent with DoD 90 on a full pack. It exists so validate.py
|
||||
# rule 8 can refuse a `peak_soc_floor` at or below it - see that rule.
|
||||
"bms_soc_floor": 10,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PeakDecision:
|
||||
@@ -309,9 +426,19 @@ class PeakDecision:
|
||||
excess_w: float = 0.0
|
||||
wanted_w: float | None = None # the floor before `achievable_w` clamped it
|
||||
saturated: bool = False
|
||||
# SAFETY-24. True when a forced discharge WAS wanted and the EMS-side SoC
|
||||
# floor refused to size it. Mutually exclusive with `saturated` by
|
||||
# construction: this branch asks for nothing, so it cannot have run out of
|
||||
# inverter. `soc_floor_reason` says WHICH of the two ways it bound.
|
||||
floor_limited: bool = False
|
||||
soc: float | None = None
|
||||
soc_floor: float | None = None
|
||||
soc_floor_reason: str | None = None
|
||||
reason: str = ""
|
||||
|
||||
def explain(self) -> str:
|
||||
if self.floor_limited:
|
||||
return self.reason
|
||||
if not self.engaged:
|
||||
if self.projected_w is None:
|
||||
return f"peak guard idle - {self.reason}"
|
||||
@@ -383,7 +510,44 @@ def blindness(projection, max_hold_s, frozen: bool) -> str | None:
|
||||
return None
|
||||
|
||||
|
||||
def decide(projection, *, batt_w, cap_w, reading, achievable_w,
|
||||
def discharge_permitted(soc, soc_floor) -> str | None:
|
||||
"""SAFETY-24. May the guard size a forced discharge, or why may it not?
|
||||
|
||||
Returns None when it may, and otherwise the REASON it may not - which the
|
||||
decision, the status page, the action log and the operator notification all
|
||||
carry, because "floor-limited" on its own tells nobody what to do.
|
||||
|
||||
THE ONE PLACE THIS QUESTION IS ANSWERED, and it is public for the reason
|
||||
`freeze.looks_frozen()`, `watchdog.loop_is_following()` and
|
||||
`shaping_suspended()` are: a safety predicate transcribed into its own check
|
||||
is one copy plus a future disagreement about what the check means, and the
|
||||
copy that drifts is always the grading one.
|
||||
|
||||
⚠️ `soc is None` IS NOT PERMISSION. A failed read is not an observation, and
|
||||
a FULL BATTERY is exactly the value that would disarm this floor - so a
|
||||
`float(soc or 100)` here would restore the unbounded discharge this card
|
||||
removes, silently, on the one install whose SoC sensor is misnamed. There is
|
||||
deliberately no default argument on either parameter.
|
||||
|
||||
⚠️ AND `soc_floor is None` IS NOT PERMISSION EITHER, for the same reason one
|
||||
step further out: a caller that has no floor configured has not thereby been
|
||||
granted an unbounded one.
|
||||
|
||||
Boundary: `<=`, not `<`. A pack sitting exactly ON the floor has already
|
||||
spent everything this guard was allowed to spend; permitting one more cycle
|
||||
there would make the floor mean "one cycle below the floor".
|
||||
"""
|
||||
if soc is None:
|
||||
return FLOOR_SOC_UNREADABLE
|
||||
if soc_floor is None:
|
||||
return ("no peak guard SoC floor is configured, so there is nothing to "
|
||||
"bound a forced discharge with")
|
||||
if float(soc) <= float(soc_floor):
|
||||
return FLOOR_AT_SOC_FLOOR
|
||||
return None
|
||||
|
||||
|
||||
def decide(projection, *, batt_w, cap_w, reading, achievable_w, soc, soc_floor,
|
||||
max_hold_s=None, frozen: bool = False) -> PeakDecision:
|
||||
"""The whole guard. Pure: no clock, no I/O, no state carried between calls.
|
||||
|
||||
@@ -395,6 +559,13 @@ def decide(projection, *, batt_w, cap_w, reading, achievable_w,
|
||||
claim can NEVER produce `lo > hi` in the resolver - a
|
||||
contradiction there resolves to 0 W, which would park the
|
||||
battery in the exact minute it should be discharging hardest.
|
||||
`soc` SAFETY-24. MEASURED state of charge, percent. None when the
|
||||
sensor for it did not answer - which is floor-limited, not
|
||||
permission.
|
||||
`soc_floor` SAFETY-24. The percent below which this guard may not spend
|
||||
the pack for money. REQUIRED, and deliberately without a
|
||||
default: a default here would be CFG-03's fourth defaults
|
||||
site and TEST-14's quiet fallback in one line.
|
||||
"""
|
||||
ceiling, from_where = quarter_ceiling_w(cap_w, reading)
|
||||
ceil_top = abs(float(achievable_w))
|
||||
@@ -454,12 +625,41 @@ def decide(projection, *, batt_w, cap_w, reading, achievable_w,
|
||||
wanted_w=wanted,
|
||||
reason="the load is under the level this quarter can afford")
|
||||
|
||||
# SAFETY-24 - AND ONLY HERE. `wanted > 0` is now established, so a forced
|
||||
# discharge really is being asked for and the SoC is the quantity that
|
||||
# decides whether it may be sized. Asked one line earlier it would grade a
|
||||
# number that governs nothing on every quiet cycle of the day, and would
|
||||
# report "floor-limited" over a guard that was never going to discharge.
|
||||
denied = discharge_permitted(soc, soc_floor)
|
||||
if denied is not None:
|
||||
# 0.0, not `wanted`, and not a negative number. Zero forbids CHARGING -
|
||||
# one-directional, cannot cost pack energy - and forbids nothing else;
|
||||
# a `lo` limit structurally cannot cap the discharge the loop is already
|
||||
# doing. See the module note: this bounds THE GUARD's forced discharge,
|
||||
# not the grid-following loop, which still has no SoC guard.
|
||||
return PeakDecision(
|
||||
engaged=True, floor_w=0.0, ceiling_w=ceiling,
|
||||
ceiling_from=from_where, projected_w=projected, excess_w=ex,
|
||||
wanted_w=wanted, floor_limited=True, soc=soc, soc_floor=soc_floor,
|
||||
soc_floor_reason=denied,
|
||||
reason=(
|
||||
f"PEAK GUARD FLOOR-LIMITED: the quarter projects "
|
||||
f"{projected:.0f} W against a {ceiling:.0f} W ceiling "
|
||||
f"({from_where}) and {wanted:+.0f} W of discharge would hold "
|
||||
f"it, but {denied}"
|
||||
+ ("" if soc is None else f" ({float(soc):.0f} %")
|
||||
+ ("" if soc is None or soc_floor is None
|
||||
else f" vs {float(soc_floor):.0f} %")
|
||||
+ ("" if soc is None else ")")
|
||||
+ " - commanding no forced discharge, so THIS QUARTER WILL GO "
|
||||
"OVER THE CEILING. Charging stays blocked (floor 0 W)"))
|
||||
|
||||
floor = min(wanted, ceil_top)
|
||||
saturated = floor < wanted
|
||||
return PeakDecision(
|
||||
engaged=True, floor_w=floor, ceiling_w=ceiling, ceiling_from=from_where,
|
||||
blind=None, projected_w=projected, excess_w=ex, wanted_w=wanted,
|
||||
saturated=saturated,
|
||||
saturated=saturated, soc=soc, soc_floor=soc_floor,
|
||||
reason=(
|
||||
f"PEAK GUARD: quarter projects {projected:.0f} W against a "
|
||||
f"{ceiling:.0f} W ceiling ({from_where}) with "
|
||||
@@ -513,5 +713,13 @@ def claim(decision: PeakDecision) -> Claim | None:
|
||||
"""
|
||||
if not decision.engaged or decision.floor_w is None:
|
||||
return None
|
||||
# Three reasons, not two. A `lo = 0 W` claim reaches the arbiter from THREE
|
||||
# different states now - blind, floor-limited, and a genuine floor that
|
||||
# happens to be zero - and the arbiter's explain() is what an operator reads
|
||||
# when the battery stops charging. Spelling them the same way would make the
|
||||
# SAFETY-24 case indistinguishable from the SAFETY-18 one on the one line
|
||||
# that gets read.
|
||||
return Claim.limit(SOURCE, P_PEAK, lo=float(decision.floor_w),
|
||||
reason=("blind" if decision.blind else "quarter peak"))
|
||||
reason=("blind" if decision.blind
|
||||
else "SoC floor" if decision.floor_limited
|
||||
else "quarter peak"))
|
||||
|
||||
@@ -607,6 +607,26 @@ PRICE_OPTION_DEFAULTS = {
|
||||
# test_price.py section 1. A second defaults dict in the new module would be
|
||||
# a fourth place option defaults live, which is exactly the open ticket.
|
||||
"static_profile": "",
|
||||
# OPT-04. §10.1 priority 3, and §14's `optimizer.cycle_cost_threshold_eur`.
|
||||
#
|
||||
# ⚠️ THEY LIVE HERE FOR CFG-03's SAKE, exactly as `static_profile` does, and
|
||||
# FCST-03's note above `WIDEN_FACTOR` names this as the place they would
|
||||
# land. `arbitrage_soc_floor` / `arbitrage_soc_ceiling` are the reserve
|
||||
# arbitrage may not spend and the headroom it may not overfill; they are
|
||||
# NOT the maintenance SoC keys, which describe a balancing cycle rather
|
||||
# than a trading reserve and would silently couple two unrelated policies.
|
||||
#
|
||||
# ⚠️ AND `cycle_cost_threshold_eur` IS 0.10 BECAUSE §10.1's NAME FOR IT SAYS
|
||||
# SO, NOT BECAUSE IT MAKES ANYTHING TRADE. On realistic Belgian day/night
|
||||
# numbers the static spread is ~0.08 EUR/kWh, so a static-tariff site does
|
||||
# NOT trade at this default. That is a reported result (test_arbitrage.py
|
||||
# section 7), and whether 0.10 - or FCST-03's 2.5/1.5 - is right is one
|
||||
# product decision that belongs to the owner. No number here was chosen to
|
||||
# make arbitrage happen.
|
||||
"arbitrage_enabled": False,
|
||||
"cycle_cost_threshold_eur": 0.10,
|
||||
"arbitrage_soc_floor": 20,
|
||||
"arbitrage_soc_ceiling": 95,
|
||||
}
|
||||
|
||||
# ⚠️ The keys the FRAMEWORK owns. A plugin that reads one of these is a plugin
|
||||
|
||||
@@ -131,6 +131,25 @@ NOT READY banner already names the problem, (b) does not apply.
|
||||
installed base is on a fixed or static tariff. Refusing to start without a
|
||||
price feed would lock those sites out of the add-on entirely.
|
||||
|
||||
8. `peak_soc_floor` at or below `bms_soc_floor`. [(a)]
|
||||
SAFETY-24, and the same shape as 1, 4, 5 and 6. The peak guard can COMMAND
|
||||
a discharge to hold a billed quarter down (SAFETY-07), and `peak_soc_floor`
|
||||
is the EMS-side bound on how far down it may take the pack. The BMS has its
|
||||
own cut-out and enforces it regardless. Set the EMS floor at or below that
|
||||
cut-out and THE BMS WINS: the EMS bound can never be the thing that stops
|
||||
the discharge, so the add-on carries an energy backstop that is incapable
|
||||
of binding while its status page reports one - rule 5's "a disarmed guard
|
||||
that advertises itself as armed", on the guard whose failure mode is a pack
|
||||
that is quietly never available when it is wanted.
|
||||
|
||||
⚠️ `<=`, NOT `<`, and the equality case is the interesting one. At exactly
|
||||
equal, the EMS floor and the pack's cut-out act at the same percent, so the
|
||||
EMS floor buys ZERO margin - it can never prevent the pack reaching the BMS
|
||||
limit, which is the entire thing this option exists to prevent. That is not
|
||||
"close to the rail", it is the disarmed case with an extra step. Rule 6
|
||||
permits equality because equality there leaves one reachable point that
|
||||
works; here it leaves none.
|
||||
|
||||
WHAT IS DELIBERATELY *NOT* REFUSED, because it is untidy rather than unsafe:
|
||||
|
||||
- `integrator_max_w` above `max_w`. DOCS.md is right that the only thing the
|
||||
@@ -151,12 +170,18 @@ WHAT IS DELIBERATELY *NOT* REFUSED, because it is untidy rather than unsafe:
|
||||
- `maintenance_discharge_w` above `max_w`. Clamped, not contradicted, and the
|
||||
clamp names itself in the arbiter's explain() - see rule 6. Warned about,
|
||||
not refused.
|
||||
- `peak_soc_floor` below `maintenance_soc_floor`, or above it. They are
|
||||
DIFFERENT QUANTITIES (SAFETY-24: a drain target versus a spend limit) and
|
||||
every ordering of the two is a legitimate site configuration. Refusing on
|
||||
an ordering would be inventing a relationship the system does not have.
|
||||
"""
|
||||
|
||||
import logging
|
||||
|
||||
from .arbitrage import enabled as arbitrage_on
|
||||
from .p1 import is_enabled, validate_meter_options
|
||||
from .price import PROVIDERS
|
||||
from .peakguard import PEAK_OPTION_DEFAULTS
|
||||
from .price import PRICE_OPTION_DEFAULTS, PROVIDERS
|
||||
from .static_tariff import ProfileInvalid, parse_static_profile
|
||||
|
||||
_LOG = logging.getLogger("goodwe.config")
|
||||
@@ -369,6 +394,36 @@ def validate_startup_config(opts: dict, dev_min=None, dev_max=None) -> list[str]
|
||||
# --- 7. two price sources on one site ------------------------------------
|
||||
problems.extend(_price_problems(opts))
|
||||
|
||||
# --- 8. an EMS energy floor the BMS does not already enforce -------------
|
||||
# ⚠️ THE FALLBACKS ARE peakguard.PEAK_OPTION_DEFAULTS, NOT LITERALS, and
|
||||
# that is CFG-03 read as a rule rather than as a complaint. Rule 6's own
|
||||
# comment documents a LIVE divergence in this repo caused by typing 2000
|
||||
# here and 2500 in main.py; this card refuses to add a second instance of
|
||||
# it. The dict is the single in-code mirror, pinned to config.yaml by
|
||||
# test_peakguard.py, and main.py reads the same one.
|
||||
soc_floor = _number(opts, "peak_soc_floor",
|
||||
PEAK_OPTION_DEFAULTS["peak_soc_floor"])
|
||||
bms_floor = _number(opts, "bms_soc_floor",
|
||||
PEAK_OPTION_DEFAULTS["bms_soc_floor"])
|
||||
if soc_floor is None or bms_floor is None:
|
||||
problems.append(
|
||||
"peak_soc_floor and bms_soc_floor must both be numbers (got "
|
||||
f"{opts.get('peak_soc_floor')!r} / {opts.get('bms_soc_floor')!r})")
|
||||
elif soc_floor <= bms_floor:
|
||||
problems.append(
|
||||
f"peak_soc_floor is {soc_floor:.0f} % but the pack's own cut-out "
|
||||
f"(bms_soc_floor) is {bms_floor:.0f} %, so SAFETY-07's forced "
|
||||
"discharge has no EMS-side bound at all - THE BMS WINS, and it is "
|
||||
"the pack's protection rather than yours. The guard would report a "
|
||||
"SoC floor on its status page while being incapable of ever being "
|
||||
"the thing that stops the discharge, and the symptom of that is a "
|
||||
"battery repeatedly sitting at the BMS limit with no record of why "
|
||||
f"- which looks exactly like the SoC drift the maintenance cycle "
|
||||
f"exists to correct. Raise peak_soc_floor above {bms_floor:.0f} % "
|
||||
f"(the shipped default is "
|
||||
f"{PEAK_OPTION_DEFAULTS['peak_soc_floor']:.0f} %), or correct "
|
||||
"bms_soc_floor if it does not match this pack.")
|
||||
|
||||
# --- warnings: real, but not worth refusing over -------------------------
|
||||
integ = opts.get("integrator_max_w")
|
||||
integ_w = None if not integ else _number(opts, "integrator_max_w", None)
|
||||
@@ -453,14 +508,21 @@ def _price_problems(opts: dict) -> list[str]:
|
||||
# arbitrage verdict, so the seam is no longer inert.
|
||||
# * main.py reads the STATIC side of that module every cycle
|
||||
# (static_status()), which makes static_profile a real writer of `cheap`.
|
||||
# * main.py does NOT yet call resolve_tariff() on the control tick, and
|
||||
# that is deliberate: a provider fetch is network I/O and the tick is
|
||||
# what feeds the ESP32 heartbeat. So a configured PROVIDER still shapes
|
||||
# no energy decision on its own; a configured STATIC PROFILE does.
|
||||
# * main.py does NOT call resolve_tariff() on the control tick, and that
|
||||
# is still deliberate: a provider fetch is network I/O and the tick is
|
||||
# what feeds the ESP32 heartbeat. OPT-04 put it on a BACKGROUND REFRESH
|
||||
# instead (Controller.run_tariff_refresh), and the tick reads the plan
|
||||
# that task last stored - an attribute read, with nothing to block on.
|
||||
# * so a configured PROVIDER now DOES shape an energy decision, but only
|
||||
# through app/arbitrage.py and only while `arbitrage_enabled` is on.
|
||||
# With it off - the shipped configuration - the series is still fetched,
|
||||
# validated and published, and still moves nothing.
|
||||
#
|
||||
# Hence two refusals below rather than one, and neither claims more than
|
||||
# that list. When OPT-04 puts resolve_tariff() on a background refresh, the
|
||||
# first message's middle clause is what changes - not the rule.
|
||||
# Hence three refusals below rather than two, and none of them claims more
|
||||
# than that list. ⚠️ THE MIDDLE CLAUSE OF THE FIRST MESSAGE IS WHAT OPT-04
|
||||
# CHANGED, exactly as the note it replaced predicted - the RULE is
|
||||
# untouched, because the reason it exists (main.py reads the Home Assistant
|
||||
# pair FIRST and the other source is silent) is unchanged.
|
||||
ha_price = [k for k in ("price_now_entity", "price_avg_entity")
|
||||
if str(opts.get(k, "") or "").strip()]
|
||||
if pid != "off" and ha_price:
|
||||
@@ -469,10 +531,12 @@ def _price_problems(opts: dict) -> list[str]:
|
||||
f"{'is' if len(ha_price) == 1 else 'are'} set, which is two price "
|
||||
"sources on one site. THE HOME ASSISTANT ENTITIES WIN: they are "
|
||||
"what main.py reduces to the `cheap` flag the maintenance cycle "
|
||||
"reads. The provider series IS consumed now - app/static_tariff.py "
|
||||
"builds a tariff plan and an arbitrage verdict from it - but it is "
|
||||
"not what main.py acts on, so the supplier you configured would "
|
||||
"shape no energy decision while the status page named it. Clear "
|
||||
"reads, and read_inputs() never looks at the other source once "
|
||||
"they are set. The provider series is consumed - "
|
||||
"app/static_tariff.py builds a tariff plan and an arbitrage "
|
||||
"verdict, and app/arbitrage.py trades on it off a background "
|
||||
"refresh - but the `cheap` flag those entities own is a separate "
|
||||
"decision they would silently keep. Clear "
|
||||
"price_now_entity and price_avg_entity to use the "
|
||||
f"{pid!r} plugin, or set price_provider to 'off' to keep the "
|
||||
"Home Assistant sensors.")
|
||||
@@ -528,6 +592,67 @@ def _price_problems(opts: dict) -> list[str]:
|
||||
"with no tariff at all rather than with a coarse one. Not refused, "
|
||||
"but filling it in is what makes that row work.", pid)
|
||||
|
||||
# ⚠️ OPT-04. THE THREE WAYS `arbitrage_enabled: true` CAN BE A LIE, refused
|
||||
# rather than logged, because a strategy that is armed in the UI and cannot
|
||||
# fire is indistinguishable from one that fires and finds no opportunity -
|
||||
# which is the §14 warning box's case exactly. None of these can break an
|
||||
# existing install: the option is new and ships `false`.
|
||||
# ⚠️ CHECKED WHETHER OR NOT ARBITRAGE IS ARMED. A threshold that is not a
|
||||
# non-negative number is a configuration error the moment it is typed, and
|
||||
# deferring the complaint until somebody turns the strategy on would report
|
||||
# it at the worst possible moment - the one where energy starts moving.
|
||||
thresh = _number(opts, "cycle_cost_threshold_eur",
|
||||
PRICE_OPTION_DEFAULTS["cycle_cost_threshold_eur"])
|
||||
if thresh is None or thresh < 0:
|
||||
problems.append(
|
||||
f"cycle_cost_threshold_eur is "
|
||||
f"{opts.get('cycle_cost_threshold_eur')!r} and must be a "
|
||||
f"non-negative number of EUR/kWh - it is what one round trip COSTS "
|
||||
f"(§10.1). A negative threshold is one that every spread clears, "
|
||||
f"including a spread of nothing, which is arbitrage with the brake "
|
||||
f"removed rather than arbitrage tuned aggressively.")
|
||||
|
||||
if arbitrage_on(opts):
|
||||
if pid == "off" and not str(opts.get("static_profile", "") or "").strip():
|
||||
problems.append(
|
||||
"arbitrage_enabled is on but price_provider is 'off' and "
|
||||
"static_profile is empty, so this site has NO PRICES AT ALL. "
|
||||
"§10.1's priority 3 is 'charge cheap, discharge expensive' and "
|
||||
"there is nothing here to tell cheap from expensive, so the "
|
||||
"strategy would refuse on every cycle forever while the "
|
||||
"configuration said it was running. Configure a price_provider "
|
||||
"or a static_profile, or turn arbitrage_enabled off.")
|
||||
if not is_enabled(opts):
|
||||
problems.append(
|
||||
"arbitrage_enabled is on but meter_source is 'off', so there "
|
||||
"is no P1 telegram stream, the capacity-tariff peak guard "
|
||||
"never runs, and nothing measures the 15-minute average this "
|
||||
"strategy's charging would land in. §9.1 gives peak shaving "
|
||||
"priority over arbitrage; an unmeasured peak cannot be shaved, "
|
||||
"so arbitrage would be deliberately creating the billed "
|
||||
"quantity with the one guard that limits it switched off. "
|
||||
"app/arbitrage.py refuses to trade in that state anyway - this "
|
||||
"refuses the configuration that asks it to. Set meter_source, "
|
||||
"or turn arbitrage_enabled off.")
|
||||
floor = _number(opts, "arbitrage_soc_floor",
|
||||
PRICE_OPTION_DEFAULTS["arbitrage_soc_floor"])
|
||||
ceiling = _number(opts, "arbitrage_soc_ceiling",
|
||||
PRICE_OPTION_DEFAULTS["arbitrage_soc_ceiling"])
|
||||
if floor is None or ceiling is None:
|
||||
problems.append(
|
||||
f"arbitrage_soc_floor {opts.get('arbitrage_soc_floor')!r} and "
|
||||
f"arbitrage_soc_ceiling {opts.get('arbitrage_soc_ceiling')!r} "
|
||||
f"must both be numbers - they are percentages of pack charge.")
|
||||
elif floor >= ceiling:
|
||||
problems.append(
|
||||
f"arbitrage_soc_floor is {floor:g} % and arbitrage_soc_ceiling "
|
||||
f"is {ceiling:g} %, so there is no band left between them: "
|
||||
f"every state of charge is either at or under the floor (no "
|
||||
f"discharge) or at or over the ceiling (no charge), and the "
|
||||
f"strategy can never act. The floor is the reserve it may not "
|
||||
f"spend and the ceiling is the headroom it may not overfill, "
|
||||
f"so the floor must be the SMALLER number.")
|
||||
|
||||
if pid == "manual_csv" and not str(opts.get("price_csv_path", "") or "").strip():
|
||||
problems.append(
|
||||
"price_provider is 'manual_csv' but price_csv_path is empty, so "
|
||||
|
||||
@@ -163,6 +163,37 @@ options:
|
||||
peak_forecast_entity: ""
|
||||
peak_cap_w: 3500
|
||||
|
||||
# --- SAFETY-24: the energy backstop under SAFETY-07's forced discharge -----
|
||||
# The peak guard can COMMAND a discharge to hold a billed quarter under
|
||||
# peak_cap_w. Until this card it was bounded from below by nothing but the
|
||||
# pack's own BMS cut-out - a real floor, but the PACK's protection rather than
|
||||
# the OPERATOR's. On the 2026-08-20 replay the guard asks for 878 Wh in a day.
|
||||
#
|
||||
# peak_soc_floor is the percent below which that discharge is not sized. When
|
||||
# it binds the guard reports FLOOR-LIMITED, keeps charging blocked (a 0 W
|
||||
# floor), and the add-on raises ONE notification saying the quarter will be
|
||||
# exceeded - failing to shave is never silent, because the euro figure is the
|
||||
# whole point of the guard.
|
||||
#
|
||||
# ⚠️ THIS IS NOT maintenance_soc_floor AND MUST NOT BE MERGED WITH IT.
|
||||
# maintenance_soc_floor is the DRAIN TARGET of the monthly recalibration: the
|
||||
# depth the cycle deliberately goes to, once a month, on purpose. This is the
|
||||
# depth a money-saving discharge may never reach on any day. The right values
|
||||
# differ, and sharing one key would couple two features that want to move
|
||||
# independently.
|
||||
#
|
||||
# ⚠️ 20 IS DERIVED, NOT PICKED. The reference pack is 13.8 kWh usable and the
|
||||
# worst modelled guard day is 878 Wh, i.e. 6.4 % - so 10 points above the
|
||||
# pack's own cut-out leaves the guard ~1.6 of its worst observed days of room
|
||||
# and still never reaches the BMS.
|
||||
peak_soc_floor: 20
|
||||
# The pack's OWN cut-out percent - what the BMS enforces whatever is set
|
||||
# above. Register 0x0560 reads 10 on the reference GW5000S-BP, consistent with
|
||||
# DoD 90. It is here so validate.py can REFUSE a peak_soc_floor at or below
|
||||
# it: such a floor can never bind, the BMS wins, and the status page would
|
||||
# report an energy backstop that does not exist.
|
||||
bms_soc_floor: 10
|
||||
|
||||
# ⚠️ THE OLD PRICE PATH, AND IT IS NOT DEAD YET. These two entities are the
|
||||
# add-on delegating the whole price question to Home Assistant: main.py reads
|
||||
# both and derives ONE BOOLEAN, `cheap = now <= avg`, which is consumed by
|
||||
@@ -240,6 +271,38 @@ options:
|
||||
# a second series: no shipped supplier publishes a dynamic injection curve.
|
||||
export_price_eur_kwh: 0.04
|
||||
|
||||
# --- price arbitrage (OPT-04, specs §10.1 priority 3 / §17) ---------------
|
||||
# Charge in the cheapest published hour, discharge in the dearest, and ONLY
|
||||
# when the spread covers what the round trip costs. Off in the shipped
|
||||
# configuration: this is the first strategy in the add-on that moves energy
|
||||
# against the meter on purpose, so it is armed deliberately or not at all.
|
||||
#
|
||||
# ⚠️ IT NEEDS A PRICE SOURCE AND A P1 METER, and validate.py refuses the
|
||||
# combinations that cannot work rather than letting them look armed. With
|
||||
# `price_provider: off` and no `static_profile` there is no series to trade
|
||||
# on; with `meter_source: off` the capacity-tariff peak guard never runs, and
|
||||
# §9.1 puts peak shaving above arbitrage - a charge nothing measures is the
|
||||
# exact failure that guard exists to prevent.
|
||||
#
|
||||
# ⚠️ AT THE SHIPPED THRESHOLD A STATIC DAY/NIGHT SITE DOES NOT TRADE. The
|
||||
# realistic Belgian day/night spread is about 0.08 EUR/kWh and the threshold
|
||||
# below is 0.10, so the verdict is "the firm spread does not cover this
|
||||
# cycle's cost" and nothing happens - correctly. That is a product decision
|
||||
# (this number, and app/price.py's widening factors) and it is open with the
|
||||
# owner; the code is correct at any value and no number was picked to make a
|
||||
# trade appear.
|
||||
arbitrage_enabled: false
|
||||
# §10.1's `cycle_cost_threshold_eur`: what one full round trip costs per kWh
|
||||
# - the wear on the pack plus the conversion losses. A spread below this is
|
||||
# a cycle that loses money and pack life, so it is not taken.
|
||||
cycle_cost_threshold_eur: 0.10
|
||||
# The reserve arbitrage may not spend, and the headroom it may not overfill.
|
||||
# Deliberately NOT the maintenance_soc_* keys: those describe a monthly
|
||||
# balancing cycle, and pointing a trading strategy at them would couple two
|
||||
# unrelated policies through one number.
|
||||
arbitrage_soc_floor: 20
|
||||
arbitrage_soc_ceiling: 95
|
||||
|
||||
# --- site ------------------------------------------------------------------
|
||||
estop_fitted: false
|
||||
log_level: info
|
||||
@@ -311,6 +374,12 @@ schema:
|
||||
|
||||
peak_forecast_entity: str?
|
||||
peak_cap_w: int(500,15000)
|
||||
# SAFETY-24. No "off" value: a guard that can command a discharge does not get
|
||||
# a supported way to have no energy bound at all. The ranges OVERLAP on
|
||||
# purpose - the schema cannot express "above bms_soc_floor", so validate.py
|
||||
# rule 8 is what refuses the pair, with a message naming both numbers.
|
||||
peak_soc_floor: int(5,60)
|
||||
bms_soc_floor: int(0,50)
|
||||
price_now_entity: str?
|
||||
price_avg_entity: str?
|
||||
|
||||
@@ -337,5 +406,15 @@ schema:
|
||||
# bottoms out at 0 would make that site inexpressible.
|
||||
export_price_eur_kwh: float(-1,1)
|
||||
|
||||
# OPT-04. ⚠️ The threshold's lower bound is 0 and not negative: it is a COST,
|
||||
# and a negative one is a threshold every spread clears - arbitrage with the
|
||||
# brake removed. app/arbitrage.py refuses it in code too, because
|
||||
# validate_startup_config() is also sim/inject.py's entry point and can be
|
||||
# handed any dict at all.
|
||||
arbitrage_enabled: bool
|
||||
cycle_cost_threshold_eur: float(0,1)
|
||||
arbitrage_soc_floor: int(5,95)
|
||||
arbitrage_soc_ceiling: int(10,100)
|
||||
|
||||
estop_fitted: bool
|
||||
log_level: list(trace|debug|info|warning|error)
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,551 @@
|
||||
#!/usr/bin/env python3
|
||||
"""DATA-02: fetch 12 months of EPEX BE day-ahead prices, in §9.3's shape.
|
||||
|
||||
python3 scenarios/fetch_epex_be.py --start 2025-09-01 --end 2026-09-01 \
|
||||
--out scenarios/epex-be-dayahead-2025-09-01_2026-09-01.json
|
||||
|
||||
Run from `goodwe_controller/`. Rewrites the output file from scratch; the
|
||||
committed dataset is exactly what this produces, and `--verify` re-derives every
|
||||
claim the file makes about itself without going to the network.
|
||||
|
||||
|
||||
WHAT THIS IS NOT
|
||||
================
|
||||
NOT wired into the add-on, and it must never be. `price.py`'s `GenericDayAhead`
|
||||
is the live path and it has its own egress check; this is a one-shot archival
|
||||
fetch that produces a file for REPLAY, run by hand by a developer. Nothing under
|
||||
`app/` imports it.
|
||||
|
||||
|
||||
THE SOURCE, AND WHY THIS ONE
|
||||
============================
|
||||
`https://api.energy-charts.info/price?bzn=BE` - Fraunhofer ISE's Energy-Charts
|
||||
API. No key, no registration.
|
||||
|
||||
⚠️ THE LICENCE WAS CHECKED BEFORE ANY DATA WAS COMMITTED, because for this
|
||||
source it is NOT uniform across bidding zones. The API's own OpenAPI document
|
||||
(`https://api.energy-charts.info/openapi.json`, /price) splits the zones in two:
|
||||
|
||||
"The data for the following bidding zones is licensed as CC BY 4.0 from
|
||||
Bundesnetzagentur | SMARD.de and is published without changes:
|
||||
AT, BE (Belgium), CH, CZ, DE-LU, ..."
|
||||
|
||||
"The data for the other bidding zones is for private and internal use only.
|
||||
The utilization of any data, whether in its raw or derived form, for
|
||||
external or commercial purposes is expressly prohibited. Should you require
|
||||
licensing for market-related data, please direct your inquiries to the
|
||||
original data providers, including but not limited to EPEX SPOT SE."
|
||||
|
||||
BE is in the FIRST list, so redistribution is permitted with attribution and the
|
||||
dataset is committed rather than left as a fetch script and a checksum. Every
|
||||
response also carries the licence per-response in `license_info`, and this
|
||||
script REFUSES to write a file if that field ever stops saying CC BY 4.0 - a
|
||||
licence checked once by a human and assumed forever is a licence that changes
|
||||
silently. Had BE been in the second list, the correct output of this card would
|
||||
have been this script and a checksum and no data.
|
||||
|
||||
Attribution, carried in the dataset's own `provenance` block and reproduced
|
||||
here: data CC BY 4.0, Bundesnetzagentur | SMARD.de, retrieved via Energy-Charts
|
||||
(Fraunhofer ISE), https://energy-charts.info, published without changes.
|
||||
|
||||
Cross-checked against the upstream on 2026-08-31: SMARD's own chart_data series
|
||||
`4996/DE` over 2026-04-10 matched this API's BE series on all 96 quarter-hours
|
||||
with a maximum absolute difference of 0.00 EUR/MWh. "Published without changes"
|
||||
is a claim worth checking once; `--verify` does not repeat it, because it is a
|
||||
claim about the SOURCE and not about the file.
|
||||
|
||||
|
||||
RAW WHOLESALE, AND THE ASSUMPTION THAT IS NOT BAKED IN
|
||||
======================================================
|
||||
⚠️ WHAT COMES BACK IS RAW WHOLESALE, EUR/MWh, DAY-AHEAD AUCTION CLEARING PRICE.
|
||||
It is NOT all-in: no supplier markup, no distribution or transmission tariff, no
|
||||
levies, no VAT. §9.3's canonical point is all-in, and TARIFF-02's whole trap is
|
||||
a figure that already contains a component being handed to something that adds
|
||||
it again.
|
||||
|
||||
So this file does NOT add one. `eur_per_kwh` in the output is the wholesale
|
||||
figure converted from EUR/MWh, `provenance.all_in` is `false` beside it, and the
|
||||
markup/tax assumption a consumer needs lives in `provenance.all_in_assumption`
|
||||
as NUMBERS TO APPLY rather than numbers already applied. test_arbitrage.py
|
||||
section 7 applies them where a reader can watch it happen.
|
||||
|
||||
The assumption's SHAPE is the part that is not arbitrary, and it decides which
|
||||
conclusions depend on the guessed numbers at all:
|
||||
|
||||
all_in(h) = (wholesale(h) + markup + network_and_levies) * (1 + vat)
|
||||
|
||||
* The two ADDITIVE terms cancel out of a spread: max - min is unchanged by
|
||||
them. So how often a day clears `cycle_cost_threshold_eur` does NOT depend
|
||||
on the markup guess - only on the VAT multiplier.
|
||||
* They do NOT cancel out of §17's negative-price guard, which compares ONE
|
||||
hour's all-in price against zero. A wholesale hour at -0.02 EUR/kWh is
|
||||
comfortably positive all-in once ~0.12 of fixed charges is added.
|
||||
|
||||
Both are asserted arithmetically in section 7 rather than believed, and the
|
||||
trade count is reported on the raw spread as well, so a reader can see exactly
|
||||
how much of the answer rests on the guess.
|
||||
|
||||
The numbers themselves (2026 Belgian residential dynamic contract, order of
|
||||
magnitude) are a STATED GUESS, labelled as one everywhere they appear. They are
|
||||
not a measurement and this card had nothing to measure them with.
|
||||
|
||||
|
||||
HOURS, QUARTER-HOURS, AND THE SEAM IN THE MIDDLE OF THE YEAR
|
||||
============================================================
|
||||
⚠️ THE SOURCE CHANGES RESOLUTION PART-WAY THROUGH ANY RECENT 12 MONTHS. EPEX
|
||||
day-ahead moved to a 15-minute market time unit in October 2025 and the feed
|
||||
follows: 2025-09-10 comes back as 24 hourly points, 2025-12-10 as 96
|
||||
quarter-hourly ones. An importer that assumed either one silently loses three
|
||||
quarters of the year, or reads a quarter as an hour.
|
||||
|
||||
The canonical format is hourly, so quarter-hours are aggregated: an hour's price
|
||||
is the unweighted mean of its four quarters. That is a STATED APPROXIMATION of
|
||||
the hourly product, not the hourly product, and it errs in the direction that
|
||||
matters here - averaging discards intra-hour variation, so the observed spread
|
||||
is a LOWER bound on what a 15-minute strategy would see.
|
||||
`provenance.resolution` records how many hours came from each source shape.
|
||||
|
||||
⚠️ AND A PARTIAL HOUR IS DROPPED, NOT AVERAGED OVER WHAT ARRIVED. An hour with
|
||||
three of four quarters is an hour nobody published a complete price for; the
|
||||
mean of three would be an invented number carrying no flag. Dropped hours are
|
||||
listed in `provenance.gaps` with their timestamps, because a missing hour in a
|
||||
feed is not a zero price, and a zero price is exactly the value that would make
|
||||
everything look free.
|
||||
|
||||
|
||||
SIZE: WHY THIS IS PLAIN JSON AND NOT A `.gz`, MEASURED
|
||||
======================================================
|
||||
The directory's existing captures are gzipped, so the default would have been to
|
||||
gzip this too. Measured on the committed file instead of assumed:
|
||||
|
||||
working tree, plain JSON 893 171 bytes
|
||||
what GIT actually stores for it (zlib, loose object) 74 590 bytes
|
||||
what git would store for a committed `.json.gz` 56 504 bytes
|
||||
|
||||
⚠️ GIT ALREADY COMPRESSES BLOBS, so gzipping buys 18 KB - 24 % of an object
|
||||
that is already 75 KB, in a repository whose three existing P1 captures cost
|
||||
27 KB, 136 KB and 140 KB. Eighteen kilobytes is not worth making the provenance
|
||||
block unreadable in a pull request, and the licence claim in that block is
|
||||
precisely the thing a reviewer of this card has to read and check. Plain JSON
|
||||
also greps, diffs line-by-line when a future refetch changes something, and is
|
||||
what `--verify` opens. `load()` still handles `.gz` for a future dataset large
|
||||
enough to want it; this one is not.
|
||||
|
||||
The P1 captures are gzipped for the opposite reason and it is not an
|
||||
inconsistency: they are hundreds of thousands of telegram lines that no human
|
||||
reads, and there is no metadata in them to review.
|
||||
|
||||
|
||||
TIME
|
||||
====
|
||||
Every instant is carried as UTC and rendered with `app.price.brussels()`, the
|
||||
module that already owns the EU DST rule without a tz database. There is no
|
||||
second answer to that question here and there must not be: the point of
|
||||
`brussels()` is that the 02:00-03:00 hour which happens twice in October is two
|
||||
instants, and a 24-slot local-hour importer collapses them.
|
||||
|
||||
So a year is 8760 hours only if it contains no transition, or contains both and
|
||||
they cancel - 25 in October, 23 in March. `--verify` therefore asserts the
|
||||
PER-DAY counts, not the total, because the total is exactly the number a broken
|
||||
importer also produces.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import gzip
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from collections import Counter, defaultdict
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from app.price import brussels # noqa: E402
|
||||
|
||||
API = "https://api.energy-charts.info/price"
|
||||
BZN = "BE"
|
||||
LICENCE_MUST_CONTAIN = "CC BY 4.0"
|
||||
UTC = timezone.utc
|
||||
|
||||
ATTRIBUTION = ("Data licensed CC BY 4.0 by Bundesnetzagentur | SMARD.de, "
|
||||
"retrieved via the Energy-Charts API (Fraunhofer ISE), "
|
||||
"https://energy-charts.info. Published without changes.")
|
||||
|
||||
# ⚠️ A GUESS, LABELLED AS ONE, AND APPLIED BY THE CONSUMER - NOT HERE.
|
||||
# Order-of-magnitude 2026 Belgian residential dynamic contract. Nothing in this
|
||||
# repository measured any part of it.
|
||||
ALL_IN_ASSUMPTION = {
|
||||
"form": "all_in = (wholesale + markup + network_and_levies) * (1 + vat)",
|
||||
"markup_eur_per_kwh": 0.020,
|
||||
"network_and_levies_eur_per_kwh": 0.100,
|
||||
"vat_fraction": 0.06,
|
||||
"status": "ASSUMPTION, NOT MEASURED. Stated so a consumer applies it "
|
||||
"deliberately; the committed eur_per_kwh has none of it applied.",
|
||||
"note": "The two additive terms cancel out of any spread, so an arbitrage "
|
||||
"trade count depends on vat_fraction only. They do NOT cancel out "
|
||||
"of the negative-price guard (17), which compares one hour's "
|
||||
"all-in price against zero.",
|
||||
}
|
||||
|
||||
|
||||
PACE_S = 3.0 # between chunks, unconditionally
|
||||
RETRIES = 6
|
||||
|
||||
|
||||
def _get(url: str) -> dict:
|
||||
"""One GET, backing off on 429 and 5xx.
|
||||
|
||||
⚠️ THIS IS A FREE PUBLIC SERVICE AND IT RATE-LIMITS. The first run of this
|
||||
script was refused with 429 on its fifth chunk. Backing off is the polite
|
||||
behaviour; it is also the correct one, because the alternative - catching
|
||||
the error and carrying on - would produce a file that is silently missing a
|
||||
month and says nothing about it. A refused request is not an empty month.
|
||||
"""
|
||||
req = urllib.request.Request(
|
||||
url,
|
||||
headers={"User-Agent": "goodwe-addon/DATA-02 (one-shot archival fetch)"})
|
||||
wait = 5.0
|
||||
for attempt in range(1, RETRIES + 1):
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=90) as r:
|
||||
return json.loads(r.read().decode("utf-8"))
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code not in (429, 500, 502, 503, 504) or attempt == RETRIES:
|
||||
raise
|
||||
print(f" HTTP {e.code}; backing off {wait:.0f}s "
|
||||
f"(attempt {attempt}/{RETRIES})", file=sys.stderr)
|
||||
time.sleep(wait)
|
||||
wait *= 2
|
||||
raise SystemExit(f"unreachable: {url}")
|
||||
|
||||
|
||||
def local_midnight_utc(day: str) -> datetime:
|
||||
"""The INSTANT at which Brussels midnight begins on `day` (YYYY-MM-DD).
|
||||
|
||||
⚠️ THE WINDOW IS LOCAL, NOT UTC, AND THE FIRST RUN OF THIS SCRIPT PROVED WHY.
|
||||
Bounding it at 00:00 UTC starts the year at 02:00 CEST, which makes the first
|
||||
Brussels day 22 hours long - and 22 is not one of the two lengths a
|
||||
transition produces, so `report()` refused the file. A dataset whose first
|
||||
day is short is a dataset whose "days per year" arithmetic is quietly wrong
|
||||
everywhere downstream.
|
||||
|
||||
Derived from `brussels()` by trying the only two offsets Belgium has and
|
||||
keeping the one that round-trips, rather than by restating the EU rule here.
|
||||
"""
|
||||
for hours in (1, 2):
|
||||
guess = datetime.fromisoformat(f"{day}T00:00:00").replace(
|
||||
tzinfo=timezone(timedelta(hours=hours))).astimezone(UTC)
|
||||
if brussels(guess).isoformat() == f"{day}T00:00:00+0{hours}:00":
|
||||
return guess
|
||||
raise SystemExit(f"no Brussels midnight for {day!r} - brussels() disagrees "
|
||||
f"with itself, which is a bug in price.py, not here.")
|
||||
|
||||
|
||||
def _months(start: datetime, end: datetime):
|
||||
"""[start, end) in month-sized chunks, so a bad chunk names its own window."""
|
||||
a = start
|
||||
while a < end:
|
||||
b = min(end, (a.replace(day=1) + timedelta(days=32)).replace(day=1))
|
||||
yield a, b
|
||||
a = b
|
||||
|
||||
|
||||
def fetch(start: datetime, end: datetime):
|
||||
"""Raw {unix_second: EUR/MWh} over [start, end), deduplicated.
|
||||
|
||||
⚠️ A `null` price is DROPPED, never coerced. The API returns null for an
|
||||
interval it holds no result for, and `or 0.0` is the version of this line
|
||||
that ships a year of free electricity.
|
||||
"""
|
||||
seen, licences, nulls = {}, Counter(), 0
|
||||
for a, b in _months(start, end):
|
||||
# ⚠️ THE API'S `end` IS AN INCLUSIVE LOCAL DAY, AND THE WINDOW'S IS AN
|
||||
# EXCLUSIVE INSTANT. Ask for the day containing the LAST hour of the
|
||||
# chunk, derived from the instant rather than by subtracting a calendar
|
||||
# day from the bound: `b - 1 day` drops the final local day whenever the
|
||||
# window ends at local midnight, which is exactly how this window ends.
|
||||
# The first run lost 2026-08-31 that way, silently - 8736 points instead
|
||||
# of 8760 - and the points-plus-gaps accounting in report() is what
|
||||
# noticed. A day nobody asked for is not a day with no prices.
|
||||
first = brussels(a).date().isoformat()
|
||||
last = brussels(b - timedelta(hours=1)).date().isoformat()
|
||||
url = f"{API}?bzn={BZN}&start={first}&end={last}"
|
||||
d = _get(url)
|
||||
lic = str(d.get("license_info", ""))
|
||||
licences[lic] += 1
|
||||
if LICENCE_MUST_CONTAIN not in lic:
|
||||
raise SystemExit(
|
||||
f"REFUSING TO WRITE. {url} came back under licence {lic!r}, "
|
||||
f"which does not say {LICENCE_MUST_CONTAIN!r}. The BE zone was "
|
||||
f"redistributable when this card ran; if that has changed, the "
|
||||
f"right output is this script and a checksum, not the data.")
|
||||
unit = str(d.get("unit", ""))
|
||||
if "MWh" not in unit:
|
||||
raise SystemExit(
|
||||
f"REFUSING TO WRITE. {url} reports unit {unit!r}, not EUR/MWh. "
|
||||
f"A silent unit change is a factor of 1000 on every price.")
|
||||
for t, p in zip(d["unix_seconds"], d["price"]):
|
||||
if p is None:
|
||||
nulls += 1
|
||||
continue
|
||||
if start.timestamp() <= t < end.timestamp():
|
||||
seen[int(t)] = float(p)
|
||||
print(f" {a.date()} .. {b.date()} {len(d['price']):5d} intervals",
|
||||
file=sys.stderr)
|
||||
time.sleep(PACE_S)
|
||||
return seen, licences, nulls
|
||||
|
||||
|
||||
def to_hourly(raw: dict):
|
||||
"""Hourly means keyed by UTC hour. Returns (hours, gaps, shape_counts).
|
||||
|
||||
An hour is emitted only when every interval the source used for it arrived:
|
||||
1 of 1 where the source was hourly, 4 of 4 where it was quarter-hourly.
|
||||
Anything else is a gap and is named as one.
|
||||
"""
|
||||
buckets = defaultdict(dict)
|
||||
for t, p in raw.items():
|
||||
buckets[t - (t % 3600)][t % 3600] = p
|
||||
|
||||
hours, gaps, shapes = {}, [], Counter()
|
||||
for h in sorted(buckets):
|
||||
got = buckets[h]
|
||||
if set(got) == {0}:
|
||||
shapes["hourly"] += 1
|
||||
hours[h] = got[0]
|
||||
elif set(got) == {0, 900, 1800, 2700}:
|
||||
shapes["quarter_hourly_mean"] += 1
|
||||
hours[h] = sum(got.values()) / 4.0
|
||||
else:
|
||||
gaps.append({
|
||||
"hour_utc": datetime.fromtimestamp(h, UTC).isoformat(),
|
||||
"intervals_present": sorted(got),
|
||||
"why": "incomplete hour - dropped rather than averaged over "
|
||||
"what arrived; a partial read is not an observation",
|
||||
})
|
||||
return hours, gaps, shapes
|
||||
|
||||
|
||||
def canonical(hours: dict) -> list:
|
||||
"""9.3 points. THREE FIELDS, rendered at the Brussels offset for the instant."""
|
||||
return [{"start": brussels(datetime.fromtimestamp(h, UTC)).isoformat(),
|
||||
"eur_per_kwh": round(hours[h] / 1000.0, 6) + 0.0,
|
||||
"provisional": False}
|
||||
for h in sorted(hours)]
|
||||
|
||||
|
||||
def expected_hours(start: datetime, end: datetime) -> int:
|
||||
"""Whole hours in the window. Derived, rather than 8760 typed in."""
|
||||
return int((end - start).total_seconds() // 3600)
|
||||
|
||||
|
||||
def per_local_day(points) -> Counter:
|
||||
"""Points per BRUSSELS calendar day - the count that exposes a DST bug."""
|
||||
n = Counter()
|
||||
for p in points:
|
||||
n[p["start"][:10]] += 1
|
||||
return n
|
||||
|
||||
|
||||
def digest(points) -> str:
|
||||
"""A checksum over the POINTS, so reformatting the envelope does not move it."""
|
||||
h = hashlib.sha256()
|
||||
for p in points:
|
||||
h.update(f"{p['start']}|{p['eur_per_kwh']:.6f}|{int(p['provisional'])}\n"
|
||||
.encode())
|
||||
return h.hexdigest()
|
||||
|
||||
|
||||
def load(path: str) -> dict:
|
||||
op = gzip.open if path.endswith(".gz") else open
|
||||
with op(path, "rt", encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
|
||||
|
||||
def report(doc, *, start=None, end=None) -> int:
|
||||
"""Re-derive every claim the file makes about itself. Returns failures."""
|
||||
pts = doc["prices"]
|
||||
prov = doc["provenance"]
|
||||
bad = 0
|
||||
|
||||
def ck(name, cond):
|
||||
nonlocal bad
|
||||
print(f" {'ok ' if cond else 'FAIL'} {name}")
|
||||
if not cond:
|
||||
bad += 1
|
||||
|
||||
ck(f"the checksum over the {len(pts)} points matches the recorded one "
|
||||
f"({prov['sha256_of_points'][:16]}...)",
|
||||
digest(pts) == prov["sha256_of_points"])
|
||||
ck("every point has exactly the three canonical fields and nothing else",
|
||||
all(set(p) == {"start", "eur_per_kwh", "provisional"} for p in pts))
|
||||
ck("every start carries a UTC offset, and it is +01:00 or +02:00 - a naive "
|
||||
"timestamp here is a wall clock plus an assumption",
|
||||
bool(pts) and all(p["start"][-6:] in ("+01:00", "+02:00") for p in pts))
|
||||
ck("the series is strictly increasing in time, so no instant is duplicated",
|
||||
all(datetime.fromisoformat(a["start"]) < datetime.fromisoformat(b["start"])
|
||||
for a, b in zip(pts, pts[1:])))
|
||||
ck("consecutive points are exactly one hour apart AS INSTANTS, so the two "
|
||||
"02:00s of the October day are two hours rather than one",
|
||||
all((datetime.fromisoformat(b["start"])
|
||||
- datetime.fromisoformat(a["start"])) == timedelta(hours=1)
|
||||
for a, b in zip(pts, pts[1:])))
|
||||
|
||||
days = per_local_day(pts)
|
||||
odd = {d: n for d, n in days.items() if n != 24}
|
||||
ck(f"exactly two Brussels days are not 24 hours long, and they are the "
|
||||
f"transitions: {sorted(odd.items())}",
|
||||
sorted(odd.values()) == [23, 25] and len(odd) == 2)
|
||||
spring = [d for d, n in odd.items() if n == 23]
|
||||
autumn = [d for d, n in odd.items() if n == 25]
|
||||
ck(f"the 23-hour day {spring} is in March and the 25-hour day {autumn} is "
|
||||
f"in October, which is the direction the EU rule actually runs",
|
||||
len(spring) == 1 and len(autumn) == 1
|
||||
and spring[0][5:7] == "03" and autumn[0][5:7] == "10")
|
||||
if autumn:
|
||||
twice = [p["start"] for p in pts
|
||||
if p["start"].startswith(autumn[0] + "T02")]
|
||||
ck(f"...and 02:00 on {autumn[0]} occurs TWICE at two different offsets "
|
||||
f"{twice} - exactly the case a 24-slot local-hour array collapses",
|
||||
len(twice) == 2 and twice[0][-6:] == "+02:00"
|
||||
and twice[1][-6:] == "+01:00")
|
||||
ck(f"...and the two carry DIFFERENT prices, so the 25th hour is a real "
|
||||
f"observation and not the same one written down twice",
|
||||
len({p["eur_per_kwh"] for p in pts
|
||||
if p["start"].startswith(autumn[0] + "T02")}) == 2)
|
||||
if spring:
|
||||
ck(f"...and 02:00 on {spring[0]} does not occur at all, because that "
|
||||
f"wall clock reading never happened in Belgium",
|
||||
not any(p["start"].startswith(spring[0] + "T02") for p in pts))
|
||||
|
||||
if start and end:
|
||||
want = expected_hours(start, end)
|
||||
ck(f"{len(pts)} points for a window of {want} whole hours, and the "
|
||||
f"{len(prov['gaps'])} recorded gaps account for the difference "
|
||||
f"exactly", len(pts) + len(prov["gaps"]) == want)
|
||||
|
||||
# ⚠️ A ZERO IS NOT AUTOMATICALLY A DEFECT HERE, AND SAYING IT WAS WOULD BE
|
||||
# THE WRONG LESSON. The Belgian market really does clear at 0.00 EUR/MWh in
|
||||
# solar-glut hours; this window contains such hours. What must be impossible
|
||||
# is a zero that arrived by COERCION - a dropped read turned into a free
|
||||
# hour. That is excluded structurally rather than by outlawing the value:
|
||||
# nulls are dropped, incomplete hours become gaps, and the accounting above
|
||||
# shows points + gaps covering every hour of the window, so every emitted
|
||||
# point stands for intervals the source actually published.
|
||||
zeros = [p["start"] for p in pts if p["eur_per_kwh"] == 0.0]
|
||||
ck(f"the {len(zeros)} hours priced at exactly zero match the count recorded "
|
||||
f"at build time, so no zero appeared between the fetch and the file",
|
||||
len(zeros) == prov["exact_zero_hours"])
|
||||
ck("...and none of them is a coerced read: every emitted hour is backed by "
|
||||
"published intervals, which is what the points-plus-gaps accounting "
|
||||
"above establishes",
|
||||
len(prov["gaps"]) == 0 or all("intervals_present" in g
|
||||
for g in prov["gaps"]))
|
||||
ck("the figures are declared RAW WHOLESALE, and the all-in assumption is "
|
||||
"recorded as an assumption rather than applied",
|
||||
prov["all_in"] is False
|
||||
and "ASSUMPTION" in prov["all_in_assumption"]["status"])
|
||||
return bad
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
ap.add_argument("--start", default="2025-09-01")
|
||||
ap.add_argument("--end", default="2026-09-01",
|
||||
help="exclusive; both bounds are BRUSSELS-local midnights")
|
||||
ap.add_argument("--out", default=None)
|
||||
ap.add_argument("--verify", default=None,
|
||||
help="re-derive the claims of an existing file; no network")
|
||||
a = ap.parse_args()
|
||||
|
||||
if a.verify:
|
||||
doc = load(a.verify)
|
||||
print(f"verifying {a.verify}")
|
||||
s = datetime.fromisoformat(doc["provenance"]["window_utc"]["start"])
|
||||
e = datetime.fromisoformat(doc["provenance"]["window_utc"]["end"])
|
||||
bad = report(doc, start=s, end=e)
|
||||
print("OK" if not bad else f"{bad} FAILED")
|
||||
return 1 if bad else 0
|
||||
|
||||
start = local_midnight_utc(a.start)
|
||||
end = local_midnight_utc(a.end)
|
||||
out = a.out or f"scenarios/epex-be-dayahead-{a.start}_{a.end}.json"
|
||||
|
||||
print(f"fetching {BZN} day-ahead {start.date()} .. {end.date()}",
|
||||
file=sys.stderr)
|
||||
raw, licences, nulls = fetch(start, end)
|
||||
hours, gaps, shapes = to_hourly(raw)
|
||||
pts = canonical(hours)
|
||||
|
||||
doc = {
|
||||
"provenance": {
|
||||
"card": "DATA-02",
|
||||
"what": "EPEX BE day-ahead auction clearing prices, hourly",
|
||||
"source": f"{API}?bzn={BZN} (Energy-Charts API, Fraunhofer ISE)",
|
||||
"upstream": "Bundesnetzagentur | SMARD.de",
|
||||
"licence": "CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)",
|
||||
"licence_checked": "before committing anything; the API's OpenAPI "
|
||||
"document lists BE among the CC BY 4.0 zones and "
|
||||
"the remaining zones as private/internal use "
|
||||
"only, so redistribution of THIS zone is "
|
||||
"permitted with attribution",
|
||||
"licence_as_returned": sorted(licences),
|
||||
"attribution": ATTRIBUTION,
|
||||
"cross_check": "SMARD chart_data 4996/DE over 2026-04-10 matched "
|
||||
"this feed on all 96 quarter-hours, max abs "
|
||||
"difference 0.00 EUR/MWh (2026-08-31)",
|
||||
"retrieved_utc": datetime.now(UTC).replace(microsecond=0).isoformat(),
|
||||
"fetched_by": "scenarios/fetch_epex_be.py",
|
||||
"window_utc": {"start": start.isoformat(), "end": end.isoformat()},
|
||||
"unit": "EUR per kWh (source publishes EUR/MWh; divided by 1000)",
|
||||
"all_in": False,
|
||||
"all_in_note": "RAW WHOLESALE. No supplier markup, no network or "
|
||||
"transmission tariff, no levies, no VAT. 9.3's "
|
||||
"canonical point is all-in and this is NOT; the "
|
||||
"consumer applies the assumption below, visibly.",
|
||||
"all_in_assumption": ALL_IN_ASSUMPTION,
|
||||
"resolution": {
|
||||
"note": "EPEX day-ahead moved to a 15-minute market time unit "
|
||||
"in Oct 2025, part-way through this window.",
|
||||
"hours_from_hourly_source": shapes["hourly"],
|
||||
"hours_from_quarter_hourly_mean": shapes["quarter_hourly_mean"],
|
||||
"aggregation": "unweighted mean of the four quarters; a stated "
|
||||
"approximation of the hourly product, and a "
|
||||
"LOWER bound on the spread a 15-minute strategy "
|
||||
"would have seen",
|
||||
},
|
||||
"timezone": "starts rendered at the Brussels offset by "
|
||||
"app.price.brussels(); instants carried in UTC, so the "
|
||||
"transition days come out 23 and 25 points by "
|
||||
"construction rather than by a special case",
|
||||
"nulls_dropped": nulls,
|
||||
"gaps": gaps,
|
||||
"gaps_note": "hours the source did not publish completely. Dropped, "
|
||||
"never zero-filled: a missing hour is not a free hour.",
|
||||
"points": len(pts),
|
||||
"exact_zero_hours": sum(1 for p in pts if p["eur_per_kwh"] == 0.0),
|
||||
"negative_hours": sum(1 for p in pts if p["eur_per_kwh"] < 0),
|
||||
"zero_note": "REAL CLEARINGS AT 0.00 EUR/MWh, not dropped reads. "
|
||||
"Nulls are dropped and incomplete hours become gaps, "
|
||||
"so points + gaps == hours in the window and no point "
|
||||
"stands in for something that failed to arrive.",
|
||||
"sha256_of_points": digest(pts),
|
||||
},
|
||||
"prices": pts,
|
||||
}
|
||||
|
||||
with open(out, "w", encoding="utf-8", newline="\n") as f:
|
||||
json.dump(doc, f, indent=1, ensure_ascii=False)
|
||||
f.write("\n")
|
||||
print(f"wrote {out} {len(pts)} points {os.path.getsize(out)} bytes",
|
||||
file=sys.stderr)
|
||||
print(f"verifying {out}")
|
||||
return 1 if report(doc, start=start, end=end) else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -212,6 +212,14 @@ def shipped(key):
|
||||
if in_schema or not line.startswith(f" {key}:"):
|
||||
continue
|
||||
raw = line.split(":", 1)[1].split("#", 1)[0].strip().strip('"').strip("'")
|
||||
# OPT-04 added the first BOOLEAN price option, and without this the
|
||||
# mirror check would compare the string "false" against `False` and go
|
||||
# red for a file that is perfectly in step. Parsing a bool as a bool is
|
||||
# what the yaml module this image does not carry would have done; it
|
||||
# widens nothing, because a key whose shipped value disagrees with the
|
||||
# mirror still drifts.
|
||||
if raw in ("true", "false"):
|
||||
return raw == "true"
|
||||
try:
|
||||
return int(raw)
|
||||
except ValueError:
|
||||
|
||||
@@ -71,6 +71,7 @@ from app.price_forecast import (
|
||||
MAINTENANCE_CHARGE, MAINTENANCE_DRAIN, Gate, Irreversible, committed_hours,
|
||||
fill_from_yesterday, price_permits, begin_phase,
|
||||
)
|
||||
from app.arbitrage import ArbitrageDecision
|
||||
from app.static_tariff import Arbitrage, TariffPlan
|
||||
|
||||
UTC = timezone.utc
|
||||
@@ -582,7 +583,13 @@ _ctors = {"Gate.permit": Gate.permit, "Gate.block": Gate.block,
|
||||
"Arbitrage.allow": Arbitrage.allow, "Arbitrage.refuse": Arbitrage.refuse,
|
||||
"Arbitrage.match": Arbitrage.match,
|
||||
"_Allowed.__init__": _st._Allowed.__init__,
|
||||
"_Refused.__init__": _st._Refused.__init__}
|
||||
"_Refused.__init__": _st._Refused.__init__,
|
||||
# ⚠️ OPT-04's outcome type is scanned HERE, in the same dict, rather
|
||||
# than by a hand-written case in test_arbitrage.py - which is TEST-14's
|
||||
# actual instruction. A new outcome type extends the SCAN; a
|
||||
# hand-written case is the thing that gets forgotten, and the whole
|
||||
# reason this dict exists instead of five separate checks.
|
||||
"ArbitrageDecision.__init__": ArbitrageDecision.__init__}
|
||||
_with_defaults = derived(lambda: {n: d for n, d in
|
||||
((n, _defaulted(f)) for n, f in _ctors.items())
|
||||
if d})
|
||||
@@ -835,13 +842,28 @@ print("7. CFG-03 and validate.py rule 7 - what this ticket did NOT widen")
|
||||
_defaults = derived(lambda: sorted(_price.PRICE_OPTION_DEFAULTS))
|
||||
_cfg = derived(lambda: open(os.path.join(HERE, "config.yaml"),
|
||||
encoding="utf-8").read())
|
||||
check(f"FCST-03 added ZERO option keys: the in-code mirror still holds exactly "
|
||||
f"TARIFF-02's and TARIFF-03's {_defaults}",
|
||||
# ⚠️ THIS LIST IS A DRIFT PIN, NOT A BAN ON EVER ADDING AN OPTION. FCST-03 added
|
||||
# none, and said so here. OPT-04 added FOUR - and added them to this ONE dict
|
||||
# and to config.yaml, which is precisely the behaviour CFG-03 asks for: the
|
||||
# alternative is a fourth freehand copy in main.py's `o.get(k, X)` fallbacks.
|
||||
# The pin is what makes such an addition a decision somebody signs rather than a
|
||||
# thing that appears; whoever adds the fifth comes here and says which card and
|
||||
# why, as this line does.
|
||||
_OPT04_KEYS = ["arbitrage_enabled", "cycle_cost_threshold_eur",
|
||||
"arbitrage_soc_floor", "arbitrage_soc_ceiling"]
|
||||
check(f"the in-code mirror holds exactly TARIFF-02's, TARIFF-03's and OPT-04's "
|
||||
f"keys - FCST-03 still added none of its own {_defaults}",
|
||||
_defaults == sorted(["price_provider", "price_markup_eur_kwh",
|
||||
"price_taxes_eur_kwh", "export_price_eur_kwh",
|
||||
"price_csv_path", "price_feed_url",
|
||||
"price_horizon_h", "price_timeout_s",
|
||||
"static_profile"]))
|
||||
"static_profile"] + _OPT04_KEYS))
|
||||
# ...and the four are genuinely NEW, not a rename of something that was already
|
||||
# there, or the line above would pass while the mirror silently lost a key.
|
||||
check("...and OPT-04's four keys are additions rather than replacements: the "
|
||||
"nine TARIFF-02/03 keys are all still present",
|
||||
isinstance(_defaults, list)
|
||||
and len(set(_defaults) - set(_OPT04_KEYS)) == 9)
|
||||
check("...and every one of them is still in config.yaml, which stays the "
|
||||
"source of truth",
|
||||
isinstance(_cfg, str)
|
||||
|
||||
Reference in New Issue
Block a user