diff --git a/goodwe_controller/CHANGELOG.md b/goodwe_controller/CHANGELOG.md index 45a0b42..d52861c 100644 --- a/goodwe_controller/CHANGELOG.md +++ b/goodwe_controller/CHANGELOG.md @@ -18,6 +18,7 @@ if it has, it needs 0.10.0. Whoever deploys decides. | _this branch_ | TARIFF-08 - the day-ahead feed is read in the shape it actually publishes, and the fixtures stop agreeing with the parser | | _this branch_ | TARIFF-09 - the day-ahead feed is asked for the horizon it was given, so tomorrow arrives firm instead of carried | | _this branch_ | TARIFF-10 - configuring a price provider is what fetches prices; trading stays its own switch | +| _this branch_ | TARIFF-12 - a supplier outage stops deleting the plan on its first failed fetch, and TARIFF_MAX_AGE_S starts governing | ### TARIFF-10 - a configured provider is fetched, whether or not you trade @@ -105,6 +106,84 @@ then the fetch is **driven** with a counting transport, because a graph is an argument and a counted request is an observation. The shipped defect - the gate back at `arbitrage.enabled(opts)`, **-4 B** - is red on seven named checks. +### TARIFF-12 - a returned degradation is not a refresh + +**`run_tariff_refresh()` promised in its own docstring that "a failure does not +clear the plan", and enforced it with `except Exception` - a branch a supplier +outage never reaches.** `load_prices()` states the reason in its own words: it +"RETURNS, NEVER RAISES, for every failure a site can actually have", because a +degradation arriving as a traceback into the planning tick would stop the +controller regulating over an expired certificate. So a dead feed **returns** a +§11.4 plan, the `try` succeeds, and the assignment below it overwrote the good +plan unconditionally. With `static_profile` unset - the shipped default, and the +normal state of a dynamic-tariff site - `_degrade()` returns +`TariffPlan(no_data, None, "none")`: **a complete 48-hour plan replaced by +nothing.** + +Measured in a 40-hour soak, `static_profile` empty, a 90-minute outage driven: + +| Brussels | refresh | plan | +|---|---|---| +| 17:26:12 | 55 | `dynamic_dayahead from generic_dayahead, 141 h, provisional` | +| **17:56:12** | **56** | **`no tariff - generic_dayahead: ClientConnectorError: Cannot connect to host`** | +| 19:26:12 | 59 | restored, one refresh after the feed returned | + +`origin: none`, `n_points: 0` for all 30 outage samples. The plan was gone ~27 +minutes after the last good fetch, **at the first failure** - so +`TARIFF_MAX_AGE_S` (5400 s) never governed anything, and there was no +serve-stale window at all: a transient blip cost exactly what a long outage did. + +**The question is now asked once, after the attempt, from the evidence that +exists** - FCST-06's answer, one module over. The evidence is +`TariffPlan.from_feed` (did this resolution observe the configured feed) and the +shipped `fresh_tariff_plan()` (is an observation still current). A degradation +replaces a standing observation only once that observation has **aged out**, so +`TARIFF_MAX_AGE_S` is now the serve-stale window it reads like. + +**A §11.4 static fallback does not replace a fresh provider plan either**, +although it carries real prices. Every point `_degrade()` builds is +`provisional=True`, so `firm_points()` is empty, `arbitrage()` refuses and +`cheap_now()` is False - everything a consumer may act on is already off - while +a day-ahead price for a given hour does not change once published. The fallback +is less actionable and no more current; it takes over when the observation +retires, which is what §11.4 is for. The rule is **not** symmetrical: a standing +plan that is itself a degradation is replaced by the next one, or an outage +would freeze the operator's static profile at the window it was first generated +for. + +**`tariff_at` does not move on a kept plan.** It is the moment of the fetch that +produced the plan standing, `fresh_tariff_plan()` judges the age from it, and an +unmoved timestamp is the only thing that tells a kept plan from a freshly +fetched one. Re-stamping it would be the refresh asserting the feed answered +when it did not - the "freeze at its last value and look current forever" hazard +the same docstring names. + +**What did NOT change.** The persisted price history still does not back the +plan during an outage: FCST-03 exists to stop stale prices being acted on, and +"recent" is not "current". Nothing about the control loop, the arbiter, or the +30 s tick - `cycle()` still reads two attributes and awaits no fetch. + +`test_arbitrage.py` section 8b drives the whole chain with **nothing stubbed but +the socket**: the transport raises the way aiohttp does, `load_prices()` returns +`no_data`, `_degrade()` builds the §11.4 plan, and the shipped +`run_tariff_refresh()` decides. A check that patched `resolve_tariff` to *raise* +would have passed against the defect. Eleven mutations, eleven reds, no +survivors - see `DECISIONS/TARIFF-12-a-degradation-is-not-a-refresh.md`. + +**Merged after TARIFF-10, which widened *which installs run this task*, and the +two configurations it newly keeps alive are graded rather than reasoned about.** +A provider with trading disarmed is the case this guard was designed for and is +covered by every driven check here, because nothing in `run_tariff_refresh()` +reads a trading option at all - asserted on the AST symbols rather than on the +source text, after two drafts of that check went red on a correct tree for +saying "arbitrage" in a docstring and in a log line. `price_provider: off` + a +`static_profile` resolves to the CONFIGURED product (`origin: "static"`), so +`standing.from_feed` is false on every refresh and the guard never latches: the +plan is regenerated for the current window exactly as it always was. Both are +driven, and a mutation that makes the guard depend on `arbitrage.enabled()` is +red on five named checks. + + ### TARIFF-09 - the feed is asked for the range the caller wanted **`generic_dayahead` took `since` and `until` and requested the bare feed URL, diff --git a/goodwe_controller/DECISIONS/TARIFF-12-a-degradation-is-not-a-refresh.md b/goodwe_controller/DECISIONS/TARIFF-12-a-degradation-is-not-a-refresh.md new file mode 100644 index 0000000..9764cc8 --- /dev/null +++ b/goodwe_controller/DECISIONS/TARIFF-12-a-degradation-is-not-a-refresh.md @@ -0,0 +1,252 @@ +# TARIFF-12 — a returned degradation is not a refresh + +Three decisions. The first is what the ticket put up for judgement; the second +and third are what fall out of it and would otherwise be decided by accident. + +--- + +## 0. What was actually wrong, stated once + +`run_tariff_refresh()`'s docstring: + +> ⚠️ **AND A FAILURE DOES NOT CLEAR THE PLAN.** … Zeroing the plan here would +> replace a coarse tariff with no tariff, which is the opposite of what §11.4 +> asks for. + +The guarantee was real. The condition it was enforced on was not: + +```python +try: + plan = await resolve_tariff(...) + self.tariff_plan, self.tariff_at = plan, now # unconditional +except Exception as err: + _LOG.warning("tariff refresh failed (%s); keeping the plan ...") +``` + +`load_prices()` says in its own docstring that it **"RETURNS, NEVER RAISES, for +every failure a site can actually have"**, because a degradation arriving as a +traceback into the planning tick would stop the controller regulating over an +expired certificate. So a supplier outage — the case the docstring names first — +is a **return**: the `try` succeeds, `_degrade()` hands back a §11.4 plan, and +the good plan is overwritten. The `except` is reachable only by something +nobody expected, and nothing in the shipped path produces it. + +This is the fourth card in one family: FCST-04's M4 (`not ok` only), FCST-05's +HIGH (`(JSONDecodeError, OSError)` only), FCST-06 (`not self.ok` only), and now +`except Exception` only, for a path that never raises. **A guarantee enforced on +the condition someone thought of, rather than the condition that occurs.** + +Reproduced before the fix, driving the shipped chain with nothing stubbed but +the socket — 48 h horizon, `static_profile` empty, one good fetch then a +`ClientConnectorError` 1800 s later, `TARIFF_MAX_AGE_S` = 5400 s: + +``` +after a good fetch origin='provider' n_points=48 'dynamic_dayahead from generic_dayahead, 48 h' +after the failed fetch origin='none' n_points=0 'no tariff - generic_dayahead: ClientConnectorError: ...' +tariff_at moved: True +``` + +and after it: + +``` +after the failed fetch origin='provider' n_points=48 'dynamic_dayahead from generic_dayahead, 48 h' +tariff_at moved: False +``` + +--- + +## 1. The question is asked ONCE, after the attempt, from the evidence that exists + +FCST-06's answer, one module over, and for the same reason: **the enumeration +of failure conditions cannot be completed.** There is no pre-check that a feed +will answer which is not itself a guess, and adding one would be a second +symptom test beside the `except`. So the predicate is an **outcome**, read off +the object the attempt produced: + +```python +standing = self.fresh_tariff_plan(now) +keep = (not plan.from_feed + and standing is not None and standing.from_feed) +``` + +Two facts, both of which exist by the time the question is asked: + +* **`plan.from_feed`** — did *this* resolution observe the configured provider? + `TariffPlan.origin` already names the four ways a plan is obtained; this + collapses them to the one question a caller holding a previous plan has to + ask. It lives on `TariffPlan` rather than as an `origin == "provider"` at the + call site for `fresh_tariff_plan()`'s own stated reason — *a check has to be + able to assert on the shipped expression rather than on a transcription that + drifts* — and because `origin`'s vocabulary belongs to `static_tariff.py`. +* **`fresh_tariff_plan(now)`** — is a standing observation still current. It + already existed, it is already the one place the age is judged, and it is + **reused, not transcribed**. `test_arbitrage.py` 8b pins that reuse on the + AST; a copy of the comparison behaves identically until the day the two + drift, which is a claim about the code's shape, so a shape check is what + grades it (FCST-06's precedent for the one source check among driven ones). + +### What was NOT done + +* **Not `except Exception` widened, or a second `if` beside it.** That is the + defect's own shape — a place where the rule is spelt twice, so a fifth cause + needs a fifth branch. The `except` stays, relabelled *"not the outage path"*, + keeps the plan on the same terms, and is now honestly the + something-nobody-expected branch. +* **Not the persisted price store as a cache.** Out of scope by the product + owner's call, and it is the right call: FCST-03 exists precisely to stop + stale prices being acted on, and "recent" is not "current". Keeping the last + **fetched** plan until it ages out is a different promise from serving the + store, and that one needs its own ticket. +* **Not a `no_data`-only guard.** `not plan.has_prices` is the tempting + spelling and it is wrong — see decision 2. It leaves the §11.4 fallback free + to replace a fresh observation, and it is a 1-byte edit from the shipped + line, which is why it is in the mutation table rather than in a comment. + +--- + +## 2. A §11.4 static fallback does NOT replace a fresh provider plan + +The ticket asked this directly, because the fallback is *not* nothing: it is +the operator's own configured profile, with real prices, for the requested +window. The answer is still no, and the reason is not that it is worse data — +it is that **nothing may be acted on it, and it is no more current.** + +* Every point `_degrade()` builds carries `provisional=True`. So + `firm_points()` is empty, `arbitrage()` finds fewer than two firm hours and + refuses, and `cheap_now()` is False. Both of the two things `TariffPlan` + exposes for a consumer to act on are already off. +* A day-ahead price for a given hour **does not change once published**. A plan + fetched 30 minutes ago is exactly as true about 19:00 as one fetched now; at + most `TARIFF_MAX_AGE_S` of its horizon has elapsed. The observation is not + stale in any sense the fallback is fresher than. + +So replacing the observation with the fallback trades a plan with firm hours +for a plan with none, and buys no currency with it. The fallback takes over +when the observation **retires** — which is exactly the row §11.4 describes, +arrived at when there is no longer an observation to prefer. + +### And the rule is deliberately NOT symmetrical + +`standing.from_feed` is a term in its own right. A standing plan that is itself +a degradation **is** replaced by the next degradation, and its timestamp moves. +Without that term: + +* an outage on a static-profile site would freeze the operator's profile at the + window it was first generated for, and the guard would be *"never overwrite"* + wearing a bound's name; +* and on a `price_provider: off` install — the shipped default — the plan would + flip between kept and re-derived every `TARIFF_MAX_AGE_S`, for no behavioural + gain, changing which reason the status page prints on a two-hour cycle. + +Each of the three terms is the only false one in some check in 8b, which is +MUTATION.md habit 17's test for whether a suite can tell an `AND` from an `OR`. + +--- + +## 3. `tariff_at` does not move on a kept plan + +The docstring's other half is that a plan must not *"freeze at its last value +and look current forever"*. `tariff_at` is what makes that work: it is the +moment of the fetch that produced the plan standing, and `fresh_tariff_plan()` +judges the age from it. + +Re-stamping it on a kept plan would be this task asserting the feed answered +when it did not. The plan would be re-certified as current at every refresh, an +outage of any length would never retire it, and `TARIFF_MAX_AGE_S` would be +**exactly as dead as it was before this card** — reachable only by the refresh +task itself dying. Leaving it alone is what makes the bound govern: the plan is +served for the age bound after its own fetch, and no longer. + +It is also the whole of **"distinguishable from a freshly-fetched one"**. A +kept plan and a fetched plan are the same object shape; what separates them is +that one's timestamp advanced and the other's did not, and every consumer that +asks how old the plan is already reads that number. The kept case additionally +logs at WARNING, naming the degradation's own reason, the fetch it is keeping, +its age, and the deadline it retires at. + +⚠️ **What this does not give the operator, and why it is not here.** The status +page has no row for "the last refresh failed" — the banner/level not moving +when the plan is lost is separately carded, and the ticket rules it out of +scope. This card's change makes that surface *more* truthful, not less: during +the first `TARIFF_MAX_AGE_S` of an outage there is now a real plan behind the +Arbitrage row instead of a no-plan refusal, so the row says something true for +longer. The no-plan state is entered later by up to 5400 s; it is not removed, +and the 30-minute lag reported on that row is untouched by anything here. + +--- + +## 4. Merged after TARIFF-10 — what the wider gate changed, and what it did not + +TARIFF-10 landed on `release/1.0` while this card was open. It changed **which +installs run this task** — `if arbitrage.enabled(opts)` became +`tariff_refresh_wanted(opts)`, i.e. `provider_configured(opts) or +arbitrage.enabled(opts)` — while this card changed **what the task does on a +failed fetch**. Git auto-merged the two hunks, which is exactly when the +interaction needs reading rather than trusting. + +**Nothing in the decisions above changed.** Two configurations are newly +reachable, and both are now driven in `test_arbitrage.py` 8b rather than +reasoned about: + +* **A provider configured with trading DISARMED** — TARIFF-10's headline case. + This is precisely the configuration the guard was designed for: the plans are + `origin == "provider"`, and a dead feed must not delete one that is still + inside `TARIFF_MAX_AGE_S`. It needs no second fixture, because **nothing on + this path reads `arbitrage_enabled` at all**, and that is asserted rather + than assumed — over the AST's symbols and the non-docstring string constants, + because two earlier drafts of that check were **red against a correct tree**: + the first matched the docstring paragraph that *says* `arbitrage_enabled` is + not read, the second matched the log line's "arbitrage, where it is armed". + A substring test over a function grades its prose as much as its behaviour, + which is habit 4 in a third costume. A fixture that set the option and + asserted no difference would have been the other defect — comparing a thing + against itself. +* **`price_provider: off` + a `static_profile`** — the configuration TARIFF-10's + `or` exists to protect, and the one where a latching guard would do real harm. + `resolve_tariff()` returns `static_plan(provisional=False)`, whose origin is + `"static"`: the CONFIGURED product, not a §11.4 row. So `standing.from_feed` + is false on every refresh, `keep` is false on every refresh, and the plan is + regenerated for the current window every 30 minutes exactly as before this + card. **Driven, twice in a row**, because a guard that latched here would + freeze a static site's plan at its first window for the life of the install — + and mutation M3, which drops the `standing.from_feed` term, is red on that + check as well as on the fallback one. + +Two further facts settle the rest of the interaction: + +* **This decision is strictly downstream of every side effect.** The §8.3 carry + into `price_history` happens inside `resolve_tariff()`, on the `series` + branch, before `keep` is computed — so which of two plans is kept can neither + add nor lose a recorded day, and TARIFF-10's *Price history* row is + unaffected either way. +* **The kept-plan log line was reworded.** It said "arbitrage stops with it", + which after TARIFF-10 is false on a site that fetches without trading; it now + says "arbitrage, **where it is armed**, stops with it". + +All eleven mutations were **re-driven on the merged tree** rather than carried +over — the surrounding code changed, so a red against the old base is not +evidence about the new one. Eleven red, no survivors, every revert +fingerprinted. The added member is M11: making the guard conditional on +`arbitrage.enabled(self.o)` (+86 B), red on five named checks. + +⚠️ **And one process note that belongs in this file.** This card's mutation +harness was written to a shared scratchpad path and was **overwritten mid-run +by another agent's harness for a different card**, pointing at a different +worktree. That is MUTATION.md habit 18's second bullet almost verbatim. It was +caught because the harness fingerprints the tree after every revert and the +runs are what proved this worktree was never left mutated; the harness now +lives at a card-specific path. **A harness that edits source must not share a +name, a directory or a backup path with another one.** + +--- + +## What this does not change + +* **Nothing on the 30 s tick.** `cycle()` still reads two attributes and awaits + no fetch; the AST pin in `test_arbitrage.py` section 8 is unchanged and green. +* **Nothing about the prices themselves.** No series is invented, extended, + re-stamped or carried. The only question this card answers is *which of two + plans the controller holds*. +* **A `price_provider: off` install is bit-for-bit unaffected.** Its standing + plan is never `origin == "provider"`, so `keep` is false on every refresh it + will ever run, and the assignment happens exactly as it did before. diff --git a/goodwe_controller/DOCS.md b/goodwe_controller/DOCS.md index e82f47d..7520c34 100644 --- a/goodwe_controller/DOCS.md +++ b/goodwe_controller/DOCS.md @@ -1453,6 +1453,14 @@ would be traded on as though it were complete. A refused fetch is the §11.4 row — static profile, provisional plan, no arbitrage — and the reason is in the log. +**A refused fetch does not take the plan you already have.** The last plan that +actually came from the feed keeps standing until it is 90 minutes old (three +missed refreshes), and only then does the §11.4 row above take over. The +add-on logs a `WARNING` naming the failure, the fetch it is keeping, its age +and the moment it retires; the plan's own age is **not** reset by the failed +refresh, so a kept plan still expires on schedule and is never presented as a +current one. One failed fetch is a blip, not a lost tariff. + **`manual_csv` — the offline fallback.** It needs no internet, no supplier account and no HTTP stack at all, and it is the supported way to run this add-on price-aware on a site with no outbound connectivity. Write a file and diff --git a/goodwe_controller/app/main.py b/goodwe_controller/app/main.py index 82387b6..9a0c00b 100644 --- a/goodwe_controller/app/main.py +++ b/goodwe_controller/app/main.py @@ -3534,13 +3534,64 @@ class Controller: raise out of here and take the task down, because then the plan freezes at its last value and looks current forever. That is what `fresh_tariff_plan()`'s age bound is the backstop for, and this is the - half that keeps the task alive so the bound rarely matters. + half that keeps the task alive. ⚠️ THE BOUND IS NO LONGER A BACKSTOP + ONLY: since TARIFF-12 it is also the serve-stale window an outage runs + on, and it is consulted below on every refresh. - ⚠️ AND A FAILURE DOES NOT CLEAR THE PLAN. `resolve_tariff()` already has - a §11.4 answer for a dead feed - degrade to the static profile, mark it - provisional, arbitrage refuses - and that answer is better than the - absence of one. Zeroing the plan here would replace a coarse tariff with - no tariff, which is the opposite of what §11.4 asks for. + ⚠️ AND A FAILURE DOES NOT CLEAR THE PLAN - WHICH IS A CLAIM ABOUT WHAT + `resolve_tariff()` RETURNS, NOT ABOUT WHAT IT RAISES (TARIFF-12). This + paragraph was enforced by the `except` below alone, and the `except` + below is the one path a supplier outage never takes. `load_prices()` + says so in its own words - it "RETURNS, NEVER RAISES, for every failure + a site can actually have" - because a degradation arriving as a + traceback into the planning tick would stop the controller regulating + over an expired certificate. So a dead feed is a RETURN: the `try` + succeeds, `_degrade()` hands back a §11.4 plan, and the assignment used + to overwrite the good one unconditionally. Measured in a 40 h soak with + `static_profile` unset (the shipped default, and the normal state of a + dynamic-tariff site): the FIRST failed fetch replaced a complete 48 h + plan with `TariffPlan(no_data, None, "none")` - 0 points - 27 minutes + after the last good one, and `TARIFF_MAX_AGE_S` governed nothing. A + transient blip cost the same as a 90-minute outage. + + ⚠️ SO THE QUESTION IS ASKED ONCE, AFTER THE ATTEMPT, FROM THE EVIDENCE + THAT EXISTS - FCST-06's answer, in this file. The evidence is + `plan.from_feed` (did this attempt observe the feed) and + `fresh_tariff_plan(now)` (is an observation still current). A + degradation replaces a standing observation only once that observation + has aged out, so `TARIFF_MAX_AGE_S` is now the serve-stale window it + reads like rather than a bound nothing could reach. + + ⚠️ AND A §11.4 STATIC FALLBACK DOES NOT REPLACE A FRESH PROVIDER PLAN + EITHER, although it carries real prices. Every point `_degrade()` + builds is `provisional=True`, so `firm_points()` is empty, + `arbitrage()` refuses and `cheap_now()` is False - everything a + consumer may act on is already off. A day-ahead price for a given hour + does not change once published, so a plan fetched half an hour ago is + exactly as true about that hour as one fetched now. The fallback is + less actionable and no more current; it takes over when the observation + retires, which is what §11.4 is for. The reverse is not symmetrical: a + standing plan that is ITSELF a degradation is replaced by a fresh one, + or an outage would freeze the operator's static profile at the window + it was generated for. + + ⚠️ AND NOTHING HERE READS `arbitrage_enabled`, WHICH TARIFF-10 MAKES + WORTH SAYING. Since that card this task also runs on a site with a + provider configured and trading DISARMED, so "keep the plan" is a + decision about what the add-on has LOOKED UP, not about what it may + trade on. It is also strictly downstream of `resolve_tariff()`: the + §8.3 carry into `price_history` happens on the `series` branch inside + that call, before this decision exists, so which of two plans is kept + cannot add or lose a recorded day. + + ⚠️ AND `tariff_at` DOES NOT MOVE ON A KEPT PLAN. It is the moment the + fetch that produced the standing plan happened, and + `fresh_tariff_plan()` judges the age from it - re-stamping it here + would be this task asserting the feed answered when it did not, and the + plan would "freeze at its last value and look current forever", which + is the hazard the paragraph above this one names. An unmoved + `tariff_at` is also the only thing that tells a kept plan from a + freshly fetched one, so it is what the check asserts. """ horizon = float(price_option(self.o, "price_horizon_h")) while not self.stopping: @@ -3549,11 +3600,33 @@ class Controller: plan = await resolve_tariff( self.o, now, now + timedelta(hours=horizon), now=now, history=self.price_history) - self.tariff_plan, self.tariff_at = plan, now - _LOG.info("tariff refreshed: %s", plan.words) + standing = self.fresh_tariff_plan(now) + # Three terms, and each of them is the only false one in some + # check: the feed answered (replace), nothing current stands + # (take the degradation - this is what makes the age bound + # govern), or what stands is itself a degradation (replace). + keep = (not plan.from_feed + and standing is not None and standing.from_feed) + if keep: + _LOG.warning( + "tariff refresh degraded (%s); KEEPING the plan " + "resolved at %s, %.0f s ago. tariff_at is NOT moved, " + "so it retires %.0f s after that fetch; arbitrage, " + "where it is armed, stops with it. A kept plan is " + "never presented as a current one.", plan.words, + self.tariff_at, + (now - self.tariff_at).total_seconds(), + TARIFF_MAX_AGE_S) + else: + self.tariff_plan, self.tariff_at = plan, now + _LOG.info("tariff refreshed: %s", plan.words) except Exception as err: # noqa: BLE001 + # ⚠️ NOT THE OUTAGE PATH - see the docstring. `load_prices()` + # returns on every failure a site can have, so what reaches + # here is something nobody expected, and the plan is kept for + # the same reason and on the same terms. _LOG.warning( - "tariff refresh failed (%s); keeping the plan resolved at " + "tariff refresh raised (%s); keeping the plan resolved at " "%s. It retires on its own after %.0f s and arbitrage " "stops with it.", err, self.tariff_at, TARIFF_MAX_AGE_S) with contextlib.suppress(asyncio.TimeoutError): diff --git a/goodwe_controller/app/static_tariff.py b/goodwe_controller/app/static_tariff.py index e4abec7..8d735e3 100644 --- a/goodwe_controller/app/static_tariff.py +++ b/goodwe_controller/app/static_tariff.py @@ -728,6 +728,33 @@ class TariffPlan: self.origin = origin # -- derived facts ------------------------------------------------------- + @property + def from_feed(self) -> bool: + """Is this series an OBSERVATION of the configured provider's feed? + + TARIFF-12. `origin` names four ways a plan is obtained; this collapses + them to the one question a caller holding a PREVIOUS plan has to ask + about a new one - did this resolution see the feed, or is it standing + in for one that did not? Only `"provider"` saw it. `"fallback"` and + `"none"` are the two §11.4 rows, and `"static"` is a site with no + feed to observe at all, where the question does not arise. + + ⚠️ IT IS NOT `has_prices`, AND THE DIFFERENCE IS THE WHOLE POINT. A + §11.4 fallback HAS prices - the operator's own static profile - but + every point `_degrade()` builds carries `provisional=True`, so + `firm_points()` is empty, `arbitrage()` refuses and `cheap_now()` is + False. It is a real tariff and it is not an observation, and a caller + deciding whether to REPLACE an observation with it needs the second + fact, not the first. + + ⚠️ AND IT IS A PROPERTY HERE RATHER THAN AN `origin == "provider"` AT + THE CALL SITE, for `main.fresh_tariff_plan()`'s stated reason: a check + has to be able to assert on the shipped expression rather than on a + transcription that drifts, and `origin`'s vocabulary belongs to this + module. + """ + return self.origin == "provider" + @property def has_prices(self) -> bool: return self.read.match(series=lambda _r: True, no_data=lambda _r: False) diff --git a/goodwe_controller/test_arbitrage.py b/goodwe_controller/test_arbitrage.py index 5509acf..4b25642 100644 --- a/goodwe_controller/test_arbitrage.py +++ b/goodwe_controller/test_arbitrage.py @@ -92,7 +92,9 @@ from app.price import ( PRICE_OPTION_DEFAULTS, Absence, PricePoint, PriceProduct, PriceRead, Uncertainty, WIDEN_FACTOR, brussels, required_spread, widen_factor, ) -from app.static_tariff import TariffPlan, parse_static_profile, static_plan +from app.static_tariff import ( + TariffPlan, parse_static_profile, resolve_tariff, static_plan, +) from app.validate import validate_startup_config UTC = timezone.utc @@ -1284,6 +1286,323 @@ check("main.py reads every arbitrage option through app/arbitrage.py or " "arbitrage_soc_ceiling"))) +# --------------------------------------------------------------------------- # +print() +print("8b. TARIFF-12 - a RETURNED degradation must not clear a fresh plan") + +# ⚠️ THE STIMULUS IS A RETURN, NOT A RAISE, AND THAT IS THE WHOLE CHECK. +# `run_tariff_refresh()` guarded the plan with `except Exception` alone, and +# `load_prices()` states in its own docstring that it "RETURNS, NEVER RAISES, +# for every failure a site can actually have" - so the one case the guard was +# written for, a supplier outage, never reached it. Everything below drives the +# SHIPPED chain and stubs nothing but the socket: the transport raises the way +# aiohttp does, `load_prices()` turns that into a `no_data` read, `_degrade()` +# turns THAT into a §11.4 plan, and `run_tariff_refresh()` decides what to do +# with it. A check that patched `resolve_tariff` to raise would have passed +# against the defective code. +import asyncio # noqa: E402 +from app import price as _price # noqa: E402 +from app.price_history import NoHistory # noqa: E402 + + +class ClientConnectorError(Exception): + """aiohttp's own class name, so the §11.4 reason this drives is the + sentence the 40 h soak recorded at refresh 56, verbatim.""" + + +_T12_FEED_UP = {"up": True} + + +async def _t12_transport(url, timeout_s): + """The socket, and nothing else, is what this replaces.""" + if not _T12_FEED_UP["up"]: + raise ClientConnectorError( + "Cannot connect to host api.energy-charts.info:443 ssl:default") + t0 = datetime.now(UTC).replace(minute=0, second=0, microsecond=0) + return json.dumps({ + "unix_seconds": [int((t0 + timedelta(hours=i)).timestamp()) + for i in range(48)], + "price": [80.0 + 40.0 * (i % 6) for i in range(48)], + "unit": "EUR / MWh"}).encode() + + +class _T12Controller: + """Just enough Controller to run the SHIPPED refresh coroutine. + + Both methods are bound off the shipped class rather than copied, for + `fresh_tariff_plan()`'s own stated reason: a transcription drifts. + """ + run_tariff_refresh = _main.Controller.run_tariff_refresh + fresh_tariff_plan = _main.Controller.fresh_tariff_plan + + def __init__(self, opts): + self.o = opts + self.tariff_plan = None + self.tariff_at = None + self.price_history = NoHistory() + self.stop_event = asyncio.Event() + self.stop_event.set() # so the inter-refresh wait returns + self._ticks = 0 + + @property + def stopping(self): + self._ticks -= 1 + return self._ticks < 0 + + def refresh_once(self): + self._ticks = 1 + asyncio.run(self.run_tariff_refresh()) + return self + + +def _t12_opts(profile=""): + # `price_timeout_s` is read with a plain `.get()` by every plugin, so it + # has to be present - an options dict that relies on price_option()'s + # defaults for it fetches with `float(None)` and this section would grade + # a TypeError instead of an outage. + return {"price_provider": "generic_dayahead", "price_timeout_s": 10, + "price_horizon_h": 48, "static_profile": profile} + + +_T12_HORIZON_H = 48 +_T12_STALE_S = _main.TARIFF_MAX_AGE_S +_t12_saved_transport = _price.https_get +_price.https_get = _t12_transport + +# -- the accepted baseline (habit 7): one good fetch, varied one thing at a +# -- time from here. +_T12_FEED_UP["up"] = True +_t12_a = derived(lambda: _T12Controller(_t12_opts()).refresh_once()) +_t12_good = derived(lambda: _t12_a.tariff_plan) +check(f"BASELINE: a fetch that succeeds stores an OBSERVATION - " + f"{getattr(_t12_good, 'words', _t12_good)!r}", + getattr(_t12_good, "origin", None) == "provider" + and _t12_good.from_feed is True + and len(_t12_good.read.points) == _T12_HORIZON_H) + +# -- the stimulus, graded before it is used on anything (habit 6/7) ---------- +_T12_FEED_UP["up"] = False +_t12_now = datetime.now(UTC) +_t12_deg = derived(lambda: asyncio.run(resolve_tariff( + _t12_opts(), _t12_now, _t12_now + timedelta(hours=_T12_HORIZON_H), + now=_t12_now, history=NoHistory()))) +check("STIMULUS: a dead feed makes resolve_tariff() RETURN a degraded plan - " + "it does not raise, so `except Exception` is not what was protecting " + "the plan and the defect was never on that branch", + isinstance(_t12_deg, TariffPlan)) +check(f"...and the returned plan is the §11.4 no-profile row the soak saw - " + f"{getattr(_t12_deg, 'words', _t12_deg)!r}", + isinstance(_t12_deg, TariffPlan) and _t12_deg.origin == "none" + and _t12_deg.from_feed is False and _t12_deg.has_prices is False + and "ClientConnectorError" in _t12_deg.words) + +# -- AC1: the degradation must not replace a plan still inside the bound ----- +# ⚠️ 1800 s IS STATED HERE, NOT DERIVED FROM THE MODULE BEING GRADED - section +# 8's own lesson. A probe at `TARIFF_MAX_AGE_S - 1` moves with a mutation of +# the constant; half an hour is inside any reading of a 5400 s bound, and it is +# the interval the soak actually ran at. +_t12_b = derived(lambda: _T12Controller(_t12_opts())) +_t12_b.tariff_plan, _t12_b.tariff_at = _t12_good, datetime.now(UTC) - \ + timedelta(seconds=1800) +_t12_at_before = _t12_b.tariff_at +check(f"PRECONDITION: the standing plan is 1800 s old and TARIFF_MAX_AGE_S is " + f"{_T12_STALE_S:.0f} s, so it IS fresh when the failed refresh runs - " + f"without this the next check grades an expired plan", + 1800 < _T12_STALE_S + and _t12_b.fresh_tariff_plan(datetime.now(UTC)) is _t12_good) +_t12_b.refresh_once() +check("a RETURNED degradation does not replace a plan still inside " + "TARIFF_MAX_AGE_S - the 48 h series is still there, by identity, and " + "not a 0-point `origin: none` read", + _t12_b.tariff_plan is _t12_good + and _t12_b.tariff_plan.has_prices is True + and len(_t12_b.tariff_plan.read.points) == _T12_HORIZON_H) +check("...and `tariff_at` did NOT move, so the kept plan still ages out from " + "the fetch that produced it and is distinguishable from a freshly " + "fetched one. Re-stamping it here is how a plan freezes at its last " + "value and looks current forever", + _t12_b.tariff_at == _t12_at_before) +check("...and the two timestamps a successful refresh and a kept plan produce " + "really are different values, or the check above compares a number " + "against itself", + _t12_a.tariff_at != _t12_at_before) + +# -- AC3: TARIFF_MAX_AGE_S governs - the bound is where the module says ------ +_t12_c = derived(lambda: _T12Controller(_t12_opts())) +_t12_c.tariff_plan, _t12_c.tariff_at = _t12_good, datetime.now(UTC) - \ + timedelta(seconds=_T12_STALE_S + 5) +_t12_c.refresh_once() +check(f"a plan that has aged PAST {_T12_STALE_S:.0f} s IS replaced by the " + f"degradation - the plan is served stale for the age bound and no " + f"longer, so TARIFF_MAX_AGE_S governs something rather than being " + f"unreachable", + _t12_c.tariff_plan is not _t12_good + and _t12_c.tariff_plan.from_feed is False + and _t12_c.tariff_plan.has_prices is False) +_t12_d = derived(lambda: _T12Controller(_t12_opts())) +_t12_d.tariff_plan, _t12_d.tariff_at = _t12_good, datetime.now(UTC) - \ + timedelta(seconds=_T12_STALE_S - 60) +_t12_d.refresh_once() +check("...and one minute SHORT of the bound it is still kept, so the line " + "above is a bound rather than 'the degradation always wins'", + _t12_d.tariff_plan is _t12_good) +_t12_e = derived(lambda: _T12Controller(_t12_opts()).refresh_once()) +check("a degradation with NO plan standing at all is stored - §11.4 still " + "reaches an install that has never had a good fetch, and the guard is " + "not 'never overwrite'", + _t12_e.tariff_plan is not None + and _t12_e.tariff_plan.from_feed is False + and _t12_e.tariff_at is not None) + +# -- the §11.4 STATIC FALLBACK, which is the judgement this card had to make - +_t12_f = derived(lambda: _T12Controller(_t12_opts(STATIC_BE))) +_t12_fb = derived(lambda: asyncio.run(resolve_tariff( + _t12_opts(STATIC_BE), _t12_now, + _t12_now + timedelta(hours=_T12_HORIZON_H), now=_t12_now, + history=NoHistory()))) +check(f"PRECONDITION: with a static_profile configured the SAME dead feed " + f"degrades to a §11.4 FALLBACK carrying real prices, not to the " + f"no-profile row - {getattr(_t12_fb, 'words', _t12_fb)!r}", + isinstance(_t12_fb, TariffPlan) and _t12_fb.origin == "fallback" + and _t12_fb.has_prices is True and _t12_fb.from_feed is False) +check("...and every one of its points is provisional, so firm_points() is " + "empty, arbitrage() refuses and cheap_now() is False - it is a real " + "tariff and nothing may be acted on it. That is why it does not replace " + "a fresh observation", + isinstance(_t12_fb, TariffPlan) and _t12_fb.provisional is True + and _t12_fb.firm_points() == () + and _t12_fb.arbitrage(peak_guard_active=False, min_spread_eur_kwh=0.0, + uncertainty=NO_WEATHER).kind == "refused") +_t12_f.tariff_plan, _t12_f.tariff_at = _t12_good, datetime.now(UTC) - \ + timedelta(seconds=1800) +_t12_f_at = _t12_f.tariff_at +_t12_f.refresh_once() +check("a §11.4 static FALLBACK does not replace a fresh provider plan either, " + "although it has prices - a day-ahead price for an hour does not change " + "once published, so the 30-minute-old observation is no less true and " + "is the only one of the two that carries a firm hour", + _t12_f.tariff_plan is _t12_good and _t12_f.tariff_at == _t12_f_at) + +# -- ...and the rule is NOT symmetrical: a degradation replaces a degradation. +_t12_g = derived(lambda: _T12Controller(_t12_opts(STATIC_BE))) +_t12_g.tariff_plan, _t12_g.tariff_at = _t12_fb, datetime.now(UTC) - \ + timedelta(seconds=1800) +_t12_g_at = _t12_g.tariff_at +_t12_g.refresh_once() +check("a standing plan that is ITSELF a §11.4 fallback IS replaced by the " + "next one, and its timestamp moves - otherwise an outage would freeze " + "the operator's static profile at the window it was first generated " + "for, and the guard would be 'never overwrite' wearing a bound's name", + _t12_g.tariff_plan is not _t12_fb and _t12_g.tariff_plan.origin == + "fallback" and _t12_g.tariff_at != _t12_g_at) + +# -- ...and a recovered feed replaces a kept plan, which is the third term --- +_T12_FEED_UP["up"] = True +_t12_h = derived(lambda: _T12Controller(_t12_opts())) +_t12_h.tariff_plan, _t12_h.tariff_at = _t12_good, datetime.now(UTC) - \ + timedelta(seconds=1800) +_t12_h_at = _t12_h.tariff_at +_t12_h.refresh_once() +check("when the feed answers again the observation replaces the kept plan and " + "`tariff_at` moves - the guard keeps a plan against a DEGRADATION, not " + "against a fetch that worked", + _t12_h.tariff_plan is not _t12_good + and _t12_h.tariff_plan.from_feed is True + and _t12_h.tariff_at != _t12_h_at) + +# -- TARIFF-10 widened WHICH INSTALLS RUN THIS TASK, so the configurations it +# -- newly keeps alive are graded here rather than reasoned about. +# ⚠️ `price_provider: off` + a `static_profile` + `arbitrage_enabled: true` is a +# configuration validate.py ACCEPTS and TARIFF-10's `or` exists to protect. On +# it `resolve_tariff()` returns `static_plan(provisional=False)` - the +# CONFIGURED product, origin "static" - so `standing.from_feed` is false on +# every refresh and this card's guard must never fire. Confirmed by driving it, +# not by reading the origin table: a guard that quietly latched here would +# freeze a static site's plan at its first window for the life of the install. +_t12_i = derived(lambda: _T12Controller( + {"price_provider": "off", "price_timeout_s": 10, "price_horizon_h": 48, + "static_profile": STATIC_BE}).refresh_once()) +_t12_i_plan, _t12_i_at = _t12_i.tariff_plan, _t12_i.tariff_at +_t12_i.refresh_once() +check("PRECONDITION: a `price_provider: off` + static_profile site resolves to " + "the CONFIGURED product, not to a §11.4 row - origin 'static', prices, " + "and from_feed false because there is no feed to have observed", + getattr(_t12_i_plan, "origin", None) == "static" + and _t12_i_plan.has_prices is True and _t12_i_plan.from_feed is False) +check("...and TARIFF-10 now keeps this task alive on exactly that site, where " + "this card's guard must NEVER latch: the second refresh replaces the " + "first and the timestamp moves, so the plan is regenerated for the " + "current window every 30 minutes as it always was", + _t12_i.tariff_plan is not _t12_i_plan + and _t12_i.tariff_plan.origin == "static" + and _t12_i.tariff_at != _t12_i_at) + +# ⚠️ AND THE OTHER CONFIGURATION TARIFF-10 ADDED - a provider configured with +# trading DISARMED - is graded by every driven check above without a second +# fixture, because nothing on this path reads `arbitrage_enabled` at all. That +# is a claim about the code rather than about a behaviour, so it is asserted as +# one (habit 4's exception, FCST-06's precedent): a fixture that set the option +# and asserted no difference would be comparing a thing against itself. +_T12_TRADING_OPTS = ("arbitrage_enabled", "cycle_cost_threshold_eur", + "arbitrage_soc_floor", "arbitrage_soc_ceiling") +# ⚠️ AND IT IS WHAT THE FUNCTION READS THAT IS SCANNED, NOT ITS TEXT. Two +# earlier drafts of this check were red against a CORRECT tree: the first +# because the docstring paragraph that SAYS `arbitrage_enabled` is not read is +# itself an occurrence of the string, the second because the log line says +# "arbitrage, where it is armed, stops with it" - which is a sentence, not a +# read. A substring test over a function grades its prose as much as its +# behaviour (habit 4). So the symbols are taken from the AST, and the string +# constants are taken separately and with the docstring dropped: naming +# arbitrage in a message is fine, READING a trading option is the thing claimed +# not to happen. +_t12_syms = derived(lambda: {getattr(n, "id", None) or getattr(n, "attr", None) + for n in ast.walk(_refresh) + if isinstance(n, (ast.Name, ast.Attribute))}) +_t12_consts = derived(lambda: {c.value for b in _refresh.body[1:] + for c in ast.walk(b) + if isinstance(c, ast.Constant) + and isinstance(c.value, str)}) +check("run_tariff_refresh() READS no trading option and calls nothing in " + "app/arbitrage.py, so keeping a plan is a decision about what the add-on " + "has LOOKED UP - TARIFF-10's own distinction, and why the checks above " + "cover a provider-configured site whether or not arbitrage is armed", + isinstance(_t12_syms, set) + and not any("arbitrage" in (n or "") for n in _t12_syms) + and not (_t12_consts & set(_T12_TRADING_OPTS)) + # ...and both scanners can see what IS there, or they grade nothing: the + # one option this task does read, and a name it really does call. + and "price_horizon_h" in _t12_consts + and "fresh_tariff_plan" in _t12_syms + # ...and the docstring really was dropped, or the first draft's defect is + # back and the constant set is grading prose again. + and not any("RETURNS, NEVER RAISES" in c for c in _t12_consts)) + +# -- the vocabulary, graded on its own so a mutation of it has a named home -- +check("TariffPlan.from_feed is true of exactly the observation - a §11.4 " + "fallback, a no-data read and a CONFIGURED static profile are all false, " + "and it is not a synonym for has_prices", + _t12_good.from_feed is True and _t12_fb.from_feed is False + and _t12_deg.from_feed is False + and static_plan({"static_profile": STATIC_BE}, D0, + D0 + timedelta(days=2), now=D0).from_feed is False + and static_plan({"static_profile": STATIC_BE}, D0, + D0 + timedelta(days=2), now=D0).has_prices is True) + +# ⚠️ A SHAPE CLAIM, AND IT IS ALLOWED TO BE ONE (FCST-06's precedent). "The age +# is judged in ONE place" is a claim about the code, not about a behaviour: a +# transcribed copy of the same comparison behaves identically until the day the +# two drift. Every other check in this section is driven. +check("run_tariff_refresh() asks fresh_tariff_plan() rather than re-deriving " + "the age bound - one place judges it, which is what that method's own " + "docstring is for", + _refresh is not None + and {n.func.attr for n in ast.walk(_refresh) + if isinstance(n, ast.Call) and isinstance(n.func, ast.Attribute)} + >= {"fresh_tariff_plan"}) + +_price.https_get = _t12_saved_transport + + # --------------------------------------------------------------------------- # print() print("9. the startup gate - the configurations that cannot work are refused")