diff --git a/.gitea/workflows/checks.yml b/.gitea/workflows/checks.yml index 425738e..44709c0 100644 --- a/.gitea/workflows/checks.yml +++ b/.gitea/workflows/checks.yml @@ -110,6 +110,37 @@ name: checks # trigger` step below then keeps it that way mechanically rather than by asking # the next editor to read this comment. # +# ⚠️ READ THE BLIND SPOTS ON THAT STEP BEFORE TRUSTING THE SENTENCE ABOVE. +# "Mechanically" holds only for a reintroduced filter that still names the +# PR's own base. For blind spots 1 and 3 the defence IS this comment, because +# the guard lives inside the workflow that a filter makes Gitea skip. Blind +# spot 3 is the one that matters: a filter naming a branch THIS pull request +# does not target is itself skipped, lands green and untested, and then every +# subsequent pull request the filter DOES NOT NAME is skipped too. That is the +# gate off, silently. +# +# ⚠️ THAT DIRECTION IS THE WHOLE POINT AND IT IS EASY TO WRITE BACKWARDS. A +# pull request that MATCHES the filter still runs; it is the one the filter +# excludes that Gitea sits out, reports `skipped`, and the rollup launders +# into `success`. An earlier draft of this very clause said "matching it is +# skipped", which is the inverse of the measurement and contradicted the +# paragraph directly below. +# +# ⚠️ AND THE DANGEROUS ONE IS NOT `branches: [main]`, WHICH AT LEAST LOOKS +# WRONG. It is `branches: [release/1.0]`, which looks like tightening onto the +# protected branch and is what a careful person would write. Every day-to-day +# pull request still runs and still goes green, so nothing appears broken - +# and the only thing switched off is the `release/1.0` -> `master` release +# PR, i.e. the one whose content reaches an installed system. SEC-2 made that +# PR the mandated release path (docs/RELEASE.md), so this is now a live route +# rather than a hypothetical one. +# +# Closing it needs a SECOND permanently unfiltered workflow posting its own +# context, with that context added to the `status_check_contexts` of BOTH +# `release/1.0` AND `master` - both carry the rule since SEC-2, and adding it +# to one alone leaves the hole open on the other. A forge change, with the +# owner. +# # THE MEASUREMENT, because the obvious guess is wrong and was nearly written # here as fact. A skip-ci keyword in a commit message (`[skip ci]`, `[ci skip]`, # `[no ci]`, `[skip actions]`, `[actions skip]`) does NOT behave like a filter diff --git a/README.md b/README.md index fdd9eb5..cd500c7 100644 --- a/README.md +++ b/README.md @@ -33,6 +33,8 @@ estop/rs485_log.py optional RS485 e-stop / bus witness docs/GoodWe-RS485-Field-Guide.pdf the guide techs carry (10pp, A4) docs/build_guide.py its single source - rebuilds the HTML and PDF FIELD-GUIDE.md how to edit and rebuild the guide +docs/RELEASE.md how a change reaches the battery - `master` is + protected, releases go via PR from `release/1.0` ``` **Picking this up after a break? Read `HANDOVER.md` first** — current state, diff --git a/docs/RELEASE.md b/docs/RELEASE.md new file mode 100644 index 0000000..ff5d899 --- /dev/null +++ b/docs/RELEASE.md @@ -0,0 +1,116 @@ +# Releasing — how a change reaches the battery + +`master` is the **default branch**, and the default branch is what the Home +Assistant Supervisor clones when it installs or updates this add-on. Whatever +sits on `master` is what runs on a real 5 kW inverter. Treat a push to `master` +as a deployment, because that is what it is. + +--- + +## 1. `master` is protected — releases go through a pull request + +Until 2026-09-23 `master` had **no protection of any kind**. Every gate this +repo built (CI-01, CI-9) sat on `release/1.0`, the branch that does *not* ship. +On 2026-09-22 version 0.10.1 went straight onto `master` — no pull request, no +CI run, no reviewer — and it succeeded (commit `0f14c2a`, the only commit on +`master` that carries no `(#NN)` merge suffix). SEC-2 closed that. + +**The release path is now:** + +1. Merge the work into `release/1.0` as usual, via a pull request that goes + green. Bump `version:` in `goodwe_controller/config.yaml` on that branch — + the Supervisor keys the built image by `version:`, so without a bump nothing + is offered and nothing changes. +2. Open a pull request **`release/1.0` → `master`**. Do not push to `master`; + it will be refused by the pre-receive hook: + + ``` + remote: error: Not allowed to push to protected branch master + ! [remote rejected] ... (pre-receive hook declined) + ``` + +3. Wait for `checks / unit (pull_request)` to go **green** on that pull request. + The workflow in `.gitea/workflows/checks.yml` carries **no `branches:` + filter**, so a `master` base matches and a real run happens — verified on + throwaway PR #107 (base `master`), which queued a genuine run. + + ⚠️ Green means green. A `skipped` status is laundered into a `success` + rollup by this Gitea and the merge gate accepts it — see the long note at + the top of `.gitea/workflows/checks.yml`. Never add an event filter there. + +4. Merge the pull request. `required_approvals` is **0**, so a green check is + the whole gate; there is no second pair of eyes unless someone asks for one. + +5. **Reload the store.** Merging to `master` does not, by itself, offer the + update. The Supervisor only sees the new version after: + + ```bash + ssh root@192.168.2.158 "ha store reload && ha addons update local_goodwe_controller" + ``` + + Until `ha store reload` runs, the add-on page still shows the old version + and the release looks like it did nothing. + +Note that `release/1.0` is normally an *ancestor* of `master` between releases, +so a `release/1.0 → master` pull request is empty until new commits land on +`release/1.0`. That is expected; it means there is nothing to release yet. + +--- + +## 2. Branch protection — before and after SEC-2 + +Forge is Gitea at `192.168.2.147:3000`, repo `admin/goodwe-addon`. Read it back +with `GET /api/v1/repos/admin/goodwe-addon/branch_protections`. + +**Before (2026-09-23):** + +| branch | protected | status check required | context | approvals | +|---|---|---|---|---| +| `master` (default) | **no rule at all** | — | — | — | +| `release/1.0` | yes | yes | `checks / unit (pull_request)` | 0 | + +**After:** + +| branch | protected | status check required | context | approvals | +|---|---|---|---|---| +| `master` (default) | yes | yes | `checks / unit (pull_request)` | 0 | +| `release/1.0` | yes (unchanged) | yes | `checks / unit (pull_request)` | 0 | + +Both rules are otherwise identical: `enable_push: false`, +`enable_force_push: false`, `block_admin_merge_override: true`, no whitelists, +no file patterns. `required_approvals` stays at **0** on purpose — raising it +would block the review agent from merging, and nobody asked for it. + +The default branch was **not** changed. Changing it would change what the +Supervisor clones, which is a different decision from this one. + +### How the gate was proven to actually bite + +A protection rule on this forge can *appear* satisfied while having checked +nothing, so the rule was tested rather than trusted. The discriminator is a +merge `POST` carrying a deliberately wrong `head_commit_id`: Gitea evaluates +branch protection *before* it checks the head, so + +* `409 head out of date` → the request **cleared** protection; +* `405 Not all required status checks successful` → protection **refused** it. + +Read the *message*, not just the code: `405` is also returned as +`Please try again later` for a pull request with **no commits**, which has +nothing to do with protection. An empty pull request on an unprotected base +gives that same `405` (verified, throwaway PR #110), so an empty +`release/1.0 → master` reports it too. + +Validated against a control on an unprotected base (PR #106), which answered +`409` while its check was still pending — i.e. protection was no obstacle +there. On `master` (PR #107) the same probe answered `405` while pending, and +`409` once the check was genuinely green. A direct `git push` to `master` was +rejected by the pre-receive hook with `master`'s SHA unmoved. + +--- + +## 3. Rolling back + +There is no force-push escape hatch: `enable_force_push` is `false` on both +branches. To undo a bad release, land a revert on `release/1.0`, bump the +version again, and take it through §1. The `/addons/` rollback for the +reference install is in `HANDOVER.md`. diff --git a/goodwe_controller/CHANGELOG.md b/goodwe_controller/CHANGELOG.md index ab9d298..02a046e 100644 --- a/goodwe_controller/CHANGELOG.md +++ b/goodwe_controller/CHANGELOG.md @@ -1,5 +1,307 @@ # Changelog +## 0.11.0 + +**A MINOR BUMP, AND ONE NEW OPTION IS THE WHOLE REASON FOR THE SECOND DIGIT: +`pv_entity` ([[TEL-11]]).** The option surface moves - **75 keys -> 76**, +`options:` and `schema:` in step - which is the same test 0.8.0, 0.9.0 and +0.10.0 were each cut on. ⚠️ **AND UNLIKE 0.10.0, NOTHING HERE CAN REFUSE A +STORED OPTIONS BLOB.** No key was removed, no default moved, no enum widened, +and **no rule was added to `validate.py`** - the validator does not mention +`pv_entity` at all, because nothing else reads it and it cannot conflict with +anything. The new key is `str?` - **note the trailing `?`, it is OPTIONAL** - +with an `options:` default of `""`, so a blob written on 0.10.0 or 0.10.1 does +not carry it, the Supervisor's merge supplies the empty string, and the empty +string is *off*. ⚠️ **A stored options blob that starts the add-on on 0.10.1 +starts it on 0.11.0. There is no `SystemExit(78)` to plan around on this hop**, +and a site that presses *Update* and changes nothing else gets an add-on that +behaves exactly as its 0.10.1 did. + +⚠️ **WHAT YOU ARE ACTUALLY DECIDING, IN ONE LINE EACH. TWO TICKETS HERE ARE +OBSERVABLE ON AN INSTALLED ADD-ON, AND NEITHER CHANGES A WATT.** + +* [[SAFETY-56]] - **three messages stop naming your hardware as the fault** in a + state where the add-on cannot tell a dead register from a battery that is + simply empty at its BMS floor. **Text only; the behaviour is measured + byte-identical.** If you have ever opened Developer Tools because the add-on + told you a sensor had failed, and found the sensor working, this is that + defect. On the reference install the state it misdiagnoses arrives on most + nights and on every cloudy morning after a deep discharge. +* [[TEL-11]] - **one new optional key, `pv_entity`, and nothing whatever happens + until you fill it in.** Set it and the add-on records what your solar is + producing beside what the meter and the battery are doing. ⚠️ **It is not a + control input and it does not change what the loop does.** + +⚠️ **THERE IS NO CONTROL-LAW CHANGE IN THIS RELEASE, WHICH IS WORTH SAYING +BECAUSE 0.10.0 HAD ONE AND NOTHING IN ITS PREAMBLE COULD WARN YOU.** +`app/control.py` is not in this release's diff at all, and neither are +`allocator.py`, `arbiter.py`, `arbitrage.py`, `clamp.py`, `peakguard.py`, +`maintenance.py` or `watchdog.py`. The runtime files that move are +`app/freshness.py` and `app/main.py` (SAFETY-56's three sentences), and +`app/pv.py` - new - with `app/main.py`, `app/mqtt.py`, `config.yaml` and +`translations/` (TEL-11). Contrast [[SAFETY-48]] in `## 0.10.0`, which altered +what the inverter is asked to do while touching no option, no default and no +validator rule, so that nothing a config check could see would mention it. This +release has nothing of that shape in it. + +⚠️ **IF YOU ARE CROSSING FROM A VERSION OLDER THAN 0.10.0, THIS SECTION IS +STILL NOT THE ONE THAT CAN REFUSE YOU - `## 0.10.0` BELOW IS.** `ha addons +update` does not walk the versions; it installs 0.11.0 in one step, so every +validator rule added in 0.8.0, 0.9.0 and 0.10.0 arrives at the same moment as +this section's nothing. Two of them can refuse a config that nobody has edited: + +> [[SAFETY-39]]'s **rule 10**, added in 0.8.0, against 0.7.0's own shipped +> default `slew_w: 1000` - the ceiling is 2/18 of the site rail, which is +> **555.6 W** on a 5000 W rail. +> +> [[TARIFF-07]]'s three withdrawn `price_provider` ids - **`engie_dynamic`, +> `totalenergies_dynamic` and `eneco_eddy`** - each of which is **still refused +> by name at startup on 0.11.0**, with the same message and for the same +> measured reasons: a 404 from a gateway with no route, a TLS handshake against +> a hostname the certificate does not cover, and a 404 with an empty body. + +⚠️ **NEITHER OF THOSE IS NEW IN 0.11.0, AND NEITHER IS NEWS TO A SITE ALREADY ON +0.10.0 OR 0.10.1 - IT HAS ALREADY MET THEM.** They are restated here only +because this is now the newest section, and a reader who opens the newest section +and stops would otherwise never meet the one class of thing that can stop the +add-on starting. **The full account of both is in `## 0.10.0` below**: what to +set instead of each withdrawn id, which tariff a site left on one of them has +really been running, why the refusal *is* the migration, and why all three ids +stay in `config.yaml`'s `price_provider` enum even though `PROVIDERS` is down to +three. **That is the section that withdrew them, it is where the evidence is, +and it is not superseded by this one.** ⚠️ The reference install is on +`price_provider: off` and at `slew_w: 400`, so neither applies to it. +`tools/predeploy_check.py` checks both against your live options blob before you +press *Update*, and on this tree it reports a real version change rather than +0.10.1's no-op. + +Contents of this release, in merge order since 0.10.1 (`b0b24e5`): + +| | | +|---|---| +| `c9aaf7f` | SAFETY-56 - the gate cannot tell a dead register from a pack that cannot comply, so stop saying it can **⚠️ TEXT ONLY, BEHAVIOUR BYTE-IDENTICAL** | +| `94da37f` | TEL-11 - PV production stops being invisible, and stays out of the control law **⚠️ NEW OPTIONAL OPTION, UNSET ON EVERY EXISTING INSTALL** | + +⚠️ **TWO MORE MERGES LANDED IN THIS WINDOW AND ARE DELIBERATELY NOT IN THE TABLE +ABOVE: [[CI-9]]'S FOLLOW-UP (`9f6b9f1`) AND [[SEC-2]] (`239e120`). THE RULE IS +[[PKG-15]]'S - AUDIENCE, NOT IMPORTANCE - AND BOTH FAIL IT THE SAME WAY.** This +file is the **add-on's** release note: the text the Supervisor renders beside the +version number, read by somebody deciding whether to press *Update* on a battery +that is currently regulating. A row here has to be something that reader can act +on. + +* **CI-9's follow-up changes one file, `.gitea/workflows/checks.yml`**, exactly + as CI-9 itself did, and it is comment-only within that file. It is not under + `goodwe_controller/`, it is not in `predeploy_check.py`'s `RUNTIME` tuple, it + is not in the built image, and **there is no version of this add-on on which an + operator can observe it.** `## 0.10.0` excluded CI-9 on exactly that reasoning; + excluding the follow-up on any other reasoning would make the rule mean + nothing. +* **SEC-2 adds `docs/RELEASE.md` and two lines to `README.md`, and this one is + the closer call, so here is the whole of it.** SEC-2 is by some distance the + more consequential of the two: it is the ticket that made `master` refuse a + direct push, and it is *why this release reaches you as a pull request into + `release/1.0` and then into `master`, rather than as a push straight to the + branch the Supervisor clones* - which is how 0.10.1 got there. It is also + **purely a procedure for whoever cuts a release**; both files sit outside + `goodwe_controller/`, neither is in the image, and no install can tell whether + that document exists or what it says. An operator reading it learns nothing + about 0.11.0 that helps them decide whether to take it. So it is excluded, and + it is excluded under the same sentence as CI-9: ⚠️ **importance to this + repository is not the test. Observability on an installed add-on is.** + +⚠️ **AND BOTH ARE WRITTEN DOWN, JUST NOT HERE.** CI-9's follow-up is PR #102 plus +the comment block at the top of `.gitea/workflows/checks.yml`, which carries the +measurements in full; SEC-2 is PR #108 plus `docs/RELEASE.md` itself, which is +now the canonical account of the release path and is the procedure this release +followed. ⚠️ **If a later CI or packaging ticket changes what the shipped image +contains, it belongs in this table.** Neither of these does. + +### SAFETY-56 — three messages stop blaming healthy hardware + +⚠️ **NOTHING THE ADD-ON DOES CHANGES. THREE THINGS IT SAYS DO.** Same +conjunction, same limits, same `MISSING`, same 0 W command, same NOT READY row, +same [[SAFETY-57]] release at 32 s. `tools/safety56_behaviour_trace.py` is what +says so rather than a commit message: it hashes a document with no sentence +anywhere in it, built from **2390 one-second reads replayed out of the committed +2026-09-21 bus capture**, a **66-cell command × register grid stepped 120 times +each**, and **six driven controller worlds**. That document is +`6030175ed2ab9a41…` on `release/1.0` and on this tree, byte for byte, while the +sentences differ. ⚠️ **And the tool REFUSES to report "behaviour unchanged" when +the sentence halves agree**, so an identical pair is a result rather than one +tree compared with itself. + +**THE DEFECT, AND IT IS NOT A TYPO.** [[SAFETY-37]]'s freshness gate fires when a +battery-power register holds one value while the standing command moves. That +conjunction is satisfied by **two disjoint worlds**: + +1. the register has stopped reporting reality - which is what the gate was built + to catch; or +2. **reality is not moving**, because the source cannot comply with the command. + A BMS refusing a discharge at its floor holds the pack still under any + setpoint, and a register reporting that honestly publishes the same unchanged + number a dead one does. + +**Nothing the gate is handed separates them, and three sentences asserted the +first.** ⚠️ **On the one episode this repository has measured, it was the +second.** On 2026-09-21 the pack read **11 %** for the whole 850 s hold, **every +command inside that hold was a discharge**, and the register **answered the first +CHARGE command three seconds later**. All three numbers are read off the +committed capture by `test_freshness.py` part 11a rather than typed there. The +register was healthy and truthful throughout, and the add-on said it had failed. + +**THE THREE SURFACES, so you can recognise what moved:** + +| where you read it | what 0.10.1 said | what 0.11.0 says | +|---|---|---| +| the **action log** and the **status page** — both quote `freshness.py`'s `STALE_READING:` line verbatim, through `_read_unit()` and `stale_note()` | `… the reading is not following, so it is reported as MISSING` | `… TWO THINGS LOOK LIKE THIS AND THIS GATE CANNOT TELL THEM APART: the register may have stopped reporting reality, or the source may be unable to comply with the command …` | +| the **add-on log**, at ERROR, from `main.py` | `SAFETY-37: battery-power register is not following its own command` | `SAFETY-37: battery-power reading did not move under a command that did, WHICH IS NOT BY ITSELF A VERDICT ON THE REGISTER` | +| the **readiness row** on the diagnostics page | `battery power: is missing or not numeric` | `battery power: no usable value this cycle from - and this row cannot tell which of two things that is … Confirm the entity in Developer Tools before replacing it` | + +⚠️ **THE THIRD ROW IS THE ONE THAT COST AN OPERATOR AN EVENING, AND IT IS THE +WORST THING A DIAGNOSTIC CAN DO: IT ASSERTED A FALSE FACT ABOUT A NAMED +ENTITY.** It read *"is missing or not numeric"* for **any** `None`, and one of +the ways that value becomes `None` is the freshness gate withholding a reading +that is present, numeric and updating - `Freshness.reading()` returns `None` +there **by design**. So the operator opened Developer Tools on the entity the row +named, found it working, and had every reason to conclude the diagnostics were +broken. **It is fixed by saying less, not by looking harder:** the row does not +branch on the gate's state, because the gate already has a surface of its own - +`stale_note()` rides `liveness_notes()` a few rows below with the whole +`STALE_READING` sentence in it - so your discriminator is *"is that row there?"*, +and this row's only job is to stop pre-empting the answer. + +⚠️ **WHAT IT DOES NOT GIVE YOU IS WHICH OF THE TWO IT WAS.** Naming which would +need the SoC and the pack floor, and those are deliberately not `freshness.py`'s +inputs - they are [[TEL-08]]'s and [[SAFETY-32]]'s numbers, on their own rows. +**SoC is STATED in these sentences and never WIRED into them**: part 11b drives +the same two worlds at 50 % and at 11 % and the sentence does not move one byte. +What changed is that the add-on now tells you it cannot tell, instead of guessing +and blaming your hardware. **The reading rule is unchanged from +`### SAFETY-57` below**: if the stale/withdrawn pair keeps repeating, the unit +really does want looking at; if it happens once at a low SoC and stops, the pack +was empty and the site has resumed. + +⚠️ **AND THE THREE WORDINGS SHARE NO HELPER, ON PURPOSE.** Two of them are log +lines about a **behaviour** and the third is a banner about an **entity**; they +answer different questions ("what did the register do" against "can I trust this +entity"), and one helper serving both would have to drop whichever half the other +caller does not need - and the half it dropped would be the operator's next step. + +**Full account: `DECISIONS/SAFETY-56-what-the-gate-cannot-tell-apart.md`.** + +### TEL-11 — `pv_entity`: PV production gets recorded, and stays out of the control law + +⚠️ **UNSET, THIS IS NOT A CHANGE AT ALL - AND THAT IS MEASURED RATHER THAN +ASSERTED.** `pv_entity` ships empty, which is what **every** install that +upgrades into this version lands on. Empty means the add-on **reads no entity, +announces no entity, publishes no topic, shows no status row and adds no page +row.** `tools/tel11_pv_trace.py` drives five grid behaviours × eight +PV-sensor behaviours through 24 cycles of the **shipped** `Controller`, on +`release/1.0` and on this tree, and hashes two separate documents - one of +**control** decisions and one of **telemetry**: + +| document | tree | `pv_entity` | sha256 | +|---|---|---|---| +| control | `release/1.0` | unset | `846a10a5b89e3210…` | +| control | TEL-11 | unset | `846a10a5b89e3210…` | +| control | TEL-11 | **set** | `846a10a5b89e3210…` | +| telemetry | `release/1.0` | unset | `a443a6ceaffe072b…` | +| telemetry | TEL-11 | unset | `a443a6ceaffe072b…` | +| telemetry | TEL-11 | **set** | `4e6d1cb289a95722…` | + +**Read the two `unset` rows first.** They are byte-identical in **both** +documents, which is a stronger claim than "the control path is untouched": an +install that does not set the option sees no change of any kind, announcements +and status rows included. **Then read the `set` row of the control document** - +identical again, while telemetry moves. ⚠️ The tool **REFUSES** unless the two +trees' `app/` fingerprints differ and unless exactly one side reacts to the +option, so a matching pair cannot be two runs of the same tree. + +⚠️ **IT IS NOT A CONTROL INPUT, AND IT DOES NOT CHANGE WHAT THE LOOP DOES.** It +is instrumentation and nothing else. It is not read by the control law, the +allocator, the arbiter, the peak guard, the maintenance cycle or any watchdog; it +is **not part of `inputs_ok`**, so a PV integration that breaks cannot stop your +battery being regulated; and its status row is a **caution** (`warn`), never a +`bad` row, so a telemetry channel can never put a battery site into NOT READY. +`test_pv.py` part 6 drives 40 cycles of the real `cycle()` and asserts the +commands are bit-identical across unset / no reading / 0 W / 8 kW, with a +**computed** census of all 33 modules under `app/` showing the material is +mentioned in exactly three of them: `pv.py`, `main.py`, `mqtt.py`. Wiring +production into the control law would be a behaviour change on a system driving a +real battery - and it would destroy the measurement, because the loop would then +be reacting to the thing being measured. + +**WHAT YOU GET IF YOU DO SET IT.** One new MQTT-discovery sensor, +`sensor.goodwe_pv_power`, `device_class: power` and ⚠️ **`state_class: +measurement`** - which is the point of the row, because that is what puts the +number into the recorder's long-term statistics, and a week of statistics is what +the attribution needs. **Positive is producing.** The unit is **read off your +entity's own `unit_of_measurement`** - `mW`, `W`, `kW`, `MW`, `GW`, `TW`, Home +Assistant's own power units, every factor an exact power of ten. A sensor with +**no** unit, or with a unit that is not power, is **refused by name, and the +status page says which entity and which unit.** There is deliberately no +magnitude heuristic: nothing can separate 30 W of dusk from 30 kW of noon, and a +silent factor of 1000 in the one channel that exists to be believed a week later +is worse than a refusal you can read. + +⚠️ **ABSENCE IS PUBLISHED AS `unknown`, NOT AS 0 AND NOT SKIPPED. THIS IS THE +DESIGN DECISION WORTH YOUR ATTENTION.** + +| what happened | `sensor.goodwe_pv_power` | the status page | +|---|---|---| +| `pv_entity` not set | the entity does not exist | no row at all | +| set, sensor answered 4100 W | `4100.0` | `PV 4 100 W` | +| set, sensor answered 0 W | `0.0` | `PV 0 W` | +| set, sensor unavailable / missing / bad unit | `unknown` | `PV —`, plus a caution naming the reason | + +The last two rows are why the key is shaped this way. `publish()` normally +**drops** a `None`, which is right for `circulation` - there the last figure +standing beats a zero that reads as health - and **exactly wrong here**: a +dropped row leaves Home Assistant holding the last number it was told, for ever, +and the recorder then draws a **flat line through the outage**, which is +indistinguishable from a measurement. Anyone reading that week back would +conclude the sun was not shining, ⚠️ **which is precisely the conclusion this +channel exists to TEST rather than to manufacture.** So the absence is published, +using Home Assistant's own no-value payload (`mqtt/const.py` declares +`PAYLOAD_NONE = "None"` and `mqtt/sensor.py` tests it *before* the numeric +branch), the entity's state becomes **`unknown`**, and the recorder stores a +**gap**. A failed read also **clears** the previous good one: the value never +outlives its own sensor. + +⚠️ **WHY IT IS A HOME ASSISTANT ENTITY AND NOT A REGISTER.** The RS485 bus this +add-on speaks carries the **battery** inverter it drives. On nearly every site the +PV inverter is a different box from a different vendor on its own connection, +reporting into Home Assistant through its own integration - so this takes +whatever `sensor.*` your PV integration already provides, **any brand Home +Assistant can read**, and nothing anywhere assumes it is on the bus. + +**THERE IS NO `pv_invert`, AND THAT IS NOT AN OVERSIGHT.** `meter_invert` and +`batt_invert` exist because those two quantities are **bipolar and are read by +the control path**, where a reversed sensor hands the site a plausible number it +acts on. Production is unipolar and is read by **nothing**, so a reversed sensor +cannot mislead a decision, and it announces itself as **−4 kW at noon** in the +very history this key exists to create. A flag would only let an installer hide +it. There is no negative-value alarm either: a PV inverter's own standby draw +makes a small negative reading at night ordinary, so any threshold would be a +number nobody has measured. + +⚠️ **WHAT THIS RELEASE DOES NOT DO, AND IT IS THE REASON THE CARD EXISTS.** It +does not answer [[OPT-10]]'s question. That card measured this site over sixteen +days: the pack reached 90 % SoC on **two** of them, and **20.5 kWh of surplus +left the site while the pack still had room** - 12.6 kWh of it with the battery +not charging at all - and it could not say **why**, because PV production was +recorded **nowhere**. Three causes need three different fixes (no irradiance: +nothing to fix; a charge-power limit; the optimiser declining to charge, which is +the only software defect), and guessing between them is how an optimiser gets +tuned to solve a cloud problem. **0.11.0 adds the channel. The attribution needs +a week of history through it, and is OPT-10's own card.** So the reason to set +`pv_entity` on the day you take this update is that the clock on that week starts +when you do. + +**Full account: `DECISIONS/TEL-11-pv-production.md`, with the operator's version +under "PV production" in `DOCS.md`.** + ## 0.10.1 **A PATCH RELEASE, AND ONE TICKET IS THE WHOLE OF IT: [[SAFETY-57]].** The last @@ -50,8 +352,13 @@ operator can observe on an installed add-on - and are excluded for the reason field history found are NOT in this release**: [[SAFETY-56]] (the stale-reading message says *"is not following its own command"* when the true cause is an empty pack, and the NOT READY row that goes with it) and [[SAFETY-58]] (a trip -on the first read after a rest). Both are open, both are wording or bound -changes rather than deadlocks, and neither was allowed to delay this one. +on the first read after a rest). Both were open when this section was cut, both +are wording or bound changes rather than deadlocks, and neither was allowed to +delay this one. ⚠️ **[[SAFETY-56]] HAS SINCE BEEN FIXED AND IS IN `## 0.11.0` +ABOVE.** The wording quoted in this paragraph is what **0.10.1 actually ships** +and is left standing for that reason - if the box in front of you reports +`0.10.1`, that is the sentence it will print at you. It is not what 0.11.0 says. +[[SAFETY-58]] is still open. ### SAFETY-57 — the stale-reading verdict gets an exit @@ -168,6 +475,17 @@ a clearance the state could not produce. ⚠️ Its **diagnosis** half still say pack is merely empty; that wording is [[SAFETY-56]]'s and is deliberately untouched here. +⚠️ **AND THAT IS NOW A STATEMENT ABOUT 0.10.1 ONLY — [[SAFETY-56]] REWROTE THE +DIAGNOSIS HALF, AND `## 0.11.0` ABOVE CARRIES THE NEW WORDING.** The quotation +above is left exactly as it stands because it is what the 0.10.1 image prints, +and a reader on 0.10.1 needs to recognise it. The same applies to the reading +rule in the paragraph above: on 0.11.0 the add-on no longer asserts that +*"the register really is not following"* — it names both explanations and says +it cannot tell them apart — so *"the pair repeats → look at the unit, it happens +once and stops → the pack was empty"* stops being a correction of the message +and becomes the message's own advice. Nothing about **what** the gate does moved +in either card. + ## 0.10.0 ⚠️ **A CONFIG THAT IS LEGAL ON 0.9.0 IS REFUSED ON 0.10.0, AND THAT - NOT THE @@ -389,7 +707,10 @@ out of it that no test in this repository had reached**: [[SAFETY-56]] and `tools/predeploy_check.py` enforces that by itself: run against a box already reporting `0.10.0` it refuses with `THE DEPLOY WOULD BE A NO-OP` and names the runtime commits since the bump. **SAFETY-57's fix is in `## 0.10.1` above**, -which is what a patch number is for. +which is what a patch number is for. ⚠️ **AND SAFETY-56'S IS IN `## 0.11.0`**, +which is where it went because it landed after 0.10.1 had already been cut. Both +defects named in this paragraph are now fixed; this section, and `## 0.10.1` +below it, describe images that still contain them, and are left saying so. Contents of this release, in merge order since 0.9.0 (`86bdf49`): diff --git a/goodwe_controller/DECISIONS/SAFETY-56-what-the-gate-cannot-tell-apart.md b/goodwe_controller/DECISIONS/SAFETY-56-what-the-gate-cannot-tell-apart.md new file mode 100644 index 0000000..3f31875 --- /dev/null +++ b/goodwe_controller/DECISIONS/SAFETY-56-what-the-gate-cannot-tell-apart.md @@ -0,0 +1,181 @@ +# SAFETY-56 — what the gate cannot tell apart, said in the sentence + +**Text only. No behaviour changes in this card**, and §5 is the measurement that +says so: the same stimuli through the tree before and after produce a +**byte-identical** trace of every verdict, every `reading()`, every `inputs_ok` +and every readiness row's `(ok, warn)`. + +--- + +## 1. The defect + +0.10.1 is regulating a real GW5000S-BP. Two messages tell the operator that +healthy hardware has failed. + +**The log line**, as emitted on the reference site on 2026-09-21: + +> SAFETY-37: inverter-a battery-power register **is not following its own +> command** - STALE_READING: -3 unchanged for 36 s while the standing command +> moved 800 W to +1600 W - **the reading is not following**, so it is reported +> as MISSING rather than as a measurement … Treating it as MISSING, which +> commands 0 W. + +**The readiness row**, at the same moment: + +> NOT READY — battery power: `sensor.kelder_goodwe_master_stage1_goodwe_inverter_ac_power` +> **is missing or not numeric** + +That entity was present, numeric and updating. + +**SAFETY-37's conjunction is correct and is not touched by this card** — +`unchanged > 35 s AND |command| > 250 W AND |travel| > 250 W`. A register +genuinely stuck against a real command is exactly what it should catch, and a +pack that cannot comply is a state the site must stop in too. What was wrong is +what the sentences **concluded** from it. + +## 2. Two worlds satisfy the conjunction + +| | the register | the pack | +|---|---|---| +| **DEAD** | has stopped reporting reality | is doing what it was told | +| **REFUSED** | is reporting reality correctly | cannot comply — the BMS is refusing at its floor | + +A refused pack holds still under any setpoint, so an honest register publishes +the same unchanged number a dead one does. **Nothing `freshness.py` is handed +separates them.** All three sentences asserted DEAD. + +## 3. The live episode was REFUSED, and it is measured, not argued + +Off the committed capture `scenarios/safety57-2026-09-21/bus-a-2026-09-21-1030-1110.log.gz`, +whose bytes and decoder are already graded by `test_freshness.py` part 10a: + +| | | +|---|---| +| the register | answered **-3 W for 850 s**, 10:51:24 → 11:05:34 | +| the commands during that hold | **+260, +660, +1060 W** — discharge only, never a charge | +| pack SoC throughout | **11 %** | +| first **charge** command after the restart | 11:05:31 at **-400 W** | +| the register moves | **11:05:34** — **3 s**, inside one 17.0 s obligation window | + +**A register that answers a reversed command inside one obligation window had +not stopped reporting.** What would not move was the pack. The sentence the +operator read named the one thing in the loop that was telling the truth. + +⚠️ What this does **not** establish: that no register is ever intermittently +wrong. It excludes a register that had simply stopped; it cannot exclude one +that was wrong and then right. That is the point — the new sentence claims +neither. + +## 4. The fix, and why it is three sentences and not one helper + +Each message states **what was observed** and names **both** explanations. + +| where | before | after | +|---|---|---| +| `app/freshness.py` `_grade()` | *"…+1600 W - the reading is not following, so it is reported as MISSING…"* | *"…+1600 W. **TWO THINGS LOOK LIKE THIS AND THIS GATE CANNOT TELL THEM APART**: the register may have stopped reporting reality, or the source may be unable to comply with the command - a pack the BMS is refusing holds still under any setpoint, and an honest register then reads exactly like a frozen one. Either way the number cannot be read as this command's effect, so it is reported as MISSING…"* | +| `app/main.py` `_read_unit()` | *"SAFETY-37: %s battery-power **register is not following its own command** - %s."* | *"SAFETY-37: %s battery-power reading **did not move under a command that did, WHICH IS NOT BY ITSELF A VERDICT ON THE REGISTER** - %s."* | +| `app/main.py` `checks()` | *"{label}: {entity} **is missing or not numeric**"* | *"{label}: **no usable value this cycle from** {entity} - **and this row cannot tell which of two things that is**: the entity is missing or not numeric, or it is publishing a number that this controller is withholding (a freshness gate does exactly that, and names itself in a row of its own). Confirm the entity in Developer Tools before replacing it"* | + +**Three sentences, three wordings, no shared helper.** The first two are a log +line about a **behaviour**; the third is a banner about a named **entity**, and +the two answer different questions ("what did the register do" against "can I +trust this entity"). A helper serving both would have to drop whichever half +the other caller does not need, and the half it dropped would be the +operator's next step. The log line names the pair **once**, inside the `%s` it +already interpolates, rather than in both of its own halves. + +**No new input, and SoC is not wired.** `BOUNDS["soc"]` stays *STATED, NOT +WIRED*; telling the two worlds apart needs the pack floor (SAFETY-32's number) +and SoC (TEL-08's wiring), and a message honest about its own ambiguity needs +neither. Part 11b drives the two worlds at **50 %** and **11 %** SoC and +asserts the sentence does not move one byte. + +**The readiness row does not branch.** `inv.batt_fresh.stale` is on the same +object at the same moment, and the row deliberately does not ask it: the same +loop renders the grid and SoC rows, which have no gate, and a branch per row +would be a second place where "why is this None" is decided. The gate already +has a surface — `stale_note()` rides `liveness_notes()` a few rows below with +the whole STALE_READING sentence in it — so the operator's discriminator is +*"is that row there?"*, and part 11d asserts that row is **present in REFUSED +and absent in ABSENT**. + +## 5. Behaviour is byte-identical, measured + +`tools/safety56_behaviour_trace.py` prints two documents from one stimulus set. +The `behaviour` half contains **no sentence**: only `stale`, `reading()`, +`releases`, `held`, the window bookkeeping, `inputs_ok`, and every readiness +row's `(ok, warn)`. Run in the CI image, purged, `:ro`, against +`origin/release/1.0` **archived out of git** and against this branch: + +| | | +|---|---| +| the committed live capture | **2390 reads**, 7 verdict edges | +| a swept grid of command × register schedules | **66 cells × 120 reads** — 1060 stale reads, 1309 `MISSING` readings, 6 releases | +| six driven controller worlds through the shipped read path | DEAD, REFUSED, ABSENT, HEALTHY, RELEASE, RETRIP — every read's `(batt, inputs_ok, stale)` and every `checks()` row's `(ok, warn)` | +| **result** | `6030175ed2ab9a4143e86e89a71d45c8ab041ac1c00121a3e3e023cd2b08996c` **on both trees** | + +⚠️ **AND THE CONTROL IS INSIDE THE TOOL, BECAUSE AN IDENTITY BETWEEN TWO TREES +THAT ARE THE SAME TREE IS HABIT 4.** `--compare` refuses to report "behaviour +unchanged" unless the `sentences` halves **differ**. This card's first +invariance run was exactly that mistake — it archived a branch whose edits were +not committed yet, so `before` and `after` were the same code and the identity +was perfect and worthless. + +Both of the tool's refusals are **driven**, not asserted: + +| | | +|---|---| +| the same file compared with itself | `REFUSED: the sentences halves are IDENTICAL…`, exit **2** | +| `MOVE_W` 250.0 → 240.0 in the after tree | `BEHAVIOUR MOVED at 'sweep_stale_reads' / 'sweep_missing_reads' / 'sweep_trace'`, exit **1** | +| this card | `OK: sentences differ, behaviour is byte-identical.`, exit **0** | + +## 6. The mutation battery + +Each in its own `mktemp`'d copy (habit 18), purged on the host, mounted `:ro`, +graded by `tools/mutation_run.sh` in the CI image. + +| | mutation | Δ bytes | named reds | +|---|---|---|---| +| M1 | `_grade()` reverted to *"the reading is not following"* | -402 | **3** | +| M2 | the log line reverted to *"register is not following its own command"* | -80 | **2** | +| M3 | the row reverted to *"{entity} is missing or not numeric"* | -619 | **2** | +| M4 | *register* and *source* transposed between the two explanations | **0** | **1** | +| M5 | the second explanation deleted, the first kept | -216 | **1** | +| M6 | *"CANNOT TELL THEM APART"* replaced by *"IN ORDER OF LIKELIHOOD"* | -13 | **1** | +| M7 | the REFUSED world's ground truth set to the DEAD world's | 0 | **2** | +| M8 | the row's second cause deleted | -309 | **1** | +| M9 | `label` and `entity` transposed in the row | **0** | **3** | +| M10 | M1 + M2 + M3 together, i.e. exactly what 0.10.1 ships | -1101 | **5** | +| M11 | the accusation literals replaced by one that matches nothing | 0 | **1** | + +⚠️ **M9's first run was a defect in this card's own check, and it is the one +worth reading.** The row was fetched with `[...][0]`. Transposing `label` and +`entity` makes the row unfindable, so the run died on an `IndexError` — a red +with **no name attached**, with every check after that line **ungraded**. That +is FCST-06's shape exactly. `_s56_row()` now returns a named absence instead of +raising, and a precondition grades it: **0 named reds before, 3 after.** + +⚠️ **M11 is the other one aimed inward.** `not any(...)` over an empty list is +True, and so is one over three mistyped literals. The positive control runs the +same predicate against the three sentences **as 0.10.1 shipped them** and +requires it to find an accusation in every one, and every literal to match at +least one of them (habit 7: vary one thing from an accepted baseline). + +## 7. What part 11 does not grade + +* It does not show the live episode was a BMS refusal beyond doubt — see §3. +* It does not grade English. A sentence that named both explanations and then + said something useless about them would pass. +* The `no entity configured` branch beside the rewritten row is untouched and + ungraded: a blank entity really is the only thing it can be. +* The **grid-power and battery-SoC rows** go through the same rewritten line and + have no freshness gate today, so for them the second cause is a hypothesis + about this controller rather than a live possibility. That is weaker than the + battery-power case and is the price of not branching per row. A gate on either + signal — TEL-08 would be one — makes it exact. +* **SAFETY-58 is not touched** (the first read after a long rest trips before a + healthy register could answer), and neither is the 710/790 disagreement + between `test_freshness.py` and the changelog. Both are separate cards; the + second has a **fourth site** at + `DECISIONS/SAFETY-57-the-exit-from-a-stale-verdict.md:274` that a fix written + off the PR table alone would miss. diff --git a/goodwe_controller/DECISIONS/SAFETY-57-the-exit-from-a-stale-verdict.md b/goodwe_controller/DECISIONS/SAFETY-57-the-exit-from-a-stale-verdict.md index 7fefa0b..18b3626 100644 --- a/goodwe_controller/DECISIONS/SAFETY-57-the-exit-from-a-stale-verdict.md +++ b/goodwe_controller/DECISIONS/SAFETY-57-the-exit-from-a-stale-verdict.md @@ -360,6 +360,14 @@ two cards editing one sentence is how one of them silently loses. Only the false, and part 10h has a named check asserting the diagnosis half is still there so that rewrite has to be a deliberate act. +> ⚠️ **SINCE SUPERSEDED, WHICH IS WHAT THAT CHECK WAS FOR.** [[SAFETY-56]] is +> the deliberate act: all three sentences now name **both** explanations and +> accuse neither, and part 10h's check is inverted — it requires the accusation +> to be **gone**. Nothing else in this section moved; the escape clause above is +> still this card's and is untouched by it. See +> `DECISIONS/SAFETY-56-what-the-gate-cannot-tell-apart.md`, and +> `test_freshness.py` part 11 for the graded pair of worlds. + **Survivor 5 is older than this card and is not closed by it.** After a long rest the unchanged clock is already past 35 s, so the **first** read carrying a travelled command trips the gate — before the 17.0 s in which a healthy register diff --git a/goodwe_controller/DECISIONS/TEL-11-pv-production.md b/goodwe_controller/DECISIONS/TEL-11-pv-production.md new file mode 100644 index 0000000..a941683 --- /dev/null +++ b/goodwe_controller/DECISIONS/TEL-11-pv-production.md @@ -0,0 +1,351 @@ +# TEL-11 — the PV production channel + +*What was decided, what it was decided against, and what the evidence was.* + +## 0. The problem, in OPT-10's numbers + +Over sixteen days at the reference site the pack reached 90 % SoC on **two** of +them, and **20.5 kWh of surplus left the site while the pack still had room** — +**12.6 kWh** of it with the battery **not charging at all**. That single +measurement is upstream of most of what else was found that week: the nightly +floor, SAFETY-37's misdiagnosis, the SAFETY-57 deadlock, a 2 h 47 min saturation +freeze. + +It cannot be **attributed**, because **PV production is recorded nowhere in this +system**. Three causes need three different fixes: + +| cause | fix | +|---|---| +| there was no irradiance | none — the site behaved correctly | +| production was there, charge power was limited | a limit, somewhere | +| production was there, the pack had room, the optimiser declined | **the only software defect** | + +Guessing between them is how an optimiser gets tuned to solve a cloud problem. +**This card adds the channel and does not do the attribution.** Ship it, let it +record a week, OPT-10 picks it up with data. + +## 1. It is an HA entity, not a register — and that is a product decision + +The RS485 bus this add-on speaks carries the **battery** inverter it drives. On +nearly every site the PV inverter is a different box from a different vendor, +reporting into Home Assistant through its own integration; it is **not** on this +bus and there is no reason it would be. So the signal comes from an entity: +`pv_entity` takes whatever `sensor.*` the site's PV integration provides, any +brand HA can read. + +Reading a register would have been cheaper on one house and wrong for a fleet — +the trade [[product-not-one-house]] names. + +## 2. The sign convention: **positive is production**, and there is no `pv_invert` + +### The convention is derived, not assumed + +Two sign rules already ship, and both are recorded in `DOCS.md`'s *Sources* +table rather than inferred here: + +* `meter_entity` — **positive must mean importing** (`meter_invert` flips it); +* `batt_entity` — **`+` = discharging** (`batt_invert` flips it). + +Both are *"positive = power arriving at the house's AC bus"*. Production arrives +there too, so **positive = producing** is the continuation of the shipped +convention rather than a fresh choice. + +### Why an invert flag would be over-building + +The question is not "could a sensor be reversed" — it could. It is what a flag +would **buy**, and the answer is nothing, for two reasons that do not apply to +the two keys it would sit beside: + +1. **Those two quantities are read by the control path; this one is read by + nothing.** `inv.batt` feeds SAFETY-30's per-unit sign check and SAFETY-31's + circulating-power sensor (`_read_unit()`'s own docstring says so), so a + reversed battery sensor hands the site a plausible number it then **acts** + on — a latch that trips when it should not, or does not trip when it should. + `pv_w` reaches no decision at all (§4), so a reversed PV sensor cannot + mislead one. +2. **Production is unipolar, so the error is self-announcing.** Battery power + and grid power are legitimately negative half the time, which is exactly why + a flipped sign is invisible in them. A PV array does not consume: a reversed + sensor reads **−4 kW at noon**, in the first day of the very history this card + exists to create. An invert flag would let an installer *hide* that by + flipping it to something that looks right, and a second place for the sign to + be wrong is a second thing to get wrong. + +So: the convention is documented in `config.yaml`, both translations, `DOCS.md` +and `app/pv.py`, the value is published **exactly as the sensor reports it**, +and there is no flag. `test_pv.py` part 1 pins `-8 W` through unchanged. + +### And no negative-value alarm either + +The tempting middle path — warn when the reading is negative — needs a +threshold, because a PV inverter's own standby draw makes a small negative +reading at night entirely ordinary. Any threshold here would be a number nobody +measured, in a card whose whole subject is not fabricating numbers. The +published row is the detector. + +## 3. Absence is never zero, and it is not "dropped" either + +This is the constraint that shaped the publication, and the obvious +implementation fails it. + +`MqttPublisher.publish()` **skips** a `None` value. That is right for +`circulation` — its note in `main.py` says a missing per-unit reading should +leave the last figure standing rather than publish a zero that reads as health — +and it is **exactly wrong here**: a skipped row leaves Home Assistant holding +the last number it was told, *forever*, and the recorder then draws a **flat +line at that value through the outage**. Read back a week later that is +indistinguishable from a real measurement, and if the last value happened to be +small it manufactures precisely the *"there was no surplus to store"* conclusion +the card exists to **test**. + +So TEL-11 publishes the absence, using Home Assistant's own mechanism: + +> `homeassistant/components/mqtt/const.py`: `PAYLOAD_NONE = "None"`. +> `homeassistant/components/mqtt/sensor.py`, in the state-message handler: +> `if payload == PAYLOAD_NONE: self._attr_native_value = None; return` — and it +> is tested **before** the numeric branch, so it holds for a row carrying +> `device_class: power`. +> (github.com/home-assistant/core, branch `dev`, retrieved 2026-09-24. The +> shipped MQTT-sensor documentation says the same thing in its `state_topic` +> entry: *"a 'None' value will set the sensor to an unknown state"*.) + +The entity therefore goes to **`unknown`** and the recorder stores a **gap**. +`mqtt.ABSENT_AS_UNKNOWN` is a one-key set and `test_pv.py` part 3 asserts both +that the pv row gets this treatment **and that `circulation` still does not** — +widening the rule silently would be a behaviour change to another card's row. + +⚠️ **What was NOT built, and why.** A per-entity `availability` block +(`availability_mode: all`, an extra retained topic) would also produce a +distinguishable state — `unavailable` rather than `unknown` — but it needs a +second topic, a second retained message, a change to `_payload()`'s +byte-order-sensitive key construction, and it survives restarts only if the +retention is right. The payload route is one branch in `publish()` and one +constant, and it is graded as **bytes on the topic** rather than as a claim +about HA's availability handling. + +### Three states, not two + +| state | frame | wire | page | +|---|---|---|---| +| no source (`pv_entity` unset) | no `pv` key | no topic, no entity | no row | +| no reading | `pv: None` | `goodwe_ctl/pv` = `None` | `PV —` + a caution naming the reason | +| zero watts | `pv: 0.0` | `goodwe_ctl/pv` = `0.0` | `PV 0 W` | + +A failed read also **clears** the last good value (`_read_pv()` assigns on every +call), so a reading never outlives its own sensor. `test_pv.py` part 2 drives a +good read followed by a dead one, because a fixture that answers the same thing +forever cannot reach that failure — MUTATION.md habit 9's shape. + +## 4. It must not become a control input, and that is graded twice + +### In the suite + +`test_pv.py` part 6 drives twelve cycles of the real `Controller.cycle()` +against a 6 kW export and asserts the commands written to the inverter are +**bit-identical** across `unset` / `no reading` / `0 W` / `8 kW` of production, +with a precondition check that the stimulus commands something at all (habit 1). +`inputs_ok` is asserted not to move either: a broken PV integration is **not** a +missing input, because nothing computes with it. + +A computed census (habit 12 — the scope is derived from the tree, not typed) +then reports which of the 33 modules under `app/` mention the material in any +spelling, comments included. It is **three**: `pv.py`, `main.py`, `mqtt.py`. The +scanner's own red direction is exercised on every run. + +### Across trees, by hash + +`tools/safety56_behaviour_trace.py` is the pattern; `tools/tel11_pv_trace.py` is +this card's version of it, and **its control clause had to be different**, which +is worth stating because the difference is the interesting part. + +SAFETY-56 changed wording and had to show behaviour did not move, so its +`--compare` **refuses** unless the `sentences` halves **differ** — otherwise the +two runs might be one tree twice. TEL-11's headline claim is the opposite shape: +with the option unset, **nothing** differs. "The documents are identical" is the +*result*, so it cannot also be the control. The two things proved instead are: + +* the two runs came from **different trees** (`app/*.py` sha256 + byte count, + compared, and an identity is **refused**); +* **exactly one** side reacts to `pv_entity` — that side is the HEAD, the other + is a tree that does not have the option. Neither, or both, is refused. + +It drives 41 stimulus cells (5 grid worlds × 8 PV schedules, plus a run stopped +part-way) and prints two documents per setting: `control` — every command +written, every `self.target`, `inputs_ok`, the loop's three fields, the +arbiter's resolution, the maintenance phase, tick by tick, **with no TEL-11 +material in it** — and `telemetry`, which is everything the card is allowed to +move. + +Measured, in the CI image, on purged trees: + +``` + HEAD is B; baseline is A (it does not react to `pv_entity`). + tree A 73296909170ec789237d6e704cce9ab0d9f83b3df1495a14d2497285c3041a12 + tree B d5011b6a18e2e3803dc09cac0214b0378fa4c7388fc0ad8ad89b832a5c92af48 + control A unset 846a10a5b89e321086d535825925721e2f791012e8405948d4fd38d3f9d0be5f + control B unset 846a10a5b89e321086d535825925721e2f791012e8405948d4fd38d3f9d0be5f + control B set 846a10a5b89e321086d535825925721e2f791012e8405948d4fd38d3f9d0be5f + telemetry A unset a443a6ceaffe072b44bcd5f2ad946c1ed6c9e01b69e278eca631f7f4d7892164 + telemetry B unset a443a6ceaffe072b44bcd5f2ad946c1ed6c9e01b69e278eca631f7f4d7892164 + telemetry B set 4e6d1cb289a957223ad0205f732a6fec6110ce1f2032c98ad60694e67625274f +OK 1/3 UNSET: control AND telemetry are byte-identical to the baseline tree. +OK 2/3 SET: the control document is byte-identical with the channel switched on. +OK 3/3 CONTROL OF THE CONTROL: the telemetry document DOES move when the + option is set (42 stimulus cells differ). +``` + +Note what clause 1 claims and clause 2 does not: **unset** is identical in +`control` *and* `telemetry`, which is stronger than "the control path is +untouched" — an install that does not set the option sees no change of any kind, +including its announcement and its status rows. **Set** is identical in +`control` only, which is the whole of "instrumentation, not an input". + +## 5. Genuinely optional — and not a second `price_now_entity` + +`price_now_entity` / `price_avg_entity` are legal **only** while +`price_provider` is `off`, and the add-on **refuses to start** otherwise. That +is the trap this card was told not to rebuild, and it is avoided structurally +rather than by intention: + +* `validate.py` does not mention `pv_entity` at all. It cannot conflict with + anything, because nothing else reads it. +* the MQTT row lives in `OPTIONAL_SENSORS`, not `SENSORS`, and + `discovery_payloads()`'s new `include` argument **defaults to empty**. So a + caller that has never heard of `pv_entity` — every fixture in `test_mqtt.py`, + `tools/rehearse_roster.py`, anything written next year — announces exactly + what it announced before. `p1_age` solves the same problem the other way + round, with `omit`, because it is announced by default and withdrawn when the + meter is off; getting that direction wrong here would have left most installs + with an entity nothing feeds, which is the hazard `omit`'s own note names. +* the read is gated on the option, so an unset install makes **no state call at + all** — not a read that fails quietly. +* there is **no status row** when it is unset. `peak_forecast_entity` nags when + it is unset because an unset forecast leaves a money guard without an input; + this is instrumentation, and a permanent caution on every install that never + asked for a PV channel is the warning spam the card forbids. +* when it **is** set and the read fails, the row is `ok=False, warn=True` — a + **caution**. `status_payload()` builds `NOT READY` out of rows that are + neither `ok` nor `warn`, so a telemetry channel can never put a battery site + into `NOT READY`. + +## 6. The unit is read off the entity, and an unreadable one is refused + +`read_inputs()` guesses kW-or-W for `peak_forecast_entity` with +`abs(raw) < 50`, and that is safe there: nobody's quarter-hour demand forecast +is 50 W. It is **not** safe here — 30 W of real production at dusk is an +ordinary reading, and that heuristic would publish it as 30 kW. + +So `pv.watts()` reads the entity's own `unit_of_measurement` and converts +against Home Assistant's own `UnitOfPower` table (`mW`, `W`, `kW`, `MW`, `GW`, +`TW`; every factor an exact power of ten). A sensor with **no** unit, or a unit +outside that table, is **refused by name** and the status page says which. The +refusal is loud; a guess would be silent, and a factor of 1000 in the one +channel that exists to be believed a week later is the worse failure. + +⚠️ `BTU/h` is in HA's enum and is deliberately **not** in the table: its factor +is a measured conversion rather than a decimal shift, no PV integration +publishes in it, and a fabricated constant is what this repository's whole +process exists to prevent. It lands in the refusal, by name, and `test_pv.py` +part 1 pins that. + +## 7. The mutation battery + +Every check this card adds was driven by the mutation that defeats the claim it +names. Method: `cp -a` of the tree per mutation into a **unique** directory, +the edit applied there, `__pycache__` purged, the suites run in the CI image +with the tree bind-mounted `:ro`, one file at a time. **The shipped tree is +never edited**, so habit 18's revert hazard — SAFETY-37's harness killed between +the edit and the `cp` that put the file back, and two harnesses sharing one +backup path — is closed by construction rather than by a verified revert. Each +row below records the byte delta, the file's sha256 before and after, and that +the mutated line was **read back off disk** before the run started. + +| # | mutation | file | bytes | sha256 (first 16) before → after | red on | +|---|---|---|---|---|---| +| M1 | the `pv` row moved into `SENSORS`, i.e. announced on every install | `mqtt.py` | −79 B | `328d249b9d698ef6` → `532d3954c023bf3d` | 3 named in `test_mqtt.py`, 1 in `test_pv.py` | +| M2 | `publish()` drops a `None` `pv` row again, so HA keeps the last number | `mqtt.py` | −98 B | `328d249b9d698ef6` → `dd54be458f446475` | 1 named (the acceptance criterion itself) | +| M3 | `ABSENT_AS_UNKNOWN` widened past the one key it was written for | `mqtt.py` | +23 B | `328d249b9d698ef6` → `8fe571ebff3dacf4` | 1 named (`circulation` must still be dropped) | +| M4 | a dead sensor reported as **0 W** instead of an absence | `pv.py` | −1 B | `55ba7471e9995436` → `9eecc34152b7ac12` | 4 named | +| M5 | `_read_pv()` assigns only on a good read — a value outlives its sensor | `main.py` | +46 B | `c4d2cab34e50b5fd` → `189802aa69ce166d` | 2 named | +| M6 | the unit is read and then ignored — a kW sensor published as watts | `pv.py` | −18 B | `55ba7471e9995436` → `ab19ef66c52f9379` | 2 named | +| **M7** | **`self.grid = self.grid − self.pv_w` in `read_inputs()` — PV wired into the control path** | `main.py` | +106 B | `c4d2cab34e50b5fd` → `c38216edb7c5c8db` | **2 named, plus `tel11_pv_trace.py` clause 2/3 on 26 of the 41 stimulus cells** | +| M8 | `optional_keys()` returns `("pv",)` unconditionally | `mqtt.py` | −52 B | `328d249b9d698ef6` → `86ff23efe3088ad8` | 3 named | +| M9 | the PV caution made a `bad` row, i.e. a dead sensor is a `NOT READY` | `main.py` | +1 B | `c4d2cab34e50b5fd` → `45c74cab89943564` | 1 named | +| M10 | the read gate removed — an unset option is read once a second forever | `main.py` | −10 B | `c4d2cab34e50b5fd` → `b67baa8aba3019e8` | 2 named, **plus two crashes** | + +`on_disk = True` for all ten: the mutated line was grepped back out of the file +after the edit and before the run (MUTATION.md reporting rule 2), and no +mutation is size-preserving, so rule 1's `+0 B` caveat does not arise. + +⚠️ **M10's two extra reds are CRASHES, not named checks, and that is recorded +rather than tidied away.** Every `Hass` double in `test_inverter.py` and +`test_telemetry.py` predates this read and has no `state()` method, so an +unconditional PV read is an `AttributeError` rather than a failed assertion — +habit 20's shape, a red with no name attached. Fixing those fixtures would be a +change to what *their* checks report and belongs to whoever next touches them; +this card's own fixture has `state()` and grades the property by name. + +⚠️ **M7's trace limb is where the two halves meet.** The mutant moves the +control document on 26 of 41 cells — every cell whose PV schedule produces a +READING. The five `none` cells and the five `zero` cells do **not** move, which +is correct and is worth seeing: subtracting nothing, and subtracting zero, +change nothing. A battery that only drove a dead sensor would have reported this +mutation as a survivor for the third time. + +The clean tree those `before` hashes belong to: `app/main.py` +`c4d2cab34e50b5fd`, `app/mqtt.py` `328d249b9d698ef6`, `app/pv.py` +`55ba7471e9995436`. + +### ⚠️ M7 was a SURVIVOR on the first battery, and the fixture was the reason + +The card's **central** mutation — `self.grid = self.grid - self.pv_w` in +`read_inputs()`, i.e. exactly the defect the whole card forbids — came back +**green** on a six-file run at **+106 B**, on disk, purged, genuinely running. + +The property did not hold; the fixture could not reach it. `test_pv.py` part 6 +drove a **6 kW export over 12 cycles**, which puts the loop **against its +rail**: the command moves `slew_w` every cycle and −6000 W and −14 000 W of +apparent error produce the **same ramp, tick for tick**. MUTATION.md habit 23, +with a saturated actuator in place of a coinciding value — and the habit's list +of no-op-prone constructs is all *expressions*, so this is worth its own entry, +which it now has. + +The fixture now drives a **600 W import the loop settles on**, with a +responsive meter (`grid = base − batt`) so the error actually closes, asserts +the resting target is well inside the rail, and asks habit 23's own question as +its own check: **shift the meter by the same 8 kW and confirm the commands +really would have moved.** On the first fixture that question answers *no*, in +one line. Every number in the table above is from the battery re-run against +the fixed tree. + +⚠️ **A red count is a property of the suite as much as of the mutation.** These +name **this** tree at this commit; a later card that adds or re-bases a check +moves the number with the mutation untouched. Re-measure rather than inherit. + +## 8. What none of this covers, with shapes + +* **Home Assistant itself.** That `None` renders as `unknown` and that the + recorder keeps a gap is read out of HA's source and quoted in §3; no HA runs + in this suite. What is graded here is the bytes this add-on publishes. +* **Whether the operator's sensor tells the truth.** A sensor wired backwards, + or reporting DC where AC was meant, or a string instead of the array, is + published faithfully. §2 is the argument for why that is the right failure + mode; it is not a claim that it cannot happen. +* **A roster of more than one.** The row is a **site** row — there is one PV + array as far as this add-on is concerned — so nothing per-unit exists to + separate, and every fixture is the shipped roster of one. +* **The attribution.** This card records production. Whether the 20.5 kWh was + cloud, a limit or the optimiser is OPT-10's, and it needs a week of this + channel's history before it can be asked. +* **`tools/`, the firmware and the ingress HTML** are outside part 6's census. +* **The identity is only as wide as the stimulus.** `tel11_pv_trace.py` prints + its 41 cells with the documents for exactly that reason. + +## 9. No `CHANGELOG.md` entry + +`test_version_gate.py` 11b refuses a newest `## ` heading that is not +`config.yaml`'s `version:`, and opening an `## Unreleased` section would mean +taking a packaging decision this card does not get to take. The operator account +is in `DOCS.md`; the release note belongs to the next PKG ticket, which should +say that **0.10.1 → next** adds one optional key, announces no new entity on an +install that does not set it, and changes no control behaviour. diff --git a/goodwe_controller/DOCS.md b/goodwe_controller/DOCS.md index 14df547..faba10a 100644 --- a/goodwe_controller/DOCS.md +++ b/goodwe_controller/DOCS.md @@ -139,6 +139,7 @@ either direction, and the check is mutation-tested both ways. | `batt_invert` | | Flip if needed | | `capacity_kwh` | | This pack's usable capacity. Read only to divide the site target between packs by **energy** headroom — see below | | `setpoint_entity` | yes | The ESPHome `number.*_goodwe_setpoint_w` | +| `pv_entity` | | Optional. Your **solar** inverter's power sensor, `+` = producing. Measurement only — it changes no control behaviour. See [PV production](#pv-production-pv_entity-tel-11) | Use the ESP32's readings rather than the inverter's cloud or dongle sensors: those serve cached values, and a stale reading here ends the maintenance charge @@ -413,6 +414,97 @@ and `Inverter B` — are two names to the validator and one id once slugged. The add-on gives the second one a `_2` suffix so nothing collides, but the ids you get then depend on roster order. Give your inverters plainly different names. +### PV production (`pv_entity`, TEL-11) + +**Optional, off by default, and it changes no control behaviour whatsoever.** +Set it and the add-on records what your solar is producing beside what the +meter and the battery are doing. Leave it empty and the add-on reads no entity, +announces no entity, publishes no topic and shows no row — which is what every +install that upgrades into this version does. + +#### Why it is a Home Assistant entity and not a register + +This add-on's RS485 bus carries the **battery** inverter it drives. On nearly +every site the PV inverter is a different box from a different vendor, on its +own connection, reporting into Home Assistant through its own integration. So +`pv_entity` takes whatever `sensor.*` your PV integration already provides — +any brand Home Assistant can read. It does not have to be on the bus, and there +is no assumption anywhere that it is. + +#### What it is for + +OPT-10 measured this site over sixteen days: the battery reached 90 % on **two +of them**, and **20.5 kWh of surplus left the site while the pack still had +room** — 12.6 kWh of it with the battery not charging at all. Three different +things produce that number and they need three different fixes: + +* there was no irradiance to store — nothing to fix; +* production was there and something limited the charge power; +* production was there, the battery could take it, and the optimiser declined. + +Only the third is a software defect, and **none of them can be told apart +without a record of production.** This entity is that record. The attribution +itself is OPT-10's, with a week of history behind it. + +#### The sign, and why there is no `pv_invert` + +**Positive must mean producing.** That is the same convention as the two sign +rules the add-on already ships — `meter_entity` is positive when **importing** +and `batt_entity` is positive when **discharging**, both of which are "power +arriving at the house" — and production arrives there too. + +`meter_invert` and `batt_invert` exist because those two quantities are +**bipolar and are read by the control path**: a reversed battery sensor is a +perfectly plausible number that the per-inverter sign failsafe then acts on. +Production is unipolar and is read by **nothing**, so a reversed sensor here +cannot mislead a decision — and it announces itself immediately as −4 kW at +noon in the very history this entity creates. A flag would only hide it, so +there is not one. If your sensor really is inverted, fix it where it is +produced. + +#### The unit is read off the entity + +The add-on reads your entity's own `unit_of_measurement` and converts: `mW`, +`W`, `kW`, `MW`, `GW`, `TW` — Home Assistant's own power units. A sensor with +**no** unit, or with a unit that is not one of those, is **refused**, and the +status page says which entity and which unit. That is deliberate: there is no +magnitude test that can separate 30 W of dusk from 30 kW of noon, so a guess +here would be a silent factor of 1000 in the one channel that exists to be +believed a week later. + +#### ⚠️ Absence is never zero + +| what happened | `sensor.goodwe_pv_power` | on the status page | +|---|---|---| +| `pv_entity` not set | the entity does not exist | no row at all | +| set, sensor answered `4100 W` | `4100.0` | `PV 4 100 W` | +| set, sensor answered `0 W` | `0.0` | `PV 0 W` | +| set, sensor unavailable / missing / bad unit | `unknown` | `PV —`, plus a caution naming the reason | + +The last two rows are the point. A dead sensor publishes Home Assistant's own +"no value" payload, so the entity goes to **`unknown`** and the recorder stores +a **gap** — not a flat line at the last number it saw. Reading a week of +history back, *"no PV was reported"* and *"PV reported 0 W"* are different +pictures, which is exactly the distinction the whole card exists to preserve. + +A failed read also **clears** the previous good one: the value never outlives +its own sensor. + +#### ⚠️ What it does NOT do + +It is **instrumentation**. It is not an input to the control law, the +allocator, the arbiter, the peak guard, the maintenance cycle or any watchdog; +it is not part of `inputs_ok`, so a PV integration that breaks cannot stop the +battery being regulated; and its status row is a **caution**, never a +`NOT READY`. Wiring production into the control law would be a behaviour change +on a system driving a real battery — and it would destroy the measurement, +because the loop would then be reacting to the thing being measured. + +`tools/tel11_pv_trace.py` is what says so rather than claims it: it drives 41 +stimulus worlds through the shipped controller and hashes a document of control +decisions containing none of this material, on this tree and on `release/1.0`, +with the option set and unset. + ### P1 meter ingestion `meter_entity` above expects one signed sensor, which usually means a template diff --git a/goodwe_controller/MUTATION.md b/goodwe_controller/MUTATION.md index 14fec18..c9e83a2 100644 --- a/goodwe_controller/MUTATION.md +++ b/goodwe_controller/MUTATION.md @@ -896,6 +896,33 @@ Cheap habits fall out of it, and all of them are now used in this repository: disjointness and the suite pins **that** - at 0, 100, 249, 250, 251 and 1300 W - rather than an order nothing depends on. + ⚠️ **SECOND INSTANCE, TEL-11, AND THE NO-OP WAS A SATURATED RAIL RATHER + THAN A VALUE.** The card's whole subject is that a PV reading must never + reach the control law, so its central mutation is + `self.grid = self.grid - self.pv_w` in `read_inputs()` - **+106 B**, on + disk, purged, genuinely running - and it came back a **SURVIVOR** of a + six-file battery. The property held in no sense: the fixture drove a 6 kW + export over 12 cycles, which puts the loop **against its rail**, and a + command moving `slew_w` every cycle produces the SAME ramp for -6000 W and + for -14000 W of apparent error. The mutant ran and could not change the + answer. + Two things generalise past PV: + * **A SATURATED ACTUATOR IS A VALUE AT WHICH EVERY INPUT COINCIDES.** The + habit-23 list above names `or`, `and`, truthiness, default arguments and + `getattr` fallbacks - all *expressions* with a coinciding value. A rail, + a clamp, a slew limit, a deadband and a latched alarm are the same hazard + in the **plant**: they are where a controller's output stops being a + function of its input. A fixture that parks there cannot separate any two + inputs, and this repository's control fixtures park there by default, + because the dramatic stimulus is the saturating one. + * **SO THE PRECONDITION IS NOT "DOES IT COMMAND SOMETHING", IT IS "DOES IT + REST SOMEWHERE THE MUTATION COULD MOVE IT FROM".** `test_pv.py` part 6 + now drives a 600 W import the loop SETTLES on, asserts the resting target + is well inside the rail, and asks habit 23's own question as a check: + shift the METER by the same 8 kW and confirm the commands really would + have moved. On the first fixture that question answers NO, in one line, + and it is the line that turns the identity into evidence. + And the mutation-table consequence: **a variant is not covered because a near-miss of it is.** DEV-02's table listed the *actuator* variant of the per-unit claim defect (`device`/`supervised` filed without `inv`), which the diff --git a/goodwe_controller/app/freshness.py b/goodwe_controller/app/freshness.py index 5b04f2c..4527243 100644 --- a/goodwe_controller/app/freshness.py +++ b/goodwe_controller/app/freshness.py @@ -565,10 +565,44 @@ class Freshness: # and different afterwards, and an operator comparing this line against # a history graph is reading the register. shown_s = self.unchanged_s(now_mono) + # ⚠️ SAFETY-56. THE OBSERVATION IS A FACT, THE DIAGNOSIS WAS NOT, AND + # THE SENTENCE NOW STOPS WHERE THE EVIDENCE DOES. The conjunction above + # is satisfied by TWO disjoint worlds, and nothing this object is handed + # separates them: + # (1) the register has stopped reporting reality - what the gate was + # built for - or + # (2) reality is not moving, because the SOURCE cannot comply with + # the command. A pack the BMS is refusing at its floor holds + # still under any setpoint, and a register reporting that + # truthfully publishes the same unchanged number a dead one does. + # The live 2026-09-21 episode was (2) - the pack read 11 % for the + # whole 850 s hold, the commands were discharge-only, and the register + # answered the first CHARGE command within one obligation window + # (test_freshness.py 11a measures all three off the committed capture) + # - and this sentence asserted (1), which sent the operator to look at + # healthy hardware. + # Naming both is what the module can honestly say. Naming WHICH would + # need SoC and the pack floor - deliberately not this module's inputs + # (see the `soc` row in BOUNDS and the docstring's last section), and + # TEL-08's and SAFETY-32's numbers rather than this sentence's. + # + # ⚠️ AND IT IS NOT SHARED WITH `main.py`'s NOT READY ROW, WHICH SAYS + # THE SAME AMBIGUITY IN DIFFERENT WORDS ON PURPOSE. This sentence is a + # LOG LINE about a behaviour, quoted verbatim by `note()` onto the + # status page and by `_read_unit()` into the action log; that one is a + # BANNER about a named ENTITY, and the two answer different questions + # ("what did the register do" against "can I trust this entity"). One + # helper serving both would have to drop whichever half the other does + # not need, and the half it dropped would be the operator's next step. self.reason = ( f"STALE_READING: {self.held:+.0f} unchanged for {shown_s:.0f} s " f"while the standing command moved {moved_w:.0f} W to " - f"{standing_w:+.0f} W - the reading is not following, so it is " + f"{standing_w:+.0f} W. TWO THINGS LOOK LIKE THIS AND THIS GATE " + f"CANNOT TELL THEM APART: the register may have stopped reporting " + f"reality, or the source may be unable to comply with the command " + f"- a pack the BMS is refusing holds still under any setpoint, and " + f"an honest register then reads exactly like a frozen one. Either " + f"way the number cannot be read as this command's effect, so it is " f"reported as MISSING rather than as a measurement (limits: " f"unchanged over {self.bound.unchanged_s:.0f} s, command over " f"{self.bound.move_w:.0f} W and moved by over " diff --git a/goodwe_controller/app/main.py b/goodwe_controller/app/main.py index b62287a..67d3bee 100644 --- a/goodwe_controller/app/main.py +++ b/goodwe_controller/app/main.py @@ -62,10 +62,11 @@ from .inverter import (LINK_STALE_S, UNIT_A, build_roster, from .maintenance import ( IDLE, POWER_MOVING, MaintConfig, Maintenance, arbitrate as maint_arbitrate, ) -from .mqtt import (MqttPublisher, announced_units, unit_slugs, +from .mqtt import (MqttPublisher, announced_units, optional_keys, unit_slugs, unit_state_key) from . import peakguard from . import packshare +from . import pv as pv_read from . import shadow from .p1 import P1Ingest, build_source, is_enabled from .price import Uncertainty, price_option, provider_configured @@ -872,6 +873,22 @@ class Controller: self.loop_i_lead = 0.0 self.loop_reason = "not-running" self.loop_w = None # what the loop asked for last cycle, or None + # --- TEL-11: the site's PV production, and NOTHING READS IT ---------- + # ⚠️ INSTRUMENTATION. It is published, it is shown on the status page, + # and it reaches no decision: not `inputs_ok`, not the allocator, not + # the arbiter, not a claim, not a watchdog. Wiring it into the control + # law would be an unrequested behaviour change on a system driving a + # real battery AND would destroy the measurement, because the loop would + # then react to the quantity being measured. `tools/tel11_pv_trace.py` + # is the instrument that says so, on both trees and both settings. + # + # ⚠️ `None` IS A READING THAT DOES NOT EXIST, AND IT IS SEEDED AS ONE. + # Nothing here ever becomes 0.0 by default; see `app/pv.py`. + self.pv_w = None + # Why there is no reading, in words the page prints - or, on a good + # read, the UNIT the value came in. Never empty. + self.pv_why = ("not configured" if not self.pv_entity + else "not read yet") # Did the grid-following loop actually win the LAST arbitration? That is # the question SAFETY-02 has to ask about the command standing on the # wire right now, so it is recorded when the arbitration happens rather @@ -1651,6 +1668,38 @@ class Controller: for inv in self.inverters) # -- io ------------------------------------------------------------------ + @property + def pv_entity(self) -> str: + """The configured PV production entity, or `""`. TEL-11. + + ⚠️ ONE READING OF THE OPTION, and `mqtt.optional_keys()` is graded + against it: the thing that decides whether the row is ANNOUNCED and the + thing that decides whether it is READ have to be the same predicate, or + the add-on ends up with an entity nothing feeds or a topic nothing + declared. `announced_units()` in mqtt.py is the same argument for the + roster, in its own words. + """ + return str(self.o.get("pv_entity") or "").strip() + + async def _read_pv(self) -> None: + """This cycle's PV production, or an ABSENCE with a reason. TEL-11. + + ⚠️ IT ASSIGNS ON EVERY CALL, INCLUDING THE FAILING ONES. Leaving the + last good number standing through an outage is what `circulation` does + on purpose - there, a stale figure beats a zero that reads as health - + and it is exactly wrong here: this row is read back as HISTORY, and a + value that persists through an outage is indistinguishable from a real + one at the moment somebody is trying to work out whether the sun was + shining. + + ⚠️ `hass.state()` RATHER THAN `hass.number()`, because the UNIT is half + the reading. See `pv.watts()`: a PV integration may publish kW, and + there is no magnitude heuristic that can separate 30 W of dusk from + 30 kW of noon. + """ + self.pv_w, self.pv_why = pv_read.watts( + await self.hass.state(self.pv_entity)) + async def read_inputs(self, now_mono: float | None = None) -> None: """Every input this cycle needs, and SAFETY-37's freshness verdict. @@ -1682,8 +1731,26 @@ class Controller: # names for the WRITE path, one function up. Within a unit the order is # unchanged (SoC, then battery power), so at the shipped roster of one # this is the same two calls in the same order. - await asyncio.gather(*(self._read_unit(inv, now_mono) - for inv in self.inverters)) + # + # --- TEL-11: AND THE PV READ RIDES IN THIS SAME GATHER --------------- + # ⚠️ IN THE GATHER RATHER THAN ON A LINE OF ITS OWN, AND THAT IS THE + # WHOLE "OBSERVABILITY MUST NOT BE ABLE TO DELAY CONTROL" DECISION. + # This coroutine feeds the ESP32 heartbeat; a sequential read would add + # its own worst case - `hass.state()`'s 10 s timeout - to every cycle, + # so a hung PV sensor at a client site would push a healthy battery + # towards layer 1's ~30 s veto for the sake of a telemetry row. + # Gathered, its latency OVERLAPS reads that already happen and the + # added worst case is zero. `peak_forecast_entity` below is read + # serially and stays that way because it is a CONTROL input (the + # capacity guard sizes off it), so the site is entitled to wait for it. + # + # ⚠️ AND AN UNSET `pv_entity` APPENDS NOTHING. Not a coroutine, not an + # HTTP call, not a log line: at the shipped default this gather is the + # same list of the same coroutines it has always been. + _reads = [self._read_unit(inv, now_mono) for inv in self.inverters] + if self.pv_entity: + _reads.append(self._read_pv()) + await asyncio.gather(*_reads) if o.get("peak_forecast_entity"): raw = await self.hass.number(o["peak_forecast_entity"]) # Accept kW or W - nobody's quarter-hour forecast is 50 W. @@ -1785,13 +1852,28 @@ class Controller: # 0 W it announces in the sentence before is exactly what stops the # register ever being asked to report anything different. The live # site sat in it for 790 s on 2026-09-21 and was freed by a human. - # ⚠️ THE DIAGNOSIS HALF IS SAFETY-56'S AND IS DELIBERATELY UNTOUCHED - # HERE - "is not following its own command" is wrong in a different - # way (a pack the BMS is refusing is not a faulty register) and that - # card is rewriting it. Two cards editing one sentence is how one of - # them silently loses. - _LOG.error("SAFETY-37: %s battery-power register is not following " - "its own command - %s. Treating it as MISSING, which " + # ⚠️ SAFETY-56 REWROTE THE DIAGNOSIS HALF, WHICH SAFETY-57 LEFT + # STANDING ON PURPOSE. It used to open "battery-power register is + # not following its own command" - an accusation against a named + # piece of hardware, and FALSE on the very episode above. The + # pack read 11 % for the whole hold, every command in it was a + # DISCHARGE, and the register answered the first CHARGE command + # three seconds later - so the BMS was refusing, the register was + # publishing that refusal correctly, and the operator was sent to + # look at a working sensor. All three numbers are measured off the + # committed capture by test_freshness.py part 11a. The gate's + # conjunction is right and is NOT touched; only its conclusion was. + # + # ⚠️ AND THE TWO EXPLANATIONS ARE NAMED ONCE, IN `%s`, NOT TWICE. + # This line's own words are the OBSERVATION plus the one thing a + # log reader has to be told before acting - that the observation is + # not yet a verdict - and `freshness._grade()`'s sentence, which + # this line already interpolates, is where the pair is spelled out. + # Saying it in both halves of one sentence is how an operator + # learns to skim it. + _LOG.error("SAFETY-37: %s battery-power reading did not move under " + "a command that did, WHICH IS NOT BY ITSELF A VERDICT " + "ON THE REGISTER - %s. Treating it as MISSING, which " "commands 0 W. It clears itself the moment the register " "reports a different value, and failing that %.0f s " "after the standing command falls below %.0f W " @@ -4382,6 +4464,24 @@ class Controller: "loop_lead": round(self.loop_i_lead, 1), "loop_reason": self.loop_reason, } + # --- TEL-11 ---------------------------------------------------------- + # ⚠️ THE KEY IS PRESENT WITH THE VALUE `None` WHEN THERE IS NO READING, + # AND ABSENT ENTIRELY WHEN THERE IS NO SOURCE. Those are two different + # states and the publisher treats them as two: `mqtt.ABSENT_AS_UNKNOWN` + # turns the first into HA's own `None` payload, so the entity's state + # goes to `unknown` and the recorder records a GAP - which is how a + # reader of a week of history tells "no PV reported" from "PV reported + # 0 W". The second never reaches the broker at all, because an install + # that never set `pv_entity` has no such entity announced. + # + # ⚠️ AND THE GATE IS `optional_keys()`, THE SAME EXPRESSION THE + # ANNOUNCEMENT USES. Written as a second predicate here the two could + # disagree, and both directions are bad - an announced entity nothing + # feeds, or a topic nothing declared. `announced_units()` two blocks + # down is the same discipline for the roster. + if "pv" in optional_keys(self.o): + values["pv"] = (None if self.pv_w is None + else round(self.pv_w, 1)) # ⚠️ ONLY when P1 ingestion is actually running. The ESP32's stale-input # watchdog subscribes to sensor.goodwe_p1_sample_age - see the entity-id # note in mqtt.py; the composed name is not the object_id - and forces layer-1 @@ -4663,8 +4763,38 @@ class Controller: if not entity: out.append({"ok": False, "warn": False, "text": f"{label}: no entity configured"}) elif value is None: + # ⚠️ SAFETY-56. THIS ROW ASSERTED A FALSE FACT ABOUT A NAMED + # ENTITY, WHICH IS THE WORST THING A DIAGNOSTIC CAN DO. It read + # " is missing or not numeric" for ANY None, and one of + # the ways this value becomes None is SAFETY-37's freshness gate + # withholding a reading that is present, numeric and updating - + # `Freshness.reading()` returns None BY DESIGN there. On the + # live site on 2026-09-21 the operator opened Developer Tools on + # the entity this row named, found it working, and had every + # reason to conclude the diagnostics were broken. + # + # ⚠️ AND IT IS FIXED BY SAYING LESS, NOT BY LOOKING HARDER. The + # row could ask `inv.batt_fresh.stale` - it is on the same + # object at the same moment - but this loop also renders the + # grid and SoC rows, which have no gate, and a branch per row + # would be a second place where "why is this None" is decided. + # The gate already has a surface of its own: `stale_note()` + # rides `liveness_notes()` a few rows below with the whole + # STALE_READING sentence in it, so the operator's discriminator + # is "is that row there?" and this row's job is only to stop + # pre-empting the answer. Same reason the wording is not shared + # with `freshness._grade()`: that one is a log line about a + # BEHAVIOUR, this one is a banner about an ENTITY. out.append({"ok": False, "warn": False, - "text": f"{label}: {entity} is missing or not numeric"}) + "text": f"{label}: no usable value this cycle from " + f"{entity} - and this row cannot tell which " + f"of two things that is: the entity is " + f"missing or not numeric, or it is " + f"publishing a number that this controller " + f"is withholding (a freshness gate does " + f"exactly that, and names itself in a row " + f"of its own). Confirm the entity in " + f"Developer Tools before replacing it"}) else: out.append({"ok": True, "warn": False, "text": f"{label}: {entity} = {value:g}"}) @@ -4869,6 +4999,34 @@ class Controller: # is both is a check nobody reads. out.append({"ok": False, "warn": True, "text": "no peak forecast: maintenance charging is not capacity-capped"}) + # --- TEL-11 ---------------------------------------------------------- + # ⚠️ NO ROW AT ALL WHEN `pv_entity` IS UNSET, AND THAT IS A DELIBERATE + # DIFFERENCE FROM THE PEAK-FORECAST CAUTION DIRECTLY ABOVE. That one + # nags because an unset forecast entity leaves a MONEY GUARD without an + # input; this one is pure instrumentation, and a permanent caution on + # every install that never asked for a PV channel is exactly the + # "no warning spam, no degraded status row" the card forbids. + # + # ⚠️ AND WHEN IT IS SET, A FAILED READ IS `ok=False, warn=True` - a + # CAUTION, never `bad`. `status_payload()` builds "NOT READY" from rows + # that are neither ok nor warn, and a telemetry channel that could put a + # battery site into NOT READY would be observability with a foot in the + # control path. The reason comes from `pv.watts()` rather than being + # re-derived here, so the page cannot disagree with the reading. + if self.pv_entity: + if self.pv_w is None: + out.append({"ok": False, "warn": True, + "text": f"PV production: no reading from " + f"{self.pv_entity} - {self.pv_why}"}) + else: + # habit 14: the value sits inside its own literal text, in its + # own slot, beside the entity it was read from and the unit it + # was read in - so a transposition of any two of the three is + # visible in the sentence rather than merely present in it. + out.append({"ok": True, "warn": False, + "text": f"PV production: {self.pv_w:.0f} W from " + f"{self.pv_entity} (reported in " + f"{self.pv_why})"}) return out def status_payload(self) -> dict: @@ -4988,6 +5146,13 @@ class Controller: # identically zero and a permanent "0 W" row would be noise. *(() if len(self.inverters) <= 1 else ((("Circulating", w(self.site_circulating_w)),))), + # TEL-11, and ONLY when a source is configured - see `checks()`. + # `w(None)` renders "—", which is how this row says "no reading" + # rather than "0 W". It sits beside Grid and Battery because the + # question it answers is the one those two cannot: at "Grid 0 W, + # Battery 0 W" the site is either becalmed or throwing away every + # watt it makes, and only this row separates them. + *(() if not self.pv_entity else ((("PV", w(self.pv_w)),))), ("State of charge", "—" if self.pack_floor_soc is None else f"{self.pack_floor_soc:g} %"), ("Commanded", w(self.target)), @@ -5261,6 +5426,11 @@ async def amain() -> None: broker.get("username") if broker else None, broker.get("password") if broker else None, omit=() if is_enabled(opts) else ("p1_age",), + # TEL-11. ⚠️ THE SHIPPED PREDICATE, NOT A SECOND READING OF THE + # OPTION - `Controller.publish()` gates the row on this same + # call. An install that never set `pv_entity` announces exactly + # what it announced before: no topic, no entity, no row. + include=optional_keys(opts), # TEL-07. ⚠️ THE NAMES, NOT A ROSTER. `build_roster()` here # would construct a SECOND set of `Inverter`s off the same # Store, minutes before `Controller.__init__` constructs the diff --git a/goodwe_controller/app/mqtt.py b/goodwe_controller/app/mqtt.py index a3c1115..a121af2 100644 --- a/goodwe_controller/app/mqtt.py +++ b/goodwe_controller/app/mqtt.py @@ -117,6 +117,62 @@ SENSORS = [ ("loop_reason", "goodwe_loop_reason", "Loop reason", None, None, None, "mdi:sigma"), ] +# --- TEL-11: rows that exist ONLY when the operator configured a source ------ +# +# ⚠️ A SECOND LIST RATHER THAN A TENTH ENTRY IN `SENSORS`, AND THE DEFAULT IS +# WHAT MAKES IT ONE. `SENSORS` is announced by every caller of +# `discovery_payloads()`; `OPTIONAL_SENSORS` is announced only by a caller that +# ASKS, so a caller that has not been taught about `pv_entity` - `test_mqtt.py`'s +# fixtures, `tools/rehearse_roster.py`, anything written next year - announces +# exactly what it announced before. `p1_age` solves the same problem the other +# way round, with `omit`, because `p1_age` is announced by DEFAULT and withdrawn +# when the meter is off; this one does not exist by default and appears when a +# source for it does. Getting that direction wrong is the hazard `omit`'s own +# note names: an entity that exists and is never fed is a worse signal than one +# that does not exist at all, and MOST installs will never set `pv_entity`. +# +# ⚠️ `state_class: measurement` IS THE WHOLE POINT OF THE ROW. It is what puts +# the number into the recorder's long-term statistics, and a week of statistics +# is what OPT-10 needs to say whether the surplus that left this site was cloud, +# a charge-power limit, or the optimiser declining to charge. +OPTIONAL_SENSORS = [ + ("pv", "goodwe_pv_power", "PV power", "W", "power", "measurement", None), +] + +# ⚠️ HA's OWN "this sensor has no value" PAYLOAD, and the one place this add-on +# spells it. `homeassistant/components/mqtt/const.py` declares +# `PAYLOAD_NONE = "None"` and `mqtt/sensor.py` tests it BEFORE the numeric +# branch - `if payload == PAYLOAD_NONE: self._attr_native_value = None` - so the +# entity's state becomes `unknown` rather than a number. (Read out of +# github.com/home-assistant/core `dev` on 2026-09-24; the shipped MQTT-sensor +# documentation says the same thing in its `state_topic` entry: "a 'None' value +# will set the sensor to an unknown state".) +PAYLOAD_NONE = "None" + +# The keys whose ABSENCE MUST BE PUBLISHED rather than skipped. ⚠️ THIS IS THE +# DIFFERENCE BETWEEN "no PV reported" AND "PV reported 0 W" IN A WEEK OF +# HISTORY, AND WITHOUT IT THE CHANNEL LIES IN THE WORSE DIRECTION. `publish()` +# below drops a `None` - which is right for `circulation`, where the last figure +# standing is better than a zero that reads as health - but a DROPPED pv row +# leaves Home Assistant holding the last number it was told, forever, and the +# recorder then draws a flat line at that value through an outage. Someone +# reading it back would conclude the sun was not shining, which is precisely the +# conclusion this card exists to TEST rather than to manufacture. +ABSENT_AS_UNKNOWN = frozenset({"pv"}) + + +def optional_keys(opts: dict) -> tuple: + """Which `OPTIONAL_SENSORS` rows this configuration has a source for. + + ⚠️ ONE ANSWER TO "IS THE PV ROW LIVE", read by the ANNOUNCEMENT (through + `MqttPublisher`) and by `Controller.publish()`. `announced_units()` above + exists for exactly this reason and says it in full: two spellings of one + predicate is how an entity gets announced that nothing feeds, or fed on a + topic nothing announced. + """ + return ("pv",) if str(opts.get("pv_entity") or "").strip() else () + + BASE = "goodwe_ctl" AVAILABILITY = f"{BASE}/availability" @@ -299,15 +355,23 @@ def _payload(name, object_id, unique_id, state_topic, device, return cfg -def discovery_payloads(units=(), omit=()) -> list: +def discovery_payloads(units=(), omit=(), include=()) -> list: """Every `(topic, payload)` this add-on announces, in announcement order. Split out of `_announce()` so that the acceptance criterion can be graded as a DIFF OF THE ANNOUNCED PAYLOADS against `release/1.0`, rather than as a reading of the code. `_announce()` publishes exactly this list and nothing else, which is itself checked. + + `include` names `OPTIONAL_SENSORS` keys to announce. ⚠️ IT DEFAULTS TO + NOTHING AND THE ROWS GO LAST, so no existing row's POSITION moves under any + configuration - not the site rows, not the per-unit ones. Order is part of + the output here (`_payload()`'s note, and test_mqtt.py part 1 asserts an + ordered LIST precisely because a re-order is a zero-byte edit that + re-announces every entity on a live install). """ omit = set(omit) + include = [k for k in include if k not in omit] out = [] for key, object_id, name, unit, dev_class, state_class, icon in SENSORS: if key in omit: @@ -336,12 +400,19 @@ def discovery_payloads(units=(), omit=()) -> list: unique_id, f"{BASE}/{unit_state_key(slug, key)}", device, unit, dev_class, state_class, icon))) + for key, object_id, name, unit, dev_class, state_class, icon in OPTIONAL_SENSORS: + if key not in include: + continue + out.append(( + f"homeassistant/sensor/{BASE}_{key}/config", + _payload(name, object_id, f"{BASE}_{key}", f"{BASE}/{key}", + DEVICE, unit, dev_class, state_class, icon))) return out class MqttPublisher: def __init__(self, host, port, username=None, password=None, omit=(), - units=()): + units=(), include=()): # `omit` drops sensor keys from discovery entirely. ⚠️ Announcing a # sensor that nothing will ever publish to is not harmless here: # p1_sample_age_s is a watchdog input, and an entity that exists but is @@ -353,6 +424,10 @@ class MqttPublisher: # per-unit is announced, so every caller that has not been taught about # the roster keeps publishing what it published before. self.units = [str(u) for u in units] + # TEL-11. `mqtt.optional_keys(opts)` - the shipped predicate, not a + # second reading of the option. Defaults to empty, so a caller that has + # not been taught about it announces exactly what it announced before. + self.include = [str(k) for k in include] self.enabled = mqtt is not None and bool(host) self.client = None if not self.enabled: @@ -389,7 +464,8 @@ class MqttPublisher: # announcement, so the check that diffs it against `release/1.0` is # diffing what actually goes on the wire - and a payload built here # instead would be one this add-on publishes and no check has ever seen. - for topic, cfg in discovery_payloads(self.units, self.omit): + for topic, cfg in discovery_payloads(self.units, self.omit, + self.include): self.client.publish(topic, json.dumps(cfg), retain=True) self.client.publish(AVAILABILITY, "online", retain=True) @@ -399,7 +475,16 @@ class MqttPublisher: try: for key, value in values.items(): if value is None: - continue + # ⚠️ TEL-11. A DROPPED ROW IS NOT AN ABSENT ROW. Skipping + # leaves Home Assistant holding the last value it was told, + # which is what `circulation` wants (its note in main.py + # says why) and is exactly wrong for a measurement somebody + # will read back as history: a flat line at the last number + # is indistinguishable from a real one. See + # `ABSENT_AS_UNKNOWN`. + if key not in ABSENT_AS_UNKNOWN: + continue + value = PAYLOAD_NONE self.client.publish(f"{BASE}/{key}", str(value)) except OSError as err: # pragma: no cover _LOG.debug("MQTT publish failed: %s", err) diff --git a/goodwe_controller/app/pv.py b/goodwe_controller/app/pv.py new file mode 100644 index 0000000..e5b9288 --- /dev/null +++ b/goodwe_controller/app/pv.py @@ -0,0 +1,117 @@ +"""TEL-11 - the site's PV production, read from a Home Assistant entity. + +WHY THIS IS AN ENTITY AND NOT A REGISTER. This add-on's RS485 bus carries the +BATTERY inverter it drives. A household's PV inverter is, on nearly every site, +a different box from a different vendor that reports into Home Assistant +through its own integration - so the only place a product can read production +for an arbitrary install is an HA entity. `pv_entity` therefore takes whatever +`sensor.*` the site's own PV integration provides. + +WHY THE CHANNEL EXISTS AT ALL. OPT-10 measured that 20.5 kWh of surplus left the +site in 16 days while the pack had room, 12.6 kWh of it with the battery not +charging at all - and could not say WHY, because PV production is recorded +NOWHERE. Three causes need three different fixes (no irradiance: nothing to fix; +a charge-power limit; the optimiser declining to charge - the only software +defect), and guessing between them is how an optimiser gets tuned to solve a +cloud problem. This module is the missing measurement and NOTHING ELSE. + +⚠️ NOTHING HERE MAY EVER REACH A CONTROL DECISION, AND THAT IS THE CARD'S FIRST +CONSTRAINT RATHER THAN A STYLE NOTE. The add-on regulates a real battery; adding +production as a term of the control law would be an unrequested behaviour change +AND it would destroy the measurement, because the loop would then be reacting to +the quantity being measured. `tools/tel11_pv_trace.py` is what says the path is +untouched: it grades a control document that contains none of this material on +both trees and with `pv_entity` set and unset. + +⚠️ ABSENCE IS NEVER ZERO, AND THAT IS THE SECOND CONSTRAINT. Every refusal below +returns `None` with a REASON, never `0.0`. A PV channel that reported 0 W for a +dead sensor would manufacture exactly the "there was no surplus to store" +conclusion the card exists to TEST - and it would do it silently, in a week of +recorder history nobody would re-derive. `app/hass.py`'s own note on `state()` +is the same rule one layer down. + +THE SIGN CONVENTION: POSITIVE IS PRODUCTION. Derived from the two conventions +this add-on already ships rather than assumed: + + `meter_entity` positive = IMPORTING from the grid (DOCS.md, Sources) + `batt_entity` positive = DISCHARGING the pack (DOCS.md, Sources) + +Both are "positive = power arriving at the house's AC bus", and production +arrives there too. There is deliberately NO `pv_invert` to go with +`meter_invert` and `batt_invert`: those two quantities are BIPOLAR and are READ +BY THE CONTROL PATH - a reversed `batt_entity` feeds SAFETY-30's per-unit sign +check and SAFETY-31's circulating-power sensor with a plausible number the site +then acts on - whereas production is unipolar and is read by nothing. A reversed +PV sensor announces itself as -4 kW at noon in the very history this card +creates, and flipping it here would only hide it. See +`DECISIONS/TEL-11-pv-production.md` §2. +""" + +from .hass import BAD_STATES + +# Home Assistant's OWN power units, each as the number of watts one of it is. +# ⚠️ READ OUT OF HA, NOT LISTED FROM MEMORY: `UnitOfPower` in +# `homeassistant/const.py` (github.com/home-assistant/core, `dev`, retrieved +# 2026-09-24) declares mW, W, kW, MW, GW, TW and BTU/h. Every factor here is an +# exact power of ten, so none of them is a number anybody chose. +# +# ⚠️ BTU/h IS IN HA's ENUM AND DELIBERATELY NOT HERE. Its factor (0.293071…) +# is a measured conversion rather than a decimal shift, no PV integration +# publishes in it, and a wrong constant in the one channel that exists to +# attribute a 20.5 kWh shortfall is worse than a refusal an operator can read. +# It lands in the "not a power unit" refusal below, BY NAME. +WATTS_PER = { + "mW": 0.001, + "W": 1.0, + "kW": 1000.0, + "MW": 1000000.0, + "GW": 1000000000.0, + "TW": 1000000000000.0, +} + +# What `watts()` answers when there is nothing to report. ⚠️ THE REASON IS PART +# OF THE RETURN VALUE, not a log line: it is what the status page prints, and an +# absence whose cause is invisible sends an operator to look at the wrong box. +UNIT_KEY = "unit_of_measurement" + + +def watts(state: dict | None) -> tuple: + """`(watts, why)` from a Home Assistant state object. + + `watts` is `None` whenever there is no reading - and `why` then says which + of the five refusals it was, in words an operator can act on. On a good read + `why` is the unit the value was read in, because "1234 W" and "1234, unit + unknown" are different claims and the page has to be able to make the first. + + ⚠️ THE UNIT IS READ OFF THE ENTITY AND NOT ASSUMED, AND THERE IS NO + MAGNITUDE HEURISTIC. `read_inputs()` guesses kW-or-W for + `peak_forecast_entity` with `abs(raw) < 50`, which is safe there because + nobody's quarter-hour demand forecast is 50 W. It is NOT safe here: 30 W of + real production at dusk is an ordinary reading, and that heuristic would + publish it as 30 kW - a fabricated number, in the channel whose entire + purpose is to be believed a week later. A sensor whose unit this cannot read + is REFUSED rather than guessed at. + """ + if state is None: + # `hass.state()` answers None for a non-existent entity, a 404 and a + # transport failure alike. All three are "no reading", and HA does not + # consider a non-existent entity an error - see `hass.state()`. + return None, "no state (entity missing, or the read failed)" + raw = str(state.get("state", "")).strip() + if raw.lower() in BAD_STATES: + # ⚠️ THE SAME LIST `hass.number()` REFUSES ON, IMPORTED RATHER THAN + # RESPELLED. A second copy of it is how one path starts treating + # `unavailable` as a number the other one rejects. + return None, f"entity is {raw or 'empty'}" + try: + value = float(raw) + except ValueError: + return None, f"state {raw!r} is not numeric" + unit = str((state.get("attributes") or {}).get(UNIT_KEY) or "").strip() + if not unit: + return None, ("no unit_of_measurement on the entity - it cannot be " + "read in watts") + if unit not in WATTS_PER: + return None, (f"unit {unit!r} is not one of " + f"{', '.join(WATTS_PER)} - it cannot be read in watts") + return value * WATTS_PER[unit], unit diff --git a/goodwe_controller/config.yaml b/goodwe_controller/config.yaml index f2aa493..5ad4e0c 100644 --- a/goodwe_controller/config.yaml +++ b/goodwe_controller/config.yaml @@ -1,5 +1,5 @@ name: GoodWe RS485 Controller -version: "0.10.1" +version: "0.11.0" slug: goodwe_controller description: >- Drives a GoodWe ES/BP battery inverter over RS485 by emulating its smart @@ -147,6 +147,33 @@ options: inverters: [] estop_fitted: false + # TEL-11 - THE SITE'S PV PRODUCTION, IN WATTS, AND IT IS READ BY NOTHING THAT + # DECIDES ANYTHING. Optional, empty on every install that does not set it, and + # when it is empty the add-on reads no entity, announces no entity, publishes + # no topic and shows no status row. + # + # ⚠️ AN HA ENTITY AND NOT A REGISTER, BECAUSE MOST SOLAR INVERTERS ARE NOT ON + # THIS BUS. The RS485 bus this add-on speaks carries the BATTERY inverter it + # drives; a household's PV inverter is normally a different box from a + # different vendor, reporting into Home Assistant through its own integration. + # So this takes whatever `sensor.*` the site's PV integration provides, any + # brand Home Assistant can read. + # + # ⚠️ POSITIVE IS PRODUCTION, and there is deliberately no `pv_invert` beside + # `meter_invert` and `batt_invert`. Those two quantities are bipolar AND are + # read by the control path, so a reversed sensor there is a plausible number + # the site acts on; production is unipolar and is read by nothing, so a + # reversed sensor announces itself as -4 kW at noon in the history this key + # exists to create. See DECISIONS/TEL-11-pv-production.md. + # + # ⚠️ THE ENTITY'S OWN `unit_of_measurement` IS WHAT IS BELIEVED - W, kW, MW + # and the rest of Home Assistant's power units. A sensor with no unit, or with + # a unit that is not power, is REFUSED and the status page says which: there + # is no magnitude heuristic that can tell 30 W of dusk from 30 kW of noon, and + # a factor of 1000 in the one channel that exists to be believed a week later + # is worse than a refusal an operator can read. + pv_entity: "" + # ========================================================================== # METER - where the grid reading comes from # ========================================================================== @@ -872,6 +899,12 @@ schema: capacity_kwh: float(0.1,200)? estop_fitted: bool + # TEL-11. Optional, and OPTIONAL WITHOUT A COMPANION RULE - unlike + # `price_now_entity`/`price_avg_entity`, which are legal only while + # `price_provider` is `off` and make the add-on REFUSE TO START otherwise. + # Nothing in validate.py mentions this key: it cannot conflict with anything, + # because nothing else reads it. + pv_entity: str? # ========================================================================== # METER - where the grid reading comes from diff --git a/goodwe_controller/test_freshness.py b/goodwe_controller/test_freshness.py index d7c4a6b..e04f6ab 100644 --- a/goodwe_controller/test_freshness.py +++ b/goodwe_controller/test_freshness.py @@ -35,6 +35,12 @@ module does honour. 7. THE POLICY TABLE tells the truth about what is wired. 8. THE MEASUREMENT the bound rests on, replayed from the committed capture. 9. KNOWN SURVIVORS, graded so they are residuals rather than surprises. + 10. SAFETY-57's EXIT from the verdict, against the committed live capture. + 11. SAFETY-56: THE SENTENCE THIS GATE CAN HONESTLY SAY. The conjunction is + satisfied by two disjoint worlds - a register that has stopped reporting, + and a source that cannot comply - and nothing here separates them. Part + 11 builds both, shows the verdict is byte-identical across them, and + grades the three operator-facing sentences against that ambiguity. ⚠️ INVERTER B DOES NOT PHYSICALLY EXIST (~2026-09-12; goodwe/NOTES.md §1.1b). Every two-unit case below is two copies of inverter A's behaviour with @@ -1608,12 +1614,17 @@ check("...and the 35.0 s the register is re-graded over is named in the note " "too, because it is what the operator is being told to wait out", f"re-graded over a fresh {BOUNDS['batt'].unchanged_s:.0f} s window" in _rel57[0]) -check("⚠️ AND THE DIAGNOSIS HALF OF THE STALE LINE IS UNTOUCHED, WHICH IS " - "SAFETY-56'S. That card is rewriting 'is not following its own command' " - "- the live episode's register was following perfectly and the pack was " - "simply empty - and two cards editing one sentence is how one of them " - "silently loses. This check is here so that rewrite is a deliberate act", - "battery-power register is not following its own command" in _stale57[0]) +check("⚠️ AND THE DIAGNOSIS HALF IS SAFETY-56'S AND HAS NOW BEEN REWRITTEN, " + "WHICH IS WHAT THIS CHECK WAS FOR. It used to require 'battery-power " + "register is not following its own command' to still be present, so " + "that removing it had to be a deliberate act rather than a side effect " + "of editing the escape clause beside it. SAFETY-56 is that act, and the " + "requirement is now the inverse: the accusation is GONE from this line " + "and part 11 grades what replaced it. The escape clause above is " + "SAFETY-57's and is untouched by it", + "battery-power register is not following its own command" + not in _stale57[0] + and "did not move under a command that did" in _stale57[0]) check("...and a re-trip's reason names the TWO durations separately, each " "beside its own value, because after a release the verdict's window and " "the register's own unchanged time are different facts", @@ -1630,6 +1641,335 @@ check("...and a re-trip's reason names the TWO durations separately, each " Freshness(BOUNDS["batt"]))) +# ============================================================================== +print() +print("11. SAFETY-56: the sentence this gate can honestly say") +# ⚠️ THE SUBJECT OF THIS PART IS A CLAIM, NOT A STRING, AND NOTHING IN IT +# CHANGES A DECISION. SAFETY-37's conjunction is correct and is untouched; +# what was wrong is what two sentences CONCLUDED from it. The conjunction is +# satisfied by two disjoint worlds - +# +# DEAD the register has stopped reporting reality. The pack really is +# doing what it was told; the published number does not say so. +# REFUSED the register is honest and the SOURCE cannot comply. The BMS is +# refusing at the pack floor, so the pack does nothing whatever the +# command asks, and an unchanged reading is the TRUTH. +# +# - and nothing this module is handed separates them. Three sentences named +# DEAD as fact: `freshness._grade()`'s "the reading is not following", +# `main.py`'s log line "battery-power register is not following its own +# command", and the NOT READY row's " is missing or not numeric". On +# the live 2026-09-21 episode all three were the wrong one of the two, and the +# third was the worst because it asserts a false fact about a NAMED entity: the +# operator opens Developer Tools, finds it working, and concludes the +# diagnostics are broken. +# +# So the property graded below is not "the new words are present". It is that +# the two worlds are REAL AND DIFFERENT (11a on the committed capture, 11b in a +# driven pair), that the gate CANNOT separate them (11b), and therefore that a +# clause asserting either one is false half the time - which is what makes 11c +# and 11d a claim rather than decoration. Habit 4: without 11a and 11b those +# two would be substring tests over a sentence this file also wrote. + +# --- 11a. the old diagnosis was FALSE on the site it was printed on ---------- +# ⚠️ OFF THE COMMITTED CAPTURE, NOT OFF A FIXTURE. Part 10a has already graded +# this file's bytes against the manifest and its decoder's sign convention +# against the frames, so the three series below are evidence rather than a +# story. Every boundary is derived in part 10b from the register's own longest +# hold; nothing here is typed. +_S56_CMDS_HELD = sorted({_D.held_at(_sp, t) + for t in range(int(_HOLD_FROM), int(_PARK))}) +_S56_SOC_HELD = sorted({_D.held_at(_soc, t) + for t in range(int(_HOLD_FROM), int(_HUMAN))}) +_S56_CHARGE_T, _S56_CHARGE_W = next((t, v) for t, v in _sp + if t > _PARK and v < 0.0) +_S56_AFTER = _D.held_at(_bt, _REG_MOVES + 5.0) +print(f" during the 850 s hold: commands {_S56_CMDS_HELD} W, " + f"SoC {_S56_SOC_HELD} %, register {_HOLD_W:+.0f} W") +print(f" first CHARGE command {hms(_S56_CHARGE_T)} at " + f"{_S56_CHARGE_W:+.0f} W -> register moves {hms(_REG_MOVES)} " + f"({_REG_MOVES - _S56_CHARGE_T:.0f} s) to {_S56_AFTER:+.0f} W") +check("11a. ⚠️ THE OLD DIAGNOSIS WAS FALSE ON THE ONE SITE IT WAS EVER " + "PRINTED ON. Through the whole hold the site asked for DISCHARGE only " + "(+260 to +1060 W) out of a pack reading 11 %, and the register answered " + "-3 W. Then the first CHARGE command arrives and it moves within ONE " + "17.0 s obligation window. A register that answers a reversed command in " + "one window had not stopped reporting; what would not move was the PACK", + all(v > 0.0 for v in _S56_CMDS_HELD) + and min(_S56_CMDS_HELD) == 260.0 and max(_S56_CMDS_HELD) == 1060.0 + and _S56_SOC_HELD == [11.0] and _HOLD_W == -3.0 + and _S56_CHARGE_W < 0.0 + and 0.0 < _REG_MOVES - _S56_CHARGE_T <= F.BATT_OBLIGED_S + and _S56_AFTER != _HOLD_W) +check("...and the gate was RIGHT to fire anyway, which is why this card touches " + "no term of the conjunction. The three limits are still the derived ones " + "and the same capture still arms at 10:52:00 with the verdict standing: " + "a register genuinely stuck against a real command is exactly what " + "SAFETY-37 should catch, and a pack that cannot comply is a state the " + "site must stop in too. What was wrong was never the firing", + BOUNDS["batt"].unchanged_s == BATT_UNCHANGED_S + and BOUNDS["batt"].move_w == MOVE_W + and BOUNDS["batt"].release_s == F.BATT_OBLIGED_S + and hms(_TRIP) == "10:52:00" and _f_after.releases == 1 + and next(r for r in _rows_after if r[0] == _TRIP)[1] is True) + +# --- 11b. two worlds, one published stream, one verdict ---------------------- +# ⚠️ GROUND TRUTH IS WRITTEN DOWN, NOT DERIVED FROM THE PUBLISHED STREAM +# (habit 2). A check that computes "what the pack was really doing" out of the +# number the register published is comparing a thing with itself, which is the +# defect MUTATION.md has recorded fifteen times. +_BATT_ENT = "sensor.batt_a" +_S56_N = 40 +_S56_CMD = [800.0 if t < 1 else 1600.0 for t in range(_S56_N)] +_S56_PUB = [-3.0] * _S56_N # what the ENTITY publishes, both +_S56_TRUE_DEAD = list(_S56_CMD) # the pack obeyed; the register lied +_S56_TRUE_REFUSED = [-3.0] * _S56_N # the pack refused; the register told + + +async def _s56_world(soc_pct): + """The shipped read path at N=1. The ground-truth stream above is never + handed to the controller - it is what the WORLD is, not an input.""" + c, hass = one_site() + hass.numbers["sensor.soc_a"] = soc_pct + for t in range(_S56_N): + hass.numbers[_BATT_ENT] = _S56_PUB[t] + c.inverters[0].last_write = _S56_CMD[t] + await c.read_inputs(now_mono=float(t)) + return c, hass + + +_S56_LOG0 = len(_LOGREC) +# ⚠️ THE TWO WORLDS ARE DRIVEN AT DIFFERENT SoC ON PURPOSE - 50 % against the +# 11 % the live pack sat at - so that "the sentence did not move" below is a +# measurement of this module's `soc` bound being STATED, NOT WIRED, and not an +# artefact of both runs being handed the same percentage. Wiring it is TEL-08's. +_c56_dead, _h56_dead = run(_s56_world(50.0)) +_c56_ref, _h56_ref = run(_s56_world(11.0)) +_pub_dead = [v for e, v in _h56_dead.served if e == _BATT_ENT] +_pub_ref = [v for e, v in _h56_ref.served if e == _BATT_ENT] +_S56_DIFFER = sum(1 for a, b in zip(_S56_TRUE_DEAD, _S56_TRUE_REFUSED) if a != b) +print(f" DEAD truth {_S56_TRUE_DEAD[:2]}... published {_pub_dead[:2]}...") +print(f" REFUSED truth {_S56_TRUE_REFUSED[:2]}... published {_pub_ref[:2]}...") +check(f"11b. PRECONDITION: the two worlds are REALLY TWO - their ground truth " + f"differs on {_S56_DIFFER} of {_S56_N} reads - and the PUBLISHED stream " + f"is bit-identical in both, so nothing downstream of {_BATT_ENT} can " + f"tell them apart. Without this the rest of part 11 is one world twice", + _S56_DIFFER == _S56_N and _pub_dead == _pub_ref == _S56_PUB + and len(_pub_dead) == _S56_N) +check("...and in REFUSED the register is RIGHT on every one of the 40 reads " + "(published == truth) while in DEAD it is wrong on all 40. That is the " + "fact the old sentence got backwards: it named the register in a world " + "where the register is the only thing telling the truth", + _S56_PUB == _S56_TRUE_REFUSED + and not any(p == t for p, t in zip(_S56_PUB, _S56_TRUE_DEAD))) +_r56_dead = _c56_dead.inverters[0].batt_fresh.reason +_r56_ref = _c56_ref.inverters[0].batt_fresh.reason +check("⚠️ AND THE GATE FIRES IDENTICALLY IN BOTH, WHICH IS THE WHOLE REASON " + "ITS SENTENCE MAY NOT NAME ONE OF THEM: same verdict, same MISSING " + "reading, same site stop - and the sentence is BYTE-IDENTICAL across two " + "worlds that share no ground truth at all", + _c56_dead.inverters[0].batt_fresh.stale is True + and _c56_ref.inverters[0].batt_fresh.stale is True + and _c56_dead.inverters[0].batt is None + and _c56_ref.inverters[0].batt is None + and _c56_dead.inputs_ok is False and _c56_ref.inputs_ok is False + and _r56_dead == _r56_ref and _r56_dead != "") +check("...and the two runs were driven at 50 % and 11 % SoC and the sentence " + "did not move one byte, because `BOUNDS['soc']` is STATED, NOT WIRED. A " + "message honest about its own ambiguity needs no new input, and adding " + "one here would make this sentence depend on a second measurement being " + "right - TEL-08's card, not this one", + _c56_dead.inverters[0].soc == 50.0 and _c56_ref.inverters[0].soc == 11.0 + and _r56_dead == _r56_ref) + +# --- 11c. both explanations, and no accusation, on every surface ------------- +# The two explanations, each tagged with the world it is true OF. The emitted +# sentence is the same in both worlds (11b), so a sentence naming only one is +# false half the time - which is why BOTH are required, as an ORDERED PAIR +# (habit 14): two clauses in one sentence can always be transposed at zero +# bytes, and "the register may be unable to comply / the source may have +# stopped reporting reality" is a different and wrong claim. +_S56_EXPLANATIONS = [ + ("DEAD", "the register may have stopped reporting reality"), + ("REFUSED", "the source may be unable to comply with the command"), +] +# ⚠️ WHAT THE THREE SENTENCES USED TO ASSERT. Each is a claim about the +# REGISTER or about the ENTITY, and 11a measured the register to be alive on +# the live episode while 11d measures the entity to be serving a float. They +# are listed with the file they came from so a reviewer can see the set is the +# whole set rather than the ones that were easy to find. +_S56_ACCUSATIONS = [ + ("app/freshness.py _grade()", "the reading is not following"), + ("app/main.py _read_unit()", "register is not following its own command"), + ("app/main.py checks()", f"{_BATT_ENT} is missing or not numeric"), +] +_log56 = [r.getMessage() for r in _LOGREC[_S56_LOG0:] if "SAFETY-37" in r.getMessage()] + + +def _s56_row(c): + """The battery-power readiness row, or a NAMED absence - never a raise. + + ⚠️ IT MUST NOT RAISE, AND THAT IS FCST-06'S RULE RATHER THAN TIDINESS. The + first draft of this part was `[... ][0]`, and the mutation that transposes + `label` and `entity` in the row - zero bytes - makes the row unfindable. The + run then died on an IndexError with every check after this line UNGRADED: + a red with no name attached, which reads like broken infrastructure and gets + retried rather than read. Measured: 0 named reds before, 2 after. + """ + hits = [r for r in c.checks() if r["text"].startswith("battery power")] + if len(hits) != 1: + return {"ok": None, "warn": None, + "text": f""} + return hits[0] + + +_row56_ref = _s56_row(_c56_ref) +_S56_SURFACES = { + "reason (status page + action log)": _r56_ref, + "log line (SAFETY-37 ERROR)": _log56[-1] if _log56 else "", + "stale_note() (NOT READY row)": _c56_ref.inverters[0].stale_note(), + "checks() battery-power row": _row56_ref["text"], +} +for _n, _s in _S56_SURFACES.items(): + print(f" {_n}: {_s[:96]}...") +check("PRECONDITION: the readiness row is FINDABLE BY ITS LABEL in both worlds " + "- exactly one `battery power` row each - so every row graded below is " + "that row, and a mutation that moves `label` out of the first slot is a " + "NAMED red here rather than an IndexError three checks later (FCST-06). " + f"REFUSED: {_row56_ref['text'][:44]!r}", + _row56_ref["ok"] is not None) +check("PRECONDITION: all four operator-facing surfaces were really produced by " + "this run and are non-empty, so the two quantifiers below are not over " + "an empty set - and the log line is the one the SHIPPED read path " + "emitted, not one this file formatted. Both worlds emitted one and the " + "two are the same string, which is 11b again on the log surface", + len(_log56) == 2 and _log56[0] == _log56[1] + and all(len(s) > 30 for s in _S56_SURFACES.values()) + and _r56_ref in _log56[-1]) +_idx56 = [_r56_ref.find(t) for _w, t in _S56_EXPLANATIONS] +check(f"11c. BOTH EXPLANATIONS ARE NAMED, AND IN THAT ORDER: " + f"{_S56_EXPLANATIONS[0][1]!r} at {_idx56[0]} then " + f"{_S56_EXPLANATIONS[1][1]!r} at {_idx56[1]}. Pinned as an ordered pair " + f"because transposing 'register' and 'source' between the two clauses is " + f"a zero-byte edit that leaves both words in the sentence (habit 14)", + all(i >= 0 for i in _idx56) and _idx56 == sorted(_idx56) + and len(set(_idx56)) == len(_idx56)) +check("...and the sentence says in its own words that it CANNOT choose, rather " + "than leaving two explanations side by side for a reader to rank - an " + "operator who reads a list picks the first item", + "CANNOT TELL THEM APART" in _r56_ref) +# ⚠️ THE POSITIVE CONTROL FOR THE `not any(...)` BELOW, AND IT IS NOT OPTIONAL: +# a quantifier over an empty list is True, and so is one over three literals +# with a typo in each. These are the three sentences as 0.10.1 SHIPPED them, +# quoted from the release rather than rebuilt, and the same predicate is +# required to FIND an accusation in every one of them (habit 7: vary one thing +# from an ACCEPTED baseline). +_S56_SHIPPED_010_1 = { + "reason": "STALE_READING: -3 unchanged for 35 s while the standing command " + "moved 800 W to +1600 W - the reading is not following, so it is " + "reported as MISSING rather than as a measurement (limits: " + "unchanged over 35 s, command over 250 W and moved by over 250 W)", + "log": "SAFETY-37: inverter-a battery-power register is not following its " + "own command - STALE_READING: ... Treating it as MISSING, which " + "commands 0 W.", + "row": f"battery power: {_BATT_ENT} is missing or not numeric", +} +check(f"POSITIVE CONTROL: the very same predicate, run against the three " + f"sentences as 0.10.1 SHIPPED them, finds an accusation in EVERY one - " + f"and every one of the {len(_S56_ACCUSATIONS)} literals matches at least " + f"one of them, so neither an empty list nor a mistyped literal can make " + f"the check below pass by vacuity", + len(_S56_ACCUSATIONS) == 3 and len(_S56_EXPLANATIONS) == 2 + and all(any(t in old for _w, t in _S56_ACCUSATIONS) + for old in _S56_SHIPPED_010_1.values()) + and all(any(t in old for old in _S56_SHIPPED_010_1.values()) + for _w, t in _S56_ACCUSATIONS)) +check("⚠️ AND NOT ONE OF THE THREE ACCUSATIONS SURVIVES, ON ANY OF THE FOUR " + "SURFACES: " + "; ".join(f"{w} -> {t!r}" for w, t in _S56_ACCUSATIONS) + + ". This is the check that goes red if any of the three sentences is " + "reverted, and it is stated over the SURFACES rather than over the " + "source files because two of the three are assembled from two modules", + not any(t in s for _w, t in _S56_ACCUSATIONS + for s in _S56_SURFACES.values())) + +# --- 11d. the NOT READY row that named a working entity ---------------------- +async def _s56_absent(): + """The other half of the row's ambiguity: the entity really is gone.""" + c, hass = one_site() + for t in range(3): + hass.numbers[_BATT_ENT] = None # Home Assistant has no state + c.inverters[0].last_write = 0.0 + await c.read_inputs(now_mono=float(t)) + return c, hass + + +_c56_abs, _h56_abs = run(_s56_absent()) +_last_ref = [v for e, v in _h56_ref.served if e == _BATT_ENT][-1] +_last_abs = [v for e, v in _h56_abs.served if e == _BATT_ENT][-1] +_row56_abs = _s56_row(_c56_abs) +print(f" WITHHELD: {_BATT_ENT} served {_last_ref!r}, stale=" + f"{_c56_ref.inverters[0].batt_fresh.stale} ABSENT: served " + f"{_last_abs!r}, stale={_c56_abs.inverters[0].batt_fresh.stale}") +check("11d. PRECONDITION: the row's two causes are both REACHED. In REFUSED " + f"{_BATT_ENT} served a real float on the last read and the gate is " + f"holding the value back; in ABSENT Home Assistant served None and the " + f"gate is not armed at all. Two different causes, one `inv.batt is None`", + _last_ref == -3.0 and _last_abs is None + and _c56_ref.inverters[0].batt_fresh.stale is True + and _c56_abs.inverters[0].batt_fresh.stale is False + and _c56_ref.inverters[0].batt is None + and _c56_abs.inverters[0].batt is None) +check("⚠️ AND THE VERDICT IS UNCHANGED BY THIS CARD - both rows are still " + "ok=False, warn=False, i.e. still NOT READY. The defect was never the " + "pass/fail, it was the row asserting a fact about a named entity that " + "was at that moment present, numeric and updating", + _row56_ref["ok"] is False and _row56_ref["warn"] is False + and _row56_abs["ok"] is False and _row56_abs["warn"] is False + and _row56_ref["text"] == _row56_abs["text"]) +check("...and what it says instead is the observation plus BOTH causes, with " + f"{_BATT_ENT} in its own slot - `label` and `entity` are two " + f"interpolations in one sentence and swapping them costs zero bytes, so " + f"the literal around each is pinned (habit 14)", + f"battery power: no usable value this cycle from {_BATT_ENT} - and this " + f"row cannot tell which of two things that is" in _row56_ref["text"] + and "the entity is missing or not numeric" in _row56_ref["text"] + and "a number that this controller is withholding" in _row56_ref["text"]) +check("⚠️ AND THE DISCRIMINATOR IT POINTS AT EXISTS AND IS SELECTIVE, which is " + "why the row does not need to ask `batt_fresh.stale` itself: `checks()` " + "carries a STALE_READING row in REFUSED and NONE in ABSENT, so 'is that " + "row below?' answers the question this one declines to answer. A row " + "that pointed at a surface present in both states would be worse than " + "the accusation it replaced", + any("STALE_READING" in r["text"] for r in _c56_ref.checks()) + and not any("STALE_READING" in r["text"] for r in _c56_abs.checks())) +check("...and the row's wording is NOT shared with the log line's, which is a " + "design choice rather than an omission: the two sentences have no clause " + "in common longer than the word 'entity', because one explains a " + "BEHAVIOUR to a log reader and the other tells a page reader whether to " + "trust a NAME. A shared helper would have to drop whichever half the " + "other caller does not need", + _S56_EXPLANATIONS[0][1] not in _row56_ref["text"] + and _S56_EXPLANATIONS[1][1] not in _row56_ref["text"] + and "no usable value this cycle" not in _r56_ref) + +# --- 11e. what part 11 does NOT grade, with its shape (habit 13) ------------- +# * It does not show the live episode was a BMS refusal beyond doubt. 11a +# measures that the register answered a reversed command inside one +# obligation window, which excludes a register that had simply stopped; it +# does not exclude one that was intermittently wrong. That is the point: the +# sentence no longer claims to know either. +# * It does not grade ENGLISH. A wording that named both explanations and then +# said something useless about them would pass 11c. +# * The `no entity configured` branch beside the rewritten row is untouched and +# ungraded here - it has no second explanation, because a blank entity really +# is the only thing it can be. +# * The grid-power and battery-SoC rows go through the same rewritten line and +# have no freshness gate today, so for them the second cause is a hypothesis +# about this controller rather than a live possibility. That is weaker than +# the battery-power case and is the price of not branching per row; a gate on +# either of those signals (TEL-08 would be one) makes it exact. + + print() if fails: print(f"{len(fails)} FAILED: {', '.join(fails)}") diff --git a/goodwe_controller/test_mqtt.py b/goodwe_controller/test_mqtt.py index 5d44b92..7ea360a 100644 --- a/goodwe_controller/test_mqtt.py +++ b/goodwe_controller/test_mqtt.py @@ -203,15 +203,21 @@ class FakeClient: self.msgs.append((topic, payload, retain)) -def announce(units=(), omit=(), module=mqtt): +def announce(units=(), omit=(), module=mqtt, include=()): """The messages `_announce()` publishes, through the SHIPPED method. ⚠️ Not `discovery_payloads()` directly. Part 2 proves the two agree; every other part goes through this one, so what is diffed is what paho is told. + + `include` is TEL-11's optional-row set and DEFAULTS TO EMPTY here for the + same reason it does in the shipped constructor: every check in this file + written before that key existed must go on grading the announcement an + install that never set `pv_entity` receives, which is all of them. """ pub = module.MqttPublisher.__new__(module.MqttPublisher) pub.omit = set(omit) pub.units = [str(u) for u in units] + pub.include = [str(k) for k in include] pub.enabled = True pub.client = FakeClient() pub._announce() @@ -316,6 +322,52 @@ check("...and the `omit` path is unchanged too: with P1 ingestion off the " "watchdog input is still not announced, and nothing else moved", (m, c, a) == ([], [], [])) +# --- TEL-11 ------------------------------------------------------------------ +# ⚠️ THE OPTIONAL ROW'S DEFAULT IS THE CLAIM, AND IT IS THE OPPOSITE DEFAULT +# FROM `omit`'s. `p1_age` is announced unless withdrawn; `pv` does not exist +# unless asked for. Every check above this line passes `include` nothing, so +# they are all grading the announcement an install that never set `pv_entity` +# receives - which is MOST installs, and is the acceptance criterion "unset +# behaves exactly as today". +check("⭐ TEL-11 ADDS NOTHING AT THE SHIPPED DEFAULT: a publisher that is told " + "nothing about `pv_entity` announces the same list, in the same order, " + "with the same bytes - so an install that never sets the option sees no " + "new entity, no new topic and no re-announcement of an old one", + announce(units=roster_names({})) == SITE_NOW) +_PV_TOPIC = "homeassistant/sensor/goodwe_ctl_pv/config" +_PV_PAYLOAD = ('{"name": "PV power", "object_id": "goodwe_pv_power", ' + '"unique_id": "goodwe_ctl_pv", "state_topic": "goodwe_ctl/pv", ' + '"availability_topic": "goodwe_ctl/availability", "device": ' + + _DEV + ', "unit_of_measurement": "W", "device_class": ' + '"power", "state_class": "measurement"}') +_pv_one = announce(units=roster_names({}), include=("pv",)) +_m2, _c2, _a2 = diff(_pv_one, SITE_NOW) +check("⭐ AND WITH `pv_entity` SET IT IS EXACTLY ONE NEW TOPIC AND NOTHING " + "ELSE MOVES: same eleven configs first, same bytes, same order - the " + "additions this card makes are additions, not a re-composition", + (_m2, _c2) == ([], []) and _a2 == [_PV_TOPIC] + and _pv_one[:len(SITE_NOW) - 1] == SITE_NOW[:-1]) +check("...and the payload is the site device, in watts, with " + "`state_class: measurement` - which is what puts it in the recorder's " + "statistics, and a week of statistics is the entire point of the card", + (_PV_TOPIC, _PV_PAYLOAD, True) in _pv_one) +# ⚠️ AND IT GOES LAST, WHICH IS WHY A ROSTER OF TWO IS ASKED THE SAME QUESTION. +# Placed among the site rows it would shift every per-unit entity's position in +# the retained announcement - a zero-byte edit that re-announces inverter B. +_pv_two = announce(units=NAMES2, include=("pv",)) +check("...and at a roster of two it is STILL the last config, after every " + "per-unit row, so no existing entity's position in the retained " + "announcement moves when the option is turned on", + [t for t, _p, _r in _pv_two if t.startswith("homeassistant/")][-1] + == _PV_TOPIC + and _pv_two == announce(units=NAMES2)[:-1] + [ + (_PV_TOPIC, _PV_PAYLOAD, True)] + announce(units=NAMES2)[-1:]) +check("...and `omit` still wins over `include`, so the two arguments cannot " + "disagree about one key", + [t for t, _p, _r in announce(units=roster_names({}), include=("pv",), + omit=("pv",)) + if t == _PV_TOPIC] == []) + # --------------------------------------------------------------------------- print("\n2. what is diffed is what goes on the wire") # --------------------------------------------------------------------------- diff --git a/goodwe_controller/test_pv.py b/goodwe_controller/test_pv.py new file mode 100644 index 0000000..6939fc7 --- /dev/null +++ b/goodwe_controller/test_pv.py @@ -0,0 +1,643 @@ +"""Runnable check for TEL-11 - the PV production channel. `python3 test_pv.py` + +OPT-10 measured that this site's battery reaches 90 % SoC on 2 days of 16 and +that 20.5 kWh of surplus left it while the pack had room - 12.6 kWh of that with +the battery not charging at all - and could not say WHY, because PV production +is recorded NOWHERE. Three causes need three different fixes: no irradiance +(nothing to fix), a charge-power limit, or the optimiser declining to charge +(the only software defect). This card adds the missing channel. It does not do +the attribution; that is OPT-10's, with a week of history behind it. + +⚠️ WHAT THIS FILE REFUSES TO ASSERT, SAID FIRST. It does not grade that a row +EXISTS. `publish()` handing a dict with `pv` in it is satisfied by `pv: 0.0` +forever, and a PV channel that reports 0 W for a dead sensor MANUFACTURES the +conclusion the card exists to test - "there was no surplus to store" - in a week +of history nobody will re-derive. So every check below separates three states +that a weaker one would collapse into one: + + NO SOURCE `pv_entity` unset. No read, no entity, no topic, no row. + NO READING configured, and the sensor did not answer. The topic carries + HA's own `None`, the entity's state is `unknown`, the + recorder records a GAP. + ZERO WATTS configured, and the sensor said 0. The topic carries `0.0`. + +The last two are the pair the acceptance criterion is about and part 3 asserts +the PAYLOAD BYTES of both, because they differ by exactly one thing that no +check on the VALUE alone can see. + +⚠️ AND THE OTHER HALF OF THE CARD IS A NEGATIVE, GRADED AS ONE. This is +instrumentation: it must not become a control input, or the loop would react to +the quantity being measured and the measurement would be worthless. Part 6 is +the in-suite half of that claim (the shipped command is bit-identical across +absent / 0 W / 8 kW of PV, `inputs_ok` never moves, and an AST scan says which +modules can even see the material). `tools/tel11_pv_trace.py` is the other half: +it hashes a control document on THIS tree and on `release/1.0`, with the option +set and unset. + + Part 1 `pv.watts()`: the unit table, and every refusal, by name. + Part 2 the shipped read path, including what a FAILED read does to the + value a good one left behind. + Part 3 ⭐ publication: no source / no reading / zero watts, as payload bytes. + Part 4 announced and published are the same key, from the same predicate. + Part 5 the operator surfaces, and that an unset option adds none of them. + Part 6 ⭐ NOT A CONTROL INPUT. + Part 7 what this file cannot see, with sizes, and the mutation table. + +⚠️ MUTATING A MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run +`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it +stops bytecode being WRITTEN, not a stale one being READ. Read MUTATION.md. +""" + +import ast +import asyncio +import sys +from pathlib import Path + +from app import mqtt, pv +from app.main import Controller + +HERE = Path(__file__).resolve().parent +fails = [] + + +def check(name, cond): + print(f" {'ok ' if cond else 'FAIL'} {name}") + if not cond: + fails.append(name) + + +def state(value, unit="W", **attrs): + """A Home Assistant state object, in the shape `hass.state()` returns.""" + return {"state": value, "attributes": dict({"unit_of_measurement": unit}, + **attrs)} + + +# ============================================================================== +print("1. pv.watts(): the unit is read off the entity, and a refusal is named") +# ============================================================================== +# ⚠️ LITERALS ON THE RIGHT-HAND SIDE, NOT `raw * WATTS_PER[u]` (habit 2). An +# assertion written against the expression under test cannot tell a correct +# table from a table that is wrong in the same direction twice. +CASES = [ + (state("1234", "W"), 1234.0, "W"), + (state("5.2", "kW"), 5200.0, "kW"), + (state("0.0072", "MW"), 7200.0, "MW"), + (state("4500000", "mW"), 4500.0, "mW"), + (state("0.0000072", "GW"), 7200.0, "GW"), + (state("0.0000000072", "TW"), 7200.0, "TW"), + (state("-8", "W"), -8.0, "W"), +] +print(" state unit -> watts") +_bad = [] +for st, want, unit in CASES: + got, why = pv.watts(st) + print(f" {st['state']:>14} {unit:<4} -> {got} ({why})") + if got is None or abs(got - want) > 1e-6 or why != unit: + _bad.append((st, got, want)) +check(f"every unit in the table converts to watts against a literal, and the " + f"reason string is the unit it was read in. Wrong: {_bad}", not _bad) +check("...and the table is HOME ASSISTANT's own power units, so a card that " + "adds one has to say so here: mW, W, kW, MW, GW, TW", + sorted(pv.WATTS_PER) == sorted(["mW", "W", "kW", "MW", "GW", "TW"])) +check("...and every factor is an exact power of ten - nothing in this table is " + "a number anybody chose", + all(abs(v * 1000 - round(v * 1000)) < 1e-9 and v > 0 + for v in pv.WATTS_PER.values())) + +# ⚠️ THE CASE THE WHOLE CARD TURNS ON, AND IT IS ASSERTED IN BOTH DIRECTIONS. +check("⭐ ZERO WATTS IS A READING: a sensor that says 0 gives 0.0, not None - " + "the pack can be full and the sun can be down and the honest report of " + "the second one is a number", + pv.watts(state("0", "W")) == (0.0, "W") + and pv.watts(state("0.0", "kW")) == (0.0, "kW")) + +REFUSALS = [ + ("no state object at all (entity missing, or the read failed)", None, + "no state"), + ("the entity is `unavailable`", state("unavailable"), "unavailable"), + ("the entity is `unknown`", state("unknown"), "unknown"), + ("an empty state", state(""), "empty"), + ("a state that is not a number", state("sunny"), "not numeric"), + ("no unit_of_measurement at all", state("1234", ""), "no unit"), + ("a unit that is not power", state("1234", "%"), "'%'"), + # ⚠️ IN HA's `UnitOfPower` ENUM AND DELIBERATELY NOT IN THE TABLE. Its + # factor is a measured conversion rather than a decimal shift, and a wrong + # constant in the channel that exists to attribute a 20.5 kWh shortfall is + # worse than a refusal somebody can read. + ("BTU/h, which IS one of HA's power units", state("1234", "BTU/h"), + "'BTU/h'"), +] +_r = [] +print(" refusal -> (watts, why)") +for label, st, needle in REFUSALS: + got, why = pv.watts(st) + print(f" {label:<42} -> ({got}, {why!r})") + if got is not None or needle not in why: + _r.append(label) +check(f"⭐ EVERY REFUSAL ANSWERS None AND SAYS WHICH REFUSAL IT WAS - never " + f"0.0, and never a bare 'no'. A reason an operator cannot act on sends " + f"them to look at the wrong box. Wrong: {_r}", not _r) +check("...and the refused states are the SAME list `hass.number()` refuses, " + "imported rather than respelled - one path cannot start accepting what " + "the other rejects", + pv.BAD_STATES is __import__("app.hass", fromlist=["x"]).BAD_STATES) +# habit 1: the precondition, as its own check. Without it the row above could +# be true of a table that had quietly become empty. +check("PRECONDITION: the refusal list and the unit table are both non-empty, " + "so the two checks above quantify over something", + len(REFUSALS) >= 8 and len(pv.WATTS_PER) >= 6) + + +# ============================================================================== +print() +print("2. the shipped read path") +# ============================================================================== +LIVE = {"max_w": 5000.0, "gain": 0.6, "slew_w": 400.0, "target_grid_w": -10.0, + "deadband_w": 15, "step_w": 10} +BASE_OPTS = {"setpoint_entity": "number.sp", "meter_entity": "sensor.grid", + "soc_entity": "sensor.soc", "batt_entity": "sensor.batt", + "meter_source": "off", "heartbeat_s": 10, "stale_input_s": 15} + + +class _Store: + def __init__(self): + self.data = {} + + def set(self, *a): + pass + + def get_time(self, *a): + return None + + +class _Pub: + """Captures what `Controller.publish()` hands the publisher.""" + enabled = False + + def __init__(self): + self.frames = [] + + def publish(self, values): + self.frames.append(dict(values)) + + def close(self): + pass + + +class _Hass: + """A scripted meter, an inverter that follows, and a scripted PV sensor. + + ⚠️ `states` IS A SCRIPT, ONE ENTRY PER CYCLE, so a check can drive a GOOD + read and then a FAILED one and ask what the failure did to the value the + good one left behind. A fixture that answers the same thing forever cannot + reach that failure - MUTATION.md habit 9's shape, one surface over. + """ + + def __init__(self, grid=0.0, soc=50.0, states=(), follow=False): + self.grid = grid + # ⚠️ `follow` MAKES THE METER RESPOND TO THE BATTERY, and part 6 needs + # it. With a FIXED meter the error never shrinks, the integrator winds + # to its bound and the command rides `slew_w` to the rail on every + # input - so every stimulus produces the same ramp and an identity over + # them is free. `grid = base - batt` is the site: a battery discharging + # 610 W into a 600 W import leaves the meter at the -10 W this add-on + # rests at. + self.follow = follow + self.soc = soc + self.batt = 0.0 + self.n = 0 + self.writes = [] + self.state_calls = [] + self.states = list(states) + + async def limits(self, entity): + return (-LIVE["max_w"], LIVE["max_w"], 1.0) + + async def number(self, entity, invert=False): + grid = self.grid - self.batt if self.follow else self.grid + return {"sensor.grid": grid, "sensor.soc": self.soc, + "sensor.batt": self.batt}.get(entity) + + async def state(self, entity): + self.state_calls.append((self.n, entity)) + if not self.states: + return None + return self.states[min(self.n, len(self.states) - 1)] + + async def set_number(self, entity, value): + self.writes.append(float(value)) + self.batt = float(value) + return True + + async def call(self, domain, service, data): + return True + + +def run(cycles=3, grid=0.0, states=(), auto=True, follow=False, **over): + """`cycles` real read/cycle/publish turns. Returns (controller, hass, pub).""" + async def go(): + hass = _Hass(grid=grid, states=states, follow=follow) + pub = _Pub() + c = Controller(dict(BASE_OPTS, **LIVE, **over), hass, _Store(), pub, + stop=asyncio.Event()) + c.auto = auto + c.dev_min, c.dev_max = -LIVE["max_w"], LIVE["max_w"] + for _k in range(cycles): + await c.read_inputs() + await c.cycle() + c.publish() + hass.n += 1 + return c, hass, pub + return asyncio.run(go()) + + +PV_ON = {"pv_entity": "sensor.pv_power"} + +c_off, h_off, p_off = run(cycles=3) +check("⚠️ UNSET IS NOT READ AT ALL: with no `pv_entity` the add-on makes NO " + "state call, for any entity, on any cycle - not a read that fails " + "quietly, no read. This is what 'no warning spam' is made of", + h_off.state_calls == [] and c_off.pv_w is None + and c_off.pv_why == "not configured") + +c_on, h_on, _ = run(cycles=3, states=[state("3210", "W")], **PV_ON) +print(f" state calls: {h_on.state_calls}") +check("configured: the entity is read once per cycle, by name, and the value " + "reaches `self.pv_w` in watts", + h_on.state_calls == [(0, "sensor.pv_power"), (1, "sensor.pv_power"), + (2, "sensor.pv_power")] + and c_on.pv_w == 3210.0 and c_on.pv_why == "W") + +c_kw, _, _ = run(cycles=1, states=[state("3.21", "kW")], **PV_ON) +check("...and a kW sensor is read in kW: 3.21 kW is 3210 W, not 3.21 W. There " + "is no magnitude heuristic anywhere on this path - 30 W of dusk and " + "30 kW of noon are both ordinary readings", + c_kw.pv_w == 3210.0 and c_kw.pv_why == "kW") + +# ⚠️ THE ONE THAT SEPARATES "ABSENT" FROM "STALE", AND IT NEEDS TWO CYCLES. +c_drop, _, p_drop = run(cycles=2, states=[state("4100", "W"), None], **PV_ON) +print(f" published pv by cycle: {[f['pv'] for f in p_drop.frames]}") +check("⭐ A FAILED READ CLEARS THE PREVIOUS GOOD ONE. The cycle after a 4100 W " + "reading disappears publishes an ABSENCE, not 4100 W again - a value " + "that survives its own sensor is indistinguishable from a real one at " + "the moment somebody is working out whether the sun was shining", + [f["pv"] for f in p_drop.frames] == [4100.0, None] + and c_drop.pv_w is None and "no state" in c_drop.pv_why) + + +# ============================================================================== +print() +print("3. ⭐ publication: no source, no reading, zero watts - as payload bytes") +# ============================================================================== +class FakeClient: + def __init__(self): + self.msgs = [] + + def publish(self, topic, payload=None, retain=False): + self.msgs.append((topic, payload, retain)) + + +def wire(values, include=("pv",)): + """What the SHIPPED `MqttPublisher.publish()` hands paho for `values`.""" + pub = mqtt.MqttPublisher.__new__(mqtt.MqttPublisher) + pub.omit, pub.units, pub.include, pub.enabled = set(), [], list(include), True + pub.client = FakeClient() + pub.publish(values) + return pub.client.msgs + + +_zero = run(cycles=1, states=[state("0", "W")], **PV_ON)[2].frames[-1] +_absent = run(cycles=1, states=[None], **PV_ON)[2].frames[-1] +_nosrc = p_off.frames[-1] +print(f" no source : 'pv' in frame = {'pv' in _nosrc}") +def pv_msg(values): + """The `goodwe_ctl/pv` message the shipped publisher emits, or None.""" + got = [m for m in wire(values) if m[0] == "goodwe_ctl/pv"] + return got[0] if got else None + + +print(f" no reading: {_absent['pv']!r} -> wire {pv_msg(_absent)}") +print(f" zero watts: {_zero['pv']!r} -> wire {pv_msg(_zero)}") +check("NO SOURCE: an install that never set `pv_entity` publishes no `pv` key " + "at all - there is no entity to feed, so feeding one would be a topic " + "nothing announced", + "pv" not in _nosrc) +check("NO READING: the key is PRESENT and its value is None, which is a " + "different frame from the one above. 'the option is off' and 'the sensor " + "did not answer' are two facts and one field cannot carry both", + "pv" in _absent and _absent["pv"] is None) +check("ZERO WATTS: 0.0, published as a number", _zero["pv"] == 0.0) +check("⭐ AND ON THE WIRE THE LAST TWO ARE DIFFERENT BYTES ON THE SAME TOPIC: " + "`None` - which is Home Assistant's own payload for 'this sensor has no " + "value', so the entity goes to `unknown` and the recorder records a GAP " + "- against `0.0`. THIS IS THE ACCEPTANCE CRITERION: a week later, no PV " + "reported is not PV reported 0 W", + pv_msg(_absent) == ("goodwe_ctl/pv", "None", False) + and pv_msg(_zero) == ("goodwe_ctl/pv", "0.0", False) + and pv_msg(_absent) != pv_msg(_zero) + # ...and the pv row is the ONLY thing that differs between the two + # frames on the wire, so the claim above is about that row and not + # about some other number the two fixtures happened to move. + and [m for m in wire(_absent) if m[0] != "goodwe_ctl/pv"] + == [m for m in wire(_zero) if m[0] != "goodwe_ctl/pv"]) +check("...and the payload is HA's literal, from the module's own constant, not " + "a string spelled here", mqtt.PAYLOAD_NONE == "None") +# ⚠️ AND THE BLAST RADIUS IS ONE KEY. The None-dropping in `publish()` is what +# `circulation` depends on - its note in main.py says a missing per-unit reading +# must leave the last figure standing rather than publish a zero that reads as +# health - so widening it would be a behaviour change to another card's row. +check("⭐ AND NO OTHER ROW CHANGED BEHAVIOUR: `circulation: None` is still " + "DROPPED, not published as `None` - TEL-11 widened the rule for exactly " + "one key and `ABSENT_AS_UNKNOWN` is the list", + wire({"circulation": None, "grid": None, "loop_reason": None}) == [] + and mqtt.ABSENT_AS_UNKNOWN == frozenset({"pv"})) + + +# ============================================================================== +print() +print("4. announced and published are the same key, from the same predicate") +# ============================================================================== +# mqtt.py's `omit` note: an entity that exists and is never fed is a worse +# signal than one that does not exist. This is that claim for TEL-11's row, and +# it is taken off the SHIPPED announcement and the SHIPPED `publish()`. +def announced(opts): + return {t.rsplit("/", 2)[-2].replace(f"{mqtt.BASE}_", "") + for t, _c in mqtt.discovery_payloads( + include=mqtt.optional_keys(opts))} + + +check("with a source: `pv` is announced AND published", + "pv" in announced(PV_ON) and "pv" in _absent) +check("without one: announced by nothing and published by nothing", + "pv" not in announced({}) and "pv" not in _nosrc) +check("...and the predicate is ONE expression, `mqtt.optional_keys()`, read by " + "the announcement and by `Controller.publish()` alike - two spellings is " + "how an entity gets announced that nothing feeds", + mqtt.optional_keys({}) == () + and mqtt.optional_keys({"pv_entity": ""}) == () + and mqtt.optional_keys({"pv_entity": " "}) == () + and mqtt.optional_keys({"pv_entity": "sensor.x"}) == ("pv",)) +check("...and a whitespace-only option is the SAME as an empty one everywhere " + "- the controller does not read it either", + run(cycles=1, pv_entity=" ")[1].state_calls == []) + +# ⚠️ AND THE RESTART, WHICH IS THE OTHER HALF OF "RECORDABLE". Home Assistant +# keeps the entity_id it composed at FIRST discovery and keys its registry on +# `unique_id`, so a row whose id moved between two starts is a NEW entity beside +# a dead one and the history stays behind on the old id. Two independent +# announcements are asserted byte-identical, and the composed id is modelled +# INDEPENDENTLY of app/mqtt.py (the model is `tools/rehearse_roster.py`'s, and +# it is anchored there on the one composed id that was verified against the live +# system). +import re as _re # noqa: E402 + + +def compose(device_name, entity_name): + s_ = _re.sub(r"[^a-z0-9]+", "_", f"{device_name} {entity_name}".lower()) + return "sensor." + s_.strip("_") + + +_a1 = mqtt.discovery_payloads(include=mqtt.optional_keys(PV_ON)) +_a2 = mqtt.discovery_payloads(include=mqtt.optional_keys(PV_ON)) +_pv_cfg = [c for _t, c in _a1 if c["unique_id"] == "goodwe_ctl_pv"] +print(f" composed entity_id: " + f"{compose(mqtt.DEVICE['name'], _pv_cfg[0]['name'])}") +check("a restart re-announces the SAME bytes, so Home Assistant sees the entity " + "it already has rather than a new one beside a dead one - the history is " + "continuous across a restart of the add-on", + [(t, __import__("json").dumps(c, sort_keys=True)) for t, c in _a1] + == [(t, __import__("json").dumps(c, sort_keys=True)) for t, c in _a2] + and len(_pv_cfg) == 1) +check("...and the id Home Assistant composes for it is " + "`sensor.goodwe_pv_power`, which is what DOCS.md tells an operator to put " + "on a dashboard - device name plus entity name, NOT the object_id", + compose(mqtt.DEVICE["name"], _pv_cfg[0]["name"]) + == "sensor.goodwe_pv_power" + and _pv_cfg[0]["state_topic"] == "goodwe_ctl/pv") + + +# ============================================================================== +print() +print("5. the operator surfaces") +# ============================================================================== +def rows_of(c): + return [r["text"] for r in c.checks()] + + +def page_rows(c): + return c.status_payload()["rows"] + + +_off_rows, _on_rows = rows_of(c_off), rows_of(c_on) +print(f" unset: {len(_off_rows)} status rows; set: {len(_on_rows)}") +check("⚠️ AN UNSET OPTION ADDS NO STATUS ROW AND NO PAGE ROW - not a caution, " + "not a 'no PV configured' nag. `peak_forecast_entity` DOES nag when it " + "is unset, because an unset forecast leaves a money guard without an " + "input; this is instrumentation and a permanent caution on every install " + "that never asked for it is the warning spam the card forbids", + not [t for t in _off_rows if "PV" in t] + and not [r for r in page_rows(c_off) if r[0] == "PV"]) +check("...and the whole row list is otherwise the same length, so nothing else " + "moved either", len(_on_rows) == len(_off_rows) + 1) +_pv_row = [t for t in _on_rows if t.startswith("PV production")] +print(f" {_pv_row}") +# habit 14: the ordered pair - value, entity and unit each in its OWN slot in +# the literal sentence, so a transposition is visible rather than merely +# present. +check("a good read says the watts, the entity it came from and the unit it was " + "reported in, each in its own slot", + _pv_row == ["PV production: 3210 W from sensor.pv_power " + "(reported in W)"] + and ("PV", "3 210 W") in page_rows(c_on)) + +c_bad, _, _ = run(cycles=1, states=[state("1234", "%")], **PV_ON) +_bad_row = [r for r in c_bad.checks() if r["text"].startswith("PV production")] +print(f" {[r['text'] for r in _bad_row]}") +check("a refused read says WHICH refusal, and names the entity - 'no reading' " + "on its own sends an operator to look at a working sensor", + len(_bad_row) == 1 + and _bad_row[0]["text"] == ("PV production: no reading from " + "sensor.pv_power - unit '%' is not one of " + "mW, W, kW, MW, GW, TW - it cannot be read " + "in watts")) +check("⭐ ...and it is a CAUTION, never a `bad` row: `status_payload()` builds " + "NOT READY out of rows that are neither ok nor warn, and a telemetry " + "channel that could put a battery site into NOT READY would be " + "observability with a foot in the control path", + _bad_row[0] == {"ok": False, "warn": True, "text": _bad_row[0]["text"]} + and not c_bad.status_payload()["banner"].startswith("NOT READY") + and c_bad.status_payload()["level"] != "bad" + and c_bad.status_payload()["banner"] + == c_off.status_payload()["banner"]) +check("...and the page row renders an absence as an em dash, not as 0 W", + ("PV", "—") in page_rows(c_bad)) + + +# ============================================================================== +print() +print("6. ⭐ NOT A CONTROL INPUT") +# ============================================================================== +# ⚠️ THE CLAIM IS NOT "pv is not in control.py". It is that the COMMANDS do not +# move, and that is graded by driving the same stimulus with the PV channel +# absent, at zero, and at 8 kW - a spread that would be impossible to miss if +# any term of the control law had learned to read it. +# +# ⚠️ AND THE FIRST VERSION OF THIS FIXTURE COULD NOT HAVE SEEN IT, WHICH IS +# MUTATION.md HABIT 23 AND THE CARD'S OWN BATTERY FOUND IT. The stimulus was a +# 6 kW export over 12 cycles; the card's central mutation - `self.grid -= +# self.pv_w` in `read_inputs()`, +106 B, on disk, purged, genuinely running - +# came back a SURVIVOR. Not because the property held: because at 6 kW of +# export the loop is against its rail and the command is moving at `slew_w` +# every cycle, so -6000 W and -14000 W of apparent error produce the SAME +# ramp, tick for tick. The mutant ran and could not change the answer on the +# data it was given. +# +# The stimulus is now a SMALL import the loop can actually settle on, far +# inside the rail, and the separation is asserted as its own precondition +# below: a check that cannot go red on the one mutation the card is about is +# the card's own habit-4. +_stim = dict(cycles=40, grid=600.0, follow=True) +_c_none, _h_none, _ = run(**_stim) +_c_absent, _h_absent, _ = run(**_stim, states=[None], **PV_ON) +_c_zero, _h_zero, _ = run(**_stim, states=[state("0", "W")], **PV_ON) +_c_big, _h_big, _ = run(**_stim, states=[state("8000", "W")], **PV_ON) +_w_none, _w_big = _h_none.writes, _h_big.writes +print(f" unset : {_w_none} -> settled at {_c_none.target:g} W") +print(f" 8000 W : {_w_big} -> settled at {_c_big.target:g} W") +check("⭐ THE COMMANDS ARE BIT-IDENTICAL ACROSS unset / no reading / 0 W / " + "8 kW of production, over 40 cycles of a real `cycle()` against a 600 W " + "import the loop SETTLES on - every write, in order, and the same " + "resting target. The PV channel changes no number that reaches an " + "inverter", + _w_none == _h_absent.writes == _h_zero.writes == _w_big + and _c_none.target == _c_absent.target == _c_zero.target + == _c_big.target) +check("PRECONDITION: that stimulus really does command something, and RESTS " + "well inside the rail rather than riding the slew limit to it - at the " + "rail every input produces the same ramp and the identity above would " + "be free (habit 1)", + len(_w_none) >= 4 and abs(_c_none.target) > 100 + and abs(_c_none.target) < 0.5 * LIVE["max_w"]) +# ⚠️ THE HABIT-23 PRECONDITION, AND IT IS THE ONE THAT MAKES THE IDENTITY MEAN +# SOMETHING. "What would the mutation have had to change, and does the fixture +# reach that value?" Here: if the 8 kW HAD been subtracted from the meter +# reading, would this stimulus produce a different command? It is one line to +# ask, and on the first version of this fixture the answer was NO - see the +# note above `_stim`. +_c_shift, _h_shift, _ = run(cycles=40, grid=600.0 - 8000.0, follow=True) +print(f" meter shifted by the same 8 kW -> settled at " + f"{_c_shift.target:g} W") +check("⭐ PRECONDITION (habit 23): the same stimulus with the METER shifted by " + "that 8 kW rests somewhere visibly different, so 'the commands did not " + "move' is a fact about the code and not about a rail that would have " + "flattened any input", + _h_shift.writes != _w_big + and abs(_c_shift.target - _c_big.target) > 4000) +check("...and `inputs_ok` does not move either: a PV sensor that never answers " + "is not a missing INPUT, because nothing computes with it. An install " + "whose solar integration breaks keeps regulating", + _c_none.inputs_ok is True and _c_absent.inputs_ok is True) + +# --- the structural half, and its scope is COMPUTED (habit 12) --------------- +APP = sorted((HERE / "app").glob("*.py")) +NEEDLES = ("pv_entity", "pv_w", "pv_why", "optional_keys", "ABSENT_AS_UNKNOWN") + + +def touches(src: str) -> bool: + """Does this module MENTION TEL-11's material, by any spelling? + + ⚠️ A TEXT SCAN ON PURPOSE, AND IT IS THE CONSERVATIVE DIRECTION HERE. An + AST scan would have to enumerate the forms a reference can take (attribute, + subscript, `getattr`, a string key) and habit 12 says a hand-written list of + forms is the same defect as a hand-written scope. Text over-reports - a + comment counts - and over-reporting is safe for a check whose claim is + "these modules do NOT mention it". + """ + return any(n in src for n in NEEDLES) or "from . import pv" in src \ + or "from .pv import" in src + + +_touch = sorted(f.name for f in APP if touches(f.read_text(encoding="utf-8"))) +print(f" {len(APP)} modules under app/; TEL-11 material is in: {_touch}") +check(f"PRECONDITION: the scan found the whole package, not a subset - " + f"{len(APP)} modules", len(APP) >= 25 and "control.py" in + [f.name for f in APP]) +check("⭐ EXACTLY THREE MODULES CAN SEE THE MATERIAL: `pv.py` (the reader), " + "`main.py` (the read and the two operator surfaces) and `mqtt.py` (the " + "publication). `control.py`, `arbiter.py`, `allocator.py`, `peakguard.py`, " + "`arbitrage.py`, `maintenance.py`, `clamp.py`, `freeze.py`, " + "`watchdog.py`, `delivery.py` and `circulation.py` do not mention it in " + "any spelling, comments included", + _touch == ["main.py", "mqtt.py", "pv.py"]) +# habit 4 in the instrument: a scanner that cannot go red is not a scanner. +check("...and the scan CAN go red - a module body with the material in it is " + "caught, so the green above is the tree speaking and not the scanner " + "being blind", + touches("x = opts['pv_entity']\n") and touches("from . import pv\n") + and not touches("# nothing to see, a pvc pipe\n")) + + +# ============================================================================== +print() +print("7. what this file CANNOT see, with sizes - and the mutation table") +# ============================================================================== +# habit 13. An enumerated list with an honest gap is worth more than a claim of +# completeness a reviewer falsifies in ten minutes. +for line in [ + "Home Assistant itself. That `None` on the state topic renders as state", + " `unknown`, and that the recorder keeps it, is READ OUT OF HA's source", + " (mqtt/const.py PAYLOAD_NONE, mqtt/sensor.py's `_message_received`,", + " retrieved 2026-09-24) and quoted in app/mqtt.py. No HA runs in this", + " suite, so what is graded here is the BYTES this add-on publishes.", + "Whether the operator's PV sensor is telling the truth. A sensor wired", + " backwards reads -4 kW at noon; this file asserts that the sign is", + " published unchanged, never that it is right. That is what the history", + " the card creates is FOR - see DECISIONS/TEL-11-pv-production.md §2.", + "The control-path claim OUTSIDE app/. `tools/`, the ESPHome firmware and", + " the ingress page are not scanned by part 6's census.", + "A roster of more than one. Every fixture here is the shipped roster of", + " one; the row is a SITE row and there is nothing per-unit to separate.", + "Wall-clock behaviour: no fixture here sleeps, and the read is driven by", + " the caller's loop rather than by the 1 s tick.", +]: + print(" " + line) + +print() +print(" mutation bytes reds") +print(" ------------------------------------------------ ----- --------") +for row in [ + ("M1 the pv row moved into SENSORS", "-79 B", "3 test_mqtt + 1 here"), + ("M2 publish() drops a None pv row again", "-98 B", "1 here"), + ("M3 ABSENT_AS_UNKNOWN widened past the one key", "+23 B", "1 here"), + ("M4 a dead sensor reported as 0 W", "-1 B", "4 here"), + ("M5 a failed read leaves the last value standing", "+46 B", "2 here"), + ("M6 the unit read and then ignored", "-18 B", "2 here"), + ("M7 `self.grid -= self.pv_w` in read_inputs()", "+106 B", + "2 here + the trace"), + ("M8 the row announced and fed unconditionally", "-52 B", "3 here"), + ("M9 the pv caution made a `bad` row (NOT READY)", "+1 B", "1 here"), + ("M10 the read gate removed - unset is read anyway", "-10 B", + "2 here + 2 crashes"), +]: + print(f" {row[0]:<48} {row[1]:>5} {row[2]}") +print(" ⚠️ M7 WAS A SURVIVOR ON THE FIRST BATTERY and part 6's note says why:") +print(" the fixture drove the loop to its RAIL, where every input produces") +print(" the same ramp. MUTATION.md habit 23, with a saturated actuator in") +print(" place of a coinciding value - and it is in that file now.") +print(" ⚠️ M10's two extra reds are CRASHES, not named checks: every") +print(" fixture in test_inverter.py and test_telemetry.py predates the PV") +print(" read and has no `state()` on its Hass double, so an unconditional") +print(" read is an AttributeError. A red, but an unnamed one - habit 20's") +print(" shape - and it is recorded rather than tidied away.") +print(" Every row above was RUN, in the CI image, on a purged tree, one at a") +print(" time - see DECISIONS/TEL-11-pv-production.md §5 for the transcript.") +print(" ⚠️ A RED COUNT IS A PROPERTY OF THE SUITE AS MUCH AS OF THE MUTATION:") +print(" these name THIS tree. Re-measure rather than inherit.") + +print() +if fails: + print(f"{len(fails)} FAILED:") + for f in fails: + print(f" - {f}") + sys.exit(1) +print("all checks passed") diff --git a/goodwe_controller/test_translations.py b/goodwe_controller/test_translations.py index 768c44c..822d21f 100644 --- a/goodwe_controller/test_translations.py +++ b/goodwe_controller/test_translations.py @@ -371,8 +371,14 @@ print("\n7. no behaviour and no value moved") # test_watchdog.py's six, test_price.py's seven, test_freeze.py's, and # test_peakguard.py's. This line is the cheap one that says the reorder did not # lose or duplicate a key on the way. -check("no option was lost, gained or duplicated by the reordering: the two " - "halves of config.yaml hold the same 72 keys, once each", +# ⚠️ THE COUNT IS READ OFF THE TREE, NOT TYPED. It was typed - "the same 72 +# keys" - and was 75 by the time TEL-11 read it, because three cards had added +# an option each and none of them was looking at this sentence. A number in a +# check's own message is a claim like any other (MUTATION.md habit 22) and this +# one had been false for three cards without a single run going red, because +# nothing asserts it. Derived, it cannot rot. +check(f"no option was lost, gained or duplicated by the reordering: the two " + f"halves of config.yaml hold the same {len(ORDER)} keys, once each", sorted(_opt_order) == sorted(ORDER) and len(set(ORDER)) == len(ORDER)) check("...and `inverters: []` is still shipped in the options half, which is " "what Supervisor 2026.08.0 refuses every pre-0.6.0 options blob without " diff --git a/goodwe_controller/tools/safety56_behaviour_trace.py b/goodwe_controller/tools/safety56_behaviour_trace.py new file mode 100644 index 0000000..8bcf325 --- /dev/null +++ b/goodwe_controller/tools/safety56_behaviour_trace.py @@ -0,0 +1,300 @@ +"""SAFETY-56's behaviour-invariance instrument. `tools/safety56_behaviour_trace.py` + +SAFETY-56 rewrote three operator-facing sentences and nothing else. This is what +says so: it drives a fixed stimulus set through the SHIPPED modules and prints +two documents - + + `behaviour` every verdict, every `reading()`, every `releases`, every + `inputs_ok` and every readiness row's (ok, warn). NO SENTENCE + REACHES IT, so a tree whose only change is wording must produce + this document byte for byte. + `sentences` the four operator-facing surfaces, from the same stimulus. + +Run it in two trees and compare: + + python3 tools/safety56_behaviour_trace.py > /tmp/after.json + python3 tools/safety56_behaviour_trace.py --compare /tmp/before.json /tmp/after.json + +⚠️ THE CONTROL IS PART OF THE TOOL AND NOT A SEPARATE STEP, WHICH IS MUTATION.md +HABIT 4 IN AN INSTRUMENT. `--compare` REFUSES to report "behaviour unchanged" +unless the `sentences` halves DIFFER. Without that, comparing a tree with itself +- which this card's first run did, by archiving a branch whose edits were not +committed yet - gives a perfect result that means nothing. An identity is +evidence only when the two sides could have differed, and ARCH-02's rehearsal +harness is the worked example of what happens when nobody checks. + +⚠️ IT GRADES BEHAVIOUR, NEVER WORDING. Two trees that agree here can still print +different sentences - that is the point. `test_freshness.py` part 11 grades the +sentences themselves. + +⚠️ WHAT IT CANNOT SEE, with shapes (habit 13): anything outside `freshness.py`'s +verdict, `_read_unit()`'s gating and `checks()`'s rows; a roster of more than +one; the MQTT and ingress surfaces; and wall-clock behaviour, since every clock +here is the caller's monotonic argument. It is a pin on the three things this +card is forbidden to move, not a full behavioural equivalence. + +⚠️ PURGE BEFORE USE on a tree you have just edited. `tools/mutation_run.sh` +does, and MUTATION.md says why `python3 -B` alone does not. +""" +import asyncio +import hashlib +import importlib.util +import json +import logging +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent.parent # the add-on directory +sys.path.insert(0, str(HERE)) + +BOUNDS = Freshness = Controller = DEFAULTS = None + + +def _load(): + """The add-on's modules, imported LAZILY so that `--compare` runs anywhere. + + ⚠️ NOT A STYLE CHOICE. `--compare` is the half a reviewer runs on a laptop, + against two JSON files, in a directory that has no `app/` in it - and a + top-level import would make the comparison impossible exactly where it is + most useful. + """ + global BOUNDS, Freshness, Controller, DEFAULTS + from app.freshness import BOUNDS, Freshness # noqa: E402,F401 + from app.main import Controller # noqa: E402,F401 + from app.store import DEFAULTS # noqa: E402,F401 + +A = 5000.0 +ONE = dict(meter_source="off", meter_entity="sensor.grid", + setpoint_entity="number.sp_a", soc_entity="sensor.soc_a", + batt_entity="sensor.batt_a", max_w=A, gain=0.6, slew_w=0, + deadband_w=15, target_grid_w=0, step_w=10, heartbeat_s=10, + stale_input_s=15, inverters=[], bms_soc_floor=10) + + +class MemStore: + def __init__(self, d=None): + self.data = dict(DEFAULTS, **(d or {})) + + def set(self, k, v): + self.data[k] = v + + def get_time(self, *a): + return None + + def set_time(self, *a): + pass + + +class Pub: + enabled = False + + def publish(self, *a, **k): + pass + + def close(self): + pass + + +class Hass: + def __init__(self, numbers=None, limits=None): + self.numbers = dict(numbers or {}) + self.limits_by = dict(limits or {}) + self.served = [] + self.writes = [] + + async def number(self, entity, invert=False): + v = self.numbers.get(entity) + self.served.append((entity, v)) + return None if v is None else (-v if invert else v) + + async def limits(self, entity): + return self.limits_by.get(entity, (None, None, None)) + + async def set_number(self, entity, value): + self.writes.append((entity, value)) + return True + + async def call(self, domain, service, data): + return True + + +def one_site(soc=50.0): + hass = Hass({"sensor.grid": 0.0, "sensor.soc_a": soc, + "sensor.batt_a": -1200.0}, + {"number.sp_a": (-A, A, 1.0)}) + c = Controller(dict(ONE), hass, MemStore(), Pub(), stop=asyncio.Event()) + c.inverters[0].dev_min, c.inverters[0].dev_max = -A, A + return c, hass + + +async def _world(name, soc, reg_of, cmd_of, n): + c, hass = one_site(soc) + seen = [] + for t in range(n): + hass.numbers["sensor.batt_a"] = reg_of(t) + c.inverters[0].last_write = cmd_of(t) + await c.read_inputs(now_mono=float(t)) + seen.append([c.inverters[0].batt, c.inputs_ok, + c.inverters[0].batt_fresh.stale]) + return {"name": name, "reads": seen, + "rows": [[r["ok"], r["warn"]] for r in c.checks()], + "n_rows": len(c.checks()), "events": len(c.events), + "inputs_bad": c.inputs_bad_since is not None, + "releases": [i.batt_fresh.releases for i in c.inverters]} + + +# The command and register schedules. Each of the conjunction's three terms is +# exercised alone and with the others, and the last two schedules take a verdict +# all the way through SAFETY-57's release and back into a re-trip. +CMDS = [lambda t: 0.0, lambda t: 1300.0, lambda t: -1200.0, + lambda t: 1300.0 if t >= 1 else -1200.0, + lambda t: 249.0 if t >= 1 else 0.0, + lambda t: 251.0 if t >= 1 else 0.0, + lambda t: 1300.0 if t < 40 else 0.0, + lambda t: 5.0 if t >= 1 else 1300.0, + lambda t: (t % 7) * 200.0 - 600.0, + lambda t: (-1200.0 if t < 1 else 1300.0) if t < 45 else ( + 0.0 if t < 90 else 1300.0), + lambda t: (-1200.0 if t < 1 else 1300.0) if t < 45 else 100.0] +REGS = [lambda t: -3.0, lambda t: -1200.0, lambda t: -1200.0 + (t % 3), + lambda t: None if t % 5 == 0 else -1200.0, + lambda t: 0.0, lambda t: float((t * 37) % 11) - 5.0] + +WORLDS = [ + ("DEAD", 50.0, lambda t: -3.0, lambda t: 800.0 if t < 1 else 1600.0, 40), + ("REFUSED", 11.0, lambda t: -3.0, lambda t: 800.0 if t < 1 else 1600.0, 40), + ("ABSENT", 50.0, lambda t: None, lambda t: 0.0, 3), + ("HEALTHY", 50.0, lambda t: -1200.0 + (t % 3), + lambda t: 1300.0 if t % 2 else -1200.0, 60), + ("RELEASE", 50.0, lambda t: -1200.0, + lambda t: (-1200.0 if t < 1 else 1300.0) if t < 40 else 0.0, 60), + ("RETRIP", 50.0, lambda t: -1200.0, + lambda t: (-1200.0 if t < 1 else 1300.0) if t < 40 else + (0.0 if t < 80 else 1300.0), 130), +] + + +def behaviour(): + """Everything this card is forbidden to move. No sentence may enter it.""" + _load() + logging.disable(logging.CRITICAL) + out = {} + + # 1. the committed live capture, read by read. + spec = importlib.util.spec_from_file_location( + "_s56decode", HERE / "tools" / "decode_bus_capture.py") + dec = importlib.util.module_from_spec(spec) + spec.loader.exec_module(dec) + cap_file = (HERE / "scenarios" / "safety57-2026-09-21" + / "bus-a-2026-09-21-1030-1110.log.gz") + events, _census = dec.decode(cap_file) + sp, bt = dec.series(events, "setpoint_w"), dec.series(events, "batt_w") + f = Freshness(BOUNDS["batt"]) + t = float(int(max(sp[0][0], bt[0][0])) + 1) + t1 = int(min(sp[-1][0], bt[-1][0])) + cap = [] + while t <= t1: + f.submit(dec.held_at(bt, t), dec.held_at(sp, t), t) + cap.append([t, f.stale, f.reading(), f.releases, f.held, + f.rested_since, f.graded_from, f.standing_at_grade, + f.released_at]) + t += 1.0 + out["capture_reads"] = len(cap) + out["capture_trace"] = hashlib.sha256( + json.dumps(cap, sort_keys=True).encode()).hexdigest() + out["capture_edges"] = [r[:4] for i, r in enumerate(cap) + if i == 0 or r[1] != cap[i - 1][1]] + + # 2. the swept grid over the policy object. + sweep = [] + for ci, cf in enumerate(CMDS): + for ri, rf in enumerate(REGS): + g = Freshness(BOUNDS["batt"]) + row = [] + for n in range(120): + g.submit(rf(n), cf(n), float(n)) + row.append([g.stale, g.reading(), g.releases]) + sweep.append([ci, ri, row]) + out["sweep_cells"] = len(sweep) + out["sweep_trace"] = hashlib.sha256( + json.dumps(sweep, sort_keys=True).encode()).hexdigest() + out["sweep_stale_reads"] = sum(1 for _c, _r, row in sweep + for s, _v, _n in row if s) + out["sweep_missing_reads"] = sum(1 for _c, _r, row in sweep + for _s, v, _n in row if v is None) + out["sweep_releases"] = sum(row[-1][2] for _c, _r, row in sweep) + + # 3. the shipped read path and every readiness row's verdict. + loop = asyncio.new_event_loop() + out["worlds"] = [loop.run_until_complete(_world(*w)) for w in WORLDS] + loop.close() + return out + + +def sentences(): + """The four operator-facing surfaces, from the REFUSED world.""" + _load() + recs = [] + + class _H(logging.Handler): + def emit(self, record): + recs.append(record.getMessage()) + + logging.disable(logging.NOTSET) + log = logging.getLogger("goodwe") + log.addHandler(_H()) + log.setLevel(logging.DEBUG) + + async def go(): + c, hass = one_site(11.0) + for t in range(40): + hass.numbers["sensor.batt_a"] = -3.0 + c.inverters[0].last_write = 800.0 if t < 1 else 1600.0 + await c.read_inputs(now_mono=float(t)) + return c + + loop = asyncio.new_event_loop() + c = loop.run_until_complete(go()) + loop.close() + return { + "reason": c.inverters[0].batt_fresh.reason, + "log": [m for m in recs if "SAFETY-37" in m], + "stale_note": c.inverters[0].stale_note(), + "not_ready_row": [r["text"] for r in c.checks() + if r["text"].startswith("battery power")], + } + + +def compare(a_path, b_path): + a = json.loads(Path(a_path).read_text(encoding="utf-8")) + b = json.loads(Path(b_path).read_text(encoding="utf-8")) + beh_same = a["behaviour"] == b["behaviour"] + sent_same = a["sentences"] == b["sentences"] + ha = hashlib.sha256(json.dumps(a["behaviour"], sort_keys=True).encode()) + hb = hashlib.sha256(json.dumps(b["behaviour"], sort_keys=True).encode()) + print(f"behaviour A {ha.hexdigest()}") + print(f"behaviour B {hb.hexdigest()}") + if sent_same: + # ⚠️ THE REFUSAL, AND IT IS THE POINT OF THE TOOL. Two runs whose + # sentences agree are either the same tree twice or a card that changed + # nothing an operator reads. Either way "behaviour unchanged" is not a + # result, and reporting it would be an identity measured against itself. + print("REFUSED: the `sentences` halves are IDENTICAL, so these two runs " + "cannot be evidence that behaviour survived a WORDING change - " + "they may be the same tree twice. Compare a tree against one " + "whose sentences differ.") + return 2 + if not beh_same: + for k in sorted(set(a["behaviour"]) | set(b["behaviour"])): + if a["behaviour"].get(k) != b["behaviour"].get(k): + print(f"BEHAVIOUR MOVED at {k!r}") + return 1 + print("OK: sentences differ, behaviour is byte-identical.") + return 0 + + +if __name__ == "__main__": + if len(sys.argv) == 4 and sys.argv[1] == "--compare": + sys.exit(compare(sys.argv[2], sys.argv[3])) + print(json.dumps({"behaviour": behaviour(), "sentences": sentences()}, + sort_keys=True, indent=1)) diff --git a/goodwe_controller/tools/tel11_pv_trace.py b/goodwe_controller/tools/tel11_pv_trace.py new file mode 100644 index 0000000..4d77554 --- /dev/null +++ b/goodwe_controller/tools/tel11_pv_trace.py @@ -0,0 +1,390 @@ +"""TEL-11's behaviour-invariance instrument. `tools/tel11_pv_trace.py` + +TEL-11 adds a PV production channel and claims two things a reviewer cannot +check by reading a diff: + + 1. with `pv_entity` UNSET - which is every install today and most installs + ever - the add-on behaves exactly as `release/1.0` does. Not "the control + law is untouched": the whole observable surface, commands and telemetry + alike. + 2. with `pv_entity` SET, the CONTROL decisions are still exactly those. The + channel is instrumentation and must never become a control input, because + a loop that reacted to production would destroy the very measurement the + card exists to make. + +This is what says so. It drives a fixed stimulus set through the SHIPPED +`Controller` and prints two documents per setting - + + `control` every command written to an actuator, every `self.target`, every + `inputs_ok`, the loop's own three fields and the arbiter's + resolution, tick by tick. NO TEL-11 MATERIAL REACHES IT. + `telemetry` the published frames, the status rows and the announcement - + i.e. everything the card is ALLOWED to move. + +- and a `tree` fingerprint: sha256 and byte count of every `app/*.py` it +imported, so the comparison can say WHICH TREES it graded. + + python3 tools/tel11_pv_trace.py > /tmp/tel11-head.json + # …in a release/1.0 checkout, with this file copied in… + python3 tools/tel11_pv_trace.py > /tmp/tel11-base.json + python3 tools/tel11_pv_trace.py --compare /tmp/tel11-base.json \\ + /tmp/tel11-head.json + +⚠️ THE CONTROL IS PART OF THE TOOL, WHICH IS MUTATION.md HABIT 4 IN AN +INSTRUMENT - AND IT IS A DIFFERENT CONTROL FROM `safety56_behaviour_trace.py`'s, +ON PURPOSE. That card CHANGED sentences, so its `--compare` refuses to report +"behaviour unchanged" unless the sentences differ. THIS card's headline claim is +that with the option unset NOTHING differs at all, so "the documents are +identical" is the result rather than the warning sign - and the thing that has +to be proved instead is that the two runs came from DIFFERENT TREES and that the +instrument can see this card's material when it is switched on. So `--compare` +refuses unless: + + * the two `tree` fingerprints DIFFER (otherwise it is one tree twice, which + is the trap ARCH-02's rehearsal harness walked into and SAFETY-56 names); + * exactly ONE side's `set` document differs from its own `unset` document. + That side is the HEAD; the other is the baseline, which has no such option + and cannot react to it. Both, or neither, is ambiguous and is refused. + +⚠️ WHAT IT CANNOT SEE, with shapes (habit 13): + * a roster of more than one, and any per-unit surface; + * the real MQTT transport, Home Assistant, and the recorder. The publication + is graded as the dict `Controller.publish()` hands the publisher plus the + announcement `discovery_payloads()` returns - not as bytes on a broker; + * wall-clock behaviour and the 1 s tick: every clock here is an argument; + * the ingress HTML, the action log and the firmware; + * anything a `pv_entity` whose sensor is HEALTHY does not exercise - the + stimulus set drives a good read, a refused unit and a dead sensor, and + nothing else; + * ⚠️ and it is an IDENTITY over a fixed stimulus, so a divergence that needs + an input this set never produces is invisible to it by construction. The + stimulus is printed with the documents for exactly that reason. + +⚠️ PURGE `__pycache__` BEFORE USE on a tree you have just edited. +`tools/mutation_run.sh` does, and MUTATION.md says why `python3 -B` alone does +not. +""" +import asyncio +import hashlib +import json +import logging +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent.parent # the add-on directory +sys.path.insert(0, str(HERE)) + +Controller = mqtt = None + + +def _load(): + """The add-on's modules, imported LAZILY so `--compare` runs anywhere. + + ⚠️ NOT A STYLE CHOICE - `safety56_behaviour_trace.py`'s note is this one's + too. `--compare` is the half a reviewer runs against two JSON files, in a + directory with no `app/` in it, and a top-level import would make the + comparison impossible exactly where it is most useful. + """ + global Controller, mqtt + from app.main import Controller # noqa: E402,F401 + from app import mqtt # noqa: E402,F401 + + +A = 5000.0 +TUNING = {"max_w": A, "gain": 0.6, "slew_w": 400.0, "target_grid_w": -10.0, + "deadband_w": 15, "step_w": 10} +ONE = dict(TUNING, meter_source="off", meter_entity="sensor.grid", + setpoint_entity="number.sp_a", soc_entity="sensor.soc_a", + batt_entity="sensor.batt_a", heartbeat_s=10, stale_input_s=15, + inverters=[], bms_soc_floor=10) +PV_ENTITY = "sensor.pv_power" + + +class MemStore: + def __init__(self): + self.data = {} + + def set(self, *a): + pass + + def get_time(self, *a): + return None + + def set_time(self, *a): + pass + + +class Pub: + enabled = False + + def __init__(self): + self.frames = [] + + def publish(self, values): + self.frames.append(dict(values)) + + def close(self): + pass + + +class Hass: + """A scripted meter, a following inverter, and a scripted PV sensor.""" + + def __init__(self, grid_of, soc, pv_of): + self.grid_of, self.soc, self.pv_of = grid_of, soc, pv_of + self.batt = 0.0 + self.n = 0 + self.writes = [] + + async def limits(self, entity): + return (-A, A, 1.0) + + async def number(self, entity, invert=False): + v = {"sensor.grid": self.grid_of(self.n), "sensor.soc": self.soc, + "sensor.soc_a": self.soc, "sensor.batt_a": self.batt, + "sensor.batt": self.batt}.get(entity) + return None if v is None else (-v if invert else v) + + async def state(self, entity): + return self.pv_of(self.n) + + async def set_number(self, entity, value): + self.writes.append((entity, float(value))) + self.batt = float(value) + return True + + async def call(self, domain, service, data): + return True + + +def _pv_state(watts, unit="W"): + return {"state": str(watts), + "attributes": {"unit_of_measurement": unit}} + + +# ⚠️ THE STIMULUS SET, PRINTED WITH THE DOCUMENTS. An identity is only as wide +# as what was driven through it, so the worlds and the PV schedules are part of +# the output rather than a fact about the source. +# +# Each PV schedule is exercised against each grid world. A schedule that only +# ever answered healthily could not separate "the loop ignores production" from +# "the loop ignores a production channel that is always saying the same thing". +GRIDS = { + "EXPORT-6K": lambda t: -6000.0, + "IMPORT-3K": lambda t: 3000.0, + "REST": lambda t: -10.0, + "SWINGING": lambda t: (-4000.0 if (t // 5) % 2 else 4000.0), + "METER-GAPS": lambda t: (None if t % 7 == 3 else -1500.0), +} +PVS = { + "none": lambda t: None, + "zero": lambda t: _pv_state(0), + "noon": lambda t: _pv_state(8000), + "ramp": lambda t: _pv_state(t * 250), + "kilowatts": lambda t: _pv_state(4.25, "kW"), + "bad-unit": lambda t: _pv_state(1234, "%"), + "flapping": lambda t: (None if t % 3 else _pv_state(3300)), + "negative": lambda t: _pv_state(-4200), +} +CYCLES = 24 + + +async def _run(grid_of, pv_of, pv_set, soc=50.0, stop_at=None): + hass = Hass(grid_of, soc, pv_of) + pub = Pub() + opts = dict(ONE) + if pv_set: + opts["pv_entity"] = PV_ENTITY + c = Controller(opts, hass, MemStore(), pub, stop=asyncio.Event()) + c.auto = True + c.dev_min, c.dev_max = -A, A + c.inverters[0].dev_min, c.inverters[0].dev_max = -A, A + ctrl = [] + for t in range(CYCLES): + if stop_at is not None and t == stop_at: + c.auto = False + await c.read_inputs() + await c.cycle() + c.publish() + # ⚠️ THE CONTROL DOCUMENT, AND NOTHING FROM THIS CARD IS IN IT. Every + # field here existed on release/1.0 and is a DECISION: what was + # commanded, what the site is aiming at, whether the inputs were good + # enough to act on, what the loop did and which claim won. + ctrl.append([hass.writes[-1] if hass.writes else None, + c.target, c.inputs_ok, round(c.i_w, 6), + round(c.loop_i_w, 6), round(c.loop_i_lead, 6), + c.loop_reason, c.reason, c.following, + c.maint.phase, len(c.events)]) + hass.n += 1 + return c, hass, pub, ctrl + + +def documents(pv_set: bool) -> dict: + """`{"control": …, "telemetry": …}` for one setting of `pv_entity`.""" + _load() + logging.disable(logging.CRITICAL) + loop = asyncio.new_event_loop() + control, tele = {}, {} + for gname, gof in GRIDS.items(): + for pname, pof in PVS.items(): + key = f"{gname}/{pname}" + c, hass, pub, ctrl = loop.run_until_complete( + _run(gof, pof, pv_set)) + control[key] = { + "ticks": ctrl, + "writes": hass.writes, + "sha": hashlib.sha256( + json.dumps(ctrl, sort_keys=True).encode()).hexdigest(), + } + rows = c.checks() + tele[key] = { + "frames": pub.frames, + "rows": [[r["ok"], r["warn"], r["text"]] for r in rows], + "n_rows": len(rows), + "page": c.status_payload()["rows"], + "level": c.status_payload()["level"], + "banner": c.status_payload()["banner"], + } + # A stopped run, because the "stopped or blind" branch writes its own rows. + c, hass, pub, ctrl = loop.run_until_complete( + _run(GRIDS["EXPORT-6K"], PVS["noon"], pv_set, stop_at=8)) + control["STOPPED"] = {"ticks": ctrl, "writes": hass.writes, + "sha": hashlib.sha256( + json.dumps(ctrl, sort_keys=True).encode() + ).hexdigest()} + tele["STOPPED"] = {"frames": pub.frames, + "rows": [[r["ok"], r["warn"], r["text"]] + for r in c.checks()]} + loop.close() + # The announcement is telemetry too, and it is the surface an install's + # entity registry is keyed on. + tele["ANNOUNCEMENT"] = _announcement(pv_set) + return {"control": control, "telemetry": tele} + + +def _supports_include() -> bool: + """Does THIS tree's `discovery_payloads()` take TEL-11's `include`? + + ⚠️ A PROPERTY OF THE TREE, AND IT IS RECORDED IN `tree` RATHER THAN IN A + DOCUMENT. `release/1.0` has neither the argument nor `optional_keys()`, so + the tool has to run on a tree that predates both or it cannot produce a + baseline at all. Putting the answer in the documents instead would make + `unset` differ between the two trees for a reason that is not behaviour - + which is the "check that goes red for the wrong reason" MUTATION.md keeps + finding - so it goes beside the file hashes, where a reader comparing two + trees is already looking. + """ + import inspect + return ("include" + in inspect.signature(mqtt.discovery_payloads).parameters + and hasattr(mqtt, "optional_keys")) + + +def _announcement(pv_set: bool) -> list: + """The retained discovery set, in announcement order. + + ⚠️ AT `pv_set` FALSE THIS IS THE BARE CALL ON BOTH TREES, which is the + point: an install that never set the option must receive the SAME + announcement from both, and that identity is graded rather than argued. + """ + if pv_set and _supports_include(): + payloads = mqtt.discovery_payloads( + include=mqtt.optional_keys({"pv_entity": PV_ENTITY})) + else: + payloads = mqtt.discovery_payloads() + return [[t, json.dumps(cfg, sort_keys=True)] for t, cfg in payloads] + + +def tree() -> dict: + """sha256 and byte count of every `app/*.py` this run imported from.""" + _load() + out = {"supports_include": _supports_include()} + for f in sorted((HERE / "app").glob("*.py")): + raw = f.read_bytes() + out[f"app/{f.name}"] = [hashlib.sha256(raw).hexdigest(), len(raw)] + return out + + +def _h(obj) -> str: + return hashlib.sha256(json.dumps(obj, sort_keys=True).encode()).hexdigest() + + +def compare(a_path, b_path) -> int: + a = json.loads(Path(a_path).read_text(encoding="utf-8")) + b = json.loads(Path(b_path).read_text(encoding="utf-8")) + + # --- the control, first: two DIFFERENT trees, and one of them reacts ----- + if a["tree"] == b["tree"]: + print("REFUSED: the two runs have the SAME `app/` fingerprint, so they " + "are one tree twice. An identity is evidence only when the two " + "sides could have differed.") + return 2 + moved = [side for side, doc in (("A", a), ("B", b)) + if doc["set"] != doc["unset"]] + if len(moved) != 1: + print(f"REFUSED: `set` differs from `unset` on {moved or 'NEITHER'} " + "side. Exactly one side must react to `pv_entity` - the HEAD. " + "Neither means the instrument cannot see this card's material " + "at all, and both means the baseline is not a baseline.") + return 2 + head, base = (b, a) if moved == ["B"] else (a, b) + hlabel = "B" if moved == ["B"] else "A" + blabel = "A" if hlabel == "B" else "B" + print(f" HEAD is {hlabel}; baseline is {blabel} (it does not react to " + f"`pv_entity`, which is what a tree without the option does).") + print(f" tree {blabel} {_h(base['tree'])}") + print(f" tree {hlabel} {_h(head['tree'])}") + for what in ("control", "telemetry"): + print(f" {what:<9} {blabel} unset {_h(base['unset'][what])}") + print(f" {what:<9} {hlabel} unset {_h(head['unset'][what])}") + print(f" {what:<9} {hlabel} set {_h(head['set'][what])}") + + rc = 0 + # 1. the acceptance criterion: unset is the baseline, in full. + if head["unset"] == base["unset"]: + print("OK 1/3 UNSET: control AND telemetry are byte-identical to the " + "baseline tree. An install that does not set `pv_entity` sees " + "no change of any kind.") + else: + rc = 1 + for what in ("control", "telemetry"): + for k in sorted(set(head["unset"][what]) | set(base["unset"][what])): + if head["unset"][what].get(k) != base["unset"][what].get(k): + print(f"FAIL 1/3 UNSET {what} MOVED at {k!r}") + # 2. the card's other half: setting it moves no DECISION. + if head["set"]["control"] == head["unset"]["control"]: + print("OK 2/3 SET: the control document is byte-identical with the " + "channel switched on - PV reaches no decision.") + else: + rc = 1 + for k in sorted(head["set"]["control"]): + if head["set"]["control"][k] != head["unset"]["control"][k]: + print(f"FAIL 2/3 CONTROL MOVED at {k!r} when `pv_entity` was " + f"set") + # 3. …and the instrument could have seen it. Without this the first two + # are an identity measured against a document that reaches nothing. + if head["set"]["telemetry"] != head["unset"]["telemetry"]: + n = sum(1 for k in head["set"]["telemetry"] + if head["set"]["telemetry"][k] + != head["unset"]["telemetry"].get(k)) + print(f"OK 3/3 CONTROL OF THE CONTROL: the telemetry document DOES " + f"move when the option is set ({n} stimulus cells differ), so " + f"1/3 and 2/3 are identities the instrument could have broken.") + else: + rc = 2 + print("REFUSED 3/3: the telemetry document did NOT move when " + "`pv_entity` was set. This instrument cannot see the card's " + "material, so nothing above it means anything.") + return rc + + +if __name__ == "__main__": + if len(sys.argv) == 4 and sys.argv[1] == "--compare": + sys.exit(compare(sys.argv[2], sys.argv[3])) + print(json.dumps({ + "tree": tree(), + "stimulus": {"grids": sorted(GRIDS), "pv": sorted(PVS), + "cycles": CYCLES}, + "unset": documents(False), + "set": documents(True), + }, sort_keys=True, indent=1)) diff --git a/goodwe_controller/translations/en.yaml b/goodwe_controller/translations/en.yaml index e31a503..ae790bd 100644 --- a/goodwe_controller/translations/en.yaml +++ b/goodwe_controller/translations/en.yaml @@ -84,6 +84,9 @@ configuration: estop_fitted: name: "Hardware · RS485 emergency stop is fitted" description: "Whether the RS485 emergency stop is physically installed. It only drives the warning banner on the status page; it changes no control behaviour." + pv_entity: + name: "Hardware · PV production entity (W)" + description: "Optional, and MEASUREMENT ONLY - it changes no control behaviour whatsoever. Point it at your solar inverter's own power sensor (any brand Home Assistant reads; it does not have to be on the RS485 bus) and the add-on records production beside the grid and battery figures, so a day of unstored surplus can be told apart from a day with no sun. POSITIVE MUST MEAN PRODUCING. The unit is read from the entity itself - W, kW and the rest of Home Assistant's power units - so a sensor with no unit is refused rather than guessed at, and the status page says so. Leave it empty and nothing is read, nothing is published and nothing is shown." meter_source: name: "Meter · Where the meter reading comes from" description: "'off' keeps the single Home Assistant entity below, which is what an existing installation runs on until you choose otherwise. ha_dsmr subscribes to the DSMR integration; mqtt_p1 reads an MQTT topic; ha_signed reads one signed entity, which is what a HomeWizard P1 publishes; homewizard_local polls the meter's own local API and skips Home Assistant entirely. ⚠️ Only homewizard_local can tell 'the meter has not answered' apart from 'the value has not changed', so it is the only source on which the frozen-meter alarm means anything." diff --git a/goodwe_controller/translations/nl.yaml b/goodwe_controller/translations/nl.yaml index 440dce7..91174d1 100644 --- a/goodwe_controller/translations/nl.yaml +++ b/goodwe_controller/translations/nl.yaml @@ -64,6 +64,9 @@ configuration: estop_fitted: name: "Hardware · RS485-noodstop is geplaatst" description: "Of de RS485-noodstop fysiek geïnstalleerd is. Dit stuurt alleen de waarschuwingsbalk op de statuspagina aan en verandert niets aan de regeling." + pv_entity: + name: "Hardware · Zonneproductie-entiteit (W)" + description: "Optioneel, en UITSLUITEND EEN METING - het verandert niets aan de regeling. Verwijs naar de vermogenssensor van je omvormer (elk merk dat Home Assistant kan lezen; hij hoeft niet op de RS485-bus te zitten) en de add-on registreert de productie naast de meter- en batterijwaarden, zodat een dag met niet-opgeslagen overschot te onderscheiden is van een dag zonder zon. POSITIEF MOET PRODUCEREN BETEKENEN. De eenheid wordt van de entiteit zelf gelezen - W, kW en de overige vermogenseenheden van Home Assistant - dus een sensor zonder eenheid wordt geweigerd in plaats van geraden, en de statuspagina zegt waarom. Laat leeg en er wordt niets gelezen, niets gepubliceerd en niets getoond." meter_source: name: "Meter · Waar de meterstand vandaan komt" description: "'off' houdt de losse Home Assistant-entiteit hieronder aan, waar een bestaande installatie op draait tot u iets anders kiest. ha_dsmr luistert mee met de DSMR-integratie; mqtt_p1 leest een MQTT-topic; ha_signed leest één entiteit met teken, wat een HomeWizard P1 publiceert; homewizard_local bevraagt de lokale API van de meter zelf en slaat Home Assistant helemaal over. ⚠️ Alleen homewizard_local kan 'de meter heeft niet geantwoord' onderscheiden van 'de waarde is niet veranderd', en is dus de enige bron waarop het alarm voor een vastgelopen meter iets betekent."