diff --git a/goodwe_controller/DECISIONS/TEL-11-pv-production.md b/goodwe_controller/DECISIONS/TEL-11-pv-production.md new file mode 100644 index 0000000..a941683 --- /dev/null +++ b/goodwe_controller/DECISIONS/TEL-11-pv-production.md @@ -0,0 +1,351 @@ +# TEL-11 — the PV production channel + +*What was decided, what it was decided against, and what the evidence was.* + +## 0. The problem, in OPT-10's numbers + +Over sixteen days at the reference site the pack reached 90 % SoC on **two** of +them, and **20.5 kWh of surplus left the site while the pack still had room** — +**12.6 kWh** of it with the battery **not charging at all**. That single +measurement is upstream of most of what else was found that week: the nightly +floor, SAFETY-37's misdiagnosis, the SAFETY-57 deadlock, a 2 h 47 min saturation +freeze. + +It cannot be **attributed**, because **PV production is recorded nowhere in this +system**. Three causes need three different fixes: + +| cause | fix | +|---|---| +| there was no irradiance | none — the site behaved correctly | +| production was there, charge power was limited | a limit, somewhere | +| production was there, the pack had room, the optimiser declined | **the only software defect** | + +Guessing between them is how an optimiser gets tuned to solve a cloud problem. +**This card adds the channel and does not do the attribution.** Ship it, let it +record a week, OPT-10 picks it up with data. + +## 1. It is an HA entity, not a register — and that is a product decision + +The RS485 bus this add-on speaks carries the **battery** inverter it drives. On +nearly every site the PV inverter is a different box from a different vendor, +reporting into Home Assistant through its own integration; it is **not** on this +bus and there is no reason it would be. So the signal comes from an entity: +`pv_entity` takes whatever `sensor.*` the site's PV integration provides, any +brand HA can read. + +Reading a register would have been cheaper on one house and wrong for a fleet — +the trade [[product-not-one-house]] names. + +## 2. The sign convention: **positive is production**, and there is no `pv_invert` + +### The convention is derived, not assumed + +Two sign rules already ship, and both are recorded in `DOCS.md`'s *Sources* +table rather than inferred here: + +* `meter_entity` — **positive must mean importing** (`meter_invert` flips it); +* `batt_entity` — **`+` = discharging** (`batt_invert` flips it). + +Both are *"positive = power arriving at the house's AC bus"*. Production arrives +there too, so **positive = producing** is the continuation of the shipped +convention rather than a fresh choice. + +### Why an invert flag would be over-building + +The question is not "could a sensor be reversed" — it could. It is what a flag +would **buy**, and the answer is nothing, for two reasons that do not apply to +the two keys it would sit beside: + +1. **Those two quantities are read by the control path; this one is read by + nothing.** `inv.batt` feeds SAFETY-30's per-unit sign check and SAFETY-31's + circulating-power sensor (`_read_unit()`'s own docstring says so), so a + reversed battery sensor hands the site a plausible number it then **acts** + on — a latch that trips when it should not, or does not trip when it should. + `pv_w` reaches no decision at all (§4), so a reversed PV sensor cannot + mislead one. +2. **Production is unipolar, so the error is self-announcing.** Battery power + and grid power are legitimately negative half the time, which is exactly why + a flipped sign is invisible in them. A PV array does not consume: a reversed + sensor reads **−4 kW at noon**, in the first day of the very history this card + exists to create. An invert flag would let an installer *hide* that by + flipping it to something that looks right, and a second place for the sign to + be wrong is a second thing to get wrong. + +So: the convention is documented in `config.yaml`, both translations, `DOCS.md` +and `app/pv.py`, the value is published **exactly as the sensor reports it**, +and there is no flag. `test_pv.py` part 1 pins `-8 W` through unchanged. + +### And no negative-value alarm either + +The tempting middle path — warn when the reading is negative — needs a +threshold, because a PV inverter's own standby draw makes a small negative +reading at night entirely ordinary. Any threshold here would be a number nobody +measured, in a card whose whole subject is not fabricating numbers. The +published row is the detector. + +## 3. Absence is never zero, and it is not "dropped" either + +This is the constraint that shaped the publication, and the obvious +implementation fails it. + +`MqttPublisher.publish()` **skips** a `None` value. That is right for +`circulation` — its note in `main.py` says a missing per-unit reading should +leave the last figure standing rather than publish a zero that reads as health — +and it is **exactly wrong here**: a skipped row leaves Home Assistant holding +the last number it was told, *forever*, and the recorder then draws a **flat +line at that value through the outage**. Read back a week later that is +indistinguishable from a real measurement, and if the last value happened to be +small it manufactures precisely the *"there was no surplus to store"* conclusion +the card exists to **test**. + +So TEL-11 publishes the absence, using Home Assistant's own mechanism: + +> `homeassistant/components/mqtt/const.py`: `PAYLOAD_NONE = "None"`. +> `homeassistant/components/mqtt/sensor.py`, in the state-message handler: +> `if payload == PAYLOAD_NONE: self._attr_native_value = None; return` — and it +> is tested **before** the numeric branch, so it holds for a row carrying +> `device_class: power`. +> (github.com/home-assistant/core, branch `dev`, retrieved 2026-09-24. The +> shipped MQTT-sensor documentation says the same thing in its `state_topic` +> entry: *"a 'None' value will set the sensor to an unknown state"*.) + +The entity therefore goes to **`unknown`** and the recorder stores a **gap**. +`mqtt.ABSENT_AS_UNKNOWN` is a one-key set and `test_pv.py` part 3 asserts both +that the pv row gets this treatment **and that `circulation` still does not** — +widening the rule silently would be a behaviour change to another card's row. + +⚠️ **What was NOT built, and why.** A per-entity `availability` block +(`availability_mode: all`, an extra retained topic) would also produce a +distinguishable state — `unavailable` rather than `unknown` — but it needs a +second topic, a second retained message, a change to `_payload()`'s +byte-order-sensitive key construction, and it survives restarts only if the +retention is right. The payload route is one branch in `publish()` and one +constant, and it is graded as **bytes on the topic** rather than as a claim +about HA's availability handling. + +### Three states, not two + +| state | frame | wire | page | +|---|---|---|---| +| no source (`pv_entity` unset) | no `pv` key | no topic, no entity | no row | +| no reading | `pv: None` | `goodwe_ctl/pv` = `None` | `PV —` + a caution naming the reason | +| zero watts | `pv: 0.0` | `goodwe_ctl/pv` = `0.0` | `PV 0 W` | + +A failed read also **clears** the last good value (`_read_pv()` assigns on every +call), so a reading never outlives its own sensor. `test_pv.py` part 2 drives a +good read followed by a dead one, because a fixture that answers the same thing +forever cannot reach that failure — MUTATION.md habit 9's shape. + +## 4. It must not become a control input, and that is graded twice + +### In the suite + +`test_pv.py` part 6 drives twelve cycles of the real `Controller.cycle()` +against a 6 kW export and asserts the commands written to the inverter are +**bit-identical** across `unset` / `no reading` / `0 W` / `8 kW` of production, +with a precondition check that the stimulus commands something at all (habit 1). +`inputs_ok` is asserted not to move either: a broken PV integration is **not** a +missing input, because nothing computes with it. + +A computed census (habit 12 — the scope is derived from the tree, not typed) +then reports which of the 33 modules under `app/` mention the material in any +spelling, comments included. It is **three**: `pv.py`, `main.py`, `mqtt.py`. The +scanner's own red direction is exercised on every run. + +### Across trees, by hash + +`tools/safety56_behaviour_trace.py` is the pattern; `tools/tel11_pv_trace.py` is +this card's version of it, and **its control clause had to be different**, which +is worth stating because the difference is the interesting part. + +SAFETY-56 changed wording and had to show behaviour did not move, so its +`--compare` **refuses** unless the `sentences` halves **differ** — otherwise the +two runs might be one tree twice. TEL-11's headline claim is the opposite shape: +with the option unset, **nothing** differs. "The documents are identical" is the +*result*, so it cannot also be the control. The two things proved instead are: + +* the two runs came from **different trees** (`app/*.py` sha256 + byte count, + compared, and an identity is **refused**); +* **exactly one** side reacts to `pv_entity` — that side is the HEAD, the other + is a tree that does not have the option. Neither, or both, is refused. + +It drives 41 stimulus cells (5 grid worlds × 8 PV schedules, plus a run stopped +part-way) and prints two documents per setting: `control` — every command +written, every `self.target`, `inputs_ok`, the loop's three fields, the +arbiter's resolution, the maintenance phase, tick by tick, **with no TEL-11 +material in it** — and `telemetry`, which is everything the card is allowed to +move. + +Measured, in the CI image, on purged trees: + +``` + HEAD is B; baseline is A (it does not react to `pv_entity`). + tree A 73296909170ec789237d6e704cce9ab0d9f83b3df1495a14d2497285c3041a12 + tree B d5011b6a18e2e3803dc09cac0214b0378fa4c7388fc0ad8ad89b832a5c92af48 + control A unset 846a10a5b89e321086d535825925721e2f791012e8405948d4fd38d3f9d0be5f + control B unset 846a10a5b89e321086d535825925721e2f791012e8405948d4fd38d3f9d0be5f + control B set 846a10a5b89e321086d535825925721e2f791012e8405948d4fd38d3f9d0be5f + telemetry A unset a443a6ceaffe072b44bcd5f2ad946c1ed6c9e01b69e278eca631f7f4d7892164 + telemetry B unset a443a6ceaffe072b44bcd5f2ad946c1ed6c9e01b69e278eca631f7f4d7892164 + telemetry B set 4e6d1cb289a957223ad0205f732a6fec6110ce1f2032c98ad60694e67625274f +OK 1/3 UNSET: control AND telemetry are byte-identical to the baseline tree. +OK 2/3 SET: the control document is byte-identical with the channel switched on. +OK 3/3 CONTROL OF THE CONTROL: the telemetry document DOES move when the + option is set (42 stimulus cells differ). +``` + +Note what clause 1 claims and clause 2 does not: **unset** is identical in +`control` *and* `telemetry`, which is stronger than "the control path is +untouched" — an install that does not set the option sees no change of any kind, +including its announcement and its status rows. **Set** is identical in +`control` only, which is the whole of "instrumentation, not an input". + +## 5. Genuinely optional — and not a second `price_now_entity` + +`price_now_entity` / `price_avg_entity` are legal **only** while +`price_provider` is `off`, and the add-on **refuses to start** otherwise. That +is the trap this card was told not to rebuild, and it is avoided structurally +rather than by intention: + +* `validate.py` does not mention `pv_entity` at all. It cannot conflict with + anything, because nothing else reads it. +* the MQTT row lives in `OPTIONAL_SENSORS`, not `SENSORS`, and + `discovery_payloads()`'s new `include` argument **defaults to empty**. So a + caller that has never heard of `pv_entity` — every fixture in `test_mqtt.py`, + `tools/rehearse_roster.py`, anything written next year — announces exactly + what it announced before. `p1_age` solves the same problem the other way + round, with `omit`, because it is announced by default and withdrawn when the + meter is off; getting that direction wrong here would have left most installs + with an entity nothing feeds, which is the hazard `omit`'s own note names. +* the read is gated on the option, so an unset install makes **no state call at + all** — not a read that fails quietly. +* there is **no status row** when it is unset. `peak_forecast_entity` nags when + it is unset because an unset forecast leaves a money guard without an input; + this is instrumentation, and a permanent caution on every install that never + asked for a PV channel is the warning spam the card forbids. +* when it **is** set and the read fails, the row is `ok=False, warn=True` — a + **caution**. `status_payload()` builds `NOT READY` out of rows that are + neither `ok` nor `warn`, so a telemetry channel can never put a battery site + into `NOT READY`. + +## 6. The unit is read off the entity, and an unreadable one is refused + +`read_inputs()` guesses kW-or-W for `peak_forecast_entity` with +`abs(raw) < 50`, and that is safe there: nobody's quarter-hour demand forecast +is 50 W. It is **not** safe here — 30 W of real production at dusk is an +ordinary reading, and that heuristic would publish it as 30 kW. + +So `pv.watts()` reads the entity's own `unit_of_measurement` and converts +against Home Assistant's own `UnitOfPower` table (`mW`, `W`, `kW`, `MW`, `GW`, +`TW`; every factor an exact power of ten). A sensor with **no** unit, or a unit +outside that table, is **refused by name** and the status page says which. The +refusal is loud; a guess would be silent, and a factor of 1000 in the one +channel that exists to be believed a week later is the worse failure. + +⚠️ `BTU/h` is in HA's enum and is deliberately **not** in the table: its factor +is a measured conversion rather than a decimal shift, no PV integration +publishes in it, and a fabricated constant is what this repository's whole +process exists to prevent. It lands in the refusal, by name, and `test_pv.py` +part 1 pins that. + +## 7. The mutation battery + +Every check this card adds was driven by the mutation that defeats the claim it +names. Method: `cp -a` of the tree per mutation into a **unique** directory, +the edit applied there, `__pycache__` purged, the suites run in the CI image +with the tree bind-mounted `:ro`, one file at a time. **The shipped tree is +never edited**, so habit 18's revert hazard — SAFETY-37's harness killed between +the edit and the `cp` that put the file back, and two harnesses sharing one +backup path — is closed by construction rather than by a verified revert. Each +row below records the byte delta, the file's sha256 before and after, and that +the mutated line was **read back off disk** before the run started. + +| # | mutation | file | bytes | sha256 (first 16) before → after | red on | +|---|---|---|---|---|---| +| M1 | the `pv` row moved into `SENSORS`, i.e. announced on every install | `mqtt.py` | −79 B | `328d249b9d698ef6` → `532d3954c023bf3d` | 3 named in `test_mqtt.py`, 1 in `test_pv.py` | +| M2 | `publish()` drops a `None` `pv` row again, so HA keeps the last number | `mqtt.py` | −98 B | `328d249b9d698ef6` → `dd54be458f446475` | 1 named (the acceptance criterion itself) | +| M3 | `ABSENT_AS_UNKNOWN` widened past the one key it was written for | `mqtt.py` | +23 B | `328d249b9d698ef6` → `8fe571ebff3dacf4` | 1 named (`circulation` must still be dropped) | +| M4 | a dead sensor reported as **0 W** instead of an absence | `pv.py` | −1 B | `55ba7471e9995436` → `9eecc34152b7ac12` | 4 named | +| M5 | `_read_pv()` assigns only on a good read — a value outlives its sensor | `main.py` | +46 B | `c4d2cab34e50b5fd` → `189802aa69ce166d` | 2 named | +| M6 | the unit is read and then ignored — a kW sensor published as watts | `pv.py` | −18 B | `55ba7471e9995436` → `ab19ef66c52f9379` | 2 named | +| **M7** | **`self.grid = self.grid − self.pv_w` in `read_inputs()` — PV wired into the control path** | `main.py` | +106 B | `c4d2cab34e50b5fd` → `c38216edb7c5c8db` | **2 named, plus `tel11_pv_trace.py` clause 2/3 on 26 of the 41 stimulus cells** | +| M8 | `optional_keys()` returns `("pv",)` unconditionally | `mqtt.py` | −52 B | `328d249b9d698ef6` → `86ff23efe3088ad8` | 3 named | +| M9 | the PV caution made a `bad` row, i.e. a dead sensor is a `NOT READY` | `main.py` | +1 B | `c4d2cab34e50b5fd` → `45c74cab89943564` | 1 named | +| M10 | the read gate removed — an unset option is read once a second forever | `main.py` | −10 B | `c4d2cab34e50b5fd` → `b67baa8aba3019e8` | 2 named, **plus two crashes** | + +`on_disk = True` for all ten: the mutated line was grepped back out of the file +after the edit and before the run (MUTATION.md reporting rule 2), and no +mutation is size-preserving, so rule 1's `+0 B` caveat does not arise. + +⚠️ **M10's two extra reds are CRASHES, not named checks, and that is recorded +rather than tidied away.** Every `Hass` double in `test_inverter.py` and +`test_telemetry.py` predates this read and has no `state()` method, so an +unconditional PV read is an `AttributeError` rather than a failed assertion — +habit 20's shape, a red with no name attached. Fixing those fixtures would be a +change to what *their* checks report and belongs to whoever next touches them; +this card's own fixture has `state()` and grades the property by name. + +⚠️ **M7's trace limb is where the two halves meet.** The mutant moves the +control document on 26 of 41 cells — every cell whose PV schedule produces a +READING. The five `none` cells and the five `zero` cells do **not** move, which +is correct and is worth seeing: subtracting nothing, and subtracting zero, +change nothing. A battery that only drove a dead sensor would have reported this +mutation as a survivor for the third time. + +The clean tree those `before` hashes belong to: `app/main.py` +`c4d2cab34e50b5fd`, `app/mqtt.py` `328d249b9d698ef6`, `app/pv.py` +`55ba7471e9995436`. + +### ⚠️ M7 was a SURVIVOR on the first battery, and the fixture was the reason + +The card's **central** mutation — `self.grid = self.grid - self.pv_w` in +`read_inputs()`, i.e. exactly the defect the whole card forbids — came back +**green** on a six-file run at **+106 B**, on disk, purged, genuinely running. + +The property did not hold; the fixture could not reach it. `test_pv.py` part 6 +drove a **6 kW export over 12 cycles**, which puts the loop **against its +rail**: the command moves `slew_w` every cycle and −6000 W and −14 000 W of +apparent error produce the **same ramp, tick for tick**. MUTATION.md habit 23, +with a saturated actuator in place of a coinciding value — and the habit's list +of no-op-prone constructs is all *expressions*, so this is worth its own entry, +which it now has. + +The fixture now drives a **600 W import the loop settles on**, with a +responsive meter (`grid = base − batt`) so the error actually closes, asserts +the resting target is well inside the rail, and asks habit 23's own question as +its own check: **shift the meter by the same 8 kW and confirm the commands +really would have moved.** On the first fixture that question answers *no*, in +one line. Every number in the table above is from the battery re-run against +the fixed tree. + +⚠️ **A red count is a property of the suite as much as of the mutation.** These +name **this** tree at this commit; a later card that adds or re-bases a check +moves the number with the mutation untouched. Re-measure rather than inherit. + +## 8. What none of this covers, with shapes + +* **Home Assistant itself.** That `None` renders as `unknown` and that the + recorder keeps a gap is read out of HA's source and quoted in §3; no HA runs + in this suite. What is graded here is the bytes this add-on publishes. +* **Whether the operator's sensor tells the truth.** A sensor wired backwards, + or reporting DC where AC was meant, or a string instead of the array, is + published faithfully. §2 is the argument for why that is the right failure + mode; it is not a claim that it cannot happen. +* **A roster of more than one.** The row is a **site** row — there is one PV + array as far as this add-on is concerned — so nothing per-unit exists to + separate, and every fixture is the shipped roster of one. +* **The attribution.** This card records production. Whether the 20.5 kWh was + cloud, a limit or the optimiser is OPT-10's, and it needs a week of this + channel's history before it can be asked. +* **`tools/`, the firmware and the ingress HTML** are outside part 6's census. +* **The identity is only as wide as the stimulus.** `tel11_pv_trace.py` prints + its 41 cells with the documents for exactly that reason. + +## 9. No `CHANGELOG.md` entry + +`test_version_gate.py` 11b refuses a newest `## ` heading that is not +`config.yaml`'s `version:`, and opening an `## Unreleased` section would mean +taking a packaging decision this card does not get to take. The operator account +is in `DOCS.md`; the release note belongs to the next PKG ticket, which should +say that **0.10.1 → next** adds one optional key, announces no new entity on an +install that does not set it, and changes no control behaviour. diff --git a/goodwe_controller/DOCS.md b/goodwe_controller/DOCS.md index 14df547..faba10a 100644 --- a/goodwe_controller/DOCS.md +++ b/goodwe_controller/DOCS.md @@ -139,6 +139,7 @@ either direction, and the check is mutation-tested both ways. | `batt_invert` | | Flip if needed | | `capacity_kwh` | | This pack's usable capacity. Read only to divide the site target between packs by **energy** headroom — see below | | `setpoint_entity` | yes | The ESPHome `number.*_goodwe_setpoint_w` | +| `pv_entity` | | Optional. Your **solar** inverter's power sensor, `+` = producing. Measurement only — it changes no control behaviour. See [PV production](#pv-production-pv_entity-tel-11) | Use the ESP32's readings rather than the inverter's cloud or dongle sensors: those serve cached values, and a stale reading here ends the maintenance charge @@ -413,6 +414,97 @@ and `Inverter B` — are two names to the validator and one id once slugged. The add-on gives the second one a `_2` suffix so nothing collides, but the ids you get then depend on roster order. Give your inverters plainly different names. +### PV production (`pv_entity`, TEL-11) + +**Optional, off by default, and it changes no control behaviour whatsoever.** +Set it and the add-on records what your solar is producing beside what the +meter and the battery are doing. Leave it empty and the add-on reads no entity, +announces no entity, publishes no topic and shows no row — which is what every +install that upgrades into this version does. + +#### Why it is a Home Assistant entity and not a register + +This add-on's RS485 bus carries the **battery** inverter it drives. On nearly +every site the PV inverter is a different box from a different vendor, on its +own connection, reporting into Home Assistant through its own integration. So +`pv_entity` takes whatever `sensor.*` your PV integration already provides — +any brand Home Assistant can read. It does not have to be on the bus, and there +is no assumption anywhere that it is. + +#### What it is for + +OPT-10 measured this site over sixteen days: the battery reached 90 % on **two +of them**, and **20.5 kWh of surplus left the site while the pack still had +room** — 12.6 kWh of it with the battery not charging at all. Three different +things produce that number and they need three different fixes: + +* there was no irradiance to store — nothing to fix; +* production was there and something limited the charge power; +* production was there, the battery could take it, and the optimiser declined. + +Only the third is a software defect, and **none of them can be told apart +without a record of production.** This entity is that record. The attribution +itself is OPT-10's, with a week of history behind it. + +#### The sign, and why there is no `pv_invert` + +**Positive must mean producing.** That is the same convention as the two sign +rules the add-on already ships — `meter_entity` is positive when **importing** +and `batt_entity` is positive when **discharging**, both of which are "power +arriving at the house" — and production arrives there too. + +`meter_invert` and `batt_invert` exist because those two quantities are +**bipolar and are read by the control path**: a reversed battery sensor is a +perfectly plausible number that the per-inverter sign failsafe then acts on. +Production is unipolar and is read by **nothing**, so a reversed sensor here +cannot mislead a decision — and it announces itself immediately as −4 kW at +noon in the very history this entity creates. A flag would only hide it, so +there is not one. If your sensor really is inverted, fix it where it is +produced. + +#### The unit is read off the entity + +The add-on reads your entity's own `unit_of_measurement` and converts: `mW`, +`W`, `kW`, `MW`, `GW`, `TW` — Home Assistant's own power units. A sensor with +**no** unit, or with a unit that is not one of those, is **refused**, and the +status page says which entity and which unit. That is deliberate: there is no +magnitude test that can separate 30 W of dusk from 30 kW of noon, so a guess +here would be a silent factor of 1000 in the one channel that exists to be +believed a week later. + +#### ⚠️ Absence is never zero + +| what happened | `sensor.goodwe_pv_power` | on the status page | +|---|---|---| +| `pv_entity` not set | the entity does not exist | no row at all | +| set, sensor answered `4100 W` | `4100.0` | `PV 4 100 W` | +| set, sensor answered `0 W` | `0.0` | `PV 0 W` | +| set, sensor unavailable / missing / bad unit | `unknown` | `PV —`, plus a caution naming the reason | + +The last two rows are the point. A dead sensor publishes Home Assistant's own +"no value" payload, so the entity goes to **`unknown`** and the recorder stores +a **gap** — not a flat line at the last number it saw. Reading a week of +history back, *"no PV was reported"* and *"PV reported 0 W"* are different +pictures, which is exactly the distinction the whole card exists to preserve. + +A failed read also **clears** the previous good one: the value never outlives +its own sensor. + +#### ⚠️ What it does NOT do + +It is **instrumentation**. It is not an input to the control law, the +allocator, the arbiter, the peak guard, the maintenance cycle or any watchdog; +it is not part of `inputs_ok`, so a PV integration that breaks cannot stop the +battery being regulated; and its status row is a **caution**, never a +`NOT READY`. Wiring production into the control law would be a behaviour change +on a system driving a real battery — and it would destroy the measurement, +because the loop would then be reacting to the thing being measured. + +`tools/tel11_pv_trace.py` is what says so rather than claims it: it drives 41 +stimulus worlds through the shipped controller and hashes a document of control +decisions containing none of this material, on this tree and on `release/1.0`, +with the option set and unset. + ### P1 meter ingestion `meter_entity` above expects one signed sensor, which usually means a template diff --git a/goodwe_controller/MUTATION.md b/goodwe_controller/MUTATION.md index 14fec18..c9e83a2 100644 --- a/goodwe_controller/MUTATION.md +++ b/goodwe_controller/MUTATION.md @@ -896,6 +896,33 @@ Cheap habits fall out of it, and all of them are now used in this repository: disjointness and the suite pins **that** - at 0, 100, 249, 250, 251 and 1300 W - rather than an order nothing depends on. + ⚠️ **SECOND INSTANCE, TEL-11, AND THE NO-OP WAS A SATURATED RAIL RATHER + THAN A VALUE.** The card's whole subject is that a PV reading must never + reach the control law, so its central mutation is + `self.grid = self.grid - self.pv_w` in `read_inputs()` - **+106 B**, on + disk, purged, genuinely running - and it came back a **SURVIVOR** of a + six-file battery. The property held in no sense: the fixture drove a 6 kW + export over 12 cycles, which puts the loop **against its rail**, and a + command moving `slew_w` every cycle produces the SAME ramp for -6000 W and + for -14000 W of apparent error. The mutant ran and could not change the + answer. + Two things generalise past PV: + * **A SATURATED ACTUATOR IS A VALUE AT WHICH EVERY INPUT COINCIDES.** The + habit-23 list above names `or`, `and`, truthiness, default arguments and + `getattr` fallbacks - all *expressions* with a coinciding value. A rail, + a clamp, a slew limit, a deadband and a latched alarm are the same hazard + in the **plant**: they are where a controller's output stops being a + function of its input. A fixture that parks there cannot separate any two + inputs, and this repository's control fixtures park there by default, + because the dramatic stimulus is the saturating one. + * **SO THE PRECONDITION IS NOT "DOES IT COMMAND SOMETHING", IT IS "DOES IT + REST SOMEWHERE THE MUTATION COULD MOVE IT FROM".** `test_pv.py` part 6 + now drives a 600 W import the loop SETTLES on, asserts the resting target + is well inside the rail, and asks habit 23's own question as a check: + shift the METER by the same 8 kW and confirm the commands really would + have moved. On the first fixture that question answers NO, in one line, + and it is the line that turns the identity into evidence. + And the mutation-table consequence: **a variant is not covered because a near-miss of it is.** DEV-02's table listed the *actuator* variant of the per-unit claim defect (`device`/`supervised` filed without `inv`), which the diff --git a/goodwe_controller/app/main.py b/goodwe_controller/app/main.py index b853a79..67d3bee 100644 --- a/goodwe_controller/app/main.py +++ b/goodwe_controller/app/main.py @@ -62,10 +62,11 @@ from .inverter import (LINK_STALE_S, UNIT_A, build_roster, from .maintenance import ( IDLE, POWER_MOVING, MaintConfig, Maintenance, arbitrate as maint_arbitrate, ) -from .mqtt import (MqttPublisher, announced_units, unit_slugs, +from .mqtt import (MqttPublisher, announced_units, optional_keys, unit_slugs, unit_state_key) from . import peakguard from . import packshare +from . import pv as pv_read from . import shadow from .p1 import P1Ingest, build_source, is_enabled from .price import Uncertainty, price_option, provider_configured @@ -872,6 +873,22 @@ class Controller: self.loop_i_lead = 0.0 self.loop_reason = "not-running" self.loop_w = None # what the loop asked for last cycle, or None + # --- TEL-11: the site's PV production, and NOTHING READS IT ---------- + # ⚠️ INSTRUMENTATION. It is published, it is shown on the status page, + # and it reaches no decision: not `inputs_ok`, not the allocator, not + # the arbiter, not a claim, not a watchdog. Wiring it into the control + # law would be an unrequested behaviour change on a system driving a + # real battery AND would destroy the measurement, because the loop would + # then react to the quantity being measured. `tools/tel11_pv_trace.py` + # is the instrument that says so, on both trees and both settings. + # + # ⚠️ `None` IS A READING THAT DOES NOT EXIST, AND IT IS SEEDED AS ONE. + # Nothing here ever becomes 0.0 by default; see `app/pv.py`. + self.pv_w = None + # Why there is no reading, in words the page prints - or, on a good + # read, the UNIT the value came in. Never empty. + self.pv_why = ("not configured" if not self.pv_entity + else "not read yet") # Did the grid-following loop actually win the LAST arbitration? That is # the question SAFETY-02 has to ask about the command standing on the # wire right now, so it is recorded when the arbitration happens rather @@ -1651,6 +1668,38 @@ class Controller: for inv in self.inverters) # -- io ------------------------------------------------------------------ + @property + def pv_entity(self) -> str: + """The configured PV production entity, or `""`. TEL-11. + + ⚠️ ONE READING OF THE OPTION, and `mqtt.optional_keys()` is graded + against it: the thing that decides whether the row is ANNOUNCED and the + thing that decides whether it is READ have to be the same predicate, or + the add-on ends up with an entity nothing feeds or a topic nothing + declared. `announced_units()` in mqtt.py is the same argument for the + roster, in its own words. + """ + return str(self.o.get("pv_entity") or "").strip() + + async def _read_pv(self) -> None: + """This cycle's PV production, or an ABSENCE with a reason. TEL-11. + + ⚠️ IT ASSIGNS ON EVERY CALL, INCLUDING THE FAILING ONES. Leaving the + last good number standing through an outage is what `circulation` does + on purpose - there, a stale figure beats a zero that reads as health - + and it is exactly wrong here: this row is read back as HISTORY, and a + value that persists through an outage is indistinguishable from a real + one at the moment somebody is trying to work out whether the sun was + shining. + + ⚠️ `hass.state()` RATHER THAN `hass.number()`, because the UNIT is half + the reading. See `pv.watts()`: a PV integration may publish kW, and + there is no magnitude heuristic that can separate 30 W of dusk from + 30 kW of noon. + """ + self.pv_w, self.pv_why = pv_read.watts( + await self.hass.state(self.pv_entity)) + async def read_inputs(self, now_mono: float | None = None) -> None: """Every input this cycle needs, and SAFETY-37's freshness verdict. @@ -1682,8 +1731,26 @@ class Controller: # names for the WRITE path, one function up. Within a unit the order is # unchanged (SoC, then battery power), so at the shipped roster of one # this is the same two calls in the same order. - await asyncio.gather(*(self._read_unit(inv, now_mono) - for inv in self.inverters)) + # + # --- TEL-11: AND THE PV READ RIDES IN THIS SAME GATHER --------------- + # ⚠️ IN THE GATHER RATHER THAN ON A LINE OF ITS OWN, AND THAT IS THE + # WHOLE "OBSERVABILITY MUST NOT BE ABLE TO DELAY CONTROL" DECISION. + # This coroutine feeds the ESP32 heartbeat; a sequential read would add + # its own worst case - `hass.state()`'s 10 s timeout - to every cycle, + # so a hung PV sensor at a client site would push a healthy battery + # towards layer 1's ~30 s veto for the sake of a telemetry row. + # Gathered, its latency OVERLAPS reads that already happen and the + # added worst case is zero. `peak_forecast_entity` below is read + # serially and stays that way because it is a CONTROL input (the + # capacity guard sizes off it), so the site is entitled to wait for it. + # + # ⚠️ AND AN UNSET `pv_entity` APPENDS NOTHING. Not a coroutine, not an + # HTTP call, not a log line: at the shipped default this gather is the + # same list of the same coroutines it has always been. + _reads = [self._read_unit(inv, now_mono) for inv in self.inverters] + if self.pv_entity: + _reads.append(self._read_pv()) + await asyncio.gather(*_reads) if o.get("peak_forecast_entity"): raw = await self.hass.number(o["peak_forecast_entity"]) # Accept kW or W - nobody's quarter-hour forecast is 50 W. @@ -4397,6 +4464,24 @@ class Controller: "loop_lead": round(self.loop_i_lead, 1), "loop_reason": self.loop_reason, } + # --- TEL-11 ---------------------------------------------------------- + # ⚠️ THE KEY IS PRESENT WITH THE VALUE `None` WHEN THERE IS NO READING, + # AND ABSENT ENTIRELY WHEN THERE IS NO SOURCE. Those are two different + # states and the publisher treats them as two: `mqtt.ABSENT_AS_UNKNOWN` + # turns the first into HA's own `None` payload, so the entity's state + # goes to `unknown` and the recorder records a GAP - which is how a + # reader of a week of history tells "no PV reported" from "PV reported + # 0 W". The second never reaches the broker at all, because an install + # that never set `pv_entity` has no such entity announced. + # + # ⚠️ AND THE GATE IS `optional_keys()`, THE SAME EXPRESSION THE + # ANNOUNCEMENT USES. Written as a second predicate here the two could + # disagree, and both directions are bad - an announced entity nothing + # feeds, or a topic nothing declared. `announced_units()` two blocks + # down is the same discipline for the roster. + if "pv" in optional_keys(self.o): + values["pv"] = (None if self.pv_w is None + else round(self.pv_w, 1)) # ⚠️ ONLY when P1 ingestion is actually running. The ESP32's stale-input # watchdog subscribes to sensor.goodwe_p1_sample_age - see the entity-id # note in mqtt.py; the composed name is not the object_id - and forces layer-1 @@ -4914,6 +4999,34 @@ class Controller: # is both is a check nobody reads. out.append({"ok": False, "warn": True, "text": "no peak forecast: maintenance charging is not capacity-capped"}) + # --- TEL-11 ---------------------------------------------------------- + # ⚠️ NO ROW AT ALL WHEN `pv_entity` IS UNSET, AND THAT IS A DELIBERATE + # DIFFERENCE FROM THE PEAK-FORECAST CAUTION DIRECTLY ABOVE. That one + # nags because an unset forecast entity leaves a MONEY GUARD without an + # input; this one is pure instrumentation, and a permanent caution on + # every install that never asked for a PV channel is exactly the + # "no warning spam, no degraded status row" the card forbids. + # + # ⚠️ AND WHEN IT IS SET, A FAILED READ IS `ok=False, warn=True` - a + # CAUTION, never `bad`. `status_payload()` builds "NOT READY" from rows + # that are neither ok nor warn, and a telemetry channel that could put a + # battery site into NOT READY would be observability with a foot in the + # control path. The reason comes from `pv.watts()` rather than being + # re-derived here, so the page cannot disagree with the reading. + if self.pv_entity: + if self.pv_w is None: + out.append({"ok": False, "warn": True, + "text": f"PV production: no reading from " + f"{self.pv_entity} - {self.pv_why}"}) + else: + # habit 14: the value sits inside its own literal text, in its + # own slot, beside the entity it was read from and the unit it + # was read in - so a transposition of any two of the three is + # visible in the sentence rather than merely present in it. + out.append({"ok": True, "warn": False, + "text": f"PV production: {self.pv_w:.0f} W from " + f"{self.pv_entity} (reported in " + f"{self.pv_why})"}) return out def status_payload(self) -> dict: @@ -5033,6 +5146,13 @@ class Controller: # identically zero and a permanent "0 W" row would be noise. *(() if len(self.inverters) <= 1 else ((("Circulating", w(self.site_circulating_w)),))), + # TEL-11, and ONLY when a source is configured - see `checks()`. + # `w(None)` renders "—", which is how this row says "no reading" + # rather than "0 W". It sits beside Grid and Battery because the + # question it answers is the one those two cannot: at "Grid 0 W, + # Battery 0 W" the site is either becalmed or throwing away every + # watt it makes, and only this row separates them. + *(() if not self.pv_entity else ((("PV", w(self.pv_w)),))), ("State of charge", "—" if self.pack_floor_soc is None else f"{self.pack_floor_soc:g} %"), ("Commanded", w(self.target)), @@ -5306,6 +5426,11 @@ async def amain() -> None: broker.get("username") if broker else None, broker.get("password") if broker else None, omit=() if is_enabled(opts) else ("p1_age",), + # TEL-11. ⚠️ THE SHIPPED PREDICATE, NOT A SECOND READING OF THE + # OPTION - `Controller.publish()` gates the row on this same + # call. An install that never set `pv_entity` announces exactly + # what it announced before: no topic, no entity, no row. + include=optional_keys(opts), # TEL-07. ⚠️ THE NAMES, NOT A ROSTER. `build_roster()` here # would construct a SECOND set of `Inverter`s off the same # Store, minutes before `Controller.__init__` constructs the diff --git a/goodwe_controller/app/mqtt.py b/goodwe_controller/app/mqtt.py index a3c1115..a121af2 100644 --- a/goodwe_controller/app/mqtt.py +++ b/goodwe_controller/app/mqtt.py @@ -117,6 +117,62 @@ SENSORS = [ ("loop_reason", "goodwe_loop_reason", "Loop reason", None, None, None, "mdi:sigma"), ] +# --- TEL-11: rows that exist ONLY when the operator configured a source ------ +# +# ⚠️ A SECOND LIST RATHER THAN A TENTH ENTRY IN `SENSORS`, AND THE DEFAULT IS +# WHAT MAKES IT ONE. `SENSORS` is announced by every caller of +# `discovery_payloads()`; `OPTIONAL_SENSORS` is announced only by a caller that +# ASKS, so a caller that has not been taught about `pv_entity` - `test_mqtt.py`'s +# fixtures, `tools/rehearse_roster.py`, anything written next year - announces +# exactly what it announced before. `p1_age` solves the same problem the other +# way round, with `omit`, because `p1_age` is announced by DEFAULT and withdrawn +# when the meter is off; this one does not exist by default and appears when a +# source for it does. Getting that direction wrong is the hazard `omit`'s own +# note names: an entity that exists and is never fed is a worse signal than one +# that does not exist at all, and MOST installs will never set `pv_entity`. +# +# ⚠️ `state_class: measurement` IS THE WHOLE POINT OF THE ROW. It is what puts +# the number into the recorder's long-term statistics, and a week of statistics +# is what OPT-10 needs to say whether the surplus that left this site was cloud, +# a charge-power limit, or the optimiser declining to charge. +OPTIONAL_SENSORS = [ + ("pv", "goodwe_pv_power", "PV power", "W", "power", "measurement", None), +] + +# ⚠️ HA's OWN "this sensor has no value" PAYLOAD, and the one place this add-on +# spells it. `homeassistant/components/mqtt/const.py` declares +# `PAYLOAD_NONE = "None"` and `mqtt/sensor.py` tests it BEFORE the numeric +# branch - `if payload == PAYLOAD_NONE: self._attr_native_value = None` - so the +# entity's state becomes `unknown` rather than a number. (Read out of +# github.com/home-assistant/core `dev` on 2026-09-24; the shipped MQTT-sensor +# documentation says the same thing in its `state_topic` entry: "a 'None' value +# will set the sensor to an unknown state".) +PAYLOAD_NONE = "None" + +# The keys whose ABSENCE MUST BE PUBLISHED rather than skipped. ⚠️ THIS IS THE +# DIFFERENCE BETWEEN "no PV reported" AND "PV reported 0 W" IN A WEEK OF +# HISTORY, AND WITHOUT IT THE CHANNEL LIES IN THE WORSE DIRECTION. `publish()` +# below drops a `None` - which is right for `circulation`, where the last figure +# standing is better than a zero that reads as health - but a DROPPED pv row +# leaves Home Assistant holding the last number it was told, forever, and the +# recorder then draws a flat line at that value through an outage. Someone +# reading it back would conclude the sun was not shining, which is precisely the +# conclusion this card exists to TEST rather than to manufacture. +ABSENT_AS_UNKNOWN = frozenset({"pv"}) + + +def optional_keys(opts: dict) -> tuple: + """Which `OPTIONAL_SENSORS` rows this configuration has a source for. + + ⚠️ ONE ANSWER TO "IS THE PV ROW LIVE", read by the ANNOUNCEMENT (through + `MqttPublisher`) and by `Controller.publish()`. `announced_units()` above + exists for exactly this reason and says it in full: two spellings of one + predicate is how an entity gets announced that nothing feeds, or fed on a + topic nothing announced. + """ + return ("pv",) if str(opts.get("pv_entity") or "").strip() else () + + BASE = "goodwe_ctl" AVAILABILITY = f"{BASE}/availability" @@ -299,15 +355,23 @@ def _payload(name, object_id, unique_id, state_topic, device, return cfg -def discovery_payloads(units=(), omit=()) -> list: +def discovery_payloads(units=(), omit=(), include=()) -> list: """Every `(topic, payload)` this add-on announces, in announcement order. Split out of `_announce()` so that the acceptance criterion can be graded as a DIFF OF THE ANNOUNCED PAYLOADS against `release/1.0`, rather than as a reading of the code. `_announce()` publishes exactly this list and nothing else, which is itself checked. + + `include` names `OPTIONAL_SENSORS` keys to announce. ⚠️ IT DEFAULTS TO + NOTHING AND THE ROWS GO LAST, so no existing row's POSITION moves under any + configuration - not the site rows, not the per-unit ones. Order is part of + the output here (`_payload()`'s note, and test_mqtt.py part 1 asserts an + ordered LIST precisely because a re-order is a zero-byte edit that + re-announces every entity on a live install). """ omit = set(omit) + include = [k for k in include if k not in omit] out = [] for key, object_id, name, unit, dev_class, state_class, icon in SENSORS: if key in omit: @@ -336,12 +400,19 @@ def discovery_payloads(units=(), omit=()) -> list: unique_id, f"{BASE}/{unit_state_key(slug, key)}", device, unit, dev_class, state_class, icon))) + for key, object_id, name, unit, dev_class, state_class, icon in OPTIONAL_SENSORS: + if key not in include: + continue + out.append(( + f"homeassistant/sensor/{BASE}_{key}/config", + _payload(name, object_id, f"{BASE}_{key}", f"{BASE}/{key}", + DEVICE, unit, dev_class, state_class, icon))) return out class MqttPublisher: def __init__(self, host, port, username=None, password=None, omit=(), - units=()): + units=(), include=()): # `omit` drops sensor keys from discovery entirely. ⚠️ Announcing a # sensor that nothing will ever publish to is not harmless here: # p1_sample_age_s is a watchdog input, and an entity that exists but is @@ -353,6 +424,10 @@ class MqttPublisher: # per-unit is announced, so every caller that has not been taught about # the roster keeps publishing what it published before. self.units = [str(u) for u in units] + # TEL-11. `mqtt.optional_keys(opts)` - the shipped predicate, not a + # second reading of the option. Defaults to empty, so a caller that has + # not been taught about it announces exactly what it announced before. + self.include = [str(k) for k in include] self.enabled = mqtt is not None and bool(host) self.client = None if not self.enabled: @@ -389,7 +464,8 @@ class MqttPublisher: # announcement, so the check that diffs it against `release/1.0` is # diffing what actually goes on the wire - and a payload built here # instead would be one this add-on publishes and no check has ever seen. - for topic, cfg in discovery_payloads(self.units, self.omit): + for topic, cfg in discovery_payloads(self.units, self.omit, + self.include): self.client.publish(topic, json.dumps(cfg), retain=True) self.client.publish(AVAILABILITY, "online", retain=True) @@ -399,7 +475,16 @@ class MqttPublisher: try: for key, value in values.items(): if value is None: - continue + # ⚠️ TEL-11. A DROPPED ROW IS NOT AN ABSENT ROW. Skipping + # leaves Home Assistant holding the last value it was told, + # which is what `circulation` wants (its note in main.py + # says why) and is exactly wrong for a measurement somebody + # will read back as history: a flat line at the last number + # is indistinguishable from a real one. See + # `ABSENT_AS_UNKNOWN`. + if key not in ABSENT_AS_UNKNOWN: + continue + value = PAYLOAD_NONE self.client.publish(f"{BASE}/{key}", str(value)) except OSError as err: # pragma: no cover _LOG.debug("MQTT publish failed: %s", err) diff --git a/goodwe_controller/app/pv.py b/goodwe_controller/app/pv.py new file mode 100644 index 0000000..e5b9288 --- /dev/null +++ b/goodwe_controller/app/pv.py @@ -0,0 +1,117 @@ +"""TEL-11 - the site's PV production, read from a Home Assistant entity. + +WHY THIS IS AN ENTITY AND NOT A REGISTER. This add-on's RS485 bus carries the +BATTERY inverter it drives. A household's PV inverter is, on nearly every site, +a different box from a different vendor that reports into Home Assistant +through its own integration - so the only place a product can read production +for an arbitrary install is an HA entity. `pv_entity` therefore takes whatever +`sensor.*` the site's own PV integration provides. + +WHY THE CHANNEL EXISTS AT ALL. OPT-10 measured that 20.5 kWh of surplus left the +site in 16 days while the pack had room, 12.6 kWh of it with the battery not +charging at all - and could not say WHY, because PV production is recorded +NOWHERE. Three causes need three different fixes (no irradiance: nothing to fix; +a charge-power limit; the optimiser declining to charge - the only software +defect), and guessing between them is how an optimiser gets tuned to solve a +cloud problem. This module is the missing measurement and NOTHING ELSE. + +⚠️ NOTHING HERE MAY EVER REACH A CONTROL DECISION, AND THAT IS THE CARD'S FIRST +CONSTRAINT RATHER THAN A STYLE NOTE. The add-on regulates a real battery; adding +production as a term of the control law would be an unrequested behaviour change +AND it would destroy the measurement, because the loop would then be reacting to +the quantity being measured. `tools/tel11_pv_trace.py` is what says the path is +untouched: it grades a control document that contains none of this material on +both trees and with `pv_entity` set and unset. + +⚠️ ABSENCE IS NEVER ZERO, AND THAT IS THE SECOND CONSTRAINT. Every refusal below +returns `None` with a REASON, never `0.0`. A PV channel that reported 0 W for a +dead sensor would manufacture exactly the "there was no surplus to store" +conclusion the card exists to TEST - and it would do it silently, in a week of +recorder history nobody would re-derive. `app/hass.py`'s own note on `state()` +is the same rule one layer down. + +THE SIGN CONVENTION: POSITIVE IS PRODUCTION. Derived from the two conventions +this add-on already ships rather than assumed: + + `meter_entity` positive = IMPORTING from the grid (DOCS.md, Sources) + `batt_entity` positive = DISCHARGING the pack (DOCS.md, Sources) + +Both are "positive = power arriving at the house's AC bus", and production +arrives there too. There is deliberately NO `pv_invert` to go with +`meter_invert` and `batt_invert`: those two quantities are BIPOLAR and are READ +BY THE CONTROL PATH - a reversed `batt_entity` feeds SAFETY-30's per-unit sign +check and SAFETY-31's circulating-power sensor with a plausible number the site +then acts on - whereas production is unipolar and is read by nothing. A reversed +PV sensor announces itself as -4 kW at noon in the very history this card +creates, and flipping it here would only hide it. See +`DECISIONS/TEL-11-pv-production.md` §2. +""" + +from .hass import BAD_STATES + +# Home Assistant's OWN power units, each as the number of watts one of it is. +# ⚠️ READ OUT OF HA, NOT LISTED FROM MEMORY: `UnitOfPower` in +# `homeassistant/const.py` (github.com/home-assistant/core, `dev`, retrieved +# 2026-09-24) declares mW, W, kW, MW, GW, TW and BTU/h. Every factor here is an +# exact power of ten, so none of them is a number anybody chose. +# +# ⚠️ BTU/h IS IN HA's ENUM AND DELIBERATELY NOT HERE. Its factor (0.293071…) +# is a measured conversion rather than a decimal shift, no PV integration +# publishes in it, and a wrong constant in the one channel that exists to +# attribute a 20.5 kWh shortfall is worse than a refusal an operator can read. +# It lands in the "not a power unit" refusal below, BY NAME. +WATTS_PER = { + "mW": 0.001, + "W": 1.0, + "kW": 1000.0, + "MW": 1000000.0, + "GW": 1000000000.0, + "TW": 1000000000000.0, +} + +# What `watts()` answers when there is nothing to report. ⚠️ THE REASON IS PART +# OF THE RETURN VALUE, not a log line: it is what the status page prints, and an +# absence whose cause is invisible sends an operator to look at the wrong box. +UNIT_KEY = "unit_of_measurement" + + +def watts(state: dict | None) -> tuple: + """`(watts, why)` from a Home Assistant state object. + + `watts` is `None` whenever there is no reading - and `why` then says which + of the five refusals it was, in words an operator can act on. On a good read + `why` is the unit the value was read in, because "1234 W" and "1234, unit + unknown" are different claims and the page has to be able to make the first. + + ⚠️ THE UNIT IS READ OFF THE ENTITY AND NOT ASSUMED, AND THERE IS NO + MAGNITUDE HEURISTIC. `read_inputs()` guesses kW-or-W for + `peak_forecast_entity` with `abs(raw) < 50`, which is safe there because + nobody's quarter-hour demand forecast is 50 W. It is NOT safe here: 30 W of + real production at dusk is an ordinary reading, and that heuristic would + publish it as 30 kW - a fabricated number, in the channel whose entire + purpose is to be believed a week later. A sensor whose unit this cannot read + is REFUSED rather than guessed at. + """ + if state is None: + # `hass.state()` answers None for a non-existent entity, a 404 and a + # transport failure alike. All three are "no reading", and HA does not + # consider a non-existent entity an error - see `hass.state()`. + return None, "no state (entity missing, or the read failed)" + raw = str(state.get("state", "")).strip() + if raw.lower() in BAD_STATES: + # ⚠️ THE SAME LIST `hass.number()` REFUSES ON, IMPORTED RATHER THAN + # RESPELLED. A second copy of it is how one path starts treating + # `unavailable` as a number the other one rejects. + return None, f"entity is {raw or 'empty'}" + try: + value = float(raw) + except ValueError: + return None, f"state {raw!r} is not numeric" + unit = str((state.get("attributes") or {}).get(UNIT_KEY) or "").strip() + if not unit: + return None, ("no unit_of_measurement on the entity - it cannot be " + "read in watts") + if unit not in WATTS_PER: + return None, (f"unit {unit!r} is not one of " + f"{', '.join(WATTS_PER)} - it cannot be read in watts") + return value * WATTS_PER[unit], unit diff --git a/goodwe_controller/config.yaml b/goodwe_controller/config.yaml index f2aa493..da627ef 100644 --- a/goodwe_controller/config.yaml +++ b/goodwe_controller/config.yaml @@ -147,6 +147,33 @@ options: inverters: [] estop_fitted: false + # TEL-11 - THE SITE'S PV PRODUCTION, IN WATTS, AND IT IS READ BY NOTHING THAT + # DECIDES ANYTHING. Optional, empty on every install that does not set it, and + # when it is empty the add-on reads no entity, announces no entity, publishes + # no topic and shows no status row. + # + # ⚠️ AN HA ENTITY AND NOT A REGISTER, BECAUSE MOST SOLAR INVERTERS ARE NOT ON + # THIS BUS. The RS485 bus this add-on speaks carries the BATTERY inverter it + # drives; a household's PV inverter is normally a different box from a + # different vendor, reporting into Home Assistant through its own integration. + # So this takes whatever `sensor.*` the site's PV integration provides, any + # brand Home Assistant can read. + # + # ⚠️ POSITIVE IS PRODUCTION, and there is deliberately no `pv_invert` beside + # `meter_invert` and `batt_invert`. Those two quantities are bipolar AND are + # read by the control path, so a reversed sensor there is a plausible number + # the site acts on; production is unipolar and is read by nothing, so a + # reversed sensor announces itself as -4 kW at noon in the history this key + # exists to create. See DECISIONS/TEL-11-pv-production.md. + # + # ⚠️ THE ENTITY'S OWN `unit_of_measurement` IS WHAT IS BELIEVED - W, kW, MW + # and the rest of Home Assistant's power units. A sensor with no unit, or with + # a unit that is not power, is REFUSED and the status page says which: there + # is no magnitude heuristic that can tell 30 W of dusk from 30 kW of noon, and + # a factor of 1000 in the one channel that exists to be believed a week later + # is worse than a refusal an operator can read. + pv_entity: "" + # ========================================================================== # METER - where the grid reading comes from # ========================================================================== @@ -872,6 +899,12 @@ schema: capacity_kwh: float(0.1,200)? estop_fitted: bool + # TEL-11. Optional, and OPTIONAL WITHOUT A COMPANION RULE - unlike + # `price_now_entity`/`price_avg_entity`, which are legal only while + # `price_provider` is `off` and make the add-on REFUSE TO START otherwise. + # Nothing in validate.py mentions this key: it cannot conflict with anything, + # because nothing else reads it. + pv_entity: str? # ========================================================================== # METER - where the grid reading comes from diff --git a/goodwe_controller/test_mqtt.py b/goodwe_controller/test_mqtt.py index 5d44b92..7ea360a 100644 --- a/goodwe_controller/test_mqtt.py +++ b/goodwe_controller/test_mqtt.py @@ -203,15 +203,21 @@ class FakeClient: self.msgs.append((topic, payload, retain)) -def announce(units=(), omit=(), module=mqtt): +def announce(units=(), omit=(), module=mqtt, include=()): """The messages `_announce()` publishes, through the SHIPPED method. ⚠️ Not `discovery_payloads()` directly. Part 2 proves the two agree; every other part goes through this one, so what is diffed is what paho is told. + + `include` is TEL-11's optional-row set and DEFAULTS TO EMPTY here for the + same reason it does in the shipped constructor: every check in this file + written before that key existed must go on grading the announcement an + install that never set `pv_entity` receives, which is all of them. """ pub = module.MqttPublisher.__new__(module.MqttPublisher) pub.omit = set(omit) pub.units = [str(u) for u in units] + pub.include = [str(k) for k in include] pub.enabled = True pub.client = FakeClient() pub._announce() @@ -316,6 +322,52 @@ check("...and the `omit` path is unchanged too: with P1 ingestion off the " "watchdog input is still not announced, and nothing else moved", (m, c, a) == ([], [], [])) +# --- TEL-11 ------------------------------------------------------------------ +# ⚠️ THE OPTIONAL ROW'S DEFAULT IS THE CLAIM, AND IT IS THE OPPOSITE DEFAULT +# FROM `omit`'s. `p1_age` is announced unless withdrawn; `pv` does not exist +# unless asked for. Every check above this line passes `include` nothing, so +# they are all grading the announcement an install that never set `pv_entity` +# receives - which is MOST installs, and is the acceptance criterion "unset +# behaves exactly as today". +check("⭐ TEL-11 ADDS NOTHING AT THE SHIPPED DEFAULT: a publisher that is told " + "nothing about `pv_entity` announces the same list, in the same order, " + "with the same bytes - so an install that never sets the option sees no " + "new entity, no new topic and no re-announcement of an old one", + announce(units=roster_names({})) == SITE_NOW) +_PV_TOPIC = "homeassistant/sensor/goodwe_ctl_pv/config" +_PV_PAYLOAD = ('{"name": "PV power", "object_id": "goodwe_pv_power", ' + '"unique_id": "goodwe_ctl_pv", "state_topic": "goodwe_ctl/pv", ' + '"availability_topic": "goodwe_ctl/availability", "device": ' + + _DEV + ', "unit_of_measurement": "W", "device_class": ' + '"power", "state_class": "measurement"}') +_pv_one = announce(units=roster_names({}), include=("pv",)) +_m2, _c2, _a2 = diff(_pv_one, SITE_NOW) +check("⭐ AND WITH `pv_entity` SET IT IS EXACTLY ONE NEW TOPIC AND NOTHING " + "ELSE MOVES: same eleven configs first, same bytes, same order - the " + "additions this card makes are additions, not a re-composition", + (_m2, _c2) == ([], []) and _a2 == [_PV_TOPIC] + and _pv_one[:len(SITE_NOW) - 1] == SITE_NOW[:-1]) +check("...and the payload is the site device, in watts, with " + "`state_class: measurement` - which is what puts it in the recorder's " + "statistics, and a week of statistics is the entire point of the card", + (_PV_TOPIC, _PV_PAYLOAD, True) in _pv_one) +# ⚠️ AND IT GOES LAST, WHICH IS WHY A ROSTER OF TWO IS ASKED THE SAME QUESTION. +# Placed among the site rows it would shift every per-unit entity's position in +# the retained announcement - a zero-byte edit that re-announces inverter B. +_pv_two = announce(units=NAMES2, include=("pv",)) +check("...and at a roster of two it is STILL the last config, after every " + "per-unit row, so no existing entity's position in the retained " + "announcement moves when the option is turned on", + [t for t, _p, _r in _pv_two if t.startswith("homeassistant/")][-1] + == _PV_TOPIC + and _pv_two == announce(units=NAMES2)[:-1] + [ + (_PV_TOPIC, _PV_PAYLOAD, True)] + announce(units=NAMES2)[-1:]) +check("...and `omit` still wins over `include`, so the two arguments cannot " + "disagree about one key", + [t for t, _p, _r in announce(units=roster_names({}), include=("pv",), + omit=("pv",)) + if t == _PV_TOPIC] == []) + # --------------------------------------------------------------------------- print("\n2. what is diffed is what goes on the wire") # --------------------------------------------------------------------------- diff --git a/goodwe_controller/test_pv.py b/goodwe_controller/test_pv.py new file mode 100644 index 0000000..6939fc7 --- /dev/null +++ b/goodwe_controller/test_pv.py @@ -0,0 +1,643 @@ +"""Runnable check for TEL-11 - the PV production channel. `python3 test_pv.py` + +OPT-10 measured that this site's battery reaches 90 % SoC on 2 days of 16 and +that 20.5 kWh of surplus left it while the pack had room - 12.6 kWh of that with +the battery not charging at all - and could not say WHY, because PV production +is recorded NOWHERE. Three causes need three different fixes: no irradiance +(nothing to fix), a charge-power limit, or the optimiser declining to charge +(the only software defect). This card adds the missing channel. It does not do +the attribution; that is OPT-10's, with a week of history behind it. + +⚠️ WHAT THIS FILE REFUSES TO ASSERT, SAID FIRST. It does not grade that a row +EXISTS. `publish()` handing a dict with `pv` in it is satisfied by `pv: 0.0` +forever, and a PV channel that reports 0 W for a dead sensor MANUFACTURES the +conclusion the card exists to test - "there was no surplus to store" - in a week +of history nobody will re-derive. So every check below separates three states +that a weaker one would collapse into one: + + NO SOURCE `pv_entity` unset. No read, no entity, no topic, no row. + NO READING configured, and the sensor did not answer. The topic carries + HA's own `None`, the entity's state is `unknown`, the + recorder records a GAP. + ZERO WATTS configured, and the sensor said 0. The topic carries `0.0`. + +The last two are the pair the acceptance criterion is about and part 3 asserts +the PAYLOAD BYTES of both, because they differ by exactly one thing that no +check on the VALUE alone can see. + +⚠️ AND THE OTHER HALF OF THE CARD IS A NEGATIVE, GRADED AS ONE. This is +instrumentation: it must not become a control input, or the loop would react to +the quantity being measured and the measurement would be worthless. Part 6 is +the in-suite half of that claim (the shipped command is bit-identical across +absent / 0 W / 8 kW of PV, `inputs_ok` never moves, and an AST scan says which +modules can even see the material). `tools/tel11_pv_trace.py` is the other half: +it hashes a control document on THIS tree and on `release/1.0`, with the option +set and unset. + + Part 1 `pv.watts()`: the unit table, and every refusal, by name. + Part 2 the shipped read path, including what a FAILED read does to the + value a good one left behind. + Part 3 ⭐ publication: no source / no reading / zero watts, as payload bytes. + Part 4 announced and published are the same key, from the same predicate. + Part 5 the operator surfaces, and that an unset option adds none of them. + Part 6 ⭐ NOT A CONTROL INPUT. + Part 7 what this file cannot see, with sizes, and the mutation table. + +⚠️ MUTATING A MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run +`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it +stops bytecode being WRITTEN, not a stale one being READ. Read MUTATION.md. +""" + +import ast +import asyncio +import sys +from pathlib import Path + +from app import mqtt, pv +from app.main import Controller + +HERE = Path(__file__).resolve().parent +fails = [] + + +def check(name, cond): + print(f" {'ok ' if cond else 'FAIL'} {name}") + if not cond: + fails.append(name) + + +def state(value, unit="W", **attrs): + """A Home Assistant state object, in the shape `hass.state()` returns.""" + return {"state": value, "attributes": dict({"unit_of_measurement": unit}, + **attrs)} + + +# ============================================================================== +print("1. pv.watts(): the unit is read off the entity, and a refusal is named") +# ============================================================================== +# ⚠️ LITERALS ON THE RIGHT-HAND SIDE, NOT `raw * WATTS_PER[u]` (habit 2). An +# assertion written against the expression under test cannot tell a correct +# table from a table that is wrong in the same direction twice. +CASES = [ + (state("1234", "W"), 1234.0, "W"), + (state("5.2", "kW"), 5200.0, "kW"), + (state("0.0072", "MW"), 7200.0, "MW"), + (state("4500000", "mW"), 4500.0, "mW"), + (state("0.0000072", "GW"), 7200.0, "GW"), + (state("0.0000000072", "TW"), 7200.0, "TW"), + (state("-8", "W"), -8.0, "W"), +] +print(" state unit -> watts") +_bad = [] +for st, want, unit in CASES: + got, why = pv.watts(st) + print(f" {st['state']:>14} {unit:<4} -> {got} ({why})") + if got is None or abs(got - want) > 1e-6 or why != unit: + _bad.append((st, got, want)) +check(f"every unit in the table converts to watts against a literal, and the " + f"reason string is the unit it was read in. Wrong: {_bad}", not _bad) +check("...and the table is HOME ASSISTANT's own power units, so a card that " + "adds one has to say so here: mW, W, kW, MW, GW, TW", + sorted(pv.WATTS_PER) == sorted(["mW", "W", "kW", "MW", "GW", "TW"])) +check("...and every factor is an exact power of ten - nothing in this table is " + "a number anybody chose", + all(abs(v * 1000 - round(v * 1000)) < 1e-9 and v > 0 + for v in pv.WATTS_PER.values())) + +# ⚠️ THE CASE THE WHOLE CARD TURNS ON, AND IT IS ASSERTED IN BOTH DIRECTIONS. +check("⭐ ZERO WATTS IS A READING: a sensor that says 0 gives 0.0, not None - " + "the pack can be full and the sun can be down and the honest report of " + "the second one is a number", + pv.watts(state("0", "W")) == (0.0, "W") + and pv.watts(state("0.0", "kW")) == (0.0, "kW")) + +REFUSALS = [ + ("no state object at all (entity missing, or the read failed)", None, + "no state"), + ("the entity is `unavailable`", state("unavailable"), "unavailable"), + ("the entity is `unknown`", state("unknown"), "unknown"), + ("an empty state", state(""), "empty"), + ("a state that is not a number", state("sunny"), "not numeric"), + ("no unit_of_measurement at all", state("1234", ""), "no unit"), + ("a unit that is not power", state("1234", "%"), "'%'"), + # ⚠️ IN HA's `UnitOfPower` ENUM AND DELIBERATELY NOT IN THE TABLE. Its + # factor is a measured conversion rather than a decimal shift, and a wrong + # constant in the channel that exists to attribute a 20.5 kWh shortfall is + # worse than a refusal somebody can read. + ("BTU/h, which IS one of HA's power units", state("1234", "BTU/h"), + "'BTU/h'"), +] +_r = [] +print(" refusal -> (watts, why)") +for label, st, needle in REFUSALS: + got, why = pv.watts(st) + print(f" {label:<42} -> ({got}, {why!r})") + if got is not None or needle not in why: + _r.append(label) +check(f"⭐ EVERY REFUSAL ANSWERS None AND SAYS WHICH REFUSAL IT WAS - never " + f"0.0, and never a bare 'no'. A reason an operator cannot act on sends " + f"them to look at the wrong box. Wrong: {_r}", not _r) +check("...and the refused states are the SAME list `hass.number()` refuses, " + "imported rather than respelled - one path cannot start accepting what " + "the other rejects", + pv.BAD_STATES is __import__("app.hass", fromlist=["x"]).BAD_STATES) +# habit 1: the precondition, as its own check. Without it the row above could +# be true of a table that had quietly become empty. +check("PRECONDITION: the refusal list and the unit table are both non-empty, " + "so the two checks above quantify over something", + len(REFUSALS) >= 8 and len(pv.WATTS_PER) >= 6) + + +# ============================================================================== +print() +print("2. the shipped read path") +# ============================================================================== +LIVE = {"max_w": 5000.0, "gain": 0.6, "slew_w": 400.0, "target_grid_w": -10.0, + "deadband_w": 15, "step_w": 10} +BASE_OPTS = {"setpoint_entity": "number.sp", "meter_entity": "sensor.grid", + "soc_entity": "sensor.soc", "batt_entity": "sensor.batt", + "meter_source": "off", "heartbeat_s": 10, "stale_input_s": 15} + + +class _Store: + def __init__(self): + self.data = {} + + def set(self, *a): + pass + + def get_time(self, *a): + return None + + +class _Pub: + """Captures what `Controller.publish()` hands the publisher.""" + enabled = False + + def __init__(self): + self.frames = [] + + def publish(self, values): + self.frames.append(dict(values)) + + def close(self): + pass + + +class _Hass: + """A scripted meter, an inverter that follows, and a scripted PV sensor. + + ⚠️ `states` IS A SCRIPT, ONE ENTRY PER CYCLE, so a check can drive a GOOD + read and then a FAILED one and ask what the failure did to the value the + good one left behind. A fixture that answers the same thing forever cannot + reach that failure - MUTATION.md habit 9's shape, one surface over. + """ + + def __init__(self, grid=0.0, soc=50.0, states=(), follow=False): + self.grid = grid + # ⚠️ `follow` MAKES THE METER RESPOND TO THE BATTERY, and part 6 needs + # it. With a FIXED meter the error never shrinks, the integrator winds + # to its bound and the command rides `slew_w` to the rail on every + # input - so every stimulus produces the same ramp and an identity over + # them is free. `grid = base - batt` is the site: a battery discharging + # 610 W into a 600 W import leaves the meter at the -10 W this add-on + # rests at. + self.follow = follow + self.soc = soc + self.batt = 0.0 + self.n = 0 + self.writes = [] + self.state_calls = [] + self.states = list(states) + + async def limits(self, entity): + return (-LIVE["max_w"], LIVE["max_w"], 1.0) + + async def number(self, entity, invert=False): + grid = self.grid - self.batt if self.follow else self.grid + return {"sensor.grid": grid, "sensor.soc": self.soc, + "sensor.batt": self.batt}.get(entity) + + async def state(self, entity): + self.state_calls.append((self.n, entity)) + if not self.states: + return None + return self.states[min(self.n, len(self.states) - 1)] + + async def set_number(self, entity, value): + self.writes.append(float(value)) + self.batt = float(value) + return True + + async def call(self, domain, service, data): + return True + + +def run(cycles=3, grid=0.0, states=(), auto=True, follow=False, **over): + """`cycles` real read/cycle/publish turns. Returns (controller, hass, pub).""" + async def go(): + hass = _Hass(grid=grid, states=states, follow=follow) + pub = _Pub() + c = Controller(dict(BASE_OPTS, **LIVE, **over), hass, _Store(), pub, + stop=asyncio.Event()) + c.auto = auto + c.dev_min, c.dev_max = -LIVE["max_w"], LIVE["max_w"] + for _k in range(cycles): + await c.read_inputs() + await c.cycle() + c.publish() + hass.n += 1 + return c, hass, pub + return asyncio.run(go()) + + +PV_ON = {"pv_entity": "sensor.pv_power"} + +c_off, h_off, p_off = run(cycles=3) +check("⚠️ UNSET IS NOT READ AT ALL: with no `pv_entity` the add-on makes NO " + "state call, for any entity, on any cycle - not a read that fails " + "quietly, no read. This is what 'no warning spam' is made of", + h_off.state_calls == [] and c_off.pv_w is None + and c_off.pv_why == "not configured") + +c_on, h_on, _ = run(cycles=3, states=[state("3210", "W")], **PV_ON) +print(f" state calls: {h_on.state_calls}") +check("configured: the entity is read once per cycle, by name, and the value " + "reaches `self.pv_w` in watts", + h_on.state_calls == [(0, "sensor.pv_power"), (1, "sensor.pv_power"), + (2, "sensor.pv_power")] + and c_on.pv_w == 3210.0 and c_on.pv_why == "W") + +c_kw, _, _ = run(cycles=1, states=[state("3.21", "kW")], **PV_ON) +check("...and a kW sensor is read in kW: 3.21 kW is 3210 W, not 3.21 W. There " + "is no magnitude heuristic anywhere on this path - 30 W of dusk and " + "30 kW of noon are both ordinary readings", + c_kw.pv_w == 3210.0 and c_kw.pv_why == "kW") + +# ⚠️ THE ONE THAT SEPARATES "ABSENT" FROM "STALE", AND IT NEEDS TWO CYCLES. +c_drop, _, p_drop = run(cycles=2, states=[state("4100", "W"), None], **PV_ON) +print(f" published pv by cycle: {[f['pv'] for f in p_drop.frames]}") +check("⭐ A FAILED READ CLEARS THE PREVIOUS GOOD ONE. The cycle after a 4100 W " + "reading disappears publishes an ABSENCE, not 4100 W again - a value " + "that survives its own sensor is indistinguishable from a real one at " + "the moment somebody is working out whether the sun was shining", + [f["pv"] for f in p_drop.frames] == [4100.0, None] + and c_drop.pv_w is None and "no state" in c_drop.pv_why) + + +# ============================================================================== +print() +print("3. ⭐ publication: no source, no reading, zero watts - as payload bytes") +# ============================================================================== +class FakeClient: + def __init__(self): + self.msgs = [] + + def publish(self, topic, payload=None, retain=False): + self.msgs.append((topic, payload, retain)) + + +def wire(values, include=("pv",)): + """What the SHIPPED `MqttPublisher.publish()` hands paho for `values`.""" + pub = mqtt.MqttPublisher.__new__(mqtt.MqttPublisher) + pub.omit, pub.units, pub.include, pub.enabled = set(), [], list(include), True + pub.client = FakeClient() + pub.publish(values) + return pub.client.msgs + + +_zero = run(cycles=1, states=[state("0", "W")], **PV_ON)[2].frames[-1] +_absent = run(cycles=1, states=[None], **PV_ON)[2].frames[-1] +_nosrc = p_off.frames[-1] +print(f" no source : 'pv' in frame = {'pv' in _nosrc}") +def pv_msg(values): + """The `goodwe_ctl/pv` message the shipped publisher emits, or None.""" + got = [m for m in wire(values) if m[0] == "goodwe_ctl/pv"] + return got[0] if got else None + + +print(f" no reading: {_absent['pv']!r} -> wire {pv_msg(_absent)}") +print(f" zero watts: {_zero['pv']!r} -> wire {pv_msg(_zero)}") +check("NO SOURCE: an install that never set `pv_entity` publishes no `pv` key " + "at all - there is no entity to feed, so feeding one would be a topic " + "nothing announced", + "pv" not in _nosrc) +check("NO READING: the key is PRESENT and its value is None, which is a " + "different frame from the one above. 'the option is off' and 'the sensor " + "did not answer' are two facts and one field cannot carry both", + "pv" in _absent and _absent["pv"] is None) +check("ZERO WATTS: 0.0, published as a number", _zero["pv"] == 0.0) +check("⭐ AND ON THE WIRE THE LAST TWO ARE DIFFERENT BYTES ON THE SAME TOPIC: " + "`None` - which is Home Assistant's own payload for 'this sensor has no " + "value', so the entity goes to `unknown` and the recorder records a GAP " + "- against `0.0`. THIS IS THE ACCEPTANCE CRITERION: a week later, no PV " + "reported is not PV reported 0 W", + pv_msg(_absent) == ("goodwe_ctl/pv", "None", False) + and pv_msg(_zero) == ("goodwe_ctl/pv", "0.0", False) + and pv_msg(_absent) != pv_msg(_zero) + # ...and the pv row is the ONLY thing that differs between the two + # frames on the wire, so the claim above is about that row and not + # about some other number the two fixtures happened to move. + and [m for m in wire(_absent) if m[0] != "goodwe_ctl/pv"] + == [m for m in wire(_zero) if m[0] != "goodwe_ctl/pv"]) +check("...and the payload is HA's literal, from the module's own constant, not " + "a string spelled here", mqtt.PAYLOAD_NONE == "None") +# ⚠️ AND THE BLAST RADIUS IS ONE KEY. The None-dropping in `publish()` is what +# `circulation` depends on - its note in main.py says a missing per-unit reading +# must leave the last figure standing rather than publish a zero that reads as +# health - so widening it would be a behaviour change to another card's row. +check("⭐ AND NO OTHER ROW CHANGED BEHAVIOUR: `circulation: None` is still " + "DROPPED, not published as `None` - TEL-11 widened the rule for exactly " + "one key and `ABSENT_AS_UNKNOWN` is the list", + wire({"circulation": None, "grid": None, "loop_reason": None}) == [] + and mqtt.ABSENT_AS_UNKNOWN == frozenset({"pv"})) + + +# ============================================================================== +print() +print("4. announced and published are the same key, from the same predicate") +# ============================================================================== +# mqtt.py's `omit` note: an entity that exists and is never fed is a worse +# signal than one that does not exist. This is that claim for TEL-11's row, and +# it is taken off the SHIPPED announcement and the SHIPPED `publish()`. +def announced(opts): + return {t.rsplit("/", 2)[-2].replace(f"{mqtt.BASE}_", "") + for t, _c in mqtt.discovery_payloads( + include=mqtt.optional_keys(opts))} + + +check("with a source: `pv` is announced AND published", + "pv" in announced(PV_ON) and "pv" in _absent) +check("without one: announced by nothing and published by nothing", + "pv" not in announced({}) and "pv" not in _nosrc) +check("...and the predicate is ONE expression, `mqtt.optional_keys()`, read by " + "the announcement and by `Controller.publish()` alike - two spellings is " + "how an entity gets announced that nothing feeds", + mqtt.optional_keys({}) == () + and mqtt.optional_keys({"pv_entity": ""}) == () + and mqtt.optional_keys({"pv_entity": " "}) == () + and mqtt.optional_keys({"pv_entity": "sensor.x"}) == ("pv",)) +check("...and a whitespace-only option is the SAME as an empty one everywhere " + "- the controller does not read it either", + run(cycles=1, pv_entity=" ")[1].state_calls == []) + +# ⚠️ AND THE RESTART, WHICH IS THE OTHER HALF OF "RECORDABLE". Home Assistant +# keeps the entity_id it composed at FIRST discovery and keys its registry on +# `unique_id`, so a row whose id moved between two starts is a NEW entity beside +# a dead one and the history stays behind on the old id. Two independent +# announcements are asserted byte-identical, and the composed id is modelled +# INDEPENDENTLY of app/mqtt.py (the model is `tools/rehearse_roster.py`'s, and +# it is anchored there on the one composed id that was verified against the live +# system). +import re as _re # noqa: E402 + + +def compose(device_name, entity_name): + s_ = _re.sub(r"[^a-z0-9]+", "_", f"{device_name} {entity_name}".lower()) + return "sensor." + s_.strip("_") + + +_a1 = mqtt.discovery_payloads(include=mqtt.optional_keys(PV_ON)) +_a2 = mqtt.discovery_payloads(include=mqtt.optional_keys(PV_ON)) +_pv_cfg = [c for _t, c in _a1 if c["unique_id"] == "goodwe_ctl_pv"] +print(f" composed entity_id: " + f"{compose(mqtt.DEVICE['name'], _pv_cfg[0]['name'])}") +check("a restart re-announces the SAME bytes, so Home Assistant sees the entity " + "it already has rather than a new one beside a dead one - the history is " + "continuous across a restart of the add-on", + [(t, __import__("json").dumps(c, sort_keys=True)) for t, c in _a1] + == [(t, __import__("json").dumps(c, sort_keys=True)) for t, c in _a2] + and len(_pv_cfg) == 1) +check("...and the id Home Assistant composes for it is " + "`sensor.goodwe_pv_power`, which is what DOCS.md tells an operator to put " + "on a dashboard - device name plus entity name, NOT the object_id", + compose(mqtt.DEVICE["name"], _pv_cfg[0]["name"]) + == "sensor.goodwe_pv_power" + and _pv_cfg[0]["state_topic"] == "goodwe_ctl/pv") + + +# ============================================================================== +print() +print("5. the operator surfaces") +# ============================================================================== +def rows_of(c): + return [r["text"] for r in c.checks()] + + +def page_rows(c): + return c.status_payload()["rows"] + + +_off_rows, _on_rows = rows_of(c_off), rows_of(c_on) +print(f" unset: {len(_off_rows)} status rows; set: {len(_on_rows)}") +check("⚠️ AN UNSET OPTION ADDS NO STATUS ROW AND NO PAGE ROW - not a caution, " + "not a 'no PV configured' nag. `peak_forecast_entity` DOES nag when it " + "is unset, because an unset forecast leaves a money guard without an " + "input; this is instrumentation and a permanent caution on every install " + "that never asked for it is the warning spam the card forbids", + not [t for t in _off_rows if "PV" in t] + and not [r for r in page_rows(c_off) if r[0] == "PV"]) +check("...and the whole row list is otherwise the same length, so nothing else " + "moved either", len(_on_rows) == len(_off_rows) + 1) +_pv_row = [t for t in _on_rows if t.startswith("PV production")] +print(f" {_pv_row}") +# habit 14: the ordered pair - value, entity and unit each in its OWN slot in +# the literal sentence, so a transposition is visible rather than merely +# present. +check("a good read says the watts, the entity it came from and the unit it was " + "reported in, each in its own slot", + _pv_row == ["PV production: 3210 W from sensor.pv_power " + "(reported in W)"] + and ("PV", "3 210 W") in page_rows(c_on)) + +c_bad, _, _ = run(cycles=1, states=[state("1234", "%")], **PV_ON) +_bad_row = [r for r in c_bad.checks() if r["text"].startswith("PV production")] +print(f" {[r['text'] for r in _bad_row]}") +check("a refused read says WHICH refusal, and names the entity - 'no reading' " + "on its own sends an operator to look at a working sensor", + len(_bad_row) == 1 + and _bad_row[0]["text"] == ("PV production: no reading from " + "sensor.pv_power - unit '%' is not one of " + "mW, W, kW, MW, GW, TW - it cannot be read " + "in watts")) +check("⭐ ...and it is a CAUTION, never a `bad` row: `status_payload()` builds " + "NOT READY out of rows that are neither ok nor warn, and a telemetry " + "channel that could put a battery site into NOT READY would be " + "observability with a foot in the control path", + _bad_row[0] == {"ok": False, "warn": True, "text": _bad_row[0]["text"]} + and not c_bad.status_payload()["banner"].startswith("NOT READY") + and c_bad.status_payload()["level"] != "bad" + and c_bad.status_payload()["banner"] + == c_off.status_payload()["banner"]) +check("...and the page row renders an absence as an em dash, not as 0 W", + ("PV", "—") in page_rows(c_bad)) + + +# ============================================================================== +print() +print("6. ⭐ NOT A CONTROL INPUT") +# ============================================================================== +# ⚠️ THE CLAIM IS NOT "pv is not in control.py". It is that the COMMANDS do not +# move, and that is graded by driving the same stimulus with the PV channel +# absent, at zero, and at 8 kW - a spread that would be impossible to miss if +# any term of the control law had learned to read it. +# +# ⚠️ AND THE FIRST VERSION OF THIS FIXTURE COULD NOT HAVE SEEN IT, WHICH IS +# MUTATION.md HABIT 23 AND THE CARD'S OWN BATTERY FOUND IT. The stimulus was a +# 6 kW export over 12 cycles; the card's central mutation - `self.grid -= +# self.pv_w` in `read_inputs()`, +106 B, on disk, purged, genuinely running - +# came back a SURVIVOR. Not because the property held: because at 6 kW of +# export the loop is against its rail and the command is moving at `slew_w` +# every cycle, so -6000 W and -14000 W of apparent error produce the SAME +# ramp, tick for tick. The mutant ran and could not change the answer on the +# data it was given. +# +# The stimulus is now a SMALL import the loop can actually settle on, far +# inside the rail, and the separation is asserted as its own precondition +# below: a check that cannot go red on the one mutation the card is about is +# the card's own habit-4. +_stim = dict(cycles=40, grid=600.0, follow=True) +_c_none, _h_none, _ = run(**_stim) +_c_absent, _h_absent, _ = run(**_stim, states=[None], **PV_ON) +_c_zero, _h_zero, _ = run(**_stim, states=[state("0", "W")], **PV_ON) +_c_big, _h_big, _ = run(**_stim, states=[state("8000", "W")], **PV_ON) +_w_none, _w_big = _h_none.writes, _h_big.writes +print(f" unset : {_w_none} -> settled at {_c_none.target:g} W") +print(f" 8000 W : {_w_big} -> settled at {_c_big.target:g} W") +check("⭐ THE COMMANDS ARE BIT-IDENTICAL ACROSS unset / no reading / 0 W / " + "8 kW of production, over 40 cycles of a real `cycle()` against a 600 W " + "import the loop SETTLES on - every write, in order, and the same " + "resting target. The PV channel changes no number that reaches an " + "inverter", + _w_none == _h_absent.writes == _h_zero.writes == _w_big + and _c_none.target == _c_absent.target == _c_zero.target + == _c_big.target) +check("PRECONDITION: that stimulus really does command something, and RESTS " + "well inside the rail rather than riding the slew limit to it - at the " + "rail every input produces the same ramp and the identity above would " + "be free (habit 1)", + len(_w_none) >= 4 and abs(_c_none.target) > 100 + and abs(_c_none.target) < 0.5 * LIVE["max_w"]) +# ⚠️ THE HABIT-23 PRECONDITION, AND IT IS THE ONE THAT MAKES THE IDENTITY MEAN +# SOMETHING. "What would the mutation have had to change, and does the fixture +# reach that value?" Here: if the 8 kW HAD been subtracted from the meter +# reading, would this stimulus produce a different command? It is one line to +# ask, and on the first version of this fixture the answer was NO - see the +# note above `_stim`. +_c_shift, _h_shift, _ = run(cycles=40, grid=600.0 - 8000.0, follow=True) +print(f" meter shifted by the same 8 kW -> settled at " + f"{_c_shift.target:g} W") +check("⭐ PRECONDITION (habit 23): the same stimulus with the METER shifted by " + "that 8 kW rests somewhere visibly different, so 'the commands did not " + "move' is a fact about the code and not about a rail that would have " + "flattened any input", + _h_shift.writes != _w_big + and abs(_c_shift.target - _c_big.target) > 4000) +check("...and `inputs_ok` does not move either: a PV sensor that never answers " + "is not a missing INPUT, because nothing computes with it. An install " + "whose solar integration breaks keeps regulating", + _c_none.inputs_ok is True and _c_absent.inputs_ok is True) + +# --- the structural half, and its scope is COMPUTED (habit 12) --------------- +APP = sorted((HERE / "app").glob("*.py")) +NEEDLES = ("pv_entity", "pv_w", "pv_why", "optional_keys", "ABSENT_AS_UNKNOWN") + + +def touches(src: str) -> bool: + """Does this module MENTION TEL-11's material, by any spelling? + + ⚠️ A TEXT SCAN ON PURPOSE, AND IT IS THE CONSERVATIVE DIRECTION HERE. An + AST scan would have to enumerate the forms a reference can take (attribute, + subscript, `getattr`, a string key) and habit 12 says a hand-written list of + forms is the same defect as a hand-written scope. Text over-reports - a + comment counts - and over-reporting is safe for a check whose claim is + "these modules do NOT mention it". + """ + return any(n in src for n in NEEDLES) or "from . import pv" in src \ + or "from .pv import" in src + + +_touch = sorted(f.name for f in APP if touches(f.read_text(encoding="utf-8"))) +print(f" {len(APP)} modules under app/; TEL-11 material is in: {_touch}") +check(f"PRECONDITION: the scan found the whole package, not a subset - " + f"{len(APP)} modules", len(APP) >= 25 and "control.py" in + [f.name for f in APP]) +check("⭐ EXACTLY THREE MODULES CAN SEE THE MATERIAL: `pv.py` (the reader), " + "`main.py` (the read and the two operator surfaces) and `mqtt.py` (the " + "publication). `control.py`, `arbiter.py`, `allocator.py`, `peakguard.py`, " + "`arbitrage.py`, `maintenance.py`, `clamp.py`, `freeze.py`, " + "`watchdog.py`, `delivery.py` and `circulation.py` do not mention it in " + "any spelling, comments included", + _touch == ["main.py", "mqtt.py", "pv.py"]) +# habit 4 in the instrument: a scanner that cannot go red is not a scanner. +check("...and the scan CAN go red - a module body with the material in it is " + "caught, so the green above is the tree speaking and not the scanner " + "being blind", + touches("x = opts['pv_entity']\n") and touches("from . import pv\n") + and not touches("# nothing to see, a pvc pipe\n")) + + +# ============================================================================== +print() +print("7. what this file CANNOT see, with sizes - and the mutation table") +# ============================================================================== +# habit 13. An enumerated list with an honest gap is worth more than a claim of +# completeness a reviewer falsifies in ten minutes. +for line in [ + "Home Assistant itself. That `None` on the state topic renders as state", + " `unknown`, and that the recorder keeps it, is READ OUT OF HA's source", + " (mqtt/const.py PAYLOAD_NONE, mqtt/sensor.py's `_message_received`,", + " retrieved 2026-09-24) and quoted in app/mqtt.py. No HA runs in this", + " suite, so what is graded here is the BYTES this add-on publishes.", + "Whether the operator's PV sensor is telling the truth. A sensor wired", + " backwards reads -4 kW at noon; this file asserts that the sign is", + " published unchanged, never that it is right. That is what the history", + " the card creates is FOR - see DECISIONS/TEL-11-pv-production.md §2.", + "The control-path claim OUTSIDE app/. `tools/`, the ESPHome firmware and", + " the ingress page are not scanned by part 6's census.", + "A roster of more than one. Every fixture here is the shipped roster of", + " one; the row is a SITE row and there is nothing per-unit to separate.", + "Wall-clock behaviour: no fixture here sleeps, and the read is driven by", + " the caller's loop rather than by the 1 s tick.", +]: + print(" " + line) + +print() +print(" mutation bytes reds") +print(" ------------------------------------------------ ----- --------") +for row in [ + ("M1 the pv row moved into SENSORS", "-79 B", "3 test_mqtt + 1 here"), + ("M2 publish() drops a None pv row again", "-98 B", "1 here"), + ("M3 ABSENT_AS_UNKNOWN widened past the one key", "+23 B", "1 here"), + ("M4 a dead sensor reported as 0 W", "-1 B", "4 here"), + ("M5 a failed read leaves the last value standing", "+46 B", "2 here"), + ("M6 the unit read and then ignored", "-18 B", "2 here"), + ("M7 `self.grid -= self.pv_w` in read_inputs()", "+106 B", + "2 here + the trace"), + ("M8 the row announced and fed unconditionally", "-52 B", "3 here"), + ("M9 the pv caution made a `bad` row (NOT READY)", "+1 B", "1 here"), + ("M10 the read gate removed - unset is read anyway", "-10 B", + "2 here + 2 crashes"), +]: + print(f" {row[0]:<48} {row[1]:>5} {row[2]}") +print(" ⚠️ M7 WAS A SURVIVOR ON THE FIRST BATTERY and part 6's note says why:") +print(" the fixture drove the loop to its RAIL, where every input produces") +print(" the same ramp. MUTATION.md habit 23, with a saturated actuator in") +print(" place of a coinciding value - and it is in that file now.") +print(" ⚠️ M10's two extra reds are CRASHES, not named checks: every") +print(" fixture in test_inverter.py and test_telemetry.py predates the PV") +print(" read and has no `state()` on its Hass double, so an unconditional") +print(" read is an AttributeError. A red, but an unnamed one - habit 20's") +print(" shape - and it is recorded rather than tidied away.") +print(" Every row above was RUN, in the CI image, on a purged tree, one at a") +print(" time - see DECISIONS/TEL-11-pv-production.md §5 for the transcript.") +print(" ⚠️ A RED COUNT IS A PROPERTY OF THE SUITE AS MUCH AS OF THE MUTATION:") +print(" these name THIS tree. Re-measure rather than inherit.") + +print() +if fails: + print(f"{len(fails)} FAILED:") + for f in fails: + print(f" - {f}") + sys.exit(1) +print("all checks passed") diff --git a/goodwe_controller/test_translations.py b/goodwe_controller/test_translations.py index 768c44c..822d21f 100644 --- a/goodwe_controller/test_translations.py +++ b/goodwe_controller/test_translations.py @@ -371,8 +371,14 @@ print("\n7. no behaviour and no value moved") # test_watchdog.py's six, test_price.py's seven, test_freeze.py's, and # test_peakguard.py's. This line is the cheap one that says the reorder did not # lose or duplicate a key on the way. -check("no option was lost, gained or duplicated by the reordering: the two " - "halves of config.yaml hold the same 72 keys, once each", +# ⚠️ THE COUNT IS READ OFF THE TREE, NOT TYPED. It was typed - "the same 72 +# keys" - and was 75 by the time TEL-11 read it, because three cards had added +# an option each and none of them was looking at this sentence. A number in a +# check's own message is a claim like any other (MUTATION.md habit 22) and this +# one had been false for three cards without a single run going red, because +# nothing asserts it. Derived, it cannot rot. +check(f"no option was lost, gained or duplicated by the reordering: the two " + f"halves of config.yaml hold the same {len(ORDER)} keys, once each", sorted(_opt_order) == sorted(ORDER) and len(set(ORDER)) == len(ORDER)) check("...and `inverters: []` is still shipped in the options half, which is " "what Supervisor 2026.08.0 refuses every pre-0.6.0 options blob without " diff --git a/goodwe_controller/tools/tel11_pv_trace.py b/goodwe_controller/tools/tel11_pv_trace.py new file mode 100644 index 0000000..4d77554 --- /dev/null +++ b/goodwe_controller/tools/tel11_pv_trace.py @@ -0,0 +1,390 @@ +"""TEL-11's behaviour-invariance instrument. `tools/tel11_pv_trace.py` + +TEL-11 adds a PV production channel and claims two things a reviewer cannot +check by reading a diff: + + 1. with `pv_entity` UNSET - which is every install today and most installs + ever - the add-on behaves exactly as `release/1.0` does. Not "the control + law is untouched": the whole observable surface, commands and telemetry + alike. + 2. with `pv_entity` SET, the CONTROL decisions are still exactly those. The + channel is instrumentation and must never become a control input, because + a loop that reacted to production would destroy the very measurement the + card exists to make. + +This is what says so. It drives a fixed stimulus set through the SHIPPED +`Controller` and prints two documents per setting - + + `control` every command written to an actuator, every `self.target`, every + `inputs_ok`, the loop's own three fields and the arbiter's + resolution, tick by tick. NO TEL-11 MATERIAL REACHES IT. + `telemetry` the published frames, the status rows and the announcement - + i.e. everything the card is ALLOWED to move. + +- and a `tree` fingerprint: sha256 and byte count of every `app/*.py` it +imported, so the comparison can say WHICH TREES it graded. + + python3 tools/tel11_pv_trace.py > /tmp/tel11-head.json + # …in a release/1.0 checkout, with this file copied in… + python3 tools/tel11_pv_trace.py > /tmp/tel11-base.json + python3 tools/tel11_pv_trace.py --compare /tmp/tel11-base.json \\ + /tmp/tel11-head.json + +⚠️ THE CONTROL IS PART OF THE TOOL, WHICH IS MUTATION.md HABIT 4 IN AN +INSTRUMENT - AND IT IS A DIFFERENT CONTROL FROM `safety56_behaviour_trace.py`'s, +ON PURPOSE. That card CHANGED sentences, so its `--compare` refuses to report +"behaviour unchanged" unless the sentences differ. THIS card's headline claim is +that with the option unset NOTHING differs at all, so "the documents are +identical" is the result rather than the warning sign - and the thing that has +to be proved instead is that the two runs came from DIFFERENT TREES and that the +instrument can see this card's material when it is switched on. So `--compare` +refuses unless: + + * the two `tree` fingerprints DIFFER (otherwise it is one tree twice, which + is the trap ARCH-02's rehearsal harness walked into and SAFETY-56 names); + * exactly ONE side's `set` document differs from its own `unset` document. + That side is the HEAD; the other is the baseline, which has no such option + and cannot react to it. Both, or neither, is ambiguous and is refused. + +⚠️ WHAT IT CANNOT SEE, with shapes (habit 13): + * a roster of more than one, and any per-unit surface; + * the real MQTT transport, Home Assistant, and the recorder. The publication + is graded as the dict `Controller.publish()` hands the publisher plus the + announcement `discovery_payloads()` returns - not as bytes on a broker; + * wall-clock behaviour and the 1 s tick: every clock here is an argument; + * the ingress HTML, the action log and the firmware; + * anything a `pv_entity` whose sensor is HEALTHY does not exercise - the + stimulus set drives a good read, a refused unit and a dead sensor, and + nothing else; + * ⚠️ and it is an IDENTITY over a fixed stimulus, so a divergence that needs + an input this set never produces is invisible to it by construction. The + stimulus is printed with the documents for exactly that reason. + +⚠️ PURGE `__pycache__` BEFORE USE on a tree you have just edited. +`tools/mutation_run.sh` does, and MUTATION.md says why `python3 -B` alone does +not. +""" +import asyncio +import hashlib +import json +import logging +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent.parent # the add-on directory +sys.path.insert(0, str(HERE)) + +Controller = mqtt = None + + +def _load(): + """The add-on's modules, imported LAZILY so `--compare` runs anywhere. + + ⚠️ NOT A STYLE CHOICE - `safety56_behaviour_trace.py`'s note is this one's + too. `--compare` is the half a reviewer runs against two JSON files, in a + directory with no `app/` in it, and a top-level import would make the + comparison impossible exactly where it is most useful. + """ + global Controller, mqtt + from app.main import Controller # noqa: E402,F401 + from app import mqtt # noqa: E402,F401 + + +A = 5000.0 +TUNING = {"max_w": A, "gain": 0.6, "slew_w": 400.0, "target_grid_w": -10.0, + "deadband_w": 15, "step_w": 10} +ONE = dict(TUNING, meter_source="off", meter_entity="sensor.grid", + setpoint_entity="number.sp_a", soc_entity="sensor.soc_a", + batt_entity="sensor.batt_a", heartbeat_s=10, stale_input_s=15, + inverters=[], bms_soc_floor=10) +PV_ENTITY = "sensor.pv_power" + + +class MemStore: + def __init__(self): + self.data = {} + + def set(self, *a): + pass + + def get_time(self, *a): + return None + + def set_time(self, *a): + pass + + +class Pub: + enabled = False + + def __init__(self): + self.frames = [] + + def publish(self, values): + self.frames.append(dict(values)) + + def close(self): + pass + + +class Hass: + """A scripted meter, a following inverter, and a scripted PV sensor.""" + + def __init__(self, grid_of, soc, pv_of): + self.grid_of, self.soc, self.pv_of = grid_of, soc, pv_of + self.batt = 0.0 + self.n = 0 + self.writes = [] + + async def limits(self, entity): + return (-A, A, 1.0) + + async def number(self, entity, invert=False): + v = {"sensor.grid": self.grid_of(self.n), "sensor.soc": self.soc, + "sensor.soc_a": self.soc, "sensor.batt_a": self.batt, + "sensor.batt": self.batt}.get(entity) + return None if v is None else (-v if invert else v) + + async def state(self, entity): + return self.pv_of(self.n) + + async def set_number(self, entity, value): + self.writes.append((entity, float(value))) + self.batt = float(value) + return True + + async def call(self, domain, service, data): + return True + + +def _pv_state(watts, unit="W"): + return {"state": str(watts), + "attributes": {"unit_of_measurement": unit}} + + +# ⚠️ THE STIMULUS SET, PRINTED WITH THE DOCUMENTS. An identity is only as wide +# as what was driven through it, so the worlds and the PV schedules are part of +# the output rather than a fact about the source. +# +# Each PV schedule is exercised against each grid world. A schedule that only +# ever answered healthily could not separate "the loop ignores production" from +# "the loop ignores a production channel that is always saying the same thing". +GRIDS = { + "EXPORT-6K": lambda t: -6000.0, + "IMPORT-3K": lambda t: 3000.0, + "REST": lambda t: -10.0, + "SWINGING": lambda t: (-4000.0 if (t // 5) % 2 else 4000.0), + "METER-GAPS": lambda t: (None if t % 7 == 3 else -1500.0), +} +PVS = { + "none": lambda t: None, + "zero": lambda t: _pv_state(0), + "noon": lambda t: _pv_state(8000), + "ramp": lambda t: _pv_state(t * 250), + "kilowatts": lambda t: _pv_state(4.25, "kW"), + "bad-unit": lambda t: _pv_state(1234, "%"), + "flapping": lambda t: (None if t % 3 else _pv_state(3300)), + "negative": lambda t: _pv_state(-4200), +} +CYCLES = 24 + + +async def _run(grid_of, pv_of, pv_set, soc=50.0, stop_at=None): + hass = Hass(grid_of, soc, pv_of) + pub = Pub() + opts = dict(ONE) + if pv_set: + opts["pv_entity"] = PV_ENTITY + c = Controller(opts, hass, MemStore(), pub, stop=asyncio.Event()) + c.auto = True + c.dev_min, c.dev_max = -A, A + c.inverters[0].dev_min, c.inverters[0].dev_max = -A, A + ctrl = [] + for t in range(CYCLES): + if stop_at is not None and t == stop_at: + c.auto = False + await c.read_inputs() + await c.cycle() + c.publish() + # ⚠️ THE CONTROL DOCUMENT, AND NOTHING FROM THIS CARD IS IN IT. Every + # field here existed on release/1.0 and is a DECISION: what was + # commanded, what the site is aiming at, whether the inputs were good + # enough to act on, what the loop did and which claim won. + ctrl.append([hass.writes[-1] if hass.writes else None, + c.target, c.inputs_ok, round(c.i_w, 6), + round(c.loop_i_w, 6), round(c.loop_i_lead, 6), + c.loop_reason, c.reason, c.following, + c.maint.phase, len(c.events)]) + hass.n += 1 + return c, hass, pub, ctrl + + +def documents(pv_set: bool) -> dict: + """`{"control": …, "telemetry": …}` for one setting of `pv_entity`.""" + _load() + logging.disable(logging.CRITICAL) + loop = asyncio.new_event_loop() + control, tele = {}, {} + for gname, gof in GRIDS.items(): + for pname, pof in PVS.items(): + key = f"{gname}/{pname}" + c, hass, pub, ctrl = loop.run_until_complete( + _run(gof, pof, pv_set)) + control[key] = { + "ticks": ctrl, + "writes": hass.writes, + "sha": hashlib.sha256( + json.dumps(ctrl, sort_keys=True).encode()).hexdigest(), + } + rows = c.checks() + tele[key] = { + "frames": pub.frames, + "rows": [[r["ok"], r["warn"], r["text"]] for r in rows], + "n_rows": len(rows), + "page": c.status_payload()["rows"], + "level": c.status_payload()["level"], + "banner": c.status_payload()["banner"], + } + # A stopped run, because the "stopped or blind" branch writes its own rows. + c, hass, pub, ctrl = loop.run_until_complete( + _run(GRIDS["EXPORT-6K"], PVS["noon"], pv_set, stop_at=8)) + control["STOPPED"] = {"ticks": ctrl, "writes": hass.writes, + "sha": hashlib.sha256( + json.dumps(ctrl, sort_keys=True).encode() + ).hexdigest()} + tele["STOPPED"] = {"frames": pub.frames, + "rows": [[r["ok"], r["warn"], r["text"]] + for r in c.checks()]} + loop.close() + # The announcement is telemetry too, and it is the surface an install's + # entity registry is keyed on. + tele["ANNOUNCEMENT"] = _announcement(pv_set) + return {"control": control, "telemetry": tele} + + +def _supports_include() -> bool: + """Does THIS tree's `discovery_payloads()` take TEL-11's `include`? + + ⚠️ A PROPERTY OF THE TREE, AND IT IS RECORDED IN `tree` RATHER THAN IN A + DOCUMENT. `release/1.0` has neither the argument nor `optional_keys()`, so + the tool has to run on a tree that predates both or it cannot produce a + baseline at all. Putting the answer in the documents instead would make + `unset` differ between the two trees for a reason that is not behaviour - + which is the "check that goes red for the wrong reason" MUTATION.md keeps + finding - so it goes beside the file hashes, where a reader comparing two + trees is already looking. + """ + import inspect + return ("include" + in inspect.signature(mqtt.discovery_payloads).parameters + and hasattr(mqtt, "optional_keys")) + + +def _announcement(pv_set: bool) -> list: + """The retained discovery set, in announcement order. + + ⚠️ AT `pv_set` FALSE THIS IS THE BARE CALL ON BOTH TREES, which is the + point: an install that never set the option must receive the SAME + announcement from both, and that identity is graded rather than argued. + """ + if pv_set and _supports_include(): + payloads = mqtt.discovery_payloads( + include=mqtt.optional_keys({"pv_entity": PV_ENTITY})) + else: + payloads = mqtt.discovery_payloads() + return [[t, json.dumps(cfg, sort_keys=True)] for t, cfg in payloads] + + +def tree() -> dict: + """sha256 and byte count of every `app/*.py` this run imported from.""" + _load() + out = {"supports_include": _supports_include()} + for f in sorted((HERE / "app").glob("*.py")): + raw = f.read_bytes() + out[f"app/{f.name}"] = [hashlib.sha256(raw).hexdigest(), len(raw)] + return out + + +def _h(obj) -> str: + return hashlib.sha256(json.dumps(obj, sort_keys=True).encode()).hexdigest() + + +def compare(a_path, b_path) -> int: + a = json.loads(Path(a_path).read_text(encoding="utf-8")) + b = json.loads(Path(b_path).read_text(encoding="utf-8")) + + # --- the control, first: two DIFFERENT trees, and one of them reacts ----- + if a["tree"] == b["tree"]: + print("REFUSED: the two runs have the SAME `app/` fingerprint, so they " + "are one tree twice. An identity is evidence only when the two " + "sides could have differed.") + return 2 + moved = [side for side, doc in (("A", a), ("B", b)) + if doc["set"] != doc["unset"]] + if len(moved) != 1: + print(f"REFUSED: `set` differs from `unset` on {moved or 'NEITHER'} " + "side. Exactly one side must react to `pv_entity` - the HEAD. " + "Neither means the instrument cannot see this card's material " + "at all, and both means the baseline is not a baseline.") + return 2 + head, base = (b, a) if moved == ["B"] else (a, b) + hlabel = "B" if moved == ["B"] else "A" + blabel = "A" if hlabel == "B" else "B" + print(f" HEAD is {hlabel}; baseline is {blabel} (it does not react to " + f"`pv_entity`, which is what a tree without the option does).") + print(f" tree {blabel} {_h(base['tree'])}") + print(f" tree {hlabel} {_h(head['tree'])}") + for what in ("control", "telemetry"): + print(f" {what:<9} {blabel} unset {_h(base['unset'][what])}") + print(f" {what:<9} {hlabel} unset {_h(head['unset'][what])}") + print(f" {what:<9} {hlabel} set {_h(head['set'][what])}") + + rc = 0 + # 1. the acceptance criterion: unset is the baseline, in full. + if head["unset"] == base["unset"]: + print("OK 1/3 UNSET: control AND telemetry are byte-identical to the " + "baseline tree. An install that does not set `pv_entity` sees " + "no change of any kind.") + else: + rc = 1 + for what in ("control", "telemetry"): + for k in sorted(set(head["unset"][what]) | set(base["unset"][what])): + if head["unset"][what].get(k) != base["unset"][what].get(k): + print(f"FAIL 1/3 UNSET {what} MOVED at {k!r}") + # 2. the card's other half: setting it moves no DECISION. + if head["set"]["control"] == head["unset"]["control"]: + print("OK 2/3 SET: the control document is byte-identical with the " + "channel switched on - PV reaches no decision.") + else: + rc = 1 + for k in sorted(head["set"]["control"]): + if head["set"]["control"][k] != head["unset"]["control"][k]: + print(f"FAIL 2/3 CONTROL MOVED at {k!r} when `pv_entity` was " + f"set") + # 3. …and the instrument could have seen it. Without this the first two + # are an identity measured against a document that reaches nothing. + if head["set"]["telemetry"] != head["unset"]["telemetry"]: + n = sum(1 for k in head["set"]["telemetry"] + if head["set"]["telemetry"][k] + != head["unset"]["telemetry"].get(k)) + print(f"OK 3/3 CONTROL OF THE CONTROL: the telemetry document DOES " + f"move when the option is set ({n} stimulus cells differ), so " + f"1/3 and 2/3 are identities the instrument could have broken.") + else: + rc = 2 + print("REFUSED 3/3: the telemetry document did NOT move when " + "`pv_entity` was set. This instrument cannot see the card's " + "material, so nothing above it means anything.") + return rc + + +if __name__ == "__main__": + if len(sys.argv) == 4 and sys.argv[1] == "--compare": + sys.exit(compare(sys.argv[2], sys.argv[3])) + print(json.dumps({ + "tree": tree(), + "stimulus": {"grids": sorted(GRIDS), "pv": sorted(PVS), + "cycles": CYCLES}, + "unset": documents(False), + "set": documents(True), + }, sort_keys=True, indent=1)) diff --git a/goodwe_controller/translations/en.yaml b/goodwe_controller/translations/en.yaml index e31a503..ae790bd 100644 --- a/goodwe_controller/translations/en.yaml +++ b/goodwe_controller/translations/en.yaml @@ -84,6 +84,9 @@ configuration: estop_fitted: name: "Hardware · RS485 emergency stop is fitted" description: "Whether the RS485 emergency stop is physically installed. It only drives the warning banner on the status page; it changes no control behaviour." + pv_entity: + name: "Hardware · PV production entity (W)" + description: "Optional, and MEASUREMENT ONLY - it changes no control behaviour whatsoever. Point it at your solar inverter's own power sensor (any brand Home Assistant reads; it does not have to be on the RS485 bus) and the add-on records production beside the grid and battery figures, so a day of unstored surplus can be told apart from a day with no sun. POSITIVE MUST MEAN PRODUCING. The unit is read from the entity itself - W, kW and the rest of Home Assistant's power units - so a sensor with no unit is refused rather than guessed at, and the status page says so. Leave it empty and nothing is read, nothing is published and nothing is shown." meter_source: name: "Meter · Where the meter reading comes from" description: "'off' keeps the single Home Assistant entity below, which is what an existing installation runs on until you choose otherwise. ha_dsmr subscribes to the DSMR integration; mqtt_p1 reads an MQTT topic; ha_signed reads one signed entity, which is what a HomeWizard P1 publishes; homewizard_local polls the meter's own local API and skips Home Assistant entirely. ⚠️ Only homewizard_local can tell 'the meter has not answered' apart from 'the value has not changed', so it is the only source on which the frozen-meter alarm means anything." diff --git a/goodwe_controller/translations/nl.yaml b/goodwe_controller/translations/nl.yaml index 440dce7..91174d1 100644 --- a/goodwe_controller/translations/nl.yaml +++ b/goodwe_controller/translations/nl.yaml @@ -64,6 +64,9 @@ configuration: estop_fitted: name: "Hardware · RS485-noodstop is geplaatst" description: "Of de RS485-noodstop fysiek geïnstalleerd is. Dit stuurt alleen de waarschuwingsbalk op de statuspagina aan en verandert niets aan de regeling." + pv_entity: + name: "Hardware · Zonneproductie-entiteit (W)" + description: "Optioneel, en UITSLUITEND EEN METING - het verandert niets aan de regeling. Verwijs naar de vermogenssensor van je omvormer (elk merk dat Home Assistant kan lezen; hij hoeft niet op de RS485-bus te zitten) en de add-on registreert de productie naast de meter- en batterijwaarden, zodat een dag met niet-opgeslagen overschot te onderscheiden is van een dag zonder zon. POSITIEF MOET PRODUCEREN BETEKENEN. De eenheid wordt van de entiteit zelf gelezen - W, kW en de overige vermogenseenheden van Home Assistant - dus een sensor zonder eenheid wordt geweigerd in plaats van geraden, en de statuspagina zegt waarom. Laat leeg en er wordt niets gelezen, niets gepubliceerd en niets getoond." meter_source: name: "Meter · Waar de meterstand vandaan komt" description: "'off' houdt de losse Home Assistant-entiteit hieronder aan, waar een bestaande installatie op draait tot u iets anders kiest. ha_dsmr luistert mee met de DSMR-integratie; mqtt_p1 leest een MQTT-topic; ha_signed leest één entiteit met teken, wat een HomeWizard P1 publiceert; homewizard_local bevraagt de lokale API van de meter zelf en slaat Home Assistant helemaal over. ⚠️ Alleen homewizard_local kan 'de meter heeft niet geantwoord' onderscheiden van 'de waarde is niet veranderd', en is dus de enige bron waarop het alarm voor een vastgelopen meter iets betekent."