diff --git a/.gitea/workflows/checks.yml b/.gitea/workflows/checks.yml index 0d4f451..46fb54f 100644 --- a/.gitea/workflows/checks.yml +++ b/.gitea/workflows/checks.yml @@ -101,10 +101,22 @@ jobs: # hwsim section was skipped - so "test_freeze.py: PASS" below is a weaker # claim than it looks. That belongs to whoever owns test_freeze.py next; # naming it here is the least this gate can do about it. + # ⚠️ PYTHONDONTWRITEBYTECODE=1 IS BELT AND BRACES, NOT A FIX FOR A LIVE + # HOLE. TEST-15 measured this runner's actual exposure and found none: the + # checkout below is bind mounted `:ro`, so the interpreter physically + # cannot write a __pycache__ into the tree it is grading - it fails + # SILENTLY, no error and exit status 0 - and every job `rm -rf`s $WS and + # clones afresh, so nothing survives from a previous job on this shared + # runner either. The variable is here because the `:ro` is the load-bearing + # part and it is one character wide: delete it while debugging a write and + # the gate quietly starts caching bytecode across a rebuild again. Note + # what it does NOT do, per TEST-15 part 2 - it stops bytecode being + # WRITTEN, never a stale one being READ. It is not a substitute for the + # purge in goodwe_controller/tools/mutation_run.sh. - name: the add-on's own tests run: | set -eu - docker run --rm -v "$WS:/w:ro" goodwe-addon-checks:1 sh -eu -c ' + docker run --rm -e PYTHONDONTWRITEBYTECODE=1 -v "$WS:/w:ro" goodwe-addon-checks:1 sh -eu -c ' cd /w/goodwe-addon/goodwe_controller set -- test_*.py if [ "$1" = "test_*.py" ]; then diff --git a/HANDOVER.md b/HANDOVER.md index 5e242d9..192db71 100644 --- a/HANDOVER.md +++ b/HANDOVER.md @@ -57,6 +57,14 @@ python3 test_control.py && python3 test_maintenance.py && python3 test_arbiter.p && python3 test_loop.py ``` +⚠️ **Mutating a module to prove a check bites? Read `goodwe_controller/ +MUTATION.md` first, and run `goodwe_controller/tools/mutation_run.sh`.** A +same-size edit reverted inside one second changes neither the file size nor the +whole-second mtime, so CPython reuses the bytecode it already compiled and the +suite grades code that is not there - once already, reported as a survivor and +disproved by disassembly (TEST-15). `python3 -B` does **not** save you: it stops +bytecode being written, not a stale `.pyc` being read. The purge is the fix. + `test_watchdog.py` covers SAFETY-02's logic, its arming gate (derived from the real arbiter, not passed in), its trip path, and the fact that the latch survives an add-on restart. Its headline validation is not here at all: `sim/replay_sign.py` @@ -165,6 +173,11 @@ Each of these was found the expensive way. They are in the code comments too. `timestamp` attribute. - **Alpine ships paho-mqtt 1.x**, which has no `CallbackAPIVersion`, and musl has no aiohttp wheel — install from apk, not pip. +- **A mutation run can grade code that is not there.** CPython validates a + `.pyc` on (source size, source mtime in whole seconds), so a same-size edit + reverted within one second is invisible to it and the old bytecode runs. It + manufactures *survivors*, never reds. Purge `__pycache__` - + `tools/mutation_run.sh` does - and see `goodwe_controller/MUTATION.md`. - **The inverter holds its last command forever.** It has no meter-timeout. Every failsafe here exists because of that one fact, and anything in doubt commands **0 W** rather than holding. diff --git a/README.md b/README.md index 7a00479..e40327d 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,8 @@ goodwe_controller/ the add-on test_control.py runnable checks — no framework needed test_maintenance.py runnable checks — walks a full cycle in fake time DOCS.md the add-on's documentation tab + MUTATION.md how to run a mutation so the result means something + tools/mutation_run.sh the mutation harness - purges bytecode, then grades firmware/goodwe-master.yaml ESPHome config for the T-CAN485 estop/rs485_log.py optional RS485 e-stop / bus witness docs/GoodWe-RS485-Field-Guide.pdf the guide techs carry (10pp, A4) @@ -82,7 +84,12 @@ python3 test_control.py python3 test_maintenance.py ``` -Both must pass before shipping any change. They are not unit-test theatre — +Both must pass before shipping any change. To prove a check *bites* - the way +almost every claim in this repository was established - mutate the module and +run `tools/mutation_run.sh`, which clears `__pycache__` first. It is not +optional: a same-size edit reverted inside one second leaves CPython reusing the +old bytecode, and the run grades code that is not there. `goodwe_controller/MUTATION.md` +has the mechanism, the measurement, and what the flags do not do. They are not unit-test theatre — each assertion corresponds to a rule whose absence produced an observed failure on real hardware, and the comments in `control.py` say which. diff --git a/goodwe_controller/CHANGELOG.md b/goodwe_controller/CHANGELOG.md index ba6e6ff..69ea3ca 100644 --- a/goodwe_controller/CHANGELOG.md +++ b/goodwe_controller/CHANGELOG.md @@ -2,6 +2,62 @@ ## Unreleased +**A mutation run could grade code that was not there, and the fix is a purge +rather than a flag (TEST-15).** FCST-03's developer reverted `self.provisional` +to `here.provisional`, re-ran, and reported a survivor; disassembly showed the +source read `here` while the running code loaded `self`. CPython validates a +`.pyc` on (source size, source mtime truncated to whole seconds), and a +size-preserving edit reverted inside one second moves neither number, so the +stale bytecode was reused. That does not make past results wrong - it made them +unverified, and the distinction is the whole card. + +Reproduced rather than reasoned about. `test_bytecode.py` builds a real package +whose two return values are the same length, imports it in a real subprocess, +rewrites it inside the same second and reads back which value ran. Part 1 shows +the interpreter running `SELF` off a file that says `HERE`. + +**And it shows the two obvious mitigations do not work.** `python3 -B` and +`PYTHONDONTWRITEBYTECODE=1` are write-side only: neither stops an EXISTING stale +`.pyc` being read, and a mutation run always has one because the previous run +wrote it. The advice already in `test_control.py`'s coverage audit led with `-B` +and is corrected here - the purge is the half that works, and `-B` earns its +place by protecting the NEXT edit rather than this one. No `sleep` anywhere: +one-second mtime resolution is the mechanism, and waiting it out would slow every +run down while still only covering the edit somebody remembered to slow. + +`goodwe_controller/tools/mutation_run.sh` is the harness the house adopts: it +purges, prints a sha256/byte fingerprint of every `app/*.py` it is about to +grade - so a mutation report can say WHICH source produced it - runs each suite +under `python3 -B`, refuses a tree with no `test_*.py` (CI-01's rule), and fails +if a cache appears during the run. It takes the tree as an argument so the infra +repo's replay harnesses can call it. + +`MUTATION.md` states the guarantee where a mutation gets written, and every one +of the 16 suites now points at it from its docstring - a list `test_bytecode.py` +derives from the tree, so a new suite joining without the pointer is red. + +CI was already immune and is described that way rather than as a hole being +closed: `checks.yml` mounts the checkout `-v "$WS:/w:ro"`, and a read-only tree +cannot take a `__pycache__` at all - measured, and it fails silently, exit +status 0 - while each job `rm -rf`s its workspace and clones afresh. +`PYTHONDONTWRITEBYTECODE=1` is added as belt and braces, because the `:ro` is one +character wide. The shipped add-on was never affected: `Dockerfile` has set +`PYTHONDONTWRITEBYTECODE=1` since it was written. + +**The survey of this week's mutation claims is in `MUTATION.md`, scoped by the +property rather than by re-running everything.** The defect is one-directional - +stale bytecode turns a red into a green survivor and never the reverse - so every +"all red, no survivors" result in the window is not at risk, and the exposure is +the recorded SURVIVORS from source-edit runs made before the purge discipline +landed on 2026-08-31. Two of them were re-run here under the harness and both +still stand (TARIFF-01's `record_block()` ignoring its `policy` argument; +SAFETY-03's `abs()` on the clamp limit). The rest are labelled still-unverified +rather than quietly assumed or noisily re-run. + +Out of scope and recorded: the infra repo's `sim/` replay harnesses import the +add-on and write bytecode into both trees. Anyone mutating a module graded by a +replay must purge both by hand until that repo adopts the harness. + **`release/1.0` went red at midnight on a commit nobody had touched (TEST-17).** `Controller.checks()` read the capacity-tariff peak with `self.peak.peak()` - no argument - so `tariff.py` resolved the month from `datetime.now()`, while diff --git a/goodwe_controller/MUTATION.md b/goodwe_controller/MUTATION.md new file mode 100644 index 0000000..54309cd --- /dev/null +++ b/goodwe_controller/MUTATION.md @@ -0,0 +1,153 @@ +# Mutation runs in this repository — and the one way they lie + +Nearly every safety claim in this add-on was established the same way: change +the shipped module, run the suites, watch one go **red**, put the module back, +watch it go **green**. There is no coverage tool here and there is not meant to +be — the check that matters is whether a *check* notices when the behaviour it +names is removed. + +That method has exactly one silent failure mode, and TEST-15 exists because we +walked into it. + +## The defect + +FCST-03's developer reverted `self.provisional` to `here.provisional`, re-ran, +and reported the mutation as a **survivor**. It was not. CPython decides a +cached `.pyc` is still valid from two numbers: + +* the source file's **size in bytes**, and +* the source file's **mtime truncated to whole seconds**. + +`self` and `here` are the same length, and the revert happened in the same +second as the edit. Neither number moved, so the interpreter loaded the +bytecode it already had. Confirmed at the time by disassembly: **the source read +`here`, the running code loaded `self`.** The suite had graded code that was not +there — no error, no warning, exit status 0. + +The same collision, in the other direction, can leave a **red** standing on +correct source after a revert that never took. + +### Which way it lies, and why that scopes everything + +A mutation that never took effect leaves the suite **green**, and a green run of +a mutated tree is reported as a survivor. So stale bytecode **manufactures +survivors**; it cannot manufacture a red. The false-red case exists but +announces itself, because a developer chases a failure they cannot explain. + +Two consequences, and both are used below: + +* a recorded **"all red, no survivors"** result is not at risk from this defect; +* a recorded **survivor** — and a recorded *zero*, such as a coverage-audit + count of 0 — is exactly what this defect produces. + +`test_control.py`'s coverage-audit table says the same thing in its own terms: +the error is one-directional, so every figure there is a lower bound at worst. + +## The rule + +**Purge `__pycache__` before every mutation run.** Not afterwards, not usually, +not when the edit looks big enough. + +```sh +cd goodwe_controller +tools/mutation_run.sh # every test_*.py in this directory +tools/mutation_run.sh . test_tariff.py # or just the ones you are watching +tools/mutation_run.sh "$GOODWE_ADDON_REPO/goodwe_controller" # from elsewhere +``` + +The harness purges, prints a `sha256`/byte-count fingerprint of every `app/*.py` +it is about to grade, runs each suite under `python3 -B`, and fails if a cache +appeared during the run. Diff two fingerprints and the file you mutated is the +line that changed — which is also how a mutation report proves *which source* +produced it. + +### What does **not** fix this + +| | writes bytecode | reads a stale `.pyc` | +|---|---|---| +| `python3 -B` | no | **yes** | +| `PYTHONDONTWRITEBYTECODE=1` | no | **yes** | +| purging `__pycache__` | (unchanged) | **no — nothing to read** | + +Both flags are *write-side only*. A mutation run always has a cache already, +because the previous run wrote it, so a harness that only adds `-B` closes +nothing. This is measured, not read off documentation: `test_bytecode.py` +part 2 arranges the collision and shows both flags still running the old code. + +`-B` is still worth passing — it stops **this** run leaving a cache for the +**next** edit to inherit, which is the one nobody is thinking about three +mutations into a batch. Purge for this run, `-B` for the next one. + +### And not a `sleep` + +One-second mtime resolution is the *mechanism*, not the enemy. Waiting out the +clock would slow every run down, and would still only cover the edit somebody +remembered to slow down. Stop trusting mtime instead. + +## Reporting a mutation + +State the things a reviewer cannot reconstruct afterwards: + +1. **the byte delta** of the mutated file (`+5 B`, `+0 B`). `+0 B` is the risky + one — say so; +2. **that the mutated line was on disk** when the run started (the harness + fingerprint, or a `grep` of the line); +3. **which suite went red, and on which named check** — "the suite is red" and + "the check that watches this is red" are different claims; +4. for a **survivor**, that the run was purged. An unpurged survivor is not a + result. + +## Where each surface stands + +* **Local and dev-harness runs — the exposed one.** This is where FCST-03 hit + it. `goodwe_controller/__pycache__` and `app/__pycache__` are gitignored and + are sitting in most working trees right now. Use the harness. +* **CI — structurally immune, and it was before this card.** `.gitea/workflows/ + checks.yml` bind mounts the checkout `-v "$WS:/w:ro"`, so the interpreter + cannot write a `__pycache__` into the tree under test at all; measured for + TEST-15, and it fails *silently* — no error, exit status 0. Each job also + `rm -rf`s its workspace and clones afresh, so nothing crosses between jobs on + the shared runner. `PYTHONDONTWRITEBYTECODE=1` is now set there too, as belt + and braces: the `:ro` is one character wide and the next person debugging a + write will be tempted by it. +* **The shipped add-on — not affected, and this is closed.** + `goodwe_controller/Dockerfile` has carried `PYTHONDONTWRITEBYTECODE=1` since + it was written. There is no deploy-side finding here; do not re-open it. +* **The infra repo's `sim/` harnesses — exposed, and NOT fixed by this card.** + `sim/replay_sign.py` and friends import the add-on out of + `$GOODWE_ADDON_REPO` and write `__pycache__` into *both* trees + (`goodwe/sim/__pycache__` exists today). SAFETY-02's headline validation is a + replay, not a suite, so a mutation graded through it carries the same + exposure. `tools/mutation_run.sh` takes the tree to purge as its first + argument precisely so that repo can call it, but the change belongs in that + repo and some of those harnesses want the rig. **Anyone mutating a module + graded by a replay must purge both trees by hand until then.** + +## Survey — mutation claims of 2026-08-25 … 2026-09-01 + +Scoped **by the property, not by re-running everything**. A claim is at risk +only if a mutation could have preserved the file's byte count *and* been +reverted inside the same second, *and* the claim is of the kind stale bytecode +can fabricate (see "which way it lies"). Sources: `git log release/1.0` and the +commit and PR bodies. + +| claim | verdict | why | +|---|---|---| +| **FCST-03** M9 `self.provisional`→`here.provisional` (dd40c56, 71cd784) | **re-verified** | this is the original false survivor. Re-run with the cache cleared, M9 is caught. The shipped source was correct throughout. | +| **FCST-03** X1, X6 degenerate stimuli (71cd784) | **not at risk** | run after the purge discipline landed in that branch. | +| **SAFETY-24** 11 author + 9 reviewer mutations (e709554, 4601682) | **not at risk** | the commit records them run under `python -B` with a purged `__pycache__` and real byte deltas. Its one survivor is an equivalent mutant, argued structurally rather than by a green run. | +| **SAFETY-26** tolerance sweep over `arbitrage.py` and `static_tariff.py`, 2 survivors (2b8b8cb) | **not at risk** | purge, printed `+5 B` delta, mutated line asserted present, tree re-read before each run. One survivor is explicitly recorded at `+0 B` — the risky delta — and was produced *under* the purge. | +| **OPT-04** mutations reported on PR #22 (bdd7f18) | **not at risk** | `test_arbitrage.py`'s header records the exact command: purge, then `python3 -B`. | +| **TEST-17** `checks()` dropping `when` (504e8da) | **not at risk** | a *red* claim, and dropping an argument changes the file length. Neither half of the collision is available. | +| **SAFETY-07** wiring mutations B1/B2/B3 and `test_peakguard.py` parts 7 and 10 (2d570d8, e7c95bf) | **not at risk** | those mutations are **executed in-process**, monkeypatched onto the imported module. No source file is edited, so no `.pyc` is consulted. | +| every **"all red, no survivors"** claim in the window — TARIFF-02 (24/0), TARIFF-03 (11), MAINT-04 (10), SAFETY-18 (6+4), SAFETY-03 (6), TEL-04 (10), TEL-05 (6) | **not at risk** | direction. Stale bytecode turns a red into a green survivor, never the reverse. A red is evidence that *something* changed under the check. | +| **TEST-06**'s deleted assertion, "no new notification while parked at 0 W" (c34b5a2) | **not at risk / superseded** | it was recorded as undecidable and then *falsified* — `test_freeze.py` records the one-line mutation that takes the file red 9 times. The stronger later claim is a red one. | +| **TARIFF-01** reviewer survivor: `record_block()` ignoring its `policy` argument (0dbd485) | **re-verified — still a survivor** | re-run here through `tools/mutation_run.sh` against the 15 suites that were on the record, all green with `billed_w(block, policy)` mutated to `billed_w(block, POLICY_NET)`. The recorded diagnosis holds: every block reaching the store is single-phase, where NET, SUM and MAX collapse. | +| **SAFETY-03** reviewer survivor: `abs()` dropped from `clamp_to_rating`'s limit (6aabb83) | **re-verified — still a survivor** | re-run the same way, all 15 green. Also as recorded: unreachable under the `int(100, 5000)` schema. | +| **TARIFF-01**'s other three survivors, **SAFETY-03**'s `describe()` survivor, **TARIFF-02**'s M3/M5/M7, **SAFETY-18**'s three | **still unverified** | recorded before the purge discipline, with no byte delta on the record, so a `+0 B` same-second revert cannot be excluded. All are survivors, which is the direction this defect fabricates. TARIFF-02's and SAFETY-18's were **closed by later commits whose claims are reds**, so what is unverified is the *diagnosis*, not the current state of the code. | + +**Nothing here says a past result is wrong.** The two survivors re-run above +both stood. What changed is that they are now on the record as measured rather +than assumed, and the rest are labelled honestly. The still-unverified rows are +cheap to close — one harness run each — and the right moment is when somebody +next touches that module, not in a re-run campaign nobody will read. diff --git a/goodwe_controller/test_arbiter.py b/goodwe_controller/test_arbiter.py index 3ff0b08..11401b7 100644 --- a/goodwe_controller/test_arbiter.py +++ b/goodwe_controller/test_arbiter.py @@ -3,6 +3,13 @@ Each assertion is a precedence question someone will eventually ask in the field: "why did it charge during a peak?", "why did it discharge while the maintenance cycle was charging?", "what happens if two things disagree?". + +⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run +`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it +stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving +edit reverted inside one second is invisible to CPython's staleness test. That +is TEST-15, and it has already turned one live mutation into a false survivor. +Read MUTATION.md before you report a survivor. """ import sys diff --git a/goodwe_controller/test_arbitrage.py b/goodwe_controller/test_arbitrage.py index 1f030cf..f4882c8 100644 --- a/goodwe_controller/test_arbitrage.py +++ b/goodwe_controller/test_arbitrage.py @@ -62,6 +62,13 @@ THREE RULES THIS FILE HOLDS ITSELF TO, because the reviewer will: aged out, an hour the series does not cover, an unread state of charge and a peak guard that never ran are five DIFFERENT reasons - and none of them is an answer. Section 5 drives all five and asserts they are distinguishable. + +⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run +`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it +stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving +edit reverted inside one second is invisible to CPython's staleness test. That +is TEST-15, and it has already turned one live mutation into a false survivor. +Read MUTATION.md before you report a survivor. """ import ast diff --git a/goodwe_controller/test_bytecode.py b/goodwe_controller/test_bytecode.py new file mode 100644 index 0000000..f7916e9 --- /dev/null +++ b/goodwe_controller/test_bytecode.py @@ -0,0 +1,296 @@ +"""Runnable check for TEST-15 - the mutation harness's own honesty. +`python3 test_bytecode.py` + +Every other file in this directory grades the add-on. This one grades the thing +that grades the add-on, because in this repository a mutation run IS the +evidence: "I broke it, the suite went red, I put it back, the suite went green" +is how nearly every safety claim here was established. + +THE DEFECT, found by FCST-03's developer and reproduced below. CPython decides a +cached `.pyc` is still good from two numbers: the source's SIZE, and its +MTIME TRUNCATED TO WHOLE SECONDS. Reverting `self.provisional` to +`here.provisional` moves neither - same length, same second - so the interpreter +loaded the bytecode it already had. Confirmed then by disassembly: the source +read `here`, the running code loaded `self`. The mutation was reported as a +survivor when in truth it had never run. + +WHICH DIRECTION IT LIES IN. A mutation that never took effect makes the suite +stay GREEN, so it is reported as a survivor: stale bytecode MANUFACTURES +SURVIVORS. The opposite - a revert that never took, leaving a red on correct +source - is possible too but announces itself, because a developer chases an +unexplained failure. That asymmetry is why the coverage-audit table in +test_control.py is a lower bound rather than nonsense, and it is what the survey +in MUTATION.md is scoped by. + +WHAT THIS FILE PINS, in the order the card's acceptance criteria name it: + + AC1 The defect reproduces (part 1), the two mitigations that DO NOT fix it + are shown not to (part 2), the one that does is shown to (part 3), and + the shipped harness is shown to close it end to end on a tree that would + otherwise mis-grade (part 4). Executed, not reasoned: every part builds a + real package, imports it in a real subprocess, and reads back which + value the running code returned. + AC2 The guarantee is where a developer writing a mutation reads it: part 5 + requires EVERY test_*.py in this directory to point at MUTATION.md. The + list is derived from the tree, so a new suite that joins without the + pointer is red - CI-01's rule, for CI-01's reason. + AC4 Part 6 pins the two properties that make CI immune, on the workflow file + itself. + +⚠️ NOTHING HERE SLEEPS. One-second mtime resolution is the mechanism; a harness +that waits out the clock would be slower on every run and would still only cover +the edit somebody remembered to slow down. Part 1 REPRODUCES the collision +deliberately (it retries until two writes land in the same second, which takes +milliseconds) rather than avoiding it. + +⚠️ This file writes only under a temporary directory. It must stay that way: CI +mounts the checkout READ-ONLY, and a check that needs to write into the tree +would fail there for a reason that has nothing to do with what it grades. + +⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or use +`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it +stops bytecode being WRITTEN, not a stale one being READ. See MUTATION.md. +""" + +import os +import shutil +import subprocess +import sys +import tempfile +from pathlib import Path + +HERE = Path(__file__).resolve().parent +fails = [] + + +def check(name, cond): + if cond: + print(f" ok {name}") + else: + print(f" FAIL {name}") + fails.append(name) + + +# --------------------------------------------------------------------------- +# The fixture. `who()` returns one of two names that are THE SAME LENGTH, so +# swapping them leaves the file byte count identical - the first half of +# CPython's staleness test. The second half is the whole-second mtime, which +# part 1 arranges by writing both versions inside one second. +# +# It is deliberately not a copy of a shipped module: the property under test is +# CPython's, and using real code here would make this file go red whenever that +# module changed for unrelated reasons. +# --------------------------------------------------------------------------- +V_OLD = 'def who():\n self = "SELF"\n here = "HERE"\n return self\n' +V_NEW = 'def who():\n self = "SELF"\n here = "HERE"\n return here\n' +assert len(V_OLD) == len(V_NEW), "the fixture must be size-preserving or it grades nothing" + +PROBE = "from app.m import who\nprint(who())\n" +# The harness runs `test_*.py` files and grades them by exit status, so the +# probe wears that shape in part 4: it FAILS when it sees the pre-mutation value. +TEST_PROBE = ( + "from app.m import who\n" + "v = who()\n" + "print('RUNS:', v)\n" + "raise SystemExit(0 if v == 'HERE' else 1)\n" +) + + +def run(tree, args, env_extra=None, script="probe.py"): + env = dict(os.environ) + env.pop("PYTHONDONTWRITEBYTECODE", None) + if env_extra: + env.update(env_extra) + out = subprocess.run( + [sys.executable] + args + [script], + cwd=tree, env=env, capture_output=True, text=True, + ) + return out.stdout.strip() + + +def caches(tree): + return sorted(str(p.relative_to(tree)) for p in Path(tree).rglob("__pycache__")) + + +def seed(tree): + """Leave `tree` holding V_NEW on disk and V_OLD in its bytecode cache. + + Returns True when the collision was actually arranged. Both writes must land + in the same whole second; at a few milliseconds per cycle that is the normal + case, and the caller retries rather than asserting on one attempt, because a + machine that happens to cross a second boundary mid-cycle would otherwise + report a defect in CPython instead of a slow disk. + """ + app = Path(tree) / "app" + shutil.rmtree(app, ignore_errors=True) + app.mkdir(parents=True) + (app / "__init__.py").write_text("") + (Path(tree) / "probe.py").write_text(PROBE) + (Path(tree) / "test_probe.py").write_text(TEST_PROBE) + + m = app / "m.py" + m.write_text(V_OLD) + t0 = m.stat().st_mtime + run(tree, []) # compiles V_OLD, writes app/__pycache__ + m.write_text(V_NEW) # the "revert": same size, same second + t1 = m.stat().st_mtime + return int(t0) == int(t1) and caches(tree) != [] + + +def seeded_tree(stack): + tree = tempfile.mkdtemp(prefix="test_bytecode.") + stack.append(tree) + for _ in range(40): + if seed(tree): + return tree + return None + + +tmpdirs = [] +print("stale bytecode (TEST-15)") + +# --------------------------------------------------------------------------- +print("\n1. the defect: a same-size edit inside one second is not seen at all") +# --------------------------------------------------------------------------- +tree = seeded_tree(tmpdirs) +check("the collision can be arranged at all (same size, same second)", tree is not None) +if tree is None: + print(" cannot continue - the fixture never landed two writes in one second") + raise SystemExit(1) + +src = (Path(tree) / "app" / "m.py").read_text() +check("the source on disk says HERE", "return here" in src and "return self" not in src) +check("...and a plain interpreter runs SELF anyway", run(tree, []) == "SELF") +# Stated as its own check because this is the whole ticket in one line: what the +# file says and what the process executes have come apart, with no error, no +# warning, and a zero exit status. + +# --------------------------------------------------------------------------- +print("\n2. the two mitigations that do NOT close it") +# --------------------------------------------------------------------------- +# ⚠️ READ THIS BEFORE 'FIXING' A HARNESS WITH -B ALONE. Both of these stop +# bytecode being WRITTEN. Neither stops a stale one being READ, and a mutation +# run always has one already: the previous run wrote it. The advice in +# test_control.py's coverage audit said `-B` first and CLEAR the cache second; +# the order is not a style choice, the clear is the half that works. +check("`python3 -B` over an existing cache still runs SELF", run(tree, ["-B"]) == "SELF") +check( + "PYTHONDONTWRITEBYTECODE=1 over an existing cache still runs SELF", + run(tree, [], {"PYTHONDONTWRITEBYTECODE": "1"}) == "SELF", +) + +# --------------------------------------------------------------------------- +print("\n3. the purge closes it, and -B keeps the next run clean") +# --------------------------------------------------------------------------- +for p in Path(tree).rglob("__pycache__"): + shutil.rmtree(p, ignore_errors=True) +check("after the purge, the plain interpreter runs HERE", run(tree, []) == "HERE") +check("...and it has written a fresh cache back", caches(tree) != []) + +tree2 = seeded_tree(tmpdirs) +check("second fixture seeded", tree2 is not None) +if tree2: + for p in Path(tree2).rglob("__pycache__"): + shutil.rmtree(p, ignore_errors=True) + check("purge + -B runs HERE", run(tree2, ["-B"]) == "HERE") + check("...and leaves NO cache for the next mutation to inherit", caches(tree2) == []) + # The second half is why the harness passes -B at all. Purging before a run + # protects THIS run; not writing one protects the NEXT edit, which is the + # one nobody is thinking about when they are three mutations into a batch. + +# --------------------------------------------------------------------------- +print("\n4. the shipped harness, executed against a tree that would mis-grade") +# --------------------------------------------------------------------------- +harness = HERE / "tools" / "mutation_run.sh" +check("tools/mutation_run.sh exists", harness.is_file()) +check("...and is executable", os.access(harness, os.X_OK)) + +tree3 = seeded_tree(tmpdirs) +check("third fixture seeded", tree3 is not None) +if tree3 and harness.is_file(): + # NON-VACUITY FIRST. If the naked run below were already green, the harness + # run after it would prove nothing - it would be passing on a tree that + # needed no help. This is the same trap test_peakguard.py part 7 documents: + # a mutation harness that patches nothing passes beautifully. + naked = subprocess.run( + [sys.executable, "test_probe.py"], cwd=tree3, capture_output=True, text=True + ) + check("without the harness the tree grades the STALE source (rc!=0)", naked.returncode != 0) + check("...and says so: it saw SELF", "SELF" in naked.stdout) + + got = subprocess.run( + ["sh", str(harness), tree3], cwd=tree3, capture_output=True, text=True + ) + check("the harness grades the source on disk (rc==0)", got.returncode == 0) + check("...having reported the purge", "no bytecode cache" in got.stdout) + check("...and it leaves no cache behind", caches(tree3) == []) + if got.returncode != 0: + print(got.stdout[-2000:]) + print(got.stderr[-2000:]) + + # The harness must also refuse to report on nothing - CI-01's zero-file rule. + empty = tempfile.mkdtemp(prefix="test_bytecode.empty.") + tmpdirs.append(empty) + none = subprocess.run( + ["sh", str(harness), empty], cwd=empty, capture_output=True, text=True + ) + check("the harness refuses a tree with no test_*.py", none.returncode != 0) + +# --------------------------------------------------------------------------- +print("\n5. the guarantee is stated where a mutation gets written") +# --------------------------------------------------------------------------- +# ⚠️ THE LIST IS DERIVED FROM THE TREE. A hand-written one is how a new suite +# joins this directory carrying no warning at all, which is the exact shape of +# failure CI-01's file glob exists to prevent. If you add a test_*.py, put the +# pointer in its docstring; that is the whole cost. +doc = HERE / "MUTATION.md" +check("MUTATION.md exists", doc.is_file()) +if doc.is_file(): + text = doc.read_text() + check("...and it names the purge as the load-bearing half", "__pycache__" in text and "purge" in text.lower()) + check("...and warns that -B alone does not read fresh", "-B" in text) + check("...and points at the harness", "mutation_run.sh" in text) + +suites = sorted(p.name for p in HERE.glob("test_*.py")) +check("there are test suites to survey at all", len(suites) >= 15) +missing = [n for n in suites if "MUTATION.md" not in (HERE / n).read_text()] +check(f"every one of the {len(suites)} suites points at MUTATION.md", missing == []) +if missing: + print(" missing the pointer:", ", ".join(missing)) + +# --------------------------------------------------------------------------- +print("\n6. CI cannot carry a stale artefact between jobs") +# --------------------------------------------------------------------------- +# ⚠️ THE FIRST TWO PROPERTIES ARE THE ONES THAT ACTUALLY DO THE WORK, AND THEY +# PREDATE THIS CARD. Measured for TEST-15, not assumed: with the checkout bind +# mounted READ-ONLY the interpreter writes no __pycache__ into the tree at all, +# and it does so SILENTLY - no error, no warning, exit status 0. The workflow +# also `rm -rf`s its workspace and clones afresh, so nothing survives from a +# previous job even on the same runner. PYTHONDONTWRITEBYTECODE=1 is belt and +# braces on top: it is what keeps this true if somebody ever needs to drop the +# `:ro`, and dropping the `:ro` is a one-character edit that nothing else here +# would notice. +wf = HERE.parent / ".gitea" / "workflows" / "checks.yml" +check("the workflow file is where this expects it", wf.is_file()) +if wf.is_file(): + y = wf.read_text() + # ⚠️ GRADED ON THE COMMAND LINE, NOT ON THE FILE. A `in y` search over the + # whole workflow passes on the PROSE around the step - this file's first + # draft did exactly that, and stayed green after the `-e` flag and the `:ro` + # were deleted, because the comment explaining them still named both. + runs = [l for l in y.splitlines() if "docker run" in l and ":/w" in l] + check("exactly one docker run mounts the checkout", len(runs) == 1) + line = runs[0] if runs else "" + check("the checkout is mounted read-only", ':/w:ro"' in line) + check("PYTHONDONTWRITEBYTECODE=1 is set on that run", "-e PYTHONDONTWRITEBYTECODE=1" in line) + check("the workspace is removed before each checkout", 'rm -rf "$WS"' in y) + check("no sleep was introduced to dodge the mtime resolution", "sleep " not in y) + +for d in tmpdirs: + shutil.rmtree(d, ignore_errors=True) + +print() +if fails: + print(f"{len(fails)} FAILED: " + "; ".join(fails)) + sys.exit(1) +print("all checks passed") diff --git a/goodwe_controller/test_clamp.py b/goodwe_controller/test_clamp.py index b9e3d0c..b8c9958 100644 --- a/goodwe_controller/test_clamp.py +++ b/goodwe_controller/test_clamp.py @@ -47,6 +47,13 @@ WHAT THIS CHECK DOES NOT COVER asserted here over every value that reached `set_number()`, which is the last thing this process controls; proving it at the inverter needs the witness log and belongs with DRIVER-07. + +⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run +`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it +stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving +edit reverted inside one second is invisible to CPython's staleness test. That +is TEST-15, and it has already turned one live mutation into a false survivor. +Read MUTATION.md before you report a survivor. """ import asyncio diff --git a/goodwe_controller/test_control.py b/goodwe_controller/test_control.py index 9dd7458..77b27c8 100644 --- a/goodwe_controller/test_control.py +++ b/goodwe_controller/test_control.py @@ -6,6 +6,13 @@ its absence caused an observed failure on real hardware. If you change control.py, run this. If it fails, the inverter would have done something you did not intend. + +⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run +`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it +stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving +edit reverted inside one second is invisible to CPython's staleness test. That +is TEST-15, and it has already turned one live mutation into a false survivor. +Read MUTATION.md before you report a survivor. """ import sys @@ -71,14 +78,22 @@ def check(name, cond): # Hence: A FIXTURE MUST SIT CLEAR OF EVERY RAIL IT IS NOT TESTING. Where a test # names one mechanism, make that mechanism the binding one (see TCLAMP and TF). # -# ⚠️ RUN MUTATIONS WITH `python -B` AND CLEAR app/__pycache__. CPython -# invalidates a .pyc on (source mtime in whole seconds, source size), so a -# same-second rewrite that also preserves the file size reuses stale bytecode -# and the suite reports on code you are no longer running. It under-reported one -# mutation here as 2 where the true figure is 6. The error is one-directional - -# stale bytecode can only under-report - so every figure above is a lower bound -# at worst, and the two zeros ever recorded were both confirmed by fixing them -# and watching the count rise, which a caching artefact cannot do. +# ⚠️ RUN MUTATIONS THROUGH `tools/mutation_run.sh`, WHICH CLEARS __pycache__ +# BEFORE IT RUNS ANYTHING. CPython invalidates a .pyc on (source mtime in whole +# seconds, source size), so a same-second rewrite that also preserves the file +# size reuses stale bytecode and the suite reports on code you are no longer +# running. It under-reported one mutation here as 2 where the true figure is 6. +# +# ⚠️ AND THE CLEAR IS THE HALF THAT WORKS. This note used to lead with +# `python -B`; TEST-15 measured what that flag actually does, and it is +# write-only - it does not stop an EXISTING stale .pyc being read, and a +# mutation run always has one, because the previous run wrote it. -B is still +# worth passing, for the NEXT edit rather than this one. +# +# The error is one-directional - stale bytecode can only under-report - so +# every figure above is a lower bound at worst, and the two zeros ever recorded +# were both confirmed by fixing them and watching the count rise, which a +# caching artefact cannot do. # --------------------------------------------------------------------------- print("control law") diff --git a/goodwe_controller/test_freeze.py b/goodwe_controller/test_freeze.py index d9f6a9f..fe44e1d 100644 --- a/goodwe_controller/test_freeze.py +++ b/goodwe_controller/test_freeze.py @@ -62,6 +62,13 @@ load the meter's POWER FIGURE never moved - 10130.0 W for 26 consecutive polls - and only the kWh registers kept `unchanged_s` at zero. See the note in freeze.py; it is why the fingerprint's register terms are the negative case rather than a nicety. + +⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run +`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it +stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving +edit reverted inside one second is invisible to CPython's staleness test. That +is TEST-15, and it has already turned one live mutation into a false survivor. +Read MUTATION.md before you report a survivor. """ import asyncio diff --git a/goodwe_controller/test_loop.py b/goodwe_controller/test_loop.py index 61292b6..69eba3a 100644 --- a/goodwe_controller/test_loop.py +++ b/goodwe_controller/test_loop.py @@ -19,6 +19,13 @@ HANDOVER.md. An understated runtime reads as a hang. `cycle_floor_s` is read from options with no lower bound in code, so a huge value reproduces the pre-fix behaviour exactly. Every claim below is therefore an A/B on one binary, not a comparison against a remembered old one. + +⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run +`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it +stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving +edit reverted inside one second is invisible to CPython's staleness test. That +is TEST-15, and it has already turned one live mutation into a false survivor. +Read MUTATION.md before you report a survivor. """ import asyncio diff --git a/goodwe_controller/test_maintenance.py b/goodwe_controller/test_maintenance.py index 54172a6..b582f0d 100644 --- a/goodwe_controller/test_maintenance.py +++ b/goodwe_controller/test_maintenance.py @@ -3,6 +3,13 @@ Walks a whole cycle in simulated time, which is the test the YAML version never had - and it is exactly the phase transitions that a real cycle takes ten hours to exercise once a month. + +⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run +`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it +stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving +edit reverted inside one second is invisible to CPython's staleness test. That +is TEST-15, and it has already turned one live mutation into a false survivor. +Read MUTATION.md before you report a survivor. """ import sys diff --git a/goodwe_controller/test_p1.py b/goodwe_controller/test_p1.py index 9d6e5b8..4983200 100644 --- a/goodwe_controller/test_p1.py +++ b/goodwe_controller/test_p1.py @@ -8,6 +8,13 @@ Every assert here is a rule whose absence poisons something downstream: an inverted sign inverts the control loop, a fabricated zero hides a dead meter, a naive per-phase sum misbills the capacity tariff, and a plain mean of samples computes the wrong quarter-hour figure whenever the telegram cadence changes. + +⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run +`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it +stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving +edit reverted inside one second is invisible to CPython's staleness test. That +is TEST-15, and it has already turned one live mutation into a false survivor. +Read MUTATION.md before you report a survivor. """ import asyncio diff --git a/goodwe_controller/test_peakguard.py b/goodwe_controller/test_peakguard.py index 0485991..15fc55f 100644 --- a/goodwe_controller/test_peakguard.py +++ b/goodwe_controller/test_peakguard.py @@ -72,6 +72,13 @@ Two days is also not thirty. AC 3's 95 %-over-30-days criterion is TARIFF-01's and is still not met; `test_tariff.py`'s part 9 says so at length and this file does not re-litigate it. What part 6 reports is the comparison that two days CAN support: the same quarters, with and without the guard. + +⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run +`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it +stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving +edit reverted inside one second is invisible to CPython's staleness test. That +is TEST-15, and it has already turned one live mutation into a false survivor. +Read MUTATION.md before you report a survivor. """ import asyncio diff --git a/goodwe_controller/test_price.py b/goodwe_controller/test_price.py index a454d7f..68132d0 100644 --- a/goodwe_controller/test_price.py +++ b/goodwe_controller/test_price.py @@ -34,6 +34,13 @@ TWO RULES THIS FILE HOLDS ITSELF TO, because the reviewer will: - the DST checks run on the 23-hour and 25-hour days, not on a normal 24-hour day where a fixed-24-slot implementation and a correct one agree exactly. + +⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run +`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it +stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving +edit reverted inside one second is invisible to CPython's staleness test. That +is TEST-15, and it has already turned one live mutation into a false survivor. +Read MUTATION.md before you report a survivor. """ import ast diff --git a/goodwe_controller/test_price_forecast.py b/goodwe_controller/test_price_forecast.py index 8e1e9d7..26e1194 100644 --- a/goodwe_controller/test_price_forecast.py +++ b/goodwe_controller/test_price_forecast.py @@ -48,6 +48,13 @@ THREE RULES THIS FILE HOLDS ITSELF TO, because the reviewer will: and a stale one is not a fresh one: the gate blocks on `no_data`, blocks on an hour the series does not cover, and blocks on a provisional hour - three different reasons, none of them an answer. + +⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run +`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it +stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving +edit reverted inside one second is invisible to CPython's staleness test. That +is TEST-15, and it has already turned one live mutation into a false survivor. +Read MUTATION.md before you report a survivor. """ import ast diff --git a/goodwe_controller/test_static_tariff.py b/goodwe_controller/test_static_tariff.py index 145b81e..f36a185 100644 --- a/goodwe_controller/test_static_tariff.py +++ b/goodwe_controller/test_static_tariff.py @@ -45,6 +45,13 @@ THREE RULES THIS FILE HOLDS ITSELF TO, because the reviewer will: price, and zero is exactly the value that makes everything look free. So `static_plan` with no profile is checked to produce a read with NO `.points`, and `price_at` on it must RAISE rather than answer 0.0. + +⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run +`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it +stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving +edit reverted inside one second is invisible to CPython's staleness test. That +is TEST-15, and it has already turned one live mutation into a false survivor. +Read MUTATION.md before you report a survivor. """ import gzip diff --git a/goodwe_controller/test_tariff.py b/goodwe_controller/test_tariff.py index 636761a..966186a 100644 --- a/goodwe_controller/test_tariff.py +++ b/goodwe_controller/test_tariff.py @@ -85,6 +85,13 @@ regression fixture that CI cannot see is not a regression fixture. The recorder's window rolls: 2026-08-01 leaves it around 2026-09-15, which is why the committed copy is the artefact and not the API call. + +⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run +`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it +stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving +edit reverted inside one second is invisible to CPython's staleness test. That +is TEST-15, and it has already turned one live mutation into a false survivor. +Read MUTATION.md before you report a survivor. """ import asyncio diff --git a/goodwe_controller/test_validate.py b/goodwe_controller/test_validate.py index 4645058..0a258b6 100644 --- a/goodwe_controller/test_validate.py +++ b/goodwe_controller/test_validate.py @@ -5,6 +5,13 @@ three refusable configurations are refused. The second asserts that everything on validate.py's "deliberately not refused" list still STARTS - because the way a validator dies is not by missing a fault, it is by refusing something harmless until somebody deletes the call. + +⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run +`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it +stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving +edit reverted inside one second is invisible to CPython's staleness test. That +is TEST-15, and it has already turned one live mutation into a false survivor. +Read MUTATION.md before you report a survivor. """ import asyncio diff --git a/goodwe_controller/test_watchdog.py b/goodwe_controller/test_watchdog.py index 9d78e69..af422e9 100644 --- a/goodwe_controller/test_watchdog.py +++ b/goodwe_controller/test_watchdog.py @@ -8,6 +8,13 @@ person switches it off. The headline validation is not here - it is `sim/replay_sign.py` in the infra repo, which drives this module over the two captured bus days. This file is the table-driven half: boundaries, timers, the latch, and the arming gate. + +⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run +`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it +stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving +edit reverted inside one second is invisible to CPython's staleness test. That +is TEST-15, and it has already turned one live mutation into a false survivor. +Read MUTATION.md before you report a survivor. """ import sys diff --git a/goodwe_controller/tools/mutation_run.sh b/goodwe_controller/tools/mutation_run.sh new file mode 100755 index 0000000..37ce243 --- /dev/null +++ b/goodwe_controller/tools/mutation_run.sh @@ -0,0 +1,95 @@ +#!/bin/sh +# The house mutation harness (TEST-15). `tools/mutation_run.sh [TREE] [test_*.py ...]` +# +# WHY THIS EXISTS. Nearly every safety claim in this repository was verified by +# mutation: change the shipped module, run the suite, watch it go red, revert, +# watch it go green. FCST-03's developer found that the middle two steps can be +# a lie. CPython decides a `.pyc` is still valid from (source size, source mtime +# TRUNCATED TO WHOLE SECONDS). Revert `self.provisional` to `here.provisional` +# and neither term moves - same length, same second - so the interpreter loads +# the bytecode it already has. Confirmed by disassembly: the source read `here`, +# the running code loaded `self`. The mutation "survived" because it never ran. +# +# ⚠️ `python3 -B` IS NOT THE FIX, AND NEITHER IS PYTHONDONTWRITEBYTECODE=1. +# Both stop bytecode being WRITTEN. Neither stops a stale `.pyc` that is already +# on disk from being READ - measured, and graded by test_bytecode.py part 2. +# The load-bearing half is the PURGE. `-B` is here so the run leaves no cache +# behind for the NEXT mutation to inherit. +# +# ⚠️ NOT A SLEEP. One-second mtime resolution is the mechanism, not the enemy; +# the answer is to stop trusting mtime, not to make every edit wait a second. +# A sleep also only covers the edit you remembered to slow down. +# +# It prints a fingerprint of the tree it is about to grade, because the other +# half of a mutation report is WHICH SOURCE produced it: a report that cannot +# say what was on disk is the same unverifiable claim in a different dress. +# Diff two fingerprints and the mutated file is the line that changed. +set -eu + +TREE="" +case "${1-}" in + "" ) : ;; + -h|--help ) sed -n '2,30p' "$0"; exit 0 ;; + * ) if [ -d "$1" ]; then TREE=$1; shift; fi ;; +esac +if [ -z "$TREE" ]; then + # Default: the add-on directory this script ships in, so `tools/mutation_run.sh` + # works from anywhere. An explicit TREE is what lets the infra repo's replay + # harnesses purge $GOODWE_ADDON_REPO before they grade it - see MUTATION.md. + TREE=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd) +fi +PY=${PY:-python3} + +cd "$TREE" + +# CI-01's rule, for the same reason: a loop over an empty glob exits 0, and a +# green run that examined nothing is worse than a red one. +if [ $# -eq 0 ]; then set -- test_*.py; fi +if [ ! -f "$1" ]; then + echo "mutation_run: no test_*.py under $TREE - NOTHING has been graded." >&2 + exit 1 +fi + +echo "== purge" +find . -name '__pycache__' -type d -prune -exec rm -rf {} + 2>/dev/null || true +find . -name '*.pyc' -type f -exec rm -f {} + 2>/dev/null || true +left=$(find . -name '__pycache__' -type d -o -name '*.pyc' -type f | head -5) +if [ -n "$left" ]; then + # A read-only tree cannot hold a cache written during this run either, so this + # is only reachable when the purge itself failed on a tree that HAS one - the + # one state where continuing would grade unknown bytecode. + echo "mutation_run: could not purge, and a cache remains:" >&2 + echo "$left" >&2 + exit 1 +fi +echo " no bytecode cache under $TREE" + +echo "== tree graded (sha256, bytes)" +find app -name '*.py' 2>/dev/null | sort | while read -r f; do + printf ' %s %8d %s\n' "$(sha256sum "$f" | cut -c1-16)" "$(wc -c < "$f")" "$f" +done + +echo "== run ($PY -B)" +rc=0 +for f in "$@"; do + start=$(date +%s) + if "$PY" -B "$f" > /tmp/mutation_run.out 2>&1; then + echo " PASS $f ($(( $(date +%s) - start ))s)" + else + echo " FAIL $f ($(( $(date +%s) - start ))s)" + tail -40 /tmp/mutation_run.out + rc=1 + fi +done + +# -B was asked for; check it was honoured rather than assuming. A cache here +# would be inherited by the next mutation in the batch, which is exactly the +# state this script exists to prevent. +after=$(find . -name '__pycache__' -type d -o -name '*.pyc' -type f | head -5) +if [ -n "$after" ]; then + echo "mutation_run: a bytecode cache appeared DURING the run - the next" >&2 + echo "mutation in this batch would inherit it:" >&2 + echo "$after" >&2 + rc=1 +fi +exit $rc