TEST-15: purge the bytecode cache before a mutation run, not after #27
@@ -101,10 +101,22 @@ jobs:
|
||||
# hwsim section was skipped - so "test_freeze.py: PASS" below is a weaker
|
||||
# claim than it looks. That belongs to whoever owns test_freeze.py next;
|
||||
# naming it here is the least this gate can do about it.
|
||||
# ⚠️ PYTHONDONTWRITEBYTECODE=1 IS BELT AND BRACES, NOT A FIX FOR A LIVE
|
||||
# HOLE. TEST-15 measured this runner's actual exposure and found none: the
|
||||
# checkout below is bind mounted `:ro`, so the interpreter physically
|
||||
# cannot write a __pycache__ into the tree it is grading - it fails
|
||||
# SILENTLY, no error and exit status 0 - and every job `rm -rf`s $WS and
|
||||
# clones afresh, so nothing survives from a previous job on this shared
|
||||
# runner either. The variable is here because the `:ro` is the load-bearing
|
||||
# part and it is one character wide: delete it while debugging a write and
|
||||
# the gate quietly starts caching bytecode across a rebuild again. Note
|
||||
# what it does NOT do, per TEST-15 part 2 - it stops bytecode being
|
||||
# WRITTEN, never a stale one being READ. It is not a substitute for the
|
||||
# purge in goodwe_controller/tools/mutation_run.sh.
|
||||
- name: the add-on's own tests
|
||||
run: |
|
||||
set -eu
|
||||
docker run --rm -v "$WS:/w:ro" goodwe-addon-checks:1 sh -eu -c '
|
||||
docker run --rm -e PYTHONDONTWRITEBYTECODE=1 -v "$WS:/w:ro" goodwe-addon-checks:1 sh -eu -c '
|
||||
cd /w/goodwe-addon/goodwe_controller
|
||||
set -- test_*.py
|
||||
if [ "$1" = "test_*.py" ]; then
|
||||
|
||||
+13
@@ -57,6 +57,14 @@ python3 test_control.py && python3 test_maintenance.py && python3 test_arbiter.p
|
||||
&& python3 test_loop.py
|
||||
```
|
||||
|
||||
⚠️ **Mutating a module to prove a check bites? Read `goodwe_controller/
|
||||
MUTATION.md` first, and run `goodwe_controller/tools/mutation_run.sh`.** A
|
||||
same-size edit reverted inside one second changes neither the file size nor the
|
||||
whole-second mtime, so CPython reuses the bytecode it already compiled and the
|
||||
suite grades code that is not there - once already, reported as a survivor and
|
||||
disproved by disassembly (TEST-15). `python3 -B` does **not** save you: it stops
|
||||
bytecode being written, not a stale `.pyc` being read. The purge is the fix.
|
||||
|
||||
`test_watchdog.py` covers SAFETY-02's logic, its arming gate (derived from the
|
||||
real arbiter, not passed in), its trip path, and the fact that the latch survives
|
||||
an add-on restart. Its headline validation is not here at all: `sim/replay_sign.py`
|
||||
@@ -165,6 +173,11 @@ Each of these was found the expensive way. They are in the code comments too.
|
||||
`timestamp` attribute.
|
||||
- **Alpine ships paho-mqtt 1.x**, which has no `CallbackAPIVersion`, and musl has
|
||||
no aiohttp wheel — install from apk, not pip.
|
||||
- **A mutation run can grade code that is not there.** CPython validates a
|
||||
`.pyc` on (source size, source mtime in whole seconds), so a same-size edit
|
||||
reverted within one second is invisible to it and the old bytecode runs. It
|
||||
manufactures *survivors*, never reds. Purge `__pycache__` -
|
||||
`tools/mutation_run.sh` does - and see `goodwe_controller/MUTATION.md`.
|
||||
- **The inverter holds its last command forever.** It has no meter-timeout. Every
|
||||
failsafe here exists because of that one fact, and anything in doubt commands
|
||||
**0 W** rather than holding.
|
||||
|
||||
@@ -19,6 +19,8 @@ goodwe_controller/ the add-on
|
||||
test_control.py runnable checks — no framework needed
|
||||
test_maintenance.py runnable checks — walks a full cycle in fake time
|
||||
DOCS.md the add-on's documentation tab
|
||||
MUTATION.md how to run a mutation so the result means something
|
||||
tools/mutation_run.sh the mutation harness - purges bytecode, then grades
|
||||
firmware/goodwe-master.yaml ESPHome config for the T-CAN485
|
||||
estop/rs485_log.py optional RS485 e-stop / bus witness
|
||||
docs/GoodWe-RS485-Field-Guide.pdf the guide techs carry (10pp, A4)
|
||||
@@ -82,7 +84,12 @@ python3 test_control.py
|
||||
python3 test_maintenance.py
|
||||
```
|
||||
|
||||
Both must pass before shipping any change. They are not unit-test theatre —
|
||||
Both must pass before shipping any change. To prove a check *bites* - the way
|
||||
almost every claim in this repository was established - mutate the module and
|
||||
run `tools/mutation_run.sh`, which clears `__pycache__` first. It is not
|
||||
optional: a same-size edit reverted inside one second leaves CPython reusing the
|
||||
old bytecode, and the run grades code that is not there. `goodwe_controller/MUTATION.md`
|
||||
has the mechanism, the measurement, and what the flags do not do. They are not unit-test theatre —
|
||||
each assertion corresponds to a rule whose absence produced an observed failure
|
||||
on real hardware, and the comments in `control.py` say which.
|
||||
|
||||
|
||||
@@ -2,6 +2,62 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
**A mutation run could grade code that was not there, and the fix is a purge
|
||||
rather than a flag (TEST-15).** FCST-03's developer reverted `self.provisional`
|
||||
to `here.provisional`, re-ran, and reported a survivor; disassembly showed the
|
||||
source read `here` while the running code loaded `self`. CPython validates a
|
||||
`.pyc` on (source size, source mtime truncated to whole seconds), and a
|
||||
size-preserving edit reverted inside one second moves neither number, so the
|
||||
stale bytecode was reused. That does not make past results wrong - it made them
|
||||
unverified, and the distinction is the whole card.
|
||||
|
||||
Reproduced rather than reasoned about. `test_bytecode.py` builds a real package
|
||||
whose two return values are the same length, imports it in a real subprocess,
|
||||
rewrites it inside the same second and reads back which value ran. Part 1 shows
|
||||
the interpreter running `SELF` off a file that says `HERE`.
|
||||
|
||||
**And it shows the two obvious mitigations do not work.** `python3 -B` and
|
||||
`PYTHONDONTWRITEBYTECODE=1` are write-side only: neither stops an EXISTING stale
|
||||
`.pyc` being read, and a mutation run always has one because the previous run
|
||||
wrote it. The advice already in `test_control.py`'s coverage audit led with `-B`
|
||||
and is corrected here - the purge is the half that works, and `-B` earns its
|
||||
place by protecting the NEXT edit rather than this one. No `sleep` anywhere:
|
||||
one-second mtime resolution is the mechanism, and waiting it out would slow every
|
||||
run down while still only covering the edit somebody remembered to slow.
|
||||
|
||||
`goodwe_controller/tools/mutation_run.sh` is the harness the house adopts: it
|
||||
purges, prints a sha256/byte fingerprint of every `app/*.py` it is about to
|
||||
grade - so a mutation report can say WHICH source produced it - runs each suite
|
||||
under `python3 -B`, refuses a tree with no `test_*.py` (CI-01's rule), and fails
|
||||
if a cache appears during the run. It takes the tree as an argument so the infra
|
||||
repo's replay harnesses can call it.
|
||||
|
||||
`MUTATION.md` states the guarantee where a mutation gets written, and every one
|
||||
of the 16 suites now points at it from its docstring - a list `test_bytecode.py`
|
||||
derives from the tree, so a new suite joining without the pointer is red.
|
||||
|
||||
CI was already immune and is described that way rather than as a hole being
|
||||
closed: `checks.yml` mounts the checkout `-v "$WS:/w:ro"`, and a read-only tree
|
||||
cannot take a `__pycache__` at all - measured, and it fails silently, exit
|
||||
status 0 - while each job `rm -rf`s its workspace and clones afresh.
|
||||
`PYTHONDONTWRITEBYTECODE=1` is added as belt and braces, because the `:ro` is one
|
||||
character wide. The shipped add-on was never affected: `Dockerfile` has set
|
||||
`PYTHONDONTWRITEBYTECODE=1` since it was written.
|
||||
|
||||
**The survey of this week's mutation claims is in `MUTATION.md`, scoped by the
|
||||
property rather than by re-running everything.** The defect is one-directional -
|
||||
stale bytecode turns a red into a green survivor and never the reverse - so every
|
||||
"all red, no survivors" result in the window is not at risk, and the exposure is
|
||||
the recorded SURVIVORS from source-edit runs made before the purge discipline
|
||||
landed on 2026-08-31. Two of them were re-run here under the harness and both
|
||||
still stand (TARIFF-01's `record_block()` ignoring its `policy` argument;
|
||||
SAFETY-03's `abs()` on the clamp limit). The rest are labelled still-unverified
|
||||
rather than quietly assumed or noisily re-run.
|
||||
|
||||
Out of scope and recorded: the infra repo's `sim/` replay harnesses import the
|
||||
add-on and write bytecode into both trees. Anyone mutating a module graded by a
|
||||
replay must purge both by hand until that repo adopts the harness.
|
||||
|
||||
**`release/1.0` went red at midnight on a commit nobody had touched (TEST-17).**
|
||||
`Controller.checks()` read the capacity-tariff peak with `self.peak.peak()` - no
|
||||
argument - so `tariff.py` resolved the month from `datetime.now()`, while
|
||||
|
||||
@@ -0,0 +1,153 @@
|
||||
# Mutation runs in this repository — and the one way they lie
|
||||
|
||||
Nearly every safety claim in this add-on was established the same way: change
|
||||
the shipped module, run the suites, watch one go **red**, put the module back,
|
||||
watch it go **green**. There is no coverage tool here and there is not meant to
|
||||
be — the check that matters is whether a *check* notices when the behaviour it
|
||||
names is removed.
|
||||
|
||||
That method has exactly one silent failure mode, and TEST-15 exists because we
|
||||
walked into it.
|
||||
|
||||
## The defect
|
||||
|
||||
FCST-03's developer reverted `self.provisional` to `here.provisional`, re-ran,
|
||||
and reported the mutation as a **survivor**. It was not. CPython decides a
|
||||
cached `.pyc` is still valid from two numbers:
|
||||
|
||||
* the source file's **size in bytes**, and
|
||||
* the source file's **mtime truncated to whole seconds**.
|
||||
|
||||
`self` and `here` are the same length, and the revert happened in the same
|
||||
second as the edit. Neither number moved, so the interpreter loaded the
|
||||
bytecode it already had. Confirmed at the time by disassembly: **the source read
|
||||
`here`, the running code loaded `self`.** The suite had graded code that was not
|
||||
there — no error, no warning, exit status 0.
|
||||
|
||||
The same collision, in the other direction, can leave a **red** standing on
|
||||
correct source after a revert that never took.
|
||||
|
||||
### Which way it lies, and why that scopes everything
|
||||
|
||||
A mutation that never took effect leaves the suite **green**, and a green run of
|
||||
a mutated tree is reported as a survivor. So stale bytecode **manufactures
|
||||
survivors**; it cannot manufacture a red. The false-red case exists but
|
||||
announces itself, because a developer chases a failure they cannot explain.
|
||||
|
||||
Two consequences, and both are used below:
|
||||
|
||||
* a recorded **"all red, no survivors"** result is not at risk from this defect;
|
||||
* a recorded **survivor** — and a recorded *zero*, such as a coverage-audit
|
||||
count of 0 — is exactly what this defect produces.
|
||||
|
||||
`test_control.py`'s coverage-audit table says the same thing in its own terms:
|
||||
the error is one-directional, so every figure there is a lower bound at worst.
|
||||
|
||||
## The rule
|
||||
|
||||
**Purge `__pycache__` before every mutation run.** Not afterwards, not usually,
|
||||
not when the edit looks big enough.
|
||||
|
||||
```sh
|
||||
cd goodwe_controller
|
||||
tools/mutation_run.sh # every test_*.py in this directory
|
||||
tools/mutation_run.sh . test_tariff.py # or just the ones you are watching
|
||||
tools/mutation_run.sh "$GOODWE_ADDON_REPO/goodwe_controller" # from elsewhere
|
||||
```
|
||||
|
||||
The harness purges, prints a `sha256`/byte-count fingerprint of every `app/*.py`
|
||||
it is about to grade, runs each suite under `python3 -B`, and fails if a cache
|
||||
appeared during the run. Diff two fingerprints and the file you mutated is the
|
||||
line that changed — which is also how a mutation report proves *which source*
|
||||
produced it.
|
||||
|
||||
### What does **not** fix this
|
||||
|
||||
| | writes bytecode | reads a stale `.pyc` |
|
||||
|---|---|---|
|
||||
| `python3 -B` | no | **yes** |
|
||||
| `PYTHONDONTWRITEBYTECODE=1` | no | **yes** |
|
||||
| purging `__pycache__` | (unchanged) | **no — nothing to read** |
|
||||
|
||||
Both flags are *write-side only*. A mutation run always has a cache already,
|
||||
because the previous run wrote it, so a harness that only adds `-B` closes
|
||||
nothing. This is measured, not read off documentation: `test_bytecode.py`
|
||||
part 2 arranges the collision and shows both flags still running the old code.
|
||||
|
||||
`-B` is still worth passing — it stops **this** run leaving a cache for the
|
||||
**next** edit to inherit, which is the one nobody is thinking about three
|
||||
mutations into a batch. Purge for this run, `-B` for the next one.
|
||||
|
||||
### And not a `sleep`
|
||||
|
||||
One-second mtime resolution is the *mechanism*, not the enemy. Waiting out the
|
||||
clock would slow every run down, and would still only cover the edit somebody
|
||||
remembered to slow down. Stop trusting mtime instead.
|
||||
|
||||
## Reporting a mutation
|
||||
|
||||
State the things a reviewer cannot reconstruct afterwards:
|
||||
|
||||
1. **the byte delta** of the mutated file (`+5 B`, `+0 B`). `+0 B` is the risky
|
||||
one — say so;
|
||||
2. **that the mutated line was on disk** when the run started (the harness
|
||||
fingerprint, or a `grep` of the line);
|
||||
3. **which suite went red, and on which named check** — "the suite is red" and
|
||||
"the check that watches this is red" are different claims;
|
||||
4. for a **survivor**, that the run was purged. An unpurged survivor is not a
|
||||
result.
|
||||
|
||||
## Where each surface stands
|
||||
|
||||
* **Local and dev-harness runs — the exposed one.** This is where FCST-03 hit
|
||||
it. `goodwe_controller/__pycache__` and `app/__pycache__` are gitignored and
|
||||
are sitting in most working trees right now. Use the harness.
|
||||
* **CI — structurally immune, and it was before this card.** `.gitea/workflows/
|
||||
checks.yml` bind mounts the checkout `-v "$WS:/w:ro"`, so the interpreter
|
||||
cannot write a `__pycache__` into the tree under test at all; measured for
|
||||
TEST-15, and it fails *silently* — no error, exit status 0. Each job also
|
||||
`rm -rf`s its workspace and clones afresh, so nothing crosses between jobs on
|
||||
the shared runner. `PYTHONDONTWRITEBYTECODE=1` is now set there too, as belt
|
||||
and braces: the `:ro` is one character wide and the next person debugging a
|
||||
write will be tempted by it.
|
||||
* **The shipped add-on — not affected, and this is closed.**
|
||||
`goodwe_controller/Dockerfile` has carried `PYTHONDONTWRITEBYTECODE=1` since
|
||||
it was written. There is no deploy-side finding here; do not re-open it.
|
||||
* **The infra repo's `sim/` harnesses — exposed, and NOT fixed by this card.**
|
||||
`sim/replay_sign.py` and friends import the add-on out of
|
||||
`$GOODWE_ADDON_REPO` and write `__pycache__` into *both* trees
|
||||
(`goodwe/sim/__pycache__` exists today). SAFETY-02's headline validation is a
|
||||
replay, not a suite, so a mutation graded through it carries the same
|
||||
exposure. `tools/mutation_run.sh` takes the tree to purge as its first
|
||||
argument precisely so that repo can call it, but the change belongs in that
|
||||
repo and some of those harnesses want the rig. **Anyone mutating a module
|
||||
graded by a replay must purge both trees by hand until then.**
|
||||
|
||||
## Survey — mutation claims of 2026-08-25 … 2026-09-01
|
||||
|
||||
Scoped **by the property, not by re-running everything**. A claim is at risk
|
||||
only if a mutation could have preserved the file's byte count *and* been
|
||||
reverted inside the same second, *and* the claim is of the kind stale bytecode
|
||||
can fabricate (see "which way it lies"). Sources: `git log release/1.0` and the
|
||||
commit and PR bodies.
|
||||
|
||||
| claim | verdict | why |
|
||||
|---|---|---|
|
||||
| **FCST-03** M9 `self.provisional`→`here.provisional` (dd40c56, 71cd784) | **re-verified** | this is the original false survivor. Re-run with the cache cleared, M9 is caught. The shipped source was correct throughout. |
|
||||
| **FCST-03** X1, X6 degenerate stimuli (71cd784) | **not at risk** | run after the purge discipline landed in that branch. |
|
||||
| **SAFETY-24** 11 author + 9 reviewer mutations (e709554, 4601682) | **not at risk** | the commit records them run under `python -B` with a purged `__pycache__` and real byte deltas. Its one survivor is an equivalent mutant, argued structurally rather than by a green run. |
|
||||
| **SAFETY-26** tolerance sweep over `arbitrage.py` and `static_tariff.py`, 2 survivors (2b8b8cb) | **not at risk** | purge, printed `+5 B` delta, mutated line asserted present, tree re-read before each run. One survivor is explicitly recorded at `+0 B` — the risky delta — and was produced *under* the purge. |
|
||||
| **OPT-04** mutations reported on PR #22 (bdd7f18) | **not at risk** | `test_arbitrage.py`'s header records the exact command: purge, then `python3 -B`. |
|
||||
| **TEST-17** `checks()` dropping `when` (504e8da) | **not at risk** | a *red* claim, and dropping an argument changes the file length. Neither half of the collision is available. |
|
||||
| **SAFETY-07** wiring mutations B1/B2/B3 and `test_peakguard.py` parts 7 and 10 (2d570d8, e7c95bf) | **not at risk** | those mutations are **executed in-process**, monkeypatched onto the imported module. No source file is edited, so no `.pyc` is consulted. |
|
||||
| every **"all red, no survivors"** claim in the window — TARIFF-02 (24/0), TARIFF-03 (11), MAINT-04 (10), SAFETY-18 (6+4), SAFETY-03 (6), TEL-04 (10), TEL-05 (6) | **not at risk** | direction. Stale bytecode turns a red into a green survivor, never the reverse. A red is evidence that *something* changed under the check. |
|
||||
| **TEST-06**'s deleted assertion, "no new notification while parked at 0 W" (c34b5a2) | **not at risk / superseded** | it was recorded as undecidable and then *falsified* — `test_freeze.py` records the one-line mutation that takes the file red 9 times. The stronger later claim is a red one. |
|
||||
| **TARIFF-01** reviewer survivor: `record_block()` ignoring its `policy` argument (0dbd485) | **re-verified — still a survivor** | re-run here through `tools/mutation_run.sh` against the 15 suites that were on the record, all green with `billed_w(block, policy)` mutated to `billed_w(block, POLICY_NET)`. The recorded diagnosis holds: every block reaching the store is single-phase, where NET, SUM and MAX collapse. |
|
||||
| **SAFETY-03** reviewer survivor: `abs()` dropped from `clamp_to_rating`'s limit (6aabb83) | **re-verified — still a survivor** | re-run the same way, all 15 green. Also as recorded: unreachable under the `int(100, 5000)` schema. |
|
||||
| **TARIFF-01**'s other three survivors, **SAFETY-03**'s `describe()` survivor, **TARIFF-02**'s M3/M5/M7, **SAFETY-18**'s three | **still unverified** | recorded before the purge discipline, with no byte delta on the record, so a `+0 B` same-second revert cannot be excluded. All are survivors, which is the direction this defect fabricates. TARIFF-02's and SAFETY-18's were **closed by later commits whose claims are reds**, so what is unverified is the *diagnosis*, not the current state of the code. |
|
||||
|
||||
**Nothing here says a past result is wrong.** The two survivors re-run above
|
||||
both stood. What changed is that they are now on the record as measured rather
|
||||
than assumed, and the rest are labelled honestly. The still-unverified rows are
|
||||
cheap to close — one harness run each — and the right moment is when somebody
|
||||
next touches that module, not in a re-run campaign nobody will read.
|
||||
@@ -3,6 +3,13 @@
|
||||
Each assertion is a precedence question someone will eventually ask in the
|
||||
field: "why did it charge during a peak?", "why did it discharge while the
|
||||
maintenance cycle was charging?", "what happens if two things disagree?".
|
||||
|
||||
⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run
|
||||
`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it
|
||||
stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving
|
||||
edit reverted inside one second is invisible to CPython's staleness test. That
|
||||
is TEST-15, and it has already turned one live mutation into a false survivor.
|
||||
Read MUTATION.md before you report a survivor.
|
||||
"""
|
||||
|
||||
import sys
|
||||
|
||||
@@ -62,6 +62,13 @@ THREE RULES THIS FILE HOLDS ITSELF TO, because the reviewer will:
|
||||
aged out, an hour the series does not cover, an unread state of charge and a
|
||||
peak guard that never ran are five DIFFERENT reasons - and none of them is
|
||||
an answer. Section 5 drives all five and asserts they are distinguishable.
|
||||
|
||||
⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run
|
||||
`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it
|
||||
stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving
|
||||
edit reverted inside one second is invisible to CPython's staleness test. That
|
||||
is TEST-15, and it has already turned one live mutation into a false survivor.
|
||||
Read MUTATION.md before you report a survivor.
|
||||
"""
|
||||
|
||||
import ast
|
||||
|
||||
@@ -0,0 +1,296 @@
|
||||
"""Runnable check for TEST-15 - the mutation harness's own honesty.
|
||||
`python3 test_bytecode.py`
|
||||
|
||||
Every other file in this directory grades the add-on. This one grades the thing
|
||||
that grades the add-on, because in this repository a mutation run IS the
|
||||
evidence: "I broke it, the suite went red, I put it back, the suite went green"
|
||||
is how nearly every safety claim here was established.
|
||||
|
||||
THE DEFECT, found by FCST-03's developer and reproduced below. CPython decides a
|
||||
cached `.pyc` is still good from two numbers: the source's SIZE, and its
|
||||
MTIME TRUNCATED TO WHOLE SECONDS. Reverting `self.provisional` to
|
||||
`here.provisional` moves neither - same length, same second - so the interpreter
|
||||
loaded the bytecode it already had. Confirmed then by disassembly: the source
|
||||
read `here`, the running code loaded `self`. The mutation was reported as a
|
||||
survivor when in truth it had never run.
|
||||
|
||||
WHICH DIRECTION IT LIES IN. A mutation that never took effect makes the suite
|
||||
stay GREEN, so it is reported as a survivor: stale bytecode MANUFACTURES
|
||||
SURVIVORS. The opposite - a revert that never took, leaving a red on correct
|
||||
source - is possible too but announces itself, because a developer chases an
|
||||
unexplained failure. That asymmetry is why the coverage-audit table in
|
||||
test_control.py is a lower bound rather than nonsense, and it is what the survey
|
||||
in MUTATION.md is scoped by.
|
||||
|
||||
WHAT THIS FILE PINS, in the order the card's acceptance criteria name it:
|
||||
|
||||
AC1 The defect reproduces (part 1), the two mitigations that DO NOT fix it
|
||||
are shown not to (part 2), the one that does is shown to (part 3), and
|
||||
the shipped harness is shown to close it end to end on a tree that would
|
||||
otherwise mis-grade (part 4). Executed, not reasoned: every part builds a
|
||||
real package, imports it in a real subprocess, and reads back which
|
||||
value the running code returned.
|
||||
AC2 The guarantee is where a developer writing a mutation reads it: part 5
|
||||
requires EVERY test_*.py in this directory to point at MUTATION.md. The
|
||||
list is derived from the tree, so a new suite that joins without the
|
||||
pointer is red - CI-01's rule, for CI-01's reason.
|
||||
AC4 Part 6 pins the two properties that make CI immune, on the workflow file
|
||||
itself.
|
||||
|
||||
⚠️ NOTHING HERE SLEEPS. One-second mtime resolution is the mechanism; a harness
|
||||
that waits out the clock would be slower on every run and would still only cover
|
||||
the edit somebody remembered to slow down. Part 1 REPRODUCES the collision
|
||||
deliberately (it retries until two writes land in the same second, which takes
|
||||
milliseconds) rather than avoiding it.
|
||||
|
||||
⚠️ This file writes only under a temporary directory. It must stay that way: CI
|
||||
mounts the checkout READ-ONLY, and a check that needs to write into the tree
|
||||
would fail there for a reason that has nothing to do with what it grades.
|
||||
|
||||
⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or use
|
||||
`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it
|
||||
stops bytecode being WRITTEN, not a stale one being READ. See MUTATION.md.
|
||||
"""
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
fails = []
|
||||
|
||||
|
||||
def check(name, cond):
|
||||
if cond:
|
||||
print(f" ok {name}")
|
||||
else:
|
||||
print(f" FAIL {name}")
|
||||
fails.append(name)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The fixture. `who()` returns one of two names that are THE SAME LENGTH, so
|
||||
# swapping them leaves the file byte count identical - the first half of
|
||||
# CPython's staleness test. The second half is the whole-second mtime, which
|
||||
# part 1 arranges by writing both versions inside one second.
|
||||
#
|
||||
# It is deliberately not a copy of a shipped module: the property under test is
|
||||
# CPython's, and using real code here would make this file go red whenever that
|
||||
# module changed for unrelated reasons.
|
||||
# ---------------------------------------------------------------------------
|
||||
V_OLD = 'def who():\n self = "SELF"\n here = "HERE"\n return self\n'
|
||||
V_NEW = 'def who():\n self = "SELF"\n here = "HERE"\n return here\n'
|
||||
assert len(V_OLD) == len(V_NEW), "the fixture must be size-preserving or it grades nothing"
|
||||
|
||||
PROBE = "from app.m import who\nprint(who())\n"
|
||||
# The harness runs `test_*.py` files and grades them by exit status, so the
|
||||
# probe wears that shape in part 4: it FAILS when it sees the pre-mutation value.
|
||||
TEST_PROBE = (
|
||||
"from app.m import who\n"
|
||||
"v = who()\n"
|
||||
"print('RUNS:', v)\n"
|
||||
"raise SystemExit(0 if v == 'HERE' else 1)\n"
|
||||
)
|
||||
|
||||
|
||||
def run(tree, args, env_extra=None, script="probe.py"):
|
||||
env = dict(os.environ)
|
||||
env.pop("PYTHONDONTWRITEBYTECODE", None)
|
||||
if env_extra:
|
||||
env.update(env_extra)
|
||||
out = subprocess.run(
|
||||
[sys.executable] + args + [script],
|
||||
cwd=tree, env=env, capture_output=True, text=True,
|
||||
)
|
||||
return out.stdout.strip()
|
||||
|
||||
|
||||
def caches(tree):
|
||||
return sorted(str(p.relative_to(tree)) for p in Path(tree).rglob("__pycache__"))
|
||||
|
||||
|
||||
def seed(tree):
|
||||
"""Leave `tree` holding V_NEW on disk and V_OLD in its bytecode cache.
|
||||
|
||||
Returns True when the collision was actually arranged. Both writes must land
|
||||
in the same whole second; at a few milliseconds per cycle that is the normal
|
||||
case, and the caller retries rather than asserting on one attempt, because a
|
||||
machine that happens to cross a second boundary mid-cycle would otherwise
|
||||
report a defect in CPython instead of a slow disk.
|
||||
"""
|
||||
app = Path(tree) / "app"
|
||||
shutil.rmtree(app, ignore_errors=True)
|
||||
app.mkdir(parents=True)
|
||||
(app / "__init__.py").write_text("")
|
||||
(Path(tree) / "probe.py").write_text(PROBE)
|
||||
(Path(tree) / "test_probe.py").write_text(TEST_PROBE)
|
||||
|
||||
m = app / "m.py"
|
||||
m.write_text(V_OLD)
|
||||
t0 = m.stat().st_mtime
|
||||
run(tree, []) # compiles V_OLD, writes app/__pycache__
|
||||
m.write_text(V_NEW) # the "revert": same size, same second
|
||||
t1 = m.stat().st_mtime
|
||||
return int(t0) == int(t1) and caches(tree) != []
|
||||
|
||||
|
||||
def seeded_tree(stack):
|
||||
tree = tempfile.mkdtemp(prefix="test_bytecode.")
|
||||
stack.append(tree)
|
||||
for _ in range(40):
|
||||
if seed(tree):
|
||||
return tree
|
||||
return None
|
||||
|
||||
|
||||
tmpdirs = []
|
||||
print("stale bytecode (TEST-15)")
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
print("\n1. the defect: a same-size edit inside one second is not seen at all")
|
||||
# ---------------------------------------------------------------------------
|
||||
tree = seeded_tree(tmpdirs)
|
||||
check("the collision can be arranged at all (same size, same second)", tree is not None)
|
||||
if tree is None:
|
||||
print(" cannot continue - the fixture never landed two writes in one second")
|
||||
raise SystemExit(1)
|
||||
|
||||
src = (Path(tree) / "app" / "m.py").read_text()
|
||||
check("the source on disk says HERE", "return here" in src and "return self" not in src)
|
||||
check("...and a plain interpreter runs SELF anyway", run(tree, []) == "SELF")
|
||||
# Stated as its own check because this is the whole ticket in one line: what the
|
||||
# file says and what the process executes have come apart, with no error, no
|
||||
# warning, and a zero exit status.
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
print("\n2. the two mitigations that do NOT close it")
|
||||
# ---------------------------------------------------------------------------
|
||||
# ⚠️ READ THIS BEFORE 'FIXING' A HARNESS WITH -B ALONE. Both of these stop
|
||||
# bytecode being WRITTEN. Neither stops a stale one being READ, and a mutation
|
||||
# run always has one already: the previous run wrote it. The advice in
|
||||
# test_control.py's coverage audit said `-B` first and CLEAR the cache second;
|
||||
# the order is not a style choice, the clear is the half that works.
|
||||
check("`python3 -B` over an existing cache still runs SELF", run(tree, ["-B"]) == "SELF")
|
||||
check(
|
||||
"PYTHONDONTWRITEBYTECODE=1 over an existing cache still runs SELF",
|
||||
run(tree, [], {"PYTHONDONTWRITEBYTECODE": "1"}) == "SELF",
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
print("\n3. the purge closes it, and -B keeps the next run clean")
|
||||
# ---------------------------------------------------------------------------
|
||||
for p in Path(tree).rglob("__pycache__"):
|
||||
shutil.rmtree(p, ignore_errors=True)
|
||||
check("after the purge, the plain interpreter runs HERE", run(tree, []) == "HERE")
|
||||
check("...and it has written a fresh cache back", caches(tree) != [])
|
||||
|
||||
tree2 = seeded_tree(tmpdirs)
|
||||
check("second fixture seeded", tree2 is not None)
|
||||
if tree2:
|
||||
for p in Path(tree2).rglob("__pycache__"):
|
||||
shutil.rmtree(p, ignore_errors=True)
|
||||
check("purge + -B runs HERE", run(tree2, ["-B"]) == "HERE")
|
||||
check("...and leaves NO cache for the next mutation to inherit", caches(tree2) == [])
|
||||
# The second half is why the harness passes -B at all. Purging before a run
|
||||
# protects THIS run; not writing one protects the NEXT edit, which is the
|
||||
# one nobody is thinking about when they are three mutations into a batch.
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
print("\n4. the shipped harness, executed against a tree that would mis-grade")
|
||||
# ---------------------------------------------------------------------------
|
||||
harness = HERE / "tools" / "mutation_run.sh"
|
||||
check("tools/mutation_run.sh exists", harness.is_file())
|
||||
check("...and is executable", os.access(harness, os.X_OK))
|
||||
|
||||
tree3 = seeded_tree(tmpdirs)
|
||||
check("third fixture seeded", tree3 is not None)
|
||||
if tree3 and harness.is_file():
|
||||
# NON-VACUITY FIRST. If the naked run below were already green, the harness
|
||||
# run after it would prove nothing - it would be passing on a tree that
|
||||
# needed no help. This is the same trap test_peakguard.py part 7 documents:
|
||||
# a mutation harness that patches nothing passes beautifully.
|
||||
naked = subprocess.run(
|
||||
[sys.executable, "test_probe.py"], cwd=tree3, capture_output=True, text=True
|
||||
)
|
||||
check("without the harness the tree grades the STALE source (rc!=0)", naked.returncode != 0)
|
||||
check("...and says so: it saw SELF", "SELF" in naked.stdout)
|
||||
|
||||
got = subprocess.run(
|
||||
["sh", str(harness), tree3], cwd=tree3, capture_output=True, text=True
|
||||
)
|
||||
check("the harness grades the source on disk (rc==0)", got.returncode == 0)
|
||||
check("...having reported the purge", "no bytecode cache" in got.stdout)
|
||||
check("...and it leaves no cache behind", caches(tree3) == [])
|
||||
if got.returncode != 0:
|
||||
print(got.stdout[-2000:])
|
||||
print(got.stderr[-2000:])
|
||||
|
||||
# The harness must also refuse to report on nothing - CI-01's zero-file rule.
|
||||
empty = tempfile.mkdtemp(prefix="test_bytecode.empty.")
|
||||
tmpdirs.append(empty)
|
||||
none = subprocess.run(
|
||||
["sh", str(harness), empty], cwd=empty, capture_output=True, text=True
|
||||
)
|
||||
check("the harness refuses a tree with no test_*.py", none.returncode != 0)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
print("\n5. the guarantee is stated where a mutation gets written")
|
||||
# ---------------------------------------------------------------------------
|
||||
# ⚠️ THE LIST IS DERIVED FROM THE TREE. A hand-written one is how a new suite
|
||||
# joins this directory carrying no warning at all, which is the exact shape of
|
||||
# failure CI-01's file glob exists to prevent. If you add a test_*.py, put the
|
||||
# pointer in its docstring; that is the whole cost.
|
||||
doc = HERE / "MUTATION.md"
|
||||
check("MUTATION.md exists", doc.is_file())
|
||||
if doc.is_file():
|
||||
text = doc.read_text()
|
||||
check("...and it names the purge as the load-bearing half", "__pycache__" in text and "purge" in text.lower())
|
||||
check("...and warns that -B alone does not read fresh", "-B" in text)
|
||||
check("...and points at the harness", "mutation_run.sh" in text)
|
||||
|
||||
suites = sorted(p.name for p in HERE.glob("test_*.py"))
|
||||
check("there are test suites to survey at all", len(suites) >= 15)
|
||||
missing = [n for n in suites if "MUTATION.md" not in (HERE / n).read_text()]
|
||||
check(f"every one of the {len(suites)} suites points at MUTATION.md", missing == [])
|
||||
if missing:
|
||||
print(" missing the pointer:", ", ".join(missing))
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
print("\n6. CI cannot carry a stale artefact between jobs")
|
||||
# ---------------------------------------------------------------------------
|
||||
# ⚠️ THE FIRST TWO PROPERTIES ARE THE ONES THAT ACTUALLY DO THE WORK, AND THEY
|
||||
# PREDATE THIS CARD. Measured for TEST-15, not assumed: with the checkout bind
|
||||
# mounted READ-ONLY the interpreter writes no __pycache__ into the tree at all,
|
||||
# and it does so SILENTLY - no error, no warning, exit status 0. The workflow
|
||||
# also `rm -rf`s its workspace and clones afresh, so nothing survives from a
|
||||
# previous job even on the same runner. PYTHONDONTWRITEBYTECODE=1 is belt and
|
||||
# braces on top: it is what keeps this true if somebody ever needs to drop the
|
||||
# `:ro`, and dropping the `:ro` is a one-character edit that nothing else here
|
||||
# would notice.
|
||||
wf = HERE.parent / ".gitea" / "workflows" / "checks.yml"
|
||||
check("the workflow file is where this expects it", wf.is_file())
|
||||
if wf.is_file():
|
||||
y = wf.read_text()
|
||||
# ⚠️ GRADED ON THE COMMAND LINE, NOT ON THE FILE. A `in y` search over the
|
||||
# whole workflow passes on the PROSE around the step - this file's first
|
||||
# draft did exactly that, and stayed green after the `-e` flag and the `:ro`
|
||||
# were deleted, because the comment explaining them still named both.
|
||||
runs = [l for l in y.splitlines() if "docker run" in l and ":/w" in l]
|
||||
check("exactly one docker run mounts the checkout", len(runs) == 1)
|
||||
line = runs[0] if runs else ""
|
||||
check("the checkout is mounted read-only", ':/w:ro"' in line)
|
||||
check("PYTHONDONTWRITEBYTECODE=1 is set on that run", "-e PYTHONDONTWRITEBYTECODE=1" in line)
|
||||
check("the workspace is removed before each checkout", 'rm -rf "$WS"' in y)
|
||||
check("no sleep was introduced to dodge the mtime resolution", "sleep " not in y)
|
||||
|
||||
for d in tmpdirs:
|
||||
shutil.rmtree(d, ignore_errors=True)
|
||||
|
||||
print()
|
||||
if fails:
|
||||
print(f"{len(fails)} FAILED: " + "; ".join(fails))
|
||||
sys.exit(1)
|
||||
print("all checks passed")
|
||||
@@ -47,6 +47,13 @@ WHAT THIS CHECK DOES NOT COVER
|
||||
asserted here over every value that reached `set_number()`, which is the last
|
||||
thing this process controls; proving it at the inverter needs the witness log
|
||||
and belongs with DRIVER-07.
|
||||
|
||||
⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run
|
||||
`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it
|
||||
stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving
|
||||
edit reverted inside one second is invisible to CPython's staleness test. That
|
||||
is TEST-15, and it has already turned one live mutation into a false survivor.
|
||||
Read MUTATION.md before you report a survivor.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
@@ -6,6 +6,13 @@ its absence caused an observed failure on real hardware.
|
||||
|
||||
If you change control.py, run this. If it fails, the inverter would have done
|
||||
something you did not intend.
|
||||
|
||||
⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run
|
||||
`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it
|
||||
stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving
|
||||
edit reverted inside one second is invisible to CPython's staleness test. That
|
||||
is TEST-15, and it has already turned one live mutation into a false survivor.
|
||||
Read MUTATION.md before you report a survivor.
|
||||
"""
|
||||
|
||||
import sys
|
||||
@@ -71,14 +78,22 @@ def check(name, cond):
|
||||
# Hence: A FIXTURE MUST SIT CLEAR OF EVERY RAIL IT IS NOT TESTING. Where a test
|
||||
# names one mechanism, make that mechanism the binding one (see TCLAMP and TF).
|
||||
#
|
||||
# ⚠️ RUN MUTATIONS WITH `python -B` AND CLEAR app/__pycache__. CPython
|
||||
# invalidates a .pyc on (source mtime in whole seconds, source size), so a
|
||||
# same-second rewrite that also preserves the file size reuses stale bytecode
|
||||
# and the suite reports on code you are no longer running. It under-reported one
|
||||
# mutation here as 2 where the true figure is 6. The error is one-directional -
|
||||
# stale bytecode can only under-report - so every figure above is a lower bound
|
||||
# at worst, and the two zeros ever recorded were both confirmed by fixing them
|
||||
# and watching the count rise, which a caching artefact cannot do.
|
||||
# ⚠️ RUN MUTATIONS THROUGH `tools/mutation_run.sh`, WHICH CLEARS __pycache__
|
||||
# BEFORE IT RUNS ANYTHING. CPython invalidates a .pyc on (source mtime in whole
|
||||
# seconds, source size), so a same-second rewrite that also preserves the file
|
||||
# size reuses stale bytecode and the suite reports on code you are no longer
|
||||
# running. It under-reported one mutation here as 2 where the true figure is 6.
|
||||
#
|
||||
# ⚠️ AND THE CLEAR IS THE HALF THAT WORKS. This note used to lead with
|
||||
# `python -B`; TEST-15 measured what that flag actually does, and it is
|
||||
# write-only - it does not stop an EXISTING stale .pyc being read, and a
|
||||
# mutation run always has one, because the previous run wrote it. -B is still
|
||||
# worth passing, for the NEXT edit rather than this one.
|
||||
#
|
||||
# The error is one-directional - stale bytecode can only under-report - so
|
||||
# every figure above is a lower bound at worst, and the two zeros ever recorded
|
||||
# were both confirmed by fixing them and watching the count rise, which a
|
||||
# caching artefact cannot do.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
print("control law")
|
||||
|
||||
@@ -62,6 +62,13 @@ load the meter's POWER FIGURE never moved - 10130.0 W for 26 consecutive polls -
|
||||
and only the kWh registers kept `unchanged_s` at zero. See the note in
|
||||
freeze.py; it is why the fingerprint's register terms are the negative case
|
||||
rather than a nicety.
|
||||
|
||||
⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run
|
||||
`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it
|
||||
stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving
|
||||
edit reverted inside one second is invisible to CPython's staleness test. That
|
||||
is TEST-15, and it has already turned one live mutation into a false survivor.
|
||||
Read MUTATION.md before you report a survivor.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
@@ -19,6 +19,13 @@ HANDOVER.md. An understated runtime reads as a hang.
|
||||
`cycle_floor_s` is read from options with no lower bound in code, so a huge
|
||||
value reproduces the pre-fix behaviour exactly. Every claim below is therefore
|
||||
an A/B on one binary, not a comparison against a remembered old one.
|
||||
|
||||
⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run
|
||||
`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it
|
||||
stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving
|
||||
edit reverted inside one second is invisible to CPython's staleness test. That
|
||||
is TEST-15, and it has already turned one live mutation into a false survivor.
|
||||
Read MUTATION.md before you report a survivor.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
@@ -3,6 +3,13 @@
|
||||
Walks a whole cycle in simulated time, which is the test the YAML version never
|
||||
had - and it is exactly the phase transitions that a real cycle takes ten hours
|
||||
to exercise once a month.
|
||||
|
||||
⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run
|
||||
`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it
|
||||
stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving
|
||||
edit reverted inside one second is invisible to CPython's staleness test. That
|
||||
is TEST-15, and it has already turned one live mutation into a false survivor.
|
||||
Read MUTATION.md before you report a survivor.
|
||||
"""
|
||||
|
||||
import sys
|
||||
|
||||
@@ -8,6 +8,13 @@ Every assert here is a rule whose absence poisons something downstream: an
|
||||
inverted sign inverts the control loop, a fabricated zero hides a dead meter, a
|
||||
naive per-phase sum misbills the capacity tariff, and a plain mean of samples
|
||||
computes the wrong quarter-hour figure whenever the telegram cadence changes.
|
||||
|
||||
⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run
|
||||
`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it
|
||||
stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving
|
||||
edit reverted inside one second is invisible to CPython's staleness test. That
|
||||
is TEST-15, and it has already turned one live mutation into a false survivor.
|
||||
Read MUTATION.md before you report a survivor.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
@@ -72,6 +72,13 @@ Two days is also not thirty. AC 3's 95 %-over-30-days criterion is TARIFF-01's
|
||||
and is still not met; `test_tariff.py`'s part 9 says so at length and this file
|
||||
does not re-litigate it. What part 6 reports is the comparison that two days CAN
|
||||
support: the same quarters, with and without the guard.
|
||||
|
||||
⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run
|
||||
`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it
|
||||
stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving
|
||||
edit reverted inside one second is invisible to CPython's staleness test. That
|
||||
is TEST-15, and it has already turned one live mutation into a false survivor.
|
||||
Read MUTATION.md before you report a survivor.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
@@ -34,6 +34,13 @@ TWO RULES THIS FILE HOLDS ITSELF TO, because the reviewer will:
|
||||
- the DST checks run on the 23-hour and 25-hour days, not on a normal
|
||||
24-hour day where a fixed-24-slot implementation and a correct one
|
||||
agree exactly.
|
||||
|
||||
⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run
|
||||
`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it
|
||||
stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving
|
||||
edit reverted inside one second is invisible to CPython's staleness test. That
|
||||
is TEST-15, and it has already turned one live mutation into a false survivor.
|
||||
Read MUTATION.md before you report a survivor.
|
||||
"""
|
||||
|
||||
import ast
|
||||
|
||||
@@ -48,6 +48,13 @@ THREE RULES THIS FILE HOLDS ITSELF TO, because the reviewer will:
|
||||
and a stale one is not a fresh one: the gate blocks on `no_data`, blocks on
|
||||
an hour the series does not cover, and blocks on a provisional hour - three
|
||||
different reasons, none of them an answer.
|
||||
|
||||
⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run
|
||||
`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it
|
||||
stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving
|
||||
edit reverted inside one second is invisible to CPython's staleness test. That
|
||||
is TEST-15, and it has already turned one live mutation into a false survivor.
|
||||
Read MUTATION.md before you report a survivor.
|
||||
"""
|
||||
|
||||
import ast
|
||||
|
||||
@@ -45,6 +45,13 @@ THREE RULES THIS FILE HOLDS ITSELF TO, because the reviewer will:
|
||||
price, and zero is exactly the value that makes everything look free. So
|
||||
`static_plan` with no profile is checked to produce a read with NO
|
||||
`.points`, and `price_at` on it must RAISE rather than answer 0.0.
|
||||
|
||||
⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run
|
||||
`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it
|
||||
stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving
|
||||
edit reverted inside one second is invisible to CPython's staleness test. That
|
||||
is TEST-15, and it has already turned one live mutation into a false survivor.
|
||||
Read MUTATION.md before you report a survivor.
|
||||
"""
|
||||
|
||||
import gzip
|
||||
|
||||
@@ -85,6 +85,13 @@ regression fixture that CI cannot see is not a regression fixture.
|
||||
|
||||
The recorder's window rolls: 2026-08-01 leaves it around 2026-09-15, which is
|
||||
why the committed copy is the artefact and not the API call.
|
||||
|
||||
⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run
|
||||
`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it
|
||||
stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving
|
||||
edit reverted inside one second is invisible to CPython's staleness test. That
|
||||
is TEST-15, and it has already turned one live mutation into a false survivor.
|
||||
Read MUTATION.md before you report a survivor.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
@@ -5,6 +5,13 @@ three refusable configurations are refused. The second asserts that everything
|
||||
on validate.py's "deliberately not refused" list still STARTS - because the way
|
||||
a validator dies is not by missing a fault, it is by refusing something harmless
|
||||
until somebody deletes the call.
|
||||
|
||||
⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run
|
||||
`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it
|
||||
stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving
|
||||
edit reverted inside one second is invisible to CPython's staleness test. That
|
||||
is TEST-15, and it has already turned one live mutation into a false survivor.
|
||||
Read MUTATION.md before you report a survivor.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
@@ -8,6 +8,13 @@ person switches it off.
|
||||
The headline validation is not here - it is `sim/replay_sign.py` in the infra
|
||||
repo, which drives this module over the two captured bus days. This file is the
|
||||
table-driven half: boundaries, timers, the latch, and the arming gate.
|
||||
|
||||
⚠️ MUTATING THE MODULE THIS FILE GRADES? PURGE `__pycache__` FIRST - or run
|
||||
`tools/mutation_run.sh`, which does. `python3 -B` alone does NOT save you: it
|
||||
stops bytecode being WRITTEN, not a stale one being READ, and a size-preserving
|
||||
edit reverted inside one second is invisible to CPython's staleness test. That
|
||||
is TEST-15, and it has already turned one live mutation into a false survivor.
|
||||
Read MUTATION.md before you report a survivor.
|
||||
"""
|
||||
|
||||
import sys
|
||||
|
||||
Executable
+95
@@ -0,0 +1,95 @@
|
||||
#!/bin/sh
|
||||
# The house mutation harness (TEST-15). `tools/mutation_run.sh [TREE] [test_*.py ...]`
|
||||
#
|
||||
# WHY THIS EXISTS. Nearly every safety claim in this repository was verified by
|
||||
# mutation: change the shipped module, run the suite, watch it go red, revert,
|
||||
# watch it go green. FCST-03's developer found that the middle two steps can be
|
||||
# a lie. CPython decides a `.pyc` is still valid from (source size, source mtime
|
||||
# TRUNCATED TO WHOLE SECONDS). Revert `self.provisional` to `here.provisional`
|
||||
# and neither term moves - same length, same second - so the interpreter loads
|
||||
# the bytecode it already has. Confirmed by disassembly: the source read `here`,
|
||||
# the running code loaded `self`. The mutation "survived" because it never ran.
|
||||
#
|
||||
# ⚠️ `python3 -B` IS NOT THE FIX, AND NEITHER IS PYTHONDONTWRITEBYTECODE=1.
|
||||
# Both stop bytecode being WRITTEN. Neither stops a stale `.pyc` that is already
|
||||
# on disk from being READ - measured, and graded by test_bytecode.py part 2.
|
||||
# The load-bearing half is the PURGE. `-B` is here so the run leaves no cache
|
||||
# behind for the NEXT mutation to inherit.
|
||||
#
|
||||
# ⚠️ NOT A SLEEP. One-second mtime resolution is the mechanism, not the enemy;
|
||||
# the answer is to stop trusting mtime, not to make every edit wait a second.
|
||||
# A sleep also only covers the edit you remembered to slow down.
|
||||
#
|
||||
# It prints a fingerprint of the tree it is about to grade, because the other
|
||||
# half of a mutation report is WHICH SOURCE produced it: a report that cannot
|
||||
# say what was on disk is the same unverifiable claim in a different dress.
|
||||
# Diff two fingerprints and the mutated file is the line that changed.
|
||||
set -eu
|
||||
|
||||
TREE=""
|
||||
case "${1-}" in
|
||||
"" ) : ;;
|
||||
-h|--help ) sed -n '2,30p' "$0"; exit 0 ;;
|
||||
* ) if [ -d "$1" ]; then TREE=$1; shift; fi ;;
|
||||
esac
|
||||
if [ -z "$TREE" ]; then
|
||||
# Default: the add-on directory this script ships in, so `tools/mutation_run.sh`
|
||||
# works from anywhere. An explicit TREE is what lets the infra repo's replay
|
||||
# harnesses purge $GOODWE_ADDON_REPO before they grade it - see MUTATION.md.
|
||||
TREE=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
|
||||
fi
|
||||
PY=${PY:-python3}
|
||||
|
||||
cd "$TREE"
|
||||
|
||||
# CI-01's rule, for the same reason: a loop over an empty glob exits 0, and a
|
||||
# green run that examined nothing is worse than a red one.
|
||||
if [ $# -eq 0 ]; then set -- test_*.py; fi
|
||||
if [ ! -f "$1" ]; then
|
||||
echo "mutation_run: no test_*.py under $TREE - NOTHING has been graded." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "== purge"
|
||||
find . -name '__pycache__' -type d -prune -exec rm -rf {} + 2>/dev/null || true
|
||||
find . -name '*.pyc' -type f -exec rm -f {} + 2>/dev/null || true
|
||||
left=$(find . -name '__pycache__' -type d -o -name '*.pyc' -type f | head -5)
|
||||
if [ -n "$left" ]; then
|
||||
# A read-only tree cannot hold a cache written during this run either, so this
|
||||
# is only reachable when the purge itself failed on a tree that HAS one - the
|
||||
# one state where continuing would grade unknown bytecode.
|
||||
echo "mutation_run: could not purge, and a cache remains:" >&2
|
||||
echo "$left" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo " no bytecode cache under $TREE"
|
||||
|
||||
echo "== tree graded (sha256, bytes)"
|
||||
find app -name '*.py' 2>/dev/null | sort | while read -r f; do
|
||||
printf ' %s %8d %s\n' "$(sha256sum "$f" | cut -c1-16)" "$(wc -c < "$f")" "$f"
|
||||
done
|
||||
|
||||
echo "== run ($PY -B)"
|
||||
rc=0
|
||||
for f in "$@"; do
|
||||
start=$(date +%s)
|
||||
if "$PY" -B "$f" > /tmp/mutation_run.out 2>&1; then
|
||||
echo " PASS $f ($(( $(date +%s) - start ))s)"
|
||||
else
|
||||
echo " FAIL $f ($(( $(date +%s) - start ))s)"
|
||||
tail -40 /tmp/mutation_run.out
|
||||
rc=1
|
||||
fi
|
||||
done
|
||||
|
||||
# -B was asked for; check it was honoured rather than assuming. A cache here
|
||||
# would be inherited by the next mutation in the batch, which is exactly the
|
||||
# state this script exists to prevent.
|
||||
after=$(find . -name '__pycache__' -type d -o -name '*.pyc' -type f | head -5)
|
||||
if [ -n "$after" ]; then
|
||||
echo "mutation_run: a bytecode cache appeared DURING the run - the next" >&2
|
||||
echo "mutation in this batch would inherit it:" >&2
|
||||
echo "$after" >&2
|
||||
rc=1
|
||||
fi
|
||||
exit $rc
|
||||
Reference in New Issue
Block a user