FCST-03's developer reverted self.provisional to here.provisional, re-ran, and reported the mutation as a survivor. It was not one. CPython validates a cached .pyc on (source size, source mtime truncated to whole seconds), and that edit moved neither number — same length, same second — so the interpreter loaded the bytecode it already had. Disassembly settled it at the time: the source read here, the running code loaded self. A mutation run had graded code that was not there, with no error and exit status 0.
This does not make the results already on the record wrong. It makes them unverified, and that distinction is what the card is about — so nothing here re-runs a campaign and nothing here quietly assumes the old numbers hold.
Evidence
Reproduced (AC 1).test_bytecode.py part 1 builds a real package whose two return values are the same length, imports it in a real subprocess, rewrites it inside the same second (retrying until two writes land in one whole second — milliseconds, no sleep) and reads back which value ran. The interpreter runs SELF off a file that says HERE.
And the two obvious mitigations do not close it. Measured, not read off documentation:
writes bytecode
reads a stale .pyc
python3 -B
no
yes
PYTHONDONTWRITEBYTECODE=1
no
yes
purging __pycache__
(unchanged)
no — nothing to read
Both flags are write-side only, and a mutation run always has a cache already because the previous run wrote it. The advice in test_control.py's coverage audit led with -B; it is corrected here. -B still earns its place — it keeps this run from leaving a cache for the next edit to inherit.
No sleep anywhere. One-second mtime resolution is the mechanism, not the enemy; waiting it out would slow every run down and still only cover the edit somebody remembered to slow down.
What I chose, and why
goodwe_controller/tools/mutation_run.sh — purge, then a sha256/byte fingerprint of every app/*.py it is about to grade (so a mutation report can say which source produced it — the reviewer's addition to this card), then each suite under python3 -B, refusing a tree with no test_*.py (CI-01's rule, for CI-01's reason) and failing if a cache appears during the run. It takes the tree as an argument so the infra repo's replay harnesses can call it without being moved.
Where the guarantee is stated (AC 2).goodwe_controller/MUTATION.md carries the mechanism, the measurement, the reporting rules and the survey; every one of the 16 suites points at it from its docstring. That list is derived from the tree by part 5, so a new suite joining without the pointer is red.
CI (AC 4) — belt and braces, not a live hole. Verified rather than trusted: with the checkout bind mounted -v "$WS:/w:ro" the interpreter writes no__pycache__ into the tree at all, and it fails silently (no error, exit 0); the same probe on a writable mount produces one. Each job also rm -rfs its workspace and clones afresh. PYTHONDONTWRITEBYTECODE=1 is added because the :ro is one character wide and the next person debugging a write will be tempted by it. Part 6 grades both on the docker command line, because the first draft searched the whole workflow file and stayed green after the flag was deleted — the comment explaining it still named it.
The shipped add-on is not affected and the question is closed:goodwe_controller/Dockerfile has set PYTHONDONTWRITEBYTECODE=1 since it was written. No deploy-side finding.
The survey (AC 3)
In MUTATION.md, scoped by the property. The defect is one-directional: a mutation that never took leaves the suite green, so stale bytecode manufactures survivors and cannot manufacture a red. That puts out of scope every "all red, no survivors" claim in the window; every mutation executed in-process (SAFETY-07's B1/B2/B3, test_peakguard parts 7 and 10 — no source file is edited); and everything run after the purge discipline landed on 2026-08-31 (SAFETY-24, SAFETY-26, OPT-04, FCST-03's own re-run).
What is left is the recorded survivors from source-edit runs before that date. Two were re-run here under the harness and both still stand — TARIFF-01's record_block() ignoring its policy argument, and SAFETY-03's abs() on the clamp limit, 15 suites green each on the tree as it stood. The rest are labelled still-unverified: one harness run each to close, and the moment to spend it is when somebody next touches that module.
Red and green
Every new check was broken and watched fail: purge commented out of the harness → part 4 red; one docstring pointer removed → part 5 red, naming the file; :ro and the -e flag removed → part 6 red. Full suite green, 16 files, in the CI image on a read-only mount — the shape the runner uses.
What I am unsure about
The infra repo's sim/ harnesses are still exposed and this PR does not fix them.replay_sign.py and friends import the add-on out of $GOODWE_ADDON_REPO and write __pycache__ into both trees (goodwe/sim/__pycache__ exists today), so SAFETY-02's headline replay carries the same risk. The harness takes a tree argument for exactly that, but the change belongs in that repo and some of those harnesses want the rig. Documented in MUTATION.md, not done.
Part 1 depends on a CPython implementation detail. If a future interpreter validates by hash (PEP 552) rather than mtime, part 1 goes red on a machine where the defect no longer exists. I would rather it say so loudly than pass silently — but a reader hitting that should change the file, not the harness.
The docstring banner touches 15 suites for one paragraph each. It is the only form I found that puts the rule in front of the person writing the mutation, and part 5 keeps it true; a reviewer who wants it thinner should say so.
The survey's still-unverified rows are a judgement call. I did not re-run them, per the card's scoping. If any of those diagnoses is load-bearing for a decision, it is one tools/mutation_run.sh away.
## The defect
FCST-03's developer reverted `self.provisional` to `here.provisional`, re-ran, and reported the mutation as a **survivor**. It was not one. CPython validates a cached `.pyc` on **(source size, source mtime truncated to whole seconds)**, and that edit moved neither number — same length, same second — so the interpreter loaded the bytecode it already had. Disassembly settled it at the time: the source read `here`, the running code loaded `self`. A mutation run had graded code that was not there, with no error and exit status 0.
This does **not** make the results already on the record wrong. It makes them *unverified*, and that distinction is what the card is about — so nothing here re-runs a campaign and nothing here quietly assumes the old numbers hold.
## Evidence
**Reproduced (AC 1).** `test_bytecode.py` part 1 builds a real package whose two return values are the same length, imports it in a real subprocess, rewrites it inside the same second (retrying until two writes land in one whole second — milliseconds, **no sleep**) and reads back which value ran. The interpreter runs `SELF` off a file that says `HERE`.
**And the two obvious mitigations do not close it.** Measured, not read off documentation:
| | writes bytecode | reads a stale `.pyc` |
|---|---|---|
| `python3 -B` | no | **yes** |
| `PYTHONDONTWRITEBYTECODE=1` | no | **yes** |
| purging `__pycache__` | (unchanged) | **no — nothing to read** |
Both flags are write-side only, and a mutation run always has a cache already because the previous run wrote it. The advice in `test_control.py`'s coverage audit led with `-B`; it is corrected here. `-B` still earns its place — it keeps *this* run from leaving a cache for the *next* edit to inherit.
**No sleep anywhere.** One-second mtime resolution is the mechanism, not the enemy; waiting it out would slow every run down and still only cover the edit somebody remembered to slow down.
## What I chose, and why
`goodwe_controller/tools/mutation_run.sh` — purge, then a `sha256`/byte fingerprint of every `app/*.py` it is about to grade (so a mutation report can say *which* source produced it — the reviewer's addition to this card), then each suite under `python3 -B`, refusing a tree with no `test_*.py` (CI-01's rule, for CI-01's reason) and failing if a cache appears during the run. It takes the tree as an argument so the infra repo's replay harnesses can call it without being moved.
**Where the guarantee is stated (AC 2).** `goodwe_controller/MUTATION.md` carries the mechanism, the measurement, the reporting rules and the survey; every one of the 16 suites points at it from its docstring. That list is **derived from the tree** by part 5, so a new suite joining without the pointer is red.
**CI (AC 4) — belt and braces, not a live hole.** Verified rather than trusted: with the checkout bind mounted `-v "$WS:/w:ro"` the interpreter writes **no** `__pycache__` into the tree at all, and it fails *silently* (no error, exit 0); the same probe on a writable mount produces one. Each job also `rm -rf`s its workspace and clones afresh. `PYTHONDONTWRITEBYTECODE=1` is added because the `:ro` is one character wide and the next person debugging a write will be tempted by it. Part 6 grades both **on the docker command line**, because the first draft searched the whole workflow file and stayed green after the flag was deleted — the comment explaining it still named it.
**The shipped add-on is not affected and the question is closed:** `goodwe_controller/Dockerfile` has set `PYTHONDONTWRITEBYTECODE=1` since it was written. No deploy-side finding.
## The survey (AC 3)
In `MUTATION.md`, scoped **by the property**. The defect is one-directional: a mutation that never took leaves the suite green, so stale bytecode **manufactures survivors and cannot manufacture a red**. That puts out of scope every "all red, no survivors" claim in the window; every mutation executed **in-process** (SAFETY-07's B1/B2/B3, `test_peakguard` parts 7 and 10 — no source file is edited); and everything run after the purge discipline landed on 2026-08-31 (SAFETY-24, SAFETY-26, OPT-04, FCST-03's own re-run).
What is left is the recorded **survivors** from source-edit runs before that date. Two were re-run here under the harness and **both still stand** — TARIFF-01's `record_block()` ignoring its `policy` argument, and SAFETY-03's `abs()` on the clamp limit, 15 suites green each on the tree as it stood. The rest are labelled **still-unverified**: one harness run each to close, and the moment to spend it is when somebody next touches that module.
## Red and green
Every new check was broken and watched fail: purge commented out of the harness → part 4 red; one docstring pointer removed → part 5 red, naming the file; `:ro` and the `-e` flag removed → part 6 red. Full suite green, **16 files**, in the CI image on a read-only mount — the shape the runner uses.
## What I am unsure about
* **The infra repo's `sim/` harnesses are still exposed and this PR does not fix them.** `replay_sign.py` and friends import the add-on out of `$GOODWE_ADDON_REPO` and write `__pycache__` into both trees (`goodwe/sim/__pycache__` exists today), so SAFETY-02's headline replay carries the same risk. The harness takes a tree argument for exactly that, but the change belongs in that repo and some of those harnesses want the rig. Documented in `MUTATION.md`, not done.
* **Part 1 depends on a CPython implementation detail.** If a future interpreter validates by hash (PEP 552) rather than mtime, part 1 goes red on a machine where the defect no longer exists. I would rather it say so loudly than pass silently — but a reader hitting that should change the file, not the harness.
* **The docstring banner touches 15 suites** for one paragraph each. It is the only form I found that puts the rule in front of the person writing the mutation, and part 5 keeps it true; a reviewer who wants it thinner should say so.
* **The survey's still-unverified rows are a judgement call.** I did not re-run them, per the card's scoping. If any of those diagnoses is load-bearing for a decision, it is one `tools/mutation_run.sh` away.
FCST-03's developer reverted `self.provisional` to `here.provisional`, re-ran,
and reported the mutation as a survivor. It was not one. CPython validates a
cached `.pyc` on (source size, source mtime truncated to whole seconds), and
that edit moved neither number - same length, same second - so the interpreter
loaded the bytecode it already had. Disassembly settled it at the time: the
source read `here`, the running code loaded `self`. A mutation run had graded
code that was not there, with no error and exit status 0.
This does not make the results already on the record wrong. It makes them
UNVERIFIED, and that distinction is the whole card - so nothing here re-runs a
campaign and nothing here quietly assumes the old numbers hold.
REPRODUCED, THEN CLOSED. test_bytecode.py builds a real package whose two
return values are the same length, imports it in a real subprocess, rewrites it
inside the same second (retrying until both writes land in one whole second -
milliseconds, no sleep) and reads back which value ran. Part 1 shows the
interpreter running SELF off a file that says HERE.
AND THE TWO OBVIOUS MITIGATIONS DO NOT WORK. Measured, not read off
documentation: `python3 -B` and PYTHONDONTWRITEBYTECODE=1 are write-side only.
Neither stops an EXISTING stale .pyc being read, and a mutation run always has
one, because the previous run wrote it. Part 2 grades both still running the old
code. The advice in test_control.py's coverage audit led with `-B`; it is
corrected here - the purge is the half that works, and `-B` earns its place by
keeping THIS run from leaving a cache for the NEXT edit to inherit.
NO SLEEP ANYWHERE. One-second mtime resolution is the mechanism, not the enemy.
Waiting it out would slow every run down and would still only cover the edit
somebody remembered to slow down.
THE HARNESS. tools/mutation_run.sh purges, prints a sha256/byte fingerprint of
every app/*.py it is about to grade - so a mutation report can say WHICH source
produced it, which is the reviewer's addition to this card - runs each suite
under `python3 -B`, refuses a tree with no test_*.py (CI-01's rule, for CI-01's
reason), and fails if a cache appears during the run. It takes the tree as an
argument so the infra repo's replay harnesses can call it without being moved.
WHERE IT IS SAID (AC 2). MUTATION.md carries the mechanism, the measurement,
the reporting rules and the survey, and every one of the 16 suites now points at
it from its docstring. That list is DERIVED FROM THE TREE by test_bytecode.py
part 5, so a new suite that joins this directory without the pointer is red -
the same failure shape CI-01's file glob exists to prevent.
CI WAS ALREADY IMMUNE, AND IS DESCRIBED THAT WAY (AC 4). Verified rather than
reasoned: with the checkout bind mounted `-v "$WS:/w:ro"` the interpreter writes
no __pycache__ into the tree at all, and it fails SILENTLY - no error, exit
status 0 - while the same probe on a writable mount produces one. Each job also
rm -rf's its workspace and clones afresh, so nothing crosses between jobs on the
shared runner. PYTHONDONTWRITEBYTECODE=1 is added as belt and braces only: the
`:ro` is one character wide and the next person debugging a write will be
tempted by it. Part 6 grades both ON THE DOCKER COMMAND LINE rather than on the
file - the first draft searched the whole workflow and stayed green after the
flag was deleted, because the comment explaining it still named it.
THE SHIPPED ADD-ON IS NOT AFFECTED and the question is closed:
goodwe_controller/Dockerfile has set PYTHONDONTWRITEBYTECODE=1 since it was
written. There is no deploy-side finding.
THE SURVEY (AC 3), in MUTATION.md, scoped BY THE PROPERTY. The defect is
one-directional - a mutation that never took leaves the suite green, so stale
bytecode manufactures SURVIVORS and cannot manufacture a red - which puts every
"all red, no survivors" claim in the window out of scope, along with mutations
executed in-process (SAFETY-07's B1/B2/B3, test_peakguard parts 7 and 10, where
no source file is edited) and everything run after the purge discipline landed
on 2026-08-31 (SAFETY-24, SAFETY-26, OPT-04, FCST-03's re-run). What is left is
the recorded SURVIVORS from source-edit runs before that date. Two were re-run
here under the harness and both still stand: TARIFF-01's record_block() ignoring
its policy argument, and SAFETY-03's abs() on the clamp limit - 15 suites green
each, on the tree as it stood. The rest are labelled still-unverified rather
than assumed or re-run; each is one harness run to close, and the moment to
spend it is when somebody next touches that module.
OUT OF SCOPE AND RECORDED: the infra repo's sim/ replay harnesses import the
add-on and write bytecode into both trees (goodwe/sim/__pycache__ exists today),
so SAFETY-02's headline replay carries the same exposure. mutation_run.sh takes
a tree argument for exactly that, but the change belongs in that repo and some
of those harnesses want the rig.
RED AND GREEN. Every new check was broken and watched fail: the purge commented
out of the harness turns part 4 red (and its non-vacuity probe already proves
the fixture would mis-grade without it); one docstring pointer removed turns
part 5 red naming the file; `:ro` and the `-e` flag removed from the workflow
turn part 6 red. Full suite green, 16 files, in the CI image on a read-only
mount - the shape the runner uses.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
admin
merged commit d64790b3e2 into release/1.02026-09-01 02:22:16 +02:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
The defect
FCST-03's developer reverted
self.provisionaltohere.provisional, re-ran, and reported the mutation as a survivor. It was not one. CPython validates a cached.pycon (source size, source mtime truncated to whole seconds), and that edit moved neither number — same length, same second — so the interpreter loaded the bytecode it already had. Disassembly settled it at the time: the source readhere, the running code loadedself. A mutation run had graded code that was not there, with no error and exit status 0.This does not make the results already on the record wrong. It makes them unverified, and that distinction is what the card is about — so nothing here re-runs a campaign and nothing here quietly assumes the old numbers hold.
Evidence
Reproduced (AC 1).
test_bytecode.pypart 1 builds a real package whose two return values are the same length, imports it in a real subprocess, rewrites it inside the same second (retrying until two writes land in one whole second — milliseconds, no sleep) and reads back which value ran. The interpreter runsSELFoff a file that saysHERE.And the two obvious mitigations do not close it. Measured, not read off documentation:
.pycpython3 -BPYTHONDONTWRITEBYTECODE=1__pycache__Both flags are write-side only, and a mutation run always has a cache already because the previous run wrote it. The advice in
test_control.py's coverage audit led with-B; it is corrected here.-Bstill earns its place — it keeps this run from leaving a cache for the next edit to inherit.No sleep anywhere. One-second mtime resolution is the mechanism, not the enemy; waiting it out would slow every run down and still only cover the edit somebody remembered to slow down.
What I chose, and why
goodwe_controller/tools/mutation_run.sh— purge, then asha256/byte fingerprint of everyapp/*.pyit is about to grade (so a mutation report can say which source produced it — the reviewer's addition to this card), then each suite underpython3 -B, refusing a tree with notest_*.py(CI-01's rule, for CI-01's reason) and failing if a cache appears during the run. It takes the tree as an argument so the infra repo's replay harnesses can call it without being moved.Where the guarantee is stated (AC 2).
goodwe_controller/MUTATION.mdcarries the mechanism, the measurement, the reporting rules and the survey; every one of the 16 suites points at it from its docstring. That list is derived from the tree by part 5, so a new suite joining without the pointer is red.CI (AC 4) — belt and braces, not a live hole. Verified rather than trusted: with the checkout bind mounted
-v "$WS:/w:ro"the interpreter writes no__pycache__into the tree at all, and it fails silently (no error, exit 0); the same probe on a writable mount produces one. Each job alsorm -rfs its workspace and clones afresh.PYTHONDONTWRITEBYTECODE=1is added because the:rois one character wide and the next person debugging a write will be tempted by it. Part 6 grades both on the docker command line, because the first draft searched the whole workflow file and stayed green after the flag was deleted — the comment explaining it still named it.The shipped add-on is not affected and the question is closed:
goodwe_controller/Dockerfilehas setPYTHONDONTWRITEBYTECODE=1since it was written. No deploy-side finding.The survey (AC 3)
In
MUTATION.md, scoped by the property. The defect is one-directional: a mutation that never took leaves the suite green, so stale bytecode manufactures survivors and cannot manufacture a red. That puts out of scope every "all red, no survivors" claim in the window; every mutation executed in-process (SAFETY-07's B1/B2/B3,test_peakguardparts 7 and 10 — no source file is edited); and everything run after the purge discipline landed on 2026-08-31 (SAFETY-24, SAFETY-26, OPT-04, FCST-03's own re-run).What is left is the recorded survivors from source-edit runs before that date. Two were re-run here under the harness and both still stand — TARIFF-01's
record_block()ignoring itspolicyargument, and SAFETY-03'sabs()on the clamp limit, 15 suites green each on the tree as it stood. The rest are labelled still-unverified: one harness run each to close, and the moment to spend it is when somebody next touches that module.Red and green
Every new check was broken and watched fail: purge commented out of the harness → part 4 red; one docstring pointer removed → part 5 red, naming the file;
:roand the-eflag removed → part 6 red. Full suite green, 16 files, in the CI image on a read-only mount — the shape the runner uses.What I am unsure about
sim/harnesses are still exposed and this PR does not fix them.replay_sign.pyand friends import the add-on out of$GOODWE_ADDON_REPOand write__pycache__into both trees (goodwe/sim/__pycache__exists today), so SAFETY-02's headline replay carries the same risk. The harness takes a tree argument for exactly that, but the change belongs in that repo and some of those harnesses want the rig. Documented inMUTATION.md, not done.tools/mutation_run.shaway.