checks / unit (pull_request) Successful in 3m35s
FCST-03's developer reverted `self.provisional` to `here.provisional`, re-ran, and reported the mutation as a survivor. It was not one. CPython validates a cached `.pyc` on (source size, source mtime truncated to whole seconds), and that edit moved neither number - same length, same second - so the interpreter loaded the bytecode it already had. Disassembly settled it at the time: the source read `here`, the running code loaded `self`. A mutation run had graded code that was not there, with no error and exit status 0. This does not make the results already on the record wrong. It makes them UNVERIFIED, and that distinction is the whole card - so nothing here re-runs a campaign and nothing here quietly assumes the old numbers hold. REPRODUCED, THEN CLOSED. test_bytecode.py builds a real package whose two return values are the same length, imports it in a real subprocess, rewrites it inside the same second (retrying until both writes land in one whole second - milliseconds, no sleep) and reads back which value ran. Part 1 shows the interpreter running SELF off a file that says HERE. AND THE TWO OBVIOUS MITIGATIONS DO NOT WORK. Measured, not read off documentation: `python3 -B` and PYTHONDONTWRITEBYTECODE=1 are write-side only. Neither stops an EXISTING stale .pyc being read, and a mutation run always has one, because the previous run wrote it. Part 2 grades both still running the old code. The advice in test_control.py's coverage audit led with `-B`; it is corrected here - the purge is the half that works, and `-B` earns its place by keeping THIS run from leaving a cache for the NEXT edit to inherit. NO SLEEP ANYWHERE. One-second mtime resolution is the mechanism, not the enemy. Waiting it out would slow every run down and would still only cover the edit somebody remembered to slow down. THE HARNESS. tools/mutation_run.sh purges, prints a sha256/byte fingerprint of every app/*.py it is about to grade - so a mutation report can say WHICH source produced it, which is the reviewer's addition to this card - runs each suite under `python3 -B`, refuses a tree with no test_*.py (CI-01's rule, for CI-01's reason), and fails if a cache appears during the run. It takes the tree as an argument so the infra repo's replay harnesses can call it without being moved. WHERE IT IS SAID (AC 2). MUTATION.md carries the mechanism, the measurement, the reporting rules and the survey, and every one of the 16 suites now points at it from its docstring. That list is DERIVED FROM THE TREE by test_bytecode.py part 5, so a new suite that joins this directory without the pointer is red - the same failure shape CI-01's file glob exists to prevent. CI WAS ALREADY IMMUNE, AND IS DESCRIBED THAT WAY (AC 4). Verified rather than reasoned: with the checkout bind mounted `-v "$WS:/w:ro"` the interpreter writes no __pycache__ into the tree at all, and it fails SILENTLY - no error, exit status 0 - while the same probe on a writable mount produces one. Each job also rm -rf's its workspace and clones afresh, so nothing crosses between jobs on the shared runner. PYTHONDONTWRITEBYTECODE=1 is added as belt and braces only: the `:ro` is one character wide and the next person debugging a write will be tempted by it. Part 6 grades both ON THE DOCKER COMMAND LINE rather than on the file - the first draft searched the whole workflow and stayed green after the flag was deleted, because the comment explaining it still named it. THE SHIPPED ADD-ON IS NOT AFFECTED and the question is closed: goodwe_controller/Dockerfile has set PYTHONDONTWRITEBYTECODE=1 since it was written. There is no deploy-side finding. THE SURVEY (AC 3), in MUTATION.md, scoped BY THE PROPERTY. The defect is one-directional - a mutation that never took leaves the suite green, so stale bytecode manufactures SURVIVORS and cannot manufacture a red - which puts every "all red, no survivors" claim in the window out of scope, along with mutations executed in-process (SAFETY-07's B1/B2/B3, test_peakguard parts 7 and 10, where no source file is edited) and everything run after the purge discipline landed on 2026-08-31 (SAFETY-24, SAFETY-26, OPT-04, FCST-03's re-run). What is left is the recorded SURVIVORS from source-edit runs before that date. Two were re-run here under the harness and both still stand: TARIFF-01's record_block() ignoring its policy argument, and SAFETY-03's abs() on the clamp limit - 15 suites green each, on the tree as it stood. The rest are labelled still-unverified rather than assumed or re-run; each is one harness run to close, and the moment to spend it is when somebody next touches that module. OUT OF SCOPE AND RECORDED: the infra repo's sim/ replay harnesses import the add-on and write bytecode into both trees (goodwe/sim/__pycache__ exists today), so SAFETY-02's headline replay carries the same exposure. mutation_run.sh takes a tree argument for exactly that, but the change belongs in that repo and some of those harnesses want the rig. RED AND GREEN. Every new check was broken and watched fail: the purge commented out of the harness turns part 4 red (and its non-vacuity probe already proves the fixture would mis-grade without it); one docstring pointer removed turns part 5 red naming the file; `:ro` and the `-e` flag removed from the workflow turn part 6 red. Full suite green, 16 files, in the CI image on a read-only mount - the shape the runner uses. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
149 lines
8.0 KiB
YAML
149 lines
8.0 KiB
YAML
# CI-01: this repository's FIRST automated check. Before this file, admin/goodwe-addon
|
|
# had no `.gitea/workflows/` directory at all - eight runnable test files, ~100 s
|
|
# of real coverage including a wall-clock cadence test, and not one machine ever
|
|
# ran them. Every merge here was a human reading a diff.
|
|
#
|
|
# ⚠️ NO `paths:` FILTER, AND ITS ABSENCE IS LOAD-BEARING. Branch protection on
|
|
# release/1.0 requires the context `checks / unit (pull_request)`. Gitea leaves a
|
|
# required context it never receives PENDING forever, so the moment this workflow
|
|
# learns to sit a PR out, that PR becomes unmergeable. Condition a STEP if you
|
|
# must; never the workflow.
|
|
#
|
|
# ⚠️ THE RUNNER IS SHARED WITH THE OTHER REPO'S RIG. `goodwe-runner` on
|
|
# 192.168.2.147 carries the labels `self-hosted` and `goodwe-rig` and is
|
|
# registered `:host`, and it is the only runner there is - so this job wears the
|
|
# rig label without wanting a rig. Nothing below starts, stops, or reads the
|
|
# `goodwe-sim` compose project or takes `sim/riglock.py`; it runs one short
|
|
# container and exits. It does occupy the runner's single slot (`capacity: 1`),
|
|
# so it QUEUES behind a rig campaign - a delayed verdict, never a destroyed one.
|
|
#
|
|
# NO JS ACTIONS: the runner image has no node, so `actions/checkout` cannot run.
|
|
# Checkout is seven lines of git below.
|
|
name: checks
|
|
|
|
on:
|
|
pull_request:
|
|
branches: [release/1.0, master]
|
|
workflow_dispatch:
|
|
|
|
concurrency:
|
|
# ⚠️ PER PULL REQUEST, NOT A BARE NAME. A shared group holds one running and
|
|
# one pending run, so a bare name lets a second PR supersede a first PR's run
|
|
# before any runner has picked it up - which is exactly what happened to the
|
|
# other repo's run 521 (four jobs, `started_at` at the Unix epoch,
|
|
# `runner_id: 0`, concluded `cancelled`, and the PR merged anyway). Do not
|
|
# replace this with a constant.
|
|
group: addon-checks-${{ github.event.pull_request.number || github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
jobs:
|
|
unit:
|
|
runs-on: [self-hosted, goodwe-rig]
|
|
timeout-minutes: 70 # ~4 to run; the rest is queueing for the single slot
|
|
env:
|
|
# ⚠️ UNDER /home/homelab, NOT $GITHUB_WORKSPACE (ENV-01, and re-measured
|
|
# during CI-01). act's workspace is `/root/.cache/act/<id>/hostexecutor`, a
|
|
# path that exists only INSIDE the runner container. Hand it to the host's
|
|
# snap-confined dockerd and the build fails with "failed to read
|
|
# dockerfile", or - worse, and this is the documented case - a bind mount
|
|
# resolves to an EMPTY DIRECTORY with no error at all. /home/homelab is
|
|
# mirrored 1:1 into the runner and is under $HOME, where snap confinement
|
|
# accepts it. The same probe under /tmp failed and under $HOME succeeded.
|
|
WS: /home/homelab/ci/addon-checks-${{ github.run_number }}
|
|
steps:
|
|
# ⚠️ THE CHECKOUT IS TWO LEVELS BELOW THE MOUNT ROOT, DELIBERATELY.
|
|
# test_freeze.py's find_hwsim() reads `HERE.parents[1] / "goodwe" / "sim"`,
|
|
# looking for the infra repo as a SIBLING of this one. Check out flat -
|
|
# goodwe_controller directly at the mount root - and `parents[1]` does not
|
|
# exist, so the test dies on `IndexError: 1` before its first assertion.
|
|
# Measured during CI-01; the flat layout was tried first and did exactly
|
|
# that. $WS/goodwe-addon/goodwe_controller gives parents[1] == $WS.
|
|
- name: checkout into a path the host shares (git; no node for actions/checkout)
|
|
env:
|
|
URL: ${{ github.server_url }}/${{ github.repository }}.git
|
|
TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
run: |
|
|
set -eu
|
|
rm -rf "$WS" && mkdir -p "$WS/goodwe-addon" && cd "$WS/goodwe-addon"
|
|
git init -q .
|
|
git remote add origin "$(echo "$URL" | sed "s#://#://x-access-token:${TOKEN}@#")"
|
|
git fetch -q --depth 1 origin "$GITHUB_REF" || git fetch -q --depth 1 origin "$GITHUB_SHA"
|
|
git checkout -qf FETCH_HEAD
|
|
echo "checked out $(git log --oneline -1)"
|
|
|
|
- name: build the test image (shipped deps, from apk - see .gitea/ci/Dockerfile)
|
|
run: |
|
|
set -eu
|
|
docker build -q -t goodwe-addon-checks:1 "$WS/goodwe-addon/.gitea/ci" >/dev/null
|
|
docker run --rm goodwe-addon-checks:1 python3 -c \
|
|
'import aiohttp, paho.mqtt.client as m, sys; print("python", sys.version.split()[0], "aiohttp", aiohttp.__version__)'
|
|
|
|
# ⚠️ THE FILE LIST IS DERIVED FROM THE TREE, NOT TYPED HERE. A hand-written
|
|
# list is how a new test file joins the repo and is never run - the exact
|
|
# shape of failure this ticket exists to close. The step globs `test_*.py`,
|
|
# prints the count, and REFUSES A COUNT OF ZERO: a loop over an empty glob
|
|
# exits 0 and would report a green gate that examined nothing.
|
|
#
|
|
# ⚠️ test_loop.py TAKES ~93 SECONDS OF WALL CLOCK ON PURPOSE (its own
|
|
# docstring says so, and says why: it drives the real `run_control()`
|
|
# against a real wall-clock floor, and mocking the clock would leave the
|
|
# assertion testing the mock). It is not hung. Do not "fix" it with a
|
|
# shorter timeout - a 60 s limit was tried during CI-01 and reported
|
|
# exactly the false hang the docstring predicts.
|
|
#
|
|
# ⚠️ WHAT THIS RUN DOES NOT COVER, stated because a pass here would
|
|
# otherwise over-claim. test_freeze.py's third section wants the infra
|
|
# repo's `sim/hwsim.py` to drive a real HTTP meter through the real
|
|
# transport. admin/goodwe is PRIVATE (checked via the API) and this
|
|
# workflow's GITHUB_TOKEN is minted for THIS repository only, so hwsim is
|
|
# absent here and `find_hwsim()` returns None. The rest of the file runs
|
|
# and passes. It does so SILENTLY - the file prints nothing to say the
|
|
# hwsim section was skipped - so "test_freeze.py: PASS" below is a weaker
|
|
# claim than it looks. That belongs to whoever owns test_freeze.py next;
|
|
# naming it here is the least this gate can do about it.
|
|
# ⚠️ PYTHONDONTWRITEBYTECODE=1 IS BELT AND BRACES, NOT A FIX FOR A LIVE
|
|
# HOLE. TEST-15 measured this runner's actual exposure and found none: the
|
|
# checkout below is bind mounted `:ro`, so the interpreter physically
|
|
# cannot write a __pycache__ into the tree it is grading - it fails
|
|
# SILENTLY, no error and exit status 0 - and every job `rm -rf`s $WS and
|
|
# clones afresh, so nothing survives from a previous job on this shared
|
|
# runner either. The variable is here because the `:ro` is the load-bearing
|
|
# part and it is one character wide: delete it while debugging a write and
|
|
# the gate quietly starts caching bytecode across a rebuild again. Note
|
|
# what it does NOT do, per TEST-15 part 2 - it stops bytecode being
|
|
# WRITTEN, never a stale one being READ. It is not a substitute for the
|
|
# purge in goodwe_controller/tools/mutation_run.sh.
|
|
- name: the add-on's own tests
|
|
run: |
|
|
set -eu
|
|
docker run --rm -e PYTHONDONTWRITEBYTECODE=1 -v "$WS:/w:ro" goodwe-addon-checks:1 sh -eu -c '
|
|
cd /w/goodwe-addon/goodwe_controller
|
|
set -- test_*.py
|
|
if [ "$1" = "test_*.py" ]; then
|
|
echo "NO test_*.py FOUND - the mount is empty or the layout moved. NOTHING has been checked."
|
|
exit 1
|
|
fi
|
|
echo "$# test file(s) to run"
|
|
rc=0
|
|
for f in "$@"; do
|
|
start=$(date +%s)
|
|
if python3 "$f" > /tmp/out 2>&1; then
|
|
echo "PASS $f ($(( $(date +%s) - start ))s)"
|
|
else
|
|
echo "FAIL $f ($(( $(date +%s) - start ))s)"
|
|
tail -40 /tmp/out
|
|
rc=1
|
|
fi
|
|
done
|
|
exit $rc
|
|
'
|
|
|
|
- name: clean up
|
|
if: always()
|
|
run: |
|
|
rm -rf "$WS" || true
|
|
# Root-owned by this runner inside a directory the `homelab` user owns,
|
|
# and sudo on that box is interactive-only, so the user cannot clean
|
|
# this up and the job that made it has to.
|
|
ls -1dt /home/homelab/ci/addon-checks-*/ 2>/dev/null | tail -n +6 | xargs -r rm -rf || true
|