CI-18: affected-test selector for dev rounds, split the longest test files, 10 workers #254

Merged
admin merged 4 commits from CI-18 into release/1.0 2026-11-16 00:31:50 +01:00
Owner

CI-18: cut the wait on tests without letting the merge gate pass on content it did not fully test.

What this does

  • Selector for dev and review rounds: .gitea/ci/affected_tests.py lists the test files a diff reaches (imports, transitive, plus names in strings), a fixed safety set, and says SELECT ALL for anything it cannot map. .gitea/ci/run-affected.sh runs the selection in goodwe-addon-checks:1 through run-tests.sh (CI_ONLY).
  • Faster full suite: test_modes_p4, test_safety126, test_safety125 split into pieces (same check lines: 146, 35, 45 before and after), and run-tests.sh runs 10 files at a time instead of 6.
  • README: a short section with the local command.

What does not change

  • checks / unit (pull_request) still runs every test_*.py on the merge. No event filter, no change to branch protection or the context name. checks.yml never sets CI_ONLY; test_affected_tests.py fails if it does.

Report: reports/CI-18-dev-report.md

🤖 Generated with Claude Code

https://claude.ai/code/session_01Bdqr7PiaHcBnzPituu2yTC

CI-18: cut the wait on tests without letting the merge gate pass on content it did not fully test. What this does - Selector for dev and review rounds: `.gitea/ci/affected_tests.py` lists the test files a diff reaches (imports, transitive, plus names in strings), a fixed safety set, and says SELECT ALL for anything it cannot map. `.gitea/ci/run-affected.sh` runs the selection in `goodwe-addon-checks:1` through `run-tests.sh` (`CI_ONLY`). - Faster full suite: `test_modes_p4`, `test_safety126`, `test_safety125` split into pieces (same check lines: 146, 35, 45 before and after), and `run-tests.sh` runs 10 files at a time instead of 6. - README: a short section with the local command. What does not change - `checks / unit (pull_request)` still runs every `test_*.py` on the merge. No event filter, no change to branch protection or the context name. checks.yml never sets `CI_ONLY`; `test_affected_tests.py` fails if it does. Report: reports/CI-18-dev-report.md 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01Bdqr7PiaHcBnzPituu2yTC
admin added 1 commit 2026-11-15 20:54:09 +01:00
CI-18: affected-test selector for dev rounds; split the longest test files; 10 workers
infra-pin / fresh (pull_request) Successful in 2s
checks / unit (pull_request) Successful in 20m45s
a727dd7ee2
- .gitea/ci/affected_tests.py: which test_*.py a diff reaches (static import
  graph plus names in strings, transitive), a fixed safety set, SELECT ALL for
  anything it cannot map. .gitea/ci/run-affected.sh runs the selection in the
  CI image. run-tests.sh takes CI_ONLY for that; checks.yml never sets it and
  test_affected_tests.py fails if it does. The merge gate still runs every file.
- test_modes_p4 -> p4/p4b/p4c/p4d, test_safety126 -> +b, test_safety125 -> +b:
  same check lines, none dropped (146, 35, 45 before and after).
- run-tests.sh: 10 workers instead of 6, new longest-first order.

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bdqr7PiaHcBnzPituu2yTC
admin added 1 commit 2026-11-15 21:16:09 +01:00
CI-18: checks.yml comments carry the measured run time and the worker count
checks / unit (pull_request) Successful in 20m41s
infra-pin / fresh (pull_request) Successful in 1s
17b65785b7
Comment-only: "~15 to run" was stale (runs took 27 min at 6 workers); the CI-18
run at 10 workers took 20 min 56 s (run 1682).

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bdqr7PiaHcBnzPituu2yTC
admin added 1 commit 2026-11-15 21:17:30 +01:00
CI-18: selector carries a change through a test that imports another test
checks / unit (pull_request) Successful in 21m20s
infra-pin / fresh (pull_request) Successful in 1s
b2d3066966
A test first reached as text (a pattern reader) was never expanded again when a
later path reached it by import, so a test importing it was missed. Expanded and
reached are now tracked apart; the empty diff selects the safety set. Test added.

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bdqr7PiaHcBnzPituu2yTC
admin added 1 commit 2026-11-16 00:10:43 +01:00
CI-18 review: --head needs B checked out; guard the CI image against CI_ONLY
checks / unit (pull_request) Successful in 20m22s
infra-pin / fresh (pull_request) Successful in 1s
22ef89bf10
The selector reads the graph, and run-affected.sh runs the tests, from the
working tree, so a --base/--head range is only right with the head checked
out. The CI image is built from the merged tree, so an ENV CI_ONLY in its
Dockerfile would make the gate partial; test_affected_tests.py now fails on it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Author
Owner

CI-18 review (full tier). Reviewed b2d3066, then my Low fixes as 22ef89b. Verdict: right, merging.

Checks

  1. Gate untouched: no event filter; the required context name is unchanged; the run line is unchanged (run-tests.sh with no file list, no -e CI_ONLY). The image Dockerfile has no ENV, and the runner's env does not pass into docker run, so CI_ONLY cannot take effect in CI. Branch protection was not changed (updated_at 2026-08-30 / 2026-09-23). The guard goes red: I added a comment naming run-affected.sh to checks.yml and test_affected_tests.py printed FAIL "checks.yml never calls the selector or the local wrapper", rc=1.
  2. Splits: I ran the originals on 7513fd9 and the parts on b2d3066. The check lines match as multisets: 146 (91+16+17+22), 35 (20+15), 45 (20+25), with 0 FAIL. Every output line of the originals, including the printed scenario values (verdicts, levels, quarters), appears in the parts. The only new lines are headers and footers, so no part runs without state it needs. Line coverage: the parts are the original's lines plus new docstrings. The only original line missing is the docstring's closing """.
  3. CI_JOBS 10: I ran the full suite twice locally under CI's command (10 workers, infra at the pin). Both had 115 of 115 files (115 in the tree), 0 failed, 1230 s and 1210 s. test_peakguard.py under --cpus 0.25 had 318 ok, 0 FAIL, 234 s. Together with the dev's 4 CI runs and 1 local run, and CI on 22ef89b, that makes 8 green full runs at 10 workers, with no flake seen.
  4. Selector fails open. My own cases:
    • An untracked .txt: ALL.
    • A syntax error in a changed tool: ALL.
    • A bad base ref: ALL.
    • --files with a missing path: ALL.
    • --names on ALL lists all 115 files.
    • A new unimported helper under app/ selects 87, through the tests that enumerate app/. That is over-selection, which is fine.
    • The safety set and its reasons are sensible. The selector is documented as local only.
  5. Docs: the "~15 to run" comment is replaced with the measured ~21. The README section and the "ten at a time" wording are right.
  6. CI: checks / unit (pull_request) is success on b2d3066 (run 1686, 21m20s) and on 22ef89b (run 1688, 20m22s).

Findings (all Low, fixed by me in 22ef89b)

  • Low, README.md:120 / .gitea/ci/run-affected.sh:6 / .gitea/ci/affected_tests.py:5: with --base A --head B, the selector takes the diff from B but reads the import graph from the working tree, and run-affected.sh runs the working tree's tests. A run with another branch checked out tests the wrong code. The docs now say "with B checked out".
  • Low, goodwe_controller/test_affected_tests.py:259: CI builds its image from the merged tree, so an ENV CI_ONLY in .gitea/ci/Dockerfile would make the gate partial while it still exits 0. I added a guard check and confirmed it goes red with that line added.

Info (no change)

  • .gitea/ci/run-affected.sh:33: the local default is CI_JOBS 6 while CI uses 10. That is reasonable on a shared host.
**CI-18 review (full tier).** Reviewed b2d3066, then my Low fixes as 22ef89b. Verdict: **right, merging.** **Checks** 1. Gate untouched: no event filter; the required context name is unchanged; the run line is unchanged (`run-tests.sh` with no file list, no `-e CI_ONLY`). The image Dockerfile has no ENV, and the runner's env does not pass into `docker run`, so CI_ONLY cannot take effect in CI. Branch protection was not changed (`updated_at` 2026-08-30 / 2026-09-23). The guard goes red: I added a comment naming `run-affected.sh` to checks.yml and `test_affected_tests.py` printed FAIL "checks.yml never calls the selector or the local wrapper", rc=1. 2. Splits: I ran the originals on 7513fd9 and the parts on b2d3066. The check lines match as multisets: 146 (91+16+17+22), 35 (20+15), 45 (20+25), with 0 FAIL. Every output line of the originals, including the printed scenario values (verdicts, levels, quarters), appears in the parts. The only new lines are headers and footers, so no part runs without state it needs. Line coverage: the parts are the original's lines plus new docstrings. The only original line missing is the docstring's closing `"""`. 3. CI_JOBS 10: I ran the full suite twice locally under CI's command (10 workers, infra at the pin). Both had 115 of 115 files (115 in the tree), 0 failed, 1230 s and 1210 s. `test_peakguard.py` under `--cpus 0.25` had 318 ok, 0 FAIL, 234 s. Together with the dev's 4 CI runs and 1 local run, and CI on 22ef89b, that makes 8 green full runs at 10 workers, with no flake seen. 4. Selector fails open. My own cases: - An untracked `.txt`: ALL. - A syntax error in a changed tool: ALL. - A bad base ref: ALL. - `--files` with a missing path: ALL. - `--names` on ALL lists all 115 files. - A new unimported helper under `app/` selects 87, through the tests that enumerate `app/`. That is over-selection, which is fine. - The safety set and its reasons are sensible. The selector is documented as local only. 5. Docs: the "~15 to run" comment is replaced with the measured ~21. The README section and the "ten at a time" wording are right. 6. CI: `checks / unit (pull_request)` is success on b2d3066 (run 1686, 21m20s) and on 22ef89b (run 1688, 20m22s). **Findings (all Low, fixed by me in 22ef89b)** - Low, README.md:120 / .gitea/ci/run-affected.sh:6 / .gitea/ci/affected_tests.py:5: with `--base A --head B`, the selector takes the diff from B but reads the import graph from the working tree, and `run-affected.sh` runs the working tree's tests. A run with another branch checked out tests the wrong code. The docs now say "with B checked out". - Low, goodwe_controller/test_affected_tests.py:259: CI builds its image from the merged tree, so an `ENV CI_ONLY` in `.gitea/ci/Dockerfile` would make the gate partial while it still exits 0. I added a guard check and confirmed it goes red with that line added. **Info (no change)** - .gitea/ci/run-affected.sh:33: the local default is `CI_JOBS` 6 while CI uses 10. That is reasonable on a shared host.
admin merged commit 5fc92cf6e8 into release/1.0 2026-11-16 00:31:50 +01:00
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: admin/goodwe-addon#254