CI-15: run the add-on's test files in parallel (the serial suite passes the runner's time limit).
What changes
.gitea/ci/run-tests.sh (new) runs the test_*.py files 6 at a time (xargs -P 6), longest first. Then test_loop.py, test_freeze.py and test_safety87.py run alone, one after the other, once the batch has finished (wall-clock margins; reasons are in the script). test_safety87.py was found by this ticket: its status page goes "Not ready - Meter: no reading for N seconds, limit 30 seconds" when the file runs slowly (it failed in a loaded CI run, 112 s, and alone under --cpus 0.15). No test file is changed.
checks.yml, step "the add-on's own tests": the loop is replaced by one line that runs the script inside the same docker run (same :ro mount, PYTHONDONTWRITEBYTECODE=1, GOODWE_REQUIRE_HWSIM=1, same image). Comment added. timeout-minutes 70 -> 40, with its comment corrected.
Kept: PASS f (Ns) / FAIL f (Ns) lines (one block per file, printed whole under a lock), the CI: echo on PASS, tail -40 on FAIL, the "NO test_*.py FOUND" failure, a failing file fails the job. Each file writes to its own output file, not one shared /tmp/out. A file that never reports a result also fails the job.
The file list is still taken from the tree; the typed lists in the script only sort or set aside files that exist.
README.md: one clause on how CI runs the tests.
Job name and context checks / unit (pull_request) unchanged. No test file changed. No version bump; the CHANGELOG has no unreleased section, so no line added.
Results
CI run 1432 (head ffb0d61, before test_safety87 moved): green, job 19:39:44Z -> 19:59:36Z (19 m 52 s); 86 files, 0 failed; parallel batch 1050 s (test_modes_p4 is the longest file and sets it), then test_loop 94 s and test_freeze 35 s alone.
CI run on this head: see the comment below.
Local (goodwe-addon-checks:1, :ro, GOODWE_REQUIRE_HWSIM=1): serial 3166 s; parallel 897 s, 1123 s (host busy), 0 failed; a third parallel run under host load ~30 failed only test_safety87 (now serial). test_loop and test_freeze passed in all 3 parallel-mode runs, same as serial.
Fail proof: draft PR #216 (throwaway, adds a failing test file; not to be merged).
CI-15: run the add-on's test files in parallel (the serial suite passes the runner's time limit).
## What changes
- `.gitea/ci/run-tests.sh` (new) runs the `test_*.py` files 6 at a time (`xargs -P 6`), longest first. Then `test_loop.py`, `test_freeze.py` and `test_safety87.py` run alone, one after the other, once the batch has finished (wall-clock margins; reasons are in the script). `test_safety87.py` was found by this ticket: its status page goes "Not ready - Meter: no reading for N seconds, limit 30 seconds" when the file runs slowly (it failed in a loaded CI run, 112 s, and alone under `--cpus 0.15`). No test file is changed.
- `checks.yml`, step "the add-on's own tests": the loop is replaced by one line that runs the script inside the same `docker run` (same `:ro` mount, `PYTHONDONTWRITEBYTECODE=1`, `GOODWE_REQUIRE_HWSIM=1`, same image). Comment added. `timeout-minutes` 70 -> 40, with its comment corrected.
- Kept: `PASS f (Ns)` / `FAIL f (Ns)` lines (one block per file, printed whole under a lock), the `CI: ` echo on PASS, `tail -40` on FAIL, the "NO test_*.py FOUND" failure, a failing file fails the job. Each file writes to its own output file, not one shared `/tmp/out`. A file that never reports a result also fails the job.
- The file list is still taken from the tree; the typed lists in the script only sort or set aside files that exist.
- `README.md`: one clause on how CI runs the tests.
- Job name and context `checks / unit (pull_request)` unchanged. No test file changed. No version bump; the CHANGELOG has no unreleased section, so no line added.
## Results
- CI run 1432 (head ffb0d61, before test_safety87 moved): green, job 19:39:44Z -> 19:59:36Z (19 m 52 s); 86 files, 0 failed; parallel batch 1050 s (test_modes_p4 is the longest file and sets it), then test_loop 94 s and test_freeze 35 s alone.
- CI run on this head: see the comment below.
- Local (goodwe-addon-checks:1, `:ro`, GOODWE_REQUIRE_HWSIM=1): serial 3166 s; parallel 897 s, 1123 s (host busy), 0 failed; a third parallel run under host load ~30 failed only test_safety87 (now serial). test_loop and test_freeze passed in all 3 parallel-mode runs, same as serial.
- Fail proof: draft PR #216 (throwaway, adds a failing test file; not to be merged).
The serial loop summed 3 557 s on SAFETY-125's run and passed the runner's
60-minute cap. The loop moves to .gitea/ci/run-tests.sh: xargs -P 6, longest
files first, then test_loop.py and test_freeze.py (TEST-30 / CI-06) alone on an
idle machine. Same PASS/FAIL lines, CI: echo, tail -40, :ro mount,
PYTHONDONTWRITEBYTECODE=1, GOODWE_REQUIRE_HWSIM=1 and NO test_*.py FOUND guard;
one output file per test file; any failing or unreported file fails the job.
timeout-minutes 70 -> 40, comment corrected. No test file changed.
Co-Authored-By: Claude <noreply@anthropic.com>
Its status page says "Meter: no reading for N seconds, limit 30 seconds" when
N passes 30; N is about half the file's run time. It failed alone under
--cpus 0.15 (94 s), in a loaded local parallel run (103 s) and in a loaded CI
run (112 s, run 1434). Idle it takes ~25 s. No test file changed, so it runs
alone after the parallel batch, like test_loop.py and test_freeze.py.
Co-Authored-By: Claude <noreply@anthropic.com>
Reviewed head a645d899f48c48585e9d8f4780be7b128c9eb885 against release/1.0 @ 4a1f6cc. CI: run 1442 / job 2593, checks / unit (pull_request)success on that sha (22:54:55 to 23:10:58 UTC, 16 min 03 s; 86 PASS, 86 file(s), 0 failed, 953s wall). Run 1443 infra-pin / fresh success.
Fail-safe, proven locally in goodwe-addon-checks:1 (throwaway files, nothing pushed). Each case below makes the job exit 1: a file that exits 1 (FAIL plus tail -40); a file killed by SIGKILL; a file killed by OOM (--memory 256m); the worker shell killed (busybox xargs stops, the files left over show no result); xargs itself killed; a failing serial file; a killed serial file; an empty tree (NO test_*.py FOUND); /tmp read-only (mktemp fails); a file name with a space. No fail-open path found.
Output: one block per file under flock, no interleaving in CI or locally. Each file writes to its own $RUN_DIR/<f>.out in the container's /tmp. The mount stays :ro. Guards kept: job name and context, all guard steps, :ro, PYTHONDONTWRITEBYTECODE=1, GOODWE_REQUIRE_HWSIM=1, same image, infra pin. No test file changed. Full suite locally on a645d89: 86 PASS in 771 s (serial: 3166 s from the dev run). The PASS/FAIL set and the CI: lines match the serial run exactly. No file left a fixed-name path (checked with docker diff). Serial group: I agree with all three. test_safety87.py failed again at --cpus 0.15 ("no reading for 48 seconds, limit 30"). 23 other clock, sleep or status files all passed alone at --cpus 0.15, so none of them needs to move.
Findings (none block the merge; all fail closed)
Low.gitea/ci/run-tests.sh:104: busybox xargs runs the command once on empty input. If every file were in the serial group, the job would fail with line 57: 2: parameter not set and no FAIL line. Fix: xargs -r.
Low.gitea/ci/run-tests.sh:60 and :106-107: in the serial loop the test gets the while read stdin. A serial file that reads stdin eats the names after it, and those show as no result (fails closed, but blames the wrong file). Fix: python3 "$f" < /dev/null.
Low.gitea/ci/run-tests.sh:5,53,104: the workers re-run "$0" after cd. With a relative path (sh ./run-tests.sh from a clone), every file shows no result. Fix: make $0 absolute before the cd, or correct the comment.
Low.gitea/workflows/checks.yml:212: timeout-minutes: 40. CI took 16 and 20 min, but the dev's local run under load 30+ took 2269 s (38 min). 45 to 50 gives more room. A timeout fails closed.
Info.gitea/ci/run-tests.sh:60: there is no per-file timeout. A hung file is only stopped by the job timeout, and the log does not name it.
Info: a few tests take a port with bind-0, close, then rebind (test_modes.py:488, test_maint11.py:283, test_batteries_screen.py:357, test_setup_only.pyfree_port). In parallel, another file could take the port in that gap. The odds are tiny, and it would fail closed.
Follow-up: the three script fixes fit in one small PR.
## CI-15 review (Opus): approve, merging
Reviewed head `a645d899f48c48585e9d8f4780be7b128c9eb885` against `release/1.0` @ `4a1f6cc`. CI: run 1442 / job 2593, `checks / unit (pull_request)` **success** on that sha (22:54:55 to 23:10:58 UTC, 16 min 03 s; 86 PASS, `86 file(s), 0 failed, 953s wall`). Run 1443 `infra-pin / fresh` success.
**Fail-safe, proven locally** in `goodwe-addon-checks:1` (throwaway files, nothing pushed). Each case below makes the job exit 1: a file that exits 1 (FAIL plus tail -40); a file killed by SIGKILL; a file killed by OOM (`--memory 256m`); the worker shell killed (busybox xargs stops, the files left over show `no result`); xargs itself killed; a failing serial file; a killed serial file; an empty tree (`NO test_*.py FOUND`); `/tmp` read-only (mktemp fails); a file name with a space. No fail-open path found.
**Output:** one block per file under flock, no interleaving in CI or locally. Each file writes to its own `$RUN_DIR/<f>.out` in the container's `/tmp`. The mount stays `:ro`.
**Guards kept:** job name and context, all guard steps, `:ro`, `PYTHONDONTWRITEBYTECODE=1`, `GOODWE_REQUIRE_HWSIM=1`, same image, infra pin. No test file changed.
**Full suite locally** on a645d89: 86 PASS in 771 s (serial: 3166 s from the dev run). The PASS/FAIL set and the `CI: ` lines match the serial run exactly. No file left a fixed-name path (checked with `docker diff`).
**Serial group:** I agree with all three. `test_safety87.py` failed again at `--cpus 0.15` ("no reading for 48 seconds, limit 30"). 23 other clock, sleep or status files all passed alone at `--cpus 0.15`, so none of them needs to move.
### Findings (none block the merge; all fail closed)
- **Low** `.gitea/ci/run-tests.sh:104`: busybox `xargs` runs the command once on empty input. If every file were in the serial group, the job would fail with `line 57: 2: parameter not set` and no FAIL line. Fix: `xargs -r`.
- **Low** `.gitea/ci/run-tests.sh:60` and `:106-107`: in the serial loop the test gets the `while read` stdin. A serial file that reads stdin eats the names after it, and those show as `no result` (fails closed, but blames the wrong file). Fix: `python3 "$f" < /dev/null`.
- **Low** `.gitea/ci/run-tests.sh:5,53,104`: the workers re-run `"$0"` after `cd`. With a relative path (`sh ./run-tests.sh` from a clone), every file shows `no result`. Fix: make `$0` absolute before the `cd`, or correct the comment.
- **Low** `.gitea/workflows/checks.yml:212`: `timeout-minutes: 40`. CI took 16 and 20 min, but the dev's local run under load 30+ took 2269 s (38 min). 45 to 50 gives more room. A timeout fails closed.
- **Info** `.gitea/ci/run-tests.sh:60`: there is no per-file timeout. A hung file is only stopped by the job timeout, and the log does not name it.
- **Info**: a few tests take a port with bind-0, close, then rebind (`test_modes.py:488`, `test_maint11.py:283`, `test_batteries_screen.py:357`, `test_setup_only.py` `free_port`). In parallel, another file could take the port in that gap. The odds are tiny, and it would fail closed.
Follow-up: the three script fixes fit in one small PR.
admin
merged commit 6be7e41de1 into release/1.02026-10-30 00:29:18 +01:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
CI-15: run the add-on's test files in parallel (the serial suite passes the runner's time limit).
What changes
.gitea/ci/run-tests.sh(new) runs thetest_*.pyfiles 6 at a time (xargs -P 6), longest first. Thentest_loop.py,test_freeze.pyandtest_safety87.pyrun alone, one after the other, once the batch has finished (wall-clock margins; reasons are in the script).test_safety87.pywas found by this ticket: its status page goes "Not ready - Meter: no reading for N seconds, limit 30 seconds" when the file runs slowly (it failed in a loaded CI run, 112 s, and alone under--cpus 0.15). No test file is changed.checks.yml, step "the add-on's own tests": the loop is replaced by one line that runs the script inside the samedocker run(same:romount,PYTHONDONTWRITEBYTECODE=1,GOODWE_REQUIRE_HWSIM=1, same image). Comment added.timeout-minutes70 -> 40, with its comment corrected.PASS f (Ns)/FAIL f (Ns)lines (one block per file, printed whole under a lock), theCI:echo on PASS,tail -40on FAIL, the "NO test_*.py FOUND" failure, a failing file fails the job. Each file writes to its own output file, not one shared/tmp/out. A file that never reports a result also fails the job.README.md: one clause on how CI runs the tests.checks / unit (pull_request)unchanged. No test file changed. No version bump; the CHANGELOG has no unreleased section, so no line added.Results
ffb0d61, before test_safety87 moved): green, job 19:39:44Z -> 19:59:36Z (19 m 52 s); 86 files, 0 failed; parallel batch 1050 s (test_modes_p4 is the longest file and sets it), then test_loop 94 s and test_freeze 35 s alone.:ro, GOODWE_REQUIRE_HWSIM=1): serial 3166 s; parallel 897 s, 1123 s (host busy), 0 failed; a third parallel run under host load ~30 failed only test_safety87 (now serial). test_loop and test_freeze passed in all 3 parallel-mode runs, same as serial.CI-15 review (Opus): approve, merging
Reviewed head
a645d899f48c48585e9d8f4780be7b128c9eb885againstrelease/1.0@4a1f6cc. CI: run 1442 / job 2593,checks / unit (pull_request)success on that sha (22:54:55 to 23:10:58 UTC, 16 min 03 s; 86 PASS,86 file(s), 0 failed, 953s wall). Run 1443infra-pin / freshsuccess.Fail-safe, proven locally in
goodwe-addon-checks:1(throwaway files, nothing pushed). Each case below makes the job exit 1: a file that exits 1 (FAIL plus tail -40); a file killed by SIGKILL; a file killed by OOM (--memory 256m); the worker shell killed (busybox xargs stops, the files left over showno result); xargs itself killed; a failing serial file; a killed serial file; an empty tree (NO test_*.py FOUND);/tmpread-only (mktemp fails); a file name with a space. No fail-open path found.Output: one block per file under flock, no interleaving in CI or locally. Each file writes to its own
$RUN_DIR/<f>.outin the container's/tmp. The mount stays:ro.Guards kept: job name and context, all guard steps,
:ro,PYTHONDONTWRITEBYTECODE=1,GOODWE_REQUIRE_HWSIM=1, same image, infra pin. No test file changed.Full suite locally on
a645d89: 86 PASS in 771 s (serial: 3166 s from the dev run). The PASS/FAIL set and theCI:lines match the serial run exactly. No file left a fixed-name path (checked withdocker diff).Serial group: I agree with all three.
test_safety87.pyfailed again at--cpus 0.15("no reading for 48 seconds, limit 30"). 23 other clock, sleep or status files all passed alone at--cpus 0.15, so none of them needs to move.Findings (none block the merge; all fail closed)
.gitea/ci/run-tests.sh:104: busyboxxargsruns the command once on empty input. If every file were in the serial group, the job would fail withline 57: 2: parameter not setand no FAIL line. Fix:xargs -r..gitea/ci/run-tests.sh:60and:106-107: in the serial loop the test gets thewhile readstdin. A serial file that reads stdin eats the names after it, and those show asno result(fails closed, but blames the wrong file). Fix:python3 "$f" < /dev/null..gitea/ci/run-tests.sh:5,53,104: the workers re-run"$0"aftercd. With a relative path (sh ./run-tests.shfrom a clone), every file showsno result. Fix: make$0absolute before thecd, or correct the comment..gitea/workflows/checks.yml:212:timeout-minutes: 40. CI took 16 and 20 min, but the dev's local run under load 30+ took 2269 s (38 min). 45 to 50 gives more room. A timeout fails closed..gitea/ci/run-tests.sh:60: there is no per-file timeout. A hung file is only stopped by the job timeout, and the log does not name it.test_modes.py:488,test_maint11.py:283,test_batteries_screen.py:357,test_setup_only.pyfree_port). In parallel, another file could take the port in that gap. The odds are tiny, and it would fail closed.Follow-up: the three script fixes fit in one small PR.