Verifier fails rounds on suite TIMEOUTs: 1500 s limit too short for this suite, and no rerun of the timed-out file #5

Closed
opened 2026-11-14 00:13:42 +01:00 by admin · 0 comments
Owner

Seen in a goodwe session (add-on suite: about 107 files, 1300-2300 s wall on a shared 12-core host).

  • MAINT-28 T307 round 2: the verifier's suite ended at 1500 s with 77 pass / 0 fail / 30 TIMEOUT and counted that against the dev. The dev's own full run was 107/0 in 2161 s; under light load 1390 s.
  • MAINT-28 T325 round 2: one FAIL point was test_tariff24_weather.py TIMEOUT. The file passes alone in 230 s on head and 234 s on base, and in all five of the dev's full runs.

Expected: a suite limit that fits the project (configurable per repo), and when a file times out, rerun that file alone (and on base) before listing it as an issue. The verdict should name the failing/timed-out file and say whether base has it too.

Seen in a goodwe session (add-on suite: about 107 files, 1300-2300 s wall on a shared 12-core host). - **MAINT-28 T307 round 2:** the verifier's suite ended at 1500 s with 77 pass / 0 fail / 30 TIMEOUT and counted that against the dev. The dev's own full run was 107/0 in 2161 s; under light load 1390 s. - **MAINT-28 T325 round 2:** one FAIL point was `test_tariff24_weather.py TIMEOUT`. The file passes alone in 230 s on head and 234 s on base, and in all five of the dev's full runs. Expected: a suite limit that fits the project (configurable per repo), and when a file times out, rerun that file alone (and on base) before listing it as an issue. The verdict should name the failing/timed-out file and say whether base has it too.
admin closed this issue 2026-11-14 01:10:56 +01:00
Sign in to join this conversation.
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: admin/model-router#5