Commit Graph
6 Commits
Author SHA1 Message Date
glenn schrooyenandClaude Opus 5.5 86bb9d79cc 0.5.0: pipeline support for ems-dev-tooling
- test-designer and release-reviewer keep a named model (Opus when none, never Haiku)
- verifier checks every T-id and N-id of a "## Test plan" section
- manager guidance: record each verdict with `ticket` before resume or rewrite

Already running on the homelab install (applied from ems-dev-tooling patches/model-router-0.5.0-pipeline.patch).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bdqr7PiaHcBnzPituu2yTC
2026-11-17 21:28:36 +01:00
adminandClaude Opus 5.5 d5c762b339 Fix #5: rerun reported files alone, and on base, before judging
The full suite run can time out files that pass alone (an overloaded
shared host, a suite-wide deadline in the runner): MAINT-28 T307 was
judged on 30 TIMEOUTs, T325 on a file that passes alone in 230 s.

- With testFileCommand set, every file the full run reports as TIMEOUT
  or FAIL (testIssuePattern, default ^(TIMEOUT|FAIL)\s+(\S+); lines are
  read as the run streams) is rerun alone in the clone,
  testRerunJobs at a time, testFileTimeoutSec each, at most
  testRerunMax files.
- A file that still fails or times out alone runs again on the base,
  in a git worktree next to the clone (<clone>-verifier-base), which is
  removed and pruned afterwards. Without a worktree, base stays "not
  run".
- The verifier gets a table (file, full run, alone on head, on base,
  meaning) and is told: passes alone does not count; fails on base too
  is not caused by the change; name every file it counts.
- Only file names of letters, digits and _ . - / reach the shell; no
  ".." parts, no leading "-".
- 12 new tests; each of the 23 new rules was also broken on purpose and
  caught.

Version 0.4.9.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MdqPWvuDpbpRvDEdURoAwo
2026-11-14 00:09:18 +00:00
adminandClaude Opus 5.5 fc41ad21aa Fix TOOL-12 follow-up: no line or verifier on interim stops
A dev that waits on its own background work (a 35-45 min test suite)
ends several turns before it hands back. Every turn end posted a new
"verifying on Opus..." line and started the verifier on an unfinished
tree (MAINT-28 T307: 5 lines; T276/T278 judged an uncommitted tree).

- After each turn end of a tracked agent, its loop status decides:
  completed, failed, killed, or already dropped by the engine is the
  hand-back; waiting, idle, running or pending is an interim stop.
- An interim stop posts nothing and starts no verifier. The router
  checks the status again every 15 s, up to 24 h, and runs the round
  once the run has ended. A newer turn end takes the watch over.
- The hand-back uses the last turn's model and end reason.
- 8 new tests, including the ticket's acceptance (3 interim stops: one
  line, one verdict, both after the hand-back); each new rule was also
  broken on purpose and caught, apart from one redundant guard.

Version 0.4.7.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MdqPWvuDpbpRvDEdURoAwo
2026-11-12 21:36:04 +00:00
adminandClaude Opus 5.5 eba7a16b32 Fix TOOL-12: one line on every dev hand-back, by the model that ran
An Opus-run dev got no verification and no message, so silence meant
both "still verifying" and "will never verify" (SAFETY-135 waited
overnight, SAFETY-126 8 h). Verdicts were also labelled with the
routing choice, not the model the run used.

- Every dev hand-back posts one line at once:
  "[model-router] Tn verifying on Opus..." (a verdict follows), or
  "[model-router] Tn no verification: dev ran on Opus. Go straight to
  review." A run that ends with an error or names no clone says so.
- Each round is decided and labelled by the model the run reported
  (its end usage), the routing choice only as fallback. A difference
  is shown as "(routed to X, ran on Y)", also on later rounds.
- 8 new tests; each new rule was also broken on purpose and caught,
  apart from one equivalent change.

Version 0.4.6.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MdqPWvuDpbpRvDEdURoAwo
2026-11-11 14:23:51 +00:00
adminandClaude Opus 5.5 05f9663129 Fix TOOL-02: read verifier verdicts with prose around them
The verdict reader matched from the first { to the last } in the whole
reply, so a fenced JSON verdict followed by prose containing a } failed
to parse, counted as FAIL and triggered a rewrite (SAFETY-129, T192).

- Read a ```json fenced block first, then the first balanced {...} that
  parses and has a PASS or FAIL verdict. Braces inside strings are
  ignored.
- An unreadable reply is re-asked once for the JSON only. Still
  unreadable: NOT VERIFIED, "verdict unreadable", handed back to the
  manager with the start of the first reply. No fail counted, no
  resume, no rewrite.
- 10 new tests; each new rule was also broken on purpose and caught.

Version 0.4.5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MdqPWvuDpbpRvDEdURoAwo
2026-11-07 00:45:56 +00:00
adminandClaude Opus 5.5 92bc4dc348 model-router v0.4.4
Claude Code plugin: routes subagents by how well-defined the brief is,
verifies dev work on a second model in the agent's own clone, and
estimates savings against Opus 5.5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MdqPWvuDpbpRvDEdURoAwo
2026-11-06 23:46:30 +00:00