Commit Graph
6 Commits
Author SHA1 Message Date
adminandClaude Opus 5.5 d5c762b339 Fix #5: rerun reported files alone, and on base, before judging
The full suite run can time out files that pass alone (an overloaded
shared host, a suite-wide deadline in the runner): MAINT-28 T307 was
judged on 30 TIMEOUTs, T325 on a file that passes alone in 230 s.

- With testFileCommand set, every file the full run reports as TIMEOUT
  or FAIL (testIssuePattern, default ^(TIMEOUT|FAIL)\s+(\S+); lines are
  read as the run streams) is rerun alone in the clone,
  testRerunJobs at a time, testFileTimeoutSec each, at most
  testRerunMax files.
- A file that still fails or times out alone runs again on the base,
  in a git worktree next to the clone (<clone>-verifier-base), which is
  removed and pruned afterwards. Without a worktree, base stays "not
  run".
- The verifier gets a table (file, full run, alone on head, on base,
  meaning) and is told: passes alone does not count; fails on base too
  is not caused by the change; name every file it counts.
- Only file names of letters, digits and _ . - / reach the shell; no
  ".." parts, no leading "-".
- 12 new tests; each of the 23 new rules was also broken on purpose and
  caught.

Version 0.4.9.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MdqPWvuDpbpRvDEdURoAwo
2026-11-14 00:09:18 +00:00
adminandClaude Opus 5.5 fd75c183ba Fix #4, #6: resumes keep their task; later instructions reach the verifier
Resuming an agent (SendMessage to a finished agent) raises agent.spawn
again, with resume.agentId set. The router treated every resume as a
new task: new Tn, fresh routing, the brief replaced by the resume
message, the failure count and round base reset. Hence T322's
"routed to Sonnet, ran on Opus" for an Opus agent that was only
resumed (#4), and verifiers that judged later rounds against the wrong
text (#6).

- A resume passes through untouched; its message is kept as a later
  instruction to that agent. Task, routing, failures and rounds stay.
- A SendMessage to a tracked agent (by id or by the name its Agent call
  gave it) that was delivered is kept as a later instruction too.
- The verifier prompt lists later instructions after the brief and
  judges scope added there as in scope.
- [router-rewrite Tn]: the stored rewrite is used up only once its
  agent has started (a failed start kept nothing before: T332 was then
  routed afresh, to Haiku). With nothing stored the call is refused
  instead of starting an agent with no brief. The task keeps Tn.
- The routing label compares with the model the engine resolved at
  start, not the one the router asked for.
- 11 new tests; each new rule was also broken on purpose and caught.

Version 0.4.8.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MdqPWvuDpbpRvDEdURoAwo
2026-11-13 23:41:48 +00:00
adminandClaude Opus 5.5 fc41ad21aa Fix TOOL-12 follow-up: no line or verifier on interim stops
A dev that waits on its own background work (a 35-45 min test suite)
ends several turns before it hands back. Every turn end posted a new
"verifying on Opus..." line and started the verifier on an unfinished
tree (MAINT-28 T307: 5 lines; T276/T278 judged an uncommitted tree).

- After each turn end of a tracked agent, its loop status decides:
  completed, failed, killed, or already dropped by the engine is the
  hand-back; waiting, idle, running or pending is an interim stop.
- An interim stop posts nothing and starts no verifier. The router
  checks the status again every 15 s, up to 24 h, and runs the round
  once the run has ended. A newer turn end takes the watch over.
- The hand-back uses the last turn's model and end reason.
- 8 new tests, including the ticket's acceptance (3 interim stops: one
  line, one verdict, both after the hand-back); each new rule was also
  broken on purpose and caught, apart from one redundant guard.

Version 0.4.7.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MdqPWvuDpbpRvDEdURoAwo
2026-11-12 21:36:04 +00:00
adminandClaude Opus 5.5 eba7a16b32 Fix TOOL-12: one line on every dev hand-back, by the model that ran
An Opus-run dev got no verification and no message, so silence meant
both "still verifying" and "will never verify" (SAFETY-135 waited
overnight, SAFETY-126 8 h). Verdicts were also labelled with the
routing choice, not the model the run used.

- Every dev hand-back posts one line at once:
  "[model-router] Tn verifying on Opus..." (a verdict follows), or
  "[model-router] Tn no verification: dev ran on Opus. Go straight to
  review." A run that ends with an error or names no clone says so.
- Each round is decided and labelled by the model the run reported
  (its end usage), the routing choice only as fallback. A difference
  is shown as "(routed to X, ran on Y)", also on later rounds.
- 8 new tests; each new rule was also broken on purpose and caught,
  apart from one equivalent change.

Version 0.4.6.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MdqPWvuDpbpRvDEdURoAwo
2026-11-11 14:23:51 +00:00
adminandClaude Opus 5.5 05f9663129 Fix TOOL-02: read verifier verdicts with prose around them
The verdict reader matched from the first { to the last } in the whole
reply, so a fenced JSON verdict followed by prose containing a } failed
to parse, counted as FAIL and triggered a rewrite (SAFETY-129, T192).

- Read a ```json fenced block first, then the first balanced {...} that
  parses and has a PASS or FAIL verdict. Braces inside strings are
  ignored.
- An unreadable reply is re-asked once for the JSON only. Still
  unreadable: NOT VERIFIED, "verdict unreadable", handed back to the
  manager with the start of the first reply. No fail counted, no
  resume, no rewrite.
- 10 new tests; each new rule was also broken on purpose and caught.

Version 0.4.5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MdqPWvuDpbpRvDEdURoAwo
2026-11-07 00:45:56 +00:00
adminandClaude Opus 5.5 92bc4dc348 model-router v0.4.4
Claude Code plugin: routes subagents by how well-defined the brief is,
verifies dev work on a second model in the agent's own clone, and
estimates savings against Opus 5.5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MdqPWvuDpbpRvDEdURoAwo
2026-11-06 23:46:30 +00:00