TOOL-02: parseVerdict reads a verifier PASS as FAIL when prose after the JSON has a brace, then orders a needless rewrite #1

Closed
opened 2026-11-07 01:37:39 +01:00 by admin · 1 comment
Owner

Trello: TOOL-02 https://trello.com/c/6aee55de45577a3362fb97ba

What happens
On goodwe-addon SAFETY-129 (router task T192, round 2, 2026-11-06), the verifier (Opus) answered with a fenced ```json block {"verdict": "PASS", "issues": [], "rewrite_brief": ""} followed by prose. The router reported "FAIL again ... Verifier reply was not valid JSON" and ordered a rewrite on a new Opus agent. The owner overrode it.

Where
hooks/router.ts parseVerdict() (v0.4.4, lines 186-198). The regex /\{[\s\S]*\}/ is greedy: it spans from the first { to the LAST } in the whole reply. Likely cause (not proven, the full reply was not kept): the prose after the JSON contained a }, so JSON.parse got JSON + prose and threw. Any unreadable reply counts as FAIL, which escalates to a rewrite.

Wanted

  • Read a ```json fenced block first, then the first balanced {...} object that parses and has a verdict key; tolerate prose before and after.
  • An unreadable reply is not a FAIL that escalates: re-ask the verifier once for the JSON only, then hand back to the manager with "verdict unreadable". Never auto-rewrite on a parse error.
  • The posted message shows the parsed verdict, or the first 300 chars plus "unreadable".

Acceptance
tests/router.test.ts: fenced JSON + prose containing } -> PASS; bare JSON -> PASS; FAIL with issues; garbage -> re-ask, then hand-back, no rewrite.

Priority P2, size S.

Trello: TOOL-02 https://trello.com/c/6aee55de45577a3362fb97ba **What happens** On goodwe-addon SAFETY-129 (router task T192, round 2, 2026-11-06), the verifier (Opus) answered with a fenced ```json block `{"verdict": "PASS", "issues": [], "rewrite_brief": ""}` followed by prose. The router reported "FAIL again ... Verifier reply was not valid JSON" and ordered a rewrite on a new Opus agent. The owner overrode it. **Where** `hooks/router.ts` `parseVerdict()` (v0.4.4, lines 186-198). The regex `/\{[\s\S]*\}/` is greedy: it spans from the first `{` to the LAST `}` in the whole reply. Likely cause (not proven, the full reply was not kept): the prose after the JSON contained a `}`, so `JSON.parse` got JSON + prose and threw. Any unreadable reply counts as FAIL, which escalates to a rewrite. **Wanted** - Read a ```json fenced block first, then the first balanced `{...}` object that parses and has a `verdict` key; tolerate prose before and after. - An unreadable reply is not a FAIL that escalates: re-ask the verifier once for the JSON only, then hand back to the manager with "verdict unreadable". Never auto-rewrite on a parse error. - The posted message shows the parsed verdict, or the first 300 chars plus "unreadable". **Acceptance** `tests/router.test.ts`: fenced JSON + prose containing `}` -> PASS; bare JSON -> PASS; FAIL with issues; garbage -> re-ask, then hand-back, no rewrite. Priority P2, size S.
Author
Owner

Fixed in v0.4.5 (05f9663). Installed into ~/projects/goodwe/tools/model-router and reloaded in the goodwe session 2026-11-07.

Fixed in v0.4.5 (05f9663). Installed into ~/projects/goodwe/tools/model-router and reloaded in the goodwe session 2026-11-07.
admin closed this issue 2026-11-07 01:54:02 +01:00
Sign in to join this conversation.
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: admin/model-router#1