mirror of
https://github.com/pbakaus/impeccable.git
synced 2026-09-12 06:06:37 +03:00
Paul BakausandClaude Opus 5
121602079c
Two failures the trace test found were structural, not model quirks. critique.md described the report's format and then went straight to writing a temp file, with no step saying to output the report. gemini-3.6-flash and luna both responded by bundling heredoc, snapshot write, trend read, and cleanup into one bash call and stopping, leaving a perfect archive nobody had read. A "Deliver the Report" step now precedes persistence, and persistence describes itself as a copy of what was already sent. gemini-3.6-flash failed three consecutive runs before this and its failures afterward all show the report reaching chat. The close is also step 6 of the persistence list rather than a section after it, since the same shape is what fixed delivery. Lineup: gpt-5.6-luna and deepseek-v4-flash out, gpt-5.6-terra in at reasoningEffort high (IMPECCABLE_SKILL_BEHAVIOR_EFFORT overrides), gemini 3.5 to 3.6. Provider options resolve from the model object inside the harness so the 21 runTurn call sites are untouched. Verified the effort actually reaches the API rather than being silently dropped. The Gemini bump was not cosmetic: 3.5-flash passed critique closes twice and 3.6-flash then failed three times against identical text. A version bump inside one family changed the outcome, so the README now treats cross-version carryover as unmeasured. Known floor, recorded: critique closes is flaky on gemini-3.6-flash, 1 run in 3. Two structural attempts moved it from consistently failing to intermittently passing and then stopped paying. claude-sonnet-5 and gpt-5.6-terra are clean. Prepared with AI assistance (Claude Code). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>