Adds three new LLM-backed scenarios to tests/skill-behavior:
- S6: `/impeccable polish` → loads polish.md
- S7: `/impeccable audit` → loads audit.md
- S8: existing SvelteKit project (PRODUCT.md + DESIGN.md + src/app.css +
src/lib/components/*.svelte + src/routes/+page.svelte) → agent reads
at least one project code file to understand the existing design system
S6/S7 surface a real model-floor: gpt-5.4-mini reads brand.md, reads the
target index.html, and just does the polish/audit without ever loading
the sub-command reference. Stronger SKILL.md wording didn't move it.
Captured in the README baseline as a known weakness. Claude and Gemini
honor the load reliably.
To fix Gemini on S6/S7, sub-command reference loading is now Setup step 2
(right after context.mjs), not step 4 — placing it before the model gets
focused on "doing the work". Step 3 (design-system familiarization) is
tightened to require at least one project code read even when a
sub-command reference loads in step 2, so Claude doesn't laser-focus on
the sub-command flow and skip the broader exploration.
Two new fixtures: MINIMAL_LANDING_HTML (a tiny static landing page for
S6/S7) and SVELTE_PROJECT_FILES (a minimal SvelteKit scaffold with
tokens, components, and a routes/+page.svelte for S8). Both designed to
look real enough that agents treat them as production code.
Suite is now 24 tests across three providers; baseline is 21-22/24, with
the stable failures being gpt-5.4-mini scenarios 6 and 7.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces load-context.mjs's JSON output with a tight markdown block from
the renamed context.mjs. The script now extracts PRODUCT.md's `## Register`
field and appends a `NEXT STEP:` directive naming the matching reference
(brand.md / product.md), which moved Gemini from skipping the register
load entirely to honoring it. Drops the `.impeccable.md` auto-migration;
makes IMPECCABLE_CONTEXT_DIR a lazy escape hatch consulted only when the
default paths come up empty.
Setup is now four bullets in one list. The DESIGN.md nudge is gone; in
its place, a "familiarize with the existing design system" step that
calls out CSS / tokens / running app as authoritative sources alongside
DESIGN.md. The standalone `### Register` H3 stays for the cascade rules
(task cue → surface → register field).
New LLM-backed test suite at tests/skill-behavior/ runs five scenarios
against claude-haiku-4-5, gpt-5.4-mini, and gemini-3.1-flash-lite via
Vercel AI SDK. Captures real tool traces, asserts on context.mjs calls,
brand.md loads, and teach.md fallback. Skips cleanly when API keys are
unset. 13-14/15 pass; only stable failure is the v3.2.0-era gpt-mini S4
"don't re-run" regression. Adds @ai-sdk/google as devDep and the
test:skill-behavior npm script.
Touches em-dashes in skill/SKILL.md and four reference files so
`bun run build:skills` passes its skill-prose validator. teach.md and
document.md drop their "re-run the loader to refresh session cache"
steps since the agent's own write is now the freshest source.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Paul BakausandClaude Opus 4.7