mirror of
https://github.com/pbakaus/impeccable.git
synced 2026-09-11 21:57:14 +03:00
dca8f1ca6f860784ca9da26e76ded0b827c67ed7
149
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Paul BakausandClaude Opus 5
|
d4e1b0902f |
Enforce the ask_instruction sentence-initial contract
Review on #576 caught document.md:71 splicing {{ask_instruction}} after "then", which is the same defect this branch set out to fix. Rendered for Codex it produced "Show the user the existing file, then STOP and use Codex's structured user-input/question tool...". The line now starts a new sentence. The comment added to PROVIDER_PLACEHOLDERS asserted the contract without enforcing it, which is exactly how four reference files shipped the splice in the first place. validateAskInstructionSites() in build.js now checks every call site and fails the build on a mid-sentence interpolation, and the comment points at the gate instead of asking authors to remember. Prepared with AI assistance (Claude Code). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Paul BakausandClaude Opus 5
|
0e5c6cbe17 |
Keep critique's report out of the question's message
The critique report and the AskUserQuestion call shipped in one assistant message, so the report stayed hidden until the user answered the picker and the command read as if it had never run. Reorder critique's persistence steps so the temp-file cleanup runs after the report and trend line are sent. That cleanup now ends the message carrying the report, leaving the questions to open a fresh one. Both critique.md and overdrive.md state the constraint and why it exists, so the ordering is not an unexplained sequence a model can optimize away. Overdrive additionally moves its direction descriptions inside the question options, where the user is actually reading them. Also fix the ask_instruction splices. The placeholder is a complete sentence, but five call sites spliced it mid-sentence and shipped text like "stop and STOP and call the AskUserQuestion tool to clarify. before expanding it". Every call site is now sentence-initial and the twelve lowercase provider values are capitalized to match, with a comment in utils.js pinning the contract. Record a workflow-contract baseline for the current model lineup. The two failures seen while validating this change are pre-existing: bolder refinement fails on deepseek-v4-flash identically with bolder.md reverted to HEAD, and redesign replaces DESIGN is flaky on assertions driven by new-work.md, which this change does not touch. Prepared with AI assistance (Claude Code). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
 Paul BakausandGitHub
|
e36833ce21 |
Merge pull request #521 from digitallamb/pr/hermes-provider
Add Hermes Agent as a supported provider |
||
|
|
045865918a |
Held for review: agent placeholder substitution, reviewer recapture contract, base-directory script form (#544)
* Resolve {{scripts_path}} in the agent bodies Codex ships
Three code paths emit an agent body: the degraded fallback reference, the
.toml nested inside the skill for Codex, and the native agent file. Only the
nested .toml skipped placeholder substitution and rule-marker stripping, so
the codex and .agents dists shipped `node {{scripts_path}}/embed-prompt.mjs`
verbatim in the asset producer, and every caller had to substitute the token
itself at load time.
All three now render through renderAgentBody(), and the new regression test
asserts a runnable embed-prompt command on each emitted surface plus a
synthetic agent proving markers and placeholders resolve in the nested .toml.
Prepared by an AI agent (Claude Code) under pbakaus's instruction.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Give the finish reviewer's screenshots one fixed address
The Input Contract asked for "desktop and mobile screenshot paths captured by
the parent" and named none, so each session invented a filename and the
verdict pass went looking for a recapture that was never written there. Two
reviewer passes burned on that in the eval runs.
The parent now captures and recaptures to .impeccable/review/desktop.png and
.impeccable/review/mobile.png, and the reviewer reads those two first,
treating a brief-named path as the fallback for a parent that wrote elsewhere.
Prepared by an AI agent (Claude Code) under pbakaus's instruction.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Lead Setup with the base directory the runtime reports
The rendered claude and codex skills opened with
`node .claude/skills/impeccable/scripts/context.mjs`, a project-relative path
that resolves in this repo and in nothing a user installs: a personal or
plugin install puts the scripts outside the project entirely. The working form
was already in the text, parenthesized, after the one that fails.
Setup now leads with `node <skill-base-dir>/scripts/context.mjs` and says once
that the base directory resolves every scripts-path command in the skill and
its references, leaving the project-relative path as the fallback for runtimes
that report no base directory.
Prepared by an AI agent (Claude Code) under pbakaus's instruction.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Answer the Copilot review: brittle model assertion, missing review dir
Assert that {{model}} resolved rather than that it resolved to "GPT", which
belongs to PROVIDER_PLACEHOLDERS and can change without touching what the test
guards. And have the parent create .impeccable/review/ when the harness does
not, so a fresh project's first capture has somewhere to land.
Prepared by an AI agent (Claude Code) under pbakaus's instruction.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Make the review-screenshot contract directory-based, not web-viewport-named
Two amendments to the recapture contract from review feedback:
1. The canonical location is the directory .impeccable/review/, one file
per captured viewport; desktop.png and mobile.png are the web case,
not the contract. Baking web-viewport names into the reviewer's spec
would have hardened a web assumption into paths that a native
(ios/android/adaptive) build cannot honestly write.
2. Precedence restored to explicit-beats-convention: paths the calling
brief names are authoritative when the files exist; the canonical
directory is where the reviewer looks when the brief names none or a
named path is missing. This avoids stale canonical files from an
earlier run silently winning over fresh explicit paths. The observed
failure (the verdict round inventing a round-stamped filename) stays
fixed: recapture happens over the same files, and invented filenames
are still called out as pointing at nothing.
Assisted-by: Claude Code
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
  Forgeandplamb
|
5ce4a5c6b5 |
Add Hermes Agent as a supported provider
Impeccable now ships a Hermes-compatible bundle under dist/hermes/.hermes/skills/. Hermes reads the Agent Skills spec as-is, so the bundle uses the four spec frontmatter fields (name, description, version, license) plus metadata/compatibility and drops the Claude/Codex-specific extensions Hermes would silently ignore. The .hermes/skills/ tracked root and the regenerated pin.mjs mirrors will be produced by .github/workflows/sync-generated-output.yml after this lands; per AGENTS.md, generated harness churn stays out of feature PRs. What a Hermes user gets: - npx impeccable install --providers=hermes --scope=project writes .hermes/skills/impeccable/ into the cwd. - npx impeccable install --providers=hermes --scope=user honors $HERMES_HOME, so a profile-scoped install (HERMES_HOME=~/.hermes/profiles/forge) lands in the active profile's skills dir, not the default ~/.hermes/. Cross-home HERMES_HOME inheritance is ignored so test isolation holds. - /impeccable registers as a Hermes slash command and routes sub-commands via the Commands table in the skill body, since user-invocable / argument-hint are not honored by Hermes' skill loader. What a Hermes user does NOT get, and why: - No hook surface. Impeccable's PostToolUse/Stop anti-pattern detector on Claude/Codex/Cursor/Grok/GitHub does not translate to Hermes, which has no equivalent tool event lifecycle. The skill body still ships. - No writeOpenAIMetadata, agentFormat, or emitHooks. Hermes has no per-skill tool ACL, no subagent on-disk format, and no hooks.json equivalent. Verified end-to-end with hermes-agent v0.18.2: /impeccable polish and /impeccable critique both load reference/<command>.md and return the documented first step. parse_frontmatter accepts the generated SKILL.md, scan_skill_commands registers /impeccable, and the full default test suite passes (267/267 in the critical files; 0 fail across all suites). |
||
 Rex LorenzoandGitHub
|
14d2641685 |
Fix: keep the node runtime probe clear of cmd.exe metacharacters (#458)
Volta's Windows shims exec through `cmd /C`, which re-parses the argument list, so the `>=` inside the probe's `node -e` payload was read as output redirection. The command died with "The filename, directory name, or volume label syntax is incorrect" before node started, the guard read that as a missing runtime, and the hook it exists to protect was disabled on every PostToolUse and Stop. A user on a supported Node 24 got a one-time notice telling them to install Node 22, then silence. Clamping with Math.min is the same floor test in the same ES5-only syntax, with no character cmd.exe can claim. Verified through the Volta shim on Node 24.16.0 and 22.18.0 (exit 0) and against a real Node 20.6.1 binary (exit 1), so the floor is unchanged. Adds a regression test asserting no `<`, `>`, or newline reaches any generated `node -e` payload. Upstream cause: volta-cli/volta#1791. Prepared with AI assistance (Claude Code). |
||
|
Paul BakausandGitHub
|
d6a9891066 |
Share browser detector bundling (#498)
Centralize the browser-safe module set and source transformation so the browser and extension builders cannot drift. AI assistance: This refactor was prepared by Codex under pbakaus's scheduled architecture-simplification authorization. |
||
|
|
6d2af3f800 |
Guard the plugin loader contract that PR #494 exposed (#499)
* test: guard the plugin loader contract that PR #494 exposed The agents manifest key shipped for months and silently loaded zero of the four subagents; no validator looked at the generated plugin manifest's shape and claude plugin validate never checks it. Three layers now do: - scripts/lib/validate-plugin-manifest.js pins the verified loader contract (KNOWN_LOADER_KEYS allowlist, no agents key, trailing-slash skills path from issue #86, every skill/agents/*.md shipped in plugin/agents/), unit-tested in tests/validate-plugin-manifest.test.js including a check of the real committed subtree. - The same check gates bun run build next to the version-drift guard. - tests/plugin-e2e.test.mjs installs the committed ./plugin subtree into a real Claude Code (sandboxed via CLAUDE_CONFIG_DIR in a temp dir) and asserts the component inventory: skill parses, all agents visible, hooks discovered. In the default suite; runs in about a second and skips cleanly when the claude CLI is absent, so CI is unaffected. All three failed against the pre-#494 tree for the shipped reason (Agents 0 of 4) and pass against current main. AI-assisted via Claude Code under maintainer direction. Co-Authored-By: Claude Code <noreply@anthropic.com> * fix: address PR review bot findings - Copilot: guard collectPluginManifestFindings against valid JSON that is not an object (null, string, number, array) so a broken manifest is a finding instead of a build crash; unit test added - Copilot: update the plugin-e2e header comment, the suite is in the default lineup rather than opt-in AI-assisted via Claude Code under maintainer direction. Co-Authored-By: Claude Code <noreply@anthropic.com> * fix: harden plugin E2E sandbox isolation Bugbot: create the sandbox CLAUDE_CONFIG_DIR up front and redirect HOME and USERPROFILE into the temp workDir too, so a CLI code path that derives config or cache locations from the home directory instead of CLAUDE_CONFIG_DIR still cannot touch the developer's real Claude config when the default suite runs. AI-assisted via Claude Code under maintainer direction. Co-Authored-By: Claude Code <noreply@anthropic.com> * fix: agent parity check mirrors the build's emit rules Bugbot: the shipped filename is `${claude-name || name}.md` and a providers: list may exclude claude-code, so comparing raw source basenames could fail the build on a renamed or provider-scoped agent with a build:release hint that cannot fix it. The validator now derives expected filenames the same way the transformer factory does (shared parseFrontmatter, same providers gate) with unit coverage for renames, name overrides, and provider-scoped agents. AI-assisted via Claude Code under maintainer direction. Co-Authored-By: Claude Code <noreply@anthropic.com> * fix: run the plugin E2E through a shell on Windows Bugbot: the claude CLI is a .cmd shim on Windows and Node refuses to spawn those via execFile without a shell, so the availability probe always failed and the suite silently skipped there. Windows now invokes through a shell with every argument double-quoted (temp paths routinely contain spaces); the POSIX path is unchanged. AI-assisted via Claude Code under maintainer direction. Co-Authored-By: Claude Code <noreply@anthropic.com> --------- Co-authored-by: Claude Code <noreply@anthropic.com> |
||
  Abdul WahabandCursor
|
7d6109b723 |
Drop deprecated craft alias from the generated argument-hint
craft is a deprecated compatibility alias, but its SKILL_CATEGORIES entry kept it advertised in every generated SKILL.md argument-hint. Unmapping it removes it from the hint while the alias keeps routing through the router table and command-metadata.json. AI-assisted change (reviewed by maintainer). Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Paul BakausandGitHub
|
c91f3717d4 |
Merge pull request #466 from pbakaus/codex/remove-legacy-pattern-parser
Simplify curated pattern loading |
||
|
Paul Bakaus
|
33f824b5b5 |
Clarify curated pattern source
Describe the catalog as independent of SKILL.md extraction rather than repository content, addressing Copilot's review feedback. Prepared with Codex assistance under pbakaus's scheduled architecture cleanup authorization. |
||
|
Paul Bakaus
|
358fc2e716 |
Simplify curated pattern loading
Remove the unreachable legacy SKILL.md pattern parser now that readPatterns uses the curated catalog exclusively. Prepared with Codex assistance under pbakaus's scheduled architecture cleanup authorization. |
||
 Paweł Adamczak
|
bf957452c4 | Add Google Antigravity provider support | ||
|
Paul BakausandClaude Fable 5
|
fa1177ed9c |
Ship native subagent definitions for GitHub Copilot and Cursor
The github and cursor providers previously received only the generated
degraded/ inline fallbacks. Both harnesses support real custom subagents,
so the build now emits them from the same skill/agents/ source:
- GitHub Copilot: .github/agents/impeccable-<role>.agent.md with portable
frontmatter only (name + description; omitting tools grants all tools,
and Copilot has no documented model/effort/max-turns equivalents).
- Cursor: .cursor/agents/impeccable-<role>.md with name, description,
model: inherit, is_background: false, and readonly derived from the
agent's tool list (true only for the finish reviewer, which declares
neither Write nor Edit). effort/max-turns are skipped because Cursor's
effort option requires an explicit model id.
Agent bodies now also resolve {{scripts_path}} and strip rule markers in
the shared agentFormat pipeline, which fixes the previously unresolved
placeholder in the emitted Claude asset-producer agent.
The CLI installer places agents per scope: project installs write
<repo>/.github/agents/ and <repo>/.cursor/agents/; user-level installs
write ~/.copilot/agents/ (Copilot's user dir, not ~/.github/) and
~/.cursor/agents/, overwriting stale impeccable-* copies. Because
Copilot lets user-level agents shadow same-named project ones, a project
install warns when shadowing copies exist; Cursor gives project agents
precedence, so no warning there.
new-work.md and visualize.md extend their harness-naming clauses with
the Cursor and Copilot invocations. The degraded/ fallbacks keep
shipping for surfaces where the model still fails to delegate.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Paul BakausandGitHub
|
5e572c8b8a |
Merge pull request #423 from pbakaus/hook-guard-unsupported-node
Stop the design hook erroring on a node too old for ESM |
||
|
|
fd9076f4f0 |
Enforce the engines floor in the probe instead of a capability check
The probe asked whether node could load ESM, while the notice promised a Node 22 floor and package.json engines declares >=22.12.0. Reviewers kept flagging the gap, and they were right to: a 14.18-to-21 runtime passed the probe on the strength of one import while the hook and its detector bundle are only ever exercised on the engines floor, so "can load our code" was a weaker claim than the one being made for it. Check the floor directly: parseInt(process.versions.node) >= 22, in ES5-only syntax that parses on any node old enough to fail it. Probe and notice now derive from one NODE_MAJOR_FLOOR constant, so they cannot disagree, and the archaeology about node: scheme support and pre-15 unhandled-rejection semantics goes with the import it explained. Add the missing contract test: every generated hook command carries the probe, the notice appears exactly where a harness can render it (Claude and Codex, project and plugin), and the expected floor is read from package.json engines rather than repeated by hand. Verified against a fake pre-22 node, no node, and a real node: one notice then the marker holds it silent, exit 0 in every failure shape, and the hook's own exit code still passes through on a supported runtime. Co-Authored-By: Claude Fable 5 (via Cursor) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Abdul WahabandClaude Opus 5
|
86cdf528c5 |
Probe the import the hook actually uses, and fail closed on rejection
Greptile flagged that the probe does not enforce the Node 22 engines floor.
Two parts to that, and they land differently.
The real defect is narrower and worse than stated: the hook closure imports
`node:fs`, `node:os`, `node:path` and `node:url`, and the `node:` scheme needs
14.18, so a bare `import('fs')` probe passed on 12 and 13 and those runtimes
then died on the real import, which is the banner this branch exists to remove.
Probing `node:fs` closes that. The added `.catch(()=>process.exit(1))` is load
bearing rather than tidiness: before Node 15 an unhandled rejection is only a
warning and the process still exits 0, so a rejected probe would have read as a
pass on exactly the versions in question.
Not enforcing 22 is deliberate and stays. The probe asks whether this runtime
can load our code, not whether it is a supported one, so a 14.18-to-21 runtime
that works today keeps working rather than being silently switched off. The
notice names 22 because that is the version worth installing, and it only ever
reaches someone whose runtime already failed the probe, so no user is shown a
threshold that contradicts what ran.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Abdul WahabandClaude Opus 5
|
4f999ceff8 |
Give Codex the notice too; its hook reference documents systemMessage
Commit
|
||
|
Abdul WahabandClaude Opus 5
|
0c19098754 |
Guard the remaining harness manifests against a dead node runtime
Bugbot caught the Codex plugin builder still invoking node directly, and the same reasoning covers GitHub Copilot and Grok Build: all three shipped the exact failure this branch exists to stop, and sat visibly inconsistent with their guarded siblings. Route them through guardedNode with no notice, matching Codex and Cursor. GitHub gains a second property from it: outside a git repository `$(git rev-parse --show-toplevel)` expands to nothing, so the old command handed node a path that could not exist and failed the turn. The file test now short-circuits that to exit 0. Every builder carries the probe; only the two Claude manifests carry the notice, which is the only harness whose response shape is confirmed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Abdul WahabandClaude Opus 5
|
8397d532b9 |
Keep the unsupported-node notice to the harness that can render it
`systemMessage` on stdout is a Claude Code contract. The shared guard was emitting it for Codex and Cursor too, where what a harness does with stdout it did not ask for is unconfirmed, and a Cursor preToolUse hook printing an unexpected JSON object is the wrong thing to guess about. Pass the notice in per harness instead of baking it into the guard. Claude manifests opt in; Codex and Cursor take the runtime probe alone, so an unsupported runtime stays as quiet there as it was before the probe existed. Giving them their own shape later is one more argument at the call site. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Abdul WahabandClaude Opus 5
|
0db59088ff |
Stop the design hook erroring on a node too old for ESM
The hook command invokes bare `node`. When that node predates ESM, `hook.mjs` dies while it is still being parsed, before the script's own always-exit-0 contract can run, so node exits 1 and the harness reports a hook error on every Stop and every edit. Probe the runtime in the command string before invoking the hook, and route the Claude plugin manifest through the guard that already covered the project-local manifests. On probe failure the command exits 0 and emits a one-time `systemMessage` naming the two fixes available to the user, since nothing written in ESM can report this condition. Fixes #410. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul BakausandClaude Fable 5
|
916b0a1fdf |
Generate degraded-mode fallback references from the subagent definitions
Harnesses with no subagent capability now run each role inline from the
same single source. The build emits reference/degraded/<role>.md for every
agent in skill/agents/ (role name is the agent name minus the impeccable-
prefix), stripping frontmatter and prepending the inline-substitution
preamble. These pass through the same provider-block compilation and
placeholder replacement as ordinary reference files, so <codex> blocks and
{{placeholders}} resolve per target, and they land in the committed harness
dirs on build:release like every reference file.
Repoint the three capability-first fallback sites in the prose at the
generated files: new-work.md reviewer and documenter fallbacks, and
visualize.md asset-producer fallback.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Paul BakausandClaude Fable 5
|
bcf354cd0c |
Fix Codex hook path so .codex-directory installs run the detector
The committed .codex/hooks.json hardcoded .agents/skills/impeccable/scripts/
hook.mjs. On a .codex-directory install the skill payload lives at .codex/
skills/..., so the guarded command ([ ! -f X ] || node X) found no file and
silently no-opped, leaving the design detector dead for those users.
Derive the hook payload path from the emitting provider's own configDir rather
than hardcoding .agents:
- buildCodexHooksManifest(skillDir) now builds `${skillDir}/skills/impeccable/
scripts/hook.mjs`; hooksJsonFor threads each provider's configDir through. The
Codex provider (configDir .codex) emits .codex/skills; the root sync and the
self-consistent dist/codex bundle both point at their own payload.
- CLI installer: project-scope hook rewriting now derives the provider's own
project-relative path instead of preserving the bundle token. The Codex bundle
ships a .codex/skills command, but the CLI lays the skill at .agents/skills, so
the installed .codex/hooks.json is rewritten to .agents/skills (Claude keeps
its ${CLAUDE_PROJECT_DIR} token; global installs keep the absolute rewrite).
Per-provider hook payload path after the fix:
Emission hook path
dist/codex/.codex/hooks.json .codex/skills/impeccable/scripts/hook.mjs
root .codex/hooks.json (build sync) .codex/skills/impeccable/scripts/hook.mjs
CLI .agents (codex) project install .agents/skills/impeccable/scripts/hook.mjs
CLI .agents (codex) global install <home>/.agents/skills/.../hook.mjs (abs)
.claude / .cursor unchanged
Tests: extended hook-build (codex-dir -> .codex/skills, agents-dir -> .agents/
skills) and skills-cli (bundle ships .codex/skills, install rewrites to .agents/
skills). Regenerated tracked .codex/hooks.json via build:release.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Paul BakausandClaude Fable 5
|
eda81f0937 |
Release prep: skill v4.0.1
Bump plugin + marketplace to 4.0.1 and sync the regenerated provider output: the guarded hook commands from issue #399 (a missing hook file exits 0 instead of crashing every turn of a user-level install), the canon standing exit, the visualize flow, the two shipped subagents, and the interactive-spine fixes from today's live testing. Detector count validates at 59 with undersized-ui-text. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Paul Bakaus
|
06d21dea7d |
Add first-class Grok Build harness support
Emit .grok skills, agents, and PostToolUse/Stop hooks; wire the CLI installer and downloads; fix the plugin install path to #plugin; and document Grok in HARNESSES.md and README. AI assistance: written with Grok Build. |
||
|
Paul BakausandClaude
|
d7d10277d1 |
Merge main into oneshot-v4, keeping the service layer split out
main still carries the site, so every `site/` path resolves to deleted. `tests/docs-integrity.test.js` goes with it (it imports the site's demo renderer), and `package.json` keeps main's `@anthropic-ai/sdk` bump while dropping `@google/genai` and `@paper-design/shaders`, which nothing in the product layer imports. Real code merges: - hook-lib: main's #391 cache fix (sync the remembered set to the live scan so fixed findings stop being named and a reintroduced one fires again) now runs on the immediate tier rather than the whole filtered set. Remembering a deferred finding the per-edit pass never reported would let the Stop deep pass dedupe it away. main's `maxFileBytes` ceiling, `cleanAcked` once-per-file ack, and template-extensions re-export all land alongside the tiering work. - live-browser: main's `hasParams` gate on the Tune badge, keeping this branch's `C.ink` badge text so it stays legible on kinpaku gold. - detect-text: both the block-level codex-grid-background scan and main's inset-stripe CSS check. - test-suites: union of both trigger sets and file lists, minus the site-only entries (`shiki-theme`, `docs-integrity`). - Two hook tests moved off deferred-tier rules (`overused-font`, `side-tab`) onto immediate-tier ones. They assert cache bookkeeping, which the per-edit pass only reaches for the immediate tier. Also drops the site waivers from `.impeccable/config.json` and stops `build:browser` recreating a stray `site/` tree just to write a bundle the other repo builds itself. Co-Authored-By: Claude <noreply@anthropic.com> |
||
|
Paul BakausandClaude Fable 5
|
90f9eeb99b |
Split service layer into private impeccable-site repo
The public repo keeps the OSS promise surface: skill, CLI, extension, tests, and the provider build. The site, labs, concept/composition catalogs, image pipeline, Cloudflare functions, and authoring guide move to pbakaus/impeccable-site. concept-seed tests run against a synthetic fixture catalog; the plugin icon and skill categories moved in-repo; build validation narrows to README prose and non-site counts; release notes read from a sibling impeccable-site checkout. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
 
|
83c37e453a |
feat: add Mistral Vibe harness support (#373)
* feat: add Mistral Vibe harness support
Mistral Vibe is Mistral AI's open-source CLI coding assistant that ships an
Agent Skills system at .vibe/skills/{name}/SKILL.md with slash-command
invocation, mapping cleanly onto the existing transformer pipeline. Adds
Vibe as a 14th first-class harness:
- PROVIDER_PLACEHOLDERS entry in scripts/lib/utils.js (model, config_file
= AGENTS.md, ask_instruction, command_prefix) mirroring the Qoder shape.
- PROVIDERS entry in scripts/lib/transformers/providers.js with configDir
.vibe and frontmatterFields user-invocable, license, compatibility,
metadata, allowed-tools (Vibe docs do not document argument-hint).
- transformVibe named export in scripts/lib/transformers/index.js for
test-spy parity.
- vibe added to FILE_DOWNLOAD_PROVIDER_CONFIG_DIRS so the download
endpoint accepts /api/download/skill/vibe/* and resolves to
dist/vibe/.vibe/.
- .vibe added to PROVIDER_DIRS, PROVIDER_ALIASES, PROVIDER_DISPLAY,
PROVIDER_INPUT_ORDER, GLOBAL_HARNESS_HINTS, and the normalizeForHash
provider regex in cli/bin/commands/skills.mjs so the CLI detects
existing Vibe installs.
- docs/HARNESSES.md updated: official docs row, frontmatter support
column, directory structure row, and Last verified date bumped.
- docs/DEVELOP.md, README.md (install instructions, providers list,
Supported Tools), .github issue/PR templates, sync-generated-output
workflow, and AGENTS.md extended with Vibe.
The dynamic providers.test.js loop picks up Vibe automatically; all 160
provider tests pass including the 14 new Mistral Vibe cases. The build
regenerates dist/vibe/.vibe/skills/impeccable/ with correct frontmatter
and .vibe-substituted script paths. Generated harness output is left
unstaged per the source-first policy; the sync-generated-output workflow
(now listing .vibe) will commit it back to main after merge.
Co-Authored-By: Mistral Vibe <noreply@mistral.ai>
* Address review on Vibe harness support
Verified the Vibe claims against the docs and the mistralai/mistral-vibe
source, then tightened what the tables say.
- model placeholder: 'Mistral', not 'the model'. Vibe is Mistral's own
first-party CLI, so it belongs with gemini -> Gemini and codex -> GPT
rather than with the provider-agnostic harnesses. SKILL.src.md's one
use of {{model}} now renders 'Mistral is capable of extraordinary
work.' instead of a lowercase 'the model'.
- Docs links point at the skills page, not the product overview, matching
every other row in both tables.
- disable-model-invocation is No, not TBD. The field appears nowhere in
Vibe's source; unknown frontmatter keys are silently ignored.
- Split the directory row into project and global scopes the way the Pi
row already does. Vibe reads .vibe/skills/ and .agents/skills/ at the
project level and ~/.vibe/skills/ and ~/.agents/skills/ globally; the
global .agents dir was missing, and project .agents/skills/ was sitting
in the global column. Sources: vibe/core/paths/_local_config_files.py
and vibe/core/config/harness_files/_paths.py.
- Restored 'Last verified' to 2026-04-28 and dated the Vibe row on its
own. Only that row was checked, and this file warns against trusting
stale claims, so a blanket re-date made ten other rows look fresher
than they are.
AI assistance: written with Claude Code.
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Mistral Vibe <noreply@mistral.ai>
Co-authored-by: Paul Bakaus <paul.bakaus@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
||
|
|
d0ac67c6e9 |
Live: polling rework, source locks, preflight scaffolding (#381)
* Improve Live polling responsiveness and reliability Restore foreground/background polling as the primary harness architecture, add progressive publication and framework-safe previews, and harden quality and regression coverage. The experimental app-server runtime is intentionally excluded.\n\nPrepared with AI assistance under maintainer direction. * Fix source-safety, detector, and lock defects in Live polling work Addresses the review findings on #371, plus several the bots did not catch. All fixes have regression coverage that fails on the prior code. Source corruption: - Vue accept dropped valueless root attrs (disabled, v-cloak) and, worse, rewrote @click="x" as a literal click="x" DOM attribute, because the attr parser was name-anchored and skipped the sigil. Tokenize the whole Vue attr grammar and normalize shorthands so accept round-trips directives. - --variant was interpolated unescaped into a RegExp, so --variant '.*' matched the original block first and reported a successful accept while silently restoring the original. Validate against the digits pattern the browser and the /events schema already enforce. - --id reached path.join unvalidated, so --id ../../../../etc/evil wrote and read receipts outside the project. Hoist the existing safeSessionId check into impeccable-paths and apply it at every id-to-path sink. Accept/lock correctness: - Plain HTML/JSX accept and discard did not catch SOURCE_LOCKED, so contention exited non-zero with empty stdout and the agent got no JSON to retry on. - Lock staleness was mtime-only and never read the pid it records: a holder whose critical section outran 60s had its live lock swept, admitting a second writer to the same file, while a crashed holder blocked accepts for a full 60s. Decide staleness by owner liveness, and release only our own lock. Detector: - isNeutralColor only parses computed color forms, so routing authored CSS through it reported inset 4px 0 0 #000 / black / #e5e7eb as chromatic side-tab stripes. Add an authored-color neutrality test covering hex and named neutrals; the fixture had no literal-color cases at all. - Rule line numbers were off by one for every rule after the first, and commented-out CSS was scanned as live rules. Server: - An error reply carries no sourceEventType, and inferSourceEventType returned undefined, which acknowledgePendingEvent treats as a wildcard: a stale generate worker's failure consumed the user's queued Accept, which then reached no agent and left the browser in SAVING forever. - The generate preflight spawned live-wrap.mjs synchronously inside the request handler, freezing the single-threaded server for the whole scaffold (~7.6s measured on this repo, 15s ceiling) and stalling Accept/Discard/SSE. Make it async, claiming the lease before the first await so no event double-delivers. - Every browser checkpoint was echoed back as variant_progress, so a Tune slider drag remounted the preview under the user's cursor and latched the *_reviewable phases from the wrong trigger. Gate on the reason. Cleanup: - Collapse four divergent benchmark argv parsers into scripts/lib/cli-args.mjs. Three silently misread flags: --iterations 20 benchmarked 5, --agent llm ran the fake agent, --median-target=0.4 used the default threshold. - Drop a snapshot cache this branch made write-only (it grew per session for the server's lifetime and was never read), a dead exported reconcile helper, and the unused deferReply branch. Prepared with AI assistance under maintainer direction. Co-Authored-By: Claude <noreply@anthropic.com> * Route the last two benchmark scripts through the shared argv parser Follow-up on review feedback. The previous commit consolidated four of the six Live benchmark parsers and left these two on their own hand-rolled `arg()`, which was the inconsistency the first pass was meant to remove. - benchmark-live-control.mjs and benchmark-live-init.mjs parsed --iterations with Number(), so a non-numeric value became NaN and `index < NaN` ran the benchmark zero times before failing on the metrics file. They also accepted only the space-separated form, so --iterations=20 silently measured the default. Both now use parseArgs + positiveIntFlag, which throws on a value that was clearly meant as a number. - benchmark-live-control.mjs read the metrics file with no handling for the case where the run produced nothing: a missing file surfaced as a raw ENOENT stack and a malformed line as a bare SyntaxError. Report both with a diagnostic naming the file and the env var that populates it. - summarize() now reports a `samples` count and nulls instead of letting percentile() read past an empty array, where the NaN serialized to null and a report of nothing measured looked like a real measurement. Prepared with AI assistance under maintainer direction. Co-Authored-By: Claude <noreply@anthropic.com> * Stop telling users a busy agent is disconnected The agent-poll indicator tracks whether a poll is parked, which is the right signal for "can steering reach the agent right now" and is why the flag itself is left alone. But it goes quiet for two different reasons, and both got the same copy: "Agent disconnected - run live-poll.mjs to connect". Under the one-shot foreground polling that live.md calls the primary contract, no poll is parked while the agent works, so the second reason is every normal generation. For its whole duration the bar told the user a healthy session was broken and advised them to start a poll loop that was already running. Pick the copy from the live state, which the browser already tracks: GENERATING and SAVING mean the agent holds work it was handed, so say it is working. Every other state with no parked poll keeps the original, actionable wording. The aria-label carries the same distinction, since the tooltip is mouse-only. The text is derived at read time rather than cached, because the live state moves between the 5s status polls and a finished generation would otherwise keep reading "Agent is working" until the next one landed. Deriving it also keeps the read out of setLiveState, which runs long before agentPollingConnected's declaration and would hit its temporal dead zone. Prepared with AI assistance under maintainer direction. Co-Authored-By: Claude <noreply@anthropic.com> * Scope design-system-font-size off the injected live overlay live-browser.js builds a self-contained UI that renders over arbitrary host pages, so its inline type scale is deliberately independent of DESIGN.md, which documents the impeccable website's ramp. The rule fired 32 times there and is the only rule that fires on that file. Suppress it as a file-scoped value wildcard rather than via ignoreFiles: an ignoreFiles glob would silence every rule for the file, and the overlay is real user-facing chrome where a future contrast or side-tab finding should still be heard. Scoped to this one file, so the rule keeps working everywhere else. Written by hand because hook-admin's ignore-value cannot emit the `files` array that detector.ignoreValues supports and existing entries already use. Prepared with AI assistance under maintainer direction. Co-Authored-By: Claude <noreply@anthropic.com> * Let hooks ignore-value scope a rule to files, and stop churning the config Fallout from suppressing the overlay's font-size findings: the narrowest exception detector.ignoreValues supports was unreachable from the path the hook tells the model to use, so the guidance steered to the blunt instrument instead. - hook-admin's ignore-value now takes --file / --files / --file= / --files=, matching `impeccable ignores add-value`, which already had them. Without it the only file-scoped option was ignore-file, which silences every rule for a path permanently, including rules not yet written. - A bare wildcard value is now refused with a message pointing at either --file or ignore-rule. Previously `ignore-value <rule> "*"` quietly wrote a project-wide suppression from a single file's finding. - ignore-value keyed entries on rule+value only, so a second scope for the same rule overwrote the first instead of coexisting. Key on the file scope too. - An unknown flag folded into the value: `ignore-value overused-font Inter --shard` stored "inter --shard", matched nothing, and reported success. Reject it, as the sibling command does. Config churn: normalizeIgnoreValueEntries runs on every write and emitted keys as rule, value, files, reason, createdAt while the config on disk uses createdAt before reason. Any edit therefore rewrote every untouched entry (35 churned lines for a one-line change). Pin the canonical order in both copies of the normalizer and in ignores.mjs, and add a test that the two copies cannot drift apart. Also point the hook's own footer and reference/hooks.md at the file-scoped form first, and say plainly what ignore-file costs. Prepared with AI assistance under maintainer direction. Co-Authored-By: Claude <noreply@anthropic.com> * Correct the prose-gate docs and write down the no-bump-in-a-PR rule CLAUDE.md said the prose validator "deliberately skips skill/", which is only half true and cost a build failure this week: validateProse skips it, but validateSkillProse then scans skill/**/*.md and fails the build on em dashes plus the phrases with no technical reading. Document both gates, which files each one reads, and the line that actually matters in practice: an em dash in skill/reference/*.md fails the build, one in a skill/scripts/*.mjs comment does not. Each claim was checked against a real `bun run build`. Also record that feature PRs do not bump manifest versions or add changelog entries. It was not written down anywhere: not CLAUDE.md, not AGENTS.md, not the PR template. CLAUDE.md's "Bump when: CLI code changes" reads as an instruction to bump inside the PR that touches cli/, so say plainly that it names which component a change belongs to rather than when to edit the manifest. Put the rule in AGENTS.md too. That is the guide the agents opening PRs here actually read, so a rule about PR hygiene living only in CLAUDE.md would not reach them. Prepared with AI assistance under maintainer direction. Co-Authored-By: Claude <noreply@anthropic.com> * Bring Live progressive delivery and the generator subagent to Claude Code Almost none of this branch's Live work was actually Codex-specific. The publisher, the fences, the source locks and the browser's partial-arrival UI are plain node and DOM with zero provider references, and the progressive E2E already passes on five frameworks driven by a non-Codex agent. The Codex-only part was policy prose and one frontmatter line, so Claude Code shipped the progressive browser UI it could never trigger. Progressive delivery, Codex and Claude Code: - Add a `live-progressive` capability tag and opt codex, agents, and claude-code in. A provider block takes one tag, so naming harnesses would have meant duplicating the recipe per tag; a capability reads better than a provider list anyway. Cursor and everyone else keep the atomic path until their poll loop is known not to stall on the extra publish calls. - Claude Code publishes variant 1 as soon as it validates rather than waiting to write the whole trio in one edit. Nothing about the arrival path needed changing: the publisher writes, framework HMR pushes, and the browser's MutationObserver counts variants. The parent conversation was never in that path, which is why Claude Code's lack of subagent progress streaming does not matter here. Generator subagent: - Drop `providers: codex` from impeccable-live-generator. The build already maps its frontmatter correctly for Claude Code, and impeccable-manual-edit-applier has shipped to .claude/agents/ this way all along. - The reason differs per harness, so the reference says so: Codex delegates to unblock a foreground poll, Claude Code delegates to keep a long session's screenshots and variant CSS out of the main context. Follows the existing manual-edit-applier convention: both agent names, and an inline fallback when native subagents are unavailable. Fixes found on the way: - The two publish commands hardcoded `.agents/skills/impeccable/scripts/` while the other thirteen commands in live.md use {{scripts_path}}. Correct only for the Codex repo-skills bundle; it would have pointed Claude Code at a directory its install never creates. The shipped .codex variant was already internally inconsistent. Now covered by a test. - `--agent=codex` resolved to the canned fake agent, because the flag parsed as `x === 'llm' ? 'llm' : 'fake'`. The private evals Live runner passes exactly that, so a real-harness run would have scored deterministic stub variants and reported them as Codex output. Unknown values for --agent, --scenario and --delivery now fail loudly. - live-reference tests now compile with each provider's real providerTags instead of hand-written lists, so a providers.js misconfiguration fails in tests rather than shipping. Verified: progressive E2E green on vite8-react-plain against a real Vite server and Chromium; every provider variant's publish and poll paths now agree; Cursor and Gemini still compile to atomic only. Prepared with AI assistance under maintainer direction. Co-Authored-By: Claude <noreply@anthropic.com> * Fix inset-order detection, the unlocked artifact discard, and stray boolean flags Three of the four open review findings. The fourth is declined below. - The inset-stripe scan only matched layers starting with `inset`, but the keyword is order-independent: `box-shadow: 4px 0 0 var(--brand-accent) inset` paints the same stripe and was silently missed. Strip the keyword wherever it sits, but only as a standalone token, so a color like var(--inset-accent) is not mangled into `var(-- -accent)` and quietly reclassified as neutral. The fixture now covers both orders plus that token, and a trailing-inset neutral still passes. - The source-artifact discard deleted the preview without the source lock, unlike every other discard path. Take the lock. Narrower than reported, though: the server journals `discard_requested` as a fenced phase before live-accept runs and the publisher checks it three times, so a publish could never land on a discarded session. What this actually prevents is deleting the artifact under a publisher mid-critical-section, turning a clean stale_generation_epoch into an ENOENT crash. - benchmark-live-providers.mjs still compared `--headed` and `--skip-cleanup-control` against a boolean sentinel, so the `=true` spelling silently did nothing. My gap: I introduced boolFlag and converted benchmark-live.mjs but not this one. skipCleanupControl is now read once rather than twice, so the two call sites cannot drift. Declined: tightening the selector guard that skips `active` / `current` / `selected` tokens. It does cause false negatives on names like `.selected-feature`, but the rule's contract makes selection and focus indicators its one exception, and `.active-tab` / `.current-step` / `.selected-row` are syntactically identical to `.selected-feature`. No regex separates them, so tightening the guard trades missed stripes for false positives on exactly the case the rule exempts. The conservative skip is the intended behavior. Prepared with AI assistance under maintainer direction. Co-Authored-By: Claude <noreply@anthropic.com> * Classify failed accepts as errors, and fix parallel lane race/all misuse Two of the three new findings, plus the bug that chasing them exposed in my own earlier fix. The third is mitigated rather than broken; details below. Failed accepts reported success: live/completion.mjs only classifies a result as `error` when it carries `mode: 'error'`. Everything else unhandled falls through to `agent_done` with an ok ack, which is deliberate for the documented fallback paths (two tests pin it) but wrong for a real failure. So `accept_receipt_conflict` reported success, and reference/live.md's `handled: false` without `mode` bullet told the agent to "read file, find markers, edit" — hand-applying a second accept on top of the one the receipt already recorded. The same hole swallowed `source_locked`, which is mine: the earlier commit made lock contention return clean JSON so the agent could retry, but the classifier turned that failure into agent_done/ok, so the accept was dequeued and silently lost. Mark genuine failures with `mode: 'error'` through one `operationFailure` helper, and give live.md a `mode: "error"` bullet with per-error guidance: retry the same command on `source_locked`, never hand-edit, and on a receipt conflict report what the session actually resolved to. The deliberate fallback and markers-not-found handoffs stay untouched. parallel-compact lane orchestration: `Promise.race` settles on the first *settlement*, so one lane failing fast rejected the whole first-variant step while two lanes were still on their way to succeeding. `Promise.any` now takes the first success and only a total wipeout is fatal, reporting every lane's reason. The tail step's `Promise.all` surfaced a raw lane error non-deterministically; `Promise.allSettled` now reports how many lanes failed and why. Added a `requestImpl` seam so lane orchestration is testable without a provider key. Not a defect: the browser releasing Accept before the source write. That is the intended optimistic design, and it is safe because poll-lanes ranks accept at priority 0 against generate at 2, so a queued accept is always leased before a generate the user queues afterwards, even if the generate arrived first. Its source write lands inside the poll script before the next generate preflights. That invariant is load-bearing and had no tests at all; poll-lanes.mjs now has a suite covering it plus lease and type filtering. Prepared with AI assistance under maintainer direction. Co-Authored-By: Claude <noreply@anthropic.com> * Finish the failed-accept classification my last commit only half did All three new findings are the same root cause, and it is my incomplete fix: operationFailure only covered results built from a *thrown* error. Two paths it missed: - Two catches wrote the failure result as a multi-line literal, so the single-line replace skipped them. The Vue accept catch was still bare, exactly as reported; the Svelte one too, though its failures happened to be caught by completion.mjs's Svelte-only special case. - The accept implementations also *return* `{handled: false, error}` for their own checks (variant missing, template empty, original text ambiguous). Those never throw, so no catch ran and no `mode` was set. Both layers now agree, because each is reachable on its own: - live-accept marks any unhandled preview-path result via markPreviewFailure, keyed on `previewMode` — a clean discriminator, since only the preview branches set it and a plain wrapper never does. This is what the agent reads: reference/live.md routes on `mode`, so without it the agent was told "read file, find markers, edit" for a preview that has no markers in source. - completion.mjs replaces its arbitrary svelte-component special case with the set of preview modes whose variants live outside the user's source. That case existed for precisely this reason; Vue and source-artifact were simply never added, so the identical failure on those paths acknowledged as success. The plain wrapper keeps its manual handoff, which is the one shape with editable markers in source. Both deliberate handoffs (mode: 'fallback' and markers not found) still classify as agent_done, now pinned by a test so the generalization cannot swallow them. Prepared with AI assistance under maintainer direction. Co-Authored-By: Claude <noreply@anthropic.com> * Stop the progressive benchmark agent inventing a second variant on count:1 `Math.max(1, event.count - 1)` floored the tail request at one variant, so a one-variant request fetched a second direction and assembled two. Ask for `count - 1` and return the first variant untouched when there is no tail. Latent rather than live: the only caller hardcodes `count: 3`. The reason it is worth fixing is the caller inconsistency it exposed. tests/live-e2e/agent.mjs gates its split-progressive path on `event.count > 1`; benchmark-live-providers.mjs had no such guard, so it would have run the tail for a one-variant request, and the parallel strategy would have assembled its three fixed lanes regardless of what was asked for. Guard the caller the same way. Prepared with AI assistance under maintainer direction. Co-Authored-By: Claude <noreply@anthropic.com> * Drop the live generator subagent; fix the artifact decoy that broke accept The first real Claude Code Live run failed, and the subagent was not the cause. Root cause: progressive publication stages each revision as `.impeccable/live/artifacts/<id>-r<n>.<source-ext>`, nothing ever deleted them, and findSessionFile's walker skipped only node_modules/.git/dist/build. It searches src, app, pages, ... then `.`; a project whose source is not under one of those (this repo's own site lives in site/pages/) falls through to the `.` walk, where dot-directories sort before letters. So accept found the artifact instead of the real file. Two outcomes, both reproduced: where isGeneratedFile returns true it declines with mode: 'fallback' (what the run hit, after which the agent hand-carbonized several hundred lines across three stylesheets, including unrequested drive-by edits); where it returns false, accept writes the variant into the throwaway artifact and reports handled: true while real source never changes. The E2E suite could not have caught this. Every fixture puts source under `src/`, which is searched before the `.` walk can reach `.impeccable`. Five framework fixtures and three progressive scenarios pass because of fixture layout, not because the path works. I read that as evidence and shouldn't have. - Never search `.impeccable`: it is Impeccable's own state, never project source. - Retire a session's staged artifacts on accept/discard, so they cannot outlive the session and become a decoy for anything else that walks the tree. - Regression tests use a site/pages layout with artifacts present. All three fail against the previous code. Generator subagent removed, on both harnesses: The parent must hand-compress the design system into the handoff, and compression is lossy. Measured on the real run: a 6,826-char handoff carrying exactly one token reference, after the parent had itself read kinpaku-tokens.css. The subagent then spent 3 of its first 9 turns hunting DESIGN.md, gave up, and emitted 0 var(--token) uses and 22 raw oklch literals — violating its own spec's "Never invent raw colors when tokens exist" — including a 1:1 gold-on-gold contrast bug. Isolation is not a benefit here; knowing the design system is the job. Generation stays in the main thread, which already holds the tokens and writes them from the first byte, so carbonize is a move rather than a translation. Copy edits keep their subagent: applying a known set of ops to a named file is self-contained, so an isolated context costs nothing. That is the line. Progressive delivery stays for Codex and Claude Code, main-thread driven. Claude Code keeps the full benefit because its poll is a background task. Codex's poll blocks the foreground, so with no subagent the user sees variant 1 early via HMR but cannot accept it until the trio finishes; that is the cost of the simplification and it is worth naming. Prepared with AI assistance under maintainer direction. Co-Authored-By: Claude <noreply@anthropic.com> * Rip out the dead isolated-preview mode and the private repo's job Comparing this branch's live against main's turned up two whole features that never made sense here. -2,466 lines. 1. The isolated source-artifact preview was never switched on. `scaffoldSourceArtifactSession` is only reachable via live-wrap's `--isolated`, and nothing passes it: not the server's preflight, not live.md, nothing. Proved it end-to-end — the default wrap writes markers straight into real source and creates no previews/ session. So the mode was wired through three modules, carried its own accept/discard branches, browser branches, server metadata resolution, preview-mode classifier entry, and test suites, and none of it could run. Worse, live.md documented it as the active path and told the agent "The true source is only the publisher's hash fence and must remain byte-identical until Accept." That is false: the wrapper lands in source at scaffold time and each revision rewrites it. An agent following that sentence believes source is protected when it isn't, and the leftover artifacts are what made accept resolve the wrong file in the first real run. live.md now describes what actually happens, including that markers are visible in source until Accept or Discard. Removed: source-artifact.mjs, --isolated, the preflight's isolated option, the accept/discard branches, four dead browser branches, the server's previews/ resolution, the classifier entry, and their tests. Kept the previews/ gitignore pattern: an ignore line for a directory that cannot exist is free, and a test pins it. 2. Quality judging belongs to the private evals repo, which says so. runner/live/README.md there is explicit: the public repo owns protocol correctness, framework coverage, timing, source commit, recovery, and a rubric-free evidence bundle; the private repo owns the task corpus, baselines, comparative judges, and release-quality decisions — "Do not add quality rubrics, competitor comparisons, or broad fixture corpora to the public Live benchmark." This branch added exactly those: an LLM judge scoring 1-10 on "off-brand, generic-AI" (live-rendered-quality.mjs, judge-live-rendered.mjs), a cross-provider comparison with a BRAND_CONTRACT rubric (live-provider-benchmark .mjs, benchmark-live-providers.mjs), and a brand-fidelity fixture corpus. All removed, with bench:live:providers and their suite entries. Also removed tests/framework-fixtures/README.md's "External quality-eval fixtures" section: it documented a bench:live workflow using --fixture-dir, --agent=codex, --action and --evidence-bundle, none of which benchmark-live.mjs implements, plus an evidenceCapture block nothing reads. Kept: timing benchmarks (the public repo's half of that boundary), progressive publication, the source lock, poll lanes, and Nuxt/Vue component previews. Coverage note: deleting the isolated suites took the only tests for `source_locked` classification with them, so the plain wrapper path — now the only non-component preview — gets equivalent accept and discard coverage. Both new tests fail if mode:'error' is removed. Prepared with AI assistance under maintainer direction. Co-Authored-By: Claude <noreply@anthropic.com> * Flag inset stripes written with the two-length box-shadow form box-shadow takes <length>{2,4}: only the two offsets are required, so `inset 4px 0 red` is valid and paints the same single-edge stripe as `inset 4px 0 0 red`. The scan demanded a third length, so the short form was silently missed. Blur and spread now default to 0 when omitted, which is exactly the stripe shape the rule looks for. The neutral-color and blur/spread exclusions still hold: `inset 4px 0 #000` and `inset 4px 0 5px var(--brand-accent)` both pass. Fixture covers both orders of the short form plus those two exclusions, and fails against the previous regex. Third false negative found in this rule (after trailing `inset` and literal neutral colors), all from the same cause: the scan was written against one spelling of the syntax rather than the grammar. Prepared with AI assistance under maintainer direction. Co-Authored-By: Claude <noreply@anthropic.com> * Live: polling rework, source locks, preflight scaffolding, Vue previews Carved out of #371, minus progressive publication. Everything here works against real project source the way main's Live already does: the agent writes variants into the file the browser loaded, HMR fires, Accept promotes and carbonizes. Nothing is staged anywhere. Poll lanes. Events now carry an explicit priority: accept/discard/exit ahead of manual_edit_apply/steer/carbonize_cleanup ahead of generate. A long generate can no longer sit in front of the Accept the user just clicked. leaseEvent claims its lease before awaiting, so a slow prepare cannot hand the same event to two pollers. Source locks. A per-file mutex around every accept and discard path, keyed on a digest of the absolute path. Staleness is decided by owner-pid liveness rather than mtime, so a wedged lock clears when its owner dies instead of after an arbitrary timeout, and a slow-but-live accept is never stolen from. Only the owning process can release a lock. Preflight scaffolding. The server runs live-wrap (or live-insert) before the poll returns and hands the result back as event.scaffold. That walk is measured at ~7.6s on a large repo; moving it off the agent's critical path removes a deterministic tool round trip without touching the generated design. Falls back cleanly to the agent running the helper itself. Vue previews. previewMode: "vue-component" for Nuxt/Vue targets, matching the existing Svelte component path: variants compile as real SFCs from a dev-only directory so the route is never rewritten during generation, and Vite mounts them without invalidating page state. Accept is the only route write. Includes a Vue attr tokenizer that normalizes shorthand bindings (@x, :x, #x) to their canonical forms. Accept hardening. Every thrown failure now returns mode: 'error' rather than an ambiguous unhandled result, so a real failure is never classified as a deliberate manual handoff and silently dropped. The marker search skips node_modules/.git/dist/build/.impeccable. Shared CLI arg parsing extracted to scripts/lib/cli-args.mjs. Assisted-by: Claude Code * Drop the progressive benchmark, remove dead wrap scaffolding Review fallout from removing progressive publication. The Live benchmark existed to compare atomic against progressive delivery: compareModelBackedReports measures goToFirstVariantMs improvement of one over the other. With progressive gone it measures nothing against nothing. Worse, benchmark-live.mjs still passed `progressive` to bootFixtureSession, which no longer accepts it, so `--delivery progressive` was silently ignored and would have emitted reports labeled progressive that actually ran atomic. Silent wrong data is worse than a crash. It was built for progressive, so it goes with progressive: benchmark-live.mjs, its lib, its test, and the bench:live script. If an atomic latency baseline is wanted later, that is a smaller thing built on purpose. live-wrap.mjs: sourceOriginalLines was assigned and never read. Both found by review bots on #381 (Copilot). Assisted-by: Claude Code * Drop the Vue preview mode; it never reached Svelte's accept path Cursor found that inlineVueComponentAccept never receives paramValues, while the Svelte equivalent uses them in 23 places: Accept on a tuned Vue variant silently persisted the default and threw the user's tuning away. Chasing that corrected something I had asserted the other way round. I said Vue's raw-CSS-append was inherited from the Svelte path. It is not. svelte-component.mjs calls sanitizeAcceptedSvelteCss before writing, which sanitizes the CSS and bakes tuned params into it. vue-component.mjs had no sanitize step at all — it appended the variant's <style scoped> body into whatever style block came last, so a variant could leak CSS site-wide when the last block was global, and brace CSS landed in a lang="sass" block. Both are the same defect: the Vue mode mirrored Svelte's preview path without its accept-side subsystem (bakeParamValuesInCss, sanitizeAcceptedSvelteCss, appendSanitizedCssRule, rewriteAcceptedSvelteSelector, rewriteParamSelectors — roughly 200 lines of CSS rewriting). Both were introduced here, not inherited. A shipped Vue session could leak styles and discard tuning without saying so. So it comes out. The poll lanes, source locks, preflight scaffolding, and accept hardening do not depend on it and are worth landing now. Vue returns when its accept path reaches parity. The nuxt-vite7 fixture goes back to main's plain-wrapper shape. Assisted-by: Claude Code * Stop the lease redelivery test racing the scheduler CI failed `does not drop polled events until the agent acknowledges them` on a commit whose content was byte-identical to one that passed, which is the signature of a flake rather than a regression. The test leased an event for 50ms, then asserted a second poll saw a timeout because the lease was still held. That gave the whole second HTTP round trip a 50ms real-time budget: cross it and the lease expires, the event is redelivered, and the assertion fails for a scheduling hiccup instead of a bookkeeping bug. Locally it passed 6/6; a loaded runner is where it bites. Hold the lease for 1000ms so a round trip cannot cross it, and wait LEASE_MS + 300 before asserting redelivery, so each half has headroom in the direction it asserts. Verified by injecting a 60ms stall before the second poll: the old test fails with exactly the CI message, the new one passes. Assisted-by: Claude Code * Recover live sessions that reload past the generation done broadcast The preflight scaffold write (new in this PR) triggers a framework full-reload — Astro reloads the page for any .astro edit. When the agent's variant write and its done SSE land while the browser is mid-reload, the resumed page misses both the second HMR reload and the done broadcast: it comes back up on the scaffold-only source and waits in GENERATING at 0/N forever, with the finished variants sitting in source. This is the astro-vite7 CI timeout; the failure artifacts show the full sequence (scaffold at 26.319s, done at 26.515s, the new page's browser_resumed checkpoint at 26.653s, DOM still scaffold-only). Three-part fix: - session-store: agent_done now stamps a monotone generationCompletedAt on the snapshot. Browser checkpoints legitimately regress phase and arrivedVariants (a resumed page reports what it sees), so completion needed a field checkpoints cannot un-set. - live-browser: on every SSE (re)connect, compare the session summary's generationCompletedAt against local progress; when behind while GENERATING, pull the finished variants from source (same settle delay as the done handler's HMR-first fallback). Covers both orderings of resumed-checkpoint vs agent_done. Also, the source-fallback empty- wrapper branch no longer tears the session down mid-generation — a scaffold-only wrapper is a legitimate in-flight state, so stay in GENERATING instead of destroying a session the agent is still filling. - live-server: a browser checkpoint reporting generating/behind for a session whose generation already completed re-broadcasts the stored done (idempotent for every other tab), and connected-payload summaries expose generationCompletedAt for the browser-side check. Coverage: live-server unit tests for redelivery, the no-redelivery guard, and marker durability across checkpoint regression; plus a deterministic live-e2e scenario on astro-vite7 that blocks the reloaded page's SSE stream and mocks its HMR websocket dead until after the agent finishes, forcing the missed-broadcast window every run. All new tests fail against the pre-fix code. The e2e harness additionally gains an IMPECCABLE_E2E_ATOMIC_DELAY_MS lever (widens the scaffold-to-write window) and env-gated console/nav tracing (IMPECCABLE_E2E_CONSOLE=1) used to diagnose this. The hypothesis that preflight opens a wrapper-with-no-variants window came from Copilot's review sketch in the follow-up WIP PR; the killing mechanism differs from that sketch (nothing calls recoverEmptyCycling in the CI trace — the session hangs precisely because no code path runs at all), but the window is real and the guard it suggested is folded into the source-fallback fix. Assisted-by: Claude Code Co-Authored-By: Claude Code <noreply@anthropic.com> * Retry a completion-driven source fallback that reads only the scaffold Greptile flagged a hole in the previous commit's empty-scaffold guard: when a `done` has already been delivered, the source fallback gets exactly one read. If that read returns the preflight-only scaffold (a stale source view, or an agent whose write lands in multiple steps), the guard's silent return left the tab in GENERATING with no further event ever coming — the same stuck state the previous commit fixed, reintroduced through a different door. Callers that know generation finished (the done handler's fallback and the SSE-reconnect self-heal) now pass generationCompleted, and an empty read on that path re-reads the source up to 3 times before surfacing recoverEmptyCycling instead of hanging. Mid-generation callers are unchanged and still wait indefinitely — a real agent can legitimately take minutes between scaffold and write, and tearing that down was the original #385 hazard. The missed-done e2e scenario now also serves a captured scaffold-only copy for the first post-reconnect /source read, forcing the retry path every run. Verified failing against the pre-retry code (tab stranded in GENERATING, test timeout) and passing with it. Assisted-by: Claude Code Co-Authored-By: Claude Code <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Paul Bakaus
|
2b1f36c43e |
Add concept world catalog and review workflow
AI-assisted: prepared by Codex at Paul's request. |
||
|
Paul Bakaus
|
bbed6eef08 |
Refresh the Impeccable product experience
Rework the landing page proof, steering demo, feature grid, slop catalog, detector coverage, theming, Live workflow, and responsive behavior.\n\nAI-assisted implementation by OpenAI Codex. |
||
|
Paul Bakaus
|
e7ed663ecd |
Fix target-mode Live evidence capture
Capture below-fold selected targets and each progressive variant reliably for portable evidence bundles. AI-assisted implementation under maintainer direction. |
||
|
Paul Bakaus
|
e0c19eff28 |
Export portable Live evidence bundles
AI-assisted implementation under maintainer direction. |
||
|
Paul Bakaus
|
4c07124198 | Deliver Live variant two independently | ||
|
Paul Bakaus
|
2f619f9bdb | Prove Live benchmark target selection | ||
|
Paul Bakaus
|
d293b803d0 | Calibrate Live rendered quality review | ||
|
Paul Bakaus
|
a274f93c4e |
Add atomic Live benchmark controls
AI-assisted: Codex |
||
|
Paul Bakaus
|
78b1b5878d |
Add rendered Live quality evidence
AI-assisted: Codex |
||
|
Paul Bakaus
|
1e4927d86a |
Prevent duplicate long-running Live turns
Keep short crash-recovery leases without allowing a healthy worker to queue its own generation twice, and surface non-monotonic benchmark journals as errors.\n\nAI-assisted: OpenAI Codex. |
||
|
Paul Bakaus
|
753820f25b |
Show real Codex Live loop evidence
Replace stale startup and synthetic claims with production browser timing, matched architecture comparisons, Accept latency, and honest run counts.\n\nAI-assisted: OpenAI Codex. |
||
|
Paul Bakaus
|
46d4fc4877 |
Measure Accept-to-next-generation latency
Exercise accepting the first progressive variant, immediately preparing another task, and receiving its first result through the independent Codex worker.\n\nAI-assisted: OpenAI Codex. |
||
|
Paul Bakaus
|
10183bd91c |
Measure production Live worker phases
Trace worker pickup and derive generation, validation, and publication latency from the durable Live session journal.\n\nAI-assisted: OpenAI Codex. |
||
|
Paul Bakaus
|
27bbf90ede |
Improve Codex Live architecture benchmark
Compare production-equivalent direct and app-server paths, include annotated UI work, expose first-usable latency, and keep semantic quality gates honest.\n\nAI-assisted: OpenAI Codex. |
||
|
Paul Bakaus
|
27a210b04c |
Add Codex Live architecture benchmark
Compare direct Sol execution with cold and persistent app-server paths using identical full-task quality gates, lifecycle timings, and token metrics.\n\nAI-assisted: OpenAI Codex. |
||
|
Paul Bakaus
|
6d1bd40959 |
Fix Codex quality benchmark gates
Allow valid CSS-only design work and compare JSX component contracts independent of formatting.\n\nAI-assisted: OpenAI Codex. |
||
|
Paul Bakaus
|
92a5589d5d |
Add Codex Live quality benchmark
Benchmark realistic bolder and polish tasks across fast, full-model, and full-context worker profiles with deterministic and independent quality gates.\n\nAI-assisted: OpenAI Codex. |
||
|
Paul Bakaus
|
02b1040280 |
Add Live performance lab and benchmarks
Measure framework and provider latency, enforce fidelity and cleanup gates, and publish reproducible results on the dev-only Live Lab.\n\nAI-assisted: OpenAI Codex. |
||
|
Paul BakausandClaude Fable 5
|
c3aba1e343 |
hooks: two-tier design hook — immediate per-edit rules + full-set Stop deep pass
Eval evidence showed the per-edit PostToolUse stream fires overwhelmingly
on copy-level rules (em-dash-overuse ~97x/session) and measurably makes
models more conservative, while a full-detector pass at completion is what
actually fixes contrast/padding/glow. Split the hook accordingly:
- Per-edit (PostToolUse) now surfaces only IMMEDIATE_TIER_RULES: broken
output (broken-image, text-overflow, clipped-overflow-container,
body-text-viewport-edge), objective contrast/legibility failures
(low-contrast, gray-on-color, tiny-text), single-property mechanical
slop (gradient-text, dark-glow), and design-system drift (the four
design-system-* rules, which compound if left uncorrected). Everything
else defers. Override with hook.perEditRules: "all" in
.impeccable/config.json. Tiering is off for Cursor/Copilot harnesses,
which have no Stop pass wired, so nothing gets silently dropped there.
- Stop deep pass (runStopHook): runs the FULL rule set over every UI file
touched this session (tracked via the existing hook.cache.json session
state; deferred-only edits now mark the file touched), dedupes against
everything already surfaced per-edit, honors ignore-rule/file/value and
inline disables, reuses the [impeccable@1] envelope, and no-ops fast
when no UI files were touched. Emits hookSpecificOutput
{ hookEventName: "Stop", additionalContext } per the Claude Code SDK
Stop contract (conversation continues so the model can act on it).
Second Stop fire is silent - deep-pass findings are remembered.
- Wiring: Stop entries (timeout 30) in plugin/hooks/hooks.json, the
.claude settings + .codex hooks manifests (transformers + hook-admin
repair path). Claude Code and Codex both dispatch a native Stop event;
Cursor's stop hook is inconsistently dispatched (pre-write gate stays)
and Copilot's agentStop/sessionEnd don't inject model context, so
neither gets a Stop entry - documented in reference/hooks.md.
- Tests: tiering split/override/harness gating, Stop dedupe + silent
no-touched-files + ignore machinery + kill switches; existing per-edit
tests moved to immediate-tier rule ids. 181 tests green; smoke-tested
the built dist skill end to end (glow surfaced per-edit, em-dash only
at Stop, second Stop silent).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Paul BakausandGitHub
|
da99645a58 |
Add OpenAI plugin submission bundle (#363)
* Add OpenAI plugin submission bundle Build a Codex-native OpenAI plugin with bundled hooks, public listing metadata, submission guidance, privacy coverage, and regression tests. AI assistance: OpenAI Codex prepared and validated these changes under maintainer direction. * Fix provider script command rendering Replace heuristic rewrites across executable scripts with one explicit provider marker, render pinned shortcuts per target harness, and remove the personal email from the public publisher manifest. Addresses automated review feedback on PR #363. AI assistance: OpenAI Codex prepared and validated these changes under maintainer direction. |
||
|
|
44c27a72af |
Fix Codex plugin hook load failure; bump skill to 3.9.1 (#333)
Codex loads bundled plugin lifecycle hooks from `hooks/hooks.json` using a strict schema that accepts only the top-level `hooks` field. The plugin-packaged manifest carried a top-level `description`, so Codex rejected the whole manifest with `unknown field description, expected hooks` and the post-edit design detector never registered (issue #330). Drop `description` from `buildClaudePluginHooksManifest()` and regenerate `plugin/hooks/hooks.json`. The Claude Code plugin path is unaffected (it only reads the `hooks` object). Add a regression assertion for the plugin artifact and bump the skill version to 3.9.1. Claude-Session: https://claude.ai/code/session_013GTTHY6uHwESUyAgjUEm7x Co-authored-by: Claude <noreply@anthropic.com> |