From fc620b9620f4829beb16e751bf0bf2747e1dbc98 Mon Sep 17 00:00:00 2001 From: Paul Bakaus Date: Fri, 17 Jul 2026 15:43:34 -0700 Subject: [PATCH] Bring Live progressive delivery and the generator subagent to Claude Code Almost none of this branch's Live work was actually Codex-specific. The publisher, the fences, the source locks and the browser's partial-arrival UI are plain node and DOM with zero provider references, and the progressive E2E already passes on five frameworks driven by a non-Codex agent. The Codex-only part was policy prose and one frontmatter line, so Claude Code shipped the progressive browser UI it could never trigger. Progressive delivery, Codex and Claude Code: - Add a `live-progressive` capability tag and opt codex, agents, and claude-code in. A provider block takes one tag, so naming harnesses would have meant duplicating the recipe per tag; a capability reads better than a provider list anyway. Cursor and everyone else keep the atomic path until their poll loop is known not to stall on the extra publish calls. - Claude Code publishes variant 1 as soon as it validates rather than waiting to write the whole trio in one edit. Nothing about the arrival path needed changing: the publisher writes, framework HMR pushes, and the browser's MutationObserver counts variants. The parent conversation was never in that path, which is why Claude Code's lack of subagent progress streaming does not matter here. Generator subagent: - Drop `providers: codex` from impeccable-live-generator. The build already maps its frontmatter correctly for Claude Code, and impeccable-manual-edit-applier has shipped to .claude/agents/ this way all along. - The reason differs per harness, so the reference says so: Codex delegates to unblock a foreground poll, Claude Code delegates to keep a long session's screenshots and variant CSS out of the main context. Follows the existing manual-edit-applier convention: both agent names, and an inline fallback when native subagents are unavailable. Fixes found on the way: - The two publish commands hardcoded `.agents/skills/impeccable/scripts/` while the other thirteen commands in live.md use {{scripts_path}}. Correct only for the Codex repo-skills bundle; it would have pointed Claude Code at a directory its install never creates. The shipped .codex variant was already internally inconsistent. Now covered by a test. - `--agent=codex` resolved to the canned fake agent, because the flag parsed as `x === 'llm' ? 'llm' : 'fake'`. The private evals Live runner passes exactly that, so a real-harness run would have scored deterministic stub variants and reported them as Codex output. Unknown values for --agent, --scenario and --delivery now fail loudly. - live-reference tests now compile with each provider's real providerTags instead of hand-written lists, so a providers.js misconfiguration fails in tests rather than shipping. Verified: progressive E2E green on vite8-react-plain against a real Vite server and Chromium; every provider variant's publish and poll paths now agree; Cursor and Gemini still compile to atomic only. Prepared with AI assistance under maintainer direction. Co-Authored-By: Claude --- scripts/benchmark-live.mjs | 8 +-- scripts/lib/cli-args.mjs | 16 +++++ scripts/lib/transformers/providers.js | 9 ++- scripts/lib/utils.js | 5 ++ skill/agents/impeccable-live-generator.md | 3 +- skill/reference/live.md | 22 +++---- tests/cli-args.test.mjs | 28 ++++++++- tests/live-reference.test.mjs | 76 +++++++++++++++++++---- 8 files changed, 134 insertions(+), 33 deletions(-) diff --git a/scripts/benchmark-live.mjs b/scripts/benchmark-live.mjs index 5e6755622..126626363 100644 --- a/scripts/benchmark-live.mjs +++ b/scripts/benchmark-live.mjs @@ -15,7 +15,7 @@ import { waitForCycling, waitForHandshake, } from '../tests/live-e2e/ui.mjs'; -import { boolFlag, parseArgs, positiveIntFlag } from './lib/cli-args.mjs'; +import { boolFlag, parseArgs, positiveIntFlag, resolveEnum } from './lib/cli-args.mjs'; import { buildInteractionRun, assembleSplitProgressiveOutput, @@ -28,9 +28,9 @@ const ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..'); const args = parseArgs(process.argv.slice(2)); const fixtureName = String(args.fixture || 'vite8-react-plain'); const iterations = positiveIntFlag(args.iterations, 5); -const agentMode = args.agent === 'llm' ? 'llm' : 'fake'; -const scenario = args.scenario === 'annotated' ? 'annotated' : 'plain'; -const delivery = args.delivery === 'progressive' ? 'progressive' : 'atomic'; +const agentMode = resolveEnum(args.agent, ['fake', 'llm'], 'fake', '--agent'); +const scenario = resolveEnum(args.scenario, ['plain', 'annotated'], 'plain', '--scenario'); +const delivery = resolveEnum(args.delivery, ['atomic', 'progressive'], 'atomic', '--delivery'); const simulatedTailMs = positiveIntFlag(args.simulatedTailMs, 0); const quiet = boolFlag(args.quiet); const outputPath = args.output ? resolve(ROOT, String(args.output)) : null; diff --git a/scripts/lib/cli-args.mjs b/scripts/lib/cli-args.mjs index 3356d6b30..8e0a58efd 100644 --- a/scripts/lib/cli-args.mjs +++ b/scripts/lib/cli-args.mjs @@ -69,3 +69,19 @@ export function positiveIntFlag(value, fallback) { } return parsed; } + +/** + * Resolve a flag that must be one of a fixed set. + * + * A silent `x === 'known' ? 'known' : fallback` is the trap this replaces: the + * private evals Live runner passes `--agent=codex`, which fell through to the + * canned fake agent and produced a clean-looking report of a deterministic stub + * labelled as a real harness run. An unrecognized value is a mistake, not a + * request for the default. + */ +export function resolveEnum(value, allowed, fallback, flagName) { + if (value === undefined || value === true) return fallback; + const normalized = String(value).trim().toLowerCase(); + if (allowed.includes(normalized)) return normalized; + throw new Error(`${flagName} must be one of ${allowed.join(', ')}; got: ${value}`); +} diff --git a/scripts/lib/transformers/providers.js b/scripts/lib/transformers/providers.js index 8351c4b22..009042d5f 100644 --- a/scripts/lib/transformers/providers.js +++ b/scripts/lib/transformers/providers.js @@ -22,7 +22,10 @@ export const PROVIDERS = { }, 'claude-code': { provider: 'claude-code', - providerTags: ['claude-code', 'claude'], + // live-progressive: Live delivers variant 1 as soon as it validates instead of + // one atomic edit. Claude Code polls in a background task, so the extra + // publish calls do not stall its control lane. + providerTags: ['claude-code', 'claude', 'live-progressive'], configDir: '.claude', displayName: 'Claude Code', frontmatterFields: ['user-invocable', 'argument-hint', 'license', 'compatibility', 'metadata', 'allowed-tools'], @@ -40,7 +43,7 @@ export const PROVIDERS = { }, codex: { provider: 'codex', - providerTags: ['codex'], + providerTags: ['codex', 'live-progressive'], configDir: '.codex', displayName: 'Codex', frontmatterFields: [], @@ -54,7 +57,7 @@ export const PROVIDERS = { }, agents: { provider: 'agents', - providerTags: ['agents', 'codex'], + providerTags: ['agents', 'codex', 'live-progressive'], configDir: '.agents', displayName: 'Codex Repo Skills', placeholderProvider: 'codex', diff --git a/scripts/lib/utils.js b/scripts/lib/utils.js index 48a4c2c27..ca2954334 100644 --- a/scripts/lib/utils.js +++ b/scripts/lib/utils.js @@ -645,6 +645,11 @@ export const PROVIDER_BLOCK_TAGS = new Set([ 'rovo-dev', 'trae', 'trae-cn', + // Capability tags. Not harness names: they mark instructions that belong to a + // shared capability several harnesses opt into. Listing the harnesses instead + // would mean duplicating the block body per provider tag, since a block takes + // one tag. Opt a provider in by adding the tag to its providerTags. + 'live-progressive', ]); /** diff --git a/skill/agents/impeccable-live-generator.md b/skill/agents/impeccable-live-generator.md index 2935c9746..38cbf30ff 100644 --- a/skill/agents/impeccable-live-generator.md +++ b/skill/agents/impeccable-live-generator.md @@ -6,7 +6,6 @@ tools: Read, Write, Edit, Bash, Glob, Grep model: inherit effort: low max-turns: 16 -providers: codex nickname-candidates: - Variant Producer - Live Composer @@ -15,7 +14,7 @@ nickname-candidates: # Impeccable Live Generator -You own one leased Impeccable Live `generate` event. The parent thread owns browser control and the foreground poll loop. Never poll, Accept, Discard, commit, stage, or edit generated provider output. +You own one leased Impeccable Live `generate` event. The parent thread owns browser control and the poll loop. Never poll, Accept, Discard, commit, stage, or edit generated provider output. ## Compact input contract diff --git a/skill/reference/live.md b/skill/reference/live.md index 4c939757c..560e2e685 100644 --- a/skill/reference/live.md +++ b/skill/reference/live.md @@ -17,24 +17,24 @@ Execute in order. No step skipped, no step reordered. 3. Poll loop with the default long timeout (600000 ms). Run `live-poll.mjs` again immediately after every event or `--reply`; Codex runs this one-shot poll in the foreground. Never pass a short `--timeout=`. The global bar **Impeccable mark** dims and shows a pulsing amber dot when no agent is long-polling `/poll`. Hover the mark for the hint; restart `live-poll.mjs` to reconnect. -4. On `generate`: reuse `event.scaffold` when present; read the screenshot if present; load the action's reference; deliver variants using the harness policy below; `--reply done`; poll again. In Codex, delegate the complete event to `impeccable_live_generator` and resume the foreground poll immediately; the generator owns publication and the reply. +4. On `generate`: reuse `event.scaffold` when present; read the screenshot if present; load the action's reference; deliver variants using the harness policy below; `--reply done`; poll again. In Codex and Claude Code, delegate the complete event to `impeccable_live_generator` / `impeccable-live-generator` and resume polling immediately; the generator owns publication and the reply. 5. On `steer`: read the message and `pageUrl`; do the work (page edits, navigation help, or a short reply in the `--reply` message); `--reply steer_done`; poll again. No pickup ack. The Steer bar unlocks when `steer_done` arrives over SSE. 6. On `accept` / `discard`: the poll script runs `live-accept.mjs`, acknowledges the delivered event, and prints `_completionAck`. Plain accepts/discards are terminal immediately. Carbonize accepts remain recoverable until the foreground task runs `live-complete.mjs --id EVENT_ID`; finish that cleanup before polling again. 7. If interrupted, run `live-status.mjs` or `live-resume.mjs` before guessing. The durable journal replays unacknowledged work after helper restart. 8. On `exit`: run the cleanup at the bottom. Harness policy: -- **Claude Code**: run the poll as a **background task** (no short timeout). The harness notifies you when it completes, so the main conversation stays free. Do not block the shell. +- **Claude Code**: run the poll as a **background task** (no short timeout). The harness notifies you when it completes, so the main conversation stays free. Do not block the shell. When `generate` arrives and native subagents are available, delegate the complete event to `impeccable-live-generator` with a compact handoff, then start the next poll immediately while that subagent publishes and replies; if unavailable, generate inline with the same contract. Do not paste this full reference into the handoff. The subagent's context is separate from yours, which is the point: a long session's screenshots, references, and variant CSS stay out of the main conversation. Handle Steer, Accept/Discard, manual Apply, carbonize, and Exit in the main conversation. - **Cursor**: run **one-shot** poll in a **background terminal** with notify on `"type":"(steer|generate|accept|discard|exit)"`. After each event the poll exits; handle it, `--reply`, then start `live-poll.mjs` again. Do **not** use `--stream` on Cursor: incremental stdout notify is slower in practice than exit-based notify (~5s vs sub-second in testing). - **Codex**: run the default one-shot poll in a **yielded foreground exec session**. Do not suffix it with `&`, use `--stream`, or leave Live without an active foreground poll. When `generate` arrives, delegate to the low-effort `impeccable_live_generator` agent with a compact handoff, then immediately start the next foreground poll while that agent publishes and replies. Do not paste this full reference into the handoff. Handle Steer, Accept/Discard, manual Apply, carbonize, and Exit in the main task; after each handler/reply, restart the foreground poll. - **Other harnesses**: one-shot foreground unless you know stdout reliably returns to this session when a shell exits. Generation delivery policy: -- **Default (Claude Code, Cursor, and other harnesses):** keep the established atomic single-edit delivery unless that harness has independently demonstrated that progressive tool calls are faster and reliable. This avoids trading model latency for extra tool-call latency on harnesses with different streaming behavior. +- **Default (Cursor and other harnesses):** keep the established atomic single-edit delivery. Do not switch a harness to progressive until its poll loop is known not to block on the extra publish calls. This avoids trading model latency for extra tool-call latency on harnesses with different streaming behavior. - -- **Codex progressive override:** deliver progressively through `live-publish.mjs`, never by editing project source directly. Publish variant 1 as soon as it is complete, then publish each additional validated variant (or the largest ready prefix) without waiting for later siblings. Attach parameter CSS/manifests only with the complete set. The browser makes every arrived variant immediately reviewable and acceptable; Accept/Discard durably cancel unfinished revisions. - + +- **Progressive delivery (Codex, Claude Code):** deliver progressively through `live-publish.mjs`, never by editing project source directly. Publish variant 1 as soon as it is complete, then publish each additional validated variant (or the largest ready prefix) without waiting for later siblings. Attach parameter CSS/manifests only with the complete set. The browser makes every arrived variant immediately reviewable and acceptable; Accept/Discard durably cancel unfinished revisions. The user reviews the first direction while the rest are still being written, so time-to-first-variant is what matters, not total time. + Chat is overhead. No recap, no tutorial output, no pasting PRODUCT / DESIGN bodies. Spend tokens on tools and edits; on failure, one or two short sentences. @@ -314,14 +314,14 @@ Colocate preview CSS as a `