feat: just-in-time event instructions + frontier default for the LLM e2e agent

Field feedback from two more Codex sessions drove both changes.

JIT instructions (live/instructions.mjs): every event live-poll prints
now carries _instructions, the authoritative next step for that exact
situation with real ids, paths, and line numbers substituted, and only
the active path's rules (a svelte-component session never sees JSX
guidance). The boot payload carries loop instructions the same way.
Instructions are versioned with the scripts, so they cannot drift from
behavior, and live.md's plumbing can keep shrinking toward contract plus
craft guidance. The Codex poll-discipline failure observed in the field
("the long poll was started, but I yielded the task instead of actively
servicing its result") gets a named anti-pattern in both the harness
policy and the boot instructions.

LLM e2e agent: default provider/model moves from Claude Haiku 4.5 to
OpenAI gpt-5.6-terra at medium reasoning effort via an Anthropic-shaped
shim over the ai SDK (the three call sites stay provider-agnostic;
Anthropic and DeepSeek remain selectable). The harness should exercise
the model tier that actually drives live sessions. Both the react and
sveltekit fixtures pass end to end with terra driving the trimmed
live.md and the new _instructions.

This work was produced with AI assistance (Claude Code).

Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
Paul Bakaus
2026-07-27 19:20:02 -07:00
co-authored by Claude Code
parent b4f1c1786e
commit 26f54d15c2
9 changed files with 294 additions and 9 deletions
+1 -1
View File
@@ -167,7 +167,7 @@ Three live-mode invariants worth knowing before editing (established by the 2026
The agent is pluggable via a one-method interface in `tests/live-e2e/agent.mjs`: `generateVariants(event, context) → { scopedCss, variants[] }`. The default fake agent emits canned variants that exercise all three param kinds (`range`, `steps`, `toggle`). The orchestrator (wrap, write, accept, carbonize) is agent-agnostic.
**LLM agent (opt-in)**: set `IMPECCABLE_E2E_AGENT=llm` to swap the fake agent for `tests/live-e2e/agents/llm-agent.mjs`, which calls Claude (default Haiku 4.5) via `@anthropic-ai/sdk`. Requires `ANTHROPIC_API_KEY` in env; the test runner skips with a clear message when it's unset. Override the model with `IMPECCABLE_E2E_LLM_MODEL=claude-sonnet-4-6` if Haiku produces unreliable JSON. Caching is on — live.md is the cacheable prefix, and after the first call subsequent fixtures pay only the cache-read rate. Pass rate on a typical sweep is 18/19; the modal fixture's intrinsic state-loss flake is amplified by LLM latency and may need a re-run. **This path hits the API and costs money** — keep it out of CI unless you really want it there.
**LLM agent (opt-in)**: set `IMPECCABLE_E2E_AGENT=llm` to swap the fake agent for `tests/live-e2e/agents/llm-agent.mjs`. Default provider/model: OpenAI `gpt-5.6-terra` at medium reasoning effort (a frontier tier, matching what drives real live sessions); Anthropic and DeepSeek remain selectable via `IMPECCABLE_E2E_LLM_PROVIDER`. Requires the selected provider's key in env (`OPENAI_API_KEY` by default); the test runner skips with a clear message when it's unset. Override the model with `IMPECCABLE_E2E_LLM_MODEL` and the effort with `IMPECCABLE_E2E_LLM_EFFORT`. Caching is on — live.md is the cacheable prefix, and after the first call subsequent fixtures pay only the cache-read rate. Pass rate on a typical sweep is 18/19; the modal fixture's intrinsic state-loss flake is amplified by LLM latency and may need a re-run. **This path hits the API and costs money** — keep it out of CI unless you really want it there.
Adding a new fixture is a matter of cloning a directory under `tests/framework-fixtures/`, swapping the source files, and writing a `fixture.json`. See `tests/framework-fixtures/README.md` for the full schema.
+2 -2
View File
@@ -10,7 +10,7 @@ Codex: run live helper commands, the app dev server, and any dependency-installi
## The contract (read once)
Execute in order. No step skipped, no step reordered.
Execute in order. No step skipped, no step reordered. Every tool output in live mode may carry an `_instructions` field: it is the authoritative next step for that exact situation, with real ids and paths substituted; when it conflicts with your recollection of this document, `_instructions` wins.
1. `live.mjs`: boot. If the request names or implies a file, route, or app inside a monorepo, infer the concrete path and run `node {{scripts_path}}/live.mjs --target <path>` instead; then run the rest of this live session from the returned `projectRoot`. The boot resolves the app root from dev-server config files and persists it in `.impeccable/live/roots.json`; every helper re-anchors to that manifest at startup (a wrong cwd cannot fork session state), PRODUCT.md / DESIGN.md are discovered upward to the git root, and relative helper args like `--file` resolve against the app root.
2. Open the app URL that serves `pageFile` (infer from `package.json`, docs, terminal output, or an open tab). Never use `serverPort`; it's the helper, not the app. **Cursor:** `browser_navigate` to that URL before polling; do not skip. **Other harnesses:** use the available browser tool; if the URL is uncertain, ask the user once.
@@ -24,7 +24,7 @@ Execute in order. No step skipped, no step reordered.
Harness policy:
- **Claude Code**: run the poll as a **background task** (no short timeout); the harness notifies you on completion. Do not block the shell.
- **Cursor**: **one-shot** poll in a **background terminal** with notify on `"type":"(steer|generate|accept|discard|exit)"`; handle, `--reply`, restart the poll. Do **not** use `--stream` on Cursor (measured ~5s pickup vs sub-second one-shot).
- **Codex**: default one-shot poll in a **yielded foreground exec session**. No `&`, no `--stream`, never leave Live without an active foreground poll.
- **Codex**: default one-shot poll in a **yielded foreground exec session**. No `&`, no `--stream`, never leave Live without an active foreground poll. Starting the poll is not enough: SERVICE it (keep reading the exec session until it returns an event). Never announce "waiting for the user" and idle; a yielded poll nobody reads is a dead session, and the user's Go sits unanswered.
- **Other harnesses**: one-shot foreground unless you know stdout reliably returns when a shell exits.
Delivery policy: atomic single-edit delivery everywhere; do not switch a harness to progressive publishing unless its poll loop is known not to block on the extra calls.
+8
View File
@@ -15,6 +15,7 @@ import { fileURLToPath } from 'node:url';
import { completionAckForAcceptResult, completionTypeForAcceptResult } from './live/completion.mjs';
import { readLiveServerInfo } from './lib/impeccable-paths.mjs';
import { enterLiveRoot } from './live/roots.mjs';
import { instructionsForEvent } from './live/instructions.mjs';
// Absolute path to a sibling script in this skill's scripts dir, so runtime
// error hints print a directly-runnable command instead of a placeholder.
@@ -262,6 +263,13 @@ export function writeCarbonizeBanner(event) {
}
export function printPollEvent(event) {
// Situational plumbing rides with the event itself: `_instructions` is the
// authoritative next step, with real ids and paths substituted, so the
// reference doc can stay lean and can never drift from script behavior.
if (event && typeof event === 'object' && !event._instructions) {
const instructions = instructionsForEvent(event, { scriptsPath: SELF_DIR });
if (instructions) event._instructions = instructions;
}
console.log(JSON.stringify(event));
}
+2
View File
@@ -26,6 +26,7 @@ import { resolveFiles } from './live-inject.mjs';
import { readLiveServerInfo } from './lib/impeccable-paths.mjs';
import { resolveSurfaceBrief } from './lib/surface-briefs.mjs';
import { resolveLiveTarget } from './live-target.mjs';
import { bootInstructions } from './live/instructions.mjs';
import { resolveRoots, writeRootsManifest } from './live/roots.mjs';
const __dirname = path.dirname(fileURLToPath(import.meta.url));
@@ -196,6 +197,7 @@ The agent should then:
hasSurfaceBrief: !!surfaceBrief,
surfaceBrief,
surfaceBriefPath,
_instructions: bootInstructions({ scriptsPath: __dirname }),
}, null, 2));
}
+142
View File
@@ -0,0 +1,142 @@
/**
* Just-in-time agent instructions for live mode.
*
* The live scripts, not the reference doc, own situational plumbing: every
* event printed by live-poll carries an `_instructions` string describing
* exactly what to do NEXT, with real ids, paths, and line numbers already
* substituted and only the active path's rules included (a svelte-component
* session never sees JSX guidance, and vice versa). live.md stays lean: the
* session contract, harness policy, and design-quality guidance that is not
* situational (identity lock, variation axes, parameter budgets).
*
* Keep these strings imperative, concrete, and short. They are read by an
* agent mid-session; every sentence must earn its tokens. Instructions are
* versioned with the scripts, so they cannot drift from behavior the way a
* hand-maintained doc can.
*/
const PLAN_POINTER = 'Plan per live.md section 4: extract the identity lock, pick default vs departure mode, commit each variant to a DIFFERENT primary axis, squint-test the trio. Size parameter knobs per section 7 budgets.';
function pollCmd(scriptsPath) {
return `node ${scriptsPath}/live-poll.mjs`;
}
function replyCmd(scriptsPath, id, rest) {
return `${pollCmd(scriptsPath)} --reply ${id} ${rest}`;
}
export function instructionsForEvent(event, { scriptsPath = '{{scripts_path}}' } = {}) {
if (!event || typeof event !== 'object') return undefined;
switch (event.type) {
case 'generate':
return generateInstructions(event, scriptsPath);
case 'steer':
return `Do what the message asks (page edits, navigation help, or a short answer). Then reply exactly once: ${replyCmd(scriptsPath, event.id, 'steer_done ["optional short toast"]')} (on failure: --reply ${event.id} error "Short reason"). No pickup ack; poll again immediately after.`;
case 'prefetch':
return `Speculative pre-read, no reply owed: resolve ${JSON.stringify(event.pageUrl || '/')} to its source file (root "/" is usually the boot's pageFile; multi-page sites map /foo to public/foo/index.html; SPAs map all routes to one entry), read it into context, then poll again. Skip if you cannot resolve it confidently.`;
case 'variant_mount_failed':
return `The browser could NOT render variant ${event.variant}${event.url ? ` (module: ${event.url})` : ''}${event.error ? `: ${String(event.error).slice(0, 200)}` : ''}. The user sees a persistent error card, not variants. Fix the variant source files, then reply ${replyCmd(scriptsPath, event.id, 'done --file <manifest or source path>')}; the browser retries on its own. Poll again after the reply.`;
case 'accept':
return acceptInstructions(event, scriptsPath);
case 'discard':
return event?._completionAck?.ok === true
? 'Original restored and durable completion acknowledged; nothing to do. Poll again.'
: `Completion was not acknowledged: run node ${scriptsPath}/live-complete.mjs --id ${event.id} --discarded, then poll again.`;
case 'manual_edit_apply':
return `The user already clicked Apply; never ask, discard, or redirect. Delegate the source edits to the impeccable_manual_edit_applier subagent when available (pass cwd, scripts path, event id, page URL, chunk/deadline, batch, evidencePath); it must not poll or reply. ${event.repair ? 'A `repair` payload is present: the previous Apply changed source but validation failed; fix the CURRENT source, never roll back yourself. ' : ''}Reply exactly once: ${replyCmd(scriptsPath, event.id, `done --data '{"status":"done","appliedEntryIds":[...],"failed":[],"files":[...],"notes":[]}'`)} (status "partial"/"error" with failed[] when not every entry applied). Then poll again.`;
case 'timeout':
return 'No event arrived; poll again immediately.';
case 'exit':
return `Session over: kill any background poll, then node ${scriptsPath}/live-server.mjs stop (removes the injected script tag). Sweep leftover impeccable-variants-start / impeccable-carbonize-start markers from source.`;
default:
return undefined;
}
}
function generateInstructions(event, scriptsPath) {
const id = event.id;
const scaffold = event.scaffold;
const steps = [];
if (event.screenshotPath) {
steps.push(`Read the annotated screenshot first: ${event.screenshotPath}. Comment {x,y} positions bind text to the child under that point; strokes read by shape (loop = emphasis on this thing, arrow = direction, cross = delete).`);
} else {
steps.push('No screenshot was sent (the user did not annotate); do not ask for one and do not screenshot the page. Work from element.outerHTML, the computed styles, and the prompt.');
}
if (event.mode === 'insert') {
steps.push(insertScaffoldInstructions(event, scriptsPath));
} else if (scaffold?.previewMode === 'svelte-component') {
steps.push(svelteComponentInstructions(event, scaffold, scriptsPath));
} else if (scaffold && scaffold.sourceWritten === false) {
steps.push(deferredWrapperInstructions(event, scaffold, scriptsPath));
} else if (scaffold) {
steps.push(`The wrapper is already written into ${scaffold.file}. Splice preview CSS plus all ${event.count} variants at line ${scaffold.insertLine} in ONE edit, following the returned cssAuthoring contract (styleTag, selector strategy, forbidden patterns). Each variant div holds exactly ONE top-level element (same tag as the original); first visible, others display: none.`);
} else {
steps.push(`Preflight could not scaffold${event.scaffoldError ? ` (${event.scaffoldError})` : ''}. Run node ${scriptsPath}/live-wrap.mjs --id ${id} --count ${event.count} --element-id "${event.element?.id || ''}" --classes "${(event.element?.classes || []).join(',')}" --tag "${event.element?.tagName || ''}" --text "<first ~80 chars of the picked element's textContent>". Keep the flags separate; --text disambiguates repeated siblings. On a fallback error, follow live.md's Handle fallback.`);
}
steps.push(event.action && event.action !== 'impeccable'
? `Action is "${event.action}": read reference/${event.action}.md before planning; its MUST params are non-negotiable. ${PLAN_POINTER}`
: `Freeform action: work from SKILL.md rules plus craft-floor.md; no sub-command file. ${PLAN_POINTER}`);
steps.push(`When all ${event.count} variants are delivered: ${replyCmd(scriptsPath, id, 'done --file <project-root-relative path you wrote>')}. Then poll again. If generation fails after the browser flipped to GENERATING, reply --reply ${id} error "Short reason" so the bar resets (never live-accept --discard for this).`);
return steps.map((s, i) => `${i + 1}. ${s}`).join('\n');
}
function svelteComponentInstructions(event, scaffold, scriptsPath) {
const dir = scaffold.componentDir;
const count = event.count;
return `Svelte component preview. EDIT the existing stubs ${dir}/v1.svelte ... v${count}.svelte in place; never delete or recreate them; do not read them back (the prop-substituted markup is in scaffold.componentStubMarkup). Keep the stub's control flow ({#each}, {#if}) and propContract prop names exactly; never flatten a loop into literal items. The stub <style> is seeded with the source rules that style the selection; restyle or delete freely, and know that any seeded rule you do not re-declare is REMOVED from source on accept (the preview never applied it). Semantic class selectors only: no @scope, no data-impeccable-* attributes. Params go in ${dir}/params.json keyed by variant number (never an attribute); author knob CSS against var(--p-<id>, default) and :global([data-p-<id>="..."]). Reply with --file ${scaffold.file}. Accept later merges everything into ${scaffold.sourceFile} mechanically; you have no post-accept cleanup.`;
}
function deferredWrapperInstructions(event, scaffold, scriptsPath) {
const insertNote = Number(scaffold.replaceEndLine) < Number(scaffold.replaceStartLine)
? ` (replaceEndLine < replaceStartLine: this is an INSERTION at line ${scaffold.replaceStartLine}; remove nothing)`
: '';
return `The wrapper is NOT in source yet. In ONE edit to ${scaffold.file}: splice preview CSS plus all ${event.count} variants into scaffold.wrapperBlock at the "Variants: insert below this line" marker, then replace lines ${scaffold.replaceStartLine}-${scaffold.replaceEndLine}${insertNote} with the result. Two separate writes reload the framework mid-publish and strand the browser at 0/N. Author CSS per the returned cssAuthoring contract; each variant div holds exactly ONE top-level element (same tag as the original); first visible, others display: none. On JSX/TSX wrap the <style> content in a template literal and use className / style={{...}}.`;
}
function insertScaffoldInstructions(event, scriptsPath) {
const scaffold = event.scaffold;
const base = `Insert mode: net-new content sized around ${event.placeholder?.width || '?'}x${event.placeholder?.height || '?'} at the chosen anchor; load craft-floor.md before writing net-new markup.`;
if (scaffold?.previewMode === 'svelte-component') {
return `${base} Write each inserted variant as a single-root Svelte component under ${scaffold.componentDir} (no data-impeccable-* attributes, CSS in each component's <style>). Never edit the route during generation; reply with --file ${scaffold.file}.`;
}
if (scaffold && scaffold.sourceWritten === false) {
return `${base} Splice your variants into scaffold.wrapperBlock at the marker and insert the result at line ${scaffold.replaceStartLine} of ${scaffold.file} in ONE edit.`;
}
return `${base} If no scaffold payload is present, run node ${scriptsPath}/live-insert.mjs --id ${event.id} --count ${event.count} --position ${event.insert?.position || 'after'} with the anchor flags from event.insert.anchor, then splice variants at the returned insertLine.`;
}
function acceptInstructions(event, scriptsPath) {
const result = event._acceptResult || {};
const ackOk = event._completionAck?.ok === true;
const prefix = ackOk ? '' : `Completion was NOT acknowledged: run node ${scriptsPath}/live-status.mjs, finish any cleanup, then node ${scriptsPath}/live-complete.mjs --id ${event.id}. `;
if (result.handled === true && result.carbonize === true) {
return `${prefix}Carbonize cleanup is REQUIRED now, before the next poll, in ${result.file}: (1) locate the impeccable-carbonize-start/end block and read the impeccable-param-values comment; (2) move the CSS rules into the stylesheet that owns this area; (3) bake params while rewriting selectors (@scope wrappers to semantic classes, keep only the chosen data-p branch, substitute range literals); (4) unwrap the accepted content and drop every data-impeccable-* / data-p-* attribute; (5) delete the inline <style>, the param-values comment, and both markers plus dead @scope rules. Then run node ${scriptsPath}/live-complete.mjs --id ${event.id} and verify phase "completed"; it refuses with source_dirty while leftovers remain. Poll again only after that.`;
}
if (result.handled === true) {
return `${prefix}Accept was merged into source mechanically; nothing to clean up. Poll again.`;
}
if (result.mode === 'fallback') {
return `${prefix}The session lived in a generated file, so accept refused to persist there. Write the accepted variant into the true source you identified during Handle fallback, remove the temporary wrapper from the served file, then poll again.`;
}
if (result.mode === 'error') {
if (result.error === 'source_locked') {
return `${prefix}The source file is briefly locked by a publisher. Re-run the exact same live-accept.mjs command (idempotent); do NOT hand-edit the file, and do not poll past this.`;
}
if (result.error === 'accept_receipt_conflict') {
return `${prefix}This session already resolved as ${result.priorOperation || 'a prior operation'}; do not edit anything. Run node ${scriptsPath}/live-status.mjs and tell the user what the session resolved to.`;
}
return `${prefix}Accept failed: ${result.error || 'unknown error'}. Source was not touched; do not hand-edit. Run node ${scriptsPath}/live-status.mjs before continuing.`;
}
return `${prefix}No mechanical accept result; read ${result.file || 'the session source file'}, find the impeccable markers, and finish the merge by hand. Poll again after.`;
}
/** Boot instructions attached to live.mjs's success payload. */
export function bootInstructions({ scriptsPath = '{{scripts_path}}' } = {}) {
return `Open the app URL that serves a pageFiles entry (never serverPort; that is the helper). Then start the poll loop per your harness policy in live.md and re-run ${pollCmd(scriptsPath)} immediately after every event or reply. Every event carries _instructions: follow them; they are the authoritative next step with real ids and paths filled in. A poll that is running is a poll you are SERVICING: never announce you are waiting and idle your turn; stay on the exec session until it returns an event, and never end a turn while a poll is outstanding.`;
}
+12 -3
View File
@@ -20,14 +20,23 @@ import {
} from './live-e2e/agents/llm-agent.mjs';
describe('live-e2e LLM agent provider config', () => {
it('defaults to Anthropic and Claude Haiku when no keys are present', () => {
it('defaults to OpenAI gpt-5.6-terra at medium reasoning effort', () => {
const config = resolveLlmAgentConfig({}, {});
assert.equal(config.provider, 'openai');
assert.equal(config.model, 'gpt-5.6-terra');
assert.equal(config.reasoningEffort, 'medium');
assert.equal(config.requiredEnv, 'OPENAI_API_KEY');
assert.equal(config.apiKey, undefined);
assert.equal(config.baseURL, undefined);
});
it('still resolves Anthropic when explicitly selected', () => {
const config = resolveLlmAgentConfig({}, { IMPECCABLE_E2E_LLM_PROVIDER: 'anthropic', ANTHROPIC_API_KEY: 'k' });
assert.equal(config.provider, 'anthropic');
assert.equal(config.model, 'claude-haiku-4-5');
assert.equal(config.requiredEnv, 'ANTHROPIC_API_KEY');
assert.equal(config.apiKey, undefined);
assert.equal(config.baseURL, undefined);
});
it('prefers Anthropic when both provider keys are present', () => {
+63 -2
View File
@@ -36,6 +36,12 @@ const __dirname = path.dirname(fileURLToPath(import.meta.url));
const REPO_ROOT = path.join(__dirname, '..', '..', '..');
const LIVE_MD_PATH = path.join(REPO_ROOT, 'skill', 'reference', 'live.md');
// Frontier default: gpt-5.6-terra at medium reasoning effort. Haiku stayed
// too far below the models that actually drive live sessions in the field;
// a spec change Haiku tolerates can still confuse or be confused by the
// frontier tier, so the harness should exercise the tier users run.
const DEFAULT_OPENAI_MODEL = 'gpt-5.6-terra';
const DEFAULT_OPENAI_REASONING_EFFORT = 'medium';
const DEFAULT_ANTHROPIC_MODEL = 'claude-haiku-4-5';
// DeepSeek model list: https://api-docs.deepseek.com/api/list-models
const DEFAULT_DEEPSEEK_MODEL = 'deepseek-v4-flash';
@@ -199,6 +205,17 @@ const STEER_SYSTEM_INSTRUCTIONS = [
export function resolveLlmAgentConfig(opts = {}, env = process.env) {
const provider = resolveProvider(opts, env);
if (provider === 'openai') {
return {
provider,
model: opts.model || env.IMPECCABLE_E2E_LLM_MODEL || DEFAULT_OPENAI_MODEL,
apiKey: opts.apiKey || env.OPENAI_API_KEY,
requiredEnv: 'OPENAI_API_KEY',
baseURL: opts.baseURL || env.OPENAI_BASE_URL,
reasoningEffort: opts.reasoningEffort || env.IMPECCABLE_E2E_LLM_EFFORT || DEFAULT_OPENAI_REASONING_EFFORT,
};
}
if (provider === 'anthropic') {
return {
provider,
@@ -225,9 +242,51 @@ export function resolveLlmAgentConfig(opts = {}, env = process.env) {
function resolveProvider(opts, env) {
const explicit = opts.provider || env.IMPECCABLE_E2E_LLM_PROVIDER;
if (explicit) return String(explicit).trim().toLowerCase();
if (env.OPENAI_API_KEY) return 'openai';
if (env.ANTHROPIC_API_KEY) return 'anthropic';
if (env.DEEPSEEK_API_KEY) return 'deepseek';
return 'anthropic';
return 'openai';
}
/**
* Anthropic-SDK-shaped shim over the `ai` SDK for OpenAI models, so the
* three text-only call sites in this file stay provider-agnostic. system
* blocks are joined (OpenAI caches long prefixes automatically; the
* cache_control marker is Anthropic-specific), temperature is omitted
* (reasoning models reject it), and the reasoning effort rides through
* providerOptions.
*/
async function createOpenAiShim({ apiKey, baseURL, reasoningEffort, }) {
const [{ generateText }, { createOpenAI }] = await Promise.all([
import('ai'),
import('@ai-sdk/openai'),
]);
const provider = createOpenAI({ apiKey, ...(baseURL ? { baseURL } : {}) });
return {
messages: {
async create({ model, system, messages, max_tokens }, { timeout } = {}) {
const systemText = Array.isArray(system)
? system.map((block) => block?.text || '').filter(Boolean).join('\n\n')
: String(system || '');
const result = await generateText({
model: provider(model),
system: systemText,
messages: messages.map((m) => ({ role: m.role, content: String(m.content) })),
maxOutputTokens: max_tokens,
abortSignal: timeout ? AbortSignal.timeout(timeout) : undefined,
providerOptions: { openai: { reasoningEffort } },
});
return {
content: [{ type: 'text', text: result.text }],
usage: {
input_tokens: result.usage?.inputTokens ?? 0,
output_tokens: result.usage?.outputTokens ?? 0,
cache_read_input_tokens: result.usage?.cachedInputTokens ?? 0,
},
};
},
},
};
}
/**
@@ -242,7 +301,9 @@ export async function createLlmAgent(opts = {}) {
const log = opts.log || (() => {});
const liveMd = opts.includeLiveSpec === false ? null : await fs.readFile(LIVE_MD_PATH, 'utf-8');
const client = new Anthropic({ apiKey, ...(baseURL ? { baseURL } : {}) });
const client = provider === 'openai'
? await createOpenAiShim({ apiKey, baseURL, reasoningEffort: config.reasoningEffort })
: new Anthropic({ apiKey, ...(baseURL ? { baseURL } : {}) });
const systemBlocks = (instructions) => [
{
type: 'text',
+63
View File
@@ -169,3 +169,66 @@ describe('live-poll stream helpers', () => {
});
});
describe('just-in-time event instructions', () => {
it('attaches situation-specific _instructions per event type', async () => {
const { instructionsForEvent } = await import('../skill/scripts/live/instructions.mjs');
const sp = '/scripts';
const steer = instructionsForEvent({ type: 'steer', id: 'ev1', message: 'x' }, { scriptsPath: sp });
assert.match(steer, /--reply ev1 steer_done/);
const mountFailed = instructionsForEvent({ type: 'variant_mount_failed', id: 'ev2', variant: 2, url: 'http://x/v2.svelte', error: 'boom' }, { scriptsPath: sp });
assert.match(mountFailed, /variant 2/);
assert.match(mountFailed, /--reply ev2 done --file/);
// Svelte component generate: only svelte guidance, with concrete paths.
const svelteGen = instructionsForEvent({
type: 'generate', id: 'ev3', count: 3, action: 'impeccable',
scaffold: { previewMode: 'svelte-component', componentDir: 'node_modules/.impeccable-live/ev3', file: 'node_modules/.impeccable-live/ev3/manifest.json', sourceFile: 'src/routes/+page.svelte' },
}, { scriptsPath: sp });
assert.match(svelteGen, /EDIT the existing stubs node_modules\/\.impeccable-live\/ev3\/v1\.svelte/);
assert.match(svelteGen, /params\.json/);
assert.doesNotMatch(svelteGen, /JSX|template literal/);
assert.match(svelteGen, /--reply ev3 done --file/);
// Deferred source-preview generate: single-edit rule with real line numbers.
const deferredGen = instructionsForEvent({
type: 'generate', id: 'ev4', count: 3, action: 'bolder',
scaffold: { sourceWritten: false, file: 'src/App.tsx', wrapperBlock: 'x', replaceStartLine: 12, replaceEndLine: 40 },
}, { scriptsPath: sp });
assert.match(deferredGen, /replace lines 12-40/);
assert.match(deferredGen, /ONE edit/);
assert.match(deferredGen, /reference\/bolder\.md/);
assert.doesNotMatch(deferredGen, /params\.json sidecar|componentDir/);
// Carbonize accept: the five steps inline with the real file + complete cmd.
const accept = instructionsForEvent({
type: 'accept', id: 'ev5', _acceptResult: { handled: true, carbonize: true, file: 'public/index.html' }, _completionAck: { ok: true },
}, { scriptsPath: sp });
assert.match(accept, /public\/index\.html/);
assert.match(accept, /live-complete\.mjs --id ev5/);
const mechanicalAccept = instructionsForEvent({
type: 'accept', id: 'ev6', _acceptResult: { handled: true, carbonize: false }, _completionAck: { ok: true },
}, { scriptsPath: sp });
assert.match(mechanicalAccept, /nothing to clean up/i);
const timeout = instructionsForEvent({ type: 'timeout' }, { scriptsPath: sp });
assert.match(timeout, /poll again/i);
});
it('printPollEvent embeds _instructions in the emitted JSON', async () => {
const { printPollEvent } = await import('../skill/scripts/live-poll.mjs');
const lines = [];
const orig = console.log;
console.log = (s) => lines.push(s);
try {
printPollEvent({ type: 'steer', id: 'zz1', message: 'hello' });
} finally {
console.log = orig;
}
const parsed = JSON.parse(lines[0]);
assert.match(parsed._instructions, /--reply zz1 steer_done/);
});
});
+1 -1
View File
@@ -76,7 +76,7 @@ describe('live target-aware monorepo context', () => {
const poll = runNode(LIVE_POLL_SCRIPT, ['--timeout=50'], payload.projectRoot);
assert.equal(poll.status, 0, `stdout:\n${poll.stdout}\nstderr:\n${poll.stderr}`);
assert.deepEqual(JSON.parse(poll.stdout), { type: 'timeout' });
assert.deepEqual(JSON.parse(poll.stdout), { type: 'timeout', _instructions: 'No event arrived; poll again immediately.' });
const stop = runNode(LIVE_SERVER_SCRIPT, ['stop', '--keep-inject'], payload.projectRoot);
assert.equal(stop.status, 0, `stdout:\n${stop.stdout}\nstderr:\n${stop.stderr}`);