Keep workflow preparation bounded and supply complete test evidence

AI assistance: Codex, under maintainer direction.

skill/reference/new-work.md

@@ -72,7 +72,7 @@ Your measured rendition prior: warm, bookish, family, and child-facing subjects
 
 ## 5. Record the decision
 
-Before code, record the chosen direction as a development-only contract under `## Direction contract` in the relevant surface brief. A direction contract is durable route or artifact strategy, so create or update the brief even when no other surface strategy needs persistence. Keep the contract to six short blocks and 150 words at most. THESIS: the one idea this surface owns and the category-default arrangement it refuses. OWN-WORLD: the palette and component language, specific enough to be recognizable with all content removed. STORY: what the visitor understands, believes, and does. FIRST VIEWPORT: the exact composition, what is where and at what scale, and where the primary action sits. FORM: the chosen form, its position on your ordered list, and the seed key the script printed. Close with one more line, FINISH: the run's exit condition, verbatim "unreviewed and undocumented is unfinished; this build ends with the finish review, the verdict, DESIGN.md, and every shipping raster carrying its provenance". The surface brief is the reminder later agents reload across edits and sessions: a page that looks complete with the FINISH line undischarged is not done, it is abandoned at the finish line. If a block reads like a mood, the direction is not decided yet; the finishing review audits the render against this contract. <!-- rule:skill-decide-then-build -->
+Before code, record the chosen direction as a development-only contract under `## Direction contract` in the relevant surface brief. A direction contract is durable route or artifact strategy, so create or update the brief even when no other surface strategy needs persistence. Use six short blocks, roughly 150 words; do not spend tool calls counting words. THESIS: the one idea this surface owns and the category-default arrangement it refuses. OWN-WORLD: the palette and component language, specific enough to be recognizable with all content removed. STORY: what the visitor understands, believes, and does. FIRST VIEWPORT: the exact composition, what is where and at what scale, and where the primary action sits. FORM: the chosen form, its position on your ordered list, and the seed key the script printed. Close with one more line, FINISH: the run's exit condition, verbatim "unreviewed and undocumented is unfinished; this build ends with the finish review, the verdict, DESIGN.md, and every shipping raster carrying its provenance". The surface brief is the reminder later agents reload across edits and sessions: a page that looks complete with the FINISH line undischarged is not done, it is abandoned at the finish line. If a block reads like a mood, the direction is not decided yet; the finishing review audits the render against this contract. <!-- rule:skill-decide-then-build -->
 
 Never copy the direction contract into implementation source or any browser-delivered artifact. This includes HTML or framework comments, hidden DOM, `<template>` elements, `data-*` attributes, rendered JSX or TSX output, serialized props or state, React Server Component payloads, client bundles, metadata or JSON-LD, accessibility-only text, and files served beside the artifact. A compiler or optimizer removing development metadata is not a safety boundary. Reviewers and documenters receive the contract from the surface brief.
 

tests/skill-behavior-harness.test.mjs

@@ -5,6 +5,24 @@ import path from 'node:path';
 import { MockLanguageModelV3 } from 'ai/test';
 import { prepareWorkspace, cleanupWorkspace, makeTools, runTurn, fileLoaded, SKILL_BODY } from './skill-behavior/harness.mjs';
 import { assertPlanningFallbackWarning, assertNewWorkLifecycle } from './skill-behavior/assertions.mjs';
+import { CASE_STUDY_ANSWER } from './skill-behavior/fixtures.mjs';
+
+it('case-study user supplies evidence now instead of promising a future message', async () => {
+  const workspace = prepareWorkspace();
+  try {
+    const { tools, trace } = makeTools(workspace, {}, { answer: () => CASE_STUDY_ANSWER });
+    for (const question of ['What real customer proof do you have?', 'Please paste the promised details.']) {
+      const result = JSON.parse(await tools.ask_user_question.execute({ questions: [{
+        question, options: [{ label: 'I will paste real customer quotes in my next message' }],
+      }] }));
+      assert.equal(result.answers[question], CASE_STUDY_ANSWER);
+      assert.match(result.answers[question], /clearly labeled synthetic case/);
+    }
+    assert.equal(trace.questionAnswers.length, 2);
+  } finally {
+    cleanupWorkspace(workspace);
+  }
+});
 
 it('new-work requires approval and a brief before code, then documents the finished redesign', () => {
   const ask = { name: 'ask_user_question' };

tests/skill-behavior/README.md

@@ -112,6 +112,22 @@ the same six-step setup allowance as Gemini; reference and edit-order assertions
 are unchanged. A full-build baseline separately hit the existing 840-second
 deadline and is not counted as a pass.
 
+The complete routing-only sweep retained its original budgets and passed 55/57
+(Claude 17/19, OpenAI 19/19, Gemini 19/19). A six-step Claude rerun passed the two
+previously clipped cases but exposed a separate context reload on turn two:
+the model queried the loader again for image-tool availability. That run was
+stopped after five passes and this failure rather than finishing another billed
+sweep; the once-per-session assertion remains unchanged.
+
+A saved Claude build trace used five calls shortening/counting the direction
+contract, then reached the 22-step cap before implementation. The word target
+is now approximate, with the six required blocks retained. A subsequent run
+correctly stopped for missing customer evidence: the default simulated user had
+selected “I have real details,” then promised them in a future message. The
+case-study test now supplies a complete, explicitly synthetic brief when asked;
+fresh-init and other user simulations are unchanged. Neither incomplete build
+is counted as a pass.
+
 ## Scenarios
 
 | # | Setup | Assertion |

tests/skill-behavior/fixtures.mjs

@@ -5,6 +5,9 @@
  * uses to decide whether to gate on `init`. Plausible enough that the agent
  * treats them as real context rather than test scaffolding.
  */
+// This user can supply a complete synthetic case now, not promise future assets.
+export const CASE_STUDY_ANSWER = 'Build a standalone index.html for the researchers described in PRODUCT.md, preserving DESIGN.md. Use a clearly labeled synthetic case: a doctoral researcher reconstructs why a literature-review conclusion changed by following linked notes and citations. No real customer names, quotes, metrics, or assets are available; author illustrative content and label it, with no invented commercial claims. The reader should understand the preserved reasoning trail and follow an in-page link to the method. For a composition or concept choice, use the first direction you presented. No additional material is coming from me.';
+
 export const PRODUCT_MD_SAMPLE = `# Acme Notes
 
 ## Platform

tests/skill-behavior/workflow-contract.test.mjs

@@ -18,7 +18,7 @@ import {
 } from './harness.mjs';
 import { detectProvider, getModel, hasKey, resolveModelList, PROVIDERS } from './providers.mjs';
 import { assertNewWorkLifecycle } from './assertions.mjs';
-import { PRODUCT_MD_SAMPLE, DESIGN_MD_SAMPLE } from './fixtures.mjs';
+import { PRODUCT_MD_SAMPLE, DESIGN_MD_SAMPLE, CASE_STUDY_ANSWER } from './fixtures.mjs';
 
 const LEGACY_DESIGN = `# Design
 
@@ -148,6 +148,7 @@ for (const modelId of resolveModelList()) {
           workspace,
           model,
           userPrompt: '/impeccable create a concise evidence-led case-study page. Leave it at index.html.',
+          simulatedUser: { answer: () => CASE_STUDY_ANSWER },
           maxSteps: 22,
         });
         const question = firstCall(trace, ({ name }) => name === 'ask_user_question');