From c4893357996516c48f44c71cda4aea738876b2ab Mon Sep 17 00:00:00 2001 From: Paul Bakaus Date: Thu, 13 Aug 2026 21:52:17 -0400 Subject: [PATCH] Build path becomes a config key existing projects can actually reach The build-path preference shipped as a question only `init` asks, written to a file only `init` writes. Nothing routes an initialized project back through init, so every existing project took the comp-first default without anyone choosing it, and the only recourse was a footer toggle that binds one session. Neither the setting nor the round that preceded it ever reached a release (skill-v4.0.4 has no `buildPath`, no `comp-led`, no `.impeccable/settings.json`), so the PRODUCT.md standing-commitment fallback describes an era that never existed publicly. It is deleted rather than honored: told a field might exist, models go hunting for it and preserve it. - `buildPath` moves from `.impeccable/settings.json` into the unified `.impeccable/config.json`, which already has a known-keys registry, doctor coverage, and a gitignored `config.local.json` override. Whether a machine has an image tool is a property of that machine, so the local file wins. - new-work captures the answer from behavior instead of an interview: a toggle flip on a project recording nothing asks once, after the round closes, whether to keep it. The answer is written either way, because a declined offer nothing writes down is an offer the next session makes again. - Two findings: `config-invalid-build-path` (an unread value rides the default rather than the opposite path) and `config-build-path-unset`, gated on a product record plus evidence of direction work so polish-and-audit projects never hear about a setting they do not use. - init treats a recorded value as a confirmed answer, resolving its conflict with Step 1's "do not reopen confirmed fields". - The setting was undocumented in the README and doctor.md. Both now cover it. Also records a measured skill-behavior baseline. Three cells fail on unmodified main (scenarios 9 and 15, `initialized natural build`), verified against a clean worktree; the suite README now says so, so the next person does not spend the hour attributing them to their own branch. Written with AI assistance (Claude Code). Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 10 +++++ skill/reference/doctor.md | 1 + skill/reference/init.md | 2 +- skill/reference/new-work.md | 4 +- skill/scripts/concept-seed.mjs | 2 +- skill/scripts/context.mjs | 41 ++++++++++++----- skill/scripts/lib/staleness.mjs | 75 +++++++++++++++++++++++++++++++- skill/scripts/serve-question.mjs | 2 +- tests/context.test.mjs | 36 +++++++++++++++ tests/skill-behavior/README.md | 49 +++++++++++++++++---- tests/staleness.test.mjs | 59 +++++++++++++++++++++++++ 11 files changed, 254 insertions(+), 27 deletions(-) diff --git a/README.md b/README.md index 80b17bc98..cc9e4cec6 100644 --- a/README.md +++ b/README.md @@ -361,6 +361,16 @@ On an interactive `install`/`update`, Impeccable explains the hook and offers to For debugging, set `hook.auditLog` in `.impeccable/config.json` to a path (or the legacy `IMPECCABLE_HOOK_LOG` env var) to write one NDJSON line per hook invocation. Leave it unset for normal use. +## Build path: comp-first or code-first + +When a new surface gets designed, Impeccable either generates a full-fidelity comp first and builds to match it, or builds straight in code with the ambition written into the direction contract and checked at the finish. Comp-first composes bolder and takes longer; code-first is leaner and faster. `/impeccable init` asks once and records the answer as `buildPath` in `.impeccable/config.json`: + +```json +{ "buildPath": "comp" } +``` + +The values are `comp` and `code`, and nothing else is read. Set it in the gitignored `.impeccable/config.local.json` to override the team's committed value on one machine, which is what you want when your harness has no image generation. Whatever is recorded is a default rather than a lock: every decision page carries a footer toggle, and flipping it binds that session only. The choice appears at all only where image generation is available, since without it there is nothing to comp. + Codex requires one platform step that Impeccable cannot safely skip: open `/hooks` after install or update and approve the project hook. There is no Codex marketplace/plugin install flow for this hook. Full hook docs: [impeccable.style/docs/hooks](https://impeccable.style/docs/hooks). diff --git a/skill/reference/doctor.md b/skill/reference/doctor.md index 0de7f9221..75c6eadd6 100644 --- a/skill/reference/doctor.md +++ b/skill/reference/doctor.md @@ -46,6 +46,7 @@ The same restraint applies to `workspace-context-inherited`. Inheritance is a de - `workspace-platform-native-evidence` is the finding that matters most here: a workspace carrying native build files while inheriting a root record that resolves to web gets web guidance for its whole life and never loads [ios.md](ios.md) or [android.md](android.md). The repair is a child PRODUCT.md in that workspace, because one inherited record cannot hold two platforms. - `config-project-roots-match-nothing` means every `projectRoots` glob missed, so the repo root is silently standing in as the active project. A renamed workspace directory is the usual cause. Report the patterns and ask which directories they should name. +- `config-invalid-build-path` and `config-build-path-unset` both concern one key, `buildPath` in `.impeccable/config.json` (or the gitignored `.impeccable/config.local.json`, which wins for that developer). It holds `comp` or `code` and sets whether new surfaces are built from a generated comp or straight in code. An unread value does not fall back to the opposite path, so a project meaning `code` has been building comp-led; report the exact value. The unset finding fires only where a project has done direction work and never recorded a preference, and the offer belongs in it only when image generation exists in your tool surface. Without image generation there is nothing to choose and nothing to say. - Use the `workspaces` table to show the user which apps carry their own context, which inherit, and which have none, before proposing any change. ## Opting out of the boot check diff --git a/skill/reference/init.md b/skill/reference/init.md index 444117a6b..34ef41548 100644 --- a/skill/reference/init.md +++ b/skill/reference/init.md @@ -109,7 +109,7 @@ Before loading new-work or resuming shape/build, verify that PRODUCT.md exists a ## Step 5: Record workflow defaults -When image generation is available (context.mjs reports it), ask once how new surfaces should be built, stated as the trade it is: **comp-first** (an image sets the bar before any code; bolder composition, slower, and the build must match the image) or **code-first** (build directly; the ambition is written into the direction contract and audited at the finish; leaner, faster). Write the answer to `.impeccable/settings.json` as `{ "buildPath": "comp" }` or `{ "buildPath": "code" }`, merging with any keys already there. This is a default, not a lock: the decision page renders a toggle whose flip binds a single session and is never written back. Without image generation there is no choice to record; code-first is the only path. +When image generation is available (context.mjs reports it) and no `buildPath` is recorded yet, ask once how new surfaces should be built, stated as the trade it is: **comp-first** (an image sets the bar before any code; bolder composition, slower, and the build must match the image) or **code-first** (build directly; the ambition is written into the direction contract and audited at the finish; leaner, faster). Write the answer to `.impeccable/config.json` as `"buildPath": "comp"` or `"buildPath": "code"`, merging with the keys already there. A value already recorded in `.impeccable/config.json` or the gitignored `.impeccable/config.local.json` is a confirmed answer: on a re-run, honor it in silence rather than asking again. This is a default, not a lock: the decision page renders a toggle whose flip binds a single session and is never written back. Without image generation there is no choice to record; code-first is the only path. Then configure live mode when useful: skip native or non-runnable projects and leave existing config untouched. Otherwise follow [live.md](live.md)'s first-time setup. Any CSP source edit still requires its stated consent. diff --git a/skill/reference/new-work.md b/skill/reference/new-work.md index d6d4c4c31..886fda31b 100644 --- a/skill/reference/new-work.md +++ b/skill/reference/new-work.md @@ -36,7 +36,7 @@ Keep the visual system fixed. Derive five to seven materially different structur `node {{scripts_path}}/concept-seed.mjs --scope surface --mode ` -The script deals three of your structures to the table; the dice decide which three reach the user, so the ranking rut stays broken while the user still holds a real choice. Present the three dealt structures on the decision page as full cards of equal salience, the dealt lead carrying kicker THE ROLL, with steer and re-roll; the user locks one in. No canon card and no pick card at surface scope: the world is settled, so every card visualizes composition, not identity. With image generation available and a comp-led default (the build-path paragraph below: `.impeccable/settings.json`, the toggle handles the exception), each card declares a `comp` under `.impeccable/mocks/decision/`, generated after serving in reading order under the comp discipline in [visualize.md](visualize.md); anchor each of these comps on the established identity by passing a captured screenshot of a representative existing page as a reference image (the harness image tool's input image, or `generate-image.mjs --ref`) beside a prompt that leads with the new surface's structure and names DESIGN.md's palette, type, and component character, because a prose paraphrase of a design system drifts where a pixel reference does not. Without image generation, or under a code-led default, each card instead carries a `wireframe` layout schematic (see `serve-question.mjs --schema`) that the page draws itself. Locking a card is the approval and sets the build path: a locked comp builds comp-led with that comp as the approved comp, discharging [visualize.md](visualize.md)'s three-option round with no second approval point; a locked wireframe builds code-led, its ambition carried by the direction contract. Never run the script for a local extension or a precisely specified narrow request; shape those directly. +The script deals three of your structures to the table; the dice decide which three reach the user, so the ranking rut stays broken while the user still holds a real choice. Present the three dealt structures on the decision page as full cards of equal salience, the dealt lead carrying kicker THE ROLL, with steer and re-roll; the user locks one in. No canon card and no pick card at surface scope: the world is settled, so every card visualizes composition, not identity. With image generation available and a comp-led default (the build-path paragraph below: `.impeccable/config.json`, the toggle handles the exception), each card declares a `comp` under `.impeccable/mocks/decision/`, generated after serving in reading order under the comp discipline in [visualize.md](visualize.md); anchor each of these comps on the established identity by passing a captured screenshot of a representative existing page as a reference image (the harness image tool's input image, or `generate-image.mjs --ref`) beside a prompt that leads with the new surface's structure and names DESIGN.md's palette, type, and component character, because a prose paraphrase of a design system drifts where a pixel reference does not. Without image generation, or under a code-led default, each card instead carries a `wireframe` layout schematic (see `serve-question.mjs --schema`) that the page draws itself. Locking a card is the approval and sets the build path: a locked comp builds comp-led with that comp as the approved comp, discharging [visualize.md](visualize.md)'s three-option round with no second approval point; a locked wireframe builds code-led, its ambition carried by the direction contract. Never run the script for a local extension or a precisely specified narrow request; shape those directly. ### Create or replace the visual world @@ -50,7 +50,7 @@ The standing exit: every direction round offers one quiet, permanent alternative When image generation exists, every card also declares a `comp` path under `.impeccable/mocks/decision/`, the canon card included. Where the harness sandboxes its shell, start the page through the least-sandboxed command path it offers: a sandboxed shell cannot bind the board's port, and the first-attempt failure costs a retry every session. Serve the page first, then produce the comps; the page shimmer-waits per slot and the user may answer before they land. Each card's image is that direction's north-star comp at full fidelity, produced under the comp discipline in [visualize.md](visualize.md): the requested surface's first viewport, structure-led prompt, real product name and real content, no invented commercial claims, in that card's own palette, type character, and material world, committed all the way; visualize.md's self-checks bind decision comps identically. Generation takes the same time at any fidelity, so an unfinished draft pays draft quality for comp cost; fairness between cards comes from equal fidelity in each card's own grammar, one surface, one aspect, never from shared unfinishedness. The frame's aspect is the surface's own: a native app or mobile-first surface comps portrait at its device viewport, a desktop web surface landscape, and the decision page adapts to either, so a phone screen comped landscape is a broken frame, not a neutral default. Produce in the order the user reads, the assigned card, then the pick, then the full-card hand, then canon, each file written with its prompt sidecar the moment it is done, so a re-roll's spend front-loads onto the cards read first; declined challengers get no comp, their catalog thumb is their face. When the harness runs subagents in parallel, fan the set out as one agent per card: each spawn is the shipped asset producer with a single-comp packet, that card's fields, PRODUCT.md, the shared frame, and the card's declared path, up to four in flight at once. A slot still empty when its agent returns is regenerated inline, and a slot still empty when the user answers is dropped without ceremony; no other supervision is owed. Without parallel subagents, generate in the main thread after serving, in the same reading order, and let the harness's own generation display carry the progress; the wait for the answer follows the last file. The chosen card's comp is not spent by the choice: on a comp-led build it enters the comp round as compositional option one, and on a code-led build it returns at the finish review as the critique reference, what the image dared that the build did not. The unchosen comps stay in `.impeccable/mocks/decision/` as the round's spent hand; they carry no approval and imply none. With no image generation, the cards carry their identity in palette chips and facts, and that page is complete, not a lesser version; the page then also demotes every challenger's catalog art to a labeled thumbnail on its own, because salience must encode the verdict, never the accident of which cards have images. -The execution contract, comp-led or code-led, is a workflow preference, not a per-surface decision, so no round asks it: the recorded default rides every round and the page's toggle handles the exception. Read the default from `.impeccable/settings.json` (`buildPath`), falling back to a standing brand commitment in PRODUCT.md recorded before settings existed; with neither, comp-led is the default whenever image generation exists. Author every direction and surface payload with `buildPath: { "value": , "toggle": true }`; the page renders a footer toggle with the trade stated beside it, and the ANSWER returns `buildPath` plus `buildPathFlipped`. A flipped value binds that session only and is never written back; when the user asks in words to change the standing default, update `.impeccable/settings.json`. **Comp-led**: the chosen card's comp is law, generated before building when it does not exist yet, and the finish review audits the build against it; boldest composition on the table, fix rounds expected; comp-led makes the comp non-optional, no silent skipping. **Code-led**: no comp of this page and no apology for it; the QUALITY BAR boards still calibrate finish, and the ambition moves into the written contract, the FIRST VIEWPORT block plus a named signature interaction and motion grammar, which the finish reviewer audits in behavior; code-led is not a discount on commitment, the direction still lands fully committed in code. A code-led round still declares each card's comp path as a flip reserve: when the user flips the toggle to comp mid-round, `--wait` returns once with BUILD PATH FLIPPED while the page shimmers the slots; generate each open card's comp into its declared path then, lead first, and wait again. The flip back is free, and a comp that already rendered rides at the finish review as the critique reference. Without image generation there is no toggle and no choice: code-led is the only path, stated in one line rather than asked. The old two-card execution-contract round is retired; `followup: true` remains the general mechanism for delivering any later round over the same table via `--update`. +The execution contract, comp-led or code-led, is a workflow preference, not a per-surface decision, so no round asks it: the recorded default rides every round and the page's toggle handles the exception. Read the default from `.impeccable/config.json` (`buildPath`), with the gitignored `.impeccable/config.local.json` winning where one machine differs from the team's committed value; with neither, comp-led is the default whenever image generation exists. Author every direction and surface payload with `buildPath: { "value": , "toggle": true }`; the page renders a footer toggle with the trade stated beside it, and the ANSWER returns `buildPath` plus `buildPathFlipped`. A flipped value binds that session only and is never written back, with one exception, and it is the only thing inside a round that earns a question about this preference (init records it up front on projects that get the chance): when `buildPathFlipped` comes back true on a project that records no `buildPath` at all, ask once after the round closes whether to keep it as the standing default, and write it to `.impeccable/config.json` when the user says yes. Ask on the flip and never on the untouched default, because a user who left the toggle alone has told you nothing. Record the answer either way: "no, just this once" means the standing default is the one they flipped away from, so write that value. A declined offer nothing writes down is an offer the next session makes again. When the user asks in words to change the standing default, update the file without asking. **Comp-led**: the chosen card's comp is law, generated before building when it does not exist yet, and the finish review audits the build against it; boldest composition on the table, fix rounds expected; comp-led makes the comp non-optional, no silent skipping. **Code-led**: no comp of this page and no apology for it; the QUALITY BAR boards still calibrate finish, and the ambition moves into the written contract, the FIRST VIEWPORT block plus a named signature interaction and motion grammar, which the finish reviewer audits in behavior; code-led is not a discount on commitment, the direction still lands fully committed in code. A code-led round still declares each card's comp path as a flip reserve: when the user flips the toggle to comp mid-round, `--wait` returns once with BUILD PATH FLIPPED while the page shimmers the slots; generate each open card's comp into its declared path then, lead first, and wait again. The flip back is free, and a comp that already rendered rides at the finish review as the critique reference. Without image generation there is no toggle and no choice: code-led is the only path, stated in one line rather than asked. The old two-card execution-contract round is retired; `followup: true` remains the general mechanism for delivering any later round over the same table via `--update`. Catalog worlds are working systems, not mood references. When one survives, carry its palette and material, type and composition, topology, controls and state, and responsive rules into the product. When the source is itself an interface language, commit to its native grammar across navigation, content, controls, and states. Open the QUALITY BAR board and hero for the world you build the moment the choice lands, even if you viewed another card earlier; the ANSWER line names the chosen card's images (when the harness only reads files or runs sandboxed, download them into the workspace and open the relative path; sandboxed viewers reject absolute paths outside it). They set the craft level the build must reach, a rendered reference's finish, commitment, and art direction, never the composition; your surface serves this product. diff --git a/skill/scripts/concept-seed.mjs b/skill/scripts/concept-seed.mjs index 4b81e8de4..fc44ed067 100644 --- a/skill/scripts/concept-seed.mjs +++ b/skill/scripts/concept-seed.mjs @@ -434,7 +434,7 @@ export function renderConceptSeed({ lead carrying kicker THE ROLL, with steer and re-roll, and let the user lock one in; the world is already settled, so this choice is composition. Visualize every dealt card: with image generation available and a - comp-led default (.impeccable/settings.json buildPath; the page toggle + comp-led default (.impeccable/config.json buildPath; the page toggle handles the exception), declare a comp per card and generate after serving, lead first; otherwise author each card's wireframe field (see serve-question --schema) and the page draws the schematic. Carry the diff --git a/skill/scripts/context.mjs b/skill/scripts/context.mjs index 7830b68dc..2c9ca46ea 100644 --- a/skill/scripts/context.mjs +++ b/skill/scripts/context.mjs @@ -1142,7 +1142,7 @@ async function cli() { parts.push(buildResolvedContextDirective(ctx, cliOptions, { targetExists })); appendDetectorFallback(parts, ctx); appendImageGenDirective(parts); - appendBuildPathDirective(parts); + appendBuildPathDirective(parts, ctx); appendAutonomyCounterDirective(parts); appendSubagentAuthorizationDirective(parts); if (shouldWarnMissingTarget(ctx, targetProvided, targetExists)) { @@ -1162,7 +1162,7 @@ async function cli() { parts.push(buildResolvedContextDirective(ctx, cliOptions, { targetExists })); appendDetectorFallback(parts, ctx); appendImageGenDirective(parts); - appendBuildPathDirective(parts); + appendBuildPathDirective(parts, ctx); appendAutonomyCounterDirective(parts); appendSubagentAuthorizationDirective(parts); if (shouldWarnMissingTarget(ctx, targetProvided, targetExists)) { @@ -1271,17 +1271,34 @@ function automaticHookMode(ctx) { } -// Build-path preference: a workflow setting (comp-led vs code-led), read -// here so every session starts knowing it without a file hunt. Absence -// stays silent; new-work's own default applies, and the decision page -// toggle can flip the value for a single session. -function appendBuildPathDirective(parts) { - try { - const settings = JSON.parse(fs.readFileSync(path.join(process.cwd(), '.impeccable', 'settings.json'), 'utf8')); - if (settings.buildPath === 'comp' || settings.buildPath === 'code') { - parts.push(`BUILD_PATH_DEFAULT: ${settings.buildPath} (from .impeccable/settings.json). Author direction and surface rounds with this as buildPath.value and toggle: true; a flip on the page binds that session only and is never written back to settings.`); +// Build-path preference: a workflow setting (comp-led vs code-led), read here +// so every session starts knowing it without a file hunt. It rides the unified +// config beside the hook and detector settings, and the gitignored local file +// wins, because whether a machine has an image tool is a property of that +// machine, not of the team's committed default. Absence stays silent; +// new-work's own default applies, and the decision page toggle can flip the +// value for a single session. +function readBuildPathAt(root) { + let value = null; + let source = null; + for (const name of ['config.json', 'config.local.json']) { + const raw = readJson(path.join(root, '.impeccable', name)); + if (raw?.buildPath === 'comp' || raw?.buildPath === 'code') { + value = raw.buildPath; + source = `.impeccable/${name}`; } - } catch { /* no settings file */ } + } + return value ? { value, source } : null; +} + +function appendBuildPathDirective(parts, ctx) { + const roots = [...new Set([ctx?.projectRoot, process.cwd()].filter(Boolean).map((root) => path.resolve(root)))]; + for (const root of roots) { + const found = readBuildPathAt(root); + if (!found) continue; + parts.push(`BUILD_PATH_DEFAULT: ${found.value} (from ${found.source}). Author direction and surface rounds with this as buildPath.value and toggle: true; a flip on the page binds that session only and is never written back to the config.`); + return; + } } // Image generation availability: harness-native tools always win, but when the diff --git a/skill/scripts/lib/staleness.mjs b/skill/scripts/lib/staleness.mjs index 5a401804d..80599095b 100644 --- a/skill/scripts/lib/staleness.mjs +++ b/skill/scripts/lib/staleness.mjs @@ -47,18 +47,33 @@ import { // Top-level keys any reader honors: `hook` and `detector` subtrees (hook-lib's // readConfig), `updateCheck` (context.mjs), `projectRoots` (context.mjs's -// monorepo resolution), plus `stalenessCheck` below. `$schema` and `version` -// are allowed as conventional metadata nobody reads. +// monorepo resolution), `buildPath` (context.mjs's build-path directive), plus +// `stalenessCheck` below. `$schema` and `version` are allowed as conventional +// metadata nobody reads. const KNOWN_CONFIG_KEYS = new Set([ 'hook', 'detector', 'updateCheck', 'stalenessCheck', 'projectRoots', + 'buildPath', '$schema', 'version', ]); +// The only two values context.mjs and new-work honor. A near miss reads as a +// working preference and silently rides the opposite path, so it is worth +// reporting rather than coercing. +const BUILD_PATH_VALUES = Object.freeze(['comp', 'code']); + +// Evidence that this project does the kind of work `buildPath` governs. A +// project that only ever ran polish or audit has no use for the setting and +// should never be told it exists. Two stats, so Tier 1 can afford it. +const DIRECTION_WORK_PATHS = Object.freeze([ + path.join('.impeccable', 'surfaces'), + path.join('.impeccable', 'mocks', 'decision'), +]); + // `detector` is a closed set, so a typo here is worth reporting. `hook` is not // checked: it carries runtime settings from several writers and the false // positive rate would outweigh the catch. @@ -325,6 +340,20 @@ export function checkConfig({ projectRoot, repoRoot }) { })); } + if (Object.prototype.hasOwnProperty.call(raw, 'buildPath') + && !BUILD_PATH_VALUES.includes(raw.buildPath)) { + findings.push(finding({ + id: 'config-invalid-build-path', + artifact: 'config.json', + filePath: rel, + severity: 'mention', + summary: `${rel} sets \`buildPath\` to ${JSON.stringify(raw.buildPath)}, which nothing reads. ` + + `The values are ${BUILD_PATH_VALUES.map((value) => `\`${value}\``).join(' and ')}.`, + fix: 'Report the value. An unread `buildPath` does not fall back to the other path; ' + + 'it falls back to the default, so a project meaning `code` has been building comp-led.', + })); + } + const detector = raw.detector; if (detector && typeof detector === 'object' && !Array.isArray(detector)) { const unknownDetector = Object.keys(detector).filter((key) => !KNOWN_DETECTOR_KEYS.has(key)); @@ -345,6 +374,47 @@ export function checkConfig({ projectRoot, repoRoot }) { return findings; } +/** + * No recorded build-path preference on a project that plainly does visual + * direction work. Not drift in the usual sense: the setting is newer than the + * project, so every project that predates it lands here at once. That is why + * it is gated twice, on a product record and on evidence of the work the + * setting governs, and why it says the choice rather than assuming a harness + * can make it. Image generation is the real precondition and this module + * cannot see it: a harness-native image tool leaves no trace on disk, so the + * finding hands the question to the one reader that knows. + */ +export function checkBuildPathUnset({ projectRoot, repoRoot, product }) { + if (!projectRoot || !product) return []; + const roots = [...new Set([projectRoot, repoRoot].filter(Boolean).map((root) => path.resolve(root)))]; + + for (const root of roots) { + for (const name of ['config.json', 'config.local.json']) { + const raw = readJson(path.join(root, '.impeccable', name)); + // Any declared value ends this, valid or not: an invalid one already has + // its own finding and two reports of one key is noise. + if (raw && Object.prototype.hasOwnProperty.call(raw, 'buildPath')) return []; + } + } + + const evidence = DIRECTION_WORK_PATHS.filter((rel) => fs.existsSync(path.join(projectRoot, rel))); + if (!evidence.length) return []; + + return [finding({ + id: 'config-build-path-unset', + artifact: 'config.json', + filePath: '.impeccable/config.json', + severity: 'mention', + summary: 'This project has run visual direction work but records no `buildPath`, ' + + 'so every direction round takes the comp-first default without anyone having chosen it.', + fix: 'Only when image generation exists in your tool surface, offer the choice once: ' + + '**comp-first** (an image sets the bar before any code; bolder composition, slower) or ' + + '**code-first** (build directly; ambition carried by the direction contract; leaner, faster). ' + + 'Write the answer to `.impeccable/config.json` as `"buildPath": "comp"` or `"buildPath": "code"`, ' + + 'merging with the keys already there. Without image generation there is no choice to record: stay silent.', + })]; +} + // ─── Surface briefs ──────────────────────────────────────────────────────── /** @@ -446,6 +516,7 @@ export function collectBootFindings(ctx, extras = {}) { projectRoot, }), ...checkConfig({ projectRoot, repoRoot: ctx.repoRoot }), + ...checkBuildPathUnset({ projectRoot, repoRoot: ctx.repoRoot, product: ctx.product }), ...checkSurfaceBriefs({ candidates: ctx.surfaceBriefCandidates, projectRoot }), ...(extras.projectRootPatterns ? checkProjectRoots({ diff --git a/skill/scripts/serve-question.mjs b/skill/scripts/serve-question.mjs index dd63f989c..4d972e957 100644 --- a/skill/scripts/serve-question.mjs +++ b/skill/scripts/serve-question.mjs @@ -199,7 +199,7 @@ if (hasFlag('schema')) { canonCard: { label: 'The category standard', thesis: 'What this category ships, executed impeccably.', palette: ['#ffffff', '#111827', '#2563eb'], materials: ['clean grid', 'product photography'], viewport: 'The arrangement a visitor expects, at full craft.', risk: 'Indistinguishable from the competition by design.', comp: '.impeccable/mocks/decision/canon.webp' }, steer: true, }, null, 2)); - console.log('\nOption ids return verbatim in ANSWER; "reroll" and "canon" are reserved. hero/board/comp accept URLs or local paths; comp slots may point at files that do not exist yet (serve first, generate after; the page polls until they land, so never block serving on generation). hero on a challenger is the inspiration it draws from and renders picture-in-picture beside the comp, never as the promise of the build. verdict routes rendering: "wins" and "competitive" challengers keep full cards, "declined" ones render demoted after them (narrow, quiet, art as a labeled thumb, "Adopt anyway"), with their kept line on the front; the page reorders declined cards to the end on its own. raised on the assigned card renders each donation as a named raise line. Salience parity: when the assigned card declares no comp (no image generation this round), catalog art on every card demotes to a labeled thumb, so what looks important is the verdict’s call, never rendering luck. canonCard renders the standing exit as a subordinate card with the same anatomy; without it, canon stays a quiet footer action. Include canon only for visual-direction rounds; never present it as your own recommendation. The pick card is a kicker convention, not a field: kicker "IMPECCABLE’S PICK" on your top-ranked grounded candidate, one at most, never in the lead slot. Every card gets the full anatomy, challengers, canon, and declined included: thesis, palette, materials, viewport, risk; the seed already hands you each challenger’s system rules, so a card with no palette chips is an authoring gap, not a data gap. Keep thesis and each fact to one short sentence: the card front shows thesis, identity, and a two-line risk, while first viewport and the case read on the card back behind the Details chip, so long facts cost the reader a flip, not the page its scanability. A card with no imagery at all has no back; its full read renders on the front, so a text-only round loses nothing. A card may instead declare "wireframe" ({"cols":12,"rows":10,"regions":[{"label":"nav rail","x":0,"y":0,"w":3,"h":10,"accent":true}]}): the page draws it as a layout schematic in the media slot; surface-scope rounds use it on code-led builds, it never counts toward salience, and the card keeps its full read on the front. The comp slot carries the card’s full-fidelity direction comp (the legacy key "sketch" is accepted as an alias). Comp aspect follows the surface: portrait at device viewport for native or mobile-first surfaces, landscape otherwise; the page adapts its cards to either. reroll accepts true or { "registers": ["safer", "bolder"] }: the register buttons steer the next hand along the familiar-to-bold axis, the answer carries "register", and you re-run concept-seed with --register for the next round; offer the registers on direction rounds, and never pre-select one. buildPath rides the payload as { "value": "comp"|"code", "toggle": true }: the value is the recorded default (.impeccable/settings.json, or a PRODUCT.md standing commitment as fallback) and the toggle renders a footer switch whose flip binds that session only; the ANSWER then carries buildPath plus buildPathFlipped. On a code-led round each card still declares its comp path as a flip reserve: wireframes render, and a flip to comp makes --wait return once with BUILD PATH FLIPPED so you generate the comps into the declared slots while the round stays open; a flip back to code is free, and a comp that already landed stays as the critique reference. The toggle may only be offered when image generation exists: a harness with no image tool and no API key never sets toggle: true, so the choice never renders where comps cannot be made, and code-led simply rides as the untoggleable value. followup: true keeps the table open after a pick for a second round via --update; send the next payload immediately, the page is waiting on it.'); + console.log('\nOption ids return verbatim in ANSWER; "reroll" and "canon" are reserved. hero/board/comp accept URLs or local paths; comp slots may point at files that do not exist yet (serve first, generate after; the page polls until they land, so never block serving on generation). hero on a challenger is the inspiration it draws from and renders picture-in-picture beside the comp, never as the promise of the build. verdict routes rendering: "wins" and "competitive" challengers keep full cards, "declined" ones render demoted after them (narrow, quiet, art as a labeled thumb, "Adopt anyway"), with their kept line on the front; the page reorders declined cards to the end on its own. raised on the assigned card renders each donation as a named raise line. Salience parity: when the assigned card declares no comp (no image generation this round), catalog art on every card demotes to a labeled thumb, so what looks important is the verdict’s call, never rendering luck. canonCard renders the standing exit as a subordinate card with the same anatomy; without it, canon stays a quiet footer action. Include canon only for visual-direction rounds; never present it as your own recommendation. The pick card is a kicker convention, not a field: kicker "IMPECCABLE’S PICK" on your top-ranked grounded candidate, one at most, never in the lead slot. Every card gets the full anatomy, challengers, canon, and declined included: thesis, palette, materials, viewport, risk; the seed already hands you each challenger’s system rules, so a card with no palette chips is an authoring gap, not a data gap. Keep thesis and each fact to one short sentence: the card front shows thesis, identity, and a two-line risk, while first viewport and the case read on the card back behind the Details chip, so long facts cost the reader a flip, not the page its scanability. A card with no imagery at all has no back; its full read renders on the front, so a text-only round loses nothing. A card may instead declare "wireframe" ({"cols":12,"rows":10,"regions":[{"label":"nav rail","x":0,"y":0,"w":3,"h":10,"accent":true}]}): the page draws it as a layout schematic in the media slot; surface-scope rounds use it on code-led builds, it never counts toward salience, and the card keeps its full read on the front. The comp slot carries the card’s full-fidelity direction comp (the legacy key "sketch" is accepted as an alias). Comp aspect follows the surface: portrait at device viewport for native or mobile-first surfaces, landscape otherwise; the page adapts its cards to either. reroll accepts true or { "registers": ["safer", "bolder"] }: the register buttons steer the next hand along the familiar-to-bold axis, the answer carries "register", and you re-run concept-seed with --register for the next round; offer the registers on direction rounds, and never pre-select one. buildPath rides the payload as { "value": "comp"|"code", "toggle": true }: the value is the recorded default (.impeccable/config.json buildPath, or .impeccable/config.local.json where one machine differs) and the toggle renders a footer switch whose flip binds that session only; the ANSWER then carries buildPath plus buildPathFlipped. On a code-led round each card still declares its comp path as a flip reserve: wireframes render, and a flip to comp makes --wait return once with BUILD PATH FLIPPED so you generate the comps into the declared slots while the round stays open; a flip back to code is free, and a comp that already landed stays as the critique reference. The toggle may only be offered when image generation exists: a harness with no image tool and no API key never sets toggle: true, so the choice never renders where comps cannot be made, and code-led simply rides as the untoggleable value. followup: true keeps the table open after a pick for a second round via --update; send the next payload immediately, the page is waiting on it.'); process.exit(0); } diff --git a/tests/context.test.mjs b/tests/context.test.mjs index e02934ed3..1378d123f 100644 --- a/tests/context.test.mjs +++ b/tests/context.test.mjs @@ -1060,6 +1060,42 @@ describe('context.mjs CLI', () => { assert.match(res.stdout, /detect\.mjs --json /); }); + // The build-path preference rides the unified config beside hook and + // detector settings. The local file wins because whether a machine can + // generate images is a property of that machine, not of the committed + // default the rest of the team shares. + describe('BUILD_PATH_DEFAULT', () => { + const run = () => spawnSync(process.execPath, [SCRIPT_PATH], { + cwd: scratch, + encoding: 'utf8', + env: { ...process.env, IMPECCABLE_NO_UPDATE_CHECK: '1', IMPECCABLE_NO_STALENESS_CHECK: '1' }, + }); + + it('reports a recorded preference from the shared config', () => { + write('PRODUCT.md', '# Acme\n'); + write('.impeccable/config.json', JSON.stringify({ buildPath: 'code' })); + assert.match(run().stdout, /BUILD_PATH_DEFAULT: code \(from \.impeccable\/config\.json\)/); + }); + + it('lets the gitignored local config win over the committed one', () => { + write('PRODUCT.md', '# Acme\n'); + write('.impeccable/config.json', JSON.stringify({ buildPath: 'comp' })); + write('.impeccable/config.local.json', JSON.stringify({ buildPath: 'code' })); + assert.match(run().stdout, /BUILD_PATH_DEFAULT: code \(from \.impeccable\/config\.local\.json\)/); + }); + + it('stays silent when nothing is recorded, leaving new-work its own default', () => { + write('PRODUCT.md', '# Acme\n'); + assert.equal(run().stdout.includes('BUILD_PATH_DEFAULT'), false); + }); + + it('stays silent on a value nothing reads rather than guessing at it', () => { + write('PRODUCT.md', '# Acme\n'); + write('.impeccable/config.json', JSON.stringify({ buildPath: 'code-first' })); + assert.equal(run().stdout.includes('BUILD_PATH_DEFAULT'), false); + }); + }); + it('keeps the manual-detector directive out of early context when the current provider hook is active', () => { const scripts = path.join(scratch, 'bundle', 'skills', 'impeccable', 'scripts'); stageContextBundle(scripts, { providerId: 'codex' }); diff --git a/tests/skill-behavior/README.md b/tests/skill-behavior/README.md index 1e904baab..d3ff2c5c4 100644 --- a/tests/skill-behavior/README.md +++ b/tests/skill-behavior/README.md @@ -86,17 +86,50 @@ stay here because they are the record of what a weaker model does with this text and that is the useful part. Reproduce with `IMPECCABLE_SKILL_BEHAVIOR_MODELS=gpt-5.6-luna,deepseek-v4-flash`. -Against the current default lineup, two cells are the known floor: -`redesign replaces DESIGN` is flaky, and `critique closes` is flaky on -gemini-3.6-flash. A regression is a failure beyond those two. +Against the current default lineup, three cells are the known floor: +`redesign replaces DESIGN` is flaky on every model, `critique closes` is flaky on +gemini-3.6-flash, and `initialized natural build` fails on claude-sonnet-5. +A regression is a failure beyond those three. | Scenario | claude-sonnet-5 | gpt-5.6-terra | gemini-3.6-flash | luna / deepseek (dropped) | |---|---|---|---|---| -| attended fresh init | not measured | not measured | not measured | not measured | -| initialized natural build | not measured | not measured | not measured | not measured | -| redesign replaces DESIGN | flaky | not measured | not measured | not measured | -| bolder refinement | not measured | not measured | pass (on 3.5) | luna pass, deepseek **fail** | -| critique closes | pass (2 of 2) | pass (2 of 2) | **flaky (1 of 3)** | luna **fail (1 of 6)**, deepseek flaky | +| attended fresh init | pass | pass | pass | not measured | +| initialized natural build | **fail (3 of 3)** | pass | pass | not measured | +| redesign replaces DESIGN | flaky (timeout this run) | **fail** | **fail (timeout)** | not measured | +| bolder refinement | pass | pass | pass (on 3.5) | luna pass, deepseek **fail** | +| critique closes | pass (4 of 4) | pass (3 of 3) | **flaky (1 of 4)** | luna **fail (1 of 6)**, deepseek flaky | + +Gemini's `bolder refinement` and `critique closes` runs in this sweep died on +`AI_APICallError` / `ETIMEDOUT` before completing a turn. Network failures are +not behavior measurements and are excluded from the counts above. + +## Scenario baseline (2026-08-13, current lineup) + +Measured on the same sweep. `scenarios.test.mjs` passes 15 of 15 on +gpt-5.6-terra and gemini-3.6-flash. Only claude-sonnet-5 fails anything, and +that asymmetry is the finding: the two cells below fail on the frontier model +while two weaker-on-paper lineups route correctly, so read them as a text +problem that one model's priors expose rather than as a model floor. + +| Scenario | claude-sonnet-5 | gpt-5.6-terra | gemini-3.6-flash | +|---|---|---|---| +| 1-7, 10, 12-14 | pass | pass | pass | +| 8 (SvelteKit exploration) | flaky | pass | pass | +| 9 (update surfaced, never auto-run) | **fail (3 of 3)** | pass | pass | +| 11 (shape resolves the build gate) | flaky | pass | pass | +| 15 (native audit variant) | **fail (3 of 3)** | pass | pass | + +**Scenarios 9 and 15 and `initialized natural build` fail on unmodified `main`.** +Confirmed against a clean worktree at `ddd23b18` with the same model and prompt: +same assertion, same shape. They are open defects in the current text, not +regressions from whatever change you are testing. Scenario 9 fails by auto-running +the skill update instead of surfacing it; scenario 15 loads `audit.md` where the +iOS platform should route it to `audit.native.md`; the natural-build contract +begins implementation before the attended concept checkpoint. Check them against +`main` before attributing any of the three to your branch. + +Scenarios 8 and 11 pass on re-run, so treat a single failure there as flake and +confirm with a second run before investigating. Gemini cells marked `on 3.5` were measured on the superseded `gemini-3.5-flash` and have not been re-run on 3.6. That distinction is not pedantic. `critique diff --git a/tests/staleness.test.mjs b/tests/staleness.test.mjs index 2b034d55f..5abb5ca44 100644 --- a/tests/staleness.test.mjs +++ b/tests/staleness.test.mjs @@ -23,6 +23,7 @@ import { stampProductSchema, } from '../skill/scripts/lib/artifact-schema.mjs'; import { + checkBuildPathUnset, checkConfig, checkDesignSidecar, checkNativePlatformEvidence, @@ -317,6 +318,64 @@ describe('checkConfig', () => { write('.impeccable/config.json', '{ not json'); assert.deepEqual(checkConfig({ projectRoot: scratch, repoRoot: scratch }), []); }); + + it('accepts both build-path values', () => { + for (const value of ['comp', 'code']) { + write('.impeccable/config.json', JSON.stringify({ buildPath: value })); + assert.deepEqual(checkConfig({ projectRoot: scratch, repoRoot: scratch }), []); + } + }); + + it('flags a build-path value nothing reads', () => { + write('.impeccable/config.json', JSON.stringify({ buildPath: 'code-first' })); + const findings = checkConfig({ projectRoot: scratch, repoRoot: scratch }); + assert.deepEqual(ids(findings), ['config-invalid-build-path']); + assert.match(findings[0].summary, /"code-first"/); + }); +}); + +// ─── build path ──────────────────────────────────────────────────────────── + +describe('checkBuildPathUnset', () => { + const args = () => ({ projectRoot: scratch, repoRoot: scratch, product: '# Product' }); + + it('flags a project that has done direction work and recorded nothing', () => { + fs.mkdirSync(path.join(scratch, '.impeccable', 'surfaces'), { recursive: true }); + const findings = checkBuildPathUnset(args()); + assert.deepEqual(ids(findings), ['config-build-path-unset']); + // The one precondition this module cannot see must reach the reader that can. + assert.match(findings[0].fix, /image generation/); + }); + + it('accepts decision mocks as the same evidence', () => { + fs.mkdirSync(path.join(scratch, '.impeccable', 'mocks', 'decision'), { recursive: true }); + assert.deepEqual(ids(checkBuildPathUnset(args())), ['config-build-path-unset']); + }); + + it('stays silent on a project that has never done direction work', () => { + assert.deepEqual(checkBuildPathUnset(args()), []); + }); + + it('stays silent without a product record, where init already routes', () => { + fs.mkdirSync(path.join(scratch, '.impeccable', 'surfaces'), { recursive: true }); + assert.deepEqual(checkBuildPathUnset({ ...args(), product: null }), []); + }); + + it('stays silent once a value is recorded, in either config file', () => { + fs.mkdirSync(path.join(scratch, '.impeccable', 'surfaces'), { recursive: true }); + write('.impeccable/config.json', JSON.stringify({ buildPath: 'code' })); + assert.deepEqual(checkBuildPathUnset(args()), []); + + fs.rmSync(path.join(scratch, '.impeccable', 'config.json')); + write('.impeccable/config.local.json', JSON.stringify({ buildPath: 'comp' })); + assert.deepEqual(checkBuildPathUnset(args()), []); + }); + + it('defers to the invalid-value finding rather than reporting the key twice', () => { + fs.mkdirSync(path.join(scratch, '.impeccable', 'surfaces'), { recursive: true }); + write('.impeccable/config.json', JSON.stringify({ buildPath: 'comp-first' })); + assert.deepEqual(checkBuildPathUnset(args()), []); + }); }); // ─── surface briefs ────────────────────────────────────────────────────────