mirror of
https://github.com/pbakaus/impeccable.git
synced 2026-09-12 06:06:37 +03:00
Merge pull request #579 from pbakaus/fix/build-path-config-migration
Build path becomes a config key existing projects can actually reach
This commit is contained in:
Paul Bakaus
@@ -361,6 +361,18 @@ On an interactive `install`/`update`, Impeccable explains the hook and offers to
|
||||
|
||||
For debugging, set `hook.auditLog` in `.impeccable/config.json` to a path (or the legacy `IMPECCABLE_HOOK_LOG` env var) to write one NDJSON line per hook invocation. Leave it unset for normal use.
|
||||
|
||||
## Build path: comp-first or code-first
|
||||
|
||||
When a new surface gets designed, Impeccable either generates a full-fidelity comp first and builds to match it, or builds straight in code with the ambition written into the direction contract and checked at the finish. Comp-first composes bolder and takes longer; code-first is leaner and faster. `/impeccable init` asks once and records the answer as `buildPath` in `.impeccable/config.json`:
|
||||
|
||||
```json
|
||||
{ "buildPath": "comp" }
|
||||
```
|
||||
|
||||
The values are `comp` and `code`, and nothing else is read. Set it in the gitignored `.impeccable/config.local.json` to override the team's committed value on one machine, which is what you want when your harness has no image generation. In a monorepo, commit it once at the repo root and any workspace that wants something else sets its own. The choice appears at all only where image generation is available, since without it there is nothing to comp.
|
||||
|
||||
You do not have to re-run `init` to set it on a project that predates the setting, and you do not have to edit the file by hand either. Whatever is recorded is a default rather than a lock: every decision page carries a footer toggle, and flipping it binds that session only. Flip it on a project that has recorded nothing and Impeccable asks once, after the round, whether to keep it, then writes your answer. That is the whole migration path for an existing project: use the toggle when the default is wrong, and answer the question that follows.
|
||||
|
||||
Codex requires one platform step that Impeccable cannot safely skip: open `/hooks` after install or update and approve the project hook. There is no Codex marketplace/plugin install flow for this hook.
|
||||
|
||||
Full hook docs: [impeccable.style/docs/hooks](https://impeccable.style/docs/hooks).
|
||||
|
||||
@@ -311,7 +311,15 @@ export const SUITES = {
|
||||
commands: [
|
||||
{
|
||||
runner: 'node',
|
||||
timeoutMs: 300000,
|
||||
// 300000 was too low to measure what these scenarios assert. The
|
||||
// workflow-contract turns run 20+ steps against a frontier model, and
|
||||
// the *correct* path is the slow one: a run that stops to put the
|
||||
// concept to the user before building was measured at 579s, while the
|
||||
// runs that skipped that checkpoint and failed the assertion finished
|
||||
// in 130-200s. At a 300s cap the thorough path is killed and the hasty
|
||||
// path is graded, so the cap was selecting for the behavior the suite
|
||||
// exists to forbid.
|
||||
timeoutMs: 900000,
|
||||
files: [
|
||||
'tests/skill-behavior/scenarios.test.mjs',
|
||||
'tests/skill-behavior/workflow-contract.test.mjs',
|
||||
|
||||
@@ -46,6 +46,7 @@ The same restraint applies to `workspace-context-inherited`. Inheritance is a de
|
||||
|
||||
- `workspace-platform-native-evidence` is the finding that matters most here: a workspace carrying native build files while inheriting a root record that resolves to web gets web guidance for its whole life and never loads [ios.md](ios.md) or [android.md](android.md). The repair is a child PRODUCT.md in that workspace, because one inherited record cannot hold two platforms.
|
||||
- `config-project-roots-match-nothing` means every `projectRoots` glob missed, so the repo root is silently standing in as the active project. A renamed workspace directory is the usual cause. Report the patterns and ask which directories they should name.
|
||||
- `config-invalid-build-path` and `config-build-path-unset` both concern one key, `buildPath` in `.impeccable/config.json` (or the gitignored `.impeccable/config.local.json`, which wins for that developer). It holds `comp` or `code` and sets whether new surfaces are built from a generated comp or straight in code. An unread value does not fall back to the opposite path, so a project meaning `code` has been building comp-led; report the exact value. The unset finding fires only where a project has done direction work and never recorded a preference, and the offer belongs in it only when image generation exists in your tool surface. Without image generation there is nothing to choose and nothing to say.
|
||||
- Use the `workspaces` table to show the user which apps carry their own context, which inherit, and which have none, before proposing any change.
|
||||
|
||||
## Opting out of the boot check
|
||||
|
||||
@@ -109,7 +109,7 @@ Before loading new-work or resuming shape/build, verify that PRODUCT.md exists a
|
||||
|
||||
## Step 5: Record workflow defaults
|
||||
|
||||
When image generation is available (context.mjs reports it), ask once how new surfaces should be built, stated as the trade it is: **comp-first** (an image sets the bar before any code; bolder composition, slower, and the build must match the image) or **code-first** (build directly; the ambition is written into the direction contract and audited at the finish; leaner, faster). Write the answer to `.impeccable/settings.json` as `{ "buildPath": "comp" }` or `{ "buildPath": "code" }`, merging with any keys already there. This is a default, not a lock: the decision page renders a toggle whose flip binds a single session and is never written back. Without image generation there is no choice to record; code-first is the only path.
|
||||
When image generation is available (context.mjs reports it) and no `buildPath` is recorded yet, ask once how new surfaces should be built, stated as the trade it is: **comp-first** (an image sets the bar before any code; bolder composition, slower, and the build must match the image) or **code-first** (build directly; the ambition is written into the direction contract and audited at the finish; leaner, faster). Write the answer to `.impeccable/config.json` as `"buildPath": "comp"` or `"buildPath": "code"`, merging with the keys already there. A value already recorded in `.impeccable/config.json` or the gitignored `.impeccable/config.local.json` is a confirmed answer: on a re-run, honor it in silence rather than asking again. This is a default, not a lock: the decision page renders a toggle whose flip binds a single session and is never written back. Without image generation there is no choice to record; code-first is the only path.
|
||||
|
||||
Then configure live mode when useful: skip native or non-runnable projects and leave existing config untouched. Otherwise follow [live.md](live.md)'s first-time setup. Any CSP source edit still requires its stated consent.
|
||||
|
||||
|
||||
@@ -36,7 +36,7 @@ Keep the visual system fixed. Derive five to seven materially different structur
|
||||
|
||||
`node {{scripts_path}}/concept-seed.mjs --scope surface --mode <mode>`
|
||||
|
||||
The script deals three of your structures to the table; the dice decide which three reach the user, so the ranking rut stays broken while the user still holds a real choice. Present the three dealt structures on the decision page as full cards of equal salience, the dealt lead carrying kicker THE ROLL, with steer and re-roll; the user locks one in. No canon card and no pick card at surface scope: the world is settled, so every card visualizes composition, not identity. With image generation available and a comp-led default (the build-path paragraph below: `.impeccable/settings.json`, the toggle handles the exception), each card declares a `comp` under `.impeccable/mocks/decision/`, generated after serving in reading order under the comp discipline in [visualize.md](visualize.md); anchor each of these comps on the established identity by passing a captured screenshot of a representative existing page as a reference image (the harness image tool's input image, or `generate-image.mjs --ref`) beside a prompt that leads with the new surface's structure and names DESIGN.md's palette, type, and component character, because a prose paraphrase of a design system drifts where a pixel reference does not. Without image generation, or under a code-led default, each card instead carries a `wireframe` layout schematic (see `serve-question.mjs --schema`) that the page draws itself. Locking a card is the approval and sets the build path: a locked comp builds comp-led with that comp as the approved comp, discharging [visualize.md](visualize.md)'s three-option round with no second approval point; a locked wireframe builds code-led, its ambition carried by the direction contract. Never run the script for a local extension or a precisely specified narrow request; shape those directly.
|
||||
The script deals three of your structures to the table; the dice decide which three reach the user, so the ranking rut stays broken while the user still holds a real choice. Present the three dealt structures on the decision page as full cards of equal salience, the dealt lead carrying kicker THE ROLL, with steer and re-roll; the user locks one in. No canon card and no pick card at surface scope: the world is settled, so every card visualizes composition, not identity. With image generation available and a comp-led default (the build-path paragraph below: `.impeccable/config.json`, the toggle handles the exception), each card declares a `comp` under `.impeccable/mocks/decision/`, generated after serving in reading order under the comp discipline in [visualize.md](visualize.md); anchor each of these comps on the established identity by passing a captured screenshot of a representative existing page as a reference image (the harness image tool's input image, or `generate-image.mjs --ref`) beside a prompt that leads with the new surface's structure and names DESIGN.md's palette, type, and component character, because a prose paraphrase of a design system drifts where a pixel reference does not. Without image generation, or under a code-led default, each card instead carries a `wireframe` layout schematic (see `serve-question.mjs --schema`) that the page draws itself. Locking a card is the approval and sets the build path: a locked comp builds comp-led with that comp as the approved comp, discharging [visualize.md](visualize.md)'s three-option round with no second approval point; a locked wireframe builds code-led, its ambition carried by the direction contract. Never run the script for a local extension or a precisely specified narrow request; shape those directly.
|
||||
|
||||
### Create or replace the visual world
|
||||
|
||||
@@ -50,7 +50,7 @@ The standing exit: every direction round offers one quiet, permanent alternative
|
||||
|
||||
When image generation exists, every card also declares a `comp` path under `.impeccable/mocks/decision/`, the canon card included. Where the harness sandboxes its shell, start the page through the least-sandboxed command path it offers: a sandboxed shell cannot bind the board's port, and the first-attempt failure costs a retry every session. Serve the page first, then produce the comps; the page shimmer-waits per slot and the user may answer before they land. Each card's image is that direction's north-star comp at full fidelity, produced under the comp discipline in [visualize.md](visualize.md): the requested surface's first viewport, structure-led prompt, real product name and real content, no invented commercial claims, in that card's own palette, type character, and material world, committed all the way; visualize.md's self-checks bind decision comps identically. Generation takes the same time at any fidelity, so an unfinished draft pays draft quality for comp cost; fairness between cards comes from equal fidelity in each card's own grammar, one surface, one aspect, never from shared unfinishedness. The frame's aspect is the surface's own: a native app or mobile-first surface comps portrait at its device viewport, a desktop web surface landscape, and the decision page adapts to either, so a phone screen comped landscape is a broken frame, not a neutral default. Produce in the order the user reads, the assigned card, then the pick, then the full-card hand, then canon, each file written with its prompt sidecar the moment it is done, so a re-roll's spend front-loads onto the cards read first; declined challengers get no comp, their catalog thumb is their face. When the harness runs subagents in parallel, fan the set out as one agent per card: each spawn is the shipped asset producer with a single-comp packet, that card's fields, PRODUCT.md, the shared frame, and the card's declared path, up to four in flight at once. A slot still empty when its agent returns is regenerated inline, and a slot still empty when the user answers is dropped without ceremony; no other supervision is owed. Without parallel subagents, generate in the main thread after serving, in the same reading order, and let the harness's own generation display carry the progress; the wait for the answer follows the last file. The chosen card's comp is not spent by the choice: on a comp-led build it enters the comp round as compositional option one, and on a code-led build it returns at the finish review as the critique reference, what the image dared that the build did not. The unchosen comps stay in `.impeccable/mocks/decision/` as the round's spent hand; they carry no approval and imply none. With no image generation, the cards carry their identity in palette chips and facts, and that page is complete, not a lesser version; the page then also demotes every challenger's catalog art to a labeled thumbnail on its own, because salience must encode the verdict, never the accident of which cards have images. <!-- rule:skill-decision-comps-full-fidelity --> <!-- rule:skill-salience-parity -->
|
||||
|
||||
The execution contract, comp-led or code-led, is a workflow preference, not a per-surface decision, so no round asks it: the recorded default rides every round and the page's toggle handles the exception. Read the default from `.impeccable/settings.json` (`buildPath`), falling back to a standing brand commitment in PRODUCT.md recorded before settings existed; with neither, comp-led is the default whenever image generation exists. Author every direction and surface payload with `buildPath: { "value": <default>, "toggle": true }`; the page renders a footer toggle with the trade stated beside it, and the ANSWER returns `buildPath` plus `buildPathFlipped`. A flipped value binds that session only and is never written back; when the user asks in words to change the standing default, update `.impeccable/settings.json`. **Comp-led**: the chosen card's comp is law, generated before building when it does not exist yet, and the finish review audits the build against it; boldest composition on the table, fix rounds expected; comp-led makes the comp non-optional, no silent skipping. **Code-led**: no comp of this page and no apology for it; the QUALITY BAR boards still calibrate finish, and the ambition moves into the written contract, the FIRST VIEWPORT block plus a named signature interaction and motion grammar, which the finish reviewer audits in behavior; code-led is not a discount on commitment, the direction still lands fully committed in code. A code-led round still declares each card's comp path as a flip reserve: when the user flips the toggle to comp mid-round, `--wait` returns once with BUILD PATH FLIPPED while the page shimmers the slots; generate each open card's comp into its declared path then, lead first, and wait again. The flip back is free, and a comp that already rendered rides at the finish review as the critique reference. Without image generation there is no toggle and no choice: code-led is the only path, stated in one line rather than asked. The old two-card execution-contract round is retired; `followup: true` remains the general mechanism for delivering any later round over the same table via `--update`. <!-- rule:skill-build-path-round -->
|
||||
The execution contract, comp-led or code-led, is a workflow preference, not a per-surface decision, so no round asks it: the recorded default rides every round and the page's toggle handles the exception. Read the default from `.impeccable/config.json` (`buildPath`), with the gitignored `.impeccable/config.local.json` winning where one machine differs from the team's committed value; with neither, comp-led is the default whenever image generation exists. Author every direction and surface payload with `buildPath: { "value": <default>, "toggle": true }`; the page renders a footer toggle with the trade stated beside it, and the ANSWER returns `buildPath` plus `buildPathFlipped`. A flipped value binds that session only and is never written back, with one exception, and it is the only thing inside a round that earns a question about this preference (init records it up front on projects that get the chance): when `buildPathFlipped` comes back true on a project that records no `buildPath` at all, ask once after the round closes whether to keep it as the standing default. Either answer ends in a write to `.impeccable/config.json`; the answer picks the value, never whether to record one. Yes writes the flipped value, and "no, just this once" writes the value they flipped away from, which is the standing default they just confirmed by declining. Ask on the flip and never on the untouched default, because a user who left the toggle alone has told you nothing. A declined offer nothing writes down is an offer the next session makes again. When the user asks in words to change the standing default, update the file without asking. **Comp-led**: the chosen card's comp is law, generated before building when it does not exist yet, and the finish review audits the build against it; boldest composition on the table, fix rounds expected; comp-led makes the comp non-optional, no silent skipping. **Code-led**: no comp of this page and no apology for it; the QUALITY BAR boards still calibrate finish, and the ambition moves into the written contract, the FIRST VIEWPORT block plus a named signature interaction and motion grammar, which the finish reviewer audits in behavior; code-led is not a discount on commitment, the direction still lands fully committed in code. A code-led round still declares each card's comp path as a flip reserve: when the user flips the toggle to comp mid-round, `--wait` returns once with BUILD PATH FLIPPED while the page shimmers the slots; generate each open card's comp into its declared path then, lead first, and wait again. The flip back is free, and a comp that already rendered rides at the finish review as the critique reference. Without image generation there is no toggle and no choice: code-led is the only path, stated in one line rather than asked. The old two-card execution-contract round is retired; `followup: true` remains the general mechanism for delivering any later round over the same table via `--update`. <!-- rule:skill-build-path-round -->
|
||||
|
||||
Catalog worlds are working systems, not mood references. When one survives, carry its palette and material, type and composition, topology, controls and state, and responsive rules into the product. When the source is itself an interface language, commit to its native grammar across navigation, content, controls, and states. Open the QUALITY BAR board and hero for the world you build the moment the choice lands, even if you viewed another card earlier; the ANSWER line names the chosen card's images (when the harness only reads files or runs sandboxed, download them into the workspace and open the relative path; sandboxed viewers reject absolute paths outside it). They set the craft level the build must reach, a rendered reference's finish, commitment, and art direction, never the composition; your surface serves this product.
|
||||
|
||||
|
||||
@@ -434,7 +434,7 @@ export function renderConceptSeed({
|
||||
lead carrying kicker THE ROLL, with steer and re-roll, and let the user
|
||||
lock one in; the world is already settled, so this choice is composition.
|
||||
Visualize every dealt card: with image generation available and a
|
||||
comp-led default (.impeccable/settings.json buildPath; the page toggle
|
||||
comp-led default (.impeccable/config.json buildPath; the page toggle
|
||||
handles the exception), declare a comp per card and generate after
|
||||
serving, lead first; otherwise author each card's wireframe field (see
|
||||
serve-question --schema) and the page draws the schematic. Carry the
|
||||
|
||||
+57
-15
@@ -1013,14 +1013,22 @@ async function fetchLatestSkillVersion() {
|
||||
}
|
||||
}
|
||||
|
||||
// Two instructions used to sit in one directive: ask, and "if they agree, run
|
||||
// it". Nothing gated the second on an answer, and the same sentence said to
|
||||
// continue without waiting, so a run that could never establish agreement was
|
||||
// still spelled out as the next command. The offer stays; the command leaves
|
||||
// this turn entirely, because installing over the skill mid-session changes
|
||||
// files the session is reading and only takes effect in the next one anyway.
|
||||
function buildUpdateDirective(localVersion, latestVersion) {
|
||||
return (
|
||||
`UPDATE_AVAILABLE: A newer Impeccable skill is available ` +
|
||||
`(installed v${localVersion}, latest v${latestVersion}). ` +
|
||||
`Before continuing, ask the user once: "A newer Impeccable (v${latestVersion}) is available. ` +
|
||||
`Mention it once, in this form: "A newer Impeccable (v${latestVersion}) is available. ` +
|
||||
`Update now? It runs \`npx impeccable update\`." ` +
|
||||
`If they agree, run \`npx impeccable update\` (the update applies to the next session, not this one). ` +
|
||||
`Either way, continue the current task without waiting, and do not raise this again.`
|
||||
`Do not run \`npx impeccable update\` in this turn, whatever the user answers: it rewrites the skill files ` +
|
||||
`this session is reading, and the update only takes effect in the next session, so there is nothing to gain now. ` +
|
||||
`Run it in a later turn, only after the user has asked for it in their own words. ` +
|
||||
`Continue the current task now without waiting, and do not raise this again.`
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1142,7 +1150,7 @@ async function cli() {
|
||||
parts.push(buildResolvedContextDirective(ctx, cliOptions, { targetExists }));
|
||||
appendDetectorFallback(parts, ctx);
|
||||
appendImageGenDirective(parts);
|
||||
appendBuildPathDirective(parts);
|
||||
appendBuildPathDirective(parts, ctx);
|
||||
appendAutonomyCounterDirective(parts);
|
||||
appendSubagentAuthorizationDirective(parts);
|
||||
if (shouldWarnMissingTarget(ctx, targetProvided, targetExists)) {
|
||||
@@ -1162,7 +1170,7 @@ async function cli() {
|
||||
parts.push(buildResolvedContextDirective(ctx, cliOptions, { targetExists }));
|
||||
appendDetectorFallback(parts, ctx);
|
||||
appendImageGenDirective(parts);
|
||||
appendBuildPathDirective(parts);
|
||||
appendBuildPathDirective(parts, ctx);
|
||||
appendAutonomyCounterDirective(parts);
|
||||
appendSubagentAuthorizationDirective(parts);
|
||||
if (shouldWarnMissingTarget(ctx, targetProvided, targetExists)) {
|
||||
@@ -1271,17 +1279,51 @@ function automaticHookMode(ctx) {
|
||||
}
|
||||
|
||||
|
||||
// Build-path preference: a workflow setting (comp-led vs code-led), read
|
||||
// here so every session starts knowing it without a file hunt. Absence
|
||||
// stays silent; new-work's own default applies, and the decision page
|
||||
// toggle can flip the value for a single session.
|
||||
function appendBuildPathDirective(parts) {
|
||||
try {
|
||||
const settings = JSON.parse(fs.readFileSync(path.join(process.cwd(), '.impeccable', 'settings.json'), 'utf8'));
|
||||
if (settings.buildPath === 'comp' || settings.buildPath === 'code') {
|
||||
parts.push(`BUILD_PATH_DEFAULT: ${settings.buildPath} (from .impeccable/settings.json). Author direction and surface rounds with this as buildPath.value and toggle: true; a flip on the page binds that session only and is never written back to settings.`);
|
||||
// Build-path preference: a workflow setting (comp-led vs code-led), read here
|
||||
// so every session starts knowing it without a file hunt. It rides the unified
|
||||
// config beside the hook and detector settings, and the gitignored local file
|
||||
// wins, because whether a machine has an image tool is a property of that
|
||||
// machine, not of the team's committed default. Absence stays silent;
|
||||
// new-work's own default applies, and the decision page toggle can flip the
|
||||
// value for a single session.
|
||||
function readBuildPathAt(root) {
|
||||
let value = null;
|
||||
let source = null;
|
||||
for (const name of ['config.json', 'config.local.json']) {
|
||||
const raw = readJson(path.join(root, '.impeccable', name));
|
||||
if (raw?.buildPath === 'comp' || raw?.buildPath === 'code') {
|
||||
value = raw.buildPath;
|
||||
source = `.impeccable/${name}`;
|
||||
}
|
||||
} catch { /* no settings file */ }
|
||||
}
|
||||
return value ? { value, source } : null;
|
||||
}
|
||||
|
||||
// Roots in precedence order, nearest first: the resolved project decides, and
|
||||
// the repo root is the fallback a monorepo commits once for every app in it.
|
||||
// `checkBuildPathUnset` reads exactly these two, and the pair has to match:
|
||||
// when they disagree the finding goes silent because a value exists while the
|
||||
// directive never names it, which is the one combination nobody can debug.
|
||||
//
|
||||
// The invoking directory is deliberately not in the chain. With `--target`
|
||||
// selecting another workspace, cwd is the caller's app, not the target's, and
|
||||
// letting it rank above the repo root hands one workspace another's workflow.
|
||||
// It stands in only when no project resolved at all.
|
||||
function appendBuildPathDirective(parts, ctx) {
|
||||
const roots = [...new Set(
|
||||
[ctx?.projectRoot || process.cwd(), ctx?.repoRoot].filter(Boolean).map((root) => path.resolve(root)),
|
||||
)];
|
||||
for (const root of roots) {
|
||||
const found = readBuildPathAt(root);
|
||||
if (!found) continue;
|
||||
// "Never written back" is scoped by the fact that this directive exists at
|
||||
// all: it is emitted only where a value is already recorded, which is the
|
||||
// case where a flip really is session-only. Saying so inline because the
|
||||
// bare absolute reads as a rule that overrides new-work's one-time offer,
|
||||
// which is exactly how the same wording misfired in serve-question.
|
||||
parts.push(`BUILD_PATH_DEFAULT: ${found.value} (from ${found.source}). Author direction and surface rounds with this as buildPath.value and toggle: true; a flip on the page binds that session only and is never written back, because a default is already recorded here. New-work's one-time offer to record a flipped value applies only where no default exists, which is why you are not seeing this line on those projects.`);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// Image generation availability: harness-native tools always win, but when the
|
||||
|
||||
@@ -33,6 +33,7 @@ import {
|
||||
stampProductSchema,
|
||||
} from './lib/artifact-schema.mjs';
|
||||
import {
|
||||
checkBuildPathUnset,
|
||||
checkConfig,
|
||||
checkDesignSidecar,
|
||||
checkNativePlatformEvidence,
|
||||
@@ -120,6 +121,7 @@ async function collect(cwd, targetOptions) {
|
||||
...checkDesignDrift({ designPath: absDesignPath, projectRoot }),
|
||||
...checkDesignCoverage({ design: ctx.design, designPath: ctx.designPath, parseDesignMd }),
|
||||
...checkConfig({ projectRoot, repoRoot: ctx.repoRoot }),
|
||||
...checkBuildPathUnset({ projectRoot, repoRoot: ctx.repoRoot, product: ctx.product }),
|
||||
...checkDetectorIgnores({ projectRoot, knownRuleIds }),
|
||||
...checkSurfaceBriefs({ candidates: ctx.surfaceBriefCandidates, projectRoot }),
|
||||
...checkHookInstallation({
|
||||
|
||||
@@ -47,18 +47,33 @@ import {
|
||||
|
||||
// Top-level keys any reader honors: `hook` and `detector` subtrees (hook-lib's
|
||||
// readConfig), `updateCheck` (context.mjs), `projectRoots` (context.mjs's
|
||||
// monorepo resolution), plus `stalenessCheck` below. `$schema` and `version`
|
||||
// are allowed as conventional metadata nobody reads.
|
||||
// monorepo resolution), `buildPath` (context.mjs's build-path directive), plus
|
||||
// `stalenessCheck` below. `$schema` and `version` are allowed as conventional
|
||||
// metadata nobody reads.
|
||||
const KNOWN_CONFIG_KEYS = new Set([
|
||||
'hook',
|
||||
'detector',
|
||||
'updateCheck',
|
||||
'stalenessCheck',
|
||||
'projectRoots',
|
||||
'buildPath',
|
||||
'$schema',
|
||||
'version',
|
||||
]);
|
||||
|
||||
// The only two values context.mjs and new-work honor. A near miss reads as a
|
||||
// working preference and silently rides the opposite path, so it is worth
|
||||
// reporting rather than coercing.
|
||||
const BUILD_PATH_VALUES = Object.freeze(['comp', 'code']);
|
||||
|
||||
// Evidence that this project does the kind of work `buildPath` governs. A
|
||||
// project that only ever ran polish or audit has no use for the setting and
|
||||
// should never be told it exists. Two stats, so Tier 1 can afford it.
|
||||
const DIRECTION_WORK_PATHS = Object.freeze([
|
||||
path.join('.impeccable', 'surfaces'),
|
||||
path.join('.impeccable', 'mocks', 'decision'),
|
||||
]);
|
||||
|
||||
// `detector` is a closed set, so a typo here is worth reporting. `hook` is not
|
||||
// checked: it carries runtime settings from several writers and the false
|
||||
// positive rate would outweigh the catch.
|
||||
@@ -325,6 +340,20 @@ export function checkConfig({ projectRoot, repoRoot }) {
|
||||
}));
|
||||
}
|
||||
|
||||
if (Object.prototype.hasOwnProperty.call(raw, 'buildPath')
|
||||
&& !BUILD_PATH_VALUES.includes(raw.buildPath)) {
|
||||
findings.push(finding({
|
||||
id: 'config-invalid-build-path',
|
||||
artifact: 'config.json',
|
||||
filePath: rel,
|
||||
severity: 'mention',
|
||||
summary: `${rel} sets \`buildPath\` to ${JSON.stringify(raw.buildPath)}, which nothing reads. `
|
||||
+ `The values are ${BUILD_PATH_VALUES.map((value) => `\`${value}\``).join(' and ')}.`,
|
||||
fix: 'Report the value. An unread `buildPath` does not fall back to the other path; '
|
||||
+ 'it falls back to the default, so a project meaning `code` has been building comp-led.',
|
||||
}));
|
||||
}
|
||||
|
||||
const detector = raw.detector;
|
||||
if (detector && typeof detector === 'object' && !Array.isArray(detector)) {
|
||||
const unknownDetector = Object.keys(detector).filter((key) => !KNOWN_DETECTOR_KEYS.has(key));
|
||||
@@ -345,6 +374,47 @@ export function checkConfig({ projectRoot, repoRoot }) {
|
||||
return findings;
|
||||
}
|
||||
|
||||
/**
|
||||
* No recorded build-path preference on a project that plainly does visual
|
||||
* direction work. Not drift in the usual sense: the setting is newer than the
|
||||
* project, so every project that predates it lands here at once. That is why
|
||||
* it is gated twice, on a product record and on evidence of the work the
|
||||
* setting governs, and why it says the choice rather than assuming a harness
|
||||
* can make it. Image generation is the real precondition and this module
|
||||
* cannot see it: a harness-native image tool leaves no trace on disk, so the
|
||||
* finding hands the question to the one reader that knows.
|
||||
*/
|
||||
export function checkBuildPathUnset({ projectRoot, repoRoot, product }) {
|
||||
if (!projectRoot || !product) return [];
|
||||
const roots = [...new Set([projectRoot, repoRoot].filter(Boolean).map((root) => path.resolve(root)))];
|
||||
|
||||
for (const root of roots) {
|
||||
for (const name of ['config.json', 'config.local.json']) {
|
||||
const raw = readJson(path.join(root, '.impeccable', name));
|
||||
// Any declared value ends this, valid or not: an invalid one already has
|
||||
// its own finding and two reports of one key is noise.
|
||||
if (raw && Object.prototype.hasOwnProperty.call(raw, 'buildPath')) return [];
|
||||
}
|
||||
}
|
||||
|
||||
const evidence = DIRECTION_WORK_PATHS.filter((rel) => fs.existsSync(path.join(projectRoot, rel)));
|
||||
if (!evidence.length) return [];
|
||||
|
||||
return [finding({
|
||||
id: 'config-build-path-unset',
|
||||
artifact: 'config.json',
|
||||
filePath: '.impeccable/config.json',
|
||||
severity: 'mention',
|
||||
summary: 'This project has run visual direction work but records no `buildPath`, '
|
||||
+ 'so every direction round takes the comp-first default without anyone having chosen it.',
|
||||
fix: 'Only when image generation exists in your tool surface, offer the choice once: '
|
||||
+ '**comp-first** (an image sets the bar before any code; bolder composition, slower) or '
|
||||
+ '**code-first** (build directly; ambition carried by the direction contract; leaner, faster). '
|
||||
+ 'Write the answer to `.impeccable/config.json` as `"buildPath": "comp"` or `"buildPath": "code"`, '
|
||||
+ 'merging with the keys already there. Without image generation there is no choice to record: stay silent.',
|
||||
})];
|
||||
}
|
||||
|
||||
// ─── Surface briefs ────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
@@ -446,6 +516,7 @@ export function collectBootFindings(ctx, extras = {}) {
|
||||
projectRoot,
|
||||
}),
|
||||
...checkConfig({ projectRoot, repoRoot: ctx.repoRoot }),
|
||||
...checkBuildPathUnset({ projectRoot, repoRoot: ctx.repoRoot, product: ctx.product }),
|
||||
...checkSurfaceBriefs({ candidates: ctx.surfaceBriefCandidates, projectRoot }),
|
||||
...(extras.projectRootPatterns
|
||||
? checkProjectRoots({
|
||||
|
||||
@@ -161,8 +161,12 @@ function printAnswer(raw) {
|
||||
console.log('FOLLOWUP OPEN: the table stays open and the page is showing a loading hand. Deliver the next round now with --update --key <key> --payload <file>, then collect it with --wait; never leave the page waiting on a round you have not sent.');
|
||||
}
|
||||
if (a.buildPath === 'comp' || a.buildPath === 'code') {
|
||||
// The page never writes the flip itself, but "never write it" overstated
|
||||
// that into a rule the agent then applied to new-work's one-time offer,
|
||||
// which exists for exactly this case: a flip on a project that had no
|
||||
// recorded default is the only moment the preference is ever asked for.
|
||||
const origin = a.buildPathFlipped
|
||||
? 'flipped on the page, so it binds this session only; never write it to settings'
|
||||
? 'flipped on the page, so it binds this session only, and the page never writes it back; the sole exception is new-work’s one-time offer, on a project that had no recorded default at all, which asks after the round closes and writes the answer to .impeccable/config.json'
|
||||
: 'the round’s recorded default';
|
||||
console.log(`BUILD PATH: ${a.buildPath} (${origin}). ${a.buildPath === 'comp'
|
||||
? 'Comp-led: the chosen card’s comp is law; generate it before building when it does not exist yet, and the finish review audits the build against it.'
|
||||
@@ -199,7 +203,7 @@ if (hasFlag('schema')) {
|
||||
canonCard: { label: 'The category standard', thesis: 'What this category ships, executed impeccably.', palette: ['#ffffff', '#111827', '#2563eb'], materials: ['clean grid', 'product photography'], viewport: 'The arrangement a visitor expects, at full craft.', risk: 'Indistinguishable from the competition by design.', comp: '.impeccable/mocks/decision/canon.webp' },
|
||||
steer: true,
|
||||
}, null, 2));
|
||||
console.log('\nOption ids return verbatim in ANSWER; "reroll" and "canon" are reserved. hero/board/comp accept URLs or local paths; comp slots may point at files that do not exist yet (serve first, generate after; the page polls until they land, so never block serving on generation). hero on a challenger is the inspiration it draws from and renders picture-in-picture beside the comp, never as the promise of the build. verdict routes rendering: "wins" and "competitive" challengers keep full cards, "declined" ones render demoted after them (narrow, quiet, art as a labeled thumb, "Adopt anyway"), with their kept line on the front; the page reorders declined cards to the end on its own. raised on the assigned card renders each donation as a named raise line. Salience parity: when the assigned card declares no comp (no image generation this round), catalog art on every card demotes to a labeled thumb, so what looks important is the verdict’s call, never rendering luck. canonCard renders the standing exit as a subordinate card with the same anatomy; without it, canon stays a quiet footer action. Include canon only for visual-direction rounds; never present it as your own recommendation. The pick card is a kicker convention, not a field: kicker "IMPECCABLE’S PICK" on your top-ranked grounded candidate, one at most, never in the lead slot. Every card gets the full anatomy, challengers, canon, and declined included: thesis, palette, materials, viewport, risk; the seed already hands you each challenger’s system rules, so a card with no palette chips is an authoring gap, not a data gap. Keep thesis and each fact to one short sentence: the card front shows thesis, identity, and a two-line risk, while first viewport and the case read on the card back behind the Details chip, so long facts cost the reader a flip, not the page its scanability. A card with no imagery at all has no back; its full read renders on the front, so a text-only round loses nothing. A card may instead declare "wireframe" ({"cols":12,"rows":10,"regions":[{"label":"nav rail","x":0,"y":0,"w":3,"h":10,"accent":true}]}): the page draws it as a layout schematic in the media slot; surface-scope rounds use it on code-led builds, it never counts toward salience, and the card keeps its full read on the front. The comp slot carries the card’s full-fidelity direction comp (the legacy key "sketch" is accepted as an alias). Comp aspect follows the surface: portrait at device viewport for native or mobile-first surfaces, landscape otherwise; the page adapts its cards to either. reroll accepts true or { "registers": ["safer", "bolder"] }: the register buttons steer the next hand along the familiar-to-bold axis, the answer carries "register", and you re-run concept-seed with --register <value> for the next round; offer the registers on direction rounds, and never pre-select one. buildPath rides the payload as { "value": "comp"|"code", "toggle": true }: the value is the recorded default (.impeccable/settings.json, or a PRODUCT.md standing commitment as fallback) and the toggle renders a footer switch whose flip binds that session only; the ANSWER then carries buildPath plus buildPathFlipped. On a code-led round each card still declares its comp path as a flip reserve: wireframes render, and a flip to comp makes --wait return once with BUILD PATH FLIPPED so you generate the comps into the declared slots while the round stays open; a flip back to code is free, and a comp that already landed stays as the critique reference. The toggle may only be offered when image generation exists: a harness with no image tool and no API key never sets toggle: true, so the choice never renders where comps cannot be made, and code-led simply rides as the untoggleable value. followup: true keeps the table open after a pick for a second round via --update; send the next payload immediately, the page is waiting on it.');
|
||||
console.log('\nOption ids return verbatim in ANSWER; "reroll" and "canon" are reserved. hero/board/comp accept URLs or local paths; comp slots may point at files that do not exist yet (serve first, generate after; the page polls until they land, so never block serving on generation). hero on a challenger is the inspiration it draws from and renders picture-in-picture beside the comp, never as the promise of the build. verdict routes rendering: "wins" and "competitive" challengers keep full cards, "declined" ones render demoted after them (narrow, quiet, art as a labeled thumb, "Adopt anyway"), with their kept line on the front; the page reorders declined cards to the end on its own. raised on the assigned card renders each donation as a named raise line. Salience parity: when the assigned card declares no comp (no image generation this round), catalog art on every card demotes to a labeled thumb, so what looks important is the verdict’s call, never rendering luck. canonCard renders the standing exit as a subordinate card with the same anatomy; without it, canon stays a quiet footer action. Include canon only for visual-direction rounds; never present it as your own recommendation. The pick card is a kicker convention, not a field: kicker "IMPECCABLE’S PICK" on your top-ranked grounded candidate, one at most, never in the lead slot. Every card gets the full anatomy, challengers, canon, and declined included: thesis, palette, materials, viewport, risk; the seed already hands you each challenger’s system rules, so a card with no palette chips is an authoring gap, not a data gap. Keep thesis and each fact to one short sentence: the card front shows thesis, identity, and a two-line risk, while first viewport and the case read on the card back behind the Details chip, so long facts cost the reader a flip, not the page its scanability. A card with no imagery at all has no back; its full read renders on the front, so a text-only round loses nothing. A card may instead declare "wireframe" ({"cols":12,"rows":10,"regions":[{"label":"nav rail","x":0,"y":0,"w":3,"h":10,"accent":true}]}): the page draws it as a layout schematic in the media slot; surface-scope rounds use it on code-led builds, it never counts toward salience, and the card keeps its full read on the front. The comp slot carries the card’s full-fidelity direction comp (the legacy key "sketch" is accepted as an alias). Comp aspect follows the surface: portrait at device viewport for native or mobile-first surfaces, landscape otherwise; the page adapts its cards to either. reroll accepts true or { "registers": ["safer", "bolder"] }: the register buttons steer the next hand along the familiar-to-bold axis, the answer carries "register", and you re-run concept-seed with --register <value> for the next round; offer the registers on direction rounds, and never pre-select one. buildPath rides the payload as { "value": "comp"|"code", "toggle": true }: the value is the recorded default (.impeccable/config.json buildPath, or .impeccable/config.local.json where one machine differs) and the toggle renders a footer switch whose flip binds that session only; the ANSWER then carries buildPath plus buildPathFlipped. On a code-led round each card still declares its comp path as a flip reserve: wireframes render, and a flip to comp makes --wait return once with BUILD PATH FLIPPED so you generate the comps into the declared slots while the round stays open; a flip back to code is free, and a comp that already landed stays as the critique reference. The toggle may only be offered when image generation exists: a harness with no image tool and no API key never sets toggle: true, so the choice never renders where comps cannot be made, and code-led simply rides as the untoggleable value. followup: true keeps the table open after a pick for a second round via --update; send the next payload immediately, the page is waiting on it.');
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
|
||||
@@ -1060,6 +1060,92 @@ describe('context.mjs CLI', () => {
|
||||
assert.match(res.stdout, /detect\.mjs --json <changed targets>/);
|
||||
});
|
||||
|
||||
// The build-path preference rides the unified config beside hook and
|
||||
// detector settings. The local file wins because whether a machine can
|
||||
// generate images is a property of that machine, not of the committed
|
||||
// default the rest of the team shares.
|
||||
describe('BUILD_PATH_DEFAULT', () => {
|
||||
const run = () => spawnSync(process.execPath, [SCRIPT_PATH], {
|
||||
cwd: scratch,
|
||||
encoding: 'utf8',
|
||||
env: { ...process.env, IMPECCABLE_NO_UPDATE_CHECK: '1', IMPECCABLE_NO_STALENESS_CHECK: '1' },
|
||||
});
|
||||
|
||||
it('reports a recorded preference from the shared config', () => {
|
||||
write('PRODUCT.md', '# Acme\n');
|
||||
write('.impeccable/config.json', JSON.stringify({ buildPath: 'code' }));
|
||||
assert.match(run().stdout, /BUILD_PATH_DEFAULT: code \(from \.impeccable\/config\.json\)/);
|
||||
});
|
||||
|
||||
it('lets the gitignored local config win over the committed one', () => {
|
||||
write('PRODUCT.md', '# Acme\n');
|
||||
write('.impeccable/config.json', JSON.stringify({ buildPath: 'comp' }));
|
||||
write('.impeccable/config.local.json', JSON.stringify({ buildPath: 'code' }));
|
||||
assert.match(run().stdout, /BUILD_PATH_DEFAULT: code \(from \.impeccable\/config\.local\.json\)/);
|
||||
});
|
||||
|
||||
it('stays silent when nothing is recorded, leaving new-work its own default', () => {
|
||||
write('PRODUCT.md', '# Acme\n');
|
||||
assert.equal(run().stdout.includes('BUILD_PATH_DEFAULT'), false);
|
||||
});
|
||||
|
||||
it('stays silent on a value nothing reads rather than guessing at it', () => {
|
||||
write('PRODUCT.md', '# Acme\n');
|
||||
write('.impeccable/config.json', JSON.stringify({ buildPath: 'code-first' }));
|
||||
assert.equal(run().stdout.includes('BUILD_PATH_DEFAULT'), false);
|
||||
});
|
||||
|
||||
// A monorepo commits the preference once at the repo root for every app in
|
||||
// it. Reading only projectRoot left the staleness finding (which does read
|
||||
// both) silent while the directive never named the value.
|
||||
describe('in a monorepo', () => {
|
||||
const writeWorkspace = () => {
|
||||
write('package.json', JSON.stringify({ private: true, workspaces: ['apps/*'] }));
|
||||
write('turbo.json', JSON.stringify({ tasks: {} }));
|
||||
write('PRODUCT.md', '# Root product\n');
|
||||
write('apps/dashboard/src/App.jsx', 'export default function App() { return "d"; }\n');
|
||||
};
|
||||
const runFromWorkspace = () => spawnSync(process.execPath, [SCRIPT_PATH, '--target', 'apps/dashboard/src/App.jsx'], {
|
||||
cwd: scratch,
|
||||
encoding: 'utf8',
|
||||
env: { ...process.env, IMPECCABLE_NO_UPDATE_CHECK: '1', IMPECCABLE_NO_STALENESS_CHECK: '1' },
|
||||
});
|
||||
|
||||
it('falls back to the repo-root value for a workspace that sets none', () => {
|
||||
writeWorkspace();
|
||||
write('.impeccable/config.json', JSON.stringify({ buildPath: 'code' }));
|
||||
assert.match(runFromWorkspace().stdout, /BUILD_PATH_DEFAULT: code/);
|
||||
});
|
||||
|
||||
it('lets the workspace override the repo root, nearest root first', () => {
|
||||
writeWorkspace();
|
||||
write('.impeccable/config.json', JSON.stringify({ buildPath: 'code' }));
|
||||
write('apps/dashboard/.impeccable/config.json', JSON.stringify({ buildPath: 'comp' }));
|
||||
assert.match(runFromWorkspace().stdout, /BUILD_PATH_DEFAULT: comp/);
|
||||
});
|
||||
|
||||
// Running from one workspace while targeting another must not hand the
|
||||
// target the caller's preference. The invoking directory is not evidence
|
||||
// about the project being worked on.
|
||||
it('does not let the invoking workspace lend its value to the target', () => {
|
||||
writeWorkspace();
|
||||
write('apps/marketing/src/App.jsx', 'export default function App() { return "m"; }\n');
|
||||
write('.impeccable/config.json', JSON.stringify({ buildPath: 'code' }));
|
||||
write('apps/marketing/.impeccable/config.json', JSON.stringify({ buildPath: 'comp' }));
|
||||
// Absolute, so the target actually resolves onto dashboard. A relative
|
||||
// path would be read against the caller's cwd, which resolves the
|
||||
// project back to marketing and tests nothing.
|
||||
const res = spawnSync(process.execPath, [SCRIPT_PATH, '--target', path.join(scratch, 'apps', 'dashboard', 'src', 'App.jsx')], {
|
||||
cwd: path.join(scratch, 'apps', 'marketing'),
|
||||
encoding: 'utf8',
|
||||
env: { ...process.env, IMPECCABLE_NO_UPDATE_CHECK: '1', IMPECCABLE_NO_STALENESS_CHECK: '1' },
|
||||
});
|
||||
// The repo-root default, not marketing's comp.
|
||||
assert.match(res.stdout, /BUILD_PATH_DEFAULT: code/);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it('keeps the manual-detector directive out of early context when the current provider hook is active', () => {
|
||||
const scripts = path.join(scratch, 'bundle', 'skills', 'impeccable', 'scripts');
|
||||
stageContextBundle(scripts, { providerId: 'codex' });
|
||||
@@ -1415,6 +1501,18 @@ describe('context.mjs update check', () => {
|
||||
assert.match(res.stdout, /^# PRODUCT\.md/);
|
||||
});
|
||||
|
||||
// The directive used to say "ask once" and "if they agree, run it" while also
|
||||
// saying to continue without waiting. Nothing gated the run on an answer that
|
||||
// could not arrive, so the command read as the next step. It now forbids
|
||||
// running in this turn outright, whatever the answer.
|
||||
it('forbids running the update in the same turn, on any answer', () => {
|
||||
const { stdout } = run({ lastCheck: Date.now(), latestVersion: '2.0.0' });
|
||||
assert.match(stdout, /Do not run `npx impeccable update` in this turn, whatever the user answers/);
|
||||
assert.match(stdout, /only after the user has asked for it in their own words/);
|
||||
// The conditional that made the command look reachable must be gone.
|
||||
assert.equal(/If they agree, run/.test(stdout), false);
|
||||
});
|
||||
|
||||
it('stays silent when the cached latest version is not newer', () => {
|
||||
const res = run({ lastCheck: Date.now(), latestVersion: '0.0.1' });
|
||||
assert.equal(res.status, 0);
|
||||
|
||||
@@ -87,16 +87,70 @@ and that is the useful part. Reproduce with
|
||||
`IMPECCABLE_SKILL_BEHAVIOR_MODELS=gpt-5.6-luna,deepseek-v4-flash`.
|
||||
|
||||
Against the current default lineup, two cells are the known floor:
|
||||
`redesign replaces DESIGN` is flaky, and `critique closes` is flaky on
|
||||
gemini-3.6-flash. A regression is a failure beyond those two.
|
||||
`redesign replaces DESIGN` is flaky on every model, and `critique closes` is
|
||||
flaky on gemini-3.6-flash. A regression is a failure beyond those two.
|
||||
|
||||
**Read any failure against the clock before calling it behavior.** The suite ran
|
||||
at a 300s per-test timeout until 2026-08-13, and for the workflow-contract
|
||||
scenarios that cap was below the runtime of a correct run. `initialized natural
|
||||
build` on claude-sonnet-5 was measured at 579s when it stopped to put the
|
||||
concept to the user before building, while the runs that skipped that checkpoint
|
||||
and failed the assertion finished in 130-200s. The cap was therefore selecting
|
||||
for the behavior the scenario forbids: thorough runs were killed, hasty ones
|
||||
were graded. The timeout is now 900s (`scripts/test-suites.mjs`). A duration at
|
||||
or just past the cap is a timeout, not a verdict.
|
||||
|
||||
| Scenario | claude-sonnet-5 | gpt-5.6-terra | gemini-3.6-flash | luna / deepseek (dropped) |
|
||||
|---|---|---|---|---|
|
||||
| attended fresh init | not measured | not measured | not measured | not measured |
|
||||
| initialized natural build | not measured | not measured | not measured | not measured |
|
||||
| redesign replaces DESIGN | flaky | not measured | not measured | not measured |
|
||||
| bolder refinement | not measured | not measured | pass (on 3.5) | luna pass, deepseek **fail** |
|
||||
| critique closes | pass (2 of 2) | pass (2 of 2) | **flaky (1 of 3)** | luna **fail (1 of 6)**, deepseek flaky |
|
||||
| attended fresh init | pass | pass | pass | not measured |
|
||||
| initialized natural build | flaky (1 of 4, and see the clock note) | pass | pass | not measured |
|
||||
| redesign replaces DESIGN | flaky (timeout this run) | **fail** | **fail (timeout)** | not measured |
|
||||
| bolder refinement | pass | pass | pass (on 3.5) | luna pass, deepseek **fail** |
|
||||
| critique closes | pass (4 of 4) | pass (3 of 3) | **flaky (1 of 4)** | luna **fail (1 of 6)**, deepseek flaky |
|
||||
|
||||
Gemini's `bolder refinement` and `critique closes` runs in this sweep died on
|
||||
`AI_APICallError` / `ETIMEDOUT` before completing a turn. Network failures are
|
||||
not behavior measurements and are excluded from the counts above.
|
||||
|
||||
## Scenario baseline (2026-08-13, current lineup)
|
||||
|
||||
Measured on the same sweep. `scenarios.test.mjs` passes 15 of 15 on
|
||||
gpt-5.6-terra and gemini-3.6-flash. Only claude-sonnet-5 fails anything, and
|
||||
that asymmetry is the finding: the two cells below fail on the frontier model
|
||||
while two weaker-on-paper lineups route correctly, so read them as a text
|
||||
problem that one model's priors expose rather than as a model floor.
|
||||
|
||||
| Scenario | claude-sonnet-5 | gpt-5.6-terra | gemini-3.6-flash |
|
||||
|---|---|---|---|
|
||||
| 1-7, 10, 12-15 | pass | pass | pass |
|
||||
| 8 (SvelteKit exploration) | flaky | pass | pass |
|
||||
| 11 (shape resolves the build gate) | flaky | pass | pass |
|
||||
|
||||
Scenarios 8 and 11 pass on re-run, so treat a single failure there as flake and
|
||||
confirm with a second run before investigating.
|
||||
|
||||
Scenarios 9 and 15 both failed on sonnet when this baseline was first measured,
|
||||
and the two causes are worth keeping because neither was where it looked:
|
||||
|
||||
- **9 was a real defect in the directive.** `UPDATE_AVAILABLE` said to ask the
|
||||
user, then "If they agree, run `npx impeccable update`", then to continue
|
||||
without waiting. With no wait there is no agreement to read, so the command
|
||||
was the only concrete instruction left standing and sonnet ran it. Fixed by
|
||||
removing the command from the turn entirely rather than by strengthening the
|
||||
warning around it.
|
||||
- **15 was a broken fixture.** The iOS workspace held PRODUCT.md and nothing
|
||||
else, so `audit the app in this workspace` named an app that did not exist.
|
||||
Sonnet spent its whole step budget looking for it and read no reference file
|
||||
at all, which the assertion reported as "loaded `audit.md` instead of the
|
||||
variant". The fixture now ships one SwiftUI screen, the same courtesy
|
||||
`MINIMAL_LANDING_HTML` already did for the web scenarios. The scenario passes
|
||||
on unmodified `main` once the fixture is answerable, which is the proof the
|
||||
routing text was never at fault.
|
||||
|
||||
The general lesson is worth more than either fix: **an assertion reports the
|
||||
property it checks, not the reason it failed.** Both of these read as routing
|
||||
defects and neither was one. Pull the trace before writing the diagnosis, and
|
||||
prefer `IMPECCABLE_SKILL_BEHAVIOR_VERBOSE=1` over inference from the message.
|
||||
|
||||
Gemini cells marked `on 3.5` were measured on the superseded `gemini-3.5-flash`
|
||||
and have not been re-run on 3.6. That distinction is not pedantic. `critique
|
||||
|
||||
@@ -162,6 +162,49 @@ Dynamic Type, VoiceOver, reduced motion, high contrast in direct sun, and
|
||||
targets usable one-handed with wet hands.
|
||||
`;
|
||||
|
||||
/**
|
||||
* The native counterpart to MINIMAL_LANDING_HTML, and it exists for the same
|
||||
* reason. A native scenario carrying only PRODUCT.md gives an audit nothing to
|
||||
* audit: the agent goes looking for the app it was told exists, and a routing
|
||||
* assertion ends up measuring how a model copes with an empty workspace
|
||||
* instead. One screen is enough to make the request answerable.
|
||||
*/
|
||||
export const MINIMAL_IOS_SOURCE = `import SwiftUI
|
||||
|
||||
struct TideDetailView: View {
|
||||
let station: String
|
||||
@State private var showsLog = false
|
||||
|
||||
var body: some View {
|
||||
NavigationStack {
|
||||
List {
|
||||
Section("Next window") {
|
||||
HStack {
|
||||
Text("High")
|
||||
Spacer()
|
||||
Text("4:12 PM").foregroundStyle(.secondary)
|
||||
}
|
||||
HStack {
|
||||
Text("Low")
|
||||
Spacer()
|
||||
Text("10:38 PM").foregroundStyle(.secondary)
|
||||
}
|
||||
}
|
||||
Section {
|
||||
Button("Log a catch") { showsLog = true }
|
||||
}
|
||||
}
|
||||
.navigationTitle(station)
|
||||
.toolbar {
|
||||
ToolbarItem(placement: .topBarTrailing) {
|
||||
Button("Refresh") { }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
`;
|
||||
|
||||
/**
|
||||
* Tiny static landing page fixture for scenarios that invoke sub-commands
|
||||
* (polish, audit) without standing up a full framework project. Gives the
|
||||
|
||||
@@ -29,6 +29,7 @@ import {
|
||||
PRODUCT_MD_SAMPLE,
|
||||
PRODUCT_MD_SAMPLE_NO_REGISTER,
|
||||
PRODUCT_MD_SAMPLE_IOS,
|
||||
MINIMAL_IOS_SOURCE,
|
||||
DESIGN_MD_SAMPLE,
|
||||
MINIMAL_LANDING_HTML,
|
||||
SVELTE_PROJECT_FILES,
|
||||
@@ -531,7 +532,7 @@ for (const modelId of resolveModelList()) {
|
||||
// reference/ios.md itself, so native guidance enters the conversation
|
||||
// without relying on a second model-directed file read.
|
||||
const workspace = prepareWorkspace({
|
||||
files: { 'PRODUCT.md': PRODUCT_MD_SAMPLE_IOS },
|
||||
files: { 'PRODUCT.md': PRODUCT_MD_SAMPLE_IOS, 'TideDetailView.swift': MINIMAL_IOS_SOURCE },
|
||||
});
|
||||
try {
|
||||
const { trace, text } = await runTurn({
|
||||
@@ -566,7 +567,7 @@ for (const modelId of resolveModelList()) {
|
||||
// switching via its web-only guard is acceptable; never reaching the
|
||||
// variant is the failure).
|
||||
const workspace = prepareWorkspace({
|
||||
files: { 'PRODUCT.md': PRODUCT_MD_SAMPLE_IOS },
|
||||
files: { 'PRODUCT.md': PRODUCT_MD_SAMPLE_IOS, 'TideDetailView.swift': MINIMAL_IOS_SOURCE },
|
||||
});
|
||||
try {
|
||||
const { trace, text } = await runTurn({
|
||||
|
||||
@@ -23,6 +23,7 @@ import {
|
||||
stampProductSchema,
|
||||
} from '../skill/scripts/lib/artifact-schema.mjs';
|
||||
import {
|
||||
checkBuildPathUnset,
|
||||
checkConfig,
|
||||
checkDesignSidecar,
|
||||
checkNativePlatformEvidence,
|
||||
@@ -317,6 +318,64 @@ describe('checkConfig', () => {
|
||||
write('.impeccable/config.json', '{ not json');
|
||||
assert.deepEqual(checkConfig({ projectRoot: scratch, repoRoot: scratch }), []);
|
||||
});
|
||||
|
||||
it('accepts both build-path values', () => {
|
||||
for (const value of ['comp', 'code']) {
|
||||
write('.impeccable/config.json', JSON.stringify({ buildPath: value }));
|
||||
assert.deepEqual(checkConfig({ projectRoot: scratch, repoRoot: scratch }), []);
|
||||
}
|
||||
});
|
||||
|
||||
it('flags a build-path value nothing reads', () => {
|
||||
write('.impeccable/config.json', JSON.stringify({ buildPath: 'code-first' }));
|
||||
const findings = checkConfig({ projectRoot: scratch, repoRoot: scratch });
|
||||
assert.deepEqual(ids(findings), ['config-invalid-build-path']);
|
||||
assert.match(findings[0].summary, /"code-first"/);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── build path ────────────────────────────────────────────────────────────
|
||||
|
||||
describe('checkBuildPathUnset', () => {
|
||||
const args = () => ({ projectRoot: scratch, repoRoot: scratch, product: '# Product' });
|
||||
|
||||
it('flags a project that has done direction work and recorded nothing', () => {
|
||||
fs.mkdirSync(path.join(scratch, '.impeccable', 'surfaces'), { recursive: true });
|
||||
const findings = checkBuildPathUnset(args());
|
||||
assert.deepEqual(ids(findings), ['config-build-path-unset']);
|
||||
// The one precondition this module cannot see must reach the reader that can.
|
||||
assert.match(findings[0].fix, /image generation/);
|
||||
});
|
||||
|
||||
it('accepts decision mocks as the same evidence', () => {
|
||||
fs.mkdirSync(path.join(scratch, '.impeccable', 'mocks', 'decision'), { recursive: true });
|
||||
assert.deepEqual(ids(checkBuildPathUnset(args())), ['config-build-path-unset']);
|
||||
});
|
||||
|
||||
it('stays silent on a project that has never done direction work', () => {
|
||||
assert.deepEqual(checkBuildPathUnset(args()), []);
|
||||
});
|
||||
|
||||
it('stays silent without a product record, where init already routes', () => {
|
||||
fs.mkdirSync(path.join(scratch, '.impeccable', 'surfaces'), { recursive: true });
|
||||
assert.deepEqual(checkBuildPathUnset({ ...args(), product: null }), []);
|
||||
});
|
||||
|
||||
it('stays silent once a value is recorded, in either config file', () => {
|
||||
fs.mkdirSync(path.join(scratch, '.impeccable', 'surfaces'), { recursive: true });
|
||||
write('.impeccable/config.json', JSON.stringify({ buildPath: 'code' }));
|
||||
assert.deepEqual(checkBuildPathUnset(args()), []);
|
||||
|
||||
fs.rmSync(path.join(scratch, '.impeccable', 'config.json'));
|
||||
write('.impeccable/config.local.json', JSON.stringify({ buildPath: 'comp' }));
|
||||
assert.deepEqual(checkBuildPathUnset(args()), []);
|
||||
});
|
||||
|
||||
it('defers to the invalid-value finding rather than reporting the key twice', () => {
|
||||
fs.mkdirSync(path.join(scratch, '.impeccable', 'surfaces'), { recursive: true });
|
||||
write('.impeccable/config.json', JSON.stringify({ buildPath: 'comp-first' }));
|
||||
assert.deepEqual(checkBuildPathUnset(args()), []);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── surface briefs ────────────────────────────────────────────────────────
|
||||
|
||||
Reference in New Issue
Block a user