Files
pbakaus_impeccable/tests/oracle
Paul BakausandClaude Fable 5.1 d1f81fc1c7 A rendered line is the whole line: the fragments merge, the snapshot carries them or the rule stands down, an ignored subtree gets no vote in the palette, and a class a design document forbids declares nothing
The review threads on #840 found four ways the new rendered rules were
still measuring something other than what the reader sees.

- `line-length` read one `getClientRects()` box per direct text node and
  called each of them a rendered line. An inline `<strong>` in the middle
  of a sentence is its own text node, so one visual line arrived as two
  fragments and the paragraph's characters were divided between them —
  and the `<strong>`'s own text was never measured at all while its
  characters still counted toward `text_len`. Both halves of the measure
  are the same text now: the probe collects the rects of every text node
  under the element, and `Dom::text_line_rects` merges the ones that share
  a row back into the line they rendered as. A column of long lines split
  by inline markup used to charge nothing; it charges now, and a wrapped
  sentence in two fragments is one line, not two.

- The same function divided every rect by the line box to recover a line
  count. On live per-line rects that was double-counting: a leading
  tighter than the glyph box makes `round(height / line_box)` 2, and one
  long line pushed twice satisfied "at least two long lines". Nothing is
  divided any more, because nothing that reaches the rule is a union.

- The union was what a snapshot-backed scan had — the extension's
  offscreen document and any strict-CSP page — and a union of a long
  first line and a short tail is the same union as two even lines, so
  every line inferred from it was invented. The capture records the rects
  now (`dl`, with `textLines` on the snapshot saying it did), and a
  capture that did not answers `None`: the rule stands down rather than
  guessing. That is also what any other DOM that cannot split a wrapped
  run answers.

- `ai-color-palette` accumulated its tell hues before the scoped ignores
  ran. A cyan tell inside a `data-impeccable-ignore="ai-color-palette"`
  subtree opened the page-wide two-hue gate and charged neon ink on an
  element nobody had waived. Ignored content now gets no vote.

- `declared_component_selectors` took every backticked class in DESIGN.md
  as a declaration, including the ones the document writes down in order
  to forbid them. "Do not write a new `.hero-cta-primary`" exempted
  `.hero-cta-primary` from `kicker-above-heading` — the parser silencing
  exactly the misuse the document was written to catch. Each occurrence
  is read in the document's own structure now: the heading chain above it
  (a "Don't" section, and its subsections with it) and the clause it sits
  in, where a clause is cut on punctuation and on the phrases that turn a
  sentence around. "No ALL CAPS outside the `.eyebrow` class" still
  declares `.eyebrow`, because what "no" governs ends at "outside"; a
  class the document calls deprecated anywhere is declared nowhere.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LQBUunp8QttxZqihybNmtL
2026-09-20 19:02:46 -07:00
..

Oracle: behavior goldens for every impeccable verb

lib.mjs runs each case (verb + args + staged workspace + stdin) against an implementation and captures stdout, stderr, exit code, and named files, with machine-specific paths and timestamps normalized.

  • The goldens are frozen JS behavior: they were recorded from the Node scripts (skill/scripts, cli/bin) before those left the tree with the launcher swap, plus the reviewed deltas in DELTAS.md. They are the behavior contract the engine binary is held to.
  • record.mjs --bin (with $IMPECCABLE_BIN or --bin=/path) writes goldens from the binary, for new cases or a delta a review accepted. Plain record.mjs still targets the JS scripts and only works on a checkout that has them (history before the swap).
  • run.mjs replays the corpus against $IMPECCABLE_BIN (or --js for a self-check on a pre-swap checkout) and diffs. Byte-equal is the bar; DELTAS.md lists reviewed exceptions.
  • tests/oracle.test.mjs runs run.mjs under bun run test and skips when no binary is found (IMPECCABLE_BIN or skill/scripts/bin/<os>-<arch>/, filled by bun run fetch:engine).
  • cases/*.mjs define the corpus (default export: array or async function returning an array). workspaces/ holds project fixtures that are copied to a temp dir per run, so cases can write freely.

Adding a case: append to the matching cases/*.mjs, run node tests/oracle/record.mjs --bin <prefix>, review the golden by hand (the binary is now the recorder, so a bug in it would be frozen too), commit the golden.

Verb names are the binary's subcommands. cli-help and cli-version map to impeccable --help / --version. lib.mjs still carries the JS_VERBS table that maps each verb to the script it was recorded from.

vectors/ holds the function-level vectors recorded from the JS engine's pure functions; see vectors/README.md.

Corpus files

  • cases/detect.mjs: detect, cli-help, cli-version, ignores.
  • cases/hooks.mjs: hook, hook-before-edit, hook-admin.
  • cases/context.mjs: context, doctor, pin, surface-brief, critique-storage, palette, embed-prompt, context-signals (id prefix signals-), detect-csp (csp-), concept-seed (seed-), generate-image (genimg-), serve-question (question-). Only offline paths: the local catalog fixture or an unreachable roll API, fake image generation, and serve-question modes that never open a browser or listen. Workspaces are workspaces/ctx-*; the header comment in the case file describes each one. Machine-specific env (OPENAI_API_KEY, catalog and context overrides, CI) is pinned per case so the recording host does not leak into goldens.

Normalizations

Beyond paths and ISO timestamps, normalize() masks these run- or machine-dependent fragments. Each is targeted at one script's output:

  • IMAGE_TOOLS: <IMAGE_TOOLS_PROBE>: context probes which cwebp sips magick ffmpeg; the set found describes the machine, not the script.
  • "devServer": <DEV_SERVER_PROBE>: context-signals probes localhost ports 4321/3000/5173/5174/8080/8000/4200; whatever is listening on the recording host is not part of the contract.
  • <STAMP>: critique-storage stamps snapshots with the wall clock in dash form (2026-05-12T18-30-00Z), in the file name and the timestamp: frontmatter it writes. Cases that write a snapshot do not snapshot the file; they run latest / trend afterwards instead.
  • "<finding-id>": <EPOCH>: the staleness notice cache (~/.impeccable/staleness-check.json) keys epoch stamps by finding id.
  • <IMPECCABLE> <verb> / <HOOK_ADMIN_CMD>: self-referential command lines.

Not covered on purpose: palette with no --id / --from / env seed (random), concept-seed against the live roll API, generate-image real mode, serve-question --start / blocking mode (opens a browser and binds a port), and unhandled-exception paths whose stack traces carry Node line numbers.

Live-mode cases (cases/live-*.mjs, workspaces live-*)

Helpers live in live-helpers.mjs (staged journals, buffers, wrapped source files with the fake-agent variant block, a .git FILE pointing at a non-repo gitdir so roots resolution sees a git boundary while git check-ignore exits 128 everywhere and the ignore block lands in the snapshotable .gitfake/info/exclude). Svelte component preview cases symlink this repo's node_modules/svelte into the staged app, exactly like the unit tests.

Harness additions made for live:

  • steps[] entries may carry their own setup(ws) (run right before that step) and daemon: true with readyFile / readyTimeoutMs: the verb is spawned detached, the harness waits for the ready file, later steps run against it, and teardown SIGTERMs (then SIGKILLs) it. Its stdout/stderr land in the golden as daemon: [{stdout, stderr}].
  • normalize: [[regexSource, flags, replacement], ...] on a case applies extra masks to that case only. Live uses it for the dynamic helper port (localhost:<PORT>, "port": <PORT>), lease and phase stamps (<EPOCH>), and float durations (<N>).
  • Global masks added: "pid": <PID> / (pid <PID>) and UUID tokens <UUID>.
  • snapshotFiles walks node_modules/.impeccable-live (the Svelte preview tree) and nothing else under node_modules.

Deliberately not covered here (rely on tests/live-e2e): the browser handshake and /live.js bundle, SSE, generate/accept round-trips through a real browser, variant_mount_failed republish, manual-edit chat routing and the codex/claude subprocess providers, Svelte revision-dir publishing, and live.mjs's dev-server-dependent flows. Lock-file names hash the absolute source path, so lock cases do not snapshot .impeccable/live/locks/. live-poll-*-connection-refused assumes nothing listens on 127.0.0.1:65531.