Files
pbakaus_impeccable/tests/oracle
9f42da8e18 Five rendered rules measure what they claim, and a hit records the width it fired at (#840)
* The rendered rules measure what rendered: the line instead of the box that holds it, the space around the text instead of the declared padding, gray by chroma at its own lightness, the AI palette by two tell hues rather than one accent, and the eyebrow named as the element its own design document declares

Five browser rules were reporting something other than what the reader sees,
and a review that charges those numbers is charging noise.

- `line-length` measured `rect.width / (fontSize * 0.5)`, the box's capacity.
  A paragraph in a 1022px column whose text stops at 571px was charged 142
  characters a line it never rendered. The probe now hands back the client
  rects of the direct text one per line box (`direct_text_line_rects`), the
  characters divide between the lines in proportion to the ink each carries,
  and the charge needs more than one long line: the harm named is the eye
  tracking back to the start of the next line, which takes a column to do.
- `cramped-padding` read the declared padding. A 44px control with
  `padding: 0 16px` and a flex-centred label has 12px of air above the label
  and was charged "0px vertical padding"; the measurement is now the inset
  between the rendered text and the inside of the border box. Its wrapper half
  read a text-bearing child's border box the same way, so a `<td>` that fills
  its table and insets its own text counted as flush; it reads the text now.
- `gray-on-color` called anything under 0.85 relative luminance gray, which
  takes in every off-white: `#e8edf2` measures 0.84 there and 0.93 as
  lightness. Gray is now low chroma at the lightness the ink actually sits at
  (saturation, which is chroma normalized for lightness) and neither of the
  two neutral inks a coloured surface carries. The contrast check beside it is
  untouched, and the recorded vectors still pass.
- `ai-color-palette` charged every hue between 160° and 200° on a dark ground
  as neon, which lit one ordinary teal accent 18 places on a page with nothing
  wrong with it. A gradient in a tell hue is still the pattern on its own;
  flat neon ink on near-black waits for a second tell hue to turn up somewhere
  on the page, because one saturated accent on a dark system is an accent.
- `kicker-above-heading` reported against `body`, so a charged row had nothing
  to point at, and it fired on eyebrows a design document documents. It names
  the eyebrow element now, and stands down where the repository's DESIGN.md
  declares the class by name — the prose's backticked class selectors travel
  on the design-system config the colour and radius rules already read.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LQBUunp8QttxZqihybNmtL

* A rendered line is the whole line: the fragments merge, the snapshot carries them or the rule stands down, an ignored subtree gets no vote in the palette, and a class a design document forbids declares nothing

The review threads on #840 found four ways the new rendered rules were
still measuring something other than what the reader sees.

- `line-length` read one `getClientRects()` box per direct text node and
  called each of them a rendered line. An inline `<strong>` in the middle
  of a sentence is its own text node, so one visual line arrived as two
  fragments and the paragraph's characters were divided between them —
  and the `<strong>`'s own text was never measured at all while its
  characters still counted toward `text_len`. Both halves of the measure
  are the same text now: the probe collects the rects of every text node
  under the element, and `Dom::text_line_rects` merges the ones that share
  a row back into the line they rendered as. A column of long lines split
  by inline markup used to charge nothing; it charges now, and a wrapped
  sentence in two fragments is one line, not two.

- The same function divided every rect by the line box to recover a line
  count. On live per-line rects that was double-counting: a leading
  tighter than the glyph box makes `round(height / line_box)` 2, and one
  long line pushed twice satisfied "at least two long lines". Nothing is
  divided any more, because nothing that reaches the rule is a union.

- The union was what a snapshot-backed scan had — the extension's
  offscreen document and any strict-CSP page — and a union of a long
  first line and a short tail is the same union as two even lines, so
  every line inferred from it was invented. The capture records the rects
  now (`dl`, with `textLines` on the snapshot saying it did), and a
  capture that did not answers `None`: the rule stands down rather than
  guessing. That is also what any other DOM that cannot split a wrapped
  run answers.

- `ai-color-palette` accumulated its tell hues before the scoped ignores
  ran. A cyan tell inside a `data-impeccable-ignore="ai-color-palette"`
  subtree opened the page-wide two-hue gate and charged neon ink on an
  element nobody had waived. Ignored content now gets no vote.

- `declared_component_selectors` took every backticked class in DESIGN.md
  as a declaration, including the ones the document writes down in order
  to forbid them. "Do not write a new `.hero-cta-primary`" exempted
  `.hero-cta-primary` from `kicker-above-heading` — the parser silencing
  exactly the misuse the document was written to catch. Each occurrence
  is read in the document's own structure now: the heading chain above it
  (a "Don't" section, and its subsections with it) and the clause it sits
  in, where a clause is cut on punctuation and on the phrases that turn a
  sentence around. "No ALL CAPS outside the `.eyebrow` class" still
  declares `.eyebrow`, because what "no" governs ends at "outside"; a
  class the document calls deprecated anywhere is declared nowhere.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LQBUunp8QttxZqihybNmtL

* A snapshot records each line once, a line stops at the gutter, and a design document's directive governs what follows it

The re-review of the first round found four places where the new
measurements reached further than they should.

- The snapshot recorded the rects of every text node *under* each element,
  so a line rendered deep in a page was written down again for every
  ancestor above it. On a deep, text-heavy page that multiplies the
  capture by its depth and can carry it past the byte cap, and a capture
  that fails is a scan that returns nothing at all. Each element records
  only its own text now — one line, one entry — and `text_line_rects`
  assembles an element's lines by walking the tree the capture already
  serialized.

- Merging rects by vertical band alone made two columns that happen to
  sit on the same rows into one page-wide line. A fragment joins a row now
  only when it runs on from it: a horizontal gap no wider than the row's
  own line box. The fragments of a wrapped line are contiguous; a gutter
  is not. An inline image wider than the leading splits its line by the
  same test, which understates a line rather than overstating it, and that
  is the direction this rule should err in.

- In DESIGN.md, where the negative word sits decides what it governs. A
  state ("`.card-old` is deprecated") describes whatever its clause is
  about; a directive ("never use `.x`") condemns what follows it and
  nothing before it. Reading the whole clause for either lost the
  sanctioned half of "Use `.kicker` and never `.tagline`". The headings a
  document uses to retire a set — "Retired components", "Unsupported
  patterns" — are read as negative now, and a heading that names both
  sides ("Dos and Don'ts") heads a section of both, so its subsections are
  what say which is which.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LQBUunp8QttxZqihybNmtL

* A heading names both sides only when something joins them

`DESIGN_BOTH_SIDES_HEADING` asked for a `do` within twelve characters of a
`don't`, which "Don't do this" and "What we don't do" both satisfy while
meaning only the one thing. A section headed either of those stopped
condemning the classes it names, so its counter-examples became declared
components and the rules stood down on exactly the misuse the heading
introduces.

The two sides have to be joined by something that pairs them now — and,
or, a slash, a comma — which is what "Dos and Don'ts" and "Do / Do not"
have and a sentence that happens to contain both words does not.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LQBUunp8QttxZqihybNmtL

* A compound separator pairs the two sides too

"Do's, and Don'ts" and "Do and/or Don't" put a comma and a conjunction, or
a conjunction and a slash, between the two halves, and a joiner that
accepted exactly one token read those headings as negative and dropped the
classes their "Do" sections sanction. One joiner or several, up to four.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LQBUunp8QttxZqihybNmtL

* The probe's two text-rect helpers say which text they read

`__textLineRects` collected only the direct text once the line probe grew
its own deep walk, so its name promised lines it no longer produced: it is
`__directTextRects`, the list the union rect is built from. The snapshot's
union helper takes the rects rather than the node now, and its comment says
so. No behavior.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LQBUunp8QttxZqihybNmtL

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-21 08:22:08 -07:00
..

Oracle: behavior goldens for every impeccable verb

lib.mjs runs each case (verb + args + staged workspace + stdin) against an implementation and captures stdout, stderr, exit code, and named files, with machine-specific paths and timestamps normalized.

  • The goldens are frozen JS behavior: they were recorded from the Node scripts (skill/scripts, cli/bin) before those left the tree with the launcher swap, plus the reviewed deltas in DELTAS.md. They are the behavior contract the engine binary is held to.
  • record.mjs --bin (with $IMPECCABLE_BIN or --bin=/path) writes goldens from the binary, for new cases or a delta a review accepted. Plain record.mjs still targets the JS scripts and only works on a checkout that has them (history before the swap).
  • run.mjs replays the corpus against $IMPECCABLE_BIN (or --js for a self-check on a pre-swap checkout) and diffs. Byte-equal is the bar; DELTAS.md lists reviewed exceptions.
  • tests/oracle.test.mjs runs run.mjs under bun run test and skips when no binary is found (IMPECCABLE_BIN or skill/scripts/bin/<os>-<arch>/, filled by bun run fetch:engine).
  • cases/*.mjs define the corpus (default export: array or async function returning an array). workspaces/ holds project fixtures that are copied to a temp dir per run, so cases can write freely.

Adding a case: append to the matching cases/*.mjs, run node tests/oracle/record.mjs --bin <prefix>, review the golden by hand (the binary is now the recorder, so a bug in it would be frozen too), commit the golden.

Verb names are the binary's subcommands. cli-help and cli-version map to impeccable --help / --version. lib.mjs still carries the JS_VERBS table that maps each verb to the script it was recorded from.

vectors/ holds the function-level vectors recorded from the JS engine's pure functions; see vectors/README.md.

Corpus files

  • cases/detect.mjs: detect, cli-help, cli-version, ignores.
  • cases/hooks.mjs: hook, hook-before-edit, hook-admin.
  • cases/context.mjs: context, doctor, pin, surface-brief, critique-storage, palette, embed-prompt, context-signals (id prefix signals-), detect-csp (csp-), concept-seed (seed-), generate-image (genimg-), serve-question (question-). Only offline paths: the local catalog fixture or an unreachable roll API, fake image generation, and serve-question modes that never open a browser or listen. Workspaces are workspaces/ctx-*; the header comment in the case file describes each one. Machine-specific env (OPENAI_API_KEY, catalog and context overrides, CI) is pinned per case so the recording host does not leak into goldens.

Normalizations

Beyond paths and ISO timestamps, normalize() masks these run- or machine-dependent fragments. Each is targeted at one script's output:

  • IMAGE_TOOLS: <IMAGE_TOOLS_PROBE>: context probes which cwebp sips magick ffmpeg; the set found describes the machine, not the script.
  • "devServer": <DEV_SERVER_PROBE>: context-signals probes localhost ports 4321/3000/5173/5174/8080/8000/4200; whatever is listening on the recording host is not part of the contract.
  • <STAMP>: critique-storage stamps snapshots with the wall clock in dash form (2026-05-12T18-30-00Z), in the file name and the timestamp: frontmatter it writes. Cases that write a snapshot do not snapshot the file; they run latest / trend afterwards instead.
  • "<finding-id>": <EPOCH>: the staleness notice cache (~/.impeccable/staleness-check.json) keys epoch stamps by finding id.
  • <IMPECCABLE> <verb> / <HOOK_ADMIN_CMD>: self-referential command lines.

Not covered on purpose: palette with no --id / --from / env seed (random), concept-seed against the live roll API, generate-image real mode, serve-question --start / blocking mode (opens a browser and binds a port), and unhandled-exception paths whose stack traces carry Node line numbers.

Live-mode cases (cases/live-*.mjs, workspaces live-*)

Helpers live in live-helpers.mjs (staged journals, buffers, wrapped source files with the fake-agent variant block, a .git FILE pointing at a non-repo gitdir so roots resolution sees a git boundary while git check-ignore exits 128 everywhere and the ignore block lands in the snapshotable .gitfake/info/exclude). Svelte component preview cases symlink this repo's node_modules/svelte into the staged app, exactly like the unit tests.

Harness additions made for live:

  • steps[] entries may carry their own setup(ws) (run right before that step) and daemon: true with readyFile / readyTimeoutMs: the verb is spawned detached, the harness waits for the ready file, later steps run against it, and teardown SIGTERMs (then SIGKILLs) it. Its stdout/stderr land in the golden as daemon: [{stdout, stderr}].
  • normalize: [[regexSource, flags, replacement], ...] on a case applies extra masks to that case only. Live uses it for the dynamic helper port (localhost:<PORT>, "port": <PORT>), lease and phase stamps (<EPOCH>), and float durations (<N>).
  • Global masks added: "pid": <PID> / (pid <PID>) and UUID tokens <UUID>.
  • snapshotFiles walks node_modules/.impeccable-live (the Svelte preview tree) and nothing else under node_modules.

Deliberately not covered here (rely on tests/live-e2e): the browser handshake and /live.js bundle, SSE, generate/accept round-trips through a real browser, variant_mount_failed republish, manual-edit chat routing and the codex/claude subprocess providers, Svelte revision-dir publishing, and live.mjs's dev-server-dependent flows. Lock-file names hash the absolute source path, so lock cases do not snapshot .impeccable/live/locks/. live-poll-*-connection-refused assumes nothing listens on 127.0.0.1:65531.