Snapshot the current app-server implementation, shared Live optimizations, generated harness output, and in-progress site work before restoring polling as the primary runtime path.
Prepared with Codex assistance under maintainer direction.
Two browser-engine quality rules, both warning severity.
text-occlusion / element-overlap fires on three shapes: an opaque
decorated box painted over a text element (elementFromPoint confirms
real coverage, box >= 30%), one text run buried under another when at
least one side is a positioned layer (text >= 45%, so line-box leading
bleed between stacked flow blocks does not count), and an inline element
whose opaque fill leaks past its line onto a neighbour (the class-name
collision bug). A large headline whose edge overhangs a bounded content
card is caught as an element collision even when the text stays on top.
Gradient scrims, decorative SVG emblems, fixed/sticky overlays, floats,
and raw image backdrops (contrast territory, deduped against the pixel
low-contrast rule) are exempt.
first-viewport-column-overflow fires when a multi-column opening section
runs one column past 140% of the viewport while a sibling fits inside
one screen, the stretched-hero signature. Single-column pages and
full-page heroes with no fitting sibling are exempt.
Validated: fires on the diagnosed repros, clean across a 60-sample
sweep. Fixtures + browser tests added.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Item 1 (hero liveness theater):
- pulsing-dot now merges declarations per selector across rule blocks
(cascade-approximate), descends into media queries, and strips
prefers-reduced-motion: reduce overrides before the predicate runs.
Catches the shipped split-block constructions (size in the base rule,
animation added later or inside a no-preference media block).
- Dots whose element sits inside a header/nav landmark are promoted to
error severity (string-level landmark ranges in both engines); the
browser engine additionally promotes dots resting in the first ~900px.
- blinking-cursor findings in the first ~900px or inside header/nav are
promoted from advisory to warning.
- Per-finding severity overrides now flow through static-html,
browser-injected serialization, and detect-url.
Item 2 (nav-CTA contrast constructions):
- The a24-opus 01/002 header CTA already fires (specificity cascade +
oklch + var() all resolved); systematic sweep found two remaining
escapes and closes both:
- own gradient background on a SAFE_TAGS element (checkColors styled-
control exception now treats an own gradient as an own surface,
contrast measured against the worst stop)
- ::before/::after full-cover surface (static cascade marks pseudo
surfaces; browser adapter reads the pseudo computed style) so text is
measured against the surface the browser actually paints
- nav-cta-constructions fixture locks all eight computable construction
families; background-image: url() remains unflaggable by design.
Item 3 (shape-assembled-illustration, slop/advisory):
- New rule for large inline SVGs composing a pictorial scene from >= 8
primitive shapes at >= 200x200 intrinsic size with >= 3 distinct fills.
Charts (axis labels), stroke-only technical drawings, icons/logos
(small explicit size), and pattern-tiled backgrounds are exempt.
1.8 percent fire rate over the 3069-sample eval corpus, all verified
pictorial scenes; zero fires across val-a22/val-a24.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Detect a missing CLI before worker startup, keep Live usable through the foreground poller, and surface actionable status in Live and Live Lab.\n\nAI-assisted implementation.
Three new rules and three widenings, all from confirmed eval-corpus
escapes found by eye:
script-error (quality, error severity, URL engine): pageerror listener
attached before goto catches uncaught exceptions AND parse errors (a
syntax error fires during the initial parse, long before load). Deduped
by message, capped at 3. A JS typo was silently deleting whole pages.
content-hidden-at-rest (quality, error, URL engine): after the main
at-rest scan, an instant-scroll reveal sweep (bypasses scroll-behavior:
smooth, which silently defeated the first sweep design) gives every
IntersectionObserver reveal its chance to fire, returns to top, then
measures the share of text characters still at opacity 0 / visibility
hidden. display:none / [hidden] / aria-hidden subtrees stay out of the
denominator. Fires above 30% with a 200/150-char floor. Calibration on
30 corpus samples: broken repro holds 83% after the sweep, all clean
samples (including 0.75-0.93 at-rest reveal pages) drop to <= 7%.
edge-flush-cards (quality, warning, browser): cards with their own
opaque background or 2+ borders inside a horizontal scroller, flush
against one edge of the clip box at rest (< 8px, > -24px so deliberate
mid-card peeks stay exempt) while keeping a gutter on the other side.
Grouped per scroller. Repro: transit-mobile pager whose first snap
panel is 407px wide inside a 390px clip. New --viewport WxH CLI flag
makes mobile-width URL scans reachable (--viewport 390x844).
Chip/badge contrast widening: the SAFE_TAGS styled-button exception in
checkColors now covers any text-bearing element painting its own opaque
background at >= 9px font, not just a/button. The shipped miss: a span
SEV-2 chip whose white text lost a specificity fight and rendered
muted-on-red at 1.2:1. Static adapter also resolves var() own-bg via
the custom-property map so the gate engages on token backgrounds.
background:none cascade fix: the background shorthand now resets
background-color/-image when it names neither (and no var()). Exposed
by the chip widening: pre code { background: none } left an earlier
surface color standing and manufactured 1.1:1 phantom findings.
text-overflow inline-owner widening: inline elements have no client
geometry (clientWidth 0) so the scrollWidth path never saw them, and
their block parent owns no direct text. New branch measures the inline
rect against the nearest block container's padding box (16px floor,
transform-path exempt). Repro: nowrap span.v spilling 45px past its
grid cell.
The round-3 nav-CTA contrast escape (val-a22-opus obs 003 header CTA)
was verified already covered at HEAD by the earlier parseAnyColor
oklch fallback; both engines fire 3.6:1 on the repro, no change needed.
FP sweep across 36 val-a21/a22/a23 samples: new rules fire only on
their repros (script-error also catches a second genuinely broken
sample); static-engine delta is limited to the chip repro plus two
borderline-but-real chip findings on one sample.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
heading-rhythm (quality): a heading binds to the content it introduces,
so its rendered space above must exceed its space below. Browser-only:
measures real getBoundingClientRect gaps (margin collapsing, flex rows,
and section padding make authored margins untrustworthy), merges eyebrow
labels into the heading cluster, requires same-column edges, and exempts
first-in-container headings, bounded bands, and small cards. Fires only
when 2+ headings on a page invert the rhythm.
blinking-cursor (slop, advisory): a decorative blinking caret (solid
block, underscore bar, or block glyph) bound to an infinite blink
animation in the landing region of a page. Real editable surfaces
(contenteditable, role=textbox, inputs) are exempt; round pulsing dots
stay with the pulsing-dot rule.
Verified against eval corpus repros: heading-rhythm fires on the
val-a18 observability sample Paul flagged (6 headings, 0px above vs
40px below) and blinking-cursor on the val-a19 hero terminal cursor;
10 other samples across both runs stay clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The skill's decide-then-build step opens the built HTML artifact with a
DIRECTION CONTRACT comment (UNIQUE / NOT-TEMPLATE / OWN-WORLD / STORY /
FIRST VIEWPORT / FORM). Until now nothing ever judged the finished build
against that contract; the eval harness proved sample contracts promised
radical compositions while the build shipped the standard template anyway.
The Stop deep pass now extracts the leading contract comment from each
session-touched HTML file (marker match in the first 200 chars, body
capped at 1800 chars) and appends a contract-audit section after the
detector findings: audit the render promise by promise, naming the two
observed failure shapes (a promise not in the pixels; a contract whose
own plan is the standard template wearing the concept's nouns). Zero
extra API calls; the audit rides the existing single Stop emission and
fires at most once per file per session via a contractAudited flag on
the same session cache entry the finding dedupe uses.
Ported from the eval harness reference implementation
(extractDirectionContract / composeContractAuditMessage in
impeccable-evals runner/workers/anthropic-native.ts). No hooks.json
changes needed: Claude Code and Codex both already dispatch Stop to
hook.mjs.
Tests: 163 -> 179 in tests/hook.test.mjs (extraction unit coverage plus
Stop-pass integration: present/absent/once-per-session/non-HTML/
malformed/oversized). hook-build 18/18, build:skills prose gate clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Capture below-fold selected targets and each progressive variant reliably for portable evidence bundles. AI-assisted implementation under maintainer direction.
Four gaps found shipping in Opus 4.8 eval samples during human review:
1. low-contrast (extended): the browser adapters parsed text/own-bg
colors with parseRgb only, so Chrome's oklch()-serialized computed
colors silently skipped every contrast check — a flat dark-on-dark
nav CTA (broader nav selector beating the button class) shipped at
1.5:1 undetected. checkElementColorsDOM and readOwnBackgroundColor
now fall back to parseAnyColor. Near-threshold ratios print two
decimals so a 4.497 finding no longer reads "4.5 needs 4.5".
2. NEW numbered-section-labels (slop, advisory): tiny (<=13px) styled
numeric index labels riding beside section headings, repeated across
2+ sections with distinct indices. Sibling of repeated-section-kickers
(which deliberately excludes bare numeric labels); handles both the
direct prev-sibling shape and label-before-heading-wrapper shape.
List/nav/table/card-item numbering is exempt.
3. side-tab (extended): the vertical pseudo-element stripe scan required
the stripe to touch both corners (top/bottom 0 or height 100%), so a
left accent bar inset a few px from each end evaded it; small end
insets (<=20px each) now count. Added a browser-side pseudo-element
check (getComputedStyle(el, '::before'/'::after')) since runtime-
assigned custom-property colors are invisible to the text scanner.
Selection-state exemptions stay as narrowed: only aria-selected=true /
aria-current / active-class markers exempt, plus button/link
affordances on the horizontal variant.
4. NEW repeated-container-text (quality): the same literal string (>=4
chars, contains letters) rendered 3+ times at 3+ structurally distinct
positions inside one bordered/elevated container. Parallel/templated
repetition (table cells, calendar grids, nav lists, identical sibling
rows) never counts — structural signatures, not word lists.
Verified: each rule fires on its repro sample via the file:// browser
scan; clean eval samples add no new findings (the new low-contrast hits
on other samples are genuine sub-AA oklch button pairs). Full test
suite green; browser bundle regenerated; README/homepage rule counts
bumped 49 -> 51 (docs-integrity test enforces them).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Keep short crash-recovery leases without allowing a healthy worker to queue its own generation twice, and surface non-monotonic benchmark journals as errors.\n\nAI-assisted: OpenAI Codex.
Return after a durable starting record, overlap app-server initialization with page startup, dynamically reclaim generation after worker failure, and cap hard-crash leases at 15 seconds.\n\nAI-assisted: OpenAI Codex.
Let the browser E2E harness launch an independent production worker, exercise real sub-command selection, and carry realistic product/design context.\n\nAI-assisted: OpenAI Codex.
Validate and transactionally publish complete structured agent messages as soon as they arrive while retaining turn-completion serialization for subsequent phases.\n\nAI-assisted: OpenAI Codex.
Journal and stream dedicated worker phases so Live distinguishes first-variant design and validation from remaining-direction work without adding pollable events.\n\nAI-assisted: OpenAI Codex.
Compare direct Sol execution with cold and persistent app-server paths using identical full-task quality gates, lifecycle timings, and token metrics.\n\nAI-assisted: OpenAI Codex.
Tab-strip MEMBERSHIP no longer exempts chromatic top/bottom stripes —
only a real selection marker does: aria-selected="true", aria-current
(any non-false value), or an active/current/selected class hint. A
stripe repeated on every tab in the group ([role=tab], .tabs items,
aria-selected="false" tabs) is decoration and flags as side-tab; the
selected tab's own underline — including the reserved-space
transparent-border pattern — stays legal. Applied consistently across
the element border path (isTabContextElement), the pseudo-element
stripe scan, and the inset box-shadow stripe scan.
Also replaces a stray NUL byte in the marquee scanner's dedupe key
that made tools treat checks.mjs as binary.
Browser bundle regenerated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Route the full-context benchmark through production worker inputs and preserve established shared-control visual roles during variant amplification.\n\nAI-assisted: OpenAI Codex.
Default Codex to a dedicated Sol/medium app-server worker with native skill and image inputs, inherited project context, bounded source neighborhood evidence, and progressive context refresh. Other harnesses retain the portable foreground path.\n\nAI-assisted: OpenAI Codex.
Benchmark realistic bolder and polish tasks across fast, full-model, and full-context worker profiles with deterministic and independent quality gates.\n\nAI-assisted: OpenAI Codex.
Introduce a Live-owned app-server supervisor with progressive fenced publishing, partitioned control polling, cancellation and recovery safety, and measured integration coverage.
AI-assisted implementation under maintainer direction.
Four gaps from human review of gpt-5.6 eval artifacts:
1. codex-grid-background variants: the block scan now also matches the
inverted end-of-tile hairline form (transparent calc(100% - Npx))
and reads the tile cell from the background shorthand's `/ Npx Npx`
slot, not just background-size declarations. A single hairline layer
qualifies when tiled by a px pair cell (page-scale line field);
percent-tiled single hairlines (background-size: 25% 100% rules on
data-viz tracks/graphs) stay legal.
2. hero-eyebrow-chip branch C (dash-prefix): sentence-case, regular-
weight microlabels above the h1 announced by a short chromatic
::before/::after bar (8-80px x 1-6px, accent fill). Static cascade
marks dash-pseudo targets during rule collection; the browser path
reads getComputedStyle(el, '::before'/'::after').
3. New `marquee` slop rule: <marquee> elements, and infinite animations
bound to keyframes with >= 20 percentage points of X travel. Percent
travel only — px-travel loops are bespoke product animations
(waveform playheads, progress sweeps). Centered elements animating
other properties (constant -50% X), non-infinite slide-ins, rotations,
and pulses never qualify.
4. side-tab inset box-shadow variant: single-edge inset shadows
(3-12px offset on one axis, no blur/spread, chromatic) drawn as
stripes on cards/badges/menu items. Selection-state indicators
([aria-current], [aria-selected], [role=tab], active/current/selected
hints, interaction states) stay exempt; the same stripe repeated
unconditionally on every item flags. Narrow fixed-width glyphs
(logo marks) are exempt. isTabContextElement narrowed to match:
bare nav ancestry no longer blanket-exempts top/bottom border
stripes — only explicit tab semantics or state markers do.
Browser bundle regenerated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Four changes driven by human design review of eval artifacts:
1. Static engine contrast fidelity (nav-CTA cascade miss):
- parseAnyColor evaluates color-mix() (premultiplied sRGB mix; exact
for the dominant `color-mix(in oklab, C n%, transparent)` chip form)
- extractStaticColor captures color-mix() balanced instead of plucking
"transparent" out of the expression
- resolveBackground composites translucent layers over the opaque base
in both engines instead of skipping (static) or returning them
as-if-opaque (browser)
- NEW hover pass in the static cascade: :hover rules are matched via
state-stripped selectors, merged per-property against the resting
cascade with real specificity, and checked for WCAG contrast on
styled controls (checkHoverContrast). Catches the recurring miss
where a broader selector (.nav-links a:hover) beats the CTA's own
hover color and drops the pair below AA.
2. New `radial-halo` slop rule: chromatic radial-gradient wash (visible
saturated center -> transparent) as a decorative background on a dark
page. Exempts achromatic vignettes, opaque-end sheens, px-stop dot
textures, url() photo layers, and translucent (<0.7 alpha) staged-
light washes. Separate id from dark-glow so dashboards track the
gradient-drawn variant independently.
3. side-tab horizontal variant: 3-12px chromatic border-top/bottom (and
top/bottom-anchored full-width pseudo stripes) on cards/badges flag as
side-tab. Exempt: tablist/nav/aria-selected underlines, link/button
affordances, table cells, hr, state-conditional pseudo stripes, and
>12px bands. Badge-shaped spans (own visible background) participate.
4. CLI: file:// URLs route to the Puppeteer browser engine (~2s on a
50KB page), and detect --json findings now carry the registry
`category` field so downstream QA loops can separate mechanical slop
tells from judgment calls.
Fixture policy update: flat 3px top-accent cards moved from should-pass
to flag columns; tablist-underline and 16px-band pass cases added.
Browser bundle regenerated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Measure framework and provider latency, enforce fidelity and cleanup gates, and publish reproducible results on the dev-only Live Lab.\n\nAI-assisted: OpenAI Codex.
Two gaps surfaced by human eval review of real artifacts:
1. side-tab missed the pseudo-element variant. The accent stripe drawn as
an absolutely-positioned ::before/::after (left/right: 0, top+bottom: 0
or height: 100%, narrow width, colored background) uses no border
property at all, so neither the element-level border checks (pseudo
elements never enter the static cascade or DOM walk) nor the
border-left/right regexes could see it. New scanCssTextForPseudoStripe
scans stylesheet text for that shape, mirroring the border rule's
gates: >= 3px thick (<= 12px), chromatic fill (var()-resolved, neutral
dividers skipped), full height against a side edge, with the
blockquote/prose exemptions preserved.
2. New pulsing-dot rule (slop): small circular "live" indicator dots
(<= 16px, border-radius >= 40% or pill values) bound to an infinite
animation whose keyframes vary opacity, scale, or box-shadow — or
pulse/blink/ping names when the keyframes aren't in the scanned text —
plus the Tailwind animate-ping/pulse + rounded-full + tiny-size utility
combo. Rotation-only keyframes (spinners) never flag, including when
they hide behind a pulse-like name.
Both scanners live in checkHtmlPatterns, so the static-html engine and
the browser bundle share the same detection path. Browser/extension
bundles regenerated; docs rule count bumped to 47.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Paul Bakaus
Abdul WahabandGitHub