Files
pbakaus_impeccable/.github/workflows/ci.yml
T
Paul BakausandClaude Fable 5.1 e355ebf714 reorg: public plumbing for the in-repo Rust workspace and the two-release flow
The engine binaries move from the impeccable-dist channel to this repo's own
GitHub Releases (tag engine-v<ENGINE_VERSION>), and the closed detector the
engine links arrives as detector-v<DETECTOR_VERSION> releases on the same
repo. This commit wires the public side for that; the crates themselves land
in the next commit.

- Launcher (sh + cmd), npm shim, fetch-engine and check-engine-release now
  download from github.com/pbakaus/impeccable/releases/download/engine-v<X>/.
- release.mjs gains `engine`: verifies ENGINE_VERSION against the platform
  package pins and the detector release, tags, pushes; release-engine.yml
  builds the five targets and publishes. check-detector-release.mjs is the
  matching release-order guard (with tests).
- Root Cargo.toml (workspace, lto = false with the reason), rust-toolchain.toml
  (exact pin), DETECTOR_VERSION, /target ignored.
- CI: rust + rust-windows jobs and an oracle job that replays the goldens
  against a source build, warn-only until the first detector release exists;
  ci-test-plan exposes a `rust` output.
- docs/ENGINE.md (the crate map and the closed-detector mechanism) and the
  CLAUDE.md engine, release-order and rules sections.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vau2X53xGTjjTCXWMVBoNY
2026-09-01 14:05:39 -07:00

602 lines
21 KiB
YAML

name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
workflow_dispatch:
# Nightly full live-e2e matrix. The smoke groups already gate every PR; the
# full sweep is too slow for that, so it runs once a day against main.
schedule:
- cron: '0 7 * * *'
concurrency:
# Scheduled runs get their own group: the 07:00 UTC nightly and a push to
# main share github.ref, and cancel-in-progress would let them kill each
# other mid-run.
group: ${{ github.workflow }}-${{ github.event_name == 'schedule' && 'nightly' || github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
changes:
runs-on: ubuntu-latest
outputs:
core: ${{ steps.plan.outputs.core }}
rust: ${{ steps.plan.outputs.rust }}
detector: ${{ steps.plan.outputs.detector }}
live: ${{ steps.plan.outputs.live }}
framework: ${{ steps.plan.outputs.framework }}
cli_remote_e2e: ${{ steps.plan.outputs.cli_remote_e2e }}
live_e2e: ${{ steps.plan.outputs.live_e2e }}
live_e2e_accept_cleanup: ${{ steps.plan.outputs.live_e2e_accept_cleanup }}
skill_behavior: ${{ steps.plan.outputs.skill_behavior }}
live_svelte_adapter_deepseek: ${{ steps.plan.outputs.live_svelte_adapter_deepseek }}
steps:
- name: Checkout repository
uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Setup Node
uses: actions/setup-node@v7
with:
node-version: 24
- name: Detect test plan
id: plan
env:
GITHUB_EVENT_BEFORE: ${{ github.event.before }}
run: node scripts/ci-test-plan.mjs
test-matrix:
name: test (Node ${{ matrix.node-version }})
runs-on: ubuntu-latest
needs: changes
strategy:
fail-fast: false
matrix:
node-version: [22.18.0, 24]
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Setup Node
uses: actions/setup-node@v7
with:
node-version: ${{ matrix.node-version }}
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Run core tests
run: bun run test:core
- name: Install Puppeteer browser
if: needs.changes.outputs.detector == 'true'
run: bunx puppeteer browsers install chrome
- name: Run detector tests
if: needs.changes.outputs.detector == 'true'
run: bun run test:detector
- name: Run live unit tests
if: needs.changes.outputs.live == 'true'
run: bun run test:live
- name: Run framework fixture tests
if: needs.changes.outputs.framework == 'true'
run: bun run test:framework
- name: Rebuild browser detector
if: needs.changes.outputs.detector == 'true'
run: bun run build:browser
- name: Build
run: bun run build
- name: Build extension
if: needs.changes.outputs.detector == 'true'
run: bun run build:extension
- name: Lint Firefox extension (web-ext)
if: needs.changes.outputs.detector == 'true'
# Pinned for reproducible CI. Fails on AMO errors; innerHTML style
# warnings in the panel renderer are non-blocking and not promoted to
# errors here.
run: npx --yes web-ext@10 lint --source-dir dist/extension-firefox
- name: Verify generated tracked outputs
# cli/engine/ left the tree with the Rust engine swap; the vendored
# extension/detector/ path stays listed for when its vendoring lands.
run: git diff --exit-code -- .agents .claude .cursor .gemini .github/skills plugin extension/detector
- name: Upload build artifacts
uses: actions/upload-artifact@v7
with:
name: impeccable-build-node-${{ matrix.node-version }}
# Ship the packaged zips, not the unpacked Firefox staging tree.
path: |
dist/
!dist/extension-firefox/
retention-days: 7
# The Rust workspace: the engine binary and every crate behind it.
# crates/core/build.rs downloads the prebuilt closed detector archive for
# the pinned DETECTOR_VERSION (docs/ENGINE.md), so this job needs that
# release to exist.
#
# continue-on-error is a release-time toggle: until detector-v<DETECTOR_VERSION>
# is published, the archive cannot be fetched and the job would block every
# PR. Once it is live, flip continue-on-error to false so a Rust regression
# fails CI instead of only annotating it.
rust:
runs-on: ubuntu-latest
needs: changes
if: needs.changes.outputs.rust == 'true'
continue-on-error: true
steps:
- name: Checkout repository
uses: actions/checkout@v7
# rust-toolchain.toml pins the exact rustc the detector was built with;
# `rustup show` installs it. Never override the toolchain here.
- name: Install the pinned toolchain
run: rustup show
- uses: Swatinem/rust-cache@v2
- name: Build
id: build
continue-on-error: true
run: cargo build --workspace --all-targets
- name: Test
if: steps.build.outcome == 'success'
run: cargo test --workspace
- name: Annotate missing detector release
if: steps.build.outcome != 'success'
run: |
echo "::warning title=Rust workspace not built::cargo build failed. If crates/core/build.rs could not download detector-v$(cat DETECTOR_VERSION), the detector release is not published yet; publish it (tag the private detector repo) and flip this job's continue-on-error to false. Otherwise this is a real build failure."
exit 1
# The engine ships a windows-x64 binary (release-engine.yml), so the
# workspace has to build and pass its own tests there. Tests that need a
# browser or the oracle skip when those are absent.
rust-windows:
runs-on: windows-latest
needs: changes
if: needs.changes.outputs.rust == 'true'
continue-on-error: true
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Install the pinned toolchain
run: rustup show
- uses: Swatinem/rust-cache@v2
- run: cargo build --workspace --all-targets
- run: cargo test --workspace
# Behavior gate: replays the tests/oracle/ goldens against a release build
# of the engine from THIS checkout (so a PR is judged on its own source,
# not on the last published binary). Without this job the oracle only ever
# runs on developer laptops: tests/oracle.test.mjs skips cleanly when no
# binary is present, so the default suite is silent about it on CI.
#
# continue-on-error: same release-time toggle as `rust` above.
oracle:
runs-on: ubuntu-latest
needs: changes
if: needs.changes.outputs.oracle == 'true' || needs.changes.outputs.rust == 'true'
continue-on-error: true
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Setup Node
uses: actions/setup-node@v7
with:
node-version: 24
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Install the pinned toolchain
run: rustup show
- uses: Swatinem/rust-cache@v2
- name: Build the engine from source
id: build
continue-on-error: true
run: cargo build --release -p impeccable
- name: Replay oracle goldens
if: steps.build.outcome == 'success'
env:
IMPECCABLE_BIN: ${{ github.workspace }}/target/release/impeccable
run: node tests/oracle/run.mjs
- name: Annotate missing detector release
if: steps.build.outcome != 'success'
run: |
echo "::warning title=Oracle not run::the engine did not build (see the rust job). The oracle behavior gate did NOT run. Expected until detector-v$(cat DETECTOR_VERSION) is published; after that, flip this job's continue-on-error to false."
exit 1
# Release-order guard (triage decision D4). Verifies that the engine release for
# the pinned ENGINE_VERSION is fully published — the five dist binaries + .sha256
# AND the five @impeccable/cli-<os>-<arch> npm platform packages — before a skill
# release/merge that depends on them. The launcher, npm shim, and
# `impeccable install` all dead-end without those assets.
#
# continue-on-error is a release-time toggle: until the first engine release is
# published, the assets cannot exist and this job would block
# every PR. It emits a loud ::warning instead. Once v<ENGINE_VERSION> is live,
# flip `continue-on-error` to false so a MIS-ORDERED release (skill/CLI ahead of
# the engine) fails CI. release.mjs already hard-fails `release:skill`/`release:cli`.
engine-release-ready:
runs-on: ubuntu-latest
continue-on-error: true
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Setup Node
uses: actions/setup-node@v7
with:
node-version: 24
- name: Check engine release assets for pinned ENGINE_VERSION
id: check
continue-on-error: true
run: node scripts/check-engine-release.mjs
- name: Annotate missing engine release
if: steps.check.outcome != 'success'
run: |
echo "::warning title=Engine release not ready::The engine release for v$(cat ENGINE_VERSION) is not fully published (engine-v$(cat ENGINE_VERSION) release) and/or the @impeccable/cli-<os>-<arch> npm platform packages. Releasing the skill/CLI (or merging) now would dead-end the launcher, the npm shim, and impeccable install. Expected until the first engine release exists; after that, publish the engine + platform packages and flip this job's continue-on-error to false so a mis-ordered release fails CI."
test:
runs-on: ubuntu-latest
needs: test-matrix
if: always()
steps:
- name: Verify Node test matrix
run: |
if [ "${{ needs.test-matrix.result }}" != "success" ]; then
echo "test-matrix result: ${{ needs.test-matrix.result }}"
exit 1
fi
echo "test matrix passed"
cli-remote-e2e:
runs-on: ubuntu-latest
needs: changes
if: needs.changes.outputs.cli_remote_e2e == 'true'
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Setup Node
uses: actions/setup-node@v7
with:
node-version: 24
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Run remote CLI E2E smoke
run: bun run test:cli-remote-e2e
live-e2e-smoke:
name: live-e2e smoke (${{ matrix.group }})
runs-on: ubuntu-latest
needs: changes
if: needs.changes.outputs.live_e2e == 'true' && github.event_name != 'workflow_dispatch' && github.event_name != 'schedule'
timeout-minutes: 15
strategy:
fail-fast: true
matrix:
include:
- group: platform
fixtures: astro-vite7,monorepo-nested-vite,nextjs-app-router,vite8-sveltekit
- group: svelte
fixtures: vite8-sveltekit-stateful
- group: react
fixtures: vite8-react-css-modules,vite8-react-insert,vite8-react-plain
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Setup Node
uses: actions/setup-node@v7
with:
node-version: 24
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Cache fixture npm downloads
uses: actions/cache@v6
with:
path: ~/.npm
key: ${{ runner.os }}-fixture-npm-${{ hashFiles('tests/framework-fixtures/**/files/package.json') }}
restore-keys: |
${{ runner.os }}-fixture-npm-
- name: Cache Playwright Chromium
uses: actions/cache@v6
with:
path: ~/.cache/ms-playwright
key: ${{ runner.os }}-playwright-chromium-${{ hashFiles('package.json', 'bun.lock') }}
restore-keys: |
${{ runner.os }}-playwright-chromium-
- name: Install dependencies
run: bun install
- name: Install Playwright Chromium
run: npx playwright install chromium
- name: Run live E2E tests
run: bun run test:live-e2e
env:
IMPECCABLE_E2E_ONLY: ${{ matrix.fixtures }}
IMPECCABLE_E2E_SCENARIOS: core
IMPECCABLE_E2E_TEST_TIMEOUT_MS: 180000
IMPECCABLE_E2E_INSTALL_TIMEOUT_MS: 120000
IMPECCABLE_E2E_DEV_READY_TIMEOUT_MS: 60000
IMPECCABLE_E2E_ARTIFACT_DIR: test-results/live-e2e/${{ matrix.group }}
- name: Upload live E2E failure artifacts
if: failure()
uses: actions/upload-artifact@v7
with:
name: live-e2e-smoke-${{ matrix.group }}-artifacts
path: test-results/live-e2e
if-no-files-found: ignore
live-e2e-full:
name: live-e2e full (${{ matrix.group }})
runs-on: ubuntu-latest
needs: changes
if: needs.changes.outputs.live_e2e == 'true' && (github.event_name == 'workflow_dispatch' || github.event_name == 'schedule')
timeout-minutes: 25
strategy:
fail-fast: true
matrix:
include:
- group: platform
fixtures: astro-vite7,monorepo-nested-vite,nextjs-app-router
- group: svelte
fixtures: vite8-sveltekit,vite8-sveltekit-stateful
- group: react-a
fixtures: vite8-https,vite8-react-base-path,vite8-react-csp-meta,vite8-react-css-modules,vite8-react-emotion
- group: react-b
fixtures: vite8-react-insert,vite8-react-mapped-list,vite8-react-modal,vite8-react-plain
- group: stateful
fixtures: vite8-react-radix-dialog,vite8-react-router-spa,vite8-react-styled-components,vite8-react-tabs
- group: styling
fixtures: vite8-react-tailwindv3,vite8-react-tailwindv4,vite8-react-ts,vite8-react-tsx-repeated-aside,vite8-react-unocss,vite8-react-vanilla-extract
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Setup Node
uses: actions/setup-node@v7
with:
node-version: 24
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Cache fixture npm downloads
uses: actions/cache@v6
with:
path: ~/.npm
key: ${{ runner.os }}-fixture-npm-${{ hashFiles('tests/framework-fixtures/**/files/package.json') }}
restore-keys: |
${{ runner.os }}-fixture-npm-
- name: Cache Playwright Chromium
uses: actions/cache@v6
with:
path: ~/.cache/ms-playwright
key: ${{ runner.os }}-playwright-chromium-${{ hashFiles('package.json', 'bun.lock') }}
restore-keys: |
${{ runner.os }}-playwright-chromium-
- name: Install dependencies
run: bun install
- name: Install Playwright Chromium
run: npx playwright install chromium
- name: Run live E2E tests
run: bun run test:live-e2e
env:
IMPECCABLE_E2E_ONLY: ${{ matrix.fixtures }}
IMPECCABLE_E2E_TEST_TIMEOUT_MS: 300000
IMPECCABLE_E2E_INSTALL_TIMEOUT_MS: 180000
IMPECCABLE_E2E_DEV_READY_TIMEOUT_MS: 120000
IMPECCABLE_E2E_ARTIFACT_DIR: test-results/live-e2e/${{ matrix.group }}
- name: Upload live E2E failure artifacts
if: failure()
uses: actions/upload-artifact@v7
with:
name: live-e2e-full-${{ matrix.group }}-artifacts
path: test-results/live-e2e
if-no-files-found: ignore
live-e2e-accept-cleanup:
runs-on: ubuntu-latest
needs: changes
if: needs.changes.outputs.live_e2e_accept_cleanup == 'true'
timeout-minutes: 15
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
steps:
- name: Skip without provider key
if: ${{ env.ANTHROPIC_API_KEY == '' && env.DEEPSEEK_API_KEY == '' }}
run: echo "Skipping provider-backed accept-cleanup regression because no provider API key is configured."
- name: Checkout repository
if: ${{ env.ANTHROPIC_API_KEY != '' || env.DEEPSEEK_API_KEY != '' }}
uses: actions/checkout@v7
- name: Setup Node
if: ${{ env.ANTHROPIC_API_KEY != '' || env.DEEPSEEK_API_KEY != '' }}
uses: actions/setup-node@v7
with:
node-version: 24
- name: Setup Bun
if: ${{ env.ANTHROPIC_API_KEY != '' || env.DEEPSEEK_API_KEY != '' }}
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Cache fixture npm downloads
if: ${{ env.ANTHROPIC_API_KEY != '' || env.DEEPSEEK_API_KEY != '' }}
uses: actions/cache@v6
with:
path: ~/.npm
key: ${{ runner.os }}-fixture-npm-${{ hashFiles('tests/framework-fixtures/**/files/package.json') }}
restore-keys: |
${{ runner.os }}-fixture-npm-
- name: Cache Playwright Chromium
if: ${{ env.ANTHROPIC_API_KEY != '' || env.DEEPSEEK_API_KEY != '' }}
uses: actions/cache@v6
with:
path: ~/.cache/ms-playwright
key: ${{ runner.os }}-playwright-chromium-${{ hashFiles('package.json', 'bun.lock') }}
restore-keys: |
${{ runner.os }}-playwright-chromium-
- name: Install dependencies
if: ${{ env.ANTHROPIC_API_KEY != '' || env.DEEPSEEK_API_KEY != '' }}
run: bun install
- name: Install Playwright Chromium
if: ${{ env.ANTHROPIC_API_KEY != '' || env.DEEPSEEK_API_KEY != '' }}
run: npx playwright install chromium
- name: Run accept cleanup regression
if: ${{ env.ANTHROPIC_API_KEY != '' || env.DEEPSEEK_API_KEY != '' }}
run: |
if [ -n "$DEEPSEEK_API_KEY" ]; then
export IMPECCABLE_E2E_LLM_PROVIDER=deepseek
else
export IMPECCABLE_E2E_LLM_PROVIDER=anthropic
fi
bun run test:live-e2e-accept-cleanup
live-svelte-adapter-deepseek:
runs-on: ubuntu-latest
needs: changes
if: needs.changes.outputs.live_svelte_adapter_deepseek == 'true'
timeout-minutes: 25
env:
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
steps:
- name: Skip without DeepSeek key
if: ${{ env.DEEPSEEK_API_KEY == '' }}
run: echo "Skipping Svelte adapter DeepSeek sweep because DEEPSEEK_API_KEY is not configured."
- name: Checkout repository
if: ${{ env.DEEPSEEK_API_KEY != '' }}
uses: actions/checkout@v7
- name: Setup Node
if: ${{ env.DEEPSEEK_API_KEY != '' }}
uses: actions/setup-node@v7
with:
node-version: 24
- name: Setup Bun
if: ${{ env.DEEPSEEK_API_KEY != '' }}
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Cache fixture npm downloads
if: ${{ env.DEEPSEEK_API_KEY != '' }}
uses: actions/cache@v6
with:
path: ~/.npm
key: ${{ runner.os }}-fixture-npm-${{ hashFiles('tests/framework-fixtures/**/files/package.json') }}
restore-keys: |
${{ runner.os }}-fixture-npm-
- name: Cache Playwright Chromium
if: ${{ env.DEEPSEEK_API_KEY != '' }}
uses: actions/cache@v6
with:
path: ~/.cache/ms-playwright
key: ${{ runner.os }}-playwright-chromium-${{ hashFiles('package.json', 'bun.lock') }}
restore-keys: |
${{ runner.os }}-playwright-chromium-
- name: Install dependencies
if: ${{ env.DEEPSEEK_API_KEY != '' }}
run: bun install
- name: Install Playwright Chromium
if: ${{ env.DEEPSEEK_API_KEY != '' }}
run: npx playwright install chromium
- name: Run Svelte adapter DeepSeek sweep
if: ${{ env.DEEPSEEK_API_KEY != '' }}
run: bun run test:live-svelte-adapter-deepseek
skill-behavior:
runs-on: ubuntu-latest
needs: changes
if: needs.changes.outputs.skill_behavior == 'true' && github.event_name != 'pull_request'
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
GOOGLE_CLOUD_API_KEY: ${{ secrets.GOOGLE_CLOUD_API_KEY }}
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Setup Node
uses: actions/setup-node@v7
with:
node-version: 24
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Run skill behavior tests
run: bun run test:skill-behavior