skills/pbakaus_impeccable
1
0
Fork 0
mirror of https://github.com/pbakaus/impeccable.git synced 2026-09-11 21:57:14 +03:00
Code Issues Packages Projects Releases Wiki Activity
Files
4c5243fcd42d39c1fc281adcaf10be0913095f74
pbakaus_impeccable/scripts/test-suites.mjs
T
Paul BakausGitHubClaude Fable 5.1
 4c5243fcd4 Tests: stop the harness leaking live-server processes (#718) 
* Tests: stop the harness leaking live-server processes

Nothing owned a live server past the exit paths JavaScript can observe. The
live unit tests spawn the server as a direct child and stop it with an HTTP
/stop plus proc.kill() inside an after() hook; the e2e session and the
target-context tests boot it through `live-server --background` / live.mjs,
which spawns a detached, unref'd daemon that only the `stop` verb ever ends.
A POSIX child does not die with its parent, and a detached daemon is orphaned
to pid 1 from birth, so any exit that skipped teardown (a node:test timeout, a
SIGKILL of the runner, a Ctrl-C, an assertion that threw before the hook) left
the server listening on a fixed live-suite port for good. scripts/run-tests.mjs
did not compensate: it used blocking spawnSync, so no signal handler could run;
it left suite commands in its own process group with nothing that could kill
that group; and it never checked afterwards whether anything survived. Days of
local runs accumulated 197 orphans on one machine, the oldest four days old,
until `bun run test:live` could not claim its ports.

The fix is structural rather than a cleanup sweep bolted on the end, and it is
deliberately implementation-agnostic so it holds for the Node scripts here and
for the Rust `impeccable live-server` on rust-swap:

- tests/lib/live-servers.mjs. armLiveServerReaper(), called once at module
  scope by every test file that starts a server, stamps the process env with a
  unique marker, installs exit and signal handlers, and spawns a detached
  reaper holding a pipe to the process. SIGKILL the process and the pipe closes,
  the reaper wakes on EOF and kills the servers carrying that marker. That is
  the one case no in-process cleanup can reach. trackServerChild() also
  registers direct children (live servers and fixture dev servers) so the
  ordinary exits are a cheap kill by handle.
- scripts/lib/live-server-processes.mjs. The scan and kill primitives, shared
  by the reaper and the runner. Processes are matched by the environment marker
  the harness exported, never by name or port, so a sweep can only ever reach a
  server this repo's tests started.
- scripts/run-tests.mjs. Each suite command now runs as its own process-group
  leader with SIGINT/SIGTERM/SIGHUP forwarded to the group, and after every
  suite the runner checks for live servers carrying that suite's run id. A
  survivor is killed and fails the run, so the next leak surfaces in the run
  that caused it instead of on a laptop days later. IMPECCABLE_SKIP_LEAK_CHECK=1
  bypasses it. `bun run test:cleanup` sweeps leftovers from earlier runs.
- tests/live-server-leak.test.mjs pins the guarantee: it boots a real server
  under a process it then SIGKILLs, and fails if the server outlives it. With
  IMPECCABLE_NO_TEST_REAPER=1 the test fails, which is what makes it a
  regression test rather than a tautology.

Verified: bun run test:live green with zero survivors; scoped live-e2e
(vite8-react-plain) matches pristine main test for test; the SIGKILL repro goes
from 2 orphans to 0; SIGINT and SIGKILL of the runner itself both leave nothing
behind; bun run build green.

Fixes #717

AI assistance: prepared by Claude Code under pbakaus's direction.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vau2X53xGTjjTCXWMVBoNY

* Review fixes: scope the sweep to whole env entries only

Five review findings on #718, all in the matching layer that decides which
processes a sweep may touch.

The repository-path fallback is gone (Greptile P1). `bun run test:cleanup`
passed REPO_ROOT to findLiveServers, which then also matched any live-server
command line under the checkout, marker or not. A developer running
`impeccable live` in this repo has exactly that command line, so the cleanup
could have killed their own session. The PR promised matching on the exported
environment marker and nothing else; now it does. The cost is that a server
from a run predating the marker is no longer found and has to be killed by
hand, which is the right trade.

Environment entries are compared whole on macOS and BSD (Greptile P1). `ps -E`
flattens the environment into the command column, and that line was searched
with a plain substring test, so IMPECCABLE_TEST_REPO=/work/impeccable also
matched /work/impeccable-copy and one checkout's cleanup could reach a
neighbouring checkout's servers. envLineHasEntry() now requires the marker to
start an entry (line start or whitespace) and to end one (line end, or
whitespace followed by the next KEY=), which is the same whole-entry
comparison the Linux /proc branch already did. Six unit tests cover it,
including the adjacent-path negative case, and a live probe against real
`ps -E` output confirms an exact repo matches while /work/impeccable-copy and
a run-id prefix do not.

The SIGKILL regression test now skips on win32 with a stated reason (Copilot).
The reaper is a POSIX mechanism and armLiveServerReaper() does not arm it
there, so the test asserted a guarantee Windows does not make yet.

Signal exits use the shell convention 128 + signum in both the runner and the
test helper (Copilot, two threads). SIGHUP returned 143; it is 129. Read from
os.constants.signals rather than a hand-written table.

Verified: leak test 7/7 (2 guard, 5 matcher); bun run test:live 895 tests, 0
fail, 0 survivors; scoped live-e2e (vite8-react-plain) 3 pass / 1 fail,
matching pristine main; SIGKILL repro 3 servers up, 0 after; bun run build
green.

AI assistance: prepared by Claude Code under pbakaus's direction.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vau2X53xGTjjTCXWMVBoNY

* Review fix: make marker values opaque so the matcher has no ambiguous case

Greptile's follow-up P1 on the parser was right, and the parser was the wrong
place to answer it. envLineHasEntry ended an entry at "whitespace followed by
the next KEY=", so a checkout path that extended another one with whitespace
plus a KEY=-shaped token still defeated it, which is exactly the ambiguity the
docblock admitted to. A format that cannot be parsed unambiguously should not
be handed ambiguous input.

So the fix is at the source: no marker value is a path any more. IMPECCABLE_TEST_REPO
now carries repoMarker(), the first 16 hex characters of the sha256 of the
checkout's real path, and the runner and the cleanup command both compute it
the same way from REPO_ROOT. Two checkouts whose paths share a prefix get
unrelated hashes, so a substring cannot arise in the first place, and every
spelling of one checkout (trailing slash, `.` segment, symlink, /private
prefix) resolves to one marker. The run id is now repoMarker plus 8 random
bytes of hex, and the process id p<pid> plus the same, both from a
whitespace-free alphabet.

With every value fixed-alphabet, envLineHasEntry needs only "starts an entry
and ends at whitespace or line end". The KEY= lookahead is gone and so is the
documented unresolvable case. assertMarkerValue keeps the invariant honest: it
refuses any value outside [A-Za-z0-9_-] with a message that says to hash it,
so a future caller that passes a path gets a loud error instead of a silent
mismatch. The readable path is still available for a human reading `ps -E`
output, exported separately as IMPECCABLE_TEST_REPO_PATH, which nothing
matches on and the docblock says so.

Matcher tests: the space-in-value case is gone, since that value can no longer
exist. Added a strict-prefix case (a longer hash-shaped value starting with the
marker), an adjacent-checkout case asserting the two hashes do not even share a
prefix, a symlink/trailing-slash case against real directories, an alphabet
check on all three generators, and one asserting assertMarkerValue throws.

Verified: leak test 10/10; bun run test:live 898 tests, 0 fail, 0 survivors;
scoped live-e2e (vite8-react-plain) 3 pass / 1 fail, matching pristine main;
SIGKILL repro 1 server up, 0 after; bun run build green. A probe against real
`ps -E` output with a hashed marker: this checkout 1 match, its trailing-slash
spelling 1, an adjacent checkout 0, exact run id 1, a run-id prefix 0.

AI assistance: prepared by Claude Code under pbakaus's direction.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vau2X53xGTjjTCXWMVBoNY

* Review fixes: async group shutdown, and a Windows-safe symlink test

Two Cursor Bugbot findings, both real.

killCurrentGroup busy-waited on alive(child.pid) after sending SIGTERM, which
could never work. A dead child stays a zombie until its parent reaps it, the
parent here is the runner, and the runner reaps through libuv when the event
loop runs. The spin blocked the very loop that would have done the reaping and
then read the unreaped zombie as alive, so every SIGINT, SIGTERM and SIGHUP
burned the full 2s grace and ended in a needless SIGKILL. There is no waitpid
from JavaScript that sees through this, so the wait is now asynchronous and
keyed on the child's own exit event. The logic moved to
scripts/lib/process-group.mjs: trackChildExit exposes the exit as a flag and a
promise, stopGroup races that promise against the grace period and escalates to
SIGKILL only if it loses, and killGroupSync stays synchronous for
process.on('exit'), where nothing can be awaited, so it sends SIGTERM then
SIGKILL without pretending to wait. A second Ctrl-C now skips the grace period
entirely rather than queueing behind it.

Measured on a real SIGINT to a running live suite: 2027ms before, 34ms after.
tests/process-group.test.mjs pins both halves, including the escalation path
against a child that traps SIGTERM, which is not otherwise reachable from a
registered suite.

The repoMarker symlink test called symlinkSync with no type, which throws EPERM
on Windows without Developer Mode. It now passes 'junction' there and 'dir'
elsewhere, the same shape tests/concept-seed.test.mjs uses, and the
trailing-slash and dot-segment cases split into their own test so they keep
running on every platform regardless.

Merged origin/main (through #716) to re-level the branch.

Verified: leak and process-group tests 16/16; bun run test:live 900 tests, 0
fail, 0 survivors; scoped live-e2e (vite8-react-plain) now 4/4, with the
orphaned-session test that #716 fixed passing in 7.2s; bun run build green.

AI assistance: prepared by Claude Code under pbakaus's direction.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vau2X53xGTjjTCXWMVBoNY

* Review fix: a second Ctrl-C must reach the group the first one is stopping

Cursor Bugbot caught a bug I introduced with the async shutdown, and it is the
same class of leak this PR exists to close. The signal handler cleared
currentChild before awaiting stopGroup, so a second Ctrl-C read a null handle:
killGroupSync did nothing, process.exit walked away from the SIGKILL escalation
still in flight, and because the suite is spawned detached it kept running
after the runner was gone. Impatience with a stuck suite produced exactly the
orphan the change is supposed to prevent.

The shutdown state machine moved into scripts/lib/process-group.mjs as
createGroupShutdown, which holds the group in `stopping` for as long as it is
being ended rather than dropping the only reference to it. A second signal
kills that handle and leaves; process.on('exit') looks at `current` or
`stopping`, so the last-resort path reaches a group mid-shutdown too. The
runner keeps no shutdown state of its own now, which is what made the bug
possible to write in the first place.

The extraction is what makes it testable: `exit` is injectable, so
tests/process-group.test.mjs can drive two signals at a stubborn child that
traps SIGTERM and assert the group dies in under 2s against a 30s grace. Point
that test at the old logic (killGroupSync on the cleared reference) and it
hangs out the full grace and fails, which is the check that it pins something
real. Five cases in all, including the exit-handler path and the no-child case.

Verified: process-group 10/10, live-server-leak 11/11; real double SIGINT to a
running live suite exits in 24ms with zero group members and zero servers left;
bun run test:live 900 tests, 0 fail, 0 survivors; scoped live-e2e
(vite8-react-plain) 4/4; bun run build green.

The core suite wedged twice locally in tests/build-phase.test.mjs, the
pre-existing unbounded-spawnSync hang noted in the PR description that
rust-swap's 47f18713 fixes. Unrelated to this change: CI is green on both Node
versions, and process-group.test.mjs passes inside that batch.

AI assistance: prepared by Claude Code under pbakaus's direction.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vau2X53xGTjjTCXWMVBoNY

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 19:21:30 -07:00

441 lines
15 KiB
JavaScript
Raw Blame History

import fs from 'node:fs';
import path from 'node:path';
export const DEFAULT_SUITES = ['core', 'detector', 'live', 'framework', 'plugin-e2e'];
export const OPT_IN_SUITES = [
'cli-remote-e2e',
'live-e2e',
'live-e2e-accept-cleanup',
'new-work-e2e',
'skill-behavior',
'live-svelte-adapter-deepseek',
];
const COMMON_INFRA_PATTERNS = [
/^package\.json$/,
/^bun\.lock$/,
/^scripts\/run-tests\.mjs$/,
/^scripts\/test-suites\.mjs$/,
/^scripts\/ci-test-plan\.mjs$/,
/^scripts\/lib\/(live-server-processes|process-group|test-orphan-reaper)\.mjs$/,
/^tests\/lib\/live-servers\.mjs$/,
/^\.github\/workflows\/ci\.yml$/,
];
export const SUITES = {
core: {
description: 'Build, provider transforms, CLI helpers, context, and storage unit tests.',
triggers: [
...COMMON_INFRA_PATTERNS,
/^scripts\/(?!benchmark-detector|build-browser-detector|build-extension)/,
/^skill\/(SKILL\.src\.md|agents\/|reference\/|scripts\/(cleanup-deprecated|comp-diff|comp-spec|build-phase|font-match|data\/font-index|concept-seed|generate-image|context|context-signals|critique-storage|design-parser|doctor|hook|impeccable-paths|is-generated|lib\/(artifact-schema|png|raster|image-metrics|font-fingerprint|font-index|hero-checks|composition-catalog|concept-catalog|provider|staleness|staleness-deep|staleness-notice|surface-briefs|target-slug|template-extensions)|pin|surface-brief))/,
/^README(\.npm)?\.md$/,
/^cli\/bin\//,
],
commands: [
{
runner: 'bun',
files: [
'tests/build.test.js',
'tests/cli-ignores.test.js',
'tests/copy-provider-commands.test.js',
'tests/windows-path-fix.test.js',
'tests/lib/provider-blocks.test.js',
'tests/lib/transformers/provider-blocks.test.js',
'tests/lib/utils.test.js',
'tests/lib/impeccable-config.test.js',
'tests/lib/transformers/factory.test.js',
'tests/lib/transformers/opencode-commands.test.js',
'tests/lib/transformers/providers.test.js',
'tests/root-commands-sync.test.js',
'tests/skills-cli.test.js',
'tests/validate-plugin-versions.test.js',
'tests/validate-plugin-manifest.test.js',
'tests/plugin-paths.test.js',
],
},
{
runner: 'node',
files: [
'tests/ci-test-plan.test.mjs',
'tests/cli-args.test.mjs',
'tests/concept-seed.test.mjs',
'tests/generate-image-embed.test.mjs',
'tests/comp-diff.test.mjs',
'tests/build-phase.test.mjs',
'tests/font-match.test.mjs',
'tests/hero-checks.test.mjs',
'tests/serve-question.test.mjs',
'tests/context.test.mjs',
'tests/context-signals.test.mjs',
'tests/critique-storage.test.mjs',
'tests/design-parser.test.mjs',
'tests/github-sheriff.test.mjs',
'tests/hook-build.test.mjs',
'tests/hook.test.mjs',
'tests/impeccable-paths.test.mjs',
'tests/openai-plugin.test.mjs',
'tests/pin.test.mjs',
'tests/process-group.test.mjs',
'tests/release.test.mjs',
'tests/doctor.test.mjs',
'tests/staleness.test.mjs',
'tests/skill-reference.test.mjs',
'tests/readme-gitignore.test.mjs',
'tests/target-args.test.mjs',
'tests/surface-brief.test.mjs',
'tests/template-extensions.test.mjs',
'tests/test-suites.test.mjs',
'tests/zip.test.mjs',
],
},
],
},
detector: {
description: 'Anti-pattern detector tests across text, jsdom fixtures, and Puppeteer browser paths.',
needsPuppeteer: true,
triggers: [
...COMMON_INFRA_PATTERNS,
/^cli\/engine\//,
/^extension\/(background|content|detector|devtools|popup|manifest\.json)/,
/^scripts\/(benchmark-detector|build-browser-detector|build-extension)\.js$/,
/^site\/(pages\/detector|public\/antipattern|data\/anti-patterns-catalog\.js)/,
/^tests\/fixtures\/antipatterns/,
],
commands: [
{
runner: 'bun',
files: [
'tests/detect-antipatterns.test.js',
'tests/detect-url-launch.test.mjs',
'tests/inline-ignores.test.mjs',
'tests/lib/detector-bundle.test.js',
],
},
{
runner: 'node',
files: [
'tests/extension-build.test.mjs',
'tests/design-system.test.mjs',
'tests/detect-antipatterns-fixtures.test.mjs',
'tests/detect-antipatterns-browser.test.mjs',
'tests/detect-cli-design-contamination.test.mjs',
'tests/detect-cli-design-monorepo.test.mjs',
'tests/detect-cli-stdin-dispatch.test.mjs',
],
},
],
},
live: {
description: 'Fast live-mode unit and local-server integration tests, excluding full browser fixture sweeps.',
triggers: [
...COMMON_INFRA_PATTERNS,
// `palette` is deliberately absent: skill/scripts/palette.mjs has no
// test anywhere, and listing it here made edits run a suite that never
// touches it, which reads as coverage that does not exist.
/^skill\/(reference\/live\.md|scripts\/(detect-csp|lib\/is-generated|lib\/template-extensions|live\/|live|live-|modern-screenshot|pin))/,
/^tests\/live-/,
],
commands: [
{
runner: 'node',
files: [
'tests/live-accept.test.mjs',
'tests/live-accept-css.test.mjs',
'tests/live-accept-scrub.test.mjs',
'tests/live-browser-dom.test.mjs',
'tests/live-browser-ignores.test.mjs',
'tests/live-browser-script-parts.test.mjs',
'tests/live-browser-regression.test.mjs',
'tests/live-browser-session.test.mjs',
'tests/live-browser-source.test.mjs',
'tests/live-commit-manual-edits.test.mjs',
'tests/live-completion.test.mjs',
'tests/live-copy-edit-agent.test.mjs',
'tests/live-discard-manual-edits.test.mjs',
'tests/live-e2e-agent-output.test.mjs',
'tests/live-e2e-cli-options.test.mjs',
'tests/live-e2e-llm-agent.test.mjs',
'tests/live-e2e-steer-agent.test.mjs',
'tests/live-e2e/agent-insert.test.mjs',
'tests/live-event-validation.test.mjs',
'tests/live-frameworks.test.mjs',
'tests/live-generation-preflight.test.mjs',
'tests/live-inject.test.mjs',
'tests/live-insert.test.mjs',
'tests/live-insert-ui.test.mjs',
'tests/live-manual-edits-buffer.test.mjs',
'tests/live-poll.test.mjs',
'tests/live-project-ignores.test.mjs',
'tests/live-poll-lanes.test.mjs',
'tests/live-poll-stream.test.mjs',
'tests/live-recovery-commands.test.mjs',
'tests/live-reference.test.mjs',
'tests/live-roots.test.mjs',
'tests/live-server.test.mjs',
'tests/live-server-leak.test.mjs',
'tests/live-session-store.test.mjs',
'tests/live-source-lock.test.mjs',
'tests/live-source-search.test.mjs',
'tests/live-svelte-ast.test.mjs',
'tests/live-svelte-component-accept.test.mjs',
'tests/live-svelte-props-script.test.mjs',
'tests/live-tanstack-adapter.test.mjs',
'tests/live-target-context.test.mjs',
'tests/live-ui-surfaces.test.mjs',
'tests/live-wrap.test.mjs',
'tests/live-wrap-buffer-aware.test.mjs',
],
},
],
},
framework: {
description: 'Framework fixture coverage for live injection, CSP, generated-file detection, and wrapping.',
triggers: [
...COMMON_INFRA_PATTERNS,
/^tests\/framework-fixtures/,
/^tests\/framework-fixtures\.test\.mjs$/,
/^skill\/scripts\/(detect-csp|live-inject|live-wrap)\.mjs$/,
/^skill\/scripts\/lib\/is-generated\.mjs$/,
/^skill\/scripts\/lib\/template-extensions\.mjs$/,
/^skill\/scripts\/live\/(source-search|sveltekit-adapter|tanstack-adapter)\.mjs$/,
/^skill\/scripts\/live\/frameworks\//,
],
commands: [
{
runner: 'node',
files: ['tests/framework-fixtures.test.mjs'],
},
],
},
'cli-e2e': {
description: 'Deterministic CLI install/update tests against a local universal bundle.',
commands: [
{
runner: 'bun',
files: ['tests/skills-cli.test.js'],
},
],
},
'cli-remote-e2e': {
description: 'Remote CLI install/update smoke tests against impeccable.style.',
optIn: true,
triggers: [
...COMMON_INFRA_PATTERNS,
/^cli\/bin\/commands\/skills\.mjs$/,
/^tests\/skills-cli\.test\.js$/,
],
commands: [
{
runner: 'bun',
env: { IMPECCABLE_CLI_REMOTE_E2E: '1' },
files: ['tests/skills-cli.test.js'],
},
],
},
'plugin-e2e': {
description: 'Install the committed ./plugin subtree into a real (sandboxed) Claude Code and assert skills, agents, and hooks all load. Skips when the claude CLI is not on PATH.',
triggers: [
...COMMON_INFRA_PATTERNS,
/^plugin\//,
/^skill\/agents\//,
/^scripts\/build\.js$/,
/^scripts\/lib\/validate-plugin-manifest\.js$/,
/^scripts\/lib\/plugin-paths\.js$/,
/^tests\/plugin-e2e\.test\.mjs$/,
],
commands: [
{
runner: 'node',
timeoutMs: 300000,
forceExit: true,
files: ['tests/plugin-e2e.test.mjs'],
},
],
},
'live-e2e': {
description: 'Full Playwright live-mode click-to-accept sweep across runtime framework fixtures.',
optIn: true,
needsPlaywright: true,
triggers: [
...COMMON_INFRA_PATTERNS,
/^skill\/scripts\/live/,
/^tests\/framework-fixtures/,
/^tests\/live-e2e(\.test\.mjs|\/)/,
],
commands: [
{
runner: 'node',
timeoutMs: 600000,
forceExit: true,
files: ['tests/live-e2e.test.mjs'],
},
],
},
'new-work-e2e': {
description: 'Playwright smoke sweep of the new-work concept/serve-question decision page plus the offline fake image generator.',
optIn: true,
needsPlaywright: true,
triggers: [
...COMMON_INFRA_PATTERNS,
/^skill\/scripts\/(serve-question|generate-image|concept-seed)\.mjs$/,
/^tests\/new-work-e2e(\.test\.mjs|\/)/,
],
commands: [
{
runner: 'node',
timeoutMs: 600000,
forceExit: true,
files: ['tests/new-work-e2e.test.mjs'],
},
],
},
'live-e2e-accept-cleanup': {
description: 'Provider-backed post-accept cleanup regression.',
optIn: true,
needsPlaywright: true,
triggers: [
...COMMON_INFRA_PATTERNS,
/^skill\/scripts\/(live-accept|live-browser|live-server|live-wrap)\.mjs$/,
/^skill\/scripts\/live\/sveltekit-adapter\.mjs$/,
/^tests\/live-e2e-accept-cleanup-regression\.test\.mjs$/,
/^tests\/live-e2e\//,
],
commands: [
{
runner: 'node',
timeoutMs: 600000,
files: ['tests/live-e2e-accept-cleanup-regression.test.mjs'],
},
],
},
'live-e2e-agent': {
description: 'Focused insert-mode fake-agent helper tests.',
commands: [
{
runner: 'node',
files: ['tests/live-e2e/agent-insert.test.mjs'],
},
],
},
'skill-behavior': {
description: 'LLM-backed skill setup behavior scenarios.',
optIn: true,
triggers: [
...COMMON_INFRA_PATTERNS,
/^skill\/SKILL\.src\.md$/,
/^skill\/reference\/(init|document|brand|product|shape|craft|audit|polish|live)\.md$/,
/^skill\/scripts\/(context|context-signals|detect|detect-csp)\.mjs$/,
/^tests\/skill-behavior\//,
],
commands: [
{
runner: 'node',
// 300000 was too low to measure what these scenarios assert. The
// workflow-contract turns run 20+ steps against a frontier model, and
// the *correct* path is the slow one: a run that stops to put the
// concept to the user before building was measured at 579s, while the
// runs that skipped that checkpoint and failed the assertion finished
// in 130-200s. At a 300s cap the thorough path is killed and the hasty
// path is graded, so the cap was selecting for the behavior the suite
// exists to forbid.
timeoutMs: 900000,
files: [
'tests/skill-behavior/scenarios.test.mjs',
'tests/skill-behavior/workflow-contract.test.mjs',
],
},
],
},
'live-svelte-adapter-deepseek': {
description: 'DeepSeek-backed Svelte adapter browser sweep.',
optIn: true,
needsPlaywright: true,
triggers: [
...COMMON_INFRA_PATTERNS,
/^skill\/scripts\/(live-server|live-wrap)\.mjs$/,
/^skill\/scripts\/live\/(sveltekit-adapter|svelte-component)\.mjs$/,
/^tests\/framework-fixtures\/vite8-sveltekit-stateful\//,
/^tests\/live-svelte-adapter-deepseek\.test\.mjs$/,
],
commands: [
{
runner: 'node',
timeoutMs: 1200000,
files: ['tests/live-svelte-adapter-deepseek.test.mjs'],
},
],
},
};
function escapeRegExp(value) {
return value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
}
// Every suite must select itself when one of its own test files changes.
// Generated from the files lists so the hand-written trigger patterns above
// only carry source paths and fixture directories; before this, four test
// files were registered in a suite that change-based CI could never select
// by editing them (serve-question, ci-test-plan, both validate-plugin-*),
// and tests/lib/detector-bundle.test.js triggered core while running in
// detector. The meta-test in tests/test-suites.test.mjs pins this invariant.
for (const suite of Object.values(SUITES)) {
const ownFiles = suite.commands.flatMap((command) => command.files);
suite.triggers = [
...(suite.triggers ?? []),
...ownFiles.map((file) => new RegExp(`^${escapeRegExp(file)}$`)),
];
}
export function expandSuites(requested) {
const names = requested.length === 0 ? ['default'] : requested;
const expanded = [];
for (const name of names) {
if (name === 'default' || name === 'all-local') {
expanded.push(...DEFAULT_SUITES);
} else if (name === 'all') {
expanded.push(...DEFAULT_SUITES, ...OPT_IN_SUITES);
} else if (SUITES[name]) {
expanded.push(name);
} else {
throw new Error(`Unknown test suite "${name}". Run: node scripts/run-tests.mjs --list`);
}
}
return [...new Set(expanded)];
}
export function suiteFiles(suiteNames) {
const files = [];
for (const name of suiteNames) {
const suite = SUITES[name];
if (!suite) throw new Error(`Unknown test suite "${name}"`);
for (const command of suite.commands) {
files.push(...command.files);
}
}
return files;
}
export function findTestFiles(root = process.cwd()) {
const out = [];
const stack = [path.join(root, 'tests')];
while (stack.length) {
const dir = stack.pop();
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
const abs = path.join(dir, entry.name);
if (entry.isDirectory()) {
stack.push(abs);
} else if (/\.test\.(js|mjs)$/.test(entry.name)) {
out.push(path.relative(root, abs).split(path.sep).join('/'));
}
}
}
return out.sort();
}
export function matchesSuiteTriggers(suiteName, changedFiles) {
const suite = SUITES[suiteName];
if (!suite) throw new Error(`Unknown test suite "${suiteName}"`);
return changedFiles.some((file) => suite.triggers?.some((pattern) => pattern.test(file)));
}
Reference in New Issue View Git Blame Copy Permalink