mirror of https://github.com/magnus919/agent-skills.git synced 2026-09-11 19:47:12 +03:00

Files

T

ac1beb117d feat(skill): add Playwright skill (E2E testing + scraping + headless browsing) (#264 )

Add ONE tool skill for Playwright: SKILL.md covering E2E test authoring,
selector robustness, network interception/mocking, parallel workers, CI
integration, scraping/headless patterns, accessibility snapshot checks, and
headed debugging; scripts/pwrun (agent-first smoke harness with --json,
fixture-tested); templates/ test-suite scaffold; eight dated references; a
schema-valid evals/evals.json (6 cases); a human-facing README; reverse
routing from qa-methodology and frontend-engineering; top-level README index
entry; and regenerated catalogs (llms.txt, marketplace, codex).

Closes #244.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>

2026-08-03 17:59:38 -04:00

3.3 KiB

Raw Blame History

CI Integration

Last Updated: 2026-08-03

Playwright in CI is: install browsers + OS deps, pin the version, run the suite with retries and tracing, and surface a debuggable report. This reference assumes GitHub Actions; the same shape applies to any runner.

Minimal GitHub Actions workflow

name: e2e
on: [push, pull_request]
jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20, cache: npm }
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npx playwright test
      - if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 14

Non-negotiables

Install browsers with OS deps: npx playwright install --with-deps (not bare install) on Linux runners; --with-deps installs the system libraries Chromium/Firefox/WebKit need.
Pin and cache:
- npm ci with a committed package-lock.json (never npm install).
- Cache the browser download: ~/.cache/ms-playwright (Linux), ~/Library/Caches/ms-playwright (macOS), %USERPROFILE%\AppData\Local\ms-playwright (Windows).
- Cache node_modules via the setup-node cache: npm option.

Retry flaky tests on CI only, with traces on retry so failures are debuggable:

retries: process.env.CI ? 2 : 0,
use: { trace: 'on-first-retry' },

Configure webServer in the config so the runner starts and waits for the app; never assume a long-lived dev server on a runner.
Upload artifacts on failure: the HTML report, the JSON report, and the test-results/ dir (traces). Retention bounded (7–14 days) — see hard boundaries in SKILL.md about keeping evidence bounded.

Reporters

list/line — human-readable run output.
html — the browsable report (upload on failure).
json — the machine-readable report for agent triage:
```
scripts/pwrun report --report test-results/test-results.json --json
```
It prints stats (expected/unexpected/flaky/skipped), the failing specs, and the error message from the last retry — enough to triage without opening a browser.
github — inline annotations on GitHub Actions, keyed to the failing spec line.

Sharding across jobs

For large suites, split the run:

strategy:
  matrix:
    shard: [1/4, 2/4, 3/4, 4/4]
steps:
  - run: npx playwright test --shard=${{ matrix.shard }}

Merge reports from all shards with playwright merge-reports (see 04-parallel-workers-and-sharding.md for the math).

Triage loop for a red CI run

scripts/pwrun report --report <json> --json — get the failing specs and messages.
Download the trace artifact (trace.zip) and open it in the Trace Viewer to see the failing action, network, and console.
Classify: environment (missing dep/browser), selector (see 02-selectors.md), timing (webServer readiness, webServer.timeout), or app regression (real bug — the test did its job).
Fix, re-run, and confirm the shard matrix is green.

Parallelism and sharding configuration: 04-parallel-workers-and-sharding.md.
Trace reading and headed debugging: 07-accessibility-and-debugging.md.

3.3 KiB Raw Blame History Unescape Escape