mirror of
https://github.com/pbakaus/impeccable.git
synced 2026-09-12 06:06:37 +03:00
Paul BakausandClaude Fable 5
6ece0e588f
A cheap, LLM-free E2E tier for the interactive parts of new-work, mirroring the two-layer live-e2e pattern (deterministic now, opt-in LLM tier later). - generate-image.mjs: IMPECCABLE_IMAGE_GEN_FAKE=1 writes a deterministic offline image (SVG with wrapped prompt + SYNTHETIC COMP label, or a valid palette-stripe PNG carrying the prompt/marker in a tEXt chunk). Same CLI contract, no key, no network, $0.00 cost line. - tests/new-work-e2e/user-bot.mjs: scripted user bot (module + CLI) that resolves the serve-question daemon from the workspace and drives the real page via Playwright (pick, re-roll + steer, canon, tab close). - tests/new-work-e2e.test.mjs: node --test coverage of the serve-question cycles (pick + CHOSEN CARD, re-roll + --update re-deal, canon + CANON CHOSEN, tab-close exit-4, text-only card) plus fake image determinism. - Registered as the opt-in new-work-e2e suite; added test:new-work-e2e. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
66 lines
3.0 KiB
Markdown
66 lines
3.0 KiB
Markdown
# new-work E2E
|
|
|
|
A cheap, deterministic smoke suite for the interactive parts of new-work: the
|
|
serve-question decision page and the offline image generator. It is kept out of
|
|
`bun run test` and runs on demand.
|
|
|
|
```bash
|
|
bun run test:new-work-e2e
|
|
```
|
|
|
|
One-time setup: `npx playwright install chromium` (the suite drives a real
|
|
Chromium so the page runs its own JS, exactly as a user's tab would).
|
|
|
|
## What it covers
|
|
|
|
`tests/new-work-e2e.test.mjs` opens the served decision page with a real
|
|
browser and drives it through the scripted user bot, then asserts on the
|
|
serve-question protocol output:
|
|
|
|
- **pick assigned** returns the chosen `optionId`, the typed steer, the
|
|
`hero`/`board` fields, and the `CHOSEN CARD` directive printed by `--wait`.
|
|
- **re-roll with steer** keeps the daemon alive, `--update` re-deals the next
|
|
hand, the page reloads itself, and the following pick is terminal (state file
|
|
cleaned up).
|
|
- **canon** returns `optionId: canon` and prints the `CANON CHOSEN` directive.
|
|
- **tab close** stops the page heartbeats so `--wait` exits 4 `PAGE CLOSED`.
|
|
- **text-only card** renders with no `.media` region when an option has no hero.
|
|
- **fake image generation**: same prompt yields identical bytes, the file
|
|
exists, the `SYNTHETIC` marker is present, and different prompts produce
|
|
different palettes.
|
|
|
|
The concept-seed direction roll (challengers, `ASSIGNED INDEX`, the no
|
|
PRODUCT.md gate) is already covered by `tests/concept-seed.test.mjs` and is not
|
|
repeated here.
|
|
|
|
## Pieces
|
|
|
|
- `user-bot.mjs` is a module plus CLI. Given a workspace dir it resolves the
|
|
running daemon from `.impeccable/questions/<key>.state.json`, opens the page,
|
|
and runs a JSON policy of real clicks: `{"pick":"assigned"}`,
|
|
`{"reroll":true,"steer":"warmer"}`, `{"pick":"challenger-*"}`,
|
|
`{"canon":true}`, `{"close":true}`. The deterministic tier passes an
|
|
already-launched browser in; the CLI launches its own Chromium.
|
|
- `IMPECCABLE_IMAGE_GEN_FAKE=1` switches `skill/scripts/generate-image.mjs` to
|
|
the offline stand-in: no OpenAI call, no key, a `$0.00` cost line, and a
|
|
deterministic image (SVG for `.svg` out with the wrapped prompt text and a
|
|
`SYNTHETIC COMP` label; a valid palette-stripe PNG otherwise, with the prompt
|
|
and marker in a PNG `tEXt` chunk).
|
|
|
|
## Planned LLM tier (not built yet)
|
|
|
|
The same scaffolding supports an opt-in LLM tier later, mirroring the two-layer
|
|
pattern in `tests/live-e2e`:
|
|
|
|
- A real model plays the user through the same scripted `user-bot.mjs` policy,
|
|
choosing and steering instead of following canned actions.
|
|
- `IMPECCABLE_IMAGE_GEN_FAKE` still stands in for image spend, so a full
|
|
concept-to-card cycle runs without paying per render.
|
|
- Assertions run against the tool-call trace via the skill-behavior harness,
|
|
the same way `tests/skill-behavior` keys on the trace rather than free-form
|
|
output.
|
|
|
|
Cost posture: the deterministic tier is free (no API calls, local Chromium).
|
|
The LLM tier hits a provider and costs money, so it stays opt-in and out of CI,
|
|
matching how `test:live-e2e` and `test:skill-behavior` are gated today.
|