mirror of
https://github.com/pbakaus/impeccable.git
synced 2026-09-17 00:26:41 +03:00
Export portable Live evidence bundles
AI-assisted implementation under maintainer direction.
This commit is contained in:
@@ -120,3 +120,38 @@ When `preActions` is omitted, steer smoke inherits `runtime.preActions` to revea
|
||||
| `nuxt-csp/` | Nuxt `routeRules` with literal CSP header in `nuxt.config.ts`. CSP shape `append-string`. |
|
||||
|
||||
Add new fixtures by cloning a directory, swapping files, and updating `fixture.json`.
|
||||
|
||||
## External quality-eval fixtures
|
||||
|
||||
The public Live benchmark can execute a fixture owned by another repository
|
||||
without copying its task corpus or rubric into Impeccable:
|
||||
|
||||
```sh
|
||||
bun run bench:live -- \
|
||||
--fixture-dir=/absolute/path/to/private-fixture \
|
||||
--agent=codex \
|
||||
--action=bolder \
|
||||
--iterations=1 \
|
||||
--evidence-bundle=/absolute/path/to/output-bundle
|
||||
```
|
||||
|
||||
An external fixture has the same shape as a directory in this folder:
|
||||
`fixture.json`, `gitignore.txt`, and `files/`. Use the optional
|
||||
`evidenceCapture` block in `fixture.json` for rubric-free capture metadata:
|
||||
|
||||
```json
|
||||
{
|
||||
"evidenceCapture": {
|
||||
"captureSelector": "section.case-study",
|
||||
"viewport": { "width": 1440, "height": 1080 },
|
||||
"action": "bolder"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The bundle contains `report.json`, the original capture, each progressively
|
||||
delivered variant capture, geometry/overflow facts, hashes, and timing data.
|
||||
It deliberately cannot run `--judge-rendered`; comparative rubrics, private
|
||||
fixtures, human calibration, and quality decisions belong in the consuming
|
||||
evaluation harness. The normal public E2E suite remains responsible for Live
|
||||
protocol, framework, source-commit, cleanup, and recovery correctness.
|
||||
|
||||
Reference in New Issue
Block a user