skills/pbakaus_impeccable
1
0
Fork 0
mirror of https://github.com/pbakaus/impeccable.git synced 2026-09-21 10:36:27 +03:00
Code Issues Packages Projects Releases Wiki Activity

docs: add workflow guide for command entry points (#737)

Browse Source 
* docs: add workflow guide for command entry points

* Refine workflow guidance into advice-only routing

Reuse the existing routing reference and docs map instead of shipping a parallel workflow catalog. Add reference-backed command comparisons, advice-only tests, and explicit-command precedence coverage.

AI-assisted maintainer revision prepared with Codex.

* Include routing guidance in behavior-test triggers

AI-assisted maintainer revision prepared with Codex.

* Constrain routing behavior tests to fixture-safe tools

Keep the real context loader but reject arbitrary host shell searches in the new advice scenarios. Preserve observable project writes and protect the staged skill; cover the restriction with offline regression tests.

AI-assisted maintainer revision prepared with Codex.

* Require actual reference reads in restricted routing tests

Do not count rejected shell reads as reference loading. Record the nine measured advice cases; explicit-command measurements remain pending the stricter retest.

AI-assisted maintainer revision prepared with Codex.

* Record measured workflow-routing baseline

All twelve focused cases pass across Claude Sonnet 5, GPT-5.6 Terra, and Gemini 3.7 Flash, including the stricter explicit-command retest.

AI-assisted verification prepared with Codex.

* Trim workflow routing guidance

Reduce added skill prose from 286 to 59 words while retaining the routing regression assertions. Record the missing-context reference-read flake and passing repeat.

AI assistance: prepared and verified with Codex under maintainer direction.

---------

Co-authored-by: Paul Bakaus <paul.bakaus@gmail.com>
This commit is contained in:
babylon1x
2026-09-05 15:01:58 -07:00
committed by GitHub
co-authored by Paul Bakaus
parent 0ac0b6866f
commit 044a04fd0d
8 changed files with 179 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/skill-behavior/README.md
+32
View File
@@ -72,6 +72,38 @@ The trace is the source of truth, not the model's free-form reply.
| 13 | empty workspace; prompt is `/impeccable teach` | runs `impeccable context` and diverts into `reference/init.md` because `teach` aliases `init` |
| 14 | PRODUCT.md with `## Platform: ios` (native iOS app); prompt is `/impeccable craft a tide detail screen` | `impeccable context` runs and emits the contents of `reference/ios.md` directly, placing native conventions in context without a second model-directed read |
| 15 | same iOS fixture; prompt is `/impeccable audit` | agent loads `reference/audit.native.md` (the Commands-table native variant, routed instead of `audit.md`) |
| 16 | existing surface, with and without PRODUCT.md; asks where to start | loads `routing.md`, delivers advice, and does not edit project files, start an interview, archive a critique, or run menu scans |
| 17 | existing surface; asks whether critique is required before polish | loads `routing.md` and both command references, then delivers advice without executing the playbooks |
| 18 | existing surface; explicitly requests polish followed by a next-command recommendation | loads `polish.md` rather than substituting workflow advice for the requested work |
## Workflow-advice baseline (2026-09-05, PR #737)
The four cases in scenarios 16-18 are new; prior scenario results do not
establish their behavior. The advice-only assertions inspect write-tool calls
and file mutations from bash (excluding context's internal `.impeccable/`
state, but not critique reports), require an actual answer, and reject
interviews and menu scans. Scenario 18 checks command-reference precedence;
it does not assert completion of a full polish pass. These cases allow only
the exact context-loader command through bash; references use read/list, and
the write tool remains available for project files so unsolicited edits still
fail the test. Writes to the staged skill are rejected. This keeps the routing
measurement from running arbitrary shell searches outside its fixture.
| Scenario | claude-sonnet-5 | gpt-5.6-terra | gemini-3.7-flash |
|---|---|---|---|
| 16 (existing project) | pass | pass | pass |
| 16 (missing product context) | flaky (1 of 2) | pass | pass |
| 17 (command comparison) | pass | pass | pass |
| 18 (explicit command) | pass | pass | pass |
Rechecked after reducing the skill addition to 59 words. Claude's first
missing-context response stayed read-only but skipped `routing.md`; an unchanged
repeat passed on all three providers. Keep that reference-read miss visible as
a flake rather than adding instructions for a single observation. All assertions
remain unchanged. Scenario 18 uses an eight-step budget and actual read-tool
evidence, not rejected bash reads. The initial unrestricted run (stopped after
host-wide search attempts), earlier rejected-read results, and broader suite's
sandboxed provider DNS errors are excluded from this baseline.
The workflow-contract file adds end-to-end assertions for attended fresh init,
an initialized natural build request, replacement-world redesign, scope-preserving bolder