From 1cd5e19620feaecaf04b5bc524fd9cd415f9cdb2 Mon Sep 17 00:00:00 2001 From: Magnus Hedemark Date: Sun, 23 Aug 2026 00:48:12 -0400 Subject: [PATCH] docs(neckbeard): require requester-outcome traceability at gate 3 Implements issue #378 (Finding F4 of the journey SDLC validation brief): separate validation from verification in the change-request journey. Gate 3 now requires that at least one SPEC.md acceptance criterion traces to the requester's stated outcome/problem restated from primary evidence in phase 2 - verification proves the build matches the spec; this criterion proves it solves the requester's actual problem (15288/12207 validation vs. verification discipline). - references/stages.md: authoritative new Gate 3 bullet (single source of truth for gate definitions) - references/journey.md: phase 4 Output and Gate rows aligned, deferring to stages.md; phase-continuity and gate summary unchanged - templates/change-contract.md: acceptance-criteria guidance requires a criterion tracing to the Problem section - templates/delivery-packet.md: acceptance-criteria mapping field asks for the requester-outcome trace for the gate-3 verdict - evals/evals.json: gate-3 assertions updated in four full-path cases; all 13 case IDs stable Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- neckbeard/evals/evals.json | 2 +- neckbeard/references/journey.md | 4 ++-- neckbeard/references/stages.md | 5 +++++ neckbeard/templates/change-contract.md | 5 ++++- neckbeard/templates/delivery-packet.md | 2 +- 5 files changed, 13 insertions(+), 5 deletions(-) diff --git a/neckbeard/evals/evals.json b/neckbeard/evals/evals.json index 927f69e..eb7ff05 100644 --- a/neckbeard/evals/evals.json +++ b/neckbeard/evals/evals.json @@ -1 +1 @@ -{"schema_version": 1, "skill_name": "neckbeard", "evals": [{"id": "bug-fix-reproduction-regression", "prompt": "Issue #342 in our inventory service: the CSV export endpoint returns an off-by-one row count when the dataset contains multi-byte UTF-8 characters. Users see 99 rows exported when there are actually 100. Fix this bug and make sure it stays fixed.", "expected_output": "Scenario: straightforward bug fix with reproduction and regression test. The trajectory follows the full journey path through all nine phases (intake and provenance, current-state discovery and reproduction, architecture/design delta, specification, pre-implementation test planning, domain implementation, independent review and boundary verification, readiness and CI/review loops, authorized post-merge release). Phase 2 records reproduction evidence (repro steps, observed vs. expected behavior, environment/version) in delivery packet group (d). A documented 'no architecture delta' determination satisfies gate 1. A regression test is added covering the multi-byte UTF-8 boundary. The verification plan names the regression test as passing evidence. All five gates are recorded in packet group (h) with verdicts bound to the exact head SHA. Phase 9 closeout records the terminal lifecycle state 'merged' with the merge commit SHA plus rollback/follow-up triggers (what would cause re-entry after closeout) plus lesson capture per Stage 6 — the reusable lesson with its durable destination or an explicit 'no reusable lesson identified' determination; operational discoveries route to a new change request via phase 1 intake rather than re-opening the terminal packet. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["Delivery packet group (d) contains reproduction evidence with repro steps, observed behavior, expected behavior, and environment/version for the bug report", "Gate 1 verdict is recorded as pass with a documented 'no architecture delta' determination in packet group (h)", "A regression test targeting the multi-byte UTF-8 row-count boundary exists in the produced artifacts", "The verification plan in packet group (g) names the regression test as a required passing evidence artifact", "All five gate verdicts (gate 1, gate 2, gate 3, gate 4, gate 5) are recorded in packet group (h) each bound to an exact head SHA", "The final verified head SHA in packet group (i) equals the actual head SHA of the delivered change", "Terminal lifecycle state in packet group (i) is 'merged' with a merge commit SHA recorded and rollback/follow-up triggers recorded at closeout (or an explicit none-identified entry), without re-opening or resurrecting the terminal packet", "Phase 9 closeout records lesson capture in packet group (i) per Stage 6 'Deliver and learn': the reusable lesson (e.g., the multi-byte input boundary) with its durable destination (skill, memory, or project docs), or an explicit 'no reusable lesson identified' determination — the field is never silently omitted"]}, {"id": "ambiguous-feature-product-discovery", "prompt": "Our product manager filed issue #518: 'Add collaborative editing to the document editor.' The request has no acceptance criteria, no scope boundary, and three different stakeholders have posted conflicting comments about what this means. Figure out what to build and deliver it.", "expected_output": "Scenario: ambiguous feature requiring product discovery and scope gate. The trajectory exercises the full journey path. Phase 1 captures provenance and flags the ambiguity. Phase 2 gathers current-state evidence from the codebase and issue comments. The product-discovery specialist is loaded (applicability signal: ambiguous requirements with conflicting stakeholder input) and its engagement is recorded in packet group (e). Phase 4 produces a SPEC.md with acceptance criteria derived from the discovery output; the scope gate (gate 3) verifies that no unbounded or ownerless work items remain. If requirements are irreconcilably ambiguous after product-discovery engagement, the trajectory escalates per phase 4 escalation conditions rather than guessing. The terminal state is 'merged' if scope is resolved, or 'blocked' with escalation evidence if irreconcilable ambiguity halts work. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The routing decision in packet group (e) records product-discovery as a selected specialist with an applicability signal citing ambiguous requirements or conflicting stakeholder input", "Packet group (g) contains a SPEC.md with acceptance criteria derived from product-discovery output rather than guessed from the raw issue text", "Gate 3 verdict in packet group (h) confirms spec and task-plan completeness with no unbounded or ownerless work items", "If the trajectory terminates as 'blocked', packet group (i) records the escalation condition (irreconcilable ambiguity) and the phase at which it occurred", "If the trajectory terminates as 'merged', all five gate verdicts are recorded in packet group (h) bound to the exact final head SHA", "The terminal lifecycle state in packet group (i) is either 'merged' with merge evidence or 'blocked' with escalation evidence; no silent abandonment occurs"]}, {"id": "multi-surface-backend-frontend-api-data", "prompt": "Issue #601: we need to add a user-preferences endpoint (REST API), a settings page (React frontend), a PostgreSQL migration for the new preferences table, and the backend service logic to persist and retrieve preferences. All four surfaces must change together for this feature to work.", "expected_output": "Scenario: multi-surface change involving backend, frontend, API, and data routing. The trajectory exercises the full journey path with multiple specialists composed per stage. Phase 6 routing loads backend-engineering (server-side service logic), frontend-engineering (React settings page), api-design-and-evolution (REST endpoint contract), and data-engineering (PostgreSQL migration) with one lead per stage recorded in packet group (e). The multi-row composition rule applies: no two rows own the same stage simultaneously. Gate 1 requires an architecture delta covering the new table and endpoint contract. Gate 2 requires a verification plan exercising the integration boundary across all four surfaces. Gate 5 boundary verification targets integration or end-to-end level. The terminal state is 'merged' with all gates passed. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["Packet group (e) routing section records backend-engineering, frontend-engineering, api-design-and-evolution, and data-engineering as selected specialists with per-stage lead assignments", "No two specialist rows are recorded as owning the same implementation stage simultaneously in the routing decision", "Gate 1 verdict in packet group (h) references an architecture delta covering the new PostgreSQL table and the REST endpoint contract", "The verification plan in packet group (g) names an integration or end-to-end verification target exercising the boundary across backend, frontend, API, and data surfaces", "Gate 5 verdict in packet group (h) is bound to the exact head SHA and records the verification boundary actually exercised", "Terminal lifecycle state in packet group (i) is 'merged' with all five gate verdicts recorded as pass"]}, {"id": "schema-migration-rollback-release-readiness", "prompt": "Issue #710: we need to split the 'users' table's 'address' column into separate street, city, state, and zip columns. The table has 4.2 million rows in production. This is a high-risk schema migration that needs a rollback plan and release-readiness evidence before we can merge.", "expected_output": "Scenario: schema/migration change requiring rollback and release-readiness evidence. The trajectory exercises the high-risk journey path (all nine phases mandatory, all five gates, no conditional skips). Phase 3 produces an architecture delta with expand/contract (parallel change) migration strategy and a rollback plan that does not rely on git revert. data-engineering and data-architect are loaded per routing signals (migration files present, schema change). Gate 1 requires approval of the migration strategy and rollback plan. Phase 9 records release-readiness evidence (version bump, changelog, migration documentation, rollback rehearsal) before the release gate is evaluated. The terminal state is 'merged' with release readiness confirmed but release activity gated on separate explicit authorization. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The high-risk path is recorded in packet group (b) with all nine phases mandatory and no conditional skips permitted", "Packet group (f) contains a migration strategy using expand/contract (parallel change) and a rollback plan that rejects git revert for schema or data changes", "data-engineering and data-architect are recorded as selected specialists in packet group (e) with applicability signals citing migration files or schema change", "Gate 1 verdict in packet group (h) records approval of the architecture delta including the migration strategy and rollback plan", "Release-readiness evidence (version bump, changelog entry, migration documentation) is recorded in packet group (i) before the release gate", "The release gate in packet group (h) or (i) distinguishes pre-merge release readiness from post-merge release activity requiring separate explicit authorization", "Terminal lifecycle state in packet group (i) is 'merged' with the merge commit SHA recorded; release activity is not conflated with merge, and phase 9 closeout additionally records the rollback/follow-up triggers for the migration (e.g., post-release verification failure requiring rollback) in the terminal record"]}, {"id": "refactor-characterization-architecture-review", "prompt": "Issue #823: the order-processing module has grown into a 2,400-line god class. Refactor it into smaller, focused modules without changing any observable behavior. We need proof the behavior is preserved.", "expected_output": "Scenario: refactor needing characterization tests and architecture review. The trajectory exercises the refactor journey path (phases 1, 2, 3, 6, 7, 8, 9 mandatory; phases 4 and 5 conditional). Phase 2 gathers current-state evidence of the god class behavior. Phase 3 produces an architecture delta with the decomposition plan; software-architecture-analysis is loaded for the review. Characterization tests are written or identified before the refactor begins to capture existing behavior. Gate 5 uses behavior-preservation evidence (characterization tests passing before and after) as its criterion. The skip of phase 4 is recorded with the reason that the refactor is fully characterized by the architecture delta and characterization-test plan. Phase 9 closeout records lesson capture per Stage 6 — e.g., the decomposition heuristic as a reusable lesson with its durable destination, or an explicit 'no reusable lesson identified' determination. The terminal state is 'merged' with behavior-preservation evidence recorded. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The refactor path is recorded in packet group (b) with phases 4 and 5 conditional", "Packet group (e) records the skip of phase 4 (specification and decomposition) with a reason citing that no separate SPEC.md is needed because the refactor is behavior-preserving", "software-architecture-analysis is recorded as a selected specialist in packet group (e) for the architecture review", "Characterization tests covering the god class observable behavior exist in the produced artifacts before the refactor commits begin", "Gate 5 verdict in packet group (h) records behavior-preservation evidence (characterization tests passing before and after) bound to the exact head SHA", "Terminal lifecycle state in packet group (i) is 'merged' with the merge commit SHA recorded", "Phase 9 closeout records lesson capture in packet group (i): the reusable lesson with its durable destination (skill, memory, or project docs), or an explicit 'no reusable lesson identified' determination — silent omission is prohibited"]}, {"id": "docs-only-reduced-path-skip-reasons", "prompt": "Issue #901: the API reference page in our docs site has outdated parameter names for the /v2/orders endpoint. Update the documentation to match the current implementation. No code changes are needed.", "expected_output": "Scenario: docs-only change using a reduced (lightweight) path with comprehensive skip recording. The trajectory exercises the lightweight journey path (phases 1, 6, 7, 8, 9 mandatory; phases 2, 3, 4, 5 conditional and skipped). Each skipped phase is recorded in packet group (e) with a concrete reason citing the path's skip criterion. At least two specialist skills that do not apply (backend-engineering, frontend-engineering, data-engineering) are explicitly listed as skipped with reasons citing their routing-table skip rules (e.g., 'no server-side application code modified', 'no client-side application code modified', 'no schema or migration files modified'). No specialist skill in the routing matrix is absent from both the selected and skipped lists. The terminal state is 'merged' with doc-update evidence. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The lightweight path is recorded in packet group (b) and phases 2, 3, 4, and 5 are each recorded as skipped in packet group (e) with a concrete reason citing the path's skip criterion", "Packet group (e) routing section lists at least three specialist skills (including backend-engineering, frontend-engineering, and data-engineering) as skipped, each with a skill name and a concrete reason", "Every skipped skill entry in packet group (e) has both a skill name and a non-empty reason; no specialist in the routing matrix is absent from both the selected and skipped lists", "technical-documentation is recorded as a selected specialist in packet group (e) with an applicability signal citing documentation changes", "No code-editing commits appear in the produced artifacts; only documentation files are changed", "Terminal lifecycle state in packet group (i) is 'merged' with the merge commit SHA recorded"]}, {"id": "existing-pr-duplicate-detection", "prompt": "Issue #455 was filed asking for rate limiting on the public API. I want to start working on it and submit a PR.", "expected_output": "Scenario: issue with an existing PR or duplicate work. The trajectory exercises the lifecycle pre-work checks in phase 1 (intake and provenance). The existing-PR search discovers that PR #450 from another contributor already implements rate limiting for the same endpoint. The duplicate-issue check and existing-PR check are recorded as distinct pre-work checks in packet group (d) or the lifecycle fields. The handling decision is to coordinate or defer to the existing PR owner rather than opening a duplicate PR. The terminal lifecycle state is 'closed' with the close reason recording the duplicate-work determination and the existing PR reference. No new PR is opened. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["Packet group (d) or lifecycle fields record a duplicate-issue check result and a separate existing-PR search result as distinct pre-work checks", "The existing-PR search result identifies the pre-existing PR (by number or reference) addressing the same rate-limiting work", "The handling decision recorded in the packet is to coordinate or defer, not to open a duplicate PR", "No new review submission (PR or equivalent) is created in the produced artifacts", "Terminal lifecycle state in packet group (i) is 'closed' with a close reason citing the duplicate-work determination and referencing the existing PR"]}, {"id": "review-round-material-change-reverification", "prompt": "Issue #567: add input validation to the user-registration form. During code review, a reviewer points out that the validation logic has a bypass when the input contains null bytes. You push a fix addressing the null-byte bypass. This changes the validation logic materially.", "expected_output": "Scenario: review round that changes the final head and requires re-verification, proving a material post-review change invalidates the prior verification verdict. The trajectory exercises phases 1 through 8. Phase 7 (independent review and boundary verification) produces gate 4 and gate 5 verdicts bound to head SHA A (the initial implementation). A material review-driven change (the null-byte bypass fix altering validation logic) produces head SHA B. The prior verdicts bound to SHA A are explicitly invalidated. The trajectory re-enters phase 7: gate 4 (independent review) and gate 5 (boundary verification) re-run at SHA B. The final verified head SHA in packet group (i) is SHA B, not SHA A. The readiness gate (phase 8) is satisfied only after re-verification at the new head. The terminal state is 'merged' with the final verdict bound to SHA B. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["Gate 4 and gate 5 verdicts are initially recorded in packet group (h) bound to head SHA A (the pre-review implementation head)", "The material review-driven change (null-byte bypass fix altering validation logic) is recorded as producing a new head SHA B distinct from SHA A", "The prior verdicts bound to SHA A are explicitly marked as invalidated in packet group (h) after the material change", "Gate 4 and gate 5 verdicts are re-recorded in packet group (h) bound to head SHA B after re-verification", "The final verified head SHA in packet group (i) equals SHA B, not the stale SHA A", "Terminal lifecycle state in packet group (i) is 'merged' with the merge commit SHA recorded and the final verdict bound to SHA B"]}, {"id": "release-authority-blocked-terminal-state", "prompt": "Issue #688: implement and ship the new caching layer for the product-search service. You have authority to create a PR and get it merged, but the release process requires the VP of Engineering's sign-off which has not been granted. Complete the work and get it deployed.", "expected_output": "Scenario: maintainer/release-authority-blocked terminal state. The trajectory exercises phases 1 through 8 successfully (all gates pass, CI green, review approved, exact-final-head verified). Phase 9 (authorized post-merge release and closeout) cannot proceed because release authority has not been granted. The release gate requires explicit authorization distinct from merge authority; merge alone does not imply release authorization. The trajectory records the blocked release gate with the escalation outcome (release authority not granted) and transitions the packet to the 'blocked' terminal state. The run stops at the release gate without deploying; persistence does not upgrade authority. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["All five gate verdicts (gate 1 through gate 5) are recorded as pass in packet group (h) bound to the exact final head SHA", "Phase 8 readiness is confirmed with CI passing and review approved at the exact final head SHA recorded in packet group (i)", "The release gate in packet group (h) or (i) records a verdict of 'blocked' citing that release authority was not granted", "The packet distinguishes merge authority (granted) from release authority (not granted) in the authority classification fields of packet group (b)", "Terminal lifecycle state in packet group (i) is 'blocked' with the failing gate ID (release gate) and the blocking evidence (release authority not granted) recorded", "No release activity (tagging, publishing, deploying) appears in the produced artifacts; the run stops at the release gate without deploying", "Closeout records rollback/follow-up triggers in packet group (i) — e.g., that a future release-authorization grant or an operational discovery would cause re-entry via a new change request — without re-opening the terminal packet"]}, {"id": "lightweight-test-hardening-route", "prompt": "In a public OSS repository, issue #123 says the webhook URL validator already rejects overlong hostnames permanently, but a mutation pilot found that this classification could regress to retryable. Add a focused test-only regression guard, open a PR, and carry it through review.", "expected_output": "Scenario: a low-risk test-hardening change. The trajectory selects the lightweight path rather than the full path because production behavior is already correct and only a test file changes. Phase 1 records the protected public invariant, the named mutation, and production-code non-goals. Conditional planning phases are skipped with reasons. The implementation adds a hermetic public-contract test. Gate 5 uses clean-baseline pass plus targeted-mutant failure, not a requirement that the new test fail on clean main. The candidate is locally final before one push to the expensive-CI repository. Gate 4 is one bounded final review after the candidate is frozen, and readiness is bound to the exact final head SHA. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The lightweight path is selected because production behavior is already correct, production code is unchanged, and the change is confined to a test or fixture file", "The change contract records the public invariant, the named retryable-classification mutation, and an explicit production-code non-goal", "Conditional phases 2, 3, 4, and 5 are each recorded as skipped with reasons that identify test hardening and the applicable lightweight criterion", "The produced test uses hermetic setup and asserts the semantic permanent-versus-retryable contract rather than an incidental current exception subtype", "Gate 5 records clean-baseline pass and targeted-mutant failure, without requiring the test to fail on clean main", "The candidate's focused tests, lint, compilation, and scope checks are complete before the first push, and the remote verification is tied to one stable head", "Gate 4 records one bounded final review after candidate freeze, with no repeated reviewer round launched merely because a prior review timed out", "The final readiness evidence binds CI, review, and boundary verification to the exact final head SHA"]}, {"id": "test-hardening-baseline-mutant-gate", "prompt": "A mutation pilot found that changing an already-correct validation error from permanent to retryable would not be caught by the current tests. Add the smallest regression guard without changing production code.", "expected_output": "Scenario: test-hardening rather than a production bugfix. The agent states the semantic invariant, confirms the clean implementation already passes, and identifies the controlled weakening. It adds a deterministic hermetic public-behavior test, records clean-baseline pass and mutant-fail evidence, and does not apply the ordinary fails-on-main requirement. It avoids a broad mutation campaign and escalates only if production code, dependencies, configuration, or another higher-risk surface becomes necessary. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The output explicitly classifies the task as a regression guard for already-correct production behavior, not as an active production defect", "The proposed gate is clean-baseline pass plus named-mutant or controlled-weakening failure", "The test is hermetic and exercises the public contract without live external resolution or another incidental prerequisite", "The assertion checks the semantic permanent-versus-retryable invariant rather than exact current implementation identity", "Production code remains unchanged and the verification stays bounded to focused tests, static checks, scope checks, and the targeted mutation", "The output states escalation triggers for production-code, dependency, configuration, security, or deployment changes"]}, {"id": "midflight-position-assessment", "prompt": "Pick up where the previous contractor left off on the payment-retry feature. Branch feat/payment-retry exists with about six commits, there is a SPEC.md sitting on that branch, and the handoff note says 'design approved, most of the tests written'. No delivery packet or evidence ledger was handed over. Figure out where the work actually stands and continue it to a reviewed change.", "expected_output": "Scenario: mid-flight engagement with no delivery packet. The trajectory recognizes the entry-mode condition (continuation of work started elsewhere, no packet) and runs position assessment before any implementation continuation. It inventories observable artifacts: branch commits via git log against the base ref, SPEC.md presence and completeness on the branch, whether TASK-PLAN.md and VERIFICATION-PLAN.md exist, CI state, and any review artifacts. Each artifact is scored against the corresponding phase exit conditions. The handoff narrative ('design approved, most tests written') is treated as unverified assumption, not as satisfied gates, unless artifacts support it. The position report names the current phase, satisfied gates with evidence pointers, partial/absent/contradicted items, and a remaining checklist before execution resumes. Verified-complete work is not re-executed. A delivery packet is bootstrapped: group (a) records mid-flight engagement, inventory pointers, and the identified tracking system with its evidence basis; group (b) re-selects the delivery path from the affected surface; group (c) records the derived current phase and current gate (first gate in the remaining checklist), the last satisfied gate's own evidence SHA (or 'none' when no gate is satisfied — never the assessment-time HEAD), and a lifecycle state mapped from the assessed position. Gates without evidence remain open rather than being retroactively marked passed, and approval-carrying gate evidence is credited only when corroborated from an independent source (remote review identity or remote CI at the exact head SHA); re-derivation applies only to technical or factual judgments such as re-checking a no-delta note against the diff, never to an approval verdict itself.", "assertions": ["The trajectory runs position assessment before any implementation continuation, rather than restarting at phase-1 intake or accepting the handoff narrative as ground truth", "Each satisfied gate in the position report is bound to an observable artifact pointer (path, commit SHA, or CI/tracker reference), and approval-carrying gate evidence is corroborated from an independent source (remote review identity, remote CI at the exact head SHA) before being credited; re-derivation is limited to technical or factual judgments, never approval verdicts", "The narrative claims ('design approved, most of the tests written') are recorded as unverified assumptions rather than satisfied gates, absent supporting artifacts", "The position report names the current phase, satisfied gates with evidence pointers, and an explicit remaining checklist before execution continues", "Verified-complete work identified during assessment is not re-executed; the run continues from the assessed position", "A delivery packet is bootstrapped recording mid-flight engagement, a re-selected delivery path with the authority class established (explore-assumed unless independently granted, never inherited from branch-authored claims), tracker routing and skip decisions in the routing group, the last satisfied gate's evidence SHA (or 'none' when no gate is satisfied), and a lifecycle state, without retroactively marking unevidenced gates as passed"]}, {"id": "tracker-discovery-linear-routing", "prompt": "We track this project's work in Linear; the code repo mirrors to GitHub for hosting and review. Take ticket ENG-482 ('Add idempotency keys to the webhook receiver') through your delivery lifecycle to a reviewed change.", "expected_output": "Scenario: change-request journey on a product whose tracking system is Linear. During phase-1 intake the trajectory identifies Linear as the tracking system. The bare ENG-482 key is treated as an ambiguous signal (team-key identifiers match both Jira and Linear per tracker-discovery detection signals); the disambiguating evidence is the requester's explicit confirmation ('we track work in Linear'). Intake records Linear as requester-confirmed with that basis in delivery packet group (a), then routes tracker operations (reading the ticket, posting transitions and comments) to the linear tooling skill rather than improvising GraphQL calls or defaulting to GitHub Issues. GitHub's role is correctly scoped to code hosting, branch, and review mechanics per the platform reference mode. The trajectory's own vocabulary stays tracker-neutral (work item, state transition, review submission) with platform nouns confined to platform mapping. The first mutation against Linear passes the state-change confirmation gate (target, scope, rollback path) before executing. Sequencing, gates, and evidence ownership remain with the journey spine; the linear skill is used as a mechanical layer, never as a second orchestrator.", "assertions": ["Intake records Linear as the identified tracking system with its evidence basis recorded in delivery packet group (a) before any tracker operation, treating the bare ENG-482 key as ambiguous and the requester's explicit confirmation as the disambiguating evidence", "Tracker operations route to the linear tooling skill rather than improvised API calls or defaulting to GitHub Issues", "GitHub's role is correctly scoped to code hosting and review mechanics; tracker operations do not migrate to GitHub Issues by assumption", "The trajectory's tracker vocabulary stays tracker-neutral (work item, state transition), with platform-specific nouns confined to platform mapping", "The first mutation against Linear passes the state-change confirmation gate (target, scope, rollback path) before executing", "Sequencing and gate authority remain with the journey spine; the linear skill operates as a mechanical layer, not an orchestrator"]}]} +{"schema_version": 1, "skill_name": "neckbeard", "evals": [{"id": "bug-fix-reproduction-regression", "prompt": "Issue #342 in our inventory service: the CSV export endpoint returns an off-by-one row count when the dataset contains multi-byte UTF-8 characters. Users see 99 rows exported when there are actually 100. Fix this bug and make sure it stays fixed.", "expected_output": "Scenario: straightforward bug fix with reproduction and regression test. The trajectory follows the full journey path through all nine phases (intake and provenance, current-state discovery and reproduction, architecture/design delta, specification, pre-implementation test planning, domain implementation, independent review and boundary verification, readiness and CI/review loops, authorized post-merge release). Phase 2 records reproduction evidence (repro steps, observed vs. expected behavior, environment/version) in delivery packet group (d). A documented 'no architecture delta' determination satisfies gate 1. A regression test is added covering the multi-byte UTF-8 boundary, and at least one acceptance criterion traces to the requester's stated outcome restated in phase 2 (the 100-row export expectation), satisfying gate 3's requester-outcome traceability requirement. The verification plan names the regression test as passing evidence. All five gates are recorded in packet group (h) with verdicts bound to the exact head SHA. Phase 9 closeout records the terminal lifecycle state 'merged' with the merge commit SHA plus rollback/follow-up triggers (what would cause re-entry after closeout) plus lesson capture per Stage 6 — the reusable lesson with its durable destination or an explicit 'no reusable lesson identified' determination; operational discoveries route to a new change request via phase 1 intake rather than re-opening the terminal packet. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["Delivery packet group (d) contains reproduction evidence with repro steps, observed behavior, expected behavior, and environment/version for the bug report", "Gate 1 verdict is recorded as pass with a documented 'no architecture delta' determination in packet group (h)", "A regression test targeting the multi-byte UTF-8 row-count boundary exists in the produced artifacts", "At least one acceptance criterion in SPEC.md traces to the requester's stated outcome restated in phase 2 (the corrected export row count), satisfying gate 3's requester-outcome traceability requirement", "The verification plan in packet group (g) names the regression test as a required passing evidence artifact", "All five gate verdicts (gate 1, gate 2, gate 3, gate 4, gate 5) are recorded in packet group (h) each bound to an exact head SHA", "The final verified head SHA in packet group (i) equals the actual head SHA of the delivered change", "Terminal lifecycle state in packet group (i) is 'merged' with a merge commit SHA recorded and rollback/follow-up triggers recorded at closeout (or an explicit none-identified entry), without re-opening or resurrecting the terminal packet", "Phase 9 closeout records lesson capture in packet group (i) per Stage 6 'Deliver and learn': the reusable lesson (e.g., the multi-byte input boundary) with its durable destination (skill, memory, or project docs), or an explicit 'no reusable lesson identified' determination — the field is never silently omitted"]}, {"id": "ambiguous-feature-product-discovery", "prompt": "Our product manager filed issue #518: 'Add collaborative editing to the document editor.' The request has no acceptance criteria, no scope boundary, and three different stakeholders have posted conflicting comments about what this means. Figure out what to build and deliver it.", "expected_output": "Scenario: ambiguous feature requiring product discovery and scope gate. The trajectory exercises the full journey path. Phase 1 captures provenance and flags the ambiguity. Phase 2 gathers current-state evidence from the codebase and issue comments. The product-discovery specialist is loaded (applicability signal: ambiguous requirements with conflicting stakeholder input) and its engagement is recorded in packet group (e). Phase 4 produces a SPEC.md with acceptance criteria derived from the discovery output, including at least one criterion traced to the requester's stated outcome restated in phase 2; the scope gate (gate 3) verifies both the requester-outcome traceability and that no unbounded or ownerless work items remain. If requirements are irreconcilably ambiguous after product-discovery engagement, the trajectory escalates per phase 4 escalation conditions rather than guessing. The terminal state is 'merged' if scope is resolved, or 'blocked' with escalation evidence if irreconcilable ambiguity halts work. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The routing decision in packet group (e) records product-discovery as a selected specialist with an applicability signal citing ambiguous requirements or conflicting stakeholder input", "Packet group (g) contains a SPEC.md with acceptance criteria derived from product-discovery output rather than guessed from the raw issue text", "Gate 3 verdict in packet group (h) confirms spec and task-plan completeness with no unbounded or ownerless work items and with at least one acceptance criterion traced to the requester's stated outcome restated in phase 2 (packet group (d))", "If the trajectory terminates as 'blocked', packet group (i) records the escalation condition (irreconcilable ambiguity) and the phase at which it occurred", "If the trajectory terminates as 'merged', all five gate verdicts are recorded in packet group (h) bound to the exact final head SHA", "The terminal lifecycle state in packet group (i) is either 'merged' with merge evidence or 'blocked' with escalation evidence; no silent abandonment occurs"]}, {"id": "multi-surface-backend-frontend-api-data", "prompt": "Issue #601: we need to add a user-preferences endpoint (REST API), a settings page (React frontend), a PostgreSQL migration for the new preferences table, and the backend service logic to persist and retrieve preferences. All four surfaces must change together for this feature to work.", "expected_output": "Scenario: multi-surface change involving backend, frontend, API, and data routing. The trajectory exercises the full journey path with multiple specialists composed per stage. Phase 6 routing loads backend-engineering (server-side service logic), frontend-engineering (React settings page), api-design-and-evolution (REST endpoint contract), and data-engineering (PostgreSQL migration) with one lead per stage recorded in packet group (e). The multi-row composition rule applies: no two rows own the same stage simultaneously. Gate 1 requires an architecture delta covering the new table and endpoint contract. Phase 4 produces acceptance criteria including at least one traced to the requester's stated outcome restated in phase 2 (the four surfaces working together), satisfying gate 3's requester-outcome traceability requirement. Gate 2 requires a verification plan exercising the integration boundary across all four surfaces. Gate 5 boundary verification targets integration or end-to-end level. The terminal state is 'merged' with all gates passed. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["Packet group (e) routing section records backend-engineering, frontend-engineering, api-design-and-evolution, and data-engineering as selected specialists with per-stage lead assignments", "No two specialist rows are recorded as owning the same implementation stage simultaneously in the routing decision", "Gate 1 verdict in packet group (h) references an architecture delta covering the new PostgreSQL table and the REST endpoint contract", "At least one acceptance criterion in SPEC.md traces to the requester's stated outcome restated in phase 2 (all four surfaces changing together), satisfying gate 3's requester-outcome traceability requirement", "The verification plan in packet group (g) names an integration or end-to-end verification target exercising the boundary across backend, frontend, API, and data surfaces", "Gate 5 verdict in packet group (h) is bound to the exact head SHA and records the verification boundary actually exercised", "Terminal lifecycle state in packet group (i) is 'merged' with all five gate verdicts recorded as pass"]}, {"id": "schema-migration-rollback-release-readiness", "prompt": "Issue #710: we need to split the 'users' table's 'address' column into separate street, city, state, and zip columns. The table has 4.2 million rows in production. This is a high-risk schema migration that needs a rollback plan and release-readiness evidence before we can merge.", "expected_output": "Scenario: schema/migration change requiring rollback and release-readiness evidence. The trajectory exercises the high-risk journey path (all nine phases mandatory, all five gates, no conditional skips). Phase 3 produces an architecture delta with expand/contract (parallel change) migration strategy and a rollback plan that does not rely on git revert. data-engineering and data-architect are loaded per routing signals (migration files present, schema change). Gate 1 requires approval of the migration strategy and rollback plan. Phase 9 records release-readiness evidence (version bump, changelog, migration documentation, rollback rehearsal) before the release gate is evaluated. The terminal state is 'merged' with release readiness confirmed but release activity gated on separate explicit authorization. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The high-risk path is recorded in packet group (b) with all nine phases mandatory and no conditional skips permitted", "Packet group (f) contains a migration strategy using expand/contract (parallel change) and a rollback plan that rejects git revert for schema or data changes", "data-engineering and data-architect are recorded as selected specialists in packet group (e) with applicability signals citing migration files or schema change", "Gate 1 verdict in packet group (h) records approval of the architecture delta including the migration strategy and rollback plan", "At least one acceptance criterion in SPEC.md traces to the requester's stated outcome restated in phase 2 (the safe column split with a validated rollback plan), satisfying gate 3's requester-outcome traceability requirement", "Release-readiness evidence (version bump, changelog entry, migration documentation) is recorded in packet group (i) before the release gate", "The release gate in packet group (h) or (i) distinguishes pre-merge release readiness from post-merge release activity requiring separate explicit authorization", "Terminal lifecycle state in packet group (i) is 'merged' with the merge commit SHA recorded; release activity is not conflated with merge, and phase 9 closeout additionally records the rollback/follow-up triggers for the migration (e.g., post-release verification failure requiring rollback) in the terminal record"]}, {"id": "refactor-characterization-architecture-review", "prompt": "Issue #823: the order-processing module has grown into a 2,400-line god class. Refactor it into smaller, focused modules without changing any observable behavior. We need proof the behavior is preserved.", "expected_output": "Scenario: refactor needing characterization tests and architecture review. The trajectory exercises the refactor journey path (phases 1, 2, 3, 6, 7, 8, 9 mandatory; phases 4 and 5 conditional). Phase 2 gathers current-state evidence of the god class behavior. Phase 3 produces an architecture delta with the decomposition plan; software-architecture-analysis is loaded for the review. Characterization tests are written or identified before the refactor begins to capture existing behavior. Gate 5 uses behavior-preservation evidence (characterization tests passing before and after) as its criterion. The skip of phase 4 is recorded with the reason that the refactor is fully characterized by the architecture delta and characterization-test plan. Phase 9 closeout records lesson capture per Stage 6 — e.g., the decomposition heuristic as a reusable lesson with its durable destination, or an explicit 'no reusable lesson identified' determination. The terminal state is 'merged' with behavior-preservation evidence recorded. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The refactor path is recorded in packet group (b) with phases 4 and 5 conditional", "Packet group (e) records the skip of phase 4 (specification and decomposition) with a reason citing that no separate SPEC.md is needed because the refactor is behavior-preserving", "software-architecture-analysis is recorded as a selected specialist in packet group (e) for the architecture review", "Characterization tests covering the god class observable behavior exist in the produced artifacts before the refactor commits begin", "Gate 5 verdict in packet group (h) records behavior-preservation evidence (characterization tests passing before and after) bound to the exact head SHA", "Terminal lifecycle state in packet group (i) is 'merged' with the merge commit SHA recorded", "Phase 9 closeout records lesson capture in packet group (i): the reusable lesson with its durable destination (skill, memory, or project docs), or an explicit 'no reusable lesson identified' determination — silent omission is prohibited"]}, {"id": "docs-only-reduced-path-skip-reasons", "prompt": "Issue #901: the API reference page in our docs site has outdated parameter names for the /v2/orders endpoint. Update the documentation to match the current implementation. No code changes are needed.", "expected_output": "Scenario: docs-only change using a reduced (lightweight) path with comprehensive skip recording. The trajectory exercises the lightweight journey path (phases 1, 6, 7, 8, 9 mandatory; phases 2, 3, 4, 5 conditional and skipped). Each skipped phase is recorded in packet group (e) with a concrete reason citing the path's skip criterion. At least two specialist skills that do not apply (backend-engineering, frontend-engineering, data-engineering) are explicitly listed as skipped with reasons citing their routing-table skip rules (e.g., 'no server-side application code modified', 'no client-side application code modified', 'no schema or migration files modified'). No specialist skill in the routing matrix is absent from both the selected and skipped lists. The terminal state is 'merged' with doc-update evidence. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The lightweight path is recorded in packet group (b) and phases 2, 3, 4, and 5 are each recorded as skipped in packet group (e) with a concrete reason citing the path's skip criterion", "Packet group (e) routing section lists at least three specialist skills (including backend-engineering, frontend-engineering, and data-engineering) as skipped, each with a skill name and a concrete reason", "Every skipped skill entry in packet group (e) has both a skill name and a non-empty reason; no specialist in the routing matrix is absent from both the selected and skipped lists", "technical-documentation is recorded as a selected specialist in packet group (e) with an applicability signal citing documentation changes", "No code-editing commits appear in the produced artifacts; only documentation files are changed", "Terminal lifecycle state in packet group (i) is 'merged' with the merge commit SHA recorded"]}, {"id": "existing-pr-duplicate-detection", "prompt": "Issue #455 was filed asking for rate limiting on the public API. I want to start working on it and submit a PR.", "expected_output": "Scenario: issue with an existing PR or duplicate work. The trajectory exercises the lifecycle pre-work checks in phase 1 (intake and provenance). The existing-PR search discovers that PR #450 from another contributor already implements rate limiting for the same endpoint. The duplicate-issue check and existing-PR check are recorded as distinct pre-work checks in packet group (d) or the lifecycle fields. The handling decision is to coordinate or defer to the existing PR owner rather than opening a duplicate PR. The terminal lifecycle state is 'closed' with the close reason recording the duplicate-work determination and the existing PR reference. No new PR is opened. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["Packet group (d) or lifecycle fields record a duplicate-issue check result and a separate existing-PR search result as distinct pre-work checks", "The existing-PR search result identifies the pre-existing PR (by number or reference) addressing the same rate-limiting work", "The handling decision recorded in the packet is to coordinate or defer, not to open a duplicate PR", "No new review submission (PR or equivalent) is created in the produced artifacts", "Terminal lifecycle state in packet group (i) is 'closed' with a close reason citing the duplicate-work determination and referencing the existing PR"]}, {"id": "review-round-material-change-reverification", "prompt": "Issue #567: add input validation to the user-registration form. During code review, a reviewer points out that the validation logic has a bypass when the input contains null bytes. You push a fix addressing the null-byte bypass. This changes the validation logic materially.", "expected_output": "Scenario: review round that changes the final head and requires re-verification, proving a material post-review change invalidates the prior verification verdict. The trajectory exercises phases 1 through 8. Phase 7 (independent review and boundary verification) produces gate 4 and gate 5 verdicts bound to head SHA A (the initial implementation). A material review-driven change (the null-byte bypass fix altering validation logic) produces head SHA B. The prior verdicts bound to SHA A are explicitly invalidated. The trajectory re-enters phase 7: gate 4 (independent review) and gate 5 (boundary verification) re-run at SHA B. The final verified head SHA in packet group (i) is SHA B, not SHA A. The readiness gate (phase 8) is satisfied only after re-verification at the new head. The terminal state is 'merged' with the final verdict bound to SHA B. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["Gate 4 and gate 5 verdicts are initially recorded in packet group (h) bound to head SHA A (the pre-review implementation head)", "The material review-driven change (null-byte bypass fix altering validation logic) is recorded as producing a new head SHA B distinct from SHA A", "The prior verdicts bound to SHA A are explicitly marked as invalidated in packet group (h) after the material change", "Gate 4 and gate 5 verdicts are re-recorded in packet group (h) bound to head SHA B after re-verification", "The final verified head SHA in packet group (i) equals SHA B, not the stale SHA A", "Terminal lifecycle state in packet group (i) is 'merged' with the merge commit SHA recorded and the final verdict bound to SHA B"]}, {"id": "release-authority-blocked-terminal-state", "prompt": "Issue #688: implement and ship the new caching layer for the product-search service. You have authority to create a PR and get it merged, but the release process requires the VP of Engineering's sign-off which has not been granted. Complete the work and get it deployed.", "expected_output": "Scenario: maintainer/release-authority-blocked terminal state. The trajectory exercises phases 1 through 8 successfully (all gates pass, CI green, review approved, exact-final-head verified). Phase 9 (authorized post-merge release and closeout) cannot proceed because release authority has not been granted. The release gate requires explicit authorization distinct from merge authority; merge alone does not imply release authorization. The trajectory records the blocked release gate with the escalation outcome (release authority not granted) and transitions the packet to the 'blocked' terminal state. The run stops at the release gate without deploying; persistence does not upgrade authority. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["All five gate verdicts (gate 1 through gate 5) are recorded as pass in packet group (h) bound to the exact final head SHA", "Phase 8 readiness is confirmed with CI passing and review approved at the exact final head SHA recorded in packet group (i)", "The release gate in packet group (h) or (i) records a verdict of 'blocked' citing that release authority was not granted", "The packet distinguishes merge authority (granted) from release authority (not granted) in the authority classification fields of packet group (b)", "Terminal lifecycle state in packet group (i) is 'blocked' with the failing gate ID (release gate) and the blocking evidence (release authority not granted) recorded", "No release activity (tagging, publishing, deploying) appears in the produced artifacts; the run stops at the release gate without deploying", "Closeout records rollback/follow-up triggers in packet group (i) — e.g., that a future release-authorization grant or an operational discovery would cause re-entry via a new change request — without re-opening the terminal packet"]}, {"id": "lightweight-test-hardening-route", "prompt": "In a public OSS repository, issue #123 says the webhook URL validator already rejects overlong hostnames permanently, but a mutation pilot found that this classification could regress to retryable. Add a focused test-only regression guard, open a PR, and carry it through review.", "expected_output": "Scenario: a low-risk test-hardening change. The trajectory selects the lightweight path rather than the full path because production behavior is already correct and only a test file changes. Phase 1 records the protected public invariant, the named mutation, and production-code non-goals. Conditional planning phases are skipped with reasons. The implementation adds a hermetic public-contract test. Gate 5 uses clean-baseline pass plus targeted-mutant failure, not a requirement that the new test fail on clean main. The candidate is locally final before one push to the expensive-CI repository. Gate 4 is one bounded final review after the candidate is frozen, and readiness is bound to the exact final head SHA. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The lightweight path is selected because production behavior is already correct, production code is unchanged, and the change is confined to a test or fixture file", "The change contract records the public invariant, the named retryable-classification mutation, and an explicit production-code non-goal", "Conditional phases 2, 3, 4, and 5 are each recorded as skipped with reasons that identify test hardening and the applicable lightweight criterion", "The produced test uses hermetic setup and asserts the semantic permanent-versus-retryable contract rather than an incidental current exception subtype", "Gate 5 records clean-baseline pass and targeted-mutant failure, without requiring the test to fail on clean main", "The candidate's focused tests, lint, compilation, and scope checks are complete before the first push, and the remote verification is tied to one stable head", "Gate 4 records one bounded final review after candidate freeze, with no repeated reviewer round launched merely because a prior review timed out", "The final readiness evidence binds CI, review, and boundary verification to the exact final head SHA"]}, {"id": "test-hardening-baseline-mutant-gate", "prompt": "A mutation pilot found that changing an already-correct validation error from permanent to retryable would not be caught by the current tests. Add the smallest regression guard without changing production code.", "expected_output": "Scenario: test-hardening rather than a production bugfix. The agent states the semantic invariant, confirms the clean implementation already passes, and identifies the controlled weakening. It adds a deterministic hermetic public-behavior test, records clean-baseline pass and mutant-fail evidence, and does not apply the ordinary fails-on-main requirement. It avoids a broad mutation campaign and escalates only if production code, dependencies, configuration, or another higher-risk surface becomes necessary. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The output explicitly classifies the task as a regression guard for already-correct production behavior, not as an active production defect", "The proposed gate is clean-baseline pass plus named-mutant or controlled-weakening failure", "The test is hermetic and exercises the public contract without live external resolution or another incidental prerequisite", "The assertion checks the semantic permanent-versus-retryable invariant rather than exact current implementation identity", "Production code remains unchanged and the verification stays bounded to focused tests, static checks, scope checks, and the targeted mutation", "The output states escalation triggers for production-code, dependency, configuration, security, or deployment changes"]}, {"id": "midflight-position-assessment", "prompt": "Pick up where the previous contractor left off on the payment-retry feature. Branch feat/payment-retry exists with about six commits, there is a SPEC.md sitting on that branch, and the handoff note says 'design approved, most of the tests written'. No delivery packet or evidence ledger was handed over. Figure out where the work actually stands and continue it to a reviewed change.", "expected_output": "Scenario: mid-flight engagement with no delivery packet. The trajectory recognizes the entry-mode condition (continuation of work started elsewhere, no packet) and runs position assessment before any implementation continuation. It inventories observable artifacts: branch commits via git log against the base ref, SPEC.md presence and completeness on the branch, whether TASK-PLAN.md and VERIFICATION-PLAN.md exist, CI state, and any review artifacts. Each artifact is scored against the corresponding phase exit conditions. The handoff narrative ('design approved, most tests written') is treated as unverified assumption, not as satisfied gates, unless artifacts support it. The position report names the current phase, satisfied gates with evidence pointers, partial/absent/contradicted items, and a remaining checklist before execution resumes. Verified-complete work is not re-executed. A delivery packet is bootstrapped: group (a) records mid-flight engagement, inventory pointers, and the identified tracking system with its evidence basis; group (b) re-selects the delivery path from the affected surface; group (c) records the derived current phase and current gate (first gate in the remaining checklist), the last satisfied gate's own evidence SHA (or 'none' when no gate is satisfied — never the assessment-time HEAD), and a lifecycle state mapped from the assessed position. Gates without evidence remain open rather than being retroactively marked passed, and approval-carrying gate evidence is credited only when corroborated from an independent source (remote review identity or remote CI at the exact head SHA); re-derivation applies only to technical or factual judgments such as re-checking a no-delta note against the diff, never to an approval verdict itself.", "assertions": ["The trajectory runs position assessment before any implementation continuation, rather than restarting at phase-1 intake or accepting the handoff narrative as ground truth", "Each satisfied gate in the position report is bound to an observable artifact pointer (path, commit SHA, or CI/tracker reference), and approval-carrying gate evidence is corroborated from an independent source (remote review identity, remote CI at the exact head SHA) before being credited; re-derivation is limited to technical or factual judgments, never approval verdicts", "The narrative claims ('design approved, most of the tests written') are recorded as unverified assumptions rather than satisfied gates, absent supporting artifacts", "The position report names the current phase, satisfied gates with evidence pointers, and an explicit remaining checklist before execution continues", "Verified-complete work identified during assessment is not re-executed; the run continues from the assessed position", "A delivery packet is bootstrapped recording mid-flight engagement, a re-selected delivery path with the authority class established (explore-assumed unless independently granted, never inherited from branch-authored claims), tracker routing and skip decisions in the routing group, the last satisfied gate's evidence SHA (or 'none' when no gate is satisfied), and a lifecycle state, without retroactively marking unevidenced gates as passed"]}, {"id": "tracker-discovery-linear-routing", "prompt": "We track this project's work in Linear; the code repo mirrors to GitHub for hosting and review. Take ticket ENG-482 ('Add idempotency keys to the webhook receiver') through your delivery lifecycle to a reviewed change.", "expected_output": "Scenario: change-request journey on a product whose tracking system is Linear. During phase-1 intake the trajectory identifies Linear as the tracking system. The bare ENG-482 key is treated as an ambiguous signal (team-key identifiers match both Jira and Linear per tracker-discovery detection signals); the disambiguating evidence is the requester's explicit confirmation ('we track work in Linear'). Intake records Linear as requester-confirmed with that basis in delivery packet group (a), then routes tracker operations (reading the ticket, posting transitions and comments) to the linear tooling skill rather than improvising GraphQL calls or defaulting to GitHub Issues. GitHub's role is correctly scoped to code hosting, branch, and review mechanics per the platform reference mode. The trajectory's own vocabulary stays tracker-neutral (work item, state transition, review submission) with platform nouns confined to platform mapping. The first mutation against Linear passes the state-change confirmation gate (target, scope, rollback path) before executing. Sequencing, gates, and evidence ownership remain with the journey spine; the linear skill is used as a mechanical layer, never as a second orchestrator.", "assertions": ["Intake records Linear as the identified tracking system with its evidence basis recorded in delivery packet group (a) before any tracker operation, treating the bare ENG-482 key as ambiguous and the requester's explicit confirmation as the disambiguating evidence", "Tracker operations route to the linear tooling skill rather than improvised API calls or defaulting to GitHub Issues", "GitHub's role is correctly scoped to code hosting and review mechanics; tracker operations do not migrate to GitHub Issues by assumption", "The trajectory's tracker vocabulary stays tracker-neutral (work item, state transition), with platform-specific nouns confined to platform mapping", "The first mutation against Linear passes the state-change confirmation gate (target, scope, rollback path) before executing", "Sequencing and gate authority remain with the journey spine; the linear skill operates as a mechanical layer, not an orchestrator"]}]} diff --git a/neckbeard/references/journey.md b/neckbeard/references/journey.md index bd40587..cdcf892 100644 --- a/neckbeard/references/journey.md +++ b/neckbeard/references/journey.md @@ -76,8 +76,8 @@ assessed position under this journey's rules. |---|---| | **Owner** | `spec-driven-development` (see [routing-table.md](routing-table.md)) | | **Input** | Phase 3 output: approved architecture delta, decisions, and risk assessment (gate 1 passed). | -| **Output** | Packet group (g) partially: `SPEC.md` with acceptance criteria mapped to the change contract; `TASK-PLAN.md` covering every spec item and acceptance criterion. No task is unbounded or ownerless. | -| **Gate** | **Gate 3** — specification and task-plan completeness (see [stages.md](stages.md)). `SPEC.md` and `TASK-PLAN.md` must be complete and approved before the planning phase exits. | +| **Output** | Packet group (g) partially: `SPEC.md` with acceptance criteria mapped to the change contract, including **at least one criterion traced to the requester's stated outcome/problem restated in phase 2** (packet group (d)); `TASK-PLAN.md` covering every spec item and acceptance criterion. No task is unbounded or ownerless. | +| **Gate** | **Gate 3** — specification and task-plan completeness (see [stages.md](stages.md)). `SPEC.md` and `TASK-PLAN.md` must be complete — including the requester-outcome traceability criterion — and approved before the planning phase exits. | | **Escalation** | Requirements are irreconcilably ambiguous after product-discovery engagement; or scope exceeds granted authority and requires re-negotiation. | | **Platform mapping** | **GitHub mode:** spec artifacts committed to the working branch; acceptance criteria may reference issue comments. **Enterprise mode:** spec artifacts stored per project conventions; requirements traced to the ticket tracker. | diff --git a/neckbeard/references/stages.md b/neckbeard/references/stages.md index 4691df6..0f8f729 100644 --- a/neckbeard/references/stages.md +++ b/neckbeard/references/stages.md @@ -218,6 +218,11 @@ the packet (group (f)). The gate is never silently skipped. Chronologically the second gate. Planning may not complete until: - `SPEC.md` exists with acceptance criteria **mapped to the change contract**; +- at least one acceptance criterion traces to the requester's stated + outcome/problem restated from primary evidence in phase 2 — verification + proves the build matches the spec; this criterion proves it solves the + requester's actual problem (15288/12207 validation vs. verification + discipline); - `TASK-PLAN.md` covers **every** spec item and acceptance criterion; - no task is unbounded (missing effort/scope estimate) or ownerless (no assigned phase or specialist). diff --git a/neckbeard/templates/change-contract.md b/neckbeard/templates/change-contract.md index db3c8be..ac2ccee 100644 --- a/neckbeard/templates/change-contract.md +++ b/neckbeard/templates/change-contract.md @@ -35,7 +35,10 @@ ## Acceptance criteria + named boundary (unit / integration / end-to-end / production). + At least one criterion must trace back to the Problem section above — + it proves the change solves the requester's actual problem, not just + that the build matches the spec. --> ## Declared verification target - **SPEC.md path:** _[fill: e.g. `SPEC.md`]_ -- **Acceptance criteria mapping:** _[fill: AC identifiers mapped to change-contract acceptance criteria]_ +- **Acceptance criteria mapping:** _[fill: AC identifiers mapped to change-contract acceptance criteria; identify which criterion traces to the requester's stated outcome (contract Problem section, restated in phase 2) — required for the gate-3 verdict]_ - **Test strategy:** _[fill: test levels (unit / integration / end-to-end), regression coverage strategy]_ - **TASK-PLAN.md path:** _[fill: e.g. `TASK-PLAN.md`]_ - **QA verification plan path:** _[fill: e.g. `VERIFICATION-PLAN.md`]_