diff --git a/neckbeard/evals/evals.json b/neckbeard/evals/evals.json index 2a30eb7..927f69e 100644 --- a/neckbeard/evals/evals.json +++ b/neckbeard/evals/evals.json @@ -1 +1 @@ -{"schema_version": 1, "skill_name": "neckbeard", "evals": [{"id": "bug-fix-reproduction-regression", "prompt": "Issue #342 in our inventory service: the CSV export endpoint returns an off-by-one row count when the dataset contains multi-byte UTF-8 characters. Users see 99 rows exported when there are actually 100. Fix this bug and make sure it stays fixed.", "expected_output": "Scenario: straightforward bug fix with reproduction and regression test. The trajectory follows the full journey path through all nine phases (intake and provenance, current-state discovery and reproduction, architecture/design delta, specification, pre-implementation test planning, domain implementation, independent review and boundary verification, readiness and CI/review loops, authorized post-merge release). Phase 2 records reproduction evidence (repro steps, observed vs. expected behavior, environment/version) in delivery packet group (d). A documented 'no architecture delta' determination satisfies gate 1. A regression test is added covering the multi-byte UTF-8 boundary. The verification plan names the regression test as passing evidence. All five gates are recorded in packet group (h) with verdicts bound to the exact head SHA. Phase 9 closeout records the terminal lifecycle state 'merged' with the merge commit SHA plus rollback/follow-up triggers (what would cause re-entry after closeout); operational discoveries route to a new change request via phase 1 intake rather than re-opening the terminal packet. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["Delivery packet group (d) contains reproduction evidence with repro steps, observed behavior, expected behavior, and environment/version for the bug report", "Gate 1 verdict is recorded as pass with a documented 'no architecture delta' determination in packet group (h)", "A regression test targeting the multi-byte UTF-8 row-count boundary exists in the produced artifacts", "The verification plan in packet group (g) names the regression test as a required passing evidence artifact", "All five gate verdicts (gate 1, gate 2, gate 3, gate 4, gate 5) are recorded in packet group (h) each bound to an exact head SHA", "The final verified head SHA in packet group (i) equals the actual head SHA of the delivered change", "Terminal lifecycle state in packet group (i) is 'merged' with a merge commit SHA recorded and rollback/follow-up triggers recorded at closeout (or an explicit none-identified entry), without re-opening or resurrecting the terminal packet"]}, {"id": "ambiguous-feature-product-discovery", "prompt": "Our product manager filed issue #518: 'Add collaborative editing to the document editor.' The request has no acceptance criteria, no scope boundary, and three different stakeholders have posted conflicting comments about what this means. Figure out what to build and deliver it.", "expected_output": "Scenario: ambiguous feature requiring product discovery and scope gate. The trajectory exercises the full journey path. Phase 1 captures provenance and flags the ambiguity. Phase 2 gathers current-state evidence from the codebase and issue comments. The product-discovery specialist is loaded (applicability signal: ambiguous requirements with conflicting stakeholder input) and its engagement is recorded in packet group (e). Phase 4 produces a SPEC.md with acceptance criteria derived from the discovery output; the scope gate (gate 3) verifies that no unbounded or ownerless work items remain. If requirements are irreconcilably ambiguous after product-discovery engagement, the trajectory escalates per phase 4 escalation conditions rather than guessing. The terminal state is 'merged' if scope is resolved, or 'blocked' with escalation evidence if irreconcilable ambiguity halts work. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The routing decision in packet group (e) records product-discovery as a selected specialist with an applicability signal citing ambiguous requirements or conflicting stakeholder input", "Packet group (g) contains a SPEC.md with acceptance criteria derived from product-discovery output rather than guessed from the raw issue text", "Gate 3 verdict in packet group (h) confirms spec and task-plan completeness with no unbounded or ownerless work items", "If the trajectory terminates as 'blocked', packet group (i) records the escalation condition (irreconcilable ambiguity) and the phase at which it occurred", "If the trajectory terminates as 'merged', all five gate verdicts are recorded in packet group (h) bound to the exact final head SHA", "The terminal lifecycle state in packet group (i) is either 'merged' with merge evidence or 'blocked' with escalation evidence; no silent abandonment occurs"]}, {"id": "multi-surface-backend-frontend-api-data", "prompt": "Issue #601: we need to add a user-preferences endpoint (REST API), a settings page (React frontend), a PostgreSQL migration for the new preferences table, and the backend service logic to persist and retrieve preferences. All four surfaces must change together for this feature to work.", "expected_output": "Scenario: multi-surface change involving backend, frontend, API, and data routing. The trajectory exercises the full journey path with multiple specialists composed per stage. Phase 6 routing loads backend-engineering (server-side service logic), frontend-engineering (React settings page), api-design-and-evolution (REST endpoint contract), and data-engineering (PostgreSQL migration) with one lead per stage recorded in packet group (e). The multi-row composition rule applies: no two rows own the same stage simultaneously. Gate 1 requires an architecture delta covering the new table and endpoint contract. Gate 2 requires a verification plan exercising the integration boundary across all four surfaces. Gate 5 boundary verification targets integration or end-to-end level. The terminal state is 'merged' with all gates passed. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["Packet group (e) routing section records backend-engineering, frontend-engineering, api-design-and-evolution, and data-engineering as selected specialists with per-stage lead assignments", "No two specialist rows are recorded as owning the same implementation stage simultaneously in the routing decision", "Gate 1 verdict in packet group (h) references an architecture delta covering the new PostgreSQL table and the REST endpoint contract", "The verification plan in packet group (g) names an integration or end-to-end verification target exercising the boundary across backend, frontend, API, and data surfaces", "Gate 5 verdict in packet group (h) is bound to the exact head SHA and records the verification boundary actually exercised", "Terminal lifecycle state in packet group (i) is 'merged' with all five gate verdicts recorded as pass"]}, {"id": "schema-migration-rollback-release-readiness", "prompt": "Issue #710: we need to split the 'users' table's 'address' column into separate street, city, state, and zip columns. The table has 4.2 million rows in production. This is a high-risk schema migration that needs a rollback plan and release-readiness evidence before we can merge.", "expected_output": "Scenario: schema/migration change requiring rollback and release-readiness evidence. The trajectory exercises the high-risk journey path (all nine phases mandatory, all five gates, no conditional skips). Phase 3 produces an architecture delta with expand/contract (parallel change) migration strategy and a rollback plan that does not rely on git revert. data-engineering and data-architect are loaded per routing signals (migration files present, schema change). Gate 1 requires approval of the migration strategy and rollback plan. Phase 9 records release-readiness evidence (version bump, changelog, migration documentation, rollback rehearsal) before the release gate is evaluated. The terminal state is 'merged' with release readiness confirmed but release activity gated on separate explicit authorization. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The high-risk path is recorded in packet group (b) with all nine phases mandatory and no conditional skips permitted", "Packet group (f) contains a migration strategy using expand/contract (parallel change) and a rollback plan that rejects git revert for schema or data changes", "data-engineering and data-architect are recorded as selected specialists in packet group (e) with applicability signals citing migration files or schema change", "Gate 1 verdict in packet group (h) records approval of the architecture delta including the migration strategy and rollback plan", "Release-readiness evidence (version bump, changelog entry, migration documentation) is recorded in packet group (i) before the release gate", "The release gate in packet group (h) or (i) distinguishes pre-merge release readiness from post-merge release activity requiring separate explicit authorization", "Terminal lifecycle state in packet group (i) is 'merged' with the merge commit SHA recorded; release activity is not conflated with merge, and phase 9 closeout additionally records the rollback/follow-up triggers for the migration (e.g., post-release verification failure requiring rollback) in the terminal record"]}, {"id": "refactor-characterization-architecture-review", "prompt": "Issue #823: the order-processing module has grown into a 2,400-line god class. Refactor it into smaller, focused modules without changing any observable behavior. We need proof the behavior is preserved.", "expected_output": "Scenario: refactor needing characterization tests and architecture review. The trajectory exercises the refactor journey path (phases 1, 2, 3, 6, 7, 8, 9 mandatory; phases 4 and 5 conditional). Phase 2 gathers current-state evidence of the god class behavior. Phase 3 produces an architecture delta with the decomposition plan; software-architecture-analysis is loaded for the review. Characterization tests are written or identified before the refactor begins to capture existing behavior. Gate 5 uses behavior-preservation evidence (characterization tests passing before and after) as its criterion. The skip of phase 4 is recorded with the reason that the refactor is fully characterized by the architecture delta and characterization-test plan. The terminal state is 'merged' with behavior-preservation evidence recorded. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The refactor path is recorded in packet group (b) with phases 4 and 5 conditional", "Packet group (e) records the skip of phase 4 (specification and decomposition) with a reason citing that no separate SPEC.md is needed because the refactor is behavior-preserving", "software-architecture-analysis is recorded as a selected specialist in packet group (e) for the architecture review", "Characterization tests covering the god class observable behavior exist in the produced artifacts before the refactor commits begin", "Gate 5 verdict in packet group (h) records behavior-preservation evidence (characterization tests passing before and after) bound to the exact head SHA", "Terminal lifecycle state in packet group (i) is 'merged' with the merge commit SHA recorded"]}, {"id": "docs-only-reduced-path-skip-reasons", "prompt": "Issue #901: the API reference page in our docs site has outdated parameter names for the /v2/orders endpoint. Update the documentation to match the current implementation. No code changes are needed.", "expected_output": "Scenario: docs-only change using a reduced (lightweight) path with comprehensive skip recording. The trajectory exercises the lightweight journey path (phases 1, 6, 7, 8, 9 mandatory; phases 2, 3, 4, 5 conditional and skipped). Each skipped phase is recorded in packet group (e) with a concrete reason citing the path's skip criterion. At least two specialist skills that do not apply (backend-engineering, frontend-engineering, data-engineering) are explicitly listed as skipped with reasons citing their routing-table skip rules (e.g., 'no server-side application code modified', 'no client-side application code modified', 'no schema or migration files modified'). No specialist skill in the routing matrix is absent from both the selected and skipped lists. The terminal state is 'merged' with doc-update evidence. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The lightweight path is recorded in packet group (b) and phases 2, 3, 4, and 5 are each recorded as skipped in packet group (e) with a concrete reason citing the path's skip criterion", "Packet group (e) routing section lists at least three specialist skills (including backend-engineering, frontend-engineering, and data-engineering) as skipped, each with a skill name and a concrete reason", "Every skipped skill entry in packet group (e) has both a skill name and a non-empty reason; no specialist in the routing matrix is absent from both the selected and skipped lists", "technical-documentation is recorded as a selected specialist in packet group (e) with an applicability signal citing documentation changes", "No code-editing commits appear in the produced artifacts; only documentation files are changed", "Terminal lifecycle state in packet group (i) is 'merged' with the merge commit SHA recorded"]}, {"id": "existing-pr-duplicate-detection", "prompt": "Issue #455 was filed asking for rate limiting on the public API. I want to start working on it and submit a PR.", "expected_output": "Scenario: issue with an existing PR or duplicate work. The trajectory exercises the lifecycle pre-work checks in phase 1 (intake and provenance). The existing-PR search discovers that PR #450 from another contributor already implements rate limiting for the same endpoint. The duplicate-issue check and existing-PR check are recorded as distinct pre-work checks in packet group (d) or the lifecycle fields. The handling decision is to coordinate or defer to the existing PR owner rather than opening a duplicate PR. The terminal lifecycle state is 'closed' with the close reason recording the duplicate-work determination and the existing PR reference. No new PR is opened. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["Packet group (d) or lifecycle fields record a duplicate-issue check result and a separate existing-PR search result as distinct pre-work checks", "The existing-PR search result identifies the pre-existing PR (by number or reference) addressing the same rate-limiting work", "The handling decision recorded in the packet is to coordinate or defer, not to open a duplicate PR", "No new review submission (PR or equivalent) is created in the produced artifacts", "Terminal lifecycle state in packet group (i) is 'closed' with a close reason citing the duplicate-work determination and referencing the existing PR"]}, {"id": "review-round-material-change-reverification", "prompt": "Issue #567: add input validation to the user-registration form. During code review, a reviewer points out that the validation logic has a bypass when the input contains null bytes. You push a fix addressing the null-byte bypass. This changes the validation logic materially.", "expected_output": "Scenario: review round that changes the final head and requires re-verification, proving a material post-review change invalidates the prior verification verdict. The trajectory exercises phases 1 through 8. Phase 7 (independent review and boundary verification) produces gate 4 and gate 5 verdicts bound to head SHA A (the initial implementation). A material review-driven change (the null-byte bypass fix altering validation logic) produces head SHA B. The prior verdicts bound to SHA A are explicitly invalidated. The trajectory re-enters phase 7: gate 4 (independent review) and gate 5 (boundary verification) re-run at SHA B. The final verified head SHA in packet group (i) is SHA B, not SHA A. The readiness gate (phase 8) is satisfied only after re-verification at the new head. The terminal state is 'merged' with the final verdict bound to SHA B. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["Gate 4 and gate 5 verdicts are initially recorded in packet group (h) bound to head SHA A (the pre-review implementation head)", "The material review-driven change (null-byte bypass fix altering validation logic) is recorded as producing a new head SHA B distinct from SHA A", "The prior verdicts bound to SHA A are explicitly marked as invalidated in packet group (h) after the material change", "Gate 4 and gate 5 verdicts are re-recorded in packet group (h) bound to head SHA B after re-verification", "The final verified head SHA in packet group (i) equals SHA B, not the stale SHA A", "Terminal lifecycle state in packet group (i) is 'merged' with the merge commit SHA recorded and the final verdict bound to SHA B"]}, {"id": "release-authority-blocked-terminal-state", "prompt": "Issue #688: implement and ship the new caching layer for the product-search service. You have authority to create a PR and get it merged, but the release process requires the VP of Engineering's sign-off which has not been granted. Complete the work and get it deployed.", "expected_output": "Scenario: maintainer/release-authority-blocked terminal state. The trajectory exercises phases 1 through 8 successfully (all gates pass, CI green, review approved, exact-final-head verified). Phase 9 (authorized post-merge release and closeout) cannot proceed because release authority has not been granted. The release gate requires explicit authorization distinct from merge authority; merge alone does not imply release authorization. The trajectory records the blocked release gate with the escalation outcome (release authority not granted) and transitions the packet to the 'blocked' terminal state. The run stops at the release gate without deploying; persistence does not upgrade authority. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["All five gate verdicts (gate 1 through gate 5) are recorded as pass in packet group (h) bound to the exact final head SHA", "Phase 8 readiness is confirmed with CI passing and review approved at the exact final head SHA recorded in packet group (i)", "The release gate in packet group (h) or (i) records a verdict of 'blocked' citing that release authority was not granted", "The packet distinguishes merge authority (granted) from release authority (not granted) in the authority classification fields of packet group (b)", "Terminal lifecycle state in packet group (i) is 'blocked' with the failing gate ID (release gate) and the blocking evidence (release authority not granted) recorded", "No release activity (tagging, publishing, deploying) appears in the produced artifacts; the run stops at the release gate without deploying", "Closeout records rollback/follow-up triggers in packet group (i) — e.g., that a future release-authorization grant or an operational discovery would cause re-entry via a new change request — without re-opening the terminal packet"]}, {"id": "lightweight-test-hardening-route", "prompt": "In a public OSS repository, issue #123 says the webhook URL validator already rejects overlong hostnames permanently, but a mutation pilot found that this classification could regress to retryable. Add a focused test-only regression guard, open a PR, and carry it through review.", "expected_output": "Scenario: a low-risk test-hardening change. The trajectory selects the lightweight path rather than the full path because production behavior is already correct and only a test file changes. Phase 1 records the protected public invariant, the named mutation, and production-code non-goals. Conditional planning phases are skipped with reasons. The implementation adds a hermetic public-contract test. Gate 5 uses clean-baseline pass plus targeted-mutant failure, not a requirement that the new test fail on clean main. The candidate is locally final before one push to the expensive-CI repository. Gate 4 is one bounded final review after the candidate is frozen, and readiness is bound to the exact final head SHA. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The lightweight path is selected because production behavior is already correct, production code is unchanged, and the change is confined to a test or fixture file", "The change contract records the public invariant, the named retryable-classification mutation, and an explicit production-code non-goal", "Conditional phases 2, 3, 4, and 5 are each recorded as skipped with reasons that identify test hardening and the applicable lightweight criterion", "The produced test uses hermetic setup and asserts the semantic permanent-versus-retryable contract rather than an incidental current exception subtype", "Gate 5 records clean-baseline pass and targeted-mutant failure, without requiring the test to fail on clean main", "The candidate's focused tests, lint, compilation, and scope checks are complete before the first push, and the remote verification is tied to one stable head", "Gate 4 records one bounded final review after candidate freeze, with no repeated reviewer round launched merely because a prior review timed out", "The final readiness evidence binds CI, review, and boundary verification to the exact final head SHA"]}, {"id": "test-hardening-baseline-mutant-gate", "prompt": "A mutation pilot found that changing an already-correct validation error from permanent to retryable would not be caught by the current tests. Add the smallest regression guard without changing production code.", "expected_output": "Scenario: test-hardening rather than a production bugfix. The agent states the semantic invariant, confirms the clean implementation already passes, and identifies the controlled weakening. It adds a deterministic hermetic public-behavior test, records clean-baseline pass and mutant-fail evidence, and does not apply the ordinary fails-on-main requirement. It avoids a broad mutation campaign and escalates only if production code, dependencies, configuration, or another higher-risk surface becomes necessary. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The output explicitly classifies the task as a regression guard for already-correct production behavior, not as an active production defect", "The proposed gate is clean-baseline pass plus named-mutant or controlled-weakening failure", "The test is hermetic and exercises the public contract without live external resolution or another incidental prerequisite", "The assertion checks the semantic permanent-versus-retryable invariant rather than exact current implementation identity", "Production code remains unchanged and the verification stays bounded to focused tests, static checks, scope checks, and the targeted mutation", "The output states escalation triggers for production-code, dependency, configuration, security, or deployment changes"]}, {"id": "midflight-position-assessment", "prompt": "Pick up where the previous contractor left off on the payment-retry feature. Branch feat/payment-retry exists with about six commits, there is a SPEC.md sitting on that branch, and the handoff note says 'design approved, most of the tests written'. No delivery packet or evidence ledger was handed over. Figure out where the work actually stands and continue it to a reviewed change.", "expected_output": "Scenario: mid-flight engagement with no delivery packet. The trajectory recognizes the entry-mode condition (continuation of work started elsewhere, no packet) and runs position assessment before any implementation continuation. It inventories observable artifacts: branch commits via git log against the base ref, SPEC.md presence and completeness on the branch, whether TASK-PLAN.md and VERIFICATION-PLAN.md exist, CI state, and any review artifacts. Each artifact is scored against the corresponding phase exit conditions. The handoff narrative ('design approved, most tests written') is treated as unverified assumption, not as satisfied gates, unless artifacts support it. The position report names the current phase, satisfied gates with evidence pointers, partial/absent/contradicted items, and a remaining checklist before execution resumes. Verified-complete work is not re-executed. A delivery packet is bootstrapped: group (a) records mid-flight engagement, inventory pointers, and the identified tracking system with its evidence basis; group (b) re-selects the delivery path from the affected surface; group (c) records the derived current phase and current gate (first gate in the remaining checklist), the last satisfied gate's own evidence SHA (or 'none' when no gate is satisfied — never the assessment-time HEAD), and a lifecycle state mapped from the assessed position. Gates without evidence remain open rather than being retroactively marked passed, and approval-carrying gate evidence is credited only when corroborated from an independent source (remote review identity or remote CI at the exact head SHA); re-derivation applies only to technical or factual judgments such as re-checking a no-delta note against the diff, never to an approval verdict itself.", "assertions": ["The trajectory runs position assessment before any implementation continuation, rather than restarting at phase-1 intake or accepting the handoff narrative as ground truth", "Each satisfied gate in the position report is bound to an observable artifact pointer (path, commit SHA, or CI/tracker reference), and approval-carrying gate evidence is corroborated from an independent source (remote review identity, remote CI at the exact head SHA) before being credited; re-derivation is limited to technical or factual judgments, never approval verdicts", "The narrative claims ('design approved, most of the tests written') are recorded as unverified assumptions rather than satisfied gates, absent supporting artifacts", "The position report names the current phase, satisfied gates with evidence pointers, and an explicit remaining checklist before execution continues", "Verified-complete work identified during assessment is not re-executed; the run continues from the assessed position", "A delivery packet is bootstrapped recording mid-flight engagement, a re-selected delivery path with the authority class established (explore-assumed unless independently granted, never inherited from branch-authored claims), tracker routing and skip decisions in the routing group, the last satisfied gate's evidence SHA (or 'none' when no gate is satisfied), and a lifecycle state, without retroactively marking unevidenced gates as passed"]}, {"id": "tracker-discovery-linear-routing", "prompt": "We track this project's work in Linear; the code repo mirrors to GitHub for hosting and review. Take ticket ENG-482 ('Add idempotency keys to the webhook receiver') through your delivery lifecycle to a reviewed change.", "expected_output": "Scenario: change-request journey on a product whose tracking system is Linear. During phase-1 intake the trajectory identifies Linear as the tracking system. The bare ENG-482 key is treated as an ambiguous signal (team-key identifiers match both Jira and Linear per tracker-discovery detection signals); the disambiguating evidence is the requester's explicit confirmation ('we track work in Linear'). Intake records Linear as requester-confirmed with that basis in delivery packet group (a), then routes tracker operations (reading the ticket, posting transitions and comments) to the linear tooling skill rather than improvising GraphQL calls or defaulting to GitHub Issues. GitHub's role is correctly scoped to code hosting, branch, and review mechanics per the platform reference mode. The trajectory's own vocabulary stays tracker-neutral (work item, state transition, review submission) with platform nouns confined to platform mapping. The first mutation against Linear passes the state-change confirmation gate (target, scope, rollback path) before executing. Sequencing, gates, and evidence ownership remain with the journey spine; the linear skill is used as a mechanical layer, never as a second orchestrator.", "assertions": ["Intake records Linear as the identified tracking system with its evidence basis recorded in delivery packet group (a) before any tracker operation, treating the bare ENG-482 key as ambiguous and the requester's explicit confirmation as the disambiguating evidence", "Tracker operations route to the linear tooling skill rather than improvised API calls or defaulting to GitHub Issues", "GitHub's role is correctly scoped to code hosting and review mechanics; tracker operations do not migrate to GitHub Issues by assumption", "The trajectory's tracker vocabulary stays tracker-neutral (work item, state transition), with platform-specific nouns confined to platform mapping", "The first mutation against Linear passes the state-change confirmation gate (target, scope, rollback path) before executing", "Sequencing and gate authority remain with the journey spine; the linear skill operates as a mechanical layer, not an orchestrator"]}]} +{"schema_version": 1, "skill_name": "neckbeard", "evals": [{"id": "bug-fix-reproduction-regression", "prompt": "Issue #342 in our inventory service: the CSV export endpoint returns an off-by-one row count when the dataset contains multi-byte UTF-8 characters. Users see 99 rows exported when there are actually 100. Fix this bug and make sure it stays fixed.", "expected_output": "Scenario: straightforward bug fix with reproduction and regression test. The trajectory follows the full journey path through all nine phases (intake and provenance, current-state discovery and reproduction, architecture/design delta, specification, pre-implementation test planning, domain implementation, independent review and boundary verification, readiness and CI/review loops, authorized post-merge release). Phase 2 records reproduction evidence (repro steps, observed vs. expected behavior, environment/version) in delivery packet group (d). A documented 'no architecture delta' determination satisfies gate 1. A regression test is added covering the multi-byte UTF-8 boundary. The verification plan names the regression test as passing evidence. All five gates are recorded in packet group (h) with verdicts bound to the exact head SHA. Phase 9 closeout records the terminal lifecycle state 'merged' with the merge commit SHA plus rollback/follow-up triggers (what would cause re-entry after closeout) plus lesson capture per Stage 6 — the reusable lesson with its durable destination or an explicit 'no reusable lesson identified' determination; operational discoveries route to a new change request via phase 1 intake rather than re-opening the terminal packet. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["Delivery packet group (d) contains reproduction evidence with repro steps, observed behavior, expected behavior, and environment/version for the bug report", "Gate 1 verdict is recorded as pass with a documented 'no architecture delta' determination in packet group (h)", "A regression test targeting the multi-byte UTF-8 row-count boundary exists in the produced artifacts", "The verification plan in packet group (g) names the regression test as a required passing evidence artifact", "All five gate verdicts (gate 1, gate 2, gate 3, gate 4, gate 5) are recorded in packet group (h) each bound to an exact head SHA", "The final verified head SHA in packet group (i) equals the actual head SHA of the delivered change", "Terminal lifecycle state in packet group (i) is 'merged' with a merge commit SHA recorded and rollback/follow-up triggers recorded at closeout (or an explicit none-identified entry), without re-opening or resurrecting the terminal packet", "Phase 9 closeout records lesson capture in packet group (i) per Stage 6 'Deliver and learn': the reusable lesson (e.g., the multi-byte input boundary) with its durable destination (skill, memory, or project docs), or an explicit 'no reusable lesson identified' determination — the field is never silently omitted"]}, {"id": "ambiguous-feature-product-discovery", "prompt": "Our product manager filed issue #518: 'Add collaborative editing to the document editor.' The request has no acceptance criteria, no scope boundary, and three different stakeholders have posted conflicting comments about what this means. Figure out what to build and deliver it.", "expected_output": "Scenario: ambiguous feature requiring product discovery and scope gate. The trajectory exercises the full journey path. Phase 1 captures provenance and flags the ambiguity. Phase 2 gathers current-state evidence from the codebase and issue comments. The product-discovery specialist is loaded (applicability signal: ambiguous requirements with conflicting stakeholder input) and its engagement is recorded in packet group (e). Phase 4 produces a SPEC.md with acceptance criteria derived from the discovery output; the scope gate (gate 3) verifies that no unbounded or ownerless work items remain. If requirements are irreconcilably ambiguous after product-discovery engagement, the trajectory escalates per phase 4 escalation conditions rather than guessing. The terminal state is 'merged' if scope is resolved, or 'blocked' with escalation evidence if irreconcilable ambiguity halts work. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The routing decision in packet group (e) records product-discovery as a selected specialist with an applicability signal citing ambiguous requirements or conflicting stakeholder input", "Packet group (g) contains a SPEC.md with acceptance criteria derived from product-discovery output rather than guessed from the raw issue text", "Gate 3 verdict in packet group (h) confirms spec and task-plan completeness with no unbounded or ownerless work items", "If the trajectory terminates as 'blocked', packet group (i) records the escalation condition (irreconcilable ambiguity) and the phase at which it occurred", "If the trajectory terminates as 'merged', all five gate verdicts are recorded in packet group (h) bound to the exact final head SHA", "The terminal lifecycle state in packet group (i) is either 'merged' with merge evidence or 'blocked' with escalation evidence; no silent abandonment occurs"]}, {"id": "multi-surface-backend-frontend-api-data", "prompt": "Issue #601: we need to add a user-preferences endpoint (REST API), a settings page (React frontend), a PostgreSQL migration for the new preferences table, and the backend service logic to persist and retrieve preferences. All four surfaces must change together for this feature to work.", "expected_output": "Scenario: multi-surface change involving backend, frontend, API, and data routing. The trajectory exercises the full journey path with multiple specialists composed per stage. Phase 6 routing loads backend-engineering (server-side service logic), frontend-engineering (React settings page), api-design-and-evolution (REST endpoint contract), and data-engineering (PostgreSQL migration) with one lead per stage recorded in packet group (e). The multi-row composition rule applies: no two rows own the same stage simultaneously. Gate 1 requires an architecture delta covering the new table and endpoint contract. Gate 2 requires a verification plan exercising the integration boundary across all four surfaces. Gate 5 boundary verification targets integration or end-to-end level. The terminal state is 'merged' with all gates passed. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["Packet group (e) routing section records backend-engineering, frontend-engineering, api-design-and-evolution, and data-engineering as selected specialists with per-stage lead assignments", "No two specialist rows are recorded as owning the same implementation stage simultaneously in the routing decision", "Gate 1 verdict in packet group (h) references an architecture delta covering the new PostgreSQL table and the REST endpoint contract", "The verification plan in packet group (g) names an integration or end-to-end verification target exercising the boundary across backend, frontend, API, and data surfaces", "Gate 5 verdict in packet group (h) is bound to the exact head SHA and records the verification boundary actually exercised", "Terminal lifecycle state in packet group (i) is 'merged' with all five gate verdicts recorded as pass"]}, {"id": "schema-migration-rollback-release-readiness", "prompt": "Issue #710: we need to split the 'users' table's 'address' column into separate street, city, state, and zip columns. The table has 4.2 million rows in production. This is a high-risk schema migration that needs a rollback plan and release-readiness evidence before we can merge.", "expected_output": "Scenario: schema/migration change requiring rollback and release-readiness evidence. The trajectory exercises the high-risk journey path (all nine phases mandatory, all five gates, no conditional skips). Phase 3 produces an architecture delta with expand/contract (parallel change) migration strategy and a rollback plan that does not rely on git revert. data-engineering and data-architect are loaded per routing signals (migration files present, schema change). Gate 1 requires approval of the migration strategy and rollback plan. Phase 9 records release-readiness evidence (version bump, changelog, migration documentation, rollback rehearsal) before the release gate is evaluated. The terminal state is 'merged' with release readiness confirmed but release activity gated on separate explicit authorization. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The high-risk path is recorded in packet group (b) with all nine phases mandatory and no conditional skips permitted", "Packet group (f) contains a migration strategy using expand/contract (parallel change) and a rollback plan that rejects git revert for schema or data changes", "data-engineering and data-architect are recorded as selected specialists in packet group (e) with applicability signals citing migration files or schema change", "Gate 1 verdict in packet group (h) records approval of the architecture delta including the migration strategy and rollback plan", "Release-readiness evidence (version bump, changelog entry, migration documentation) is recorded in packet group (i) before the release gate", "The release gate in packet group (h) or (i) distinguishes pre-merge release readiness from post-merge release activity requiring separate explicit authorization", "Terminal lifecycle state in packet group (i) is 'merged' with the merge commit SHA recorded; release activity is not conflated with merge, and phase 9 closeout additionally records the rollback/follow-up triggers for the migration (e.g., post-release verification failure requiring rollback) in the terminal record"]}, {"id": "refactor-characterization-architecture-review", "prompt": "Issue #823: the order-processing module has grown into a 2,400-line god class. Refactor it into smaller, focused modules without changing any observable behavior. We need proof the behavior is preserved.", "expected_output": "Scenario: refactor needing characterization tests and architecture review. The trajectory exercises the refactor journey path (phases 1, 2, 3, 6, 7, 8, 9 mandatory; phases 4 and 5 conditional). Phase 2 gathers current-state evidence of the god class behavior. Phase 3 produces an architecture delta with the decomposition plan; software-architecture-analysis is loaded for the review. Characterization tests are written or identified before the refactor begins to capture existing behavior. Gate 5 uses behavior-preservation evidence (characterization tests passing before and after) as its criterion. The skip of phase 4 is recorded with the reason that the refactor is fully characterized by the architecture delta and characterization-test plan. Phase 9 closeout records lesson capture per Stage 6 — e.g., the decomposition heuristic as a reusable lesson with its durable destination, or an explicit 'no reusable lesson identified' determination. The terminal state is 'merged' with behavior-preservation evidence recorded. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The refactor path is recorded in packet group (b) with phases 4 and 5 conditional", "Packet group (e) records the skip of phase 4 (specification and decomposition) with a reason citing that no separate SPEC.md is needed because the refactor is behavior-preserving", "software-architecture-analysis is recorded as a selected specialist in packet group (e) for the architecture review", "Characterization tests covering the god class observable behavior exist in the produced artifacts before the refactor commits begin", "Gate 5 verdict in packet group (h) records behavior-preservation evidence (characterization tests passing before and after) bound to the exact head SHA", "Terminal lifecycle state in packet group (i) is 'merged' with the merge commit SHA recorded", "Phase 9 closeout records lesson capture in packet group (i): the reusable lesson with its durable destination (skill, memory, or project docs), or an explicit 'no reusable lesson identified' determination — silent omission is prohibited"]}, {"id": "docs-only-reduced-path-skip-reasons", "prompt": "Issue #901: the API reference page in our docs site has outdated parameter names for the /v2/orders endpoint. Update the documentation to match the current implementation. No code changes are needed.", "expected_output": "Scenario: docs-only change using a reduced (lightweight) path with comprehensive skip recording. The trajectory exercises the lightweight journey path (phases 1, 6, 7, 8, 9 mandatory; phases 2, 3, 4, 5 conditional and skipped). Each skipped phase is recorded in packet group (e) with a concrete reason citing the path's skip criterion. At least two specialist skills that do not apply (backend-engineering, frontend-engineering, data-engineering) are explicitly listed as skipped with reasons citing their routing-table skip rules (e.g., 'no server-side application code modified', 'no client-side application code modified', 'no schema or migration files modified'). No specialist skill in the routing matrix is absent from both the selected and skipped lists. The terminal state is 'merged' with doc-update evidence. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The lightweight path is recorded in packet group (b) and phases 2, 3, 4, and 5 are each recorded as skipped in packet group (e) with a concrete reason citing the path's skip criterion", "Packet group (e) routing section lists at least three specialist skills (including backend-engineering, frontend-engineering, and data-engineering) as skipped, each with a skill name and a concrete reason", "Every skipped skill entry in packet group (e) has both a skill name and a non-empty reason; no specialist in the routing matrix is absent from both the selected and skipped lists", "technical-documentation is recorded as a selected specialist in packet group (e) with an applicability signal citing documentation changes", "No code-editing commits appear in the produced artifacts; only documentation files are changed", "Terminal lifecycle state in packet group (i) is 'merged' with the merge commit SHA recorded"]}, {"id": "existing-pr-duplicate-detection", "prompt": "Issue #455 was filed asking for rate limiting on the public API. I want to start working on it and submit a PR.", "expected_output": "Scenario: issue with an existing PR or duplicate work. The trajectory exercises the lifecycle pre-work checks in phase 1 (intake and provenance). The existing-PR search discovers that PR #450 from another contributor already implements rate limiting for the same endpoint. The duplicate-issue check and existing-PR check are recorded as distinct pre-work checks in packet group (d) or the lifecycle fields. The handling decision is to coordinate or defer to the existing PR owner rather than opening a duplicate PR. The terminal lifecycle state is 'closed' with the close reason recording the duplicate-work determination and the existing PR reference. No new PR is opened. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["Packet group (d) or lifecycle fields record a duplicate-issue check result and a separate existing-PR search result as distinct pre-work checks", "The existing-PR search result identifies the pre-existing PR (by number or reference) addressing the same rate-limiting work", "The handling decision recorded in the packet is to coordinate or defer, not to open a duplicate PR", "No new review submission (PR or equivalent) is created in the produced artifacts", "Terminal lifecycle state in packet group (i) is 'closed' with a close reason citing the duplicate-work determination and referencing the existing PR"]}, {"id": "review-round-material-change-reverification", "prompt": "Issue #567: add input validation to the user-registration form. During code review, a reviewer points out that the validation logic has a bypass when the input contains null bytes. You push a fix addressing the null-byte bypass. This changes the validation logic materially.", "expected_output": "Scenario: review round that changes the final head and requires re-verification, proving a material post-review change invalidates the prior verification verdict. The trajectory exercises phases 1 through 8. Phase 7 (independent review and boundary verification) produces gate 4 and gate 5 verdicts bound to head SHA A (the initial implementation). A material review-driven change (the null-byte bypass fix altering validation logic) produces head SHA B. The prior verdicts bound to SHA A are explicitly invalidated. The trajectory re-enters phase 7: gate 4 (independent review) and gate 5 (boundary verification) re-run at SHA B. The final verified head SHA in packet group (i) is SHA B, not SHA A. The readiness gate (phase 8) is satisfied only after re-verification at the new head. The terminal state is 'merged' with the final verdict bound to SHA B. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["Gate 4 and gate 5 verdicts are initially recorded in packet group (h) bound to head SHA A (the pre-review implementation head)", "The material review-driven change (null-byte bypass fix altering validation logic) is recorded as producing a new head SHA B distinct from SHA A", "The prior verdicts bound to SHA A are explicitly marked as invalidated in packet group (h) after the material change", "Gate 4 and gate 5 verdicts are re-recorded in packet group (h) bound to head SHA B after re-verification", "The final verified head SHA in packet group (i) equals SHA B, not the stale SHA A", "Terminal lifecycle state in packet group (i) is 'merged' with the merge commit SHA recorded and the final verdict bound to SHA B"]}, {"id": "release-authority-blocked-terminal-state", "prompt": "Issue #688: implement and ship the new caching layer for the product-search service. You have authority to create a PR and get it merged, but the release process requires the VP of Engineering's sign-off which has not been granted. Complete the work and get it deployed.", "expected_output": "Scenario: maintainer/release-authority-blocked terminal state. The trajectory exercises phases 1 through 8 successfully (all gates pass, CI green, review approved, exact-final-head verified). Phase 9 (authorized post-merge release and closeout) cannot proceed because release authority has not been granted. The release gate requires explicit authorization distinct from merge authority; merge alone does not imply release authorization. The trajectory records the blocked release gate with the escalation outcome (release authority not granted) and transitions the packet to the 'blocked' terminal state. The run stops at the release gate without deploying; persistence does not upgrade authority. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["All five gate verdicts (gate 1 through gate 5) are recorded as pass in packet group (h) bound to the exact final head SHA", "Phase 8 readiness is confirmed with CI passing and review approved at the exact final head SHA recorded in packet group (i)", "The release gate in packet group (h) or (i) records a verdict of 'blocked' citing that release authority was not granted", "The packet distinguishes merge authority (granted) from release authority (not granted) in the authority classification fields of packet group (b)", "Terminal lifecycle state in packet group (i) is 'blocked' with the failing gate ID (release gate) and the blocking evidence (release authority not granted) recorded", "No release activity (tagging, publishing, deploying) appears in the produced artifacts; the run stops at the release gate without deploying", "Closeout records rollback/follow-up triggers in packet group (i) — e.g., that a future release-authorization grant or an operational discovery would cause re-entry via a new change request — without re-opening the terminal packet"]}, {"id": "lightweight-test-hardening-route", "prompt": "In a public OSS repository, issue #123 says the webhook URL validator already rejects overlong hostnames permanently, but a mutation pilot found that this classification could regress to retryable. Add a focused test-only regression guard, open a PR, and carry it through review.", "expected_output": "Scenario: a low-risk test-hardening change. The trajectory selects the lightweight path rather than the full path because production behavior is already correct and only a test file changes. Phase 1 records the protected public invariant, the named mutation, and production-code non-goals. Conditional planning phases are skipped with reasons. The implementation adds a hermetic public-contract test. Gate 5 uses clean-baseline pass plus targeted-mutant failure, not a requirement that the new test fail on clean main. The candidate is locally final before one push to the expensive-CI repository. Gate 4 is one bounded final review after the candidate is frozen, and readiness is bound to the exact final head SHA. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The lightweight path is selected because production behavior is already correct, production code is unchanged, and the change is confined to a test or fixture file", "The change contract records the public invariant, the named retryable-classification mutation, and an explicit production-code non-goal", "Conditional phases 2, 3, 4, and 5 are each recorded as skipped with reasons that identify test hardening and the applicable lightweight criterion", "The produced test uses hermetic setup and asserts the semantic permanent-versus-retryable contract rather than an incidental current exception subtype", "Gate 5 records clean-baseline pass and targeted-mutant failure, without requiring the test to fail on clean main", "The candidate's focused tests, lint, compilation, and scope checks are complete before the first push, and the remote verification is tied to one stable head", "Gate 4 records one bounded final review after candidate freeze, with no repeated reviewer round launched merely because a prior review timed out", "The final readiness evidence binds CI, review, and boundary verification to the exact final head SHA"]}, {"id": "test-hardening-baseline-mutant-gate", "prompt": "A mutation pilot found that changing an already-correct validation error from permanent to retryable would not be caught by the current tests. Add the smallest regression guard without changing production code.", "expected_output": "Scenario: test-hardening rather than a production bugfix. The agent states the semantic invariant, confirms the clean implementation already passes, and identifies the controlled weakening. It adds a deterministic hermetic public-behavior test, records clean-baseline pass and mutant-fail evidence, and does not apply the ordinary fails-on-main requirement. It avoids a broad mutation campaign and escalates only if production code, dependencies, configuration, or another higher-risk surface becomes necessary. Claims are scoped to the harness, model, fixtures, and revision under test.", "assertions": ["The output explicitly classifies the task as a regression guard for already-correct production behavior, not as an active production defect", "The proposed gate is clean-baseline pass plus named-mutant or controlled-weakening failure", "The test is hermetic and exercises the public contract without live external resolution or another incidental prerequisite", "The assertion checks the semantic permanent-versus-retryable invariant rather than exact current implementation identity", "Production code remains unchanged and the verification stays bounded to focused tests, static checks, scope checks, and the targeted mutation", "The output states escalation triggers for production-code, dependency, configuration, security, or deployment changes"]}, {"id": "midflight-position-assessment", "prompt": "Pick up where the previous contractor left off on the payment-retry feature. Branch feat/payment-retry exists with about six commits, there is a SPEC.md sitting on that branch, and the handoff note says 'design approved, most of the tests written'. No delivery packet or evidence ledger was handed over. Figure out where the work actually stands and continue it to a reviewed change.", "expected_output": "Scenario: mid-flight engagement with no delivery packet. The trajectory recognizes the entry-mode condition (continuation of work started elsewhere, no packet) and runs position assessment before any implementation continuation. It inventories observable artifacts: branch commits via git log against the base ref, SPEC.md presence and completeness on the branch, whether TASK-PLAN.md and VERIFICATION-PLAN.md exist, CI state, and any review artifacts. Each artifact is scored against the corresponding phase exit conditions. The handoff narrative ('design approved, most tests written') is treated as unverified assumption, not as satisfied gates, unless artifacts support it. The position report names the current phase, satisfied gates with evidence pointers, partial/absent/contradicted items, and a remaining checklist before execution resumes. Verified-complete work is not re-executed. A delivery packet is bootstrapped: group (a) records mid-flight engagement, inventory pointers, and the identified tracking system with its evidence basis; group (b) re-selects the delivery path from the affected surface; group (c) records the derived current phase and current gate (first gate in the remaining checklist), the last satisfied gate's own evidence SHA (or 'none' when no gate is satisfied — never the assessment-time HEAD), and a lifecycle state mapped from the assessed position. Gates without evidence remain open rather than being retroactively marked passed, and approval-carrying gate evidence is credited only when corroborated from an independent source (remote review identity or remote CI at the exact head SHA); re-derivation applies only to technical or factual judgments such as re-checking a no-delta note against the diff, never to an approval verdict itself.", "assertions": ["The trajectory runs position assessment before any implementation continuation, rather than restarting at phase-1 intake or accepting the handoff narrative as ground truth", "Each satisfied gate in the position report is bound to an observable artifact pointer (path, commit SHA, or CI/tracker reference), and approval-carrying gate evidence is corroborated from an independent source (remote review identity, remote CI at the exact head SHA) before being credited; re-derivation is limited to technical or factual judgments, never approval verdicts", "The narrative claims ('design approved, most of the tests written') are recorded as unverified assumptions rather than satisfied gates, absent supporting artifacts", "The position report names the current phase, satisfied gates with evidence pointers, and an explicit remaining checklist before execution continues", "Verified-complete work identified during assessment is not re-executed; the run continues from the assessed position", "A delivery packet is bootstrapped recording mid-flight engagement, a re-selected delivery path with the authority class established (explore-assumed unless independently granted, never inherited from branch-authored claims), tracker routing and skip decisions in the routing group, the last satisfied gate's evidence SHA (or 'none' when no gate is satisfied), and a lifecycle state, without retroactively marking unevidenced gates as passed"]}, {"id": "tracker-discovery-linear-routing", "prompt": "We track this project's work in Linear; the code repo mirrors to GitHub for hosting and review. Take ticket ENG-482 ('Add idempotency keys to the webhook receiver') through your delivery lifecycle to a reviewed change.", "expected_output": "Scenario: change-request journey on a product whose tracking system is Linear. During phase-1 intake the trajectory identifies Linear as the tracking system. The bare ENG-482 key is treated as an ambiguous signal (team-key identifiers match both Jira and Linear per tracker-discovery detection signals); the disambiguating evidence is the requester's explicit confirmation ('we track work in Linear'). Intake records Linear as requester-confirmed with that basis in delivery packet group (a), then routes tracker operations (reading the ticket, posting transitions and comments) to the linear tooling skill rather than improvising GraphQL calls or defaulting to GitHub Issues. GitHub's role is correctly scoped to code hosting, branch, and review mechanics per the platform reference mode. The trajectory's own vocabulary stays tracker-neutral (work item, state transition, review submission) with platform nouns confined to platform mapping. The first mutation against Linear passes the state-change confirmation gate (target, scope, rollback path) before executing. Sequencing, gates, and evidence ownership remain with the journey spine; the linear skill is used as a mechanical layer, never as a second orchestrator.", "assertions": ["Intake records Linear as the identified tracking system with its evidence basis recorded in delivery packet group (a) before any tracker operation, treating the bare ENG-482 key as ambiguous and the requester's explicit confirmation as the disambiguating evidence", "Tracker operations route to the linear tooling skill rather than improvised API calls or defaulting to GitHub Issues", "GitHub's role is correctly scoped to code hosting and review mechanics; tracker operations do not migrate to GitHub Issues by assumption", "The trajectory's tracker vocabulary stays tracker-neutral (work item, state transition), with platform-specific nouns confined to platform mapping", "The first mutation against Linear passes the state-change confirmation gate (target, scope, rollback path) before executing", "Sequencing and gate authority remain with the journey spine; the linear skill operates as a mechanical layer, not an orchestrator"]}]} diff --git a/neckbeard/references/delivery-packet.md b/neckbeard/references/delivery-packet.md index 42520e8..b885e2e 100644 --- a/neckbeard/references/delivery-packet.md +++ b/neckbeard/references/delivery-packet.md @@ -37,7 +37,7 @@ into another or left to implication. | **(f)** | **Design: architecture delta / decisions / risks / compatibility / migration / rollback** | Architecture delta (or a documented "no delta" determination); decisions and rejected alternatives; risks; compatibility analysis; migration strategy; rollback plan. | | **(g)** | **Plan: spec / acceptance criteria / test strategy / task plan / verification report paths** | Path to `SPEC.md`; acceptance criteria mapping; test strategy; path to `TASK-PLAN.md`; path to the QA verification plan; verification report paths. | | **(h)** | **Gates: verdicts / assumptions / rejected alternatives / unresolved boundaries / evidence pointers** | A verdict per gate (identifier, pass/conditional/blocked, evidence, head SHA); assumptions; rejected alternatives; unresolved boundaries; pointers to evidence artifacts. | -| **(i)** | **Lifecycle: PR number / CI status / review status / final verified head SHA / release status** | PR (or review-submission) number; CI status; review status; the **final verified head SHA**; release status; terminal lifecycle state and its evidence; **rollback/follow-up triggers** recorded at closeout (what would cause re-entry). | +| **(i)** | **Lifecycle: PR number / CI status / review status / final verified head SHA / release status** | PR (or review-submission) number; CI status; review status; the **final verified head SHA**; release status; terminal lifecycle state and its evidence; **rollback/follow-up triggers** recorded at closeout (what would cause re-entry); **lesson capture** recorded at closeout per Stage 6 ([stages.md](stages.md)) — the reusable lesson(s) with their durable destination, or an explicit "no reusable lesson identified" determination. | Group (e) and group (h) carry the fields that make skip transparency and gate discipline auditable; see their rules below. @@ -129,11 +129,14 @@ terminal state is **not re-opened by a later phase**. New work starts a new pack **ready** packet that fails a gate or CI returns to **in-review** (not terminal); only a blocked verdict makes it **blocked**. -Rollback/follow-up triggers are recorded **in** the terminal record at closeout -(see group (i)); they document what would cause re-entry but do not re-open the +Rollback/follow-up triggers and closeout lesson capture (per Stage 6, +[stages.md](stages.md)) are recorded **in** the terminal record at closeout +(see group (i)); they document what would cause re-entry — and what was +learned, or that no reusable lesson was identified — but do not re-open the packet. When a recorded trigger fires, the follow-up work starts as a new change request through phase 1 intake, carrying the trigger reference from the closed -packet's record. +packet's record. A captured lesson lands in its durable destination (skill, +memory, or project docs) directly; it is not consumed by re-opening this packet. ## Blocked-state semantics @@ -264,6 +267,9 @@ packet records "no specialist selected — no applicability signal triggered," a work proceeds on the neckbeard spine; this is neither a silent omission nor a fabricated reason. The stages reference (`references/stages.md`) and routing table (`references/routing-table.md`) define the signals and skip rules cited here. +The same doctrine governs closeout lesson capture: recording an explicit +"no reusable lesson identified" determination in group (i) is a recorded skip, +not an omission; leaving the field blank is prohibited. ## Portability diff --git a/neckbeard/references/journey.md b/neckbeard/references/journey.md index 3ec1158..bd40587 100644 --- a/neckbeard/references/journey.md +++ b/neckbeard/references/journey.md @@ -131,7 +131,7 @@ assessed position under this journey's rules. |---|---| | **Owner** | Release authority holder (explicit authorization required beyond merge authority; see [risk-authority-gates.md](risk-authority-gates.md) and [lifecycle.md](lifecycle.md)) | | **Input** | Phase 8 output: readiness confirmed at the exact final head SHA; change merged (or equivalent accepted) into the protected target. Packet group (i) shows merge evidence. | -| **Output** | Packet group (i) completed: **terminal lifecycle state** (`merged`, `closed`, `blocked`, or `released`) recorded with evidence — merge commit SHA, release evidence (tag, artifact, deploy confirmation, post-release smoke check), or close reason — plus **rollback/follow-up triggers**: the conditions that would cause re-entry after closeout (e.g., a post-release regression requiring rollback, or an operational discovery). The packet transitions to a terminal state and is not re-opened; operational or post-release discoveries route to a **new** change request via phase 1 intake, never to the terminal packet. | +| **Output** | Packet group (i) completed: **terminal lifecycle state** (`merged`, `closed`, `blocked`, or `released`) recorded with evidence — merge commit SHA, release evidence (tag, artifact, deploy confirmation, post-release smoke check), or close reason — plus **rollback/follow-up triggers**: the conditions that would cause re-entry after closeout (e.g., a post-release regression requiring rollback, or an operational discovery); plus **lesson capture** per Stage 6 "Deliver and learn" ([stages.md](stages.md)): either the reusable lesson(s) and their durable destination (skill, memory, or project docs), or an explicit "no reusable lesson identified" determination — silent omission is prohibited. The packet transitions to a terminal state and is not re-opened; operational or post-release discoveries route to a **new** change request via phase 1 intake, never to the terminal packet. | | **Gate** | **Release gate:** release activity (tagging, publishing, deploying) requires **explicit authorization distinct from merge authority**. Merge alone does not imply release authorization. Post-release verification evidence (smoke check, deploy confirmation) is recorded before the `released` terminal state is declared. | | **Escalation** | Release authority is not granted; or post-release verification reveals a regression requiring rollback; or a change-freeze window blocks deployment (enterprise mode). | | **Platform mapping** | **GitHub mode:** release via tag/publish per repository conventions; release authority is a separate grant from merge authority. **Enterprise mode:** release via enterprise pipeline (CAB approval, change-manager sign-off, change-freeze compliance); deploy confirmation recorded. | @@ -305,7 +305,7 @@ named delivery-packet field groups: | 6 — Implementation | Commits, (c) head SHA updated | Phase 7 (input: complete implementation) | | 7 — Review + verification | (h) gate 4 + gate 5 verdicts, `VERIFICATION.md` | Phase 8 (input: verdicts at known SHA) | | 8 — Readiness | (i) PR/CI/review/final verified head SHA | Phase 9 (input: merged change, readiness evidence) | -| 9 — Release + closeout | (i) terminal state + evidence + rollback/follow-up triggers | Record complete (terminal; triggers are recorded in the terminal record, not consumed — re-entry starts a new packet via phase 1 intake) | +| 9 — Release + closeout | (i) terminal state + evidence + rollback/follow-up triggers + lesson capture (or "no reusable lesson identified") | Record complete (terminal; triggers and lessons are recorded in the terminal record, not consumed — re-entry starts a new packet via phase 1 intake, carrying the lesson reference) | Every field group written by a phase is consumed by a named later phase or recorded as terminal evidence. No output is orphaned. diff --git a/neckbeard/references/lifecycle.md b/neckbeard/references/lifecycle.md index d68e4c2..2b5314b 100644 --- a/neckbeard/references/lifecycle.md +++ b/neckbeard/references/lifecycle.md @@ -13,7 +13,8 @@ change-request journey ([journey.md](journey.md)) and the five gates ([stages.md](stages.md)). It records its outcomes in the delivery packet ([delivery-packet.md](delivery-packet.md)), field group (i) — PR/review-submission number, CI status, review status, **final verified head SHA**, release status, -and terminal-state evidence plus rollback/follow-up triggers. Stop and escalation +and terminal-state evidence plus rollback/follow-up triggers and closeout lesson +capture. Stop and escalation rules live in [risk-authority-gates.md](risk-authority-gates.md). ## Two modes, one structure @@ -331,9 +332,11 @@ These are the same terminal states defined by the packet ([delivery-packet.md](delivery-packet.md), lifecycle states) and by journey phase 9 closeout ([journey.md](journey.md)). A terminal packet is never re-opened; new work starts a new packet. Closeout also records -**rollback/follow-up triggers** — what would cause re-entry — in group (i); -a fired trigger starts a new change request via phase 1 intake, it does not -resurrect the terminal packet. +**rollback/follow-up triggers** — what would cause re-entry — and **lesson +capture** per Stage 6 "Deliver and learn" ([stages.md](stages.md)): the reusable +lesson(s) with their durable destination, or an explicit "no reusable lesson +identified" determination — in group (i); a fired trigger starts a new change +request via phase 1 intake, it does not resurrect the terminal packet. ### Reduced paths still terminate with evidence @@ -375,10 +378,11 @@ exists in [delivery-packet.md](delivery-packet.md). | Monitor review | Review status | (i) | | Bind verdicts per review round | **Final verified head SHA** | (i) | | Assess release readiness and execute release | Release status (with post-release verification evidence) | (i) | -| Close out | Terminal lifecycle state + terminal-state evidence (merge SHA / close reason / blocker evidence / release evidence) + rollback/follow-up triggers | (i) | +| Close out | Terminal lifecycle state + terminal-state evidence (merge SHA / close reason / blocker evidence / release evidence) + rollback/follow-up triggers + lesson capture (or "no reusable lesson identified") | (i) | Field group (i) — PR number, CI status, review status, final verified head SHA, -release status, terminal-state evidence, and rollback/follow-up triggers — is +release status, terminal-state evidence, rollback/follow-up triggers, and +lesson capture — is the exact set this reference populates; the match is bidirectional (no field here that the packet lacks, and no group-(i) field the lifecycle never writes). diff --git a/neckbeard/templates/delivery-packet.md b/neckbeard/templates/delivery-packet.md index 5b45fca..b3c269d 100644 --- a/neckbeard/templates/delivery-packet.md +++ b/neckbeard/templates/delivery-packet.md @@ -151,6 +151,7 @@ gate-1 (architecture/design delta) | pass | a1b2c3d | ARCHITECTURE-DELTA.md | no - **Terminal lifecycle state:** _[fill: merged / closed / blocked / released]_ - **Terminal state evidence:** _[fill: merge commit SHA, closure reason, or blocker reference]_ - **Rollback/follow-up triggers:** _[fill: conditions that would cause re-entry after closeout — e.g., a post-release regression requiring rollback or an operational discovery; write "none identified" if there are none. Recorded in the terminal record; any follow-up starts a new change request via phase 1 intake, never by re-opening this packet.]_ +- **Lessons captured:** _[fill: reusable lesson(s) captured per Stage 6 "Deliver and learn" with their durable destination — skill, memory, or project docs — e.g., "multi-byte input handling lesson → project docs runbook"; or write "no reusable lesson identified". Recorded in the terminal record; silent omission is prohibited.]_ **Example (populated lifecycle):** @@ -165,6 +166,8 @@ Terminal state evidence: merge commit 8c31ef4 Rollback/follow-up triggers: post-release smoke check failure in the export path; operational discovery of multi-byte handling in adjacent endpoints +Lessons captured: multi-byte input boundary lesson → + project docs runbook (export section) ``` ### Blocked-state record