* feat(validation): enforce 60K-char cap on skill reference files
Implements issue #277:
- validate-references.rb: new oversized_reference_errors check — every
references/*.md must be <= 60,000 characters; error reports path, size,
and the split-and-reindex remediation; wired into validate-skills.rb
- test-validate-skills.rb: 5 fixture tests (under-limit passes, over-limit
fails with path+size, exactly-at-limit passes, remediation message,
non-.md ignored); the suite now runs in validate.yml after the format
check (it was previously untested in CI)
- Docs: agent-skills/SKILL.md, agent-skills/references/best-practices.md,
and the AGENTS.md Format Compliance table document the cap and the
split-and-reindex procedure
- Compliance: split remote-systems-administration/references/ansible.md
and programming-principles/references/refactoring-guru.full.md into an
index + focused parts (content moved verbatim); SKILL.md routing,
README, and source-index references updated; pre-existing stale
refactoring-guru-smells.md reference repointed to the index
- Fix pre-existing quality-gate violations in the programming-principles
and remote-systems-administration descriptions (imperative verb +
negative boundary) so this PR's CI quality step passes; regenerated
llms.txt and marketplace artifacts
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
* test(evals): add eval manifests to modified skills for ratchet
The eval-coverage ratchet requires schema-valid eval manifests for any
skill modified once coverage is past 50%. This PR modifies
programming-principles and remote-systems-administration (splitting
their oversized references), so add evals/evals.json to both:
- programming-principles: 6 output-quality cases (task-to-book mapping,
principled code review, refactor-vs-rewrite, no-op detection, rule
distillation, principle conflicts)
- remote-systems-administration: 6 output-quality cases (discovery
before change, smallest control plane, rollback planning, platform
identification, verification evidence, escalation on missing
authority)
Coverage: 87/145 (60.0%) schema-valid; ratchet clean.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
---------
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-08-04 22:39:14 -04:00
Magnus HedemarkGitHubfactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
* feat(bundles): add bundle manifest schema, manifests, and validation (#203)
Introduce a machine-readable composition contract for canonical bundles:
purpose, audience, stages, included skills, prerequisites, outputs,
handoffs, conflicts, and eval suite (schemas/bundle-manifest-v1.schema.json,
following the evals-v1 versioned-schema convention). Ship the bounded design
note (docs/bundle-manifest-design.md), a schema-conformant example, canonical
manifests for the three new milestone bundles, and a stdlib-only validator
(scripts/validate-bundles.rb) that rejects incomplete, contradictory, and
undeclared-overlapping manifests while keeping bundles an optional layer.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
* feat(bundles): add lifecycle capability matrix generator and validators (#203)
Add scripts/gen-lifecycle-matrix.rb, which deterministically produces the
human-readable docs/lifecycle-capability-matrix.md (one row per canonical
bundle) and the machine-readable docs/lifecycle-capability-matrix.json (with
per-cell source provenance) reusing the gen-*.rb conventions. Add
scripts/validate-lifecycle-matrix.rb to check bundle coverage, cell
traceability, artifact currency, and catalog-exactness of nested bundle
helpers.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
* test(bundles): add bundle manifest validation tests (#203)
Add scripts/test-validate-bundles.rb covering schema conformance of the
committed example, valid-manifest and declared-conflict positives, per-field
incomplete-manifest rejections, contradictory-manifest rejections (missing
skill, undeclared handoff artifact, non-catalog conflict), undeclared-overlap
rejection naming both manifests, and matrix generator/validator
completeness and drift detection.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
* ci(bundles): wire bundle manifest validation into the gate (#203)
Add validate-bundles.rb, test-validate-bundles.rb, the lifecycle matrix
generator check, and the matrix validator to .github/workflows/validate.yml
alongside the existing validator steps.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
---------
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-08-02 19:48:09 -04:00
Magnus HedemarkGitHubusername <username>factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
* fix(product-strategy): correct stale RICE reference
Fix the misspelled prose reference at product-strategy/references/product-strategy.md:69
to point at the canonical rice-framework.md owned by product-methodology. Add a
repository check that scans references/*.md for stale prose backtick references
to nonexistent files (the bug class the SKILL.md link-resolution pass cannot
see), wired into validate-skills.rb, with a regression test suite proving the
stale reference is caught when reintroduced. product-strategy and
product-methodology remain grandfathered; no evals manifests are added.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
* fix(validate-references): case-sensitive resolution for stale-reference scan
The references scan resolved backtick tokens case-insensitively on hosts with
case-insensitive filesystems (default macOS APFS), so a token such as
`EVIDENCE-LEDGER.md` matched an existing lowercase `evidence-ledger.md` and
escaped detection locally while failing CI's Linux runners. Resolve candidates
against exact directory entries so results match CI on every host, and treat
neckbeard delivery-packet field names (EVIDENCE-LEDGER, DELIVERY-SPEC, REVIEW,
V2-SPEC) as doc-type names rather than file references. Adds regression tests
for case-mismatched and exact-case references.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
---------
Co-authored-by: username <username>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
When unittest discover processes skills/binary-analysis/tests/ before
tests/, the integration package gets cached in sys.modules from the
first discover call. Later discover calls reuse the cached package
which doesn't contain the expected test modules, causing spurious
ModuleNotFoundError failures.
Fix: snapshot and restore sys.modules between discover iterations.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Wire the Phase 3 ratchet into CI by passing the PR base SHA to
eval-coverage.py --modified-from. Expand changed-skill detection from
SKILL.md-only diffs to the entire skill directory so that references,
scripts, fixtures, README, and eval manifest edits all count as
modifications. Add a monotonic coverage floor that fails CI when
coverage decreases between the base and candidate revisions.
Add script tests for ratchet-mode detection and coverage-decrease
behaviour. Update AGENTS.md and CONTRIBUTING.md to describe the
behaviour CI now enforces.
Closes#102
Signed-off-by: Magnus Hedemark <magnus919@pm.me>
Phase 1: New skills (not in grandfathered-skills.txt) must have
evals/evals.json with at least 5 test cases. All 107 existing skills
are grandfathered.
Phase 2: scripts/eval-coverage.py reports coverage (skills with/without
evals, case counts, reference-priority sorting). Added as informational
CI step.
Phase 3: Ratchet thresholds — at 25% coverage, modified skills without
evals get a warning; at 50%, they fail CI. Enforced via
--modified-from flag for PR-scoped checks.
Closes#90
Generate a root discovery catalog from public skill frontmatter and fail CI
when the committed index drifts. Add a fixture-based regression test for
bundle paths, nested-helper exclusion, normalized descriptions,
deterministic ordering, and stale-file recovery.
Closes#78
AI-assisted: yes (Jasper/Hermes Agent)
Co-authored-by: magnus919 <magnus919>
The generator used File.basename which stripped the bundles/ prefix,
emitting ./neckbeard instead of ./bundles/neckbeard. Codex discovered
92/96 skills — the 4 bundle entrypoints were missing because their
paths didn't resolve.
Verified with live Codex CLI: all 96 skills now discoverable.
AI-assisted: yes (Jasper/Hermes Agent)
* feat: add Codex plugin packaging (single-plugin, metadata-only)
Adds .codex-plugin/plugin.json with a skills array listing all public
skills, plus .agents/plugins/marketplace.json for one-command install:
codex plugin marketplace add magnus919/agent-skills
codex plugin install magnus919
Same pattern as mattpocock/skills — one plugin, explicit skill paths,
no dist/, no curation, no duplication. Bundle-internal helpers excluded
by the shared glob. CI check mode fails if the manifest drifts.
Closes#79
AI-assisted: yes (Jasper/Hermes Agent)
* chore: trigger CI
* chore: regenerate Codex plugin manifest to include neckbeard bundle
Adds .claude-plugin/marketplace.json exposing all 95 public skills as
installable plugins via /plugin marketplace add magnus919/agent-skills.
Metadata-only approach: each entry uses source './' + skills ['./<name>']
+ strict:false, so no per-skill plugin.json or directory restructuring is
needed. Bundle-internal helper skills are excluded; bundle entrypoints are
included.
- scripts/gen-claude-marketplace.rb: generates and validates the manifest
- CI step fails if marketplace.json drifts from the skill tree
- README: Claude Code install instructions
Closes#76
AI-assisted: yes (Jasper/Hermes Agent)
usernameandfactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Jasper (AI Assistant)