Pin the epub skill's test-only dependencies (EbookLib, beautifulsoup4) with
exact versions in requirements-epub-test.txt and install from that file in
CI, instead of unpinned package names. The file is test-only and notes that
EbookLib is AGPL; requirements-dev.txt is untouched.
Also surface the epublib skip in epub/scripts/test_epub_skill.sh: when
epublib is not importable (it requires Python 3.13+), epub-edit and
epub-convert now print an explicit SKIPPED line and count as SKIP in the
summary instead of silently passing.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-08-14 23:42:25 -04:00
Magnus Hedemarkandfactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Install the epub skill's documented test dependencies (EbookLib and
beautifulsoup4) on the runner just before the shell-test step, and move
epub/scripts/test_epub_skill.sh from the manual registry into the run
registry. EbookLib is AGPL-licensed, so the install step is named and
scoped as test-only and does not touch requirements-dev.txt.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-08-14 23:22:29 -04:00
Magnus Hedemarkandfactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Add scripts/check-skill-tests.py as the single source of truth for
skill-local shell tests. --run executes the registered shell tests in CI;
--check (default) fails on any unregistered skill test file, stale registry
entry, or registry inconsistency, so new un-run shell tests cannot slip in.
Register the 9 deterministic shell suites as run entries and the 4 suites
that need network, credentials, or third-party libraries not installed by
CI as manual entries.
Wire both modes into validate.yml right after the skill-local pytest step
and run the new guardrail tests in the existing pytest invocations.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-08-14 22:41:53 -04:00
Magnus Hedemarkandfactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
test_rejects_duplicate_catalog_names failed on case-sensitive CI
filesystems (Ubuntu ext4): the flat-layout fixture wrote both duplicate/
and Duplicate/ with frontmatter name: duplicate, so Duplicate/SKILL.md
tripped the name-match guard ('name must match directory name
"Duplicate"') before the duplicate-catalog check ran, hiding the
expected 'duplicate catalog name "duplicate"' error. Local macOS APFS
collapsed the two dirs and passed, concealing the defect.
Detect filesystem case-sensitivity with a Probe/probe probe. On
case-sensitive filesystems write duplicate/ (name: duplicate) and
Duplicate/ (name: Duplicate) so both pass the name-match guard and the
downcased-name collision fires the duplicate-catalog error. On
case-insensitive filesystems the dirs collapse into one, so write the
single dir with name: duplicate and assert the generator succeeds.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-08-14 17:44:11 -04:00
Magnus Hedemarkandfactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Rename the nested-skill fixture paths in test_nested_change_maps_to_nearest_skill_owner
from bundles/example to example so no tracked file (other than the OKF upstream
sample file) references the removed bundles/ directory, satisfying the final
repo-wide sweep (VAL-REF-017).
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-08-14 16:55:35 -04:00
Magnus Hedemarkandfactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Re-point promise-theory reference links to ../../workflow-architect/SKILL.md
(one level deeper than SKILL.md), drop the bundles/ prefix from the
pace-plan evidence-ledger path, and refresh the stale eval-coverage.py
comment to describe only the flat <root>/<skill-name>/SKILL.md shape.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-08-14 16:50:06 -04:00
Magnus Hedemarkandfactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Author schema-valid evals/evals.json manifests (>=5 cases each) for the
tailscale sub-skills headscale-backup, headscale-deploy, headscale-derp,
headscale-node-lifecycle, headscale-routing, tailnet-policy, tailscale-client
and the workflow-architect sub-skills bundle-builder, interviewer, observer,
raising schema-valid coverage from 96/152 (63.2%) to 106/152 (69.7%). Remove
all 14 bundles/ entries from scripts/grandfathered-skills.txt now that the
8 umbrellas and 10 sub-skills all carry valid eval manifests.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-08-14 16:23:11 -04:00
Magnus Hedemarkandfactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Update validate-skills.rb expected_catalog_paths to the flat */SKILL.md
glob (drop the bundles/ term), repoint the 8 README catalog headings to
<name>/SKILL.md, and fix the promise-theory and semantic-spacetime
workflow-architect links to ../workflow-architect/SKILL.md.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-08-14 16:07:01 -04:00
Magnus Hedemarkandfactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Update gen-lifecycle-matrix.rb and validate-lifecycle-matrix.rb to drop all
hardcoded bundles/ path references after the directory flatten:
- CANONICAL_BUNDLES now selects canonical bundles via */manifest.yaml (the
8 bundle dirs) instead of */SKILL.md (which would match every top-level
skill); keeps the 8-bundle invariant required by the validation contract.
- source strings, manifest paths, fallback frontmatter, and base dirs use
<name>/... at the repo root; the top-level JSON key stays "bundles".
- validate-lifecycle-matrix.rb re-anchors NESTED_HELPER_PATTERN to
%r{(?:^|/)skills/} so nested helpers (tailscale/skills/*,
workflow-architect/skills/*) are still detected in the catalogs, and
canonical_bundles matches the generator's */manifest.yaml glob.
- align the two matrix tests in test-validate-bundles.rb: canonical bundles
are manifest-having dirs (a manifest-less top-level skill is not a row),
and the missing-row rejection targets the demo manifest bundle.
- regenerate docs/lifecycle-capability-matrix.{md,json}.
gen check mode: "is current (8 bundles)"; validate:
"Validated lifecycle capability matrix (8 bundle(s)): complete, traceable, and current."
test-validate-bundles.rb: 14 runs, 307 assertions, 0 failures.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-08-14 15:53:43 -04:00
Magnus Hedemarkandfactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Drop the Dir.glob("#{ROOT}/bundles/*/SKILL.md") term from gen-llms-txt.rb,
gen-claude-marketplace.rb, and gen-codex-plugin.rb (the first glob */SKILL.md
now covers the 8 former bundle umbrellas) and refresh the header comments.
Regenerate llms.txt and .codex-plugin/plugin.json (./bundles/<name> ->
./<name>); .claude-plugin/marketplace.json is byte-identical. Update
test-gen-llms-txt.rb fixtures from bundles/zulu etc. to top-level zulu/ with
assertions [zulu](zulu/SKILL.md), and make the duplicate-guard test robust to
case-insensitive filesystems where case-variant dir names collapse.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-08-14 15:47:05 -04:00
Magnus Hedemarkandfactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Change bundle-manifest discovery from bundles/*/manifest.yaml to */manifest.yaml
and rebuild the tmpdir fixtures in test-validate-bundles.rb at the tmpdir root
so manifests live at <skill>/manifest.yaml and path fields use the flattened
relative depth. Refresh the validator's header comment.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-08-14 15:29:59 -04:00
Magnus HedemarkGitHubfactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
* feat(validation): enforce 60K-char cap on skill reference files
Implements issue #277:
- validate-references.rb: new oversized_reference_errors check — every
references/*.md must be <= 60,000 characters; error reports path, size,
and the split-and-reindex remediation; wired into validate-skills.rb
- test-validate-skills.rb: 5 fixture tests (under-limit passes, over-limit
fails with path+size, exactly-at-limit passes, remediation message,
non-.md ignored); the suite now runs in validate.yml after the format
check (it was previously untested in CI)
- Docs: agent-skills/SKILL.md, agent-skills/references/best-practices.md,
and the AGENTS.md Format Compliance table document the cap and the
split-and-reindex procedure
- Compliance: split remote-systems-administration/references/ansible.md
and programming-principles/references/refactoring-guru.full.md into an
index + focused parts (content moved verbatim); SKILL.md routing,
README, and source-index references updated; pre-existing stale
refactoring-guru-smells.md reference repointed to the index
- Fix pre-existing quality-gate violations in the programming-principles
and remote-systems-administration descriptions (imperative verb +
negative boundary) so this PR's CI quality step passes; regenerated
llms.txt and marketplace artifacts
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
* test(evals): add eval manifests to modified skills for ratchet
The eval-coverage ratchet requires schema-valid eval manifests for any
skill modified once coverage is past 50%. This PR modifies
programming-principles and remote-systems-administration (splitting
their oversized references), so add evals/evals.json to both:
- programming-principles: 6 output-quality cases (task-to-book mapping,
principled code review, refactor-vs-rewrite, no-op detection, rule
distillation, principle conflicts)
- remote-systems-administration: 6 output-quality cases (discovery
before change, smallest control plane, rollback planning, platform
identification, verification evidence, escalation on missing
authority)
Coverage: 87/145 (60.0%) schema-valid; ratchet clean.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
---------
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-08-04 22:39:14 -04:00
Magnus HedemarkGitHubfactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
* feat(bundles): add bundle manifest schema, manifests, and validation (#203)
Introduce a machine-readable composition contract for canonical bundles:
purpose, audience, stages, included skills, prerequisites, outputs,
handoffs, conflicts, and eval suite (schemas/bundle-manifest-v1.schema.json,
following the evals-v1 versioned-schema convention). Ship the bounded design
note (docs/bundle-manifest-design.md), a schema-conformant example, canonical
manifests for the three new milestone bundles, and a stdlib-only validator
(scripts/validate-bundles.rb) that rejects incomplete, contradictory, and
undeclared-overlapping manifests while keeping bundles an optional layer.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
* feat(bundles): add lifecycle capability matrix generator and validators (#203)
Add scripts/gen-lifecycle-matrix.rb, which deterministically produces the
human-readable docs/lifecycle-capability-matrix.md (one row per canonical
bundle) and the machine-readable docs/lifecycle-capability-matrix.json (with
per-cell source provenance) reusing the gen-*.rb conventions. Add
scripts/validate-lifecycle-matrix.rb to check bundle coverage, cell
traceability, artifact currency, and catalog-exactness of nested bundle
helpers.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
* test(bundles): add bundle manifest validation tests (#203)
Add scripts/test-validate-bundles.rb covering schema conformance of the
committed example, valid-manifest and declared-conflict positives, per-field
incomplete-manifest rejections, contradictory-manifest rejections (missing
skill, undeclared handoff artifact, non-catalog conflict), undeclared-overlap
rejection naming both manifests, and matrix generator/validator
completeness and drift detection.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
* ci(bundles): wire bundle manifest validation into the gate (#203)
Add validate-bundles.rb, test-validate-bundles.rb, the lifecycle matrix
generator check, and the matrix validator to .github/workflows/validate.yml
alongside the existing validator steps.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
---------
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-08-02 19:48:09 -04:00
Magnus HedemarkGitHubusername <username>factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
* fix(product-strategy): correct stale RICE reference
Fix the misspelled prose reference at product-strategy/references/product-strategy.md:69
to point at the canonical rice-framework.md owned by product-methodology. Add a
repository check that scans references/*.md for stale prose backtick references
to nonexistent files (the bug class the SKILL.md link-resolution pass cannot
see), wired into validate-skills.rb, with a regression test suite proving the
stale reference is caught when reintroduced. product-strategy and
product-methodology remain grandfathered; no evals manifests are added.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
* fix(validate-references): case-sensitive resolution for stale-reference scan
The references scan resolved backtick tokens case-insensitively on hosts with
case-insensitive filesystems (default macOS APFS), so a token such as
`EVIDENCE-LEDGER.md` matched an existing lowercase `evidence-ledger.md` and
escaped detection locally while failing CI's Linux runners. Resolve candidates
against exact directory entries so results match CI on every host, and treat
neckbeard delivery-packet field names (EVIDENCE-LEDGER, DELIVERY-SPEC, REVIEW,
V2-SPEC) as doc-type names rather than file references. Adds regression tests
for case-mismatched and exact-case references.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
---------
Co-authored-by: username <username>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
When unittest discover processes skills/binary-analysis/tests/ before
tests/, the integration package gets cached in sys.modules from the
first discover call. Later discover calls reuse the cached package
which doesn't contain the expected test modules, causing spurious
ModuleNotFoundError failures.
Fix: snapshot and restore sys.modules between discover iterations.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Wire the Phase 3 ratchet into CI by passing the PR base SHA to
eval-coverage.py --modified-from. Expand changed-skill detection from
SKILL.md-only diffs to the entire skill directory so that references,
scripts, fixtures, README, and eval manifest edits all count as
modifications. Add a monotonic coverage floor that fails CI when
coverage decreases between the base and candidate revisions.
Add script tests for ratchet-mode detection and coverage-decrease
behaviour. Update AGENTS.md and CONTRIBUTING.md to describe the
behaviour CI now enforces.
Closes#102
Signed-off-by: Magnus Hedemark <magnus919@pm.me>
Phase 1: New skills (not in grandfathered-skills.txt) must have
evals/evals.json with at least 5 test cases. All 107 existing skills
are grandfathered.
Phase 2: scripts/eval-coverage.py reports coverage (skills with/without
evals, case counts, reference-priority sorting). Added as informational
CI step.
Phase 3: Ratchet thresholds — at 25% coverage, modified skills without
evals get a warning; at 50%, they fail CI. Enforced via
--modified-from flag for PR-scoped checks.
Closes#90
Generate a root discovery catalog from public skill frontmatter and fail CI
when the committed index drifts. Add a fixture-based regression test for
bundle paths, nested-helper exclusion, normalized descriptions,
deterministic ordering, and stale-file recovery.
Closes#78
AI-assisted: yes (Jasper/Hermes Agent)
Co-authored-by: magnus919 <magnus919>
The generator used File.basename which stripped the bundles/ prefix,
emitting ./neckbeard instead of ./bundles/neckbeard. Codex discovered
92/96 skills — the 4 bundle entrypoints were missing because their
paths didn't resolve.
Verified with live Codex CLI: all 96 skills now discoverable.
AI-assisted: yes (Jasper/Hermes Agent)
* feat: add Codex plugin packaging (single-plugin, metadata-only)
Adds .codex-plugin/plugin.json with a skills array listing all public
skills, plus .agents/plugins/marketplace.json for one-command install:
codex plugin marketplace add magnus919/agent-skills
codex plugin install magnus919
Same pattern as mattpocock/skills — one plugin, explicit skill paths,
no dist/, no curation, no duplication. Bundle-internal helpers excluded
by the shared glob. CI check mode fails if the manifest drifts.
Closes#79
AI-assisted: yes (Jasper/Hermes Agent)
* chore: trigger CI
* chore: regenerate Codex plugin manifest to include neckbeard bundle
Adds .claude-plugin/marketplace.json exposing all 95 public skills as
installable plugins via /plugin marketplace add magnus919/agent-skills.
Metadata-only approach: each entry uses source './' + skills ['./<name>']
+ strict:false, so no per-skill plugin.json or directory restructuring is
needed. Bundle-internal helper skills are excluded; bundle entrypoints are
included.
- scripts/gen-claude-marketplace.rb: generates and validates the manifest
- CI step fails if marketplace.json drifts from the skill tree
- README: Claude Code install instructions
Closes#76
AI-assisted: yes (Jasper/Hermes Agent)
Magnus Hedemarkandfactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
usernameandfactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Jasper (AI Assistant)