feat(validation): enforce 60K-char cap on skill reference files (#279)

* feat(validation): enforce 60K-char cap on skill reference files

Implements issue #277:

- validate-references.rb: new oversized_reference_errors check — every
  references/*.md must be <= 60,000 characters; error reports path, size,
  and the split-and-reindex remediation; wired into validate-skills.rb
- test-validate-skills.rb: 5 fixture tests (under-limit passes, over-limit
  fails with path+size, exactly-at-limit passes, remediation message,
  non-.md ignored); the suite now runs in validate.yml after the format
  check (it was previously untested in CI)
- Docs: agent-skills/SKILL.md, agent-skills/references/best-practices.md,
  and the AGENTS.md Format Compliance table document the cap and the
  split-and-reindex procedure
- Compliance: split remote-systems-administration/references/ansible.md
  and programming-principles/references/refactoring-guru.full.md into an
  index + focused parts (content moved verbatim); SKILL.md routing,
  README, and source-index references updated; pre-existing stale
  refactoring-guru-smells.md reference repointed to the index
- Fix pre-existing quality-gate violations in the programming-principles
  and remote-systems-administration descriptions (imperative verb +
  negative boundary) so this PR's CI quality step passes; regenerated
  llms.txt and marketplace artifacts

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>

* test(evals): add eval manifests to modified skills for ratchet

The eval-coverage ratchet requires schema-valid eval manifests for any
skill modified once coverage is past 50%. This PR modifies
programming-principles and remote-systems-administration (splitting
their oversized references), so add evals/evals.json to both:

- programming-principles: 6 output-quality cases (task-to-book mapping,
  principled code review, refactor-vs-rewrite, no-op detection, rule
  distillation, principle conflicts)
- remote-systems-administration: 6 output-quality cases (discovery
  before change, smallest control plane, rollback planning, platform
  identification, verification evidence, escalation on missing
  authority)

Coverage: 87/145 (60.0%) schema-valid; ratchet clean.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>

---------

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
This commit is contained in:
Magnus Hedemark
2026-08-04 22:39:14 -04:00
committed by GitHub
co-authored by factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
parent c312b36166
commit 3eb7bd4096
21 changed files with 2020 additions and 1677 deletions
+2 -2
View File
@@ -867,7 +867,7 @@
"./programming-principles"
],
"strict": false,
"description": "Distilled coding principles from 14 classic software books."
"description": "Apply distilled coding principles from 14 classic software books to code review, refactoring, design, and implementation decisions. Do not use for language- or framework-specific tutorials, tool manuals, or tasks already governed by a project's established conventions."
},
{
"name": "pydanticai",
@@ -912,7 +912,7 @@
"./remote-systems-administration"
],
"strict": false,
"description": "Administer and troubleshoot remote Linux, FreeBSD, NetBSD, OpenBSD, and macOS systems safely, one host or a fleet at a time. Use when a task requires SSH, Ansible, Paramiko, POSIX diagnostics, service management, software updates, system configuration, firewall changes, or evidence-led remote operations."
"description": "Administer and troubleshoot remote Linux, FreeBSD, NetBSD, OpenBSD, and macOS systems safely, one host or a fleet at a time. Use when a task requires SSH, Ansible, Paramiko, POSIX diagnostics, service management, software updates, system configuration, firewall changes, or evidence-led remote operations. Do not use for local-only desktop administration, container or cluster orchestration (route to kubernetes), or cloud consoles that expose no remote shell."
},
{
"name": "research-and-vault",