Guard the plugin loader contract that PR #494 exposed (#499)

* test: guard the plugin loader contract that PR #494 exposed

The agents manifest key shipped for months and silently loaded zero of
the four subagents; no validator looked at the generated plugin
manifest's shape and claude plugin validate never checks it. Three
layers now do:

- scripts/lib/validate-plugin-manifest.js pins the verified loader
  contract (KNOWN_LOADER_KEYS allowlist, no agents key, trailing-slash
  skills path from issue #86, every skill/agents/*.md shipped in
  plugin/agents/), unit-tested in tests/validate-plugin-manifest.test.js
  including a check of the real committed subtree.
- The same check gates bun run build next to the version-drift guard.
- tests/plugin-e2e.test.mjs installs the committed ./plugin subtree into
  a real Claude Code (sandboxed via CLAUDE_CONFIG_DIR in a temp dir) and
  asserts the component inventory: skill parses, all agents visible,
  hooks discovered. In the default suite; runs in about a second and
  skips cleanly when the claude CLI is absent, so CI is unaffected.

All three failed against the pre-#494 tree for the shipped reason
(Agents 0 of 4) and pass against current main.

AI-assisted via Claude Code under maintainer direction.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix: address PR review bot findings

- Copilot: guard collectPluginManifestFindings against valid JSON that is
  not an object (null, string, number, array) so a broken manifest is a
  finding instead of a build crash; unit test added
- Copilot: update the plugin-e2e header comment, the suite is in the
  default lineup rather than opt-in

AI-assisted via Claude Code under maintainer direction.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix: harden plugin E2E sandbox isolation

Bugbot: create the sandbox CLAUDE_CONFIG_DIR up front and redirect HOME
and USERPROFILE into the temp workDir too, so a CLI code path that
derives config or cache locations from the home directory instead of
CLAUDE_CONFIG_DIR still cannot touch the developer's real Claude config
when the default suite runs.

AI-assisted via Claude Code under maintainer direction.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix: agent parity check mirrors the build's emit rules

Bugbot: the shipped filename is `${claude-name || name}.md` and a
providers: list may exclude claude-code, so comparing raw source
basenames could fail the build on a renamed or provider-scoped agent
with a build:release hint that cannot fix it. The validator now derives
expected filenames the same way the transformer factory does (shared
parseFrontmatter, same providers gate) with unit coverage for renames,
name overrides, and provider-scoped agents.

AI-assisted via Claude Code under maintainer direction.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix: run the plugin E2E through a shell on Windows

Bugbot: the claude CLI is a .cmd shim on Windows and Node refuses to
spawn those via execFile without a shell, so the availability probe
always failed and the suite silently skipped there. Windows now invokes
through a shell with every argument double-quoted (temp paths routinely
contain spaces); the POSIX path is unchanged.

AI-assisted via Claude Code under maintainer direction.

Co-Authored-By: Claude Code <noreply@anthropic.com>

---------

Co-authored-by: Claude Code <noreply@anthropic.com>
This commit is contained in:
Paul Bakaus
2026-08-03 13:13:37 -07:00
committed by GitHub
co-authored by Claude Code
parent 69b63d36b6
commit 6d2af3f800
7 changed files with 552 additions and 3 deletions
+27 -1
View File
@@ -23,6 +23,7 @@ import { createTransformer, PROVIDERS } from './lib/transformers/index.js';
import { hooksJsonFor, buildClaudePluginHooksManifest } from './lib/transformers/hooks.js';
import { createAllZips, createProviderZip } from './lib/zip.js';
import { collectPluginVersions } from './lib/validate-plugin-versions.js';
import { collectPluginManifestFindings } from './lib/validate-plugin-manifest.js';
import { stageOpenAIPlugin } from './lib/openai-plugin.js';
import { ANTIPATTERNS } from '../cli/engine/registry/antipatterns.mjs';
// Sub-page generation is now handled by Astro content collections.
@@ -140,6 +141,27 @@ function validatePluginVersions(rootDir) {
return total;
}
/**
* Guard against unverified plugin manifest shapes (PR #494). The pure check
* lives in ./lib/validate-plugin-manifest.js (so it's unit-tested directly);
* this wrapper owns the console output and the error count the build gates on.
*/
function validatePluginManifestShape(rootDir) {
const findings = collectPluginManifestFindings(rootDir);
for (const { relPath, reason } of findings) {
console.error(`${relPath}: ${reason}`);
}
if (findings.length > 0) {
console.error(
`\n${findings.length} plugin manifest shape problem(s). Every manifest key is a ` +
'claim about the Claude Code loader; see scripts/lib/validate-plugin-manifest.js (PR #494).',
);
} else {
console.log('✓ Plugin manifest shape matches the verified loader contract');
}
return findings.length;
}
function validateSkillFrontmatter(skills) {
let errors = 0;
@@ -704,6 +726,10 @@ async function build() {
// match root plugin.json so marketplace installs never ship a stale version.
const versionErrors = validatePluginVersions(ROOT_DIR);
// Guard the generated plugin manifest's shape: a key the Claude Code loader
// does not honor (like the agents array, PR #494) ships silently broken.
const manifestShapeErrors = validatePluginManifestShape(ROOT_DIR);
// Scan user-facing copy for AI tells (em dashes, marketing fluff, denylisted phrases)
const proseErrors = validateProse(ROOT_DIR);
@@ -711,7 +737,7 @@ async function build() {
// that has no technical reading. Hardening repetition is intentionally allowed.
const skillProseErrors = validateSkillProse(ROOT_DIR);
if (countErrors > 0 || versionErrors > 0 || proseErrors > 0 || skillProseErrors > 0) {
if (countErrors > 0 || versionErrors > 0 || manifestShapeErrors > 0 || proseErrors > 0 || skillProseErrors > 0) {
process.exit(1);
}