Files
pbakaus_impeccable/scripts/lib/validate-plugin-manifest.js
T
6d2af3f800 Guard the plugin loader contract that PR #494 exposed (#499)
* test: guard the plugin loader contract that PR #494 exposed

The agents manifest key shipped for months and silently loaded zero of
the four subagents; no validator looked at the generated plugin
manifest's shape and claude plugin validate never checks it. Three
layers now do:

- scripts/lib/validate-plugin-manifest.js pins the verified loader
  contract (KNOWN_LOADER_KEYS allowlist, no agents key, trailing-slash
  skills path from issue #86, every skill/agents/*.md shipped in
  plugin/agents/), unit-tested in tests/validate-plugin-manifest.test.js
  including a check of the real committed subtree.
- The same check gates bun run build next to the version-drift guard.
- tests/plugin-e2e.test.mjs installs the committed ./plugin subtree into
  a real Claude Code (sandboxed via CLAUDE_CONFIG_DIR in a temp dir) and
  asserts the component inventory: skill parses, all agents visible,
  hooks discovered. In the default suite; runs in about a second and
  skips cleanly when the claude CLI is absent, so CI is unaffected.

All three failed against the pre-#494 tree for the shipped reason
(Agents 0 of 4) and pass against current main.

AI-assisted via Claude Code under maintainer direction.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix: address PR review bot findings

- Copilot: guard collectPluginManifestFindings against valid JSON that is
  not an object (null, string, number, array) so a broken manifest is a
  finding instead of a build crash; unit test added
- Copilot: update the plugin-e2e header comment, the suite is in the
  default lineup rather than opt-in

AI-assisted via Claude Code under maintainer direction.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix: harden plugin E2E sandbox isolation

Bugbot: create the sandbox CLAUDE_CONFIG_DIR up front and redirect HOME
and USERPROFILE into the temp workDir too, so a CLI code path that
derives config or cache locations from the home directory instead of
CLAUDE_CONFIG_DIR still cannot touch the developer's real Claude config
when the default suite runs.

AI-assisted via Claude Code under maintainer direction.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix: agent parity check mirrors the build's emit rules

Bugbot: the shipped filename is `${claude-name || name}.md` and a
providers: list may exclude claude-code, so comparing raw source
basenames could fail the build on a renamed or provider-scoped agent
with a build:release hint that cannot fix it. The validator now derives
expected filenames the same way the transformer factory does (shared
parseFrontmatter, same providers gate) with unit coverage for renames,
name overrides, and provider-scoped agents.

AI-assisted via Claude Code under maintainer direction.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix: run the plugin E2E through a shell on Windows

Bugbot: the claude CLI is a .cmd shim on Windows and Node refuses to
spawn those via execFile without a shell, so the availability probe
always failed and the suite silently skipped there. Windows now invokes
through a shell with every argument double-quoted (temp paths routinely
contain spaces); the POSIX path is unchanged.

AI-assisted via Claude Code under maintainer direction.

Co-Authored-By: Claude Code <noreply@anthropic.com>

---------

Co-authored-by: Claude Code <noreply@anthropic.com>
2026-08-03 13:13:37 -07:00

163 lines
6.5 KiB
JavaScript

/**
* Plugin manifest shape validation (PR #494).
*
* The Claude Code marketplace installs from the committed `./plugin` subtree,
* and every key in its `.claude-plugin/plugin.json` is a claim about how
* Claude Code's plugin loader behaves. Those claims are not checked by
* `claude plugin validate` (it validates the marketplace manifest, not the
* plugin manifest), and a wrong one fails silently: shipping an `agents` key
* as an array of file paths made Claude Code load ZERO agents, while the
* loader auto-discovers `agents/*.md` on its own when the key is absent. The
* four shipped subagents were unreachable for every marketplace install and
* nothing flagged it.
*
* This guard pins the verified contract:
*
* - `KNOWN_LOADER_KEYS` is the set of keys confirmed to load correctly in a
* real Claude Code install (checked via the `claude plugin details`
* component inventory). A key outside the set fails the build until
* someone verifies it end to end and adds it here with a note.
* - The `agents` key must never appear. Omission is the only shape that
* works: an array of file paths loads zero agents, and a string or a
* directory entry fails the whole plugin.
* - Because omission means the files themselves are the only carrier, every
* agent in `skill/agents/` that ships to claude-code must have its emitted
* copy in `plugin/agents/`. The expected filename follows the build's emit
* rules (`claude-name` / `name` frontmatter override the source basename,
* and a `providers:` list may exclude claude-code entirely).
* - `skills` must keep the trailing-slash `./skills/` form (issue #86: the
* bare form fails to register slash commands).
*
* The collector is pure (filesystem-in, data-out) so it can be unit-tested
* against fixtures; build.js owns the logging and the non-zero exit.
*/
import fs from 'fs';
import path from 'path';
import { parseFrontmatter } from './utils.js';
/**
* Keys verified against a live Claude Code install (2026-08, Claude Code
* 2.1.220). Descriptive metadata keys pass through harmlessly; `skills` is
* the one component path the loader reads from this manifest. Do not add a
* component key (`agents`, `hooks`, `commands`, `mcpServers`, ...) without
* installing the built subtree and confirming the component inventory in
* `claude plugin details` counts every shipped piece.
*/
export const KNOWN_LOADER_KEYS = [
'name',
'description',
'version',
'author',
'homepage',
'repository',
'skills',
];
function listMarkdown(dir) {
if (!fs.existsSync(dir)) return [];
return fs.readdirSync(dir).filter((file) => file.endsWith('.md')).sort();
}
/**
* Which agent files the claude-code build actually emits, mirroring
* readSourceFiles (name and providers parsing) and the transformer factory
* (`${claudeName || name}.md`, providers gate). A drift between this and the
* build shows up as a false finding, so change them together.
*/
function expectedClaudeAgentFiles(rootDir) {
const agentsDir = path.join(rootDir, 'skill', 'agents');
const expected = [];
for (const sourceFile of listMarkdown(agentsDir)) {
const { frontmatter } = parseFrontmatter(
fs.readFileSync(path.join(agentsDir, sourceFile), 'utf-8'),
);
const providersRaw = frontmatter.providers;
const providers = Array.isArray(providersRaw)
? providersRaw.map((p) => String(p).trim()).filter(Boolean)
: typeof providersRaw === 'string' && providersRaw.trim()
? providersRaw.split(',').map((p) => p.trim()).filter(Boolean)
: null;
if (providers && !providers.includes('claude-code')) continue;
const name = frontmatter.name || path.basename(sourceFile, '.md');
expected.push({ sourceFile, shippedFile: `${frontmatter['claude-name'] || name}.md` });
}
return expected;
}
/**
* Validate the generated plugin manifest's shape against the verified loader
* contract.
*
* @param {string} rootDir repository root
* @returns {Array<{relPath:string, reason:string}>}
* Empty when the subtree is absent (nothing generated yet) or fully valid.
*/
export function collectPluginManifestFindings(rootDir) {
const manifestRel = 'plugin/.claude-plugin/plugin.json';
const manifestPath = path.join(rootDir, manifestRel);
if (!fs.existsSync(manifestPath)) return [];
let manifest;
try {
manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf-8'));
} catch (err) {
return [{ relPath: manifestRel, reason: `could not parse (${err.message})` }];
}
// JSON.parse accepts null, strings, numbers, and arrays; the key checks
// below need a plain object, so anything else is a finding, not a crash.
if (manifest === null || typeof manifest !== 'object' || Array.isArray(manifest)) {
return [{
relPath: manifestRel,
reason: `manifest is ${Array.isArray(manifest) ? 'an array' : manifest === null ? 'null' : `a ${typeof manifest}`}, not a JSON object`,
}];
}
const findings = [];
if ('agents' in manifest) {
findings.push({
relPath: manifestRel,
reason:
'declares an "agents" key; Claude Code loads zero agents when it is present ' +
'as file paths and fails the whole plugin on other shapes. Omit it and let ' +
'the loader auto-discover plugin/agents/*.md (PR #494)',
});
}
for (const key of Object.keys(manifest)) {
if (key === 'agents') continue; // already reported with the specific fix
if (!KNOWN_LOADER_KEYS.includes(key)) {
findings.push({
relPath: manifestRel,
reason:
`unverified manifest key "${key}"; confirm it loads in a real Claude Code ` +
'install, then add it to KNOWN_LOADER_KEYS in scripts/lib/validate-plugin-manifest.js',
});
}
}
if (manifest.skills !== './skills/') {
findings.push({
relPath: manifestRel,
reason:
`"skills" is ${JSON.stringify(manifest.skills)}; must be "./skills/" ` +
'(trailing-slash form, issue #86)',
});
}
// With no `agents` key, shipped files are the only thing the loader sees.
const shippedAgents = listMarkdown(path.join(rootDir, 'plugin', 'agents'));
for (const { sourceFile, shippedFile } of expectedClaudeAgentFiles(rootDir)) {
if (!shippedAgents.includes(shippedFile)) {
findings.push({
relPath: `plugin/agents/${shippedFile}`,
reason:
'missing from the plugin subtree; auto-discovery relies on the shipped ' +
`file, so skill/agents/${sourceFile} would never load. Run \`bun run build:release\``,
});
}
}
return findings;
}