mirror of
https://github.com/magnus919/agent-skills.git
synced 2026-09-22 00:56:35 +03:00
Merge remote-tracking branch 'origin/main' into ci/raleigh-tests-119
This commit is contained in:
magnus919
@@ -831,7 +831,7 @@
|
||||
"./verification-methodology"
|
||||
],
|
||||
"strict": false,
|
||||
"description": "Verify work against explicit criteria using evidence, reproducible checks, and clear pass, conditional, or blocked verdicts. Use before declaring an artifact, implementation, or claim complete."
|
||||
"description": "Verify work against explicit criteria using direct, source-faithful evidence, reproducible checks, and clear verdicts. Use before declaring an artifact, implementation, or claim complete; do not use for exploratory research without pass/fail criteria."
|
||||
},
|
||||
{
|
||||
"name": "web-accessibility",
|
||||
|
||||
@@ -93,7 +93,7 @@
|
||||
- [trakt](trakt/SKILL.md): Discover trending, anticipated, and popular movies and TV shows via the Trakt.tv API from the terminal. No authentication required for read-only discovery. Use when the user asks about what to watch, trending movies, popular shows, or media discovery.
|
||||
- [transistor](transistor/SKILL.md): Manage Transistor.fm podcast hosting from the terminal: view shows, list episodes, check analytics, and get subscriber counts. Use when the user mentions Transistor, podcast hosting, podcast analytics, show management, or episode tracking.
|
||||
- [vercel-eve](vercel-eve/SKILL.md): Build, develop, deploy, self-host, secure, and troubleshoot durable backend AI agents with Vercel Eve. Use when creating an Eve agent, adding tools, skills, subagents, channels, schedules, sandboxing, durable sessions, observability, or deploying Eve on Vercel or a Node host. Do not use for the separate Vercel AI SDK Agent APIs such as ToolLoopAgent or WorkflowAgent; use an AI SDK-specific skill for those.
|
||||
- [verification-methodology](verification-methodology/SKILL.md): Verify work against explicit criteria using evidence, reproducible checks, and clear pass, conditional, or blocked verdicts. Use before declaring an artifact, implementation, or claim complete.
|
||||
- [verification-methodology](verification-methodology/SKILL.md): Verify work against explicit criteria using direct, source-faithful evidence, reproducible checks, and clear verdicts. Use before declaring an artifact, implementation, or claim complete; do not use for exploratory research without pass/fail criteria.
|
||||
- [web-accessibility](web-accessibility/SKILL.md): Design, build, and review accessible web interfaces with native semantics, keyboard and focus behavior, forms and recovery, responsive input, motion, assistive-technology testing, and WCAG 2.2-informed evidence. Use for a11y, WCAG, ARIA, screen-reader, keyboard, focus, dialog, form, widget, or accessibility review work across frameworks.
|
||||
- [woodpecker-ci](woodpecker-ci/SKILL.md): Operate Woodpecker CI from installation through production troubleshooting: configure servers and agents, connect Forgejo/Gitea or another forge, write and validate pipelines, manage secrets and plugins, use Docker or Kubernetes backends, run the CLI, and diagnose failed builds. Use when setting up, administering, or debugging Woodpecker CI.
|
||||
- [workflow-architect](bundles/workflow-architect/SKILL.md): Discover your actual workflow through conversation or observation, then generate a tailored skills bundle that encodes it as loadable agent skills with trigger conditions. Use when you want to understand your own process, formalize it, or share it with collaborators. Also use when a session feels aimless — this skill gives it structure.
|
||||
|
||||
@@ -100,7 +100,6 @@ traefik
|
||||
trakt
|
||||
transistor
|
||||
vercel-eve
|
||||
verification-methodology
|
||||
web-accessibility
|
||||
woodpecker-ci
|
||||
yc-default-alive-calculator
|
||||
|
||||
@@ -14,16 +14,17 @@ Use it when the work needs a repeatable process and an inspectable result. It is
|
||||
|---|---|
|
||||
| `SKILL.md` | Trigger conditions, workflow, and guidance for loading deeper resources. |
|
||||
| `references/` | Reference material: `criteria-assessment.md`, `evidence-standards.md`, `verdict-template.md` |
|
||||
| `evals/evals.json` | Regression cases for direct-source selection, bundled executable discovery, failure reporting, and explicit fallback handling. |
|
||||
|
||||
## Quick Start
|
||||
|
||||
Define the criteria, collect direct evidence, and produce a verdict using `references/verdict-template.md`.
|
||||
Define the criteria, collect evidence from the requested source before any substitute, and produce a verdict using `references/verdict-template.md`.
|
||||
|
||||
Install or expose this directory using your agent's standard Agent Skills loading mechanism, then ask for work that matches the triggers below.
|
||||
|
||||
## Triggers
|
||||
|
||||
- Verify work against explicit criteria using evidence, reproducible checks, and clear pass, conditional, or blocked verdicts. Use before declaring an artifact, implementation, or claim complete.
|
||||
- Verify work against explicit criteria using direct, source-faithful evidence, reproducible checks, and clear verdicts. Use before declaring an artifact, implementation, or claim complete.
|
||||
- Requests involving the method, deliverables, or review process described in `SKILL.md`.
|
||||
- Work where a reusable template or reference from this skill would reduce avoidable mistakes.
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
name: verification-methodology
|
||||
description: Verify work against explicit criteria using evidence, reproducible checks, and clear pass, conditional, or blocked verdicts. Use before declaring an artifact, implementation, or claim complete.
|
||||
description: Verify work against explicit criteria using direct, source-faithful evidence, reproducible checks, and clear verdicts. Use before declaring an artifact, implementation, or claim complete; do not use for exploratory research without pass/fail criteria.
|
||||
license: MIT
|
||||
compatibility: No runtime dependency.
|
||||
metadata:
|
||||
@@ -17,12 +17,27 @@ Pass/fail assessment against pre-defined criteria.
|
||||
|
||||
1. **Receive** — restate the artifact, claim, or implementation being verified and the decision it will support.
|
||||
2. **Assess criteria** — convert requirements into observable pass/fail conditions; identify what would disprove each claim.
|
||||
3. **Investigate** — collect direct, reproducible evidence and record commands, source locations, or source URLs. Own this collection when the source is accessible: do not ask the user to relay evidence you can retrieve yourself. Ask only for access you genuinely lack.
|
||||
3. **Investigate** — collect direct, reproducible evidence from the source named by the request and record commands, source locations, or source URLs. Own this collection when the source is accessible: do not ask the user to relay evidence you can retrieve yourself. Ask only for access you genuinely lack.
|
||||
4. **Decide** — mark each criterion passed, failed, blocked, or not applicable. Do not convert missing evidence into a pass.
|
||||
5. **Report** — use the verdict template to distinguish verified facts, assumptions, and remaining work.
|
||||
|
||||
Stop when every criterion has direct evidence or an explicit blocked/not-applicable verdict. Escalate when the criterion is ambiguous, evidence conflicts, or the required access is unavailable.
|
||||
|
||||
## Source Fidelity
|
||||
|
||||
Treat the requested or configured local service as part of the verification criterion, not as an interchangeable topic label.
|
||||
|
||||
1. Load the matching skill and use its documented executable or service path before adjacent integrations, generic web search, or public project sources.
|
||||
2. Use the path resolved by the skill itself. A missing global `PATH` entry does not prove that a bundled executable is unavailable.
|
||||
3. If the direct source fails, report the attempted command or endpoint and its exact failure. Do not silently substitute evidence from another source.
|
||||
4. Use a substitute only when the user requests broader context or explicitly accepts the fallback. Label substitute evidence as secondary and do not present it as the requested source's state.
|
||||
|
||||
Example: for “What’s new on Jellyfin?” in an environment with a configured Jellyfin skill and bundled CLI, query that server through the bundled CLI first. Home Assistant entities and public Jellyfin project activity answer different questions.
|
||||
|
||||
## When not to use
|
||||
|
||||
Do not use this skill for open-ended exploration that has no artifact, claim, decision, or observable completion criterion. Use a research or discovery skill first, then return here when there is something falsifiable to verify.
|
||||
|
||||
## Reference Files
|
||||
|
||||
| Reference | When to load |
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
{
|
||||
"skill_name": "verification-methodology",
|
||||
"evals": [
|
||||
{
|
||||
"id": "configured-jellyfin-server-first",
|
||||
"prompt": "The jellyfin-cli skill is installed, its bundled CLI and valid server configuration are available, and the user asks: ‘What’s new on Jellyfin?’ Verify the answer.",
|
||||
"expected_output": "A source-faithful investigation that loads jellyfin-cli and uses the skill-resolved bundled CLI to query recently added items on the configured server before considering adjacent integrations or public project sources.",
|
||||
"assertions": [
|
||||
"Loads the jellyfin-cli skill before investigating service content.",
|
||||
"The first content query uses the bundled CLI's documented recent-items command through the path resolved by the skill.",
|
||||
"Does not query Home Assistant, GitHub, web search, or public Jellyfin project activity before attempting the configured server.",
|
||||
"Reports the configured server's results as server state rather than conflating them with project news."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "bundled-executable-not-on-path",
|
||||
"prompt": "The storage-audit skill resolves its executable to /opt/skills/storage-audit/scripts/storage-audit, but `which storage-audit` returns nothing. Verify the current storage report.",
|
||||
"expected_output": "An investigation that executes the skill-resolved absolute path instead of concluding that the tool is unavailable because it is absent from the global PATH.",
|
||||
"assertions": [
|
||||
"Attempts /opt/skills/storage-audit/scripts/storage-audit before trying a substitute.",
|
||||
"Does not treat a failed `which` lookup as proof that the bundled executable is unavailable.",
|
||||
"Records the exact executable path used as reproducible evidence.",
|
||||
"Bases the verdict on the bundled tool's result or exact failure."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "direct-source-failure-no-silent-substitution",
|
||||
"prompt": "Use the configured inventory CLI to verify which laptops are currently enrolled. The direct command exits with an authentication error, while a public vendor status page is reachable.",
|
||||
"expected_output": "A blocked or hold verdict that reports the exact direct-source authentication failure and identifies the access needed, without presenting the public vendor status page as evidence of local enrollment state.",
|
||||
"assertions": [
|
||||
"Reports the attempted inventory command or endpoint and the authentication failure.",
|
||||
"Marks local enrollment verification blocked or on hold rather than passed.",
|
||||
"Does not substitute the public vendor status page for the requested inventory source.",
|
||||
"States what access or configuration would resolve the block."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "explicit-broader-context-allows-secondary-source",
|
||||
"prompt": "First verify what version my configured Jellyfin server runs, then compare it with the latest upstream Jellyfin release notes.",
|
||||
"expected_output": "A two-source answer that queries the configured server first, then consults an authoritative upstream release source because the user explicitly requested the comparison, with each source and claim clearly labeled.",
|
||||
"assertions": [
|
||||
"Queries the configured Jellyfin server for its installed version before the upstream comparison.",
|
||||
"Uses an authoritative upstream release source for the latest release claim.",
|
||||
"Labels local server evidence and upstream project evidence separately.",
|
||||
"Does not imply that the latest upstream version is already installed locally."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "requested-trakt-source-before-adjacent-catalog",
|
||||
"prompt": "A configured Trakt skill documents a canonical CLI and credential source. Verify the user's Trakt watchlist. A movie-catalog integration is also available.",
|
||||
"expected_output": "A Trakt-first investigation that follows the skill's documented CLI and credential path, without switching to the adjacent movie catalog unless the user explicitly accepts that fallback.",
|
||||
"assertions": [
|
||||
"Loads the Trakt skill and follows its documented executable and credential discovery path.",
|
||||
"Attempts the Trakt watchlist query before any adjacent movie-catalog query.",
|
||||
"Does not replace a Trakt authentication failure with results from another catalog.",
|
||||
"Reports Trakt evidence or the exact Trakt failure as the basis for the verdict."
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user