diff --git a/README.md b/README.md index d1764c0..8c67a5c 100644 --- a/README.md +++ b/README.md @@ -8,9 +8,15 @@ This repo is being built incrementally. Skills will be added over time as they'r Each skill lives in its own directory with a `SKILL.md` as the entry point, optionally backed by `references/`, `templates/`, and `scripts/` for supporting material. Skills are designed to be loaded by AI agents (Hermes Agent, Claude Code, OpenCode, etc.) as procedural memory — giving them structured domain knowledge and proven approaches for specific tasks. -## Progress +## Skills -Started May 2026. Nothing here yet — check back. +### [cli-builder](cli-builder/SKILL.md) + +Design and build CLI tools for AI agent consumption. 10 universal patterns (non-interactive, `--json`, `--dry-run`, idempotent, lazy auth, progressive help), an agent-compatibility test suite, and a bash scaffold template. Principles grounded in real failures from building 15+ agent-facing CLIs. + +### [agent-skills](agent-skills/SKILL.md) + +Reference for the Agent Skills open format itself — directory structure, frontmatter schema, naming conventions, and progressive disclosure model. Use this meta-skill when creating or reviewing any other skill in this repository. ## License diff --git a/cli-builder/SKILL.md b/cli-builder/SKILL.md new file mode 100644 index 0000000..ed5eb32 --- /dev/null +++ b/cli-builder/SKILL.md @@ -0,0 +1,371 @@ +--- +name: cli-builder +description: >- + Build or refactor CLI tools designed for AI agent consumption: non-interactive, + flag-driven, idempotent, with --json output and --dry-run preview. Use when + creating a new script the agent will call, adding agent-friendly flags to an + existing tool, or debugging why an agent keeps failing to use your CLI. +license: MIT +compatibility: Requires bash, Python 3.8+, jq, and standard Unix CLI environment. +metadata: + tags: [cli, agent-tooling, design-patterns, automation] + sources: + - https://x.com/ericzakariasson/status/2036762680401223946 + - https://www.scalekit.com/blog/mcp-vs-cli-use + - https://github.com/ComposioHQ/awesome-agent-clis + - https://ronnierocha.dev/blog/dont-build-mcps-build-cli-tools +--- + +# CLI Builder — Agent-Friendly Tool Design + +## Overview + +A CLI tool is a **contract** between your code and the agent that calls it. Every design decision is part of that contract: + +| CLI Element | Contract Purpose | +|---|---| +| `--help` output | Schema — what the tool offers, what flags it accepts | +| Subcommand structure | API surface — the operations the agent can perform | +| `--json` output fields | Data contract — guaranteed keys and their types | +| Exit codes | Status signals — success, usage error, runtime failure | +| Stderr messages | Error contract — what went wrong and how to fix it | +| `--dry-run` output | Preview contract — what would happen | + +An agent discovers this contract by calling `--help`. The tool needs to be **predictable, structured, and complete** — no interactive surprises, no missing examples, no silent failures. + +## When to Use + +- Building a new script the agent will call +- Refactoring an existing tool that causes agent friction (interactive prompts, unclear errors, non-idempotent operations) +- Adding `--json`, `--dry-run`, or `--yes` flags to an existing script +- Designing a CLI subcommand for an agent framework + +**Don't use for:** One-off terminal commands the human runs interactively. The principles here optimize for machine consumption, which can make human-facing CLIs feel overly verbose. + +## Build Workflow + +A CLI tool is built in three phases: + +``` +Phase 1: Discover → Phase 2: Build → Phase 3: Verify +curl the live server → Implement → Run test suite +Confirm auth → Wire client auth → Test against live +Inventory endpoints → Write help text → Verify dry-run paths +Capture data shapes → Run tests as-you-go → Fix failures +``` + +## Phase 1: Plan — Before Writing Code + +### Architecture: One CLI Per Service + +Each API or data source gets its own CLI. Do not combine disparate services into one tool. + +**Correct:** `tmdb-cli` (TMDb only), `ghost-cli` (Ghost CMS only) +**Wrong:** `media-cli` (combines TMDb + Trakt + Radarr) + +Exception: services from the same vendor sharing auth (e.g. Radarr + Sonarr). + +### Live-Server Discovery + +Before writing any code, verify against the actual server: + +```bash +# 1. Test auth against a NON-WHITELISTED endpoint +curl -s -w "\nHTTP: %{http_code}" \ + -H "X-API-Key: $KEY" \ + https://server.example.com/api/items + +# 2. Try alternate auth mechanisms if that 401s +curl -s -w "\nHTTP: %{http_code}" \ + -H "Authorization: Bearer $TOKEN" \ + https://server.example.com/api/items + +# 3. Capture response shapes for a read endpoint +curl -s -H "X-API-Key: $KEY" \ + https://server.example.com/api/items?limit=1 | head -c 2000 +``` + +**Why this matters:** The health endpoint is often whitelisted and won't catch a wrong auth header. Test against a real data endpoint. Field names in the live response are the only truth — docs are often for a different version. + +### Bash vs Python Decision + +| Concern | Bash | Python | +|---------|------|--------| +| HTTP requests | Pipe to curl, parse with jq | `requests` library, proper error handling | +| JSON handling | jq, fragile escaping | Native `json` module | +| Auth tokens | Write/read files | Class with `_token` state | +| Multipart uploads | curl -F, painful | requests `files=` param | +| Subcommands | Case statements | argparse subparsers | +| Testing | bats, shunit2 | pytest | + +**Use Python** when: the tool sends HTTP requests, manages auth state, parses JSON responses, or has 3+ subcommand levels. + +**Use Bash** when: the tool wraps local binaries, does filesystem operations, or pipes commands together. Bash must have `set -euo pipefail` at the top. + +## Phase 2: Build — Design Patterns + +### Pattern 1: Non-Interactive by Default + +No prompts mid-execution. Everything passable as a flag or environment variable. + +```bash +# BAD — agent blocks forever +read -p "Are you sure? (y/n) " confirm + +# GOOD — flag-driven +FORCE=${FORCE:-false} +if [[ "$1" == "--force" || "$FORCE" == "true" ]]; then + : # proceed +fi +``` + +### Pattern 2: Progressive Help Discovery + +Every subcommand's `--help` includes concrete examples. Agents pattern-match off examples faster than prose. + +```bash +usage() { + case "${1:-}" in + create) + echo "Usage: $0 create --name [OPTIONS]" + echo "" + echo "Examples:" + echo " $0 create --name my-resource" + echo " $0 create --name my-resource --dry-run" + echo " $0 create --name my-resource --force" + ;; + *) + echo "Commands:" + echo " list List resources" + echo " create Create a resource" + echo "Run '$0 --help' for command-specific options." + ;; + esac +} +``` + +### Pattern 3: `--json` for Machine-Readable Output + +Support `--json` flag. Humans get text; agents get parseable data. + +```bash +if [[ "$JSON_OUTPUT" == "true" ]]; then + cat <&2; } +die() { echo "Error: $*" >&2; exit 1; } +info() { [[ "$VERBOSE" == "true" ]] && echo "[info] $*" >&2 || true; } +``` + +**Critical:** `log()` must be suppressed in `--json` mode. A stray "Processing..." line before the JSON payload breaks all consumers. `emit()` handles this correctly; the danger is auxiliary `log()`/`print()` calls that don't go through `emit()`. + +### Pattern 8: `--force` / `--yes` to Skip Confirmations + +Safe default, bypassable for automation. + +```bash +for arg in "$@"; do + case "$arg" in + --force|--yes|-y) FORCE=true ;; + --dry-run|-n) DRY_RUN=true ;; + --json) JSON_OUTPUT=true ;; + esac +done +``` + +### Pattern 9: Consistent Subcommand Structure + +Pick `resource verb` or `verb resource` and stick to it everywhere. + +``` +tool service list ✓ +tool service create ✓ +tool service delete ✓ +tool config list ✓ (agent can guess this pattern) + +tool list services ✗ (verb resource — inconsistent) +tool create-service ✗ (hyphenated verb-resource) +``` + +### Pattern 10: Lazy Auth — Help Works Without Credentials + +Credentials checked at request time, not client creation time. `--help` and `--dry-run` work without any key. + +```python +class MyClient: + def __init__(self, api_key=""): + self.api_key = api_key # Accept empty key — don't check yet + + def _request(self, method, path, ...): + if not self.api_key and not DRY_RUN: + die("API key not found. Set MYTOOL_API_KEY in your environment.") + if DRY_RUN: + return {"dry_run": True} # Safe empty response + # Real HTTP call follows +``` + +This way `tool cmd --help` and `tool cmd --dry-run` never need credentials. + +## Phase 3: QA — Agent Compatibility Testing + +### Essential Test Suite + +```bash +# 1. Syntax check +python3 -c "import py_compile; py_compile.compile('./tool.py', doraise=True)" + +# 2. --help on every subcommand has examples +./tool.sh create --help | grep -qi "example" && echo "PASS" + +# 3. --json output is valid parseable JSON +./tool.sh list --json 2>/dev/null | jq . >/dev/null && echo "PASS" + +# 4. Missing required args → immediate error with corrective usage +result=$(./tool.sh create 2>&1 || true) +echo "$result" | grep -qi "\-\-name" && echo "PASS" + +# 5. --dry-run returns meaningful preview +result=$(./tool.sh delete --name x --dry-run 2>&1 || true) +echo "$result" | grep -qi "dry-run\|would" && echo "PASS" + +# 6. Errors to stderr, data to stdout +result=$(./tool.sh create 2>&1 1>/dev/null || true) +echo "$result" | grep -qi "Error" && echo "PASS: errors to stderr" + +# 7. Idempotent — second call succeeds +result=$(./tool.sh create --name test 2>&1 || true) +result=$(./tool.sh create --name test 2>&1 || true) +echo "$result" | grep -qi "no-op\|already\|exists" && echo "PASS" + +# 8. --dry-run on chained commands doesn't crash +result=$(./tool.sh list --dry-run 2>&1 || true) +echo "$result" | grep -qi "dry-run\|would\|preview" && echo "PASS" +``` + +### Live-Server Verification + +The syntax tests above catch coding errors. They don't catch API mismatches. Run every read command against a real server: + +```bash +# Auth verification (non-whitelisted endpoint) +curl -s -H "X-API-Key: $KEY" https://api.example.com/items?limit=1 + +# Read command smoke test +./tool.sh list --json > /dev/null && echo "PASS" + +# Dry-run every mutating command to verify payload structure +./tool.sh create --name test --dry-run --json 2>/dev/null +``` + +Common bugs only found this way: +- **Wrong auth header** — health endpoint was whitelisted, you never tested a real endpoint +- **Wrong field names** — API returns `items[].id` but you wrote `entity_name` +- **Wrong content-type** — login needs form data, not JSON +- **Wrong nesting** — Swagger shows `daily` at top level, real API nests it under `forecast` + +## Gotchas — The Eight Most Common Agent-CLI Bugs + +These are the failures observed across every agent-built CLI: + +1. **Errors on stdout** — `echo "Error"` (no `>&2`) breaks pipeline consumers. Always use `die()` which writes to stderr. + +2. **No examples in `--help`** — Agents can't guess argument order from a field description. Every subcommand needs at least two concrete examples. + +3. **`--json` output has auxiliary text** — A "Processing..." line before the JSON payload makes `json.load()` fail. Gate ALL output through `emit()`. + +4. **No `--dry-run` for chained commands** — Handler fetches data first (e.g. station ID lookup), dry-run crashes before reaching the preview. Short-circuit BEFORE data-fetching logic. + +5. **Auth header format guessed wrong** — Some APIs use `X-API-Key`, others use `Authorization: Bearer`, some use both for different auth mechanisms. Always curl a non-whitelisted endpoint first. + +6. **Content-type mismatch on login** — Login/oauth endpoints usually use `application/x-www-form-urlencoded`, not JSON. Build `_form_post()` separately. + +7. **Idempotency not checked** — `create` called twice creates duplicate state. Always guard creation/deletion with an existence check. + +8. **Hyphenated positional arguments** — Python argparse converts `--flag-name` to `args.flag_name` for flags, but `parser.add_argument("resource-id")` stays as `getattr(args, "resource-id")`, not `args.resource_id`. + +## Agent-Readiness Checklist + +- [ ] No interactive prompts (`read`, `select`, `dialog`) +- [ ] All inputs via flags or env vars +- [ ] `--help` on every subcommand with examples +- [ ] `--json` output is valid parseable JSON +- [ ] `--dry-run` on every destructive command +- [ ] `--force` / `--yes` to skip confirmations +- [ ] Idempotent: second call returns no-op, not error +- [ ] Consistent `resource verb` structure +- [ ] Errors go to stderr (`>&2` or `die()`) +- [ ] `--json` mode suppresses all non-JSON stdout +- [ ] Lazy auth: `--help` and `--dry-run` work without credentials +- [ ] Exit codes: 0=success, 1=usage error, 2=runtime failure +- [ ] Live-server verified for every read endpoint +- [ ] Dry-run verified for every chained-API path + +## References + +- [templates/bash-cli-scaffold.sh](templates/bash-cli-scaffold.sh) — Full bash project template with pre-wired global flags, logging helpers, and subcommand dispatch. Use as a starting point for any bash CLI. +- [references/python-api-client.md](references/python-api-client.md) — Complete Python API client pattern with lazy auth, centralized error handling, form-login support, and argparse dispatch with pre-parsed global flags. Read when building a Python CLI that wraps an HTTP API. +- [references/advanced-patterns.md](references/advanced-patterns.md) — Edge case patterns: morphological text matching, version-dependent imports, robust JSON consumption from third-party tools, dry-run short-circuit for chained APIs. Read when a specific edge case from the gotchas section bites you. +- [references/mcp-vs-cli.md](references/mcp-vs-cli.md) — Summary of the MCP-vs-CLI discourse with a decision framework. Read when debating whether to build a CLI or an MCP server for a new integration. +- [references/improvement-cycle.md](references/improvement-cycle.md) — Structured feedback schema and HALO-style prioritization for improving CLIs over time. Read after shipping your first version and collecting usage traces. diff --git a/cli-builder/references/advanced-patterns.md b/cli-builder/references/advanced-patterns.md new file mode 100644 index 0000000..94d8973 --- /dev/null +++ b/cli-builder/references/advanced-patterns.md @@ -0,0 +1,184 @@ +# Advanced Patterns + +Edge case patterns that don't apply to every CLI but are essential when they do. + +## Morphological Matching for Text-Based Filters + +When a CLI provides `--category`, `--type`, or similar text matching against section headers, **exact substring matching fails on morphological variants**: + +| User passes | Header reads | Substring match? | +|---|---|---| +| `warehouse` | DATA WAREHOUSING | ❌ "warehouse" ≠ substring of "warehousing" | +| `model` | DATA MODELING | ❌ "model" ≠ substring of "modeling" | +| `format` | STORAGE FORMATS | ❌ "format" ≠ substring of "formats" | + +**Fix: Match on a shared word stem.** Take the first N characters of the user's filter term and check if that stem appears in the lowercased header: + +```python +def _header_matches(header: str, category: str, stem_len: int = 5) -> bool: + """Check if a category filter matches a section header on shared stem.""" + cat_lower = category.lower() + header_lower = header.lower() + # Exact match first (fast path) + if cat_lower in header_lower: + return True + # Stem match (handles morphological variants) + stem = cat_lower[:min(len(cat_lower), stem_len)] + return len(stem) >= 3 and stem in header_lower +``` + +**When to use:** Any CLI with `--category`, `--type`, or free-text filtering against known labels where the headers may use different morphological forms. + +## Version-Dependent Imports After Dry-Run + +When a CLI handler wraps an optional library module, imports must respect the dry-run first check: + +```python +async def cmd_foo(**kw): + url = kw["url"] + # ... parse ALL params BEFORE any imports ... + + if DRY_RUN: + emit("[dry-run] Would foo", {"dry_run": True, "url": url}) + return + + # Lazy imports — after dry-run, so --dry-run works without the module + from some_library.optional import CoolFeature + try: + from some_library.newer_module import NewThing + except ImportError: + NewThing = None # graceful fallback for older versions + + # ... rest of handler using CoolFeature / NewThing ... +``` + +**The failure mode:** If the import is at the top of the handler, `--dry-run` crashes with `ModuleNotFoundError` even though it should be safe. + +**Detection:** Syntax checks don't catch this. Run `--dry-run` against the actual target environment. + +## Robust JSON Consumption from Subprocesses + +When consuming JSON from a CLI tool you don't control, filter stdout lines before parsing: + +```python +def _parse_json_output(result: subprocess.CompletedProcess) -> list | dict: + """Parse subprocess stdout as JSON, filtering out non-JSON noise.""" + json_lines = [] + for line in result.stdout.splitlines(): + stripped = line.strip() + if stripped.startswith(("[", "{")): + json_lines.append(stripped) + if not json_lines: + return [] + return json.loads("\n".join(json_lines)) +``` + +This acts as a "JSON line filter" — anything that doesn't start with `[` or `{` is discarded. Safe because: +- JSON arrays always start with `[` +- JSON objects always start with `{` +- Warning/status messages rarely start with either character + +**When to use:** Consuming JSON from tools you didn't build, or tools with environment-specific logging you can't suppress. + +## Dry-Run Short-Circuit for Chained-API Commands + +The basic dry-run pattern breaks down when a command handler chains multiple API calls where the first call provides context for subsequent calls. Example: + +```python +def cmd_current(client, args): + # First call: fetch station list to get device IDs + stations = client.get_stations() # Fails in dry-run — returns [] + # Second call: fetch observations for that device + obs = client.get_observations(stations[0]["id"]) +``` + +In dry-run mode, the first call returns an empty list. The handler errors: "No stations found." The dry-run never reaches the preview logic. + +**Fix: Add a command-level dry-run short-circuit BEFORE any data-fetching calls.** + +```python +def cmd_current(client, args): + # Short-circuit at command level, above all data-fetching + if client.dry_run: + emit("[dry-run] Would query latest station observations.", + {"dry_run": True, "command": "current"}) + return + + # All real logic follows — stations lookup, observations fetch + stations = client.get_stations() + obs = client.get_observations(stations[0]["id"]) + # ... format and emit output ... +``` + +**Pattern rules:** +1. The short-circuit must emit a meaningful preview of what the command would do +2. It must include all parameters the command received (IDs, flags, etc.) +3. It must return — not fall through — so the chained API calls never execute +4. Every command that chains API calls needs its own short-circuit + +**Detection:** If a command emits a fatal error (not a dry-run preview) when run with `--dry-run`, it has this problem. + +## Container-Key Wrapper Ambiguity + +When a CLI wraps an API that expects the POST body wrapped in a container key (e.g., `{"dashboard": {...}, "overwrite": true}`), there's ambiguity about what `--file` should contain: the inner resource only, or the full POST body. + +**Rule:** Accept the inner resource body only. Add the wrapper yourself in the handler. + +```python +def cmd_create(json_file): + with open(json_file) as f: + data = json.load(f) # expected: {...}, not {"resource": {...}} + body = {"resource": data, "overwrite": True} + client._request("POST", "/api/resources", json_data=body) +``` + +Document this explicitly in `--file` help text: "Path to a JSON file containing just the resource body — the CLI adds the API envelope." + +## Library Init Banners in Stdout + +Some libraries (Crawl4AI, Playwright) print init banners to stdout via bare `print()`, not logging. These appear before any of your code runs and contaminate `--json` output. + +**Preferred fix: route ALL human output to stderr, redirect stdout globally in `main()`.** + +```python +import sys +_REAL_STDOUT = None + +def emit(human: str, machine: dict) -> None: + if JSON_OUTPUT: + print(json.dumps(machine), file=_REAL_STDOUT) + else: + print(human, file=sys.stderr) + +def main(): + global _REAL_STDOUT + # ... parse args, detect JSON mode ... + + if JSON_OUTPUT: + _REAL_STDOUT = sys.stdout + sys.stdout = sys.stderr # Library noise → stderr + # ... dispatch handlers ... +``` + +No `with` blocks needed in any handler. Only `emit()` writes to the real stdout. + +## Input Sanitization for Embedded Queries + +When user input gets interpolated into SQL, Cypher, or shell commands, sanitize first: + +```bash +sanitize() { + printf '%s' "$1" | sed -e "s/'//g" -e 's/;/./g' +} +name=$(sanitize "$RAW_NAME") +``` + +In Python: + +```python +def sanitize(value: str) -> str: + """Strip characters that break string interpolation.""" + for char in ["'", ";", "\\"]: + value = value.replace(char, "") + return value +``` diff --git a/cli-builder/references/improvement-cycle.md b/cli-builder/references/improvement-cycle.md new file mode 100644 index 0000000..ba772f0 --- /dev/null +++ b/cli-builder/references/improvement-cycle.md @@ -0,0 +1,58 @@ +# CLI Improvement Cycle + +After a CLI ships, real usage reveals what the tests didn't catch. This cycle captures feedback and prioritizes fixes systematically. + +## The Flywheel + +``` +Traces → Feedback → Triage → Fix → Deploy + ↑ │ + └──────────────────────────────┘ +``` + +## Structured Feedback Schema + +When an agent session reveals a problem with a CLI tool, capture it as structured data — not a note-to-self. This makes patterns visible across multiple sessions: + +```python +feedback = { + "tool": "my-cli", + "trace_id": "", + "theme": "ambiguous_error", + # one of: missing_flag, silent_failure, wrong_output, + # unparseable_json, confusing_help + "command": "my-cli deploy --env staging --tag v1.2", + "observed": "Agent ran command with correct flags but got " + "non-zero exit with no stderr output", + "expected": "Non-zero exit should always include a stderr message " + "explaining what went wrong", + "frequency": "single_occurrence", + # or "recurring" — if recurring, escalate to High priority +} +``` + +Review the feedback log before each new CLI build to identify recurring pain points. + +## HALO-Style Prioritization + +When the feedback log accumulates, triage findings by four tiers: + +| Priority | Criteria | Action | +|----------|----------|--------| +| **Blocking** | Tool returns wrong output, errors on valid input, or crashes | Fix immediately, add regression test | +| **High** | Agent misuses a flag or pattern across multiple sessions (2+ feedback entries with same theme) | Fix this sprint, update help text | +| **Medium** | Missing `--json`, missing help examples, inconsistent naming | Schedule next sprint | +| **Low** | Stderr hygiene, edge-case idempotency, non-idiomatic flag names | Defer, log for next version | + +**Triage rule:** Pattern frequency overrides tier. A "Medium" finding that appears in 3+ sessions is actually High. A "Blocking" finding that only appeared once with a workaround may be Medium. + +The goal is not to fix everything — it's to have a defensible reason for what you're fixing now vs. deferring. + +## Applying the Cycle + +1. Collect traces from agent sessions using the tool +2. When a pattern emerges, write a structured feedback entry +3. Before the next development cycle, review the backlog +4. Fix the top priority items +5. Add regression tests for each fix +6. Deploy the updated tool diff --git a/cli-builder/references/mcp-vs-cli.md b/cli-builder/references/mcp-vs-cli.md new file mode 100644 index 0000000..562bd9d --- /dev/null +++ b/cli-builder/references/mcp-vs-cli.md @@ -0,0 +1,43 @@ +# MCP vs CLI — Discourse Summary & Decision Framework + +## The Core Argument + +CLI-first design for agent tools vs MCP servers — what the debate is actually about, not the noise. + +### Key Sources + +| Source | Key Claim | +|--------|-----------| +| **Eric Zakariasson** — "Building CLIs for Agents" ([X thread](https://x.com/ericzakariasson/status/2036762680401223946)) | CLIs designed for agents need `--json`, `--dry-run`, examples in help, and non-interactive mode. Most CLIs assume a human. | +| **ScaleKit Benchmarks** ([scalekit.com/blog/mcp-vs-cli-use](https://www.scalekit.com/blog/mcp-vs-cli-use)) | 9-32× token savings for CLI over MCP on GitHub automation. 100% reliability (CLI) vs 72% (MCP). The gap is schema injection — 43 tool definitions per turn, agent uses 1-2. | +| **Ronnie Rocha** — "Don't Build MCPs, Build CLI Tools" ([ronnierocha.dev](https://ronnierocha.dev/blog/dont-build-mcps-build-cli-tools/)) | MCP was designed for sandboxed agents (IDE plugins, web assistants). Terminal-native agents already have access — they don't need a bridge. MCP tax: context bloat, no composability, no pipes, serialized overhead. | +| **Garry Tan** ([X](https://x.com/garrytan/status/2031910564344262988)) | "MCP sucks honestly. It eats too much context window… I vibe coded a CLI wrapper for Playwright tonight in 30 minutes… worked 100x better." | +| **Peter Steinberger (steipete)** — MCPorter ([github.com/steipete/mcporter](https://github.com/steipete/mcporter)) | Converts MCP tools to CLI commands. Describes MCP as "a crutch" for environments without terminal access. | + +## Token Cost Breakdown + +The ScaleKit benchmark reveals why CLI wins for agent consumption: + +| Metric | CLI | MCP | +|--------|-----|-----| +| Schema overhead per turn | 0 tokens (agent calls `--help` on demand) | 500-2000 tokens (full tool definitions injected every turn) | +| Output shape | Agent requests exactly what it needs via flags | Full JSON-RPC response, unfiltered | +| Composition | Piped through `jq`, `grep`, `mlr` | Atomic calls only | +| Failure mode | Exit code + stderr message | ConnectTimeout to MCP endpoint (36% of MCP failures) | + +The gap grows with tool count. A CLI aggregate (one binary with subcommands) costs ~100 tokens in `--help` output. An MCP aggregate costs ~43 tool schemas × ~500 tokens each = ~21,500 tokens per turn. + +## Decision Framework + +| Situation | Default | Rationale | +|-----------|---------|-----------| +| Agent already has a terminal | **CLI** | No bridge needed. Agent pipes output directly. | +| Single-user, personal/homelab | **CLI** | Simpler to build and debug. No server to maintain. | +| Multi-tenant, end-user OAuth | **MCP** | Auth delegation and credential management handled by the protocol. | +| Token-sensitive at scale | **CLI** | 9-32× cheaper per operation. | +| Need composability (pipes) | **CLI** | `cmd | grep | jq` is zero-cost composition. | +| Need dynamic resource discovery | **MCP** | MCP's `list_resources` + `subscribe` provides real-time schema discovery. | +| Enterprise audit/traceability | **MCP** | JSON-RPC has structured request/response logging at the protocol level. | +| Internal bespoke API | **Either** — CLI with `--json` is simpler; MCP if governed access is required | + +**Bottom line:** CLI is the default for most agent-facing tools. MCP wins for governed multi-tenant deployments where credential management and audit trails are the primary value. In practice, many deployments end up hybrid: CLI for high-frequency known tools, MCP for sandboxed or governed integrations. diff --git a/cli-builder/references/python-api-client.md b/cli-builder/references/python-api-client.md new file mode 100644 index 0000000..a0a49f3 --- /dev/null +++ b/cli-builder/references/python-api-client.md @@ -0,0 +1,271 @@ +# Python API Client Pattern + +Use this pattern when your CLI wraps an HTTP API. Two key design decisions: +1. **Lazy auth** — credentials checked at request time, not client creation time +2. **Pre-parsed global flags** — `--json` and `--dry-run` work in any position + +## Client Class + +```python +import json, os, sys, warnings +import requests +from typing import Dict, Any, Optional, List + +DEFAULT_SERVER = os.getenv("MYTOOL_SERVER", "http://localhost:8080") +ENV_API_KEY = os.getenv("MYTOOL_API_KEY", "") + + +class MyToolError(Exception): + pass + + +class MyToolClient: + """API client with lazy auth, dry-run, and error wrapping.""" + + def __init__(self, server: str, api_key: str = "", dry_run: bool = False): + self.server = server.rstrip("/") + self.api_key = api_key or ENV_API_KEY + self.dry_run = dry_run + self._token: Optional[str] = None + self._token_file = os.path.expanduser("~/.mytool_token") + + # Auto-load saved JWT token + if not self.api_key: + try: + with open(self._token_file) as f: + saved = f.read().strip() + if saved: + self._token = saved + except (OSError, IOError): + pass + + # ── Auth Headers ────────────────────────────────────────── + + def _headers(self) -> Dict[str, str]: + """Build auth headers. Priority: JWT token > API key > no auth.""" + h: Dict[str, str] = {} + if self._token: + h["Authorization"] = f"Bearer {self._token}" + elif self.api_key: + # VERIFY header name against YOUR server. + # Common options: X-API-Key, Authorization: Bearer, Authorization: Token + h["X-API-Key"] = self.api_key + if not any("files" in k for k in ["_files"]): + h["Content-Type"] = "application/json" + return h + + # ── Centralized HTTP Request ────────────────────────────── + + def _request(self, method: str, path: str, + params: Optional[Dict] = None, + json_data: Any = None, + files: Optional[Dict] = None) -> Dict[str, Any]: + """Centralized HTTP request with error wrapping. + + Credentials checked HERE, not in __init__. + This lets --help and --dry-run work without any API key configured. + """ + url = f"{self.server}{path}" + headers = self._headers() + + if files: + headers.pop("Content-Type", None) # requests sets multipart boundary + + # Dry-run: return a safe empty shape matching what the handler expects + if self.dry_run: + msg = f"[dry-run] {method.upper()} {path}" + info = {"dry_run": True, "method": method.upper(), "url": url, + "params": params, "json": json_data} + print(msg, file=sys.stderr) + # Return empty shape that won't crash the handler + return {"items": [], "total_count": 0} + + # Check credentials only on real API calls + if not self.api_key and not self._token: + raise MyToolError( + "No credentials configured.\n" + " Set MYTOOL_API_KEY environment variable or login with:\n" + " mytool login --username --password " + ) + + try: + resp = requests.request(method=method, url=url, + params=params, json=json_data, + files=files, headers=headers, timeout=120) + except requests.ConnectionError as e: + raise MyToolError( + f"Cannot connect to {self.server}: {e}\n" + f" Is the server running? Set MYTOOL_SERVER or use --server." + ) + + if resp.status_code == 204: + return {} + + try: + body = resp.json() + except (json.JSONDecodeError, ValueError): + body_text = resp.text.strip() + if not body_text: + return {} + body = {"raw": body_text[:500]} + + if resp.status_code == 401: + raise MyToolError( + f"Auth failed (401). Check your credentials.\n" + f" Server response: {body.get('detail', body.get('message', str(body)))}" + ) + if resp.status_code >= 400: + detail = body.get("detail", body.get("message", str(body))) + raise MyToolError(f"API error ({resp.status_code}): {detail}") + + return body + + # ── Form-Encoded Login ──────────────────────────────────── + + def _form_post(self, path: str, data: Dict[str, str]) -> Dict[str, Any]: + """For login/oauth endpoints that need form-encoded data, not JSON.""" + url = f"{self.server}{path}" + if self.dry_run: + return {"dry_run": True, "method": "POST", "url": url, "form": data} + try: + resp = requests.post(url, data=data, timeout=30) + except requests.ConnectionError as e: + raise MyToolError(f"Cannot connect: {e}") + try: + body = resp.json() + except (json.JSONDecodeError, ValueError): + body = {"raw": resp.text[:500]} + if resp.status_code == 401: + raise MyToolError("Login failed: incorrect credentials") + if resp.status_code >= 400: + detail = body.get("detail", body.get("message", str(body))) + raise MyToolError(f"Login error ({resp.status_code}): {detail}") + return body + + def login(self, username: str, password: str) -> Dict[str, Any]: + """Form-based login with token persistence.""" + result = self._form_post("/login", {"username": username, "password": password}) + if "token" in result: + self._token = result["token"] + try: + with open(self._token_file, "w") as f: + f.write(self._token) + except OSError: + pass + return result + + # ── Endpoint Methods ────────────────────────────────────── + + def list_items(self, limit: int = 50) -> Dict[str, Any]: + return self._request("GET", "/items", params={"limit": limit}) + + def get_item(self, item_id: str) -> Dict[str, Any]: + return self._request("GET", f"/items/{item_id}") + + def create_item(self, name: str, **kwargs) -> Dict[str, Any]: + return self._request("POST", "/items", json_data={"name": name, **kwargs}) + + def delete_item(self, item_id: str) -> Dict[str, Any]: + return self._request("DELETE", f"/items/{item_id}") + + def upload_file(self, file_path: str) -> Dict[str, Any]: + """File upload using multipart — Content-Type set by requests.""" + with open(os.path.abspath(file_path), "rb") as f: + files = {"file": (os.path.basename(file_path), f)} + return self._request("POST", "/items/upload", files=files) +``` + +## Argparse Dispatch with Pre-Parsed Global Flags + +The fundamental argparse pitfall: argparse routes all unrecognized flags after a subcommand name to that subparser. If `--json` is only defined on the main parser, `tool subcommand --json` fails. + +**Fix: pre-parse global flags from argv before argparse sees them.** + +```python +def _preparse_global_flags(argv: List[str]) -> tuple[Dict[str, Any], List[str]]: + """Strip global flags from argv regardless of position. + + Returns (globals_dict, filtered_argv) where filtered_argv has only + positional args and subcommand-specific flags, ready for argparse. + """ + GLOBAL_BOOLS = {"--json", "--dry-run", "--force", "--quiet", "--verbose"} + GLOBAL_VALUES = {"--server", "--key"} + globals_map: Dict[str, Any] = {} + filtered: List[str] = [argv[0]] + i = 1 + while i < len(argv): + arg = argv[i] + if arg in GLOBAL_BOOLS: + globals_map[arg.lstrip("-").replace("-", "_")] = True + i += 1 + elif arg in GLOBAL_VALUES: + key = arg.lstrip("-").replace("-", "_") + if i + 1 < len(argv) and not argv[i + 1].startswith("-"): + globals_map[key] = argv[i + 1] + i += 2 + else: + globals_map[key] = "" + i += 1 + elif arg == "--": + filtered.extend(argv[i:]) + break + else: + filtered.append(arg) + i += 1 + return globals_map, filtered + + +def main(): + # 1. Strip global flags from anywhere in argv + global_flags, filtered_argv = _preparse_global_flags(sys.argv) + + # 2. Suppress Python warnings in machine mode + if global_flags.get("json"): + warnings.simplefilter("ignore") + + # 3. Let argparse handle the filtered args + parser = argparse.ArgumentParser(prog="mytool") + parser.add_argument("--server", default="") + # ... subparsers, etc. + args = parser.parse_args(filtered_argv[1:]) + + # 4. Merge: explicit argparse value > pre-parsed global > env var + server = args.server or global_flags.get("server") or os.getenv("MYTOOL_SERVER", DEFAULT_SERVER) + dry_run = global_flags.get("dry_run", False) + json_mode = global_flags.get("json", False) + + client = MyToolClient(server=server, dry_run=dry_run) + # ... dispatch to subcommand handlers ... +``` + +This handles `--json` in any position: +- `mytool --json subcommand --flag value` (before subcommand) +- `mytool subcommand --flag value --json` (after subcommand) +- `mytool subcommand --json subsub --name foo` (deep nesting) + +## Env Var File-Read Fallback + +Terminal subprocesses may not inherit environment variables from the parent agent process. Always implement a file-read fallback: + +```python +def _get_env(key: str, default: str = "") -> str: + """Get env var with ~/.mytool.env file-read fallback.""" + val = os.getenv(key) + if val: + return val + env_path = os.path.expanduser("~/.mytool.env") + if os.path.isfile(env_path): + try: + with open(env_path) as f: + for line in f: + line = line.strip() + if line.startswith("export "): + line = line[len("export "):] + if line.startswith(f"{key}="): + return line.split("=", 1)[1].strip("\"'") + except OSError: + pass + return default +``` + +Call this at module level to set defaults even when env vars aren't inherited. diff --git a/cli-builder/templates/bash-cli-scaffold.sh b/cli-builder/templates/bash-cli-scaffold.sh new file mode 100644 index 0000000..a7e37a6 --- /dev/null +++ b/cli-builder/templates/bash-cli-scaffold.sh @@ -0,0 +1,242 @@ +#!/usr/bin/env bash +# tool.sh — +# Usage: ./tool.sh [OPTIONS] +# +# Agent-friendly CLI scaffold following the cli-builder skill patterns. +# Replace and implement cmd_* functions for your use case. +# +# Pre-wired patterns: +# --json Machine-readable JSON output +# --dry-run Preview destructive operations +# --force Skip confirmations +# --quiet/-q Suppress non-essential output +# --verbose/-v Additional diagnostic output to stderr +# +# Available helpers: +# log() stdout, suppressed in --json and --quiet modes +# warn() stderr, always visible +# die() stderr + exit 1 +# info() stderr, visible only with --verbose +# emit() dual output: machine string vs human string + +set -euo pipefail + +# === Configuration (override via env vars) === +TOOL_DB="${TOOL_DB:-/path/to/default.db}" +TOOL_SERVER="${TOOL_SERVER:-http://localhost:8080}" + +# === State (modified by parse_args) === +COMMAND="" +FORCE=false +DRY_RUN=false +JSON_OUTPUT=false +QUIET=false +VERBOSE=false + +# === Argument Parsing (indexed approach) === +# Uses while+shift inside the loop, NOT for arg in "$@" — shifts inside +# for-each loops don't affect the iteration variable and cause consumed +# flag values to appear as positional arguments. +parse_args() { + while [[ $# -gt 0 ]]; do + case "$1" in + --force|--yes|-y) FORCE=true; shift ;; + --dry-run|-n) DRY_RUN=true; shift ;; + --json) JSON_OUTPUT=true; shift ;; + --quiet|-q) QUIET=true; shift ;; + --verbose|-v) VERBOSE=true; shift ;; + --help|-h) usage "${COMMAND:-}"; exit 0 ;; + --) shift; break ;; + -*) die "Unknown flag '$1'. Run '$0 --help' for usage." ;; + *) + if [[ -z "$COMMAND" ]]; then + COMMAND="$1" + else + die "Unexpected argument '$1'. Run '$0 $COMMAND --help' for usage." + fi + shift + ;; + esac + done + + [[ -z "$COMMAND" ]] && { usage; exit 1; } +} + +# === Logging Helpers === +log() { + if [[ "$QUIET" != "true" && "$JSON_OUTPUT" != "true" ]]; then + echo "$@" + fi +} +warn() { echo "Warning: $*" >&2; } +die() { echo "Error: $*" >&2; exit 1; } +info() { [[ "$VERBOSE" == "true" ]] && echo "[info] $*" >&2 || true; } + +# === Dual Output Helper === +# Every command calls emit() exactly once. This is the only path to stdout. +emit() { + if [[ "$JSON_OUTPUT" == "true" ]]; then + echo "$1" # machine-readable JSON string + else + echo "$2" # human-readable text + fi +} + +# === Usage / Help === +usage() { + local cmd="${1:-}" + case "$cmd" in + list) + cat <<'HELP' +Usage: tool.sh list [OPTIONS] + +List resources. + +Options: + --json Output as JSON + --quiet Suppress non-essential output + +Examples: + tool.sh list + tool.sh list --json | jq '.[].name' +HELP + ;; + create) + cat <<'HELP' +Usage: tool.sh create --name [OPTIONS] + +Create a resource. + +Options: + --name Resource name (required) + --dry-run Preview without creating + --force Overwrite if exists + +Examples: + tool.sh create --name my-resource + tool.sh create --name my-resource --dry-run +HELP + ;; + *) + cat <<'HELP' +Usage: tool.sh [OPTIONS] + +Commands: + list List resources + create Create a resource + delete Delete a resource + +Global flags (work in any position): + --force, -y Skip confirmations + --dry-run, -n Preview changes + --json Machine-readable output + --quiet, -q Minimal output + --verbose, -v Detailed output to stderr + +Run 'tool.sh --help' for command-specific options. +HELP + ;; + esac +} + +# === Commands === + +cmd_list() { + info "Listing resources..." + + if [[ "$DRY_RUN" == "true" ]]; then + emit '{"dry_run":true,"command":"list"}' "[dry-run] Would list resources" + return 0 + fi + + # TODO: implement list logic + # Use log() for progress, emit() for output + # Use $JSON_OUTPUT to decide format + + emit '{"items":[]}' "No resources found" +} + +cmd_create() { + local NAME="" + + # Parse command-specific flags + while [[ $# -gt 0 ]]; do + case "$1" in + --name) NAME="$2"; shift 2 ;; + --name=*) NAME="${1#--name=}"; shift ;; + --) shift; break ;; + *) die "Unknown flag '$1'. Run 'tool.sh create --help' for usage." ;; + esac + done + + # Validate required flags + [[ -z "$NAME" ]] && die "--name is required" + + # Sanitize input + NAME=$(printf '%s' "$NAME" | sed -e "s/'//g" -e 's/;//g') + + # Idempotency check + if resource_exists "$NAME"; then + if [[ "$FORCE" != "true" ]]; then + emit "{\"status\":\"exists\",\"name\":\"$NAME\"}" "Resource '$NAME' already exists — no-op" + return 0 + fi + info "Overwriting existing resource '$NAME'" + fi + + # Dry-run + if [[ "$DRY_RUN" == "true" ]]; then + emit "{\"dry_run\":true,\"name\":\"$NAME\"}" "[dry-run] Would create '$NAME'" + return 0 + fi + + # TODO: implement create logic + create_resource "$NAME" + + emit "{\"status\":\"created\",\"name\":\"$NAME\"}" "Created '$NAME'" +} + +cmd_delete() { + local NAME="" + + while [[ $# -gt 0 ]]; do + case "$1" in + --name) NAME="$2"; shift 2 ;; + --name=*) NAME="${1#--name=}"; shift ;; + --) shift; break ;; + *) die "Unknown flag '$1'. Run 'tool.sh delete --help' for usage." ;; + esac + done + + [[ -z "$NAME" ]] && die "--name is required" + + if ! resource_exists "$NAME"; then + emit "{\"status\":\"not_found\",\"name\":\"$NAME\"}" "Resource '$NAME' not found" + return 1 + fi + + if [[ "$DRY_RUN" == "true" ]]; then + emit "{\"dry_run\":true,\"name\":\"$NAME\"}" "[dry-run] Would delete '$NAME'" + return 0 + fi + + # TODO: implement delete logic + delete_resource "$NAME" + + emit "{\"status\":\"deleted\",\"name\":\"$NAME\"}" "Deleted '$NAME'" +} + +# === Stub Functions (implement for your use case) === +resource_exists() { return 1; } +create_resource() { :; } +delete_resource() { :; } + +# === Main Dispatch === +parse_args "$@" + +case "$COMMAND" in + list) cmd_list "$@" ;; + create) cmd_create "$@" ;; + delete) cmd_delete "$@" ;; + *) die "Unknown command '$COMMAND'" ;; +esac