feat: add llama.cpp operations skill (#145)

Co-authored-by: magnus919 <magnus919>

.claude-plugin/marketplace.json

@@ -428,6 +428,15 @@
       "strict": false,
       "description": "Manage Linear teams, projects, cycles, issues, comments, workflow state, and documents from a terminal through Linear's public GraphQL API. Use when a user asks to list, search, inspect, create, update, move, or comment on Linear work, or to find Linear documents. Do not use to embed a live agent inside Linear or to build an MCP integration."
     },
+    {
+      "name": "llama-cpp",
+      "source": "./",
+      "skills": [
+        "./llama-cpp"
+      ],
+      "strict": false,
+      "description": "Operate, configure, benchmark, and troubleshoot llama.cpp across CPU, Metal, CUDA, HIP/ROCm, Vulkan, SYCL, and hybrid or multi-GPU systems. Use when installing or building llama.cpp, selecting or inspecting GGUF models, running llama-cli, serving an OpenAI-compatible API with llama-server, tuning memory and performance, or diagnosing backend, context, template, and API failures. Do not use for model training or fine-tuning, general inference-framework selection, llama-cpp-python or other bindings, LlamaIndex, Ollama, or LM Studio operation."
+    },
     {
       "name": "llamaindex",
       "source": "./",

.codex-plugin/plugin.json

@@ -69,6 +69,7 @@
     "./lastfm",
     "./legal-strategy",
     "./linear",
+    "./llama-cpp",
     "./llamaindex",
     "./mermaid-diagrams",
     "./meshcore-packet-capture",

README.md

@@ -200,6 +200,10 @@ CLO/General Counsel methodology — regulatory landscape analysis (GDPR, CCPA, A
 
 Work with Linear teams, projects, cycles, issues, comments, workflow transitions, and documents using a small, dependency-free GraphQL CLI with bounded reads, dry-run previews, and focused reference guidance.
 
+### [llama-cpp](llama-cpp/SKILL.md)
+
+Operate llama.cpp from hardware-aware installation and GGUF selection through verified local inference, OpenAI-compatible serving, reproducible tuning, multi-GPU operation, and evidence-led troubleshooting. Uses dated upstream sources, operation and benchmark templates, and six output-quality eval cases without adding a wrapper CLI.
+
 ### [llamaindex](llamaindex/SKILL.md)
 
 An expert-level skill for building LLM applications over your data with LlamaIndex. RAG pipelines, multi-agent orchestration, event-driven workflows, knowledge graph construction, and production deployment.

llama-cpp/EVIDENCE-LEDGER.md

@@ -0,0 +1,83 @@
+# Evidence Ledger
+
+## Intent
+
+Add the source-grounded llama.cpp operations skill requested by issue #143, covering installation, GGUF/model handling, verified inference and serving, tuning, benchmarking, and diagnosis without freezing volatile interfaces in the core skill.
+
+## Authority
+
+The user granted modify, publish, and merge authority for this repository and requested commit, push, PR, green CI, and merge.
+
+## Inspected artifacts
+
+- Issue `magnus919/agent-skills#143` and repository `AGENTS.md`, validators, eval schema, CI workflows, catalogs, and recent merged PR conventions.
+- Existing `agent-skills`, `research-methodology`, `spec-driven-development`, `neckbeard`, `ml-engineering`, `restic`, `supabase`, and `esp32-development` guidance.
+- llama.cpp commit `555881ebc8b0fc0402b30e09258a32a7bfd13c52`, release `b10107`, official build/install/server/CLI/GGUF/quantization/benchmark/multi-GPU sources, and REST changelog.
+
+## Assumptions
+
+- The repository's current schema-version-1 eval contract remains authoritative for this change.
+- Source-backed command examples are useful without a bundled wrapper CLI.
+- Runtime behavior must be refreshed against the installed binary because upstream interfaces are volatile.
+
+## Alternatives rejected
+
+- Expanding `ml-engineering/references/quantization-inference.md`: too broad for the llama.cpp lifecycle and operational failure modes.
+- Separate GGUF/server/benchmark skills: fragments one operational workflow.
+- Bundled preflight wrapper: native `--version`, `--help`, `--list-devices`, logs, and templates cover the first version without another maintenance surface.
+- One known-good launch command: hardware/model-specific and rapidly stale.
+
+## Files changed
+
+- Added `llama-cpp/SKILL.md`, human `README.md`, this ledger, six focused references, two operation templates, and a six-case eval manifest.
+- Updated root `README.md` and `references/skill-triggers.md` for discovery.
+- Regenerated `.claude-plugin/marketplace.json`, `.codex-plugin/plugin.json`, and `llms.txt`; `.agents/plugins/marketplace.json` remained unchanged.
+
+## Commands / checks run
+
+- `ruby scripts/validate-skills.rb`
+- `ruby scripts/validate-skill-quality.rb --base origin/main`
+- `python3 scripts/test-eval-validation.py`
+- `python3 scripts/validate-evals.py`
+- `python3 scripts/test-eval-coverage.py`
+- `python3 scripts/eval-coverage.py --modified-from origin/main`
+- `python3 scripts/check-artifacts.py`
+- `ruby scripts/test-validate-skill-quality.rb`
+- Claude, Codex, and `llms.txt` generator check modes plus the `llms.txt` generator tests
+- Paired, release, and existing eval-runner test suites
+- Independent final-diff review against issue #143, Agent Skills rules, research fidelity, and neckbeard boundaries
+
+## Observed outputs
+
+- Worktree began clean on `main` at `1f5cd6a`.
+- Upstream llama.cpp reviewed at `555881eb`; latest reviewed release was `b10107`.
+- No `llama-cli` executable was present on the authoring host.
+- Skill validation found 108 canonical skills with no format/link failure.
+- Changed-skill quality checked one new skill with 0 errors and 0 warnings.
+- All 9 present eval manifests passed schema-v1 and semantic validation; eval validation and coverage tests passed.
+- Artifact checks and their repository test suites passed.
+- Generated catalogs were current at 98 public skills/plugins.
+- Paired, release, and existing eval-runner tests passed.
+- Independent review found no skill-content, trigger, safety, eval, catalog, or issue-scope defect after this ledger/README correction.
+
+## Verification boundary
+
+- Research: source and repository boundary.
+- Component: Agent Skills structure, links, README sections, eval schema, line budget, and generated catalogs.
+- Integration: whole-repository artifact and validator suites plus fake-adapter eval-runner plumbing.
+- Delivery: GitHub PR checks and final-head review remain pending until publication.
+
+## Unverified boundaries
+
+- Live llama.cpp build, model load, inference, accelerator, and server behavior are not locally exercised because no binary/model is installed and host installation/model download is outside repository scope.
+- Real-model output-quality grading is not established by the schema-v1 manifest or fake-adapter CI smoke; repository policy reports executable grader bindings, recent run evidence, and release-gated evidence as not assessed.
+
+## Rollback / follow-up triggers
+
+- Revert if repository validation or CI cannot pass without weakening required quality gates.
+- Refresh when upstream command help, backends, GGUF metadata, REST behavior, or benchmark boundaries change materially.
+- Add a script only if repeated eval traces show agents independently reimplementing the same error-prone inspection.
+
+## Status
+
+Local implementation and integration verification passed. Delivery is pending publication, final-head CI, and merge; live llama.cpp runtime behavior remains an explicitly disclosed gap.

llama-cpp/README.md

@@ -0,0 +1,46 @@
+# llama.cpp Skill
+
+Operate llama.cpp from hardware discovery through verified local inference, API serving, benchmarking, and failure diagnosis.
+
+## Why Install This Skill
+
+llama.cpp can run on laptops, CPU servers, and heterogeneous accelerator systems, but a command that starts is not necessarily using the intended backend or fitting the intended workload safely. Build options, GPU offload, GGUF quantization, context sizing, chat templates, and server behavior interact in ways that generic local-LLM advice does not capture.
+
+This skill gives your agent a discovery-first operating procedure. It checks the actual binary, host, model metadata, startup logs, and workload before selecting a launch configuration, then verifies inference or serving at the requested boundary. It also keeps fast-moving flags and API details tied to dated upstream sources instead of presenting one launch command as timeless.
+
+## What You Get
+
+| Resource | Purpose |
+|---|---|
+| `SKILL.md` | Core operating contract, task routing, safety boundaries, and completion criteria |
+| `references/installation-and-backends.md` | Package, release, Docker, source-build, and backend verification workflows |
+| `references/models-gguf-and-memory.md` | Model provenance, GGUF inspection, quantization, context, and capacity planning |
+| `references/inference-and-serving.md` | CLI smoke tests, server readiness, APIs, templates, structured output, embeddings, and reranking |
+| `references/performance-and-benchmarking.md` | Reproducible tuning and matched benchmark comparisons |
+| `references/troubleshooting.md` | Evidence-led diagnosis by symptom |
+| `references/source-index.md` | Dated primary sources and refresh rules |
+| `templates/` | Reusable operation and benchmark evidence records |
+| `evals/evals.json` | Output-quality cases for installation, fit, serving, performance, templates, and multi-GPU diagnosis |
+| `EVIDENCE-LEDGER.md` | Auditable implementation, verification, and known-gap record for this skill release |
+
+## Quick Start
+
+With a current llama.cpp installation and a compatible local GGUF model:
+
+```sh
+llama-cli --version
+llama-cli --list-devices
+llama-cli -m /path/to/model.gguf -p "Reply with exactly: llama.cpp ready" -n 16
+```
+
+Inspect the startup log and generated response. Before relying on a GPU, server, long context, or downloaded model, follow the corresponding workflow in `SKILL.md`.
+
+## Triggers
+
+Use this skill for llama.cpp installation and builds, CMake backend selection, GGUF acquisition and inspection, `llama-cli`, `llama-server`, OpenAI-compatible endpoints, chat templates, GPU offload, KV cache and context sizing, `llama-bench`, multi-GPU operation, or llama.cpp-specific troubleshooting.
+
+Do not use it for training or fine-tuning models, comparing inference frameworks generally, operating LlamaIndex/Ollama/LM Studio, or configuring language bindings such as `llama-cpp-python`.
+
+## Requirements
+
+Operations require a llama.cpp binary or a CMake/C++ build environment. Model runs require a supported GGUF model, adequate disk and RAM, and optional accelerator drivers/toolkits. Hugging Face acquisition requires network access and possibly an access token for gated repositories. Server verification uses an HTTP client such as `curl`.

llama-cpp/SKILL.md

@@ -0,0 +1,92 @@
+---
+name: llama-cpp
+description: >-
+  Operate, configure, benchmark, and troubleshoot llama.cpp across CPU, Metal, CUDA, HIP/ROCm, Vulkan, SYCL, and hybrid or multi-GPU systems. Use when installing or building llama.cpp, selecting or inspecting GGUF models, running llama-cli, serving an OpenAI-compatible API with llama-server, tuning memory and performance, or diagnosing backend, context, template, and API failures. Do not use for model training or fine-tuning, general inference-framework selection, llama-cpp-python or other bindings, LlamaIndex, Ollama, or LM Studio operation.
+license: MIT
+compatibility: Requires a supported llama.cpp binary or a build environment. Model use requires a compatible GGUF file and sufficient disk and memory; accelerator paths require the matching driver and SDK.
+metadata:
+  source: https://github.com/ggml-org/llama.cpp
+  source_index: references/source-index.md
+  research_checked: "2026-07-25"
+---
+
+# llama.cpp Operations
+
+Treat every launch recipe as a hypothesis about a specific build, model, host, and workload. Discover capabilities from the installed binary, inspect the model and startup logs, then measure the requested boundary.
+
+## Operating contract
+
+1. Record the exact llama.cpp version or commit, installation method, OS and architecture, CPU and RAM, accelerator and memory, driver/toolkit, available devices, model provenance and quantization, intended context, concurrency, and workload.
+2. Read the installed command's `--help` before using a flag from documentation. llama.cpp flags, defaults, binary names, and REST behavior change frequently.
+3. Confirm the target, scope, and rollback path before acting. Read-only discovery may proceed without confirmation.
+4. Verify the backend from `--list-devices` and model-load logs. A successful build or an accepted GPU flag does not prove acceleration is active.
+5. Start with a bounded CLI smoke test on loopback or local input. Establish a measured baseline before changing threads, batches, context, cache types, offload, or split mode.
+6. Call work complete only at the requested boundary: binary, model load, generated output, API response, benchmark comparison, or diagnosed failure with evidence.
+
+## When not to use
+
+Use `ml-engineering` for model training, fine-tuning, broad quantization methodology, evaluation design, or choosing among llama.cpp, vLLM, TGI, and other engines. Use the relevant product skill for Ollama, LM Studio, or LlamaIndex. Use binding-specific documentation for `llama-cpp-python`, node-llama-cpp, or other language wrappers.
+
+## Read-only preflight
+
+Run only commands that exist in the installed build:
+
+```sh
+llama-cli --version
+llama-cli --help
+llama-cli --list-devices
+llama-server --version
+llama-server --help
+llama-bench --help
+```
+
+Also inspect host memory and accelerator state with native OS/vendor tools. Record results in [the operation record](templates/operation-record.md). If no binary exists, choose an installation path only after reading [installation and backends](references/installation-and-backends.md).
+
+## Route the task
+
+| Need | Read first |
+|---|---|
+| Install, build, choose CPU/Metal/CUDA/HIP/Vulkan/SYCL, Docker, or prove backend use | [installation and backends](references/installation-and-backends.md) |
+| Acquire, convert, inspect, license-check, quantize, or fit a GGUF model | [models, GGUF, and memory](references/models-gguf-and-memory.md) |
+| Run `llama-cli`, expose `llama-server`, call compatible APIs, use templates, structured output, embeddings, reranking, or tools | [inference and serving](references/inference-and-serving.md) |
+| Tune threads, batches, context, cache, offload, concurrency, or multi-GPU and compare results | [performance and benchmarking](references/performance-and-benchmarking.md) |
+| Diagnose load, backend, OOM, speed, context, template, output, or API failures | [troubleshooting](references/troubleshooting.md) |
+| Check the evidence, research date, upstream revision, or refresh rule behind a claim | [source index](references/source-index.md) |
+
+## Safe workflow
+
+### 1. Select and verify the installation
+
+Prefer a supported package or release binary when its compiled backend matches the target. Build from a pinned revision when backend options, portability, or reproducibility require it. Use Docker when host isolation is useful and device passthrough is understood. After installation, capture version, help, device listing, and a model-load log before claiming success.
+
+### 2. Select and inspect the model
+
+Accept a user-specified local GGUF path or Hugging Face repository. Before downloading, record the repository, revision, file, size, model card, license, base-model lineage, and quantizer when available. Inspect GGUF metadata and model-load output for architecture, quantization, context, tokenizer, chat template, and sidecars. Plan capacity from actual file size plus KV cache, context, batch/concurrency, compute buffers, and backend overhead; parameter count alone is insufficient.
+
+Do not call one quantization universally best. Start from workload quality and capacity constraints, avoid requantizing an already quantized model when a higher-precision source is available, and compare candidate quants with the same task-quality and performance workload.
+
+### 3. Prove local inference
+
+Use a short, fixed prompt and bounded token count. Record the exact command, seed or sampling settings, startup log, output, timings, and whether the expected backend loaded. If the model has a chat template, test the template path required by the intended workload rather than treating plain completion as chat proof.
+
+### 4. Prove serving
+
+Bind to `127.0.0.1` for the first launch. Wait for `/health` to report ready, query `/v1/models`, then make a representative request using a reported model identifier. A listening process or HTTP 200 from a shallow endpoint is not inference proof. External exposure requires an explicit decision about bind address, API keys, TLS or reverse proxy, firewall, CORS, rate limits, logging, and whether experimental built-in tools are disabled.
+
+### 5. Tune one dimension at a time
+
+Preserve a baseline before changing context size, generation and batch threads, logical or physical batch size, GPU layers, KV cache type/offload, Flash Attention, parallel slots, or multi-GPU split. Use `llama-bench` for prompt-processing and token-generation comparisons, and an end-to-end client or server benchmark for TTFT and request latency. Record each comparison in [the benchmark template](templates/benchmark-comparison.md).
+
+## Hard boundaries
+
+- Do not infer accelerator use from the command line alone; require device and load-log evidence.
+- Do not expose an unauthenticated server beyond loopback by accident. Authentication is not a substitute for network and TLS controls.
+- Do not enable `llama-server` built-in filesystem or shell tools in an untrusted environment.
+- Do not override a chat template until model metadata, the original model card, and rendered behavior have been inspected.
+- Do not compare benchmark numbers from different models, quants, commits, backends, contexts, batches, thermal states, or workloads as if only one variable changed.
+- Do not treat `llama-bench` tokens per second as TTFT; its measurements exclude tokenization and sampling.
+- Do not claim a larger configured context preserves quality unless the model and scaling behavior support it and the workload was evaluated.
+
+## Exit criteria
+
+The task is complete when the requested boundary is evidenced: the expected binary and backend are observed; the selected model's provenance and fit are recorded; a bounded prompt returns usable output; a server reaches readiness and completes a representative API request; a tuning change beats or preserves the declared metrics under matched conditions; or a failure is reduced to a supported cause with a safe next action. List any stronger boundary that was not tested.

llama-cpp/evals/evals.json

@@ -0,0 +1,72 @@
+{
+  "schema_version": 1,
+  "skill_name": "llama-cpp",
+  "evals": [
+    {
+      "id": "hardware-aware-cuda-build",
+      "prompt": "Build llama.cpp with CUDA for this Linux workstation and tune it for the installed NVIDIA GPU. Tell me when GPU acceleration is working.",
+      "expected_output": "The agent discovers the exact host, GPU, driver, CUDA toolkit, compiler, CMake, and portability target; pins a llama.cpp revision; chooses current CMake options; inspects configure output; and proves the built binary sees and uses the intended device during a bounded model load.",
+      "assertions": [
+        "Discovers the GPU model, compute/toolkit compatibility, driver, host architecture, compiler, and whether the artifact is host-native or portable before selecting build options.",
+        "Uses current CMake-based CUDA guidance from installed or pinned upstream documentation rather than obsolete Makefile variables.",
+        "Verifies version, generated help, and --list-devices, then requires model-load logs showing device buffer or layer placement before claiming acceleration.",
+        "Does not infer GPU use merely from a successful build, an accepted --n-gpu-layers argument, or nvidia-smi listing the device."
+      ]
+    },
+    {
+      "id": "gguf-memory-fit-and-provenance",
+      "prompt": "I have 24 GB VRAM and 64 GB RAM. Pick a GGUF of a 32B chat model from Hugging Face, use a 64K context, and download whatever you recommend.",
+      "expected_output": "The agent identifies the exact model and workload, checks model-card license and lineage, pins the repository revision and file, dry-runs the download, inspects GGUF metadata, and budgets weights, KV cache, concurrency, buffers, and margin before proposing a quant and launch experiment.",
+      "assertions": [
+        "Does not select a quantization from parameter count and VRAM alone or present Q4_K_M as universally best.",
+        "Checks the model card, license, base-model lineage, quantizer/provenance, exact revision, file, size, shards, and sidecars before acquisition.",
+        "Accounts for trained context and scaling support, KV cache type and size, parallel sequences, batch buffers, host/device placement, and safety margin.",
+        "Uses a download dry run or equivalent size evidence and treats 64K quality and sustained fit as measurements to verify, not assumptions."
+      ]
+    },
+    {
+      "id": "verified-secure-server-startup",
+      "prompt": "Start llama-server as an OpenAI-compatible endpoint for the model at /models/chat.gguf. Make it available to other machines on my network and tell me when it is ready.",
+      "expected_output": "The agent first proves a loopback launch, waits for model readiness, obtains the reported model identifier, completes a representative OpenAI-compatible inference request, then confirms the LAN exposure, authentication, TLS/proxy, firewall, CORS, logging, and rollback boundary before changing the bind address.",
+      "assertions": [
+        "Starts with or proposes loopback verification before binding to a non-loopback address.",
+        "Distinguishes a 503 loading health response from ready state and verifies a representative inference response rather than process existence alone.",
+        "Queries the running server for its model identifier and tests the exact client route and response fields instead of assuming full OpenAI API parity.",
+        "Requires an explicit network boundary plus API-key handling, TLS or a trusted reverse proxy, firewall/CORS decisions, and does not expose experimental built-in shell or filesystem tools."
+      ]
+    },
+    {
+      "id": "performance-regression-diagnosis",
+      "prompt": "After updating llama.cpp, generation dropped from 42 to 29 tokens per second. Fix the regression. The model file did not change.",
+      "expected_output": "The agent preserves old and new versions, reconstructs matched build, backend, device, model, context, batch, cache, offload, thermal, and workload conditions; separates prompt processing from token generation and end-to-end latency; repeats structured benchmarks; and changes one variable at a time.",
+      "assertions": [
+        "Does not start by changing multiple tuning flags or assume the update itself is the cause.",
+        "Compares exact commits, build options, resolved devices/offload, drivers, model hash, context, batches, threads, cache, and thermal/background conditions.",
+        "Uses repeated llama-bench prompt-processing and token-generation measurements with raw structured output and reports variance.",
+        "Recognizes that llama-bench excludes tokenization and sampling and uses an end-to-end measurement if the reported regression is user-visible latency."
+      ]
+    },
+    {
+      "id": "chat-template-tool-call-failure",
+      "prompt": "My GGUF chats normally, but llama-server returns role markers in content and malformed JSON whenever I send OpenAI tools. Force the ChatML template and lower the temperature so it works.",
+      "expected_output": "The agent resists the requested speculative fix, identifies the exact fine-tune, inspects model-card and GGUF templates plus server props/log format, reproduces one minimal tool call, and only tests a source-supported tool-use template override while preserving the original configuration.",
+      "assertions": [
+        "Does not immediately force ChatML or treat lower temperature as a template/parser repair.",
+        "Inspects the original model card, GGUF chat and tool-use metadata, selected server format, special tokens, and current function-calling documentation.",
+        "Reproduces a minimal deterministic tool with required arguments and verifies tool name, JSON arguments, finish reason, and follow-up flow without executing the proposed tool.",
+        "Checks cache precision and template override effects on output quality and preserves a rollback to the original launch."
+      ]
+    },
+    {
+      "id": "multi-gpu-oom-and-slowdown",
+      "prompt": "A model fits across two unequal NVIDIA GPUs, but tensor split now OOMs on long prompts and is slower than one GPU. Enable peer-to-peer and find the fastest split.",
+      "expected_output": "The agent records device order, memory, interconnect, collectives, split mode, resolved allocation, context, slots, cache, and batch; recognizes tensor mode constraints and P2P risk; compares stable layer, tensor, and single-GPU or partial-offload baselines under matched workloads; and retains the fastest stable configuration with quality checks.",
+      "assertions": [
+        "Does not enable peer-to-peer before confirming support and a rollback, and notes that it can cause instability or corrupted output on some systems.",
+        "Accounts for long-context KV cache, parallel slots, batch buffers, per-device margin, Flash Attention, cache types, architecture support, and automatic-fit limitations.",
+        "Treats layer split as the stable comparison, row split as deprecated, and tensor split as experimental at the reviewed upstream revision.",
+        "Benchmarks against a matched single-GPU or partial-offload baseline and does not assume two GPUs must be faster."
+      ]
+    }
+  ]
+}

llama-cpp/references/inference-and-serving.md

@@ -0,0 +1,93 @@
+# Inference and serving
+
+Use this reference for `llama-cli`, `llama-server`, chat templates, structured output, embeddings, reranking, function calling, and network exposure.
+
+## Bounded CLI smoke test
+
+Start with one model, one short prompt, a bounded output, and explicit sampling where reproducibility matters:
+
+```sh
+llama-cli -m /path/to/model.gguf \
+  -p "Reply with exactly: llama.cpp ready" \
+  -n 16 -s 42
+```
+
+Confirm against the installed help that each option still has the intended meaning. Capture startup logs separately from generated output. Pass criteria are not merely exit code zero: the expected model and backend load, memory remains stable, output is usable for the intended mode, and timings are recorded.
+
+For chat models, inspect the embedded `tokenizer.chat_template`, original model card, and llama.cpp logs. Conversation mode may auto-enable when a template exists. Test the real system/user message behavior needed by the application. A plain prompt completion does not prove chat-template correctness.
+
+## Template diagnosis and overrides
+
+Use this order:
+
+1. identify the exact base/fine-tune model and source template;
+2. inspect GGUF template metadata and server `/props` when available;
+3. check logs for the selected chat/tool format;
+4. render or send a minimal representative conversation;
+5. compare special tokens, role boundaries, stop behavior, reasoning fields, and tool-call shape with the model documentation;
+6. only then test a current built-in or file-based template override.
+
+Preserve the original launch and model. Do not apply a familiar template merely because the model family name looks similar. Tool-use variants can differ from default chat variants, and aggressive KV-cache quantization can damage structured or tool-calling behavior.
+
+## Local server proof
+
+First bind to loopback:
+
+```sh
+llama-server -m /path/to/model.gguf --host 127.0.0.1 --port 8080
+```
+
+In another shell:
+
+```sh
+curl --fail-with-body http://127.0.0.1:8080/health
+curl --fail-with-body http://127.0.0.1:8080/v1/models
+```
+
+The health endpoint can return 503 while the model loads and 200 when ready. Select a model identifier reported by the running server, then make a representative request:
+
+```sh
+curl --fail-with-body http://127.0.0.1:8080/v1/chat/completions \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "model": "<reported-model-id>",
+    "messages": [{"role": "user", "content": "Reply with exactly: server ready"}],
+    "max_tokens": 16,
+    "temperature": 0
+  }'
+```
+
+Verify status, response schema, finish reason, content, usage/timings when exposed, and server logs. Do not claim complete OpenAI API parity; test the exact routes and fields the client needs and review the REST API changelog before upgrades.
+
+## External exposure gate
+
+Changing from loopback is a consequential operation. Confirm:
+
+- intended clients and network boundary;
+- bind address, firewall/security group, and reverse-proxy path;
+- API-key source, rotation, and log redaction;
+- TLS termination and forwarded headers;
+- CORS origins, methods, headers, and credential behavior;
+- request/body/time limits, concurrency, rate limiting, and denial-of-service controls;
+- prompt/response logging policy and data retention;
+- monitoring, restart policy, rollback binary/model/config, and external smoke test.
+
+Prefer an API-key file or supported secret injection over a literal key in a command. A public health endpoint and an authenticated inference endpoint have different exposure properties. Test from both an allowed client and a denied/untrusted path.
+
+The server includes experimental built-in filesystem and shell tools. Do not enable them in untrusted environments. If explicitly required, isolate the process, constrain filesystem/network permissions, enumerate only needed tools, and verify the trust boundary separately.
+
+## Structured output and grammars
+
+Use the installed help and current server docs to choose JSON Schema or grammar support. Validate both syntactic conformance and task semantics. A grammar can make invalid output impossible while still producing a semantically wrong value. Keep schema complexity within currently supported features and test error behavior.
+
+## Embeddings and reranking
+
+Use a model designed for the requested task and start the server in the corresponding mode. Check `/v1/models` or current capability reporting, pooling configuration, vector dimension, normalization, input limits, and batch behavior. Verify with a small known-similarity or ranking fixture; an endpoint returning numbers is not semantic proof.
+
+## Function calling
+
+Function calling depends on the model, template, parser/handler, and request schema. Inspect `/props` and logs, use a tools-aware template, and test a minimal deterministic function with required arguments. Verify tool name, JSON arguments, finish reason, parallel-call behavior if requested, and the follow-up message flow. Do not execute model-proposed tools as part of protocol verification.
+
+## Service completion criteria
+
+A service is ready only when the selected model reaches readiness, a representative request succeeds through the intended network boundary, authentication and denial paths behave as designed, response shape matches the actual client contract, resource use is within budget, and rollback is available.

llama-cpp/references/installation-and-backends.md

@@ -0,0 +1,80 @@
+# Installation and backends
+
+Use this reference when selecting an installation method, building llama.cpp, choosing a compute backend, or proving that acceleration is active. Recheck commands against the installed `--help` and the dated sources in `source-index.md`.
+
+## Discover the host first
+
+Record:
+
+- OS, release, kernel, architecture, and whether the environment is native, containerized, WSL, or virtualized;
+- CPU model, physical cores, instruction-set support, NUMA topology, and available RAM/swap;
+- every accelerator, dedicated or unified memory, driver version, runtime/toolkit, and interconnect;
+- compiler, CMake, build generator, package manager, Docker runtime, and device passthrough;
+- whether the artifact must run only on this host or on a wider hardware fleet.
+
+Use native evidence such as `uname`, `sysctl`, `lscpu`, `system_profiler`, `nvidia-smi`, `rocminfo`, `vulkaninfo`, or vendor equivalents. Do not install an SDK merely because a GPU vendor is present; verify that llama.cpp supports the exact backend/device combination.
+
+## Choose an installation path
+
+| Path | Prefer when | Main tradeoff |
+|---|---|---|
+| Package manager | Fast local setup and its compiled backend is known | Package revision and build options follow the distributor |
+| Official release binary | A published artifact matches OS, architecture, and backend | Verify artifact provenance and included backend |
+| Docker image | Isolation and reproducible image selection outweigh device-passthrough complexity | Host drivers and runtime passthrough still matter |
+| Source build | Exact revision, backend options, native tuning, or multi-backend output is required | Toolchain and SDK become part of the support surface |
+
+Official package paths currently include conda-forge, Winget, Homebrew, MacPorts, and Nix. Official container families include CPU plus backend-specific variants. Inspect the current install, release, and Docker documentation rather than assuming every package exists for every platform.
+
+## Source-build baseline
+
+Pin the intended revision before configuring:
+
+```sh
+git clone https://github.com/ggml-org/llama.cpp
+git -C llama.cpp checkout <release-tag-or-commit>
+cmake -S llama.cpp -B llama.cpp/build -DCMAKE_BUILD_TYPE=Release
+cmake --build llama.cpp/build --config Release -j <jobs>
+```
+
+Add only the backend options justified by discovery. At the reviewed upstream revision, common options include:
+
+| Target | CMake direction | Verification evidence |
+|---|---|---|
+| CPU | default build; optional BLAS/CPU-specific options | CPU backend and instruction/path logs |
+| Apple Silicon | Metal is enabled by default | Metal device listed and model buffers assigned to Metal |
+| NVIDIA | `-DGGML_CUDA=ON` | CUDA device listed, layers/buffers offloaded, vendor utilization during run |
+| AMD ROCm/HIP | `-DGGML_HIP=ON`, with target details when needed | HIP device and buffer/offload logs |
+| Vulkan | `-DGGML_VULKAN=ON` after SDK/loader validation | `vulkaninfo`, listed device, Vulkan load log |
+| Intel GPU | Follow the current SYCL backend guide | SYCL device and load log |
+
+The project can build multiple backends together and select devices at runtime. Do not assume an accepted CMake option means its SDK was found; inspect configure output and fail on missing expected dependencies.
+
+## Verify the artifact
+
+```sh
+<binary-dir>/llama-cli --version
+<binary-dir>/llama-cli --help
+<binary-dir>/llama-cli --list-devices
+```
+
+Then run a bounded model load and inspect startup output. Verification should answer:
+
+1. Which revision/build number is running?
+2. Which devices and backends are compiled and visible?
+3. Which device received model weights, KV cache, and compute buffers?
+4. How many layers or tensors remained on CPU?
+5. Did the vendor tool show activity and expected memory use during inference?
+
+On current builds, automatic fitting and GPU-layer selection may adjust unset values. Record the resolved startup configuration rather than only the submitted arguments. To prove a CPU-only control, use the installed help to identify the current device-disable mechanism; `--n-gpu-layers 0` may still allow some accelerator work, while reviewed upstream documents `--device none` as the full disable path.
+
+## Portable and fleet builds
+
+Native builds may optimize for attached hardware. For an artifact intended for other machines, define the target CPU and GPU architecture set explicitly and test on representative hosts. Record compiler, CMake cache/options, linked libraries, driver minimums, and artifact hash. A build that runs on the build host is not portability evidence.
+
+## Docker boundaries
+
+Pin an image tag or digest, mount models read-only where practical, run as a non-root user when feasible, and expose only intended ports. GPU images still require compatible host drivers and container runtime configuration. Verify acceleration inside the container with device listing, load logs, and host-side utilization; successful `docker run` is not backend proof.
+
+## Rollback
+
+Keep the previous binary/image and its launch record until the replacement passes the same smoke and benchmark workload. Source builds should use separate build/install prefixes. Package upgrades need the distributor's downgrade path and the prior version identifier.

llama-cpp/references/models-gguf-and-memory.md

@@ -0,0 +1,90 @@
+# Models, GGUF, and memory
+
+Use this reference when acquiring, converting, inspecting, quantizing, or deciding whether a model fits the target workload.
+
+## Model acquisition contract
+
+Before download or conversion, record:
+
+- original model repository and exact revision;
+- GGUF repository, file, shard set, quantization, and quantizer identity;
+- base-model and fine-tune lineage;
+- model card, intended use, known limitations, and license terms;
+- file sizes, available disk, cache location, and expected sidecars such as multimodal projectors;
+- whether the repository is gated and how credentials will be supplied without logging them.
+
+GGUF metadata can carry license, source repository, base-model lineage, quantizer, architecture, context, tokenizer, and chat-template data, but these fields are not guaranteed to be complete. Reconcile metadata with the model card and original source. Missing license or provenance is a decision blocker, not permission to guess.
+
+For reproducible Hugging Face acquisition, prefer an exact revision and file. Use the current Hugging Face CLI dry run to inspect download size before fetching:
+
+```sh
+hf download <repo> <file> --revision <full-commit> --dry-run
+hf download <repo> <file> --revision <full-commit>
+```
+
+The llama.cpp `-hf` path is useful for interactive acquisition and shared cache use, but record the resolved repository revision and file if the result must be reproducible. Keep access tokens in the supported environment/credential mechanism, never in committed commands.
+
+## Inspect before launch
+
+Use a current GGUF inspection tool from the upstream `gguf-py` package or a bounded llama.cpp model load. Capture at least:
+
+- GGUF version and tensor types;
+- architecture, parameter/size label, block count, and expert topology;
+- trained context and RoPE/scaling metadata;
+- tokenizer and special-token metadata;
+- embedded chat template and tool-use template, if present;
+- quantization type/version and quantizer/source fields;
+- shard and sidecar requirements;
+- model file hash for a locally controlled artifact.
+
+Do not edit metadata to hide incompatibility. An override is an experiment that must preserve the original file and prove output behavior.
+
+## Capacity planning
+
+Model file size is the weight-floor estimate, not total runtime memory. Budget separately for:
+
+1. mapped or loaded model weights and any duplicated host/device placement;
+2. KV cache, driven by architecture, context, parallel sequences, cache types, and offload;
+3. prompt and micro-batch compute buffers;
+4. backend/runtime, driver, graph, and allocator overhead;
+5. multimodal projectors, adapters, draft models, or multiple loaded models;
+6. safety margin for the OS and other workloads.
+
+Use GGUF metadata and startup allocation logs for the exact model. Parameter-count formulas are rough screening tools because MoE topology, tensor mixtures, metadata, cache architecture, and backend placement differ. For `llama-server`, concurrency can multiply context/cache pressure. Reduce context or parallel slots before sacrificing full accelerator offload when that matches the service objective, then measure the effect.
+
+Current llama.cpp may auto-fit unset arguments and report resolved settings. Treat that as a starting proposal, not a guarantee that sustained workload, latency, or quality targets are met.
+
+## Choose a quantization
+
+Start with constraints, not a universal recommendation:
+
+- minimum task-quality threshold;
+- model and cache capacity on the target host;
+- prompt-processing and generation performance;
+- architecture/backend kernel support;
+- context and concurrency needs;
+- whether an importance matrix exists for representative data.
+
+Compare candidate quants against a higher-precision baseline using the same task-specific prompts and, where useful, perplexity or KL-divergence tooling. Perplexity is most meaningful for comparing variants of the same model/tokenizer; it is not a universal cross-model quality score.
+
+## Conversion and quantization
+
+Upstream defines two separate phases:
+
+1. convert a supported source model to a high-quality GGUF;
+2. quantize that GGUF with `llama-quantize`.
+
+Use the conversion script and requirements from the same pinned llama.cpp revision. Validate representative outputs against the source model before quantization. Preserve the high-precision GGUF and conversion record.
+
+Avoid requantizing an already quantized input when a higher-precision source exists; upstream warns this can severely reduce quality. For importance-matrix use, select representative calibration text, record its provenance, generate the matrix with `llama-imatrix`, inspect its statistics, and compare output quality with and without it. An importance matrix is evidence for a particular model and data distribution, not a universal quality certificate.
+
+## Model acceptance gate
+
+Accept a model for the requested use only when:
+
+- provenance and license are understood;
+- all required shards and sidecars are present;
+- architecture and tensor types load in the chosen llama.cpp revision;
+- capacity includes context, concurrency, and margin;
+- the intended chat/template or completion behavior passes a representative smoke test;
+- the selected quantization meets the declared quality and performance thresholds.

llama-cpp/references/performance-and-benchmarking.md

@@ -0,0 +1,82 @@
+# Performance and benchmarking
+
+Use this reference when tuning llama.cpp or comparing builds, models, quants, backends, cache settings, or server configurations.
+
+## Define the objective
+
+Choose explicit primary and guardrail metrics:
+
+| Concern | Useful evidence |
+|---|---|
+| Interactive latency | TTFT, time per output token, end-to-end latency percentiles |
+| Offline throughput | prompt tokens/s, generated tokens/s, requests/s, total completion time |
+| Capacity | peak/resident RAM, VRAM per device, KV/cache use, maximum stable context/concurrency |
+| Quality | task success, schema/tool-call success, paired output review, perplexity/KL where appropriate |
+| Stability | error/OOM rate, variance, throttling, thermal behavior, long-run memory trend |
+
+Tokens per second alone is not an objective. State the workload: prompt/output lengths, context depth, number of slots/users, streaming, structured output, and expected hardware contention.
+
+## Freeze the comparison contract
+
+Keep constant:
+
+- llama.cpp commit/build options and backend unless that is the tested variable;
+- model file hash, quant, sidecars, template, and sampling;
+- host, driver, power mode, device visibility, and interconnect;
+- context, prompt and generated token counts, batches, cache types, offload, and concurrency;
+- warmup, repetitions, delays, background load, and thermal state.
+
+Change one dimension at a time. Record raw machine-readable output and observed startup configuration in `templates/benchmark-comparison.md`.
+
+## Use `llama-bench` correctly
+
+At the reviewed revision, `llama-bench` distinguishes:
+
+- prompt processing (`pp`);
+- text generation (`tg`);
+- combined prompt plus generation (`pg`);
+- context depth and repeated runs.
+
+Use its current `--help` to construct tests and prefer JSON/JSONL/CSV for durable comparisons. A representative shape is:
+
+```sh
+llama-bench -m /path/to/model.gguf \
+  -p 512 -n 128 -r 5 -o json
+```
+
+The upstream tool states that measurements exclude tokenization and sampling. Therefore use an end-to-end client or server benchmark for TTFT and user-visible latency. Preserve individual repetitions and standard deviation; do not report only the best run.
+
+## Tuning order
+
+Start from the working baseline and test in this order when relevant:
+
+1. **Backend and placement:** verify expected device, full or partial offload, and CPU fallback.
+2. **Context and concurrency:** set only what the workload needs; both can drive KV/cache memory.
+3. **Generation threads:** sweep from a small value upward; oversubscription can reduce decode speed.
+4. **Prompt/batch threads and sizes:** tune prompt processing separately from generation.
+5. **Logical/physical batch:** compare speed against compute-buffer memory and latency.
+6. **KV cache type/offload and Flash Attention:** measure memory, speed, and task quality together.
+7. **GPU layers/device split:** compare full offload, partial offload, and stable margin.
+8. **Advanced or experimental paths:** multi-GPU tensor mode, speculative decoding, and backend-specific knobs require a new baseline and quality/stability checks.
+
+## Multi-GPU
+
+Run `--list-devices` first and record device order and memory. At the reviewed revision:
+
+- `layer` is the default and most compatible split, useful for capacity and batch throughput;
+- `row` is deprecated;
+- `tensor` is experimental, communication-sensitive, architecture-limited, incompatible with automatic fitting in documented cases, and requires supported Flash Attention/cache combinations.
+
+Use automatic splitting as a baseline, then explicit device and tensor proportions only when measured imbalance justifies them. Compare against the best single-GPU or partial-offload baseline. More GPUs can be slower when interconnect communication dominates. Record NCCL/RCCL or peer-access availability and any warnings; revert peer-access experiments if instability or corrupted output appears.
+
+## Server benchmark
+
+Test the actual request shape and concurrency. Capture at least request success/error rate, TTFT, completion latency, prompt/completion tokens, truncation/finish reasons, throughput, and memory. The upstream server benchmark uses k6 and distinguishes client metrics from server metrics; its simple local tokenizer can differ from actual token counts, so rely on server usage for authoritative request accounting when available.
+
+## Quality guardrail
+
+Performance changes can alter output through quantization, cache precision, templates, context scaling, batching, or backend numerical behavior. Keep a fixed representative prompt/eval set. For quant comparisons, pair task-specific outcomes with perplexity or KL divergence where meaningful. Perplexity from different tokenizers/models is not directly comparable.
+
+## Verdict
+
+Accept a change only when the primary metric improves beyond observed variance, guardrails remain within thresholds, memory has safe margin, and the target workload passes. Otherwise retain the baseline and document whether the candidate was slower, unstable, too large, or outside the quality budget.

llama-cpp/references/source-index.md

@@ -0,0 +1,63 @@
+# Source index
+
+Research checked 2026-07-25 against llama.cpp commit [`555881ebc8b0fc0402b30e09258a32a7bfd13c52`](https://github.com/ggml-org/llama.cpp/commit/555881ebc8b0fc0402b30e09258a32a7bfd13c52), immediately after release [`b10107`](https://github.com/ggml-org/llama.cpp/releases/tag/b10107). Links to `master`, generated command help, changelog issues, package repositories, and Hugging Face remain live sources and must be rechecked before relying on current flags, defaults, routes, or artifacts.
+
+## Coverage matrix
+
+| Requested dimension | Primary evidence | Skill reference | Status |
+|---|---|---|---|
+| Packages, releases, Docker, source builds | Install, build, Docker docs and releases | `installation-and-backends.md` | Covered with host-specific discovery |
+| CPU, Metal, CUDA, HIP, Vulkan, SYCL | README backend table, build/backend guides, generated help | `installation-and-backends.md` | Covered; exact device support remains a task input |
+| GGUF, provenance, conversion, quantization, imatrix | GGUF specification, quantize and imatrix docs, HF model cards | `models-gguf-and-memory.md` | Covered without a universal quant recommendation |
+| Memory, context, cache, offload | Generated CLI/server help, model metadata, load logs, multi-GPU guide | `models-gguf-and-memory.md`, `performance-and-benchmarking.md` | Covered by measured procedure, not a fixed formula |
+| CLI inference and chat templates | README quick start, generated CLI help, template and function docs | `inference-and-serving.md` | Covered with installed-help refresh gate |
+| Server and OpenAI-compatible routes | Server README, tests, REST changelog | `inference-and-serving.md` | Covered; exact client contract must be tested |
+| Structured output, embeddings, reranking, tools | Server README and function-calling docs | `inference-and-serving.md` | Covered with model/capability checks |
+| Benchmarking and quality | llama-bench, server bench, perplexity docs | `performance-and-benchmarking.md` | Covered with matched-comparison contract |
+| Multi-GPU | Current multi-GPU guide and generated help | `performance-and-benchmarking.md`, `troubleshooting.md` | Covered; experimental tensor support must be refreshed |
+| Failure diagnosis | Build/server/multi-GPU docs, changelog, startup evidence | `troubleshooting.md` | Covered by symptom routing |
+
+## Primary llama.cpp sources
+
+| Area | Source | Claims used |
+|---|---|---|
+| Project scope, quick start, backends, tools | [README at reviewed commit](https://github.com/ggml-org/llama.cpp/blob/555881ebc8b0fc0402b30e09258a32a7bfd13c52/README.md) | Supported operating surfaces, model acquisition, primary binaries |
+| Pre-built installation | [Install guide](https://github.com/ggml-org/llama.cpp/blob/555881ebc8b0fc0402b30e09258a32a7bfd13c52/docs/install.md) | Package-manager matrix and distributor boundaries |
+| Source build and backends | [Build guide](https://github.com/ggml-org/llama.cpp/blob/555881ebc8b0fc0402b30e09258a32a7bfd13c52/docs/build.md) | CMake paths, backend selection, multi-backend/device discovery |
+| Containers | [Docker guide](https://github.com/ggml-org/llama.cpp/blob/555881ebc8b0fc0402b30e09258a32a7bfd13c52/docs/docker.md) | Image families, host driver and passthrough requirements |
+| CLI interface | [Generated CLI documentation](https://github.com/ggml-org/llama.cpp/blob/555881ebc8b0fc0402b30e09258a32a7bfd13c52/tools/cli/README.md) | Current flags and defaults; generated and volatile |
+| Server interface | [Server documentation](https://github.com/ggml-org/llama.cpp/blob/555881ebc8b0fc0402b30e09258a32a7bfd13c52/tools/server/README.md) | Readiness, APIs, auth/TLS, capabilities, generated flags |
+| REST changes | [REST API changelog](https://github.com/ggml-org/llama.cpp/issues/9291) | Upgrade-sensitive route, response, default, and environment changes |
+| Multi-GPU | [Multi-GPU guide](https://github.com/ggml-org/llama.cpp/blob/555881ebc8b0fc0402b30e09258a32a7bfd13c52/docs/multi-gpu.md) | Split-mode status, fit/cache constraints, troubleshooting |
+| Quantization | [Quantize guide](https://github.com/ggml-org/llama.cpp/blob/555881ebc8b0fc0402b30e09258a32a7bfd13c52/tools/quantize/README.md) | Conversion/quantization phases, requantization warning, example data |
+| Importance matrices | [Imatrix guide](https://github.com/ggml-org/llama.cpp/blob/555881ebc8b0fc0402b30e09258a32a7bfd13c52/tools/imatrix/README.md) | Calibration inputs, outputs, statistics, quantization use |
+| Model performance | [llama-bench guide](https://github.com/ggml-org/llama.cpp/blob/555881ebc8b0fc0402b30e09258a32a7bfd13c52/tools/llama-bench/README.md) | pp/tg/pg methodology, repetitions, structured outputs, exclusions |
+| Service performance | [Server benchmark guide](https://github.com/ggml-org/llama.cpp/blob/555881ebc8b0fc0402b30e09258a32a7bfd13c52/tools/server/bench/README.md) | Concurrent request benchmark and client/server metrics |
+| Quant quality | [Perplexity guide](https://github.com/ggml-org/llama.cpp/blob/555881ebc8b0fc0402b30e09258a32a7bfd13c52/tools/perplexity/README.md) | Same-model comparison, uncertainty, KL, cross-model limits |
+| Templates and tools | [Function-calling guide](https://github.com/ggml-org/llama.cpp/blob/555881ebc8b0fc0402b30e09258a32a7bfd13c52/docs/function-calling.md) | Template/handler coupling, props/log verification, cache-quality warning |
+| Template provenance | [Template maintenance](https://github.com/ggml-org/llama.cpp/blob/555881ebc8b0fc0402b30e09258a32a7bfd13c52/models/templates/README.md) | Model-source template acquisition |
+
+## Format and model provenance sources
+
+| Area | Source | Claims used |
+|---|---|---|
+| GGUF format and metadata | [GGUF specification](https://github.com/ggml-org/ggml/blob/master/docs/gguf.md) | Extensibility, mmap, architecture, context, license/source/base-model and template metadata |
+| Model cards | [Hugging Face model cards](https://huggingface.co/docs/hub/model-cards) | License, base-model lineage, intended use, limitations, datasets, evaluation |
+| Reproducible downloads | [Hugging Face download guide](https://huggingface.co/docs/huggingface_hub/guides/download) | Exact revisions, file filtering, caching, CLI dry runs |
+
+## Source evaluation
+
+The operational claims above are primarily Tier 1 official documentation, generated help, specifications, tests, and project changelogs. Upstream examples establish supported command shapes, not performance guarantees for other hardware or models. No local llama.cpp binary or model was available during skill authoring, so commands were source-verified but not represented as locally executed runtime evidence.
+
+## Refresh rules
+
+Recheck the installed help and upstream sources when any of these change:
+
+- llama.cpp build/release, binary name, package, or container image;
+- backend SDK, driver, GPU architecture, build target, or device topology;
+- model revision, GGUF metadata/version, quantization, sidecar, template, or license;
+- REST route, stream/error schema, environment variable, authentication, CORS, or default bind behavior;
+- context, cache, fit, batch, offload, split-mode, or speculative-decoding default;
+- benchmark tool output/schema or measurement boundary.
+
+Always recheck external-exposure and built-in-tool security options immediately before enabling them.

llama-cpp/references/troubleshooting.md

@@ -0,0 +1,71 @@
+# Troubleshooting
+
+Use this reference after preserving the failing command, exact version, model identity, host/device inventory, startup logs, error response, and recent change. Diagnose one layer at a time: artifact, model, backend/placement, memory, prompt/template, then API/client.
+
+## First evidence
+
+```sh
+llama-cli --version
+llama-cli --list-devices
+llama-cli --help
+llama-server --version
+llama-server --help
+```
+
+Also capture vendor device status, free RAM/disk, the model hash and metadata, and the last known-good command. Redact API/Hugging Face tokens and sensitive prompts.
+
+## Symptom routing
+
+| Symptom | Check first | Safe next action |
+|---|---|---|
+| Unknown flag or changed response | Installed `--help`, version, REST changelog | Translate configuration to current interface; do not retry obsolete flags blindly |
+| Unsupported architecture/tensor/GGUF | Model metadata, complete shards, llama.cpp revision | Confirm current architecture support or use a supported build/model conversion |
+| GPU absent | Build configure log, `--list-devices`, driver/runtime visibility | Fix build/runtime visibility before tuning offload |
+| GPU listed but CPU inference | Resolved device, offloaded layer/buffer logs, environment device filters | Select the intended device and verify load placement |
+| OOM at startup | Weight placement, context, slots, KV type/offload, batch buffers, sidecars | Reduce context/parallel slots or buffers, then offload; preserve safety margin |
+| OOM during requests | Actual prompt depth, concurrency, cache growth, multimodal inputs | Reproduce with one slot and bounded context, then scale deliberately |
+| Slow generation | Backend fallback, generation threads, partial offload, memory pressure, thermals | Establish `llama-bench` tg baseline and sweep one setting |
+| Slow prompt/TTFT | Prompt length, batch/ubatch, prompt threads, model load/cache misses | Separate pp from tokenization, queueing, and first-request warmup |
+| More threads are slower | Physical cores, SMT, NUMA, oversubscription | Sweep from one thread upward and retain measured optimum |
+| Multi-GPU slower | Split mode, interconnect, collective library, device balance | Compare `layer` with single GPU; use experimental tensor mode only when supported |
+| Garbled or role-leaking chat | Embedded/source template, conversation mode, special tokens | Test a minimal rendered chat before overriding template |
+| Broken tool calls/JSON | Tool-aware template, parser format, schema, cache quantization | Test one deterministic tool/schema and inspect logs/response shape |
+| Context exhausted/truncated | Trained context, configured context, prompt tokens, slots, finish reason | Reduce prompt/output or use supported scaling with quality evaluation |
+| Health stays 503 | Model load progress, file access, OOM, startup error | Fix load failure; do not put traffic on the instance |
+| API client fails after upgrade | Exact route/fields, model identifier, auth, stream framing, changelog | Reproduce with `curl` against the documented current contract |
+
+## Backend fallback
+
+An accelerator appearing in a vendor tool is not enough. Check that the binary was compiled with the backend, the device appears to llama.cpp, the runtime did not hide it, and model buffers/layers were actually assigned. Use a CPU-only control and vendor utilization only after confirming identical workload. Avoid changing backend, offload, threads, and batch simultaneously.
+
+## OOM triage
+
+Account separately for weights, KV cache, parallel sequences, context, batch/compute buffers, sidecars/draft models, and runtime margin. Read resolved allocations from startup logs. For a server, reproduce with one parallel slot and a bounded prompt. Reduce the largest workload-driven allocation first; moving model layers to CPU can restore fit but may sharply reduce speed.
+
+On current multi-GPU tensor mode, upstream documents additional constraints around automatic fitting, cache quantization, Flash Attention, and architecture support. Fall back to `layer` rather than forcing unsupported combinations.
+
+## Output and template triage
+
+Determine whether the model is base, instruct, chat, reasoning, or tool-tuned. Compare the original model card template, GGUF metadata, selected llama.cpp format, and response parser. Look for leaked role markers, duplicated BOS/EOS, wrong stops, missing tool delimiters, and reasoning content in unexpected fields. Use a minimal deterministic conversation before testing long prompts.
+
+## Performance triage
+
+Separate:
+
+1. startup/model load;
+2. tokenization and queueing;
+3. prompt processing;
+4. first-token latency;
+5. token generation;
+6. sampling/stream rendering;
+7. end-to-end request latency.
+
+`llama-bench` isolates model prompt/generation performance but excludes tokenization and sampling. If its numbers are stable while the API is slow, investigate queueing, templates, prompt length, streaming/client rendering, network, and concurrency.
+
+## API compatibility triage
+
+Reproduce with the smallest direct `curl` request, query current model/capability endpoints, and inspect server logs. Compare the installed build with the REST changelog. Do not assume that OpenAI-compatible means every OpenAI route, field, stream event, model-name behavior, or error shape is implemented identically.
+
+## Stop conditions
+
+Stop after two materially different fixes fail, when the model/license or target hardware is unknown, when only an unsupported architecture/backend combination would satisfy the request, or when external exposure lacks an approved security boundary. Report evidence, the exact blocker, and the smallest decision needed next.

llama-cpp/templates/benchmark-comparison.md

@@ -0,0 +1,53 @@
+# llama.cpp Benchmark Comparison
+
+## Objective
+
+- Hypothesis:
+- Primary metric and threshold:
+- Guardrail metrics and thresholds:
+- Workload represented:
+
+## Frozen conditions
+
+- Host / power / thermal / background load:
+- llama.cpp commit and build options:
+- Driver / backend / devices / interconnect:
+- Model file hash / quant / template / sidecars:
+- Prompt and output lengths / context depth:
+- Context / batch / ubatch / threads / slots:
+- Cache / Flash Attention / offload / split:
+- Warmup / repetitions / delay:
+
+## Compared variable
+
+- Baseline value:
+- Candidate value:
+- All other known differences:
+
+## Commands and raw artifacts
+
+- Baseline command/output path:
+- Candidate command/output path:
+- End-to-end or server command/output path:
+- Quality fixture/results:
+
+## Results
+
+| Metric | Baseline | Candidate | Delta | Variance / confidence | Threshold |
+|---|---:|---:|---:|---:|---:|
+| Prompt processing | | | | | |
+| Token generation | | | | | |
+| TTFT | | | | | |
+| End-to-end latency | | | | | |
+| Throughput | | | | | |
+| Peak RAM | | | | | |
+| Peak device memory | | | | | |
+| Task quality / success | | | | | |
+| Error rate | | | | | |
+
+## Verdict
+
+- Accept / reject / inconclusive:
+- Evidence:
+- Confounders or unverified boundaries:
+- Rollback or next experiment:

llama-cpp/templates/operation-record.md

@@ -0,0 +1,54 @@
+# llama.cpp Operation Record
+
+## Intent and boundary
+
+- Requested outcome:
+- Verification target: binary / model load / CLI output / API / benchmark / diagnosis
+- Target and scope confirmed:
+- Rollback path:
+
+## Host
+
+- OS / architecture / environment:
+- CPU / physical cores / NUMA:
+- RAM / swap / available disk:
+- Accelerators / memory / interconnect:
+- Driver / toolkit / container runtime:
+
+## Artifact
+
+- llama.cpp version / commit / build number:
+- Installation method or image digest:
+- Build options / compiler / CMake:
+- `--list-devices` result:
+
+## Model
+
+- Local path or repository / revision / file:
+- SHA-256:
+- License / model card / base-model lineage:
+- GGUF architecture / quantization / context / template:
+- Shards / sidecars / adapters / draft model:
+
+## Workload and launch
+
+- Exact command or request:
+- Prompt/input and sensitivity:
+- Context / output / batch / concurrency:
+- Sampling / schema / template:
+- Network / auth / TLS boundary:
+
+## Observations
+
+- Startup device and buffer placement:
+- Memory by host/device:
+- Response/output:
+- Timings and finish reason:
+- Logs/errors, with secrets redacted:
+
+## Verdict
+
+- Pass / fail / partially verified:
+- Boundary exercised:
+- Unverified boundary:
+- Next action or rollback trigger:

llms.txt

@@ -49,6 +49,7 @@
 - [lastfm](lastfm/SKILL.md): Interact with the Last.fm music data API: lookup user listening history, get artist/album/track metadata, discover similar music via collaborative filtering, explore global and per-country charts, search by artist/album/track, manage tags, and scrobble listening events. Use when the user asks about music data, listening statistics, music recommendations, similar artists, charts, or wants to scrobble or love tracks.
 - [legal-strategy](legal-strategy/SKILL.md): CLO/General Counsel methodology — regulatory landscape analysis (GDPR, CCPA, AI Act, sector-specific), IP strategy (patent, trademark, trade secret, open source licensing), contract risk assessment (indemnification, liability caps, force majeure), data privacy frameworks (privacy-by-design, DPIAs, data mapping), corporate governance (board responsibilities, fiduciary duties, shareholder rights), employment law (classification, IP assignment, non-competes).
 - [linear](linear/SKILL.md): Manage Linear teams, projects, cycles, issues, comments, workflow state, and documents from a terminal through Linear's public GraphQL API. Use when a user asks to list, search, inspect, create, update, move, or comment on Linear work, or to find Linear documents. Do not use to embed a live agent inside Linear or to build an MCP integration.
+- [llama-cpp](llama-cpp/SKILL.md): Operate, configure, benchmark, and troubleshoot llama.cpp across CPU, Metal, CUDA, HIP/ROCm, Vulkan, SYCL, and hybrid or multi-GPU systems. Use when installing or building llama.cpp, selecting or inspecting GGUF models, running llama-cli, serving an OpenAI-compatible API with llama-server, tuning memory and performance, or diagnosing backend, context, template, and API failures. Do not use for model training or fine-tuning, general inference-framework selection, llama-cpp-python or other bindings, LlamaIndex, Ollama, or LM Studio operation.
 - [llamaindex](llamaindex/SKILL.md): Expert skill for building LLM applications with the LlamaIndex framework — RAG pipelines, multi-agent orchestration, event-driven workflows, knowledge graph construction, production deployment, and evaluation. Use when working with LlamaIndex or comparing RAG and agent orchestration frameworks.
 - [mermaid-diagrams](mermaid-diagrams/SKILL.md): Author, render, and troubleshoot Mermaid diagrams for documentation, architecture, processes, and technical communication. Use when a text-based diagram needs to stay versionable.
 - [meshcore-packet-capture](meshcore-packet-capture/SKILL.md): Capture MeshCore Companion packets via BLE, serial, or TCP.

references/skill-triggers.md

@@ -29,6 +29,7 @@ Each skill's `description` field is the canonical routing contract. This conveni
 | "kubernetes", "k8s", "kubectl", "k3s", "RKE2", "MicroK8s", "k0s", "Talos", "OpenShift", "EKS", "AKS", "GKE", "Pod", "Deployment", "StatefulSet", "CRD", "RBAC", "NetworkPolicy", "Helm on Kubernetes", "cluster upgrade", "Kubernetes troubleshooting" | [kubernetes](../kubernetes/SKILL.md) |
 | "langgraph", "multi-agent", "state machine", "graph-based workflow", "LangGraph", "supervisor pattern", "swarm pattern", "agent orchestration", "graph state", "subgraph", "agent routing", "tool-calling loop", "agent loop", "stateful agent", "durable execution", "human in the loop langgraph", "checkpointer", "langgraph persistence" | [langgraph](../langgraph/SKILL.md) |
 | "Linear", "Linear API", "Linear issue", "Linear project", "Linear cycle", "Linear comment", "Linear document" | [linear](../linear/SKILL.md) |
+| "llama.cpp", "llama-cpp", "llama-cli", "llama-server", "llama-bench", "GGUF model", "GGUF quantization", "GPU layers", "KV cache", "Metal llama.cpp", "CUDA llama.cpp", "ROCm llama.cpp", "Vulkan llama.cpp", "llama.cpp OpenAI API", "llama.cpp chat template", "llama.cpp multi-GPU" | [llama-cpp](../llama-cpp/SKILL.md) |
 | "debate", "council", "multi-perspective", "structured debate", "get multiple perspectives", "expert panel", "decision landscape", "what would experts say", "what are we missing", "convergence", "false consensus", "agent-council", "pre-mortem" | [agent-council](../agent-council/SKILL.md) |
 | "skill format", "how do I make a skill", "agentskills.io" | [agent-skills](../agent-skills/SKILL.md) |
 | "Fireflies", "Fireflies.ai", "meeting transcripts", "meeting notes", "audio upload", "webhook verification", "AskFred", "AI meeting analytics" | [fireflies](../fireflies/SKILL.md) |