feat(skill): add vLLM inference-serving skill (#247) (#267)

* feat(skill): add vLLM inference-serving skill (#247)

Add a single-tool vllm skill covering Docker/Kubernetes deployment,
quantization-aware model configuration (tensor parallelism, KV cache),
the OpenAI-compatible API surface, throughput/latency benchmarking,
continuous batching tuning, GPU operation, and upgrade/rollback.

Ships a read-only vllm-health probe (stdlib-only, --json), fillable
serving-config and benchmark-run-record templates, seven dated
references with upstream sources, a human-facing README, tests, and a
schema-v1 eval manifest with six cases covering config, benchmarking,
and troubleshooting.

Route ml-engineering to the new skill via a resolvable link alongside
llama-cpp, add the vllm entry to the top-level README index, and
regenerate the tracked catalogs.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>

* fix(skill): emit timeout exit 124 and bound /metrics reads in vllm-health

Address the review observations on the bundled probe: requests that exceed
--timeout now raise ProbeTimeout and make the tool exit 124 as documented
(previously they surfaced as exit 1), and the metrics check reads at most
64 KiB of /metrics and reports truncation instead of reading the whole body.
Adds tests for both behaviors.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>

---------

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
This commit is contained in:
Magnus Hedemark
2026-08-03 19:31:36 -04:00
committed by GitHub
co-authored by factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
parent f83d48ba53
commit 6181f1746d
19 changed files with 1495 additions and 5 deletions
+10 -1
View File
@@ -579,7 +579,7 @@
"./ml-engineering"
],
"strict": false,
"description": "Plan and execute production ML engineering work — model training and fine-tuning (LoRA/QLoRA), evaluation and eval-set design, quantization decisions, inference deployment, and regression triage, grounded in practical engineering patterns for production ML systems. Do not use for statistical modeling and experimental design (that's the data scientist) or for operating a specific inference engine (that's a tool skill such as llama-cpp)."
"description": "Plan and execute production ML engineering work — model training and fine-tuning (LoRA/QLoRA), evaluation and eval-set design, quantization decisions, inference deployment, and regression triage, grounded in practical engineering patterns for production ML systems. Do not use for statistical modeling and experimental design (that's the data scientist) or for operating a specific inference engine (that's a tool skill such as llama-cpp or vllm)."
},
{
"name": "neckbeard",
@@ -1103,6 +1103,15 @@
"strict": false,
"description": "Verify work against explicit criteria using direct, source-faithful evidence, reproducible checks, and clear verdicts. Use before declaring an artifact, implementation, or claim complete; do not use for exploratory research without pass/fail criteria."
},
{
"name": "vllm",
"source": "./",
"skills": [
"./vllm"
],
"strict": false,
"description": "Operate, configure, benchmark, and troubleshoot vLLM inference servers: Docker and Kubernetes deployment, quantization-aware model configuration (tensor parallelism, KV cache), OpenAI-compatible API serving, throughput and latency benchmarking, continuous batching tuning, GPU operation, and upgrade/rollback. Use when deploying or running a vLLM server (vllm serve, vllm/vllm-openai), sizing a model and its KV cache for GPUs, selecting quantization and parallelism, serving via /v1 endpoints, measuring serving throughput or latency, tuning batching, or diagnosing GPU, OOM, or startup failures in a vLLM deployment. Do not use for model training, fine-tuning, evaluation-set design, or engine-selection methodology (that is ml-engineering), or for operating the llama.cpp stack with GGUF models (that is llama-cpp); other inference engines (TGI, Ollama, Triton) are out of scope."
},
{
"name": "web-accessibility",
"source": "./",