* feat(skill): add vLLM inference-serving skill (#247) Add a single-tool vllm skill covering Docker/Kubernetes deployment, quantization-aware model configuration (tensor parallelism, KV cache), the OpenAI-compatible API surface, throughput/latency benchmarking, continuous batching tuning, GPU operation, and upgrade/rollback. Ships a read-only vllm-health probe (stdlib-only, --json), fillable serving-config and benchmark-run-record templates, seven dated references with upstream sources, a human-facing README, tests, and a schema-v1 eval manifest with six cases covering config, benchmarking, and troubleshooting. Route ml-engineering to the new skill via a resolvable link alongside llama-cpp, add the vllm entry to the top-level README index, and regenerate the tracked catalogs. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> * fix(skill): emit timeout exit 124 and bound /metrics reads in vllm-health Address the review observations on the bundled probe: requests that exceed --timeout now raise ProbeTimeout and make the tool exit 124 as documented (previously they surfaced as exit 1), and the metrics check reads at most 64 KiB of /metrics and reports truncation instead of reading the whole body. Adds tests for both behaviors. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --------- Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
ML Engineering
Machine learning engineering methodology — model training, fine-tuning (LoRA/QLoRA), evaluation, quantization, deployment, and MLOps pipeline design. Grounded in practical engineering patterns for production ML systems.
Why Install This Skill
Your agent makes informed decisions about fine-tuning approaches, quantization trade-offs, GPU selection, and serving architecture with real VRAM budgets and benchmarks. Fillable templates turn training runs, eval comparisons, and quantization decisions into reviewable records, and the bundled eval-overlap checker catches train/eval contamination before it invalidates a benchmark.
What You Get
| Directory | Purpose |
|---|---|
SKILL.md |
Core methodology, trigger conditions, reference index |
references/ |
Deep-dive reference files loaded on demand |
templates/ |
Fillable records: training-run record, eval regression table, quantization decision record |
scripts/ |
check-eval-overlap.py — detects test-set leakage between train and eval corpora |
evals/ |
Output-quality eval manifest for the skill's methodology cases |
Triggers
Setting up fine-tuning runs, quantizing models, selecting training infrastructure, deploying inference servers, evaluating model quality, or triaging a model regression.
Requirements
Assumes familiarity with PyTorch/HuggingFace ecosystem. References cover vLLM, llama.cpp, TGI, DeepSpeed, and accelerate. The bundled script needs only Python 3 (standard library).
Quick Start
Check an eval corpus for leakage against your training data before trusting any eval score:
python3 ml-engineering/scripts/check-eval-overlap.py --train data/train/ --eval data/eval/
Each eval file is reported with its overlap fraction against the training corpus; an eval file that shares more than 10% of its text with training is flagged LEAK and the script exits 1, so it can gate a CI pipeline. Add --json for machine-readable output, --token-ngram 5 to compare token sequences instead of character shingles, and --max-overlap-fraction 0.05 to tighten the threshold.
Load SKILL.md for the methodology overview and reference table, then load specific references or templates as needed for the task at hand.