mirror of https://github.com/magnus919/agent-skills.git synced 2026-09-12 12:06:29 +03:00

Files

T

Magnus Hedemarkandfactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> 030e6dfd47 feat(skill): add LiteLLM AI gateway operations skill

Add litellm/, an operational tool skill for the LiteLLM AI gateway (proxy)
and Python SDK, in the same vein as the vllm and llama-cpp engine skills.

Contents:
- SKILL.md: operating contract, operating loop, verification boundaries,
  and hard boundaries; concise core sections routing depth to references
- README.md: human-facing install/use guide with required sections
- references/: nine dated, source-indexed references (source index,
  quickstart + SDK, config & routing, keys/teams/budgets/spend, caching &
  guardrails, observability & logging, deployment, security & public
  hosting, troubleshooting), researched against litellm 1.97.0
  (2026-08-22) including a live proxy probe of the health endpoints
- scripts/litellm-health: read-only GET-only probe (liveliness, readiness,
  /v1/models, /model/info); stdlib-only Python 3.9+, --json, --help
  without a server
- tests/test_litellm_health.py: 18 deterministic tests against a local
  stub HTTP server, including the observed-traffic GET-only contract
- templates/proxy-config-record.md and proxy-deployment.md: fillable
  records; the config record is the rollback unit
- evals/evals.json: schema_version 1, six output-quality cases

Also regenerates tracked catalog artifacts (.claude-plugin/marketplace.json,
.codex-plugin/plugin.json, llms.txt) and adds the root README catalog entry
plus the skill-triggers.md index row.

AI assistance: authored with AI assistance (Factory Droid) under human
direction; facts verified against litellm 1.97.0 and official docs dated
2026-08-22.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>

2026-08-22 15:31:16 -04:00

3.2 KiB

Raw Permalink Blame History

LiteLLM Proxy Deployment Record

Fill this record for the runtime deployment: how the proxy runs, where its state lives, and how it is rolled back. Pair it with the config record (proxy-config-record.md), which captures what the proxy serves.

Deployment identity

Requested outcome: [fill: availability, scale, and exposure requirements]
Runtime: [fill: Docker / docker compose / Kubernetes + Helm / raw manifests]
Target and scope confirmed with: [fill: who confirmed, when]
Rollback path: [fill: previous image tag/digest + previous deployment record]

Pinned image

Image and tag: [fill: ghcr.io/berriai/litellm:vX.Y.Z — never latest/main-latest]
Digest: [fill: sha256:...]
Cosign verification: [fill: command/CI gate used]
Version floor check: [fill: >=1.83.7 confirmed for public deployments]
Component images (microservices chart): [fill: gateway/backend/ui tags]

Runtime shape

Replicas and worker count: [fill: --num_workers 1 per pod on K8s]
CPU/memory per replica: [fill: ~1 vCPU / 4Gi floor per worker; memory ratchets]
Autoscaling: [fill: HPA CPU target ~60% or KEDA; maxReplicas bounded by DB pool math]
Job role split: [fill: LITELLM_JOB_ROLE=serving pods + dedicated worker replica]
Security context: [fill: runAsNonRoot, readOnlyRootFilesystem, writable emptyDirs]

Network and exposure

Ports: [fill: 4000 gateway; 4001 backend; 3000 ui if microservices]
Bind address: [fill: explicit --host; default 0.0.0.0 acknowledged]
TLS termination: [fill: LB/reverse proxy; port 4000 never raw]
Edge route policy: [fill: LLM routes + health probes exposed; management paths denied]
Admin UI policy: [fill: restricted network / SSO / DISABLE_ADMIN_UI]
Probes: [fill: liveness /health/liveliness; readiness /health/readiness; thresholds]

Environment (references only — values live in the secret manager)

DATABASE_URL: [fill: secret reference]
LITELLM_MASTER_KEY: [fill: secret reference]
LITELLM_SALT_KEY: [fill: secret reference; set once, never rotate]
STORE_MODEL_IN_DB: [fill: True/False]
DISABLE_SCHEMA_UPDATE: [fill: true on pods when a migration job runs]
LITELLM_JOB_ROLE: [fill: serving | worker]
Provider keys: [fill: os.environ/ references only — never values]

Data stores

Postgres: [fill: endpoint, version, private subnet, TLS, least-privilege role]
Redis: [fill: endpoint, >=7.0, private subnet, TLS; required when >1 replica]
Backup/restore: [fill: schedule, last restore test date]

Migrations

Migration strategy: [fill: startup default vs dedicated job (Helm PreSync/hook)]
DB backup taken before last migration: [fill: date]

Verification checklist

litellm-health --check health --check readiness --json passes via the service route
x-litellm-version on a live response matches the pinned tag
A representative chat request returns tokens through the public edge
Management routes return 403/404 from outside the trust boundary
Blocked test key fails immediately (revocation path works)

Changes from the previous record

[fill: what changed, why, and the verification that backs it]

3.2 KiB Raw Permalink Blame History