mirror of
https://github.com/magnus919/agent-skills.git
synced 2026-09-14 21:16:36 +03:00
54 lines
1.3 KiB
Markdown
54 lines
1.3 KiB
Markdown
# llama.cpp Benchmark Comparison
|
|
|
|
## Objective
|
|
|
|
- Hypothesis:
|
|
- Primary metric and threshold:
|
|
- Guardrail metrics and thresholds:
|
|
- Workload represented:
|
|
|
|
## Frozen conditions
|
|
|
|
- Host / power / thermal / background load:
|
|
- llama.cpp commit and build options:
|
|
- Driver / backend / devices / interconnect:
|
|
- Model file hash / quant / template / sidecars:
|
|
- Prompt and output lengths / context depth:
|
|
- Context / batch / ubatch / threads / slots:
|
|
- Cache / Flash Attention / offload / split:
|
|
- Warmup / repetitions / delay:
|
|
|
|
## Compared variable
|
|
|
|
- Baseline value:
|
|
- Candidate value:
|
|
- All other known differences:
|
|
|
|
## Commands and raw artifacts
|
|
|
|
- Baseline command/output path:
|
|
- Candidate command/output path:
|
|
- End-to-end or server command/output path:
|
|
- Quality fixture/results:
|
|
|
|
## Results
|
|
|
|
| Metric | Baseline | Candidate | Delta | Variance / confidence | Threshold |
|
|
|---|---:|---:|---:|---:|---:|
|
|
| Prompt processing | | | | | |
|
|
| Token generation | | | | | |
|
|
| TTFT | | | | | |
|
|
| End-to-end latency | | | | | |
|
|
| Throughput | | | | | |
|
|
| Peak RAM | | | | | |
|
|
| Peak device memory | | | | | |
|
|
| Task quality / success | | | | | |
|
|
| Error rate | | | | | |
|
|
|
|
## Verdict
|
|
|
|
- Accept / reject / inconclusive:
|
|
- Evidence:
|
|
- Confounders or unverified boundaries:
|
|
- Rollback or next experiment:
|