# llama.cpp Benchmark Comparison

## Objective

- Hypothesis:
- Primary metric and threshold:
- Guardrail metrics and thresholds:
- Workload represented:

## Frozen conditions

- Host / power / thermal / background load:
- llama.cpp commit and build options:
- Driver / backend / devices / interconnect:
- Model file hash / quant / template / sidecars:
- Prompt and output lengths / context depth:
- Context / batch / ubatch / threads / slots:
- Cache / Flash Attention / offload / split:
- Warmup / repetitions / delay:

## Compared variable

- Baseline value:
- Candidate value:
- All other known differences:

## Commands and raw artifacts

- Baseline command/output path:
- Candidate command/output path:
- End-to-end or server command/output path:
- Quality fixture/results:

## Results

| Metric | Baseline | Candidate | Delta | Variance / confidence | Threshold |
|---|---:|---:|---:|---:|---:|
| Prompt processing | | | | | |
| Token generation | | | | | |
| TTFT | | | | | |
| End-to-end latency | | | | | |
| Throughput | | | | | |
| Peak RAM | | | | | |
| Peak device memory | | | | | |
| Task quality / success | | | | | |
| Error rate | | | | | |

## Verdict

- Accept / reject / inconclusive:
- Evidence:
- Confounders or unverified boundaries:
- Rollback or next experiment: