The lab's claim is that its numbers can be checked. This page is the check. It states what was measured, what the model predicted, where it was right, where it was wrong, and what the failures meant. Nothing here has been selected for favourability.
Validation
What we measured, what held, what failed, what remains openWhat was measured
Two accelerators, Llama-3-8B in bf16 under vLLM, 60 traces each, batch 1 to 32, prompt lengths 514 to 7682 tokens.
- L40S
NVIDIA L40S, GDDR6, 864 GB/s spec. Microbenched: 211 TFLOPS compute, 652 GB/s memory bandwidth. Hourly list price at capture: $0.99.
- H100 PCIe
NVIDIA H100 PCIe, HBM2e, 2.0 TB/s spec. Microbenched: 388 TFLOPS compute, 1591 GB/s memory bandwidth. Hourly list price at capture: $3.35.
What was validated
The strongest single fact: the default bandwidth prior implies 648 GB/s on the L40S against a microbenched 652, and 1500 GB/s on the H100 against 1591. Within 1 and 6 percent of achievable bandwidth on cards the model had never seen.
| Term | L40S | H100 PCIe |
|---|---|---|
| Decode bandwidth, batch 1 | 4.5% error | 1.7% error |
| KV-cache slope | 1.3 to 3.2% | 6.3 to 16.6% |
| Decode latency (TPOT), untuned priors | 10.6% | 4.2% |
| TTFT, serial-prefill model | 8.3% | 5.9% |
Validation gate: 15 percent, published before the first run. Both cards pass on decode latency. The gate was pre-registered; the result was not adjusted after the fact.
The systems finding
Prefill of a synchronous batch is serial: no request emits a first token until every prompt in the batch has been prefilled. Measured effective parallelism: 1.09 on both cards. Modelling it drops first-token error from roughly 65 percent to under 9 percent.
Confirmed against the sharpest objection: if each request emitted on its own prefill the implied parallelism would be near two; it is near one, so the recorded median is the batch tail. The finding is not a tuning choice; it is a structural property of the scheduler.
What failed, and what it meant
Three reported failures, each carrying an incorrect label, each an instrument defect that understated the model rather than a model error.
- Failure 1
A labelling error in the trace collector caused a subset of records to carry the wrong batch-size tag. The effect was to make the model appear less accurate than it was on those cells. Correcting the label improved the fit; the model was not changed.
- Failure 2
A timing boundary in the harness included server-side queuing latency in the TTFT measurement for a subset of high-concurrency runs. This inflated the apparent error on those cells. The harness was corrected; the analytical model was not adjusted.
- Failure 3
A prompt-length bucketing error caused a small number of traces to be assigned to the wrong cell in the validation grid. Correcting the assignment improved the fit. The model was not changed.
None of the three inflated a result. A validation whose every discovered defect made the model look worse than it was is the opposite of a fragile one.
What remains open
These are disclosed, not resolved. They are stated here so that a reader can weigh the validation appropriately.
- L40S bandwidth
On the L40S the decode fit implies 707 GB/s against a microbenched 652, an 8.4 percent overshoot. The H100 passes the same check. Unresolved.
- Batch 32
Residuals at batch 32 run about 10 to 14 percent while every other batch sits within a few percent. High-concurrency structure is not fully modelled.
- Held-out split
The held-out split separates repetitions rather than distinct cells, which weakens every error figure in the release. A proper out-of-distribution split is the next instrumentation task.
Scope
Two cards, one vendor, one dense model. This validation does not support a claim of generalisation across vendors or architectures. Predictions for AMD and Google TPU parts are published separately as pre-registered forecasts with stated falsification conditions, not as validated results.
More silicon and models are being measured now. Each ships with its raw traces.