Hardware
Same weights. Different serving stack.
Llama 4 405B across Together, Fireworks, Groq, self-host H100, and self-host B200. MLPerf’s split — workload quality versus system performance, including measured rack power — is the rigor we borrow.
B200 tok/s
210
GBU lab, measured
Groq tok/s
420
LPU, LIVE_MEASURED speed
B200 J/token
20.0
Wall power / throughput
H100 watts
5.6 kW
TP8 rack, LIVE_MEASURED
| Stack | GPU | VRAM | TTFT | tok/s | RPS | Watts | J/tok | J/task | $/task | Evidence |
|---|---|---|---|---|---|---|---|---|---|---|
| Together H200 Llama 4 405B | H200 SXM | 141 GB | 280 ms | 110 | 9.4 | n/a (API) | — | — | $0.022 | THIRD_PARTYLIVE_MEASURED |
| Fireworks H200 Llama 4 405B | H200 SXM | 141 GB | 220 ms | 155 | 12.8 | n/a (API) | — | — | $0.024 | THIRD_PARTYLIVE_MEASURED |
| Groq LPU Llama 4 405B | Groq LPU | — | 90 ms | 420 | 38.0 | n/a (API) | — | — | $0.020 | OFFICIAL_PROVIDERLIVE_MEASURED |
| GBU lab · TP8 H100 Llama 4 405B | 8× H100 80GB | 640 GB | 180 ms | 85 | 4.2 | 5,600 | 65.9 | 34,500 | $0.011 | LIVE_MEASUREDREPRODUCED |
| GBU lab · TP4 B200 Llama 4 405B | 4× B200 192GB | 768 GB | 95 ms | 210 | 11.4 | 4,200 | 20.0 | 10,800 | $0.0078 | LIVE_MEASUREDREPRODUCED |
Power columns are empty for APIs on purpose. We do not invent joules for a black-box fleet. Lab racks are instrumented at the PDU.