Speed
Continuous sampling, not a quarterly bake-off
Workers in five regions send standardized requests: small, large, long-context, streaming, JSON, tool-call, reasoning, burst, and sustained load. We publish percentiles, not averages.
TTFT percentiles · US-East · Streaming
Ice = p50. Sage = p95. p99 stays in the table — averages are not published as ranks.
| Endpoint | TTFT p50 | p95 | p99 | tok/s | e2e p50 | Fail | Timeout | Evidence |
|---|---|---|---|---|---|---|---|---|
| Llama 4 70B Groq LPU | 48 ms | 84 ms | 141 ms | 890 | 720 ms | 0.14% | 0.03% | LIVE_MEASURED |
| Llama 4 405B Groq LPU | 79 ms | 141 ms | 246 ms | 420 | 1.40 s | 0.27% | 0.08% | LIVE_MEASURED |
| Llama 4 405B Self-host B200 | 84 ms | 158 ms | 282 ms | 210 | 2.80 s | 0.29% | 0.10% | REPRODUCED |
| Gemini 2.5 Flash Google API | 106 ms | 185 ms | 343 ms | 280 | 1.60 s | 0.09% | 0.02% | LIVE_MEASURED |
| Llama 4 70B Together | 123 ms | 229 ms | 422 ms | 195 | 2.40 s | 0.22% | 0.07% | LIVE_MEASURED |
| Grok 3 Mini xAI API | 158 ms | 299 ms | 537 ms | 220 | 2.10 s | 0.12% | 0.04% | LIVE_MEASURED |
| Llama 4 405B Self-host H100 | 158 ms | 317 ms | 563 ms | 85 | 6.20 s | 0.51% | 0.22% | REPRODUCED |
| Llama 4 405B Fireworks | 194 ms | 361 ms | 669 ms | 155 | 3.90 s | 0.33% | 0.12% | LIVE_MEASURED |
| GPT-5 Mini OpenAI API | 211 ms | 378 ms | 686 ms | 160 | 2.80 s | 0.18% | 0.06% | LIVE_MEASURED |
| Qwen3-Max Together | 229 ms | 449 ms | 827 ms | 92 | 5.40 s | 0.36% | 0.13% | LIVE_MEASURED |
| Llama 4 405B Together | 246 ms | 475 ms | 862 ms | 110 | 5.10 s | 0.38% | 0.14% | LIVE_MEASURED |
| DeepSeek V3.1 DeepSeek API | 273 ms | 607 ms | 1.25 s | 78 | 5.60 s | 0.55% | 0.27% | LIVE_MEASURED |
| Gemini 2.5 Pro Google API | 299 ms | 563 ms | 986 ms | 145 | 4.30 s | 0.16% | 0.05% | LIVE_MEASURED |
| Mistral Large 3 Mistral API | 308 ms | 598 ms | 1.06 s | 98 | 4.90 s | 0.31% | 0.12% | LIVE_MEASURED |
| Claude 4.1 Sonnet Anthropic API | 343 ms | 634 ms | 1.04 s | 108 | 5.20 s | 0.19% | 0.07% | LIVE_MEASURED |
| Grok 4 xAI API | 361 ms | 686 ms | 1.09 s | 138 | 4.80 s | 0.21% | 0.08% | LIVE_MEASURED |
| GPT-5 OpenAI API | 458 ms | 827 ms | 1.42 s | 95 | 6.10 s | 0.28% | 0.11% | LIVE_MEASURED |
| GPT-5 Azure OpenAI | 537 ms | 1.04 s | 1.74 s | 88 | 6.90 s | 0.41% | 0.18% | LIVE_MEASURED |
| Claude 4.1 Opus Anthropic API | 598 ms | 1.09 s | 1.85 s | 72 | 8.40 s | 0.24% | 0.09% | LIVE_MEASURED |
| Grok 4 Heavy xAI API | 783 ms | 1.48 s | 2.41 s | 62 | 14.2 s | 0.34% | 0.19% | LIVE_MEASURED |
| o4-pro OpenAI API | 2.11 s | 4.49 s | 7.74 s | 48 | 28.6 s | 0.62% | 0.41% | LIVE_MEASURED |