public · coding
LiveCodeBench
Continuously refreshed coding problems after model cutoffs.
Items
612
Revision
2026.08
Contamination
clean
Scorer
scorer.lcb.v6
- Dataset revision
- [email protected]
- Scorer
- hidden-tests · scorer.lcb.v6
- License
- MIT
- Kind
- Public reproduction suite
| Model | Score | $/task | Median s | Tools | Evidence |
|---|---|---|---|---|---|
| GPT-5 | 84.1 | $0.081 | 12.9 | 1.4 | LIVE_MEASURED |
| Claude 4.1 Opus | 83.8 | $0.328 | 16.9 | 1.4 | LIVE_MEASURED |
| Grok 4 Heavy | 83.4 | $0.150 | 22.3 | 1.4 | LIVE_MEASURED |
| DeepSeek V3.1 | 82.8 | $0.0051 | 8.3 | 1.4 | REPRODUCED |
| o4-pro | 82.1 | $0.695 | 37.3 | 1.4 | LIVE_MEASURED |
| Grok 4 | 81.6 | $0.048 | 10.3 | 1.4 | LIVE_MEASURED |
| Gemini 2.5 Pro | 80.6 | $0.029 | 8.7 | 1.4 | LIVE_MEASURED |
| Claude 4.1 Sonnet | 80.2 | $0.058 | 11.5 | 1.4 | LIVE_MEASURED |
| Qwen3-Max | 79.4 | $0.0070 | 9.7 | 1.4 | LIVE_MEASURED |
| Mistral Large 3 | 76.8 | $0.022 | 10.0 | 1.4 | LIVE_MEASURED |
| Llama 4 405B | 73.2 | $0.0070 | 9.2 | 1.4 | REPRODUCED |
| GPT-5 Mini | 72.6 | $0.0044 | 5.1 | 1.4 | LIVE_MEASURED |
| Gemini 2.5 Flash | 70.4 | $0.0016 | 3.3 | 1.4 | LIVE_MEASURED |
| Grok 3 Mini | 64.8 | $0.0025 | 4.0 | 1.4 | LIVE_MEASURED |
| Llama 4 70B | 64.1 | $0.0016 | 1.9 | 1.4 | REPRODUCED |