public · math
FrontierMath
Original research-level math. Licensed subset only.
Items
300
Revision
2026.01
Contamination
clean
Scorer
scorer.fm.v2
- Dataset revision
- [email protected]
- Scorer
- equiv-numeric · scorer.fm.v2
- License
- Epoch / FrontierMath license
- Kind
- Public reproduction suite
| Model | Score | $/task | Median s | Tools | Evidence |
|---|---|---|---|---|---|
| o4-pro | 48.8 | $0.638 | 81.3 | 0.2 | LIVE_MEASURED |
| Grok 4 Heavy | 36.2 | $0.132 | 46.7 | 0.2 | LIVE_MEASURED |
| DeepSeek V3.1 | 34.1 | $0.0042 | 17.1 | 0.2 | REPRODUCED |
| GPT-5 | 31.2 | $0.063 | 26.5 | 0.2 | LIVE_MEASURED |
| Qwen3-Max | 29.6 | $0.0056 | 20.1 | 0.2 | LIVE_MEASURED |
| Grok 4 | 28.4 | $0.038 | 21.1 | 0.2 | LIVE_MEASURED |
| Gemini 2.5 Pro | 27.4 | $0.025 | 17.7 | 0.2 | LIVE_MEASURED |
| Claude 4.1 Opus | 26.8 | $0.267 | 34.2 | 0.2 | LIVE_MEASURED |
| Claude 4.1 Sonnet | 19.6 | $0.047 | 23.0 | 0.2 | LIVE_MEASURED |
| Mistral Large 3 | 16.2 | $0.015 | 8.3 | 0.2 | LIVE_MEASURED |
| Llama 4 405B | 14.8 | $0.0043 | 7.7 | 0.2 | REPRODUCED |
| GPT-5 Mini | 12.4 | $0.0031 | 4.2 | 0.2 | LIVE_MEASURED |
| Gemini 2.5 Flash | 11.2 | $0.0011 | 2.8 | 0.2 | LIVE_MEASURED |
| Grok 3 Mini | 8.6 | $0.0016 | 3.3 | 0.2 | LIVE_MEASURED |
| Llama 4 70B | 7.4 | $0.00094 | 1.6 | 0.2 | REPRODUCED |