public · agents
OSWorld
Desktop computer-use tasks across apps and file systems.
Items
369
Revision
2026.03
Contamination
watched
Scorer
scorer.osworld.v4
- Dataset revision
- [email protected]
- Scorer
- desktop-oracle · scorer.osworld.v4
- License
- MIT
- Kind
- Public reproduction suite
| Model | Score | $/task | Median s | Tools | Evidence |
|---|---|---|---|---|---|
| Claude 4.1 Opus | 54.8 | $0.328 | 49.8 | 3.8 | LIVE_MEASURED |
| Claude 4.1 Sonnet | 48.2 | $0.058 | 33.6 | 3.4 | LIVE_MEASURED |
| Grok 4 Heavy | 47.6 | $0.150 | 64.5 | 2.8 | LIVE_MEASURED |
| GPT-5 | 46.8 | $0.081 | 37.1 | 3.0 | LIVE_MEASURED |
| Grok 4 | 44.2 | $0.048 | 29.6 | 3.2 | LIVE_MEASURED |
| Gemini 2.5 Pro | 43.6 | $0.029 | 24.9 | 2.9 | LIVE_MEASURED |
| o4-pro | 38.2 | $0.695 | 104.9 | 2.2 | LIVE_MEASURED |
| Mistral Large 3 | 36.8 | $0.022 | 28.4 | 2.9 | LIVE_MEASURED |
| DeepSeek V3.1 | 33.6 | $0.0051 | 22.8 | 2.7 | REPRODUCED |
| Qwen3-Max | 32.1 | $0.0070 | 27.0 | 2.8 | LIVE_MEASURED |
| Llama 4 405B | 31.4 | $0.0070 | 25.9 | 2.6 | REPRODUCED |
| GPT-5 Mini | 28.1 | $0.0044 | 14.3 | 2.4 | LIVE_MEASURED |
| Gemini 2.5 Flash | 27.8 | $0.0016 | 9.4 | 2.2 | LIVE_MEASURED |
| Grok 3 Mini | 22.4 | $0.0025 | 11.1 | 2.1 | LIVE_MEASURED |
| Llama 4 70B | 21.2 | $0.0016 | 5.3 | 2.0 | REPRODUCED |