This leaderboard is based on the following benchmarks.
- Arena - a crowdsourced, randomized battle platform for large language models (LLMs). We use 6M+ user votes to compute Elo ratings.
- AAII - Artificial Analysis Intelligence Index v3 aggregating 10 challenging evaluations.
- ARC-AGI - Artificial General Intelligence benchmark v2 to measure fluid intelligence.
| Model | Arena Elo | Coding | Vision | AAII | MMLU-Pro | ARC-AGI | Organization | License | |
|---|---|---|---|---|---|---|---|---|---|
| 🏆 | Claude Fable 5 | 1510 | 1566 | 1310 | 81 | 91.5 | 86 |  Anthropic | Proprietary |
| 🥇 | Claude Opus 4.8 Thinking | 1506 | 1562 | 1311 | 77 | 90.1 | 76 | Anthropic | Proprietary |
| 🥇 | GPT-5.5-high | 1506 | 1561 | 1312 | 76 | 89.6 | 85 | OpenAI | Proprietary |
| 🥇 | Claude Opus 4.7 Thinking | 1505 | 1560 | 1310 | 76 | 90 | 75.8 | Anthropic | Proprietary |
| 🥇 | Gemini-3.1-Pro | 1505 | 1531 | 1309 | 76 | 91 | 77.1 |  Google | Proprietary |
| 🥇 | Claude Opus 4.8 | 1504 | 1558 | 1306 | 74 | 90 | 62.2 | Anthropic | Proprietary |
| 🥇 | Gemini-3.5-Flash | 1504 | 1535 | 1301 | 74 | 91 | 72.1 | Google | Proprietary |
| 🥇 | Claude Opus 4.7 | 1503 | 1554 | 1300 | 73 | 89.9 | 62.1 | Anthropic | Proprietary |
| 🥇 | Claude Opus 4.6 Thinking | 1503 | 1545 | 1304 | 73 | 89.7 | 69.2 | Anthropic | Proprietary |
| 🥇 | Grok-4.20 | 1496 | 1518 | 1279 | 72 | 89.6 | 65.1 | xAI | Proprietary |
| 🥇 | GPT-5.4-high | 1495 | 1538 | 1290 | 73 | 88.5 | 74 | OpenAI | Proprietary |
| 🥇 | Gemini-3-Pro | 1492 | 1501 | 1308 | 73 | 90 | 33.6 | Google | Proprietary |
| 🥇 | Claude Opus 4.6 | 1490 | 1535 | 1298 | 71 | 89.5 | 64.6 | Anthropic | Proprietary |
| 🥇 | GLM-5.2 ✅ | 1488 | 1525 | 73 | 87.5 | 22.8 | Z.ai | MIT | |
| 🥇 | Qwen3.7-Max | 1486 | 1505 | 1289 | 72 | 89.6 |  Alibaba | Proprietary | |
| 🥇 | Muse Spark | 1485 | 1493 | 1294 | 71 | 87.3 |  Meta | Proprietary | |
| 🥇 | Grok-4.1-Thinking | 1482 | 1483 | 70 | 89 | 26 | xAI | Proprietary | |
| 🥇 | ERNIE-5.1 | 1475 | 1495 | 71 | 87.1 |  Baidu | Proprietary | ||
| 🥇 | Claude Opus 4.5 (thinking-32k) | 1469 | 1510 | 70 | 89.5 | 30.6 | Anthropic | Proprietary | |
| 🥇 | Claude Sonnet 4.6 Thinking | 1468 | 1511 | 1278 | 71 | 88 | 60.4 | Anthropic | Proprietary |
| 🥈 | GLM-5.1 | 1467 | 1506 | 71 | 87.1 | 5.1 | Z.ai | MIT | |
| 🥈 | DeepSeek-V4-Pro ✅ | 1467 | 1491 | 71 | 87.5 |  DeepSeek | MIT | ||
| 🥈 | Seed2.0 Pro | 1466 | 1495 | 1288 | 70 | 87.8 |  ByteDance | Proprietary | |
| 🥈 | Kimi-K2.6-Thinking ✅ | 1466 | 1493 | 1286 | 71 | 87.3 |  Moonshot | Modified MIT | |
| 🥈 | Qwen3.5-Max | 1466 | 1493 | 70 | 87.8 | Alibaba | Proprietary | ||
| 🥈 | MiMo-V2.5-Pro | 1466 | 1485 | 70 | 87 |  Xiaomi | MIT | ||
| 🥈 | GPT-5.2-high | 1465 | 1470 | 1280 | 72 | 87.5 | 52.9 | OpenAI | Proprietary |
| 🥈 | Gemini-3-Flash | 1465 | 1469 | 1292 | 71 | 89 | 31.1 | Google | Proprietary |
| 🥈 | GPT-5.4 | 1465 | 1468 | 1275 | 70 | 88.4 | 29.2 | OpenAI | Proprietary |
| 🥈 | GPT-5.1-high | 1464 | 1466 | 1250 | 70 | 87.1 | 17.6 | OpenAI | Proprietary |
| 🥈 | GPT-5.2 | 1464 | 1465 | 1248 | 68 | 87.4 | 26.7 | OpenAI | Proprietary |
| 🥈 | Grok-4.1 | 1463 | 1463 | 68 | 88 | xAI | Proprietary | ||
| 🥈 | Claude Opus 4.5 | 1462 | 1496 | 60 | 88.8 | 7.8 | Anthropic | Proprietary | |
| 🥈 | GPT-5.4-mini-high | 1461 | 1461 | 68 | 87 | 18.9 | OpenAI | Proprietary | |
| 🥈 | Claude Sonnet 4.6 | 1460 | 1500 | 1277 | 65 | 87.3 | 37.6 | Anthropic | Proprietary |
| 🥈 | Gemini-2.5-Pro | 1460 | 1465 | 1266 | 63 | 86.2 | 4.9 | Google | Proprietary |
| 🥈 | ERNIE-5.0 | 1458 | 1461 | 1251 | 65 | 86 | Baidu | Proprietary | |
| 🥈 | Qwen3.6-Plus | 1456 | 1482 | 70 | 88.5 | Alibaba | Proprietary | ||
| 🥈 | Minimax-M3 | 1452 | 1485 | 70 | 87.2 |  MiniMax | Non-commercial | ||
| 🥈 | GLM-5 | 1452 | 1461 | 70 | 87 | 5 | Z.ai | MIT | |
| 🥈 | Kimi-K2.5-Thinking | 1451 | 1480 | 1271 | 69 | 87.1 | 11.8 | Moonshot | Modified MIT |
| 🥈 | Qwen3.5-397B-A17B | 1450 | 1463 | 69 | 87.8 | 5 | Alibaba | Apache 2.0 | |
| 🥉 | Gemma-4-31B-it | 1449 | 1462 | 63 | 85.2 | Google | Apache 2.0 | ||
| 🥉 | GLM-4.7 | 1445 | 1460 | 68 | 85.6 | Z.ai | MIT | ||
| 🥉 | DeepSeek-V4-Flash | 1445 | 1460 | 64 | 86.2 | DeepSeek | MIT | ||
| 🥉 | GPT-5-high | 1444 | 1460 | 1232 | 66 | 87.1 | 9.9 | OpenAI | Proprietary |
| 🥉 | Qwen3-Max | 1443 | 1468 | 64 | 85.3 | Alibaba | Proprietary | ||
| 🥉 | Gemma-4-26B-A4B-it | 1443 | 1460 | 56 | 82.6 | Google | Apache 2.0 | ||
| 🥉 | Grok-4 | 1442 | 1453 | 1221 | 66 | 86.6 | 16 | xAI | Proprietary |
| 🥉 | GLM-4.6 | 1441 | 1458 | 56 | 83.5 | Z.ai | MIT | ||
| 🥉 | GPT-5.1 | 1440 | 1450 | 1243 | 66 | 87 | 7.5 | OpenAI | Proprietary |
| 🥉 | Kimi-K2-Thinking | 1438 | 1450 | 67 | 84.8 | Moonshot | Modified MIT | ||
| 🥉 | Claude Sonnet 4.5 (thinking-32k) | 1432 | 1485 | 61 | 87.5 | 13.6 | Anthropic | Proprietary | |
| 🥉 | MiMo-V2-Pro | 1431 | 1441 | 69 | 86.8 | Xiaomi | Proprietary | ||
| 🥉 | GLM-4.5 | 1430 | 1448 | 54 | 83.5 | Z.ai | MIT | ||
| 🥉 | Qwen3-VL-235B-A22B-Instruct | 1429 | 1457 | 1246 | 49 | 82.8 | Alibaba | Apache 2.0 | |
| 🥉 | Mistral Large 3 | 1428 | 1450 | 40 | 81 |  Mistral | Apache 2.0 | ||
| 🥉 | ChatGPT-4o-latest (2025-03-26) | 1427 | 1434 | 1242 | 38 | 80.3 | OpenAI | Proprietary | |
| 🥉 | DeepSeek-R1-0528 | 1426 | 1436 | 57 | 84.9 | 1.3 | DeepSeek | MIT | |
| 🥉 | Claude Opus 4.1 (thinking-16k) | 1425 | 1475 | 59 | 87.8 | Anthropic | Proprietary | ||
| 🥉 | o3-2025-04-16 | 1424 | 1441 | 1218 | 65 | 85.3 | 6.5 | OpenAI | Proprietary |
| 🥉 | Grok-3-Preview-02-24 | 1423 | 1439 | 44 | 79.9 | xAI | Proprietary | ||
| 🥉 | Hunyuan-Hy3 | 1422 | 1448 | 66 | 86.2 |  Tencent | Tencent | ||
| 🥉 | DeepSeek-V3.2-Thinking | 1422 | 1438 | 66 | 86.2 | 4 | DeepSeek | MIT | |
| 🪙 | Gemini-3.1-Flash-Lite | 1421 | 1402 | 1230 | 60 | 86.2 | Google | Proprietary | |
| 🪙 | Claude Sonnet 4.5 | 1420 | 1464 | 49 | 86 | Anthropic | Proprietary | ||
| 🪙 | LongCat-Flash-Chat | 1420 | 1461 | 49 | 82.7 |  Meituan | MIT | ||
| 🪙 | Claude Opus 4.1 | 1419 | 1465 | 47 | 87.3 | Anthropic | Proprietary | ||
| 🪙 | Grok-4-Fast | 1419 | 1441 | 62 | 85 | xAI | Proprietary | ||
| 🪙 | Qwen3-235B-A22B-Instruct-2507 | 1418 | 1457 | 49 | 82.8 | 1.3 | Alibaba | Apache 2.0 | |
| 🪙 | Nemotron-3-Ultra-550B-A55B | 1418 | 1452 | 70 | 86.8 |  Nvidia | Nvidia Open | ||
| 🪙 | DeepSeek-V3.1-Thinking | 1418 | 1437 | 58 | 85.1 | DeepSeek | MIT | ||
| 🪙 | DeepSeek-V3.2 | 1418 | 1431 | 52 | 83.7 | DeepSeek | MIT | ||
| 🪙 | Qwen3-Next-80B-A3B-Instruct | 1417 | 1456 | 57 | 82.4 | Alibaba | Apache 2.0 | ||
| 🪙 | Amazon-Nova-Chat-11-10 | 1417 | 1431 | 62 | 83 |  Amazon | Proprietary | ||
| 🪙 | DeepSeek-V3.1 | 1417 | 1430 | 47 | 83.3 | DeepSeek | MIT | ||
| 🪙 | Minimax-M2.7 | 1416 | 1448 | 69 | 87.1 | MiniMax | Non-commercial | ||
| 🪙 | Qwen3-235B-A22B-Thinking-2507 | 1416 | 1442 | 62 | 84.3 | Alibaba | Apache 2.0 | ||
| 🪙 | GPT-4.5-Preview | 1415 | 1419 | 1190 | 40 | 81 | 0.8 | OpenAI | Proprietary |
| 🪙 | GPT-5-chat | 1413 | 1421 | 1238 | 65 | 86.6 | 7.5 | OpenAI | Proprietary |
| 🪙 | Gemini-2.5-Flash | 1412 | 1420 | 1235 | 56 | 83.2 | 2.5 | Google | Proprietary |
| 🪙 | Qwen3-VL-235B-A22B-Thinking | 1411 | 1432 | 1215 | 62 | 84.3 | Alibaba | Apache 2.0 | |
| 🪙 | Hunyuan-Vision-1.5-Thinking | 1410 | 1419 | 1220 | 86.2 | Tencent | Proprietary | ||
| 🪙 | Minimax-M2.5 | 1408 | 1436 | 65 | 87 | 4.9 | MiniMax | Modified MIT | |
| 🪙 | MiMo-V2-Flash | 1402 | 1411 | 61 | 84.8 | Xiaomi | MIT | ||
| 🪙 | Mistral Medium 3.1 | 1401 | 1412 | 38 | 77.2 | Mistral | Proprietary | ||
| 🪙 | Hunyuan-T1-20250711 | 1400 | 1409 | 86.2 | Tencent | Proprietary | |||
| 🪙 | Minimax-M2.1 | 1399 | 1430 | 64 | 87 | MiniMax | Modified MIT | ||
| 🪙 | MAI-1-Preview | 1398 | 1406 |  Microsoft | Proprietary | ||||
| 🪙 | Gemini-2.0-Pro-Exp-02-05 | 1398 | 1396 | 1170 | 36 | 80.5 | Google | Proprietary | |
| 🪙 | Gemini-2.0-Flash-Thinking-Exp-01-21 | 1397 | 1383 | 1208 | 40 | 79.8 | Google | Proprietary | |
| 🪙 | Step-3.5-Flash | 1387 | 1433 |  StepFun | Apache 2.0 | ||||
| 🪙 | GLM-4.5-Air | 1386 | 1410 | 47 | 81.5 | Z.ai | MIT | ||
| 🪙 | Qwen3-30B-A3B-Instruct-2507 | 1382 | 1425 | 44 | 77.7 | Alibaba | Apache 2.0 | ||
| 🪙 | Kimi-K2-0905-Preview | 1382 | 1403 | 49 | 82.4 | Moonshot | Modified MIT | ||
| 🪙 | Qwen-VL-Max-2025-08-13 | 1381 | 1440 | 1213 | Alibaba | Proprietary | |||
| 🪙 | GPT-4.1-2025-04-14 | 1381 | 1396 | 1207 | 45 | 80.6 | 0.4 | OpenAI | Proprietary |
| 🪙 | Kimi-K2-0711-Preview | 1380 | 1402 | 47 | 82.4 | Moonshot | Modified MIT | ||
| 🪙 | Claude Haiku 4.5 | 1378 | 1436 | 42 | 80 | Anthropic | Proprietary | ||
| 🪙 | DeepSeek-V3-0324 | 1377 | 1391 | 42 | 81.9 | DeepSeek | MIT | ||
| 🪙 | Hunyuan-Turbos-20250416 | 1377 | 1390 | 78 | Tencent | Proprietary | |||
| 🪙 | Claude Opus 4 (thinking-16k) | 1376 | 1434 | 1188 | 57 | 87.3 | 8.6 | Anthropic | Proprietary |
| 🪙 | GPT-5-mini | 1375 | 1419 | 1214 | 62 | 82.8 | 4.4 | OpenAI | Proprietary |
| 🪙 | DeepSeek-R1 | 1373 | 1382 | 48 | 84.4 | 1.3 | DeepSeek | MIT | |
| 🪙 | Gemini-2.0-Flash-Exp | 1370 | 1371 | 1191 | 36 | 78.2 | 1.3 | Google | Proprietary |
| 🪙 | Qwen3-235B-A22B | 1369 | 1394 | 46 | 82.8 | Alibaba | Apache 2.0 | ||
| 🪙 | Mistral Medium 3 | 1369 | 1387 | 1159 | 37 | 76 | Mistral | Proprietary | |
| 🪙 | gpt-oss-120b | 1368 | 1398 | 59 | 80.8 | OpenAI | Apache 2.0 | ||
| 🪙 | Qwen2.5-Max | 1367 | 1373 | 32 | 76.2 | Alibaba | Proprietary | ||
| 🪙 | Claude Opus 4 | 1366 | 1405 | 1173 | 45 | 86 | 1.3 | Anthropic | Proprietary |
| 🪙 | Grok-3-mini-high | 1366 | 1380 | 56 | 82.8 | xAI | Proprietary | ||
| 🪙 | o1-2024-12-17 | 1366 | 1378 | 1166 | 50 | 84.1 | 1.3 | OpenAI | Proprietary |
| 🪙 | o4-mini-2025-04-16 | 1362 | 1385 | 1192 | 63 | 83.2 | 6.1 | OpenAI | Proprietary |
| 🪙 | Step-3 | 1360 | 1400 | 1190 | StepFun | Proprietary | |||
| 🪙 | Qwen3-Coder-480B-A35B-Instruct | 1358 | 1406 | 43 | 78.8 | Alibaba | Apache 2.0 | ||
| 🪙 | Nemotron-3-Super-120B-A12B | 1357 | 1401 | 60 | 83.7 | Nvidia | Nvidia Open | ||
| 🪙 | Gemma-3-27B-it | 1356 | 1350 | 1163 | 23 | 66.9 | Google | Gemma | |
| 🪙 | INTELLECT-3 | 1355 | 1376 | Prime Intellect | MIT | ||||
| 🪙 | Claude Sonnet 4 (thinking-32k) | 1351 | 1412 | 1187 | 57 | 84.2 | 5.9 | Anthropic | Proprietary |
| 🪙 | Minimax-M1 | 1351 | 1369 | 51 | 81.6 | MiniMax | Apache 2.0 | ||
| 🪙 | Qwen3-32B | 1342 | 1376 | 42 | 79.8 | Alibaba | Apache 2.0 | ||
| 🪙 | Llama-3.3-Nemotron-Super-49B-v1.5 | 1340 | 1359 | 50 | 81.4 | Nvidia | Nvidia Open | ||
| 🪙 | Step-1o-Turbo-202506 | 1339 | 1361 | 1182 | StepFun | Proprietary | |||
| 🪙 | o3-mini-high | 1338 | 1380 | 53 | 80.2 | 3 | OpenAI | Proprietary | |
| 🪙 | GPT-4.1-mini-2025-04-14 | 1338 | 1370 | 1177 | 40 | 78.1 | OpenAI | Proprietary | |
| 🪙 | Gemini-2.5-Flash-Lite | 1337 | 1362 | 1205 | 42 | 75.9 | Google | Proprietary | |
| 🪙 | Mistral-Small-3.2-2506 | 1337 | 1361 | 1145 | 30 | 68.1 | Mistral | Apache 2.0 | |
| 🪙 | Claude Sonnet 4 | 1335 | 1384 | 1170 | 44 | 83.7 | 1.3 | Anthropic | Proprietary |
| 🪙 | Gemma-3-12B-it | 1335 | 1310 | 22 | 59.5 | Google | Gemma | ||
| 🪙 | DeepSeek-V3 | 1334 | 1337 | 33 | 75.2 | DeepSeek | DeepSeek | ||
| 🪙 | GPT-5-nano | 1333 | 1363 | 1166 | 52 | 77.2 | 2.6 | OpenAI | Proprietary |
| 🪙 | QwQ-32B | 1332 | 1351 | 46 | 76.4 | Alibaba | Apache 2.0 | ||
| 🪙 | GLM-4-Plus-0111 | 1332 | 1310 | 78.6 | Z.ai | Proprietary | |||
| 🪙 | Gemini-2.0-Flash-Lite | 1330 | 1338 | 1097 | 28 | 72.4 | Google | Proprietary | |
| 🪙 | Qwen-Plus-0125 | 1327 | 1339 | Alibaba | Proprietary | ||||
| 🪙 | Command A (03-2025) | 1327 | 1336 | 30 | 71.2 |  Cohere | CC-BY-NC-4.0 | ||
| 🪙 | Amazon-Nova-Chat-05-14 | 1324 | 1337 | 33 | 73.3 | Amazon | Proprietary | ||
| 🪙 | Llama-3.1-Nemotron-Ultra-253B-v1 | 1321 | 1345 | 44 | 82.5 | Nvidia | Nvidia Open | ||
| 🪙 | Step-2-16K-Exp | 1321 | 1313 | StepFun | Proprietary | ||||
| 🪙 | Qwen3-30B-A3B | 1320 | 1346 | 40 | 77.7 | Alibaba | Apache 2.0 | ||
| 🪙 | Gemini-1.5-Pro-002 | 1320 | 1311 | 1158 | 32 | 75 | 0.8 | Google | Proprietary |
| 🪙 | o1-mini | 1318 | 1366 | 41 | 74.2 | 0.8 | OpenAI | Proprietary | |
| 🪙 | o3-mini | 1318 | 1361 | 51 | 79.1 | 2.1 | OpenAI | Proprietary | |
| 🪙 | Claude 3.7 Sonnet (thinking-32k) | 1316 | 1355 | 1167 | 45 | 83.7 | 0.9 | Anthropic | Proprietary |
| 🪙 | gpt-oss-20b | 1315 | 1371 | 47 | 73.6 | OpenAI | Apache 2.0 | ||
| 🪙 | Hunyuan-Turbo-0110 | 1314 | 1335 | Tencent | Proprietary | ||||
| 🪙 | Llama-3.3-Nemotron-Super-49B-v1 | 1310 | 1320 | 38 | 78.5 | Nvidia | Nvidia Open | ||
| 🪙 | OLMo-3-32b-think | 1306 | 1327 | 36 | 75.9 |  Ai2 | Apache-2.0 | ||
| 🪙 | Grok-2-08-13 | 1305 | 1298 | 26 | 70.9 | xAI | Grok 2 | ||
| 🪙 | Gemma-3n-e4b-it | 1304 | 1297 | 16 | 48.8 | Google | Gemma | ||
| 🪙 | Yi-Lightning | 1303 | 1321 |  01 AI | Proprietary | ||||
| 🪙 | GPT-4o-2024-05-13 | 1302 | 1307 | 1134 | 28 | 74.8 | OpenAI | Proprietary | |
| 🪙 | Claude 3.7 Sonnet | 1301 | 1341 | 1145 | 35 | 80.3 | Anthropic | Proprietary | |
| 🪙 | Claude 3.5 Sonnet (20241022) | 1299 | 1340 | 1122 | 31 | 77.2 | Anthropic | Proprietary | |
| 🪙 | Deepseek-v2.5-1210 | 1296 | 1316 | 22 | 67.2 | DeepSeek | DeepSeek | ||
| 🪙 | Athene-v2-Chat-72B | 1294 | 1320 | NexusFlow | NexusFlow | ||||
| 🪙 | Gemma-3-4B-it | 1293 | 1265 | 12 | 41.7 | Google | Gemma | ||
| 🪙 | Llama-4-Maverick-17B-128E-Instruct | 1292 | 1312 | 1135 | 40 | 80.9 | Meta | Llama 4 | |
| 🪙 | GLM-4-Plus | 1292 | 1301 | 70.2 | Z.ai | Proprietary | |||
| 🪙 | Hunyuan-Large-2025-02-10 | 1291 | 1311 | Tencent | Proprietary | ||||
| 🪙 | Gemini-1.5-Flash-002 | 1290 | 1273 | 1137 | 26 | 68 | Google | Proprietary | |
| 🪙 | GPT-4o-mini-2024-07-18 | 1289 | 1300 | 1063 | 22 | 64.8 | OpenAI | Proprietary | |
| 🪙 | GPT-4.1-nano-2025-04-14 | 1287 | 1312 | 1060 | 28 | 65.7 | OpenAI | Proprietary | |
| 🪙 | Llama-3.1-405B-Instruct-bf16 | 1286 | 1299 | 27 | 73.2 | Meta | Llama 3.1 | ||
| 🪙 | Llama-3.1-Nemotron-70B-Instruct | 1285 | 1289 | 24 | 69 | Nvidia | Llama 3.1 | ||
| 🪙 | Qwen-Max-0919 | 1284 | 1296 | Alibaba | Proprietary | ||||
| 🪙 | Llama-3.1-405B-Instruct-fp8 | 1284 | 1292 | 27 | 73.2 | Meta | Llama 3.1 | ||
| 🪙 | Yi-Lightning-lite | 1284 | 1286 | 01 AI | Proprietary | ||||
| 🪙 | Claude 3.5 Sonnet (20240620) | 1283 | 1309 | 1117 | 27 | 75.1 | Anthropic | Proprietary | |
| 🪙 | Grok-2-mini-08-13 | 1283 | 1279 | xAI | Proprietary | ||||
| 🪙 | Llama-4-Scout-17B-16E-Instruct | 1276 | 1290 | 1126 | 31 | 75.2 | Meta | Llama 4 | |
| 🪙 | Hunyuan-Standard-2025-02-10 | 1276 | 1289 | Tencent | Proprietary | ||||
| 🪙 | Llama-3.3-70B-Instruct | 1276 | 1279 | 29 | 71.3 | Meta | Llama 3.3 | ||
| 🪙 | Deepseek-v2.5 | 1275 | 1306 | 21 | 66.2 | DeepSeek | DeepSeek | ||
| 🪙 | GPT-4-Turbo-2024-04-09 | 1275 | 1280 | 1087 | 26 | 69.4 | OpenAI | Proprietary | |
| 🪙 | Qwen2.5-72B-Instruct | 1272 | 1302 | 27 | 72 | Alibaba | Qwen | ||
| 🪙 | Hunyuan-Large-Vision | 1270 | 1298 | 1183 | Tencent | Proprietary | |||
| 🪙 | Mistral-Small-3.1-24B-Instruct-2503 | 1269 | 1295 | 1121 | 22 | 65.9 | Mistral | Apache 2.0 | |
| 🪙 | Mistral-Large-2411 | 1269 | 1284 | 25 | 69.7 | Mistral | MRL | ||
| 🪙 | Athene-70B | 1268 | 1274 | NexusFlow | CC-BY-NC-4.0 | ||||
| 🪙 | GPT-4-1106-preview | 1267 | 1269 | 23 | 63.7 | OpenAI | Proprietary | ||
| 🪙 | GPT-4-0125-preview | 1266 | 1261 | OpenAI | Proprietary | ||||
| 🪙 | Claude 3 Opus | 1265 | 1269 | 1020 | 22 | 69.6 | Anthropic | Proprietary | |
| 🪙 | Llama-3.1-70B-Instruct | 1265 | 1268 | 22 | 67.6 | Meta | Llama 3.1 | ||
| 🪙 | Amazon Nova Pro 1.0 | 1262 | 1282 | 979 | 27 | 69.1 | Amazon | Proprietary | |
| 🪙 | Llama-3.1-Tulu-3-70B | 1260 | 1251 | Ai2 | Llama 3.1 | ||||
| 🪙 | Claude 3.5 Haiku (20241022) | 1256 | 1287 | 1095 | 21 | 63.4 | Anthropic | Proprietary | |
| 🪙 | magistral-medium-2506 | 1253 | 1307 | 36 | 75.3 | Mistral | Proprietary | ||
| 🪙 | Reka-Core-20240904 | 1252 | 1238 | 20 |  Reka AI | Proprietary | |||
| 🪙 | Reka-Core-20240722 | 1250 | 1226 | Reka AI | Proprietary | ||||
| 🪙 | Qwen-Plus-0828 | 1242 | 1263 | Alibaba | Proprietary | ||||
| 🪙 | Jamba-1.5-Large | 1242 | 1244 | 16 | 57.2 |  AI21 Labs | Jamba Open | ||
| 🪙 | Deepseek-v2-API-0628 | 1240 | 1260 | DeepSeek | DeepSeek | ||||
| 🪙 | Mistral-Small-3-24B-Instruct-2501 | 1238 | 1251 | 22 | 65.2 | Mistral | Apache 2.0 | ||
| 🪙 | Deepseek-Coder-v2-0724 | 1237 | 1286 | 15 | 58.5 | DeepSeek | DeepSeek | ||
| 🪙 | Yi-Large | 1236 | 1238 | 14 | 58.6 | 01 AI | Proprietary | ||
| 🪙 | Gemma-2-27B-it | 1236 | 1226 | 18 | 57.5 | Google | Gemma | ||
| 🪙 | Qwen2.5-Coder-32B-Instruct | 1235 | 1279 | 23 | 63.5 | Alibaba | Apache 2.0 | ||
| 🪙 | Amazon Nova Lite 1.0 | 1233 | 1253 | 989 | 23 | 59 | Amazon | Proprietary | |
| 🪙 | Gemma-2-9B-it-SimPO | 1233 | 1211 | Princeton | MIT | ||||
| 🪙 | Command R+ (08-2024) | 1233 | 1200 | 7 | 43.2 | Cohere | CC-BY-NC-4.0 | ||
| 🪙 | Gemini-1.5-Flash-8B-001 | 1231 | 1228 | 1042 | 17 | 56.9 | Google | Proprietary | |
| 🪙 | Llama-3.1-Nemotron-51B-Instruct | 1231 | 1227 | Nvidia | Llama 3.1 | ||||
| 🪙 | GLM-4-0520 | 1230 | 1237 | Z.ai | Proprietary | ||||
| 🪙 | Nemotron-4-340B-Instruct | 1229 | 1220 | Nvidia | Nvidia Open | ||||
| 🪙 | Aya-Expanse-32B | 1229 | 1211 | 6 | 37.7 | Cohere | CC-BY-NC-4.0 | ||
| 🪙 | Reka-Flash-20240904 | 1225 | 1208 | 20 | Reka AI | Proprietary | |||
| 🪙 | Llama-3-70B-Instruct | 1224 | 1216 | 14 | 57.4 | Meta | Llama 3 | ||
| 🪙 | Claude 3 Sonnet | 1223 | 1232 | 983 | 14 | 57.9 | Anthropic | Proprietary | |
| 🪙 | OLMo-2-0325-32B-Instruct | 1223 | 1215 | 14 | 51.1 | Ai2 | Apache-2.0 | ||
| 🪙 | Phi-4 | 1222 | 1242 | 26 | 71.4 | Microsoft | MIT | ||
| 🪙 | Reka-Flash-20240722 | 1218 | 1201 | Reka AI | Proprietary | ||||
| 🪙 | Amazon Nova Micro 1.0 | 1215 | 1228 | 18 | 53.1 | Amazon | Proprietary | ||
| 🪙 | Gemma-2-9B-it | 1213 | 1194 | 8 | 49.5 | Google | Gemma | ||
| 🪙 | Hunyuan-Standard-256K | 1209 | 1244 | Tencent | Proprietary | ||||
| 🪙 | Command R+ (04-2024) | 1209 | 1184 | 6 | 42.7 | Cohere | CC-BY-NC-4.0 | ||
| 🪙 | Qwen2-72B-Instruct | 1208 | 1206 | 19 | 62.2 | Alibaba | Qianwen | ||
| 🪙 | Claude 3 Haiku | 1200 | 1208 | 950 | 10 | 50 | Anthropic | Proprietary | |
| 🪙 | Llama-3.1-Tulu-3-8B | 1200 | 1197 | Ai2 | Llama 3.1 | ||||
| 🪙 | Qwen-Max-0428 | 1199 | 1208 | Alibaba | Proprietary | ||||
| 🪙 | Ministral-8B-2410 | 1198 | 1219 | 8 | 38.9 | Mistral | MRL | ||
| 🪙 | GLM-4-0116 | 1198 | 1209 | Z.ai | Proprietary | ||||
| 🪙 | DeepSeek-Coder-V2-Instruct | 1196 | 1259 | DeepSeek | DeepSeek | ||||
| 🪙 | Command R (08-2024) | 1195 | 1180 | 1 | 33.8 | Cohere | CC-BY-NC-4.0 | ||
| 🪙 | Llama-3.1-8B-Instruct | 1193 | 1203 | 10 | 47.6 | Meta | Llama 3.1 | ||
| 🪙 | Jamba-1.5-Mini | 1193 | 1197 | AI21 Labs | Jamba Open | ||||
| 🪙 | Aya-Expanse-8B | 1193 | 1184 | 2 | 31.2 | Cohere | CC-BY-NC-4.0 | ||
| 🪙 | Qwen1.5-110B-Chat | 1180 | 1192 | 11 | Alibaba | Qianwen | |||
| 🪙 | Yi-1.5-34B-Chat | 1178 | 1181 | 01 AI | Apache-2.0 | ||||
| 🪙 | Claude-1 | 1178 | 1161 | Anthropic | Proprietary | ||||
| 🪙 | Qwen1.5-72B-Chat | 1172 | 1175 | Alibaba | Qianwen | ||||
| 🪙 | Mistral Medium | 1171 | 1172 | 9 | 49.1 | Mistral | Proprietary | ||
| 🪙 | Llama-3-8B-Instruct | 1171 | 1164 | 7 | 40.5 | Meta | Llama 3 | ||
| 🪙 | Command R (04-2024) | 1169 | 1141 | 1 | 33.7 | Cohere | CC-BY-NC-4.0 | ||
| 🪙 | InternLM2.5-20B-chat | 1168 | 1179 | InternLM | Other | ||||
| 🪙 | Mixtral-8x22b-Instruct-v0.1 | 1168 | 1175 | 12 | 53.7 | Mistral | Apache 2.0 | ||
| 🪙 | Gemma-2-2b-it | 1163 | 1130 | Google | Gemma | ||||
| 🪙 | Granite-3.1-8B-Instruct | 1158 | 1191 | IBM | Apache 2.0 | ||||
| 🪙 | Claude-2.0 | 1158 | 1160 | 9 | 48.6 | Anthropic | Proprietary | ||
| 🪙 | Gemini-1.0-Pro-001 | 1155 | 1125 | Google | Proprietary | ||||
| 🪙 | Zephyr-ORPO-141b-A35b-v0.1 | 1150 | 1144 | HuggingFace | Apache 2.0 | ||||
| 🪙 | Claude-2.1 | 1146 | 1158 | 10 | 49.5 | Anthropic | Proprietary | ||
| 🪙 | GPT-3.5-Turbo-0613 | 1145 | 1164 | 9 | 46.2 | OpenAI | Proprietary | ||
| 🪙 | Qwen1.5-32B-Chat | 1144 | 1163 | Alibaba | Qianwen | ||||
| 🪙 | Phi-3-Medium-4k-Instruct | 1144 | 1146 | 11 | 54.3 | Microsoft | MIT | ||
| 🪙 | Starling-LM-7B-beta | 1139 | 1151 | Nexusflow | Apache-2.0 | ||||
| 🪙 | Mixtral-8x7B-Instruct-v0.1 | 1138 | 1136 | 3 | 38.7 | Mistral | Apache 2.0 | ||
| 🪙 | GPT-3.5-Turbo-0314 | 1138 | 1136 | OpenAI | Proprietary | ||||
| 🪙 | Granite-3.1-2B-Instruct | 1136 | 1166 | IBM | Apache 2.0 | ||||
| 🪙 | Qwen1.5-14B-Chat | 1135 | 1144 | Alibaba | Qianwen | ||||
| 🪙 | Claude-Instant-1 | 1135 | 1136 | 1 | 43.4 | Anthropic | Proprietary | ||
| 🪙 | Yi-34B-Chat | 1134 | 1129 | 01 AI | Yi | ||||
| 🪙 | Tulu-2-DPO-70B | 1127 | 1120 | Ai2 | Ai2 ImpACT | ||||
| 🪙 | DBRX-Instruct-Preview | 1126 | 1141 | Databricks | DBRX | ||||
| 🪙 | WizardLM-70B-v1.0 | 1126 | 1093 | Microsoft | Llama 2 | ||||
| 🪙 | Llama-2-70B-chat | 1122 | 1099 | 6 | 40.7 | Meta | Llama 2 | ||
| 🪙 | Nous-Hermes-2-Mixtral-8x7B-DPO | 1119 | 1103 | NousResearch | Apache-2.0 | ||||
| 🪙 | Llama-3.2-3B-Instruct | 1118 | 1097 | 5 | 34.7 | Meta | Llama 3.2 | ||
| 🪙 | Phi-3-Small-8k-Instruct | 1117 | 1123 | Microsoft | MIT | ||||
| 🪙 | OpenChat-3.5-0106 | 1114 | 1119 | OpenChat | Apache-2.0 | ||||
| 🪙 | Starling-LM-7B-alpha | 1114 | 1104 | UC Berkeley | CC-BY-NC-4.0 | ||||
| 🪙 | Vicuna-33B | 1113 | 1091 | LMSYS | Non-commercial | ||||
| 🪙 | DeepSeek-LLM-67B-Chat | 1111 | 1106 | 6 | DeepSeek | DeepSeek | |||
| 🪙 | Snowflake Arctic Instruct | 1109 | 1101 | Snowflake | Apache 2.0 | ||||
| 🪙 | Granite-3.0-8B-Instruct | 1108 | 1115 | IBM | Apache 2.0 | ||||
| 🪙 | NV-Llama2-70B-SteerLM-Chat | 1106 | 1047 | Nvidia | Llama 2 | ||||
| 🪙 | OpenChat-3.5 | 1103 | 1077 | OpenChat | Apache-2.0 | ||||
| 🪙 | Gemma-1.1-7B-it | 1102 | 1105 | Google | Gemma | ||||
| 🪙 | OpenHermes-2.5-Mistral-7B | 1100 | 1083 | NousResearch | Apache-2.0 | ||||
| 🪙 | pplx-70B-online | 1099 | 1055 | Perplexity AI | Proprietary | ||||
| 🪙 | Mistral-7B-Instruct-v0.2 | 1097 | 1094 | 1 | 24.5 | Mistral | Apache-2.0 | ||
| 🪙 | Llama-2-13b-chat | 1093 | 1077 | 6 | 40.6 | Meta | Llama 2 | ||
| 🪙 | Granite-3.0-2B-Instruct | 1091 | 1104 | IBM | Apache 2.0 | ||||
| 🪙 | SOLAR-10.7B-Instruct-v1.0 | 1091 | 1073 | Upstage AI | CC-BY-NC-4.0 | ||||
| 🪙 | Qwen1.5-7B-Chat | 1090 | 1110 | Alibaba | Qianwen | ||||
| 🪙 | Phi-3-Mini-4K-Instruct-June-24 | 1088 | 1098 | Microsoft | MIT | ||||
| 🪙 | Dolphin-2.2.1-Mistral-7B | 1088 | 1049 | Cognitive | Apache-2.0 | ||||
| 🪙 | WizardLM-13b-v1.2 | 1084 | 1048 | Microsoft | Llama 2 | ||||
| 🪙 | Phi-3-Mini-4k-Instruct | 1082 | 1102 | Microsoft | MIT | ||||
| 🪙 | MPT-30B-chat | 1076 | 1055 | MosaicML | CC-BY-NC-SA-4.0 | ||||
| 🪙 | Zephyr-7B-beta | 1076 | 1053 | HuggingFace | MIT | ||||
| 🪙 | CodeLlama-34B-instruct | 1073 | 1065 | Meta | Llama 2 | ||||
| 🪙 | Llama-3.2-1B-Instruct | 1067 | 1063 | 1 | 20 | Meta | Llama 3.2 | ||
| 🪙 | Qwen2.5-VL-32B-Instruct | 1149 | Alibaba | Apache 2.0 | |||||
| 🪙 | Step-1o-Vision-32k (highres) | 1119 | StepFun | Proprietary | |||||
| 🪙 | Qwen2.5-VL-72B-Instruct | 1104 | Alibaba | Qwen | |||||
| 🪙 | Pixtral-Large-2411 | 1088 | 24 | 70.1 | Mistral | MRL | |||
| 🪙 | Qwen-VL-Max-1119 | 1056 | Alibaba | Proprietary | |||||
| 🪙 | Qwen2-VL-72b-Instruct | 1044 | Alibaba | Qwen | |||||
| 🪙 | Step-1V-32K | 1043 | StepFun | Proprietary | |||||
| 🪙 | Molmo-72B-0924 | 1010 | Ai2 | Apache 2.0 | |||||
| 🪙 | Pixtral-12B-2409 | 1006 | 9 | 47.3 | Mistral | Apache 2.0 | |||
| 🪙 | InternVL2-26B | 1003 | OpenGVLab | MIT | |||||
| 🪙 | Llama-3.2-90B-Vision-Instruct | 997 | 20 | 67.1 | Meta | Llama 3.2 | |||
| 🪙 | Hunyuan-Standard-Vision-2024-12-31 | 996 | Tencent | Proprietary | |||||
| 🪙 | Aya-Vision-32B | 992 | Cohere | CC-BY-NC-4.0 | |||||
| 🪙 | Qwen2-VL-7B-Instruct | 988 | Alibaba | Apache 2.0 | |||||
| 🪙 | Yi-Vision | 978 | 01 AI | Proprietary | |||||
| 🪙 | Llama-3.2-11B-Vision-Instruct | 964 | 11 | 46.4 | Meta | Llama 3.2 | |||
| 🪙 | Molmo-7B-D-0924 | 957 | Ai2 | Apache 2.0 |
OproAI
💡 AAII v3 incorporates 10 evaluations: MMLU-Pro, Humanity’s Last Exam, AA-LCR, GPQA Diamond, AIME, IFBench, SciCode, LiveCodeBench, Terminal-Bench Hard, and 𝜏²-Bench Telecom.
💻 Arena Elo ratings are computed by this notebook. Higher values are better for all benchmarks. Empty cells mean not available.
Transition from online Elo rating system to Bradley-Terry model
We adopted the Elo rating system for ranking models since the launch of the Arena. It has been useful to transform pairwise human preference to Elo ratings that serve as a predictor of winrate between models. Specifically, if player A has a rating of RA and player B a rating of RB, the probability of player A winning is
ELO rating has been used to rank chess players by the international community for over 60 years. Standard Elo rating systems assume a player’s performance changes overtime. So an online algorithm is needed to capture such dynamics, meaning recent games should weigh more than older games. Specifically, after each game, a player’s rating is updated according to the difference between predicted outcome and actual outcome.
This algorithm has two distinct features:
- It can be computed asynchronously by players around the world.
- It allows for players performance to change dynamically – it does not assume a fixed unknown value for the players rating.
This ability to adapt is determined by the parameter K which controls the magnitude of rating changes that can affect the overall result. A larger K essentially put more weight on the recent games, which may make sense for new players whose performance improves quickly. However as players become more senior and their performance “converges” then a smaller value of K is more appropriate. As a result, USCF adopted K based on the number of games and tournaments completed by the player (reference). That is, the Elo rating of a senior player changes slower than a new player.
When we launched the Arena, we noticed considerable variability in the ratings using the classic online algorithm. We tried to tune the K to be sufficiently stable while also allowing new models to move up quickly in the leaderboard. We ultimately decided to adopt a bootstrap-like technique to shuffle the data and sample Elo scores from 1000 permutations of the online plays. This provided consistent stable scores and allowed us to incorporate new models quickly.
In the context of LLM ranking, there are two important differences from the classic Elo chess ranking system. First, we have access to the entire history of all games for all models and so we don’t need a decentralized algorithm. Second, most models are static (we have access to the weights) and so we don’t expect their performance to change. However, it is worth noting that the hosted proprietary models may not be static and their behavior can change without notice. We try our best to pin specific model API versions if possible.
To improve the quality of our rankings and their confidence estimates, we are adopting another widely used rating system called the Bradley–Terry (BT) model. This model actually is the maximum likelihood (MLE) estimate of the underlying Elo model assuming a fixed but unknown pairwise win-rate. Similar to Elo rating, BT model is also based on pairwise comparison to derive ratings of players to estimate win rate between each other. The core difference between BT model vs the online Elo system is the assumption that player’s performance does not change (i.e., game order does not matter) and the computation takes place in a centralized fashion.