AI Summary
ARC-AGI has progressed to its third version, which evaluates AI agents' adaptability in new environments. The latest leaderboard emphasizes the importance of efficiency, showcasing systems that operate under a $10,000 budget.
%22%2F%3E%3Crect%20width%3D%22800%22%20height%3D%22450%22%20fill%3D%22url(%23glow)%22%2F%3E%3Ctext%20x%3D%22400%22%20y%3D%22275%22%20text-anchor%3D%22middle%22%20fill%3D%22%2360a5fa%22%20font-family%3D%22system-ui%2C-apple-system%2Csans-serif%22%20font-size%3D%22132%22%20font-weight%3D%22800%22%20letter-spacing%3D%22-6%22%20opacity%3D%220.9%22%3Ecn%3C%2Ftext%3E%3Ctext%20x%3D%22400%22%20y%3D%22318%22%20text-anchor%3D%22middle%22%20fill%3D%22%2360a5fa%22%20font-family%3D%22system-ui%2C-apple-system%2Csans-serif%22%20font-size%3D%2211%22%20font-weight%3D%22600%22%20letter-spacing%3D%223%22%20opacity%3D%220.38%22%3ECONTEXT%20NEWS%3C%2Ftext%3E%3C%2Fsvg%3E)
- ARC-AGI has transitioned from earlier versions that focused on passive fluid intelligence to a new version that tests adaptability in interactive environments.
- The performance of AI agents is visualized in a scatter plot that highlights the relationship between cost-per-task and efficiency.
- Only systems with operational costs below $10,000 are included in the results.
- Incomplete test outputs are marked as incorrect, and results labeled as 'preview' are unofficial and may not reflect complete testing.
- The ARC-AGI-2 score is based on partial testing, and cost estimates for some models are provisional pending further testing.
arc-agileaderboardai researchperformance metricsartificial intelligence