AI Research
Oct 16, 2026
Limited AI Benchmark Coverage for US Jobs Affects 90% of Working Hours
Oct 16, 2026
AI Summary
A recent analysis reveals that 90% of American working hours are spent on tasks not well represented by existing AI benchmarks. Only 208 out of 923 occupations have sufficient benchmark coverage, highlighting significant gaps in understanding AI's capabilities in the workforce.
- AI benchmarks have increased, but their coverage of economically valuable work is limited.
- A study mapped 923 US occupations to assess benchmark coverage, finding that only 208 occupations have tasks represented by at least five benchmark prompts.
- The analysis indicates that 90% of working hours are spent on tasks with little or no benchmark coverage, complicating the evaluation of AI model performance in real-world job settings.
- The focus of AI benchmarks tends to be on larger labor markets, with significant underrepresentation of middle-income knowledge jobs.
- The study utilized the US Department of Labor’s O*NET taxonomy to categorize occupations and tasks, totaling 18,797 tasks across all occupations.
- Existing benchmarks disproportionately cover tasks that are more exposed to AI automation, particularly in high-wage occupations, while lower-wage and less AI-exposed tasks remain underrepresented.
- The findings suggest a need for more diverse and comprehensive benchmarks to accurately assess AI capabilities across various job sectors.
us workforceai benchmarkslabor indexproductivitytask analysis