About ARC Prize Foundation

AI benchmarks that measure general intelligence and inspire new ideas

Tech description:

Job description:

A technical researcher to own how we evaluate frontier models on the ARC-AGI benchmarks. This person will run new models end-to-end, mine the data exhaust from every run, and translate what we learn into reports and public communication that shape the conversation on where model capability is heading. This is a remote, full-time role. What You'll Do: * Own our model benchmarking and testing process, and run new frontier models against ARC-AGI-1, ARC-AGI-2, and ARC-AGI-3 as they ship * Build and own the ARC Prize Analysis Package - a repeatable report produced for every new frontier model, turning raw logs into insight on capability, failure modes, and gaps * Own the [official](https://arcprize.org/leaderboard) and [community](https://arcprize.org/leaderboard/community) leaderboards end-to-end - from scoring pipeline to public page * Serve as primary contact for new labs testing on ARC-AGI, and communicate findings externally via Twitter, newsletter, and policy and partner briefings What We're Looking For: * Research background with hands-on model evaluation experience - you've run evals before and know how to read the results (model training experience not required) * Deep understanding of how modern models work and fail, and comfortable building your own tooling and analysis to answer the questions you care about * Strong ownership instinct and clear technical communicator Example outputs this role would produce: a [model score announcement](https://x.com/arcprize/status/1999182732845547795) and a [model analysis blog post](https://arcprize.org/blog/analyzing-o3-with-arc-agi).

Skills:

Python

ARC Prize Foundation remote jobs

Game Platform Engineering Lead

Benchmark Testing and Analysis Lead