Source record · content verified
Measuring AI Ability to Complete Long Software Tasks
research organization2025
Why this source is here
The primary definition of the 50%-task-completion time horizon. Abstract read during this pass; it is the only verified source for any time-horizon figure in this map.
- Verification
- Content-verified on 2026-07-27: the canonical source and its title were resolved during the Atlas review. This is not an endorsement of the source’s argument.
- Authors
- Kwa et al. (METR)
- Identifier
- arXiv:2503.14499
Claims citing this source · 2
- si-018
METR defines a 50%-task-completion time horizon: the time humans typically take on tasks a model completes with 50% success. Its published measurement reports a horizon of around 50 minutes for one frontier model of that period, and a doubling of the horizon roughly every seven months since 2019, with a possible acceleration in 2024.
- si-019
Increases in measured long-horizon performance are attributed in part to improved reliability and error recovery rather than to reasoning improvements alone.