Concept record · edition 1.0.0
Inference-time compute
What the term means here
Additional computation spent at generation time — longer reasoning traces, sampling with verification, or search over candidate outputs.
Why it matters
It breaks the assumption that a benchmark score is a property of a model. The same weights score differently under different inference budgets.
What this does not establish
That inference-time gains substitute indefinitely for training scale. The trade-off is reported within studied regimes, not as a general law.
Related claim records · 2
Sources used here · 3
primary paperContent verified
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei et al. · 2022
Established that intermediate reasoning steps at inference change measured performance.
Source record →primary paperContent verified
Training Verifiers to Solve Math Word Problems
Cobbe et al. · 2021
Verifier-based selection over sampled solutions, an early inference-time compute method.
Source record →primary paperContent verified
Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
Snell et al. · 2024
The inference-time compute trade-off against parameter scaling.
Source record →