Concept record · edition 1.0.0
Local and on-device inference
What the term means here
Running models on user-controlled hardware rather than a hosted API.
Why it matters
It changes the cost, privacy, and reproducibility profile of a system, and makes an evaluation reproducible in a way a served endpoint is not.
What this does not establish
Any claim about the current capability gap between local and hosted models. No source in this map measures that gap as of the evidence cutoff.
Related claim records · 1
Sources used here · 2
provider self reportContent verified
The Llama 3 Herd of Models
Grattafiori et al. (Meta) · 2024
A comparatively detailed provider training report; still a self-report.
Source record →primary paperContent verified
How is ChatGPT's behavior changing over time?
Chen, Zaharia, Zou · 2023
Documents that a fixed prompt against a served model is not a stable measurement over time.
Source record →