Claim record · edition 1.0.0
si-017Active research
The behaviour of a served model accessed through an API can change over time on fixed prompts.
Longitudinal measurement of a hosted service found behaviour changes across dates on identical inputs.
Limits of this claim
A measurement of a hosted endpoint is dated. Reproducibility claims about API-based evaluations require the date and, where available, the model version.
Supporting source records · 1
primary paperContent verified
How is ChatGPT's behavior changing over time?
Chen, Zaharia, Zou · 2023
Documents that a fixed prompt against a served model is not a stable measurement over time.
Source record →