Reconciling an AI Risk Estimate Against What Truly Happened
A model produces a figure, an event happens, and somebody asks whether the figure was right. It is the obvious question and it has almost no published answer, because the comparison is harder than it looks. A single realized loss cannot falsify a distribution. If a model puts a one percent chance on exceeding a threshold and the threshold is exceeded, the one percent case occurred, which is what the model said would sometimes happen.