Distinguishing ``Approximately Correct'' from ``Completely Wrong''; a Comparison of Bayesian Methods for Evaluating Fit

Kert Viele

Abstract:

Model embeddings have been used by several authors to examine fit. We propose an embedding method for examining fit designed to quantify how close a proposed model is to the actual process generating the data. This method is asymptotically consistent, allowing good approximate models to be distinguished from poor approximations. In simulations with moderate samples, the method also performs well. While predictive p-values perform well at distinguishing whether a model is correct, asymptotics and simulations performed indicate it is difficult to quantify the amount of inaccuracy in a model using predictive p-values.

Keywords: Goodness of Fit, Kullback-Leibler Information, Consistency, Posterior Predictive p-value

Here is the full postscript text for this technical report. It is 253335 bytes long.