Automatic GoF

[alexandrebouchard]

• Check generative distribution emitting an observed real/int/.. and with no outgoing edges
• Doctor the fixed real/int value to be actually modifiable
• Then at each iteration:

1. Compute the likelihood, compute and store importance weight
2. Keep a copy of observation
3. Forward sample the observation, record
4. Roll back to the observation

At the end,

1. sort the values
2. normalize the weights
3. build a alpha% CI for a bunch of alphas
4. plots these

Can also do for first vs second half of sample to help disentangle model vs inference errors. Also plot the ESS for these GoF estimates to make sure they are reliable. For facetted can be seen as residuals (plot 25, 50, 75)