Summary:
Add PSIS diagnostic for ADVI .
Description:
This paper proposes PSIS diagnostic for ADVI: Yuling Yao, Aki Vehtari, Daniel Simpson, and Andrew Gelman (2018). Yes, but Did It Work?: Evaluating Variational Inference https://arxiv.org/abs/1802.02538
Currently ADVI samples N draws from the approximate distribution and outputs those. In addition lp__ column is inserted, but it has all 0's (it seems stansummary requires lp__ column).
To follow the current design with stansummary, what we propose is
Note that initially we start just computing the diagnostics, but eventually we can use diagnostics to improve the algorithm.
Summary:
Add PSIS diagnostic for ADVI .
Description:
This paper proposes PSIS diagnostic for ADVI: Yuling Yao, Aki Vehtari, Daniel Simpson, and Andrew Gelman (2018). Yes, but Did It Work?: Evaluating Variational Inference https://arxiv.org/abs/1802.02538
Currently ADVI samples N draws from the approximate distribution and outputs those. In addition lp__ column is inserted, but it has all 0's (it seems stansummary requires lp__ column).
To follow the current design with stansummary, what we propose is
after ADVI finishes, compute lp__ and lg__ (log density of of the approximation) in stan/src/stan/variational/advi.hpp
output lp__ and lg__ along with parameter sample (for lp__ provide values instead of 0's and add lg__ column). Something needs to be added also in stan/src/stan/services/experimental/advi/meanfield.hpp
add algorithms for computing khat and neff in services
Given the output we can compute diagnostic afterwards, e.g., using stansummary for CmdStan
for simplicity we would implement this first for CmdStan (needs separate issue), ie, modify stansummary to output diagnostics cmdstan/src/cmdstan/stansummary.cpp
Note that initially we start just computing the diagnostics, but eventually we can use diagnostics to improve the algorithm.