Bayesian nonparametrics

Overview document describing various BNP models and samplers.

Overview document tex .tex file for document describing various BNP models and samplers.

Todo items for DP-related work:

• finish nonconjugate dCRP sampler
  • need MCMC configuration to recognize this case and assign sampler (Chris)
  • finalize sampler (Claudia)
  • allow zero intermediate nodes (Claudia / Chris)
  • what BUGS code could a user write that would make the sampler illegitimate (Claudia)
  • allow upper bound on number of components
  • discuss inefficiency of sampling all tilde variables even though most have no observations
  • (HIGH) add new sampler for CRP that iterates over unique labels instead of n labels. Claudia has draft; Chris to look over.
  • (HIGH) add test that has basic usage of a dmnorm mixture. (Claudia, I just added this August 2018)
  • discuss 'links' version of CRP - see Johnson and Sinclair - Environmetrics. 2017;28:e2440 "Modeling joint abundance of multiple species using Dirichlet process mixtures"
• initial conjugate dCRP sampler (i.e., marginalized with respect to new component)
  • determine conjugacy in setup code, including for zero intermediate nodes (Chris)
  • determine conjugacy for normal-inverse gamma and normal-inverse Wishart priors in normal model (Claudia; Chris to first write the bivariate N-IG and N-IW distributions as nimble distributions)
  • check what happens with MVN cases
  • use conjugacy determination to develop conjugate sampler run code - all common univariate cases are now handled (Claudia). List of examples:
  • immediately sample new component parameters when create new component (Claudia)
  • try to reuse setup code and run code so we don't duplicate code (Chris)
  • ask Perry if using model[ something ][i] or model$values(something) makes a difference in efficiency
  • (HIGH) add non-identity-link conjugacy - will likely need variations on 'offset' and 'coeff' from our conjugate sampler setup to allow us to handle things like y~dnorm(c*thetatilde[xi[i]] + x[i]*beta, 1) and correctly account for 'c' and x[i]*beta (Chris to help get this started)
• avoid updating cluster parameters for unfilled clusters (except the 'next' cluster)
• stickbreaking approach
  • determine syntax for stickbreaking and write stickbreaking function (Claudia)
    • determine possible NaN situations
  • detect conjugacy in this setting (Chris)
  • clean up conjugacy for this setting (Chris)
• (HIGH) compare speed and mixing of Nimble BNP to one or two other popular BNP (e.g., DPpackage) packages; do this after we change CRP sampler to not iterate through all observations (Claudia; let's check in on this once the unique labels sampler is fully done)
• (HIGH) blog post on BNP functionality - Chris/Abel decided the Avanda GLMM example would be good; also possibly add simpl density estimation example and show NIMBLE supports different kernels (Chris/Claudia)
• (HIGH) testing
  • formal comparison (e.g., K-L calculation) of density estimates with truth and between CRP and stick-breaking
  • we might also think of other good test cases where we know the right answer (perhaps models Abel has fit previously using his own code)
• CRP distribution
  • determine possible NaN situations (Claudia)
  • write help (Claudia)
• standardized output for G when using dCRP (input posterior modelValues and augment with columns for weights and atoms). (Claudia / Nick / Chris)
  • figure out how a user will call this (current rough plan is to have an R function that user calls with that R function using a stand-alone nimbleFunction that sets number of columns in a matrix (not a modelValues)) (Chris / Claudia / Abel / other nimble-devs)
  • handle multivariate clusters - e.g. mixtures of dmnorm components
• write help for BNP sampler with some examples.
• write a quasi conjugate sampler for the "conc" parameter when a gamma distribution is assumed
  • write the sampler (Claudia)
  • write help with some examples (Claudia)
  • automatically assign this sampler (Claudia / Chris)
• fully marginalized dCRP sampler for conjugate models - low priority
  • write sampler
  • write help
  • add sampling for tilde variables only every 'thin' iterations if monitoring for them requested
  • determine conjugacy for normal-inverse gamma prior in normal model
• more complicated cases, in approximate order of decreasing priority (Claudia, except where indicated)
  • HDP (allows a Polya urn so straightforward)
  • check conjugacy detection in stick-breaking case for Pitman-Yor and generalized Dirichlet cases (Chris)
  • Pitman-Yor (i.e., Beta(1-alpha, beta+k*alpha)) (allows Polya urn so straightforward)
  • nested DP - possibly use Polya urn, but Abel to check with Peter Mueller about sampling (Abel + Claudia)
  • generalized Dirichlet (Beta(a_k, b_k) or Beta(a, b)) with dCRP (no easy Polya urn)
  • (possibly) NRMs with slice sampler (Italian school); Abel is checking with researchers about whether they have C/C++ code we might use or at least learn from
  • dependent DP
• work on rich examples in NIMBLE, possibly for blog post or papers, perhaps by Masters students; some possibilities are:
  • basic analysis of Kevin Quinn model
  • comparison of biclustering with Kevin Quinn model
  • blog post or paper focused on simple use of DPM for random effects to reach out to poli sci or education (or possibly ecology, though we have connections there already) (Abel suggested the spatial cluster model and there is the Kevin Quinn case we could also revisit)
• revisit our overall framework for basic DP models only after we get user feedback