parametric vs non-parametric bootstrap?

[joseah]

Dear MixedModels team,

I have been successfully applying the parametric bootstrap implemented in the MixedModels package. However, I was recently wondering about the rationale of implementing a parametric bootstrapping for linear mixed models as opposed to a non-parametric approach. Unfortunately I haven't found a reliable literature reference for preferring a parametric approach for bootstrapping mixed models.

Many thanks for the amazing library!

[palday]

The reason is largely pragmatic: defining a nonparametric resampling method for multilevel/hierarchical data is non trivial. I am involved in a research project to get such methods defined and tested, but I haven't had much time for it lately. You can see that work over at MixedModelsPermutations.jl.

Meanwhile, defining a parametric method is straightforward and is isomorphic with data simulation from known/assumed parameter values, which is useful for things like power analysis. In other words, the parametric bootstrap is easier to implement codewise, has a clear definition mathematically, and has other uses, so it's the obvious choice to implement first. 😄

In terms of performance relative to nominal error rates, the parametric bootstrap performs fairly well, but the shortest tail confidence intervals can be slightly anticonservative. The equal tail probability ones seem to perform slightly better, at least for the fixed effects parameters. The best performance is probably from the profile confidence intervals.

[joseah]

Hi Phillip,

thanks for the comprehensive explanation!
Just a quick follow-up question. Would it be fair to say that using a non-parametric approach might be appropriate for simple hierarchical structures (e.g. two random effects and no cross-effects) whilst a parametric approach would be more adequate for more complex multilevel/hierarchical data structures?

Many thanks!

[palday]

Would it be fair to say that using a non-parametric approach might be appropriate for simple hierarchical structures (e.g. two random effects and no cross-effects)

Depending on your inference goals, a parametric method might also be appropriate. If you can define a good non-parametric resampling regime for the specific case(s) you care about, then of course feel free to use that! 😄 Not every problem has to be solved in general.

whilst a parametric approach would be more adequate for more complex multilevel/hierarchical data structures?

Yes-ish if only because it's easier to define the general case. 😄

[joseah]

Makes sense, thanks very much @palday!