Recent advances in bootstrap test for comparing two or more groups

Alok Kumar Dwivedi

Associate Professor, Division of Biostatistics and Epidemiology
Director of Biostatistics and Epidemiology Consulting Lab (BECL)
Department of Biomedical Sciences, Paul L. Foster School of Medicine
Texas Tech University Health Sciences Center El Paso (TTUHSC EP)
Room no-3103A, Medical Science Building
5001 El Paso Drive, El Paso, Texas-79905
Cell: 513-917-4242 ; Office: 915-215-4177
Fax: 915-545-5716

Randomized experimental design is a common design in biomedical

studies which typically requires univariate statistical tests for the
analysis. In such studies, investigator generally wants to compare
quantitative outcomes according to two or more conditions/groups.
However, non-normal data distribution, small sample size, and unequal
variances between groups frequently pose analytical challenges in these
studies. A number of parametric and nonparametric statistical tests
have been recommended under different assumptions and conditions
for data analysis. The validity of these statistical tests in terms of
statistical power and type I error probability has been questioned
especially for non-normal data with small to moderate sample size
studies. To overcome this problem, methodologists have started
exploring the utility of semi-parametric approaches such bootstrap and
permutation tests which combine the benefits of parametric and
nonparametric tests. However, these alternative tests also have sample
size limitations. Further, there are many versions of bootstrap test which
also produced varying statistical powers and type I error probability. We
developed pooled bootstrap test, composite bootstrap test, and
bootstrap test based on rank t-statistic with counterintuitive resampling
method for comparing distributions and means across groups. The
performance of these novel bootstrap tests as compared with common
parametric and nonparametric statistical tests for comparing means
were assessed using extensive simulations under various conditions and
using real data examples. The proposed bootstrap tests reasonably
controlled type I error probability in most situations. All the proposed
bootstrap tests provided greater or equal statistical power for comparing
two means as compared with unpaired t-test, Welch t-test, Wilcoxon
rank sum test, and permutation t-test in many conditions. Similarly, all
the proposed bootstrap tests outperformed over other alternatives for
comparing paired means and for comparing more than two means. For
exponential family distributions, we recommend to use the pooled
bootstrap test while the use of composite bootstrap test or bootstrap
based on rank t-statistic should be preferred for extreme variable
distributions such as Cauchy or log normal data, or under non-
exchangeability conditions. The proposed bootstrap tests should also be
used for comparing paired or unpaired groups.