Equivalence tests establish whether treatments are similar in their intended outcomes. This is in contrast to superiority tests, which establish whether a new treatment is better than a standard treatment or placebo. Few equivalence trials have employed a cluster randomized design, but they are subject to some of the same analysis pitfalls that are common to superiority trials—namely, a failure to adjust for either cluster effects or covariate imbalances resulting from randomization. Using real and simulated data from a cluster randomized trial comparing exercise protocols among U.S. Army soldiers, this study empirically demonstrates the consequences for power and Type I error rates when either or both of these effects have been ignored in the analysis. Analysis of real trial data showed that equivalence test outcomes can change depending on whether appropriate adjustments are applied. Simulations demonstrated that failing to adjust for important baseline covariates severely reduces statistical power, and failing to adjust for cluster effects increases the risk of false declarations of equivalence. As cluster randomized designs are increasingly employed for equivalence trials, analysts must be aware of the importance of adjusting for cluster effects and covariate imbalances to avoid false conclusions.

Equivalence tests establish whether treatments are similar in their intended outcomes. This is in contrast to superiority tests, which establish whether a new treatment is better than a standard treatment or placebo. Few equivalence trials have employed a cluster randomized design, but they are subject to some of the same analysis pitfalls that are common to superiority trials — namely, a failure to adjust for either cluster effects or covariate imbalances resulting from randomization. Using real and simulated data from a cluster randomized trial comparing exercise protocols among U.S. Army soldiers, this study empirically demonstrates the consequences for power and Type I error rates when either or both of these effects have been ignored in the analysis. Analysis of real trial data showed that equivalence test outcomes can change depending on whether appropriate adjustments are applied. Simulations demonstrated that failing to adjust for important baseline covariates severely reduces statistical power, and failing to adjust for cluster effects increases the risk of false declarations of equivalence. As cluster randomized designs are increasingly employed for equivalence trials, analysts must be aware of the importance of adjusting for cluster effects and covariate imbalances to avoid false conclusions.

Equivalence tests establish whether treatments are similar in their intended outcomes. This is in contrast to superiority tests, which establish whether a new treatment is better than a standard treatment or placebo. Few equivalence trials have employed a cluster randomized design, but they are subject to some of the same analysis pitfalls that are common to superiority trials—namely, a failure to adjust for either cluster effects or covariate imbalances resulting from randomization. Using real and simulated data from a cluster randomized trial comparing exercise protocols among U.S. Army soldiers, this study empirically demonstrates the consequences for power and Type I error rates when either or both of these effects have been ignored in the analysis. Analysis of real trial data showed that equivalence test outcomes can change depending on whether appropriate adjustments are applied. Simulations demonstrated that failing to adjust for important baseline covariates severely reduces statistical power, and failing to adjust for cluster effects increases the risk of false declarations of equivalence. As cluster randomized designs are increasingly employed for equivalence trials, analysts must be aware of the importance of adjusting for cluster effects and covariate imbalances to avoid false conclusions.