A permutation procedure to detect heterogeneous treatment effects in randomized clinical trials while controlling the type I error rate

Abstract

Secondary analyses of randomized clinical trials often seek to identify subgroups with differential treatment effects. These discoveries can help guide individual treatment decisions based on patient characteristics and identify populations for which additional treatments are needed. Traditional analyses require researchers to pre-specify potential subgroups to reduce the risk of reporting spurious results. There is a need for methods that can detect such subgroups without \emph{a priori} specification while allowing researchers to control the probability of falsely detecting heterogeneous subgroups when treatment effects are uniform across the study population. We propose a permutation procedure for tuning parameter selection that allows for Type-I error control when testing for heterogeneous treatment effects framed within the Virtual Twins procedure for subgroup identification. We verify that the Type-I error rate can be controlled at the nominal rate and investigate the power for detecting heterogeneous effects when present through extensive simulation studies. An application of our approach to a recently completed trial of very low nicotine content cigarettes identifies several variables with potentially heterogeneous treatment effects. The proposed permutation procedure allows researchers to engage in secondary analyses of clinical trials for treatment effect heterogeneity while maintaining the Type-I error rate without pre-specifying subgroups.