Ying Meng and Jianhai Su Abstract: Despite achieving state-of-the-art performance across many domains, deep neural networks (DNN) are highly vulnerable to subtle adversarial perturbations. Defense approaches have been proposed in recent years, many of which have been shown inefficient by researchers. Early study suggests that ensembles created by combining multiple weak defenses are still weak. However, we observe that it is possible to construct efficient ensembles using many weak defenses. In this work, we implement and present 5 strategies to construct efficient ensembles from many (possibly weak) defenses that comprise transforming the inputs (e.g. rotation, shifting, noising, denoising, and many more) before feeding them to the classifier. We test our ensembles with adversarial examples generated by various adversaries (27 sets generated by 9 different adversarial attack methods, such as FGSM, JSMA, One-Pixel, etc.) on MNIST and investigate the factors that may impact the effectiveness of an ensemble model. We evaluate our ensembles via 4 threat models (i.e., white-box, gray-box, black-box, and zero-knowledge attacks). Also, we study and attempt to explain, empirically, how a transformation blocks perturbations generated by an adversary.