Adityanarayanan Radhakrishnan

Bio: Adityanarayanan (Adit) Radhakrishnan is a Ph.D. candidate in the Electrical Engineering and Computer Science (EECS) department at MIT advised by Caroline Uhler. He is also a Ph.D. fellow at the Eric and Wendy Schmidt Center of the Broad Institute of Harvard and MIT. Prior to his Ph.D., Adit completed his Masters in EECS and Bachelors in EECS and mathematics at MIT. Adit’s research focuses on advancing theoretical understanding of over-parameterized neural networks in order to develop simple and effective methods for applications in healthcare and computational biology.

Talk Title: Wide and Deep Networks Achieve Consistency in Classification

Talk Abstract: While neural networks are used for classification tasks across domains, a long-standing open problem in machine learning is determining whether neural networks trained using standard procedures are consistent for classification, i.e., whether such models minimize the probability of misclassification for arbitrary data distributions. In this work, we identify and construct an explicit set of neural network classifiers that are consistent. Since effective neural networks in practice are typically both wide and deep, we analyze infinitely wide networks that are also infinitely deep. In particular, using the recent connection between infinitely wide neural networks and Neural Tangent Kernels, we provide explicit activation functions that can be used to construct networks that achieve consistency. Interestingly, these activation functions are simple and easy to implement, yet differ from commonly used activations such as ReLU or sigmoid. More generally, we create a taxonomy of infinitely wide and deep networks and show that these models implement one of three well-known classifiers depending on the activation function used: (1) 1-nearest neighbor (model predictions are given by the label of the nearest training example); (2) majority vote (model predictions are given by the label of the class with greatest representation in the training set); or (3) singular kernel classifiers (a set of classifiers containing those that achieve consistency). Our results highlight the benefit of using deep networks for classification tasks, in contrast to regression tasks, where excessive depth is harmful.