Ben Lengerich

Bio: Ben Lengerich is a Postdoctoral Associate and Alana Fellow at MIT’s Computer Science and Artificial Intelligence Lab (CSAIL) and the Broad Institute of MIT and Harvard, where he is advised by Manolis Kellis. His research in machine learning and computational biology emphasizes the use of context-adaptive models to understand complex diseases and advance precision medicine. Through his work, Ben aims to bridge the gap between data-driven insights and actionable medical interventions. He holds a PhD in Computer Science and MS in Machine Learning from Carnegie Mellon University, where he was advised by Eric Xing. His work has been recognized with spotlight presentations at conferences including NeurIPS, ISMB, AMIA, and SMFM, and is currently financially supported by the Alana Foundation.

Talk Title: Decoding Real-World Evidence: Bridging GAMs and LLMs to Expose “Death by Round Numbers”

Abstract: Large language models (LLMs) are revolutionizing natural language processing, but combining their implicit knowledge with explicit statistical machine learning models is an ongoing challenge. To bridge these systems, we propose to use generalized additive models (GAMs). GAMs decompose complex outcomes into separable component functions, creating a modular structure that fits seamlessly into LLM context windows. As a result, GAMs can unleash the power of LLMs for traditional machine learning tasks.

We demonstrate an exciting application of this perspective: automated surprise finding. Real-world data science grapples with complications like hidden confounding, often necessitating manual model inspections by domain experts. For instance, treatment-based confounding in medical data may inadvertently teach data- driven systems to perpetuate inherent biases. However, by bridging GAMs with LLMs, we harness the vast knowledge reservoir within LLMs to auto-detect anomalies that appear to contradict domain expertise.

This approach not only improves machine learning models, but also reveals quirks of the underlying system. Using this combination of glass-box GAMs and foundational LLMs, we evaluate four datasets of real-world medical evidence, identifying two characteristic types of medical biases: (1) discontinuous risk at sharp treatment thresholds and (2) counter-causal paradoxes where aggressive treatment reduces perceived risk.

Finally, we present the TalkToEBM package to establish this bridge between GAMs and LLMs and offer a lens to automatically identify surprising effects in statistical models.