Emergency department (ED) discharge notes contain critical follow-up instructions often buried in free-text, making them difficult to extract and act upon. This project, conducted in partnership with HealthLab Innovations Inc., evaluates whether large language models (LLMs) can reliably identify and structure key clinical elements—such as follow-up intervals, provider names, and imaging recommendations—within real-world ED documentation. A dual-model pipeline was developed to compare outputs from multiple LLMs and flag inconsistencies for further review. The evaluation involved four open-source models tested across prompt variations and preprocessing configurations. Initial results show strong agreement rates in structured output and support the feasibility of scalable, real-time deployment. This work demonstrates the potential for AI-driven systems to improve care continuity by transforming unstructured clinical data into actionable information.
Watch the team present this project at 29:18 in the session recording here.
Keywords: dark data, large language models, emergency department, clinical NLP, healthcare automation, follow-up care
Faculty Advisor
Ashish Pujari is a Data and AI consultant, practitioner, and educator with over 20 years of experience in in machine learning, big data, and cloud computing. He has led large global technology and data science teams and consulted for Fortune 500 companies in banking, finance, healthcare, insurance, and manufacturing.
As a Principal ML/AI Architect at AWS, Ashish provides strategic guidance to enterprise customers on leveraging the cloud for AI and Machine Learning. Prior to joining Amazon, he served in various technology leadership roles at Credera, GLG, IRI, and Pegasystems. Ashish holds a Master of Science in Analytics from the University of Chicago and BS in Electrical Engineering from the National Institute of Technology, Rourkela.