The Leap of Faith capstone team developed an AI-driven solution to automate the review of physician and emergency room visit notes for Medicare’s Merit-based Incentive Payment System (MIPS), a program that adjusts healthcare provider payments based on quality measures. Traditionally, medical professionals manually review clinical notes to determine compliance, creating a process that is costly, time-consuming, and prone to human error. Using Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG), the team built a system capable of extracting relevant clinical information from physician notes while reducing hallucinations and improving reliability. Tested across multiple healthcare quality measures, including appropriate CT scan use for pulmonary embolism cases, the model achieved 92–96% accuracy with 80–100% precision. By replacing manual chart review with AI-powered automation, the project demonstrates the potential to reduce operational costs by up to 80% while improving efficiency, scalability, and healthcare quality assurance for hospitals and providers participating in MIPS programs.

Keywords: Healthcare AI, Large Language Models, Retrieval-Augmented Generation (RAG), Clinical NLP, Natural Language Processing, Medicare MIPS, Healthcare Quality Measures, Physician Notes, Clinical Documentation, AI in Healthcare, Medical Record Analysis, Healthcare Automation
Watch the team present this project in the session recording here. You can also watch the team discuss their project in this video.

Faculty Advisor

Dr. Dmitri Sidorov is a Principal Data Scientist at Abbott Laboratories, a multinational healthcare company, specializing in research-based drugs, medical devices, diagnostics, and nutritional products.

He earned a PhD in Physics from Oklahoma State University with a focus in analysis of data from particle physics experiments and applying statistics.

Dmitri has over a decade of professional experience analyzing Big Data, using various methods and techniques, and generating relevant and actionable insights. He teaches R for Data Science and Big Data and Cloud Computing courses in the MS in Applied Data Science program at the University of Chicago.

This experience gave him a strong belief that any endeavor can be improved by intelligent use of data.

arrow-left-smallarrow-right-large-greyarrow-right-large-yellowarrow-right-largearrow-right-long-yellowarrow-right-smallclosefacet-arrow-down-whitefacet-arrow-downCheckedCheckedlink-outmag-glass