Jake Hoffman: An illusion of predictability in scientific results

Abstract:  In many fields there has been a long-standing emphasis on inference (obtaining unbiased estimates of individual effects) over prediction (forecasting future outcomes), perhaps because the latter can be quite difficult, especially when compared with the former. Here we show that this focus on inference over prediction can mislead readers into thinking that the results of scientific studies are more definitive than they actually are. Through a series of randomized experiments, we demonstrate that this confusion arises for one of the most basic ways of presenting statistical findings and affects even experts whose jobs involve producing and interpreting such results. In contrast, we show that communicating both inferential and predictive information side by side provides a simple and effective alternative, leading to calibrated interpretations of scientific results. We conclude with a more general discussion about integrative modeling, where prediction and inference are combined to complement, rather than compete with, each other.

Attendance: The presentation will be held in Room 295, 1155 E. 60th Street, with overflow and remote participants on Zoom at this address.