Making Statistics Work: Information Theory and Bayesian Inference

Conventional “frequentist” methods that dominate the field of statistics are generally inconsistent and liable to catastrophic failure in some contexts. These weaknesses have become particularly concerning in relation to crises of replicability and credibility in science. Two alternatives have been proposed to address these flaws—classical Bayesian inference and the principle of maximum entropy—but the connections between them remain controversial. Making Statistics Work presents a synthesis of information theory and Bayesian inference that addresses these fundamental problems. It provides a consistent, powerful, and flexible framework for data inference based on rigorous logic derived from first principles, allowing for new approaches to many of the unresolved questions of statistics. Duncan K. Foley and Ellis Scharfenaker illustrate the application of this framework and the reasoning behind it across a variety of important statistical problems, such as the inference underlying “gold standard” clinical trials, models of human behavior employed in behavioral finance and psychology, analysis of macroeconomic policy, the relationship of classical probability to quantum physics, and the limitations of linear regression analysis. Making Statistics Work offers new insight into contentious topics, from problems of causality and confounding variables in randomized experimental trials to the foundations of Bayesian and frequentist probability theory.