Blogging the Bookshelf

Blogging my bookshelf – one book at a time

Blogging the Bookshelf header image 2

Out of Sample Problems – “The Signal and the Noise: The Art and Science of Prediction”, Nate Silver

November 16th, 2012 · No Comments · Causation and Correlation, Data, Prediction, Statistics

But forecasters often resist considering these out-of-sample problems. When we expand our sample to include events further apart from us in time and space, it often means that we will encounter cases in which the relationships we are studying did not hold up as well as we are accustomed to. The model will seem to be less powerful. It will look less impressive in a PowerPoint presentation (or a journal article or a blog post). We will be forced to acknowledge that we know less about the world than we thought we did. Our personal and professional incentives almost always discourage us from doing this.

We forget—or we willfully ignore—that our models are simplifications of the world. We figure that if we make a mistake, it will be at the margin.

Tags: Bayesian Analysis·Bias·Incentives·prediction·