Blogging the Bookshelf

Blogging my bookshelf – one book at a time

Blogging the Bookshelf header image 4

Entries Tagged as 'Statistics'

The Origins of Statistical Inference – “The Victory Lab: The Secret Science of Winning Campaigns”, Sasha Issenberg

December 4th, 2012 · No Comments · Data, Policy, Statistical Inference, Statistics

Not far from Fisher, a young economist named Austin Bradford Hill was growing similarly impatient with the limits of statistics to account for cause and effect in health care. In 1923, for example, Hill received a grant from Britain’s Medical Research Council that sent him to the rural parts of Essex, east of London, to […]

[Read more →]

Tags:····

The Limitations of Statistical Significance – “The Signal and the Noise: The Art and Science of Prediction”, Nate Silver

November 22nd, 2012 · No Comments · Data, Statistical Inference, Statistics

The bigger problem, however, is that the frequentist methods—in striving for immaculate statistical procedures that can’t be contaminated by the researcher’s bias—keep him hermetically sealed off from the real world. These methods discourage the researcher from considering the underlying context or plausibility of his hypothesis, something that the Bayesian method demands in the form of […]

[Read more →]

Tags:Bayesian Analysis··Modelling··Statistical Significance·

More Data Means More Noise – “The Signal and the Noise: The Art and Science of Prediction”, Nate Silver

November 21st, 2012 · No Comments · Data, Prediction, Statistical Inference, Statistics

As there is an exponential increase in the amount of available information, there is likewise an exponential increase in the number of hypotheses to investigate. For instance, the U.S. government now publishes data on about 45,000 economic statistics. If you want to test for relationships between all combinations of two pairs of these statistics—is there […]

[Read more →]

Tags:Big Data··Modelling··

Bayes’ Theorem – “The Signal and the Noise: The Art and Science of Prediction”, Nate Silver

November 21st, 2012 · No Comments · Policy, Prediction, Statistics

Bayes’s theorem is concerned with conditional probability. That is, it tells us the probability that a theory or hypothesis is true if some event has happened. Suppose you are living with a partner and come home from a business trip to discover a strange pair of underwear in your dresser drawer. You will probably ask […]

[Read more →]

Tags:analysis·Bayesian Analysis·Forecasting··prediction·

The Limitations of Economic Forecasts – “The Signal and the Noise: The Art and Science of Prediction”, Nate Silver

November 19th, 2012 · Comments Off on The Limitations of Economic Forecasts – “The Signal and the Noise: The Art and Science of Prediction”, Nate Silver · Economics, Policy, quote

Instead, economic forecasts are blunt instruments at best, rarely being able to anticipate economic turning points more than a few months in advance. Fairly often, in fact, these forecasts have failed to “predict” recessions even once they were already under way: a majority of economists did not think we were in one when the three […]

[Read more →]

Tags:Economic Forecasts·Economic Modelling·Economic Policy·economics··

Overfit Models – “The Signal and the Noise: The Art and Science of Prediction”, Nate Silver

November 19th, 2012 · No Comments · Modelling, Quotes, Statistics

But the overfit model scores those extra points in essence by cheating—by fitting noise rather than signal. It actually does a much worse job of explaining the real world. As obvious as this might seem when explained in this way, many forecasters completely ignore this problem. The wide array of statistical methods available to researchers […]

[Read more →]

Tags:Economic Modelling·Modelling···

The Limited Accuracy of Polls in Primary Contests – “The Signal and the Noise: The Art and Science of Prediction”, Nate Silver

November 18th, 2012 · No Comments · Electoralism, Politics

During the 2008 Democratic primaries, the average poll missed by about eight points, far more than implied by its margin of error. The problems in polls of the Republican primaries of 2012 may have been even worse. In many of the major states, in fact—including Iowa, South Carolina, Florida, Michigan, Washington, Colorado, Ohio, Alabama, and […]

[Read more →]

Tags:House Bias···

The Unimportance of Political News – “The Signal and the Noise: The Art and Science of Prediction”, Nate Silver

November 18th, 2012 · No Comments · Electoralism, Journalism, Newspapers, Politics, Spin, The Media

Political news, and especially the important news that really affects the campaign, proceeds at an irregular pace. But news coverage is produced every day. Most of it is filler, packaged in the form of stories that are designed to obscure its unimportance.* Not only does political coverage often lose the signal—it frequently accentuates the noise. […]

[Read more →]

Tags:Horse Race·journalism·journalists·Political Coverage····The Media

The Narrative and Political Prediction – “The Signal and the Noise: The Art and Science of Prediction”, Nate Silver

November 17th, 2012 · No Comments · Electoralism, Journalism, Newspapers, Politics, Prediction, Psychology, Spin, The Media

You can get lost in the narrative. Politics may be especially susceptible to poor predictions precisely because of its human elements: a good election engages our dramatic sensibilities.

[Read more →]

Tags:Experts·journalism·journalists·Political Coverage···prediction··The Media

Out of Sample Problems – “The Signal and the Noise: The Art and Science of Prediction”, Nate Silver

November 16th, 2012 · No Comments · Causation and Correlation, Data, Prediction, Statistics

But forecasters often resist considering these out-of-sample problems. When we expand our sample to include events further apart from us in time and space, it often means that we will encounter cases in which the relationships we are studying did not hold up as well as we are accustomed to. The model will seem to […]

[Read more →]

Tags:Bayesian Analysis·Bias·Incentives·prediction·