Ever seen statistics reverse themselves? Try this on for size:
In 2002 Mo Vaughn batted a .259 batting average for the New York Mets, while Raul Gonzalez batted a .260 (for the New York Mets and the Cincinnati Reds). In 2003 Vaughn delivered a .190 batting average, and Gonzalez batted a .230. Clearly in each year, Gonzalez had a higher batting average than Vaughn. However, between the years 2002 and 2003 seasons, Vaughn had a batting average of .258, while Gonzalez only kept up a .240. How is it possible that across two seasons, Mo Vaughn had a higher batting average? Simpson’s paradox.
This is the phenomenon wherein probabilities or rates appear reversed when we divide a population into subgroups (i.e. split the batting averages according to year). We see that overall, Mo Vaughn held the higher batting average, but during each season, Gonzalez proved better numbers. The trick is the number of at bats (AB) per season.
2002 | 2003 | Combined | ||||
Mo Vaughn | 126/487 | .259 | 15/97 | .190 | 141/546 | .258 |
Raul Gonzalez | 27/104 | .260 | 50/217 | .230 | 77/321 | .240 |
This reverse in leading batting average is due to the difference in weighting when we calculate across years. The combined batting average is not simply an average of the two years, rather we calculate all the hits divided by all at bats. Note that in 2003, Vaughn had only 97 at bats, compared to 2002 when he had 487. Averaging the two statistics would mean that each at bat does not have the same weight (over-weighting the at bats from 2003). In the same way, averaging Gonzalez’s statistics would over-weight the at bats from 2002. Although unbelievably counterintuitive, Simpson’s Paradox proves the need to know all the facts.
