Bias: curse of polls & surveys
Brexit is all over the web. Some of the news triggered my thoughts on sample bias.
One of them is 'In the Cambridge City Council district, 26.2% voted Leave, while 73.8% voted Remain, compared to the UK totals of 52% Leave and 48% Remain.'
This is similar to one classic case study in Statistics: The 1936 Literary Digest Poll
For the 1936 election, the Literary Digest prediction was that Landon would get 57% of the vote against Roosevelt 43%. The actual results were 62% for Roosevelt against 38% for Landon. This poll definitely was (one of) the largest polls ever conducted. They collected feedback from around 2 millions people (2000000 - big data !!) out of the 10 millions names selected from magazine readers, car owners or telephone customers - the rich people during depression. It was not a representative sample.
Another possible source of bias is response bias, 2 out 10 millions people responded to the survey. People who responded may have different opinion from people who did not. This may not be obvious in this case, but need serious consideration for all kind of survey data.
Lesson learned:
- A big sample sometime is worse than a well-selected small sample
- Be careful about experiment design to eliminate/reduce sample bias and response bias