Polling Gaza

I’m always curious as to why the level of scrutiny of polling in the United States is never applied to some other pretty important polls elsewhere. Maybe it's because election polls are eventually proven accurate or inaccurate, whereas others are onetime snapshots of public opinion that can't really be challenged.

I mention this because of an article in the AFP discussing attitudes of Palestinians after the war in Gaza. Now the poll may be accurate, I’m not sure, but there were some red flags that were raised which the article doesn't address. I don't' really fault the pollster here – like in the US, it's mostly the fault of the media in the way they report polls.

The article leads with the headline: Poll: Gaza Boosts Hamas Support.

It then goes on to say:

Hamas would get 28.6 percent of the vote compared with 27.9 percent for the rival Fatah faction of Western-backed president Mahmud Abbas if elections were held today, according to the survey by the Jerusalem Media and Communications Centre.

In Gaza, the poll put Hamas at 28 percent against 33.6 percent for Abbas' Fatah. In the West Bank, the poll gave Hamas 29 percent support against 24.5 percent for its rival.

Why do I have a problem with this? First of all, the headline, which states: "Gaza Boosts Hamas Support" is inaccurate. In order to show that it boosted Hamas' support, we would have needed an earlier poll taken before the war to measure support then. If such a poll was taken, it would have to be by the same firm with the same exact methodology. The article doesn't mention any such poll. So whatever the actual current level of support is, we don't really know if it was an increase or a decrease. This didn't stop the writer from blindly assuming a phenomenon based on no empirical evidence.

The article then goes on to mention the numbers listed above, which is fine, but uses them to make blind assertions about who actually enjoys more popularity. Given the narrow gap between the figures however, I think this is rather reckless. For example, it's declared that Hamas would win an election because it gets 28.6 percent compared to 27.9 for Fatah. It doesn't take much to observe that that's only slightly over one percentage point. Given that the polls margin of error was stated to be plus or minus 3, we can't really assume a victory based on a 1-point difference. A margin of error of 3 means that the actually results could be anywhere from Hamas - 25.6% and Fatah – 30.9, to Hamas - 31.6 and Fatah - 24.9. We can predict the real numbers would lay somewhere close to what was stated, but the article doesn't predict, it makes a bold statement about something the poll actually doesn't say.

I also wonder about the way the margin of error was calculated. The article states:

The pollsters surveyed a sample of 1,198 people -- 758 in the West Bank and 440 in Gaza -- and gave a margin of error of plus or minus three percent.

Okay, that’s fine, but if you're going to poll both territories together, and then divide them separately, it's not the same margin of error. Anytime a pollster in the US looks into demographic subgroups of a poll they know that the data isn't as accurate as the entire poll because they're not dealing with as large as a sample size. If we plug the number of respondents in the West Bank, compared with a rough estimate of the population (2,345,000) and assume a standard deviation of 95, then the margin of error for that territory would be 3.55. If we do the same for Gaza, the margin is 4.29.
That’s using the equation: square root of p (p -1) / n x 1.96. Where p = percent of respondents polled to the population, n = total population.

I also have to wonder about the accuracy of the survey given the difficulty in taking an accurate poll at this point in time. Polling a territory that was just bombed seems like it would be prone to all sorts of errors. First of all, how many people have working land lines at this point in time? Of those who do or don't, there might be a difference in affiliation. Maybe somebody who wasn't able to answer a phone for whatever reason would be more likely to support Hamas. This may be because their house was destroyed, are still at a hospital, taking care of others, etc… Or maybe any of these situations would make them more likely to support Fatah, or neither, we don't know, but there could be a difference that would greatly throw off the findings of the poll.