Different polls put different candidates ahead; methodologies questioned
Exit poll results from the Oct. 31 presidential elections are raising more questions than answers, challenging the notion that polls can keep official Central Election Commission vote counts honest.
Different polling groups have been critical of each other’s work during the last few weeks, and some have even criticized their own polls. The bickering has raised questions about the pollster’s abilities to gauge the legitimacy of the tight vote count expected in the Nov. 21 run-off election between Prime Minister Viktor Yanukovych and Our Ukraine opposition leader Viktor Yushchenko.
Election commission counts show that Yanukovych and Yushchenko each received about 40 percent support in the first round.
Of the three nationwide exit polls conducted during the first round, two mirrored each other, indicating that Yanukovych edged out Yushchenko by several points. Both conducted traditional face-to-face surveys asking voters whom they voted for as they exited polling stations, followed by additional questions.
Ukrainian Exit Poll, a consortium of sociological teams from Ukrainian educational institutions, and Russia’s Obshestvenoye Mnenie Fund (known as FOM) polling team produced nearly identical results. Obshestvenoye Mnenie’s poll was called off on Election Day after over 70 percent of respondents refused to participate, and Ukrainian Exit Poll, which surveyed just over 13,000 Ukrainians with a margin error of 1.5 percent, reported a refusal rate of about 25 percent.
The third group, a consortium of polling firms coordinated by the Kyiv-based Democratic Initiatives Foundation, a Western-funded NGO, gathered strikingly different results. Two polling companies in the consortium – Socis and Ukrainian Monitoring – conducted traditional face-to-face surveys asking voters whom they voted for as they left polling stations. Their results showed Yanukovych holding about a four percentage point lead over Yushchenko. The other two members of the consortium – the Razumkov Ukrainian Center for Economic and Political Studies and the Kyiv International Institute for Sociology (KIIS) – conducted their own election exit polls using an as-yet untested methodology, according to which respondents fill out questionnaires anonymously. Results using this methodology showed Yushchenko with a nearly six point lead.
Opposition figures have used this poll to back up their claim that Yushchenko earned more support from voters than the approximately 39 percent granted him by the Central Election Commission.
The consortium’s exit polls surveyed nearly 50,000 respondents. Each of the two methods employed has a two percent margin of error.
Several pre-election polls conducted during recent months produced sharply different results, as well. According to some, Yushchenko had a three point to eight point lead over his main opponent. Others suggested that Yanukovych had surpassed Yushchenko in September and October by posting a lead of several percentage points.
The Democratic Initiative consortium has earned the trust of many western observers due to the fact that its polls are funded by western donors, including governments and non-profit foundations.
But their decision two weeks ago to allow two polling firms in their consortium to conduct the controversial secret-ballot surveys is fueling criticism from other polling groups and within their own camp.
Untimely debate
Democratic Initiatives officials have defended their so-called secret-ballot survey, which critics allege has not been tested by sociologists. They say it helps eliminate a unique factor that exists in Ukrainian politics: respondents’ fear of openly admitting preferences for opposition candidates or parties. According to some accounts, as many as 25 percent of respondents are affected by the fear factor. Indeed, traditional face-to-face pre-election polls conducted ahead of the 2002 parliamentary elections indicated that the oppositionist Socialist and Fatherland parties had support below four percent. When the votes were tallied, however, they posted support of about six percent.
Yet most Ukrainian polling companies, including one member of the Democratic Initiatives consortium, are questioning the validity of and need for secret-poll methodology.
Oleksandr Yaremenko, chairman of the Ukrainian Institute for Sociological Studies, which coordinated the Ukrainian Exit-Poll, cast doubt on the secret methodology used by Razumkov and KIIS.
“We do not have plans to use this method because it’s untested and we are unaware of any respected sociological institutions having used it before,” Yaremenko said, adding that Ukrainian Exit Poll will conduct a traditional face-to-face exit poll during the second round if they find about $30,000 in funding from donors.
“We consider it unacceptable to employ an untested methodology in these contested and tight elections. You can’t play games at times like these,” Yaremenko added.
Yaremenko also doubted that the fear factor was so significant. Studies indicate it was larger two years ago, but has shrunk as of late and only affects about two percent of respondents, he said.
Olha Balakireva, director of Social Monitoring, said her polling company could drop out of the DIF-led consortium in connection with their doubts about the legitimacy of the secret-ballot survey methodology. Social Monitoring will stick with the traditional face-to-face polling, she added.
“I am confident in our methodology and results,” she said. “The results of the so-called anonymous surveys are interesting, but this is experimental.”
“We don’t think we will leave the consortium, but if several issues are not resolved, we might have to leave…Our questions are not answered, and until they are, there are issues to be resolved,” she said, adding that discussions are currently underway.
Also unclear is what methodology the DIF-led consortium would use in the second round.
Ilko Kucheriv, director of Democratic Initiatives, said results gathered in the first round are currently being reviewed and talks are underway to change the strategy to be used in the second round.
“I’d like for each company to stay in the consortium, but operate as a single team, so that each gathers survey results through their own network but complies and presents the results as one,” Kucheriv said. “We have not decided yet if we will use the secret survey methodology in the second round.”
“We are discussing the issue now. The most important thing is that we work as a team, use one methodology and produce one result,” he added.
