How to Properly Interpret Polls: "Blind Trust is a No-Go, Pay Attention to Survey Methods"

[Planning] D-8, In-Depth Analysis of By-Elections ⑧ Can Polls Be Trusted?
Poll Results Should Be Questioned Due to Subsamples
Support Rate Differences Within Margin of Error Are 'Close Race,' Not 'Superior' or 'Inferior'

On the 23rd, ahead of the Seoul mayoral by-election, an election commission official is conducting a mock vote at a special early voting station set up at the Seoul Youth Hostel in Jung-gu, Seoul. Photo by Hyunmin Kim kimhyun81@

[Asia Economy Reporter Naju-seok] During election season, not only candidate camps but also voters pay close attention to the trends in opinion polls. Camps use them to devise election strategies, and voters can compare and reference the overall public sentiment with their own choices. However, there is also criticism that opinion polls do not merely reflect public opinion but can, conversely, shape it?so-called "the tail wagging the dog." With the April 7 by-elections just a week away, we asked experts how to wisely utilize opinion polls.

First, pay attention to the ‘sample size.’ There is an example. According to an opinion poll conducted by Realmeter commissioned by YTN in January this year (survey conducted January 18?20, nationwide adult men and women 1,510 respondents, wireless ARS, sampling error ±2.5% at 95% confidence level, response rate 2.5%, refer to the Central Election Poll Deliberation Commission website), the support rate for the Democratic Party in Busan, Ulsan, and Gyeongnam rose by 8.4 percentage points from the previous survey to 34.5%, surpassing the People Power Party (29.9%). At the time, the media interpreted this as the Democratic Party’s legislative efforts on the Gadeokdo New Airport Special Act influencing local public sentiment.

Experts cautioned against overinterpreting such conclusions. This poll was conducted nationwide among adult men and women, with the Busan, Ulsan, and Gyeongnam regional sample comprising only 12.6% of the total, or 191 respondents.

Kim Bong-shin, Senior Deputy Director at Realmeter, explained, "With a sample size of about 190, the sampling error at a 95% confidence level is approximately ±7.1 percentage points, so a difference of about 14?15 percentage points is needed to interpret it as a meaningful difference." In other words, there are limitations to definitively interpreting that the Democratic Party overtook the People Power Party based on this poll.

Jung Han-ul, a polling expert at Korea Research, pointed out, "Based on a poll with fewer than 200 respondents, the government actually promoted the Gadeokdo New Airport Special Act, and the opposition announced plans for an undersea tunnel."

Besides sample size, experts advise closely examining the sampling error in opinion polls. Election-related polls or related media reports often include phrases like ‘sampling error ±3 percentage points at 95% confidence level.’ For example, if candidate A has 40% support and candidate B has 35%, it means candidate A’s support is likely between 37% and 43%, and candidate B’s between 32% and 38%, each with 95% probability. Although the numbers suggest candidate A is ahead, there is still a possibility that candidate B is actually leading.

Regarding this, Gallup, an opinion polling agency, explains on its website, "It is impossible to know exactly where the overall public opinion lies; we can only estimate that it falls within a range."

An official from the Opinion Poll Deliberation Commission, which oversees election-related polls, explained, "Differences in support within the margin of error should be interpreted and understood with expressions like ‘competitive’ or ‘close race’ rather than ‘leading’ or ‘winning.’"

There are also many criticisms that the gap between opinion polls and actual public opinion must be considered. It is necessary to acknowledge that opinion polls reflect only the views of those who actively respond to surveys. Senior Deputy Director Kim analyzed, "As voices calling for judgment on the government/ruling party or opposition increase ahead of major elections, the willingness to respond to opinion polls has risen. Recent polls represent the opinions of highly engaged respondents rather than the entire public."

The results also vary depending on whether the survey uses an ARS method with recorded machine voices or an interviewer-based method. In ARS surveys, the response rate tends to be low, which may relatively reflect the voices of politically strong and clear positions more. Conversely, some polling agencies using ARS methods argue that ARS better reflects voter sentiment than interviewer methods. Since experts have differing opinions on the reliability of opinion poll results, voters are advised to adopt a more conservative approach when accepting poll outcomes.

This content was produced with the assistance of AI translation services.