The 'Killer Questions' Identified by the Ministry of Education... Question Marks Over Selection Criteria

Ministry of Education Announces 'Measures to Reduce Private Education Expenses'
22 Actual Killer Questions Released
Clear Variations in Correct Answer Rates by Subject and Question

On the 26th, Oh Seung-geol, Director of the Responsible Education Policy Office at the Ministry of Education, is presenting examples of killer questions from the College Scholastic Ability Test at the Government Seoul Office in Jongno-gu, Seoul. Photo by Yoon Dong-joo doso7@

The education authorities have released the actual questions in response to the so-called ‘killer question’ controversy stemming from President Yoon Suk-yeol’s directive for a ‘fair CSAT’.

However, since there was no clear definition of killer questions, the example questions released this time have also been criticized for having ambiguous criteria.

On the 26th, the Ministry of Education announced the ‘Measures to Reduce Private Education Expenses’ at the Government Seoul Office and disclosed 26 ultra-difficult questions that appeared in the June mock exam and the past three years of the College Scholastic Ability Test (CSAT). By subject, there were 7 in Korean, 9 in Mathematics, 6 in English, and 4 in Science.

Asia Economy analyzed the released killer questions based on the correct answer rates compiled by EBS’s online lecture site, EBSi. The Korea Institute for Curriculum and Evaluation (KICE), which is responsible for CSAT question setting, does not disclose the correct answer rates per question nor the standard scores by Korean and Math elective subjects since the introduction of the integrated liberal arts and science CSAT in 2022.

[Image source=Yonhap News]

Sharp Variations in Correct Answer Rates by Subject and Question... Is a ‘Killer Question’ One That Only 2 Out of 5 Students Get Right?

Analysis of the data released by the Ministry of Education showed significant differences in correct answer rates by subject, even among the same killer questions. Within the same subject, the variation in correct answer rates among killer questions ranged from 11.1% to 21.7%. The Korea Institute for Curriculum and Evaluation does not disclose correct answer rates per question. EBSi compiles provisional scoring results from test takers.

According to the correct answer rates compiled by EBSi, the hardest questions in the Korean (15.1%) and English (17.0%) sections had higher correct answer rates than the easiest question in the Mathematics section (14.0%), suggesting that the Mathematics killer questions were much more difficult. It also appears to have been influenced by the fact that all Mathematics killer questions were subjective (open-ended) questions.

First, in the Korean section, 7 questions were identified as killer questions: June mock exam questions 14 and 33, 2023 CSAT questions 15 and 17, and 2022 CSAT questions 8, 13, and 15. Among these, question 33 from the June mock exam had the highest correct answer rate at 36.8%, while question 17 from the 2023 CSAT had the lowest at 15.1%. The average correct answer rate for Korean killer questions was about 28.6%.

In Mathematics, 9 questions were selected: June mock exam common subjects questions 21 and 22, Calculus question 30, 2023 CSAT common subjects question 22, Probability and Statistics question 30, Calculus question 30, 2022 CSAT Calculus question 29, Geometry question 30, and 2021 CSAT Type B question 30. The question with the lowest correct answer rate was June mock exam common subject question 22 (2.9%), and the highest was 2022 CSAT Calculus question 29 (14.0%). The average correct answer rate for Mathematics killer questions was only 6.5%.

For English, the killer questions were June mock exam questions 33 and 34, 2023 CSAT questions 34 and 37, and 2022 CSAT questions 21 and 38. The 2023 CSAT question 34 had the lowest correct answer rate at 17.0%, while question 37 from the same exam had the highest at 29.1%. The average correct answer rate for English killer questions was 24.7%.

In Science, the killer questions were June mock exam Life Science II question 15, 2023 CSAT Chemistry II question 20, 2022 CSAT Physics II question 18, and Earth Science II question 20. The 2023 CSAT Chemistry II question 20 had the lowest correct answer rate at 22.0%, and the 2022 CSAT Physics II question 18 had the highest at 34.5%. The average correct answer rate for Science killer questions was about 27.5%.

Ambiguous ‘Killer Question’ Criteria... Ministry of Education’s Explanation Also One-Dimensional

Despite ongoing controversy, there is no clear definition of killer questions. President Yoon Suk-yeol emphasized that questions that cannot be solved without private education and that go beyond the public education curriculum should not be included in the CSAT, and Lee Ju-ho, Deputy Prime Minister for Social Affairs and Minister of Education, described them as questions “so twisted that even professors cannot solve them.” These are the only criteria presented.

In the admissions industry, killer questions are generally classified as ‘ultra-difficult questions with single-digit correct answer rates’ or ‘the questions with the lowest correct answer rates in a subject.’ Some of the killer questions released by the Ministry of Education this time did not meet these industry standards.

For example, in the Korean section, 2023 CSAT questions 35 (20.4%) and 39 (28.9%) had lower correct answer rates than question 15 (30.3%) but were excluded from the Ministry’s list of killer questions. Similarly, in Mathematics, 2023 CSAT Probability and Statistics question 29 had an extremely low correct answer rate of 2.6% but was not included as a killer question.

Regarding this, the Ministry of Education explained, “Killer questions are those that cover content not included in the public education curriculum and favor students who have learned problem-solving techniques and repeatedly practiced through private education. Candidate questions were selected based on opinions from field teachers, then reviewed once or twice before being reported to the Curriculum Normalization Deliberation Committee.”

Additionally, the Ministry mentioned characteristics of killer questions by subject. For Korean, these included: ▲questions using passages and technical terms difficult for high school students to understand, making it relatively easier and faster for students with background knowledge ▲questions that do not provide sufficient information needed to solve the problem, making comprehension difficult ▲questions with complex meanings and structures in the answer choices that intentionally induce mistakes.

For Mathematics, the characteristics were: ▲questions combining multiple mathematical concepts requiring excessively complex thinking or advanced solving methods ▲questions where students who have studied university-level content can solve the problem by methods other than those expected by the question setter, causing advantages and disadvantages among students. For English, the characteristics included: ▲questions with professional or abstract content that are difficult to understand even after translating the English ▲questions using excessively long and complex sentences beyond the general level covered in public education, making interpretation difficult ▲questions with long and complex syntax and difficult vocabulary in the answer choices, making it hard to solve the problem even after understanding the passage.

This content was produced with the assistance of AI translation services.