Models

Claude Opus 5.5 solved all 74 CSAT questions in under five minutes

Claude Opus 5.5 correctly answered all 74 CSAT reading and math questions in an estimated five minutes or less. Perplexity scored 71, while free ChatGPT managed 32 of 46 math questions.

It takes only minutes for AI tool to ace all English, math CSAT questions
It takes only minutes for AI tool to ace all English, math CSAT questionsMonoar_CGI_Artist / pixabay
By Rebecca Stone7 min read

Why it matters

  • Claude Opus 5.5 answered all 74 tested CSAT English and mathematics questions correctly in an estimated five minutes or less.
  • Perplexity’s free version answered 71 of 74 questions correctly, left two visual questions unanswered and missed one geometry question.
  • Free ChatGPT answered all 28 English reading questions correctly in about 10 seconds but only 32 of 46 mathematics questions.
  • Claude used images to interpret garbled PDF formulas and Python to check Probability and Statistics Questions 28 and 30.
  • The test was not controlled because timings were approximate, settings differed and tool use was not standardized.

By Rebecca Stone

Claude Opus 5.5 answered all 74 English reading and mathematics questions from the 2027 academic-year CSAT correctly in a reporter-run test, with the model estimating the work took less than five minutes.

Perplexity’s free version scored 71 of 74 in the same exercise, while free ChatGPT answered all 28 English questions but only 32 of 46 mathematics questions.

The test did not establish a controlled model ranking: timing estimates were approximate, settings differed and tool use was not standardized.

Together, the results put general-purpose AI tools directly against the questions used for university admission in South Korea. They also show that success varied sharply once the systems encountered visual material, uncertain answers and different levels of access to code.

What did the CSAT test include?

The reporter-run test used papers for the 2027 academic-year College Scholastic Ability Test, or CSAT, university entrance examination. It covered every question in the English reading and mathematics sections tested.

The English portion contained 28 reading questions and excluded the listening test. The mathematics portion contained 46 questions drawn from the common section and three elective subjects. Together, they produced a 74-question test.

The report checked the models’ responses against answer keys published by the state-run Korea Institute for Curriculum and Evaluation.

The scope of the test was broad, but its method lacked the controls required for a definitive benchmark:

  • 28 English reading questions, with listening excluded.
  • 46 mathematics questions from the common section and three electives.
  • 74 questions checked against answer keys from the Korea Institute for Curriculum and Evaluation.
  • One paid Claude configuration compared with free versions of Perplexity and ChatGPT.

How did Claude solve the exam?

Anthropic’s paid Claude Opus 5.5 operated with its effort level set to “High.” It returned the correct answer to every one of the 74 questions.

The prompt was straightforward: “I’ll give you the mathematics and English questions. Solve them and tell me the answers.”

Claude later estimated that the full task took less than five minutes. The report treated that as an estimate rather than a controlled timing measurement.

The model also disclosed the tools behind its performance. Anthropic’s Claude said:

“The formulas came through as garbled text in the PDF, so I converted the pages into images and solved the problems by looking at them. I solved most without running code, but for Probability and Statistics questions 28 and 30, which involved many possible combinations, I used Python to enumerate all the possibilities and check my answers.”

That account matters because Claude did not answer solely by reading extracted text. It converted PDF pages into images to recover formulas, then used Python on two Probability and Statistics questions involving multiple combinations.

Claude said it solved most questions without running code. The result therefore combined model reasoning with PDF conversion, visual interpretation and limited calculation checks rather than reproducing a clean, no-tools examination response.

The test compresses work that students spend many years preparing for into a short prompt and a set of PDF pages.

How close did Perplexity get?

Perplexity’s free version returned its English answers in an estimated 20 seconds or less. It produced its mathematics answers in about one minute and 50 seconds or less.

Perplexity correctly answered all 27 English questions it attempted and 44 of the 45 mathematics questions it attempted. Its 71 correct answers came with one error and two unanswered questions.

The service left an English chart question and a mathematics function-graph question blank. Perplexity said it could not reliably recover the visual information it needed from the uploaded PDFs.

Those omissions show the practical limit of the test’s document format. Perplexity performed well on questions whose information it could extract, but the report did not record a correct answer when it could not interpret the required chart or graph.

Where did Perplexity go wrong?\n

Among the questions Perplexity attempted, its only incorrect answer came on Geometry Question 27. The problem asked for the area of a triangle in a regular hexagonal prism.

Perplexity selected option 3 instead of the correct option 1. The service later said it had submitted the answer without completing a reliable calculation.

Perplexity also said it could not determine whether a mistaken interpretation of the diagram had contributed to the error. The report therefore recorded both a wrong answer and uncertainty about the source of that answer.

The distinction separates its 71 correct responses from its total of 74: Perplexity answered 73 questions, got 72 right and submitted one wrong answer, while leaving two questions unanswered. The simpler reported score was 71 of 74 correct.

What did ChatGPT solve?

The free ChatGPT service, developed by OpenAI, answered all 28 English reading questions correctly in about 10 seconds.

Its mathematics performance was weaker. ChatGPT answered 32 of the 46 mathematics questions correctly, submitted two incorrect answers and marked 12 answers as unconfirmed.

The report did not convert those unconfirmed responses into either correct or incorrect results. It counted 32 confirmed correct mathematics answers, leaving 14 without a confirmed correct response.

ChatGPT therefore matched Claude on the 28 English reading questions but produced a different result on the 46-question mathematics section. Claude returned 46 confirmed correct mathematics answers; ChatGPT returned 32.

How do the three results compare?

The reported results were:

  • Claude Opus 5.5, paid: 74 of 74 correct, with the effort level set to “High”; less than five minutes estimated; PDF pages converted into images; Python used on Probability and Statistics Questions 28 and 30.
  • Perplexity, free: 71 of 74 correct; English answers in an estimated 20 seconds or less; mathematics answers in about one minute and 50 seconds or less; one wrong answer and two unanswered visual questions.
  • ChatGPT, free: 28 of 28 English reading questions correct in about 10 seconds; 32 of 46 mathematics questions correct; two incorrect answers and 12 unconfirmed.

The test does not support a clean ranking of the underlying models. The report explicitly says that the exercise was not controlled, that its timings were approximate, that settings differed and that tool use was not standardized.

Claude alone had a named effort setting and a disclosed workflow involving images and Python. Perplexity used a free service and encountered limits in extracting information from visual PDF content. ChatGPT also used a free service and declined to confirm 12 mathematics answers. Those differences prevent the scores from measuring model capability under identical conditions.

The comparison also did not place three equivalent products against one another. It paired a paid Claude configuration with free versions of two competing services, leaving pricing and tier differences unmeasured beyond those labels.

Even with those limitations, the report found that the systems could complete a demanding set of university entrance questions far faster than a human test-taker. It also placed the accuracy of Claude’s full-score result on par with that of top-performing students.

What do the results mean for AI testing?

The immediate research question is not whether an AI system can reach a high score on a CSAT practice paper. Claude’s result shows that it can under the conditions used in this test. The harder question is how reliably the same performance holds when prompts, settings, tools and time limits change.

The policy stakes center on the same distinction. These were university entrance questions, and the AI systems produced near-complete answer sets in minutes. That raises a concrete question about how schools and exam administrators should treat AI assistance during real admissions testing, although this reporter-run exercise did not reproduce controlled examination conditions.

A valid follow-up would require identical prompts and settings, fixed time limits and clearly controlled tool access across the services. Without those controls, Claude’s 74-for-74 result remains a striking capability demonstration, while the gaps recorded for Perplexity and ChatGPT show how service configuration, document handling and willingness to answer can alter the outcome.

Source: feed.koreatimes.co.kr

Share this article:

More from Rebecca Stone

Rebecca Stone

Show full bio

Correspondent covering companies, deals and strategy in artificial intelligence and machine learning at AI In Context.

802 articles

Related articles

  1. AI Glasses Solved Four CSAT Math Questions in Ten Seconds
  2. 551,864 Students Sit Korea's CSAT Under Anti-AI Cheating Crackdown
  3. Lee Jin-woo misses exam demo as Seoul police probe AI allegations
  4. South Korea orders A-to-Z overhaul of national exams over AI cheating
  5. AI-assisted cheater tops South Korea’s tax accountant exam

« Previous articleNext article »