What is the cancer information and accuracy of AI? When I compared ChatGPT Clova X...

Jul 30, 2025

Source of data =Annual Medical Journal

Worldwide, Large Language Model (LLM) using artificial intelligence (AI) has emerged as a tool to generate text responses similar to humans in various areas. In particular, ChatGPT has recently passed the U.S. Medical License Test (USMLE), and has also stood out in specialized fields such as science, law, and medicine.

Now is the time when AI explains treatment beyond cancer information. How reliable can AI's cancer information be?

A research team at a university hospital in Korea directly compared the accuracy of the AI language model's response to Korean cancer information.

Catholic Kwandong University International St. Mary's Hospital (Father Ko Dong-hyun) announced on the 30th that a research team led by Professor Jang Hyun of Oncology Blood Medicine, Professor Jeong Jin-woo of Urology, and Professor Kim Yong-ho of Radiation Oncology announced the results of a study comparing and analyzing the accuracy of Korean cancer information responses of representative AI language models 'ChatGPT' and 'ClovaX'.

This study is the first to compare how accurate and easy-to-understand AI language models provide answers to cancer-related questions in Korean.

Professor Jang's team entered cancer-related questions registered on the website of the National Cancer Information Center and the U.S. National Cancer Institute into ChatGPT and ClovaX, and a specialist evaluated the responses of the two models.

As a result, both models showed a response quality of 'similar level' overall and there was no statistically significant difference. However, in some questions, both models provided 'hallucination responses' containing non-medical information or unverified content.

The research team evaluated "A meaningful study that presented basic data on the level at which artificial intelligence language models can provide medical information in Korean." Professor Jang Hyun said, "We will continue our research to increase the accuracy of language models so that artificial intelligence can become a tool to provide reliable medical information to cancer patients" he said.

Meanwhile, the research paper was published in the July issue of Yonsei Medical Journal, an international academic journal, under the title Comprehensive Analysis of Large Language Models for Answering Cancer-Related Questions in Korean.