Assessing ChatGPT's performance in national nuclear medicine specialty examination: An evaluative analysis

Q: How can I download an article?

To download an article from SID, first log in to the site, search for the article title, and click on the 'Download Article' option.

Q: How can I download an ISI article?

To download an ISI article on SID, enter the keyword or article title in the search bar, view the relevant results, click on the desired article, and select the 'Download Article' option.

Q: How can I access the SID database?

To access the SID database, visit SID.ir, create an account, and log in to access scientific resources.

Q: Is downloading articles from SID free?

Some articles on SID are available for free, while others require payment. Details are specified on the article's page.

Kufel Jakub; Bielowka Michal; Rojek Marcin; Mitrega Adam; Czogalik Lukasz; Kaczynska Dominika; Kondol Dominika; Palkij Kacper; Mielcarska Sylwia

Scientific Information Database (SID) - Trusted Source for Research and Academic Resources

Journal Paper

Paper Information

Journal: IRANIAN JOURNAL OF NUCLEAR MEDICINE Year:621 | Volume: | Issue: Page(s): 60-65

Download Full-Text

View:

Download:

Cites:

Information Journal Paper

Title

Assessing ChatGPT's performance in national nuclear medicine specialty examination: An evaluative analysis

Author(s)

Keywords

Artificial intelligenceQ4

Computer scienceQ4

Language modelQ4

Nuclear medicine examQ4

Abstract

Introduction: The rapid development of Artificial intelligence (AI) has sparked a desire to analyse its potential applications in medicine. The aim of this article is to present the effectiveness of the ChatGPT advanced Language model in the context of the pass rate of the polish National Specialty Examination (PES) in nuclear medicine. It also aims to identify its strengths and limitations through an in-depth analysis of the issues raised in the exam questions.Methods: The PES exam provided by the Centre for Medical Examinations in Łódź, consisting of 120 questions, was used for the study. The questions were asked using the openai.com platform, through which free access to the GPT-3.5 model is available. All questions were classified according to Bloom's taxonomy to determine their complexity and difficulty, and according to two authors' subcategories. To assess the model's confidence in the validity of the answers, each questions was asked five times in independent sessions.Results: ChatGPT achieved 56%, which means it did not pass the exam. The pass rate is 60%. Of the 117 questions asked, 66 were answered correctly. In the percentage of each type and subtype of questions answered correctly, there were no statistically significant differences.Conclusion: Further testing is needed using the questions provided by Centre for Medical Examinations from the nuclear medicine specialty exam to evaluate the utility of the ChatGPT model. This opens the door for further research on upcoming improved versions of the ChatGPT.

Cites

No record.

References

No record.

Cite

Related Journal Papers

No record.

Related Seminar Papers

No record.

Related Plans

No record.

Recommended Workshops