Your browser does not fully support modern features. Please upgrade for a smoother experience.
Evaluating LLMs on the Certified Internal Auditor Exam
Playlist
  • internal audit
  • learning by testing
  • Certified Internal Auditor
  • large language models
  • competencies
  • certification exam
Video Introduction

This video is adapted from https://doi.org/10.3390/app16168106

The internal audit function provides consulting services and recommendations for management on performance quality, business environment security, or risk management. Internal auditors are expected to monitor the senior managers' behaviors and support them in operational and strategic decision making. They should recognize and reduce the possibility of an employee's opportunistic behavior, improve internal control, and prevent financial fraud. Therefore, they should be highly qualified, and their competencies are to be constantly updated. Hence, they are encouraged to learn and participate in certification processes organized by various institutions, e.g., the Institute of Internal Auditors (IIA). Although there are many strategies of learning in the certification process, this video focuses on learning-by-testing approach. This method emphasizes the use of testing to acquire the internal audit knowledge needed to successfully pass the professional certified exams. Beyond that, this video presents the use of large language models (LLMs) in internal auditing. However, the research goal is to how the large language models (LLMs) can be used to answer questions included in the Certified Internal Auditor (CIA) exam directed by the Institute of Internal Auditors (IIA). In the research section, fourteen LLMs were evaluated using English and Polish-language CIA test sets comprising 184 and 183 multiple-choice questions, respectively, with accuracy as the primary performance metric. The best-performing individual model achieved approximately 91% accuracy, while the Friedman test identified statistically significant overall differences among the models for both datasets. The findings indicate that a multi-agent system can achieve better performance than an individual LLM. The learning-by-testing method can be supported not only by the use of professional association materials but also by the opportunity learners have to enrich their knowledge through the use of LLMs in the learning process.

Full Transcript
${ textCharacter }/${ maxCharacter }
Submit
Cancel
There is no comment~
${ textCharacter }/${ maxCharacter }
Submit
Cancel
${ selectedItem.replyTextCharacter }/${ selectedItem.replyMaxCharacter }
Submit
Cancel
Confirm
Are you sure to Delete?
Yes No
Cite
If you have any further questions, please contact Encyclopedia Editorial Office.
Pankowska, M.; Żytniewski, M. Evaluating LLMs on the Certified Internal Auditor Exam. Encyclopedia. Available online: https://encyclopedia.pub/video/1907 (accessed on 06 October 2026).
Pankowska M, Żytniewski M. Evaluating LLMs on the Certified Internal Auditor Exam. Encyclopedia. Available at: https://encyclopedia.pub/video/1907. Accessed October 06, 2026.
Pankowska, Malgorzata, Mariusz Żytniewski. "Evaluating LLMs on the Certified Internal Auditor Exam" Encyclopedia, https://encyclopedia.pub/video/1907 (accessed October 06, 2026).
Pankowska, M., & Żytniewski, M. (2026, September 30). Evaluating LLMs on the Certified Internal Auditor Exam. In Encyclopedia. https://encyclopedia.pub/video/1907
Pankowska, Malgorzata and Mariusz Żytniewski. "Evaluating LLMs on the Certified Internal Auditor Exam." Encyclopedia. Web. 30 September, 2026.
Academic Video Service