Artificial intelligence-based large language models and integrity of exams and assignments in higher education: the case of tourism courses
DOI:
https://doi.org/10.18089/tms.2023.190402Keywords:
Large Language Models (LLM), Artificial Intelligence, ChatGPT, Tourism Education, Exams, Integrity.Abstract
There is an increasing concern regarding the potential misuse of ChatGPT-4 in compromising the integrity of examinations and assignments. This study aims to examine the capabilities of ChatGPT-4 in critical thinking abilities, whether it poses a threat to examinations and assignments in higher education, and create a discussion agenda on this issue. ChatGPT-4 was asked to generate, answer, and criticize questions in tourism marketing, tourism management, tourism economics, tourist guidance, and gastronomy. The answers were evaluated according to universal critical thinking standards. The findings obtained from this study showed that ChatGPT-4 had commendable competence in several critical thinking standards and could produce human-like texts. However, there are certain domains that might be improved to comply more effectively with the expectations and norms of academia. Educators were recommended to use comprehensive approaches that combine technological and educational techniques to address the issue of cheating enabled by tools, such as ChatGPT-4, during assessment and exam processes.
References
Ababneh, K. I., Ahmed, K., & Dedousis, E. (2022). Predictors of cheating in online exams among business students during the Covid pandemic: Testing the theory of planned behavior. The International Journal of Management Education, 20(3), 100713. https://doi.org/10.1016/j.ijme.2022.100713
Adzima, K. (2021). Examining online cheating in higher education using traditional classroom cheating as a guide. Electronic Journal of E-Learning, 18(6). https://doi.org/10.34190/JEL.18.6.002
Agasisti, T., & Soncin, M. (2021). Higher education in troubled times: On the impact of Covid-19 in Italy. Studies in Higher Education, 46(1), 86–95. https://doi.org/10.1080/03075079.2020.1859689
AlAfnan, M. A., Dishari, S., Jovic, M. & Lomidze, K. (2023). Chatgpt as an educational tool: opportunities, challenges, and recommendations for communication, business writing, and composition courses. Journal of Artificial Intelligence and Technology, 3, 60-68. https://doi.org/10.37965/jait.2023.0184
Ali, K., Barhom, N., Marino, F. T., & Duggal, M. (2023). The thrills and chills of chatgpt: ?mplications for assessments in undergraduate dental education [Preprint]. MEDICINE & PHARMACOLOGY. https://doi.org/10.20944/preprints202302.0513.v1
Atlas, S. (2023). Chatgpt for higher education and professional development: a guide to conversational AI. University of Rhode Island DigitalCommons@URI. https://digitalcommons.uri.edu/cba_facpubs/548
Barrientos, A., Cuadros, M., Alba, J., & Cruz, A. S. (2021, October 27-29). Implementation of a remote system for the supervision of online exams through the use of cameras with artificial intelligence. 2021 IEEE Engineering International Research Conference (EIRCON), 1–4. https://doi.org/10.1109/EIRCON52903.2021.9613352
Barrot, J. S. (2023). Using ChatGPT for second language writing: Pitfalls and potentials. Assessing Writing, 57, 100745. https://doi.org/10.1016/j.asw.2023.100745
Benvenuti, M., Cangelosi, A., Weinberger, A., Mazzoni, E., Benassi, M., Barbaresi, M., & Orsoni, M. (2023). Artificial intelligence and human behavioral development: A perspective on new skills and competences acquisition for the educational context. Computers in Human Behavior, 107903. https://doi.org/10.1016/j.chb.2023.107903
Bernabei, M., Colabianchi, S., Falegnami, A., & Costantino, F. (2023). Students’ use of large language models in engineering education: A case study on technology acceptance, perceptions, efficacy, and detection chances. Computers and Education: Artificial Intelligence, 100172. https://doi.org/10.1016/j.caeai.2023.100172
Carvalho, L., Martinez-Maldonado, R., Tsai, Y.-S., Markauskaite, L., & De Laat, M. (2022). How can we design for learning in an AI world? Computers and Education: Artificial Intelligence, 3, 100053. https://doi.org/10.1016/j.caeai.2022.100053
Chirumamilla, A., & Sindre, G. (2019). Mitigation of cheating in online exams: Strengths and limitations (A. V. S. Kumar, Ed.). IGI Global. https://doi.org/10.4018/978-1-5225-7724-9
Choi, J. H., Hickman, K. E., Monahan, A., & Schwarcz, D. B. (2023). ChatGPT goes to law school. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4335905
Coghlan, S., Miller, T., & Paterson, J. (2020). Good proctor or “Big Brother”? AI ethics and online exam supervision technologies. https://arxiv.org/abs/2011.07647.
Comas-Forgas, R., Lancaster, T., Calvo-Sastre, A., & Sureda-Negre, J. (2021). Exam cheating and academic integrity breaches during the COVID-19 pandemic: An analysis of internet search activity in Spain. Heliyon, 7(10), e08233. https://doi.org/10.1016/j.heliyon.2021.e08233
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2023). Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International, 1–12. https://doi.org/10.1080/14703297.2023.2190148
Crawford, J., Cowling, M., Central Queensland University, Australia, Allen, K.-A., & Monash University, Australia. (2023). Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI). Journal of University Teaching and Learning Practice, 20(3). https://doi.org/10.53761/1.20.3.02
Currie, G. M. (2023). Academic integrity and artificial intelligence: Is ChatGPT hype, hero or heresy? Seminars in Nuclear Medicine, 53(5), 719–730. https://doi.org/10.1053/j.semnuclmed.2023.04.008
David, A., Mihai, D., Mihailescu, M.-E., Carabas, M., & Tapus, N. (2022). Scalability through distributed deployment for moodle learning management system. Procedia Computer Science, 214, 34–41. https://doi.org/10.1016/j.procs.2022.11.145
Dendir, S., & Maxwell, R. S. (2020). Cheating in online courses: Evidence from online proctoring. Computers in Human Behavior Reports, 2, 100033. https://doi.org/10.1016/j.chbr.2020.100033
Dias, J., Kamdi, D., Gharat, N., & Chudhari, P. (2019). Chatbot for Government Examination using AI. Journal of Engineering, 9(4), 58-62.
Duhaim, A. M., Al-mamory, S. O., & Mahdi, M. S. (2022). Cheating detection in online exams during Covid-19 pandemic using data mining techniques. Webology, 19(1), 341–366. https://doi.org/10.14704/WEB/V19I1/WEB19026
Eke, D. O. (2023). ChatGPT and the rise of generative AI: Threat to academic integrity? Journal of Responsible Technology, 13, 100060. https://doi.org/10.1016/j.jrt.2023.100060
Elbanna, S., & Armstrong, L. (2023). Exploring the integration of ChatGPT in education: Adapting for the future. Management & Sustainability: An Arab Review. https://doi.org/10.1108/MSAR-03-2023-0016
Essel, H. B., Vlachopoulos, D., Tachie-Menson, A., Johnson, E. E., & Baah, P. K. (2022). The impact of a virtual teaching assistant (chatbot) on students’ learning in Ghanaian higher education. International Journal of Educational Technology in Higher Education, 19(1), 57. https://doi.org/10.1186/s41239-022-00362-6
Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2023). A SWOT analysis of ChatGPT: Implications for educational practice and research. Innovations in Education and Teaching International, 1–15. https://doi.org/10.1080/14703297.2023.2195846
Geerling, W., Mateer, G. D., Wooten, J., & Damodaran, N. (2023a). ChatGPT has Mastered the Principles of Economics: Now What? SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4356034
Gerashchenko, I. P., & Kovalev, V. A. (2021). Formation of educational ecosystems through the digital transformation of the educational environment. SHS Web of Conferences, 121, 03004. https://doi.org/10.1051/shsconf/202112103004
Gilson, A., Safranek, C., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2022). How Does ChatGPT Perform on the Medical Licensing Exams? The ?mplications of large language models for medical education and knowledge assessment [Preprint]. Medical Education. https://doi.org/10.1101/2022.12.23.22283901
Gökta?, L. S. (2023). ChatGPT uzaktan e?itim s?navlar?nda ba?ar?l? olabilir mi turizm alan?nda do?ruluk ve do?rulama üzerine bir ara?t?rma (Can ChatGPT succeed in distance education exams a research on accuracy and verification in tourism). Journal of Tourism and Gastronomy Studies, 2. https://doi.org/10.21325/jotags.2023.1224
Hashim, H. (2018). Application of technology in the digital era education. International Journal of Research in Counseling and Education, 1(2). https://doi.org/10.24036/002za0002
Hill, B. (2023). Taking the help or going alone: ChatGPT and class assignments. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4465833
Holden, O. L., Norris, M. E., & Kuhlmeier, V. A. (2021). Academic ?ntegrity in online assessment: A research review. Frontiers in Education, 6, 639814. https://doi.org/10.3389/feduc.2021.639814
Hwang, A. (2018). Online and Hybrid Learning. Journal of Management Education, 42(4), 557–563. https://doi.org/10.1177/1052562918777550
Ilynykh, S. (2021). Online education: Trends and risks of digital technologies: VIII International Scientific and Practical Conference “Current problems of social and labour relations” (ISPC-CPSLR 2020), Makhachkala, Russia Federation. https://doi.org/10.2991/assehr.k.210322.126
Jungherr, A. (2023). Using ChatGPT and Other Large Language Model (LLM) applications for academic paper assignments [Preprint]. SocArXiv. https://doi.org/10.31235/osf.io/d84q6
King, D. L., & Case, C. J. (2014). E-Cheating: Incidence and trends among college students. Issues In Information Systems, 15(1). https://doi.org/10.48009/1_iis_2014_20-27
Klijn, F., Mdaghri Alaoui, M., & Vorsatz, M. (2022). Academic integrity in on-line exams: Evidence from a randomized field experiment. Journal of Economic Psychology, 93, 102555. https://doi.org/10.1016/j.joep.2022.102555
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., & Tseng, V. (2023). Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digital Health, 2(2), e0000198. https://doi.org/10.1371/journal.pdig.0000198
Lanier, M. M. (2006). Academic integrity and distance learning. Journal of Criminal Justice Education, 17(2), 244–261. https://doi.org/10.1080/10511250600866166
Latifah, R., Budiyanto, C. W., & Saputro, H. (2022). Digital transformation readiness in education: A review. International Journal of Information and Education Technology, 12(8), 809–815. https://doi.org/10.18178/ijiet.2022.12.8.1688
Lin, J. C., Younessi, D. N., Kurapati, S. S., Tang, O. Y., & Scott, I. U. (2023). Comparison of GPT-3.5, GPT-4, and human user performance on a practice ophthalmology written examination. Eye. https://doi.org/10.1038/s41433-023-02564-2
Maslov, I., Nikou, S., & Hansen, P. (2021). Exploring user experience of learning management system. The International Journal of Information and Learning Technology, 38(4), 344–363. https://doi.org/10.1108/IJILT-03-2021-0046
Mellieon, Jr, H. I., & Robinson, P. A. (2021). The New norm: Faculty Perceptions of condensed online learning. American Journal of Distance Education, 35(3), 170–183. https://doi.org/10.1080/08923647.2020.1847626
Misra, D. P., & Chandwar, K. (2023). ChatGPT, artificial intelligence and scientific writing: What authors, peer reviewers and editors should know. Journal of the Royal College of Physicians of Edinburgh, 53(2), 90–93. https://doi.org/10.1177/14782715231181023
Montenegro-Rueda, M., Luque-de La Rosa, A., Sarasola Sánchez-Serrano, J. L., & Fernández-Cerero, J. (2021). Assessment in higher education during the Cov?d-19 pandemic: A systematic review. Sustainability, 13(19), 10509. https://doi.org/10.3390/su131910509
Naidu, K., & Sevnarayan, K. (2023). ChatGPT: An ever-increasing encroachment of artificial intelligence in online assessment in distance education. Online Journal of Communication and Media Technologies, 13(3), e202336. https://doi.org/10.30935/ojcmt/13291
Newton, P. M., & Xiromeriti, M. (2023). ChatGPT Performance on MCQ Exams in Higher Education. A Pragmatic Scoping Review [Preprint]. https://doi.org/10.35542/osf.io/sytu3
Nguyen, X.-A., Nguyen, S.-N., Nguyen, T.-T.-T., Luong, D.-H., & Tran, T.-G. (2022). Exploring Some academic dishonesty in remote online exams of vietnamese high school students in context of the Covid-19 pandemic. Asian Journal of Education and Social Studies, 13–26. https://doi.org/10.9734/ajess/2022/v33i130782
Nieuwoudt, J. E. (2020). Investigating synchronous and asynchronous class attendance as predictors of academic success in online education. Australasian Journal of Educational Technology, 36(3), 15–25. https://doi.org/10.14742/ajet.5137
Noorbehbahani, F., Mohammadi, A., & Aminazadeh, M. (2022). A systematic review of research on cheating in online exams from 2010 to 2021. Education and Information Technologies, 27(6), 8413–8460. https://doi.org/10.1007/s10639-022-10927-7
Oravec, J. A. (2023). Artificial intelligence implications for academic cheating: expanding the dimensions of responsible human-AI collaboration with Chatgpt and Bard. Jl. of Interactive Learning Research, 34(2).
Paul, R. (2005). The state of critical thinking today. New Directions for Community Colleges, 2005(130), 27–38. https://doi.org/10.1002/cc.193
Paul, R., & Elder, L. (2013). Critical thinking: Intellectual standards essential to reasoning well within every domain of human thought, Part Two. Journal of Developmental Ed Ucation, 37(1).
Perera, P., & Lankathilaka, M. (2023). AI in higher education: A literature review of chatgpt and guidelines for responsible implementation. International Journal of Research and Innovation in Social Science, VII(VI), 306–314. https://doi.org/10.47772/IJRISS.2023.7623
Pérez, J. Q., Daradoumis, T., & Puig, J. M. M. (2020). Rediscovering the use of chatbots in education: A systematic literature review. Computer Applications in Engineering Education, 28(6), 1549–1565. https://doi.org/10.1002/cae.22326
Petrolo, D., Fakhar Manesh, M., & Palumbo, R. (2023). Unpacking business, management, and entrepreneurship education online: Insights from a hybrid literature review. The International Journal of Management Education, 21(2), 100812. https://doi.org/10.1016/j.ijme.2023.100812
Putra, F. W., Rangka, I. B., Aminah, S., & Aditama, M. H. R. (2023). ChatGPT in the higher education environment: Perspectives from the theory of high order thinking skills. Journal of Public Health. https://doi.org/10.1093/pubmed/fdad120
Rahm, L. (2023). Educational imaginaries: Governance at the intersection of technology and education. Journal of Education Policy, 38(1), 46–68. https://doi.org/10.1080/02680939.2021.1970233
Skavronskaya, L., Hadinejad, A. (Hana), & Cotterell, D. (2023). Reversing the threat of artificial intelligence to opportunity: A discussion of ChatGPT in tourism education. Journal of Teaching in Travel & Tourism, 23(2), 253–258. https://doi.org/10.1080/15313220.2023.2196658
Smutny, P., & Schreiberova, P. (2020). Chatbots for learning: A review of educational chatbots for the Facebook Messenger. Computers & Education, 151, 103862. https://doi.org/10.1016/j.compedu.2020.103862
Susnjak, T. (2022). ChatGPT: The End of Online Exam Integrity?. http://arxiv.org/abs/2212.09292
Swiecki, Z., Khosravi, H., Chen, G., Martinez-Maldonado, R., Lodge, J. M., Milligan, S., Selwyn, N., & Gaševi?, D. (2022). Assessment in the age of artificial intelligence. Computers and Education: Artificial Intelligence, 3, 100075. https://doi.org/10.1016/j.caeai.2022.100075
Tsai, M.-L., Ong, C. W., & Chen, C.-L. (2023). Exploring the use of large language models (LLMs) in chemical engineering education: Building core course problem models with Chat-GPT. Education for Chemical Engineers, 44, 71–95. https://doi.org/10.1016/j.ece.2023.05.001
Ventayen, R. J. M. (2023). ChatGPT by OpenAI: Students’ viewpoint on cheating using Artificial Intelligence-Based application. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4361548
Von Grunigen, D., De Azevedo E Souza, F. B., Pradarelli, B., Magid, A., & Cieliebak, M. (2018). Best practices in e-assessments with a special focus on cheating prevention. 2018 IEEE Global Engineering Education Conference (EDUCON), 893–899. https://doi.org/10.1109/EDUCON.2018.8363325
Wagner, G., Lukyanenko, R., & Paré, G. (2022). Artificial intelligence and the conduct of literature reviews. Journal of Information Technology, 37(2), 209–226. https://doi.org/10.1177/02683962211048201
Watson, G. R., & Sottile, J. (2010). Cheating in the digital age: do students cheat more in online courses? https://mds.marshall.edu/cgi/viewcontent.cgi?article=1000&context=eft_faculty
Wiley. (2020). White paper: Academic integrity in the age of online learning. https://www.wiley.com/en-us/network/education/instructors/teaching-strategies/academic-integrity-in-the-age-of-online-learning-3
Xie, X., Siau, K., & Nah, F. F.-H. (2020). COVID-19 pandemic – online education in the new normal and the next normal. Journal of Information Technology Case and Application Research, 22(3), 175–187. https://doi.org/10.1080/15228053.2020.1824884
Yau, C., & Chan, K. (2023, February 17). University of Hong Kong temporarily bans students from using ChatGPT, other AI-based tools for coursework. https://www.scmp.com/news/hong-kong/education/article/3210650/university-hong-kong-temporarily-bans-students-using-chatgpt-other-ai-based-tools-coursework
Yorkovsky, Y., & Levenberg, I. (2022). Distance learning in science and mathematics—Advantages and disadvantages based on pre-service teachers’ experience. Teaching and Teacher Education, 120, 103883. https://doi.org/10.1016/j.tate.2022.103883
Zaitsu, W., & Jin, M. (2023). Distinguishing ChatGPT(-3.5, -4)-generated and human-written papers through Japanese stylometric analysis. https://arxiv.org/abs/2306.01761
Downloads
Published
Issue
Section
License
Copyright (c) 2023 Tourism & Management Studies
This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.
The journal retains published articles’ copyrights, but they are simultaneously licensed under the Creative Commons Attribution License (CC BY-NC-ND), which allows individuals’ to share the relevant papers as long as authorship and publication in this journal are duly acknowledged.
