Artificial intelligence-based large language models and integrity of exams and assignments in higher education: the case of tourism courses

Authors

  • Abdullah Ülkü Harran University, School of Tourism and Hotel Management, Department of Tourism Guidance, ?anl?urfa/Turkey

Keywords:

Large Language Models (LLM), Artificial Intelligence, ChatGPT, Tourism Education, Exams, Integrity.

Abstract

There is an increasing concern regarding the potential misuse of ChatGPT-4 in compromising the integrity of examinations and assignments. This study aims to examine the capabilities of ChatGPT-4 in critical thinking abilities, whether it poses a threat to examinations and assignments in higher education, and create a discussion agenda on this issue. ChatGPT-4 was asked to generate, answer, and criticize questions in tourism marketing, tourism management, tourism economics, tourist guidance, and gastronomy. The answers were evaluated according to universal critical thinking standards. The findings obtained from this study showed that ChatGPT-4 had commendable competence in several critical thinking standards and could produce human-like texts.  However, there are certain domains that might be improved to comply more effectively with the expectations and norms of academia. Educators were recommended to use comprehensive approaches that combine technological and educational techniques to address the issue of cheating enabled by tools, such as ChatGPT-4, during assessment and exam processes.

Author Biography

  • Abdullah Ülkü, Harran University, School of Tourism and Hotel Management, Department of Tourism Guidance, ?anl?urfa/Turkey
    Abdullah Ülkü (Orcid no: 0000-0002-0937-2252) completed  his doctorate degree at Konya Necmettin Erbakan University, Department of Tourism Management. He is currently working as Assistant Professor in the Department of Tourist Guidance at Harran University School of Tourism and Hotel Management. He is interested in digital transformation in tourism and artificial intelligence.

References

Ababneh, K. I., Ahmed, K., & Dedousis, E. (2022). Predictors of cheating in online exams among business students during the Covid pandemic: Testing the theory of planned behavior. The International Journal of Management Education, 20(3), 100713. https://doi.org/10.1016/j.ijme.2022.100713

Adzima, K. (2021). Examining online cheating in higher education using traditional classroom cheating as a guide. Electronic Journal of E-Learning, 18(6). https://doi.org/10.34190/JEL.18.6.002

Agasisti, T., & Soncin, M. (2021). Higher education in troubled times: On the impact of Covid-19 in Italy. Studies in Higher Education, 46(1), 86–95. https://doi.org/10.1080/03075079.2020.1859689

AlAfnan, M. A., Dishari, S., Jovic, M. & Lomidze, K. (2023). Chatgpt as an educational tool: opportunities, challenges, and recommendations for communication, business writing, and composition courses. Journal of Artificial Intelligence and Technology, 3, 60-68. https://doi.org/10.37965/jait.2023.0184

Ali, K., Barhom, N., Marino, F. T., & Duggal, M. (2023). The thrills and chills of chatgpt: ?mplications for assessments in undergraduate dental education [Preprint]. MEDICINE & PHARMACOLOGY. https://doi.org/10.20944/preprints202302.0513.v1

Atlas, S. (2023). Chatgpt for higher education and professional development: a guide to conversational AI. University of Rhode Island DigitalCommons@URI. https://digitalcommons.uri.edu/cba_facpubs/548

Barrientos, A., Cuadros, M., Alba, J., & Cruz, A. S. (2021, October 27-29). Implementation of a remote system for the supervision of online exams through the use of cameras with artificial intelligence. 2021 IEEE Engineering International Research Conference (EIRCON), 1–4. https://doi.org/10.1109/EIRCON52903.2021.9613352

Barrot, J. S. (2023). Using ChatGPT for second language writing: Pitfalls and potentials. Assessing Writing, 57, 100745. https://doi.org/10.1016/j.asw.2023.100745

Benvenuti, M., Cangelosi, A., Weinberger, A., Mazzoni, E., Benassi, M., Barbaresi, M., & Orsoni, M. (2023). Artificial intelligence and human behavioral development: A perspective on new skills and competences acquisition for the educational context. Computers in Human Behavior, 107903. https://doi.org/10.1016/j.chb.2023.107903

Bernabei, M., Colabianchi, S., Falegnami, A., & Costantino, F. (2023). Students’ use of large language models in engineering education: A case study on technology acceptance, perceptions, efficacy, and detection chances. Computers and Education: Artificial Intelligence, 100172. https://doi.org/10.1016/j.caeai.2023.100172

Carvalho, L., Martinez-Maldonado, R., Tsai, Y.-S., Markauskaite, L., & De Laat, M. (2022). How can we design for learning in an AI world? Computers and Education: Artificial Intelligence, 3, 100053. https://doi.org/10.1016/j.caeai.2022.100053

Chirumamilla, A., & Sindre, G. (2019). Mitigation of cheating in online exams: Strengths and limitations (A. V. S. Kumar, Ed.). IGI Global. https://doi.org/10.4018/978-1-5225-7724-9

Choi, J. H., Hickman, K. E., Monahan, A., & Schwarcz, D. B. (2023). ChatGPT goes to law school. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4335905

Coghlan, S., Miller, T., & Paterson, J. (2020). Good proctor or “Big Brother”? AI ethics and online exam supervision technologies. https://arxiv.org/abs/2011.07647.

Comas-Forgas, R., Lancaster, T., Calvo-Sastre, A., & Sureda-Negre, J. (2021). Exam cheating and academic integrity breaches during the COVID-19 pandemic: An analysis of internet search activity in Spain. Heliyon, 7(10), e08233. https://doi.org/10.1016/j.heliyon.2021.e08233

Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2023). Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International, 1–12. https://doi.org/10.1080/14703297.2023.2190148

Crawford, J., Cowling, M., Central Queensland University, Australia, Allen, K.-A., & Monash University, Australia. (2023). Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI). Journal of University Teaching and Learning Practice, 20(3). https://doi.org/10.53761/1.20.3.02

Currie, G. M. (2023). Academic integrity and artificial intelligence: Is ChatGPT hype, hero or heresy? Seminars in Nuclear Medicine, 53(5), 719–730. https://doi.org/10.1053/j.semnuclmed.2023.04.008

David, A., Mihai, D., Mihailescu, M.-E., Carabas, M., & Tapus, N. (2022). Scalability through distributed deployment for moodle learning management system. Procedia Computer Science, 214, 34–41. https://doi.org/10.1016/j.procs.2022.11.145

Dendir, S., & Maxwell, R. S. (2020). Cheating in online courses: Evidence from online proctoring. Computers in Human Behavior Reports, 2, 100033. https://doi.org/10.1016/j.chbr.2020.100033

Dias, J., Kamdi, D., Gharat, N., & Chudhari, P. (2019). Chatbot for Government Examination using AI. Journal of Engineering, 9(4), 58-62.

Duhaim, A. M., Al-mamory, S. O., & Mahdi, M. S. (2022). Cheating detection in online exams during Covid-19 pandemic using data mining techniques. Webology, 19(1), 341–366. https://doi.org/10.14704/WEB/V19I1/WEB19026

Eke, D. O. (2023). ChatGPT and the rise of generative AI: Threat to academic integrity? Journal of Responsible Technology, 13, 100060. https://doi.org/10.1016/j.jrt.2023.100060

Elbanna, S., & Armstrong, L. (2023). Exploring the integration of ChatGPT in education: Adapting for the future. Management & Sustainability: An Arab Review. https://doi.org/10.1108/MSAR-03-2023-0016

Essel, H. B., Vlachopoulos, D., Tachie-Menson, A., Johnson, E. E., & Baah, P. K. (2022). The impact of a virtual teaching assistant (chatbot) on students’ learning in Ghanaian higher education. International Journal of Educational Technology in Higher Education, 19(1), 57. https://doi.org/10.1186/s41239-022-00362-6

Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2023). A SWOT analysis of ChatGPT: Implications for educational practice and research. Innovations in Education and Teaching International, 1–15. https://doi.org/10.1080/14703297.2023.2195846

Geerling, W., Mateer, G. D., Wooten, J., & Damodaran, N. (2023a). ChatGPT has Mastered the Principles of Economics: Now What? SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4356034

Gerashchenko, I. P., & Kovalev, V. A. (2021). Formation of educational ecosystems through the digital transformation of the educational environment. SHS Web of Conferences, 121, 03004. https://doi.org/10.1051/shsconf/202112103004

Gilson, A., Safranek, C., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2022). How Does ChatGPT Perform on the Medical Licensing Exams? The ?mplications of large language models for medical education and knowledge assessment [Preprint]. Medical Education. https://doi.org/10.1101/2022.12.23.22283901

Gökta?, L. S. (2023). ChatGPT uzaktan e?itim s?navlar?nda ba?ar?l? olabilir mi turizm alan?nda do?ruluk ve do?rulama üzerine bir ara?t?rma (Can ChatGPT succeed in distance education exams a research on accuracy and verification in tourism). Journal of Tourism and Gastronomy Studies, 2. https://doi.org/10.21325/jotags.2023.1224

Hashim, H. (2018). Application of technology in the digital era education. International Journal of Research in Counseling and Education, 1(2). https://doi.org/10.24036/002za0002

Hill, B. (2023). Taking the help or going alone: ChatGPT and class assignments. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4465833

Holden, O. L., Norris, M. E., & Kuhlmeier, V. A. (2021). Academic ?ntegrity in online assessment: A research review. Frontiers in Education, 6, 639814. https://doi.org/10.3389/feduc.2021.639814

Hwang, A. (2018). Online and Hybrid Learning. Journal of Management Education, 42(4), 557–563. https://doi.org/10.1177/1052562918777550

Ilynykh, S. (2021). Online education: Trends and risks of digital technologies: VIII International Scientific and Practical Conference “Current problems of social and labour relations” (ISPC-CPSLR 2020), Makhachkala, Russia Federation. https://doi.org/10.2991/assehr.k.210322.126

Jungherr, A. (2023). Using ChatGPT and Other Large Language Model (LLM) applications for academic paper assignments [Preprint]. SocArXiv. https://doi.org/10.31235/osf.io/d84q6

King, D. L., & Case, C. J. (2014). E-Cheating: Incidence and trends among college students. Issues In Information Systems, 15(1). https://doi.org/10.48009/1_iis_2014_20-27

Klijn, F., Mdaghri Alaoui, M., & Vorsatz, M. (2022). Academic integrity in on-line exams: Evidence from a randomized field experiment. Journal of Economic Psychology, 93, 102555. https://doi.org/10.1016/j.joep.2022.102555

Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., & Tseng, V. (2023). Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digital Health, 2(2), e0000198. https://doi.org/10.1371/journal.pdig.0000198

Lanier, M. M. (2006). Academic integrity and distance learning. Journal of Criminal Justice Education, 17(2), 244–261. https://doi.org/10.1080/10511250600866166

Latifah, R., Budiyanto, C. W., & Saputro, H. (2022). Digital transformation readiness in education: A review. International Journal of Information and Education Technology, 12(8), 809–815. https://doi.org/10.18178/ijiet.2022.12.8.1688

Lin, J. C., Younessi, D. N., Kurapati, S. S., Tang, O. Y., & Scott, I. U. (2023). Comparison of GPT-3.5, GPT-4, and human user performance on a practice ophthalmology written examination. Eye. https://doi.org/10.1038/s41433-023-02564-2

Maslov, I., Nikou, S., & Hansen, P. (2021). Exploring user experience of learning management system. The International Journal of Information and Learning Technology, 38(4), 344–363. https://doi.org/10.1108/IJILT-03-2021-0046

Mellieon, Jr, H. I., & Robinson, P. A. (2021). The New norm: Faculty Perceptions of condensed online learning. American Journal of Distance Education, 35(3), 170–183. https://doi.org/10.1080/08923647.2020.1847626

Misra, D. P., & Chandwar, K. (2023). ChatGPT, artificial intelligence and scientific writing: What authors, peer reviewers and editors should know. Journal of the Royal College of Physicians of Edinburgh, 53(2), 90–93. https://doi.org/10.1177/14782715231181023

Montenegro-Rueda, M., Luque-de La Rosa, A., Sarasola Sánchez-Serrano, J. L., & Fernández-Cerero, J. (2021). Assessment in higher education during the Cov?d-19 pandemic: A systematic review. Sustainability, 13(19), 10509. https://doi.org/10.3390/su131910509

Naidu, K., & Sevnarayan, K. (2023). ChatGPT: An ever-increasing encroachment of artificial intelligence in online assessment in distance education. Online Journal of Communication and Media Technologies, 13(3), e202336. https://doi.org/10.30935/ojcmt/13291

Newton, P. M., & Xiromeriti, M. (2023). ChatGPT Performance on MCQ Exams in Higher Education. A Pragmatic Scoping Review [Preprint]. https://doi.org/10.35542/osf.io/sytu3

Nguyen, X.-A., Nguyen, S.-N., Nguyen, T.-T.-T., Luong, D.-H., & Tran, T.-G. (2022). Exploring Some academic dishonesty in remote online exams of vietnamese high school students in context of the Covid-19 pandemic. Asian Journal of Education and Social Studies, 13–26. https://doi.org/10.9734/ajess/2022/v33i130782

Nieuwoudt, J. E. (2020). Investigating synchronous and asynchronous class attendance as predictors of academic success in online education. Australasian Journal of Educational Technology, 36(3), 15–25. https://doi.org/10.14742/ajet.5137

Noorbehbahani, F., Mohammadi, A., & Aminazadeh, M. (2022). A systematic review of research on cheating in online exams from 2010 to 2021. Education and Information Technologies, 27(6), 8413–8460. https://doi.org/10.1007/s10639-022-10927-7

Oravec, J. A. (2023). Artificial intelligence implications for academic cheating: expanding the dimensions of responsible human-AI collaboration with Chatgpt and Bard. Jl. of Interactive Learning Research, 34(2).

Paul, R. (2005). The state of critical thinking today. New Directions for Community Colleges, 2005(130), 27–38. https://doi.org/10.1002/cc.193

Paul, R., & Elder, L. (2013). Critical thinking: Intellectual standards essential to reasoning well within every domain of human thought, Part Two. Journal of Developmental Ed Ucation, 37(1).

Perera, P., & Lankathilaka, M. (2023). AI in higher education: A literature review of chatgpt and guidelines for responsible implementation. International Journal of Research and Innovation in Social Science, VII(VI), 306–314. https://doi.org/10.47772/IJRISS.2023.7623

Pérez, J. Q., Daradoumis, T., & Puig, J. M. M. (2020). Rediscovering the use of chatbots in education: A systematic literature review. Computer Applications in Engineering Education, 28(6), 1549–1565. https://doi.org/10.1002/cae.22326

Petrolo, D., Fakhar Manesh, M., & Palumbo, R. (2023). Unpacking business, management, and entrepreneurship education online: Insights from a hybrid literature review. The International Journal of Management Education, 21(2), 100812. https://doi.org/10.1016/j.ijme.2023.100812

Putra, F. W., Rangka, I. B., Aminah, S., & Aditama, M. H. R. (2023). ChatGPT in the higher education environment: Perspectives from the theory of high order thinking skills. Journal of Public Health. https://doi.org/10.1093/pubmed/fdad120

Rahm, L. (2023). Educational imaginaries: Governance at the intersection of technology and education. Journal of Education Policy, 38(1), 46–68. https://doi.org/10.1080/02680939.2021.1970233

Skavronskaya, L., Hadinejad, A. (Hana), & Cotterell, D. (2023). Reversing the threat of artificial intelligence to opportunity: A discussion of ChatGPT in tourism education. Journal of Teaching in Travel & Tourism, 23(2), 253–258. https://doi.org/10.1080/15313220.2023.2196658

Smutny, P., & Schreiberova, P. (2020). Chatbots for learning: A review of educational chatbots for the Facebook Messenger. Computers & Education, 151, 103862. https://doi.org/10.1016/j.compedu.2020.103862

Susnjak, T. (2022). ChatGPT: The End of Online Exam Integrity?. http://arxiv.org/abs/2212.09292

Swiecki, Z., Khosravi, H., Chen, G., Martinez-Maldonado, R., Lodge, J. M., Milligan, S., Selwyn, N., & Gaševi?, D. (2022). Assessment in the age of artificial intelligence. Computers and Education: Artificial Intelligence, 3, 100075. https://doi.org/10.1016/j.caeai.2022.100075

Tsai, M.-L., Ong, C. W., & Chen, C.-L. (2023). Exploring the use of large language models (LLMs) in chemical engineering education: Building core course problem models with Chat-GPT. Education for Chemical Engineers, 44, 71–95. https://doi.org/10.1016/j.ece.2023.05.001

Ventayen, R. J. M. (2023). ChatGPT by OpenAI: Students’ viewpoint on cheating using Artificial Intelligence-Based application. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4361548

Von Grunigen, D., De Azevedo E Souza, F. B., Pradarelli, B., Magid, A., & Cieliebak, M. (2018). Best practices in e-assessments with a special focus on cheating prevention. 2018 IEEE Global Engineering Education Conference (EDUCON), 893–899. https://doi.org/10.1109/EDUCON.2018.8363325

Wagner, G., Lukyanenko, R., & Paré, G. (2022). Artificial intelligence and the conduct of literature reviews. Journal of Information Technology, 37(2), 209–226. https://doi.org/10.1177/02683962211048201

Watson, G. R., & Sottile, J. (2010). Cheating in the digital age: do students cheat more in online courses? https://mds.marshall.edu/cgi/viewcontent.cgi?article=1000&context=eft_faculty

Wiley. (2020). White paper: Academic integrity in the age of online learning. https://www.wiley.com/en-us/network/education/instructors/teaching-strategies/academic-integrity-in-the-age-of-online-learning-3

Xie, X., Siau, K., & Nah, F. F.-H. (2020). COVID-19 pandemic – online education in the new normal and the next normal. Journal of Information Technology Case and Application Research, 22(3), 175–187. https://doi.org/10.1080/15228053.2020.1824884

Yau, C., & Chan, K. (2023, February 17). University of Hong Kong temporarily bans students from using ChatGPT, other AI-based tools for coursework. https://www.scmp.com/news/hong-kong/education/article/3210650/university-hong-kong-temporarily-bans-students-using-chatgpt-other-ai-based-tools-coursework

Yorkovsky, Y., & Levenberg, I. (2022). Distance learning in science and mathematics—Advantages and disadvantages based on pre-service teachers’ experience. Teaching and Teacher Education, 120, 103883. https://doi.org/10.1016/j.tate.2022.103883

Zaitsu, W., & Jin, M. (2023). Distinguishing ChatGPT(-3.5, -4)-generated and human-written papers through Japanese stylometric analysis. https://arxiv.org/abs/2306.01761

Downloads

Published

31.10.2023

Issue

Section

Tourism/Hospitality: Research Papers

How to Cite

Artificial intelligence-based large language models and integrity of exams and assignments in higher education: the case of tourism courses. (2023). Tourism & Management Studies, 19(4), 21-34. https://www.tmstudies.net/index.php/ectms/article/view/2242

Most read articles by the same author(s)

1 2 3 4 5 6 7 8 9 10 > >>