Informing students of the limitations of GenAI can also help to increase understanding for why students are not allowed to use it on a particular assignment or in a particular course. Some points to touch on during a discussion of GenAI limitations are:
AI is Basically an Advanced Autocomplete Tool, and Can Be Wrong
Current AI models use advanced probabilistic models to predict the next most likely letter in a sequence of text and generate outputs in this sequence (Grassini, 2023). There is no fact-checking built into LLMs; they are sophisticated models that predict which words should come next based on the data they were trained on (Sharples, 2022). Since “ChatGPT is trained on a large corpus of unpolished, raw data” (Grassini, 2023, p. 4), outputs can be unreliable.
AI Cannot Accurately Cite Scholarly Sources
Outputs can be incorrect, and errors are more common when querying AI on specialized areas of knowledge (Grassini, 2023). ChatGPT also generates nonexistent references that nevertheless look real (Sharples, 2022).
Using AI Does Not Mean That You Now Have That Skill or Knowledge