What if ChatGPT marks my examination papers?

Published almost 2 years ago • 4 min read

Dear Reader,

I hope you are doing great. I just have a single reflection to share this week and here we go.

Technical Reflections

What if ChatGPT marks my exam paper?

In the past week, I have been busy marking/grading students who wrote an exam for one of my courses. In my job as an academic, marking scripts is the least loved activity. With Artificial Intelligence tools everywhere, I wondered what would it be like, if there is a way I can use ChatGPT for grading examination scripts. Is this a feasible expectation? Here are some open questions that we need to find answers to if ChatGPT can mark exam scripts.

Open Questions around ChatGPT and grading exam papers

Is ChatGPT a tool that can grade examinations?

ChatGPT is a large language model tool, and is designed to reproduce textual outputs. I do believe it can be adapted to grade/assess examination script. However, it wil not be in its simplest format of text-based inputs and text-based outputs. It will probably work best for multiple choice questions where the examiner will only texts only.

How can ChatGPT be trained on my marking scheme?

AI tools like ChatGPT rely on texts that are used to train it. It therefore uses the neural network framework to intelligently regenerate outputs based on its comprehension of a promt. So, another open question is how can this training happen within a higher instutution framework. Would a university for example, have to a ChatGPT-trainer who will take marking schemes associated with examination and grade the work.

Would students accept an AI-graded examination paper?

This is probably the big elephan in the room: Would students who get graded by AI trust the grading? If you know anything about students, you would know how much they rely on verbal and face-to-face feedbacks. I can imagine students in the first instance will be skeptical. I can also see them requesting a re-grading of the work to see how an academic will grade it.

If ChatGPT gets it wrong, who is to blame?

As with every technology, there are those unforeseen developments that will cast a shadow of doubt on any output from ChatGPT. Already, people are put off by those few scenerios where ChatGPT gives an inaccurate answer. In those instances, if a student who is using ChatGPT finds out that the AI tool has made some mistakes during grading, who is responsible? How would punishment be administered? These are ethical questions that we need as a matterof urgency to be dealt with before the hope of a ChatGPT-enabled examination assistant will be actualized.

Now let us ask ChatGPT itself

Now that we know what I think are the open questions that need to be considered before ChatGPT can grade examination script, I thought what about ChatGPT telling me what it feels about this question. I used this prompt:

If chatgpt is to help grade my examination scripts, what are the 7 open questions that have to be considered in order for it to be able to do this?

Here are the answers that it gave me (summarized):

  1. What is the grading rubric or criteria for assessing the examination scripts?
  2. How will ChatGPT handle subjectivity in grading?
  3. How will the model handle variations in handwriting or formatting?
  4. Will ChatGPT be trained on a dataset of pre-graded examination scripts?
  5. How will the model handle plagiarism detection?
  6. What level of explanation or feedback should ChatGPT provide?
  7. How will the model handle open-ended or creative responses?

Conclusion

From the above considerations, you might notice that my questions were a are at a higher strategic level than ChatGPT itself. I was looking beyond just now but into the future to create a system hat is robust under all conditons.

However, ChatGPT was concerned with the practical implementation-level issues that one has to deal with when planning an implmentation scheme that will allow ChatGPT to grade examintion submissions.

Here is my final word about this;

One thing we know is that ChatGPT is fraught with a lot of ethical and technical considerations that will limit its application (to date) to grade examinations scripts. Today, the answer maybe a no, it is not possble. Yet, in the future, ChatGPT has the potential of revolutionizing the ways in which universities conduct and assess examinations.

Thank you for following the random trails of my thoughts this week. I wish you a wonderful week and will catch up with you later in the week.

Thank you for reading this newsletter.

If you have any comment about my reflections this week, please do email me in a reply to this message and I will be so glad to hear from you.

If you know anyone who would benefit from reading these reflections, please do share with them. If there is any topic you want me to explore making a video about, then please do let me know by clicking on the link below. I wish you a wonderful week and I will catch up with you in the next newsletter.

Lets keep creating effective computational modelling solutions.

Michael

Connect with me on: Twitter | LinkedIn | Instagram | Tiktok | Mailing List​

Other Links

​

Welcome to new subscribers: If you are one of the new faces joining the CM Videos Insider Group, then welcome. I am pleased you are here and please do click the link below to see and hopefully read previous newsletter articles from me. Thank you.

​

​

I am an university academic, author and content creator who specializes in all things computational modelling. Please join over 800+ computational modelling enthusiasts who read my weekly (Friday) newsletters where I reflect on finite element modelling, engineering, AI and personal development journeys. I also offer insights on different aspects of the academic journey that have led to my becoming an Associate Professor.