There are plenty of reasons to panic about the effect of AI on our current assessment practices in HE and adult education. Any essay question we ask can be answered by typing in a prompt and copying and pasting an answer into the appropriate field. Plagiarism detection software can only hope to keep up with the rapid developments in AI as new iterations are released all the time. Here are some ideas for you to use when coming up with next term’s assessment activities
Worry not
There are plenty of reasons not to worry about AI and the potential downfall of humanity when designing your assessment strategy and individual assessments.
- It’s just a language model
The technology is a language model and depends heavily on the source text to produce an answer to a prompt. The source text (the internet) is limited in time, scope, language and authorship. Clever prompting is key to getting a decent result and this is a skill worth teaching.
2. It is frequently inaccurate
Where the software doesn’t find information that answers the question asked, it will make stuff up, even references. Erroneous information is easy to spot and can be a base criterion for acceptability of a submission. Fact checking and analysis are key skills for students to have and AI can be a useful tool in helping to teach them.
3. Essays are not the only way
Over dependence on generic questions that ask for general understanding of a field have long been contentious as a method of assessment. With technology on our side we have plenty of other options.
Authentic Assessment
First, refocus your assessment on more authentic activities. This means observable actions and outputs that are sufficiently personalised to make the viable source text so limited that it isn’t really usable. For example, a pitch for a non-existent product that is something your engineering students might actually produce, an infographic explaining a process, a filled in sprint plan that evidences reflective thinking and well designed experiments, a Miro board and session plan for how you would teach the target knowledge to someone else.
Critical Thinking
Alternatively, lean into the possibilities that the AI affords. For instance, ask students to get the tool to produce an essay on the Societal Impact of AI Generated Content.
When the internet became widely used, we had to encourage learners to think critically about what they were reading, to arrive at a conclusion of whether something was true or not, accurate and helpful or not or what biases informed the writing. Those same thinking skills are much in demand now too with the AI output.
Analysis
So with their AI generated essay, give students an analytical task. Ask them to read the essay, ascertain the truth of what is written in light of what they have learned on your course. they could uncover the biases inherent in the AI output and what causes them. What they submit is then not the essay but a reflective piece on the process and product of the exercise, e.g. what additional learning occurred as a result, and how to refine the output to make a viable, publishable piece of content.
Useful content?
A question worth asking is how useful is the content you are asking your students to produce. Is it useful to other learners in this field or to the public more generally or is it a pure test of recall? I would argue that recall can be tested in many other simpler ways. If we are asking students to write an essay, ie. produce content, then it should be of use to others. Think of David Wiley’s work on Renewable Assessment. We don’t want to flood the world with useless work. Ask students to produce things that are of value and add unique perspectives. Use ‘usefulness’ as a marking criterion.
Of course AI tools can generate more than just essays so you can combine the two approaches of personalised, authentic tasks with critical thinking, analysis and adaptation.
For example, (executive education) writing a business proposal in support of an innovative pivot in activities or approach. The students could have AI generate an initial proposal by providing rich prompts and responding to output with further prompts to refine, tone, feasibility, context etc. They would be working through multiple iterations of the proposal, analysing each, identifying areas that need improvement and making those changes happen either through manual updates or more precise prompting to get to a proposal that is factually accurate in a specific context and persuasively written.
The process is both authentic and rich in learning and agency and the output is immediately valuable to the learner and their business.