Good practice guide - Designing assessment for Artificial Intelligence and academic integrity

At Flinders University, we want to harness the power of artificial intelligence (AI) in our teaching and assessment but at the same time educate our students so that they know how to use it ethically and avoid potential academic integrity issues. This tip sheet aims to assist you in fearlessly supporting your students.

Design process

We recommend that you take the following steps to design for academic integrity and AI.

Click here to open image on mobile devices

Artificial Intelligence Scales

Use the following scales when designing for Artificial Intelligence:

Designing Authentic Assessment

The most important consideration of designing for academic integrity with the advent of artificial intelligence is authentic assessment design.

Providing students with authentic assessments and specifying what help they are permitted to use for each activity or task helps students to prepare for their professions.

Design that discourages the use of Artificial Intelligence:

• Carefully analyse all prompts and questions as well as the conditions under which assessments are completed to ensure that students are not able to easily use AI.
• ‘Test’ prompts by running them through ChatGPT and analyse the output. If the AI generated work is satisfactory, then rework the prompt. Note that this activity helps the tool to learn, so make sure that your final prompt is completely different.
• Ensure that emphasis is on critical thinking and reasoning in all assessments.
• Focus on the process as well as the product with drafts and outlines also assessed (Turnitin, 2023, p.1-3).
• Develop increasingly complex tasks investigated over a sustained period (Turnitin, 2023, p.2) which allow for “competing solutions and diversity of outcomes” (Sridharan & Mustard, 2015, p.4).
• Build a positive culture of learning and good relationships with your students where they are clear on task and behavioural expectations. This will also prevent anxiety and academic misconduct.
• Assess on material discussed in class (e.g. a summary of the tutorial discussion).
• Ask students to reflect on and embed their own experiences into the assessment response.

Using Artificial Intelligence in Assessment design

It is also possible to draw on the affordances of artificial intelligence in assessment design. Some ideas are provided in the section below.

Design that uses the affordances of AI:

AI in the planning stages of a task:

• Students use AI for an initial scan of the literature with the output on a topic providing them with key words and concepts to inform a search of the academic literature (Rudolph, Tan & Tan, 2023).
• Students use AI to identify the sections and elements needed for a text type.

AI as a core part of the task:

• Students use AI to improve transfer of knowledge skills. The output can be used to provide many examples related to a topic. The student can then be asked to test their understanding and name and explain inaccuracies, gaps, and missing aspects on a topic (Mollick & Mollick, 2022, pp. 3-8).
• Students use AI to learn evaluation. If the output explains a concept step by step or produces an example essay, students then enhance the output by considering the order to steps and re-thinking the depth of their knowledge about the topic” and/or critiquing and improving an essay (Mollick & Mollick, 2022, 00. 3-8).
• Students use AI to learn critical thinking and reflection skills. They reflect on inputs and outputs, annotate, and then develop their own work.
• Students use AI for divergent thinking and the generation of ideas. They then use AI in the convergent thinking process where solutions (prototypes) are developed, tested and evaluated.

AI for self-testing:

• Have students generate quiz questions for themselves on the content to facilitate revision.
• Use AI to explain why an answer/ solution is correct or what other options may be.

AI as a copyediting tool:

• After first completing a task themselves, students can compare their language and formatting choices with the AI output and make edits to their own work. They should be careful not to put their own work through the AI tool unless explicitly told to do so as this will also assist in the learning of the tool.
• Students use AI to identify and improve story style and tone.

Note:

• It is important to teach your students to formally acknowledge the use of artificial intelligence and other support services used.
• Students should also be transparent about the use of AI and clearly indicate what prompts they used to generate the results as well as reflect on the improvements.

*Acknowledgement:

An initial scan of the topic was conducted using ChatGPT output with the following question posed: How can student academic integrity breaches using ChatGPT be prevented? The document was refined with further feedback from the Learning and Teaching Innovation Committee and the following prompt developed by Prashant Pandey: What do institutions need to do to change the culture of assessment to keep up to date with the AI developments, especially ChatGPT. The document was refined through feedback by the Deans Education and Digital Literacy and Artificial Intelligence Working Party.