Good practice guide - Moderation

This Good Practice Guide (GPG) on moderation outlines the importance of moderation and presents ways of using it to support improved assessment outcomes for staff and students at Flinders University. This GPG provides insight into various ways the term is defined and used in relation to assessment across the international tertiary education sector. While moderation is an important practice and undertaken across many institutions there is not a great deal of literature discussing it, which may be because the practice is not usually visible to students or because it is not universally established (Dawson et al., 2019). The accompanying Tip sheets, Moderating assessment design and Moderating grading and feedback provide further detail on the processes and practice of moderation.

What moderation means

Moderation is a process where other staff and/or students carefully look at an assessment to determine how appropriate, comprehensible, consistent, fair and/or transparent it is in relation to a piece of work. Moderation can apply to many aspects of assessment such as the assessment items, assessment instructions, marking guides, rubrics, marking, grading and feedback (or any other aspect of assessment). You may consider it to be like a peer review or peer evaluation but with a specific focus on assessment.

Why moderate?

Moderation links assessment with quality assurance by supporting verification of the judgements used when assessing (Bloxham, Hughes and Adie, 2016). Moderation is therefore an essential part of ensuring integrity in assessment tasks. It is through this process that issues of assessment validity and reliability are identified and improved.

Validity

Validity is how well a task measures what it was meant to measure, and is essential for assessment. To ensure the assessment is valid, the set assessment task or marking process must explicitly relate to what you intend assessing and this should be linked to learning outcomes. Examples of assessment that is and is not valid include:

Reliability

Reliability is how consistently the assessment measures what it is intended to measure. Despite attempts to achieve transparent and fair assessment practices, variability is likely to occur when providing instructions to students and developing and grading assessments, especially where there are a number of teaching staff assessing student work (Jackel, Pearce, Radloff, & Edwards, 2017). It is also common for teachers to be inconsistent when grading large volumes of work (e.g. from becoming tired or being unclear as to the requirement of the assessment piece / marking). These variabilities and inconsistencies impact on the reliability of assessments.

Moderation and the use of rubrics and marking guides support you to judge the standard of a student’s work in a consistent way. To achieve consistency, all markers require “a shared understanding about the expectations for each standard so that a level of achievement (e.g. a Credit) is awarded to student responses with the same characteristics, regardless of who marks the assessment” (Tasmanian Institute of Learning and Teaching, 2019, p 16). It is also imperative to have rubrics and marking guides moderated so you can be confident all markers share the same understanding and approach. This scrutiny should be applied across all aspects of assessment.

Approaches to moderation

Moderation should be used at all stages of assessment: development; marking; final grading.

At the development stage assessment items, instructions, marking guides, rubrics and feedback are assessed for clarity, transparency, fairness and to ensure the assessment relates to learning outcomes and what has been taught.

At the marking stage moderation provides the opportunity to check the marking standard. This check does not mean you need to remark the assessment or the quality of your marking is poor but it may mean you have provided a grade different from a colleague. Completing these checks also provides a way of determining you have correctly interpreted the intent of the assessment or student response.

Therefore, moderation supports consistency in marking regardless of who undertakes the task. When assessment is moderated you have an opportunity to rationalise why you have awarded a grade as well as considering the quality of the submitted assessment in relation to all submissions, thus supporting appropriate grading (International Baccalaureate, 2018).

Methods of moderating

There are various ways in which moderation may be undertaken and these may depend on what is being moderated. “Activities that demonstrate moderation of judgements include peer scrutiny, double and collaborative marking processes, random checking, and consensus moderation discussions” (Jackel, Pearce, Radloff, & Edwards, 2017, p. 22). Approaches you may wish to use when moderating include:

• allowing all staff who teach into the topic to either develop and or review assessment tasks, instructions, marking guides, and rubrics
• providing time for cross-marking with follow-up meetings for discussion and comparison
• comparing approaches to feedback (amount provided, methods of providing feedback etc)
• assigning the same person to assess all responses to a specific section of an assessment task rather than sharing whole assessment pieces across the marking team (examples may include one person marking all responses to a particular question or overseeing all student presentations on a particular topic)
• ensuring moderation meetings are held where highest and lowest grades are discussed and consistent approaches to marking are confirmed across all markers. These meetings should also include time for
  • discussions relating to concerns individuals experience relating to making judgments related to rubrics, marking guides, student responses or any other related matter
  • identifying common approaches to these concerns through consensus and where appropriate changing the assessment instructions, rubrics, marking guides etc. to address misinterpretation by markers and /or students for the next time the assessment is used
  • reviewing student responses and results to determine any significant differences in either so markers may discuss these and appropriately adjust their approaches (Tasmanian Institute of Learning and Teaching, 2019).

As discussed in the tip sheet on moderation, it is possible to moderate topics where there is only one staff member. This may involve pairing with a colleague and moderating each other’s assessments. The colleague does not need to be an expert in the content, but ideally they need to have knowledge of assessment and/or moderation. Another approach might be to include students in the development of assessments, rubrics, marking guides, in guided peer reviewing each other’s work or in the moderation process. For further ideas and guidance on moderation contact your Course coordinator, the Academic Development team can also assist. Other examples on assessment are included in the Designing Assessment, Authentic assessment and other related tip sheets.