Observable criteria and levels

How can you use AI to build a rubric that distinguishes learning outcomes and reduces discrepancies between assessors?

1250 € per session, excl. VAT

In short

In a half-day session of 3 hours participants use generative AI to turn learning objectives into observable criteria, describe levels of proficiency and test a marking scheme. They leave with a documented rubric, revised after comparing marks awarded to fictional work.

Why this training?

A criterion such as ‘quality of the response’ leaves excessive room for interpretation. Defining the expected evidence and levels of proficiency helps explain a mark, provide useful feedback and identify learning that is not yet secure. AI can suggest criteria and rephrase descriptors, but it can also introduce requirements that are absent from the task instructions. The training teaches participants to check this alignment and test the rubric against work rather than accept its first version. Running case study: The fictional organisation Avelune Pédagogie needs to assess an exercise involving a written response to a complaint. Two assessors interpret the relevance of the proposed solution differently. Participants build a shared rubric, then test it on three fictional responses showing contrasting levels of proficiency. Transfer to the workplace: For an upcoming assessment, adapt the rubric to a real learning objective, anonymise the work and have a sample of submissions marked by two assessors. Record disagreements, clarify the relevant descriptors and date the agreed version before using it more widely.

Objectives of this training

  • Derive observable criteria from a learning objective.
  • Use AI to write four distinct levels of proficiency.
  • Allocate marks in line with the importance of the learning outcomes being assessed.
  • Compare two sets of marks and identify ambiguous descriptors.
  • Revise the rubric and justify the changes made.

Who is it for and what are the prerequisites?

Roles and tasks concerned

  • Instructional designers preparing assessments of written work.
  • Trainers and training managers wishing to harmonise their marking criteria.

Prior skills

  • Have previous experience of writing and refining a prompt in a generative AI assistant.
  • Be able to distinguish a learning objective from exercise instructions.

What you will work on

  1. Link objectives and evidence

    Objective: Define what the rubric should actually measure. Differences between an objective, a criterion, an indicator and evidence; excluding expectations not stated in the task instructions.

  2. Describe levels of proficiency

    Objective: Obtain descriptors that can be used for marking. Structured prompt: objective, task instructions, criteria, levels and format; checking for invented additions and vague wording.

  3. Build a marking scheme that can be explained

    Objective: Make the calculation and allocation of marks transparent. Weighting criteria, adding up marks, missing evidence and risks of double counting.

  4. Compare marks

    Objective: Gauge ambiguities and correct the rubric. Independent marking, comparison by criterion, justification using evidence and traceability of changes.

What you take away

  • Alignment table linking objectives, evidence and criteria.
  • Four-level rubric with descriptors and a marking scheme.
  • Comparative record of marks and reasoned corrections.
  • Instructions for use and a reusable AI prompt.

Method and assessment

  • Short input sessions followed by immediate practice using the running case study.
  • Individual work followed by work in pairs, with peer review of the work produced.
  • Guided group discussion: what works, what needs correcting and what remains a human decision.
  • Preparation: Prepare a fictional or anonymised learning objective and set of task instructions; a shared case study is provided if necessary. Check access to an AI tool authorised by the organisation and to a resource that allows you to edit a table.

Observable criteria: Each criterion refers to evidence that can be identified in the expected work. ; The four levels describe observable differences, rather than merely using evaluative adjectives. ; The marking scheme adds up to the stated total and avoids awarding marks twice for the same learning outcome. ; Marking discrepancies are recorded by criterion and linked to a revision decision. ; The instructions specify how to handle missing evidence and justify the marks awarded..

Frequently asked questions

Does AI award the final mark?

No. It helps write the rubric; the assessor checks the evidence and remains responsible for marking.

Can real submissions be used?

Only after effective anonymisation and in accordance with the organisation’s rules; the exercises primarily use fictional work.

Is marking out of twenty compulsory?

No. It is used for the practical case study; a descriptive scale may be suitable for formative assessment.

Is the rubric suitable for all exercises?

No. Each new set of task instructions requires the expected evidence, criteria and levels to be checked.

Last updated: