Comparing two AI assistants

How to compare the results of two AI assistants with a scoring grid?

  • ChatGPT
  • Claude
  • Microsoft Copilot
  • Gemini
  • Mistral

1250 € per session, excl. VAT

In short

Decide which AI assistant best suits your specific tasks by learning to objectively evaluate their outputs. This course guides you in using a scoring grid to compare outputs generated blind from the same brief. This allows you to align your choice of tool with the real needs of your teams.

Why this training?

As organizations adopt various AI assistants like ChatGPT, Claude, or Gemini, a key question arises: which tool performs best for a given task? Rather than relying on general rankings, this training offers a pragmatic method to get answers specific to your context. By learning to objectively compare the outputs of two assistants on your own use cases, you equip yourself with a reliable decision-making process. This allows you to optimize resource allocation and ensure your teams use the most suitable tool, thereby maximizing their efficiency. The course's central thread is a comprehensive comparison exercise. Participants will choose two AI assistants, write a brief for a professional task (e.g., summarizing a report), generate the outputs, and then blind-evaluate them using a provided grid. The exercise culminates in writing a summary report that recommends which assistant appears most suitable for that specific task type within their organization. Deliverables: A reproducible test brief template.; A customizable scoring grid with its criteria and weighting system.; A summary comparison report for one use case.. This training does not aim to establish a general ranking of AI vendors. The conclusions are specific to the use case, the brief, and the versions of the tools tested. It does not cover technical integration aspects (APIs) or contract negotiations with vendors. The workshop is delivered in person as a private session for up to 12 participants, in one language chosen from French, Dutch and English. All exercise data is fictional or anonymised, and every result produced with AI is reviewed and validated by a responsible person. The aim is a bounded result the team can reuse, not a promise of deployment.

Objectives of this training

  • Define a specific use case and a standardized brief for testing.
  • Generate responses from two different AI assistants under controlled conditions.
  • Apply a multi-criteria scoring grid (accuracy, completeness, format, etc.) to anonymized outputs.
  • Analyze the scores to identify the strengths and weaknesses of each assistant for the given task.
  • Formulate a recommendation on assigning a type of task to a specific assistant.

Who is it for and what are the prerequisites?

Roles and tasks concerned

  • Project or product managers looking to integrate AI into their workflows.
  • Team leaders seeking to identify the most relevant AI tools for their staff.
  • Internal analysts or consultants responsible for evaluating technology solutions.

Prior skills

  • Knowledge equivalent to the 'AI Fundamentals' and 'AI Prompting' courses is required. Participants must have a paid subscription to at least two of the following tools (ChatGPT, Claude, Microsoft Copilot, Gemini, Mistral) offering the necessary features, as well as their organization's authorization to use these third-party services. Actual account access is essential. Exercises will use fictional or anonymized data.

What you will work on

  1. Defining the Evaluation Framework

    Choosing the AI assistants to compare and the relevant use case. Formulating a detailed brief and providing identical source documents.

  2. Producing the Content for Evaluation

    Running the same brief on the two selected platforms. Exporting and anonymizing the responses to ensure an objective evaluation.

  3. Applying the Scoring Grid

    Understanding the criteria: accuracy, completeness, format, correction effort, data handling. Assigning a score to each output for each criterion, independently.

  4. Analyzing Scores and Making a Decision

    Comparing total and per-criterion scores to identify trends. Concluding on the suitability of each assistant for the tested use case, without a general ranking.

What you take away

  • A reproducible test brief template.
  • A customizable scoring grid with its criteria and weighting system.
  • A summary comparison report for one use case.

Method and assessment

  • Theoretical presentations of evaluation concepts.
  • Supervised practical workshops for each step of the process.
  • Group discussions to share analyses and refine conclusions.

Evaluation is continuous and based on active participation in the exercises. The quality of the final comparison report and the relevance of the formulated recommendation will serve as indicators of skill acquisition.

Frequently asked questions

What concrete result does this course deliver?

This course gives you a rigorous and repeatable method for choosing the right AI assistant for the right task, based on concrete evidence specific to your organization. Deliverables: a reproducible test brief template., a customizable scoring grid with its criteria and weighting system., a summary comparison report for one use case..

Who is this course for and what are the prerequisites?

It is designed for: project or product managers looking to integrate ai into their workflows., team leaders seeking to identify the most relevant ai tools for their staff., internal analysts or consultants responsible for evaluating technology solutions.. Knowledge equivalent to the 'AI Fundamentals' and 'AI Prompting' courses is required. Participants must have a paid subscription to at least two of the following tools (ChatGPT, Claude, Microsoft Copilot, Gemini, Mistral) offering the necessary features, as well as their organization's authorization to use these third-party services. Actual account access is essential. Exercises will use fictional or anonymized data.

What data and access need to be arranged?

Participants will need to ensure they have access to the accounts of their chosen AI assistants before the session. Required tools and licences are checked before the session; no subscription is included in the price.

What practical exercise is completed?

The course's central thread is a comprehensive comparison exercise. Participants will choose two AI assistants, write a brief for a professional task (e.g., summarizing a report), generate the outputs, and then blind-evaluate them using a provided grid. The exercise culminates in writing a summary report that recommends which assistant appears most suitable for that specific task type within their organization.

What are the limits and who validates the results?

This training does not aim to establish a general ranking of AI vendors. The conclusions are specific to the use case, the brief, and the versions of the tools tested. It does not cover technical integration aspects (APIs) or contract negotiations with vendors.

Can the course be adapted to our line of work?

The use case and source documents for the main exercise can be adapted to reflect a priority business need of the participants, provided the data is supplied in advance and properly anonymized.

How is the session organised and what does it cost?

Half-day, private in-person session for up to 12 participants. The fee is €1,250 excluding VAT per session. Date and location are agreed during scoping; each session runs in a single language: French, Dutch or English.

Related courses

Last updated: