LLMs: foundations and vigilance
How do you distinguish a plausible response from a verified one when querying an LLM?
1250 € per session, excl. VAT
In short
“Understanding LLMs and their limitations” is a private in-person session of half a day for iT and product staff discovering conversational assistants or project managers and support staff wishing to assess generated texts. Groups of up to 12 participants, in French, with exercises on your own work cases.
Why this training?
An LLM produces text based on learned patterns; the fluency of its response does not prove its accuracy. Understanding this distinction helps identify fabricated claims, shortcuts in reasoning and references that seem credible without being verifiable. In an IT and product context, an incorrect summary can turn a hypothesis into a technical feature. This introduction teaches participants to check outputs against the available dossier, flag missing information and retain human validation. Ongoing case study: The fictitious organisation « Veldorne Instruments » is preparing an internal briefing note on a new sensor. Its dossier contains a technical data sheet, a test report and a question about compatibility with no documented answer. Participants examine a generated summary and identify what can be retained, corrected or left unresolved. Transfer to the workplace: After the session, reuse the verification checklist on an authorised anonymised document. Compare each generated claim with its source and note the corrections needed before any distribution.
Objectives of this training
- Explain the concepts of tokens, training and context in simple terms.
- Distinguish between text generation, document research and fact-checking.
- Identify an unsupported claim in a generated summary.
- Formulate a request specifying the task, sources and missing information.
- Assess a response using an explicit assessment grid.
Who is it for and what are the prerequisites?
Roles and tasks concerned
- IT and product staff discovering conversational assistants.
- Project managers and support staff wishing to assess generated texts.
Prior skills
- No prior AI knowledge required.
- Ability to consult a digital document and enter text in an interface.
What you will work on
From text to predictions
Objective: Describe how an LLM works without excessive technical jargon. Tokens, training, step-by-step generation and the context window. The difference between learned knowledge and documents provided in a conversation.
Observing the limitations
Objective: Identify why a convincing response can be incorrect. Hallucinations, biases, potentially outdated knowledge, ambiguities and variability. Access to external sources depends on the environment used.
Framing requests
Objective: Observe the effect of a precise prompt on a summary. Task, scope of documents, expected format and handling of unknowns. A better prompt does not replace verification.
Verify before reusing
Objective: Measure the factual quality of a response and decide how to use it. Traceability of claims, coverage of expected elements, confidentiality and human validation. Limitations of checking against a small dossier.
What you take away
- Annotated diagram showing a simplified view of how an LLM works.
- Tested prompt for summarising a technical dossier.
- Table of claims with sources and corrections.
- Practical verification checklist for use before reuse.
Method and assessment
- Short teaching inputs followed by immediate practice using the ongoing case study.
- Individual work followed by work in pairs, with participants reviewing each other’s outputs.
- Facilitated group discussion: what works, what needs correcting and what remains a human decision.
- Preparation: Arrange access to a generative AI assistant authorised by the organisation; if this is unavailable, use the prepared outputs for the exercise. Consult the internal confidentiality rules; the exercises use only fictitious or anonymised data.
Observable criteria: The explanation distinguishes model training from use of context. ; The prompt explicitly asks for missing information to be flagged. ; Each claim examined is classified as supported, contradicted or undocumented. ; The proportion of supported claims is calculated across the claims examined. ; The final checklist indicates corrections and points requiring human validation..
Frequently asked questions
Do you need to know how to programme?
No. The exercises involve prompts and short documents.
Does the model systematically consult the Internet?
No. You need to check the capabilities and access permissions of the authorised environment.
Can you use internal documents?
The session is limited to fictitious or anonymised cases, using authorised tools.
Is a response accompanied by sources reliable?
Every reference must be consulted: its existence does not prove that it supports the claim.
Last updated: