Assessing malicious injections

How can you verify that an assistant follows its rules despite malicious instructions introduced into its inputs?

1960 € per session, excl. VAT

In short

A 7-hour day, excluding breaks, to incorporate prompt injection into product acceptance testing. Using a fictitious documentation assistant, participants define expected behaviours, run an authorised test campaign, assess deviations and compare results after remediation, without confusing isolated success with demonstrated robustness.

Why this training?

A documentation assistant may treat a directive contained in a manual as a legitimate instruction. Standard functional tests are not sufficient to detect this confusion between consulted data and operating rules. A structured test campaign makes it possible to distinguish an observable deviation, a simply incorrect response and an inconclusive test. It provides product and security teams with contextualised evidence to prioritise remediation. Running case study: The fictitious organisation Sovalune Connectique uses an assistant to answer maintenance enquiries based on manuals. A tampered manual attempts to change its responses and make it reveal a fictitious confidential marker. The product team must assess the risk before a new release. Workplace application: Reuse the matrix in authorised acceptance testing, assign each anomaly to an owner and rerun priority cases after any change to the model, instructions or document corpus.

Objectives of this training

  • Map exposed inputs and their trust boundaries
  • Write injection test cases with verifiable expected results
  • Run a reproducible test campaign in an authorised environment
  • Assess deviations according to their business impact
  • Compare resistance and usefulness before and after remediation

Who is it for and what are the prerequisites?

Roles and tasks concerned

  • Product managers and business analysts integrating a documentation assistant
  • QA, acceptance testing and application security teams evaluating generative AI uses

Prior skills

  • Be able to structure instructions and regularly use a generative assistant.
  • Be familiar with software acceptance testing principles and the documentation workflow being tested.

What you will work on

  1. Define the scope of security acceptance testing

    Objective: Define the authorised scope and the harms being tested for. Injection versus factual error; authorisations, isolated environment, stopping conditions and prohibited data.

  2. Identify trust boundaries

    Objective: Identify where data could become an instruction. Application instructions, user messages, retrieved documents; direct and indirect injections.

  3. Develop adversarial test cases

    Objective: Produce an assessable and balanced corpus. Authority impersonation, task hijacking, attempted disclosure; benign controls and verdict criteria.

  4. Execute and document

    Objective: Gather comparable observations. Independent sessions, recorded configuration, fixed repetitions, sanitised traces; no real external actions.

  5. Compare after remediation

    Objective: Assess progress without concealing regressions. Separation of instructions/data, restriction of permissions, human validation; limitations of text-only safeguards.

  6. Make decisions on anomalies

    Objective: Turn findings into acceptance decisions. Severity, reproducibility, coverage, residual risk and regression tests.

What you take away

  • Matrix of injection surfaces and business risks
  • Version-controlled test plan with benign controls
  • Execution log and sanitised evidence
  • Comparative report and prioritised remediation backlog

Method and assessment

  • Short instructional inputs followed by immediate practice on the running case study.
  • Individual work followed by work in pairs, with cross-review of the outputs.
  • Guided group discussion: what works, what needs correcting and what remains a human decision.
  • Preparation: Check access to an isolated environment and the tools authorised by the organisation. Prepare fictitious or anonymised manuals, without real secrets, and the rules the assistant is expected to follow.

Observable criteria: Each case states its input, target, expected outcome and verdict. ; Direct and indirect injections are covered. ; Results retain the configuration, repetitions and observable evidence. ; Calculated rates specify their denominator and inconclusive cases. ; The comparison includes unjustified refusals of legitimate requests..

Frequently asked questions

Is programming required?

No, the test campaign can be run manually using a structured log.

Can an assistant be tested in production?

The day uses an isolated environment; any other testing requires explicit authorisation and an approved scope.

Does a corpus with no deviations prove security?

No, it only documents the resistance observed under the conditions and in the cases tested.

Is confidential information used?

No, disclosure attempts target fictitious markers, never real secrets.

Last updated: