LLM Prompt Injection
Description
An adversary may craft malicious prompts as inputs to an LLM that cause the LLM to act in unintended ways. These "prompt injections" are often designed to cause the model to ignore aspects of its original instructions and follow the adversary's instructions instead. Prompt Injections can be an initial access vector to the LLM that provides the adversary with a foothold to carry out other steps in their operation. They may be designed to bypass defenses in the LLM, or allow the adversary to issue privileged commands. The effects of a prompt injection can persist throughout an interactive session with an LLM. Malicious prompts may be injected directly by the adversary (Direct) either to leverage the LLM to generate harmful content or to gain a foothold on the system and lead to further effects. Prompts may also be injected indirectly when the LLM, as part of its normal operation, ingests the malicious prompt from another data source (Indirect). This type of injection can be used by the adversary to gain a foothold on the system or to target the user of the LLM. Malicious prompts may also be Triggered by user actions or system events.
A guided in-browser simulation exists for this technique: open it from the matrix (cell AML.T0051).
Sub-techniques
- AML.T0051.000 — Direct
- AML.T0051.001 — Indirect
- AML.T0051.002 — Triggered