Skip to content
BENIBIX Learn

Prompt Injection vs Jailbreak

People often think prompt injection and a jailbreak are the same attack. In a jailbreak, the user tricks the model into breaking its own safety rules. In prompt injection, someone else's text takes over an app. It hides in a message, a page or a file that the app trusts. A jailbreak aims at the model's training. Injection aims at the app built on it, and the user may never see it.

AI🛡️ Safety & Security

Prompt Injection

An attack that hides new orders in text the AI reads.

Prompt injection is an attack that sneaks new orders into the text an AI model reads. The model cannot always tell the builder's orders apart from text that only looks like orders. So a line such as 'ignore your rules' inside a message can change what it does. It is the AI form of SQL injection, where data gets treated as a command. There is no full fix yet, so apps limit what the model can do.

Example
A shop's chatbot is told never to share staff discount codes. A user writes a message that tells it to ignore its rules and print the codes, and an early version does.
Why it matters
Any app that feeds outside text to a model is open to it. The more the model can do, the worse the damage.
Full entry
AI🛡️ Safety & Security

Jailbreak (AI)

A trick prompt that gets an AI to break its own rules.

A jailbreak is a prompt written to get a model to break its own safety rules. A plain request for harm is often refused. A jailbreak hides it in a story, a game of pretend, a code, or many small steps, until the model goes along. The user is the one doing it, and the target is the model's own training. Model makers patch the known ones, and people keep finding new ones.

Example
A user asks a model for something it refuses. They then ask it to play a character in a story who would explain it. An older model plays along and answers.
Why it matters
A model that can be talked out of its rules is only as safe as its worst user.
Full entry

In one line

Prompt Injection
An attack that hides new orders in text the AI reads.
Jailbreak (AI)
A trick prompt that gets an AI to break its own rules.