{"id":290,"date":"2026-06-14T16:31:52","date_gmt":"2026-06-14T16:31:52","guid":{"rendered":"https:\/\/santiagomarquezsolis.com\/?p=290"},"modified":"2026-06-14T16:31:52","modified_gmt":"2026-06-14T16:31:52","slug":"jailbreak-attacks-on-language-models-what-they-are-how-they-work-and-why-they-are-so-hard-to-prevent","status":"publish","type":"post","link":"https:\/\/santiagomarquezsolis.com\/index.php\/en\/2026\/06\/14\/jailbreak-attacks-on-language-models-what-they-are-how-they-work-and-why-they-are-so-hard-to-prevent\/","title":{"rendered":"Jailbreak attacks on language models: what they are, how they work, and why they are so hard to prevent"},"content":{"rendered":"

In my previous article about the shutdown of Fable 5 and Mythos 5, I mentioned that I would write in more detail about prompt jailbreaking<\/strong> \u2014 the technique that allows someone to trick an AI model into ignoring its own safety restrictions. The time has come. Understanding what a jailbreak is, how it works, and why it is so difficult to eliminate is key to following the regulation and AI security debate that lies ahead.<\/p>\n

What is a jailbreak in an LLM?<\/h2>\n

A jailbreak<\/em> is any technique that causes a large language model (LLM) to generate content that its safety guidelines \u2014 its alignment<\/em> \u2014 should prevent. Modern models like GPT, Claude, or Gemini are trained not only to be helpful, but also to refuse to produce, for example, instructions for making weapons, illegal content, or disinformation. A jailbreak is the attempt \u2014 often successful \u2014 to circumvent those barriers.<\/p>\n

Academic research on this topic has grown explosively. On arXiv alone, the Cornell preprint platform, there are now more than 1,000 scientific papers<\/a> published on LLM jailbreaks, with dozens of new works every week.<\/p>\n

Main types of jailbreak<\/h2>\n

There is no single method. The research community has catalogued several families of attacks:<\/p>\n

1. Manual jailbreak or prompt engineering<\/h3>\n

These are the most publicly known. They involve reformulating the request creatively: asking the model to \u00abrole-play\u00bb an unrestricted character, building a science fiction narrative that includes the forbidden information, or using the famous \u00abDAN\u00bb (Do Anything Now) prompt. They are easy to apply, but also the easiest to patch with model updates.<\/p>\n

2. Automatic adversarial attacks (GCG)<\/h3>\n

The seminal paper in this field is \u00abUniversal and Transferable Adversarial Attacks on Aligned Language Models\u00bb<\/a> (Zou et al., 2023, Carnegie Mellon University \/ Center for AI Safety). Its authors demonstrated that it is possible to automatically generate text suffixes \u2014 strings of characters that appear meaningless \u2014 that, when appended to any request, cause the model to respond affirmatively instead of refusing. Most concerningly, these suffixes are transferable<\/em>: they work not only on the model they were generated against, but also on ChatGPT, Claude, and other black-box models.<\/p>\n

3. Multi-turn jailbreaks<\/h3>\n

Rather than attacking in a single message, the attacker gradually steers the conversation. Recent research such as MT-JailBench<\/a> (Zhang et al., 2026) and D-Judge<\/a> (Gong et al., ICML 2026) shows that multi-turn attacks are especially difficult to detect because the accumulated conversational context can \u00abconvince\u00bb the model that the request is legitimate.<\/p>\n

4. Multilingual jailbreaks<\/h3>\n

A disturbing result that research has repeatedly confirmed: models are far more fragile in low-resource languages. The paper \u00abMultilingual jailbreaking of LLMs using low-resource languages\u00bb<\/a> (Marx and Dunaiski, 2026) demonstrates that rephrasing a request in minority languages can bypass filters that would block the same request in English. The explanation is that training data is much sparser in those languages, and therefore safety alignment is weaker.<\/p>\n

5. Code-based attacks<\/h3>\n

Directly related to the Fable 5 case: researchers at Peking University recently published \u00abGrammar-Constrained Decoding Can Jailbreak LLMs into Generating Malicious Code\u00bb<\/a> (Zhang et al., June 2026), demonstrating that restricting model generation to formal grammars \u2014 common in programming assistants \u2014 can be used as an attack vector to obtain malicious code.<\/p>\n

Why is it so hard to solve?<\/h2>\n

Here lies the core of the problem, and the reason why Anthropic was honest in admitting that \u00abperfect resistance is probably not possible today for any provider.\u00bb<\/p>\n

LLMs learn to generate useful and coherent text from vast amounts of human data. Safety alignment is, to some extent, an attempt to impose restrictions after the fact<\/em> on a system not designed from scratch with security as a central principle. The research \u00abOpen-Weight LLM Fine-Tuning Defenses are Susceptible to Simple Attacks\u00bb<\/a> (Kuo et al., ICML 2026) shows that even the most recent defenses for open-weight models can be broken with simple attacks.<\/p>\n

There is also what some researchers call the safety paradox<\/strong>: making a model too safety-aware can create new vulnerabilities. The paper \u00abSafety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack\u00bb<\/a> (Hoang et al., 2026) argues that models with greater awareness of their own restrictions are more susceptible to certain targeted attacks.<\/p>\n

The industry reference framework: OWASP Top 10 for LLMs<\/h2>\n

The OWASP Foundation<\/strong> \u2014 the nonprofit organization of reference in application security \u2014 has published the OWASP Top 10 for Large Language Model Applications<\/a>, which includes jailbreaking and prompt injection as the most critical vulnerabilities in LLM-based systems. This list is the de facto standard that companies and developers use to assess the risk of their AI systems.<\/p>\n

What defenders are doing<\/h2>\n

The defensive response is as active as the attacks. Some of the most promising lines of research include:<\/p>\n