About this paper
A novel framework addresses the limitations of current reinforcement learning-based red-teaming methods by employing dictionary learning and on-policy adversarial training to improve the robustness of LLM agents against indirect prompt injection attacks.
Large language model (LLM) agents routinely interact with untrusted external data, exposing them to indirect prompt injection (IPI) attacks that hijack agents into executing attacker-chosen tasks. To uncover these vulnerabilities at scale, automated red-teaming methods often leverage reinforcement learning (RL) to generate attacks that achieve high attack success rates (ASR). Model developers use the attacks discovered during internal red-teaming to harden agents through adversarial training before deployment, a collaborative offensive-defensive process known in cybersecurity as purple-teaming. However, we show that achieving high ASR during red-teaming does not guarantee robustness gains…
Large language model (LLM) agentsIndirect prompt injection (IPI) attacksReinforcement learning (RL)Automated red-teamingAdversarial trainingPurple-teamingMode collapseDictionary learning-based rewardOn-policy adversarial trainingAgentic benchmarks
- Upvotes
- 2
- Authors
- 4
- Published
- Nov 18, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Trending history
PIPETTE: Prompt Injection Purple-Teaming for Robust LLM Agents was first seen on Hugging Face · Daily papers on Nov 19, 2026. Since then it has appeared in 9 checks on 2 different days. Its best position was #23 of 30, reached on Nov 19, 17:29 UTC. It dropped off the list after Nov 20, 07:29 UTC.