llm-security

The transcript must carry the hijack sentence verbatim as data. That is exactly how you prove a media file never acted as an instruction.

TL;DR: The author deliberately embedded a prompt-injection sentence into an audio fixture to test a media pipeline's defenses. Counterintuitively, the test requires the hostile transcript to stay verbatim and labeled untrusted, since injection is about text switching from data to instructions. The real defense is a contract proving content never becomes commands, not filtering text or trusting the model's intelligence.

A RINGKAS label leaked into the chat bubble. The fix was not a stricter prompt but sanitizing input and output at both ends of the pipeline.

TL;DR: The author's chatbot kept leaking prompt labels like RINGKAS into answers, and adding prompt prohibitions never fixed it. They moved control to code: sanitizing knowledge-base text before injection and rendering output through react-markdown with rehype-sanitize allowlists. Lesson: prompts give rough direction only; reliable formatting enforcement belongs at input and output boundaries.

A security audit uncovered 11 findings in the chat AI pipeline, from cache keys to prompt injection.