That Time a Control Flag Showed Up in the Chat Bubble
A raw CTA_FLAG: YES marker leaked into a live chat bubble. The strip regex only matched end-of-string, and the SSE stream bypassed it entirely.
TL;DR
Marker CTA_FLAG muncul di UI karena regex-nya pakai anchor $ sehingga hanya cocok kalau flag ada di akhir teks, padahal LLM menaruhnya di tengah paragraf. Masalah makin parah karena jawaban streaming via SSE, jadi strip pasca-proses telat menangkap chunk yang sudah terkirim. Solusinya: strip di jalur streaming dan CTA kini diambil dari field terstruktur knowledge source, bukan parsing teks model.
Testing the AI answer feature on staging, I scrolled to the last chat bubble and saw CTA_FLAG: YES rendered as plain text right below a paragraph of AI-generated answer. The WhatsApp contact button was there too, looking correct, but the raw marker was just... visible. Like seeing stage directions left in a movie script.
I opened the code that strips markers. The regex was \s*CTA_FLAG:\s*YES\s*$ with a $ anchor, so it only matched if the flag sat at the very end of the string. Seemed fine when I wrote it. The LLM prompt says "put this at the end." But that's the problem: a prompt isn't a contract [2]. The model dropped the marker in the middle of a paragraph instead, and the regex didn't catch it because $ only looks at the end.
The Regex Anchor That Wasn't Enough
I removed the $ from the regex. Problem solved? Not quite.
The app streams answers over SSE. Each token delta gets pushed to the browser as it arrives [3]. The strip function runs after the full response is assembled, but the deltas are already flying out to the client in real-time. So even with the fixed regex, if the marker happened to land in an early chunk, it was already streamed to the UI before anyone tried to strip it.
Two different code paths. One gate to pass through. Classic case of assuming everything funnels through the same checkpoint.
The marker landed mid-text because the model treated the prompt instruction as a suggestion, not a format spec. And the streaming architecture meant post-processing couldn't catch what was already sent. These aren't two independent bugs. They're the same architectural weakness wearing two masks: in-band signaling is fragile because you can't control where it appears, and streaming makes it worse because you can't retroactively edit what's already out the door.
The fix touched both sides. In use_cases.py, the streaming path now strips the marker from the collected answer before it moves on, and emits its own event when it does. In routes.py, the old logic that appended CTA text to the answer is gone; the post-processor only strips. But I also changed something bigger: the CTA data now comes from a structured cta field on the knowledge source itself, not from whatever the LLM decides to write in its response.
Structured Fields Over Parsed Text
Frontend renders the WhatsApp button with the verified-check icon directly from that structured cta field in AiAnswer.tsx. The model generates the prose answer. The CTA button is pulled from the data layer. They don't touch each other.
This is the part I feel strongly about: if your output needs a specific shape, don't parse it out of free-form text. Use structured output. OpenAI's docs are clear that JSON mode guarantees valid JSON, but schema adherence requires structured outputs [1]. Even then, the model can still hallucinate fields or miss them. The real win is removing the dependency entirely, which is what pulling the CTA from the knowledge source does.
There's a pattern here that goes beyond this one flag. Any time you put a control marker in-band with the model's text output, you're betting on the model placing it exactly where you expect. The model is not a compiler. It doesn't follow format specs the way code does. OWASP's prompt injection guidance says mitigation through system prompts and input handling helps, but it isn't a complete prevention [2]. That's not a reason to give up on prompts. It's a reason not to rely on them for structural guarantees.
React's useEffect only runs on the client, after mount [4]. That's a different corner of the same mental model: paths that skip the gate you assumed covers everything. The structured-field approach sidesteps the timing question entirely, because the CTA button reads from props that are there from the start.
I'm not saying every piece of LLM output needs a structured envelope. But the boundary between "model prose" and "UI needs this to render correctly" is exactly where you should stop trusting the model's formatting and start using fields. The model generates the answer. The system provides the CTA. Clean separation, no regex hoping.
Sources
- [1] Structured model outputs | OpenAI API (accessed 2026-09-01)
- [2] LLM01:2025 Prompt Injection | OWASP Gen AI Security Project (accessed 2026-09-01)
- [3] Using server-sent events | MDN (accessed 2026-09-01)
- [4] useEffect – React (accessed 2026-09-01)
- Related: One SSE Stream, Two AI Answers