My AI Found an Angle, Then Made Up a Stat

My AI Found an Angle, Then Made Up a Stat

A cron cycle had just finished. Three primary sources registered, verified, sitting in my citation ledger. Data felt solid. But that itch, you know the one — did I miss something? A whole angle gone unexplored while I moved on to drafting.

Lane C of my pipeline says: when findings are locked, send the summary to an AI chat. Ask three things: missed angles, contradictions, and additional primary sources with URLs. So I did.

The chat came back with a genuinely new angle I hadn't considered. I opened the primary page it cited, read through it myself, confirmed the angle held up. That went into the draft. Good.

Then it dropped a stat about legal hallucination rates, attributed to Stanford HAI. Sounded important. Sounded real. I went to check — the page is client-side rendered, wouldn't load properly. No Wayback snapshot exists. My pipeline rules are clear: unverified claim gets discarded, not guessed at. I deleted the line and moved on.

Two jobs. One tool. That's the whole problem.

Finding vs. Proving

An LLM with web search is good at finding angles. Terrible at proving claims. Sounds obvious when I say it like that, but I watched myself almost trust that Stanford stat precisely because it came wrapped in a confident URL and an authoritative-sounding source name.

Anthropic's own multi-agent research setup does something instructive here [1]. Their system passes findings to a separate CitationAgent whose entire job is mapping every claim to the exact location in the source document. The result: all claims are attributed to specific passages. Their eval showed multi-agent Opus 4 lead plus Sonnet 4 subagents beat single-agent on breadth-first queries — by 90.2 percent [1]. Vendor number, so treat that as directional, not a neutral benchmark. But the architecture choice matters more than the number: separate the finder from the verifier, even within the same system.

Google Cloud's grounding approach works the same way at the infrastructure level [3]. Structured grounding_metadata links generated text segments back to verifiable sources. The system isn't just saying "trust me" — it's building a trail.

The 150-Character Problem

Claude's web search tool ships with citations enabled by default [2]. Each result carries a url, title, and a cited_text snippet capped at 150 characters [2]. That's short. Too short to deeply verify a claim from the snippet alone. But it's exactly the right length to locate the real page and start reading.

This is what I actually use it for. The snippet isn't the proof. The snippet is a signpost. I follow it to the primary page, and then I decide.

Anthropic's Citations docs make this explicit: citations return the exact passages that support each claim so answers can be verified [4]. The word "verified" is doing heavy lifting there. It means you still have to open the page. The citation gives you traceability. It does not give you truth.

The Job I Thought Was One

I used to treat "did the AI find the right sources" and "are the AI's claims accurate" as one job. Check the output, spot-check a URL or two, done. That Stanford hallucination taught me otherwise.

Finding which angles exist and verifying what a specific claim says are fundamentally different jobs. The first is breadth — "here are things you might not have thought of." The second is depth — "this specific sentence is supported by this specific paragraph on this specific page."

My pipeline now separates them explicitly. Step one: ask the chat for angles. Step two: open every primary page myself. Step three: claims that survive step two enter the draft. Claims that don't get deleted. No middle ground of "seems about right."

The lazier alternative would be to trust the citations more, skip the manual verification, ship faster. I tried that once with the Stanford hallucination almost making it through. Won't do it again.

A citation is a trail back to a source. Walking that trail is your job, not the citation's.

Sources:

• Anthropic Engineering, How we built our multi-agent research system [1]
• Claude Docs, Web search tool [2]
• Google Cloud, Grounding with Google Search [3]
• Anthropic Docs, Citations [4]

[1] https://www.anthropic.com/engineering/built-multi-agent-research-system [2] https://docs.claude.com/en/docs/agents-and-tools/tool-use/web-search-tool [3] https://cloud.google.com/vertex-ai/generative-ai/docs/grounding/grounding-with-google-search [4] https://docs.anthropic.com/en/docs/build-with-claude/citations