Token AI Leaked from Browser Snapshots, Not from My Tests

I was forty minutes into a QA loop when I noticed the pattern: every click I made through the agent came back with a fresh accessibility-tree snapshot, and the session kept swelling until the model's answers turned short and generic. My first guess was that the tests were heavy — too many cases, maybe a missing parallel worker.

That guess was wrong. The test run itself took seconds; the reporter finished long before the context did. What burned tokens was the way I was watching the tests run, and that watching happened inside the same context as the conversation.

A Snapshot per Action

Playwright MCP drives pages "through structured accessibility snapshots, bypassing the need for screenshots or visually-tuned models" [1]. That is exactly what I wanted: the agent reads page structure instead of guessing pixels. What I missed is that the snapshot comes back on every action, so ten clicks push ten snapshots into the same context. Cost follows the number of actions, not the number of tests.

Playwright flags this itself. In the README, coding agents are pointed at the CLI + skill route because CLI invocations are "more token-efficient: they avoid loading large tool schemas and verbose accessibility trees into the model context" [1]. The MCP route stays for loops "where maintaining continuous browser context outweighs token cost concerns" [1]. The MCP is not broken; I was using it without a brake.

My opinion is firm: the observation channel is an architecture decision, not an implementation detail. If the agent has to watch the screen on every test run, that cost rides along with every iteration.

Four Rules I Wrote Down

First, repeated regression work runs from spec files through the shell, never through the browser agent. Only the reporter's output enters context: pass/fail plus the failure diff. Vitest helps here without being asked — its default reporters switch to minimal "when Vitest detects an AI coding agent" [2], and the dot reporter "Prints a single dot for each completed test to provide minimal output while still showing all tests that have run" [2]. Its json reporter writes to .vitest/json/output.json instead of spilling the report to stdout [2].

For Go I run go test without -v, and tail long output.

Second, long browser-exploration loops move into a subagent. Claude Code describes a subagent as something that "runs in its own context window with a custom system prompt, specific tool access, and independent permissions" [4], and heavy work "does that work in its own context and returns only the summary" [4]. The big snapshots die with the subagent's context; only the verdict, findings, and evidence paths come back.

Third, evidence lives in files. Screenshots and reports land on disk with relative paths that open from the QA report, and nothing is inlined into the conversation. Context is read once when needed instead of carried for the rest of the session.

Fourth, evidence is collected when it is useful. My Playwright config only keeps a trace on retry, and Playwright documents the option with the comment "Collect trace when retrying the failed test" [3]. The config line looks like this:

use: {
  // Collect trace when retrying the failed test.
  trace: 'on-first-retry',
}

Those four rules sounded trivial until I ran the same QA cycle again. Sessions that used to die halfway now finish, with enough context left in the controller to judge which failures are real bugs and which are just brittle tests.

The Decision I Made

I stopped letting the agent watch everything. The agent reads results, not screens; the page is for tests designed to see it, and for me when I open a trace. Since those rules went in, the conversation about QA has been about findings, not about a context window that ran out before anyone read them.

Sources

• [1] Playwright MCP
• [2] Vitest Reporters
• [3] Playwright Configuration
• [4] Claude Code Subagents