The Mechanical Redaction Gate That Runs Before Publish
A 9,000-character silent truncation cut the redaction rules out of the writing prompt. The fix that mattered was a mechanical gate on the final artifact.
TL;DR
A style guide was silently truncated at 9,000 characters, cutting the redaction rules before they reached the writer model. The fix loads the guide whole and adds a mechanical verification gate, check_redaction, that scans the final article for non-documentation IPs and secret-like token prefixes. False positives cost minutes; published leaks are permanent, so the trade-off is easy.
While tidying up the article pipeline's style layer yesterday, I hit a snippet that made me put my coffee down: a system prompt in one of the engines was being hard-cut at 9,000 characters, while the style guide had outgrown that. Part of the guide simply never reached the writer model. And the part that got cut was not just any part: the redaction rules, exactly the section that keeps IPs and credentials from leaking.
Yesterday's commit fixed this in two layers. First, the guide is now loaded whole in both engines, no more silent slicing. Second, and more important: article verification got a new mechanical gate, check_redaction, running inside cmd_verify. The logic is simple. A prompt can get truncated, get ignored by the model, get overwritten by a revision. The final artifact about to be published has no room to negotiate.
Why good intentions can't run this
The OWASP secrets management cheat sheet still has to remind people of something that sounds ancient: many organizations still put credentials straight into source code in plaintext [3]. GitHub built secret scanning that sweeps entire Git histories for the same reason, a committed credential becomes a target for unauthorized access [2]. If a private repo can leak through commit history, a blog post that has been indexed by a search engine is far more permanent. There is no credential rotation for a paragraph that is already live. Tools in the gitleaks class bake this principle right into their exit codes: 0 means no leaks, 1 means a leak or an error [4].
That is why I don't trust the deal of "I'll just remember to check while writing". The gate has to be mechanical, fail-level, and applied to the entire content, code blocks included. A leak inside a fence is still a leak; a scanner that skips code blocks only creates false comfort.
The rules I put in place
For IPv4 addresses, the gate uses an allowlist: the only IPs allowed in an article are the three documentation blocks from RFC 5737, namely 192.0.2.0/24, 198.51.100.0/24, and 203.0.113.0/24 [1]. The RFC itself says these blocks should never appear on the public Internet [1], which makes them ideal for examples: realistic in format, guaranteed to collide with nobody's server.
For tokens, detection is prefix-based and absolute: patterns like AKIA, ghp_, sk-, and xox fail outright with no exemptions. It's a somewhat cruel decision, because even an example key from AWS's official docs gets caught. I left it that way. A prefix is structure, not a value, and structure needs no context to judge. The sk- pattern was also widened to catch hyphens, so newer key formats like sk-proj- don't slip through because of a dash.
The single compromise lives in assignment heuristics like password = "...". There, explicit placeholders are exempted: a value with a clear marker like <DB_PASSWORD> or YOUR_ passes, a random 8-plus-character string fails. Without this exemption the gate would nag at its own examples forever, and a gate that nags too much gets switched off. That is more dangerous than the compromise itself.
Check it yourself, don't take my word for it
The core of the gate is this small:
import re
IP_RE = re.compile(r"\b(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\b")
ALLOWED = ("192.0.2.", "198.51.100.", "203.0.113.")
def ip_aman(text: str) -> bool:
return all(ip.startswith(ALLOWED) for ip in IP_RE.findall(text))
assert ip_aman("ping 192.0.2.55")
assert ip_aman("backup ke 198.51.100.7")Run it and both asserts pass. Now swap the strings for your office's server addresses and the asserts blow up instantly. That is exactly what the gate does on every verify: compare every IP it finds against the three documentation prefixes, one outside means failure.
The selftest in the commit is even harsher, and the harshness is deliberate: a private IP must fail, the three TEST-NET blocks must pass, a 36-character GitHub PAT must fail, a placeholder like <YOUR_API_KEY> must be skipped instead, and a password assignment holding a random string must get caught. If any single one flips direction, verification stops before anyone gets to publish.
The trade-off I accepted is clear: sometimes there's a false positive, a legitimate example gets flagged. But a false positive costs five minutes of swapping to a documentation range. A false negative means internal topology readable forever in an archive. Not a hard choice.
If you run any publishing pipeline, the portable principle is: validate the final artifact, not the writer's intent. A system that nags early is cheap. A leak that's already indexed has no comparable price.