testing

When a selftest scores 14/19, the first instinct is to blame the validator. The real cause — broken bash fixture transforms — is a reminder that a reliable failure is more honest than a rigged green.

TL;DR: After pushing commit bc77087, the selftest scored 14/19, and I initially blamed my new validator logic. The real culprit was defective bash fixture transforms producing invalid JSON, so the validator correctly rejected corrupt input. I froze the suite at 14/19 with documented defects rather than patch the fixture, because honest reliability beats inflated vanity metrics.

A data validator flagging false positives isn't just noisy — it destroys developer trust. Here is how I added a negative-control corpus and exact reason-code assertions to turn a paranoid validator into a precise one.

TL;DR: The validator flagged a blank source as unverified because it treated missing facts as failures rather than negative controls. A new 20-sentence negative-control corpus now fails the test if false positives exceed 10 percent. Seven exact reject reason-codes plus a positive control ensure bad data gets rejected for the right reasons.

A green suite proves nothing if a fake can pass it. I fed three fake implementations into my 32-test selftest to prove the grader works.

TL;DR: After my 32-test suite went green, I worried it might be vacuous, so I built three fake validators (always pass, always fail, always emit every error code) that must each score zero passes. This is manual mutation testing, a known technique. I also pinned the selftest's sha256 so silent grader edits get caught.

How commit 6f5ae82 turned a deliberately failing selftest into a frozen CLI contract: red tests first, implementation later, no silent drift.

TL;DR: The author's selftest showed 32 failures, all intentional: that's the red step of TDD, freezing the CLI contract before any implementation exists. It's a batch-freeze variant rather than Canon TDD, accepting rework risk since a stable spec beats iteration rhythm for a CLI. Synthetic fixtures keep tests honest, and next comes implementing until green.