Event Log Integrity the Git Way: A Frame Tamper Story

Adityo Guni WaluyoSeptember 22, 202604:03

A one-byte frame tamper taught me how content-addressed sidecars, an append-only event log, and disk-truth verification keep a pipeline recoverable.

#git #event-sourcing #automation #python #devops

TL;DR

When a corrupted frame triggered media-quarantine instead of a crash, the kernel proved it rebuilds state from events and verifies files from disk, never trusting recorded hashes. Content-addressed caption sidecars give free deduplication, while the append-only log enables clean recovery through full rebuilds. But hash chains prove tamper-evidence, not authenticity—integrity is an architecture, not a checksum ritual.

While testing scenario R71, I appended a single byte X to the frame file f0001.png and re-ran the planner. I expected a hard crash, or at least a parsing error. Instead, the planner emitted a media-quarantine action. My first guess was that checksum validation had silently failed. Wrong: the planner had done exactly the right thing. After walking through the event log, I realized this kernel does not trust recorded metadata. It re-reads meta.json straight from disk at verify time. A hash recorded in a log proves nothing about a file's current content. That is the core of the pattern: state gets rebuilt from events, never trusted as-is.

Content-Addressed Sidecars and Free Dedup

Caption sidecars in the kernel live at media/captions/<src_id>/<frame_id>.<8-char-sha256>.txt. The filename embeds a hash prefix of the caption text itself, so identical text produces an identical file. That is dedup for free. Change the text and the filename changes with it. The caption event stores only {sidecar, sha256, deferred} and never the text, mirroring how Git addresses blobs: "Git is a content-addressable filesystem", in essence "a simple key-value data store" [1].

Mid-iteration, my delegate caught a real bug here. Full-state transcribe and refilter events dropped the captions{} field, so after each transcribe the caption budget looked untouched and the planner re-emitted captions that were already done. The fix was field-carry at every event site, not rewriting the log: derived states must carry the fields they do not own. The model itself only supplies raw text. The kernel wraps it with a  banner and never parses it back as structure. That is the classic defense for the case where "Injection flaws occur when an application sends untrusted data to an interpreter" [4]. An empty caption gets refused with exit code 6 and no event, and the planner simply re-emits.

Verify from Disk, Not from the Recorded Hash

At verify time the checker reads meta.json from disk instead of comparing the recorded meta_sha256. The reason is blunt: a checksum over a mutable file says nothing about that file's present content. Strip the caption_deferred marks from meta by hand and the checker still catches it, raising a hard frame-needs-caption. Tamper with a frame and the plan emits media-quarantine, and healing is boring on purpose: re-run the media step. The media-extracted status comes back, transcripts and captions ride along, the attempts counter ticks up one. Recovery works without hand-fixing because the event log is append-only. This is event sourcing's Complete Rebuild: "We can discard the application state completely and rebuild it by re-running the events from the event log" [3].

The Honest Limits of a Hash Chain

A hash chain gives tamper-evidence, not authenticity. There is no signing here, and an attacker with full write access can rebuild the whole chain from scratch. Git itself still defaults to SHA-1, with SHA-256 available as an opt-in through extensions.objectFormat=sha256 [1]. Event logs have real costs too: "Event storage, backups, and snapshots can also add additional complexity" [5]. Replay has to be idempotent, and concurrent writers need optimistic concurrency control to resolve collisions.

Small Details That Make the Difference

It is tempting to think integrity is about long hashes, but the filesystem-level details decide. Git allows exactly three blob modes: 100644 regular, 100755 executable, 120000 symlink [1]. Tight constraints like that prevent metadata anomalies that almost never announce themselves. I carried the same philosophy into the pipeline: remote-tracking state stores the last known value from fetch/push communication [2], so verification becomes comparing what is known against what is on disk. Integrity verification is not a checksum ritual. It is an architecture that makes every component prove its own case, every time, instead of trusting a system's memory.