A valid MP4 that will never transcribe
A video file can be perfectly valid and still carry no audio. A nonstate skip keeps the row visible and holds the pipeline until someone decides.
TL;DR
A supposedly broken MP4 was actually valid, just with zero audio streams. The pipeline now runs ffprobe first, stores an audio_track flag, and emits a visible skip that never marks work done. Frames follow a fixed fps cadence with capped extraction, while audio is downsampled to 16kHz mono WAV for transcription.
It looked like a broken file. An MP4 landed in my ingest queue, my player played it just fine, and yet the transcription queue had nothing to work with. My first guess: the container was corrupt, or at least half-broken.
The reality was simpler. The file was valid. The container is intact; it just carries no audio at all. Zero audio streams. The file was never the problem. My assumption that a video always comes with a sound track was.
A valid container, zero audio
The nice part: the question "does this file have audio" never has to be answered by trying and failing. ffprobe has -show_streams, which prints one STREAM section per media stream, and -select_streams a narrows the output down to audio only [1]. Empty output is the answer.
In my ingest pipeline, .mp4, .mov, and .mkv files now enter the media path. The media job runs ffprobe first, then stores an audio_track flag, true or false, in meta.json right next to the list of extracted frames.
The order matters, too. ffprobe runs before extraction so the cheapest question gets answered first: reading metadata is cheap, decoding video is expensive. The ffprobe version gets stored in the meta as well, so if its behavior ever shifts between versions, the trail is still there.
A skip that doesn't lie
Once audio_track is false, the planner emits a skip-no-audio action. I made this action deliberately NONSTATE: no "done" status gets written, the same action re-emits on every plan run, and the row stays media-extracted, still blocking the pipeline from closing. A skip that writes done is a lie. I trust a skip that keeps the row visible, so a broken file can't sneak through as finished work.
There is a second layer anyway. If some stale plan ever tells the transcription worker to transcribe a video without audio, the worker dies with exit 1 and the reason no-audio-track. Two guards for one condition, and I think that ratio is right.
A frame cadence you can count on
The video side itself I treat as a contract, not an accident. The fps filter converts video to a constant frame rate by duplicating or dropping frames, so fps=1/interval produces exactly one frame per interval second [3]. The frame_pts=1 option on the image2 muxer makes filenames carry the packet timestamp: f0000 sits at t=0, and the i-th file in sorted order sits at t = i times the interval [4]. The number reflects time, not extraction order.
Re-extraction wipes the old frames first, so a changed interval never mixes two cadences in one directory.
Extraction is capped, too. -frames:v N stops writing to the stream after N frames [2], so the extra frames are never decoded; they are only recorded as timestamped entries with a deferred flag. The audio side reuses the ordinary audio path: -vn as an input option blocks every video stream from the output, ffmpeg decodes audio only, and the result is downsampled to 16 kHz mono WAV for whisper [2].
Spending decode time on video that will never produce text is waste. More importantly, the decision has to be readable: a file like this has not failed and it is not done. It sits skipped, with a visible trail in the plan, until somebody decides what should happen to it.