Technical notes on web development, DevOps, and AI integration.
1 article
Why gated data harvests save verbatim raw text before parsing: an idempotent ledger, a resumable cursor, and retroactive rebuilds.
TL;DR: When scraping captcha-gated, session-locked court portals, save the verbatim rendered text first and parse later from local files. Resumable cursors and idempotent upserts keep crashes from corrupting data. This bronze-layer approach means parser fixes apply retroactively without ever hitting the fragile site again.