Technical notes on web development, DevOps, and AI integration.
1 article
Two-layer API key rotation, provider fallback, and DoH for lying ISP DNS: lessons from building a research script that refuses to die quietly.
TL;DR: Firecrawl's keyless search endpoint died with permanent 403s mid-research, so the author pivoted to keyed APIs. The result is web-direct.py, 125 lines doing search, scrape, and extract with two-layer rotation: next key first, then next provider. A DoH health check catches lying ISP DNS, falling back to curl --resolve.