ISP DNS Poisoning Breaks Automated Web Research
Months spent blaming an anti-bot; the culprit was an Indonesian ISP's DNS redirecting duckduckgo.com to Trustpositif.
TL;DR
Headless browser di Docker gagal membuka DuckDuckGo, dan penulis berbulan-bulan mengira penyebabnya anti-bot dan fingerprinting yang ekstrem. Ternyata cek nslookup sederhana membuktikan DNS ISP memalsukan jawaban resolusi, efek kebijakan Kominfo yang membelokkan port 53. Solusinya cuma satu flag: paksa container memakai DNS publik seperti 1.1.1.1.
Automated (headless) browser logs inside a Docker container kept failing to open DuckDuckGo. The errors were weird: sometimes a timeout, sometimes a redirect to a page that was clearly not the DDG interface. Meanwhile, on a normal laptop on the same network, DuckDuckGo opened smoothly.
The sentence that haunted me for a long time: why is this anti-bot world-class and this extreme? How brutal is their fingerprinting?
For months I went down the wrong path. Busy tweaking fingerprint configurations, manipulating cookies, even trying various elaborate bypass techniques. The culprit, it turned out, was not DuckDuckGo at all.
The nslookup moment that unraveled everything
The turning point came when I stopped and checked the most fundamental layer: domain name resolution. The command is simple, run inside the troubled container:
nslookup duckduckgo.comThe result: 103.123.248.32.
I checked again. That IP does not belong to DuckDuckGo. On my server, gethostbyname even reversed the domain name into trustpositif.moratelindo.io: an IP block owned by a local ISP known for its TrustPositif blocking mechanism. The container inherited the default DNS configuration from the host network, and that DNS casually manipulated the resolution answer. Every elaborate theory about anti-bots and fingerprinting collapsed instantly. I was not blocked by DuckDuckGo; I was "poisoned" by my own ISP's DNS.
Why this happens in Indonesia
This problem is no coincidence. History records that "On May 31, 2015, Kominfo mandated all home and cellular ISPs operating in Indonesia to redirect port 53 for censorship purposes", according to Wikipedia's entry on Internet censorship in Indonesia. Since then, port 53 DNS queries at many ISPs have been redirected to ISP servers, and some ISPs even block encrypted DNS domains. This can show up anywhere: office networks, certain mobile providers, or devices whose inherited configuration nobody has touched.
To prove it, compare the diagnosis on an affected network against a public resolver:
# Inside the container (inheriting ISP DNS)
nslookup duckduckgo.com
# answer: 103.123.248.32 (not DDG)
# Comparison via a public resolver
dig +short @1.1.1.1 duckduckgo.com
# answer: 20.43.161.105 (DDG's real IP)The fix? One line. When running the container, force trusted public resolvers, for example 1.1.1.1 from Cloudflare:
docker run --dns 1.1.1.1 --dns 8.8.8.8 --dns 9.9.9.9 -d nama-image-risetAfter that, DuckDuckGo opened normally, without changing a single line of fingerprint or cookie code.
Bonus lesson: because any single search engine can act up at any time, my automated research now uses a rotation: DDG, Bing, Yandex, Google, plus a DNS fallback whenever resolution looks weird again.
Cheap diagnosis always comes first
DNS over HTTPS (DoH) was created precisely so that DNS queries are not vulnerable to "eavesdropping and manipulation ... by man-in-the-middle attacks", as Wikipedia's DoH entry puts it. That principle describes this problem exactly: manipulation at the middle layer by the party that is supposed to be the guide.
The biggest lesson from this incident: before blaming anti-bots, fingerprinting, or high-level network complexity, make sure the lowest layer (DNS) is healthy. Do not spend months on a problem that is actually solved with one --dns flag.
The closing move is simple: every container on my research server now explicitly uses public DNS. No more default DNS inheritance that could "poison" research results.