Incident note: DNS SERVFAIL: find the cause (DNSSEC & more)

Read the signal

Start with the exact wording of the error and its timestamp in the log. Then narrow down the cause before changing anything.

;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 41234

Narrow down possible causes

• The DS record at the registry no longer matches the zone's DNSKEY after a provider change.
• Expired RRSIG signatures because automatic re-signing stopped.
• Lame delegation: listed nameservers are not authoritative for the zone or unreachable.
• Nameservers answer incorrectly or block TCP/EDNS (large responses).

Check before changing anything

1. Query with DNSSEC data

   dig example.com A +dnssec

2. Same query with checking disabled – if it works, DNSSEC is the problem

   dig example.com A +cd

3. Query the DS record at the parent zone

   dig +short DS example.com @a.gtld-servers.net

4. Query DNSKEY directly at the authoritative server

   dig +short DNSKEY example.com @ns1.example-dns.com

Fix and test again

1. Delete the wrong DS record at the registrar or replace it with the matching one; mind the DS TTL.
2. When moving a DNSSEC-signed domain, remove the DS first, wait out the TTL, then move and re-sign.
3. Check and fix automatic re-signing at the DNS provider or in BIND/PowerDNS.
4. Align NS records at the registry and in the zone.

Then repeat the original action and compare the result with the initial error.

Keep a note for the next ticket

Record the exact error, the check result, the change and the retest. That gives the team a useful starting point next time.

Why does it work for some users but not others?

Only validating resolvers (e.g. 1.1.1.1, 8.8.8.8, many ISPs) return SERVFAIL. Non-validating resolvers still hand out the answer.

Full error guide: DNS SERVFAIL: find the cause and fix it