DNS is critically important to the proper functioning of the Internet, and its implementation can significantly impact the performance to end users.
The Internet in general is not a great fan of packet loss, but the impact to DNS is disproportionately severe. Usually a program cannot begin loading a webpage, or perform other actions until it has had a successful DNS lookup. A web page may consist of many resources being loaded from a wide variety of domains before the page load is considered complete, leading to lots of DNS queries. Even minor delays in DNS resolution can have a large impact on page load time.
DNS has "stub" resolvers, these are DNS resolvers built into applications, and are usually fairly dumb. A "stub" resolver will talk to a "recursive" resolver, which goes and iteratively talks to "authoritative" resolvers on behalf of the stub resolver to collect an answer.
Stub resolvers usually get a list of recursive resolvers to use from DHCP. They will usually try ask them one at a time for a response, stopping when they get an answer. If they don't get a reply at all, they will generally wait for some fixed timeout then retry with the next recursive resolver.
Since DNS uses UDP, there's no feedback if a DNS query is dropped in the network, and so the stub resolver will be forced to wait for the maximum timeout, and the default timeouts are extremely large. This is because the recursive resolver may have to do a lot of work (talking to a lot of authoritative servers with many RTTs) to get an answer to return to the stub resolver.
By default Windows 8 has a DNS timeout of 1s [WIN8], but even worse [RFC1123] recommends a default timeout of 5 seconds, which many devices follow.
This means a single lost packet can cause a 1–5s delay for beginning a page load for a user, significantly higher than most other packet loss delays. To put this another way, if your users are using a 5s DNS timeout, and you lose 720 DNS packets throughout a day, you've cost those users an entire hour of waiting.
As the administrator of a recursive resolver you want to make sure that there is as close to 0 packet loss to the recursive resolver as possible, and monitor that aggressively, ideally with black box probes.
One way this can be mitigated by having a DNS proxy server in the end users network that has smarter timeout behaviours. [dnsmasq] however, appears to, by default have a 10s timeout, which is even worse! Although, dnsmasq will retransmit if the client has timed out and retransmitted.
A common feature of CDNs is to have multiple locations around the world that provide their content, and then use different DNS replies to steer traffic to the best location for the users that are asking for content.
For example, a CDN might have a server farm on the east coast of the US, and another replica on the west coast. DNS queries to find that content will get a response of the east coast replica if the user is closer to the east coast, and a response from the west coast if the user is closer to the west coast.
This hypothetical CDN can be thwarted if users on both coasts use the same recursive DNS server. The CDN cannot tell which coast the user that generated the DNS query is from, and therefore cannot decide if it's better to return the east or west coast. And, once the CDN has replied the result will be cached and handed to users on either coast.
A solution to this problem is RFC7871: EDNS0 Client Subnet, where the recursive DNS server includes the client network in the request, so the CDN's authoritative DNS server can make an informed decision as to which replica to reply with, and the recursive resolver keeps the two cache entries separate.
EDNS0 Client Subnet recommends aggregating users by /24 for IPv4 and /56 for IPv6, so make sure that you don't have two wildly different client populations in prefixes longer than that, or they will still get mixed together.
One disadvantage of EDNS0 Client Subnet, is that it ends up partitioning your cache, leaving you with the same name in the cache multiple times (in our example, there needs to be records for the people on the East coast, and different records for people in the west coast). By having an "east" coast recursive DNS server and a west coast DNS server, the problems with this are somewhat alleviated.
Thus it is also usually wise to have different DNS servers close to different subsets of users. Not all CDNs support EDNS0 Client Subnet, or may not have information for the subnet and may fall back on using the IP address of the recursive server. Also, a server close to your users can respond quicker than a server that is far away, making the Internet feel substantially faster. This recursive server can use the same anycast address as other servers, so clients may only see one IP address, but the "true" IP address that DNS queries come from as seen by the authoritative server should be different.
If the machines behind the recursive DNS server are using NAT, then EDNS0 Client Subnet only has internal addresses (e.g, [RFC6890] addresses) to use, then the authoritative server should ignore the addresses (as recommended by Section 11.3 of [RFC7871]). In this case, the authoritative DNS server can only use the source address of the DNS query (i.e the recursive DNS server's egress IP). Needless to say this means that you should only have one client population behind a single NAT gateway.
Different DNS servers should not come from the same network prefix. CDNs commonly treat DNS servers in the same subnet as having client populations that are the same.
By making sure that CDNs can determine where best to send a client, you will reduce latency for your end users, and will allow network traffic to flow over shorter, usually cheaper networking links such as direct peering, or Internet exchanges.
One of the most important properties of a recursive DNS server, is it's cache. The cache allows the server to answer immediately, without having to consult authoritative DNS servers.
Closely monitoring your cache hit rate is very important to the overall performance of your users.
A massive proportion of DNS queries are pure junk. This is often causes by programs trying to resolve things that look like they might be a DNS name, to see if it is or not. This leads to huge numbers of queries for useless names such as "untitled.pdf". Resolving these names is a waste of your time, and the authoritative nameservers resources. To mitigate this, you should use RFC8198: Aggressive Use of DNSSEC-Validated Cache.
RFC8198 works by using DNSSEC's NSEC and NSEC3 records to prove that there is no valid name within a range. Thus the recursive DNS server doesn't need to go and ask the authoritative name server if the name exists or not – it can immediately create a NXDOMAIN from the information from the NSEC or NSEC3 records in it's cache and respond. This can have a massive impact on DNS performance, and reduces the load on authoritative name servers, especially the root name servers.
In the example above, a lookup for "untitled.PDF" would find the NSEC records in the root zone saying there are no names between ".pccw" and ".pe", and therefore there is no valid domain that ends in ".pdf" and can immediately return NXDOMAIN.
With RFC8198 and a busy client population, your recursive DNS server will quickly load the entire root zone into it's cache, avoiding many lookups.
RFC8198 also means you don't waste cache space storing lots of spurious NXDOMAINs, but instead cache the fewer, more useful NSEC (or NSEC3) records.
[RFC8020] "NXDOMAIN: There really is nothing underneath" is similar solution with similar improvements, that does not rely on the zones being DNSSEC signed. Since most of the improvement happens in the root zone (and, to some extent, the second level domains, which generally are also signed), I suspect it's use when combined with RFC8198 is limited unless you also run a large set of authoritative name servers that are not signed. However, if the feature is available then, by all means turn it on and get even more value out of your cache.
Both of these techniques also help clients that are trying a variety of names (such as trying to qualify a name using a search path) to find the actual name they are looking for.
dnsmask RFC1123 RFC6890 RFC7129 RFC7871 RFC8020 RFC8198 WIN8
index