October 10th, 2019 There's been a lot of talk about DNS-over-HTTPS aka DoH lately, primarily due to Mozilla's and Google's respective plans to move forward with enabling it in their browsers. There's also a lot of misunderstanding and conflation of speculated plans for world domination with practical benefits of the technology; it seemed useful to me to clarify at least to myself the benefits and drawbacks from a more neutral point of view and in comparison to some other approaches to securing the DNS. The DNS is the famously insecure backbone of most things internet. The cause of endless wasted hours troubleshooting bizarre and inconsistent behavior only to find out somebody monkeyd around with /etc/hosts. The DNS is the thing we all rely on, yet which uses plain-text UDP with no assurance of authenticity or integrity of the data. It's really pretty wild, when you think about it. Anyway, so looking at how to secure the DNS, we need to consider the (modified) CIA triad: Confidentiality, Integrity, and Authenticity. But DNS traffic also exposes the user to other threats and impacts their privacy, so it's worth considering the use cases and actors (with their respective capabilities) for a few different scenarios. That is, let's compile an informal threat model: ActorsFor our discussion, we'll consider the following actors:
DNS Servers and ServicesThe DNS consists of numerous components, and different threats apply differently to each. We distinguish:
In this article, we'll focus primarily on the interactions between the stub resolver and the recursive resolver. ThreatsBefore we look at the (well-known) flow of information, let's consider the different threats to the properties identified above:
In other words, using the well-known information flow of a regular DNS query from the browser to your ISP's resolver, we are looking at an illustrated threat model (for the DNS related components of the traffic exchange) like the one below: ![]() DNSSECAlright, so this looks pretty awful. "How can we have gone on like this for so long?", you ask? Well, thing is... we haven't, really. We've had DNSSEC (discussed here previously) for a while -- RFC2535 dates back to 1999! But DNSSEC doesn't protect against all of the threats noted above. Specifically, DNSSEC only addresses the question of data integrity and authenticity, but does not in any way concern itself with the aspects of confidentiality. (What's more, data privacy within the larger discussion of what information service providers ought to (be able to) collect is left entirely out of many information security discussions.) In other words, DNSSEC defeats Mallory, forces Grace to the authoritative name server of the service they are interested in rather than allowing it on the (likely easier to control and significantly more cost-effective control point) resolver, but does absolutely nothing to combat Eve: ![]() Finally, we should note that DNSSEC also covers a specific (yet tricky) edge case: DNSSEC can prove that a given record does not exist by using something called "Authenticated Denial of Existence in the DNS" (RFC7129). This is noteworthy as tricking a client to believe a domain doesn't resolve (NXDOMAIN) can be used for censorship, but on the other hand is also frequently used by e.g., enterprises to blackhole known malware domains and the like. (See e.g., the use of DNS Response Policy Zones (RPZ); DNSSEC doesn't break this: enforcing clients will still not get results, but they will know that the answer from their resolver was not accurate.) Support for DNSSECSupport for DNSSEC is widely available in the common operating systems, DNS resolver software, and stub resolvers. Mobile operating systems support DNSSEC, but require the application developer to explicitly enable/require it (e.g., in Apple's dnssd framework)... which they, by and large, do not (see e.g., Chrome). DNSSEC by exampleI've previously shown various examples of DNSSEC in practice, so let's just quickly illustrate the authenticated data bit and the RRSIG records: $ dig www.netmeister.org +dnssec +noall +comments +answer [...] ;; flags: qr rd ra ad; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1 ;; ANSWER SECTION: www.netmeister.org. 10613 IN CNAME panix.netmeister.org. www.netmeister.org. 10613 IN RRSIG CNAME 13 3 10800 20191017000000 20190926000000 31910 netmeister.org. ohYDH8Nd0/z60thHr4uUXMBmulU4TtlGkUhUPiKFQ0jSYqSMK6bsp6Tu cvt3iV8vHuScGGGq1cwc1atHPQz5dg== panix.netmeister.org. 10613 IN A 166.84.7.99 panix.netmeister.org. 10613 IN RRSIG A 13 3 10800 20191017000000 20190926000000 31910 netmeister.org. xtqLs3rQuzcbI97rDjhrcMfpYAmN2GFt3gADihP2TWCv6C7h5Hb0LlHb jm7rb5VptAkbZuJCNEG7ebqP91pvww== $ DNS-over-TLS aka DoTOk, so what do we normally do when we want to imbue a product with "security"? Why, we rub some crypto on it! Sprinkle some of that magical crypto pixie dust all over it. Easy! See what we did with HTTP? We just added another layer, wrapped the whole thing in TLS, et voilĂ , problem solved! Let's do the same thing for DNS traffic! Enter DNS over TLS aka DoT (RFC7858 and RFC8310). Here, we are switching from UDP port 53 to TCP port 853, establish a TLS connection, and then exchange our regular DNS traffic over the encrypted channel. DNS traffic is kind of important. If things go wrong with the DNS, all sorts of things stop working. Now remember the various headaches involved in managing TLS certificates, and think about how poorly the end-user experience is going to be when their ISP fails to renew the cert for their DNS resolvers. Add to that the fact that we're using a new port, meaning clients will need to be updated. To accommodate both old-fashioned DNS over UDP (DoU?) as well as DoT, providers are likely to offer both, and a client that fails DoT (for example, due to an expired cert on the DNS resolver) is likely to retry using DoU. For this reason, DoT offers two different "Usage Profiles": an opportunistic privacy profile and a out-of-band key-pinned privacy profile, described in detail in RFC8310. Think of the way mail servers mostly use opportunistic encryption and verification: if STARTTLS is not stripped by the attacker, servers generally accept any certificate, as the alternatives are to fail the connection and not deliver mail, or to try again without TLS. (See RFC8461 for a description of SMTP MTA Strict Transport Security (MTA-STS), supported by gmail since April 2019.) The out-of-band key-pinned privacy profile profile requires the client to receive, via a previously trusted connection, pins of the leaf cert used by the resolver(s) in question. This is practical in environments where the DNS service provider also controls the client environment, such as e.g., in enterprise environments. Elsewhere, this is going to be challenging, and I expect this profile to be the minority of deployments. Now the opportunistic privacy profile still allows for validation of the server certificate, and clients can be configured to trust exactly one, a small set, or the whole regular list of public CAs. I'd wager that most deployments will reuse the same trust bundle already used by their OS or browser, bringing with it the entire litany of cert bundle issues we know so well. This also notably expands the responsibility of CAs: most of the rules and regulations are decided in the CA/B forum, which is necessarily browser focused. Adding the DNS to the list of clients ought to shift this focus, but I don't know whether this is being paid sufficient attention to. Given the different privacy profiles, the protections offered by DoT depend largely on how it is configured. In addition, DoT is not necessarily used between all resolvers and authoritative servers, so the protections provided only apply explicitly to the traffic between the client and the recursive resolver: ![]() The more complete threat model for the different privacy profiles looks like this table from RFC8310: +---------------+------------+------------------+-----------------+ | Usage Profile | Connection | Passive Attacker | Active Attacker | +---------------+------------+------------------+-----------------+ | Strict | A, E | P | P | | Opportunistic | A, E | P | P | | Opportunistic | E | P | N, D | | Opportunistic | | N, D | N, D | +---------------+------------+------------------+-----------------+ P == Protection; N == No protection; D == Detection is possible; A == Authenticated connection; E == Encrypted connection Note also that the authenticity provided by DoT only refers to the server, not the records! While records cannot be tampered with between the resolver and the client, the client has no assurance that the resolver provided the correct answer, or that Mallory or Grace did not manipulate it anywhere along the path or on the systems. Lastly, in order to speak DoT, the client will need to be updated. See this matrix for support by products. One problem here is that if the base OS stub resolver (i.e., libresolv) doesn't speak DoT, then individual applications may choose to implement DoT themselves. This yields a scenario where one application might resolve a query in a different way than another, which increases the troubleshooting burden. Just remember how many hours you've wasted troubleshooting different applications using different CA trust bundles, where those are set, what formats they use, etc. to get an idea where a bifurcated OS/application resolver may take you. Support for DoTDoT support is available in several public DNS resolvers. DNS server software varies in its support (BIND requires the use of stunnel, Unbound supports DoT out of the box), Android now automatically upgrades to DoT if the server supports it, while iOS only lets application developers implement a DNS proxy provider. For regular iOS users, there is the option of installing such an app (e.g., Cloudflare's 1.1.1.1 app). systemd supports DoT, but on most other systems you will need to install a separate stub resolver with support for DoT (e.g., Stubby or knot resolver). DoT by exampleTo simulate and inspect a sample DoT query, you can create a simple stunnel between 127.0.0.1:53 and a public resolver that supports DoT, and then use your usual DNS tools to submit the query: $ cat >/tmp/dot.conf <<EOF output = /tmp/log debug = debug [dot] client = yes accept = 127.0.0.1:53 connect = dns.quad9.net:853 EOF $ sudo stunnel /tmp/dot.conf $ dig +tcp +short @127.0.0.1 www.yahoo.com atsv2-fp-shed.wg1.b.yahoo.com. 72.30.35.9 72.30.35.10 $ As you can see, we are using the same old DNS protocol, trivially wrapped in TLS. Quite literally DNS over TLS. If you tcpdump the relevant traffic, you won't be surprised to see the query and response nicely encrypted with all the usual TLS handshake and session data: ![]() DNS-over-HTTPS aka DoHOk, so now on to DNS-over-HTTPS aka DoH, specified in RFC8484. Since HTTPS uses TLS, you could argue that technically DoH is "DNS over TLS", too, but this is misleading at best: DoT speaks the regular DNS protocol over a TLS connection on a distinct and dedicated port, while DoH uses the HTTP application layer protocol to send queries to a specific HTTP endpoint on the resolver's well-known HTTPS port, including and using all the semantics of HTTP communications. Note that per the RFC, DoH was specifically designed as a solution for web applications (i.e., browsers), not for generic stub resolvers or libraries. In addition, as the recent discussions have shown, DoH is also focusing on a different threat model whereby the operator of the default resolver or even the network are not trusted, but we're getting ahead of ourselves. Let's first look at the threat model for your web browser using your local resolver via DoH: ![]() In other words, we gain the same protections as with DoT for our web applications, but leave all other DNS traffic vulnerable. But it appears that at least the browsers' plans for adoption of DoH add another layer: the selection of a Public DNS Server. We'll discuss that part of the puzzle in some detail below. Support for DoHSupport for DoH is almost entirely application specific and driven by the two most popular browsers as noted above, albeit with a higher adoption than DoT (for example, curl supports DoH since version 7.62.0). Mobile clients again have the option of installing a separate app (e.g., Cloudflare's 1.1.1.1, or DNSCloak); for your desktop / laptop OS, you'll again need a separate stub resolver (knot resolver or dnscrypt-proxy). DoH by exampleAn example DoH query might look like so: $ curl -L -s -H 'Accept: application/dns-message' \ 'https://9.9.9.9:5053/dns-query?dns=q74BAAABAAAAAAAABXlhaG9vA2NvbQAAHAAB' This isn't very easy to use, however, since you need to create a DNS wire format packet, then base64url encode it, and of course you then get back a response in DNS wire format. Several public resolvers offer an option to make a much simpler HTTP request and get back results in JSON if you submit a Accept: application/dns-json header: $ curl -L -s -H 'Accept: application/dns-json' \ 'https://9.9.9.9:5053/dns-query?name=yahoo.com&type=AAAA' | jq { "Status": 0, "TC": false, "RD": true, "RA": true, "AD": false, "CD": false, "Question": [ { "name": "yahoo.com.", "type": 28 } ], "Answer": [ { "name": "yahoo.com.", "type": 28, "TTL": 1246, "Expires": "Wed, 09 Oct 2019 16:24:42 UTC", "data": "2001:4998:58:1836::10" }, { "name": "yahoo.com.", "type": 28, "TTL": 1246, "Expires": "Wed, 09 Oct 2019 16:24:42 UTC", "data": "2001:4998:44:41d::4" }, { "name": "yahoo.com.", "type": 28, "TTL": 1246, "Expires": "Wed, 09 Oct 2019 16:24:42 UTC", "data": "2001:4998:58:1836::11" }, { "name": "yahoo.com.", "type": 28, "TTL": 1246, "Expires": "Wed, 09 Oct 2019 16:24:42 UTC", "data": "2001:4998:c:1023::4" }, { "name": "yahoo.com.", "type": 28, "TTL": 1246, "Expires": "Wed, 09 Oct 2019 16:24:42 UTC", "data": "2001:4998:c:1023::5" }, { "name": "yahoo.com.", "type": 28, "TTL": 1246, "Expires": "Wed, 09 Oct 2019 16:24:42 UTC", "data": "2001:4998:44:41d::3" } ] } There, that's much easier to handle when all you want to do is play around with a few public resolvers. DoH with Public ResolversSelecting a public resolver instead of your network provider assigned resolver may be done for a variety of reasons, including the fact that your local resolver may not even support DoH or that you are looking to protect your privacy from the provider of the local resolver. ISPs have long had a history of monetizing DNS lookups, and are of course likely to utilize the wealth of information derived from their users' DNS queries for a number of purposes; traffic from residential networks to ports outside of the expected and approved set (by and large: 80, 443) may be blocked, thus preventing a client from switching to a Public DNS server using DoT. Censorship performed by those (directly or indirectly) in charge of the resolvers may even prevent DNS resolution of certain names altogether; China's Great Firewall is known to at least in part use DNS hijacking, injection, poisoning, and exerting control over ISP resolvers. From a browser's perspective, it may thus be advantageous to circumvent the local resolver. By stuffing the DNS resolution into regular HTTPS traffic, you're conveniently also utilizing a port and protocol that in most environments is much less restricted than the others. The properties of TLS grant you confidentiality and integrity protections, the use of HTTP as the application layer protocol lets your client use the same API and even improve efficiency, and by using a public resolver, you free yourself from several restrictions possibly imposed on you by your network provider: ![]() Alright, so that seems generally useful, and a point can well be made that ISPs equating Mozilla with Trump as "Internet Villain of the Year" over the use of DoH with a public resolver only validates its necessity. But of course there is indeed no such thing as a free lunch, and the use of public resolvers involves trade-offs... DNS CentralizationWhile there are many public resolvers, there are very few that the various discussions surrounding DoH include. For the most part, it comes down to Google's and Cloudflare's services. What's more, Mozilla made the decision to change the default for their Trusted Recursive Resolver settings to use Cloudflare. This is notably different from Google's plans for Chrome, which are much less controversial if you bothered to read them: Chrome 78 will check if the user's current DNS provider is among a list of DoH-compatible providers, and upgrade to the equivalent DoH service from the same provider. If the DNS provider isn't in the list, Chrome will continue to operate as it does today. The short list consists of Cleanbrowsing (a public resolver with content filtering), Cloudflare, DNS.SB, Google DNS, OpenDNS (owned by Cisco; unclear whether the service used by Chrome utilizes OpenDNS's phishing protection and content filtering), and Quad9 (a non-profit). If your resolver supports DoH, but is not in this list, then Chrome will not automatically select it and instead perform plain old UDP 53 DNS lookups. (I would expect there to be a setting for you to choose your own DoH service, but I don't know if this will be provided.) So for a reasonably common setup, where the host is configured to use Google's DNS, our threat model now looks like so: ![]() In this model, the widely anticipated use of DoH defeats Grace and Colleen, but only at the ISP level. DoH defeats Eve and Mallory, but only on the web application. At the same time, though, DoH increases the complexity on the client with at least two different resolvers and two different protocols in play, leading to possibly confusing results. Finally, the use of a public DNS server if encouraged (in the case of Google Chrome) or enforced (practically, in the case of Mozilla Firefox) on a browser with a significant market share necessarily increases the capabilities of both Grace and Colleen on the public resolver: ![]() If a -- any -- public resolver is made the default or in any other way favored by widely used browsers, then a significant chunk of private data is going to be delivered to this service provider. That will necessarily increase the target on the service provider's back as well as invite Colleen operating at the provider to (ab)use the information now present. It stands to reason that for most threat actors, a large company the scale and reputation of Google or Cloudflare stands a much better chance at defeating (technically or legally) them. An organization like Mozilla, with a track record and official objective of acting in the interest of the users and the health of the internet should also not be dismissed as intentionally enabling mass surveillance. For example, Mozilla assures users that it will only allow public resolvers in it's "Trusted Recursive Resolver" program if they meet strict privacy and transparency requirements. That policy reads well enough, including explicit support of e.g., QNAME Minimization and demanding the ECS extension to be disabled unless communications between the public resolver and the authoritative servers is encrypted. However, at the end of the day that's a policy, not a technical enforcement. Grace will indeed likely seek to utilize this service for mass surveillance, Colleen will inevitably find (possibly well-intentioned) policy loopholes, and changes to the terms of service post facto will be increasingly difficult to rectify once critical mass has been achieved. I view the centralization of the DNS as a major point of concern. This blog post by Geoff Huston at APNIC covers this concern in much more detail; I'm curious to see how this data is going to change with the rollout of the latest browser versions. DoH Implementation ConsiderationsDoH is going to come. It may or may not be here to stay, but it sure has a lot of momentum right now. So it's worth considering what the current plans by the browser providers mean for end-users, network operators, and enterprises. Let's look at each use case: DoH for End UsersMost users will simply not care one way or another. I know, it's difficult for us inside the bubble to grasp this, but DNS resolution is not on anybody's attention horizon. Those (comparatively) few people who do care, will find themselves in a conundrum: if you worry about your network provider monetizing or monitoring your DNS traffic, you have the problem that even with DoH to a public resolver your network provider can still see what sites you're connecting to -- they're controlling the network! By and large, they see what websites you're connecting to. (The overall number of websites relying on encrypted SNI via TLS 1.3 in a multi-hosted environment is statistically insignificant.) This, to some degree, negates the privacy benefits of using a public resolver, while at the same time providing to the public resolver -- with whom you have no business relationship, and who can change their terms of service at any time -- your private data. Will this make a difference for you? I don't know. I doubt it. If Grace interfering with your DNS lookups is within your threat model, the use of a public resolver via DoH may help you, as it raises the cost of preventing you form accessing a given service somewhat, but if you want privacy from your network service provider, you might use a VPN (thereby granting the VPN provider the same visibility previously granted to the network service provider) or use e.g., Tor. Either way, you should consider that switching to a public resolver also means that any protections previously offered by your network provider, such as malware protection, blackholing of phishing domains, parental content filters etc. are no longer available to you. Trade-offs, as always. DoH for Network Operators / ISPsIf you're a network operator, you will have to decide whether or not you want to support DoH to public resolvers. One thing you can (and should) do is to offer your users various options to secure DNS traffic: enable DNSSEC, offer DoT and DoH on your resolvers, provide instructions for your users how to configure their systems, and have support staff be able to diagnose, troubleshoot, and address the resulting problems. Haha, I kid, I kid. ISPs are not going to do any of that, are they? Well, would be nice if they did. A man can dream, a man can dream... (To be fair, some ISPs do indeed look into providing DoT and/or DoH.) Now if you do not want to support DoH to public
resolvers, you have a few options. Mozilla has built
a Now this seems a bit weird: DoH to a public resolver seemed intended to defeat an untrusted local resolver, but an untrusted local resolver can disable the use of DoH to the public resolver in this fashion. In fact, if you have content filtering in place on your DNS resolver, Firefox will also use that as a metric to decide whether or not to enable DoH to the default public resolver. In other words: if the browser detects that one of the things that DoH would help protect against (censorship / content filtering) is in place, then it will not enable DoH. ?? Well, for you as the network provider, that's good news -- you remain in charge, as so often, it boils down to having to trust the things you have to trust -- if you (the end user) don't trust your network provider, you can't use their services to bootstrap a trusted environment. A more complete discussion of the considerations for operator networks can be found in this IETF draft. DoH for EnterprisesEnterprises are, in a way, a special case of network service provider. In an enterprise environment, it is often desirable to control significant aspects of the DNS: you want to be able to blackhole malware domains, quickly disable unique and targeted phishing attacks, allow users to resolve domains that must be internally visible only, or you may want to monitor and analyze DNS queries for possible attacks or exfiltration processes. In short, you really want to be in control of the DNS. For all of these reasons, enterprises want to explicitly disable DoH to public resolvers, and adding use-application-dns.net to your RPZ to return NXDOMAIN is a trivial first step you should take. However, please do note that this will only disable DoH in Firefox if enabled by default, not if it was enabled by the user explicitly, nor for e.g. Chrome or any other application. Since enterprises generally also have some level of control over the endpoints, you likely also want to control the different browser settings to disable the use of public resolvers. However, just like for network operators, it is in your interest to actually offer the benefits provided by DoH (and DoT), so now would probably be a good time to look into adding DoH and DoT support to your organization's DNS servers. DNS Security SummaryOk, so that was a lot of words. Let's recap:
So, what do we do now? From my perspective, I believe that we should work on pushing for wider adoption of DNSSEC, because I continue to come back to the core problem of DNS results not being authenticated or verified. This strikes me as a significantly larger problem than the privacy implications surrounding our collective lookups. (I have not yet had a chance to investigate DNSCurve and DNSCrypt; perhaps another time...) However, the confidentiality of the DNS lookups is not something I dismiss, either. For that, my current preference is quite clearly DNS-over-TLS: I fear a bifurcation of DNS resolution by apps combined with the push for using public resolvers with DoH will lead to a more complex environment and threat model for many users. Either way, though, DoH is clearly coming. Most enterprises and operators would likely benefit from a better understanding of DoH and especially the impact of centralizing the DNS in a handful of public resolvers. This blog post is the outcome of my own pursuit in this direction; I hope it was of some use to you as well. October 10th, 2019 Links and additional reading:
|