Sunday, 29 November 2015

Logging DNS lookups

One of the interesting questions in relation to the Draft Investigatory Powers Bill is whether it would allow a retention order to require an ISP to log DNS lookups.

What is a DNS lookup?

The Domain Name System is a key part of the Internet - its primary use being to convert the names you use on web sites (like to the addresses used within the protocol itself (e.g. 2001:8b0:0:30::51bb:1e51).

It is actually a pretty good distributed database system, and can hold more than simply name to IP address lookups. It can do reverse lookups (IP to name), and hold text records and mail server records, and a number of other record types.

Why would you want to log DNS?

Well, the government have made it clear that they would like to see the web site names people access. Usually, when accessing a web site, before you access it you have to convert the name to an IP address, and hence to a DNS lookup. Trying to extract the name of the web site from the web site access itself it a lot harder than just logging the DNS lookup.

How easy is it to log DNS lookups?

Mostly the ISP runs DNS servers for their customers, and such servers could produce logs. To be honest, that would mean beefing up the servers, as they typically are not logging (it would be a lot of logs). Also it would mean finding a good way to store and search the logs, but it is possible.

What gets a tad more complex is when people do not use the ISPs DNS servers. Normally this is a simple thing to do, and some people use googles or OpenDNS which can provide some parental control filtering. There are ISPs that do not run DNS at all themselves and subcontract it.

However, DNS packets are not encrypted, and are always on the same port, so it is technically possible to log the requests as they go past. This is a headache to do - you cannot easily divert these packets or copy them on a normal router - you have to look at a switch mirror port of all traffic and filter out the DNS packets. The only good news is that you probably do not have to do session tracking, simply catching the DNS replies would allow you to see the (apparent) requester and the answer they got. You'd also get all DNS reflection attack traffic.

Of course, it is easy to see how protocols could advance to allow encrypted DNS lookups, and I am sure that will come.

Why would people not use their ISPs DNS resolvers?

There are lots of reasons, but one of the reasons that is increasing a lot is because bypassing ISP DNS resolvers can bypass the ISPs ability to block access to some web sites in some cases. It is somewhat ironic that the governments moves to try and ban porn, copyright infringement, and extreme content are making the public at large much more tech-savvy in ways to bypass the controls of the ISPs, and hence also logging.

Should DNS lookup logging be allowed?

This is where it gets tricky! In the telephony world a call to Directory Enquiries is essentially the same function as a DNS lookup - however telcos are not expected to record, listen to, and log the content of that call any more than they can log the content of any other call. So it seems obvious that DNS requests should not be logged.

Will the bill allow DNS lookups to be logged?

The bill tries to define content and meta data (communications data) - which is a complex task. In principle, an "identifier" or data about a communications address is considered meta data and so could be logged. On that basis, maybe they could ask to log the content of these DNS lookups.

The problem is that DNS can be used for more than just a name/IP lookup. Only some types of DNS request will come within that somewhat loose definition of communications data. Any other type of lookup would be "content" which the ISP must definitely not be logging and retaining.

Even more complex is that you do not know for sure that a name/IP lookup is actually be used to look up a protocol address. The IP address returned could be used to signal something else - one common usage is a blacklist lookup. This is using DNS as a database query system, and the reply indicates a yes or no, and not actually an IP address, even though it looks like an IP address is returned.

Ultimately the ISP has no way to know for sure what purpose exists for the DNS lookup - it is simply a database. With that in mind, and with a ban on logging the "content", I do not think any ISP could legally log the content of DNS lookups under a retention order.

How would we know?

One huge problem here is that if this is not clear in the bill, once passed an ISP could be asked to log DNS requests. If they don't appeal that, then end up making and retaining such logs. If that is in fact not allowed (and presumably even one logged request which was not a protocol "identifier" would make it illegal) then that could cause problems. The issue here is nobody knows - the retention order is secret.

Indeed this is a more general issue with the secrecy - the definitions are not crystal clear and if the government decide something is in scope of "communications data" they could include it in a retention order and simply get away with it. One level example was the idea of grabbing from emails the details of calendar events. These seem obvious that they are "content" except the define the time of an event, and that is something that is defined as "communications data" in the bill. The fact that it was within the "content" part of an email may not matter. This is yet another reason that retention orders must not be secret.

What did the Home Office say?

They seemed unsure. As per my written evidence I think this needs spelling out in the bill that DNS lookups must not be logged.


  1. Here's a fun idea though - if your ISP decides they're not going to bother logging outbound requests, only inbound replies, then you've instantly got a real easy way to stitch someone up. Just throw them lots of faked DNS replies from some DNS server containing replies to queries they didn't make. Hey presto - you can make it look like anyone's requested anything. It's not even the same as a DNS poisoning attack, as you probably wouldn't be lying in the response, you're just lying in the sense that you're answering a question nobody asked.

    1. Oh, I had already worked that one out, don't worry :-)

  2. If the government allowed me to improve just a single aspect of this nefarious bill it would have to be to remove the secrecy applied to retention orders. Even if they eventually sharpen up the ambiguous definitions in the bill they'll ensure there's still sufficient leeway to enable themselves, or any future government, to demand anything they want in secret retention orders. I can see the main opposition being the handful of very large ISPs, though. If they already think that the smaller ISPs such as AAISP are likely to be left alone, as indicated at your Home Office meeting, they may see this as unfair competition were it to become public knowledge that only the large ISPs were tracking their customers for Uncle Dave.

  3. That information is also more or less useless for determining access patterns. My local DNS will lookup, then cache it, and use the cache for as long as it wants (up to max TTL, assuming it's obeying that).

    Furthermore I may simply have visited a page with a link to on it and my browser could be precaching it.

    That means you can tell only that I made a dns request for once in the past several hours. You can't tell how often I did so, and you can't tell whether I subsequently ever visited the site. I can't see how that information is at all useful to an investigation.

    1. It is hard to see have any of the information they want kept is that useful or at all definitive.

  4. The best bit is using a combination of non ISP DNS and HTTPS to circumvent ISP blocking of sites...

  5. Presumably if anyone runs a full bind installation at home as a DNS server, there will be nothing meaningful an upstream server can log?

    1. The upstream ISP caching resolvers will see nothing but packet snooping would see all the requests - only much less frequently as they are locally cached.

    2. Alternatively, you can run something like DNSCrypt to encrypt all your DNS queries, which gets around the packet snooping entirely.

    3. I've been running DNSCrypt on my home router for about a year now - after BT broke my internet by trying to force an interstersial page on opting in to their parental controls feature! f that. Get off my DNS requests.

    4. Equally, anyone using a DNSCurve compatible client will be both sending and receiving encrypted DNS traffic, tunnelled via standard DNS queries and responses - so even if I were routing through A&A's DNS servers, and if you were logging that traffic, you'd still be getting partly meaningless ciphertext in the logs! (Thus violating the rules, since that ciphertext doesn't seem to be permitted traffic to capture...)

      Even without encryption, tunnelling alone - like your own L2TP tunnels, or HE's free IPv6 tunnels for victims of legacy ISPs - would defeat simple port 53 logging too.

      It's a shame (IMO) that DNSSEC missed this out; after the Snowden revelations I like to think privacy will be a higher priority and be incorporated in the next protocol somehow.

      They really, really, haven't thought this through at all, have they?!

  6. When I'm connected to the office VPN all my DNS lookups are served by the office DNS servers.Obviously no intention to 'hide' such traffic but that is the result. Would this mean that companies will have to keep records of all their employees activity as well?

    1. The bill, as worded, could allow a retention order on anyone providing communications, so could over an office network or even a home network, yes!

  7. You may be interested in RFC 7626 "DNS privacy considerations"

    The IETF working groups dprive and dnsop, on the basis of this RFC, are working to improve privacy of the DNS. Encryption is just one part of it (it does not protect you against spying by the authoritative name servers).

    dprive works on encryption (DNS over TLS, currently implemented in several programs, RFC under way)

    dnsop works on qname minimisation (limiting the amount of data sent), currently implemented in several programs, RFC almost ready)