Wednesday, 26 November 2014


The Counter-Terrrorism and Security Bill was published today.

There are also explanatory notes and impact assessment that give some clue to the plans of the Home Secretary, but at this stage, with the bill as it is, all it does is change a definition in the Data Retention and Investigatory Powers Act so as to allow a wider scope for secondary legislation on data retention to be made by the Home Secretary. The change of scope is to add some additional data such as port numbers to IP addresses in what is logged.

The underlying intention is not entirely clear - it seems to be an attempt to match IP addresses to individuals or devices.

This falls down for several reasons.

For a start, an IP address (and timestamp) is simply not going to be enough, and that is often all you have from the likes of web logs on a web site or some such. You need the source port at least, and in some cases the destination IP and port as well because some address translation systems use the full set of IPs and ports both ends to make a connection. Even logging all this in the ISP would not help if all you have is an IP and date/time.

But then it is not clear how they could go further than just identifying a subscriber. Getting to a device or user is pretty much impossible. There are two things in the way...

End user NAT router

It is commonly the case that the end user has a router that does network address translation (NAT) which makes all of the devices in a home or office appear to be one external IP address. This translation is not normally logged by such devices, and even if it was - the device is outside the ISP. The ISP would only have to log if they generate or process the data, so any data outside their network does not have to be logged. Maybe some large ISPs that provide the router and manage it for the end user could have some sort of back door to log this, but it seems unlikely and not something any ISP really wants the hassle of doing. All they have to do is say that the kit belongs to their customer and bingo: not in their network; not data generated or processed by them; not logged.

Carrier grade NAT (CGNAT)

There is another type of translation that is starting to happen. This is where an end user does not have a public IP of their own but is sharing it with other unrelated users by some means in the ISP network. There are a lot of ways this can work, but some include a big box that does the translation. This assigns a new port to each session at the TCP or UDP level so that the shared IP can be identified to the original customer. It may even assign different IPs to the same customer quite quickly. It could "overload" the IPs it uses where the same source IP and port is used to different target IPs and ports meaning you need all four bits of data (and the protocol and timestamp) to undo that translation.

Again, logging all of the CGNAT sessions is a massive job compared to now. At present ISPs subject to a retention notice (not A&A) need to keep their RADIUS logs where there assign an IP to a connection when it is made. That allows the IP and date/time to be traced to a subscriber. Having to log CGNAT sessions is millions of times more work. It makes CGNAT way more expensive.

Impact on IPv6

Increased cost for CGNAT should drive IPv6 deployment so as to get as little as possible running through the expensive CGNAT. That is sort of good news.

But even though IPv6 does not have the CGNAT or end user NAT router, it has privacy addressing which is not logged anywhere. So back to the issue that an end device or user cannot be traced.

Responsibility for use of you internet connection?

One thing that is definitely not being stated is that people have any responsibility for others using their Internet link. This is about tracing the IP to them, but it is still 100% legal to run an open WiFi. It is still 100% legal to run a TOR exit node or a VPN endpoint. You are not responsible for what others do with your network. Indeed, having an open WiFi or TOR exit node is a great way to create plausible deniability. In some ways this new legislation is encouraging that!

Totally pointless

There is still no tracing to end user as a person or a device with this, and it is hard to see how their ever could be. Being still legal to run a TOR exit node, and to use TOR or VPNs means that anyone can easily bypass all of this themselves, as well as having good excuses why traffic is leaving their network. The widespread use of TOR and VPNs encouraged by the default ban on porn makes this even more common and something of which terrorists will be well aware.


I am shocked that Theresa May has the audacity to make the statement on page 1 of the bill stating it complies with human rights. The ECJ said the EU data retention directive did not, and this legislation takes that an extends it. How can she say that a blanket surveillance of innocent users of the Internet in the UK is compliant with the right to privacy?

Bear in mind that in some cases CGNAT logs have to have details of what IPs you accessed to be useful, and so will basically log what web sites you visited and when. Well, "you", or someone using your Internet connection. That is a huge invasion of privacy. Notably the legislation seems to try and exclude that data, but without it many CGNAT systems are not logging enough to trace a connection back to a subscriber.

Impact on A&A?

  • We have not had a data retention notice so do not log anything for law enforcement!
  • Obviously, as now, if served with a suitable notice under RIPA to give out details of a subscriber from an IP we can do so, but that is targeted and with due legal process. We assign a fixed IP to all customers. We would always stress in such responses that the IP does not in any way identify a person that sent traffic and explain TOR and VPNs and open WiFi. I do not think we have had any valid requests for such data yet.
  • If served with a retention notice we can claim costs, and they will not be small! We can also make a few minor reorganisations which will minimise the level of logging. Quite what would happen will depend on exact wording of the new Data Retention Directive itself enabled by this if it becomes law.
  • Obviously we encourage IPv6 and are happy with people using privacy addressing which is default on so much kit these days.
  • Obviously we will clarify in our terms and conditions that we do not "run" the end user router so do not have any logs to make from that.
  • Obviously we encourage using https and TLS and your own mail servers to avoid logging.
  • If we had a retention notice, our only big NAT box which runs a public service experimental NAT64 gateway may have to change hands so not belonging to an ISP, i.e. I may personally own it and so not have to log anything. No way we are keeping CGNAT logs. Actually, Thrall Horde is a legal entity, he can own it :-)
  • In essence, nothing much changes for us, phew!
  • Though, maybe, I have to be careful if I ever leave the country :-(
  • Oh, and our voice SIMs do have a NAT unfiltered Internet connection, but the NAT is done outside the UK, so the legislation does not apply!


  1. CGNAT doesn't require that you log all traffic.

    One simple method is to simply allocate a block of (say 500) ports and IP address to each subscriber. When the CGNAT needs to do a translation for a specific subscriber it uses the next free port, and the IP address.

    This means that if any request is made to identify a subscriber, its a simple database lookup to see who is assigned those details. This would even work with the overloading solution you outlined above.

    The downside to this solution means you need to have the port capacity sat there for subscribers who are not necessarily using their connection at that time, but does mean you don't need to log anything.

  2. Dave, an ISP would still need to log something (broadly equivalent to keeping RADIUS logs: internal IP, external IP, port range, time, subscriber), for network abuse tracking as well as for lawful access data requests, but it hugely reduces requirements compared to logging every NAT session.

    Somebody doing this type of logging has mentioned that they're doing it to cope with access requests without source port information. Sorry but access requests for data from CGNAT logs will *have* to have information about the source port as well as the source IP address. Yes this is not commonly logged at present but that will have to change. A session log would allow fudging around not having source port information in some cases, but then would need highly accurate timestamps, and the costs for running the operation are enormous.

    "hugely" here isn't 10x or 50x or 100x but far far more - session logging would have more data-processing requirements than the entire rest of a typical ISP - servers, fast data storage, security (this is highly sensitive data), space to house it, power consumption. The only way I can see an ISP being able to justify that expense is if they're also doing massive data mining on this data for advertising purposes (remember Phorm?).

    1. I believe you have missed the point somewhat.

      Subscriber A is NAT'd to using ports 500-9999
      Subscriber B is NAT'd to using ports 1000-1499
      Subscriber C is NAT'd to using ports 500-9999
      Subscriber D is NAT'd to using ports 1000-1499

      A police request comes through saying "I need to know who sourced this traffic - it came from source port 832"

      Without looking at any logs you can tell that is Subscriber C. Simple. Only Subscriber C is allowed to use that IP/Port combination.

      Of course its a little more tricky than that. Subscriber C may leave, and Subscriber E join, and get reallocated their allocation. That's fine, the request should contain a date/time, and using that you can tell which sub it actually was.

    2. (I'm assuming you mean that subscribers A/C went to port 999 not 9999 in the above)

      The big issue with that sort of scenario is what happens when a subscriber needs more than 500 ports (due e.g. to a large number of short lived connections) - with CGNAT in general you can mitigate this risk by having sufficient IPs and ports that one subscriber using a lot can be handled, but if you are allocating a fixed set of ports you can't...

    3. And this also raised the other important point - your design of the technical solutions, such as CGNAT, should not have to be dictated by the government's general population snooping requirements!

    4. (I did mean 999! Oops!)

      This solution is able to allow more than 500 ports at a time (to different IP addresses) using Adrians 'overloading' idea above.

      You use the source IP, destination IP, source port and destination port all as an identifier about how to translate. IE:

      source IP:
      source port: 500
      destination IP
      destination port: 80


      source IP:
      source port: 500
      destination IP
      destination port: 80

      Can both co-exist. More memory is required for state, but memory is more available than IP addresses!

      I agree there are problems with this kind of CGNAT, but they are probably not insurmountable, and this kind of thing is leagues better than logging and storing what your customers are doing.

      Adrian - I don't think its unreasonable that the police can ask who (subscriber wise) was using an IP/Source port as part of a police investigation. As long as we are not logging vast amounts of translations unnecessarily that's fine by me. Obviously there are things (TOR, open wifi's etc) that may mean that the person paying the bill is not actually the person the police are after, but its not an unreasonable request.

      I am interested on how the government think they can identify a specific device however!

    5. Indeed, and as an ISP doing fixed IP for customers it is an easy request for us to answer if we ever got one. But we need to stress that that bill payer may not be the source of the traffic or indeed it may not be someone in the premises even. If people want privacy they can have it. Whilst I know criminals are often dumb, you would expect terrorists that are part of some group to be able to learn as a group to use TOR and the like, so I seriously doubt any of this is much help to actual organised terrorists.

    6. Dave: I think it is unreasonable to demand that any service provider preemptively log data purely for law enforcement purposes about everyone on the offchance that the police have to investigate one of those people at a later date.

      If the police have reasonable grounds for suspicion regarding an individual (which is the usual requirement for getting search warrants, arresting, etc.) then they can get a warrant and ask the ISP to log information about that individual's traffic, they can get a warrant and ask the ISP for any preexisting records that the ISP had kept for their own purposes (which are governed by the DPA), but they shouldn't have the right to spy on everything that the individual was doing in the past, any more than they should have the right to stick cameras in everyone's home "just in case" they later suspected someone of a crime and they wanted to review what that person had been doing in private.

      RevK: An interesting thought about claiming costs - if you were ordered to implement such a system without disclosing its existence, presumably the costs being claimed back would appear on the company's public accounts for all to see?

    7. Interesting. AFAIK the DRIPA stuff is not something we would not be allowed to disclose - as it could not prejudice a case if monitoring everyone. However, I think there are things in RIPA that could be gagged, and have costs, good point. Not happened yet thankfully. I really hope we would be way too much hassle to ever deal with on such things.