The new elephant in the room - meta data collection. We need your help!

The Investigatory Powers Bill has many issues, and one of these is the collection of meta data known as "Internet Connection Records", or ICRs.

Up until now the main focus on security has been on the content of communications, and we now have very powerful processors (even in our phones) and we are able to ensure that the content of our communications is secure, end to end, encrypted.

But there is a new threat, the collection of meta data. By collecting ICRs from everyone, and creating a national database that can be searched and collated we create the very definition of a nanny state or big brothersister, and the all seeing eye of police state as depicted in "1984".

The problem is that (a) meta data is actually very revealing of our lives, what we do, and who we associated with, and (b) the law sees it as less significant. This second point is important as it means that new laws can collect data from everyone, not just suspects in a crime, and can allow use of that data by a lot of people without a warrant. It is only seen as serious and needing of a warrant when you want to look in detail at the communications via some sort of "intercept", the very things that will not work with the modern "encrypt everything" culture. Of course that won't work with criminals.

The UK government want to make a national system of searchable ICRs, and that means getting data from every ISP. But that is hard. There are literally thousands of ISPs, small and large, and they each would need notices to retain data. But it is worse - each ISP needs to consider the collection, storage, and access to this data, and how that will comply with the Investigatory Powers law and Data Protection law. The ISP may have to have positively vetted staff, and secure data storage systems, and all sorts. This is far from cheap or proportionate for a small ISP with only a few hundred or even a few thousand customer lines. Current policy is government pays for this too, so even harder.

Even with this security, the data is vast and the risk for it being compromised is very real. It is a far greater threat than the terrorists we try to thwart by such measures (but then so are paper cuts, well, nearly).

The only sane approach the UK government can take, if they really are hell bent on this new police state, is to engage with the back-haul carriers, like BT Wholesale, Talk Talk Business, Virgin, and maybe a couple of others. By doing this they can get almost everyone covered, even A&A customers! And all done in secret.

So what can be done?

Well, for a start, it is important to make it clear that we are not talking about helping "terrorists", "pedophiles", or "criminals" here. They can all take measures themselves, using Tor, and so on, to protect their data very easily. Also, they are often already known and already under more detailed surveillance. What we are talking about here is the police state surveillance on every single innocent person in the country for no legitimate reason. A true police state.

An important step is for everyone to ensure they use encryption as much as possible, to protect that content, but using encryption to protect meta data is harder. Tor is a start, but that is a complicated network that really should be used for those that really need it. So how can end users feel any safer over meta data collection?

One obvious answer is use of standards based encrypted PPP links. They exist, they work, and some small ISPs do this. Well done to them. The challenge is scaling up to larger ISPs. Running proper crypto for thousands of lines and gigabits of data is quite simply not easy, yet.

This is a short term issue in a way - I am sure in a few years the hardware will be up to the job, but not quite yet, in our experience.

So what can we do - well we can obfuscate the meta data!

Basically, the PPP traffic may look like normal IP data, but actually the IP addresses, maybe the TCP and UDP ports, and perhaps a bit in DNS queries, will be "scrambled" a bit. It does not have to be processor intensive or too complex. Just something that cannot easily be automated on a large scale.

Scrambling the data is not hard, the trick is to make some sort of initial negotiation to make it hard to descramble without some work. We are thinking some Diffie Hellman exchange at the LCP level maybe, and simple XOR of meta data. Maybe change occasionally during the connection. Ideally some properly negotiated obfuscation and publishing an RFC, or specification of this, so linux pppd can do it as well.

The result is that L2TP DPI based PPP capture will not easily collect meta data. Indeed, it will actually capture screwed up meta data and create bogus ICRs.

So what would happen - well, the government will have to consider talking to each and every one of those small ISPs, and pay the price for doing it - not financially viable, surely. If nothing else, the ICRs they collect to start with will be less than useless.

So we want to make an RFC - how can we get some help?

Please comment on here, let me know if you can help, DM me on twitter. Let's make a standard, or at least a specification, and I will code it in the FireBricks at the LNS end to work with A&A customers as well as a few other ISPs using the same kit.

We do not need a police state in the UK, or any country that follows, and we can help stop it, or at least thwart it.


  1. On a related note, I was rather hoping privacy-enhanced SNI might make it into TLS 1.3, but it seems it's missed the boat this time around. (When you connect to an HTTPS server, everything else is encrypted and protected, but the hostname you're connecting to is still present in clear, making censorship or monitoring at a hostname level trivial. Sending a prefixed SHA256 hash of the hostname wanted instead means it's still trivial for the actual server to identify the target - it knows its own names, and can hash them with some random prefix as a one-off operation - but to block or log the name used each time centrally would involve prohibitive amounts of brute-forcing.)

    Now it's been revealed they are *already* harvesting "Bulk Communications Data" under Section 94, I'm not sure this is really a "new" threat at all - but I'd be happy to see any steps towards enhanced privacy on the backbone.

    Building on the existing PPP encryption negotiation seems sensible to me: perhaps a new algorithm which only encrypts the first N bytes of each packet, as a first step?

    Given hardware without dedicated encryption acceleration, Chacha20-Poly1305 seems like a good candidate? PPP sessions will be relatively long-lived compared to typical SSL use, and even the OMAP 4460 in the original Galaxy Nexus should just about cope with 1 Gbps of encryption - so the upgrade needed, or throughput hit on existing kit, may be less than you'd expect.

  2. Will the existing user data over PPP LCP - or whatever, I forget, my apologies - mechanism which has been used with Firebricks before help? Jog my memory someone?

    1. IP over LCP was a bodge for one bit of kit and happened to help with 20CN ATM priority stuff. I would not be too surprised if DPi of L2TP ignored the packet type if the data looked like IP so I suspect we need more obfuscation. I am liking the idea of a variant of pop encryption that only does the header.

    2. I sold a FireBrick FB2700 to a customer on the grounds that enabling IP-over-LCP on both ends demonstrably proved that he could get the full bandwidth of his FTTC 80/20 circuit - whereas before he would only get a few Mbit/s during peak hours.

  3. I was under the impression that the IP Bill required you to notify and seek approval for any new systems or changes so wouldn't you have to legally inform the government and ask permission if you planned to obfuscate the meta data? Of course you could do it secretly but that would defeat the object if you couldn't let your customers know that you were protecting their privacy. I do hope you can tell me I'm wrong.

    1. Notify is not same as ask permission, and I think only if you have an order (which we have not). We can tell them in advance if they want...

  4. From what I've read in your blog, the Government hasn't listened to you at all so far, so as sad as it may be, how can we expect them to start listening to you now? They are just steamrolling this through regardless of what anyone says.

    Why aren't we staging a protest outside the Houses of Parliament about this? One that doesn't involve guy fawkes masks for once, because as you said most of us have nothing to hide but plenty to fear. Perhaps that will get some media attention..?

  5. If I ever work out how to configure my FB2700 (been sat there for about three months since I bought it), this will be a good use for it. With firebrick at both ends, any solution aaisp come up with should work.


Comments are moderated purely to filter out obvious spam, but it means they may not show immediately.

TOTSCO moving goal posts, again!

One of the big issues I had in initial coding was the use of correlationID on messages. The test cases showed it being used the same on a se...