Data Retention, Spooks, and National Security

Some of the comments I have seen about my various rants over the Draft Investigatory Powers Bill have been slightly negative - basically suggesting that we as a society should accept almost any invasion of privacy to protect us from terrorists.

I think that there is perhaps some slight misunderstanding here, and worth clearing up.

The bill has several parts - one part covers bulk intercept of communications and is basically the spying done by the likes of GCHQ. They allegedly have taps on to transatlantic cables and loads of computing power to allow them to look for threats and chase leads and to address "National Security" issues. They already do this (allegedly) and the bill is primarily to put what they do on a more clear legal footing.

I have not really said a lot about that - partly because, like everyone else, I do not know a lot about what they actually do, and partly because the technical issues are sort of their problem. There are, of course, privacy issues, and I have concerns over what they do - but there are bodies like Privacy International and Open Rights Group working on these (and I am helping with that where I can).

The main issues I have been raising are not over the bulk intercept but over data retention. This is where ISPs keep data for up to 12 months to help the authorities. This is almost always normal requests from police forces investigating some normal crimes. Apparently, as I understand it, RIPA requests relating to national security are really rare compared to more normal crimes (which is not a huge surprise).

We have seen how the police handle such requests first hand, both as an ISP and as a victim of a crime, and we have seen how badly they handle the requests and the data.

The snooping that the government want ISPs to do, as opposed to GCHQ doing, is for these types of requests - so that normal police enquiries can get details. This is also the area where knowing every web site you have visited is likely to be very unhelpful (as seen in Denmark).

So accusing my comments as trying to hamper "National Security" is somewhat misguided.

Of course, as I have pointed out many times, the threat from terrorists is absolutely tiny compared to so many other threats and disproportionately treated in legislation like this.
  • Security technology is changing, largely to tackle the very real threats of so called "cyber attacks", and this will render both bulk intercept and data retention more and more useless over time.
  • Terrorists and criminals are already able to evade both bulk intercept and data retention anyway.
  • ISP data retention is not generally related to terrorist investigations and national security anyway - that is more related to GCHQ and bulk intercepts.
  • Having ISPs collect and retain this data has cost, privacy, and risks of data being disclosed or misused which far outweigh any benefits.
In my opinion we should scrap forcing ISPs to retain data at all - ISPs will have some data anyway for operational reasons, and once the police understand this technology better they will be better able to use RIPA requests to access the data that is available now. Forcing retention for a long time, and forcing logging and retaining more data is not a good idea.


  1. "the treat from terrorists is absolutely tiny" typo.

  2. Good post - I hadn't fully appreciated the distinction between the mass passive surveillance they already do and the proposals for much more logging at ISP level.

    What about if they had a properly authorised warrant and asked you to log packet headers for someone they wanted to track? This is then no longer mass surveillance and would be much more manageable (assuming only a handful of connections needing to be tracked at any one time), but more specifically targeted and more like a a phone tapping request...

    I note that you've already built the functionality to do this at A&A, as there is the "Traffic dump" option on the control pages. Presumably it wouldn't therefore need lots of costly new hardware and software either!

    1. Indeed - targeted and specific surveillance on a suspect, with proper judicial oversight and authorisation, is generally not so much of a problem. As you say, we have some very limited capability (it is there for "debug" not "wire tapping" so may struggle) which may help. If we ever had a "wire tap" type order we'd have to discuss in some detail what was needed and what kit we would have to deploy to do that. In practice I suspect it would mean moving someone to a different LNS which is not undetectable, but unlikely for a non techie to notice.


Comments are moderated purely to filter out obvious spam, but it means they may not show immediately.

ISO8601 is wasted

Why did we even bother? Why create ISO8601? A new API, new this year, as an industry standard, has JSON fields like this "nextAccessTim...