Monitor or intercept

The latest snooping proposals from the government have a lot of issues, moral, legal, technical, security, and many more. But there have to be some "lines" that have to be drawn somewhere and they need to understand them. One such line is between monitor and intercept. There are many other lines, and some are hard to draw (like difference between envelope and content), but that is something for another day.

Monitoring means "listening in", in effect. It means the communications continues as normal, the same as if you are not monitoring, but you as the person doing the monitoring get to see a copy of some or all of what is being communicated.

Intercepting means you actually take the communications, and do things with it before sending it on. You may change where it goes, block some of it, change some of it. It is much more serious in lots of different ways than monitoring.

The problem is that the government want to be able to track who is talking to who, i.e. the communications data. I am sure they would love to see what everyone is saying as well, but they know that really is going too far to get re-elected. To be honest, what they seem to be proposing now is going to far for my view.

However, there are plenty of means of communication that are not handled by someone in the UK. Facebook and twitter and games and all sorts can mean that they would have to convince a non-UK company to provide monitoring of that communications data. They don't like this, for obvious reasons.

So they want to snoop on the communications as it passes through UK ISPs. Essentially monitoring everything, so it seems.

There is another problem - some of these non-UK companies are using encryption - i.e. https (secure web page access) like your bank uses. This means that they cannot see what is being communicated simply by monitoring in the middle. The whole idea of encryption is to stop such things.

They claim to have a way around that! Why the hell to they think they can monitor encrypted traffic? Well the answer, we think, is that they have had vendors of black boxes show them it can be done. And, in a controlled corporate environment there are ways. One way is taht you mess with the settings on everyone's computer so that you can do what is called a "man in the middle" (MITM) attack without the computers being aware of it (installing a new CA). In effect you pretend to be facebook or twitter (or your bank) when talking to your computer, and you make it believe you. Then you pretend to be you when talking to facebook, twitter, etc, and pass on the content of the encrypted communications after looking at it and taking a copy. In theory this can be done if you are in bed with the certificate authorities and get a dodgy CA. And CA found doing this would go out of business quickly though.

For me, this moves very clearly from monitoring to interception. Now you are actually messing with the communications. This is very very bad for a lot of reasons.

1. It is just wrong - if you are monitoring, then that is all you should be doing
2. It undermines the whole principle of secure communications and can allow real MITM attacks behind the government system
3. It allows you to snoop on the bank, and anything else you want
4. It is detectable by anyone that is looking, and more and more people will look
5. It will break lots of things
6. I creates some really nice targets for any criminals and hackers to go after
7. It is technically a nightmare, including scaling issues and single points of failure

This last point is very important. For an ISP, monitoring communications can, in principle, be done by setting up a monitoring port on one or more switches. These are a port to which the switch tries to send a copy of every packet. Technically, this is simple, though picking where to put this in the network is harder. Also, it is low risk. If the black box breaks, the network does not. If there is too much data, the black box does not see 100% of it, but it sees some, and again, nothing actually breaks.

But, if you want to intercept traffic, that is a lot harder. It means that you send everything in to and back out of a black box. It means ensuring all of the communications goes via this one point, and does not have packets spread over several redundant links. It means your whole network relies on the black box working and having enough capacity to cope with the load. It also means some stupidly expensive black boxes. Looking on-line there are some expensive boxes that handle 100Mb/s of traffic and some really expensive ones that handle 1Gb/s of traffic. Even A&A's tiny network is going over 1Gb/s now. They need many orders of magnitude more in order to work with any of the larger UK ISPs. It is basically impossible but trying will break lots of stuff.

It won't actually help. There will be ways to communicate securely and without monitoring the communications traffic. There are well established systems in place for this designed to allow people working under oppressive regimes to communicate with the outside would - where being found out could get them shot. Such systems will always exist, and there is no reason to think that they will not be used.

One interesting discussion on the mailing list is what if a web site wants to be check they are not intercepted. It is possible a plugin or some websocket javascript or some such on a standard browser, right now, could check the certificate data on an SSL link. If that is the case, any web site could simple include a simple app to report to the user that they are being intercepted. The likes of facebook or twitter could make it so you cannot use them from an intercepted connection. This is even more possible, and perhaps likely, for games where the client is the game itself. Well, what would a hacker do to bypass this? they would change that javascript or app on the fly, so that it does not report the issue. Would this be going to far for our government? I bloody well hope so. If this is allowed, then what of a page that has on in it text giving instructions to tell the end user how to check the certificate (as we do on the A&A accounts web page)? Are they allowed to change the text on the site to remove such instructions? What else would they be allowed to remove, AKA censor?

At the end of the day, we have to consider very carefully how much freedom and privacy we want to give up. Remember, bee stings have killed as many people in the UK as terrorists so far this millennium. Think of the bees!

Update: ISPreview say "At present ISPs are already required, if requested, to maintain a very basic log of their customers’ internet website and email accesses (times, dates and IP addresses) for a year, which is made available to various government and security services via a warrant. This does NOT include the actual content of your communication." which is not quite true. We only log email that goes via our mail servers and web pages accessed on our web servers. We have not been asked to keep these logs for a year. We do not have to snoop on customers to see what web pages they access or what emails or tweets or pokes they do. This new law would require that.

Geolocation errors

A new issue seems to be causing problems for Internet users. One of my customers has suffered with this, and I would be interested to hear of any others.

The problem is that more and more companies are using geo-location databases to try and confirm if their users are within a certain area. One example was National Lottery refusing to allow a customer to buy a ticket as he was supposedly outside the UK. We was in Gloucester which is well within the UK! He now has issues with O2's retail on-line shop.

It is a concern when there is wrong data, but I am not convinced this is covered by things like the DPA as it is not really personal. I hope that, in general, an IP address is not personal, but there may be cases where it is.

What bugs me is that there is a definitive database of the country in which all EU IP addresses are located, and that is the RIPE whois. As an ISP we carefully ensure that this is correct (almost all of our IPs are in the UK). If someone wants to know if an IP is in the UK, the whois database will tell them - so why do geo-location companies make up incorrect data? The sort of people that want to use geo-location data are normally advertises who apparently want to offer you "girls waiting for you in Arnold, Nottinghamshire" for some reason :-)

It is also a concern that the National Lottery help page on this (they have one, suggesting this is really a problem) says to contact your ISP. Why on earth would they say that? It is not the ISP telling them that the IP is in another country. Then if you do get hold of them they apparently create some amazing bullshit to try and explain it!

This is what I have just sent to the ICO:-

I am trying to establish the extent to which an Internet Protocol (IP) address and location information which as been associated with the IP address can be considered "personal data", and if this can give rise to obligations on a Data Controller to correct errors.

The specific issue is where companies have information supposedly identifying the location of an IP address, and that information is incorrect. We are having trouble getting the information corrected.

A specific case we have had recently involves the National Lottery. They incorrectly identified an IP address as being in Guernsey when it is in the UK, and so would not allow access to their web site to make use of their services. We also have the issue with O2 (mobile). I can see this becoming a more and more common problem.

In this specific case the IP address happens to be permanently routed to a specific personal computer on a desk in Gloucester and there is an identifiable individual that uses that computer and no other. Does advising them of the name associated with the IP address make the information related to that IP address "personal data"? Would this then allow us to required that they correct this incorrect personal data (i.e. the wrong location information) under the Data Protection Act? Or are they under such an obligation anyway by some aspect of the Act?

I look forward to your reply.

Adrian Kennard, Director, Andrews & Arnold Ltd

Ghost town Bracknell?

I pop in to town from time to time, every week or two. I have noticed a few shops that have closed down, some for a long time now. But today it really hit me just how many are now closed, and have not been replaced with new businesses.

Walking in to Princess Square, 6 of the first 8 shops are empty. The high street is full of empty shops. I noticed today that the Thornton's chocolate stand has gone. Indeed, several of the shops I planned to visit have gone. I am pleased to see Bracknell Cycles have only moved, not closed down.

But the overall impression is that it is turning in to a ghost town. Scary.

The Queen messed up my cash flow!

Well, cash flow has been a tad off this month, and finally I have tracked it down. Things like this niggle at me, as you can imagine.

Thanks to the customer that raised this - actually telling me that we gave him extra credit. He did not have to, but it was useful understanding what happened.

The problem is the Queen's Jubilee! Or more specifically the extra bank holiday right at the start of the month.

We have a number of customers who are billed on 1st of each month but selected Direct Debit on the 9th. Well, obviously "the 9th" has some leeway as it could be a weekend, etc, but within a couple of days on or after the 9th is how it normally works. We allow people to pick one of four dates during the month for collection of Direct Debits.

The problem is that people billed on 1st June could not get the agreed notice for Direct Debits (we work on 5 working days notice) before the 9th, or 10th, or 11th. Normally this does not happen, but the extra bank holiday pushed it too far, and the cautious programming in the accounts system erred on the side of the customer and gave all those affected an extra month's credit. So hundreds of people billed on 1st June are paying on 9th July rather than around 12th June.

The system did what it was told, it was not actually an error (which is why we had trouble finding it), but I have changed the rules for the future. It now allows 3 working days after the "agreed" day in the month. By allowing working days it should cope with things like this (which could happen for an Easter). The DD rules allow 3 working days after notified collection date so this is consistent with the rules, even though it would mean notifying a later date in our case.

The invoices we sent were, of course, right, and clearly stated the extra (interest free) credit terms. The DD notices are, of course, right. The payments are not seen as "late" and no penalties charged. The system did what it was told!

What worries me with this is not just the tens of thousands of pounds of payments delayed by a month, but the fact I can pretty much guarantee that someone will complain! Someone will be unhappy paying two months in July even though they paid nothing in June, and even though the typical amount is around £20. I will be really pleased to be proved wrong on this. I am sure 99% of affected customers will not complain, and many will realise that we have given them a month's free credit, but someone will complain I bet you...

Am I just getting cynical in my old age I wonder?

At least we have made changes to avoid this confusion in future. It is one thing making any sort of mistake, but another not to learn from it.

3D TV: LG vs Sony

I posted on 3D TV a while ago, when I got the Sony KDL-55NX813. Now I have had a chance to compare with the LG 55LM960V. The differences are interesting. Overall the LG wins. There are lower models such as the 55LM760T for under £1,750 in Curry's at present.

When we got the Sony, it was the first LED lit LCD TV we had got. We were not impressed, especially with banding effects on dark images. The LG is also LED lit LCD, but the high end LG uses some rear LED lighting with their "Nano" technology. Overall this seems to be a better picture so far.

Like most TVs these days they both seem to come with a whole load of (IMHO) horrid image processing crap. I managed to find how to set the Sony to "full scan" but I don't think I ever managed to turn off the motion processing. The LG allows "Just scan" mode to see the picture as broadcast, and also a "game" mode that turns off the motion processing.

Personally I find it strange that now we have a standard for HD images, that TVs, by default, have modes to overscan the image (scaling up and cropping the edges). It made sense in analogue TV days, but not now. The scaling is done well without anti-aliasing effects but I simply cannot see the point. It is a digital signal, with a defined pixel array, and the TV has that exact array of display elements. As I say, both TVs let me set that, thankfully. Wikipedia article on overscan says "For 1080i/p overscan is undesirable, as it reduces picture quality and 1:1 pixel mapping is preferred." so I am clearly not alone.


As for the motion processing, whilst I can see some logic in the fact that as modern TVs can update the screen faster that the original image (e.g. 100Hz not 50Hz) then maybe they want to make intermediate frames. However, ultimately, you are not adding information. When it works, it works well, fair enough, but when it goes wrong it is really annoying. No system can be perfect. Examples of where it goes wrong are things like the slow scrolling credits at the end of films, and slowly scrolling horizontal text you get on news channels. Both can judder or do strange things, even making them unreadable. Even when not some slow scrolling text you can get odd juddering on panning shots. With the LG it looks like I have managed to turn this off. Watching the F1, all the motion was smooth except the slowed replays, but that is as broadcast. The trick in the case of the LG was to select "game" mode.

The sound is not bad, but we have a simple sound system (Sony HT-AS5) which is much better than the TV speakers on either. With the Sony TV it pretty much just works, as you would expect. TV volume control works the sound system not its own speakers, etc. It also controls power on/off. It uses Audio Return Channel (ARC) if another source is selected on the TV. It was still a tad strange on the Sony in that regardless of the audio source (even when 5.1 stereo) it only used the front speakers unless you set "Theatre" mode. This mode also adjusted the picture! The mode was forgotten every time you turned it off. Why would you ever not want the audio to work properly and use all the speakers? Mad!

Sadly the LG does not understand the sound system. Even though it, and the AV system, do ARC, the sound system produces no sound. I can turn off the TV speakers and use the AV system in-line, but that then means faffing with the audio/video sync and means no sound on other TV sources (such as digital tuner). I may go find an audio optical fibre cable and see if that will work. Who knows if it will drive all speakers?!

Anyway, lots about the TV and sound and none about 3D yet!

The LG uses circular polarized filters in front of the screen and uses passive glasses. This is a massive improvement in 3D watching. Massive! You use the same cheap glasses you get for cinema use. They are light. They do not flicker. They do not need batteries. With the Sony (and most, if not all, other 3D TVs) the glasses use LCD shutters and are heavier and flicker. The LG technology is much better - it means there is no flicker at all on the 3D. It also means the viewing angle is better (i.e. where in the room you sit) and the angle of your head is not critical (which is much nicer).

Glasses on left. Click to see fill size for detail.

There is one disadvantage in the way LG are doing it though, and this is something I do hope they fix in a later model. Essentially, as they show both images at once, you have some compromise. Without shutters (and hence flickering) you cannot show the same pixels for left and right in the same physical space (well, not yet). So they alternate which lines are left and right eye. This means that you see half the lines (540 not 1080) in each eye. There is a slight vertical offset, buyeyes can cope with that, generally. Sadly, I can notice this as it is stripy, but it is a very subtle effect, and to be honest the reduced resolution is worth it for the massively better usage that you get with the passive glasses. The fix is for them to do a 2160 line panel and have alternate lines so allowing full HD 3D with passive glasses.

Another big issue with the shutter glasses was bleed through or ghosting where you would see some of the other eye image like a sort of shadow. I see none of that on the LG. As you can see from the picture about, the filtering is perfect.

There is an irony which is that 3D is usually transmitted as left/right (at least by Sky, anyway), which reduces from 1920*1080 to 960*1080. But the LG 3D system then makes the images in each eye 960*540. If sky transmitted top/bottom, making the image 1920*540 then the LG system would not lose any more resolution. As the effect of 3D relies on very slight differences in horizontal placement, this would give a better 3D effect as it would maintain the 1920 horizontal resolution. Given Sky seem to like the LG 3D sets, it is a shame they are not switching to (or offering an option of) this mode of transmission.

They do include 4 normal glasses, one clip on set (which I can use at the cinema), and two game glasses. The game glasses are left+left and right+right which means a split screen game can be played by two players as full screen seeing only their own game. There is some input lag which is not ideal for gaming, but the effect is pretty good. Also, this does mean that for those few people who find even cinema 3D to be uncomfortable, they can watch a 3D film with the rest of the family in the cinema, in 2D, by using the gaming glasses.

So, I would recommend the LG over the Sony :-

• No flicker at all
• Light weight glasses - same as cinema glasses
• Cheap glasses
• No batteries in glasses
• Better viewing angle in room
• Allows head to be at different angles
• No bleed through (seeing ghost of other eye's image)
• Dual player game mode glasses included (left+left and right+right)

Negative points

• Loss of vertical resolution
• Some striping because of loss of vertical resolution if close to screen

Tricky

But at least it is a Rubik's cube that a blind person could do...

New type of email address

I have wondered if I could have a shorter email address. Something like x@e.gg is pretty short, but what of say e@gg ?

My understanding is that the guy running gg actually tried it, and found it did not work. Partly down to email clients, but annoyingly, whilst a top level domain can have an MX record, it is actually part of the RFC that disallows this.

RFC2822 makes domain :-

 Domain = (sub-domain 1*("." sub-domain)) / address-literal

But the older RFC822 made it :-

 domain = sub-domain *("." sub-domain)

So it used to allow a domain with no dots in it, but now requires dots.

Shame, especially for many of those applying for new TLDs as they cannot have things like info@google as an email address.

Update: See comments as it seems I missed something and such email addresses are valid, just unlikely to work.