Evaluating a VPN provider

At A&A we are looking in to how we can best help customers exercise their human rights for privacy and rights to net neutrality and to access to legal content.

The IP Act puts in place horrendous snooping powers, and the DE Bill as proposed puts in place a new national censor with the job of blocking porn sites - legal porn sites. We can all imagine how much further such proposals could go.

At present there is no ban on operating a VPN - it would be hard to ban without also banning https used by many web sites and businesses and banks and the VPNs used by industry and even parliament.

There are VPN providers now (sensibly) targeting the UK market - they provide an VPN endpoint which you connect to from your computer or using a router that can do VPN for your whole house. They make a point of having equipment and legal entities in countries that do not require logging and snooping, and make a point of not recording anything or accepting orders from governments like the UK.

So, the next question is how we evaluate VPN providers and even make some recommendations. We may even set up a VPN operation ourselves (well, not ourselves, a foreign company with foreign servers, so no subject to UK jurisdiction).

These are this obvious aspects I can think of, but keen on other comments.


One simple aspect of the service and the devices you choose to operate the VPN at your premises is whether the service can keep up with the speed of your Internet connection, such as an 80Mb/s VDSL service.


Obvious one, but you want a reasonable price. Free is great but there has to be a catch some how, so you expect to pay a few dollars a month at least for any reasonable service.


Are they really not logging anything, do they have a clear history of refusing information requests?


Can we really trust them? Very hard to be sure but reputation and how long they have been in business are key factors.


Can they have your traffic look like UK traffic (if that is what you want)? This may be tricky and without it things like netflix may not even work. If someone sets up a service specially for UK use they may be able to convince netflix and others that it is UK IP addresses even if plugged in via another country.


MTU issues, latency, transparency of IP protocols and ports and so on, IPv6. All things that matter from a technical point of view. Ironically, maybe even fixed IP - if blocking/censorship is your main concern.

Openreach/BT Split

As reported by BBC, OFCOM are getting Openreach split off as a separate legal entity from the rest of BT plc.

What do we think of that?

To be honest it is tricky - A&A deal with BT plc for both BT Wholesale and Openreach departments. The latter is for phone lines to support the broadband services we sell. Mostly we are dealing with the BT Wholesale part.

A lot of the reasons behind this are coming from some of the larger operators who have to deal with the Openreach part of BT much more. They are concerned that BT Retail get some preference of some sort. There are concerns over whether BT are investing enough in infrastructure.

I am not at all convinced by some of the arguments, as Openreach had 86% coverage of UK premises for VDSL (FTTC) back in April, and are pushing hard on this. So there seems to be quite a lot of investment in infrastructure. Also, we don't really see much in the way of preferrential dealing with BT Retail at all - it seems they, and Plus net, have as much hassle with Openreach as anyone else.

But the devil is in the detail, and it looks like the new Openreach will be owned by BT Group plc, so there is a source of investment via that route. But this also means one of the big issues still exists - if BT Retail pay lots to Openreach, that has no impact on my BT Group plc shares.

In practice, Openreach is already operated like a separate company - annoyingly so on occasions. This split will actually remove one useful aspect of being one company. At present, when dealing with BT Wholesale they will often blame Openreach (or "their suppliers") for a failure. Legally that could be force majeure (matters beyond their reasonable control) if they were not in fact the same company and in fact blaming themselves. Being able to throw that back at them can be useful and force them to do their job and not just blame someone else.

But otherwise I would be surprised if we see any difference at all from this move - apart from new contracts which gives them a chance to screw us over somehow.


Human Rights

The BBC did a good article on the Investigatory Powers Act (which oddly has yet to appear on the legislation.gov.uk web site).

But there is one aspect they did not make that clear...

The headline was :-

"Tech firms seek to frustrate internet history log law"

It should have been

"Tech firms seek to help people exercise their basic human rights"

We all have the right to a private life and family and correspondence,  and that is all people are after here. Nobody is aiming the thwart the law, unless, that is, if the law is trying to take away that basic human right.

So please, BBC, report it correctly. Nobody is trying to break laws, or frustrate them - we are just trying to exercise our human right in EU and UN declarations of human rights - a right to a private life - that is all.


First they came for the porn sites

If it was not bad enough with the Investigatory Powers Act passing in to law, we are now facing another wave of stupid and dangerous law - the Digital Economy Bill.

Several people have written some good pieces on that - see one of the latest by Jim Killock of Open Rights Group.

What problem are they trying to address?


Seriously, it is not clear what the specific problem is here - but the Government have been after porn sites for a long time. Those of us that are cynical see this as just one more step in censoring the Internet, one small justification for more filters and laws to back them, so that more and more can later be added to the filtering lists over time.

I will be delighted if someone reading this has some concrete evidence of studies showing what problem exists to be solved. Are there any MPs that did not see porn before 18 (or a pig's head maybe?).

Personally I see two issues, the first is younger children inadvertently encountering unsavoury content on the Internet. This is easy to address with existing tools and some education of parents. The second is older children that want to access porn on the Internet but are not yet 18 (e.g. people that are 16, can fight in the army, and can get married and have sex, those sort of people as well as those a few years younger). This is not a "problem" to solve - teenage kids have accessed porn, probably forever, and long before the Internet. The only problem is where they see porn as "reality" rather than "entertainment and fiction", and that is solved by education. No amount of blocking will ever stop a teenage kid accessing porn if they want to and that is a simple fact!

What is the solution they propose?

There are two key parts here, both of which have huge issues.

1. Age verification on porn sites. Unlike whisky selling web sites that have "Are you over 18? Yes/No", they mean something that can actually validate that you are over 18.

This is serious - a lot of people (adults) access porn. It is not unusual. However, the fact that people access porn, and the specific preferences for people's fantasies is very personal information - sensitive personal information which is valuable to criminals, may be very embarrassing, and usable for blackmail and who knows what else. Remember, until surprisingly recently a preference for same sex relationships would make you a criminal suspect! If anything, it is one's sexual preferences that are perhaps one of the main reasons for the basic human right to a private life.

The only real way to do any sort of age verification is to identify the user somehow. This is a huge challenge to do "over the Internet". Almost anything that can be used to identify a person can be copied and used by their teenage kid - and something like a credit card is one of the easiest. Also, bear in mind, kids as young as 8 can legitimately get a pre-payment visa/mastercard now.

No matter how you try - the system will be flawed somehow (what can an adult type or do on a computer that a child cannot copy?).

But no matter what you try - there will be an association of the web site access with the identity of the person accessing it. Steps can be taken to try and avoid this linking together cleanly by some means, but ultimately there will be a link somewhere, and that allows for a huge database of sexual preferences for adults in the UK. That will get hacked or sold or both.

We are talking about a database of the sexual preferences of every UK adult! But I suppose the Investigatory Powers Act allows such a database to be created as well - at least tied to an Internet connection if not a person. This database will tie to specific people.

2. Blocking of porn sites. Only UK sites would have to comply (putting them at a commercial disadvantage and hampering minority groups), so they propose that sites that do not comply can be blocked by an order on UK ISPs.

There is plenty of evidence that trying to block illegal sites that assist in copyright infringement in some way simply does not work. It is a massive game of "whack-a-mole" at best, and totally pointless at worst. This has been tried, and it simply does not work.

But trying to censor completely legitimate and legal web sites, which have financial and legal resources, is going to be a much bigger challenge. For a start, there are a lot of them, a hell of a lot. We are not talking of blocking one web site like piratebay, we are talking every single non UK porn web site that is not going to pay for UK age verification services - they would be much more successful investing in ways for UK "users" to bypass government censorship.

But as Jim Killock points out - the second "age verification" becomes the "norm" for UK porn "users", we see massive opportunity for fraud - porn sites that insist you have to enter card details to proceed and even quoting the UK law on this. Quote a law and link to it and the request seems legitimate. If all of the free sites vanish (unless you try a little to find them), then we will be swamped by the bogus sites collecting personal information. And there is almost no end to how much personal information they can ask for in the interests of "age verification" and a promise not to actually charge your card or log the details. There is no way for people to tell the "real" (and supposedly safe) age verification requests from the bogus ones, and there is a massive incentive for people that are defrauded to keep quiet rather than own up to the site they were trying to access. It will be a secret and undercover fraud that will be a nightmare to track down.

What is the right answer?

You have to assume there is a question/problem in the first place, which is not clear, but assuming there is one - what is the answer.

I think it is simple to say - education is the answer, not censorship.

But I'll try and be a tad more helpful.

For young children you need education of parents and guardians on how to use the many tools available to them, and some education that the Internet is not the ultimate baby sitter. There are many tools - just installing any operating system these days will offer a range of "parental controls". There are safe-search settings on search engines and there are controls that can be set in most ISPs systems that offer filtering as an option. ISP filters tend to be whole house and so a tad crude but there are DNS based systems which are easier to set on a per computer basis and provide controls not only on content but times of day, etc. Lots of tools exist, in the control of the parent/guardian. Yes, they are easy for some teenage kid to bypass, but we are talking here of young children not trying to access porn, and for that all of these tools work well.

For older children that want to assess porn the first thing to realise is that there really is no point trying to stop them doing so - it will never work, sorry. But education matters. Along with sex education you need education for teenagers about porn! I know it seems odd, but teenagers need to know porn exists, and that every type of porn and sexual preference you can imagine (and many you cannot) exist somewhere. They need to now that porn is entertainment and not reality. That it is fiction. That there are many things out there, with which they may feel uncomfortable, and that they have the choice of what they look at and what they do not. And that most of all they need to understand that it is not in any way a guide to any real relationship, just as many fictional and entertainment films are not a guide to real life. With some basic education people can enjoy porn, avoid things they do not enjoy, and still have meaningful sexual relationships in the real world.


IPv6 and Zen

With my FireBrick hat on for a change, one of our customers has a Zen line with IPv6

He was surprised to find the IPv6 was not working when using a FireBrick FB2700, and so was I!

As usually, within a couple of hours of reporting the issue, we have new code that solves it, even on a weekend. I have to say though that I was impressed that Zen looked at the FireBrick web site and manual in an effort to help their customers with this. Well done guys.

For so long FireBrick has been used on A&A lines for IPv6, it is nice to see how other people do it.

The problem is that the protocols used for this are horrid. I think I mentioned this before. I really think a PPP level negotiation would make a lot more sense. I even have my name on a draft RFC, but no luck on that.

What happens is, after the IPV6CP negotiation for a 64 bit interface address, you can then send IPv6 using the FE80:: based address. To get any real IPv6 addresses works a lot like a LAN but with extra bits. You can get Router Announcements on the PPP, and pick an address and you can use DHCPv6 to request an address and prefixes for your LAN.

Traditionally, as it seems the most common way, FireBrick used the latter - expecting the DHCPv6 to allocate a "real" address on the link itself (maybe) and a prefix for LAN. We actually ask for one or more /64 prefixes for different LAN interfaces as configured, but by default it is all of the interfaces we have. You can configure which interfaces to use with which PPPoE links though, if you want.

We have a bug where the IPV6CP forced a new interface address, which Zen do, specifically 00:00:00:00:00:01 for some reason. We were not then using the right FE80:: address for DHCPv6, or rather not accepting replied to the address we were using due to a silly mismatch of the two.

It also looks like Zen do RA for the PPP side address, and DHCPv6 for Prefix Delegation, which sort of almost worked. We had some bugs. For a start, we did not handle "infinity" as the validity for these (even though that is what we requested), silly error, but it meant we expired every allocation one second before we got it! Took me a while to worth that one out...

We also did not handle a case of asking for a prefix even if no interfaces are set up to use it (e.g. where Zen is a secondary ISP on separate routing table).

However, with a few tweaks we have it sorted, only using DHCPv6, not RA, but picking an address for PPP link from the delegated addresses and requesting at least one /64 by default.

Obviously, we are happy to test with other ISPs and make sure we work. IPv6 should "just work" with a default config with any ISP, not just A&A.

Our customer now says IPv6 wastes a lot of time - not because of any difficulty setting up, but because he just spent an hour playing www.loopsofzen.uk which is only on IPv6.

Well done Zen, the world is gradually moving forward. A&A started doing IPv6 in 2002.


Snooper's Charter and A&A

First off - I appreciate that my blog is not an official statement for A&A, but I have linked the status page here to give you an idea of my thoughts on the matter and how that may play out for A&A in due course.

Summary: Watch this space - more to come over coming weeks.

I have commented many times on the Investigatory Powers Bill, and submitted written evidence to parliament as well as oral evidence to the committee. I have attended meetings with privacy groups and legislators. I have spent a lot of time on this. I have tried very hard to try and get some degree of sanity in to this legislation, and I am sorry to say that on the whole I have failed to make any real changes, sorry.

Once we see it, I am planning to go through the final wording of the Act, with a lawyer friend of mine, and we are going to try and make sure we understand the nuances that finally made it in to law. Once we have done that I do plan to write up something much more comprehensive.

But how does this impact A&A and the services offered.

As I say, this is not an official statement yet - we'll be posting more details of what we are doing and when as time goes on. At this stage there is nothing that needs doing urgently - it will take time for anything to happen in relation to the new Act and a lot of time (and money) for the monitoring and logging to get in to place.

It is also worth pointing out that I don't really have a real problem helping the police investigate crimes as long as there is a proper oversight and control.

In practice a lot of the Act relates to the intelligence agencies, and whilst there a lot of problems with this, it is unlikely we can do much now, or that we would be impacted by this aspect of the Act. However, some of the steps we can take for privacy thwart those parts of the Act too!

The real issue we see is the huge invasion of privacy in collecting and storing data on innocent people - and the bulk powers for "data retention" do just that. They are designed to allow lots of personal information to be gathered on everyone - so mostly people completely innocent and almost entirely people not even suspected of a crime in any way. This is compounded by systems to search through that data over many ISPs and provide it to a wide range of people including the police, without a warrant of any sort.

We expect that it is very unlikely A&A will be asked to do anything - this is because companies like BT and Talk Talk will be asked (ordered) to and that will allow deep packet inspection in the back-haul networks that are used by A&A (and most ISPs).

So what can we do about that?

One of the biggest things we can do is provide information and advice about exercising your basic human right to a private life. This will take some time to put together in detail once we fully understand the legislation. We will start a specific section of the wiki pages as well to cover ideas people have. We are interested in suggestions people have too.

There is also a good possibility that we can engineer some services that operate in a way that bypasses the logging. A simple example would be an outgoing email server that is esmtp only (encrypted) to a service that is outside the jurisdiction of the UK and new law. This would be servers outside the UK and also set up in a way that A&A, or any people in the UK, technically have no control of them. This means that nobody under UK law could be required to comply with an order to include logging on those servers. As an ISP we, or BT/TT, would only see encrypted esmtp traffic to that server and hardly any useful meta data on the emails and nothing on the addresses involved.

Of course, even something simple like this suffers the big problem that the person at the other end of such communications (e.g. emails) will not have the same degrees of security and hence allow logging of meta data at that end. This is always a problem with any communications.

There is also a lot of advice on the use of tools and apps that help - like signal and tor. Sadly even tor has limitations and performance issues.

One answer is VPN services with endpoints outside UK jurisdiction but still reasonable latency. This is hard to scale up - but we are already talking about this in the FireBrick dev team about this.

In the short term we are seriously considering a trip to Iceland to investigate data centres and transit there - perhaps installing some tin that can run VMs as needed - but we also have to investigate the exact way such servers can be outside our control and hence not subject to orders on us to add data retention or intercepts under UK law.


It is, of course, right for everyone to expect to be able to exercise their human rights, including the right to privacy. There are a lot of people, in light of this incredibly intrusive new legislation, that wish to do so, and so there will therefore be a lot of companies working on ways to provide (sell) services to help people do that. These services will have to be designed to be outside UK law, obviously. But this means they are also outside the law where there is a specific suspect of a crime, and a more reasonable justification to provide intercept or collect data to help law enforcement (with suitable warrants). So by encouraging people to need privacy and encouraging companies to offer privacy you actually make fighting crime harder. It is worth bearing in mind that serious criminals have always been able to avoid this type of monitoring, but more and more normal people and, occasionally, those committing minor crimes will find it easier and easier to use services offering privacy now.


More on Live TV from my man cave

SkyQ is fucking useless - it did not record tonight's live RT UK News interview. It is a heap of shit is so many ways.

Update: Someone managed to record it - thank you.

However, I am pleased to report that doing a live Skype TV interview worked well.

They texted me with around 30 minutes notice - the Skype login did not work - the camera was not set up and then did not see the lens but instead of telling my it just caused the app to freeze, the headset cable did not work, and I had to reboot the Mac, and well, those 30 minutes were busy.

However, the good news is that, in the end, the audio worked properly simply by selecting 48kHz as a USB audio device. The room is a tad echoey, so that needs improving but the H4n pro worked for audio.

Also good news is that the 1DX MkII with a 24-70 f/2.8 L lens worked well with the BlackMagic HDMI box to provide full HD. The picture, as broadcast was crystal clear and clearly HD and looked excellent.

I am well underway to having a decent studio here in the man cave for TV interviews, and I think I did quite well being live on air at 30 minutes notice to be honest.

Only snag was audio in my earpiece went to zero just as we went live, I wonder if the volume control caught on my collar or something, fixed, and was live a few minutes later. Well done coping RT...

The only sad part was that the topic was the IP bill finally becoming an Act. Sad days. More on that when we have reviewed the final text.



Dreams are weird shit, aren't they. This is one of those areas where different people can be very different, as I understand it. I am one of those people that can occasionally remember dreams - usually if I somehow wake up during a dream. My latest medication woes have meant different sleep cycles and dreams, but overall for the better I think.

I have to say that I do wonder what the underlying evolutionary process is that created the state of dreams we experience, and as I understand it is experienced by animals as well. There is some logic in the idea that it is a way for us to review our experiences and "safely" try many possible strategies and see how they play out. This makes some sense, but dreams have a whole load of sanity checks turned off.

Obviously, the most basic disconnect in a dream is our actual control of our body - we may move our eyes, but for most of us we are not actually moving our muscles otherwise and getting up and walking around and doing shit - we un-plug our brain from the world for a while. Errors in that process lead to sleep walking, and the rather odd moments we can experience as we wake up some times.

To some extend that makes sense - if this is the ultimate role playing experience to try out scenarios and work our strategies for the future. I have dreams of upcoming events, whether a meeting with BT or a live TV broadcast or, well, anything new that I know is going to happen. I expect these dreams help me prepare. I also have dreams of coding and solving puzzles. Some times (as I have blogged) theses really work to find the answers. The other day I was designing an XML system in great detail for creating a comedian's script to tell jokes, in a dream - I have no idea why!

But what puzzles me is how dreams (at least for me) also un-plug so many "sanity checks" on reality. So many things can happen and not be questioned.

So, one of my latest dreams for amusement:-

The idea of flying is not uncommon - but for me this is a recurring theme where I cannot actually fly but, with a lot of mental concentration, I can sort of hover several cm over the ground and then sort of "glide" along the ground.

In this dream people can do that - not many, but I can. It is seen a lot like skateboarding - something some people can do well (not me!). There are kids that do it recreationally, in like skateboard parks, but if you saw someone doing it in the street you would think a bit childish and maybe even a bit dangerous.

In this context the fact I can do this is a bit of showing off, occasionally, but not seen as a very useful skill. Obviously someone doing the long jump would be cheating when using this skill, but that was about it.

Now, the weird bit is that in the dream I realised it was impossible - it made no sense in terms of laws of physics, and it dawned on me that it meant that the universe in which I lived was fake!

I was realising that the universe was some sort of simulation or something fake and not real as I had an impossible ability!

What was strange is that in the dream I did not make the leap that the reason the universe was fake is that I was in a dream! I was thinking simulation with a bug in it somehow.

As I awoke I had what seemed like several seconds of dread that the real universe was fake because I could do something impossible, and a sort of double take that no I cannot really float a few cm over the ground.

So yes, some times dreams can be fun!

P.S. I like someone commenting to me of dreams of "no pants", not a dream I have ever had, having spent some of my youth living in a naturist club where such was clearly not an issue!



We live in a strange world where some times we share personal information and sometimes we don't and sometimes it is a problem and sometimes not. There are those that will say I should not have said I was diabetic in a previous blog post.

To be honest, I find the lack of information, and/or overwhelming information, on the Internet to be an issue when facing some of the challenges of growing older and more broken over time. So I hope these occasional health related blog posts can be of interest to those who find themselves in the same boat as me in the future.

For a little while I have been on some blood pressure medications. I was originally not at all keen on this, but apparently high blood pressure is a "silent killer" and so it is important. The thing that swayed me was when I realised that my higher blood pressure was actually related to some headaches. I found I could not drink more than a certain amount without suffering with a crippling headache all night - nothing like simple de-hydration, and very binary - below a certain level was fine, above it was agony. Being a tad scientific I got an optic for my whisky bottle so as to understand how much I was drinking, and over time the amount I could "safely" drink got lower. Only when I was put on blood pressure medication did I realise the connection - some Perindopril and now Ramipril made all the difference. But over the years things have progressed, and so something more was needed.

I was put on Indapamide 2.5mg which works somewhat differently, and, well, works well on the blood pressure - too well in some ways! I have my first review of this with the diabetic nurse this week, and we'll see what she says - I'll add to this post.

Yes, I can drink what I like, but there have been a few odd effects. To start with, as they suggested, a lot more going to the loo, but that settled down. Headaches for a couple of weeks, but that has cleared up. Initially blood pressure somewhat all over the place, nearly passed out at one point. Again, settled down.

Overall, after a few weeks all I am left with is issues with low blood pressure (100/70) which mean standing up quickly, running up stairs and cycling up hill, are causing me problems. I hope we can address this with review of the medication and dosage.

But there is one really weird change - my blood sugar rising (one of the listed symptoms) meaning I have gone from 38 units a day to 56 units (so far) to try and address high blood sugar some of the day. However, oddly, I am actually finding myself much more awake and "with it" all day now, and sleeping just as well at night.

So overall, if you are put on Indapamide, I would say you may well have a few weeks of hassle in various ways. I got through quite a lot of ibuprofen in the first few weeks, but overall, it really works.

I hope this post was of some use...

Update: They don't have 1.25mg available, but do have a combined perindopril 5mg and 1.25mg indapamide combination - so trying that. Wish me luck :-)


Brick wall BT

Hurry up and build that wall, Trump, I want to bang my head against it as it would be easier than BT...

So, simple BT fault, suspected issue on fibre back-haul from an FTTC cab, started after MSO, so probably damaged to dirty fibre. Not a complicated fault at all.

Symptoms are idle levels of packet loss all day, sometimes peaking to 20%, that is shitty!

The dialogue sort of goes back and forth, but we finally go somewhere, after escalation, with a clear statement from BT:

'We have run diagnostics and can see numerous code violations which will require an engineer to liaise with 2nd line DCoE if the fibre escalation team are unable to assist.'

That is pretty clear, so we ask them to fix it.
  • Us: Please fix the fault.
  • BT: You need to book an SFI
  • Us: An SFI is an optional service to check the metallic path to SIN349, nothing wrong with the metallic path here, so silly, just fix it
  • BT: No, you have to book an SFI, that is the process
  • Us: OK, what is the process for fixing this fault without ordering an optional service.
  • BT: There is no other process
  • Us: Are you really saying BT have no process to fix this fault without ordering an optional extra service - if so that is breach of contract as contract says you will investigate and fix faults
  • BT: We don't see a BT fault
  • Us: OK, so you are saying a line with constant loss peaking at 20% is acceptable, is that the formal standard BT work to?
  • Us: What actual investigation did you do, and to what standard or reference did you test the line so as to decide there was no fault?
  • BT: If you want us to investigate the fault report you raised further then please book an SFI
  • Us: No, the contract says you will investigate and fix faults, are you refusing to?
  • Us: Repeat of questions about this being "not faulty" at 20% loss?
I'll add some more as it goes. But it goes round in circles.

Either BT

(a) consider this to be acceptable level of service, in which case, especially with ideas of universal broadband service obligations and automatic compensation, we need to take this up with OFCOM, BIS, and the Digital Economy Bill parliamentary committee as a matter of urgency.

(b) accept that this is a fault and BT are breaking contract by refusing to investigate and fix it.

BT just need to pick one and stick with that story to the end. Either that or just damn well fix the fault!


Scammers are obvious

We all get scam emails and calls and the like.

I find them obvious, and so do my friends, but I live in fear that someone will be cunning enough to fool me.

Indeed, I ponder what steps would be needed to fool me, and feel there are not that many, and that worries me.

But I realised now, as several have pointed out, that this is deliberate. Scammers are not so stupid! There is a reason the emails have iffy logos and bad grammar. It is to weed out, well, me!

Basically, there are people with some clue and they are the people scammers do not wish to engage. They are the people that will both take up their time and not get them any money.

So the scams look obvious and broken and badly worded. They don't need to try and fool me. In fact they want me to immediately see through it and hang up or delete the email and let them concentrate on the muggles.

Why did I not realise this before - maybe I am not that smart?!

Technically correct (@aaisp and 10Gb/s backhaul)

A&A have just upgraded one of the back-haul links to 10Gb/s today, and I have to say it went rather well. In fact we think we managed it without dropping a packet :-)

Basically, using BGP, one can move all of the traffic around and do so whilst the routers still know where to send the packets. We moved all of the traffic to one link at a time of day that was under 1Gb/s of traffic total. And when done, we moved traffic back. This did mean some staff up at 4am, including myself. Actually the whole process meant two staff in the data centre, me on a computer here, other staff working from home, tech staff at the office in case there were problems (we opened the phone lines early), staff at Talk Talk, and data centre staff, all involved to co-ordinate one fibre move.

Now we have done the first fibre, we can do the second any time of day, so that is planned for tomorrow during the day. It should be just as seamless.

But why do this upgrade now? Well, I am actually cross we did not do it a month ago. We forecast the need for more capacity and we started talking to Talk Talk several months ago, but these things seem to always talk longer than we expected. To be clear, all of the kit was in place as Talk Talk have 10G ports already. They upgraded their kit in the data centre some time ago. We upgraded our kit to allow 10G ports end of last year. The only technical step is config and change of optics in the switches which, as we see today, can be done over a couple of hours, and with a few days notice and planning. But all of the paperwork involved in ordering this, at a cost that is more than my first house, is very time consuming. Well done to my team for sorting it though.

Anyway, back to why I am cross. I am cross because our service was not as good as we expect. We don't make contractual promises as so much is out of our hands, but we aim high none the less. It is interesting to see what that actually means. I have picked a line that is on that backhaul. The congestion we saw was happening only on some of the lines, not all. But I can show you what the congestion means in practice.

This is a graph for one of the impacted lines, with our loss and latency monitoring every second of every day recording the packet loss and round trip latency for the line.

I'll explain the key here - the green dots are usage and not relevant here, what we are looking at is the blue/green at the bottom. Normally it is low, and is in fact 7ms round trip latency minimum, average, and maximum. But you can see some green bits, and a slight hump in the blue even. The green is peak, so typical one test in a hundred. The blue is the average, so needs several to be high for that to increase. So this shows that some times there are peaks of 20ms or 30m and even 50m just after 8pm even though the average is still down at 10ms or below.

Now, I know some people that would kill for a line that good and that clean from their ISP, but really, it is not the A&A way. We expect better!

This is what it should look like, and is in fact another line at the same premises on the same backhaul. As I said, only some lines affected. This is what it will look like now we have upgraded.

This is 7ms latency! In fact, to be technically correct, minimum 7.1ms, average 7.4ms, maximum 7.8ms. That is what an A&A line should look like - though the base latency will depend on line type and interleaving and so on, we are not adding any extra by way of congestion in our network.

And that is what we aim for - not being the bottleneck. That is why we have upgraded some backhaul links to 10Gb/s.

P.S. As someone will ask - the usage dots are different but similar. This is because it is bonded lines but the actual throughput depends on the line speed, and the lines are slightly different sync speeds meaning that the throughput of each will be biased slightly to match.

P.P.S This is the same line (as the first one above) last night after the upgrade, as predicted 7ms all the way down...


Giving the police more power? (#IPBill/#IPAct)

With the Investigatory Powers Bill reaching final stages I expect to have some detailed comments by next week, once we have the final text of the Act.

However, we had a rather odd email today from the police, a Cyber Distribution & Prevention Team, no less. You would hope they have some clue, but their email shows drastic lack of clue...

We do occasionally get requests, usually under RIPA, usually related to telephony, and almost never actually correct. Typical errors are:-
  • Not one of our numbers!
  • Number too short
  • Number too long
But also issues they would not know, but still a nuisance:-
  • Number simply not in use, and hence must just be spoofed CLI
  • Number leased to another telco
This request was different. It was a request to suspend a "line" which is in fact a VoIP service. But it shows some serious lack of clue here.

Firstly they are entirely going on the CLI. They have not attempted to trace the source of the calls via the telephone network in any way (else they would not have got to us as it is part of a block leased to another telco). But even though only based on CLI they are assuming the CLI is genuine. No hint that they know it could be otherwise, and indeed, asking us to suspend a line based only on CLI provides a means to "attack" a victim by using their CLI for something iffy and getting the police to get the victims line suspended! Our reply refers them to the wikipedia article on spoofing CLI.

Also, it is marked "Classification: PROTECT - INTERNAL USE ONLY" yet they have sent it externally to us. Ooops.

Then they explain "Attempts to contact the line have not been undertaken to prevent jeopardising any ongoing or potential investigation that may follow." Hang on?!? What would we say to a customer (if it was our customer) when we suspend them that would not jeopardise ongoing or potential investigation - seriously - suspended line is going to be a tad noticeable.

Then there is the actual request "We request that you consider suspending this line as soon as possible to prevent further harm to members of the public occurring and for a minimum period of 12 months." which I am not sure I understand. This is just a number. If we suspend it at all, the end user can have a new number to make calls within seconds, so not going to stop him, just alert him that they are on to him. Also it means the fraudster is now using a new number which nobody is blocking or watching out for, so actually that increases harm - by always using the same number one can alert people "don't accept calls from X" if that was a sane thing to do when considering CLI spoofing anyway. It also makes handling the reports they are getting easier to collate as they know it is the same person. But also, why suspend for 12 months? How does that help?

But then we have the fraud itself - a simple matter of someone calling an claiming to be something official (I am not giving details here), but a key point is the victim is then asked for "bank details" to pay something they have been convinced is due. As far as I know that only allows a direct debit, and a direct debit can always be reversed. So either the victims are getting really bad advice and not getting the DD reversed and their money back, or the fraudster is particularly stupid. The email is quite specific, and says the other trick is to ask the victim to get "I tune"[sic] vouchers and read the number, but that again makes no sense as this is someone claiming to be from an official body which nobody would be stupid enough to think could be paid in iTunes vouchers. What they are saying here really makes no sense. I suspect the fraudster is smarter than they are saying and the police are recording the details totally wrongly.

I hope they catch the fraudster, but in this case there is nothing we can do to assist further - as the calls are not through us or from one of our customers.

These are the people we want to give access to details of every web site visited by every person in the UK. Seriously?

NOTSCO (Not TOTSCO) One Touch Switching test platform (now launched)

I posted about how inept TOTSCO seem to be, and the call today with them was no improvement. It seems they have test stages... A "simul...