No clue

We really are hitting a problem with the "Powers that be" having no clue about the way any technology works.

I have commented as to how the "Snooper's charter" is so out of date already, but it gets worse.

I was chatting to someone at LINX who has been at some of the ICANN/IANA type meetings covering the top level operation of "the Internet". It is a complex and worrying arrangement where ultimately everything, including domain names and IP addresses, ultimately gets authority from IANA which is part of ICANN which has a contract from the US government (which is all changing, maybe).

We already see some crazy steps in verification of domain owner details, and the anecdote I was told was positively scary. Apparently, at such meetings, there are people from law enforcement like Interpol. They want things like "the ability to take an IP address off the Internet" or even remove whole ranges.

But one of the scariest and amusing comments was that they apparently were confused over verifying a phone number against an address. Surely one can check the phone book? A comment was made that "well, you can take your phone number with you when you move" and apparently the Interpol person was "No you can't?!". They seemingly had no idea that a phone number was not physically tied to an address for ever,

Even 50 years ago when phone lines and numbers were more physically associated using elect-mechanical exchanges, one could move house within a town and take a phone number or arrange an out-of-area line. These days you can do it and move to anywhere (in the world) if you want. Many phone numbers have no physical presence in the first place (as with all of our VoIP numbers). They are just an over-the-top logical service to convey voice. Why would they have "an address"?

What is worrying is that people in power to influence policy that could affect us all are so totally and utterly clueless. And for a change I am not even picking on David Cameron.

But just considering UK law, and EU law, there needs to be a massive shake up. The whole concept of communications and communications providers needs redefining to fit with reality now, and in the future. The current definitions simply don't work, and it is only going to get more complex.

There is a need for some laws and regulations, I am sure, but the current laws do not fit and do not work.

The one RIPA request we had that was actually related to some mis-use of something by a customer was one where we sent a member of staff to court for his trial, an he ended up spending most of his time explaining how things worked to both prosecution and defence. Neither of them had any clue that you could route a geographic phone number to a mobile phone and/or a SIP phone or PABX and even a landline or have them go to them all at once or have calls from somewhere present any number you like. In the end the case, of someone making stupid bomb threats (allegedly) ,was thrown out. We got bugger all for his time, and in future will "offer" consultancy services at commercial rates and not offer to be an witness. Lesson learned.

The endless possibilities of layers on layers of communications with levels of encryption and different means of addressing are just beyond anything anyone in parliament or the EU has any clue.

How do we fix this?

Democracy

The basic notion of a democratic society is that "the people" make a decision on who shall govern them.

Unfortunately democracy has a heck of a lot of flaws. Not least of which is that, even with only two choices, and even if everyone votes, and all votes counted, you could still have 49.999% of people disagreeing with the elected government. But democracy has far greater flaws than that.

The alternative idea here is not really to propose an answer but perhaps start a discussion...

This came out of a discussion last night at a Conservative Party Dinner, to which I was invited as a guest of one of the people with a table. I have many issues with some conservative policies, but it was an interesting evening.

A simple concept was discussed at the table - the idea that everyone gets a vote, but that those votes are weighted somehow. Of course, before you even get to such a concept you have to have a way to make every vote count, by some sort of proportional representation - and even that is complex.

One of the reasons to even consider this is the basic concept that people are not created equal, and that even people who are in fact equally intelligent may not care to spend the time to consider the issues and make a rational decision or have the necessary knowledge or experience to do so, when voting. The vast majority of people are essentially sheep, following friends, relatives, and media in the way they vote, and not considering the actual issues.

I am not trying to be unfair here - for the most part politics is not a useful way for anyone to spend their time, and I too lack the experience or motivation to have a rational viewpoint on a huge number of political issues. There are many issues of government that 99% of people will simply not have to consider.

But the idea of a weighted vote would create interesting problems - the criteria for deciding the weighting would be both tricky and exploitable. If one made it that those with a degree had higher weighting, then there would be a million scams to get people a degree if they will vote the right way.

So, sadly, whilst an interesting idea I wonder if it could every be workable.

There is one idea I did have though - a simple one - allow people to vote with a weighting of their choice. i.e people that actually feel strongly to vote for one party or candidate can say so, and their vote count for more than those that don't really mind either way.

In a way we have that now - you can vote with a weighting of 0 or 1 by whether you bother to vote - so how about having a system where you can say your vote, and state a weighting 0 to 1 (or 1% to 100%). Indeed, knowing you can cross out the default 10% weighting on the form may be an intelligence test in itself.

Could that work?

Update: I did ponder another daft idea: What if it was 1st and 2nd past the post in each constituency (you'd have to make constituencies bigger to have same number of MPs), but the MP's vote in parliament counted based on number of votes they got. If you combine that with a single transferrable vote in the constituencies (so least popular candidates votes transfer to 2nd choice and so on until only 2 remain) you end up with the vast majority of people having someone local that represents them in parliament and is also someone they voted for, but a degree of proportional representation and every vote counting.

TPS actually respond

I asked for my SIP URIs to be included in the TPS...

The first time I got a response of "we can't but we'll ask OFCOM", and OFCOM answered very much "TPS can only do digits".

But to my surprise, having written again (from work this time) I actually got a more comprehensive reply. This may be because I complained to the ICO, not sure.

Basically he is saying the definition of "Number" (as in "Telephone number") in the Communications Act 2003 is not carried across to the Privacy and Electronic Communications (EC Directive) Regulations 2003. So they only accept "numbers" in the traditional sense.

I may write back, as, indeed, he is right, there is no definition of "number" in the PECR, so one would indeed expect it to be "number" in the traditional sense... But "registration number", "vehicle identification number", "national insurance number", "serial number", all of which allow things other than just digits, so there is not really much "tradition" in numbers only being digits, certainly not in law.

It also seems to me that the TPS has to list numbers "allocated to a line", and "line" covers things that perform the function of a line. My SIP phone here clearly does that and the "number" allocated by my telco to that line is my SIP URI - i.e. it is my telco that has allocated the SIP URI as a "number", so it counts.

He does say that he will ensure the proper regulatory treatment of SIP addresses is brought to the attention of the relevant authorities in the future. It seems mean having a go now - as he has tried, but I'll give it a go anyway...

Update: I have replied to him :-

Update: The TPS site won't accept my Iridium mobile number, so I am writing to them to add that. That is only digits, so they should accept it. Of course, I know that no junk caller would be mad enough to ever call it, but I am entitled to have it included in the register I believe.

OFCOM drop the ball?

OFCOM have created a new system for migrating broadband lines, harmonising it with phone lines. The system is called Gaining Provider Led (GPL) because you simply order service with a new provider - no need for a Migration Code (MAC).

Dangerous!

Many of us think this is a bad idea. Broadband is not like electricity or gas, where an incorrect migration does not disrupt the actual service, it simply creates billing differences which can always be sorted later. With broadband you could find you inter office leased line that works using Ethernet over FTTC circuits suddenly changes to some generic residential broadband service, taking 10 days to fix, simply because someone put a wrong digit on their order with an ISP and somehow your organisation missed the "Notice" that was sent to the account department in another office. There are real danger of some ADR or suing if this were to happen. There is also the fact that it will now take 10 days, not 5, to migrate services, which seems a backwards step.

Details

However, looking at this in practical terms, as it all kicks off in June, we did consider that it should be relatively easy:-

• Orders to BT are the same, except we don't send a MAC, simple enough
• Notices from BT are the same, as we get them for migrate-out using a MAC, some fields will be a tad different, but we already get these notices, we just need to email the customer
• Cancelling a migrate-out is something we could do already, but don't have to as a MAC is used. We'd have to integrate it a bit more in to our systems and make it easy for customer to cancel.

Of course we expected a few other changes:-

• Update details of how to migrate on web site
• Change order form not to ask for a MAC
• No longer offer people option of getting a MAC

Not so simple!

Unfortunately, now we are working on some of the details, it is far from simple.

For a start, we have a couple of places where we have to send letters which then have a detailed list of things we have to say. Most are simple, but not all. For example, our billing system (priceless) can work out what to charge people for a minimum term when it picks up that a line has ceased and raises a bill - easy. However, the broadband management system which deals with these messages (clueless) does not have billing/pricing data, so for it to tell people the exact amount they will pay for the remainder of a minimum term (Early Termination Charge) means it needs to somehow ask the billing system, but not actually raise a bill (not something the billing system usually does). So not as simple as it sounds, so that will be fun.

Also, these letters need to go by post unless the customer has agreed we contact by email. Well, all of our customers have agreed this. It is in our terms and very clear, and we email invoices and DD notices and so on. But OFCOM are saying that being in our terms is not good enough, we have to have explicit consent from the customer for this. Well, that is easy for new orders, but are we going to have to contact all existing customers to get this explicit consent - that will be a pain for us and them. We're asking OFCOM to confirm.

We also can't talk to the customer to even ask why they are leaving as that could be seen as a reactive save (retentions call).

Anyway, obviously. we are working hard to ensure we follow all these crazy rules to the letter by the time the new process starts in a few months...

---

But what of OFCOM dropping the ball?

Well, this is where it gets interesting, and we hopefully get reaction from BT and OFCOM today on this. These rules are imposed by OFCOM General Conditions. GC22 covers the migration. 22.30(s) defines "End-User" such that it excludes large businesses (those with more than 10 people doing work for them). 22.30(n) defines "Customer" as an "End-User", so that excludes large businesses. 22.30(j) defines "Communications Provider Migration" as an "End-User" or "Customer" changing provider, so that excludes large businesses. Even 22.30(gg) defining "Migration" as a word, excludes large businesses by use of "End-User" or "Customer". Pretty much all of GC22 uses these terms which exclude large businesses. Even 22.25 which is a catch all for cases not using BT, etc, and basically says ISPs have to work together to facilitate a migration fairly/quickly uses the "Migrate" definition which excludes large businesses.

The whole of the migration process does not apply to large businesses!

This is new. This is different. This is a surprise. I am happy that such a dangerous system does not apply to large businesses, but there is no alternative. The old MAC based system is stopping, so that does not apply either.

Large businesses may simply not be able to migrate phone line or broadband services now. Obviously there will be cases where providers co-operate and do this, using the underlying BT/carrier mechanisms, the same as small businesses, but they don't have to. If a large business asks us to take over a line, the losing provider could just cancel the migration as it is not covered by the migration process at all.

Obviously if I have missed a step here I hope someone will tell me, but I can't see it.

Is this what you meant OFCOM? If not, please let us know when you consult on a new version of GC22 to fix this as we'll be happy to comment.

I'll update if I get responses from BT and OFCOM on this.

Update: No word from OFCOM yet. BT have a relatively easy job of the various messages and process changes, but are not sure of the "point of no return" and cancellations at the last minute. They also get the impression that OFCOM are being a tad wooly on several points. They were interested in the large businesses issue and they too have asked OFCOM for comment.

Update: OFCOM have apparently confirmed to BT that the new migration process does nt cover larger businesses.

OFCOM think they are above the law?

OFCOM have admitted they did not send a written reminder of our corporate TPS numbers (and neither did TPS) as required by section 26(2A) The Privacy and Electronic Communications (EC Directive) Regulations 2003 as modified by The Privacy and Electronic Communications (EC Directive) (Amendment) Regulations 2004... phew that is a long sentence.

So they have acted unlawfully, and section 30 allows anyone suffering damages from a breach by anyone under the regulations to claim damages.

Well, having had TPS drop our numbers silently before, obviously, I had to find out if we were still listed having not had the reminder. This involved recordered delivery letters to TPS and OFCOM, which apart from my time, also cost in paper, ink, envelope and postage to the tune of several pounds.

OFCOM are adamant that they do not have to pay any damages.

Are OFCOM really claiming that they are above the law? We will see. I have formally requested that the ICO exercise their enforcement action against OFCOM, and I have sent OFCOM a notice before action.

Congestion case study

The BT kind, not the sudafed kind...

I have not really needed to talk of backhaul congestion for some time. Many years ago, when BT first had congestion in their network they did not appear to have proper means to plan and manage the capacity. Thanks to our work constantly identifying congested links, they do now have departments to handle this. We've worked closely with BT on these issues over the years in a genuine effort to help them solve their problems and provide a quality service to all ISPs. Over the years this has been a bit of a roller coaster, and occasionally there have been problems for many months (such as when BT's BRAS back haul links had to be upgraded to 10G). Generally things have been OK for quite a while until a few months ago. Oddly we are seeing quite a few issues at the moment that are taking some time to get fixed, but BT are working on it.

The main way we can see congestion is because we have very good monitoring - an LCP echo every second on every line allows us to see packet loss and latency clearly. We then correlate trends over exchanges and BRASs and carriers to identify congestion before customers even need to contact us.

However, there is still the old 20CN ATM based network in BT and many 20CN only exchanges. These are still growing with more demand for bandwidth from existing lines, and more lines being added. This can also result in congestion. Obviously, over time, these are being upgraded to either 21CN ADSL or FTTC (or both).

Over the last two weeks I have done a bit of a case study on one customer on the SOUTH RAUCEBY exchange. Here are my findings...

Seeing the congestion

The first concern is that we cannot see the congestion any more! The LCP echo are not showing loss or latency even when the exchange has a lot of congestion. We think this is a change BT made many years ago to prioritise the LCP echo. This may have been to ensure routers do not drop the link due to a lost LCP echo/reply, but it could also be to make our graphs "look good" I suppose. Thankfully 20CN is a minority now, but we do have to rely on customers telling us 20CN congestion issues.

Overloaded link

This particular customer contacted us some time ago advising that there was congestion, and we contacted BT. As it happens, BT said they had just set up another DSLAM (or another shelf in a DSLAM, I am not sure), and they would move this customer over to that. This solved the problem for our customer, great. However, some months later, he once again has congestion. This does rather suggest they have issues with monitoring the links, upgrading links, and planning rules, one way or another.

This is an example :-

This shows the upload (red) and download (green). This is an attempt to fill the line, which should be able to get close to 7Mb/s of IP throughput but is in fact only getting 1Mb/s at best.

What can be done?

Sadly, we have still not managed to get BT to do anything to fix this yet - it is often an uphill struggle and they may even deny there is an issue. So we considered some alternatives. There are two things we considered. One thing to try is to order "premium" on the 20CN service. This offers a higher upload speed and also elevated weighting in the network.

However, we also had another cunning plan. We lent the customer a FireBrick FB2700. We then told our end to mark all of the IP packets as if they were LCP. This is a feature in the FireBrick which was added to work around problems with a faulty DSLAM that refused to allow IPv6 PPP packets, but we had a hunch it may help here. The fact that our LCP echo always seemed fine seems to suggest that BT prioritise LCP traffic. So we gave it a try.

This is the result :-

As you can see, the download is close to the line (which shows the BRAS rate that should be possible). It is not 100% perfect (the line is a bit wobbly), but very close, and massively better than before.

Does "premium" help?

We also added premium to see if that would help, and it does (as expected). This is an example of premium, with and without the IP over LCP being used.

Just before 20:00 is premium using normal PPP coding for IP packets. After 20:00 is premium and LCP coding for IP packets. As you can see, after 20:00 there is a solid line at the limit - the best performance yet. The graph is a log scale, so it may not be obvious, but without LCP marking the line is achieving approximately half the full speed.

Conclusion

Premium improved from under 1Mb/s to around 3.5Mb/s. However marking IP as LCP improved both premium and non premium to full line speed (with premium is slightly better).

Will BT fix the problem?

We hope so - we will, of course, continue to pursue BT over this congestion. However, our customer has three lines, and all of them can now hit the full line rate over around 7Mb/s at once when he is downloading. He has done speed tests showing nearly 20Mb/s over the three lines. Considering how poor the performance was previously, it seems likely that when our customer is downloading, everyone else on the same VP backhaul in SOUTH RAUCEBY will probably have totally unusable Internet. With any luck they will complain to their ISP (probably BT retail) and help get the backhaul fixed properly.

I'd like to thank domb for all his help on this.

Tech note: The customer is considering patching pppd. What we actually do is simply mark an IPv4 or IPv6 packet as LCP providing the packet starts 4X for IPv4 and 6X for IPv6. At the receiving end an LCP packet starting 4X or 6X is assumed to be IPv4 or IPv6 respectively, so no extra overhead. Genuine LCP codes do not get anywhere near as high as 40. Maybe we should do an RFC :-)

When is junk mail not junk mail?

As I reported recently, a company selling training courses on email marketing emailed my titanic email address (for which I am an individual subscriber). Yes, ironic isn't it. What is interesting is that he has engaged in some lengthy debate on the matter during this week (something to do whilst I'm off sick). We have finally agreed on a £75 settlement. But some of the points that have been raised are interesting.

Even though a UK business, he had not provided the details of his legal entity on the web site or emails (as required by the Companies Act 2006). This is a pain as it makes suing him somewhat harder. He also uses a correspondance address in London, not a real office. I was expecting to hear nothing and hence be able to do nothing. To my surprise he did reply. Even so, I have reported this to Trading Standards, and they are in fact investigating. It seems that they are a partnership, and the best I got was an initial and surname for each of the two partners and no other address, which, sadly, seems to comply with the rules. The difficulty with actually suing and enforcing a judgement is one reason I ended up agreeing a settlement. Fortunately most spam like this is from UK limited companies providing a proper company number in the initial email.

Another interesting issue is that he had emailed 7 times before. He claims not to have received any of the email replies I had sent, and suggests that my IP is on some sort of black list. Needless to say no emails bounced. He initially offered £25, and I said fine, and £25 for each of the other 7 makes £200. But he wanted to offer £25 for all emails. I pointed out my letter before action clearly related to just one email and that is all I would be taking to court this time. Depending on the outcome I could take action for the other 7 at a later date. Maybe, in hindsight, I should have got settlement of the £25 for this one email first and then gone after him for the other 7.

At every stage he disputes any liability, and keeps saying that he complies with all regulations. I have tried pointing out that this is factually wrong. There is no doubt he sent an unsolicited marketing email to an individual subscriber, which means he did not comply - matter of fact. Interestingly he claims that "subject to certain requirements they do permit us to legally email what we reasonably understand to be business email addresses". When I asked him to back that up, he referenced an ICO guideline which did not actually say that at all. It seems to me he genuinely thought that he was within the regulations if he believed the email address was not an individual subscriber, and if he removed the email address when told otherwise. Reading the regulations I am very much of the opinion that this is not the case, and that even one such email, regardless of intent, is a breach of the regulations.

He also said that they are trying to get clarification from the ICO on the "ambiguity" over individual subscribers and corporate subscribers. I guess, if in the business of email marketing he might like to clarify that. I don't think it is ambiguous, just something the sender cannot know. Even so he raises one interesting point, which I was a tad unclear of. Is the subscriber in "individual subscriber" the person that contacts for the "email service", or the person that contracts for the "line" over which the email is delivered? However, I think I have worked it out now...

The PECR states: “subscriber” means a person who is a party to a contract with a provider of public electronic communications services for the supply of such services; “electronic communications service” has the meaning given by section 32 of the Communications Act 2003;

Now, looking at the section 32 of the comms act: “electronic communications service” means a service consisting in, or having as its principal feature, the conveyance by means of an electronic communications network of signals, except in so far as it is a content service.

Now, an "electronic communications network" is a transmission system for the conveyance, by the use of electrical, magnetic or electro-magnetic energy, of signals of any description

A signal is (a) anything comprising speech, music, sounds, visual images or communications or data of any description; and (b) signals serving for the impartation of anything between persons, between a person and a thing or between things, or for the actuation or control of apparatus.

An email service does indeed use such a network, and I think signal covers an email. So it seems to me that the subscriber in relation to an unsolicited marketing email is the subscriber to the email service.

Anyway, once again, this means no case in front of an actual judge. We'll get there eventually.

Update: Interesting point from the comments, I missed the "public" part in the above, which is: “public electronic communications service” means any electronic communications service that is provided so as to be available for use by members of the public; 

What is interesting is the "use" part. If you run a mail server for a domain, you are allowing members of the public to "use" that service by allowing members of the public to send you email. It does not say you have to allow members of the public to "subscribe" to the service. It uses the word "use". So to my mind, pretty much any mail server is a "public" electronic communications service.