2012-03-04

Automatic IPv6

This generation of FireBrick has had IPv6 built in in from scratch. It has proved very useful for customers wanting IPv6 on their network.

We put in the PPPoE client code some time ago, and now our preferred configuration for FireBricks in an office is to use PPPoE. This works directly on FTTC/FTTP and with DSL modem or bridges.

The IPv6 works nicely. You configure the LAN with the IPv6 blocks that we (or your ISP) are routing to you.

As we got DSL routers that handled IPv6 we added DHCPv6 serving to the LNS so delegate the IPv6 prefix to the router. The router then broadcasts router announcements for the delegated prefix on the LAN. Works well.

DHCPv6 is a horrid way to do it, as I have said before. A broadcast protocol on a point to point link; using a no routable link local address on a link with source filtering; Using IP level protocol on routers where it can hit firewall rules. Also it is not associated with the link state or established as part of bringing up the link itself. All very messy. But we have it working on the LNS.

Today I have been putting the final piece of the puzzle in place with DHCPv6 client in the PPPoE code so that am FB2500 and FB2700 can get a delegated prefix automatically. I have it mostly working this morning, but need a few more bit and some testing. I have issued an alpha with what I have now anyway. I still need to handle the PPPoE link address (IA NA) and DNS servers. I am not sure the RA timing is quite right on the delegated prefixes yet either - needs testing.

In principle it is simple, but as ever there are challenges. The main one is that FireBricks are a tad flexible. Even the FB2500 can have hundreds of interfaces with ports and VLAN combinations - so which LANs do we get prefixes for exactly? The FireBrick can also handle multiple PPPoE connections.

My main objective is that it is simple for the typical case. In fact I have been working on making a factory reset FireBrick use a default login and password to get on to the Internet and in the process to get an IPv6 prefix for the LAN. But I also want it to be flexible as well.

What I have come up with is a setting on the PPPoE config allowing you to say which interfaces you want a prefix for. The PPPoE connection then requests the prefixes you have asked for and allocates them to those interfaces. That allows different PPPoE links to provide prefixes for different interfaces in a variety of combinations. Obviously if multiple PPPoE links are providing the same prefixes to the same interfaces (as with our typical bonded lines) then that works too - with each DHCPv6 response updating the prefix to the latest expiry time. The default is, however, to delegate for all interfaces on the same routing table that are not RA clients - so a sensible default. You can also set to an empty list so no prefix delegation is done.

I think this makes sense. I'll probably do more updates tomorrow to finish it off.

P.S. Now does the IPv6 WAN address, and (in theory) DNS servers. Just testing now.

2012-03-03

Slicker PPP

I have spent all morning tweaking PPP.

It is not that there was a problem or anything, but as I was working on the whole area to get a DHCPv6 client in the FireBrick PPPoE side, I was looking at it.

I had not realised we were taking a few seconds to fully negotiate.

The problem was that we did not send the next packet immediately on a state change for every normal event - we waited for a one second poll to check for timeouts and/or next thing to send.

In practice this worked well, but introduced a couple of places where there was an extra delay of up to a second in the negotiation.

On our LNS side this did not happen as the authentication is proxied which eliminated one stall leaving only the IPv6CP negotiation. The result is IPv6 took up to a second longer to come up, which is probably why nobody noticed. After all I have customers that would have complained if PPP was not working as well as it could (some are involved in the pppd stack on linux).

Where it really was more noticeable was using the FB2500/FB2700 as a client, e.g. on FTTC. This had two or three places that would stall up to a second.

Well, the changes were mostly simple. I already handled the logic of a received PPP message means sensing one back immediately - either as a reply or the next message to send.

The startup on PPPoE completing was not immediately sending the first PPP packet, so that was one stall easily fixed.

I had missed one step where CHAP was accepted - I should immediately go on to start IPCP at that point, so that was another easy fix.

The less easy fix was when we sent an ACK, and that completed a stage. We do IPCP before IPV6CP (though we could do in parallel I guess) and when we sent IPCP ACK we did not also start IPV6CP. That meant the rx packet processing had to cope with two packets to send as a result instead of one. Again, not a big change.

The end result is that on the bench I have PPPoE and PPP negotiating all the way in under 50ms, and in fact all within a second of boot up.

Next test - FTTC line. How fast can we be on line from power up on a real connection. I'll add details when we have tested that :-)

Data we have so far :-

1. BE line using vigor modem, PPP kill from our end, loss of 2.5 seconds of pings at 0.1 second intervals.
2. More tweaking and above managed a PPP kill where we lost 1 second of pings from outside of our network.
3. A software reload of an FB2500 on a vigor on a 20CN line - 2 second downtime


2012-03-02

The shortfalls

People asked what "shortfalls" we had that justify a "good will" award of £500 even though it is agreed we are not in breach of contract.

Whilst I am not sure we can publish their initial decision, I am happy to publish some of our comments to them and a summary of the shortfalls.

The first part of our reply is :-
The claimant has stated to the court in their defence of Reading case XXXXXXX that they referred the matter to the Ombudsman service (Communications) on 9th January 2012. This agrees with the details of complaint statement you sent us. Unless you are accusing the claimant of perjury then this means that the matter is outside the 9 month limit defined in 9.1(c). As there are no exceptional reasons to justify the delay this puts the Ombudsman service in breach of terms with us by accepting this case. We require you to drop the case and refund the case fee. If you do not do so within 30 days we will start proceedings in the county court against you without further notice. We will also claim compensation for our time handling this case.
But the apparent shortfalls are as follows :-
  1. An installed line without annex M is too slow. Annex M was added afterwards. Note we don't guarantee line speeds so not actually a shortfall as we did not agree a target and fall short of it.
  2. Some of the dates we advised for likely install of orders changed and were inconsistent. We basically passed on what details we had as orders progressed, and did not guarantee the dates. Apparently this is bad. We have asked if they would prefer we don't pass on information we get in case it is ever inconsistent and if that would be better customer service some how.
  3. We apparently did not reply to an email - given the case file was something like 500 pages of email I am not surprised some were missed. I am not happy about it, and more than happy to apologise for losing an email. It did not change the outcome of anything or delay anything. And as we never guaranteed any response times for emails, not a shortfall. We did reply to their follow up email. Note that the ombudsman service clearly think not replying to every email is perfectly acceptable as that is what they do to us - in fact not replying to most emails we have sent.
  4. We finally managed to get all 4 lines with annex M installed in time for the event. Even the ombudsman's decision starts by saying they wanted service "in time for the Royal Wedding", which we managed. Apparently meeting the stated deadline even though not contractually required to is a "shortfall".
  5. We billed for services from the date they were supplied. Somehow this is an error and a shortfall. We are at a loss on that one.
  6. One more case later of apparently not replying to an email promptly.
So, that is what justifies £500 "good will" award, and also writing off around £700 of unpaid invoices.

Our main issues are :-
  1. They accepted a case that was outside the 9 month limit; was already resolved (in that the £6.69 claimed had been settled); was already resolved in that the proposed resolution of "let out of contract without penalty" had already been done; and that was a frivolous case (£6.69).
  2. When they took the case they failed to look for a mutually acceptable resolution and ignored the statement from the customer to them that they would be happy with being let out of contract without penalty and also our statement to them that we agreed that.
  3. They invented targets we did not offer or agree and that the customer did not even ask for, and then penalise us for not meeting them.
  4. The ignore contract law which is that there is no requirement for us to compensate someone where we are not in breach of contract
  5. The ignore the contract which limits what we are liable for and make an award for much more
  6. They make a punitive award even though not allowed
  7. The invent a figure of £500 with no explanation and call it good will, which is  contradiction if forced to pay it.
  8. They insist services from 31st May be credited even though the customer asked for them, used them, and has not disputed them. We have an email in June stating "we want to continue with the service". They do not explain why they are saying this. They also say to waive all late payment charges even though invoices are still not paid even now.
  9. They considered and acted on numerous issues with actually post date the complaint date. i.e. This case is to resolve an unresolved complaint that was made on 17th March 2011, but many of the shortfalls and remedies appear to relate to things later than that date which were never part of that complaint and have not even been part of a later complaint.
The only point I am even vaguely happy about is that we are asked to apologise for the shortfalls. I won't do that as they were not shortfalls. I am happy to apologise for some emails not being replied to promptly. I am even happy to apologise that it took longer than expected, even though we met their target. I am pretty sure we have already apologised a few times. We gave a £272.17 good will credit as well, so I think we have been more than fair as we did not have to do that.

So, an 11 page formal response is being posted to them today, including the threat of legal action against the ombudsman. We'll see what happens next.

Actually, just to condense facts to the key points here...
  • Customers asks A&A for complicated service (4 line uplink annex M bonded) "in time for the Royal Wedding"
  • A&A delivers exactly what is wanted "in time for the Royal Wedding" and customer is happy with what was delivered so much so he wants to continue the service after the event.
  • Customer thinks A&A over charged £6.69
  • A&A credit customer £272.17
Err, right, and this is a case for ADR why exactly?

Event (not so) brite

So, first time using eventbrite. Seems simple enough.

Small snags so far...

The control pages - event in Bracknell
The event page - some place south of Sweden!

Also, can't see how to set as BST for the time of the event not GMT.

P.S. The location magically fixed itself!

P.P.S BST is in the time zone list, right down the bottom and nowhere near GMT+01:00 in the list!

2012-03-01

AAISPISSUP 31st March. Bracknell

We will clear the training room and get some drinks and nibbles in. It is a Saturday so we plan to have an afternoon event but no doubt some will stay on in to the evening and possibly involve pizza.

It is a chance to meet some of the staff in an informal atmosphere, meet the orc, and meet me.

It will be the last day of our 15th year as Andrews & Arnold Ltd. Quite a milestone.

A&A Offices in Bracknell, from 2pm (BST), 31st March 2012

Book a place
Key signing

Losing the race

Arrrg! All this work, and worrying about ADR, mean I am now losing a race to get a character to level 85 against my daughter's partner. It is getting a bit close now with him on 83 and myself on 82.

I wonder if I can sue the ombudsman for the stress caused by this :-)

<--- me on a bad day - and you wonder why we do lunar billing.

Clocks

Some time geeks (should I say Time Lords) checked out my clocks. Seems they are impressed, saying sub microsecond. I have spent all day tryi...