Thursday, 16 November 2017

Pushing the limits

We are deploying some new LNSs at A&A, three of them so increasing the eight live LNSs to eleven. This will happen over the next few days.

Why? Well, we are hitting some amazing levels of traffic - we seem to have actually exceeded 1Mb/s average peak usage per customer.

This is a metric which we discovered is used by the likes of BT Wholesale, and a few years ago, at over 100kb/s we were a really high usage ISP on their network - the highest at the time.

Basically you look at the peak usage, which for us is now in the evening, and divide by number of customers, simple as that. Bear in mind, at that peak time, pretty much every one of our business lines (which is a lot of our customers) will be idle.

Even so, that average is getting to the 1Mb/s. That is amazing. I am shocked. We have multiple 10G links to peers, and multiple 10G links to one carrier, and extra 1G links going in right now to another carrier, and more LNSs. All in aid of not being the bottleneck.

This really is the netflix generation taking off, and it is a challenge for ISPs. We are coping, and we are making sure we stick to our "not the bottleneck" aims. But it means some quick work to extend the capacity with staff on site today installing new LNS hardware.

We know that Talk Talk are working on expanding their network, no doubt seeing the same increases, and we are working with them to make the best use of that new network as soon as possible. Thanks to those customers testing things with us.

It seems not long ago the maximum a customer could manage, a far cry from average, was 64k ISDN. How times change.

With new links to BT going in, we expect to make tariff changes over the coming weeks to make "terabyte" just another level on all of the Home and SoHo packages, with easy regrades, and balancing of traffic over multiple lines. (Not for 20CN, sorry)

If you want Internet access that works, and keeps up with the ever increasing usage, you know where to come :-)

6 comments:

  1. Heh, I remember in about 2000, being in the Army and being the first one in the regiment to get consent to get a phone line installed in his barrack room. I then upgraded it to home highway, 64kbit was bliss. I rarely used 128kbit due to call charges, but it paid for itself as I charged my mates to share it via ethernet... there were about 5 of us using that connection, and it still seemed fast until someone opened Kazaa or Limewire.

    How times have changed.

    ReplyDelete
  2. When I first gained access to the Internet in the early 90s via my employer, they had a 64k link to, I think, UCL and there was only a 64 k satellite link to the US. Those were the days of Gopher and DEC's ftpmail server in California - software source packages arriving overnight as 60 or 70 uuencoded e-mails! Sorry, this sounds a bit like a 4 Yorkshiremen sketch...

    ReplyDelete
  3. Netflix are normally very keen to put their CDN boxes on ISPs internal networks - Sky has 18 Netflix CDNs on their network last time I looked.

    Perhaps you should consider contacting them? After all they can only say no and the external bandwidth saved is fairly staggering (Sky wouldn't be offering a competitor "hosting" if it didn't make lots of financial sense).

    ReplyDelete
    Replies
    1. There are several stages for things like that, and we are well aware of that. The transit and peering are generally not the issue or cost but that will happen eventually.

      Delete
    2. The problem is switching to a local cache only saves on peering bandwidth (LoNAP in this case, IIRC?), which runs £300 per 10G port per month - no saving on the BT or TalkTalk backhaul bandwidth, which is dramatically more expensive!

      For Sky, yes, it's slightly cheaper to have the boxes local than on the far end of a LoNAP link or private peering cross-connect, so makes sense - but the big cost savings are to Netflix, not the ISP.

      Delete
    3. Exactly.

      For arguments sake lets say A+A do 5Gbps of traffic 95%ile from Netflix, making them eligible for caches (probably they don't yet).

      The cost of a LINX PI intersite pair is £200/month which can be run at 10G, 40G, 100G or in future higher. LONAP 10G will be £300/month from January plus around £25/month for the XC (assuming TH). The cost of hosting just one 600W 4u cache in Telehouse North or HEX is likely to be well, around £300/month.
      Netflix usually want you to take two for redundancy and additional content so really you are talking £600/month. A Private 10G link in Telehouse is going to be <=£200/month. It's more expensive. Why bother, you are just paying Netflixes hosting fees for them! Why should you? That is their job :)

      Delete