Wednesday, 17 July 2013

You canna' change the laws of physics, cap'n

So, we are playing with a satellite line from twoway.

They seem to have rather quietly created a rather impressive service. For around £25/month for the lowest residential tariff you can get 20Mb/s down and 6Mb/s up.

It was easy to set up, in fact, after playing in the car park yesterday, today I simply plonked it on the gravel by the office and turned it on and it worked first time before I had even aligned it. A bit of tweaking got the SNR from 2 to 10dB, but it was that easy.

Plug a laptop in, download a file, and see and average over 20Mb/s transfer speed. Wow!

But, those damn physics police insist that you can't go faster than the speed of light, bugger. The round trip pings are around 700ms. Bear in mind that it includes switching delays and the up-link being in France (we think).

This is a problem. TCP does not do 20Mb/s over 700ms. Do the sums. It means 1.7MB of transmit buffer at the sender per TCP session. I checked, and the config on a typical linux box was 128kB. You can put it up, and use window scaling (TCP only does 64KB without it) but that would mean reconfiguring every server on the internet to which you wish to communicate.

The 700ms is a big issue. Consider the normal "go to a web page", which includes a DNS exchange, an initial SYN/SYN+ACK exchange, and then several stages of "slow start congestion window" on TCP to get to filling its tx buffers before hitting a limit of maybe 1.5Mb/s.

So, the service must be doing something special. What you can do is fake the TCP session at both ends, telling the sending server that the packets have, in fact, arrived at the laptop, when they have not. You then send them (reliably) over the satellite link, with your own acks and retries somehow. It means buffering in the network, and is not very nice as the sender gets a false sense of security, but for most practical purposes that is not important. It allows the transmission over the high latency link to be managed in a different way that is fine tuned to allow for the high latency, presumably with selective resends and so on.

Of course, as soon as we get clever and tunnel traffic over the link, we lose that. Hence getting a little over a 1.4Mb/s transfer rates per session. We are being clever as we want to use a mobile and/or slow ADSL link to support the satellite by sending small packets (ACKs, DNS, etc) over the low latency link. This reduces the round trip latency which helps, but still does not solve the whole TCP window and startup issue.

So, next challenge, we have to make a system to do this TCP spoofing logic ourselves. Not a small project, but the rewards should be a stunning service with low latency interactive response time, but high throughput in the middle of nowhere... I'll post when we get the time to do this.

12 comments:

  1. Very nice indeed!

    I considered getting one of those systems for portable use a while back (like you, I can probably manage to point a dish in the right direction).

    Unfortunately, they use a set of several spot beams covering the UK, and don't (apparently) allow free migration between them - I think there was some talk of a £100 charge to get yourself moved between different spots. Fine if you're just pottering around the local area, but not so much if you're going to Scotland.

    Also, watch out for the AUPs and throttling policies - I think any kind of sustained transfer is likely to trigger them and land you with a much slower speed.

    Cheers,
    Rob

    ReplyDelete
    Replies
    1. I understand the footprints are actually quite big, and the fee was not that much when we asked but will depend on resellers I expect.

      There are also higher tariffs which list as "unlimited".

      Delete
    2. To be fair, they're not small, but three cover the UK:
      http://www.satsig.net/tooway/satellite-dish-pointing-ka-sat-tooway-europe.htm

      Fine for local use, but not so much (potentially) for the seamless "go on holiday" scenario, which is a shame. I realise this is not the typical use case though!

      Delete
    3. This comment has been removed by the author.

      Delete
  2. If you manage this, please sell it in France! I'll be your beta tester!! :-)

    ReplyDelete
  3. The uplink is in Italy, I think. (At least they finally give you a UK IP address, IIRC. It took them many years to get around to that. Clearly it's *harder* than rocket science.)

    They are definitely doing something fairly aggressive: they block everything but ports 80 and 443, and proxy those. I see frequent RSTs when httptunnelling anything over 443, but that may just be a capacity limitation at the far end aging out older connections (since a web browser would just initiate a new connection: shame my SSH session can't do that).

    ReplyDelete
    Replies
    1. Outgoing UDP port 0 works :-)

      Delete
    2. Ouch. That's nasty. If that's true, the fact that they don't offer Internet access (only web access) is not mentioned on their website in even the tiniest of tiny print. I'd be mightily unhappy to buy "...high speed Internet, everywhere and for all" only to find that I couldn't access vast swathes of the Internet.

      Delete
    3. NAB, it's mentioned in the T&Cs of the underlying satellite operator... which aren't linked from anywhere, are not mentioned when you buy the service, and are only available via a Google search. (Or it was, as of 2012, last time I looked.)

      IIRC (it's been a while), they go so far as to say that terminal emulators in particular are 'ABSOLUTELY FORBIDDEN' in capital letters, so I guess low-latency interactive sessions interfere with whatever trickery they are employing.

      I guess we should feel lucky that the T&Cs are in English, rather than, say, Italian. They're just on display in the basement, in a locked cupboard, in a disused lavatory with a sign on the door reading 'BEWARE OF THE LEOPARD'.

      I didn't do an nmap scan, so I didn't find individual ports that work. If UDP 0 works, I can probably run a VPN over that :)

      Delete
  4. what attracted you to satellite? what do you want to do with it?

    ReplyDelete
    Replies
    1. Trying to find clever ways to mix satellite and slow broadband or mobile to get best of both worlds and sell a service.

      Delete
  5. You abandoned this idea then?

    ReplyDelete