WIFi: Aruba vs FS

I have used Aruba WiFi for some time - they are nice, but expensive. The latest Aruba Instant series seem more sensibly priced though.

When picking WiFi there are a number of factors, but if you are going to get more than just the WiFi that comes with your broadband router you almost certainly want something that can allow multiple access points and roaming between them.

There is also a matter of the number of antenna and radios, and the protocol - summarised as catchy names like WiFi5 and WiFi6, these days. There are also access points for outdoor use, which can be useful.

The Aruba has one cool feature, and the main reason I like it, that it does not need a controller. A controller is what directs devices to move from one access point to another as you move around, and what manages the overall configuration. With the Aruba, the controller is one of the access points, and can even fall back to anther one if the main one is off.

So what changed

I have been pretty happy with the Aruba for many years. It has a lot of features and settings (perhaps too many), and as I say, needs no separate controller.

But recently some of my IoT kit (ESP32), which is all 2.4GHz, has been playing up, sometimes sulking not reconnecting to WiFi, and so I felt it was worth changing the WiFI to see if it helped.

Long story short, the WiFi choice was not the cause of that specific problem, and I think I have fixed it. But the change of WiFi had some surprising side effects. For reasons I won't go in to, I have all my 2.4GHz on the same channel. And so that is a pretty full channel. As such I was never surprised to find s/w upgrades on a set of IoT devices was always a bit slow - I set up to do one at a time. To my surprise, with the new WiFi, it is way quicker!

What is the new WiFi?

I have got some kit from FS. It is cheaper than the Aruba, which is a start, but it needs a controller. The controller (AC-1004) is also a 4 port PoE switch which makes it actually useful for a small home installation as you almost certainly want a PoE switch for your access points anyway. So again, not that much extra cost. It can handle 64 APs. I may be a convert to the idea of a controller.

The (web based) controls are a but more clunky than the Aruba, but to be honest, pretty similar. It also has a huge command line interface!

The APs actually come with mounting brackets, unlike the Aruba! I have AP-N505 indoors and AP-T565 outdoors.

So far, so good. I'll see if I am cursing the change in a few weeks. If not, I'll take down the Aruba APs.

Fun with DHCP

We have had a slight issue at the house here, we have some Apple HomePod things. My son decided to put several in the house when staying here and now my wife is using one.

The snag is that they keep falling off the internet! A power cycle fixes, but it is very frustrating.

I have found the solution though, and I think it points a finger at the cause.

And it is all down to DHCP. Yep, not DNS this time. Not IPv6 even. DHCP!

So what's the problem?

First off, what's the kit?

• FireBrick doing DHCP and Internet gateway
• Aruba APs
• Apple HomePods

The failure did not seem to be all the time, but could be. Sandra has almost given up using them as they never work. But it seems it can usually renew its DHCP without problems, but sometimes it gets stuck. The logs on the FireBrick showed we kept sending a DHCP "Offer" to the HomePo, but it keeps asking.

I added lots of debug, and confirmed that the request being sent, the DHCP "Discover", does not request a broadcast reply, which is fine, so we send the reply to the MAC of the HomePod and its "new" IP address. This is normal.

On a whim, I decided to try fudging the code to treat the discovery as if it has asked for a broadcast reply. This then meant a Discover, Offer, Request, and Ack - but the HomePod did not see the Ack and so kept asking. I then forced the broadcast on the Ack as well, and bingo, it worked. So the issue is the broadcast used for Offer and Ack.

This is a massive clue.

So more investigating.

The RFC says the broadcast request is in the left most bit of a 16 bit flag field.

PLEASE DO NOT DO SPECIFICATIONS LIKE THIS!

I fully understand that bits in a byte may be sent "on the wire" low or high bit first, or high to low bit first. I fully understand that bytes in a word may be ordered big endian or little endian. The above diagram is for a 16 bit "network byte order" value (i.e. big endian).

They number the bits from 0 to 15. Actually they number the gaps between the bits 0 to 15.

In my view there is only one way you should number bits - by their binary power of two value. I would always write that in the way we write numbers, most significant first, so would write that as bits 15 to 0, and it is bit 15 that is the B flag. I don't mind if showing as bits 15 to 8, and 7 to 0 (big endian) or even as 7 to 0, 15 to 8 (little endian), but number each bit by its power of two value, please!

Some people number as order on the wire, starting from 1. So 1 to 8 may be 0 to 7 or 7 to 0, who knows! Please do not do that. But at least if numbering bits 1 to 8, you have some clue that something is wrong.

So, to be quite frank, I actually do not know if this is bit 0 or 15 in a network byte order (big endian) 2 byte (16 bit) value. We assumed it is bit 15, i.e. bit 7 in the first byte. But seriously, from bits numbered 0 to 15 and a reference to "left most bit" I don't actually know for sure. I started to doubt we had read the RFC correctly!

Thankfully empirical testing shows the flags as 0x8000 from other devices, so either it is bit 7 of first byte, or other devices have the same fun reading the RFC. 

So who is at fault here?

Well, my son has the same FireBrick and the same HomePods, but different APs. That all works. That is another clue.

My Aruba APs are set up to inject data in the DHCP, which is good. I get details of the AP and SSID, and can even tell the FireBrick to allocate based on SSID even if different SSIDs on the same physical network. All good.

It may be that it is stripping the broadcast bit, bit that does not explain why it works after a power cycle. Interestingly the working DHCP renewals did not have the injected AP details, it seems. This points further to the AP being "special"

My son does have different network switches as well, so it is just remotely possible that it is a switch level issue, but that seems unlikely - the DHCP discovers are from the right MAC so all switch learning should be fine.

P.S. Yes, I had changed the filtering to disabled already.

The work around...

FireBricks now have an option to force broadcast reply. And it works. Alpha out soon.

Internet in a box

I have finished my cruise now, which was mostly holiday, but also some work. I did some training for my mates (mainly in C coding) and we did various coding as well (there were a few sea days). But now I am back I am making up the next version of my "internet in a box" that I take on cruises like this. I'm doing it now whilst I remember the last cruise in detail, even though my next cruise is some way off.

OK, that is not it - we have one of those, and it would be really cool if I could fit the bits in that box, but at present is is a tad larger... More like this...

So, what's in the box?

• FireBrick FB2900
• Aruba 501
• 2x Aruba AP-303H
• 3x PoE injectors
• 1x 4 way power strip
• Magic tape to hold it all in place

This is obviously somewhat overkill, so worth some explanation...

FireBrick FB2900

The FireBrick is a "swiss army knife" of network contraptions. It does a lot.

When you are trying to use internet on a ship you have a challenging, even hostile, environment. There are blocked ports and protocols, 700ms round trip latency (or randomly much more), packet loss at various levels, strange MTU issues, and seriously messing with TCP packets (acceleration). This can all change on the fly as you travel (the Panama trip was especially complicated).

To be clear, this is not stealing internet service - it is expensive and we pay for the premium, unlimited, steaming package for multiple devices. This does allow connection of devices that do not have WiFi or have a browser.

Whenever I take a FireBrick on a cruise we find new ways to improve it. This can be changes to handle high latency, or new features to handle some of the limitations. Even simple higher level protocols can struggle with the very high latency and low level packet loss. A lot of new features are the result of testing in this harsh environment and have benefitted the FireBrick code. Not sure I can expense my cruises as R&D just yet though, shame.

So, this alone, is one of the reasons for the crazy set up. The FireBrick can do various VPNs, UDP over faked TCP, TCP relaying, all sorts.

The main objective is to connect to the ship internet (WiFi) and provide internet to laptop or apple TV. For the apple TV to work in any expected way without regional blocks, it needs a working UK IP address in some way, and the FireBrick can do that.

The FireBrick can also monitor the connection in various ways and fall back, even to simple NAT over the ship's WiFi as last resort, and report status on an LED to make it obvious. If ever I fit this in one of those black boxes, the LED will not just blink red :-)

Aruba 501

This is a rather nice WiFi client. It connects to the WiFi and can do MAC cloning, where it will associate using the same MAC address the FireBrick is using. We found that the WiFi on ship filters other MAC addresses, and even locks down the connection after a little while if it sees more than one MAC. We were changing MACs every day until we managed to lock it down to no see any others.

Aruba AP-303H

Having connected to the Internet, and set up a VPN, we then provide internet over WiFi. It can be done with cables, but WiFi is fine and not as messy or such a trip hazzard. Previously I took a larger ceiling mount AP, but that gets hot, especially if not ceiling mounted. So this time I have smaller, and lower power, AP-303H units. I also have two, one facing each way, so the box can go in the corridor. Ships have big metal walls which make WiFi tricky. Even so, I am taking some 10m ethernet cables to allow me to place the APs to cover the whole cabin if necessary.

We actually had to set a hidden SSID, as we found that in at least one port we were seeing de-auth attacks. Interestingly this was not happening once we changed to hidden SSID. Even with the metal walls, we often see people running personal hotspots when in port, so it may be an attempt to stop that (AFAIK not legal to de-auth people like that, but who knows on a ship).

Update: Having two APs powered by PoE means I have more options - running a cable to place one, or both, APs, in more suitable locations in the cabin if they don't work in the box.

PoE injectors

This is another change from previous cruise - the last couple of times I took a nice 8 port Aruba PoE switch, which is quite big and has a big chunky power supply. This time I have three small PoE injectors which take a lot less space overall. There are some multiple port in-line PoE injectors which may be a good alternative to consider, but even with just one such unit I still need a power strip to power it and the FireBrick.

The AP-303H includes a switch, so if I need more Ethernet ports, they can provide them, so the bigger switch was not needed.

Power strip

The three PoE injectors and FireBrick mean a 4 way power strip - though I am considering making a lead with daisy chained C13 plugs and a C8 all on one lead perhaps. However, the 4 way strip fits fine. One option may be an IEC socket in the side of the Peli case so it can be closed. It looks like the whole lot is not generating enough heat for that to be an issue, but something to test.

Update: One idea is to use a 4 way IEC distribution board instead, which may well take less space.

Spare space

The whole box, even with all those bits taped in to place, has a lot of space. In fact I can pack my laptop, charger, mouse, mat, Apple TV, spare cables, phone charger, and so on, all in the one case. This means all of the tech in one small Peli case which then just sits in the corridor to provide "internet in a box".

Why?

Update: This allows me to bypass much of the hostile environment, and have clean Internet access on my own IP addresses. It even allows me to have a standard VoIP phone on the table in the cabin if I want. It allows devices that could not connect to ship's WiFi on their own (I had some of my IoT stuff on it). It is not trying to be the cheapest, or even the smallest (though I am trying to make it smaller). It mainly allows testing and development of the FireBrick in such an environment, and it is fun (for me), even if it is overkill.

Update

A few more pictures. I decided to go for an IEC distribution panel inside, and fit connectors to the case itself, and add a 3G/4G dongle.

P.P.S. Using V2.0.0.1-Aruba501-B0013 on the Aruba 501 was Crashy McCrashFace, but V1.0.1.3-HP501-B0012 seems to be stable.

RevK 2.0

So, I think I have svn reverted to 1.0, or maybe 0.9 now, as I have a cold, but my mates were talking of RevK 2.0 over the last month.

Largely because I let them talk me in to :-

• Going out in the sunshine - in tropical climates
• Sitting on a sunny breach
• Going to a Jazz bar
• A tour of a warship
• A tour of an aircraft carrier
• Swimming in the sea
• Swimming with dolphins
• Snorkelling (well, I tried, but kept hyperventilating)
• Parasailing (felt sick, life jacket was too tight)
• Boat trip to see crocodiles (they were mostly tiny)
• Going on a submarine (albeit docked)
• Several open top bus tours (which I do not normally go near)
• A walking tour in a hot city (Cartagena, Colombia)
• Taking a picture of an Aruba access point, in Aruba
• Oh, and wearing a Panama hat, in Panama

So over all, a fun trip! I did not do the jet skiing, just took pictures.

Here is a small selection of the pictures...

Serious WiFi case study (a house in Wales)

Two of my staff work from their home in Wales some of the time. Working for an ISP you expect that they have good Internet, and perhaps even good WiFi. Well, they have good Internet with our standard dual bonded FTTC offering, but WiFi posed more of a challenge.

The problem is the house! Whilst old buildings with thick walls are not an issue unique to Wales, they are a problem for WiFi. The outside walls are around a metre thick, and the middle of the house has huge fire places and chimney breast making an even thicker wall. Whilst the basic layout is two rooms per floor (one front, one back), it has three stories and high ceilings.

What this means is one WiFi access point in the front room does not working the back (kitchen). Similarly WiFi on the ground floor struggles on the 2nd floor, if at all. Basically, there is no one place to put a WiFi access point (AP) that will work sensibly for the whole house, or even most of it.

This is a big problem for the traditional arrangement of ISP provided combined router and WiFi AP. Even solutions using WiFi repeaters would struggle, so the best bet is to have multiple wired in APs.

The solution, for some time, was to have several Apple Airport Express APs, around 4 of them. This is very much "on the cheap". For some reason this was not a good solution. Much of the house was not well covered, and even in the same room as an AP a speed test would rarely show more than 1Mb/s and usually the WiFi was unusable - so much so that they took to turning off WiFi on phones and tablets and using mobile data instead, in there own house!

We recently added a new package to our order forms at A&A, a "Serious WiFi" package which includes two WiFi-5 APs, a PoE switch and selection of cables and couplers. The idea is that this gives you the kit to deploy two WiFi APs in such a premises. It is far from cheap.

They finally asked me to come and sort the WiFi, and I thought this was a great chance to test this new "package" which we have started selling in a typical situation. I took the two APs, a PoE switch, and cables. The APs are Aruba IAP-305s which are "WiFi-5" (802.11ac).

The first step was to work out where to put the APs for good coverage. This was a combination of looking at the rooms and the thick walls, and also considering where the mobile devices will most commonly be used. We decided high on the wall on the ground floor at the front facing in to the house, and high on the wall in a back room, on the 1st floor facing in to the house. There are a number of other places we could have set up the APs I am sure.

We unplugged the old APs, as the new ones do a frequency scan to decide on best channels anyway, and we used a double sided sticky pad to stick the new APs to the walls in the selected locations with a loose trailing network lead. This allowed us to test the positioning easily.

We looked at mobile signal strength and speed tests in each room, the results were good. Having decided on the position, this meant a masonry drill to fix the access points properly and cable clip the cable in place. Yes, I am not a decorator, I was only there to "make it work" - there are plans for some white trunking in the near future I gather :-)

We were quite pleased that the solution worked and did not need a third access point, which was always a possibility.

Once done, more testing, and they are over the moon with the result. Phones can seamlessly roam between the two access points. And the speed tests are somewhat better than the 1Mb/s they could manage before.

To be honest I was surprised how much difference this has made. The Apple Airports may be a bit dated, but they should basically work. Obviously the new access points have 2.4GHz, and 5GHz, and multiple radios and antenna, and newer (faster protocols), so this clearly makes a difference.

It does rather prove the worth of the new "Serious WiFi" package though, and I am pleased with the result. Expensive, but compared to "unusable" WiFi before, I suspect it was worth it - and would be well worth it for a business premises.

P.S. We had a bit of fun as well...

Outside WiFi

Whilst we have excellent WiFi in the house, due to three access points, the garden was not as well covered. A brick wall and conservatory in the way, etc. The WiFi was OK in the garden but I thought I should really try out an actual proper outdoor WiFi access point.

I now have one of these, an Aruba AP-375, and yes, they do cost over £1,000, which is silly, I agree. But it works with the WiFi we have (all one controller), and is clearly robust and weather proof.

We now have excellent WiFi in our garden, and the gardens of several neighbours!

The garden really is not big enough to justify the cost, and if it was not more of an experiment to see how well it works, I doubt I would have got it.

As with the others it is run of PoE, so a simple network cable in to the loft to a PoE switch. I got proper external grade cat5e cable - from Amazon (here).

With solid core network cable you have to get the right plugs that work with solid cable. I actually got some cat6 plugs for solid cable from Amazon (here) which "just work".

I had to also order the wall bracket. Do not be fooled by pictures in the Internet, there is an "H" and "V" version for the surface on to which it is mounted being horizontal or vertical. There are sites selling one with a picture of the other! I fitted with some wall anchors, which, you guessed it, I got from Amazon (here) and drill bit (here).

I walked around the block, and even found a spot on the main road where our WiFi is just visible (through a few houses /gardens). It goes quite a way in open air from high up.

Walking around, it is amazing how many WiFi signals you see. It was nice to see, amongst the VM and BT names, at least one aa.net.uk signal from someone in the next street :-)

Obviously, where these access points make more sense is when you do have a large open space, not just a simple residential garden like mine. We have done WiFi at a beer festival a few times, and if we do that again I can see us putting a few of these in.

Decent WiFi

The Aruba AP-515 "WiFi-6" Access Point

My home needs more than one WiFi access point. This is partly because it is a 5 bed house, and partly because the garage conversion for my office involved a lot of Celotex (metal foil coated insulation) in the walls and ceiling, which kind of stops WiFi signals dead. It would be just about possible to cover the whole house with one access point in the ceiling at the top of the stairs (as my neighbour does) but that would not get to my garage. So we actually have three access points.

Getting good WiFi at home

If you have a large home, or one with thick or insulated walls, you may need more than one access point to get good WiFi coverage. BT are making a point of this in recent adverts, but please do think carefully about this. WiFi is simply part of your home or office network infrastructure and a totally separate thing to your Internet Connection. Yes, smaller homes often have one box for all (modem, router and WiFi), and that does work for smaller homes. But if you want good networking at home or office, I'd recommend thinking about your network infrastructure (WiFi, Cabling, and switches) as a separate project than your choice of ISP. Good networking at home can make all the difference. WiFi repeaters and power ethernet devices are generally no match for running a network cable to where you need an access point. With PoE (power over Ethernet) that can be one cable and no need to find a power socket near where the AP is located (important when on the ceiling). Any fixed machines on desks, or TVs, etc, are often better handled on cable as well, and there are some nice PoE powered 5 port switches that are available making it easy to run one cable to behind the TV for several devices and not need another power socket.

Please, don't do what I saw in one place. We spent a weekend at a large holiday home. WiFi did not reach from one end to the other. They had installed two totally separate ISP connections each with a separate WiFi modem/router with the default ISP set SSID and password, in order to "cover" the whole house!

Ubiquiti/Unifi

Ubiquiti have been pretty groundbreaking in making a lot of network equipment available at a reasonable price, including good WiFi access points. I tried these and we were selling them at A&A. I am using Ubiquiti IP Cameras and they are pretty good (just got some of the G3 Pro models which do proper PoE, and they are nice).

I tried these access points at home, and I did run in to a real problem with iPhones, roaming between access points, and IPv6. At one point we thought FireBricks may be a factor, but it seems that is just because A&A sold Ubiquti APs, FireBrick routers and IPv6 networking - the problem was seen with non FireBrick routers. The problem looks like it is probably iPhone related, but hard to say if iPhone bug or not. The big clue was when someone found that turning off IGMP snooping on the AP solved the issue. This makes some sense as IPv6 uses multicast for neighbour discovery, so broken IGMP and multicast could break IPv6. What is interesting is we have also seen this on another make of AP now, though still iPhone specific, and it looks like it relates to 802.11r roaming. My guess is an iPhone bug.

Aruba

I changed to Aruba access points. They are a bit more pricey than Ubiquiti - around twice the price. Ubiquiti really have done a good job on price.

They have a confusing array of model numbers which fooled me a bit. Not only for different grades of WiFi standard and speed, and number of radios, but for indoor and outdoor, and for internal and external antenna. They have models for specific countries - the one we need is "RW" (which I assume means Rest of World).

Like most systems to manage a set of access points you need a controller. Ubiquiti do controller software for multiple platforms for free. What Aruba were doing is a separate hardware controller, but they then added Aruba Instant which is where one of the APs acts as a controller for the set, providing a nice web interface. This has the advantage of no separate controller, but also, if that AP is off line another takes its place with the same config, so redundancy built in. What was confusing is that this was a separate model - the IAP-305 is the one that can be a controller and the AP-305 cannot. What fooled me further is that later models are all able to be a controller but don't have the I in the name, so an AP-375 is an outdoor AP and there is no IAP-375 which confused me. The AP-375 can be stand-alone / controller. I'll be trying an outdoor AP (AP-375) soon, to ensure good coverage of the garden for the summer, and I'll no doubt post more on that.

As for roaming iPhones and IPv6, the Aruba has control of separate roaming related settings for 802.11r, 802.11k, and 802.11v. It seems only the 802.11r breaks iPhones and IPv6, but roaming is pretty seamless with the other two settings turned on. I am sure one day this will be fixed in iPhones and so 802.11r can be enabled.

Update: It seems 802.11r is OK now, but also enable the setting to make ARP unicast (broadcast filtering).

Personally I like the Aruba better, but that does come at a price.

WiFi-6

One of the interesting things for both manufacturers is the new 802.11ax standard (aka WiFi-6). I don't have a device that can use it yet, but it is rumoured that iPhones may have this year. Aruba have an AP that does it (AP-515). Once they are both available, I'll give them a try and keep you posted.