2020-03-29

Rules

Before anyone has a go about looking for loopholes, please just stay at home. If you have to go out, keep away from people, try not to touch your face, wash your hands (take sanitiser with you even). Stay safe!

However, when the new rules actually came out (here), and I mean for England, as, confusingly, other UK countries are not quite the same, I did wonder how the police were supposed to enforce them at all.

For example, a gathering of more than 2 people is prohibited, unless from same household. How to you prove you are from the same household, or, more importantly in an "innocent until proven guilty culture" how do the police prove you are not? You don't have to carry ID.

But I assumed some sense. E.g. a couple with a child going for a walk are probably OK, but a group of 20 teenagers is probably not.

Sadly the police are not being sensible, even publishing drone footage of a couple walking way apart from anyone else, and suggesting somehow that was not allowed. Worrying times.

But then you get to some of the more complicated rules. In this case I am not so much pointing out loopholes, as highlighting some rather crazy drafting of the law.

Funerals

A gathering for a funeral is allowed. You can leave your home to attend a funeral, but only of a member of your household, a close family member, or a friend. However, you can only go to a fiend's funeral if there is not a member of the same household or close family attending. But it seems like if none of your household or close family attending the friends funeral rather than close family or same household as the friend, which is rather confusing. It looks like if two people in the same family or household know someone as a friend, only one of them can go to the funeral. This is almost the reverse of the gatherings rule where people have to be the same household - at a funeral they can't be the same household, if the deceased is a friend, but can if the decided is their family. Arrrg! How do you police that and why is that the rule? It makes no sense!

It would have been far simpler to say funerals max 25 people or something simple and enforceable, but no, they have created convoluted rules.

But wait, it is not that simple. The convoluted rules only relate to leaving your home. It is the valid excuse to leave if to attend a funeral. If you leave your home to, say, exercise, you can then go to a funeral (an allowed gathering).

So it seems the only safe way to go to a friend's funeral is to wear a jogging outfit.

I can only hope none of us have to attend many funerals though - so please stay home and stay safe.

Need

Of course, these convoluted rules are simply part of the list of things that are a reasonable excuse. You can leave home for any "reasonable excuse" even those not on the list, but you may have to convince a police officer of your reasonable excuse. Even the list has the caveat that you leave your home for a "need" to do something, and arguably nobody "needs" to go to a funeral.

2020-03-24

Business as usual, or is it?

Obviously, to slow the spread of COVID-19 cases and reduce burden on NHS, we need people to stop interacting as much.

TLDR; governments have given five different messages about going to work, from yes you can, to only key workers, shutting down the economy almost entirely, with no clear message. It is causing confusion and will cost businesses and livelihoods.

The message from the government last night seemed clear...

Don't go out, but you can get food, medicine, and importantly you CAN GO TO WORK. Ideally, you work from home if you can, but all that was being shut down is the face to face shops and places where people gather and spread the virus a lot with other random people. Indeed, stopping all contact probably does not help as the spread just comes back as soon as you relax the lockdown - this is all about statistics - reduce contact enough to allow the NHS to cope.

It was, otherwise, for all those offices and factories and businesses that keep the economy going - business as usual (working from home if you can).

Now, assuming this is enough to slow the virus, that is great, well done UK government.

So, yes, please follow advice, stay at home, let's slow this spread enough for the NHS to still work.

My issue, however, is the horrible mixed messages from the government, which is really causing a lot of confusion.

First the message was, you can go to work, but try and work from home where possible.

Then, No10 twitter tweeted (image below) that you can go to work (if you're a key worker).

Now that is a massively different message. That is shutting down the whole UK economy in one go over night with no notice. That is HUGE!

They deleted the tweet, and changed so you can go to work (but work from home if possible) as per the speech. OK, good.

This was however a massive mis-direction and confusion by a tweet that was up for a while last night.

But it gets worse, a document about the lock down (here) says, in bold and underlined: "Non-essential businesses and premises must now shut!" We think this is meant to be a heading, and wording like "Non-essential businesses and premises that must now shut:-" or some such, but that is not what it says.

Again, confusing and mixed messages.

But now they are SMSing everyone with a link to gov.uk/coronavirus which says not to go outside except for "essential work", which is yet another unclear term, and again suggests a complete UK economic shutdown.

So what the hell is the message meant to be - is this business as usual apart from shops, or what?
Why the hell are the government not giving a CLEAR MESSAGE?

P.S. Obviously offices and factories have some duty of care, so things like people being able to keep apart and hand sanitisers and so on are an important step as well.

P.P.S. as per final image below, they have now changed the web site to say "work (where this absolutely cannot be done from home)". This is a total shambles!

P.P.P.S. the BBC have yet another, subtly different wording: Travelling to and from work, but only if it is "absolutely necessary", which is not clear either, is it necessary for you to still be paid, or is it "necessary work", i.e. key worker stuff?

P.P.P.P.S. as of this morning (25th) they have finally changed the PDF guidance document. That is three separate bits of government information that have changed now, having initially given misleading or wrong information.

Here are the messages:-









2020-03-14

Pi day

As it is Pi day (14th March) I thought I would say something obvious, but slightly mind blowing about another irrational number.

When you square a decimal number, you always end up with twice as many significant digits, or one fewer. e.g. 214² = 45796 (3 digits times 2 is 6, but in this case one less, 5), 56² = 3136 (2 digits times 2 is 4). If you think about it, it is obvious. Basically, for the last digit you multiply out not to count as a significant digit, it would have to be 0. You can do that with 2*5, but not by squaring any final significant digit.

But √2 is irrational. It goes on forever. It starts 1.41421356237 (12 significant digits), which squared is be 1.9999999999912458800169 (23 significant digits, i.e. 12*2-1).

Obviously the more digits, the closer... even lots, such as 1.41421356237309504880168872420969807856967187537694807317667973799073247846210703885038753432764157273501384623091229702492483605585073721264412149709993583141322266592750559275579995050115278206057147010955997160597027453459686201472851741864088919860955232923048430871432145083976260362799525140798968725339654633180882964062061525835239505474575028775996172983557522033753185701135437460340849884716038689997069900481503054402779031645424782306849293691862158057846311159666871301301561856898723723528850926486124949771542183342042856860601468247207714358548741556570696776537202264854470158588016207584749226572260020855844665214583988939443709265918003113882464681570826301005948587040031864803421948972782906410450726368813137398552561173220402450912277002269411275736272804957381089675040183698683684507257993647290607629969413804756548237289971803268024744206292691248590521810044598421505911202494413417285314781058036033710773091828693147101711116839165817268894197587165821521282295184884720896946338628915628827659526351405422676532396946175112916024087155101351504 squared, is 1.9999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999843358064061778389918448031979379487768885015043845659822004916377513166730291211571068952082602447585053372271803775212141335211046806468017544012173246168673653215429898752600809076945040478226149804752092795249913294211896685919489781038912339790863836308797672158570101054709837854108990150947671626758096902141232172374611365869462539815599093474631680641002315522198827855254655681178183773171267162025505081207619119154315015019593781997539423459450924437955866358803983089345879502011206070716091422845617882605404958297112110314310433923158845197889301909351312924983769999151087074570462040601631719823168853951704021687236598419124498942731362178121174457523357788222619729955636686231081657732082989686914502778260949270241394967997279088545806588326980598466346571508275959469245048169832037947129583450529101421217940987273080154757322849492641724996747550397932241142493214043106291202086235967891808498928787226259087932847850782010982642427061227177199617553123685346155963957806300620208169830364794225870383219224922644130340677633853727891782934367363062016

Yes, lots of 9's after that 1. but it just gets more complex.

What must blow your mind is that ultimately, to infinite digits, the answer does actually cancel out and actually end up as exactly 2.

How?!?!


2020-03-09

Backwards compatibility!

Arrrrrrrrg!

As I said, I have made a nice new decimal maths library and am using it in one of my bits of code.

It was a simple update. The original code evaluated a simple sum and provided an answer. It was a tad quick and cheerful (a bit of a bodge) but worked well enough until recently. The new code does the same but using my new library.

What could go wrong - it was a function to evaluate a text decimal sum, simple as that. How hard could it be?

The original function just added and subtracted. It worked out how many decimal places were in play and did integer maths to that precision and printed result to that many places. I said it was a bodge.

This meant, as a fluke of the way it worked, 1.00+2 would be 3.00. I.e. the number of places in the output was dependant on the max number of places of the arguments. This was never a documented feature, but guess what, people were assuming it would work like that. This is in spite of output formatting for money to two places which is what I was using in my stuff.

Then we get the fact the old code allowed blanks for operands and treated as 0. Again, a fluke of the original code and not a documented feature, but once again, they were relying on it.

It is infuriating having to make special cases in code to reproduce old and totally undocumented side effects in some function just to keep users happy! I suspect that even if the manual had said "You cannot assume a specific number of decimal places in the output" and "Behaviour for an input that is not a well formed numeric expression is undefined and cannot be relied on" it would not be any different. Users are like that!

I am waiting for the next thing, but I have a feeling it is a side effect of the total lack of error handling and syntax checking in the earlier bodged code. I might end up refusing to fix what happens when total garbage is thrown at the function and insist they fix their use cases. We'll see how bad it is.

Arrrrrrrrrrrrg!

No comment

It seems blogger have broken something - no matter ho much I clear cookies and so on, I am not able to comment on my own blog!

It used to be that it understood it was me, allowed me to edit posts directly from my blog, and comment, and reply to comments as me.

Sadly, it is not doing that - I have to go in to blogger and from there I can make new posts and edit posts, but even then I am unable to actually comment or reply.

So if you are wondering why I have not replied lately, blame blogger.

Grr.

P.S. Apparently it is not just me! Thanks for confirming, Neil.



P.P.S. looks like using incognito actually asks me for my login, and then oddly the main browser window knows who I am, WTF? So, err, fixed now.

P.P.P.S. and magically, a little later, it forgets who I am again and won't ask me for a login. Grrr

2020-03-08

Big number maths

One of my first C programmes ever, at uni, was a calculator. I had done a lot of code in BASIC, Z80, 6502 and other languages before, but C I learned at uni.

It was a strange exercise for me as I was teamed with someone else, and basically gave up on him. To me a calculator, i.e. something that could parse 1+2*3 and get 7, and understand brackets, was inherently simple. The code was a couple of pages at most and I had added all sorts of extras over and above the basic +, -, *, / binary operators. It handle brackets and operator precedence, and even had a table of operators to allow easy expansion. My "partner" made code that was dozens of pages and every test that involved extra brackets or some unexpected sequence broke it and he had two add extra code for edge cases. Oddly I once encountered a compiler that must have been written in such a way as adding extra brackets, e.g. 1+(2*3), actually broke it and made wrong code. Scary that stuff like exists.

My code has a simple loop to process: prefixes or "("s, operand, postfixes or ")"s, operator, in a loop, ending after last operand. It stacked operands, and operators (after processing the stack for any higher operators before adding), and that was it. It allowed unary pre and post operators, and binary operators. Simple and easy to understand, IMHO.

Of course, it used the normal C code to parse and output numbers, storing internally as floats (I think).

However, having played with mechanical calculators, one of the things I have meant to code one day and had not got around to for over 30 years, is a decimal calculator library... That is until this weekend.

Basically, one tool I have written, and my friends use a lot, includes an "eval" function to evaluate a simple sum for them. It is used in loads of places (as it embeds maths in a back end for a web page). It used integer (long long) types (64 bit) and shifted by number of decimal places it saw, giving it around 18 significant digits. Sadly it broke if you went over that (my bad) and they had some silly case of some numbers plus a fraction which had rather too many decimal places. So they asked me to fix.

A simple fix was limit the size, and then limit size and apply bankers rounding at the limit.

But I decided this may be the time to finally do that decimal maths code. I googled, and there are a few decimal libraries. There are also some arbitrary precision binary libraries. To my surprise the latest GCC has decimal types, where data is stored and processed in decimal. I can only assume we have decimal maths in some processors even, these days. This is good for accounting where the rounding errors you can get are bad (essentially binary does not have a way to represent 0.1 or 0.01 without recurring digits). Sadly GCC may have it, but clib does not yet (scanf and printf), but will some time, and I can see that being useful. Even so, _Decimal128 does have limits on number of digits. There are libraries in other languages, but I wanted something for C.

So I made a C string decimal library, and put it here on GitHub for all to use.

It is a library and command line, and includes functions to add, subtract, multiple, divide, and compare, as well as a simple evaluation function that parses basic sums, just like my uni project calculator.

The key thing is that it works on C text strings for numbers. A simple sum (I learned at school, and now out of date) was 366.2564*86164.091=31558149.7789324. It was numbers from an encyclopaedia in the library. I remember because I had to do the maths manually as no calculator I had would do it to full precision. Even google calculator truncates it a bit. That was my first test, and it worked. Yes, I also learned π to 50 places (and now only remember 25).

So, for add, subtract, and multiple, it is simple to ensure you have as many digits as needed. Sadly division can go on for ever, so I had to include a limit of decimal places, and a rounding rule, and remainder option.

I included rounding on limit on division, and also a separate simple rounding function - obviously actually applied at whatever digit you set. Default is banker's rounding :-

  • Round towards 0 (i.e. -3.9 rounds to -3)
  • Round away from 0 (i.e. -3.9 rounds to -4)
  • Round towards -ve (i.e. -3.9 rounds to -4, 4.2 rounds to 4)
  • Round towards +ve (i.e -3.9 rounds to -3, 4.2 rounds to 5)
  • Simple rounding (i.e 2.4 rounds to 2, 2.5 rounds to 3)
  • Banker's rounding (i.e. 2.4 rounds to 2, 2.5 rounds to 2, 2.6 rounds to 3, 3.5 rounds to 4)
I went further though, and made the calculator work on rational numbers, that means all of the maths is done with numerator and denominator, and only finally at the end does the division get done and rounding applied. This means that 1000/7*7 is 1000, unlike bc which makes it 994. minor optimisations for adding/subtracting with same denominator, and anything with denominator of 1 or a power of 10, making it simpler. For anything without division the maths does not even need a final divide.

I had a few silly bugs, one that fooled me is that zero is a special case, which I store as a magnitude of 0 and no digits, which is fine, but comparing numbers I started by comparing magnitude before comparing digits, so 0 looked like a bigger number than 0.1 as 0 was 0*10^0 and 0.1 was 1^10-1 and magnitude 0 is greater than magnitude -1. I had to add checks for comparison with 0.

However, overall, I am quite chuffed. Heck, I even asked it to work out 1e1000000000+1 and it just worked, that is a billion significant digits!

This is pure coding fun though. But may be useful - feel free to use it.

P.S. Someone asked about Banker's rounding, and I was sure I had blogged once, but cannot find it. Unlike that which I was taught at school, and every Casio (and other) calculator I had, rounding is not as simple as you think. I used to assume that a residual of exactly 0.5, or above, rounded up, and below 0.5 was down. That is what my calculator did. However, this creates a bias. Ideally you want to round an exact 0.5 residual down half the time and up half the time to remove bias. This could be random, but that is bad as you get results which are not reproducible. One simple way, bankers rounding, is to round 0.5 residual to nearest even number. Hence 0.5 rounds to 0, 1.5 rounds to 2, 2.5 rounds to 2, 3.5 rounds to 4, and so on.

2020-02-23

Internet in a box

I have finished my cruise now, which was mostly holiday, but also some work. I did some training for my mates (mainly in C coding) and we did various coding as well (there were a few sea days). But now I am back I am making up the next version of my "internet in a box" that I take on cruises like this. I'm doing it now whilst I remember the last cruise in detail, even though my next cruise is some way off.


OK, that is not it - we have one of those, and it would be really cool if I could fit the bits in that box, but at present is is a tad larger... More like this...


So, what's in the box?
  • FireBrick FB2900
  • Aruba 501
  • 2x Aruba AP-303H
  • 3x PoE injectors
  • 1x 4 way power strip
  • Magic tape to hold it all in place
This is obviously somewhat overkill, so worth some explanation...

FireBrick FB2900

The FireBrick is a "swiss army knife" of network contraptions. It does a lot.

When you are trying to use internet on a ship you have a challenging, even hostile, environment. There are blocked ports and protocols, 700ms round trip latency (or randomly much more), packet loss at various levels, strange MTU issues, and seriously messing with TCP packets (acceleration). This can all change on the fly as you travel (the Panama trip was especially complicated).

To be clear, this is not stealing internet service - it is expensive and we pay for the premium, unlimited, steaming package for multiple devices. This does allow connection of devices that do not have WiFi or have a browser.

Whenever I take a FireBrick on a cruise we find new ways to improve it. This can be changes to handle high latency, or new features to handle some of the limitations. Even simple higher level protocols can struggle with the very high latency and low level packet loss. A lot of new features are the result of testing in this harsh environment and have benefitted the FireBrick code. Not sure I can expense my cruises as R&D just yet though, shame.

So, this alone, is one of the reasons for the crazy set up. The FireBrick can do various VPNs, UDP over faked TCP, TCP relaying, all sorts.

The main objective is to connect to the ship internet (WiFi) and provide internet to laptop or apple TV. For the apple TV to work in any expected way without regional blocks, it needs a working UK IP address in some way, and the FireBrick can do that.

The FireBrick can also monitor the connection in various ways and fall back, even to simple NAT over the ship's WiFi as last resort, and report status on an LED to make it obvious. If ever I fit this in one of those black boxes, the LED will not just blink red :-)

Aruba 501

This is a rather nice WiFi client. It connects to the WiFi and can do MAC cloning, where it will associate using the same MAC address the FireBrick is using. We found that the WiFi on ship filters other MAC addresses, and even locks down the connection after a little while if it sees more than one MAC. We were changing MACs every day until we managed to lock it down to no see any others.

Aruba AP-303H

Having connected to the Internet, and set up a VPN, we then provide internet over WiFi. It can be done with cables, but WiFi is fine and not as messy or such a trip hazzard. Previously I took a larger ceiling mount AP, but that gets hot, especially if not ceiling mounted. So this time I have smaller, and lower power, AP-303H units. I also have two, one facing each way, so the box can go in the corridor. Ships have big metal walls which make WiFi tricky. Even so, I am taking some 10m ethernet cables to allow me to place the APs to cover the whole cabin if necessary.

We actually had to set a hidden SSID, as we found that in at least one port we were seeing de-auth attacks. Interestingly this was not happening once we changed to hidden SSID. Even with the metal walls, we often see people running personal hotspots when in port, so it may be an attempt to stop that (AFAIK not legal to de-auth people like that, but who knows on a ship).

Update: Having two APs powered by PoE means I have more options - running a cable to place one, or both, APs, in more suitable locations in the cabin if they don't work in the box.

PoE injectors

This is another change from previous cruise - the last couple of times I took a nice 8 port Aruba PoE switch, which is quite big and has a big chunky power supply. This time I have three small PoE injectors which take a lot less space overall. There are some multiple port in-line PoE injectors which may be a good alternative to consider, but even with just one such unit I still need a power strip to power it and the FireBrick.

The AP-303H includes a switch, so if I need more Ethernet ports, they can provide them, so the bigger switch was not needed.

Power strip

The three PoE injectors and FireBrick mean a 4 way power strip - though I am considering making a lead with daisy chained C13 plugs and a C8 all on one lead perhaps. However, the 4 way strip fits fine. One option may be an IEC socket in the side of the Peli case so it can be closed. It looks like the whole lot is not generating enough heat for that to be an issue, but something to test.

Update: One idea is to use a 4 way IEC distribution board instead, which may well take less space.

Spare space

The whole box, even with all those bits taped in to place, has a lot of space. In fact I can pack my laptop, charger, mouse, mat, Apple TV, spare cables, phone charger, and so on, all in the one case. This means all of the tech in one small Peli case which then just sits in the corridor to provide "internet in a box".

Why?

Update: This allows me to bypass much of the hostile environment, and have clean Internet access on my own IP addresses. It even allows me to have a standard VoIP phone on the table in the cabin if I want. It allows devices that could not connect to ship's WiFi on their own (I had some of my IoT stuff on it). It is not trying to be the cheapest, or even the smallest (though I am trying to make it smaller). It mainly allows testing and development of the FireBrick in such an environment, and it is fun (for me), even if it is overkill.

Update

A few more pictures. I decided to go for an IEC distribution panel inside, and fit connectors to the case itself, and add a 3G/4G dongle.






P.P.S. Using V2.0.0.1-Aruba501-B0013 on the Aruba 501 was Crashy McCrashFace, but V1.0.1.3-HP501-B0012 seems to be stable.

2020-02-20

RevK 2.0

So, I think I have svn reverted to 1.0, or maybe 0.9 now, as I have a cold, but my mates were talking of RevK 2.0 over the last month.

Largely because I let them talk me in to :-

  • Going out in the sunshine - in tropical climates
  • Sitting on a sunny breach
  • Going to a Jazz bar
  • A tour of a warship
  • A tour of an aircraft carrier
  • Swimming in the sea
  • Swimming with dolphins
  • Snorkelling (well, I tried, but kept hyperventilating)
  • Parasailing (felt sick, life jacket was too tight)
  • Boat trip to see crocodiles (they were mostly tiny)
  • Going on a submarine (albeit docked)
  • Several open top bus tours (which I do not normally go near)
  • A walking tour in a hot city (Cartagena, Colombia)
  • Taking a picture of an Aruba access point, in Aruba
  • Oh, and wearing a Panama hat, in Panama
So over all, a fun trip! I did not do the jet skiing, just took pictures.

Here is a small selection of the pictures...






















2020-02-16

Escorted off the ship

Being "escorted off the ship" sounds a tad bad, but this was at the right point, at the end of the cruise, honest...

My mates and I are just back from a cruise (one hell of a holiday, thanks guys), and we stayed in the "Garden Villa" on the Norwegian Gem. It is big suite on the ship. It is often used for celebrities, or Sultans, and the like, but they also auction it to suite guests if not sold. Some times you end up with the likes of us in it!

[A slight aside, I do love how the concierge emailed crew to advise that even though it is three blokes in the Garden Villa, it is not a gay cruise, we have wives - even so people kept asking Mike and Simon if they were a couple, which was amusing]

Anyway, even though not actually celebrities, you do get treated as such, which is both nice, and also slightly embarrassing. I imagine real celebrities are more used to it.

I wanted to tell of one of the perks, which was slightly more embarrassing than we expected. It may be educational for anyone else staying in the Garden Villa on an NCL ship.

You get escorted off the ship!

This means that the concierge waits to the right point for when your bags will be ready (among the first bags off as "priority"), and escorts you from your room (Garden Villa) to the gangway where he/she hands off to an escort to take you off the ship. Just getting the to gangway means going via staff elevators, and past waiting queues of people, and at some points people being stopped and waiting for you to pass before they continue.

Once off the ship this continues, more queues of people and you get ushered past. We are taking to the baggage area, and our bags put on a trolly with a porter, and on we go. All very slick.

Just to be clear, we did not know quite what was going to happen, and were just following instructions from the escort.

Finally we get to immigration control, and a queue for the passport checking desks. This is a well established ship process it seems, and very slick. All the security know what they are doing and were expecting the escort and Garden Villa guests so co-ordinated to move us to the front of the queue.

We get to the border control person, and he is "who are these people, why did they jump the queue?". Our escort says "Garden Villa" which means nothing to him, and he ends up saying "nope, not unless my supervisor agrees" and calls him over. The supervisor just says "let them through and send to secondary screening". We can only assume this is somehow to punish us for jumping the queue, but secondary screening were at a loss as we were not on any lists or anything. They asked if we had any dairy products and basically sent us on with maybe a couple of minutes delay. We are then escorted to a waiting car to take us to the hotel.

[OK, last bit went wrong, and no car, and they went in to headless chicken mode, calling the ship, etc, so we just got in a taxi, which is easy enough in New York, but you get the idea. It was too cold to wait for it to be sorted.]

Now, this whole process is slick. As I say, the ship and port staff know what they are doing, and know about people being escorted, etc.

To be fair to the guy, I can see why the border control person was upset - why should someone be able to effectively pay to go to the front of the line? That said, I thought secondary screening for the hell of it was a tad petty, but what the hell. We were all being quite chilled over it - as to be honest we had imagined all sorts with the Coronavirus scares and ships being refused entry and so on. This was a breeze.

Then it struck me that this "everyone treated equally" is all very well, and laudable, but basically not how a lot of things work in practice. The ship definitely have tiers of guests. So do aircraft, and 1st/upper class get off first. EVERYONE in the queue for immigration control was on "priority disembarkation" which is paying to jump the queue of some 2,000 or so people behind them waiting on the ship, and they effectively paid for that by being in suites. Somehow none of that mattered to him. If he really did feel strongly about everyone treated equally he should have made a stand on that front, and said the people in the cheapest cabins should be coming through first, or at the same time. No, his only beef was that he actually saw that were jumped the queue.

I hind sight, it would have been better, and not really much delay, to have queued with the rest of the priority disembarkation passengers at that point, and we'd have done that if we'd have known what was happening. But we were just doing as we were told by the staff.

Anyway, if you are in the Garden Villa on an NCL ship, this is the sort of thing you can expect, and perhaps decide how far you want it to go, yourself.


I am, of course, interested in views. Should people be able to pay to speed up such tedious processes, or (at the other extreme) should royalty and presidents all have to queue with everyone else in such cases? Should 1st/upper class on planes be scrapped? Where do you draw the lines? Curious?

2020-02-13

Standard C function to read lines from a file

[update: As I hoped, there is a simple answer, getline(), see comments, thank you Charles Lecklider]

The classic is fgets(), it is simple, and easy to use...


Of course, for some reason, fgets() gives you the line endings, so I usually end up with more like.


The problem, of course, is you have a line length. This is also an advantage in that you constrain the lines and don't have random memory allocation issues, but computers have so much memory and VM these days. How many times have I seen this code, and seen someone have to change 1000 to 10000 one day?

What I would like is a simple function that reads a line and mallocs space as needed. Indeed, it could return the allocated space or NULL for error (EOF, or malloc fail). You'd have to free it, but no big issue. Would also be nice if (a) it stripped the line ending as I literally NEVER want that in the line, and (b) seamlessly handled bloody DOS style carriage returns...


So whilst trying to explain some basic C to my mates, whilst at sea, in the middle of the Atlantic, I tried to explain this whilst making a simple CSV file parsing program for them. We did some googling, and found that I am not alone in trying to find such a function. It seems that fscanf() may be the answer. [update: clearly I did not google well enough!]

To be honest fscanf() is a function I just don't use enough. It is very powerful, but I always find myself parsing things more directly. However, I had not considered it as a means to just get a line.

The magic incantation is something like...


This reads any characters up to a newline, allocates space (that is what m is), and stores in line. Just what we need. A minor variation to handle carriage returns seems to work too...


Bingo, we have our magic line malloc file reader function. Perfect.

And get this, reading the man page it is clear that using the [ function does not consume the leading white space, which is perfect... So all good


Except that is not what happens. We did the CSV stuff, and then went on to TSV (tab separated) and magically leading TABs (i.e. empty first field) were stripped by fscanf()

Why?!?!?!?!?!

Please someone tell me I am being thick and that there is a standard function to do just this. Yes, I could write my own, but this is surely so basic it should be standard C library stuff.

[code mistakes in examples left in for the reader to find]


2020-02-04

VPN on a ship

There are many reasons for a VPN on a ship, and I was quite impressed that my other hack for UDP over fake TCP worked at all.

It did work, but was slightly strange. The experience was that normally VPN (via UDP or TCP) worked, but had 700ms latency round trip.

However, some of the time, the ship would have packet loss on the back haul, and then things changed. The UDP VPN would lose enough packets that normal TCP really struggled. Indeed, some times the VPN itself struggled to establish or stay up. It was actually for this reason that we even tried the TCP based solution.

However, in such cases, the TCP based VPN is special. It is very reliable, but has silly latency (up to many seconds).

The reason is what the satellite link it doing behind the scenes with TCP. It is doing an accelerator, which (I assume) is some level of local ACK combined with some internal re-transmission for dropped packets. But this means our VPN over TCP has the problems of doing TCP properly rather than faked, in that dropped packets result in high latency for every following packet.

Given the "cleverness" going on, it is amazing our faked TCP was good enough at all!

However, we like to have proper IP addresses here. There are a lot of reasons for that, but one of them is simple, it puts us, and our apple TV, geo-located at home. This was rather important for watching the episodes of Picard as they came out, even when in the middle of the Pacific.

So the challenge was proper UK IP addresses but still using the ship's TCP acceleration.

The solution was to de-NAT the traffic. The local brick here would NAT and send via ship's internet, which NATs again, but we change the destination IP to go to a brick in the UK. That then de-NATs, mapping back to original source and destination IP addresses and ports, or NAT the source. This works a treat, but you may have spotted the challenge here... How does the far end know where you wanted to send the packets in the first place?

The answer was to tack them on to the TCP SYN and look a them at the remote end!

My first thought was push them in to TCP options, but that is a challenge as there are only 40 bytes, and MSS takes 4 of those, leaving not enough bytes for two IPv6 addresses and ports. We could for IPv4, or we could just send target IP/port and NAT in UK. So we tried that.

The buggers NOP out any TCP option they do not understand, arrrg!

Alternative was put actual data in the initial SYN. To our surprise that worked, and as a SYN does not normally carry data, it works well (it is one of those theoretical TCP things I have never seen used). But it survives, and does not have the small size limit. We do have to move the SEQ back to allow for it, and strip out at the far end before sending on, but end result is public IPv4 and public IPv6 here, working. There is even space to add some authentication. (Note, this is not encrypted as assumes we will encrypt over it with TLS/https, ssh, etc).

Sadly sessions do not persist over glitches, and no incoming sessions, but I could watch Picard, so mission accomplished.

P.S. We have a choice of how to connect now, VPN over UDP, TCP (faked), relayed TCP, or just NAT on to ship's network raw (no good for Apple TV). Today we found UDP the best - the ship's TCP was unbearably slow (relaying, or not), but UDP was way better. To be honest this is much more what we are used to - everyone on the ship saying internet is crap and we're like "high latency, but not bad really" because the UDP based VPN works so well. The main thing is we can switch between different approaches as needed on a trip like this.

2020-02-02

VPN over UDP over *fake* TCP

It has become a bit of a tradition for us now that, on a cruise with my mates, we spend maybe a day on some serious FireBrick development that is relevant to the cruise. In the past we have greatly improved the PPP stack. Last time we greatly improved the SIP handling of retries and duplicate packets and dropped packets (I had a working VoIP phone on my desk, and even got a junk call). Basically, the internet access on a cruise ship is, for want of a better word, "special". It can have horrid packet loss, and typically has 700ms latency but can be way more (as I type this I am seeing 2s). It creates problems for a variety of networking.

So this time, I have coded a special new feature on the FireBrick to make UDP in to fake TCP. The reason was that at one port (Guatemala) even UDP was being a problem.

Let's just be clear here first, this is not hacking in any computer misuse sense - we have paid for the most expensive internet. Not just the premium "unlimited", but the premium "unlimited" with steaming. That's right, "unlimited" somehow has a limitation of not allowing streaming! It specifically says it allows VPN but they are unable to say which VPN, and out-of-the-box IPsec on my phone or laptop simply does not work (even when it uses UDP). Oddly, unlike previous cruises, simple L2TP was blocked, meaning it had to be mapped to another random UDP port to work. But when even UDP struggled somehow, we considered TCP. Let's get that VPN, for which we have paid, working.

One idea was simply to open a TCP connection and push UDP packets through it. Not hard, but has issues. For a start, the entire latency / throughput has to be buffered on the firewall for that to work. But also, one dropped packets causes a backlog of all streams until received, as TCP is always in order. At 700ms+ that matters. So we wanted UDP behaviour but something that the ship would think is TCP.

Dumb idea (thanks Mike) was simply change protocol tag from UDP to TCP. After all the ports are in the same place at the start for both. Unfortunately even the dumbest NAT will look at the TCP flags for SYN, FIN and RST at the least.

So, less dumb, change to a TCP header with sensible flags (SYN on first, SYN+ACK on first reply, and ACK on the rest). But pack the otherwise lost 12 bytes (TCP uses 20, UDP 8), in to the SEQ, ACK, Window, and Urgent fields in TCP. That way, NAT can play with ports and look at the TCP flags but pass through the same data with no extra overhead. I am quite sure that would work on some NAT. I think FireBrick NAT would pass that with no problem.

However, we found that the ships system has a rather heavy handed NAT that not only changes ports but also sequence numbers (why?!). It also expects valid sequence numbers to be used in SEQ and in ACKs. Simply resending a SYN with a different SEQ or sending an ACK that is before the SEQ of the SYN caused a RST and dropped session. So we had to make the TCP look the part.

The answer was that the first packet was changed to just a SYN (and later updated to a SYN with window scale, just in case it cared). At the start we set a random sequence and store it, and for SYN send the SEQ to that minus 1. Only once we had a SYN+ACK did we consider the session properly started and we could send the UDP packets as TCP, moving on the stored SEQ for each to look like a legit TCP stream. On the other side, for a SYN, we generate a SYN+ACK response, and consider the session started. Any protocol over the top loses the first packet to the SYN, but as it is UDP it will resend, which is exactly what L2TP does. Obviously the MTU was dropped by 12, but we were working on 1280 to get through whatever shit the ship was using between us and dry land anyway. The ACK was then based on highest SEQ+length received regardless and so there is no buffering or resending - that is all handled by the VPN and ultimately individual TCP sessions over the VPN.

End result, after more hours than I would have hoped, it works. It is in FireBrick release1.53.025 Flint+ Alpha, as experimental. Part of firewalling rules allowing a protocol 6 to be set to 17, or 17 to be set to 6. Have fun. Likely to change some time, perhaps with option to try using the 12 bytes in TCP header to avoid extra overhead.


2020-02-01

Far from #Brexit - some civilised drinks

It happens that, at the time, my friends and I were far from Brexit. Indeed, we were exactly 14.48054467N and 94.175880624W (or -449786.479813 -6160429.922092 1584520.022083 in ECEF) which is in the Pacific ocean.

However, as there were a few Brits on the ship, we got the concierge to invite some to our suite for a few drinks. We engineered nearly live BBC on the TV via a convoluted set of kit and VPN back to the UK, and we had my brexit clock.

It was remarkably civilised, with those for and against agreeing that whatever we do, we now have to find a way to live with this and make the best of any opportunities created.


It is the first time we have "hosted" a party from the suit, but butler managed it quite well and the ship even provided a couple of bottles of champagne for the party.

Amusingly at least one couple assumed it was a scummy NCL promotion to try and up-sell the suites, LOL.

2020-01-17

Diabetes, and CGMs (Freestyle), non-diabetic using one!

I have been diabetic for a few years. My mum was too, since she had me, and we suspect this is what did my grandfather in (undiagnosed) to be honest. It is often hereditary.

I have often felt almost like some sort of fraud. I have insulin, as just taking tablets was not working, but the process is to review my HbA1c, maybe once a year, which is a test that sort of gives an average blood glucose over some months, which is not that good a "picture". But (having lost some weight) I am on a low daily dose of insulin now. That has advantages (one jab) and disadvantages (cannot adapt to changing circumstances easily). I have tablets too. It is "mild" compared to many people.

However, when I started losing weight, I also decided to buy, with my own money, at a cost of some £100+ a month, a continuous glucose monitor (CGM). It sticks on my arm and logs interstitial glucose levels and keeps a history. It has its quirks, like only 8 hours of data (and some times I try and sleep more than that!) so has to be scanned at least that often for a full picture. It is also maybe half an hour behind blood sugar levels, so I can feel hypo when it shows higher as it has not caught up.

However, I have found it hugely useful with managing my diabetes and diet. It is really good for making me aware of the wrong things to eat (basically sugar) and what I can eat in moderation and get away with in, and how much I can eat of something without getting away with it. This is mostly feedback of history rather than "am I really hypo now" which a blood test can do.

Sadly they are not cheap, but I feel they should be used more. They are normally only prescribed for people with severe diabetes, but I can see they should be really useful, even for people just trying to control it with diet. It is a shame they are not cheaper and prescribed more.

Recently I was able to see what a "normal person" is like on one of these. That said, it was rather odd. A friend of mine (who will, no doubt, read this blog) was diagnosed with gestational diabetes. So I gave her a CGM (and another as she knocked the first one off on a car door, FFS). She did not want to do the requested 8+ finger pricks a day, so my treat.

The thing is, having used the CGM, no way is she remotely diabetic. This is one day (with her permission)... Yes, charge that battery FFS!


Subsequent days I have seen are lower than that. She does not spike over 7 even when eating stuff she knows she should not.

For me this was really interesting as I did not know what a "normal", non-diabetic person looked like on a CGM, and now I do. It puts my graph to shame, and I am well controlled (apparently).

Like I say, I almost felt like a fraud, until I saw that, and I know I am nothing like that good. My diabetes is mild, under good control, but very real. I don't feel like a fraud any more, and even wonder if I need fast acting insulin doses when I eat as I can peak at 10 mmol/l, and occasionally more.

Interesting stuff.

P.S. As requested, here is one of mine, on a really good day... Most days I am peaking higher.


2020-01-05

New printer (Canon PRO-1000)

I have had, and used, many printers over time (for paper, not 3D).

Just off the top of my head :-

  • Simple line of pins impact dot matrix through ribbon - classic. I had a few of these.
  • Single "pin" with rotating roller behind paper to do dot matrix through ribbon - slow - prints one dot at a time moving up through the character, then the blade shaped head moves right to do next pixel one dot at a time.
  • Band printers - I did not own one, but used one - has all the letters on a band, and it is fascinating watching the line form as each letter is printed when it is over the right space, printing many in the line at a time, so the line of text sort of forms in seemingly random order in front of your eyes.
  • Daisy wheel, impact print through ribbon, but fun doing some graphics with a lot of full stops.
  • A spark based printer, single wire high drags at high speed across the paper for each row burning off a silvered surface of the paper, dot matrix - creates a black on silver text.
  • A spark jet printer, with a carbon rod in a glass tube making a spark to the paper and carrying carbon deposited on the page. Single glass tube moves at high speed back and forth over the paper. Like printing with pencil. I did my degree dissertation on that.
  • A variety of thermal printers on thermal paper - head the width of paper. Fades rather easily.
  • A variety of thermal transfer printers, transfer from film to normal paper, head the width of paper.
  • A variety of thermal transfer multi-colour ribbon printers for photo printing.
  • The plastic card printer we use at work, thermal transfer. Ive used two kinds of such printers.
  • Normal A4 laser printers, postscript
  • A3 laser printer, colour
  • Ink jet printer
  • Bubble jet printer
  • Oh, and pen plotters
Wow, I have had a lot of printers. I suspect I have missed some out even.

Actually, I left out that I have used manual, movable lead type printing machines and done typesetting using actual fonts of lead type characters. Genuine upper and lower "cases". That was a long time ago. I must be old.

The printer I have used for a long time is a wax based printer - originally Tektronix, but now bought by Xerox. I like them as they are not as messy as using toner and do solid colours really well. You just drop in these wax blocks to load the ink, neat and tidy. They are OK (ish) for photo print, but for colour letterheads (which is why we got them originally) they are really nice. I even print red wax seals to use with an embossing seal, and well, it was actually wax.

The office moved on to other laser printers some time ago, and my printer here finally started playing up (sheet feeder issues), so I have decided it is time for a new printer, and I thought it would be nice to get an ink jet type printer, but why not get one that can do photos...

What I eventually got was a Canon PRO-1000. It can have a stack of A4 plain paper for the normal use cases, but can also take a variety of sizes up to A2, and do impressive high resolution professional quality photo prints, edge to edge, on photo paper. It does pretty good photos even on plain paper.

So, yes, it will be used for simple A4 prints most of the time. These days I print quite low volumes, and can alway use printers at work for printing something with a lot of pages. It also has separately replaceable ink cartridges, which I prefer. However, there have been occasions where we do want to print bigger than A4, mainly for circuit drawings, etc.

But yes, I can print really nice photographs now. This is printing an A2 map on plain paper.


I did consider getting the wider models, they can do roll based prints up to A0, or even bigger. You can print proper posters for adverts and the like. But no way it would fit in the man-cave sensibly. I was also not sure if it would do the simple A4 plain paper as easily. The PRO-1000 seems a good compromise. The print quality really is rather impressive and having the option of large prints is nice.

2020-01-02

EICAR test QR

It seems there is something of a standard test string for anti virus (wikipedia has more on this).

The idea is that systems that look for viruses will have this string loaded as a signature of a valid virus, and so react as such. This allows you to test virus checking systems without an actual virus being used. Obviously some systems may flag as "test virus" or some such, and some may not have this "standard" string.

The string is :-
X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*
So far, so good, but what people are doing (see tweet) is putting that in a QR code, e.g. this (feel free to copy this image).


[note the white space around the image is part of the QR code spec]

And then sticking it on a car, or a hoody, etc..

The result is that some systems that happen to log the content of QR codes they see, e.g. on CCTV and the like, promptly trip their virus detection systems. Ooops.


Of course this does raise questions of whether this could count as Computer Misuse, but then should such systems be reading QR codes off a hoody anyway?

P.S. My QR code generator is on GitHub if you want... It seems to be more efficient than most (though no advantage for this particular case), and has a lot of options (png, svg, text, binary, eps, ps, hex, data URL). Have fun.

2020-01-01

0

It is funny how we like to see numbers clock over, whether a simple anniversary or birthday, or the odometer on a car, or even years.

Until the year 2000 I do not recall anyone having any issue with the common way decades were numbered. The '20s were 1920-1929 (inclusive), and so on. No issue, no doubt, no confusion. Sadly the year 2000, being a change of millennium, caused many to say "technically the new millennium does not start until 2001 as there was no year 0". This is one rare cases where I err aware from "technically correct" for a change, and even question if it is technically correct. The years are projected back from a more recently invented calendar and indeed go from 1BC to 1AD in one calendar. So on that basis, yes, a millennium starting at the start of 1AD means a new one in 2001. But why consider the start 1AD not 1BC? All evidence suggests that was not when Jesus was born, if he existed, so you should probably consider the third millennium starting maybe spring 2004 [citation needed]. You are picking an arbitrary start point - why?

However, it depends which calendar you pick obviously. We use the Gregorian calendar, but other calendars (e.g. astronomical) do have a year 0 and otherwise align with the Gregorian calendar (for current years). So one can be "technically correct" and still have a new millennium starting at the start of 2000 without any difficulty.

My point all along was that the only reason to consider the change of millennium as "special" in any way is the base 10 numbering that we use, and that a clocking round of 1000 years happens very rarely (oddly enough, every 1000 years), and so the only logical point to consider "special" is when the year number "clocks round" 1999 to 2000. If you are not doing it then, then why even consider 1000 special, why not consider multiples of 324.6 years as "special"?

I had thought this was all old news, but, to my surprise, I see people even now on social media saying the next decade does not start until 2021 as "there wasn't a year 0", continuing this nonsense. Sorry, but (Gregorian) decades only make sense as "special" if you consider them to be the years ending 0 to 9 (inclusive), end of story. So if anyone says otherwise just say you are using the astronomical calendar which does have a year 0, and see how they cope with that. Good luck.

But this may also help (thanks to xkcd)


P.S. this was raging on twitter later in the day on the 1st, and someone even posted that "At age 21 you start your third decade", LOL. No, 1st is 0-9, 2nd is 10-19, and 3rd is 20-29. Anyway, for those insisting "it" starts in 2021, point out that what "it" is, in that case, is "the 203rd decade of the Gregorian calendar" and not "the '20s". The '20s start in 2020, end of story.

P.P.S. a reminder that it is '20s, and not 20's, unless you are using a possessive, like "The '20's greatest hits".



Anyway, on a more amusing note, it seems Bulb may have finally fixed my account so I can submit a meter reading (they have been messed up since I signed up for no apparent reason, and just emailed me to say fixed). Yay, so a meter reading is needed.

I was about to submit one, on 31st Dec, and noticed it was close... very close...

This has resulted in my spending many hours on the 31st Dec, turning on extra high power kit in the house for a while, and even running the tumble drier, wasting many pence worth of electricity in order to get this picture (well, maybe not wasting as it means gas heating needs less power as house is warmer)...


It is a thing of beauty, is it not?

To my surprise Bulb had no problem with my submitting the meter reading of 00000.

All the "there was not a year 0" people would say I should not consider my meter to have rolled over until 00003 (or whatever it was when first installed).



There is one clock I'd rather not reach 0 though :-