2022-07-28

Research on misdialled numbers

I am considering if there is research on this, and even if I should do such research.

This is slightly relevant to things like W3W. I don't think they did any research of mis spoken, mis remembered, mis heard, and mis typed, random words, to be fair. Words work well in context and are shit when random from a huge dictionary (especially when beyond most people's vocabulary and available in multiple languages). Heck, even in "context" the classic game of "Chinese whispers" shows how shit this is.

But conveying numbers is a totally separate issue - a much smaller space to play with, except people "group" numbers internally. The whole concept of phone numbers in UK works well with area code (a familiar number sequence) and then number. People cope well with local numbers. They cope well with numbers in neighbouring area codes.

It has been long known that people can "transpose" digits, and this is why some check digit systems (like used on credit card numbers) specifically target digit transposition.

But people can also group sequences of digits in various ways. The ways they are presented with spaces matter. They create patterns. People are good with patterns.

I am well aware of two distinct misdials, and they are quite different.

One is seeing 0XXXX 400 000 and dialling 0XXXX 400 400. This happens a *lot*. We changed to not even publish 0XXXX 400 000 so as to avoid this.

One is seeing 0XXX 0 112 112 and dialling 0XXX 112 112 0.

This latter one is weird, in my view. I don't grasp why it happens, but it happens around 4 or 5 times a month. People misdial Screwfix's number and get me!

This is really not something I expected, which is why I wonder if numbers being mis handled is a topic for research.

We know W3W is crap, but can research make normal grid references and phone numbers better, if we understand how people get them wrong?

2022-07-19

How did the aircon cope?

Well, it coped quite well.

The red line is the temperature at my desk, which I was aiming for 22C. In practice the floor ends up around 20C.

The interesting bit is the blip around 13:00 where it got colder. It seems that the aircon is not doing what I expect. In order to control the temperature at my desk rather than at the wall controller or the air inlet in the loft, I tell the aircon a target temperature. When too cold I tell it to to cool to 5C higher than the wall controller. When too hot I tell it to cool to 5C lower than the wall controller. This works to turn the compressor off or on. But as the inlet (pink line) got hotter we got to a point where telling it to cool to 5C higher was not turning the compressor off. This means that it is clearly using the inlet as a reference, or possibly inlet and controller average. The inlet is interesting as (being ducted) the air leaves my room, via some ducting in to the aircon in the loft, and the loft temperature was up to 40C, meaning the inlet temp sensor was way hotter than the air actually leaving my office.

This is rather annoying as there is an explicit field setting to tell it what to use as a reference and that is set to the wall mounted controller. The fix was to change my code to allow me to expect the reference to be the inlet temperature or the controller or average of the two. As you can see, that fixed it.

However, from around 14:30 it was not going below 22C. The thicker line is my code making the fan speed higher in an effort to get it to go down to 22C. The compressor was on and the coolant was cool. But it was struggling. My office got up to a roasting hot 22.4C :-)

All this was on one of the hottest days of the year. So, yes, I think it is working well :-)

2022-07-15

It's bad luck to be superstitious

How was it not Friday the 13th today?

I am not one for "bad luck", but today has been quite special, so much so I felt it worth some blogging, sorry.

It all started quite innocently - the locksmith was finally coming to fit an EL560 lock on the back door with my Solar System door entry control system. The plan for that door is that it will be "unlocked" during the day, only having access within our grounds. But locked when we leave and alarm set, and locked at night. This means we need a way to "lock" the doors at night. So I decided to install this - you press the button when going to bed and that makes sure all doors are locked.

Yes, it is a mess, and that is partly because of the first issue - there was a wooden stud in the way, even though I was sure I checked. So a lot of chiseling. Plaster and paint will fix.

Installing this should be simple, need live/neutral from adjacent light switch. It is a Shelly Plus 1 running alarm code and linked to the secure alarm network, so simple. Except things went wrong.

  • First off I managed to "borrow a neutral", which does not go well with RCBOs. But easy to sort, get live and neutral from the same circuit. I forgot the light switch was on both down and up stairs lighting circuits.
  • Then, well it did not come on. In fact none of the lights in the south of the house came on, it seems.
  • The light was fed from another light switch - now this is a house rewired only a couple of years ago by the previous owners, so I take no responsibility for this mess
The light switch from hell

Somehow this was not right, no power. I checked for loose wires, and no joy. OK to be fair, one earth and one live were not connected (!) but putting them back did not help. No power on any brown wire in the box, well, any wire at all. I even re-did all the WAGOs, and no joy. I even checked the RCBO in the consumer unit. No joy. But I realised one light was on. So the issue was between the lights.

Just to add to the "not actually Friday 13th", in the middle of this, when testing on another switch I was puzzled that the switched on the upstairs lights did not show live. I was using a simple (and I know they are iffy) electrical test screwdriver, and it was showing nothing. It seems, in the middle of my trying to diagnose this, my screwdriver had actually failed! I got a new one from Screwfix.

At this point it was time to bring the sparky in - wiring an extra switch I could easily do - solving this was getting beyond me. I mean I have A level physics, and a degree, and I could work it out I am sure, but really, to be on the safe side, getting someone who knows the conventions and rules for wiring a house was in order.

So the sparky tried to trace the issue - it looks a lot like a break in live from the light that is working to that light switch from hell mess. But not completely open - showed some voltage. Great. But then we realise the cloakroom was on, and that is a second light on the same circuit, so let's test from there. One touch of the wiring and the circuit trips.

We have now moved from an open circuit to a fault that trips an RCBO, but takes maybe a second to do so. The second working light had been working, but the wiring was in an old ceiling rose shoved behind the ceiling which promptly snapped, so replaced by a nice new joint box and WAGOs. If that was the cause of the fault it should all be sorted now. No such luck!

Still tripping. Arg. OK, so on to the light switch from hell as pictured above. So the plan was to try and work through its nightmare circuits one by one. First step, chop off the 8 way WAGO for 8 of the live feeds. Test, and WTAF, all the other lights in the house now working. The fault is gone, and the short is gone, and the open circuit is gone.

OK, test them all, and not problems found, to connect back one by one, and well, all 8 back and all working. These are a number of outside lights and stuff. But just reconnecting all 8 live wires to a new WAGO and it works. Arrrg!

Then we discover that somehow tripping the RCBOs has killed 5 of the Shelly in the lights, so next to work on that - but in the morning. That should not happen - I know some of the shit involved in electrical interference immunity testing for CE, and this should not kill a Shelly, really!

So that was the power nightmare - somehow two separate and unrelated faults happen when I am doing something minor, and one remains 100% unexplained. In the end, my switch and indicator are fine, as I had wired them.

Now for the lock fun - a simple job as the locksmith had practiced on my office door and the outhouse door already and was all over confident "simple, 2 hours". Then he realised he did not have to drill the whole door width for a power cable as it could be done with a groove, and that the keep was the same place and size for the new lock, and did not need changing. So he was all super confident this would be easy. Spoiler: it was not.

It went OK to start, but then the lock was not working, and the handles not working, and then the keep not engaging, and then the spindles somehow started to slip and stopped working, and well, everything went to shit - and he is on the clock, after 6 hours here, he has to catch a plane. So we now have a door that is "technically" secure, but far from pretty until he gets bank from holiday. With him, and the sparky, alternating in finding impossible shit happening with their jobs, at the same time.

So seriously a lot broken, a hell of a lot. Stuff totally unrelated all happening at once.

Then, to top off the day, just as things calm, and I decide I am doing nothing now but watching TV and drinking whisky, on basis they cannot go far wrong, I get this from my son.

Is this a bad sign, dad?

Err, yes! But at least he has a working screwdriver! It is a really sensitive LED based one though, so lights up if you sneeze too close to it - turns out he just had a loose wire in the switch.

P.S. Some credit to Shelly. Whilst we have had, in the past, some Shelly 1 struggle with heat and fixed by replacing with Shelly Plus , and I am pretty sure we have had a Shelly die before. On this occasion it looks like it was actually intended behaviour - tasmota code has a last ditch recovery mode config reset if you power cycle it several times quickly - which is, of course, what was happening, and resulted in four of them simply being factory reset, and hence easy to fix.

2022-07-02

A flaw in GDPR

One of the aspects of the General Data Protection Regulation (GDPR, and UK GDPR) is that you can expect that the personal data an organisation holds on you to be accurate.

Specifically, that if it is inaccurate, you have a right to rectification, and you can require them to correct it and make it accurate (even if the ICO don't quite understand that, it is the law).

This is important if the information is mistakenly wrong, but also if it changes over time...

  • If you move house and your postal address changes
  • If you change your name
  • If you change your gender
  • If you change your title
  • If you change your phone number
  • If you change your email address
  • Etc...
(obviously if someone has a record of "the postal address you had when you signed up", then that does not need to change just because you move, unless it is a mistake, but a record of "current address" needs to change when you move).

The organisation has to, legally, rectify the inaccurate personal information they hold on you when you ask them to. That is the law.

But, in my opinion, there is a flaw in GDPR. When "signing up", "registering", etc, when first becoming a data subject with an organisation, it is apparently legal for that organisation to impose rules on what they consider acceptable personal information.

A perfect example is, apparently, British Airways, this week, refused to accept someone that was female and a Doctor, as the gender and title did not match!

But organisations will decide someone cannot have a first name that is one letter, of that you have to have a first and last name, or that your email address cannot have a dot before the @, etc.

Of course, the person could have recorded themselves as male and a doctor, and having been accepted they could require the incorrect personal information be corrected, under GDPR. The same is true for email addresses that an organisation decides is not valid, or a phone number, or postal address or name, etc. Ultimately, legally, they have to accept the accurate personal information in the long run if you required them to rectify the inaccurate personal information they hold and collected at "sign up".

But it seems nothing in GDPR requires that organisations accept the "accurate" personal information from data subjects "in the first place". They can make any arbitrary rules they wish. So we see shit like this, even for perfectly valid email addresses.

To be fair, companies can, and should, validate that something like an email address is valid and is the subject's email address. That is part of GDPR when it comes to rectifying personal data as well. But if it is valid, they should accept it, in my view. Making random rules on names, genders+titles, email addresses, phone numbers, etc, are all stupid and should be fixed by an update to the law.

I feel GDPR (or UK GDPR) needs updating so that no data controller can discriminate (i.e. refused to accept a new data subject) based solely on the format or syntax or rules they have created relating to any valid and accurate personal information at the point of becoming a data controller, any more than they could at the point of being required to rectify inaccurate personal data later.

The fact this is not part of the GDPR, is, in my view, a flaw, that needs fixing.

I have written to my MP asking for this, maybe you could too?

2022-06-29

The round one

As previously blogged, I created an NFC RFID reader based on the PN532 NFC chip.

It works well, and includes red/amber/green LEDs and tamper switch and even contacts for a "door bell". This makes it ideal for access control.

But I decided some cases may look better with a round modules. So I wrote code to measure track lengths in KiCad PCB files, and then code to make a spiral track, which I made the same length, and then made a round version of the same thing.

It works. It worked first time. Indeed, the solder paste and cook worked first time - no re-work - no glitches - just worked. I am really pleased.

One of the small tweaks was around the reverse mount LEDs which used to tombstone in the oven - that is all fixed nicely now.

Other changes are that the connectors are all SMD now to make the other side "clean".

Which leaves me wondering if I should add a logo or something on that side. I am really not sure. I also think purple solder resist may be nicer. The main thing is I want a distinctive appearance / brand that can compete with elechouse on Amazon. Suggestions welcome.

Of course, what is super frustrating is that these are all prototypes - I cannot really make commercially until the global component shortages are sorted and I can actually order 100 of the PN532 or indeed anything else! Once sorted, I plan to put these on Amazon.

2022-06-28

ICO and NHS

I have a short email address. Those that know, know, so not posting here.

Suffice to say it is of the form x@x.xx so is only 6 characters. It is 100% valid. I have used it for a couple of decades now - this is not new. I am not alone. [side note, I tried to sort x@xx email, which is not easy, and did not get off the ground, but some people have done this, and it is valid]

I registered to get access to on-line COVID passes with the NHS or is it NHS Digital, or what? To be honest it is not 100% clear. Privacy policies and the like should make this clear, but even now I am not sure. My MP believes it is Welsh government. The fact I am not 100% sure is part of the problem.

[update: https://access.login.nhs.uk/privacy says it is joint data controllers of the devolved (Welsh) administration and NHS Digital]

They would not allow me to register, so I created a temporary address (longer) and registered. Simple. I even have the whole domain rfc2822.uk for this purpose.

I then tried to change the email address to my normal x@x.xx email address, and their system would not accept it.

NHS expecting me to change my personal data to fit them

So I emailed their data controller requiring them, under my right of rectification under (UK) GDPR to correct my email address. They refused. Note the original (temporary) email is no longer valid, and hence meets the definition of not "accurate" personal information. Indeed, I do not even have the domain any more.

I wrote to ICO, and have exchanged several emails to ICO, and escalated and asked for review of the case.

Basically the ICO said: There is nothing in data protection legislation that prevents an organisation from having a system that has a minimum requirement for an email address.

This seems odd, as how can an organisation accurately record personal information if they do not accept a valid email address, i.e. they have a "minimum requirement" for what is "valid"?

This has gone on for some time, and I am not alone, there are others I know with similarly short email addresses that have issues with NHS (and other organisations). There are others I know with related issues on incorrect data validation at "sign up".

Just to be 100% clear, the NHS fully accept my email address is a valid email address, and have emailed me, to that email address, to say so, as have ICO.

I also asked ICO more generic questions about whether an email address is personal information, and if I can expect (require) an organisation to correct it when it is wrong. They confirmed that is the case, so I again wrote to NHS quoting them - no reply. What a surprise.

I have written to my MP as well, and asked them to chase, and they have written to NHS (Welsh Government).

Latest from ICO is "For clarification, as the NHS has not recorded your email address then we are unable to suggest that they are recording inaccurate information. 'Inaccurate' would apply to information that was recorded incorrectly. There is no suggestion that they have done this."

Seriously, I'm shocked. This has, all along, been about the NHS refusing to correct my email address. So I have explained, again, to the ICO, that the NHS have recorded my wrong email address and are refusing to enact my request under my right to rectification to correct it under UK GDPR.

We will see how it goes, but this is a matter that relates not just to email, but other things.

  • Organisations will insist someone has to meet some format for a name - a forename and surname (not all have this), a name with more than one letter (not all have this), etc.
  • Organisations will insist a UK mobile phone number has to start 07, and organisations will even blacklist some operators 07 mobile numbers as not valid mobile numbers!
  • Organisations routinely try to impose rules on email addresses.
  • I really expect organisations to have shit when it comes to recording gender, which is rather topical.

The law does not stop companies from having rules to take their service as long as not discriminating based on some protected criteria. They can refuse me because my email address is too short. IMHO this is wrong.

But once they have accepted a customer/client, perhaps with wrong, or temporary personal details, they do have to comply with GDPR and have to correct incorrect personal information. So it would be better if they accept the correct personal information in the first place. In seems to me that GDPR (or UK GDPR) has a flaw in not covering this properly for "sign up". People should not be able to use email address, mobile number, name, or gender, as a reason to refuse to accept a customer/client.

This is even more so when it is not some company, but an organisation like the NHS. I have an NHS presence, I have to, as a UK citizen, and they have data on me right now that is not "accurate". That needs fixing.

Update: 1st Jul: The ICO now seem to be suggesting that because the email address they recorded at the time was correct (accurate) that they do not have to correct it now that it has become inaccurate. This would suggest organisations do not have to update name, address, phone number, email address, well, any personal information they hold when it becomes inaccurate over time. That seems a stretch!

Update: Someone has suggested this is "the same on all GDS platforms", and that it is not fair for me to "bother" the NHS. I appreciate the NHS have a hard time, but if this is the case the all the NHS have to do is contact whoever maintains their platform for them, explain they have a legal requirement to correctly record personal data, so have 30 days to "fix" this, and the NHS will have done what they need. Instead the NHS have so far chosen to spend time arguing with me, and then updating their site to state that an email address has to be at least 7 characters (previously it accepted it but did not send the confirmation email so it did not work). At the end of the day, someone, somewhere, on some platform, just has to change a 7 to a 6 in some code (or better still, follow the RFC for validating email, which will be a simple regex or library). It is not a hard fix for whoever does it - if the layers of people above that, all the way to the NHS, simply tell them to do it.

2022-06-19

Euro profile locks - a few tips

Test door, lock sticks out a bit!
Euro profile locks are a doddle to change - it is literally one screw, and you can slide out the old lock and slide in the new locks. They are easy to buy, or order on-line. But a few quick tips:

  • You need to order the right size (inside and outside). This is the distance from the centre of the lock to the key slot each side. Usually available in 5mm steps. You want it just right, not sticking out, though good locks have a snap off part if someone does take pliers to it.
  • You can order keyed alike locks so they all have the same key, which can be very handy. Somehow people don't realise this!
  • There are loads of different quality and prices of locks.
  • You can have key both sides, or one side with a thumb turn, or just keyed one side and blank the other side even (half lock).

So, when we moved in, the first thing I did on day 1 was order new locks. Five of them. The house had all been recently re-done and the locks in the house already were all brand new, so a bit of a shame. But I wanted higher quality locks and did not want five different sets of keys (don't people know you can get locks that have the same key?).

Cheaper lock

I ordered like for like, the same size, and keyed both sides as that was what was in the house.

Unfortunately, after a little while we realised the choice of locks was wrong. You do not want keyed both sides. The reason is that the doors were all multipoint locks, and only locked by turning the key. Without that someone can literally walk in from the street (which has happened!). But this means you can only use the door from the inside if you have a key to unlock it. This means if there was a fire in the night when the doors are (obviously) locked, you need a key to get out. Remember the house originally had 5 sets of keys so you need to find the right key for the door by which you are trying to escape from a fire.

The short term fix, a key on a hook by the door, but that is far from ideal. Obviously.

Thumb turn on inside
So I ordered another complete set of locks (getting expensive now) with thumb turn on the inside. This means you can always lock or unlock the door easily from the inside without a key. Importantly it is not hard to unlock if trying to escape a fire. I actually disposed of the first set of locks as no use to me any more.

As some of you know, I have a complete door access control system and alarm system, but changing locks means a locksmith and time and money, so even though we have been here over a year I had not yet changed the locks. I was also researching the right lock for the job. Being a house, I really don't want an "exit button" and an "emergency break glass" by every door. I also wanted a "fail secure" so a power cut when we are away (long enough for battery to drain) does not leave all the doors unlocked. But I also want it "safe" so always possible to get out in a fire even when nothing is working electrically.

So the locks are Abloy EL560. I have been testing on my office/study door, meaning we now have six locks. This is great, it is fail secure (i.e. power fail is locked). But it can always be opened from the inside with the handle, and with a key. It also has signals so you know if opened by handle or key. This is great.

RFID reader
They are being installed in the rest of the house shortly, with my AES DESFire based readers to open from outside - nice and secure, and all links in to the alarm system.

However, if it was to be opened from outside using a key, I would want the alarm to know this, and disarm. The idea is we can trust a key. That is fine, I simply configure the system to disarm when the key is used. The system is very flexible and easy to configure for various types of lock.

Except... I have thumb turns on the locks on the inside. I don't need these now as you can open the door with the inside handle. But if I set the system to trust a key it trusts the thumb turn as well. Someone breaking in only has to turn the thumb turn on any door to disarm the alarm. So yes, obviously, it is not being configured to do that (yet). Indeed, one approach would be you have to enter a PIN on the keypad if not using a DESFire fob, even if you did use a key. Given how easily keys, even high security keys, can be copied or 3D printed, this may be sensible anyway.

The real answer is order locks keyed both sides, or even no key at all on the inside now.

I had a full set of locks just like that from when we first moved in. I have disposed of them. Arg! So another set of locks, six this time.

The moral is never throw anything away, ever!... I should know this already.

2022-06-15

IPv8?

I should write up my concept for IP. This is literally stuff I dream of!

This is totally "if I had a time machine and could fix IP at the start" and very much not a "this is what we should move to". The time taken to get IPv6 deployed (over 50% in US now) shows this would never fly.

So my ideas is this...

IP addresses would have multiple levels, tagged at the binary level in some way to allow each level to be different number of bytes, and allow for multiple levels - perhaps top two bits say length of each level. The exact detail on this is not that important other than the fact it is "variable" in some way and a fixed pattern for any IP address to allow hardware to cope. The displayed format is not that important either, but probably a series of decimal numbers with a separator.

The top level of any target IP would be AS number. This is still routing packet by packet, not session routing. So an ISP level core router needs a simple top level binary decisions, is target outside our AS (so send to target AS) or within (so send to next level at byte X in packet). Yes, a router could have more than one role as more than one AS maybe, but in general it is simple. This is the sort of thing that can work at a hardware level in ASICs without too much issue. A CAM at top level for sending to AS and a CAM for "within my network (AS)" level.

Routers below this level are similar - "is it my network" do routing to "next level", or I send "upstream".

The concept is that the IP would actually go in levels from AS, to areas within an AS (if needed), to customers, to devices on customer networks, and even include "port" within the device. No need for NAT ever. Ultimately extensible at ISP or customer or network level and even within device to allow more ports (which can be an issue). Some limit on levels, and bits at each level, but more than enough.

Yes, TCP and UDP would change to not have a port where it is now, but part of the IP addressing.

As for allocation and RIRs, the allocation would be AS, and anyone with an AS controls as many IPs within that as they need.

More ports

Also, the session connection to a device would be a protocol in itself, for things like TCP (maybe even UDP and others), where the connection is to the device IP address (not a port level) but the payload includes a text port name. So https would be to port "https" not port 443. The reply would confirm the actual target IP (which includes port ID) to use for that connection. This allows target port to be unique without mapping the source IP/port and target IP/port normally needed to identify TCP socket within a device, even as a server. So simpler code (yes, check the IP/ports are right when you match it).

I also think that, unlike IPv6 which has a separate standard header for encryption (which therefore does not actually work for TLS we have now) I think TLS would be an option in the that SYN, along with port name. So port "http" can request TLS or not as a standard start of that session, with things like "use previous authentication session" as an option at the SYN level for faster connections. Ideally the application calls for TCP would make it very simple for any stream to be TLS or not with minimal coding overhead.

Multihoming at TCP level

Also, you need protocols like TCP to be multi-homed at the TCP level. Mobile phones can already do this to some extent. Connect to a name, not an IP, and it has multiple IPs, but allow the IPs to change during the session if needed, either end, as part of the connection protocol. This avoids the need for multihoming at the BGP level, and IPs can start with AS at a top level regardless.

No, not "IP is the route" - still routing path redundancy

Just to be clear, this is not saying the IP is the route to the end point, either. The route taken would be determined by routing protocols like BGP, and still have the alternative paths and redundancy that exists now to get to an AS, and within an AS. The only really difference is that core routing policy would almost certainly not allow announcements below the AS level, and hence keeping routing tables smaller. At present IPv4 does not work (by policy) smaller than a /24, for the same reasons. This just makes a really simple and obvious policy as the inter-AS level. It also means each AS only needs to originate one prefix (their AS) where as now they originate loads of separate blocks. That one prefix is extensible as much as they like. Indeed, the role of RIRs for IP management would pretty much vanish as having an AS would entitle you to allocate IPs under that AS, and originate routing for those IPs from that AS.

Comments and discussion welcome - but remember this is essentially just a thought exercise.

Alternatives rejected

This idea is still per packet - a totally different approach involves a connection based system. Establish a route over the internet as you connect and each point reports the connection id, and at the start you send to the local connection ID which is mapped to next hop connection ID. It could work but is a massive amount of "state" in the core, and I don't think a viable approach. Sorry.

2022-06-12

Un-mapping CGNAT for IPv6, etc, with no overhead?

Some links are IPv4 CGNAT (notably Starlink at the moment).

What if I want IPv6.

Well, a simple approach is something like L2TP. The issue is that this is IPv6 over L2TP over UDP over IPv4 over CGNAT, and that means a much reduced MTU. This is not the end of the world if you know you have the reduced MTU, but a shame.

So would there be a way, even if only for outgoing sessions, to "tunnel" the IPv6 over IPv4 with zero overhead?

I think so. You need something in the Internet with IPv6 and something on the network that is stuck behind the IPv4 CGNAT end point.

The trick is that an IPv4 header is 20 bytes shorter. So if you reduced an IPv6 packet to IPv4 you get 20 bytes spare. You could tack something on to the end of the packet, and it should arrive at the tunnel endpoint via the CGNAT over IPv4.

So why not tack on the original external IPv6 address?

Yes, the source address would have to be tracked using NAT of the IPv6. And indeed, one could say you only need to tack on the original IPv6 at the start of the session as once you have created the NAT session you know both IPs involved.

Only catch is that TCP, for example, does not normally have extra data on SYN packets, so you have to check they arrive intact. One would hope so.

OK scrap that - I have a better idea :-)

You establish a link, maybe simply a TCP session, between your devices, outgoing via the CGNAT IPv4 to the far end. You use this to carry some control data. It could even be TLS with client cert, etc, for security.

Every outgoing session you create you send details over your TCP control link to the far end with details of that session, and it allocates an IPv4 UDP or TCP port for you and advises over the TCP control link. You can then send a new packet of the same type, e.g. TCP/UDP, to the far end. This then maps through to the IPv6 addresses you want to use, and the original ports. The CGNAT is then used to carry each new packet in or out over the session map each end.

What makes this even more clever is that you can do incoming new sessions as well. Obviously the IPv6 block being used has to be routed to the outside device. But simply carry the details of the new incoming connection over the TCP control link, and then have the NAT end start an outgoing session. Once established, that is used to carry the incoming packet and the outgoing reply packets. Yes, it could make for a slight odd TCP handshake, and that may need slight messing, e.g. send the SYN and SYN/ACK over TCP control link, but create an outgoing SYN and SYN/ACK exchange over the CGNAT outgoing, setting up the payload sequence numbers to match what the ongoing packets will be using.

This could allow full IPv6 both ways over an IPv4 only CGNAT, and not only would it have no overhead, allowing full MTU, it would actually use fewer 20 bytes per packet after the first control packets, and so be 1.3% more efficient on the wire. Technically you would lose per packet flow labels, but not a lot else. Of course things other than TCP and UDP would be a challenge, but could simply be mapped to look like UDP on the IPv4 if needed.

Of course the same could be done for IPv4, not saving 20 bytes per packet, but basically un-mapping the CGNAT at each end.

A zero overhead tunnel could save one of the classic IT/networking issues "MTU".

2022-06-10

Starlink (2)

As I say, impressed, and to be honest even at £89/month inc VAT this is good value...

A doddle to install properly, some tree cover is unavoidable, but it works, and is happily doing 185Mb/s right now. Thanks to Jim to coming round with his drill for me.

The IPv4 is all CGNAT, and the IPv6 is not working yet or reliably. But this "just works" with A&A L2TP providing fixed IPv4 and IPv6 over it. And the latency makes this a very viable service at around 30ms. The package of a Starlink + FireBrick + A&A L2TP is pretty appealing, to be honest.

I really am pretty gobsmacked with how simple this all was to buy and get working - only caveat is "remember to get the Ethernet adaptor".





2022-06-09

Security by annoyance

TOTP (Timed One Time Password) uses a secret which is a block of binary data. It suffers from being symmetric in that the server doing the checks, and the user (usually in an app on their phone) both have to know the same secret.

This presents the problem of getting this secret from one to the other. This story is about that journey.

A friend has a customer that wants to use some specific device they already have to provide OTP codes, and so has to send the secret to him to load on to the server.

Firstly, this is not quite how it is usually done - the usual way is the server makes a (new, random) secret, provides it to the customer (often as a QR code), who confirms receipt by entering the current OTP code. All done over https to be secure. Sorted.

But in this case the customer had the secret and wanted to send it.

As I say, the secret is just a small block of binary data, usually expressed as base32.

They sent, by email, an RFC6030 XML file, signed, with base64 coded encrypted secret in it. They also sent a simple 16 byte binary file which is the key, by email.

  • The base64 had to be converted to binary
  • The first 16 bytes extracted as IV
  • The remaining bytes stored, in binary, as the encrypted data
  • The IV had to be converted to hex for openssl command line
  • The 16 byte binary key file had to be converted to hex for openssl command line
  • openssl command line for AES-CBC was run and produced a binary decoded file
  • This had to be converted to base32 and loaded on the system as the secret

Was this secure?

Short answer: no. Both parts sent by email. Sent as separate email as obviously no hacker could possibly intercept both emails!!!

Alternatives?

Almost any secure messaging app, e.g. signal, interactively whilst talking to the contact, sent as base32, and ideally as a disappearing message.

Or email the base32 over PGP encrypted email.

Or, well, almost anything else! Reading the Base32 over the phone FFS (though phone calls are rarely encrypted in transit over the network).

Storing the key?

One thing done on the back end system, apart from encrypted disk, etc, is to store the secret encrypted with the users password and salt. The password is not stored, only an argon2 hash. This means you can validate a password+totp code, but you cannot extract either the password or the TOTP key from a leaked copy of the database. I mean, yes, if someone can get a copy of the system and database, and trojan a user in to entering password, they can decode the TOTP as well, but that requires that extra user interaction step for each compromised user record.

P.S. what is my involvement?

Mostly helping my friend out to understand what he had to do. We are all cautious to ensure security, as you would expect.

Shelly Plus 1 GPIO

I am quite impressed with Shelly stuff anyway, but the new "Plus" range has allowed some interesting developments - as they use ESP32, which is the processor I use for my access and alarm system.

This has meant I am able to add bits to the alarm system much more simply than using my custom boards - anywhere I need just an input and/or just an output, for £15.99 I have a device that does the job. As an output it is a dry contact relay, and can be powered from 12V DC, or 24V-240V DC, or 110V-240V AC, so very flexible. Working with alarm system 12V DC or 24V DC is easy.

The input is run at "live" so for mains working needs a proper switch rated for mains such as a light switch (or one of the nice retractive switches). But working on alarm system 12V, a simple contact like a reed switch, or connection to a PIR is simple.

Obviously there are plenty of uses for the alarm system that needs more than one input or one output, but this is pretty useful. I even have one electrically locked door that is just using one for the lock and door open inputs as it is a door that does not need a fob, just needs to be locked when alarm armed, and detect door open as an alarm input ("access", causing alarm if armed).

My new EL560 locks on the house are great as no need for break glass or exit button as they open with the inside handle. But if you have any doors you keep unlocked during the day, you need a way to lock the doors at night. And that needs a button. Well, a Shelly Plus 1 is ideal. The light shows the doors have been locked.

I have also worked out the GPIO:-

  • GPIO0: Output for small LED (hard to see through case).
  • GPIO4: Input for SW signal (external switch) but needs configuring as pull down.
  • GPIO25: Input for small blue button on the case, but needs configuring as pull up.
  • GPIO26: Output to work relay.

There are also 6 external connection pins for programming (WARNING: THESE CAN BE LIVE):

  • GND
  • GPIO0
  • EN
  • 3V3
  • GPIO3 RX to shelly
  • GPIO1 TX from shelly
  • GPIO16

2022-06-06

A bit more on air-con

I have been tweaking the air-con even more.

The control I have, basically, is to set the target temperature. I can set higher than now, and lower than now, and so make it turn on or off. But sensibly the air-con has controls. It does not run the compressor exactly to match what I say, it turns on for a minimum time, and there are also sorts of laggy effects for the temperature to react. Once compressor was on it stays cool a while and then fan blows cool for a while.

But it is getting silly.

For the last hour or so, even more so. This is not like ±0.1℃, it is more like ±0.05℃.

It is really good. But I was thinking, is this just that it is a cold day in Wales and the temperature happens to be settling where I want? At 22℃?

Well no, it is not that, looking at the power usage I see that it is turning on and off the compressor to work the temperature.

What I did was make it predict on a trend the next 2 minutes and turn on/off (set high/low target) based on that. And it was surprisingly good. This is the power usage showing it is fine tuning it to manage that. It is controlling the temperature to well within ±0.1℃ with no problem.

Wow.


P.S. Just to clarify (as someone asked), this is not switching between heating and cooling (which would use more power), this is turning cooling on and off. It will switch to heating, but only after quite a while of temperature not coming back within range when off - and that can happen on some days in spring/autumn, but usually only once (each way) in the day. It also does automatic fan control.

Starlink

Finally got one for testing, so first experiences.

1. Ordering was simple, but the site is one of these "slick modern sites" that are a pain to find any proper information - very easy to order - not so easy to realise an Ethernet adapter is separate. I have finally ordered one now.

2. Arrived very quickly (ordered 1st June, arrived 6th June, Jubilee in the middle).

3. Very very confusing re "account". The pages all have "sign in". I cannot find any "create account". What I did not know is that when I ordered by simply clicking "apple pay" and double clicking my phone (yes, very slick), is that this "created" an account using the apple pay email address. Once I worked that out I could use the "locked out" link to get a password. They could make that a lot clearer.

4. The instructions are simple, you plug it in and point at the sky and connect to wifi - it makes you then create an SSID/password, but then it works - Internet, several MB/s of it. NAT / IPv4, but yes, works. I now need to find somewhere vaguely sensible to put it - away from trees - may need a pole on the wall.

Of course the next steps are to get the Ethernet adaptor, and start working with FireBrick, see if I can get an IPv6 address; see if L2TP/IPsec/etc tunnels work over it; etc, etc... So R&D to do.

P.S. LOL, you can get to RT.com via Starlink :-)

2022-06-02

New air-con, part 6

Well, well, well... I am impressed with 4 Seasons. They have only gone and sorted it.

This is the huge difference between this install and the previous Mitsubishi install from a different company. The customer service to get to the bottom of the problem and fix things.

Firstly, they did discuss this "anti-freeze" mode with Daikin, and whilst we all agreed that turning the fan down was silly, their point, which is very valid, is that it should not be happening at all. The refrigerant should not be getting so cold in the first place - this is a fault condition. That is what needs fixing.

Naturally I leapt to the conclusion that the issue was air-flow. But I was wrong. It seems the advice from Daikin at this point was right - they said to check the refrigerant. Well, pressures had been checked several times. But they suggested there was too little refrigerant. Indeed they suggested (counter intuitively) that we would see these lower temperatures if there was not enough refrigerant. That seems crazy, obviously.

Well, the installer, and I, were rather baffled, but the only thing that basically makes sense is that the system is (over) compensating for the lack of refrigerant, and that is how it ends up colder. And, of course, at the same time, it is unable to carry the heat away properly, so does not work as well.

The symptoms were pretty simple, when cooling, the refrigerant quickly goes down to as low as -10C, which it manages for a few minutes, and then the anti-freeze mode kicks in, and everything stops for 10 minutes, and overall things are not working well.

Now, the refrigerant is settling around 5C with no problem, and cool air is coming out, and the room is quickly cooling. It is not going negative at all, not even getting below 4C. It varies, and I have seen 5C to 8C, but that is "sane".

So what did we fix?

The fix was to completely drain down the system and refill it - simple as that. The big clue was that only 0.9kg of R32 came out. It should have had around 2.5kg. So he emptied, vac'd, and refilled. What I had not appreciated is that these systems come pre-filled for installation, so it seems that somehow it must have come without the right amount - some must have leaked in transit (not good). But the only real test he can do is pressures, and they were fine, so no clue that there was not the required amount of R32.

I'll add some graphs once we have a couple of days running to confirm. But I am sat here on a warm sunny day in a nice cool office. To say that I am over the moon is putting it mildly.

Side effects

A small side effect is that my Daikin wifi modules have way more monitoring and reporting and graphing now. I did loads of work to try and find what was actually happening, turning in to a seriously useful tool.

Next job is the new controller to work with it for when people want manual control but doing a better job than the normal Daikin controller.

Another small side effect is that my previous very tight temp control is now a little less tight as the effect of cooling (or heating) is much more and continues longer, so some level of predictive processing may be needed to manage tighter temp control. Should be pretty easy.

2022-05-24

New air-con, part 5

After the fiasco of the previous air-con install, I was pleased with the new one, as you can see, and especially that I can make my WiFi controllers.

However, this month, as we have got warmer, I have run in to an issue. It was gradual, kicking in some afternoons. Initially it was something that fooled my temperature control, causing it to flip to heating and back to cooling...

I made the controller a bit more reactive, and fixed the flipping to heating, which was good, but still, in the afternoons, things were not right.

Will all this, over night was OK, mostly, but not always. After a while I realised the clue was in the power traces. Basically, after a while the unit switches in to a mode where it cycles, around 11 minutes on, and 11 minutes off, and a low fan when off too.

This is not enough to keep my office cool, and as we go on, it is getting worse and worse. It always starts the day OK for some minutes, but can go in to this mode right away after that.

This is crazy. I tried all sorts. I thought maybe it is the "econo" mode, which apparent limits cooling when below a certain temperature. I even fudged the temperature (replacing thermistor with a resistor) to test this, but no joy.

After a lot of testing for a couple of weeks, I have found it. It is an "anti freeze" mode. This is what they say (when you know what to google). Needless to say that Daikin support did not work this out for us.

What is especially annoying is that when it is pending 10 minutes with "thermostat off", it also puts the fan in what is shows as "LL" mode, i.e. slower than "L". Well, if it want to avoid the coil freezing you would think it should keep the fan at "H", but that is not what happens.

So now we know what it is, well what can I do?

For a start, I can try and pre-empt the anti-freeze logic, but that does not really solve the problem of why it is freezing the first place. It would have helped a lot if Daikin could have simply said "that looks like anti-freeze kicking in" when asked. Indeed, a fault/indicator on the controller or controller app would have helped so we knew the problem.

Even so, a way better outcome than previous air-con install, and some hope it will be "just working" soon.

Update: Clearly air-flow is the issue, but by preempting the anti-freeze I can keep it on full fan more and so have a slightly improved performance for now.

Part 6

2022-05-19

Not using solder paste?

I made the mistake of ordering a board and not ordering the solder paste stencil.

So what to do - well, not a complex board, I tinned the pads and applied some flux as it is a bit sticky...


Then I applied the components. Now this is where I wish I had applied less solder to make it flatter as they did not stay put very well.

Getting flux on the tweezers was also an issue, so lots of cleaning.


Then I cooked in the oven the same as if I had solder paste.


To my utter amazement, it worked perfectly. I did put in a sonic cleaner, which did not help much in cleaning it up...


However, a nylon brush and some IPA, and bingo..


But the damn board works!

So there you have it - an option when you have no stencil!

2022-05-09

DHCPv6

First off, an apology! FireBrick and A&A have been pioneers in making IPv6 work in practice in the UK. The current FireBrick operating system has been designed with IPv6 from the start, and not as an afterthought. This means that at every stage the notion of an IP address comes with the notion of "which sort of IP address". We are proud of that. But we have been lax in one notable area - DHCPv6. Sorry.

The problem is that IPv6 has a mechanism, SLAAC, which assigns an IP address on Ethernet. It is very simple, and works. The router sends out some details (which can include DNS servers, etc, as well), and allows devices to pick an address. With 64 bits of local address, this is easy to do in a way that avoids any collisions. Indeed, devices often pick lots of addresses, cycling them quickly, to hide how many devices you have on a subnet, etc. (privacy addressing). It works well, and in the UK it was pretty much the only thing being used.

DHCPv6 is a "stateful" way to manage addresses, and like DHCP (for IPv4) it has a lot of options. Thankfully IPv4 DCHP has been around a long time, and the sane options to implement are clear. The FireBrick IPv4 DHCP server is actually pretty flexible and powerful and works well.

But DHPCv6 is newer, and has a lot of options. One of the big ones is "prefix delegation" - telling a device it has one or more blocks of IP addresses to use on its interfaces to give out to devices it has connected. This is just one of the complications for a DHCPv6 server on Ethernet - it has to consider not only allocating and tracking an IP address, but a block of IP addresses and routing. As a client we also have to consider if we want to ask for a block of IP addresses and what interfaces we can use to assign those IP addresses.

We do this already, but we only do it for the LNS/PPPoE (broadband) side of things. We work as a server when operating as an LNS, linking to RADIUS and routing. We work as a client when PPPoE, allocating a dynamic block of prefix delegated IPs to the LAN interface. So we have a lot of the tools already.

Until now we did have an option on our router announcements on IPv6 over Ethernet to allow a DHCPv6 server, and even an option for that to be the FireBrick itself. The idea was the FireBrick would issue an address the same as SLAAC. This was specifically to allow for devices that only worked using DHCPv6. But we really did not find any such devices. Devices (like the FireBrick itself) that only handle SLAAC are actually more common from what we can see.

But the time has come for DHCPv6. And there will be several alpha releases around this. We need to take careful steps to integrate the code which currently only works for LNS and PPPoE to be used for Ethernet. We need to decide on the config for this, and the options we offer. This will not be simple.

So, watch this space. It will take a while as we are working hard on the new 10G FB9000 router right now, but we are starting. We already have an alpha which does a consistent simple IPv6 address allocation over DHCPv6 on Ethernet (a hash based address to avoid giving out the MAC). We hope next to have DHCPv6 client over Ethernet and then DHCPv6 client prefix delegation. The server side will be a lot more work considering the config, and how much we link with IPv4 config, so bear with us.

2022-04-29

The latest crazy law

The latest crazy law imposed, today, with no notice, is The Russia (Sanctions) (EU Exit) (Amendment) (No. 9) Regulations 2022. My good friend, and lawyer, Neil, has blogged on it already...

This is one of those rare cases I am blogging as director of AAISP rather than purely personally.  See here for A&A news post on this.

The main issue is we, as an ISP, have to "take reasonable steps to prevent a user of the service in the United Kingdom from accessing, by means of that service, an internet service provided by a designated person."

I do wonder why - I mean this is asking anyone providing internet access, whether for their family at home, or via free wifi, or anything, to do this? Why not ask the handful of transit providers to do this instead - much simpler, surely? But OK...

My first issue as an ISP is what are those services? I mean these are not services offered by some corporate entity that happens to have a "designated person" as shareholder, officer, or even employee, but services actually provided by a "designated person", over the internet. This list of such persons is not simple or small, and working out which provide what services over the internet will not be a simple task.

So we plan to ask, maybe, OFCOM, as they have specific enforcement requirements in that legislation, for a list of such services.

But when we get that, what then? LOL, like we will get a sane answer, ha... But, well, then we have to try and block access somehow, if reasonable.

We do not have means to block access or filter anything by IP, or DNS name, in our network!

I can't stress this enough, we have never had any order to block anything or any previous legal requirement to do so, really. It is, in my opinion, not "reasonable" to expect us (for no payment at all, or otherwise) to magically implement such a measure, especially to do so between Laid before Parliament at 5.00 p.m. on 27th April 2022 and coming into force 29th April 2022, really. Or even (as it will cost a lot) later.

Update: As some people say, we have BGP routers that could have a black hole route added, and customer facing DNS servers that could have a bogus entry added. But this is the tip of the iceberg in terms of a "system" for blocking. There needs to be the management systems to maintain the blocked IPs and domains. Systems for who can add and remove entries. Systems to ensure they are applied correctly to the various config files. Procedures for handling mistakes. Procedures for handling support queries from customers relating to blocks (and mistakes in blocks, over-blocking, etc). Systems for getting the sanctions lists, processing it, researching the services provided by those Russian companies, and making changes over time. Yes, some ISPs have (most of) these systems and procedures in place for other reasons. We don't! On top of which, actual URL blocking is a completely different matter and simply impossible when considering the current use of https.

Update: That said, for a couple of domains, it is not impossible to add a DNS entry manually, but it is far from a scaleable solution.

What could we do?

At a push we could refuse to answer DNS for some domains on our customer facing DNS servers, but customer do not have to use them, so that would not be effective in meeting the requirement. And weirdly the providers of public DNS, like 8.8.8.8 and 1.1.1.1 are not subject to this order - why?

Indeed, if we had some way to block some routing to some IPs (and remembering we must not "over block" to meet net neutrality laws), customers are allowed to, and often do, use VPNs, so again, it would not actually be effective.

I am not sure we could "reasonably" take any technical measures. The closest we could get is not answering some DNS.

So what do we do?

Well, step one is we ask OFCOM for the list of services, and see what we get. That is it for now. I expect no list, to be honest, which sort of solves the problem.

Then we consider what next.

The other consideration is that we might "ask customers nicely" not to access such services. That sounds like a reasonable step to me. We might do that once we have a list of such services.

Update: The sanctions list has been updated - two "designated persons" have been listed: TV-Novosti and Rossiya Segodnya, with the web site rossiyasegodnya.com specifically listed. What is odd is that OFCOM have seen the list and decided that the sites rt.com and sputniknews.com should be "blocked" somehow. So which is it? What is the process for finding the "services" offered by the designated persons and how did OFCOM come up with those two domains? Is every coffee shop offering WiFi to somehow research some Russian companies to find what services they offer?

In practice, it looks like our (free) customer facing DNS servers may have to fib about a couple of domains for now. Not a scalable system, but hopefully "reasonable steps".

And just to be clear, I want the war to stop. But I am not sure how these sanctions help or are in any way effective. They are, however, a break from any notion of "mere conduit" for Internet Access. If they are needed, they are in the wrong place (surely transit providers, or DNS providers like 1.1.1.1 and 8.8.8.8, are more appropriate than every coffee shop offering WiFi). So we are doing what may be the only "reasonable steps" we can do.

2022-04-26

🦣

Back on the Mammoth, having re-installed Mastodon.

You don't have to set up your own instance. You can simply join any of the bigger mastodon servers. If you want to make your own, it is still a pain to set up, but the instructions do work if you follow them carefully. Here.

The larger servers are struggling a little right now, after the Musk take-over of twitter has been announced. Seems a lot of people think Mastodon is viable alternative. It may be, but it is different.

It is much more of a "social" network in that you have to socialise, and talk to people, to get followers and find who to follow. But no adverts and no "algorithms".

But I am here, on Mastodon, again, toot.me.uk/@revk so do follow me if you like any of the stuff I do/say.

P.S. As this is a new server, some servers have the old stuff cached in some way, so I may have to follow you first to get it to sort itself out. Let me know if you have problems following me.

2022-04-20

Solar install, part 4 (export)

Read part 1 for more background.

I'm really pleased with the install, and we are making 40 to 50kWh a day at the moment. So now for some paperwork...

One bit is a micro generation certificate, this confirms it is all installed by approved installers, etc, and has the details. I need that for and export tariff.

I have been discussing with the installers - we have a 12.09kW system (i.e. 31 x 390W panels), but they are east / west split so never all get direct sunlight. With the split and roof angle they have determined that maximum is going to be around 9kW, so a 10kW inverter is more than adequate. It will be interesting to see in the height of summer - but interestingly, even though only April, the generation in June won't necessarily be as good as you expect just by sun angle, because it is hotter. Being hotter makes the panels less efficient, so may cancel out the extra sun angle.

I heard there is some trick with some panels with water cooling, i.e. using the waste heat as a heat pump and keeping the panels cool for maximum PV. But we don't have that :-)

So, I may be making 9kW some days, yay. But what do I do with that?

As you will have read, the primary target for this my own usage, and then the excess power goes to a battery, so I can then use the power later when the sun is not shining as much - maybe covering my whole 24 hour day's usage for most if the summer, which is mental.

The bad news!

The battery is likely to be August. So 4 months of no battery. So, I'll export to the power grid, simples!

Well, yes, but that needs paperwork to. The Distribution Network Operator have to give permission to connect, which they have. The DNO decide what they consider the limit for export, if everyone was exporting, before it would cause damage to transformers, etc.

It varies, it can be as low as 3kW. Thankfully, in our cause though, it is 6kW. So we cannot export more than 6kW, legitimately.

Well 9-6 is still 3kW, and whilst I have the hot tub heating during the day, normally, a daytime idle power (when hot tub not heating), is a bit more than 1kW.

The result is, on a really sunny day, we could be exporting 2kW too much. Ooops.

Well, not quite, the inverter is set up to limit to the agreed amount, 6kW. It adjusts the panel optimisers to ensure that overall they are not quite as efficient, if we go over 6kW. So on a sunny day I should see a flat top on my export stats at 6kW. It means I will be literally throwing away some sunshine, up until I get my battery.

Paperwork

However, there is another snag - the final handover paperwork is normally done when the install is complete. The permission covers the solar and the battery - but we have no battery for 4 months - so I have no paperwork. So I cannot get on an export tariff.

The installers are trying to work out with Western Power Distribution how the handle an interim paperwork handover for the solar only without losing the permission to install the battery later. Hopefully something we have sorted shortly.

Tariffs

So the next step is a tariff for the next 4 months. It looks like I can get 5p/kWh from so.energy, less from Octopus unless I move to them for import as well. Octopus also do an agile tariff, but not being able to control when I export (yet, no battery), just relying on sunshine, I am not convinced that makes sense (a variable tariff for every half hour noticed a day in advance).

So my plan for now is probably the 5p/kWh from so.energy for 4 months, if I can confirm how the 12 month minimum works penalty wise. Once I have a battery I will need to revisit tariffs as there are interesting options.