Setting the temperature

My air-con can have a temperature set and aim for it.

• It has a wide range of ± a few degrees which I don't like.
• It is not setting temperature by me, it is basing it on some sensor it has.

To fix this I have used an environmental sensor - on my desk, and for my bedroom, on my bed head. I then control the air-con - telling it I want it higher or lower than it thinks things are, so as to aim for the right temperature by me. It works well most of the time and keeps the temperature very close.

That is my bedroom, last night (cooling), units in ℃, so maybe ±0.2 ℃ at most. Good.

I also have radiators for the winter, and they have the same fun, driving the valve on the radiator with some prediction.

Which to use for heating? air-con (heat pump) or radiator... Bearing in mind some of the house does not have air-con, so has radiators anyway. I don't actually know which is most cost effective / efficient. But I can choose. Have systems in place to make sure they are never fighting each other :-)

Per room radiator control

One of the key things is per-room controls - this means a temperature sensor in the room, and a radiator control, and some way to tell the boiler that one of the rooms needs heat. We have two heating loops (up and down), my daughter has two (left and right!). But for each radiator there is one boiler control that is needed.

The hard way

So, the hard way, I have my fancy environmental monitors with a display. They have a config for temp targets throughout the day. I have systems to override for empty rooms. Using the light switch marks a room in use for the day :-)

They talk MQTT to a Shelly running Tasmota that controls the radiator. So needs an MQTT broker.

The Shelly talks MQTT to a FireBrick to say radiator is on/off.

The FireBrick has a set of profiles, and any one in each loop will cause a message to another Shelly running Tasmota to operate the heating loop valve.

This triggers the boiler as needed.

It works.

It is not cheap or simple - the env sensors cost, the FireBrick costs a lot.

The easy way - maybe

So I am working on an easy way - my daughter happens to have a FireBrick, but even so I am trying to make a really easy way to do this. Not all of my kids have one (?!).

The first step is the reference temperature. A sensor that can be placed sensibly, e.g. by a bed.

I had found some nice industrial (BlueCoinT) ones at €31, but now have found, thanks to a recommendation, some stupidly cheap ones on Amazon. They are only £9.

The next step is controlling the radiator, and something like a Shelly Plus Mini would be ideal. Not sure of price yet, but the normal Shelly Plus 1 is £16.49. Whoa, they used to be less than that! Interesting. The Mini may be less. We'll see. But up to under £26. The radiator control itself is something like this, £19.63

So yes, we are up to £45 per room. A few more £s for back box and fused spur box for the Shelly mini by the radiator. And one Shelly at the boiler.

The comms can all be BLE. Sensor to Shelly, and Shelly to boiler control Shelly. Just needs a bit of software.

Not ideal, but £50 per room for exact per room heating control is probably quickly worth it. Simple web interface (or maybe even HomeKit) for unoccupied rooms, and controls for times of day and target temperatures via a web page. This can quickly save on heating costs and make things more comfortable.

It always amazed me people ran their house with one thermostat on the wall in the hall.

What counts as R&D?

One of the things a company can do in the UK is claim some tax benefits for spending some of the company resources on R&D. It is an incentive for companies to do R&D.

As a company that does a lot of R&D, we work with a really good agency on this, and get some tax credit. This works well, and is indeed an incentive for us to do more R&D.

This year the rules have changed slightly, and I noted a couple of them. It is all a bit open to interpretation, but a couple of the points seem a bit odd.

Open source

For a start, if we, as a company, did some purely open source stuff that was not direct benefit to us - by way of giving back to the open source community, it seems that would not qualify for R&D tax credits. It has to benefit the company. Thankfully this is not really a problem as pretty much all of the open source stuff we do is used in house and so benefits us directly as well. Some of the stuff we don't use in house, is actually stuff we then make and sell, so a benefit to us. But it is a disincentive for doing something purely open source.

Super human engineers

The other one that is odd is that things are not R&D if they would be apparent to a competent engineer in the field with publicly available information.

Well, we employ competent engineers. everything they do is apparent to a competent engineer. It almost sounds like, to be valid R&D, we need super human engineers, which seems a bit much.

We had a lot of discussion on this - one thing is the "publicly available information". This has meant we have focused a lot on things where the publicly available information is wrong or incomplete. And notably where what was "apparent" to the competent engineer was in fact wrong when tried.

Though one other aspect is that we think we can validly say that anything we are doing based on closed source designs and code is something that qualifies because that closed source is not publicly available information. None of it would be apparent to our competent engineers without access to that closed source information. So again, this seems to be discouraging open source.

Even so, the end result was a good meeting today, and plenty of R&D work.

Amazon done wrong

We sell on Amazon, and for that I'll apologise at the start, but it does provide some convenience to some customers and mostly works.

A few weeks ago we got an email saying we have to upload dome documents or they will close the account, and several ongoing reminders. Oh shit...

The problem is that the staff, including me, have a login to the seller account and can do 99% of things on it, as needed, but we could not do this as it had to be the main seller account. OK, so log in to that. Easy.

Except it wants to SMS a number to one of two numbers. Both were set up (and so worked at the time), but are 01344 (i.e. landline) numbers, and between when set up and now Amazon have obviously fucked up their SMS stuff and cannot SMS either.

The interworking of SMS really is a shit show and needs OFCOM to actually do its job and ensure it works properly.

But they also for calling them with a code - yay - except that also does not work - no clue why. Both numbers valid for SMS and calls, and they do neither.

So I tried the "remove 2FA" and sent pictures of passport, or something. My accounts manager did the same several times. No joy.

Every single chat was just crazy, and got to being daily. I mean I am tempted to publish them - they are so mad. One of them insisted I wait 24 hours after trying before contacting them - I said I had done that - he asked why I did not contact them after waiting 24 hours - and I explained I did - this is me contacting them!! The chats were like that. I don't like to judge people as "stupid", honest, people are only sensible in the areas they know, and that applies to all of us. But really, these chats were "special".

Just to be clear - the only reason I could "chat" at all is I had a working amazon account from which to chat. If not I would be screwed.

I have my case handled and referred to these, all of which did not reply at all!

• Internal escalation team
• Account recovery team
• External escalation team

I finally got someone that insisted that another "remove 2FA" should work, and I tried it, and, well, it did. But the instructions were clear - once done I could log in with just a password.

This was a lie. I logged in, and it said "Enter verification code. For your security, we've sent the code to your phone ***-***-**23" which basically means 2FA, so not removed, but also a number I don't know. Not either of the two numbers before.

Many more chats. No way to progress this. Doing the "remove 2FA" again stalled as it needs the code sent to *23 to get as far as uploading a driving licence even. I kept on, emailed in again, and finally it changed.

Now I have option for OTP by email or by SMS to *23, yay, but then it again does "Enter verification code. For your security, we've sent the code to your phone ***-***-**23" as well. So email and SMS now! (yes, when I got SMS working it wanted email too, so yes, both now needed!).

Had there not been an actual mobile *23 I really think we would be screwed. They have zero way to sort this from what I can see. Removing 2FA is "still do 2FA using a *main* mobile number" which you cannot change until you log in!

Turns out, after some research, checking staff lists, and pestering someone on leave and at a party (really, thank you Mike), I found someone that had a number ending in 23, and they were indeed what was on Amazon. So they let me know the code. Yes, that is being fixed, they should not have been on the main login, really. History from when first set up.

So finally I am in - it took a couple of attempts, but now the 2FA is sorted.

Phew.

So back to step 1. What is it they need.

Well this...

Yep, it is a sort of Farage test I think!

Obviously I click Agree and submit.

And, well, predictable...

So, err...

So another chat, and really, I am at a loss on this chat...

Ok, long winded, but basically no clue why this was an issue "not understandable" somehow, and actually, no, there is no way to upload a screenshot.

Eventually got an email and replied with a screenshot. It is a "don't reply to this" email address, FFS.

LIGHT HAS A SPEED - SO WHY NOT DARK?

I was looking at today's xkcd, and am reminded of a Terry Pratchett quote: [The science of Discworld, Terry Pratchett, Ian Stewart & Jack Cohen, ISBN 9780091886578, chapter 22 "Things That Aren't"]

LIGHT HAS A SPEED - SO WHY NOT DARK?

It's a reasonable question. Let's see where it leads.

In the 1960s a biological supply company advertised a device for scientists who used micro-scopes. In order to see things under a microscope, it's often a good idea to make a very thin slice of whatever it is you're going to look at. Then you put the slice on a glass slide, stick it under the microscope lens, and peer in at the other end to see what it looks like. How do you make the slice? Not like slicing bread.

The thing you want to cut - let's assume it's a piece of liver for the sake of argument - is too floppy to be sliced on its own. Come to think of it, so is a lot of bread.

You have to hold the liver firmly while you're cutting it, so you embed it in a block of wax. Then you use a gadget called a micro-tome, something like a miniature bacon-slicer, to cut off a series of very thin slices. You drop them on the surface of warm water, stick some on to a microscope slide, dissolve away the wax, and prepare the slide for viewing. Simple ...

But the device that the company was selling wasn't a microtome: it was something to keep the wax block cool while the microtome was slicing it, so that the heat generated by the friction would not make the wax difficult to slice and damage delicate details of the specimen.

Their solution to this problem was a large concave (dish-shaped) mirror. You were supposed to build a little pile of ice cubes and  'focus the cold' on to your specimen.

Perhaps you don't see anything remarkable here. In that case you probably speak of the 'spread of ignorance', and draw the curtains in the evening to 'keep the cold out' - and the darkness.

It occurs to me that it would surely work as you would expect. Whilst you cannot focus cold on to something, you can cover part of the area that would radiate heat from the environment to something with a mirror, and that mirror pointing at ice cubes means that not as much heat is radiating from the ice cubes via the mirror to the target as would be otherwise. But heat from the target is radiating to the ice cubes that absorb that heat. The net flow of heat would be from the sample to the ice cubes for a large chunk of the area surrounding the sample.

But this would mathematically be the same as considering cold radiating from the ice cubes to the target via the mirror. I suspect the maths works out exactly as you would expect for focusing cold just the same as it does for electron holes.

Yes?

ESP32-S3-MINI-1 GPIO

I now have a few boards, including a general test board...

The ESP32-S3-MINI-1 has 39 GPIO pins, though the data sheet defines that one (GPIO26) is reserved when using PSRAM (e.g. ESP32-S3-MINI-1-N4R2).

Unlike the S1 boards, the GPIO are all available as input or output, making its very flexible. But as always there are some caveats on GPIO usage.

The following relates specifically to the ESP32-S3-MINI-1 module only.

Restricted use:

GPIO0

GPIO0 is the main strapping pin that defines BOOT mode. If low at startup the device enters boot mode. It is pulled up internally so can be left unconnected. This means you cannot use this as an input if there is any chance of the input being low at start up. It can be freely used as an output.

By far the most common usage for GPIO0 is connection to the cathode of a status LED.

GPIO19/20

By default these are USB. This may make them not ideal to use for inputs. Note that they can be configured not to be USB by a fuse setting, but will be USB on an unconfigured module. As such it is probably best to use for USB.

GPIO26

The cannot be used with a PSRAM module, e.g. ESP32-S3-MINI-1-N4R2. It can be used if not PSRAM, e.g. ESP32-S3-MINI-1-N8.

GPIO43/44

By default the boot serial console uses GPIO43 and GPIO44. This means GPIO43 is configured as an output at start up. There are settings to stop any actual output on this. This means it cannot be used as a driven input without at least a resistor to handle the clash at startup. GPIO44 is an input so can be used for any purpose without any problems generally.

GPIO46

GPIO46 only matters when in BOOT mode (GPIO0 low) and in such case must be low. It is pulled low by default so an be unconnected. But do not use this as an input that could be high at startup in BOOT mode. It can safely be used as an output.

Caution:

GPIO3

This is defined as a strapping pin, and is floating by default. However it is only relevant depending with some specific eFuse settings related to JTAG operation. When using the default USB JTAG working, this pin can be used as a general input or output.

GPIO45

GPIO46 impacts the voltage used for SPI, but depends on EFUSE_VDD_SPI_FORCE. But this efuse is already set by default on this module, which means GPIO45 is ignored. It can therefore be used as an input or output.

Serial or USB console?

The S3 modules have two ways you can operate the console - the normal serial (UART), and USB.

USB

The default USB working is as USB JTAG and Serial console. This allows flashing and debug console, and a lot more debugging using JTAG and OpenOCD. This is by far the simplest way to manage the module and what I would recommend.

In most cases my boards have USB, either a printed tab for USB-A or a USB-C connector, but in the rare cases they do not, it is simple to include a header for USB to work with a lead.

This nicely fits under the module itself, with no vias, so takes no space and so no component cost. I deliberately spaced as 0.1" header and with a polarity to avoid confusion.

Apart from using JTAG, the USB serial is not really serial, so is way quicker for flashing code than using the real serial port.

There is one small snag...

It is possible to use an efuse, and more importantly, it is possible for the application to turn off the USB pins. This can even be inadvertent (I have done this, loading code for a different board which had an LED on GPIO19). This then means you cannot use the USB port at all, so cannot reset or load code or anything.

For this one reason, it is always useful to have a fallback.

Serial

The other approach for flashing and console is the conventional serial port. For this you need to have at least Tx and Rx (GPIO43/44) pins, and boot mode (GPIO0), and a means to reset (EN). You could use power on as the reset and keep it to just 5 pins. I use a header that works the same as the Shelly 1 and so can use my Tasmotiser board.

This also fits under the processor rather nicely with no need for vias, and no component cost. In fact, it can fit under the processor along with a USB header...

ESP32-S3

I have worked through a few of these processors - starting with the ESP8266. I moved to the ESP32 quickly and have not looked back - initially with the WROOM module (which can be hand soldered), to the PICO module (which is smaller and needs solder stencil and paste).

But finally the ESP32-S3 (version 3 of the silicon) is available as a module, the ESP32-S3-MINI-1, so I am finally starting to use that. The fact that the ESP32-PICO-MINI seem to suddenly be out of stock, seems like the time to upgrade designs.

The S3 has several improvements which may be useful.

Size matters

A key difference is size. The ESP32-PICO-MINI (i.e. S1) is 13.2mm square plus antenna, but the ESP32-S3-MINI is 15.4mm square plus antenna. So 2.2mm wider and longer.

For almost all of my designs this slight change does not matter, and indeed, for some the other changes actually make things smaller. There are exceptions - the module to go in to the case of a Galaxy keypad may be too big as there is very little space.

However, there is one useful side effect of this extra size. I have a small 5 pad footprint for serial programming - it uses the same 5 pins as the Shelly, and is pretty much the minimum you can have for serial programming and debug (see on image above on the right of the components). Now the module is slightly bigger than can actually fit behind the module, so effectively taking no extra space on a single signed board. This is one of those small details that saves space - e.g. the design show above is now a couple of mm shorter as a result.

General purpose I/O

Another change is the GPIO. The S1 modules have fewer GPIO (27), but several of them are input only - all of one side of the module has no output GPIO. There are also a number of NC pins.

The S3 has more GPIO (39), and all of them are input and output. There are no NC pins (apart from GPIO26 when using the PSRAM based module). This means all three sides of the module have lots of usable GPIO. This is perfect for tracking as you can easily track to the nearest convenient GPIO, which saves space.

USB

This is where it gets both interesting and slightly disappointing.

The S3 has USB built in, so direct connection of D- and D+ pins to specific GPIO. It can be disabled, and it can be used in s/w as USB server or client side as needed, which is very cool. This saves space on many designs if no UART needed.

The default, starting point, is USB JTAG with USB serial. This appears on my Mac, for a totally un-programmed module, as a USB serial port. I can connect to it and see the boot loader serial output. This is excellent as it means I can use a much simpler interface to the module - no need for a UART. As many of my modules have USB for programming and debug, this makes it a lot simpler.

However, there is a snag, it seems. Whilst this works as you would expect as a serial console port over USB, it seems it does not provide the normal connection for RST to reset, and DTR to boot mode (GPIO0). So you can only really use it to load code if you have a button for GPIO0 and RST on your board. This is a lot of board space, and really is silly.

 UPDATE: I don't know what was wrong with my first test boards, but this is working, so, all OK! I think perhaps I simply had out of date tools on my Mac (e.g. esptool).

Now, there are a few rays of sunshine.

1. It seems when un-programmed it does work in boot mode and so can be flashed, even though no RST or DTR working. That said, it seemed oddly intermittent, so I plan to do more testing on that. It may mean I can use USB for the initial loading of code and fuse settings.
2. Obviously as a USB JTAG interface it may be that there are ways to force a reset and boot mode or load code over JTAG, but this is not obvious from the documentation. If I can work that out I can make boards which do not also need my 5 pad serial header, which is what I was aiming for.
3. It is possible to internally detach this USB JTAG/serial and run USB from code in the module - this can handle code loading as a feature you compile in it seems (though even that suggests GPIO0 needed - why?!). I have not sorted that yet. My main concern is that if the code in the module does not have this, or is crashing, or whatever, I will have no fall back unless I have my serial header.

So more to do, and any help welcome. If there is a fallback for the USB JTAG working that allows code loading without having to do GPIO0 and reset, that would be really nice. For now, I'll keep my 5 pad serial, under the chip itself, as a fallback.

In theory OpenOCD is loaded, and should work for the JTAG. No idea how to use that yet, but that is a different matter as it does not work - some obscure error which sounds like a libusb issue on modern Macs. Arg!

Security

One small concern - JTAG is usually a very powerful interface on a system. If the chip is left with USB JTAG set up, perhaps because that is the only way to load code sensibly, it may also mean it can be controlled such as to allow secure boot and flash to be accessed. I.e. single stepping code after loading, etc. It may be that for security it needs the JTAG USB disabling (a one time e-fuse setting). I need to investigate. As a whole my security threat considerations have generally be based on physical access being a reasonable challenge, as the serial interface represents a similar risk in some cases, but it depends on the application.

Royal Mail API (RMAPI v4)

We have used Royal Mail for postage for some time. They had an XML API which we have used, and it works.

What is interesting is how all of these APIs are moving from XML to JSON now. One of the big benefits of JSON is none of the FUCKING XML NAMESPACES (sorry, did I say that out loud?).

Even so JSON has its issues, but nothing like XML with namespaces.

Manuals

They have a web site with a login and loads of interactive stuff, and that includes the API and API manuals. They also have a "V4 Shipping API Consolidated Guide" PDF they sent. So how hard could it be. I had two manuals!

To put this in context, I am someone that has used (and written) many APIs that are JSON/http based over the years. I practically invented the concept (I even have a patent on form posting from over 30 years ago). I have some clue, honest.

I actually quite like the latest "web" based stuff with JSON sent and received. Even OAUTH crap. I have some experience with this, honest.

Update: Before logging in there are some extra links, and another set of manuals, which is yet again different and contradictory - arrrg!

Authentication

First challenge is authentication, and to some extent the manuals were not that bad. The one critical thing they could have done is include, in any of the manuals, one simple example.

They use OAUTH2 but in a way I have not seen anyone else use. I needed a POST to a URL with a URL encoding post of grant_type=client_credentials (and it had to be a POST not a GET) using Basic authentication of the client ID and key they allocated. This is an unusual way to OAUTH2, but valid, to my huge delay of some hours getting the hang of it was really down to me. I get back a Bearer key with expiry (1 hour).

API URL

One issue is the URL, well, the hostname. The documents state that the hostname to use is for their TEST system. This is good, they have a test system, but the other manual suggests it is the hostname for the LIVE system. Why?

I eventually worked it out, there is one hostname for authentication, and whether it is test or live depends on your account. Your account can be in "sandbox". I mean that is great, a good system, but why the fuck not state that in either of the manuals?

What adds to the fun is that hostname is actually only to authenticate. The actual hostname for the service is different, and searching both manuals I do not see the actual hostname for actual API calls listed anywhere.

Why? WTAF?

JSON blob

The big clue was some swagger.json file you can download. Loads of JSON, some "OpenAPI" thing. That has been way more helpful than either of the manuals!

It explains the actual hostname to use for the API, and each of the schemas for each API.

The manual, well one of them, lists fields you need, but omits that these may or may not be in sub objects. The JSON blob explains in way more detail.

And, of course, this JSON file, whilst really good, is missing a load of things, like creating manifests!

Errors

For example, the error "Could not find member 'ContactName' on object of type 'Destination'. Path 'Destination.ContactName'" fooled me for a while as I was supplying Destination.ContactName. What it meant is I should not be supplying that. I read "Count not find" as I was missing something, but no, it meant I had supplied something it "could not find" a use for. That is shit UI. (FYI I should have provided Destination.Address.ContactName)

Tracking

There is, of course, tracking. There is an API to get tracking and an option to have tracking posted to a web hook - yay!

But the documentation for the tracking API just says the response is a tracking number? Which, err, makes no sense, what of the status of the shipment. Hmmm.

And apparently the tracking web hook is not supported, so why the hell is it in the API documentation? Arrrg!

Working

I have it working now - but this is literally one of the worst documented APIs ever.

The raw JSON file was more documentation than either if the manuals, but even that is missing bits.

Icing on the cake

Royal Mail have What3Words embedded. I kid you not.

This is so mental! RM have PAF which has detailed (like, to 10cm) delivery point details on all UK addresses. They are the one organisation that has ZERO use for W3W.

I don't know what they do with it - I will ask?

• If a W3W supplied is way off, like hundreds of miles, do you use it?
• If a W3W is next door to postal address provided, do you use it?
• If a W3W is within target address, e.g. a shed in garden, do you use it?

If all are "no", I have to ask WHY THE FUCK it exists in their API!

WIFi: Aruba vs FS

I have used Aruba WiFi for some time - they are nice, but expensive. The latest Aruba Instant series seem more sensibly priced though.

When picking WiFi there are a number of factors, but if you are going to get more than just the WiFi that comes with your broadband router you almost certainly want something that can allow multiple access points and roaming between them.

There is also a matter of the number of antenna and radios, and the protocol - summarised as catchy names like WiFi5 and WiFi6, these days. There are also access points for outdoor use, which can be useful.

The Aruba has one cool feature, and the main reason I like it, that it does not need a controller. A controller is what directs devices to move from one access point to another as you move around, and what manages the overall configuration. With the Aruba, the controller is one of the access points, and can even fall back to anther one if the main one is off.

So what changed

I have been pretty happy with the Aruba for many years. It has a lot of features and settings (perhaps too many), and as I say, needs no separate controller.

But recently some of my IoT kit (ESP32), which is all 2.4GHz, has been playing up, sometimes sulking not reconnecting to WiFi, and so I felt it was worth changing the WiFI to see if it helped.

Long story short, the WiFi choice was not the cause of that specific problem, and I think I have fixed it. But the change of WiFi had some surprising side effects. For reasons I won't go in to, I have all my 2.4GHz on the same channel. And so that is a pretty full channel. As such I was never surprised to find s/w upgrades on a set of IoT devices was always a bit slow - I set up to do one at a time. To my surprise, with the new WiFi, it is way quicker!

What is the new WiFi?

I have got some kit from FS. It is cheaper than the Aruba, which is a start, but it needs a controller. The controller (AC-1004) is also a 4 port PoE switch which makes it actually useful for a small home installation as you almost certainly want a PoE switch for your access points anyway. So again, not that much extra cost. It can handle 64 APs. I may be a convert to the idea of a controller.

The (web based) controls are a but more clunky than the Aruba, but to be honest, pretty similar. It also has a huge command line interface!

The APs actually come with mounting brackets, unlike the Aruba! I have AP-N505 indoors and AP-T565 outdoors.

So far, so good. I'll see if I am cursing the change in a few weeks. If not, I'll take down the Aruba APs.

BMA423

The BMA423 is a small accelerometer by Bosch. The data sheet is here. It is 129 pages. It is really crap if you want to just use this chip. It repeats itself quite a bit as well. The Quick Start Guide has a nice vague bit about writing the init file as a burst write to address 0x5E.

There is reference code, and the help forums for this just have Bosch saying to use the reference code. Some of us would rather not. A good data sheet should not expect you to just use the reference code. It is also a bugger to follow.

One of the main issues is the the reference code works with registers 0x5B and 0x5D which the data sheet lists as reserved. This is silly, the data sheet could define these. The reference code also has 6kiB of binary code that seems to need to be sent.

So, here is what I worked out, and it is now working.

Initialisation

Reading and writing registers is standard I2C register read write. I.e. write the register address and then data, or write the register address and then read data, allowing multiple byte writes or reads as normal (with on exception, see below).

Checking

• You can check all is well, and init done, by reading 0x2A (INTERNAL_STATUS), and confirming it is 0x01

Soft reboot

• You can do a full soft boot by writing 0xB6 to 0x7E (CMD).
• Wait 1s after soft boot

Loading ASIC code

• Write 0x00 to 0x7C (PWR_CONF). This disables aps (advanced power save).
• Wait 500ms
• Write 0x00 to 0x59 (INIT_CTRL)
• Send the 6k block to the ASIC address 0 (see below) - see here for the data.
• Write 0x01 to 0x59 (INIT_CTRL)
• Wait 150ms
• Check status by reading 0x2A (INTERNAL_STATUS), it should now be 0x01
• Read the ASIC address at 0x5B/0x5C, this is where you load FEATURES_IN (see below)
• Write 0x01 to 0x7C (PWR_CONF). This re-enables aps

Block read/write to ASIC

It seems there is a separate ASIC block of memory, and this is accessed by three registers.

• 0x5B is low byte of address to next read/write - BUT NOT AS SIMPLS AS THAT!
• 0x5C is high byte of address to next read/write
• 0x5E is the gateway to reading and writing

Normally, with I2C it would auto increment the register on read/write when transferring multiple bytes. This does not happen with 0x5E. You can write to 0x5B/0x5C as a 2 byte transfer, as per normal I2C.

The ASIC address is stored oddly. It has to be even (bit 0=0) and you have to read/write even number of bytes at a time via 0x5E. The address is stored such that 0x5B is (address>>1)&0xF, i.e. bits 1-4 of address. 0x5C is address>>5 (i.e. bits 5-12). So you need to allow for this when reading or writing the ASIC address.

It seems from reference code you can transfer up to a maximum of 8 bytes at a time, so you have to repeatedly write the ASIC address to 0x5B/0x5C then write 8 bytes to 0x5E in one block. However, my testing suggests you can transfer the whole 6kiB init in one go, so totally pointless reference code.

Once the init file is written, and the status is 0x01, you have to read the address in 0x5B/0x5C which tells you where the FEATURES_IN block is located in ASIC memory... Shift it appropriately to get the byte ASIC address.

FEATURES_IN

FEATURES_IN is block of 64 bytes in ASIC memory. You can read/write even number of bytes, setting 0x5B/0x5C to the base address (as read above) plus the offset in to the block. The data sheet does list these additional registers reasonably well. It just lacks any explanation of how you access these bits!

You can read all 64 bytes in one go, and write all 64 in one go, or read/write even numbers of bytes if you only need to change one register.

Steps

To enable step counting, I have done the following :-

• Write 0x00 to 0x41 (ACC_RANGE) to set ±2G
• Write 0x02 to 0x56 (INT1_MAP) - only needed if you want interrupts
• Write 0x04 to 0x7D (PWR_CTRL) to enable accelerometer
• Set FEATURES_IN(0x37) OR'd 0x30 to enable activity and counter
• Set FEATURES_IN(0x3B) OR'd 0x10 - not idea what this does as reserved in data sheet! 

Reading steps is then 4 bytes from 0x1E onwards, low to high bytes for 32 bits.

Resetting the step counter is setting FEATURES_IN(0x47) OR'd 0x04 (self clearing).

NON-STERLING TRANS FEE

One thing pisses me off is cards charging a Non sterling transfer fee. I'd have expected them to make money on the exchange rate, or be big enough that they can handle currencies without such silly fees to be honest. Even more so when it is someone like American Express, that, well are American, and I am paying something in American dollars. You would think they could handle dollars.

But I have recently discovered something slightly odd.

VISA

Using my VISA debit I was charged...

AMOUNT IN USD 206.87 ON 28 JUN VISA 1.269999 FINAL GBP AMOUNT INCLUDES NON-STERLING TRANS FEE £4.48 = £167.37

But when I was then refunded, it looks like...

AMOUNT IN USD 206.87 ON 30 JUN VISA 1.2669 FINAL GBP AMOUNT INCLUDES NON-STERLING TRANS FEE £4.49 = £158.80

This means I was charged for the refund, and oddly charged at a higher rate (by 1p).

Amex

Using my Amex I was charged... (different day, I have not compared exchange rates)

71.15 UNITED STATES DOLLAR, Non-sterling transaction fee: £1.68, Exchange rate: 1.2687 = £57.76

But when I was then refunded... (next day)

71.15 UNITED STATES DOLLAR = £57.82

Comparison

VISA charge the non sterling fee on charge and refund, but Amex do not. Indeed it seems they effectively refund the non sterling admin fee.

VISAs fee appears to be around 2.7%, Amex fee appears to be around 2.9%, very similar.

Conclusion

It is actually in my companies best interests if I pay for these on my personal Amex and claim on expenses, if there is any chance of a refund (which happens quite a lot if I have arguments with JLCPCB, for example).

toot

Twice I setup Mastodon.

Once many years ago - I followed the step but step guide and got it working. It worked for quite a while and then broke - out of disk. I gave up.

Then a second time we easier - this time my ops team set me up a VM, and Mastodon had improved a bit, so set up again, following the instructions. Again it worked. It caused problems as it was a "new" instance on an existing domain, and that caused things that had cached stuff to get confused. It got sorted, people had follow requests they had to cancel and try again, that sort of thing.

These days it seems to have a lot more "cleanup" logic, and a lot more obvious on disk usage, but still far from perfect from what I can see.

WTF is it not simple?

My big issue really is that Mastodon should be like apache, or many other quite complex systems, which are basically apt install mastodon. That really is what it should be. Simple as that. The web interface or command line could take you through the initial steps for admin user password and domain name and so on, but that should be it.

There are docker instances. I know nothing about docker, but I am told they sort of just work. Maybe that is the answer?

But why is it not as simple any anything else on linux, really. It seems Ruby is the main problem, as is python it seems. A pain in the arse.

But all is well, so not problem.

Yes, all is well, and even upgrading to next mastodon version as needed was not that hard. Again, following a load of step by step instructions not a simple apt upgrade mastodon or some such. But not rocket science.

But err, then, it broke. Basically the ops team wanted to upgrade Debian to bookworm, which is fair enough, but doing so cleaned out almost all of the stuff Mastodon needs.

Now I am not just upgrading Mastodon, and not installing Mastodon (as I don't want to lose/break what I have), I am sort of trying to make it work from instructions that are one or the other, and constantly hitting stupidities like OpenSSL is not available (how the fuck is that not standard in these tools?!) and not clue how to fix that.

I gave up

I must be getting old, because, yes, I gave up, it is too much of a pain in the arse, and I have better things to do. I tried, honest, but life is too short.

Bring in the experts!

The simple answer is to call in some experts. In this case Mythic Beasts. They actually run the A&A instance because my ops team were smart enough to not go near this with a barge pole.

They are not only running my instance, but in very short order managed to copy all the stuff from the old, and now broken, instance over, and make it continue seamlessly for me, well mostly - some avatar issues they are fixing. But followers, and so on, all working. I have not missed anything and was off for a day at most.

It is not cheap, it is several hundred quid a year. But what do I value my time at?

There is an irony here. I am paying that for my social media. If the likes of Twitter had people paying that for a really good, clean, not sales/advertising/pushy/confrontational/stressy service, they would make a fortune. That is not their business model.

The only mitigation is it may end up being more than just me - some family members, when they finally catch this boat. Of course, subject to whatever stupid rules the Online Safety Bill may create, LOL!

Pipelining

I am reminded of years gone by when "compiling" would take a long time, even hours in some cases.

This lead to a "pipelined" debugging and development cycle. By the time what you did was compiled you already had fixed a load of bugs and added a load of new features that needed testing. So you had to test things from the compilation as they were when you started. Sometimes you just knew the compilation would create something broken, but some other aspects would be things you need to test, so you let it run anyway to test them. You had to make code that would include things you need to test in the best way possible.

I am sort of doing that with PCBs over the last week or so - getting a new PCB takes slightly over one week from China.

I have been working on a load of small changes to various designs.

• New 5 pin pad for programming - I made a larger set of holes as the ones on my last batch were annoyingly small and have to be held in place while programming. This is one I finally have back prototypes and my new design is too big and loose, FFS.
• New New 5 pin pad for programming, staggered more than last time to allow pins and hold them whilst programming - waiting - works perfectly
• New voltage regulator - waiting - and I really hope it is right as I have several in the pipeline with no fallback - one version works nicely (the cheaper one), the other (tantalum cap) does not work.
• New USB UART - got a board, discovered I cocked up - but was able to rework some and test the correct design - some boards were able to be used anyway
• New New USB UART - fixed layout/design issue - waiting - working
• New RGB LED - waiting, but already spotted that I have cocked up and got G and B swapped, FFS. This means the resistors will be wrong for the colours as well. May be able to rework when this arrives.
• New New RGB LED - waiting on revised, but still not been able to check if resistors right - waiting - working
• New panel design - i.e. making part in a 70x70 panel instead of JLC doing it, and if a barcode is clean in silkscreen - waiting - looks nice but JLC being a pain
• New Tx on Faikin, open drain output - waiting - working
• New Rx pull to 3V3 on Faikin, detect loopback - waiting

The whole thing is frustrating. I have had to scrap one set of boards sadly because of the CH340X issue. Others I was able to rework, or remove and use a 5 pin programming header instead. One trick is making a new board that you can "fall back" if the new bit does not work.

The RGB LED is a pain - I used one (side mounted) that worked nicely because they did not have the simple one I got from Mouser and used for my hand soldered boards. But JLC places it badly (over 10% of boards needed rework). And I found they had an equivalent simple front facing RGB LED with low enough voltage on the blue for 3.3V working. The issue is the resistors - you can work out values - as I had originally but then find green is way too bright. For my manual working I had changed resistors and so am trying those values to start with. But until I get them, and get the G and B right, I will not be sure I have the right resistors even if I have the right colours. I may have to rework some 0402 resistors to test when they arrive.

On top of it, DHL, who have managed around 2 days from China with no VAT issues, have suddenly made it 3 or 4 days and not doing proper VAT with postponed VAT accounting - that will be a fun argument as I am planning to refuse to pay and insist they re-do as PVA. We will see.

Also JLC being odd on rules, insisting multiple designs on a board even if only in solder resist. Still not resolved. Even if I end up making panels with no extra silkscreen, at least making as panels myself, instead of leaving JLC to do it, lets me make cleaner slots and edges on the designs.

I have to test if the barcode in silkscreen works, and is acceptable (inside a nearly clear, slightly pink, anti static bag) with Amazon - so avoiding having stickers.

So a very frustrating week, and another such week to come.

CH340X DTR/TNOW with ESP8266/ESP32 BOOT and RESET

The CH340X is a small chip that does USB UART, and is part of a family of CH340 chips in various sizes. A UART can need few or many pins, and they do a really small one that is just Rx, Tx and RTS pins, which is the minimum you would want. But they also do a version specially aimed at the needs of small processors like the ESP to handle boot mode.

BOOT and RESET

Both ESP8266 and ESP32 (and many other processors) have a BOOT and RESET input, well, actually GPIO0 and EN.

You need these to program and debug. To program you have to set BOOT MODE, (which is GPIO0 pulled low) and pulse RESET (EN low and then high). For debug, you want no BOOT or RESET, or maybe RESET to restart the processor, but not BOOT mode (GPIO0 staying high).

What is weird is the various ESP based modules out there. I have seen complicated FET based links of RTS and DTR from a UART linked to GPIO0 and EN, and some with R/C combinations linked to RST and GPIO0. I think the logic here is for more traditional use of DTR and RTS, i.e. usually one has RTS active the whole time, so an R/C allows RTS to be activated (low) but EN to pulse low and back high. Similarly linking the use of DTR and RTS to allow a normal UART operation not to be BOOT MODE.

However, the esptool and idf.py as part of the ESP IDF, and even the tasmotizer tool and web page all treat DTR as BOOT MODE and RTS as RESET directly.

You can connect DTR directly to GPIO0 and RTS directly to EN, and it works perfectly!

What is even more weird is the number of modules you see with actual push buttons for BOOT and RESET on the board, which costs money to fit, even though they have a USB UART.

I can only assume some legacy designs and copying reference designs and so on.

All my designs connect DTR to GPIO0 and RST to EN, and work perfectly. Yay.

CH340X DTR

The CH340X has Tx and Rx and also CTS (input) and DTR/TNOW (output).

The TNOW is multi function though, the datasheet says :-

5.3. DTR and multi-mode MCU download

For CH340X, 6# Pin defaults to TNOW, weak pull-up during power-on or reset, output TNOW during normal operation used for half-duplex transceiver switchover. By connecting an external resistor with 6# pin, TNOW can be switch to DTR#, the two options are as follows:

1 If the 4.7KΩ pull-down resistor is connected to GND with the 6# pin, it will enter the open source DTR enhanced mode, and the 6# pin will automatically switch to the open source driven DTR# pin , which is used to connect the boot mode pin of MCU. The default DTR # is not output and is kept low by the external resistor, but the DTR# pin can be set to output high level or not output by the application program, which is used for multi-mode MCU download with DTR# default low level.

2 If a 4.7KΩ resistor is connected between the 6# pin and the 5# pin, it will enter the push-pull DTR enhanced mode, and the 6# pin will automatically switch to the push-pull driven DTR# for connecting to

the control pin of the MCU. The application program sets the DTR# pin to output high or low level for multi-mode MCU download with DTR# default high level.

This is excellent, and makes it clear they have considered the use of DTR as a BOOT MODE for processors. I have not tested the DTR to GND via 4k7 - I think they are saying it inverts, i.e. DTR drives high. I may be wrong.

But the second mode, DTR to CTS via 4k7, I have tested. Well, actually I misread / assumed 4k7 to 3.3V, and that does not work, but when 4k7 to CTS it works, and drives the pin as DTR allowing direct connect to GPIO0 yay

Why a 4k7?

But I am puzzled. I understand the 4k7 to GND, as it pulls down, and allows active drive high as needed. But why 4k7 to CTS.

CTS is an input, so the only use of the link is to allow the CH340 to confirm the connection from DTR/TNOW (output) to CTS (input). Indeed, it pulses that output low a few milliseconds after startup, presumably to test it is connected to CTS.

The thing that puzzles me is why the 4k7. With CTS as an input it is quite safe and valid to connect DTR directly to CTS, output to input. And, as you would expect, doing that causes the CH340X to switch in to DTR output mode.

I suppose one could argue that you may want to make use of CTS as well, and the 4k7 allows that even if DTR is outputting something different. Except that does not work. If you are using CTS, that means you have an active signal feeding CTS from somewhere. If that is the case, the test of DTR to CTS does not work (I tried it!) as CTS will not see the signal from DTR/TNOW via the 4k7. So the 4k7 does not allow that. Indeed, the only use may be if you have something driving CTS which does not drive it at power up - that would be a rather special arrangement and somewhat unlikely. That said, I suppose if CTS was a general purpose GPIO from a CPU which starts as floating/input, it could make sense and allow CTS to be signalled from the CPU.

So I'd expect the data sheet to say just link DTR to CTS and not use CTS, or use 4k7 if CTS is floating at power up and needs to be used.

Anyway I have done that (just linked DTR to CTS), and it works nicely.

USB-A tracks to CH340X

There is a snag though - using DTR to drive GPIO0 works, but on my Shelly board it is drive GPIO0 that is connected internally in the board, with some pull something, and an LED, and so on. So the CH340 might "see" the pull down on it when it starts perhaps. So I have to drive through a buffer in that case. Thankfully that is basically the only case, as normally it just goes directly to GPIO0 on the connected ESP module.

JLCPCB and fun rules

I am using JLCPCB for my small boards. There are other places, but they seem reasonably sensibly priced, have a good automated ordering process, and are quite quick.

However, I have been trying to do something clever, and that is always a bad idea.

If I have a small circuit board, and want assembly, they will make it in to a small panel, e.g. 70x70m, with "rails" top and bottom, and V-cuts. This means when you get it, it snaps out of the 70x70mm PCB. The rails have some holes and fiducials.

As it happens, I am shipping (to Amazon) as a 70x70mm panel. This is partly because it fits better in a small anti-static bag, and partly to make clear it is sold as a component, an assembled PCB, and not a final product with UKCA/CE, etc. As it happens the main module is CE marked, with a RF can, and the only extra bits are normally just a small voltage regulator, so getting UKCA/CE marked should not be too hard one day, but for now, these are hobbyist projects.

Anyway, the 70x70mm panel has loads of wasted space. JLC add an order number (unless you pay them extra not to), but other than that, just unused blank PCB, a few holes, and fiducials.

So I decided to try and make use of it - make the 70x70mm panel myself. They allow this and have instructions for where to put fiducials. They also seem to want some holes and instructions seem to be lacking for those, but I copied what they did. However, this created a few challenges :-)

V-Cuts

The first problem is the V-cuts are cuts in to the board at the last stage, and always go right across the board (they explain this), but can go any direction (well, at least up/down or left/right). These allow the small PCBs to be snapped out. Some extra milling around bits of the PCB that are not simple straight edges allows other shapes, so works well.

For example, this is what they did when ordered simply as a tiny PCB. Four V-cuts to snap out.

KiCad

Firstly, how to tell KiCad. I cannot simply put these on the Edge.Cuts later as it would break the board outline - KiCad would fail DRC and not show the board on the 3D view and so on. So simple answer is a separate layer, easy enough.

But then KiCad does not know, for example, that tracks and copper fill should not get too close to these V Cuts. It knows for edge cuts, and so one can have milled slots for non rectangular edges, but the V cuts are not seen.

My solution was to make a footprint that is a 70mm V-cut.

• 70mm line on User.1 layer as the actual V-Cut
• Line on User.Comments and Fab layers to more visible
• A courtyard for 1mm above the V cut on one side
• An exclude for 0.2mm either side, front and back

The exclude keeps tracks and pads and so on away, just like Edge.Cuts.

The courtyard works on the basis that you have to bend the PCB to snap it, and that could be up or down. I have parts that hang over the edge (the antenna of the ESP32 module, etc), but by adding a courtyard one side I ensure I cannot accidentally have things hanging over the edge both sides. I have to flip the V-cut footprint to fit, but it allows a sane DRC.

Telling JLCPCB

This took a few attempts to get right.

For a start, I updated the JLC-Plugin to export an extra file for V.Cuts - I just added to the config file, and it exported. The problem though is that, even as a file called V_Cuts, it shows as a User.1 layer when viewed, and apparently this would mean adding a note to the order so they don't miss it. I don't trust myself to add a note.

However, after a few emails back and forth, I found that what they really wanted was V Cuts simply included in the Edge.Cuts file. It breaks some of the preview stuff as not a joined up border, but it is understood by their people it seems, and won't be missed.

I have done a MR on the plugin to include the User.1 in the Edge.Cuts file as part of the export, so I don't have to think about it.

Extra artwork on the panel

Adding extra artwork to the panel was pretty simple - just more silk screen. What took slightly more work is I wanted a barcode, and so made a tool to create a KiCad barcode module.

Then I ordered.

They I got a "we want $50 more" response. They said it was 3 designs. Actually, by their rules it was 5.

FFS

So, the rule is, if you have parts that detach, and they have any traces or silkscreen, they are treated as additional "designs" on the same order, and there is a surcharge for each extra design. So even a "design" that is just text on silk screen has a surcharge.

Simpler

A simple step was only silk screen on one of the unused parts, that made it 2 designs. So progress.

The problem is that the surcharge, whilst only $2 on the PCB manufacturer, they also surcharge the assembly, by around $30. Now I have queried as, even if there are 2 designs, I only want one assembled - the one with components on it.

It may be possible to order PCB as 2 designs and then order assembly as 1, but I cannot see any way to actually do that, sadly.

Solder resist

Another trick may be to avoid the multiple designs by following the rules. The rules cover tracks and silkscreen. Well, it seems we can do solder resist. Using this on an FR4 board with black solder resist may well have enough contrast. Even more so if copper fill and ENIG finish.

It needs more testing to see what resolution and contrast is achievable.

The latest attempt...

This is 2 designs because of the silk screen. But testing they agree with that by having the solder resist at the top as well.

Update: They magically decided this was 5 designs so I have challenged as "solder mask/resist" is not something they list as creating extra designs. We'll see...

Update: They seem to be doubling down on the "5 designs" still. I asked if the rules have changed?

Update: They are sticking to their rules, but making up bullshit for them, saying that they have to "load two programmes" if it is two designs. This is clearly bullshit, but I have challenged on basis that they do put their order number on silkscreen on one of the snap of panels, so extra silkscreen on that panel will be no more work, will it, as they are "loading two programmes" already, for their order number. We'll see.

To USB or not to USB, that is the question!

I have done quite a lot of small boards now, as I have blogged at various points.

It has helped me build up my skills in circuit and PCB design, and over time I have moved to different approaches for things as I have learned. Some things driven by practical usage, some by cost, some by simplicity.

At present I am working on ESP32 based boards, and I may yet move on to ESP32-S2 or ESP32-S3, at least one of which does some direct USB, but for now, for ESP32, I need serial connection for programming and debug. The ESP32 are especially appealing in the form of the ESP32-PICO-MINI-02 module which has extra flash and RAM which is invaluable.

I have "standardised" on a USB-C connector for my boards, where needed, for simple power. That is most boards, e.g. this LED strip controller.

It uses USB-C for the 5V supply for the LED strip, and via 3.3V regulator for the ESP32.

In many designs, I have included an FT231XQ USB UART in the design, allowing not just power, but programming and debug via the USB-C connector. This works really well. It makes it very simple to develop on. I have even learned my lesson with diodes on boards which can also have 12V DC, and USB connecting both at the same time :-)

But the FT231XQ is not a cheap part, and not a readily available part (a lot of parts are now available after COVID, but some still in very short supply). So I have been making even these boards have an alternative for when the FT231XQ cannot be fitted.

Indeed, I have actually started making boards which have no FT231XQ at all, even if they have a USB-C socket for power (like the one shown above).

This means I need some sort of header. One of the reasons I went with the UART in the first place is any sort of sensible header connector would take as much board space as the UART chip, and add cost or time, especially if a through hole connector.

But, the way Shelly do it has come to the rescue. They use a small 1/10" 5 pin header (and a 1/20th" 7 pin header) - a socket on the board, and I made this to work with them. These are useful if you are wanting to reprogram Shelly with say tasmota, or for any projects with similar devices. See Amazon.

It has header pins for the 5 and 7 way connectors. The shelly has a proper socket for these.

However, I can simply make the board have holes, instead of a proper "header", like the 5 holes on this boards.

I can simply push the pins in the holes, and hold them whilst programming. Indeed, if I can get the holes just right they will fit firmly enough not to need holding and so can be used for debug as well. It is cheap - holes don't really cost anything. It does not take a lot of space, especially as it can (like that example) be under another connector even.

I am working on slight variations of the holes, to try and make a firm fit to the square pins. The latest is slotted holes. In practice this means each set of boards I get has a slight improvement and I try it. The above design had holes slightly too small - some boards needed to be "held" whilst programming (which is not really a problem, but adds some time). This is my latest design for them.

So it is a question of whether I bother with a UART on future boards. Using USB-C for power is a good idea for a lot of boards - the "Faikin" being one of the few cases where that is not sensible as it fits and existing connector with power. If I move to ESP32-S3 for example, I'll see if that can do USB sensibly directly, but I want a module like the ESP32-PICO-MINI-02 really - not sure if there is one yet.

Oh, and as you may notice, I have adopted a policy of GPIO numbers on silkscreen - it really is damn useful when you have so many generations of so many PCBs lying around. I don't do a lot of silk screen - i.e. no R1, R2, etc, but I have to do some to stop JLCPCB asking questions about pin 1. But GPIOs is invaluable. And now KiCad allows proper fonts it is slightly more fun (the above has "faikin.revk.uk" in xkcd script, err, because I can).

Update: As suggested (and as per above image) I have now gone for staggered pins, to see how that goes.

Update: The silk screen does not work that small, so going for something simpler...

Update: I am also, as someone suggested, trying a CH340X which is physically a similar size for the FT231XQ but a lot cheaper and simper to wire up.

Superimposed CH340X (green) on FT231XQ

Design for USB-A ESP32-PICO-MINI-02 with RGB LED and CH340X

Update: Fuck sake, read the data sheet. 4k7 from CTR to DTR/TNOW is needed, not 4k7 from 3V3 to DTR/TNOW. Sadly meant some boards could not even be reworked. But was able to test this for future boards.

Companies bad at banking

I was discussing with a colleague the other day how so many companies are so bad with banking.

In some ways we have been lucky, but to be fair we have worked at it and made systems that work.

Your payment will not appear for 3 to 5 working days

I see this a lot when paying people - we live in a world of "fast payments", at least in the UK, and if I (personally) pay someone they will have the money within seconds. I mean technically the process can take longer, but in practice it does not.

I had some issue with a loan company for one of my kids, a settlement amount paid by fast payment and they still considered it several days later and extra interest. I was very much on their case from technical and legal points of view until they caved.

So what the hell are companies doing?

If someone pays A&A via normal UK fast payment, we have the money, and, in literally seconds, they get an emailed statement showing their account with the payment received. This is all thanks to Monzo web hooks. I am impressed with Monzo handling this well.

But even without that, there are ways to pull from Open Banking, allowing a payment to show on the banking day it arrives, but polled. Even if only polled once a day a company can see they are paid that day. Poll a couple of times and see money as it comes in during the day. It is not hard! It works for all banks.

Your refund will be in 5 to 10 working days

This is slightly harder, in some ways, as sending money generally needs more business controls.

We do BACS direct credits meaning we can send money on 2 working days if needed, and automate it all with direct submission to BACS.

In special circumstances we can do manual fast payments, but to be honest that is rarely needed from a business point of view. One day we may have that sorted too with the BACS system. But obviously such systems need careful business processes in place to avoid errors (either bugs or errors by staff).

Indeed, we automate refunds up to certain levels with two day BACS as a matter of course. But they have limits and above those I am involved (as director) confirming them.

But so many companies are so bad at this, sending cheques (wow) or "refunding DD payments". When Bulb sent me money it was dozens of payments that were each the amount of a previous DD.

Banking should be simple

I wish Open Banking had given companies and individuals control of their own bank accounts properly. They could do it all and all be like A&A. There are some companies offering this indirectly but why is Open Banking not something that is available to the account holder as standard - that would make sense.

Defensive coding

Business systems can be written over long periods of time and in a variety of scripts and languages, and we (A&A) are no exception. Some of the systems started life more than 26 years ago and even before A&A existed.

Obviously all of the code, and scripts and web applications and so on have been updated over the years in many ways. If nothing else the advances in security best practice alone are reasons to update systems.

One aspect to the way some of the code has been written is "defensive coding". This is basically assuming the worst can happen (either by accident or malice by an attacker) and trying to ensure the outcome of such will always be "safe" in some way.

The way we (A&A) do Direct Debits is one example of this. At every stage in a very complicated set of processes and scripts we try to assume the worse and take the safest action on failure. And one area of this is the notices sent to customers. There are a load of rules we should follow, and (seemingly unlike so many other companies) we try very hard to follow the rules very very carefully.

For example, if we have notified a customer that we will "collect £20 on, or immediately after, the 1st of each month" we record that fact and try to ensure that DDs only go in if that is the case. The rules allow up to 3 working days later, but the second we are not collecting £20 exactly, or it is not exactly on the 1st or within 3 working days, we cannot rely on that notice and so the DD is not allowed. Indeed, if we notify the customer anything else such as cancelling a collection, we discard the record of that notice so a new notice has to be issued.

Another aspect is checking it all worked, this is a key step for defensive coding - scripts and systems to check things that should work have worked. Such a system flashed lights, dinged my phone, and even set off the very loud teletype in my hall in an attempt to make quite sure I knew things were not quite right and that some intervention may be needed. That said, the comments on Mastodon and call from staff were also a clue something had happened!

So what did happen yesterday?

With all of this in mind, shit happens, and did yesterday, and a load of A&A customers were told their 1st June DD collection was cancelled. So what happened?

The root cause was an update to one of our billing systems SQL servers. We do updates to all sorts of servers all the time, and it seems the timing of this was unfortunate. Many updates are to ensure all security patches are applied, and so done as soon as possible.

Once again, defensive code - the systems generating billing records, e.g. for calls, queue them up until the SQL server is back. And the billing system using these databases to make bills will re-run a few hours later if there is any issue accessing the data so just delaying bills slightly.

The problem is that there was, apparently, a small window each month where there is a process of "working out what people will be billed in two working days". This is a test/dummy billing run. This is done to confirm the regular direct debits are as expected and ensure they are submitted to BACS two days in advance, so two working days before the 1st of the month, i.e. yesterday. That was the system that failed to access the database, and unlike normally billing runs, which can run a few hours later, it left a load of customers set up to not be expecting a bill on the 1st.

This is a small window, and as a result it meant that a load of customers were not expected to have a bill on the 1st, even though they will. If the bill on the 1st had an issue it would have run a few hours later and still billed. But this was a test/dummy bill run in advance and run only once a day.

It is not just the 1st, we have billing on 4 week cycles, and every full moon, and all of these work in the same way.

What happened then is the direct debits scheduled for the 1st were cancelled because the test/dummy billing run did not say a bill was expected. The system sending the cancel emails knows to also forget the previous notice of collection. This means that not only were "cancel" emails sent, but once the bills do happen on the 1st they will need a new Direct Debit collection notice email for 5 working days to be sent - so following the DD rules.

It means people will have a DD collections on the 8th, not the 1st, and then from the following month should be back to normal.

Whilst it is a mistake, the system, being defensively coded, has followed the DD rules for notices to the letter, delaying collections, and sending new notices.

What have we done?

We have changed the logic slightly, which means this should not happen in quite the same way, but we have also made sure the ops team know not to do an update in the middle of this test billing run in future.

Sorry for any inconvenience caused by an extra week's credit / time to pay on your bills. As always we try to learn from our mistakes. And well done to my staff handling all the calls and emails today on this.