ESP32-S3-MINI-1 GPIO

I now have a few boards, including a general test board...

The ESP32-S3-MINI-1 has 39 GPIO pins, though the data sheet defines that one (GPIO26) is reserved when using PSRAM (e.g. ESP32-S3-MINI-1-N4R2).

Unlike the S1 boards, the GPIO are all available as input or output, making its very flexible. But as always there are some caveats on GPIO usage.

The following relates specifically to the ESP32-S3-MINI-1 module only.

Restricted use:

GPIO0

GPIO0 is the main strapping pin that defines BOOT mode. If low at startup the device enters boot mode. It is pulled up internally so can be left unconnected. This means you cannot use this as an input if there is any chance of the input being low at start up. It can be freely used as an output.

By far the most common usage for GPIO0 is connection to the cathode of a status LED.

GPIO19/20

By default these are USB. This may make them not ideal to use for inputs. Note that they can be configured not to be USB by a fuse setting, but will be USB on an unconfigured module. As such it is probably best to use for USB.

GPIO26

The cannot be used with a PSRAM module, e.g. ESP32-S3-MINI-1-N4R2. It can be used if not PSRAM, e.g. ESP32-S3-MINI-1-N8.

GPIO43/44

By default the boot serial console uses GPIO43 and GPIO44. This means GPIO43 is configured as an output at start up. There are settings to stop any actual output on this. This means it cannot be used as a driven input without at least a resistor to handle the clash at startup. GPIO44 is an input so can be used for any purpose without any problems generally.

GPIO46

GPIO46 only matters when in BOOT mode (GPIO0 low) and in such case must be low. It is pulled low by default so an be unconnected. But do not use this as an input that could be high at startup in BOOT mode. It can safely be used as an output.

Caution:

GPIO3

This is defined as a strapping pin, and is floating by default. However it is only relevant depending with some specific eFuse settings related to JTAG operation. When using the default USB JTAG working, this pin can be used as a general input or output.

GPIO45

GPIO46 impacts the voltage used for SPI, but depends on EFUSE_VDD_SPI_FORCE. But this efuse is already set by default on this module, which means GPIO45 is ignored. It can therefore be used as an input or output.

Serial or USB console?

The S3 modules have two ways you can operate the console - the normal serial (UART), and USB.

USB

The default USB working is as USB JTAG and Serial console. This allows flashing and debug console, and a lot more debugging using JTAG and OpenOCD. This is by far the simplest way to manage the module and what I would recommend.

In most cases my boards have USB, either a printed tab for USB-A or a USB-C connector, but in the rare cases they do not, it is simple to include a header for USB to work with a lead.

This nicely fits under the module itself, with no vias, so takes no space and so no component cost. I deliberately spaced as 0.1" header and with a polarity to avoid confusion.

Apart from using JTAG, the USB serial is not really serial, so is way quicker for flashing code than using the real serial port.

There is one small snag...

It is possible to use an efuse, and more importantly, it is possible for the application to turn off the USB pins. This can even be inadvertent (I have done this, loading code for a different board which had an LED on GPIO19). This then means you cannot use the USB port at all, so cannot reset or load code or anything.

For this one reason, it is always useful to have a fallback.

Serial

The other approach for flashing and console is the conventional serial port. For this you need to have at least Tx and Rx (GPIO43/44) pins, and boot mode (GPIO0), and a means to reset (EN). You could use power on as the reset and keep it to just 5 pins. I use a header that works the same as the Shelly 1 and so can use my Tasmotiser board.

This also fits under the processor rather nicely with no need for vias, and no component cost. In fact, it can fit under the processor along with a USB header...

ESP32-S3

I have worked through a few of these processors - starting with the ESP8266. I moved to the ESP32 quickly and have not looked back - initially with the WROOM module (which can be hand soldered), to the PICO module (which is smaller and needs solder stencil and paste).

But finally the ESP32-S3 (version 3 of the silicon) is available as a module, the ESP32-S3-MINI-1, so I am finally starting to use that. The fact that the ESP32-PICO-MINI seem to suddenly be out of stock, seems like the time to upgrade designs.

The S3 has several improvements which may be useful.

Size matters

A key difference is size. The ESP32-PICO-MINI (i.e. S1) is 13.2mm square plus antenna, but the ESP32-S3-MINI is 15.4mm square plus antenna. So 2.2mm wider and longer.

For almost all of my designs this slight change does not matter, and indeed, for some the other changes actually make things smaller. There are exceptions - the module to go in to the case of a Galaxy keypad may be too big as there is very little space.

However, there is one useful side effect of this extra size. I have a small 5 pad footprint for serial programming - it uses the same 5 pins as the Shelly, and is pretty much the minimum you can have for serial programming and debug (see on image above on the right of the components). Now the module is slightly bigger than can actually fit behind the module, so effectively taking no extra space on a single signed board. This is one of those small details that saves space - e.g. the design show above is now a couple of mm shorter as a result.

General purpose I/O

Another change is the GPIO. The S1 modules have fewer GPIO (27), but several of them are input only - all of one side of the module has no output GPIO. There are also a number of NC pins.

The S3 has more GPIO (39), and all of them are input and output. There are no NC pins (apart from GPIO26 when using the PSRAM based module). This means all three sides of the module have lots of usable GPIO. This is perfect for tracking as you can easily track to the nearest convenient GPIO, which saves space.

USB

This is where it gets both interesting and slightly disappointing.

The S3 has USB built in, so direct connection of D- and D+ pins to specific GPIO. It can be disabled, and it can be used in s/w as USB server or client side as needed, which is very cool. This saves space on many designs if no UART needed.

The default, starting point, is USB JTAG with USB serial. This appears on my Mac, for a totally un-programmed module, as a USB serial port. I can connect to it and see the boot loader serial output. This is excellent as it means I can use a much simpler interface to the module - no need for a UART. As many of my modules have USB for programming and debug, this makes it a lot simpler.

However, there is a snag, it seems. Whilst this works as you would expect as a serial console port over USB, it seems it does not provide the normal connection for RST to reset, and DTR to boot mode (GPIO0). So you can only really use it to load code if you have a button for GPIO0 and RST on your board. This is a lot of board space, and really is silly.

 UPDATE: I don't know what was wrong with my first test boards, but this is working, so, all OK! I think perhaps I simply had out of date tools on my Mac (e.g. esptool).

Now, there are a few rays of sunshine.

1. It seems when un-programmed it does work in boot mode and so can be flashed, even though no RST or DTR working. That said, it seemed oddly intermittent, so I plan to do more testing on that. It may mean I can use USB for the initial loading of code and fuse settings.
2. Obviously as a USB JTAG interface it may be that there are ways to force a reset and boot mode or load code over JTAG, but this is not obvious from the documentation. If I can work that out I can make boards which do not also need my 5 pad serial header, which is what I was aiming for.
3. It is possible to internally detach this USB JTAG/serial and run USB from code in the module - this can handle code loading as a feature you compile in it seems (though even that suggests GPIO0 needed - why?!). I have not sorted that yet. My main concern is that if the code in the module does not have this, or is crashing, or whatever, I will have no fall back unless I have my serial header.

So more to do, and any help welcome. If there is a fallback for the USB JTAG working that allows code loading without having to do GPIO0 and reset, that would be really nice. For now, I'll keep my 5 pad serial, under the chip itself, as a fallback.

In theory OpenOCD is loaded, and should work for the JTAG. No idea how to use that yet, but that is a different matter as it does not work - some obscure error which sounds like a libusb issue on modern Macs. Arg!

Security

One small concern - JTAG is usually a very powerful interface on a system. If the chip is left with USB JTAG set up, perhaps because that is the only way to load code sensibly, it may also mean it can be controlled such as to allow secure boot and flash to be accessed. I.e. single stepping code after loading, etc. It may be that for security it needs the JTAG USB disabling (a one time e-fuse setting). I need to investigate. As a whole my security threat considerations have generally be based on physical access being a reasonable challenge, as the serial interface represents a similar risk in some cases, but it depends on the application.

Royal Mail API (RMAPI v4)

We have used Royal Mail for postage for some time. They had an XML API which we have used, and it works.

What is interesting is how all of these APIs are moving from XML to JSON now. One of the big benefits of JSON is none of the FUCKING XML NAMESPACES (sorry, did I say that out loud?).

Even so JSON has its issues, but nothing like XML with namespaces.

Manuals

They have a web site with a login and loads of interactive stuff, and that includes the API and API manuals. They also have a "V4 Shipping API Consolidated Guide" PDF they sent. So how hard could it be. I had two manuals!

To put this in context, I am someone that has used (and written) many APIs that are JSON/http based over the years. I practically invented the concept (I even have a patent on form posting from over 30 years ago). I have some clue, honest.

I actually quite like the latest "web" based stuff with JSON sent and received. Even OAUTH crap. I have some experience with this, honest.

Update: Before logging in there are some extra links, and another set of manuals, which is yet again different and contradictory - arrrg!

Authentication

First challenge is authentication, and to some extent the manuals were not that bad. The one critical thing they could have done is include, in any of the manuals, one simple example.

They use OAUTH2 but in a way I have not seen anyone else use. I needed a POST to a URL with a URL encoding post of grant_type=client_credentials (and it had to be a POST not a GET) using Basic authentication of the client ID and key they allocated. This is an unusual way to OAUTH2, but valid, to my huge delay of some hours getting the hang of it was really down to me. I get back a Bearer key with expiry (1 hour).

API URL

One issue is the URL, well, the hostname. The documents state that the hostname to use is for their TEST system. This is good, they have a test system, but the other manual suggests it is the hostname for the LIVE system. Why?

I eventually worked it out, there is one hostname for authentication, and whether it is test or live depends on your account. Your account can be in "sandbox". I mean that is great, a good system, but why the fuck not state that in either of the manuals?

What adds to the fun is that hostname is actually only to authenticate. The actual hostname for the service is different, and searching both manuals I do not see the actual hostname for actual API calls listed anywhere.

Why? WTAF?

JSON blob

The big clue was some swagger.json file you can download. Loads of JSON, some "OpenAPI" thing. That has been way more helpful than either of the manuals!

It explains the actual hostname to use for the API, and each of the schemas for each API.

The manual, well one of them, lists fields you need, but omits that these may or may not be in sub objects. The JSON blob explains in way more detail.

And, of course, this JSON file, whilst really good, is missing a load of things, like creating manifests!

Errors

For example, the error "Could not find member 'ContactName' on object of type 'Destination'. Path 'Destination.ContactName'" fooled me for a while as I was supplying Destination.ContactName. What it meant is I should not be supplying that. I read "Count not find" as I was missing something, but no, it meant I had supplied something it "could not find" a use for. That is shit UI. (FYI I should have provided Destination.Address.ContactName)

Tracking

There is, of course, tracking. There is an API to get tracking and an option to have tracking posted to a web hook - yay!

But the documentation for the tracking API just says the response is a tracking number? Which, err, makes no sense, what of the status of the shipment. Hmmm.

And apparently the tracking web hook is not supported, so why the hell is it in the API documentation? Arrrg!

Working

I have it working now - but this is literally one of the worst documented APIs ever.

The raw JSON file was more documentation than either if the manuals, but even that is missing bits.

Icing on the cake

Royal Mail have What3Words embedded. I kid you not.

This is so mental! RM have PAF which has detailed (like, to 10cm) delivery point details on all UK addresses. They are the one organisation that has ZERO use for W3W.

I don't know what they do with it - I will ask?

• If a W3W supplied is way off, like hundreds of miles, do you use it?
• If a W3W is next door to postal address provided, do you use it?
• If a W3W is within target address, e.g. a shed in garden, do you use it?

If all are "no", I have to ask WHY THE FUCK it exists in their API!

[Update: Answer is they don't use the W3W filed.... So WTF have it?!?!]

WIFi: Aruba vs FS

I have used Aruba WiFi for some time - they are nice, but expensive. The latest Aruba Instant series seem more sensibly priced though.

When picking WiFi there are a number of factors, but if you are going to get more than just the WiFi that comes with your broadband router you almost certainly want something that can allow multiple access points and roaming between them.

There is also a matter of the number of antenna and radios, and the protocol - summarised as catchy names like WiFi5 and WiFi6, these days. There are also access points for outdoor use, which can be useful.

The Aruba has one cool feature, and the main reason I like it, that it does not need a controller. A controller is what directs devices to move from one access point to another as you move around, and what manages the overall configuration. With the Aruba, the controller is one of the access points, and can even fall back to anther one if the main one is off.

So what changed

I have been pretty happy with the Aruba for many years. It has a lot of features and settings (perhaps too many), and as I say, needs no separate controller.

But recently some of my IoT kit (ESP32), which is all 2.4GHz, has been playing up, sometimes sulking not reconnecting to WiFi, and so I felt it was worth changing the WiFI to see if it helped.

Long story short, the WiFi choice was not the cause of that specific problem, and I think I have fixed it. But the change of WiFi had some surprising side effects. For reasons I won't go in to, I have all my 2.4GHz on the same channel. And so that is a pretty full channel. As such I was never surprised to find s/w upgrades on a set of IoT devices was always a bit slow - I set up to do one at a time. To my surprise, with the new WiFi, it is way quicker!

What is the new WiFi?

I have got some kit from FS. It is cheaper than the Aruba, which is a start, but it needs a controller. The controller (AC-1004) is also a 4 port PoE switch which makes it actually useful for a small home installation as you almost certainly want a PoE switch for your access points anyway. So again, not that much extra cost. It can handle 64 APs. I may be a convert to the idea of a controller.

The (web based) controls are a but more clunky than the Aruba, but to be honest, pretty similar. It also has a huge command line interface!

The APs actually come with mounting brackets, unlike the Aruba! I have AP-N505 indoors and AP-T565 outdoors.

So far, so good. I'll see if I am cursing the change in a few weeks. If not, I'll take down the Aruba APs.

BMA423

The BMA423 is a small accelerometer by Bosch. The data sheet is here. It is 129 pages. It is really crap if you want to just use this chip. It repeats itself quite a bit as well. The Quick Start Guide has a nice vague bit about writing the init file as a burst write to address 0x5E.

There is reference code, and the help forums for this just have Bosch saying to use the reference code. Some of us would rather not. A good data sheet should not expect you to just use the reference code. It is also a bugger to follow.

One of the main issues is the the reference code works with registers 0x5B and 0x5D which the data sheet lists as reserved. This is silly, the data sheet could define these. The reference code also has 6kiB of binary code that seems to need to be sent.

So, here is what I worked out, and it is now working.

Initialisation

Reading and writing registers is standard I2C register read write. I.e. write the register address and then data, or write the register address and then read data, allowing multiple byte writes or reads as normal (with on exception, see below).

Checking

• You can check all is well, and init done, by reading 0x2A (INTERNAL_STATUS), and confirming it is 0x01

Soft reboot

• You can do a full soft boot by writing 0xB6 to 0x7E (CMD).
• Wait 1s after soft boot

Loading ASIC code

• Write 0x00 to 0x7C (PWR_CONF). This disables aps (advanced power save).
• Wait 500ms
• Write 0x00 to 0x59 (INIT_CTRL)
• Send the 6k block to the ASIC address 0 (see below) - see here for the data.
• Write 0x01 to 0x59 (INIT_CTRL)
• Wait 150ms
• Check status by reading 0x2A (INTERNAL_STATUS), it should now be 0x01
• Read the ASIC address at 0x5B/0x5C, this is where you load FEATURES_IN (see below)
• Write 0x01 to 0x7C (PWR_CONF). This re-enables aps

Block read/write to ASIC

It seems there is a separate ASIC block of memory, and this is accessed by three registers.

• 0x5B is low byte of address to next read/write - BUT NOT AS SIMPLS AS THAT!
• 0x5C is high byte of address to next read/write
• 0x5E is the gateway to reading and writing

Normally, with I2C it would auto increment the register on read/write when transferring multiple bytes. This does not happen with 0x5E. You can write to 0x5B/0x5C as a 2 byte transfer, as per normal I2C.

The ASIC address is stored oddly. It has to be even (bit 0=0) and you have to read/write even number of bytes at a time via 0x5E. The address is stored such that 0x5B is (address>>1)&0xF, i.e. bits 1-4 of address. 0x5C is address>>5 (i.e. bits 5-12). So you need to allow for this when reading or writing the ASIC address.

It seems from reference code you can transfer up to a maximum of 8 bytes at a time, so you have to repeatedly write the ASIC address to 0x5B/0x5C then write 8 bytes to 0x5E in one block. However, my testing suggests you can transfer the whole 6kiB init in one go, so totally pointless reference code.

Once the init file is written, and the status is 0x01, you have to read the address in 0x5B/0x5C which tells you where the FEATURES_IN block is located in ASIC memory... Shift it appropriately to get the byte ASIC address.

FEATURES_IN

FEATURES_IN is block of 64 bytes in ASIC memory. You can read/write even number of bytes, setting 0x5B/0x5C to the base address (as read above) plus the offset in to the block. The data sheet does list these additional registers reasonably well. It just lacks any explanation of how you access these bits!

You can read all 64 bytes in one go, and write all 64 in one go, or read/write even numbers of bytes if you only need to change one register.

Steps

To enable step counting, I have done the following :-

• Write 0x00 to 0x41 (ACC_RANGE) to set ±2G
• Write 0x02 to 0x56 (INT1_MAP) - only needed if you want interrupts
• Write 0x04 to 0x7D (PWR_CTRL) to enable accelerometer
• Set FEATURES_IN(0x37) OR'd 0x30 to enable activity and counter
• Set FEATURES_IN(0x3B) OR'd 0x10 - not idea what this does as reserved in data sheet! 

Reading steps is then 4 bytes from 0x1E onwards, low to high bytes for 32 bits.

Resetting the step counter is setting FEATURES_IN(0x47) OR'd 0x04 (self clearing).

NON-STERLING TRANS FEE

One thing pisses me off is cards charging a Non sterling transfer fee. I'd have expected them to make money on the exchange rate, or be big enough that they can handle currencies without such silly fees to be honest. Even more so when it is someone like American Express, that, well are American, and I am paying something in American dollars. You would think they could handle dollars.

But I have recently discovered something slightly odd.

VISA

Using my VISA debit I was charged...

AMOUNT IN USD 206.87 ON 28 JUN VISA 1.269999 FINAL GBP AMOUNT INCLUDES NON-STERLING TRANS FEE £4.48 = £167.37

But when I was then refunded, it looks like...

AMOUNT IN USD 206.87 ON 30 JUN VISA 1.2669 FINAL GBP AMOUNT INCLUDES NON-STERLING TRANS FEE £4.49 = £158.80

This means I was charged for the refund, and oddly charged at a higher rate (by 1p).

Amex

Using my Amex I was charged... (different day, I have not compared exchange rates)

71.15 UNITED STATES DOLLAR, Non-sterling transaction fee: £1.68, Exchange rate: 1.2687 = £57.76

But when I was then refunded... (next day)

71.15 UNITED STATES DOLLAR = £57.82

Comparison

VISA charge the non sterling fee on charge and refund, but Amex do not. Indeed it seems they effectively refund the non sterling admin fee.

VISAs fee appears to be around 2.7%, Amex fee appears to be around 2.9%, very similar.

Conclusion

It is actually in my companies best interests if I pay for these on my personal Amex and claim on expenses, if there is any chance of a refund (which happens quite a lot if I have arguments with JLCPCB, for example).

toot

Twice I setup Mastodon.

Once many years ago - I followed the step but step guide and got it working. It worked for quite a while and then broke - out of disk. I gave up.

Then a second time we easier - this time my ops team set me up a VM, and Mastodon had improved a bit, so set up again, following the instructions. Again it worked. It caused problems as it was a "new" instance on an existing domain, and that caused things that had cached stuff to get confused. It got sorted, people had follow requests they had to cancel and try again, that sort of thing.

These days it seems to have a lot more "cleanup" logic, and a lot more obvious on disk usage, but still far from perfect from what I can see.

WTF is it not simple?

My big issue really is that Mastodon should be like apache, or many other quite complex systems, which are basically apt install mastodon. That really is what it should be. Simple as that. The web interface or command line could take you through the initial steps for admin user password and domain name and so on, but that should be it.

There are docker instances. I know nothing about docker, but I am told they sort of just work. Maybe that is the answer?

But why is it not as simple any anything else on linux, really. It seems Ruby is the main problem, as is python it seems. A pain in the arse.

But all is well, so not problem.

Yes, all is well, and even upgrading to next mastodon version as needed was not that hard. Again, following a load of step by step instructions not a simple apt upgrade mastodon or some such. But not rocket science.

But err, then, it broke. Basically the ops team wanted to upgrade Debian to bookworm, which is fair enough, but doing so cleaned out almost all of the stuff Mastodon needs.

Now I am not just upgrading Mastodon, and not installing Mastodon (as I don't want to lose/break what I have), I am sort of trying to make it work from instructions that are one or the other, and constantly hitting stupidities like OpenSSL is not available (how the fuck is that not standard in these tools?!) and not clue how to fix that.

I gave up

I must be getting old, because, yes, I gave up, it is too much of a pain in the arse, and I have better things to do. I tried, honest, but life is too short.

Bring in the experts!

The simple answer is to call in some experts. In this case Mythic Beasts. They actually run the A&A instance because my ops team were smart enough to not go near this with a barge pole.

They are not only running my instance, but in very short order managed to copy all the stuff from the old, and now broken, instance over, and make it continue seamlessly for me, well mostly - some avatar issues they are fixing. But followers, and so on, all working. I have not missed anything and was off for a day at most.

It is not cheap, it is several hundred quid a year. But what do I value my time at?

There is an irony here. I am paying that for my social media. If the likes of Twitter had people paying that for a really good, clean, not sales/advertising/pushy/confrontational/stressy service, they would make a fortune. That is not their business model.

The only mitigation is it may end up being more than just me - some family members, when they finally catch this boat. Of course, subject to whatever stupid rules the Online Safety Bill may create, LOL!