2019-12-24

Slow month

I spent over two weeks with a rotten cold, which meant doing bugger all.

I hate that, and even though I have recovered from the cold now, I am still a tad stuck in the "doing fuck all" Christmas spirit.

So, yes, sorry, few blog posts.

I have got my Christmas present to myself, a new watch (Matrix Powerwatch 2) which is pretty cool. And that means once again I am watching my steps and calories, which is good news. Whilst it is new and has quirks, it does indeed seem to be a pretty good smart watch that does not need charging (uses body heat and solar). So far it is working well, even with GPS tracking my cycling! OK, 4461 is not a good step count today, I was again being lazy, but I have been managing nearer 12k most days.

Whilst I am hoping to get my head in to bluetooth and such over the next few weeks, I am also looking at some impressive TI ARM processors as well. More to post on all of that later.

Merry New Year and Happy mid-winter festival to all, whatever your reason for taking a few days rest... I'll be opening some virtual presents in WoW Classic, Mirage Raceway realm, tomorrow.


2019-12-11

FTDI USB Serial MacOS Catalina Resource Busy error

In an effort not to be DenverCoder9, here is what I found.

New MacBook Pro, using an FTDI USB serial cable (used to program ESP32 chips).

The error was "Resource busy".

The problem is that googling this gets a lot of people saying they have this issue and a lot of answers which didn't help me - it seems many people do this to themselves by having the device open from some other app. Using lsof I confirmed it was not open by something else. Some pages even suggested update to python, but I confirmed that even a simple cat of /dev/cu.usbserial-A... said the same, so not python. Someone even suggested it was not a genuine FTDI chip, so I ordered a cable from RS, and the same issue persisted.

After a lot of googling, it seems to me that there must be some sort of bug in Catalina. It looks like FTDI USB serial should work on a Mac without needing custom drivers, but it does not work. It is recognised and the /dev/ created as you would expect, but shows the resource busy error.

Importantly there was no pop-up or prompt or error of any sort when connecting the lead, it simply appears in /dev/ as you would expect. No clue that there is a problem with drivers.

So, what did I do?
  • Install the VCP drivers from FTDI (2.4.2), it is a DMG you open and install. This all went swimmingly but did not help even after a reboot, and not obvious clue as to why.
  • I used the About This Mac, System Report, Software, Extensions, and saw FTDIUSBSerialDriver 2.4.2 listed, Notarised:Yes, Loaded: No, which was a clue.
  • Going to System Preferences, Security and Privacy, it showed text "Some system software was blocked from loading". However, this was subtle as the adjacent "Allow" button was not active. It was not immediately obvious but I had to "unlock" the window, and then I could click Allow, and select FTDI to allow the driver.
Then it worked! It seems you do need the FTDI driver for some reason, even though MacOS knows of the FTDI USB serial. It also seems that MacOS has made the process somewhat more complex.


2019-11-30

More fun with GPS...

The NMEA sentences from a GPS work on degrees, minutes and fractions. As a format they are horrid - the sign is a separate character N/S, E/W, and the value is fixed length so DDMM.mmmm and DDDMM.mmmm format.

The 4 places for minutes is not that precise, but more than enough for absolute GPS positioning.

But there is an alternative, and it is like the IPv6 of GPS. More precision that you could ever want - ECEF format is X/Y/Z in metres from centre of the Earth to the micrometer!!!

$ECEFPOSVEL,124015.000,3985352.629186,-54462.755461,4962832.255023,-0.008022,0.003776,0.022063

Amazing. So whilst not adding absolute accuracy, it does add some impressive relative precision.


Walking in the road to spell my name, LOL.

2019-11-28

Shit week (or two)

Some times things go wrong, it happens, but if I did not know better I would say some deity was out to get me the last couple of weeks!

  • My 3D printer started playing up, but in a subtle way. It was printing some parts slightly too narrow, which is crazy. I spent ages on this as the lid of a case did not fit on the base properly. They are a tight fit, but this was just too tight and meant it cracked the edge of the lid. I adjusted tolerances in the design, printed loads of tests, and finally the lid fitted the base just, but no idea why.
  • I then put the lid on the widget I was making which has an OLED display. This is glass, and has a very thin bit at the edge. Well, the lid is carefully designed to leave a small amount of space around this, but it printed wrong didn't it! So fitting it cracked the display.
  • So I had to remove the display from the base board, and it is fitted with square pins which are a bugger to desolder, so I cut them. But the cutting caused the pins to tilt enough to rip the tracks off the main board. So I had to make a new main board.
  • The 3D printing got worse, now it was clearly squashing the design at a particular point, but only some of the time!
  • It turns out it would print sensibly to start and then get worse, but also a small print would be fine (it broke as specific points on the X axis) all of which caused confusion as I would think I had found the cause (e.g. the X axis rails were slightly skewed) because a small test print (after tinkering) would seem to have fixed it.
  • I tried all sorts, and eventually ordered a new stepper motor. The one from Amazon was wrong current, so did not work, but I found that the original stepper motor was actually slipping on the flat edge of the spindle, only at certain points and only when it warmed up. All this, wasting nearly a whole week, because of a loose grub screw. Oddly Lulzbot totally failed to answer support emails (apart from a e-ticket number) which is really bad of them.

  • So, re-installed original stepper motor, tightened, applied thread lock, and bingo, working perfectly again, phew. Back in business.
  • I then tried to mill a new board on my milling machine. I have done a lot of these, and I am using a simple bed-levelling program I wrote to measure the corners and adjust the cut over the board to allow fine tracks to be reliably printed. But this one new board (which was bigger than previous ones, and has some long straight cuts) was not cutting correctly - turns out I had a bug in my code that got the Z level wrong (was for previous point not current point) causing some tracks to cut at wrong depth. Just enough to not show up most of the time, but create duff boards several times.
  • So I was improving the code, and making it probe several times for each point and more slowly, and so on....
  • And just then the main milling motor on my milling machine stopped working. It would spin up slowly and then grind to a halt.
  • I tried on a bench supply and it is fine, so clearly the power supply not the motor.
  • I ordered a 48V 3A power supply, and that was not beefy enough to run it.
  • I ordered a nice 10A bench supply, which I am waiting for now - will it work - I'll add a PS to this post later today :-)
  • In the mean time, with 3D printing working, I had managed to make one working PCB, so made a case for the latest widget and it did not fit. For some reason I had made a silly mistake in measuring the parts, which took my about 6 attempts (at over an hour per print) to get right - this is crazy.
  • However, the slightly wrong print (due to design, not printer, this time) meant the button cell battery holder moved, pulling tracks off the board, shorting it, and that took me ages to debug as the GPS refused to work with no backup battery.
  • I set up the Quectel M95 mobile module, and surprise surprise all of the commands I need are different to the SIM800 module I was using. Arrrg! Thankfully they have a good manual. So new GPS tracker working well.
  • Then something else stopped working on the board - I don't know what yet - I need to use my magnifying glasses to work on this board, and guess what, they just snapped.
  • And in the middle of this, other issues, involving someone having unexpected hospital visits (which have turned out fine, phew).
It has been a shitty couple of weeks really.
It can only get better :-)

P.S. And not the plastic cover thing on end of my glasses has come off so I stabbed myself putting them on.

P.P.S. the cheapest, unbranded, iffy looking DC power supply I have seen, from Amazon... Actually has a CE mark, but not sure I'll ever leave it unattended. But it works - yay!

2019-11-11

Hayes AT

A long long time ago, in galaxy far far away (no, scratch that bit), a standard evolved for talking to modems, well, specifically Hayes modems, but it was widely adopted.

It works over a serial link by sending commands starting AT for (attention). Now, I am pretty sure they were reasonably consistent, back in the day, but like many things this standard has evolved and been bastardised.

We now have things like this SIM800 module, which is basically a mobile phone in a can, with no display or keypad. It talks on a serial bus and talks Hayes AT commands, but as modified by various GSM bodies.

Now, some AT commands are consistent. You send AT and some options, and get some data followed by OK. Good.

There is a reasonably consistent format for commands and settings and queries, mostly. There are then AT+ functions which are used for a lot of the mobile stuff.

But it is far from consistent. The AT+CIPSEND, after the > sends "SEND OK" not "SENT OK" or "OK", and AT+CIPSHUT sends "SHUT OK". Why are these not just "OK" like the rest? I am also pretty sure I had one case of a response with no OK (but cannot find it now).

Then we have the asynchronous messages from the device. Most start with a +, like "+CREG: 0,1", so can be recognised if you are expecting another response. But some are just text like "SMS Ready". These smack of debug messages left in the code.

Half the interface was clearly designed for a person sat at a terminal typing stuff, and the other half was designed for talking to a machine!

But when you get to some of the higher level functions, like establishing a UDP "connection", it gets even more special. For a start, you look for "CONNECT OK" not simply "OK", and mostly at that point it seems to avoid sending asynchronous messages (though I cannot be sure). Some still can be sent, like "+PDP: DEACT" when it loses the connection. What you do get though, with no prompt, header, or indication of length is the raw binary content of any UDP packets you receive on the connection. So send a packet containing "+PDP: DEACT" and I'll think the connection has dropped!

What would have been sensible would have been a line something like "+DATA=N" and then N bytes of data, or some such, but no, it is simply the raw UDP packet.

Then we have some interesting little bugs and discrepancies. One that took a while to work but was that the length of UDP data I can send is limited. Not to 65535 which is what IP limits to, but to 1472 which is the available UDP in a 1500 byte IPv4 packet. Except it is not. It is actually 1460, any more says "ERROR". Oh, and it won't allow me to send a zero length UDP packet - no idea why, and as NAT is often used this is something I may want to send as a keep alive. I ended up having to send one byte.

But there is an even more special feature. When you receive a UDP packet, as I say, it arrives as the raw binary data via serial. I am relieved to see no mangling of any bytes, which is good. Except it truncates the last byte of the message. Yep, I have to send an extra dummy byte on my packets to the device. WTF?

I could save myself from a lot of this by coding PPP, pretending to dial on an old style modem and talking PPP which the device fakes and converts to packet data. This is a serious consideration, to be honest.

However, after a lot of messing about, my GPS trackers are sending (and receiving, when needed) tracking data over UDP over mobile - yay!

P.S. I may have maligned the SIM800 as incoming UDP looks like it may be my bug losing a byte, and there is an option for a header with length for incoming UDP. But the Quectel M95 does look a bit nicer.

2019-10-31

GPS trackers

I am playing around with GPS trackers (I'll post more on hardware for this later). This is for commercial vehicles, not anything covert, and I am not going to even touch on GDPR in this post :-)

Reducing the number of points

One of the challenges is managing how often we sample location, and if/how we reduce the number of points. There are a number of reasons to do this:-

Local Storage

One reason to reduce the number of points stored is where the tracker is storing the points - either for download later or for transmission when mobile coverage comes back, etc. The local storage of a small embedded device is likely to be limited. The GPS module I am using has several ways to do this (I have yet to work out how to do other than simple interval). Storing every second, which it will happily do, runs out of space in a few hours.

Transmission

Transmitting the data over a mobile network has costs. Yes, there are SIM packages with unlimited data, but if you have a fleet of hundreds of vehicles you can do better than each of them on a package for tens of pounds a month and unlimited data. There are packages with much lower monthly costs but data usage costs, and there are foreign SIM packages with roaming for better coverage and higher data costs. Reducing the data transmission can get the costs down to pence a month. This means not only reducing the number of points but finding ways to reduce the overhead of transmission of the data itself.

My plan is to pack data in to an encrypted UDP packet, with a simple resend packet sent back if the data has not arrived at the expected time. I.e. an entirely NAK based protocol so that normally it is one packet in one direction, probably every 10 minutes or so. This means really low mobile data usage.

Database storage

Reducing the number of points also helps with back-end storage. Hundreds of vehicles tracked every second means a lot of data. Simply increasing the interval loses useful data so you ideally need a smarter way to reduce the data.

Display

Perhaps a small point, with computers and networks being so fast, but reducing the number of points sent to a browser to display a route can make things much more responsive.

Where to reduce points

At present I have code to reduce the points when pulling from a database and sending to a browser, with an option to also delete the removed points from the database. But ideally you want to do this in the tracker itself - that way you have all of the benefits for local storage and transmission as well.

This also creates an extra advantage - if you have a means to reduce data points effectively you can start by collecting much more data in the tracker. The GPS modules I am using can do 1/10 second fixes, so why not start with this, and reduce points from there.

How to reduce points

I pondered how to do this, and came upon with something based on removing points within a certain distance. I then found there is a standard algorithm which is basically very similar to what I came up with. The Ramer Douglas Peucker algorithm.

It recursively takes a set of points. If all intermediate points are within a specified margin of the straight line between start and end, then those intermediate points are removed. If not, the one furthest away is kept and the point from start to this, and from this to end, are recursively processed. The end result is a path of straight lines where all the removed points must have been within the specified margin of that line.

This works well, and removes points on long straight roads but keeps points driving around a roundabout!



Even setting a margin as small as 1m massively reduces the number of points, and what is even better is that you can start with say 10 times as many points (e.g. 1/10 second fixes) and still end up with similar final number - just that the points you select are more precise (i.e. the exact edge of a turn, not just nearest second).

Of course I won't have a full set of points in the tracker, I'll have to work on time periods, like 10 minutes, collect data, and then apply the algorithm. This means there will be points based purely on time as the end points for the chunks of data I process.

Distances

One issue is that the raw data is lat/lon and not ground distances. There are ways to work out exact ground distances around the curved surface of the Earth, but a simpler way is to just assume we can map lat/lon on to a flat plane. What I do is, for each stage of the algorithm, take all lat/lon relative to the middle, and convert to metres. For latitude this is simply multiple my 111111 (yes, apparently this comes from French definition of a metre). For longitude it is 111111*cos(lat), remembering to convert to radians for cos(). This means I can then work out distance from a line in metres with minimal error. Of course as the algorithm recurses and processes smaller line segments, the error because the Earth is not flat (really, it is not!) reduces.

3D

For a change, I am not talking of 3D printing, but we have altitude as well. So I am including altitude in the calculations. This means that a straight road that has a peak hill gets a point kept at the top of the hill!

4D

Yes, 4D printing would be cool, but no, I am now including time in the algorithm. That way we get points if a vehicle stops and starts or changes speed. The only issue here is deciding how time works as a unit in the distance calculation, so I need a metres/second. I have made this small, treating a 1 second deviation from the line between two points in space and time as a small fraction of a metre, hence you have to change speed or stop for many seconds to create a point you keep. It seems to work well.

2019-10-29

Dropping summer time

It seems moves are afoot to stop the clock changes, and I rather like the idea.

The issue, for me, is that people talk of "permanent summer time" as the answer, sort of forgetting that the clocks are set based on lots of measurement at the Greenwich observatory and hence why we have Greenwich Mean Time. It is summer time that is the "anomaly" and it only makes sense to me to be on GMT all year.

This assumes stopping clock changes is a good idea, and that seems to be a matter of debate, but one which is leaning towards the idea of not doing it.

So, some thoughts on why, to my mind, it is obvious we stick to GMT all year.

The matter of definition

It is called GMT, and is generally the high sun at the middle of the day at 12:00. It would be crazy, IMHO, to have the mean time over Greenwich as Greenwich Mean Time plus one hour.

What would we call it?

We can't call it GMT and it makes no sense to call it BST as not only summer time. It won't be UTC if +1, so actually we need a new name. That is more of a pain that you realise as it is a lot of software and all sorts. UKT maybe?

It only matters in winter anyway!

In summer we have more than enough daylight. Yes, moving some "unused" daylight from early morning to evening is vaguely helpful, or was before the invention of street lights. But what matters is Winter.

We tried a permanent summer time in 1970 and it did not go well. We know GMT in Winter "works" well enough and fits around schools and so on.

Changing what we do in Winter (by making permanent UTC+1) would be way more problematic as we simply have less daylight to play with. Changing in summer (by stopping BST) is not an issue as we have daylight to spare, so to speak.

Give UK an advantage?

There are a lot of things that log in UTC, and confusion over UTC and local time happen all the time. Heck, even the Amazon app gets the order date wrong all summer because of this. BT get this wrong in their XML in lots of places. There is an advantage for the a country that stays on UTC+0 as their time zone as all UTC logs will be local time, and simpler for everyone.

Some have even suggested that UTC should be moved to the middle of the Atlantic so no country is UTC+0 and hence has any advantage. Let's give the UK that advantage.

2019-10-27

ESP-IDF re-partitioning the flash on the fly

The ESP32-WROOM-32 has a 4M flash, which is usually carved up for boot loader, some small areas (NVS, etc), and then three code image spaces: 1 for "factory", and two "OTA" areas which alternate. Well, that is one of the two defaults you can set.

Sadly, even the most basic of code, with even log levels of logging enabled can easily hit a 1M code space limit. So I decided to make a new partition table.

Just two OTA spaces

The first thing I confirmed is that if I just have two OTA areas, and no "factory" area, it works fine. It can alternate the OTA flashing, and make flash just flashes the first OTA area.

Oddly, it looks to me like you can have two OTA areas of 0x1F8000 but the make system said it would not fit. I ended up going for 0x1F0000 which is close to 2M.

On the fly update

The real trick was changing the partition table on the fly. The key issue here was ensuring the original factory build was in the same place as the first new OTA block. I changed from factory+ota_0+ota_1 to just ota_0+ota_1 at 0x10000. This meant, whichever image I was running, by changing the partition table a reboot would find the factory release (in ota_0) and run it.

I simply had to erase and re-flash the partition block. I obtained the signed binary partition from the build directory and embedded. Then I just reboot, and sorted. Obviously you then need an OTA upgrade to move off the original factory code.

The simplest check was to compare the current partition table with the one I want, and if different, re-flash and reboot.

You do need to tell the SPI flash component to allow you to erase/write to dangerous areas of flash, but otherwise the code was pretty simple.

End result, I have nearly twice the code space.

2019-10-22

GPS made easy

I am surprised how simple GPS is these days. For only £12 or so (from RS of all places, part 908-4085) I can get a small module with built in antenna that runs of 3.3V and provides serial GPS data and a very precise PPS output, is only 15mm square, and easily soldered to a PCB. It even has built in RTC that can run off a small button cell battery for fast fix and retaining time of day.

I have yet to decide on the best applications of this, apart from simple things like a speedo for a bicycle. There are applications with location logging (that dump to WiFi when back in range), or even if you just need a reliable time source for something like a door entry system (or a Brexit countdown clock, LOL).

A clock showing how long until sunset, wherever you are, would be cool :-)

I have made a module with GPS (the Adafruit module, as I found that before the RS part), and a display, so I can have a play around with it.

P.S. I have it set up for my bike, even with amount of daylight left...


2019-10-15

Dissolvable PVA support

The TAZ Pro has two extruders which allows me to try and use dissolvable PVA support. Support is simple enough but hard to remove from the print cleanly, so using PVA allows extra options - just dissolve it!

I have had to play with the Simpify3D settings a bit as the settings for the TAZ 6 did not quite work. I am not sure if the bed is different, the nozzle, or what but my prints were almost welded to the bed. I have tweak the settings for nice clean prints with nGen now. The next challenge was settings for PVA supports.

I ordered some from RS (yes, the price was not silly, strangely), part 174-0082. Well, actually I ordered from someone else, and realised wrong diameter, after opening it, D'Oh, but now I have the right stuff I googled a bit to find temperature.

It is funny stuff, and I ended up printing at 205C which is apparently on the high side. I could also see from simply feeding the filament that it came out thick and slow (around 1mm).

The key setting needed to stop it just curling up was speed - it needs to be very slow. In the end I ran with 2x multiplier, 1mm wide, 20% print speed, and that actually worked. Well, mostly (the eyes went a tad wonky, but worked).


For a start, the PVA comes away from the model really easily, so that is a good start.


Then, put in warm water for a while to remove the last bits, and yay, it worked.


Now to try something more complex with enclosed parts that simply could not have been printed before.

2019-10-10

LulzBot TAZ Pro (dual extruder)

I have been 3D printing for a long time - and even coded a slicer once.

I have tried many printers. My current favourite for cost and performance is the LulzBot TAZ. We have a TAZ 6 at the office and (even though not on the A&A web site) we do 3D printing if you want.

It is a nice heated bed, which works well, especially with my filament of choice ColorFabb nGen.

I have been using to make cases for various electronics / R&D.

Sadly I slightly fried my TAZ 6 units. I have a new controller board on the way, but decided to check what they have now, and saw the TAZ Pro. So I ordered one and to my surprise it arrived in a couple of days (in spite of warnings of back orders and long lead times). Nice.

It is nice...


But is it worth the price (around £5k)?

Well, possible. It has some nice features and works very well. I have only been playing with it for a day now, but I can make some comments. The bed levelling is faster and seems more consistent. The Z axis is belt driven meaning it can easily use Z moves when moving from one part to another (far too slow when Z axis is screw). The big thing is the dual extruder.

Cura?

LulzBot recommend Cura, a good, free, 3D slicer. It works well, and makes some impressive prints, but...

  • It seems way slower than I am used to (I used Simplify 3D before)
  • The prints, whilst nice, and precise, seems brittle. I expect minor tweaks to settings would help.
  • It expects nGen to be glue sticked to the bed - and I know it can be printed without, but needs hotter bed for first layer. I tried that and it worked.
  • Somehow the prints were distorted. I am not sure if cooling is right. I am sure it could be tweaked.
However, it did work, as I say. And the reason I tried it is that Simplify3D, which I was using, had no profile for the new printer.

But, Simplify3D support emailed me one, within a few hours of asking. It works!

Simplify3D

Whilst I do not usually end up recommending paid-for software, this is good. It works well. It is way faster. It is not perfect, but nothing it, but it is nice to use. So I recommend it.

Dual Extruder

This is the first time I have used a dual extruder. But the TAZ Pro does it really well. The heads are motorised to retract completely out of the way. The full bed size is still available to both extruders. It really works well.

There are several reasons to use a dual extruder.
  • Two colour prints - obviously. To be honest this is a pretty minor use case. But it works well.
  • Mixed materials - combine fixed brittle material and flexible rubber to make complex designs - not tried it yet, but fun possibility.
  • Using dissolvable supports - this is likely to be the main use.
The dissolvable supports will be subject of another blog when the PVA reel arrives. Basically, 3D printing like this cannot always print what you want as some things are "in mid air". Using support material works, but has to be cut/broken off, and is impossible to do "inside" some designs. Using dissolvable supports allows the impossible to be printed, and then put in warm water for a while. I look forward to it.

The dual head working is impressive...


In the mean time - two colour test print...



Any downsides

The only one so far is the heads are covered by a safety guard with warning about being hot - which means you cannot see what is printed as it prints.  Ironically it means you cannot see the print head, and can end up burning your finger when trying to adjust something because you cannot see where your finger is. Elf 'n' safety gone mad, IMHO. If it always "just works" then I guess it is not a problem. TBH they should have designed with angled plastic to make a guard that allowed viewing of the actual print head. This may seem trivial but I do feel is more important than they realise. Even ignoring the practical aspects of knowing a print is not working right ASAP, seeing what is being printed it important for selling the whole idea of 3D printing...

USB-C not so easy

Using a USB-C connector for power only is, sadly, not as simple as I thought.

Yes, the pins are tiny, butt the end pins are double pins on the connector and just about solderable (0.6mm wide for two pins together).


However, whilst this works when you use a legacy cable from a USB-A, giving 5V, this will not work to a true USB-C power supply or device.

Why?


Well, resistance is not futile, it seems. There are two connectors (CC1 and CC2) on the USB-C which need resistors to GND to tell the other end you want power.

Whilst the end power connections are two pins at a time (0.6mm), the CC1/2 pins are not, and are 0.3mm wide at 0.5mm spacing. That is pretty much impossible to mill, as the milled track is anything from 0.3mm to 0.6mm wide. The picture above shows around 0.5mm milled track, which is deliberately spaced to allow soldering to the pin 6th from the right. So double pin on right (GND), next double pin in (VBUS) and then another pin we don't care for and then the CC pin. Same on the other side of the connector, except the CC pin is next to the VBUS on that side.

Yes, it is possible to solder!


Then to add the 5.1kΩ resistors to GND.


The end result is power from USB-C...

P.S. Thanks to John for pointing this out to me.
P.P.S. John also pointed out that you do need the two resistors - and cannot simply common up CC1/CC2 with one resistor (as RasPi4 did, by mistake).

2019-10-08

Powering widgets

Making widgets for the alarm system means that power is quite simple - the alarm wiring has 12V DC, and a simple regulator means I can connect to that - typically with screw terminals.

However, making other widgets with an ESP32, like the environmental monitor, or the Brexit clock, means finding a way to power them. I am not yet playing with battery powered stuff (will do, eventually), so need a power lead.

So what to use?

Initially I tried a micro-USB. Leads for this are very common, and USB-A sockets providing 5V are so common they are fitted to standard power sockets even, so seems ideal.

The issue is the connectors. I have struggled to find what I need. I did see some from China (which have not yet arrived) which have only the power tabs on them making soldering easier, and have clips to hold to the PCB. Whilst waiting for those, I tried some simple surface mount connectors.

The problem is they are not very robust, even superglued to the board, they can become detached very quickly.

I have actually made a set of tools under OpenSCAD two make tight fitting 3D cases to help hold the connector in place, which has helped, but that is still a challenge, and I am not happy about it.


The other catch is the tiny tiny surface mount connectors... Thankfully I only need the ones at the ends. For reference, the three pins at the bottom are 0.1" spaced.

So, I wanted to find a solution, and turned to USB-C. These are a nightmare, not only tiny tiny tiny surface mount tabs, but tabs under the connector so you cannot get to them with a soldering iron. Finally I found a part that will do, and from a UK supplier.

It did mean some fine milling, and even finer soldering, but again, only needing the double connectors at the ends, so just about possible. Those pads are 0.5mm spaced!


The result is a nice solid, soldered to the board, connector.


2019-10-02

IoT

I have not done much on the IoT for over a week, but being rather ill for a few days has not helped 🤮🤮🤮🤮🤮.

The Brexit clock is getting scary!

My grandson wants to do something with LEDs some time!

The environmental sensors are working well, and I have nice graphs from it... This will probably lead to some fans in the office now.



I am thinking the next IoT thing to look at may be Bluetooth. Any suggestions for how that could be fun?

But yeh, the LEDs for my grandson are going to be a thing soon. Here he is abusing his first LEDs.



2019-09-24

Brexit

Back in 2016 I posted how there really was not enough information for anyone to vote on the referendum to leave the EU. How could members of the public make any sort of informed decision.

We are now way down the line, and things are sadly only slightly better.

I am amazed how this has polarised the country. Even people in "my bubble" can be massively Brexit still and there is no reasoning with them. And so much disinformation still exists, even now where Boris Johnson said we could ban Shark Fin Soup if we left the EU. WTF is that as an argument? Who cares about Shark Fin Soup?! From what I can tell it is either "not true, WTO rules means you can't ban it", or "true, WTO rules have an exception, but that means we can ban it now even in the EU". Still the mad sound bites of disinformation persist even three years later.

Is change good?

If you are a good businessman then change can always been good - you can always exploit it. Even when you look at war and the blitz spirit (as has been evoked regarding Brexit?!) there were "businessmen" that made money (and were illegal as black market traders). Imagine in the blitz, in an air-raid shelter and bombs dropping - we survived. But imagine if 52% of people in there voted for the bombs?

And if you know a change will happen you can be even better at exploiting it. It is coming out how some people in politics have direct personal gain linked to Brexit. Scary.

But "good" for some careful businessmen will always be "bad" for someone. That money they make comes from somewhere.

In my personal view I cannot see Brexit being good for the UK overall. It seems counter intuitive to be isolationist, and then still want to trade with the world.

Personally, I am hoping that I am finally doing well enough in business that I, and my family, can cope. But that is by no means certain, and I am scared! Even if it was the case, I would not wish the hardship that will ensue on everyone else, whether my friends, my staff (also friends), my customers (also many friends), or anyone else.

Limbo

One of the huge issues is uncertainty.

A50 can be revoked, or delayed, so nobody knows for sure what will happen right up to the wire!


We cannot have this again - we need the A50 process to be 100% certain, no delays and no extensions, and no way to revoke. A country leaving EU makes one decision to do so and then has time to make it happen, and that is it. That needs to be the case even if the UK decides, again, to leave.

What we have now is hugely damaging, maybe more so than having left already!

As EU members we should change the A50 process to fix this!

We import from China!

What interested me was comments from a close friend. Smart chap. We make products that use chips that come from outside the EU, like China. The comment is that Brexit is therefore not so much of an issue, surely.

But we (UK) don't import from China, we (EU) do. So when we Brexit, we (UK) have to have the trade agreement to import those parts to UK (not EU). Is that in place? - well I don't think so, even after many years.

It is so easy for anyone, even smart people, to miss some of these obvious details. Brexit is not simple.

[I am not sure China is necessarily the best example, but assuming just because we trade with a non EU country before Brexit does not mean we, as the UK, have a trade deal to allow trade after Brexit. It needs negotiating. The government claim to have managed to negotiate several "continuation" deals already, which is a good start, but they needed negotiating, and more are needed still, they are not just what automatically happens when we Brexit, which is what people were assuming.]

Will of the people!

This is perhaps the most annoying part! We accept people can change their minds as we have a general election every few years, but somehow Brexit is a "fixed point in time" (DrWho?).

In fact that the referendum was a "poll" (as stated in the legislation). It was not a mandate. And then it was (as decided by courts) conducted outside the legal constraints for such campaigns. Had it been a mandate it would have been struck down, and was not only because it was just a poll.

As I said in my 2016 blog post, I did not know what was the right way to vote. I voted to remain, but I did not do that as an informed decision by any means.

Now, we have some clues on the issues with disentangling the UK from the EU. At best we have some good deal with EU - but such a deal means we have to abide by a lot of EU law on the way we do things with no say in such law! The alternative is not having a good deal, which is worse.

We have the crazy situation now that calls for a "people's vote" are called out as undemocratic. The idea that asking the "people" to vote on this is not democratic is, well, mental!

We also have the issue that this is "one sided". We leave, and no way we ever re-join on anything like he deal we have now. We stay, and we can vote to leave again in a few years if the EU goes OTT and we don't like it. So leaving is a "really really sure you want to do this" type of thing.

Breaking the deadlock!

I see the only way to resolve this is something that everyone can see is the (current) "will of the people", such as another referendum or a general election fought on the Brexit issue. Only then can anyone say, again, "you lost, get over it".

That will still be bad. A lot of people convinced that Brexit is right will be cross. I don't know what to say. I have yet to find a single person who can explain the benefit of leaving!

I do feel that if Brexit is cancelled, whatever government we have will need to make a big issue of "why Brexit". They need to tackle each and every one of the reasons Brexiters wanted to leave, in detail, and with actual change. Some things are simple (blue passports), and some are not so simple, but they all need addressing. I feel we can sort the underlying issues and stay in the EU. Why not?

Bad EU laws?

But are there bad EU laws we don't want?

Yes, absolutely, there are laws that, in my view, go too far, or get it wrong. Not least of which was the damn cookie stuff creating those damn cookie consent warnings on every damn web site! That was technically stupid in so many ways, and I am sure many other such laws are "wrong". Sadly GDPR adds to that - mostly good - a tad bad.

The problem is that there are always wrong laws, and the UK has passed many (RIPA / IPA) that have bad things in them on both moral and technical grounds.

The EU is not immune too this, and neither is the UK. But leaving the EU means we have no say in such EU laws. We cannot fix them. It does not, however, mean we can ignore them as almost any trade deal will mean complying to standards set in such laws.

And to be fair, most of the laws are good! As a manufacturer, the hassle for CE marking is a nightmare now, but even though it is hard work and costs a lot of money it is all "good". It stops stuff catching fire, for example. The UK would mirror almost all EU safety laws as that is both sensible, and needed to sell to EU!

Conclusion

  • I'd like to stay in the EU, though a lot of that is because of the obvious chaos caused by leaving it at this point.
  • I'd like to see all of the underlying issues that Brexiters want addressed looked in to and sorted, as part of the EU. This is important - a lot of people are unhappy, and this needs addressing!
  • I'd like to see MEPs taken a lot more seriously by all - this is how we shape the EU to be what we want!
  • I'd always consider leaving EU in future as an option, but really only makes sense if EU starts to really fall apart.
  • I'd like to see A50 process fixed for any country planning to leave to remove any "left in limbo" effect - this is absolutely crazy!

2019-09-21

99% of insurance claims

I have seen/heard a couple of slightly odd averts lately. Yes, I try and avoid adverts, but these seemed a tad odd.

One was an advert for a life assurance company proudly claiming that they "pay 100% of claims". The implication (though, I think, not said) is that other companies do not.

This seems a very strange claim to me. That can't mean it, surely. After all, if true, I should put in a claim. I'm not a customer, but if they pay 100% of claims it would be a way to get free money.

So I assume they mean they pay 100% of valid claims. But then that is what I would expect, what is their contract, and indeed what I would expect of every such company. Surely no life assurance company refuses to pay a valid claim, else they just get sued.

The second advert was an insurance company proudly proclaiming that they pay "99% of claims".

Again, odd. If they mean 99% of all claims, then they seem to be proud that only 1% of claims are fraudulent. I have no idea if that is good or bad compared to the industry, but I cannot see how it is a marketing statistic in any way. How is only 1% of claims being fraudulent a "selling point". Or are they saying that actually there are more than 1% that are fraudulent but they pay them anyway, thus being proud that they are being ripped off (and hence the customers that pay premiums being ripped off, ultimately). Or are they saying they don't pay all valid claims? I.e. you have a 1 in 100 chance of your (valid) claim not being paid if you go with them.

I just cannot understand the marketing point being made by either of these adverts. Crazy!

2019-09-20

Countdown and clock changes

As I mentioned, for the Brexit clock, I had to allow for the clocks changing (something many such countdown clocks do not allow).

Basically, one of the days between now and (the current) Brexit is 25 hours long. This, in itself, is not complex.

I have tinkered with the way I do this a few times, and eventually came up with this, starting with how many whole days...

  • Work out a time for UTC midnight on target date
  • Work out a time for UTC midnight today
  • Divide time difference (as seconds) by 86400 (a day) for number of days
  • If the current local time (HH:MM:SS) is on or after the target local time (HH:MM:SS) take off one day from that total.

That is the easy bit, and means that the number of days changes at the local time (HH:MM:SS) of the target, so for Brexit, changes at 11pm local time exactly each day. What normally happens is the time part then counts down from 23:59:59 until 00:00:00 at 11pm local time the next day.

However, working out the time part is one area that took me a bit of thought. In the end I make use of the fact that linux mktime() function allows tm_mday to be any value and it adjusts.

So, what I do is :-
  • Create a date for the target date/time, local time.
  • Subtract the calculated number of days from the tm_mday part of that.
  • Work out time difference from that point to current time, and convert to hours, minutes and seconds.

The effect of that is that once the day changes, the time part goes down without a discontinuity. It means that on the day that has the clock change the clock will start counting down from 24:59:59 instead of 23:59:59.

For Brexit, that means that 11pm on Saturday 26th October 2019 it will go to from 5 days 00:00:00 to 4 days 24:59:59 and count down for 25 hours until 11pm on Sunday 27th October when it goes to 3 days 23:59:59.

Of all the ways to work this out, I think this is the best. It means days are counted to the local time on each day that matches the final target local time (so for Brexit, its counts a day at 11pm local time each day), and then that the time part counts without a discontinuity down to 00:00:00 at the same local time the next day.

Yes, I put far too much thought in to this.

2019-09-14

Brexit countdown clock

Having made the environmental sensor, I have now cut down the design to just the processor and OLED display so I can make other things.

One example is a Brexit clock. I have put the code and PCB and 3D case on GitHub (here). It may need regular code updates, obviously :-)

It did raise one issue though. Most of the Brexit clocks I see, like Sky News's (here) seem to be an hour out. Brexit is at 11pm but they show days plus time that would be midnight.

Now, I know why... Someone is being lazy and simply dividing time of Brexit minus time now by 86400 and ignoring the clock change. One of the days between now and Brexit is 25 hours long, but that does not stop it being "one day".

My clock gets it right, obviously.

There are other uses for the app - and as I am taking one of my grandsons on a his first cruise, I thought he may like this...


He heard the phones in the suite have a button labeled "pizza" :-)
He is already packing!

P.S. This is why I have a milling machine and a 3D printer! I came up with the idea over coffee this morning at around 9am. I had to design a new PCB and a new case, mill the PCB, 3D print the case, and assemble it all, as well as make the code. I had a working version by around noon. My grandson is going to love this, I am sure!

2019-09-12

Environmental sensor

For my latest little R&D project I have made a new environmental sensor. These should help us see how good the air quality is in the office, and as usual, I have published this as an open source project for anyone else to make the same.

This proved to be a bit of a challenge, as always. I have already managed to get my head around the ESP32 and the Espressif ESP-IDF now, and have a set of  MQTT based tools as a base (here).

Hardware

I used an ESP32-WROOM-32 again, as they are really nice. I also used an SCD30 CO₂ sensor, which is the most expensive bit at around £40 from RS. I got a nice 128x128 OLED display from Amazon, and a DS18B20 temperature sensor.

One challenge was space - I decided to make the the PCB the same size as the OLED PCB and sandwich the CO₂ sensor in between. This was definitely a challenge to get connectors to fit!

I also decided to try the Molex SPOX connectors as they seem relatively cheap and easy to use.

The other aspect which is especially challenging is power. Most of the bits I have done to date have been alarm and access control and so use 12V on screw terminals. As it happens the unit (pictured above) takes 12V which is in the wall for the door control. But for use in bedrooms I need power to these and so I went for a micro USB. This is simply because that is a really easy and readily available and cheap way to provide a device with power.

The challenge itself is that the connectors have tiny tiny 0.635mm spaced contacts, and are also mechanically crap and come off the board. I ended up getting some connectors that I could superglue to the PCB, and thankfully the power contract are the end, so even though milling and soldering at 0.635mm pitch is just about possible (surprisingly) I can avoid this by actually only soldering the end pins for power. I decided to make the 3D printed case design fit tightly around the connector as well for extra mechanical support, except that means removing from the case pulls the connector off the PCB!

In short, some interesting new hardware challenges.

Software

There are a lot of libraries around for ESP32 and ESP-IDF already, which is nice - but perhaps not as many as for the ESP8266/Arduino based environment.

That said, these are simple enough devices, so actually I ended up doing the I₂C interface for the CO₂ monitor myself using the ESP-IDF directly.

I used a library for the DS18B20 though, and it seems they have been quite clever using the IR remote control hardware to make the one wire bus work using DMA. Impressive. I decided to use a DS18B20 as (a) it allows the thermometer to be positioned where you want, (b) it is away from the components that could get slightly warm, and (c) it seems way more accurate than the one built in to the SCD30 CO₂ sensor. However, the code is happy to use the SCD30 if you don't want to use a DS18B20. Similarly, the code is happy to work without a display, but I am quite pleased with the display, to be honest.

For the display, there is a good Adafruit GFX library, but I ended up doing it myself. The commands for the SSD1327 OLED controller are simple enough, and only a few lines of code to send a whole frame buffer over I₂C. As such, I decided to make some 16 grey level renders of my existing font designs. There is also a logo (configurable).

I found the CO₂ reading was liable to change quite quickly. This is great, except just standing looking at it caused it to change a lot due to your breath. So I ended up damping the reading (biased average with last reading) to remove spikes.

I also decided to round the values for reporting on MQTT so that it is not flooding with updates every second. Configurable rounding, preset to CO₂ at multiples of 10ppm, RH at whole percent, and Temp at 0.1℃. I also added some hysteresis to the reporting.

Obviously I have my air-con control code working with it - just had to tell it a different topic to watch for on MQTT, but I added logging of CO₂ and R/H as well just for fun.

And finally I added settings so it can send MQTT to turn a fan on or off based on CO₂ levels.

It is interesting to see how CO₂ changes during the day.

Overall I am very pleased with it. It is all on GitHub (here) with PCB milling, 3D printed case design, and code. Have fun.

P.S. The OLED displays are made of glass, and very very very easy to accidentally break. Especially if the 3D case design is a fraction out!


P.P.S. I have one by my bed, and it is quite bright, so I now have an MQTT contrast command tied in to the sleep tracking stuff so when I get in to bed it goes dim :-)


2019-09-04

Front door locks, at last

Even though I have had electronic door locks for something like 30 years, since well before I was married, my wife, for some reason, wanted a "normal" front door for the house. First units were made from a cassette player head (Sony Walkman, I think) and read mag cards (bank cards).

This is crazy (IMHO, but obviously I am "wrong"). For a long time it was a typical double glazing unit, but she then got a custom made wooden door. Nice!

Even though custom made, she was not keen on me insisting on electronic locks. I am not sure why.

Anyway, I did manage to get good euro profile actual locks, hard to pick, etc, and you need a 3D printer to make a fake key! Yeh, I know!!

The problem is we got some keys, and could get more, but with 5 kids and their various family you run out, especially if any are staying with you or visiting enough to have a key.

The end result was rather annoying for me. In practice the house is never empty, but having a key in a box outside the door is not a good security measure! Not quite a key under the mat, but damn close!

So finally I got a locking spindle.

I fitted it and routed wires in the mechanism to the hinge, etc.

The mechanical lock still works, obviously.

Unless deadlocked (as I say, never nobody here), the door can be opened by using the handle inside. Yes, that too is multi-point locking, but you just turn the handle.

Now the outside handle also works, if 12V is applied to the locking spindle. Suitable amounts of super glue, etc, make it hard to dismantle and operate otherwise. The door supplier has just used a split spindle which could easily be opened with a screw driver and replaced - crazy!

But this little device was simple, and around £60. There are options for a completely new multi-point locking strip with motors in it, but they are a lot more work and cost. This was surprisingly simple.

I already had a card reader on the wall near the front door, now an NFC reader using an ESP32 (inside) to check cards. That simply set the alarm, previously, but now can be used to "engage" the locking spindle.

End result is that I can now easily allow some access at some times of day, using DESFire cards, or even just bank cards, if and when I like, and no keys in a box outside. We have someone staying with us for a few days, and they were suitably gobsmacked that I just enabled their bank card to work the front door for a few days!

I am surprised more locks are not like this to be honest. The tech is way better now, but still, it has always been so much better than simple mechanical keys.

PCB milling

The new CNC machine is working well, so I thought it worth explaining a bit more on the PCB milling. I am getting pretty good at it now.

Drilling

I 3D printed a support for the PCBs, holding them a couple of mm off the bed. This allows me to drill through the PCB and mill around the outline. Designs on thingiverse.

Milling

I previously mentioned milling bits, and there are lots you can buy from Amazon and the like. However, the best one I have found so far is one from RS components (part 382002). It is not cheap (£40), but don't be fooled - it works, and works, and works. I did manage to finally blunt one after a lot of PCBs, but really it is very good. The small end means it does not snap off easily either.

I am using a 0.5mm drill to make holes, and a 1mm drill to mill around the edges.

Design rules

I am now milling for the finer pitch used by the ESP32-WROOM-32 SoC modules. So I have to be able to mail quite fine tracks with 1.27mm spacing.

What I have found is that, with that bit, the simplest rule is to ensure isolation tracks are at least 1mm apart. This is quite easy, using inkscape, I just set the tracks to 1mm wide and see if any over lap. I have a layer for the parts, a layer for the drilled holes, a layer for the edge mill, and a layer for the isolation tracks. I can then simply save each layer as EPS and convert to an NC G code file.

Faster milling

I also updated my code to take the EPS output from inkscape and make a mill file. It now sorts the lines, and makes paths with a much more optimal routing. On GitHub.

2019-09-01

Pololu D24V5F3

These are really nice power supply modules from Pololu..

They run from anything up to 36V and regulate down to a nice 3.3V. They cope if I reverse the polarity. They are tiny (0.4″ × 0.5″). They just work.

They cost a few dollars, and actually cost more than the ESP32-WROOM-32 (at current pre-brexit style exchange rates), but they really just work, and I am using them everywhere.

What I did not work out, when ordering a 100 of them, is that they are so recyclable! I am changing controllers from ESP8266 to ESP32, but the power modules can be easily desoldered and re-used. I have so many I am thinking of listing them on Amazon.

I whole heartedly recommend them, very nice, work well, don't get fried when you cock up.

2019-08-31

ESP32 Interrupts

Well, it has gone well... A week in Greece and a week back home, and I have my alarm system / access control code working.

In the process I have released an ESP32 / ESP-IDF based PN532 library which works with my DESFireAES library, and also a VL53L0X library.

The only bit that has proved problematic is the 9600 Baud RS485 Galaxy Bus library. It worked well on the ESP8266 using around 28800Hz timer interrupt to bit bash RS485. I coded the same on the ESP32, with much faster processor, and, well, two processors. It is being "iffy". It is close enough for now, but I need to fix.

I could go for using one of the three UARTs which includes some RS485 stuff, but that too needs some interrupts to get timing right, I expect. Getting the UART interrupts working was even more challenging than a simple timer interrupt.

I am working on updating my door controllers now. The new design is nice, and smaller than the old design.


So I'll read up more on interrupts :-)

2019-08-21

ESP32 progress

I am, of course, reinventing wheels, yet again with my move to ESP32 for IoT and access control stuff.

It is going well - I am actually on holiday in Greece, but whilst my wife and daughter sit on sun beds, I have been messing with code. Yes, I took a laptop, programming cable, some ESP32 boards, and even a hand held oscilloscope on holiday with me. Sorry. I spent 4 hours on the plane reading the data sheet on my iPad!

My plan is to get my ESP32 development environment and toolkit working like I had on the ESP8266, and build up my "Solar System" alarm and door control module code.

In some ways this is a nice project - it is often good to rework and redesign something for the second time as you have learned lessons and can do things better. It is also nice that my code effectively has a specification to work to. The previous code was being specified, designed, and coded all at once. But now I have a system which works with the ESP8266 modules and should work just the same with the ESP32 modules.

Progress

I am making good progress. I have the build environment on Mac and linux working. The instructions were very clear. I am actually ssh'd to my linux box to develop now. I have the basic support system which allows me to connect to the WiFi, and MQTT server, and do over the air s/w updates, and store settings in non volatile storage. These are all based on the ESP-IDF tools, which seem quite good.

I have hit a few snags - for example I cannot do certificate pinning on the TLS using the SHA1 hash of the certificate, it seems. I have to include the PEM format certificate itself. This seems needlessly complicated, but not a big change.

I have a snag with MQTT that it seems not to pass on messages with zero length data for some reason. I may have to work that out and do a pull request to the library.

I also have to restructure the design in some case - for example, previously I did the settings in a block of memory that was flashed. Now I am using the provided NVS library for settings. It seems good as it does not have to re-flash a block on every change - it incrementally updates a log of settings in flash as it goes. Quite sensible really. However, it seems to lack a means to enumerate the settings that are there - I have to ask for a setting by name. As a result I have changed my code structure to register settings from the application with my library. This has meant a redesign, but as always it makes it much better in the long run.

Update: As redesigns go - I ended up re-doing it yet again, and pretty slick now.

I should be able to sort the basic GPIO (inputs and outputs) really easily, and hence use almost all of my existing door control code with that. It looks like I can compile my linux version of DESFire stuff for NFC too, so no need for an ESP specific version (if needs ESP specific AES calls).

There will be some challenges with the PN532 library and VL53L0X library I expect. But they are not rocket science, thankfully. The Honeywell Galaxy RS485 bus should just work if I can work out the timer interrupt logic. Saldy each of these is simple in theory but getting my head around the specific way of working for the ESP32 takes an unknown amount of time. Interrupts were a challenge on the ESP8266!

PCB design

This had me kicking myself the night before we left on holiday. I have made up several ESP32 based PCBs. A general break out board; a door controller board; and several Galaxy keypad boards. I had checked I could load code on the door controller. It worked.

But I realised, on reading more of the data sheet, that 6 of the pins on the bottom of the module (ESP32-WROOM-32) are connected to the internal flash and so unusable. Why even have them come out of the module then? They are literally in the way of tracking the usable pins out of the chip. I had the same issue on the ESP-12F and taped over the pads. The ESP-12S sensibly does not have them, but then has a GND pad you have to tape over if you have tracks under it. The ESP32-WROOM-32 has both unusable pins on the end and a GND pad in the way. Arrrg! Even worse, it is only 6 pins of 10 at the end that you cannot use, so not actually easy to tape over just the 6 and still sensibly solder to others. I had read a web site on "safe" GPIO pins, but that documented the ESP32 chip not the ESP32-WROOM-32 module, and for that the pins 6-11 are "safe" to use as GPIO. There are other pins that are iffy - i.e. needs to be high or low at boot. Those I knew about.

The end result is that the keypad boards I made would simply not work! So I was left with three boards in total to take on holiday. The door controller would not "work" for the GPIO pins, but they are not connected to anything that would stop the chip actually working. So usable for developing code. I have managed to trash one board already with a flash / secure boot mix up!

The first thing I did when I got here was re-work my designs on the basis of taping all 10 connectors on the end of the chip. If I had not, it would have been bugging me all holiday. I had to be a bit creative in some cases, but I managed to redesign both boards. I'll mill them when I get back (I don't think my wife would have let me bring the milling machine on holiday for some reason).

Door controller PCB

Update: The RS485 is a challenge. I made a low level bit bashed solution for ESP8266, which I could use. It uses two pins, a Tx/Rx and a DE/RE pin. But the UARTs in the ESP32 have hardware support for RS485 as do the drivers, and are better done with separate Tx, Rx, and DE (RE permanently enabled). So I may re-track the PCB and make a driver using the UART. Again, a redesign of both PCB and code. Overall this is dragging on longer than expected.

Security

Security with any IoT stuff is important, and reading the data sheets on the ESP32 and using the build environment, I can see that Espressif have done their homework. Nothing is 100% secure, but they are really trying hard here - which is good news as a lot of IoT stuff is totally crap!

The chip has a secure boot mode. This means the first stage bootloader in ROM checks the signature on the bootloader in flash before running it. The second stage bootloader checks the signature on the application. Doing an OTA (over the air) s/w update also checks the signature, even. This means you cannot load wrong code on the chip!

It also supports encrypted flash. This means that I can have the code, and all my settings (like login details to the MQTT server, and WiFi) securely stored in the flash. Even pulling it off the board won't get you the data.

It has a set of eFuses, which are built in to the chip. They can only be "blown", so are bits you cannot unset once set. These include bits that stop you reading the eFuses, but they are wired internally in the chip hardware allowing it to encrypt and decrypt the flash. You can even set a fuse to disable the JTAG debug port. This means you can really lock it down so that you cannot get credentials out of the chip or the flash. It might not stop someone de-capping the chip package and using a microscope to read the fuses, but it will stop most conceivable attacks.

They have built in AES, SHA, BIGNUM (RSA) and TRNG hardware as well, and the API makes using https and TLS simple and encouraged.

In short, security seems to have been done very well, and thought about at the outset in the hardware and software design. This is almost unheard of in the IoT world, which is generally pretty shit at security. Well done Espressif.

Update: I said nothing is 100% and a recent tweet suggests hackers have managed to crack it using clock or power glitching, but Espressif have replied saying they are adding hardware changes in the next release out soon! So they do seem to be trying, which is way better than most IoT stuff.

Opportunities

Once I have this starting point - code and PCBs - and it is working, then (apart from updating my existing door controllers), I have to think of other uses for these chips. They are packed with stuff - included a dedicated separate ultra low power processor. The opportunities for very low power devices are interesting. Even just the bluetooth looks fun - I could make it that door entry using a key fob needs to be able to see my phone in range, for example. I am sure there are a lot of opportunities!

They have some interesting direct AP-less WiFi modes in the API as well, and some low rate long range (e.g. 1km) WiFi functions too. There is even a MESH WiFi system. This all makes for some very interesting possibilities to make systems that do not rely on infrastructure such as access points.

I still have not made my environmental monitoring device yet, so that may well be the next project after this.

Missing unix/linux/posix file open option

What I would like is a file open option for "create replacement file". The idea is that this makes a new inode in the same mount p...