Brexit countdown clock

Having made the environmental sensor, I have now cut down the design to just the processor and OLED display so I can make other things.

One example is a Brexit clock. I have put the code and PCB and 3D case on GitHub (here). It may need regular code updates, obviously :-)

It did raise one issue though. Most of the Brexit clocks I see, like Sky News's (here) seem to be an hour out. Brexit is at 11pm but they show days plus time that would be midnight.

Now, I know why... Someone is being lazy and simply dividing time of Brexit minus time now by 86400 and ignoring the clock change. One of the days between now and Brexit is 25 hours long, but that does not stop it being "one day".

My clock gets it right, obviously.

There are other uses for the app - and as I am taking one of my grandsons on a his first cruise, I thought he may like this...

He heard the phones in the suite have a button labeled "pizza" :-)
He is already packing!

P.S. This is why I have a milling machine and a 3D printer! I came up with the idea over coffee this morning at around 9am. I had to design a new PCB and a new case, mill the PCB, 3D print the case, and assemble it all, as well as make the code. I had a working version by around noon. My grandson is going to love this, I am sure!

Environmental sensor

For my latest little R&D project I have made a new environmental sensor. These should help us see how good the air quality is in the office, and as usual, I have published this as an open source project for anyone else to make the same.

This proved to be a bit of a challenge, as always. I have already managed to get my head around the ESP32 and the Espressif ESP-IDF now, and have a set of  MQTT based tools as a base (here).

Hardware

I used an ESP32-WROOM-32 again, as they are really nice. I also used an SCD30 CO₂ sensor, which is the most expensive bit at around £40 from RS. I got a nice 128x128 OLED display from Amazon, and a DS18B20 temperature sensor.

One challenge was space - I decided to make the the PCB the same size as the OLED PCB and sandwich the CO₂ sensor in between. This was definitely a challenge to get connectors to fit!

I also decided to try the Molex SPOX connectors as they seem relatively cheap and easy to use.

The other aspect which is especially challenging is power. Most of the bits I have done to date have been alarm and access control and so use 12V on screw terminals. As it happens the unit (pictured above) takes 12V which is in the wall for the door control. But for use in bedrooms I need power to these and so I went for a micro USB. This is simply because that is a really easy and readily available and cheap way to provide a device with power.

The challenge itself is that the connectors have tiny tiny 0.635mm spaced contacts, and are also mechanically crap and come off the board. I ended up getting some connectors that I could superglue to the PCB, and thankfully the power contract are the end, so even though milling and soldering at 0.635mm pitch is just about possible (surprisingly) I can avoid this by actually only soldering the end pins for power. I decided to make the 3D printed case design fit tightly around the connector as well for extra mechanical support, except that means removing from the case pulls the connector off the PCB!

In short, some interesting new hardware challenges.

Software

There are a lot of libraries around for ESP32 and ESP-IDF already, which is nice - but perhaps not as many as for the ESP8266/Arduino based environment.

That said, these are simple enough devices, so actually I ended up doing the I₂C interface for the CO₂ monitor myself using the ESP-IDF directly.

I used a library for the DS18B20 though, and it seems they have been quite clever using the IR remote control hardware to make the one wire bus work using DMA. Impressive. I decided to use a DS18B20 as (a) it allows the thermometer to be positioned where you want, (b) it is away from the components that could get slightly warm, and (c) it seems way more accurate than the one built in to the SCD30 CO₂ sensor. However, the code is happy to use the SCD30 if you don't want to use a DS18B20. Similarly, the code is happy to work without a display, but I am quite pleased with the display, to be honest.

For the display, there is a good Adafruit GFX library, but I ended up doing it myself. The commands for the SSD1327 OLED controller are simple enough, and only a few lines of code to send a whole frame buffer over I₂C. As such, I decided to make some 16 grey level renders of my existing font designs. There is also a logo (configurable).

I found the CO₂ reading was liable to change quite quickly. This is great, except just standing looking at it caused it to change a lot due to your breath. So I ended up damping the reading (biased average with last reading) to remove spikes.

I also decided to round the values for reporting on MQTT so that it is not flooding with updates every second. Configurable rounding, preset to CO₂ at multiples of 10ppm, RH at whole percent, and Temp at 0.1℃. I also added some hysteresis to the reporting.

Obviously I have my air-con control code working with it - just had to tell it a different topic to watch for on MQTT, but I added logging of CO₂ and R/H as well just for fun.

And finally I added settings so it can send MQTT to turn a fan on or off based on CO₂ levels.

It is interesting to see how CO₂ changes during the day.

Overall I am very pleased with it. It is all on GitHub (here) with PCB milling, 3D printed case design, and code. Have fun.

P.S. The OLED displays are made of glass, and very very very easy to accidentally break. Especially if the 3D case design is a fraction out!

P.P.S. I have one by my bed, and it is quite bright, so I now have an MQTT contrast command tied in to the sleep tracking stuff so when I get in to bed it goes dim :-)

Front door locks, at last

Even though I have had electronic door locks for something like 30 years, since well before I was married, my wife, for some reason, wanted a "normal" front door for the house. First units were made from a cassette player head (Sony Walkman, I think) and read mag cards (bank cards).

This is crazy (IMHO, but obviously I am "wrong"). For a long time it was a typical double glazing unit, but she then got a custom made wooden door. Nice!

Even though custom made, she was not keen on me insisting on electronic locks. I am not sure why.

Anyway, I did manage to get good euro profile actual locks, hard to pick, etc, and you need a 3D printer to make a fake key! Yeh, I know!!

The problem is we got some keys, and could get more, but with 5 kids and their various family you run out, especially if any are staying with you or visiting enough to have a key.

The end result was rather annoying for me. In practice the house is never empty, but having a key in a box outside the door is not a good security measure! Not quite a key under the mat, but damn close!

So finally I got a locking spindle.

I fitted it and routed wires in the mechanism to the hinge, etc.

The mechanical lock still works, obviously.

Unless deadlocked (as I say, never nobody here), the door can be opened by using the handle inside. Yes, that too is multi-point locking, but you just turn the handle.

Now the outside handle also works, if 12V is applied to the locking spindle. Suitable amounts of super glue, etc, make it hard to dismantle and operate otherwise. The door supplier has just used a split spindle which could easily be opened with a screw driver and replaced - crazy!

But this little device was simple, and around £60. There are options for a completely new multi-point locking strip with motors in it, but they are a lot more work and cost. This was surprisingly simple.

I already had a card reader on the wall near the front door, now an NFC reader using an ESP32 (inside) to check cards. That simply set the alarm, previously, but now can be used to "engage" the locking spindle.

End result is that I can now easily allow some access at some times of day, using DESFire cards, or even just bank cards, if and when I like, and no keys in a box outside. We have someone staying with us for a few days, and they were suitably gobsmacked that I just enabled their bank card to work the front door for a few days!

I am surprised more locks are not like this to be honest. The tech is way better now, but still, it has always been so much better than simple mechanical keys.

PCB milling

The new CNC machine is working well, so I thought it worth explaining a bit more on the PCB milling. I am getting pretty good at it now.

Drilling

I 3D printed a support for the PCBs, holding them a couple of mm off the bed. This allows me to drill through the PCB and mill around the outline. Designs on thingiverse.

Milling

I previously mentioned milling bits, and there are lots you can buy from Amazon and the like. However, the best one I have found so far is one from RS components (part 382002). It is not cheap (£40), but don't be fooled - it works, and works, and works. I did manage to finally blunt one after a lot of PCBs, but really it is very good. The small end means it does not snap off easily either.

I am using a 0.5mm drill to make holes, and a 1mm drill to mill around the edges.

Design rules

I am now milling for the finer pitch used by the ESP32-WROOM-32 SoC modules. So I have to be able to mail quite fine tracks with 1.27mm spacing.

What I have found is that, with that bit, the simplest rule is to ensure isolation tracks are at least 1mm apart. This is quite easy, using inkscape, I just set the tracks to 1mm wide and see if any over lap. I have a layer for the parts, a layer for the drilled holes, a layer for the edge mill, and a layer for the isolation tracks. I can then simply save each layer as EPS and convert to an NC G code file.

Faster milling

I also updated my code to take the EPS output from inkscape and make a mill file. It now sorts the lines, and makes paths with a much more optimal routing. On GitHub.

Pololu D24V5F3

These are really nice power supply modules from Pololu..

They run from anything up to 36V and regulate down to a nice 3.3V. They cope if I reverse the polarity. They are tiny (0.4″ × 0.5″). They just work.

They cost a few dollars, and actually cost more than the ESP32-WROOM-32 (at current pre-brexit style exchange rates), but they really just work, and I am using them everywhere.

What I did not work out, when ordering a 100 of them, is that they are so recyclable! I am changing controllers from ESP8266 to ESP32, but the power modules can be easily desoldered and re-used. I have so many I am thinking of listing them on Amazon.

I whole heartedly recommend them, very nice, work well, don't get fried when you cock up.

ESP32 Interrupts

Well, it has gone well... A week in Greece and a week back home, and I have my alarm system / access control code working.

In the process I have released an ESP32 / ESP-IDF based PN532 library which works with my DESFireAES library, and also a VL53L0X library.

The only bit that has proved problematic is the 9600 Baud RS485 Galaxy Bus library. It worked well on the ESP8266 using around 28800Hz timer interrupt to bit bash RS485. I coded the same on the ESP32, with much faster processor, and, well, two processors. It is being "iffy". It is close enough for now, but I need to fix.

I could go for using one of the three UARTs which includes some RS485 stuff, but that too needs some interrupts to get timing right, I expect. Getting the UART interrupts working was even more challenging than a simple timer interrupt.

I am working on updating my door controllers now. The new design is nice, and smaller than the old design.

So I'll read up more on interrupts :-)

ESP32 progress

I am, of course, reinventing wheels, yet again with my move to ESP32 for IoT and access control stuff.

It is going well - I am actually on holiday in Greece, but whilst my wife and daughter sit on sun beds, I have been messing with code. Yes, I took a laptop, programming cable, some ESP32 boards, and even a hand held oscilloscope on holiday with me. Sorry. I spent 4 hours on the plane reading the data sheet on my iPad!

My plan is to get my ESP32 development environment and toolkit working like I had on the ESP8266, and build up my "Solar System" alarm and door control module code.

In some ways this is a nice project - it is often good to rework and redesign something for the second time as you have learned lessons and can do things better. It is also nice that my code effectively has a specification to work to. The previous code was being specified, designed, and coded all at once. But now I have a system which works with the ESP8266 modules and should work just the same with the ESP32 modules.

Progress

I am making good progress. I have the build environment on Mac and linux working. The instructions were very clear. I am actually ssh'd to my linux box to develop now. I have the basic support system which allows me to connect to the WiFi, and MQTT server, and do over the air s/w updates, and store settings in non volatile storage. These are all based on the ESP-IDF tools, which seem quite good.

I have hit a few snags - for example I cannot do certificate pinning on the TLS using the SHA1 hash of the certificate, it seems. I have to include the PEM format certificate itself. This seems needlessly complicated, but not a big change.

I have a snag with MQTT that it seems not to pass on messages with zero length data for some reason. I may have to work that out and do a pull request to the library.

I also have to restructure the design in some case - for example, previously I did the settings in a block of memory that was flashed. Now I am using the provided NVS library for settings. It seems good as it does not have to re-flash a block on every change - it incrementally updates a log of settings in flash as it goes. Quite sensible really. However, it seems to lack a means to enumerate the settings that are there - I have to ask for a setting by name. As a result I have changed my code structure to register settings from the application with my library. This has meant a redesign, but as always it makes it much better in the long run.

Update: As redesigns go - I ended up re-doing it yet again, and pretty slick now.

I should be able to sort the basic GPIO (inputs and outputs) really easily, and hence use almost all of my existing door control code with that. It looks like I can compile my linux version of DESFire stuff for NFC too, so no need for an ESP specific version (if needs ESP specific AES calls).

There will be some challenges with the PN532 library and VL53L0X library I expect. But they are not rocket science, thankfully. The Honeywell Galaxy RS485 bus should just work if I can work out the timer interrupt logic. Saldy each of these is simple in theory but getting my head around the specific way of working for the ESP32 takes an unknown amount of time. Interrupts were a challenge on the ESP8266!

PCB design

This had me kicking myself the night before we left on holiday. I have made up several ESP32 based PCBs. A general break out board; a door controller board; and several Galaxy keypad boards. I had checked I could load code on the door controller. It worked.

But I realised, on reading more of the data sheet, that 6 of the pins on the bottom of the module (ESP32-WROOM-32) are connected to the internal flash and so unusable. Why even have them come out of the module then? They are literally in the way of tracking the usable pins out of the chip. I had the same issue on the ESP-12F and taped over the pads. The ESP-12S sensibly does not have them, but then has a GND pad you have to tape over if you have tracks under it. The ESP32-WROOM-32 has both unusable pins on the end and a GND pad in the way. Arrrg! Even worse, it is only 6 pins of 10 at the end that you cannot use, so not actually easy to tape over just the 6 and still sensibly solder to others. I had read a web site on "safe" GPIO pins, but that documented the ESP32 chip not the ESP32-WROOM-32 module, and for that the pins 6-11 are "safe" to use as GPIO. There are other pins that are iffy - i.e. needs to be high or low at boot. Those I knew about.

The end result is that the keypad boards I made would simply not work! So I was left with three boards in total to take on holiday. The door controller would not "work" for the GPIO pins, but they are not connected to anything that would stop the chip actually working. So usable for developing code. I have managed to trash one board already with a flash / secure boot mix up!

The first thing I did when I got here was re-work my designs on the basis of taping all 10 connectors on the end of the chip. If I had not, it would have been bugging me all holiday. I had to be a bit creative in some cases, but I managed to redesign both boards. I'll mill them when I get back (I don't think my wife would have let me bring the milling machine on holiday for some reason).

Door controller PCB

Update: The RS485 is a challenge. I made a low level bit bashed solution for ESP8266, which I could use. It uses two pins, a Tx/Rx and a DE/RE pin. But the UARTs in the ESP32 have hardware support for RS485 as do the drivers, and are better done with separate Tx, Rx, and DE (RE permanently enabled). So I may re-track the PCB and make a driver using the UART. Again, a redesign of both PCB and code. Overall this is dragging on longer than expected.

Security

Security with any IoT stuff is important, and reading the data sheets on the ESP32 and using the build environment, I can see that Espressif have done their homework. Nothing is 100% secure, but they are really trying hard here - which is good news as a lot of IoT stuff is totally crap!

The chip has a secure boot mode. This means the first stage bootloader in ROM checks the signature on the bootloader in flash before running it. The second stage bootloader checks the signature on the application. Doing an OTA (over the air) s/w update also checks the signature, even. This means you cannot load wrong code on the chip!

It also supports encrypted flash. This means that I can have the code, and all my settings (like login details to the MQTT server, and WiFi) securely stored in the flash. Even pulling it off the board won't get you the data.

It has a set of eFuses, which are built in to the chip. They can only be "blown", so are bits you cannot unset once set. These include bits that stop you reading the eFuses, but they are wired internally in the chip hardware allowing it to encrypt and decrypt the flash. You can even set a fuse to disable the JTAG debug port. This means you can really lock it down so that you cannot get credentials out of the chip or the flash. It might not stop someone de-capping the chip package and using a microscope to read the fuses, but it will stop most conceivable attacks.

They have built in AES, SHA, BIGNUM (RSA) and TRNG hardware as well, and the API makes using https and TLS simple and encouraged.

In short, security seems to have been done very well, and thought about at the outset in the hardware and software design. This is almost unheard of in the IoT world, which is generally pretty shit at security. Well done Espressif.

Update: I said nothing is 100% and a recent tweet suggests hackers have managed to crack it using clock or power glitching, but Espressif have replied saying they are adding hardware changes in the next release out soon! So they do seem to be trying, which is way better than most IoT stuff.

Opportunities

Once I have this starting point - code and PCBs - and it is working, then (apart from updating my existing door controllers), I have to think of other uses for these chips. They are packed with stuff - included a dedicated separate ultra low power processor. The opportunities for very low power devices are interesting. Even just the bluetooth looks fun - I could make it that door entry using a key fob needs to be able to see my phone in range, for example. I am sure there are a lot of opportunities!

They have some interesting direct AP-less WiFi modes in the API as well, and some low rate long range (e.g. 1km) WiFi functions too. There is even a MESH WiFi system. This all makes for some very interesting possibilities to make systems that do not rely on infrastructure such as access points.

I still have not made my environmental monitoring device yet, so that may well be the next project after this.

ESP-01, ESP-12F, ESP-12S, ESP32

As I mentioned, I am trying to move my IoT development stuff to ESP32, and notably move from Arduino IDE to the native Espressif ESP32-IDF using mostly C.

This poses a slight problem. The development environments are sufficiently different that it is something of a pain in the arse to maintain both. Probably just about possible, but not simple. So I'd like to basically abandon the ESP8266 / Arduino stuff (I'll leave on GitHub) and start working only on the ESP32 based stuff.

This poses a slight snag as one of the use cases for this IoT stuff is a very small device placed inside a Honywell Galaxy alarm system keypad.

This is based on an ESP-01 module which is ESP8266 based. It is not ideal as not CE marked, no RF can, very limited IO pins, but it is small. It comes with a 2x4 0.1" pitch header and is just the right size to fit a Pololu 3V3 regulator between it and the PCB, which is rather neat.

This use case has the ESP-01, a 3V3 regulator, an RS485 SO8 driver, and a small 2x2 milli-grid header for programming and debugging.

The whole thing fits in the small space inside the keypad, as per the picture on the right.

As a result I assumed I'd be stuck with, at the very least, my keypad code on the ESP8266 / Arduino based code.

The other IoT devices are generally based on either ESP-12F, or preferably ESP-12S modules. These are only slightly narrower than the ESP32-WROOM-32 modules I am now using, and so easy enough to make alternative designs using the ESP32.

The ESP32, however, whilst larger, has a lot of IO pins. It is possible to track these pins so that you have quite a few usable IO pins, and GND and 3V3, all on one end of the chip if you try :-

I have actually tracked Tx/Rx programming pins, and GND and 3V3 "through" some unused GPIO pins, which allows its all to come out of the bottom end and keep the whole thing quite narrow. Obviously this is fine as long as I don't make those pins an output :-)

Once again, the new milling machine is doing a great job, and I was able to solder it.

And, to my delight, it fits, and the keyboard PCB fits over it with no problems!

So, yes, I can actually move away from ESP8266 now. I have to migrate and re-work the various tools and modules I have made, but once done, I can continue development on ESP32 only. Yay!

P.S. Always tweaking :-)

Moving to ESP32

Someone already told me that once I move to ESP32 I won't want to go back to ESP8266.

Well, apart from the fact the ESP8266 is available in a tiny ESP-01 package, I think they may be right. The ESP32 in a WROOM-32 package is slightly wider than the ESP12 package, but because the GPIO pins can be mapped any way you want, pretty much, the tracking takes less space. So overall I was able to make an equivalent board in less space using ESP32.

The big challenge for me with ESP32 was the concern over the fine pitch - 0.9mm contacts at 1.27mm spacing. I was not sure I could either mill a PCB or solder a board at that pitch reliably.

I was wrong, I can!

New door controller using ESP32

Fixing the X axis backlash on my mill was a big help as well. So yes, I can use the ESP32 modules.

The other concern was the price, but it seems that getting a proper ESP32 is as cheap, if not cheaper, than an ESP12S. Yes, some cheap ESP12S can be got, without CE marks, and missing FCC ID, and not quite working well on WiFi. But £2.91 for ESP32-WROOM-32 from a reputable supplier is pretty good.

There are advantages to the ESP32, obviously, including much more RAM, faster, dual processor, more GPIO, more UARTs, and so on. It is a good comprehensive IoT device.

The other big change I want to make is moving away from Arduino IDE. Yes, Arduino makes things easy, but it turns out the Espressif IDF for ESP32 is pretty good. It has the key things like https, MQTT, OTA, and all sorts as standard.

So my plan is to make an ESP32 IDF based version of my door controller, including PN532, VL53L0X, etc. This should not be too hard, and a lot of my existing code can easily be ported to a simple C code environment.

I am, however, impressed with the documentation - not only of the APIs in the IDF, but also the technical reference manual. It actually details all of the registers clearly.

The other impressive thing is Espressif seem to to be trying to encourage some best practice in IoT. They have secure boot and signed images as a standard feature. Even https libraries, as standard. It means that making a reasonably secure IoT device is easy. Well done.

I managed to get the build environment working on Mac and linux with no problems, following the clear instructions. I built the "hello world" example, and flashed it with no problem.

I am off on a 4 hour flight shortly, so I have the 669 page PDF of the technical reference manual on an iPad for some light reading on the flight.

Of course, once I have this off the ground, I can look at Bluetooth, and CAN bus, and all sorts of new fun things.

New milling machine

I had a CNC 3018 PRO machine from Amazon.

• It needed assembling, but I've done that now. It is something to bear in mind if buying one though.
• It is a bit of a "home made" style, with the PCB bolted to the frame, etc.
• The one I got had a bent screw for X axis, which limited the usable space a bit.
• It has a working area 30x18cm.
• I was able connect a Z-axis probe (well, contact to PCB copper clad board).
• It works well, and I used bCNC on my Mac to work it.
• It comes with a controller box, which is not needed really!
• It is surprisingly cheap.

So, especially given the bent screw, I decided to get a replacement, and went for a CNC 3020 from Amazon.

• This is a much more "industrial" unit - proper drag chains and cables. Looks good but costs a bit more.
• Work area (you guessed it) 30x20cm
• Comes pre-built
• Has a boxed controller and power supply, with nice connectors to the machine.

However, it has a challenge or two - the main one is that it works with some very specific software (a pirate copy of which is apparently supplied with it). It has parallel (!) and USB, but the USB only works with that software and only on a windows machine - it does not even appear on a Mac even in a debug log! The software did not look too good anyway! This is a bit of a bugger.

The upgrade

My solution was to upgrade the controller to a TingG controller. This adds a bit to the cost (especially with the customs/VAT and admin charge from US). But was not actually at all hard to do.

The old controller is bolted to a heat sink, and can simply be unscrewed.

The new controller is slightly smaller. I needed to drill/tap an M3 hole in the heat sink for one of the screws so that I could screw it to the heat sink. I also cut an extra hole in the case for the USB connector.

Also, I decided to use the heat sink, which meant something between the new controller board and the heatsink. I got some small copper block / heat sinks and thermal tape - and I used them between the back of the new board and the existing heatsink.

The TinyG is designed to work without the need for a heatsink, but this can not really do any harm.

The connectors on the old board were not the same, though the new board has pads which means I could have changed them. I decided instead to use the screw terminals on the new board, fitting bootlace crimps on the wires. I had to swap the cables to make them fit, and use the motors 2, 3 and 4 as closest to edge of the case. But they all fit!

I also connected the emergency stop to the reset pins. I have not worked out how to connect the spindle drive to the new board yet - that is a challenge for another time. The case has a button and a dial for that which works for now.

I also got a 6 way (plus ground) chassis DIN connector and wired up all 6 end stops (X/Y/Z Min/Max).

This meant various micro switches, and super glue on the actual machine.

And also, the Z-Min I wired to croc clips to allow me to use contact to PCB for homing Z axis.

The TinyG has good documentation, covering setting the current, etc. I was able to set the axis to the right motors, and set the polarity and travel per rotation, etc.

However, the bCNC code I was using did not like the TinyG, and I ended up installing CNC.js which works nicely on my Mac. It does lack the multiple point Z axis levelling of bCNC sadly, but works.

I did have to configure the homing for Z axis a tad as it is designed for a switch from which you back off. For PCB contact you get the exact Z home, and do not need offsets. But this is all well documented.

Slack (backlash)

However, even though it is a really good solid construction and seems to have no play, I found there was an issue with the X axis. It seems to have around 0.1mm slack on it. This means if you move right to a point it is actually around 0.05mm short of it, same if moving left (i.e. is then 0.05mm right of it). Make a row of left and right moves and lines and you see the problem clearly. This test shows it well - centre is left and right moves before each line, but left is all moves from the right (apart from first, bottom left) and right is all moves from the left. Repeatable slack!

Oddly the Y axis is absolutely spot on!

This is not really enough of a problem to cause issues making PCBs, but is annoying so I wanted to fix it.

I checked for anything loose, and also checked the TinyG for any options, but to no avail.

My solution (now updated to the eps2gcode tools) is software to compensate for slack. This seems to work.

Overall I am quite happy with the result.

P.S. Tightening this nut a bit fixed it.



P.P.S. After reading the TinyG docs I am actually running at 4 micro steps not 8, so 1/200th mm spacing, which I think will be more than adequate for anything I am milling. I suspect 1/100th mm would be fine but actually that is a lot more noisy. Now that the backlash is sorted maybe I'll go back to 8.