Reverse engineering

Reverse engineering can be fun - and I usually end up doing it where a supplier does not provide linux based drivers for something.

This is the case with the card printer we use. We have gone through many card printers over the years, including an Evolis pebble, a Zebra, a Matica printer and now a newer Matica printer. Each one lacked any linux drivers. What is also a shame is they did not publish the interface so someone could write some.

This is where regulation could be an interesting option - publishing specs to allow interworking of systems. There is much debate on this, and something for another blog.

However, with some debugging and reverse engineering, I have working drivers for all of the card printers.

Why Matica

The XID9300 and XID8600 are "retransfer" printers - they print on a transfer film, and then transfer to the card. This works really well on SIM cards. The Zebra was too, but did not like cards with gaps or contacts (e.g. around SIM), but the Matica is really good. Retransfer also means proper edge to edge with over the edge bleed on artwork.

Not your typical printer

Usually a printer is write only - you send a print job, it does it, job done. But the card printing is a little specialised. You may want to load a card to a "contact station" (e.g. a SIM) or "contactless (RFID) station", or mag stripe encode or mag stripe read. The way these contact stations work is a pain, but seems pretty industry standard - the printer itself takes a hands-off approach - you tell it to put the card on the station and then talk to something else (over USB) to "encode" the card.

It is also not quite as simple as just sending the print job, even when not encoding a card - you have colour and black, and you want control of each separately (the black is 1 bit per pixel and crisp solid black on top, but the colour is 8 bits per colour). You also have the fact the card has two side, and can even have a UV fluorescent layer. So you need to be able to send "layers" and "sides".

What I wanted was a way to do all of this but over Ethernet from remote systems. We have many systems (like our control pages) from which we need to send card print jobs and encode cards.

I decided to use a TLS over TCP link and send back to back JSON objects each way. It worked well as a system. I also decided that image data to print should be sent as PNG images coded as base64 data URLs in the JSON. This worked surprisingly well, and I have a handle on how the png linux library works now (an annoyingly complex library).

The interface also allows exchange of hex data strings to send to/from a contact or RFID card allowing any "encoding" I wish. This is the key thing that the client does not have to talk to something separate to "encode" cards. It also allows me to hold a job all the way and not have other jobs sneak in whilst trying to encode cards (which happened before).

Ethernet

The previous printer, in this case a Matica XID9300, has Ethernet or USB. I had working drivers from long ago that just did printing, and separately had scripts to talk over USB to the cards. Not ideal.

The new printer is a later model Matica XID8600. This too has Ethernet or USB, but instead of being a setting in a menu it picks one. If USB is connected on power up it is USB. If you want to use Ethernet you must not have USB connected (has an internal USB hub). But if you want to encode cards you have to have USB connected, and if you want to do what we did on the XID9300 (Ethernet print, USB for encoding) you have to unplug USB during power up every time - WTAF?

This was only really apparent after I had written some nice new Ethernet based drivers - a combined system to print over Ethernet and talk to contact stations over USB, all using this TCP TLS JSON connection to the client.

Basics

The basic steps are "use the windows drivers" and "observe what is sent to printer". Wireshark is your friend, and I was able to see the Ethernet traffic sent to the printer. There was a nice obvious message structure.

I was really lucky with this printer though as there is a diagnostics tool that the suppliers sent, one they use for maintenance/repair and testing. It has buttons and check boxes for each operation - load a card, move a card, load image data, print images, transfer image to card, etc, etc. And control of all of the settings.

This allowed me to test each operation separately and each option separately and dump the messages. This is absolute slog from a coding point of view - it is spending days trying each separate command and option and looking at hex dumps to work out what each option and setting does in the underlying message. It was tedious.

Once you have that you make test code to generate the same messages and confirm they work, and then build a driver with all of the functions you need.

USB

Whilst I know a lot about Ethernet, I know bugger all about USB. But to avoid the whole "unplug USB during power up" issue, I decided to give it a go.

I was pleased to see that Wireshark comes to the rescue again, and allows dumping of USB traffic on windows. This allowed me to do exactly the same slog of testing each operation and seeing what is exchanged with the printer.

What did I expect?

I expected the XID8600 to be similar to the XID9300, and I was pleased that they were almost the same (Ethernet drivers), but different ports.

The Ethernet has a simple packet/message system - a header with type and length and a variety of commands and parameters. I expected USB to be an alternative way to talk to the underlying printer system and so essentially to be the same with maybe different headers more suitable to USB.

How it works!

I was shocked!

Not even close, not at all, really, crazy. It is like two separate printers! Yes, the underlying operations you can do are the same but the commands and parameters are not remotely related.

Some examples of the craziness.

• Ethernet has a "transfer" command with an option of where to move the card to after (eject, reject, back to print station, etc), and a flag to say if to flip the card. The flag and "location" code match the card load and card move commands. But on USB there are three separate transfer commands (return, eject, flip) and a flag for "immediate" (no clue what that actually does).
• Ethernet has a way to get and set some "settings". These are one byte values, and you send a list of tagged values and get back a list of tag/value pairs. To change you send tag/value. So a setting has a "tag" which is a byte, no real logic to which setting is what tag, but I went through each and documented. But USB is not the same, no, not close. You get settings by two separate commands getting blocks of binary data (64 bytes and 14 bytes), and various bytes are various settings, but their position is not related in any way to the tag used in Ethernet. It gets worse, to set these you send two blocks (32 bytes and 10 bytes), where the settings are specific positions (but not related to the ones you fetch). But then if you want to set some settings just for the job, there is a single block of 24 bytes that have most settings, again in different positions.
• The mag stripe coding messages are similarly different.
• The film layers have numbers when loading the image data, on the Ethernet they were C, M, Y, K, P, U, but on USB the are K, Y, M, C, U, P, just to be different. When printing the them there is a mask of which to print - for Ethernet it is bit numbers for each layer but always prints C/M/Y together if any set. For USB there is a bit for K, YMC, U, P, instead (so obvious that YMC always together).

(Yes, they persistently use YMCK 👷🤷‍♂️ rather than the more common CMYK, which has led to lots of typos in my code)

Basically, it is like two completely separate specification written independently to meet a list of commands and options. Totally separate logic to how they are coded and the how commands are grouped, and so on.

In addition, the USB used a command structure that seems to be based on USB mass storage, using SCSI commands, or possible mis-using them as Wireshark was not entirely happy with some of them. No idea why they would use that structure, perhaps allowing some standard library to be used from Windows.

The good news

Having coded USB for this I tested on the older XID9300. It works. I was shocked!

Deleting code

This led to a decision to delete and remove all of the Ethernet driver code I wrote. The concept for this project is a Raspberry Pi next to printer acting as a single contact for TCP TLS JSON printing. I have a library for the code to run the client side of this as well. The connection from Pi to printer can be USB with no issue, and indeed is possibly easier than having Ethernet from Pi to printer (needs more config to say IP, etc). So using USB is all that is every required (and marginally faster than Ethernet, oddly).

Deleting code is not something I like doing, but in this case it made the code simpler and made for simpler ongoing maintenance. With both drivers - any changes had to be coded and tested for each way of working.

TODO

There are aspects not in the drivers (yet), e.g. we don't have the JIS (Japanese Mag stripe coding) hardware to test that (though easy to code anyway), and we don't have a laminator attachment. So a few bits are not there.

New features

This means we can update our card printing service...

• 600dpi instead of 300dpi.
• The UV print no longer has at the MAC address overlay (windows has it but I found the bits to turn it off, yay).
• We can get the "ID" of an RFID card and store when printing so the API we offer allows you to tie a printed ID to the card ID later for coding - that way we don't need your keys. We even allow the ID to be part of what you print.
• We plan to offer basic MIFARE classic coding of cards (but really, that is not very secure).
• We plan to offer MIFARE DESFIRE coding, but means giving us keys.

All in all a good few weeks work on this. I was, however, rather pleased with myself (having spent a day with wireshark) making working USB drivers in a (long) day this weekend.

Fun with QR library

My QR code generation library works well, but one of the features was generating a "colour" QR code. No, QR codes are not normally coloured, the idea is to just show the anatomy of a QR code, which parts are which. Wikipedia does a good job too.

As part of generating the code I have to create the data part, and padding, and then generate the error correction code (ECC) part, and apply various format control bits and fixed black/white units to make the image.

I have updated the library so it will make a QR code which shows what is what. It was a tad complicated by the fact that the error correction code is interleaved. This means that blocks of data and ECC are scrambled so that each block is actually spread out over the QR code. This means you can remove a chunk of the code - e.g. tear off a corner, and that is a small part of several separate blocks. You will notice the padding (green) below is spread out because of this interleaving.

Each block of data and ECC allows recreating of the data from a relatively small part of the overall block, so the distinction between data and ECC is not that relevant. But the colour coding shows how much is used for what quite nicely even so.

This has colour coding for :-

• Blue: the actual data for the content
• Green: padding bytes and bits
• Red: the ECC code
• Grey: formatting/control
• Black/White: the fixed pixels in this size QR code

So that is it, just in case you wondered...

(That one above is an NHS COVID-19 QR code).

P.S. I have been having more fun with custom padding bytes. This is not just changing units within the tolerance of the ECC (which can be done, if careful), it is changing padding so that the ECC is still valid.

Of course, if you want to get a bit meta, you can put one type of barcode in the padding of another type. This is a Datamatrix barcode in a QR barcode. A real Frankenstein barcode!

Technically the padding is meant to be a repeating pattern EC/11, and surprisingly this is not just a recommendation in the spec. But obviously nothing checks the padding on read, so allowing pretty padding and hidden data in the padding to work.

The blue M&M test

There is a famous story of the Van Halen concert contract requiring the removal of some M&Ms. It is a test to confirm that the contract had been read properly. Clever idea. More on that at snopes.

However, I first encountered it from a friend who referred to it as "The blue M&M test" and explained the story, which I indeed thought was clever.

But there is a problem! As you can see from the Snopes article, the actual story is that they banned brown M&Ms, not blue ones.

So where did this come from? Well, my friend is adamant he always heard it as blue.

Try googling.

Yep, 2 results. Guess what they are? My blog!

Sounds like my friend did not read the contract properly, and so failed the test :-)

Tech: Managing calls

I have just issued a new alpha release of FireBrick, by popular demand, allowing some call filtering based on CLI. Basically you can do a lot more than simple anonymous call reject now. Whilst most people know the FireBrick as a firewall/router it is also a SIP VoIP PABX (phone system).

But one of the things that this highlights is the increasing need to "do something" with calls you receive. Systems like the FireBrick allow you to do all sorts on your own network, making your own "phone system" for VoIP phones. If you look at systems like asterisk the level of controls you have are quite incredible - essentially a programming language for how calls are handled and even allowing recoded messages and DTMF menus and so on.

On a private system you also have means to log things and even manage call recordings yourself.

This is all great for a "desk phone" but what about your mobile?

This is where some of the stuff we sell comes in - and we have customers doing some clever stuff. Recently we have managed to make a few improvements, but basically we have means to have a normal 07 UK mobile number, and a mobile SIM, and put your own phone system (whether FireBrick, or asterisk, or anything else) in the middle.

This means you can have a mobile phone with a SIM card, and do things like log, or filter, or record calls, and texts, either way. The SIP2SIM service looks like a VoIP handset has registered and connected to your phone system, but is in fact a normal Mobile telephone service (i.e. no special app on the phone). This means you can even make internal calls on your phone system from your mobile. (The mobile leg does have call costs even for these).

The texts can be passed by email or using http/https on your own server where you can do things with them. The latest improvements mean much better handling of unicode characters as well. You can also handle the texts from the mobile. You could just join the dots to make texts or calls, in and out, like a normal mobile, or you could do much more with your own scripts on the way.

We have people doing things like opening doors using calls, and clever tricks with texts.

Obviously we also have services that simply link calls, or texts, or both, in and out between an 07 mobile number and the SIP2SIM service without needing your own phone system. We have options like call recording and logging. But we are happy for you to make your own systems, as simple or as complex, as you wish in the middle.

Of course this also allows mobile on a normal landline style number, but texting to such numbers remains a challenge in the UK, and a lot of companies and web sites will refuse to even try texting what they think is a landline number. So using a normal 07 mobile number does the trick nicely.

We can even port in an 07 mobile number to the service, and if you don't like it, port back out again. No minimum term on the SIP2SIM or VoIP services.

So if you are techie, but want a lot more control of your mobile phone service, it is worth taking a look.

One little trick I do a lot is steal a call, transferring it between my mobile and my desk phone mid call, without the other party even realising I have done it. E.g. answer on mobile, walk to desk, put on headset, switch call to desk phone and work on computer while on the call. All seamless.

UK government digitally signed my penis!

It appears that the COVID-19 QR code generator government web site does not allow emojis, but it does allow hieroglyphs, so it seems I have managed to have the UK government digitally sign a penis.

The QR code, signed by the government/NHS, includes

{"id":"53WVKKW5","opn":"𓂸","vt":"005","pc":"SW1A2AA"}

I know it is childish, sorry, and you definitely should not check in to 10 Downing Street by using this barcode, obviously. That would be bad. It does show how daft it is making these QR codes so huge by digitally signing them though.  See the other blog post for more details on these QR codes, and what else is wrong with them.

There is a practical use though - making a QR code for your own home will allow you to "check out" of where you have been, otherwise the app assumes you were there until Midnight.

For those with character set challenges, this is what it looks like

How not to QR (NHS COVID-19 App)

There is now law requiring (from 24th Sep) a QR code to be displayed in various premises so people can scan it in to the NHS COVID-19 App. See The Health Protection (Coronavirus, Collection of Contact Details etc and Related Requirements) Regulations 2020. The law has plenty of issues, but let's look at those QR codes...

It seems you can request the QR code poster for your venue, see here. The poster is emailed to you.

This is an example.

This is not how you do it - and I wonder if they got any technical advice from anyone on the matter first.

What's in this huge QR code?

The content of that QR code is: UKC19TRACING:1:eyJhbGciOiJFUzI1NiIsImtpZCI6IllycWVMVHE4ei1vZkg1bnpsYVNHbllSZkI5YnU5eVBsV1lVXzJiNnFYT1EifQ.eyJpZCI6IlJLWTMyV01SIiwib3BuIjoiUFVCTElDIFRFTEVQSE9ORSIsInZ0IjoiMDA1IiwicGMiOiJMRTE4M1RFIn0.ix66d7uRe_vhpB4BPb0Nzbq2vEC3IShdX7UOqfp0XVyg7YI88R_bOCY1DpgQZo9dy07xcga4e1MTmcKV9ZHi1A

The data contained is an RFC7515 JSON Web Signature (JWS) base64 coded string which contains:-

• {"alg":"ES256","kid":"YrqeLTq8z-ofH5nzlaSGnYRfB9bu9yPlWYU_2b6qXOQ"}
• {"id":"RKY32WMR","opn":"PUBLIC TELEPHONE","vt":"005","pc":"LE183TE"}
• Binary signature

So what's wrong?

• It is not user friendly - and requires the app installed first so you can use it (see below). This is perhaps the biggest issue. P.S. even with app installed, they have not hooked the QR code to the app, as they could have, so could be used from camera app - just lazy!
• This QR code is far too big (i.e. dense)! The denser the code the harder to read reliably. There is no need for it to be this dense. A small code is quicker and easier to read.
• One reason it is dense is poor choice of QR encoding options. It could be less dense with exact same content easily.
• Another reason it is too dense is that the content is base64 coded JSON which itself contains base64 binary. This is crazy. The actual underlying data is quite small, and even signing it, it does not have to be anywhere near as big.
• Another reason it is too dense is they have chosen to sign the data, which is pointless (see below)
• Another reason it is too dense is they have chosen to encode some simple data (venue ID and name) in JSON, when there really is no need.
• You have to use the gov web site to make the QR code, a large company could not, for example, automate making posters for all their sites centrally.
• This is not actually a valid QR code! Yes, pretty much everything will read it, but the specification requires a 4 unit white space all around, and this does not have that - it has grey at 2 units and text within the whitespace area.
• If you request a poster more than once for a venue, you get a different venue code, so the app will see each poster as a separate venue, it seems. I can easily see that happening as it may be easier to request a new poster than to find the PDF / email you previously saved if you need to print more.
• Oh, and the instructions are to display the poster and ask people to scan it with the app, as soon as you get it, even though the app is not actually working yet, so people cannot scan it with the app!
• The poster has no link to where to get the app, just the store you have to search, and guess what, searching does not work (depending on exactly what you type):-

In summary this is thrown together with some standard libraries and very little actual thought - is not even a valid QR code, and is going to be a mess with every waiter now expected to provide tech support on app installation on Android and iPhone to every customer that comes along - but this is very much what we have come to expect.

On another small technical point, base64 is a bad choice in a QR code. If designed for just the app to read, use binary coding which is 100% efficient (one dot per bit, before any ECC). Base64, however, uses byte coding, so 8 bits for each base64 character which holds 6 bits of data, so 75% efficient. If you don't want to use binary, use base32 which uses alphanumeric QR coding, 5½ bits for each character which holds 5 bits of data, so 91% efficient.

How to make it more user friendly

Many people have QR readers built in to their phone, for example an iPhone will pop up with a link from the camera app itself, so there is a really simple trick for this - make the QR code a URL which the app can read as data, but if used simply as a URL itself you end up going to a web site which redirects you to the app or the app store to download the app. The data can be after a # in the URL so not even sent to the server when used as a URL. This allows it to be used from the app or from the camera, and helps for people that don't yet have the app, and those that mistakenly did not realise they have to launch the app first. It makes it a lot simpler to use.

It is not hard, basically, instead of UKC19TRACING:1:blah use https://c19qr.uk/#blah
(well, obviously, an nhs.uk domain would be used)

(Update: Just to clarify, the use of a URL at the start is not to make the QR code usage rely on an internet connection or a web site in any way. If the app is installed it would be used purely as a version/ID confirmation, like the UKC19TRACING:1: string, and the app would then just use the data in the QR code, not visit a web site. The URL is there to make it easy for people to use from camera, and to install the app in the first place).

Why is a "big" QR code a bad idea?

I have added this to clarify a little why the large / dense QR code is not ideal.

For a start, from a purely technical point of view, it is just unnecessary. You need some extra data to avoid confusion with other uses of QR codes, which is what the UKC19TRACING: is for, and ideally a version, which is what the 1: is for. Beyond that you just need the actual data (location code, postcode, and venue name, in this case). There is no need for extra syntax (e.g. JSON). Indeed, some careful choice of data (e.g. using digits, or upper case letters and digits) can make the QR coding even more efficient. But this is far from the only reason.

In ideal conditions it does not matter if you have a large/dense QR code. As someone that has been messing with barcodes for around 40 years, I have no trouble using a camera phone to read a barcode. I know what hoops the phone / software is going through to make it work and how best to position the camera. But I cringe watching people do this and struggle (notably, someone I know reading lottery ticket QR codes). They (understandably) don't know how these things work and will randomly try getting closer or further away, usually the wrong way, not waiting for camera to focus, etc, eventually reading the code.

The way it works is the camera has to be able to see the units, i.e. the black and white squares that make up the code. The camera itself has pixels (dots) with which it can see. If the camera was perfectly aligned and square you could read a QR code with one such pixel per unit, but that never happens, and in practice you need a lot more. Throw in the possibility of poor focus, glare and reflection from glass / perspex, dirty lens, and the QR code at an angle, and you need even more. Thankfully most modern camera phones are high resolution enough to cope and read a very large and dense QR code. Not all phone cameras are made equal and some are much lower resolution and slower to focus. Print quality also matters, and whilst most printers are very good, remember this is also intended to be used displayed on a screen. The QR code itself includes error correction which allows some imperfections and errors to be corrected, but this can only help so much.

Even so, perhaps the biggest issue with a large and dense QR code is the range of distance between the code and the camera. This is a thing I observe people struggling with, for some reason. With a low density QR code the camera can read it when it is small in the view of the camera. Also, when it is small in the view, they do not have to point directly at it, it can be off to one side, etc. With a large / dense QR code the camera needs to be closer, with the QR code filling more of the frame. So the usable distance range where the code can be read relates directly to how dense the QR code is. The more usable range, the easier it is for the user to get it right first time and not hold up a queue of people trying to get in to a bar.

Of course the other issue is how big you print it. The guideline for this is to print at least A4. Why? Because it is so dense. A smaller code could be printed much smaller and still be easy to use. I note, for example, Costa have small table menu cards with the check-in QR codes they used (which are nice and small) on them, and they are much smaller than A4.

One final reason is confusion and paranoia. I already see people on twitter asking what the hell is in this huge QR code. People are concerned that it obviously contains a lot of data and it is not obvious why. The whole project has suffered from privacy concerns already, and this does not help.

Why is signing daft?

Signing means that there is an extra chunk of information in the QR code (making it a lot bigger/denser) that ensures the data is genuine, i.e. that it definitely came from the government QR code generator web page. There are many good reasons to sign things, but not in this case.

The signing a tad daft as :-

• Anyone can make a code for anywhere on the gov web site, and it gets signed.
• You can copy a code from somewhere else and it is signed.
• It makes the QR massive! and so harder to scan.
• Obviously not done to try and avoid vulnerabilities, as one can get 
  سمَـَّوُوُحخ ̷̴̐خ ̷̴̐خ ̷̴̐خ امارتخ signed, no problem (which some may remember would crash iPhones).
• To be quite frank, it would not be as fun making a barcode for Specsavers in Barnard Castle if the QR code was essentially just a postcode and name, and not government signed.
• If, instead of signing, they just published a specification (as done by other countries), large companies with lots of sites could have made posters for all their venues centrally and easily as well as allowing individual posters via the gov website.

Whilst this is not quite the way the track and trace process works, if some establishment did not want to risk being shut down, they could put the barcode for a competitor, maybe with a vague establishment name in the QR code, so not obvious when using the app. The gov website lets you make a signed QR code for anywhere, and even if they did not, you can literally copy a signed QR code you can see anywhere, and it is still signed.

A further update (Oct 20th) shows the signing checking did not even work on Android!

What I couldn't say in this discussion before, and can say now, is that the signature verification was fluffed on the Android app: https://t.co/zmJ55fyPQk

— zofrex (@zofrex) October 21, 2020

And yes, they will sign almost anything. Emojis seem to be banned, but hieroglyphs are not. So I seem to have got the UK Government to digitally sign a penis!

{"id":"53WVKKW5","opn":"𓂸","vt":"005","pc":"SW1A2AA"}

How it could be a lot less dense, so easier to read

As an example, if I just include the actual data, and some sort of signature (an MD5 in this case, there are many ways to sign things), and a URL prefix to get the app (which acts as ID/version), you could make a code like this... Way less dense, and easier to use.

If you don't sign the data (and why would you?)

All that is really needed in the code is the location, a postcode with DPS, e.g. LE183TE9Z, or maybe just a UPRN (Unique Property Reference Number), e.g. 100032050996. The postcode/DPS may be better as you can then quote the venue postcode in the app. You probably do need the venue name as well to quote in the app. That is not a lot of data that is actually needed.

If you do that, you can make a code like this which has a URL, UPRN, and premises name in it, and is way less dense and easier to scan.

With just a postcode/DPS it is possible to go even smaller!

Is the app OK though?

Just to be clear, this is criticism of the QR code not the track and trace app. There is a blog on that which is quite interesting, but does not explain why they felt it necessary to sign the QR codes, or why they did not make them a URL format for easy access to the app. Here: https://www.ncsc.gov.uk/blog-post/nhs-test-and-trace-app-security-redux (the previous app described here https://www.ncsc.gov.uk/blog-post/security-behind-nhs-contact-tracing-app)

P.S. I have a QR code generation library available free on GitHub. here.

Dull images

I have been busy on all sorts of little things, none of which seemed worth blogging.

But one of the more fun little distractions was fixing my photo site. I upload images direct from camera on wifi, and they have Exif data extracted and all put in a database. But it was slow, and I ran in to a slight snag with the latest camera. It is still slow, but a complete rewrite of the javascript and the C/database backend has made a hell of a difference.

I am recording images in HEIC (Canon HIF) files. This is 10 bit colour and set with absolute minimal compression. It seems a good compromise. Obviously Canon raw files would be better, and JPG would be easier, but I thought I would give these a try.

There is a nice library called libheif, which has a tool heif-convert which converts the image from the Canon HIF file to JPG. Sorted, or so I thought. I do have to use the latest version as Canon use something odd, it seems, in the header, but that does work.

Unfortunately I am then in a world of weird with colour profiles. The resulting JPG is reduced to 8 bits per colour, as expected, but also has a colour profile applied. This works fine when viewed on my mac and in some browsers. However, it breaks, badly, in other cases. Notably viewing in firefox (even on Mac) and if sent to some social media - twitter just strips the colour profile. Facebook makes an effort. The other issue is various tools to get a thumbnail or make a scaled image, also strip the colour profile. So I had to make things re-apply it at each stage.

The problem is that without the right colour profile the images look, shall we say, dull!

These three images a) with colour profile, b) with profile stripped, c) converted profile.

Update: Blogger fixes the "preview" on image a) here, click on it in firefox to see the problem!

The solution was to use a tool called jpgicc which will convert profiles really quickly on JPG files. Just running it with no options to understand the original profile and convert to a default sRGB for JPG works a treat.

This was not the only issue. The convert from HIF tool also had some slight challenges, but the library allowed me to make my own convertor. The main issue was that the image was coming out of the library rotated correctly - e.g. a portrait image was portrait. This is great until you copy over the Exif data, which says the image needs rotating. The end result is a JPG that claims to need rotating. I was able to strips the rotation from the Exif.

With these two tweaks I now have JPG files that look OK on the photo site, and work for preview on social media, but the raw HEIC available for printing, etc.

I also made nice social media preview links, e.g. https://k.gg/213678

And yes, I know, my photo site is k.gg now, sorry, challenging times.