We have a new card printer at last!
We have used Zebra printers for cards to date, and as some may recall I found they only had windows drivers so I made a suitable linux command line app to talk to it.
Unfortunately, I am sorry to say, that the Zebra is really not coping. Basically the issue is printing SIM cards. Normal white plastic cards are fine on any printer but SIM cards tend to be different materials (we have two types of SIM and they are different to each other and to plain white cards). They need more heat to transfer, and the only way to get them to work on the Zebra is hot enough to damage the transfer rollers and impact print quality.
So, we needed a new printer, and we have tried a few, as have the dealers with which we have been working on this. Quite a few have been tried, even the DataCard one, they all had issues with printing on the SIM cards, especially the data SIMs on Three. In the end we have finally found one that needs only very minor adjustment to settings to work on the SIM cards. Indeed, with the same setting it will do plain white cards and both types of SIM card. The Matica XID9300 printer.
It is clearly an older style, well build, solid (and heavy) printer - apparently used for ID cards by police even.
This is not that new a model as printers go, but it clearly works on slightly different mechanical and film transfer processes to some others and just copes with SIM cards seamlessly. Importantly it also copes with gaps and the contacts with no special measures - it simply manages not to print on the contacts or over the gap. Compared to the Zebra, this is magic - it needed an inhibit ribbon and print to mask where the transfer was applied - leaving a white border and alignment issues. Without it, you got this cellophane type flecks that stuck to everything and had to be wiped off the contacts. So the Matica is massively better on the printing. We set speed -1 and heat +1 and it just works. It also manages not to bend the cards (which was a challenge on the zebra).
I really feel print quality will be fine now, which is excellent news.
Printing over Ethernet
An important point for us is printing over Ethernet (TCP/IP) and from linux. Again, as expected, only windows drivers come with the printer. This is where the reverse engineering comes in.
First step is windows - I installed a copy under virtual box on my Mac. I tested some stuff on IE. I then installed the drivers. Interestingly, there is a choice of driver, including "Third party" drivers. This suggests they have published a spec for the interface. I have asked the dealers many times, and Matica, and no reply, so tcpdump it is.
Sending print jobs from the windows machine (Print Test Card) I tcpdumped the traffic on the host Mac and stored in a pcap file. I could see three types of traffic - simple pings to the printer, UDP to port 50330, and TCP to port 9100. I then used tshark with -z follow,tcp,hex,0
to extract the content of the TCP exchange.
The UDP appears to allow settings and status to be extracted and set - I have not debugged this yet but it should be quite simple. There is a "monitor" app that gets status and displays, so I can dump what it does. I'll look in to that later.
As for the print itself, over TCP port 9100, you start by looking at the dump. It quickly becomes clear that there is a message exchange protocol of some sort. The printer started with
00000000 f3 00 02 00 00 00 00 10 00 00 00 00 00 01 f9 2f
00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 50 52
00000020 49 4e 54 45 52 30 31 00 5b f0 b0 5c 00 00 00 00
00000030 00 00 00 00 00 00 00 00 58 49 44 35 38 30 69 65
00000040 00 00 00 00 00 00 00 00
Which contains the printer ID and model and other stuff. There are then a series of messages sent from the printer driver and replied to by the printer, they all have a common format and the first 16 bytes have an obvious pattern, e.g.
00000000 f2 00 03 00 00 00 00 1d 00 00 00 02 99 99 99 99
00000010 15 09 00 d2 73 09 0a 06 1e 17 00 00 4f 00 57 00
00000020 4e 00 45 00 52 00 5f 00 54 00 4f 00 44 00 4f 00
00000030 00 00 00 00 00 00 00 00 00 00 00 00 78 00 69 00
00000040 64 00 2e 00 64 00 6f 00 63 00 75 00 6d 00 65 00
00000050 6e 00 74 00 00 00 00 00 00 00 00 00 00 00 00 00
00000060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000070 00 00 00 00 00 00 00 00 00 00 00 00
00000048 f3 00 04 00 00 00 00 02 00 00 00 00 99 99 99 99
0000007C f0 00 01 00 00 00 00 03 00 00 00 00 00 00 33 b0
0000008C 01 02 00 00
00000058 f1 00 01 00 00 00 00 02 00 00 00 00 00 00 33 b0
00000090 f0 00 01 00 00 00 00 03 00 00 00 00 00 00 33 b1
000000A0 02 02 00 00
00000068 f1 00 03 00 00 00 00 03 00 00 00 00 00 00 33 b1
00000078 01 02 04 00
The first message from the printer was a generic job header (OWNER_TODO, and xid.document), and it seems clear that each message from the driver had a reply from the printer - so far so good.
The first thing to establish in any binary protocol is endianness of binary values. Fortunately we have a some sort of increasing counter (you see 33B0, 33B1), and it becomes obvious that the data is big-endian (most significant byte first).
The second thing in any block protocol is trying to find a length indicator in the blocks - this is important to ensure you can process blocks even if you do not yet understand them. Some times the length is implicit from knowing the message, but that is horrid - in this case the length is in the header. I quickly spotted that the second 32 bit word was usually 2 and sometimes 3 and sometimes much higher for longer messages. It was clear that this was number of 32 bit words following the length word, and clearly had a minimum of 2 making a 16 byte header on all messages. This also supported the idea that the data was big-endian.
It also became very clear that the 4th word was some sort of sequence, as every message from the driver got a reply quoting back the same number and the next message from the drive was one more. Even the fixed generic document header, which used 99999999 (odd, as clearly not decimal) gets that in the reply, so looks like the reply simply repeats the request and can be used just to ensure you stay in sync. That said, it seems that every message waits for a reply anyway, so not really needed.
The 3rd word was almost always 0, but occasionally had some value. It seems to be used on reply message to indicate error codes, where 0 is all good.
And finally the first word has some logic - it starts F0 or F2 from driver, and F1 or F3 from printer, always followed by 00, and then 01 to 04, and followed by 00. Only the initial exchange used F2/F3, after which F0 and F1 were used.
I quickly worked out some obvious formats :-
- F3 00 02 00 Is printer initial message
- F2 00 03 00 Is job header from server
- F3 00 04 00 Looks like ack for job header
- F0 00 01 00 Sends a command?
- F1 00 01 00 Simple ACK
- F1 00 02 00 ACK with status (e.g. printer busy) in 3rd word.
- F1 00 03 00 ACK with some data
- F0 00 02 00 Sends colour panel data
Oddly on the TCP dump after sending a print command the driver kept checking status and getting a busy response code, but when I tried the response did not come in until the requested command was done and had response error 0.
The interesting one was where it sends some sort of command, as this was a sequence of words in itself. E.g. this has a command 04 02 80 00
000000A4 f0 00 01 00 00 00 00 03 00 00 00 02 00 00 33 b2
000000B4 04 02 80 00
The command itself has a format, and looking at several it is clear that the first byte is the command code, the second is length (bytes) that follow (usually 2) and then data bytes.
By trial and error I have managed to decode many of the commands :-
- 01 appears to check the command/error state, returned in the 3rd word of the header
- 02 appears to check the state of printing, where the card is?
- 03 appears to re-initialse the printer
- 04 appears to load a card
- 05 appears to move a card about
- 06 appears to print the colour panels on to the transfer ribbon
- 07 appears to transfer the image on to the card
- 09 appears to encode mag data
- 0A appears to engage or disengage the contacts for the contact station
- 0D appears to request info, e.g. what looks like a serial number request
With 04, 05, and 07 there appears to be one byte with some flags, the only one I can make do anything is 10 meaning flip the card over, 80 means something but not sure what; then a byte with some location for card. This location is also last byte of the response to 02020000 command.
- 00 Print ready
- 01 Contact station
- 02 Contactless station
- 03 Mag coding station
- 04 Reject card
- 05 Eject card (after print)
- 06 In the printer - not sure what?
The contact station position does not do anything until you then use 0A020000 command to engage the contacts. Use 0A024000 to disengage.
Oddly the length of the 07 messages appears to be 6 bytes not 2, and have 00 00 00 00 on the end.
The 09 command has 00 and 01 as first two bytes but then has a string that starts TB (track and bits), e.g. 16 is track 1 coded in 6 bits, then length in characters, and then the encoding data (without start/end sentinels included).
The image format was easy to work out, e.g.
000000B8 f0 00 02 00 00 02 9f cd 00 00 00 00 00 00 33 b3
000000C8 01 00 00 00 00 0a 7f 24 00 0a 7f 20 00 00 00 00
000000D8 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
000A7FE4 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
The message contains (as you can see) 00029FCD(+2) words. In the data it starts 01 00 00 00, and then has two words with each appear to be bytes following, so 000A7F24 and 000A7F20.
The 01 is the colour, 01=Y, 02=M, 04=C, 08=K, and we assume 10=UV. The 06 command 4th byte is a bit map of which panels to transfer. After these initial words are bytes for the image, 00=unprinted, FF=full print, one byte per pixel for 664 rows of 1036 columns covering the card in that colour. Very simple!
Problems - SIM/IC contact station
Getting things to print was easy! The problem was the IC contact reader. We had been very clear with the dealer that we wanted to use Ethernet/IP as the computer printing this was nowhere near the printer. We needed the IC contact controller to work over Ethernet.
Sadly this appears not to be the case, just like the Zebra. There is a contact point to connect to the card, and either that is wired to the back directly, or you have an inbuilt IC contact controller which has an (additional) USB on the back. There appears to be no integration to the control board.
Looking inside, it is clear that the controller is in fact simply a cheap USB card controller with a header to connect in to the contacts in the printer - it even has the original card slot in it so I could insert a card and talk to it over USB!
The step that was lacking is how to tell the printer to place the card on the contacts, and I have tried everything. I cannot find any test or demo app to do this. There is one called cardpresso that claims to, but does not appear to do anything. I have tried loads of values on the 04 and 05 commands, and I am rather concerned that one is rejected with an "IC module not installed" type error.
At this stage I do know know if the card reader contacts are not properly installed or configured or if I have simply not found an app to try using it under windows (for which I could dump the traffic).
(Update: we now know printer works and how to load on to contact station)
Update 13th Oct: I managed to eventually get the maintenance app running - it involved loading an XP version of windows as anything later would not run it. After a lot of messing about I worked out that whilst it would talk to a printer over Ethernet it would not allow "Self test" to be available unless connected over USB. The self test allowed me to confirm the contact station does work - loading the card and you hear a clunk as it engages and the USB card reader wakes up and reports a card connected. So I know the hardware works. I tried usbsnoop but the USB data is nothing like the Ethernet data (surprisingly) so no clue. I have made some more progress - from the manuals it seems the printer will load the card on to the contacts as part of printing, but the driver refuses to even try doing that unless it has a "plugin" which it runs at that point. The plugin is expected to talk to the reader and do its stuff as needed before printing continues. I can tell the printer I want to encode a chip and get as far as it complaining about the lack of the plugin dll. I don't actually want to do any more than see what command it sends to load the card - it is so frustrating. A completely "dummy" plugin that says "yes, card encoded OK" would be fine. Googling is not helping. Some people seem to sell plugins for some printers and cards but even they don't make it obvious how you buy one or get one to try or anything. Thankfully one manual says a free development package is available demonstrating the development of a plugin - so I have asked for that.
Update 13th Oct: The manufacturers have sent the SDK, which may be handy. Now, the SDK has various licensing and some source code. I have sent it to a colleague to see if he can make the printer put a card on the contact station and I'll dump what it sends. This is partly because I have no bloody clue how to do any code dev on a windows machine or make a COM object SDK thing and Cliff does, and partly because I don't want any drivers I make to be tainted by having read their source code and so a derived work of something copyright.
Update 14th Oct 5am: Cliff spent all night working on understanding how to poke some XML at the driver as a COM object, which cleared up the use of position 01 for the card contact station, and the 0A command to engage the contacts. Today I shall be testing and working on the driver code.
Update 14th Oct: I have command line tools for composing cards from postscript and image files for colour, black and UV layers for both sides and making the raw file for printing as well as making image previews for integration with web pages, and I have a tool to print to the Matica printer. I have managed to update our various systems that print cards.
Driver for linux
Let me know if you want a copy.
I am not in fact doing anything with any of the s/w they provide other than using as a normal printer driver - not decompiling it or anything. So no issue with Matica copyright here. Even so, EU regulations would allow me to do that to make compatible products if I had to.
Thanks to Mike and Simon who came over for the weekend to see the man cave, and for whom coding is now a spectator sport! Thanks to Cliff for playing with COM objects.