2015-12-24

Royal Mail Shipping API

Update: See latest blog post on this.

Having worked out what the WSSE PasswordDigest uses, I am struggling to get past "Authorization Failure.", and sadly it sounds like they have all buggered off home.

The example they sent me matched what I worked out, it includes a Nonce, and a Created date time, and using the password supplied, I was able to create the same Password digest.

To do this I had to use a Password in the algorithm that was not the actual password.

Password_Digest = Base64 ( SHA-1 ( nonce + created + password ) )

I had to use a base64(sha1(password)) in its place. That worked to match the example. But it is not letting me log in. I used this because the specification says "For Shipping API V2 the password used in the below formula is the base 64 encoding of the SHA-1 hash of the plain text password." even though it goes on to say "Password_Digest = Base64(SHA-1(Nonce + Created + SHA-1(Password)))"

However, I have found the spec has errors, little things like it defines an object unitofMeasure which if used causes the whole fail to report "schema validation failure". The example had unitOfMeasure which works. So I cannot actually trust the spec. Not a good start.

However, they helpfully provide a zip of various versions of the authentication done in different languages. This is useful, but also shows that something that should be simple is in fact clearly very complicated.

In perl they provide :-

my $conct = $nonce . $creationdate . (sha1($password));
my $passworddigest = encode_base64(sha1($conct));

Which seems to use simply the raw SHA-1 of the password in the algorithm, not a base64 of it.

In PHP they provide :-

$nonce_date_pwd = pack("A*",$nonce) . pack("A*",$CREATIONDATE) . pack("H*",sha1($password));
 $PASSWORDDIGEST = base64_encode(pack('H*', sha1($nonce_date_pwd)));

Now, my PHP is not good, but reading up, the H* means hex encoded. So that looks like it uses a hex coding of the SHA-1 of the password in the algorithm, not a base64 coded version.

In python :-

hashedpassword = sha.new(password).digest()
digest = sha.new(nonce + CREATIONDATE + hashedpassword).digest()
PASSWORDDIGEST = base64.b64encode(digest)

Which looks like using the raw SHA-1 in the algorithm, not a base64 coded.

So I am thinking the spec, and the example they gave me, is wrong. I even tried using their python code to generate Created, Nonce, and Password fields.

Sadly, it is not working to get past "Authorization Failure.", and it looks like the have buggered off home.

This is NOT THE WAY to specify an API. You need clear documentation that is actually correct, and you need helpful error messages on your on-ramp system. Not impressed at all.

Update: Nope, they are there until 6pm - yay, well done Royal Mail - trying to find the problem for me now.

Update: We are back to the base64(SHA-1(plaintextpassword)) now, and using a different applicationId, and working. Yes, that disagrees with all of their code samples! So hoping to get on live system soon.

22 comments:

  1. If you get an answer to this then please update this, I am struggling with the schema validation failure in .Net, same with soap ui. I have tried with the bare minimum mandatory fields and still get the same. The docs are a pile of ****.

    Are you adding your wsse stuff direct to the xml? I have tried posting mine below however the code is removed by the comment box! I can send it to you via google+ etc if you woyuld like to take a look:



    I am then providing the client id, secret, contenttype and SoapAction as WebClient Headers eg

    Using wc As New WebClient
    wc.Headers.Add("Content-Type", "application/soap+xml")
    wc.Headers.Add("X-IBM-Client-Id", ClientID)
    wc.Headers.Add("X-IBM-Client-Secret", ClientSecret)
    wc.Headers.Add("SOAPAction", SoapAction)

    ReplyDelete
    Replies
    1. Latest clues!!!

      The xmlns has to be at the top level, or as per their examples, it cannot be at the level most closely encompassing the usage as you would expect. I have had to change my XML library. Yes, their system expects specific placement of xmlns

      I have managed to get working on a test application ID they provided, not the real on-boarding application ID. So I can make progress. But the WSSE password used is base64(SHA-1(password)) not as per the example code which is just SHA-1(password).

      I feel like I have been banging my head against a brick wall and may have finally knocked on small brick out.

      Delete
    2. So this:

      Password_Digest = Base64(SHA-1(Nonce + Created + Base64(Password))) ?

      Should be
      Password_Digest = Base64(SHA-1(Nonce + Created + SHA-1(Password)))

      Delete
  2. It is working on the test applicationId using
    Password_Digest = Base64(SHA-1(Nonce + Created + Base64(SHA-1(Password))))

    ReplyDelete
    Replies
    1. Hi Revk,
      I've tried with different application ids but still getting the authorization error. Please help me get past over this.

      Can you send me the code for get past over the authorization failure to vivektrial2016@gmail.com. I'd be grateful becoz we are trying that 4 months or so. Please send me the code. Thanks in advance

      Vivek

      Delete
    2. Hi Revk,

      Can you send me the code for get past over the authorization failure to vivektrial2016@gmail.com. I'd be grateful becoz we are trying that 4 months or so. Please send me the code. Thanks in advance

      Vivek

      Delete
  3. Thanks, still no joy here, just hit the E0004 Failed Schema Validation error everytime. This has to be hands down the worst api I have ever worked with, well, if you exclude Yahoo!

    ReplyDelete
    Replies
    1. OK yes, the untagged objects like <code> have to be untagged. Also, you need to use the xmlns tags they use as well. It is horrid. Check line for line with their example and remember order of fields matters. I may publish my code when I have it working, getting there slowly - just got a PDF label in base64 out of it!

      Delete
  4. I envy you!!!

    This is what I am sending http://pastebin.com/MLhbCWMT (details removed)
    Been struggling with this for a couple of days now, really is infurating!

    ReplyDelete
    Replies
    1. OK let me take a look in a moment, just working out the other messages now. I have added a label image to this post though :-)

      Delete
    2. Don't worry about it, I have noticed multiple mistakes by comparing it with the examples. I am getting closer, the message has changed to E0007 now ;-)

      Delete
    3. Try and catch me on irc if still stuck!

      Delete
    4. Cheers, I have noticed my function for sha1 is not returning the correct data so Ill fix that and see how I get on.

      Delete
  5. Should the login details for appid 0127229000 be the different from my normal login? Still hitting the E0007 error regardless of what I try. If you have a moment do you mind comparing my function for getting the wsse details compared to yours? http://pastebin.com/cNaPrPz6 Cheers

    ReplyDelete
    Replies
    1. They worked the same for me, but I'll take a look in a bit.

      Delete
  6. Never mind, I seem to be in with my application id! I used the sha-1 function from the included c# file and that seems to have done the trick.

    ReplyDelete
    Replies
    1. OK so exactly how are you doing the password digest?

      Delete
    2. Like this: http://pastebin.com/jKAeUkf8

      Now just to work through the onboarding spreadsheet!

      Delete
    3. Hi Dave

      Just wondering if you've got some sample code for this, I've been struggling with "Authorisation Failure" for what seems like an age.

      Thanks

      Chris.

      Delete
  7. Hi all

    Can't believe I have stumbled across this

    just to let you know I am now onto my fourth developer on freelancer trying to get this code developed

    I haven't worked with vb commercially for about 15 year - so after having a go and failing I outsourced to the experts, but they are struggling with this as well

    I got the soapui project working and tried the upgraded version ReadyAPI to generate .net artifacts - but it fails

    Sine I missed the SOAP revolution I totally struggle with ChannelFactory stuff

    I am really hoping this forth developer can crack it - I have been trying to get this working since october

    If he does get it I am happy to post whole solution here - I wouldn't see anyone else suffer - the RM documentation is dreadful

    ReplyDelete

Comments are moderated purely to filter out obvious spam, but it means they may not show immediately.

Missing unix/linux/posix file open option

What I would like is a file open option for "create replacement file". The idea is that this makes a new inode in the same mount p...