Monday, 12 December 2016

The TOTP seed storage dilemma

When making any sort of login system you typically used a username and password. One of the key things you should do, if at all possible, is hash the password.

This means that we do not know the password people have used! We can check a password attempt against the hash (and then forget the password), but we never store the actual password.

If the user database was to be compromised in any way the attackers would not get real passwords, and so could not use them. More importantly, given so many people re-use passwords, it does not give then the passwords people are using on other systems.

So far, so good.

In additional to passwords we are now using two factor authentication using an authenticator app that provides a new 6 digit code every 30 seconds (Timed One Time Passwords). This uses a seed that is a long random number which is stored in the app and which we have to know as well - that way we can generate the same code and check it matches. We actually generate the codes for the last few minutes and check it matches any of them.

This creates two factors - one is something you know, being the username and password, and the other is something you have being the smartphone app or authenticator device. There are, of course, issues of ensuring the two have the same seed, but using a QR code on an https page seems a good compromise.

The problem is that we have to be able to see the seed to check the code, so it is not hidden. If that seed gets out then the authenticator is compromised as someone else can always generate the same codes. So in an ideal world we do not want to be storing this seed in the clear.

The small revelation I had this morning was that we could simply encrypt the seed with the plain text password. This means that when you provide the passwords and authenticator code, we can check the password, and knowing it is right we can use it to decrypt the seed and check the code, all in one go.

This is great, and I nearly rushed off an implemented it before realising a significant number of shortcomings with this.

One big problem is lost passwords. When changing password you always have to know old and new passwords so as to decrypt the seed with the old one and re-encrypt with the new one. This is fine for a change password form, but not for any sort of lost password reset process.

Indeed, at present, we use the authenticator code to validate the reset password request that is sent by email (so working email as well as using the authenticator code as two factors). However, if the authenticator code cannot be validated without the password, you cannot do that. You either have to trust just the email working, which is not ideal, or you have to find some other validation process as well. Also, when resetting a password you have no choice but to also issue a new authenticator seed and reset up the app with that new seed.

You also cannot use the code as a validator when talking to staff as they could only check the code unless they also have the password, and we would never want to ask a customer for their password.

So this creates a trade off - transparent storage of the seed on our systems and added convenience and some extra security on password reset, or encrypted seed on our system and some much more constrained processes and less convenience.

I can see we may end up with the underlying libraries allowing both options and using for different systems as appropriate.

Isn't security fun some times :-)

P.S. Read the comments - at least one important point I had not realised.


  1. dont know if this would make it more or less secure but how about encrypting the seed with the account number? It would help where seeds are stolen but not where other info is stolen too

    1. Quite, or a fixed (hidden in the code) secret. Whatever you do is no good if it uses something on the machine / known. It pretty much has to be the users plaintext password or not bother I think.

  2. Beware, that may significantly weaken your password hashing.

    If an attacker can tell whether a seed decrypted correctly or not, then they can just try decrypting the seed with lots of possible passwords. This may be much quicker / more parallelizable / use less memory than testing a password against an Argon2 hash. In that case, you've lost all the security from Argon2.

    1. Interesting point - though they can only test the seed if they have an auth code, but they could do so with just one auth code and the time it was generated. So yes, interesting point. Hmmm.

    2. This is fairly easily avoided: encrypt the seed with the stretched (argon2) hash, and store the hash of the hash for password verification.

    3. "they can only test the seed if they have an auth code" - is that really the case?

      You might be OK if the "seed" just a chunk of random data, stored in binary, and it's encrypted using an encryption algorithm that doesn't do any authenticity checks and doesn't use padding. (Perhaps a stream cipher like AES-CTR).

      But if there's any structure to your "seed", e.g. if you convert it to ASCII hex before encrypting it, or if you have a binary header at the beginning of it, then it's easy for an attacker to see if the decrypted data "looks like a seed" or not.

      And if your encryption algorithm adds padding, then it will check the padding when it decrypts the seed, so that will tell an attacker if their password is right or not.

      And obviously, if your encryption algorithm provides authenticity, like AES-GCM does, then the decryption algorithm will tell an attacker if their password is right or not.

    4. The seed is currently random binary data, and the idea would be to encrypt in a way you cannot tell it "looks like a seed". You'd have to actually test the seed against an auth code. But even that is a weakness as the auth is just an HMAC SHA1 so less work than Argon2.

  3. You could at least encrypt the seeds in the database with a common key, and protect this key in the same way as you'd protect other stuff on the servers (SSL cert private keys for example). That way if someone finds a SQL injection attack and tips out your user database, they won't have the plaintext seed values.

    1. Indeed, something to consider, and helps against some attacks (leaked data rather than compromised server).

  4. How about just to have an isolated service which is human free and whose whole purpose is to generate feed and vlidate code. The feed never needs to leave the service. With proper setup it's quite doable on AWS.

  5. What about splitting the seed-validation in two?
    The 2FA system becomes the generator of seeds, and the point on the network where queries are sent to, but it does not hold the seeds. A second system, a "secure vault" is connected to the first system via something along the lines of a uart and is NOT on the internet.
    Seeds are inserted with Account Details and Seed IDs,
    Seeds can be disowned if thought lost by ID and account name
    Authentication attempts can be fed to the vault and validated (in Account, in Code, out pass, out IdAndValidityOfPassingSeed).
    Seeds can NOT be extracted from the vault over the link.

    A mechanism for backing up the vault and replicating it at another site need to be establisted.

    1. Indeed. One nice thing about coding the mechanisms in one package for use in half a dozen systems is we could move to systems like this easily. Though even internet connected but separate and https would be a big step as the seed stays locked in a separate box with very specific interface only. An advantage of encrypted seeds in the database though is that they are easy to replicate and backup as needed, so it is, as always, a trade off.

  6. Please consider supporting FIDO U2F (token base with transitive secrets). It is supported in Chrome and Firefox. There is a list of supporting sites at

  7. What about having two seeds, one encrypted with the password and the other one stored in the clear?

    For things where you have the original password you can use the combined output from both seeds. For the less common case where you don't have the original password, you can just use the cleartext seed and accept the lower level of security that this offers.

    That said, if you allow password reset through the cleartext seed, someone who stole your database can probably reset someone's password, which defeats the 2-factor auth down to a single (compromised) factor.

  8. The TOTP keys are on a server, it responds to a validate request but only from the small set of internal systems that need to validate TOTP requests. It will receive an account ID and a TOTP code to validate. Any one account is only allowed a small number of fails, at which point all subsequent requests fail. ie someone it trying to brute force the code. The only other transaction type is set TOTP key call.