The TOTP seed storage dilemma
This means that we do not know the password people have used! We can check a password attempt against the hash (and then forget the password), but we never store the actual password.
If the user database was to be compromised in any way the attackers would not get real passwords, and so could not use them. More importantly, given so many people re-use passwords, it does not give then the passwords people are using on other systems.
So far, so good.
In additional to passwords we are now using two factor authentication using an authenticator app that provides a new 6 digit code every 30 seconds (Timed One Time Passwords). This uses a seed that is a long random number which is stored in the app and which we have to know as well - that way we can generate the same code and check it matches. We actually generate the codes for the last few minutes and check it matches any of them.
This creates two factors - one is something you know, being the username and password, and the other is something you have being the smartphone app or authenticator device. There are, of course, issues of ensuring the two have the same seed, but using a QR code on an https page seems a good compromise.
The problem is that we have to be able to see the seed to check the code, so it is not hidden. If that seed gets out then the authenticator is compromised as someone else can always generate the same codes. So in an ideal world we do not want to be storing this seed in the clear.
The small revelation I had this morning was that we could simply encrypt the seed with the plain text password. This means that when you provide the passwords and authenticator code, we can check the password, and knowing it is right we can use it to decrypt the seed and check the code, all in one go.
This is great, and I nearly rushed off an implemented it before realising a significant number of shortcomings with this.
One big problem is lost passwords. When changing password you always have to know old and new passwords so as to decrypt the seed with the old one and re-encrypt with the new one. This is fine for a change password form, but not for any sort of lost password reset process.
Indeed, at present, we use the authenticator code to validate the reset password request that is sent by email (so working email as well as using the authenticator code as two factors). However, if the authenticator code cannot be validated without the password, you cannot do that. You either have to trust just the email working, which is not ideal, or you have to find some other validation process as well. Also, when resetting a password you have no choice but to also issue a new authenticator seed and reset up the app with that new seed.
You also cannot use the code as a validator when talking to staff as they could only check the code unless they also have the password, and we would never want to ask a customer for their password.
So this creates a trade off - transparent storage of the seed on our systems and added convenience and some extra security on password reset, or encrypted seed on our system and some much more constrained processes and less convenience.
I can see we may end up with the underlying libraries allowing both options and using for different systems as appropriate.
Isn't security fun some times :-)
P.S. Read the comments - at least one important point I had not realised.