stop storing plaintext passwords about axiom

[mithrandi]

I wonder if there are any Axiom applications that are using a non-plaintext authentication method (Quotient, maybe?); those would be broken by this change, right?

[lvh]

So, from a cryptographic standpoint, a KDF pretty much has to block. While the standard for what counts as acceptable blocking or not differs for each application, we're talking order of magnitude 10-100ms here, and that is definitely not acceptable for some, if not most, applications.

FWW, this wouldn't have broken my stuff, but only because I totally dodged the whole Axiom user model thing for precisely this reason.

@mithrandi Not sure what you mean; don't they just upgrade some objects to some interfaces? ISTR IUsernameHashedPassword is synchronous, but that just means that interface is broken too, which might mean that this ticket is a bit of a rabbit hole :-) If that's what you mean, then yes; we can't fix things that want to use that interface.

[lvh]

Also, I think there's a twisted ticket for that...

[mithrandi]

My question doesn't have anything to do with synchronous vs asynchronous.

Authentication methods that don't send the password over the transport in cleartext typically require the password in cleartext locally so that they can perform some kind of challenge-response handshake with it. For example, CRAM-MD5 (which I think is a commonly-used POP3/IMAP authentication method?) has the server send a challenge C, and the client respond with HMAC-MD5(C, password), which requires the server to know password in order to calculate the expected value to compare it against what the client sent.

In terms of interfaces, these authentication methods require IUsernamePassword and cannot use IUsernameHashedPassword.

[glyph]

@mithrandi That sort of question is exactly how we got into this mess 😄 .

CRAM-MD5 can "protect" your credentials on the wire and on your server. But I put "protect" in quotes for two reasons:

1. the way we do it, and the way most clients do it is by storing the plain text, rather than i_key_pad and o_key_pad anyway, so the server gets to see the credentials regardless
2. the protection doesn't actually work; it's reversible due to weaknesses in MD5 by capturing either the server's state or the wire conversation.

Therefore Quotient (et. al.) should just stop using it and having the pretense that it can protect credentials on the wire. You have to use TLS. It also can't protect credentials on the server because the crypto simply isn't good enough.

There are protocols that can effectively offer protection in all three places (on the wire, in the server's memory, on the server's disk), such as TLS-SRP but those are not widely deployed enough to use in practice. However, if we have a protocol where you have to give the server your plaintext credentials anyway, it's always possible to upgrade to a better protocol when it's available by doing so on login.

We should just switch over to a cred thing that authenticates only via PLAIN, and only operates over TLS.

[glyph]

@lvh -

So, from a cryptographic standpoint, a KDF pretty much has to block

I should mention that this isn't all that serious of a problem. Cred has Deferreds returned from all the relevant integration points for specifically that reason, and I don't think we have to break any compatibility at that integration point. We will have to break the .password field, obviously; that's sort of the point, and have an upgrade process which spins the reactor. But anyone calling something that relies on non-blocking auth in any fundamental way is broken regardless.

[glyph]

@lvh - I know that keybase has a thing that says "scrypt" when you log in, and it's presumably using some fancy javascript to avoid sending my real actual password to the server. Is there something one can do to make scrypt itself work as a challenge-response protocol?

[glyph]

I just had a many-hour-long conversation with @lvh about threat models and I believe the conclusion was this:

It's possible to use extra fancy authentication mechanisms with help from the client (browser-side javascript to compute a pre-hashed scrypt key, or better yet using a KDF to derive a public key to sign a server challenge) to mitigate a very slightly more sophisticated threat model and reduce the opportunity for password re-use attacks. The other approaches would be a prohibitive amount of additional work and, based on vagaries of client security, might not practically provide anything more secure (i.e. any attacker who can do X can do Y, and we'd just be adding a defense against X without defending against Y).

So, fundamentally "just use txscrypt" is the best compromise between providing an acceptable level of security for users and maintaining our own sanity.

[glyph]

(according to github timestamps that was only an hour but it felt like daaaaaays)

[glyph]

Now, regarding @mithrandi's question about whether anything will break, since I guess I didn't answer that:

Yes. Quotient will break any clients which have selected the CRAM-MD5 authentication mechanism, because it incorrectly, unconditionally advertises CRAM-MD5. However, clients may happily select LOGIN. https://github.com/twisted/quotient/blob/95f2515219da99a77905852bc01deeb27e93466e/xquotient/mail.py#L317-L322

While I was investigating this I realized that the whole way it does this is totally undocumented. There are some IMAP-specific credentials:

https://github.com/twisted/twisted/blob/trunk/twisted/mail/imap4.py#L384-L413

which are weirdly randomly manipulated with no explanation:

https://github.com/twisted/twisted/blob/trunk/twisted/mail/imap4.py#L1036-L1040

commented (but not documented) to implement an interface that doesn't actually exist anywhere:

https://github.com/twisted/twisted/blob/trunk/twisted/mail/imap4.py#L520-L521

and the way you turn on CramMD5 is to use this thing:

https://github.com/twisted/twisted/blob/trunk/twisted/cred/credentials.py#L378-L408

and appears to depend on this one weird method:

https://github.com/twisted/twisted/blob/trunk/twisted/mail/imap4.py#L390-L391

[glyph]

The place it advertises the wrong list is here: https://github.com/twisted/twisted/blob/trunk/twisted/mail/imap4.py#L541-L542

[glyph]

Or rather, it's not advertising the wrong list there, it's advertising the list that it's given, but ... Quotient unconditionally CramMD5Credentials in its challengers map, and that adds it as a capability there.

The correct thing to do architecturally within cred would be to check the credentialInterfaces of each checker in the portal's checker list.

Unfortunately, to make this "easy", IUsernamePassword also specifies a checkPassword(password) where password is supposed to be a plain-text password gotten out of a database. Ugh. However, luckily, this is merely aesthetically unfortunate. IUsernamePassword also specifies a password attribute which is the plain-text password from the wire, and we could easily write a checker which could check IUsernamePassword without ever calling checkPassword, ignoring it entirely.

[glyph]

To be more specific, when I say "check the credentialInterfaces of each checker in the portal's checker list", I mean, imap4 itself ought to select which challengers to use because a challenger may only be used with a checker that is a provider of one of the interfaces that the given challenger (i.e. callable that returns a credential).

This is a bit of an implementation accident, but since there's no documentation anywhere of what goes in the challengers attribute, just about anything is an implementation accident.

[glyph]

Sorry, a checker whose credentialInterfaces contain an interface that a given challenger is an implementer of. The checker should just provide ICredentialsChecker.

[mithrandi]

PR up! Reviews desperately desired.