signing via openssl fails about aws-kms-pkcs11

[ozbenh]

Note: I'm basically following your README.md example using an arbitrary large file (larger than 2560 bytes) as input

[ozbenh]

Hrm.. ok so you expect to be getting a digest which indeed shouldn't be that big... I'm not sure what openssl is smoking here .. ugh.

[JackOfMostTrades]

So I'm just finally circling around to looking at this, and it does seem like the pkeyutl example in the README doesn't quite play nice with RSA keys (I just end up testing EC keys in my workflow). With RSA keys it seems like you need to supply a hint about what digest you're using because of all the weird digest identifier prefixes that get prepended to digests under the weird RSASSA_PKCS1_V1_5_* spec. I think it's just a matter of passing in -pkeyopt digest:sha256 option when doing both the pkeyutl sign and verify operations.

[JackOfMostTrades]

Yea, the pkeyutl does expect foo to be the digest (not the raw message). I also just checked and for RSA keys only you have to pass in the -pkeyutl digest:sha256 option so that openssl expects/uses that digest algorithm identifier prefix. Added a note about that option to the README.

[ozbenh]

Thanks !