Compiler-free Python crypto library backed by the OS, supporting CPython and PyPy
License: MIT License
My open source projects are primarily an artifact of various personal projects. Currently the only ones under active development are Package Control, asn1crypto and oscrypto.
I generally will add contributors to repos once they have shown a pattern of useful contributions, be it responding to issues or sending PRs.
Is there documentation on the asymmetric.Certificate class?
I am trying to retrieve the subject and issuer to compare them.
Hi, the problem occurs if I want to load certificate from variable. In this case this:
`ROOT_CA = b"""
-----BEGIN CERTIFICATE-----
MIID1TCCAr2gAwIBAgIJAIOVTvMIMD7OMA0GCSqGSIb3DQEBCwUAMIGAMQswCQYD
VQQGEwJVUzENMAsGA1UECAwEVXRhaDEXMBUGA1UEBwwOU2FsdCBMYWtlIENpdHkx
DzANBgNVBAoMBlZlbmFmaTEbMBkGA1UECwwSTk9UIEZPUiBQUk9EVUNUSU9OMRsw
GQYDVQQDDBJWQ2VydCBUZXN0IE1vZGUgQ0EwHhcNMTgwMzI3MTAyNTI5WhcNMzgw
MzIyMTAyNTI5WjCBgDELMAkGA1UEBhMCVVMxDTALBgNVBAgMBFV0YWgxFzAVBgNV
BAcMDlNhbHQgTGFrZSBDaXR5MQ8wDQYDVQQKDAZWZW5hZmkxGzAZBgNVBAsMEk5P
VCBGT1IgUFJPRFVDVElPTjEbMBkGA1UEAwwSVkNlcnQgVGVzdCBNb2RlIENBMIIB
IjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA0BobDKthxG5SuMfAp2heyDQN
/IL9NTEnFJUUl/CkLEQTSQT68M9US7TCxi+FOizIoev2k4Nkovgk7uM0q94aygbh
cHyTTL64uphHwcClu99ZQ6DIwzDH2gREsLWfj+KXw4bPsne+5tGxv2+0jG2at5or
p/nOQWYD1C1HB6ZQqvP3PypDjou7Uh+Y00bOfXkbYWr8GkX4XAL6UtC0jUnsBEZX
CuwO1BlIIoKNokhOV7Jcb3l/jurjzVWfem+tqwYb/Tkj6MI1YBqt6Yy2EsGsoAv1
E5/IGcjSQnLEqDWhpY0s2fA4o+bAMzyakDFKJoQbF982QhS2fT+d87vQlnMi1QID
AQABo1AwTjAdBgNVHQ4EFgQUzqRFDvLX0mz4AjPb45tLGavm8AcwHwYDVR0jBBgw
FoAUzqRFDvLX0mz4AjPb45tLGavm8AcwDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0B
AQsFAAOCAQEAWbRgS1qUyGMh3ToJ060s5cdoKzyx/ji5pRPXRxrmzzSxP+dlKX7h
AKUgYOV9FU/k2f4C7TeCZSsir20x8fKRg4qs6r8vHTcWnkC6A08SNlT5kjyJl8vt
qQTEsemnyBFis8ZFUfYdmNYqZXuWSb7ZBfNkR7qMVna8A87NyEmTtlTBkZYSTOaB
NRuOli+/6akXg/OW/GfVUD11D413CtZsWNzKaxj1WH88mjBYwQx2pGRzMWHfWBka
f6ZUnA9hhqxO4CHqQWmKPHftbGscwx5yg/J6J7TfG+rYd5ZVVhrr2un2xpOTctjO
lriDCQa4FOwP9/x1OJRXEsSl5YFqBppX5A==
-----END CERTIFICATE-----
"""
root_ca_certificate = asymmetric.load_certificate(ROOT_CA)`
Will cause "ValueError: The data specified does not appear to be a known certificate format"
To fix I need to start certificate string from the same line as quotes. This one works:
`
ROOT_CA = b"""-----BEGIN CERTIFICATE-----
MIID1TCCAr2gAwIBAgIJAIOVTvMIMD7OMA0GCSqGSIb3DQEBCwUAMIGAMQswCQYD
VQQGEwJVUzENMAsGA1UECAwEVXRhaDEXMBUGA1UEBwwOU2FsdCBMYWtlIENpdHkx
DzANBgNVBAoMBlZlbmFmaTEbMBkGA1UECwwSTk9UIEZPUiBQUk9EVUNUSU9OMRsw
GQYDVQQDDBJWQ2VydCBUZXN0IE1vZGUgQ0EwHhcNMTgwMzI3MTAyNTI5WhcNMzgw
MzIyMTAyNTI5WjCBgDELMAkGA1UEBhMCVVMxDTALBgNVBAgMBFV0YWgxFzAVBgNV
BAcMDlNhbHQgTGFrZSBDaXR5MQ8wDQYDVQQKDAZWZW5hZmkxGzAZBgNVBAsMEk5P
VCBGT1IgUFJPRFVDVElPTjEbMBkGA1UEAwwSVkNlcnQgVGVzdCBNb2RlIENBMIIB
IjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA0BobDKthxG5SuMfAp2heyDQN
/IL9NTEnFJUUl/CkLEQTSQT68M9US7TCxi+FOizIoev2k4Nkovgk7uM0q94aygbh
cHyTTL64uphHwcClu99ZQ6DIwzDH2gREsLWfj+KXw4bPsne+5tGxv2+0jG2at5or
p/nOQWYD1C1HB6ZQqvP3PypDjou7Uh+Y00bOfXkbYWr8GkX4XAL6UtC0jUnsBEZX
CuwO1BlIIoKNokhOV7Jcb3l/jurjzVWfem+tqwYb/Tkj6MI1YBqt6Yy2EsGsoAv1
E5/IGcjSQnLEqDWhpY0s2fA4o+bAMzyakDFKJoQbF982QhS2fT+d87vQlnMi1QID
AQABo1AwTjAdBgNVHQ4EFgQUzqRFDvLX0mz4AjPb45tLGavm8AcwHwYDVR0jBBgw
FoAUzqRFDvLX0mz4AjPb45tLGavm8AcwDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0B
AQsFAAOCAQEAWbRgS1qUyGMh3ToJ060s5cdoKzyx/ji5pRPXRxrmzzSxP+dlKX7h
AKUgYOV9FU/k2f4C7TeCZSsir20x8fKRg4qs6r8vHTcWnkC6A08SNlT5kjyJl8vt
qQTEsemnyBFis8ZFUfYdmNYqZXuWSb7ZBfNkR7qMVna8A87NyEmTtlTBkZYSTOaB
NRuOli+/6akXg/OW/GfVUD11D413CtZsWNzKaxj1WH88mjBYwQx2pGRzMWHfWBka
f6ZUnA9hhqxO4CHqQWmKPHftbGscwx5yg/J6J7TfG+rYd5ZVVhrr2un2xpOTctjO
lriDCQa4FOwP9/x1OJRXEsSl5YFqBppX5A==
-----END CERTIFICATE-----
"""
root_ca_certificate = asymmetric.load_certificate(ROOT_CA)
`
The reason is this check: https://github.com/wbond/oscrypto/blob/master/oscrypto/keys.py#L140
Can this be used with requests? As replacement for the built in SSL lib (currently I'm referring to a Sublime Text 3 plugin context).
I've come here after an all-day debugging session. tl;dr I think oscrypto is picking up the wrong SSL backend for catalina (OS X 10.15.1) which causes a crash.
There's some workarounds as documented here: https://forums.developer.apple.com/thread/119429
Though that seems pretty gnarly.
Specifically I'm finding it failing in _libcrypto_cffi:
libcrypto_path = _backend_config().get('libcrypto_path')
if libcrypto_path is None:
libcrypto_path = find_library('crypto')
if not libcrypto_path:
raise LibraryNotFoundError('The library libcrypto could not be found')
try:
vffi = FFI()
vffi.cdef("const char *SSLeay_version(int type);")
version_string = vffi.string(vffi.dlopen(libcrypto_path).SSLeay_version(0)).decode('utf-8') #### fails here
except (AttributeError):
vffi = FFI()
vffi.cdef("const char *OpenSSL_version(int type);")
version_string = vffi.string(vffi.dlopen(libcrypto_path).OpenSSL_version(0)).decode('utf-8')
I suspect we are just picking a version that makes Catalina extremely upset, and so if we can pick a pinned version that would be better.
I posted something similar here: snowflakedb/snowflake-connector-python#235
today i upgrade to python3.7 with brew. but now i can't run my project and i find the problem was oscrypto.
i find the file(_libcrypto_ctypes.py) line: 258,
libcrypto.EVP_MD_CTX_new.argtypes = []
when run here then raise FFIEngineError('Error initializing ctypes')
and the pic you can see in the libcrypto object, there is no object named EVP_MD_CTX_new.
can you fix it, soon?
when i use python3.6, it runs well
Hi,
While upgrading the Alpine Linux oscrypto package to 1.3.0 I noticed the following test failure:
======================================================================
ERROR: test_load_rsa_pss_cert (tests.test_asymmetric.AsymmetricTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/soeren/src/aports/community/py3-oscrypto/src/oscrypto-1.3.0/tests/test_asymmetric.py", line 143, in test_load_rsa_pss_cert
self.assertEqual(2048, cert.bit_size)
File "/home/soeren/src/aports/community/py3-oscrypto/src/oscrypto-1.3.0/oscrypto/_asymmetric.py", line 216, in bit_size
return self.public_key.bit_size
File "/home/soeren/src/aports/community/py3-oscrypto/src/oscrypto-1.3.0/oscrypto/_asymmetric.py", line 175, in bit_size
return self.asn1.bit_size
File "/usr/lib/python3.10/site-packages/asn1crypto/keys.py", line 1225, in bit_size
self._bit_size = int(math.ceil(math.log(prime, 2)))
UnboundLocalError: local variable 'prime' referenced before assignment
Looking at the tests, the problem seems to be this test case:
oscrypto/tests/test_asymmetric.py
Lines 140 to 143 in 41c21af
This seems to create a key with algorithm rsassa_pss, however, the bit_size() implementation from /usr/lib/python3.10/site-packages/asn1crypto/keys.py does not seem to support this algorithm:
def bit_size(self):
"""
:return:
The bit size of the public key, as an integer
"""
if self._bit_size is None:
if self.algorithm == 'ec':
self._bit_size = int(((len(self['public_key'].native) - 1) / 2) * 8)
else:
if self.algorithm == 'rsa':
prime = self['public_key'].parsed['modulus'].native
elif self.algorithm == 'dsa':
prime = self['algorithm']['parameters']['p'].native
self._bit_size = int(math.ceil(math.log(prime, 2)))
modulus = self._bit_size % 8
if modulus != 0:
self._bit_size += 8 - modulus
return self._bit_sizeThat is, prime is unset if the algorithm is neither rsa nor dsa. As such, I would assume this to be a bug in oscrypto.
I bumped into a situation similar to #7, where calling asymmetric.generate_pair using Python 2.7 and virtualenv fails with OSError: The user name or passphrase you entered is not correct.. The twist is that I wasn't using the system Python 2.7, but one installed through Homebrew.
The issue can be reproduced by initializing and activating a new virtualenv using python2.7 from Homebrew. It should be noted that in this case Homebrew files aren't in the usual /usr/local path, but inside the test user's home directory.
$ which python2.7
/Users/test/homebrew/bin/python2.7
$ python2.7 -m virtualenv test
$ source test/bin/activate
(test) $ pip2.7 install oscrypto
(test) $ python2.7Now calling asymmetric.generate_pair in the virtualenv fails:
>>> from oscrypto import asymmetric
>>> asymmetric.generate_pair("rsa", 4096)
Traceback (most recent call last):
...
OSError: The user name or passphrase you entered is not correct.I suspect the problem is related to lines 57-58 in oscrypto/asymmetric.py that check the sys.real_prefix path. In my scenario real_prefix is set, but doesn't begin with _system_prefix:
>>> import sys
>>> sys.real_prefix
'/Users/test/homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7'When constructing an EC public key from invalid X, Y coordinates (ie. not on the curve), the error handling code in oscrypto/_osx/_core_foundation.py fails due to a NULL c-pointer. Root cause is expecting CFHelpers.cf_string_to_unicode to return None if given a null CFStringRef. It seems to raise an exception instead:
...
File ".../ENV/lib/python3.5/site-packages/oscrypto/_osx/_core_foundation.py", line 43, in handle_cf_error
output = CFHelpers.cf_string_to_unicode(cf_string_ref)
File ".../ENV/lib/python3.5/site-packages/oscrypto/_osx/_core_foundation_cffi.py", line 248, in cf_string_to_unicode
string = ffi.string(string_ptr)
File ".../ENV/lib/python3.5/site-packages/cffi/api.py", line 300, in string
return self._backend.string(cdata, maxlen)
RuntimeError: cannot use string() on <cdata 'char *' NULL>
Here is a snippet of code that demonstrates the issue:
from oscrypto import asymmetric
from binascii import a2b_hex, b2a_hex
from asn1crypto.keys import PublicKeyInfo, PublicKeyAlgorithmId, PublicKeyAlgorithm
from asn1crypto.keys import ECPointBitString, NamedCurve
def test_it():
if 0:
pub_key, _ = asymmetric.generate_pair('ec', curve=u'secp256r1')
x,y = pub_key.asn1['public_key'].to_coords()
else:
x, y = 123, 456
algo = PublicKeyAlgorithm()
algo['algorithm'] = PublicKeyAlgorithmId('ec')
algo['parameters'] = NamedCurve('secp256r1')
pubkey = PublicKeyInfo()
pubkey['algorithm'] = algo
pubkey['public_key'] = ECPointBitString.from_coords(x, y)
pub_key2 = asymmetric.load_public_key(pubkey.dump())I've already patched my copy of this code to catch and handle this specific case, but I wanted to report it anyway. The remainder of the error handling code works and give "Invalid Key" for this case, which seems appropriate.
Here are some version numbers from my machine:
Darwin x.local 15.6.0 Darwin Kernel Version 15.6.0: Mon Aug 29 20:21:34 PDT 2016; root:xnu-3248.60.11~1/RELEASE_X86_64 x86_64
asn1crypto==0.19.0
certbuilder==0.14.2
cffi==1.9.1
idna==2.2
oscrypto==0.17.2
pyasn1==0.1.9
pycparser==2.17
six==1.10.0
Hi,
Would it be possible to support starttls when connecting to servers ?
Some servers require this to start an encrypted connection. (mariadb, dovecot, etc.)
I have now tracked down the test case failure that we are seeing in our environment. The data returned from www.gooole.com:443 contains a HTTP 302 redirecting to www.google.ie and the html contained in the response has all it's HTML tags in caps. This breaks the test case which attempts to read until b'</html>'.
In line 1 and 3, s/assymetric/asymmetric/
Working with a blockchain that stores the 'keystring' in a base64 string when decoded expects 65 bytes where
The first 33 bytes are the public key
The remaining 32 bytes are the private key
So, for a new key generation I know I do:
from oscrypto import asymmetric
public, private = asymmetric.generate_pair('ec', curve='secp256r1')My question is how to get the bytes from the respective keys to b64encode to expected keystring?
Conversely, if I load keystring, decode to bytes... how to instantiate the public and private keys to assert test the round trip?
I'm a newbie but have thick skin :)
Hi, I'm trying to validate certificate using SNI (as I have more than 1 certificate for the same vhost in nginx), but it doesn't seem to work properly:
Python 3.6.8 (default, Apr 2 2020, 13:34:55)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from oscrypto import tls
>>> from certvalidator import CertificateValidator, ValidationContext
>>> session = tls.TLSSession(manual_validation=True)
>>> connection = tls.TLSSocket('10.4.32.37', 443, session=session)
>>> context = ValidationContext(allow_fetching=True)
>>> validator = CertificateValidator(connection.certificate, connection.intermediates, context)
>>> validator.validate_tls('umi-mobile.net')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/data/ludia_default_py36/lib/python3.6/site-packages/certvalidator/__init__.py", line 223, in validate_tls
validate_tls_hostname(self._context, self._certificate, hostname)
File "/opt/data/ludia_default_py36/lib/python3.6/site-packages/certvalidator/validate.py", line 95, in validate_tls_hostname
', '.join(cert.valid_domains)
certvalidator.errors.InvalidCertificateError: The X.509 certificate provided is not valid for umi-mobile.net. Valid hostnames include: *.ludia.net, ludia.net
and openssl says it's ok
openssl s_client -servername umi-mobile.net -connect 10.4.32.37:443 </dev/null
CONNECTED(00000003)
depth=2 C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust RSA Certification Authority
verify return:1
depth=1 C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo RSA Domain Validation Secure Server CA
verify return:1
depth=0 CN = *.umi-mobile.net
verify return:1
---
Certificate chain
0 s:/CN=*.umi-mobile.net
i:/C=GB/ST=Greater Manchester/L=Salford/O=Sectigo Limited/CN=Sectigo RSA Domain Validation Secure Server CA
1 s:/C=GB/ST=Greater Manchester/L=Salford/O=Sectigo Limited/CN=Sectigo RSA Domain Validation Secure Server CA
i:/C=US/ST=New Jersey/L=Jersey City/O=The USERTRUST Network/CN=USERTrust RSA Certification Authority
2 s:/C=US/ST=New Jersey/L=Jersey City/O=The USERTRUST Network/CN=USERTrust RSA Certification Authority
i:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
---
...
Verify return code: 0 (ok)
---
DONE
What am I doing wrong here ?
OpenSSL 1.0.x and CNG support AES GCM mode.
While not publicly documented on the Apple developer documentation, it appears that AES GCM is available via http://www.opensource.apple.com/source/CommonCrypto/CommonCrypto-60061/lib/CommonCryptorGCM.c. We need to figure out if these functions are exposed in /usr/lib/system/libcommonCrypto.dylib and for what versions of OS X they are available.
Unfortunately OpenSSL 0.9.8 does not support AES GCM, so there will never be a way to support it there. However, most Linux distros are on OpenSSL 1.0.x now, and OS X has its own native libraries.
I am on macOS Catalina and Python 3.7. While on Conda base, the application runs without errors. However, running it in a Conda virtual environment results in the following error.
Traceback (most recent call last):
File "/Users/charles.siu/git/domain_classifier_real_estate/data_collector/server.py", line 7, in <module>
from data_collector.domain import parse_train_domains, parse_predict_domains
File "/Users/charles.siu/git/domain_classifier_real_estate/data_collector/domain.py", line 7, in <module>
from data_collector.cert import get_cert, get_cert_issuer
File "/Users/charles.siu/git/domain_classifier_real_estate/data_collector/cert.py", line 6, in <module>
import oscrypto.asymmetric
File "/usr/local/anaconda3/envs/domain_classifier/lib/python3.7/site-packages/oscrypto/asymmetric.py", line 19, in <module>
from ._asymmetric import _unwrap_private_key_info
File "/usr/local/anaconda3/envs/domain_classifier/lib/python3.7/site-packages/oscrypto/_asymmetric.py", line 27, in <module>
from .kdf import pbkdf1, pbkdf2, pkcs12_kdf
File "/usr/local/anaconda3/envs/domain_classifier/lib/python3.7/site-packages/oscrypto/kdf.py", line 9, in <module>
from .util import rand_bytes
File "/usr/local/anaconda3/envs/domain_classifier/lib/python3.7/site-packages/oscrypto/util.py", line 10, in <module>
from ._mac.util import rand_bytes
File "/usr/local/anaconda3/envs/domain_classifier/lib/python3.7/site-packages/oscrypto/_mac/util.py", line 208, in <module>
from .._openssl._libcrypto import libcrypto
File "/usr/local/anaconda3/envs/domain_classifier/lib/python3.7/site-packages/oscrypto/_openssl/_libcrypto.py", line 15, in <module>
from ._libcrypto_ctypes import (
File "/usr/local/anaconda3/envs/domain_classifier/lib/python3.7/site-packages/oscrypto/_openssl/_libcrypto_ctypes.py", line 37, in <module>
libcrypto = CDLL(libcrypto_path, use_errno=True)
File "/usr/local/anaconda3/envs/domain_classifier/lib/python3.7/ctypes/__init__.py", line 364, in __init__
self._handle = _dlopen(self._name, mode)
OSError: dlopen(/usr/local/anaconda3/envs/domain_classifier/bin/../lib/libcrypto.42.dylib, 6): image not foundI have tried symbolic linking /usr/lib to ../lib but it seemed to have created more issues. Any help is appreciated!
I can see functions to generate EC keys, but not one to calculate the ECDH shared secret. Am I missing it or does it need to be added?
Thanks
Package Control will be switching to use oscrypto.trust_list for TLS trust roots to solve:
In order to do this, we need to tweak trust_list.get_list(). Ideally we need:
get_list() by one or more key usage strings/extended key usage OIDsget_list() that will return the list os trust roots along with a set() of their key usage names and extended key usage OIDsI'm in the process of implementing PDF 2.0 encryption, which is AES-256 based. At some point, the algorithm requires encrypting a specific 16-byte string using AES-256, without padding against an all-zero IV (so it's effectively one block of unpadded AES-256 in ECB mode).
This is the relevant snippet:
perms_bytes = b'\xfc\xff\xff\xff'
extd_perms_bytes = (
perms_bytes + (b'\xff' * 4) + b'Tadb' + secrets.token_bytes(4)
)
perms = struct.unpack('<i', perms_bytes)[0]
_, encrypted_perms = symmetric.aes_cbc_no_padding_encrypt(
encryption_key, extd_perms_bytes, bytes(16)
)
My tests pass on my macOS machine, but the OpenSSL-based implementation used on Linux throws an error here.
2021-01-14T23:32:31.2930994Z if cipher != 'rc4' and not padding:
2021-01-14T23:32:31.2931561Z # AES in CBC mode can be allowed with no padding if
2021-01-14T23:32:31.2932148Z # the data is an exact multiple of the key size
2021-01-14T23:32:31.2932655Z aes128_no_padding = (
2021-01-14T23:32:31.2933212Z cipher == 'aes128' and
2021-01-14T23:32:31.2933653Z padding is False and
2021-01-14T23:32:31.2934052Z len(data) % 16 == 0
2021-01-14T23:32:31.2934382Z )
2021-01-14T23:32:31.2934762Z aes192_no_padding = (
2021-01-14T23:32:31.2935390Z cipher == 'aes192' and
2021-01-14T23:32:31.2935779Z padding is False and
2021-01-14T23:32:31.2936133Z len(data) % 24 == 0
2021-01-14T23:32:31.2936436Z )
2021-01-14T23:32:31.2936747Z aes256_no_padding = (
2021-01-14T23:32:31.2937334Z cipher == 'aes256' and
2021-01-14T23:32:31.2937728Z padding is False and
2021-01-14T23:32:31.2938079Z len(data) % 32 == 0
2021-01-14T23:32:31.2938379Z )
2021-01-14T23:32:31.2938890Z if aes128_no_padding is False and aes192_no_padding is False and aes256_no_padding is False:
2021-01-14T23:32:31.2939704Z > raise ValueError('padding must be specified')
2021-01-14T23:32:31.2940252Z E ValueError: padding must be specified
So correct me if I'm wrong, but shouldn't the correct requirement be to force the data length to be a multiple of the block size (i.e. 16 bytes), and not the key size (32 bytes)? I'm happy to submit a pull request rectifying that, but I wanted to get a second opinion first.
EDIT: thanks a lot for your efforts, by the way, I've been using your crypto suite for quite some time now, it's really been a tremendous asset!
It does not seem logical to me to get this error
TypeError: certificate_or_public_key must be an instance of the Certificate or PublicKey class, not asn1crypto.keys.PublicKeyInfo
When trying to do this:
info = cms.ContentInfo.load(f.read())
pubkey = info['content']['certificates'][0].chosen.public_key
asymmetric.rsa_pkcs1v15_verify(pubkey, signature_value, signed_data, 'sha256')
But maybe I am wrong (and I cannot find an easy way to convert PublicKeyInfo to PublicKey ....)
Using Oscrypto 1.3.0 on a mac M1, I am getting a really strange error when I open a TLSSocket. Something I am doing wrong here? This works on an Intel Mac
Python 3.9.12 (main, Mar 26 2022, 15:44:31)
[Clang 13.1.6 (clang-1316.0.21.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from oscrypto import tls
>>> t=tls.TLSSession(manual_validation=True)
>>> c = tls.TLSSocket('www.google.com', 443, session=t)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/vivekv/Library/Caches/pypoetry/virtualenvs/digital-resilience-pmVk26sL-py3.9/lib/python3.9/site-packages/oscrypto/_mac/tls.py", line 532, in __init__
self._handshake()
File "/Users/vivekv/Library/Caches/pypoetry/virtualenvs/digital-resilience-pmVk26sL-py3.9/lib/python3.9/site-packages/oscrypto/_mac/tls.py", line 654, in _handshake
cipher_suite = int_to_bytes(supported_cipher_suite, width=2)
File "/Users/vivekv/Library/Caches/pypoetry/virtualenvs/digital-resilience-pmVk26sL-py3.9/lib/python3.9/site-packages/asn1crypto/util.py", line 243, in int_to_bytes
return value.to_bytes(width, byteorder='big', signed=signed)
OverflowError: int too big to convert
>>>
It would be nice to have support of Path objects for filenames.
As I read
oscrypto/oscrypto/_openssl/tls.py
Line 192 in 66ba999
oscrypto/oscrypto/_openssl/tls.py
Line 208 in 66ba999
a provided "trust_list_path" only gets applied on windows and mac, while on
other platforms trusted certificates are (only) loaded from the system default
via SSL_CTX_set_default_verify_paths(ssl_ctx) .
Later on, "extra_trust_roots" can be added, but this won't
override/replace/forget already loaded certificates from the system default.
According to
https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_default_verify_paths.html
this result might be influenced by setting environment variables.
To be able to ignore/tighten the systems trust settings without fiddling with
the environment, I tried to use the "trust_list_path" feature
for openssl on linux the same way it seems to be already possible
on windows and mac, with patch @
Is my described understanding correct?
If yes, is there a reason why "trust_list_path" should have an effect on some
platforms, but not on others?
If the second answer is "no",
might upstream be interested in taking this patch or something similar?
I updated my project to use oscrypto 1.3.0 to support Ubuntu 22.04 with Openssl 3, but I'm running into an issue.
If I'm reading the commit message of 5ae0e79 correctly, then oscrypto should support the legacy mode of openssl. However, my application fails when calling parse_pkcs12, and raises an exception in rc2_cbc_pkcs5_decrypt with the message "OpenSSL has been compiled without RC2 support".
Is there anything I can do from within Python / with oscrypto to get around that and have it still use RC2, or can I somehow enable this RC2 support in OpenSSL?
I believe applications do need to actively request this legacy provider, does oscrypto do that?
Looking at this code in _libcrypto.py:
# This enables legacy algorithms in OpenSSL 3.0, such as RC2, etc
# which are used by various tests and some old protocols and things
# like PKCS12
libcrypto_legacy_support = True
if libcrypto_version_info >= (3, ):
if libcrypto.OSSL_PROVIDER_available(null(), "legacy".encode("ascii")):
libcrypto.OSSL_PROVIDER_load(null(), "legacy".encode("ascii"))
else:
libcrypto_legacy_support = Falsethis might already be implemented if I'm reading this correctly, but why doesn't it work then?
Can I somehow test if Ubuntu 22.04 actually ships completely without this legacy module?
Hi,
currently only secp256r1 seems to be supported among the 256-bit EC curves. Windows' certreq utility offers brainpoolp256r1 by default. Would it be much of an effort to add it to oscrypto?
To implement the https://github.com/spark/firmware protocol, I need to perform a signature with PKCS15 padding, but without applying a hash.
Thanks you!
https://developer.apple.com/documentation/network seems to be the supported API set these days. The older https://developer.apple.com/documentation/security/secure_transport APIs are marked legacy / deprecated.
What does the future for macOS support look like if secure transport is assumed to go away?
Twisted has long desired something just like oscrypto, so that users get their platform's trust settings by default. oscrypto provides something like this.
However, many of the APIs in oscrypto are unfortunately hard-coded to do I/O in in a blocking way, as well as in places like constructors which makes it extra tricky to interpose.
It would be neat if oscrypto could expose a buffer-like (as opposed to socket-like) interface for TLS where I could just drop in some encrypted bytes from the wire and get back some authenticated / verified plaintext bytes, and vice versa.
When loading certain public keys through oscrypto.asymetric.load_certificate then trying to inspect any property (say algorithm) an OSError such as OSStatus 62339392 is thrown. The exact error code is always different but undefined within Apple docs.
The error appears to be coming from SecCertificateCopyPublicKey(_:) which I note has been deprecated for a long time now in favour of SecCertificateCopyKey(_:)
System Version: macOS 13.0 (22A5321d)
Kernel Version: Darwin 22.0.
Example cert:
-----BEGIN CERTIFICATE-----
MIIDuTCCAqGgAwIBAgIQfCoOUTBOfHengHo4iS1DmjANBgkqhkiG9w0BAQsFADBM
MSAwHgYDVQQLExdHbG9iYWxTaWduIFJvb3QgQ0EgLSBSMzETMBEGA1UEChMKR2xv
YmFsU2lnbjETMBEGA1UEAxMKR2xvYmFsU2lnbjAeFw0yMjA4MDcwMDAwMDBaFw0y
MjExMTUwMDAwMDBaMFsxCzAJBgNVBAYTAkJFMRkwFwYDVQQKExBHbG9iYWxTaWdu
IG52LXNhMTEwLwYDVQQDEyhHbG9iYWxTaWduIE9DU1AgZm9yIFJvb3QgUjMgLSBT
aWduZXIgMS4yMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArrYtuWRO
QRqEM8cSyLzpgbIVeZ7n/dG7DIpGHcl4SBo99ji0IEFab/Mi7k2ML9hTQ8jaWUPa
zllvmpMrKqzU/nobvcq5XGewtXPY8XNQfHDch9ruqI5Xii/mxCfX0dxbP5pHOm2Y
FfJe+ObYHpZy4jtifd3GtYlGODnfSkrL+s6NwZX3b4pAZVt7MWjKg3UQI5W3hskO
zhAdOmhWacMeM4HeyHCwMz/S9caQ2v+g8ZC8eTQoeqfGzknvMNl9GbipED4CWYgG
++4cKYmIbXlgdOm1I6jDtVjWjmXahQKYGh/EC39wlVukWoUhtSfE6oK3TXq5hdg3
JXcAlu970y0ErQIDAQABo4GHMIGEMA4GA1UdDwEB/wQEAwIHgDATBgNVHSUEDDAK
BggrBgEFBQcDCTAMBgNVHRMBAf8EAjAAMB0GA1UdDgQWBBQpps1+5UkBfCYf2+XI
r2Cd+Tw2fjAfBgNVHSMEGDAWgBSP8Et/qC5FJK5NUPpjmove4t0bvDAPBgkrBgEF
BQcwAQUEAgUAMA0GCSqGSIb3DQEBCwUAA4IBAQDEdJsZuNVxwdR4jSluEWXmVW16
RDarknBhHxd8Cq1q4FPfN+BcltDoBUDISEvOHffKm1VPCjOTxvWuvSd630u6TQSn
9Uut/IrjeAnq1eRrEkdDE09772G/Q5tfBHGB+cZdFqICv94YqGHlT1f5tNWfTmV6
/3wd86AxrBOWcYnd4HgXBid5kBiRoFcP4v8z94A9zuPsZDLmHrbrqqLH4zi9Z+ak
YMCBTfYQUAJuT/kUbGX4PuxDYT3CnYW54ukUrfmsWHtSUA/cUGY+WlyNxG2y+cD+
3GgZxwTQpy7ik8wj1McBYh6X99Nud8D29Hfkj9YtBIqrjPXUM9F2ik+x0u3l
-----END CERTIFICATE-----
Minimum reproducing code:
import asn1crypto.pem
import asn1crypto.x509
import oscrypto.asymmetric
with open('cert.pem', 'rb') as f:
pem_bytes = f.read()
_, _, der_bytes = asn1crypto.pem.unarmor(pem_bytes)
cert = asn1crypto.x509.Certificate.load(der_bytes)
key_object = oscrypto.asymmetric.load_certificate(cert)
print(key_object.algorithm)Instead of defining the struct contents, which are liable to change, sk_num() and sk_value() should be used instead.
OpenSSL 0.9.8y:
int sk_num(const STACK *);
char *sk_value(const STACK *, int);
OpenSSL 1.1.0:
int sk_num(const _STACK *);
void *sk_value(const _STACK *, int);
Not exactly a bug, more a suggestion.
Using windows, if you try to verify a 2048 bit signature with a 4096 bit key (or vice versa I assume), you get this back:
OSError: NTSTATUS error 0xC000000D: An invalid parameter was passed to a service or function
From a usability standpoint I think this condition should raise [for example] SignatureError('Signature length does not match RSA key size') instead of sending back an OSError or a ValueError for algorithm mismatch.
As a caller, I want to answer "Can I verify this signature with this public key (certificate)?" Assuming I pass in valid objects, I think it makes sense to get a "No" response (e.g. SignatureError) back and save the ValueError and OSError for stuff that can't be handled a gracefully like "I can't verify this because I don't support MD2"
For example, in CertValidator - validate.py, _self_signed(cert), if the algorithm or key size is not matched to the signature in the certificate then this error will pop past your oscrypto.errors.SignatureError catch block:
try:
key_object = asymmetric.load_certificate(cert)
verify_func(key_object, cert['signature_value'].native, cert['tbs_certificate'].dump(), hash_algo)
return True
except (oscrypto.errors.SignatureError):
return False
The _validate_path function has the same issue and could realistically expose this if the caller didn't ensure they were doing the algorithm matching and key size checks during path building.
It seems there are several ways you might address handling these conditions, including saying you want those errors to propagate back up and doing nothing! However, if you do agree that this should be addressed somewhere, I imagine you'd prefer the fix not be per-platform. That said, the particular case that brought this to my attention was mismatched rsa key/sig size on Windows. I noted that if this code were changed
failure = res == BcryptConst.STATUS_INVALID_SIGNATURE
failure = failure or (rsa_pss_padding and res == BcryptConst.STATUS_INVALID_PARAMETER)
to
failure = res == BcryptConst.STATUS_INVALID_SIGNATURE or res == BcryptConst.STATUS_INVALID_PARAMETER`
that it would give the desired SignatureError answer for this particular case. Given you're already abstracting away STATUS_INVALID_PARAMETER in the rsa_pss_padding case perhaps it makes sense to simply always do so as it might pop up for other unimagined conditions? Obviously the better solution for my exact use case would be to never get to that line and raise SignatureError('Signature length does not match RSA key size') farther up the stack.
Thoughts?
https://docs.python.org/3/library/hashlib.html
Of course sha3 and blake2 are safer than sha2 and md5. But I also noticed they were added recently in python3.6, while oscrypto supports python2. So I'm not sure about whether we should support them right now.
I intend to use a TPM on Windows to generate, store and sign, through the win32 CNG API (NCryptCreatePersistedKey and NCryptSignHash) but within a Python program. When creating a key, one have to provide the hProvider (handle the Key Storage Provider) parameter as "Microsoft Platform Crypto Provider" to select the TPM target.
oscrypto uses CNG but the oscrypto methods are too much high-level and a key generation gives the keypair data, and there's no easy way to select the KSP. Do you have any idea how to proceed in Python for what I have in mind ? Can I use the brcrypt ffi (from bcrypt.dll) in an easy way to use the win32 method ? I guess I have to add the ffi cdef in _cng_cffi about BCryptCreatePersistedKey ?
Thank you for providing the modularcrypto tools!
To get access to the verified certificate chain of TLS connections,
I tried to edit oscrypto for openssl >= 1.1.0 with
What are the chances to see something similar in your upstream code?
I'm having trouble on MacOS 10.15.3 with this line and version 1.1.1/1.2.0 on python 3.6.8
from oscrypto import asymmetric
I'm getting the following error:
dlopen(/usr/local/lib/libcrypto.42.dylib, 6): image not found
It does appear to be in/usr/lib
Hi,
I am trying to run oscrypto (latest pip install) inside a virtualenv on OSX 10.10 (python2.7).
As a test one-liner:
python -c "import oscrypto.asymmetric as asym;asym.generate_pair('rsa', 1024)"and end up with:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "env/lib/python2.7/site-packages/oscrypto/_osx/asymmetric.py", line 388, in generate_pair
handle_sec_error(result)
File "env/lib/python2.7/site-packages/oscrypto/_osx/_security.py", line 47, in handle_sec_error
raise exception_class(output)
OSError: Noneany ideas or reasons, why this wouldn't work?
Hello! Looks like we're using your software, so thanks before anything else.
I have been looking into this series of warnings thrown when running some python commands, which complained about several global variables not having the extern keyword. I'm adding the warnings below.
I finally troubleshooted it down to your code, and temporarily fixed it by the obvious solution of adding the keyword where necessary.
My question is, do you think we should actually use extern here? I am not knowledgable about this domain, so any kind of information would be helpful. I'd be happy to open a PR.
Files affected:
_mac/_security_cffi.py
_mac/_core_foundation_cffi.pyThanks!
The warning avalanche:
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecPaddingKey' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecPaddingPKCS7Key' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecPaddingPKCS5Key' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecPaddingPKCS1Key' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecPaddingOAEPKey' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecPaddingNoneKey' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecModeCBCKey' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecTransformInputAttributeName' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecDigestTypeAttribute' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecDigestLengthAttribute' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecIVKey' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecAttrIsExtractable' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecDigestSHA1' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecDigestSHA2' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecDigestMD5' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecAttrKeyType' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecAttrKeyTypeRSA' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecAttrKeyTypeDSA' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecAttrKeyTypeECDSA' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecAttrKeySizeInBits' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecAttrLabel' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecAttrCanSign' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecAttrCanVerify' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecAttrKeyTypeAES' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecAttrKeyTypeRC4' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecAttrKeyTypeRC2' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecAttrKeyType3DES' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecAttrKeyTypeDES' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kCFAllocatorDefault' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kCFTypeArrayCallBacks' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kCFBooleanTrue' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kCFTypeDictionaryKeyCallBacks' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/Users/erinc/.virtualenvs/findapro/lib/python3.6/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kCFTypeDictionaryValueCallBacks' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))(Sorry for the clickbaity title, I couldn't resist)
No compiler needed, ever. … Just distribute the Python source files, any way you want.
This is not correct on Linux systems. If you check the docs for ctypes.util.find_library(), it says
On Linux, find_library() tries to run external programs (/sbin/ldconfig, gcc, objdump and ld) to find the library file.
Which means, not only does it need a compiler, it needs it on runtime! You cannot use oscrypto on a system without a full development environment, which kind of defeats the purpose of a library.
The only way to make it work on a production system is calling use_openssl() to pass hard coded paths early in your app, which also is somewhat of a suboptimal hack.
I think oscrypto should call find_library() while installing and cache the openssl/libcrypto paths somehow, so that an install from a wheel package will work without further shenanigans?
The most complicated part of this is testing. We need to find a reliable way to test various scenarios including:
Ideally we would use something like tls-o-matic.com, however they do not provide good and bad client certs to test with.
I have no interest in creating a VM for this since it will complicate testing.
I'm fooling around with the daily builds of the upcoming (22.04) Ubuntu release.
Two things to know about 22.04 are:
libssl3 installed and defaulted.libcrypto.so.3 has a different symbol table than libcrypto.so.1.1.I'm not sure if 2 is an upstream bug or not, but I thought I'd give oscrypto notice insofar as your docs only mention <= OpenSSL 1.1.
The specific traceback I'm seeing is:
File "/home/.../lib/python3.9/site-packages/certbuilder/__init__.py", line 904, in build
signature = sign_func(signing_private_key, tbs_cert.dump(), self._hash_algo)
File "/home/.../lib/python3.9/site-packages/oscrypto/_openssl/asymmetric.py", line 1494, in rsa_pkcs1v15_sign
return _sign(private_key, data, hash_algorithm)
File "/home/.../lib/python3.9/site-packages/oscrypto/_openssl/asymmetric.py", line 1809, in _sign
buffer_size = libcrypto.EVP_PKEY_size(private_key.evp_pkey)
File "/home/.../lib/python3.9/site-packages/cffi/api.py", line 912, in __getattr__
make_accessor(name)
File "/home/.../lib/python3.9/site-packages/cffi/api.py", line 908, in make_accessor
accessors[name](name)
File "/home/.../lib/python3.9/site-packages/cffi/api.py", line 838, in accessor_function
value = backendlib.load_function(BType, name)
AttributeError: function/symbol 'EVP_PKEY_size' not found in library 'libcrypto.so.3': /lib/x86_64-linux-gnu/libcrypto.so.3: undefined symbol: EVP_PKEY_size
test_tls_connect_revoked in tests/test_tls.py fails with
oscrypto.errors.TLSVerificationError: Server certificate verification failed - certificate expired 2020-04-13 12:00:00Z
Apparently the certificate for revoked.grc.com is expired.
OpenBSD (-current, snapshot 2016-06-14) using Python 3.6 from Ports is missing Python cffi package. I didn't test other Python versions, but let me know if it's needed.
Should oscrypto depend on cffi and install it if missing? This helps users who install Python packages with pip install.
Workaround (or solution) for this issue in OpenBSD is to install py3-cffi package from Ports. However, hunting missing Python packages from ports is not solution for users without root access.
When cffi is missing, the user gets following traceback:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/oscrypto/_openssl/_libcrypto_cffi.py", line 13, in <module>
from cffi import FFI
ModuleNotFoundError: No module named 'cffi'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/oscrypto/_openssl/_libcrypto.py", line 8, in <module>
from ._libcrypto_cffi import (
File "/usr/local/lib/python3.6/site-packages/oscrypto/_openssl/_libcrypto_cffi.py", line 16, in <module>
raise FFIEngineError('Error importing cffi')
oscrypto._ffi.FFIEngineError: Error importing cffi
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/oscrypto/_openssl/_libcrypto_ctypes.py", line 258, in <module>
libcrypto.EVP_MD_CTX_new.argtypes = []
File "/usr/local/lib/python3.6/ctypes/__init__.py", line 361, in __getattr__
func = self.__getitem__(name)
File "/usr/local/lib/python3.6/ctypes/__init__.py", line 366, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: Unable to resolve symbol
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/trytls", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python3.6/site-packages/trytls/runner.py", line 177, in main
bundle = bundles.load_bundle(bundle_name)
File "/usr/local/lib/python3.6/site-packages/trytls/bundles/__init__.py", line 11, in load_bundle
return entry.load()
File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2268, in load
return self.resolve()
File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2274, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/usr/local/lib/python3.6/site-packages/trytls/bundles/https.py", line 15, in <module>
from ..gencert import gencert
File "/usr/local/lib/python3.6/site-packages/trytls/gencert.py", line 3, in <module>
from oscrypto import asymmetric
File "/usr/local/lib/python3.6/site-packages/oscrypto/asymmetric.py", line 15, in <module>
from .kdf import pbkdf2, pbkdf2_iteration_calculator
File "/usr/local/lib/python3.6/site-packages/oscrypto/kdf.py", line 9, in <module>
from .util import rand_bytes
File "/usr/local/lib/python3.6/site-packages/oscrypto/util.py", line 14, in <module>
from ._openssl.util import rand_bytes
File "/usr/local/lib/python3.6/site-packages/oscrypto/_openssl/util.py", line 6, in <module>
from ._libcrypto import libcrypto, libcrypto_version_info, handle_openssl_error
File "/usr/local/lib/python3.6/site-packages/oscrypto/_openssl/_libcrypto.py", line 14, in <module>
from ._libcrypto_ctypes import (
File "/usr/local/lib/python3.6/site-packages/oscrypto/_openssl/_libcrypto_ctypes.py", line 668, in <module>
raise FFIEngineError('Error initializing ctypes')
oscrypto._ffi.FFIEngineError: Error initializing ctypes
For cross referencing this issue is also in TryTLS repo: ouspg/trytls#307
Setup: macOS 12.1, oscrypto 1.2.1, Python 3.9.5
Not sure where to even start to look for this problem. Running the newest mac version on arm and not Intel. The OSStatus 62385568 did not give me any clues.
/Users/xxx/.local/share/virtualenvs/QpmVhfBc/lib/python3.9/site-packages/oscrypto/_mac/asymmetric.py:1027: in rsa_pkcs1v15_encrypt
key_length = certificate_or_public_key.byte_size
/Users/xxx/.local/share/virtualenvs/QpmVhfBc/lib/python3.9/site-packages/oscrypto/_asymmetric.py:225: in byte_size
return self.public_key.byte_size
/Users/xxx/.local/share/virtualenvs/QpmVhfBc/lib/python3.9/site-packages/oscrypto/_mac/asymmetric.py:255: in public_key
handle_sec_error(res)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
error = 62385568, exception_class = <class 'OSError'>
> raise exception_class(output)
E OSError: OSStatus 62385568
/Users/xxx/.local/share/virtualenvs/QpmVhfBc/lib/python3.9/site-packages/oscrypto/_mac/_security.py:57: OSError
We are facing issue with Mac OS High Sierra 10.13.2 (17C88).
We are using oscrypto library which internally uses openssl of OS. While accessing it we are receiving following error.
We don't receive below error in sierra operating system of MAC
Traceback (most recent call last):
File "/Users/rodrigo.velez/First_steps/deeplaser-core/deeplaser-core-env/lib/python2.7/site-packages/pybuilder/cli.py", line 413, in main
environments=options.environments, tasks=arguments)
File "/Users/rodrigo.velez/First_steps/deeplaser-core/deeplaser-core-env/lib/python2.7/site-packages/pybuilder/reactor.py", line 128, in build
return self.build_execution_plan(tasks, execution_plan)
File "/Users/rodrigo.velez/First_steps/deeplaser-core/deeplaser-core-env/lib/python2.7/site-packages/pybuilder/reactor.py", line 166, in build_execution_plan
reactor=self)
File "/Users/rodrigo.velez/First_steps/deeplaser-core/deeplaser-core-env/lib/python2.7/site-packages/pybuilder/execution.py", line 351, in execute_execution_plan
summaries.append(self.execute_task(task, **keyword_arguments))
File "/Users/rodrigo.velez/First_steps/deeplaser-core/deeplaser-core-env/lib/python2.7/site-packages/pybuilder/execution.py", line 298, in execute_task
task.execute(self.logger, keyword_arguments)
File "/Users/rodrigo.velez/First_steps/deeplaser-core/deeplaser-core-env/lib/python2.7/site-packages/pybuilder/execution.py", line 169, in execute
executable.execute(argument_dict)
File "/Users/rodrigo.velez/First_steps/deeplaser-core/deeplaser-core-env/lib/python2.7/site-packages/pybuilder/execution.py", line 100, in execute
self.callable(*arguments)
File "/Users/rodrigo.velez/First_steps/deeplaser-core/deeplaser-core-env/lib/python2.7/site-packages/pybuilder/plugins/python/unittest_plugin.py", line 61, in run_unit_tests
run_tests(project, logger, "unittest", "unit tests")
File "/Users/rodrigo.velez/First_steps/deeplaser-core/deeplaser-core-env/lib/python2.7/site-packages/pybuilder/plugins/python/unittest_plugin.py", line 71, in run_tests
project, logger, execution_prefix, execution_name))
File "/Users/rodrigo.velez/First_steps/deeplaser-core/deeplaser-core-env/lib/python2.7/site-packages/pybuilder/utils.py", line 323, in fork_process
raise_exception(result[1], result[2])
File "/Users/rodrigo.velez/First_steps/deeplaser-core/deeplaser-core-env/lib/python2.7/site-packages/pybuilder/utils.py", line 305, in instrumented_target
send_value = (target(*args, **kwargs), None, None)
File "/Users/rodrigo.velez/First_steps/deeplaser-core/deeplaser-core-env/lib/python2.7/site-packages/pybuilder/plugins/python/unittest_plugin.py", line 101, in do_run_tests
test_method_prefix)
File "/Users/rodrigo.velez/First_steps/deeplaser-core/deeplaser-core-env/lib/python2.7/site-packages/pybuilder/plugins/python/unittest_plugin.py", line 136, in execute_tests_matching
tests = loader.loadTestsFromNames(test_modules)
File "/Users/rodrigo.velez/.pyenv/versions/2.7.14/lib/python2.7/unittest/loader.py", line 130, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "/Users/rodrigo.velez/.pyenv/versions/2.7.14/lib/python2.7/unittest/loader.py", line 91, in loadTestsFromName
module = __import__('.'.join(parts_copy))
File "/Users/rodrigo.velez/First_steps/deeplaser-core/src/unittest/python/services/test_certificate_service.py", line 13, in <module>
from deeplaser.certificate.crypto.standard import TokenGeneratorOsCrypto
File "/Users/rodrigo.velez/First_steps/deeplaser-core/src/main/python/deeplaser/certificate/crypto/standard.py", line 11, in <module>
from oscrypto import asymmetric
File "/Users/rodrigo.velez/First_steps/deeplaser-core/deeplaser-core-env/lib/python2.7/site-packages/oscrypto/asymmetric.py", line 15, in <module>
from .kdf import pbkdf2, pbkdf2_iteration_calculator
File "/Users/rodrigo.velez/First_steps/deeplaser-core/deeplaser-core-env/lib/python2.7/site-packages/oscrypto/kdf.py", line 9, in <module>
from .util import rand_bytes
File "/Users/rodrigo.velez/First_steps/deeplaser-core/deeplaser-core-env/lib/python2.7/site-packages/oscrypto/util.py", line 10, in <module>
from ._osx.util import rand_bytes
File "/Users/rodrigo.velez/First_steps/deeplaser-core/deeplaser-core-env/lib/python2.7/site-packages/oscrypto/_osx/util.py", line 208, in <module>
from .._openssl._libcrypto import libcrypto
File "/Users/rodrigo.velez/First_steps/deeplaser-core/deeplaser-core-env/lib/python2.7/site-packages/oscrypto/_openssl/_libcrypto.py", line 14, in <module>
from ._libcrypto_ctypes import (
File "/Users/rodrigo.velez/First_steps/deeplaser-core/deeplaser-core-env/lib/python2.7/site-packages/oscrypto/_openssl/_libcrypto_ctypes.py", line 668, in <module>
raise FFIEngineError('Error initializing ctypes')
FFIEngineError: Error initializing ctypes
------------------------------------------------------------
BUILD FAILED - Error initializing ctypes
------------------------------------------------------------
I am searching for a way to use the TPM2 module to store the private keys of certificates. The idea is to use the TPM2 as secure key store for an own CA. The problem I ran into is that there is actually no implementation of that which is platform independent.
So I got a way using Linux and the openssl-tpm2 provider. The problem is that compiling this for Windows is not an easy task, especially because of the hardware interface to the TPM2 module. So I found that the "correct" way on windows would be to use the CNG for that purposey since it handles that internally (If I got that right in the Microsoft documentation).
Now I come to the point at which I need to have an integration with python, so I found this repository. So far it looks like exactly what I need, except the integration of the TPM2 in openssl on Linux.
Is it possible to integrate the tpm provider into this library to become the 'first' platform independent crypto library with TPM support?
I'm trying to call oscrypto.symmetric.aes_cbc_no_padding_encrypt with a 16 byte key, 16 byte IV, and 64 bytes of data.
This throws an error:
>>> import oscrypto.symmetric
>>>
>>> key = [0] * 16
>>> iv = [1] * 16
>>> data = [255] * 64
>>>
>>> oscrypto.symmetric.aes_cbc_no_padding_encrypt(key=bytes(key), data=bytes(data), iv=bytes(iv))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "oscrypto/_openssl/symmetric.py", line 74, in aes_cbc_no_padding_encrypt
return (iv, _encrypt(cipher, key, data, iv, False))
File "oscrypto/_openssl/symmetric.py", line 606, in _encrypt
raise ValueError('padding must be specified')
ValueError: padding must be specifiedHere's the abridged source of oscrypto._openssl.symmetric:
def aes_cbc_no_padding_encrypt(key, data, iv):
cipher = _calculate_aes_cipher(key)
return (iv, _encrypt(cipher, key, data, iv, False))
def _calculate_aes_cipher(key):
if len(key) == 16:
return 'aes128'
def _encrypt(cipher, key, data, iv, padding):
if cipher != 'rc4' and not padding:
raise ValueError('padding must be specified')
# ...
As I read it, aes_cbc_no_padding_encrypt always passes cipher != rc4 and padding = False to _encrypt, resulting in the exception.
Happy to help with a fix, just wanted to check I'm not doing anything crazy?
Loading a certificate which is signed using RSASSA PSS fails on Windows. It works using asn1crypto therefore I assume it's not a limitation of the Windows CNG API.
The following works on Linux but fails on Windows 10.
#!/usr/bin/env python3
from asn1crypto import pem, x509
from oscrypto import asymmetric
cert_string = ("""-----BEGIN CERTIFICATE-----
MIIDwzCCAnqgAwIBAgIUcLiHLd6kIhTmbL3akxJhqM5mbbUwPgYJKoZIhvcNAQEK
MDGgDTALBglghkgBZQMEAgGhGjAYBgkqhkiG9w0BAQgwCwYJYIZIAWUDBAIBogQC
AgDeMBYxFDASBgNVBAMMC0NhcmxQU1MyMDQ4MCAXDTIwMDMyNDExMTIxNVoYDzIw
NjAwMzE0MTExMjE1WjAXMRUwEwYDVQQDDAxBbGljZVBTUzIwNDgwggEgMAsGCSqG
SIb3DQEBCgOCAQ8AMIIBCgKCAQEAwZK+biuFQF+z0IZML9BJOg2J8LbmouPojVwT
Qez11nyVPR+AFX5aak9r2yq7c+pfSN7Jdvka0zrOsEjvxZYVJQgzpWnAJzjHUHE+
PEmWpxI5pDOg1HmtlFOWkfrxqoX2iSwFXeEQ3MQ6xChWqfSbhzkkzSTHJGdUfSz0
YDMFX3Np0CG0yC/s7pqSErGq90fbkJfF+HIsxu1R79ps2YFY8ruBBzb8UOfZ1hEG
0PJVm13MmuJOZ7UkzpcVT2l9qbi/LHjpVgkBgOKlnOrtuLx5kXPX377BOYGiV6z6
CA/Edw50uaYbtK4XqHacxgvH7fH3yvW2Xf7zic7jl6b9DY89VQIDAQABo4GlMIGi
MAwGA1UdEwEB/wQCMAAwDgYDVR0PAQH/BAQDAgWgMB8GA1UdIwQYMBaAFDfreWjN
UnHTpWKSAJlidTs1P4LtMB0GA1UdDgQWBBQadw8m46Wp5npvvh4DvHRkWqJ9rjAj
BgNVHREEHDAagRhBbGljZVBTUzIwNDhAZXhhbXBsZS5jb20wHQYDVR0lBBYwFAYI
KwYBBQUHAwIGCCsGAQUFBwMEMD4GCSqGSIb3DQEBCjAxoA0wCwYJYIZIAWUDBAIB
oRowGAYJKoZIhvcNAQEIMAsGCWCGSAFlAwQCAaIEAgIA3gOCAQEAwN0z12cT92zm
Vn0lRdbRCCl36iSO6g422wc5G42Rl2Akqj+lhqEmWKnidMWnniZHOHafds5ti58h
iQFax/Z1bJTOvFxzo7uz9DeWpmdKHT8buiwzPxSxdWXJNPKPDZCNAvvrocrSSeuh
cDzkJ3mWACziFQJhVpjm0XlVz4J6WQpCjxzs1GHKVKY4VOEWvpcuzPqYDwUYfTtk
joTiNcC8fWVKn9TQ6NZJ2lobDyku4bdZ1z9HZLJrRJ/EZoenGfS9HFvYQ+Nmt9zA
7JifAWknpoaxUQFmvNFE2WFkXIdxUCF+vGZ1GmSm6uodZJ3zoenKuFaJTRCd9tPa
GBauR4UtBA==
-----END CERTIFICATE-----
""")
print("Load using asn1crypto:")
_, _, der_bytes = pem.unarmor(cert_string.encode())
asn1_cert = x509.Certificate.load(der_bytes)
print(type(asn1_cert))
print("Load using oscrypto:")
oscrypto_cert = asymmetric.load_certificate(cert_string.encode())
print(type(oscrypto_cert))
Error:
Load using asn1crypto:
<class 'asn1crypto.x509.Certificate'>
Load using oscrypto:
Traceback (most recent call last):
File "User/issue.py", line 37, in <module>
oscrypto_cert = asymmetric.load_certificate(cert_string.encode())
File "C:\Users\User\venv\lib\site-packages\oscrypto\_win\asymmetric.py", line 1542, in load_certificate
return _load_key(certificate, Certificate)
File "C:\Users\User\venv\lib\site-packages\oscrypto\_win\asymmetric.py", line 1622, in _load_key
return _bcrypt_load_key(key_object, key_info, container, curve_name)
File "C:\Users\User\venv\lib\site-packages\oscrypto\_win\asymmetric.py", line 1856, in _bcrypt_load_key
}[alg_selector]
KeyError: 'rsassa_pss'
Hi, P12 parsing seems to be there but it's not yet possible to generate P12 bundles. Is it much of an effort to add the feature?
Lots of warnings are dumped when importing oscrypto.asymmetric in the environment including cffi 1.13.2:
> python -c 'from oscrypto import asymmetric'
/private/tmp/k1/lib/python3.7/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecRandomDefault' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/private/tmp/k1/lib/python3.7/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecPaddingKey' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/private/tmp/k1/lib/python3.7/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecPaddingPKCS7Key' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
/private/tmp/k1/lib/python3.7/site-packages/cffi/cparser.py:164: UserWarning: Global variable 'kSecPaddingPKCS5Key' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
"(usually 'extern')" % (decl.name,))
...
My test environment is as follows:
> system_profiler SPSoftwareDataType
Software:
System Software Overview:
System Version: macOS 10.14.6 (18G1012)
Kernel Version: Darwin 18.7.0
Python Version is as follows.
> python --version
Python 3.7.4
pip output is
> pip list
Package Version
---------- -------
asn1crypto 1.2.0
cffi 1.13.2
oscrypto 1.1.0
pip 19.0.3
pycparser 2.19
setuptools 40.8.0
If cffi is not included, no warning is dumped.
Also cffi==1.13.0 doesn't show any warning too.
I suspect cffi added more strict checking on the global variables?
It would be nice to have support for AES Encryption/Decryption with ECB cipher mode. I know that at least the OS X APIs has support for ECB and so does OpenSSL. While there are great cases that ECB mode should not be used historically, there are some use cases I have run to where data that is encrypted using AES ECB is encrypted (along with other data) a second time with AES in CBC mode. I would rather not have to depend on another encryption library just for that use case.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.