nike-inc / cerberus-python-client Goto Github PK

It looks like if I don't set a role_arn to assume (just want to use the Lambda's execution role), then this logic assumes that an ec2 instance is calling it, so will fail for Lambdas.

Example error I'm seeing from Lambda:

module initialization error: HTTPConnectionPool(host='169.254.169.254', port=80): 
Max retries exceeded with url: /latest/meta-data/iam/info 
(Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f47c888a7f0>:
 Failed to establish a new connection: [Errno 111] Connection refused',))

Current call to CerberusClient.get_secrets(vault_path) returns data in a Python dictionary in which values are not directly accessible by SDB key. As a user I have to get SDB values by first fetching dictionary element with data key and then index by SDB key.
For example, user has to do:

client = CerberusClient("https://prod.cerberus.nikecloud.com")
sdb_path = "some/cerberus/path"
sdb_key = "some_key"
bucket_secrets = client.get_secrets(sdb_path)
#Important line
sdb_secret_of_interest = bucket_secrets['data'][sdb_key]

where they would expect:

client = CerberusClient("https://prod.cerberus.nikecloud.com")
sdb_path = "some/cerberus/path"
sdb_key = "some_key"
bucket_secrets = client.get_secrets(sdb_path)
#Important line - notice change to previous block
sdb_secret_of_interest = bucket_secrets[sdb_key]

This behavior (if implemented) is a breaking change.

Two actions required:

• Update to v2/iam-principal auth
• Authenticate with Cerberus using the IAM role (instead of the Instance Profile)

Currently the python client uses the EC2 Instance Profile name when authenticating with Cerberus:
https://github.com/Nike-Inc/cerberus-python-client/blob/master/cerberus/aws_auth.py#L61

This is problematic because the real entity that Cerberus needs is the IAM role name. Usually these two names are the same in which case there isn't an issue. However, when the Instance Profile has a separate name (e.g. when generated by CloudFormation) it becomes impossible for Cerberus to find the IAM role that the given Instance Profile is associated with.

The IAM role name can be retrieved from EC2 metadata via this URL:
http://169.254.169.254/latest/meta-data/iam/security-credentials/. Also documented here:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html

The account ID will still need to be retrieved from the Instance Profile ARN.

** Note: The upgrade to v2 auth is not required, but strongly suggested.

In case this is still maintained, the pypi name (cerberus-python-client) and underlying folder name (cerberus) don't match. This means that installing both cerberus (https://pypi.org/project/Cerberus/) and cerberus-python-client via pip will cause a namespace clash and break both libraries

I believe there is a bug in this client where paths in role ARNs aren't working.

Basically, if you have an instance-profile ARN like: arn:aws:iam::1234567890123:instance-profile/foobar/foo/foo-app

The role ARN needs to be contructed to look like this: arn:aws:iam::1234567890123:role/foobar/foo/foo-app

We had the same issue in the java client and the fix can be found in how we generate role ARNs from the metadata endpoints in this class,
https://github.com/Nike-Inc/cerberus-java-client/blob/master/src/main/java/com/nike/cerberus/client/auth/aws/InstanceRoleVaultCredentialsProvider.java

See method buildIamRoleArns() and the args for that method should be gathered from the two endpoints:

1. EC2MetadataUtils.getIAMInstanceProfileInfo().instanceProfileArn; e.g. http://169.254.169.254/latest/meta-data/iam/info
2. EC2MetadataUtils.getIAMSecurityCredentials().keySet(); e.g. http://169.254.169.254/latest/meta-data/iam/security-credentials/

Above handles the pathing edge case as well as another that occurs in CloudFormation where the role name doesn't match the instance profile name.

Cerberus client should self report their name and version via HTTP header with all requests to Cerberus. This is a feature request against all Cerberus clients. This feature will be helpful in understanding:

• What clients are being used in a particular Cerberus environment
• If a bug exists in a certain client version, is that client still being used by applications

The format for this header is similar to the User-Agent string. In fact, we considered using the User-Agent Header but thought using a custom header will make some reporting easier.

Specification:
HTTP Header name: 'X-Cerberus-Client'
Example value: 'CerberusJavaClient/1.4.0'

The method CerberusClient.get_secret(vault_path, key) is ineffective.
It should either hit API that effectively fetches one SDB value, or should be deprecated.
Current implementation encourages bad practice. It misleads users to naively believe that they are getting only one value, where it actually fetches entire bucket and then filters values by key in the client. In the end this can create unnecessary load on Cerberus APIs.