Comments (13)
For part 2 we can revisit the config schema that was used for the clients
beta feature:
unclassified:
awsCredentialsProvider:
clients:
- credentialsProvider:
assumeRole:
roleArn: "arn:aws:iam::111111111111:role/foo"
roleSessionName: "jenkins"
- credentialsProvider:
assumeRole:
roleArn: "arn:aws:iam::222222222222:role/bar"
roleSessionName: "jenkins"
There's a few options for where to insert namespaces here.
Implicit auto-detection
The namespaces could be auto-sensed inside the provider - by extracting the AWS account ID from the ARN of each secret in the ListSecrets operation, and saving that as a new property on the credential.
In this case there is no need to add extra namespace
properties to the CasC configuration.
There is a small risk of a name clash from misconfiguration of Jenkins. If two clients
are configured to assumeRole into the same target account, the secrets in that account will be fetched twice, producing a name clash. In this situation the administrator would have to have pasted two roleArn
s containing the same account ID in the CasC file, which should set alarm bells ringing before they even save that config.
Explicit namespaces
The namespaces could be declared in their own section of the CasC config, then referenced from credential providers. This would also make them usable in other places besides the credentials system.
namespaces:
- 1111111111
- 2222222222
unclassified:
awsCredentialsProvider:
clients:
# TODO fill in details
Explicit semantic namespaces
If namespaces are extracted to their own section, we are not limited to using the identifier from the underlying system (e.g. the AWS account ID) as the namespace identifier. Instead, we could give each namespace a semantic name like 'staging' or 'production', and somehow map the semantic identifier down to the real identifier within the providers.
namespaces:
- staging
- production
unclassified:
awsCredentialsProvider:
clients:
# TODO fill in details
from aws-secrets-manager-credentials-provider-plugin.
We could really use this feature as well, we have secrets in another account that are shared with us, and we need to pull them into Jenkins
from aws-secrets-manager-credentials-provider-plugin.
Why does the plugin not just take a secret ARN? Looking through the code, the plugin is doing some querying with filters for secrets. If it had the ARN, couldn't it just access that secret directly, even if it was in a different account?
from aws-secrets-manager-credentials-provider-plugin.
I did experiment with using secret ARNs as credential IDs, but found that credential listing or binding was unreliable. Sometimes secret retrieval failed, and since Moto won't let us simulate cross-account operations yet, I couldn't write tests for this feature to find out why for sure. Perhaps the credentials API did not like some of the characters used in the secret ARNs.
from aws-secrets-manager-credentials-provider-plugin.
Makes sense. It seems like a limitation of the AWS API as well, that list secrets is scoped to only the requester's account. It looks like currently, the flow of the plugin is to fetch secrets at startup, then create Jenkins credentials/interpolate JSCAC yaml. Is that accurate? If that's the case, then I can understand why using the value in ${}
from jcasc (our primary use case right now) to query AWS doesn't work.
from aws-secrets-manager-credentials-provider-plugin.
You can retrieve secrets from other accounts if you do an sts:assumeRole
first, and then reference the resource by full ARN (not just its name) when fetching it. So it's technically possible, but comes with some practical issues:
- how to do that in a reliable way (each cross-account operation is another HTTP call which could fail - how do you partition them so that a failure in one doesn't knock over the others?)
- how to namespace the retrieved credentials so that if two of them have the same name, we don't get a name clash. (This is more common than you'd think; products in my company typically have one AWS account for the staging environment and one for the production environment. Terraform is used to provision the accounts so they look as similar as possible. This means that almost all the secrets would have a name clash in Jenkins if there's no namespacing mechanism!)
from aws-secrets-manager-credentials-provider-plugin.
The way the plugin works at the moment is loosely like this:
- At startup a Future is created which knows how to retrieve the credentials using ListSecrets.
- This Future is then wrapped in a memoizer function, which caches the results for 5 minutes if the list operation was successful, and but doesn't cache if not.
- When the CredentialsProvider#listCredentials() function is called (the main entry point to the plugin), the Future is invoked and the result is handed back.
- Presuming it was successful, subsequent calls to CredentialsProvider#listCredentials() within the cache period use the memoized result. This avoids hammering the AWS API unnecessarily.
- The cache resets, and when CredentialsProvider#listCredentials() is next called, steps 3-4 repeat.
Later on, when a credential is bound in a Jenkins job, the secret value is retrieved online with GetSecretValue. The credential consumer may elect to cache the value - within a job, a given credential will only be bound once. But the credential provider never caches the secret value itself.
from aws-secrets-manager-credentials-provider-plugin.
Just posting the first part of the solution in here. Comments and suggestions welcome!
My idea is to introduce a concept of namespaces to the credentials API plugin, to represent credentials that come from different places. In our case that will be different AWS accounts, but it's more generic than that.
This can be used, simply enough, by supplying an optional namespace
argument to a credentials API lookup or binding in the Jenkinsfile.
A credentials
binding in your Jenkinsfile would look like this:
pipeline {
agent any
stages {
stage('Deploy to staging') {
environment {
API_KEY = credentials('api-key', namespace: '1111111111')
}
steps {
sh 'curl -X POST -u "foo:$API_KEY" https://example.com'
}
}
stage('Deploy to production') {
environment {
API_KEY = credentials('api-key', namespace: '2222222222')
}
steps {
sh 'curl -X POST -u "foo:$API_KEY" https://example.com'
}
}
}
}
A withCredentials
binding would look like this:
node {
withCredentials(bindings: [string(credentialsId: 'api-key', variable: 'API_KEY', namespace: '1111111111')]) {
echo 'Hello world'
}
}
Any credential that does not have an explicit namespace is implicitly in the default namespace. These credentials are bound by invoking withCredentials
or credentials
in the Jenkinsfile with no namespace
argument - just as we do today. This also provides backwards compatibility; credentials from any currently existing provider would be deemed to be in the default namespace.
The next part of the problem will be to work out where in the Jenkins config (think in terms of the casc.yaml file) namespaces would be specified and how they would be connected to credentials providers that want to support them.
from aws-secrets-manager-credentials-provider-plugin.
really looking forward for this feature to be implemented.
from aws-secrets-manager-credentials-provider-plugin.
Any news on this? :)
could be really helpful
from aws-secrets-manager-credentials-provider-plugin.
Hi @gals-ma
I found an eventual solution to this, which is to implement folders support in an extension of this provider: the folder-scoped AWS Secrets Manager Credentials Provider (https://github.com/chriskilding/aws-secrets-manager-credentials-provider-folders-plugin)
Within each folder, you can choose the AWS authentication option. In effect this allows you to have a connection to a different AWS account in each folder you have, which solves both the multi-account problem as well as namespacing.
I have a draft implementation ready in the PR chriskilding/aws-secrets-manager-credentials-provider-folders-plugin#1 - BUT I need someone to beta test it in a real Jenkins (non-production!) installation, to ensure that it works before I release it. So far nobody has been forthcoming to do that, so it has been stuck in draft. But if you are willing to help test it, we can get this over the line into an initial release :)
from aws-secrets-manager-credentials-provider-plugin.
Hey @chriskilding, appreciate your update on this.
I am willing to test this, can you please share installation instructions?
from aws-secrets-manager-credentials-provider-plugin.
Great! You'll need to:
- Check out the initial-dev branch of the folder-scoped plugin: https://github.com/chriskilding/aws-secrets-manager-credentials-provider-folders-plugin/tree/initial-dev
- Build the plugin by compiling that branch from source
- Deploy the plugin .hpi file in your test Jenkins and see how it works
This assumes some knowledge of Java software development, so let me know if that works for you or not
from aws-secrets-manager-credentials-provider-plugin.
Related Issues (20)
- Icons don't display for "SSH User Private Key" & "Certificate" credentials types HOT 4
- Support for browerstack credential kind HOT 2
- Cross-account role access doesn't appear to work HOT 3
- Make this plugin configurable at folder level, not just centrally HOT 7
- AWS EKS 1.24 client is not respecting jenkins-master pod role HOT 5
- casc config reports improper filter value HOT 5
- reading json secrets HOT 2
- Create support for username-password passing without tag value limitations HOT 3
- Support the popular AmazonWebServicesCredentialsBinding credential types HOT 2
- SSH Keys not working with sshagent
- Ability to set STS endpoint
- The plugin does not pick up Jenkins' proxy settings HOT 4
- Content goes to 404 in Jenkins's documentation
- File Credentials stored in AWS cannot be validated HOT 3
- Don't remove credentials during temporary issues HOT 3
- Github app credentials integrations HOT 1
- "Could not list credentials in Secrets Manager" HOT 2
- region configuration option not respected HOT 2
- Jenkins does not recognise username with password secrets from AWS Secrets Manager Credenatials Provider plugin
- Prefix removal does not work while using Cache after Jenkins Restart HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aws-secrets-manager-credentials-provider-plugin.