hexadite / acs-keyvault-agent Goto Github PK

Would it be possible that when creating a kubernetes secret as kubernetes.io/tls, that it could support _SECRETS_DATA_KEY? We have a scenario where I'd like to have different secret names be created for conflicting certificates based on different keyvaults. Effectively, I'd like to have

env:
- name: SECRETS_KEYS
   value: ssl::ssl_cert.pem:ssl_key.pem
- name: SSL_SECRETS_DATA_KEY
  value: client-ssl
- name: CREATE_KUBERNETES_SECRETS
   value: "true"
- name: SECRETS_TYPE
   value: kubernetes.io/tls

to produce the kubernetes.io/tls secret named client-ssl rather than ssl.

As part of migration work, we need the capability of downloading CA certificates for specific certs listed even if the global parameter DOWNLOAD_CA_CERTIFICATES is set to 'false'.

Logs from init container:
|2019-08-12 03:18:29,364|INFO |1|140368635659776|keyvault-agent|Parsing Service Principle file from: /host/azure.json
...
File "./main.py", line 68, in _parse_sp_file
with open(file_path, 'r') as sp_file:
IOError: [Errno 13] Permission denied: '/host/azure.json'

This is the yaml file

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: test-keyvault
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: keyvault
    spec:
      containers:
      - image: alpine:latest
        name: test-app
        command: ["/bin/sh", "-c", "--"]
        args: ["while true; do sleep 5; done;"]
        volumeMounts:
        - name: secrets
          mountPath: /secrets
          readOnly: true

      initContainers:
      - name: keyvault-agent
        image: r11.azurecr.io/images/kvagent:v1
        env:
        - name: VAULT_BASE_URL
          value: https://r11testkv.vault.azure.net/
        - name: SERVICE_PRINCIPLE_FILE_PATH
          value: /host/azure.json
        - name: SECRETS_FOLDER
          value: /secrets
        - name: SECRETS_KEYS
          value: testkey

        volumeMounts:
        - name: secrets
          mountPath: /secrets

        - name: host-sp
          mountPath: /host/azure.json
          readOnly: true
      volumes:
      - name: secrets
        emptyDir:
          # we want the secrets to reside only in-memory
          medium: Memory

      - name: host-sp
        hostPath:
          # this file contains the cluster service-principal, it exists on every node by default
          path: /etc/kubernetes/azure.json
          type: File

Someone has been able to expose each file with the secret as environment variables to the main container.

Is there a fix for this error?

/usr/local/lib/python2.7/dist-packages/jwt/utils.py:8: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release.
from cryptography.hazmat.primitives.asymmetric.utils import (
|2022-03-08 22:08:11,297|INFO |1|127115567826752|keyvault-agent|Grabbing secrets from Key Vault
|2022-03-08 22:08:11,297|INFO |1|127115567826752|keyvault-agent|Parsing Service Principle file from: /etc/kubernetes/azure.json
|2022-03-08 22:08:11,298|INFO |1|127115567826752|keyvault-agent|AAD tenant auto detection turned off. Using tenant id from cloud config
|2022-03-08 22:08:11,298|INFO |1|127115567826752|keyvault-agent|Parsing Service Principle file completed
|2022-03-08 22:08:11,299|INFO |1|127115567826752|keyvault-agent|Using MSI

In rare occasion, init container stucks at Init stage indefinitely because MSIAuthentication library makes a http request to IMDS and the connection hangs. The MSIAuthentication library does not specified timeout in the actual request, it will stuck forever when that happen.
Although it's rare but it becomes a lot more likely to happen given the scale of our system.
The problem is at this line of code
The timeout argument provided by the library doesn't work either, because it's not used on the http request level
Related issue: Azure/msrestazure-for-python#148

https://github.com/Azure/msrestazure-for-python/blob/master/msrestazure/azure_active_directory.py
I tried to make a PR but this library has been deprecated and will not accept new feature.
and it is replaced by azure-identity.
https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/identity/azure-identity
We should migrate the current implementation to use azure-identity ASAP.

We have used this image to get certificates for our webserver from keyVault. As there is now the possibility to use MSI on AKS, I wonder how to use the managed identity instead of utilizing the serviceprincipal to authenticate to keyvault.

Is there anything planned to support this scenario?

I assume you wouln't have to change anything as MSI is transparent from application perspective.

In the README.md, the variable for secret keys is SECRET_KEYS, however, in code it is SECRETS_KEYS.

Please fix the docs or the code.

The README indicates that all secret values should be enumerated if the SECRETS_KEYS variable is not present. This is only true for the case where secrets are stored as Kubernetes secrets objects.

I have a branch forked where I've added this functionality: https://github.com/GoodSky/acs-keyvault-agent/commit/fa8eadc160befdbba2bd6c6eadb879cdd1593ac0

I have set the env variable as the tutorial says:

    - name: CREATE_KUBERNETES_SECRETS
      value: "true"

It appears set in the environment part of kubectl describe pods:

Environment:
  SERVICE_PRINCIPLE_FILE_PATH:  /host/azure.json
  CREATE_KUBERNETES_SECRETS:    true

However, the logs from the container show that the code is failing in grab_secrets, however this method should never have been called according to main.py:

if __name__ == '__main__':
    _logger.info('Grabbing secrets from Key Vault')
    if os.getenv('CREATE_KUBERNETES_SECRETS','false').lower() == "true":
        KeyVaultAgent().grab_secrets_kubernetes_objects()
    else:
        KeyVaultAgent().grab_secrets()
    _logger.info('Done!')

|2019-08-15 20:12:27,195|INFO |1|140086269146880|keyvault-agent|Grabbing secrets from Key Vault
|2019-08-15 20:12:27,195|INFO |1|140086269146880|keyvault-agent|Parsing Service Principle file from: /host/azure.json
|2019-08-15 20:12:27,195|INFO |1|140086269146880|keyvault-agent|Parsing Service Principle file completed
Traceback (most recent call last):
File "main.py", line 182, in
KeyVaultAgent().grab_secrets()
File "main.py", line 89, in grab_secrets
self._secrets_output_folder = os.path.join(output_folder, "secrets")
File "/usr/lib/python2.7/posixpath.py", line 70, in join
elif path == '' or path.endswith('/'):
AttributeError: 'NoneType' object has no attribute 'endswith'

seems like it supporting msi only , is there a way to use podidentity with this ?
for customer using aks.
if there is ill be glad for a sample.

Hi,
I am trying to deploy acs-keyvault-agent/example/acs-keyvault-deployment.yaml

The current code tried to list kubernetes secret under conn namespace by system:serviceaccount:conn:default account and this account has not permission to list secret object at current namespace.

Is it alternative service account to using any action on kubernetes cluster or namesapce ?

Traceback (most recent call last):
File "./main.py", line 374, in
KeyVaultAgent().grab_secrets_kubernetes_objects()
File "./main.py", line 248, in grab_secrets_kubernetes_objects
self._create_kubernetes_secret_objects(key_name, secret.value, secret_type)
File "./main.py", line 199, in _create_kubernetes_secret_objects
secrets_list = self._get_kubernetes_secrets_list()
File "./main.py", line 146, in _get_kubernetes_secrets_list
api_response = api_instance.list_namespaced_secret(namespace=self._secrets_namespace)
File "/usr/local/lib/python2.7/site-packages/kubernetes/client/apis/core_v1_api.py", line 12838, in list_namespaced_secret
(data) = self.list_namespaced_secret_with_http_info(namespace, **kwargs)
File "/usr/local/lib/python2.7/site-packages/kubernetes/client/apis/core_v1_api.py", line 12941, in list_namespaced_secret_with_http_info
collection_formats=collection_formats)
File "/usr/local/lib/python2.7/site-packages/kubernetes/client/api_client.py", line 321, in call_api
_return_http_data_only, collection_formats, _preload_content, _request_timeout)
File "/usr/local/lib/python2.7/site-packages/kubernetes/client/api_client.py", line 155, in __call_api
_request_timeout=_request_timeout)
File "/usr/local/lib/python2.7/site-packages/kubernetes/client/api_client.py", line 342, in request
headers=headers)
File "/usr/local/lib/python2.7/site-packages/kubernetes/client/rest.py", line 231, in GET
query_params=query_params)
File "/usr/local/lib/python2.7/site-packages/kubernetes/client/rest.py", line 222, in request
raise ApiException(http_resp=r)
kubernetes.client.rest.ApiException: (403)
Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'Date': 'Wed, 08 Apr 2020 10:27:12 GMT', 'Audit-Id': 'ef4302f4-6091-4253-913f-c3f4f7847c08', 'Content-Length': '285', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets is forbidden: User "system:serviceaccount:conn:default" cannot list resource "secrets" in API group "" in the namespace "conn"","reason":"Forbidden","details":{"kind":"secrets"},"code":403}

Is there a reason for not exporting the key vault secrets to kubernetes secrets?
Wondering if I’m missing something. Why was tmpfs and shared volumes prefered?

Tthe keyvault agent fails if the cert in keyvault is already created in PEM format.

|2020-06-18 17:51:09,774|INFO |1|139744735868672|keyvault-agent|Secret is backing certificate. Dumping private key and certificate.
|2020-06-18 17:51:09,774|ERROR|1|139744735868672|keyvault-agent|Secret is not in pkcs12 format

Hi,
Are you planning to export secrets as env variables?
Thx

In Python 3.9, base64.encodestring() was removed as it was deprecated since Python 3.1.

For more information, please see here: https://bugs.python.org/issue39351

Should be: MSI_RESOURCE_ID

How does /host/azure.json get set from /etc/kubernetes/azure.json ?

Some tools/libraries unfortunately require PFX format certificates. Azure Key Vault native supports PFX format, so it would be nice to be able to download the certificates in PFX format, alongside the PEM format (so depending on the tool I need to feed it to, I can pick the appropriate file type).

The output mechanism most important to my use case is the "dump the files to a folder" one. In other words, I would like _dump_pfx to actually save the PFX, as well, not just the extracted PEMs.

Can this script be extended to download certificate as pfx format?

Currently it is not possible to create 'kubernetes.io/tls' type secrets due to missing privateKey/cert info.

This has worked for us in the past, however with a brand new cluster, we are getting the following error when initializing:

$ kubectl logs app-65559b75cf-zjm2c -c keyvault-agent
|2018-08-31 12:34:39,684|INFO |1|140644736362240|keyvault-agent|Grabbing secrets from Key Vault
|2018-08-31 12:34:39,684|INFO |1|140644736362240|keyvault-agent|Parsing Service Principle file from: /host/azure.json
Traceback (most recent call last):
File "main.py", line 196, in
KeyVaultAgent().grab_secrets()
File "main.py", line 52, in init
self._parse_sp_file()
File "main.py", line 65, in _parse_sp_file
sp_data = json.load(sp_file)
File "/usr/local/lib/python2.7/json/init.py", line 291, in load
**kw)
File "/usr/local/lib/python2.7/json/init.py", line 339, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python2.7/json/decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python2.7/json/decoder.py", line 382, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0",
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.3",

Any ideas ?

Thanks

Hi,
The current version of repo is able to supply create one kubernetes secret that has more than key value and key differ from default "secret" as below
root@cloudyanke:/opt/certificates/rabbitmqazurerc# kubectl describe secret mqttservice-rabbitmq-ha

Name: mqttservice-rabbitmq-ha
Namespace: conn
Labels: app=rabbitmq-ha
chart=rabbitmq-ha-1.0.0
heritage=Helm
release=mqttservice
Annotations:

Type: Opaque

Data
rabbitmq-password: 8 bytes
rabbitmq-username: 8 bytes
definitions.json: 1525 bytes
rabbitmq-erlang-cookie: 32 bytes
rabbitmq-management-password: 24 bytes
rabbitmq-management-username: 10 bytes

The required environment variable for the program has “SERVICE_PRINCIPLE_FILE_PATH”. It should be “SERVICE_PRINCIPAL_FILE_PATH”.