sinhasantos / cloud-discovery

Cloud Discovery provides a point in time enumeration of all the cloud native platform services

Twistlock Cloud Discovery

Cloud Discovery provides point in time enumeration of all the cloud native platform services, such as container registries, managed Kubernetes platforms, and serverless services used across your cloud providers, accounts, and regions. Its a powerful tool for audit and security practitioners that want a simple way to discover all the 'unknown unknowns' across environments without having to manually login to multiple provider consoles, click through many pages, and manually export the data.

Cloud Discovery connects to cloud providers' native platform APIs to discover services and their metadata and requires only read permissions. Cloud Discovery also has a network discovery option that uses port scanning to sweep IP ranges and discover cloud native infrastructure and apps, such as Docker Registries and Kubernetes API servers, with weak settings or authentication. This is useful to discover 'self-installed' cloud native components not provided as a service by a cloud provider, such as a Docker Registry running on an EC2 instance. Cloud Discovery is provided as a simple Docker container image that can be run anywhere and works well for both interactive use and automation.

Cloud Discovery is another open source contribution provided by Twistlock.

Environment variables

1. BASIC_AUTH_USERNAME - This variable determines the username to use for basic authentication.
2. BASIC_AUTH_PASSWORD - This variable determines the password to use for basic authentication.
3. TLS_CERT_PATH - This variable determines the path to the TLS certificate inside the container. By default the service generates self-signed certificates for localhost usage.
4. TLS_CERT_KEY - This variable determines the path to the TLS certificate key inside the container.

Example usage

Start the container

docker run -d --name cloud-discovery --restart=always \
 -e BASIC_AUTH_USERNAME=admin -e BASIC_AUTH_PASSWORD=pass -e PORT=9083 -p 9083:9083  twistlock/cloud-discovery

Scan and list all AWS assets

curl -k -v -u admin:pass --raw --data \
'{"credentials": [{"id":"<AWS_ACCESS_KEY>","secret":"<AWS_ACCESS_PASSWORD>"}]}' \
 https://localhost:9083/discover

Output

Type    Region        ID
EKS     us-east-1     k8s-cluster-1
ECS     us-east-1     cluster-1
ECS     us-east-1     cluster-2
ECS     us-east-1     cluster-3
ECR     us-east-2     cluster-1

Scan all AWS assets and show full metadata for each of them

curl -k -v -u admin:pass --raw --data \
'{"credentials": [{"id":"<AWS_ACCESS_KEY>","secret":"<AWS_ACCESS_PASSWORD>"}]}' https://localhost:9083/discover?format=json

Scan and list all GCP assets

SERVICE_ACCOUNT=$(cat <service_account_secret> | base64 | tr -d '\n')
curl -k -v -u admin:pass --raw --data '{"credentials": [{"secret":"'${SERVICE_ACCOUNT}'", "provider":"gcp"}]}' https://localhost:9083/discover

Output

Type        Region            ID
GKE         us-central1-a     cluster-1
GKE         us-central1-a     cluster-2
GCR         gcr.io            registry-1
GCR         gcr.io            registry-2
Functions   us-central1       function-1

Scan all GCP assets and show full metadata for each of them

SERVICE_ACCOUNT=$(cat <service_account_secret> | base64 | tr -d '\n')
curl -k -v -u admin:pass --raw --data '{"credentials": [{"secret":"'${SERVICE_ACCOUNT}'", "provider":"gcp"}]}' https://localhost:9083/discover?format=json

Port scan a subnet to discover cloud native infrastructure and apps

Scan all open ports and automatically detect insecure apps (native cloud apps configured without proper authorization) Remark: If the container runs in AWS cluster, the subnet can be automatically extracted from AWS metadata API server

curl -k -v -u admin:pass --raw   --data '{"subnet":"172.17.0.1", "debug": true}'   https://localhost:9083/nmap

Output

Host           Port      App                 Insecure
172.17.0.1     5000      docker registry     true
172.17.0.1     5003      docker registry     false
172.17.0.1     27017     mongod              true