Description

Explore your AWS platform with, Dgraph, a graph database.

Thanks to Go and its goroutines, we can insert thousand of ressources in few seconds.

Prerequisites

docker
docker-compose
Go (for build only)

Install

Download and move to /usr/local/bin/ a binary from release page

Build

This project uses go.mod, so after cloning this repo, simply run :

go build && chmod +x ./aws-inventory-graph

GOBIN=/usr/local/bin/ go install && chmod +x /usr/local/bin/aws-inventory-graph

Usage

Start Dgraph server

make

make up

Access to WebUI (dgraph-ratel) : http://localhost:8000

Import ressources

Authentication is based on your .aws/config file.

Usage of aws-inventory-graph:
  -dgraph string
        Dgraph server (ip:port) (default "127.0.0.1:9080")
  -drop
        Drop all nodes and the schema
  -list
        List available ressource types
  -no-schema
        Disable the refresh schema at each run
  -profile string
        Profile from ~/.aws/config (default "default")
  -region string
        AWS Region (default "eu-west-1")
  -type string
        Get the schema for a type (only after importing some data)

Example :

aws-inventory-graph -region us-west-2 -profile xxxx

2019/11/29 17:35:58 Drop all previous data
2019/11/29 17:35:58 Add schema
2019/11/29 17:36:04 List ...
...
2019/11/29 17:36:05 Add ... Nodes
...
2019/11/29 17:36:08 Add ... Edges
...

Get schema for a type

You can get all schemas for types and predicates in dgraph-ratel WebUI:

or with binary, in JSON format :

aws-inventory-graph -type Address | jq

{
  "types": [
    {
      "fields": [
        {
          "name": "name",
          "type": "string"
        },
        {
          "name": "Service",
          "type": "string"
        },
        {
          "name": "Region",
          "type": "string"
        },
        {
          "name": "OwnerId",
          "type": "string"
        },
        {
          "name": "PrivateIpAddress",
          "type": "string"
        },
        {
          "name": "PublicIp",
          "type": "string"
        },
        {
          "name": "Domain",
          "type": "string"
        },
        {
          "name": "AllocationId",
          "type": "string"
        },
        {
          "name": "_Instance",
          "type": "Instance"
        }
      ],
      "name": "Address"
    }
  ]
}

Predicates which are prefixed with a _ are Edges, and they all have a reverse.

Stop and/or Remove Dgraph

Stop :

make stop

Remove :

make rm

Available Ressources

Here the list of currently supported ressources :

Address
AutoScalingGroup
AvailabilityZone
CacheCluster
CacheSubnetGroup
Cidr
DbCluster
DbClusterParameterGroup
DbInstance
DbParameterGroup
DbSubnetGroup
Image
Instance
InstanceProfile
KeyPair
LaunchConfiguration
LaunchTemplate
LoadBalancer
NatGateway
OptionGroup
SecurityGroup
Snapshot
Subnet
TargetGroup
Volume
Vpc
VpcPeeringConnection

All Edges between the Nodes have reversed.

⚠️ AWS API is often messy, names are not consistent from one endpoint to another, we try to fix that and keep a global coherance. This is why some Predicates don't match exact names returned by API.

Query examples

See here to get more info about Dgraph’s GraphQL+.

Get Elastic IPs + Instances + NatGateways

{
  Address(func: type(Address)) @filter(has(_Instance) or has(_NatGateway)){
    name dgraph.type PublicIp
    Instance:_Instance {name dgraph.type InstanceId}
    NatGateway: _NatGateway{name dgraph.type NatGatewayID}
  }
}

Get Classic LoadBalancers + AutoScalingGroups + Instances

{
  LoadBalancer(func: type(LoadBalancer))@filter(eq(LoadBalancerType, classic)) @cascade{
    name dgraph.type
    AutoScaling:~_LoadBalancer {
        name dgraph.type
        Instance:_Instance{
            name dgraph.type InstanceId
        }
    }
  }
}

Get Application LoadBalancers + TargetGroups + AutoScalingGroups + Instances

{
  LoadBalancerV2(func: type(LoadBalancer))@filter(eq(LoadBalancerType, application))@cascade{
    name dgraph.type
    TargetGroup:~_LoadBalancer @filter(type(TargetGroup)){
        name dgraph.type
        AutoScalingGroup:~_TargetGroup @filter(type(AutoScalingGroup)){
            name dgraph.type Instance:_Instance{
                name dgraph.type InstanceId
            }
        }
    }
  }
}

Get VpcPeeringConnections + Vpcs

{
  VpcPeeringConnection(func: type(VpcPeeringConnection)){
    name dgraph.type VpcPeeringConnectionId
    AccepterVpc:_AccepterVpc {name dgraph.type VpcId}
    RequesterVpc:_RequesterVpc {name dgraph.type VpcId}
  }
}

Get which Instances have access to which DbInstances

{
  DbInstances(func: type(DbInstance))@cascade{
    name dgraph.type
    SecurityGroup:_SecurityGroup {
      name dgraph.type GroupId
      IngressSecurityGroup:_SecurityGroup @facets {
        name dgraph.type
        Instance:~_SecurityGroup @filter(type(Instance)){
          name dgraph.type InstanceId
        }
      }
    }
  }
}

Get the Cidr which are allowed for access to DbInstances

{
  Rds(func: type(DbInstance))@cascade{
    name dgraph.type
    SecurityGroup:_SecurityGroup {
      name dgraph.type GroupId
      IngressCidr:_Cidr @facets {
        name dgraph.type
      }
    }
  }
}

Get Instance opened worldwide + associated ports

{
  OpenWorldCidr(func: eq(name, "0.0.0.0/0"))@cascade{
    name dgraph.type
    SecurityGroup:~_Cidr @filter(type(SecurityGroup)) @facets {
      name dgraph.type GroupId
      Instance:~_SecurityGroup @filter(type(Instance)){
        name dgraph.type InstanceId
      }
    }
  }
}

Get which KeyPairs give access to which Instances

{
  KeyPair(func: type(KeyPair))@cascade{
    name dgraph.type
    Instance:~_KeyName{
      name dgraph.type InstanceId
    }
  }
}

Get CacheClusters (Elasticache) with Instances which have access to them

{
  Memcached(func: type(CacheCluster))@filter(eq(Engine, memcached))@cascade{
    name dgraph.type
    SecurityGroup:_SecurityGroup @facets {
      name dgraph.type
      IngressSecurityGroup:_SecurityGroup @facets {
        name dgraph.type
        Instance:~_SecurityGroup @filter(type(Instance)){
          name dgraph.type InstanceId
        }
      }
    }
  }
  Redis(func: type(CacheCluster))@filter(eq(Engine, redis))@cascade{
    name dgraph.type
    SecurityGroup:_SecurityGroup @facets {
      name dgraph.type
      IngressSecurityGroup:_SecurityGroup @facets {
        name dgraph.type
        Instance:~_SecurityGroup @filter(type(Instance)){
          name dgraph.type InstanceId
        }
      }
    }
  }
}

Get InstanceProfiles which are not used by Instances

{
  InstanceProfile(func: type(InstanceProfile)) @filter(not has(~_InstanceProfile)) {
    name dgraph.type
  }
}

Get Instances without backup (no linked Snapshot)

{
  Instance(func: type(Instance)) @filter(eq(OwnerName, univadis-prod)) @cascade{
    name dgraph.type
    Volume:~_Instance @filter(not has(~_Volume) and type(Volume)){}
  }
}

Get the sum of Volumes by type

{
  var(func: type(Volume))  @filter(eq(VolumeType, gp2)){
    A as Size
  }
  var(func: type(Volume))  @filter(eq(VolumeType, standard)){
    B as Size
  }
  Volume(){
    gp2:sum(val(A))
    standard:sum(val(B))
  }
}

Author

Thomas Labarussias (@Issif)

claranet / aws-inventory-graph Goto Github PK

aws-inventory-graph's Introduction