jermwatt / easy_ec2

create, connect, and manage ec2 instances right from vscode

easy ec2 setup, remote usage, and teardown with vscode

easy_ec2 simplifies remote ec2 usage in vscode by adding a command line interface (CLI) and abridged Python API that allows you to easily create, manage, and tear-down AWS ec2 instances and associated resources using ec2 and awscli in a simple, easy to use, and easy to refactor .yaml configuration file.

Contents

• ec2 made easy
  • Contents
  • Installation
  • Using easy_ec2 CLI
    • Managing ec2 instances
      • Creating an ec2 instance with cloudwatch alarm
      • Show instance cloud_init logs
      • Show instance syslog logs
      • Listing ec2 instances
      • Stopping an ec2 instance
      • Starting a stopped an ec2 instance
      • Termianting ec2 instances by id
    • Managing AWS profiles
      • List all AWS profiles in ~/.aws/credentials
      • List active AWS profile (currently used by easy_ec2)
      • Set active AWS profile (currently used by easy_ec2)
  • Using easy_ec2's Python API
    • Creating an ec2 instance

Installation and setup

You can install easy_ec2 via pip as

pip install easy-ec2

To use in tandem with VScode install the ssh-Remote extension as well.

Using easy_ec2 CLI

Managing ec2 instances

Creating an ec2 instance with cloudwatch alarm

easy_ec2 allows you to translate a standard ec2 pythonic infrastructure task like instantiating an ec2 instance with an attached cloudwatch cpu usage alarm from complex pythonic implementation like the following

import boto3

# read in aws_access_key_id and aws_secret_access_key based on input profile_name using boto3
session = boto3.Session(profile_name=profile_name)

# create ec2 controller from session
ec2_controller = session.resource('ec2')

# read in startup script
with open(startup_script_path, 'r') as file:
    startup_script = file.read()

# create a new EC2 instance
instances = ec2_controller.create_instances(
    ImageId='ami-03f65b8614a860c29',
    InstanceName='example_worker',
    NetworkInterfaces=[{
        'DeviceIndex': 0,
        'Groups': ['sg-1ed8w56f12347f63d'],
        'AssociatePublicIpAddress': True}],
    UserData=startup_script,
    TagSpecifications=[{'ResourceType': 'instance',
                        'Tags': [{'Key': 'Name', 'Value': 'example_worker'}]}],
    InstanceType='t2.micro',
    KeyName=<ssh_key_name>,
    )

# wait for the instance to enter running state
instances[0].wait_until_running()
instance_id = instances[0].id

# create cloud watch client
cloudwatch_client = session.client('cloudwatch')

# enable detailed monitoring for the instance
ec2_client.monitor_instances(InstanceIds=[instance_id])

# create alarm
result = cloudwatch_client.put_metric_alarm(
        AlarmName=cpu_alarm_name,
        ComparisonOperator='GreaterThanOrEqualToThreshold',
        EvaluationPeriods=1,
        MetricName='CPUUtilization',
        Namespace='AWS/EC2',
        Period=60,
        Statistic='Average',
        Threshold=threshold_value,
        Dimensions=[
            {
                'Name': 'InstanceId',
                'Value': instance_id
            },
        ],
    )

into easier to re / use and refactor .yaml configuration file using the same boto3 option syntax for to declaration of the same task. So for example the above task can be accomplished using the analogous .yaml configuration file carrying over the same boto3 option syntax as follows:

aws_profile: <your profile name in config/credentials of ~/.aws>

ec2_instance:
  instance_details:
    InstanceName: example_worker
    InstanceType: t2.micro
    ImageId: ami-03f65b8614a860c29
    BlockDeviceMappings: 
      DeviceName: /dev/sda1
      Ebs: 
        DeleteOnTermination: true
        VolumeSize: 8
        VolumeType: gp2
    Groups:
      - <your security group>

  ssh_details: 
    Config:
      User: ubuntu
      IdentityFile: <path to ssh key>
      ForwardAgent: yes
    Options:
      add_to_known_hosts: true
      test_connection: true

  script_details: 
    filepath: <path_to_startup>
    inject_aws_creds: true
    ssh_forwarding: true
    github_host: true

alarm_details:
  ComparisonOperator: GreaterThanOrEqualToThreshold
  EvaluationPeriods: 1
  MetricName: CPUUtilization
  Namespace: AWS/EC2
  Period: 60
  Statistic: Average
  Threshold: 0.99

This example configuration - along with a simple startup script - can be found in the examples directory.

Using easy_ec2 and this configuration config.yaml the same task - instantiating an ec2 instance - can be accomplished via the command line as follows:

easy_ec2 ec2 create config.yaml

Show instance cloud_init logs

easy_ec2 ec2 check_cloud_init_logs <instance_id>

Show instance syslog logs

easy_ec2 ec2 check_syslog <instance_id>

Listing ec2 instances

You can use easy_ec2 to easy see (all/ running / stopped / terminated) instances in your AWS account as follows.

See all instances

easy_ec2 ec2 list_all

See just running instances

easy_ec2 ec2 list_running

The output of this command gives the instance id, name, type, and state of each instance in your account - looking like this

{'instance_id': 'instance_id', 'instance_state': 'running', 'instance_type': 't2.micro'}

You can filter by state - running, stopped, terminated - as follows

easy_ec2 ec2 list_running

easy_ec2 ec2 list_stopped

easy_ec2 ec2 list_terminated

Stopping an ec2 instance

easy_ec2 ec2 stop <instance_id>

Starting a stopped an ec2 instance

easy_ec2 ec2 start <instance_id>

Termianting ec2 instances by id

You can use easy_ec2 CLI to terminate an ec2 instance by id as follows

easy_ec2 ec2 terminate <instance_id>

Note: by default this will delete any cloudwatch alarms associated with the instance.

Managing AWS profiles

You can use easy_ec2 CLI to manage AWS profiles as follows

List all AWS profiles in ~/.aws/credentials

easy_ec2 profile list_all

List active AWS profile (currently used by easy_ec2)

easy_ec2 profile list_active 

Set active AWS profile (currently used by easy_ec2)

easy_ec2 profile set <profile_name>

Using easy_ec2's Python API

In addition to config driven command line use, easy_ec2 also offers a simplified python API that makes creating and managing AWS resources with ec2 easier.

Creating an ec2 instance

In this example an ec2 instance of user-specified type and AMI is created.

Note block_device_mappings and UserData startup bash script are optional.

from easy_ec2 import set_profile
from easy_ec2.startup_script_management import read_startup_script
from easy_ec2.ec2_instance_management import launch_instance

# set aws profile - optional - set to 'default' profile by default
set_profile.set('my_aws_profile') # -> returns None if profile is valid

# read in startup script from file
UserData = read_startup_script('./path/to/startup.sh')

# build ec2 launch instance command
InstanceName = 'example_worker'
InstanceType = 't2.micro'
ImageId = 'ami-03f65b8614a860c29'
Groups = ['my_security_group_id']
BlockDeviceMappings = [
    {
        'DeviceName': '/dev/sda1',
        'Ebs': {
            'VolumeSize': 300,
            'VolumeType': 'gp2'
        }
    }
]
KeyName = 'my_ssh_key_name'

# launch instance
launch_result = launch_instance(KeyName=KeyName,
                                InstanceName=InstanceName,
                                InstanceType=InstanceType,
                                ImageId=ImageId,
                                Groups=Groups,
                                BlockDeviceMappings=BlockDeviceMappings,
                                UserData=UserData)

# wait for the instance to enter running state
launch_result.wait_until_running()

# get instance id
instance_id = launch_result[0].id

Further uses of the Python API can be found in the examples/python_api directory.