dualspark / cloudformation-environmentbase Goto Github PK

It appears that the Fn::GetAZs provided by Amazon returns some invalid output if it's run in an account that was created pre-VPC.

Specifically, it returns a list that includes AZs that can not create new subnets in the VPC era. This is relevant to environmentbase because Fn::GetAZs is used in generated templates when creating HaClusters (and probably other places)

Specifically, when deploying a template with an HaCluster in us-east-1 with an older account, the following snippet of generated template:

 "privateAZ1": {
            "Properties": {
                "AvailabilityZone": {
                    "Fn::Select": [
                        1,
                        {
                            "Fn::GetAZs": ""
                        }
                    ]
                },

causes Cloudformation to explode with: privateAZ1 Value (us-east-1b) for parameter availabilityZone is invalid. Subnets can currently only be created in the following availability zones: us-east-1e, us-east-1a, us-east-1d, us-east-1c.

Note - this ONLY occurs for pre-VPC accounts, due to the invalid AZ being returned. It appears that Amazon is aware of the issue but doesn't consider it a high priority to fix. http://stackoverflow.com/questions/22744467/vpc-capable-availability-zones-in-amazon/22812138#22812138

I'm not really sure what, if anything, can be done to fix this aside from building tables of VPC-subnet-capable AZs yourself instead of using Fn::GetAZs, but I thought you guys should be aware.

I know we are working on it, but I just wanted to note down this issue before I forget and becomes untracked.

The README makes it seem like you can just pull this repo, do a pip install -e . and then run environmentbase -h to see the cmd options and even launch a VPC with 1 public subnet and 1 private, with a bastion instance etc.

But as of this version of the code, that does not happen.
On master I had this error:

IOError: [Errno 2] No such file or directory: u'ami_cache.json'

and on the feature-0.4.0 I got:

 File "/Users/gabrielreyla/dev/cloudformation-environmentbase/src/environmentbase/environmentbase.py", line 4, in <module>
    import Template
ImportError: No module named Template

just like add_parameter_idempotent I was thinking it might be a good idea to make add_output_idempotent. But maybe we should at least throw a warning when assigning a different value to the same output key.

Putting the has in the template itself does not make it easy to check it with the final template. Maybe we can come up with another way of passing the validation hash into the template (i.e. as a parameter?)

From:
*ELBDNSName
to:
*ElbDnsName

Or to something more readable

templateValidationHash and dateGenerated from template -- or at least make them optional -- to make diff-ing templates easier.

Walking through the setup for a new environment with a fresh checkout gets:

$ environmentbase create

ERROR:
    Config file missing section global.valid_regions

Try running with the --debug flag

create_elb only does internet-facing ELBs. I want the ability to set Scheme="internal" for an internal ELB.

The natAmiId for us-east-1 is ami-4f9fee26, which is a PV AMI that doesn't work with the default NAT instance type of t2.micro

I'm guessing the factory_default ami_cache.json just hasn't been rebuit in a while. I'll put in a PR with an updated ami_cache in a bit.

This piece of code doesn't handle us-east-1 region well:

Source: http://boto3.readthedocs.org/en/latest/reference/services/ec2.html#EC2.Client.describe_availability_zones

DefaultLogGroup & VPCFlowLogsIAMRole are still in the root template. We should probably move both to another template.

Worth remembering that we can't turn on VPCFlowLogs from CloudFormation at this point and that it has to be done through a separate script.

There's a use case for which EIPs attached to the NAT instances would be helpful: if an external service needs to include the IP of a trusted host in a whitelist, then the IP needs to be stable.

We could probably add a config flag to the nat section of the config, like assign_eip here: https://github.com/DualSpark/cloudformation-environmentbase/blob/master/src/environmentbase/patterns/base_network.py#L39