aws-quickstart / quickstart-hashicorp-vault Goto Github PK

View Code? Open in Web Editor NEW

125.0 16.0 70.0 4.59 MB

AWS Quick Start Team

License: Apache License 2.0

Shell 80.17% Python 19.83%

quickstart-hashicorp-vault's Introduction

quickstart-hashicorp-vault

This repository has been deprecated in favor of https://github.com/aws-ia/cfn-ps-hashicorp-vault.

We will archive this repository and keep it publicly available until May 1, 2024.

quickstart-hashicorp-vault's People

Contributors

Stargazers

Watchers

Forkers

tecracer-fgr houwill dds gibbowski gibbster jpoley alexxnica kryndex avoidik mario21ic rohanshirke dstoq daitc2004 spasupuleti17 roshithrajan supesharisuto dalavancloud samaws1 benluteijn ytupman ogeagla eshack94 drakun tonynv schooloffreelancing kbharadwaja jayabhavana342 dcallao gargana devopsotrator userhas404d iothack devopseze jjjacques etsangsplk abelgana jdubray peteroruba patrixgdd patrixgddtsang anishmourya avidevopsdemo sssriesss shouryasharmahere johntyner hashishrajan sathishstorm zeroc0d3 shay-ul othmanequarati charles545587 sjm42 uakbr rud5g gautambaghel louievandyke dhung-hashicorp annaone subbudevasani belgaied2 soumik-mukherjee wolfsrudel rdomala iamburakcinar

quickstart-hashicorp-vault's Issues

HashiCorpConsulStack fails to create due to required parameters having no values

Here's the exact error I get:

Parameters: [PublicSubnet3CIDR, PrivateSubnet2CIDR, AccessCIDR, PrivateSubnet1CIDR, PublicSubnet2CIDR, PublicSubnet1ID, PublicSubnet2ID, PrivateSubnet3CIDR, PublicSubnet3ID, PublicSubnet1CIDR] must have values

All those parameters do have values though:

AccessCIDR	0.0.0.0/0	
PrivateSubnet1CIDR	10.0.0.0/19	
PrivateSubnet2CIDR	10.0.32.0/19	
PrivateSubnet3CIDR	10.0.64.0/19	
PublicSubnet1CIDR	10.0.128.0/20	
PublicSubnet2CIDR	10.0.144.0/20	
PublicSubnet3CIDR	10.0.160.0/20

Almost forgot: I'm kicking this off from https://aws.amazon.com/quickstart/architecture/vault/ and am using the "Deploy into new VPC" option.

Stack creation fails in existing VPC

I seem to be getting the exact same error when attempting to deploy into our existing VPC. It appears to be an issue creating the ACM certificate. Manually creating the certificate and then defining it also results in the same error.

Values used:

Any assistance would be very appreciated.

Documentation: missing setup for standby vault audit logs

While the documentation states that we should use vault audit-enable file file_path=/var/log/vault_audit.logstatus to enable audit logs, it shows a screenshot for Vault1 and does not specify that this should take place on the standby Vault2 as well.

Architecture Diagram

Updated Architecture diagram

Vault: 06_install_crontab fails on missing file

quickstart-hashicorp-vault.template refers to /etc/cron.hourly/cloudwatch-monitoring.sh but /etc/cron.hourly is empty.

Reconcile Documentation to update for Vault 1.4.0

Deployment Guide
README.md

Deploy into new VPC fails at VaultServerLaunchConfiguration stage

Hi there, I used this template to deploy into a new VPC but it keeps getting this error.

I selected the parameter with CIS-Ubuntu-1604-HVM for Vault AMI. Any ideas on what is causing this error, or any troubleshooting steps I should attempt?

TLS for vault listeners

Hello,

It would be a nice improvement to have vault with TLS not disabled.

Upgrade to Vault 1.3.2

We wish to upgrade Vault to version 1.3.1

Consul is not installed on the Vault servers

Contrary to the documentation provided with the Quickstart, consul is NOT installed on the Vault servers. This causes Vault startup to fail (it's pointed at 127.0.0.1).

There are only two Vault servers started; the documentation shows three.

There is only one bation host created; the documentation shows two.

The Vault HA API and Cluster addresses point at an ENI that won't attach.

What gives? This quickstart is nonfunctional as shipping.

Add Enterprise version support

Add fetching enterprise packages
Add Input mechanism for enterprise licenses

Ubuntu 16.04 LTS (Long Term Support) period will end on Friday, April 30, 2021

Can we get the template updated to use either Ubuntu 18.04 LTS or Ubuntu 20.04 LTS?

vault init does not seem to work

After login from bastion to vault machine, consul member command works. However when running the vault init after exporting VAULT_ADDRESS, it produces error.

Convert from JSON to YAML

Convert templates from JSON to YAML

Unable to initialize vault after creating ssh tunnel

Hi,

On step 4 of https://s3.amazonaws.com/quickstart-reference/hashicorp/vault/latest/doc/hashicorp-vault-on-the-aws-cloud.pdf it says to create an ssh tunnel. I did so by doing
ssh -L 8200:10.0.3.133:8200 [email protected] -N from the bastion host and received this error from my local machine

vault init
Error initializing Vault: Put https://127.0.0.1:8200/v1/sys/init: dial tcp 127.0.0.1:8200: getsockopt: connection refused

Upgrade vault to (1.2.3)

QuickStart Vault - documentation on ssh tunnel

Reported at Forum link

On step 4 of the pdf - https://s3.amazonaws.com/quickstart-reference/hashicorp/vault/latest/doc/hashicorp-vault-on-the-aws-cloud.pdf

To initialize vault it says to start a ssh tunnel between your workstation and the bastion host.
I just wanted to point out that the sshd config on the bastion host has AllowTcpForwarding set to no, so this step will not work until changing that and restarting ssh.
Wonder if that should be added to the documentation, or the default

Solution:
We should "EnableTCPForwarding": "true" to BastionStack in quickstart-hashicorp-vault-master.template

Vault Update to latest version v1.2.2

Thanks.

Add an ELB

We wish to add and ELB

Cannot ssh from bastion host to any of the other instances

I get the Permission denied (publickey) error when I ssh from bastion host to any of the other instances.

Create into existing stack fails at VaultServerAutoScalingGroup stage

I appear to be having issues with the VaultServerAutoScalingGroup stage.

I also get a similar error when trying to deploy into a new VPC but that is already covered under #57.

Checking into the stack after the error shows that the hosts were built, and they appear healthy in the auto scaling group. Any ideas on what is causing this step to fail, or any troubleshooting steps I should attempt?

Add gpg verification to downloaded install_vault bash script

As per the Archive Checksum Verification section on https://www.hashicorp.com/security the install_vault function in https://github.com/aws-quickstart/quickstart-hashicorp-vault/blob/master/scripts/functions.sh should be updated with the following.

Some variables and other files will need to be downloaded during installation, so the below is somewhere between bash snippets and pseudocode.

Set a variable to the PGP public key readonly vault_gpg_key="..."
Import the key into gpg echo -e "${vault_gpg_key}" | gpg --import -
Download the appropriate SHA256SUMS file and SHA256SUMS.sig file for the version of vault
Verify the SHA256 sig gpg --batch --verify "vault_${vault_version}_SHA256SUMS.sig" "vault_${vault_version}_SHA256SUMS"
Validate the SHA256 of the individual downloaded vault ZIP file grep "${vault_filename}" "vault_${vault_version}_SHA256SUMS" | sha256sum -c

Add support for Vault 1.6

Adjust the leader election process since this can now be dynamically configured in the vault server config

Errors with Linux Bastion stage of Hashi quickstart for a new VPC

When launching the hashi vault quickstart for a new VPC I am getting errors during the linux bastion host stage of the overall deployment. The stack for the linux bastion host is erroring out at the BastionAutoScalingGroup resource with the message "Received 1 FAILURE signal(s) out of 1. Unable to satisfy 100% MinSuccessfulInstancesPercent requirement". Even though this message happens the EC2 instance for the AutoScaling group successfully launches.

I have tried launching this quickstart in the us-east-1 and us-west-2 regions with the same results.

I also tried just launching the linux bastion quickstart on its own. I used the same S3 bucket and prefix parameters as the Hashi quickstart is passing in and the quickstart fails with the same error. If I launch the linux bastion quickstart using the default AWS parameters then the quickstart for the linux bastion host launches successfully.

I was able to successfully launch the Hashi vault quickstart back at the beginning of April so have not always seen this error.

Let me know if there is any more info I can provide which will help.

Enforce the Combination of HostedZoneID and DomainName.

The parameters required for ACM Certificates need to checked before launching the stack.
This should help others avoid issues like #69.

Only one bastion host is created. Documentation shows two.

The documentation illustrates a bastion host being created in public subnets 1 and 2, but there is only one created (in public subnet 2):

Vault Architecture (proposed changes)

The next iteration of the Vault reference is a major rewrite and will have significant changes. Lets issues detail the suggested approach. Once concourses is reached between @dcallao (HashiCorp) and AWS (@avattathil @gargana) We can move to an alpha sprint

Generate SSL Cert

We wish to automatically generate the SSL certificate within CFN for use with the ELB

Can we get the submodule quickstart-aws-acm-certificate updated to get python37 upgrade?

Need submodule quickstart-aws-acm-certificate updated to get python37 update

Repo currently points to https://github.com/aws-quickstart/quickstart-aws-acm-certificate/blob/17e712a6989e9fb514b293eb61fd90870f98629a/templates/quickstart-aws-acm-certificate.template.yml#L164

Latest quickstart-aws-acm-certificate has py3 updated:

https://github.com/aws-quickstart/quickstart-aws-acm-certificate/blob/main/templates/quickstart-aws-acm-certificate.template.yml#L173

Changes is upstream reference (quickstart-hashicorp-consul) breaking reference

aws-quickstart/quickstart-hashicorp-consul#21

/var/awslogs/etc/awslogs.conf is not configured properly

[Note: my initial report was regarding /etc/awslogs.conf -- this file exists on Vault1, but I looked at /etc/init.d/awslogs and found that /var/awslogs/etc/awslogs.conf is the conf being used.]

The documentation says that CloudWatch Logs will stream to Vault-Audit-Logs but /var/awslogs/etc/awslogs.conf is not set up properly.

On Vault1 it has an empty log_group_name:

[general]
state_file = /var/awslogs/state/agent-state

[/var/log/syslog]
file = /var/log/vault_audit.logstatus
log_group_name =
log_stream_name = {instance_id}
datetime_format = %b %d %H:%M:%S

On Vault2 it has the VAULT_LOG_GROUP token still in place:

[general]
state_file = /var/awslogs/state/agent-state

[/var/log/syslog]
file = /var/log/vault_audit.logstatus
log_group_name = __VAULT_LOG_GROUP__
log_stream_name = {instance_id}
datetime_format = %b %d %H:%M:%S

Convert to YAML

Convert Vault reference to YAML

Reconcile Documentation

Can't use TLS auth due to tls_disable = 1

Hi,

As this module use doesn't use tls it is not possible to use TLS authentication. There is an error:
tls connection is required

Step to reproduce:

Provision Vault from the module
Enable TLS auth
Make some tests (add secret, policy and cert), e.g.:

# With Powershell on Windows
$vaulturl = "https://example.com"
$certpath = "Subject of a client cert in Windows Store"
$secret = "secret/test/test"
 
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$cert = New-Object System.Security.Cryptography.X509Certificates.X509Store([System.Security.Cryptography.X509Certificates.StoreName]::My, [System.Security.Cryptography.X509Certificates.StoreLocation]::LocalMachine)
$cert.Open([System.Security.Cryptography.X509Certificates.OpenFlags]::ReadOnly)
$result = $cert.Certificates.Find([System.Security.Cryptography.X509Certificates.X509FindType]::FindBySubjectDistinguishedName, $certpath, $false)
$cert.Close()

json = (Invoke-WebRequest -Uri "$vaulturl/v1/auth/cert/login" -Method 'POST' -Certificate $result[0] -UseBasicParsing).Content | ConvertFrom-Json
(Invoke-WebRequest -Headers @{"X-Vault-Token" = $json.auth.client_token} -ContentType "application/json" -Method 'GET' -Uri "$vaulturl/v1/$secret" -UseBasicParsing).Content

And the error is:
tls connection is required

Log Group not unique

We would like to deploy Vault in the same VPC for which we would like to have Log group name to be unique.

CopyZipsTemplate S3 error: Access Denied

I am using the quickstart to deploy vault from AWS console, deploy to existing VPC, error on this step:

CopyZipsTemplate | CREATE_FAILED | S3 error: Access Denied For more information check http://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html

The CloudFormation should create the stack under my role, and the S3 bucket was created under my login, and I have full control of this bucket,

Parameters: [QuickStartS3URL, AvailabilityZones] do not exist in the template

Hi,

The quickstart-hashicorp-vault-master.template is unable to run and rolls back with the following error:

Parameters: [QuickStartS3URL, AvailabilityZones] do not exist in the template

Refactor bootstrapping

Use configset
Add CFN signaling (via qsutils)

Update to latest Vault version

Vault is currently on version 1.5.3.

This currently only allows for version 1.4

Possibly add the ability to specify latest as a version?

Spelling error

https://github.com/aws-quickstart/quickstart-hashicorp-vault/blob/master/scripts/bootstrap_server.sh#L62-L71

The echo does not use the correct variable name.

Update to vault 1.4.0 including best practices

Update to vault 1.4 including best practices

Add ACM SSL Certificate and ALB

Add a raft snapshot after setup completes

Add a raft snapshot after the leader has been bootstrapped.

Enable Kubernetes Auth mechanism on Vault Cluster

Allow for selection of CIS Ubuntu Linux 16.04 LTS Benchmark or a base image

Adjust all IAM policies to Least privilege and review security controls

Ensure all IAM Policies need to be reviewed and adjusted for least privilege.
- Review Server Instance IAM Role Policies
- Review Client Instance IAM Role Policies
review all access to AWS Secrets Manager Secrets
review all access to KMS Keys
review instance hardening guidance (https://learn.hashicorp.com/vault/day-one/production-hardening)

Add s3 authenticated calls

GovCloud - CloudFormation S3 Denied Error

Hello,

I'm attempting to deploy this stack in GovCloud (us-gov-west-1), when entering the template URL (https://aws-quickstart.s3.amazonaws.com/quickstart-hashicorp-vault/templates/quickstart-hashicorp-vault-master.template) and validating the stack via "View/Edit template in Designer", I'm receiving the error:

Cannot open this file because of an error.: https://aws-quickstart.s3.amazonaws.com/quickstart-hashicorp-vault/templates/quickstart-hashicorp-vault-master.template must reference a valid S3 object to which you have access.

When running the stack I get an immediate:

CREATE_FAILED | AWS::CloudFormation::Stack | VPCStack | S3 error: Access Denied ...

I have attempted to create an IAM role for CloudFormation to use with appropriate permissions and that too fails. Any help/pointers would be appreciated. Thanks~

Vault stack creation fails in a new VPC

The vault stack creation fails in new VPC

Parameters:

Events:

Missing dependencies in bootstrap/user-data script causing broken deployment

The bootstrap script on Vault instances fails due to missing objects in S3, effectively breaking the entire quickstart CFN deployment.

+ wget https://s3.amazonaws.com/aws-quickstart/quickstart-hashicorp-vault/submodules/quickstart-hashicorp-consul/scripts/consul_client_bootstrap.sh
--2019-10-16 17:20:45--  https://s3.amazonaws.com/aws-quickstart/quickstart-hashicorp-vault/submodules/quickstart-hashicorp-consul/scripts/consul_client_bootstrap.sh
Resolving s3.amazonaws.com (s3.amazonaws.com)... 54.231.48.251
Connecting to s3.amazonaws.com (s3.amazonaws.com)|54.231.48.251|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2019-10-16 17:20:45 ERROR 404: Not Found.

+ chmod 755 ./consul_client_bootstrap.sh
chmod: cannot access './consul_client_bootstrap.sh': No such file or directory```