Coder Social home page Coder Social logo

acp-tf-s3's Introduction

acp-tf-s3 S3 bucket terraform module

Module usage:

 module "s3" {

    source = "git::https://github.com/UKHomeOffice/acp-tf-s3?ref=master"

    name                 = "fake"
    acl                  = "private"
    environment          = "${var.environment}"
    kms_alias            = "mykey"
    bucket_iam_user      = "fake-s3-bucket-user"
    iam_user_policy_name = "fake-s3-bucket-policy"

 }

The bucket created is always encrypted.

If the website_hosting parameter is set to true, default AES256 encryption is used.

For standard buckets, KMS encryption is used if a kms_alias is provided. If kms_alias is not provided, default AES256 encryption is used.

encryption type website_hosting is true website_hosting is false
kms_alias specified AES256 KMS
kms_alias is "" AES256 AES256

Upgrading

v2 of the module is not backwards-compatible with v1 following refactoring of the module.

Because of the limitations of terraform at the time, there were 4 versions of an aws_s3_bucket that were conditionally created, with only one out of the 4 options actually creating a bucket.

This caused issues when a tenant initially requested a bucket without logging and later on asked for logging to be turned on: this meant that the module wanted to destroy one bucket resource and create another one. This meant that the pipeline would fail (due to buckets not being empty) until the terraform state was also refactored.

In v2 of the module, there is a single aws_s3_bucket resource and the 4 options have the appropriate blocks created dynamically (standard bucket, website bucket) x (no logging, logging enabled).

If the state refactoring is performed in a terraform-toolset container, replace terraform below with /acp/bin/run.sh

Upgrading a standard bucket with no logging enabled

Replace standard_bucket below with the name of the module creating the bucket.

terraform state mv module.standard_bucket.aws_kms_alias.s3_bucket_kms_alias[0] module.standard_bucket.aws_kms_alias.this[0]
terraform state mv module.standard_bucket.aws_kms_key.s3_bucket_kms_key[0] module.standard_bucket.aws_kms_key.this[0]
terraform state mv module.standard_bucket.aws_s3_bucket.s3_bucket[0] module.standard_bucket.aws_s3_bucket.this

Upgrading a standard bucket with audit logs enabled

Replace audit_bucket below with the name of the module creating the audit bucket and bucket_with_logging with the name of the tenant bucket that has logging enabled.

# refactoring for the audit bucket
terraform state mv module.audit_bucket.aws_kms_alias.s3_bucket_kms_alias[0] module.audit_bucket.aws_kms_alias.this[0]
terraform state mv module.audit_bucket.aws_kms_key.s3_bucket_kms_key[0] module.audit_bucket.aws_kms_key.this[0]
terraform state mv module.audit_bucket.aws_s3_bucket.s3_bucket[0] module.audit_bucket.aws_s3_bucket.this
#
# refactoring for the bucket with logging enabled
terraform state mv module.bucket_with_logging.aws_kms_alias.s3_bucket_kms_alias[0] module.bucket_with_logging.aws_kms_alias.this[0]
terraform state mv module.bucket_with_logging.aws_kms_key.s3_bucket_kms_key[0] module.bucket_with_logging.aws_kms_key.this[0]
terraform state mv module.bucket_with_logging.aws_s3_bucket.s3_bucket_with_logging[0] module.bucket_with_logging.aws_s3_bucket.this

Upgrading a website bucket with no logging enabled

Replace website_bucket below with the name of the module creating the bucket.

terraform state mv module.website_bucket.aws_s3_bucket.s3_website_bucket[0] module.website_bucket.aws_s3_bucket.this

Upgrading a website bucket with audit logs enabled

Replace audit_bucket below with the name of the module creating the audit bucket and website_bucket_with_logging with the name of the tenant website bucket that has logging enabled.

# refactoring for the audit bucket
terraform state mv module.audit_bucket.aws_s3_bucket.s3_bucket[0] module.audit_bucket.aws_s3_bucket.this
#
# refactoring for the bucket with logging enabled
terraform state mv module.website_bucket_with_logging.aws_s3_bucket.s3_website_bucket_with_logging[0] module.website_bucket_with_logging.aws_s3_bucket.this

Upgrade notes

Please note the following:

  • the KMS key will be amended to enable automatic key rotation. Any already encrypted will still be able to be decrypted with any previous keys replaced by the AWS automatic key rotation process.
  • if you set the block_public_access module property to true, a new resource will be created and a number of bucket policy resources will be modified to make sure that public access is not granted.

Requirements

Name Version
terraform >= 1.0
aws ~> 3.75.1

Providers

Name Version
aws 3.75.1

Modules

Name Source Version
self_serve_access_keys git::https://github.com/UKHomeOffice/acp-tf-self-serve-access-keys v0.1.0

Resources

Name Type
aws_iam_policy.s3_bucket_iam_policy resource
aws_iam_policy.s3_bucket_iam_website_policy_1 resource
aws_iam_policy.s3_bucket_iam_whitelist_ip_and_vpc_policy resource
aws_iam_policy.s3_bucket_iam_whitelist_policy resource
aws_iam_policy.s3_bucket_with_kms_and_whitelist_iam_policy_1 resource
aws_iam_policy.s3_bucket_with_kms_and_whitelist_iam_policy_2 resource
aws_iam_policy.s3_bucket_with_kms_and_whitelist_ip_and_vpc_iam_policy_1 resource
aws_iam_policy.s3_bucket_with_kms_and_whitelist_ip_and_vpc_iam_policy_2 resource
aws_iam_policy.s3_bucket_with_kms_and_whitelist_vpc_iam_policy_1 resource
aws_iam_policy.s3_bucket_with_kms_and_whitelist_vpc_iam_policy_2 resource
aws_iam_policy.s3_bucket_with_kms_iam_policy_1 resource
aws_iam_policy.s3_bucket_with_kms_iam_policy_2 resource
aws_iam_policy.s3_bucket_with_whitelist_vpc_iam_policy resource
aws_iam_policy.s3_tls_bucket_policy resource
aws_iam_user.s3_bucket_iam_user resource
aws_iam_user_policy_attachment.attach_s3_bucket_iam_policy resource
aws_iam_user_policy_attachment.attach_s3_bucket_whitelist_iam_policy resource
aws_iam_user_policy_attachment.attach_s3_bucket_whitelist_ip_and_vpc_iam_policy resource
aws_iam_user_policy_attachment.attach_s3_bucket_with_kms_and_whitelist_iam_policy_1 resource
aws_iam_user_policy_attachment.attach_s3_bucket_with_kms_and_whitelist_iam_policy_2 resource
aws_iam_user_policy_attachment.attach_s3_bucket_with_kms_and_whitelist_ip_and_vpc_iam_policy_1 resource
aws_iam_user_policy_attachment.attach_s3_bucket_with_kms_and_whitelist_ip_and_vpc_iam_policy_2 resource
aws_iam_user_policy_attachment.attach_s3_bucket_with_kms_and_whitelist_vpc_iam_policy_1 resource
aws_iam_user_policy_attachment.attach_s3_bucket_with_kms_and_whitelist_vpc_iam_policy_2 resource
aws_iam_user_policy_attachment.attach_s3_bucket_with_kms_iam_policy_1 resource
aws_iam_user_policy_attachment.attach_s3_bucket_with_kms_iam_policy_2 resource
aws_iam_user_policy_attachment.attach_s3_bucket_with_whitelist_vpc_iam_policy resource
aws_iam_user_policy_attachment.attach_s3_tls_bucket_policy resource
aws_iam_user_policy_attachment.attach_s3_website_bucket_iam_policy_1 resource
aws_kms_alias.this resource
aws_kms_key.this resource
aws_s3_bucket.this resource
aws_s3_bucket_accelerate_configuration.this resource
aws_s3_bucket_acl.this resource
aws_s3_bucket_cors_configuration.this resource
aws_s3_bucket_lifecycle_configuration.this resource
aws_s3_bucket_logging.this resource
aws_s3_bucket_ownership_controls.this resource
aws_s3_bucket_policy.enforce_tls_bucket_policy resource
aws_s3_bucket_policy.s3_website_bucket resource
aws_s3_bucket_public_access_block.s3_bucket resource
aws_s3_bucket_server_side_encryption_configuration.aes resource
aws_s3_bucket_server_side_encryption_configuration.kms resource
aws_s3_bucket_versioning.this resource
aws_s3_bucket_website_configuration.this resource
aws_caller_identity.current data source
aws_iam_policy_document.kms_key_policy_document data source
aws_iam_policy_document.kms_key_policy_document_whitelist data source
aws_iam_policy_document.kms_key_with_whitelist_ip_and_vpc_policy_document data source
aws_iam_policy_document.kms_key_with_whitelist_vpc_policy_document data source
aws_iam_policy_document.s3_bucket_policy_document data source
aws_iam_policy_document.s3_bucket_policy_document_whitelist data source
aws_iam_policy_document.s3_bucket_with_kms_and_whitelist_ip_and_vpc_policy_document_1 data source
aws_iam_policy_document.s3_bucket_with_kms_and_whitelist_ip_and_vpc_policy_document_2 data source
aws_iam_policy_document.s3_bucket_with_kms_and_whitelist_vpc_policy_document_1 data source
aws_iam_policy_document.s3_bucket_with_kms_and_whitelist_vpc_policy_document_2 data source
aws_iam_policy_document.s3_bucket_with_kms_policy_document_1 data source
aws_iam_policy_document.s3_bucket_with_kms_policy_document_2 data source
aws_iam_policy_document.s3_bucket_with_kms_policy_document_whitelist_1 data source
aws_iam_policy_document.s3_bucket_with_kms_policy_document_whitelist_2 data source
aws_iam_policy_document.s3_bucket_with_kms_website_policy_document_1 data source
aws_iam_policy_document.s3_bucket_with_whitelist_ip_and_vpc_policy_document data source
aws_iam_policy_document.s3_bucket_with_whitelist_vpc_policy_document data source
aws_iam_policy_document.s3_tls_bucket_policy_document data source
aws_region.current data source

Inputs

Name Description Type Default Required
acceleration_status Sets the accelerate configuration of an existing bucket. Can be Enabled or Suspended. string "Suspended" no
acl The access control list assigned to this bucket string "private" no
block_public_access Blocks all public access to the bucket bool false no
bucket_iam_user The name of the iam user assigned to the created s3 bucket any n/a yes
cmk_enable_key_rotation Enables CMK key rotation bool true no
cors_allowed_headers Specifies which headers are allowed. list
[
"Authorization"
]
no
cors_allowed_methods Specifies which methods are allowed. Can be GET, PUT, POST, DELETE or HEAD. list
[
"GET"
]
no
cors_allowed_origins Specifies which origins are allowed. list
[
"*"
]
no
cors_expose_headers Specifies expose header in the response. list [] no
cors_max_age_seconds Specifies time in seconds that browser can cache the response for a preflight request. string "3000" no
email_addresses A list of email addresses for key rotation notifications. list [] no
enforce_kms_key_use Whether or not to require a PutObject request to specify the KMS key id that was created. Defaults to true. Should only be set to false to emulate the behaviour of v0.x of the module and only until the tenants have changed their code to specify the KMS key id in their requests bool true no
enforce_tls Specifies if the bucket will be enforce a TLS bucket policy bool true no
environment The environment the S3 is running in i.e. dev, prod etc any n/a yes
expire_noncurrent_versions Allow expiration/retention rules to apply for all non-current version objects bool true no
iam_user_policy_name The policy name of attached to the user any n/a yes
key_rotation Enable email notifications for old IAM keys. bool true no
kms_alias The alias name for the kms key used to encrypt and decrypt the created S3 bucket objects string "" no
kms_key_policy KMS key policy (uses a default policy if omitted) string "" no
create_lifecycle_policy Specifies whether to create lifecycle policies. bool true no
lifecycle_abort_multipart_upload_enabled Specifies Abort Multipart Uploads lifecycle rule status. bool false no
lifecycle_abort_multipart_upload_object_prefix Object key prefix identifying one or more objects to which the lifecycle rule applies. string "" no
lifecycle_abort_multipart_upload_object_tags Object tags to filter on for the abort multipart upload lifecycle rule. map {} no
lifecycle_days_to_abort_multipart_upload Specifies the number of days after which Amazon S3 aborts an incomplete multipart upload. string "7" no
lifecycle_days_to_expiration Specifies the number of days after object creation when the object expires. string "365" no
lifecycle_days_to_glacier_deep_archive_transition Specifies the number of days after object creation when it will be moved to Glacier storage. string "180" no
lifecycle_days_to_glacier_transition Specifies the number of days after object creation when it will be moved to Glacier storage. string "180" no
lifecycle_days_to_infrequent_storage_transition Specifies the number of days after object creation when it will be moved to standard infrequent access storage. string "60" no
lifecycle_expiration_enabled Specifies expiration lifecycle rule status. bool false no
lifecycle_expiration_object_prefix Object key prefix identifying one or more objects to which the lifecycle rule applies. string "" no
lifecycle_expiration_object_tags Object tags to filter on for the expire object lifecycle rule. map {} no
lifecycle_glacier_deep_archive_object_prefix Object key prefix identifying one or more objects to which the lifecycle rule applies. string "" no
lifecycle_glacier_deep_archive_object_tags Object tags to filter on for the transition to glacier lifecycle rule. map {} no
lifecycle_glacier_deep_archive_transition_enabled Specifies Glacier Deep Archive transition lifecycle rule status. bool false no
lifecycle_glacier_object_prefix Object key prefix identifying one or more objects to which the lifecycle rule applies. string "" no
lifecycle_glacier_object_tags Object tags to filter on for the transition to glacier lifecycle rule. map {} no
lifecycle_glacier_transition_enabled Specifies Glacier transition lifecycle rule status. bool false no
lifecycle_infrequent_storage_object_prefix Object key prefix identifying one or more objects to which the lifecycle rule applies. string "" no
lifecycle_infrequent_storage_object_tags Object tags to filter on for the transition to infrequent storage lifecycle rule. map {} no
lifecycle_infrequent_storage_transition_enabled Specifies infrequent storage transition lifecycle rule status. bool false no
log_target_bucket The S3 bucket that access logs should be sent to. string "" no
log_target_prefix The object prefix for access logs string "" no
logging_enabled Specifies whether server access logging is enabled or not. bool false no
name A descriptive name for the S3 instance any n/a yes
number_of_users The number of user to generate credentials for number 1 no
ownership_controls Ownership controls for the writer must be defined by default string "ObjectWriter" no
ownership_controls_object control_object_ownership needs to be set to true bool true no
tags A map of tags to add to all resources map {} no
transition_noncurrent_versions Allow lifecycle rules to apply for all non-current version objects bool true no
versioning_enabled If versioning is set for buckets in case of accidental deletion; deprecated - use versioning_status instead bool false no
versioning_status The versioning status for the bucket - valid values are: Enabled, Disabled and Suspended string "" no
website_error_document The path to the document to return in case of a 4XX error for static website hosting string "error.html" no
website_hosting Specifies if the bucket will be used for static website hosting bool false no
website_index_document The path of index document when requests are made for static website hosting string "index.html" no
whitelist_ip Whitelisted ip allowed to access the created s3 bucket (note: this allows all by default) list [] no
whitelist_vpc Whitelisted vpc allowed to access the created s3 bucket list [] no

Outputs

Name Description
s3_bucket_arn ARN of generated S3 bucket
s3_bucket_id ID of generated S3 bucket
s3_bucket_kms_key KMS Key ID of the generated bucket
s3_bucket_kms_key_arn KMS Key ARN of the generated bucket

acp-tf-s3's People

Contributors

awkwardben avatar nefischer avatar vinn946 avatar oyelekci avatar tasharnvb avatar kashifsaadat avatar marcinc avatar asmith030 avatar james-royds-ho avatar gambol99 avatar jim5252 avatar aiden-page-ho avatar aibitayo55555 avatar jaykeshur avatar lukmaan-a avatar belindac-vd avatar chej-hod avatar

Stargazers

Eva M. Barabas avatar  avatar  avatar  avatar

Watchers

 avatar Ben Marvell avatar Luke Ashe-Browne avatar Douglas Gardner avatar James Cloos avatar  avatar Chris Freeman avatar Mohammud Yassine Jaffoo avatar Martin Devlin avatar Iqbal Shaikh avatar Daniel A.C. Martin avatar Vijay Jadhav avatar Syed Rafiq avatar  avatar  avatar  avatar  avatar

acp-tf-s3's Issues

Clone request

Hello, is this module ready enough to be pulled as we want to upgrade our infrastructure with Terraform 0.12?

Thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.