Coder Social home page Coder Social logo

org-formation / org-formation-cli Goto Github PK

View Code? Open in Web Editor NEW
1.3K 22.0 125.0 35.83 MB

Better than landingzones!

License: MIT License

TypeScript 99.17% JavaScript 0.61% Dockerfile 0.02% Nunjucks 0.18% HCL 0.02%
aws organizations landing-zone controltower cloudformation devops tool cli patterns best-practices

org-formation-cli's Introduction

AWS Organization Formation

AWS Organization Formation is an Infrastructure as Code (IaC) tool for AWS Organizations.

Features

AWS Organization Formation (also: org-formation) has 3 main features:

  1. Infrastructure as Code for AWS Organizations: Infrastructure as Code for AWS Organizations Organization resources reference | Example organization file | CLI Reference

 

  1. CloudFormation annotations to provision resources cross account: CloudFormation annotations to provision resources cross account Annotated CloudFormation reference | Examples | CLI Reference

 

  1. Automation of account creation and resource provisioning: Automation of account creation and resource provisioning Automation task file reference | Example tasks file | CLI Reference

Want more? here a list of 50+ features 😎😎😎

Installation

With npm installed, run

> npm install -g aws-organization-formation

You can now execute the command line program org-formation. try:

> org-formation --help

Docker

If you choose, you can run org-formation in a docker container:

# Set the AWS_PROFILE environment variable and pass it to the container
> AWS_PROFILE=example
# Run the container
> docker run --rm -it -v $HOME/.aws:/root/.aws:ro -v $PWD:/workdir -w /workdir -e AWS_PROFILE orgformation/org-formation-cli

Optional: create an alias for the container:

> alias org-formation='docker run --rm -it -v $HOME/.aws:/root/.aws:ro -v $PWD:/workdir -w /workdir -e AWS_PROFILE orgformation/org-formation-cli'

Getting started

💡Need help getting started? Get some on slack!

📖How to set up AWS Organizations? Off to a great start

🎧 Hear about org-formation in Real-World Serverless podcast #5

📺 See org-formation in Mastering AWS Organizations with Infrastructure-As-Code

To get started you first need an org-formation template that describes all your Organization resources such as Accounts, OUs and SCPs.

After Installation you can generate this file using the following command:

> org-formation init organization.yml  --region us-east-1 [--profile org-master-account]
example output organization.yml file
AWSTemplateFormatVersion: '2010-09-09-OC'

Organization:
  Root:
    Type: OC::ORG::MasterAccount
    Properties:
      AccountName: My Organization Root
      AccountId: '123123123123'
      Tags:
        budget-alarm-threshold: '2500'
        account-owner-email: [email protected]

  OrganizationRoot:
    Type: OC::ORG::OrganizationRoot
    Properties:
      ServiceControlPolicies:
        - !Ref RestrictUnusedRegionsSCP

  ProductionAccount:
    Type: OC::ORG::Account
    Properties:
      RootEmail: [email protected]
      AccountName: Production Account
      Tags:
        budget-alarm-threshold: '2500'
        account-owner-email: [email protected]

  DevelopmentAccount:
    Type: OC::ORG::Account
    Properties:
      RootEmail: [email protected]
      AccountName: Development Account
      Tags:
        budget-alarm-threshold: '2500'
        account-owner-email: [email protected]

  DevelopmentOU:
    Type: OC::ORG::OrganizationalUnit
    Properties:
      OrganizationalUnitName: development
      Accounts:
        - !Ref DevelopmentAccount

  ProductionOU:
    Type: OC::ORG::OrganizationalUnit
    Properties:
      OrganizationalUnitName: production
      Accounts:
        - !Ref ProductionAccount

  RestrictUnusedRegionsSCP:
    Type: OC::ORG::ServiceControlPolicy
    Properties:
      PolicyName: RestrictUnusedRegions
      Description: Restrict Unused regions
      PolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Sid: DenyUnsupportedRegions
            Effect: Deny
            NotAction:
              - 'cloudfront:*'
              - 'iam:*'
              - 'route53:*'
              - 'support:*'
            Resource: '*'
            Condition:
              StringNotEquals:
                'aws:RequestedRegion':
                  - eu-west-1
                  - us-east-1
                  - eu-central-1

Note: If you prefer to set up CI/CD run org-formation init-pipeline instead. It will create a CodeCommit repository and CodePipeline that will update your organization upon every commit!

You can make changes to the file you generated and update your organization using the update command. Alternatively, you can run create-change-set and update-change-set. Read more in the cli reference

Once you got the hang of managing organization resources, use these organization resources to write smarter CloudFormation that allows you to provision resources across your organization. Read more about managing resources across accounts.

Why is this important?

Just like with the resources within your AWS Account, managing AWS Organization resources as code allows you to apply changes automatically, reducing manual work, inconsistencies and mistakes.

If you are considering to use an account vending machine (e.g. AWS Control Tower) to create and manage new accounts within your organization: Do realize that the account vending machine allows you to quickly create organization resources but only has limited facilities when it comes to updating and maintaining these resources.

Questions and Answers

My operation takes a long time to complete / is slow.  

Especially if you have a lot of accounts this can happen.

An easy way to speed things up is by specifying the command-line argument --max-concurrent-stacks 10 where 10 is the number of stacks to run in concurrently.

Another way to speed things up is to run tasks in parallel this can be done with the argument --max-concurrent-tasks 10. This, however, has the side-effect that the logging might be somewhat harder to relate to a specific task (as it might be out of order).

 

Is there a way around having to create new email accounts per account?  

Every AWS account needs a unique root email address, there is no way around this...

What you can do is to check whether your mail server allows you to append a '+' (plus sign) and another secondary name to your account to create new unique email addresses.

Email to there addresses will end up in the mailbox assigned to the alias before the plus sign and this will still be considered a valid and unique email address when creating a new AWS Account.

Example: If your email address is [email protected] you will receive email send to [email protected] and [email protected] to your inbox.

Mail servers that support this are gmail, aws workmail and hotmail.

 

How do i set up MFA for the account used by org-formation?  

Org-formation needs high privilege access to your master account. If you run org-formation manually it is wise to set up MFA.

I assume you have credentials set up in ~/.aws/credentials and this looks like (might well be called default):

[org-formation]
aws_access_key_id = AKIAxxxxxxxxx
aws_secret_access_key = xxxxxxxxxxxxxxxxx

This allows org-formation to assume the IAM User that corresponds to the access key and secret using the option --profile org-formation.

To enforce MFA you need to do the following:

  1. Assign an MFA device to the IAM User in the console.
  2. Create a role in your master account that has high privileged access and enforces the use of MFA. We call this MyOrgFormationRole.
  3. Create a profile that refers to the MyOrgFormation. We call this profile org-formation-mfa.
  4. Test whether MFA has been setup correctly by running org-formation describe-stacks --profile org-formation-mfa.
  5. If step #4 was successful you can strip the IAM user you use from permissions other than the once it needs to assume MyOrgFormationRole.

Code snippets below:

  1. Creating the MyOrgFormationRole Role (step #2) - execute with CloudFormation
AWSTemplateFormatVersion: '2010-09-09'

Resources:
  MyOrgFormationRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: MyOrgFormationRole
      ManagedPolicyArns:
      - 'arn:aws:iam::aws:policy/AdministratorAccess'
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            AWS: !Sub 'arn:aws:iam::${AWS::AccountId}:root'
          Action: sts:AssumeRole
          Condition:
            Bool:
              aws:MultiFactorAuthPresent: 'true'
  1. Creating the profile org-formation-mfa (step #3) put in your ~/.aws/config file. Replace 000000000000 with your master account id. The value for mfa_serial needs to be the value you got when setting up MFA for your user
[profile org-formation-mfa]
role_arn = arn:aws:iam::000000000000:role/MyOrgFormationRole
source_profile = org-formation
mfa_serial = arn:aws:iam::000000000000:mfa/my-user
  1. Expected output when executing a command that requires MFA (step 4):
\> org-formation describe-stacks --profile org-formation-mfa
👋 Enter MFA code for arn:aws:iam::000000000000:mfa/my-user:
XXXXXX # here you type in the  put the MFA code
{ ...regular output } # if successful the command will execute
  1. The minimum set of permissions for your user Replace 000000000000 with your master account id (or the complete ARN for your Role )
Sid: 'AssumeMFARole'
Action: 'sts:AssumeRole'
Effect: 'Allow'
Resource: 'arn:aws:iam::000000000000:role/MyOrgFormationRole'

Hope this helps

 

What is the password of the root user for newly created accounts?  

Accounts that are created have a root user but no password.

You can create a password using the 'Forgot password' process using the root email.

Note: Once you have created a password and used it consider throwing the password away. You are not supposed to log in using root anyway and storing your password somewhere could only lead to losing it. As we just figured out above you didn't need it in the first place.

Do bind an MFA on your root user! Find info under the IAM service section of the console

Needless to add? don't use a virtual MFA on the same device that has access to the email account used as RootEmail... this reduces your 'multi-factor' authentication to a single factor 🤔🤣

 

What happens when I remove an account from the organization.yml?  

If you remove an account from the organization it will not be deleted. Deleting accounts using API calls is not supported by AWS.

After running update the account that is removed from the organization will not be able to be part of organization bindings.

\> org-formation update ./examples/organization.yml --profile org-formation
OC::ORG::Account              | Development4Account           | Forget
OC::ORG::OrganizationalUnit   | DevelopmentOU                 | Detach Account (Development4Account)
OC::ORG::OrganizationalUnit   | DevelopmentOU                 | CommitHash

After running update-stacks any stack that was deployed to this account using org-formation will be deleted from the target account. Stacks that have been created by other means will not be affected.

Obviously: having a task file will do both update and update-stacks in the right sequence and you're done!

If you removed and account and want to re-add it: Just add it back to the organization.yml. Make sure you run update and update-stacks (or perform-tasks) and your account will participate in all bindings and the stacks will be re-deployed to the account.

As long as the account was not deleted in full org-formation will identify it by the RootEmail (or AccountId) attribute in the organization.yml

 

What happens when I rename an account (AccountName attribute) in org-formation?  

Renaming accounts is not possible using API's. You will have to log into the account as root to change the account name in AWS.

If you change the AccountName attribute in org-formation this will warn you about the above and will, when resolving references to the account, use the account name from the organization.yml file.

 

What happens when I rename an account (logical name) in org-formation?  

The logical name, just like with CloudFormation is how you refer to the account from within your templates. The logical account is also used as an identifier within org-formation.

If you rename an account, by its logical name, org-formation will first notice that the resource by the old logical name has gone and forget it. Later it will discover the new same account by its new logical name and match it with the physical account that already exists in AWS. It will match the two thus completing the rename.

 

Why is XYZ not supported?  

No reason other than not running into this use-case so far.

Really happy to implement this based on someone elses use-case.

 

More docs

Sponsors & collaborators

Special thanks to the following companies:

Stedi

Moneyou

ChainSlayer

Special thanks to the following individuals:

org-formation-cli's People

Contributors

arunasbendoraitis avatar benmcp avatar brcourt avatar castaples avatar craighurley avatar davidvpe avatar dependabot[bot] avatar eduardomourar avatar eoinsha avatar fredericbarthelet avatar grahamcampbell avatar hxtree avatar jimasuen avatar k-paulius avatar maafk avatar mbarneyjr avatar mhlchirosca avatar niranjansm avatar olafconijn avatar petecliff avatar pgeczyx avatar rehos avatar rene84 avatar rob3c avatar saferrer avatar stewartcampbell avatar theburningmonk avatar tstroband avatar yannickvr avatar zaro0508 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

org-formation-cli's Issues

Variable substitution in tasks file

Would be good to, within a tasks file, be able to support variable substitution.
Variables would always we resolved in the context of the Account/Region a task is being executed (this can be multiple targets, which means in the context of any target the variables will resolve to different values.

Syntax support:

  • !Ref logicalName
  • !GetAtt logicalName.attr
  • !Sub 'my-${reference.attr}
  • `!Sub
    • 'my-${var}
    • { var: reference.attr }`

Attribute support (task file attributes where variables can be substituted):

update-cdk

  • CustomDeployCommand
  • CustomRemoveCommand
  • Parameters

update-serverless.com

  • CustomDeployCommand
  • CustomRemoveCommand
  • Parameters

copy-to-s3

  • RemotePath
  • LocalPath

update-stacks

  • Parameters
  • StackDescription

Variable support (substitutions that can be made):

  • all Organization resources e.g. !Ref Account, !GetAtt Account.Id, !GetAtt Account.Alias described here
  • ${TASK::Parameters} for update-cdk and update-sls, to be used in CustomXxxxCommand
  • ${AWS::AccountId}, ${AWS::Region} while we are at it.

Pseudo Parameters?

Hi,

Does it make sense to introduce new pseudo parameters like ${OC::AwsOrgId} to reference the OrganizationId?

Or do all the usecases for this pseudoparameter have to do difficulties managing cross account permissions and therefore are a thing of the past? 😆

Action: s3:GetObject
Resource: arn:aws:s3:::this-or-that/*
Condition:
  StringEquals:
    aws:PrincipalOrgID:
    - !Ref OC::AwsOrgId

[SLS task] parameter support

it would be nice to support passing parameters to the SLS workload from within tasks. These parameters should be translated to arguments on the deploy and remove command (e.g. --param val --param2 val).

for the CustomDeployCommand and CustomRemoveCommand we could introduce a variable that gets substituted with the parameter string. e.g. !Sub npm ci && npx sls deploy ${Task::Parameters} --something else

MySlsSomething:
Type: update-serverless.com
Path: ./sls/
Parameters:
orgPrefix: myorg
multiAZ: false
instanceType: db.t3.medium
allocatedStorage: 20

Support to skip task

There are certain situations where you would like to keep some resources locked to a particular version. I believe we can achieve that by having a property called skip or locked within a task that will simply keep that resource in the current saved state.

This is very important for production environments where you want to prevent unwanted updates. We should use stack policies from CloudFormation services itself to prevent changes from org-formation or from anyone with access to the stacks in the account.

Ability to delete a stack in all relevant accounts/ou's

It would be nice to have the ability to delete a particular stack in all or specified accounts/ou's/regions. This could be a special org-formation command that takes the organization.yml and parameters to identify the accounts/ou's/regions

Support for deploying S3 files from perform-tasks file

Would be nice to be able to upload files to S3 using a tasks file.
Specific scenario: uploading trusted IP list for GuardDuty, but i bet there is many others:

ServerlessWorkload:
  Type: s3-put-object                         # type
  LocalFile: ./myfile.ext                     # path to file, used to calculate hash
  RemotePath: s3://my-bucket/path/to/object
  OrganizationBinding:
    Account: !Ref AccountA

Questions:

  • Does it make sense to support directories? is there a usecase?
  • Would be nice to support pseudo parameters in the RemotePath, but is it really needed
  • Any other suggestions w/ regards to configuration?

Execute task with configurable role

All tasks that org-formation executes are deployed using the "OrganizationAccountAccessRole" in the target account. This "full access" role (":" on "*") is created when the account is created and allows the master account to assume it.

Customizing or restricting this role is probably not a good idea. First of all, this cannot be done through infra-as-code (CloudFormation) as the role is a pre-existing role and secondly, the chances are you're going to break the org-formation pipeline, requiring to do manual work.

However, in some cases, it is required to provision a particular stack using a different role. One such case is deploying an EKS cluster through CloudFormation. The AWS::EKS::Cluster resource has the peculiar behavior of assigning (solely) admin privileges to the role that created it. When "managing" this cluster through Lambda based custom resources the Lambda's must use the role that was used to create the cluster. As the org-formation OrganizationAccountAccessRole is not configured nor suitable to allow Lambda to use it this poses a chicken-egg problem. There are several complicated workarounds that involve creating the cluster-manage-role and then have a custom lambda use that role to create the cluster through cloud formation but this seems very cumbersome and brittle. The ability to execute a particular org-formation task with a specific role is therefore very useful.

There are other cases in which this ability is also useful. TODO: describe other cases

The most basic way of doing this through org-formation would be to allow the following syntax:

MyTask:
  Type: update-stacks
  Template: ./my-template.yml
  StackName: my-stack
  StackDescription: My Stack which is created with a different role
  DefaultOrganizationBinding:
    Region: eu-west-1
    Account: !Ref MyAccount
  TaskRole: my-pre-existing-role
  Parameters:
    myParam: someValue

The constraints of this pre-existing role are:

  • The 'local' OrganizationAccountAccessRole is allowed to assume it
  • The role must have enough permissions to execute cloudformation
  • ???

Support SamlProvider Resource

To implement SAML based SSO an IAM SAML Identity Provider needs to be set up. Unfortunately, this is not possible in CloudFormation. Currently, this can only be achieved with a custom resource, which requires a lambda, role, etc.

It would be great to have this supported in AWS Organization Formation.

It could look like the following:

IdentityProvider:
  OrganizationBindings:
    Regions: eu-west-1
    Accounts: !Ref UsersAccount
  Type: OC::ORG::SAMLIdentityProvider
  Properties:
    Name: 'my-identiy-provider'
    Metadata: !Ref xmlMetadata

The !Ref to the 'IdentityProvider' should return the Name. The 'Arn' attribute should return the arn of the IdentityProvider.

The following section is the Python code for the CloudFormation custom resource that can create the Identity Provider:

  IdpHandler:
    Type: AWS::Lambda::Function
    Properties:
      FunctionName: !Sub smclp-cfn-idp-${samlProviderName}-${ouName}-handler
      Runtime: python3.6
      Handler: index.lambda_handler
      MemorySize: 128
      Role: !GetAtt IdpHandlerRole.Arn
      Timeout: 30
      Code:
        ZipFile: !Sub |

          import boto3
          from botocore.exceptions import ClientError
          import json
          import cfnresponse
          iam = boto3.client("iam")

          def create_provider(name, doc):
            try:
              resp = iam.create_saml_provider(SAMLMetadataDocument=doc,Name=name)
              return(True, resp['SAMLProviderArn'])
            except Exception as e:
              return (False, "Cannot create SAML provider: " + str(e))

          def delete_provider(arn):
            try:
              resp = iam.delete_saml_provider(SAMLProviderArn=arn)
              return (True, "SAML provider with ARN " + arn + " deleted")
            except ClientError as e:
              if e.response['Error']['Code'] == "NoSuchEntity":
                # no need to delete a thing that doesn't exist
                return (True, "SAML provider with ARN " + arn + " does not exist, deletion succeeded")
              else:
                return (False, "Cannot delete SAML provider with ARN " + arn + ": " + str(e))
            except Exception as e:
              return (False, "Cannot delete SAML provider with ARN " + arn + ": " + str(e))

          def update_provider(arn, doc):
            # Need to create the ARN from the name
            arn = "arn:aws:iam::${AWS::AccountId}:saml-provider/" + name
            try:
              resp = iam.update_saml_provider(SAMLMetadataDocument=doc, SAMLProviderArn=arn)
              return (True, "SAML provider " + arn + " updated")
            except Exception as e:
              return (False, "Cannot update SAML provider " + arn + ": " + str(e))

          def lambda_handler(event, context):
            provider_xml = event['ResourceProperties']['Metadata']
            provider_name = event['ResourceProperties']['Name']

            # create a default ARN from the name; will be overwritten if we are creating
            provider_arn = "arn:aws:iam::${AWS::AccountId}:saml-provider/" + provider_name

            if event['RequestType'] == 'Create':
              res, provider_arn = create_provider(provider_name, provider_xml)
              reason = "Creation succeeded"
            elif event['RequestType'] == 'Update':
              res, reason = update_provider(provider_arn, provider_xml)
            elif event['RequestType'] == 'Delete':
              res, reason = delete_provider(provider_arn)
            else:
              res = False
              resp = "Unknown operation: " + event['RequestType']
            responseData = {}
            responseData['Reason'] = reason

            if res:
              cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData, provider_arn)
            else:
              cfnresponse.send(event, context, cfnresponse.FAILED, responseData, provider_arn)

This function needs the following permissions:

  • arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
  • Allow 'iam:*SamlProvider' on '*'

.orgformation-rc file

Would be cool to have a .orgformation-rc (.orgfo-rc? .org-rc?) file that contains runtime configuration such as profile, stateBucketName, stateObject options.

Support CDK Workloads (typescript)

Hi, in order to add CDK support i was thinking of the following task type:

ServerlessWorkload:
  Type: update-cdk                                   # type
  Path: ./workload/                                  # used to package / calculate hash
  RunNpmInstall: true                                # see #1
  RunNpmBuild: true                                  # see #2 
  OrganizationBinding:
    AccountsWithTag: selective-few
  MaxConcurrentStacks: 10
  FailedStackTolerance: 10

Questions:

  1. RunNpmInstall will run npm ci or i (depdending on whether there is a package.lock). For other languages we can make a RunPipInstall, etc.
  2. In the examples (https://github.com/aws-samples/aws-cdk-examples) i see that a build command needs to be ran. does it make sense to pass the npm action as a parameter? or commit to a convention.

Error in 0.0.12 when executing update-accounts

TypeError: matches is not iterable
    at CfnTemplate._resolveOrganizationFunctions (/usr/local/lib/node_modules/aws-organization-formation/dist/src/cfn-binder/cfn-template.js:155:45)
    at CfnTemplate._resolveOrganizationFunctions (/usr/local/lib/node_modules/aws-organization-formation/dist/src/cfn-binder/cfn-template.js:182:42)
    at CfnTemplate._resolveOrganizationFunctions (/usr/local/lib/node_modules/aws-organization-formation/dist/src/cfn-binder/cfn-template.js:182:42)
    at CfnTemplate.resolveOrganizationFunctions (/usr/local/lib/node_modules/aws-organization-formation/dist/src/cfn-binder/cfn-template.js:62:46)
    at new CfnTemplate (/usr/local/lib/node_modules/aws-organization-formation/dist/src/cfn-binder/cfn-template.js:33:14)
    at CloudFormationBinder.enumBindings (/usr/local/lib/node_modules/aws-organization-formation/dist/src/cfn-binder/cfn-binder.js:35:33)
    at CloudFormationBinder.enumTasks (/usr/local/lib/node_modules/aws-organization-formation/dist/src/cfn-binder/cfn-binder.js:63:36)
    at HandleErrors (/usr/local/lib/node_modules/aws-organization-formation/dist/index.js:71:36)
    at process._tickCallback (internal/process/next_tick.js:68:7)

[Question] Reference AccountId in Taskfile

Given I have generated an AWS::IAM::ManagedPolicy in one specific account, I'd like to set the arn of this policy on a role via a Task file, basically the same use case as the cross account role.

Now the arn looks something like this: arn:aws:iam::<accountID>:policy/my-special-policy. The question: how to get the accountId dynamically in the Taskfile as I don't want to hardcode that?

Error with master account not in organization's root

When trying to run init in organization with master account not in its root, we get the following error:

ERROR: unable to load references for organizational resource DevOU, reason: unable to find resource named masterAccount

Support CFN template files larger than 51200

Expected behaviour
When including template files larger than 51200 bytes, OrgFormation should not throw an error, but instead recognize that the way to process larger templates is to first upload them to S3 and reference the S3 object in the CFN command

Current behaviour

}' at 'templateBody' failed to satisfy constraint: Member must have length less than or equal to 51200
ERROR: task **** failed, reason: number failed stacks 1 exceeded tolerance for failed stacks 0
number failed stacks 1 exceeded tolerance for failed stacks 0
Error: number failed stacks 1 exceeded tolerance for failed stacks 0
    at Object.onFailureToleranceExceeded (/usr/local/lib/node_modules/aws-organization-formation/dist/src/cfn-binder/cfn-task-runner.js:24:23)
    at Function.RunTasks (/usr/local/lib/node_modules/aws-organization-formation/dist/src/core/generic-task-runner.js:49:26)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async Function.RunTasks (/usr/local/lib/node_modules/aws-organization-formation/dist/src/cfn-binder/cfn-task-runner.js:29:9)
    at async UpdateStacksCommand.performCommand (/usr/local/lib/node_modules/aws-organization-formation/dist/src/commands/update-stacks.js:77:17)
    at async Function.Perform (/usr/local/lib/node_modules/aws-organization-formation/dist/src/commands/update-stacks.js:15:9)
    at async Object.perform (/usr/local/lib/node_modules/aws-organization-formation/dist/src/build-tasks/tasks/update-stacks-task.js:27:17)
    at async Function.performTask (/usr/local/lib/node_modules/aws-organization-formation/dist/src/core/generic-task-runner.js:65:17)
    at async Promise.all (index 0)
    at async Function.RunTasks (/usr/local/lib/node_modules/aws-organization-formation/dist/src/core/generic-task-runner.js:43:13)
ERROR: number failed tasks 1 exceeded tolerance for failed tasks 0

Way to reproduce
Create a tasks file larger than 51200 bytes and process that using a tasksfile.yml that includes that template in the update-stacks command

Environment Variables on SLS and CDK Invocations

Would be nice to pass environment variables to tools that are executed on the commandline.
Implementation would be on CustomDeployCommand/CustomRemoveCommand and could be backwards compatible with the current commands.

env variables would be able to overwrite everything, except for the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN passed as environment variables by org-formation.

  CustomDeployCommand:
    cmd: npm ci && npm run build && npx cdk deploy
    env: 
      SOME_VAR: some-value 
      ANOTHER_SOME_VAR: another-value 

or

  CustomDeployCommand:
    cmd: npm ci && npm run build && npx cdk deploy
    env: { SOME_VAR: some-value, ANOTHER_SOME_VAR: another-value} 

Errors occur in the init command when member account names are duplicated?

An error occurred in the init command when member accounts with the same name was included in the organization.

The status of the organization when the error occurred is as follows.
The member accounts have same name.

$ aws organizations list-accounts --profile organizations
{
    "Accounts": [
        {
            "Status": "ACTIVE",
            "Name": "Master",
            "Email": "********[email protected]",
            "JoinedMethod": "INVITED",
            "JoinedTimestamp": 1573634525.57,
            "Id": "XXXXXXXXXXXX",
            "Arn": "arn:aws:organizations::XXXXXXXXXXXX:account/o-xxxxxxxxxx/XXXXXXXXXXXX"
        },
        {
            "Status": "ACTIVE",
            "Name": "Nobuhiro Nakayama",
            "Email": "********[email protected]",
            "JoinedMethod": "CREATED",
            "JoinedTimestamp": 1573634631.151,
            "Id": "YYYYYYYYYYYY",
            "Arn": "arn:aws:organizations::XXXXXXXXXXXX:account/o-xxxxxxxxxx/YYYYYYYYYYYY"
        },
        {
            "Status": "ACTIVE",
            "Name": "Nobuhiro Nakayama",
            "Email": "********[email protected]",
            "JoinedMethod": "CREATED",
            "JoinedTimestamp": 1573634646.563,
            "Id": "ZZZZZZZZZZZZ",
            "Arn": "arn:aws:organizations::XXXXXXXXXXXX:account/o-xxxxxxxxxx/ZZZZZZZZZZZZ"
        }
    ]
}

The error message is as follows.

$ org-formation init organization.yml --region us-east-1 --profile organizations --print-stack
ERROR: unexpected error occurred...
duplicated mapping key at line 22, column -193:
      NobuhiroNakayamaAccount:
      ^
YAMLException: duplicated mapping key at line 22, column -193:
      NobuhiroNakayamaAccount:
      ^
    at generateError (/usr/local/lib/node_modules/aws-organization-formation/node_modules/js-yaml/lib/js-yaml/loader.js:167:10)
    at throwError (/usr/local/lib/node_modules/aws-organization-formation/node_modules/js-yaml/lib/js-yaml/loader.js:173:9)
    at storeMappingPair (/usr/local/lib/node_modules/aws-organization-formation/node_modules/js-yaml/lib/js-yaml/loader.js:335:7)
    at readBlockMapping (/usr/local/lib/node_modules/aws-organization-formation/node_modules/js-yaml/lib/js-yaml/loader.js:1098:9)
    at composeNode (/usr/local/lib/node_modules/aws-organization-formation/node_modules/js-yaml/lib/js-yaml/loader.js:1359:12)
    at readBlockMapping (/usr/local/lib/node_modules/aws-organization-formation/node_modules/js-yaml/lib/js-yaml/loader.js:1089:11)
    at composeNode (/usr/local/lib/node_modules/aws-organization-formation/node_modules/js-yaml/lib/js-yaml/loader.js:1359:12)
    at readDocument (/usr/local/lib/node_modules/aws-organization-formation/node_modules/js-yaml/lib/js-yaml/loader.js:1519:3)
    at loadDocuments (/usr/local/lib/node_modules/aws-organization-formation/node_modules/js-yaml/lib/js-yaml/loader.js:1575:5)
    at load (/usr/local/lib/node_modules/aws-organization-formation/node_modules/js-yaml/lib/js-yaml/loader.js:1596:19)

I was able to avoid the error by eliminating duplicate account names.
(I canceled one of member accounts)

Is this a specification or constraint? Is it a bug?

Template format error: The Value field of every Outputs member must evaluate to a String and not a Map.

When referencing a list instead of a value in a stack that spans multiple accounts the following error is produced:

ERROR: Template format error: The Value field of every Outputs member must evaluate to a String and not a Map.

Steps to reproduce:

  1. Create a hosted zone for "somedomain.com" in account A.
  2. Create a single template that defines the following resources:
  • A hosted zone for "sub.somedomain.com" in account B
  • An NS resource record for domain B in the "somedomain.com" zone in account A using the "!GetAtt SubHostedZone.NameServers" value.

Example of template for step 2:

  # hosted zone in account B
  HostedZone:
    Type: AWS::Route53::HostedZone
    OrganizationBindings:
      Regions: eu-west-1
      Accounts: !Ref AccountB
    Properties:
      HostedZoneConfig:
        Comment: Domain Hosted Zone
      Name: !Sub '${AWSAccount.Tags.subdomain}.${rootHostedZoneName}.'
      HostedZoneTags:
        - Key: Application
          Value: cso
        - Key: Component
          Value: domains
        - Key: Aspect
          Value: addressing

  # Parent NS record in root zone in account A
  ParentNsRecord:
    Type: AWS::Route53::RecordSet
    OrganizationBindings:
      Regions: eu-west-1
      Accounts: !Ref AccountA
    Properties:
      Type: NS
      HostedZoneName: !Ref rootHostedZoneName
      Name: !Sub '${AWSAccount.Tags.subdomain}.${rootHostedZoneName}.'
      TTL: 86400
      ResourceRecords: !GetAtt HostedZone.NameServers

The "HostedZone.NameServers" returns a List. As CloudFormation outputs are used to handle cross-account references the error most likely is caused by that.

Note that there's probably an easier way to reproduce this but this is what I've got...

Parameter support in tasks file

It would be nice to declare parameters in a tasks file, overwrite these parameters on the command line and pass parameters down to included task files.

I think it makes sense to keep syntax the same as CloudFormation.

e.g:

Parameters:
  resourcePrefix: # can be overwritten by cli
    Type: String
    Default: My

  required: # no Default means error if not provided by cli
    Type: String

OrganizationUpdate:
  Type: update-organization
  Template: ./organization.yml

CfnTemplate:
  Type: update-stacks
  Template: ./my-template.yml
  StackName: my-stack
  Parameters:
    Param: !Ref resourcePrefix

MyInclude:
  Type: include
  Path: ./another-orgformation-tasks.yml
  Parameters:
    Param: !Ref resourcePrefix

and on the CLI:
\> org-fromation perform-tasks my-tasks --parameters resourcePrefix=XYZ required=ABC
or if you are into typing a lot:
\> org-fromation perform-tasks my-tasks --parameters ParameterKey=resourcePrefix,ParameterValue=XYZ ParameterKey=required,ParameterValue=ABC

ConcurrentModificationException

happens sometimes when create a resource (e.g. SCP) and attaching it to another.
workaround: try again.

AWS Organizations can’t complete your request because it conflicts with another attempt to modify the same entity. Try again later

Support for excluding accounts

It would be great to bind to certain OU's but exclude some specific accounts. Something like the following:

  AdministratorRole:
    OrganizationBindings:
      Regions: eu-west-1
      Accounts: '*'
      IncludeMaster: true
      ExcludeAccounts:
        - !Ref UsersAccount
    Type: AWS::IAM::Role
    Properties:
      RoleName: Administrator

S3 Bucket creation for CloudTrail failed

Creating a stack with the following command failed:

Command:
org-formation update-accounts 01-cloudtrail.yml --stack-name cso-cloudtrail --profile my-profile

Error:

ERROR: error updating cloudformation stack cso-cloudtrail in account 74750******* (eu-west-1). 
Resource is not in the state stackCreateComplete
ERROR: Resource CloudTrailLogGroupRole failed because Resource creation cancelled.
ERROR: Resource CloudTrailBucket failed because The XML you provided was not well-formed or did not validate against our published schema (Service: Amazon S3; Status Code: 400; Error Code: MalformedXML; Request ID: 4CD90BFE570924E8; S3 Extended Request ID: Z3ARrstSaVFaVlNV3IcUnrA/2ikpgIW7MxkXI+WGuu/XCvnEniKDTBXksrCXQNttTY+qacLhf4o=).
failed executing task: ResourceNotReady: Resource is not in the state stackCreateComplete
ERROR: unexpected error occurred...
{ ResourceNotReady: Resource is not in the state stackCreateComplete
    at constructor.setError (/usr/local/lib/node_modules/aws-organization-formation/node_modules/aws-sdk/lib/resource_waiter.js:182:47)
    at Request.CHECK_ACCEPTORS (/usr/local/lib/node_modules/aws-organization-formation/node_modules/aws-sdk/lib/resource_waiter.js:44:12)
    at Request.callListeners (/usr/local/lib/node_modules/aws-organization-formation/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (/usr/local/lib/node_modules/aws-organization-formation/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (/usr/local/lib/node_modules/aws-organization-formation/node_modules/aws-sdk/lib/request.js:683:14)
    at Request.transition (/usr/local/lib/node_modules/aws-organization-formation/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/usr/local/lib/node_modules/aws-organization-formation/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /usr/local/lib/node_modules/aws-organization-formation/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/usr/local/lib/node_modules/aws-organization-formation/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/usr/local/lib/node_modules/aws-organization-formation/node_modules/aws-sdk/lib/request.js:685:12)
  message: 'Resource is not in the state stackCreateComplete',
  code: 'ResourceNotReady',
  retryable: false,
  time: 2019-09-28T16:49:46.349Z,
  statusCode: 200,
  retryDelay: 1000 }

Fragment:

  CloudTrailBucket:
    OrganizationBindings:
      Regions: eu-west-1
      Accounts: !Ref AuditAccount
    DeletionPolicy: Retain
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub 'cso-cloudtrail-${AWS::AccountId}'
      AccessControl: Private
      PublicAccessBlockConfiguration:
        BlockPublicAcls: true
        BlockPublicPolicy: true
        IgnorePublicAcls: true
        RestrictPublicBuckets: true
      BucketEncryption:
        ServerSideEncryptionConfiguration:
          - ServerSideEncryptionByDefault:
              SSEAlgorithm: AES-256
      LifecycleConfiguration:
        Rules:
          - Id: ArchiveToGlacier
            Status: Enabled
            Transitions:
              - StorageClass: GLACIER
                TransitionInDays: 365
      VersioningConfiguration:
        Status: Enabled

Specific Master Account reference not working

It seems that a reference to the master account is not working properly.
Consider the following fragment:

  RootHostedZone:
    Type: AWS::Route53::HostedZone
    OrganizationBindings:
      Regions: eu-west-1
      Accounts: !Ref MasterAccount
    Properties:
      ...

Here the reference to the master account yields the following error message:

ERROR: unable to load file somefile.yml, reason: unable to find resource named MasterAccount

A workaround is not specifying any account but to use the include-master property:

  RootHostedZone:
    Type: AWS::Route53::HostedZone
    OrganizationBindings:
      Regions: eu-west-1
      IncludeMasterAccount: true
    Properties:
      ...

Account Vending Machine?

Hi,

I wonder whether it makes sense to create an account vending machine for org-formation.

This account vending machine would probably use the service catalog to allow users to create an account by

  1. Checking out the Organization IaC file
  2. Adding the account (attaching it to the right OU?)
  3. Pushing the changes into git. Create a pull request?
  4. Have the CodePipeline (org-formation init-pipeline) run and create the account.

does this make sense? any additional requirements/ideas?

[Question] Is it possible to use org-formation without owning the master account?

Hi org-formation maintainer(s),

Recently we have been looking at Control Tower and decided to back out because our accounts will probably roll under some vendor's OU to get some discounts and such.

In that kind of set up, are we still able to use org-formation? From a quick look at the features.pdf and the readmes, seems like the answer is no - we have to own the master account to use org-formation. But would still like to check with you to make sure I understand things correctly.

Thank you in advance!
York

organization binding: AccountsWithTag -> AccountWithTag

Attribute AccountsWithTag should be renamed to AccountWithTag (without the 's') in order to have more consistent naming.

Attribute AccountsWithTag should still be supported, but log a warning.

Change should apply to

  • bindings in tasks file
  • toplevel bindings in template
  • bindings on the resource level in templates.

[CDK task] parameter support (-c)

it would be nice to support passing parameters to the CDK workload from within tasks. These parameters should be translated to -c arguments on the deploy and remove command (e.g. -c param=val -c param2=val).

for the CustomDeployCommand and CustomRemoveCommand we could introduce a variable that gets substituted with the parameter string. e.g. !Sub npm ci && npm run build && npx cdk deploy ${Task::Parameters} --requireApproval

MyCdkSomething:
  Type: update-cdk
  Path: ./cdk/
  Parameters:
    orgPrefix: myorg
    multiAZ: false
    instanceType: db.t3.medium
    allocatedStorage: 20

[Question] Add users to account

I'd like to add new users to the newly created account in an OU. Is there an example on how to do that? Or is this tool not meant to do that?

Support Nested OUs

Currently nested OU structures cannot be modelled in the organization.yml file.
Also bindings to an OU will only resolve to the accounts directly part of the OU (not accounts part of a nested child OU).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.