indavelopers / gcp-training-projects

Demo project and how-to guide to use Pulumi as an IaC (Infrastructure as Code) tool for creating GCP sandbox projects with starting resources for demos, workshops, trainings, etc.

README

Demo project and how-to guide to use Pulumi as an IaC (Infrastructure as Code) tool for creating GCP sandbox projects with starting resources for demos, workshops, trainings, etc.

Learn more about the OSS Pulumi project: www.pulumi.com

Contact maintainer:

• Marcos Manuel Ortega: [email protected]
• LinkedIn: Marcos Manuel Ortega

Use case

You are running a GCP workshop, training course, sandbox, hackathon... and you need to create several individual projects for participants.

Those project need IAM roles for participants, APIs enabled by default, billing enabled, etc., and most importantly, maybe multiple resources already created following a template.

You don't want to setup each project manually, then setting up every needed resources and config in each project...

You would like to have a template for creating projects automatically, repeating every time the environments are needed, and even collaborate sharing the project templates.

Usage

1. Clone repo and setup as working dir: git clone REPO_URL, cd gcp-training-projects
2. Install Pulumi CLI (read below first): [https://github.com/pulumi/pulumi?tab=readme-ov-file#getting-started], curl -fsSL https://get.pulumi.com/ | sh
   1. (Optional, works without it) Install Pulumi GCP Python package: pip install pulumi_gcp
   2. You can login to Pulumi or manage stack states locally:
      1. State file in $HOME/.pulumi: pulumi login --local (alias for pulumi login file://~)
      2. State file in another location: pulumi login file://path/to/pulumi-state
   3. If you want to use the code in this repo, don't create a new Pulumi project, as will rewrite __main__.py file
3. You can setup Pulumi passphrase so you don't have to input it every time: export PULUMI_CONFIG_PASSPHRASE=passphrase && echo $PULUMI_CONFIG_PASSPHRASE
4. Setup GCP authn for Pulumi CLI: gcloud auth application-default login
   1. You need Cloud SDK installed locally (or use Cloud Shell)
   2. Or use another gcloud CLI installation to create credentials file with said command
   3. In the command output, check the path to the JSON file where the credentials are stored and move it to a known path, e.g. credentials.json
   4. Use its path for envvar and check its content: export GOOGLE_APPLICATION_CREDENTIALS=path/to/credentials.json && echo $GOOGLE_APPLICATION_CREDENTIALS, e.g. export GOOGLE_APPLICATION_CREDENTIALS=$(readlink -f credentials.json) && echo $GOOGLE_APPLICATION_CREDENTIALS
5. Work under stacks dir: cd stacks
6. Create a new Pulumi stack. You can use the exercise name/ID for the Pulumi stack name: pulumi stack init
7. Include config in Pulumi.STACK_NAME.yaml. Requires:
   1. List of emails for access control to projects, with first email for the instructor
   2. List of roles to be assigned to emails
   3. List of APIs to be enabled in projects
   4. IDs for organization & billing account, folder name & display name
   5. Project prefix, e.g. Project IDs created PROJECT_PREFIX-00-EMAIL_HASH_SUFFIX, with 00-99 pseudo-random integers for each project
      1. Project prefix can e.g. reflect the name of the exercise or workshop, so can be the same as Pulumi stack name
      2. GCP project IDs must be 6 to 30 with lowercase letters, digits, hyphens and start with a letter, and trailing hyphens are prohibited, so this also applies to project prefix
   6. Name of the infrastructure resources script to be imported:
      1. For clarity, you can use STACK_NAME_infra, but it's not enforced - i.e. script for this how-to guide is gcp_course_infra
      2. Don't add .py as it's the name of the Python module script
      3. Example file in example-Pulumi.stack_name-yaml
   7. As we're creating a GCP folder and multiple projects, Pulumi config gcp:project is not used, as it, so can be setup as any valid GCP project ID
8. Include IaC for creating template GCP resources in STACK_NAME_infra.py, along Pulumi exports
9. Create resources with Pulumi CLI: pulumi up
   1. If you got an error, sometimes it needs to be run multiple times, as (for now) some resources depends on others, so they need to be created first

Multiple projects, courses or exercises

Sometimes you just want one working environment for a single workshop. Sometimes you're running several training courses, each one consisting of multiple exercises.

You can store multiple exercises in two ways, by using Pulumi projects and Pulumi Stacks:

1. Using a single Pulumi project and multiple Pulumi stacks:
   1. Recommended: Creating a new Pulumi project can rewrite __main__.py, losing all functionality
   2. Every stack represents a new exercise, so multiple courses can be organized using directories, each one hosting multiple exercises as stacks.
2. Using multiple Pulumi projects and multiple Pulumi stack:
   1. Each project represents a different course/workshop/collection of exercises
   2. Each stack represents a single exercise in said project

Each stack will also have its own state for managing GCP resources.

You can manage stacks with pulumi stack (docs):

• Create: pulumi stack init
• List: pulumi stack ls
• Choose current stack: pulumi stack select
• Remove: pulumi stack rm

Each stack will have its own config & secrets files, e.g. Pulumi.STACK_EXERCISE_NAME.yaml, where you can store its config and reference the template GCP resources file, which you can modify to follow the example-Pulumi.gcp_course-yaml example config file.

Requirements

Check requirements.txt.

• Pulumi for Python v3
• Pulumi provider: Google Cloud (GCP) Classic v7

License

GNU GPLv3

Known issues and contribution

Tested at the time of last commit:

• None known.

If you find any issues, please open a GitHub issue before (optionally) opening a PR to fix it, or contact the maintainer directly any way.

TO-DOs

See to-dos in to-dos.md