binsabbar / oracle-cloud-terraform

Module for common Oracle Cloud Infrastracture services

• OCI
  • Prerequisites:
  • Managing Different Environments
    • Environemnt Configuration
    • Considerations
    • Advantage of using the above approach:
  • Getting Started
  • Modules
• Releases
  • oci provider version
  • Using the module in your project from github.com:
• Contributors

OCI

Ready Module for common Oracle Cloud Infrastracture services that make it easy to setup and create your infrastracture as code. The repo contains some custom modules that are built on the top of official oci provider. This repository provides ready and handy modules to use to spin up production ready infra in Oracle Cloud.

Note that, you will not be able to manage your infra 100% as code, however, what we aimed at is to be 80%-90%, as some manual setup will still be required.

Prerequisites:

1. You need an account with oracle, and you must have setup your public key in your account. Consult oracle docs for how to setup your account to use oci api here.
2. terraform binary downloadable from here.

Managing Different Environments

There are multiple ways on how to manage environments in Terraform. What is suggested here is to let terraform manages multiple workspaces, each state is scoped to an environment. Environments states are stored differently in remote state file as specified in the configurations.tf file. Environments can be found under environments directory. Here is a suggestion on how to organise your different environments.

environments/
  production/
      variables.tf
      provider.tf
      output.tf
      data.tf
      configurations.tf.  <--------- using production workspace in Terraform
      main.tf   <------ specific environment infra goes here
      ...
      ...
  uat/
      variables.tf
      provider.tf
      output.tf
      data.tf
      configurations.tf.  <--------- using uat workspace in Terraform backend
      main.tf   <------ specific environment infra goes here
      ...
      ...

NOTE Since compartments, users, groups, and policies are not scoped to specific environments, and they apply at the account/tenant level, it is advisable to have a concept of gloabl environment, where states in this environment will affect the whole account (e.g environments/global/ can be for identity management at account level)

Environemnt Configuration

Your configurations.tf will look like this:

terraform {
  backend "remote" {
    hostname     = "app.terraform.io"
    organization = YOUR_ACCOUNT
    workspaces {
      name = YOUR_ENV_WORKSPACE
    }
  }
  required_providers {
    oci = {
      source  = "hashicorp/oci"
      version = "~> 4.2.0"
    }
  }
}

Considerations

It is been found useful to seperate your environments based on compartments. Using compartments make it easy to create full isolation and manage access based on need. For example, you can create production compartment and stage compartments, then create two cicd users, each with access to specific compartment. This way you can manage your environments based on compartemtns with different credentials.

To create different environments (prod, uat, dev), it is assumed that they same overall design.

1. Create a new folder under environments dir, and create configurations.tf file to configure how you want to manage your Terraform state.
2. Create the .auto.tfvars file and fill it with the needed variables.
3. customise the environment as you wish in the maint.tf. You probably want to configure the instances you want to create + security groups per environment.
4. Ensure you do not have an overlapping CIDR for your VCN if you plan to do VCN peering between different environments

Advantage of using the above approach:

1. You can seperate your cloud access to your cicd pipelines based on environments.
2. Easier to manage different environments
3. Easier to create duplicated environments

Getting Started

1. Create a directory under environments.
2. cd to environments/DESIRED_ENV (e.g environments/stage)
3. cp variables.tffrom current working directory to environments/DESIRED_ENV
4. pass in the required variables as specified in variables.tf in a file called .auto.tfvars.
5. Setup your oracle credentials as explained in Generating an API Signing Key.
6. (if you plan to use remote state store) Ensure you have setup your Terraform creds for configurations.tf file
7. run terraform plan
8. if happy run terraform apply

Modules

module	Description
common-config	Contains common configuration that can be used between other modules, such as IDs for computing shape, os versions, etc
vault	Create Vault and manages keys
file-storage-systme	Create a file storage system with exports, export paths and mount targets in a given VCN
identity	IAM management, compartment and policies for creating users, groups, compartments, and policies
instances	Create compute instances and attach a network security group id in a given subnets and VCN
volumes	Create Volumes, Backup Policy and manages volumes attachments
kuberentes	Creates k8s cluster and node pools in the given VCN
load-balancer	WIP - Not ready
mysql	Creates MYSQL Database in a given VCN
network	Creates a VCN with default routing table, and default security list alongside desired subnets
network-load-balancer	Creates an NLB, with listeners and backends alongside backendset in a given VCN
network-sg	Creates network security groups
object-storage	Creates object storage buckets
public-ip	Creates a reserved public IP that can be attached to instances or load balancers
security-list	Creates network security lists
waf	Creates a WAF.
dns	Creates a DNS management system.

Releases

oci provider version

repo release	oci terraform provider version
v1.0	4.20.0
v2.0	4.44

Using the module in your project from github.com:

As explained in Terraform Modules, you can use this repo to refer to the modules defined here. Since all modules are hosted in the same git repo, you can the special double-slash syntax as stated here. You can also set specific version using ref argument.

In summary:

• Use github.com/Binsabbar/oracle-cloud-terraform as source
• Set module name in the path by appending special double-slash: github.com/Binsabbar/oracle-cloud-terraform//modules/identity
• Set ref if you want to avoid breaking changes: github.com/Binsabbar/oracle-cloud-terraform//modules/identity?ref=vx.x

Example

module "identity" {
  source = "github.com/Binsabbar/oracle-cloud-terraform//modules/identity?ref=v2.0"
  ...
  ...
}

module "object-storage" {
  source = "github.com/Binsabbar/oracle-cloud-terraform//modules/object-storage?ref=v2.1"
  ...
  ...
}

Contributors

Thanks to the following folks for providing suggestions and improvments to this project.

• Abdullah Aljubayri Thwwaq (waf module)
• Abeer Alotaibi octopus20 (network-load-balancer module)
• Grzegorz M grzesjam (v1.0 public-ip module)
• Mateusz Kozakiewicz mateuszkozakiewicz (v1.0 public-ip module)
• Abdullah Bin Rasheed Twirlyz (dns module)
• Remaz Altuwaim remaz2250 (v2.8.0 and v2.10.0 instance/network-sg module)

oracle-cloud-terraform's Issues

instances metadata missing user_data

Describe the bug
module: instances
version:
oci version: 4.79.0
description: Trying to use metadata to configure Cloud-Init, found the Cloud-Init script wasnt run on instance as expected. Metadata attribute suppose to have user_data attribute in order to run Cloud-Init custom scripts

Expected behavior
update the instances module to include user_data attribute

An argument named "assign_private_dns_record" is not expected here.

Describe the bug
Module: instances
Version: 2.7.0
OCI version: 4.20.0
Description: Getting error 'unsupported argument' while trying to create a new instance.

To Reproduce
Steps to reproduce the behavior:

1. Create a new instance, example below (Note: compartment ID, subnet ID and Image ID are dummy data):

locals {
  instances = {
    "uat-instance1" = {
      name                     = "instance1"
      availability_domain_name = "Alfp:ME-JEDDAH-1-AD-1"
      fault_domain_name        = "FAULT-DOMAIN-1"
      compartment_id           = "ocid1.compartment.oc1..xxxxxxx"
      volume_size              = 500
      autherized_keys          = ""
      state                    = "RUNNING"
      config = {
        shape    = "VM.Standard3.Flex"
        flex_shape_config = {
          ocpus         = 2
          memory_in_gbs = 4
        }

        image_id = "ocid1.image.oc1.me-jeddah-1.xxxxx"
        subnet   = { 
          id = "ocid1.subnet.oc1.xxxxxx"
          prohibit_public_ip_on_vnic = true
        }
        network_sgs_ids = []
        primary_vnic = {
          primary_ip = ""
          secondary_ips = {}
        }
      }
      secondary_vnics = {}
      optionals = {}
    }
 }
}
module "instances" {
  source = "github.com/Binsabbar/oracle-cloud-terraform//modules/instances?ref=v2.7.0"

  instances = local.instances
}

2. Run terraform init & terraform plan.
3. The error below returned:

╷
│ Error: Unsupported argument
│
│   on .terraform\modules\instances\modules\instances\main.tf line 66, in resource "oci_core_instance" "instances":
│   66:     assign_private_dns_record = false
│
│ An argument named "assign_private_dns_record" is not expected here.
╵
╷
│ Error: Unsupported argument
│
│   on .terraform\modules\instances\modules\instances\main.tf line 97, in resource "oci_core_vnic_attachment" "secondary_vnic_attachment":
│   97:     assign_private_dns_record = true
│
│ An argument named "assign_private_dns_record" is not expected here.

Expected behavior
The instance created successfully.

Screenshots
Errors samples:

Additional context

1. For temporary workaround, go to the instances module at https://github.com/Binsabbar/oracle-cloud-terraform/blob/master/modules/instances/main.tf
2. Comment or delete the assign_private_dns_record property on:

• resource "oci_core_instance" "instances" here.

• resource "oci_core_vnic_attachment" "secondary_vnic_attachment" here.

NLB updating port causing apply to fail

Describe the bug
module: nlb
version:
oci version:
description: what is the issue in details

Updating the port number result in error due to dependency between oci_backend and oci_listener. Currently, it requires manual action to remove all backend from backend_set before removing the backend_set from listener.

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

disable internet/nat gateway from being created

Make it possible to disable the Gateways

To make this backward compatible you can utilise the followings

• Internet Gateway: use attribute enable to control the gateway. When enable is false then remove rule from route table.
• NAT Gateway: use block_traffic attribute to control the gateway. When block_traffic is true, then remove its rule form the private route table.

Complex Object error for security-list rules

Describe the bug
module: security-list
terraform: tested on 0.13, 0.14, 0.15 and 1
version: v1.1
oci version: xxx
description:

When passing security list as collection of maps where some map contains ingress/egress rules and other do not, it produces the following error:

The given value is not suitable for child module variable "security_lists" defined at .terraform/modules/security_lists/modules/security-list/variables.tf:4,1-26: attribute types
│ must all match for conversion to map.

Here is the input:

on main.tf line 45, in module "security_lists":
│   45:   security_lists = {
│   46:     "a" = {
│   47:       egress_rules = tomap({
│   48:         "rule-1" = {
│   49:           protocol    = 12
│   50:           ports       = { min : 12, max : 12 }
│   51:           destination = "test"
│   52:           optionals    = {}
│   53:         }
│   54:       })
│   55:       ingress_rules = tomap({
│   56:         "rule-1" = {
│   57:           protocol    = 12
│   58:           ports       = { min : 12, max : 12 }
│   59:           source = "test"
│   60:           optionals    = {}
│   61:         }
│   62:       })
│   63:     }
│   64:     "b" = {
│   65:       egress_rules = tomap({
│   66:         "rule-1" = {
│   67:           protocol    = 12
│   68:           ports       = { min : 12, max : 12 }
│   69:           destination = "test"
│   70:           optionals    = {}
│   71:         }
│   72:       })
│   73:       ingress_rules = tomap({})
│   74:     }
│   75:   } 

To Reproduce
Steps to reproduce the behavior:

1. Create security rule like the above
2. run terraform validate
3. See error

Expected behavior
to be able to convert successfully into map of empty elements for b.rule-1.ingress_rules

Additional Info
When input type is changed to map(any) for optionals, it worked without any issue.

variable "security_lists" {
  type = map(object({
    egress_rules = map(object({
      protocol    = number
      ports       = object({ min : number, max : number })
      destination = string
      optionals   = map(any)
      # The followings are the keys for the optionals with defaults in brackets
      # type = string(CIDR)
    }))
    ingress_rules = map(object({
      protocol  = number
      ports     = object({ min : number, max : number })
      source    = string
      optionals = map(any)
      # The followings are the keys for the optionals with defaults in brackets
      # type = string(CIDR)
    }))
  }))
}

multiple network interface for an instance

allow passing of extra interfaces for an instance, where the instance must be attached to multiple vnic.

create_vnic_details {
    subnet_id              = each.value.config.subnet.id
    assign_public_ip       = each.value.config.subnet.prohibit_public_ip_on_vnic == false
    display_name           = "${each.key}Vnic"
    hostname_label         = each.key
    nsg_ids                = each.value.config.network_sgs_ids
    skip_source_dest_check = false
  }

k.id attribute doesn't exist in instances module

Describe the bug
module: instances
version: v2.0
oci version: 4.44
description: output of instances don't contain id.

To Reproduce
Steps to reproduce the behavior:

1. Create instance
2. Use output of the module
3. See the error k.id does not exist.

Expected behavior
Instance information without error

FEAT: Virtual IP attached to instance

Ability to attach a secondary to VNIC

FEAT: set default IP for instance in subnet

Ability to set private IP of instance (Primary VNIC) at creation time

Separate K8s node version from cluster version

Currently there is no way to split the version of k8s between the master and node pool. This might cause an issue during an upgrade.

Add `name` key for subnet object variable

Fixing this will result in breaking changes since I have to change the structure of the variable and introduce new key in the object called name. Currently we have the following:

variable "public_subnets" {
  type = map(object({
    cidr_block        = string
    security_list_ids = list(string)
    optionals         = map(any)
    # The followings are the keys for the optionals with defaults in brackets
    # route_table_id = string # id of custom route table
    # allow_tcp_egress_to_ports = list(string) # egress tcp ports to 0.0.0.0/0
    # allow_udp_egress_to_ports = list(string) # egress udp ports to 0.0.0.0/0
  }))

I need to introduce name as key of type string. In V2 I will introduce the new key

variable "public_subnets" {
  type = map(object({
    cidr_block           = string
    name                   = string
    security_list_ids = list(string)
    optionals             = map(any)
    # The followings are the keys for the optionals with defaults in brackets
    # route_table_id = string # id of custom route table
    # allow_tcp_egress_to_ports = list(string) # egress tcp ports to 0.0.0.0/0
    # allow_udp_egress_to_ports = list(string) # egress udp ports to 0.0.0.0/0
  }))

Originally posted by @Binsabbar in #1 (comment)

Issue with security-rules in security-lists module

Describe the bug
module: security-list
version: 1.0
oci version: 4.20.0
description: When adding rules under security_lists (egress_rules & ingress_rules), terraform will throw an error (The given value is not suitable for child module variable "security_lists") .. this happened because of the defined type of "optionals" variable in variables.tf file. I managed to fix this by changing the type from "any" to "map(any)" and using empty brackets as a value for optionals (optionals= {})

To Reproduce
Steps to reproduce the behavior:

1. User security-lists module
2. Create security lists and add security rules (ingress_rules & egress_rules)
3. terraform apply
4. See error

The given value is not suitable for child module variable "security_lists"
defined at
.terraform/modules/security_lists/modules/security-list/variables.tf:4,1-26:
object is required.

Create Service Gateway for OCI

Allow Service Gateway Creation as part of VCN creation. This is very useful and handy feature.

Issues with DNS-Label in network module

Describe the bug
module: network
version: 1.0
oci version: 4.20.0
description: Using network name that contains - will result in the following error Error: Service error:InvalidParameter. DNS Label public_1 does not follow Oracle requirements. http status code: 400. Opc request id: since - is not accepted in the DNS name.

To Reproduce
Steps to reproduce the behavior:

1. use the module network
2. Create private_subnet or public_subnet with a name containing -, such as network-1-public
3. run terraform apply
4. See error

Expected behavior
A network should be created

Additional context
The issue is in this line https://github.com/Binsabbar/oracle-cloud-terraform/blob/master/modules/network/main.tf#L82 and https://github.com/Binsabbar/oracle-cloud-terraform/blob/master/modules/network/main.tf#L95

Default outbound for public subnet

Describe the bug
module: network
version: 0.1
oci version: 0.4.26
description: what is the issue in details

The default behaviour of the module network makes the public and private subnet having outbound for all TCP and UDP ports. This should be optional and not enforced

Screenshots

resource "oci_core_default_security_list" "public_subnet_security_list" {
      + defined_tags               = (known after apply)
      + display_name               = "defaultSecurityList"
      + freeform_tags              = (known after apply)
      + id                         = (known after apply)
      + manage_default_resource_id = (known after apply)
      + state                      = (known after apply)
      + time_created               = (known after apply)

      + egress_security_rules {
          + description      = "Outbound All TCP"
          + destination      = "0.0.0.0/0"
          + destination_type = (known after apply)
          + protocol         = "6"
          + stateless        = (known after apply)
        }
      + egress_security_rules {
          + description      = "Outbound All UDP"
          + destination      = "0.0.0.0/0"
          + destination_type = (known after apply)
          + protocol         = "17"
          + stateless        = (known after apply)
        }

      + ingress_security_rules {
          + description = (known after apply)
          + protocol    = "1"
          + source      = "0.0.0.0/0"
          + source_type = (known after apply)
          + stateless   = true

          + icmp_options {
              + code = 4
              + type = 3
            }
        }
    }

Volume attachments to existing instance

Link more volumes to existing running instances (https://registry.terraform.io/providers/hashicorp/oci/latest/docs/resources/core_volume)

Hostname xxxxx is provided when AssignPrivateDnsRecord is false

Describe the bug
module: instances
version: v2.1.0
oci version: latest
description: Error appeared when creating an instance that contains hostname (see image)

Screenshots

Additional context
Add any other context about the problem here.

Support for VCN-Native Cluster

Current module does not support allocation of control plane (support for VCN-Native Cluster)