Something in a recent Serverless upgrade broke aws-sdk-mock so the tests are failing

Hi,
I try to to create the stack with only ONE availability zone.

In my serverless.yml I do something like:

  vpcConfig:
    createNatGateway: true
    zones:
      - eu-central-1b

But I get an error when I deploy -->

Error: The CloudFormation template is invalid: Unresolved resource dependencies [RDSSubnetGroup] in the Outputs block of the template
All is okay when I omit the zone entry. In this case three external IPs are created. (But I want only one.)

What is the problem?

Hey, thanks for this awesome plugin. I use it all the time. Currently, I am including serverless-vpc-plugin in the same stack with my microservices. This is great because your plugin automatically added all Subnets and SGs to my functions. However, as I'm sure you know, that takes up about ~70-80 of the 200 resource limit.

So in an attempt to free up some space in my CF templates I am researching how I can separate VPC resources into a separate serverless.yaml, export the values needed, and declare them in my microservice's serverless.yaml files.

You may recommend doing that this way:

provider:
  vpc:
    subnetIds:
      - 'Fn::ImportValue': myvpcstack-${opt:stage}-AppSubnet1
      - 'Fn::ImportValue': myvpcstack-${opt:stage}-AppSubnet2
      - ...

But I think we can do better than that. We should be able to dynamically discover the subnets that were created, rather than "hardcoding" them. The biggest problem this causes is that not all regions have the same number of AZs(subnets). If I had AppSubnet[1-6] declared, and tried to launch it in an non-US AWS region, it would error because that region only has 3 AZs(subnets) available.

Feature request

Please create a new stack output, call it AllAppSubnets(?), and the value should be all AppSubnets{i} as a single string separated by commas. In us-east-1 this would have 6 values. E.g. subnet-1234, subnet-2345, subnet-3456, subnet-4567, subnet-5678, subnet-6789

Use case, new code example

This is a common pattern used in serverless.yaml for this type of situation:

vpc:
  subnetIds:
    "Fn::Split":
      - ","
      - 'Fn::ImportValue': myvpcstack-${opt:stage}-AllAppSubnets

Possible to set up a multi-AZ public & private subnet VPC? E.g.:

public 1a subnet,
private 1a subnet,
public 1b subnet,
private 1b subnet,
...

Thanks!

Hi,

Are there any plans to include DocumentDB as part of the supported services? Its' CloudFormation needs are very similar to RDS. I think it would be somewhat simple.

Regards

Hi there,

I was wondering if there's a way to opt specific functions out of the global VPC config that this plugin creates?

Use case is: I have some functions that are replicated to Lambda @ Edge, and thus the source functions are not allowed to be inside a VPC.

What I'm doing currently is adding the following "empty" VPC config to the function:

vpc:
  securityGroupIds: []
  subnetIds: []

This seems to work sometimes (haven't fully tested it), but is there maybe a better way?

Seems to be a near identical error to ##382
Attempting to setup a super basic VPC just an elastic ip, public and app subnets. No db subnet nor Redshift, ElasticCache and Dax subnets getting an error message about RedshiftSubnetGroup.

Error: The CloudFormation template is invalid: Unresolved resource dependencies [RedshiftSubnetGroup] in the Outputs block of the template

vpcConfig:
    cidrBlock: '10.0.0.0/16'
    createNatGateway: 1
    createNetworkAcl: true
    createDbSubnet: false
    createParameters: true
    exportOutputs: true

The cloudformation-template-update-stack.json mentions all three subnet groups RedshiftSubnetGroup, ElastiCacheSubnetGroup, DAXSubnetGroup:

    "RedshiftSubnetGroup": {
      "Description": "Subnet Group for redshift",
      "Value": {
        "Ref": "RedshiftSubnetGroup"
      },
      "Export": {
        "Name": {
          "Fn::Sub": "${AWS::StackName}-RedshiftSubnetGroup"
        }
      }
    },
    "ElastiCacheSubnetGroup": {
      "Description": "Subnet Group for elasticache",
      "Value": {
        "Ref": "ElastiCacheSubnetGroup"
      },
      "Export": {
        "Name": {
          "Fn::Sub": "${AWS::StackName}-ElastiCacheSubnetGroup"
        }
      }
    },
    "DAXSubnetGroup": {
      "Description": "Subnet Group for dax",
      "Value": {
        "Ref": "DAXSubnetGroup"
      },
      "Export": {
        "Name": {
          "Fn::Sub": "${AWS::StackName}-DAXSubnetGroup"
        }
      }
    },

If I set createDbSubnet to false, and createParameters to true, sls deploy fails with this message:

An error occurred: ParameterDBSubnets - Property validation failure: [Length of value for property {/Value} is less than minimum allowed length {1}].

It seems you're trying to publish DBSubnet parameter even if createDbSubnet is set to false

If an RDS database is provisioned within the DB subnets and network ACLs are being used, only hosts or lambda functions within the Application subnets can access the database.

If a bastion host is provisioned, we should also allow the Public subnets access to the DB subnets through the DB network ACLs.

We created something like this:

plugins:
  - serverless-vpc-plugin

provider:
  name: aws
  runtime: nodejs12.x
  region: eu-central-1

custom:
  vpcConfig:
    cidrBlock: "172.16.0.0/16"
    createNatGateway: 2
    createNetworkAcl: true
    createDbSubnet: false # <-- NOTE: we have db subnets disabled
    createFlowLogs: true
    zones:
      - eu-central-1a
      - eu-central-1b

This creates a broken template, as it has outputs for db subnets, but no actual subnet:

"Outputs": {

    "RDSSubnetGroup": {
      "Description": "Subnet Group for rds",
      "Value": {
        "Ref": "RDSSubnetGroup"
      }
    },

The deployment fails with:

Error: The CloudFormation template is invalid: Unresolved resource dependencies [RDSSubnetGroup] in the Outputs block of the template

This works with only one AZ. Then no subnet group nor output will be created.

After deployment only Lambda Execution security group is assigned to lambda function. However function cannot connect to ElastiCache in this group. I had to manually add default security group to lambda, and create ElastiCache cluster with default security group in order to connect to redis properly.

It would be nice if we could specify an option to automatically assign vpc's default security group beside LambdaExec security group to lambda function during deployment .

Hi,

Is there a way to prevent the plugin from creating a DB subnet ?

I have no RDS and I only want 2 subnets: private and public. My lambda will run in the private subnet and route internet traffic via NAT with EIP in the public subnet.

In the interest of keeping my AWS resource config minimal, simple and secure I'd rather it didn't create the DB subnet.

My test config is below, and event when I specifying only one subnet, it creates three (db, app and public):

In provider:

vpc:
  securityGroupIds:
    - Ref: LambdaExecutionSecurityGroup
  subnetIds:
    - Ref: AppSubnet1

In custom:

vpcConfig:
  cidrBlock: "10.0.0.0/16"
  useNatGateway: true
  zones:
    - eu-west-1a
  services:
    - kms
    - secretsmanager

Thanks in advance.

This is more of a question than an issue - after making changes in the console to the app security group to allow outbound access for RDS, if I redeploy the vpc in the future will it overwrite/remove those changes?

Create Network ACL's around each subnet as follows:

• Public Subnets <-> App Subnets = ALLOW
• Public Subnets <-> DB Subnets = DENY
• App Subnets <-> DB Subnets = ALLOW

In #39, I added the ability to create a server in the Public subnet which acts as both a NAT Instance and a Bastion Host for the Application subnets.

Rethinking this, I should separate the two functions since they serve separate purposes from a security perspective (separation of duties):

• Bastion Host: inbound SSH access only
• NAT Instance: outbound HTTP/HTTPS access only

Provisioning an actual bastion host requires providing the SSH key pair name to use when creating the EC2 instance. The NAT instance doesn't require SSH access or access to the server at all (except for troubleshooting purposes).

Since the actual request in #36 was for a NAT instance, I think I can start on that first.

Add an example project to test IAM access information from within the VPC to S3. This will help ensure that the NACL, Security Groups, VPC Endpoint and S3 bucket policies all work as designed.

An error occurred: BastionSecurityGroup - CIDR block 208.157.160.147
/32 is malformed (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue;

When using the useNatGateway option, it will create a gateways per zone (so when using 6 zones - it will create 6 gateways + 6 EIP).

Sometimes, we need a limited number of EIP's.

Maybe useNatGateway should be either a boolean or a number, so when using a number it will limit the number of gateways:

useNatGateway: 2

Maybe do this in a separate project, but:

• Create a KMS key
• Create a logs
• Enable CloudTrail
• Enable AWS Config
• Enable VPC flow logs

I am using your plugin for a while now I think it is really a great plugin. (especially the example helped me a lot)

I tries to update to 1.0.1 from 0.10.0 and Cloudformation had a problem to delete the LambdaExecutionSecurityGroup. (error: resource sg-xxx has a dependent object) Cloudformation retried this for a while and then the deployment succeded.
The problem now is that the lambda cannot access the RDS instance. (I am using a RDS instance not a cluster)

DBSecurityGroup:
  Type: 'AWS::EC2::SecurityGroup'
  Properties:
    GroupDescription: Access to Postgres
    VpcId:
      Ref: VPC
    SecurityGroupIngress:
      - SourceSecurityGroupId:
          Ref: BastionHostSecurityGroup
        Description: Bastion access to Postgres
        IpProtocol: tcp
        FromPort: 5432
        ToPort: 5432
      - SourceSecurityGroupId:
          Ref: LambdaExecutionSecurityGroup
        Description: Lambda access to Postgres
        IpProtocol: tcp
        FromPort: 5432
        ToPort: 5432
    Tags:
      - Key: Name
        Value:
          'Fn::Join':
            - '-'
            - - Ref: 'AWS::StackName'
              - rds

and I am using it in the DB instance like this:

DBInstance:
  Type: 'AWS::RDS::DBInstance'
  Properties:
    AllocatedStorage: 20
    AllowMajorVersionUpgrade: true
    AutoMinorVersionUpgrade: true
    DBInstanceClass: db.t2.micro
    Port: 5432
    StorageType: gp2
    BackupRetentionPeriod: 7
    PreferredBackupWindow: '00:25-00:55'
    PreferredMaintenanceWindow: 'mon:03:37-mon:04:07'
    DBName: postgres
    Engine: postgres
    EngineVersion: '12.3'
    LicenseModel: postgresql-license
    DBSubnetGroupName:
      Ref: RDSSubnetGroup
    VPCSecurityGroups:
      - Ref: DBSecurityGroup
    AvailabilityZone:
      'Fn::Select':
        - 0
        - 'Fn::GetAZs': ''
    CopyTagsToSnapshot: true
    MasterUsername:
      'Fn::Join':
        - ''
        - - '{{resolve:secretsmanager:'
          - Ref: DBSecret
          - ':SecretString:username}}'
    MasterUserPassword:
      'Fn::Join':
        - ''
        - - '{{resolve:secretsmanager:'
          - Ref: DBSecret
          - ':SecretString:password}}'
    MultiAZ: false
    PubliclyAccessible: false
    Tags:
      - Key: Name
        Value:
          'Fn::Join':
            - '-'
            - - Ref: 'AWS::StackName'
              - rds

After the deployment I inspected the DBSecurityGroup in the AWS console and the two ingress security groups were successfully applied.
I rolled back for now, but I don't get why this is not working.

Your newest release v1.0.0 causes sls to fail with this error:

TypeError: awsService[method] is not a function

Using this very simple config:

service: vpc

provider:
  name: aws
  
plugins:
  - serverless-vpc-plugin

custom:
  vpcConfig:
    cidrBlock: '10.0.0.0/16'
    exportOutputs: true

There's a good chance I screwed something up after not fully comprehending the breaking changes that v1.0.0 brings. But if I screwed it up, chances are someone else will screw it up too. They'll google the error and his issue will guide them.

Hi,

I've got the following set up:

infrastructure/serverless.yml
apiLambda/serverless.yml

infrastructure/serverless.yml includes the RDS/IAM.yml like in your example.

Only the infrastructure.yml has the vpc plugin installed but I'm struggling to get apiLambda/serverless.yml to connect to the other VPC. I have outputExports set to true and tried setting:

provider:
  vpc: STACK-NAME-VPC

But when the lambda is deployed I can see that the VPC hasn't been connected.

Any ideas on how I can get to fix this would help me out a lot! Let me know if I can help by giving anymore information

First of all, this plugin is amazing!

Now I just want to make sure I'm not overthinking this: if I deploy with this plugin, and I create a RDS instance inside the "db" subnet, then Lambda cannot access the database by default, correct?

The only way I managed to make it work was:

plugins:
    - serverless-vpc-plugin

custom:
    vpcConfig:

resources:
    Resources:
        # RDS databases should be put inside this security group
        DBSecurityGroup:
            Type: 'AWS::EC2::SecurityGroup'
            Properties:
                GroupDescription: 'RDS database access from Lambda'
                VpcId: !Ref VPC
                SecurityGroupIngress:
                    -   Description: 'Allow inbound MySQL access from Lambda'
                        FromPort: 3306
                        IpProtocol: tcp
                        # Accept connections from the Lambda security group
                        SourceSecurityGroupId: !GetAtt LambdaExecutionSecurityGroup.GroupId
                        ToPort: 3306

i.e. I have to create a security group, and then use this one when creating the RDS instance.

Is that correct?

And if yes, would it make sense to have the plugin create this security group?

It is not clear, in the README, that the comment on line 7 of the sample code actually means "delete the following five lines that are here as examples of would be here otherwise". Not deleting them will throw errors in deployment.

Hi,

This looks like a great plugin, and might be exactly what I've been searching for. Thanks for taking the time to create and share it! But unfortunately I'm stumped in terms of usage.

Design intent

I've read the README and I've looked through (and deployed) the example directory and those gives me some guidance, but is there something else at a high level — a blog post or something — that I haven't seen in terms of documentation? I feel like perhaps I'm missing something obvious.

What is the purpose/design of the various subnets? What should I put into "public" vs "app" vs "db"? Are there others I've missed? What's the rationale behind this design? I'm guessing all this might have been in the link to the now shut-down Skyliner. I've searched the web and found nothing on it.

The best way to use the created resources

Also, how do I use the various subnets, security groups, etc this plugin creates? I presume there's a way to !Ref them as needed, but how? What names does it create?

Using the bastion host

Is there any documentation on the bastion host? I can see this as highly useful, but I can't seem to work out how to use it. I set it up but I get an error saying the key doesn't exist? I presume bastionHostKeyName must refer to something external, but what?

Fixing the example

And finally, FYI I pulled down the repo and tried to run sls deploy on the example directory but it fails out of the box right now due to an error on line 62 of serverless.yml:

  The CloudFormation template is invalid: Template error: every Fn::GetAtt object requires two non-empty parameters, the resource name and the resource attribute

The corrected line that works should be:

    'Fn::GetAtt': [ EFSAccessPoint, Arn ]

This was run on my environment:

  Your Environment Information ---------------------------
     Operating System:          darwin
     Node Version:              14.4.0
     Framework Version:         2.54.0 (local)
     Plugin Version:            5.4.3
     SDK Version:               4.2.6
     Components Version:        3.15.1

Thank you

Again, thanks for the plugin. I know this post may seem negative, but I really wish to get some better understanding so I can get some value out of it, and perhaps I can contribute to the project in terms of improved docs.

Cheers, Scott

Not able to export the CFN Output for SubnetIDs which is required for cross referencing with other stack. Added the tag in the serverless.yml but somehow its not converting the custom outputs. Using the below serverless.yml

service: vpc-creation-service

plugins:

• serverless-vpc-plugin

provider:

you do not need to provide the "vpc" section as this plugin will populate it automatically

name: aws
region: ap-southeast-2
vpc:
securityGroupIds:
- # plugin will add LambdaExecutionSecurityGroup to this list
subnetIds:
- # plugin will add the "Application" subnets to this list

custom:
vpcConfig:
cidrBlock: '172.1.0.0/16'

# if createNatGateway is a boolean "true", a NAT Gateway and EIP will be provisioned in each zone
# if createNatGateway is a number, that number of NAT Gateways will be provisioned
createNatGateway: 2

# When enabled, the DB subnet will only be accessible from the Application subnet
# Both the Public and Application subnets will be accessible from 0.0.0.0/0
createNetworkAcl: false

# Whether to create the DB subnet
#createDbSubnet: true

# Whether to enable VPC flow logging to an S3 bucket
createFlowLogs: false

# Whether to create a bastion host
createBastionHost: false
bastionHostKeyName: MyKey # required if creating a bastion host

# Whether to create a NAT instance
createNatInstance: false
zones:
      - ap-southeast-2a
      - ap-southeast-2b 


# By default, S3 and DynamoDB endpoints will be available within the VPC
# see https://docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints.html
# for a list of available service endpoints to provision within the VPC
# (varies per region)
# Output of created architecture

Outputs:
AppSubnet1:
Value:
Ref: AppSubnet1
AppSubnet2:
Value:
Ref: AppSubnet2
PublicSubnet1:
Value:
Ref: PublicSubnet1
PublicSubnet2:
Value:
Ref: PublicSubnet2
DynamodbVPCEndpoint:
Value:
Ref: DynamodbVPCEndpoint
S3VPCEndpoint:
Value:
Ref: S3VPCEndpoint
LambdaExecutionSecurityGroupId:
Value:
Ref: LambdaExecutionSecurityGroupId
vpc:
Value:
Ref: vpc

Problem

Version: 0.9.3

Configuration is empty:

plugins:
  - serverless-vpc-plugin

custom:
  vpcConfig:

With the above configuration, running serverless deploy fails with:

  An error occurred: PublicRoute2 - route table rtb-03701c7a6171e6432 and network gateway igw-0138227911f6b43af belong to different networks (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue; Request ID: 12b44f4c-f15b-4217-ab15-3ffc888c9d5f).

Fix

diff /tmp/old.js node_modules/serverless-vpc-plugin/src/vpc.js
195a196
>     DependsOn: ['InternetGatewayAttachment'],

Hi,

I have the problem that the EIP address has another value when I drop and recreate the stack.
I want use the same IP any time I have to recreate the stack.

I have in my infrastructure ec2-instances with EIPs. Could I assign this IP-addresses here too?

I see in natgw.js

function buildEIP(position) {
  const cfName = `EIP${position}`;
  return {
    [cfName]: {
      Type: 'AWS::EC2::EIP',
      Properties: {
        Domain: 'vpc'
      },
    },
  };
}

Could I add another line

function buildEIP(position) {
  const cfName = `EIP${position}`;
  return {
    [cfName]: {
      Type: 'AWS::EC2::EIP',
      Properties: {
        Domain: 'vpc',
        InstanceId : 'logical-name-of-myec2-instance'
      },
    },
  };
}

or is this idea totally wrong?

In https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-eip.html
I see a parameter "PublicIpv4Pool" but I do not own a address pool. However I could create a further EIP in aws-console. But how could I assign/use this EIP in this stack?

As bastion instances are not used heavily it makes sense to be able to stop them to reduce costs.
ie after deployment of vpc and using bastion for setup purposes and initial checks, stop the instance in the ec2 console.

It was previously possible but I cannot remember which version of this plugin.

Currently the Bastion and NatInstance ec2 instances throw this error when trying to stop them
"Failed to stop the instance i-xxxxxxxxx
You can't stop the Spot Instance 'i-xxxxxx' because it is associated with a one-time Spot Instance request. You can only stop Spot Instances associated with persistent Spot Instance requests"

Can the spot instance type be adjusted to allow this please?

By default, without a NAT Instance or NAT Gateway, the subnets associated with the functions should route all traffic to the Internet Gateway. Hence the functions should have the ability to make outbound requests. However, all the outgoing traffics are blocked and timeout.

This issue can be resolved by provisioning a NAT instance/gateway in v0.6.0.

serverless.yml

service: serverless-vpc-plugin-example

frameworkVersion: '>=1.40.0 <2.0.0'

provider:
  name: aws
  runtime: nodejs8.10
  stage: dev
  region: us-west-2
  memorySize: 128
  timeout: 6

functions:
  hello:
    handler: handler.hello
    events:
      - http:
          path: /
          method: get

custom:
  vpcConfig:
    cidrBlock: '10.123.0.0/16'
#    createNatInstance: true

plugins:
  - serverless-vpc-plugin
  - serverless-webpack

handler.js

'use strict'

const axios = require('axios')

module.exports.hello = async (event) => {
  const response = await axios.post('https://httpbin.org/post', { some: 'data' })
  return {
    statusCode: 200,
    body: JSON.stringify({
      message: 'Go Serverless v1.0! Your function executed successfully!',
      input: event,
      response: response.data
    })
  }
}

With NAT instance:

{
  "message": "Go Serverless v1.0! Your function executed successfully!",
  "input": {
    ...
  },
  "response": {
    "args": {},
    "data": "{\"some\":\"data\"}",
    "files": {},
    "form": {},
    "headers": {
      "Accept": "application/json, text/plain, */*",
      "Content-Length": "15",
      "Content-Type": "application/json;charset=utf-8",
      "Host": "httpbin.org",
      "User-Agent": "axios/0.18.0"
    },
    "json": {
      "some": "data"
    },
    "origin": "...",
    "url": "https://httpbin.org/post"
  }
}

Without NAT instance:

{
  "message": "Internal server error"
}

Sometimes, you don't want to pay for the NATGateway especially in the dev environment :P

Looks like an interesting plugin and I'm trying to give it a go.

Using version: "serverless-vpc-plugin": "^1.0.0"

My config looks like this but get an error thrown:

  An error occurred: BastionSecurityGroup - CIDR block ##.##.##.###
  /32 is malformed (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue; Request ID: #########-####-####-####-###########; Proxy: null).

 vpcConfig:
    cidrBlock: '10.0.0.0/16'
 
    # if createNatGateway is a boolean "true", a NAT Gateway and EIP will be provisioned in each zone 
    # if createNatGateway is a number, that number of NAT Gateways will be provisioned 
    # createNatGateway: 2
 
    # When enabled, the DB subnet will only be accessible from the Application subnet 
    # Both the Public and Application subnets will be accessible from 0.0.0.0/0 
    createNetworkAcl: true
 
    # Whether to create the DB subnet 
    createDbSubnet: true
 
    # Whether to enable VPC flow logging to an S3 bucket 
    createFlowLogs: false
 
    # Whether to create a bastion host 
    createBastionHost: true
    bastionHostKeyName: desktop # required if creating a bastion host 
 
    # Whether to create a NAT instance 
    createNatInstance: true
 
    # Whether to create AWS Systems Manager (SSM) Parameters 
    createParameters: true
 
    # Optionally specify AZs (defaults to auto-discover all availabile AZs) 
    zones:
      - eu-central-1a
      - eu-central-1b
 
    # By default, S3 and DynamoDB endpoints will be available within the VPC 
    # see https://docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints.html 
    # for a list of available service endpoints to provision within the VPC 
    # (varies per region) 
    services:
      - kms
      - secretsmanager
 
    # Optionally specify subnet groups to create. If not provided, subnet groups 
    # for RDS, Redshift, ElasticCache and DAX will be provisioned. 
    subnetGroups:
      - rds
 
    # Whether to export stack outputs so it may be consumed by other stacks 
    exportOutputs: true

Any thoughts on what is going on?

Hi, thanks for this awesome plugin!

Is it possible to obtain the list of all the required IAM permissions in order to fully deploy a serverless solution with VPC enabled?

Thank you.

Using serverless-vpc-plugin I'm having trouble connecting it all. I have a Lambda-function triggered via API Gateway that I want to execute a call to elastiCache. It deploys just fine but my lambda-function just timeouts after performing a redisClient.get ... 😢 I think the problem lies in missing access credentials for my lambda to the VPC where the elastiCache resides.

I have scavenged the darkest corners of the deep web in search for a solution for more than two full days, I have come up short... I need your help! 🤓

Here's my serverless.yml for the network:

service: obb-network
plugins:
  - serverless-vpc-plugin
provider:
  name: aws
  runtime: nodejs12.x
  stage: dev
  region: eu-central-1
  profile:
    obb
custom:
  vpcConfig:
    cidrBlock: '10.0.0.0/16'
     createNatGateway: false
    createNetworkAcl: false
    createDbSubnet: true
    createFlowLogs: false
    createBastionHost: false 
    createNatInstance: false
    subnetGroups:
      - elasticache
    exportOutputs: true

And here's my serverless.yml for the API Gateway and elastiCache:

service: obb-api
custom:
  networkStack: obb-network-${self:provider.stage} # this links the above network serverless
provider:
  name: aws
  runtime: nodejs12.x
  stage: dev
  region: eu-central-1
  profile:
    obb
  environment:
    redisHost:
      "Fn::GetAtt": [ElasticCacheCluster, RedisEndpoint.Address]
  iamRoleStatements: # perhaps this will provide passage to elastiCache 
    - Effect: Allow
      Action:
        - "elasticache:*"
      Resource: "*"

functions:
  hello:
    handler: hello.main
    events:
      - http:
          path: /
          method: get
          cors: true
          authorizer: aws_iam
    role: LambdaVPCAccessExecutionRole # perhaps this will provide passage to elastiCache 
    vpc: 
    securityGroupIds:
      - Fn::GetAtt: CacheSecurityGroup.GroupId
    subnetIds:
      - Fn::ImportValue: !Sub ${self:custom.networkStack}-ElastiCacheSubnetGroup

    # Role to append to lambda in order to connect to ElastiCache, I guess
    LambdaVPCAccessExecutionRole:
      Type: "AWS::IAM::Role"
      Properties:
        RoleName: "${self:service}-${self:provider.stage}-${self:provider.region}-LambdaVPCAccessExecutionRole"
        AssumeRolePolicyDocument:
          Statement:
            - Effect: Allow
              Principal:
                Service: "lambda.amazonaws.com"
              Action: "sts:AssumeRole"
        Policies:
          - PolicyName: LambdaVPCAccessExecutionRolePolicy
            PolicyDocument:
              Statement:
                - Effect: Allow
                  Action:
                    - "elasticache:*"
                  Resource: "*"
        ManagedPolicyArns:
          - "arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole"

    # Elasticache needs an inbound rule to open port 6379, I guess
    CacheSecurityGroup:
      Type: AWS::EC2::SecurityGroup
      Properties:
        GroupDescription: Ingress for Redis Cluster
        VpcId:
          Fn::ImportValue: !Sub ${self:custom.networkStack}-VPC
        SecurityGroupIngress:
          - IpProtocol: tcp
            FromPort: 6379
            ToPort: 6379
            SourceSecurityGroupId:
              Fn::ImportValue: !Sub ${self:custom.networkStack}-LambdaExecutionSecurityGroupId

    # Set up elasticache cluster, I guess
    ElasticCacheCluster:
      Type: AWS::ElastiCache::CacheCluster
      Properties:
        AutoMinorVersionUpgrade: true
        Engine: redis
        CacheNodeType: cache.t2.micro
        NumCacheNodes: 1
        CacheSubnetGroupName:
          Fn::ImportValue: !Sub ${self:custom.networkStack}-ElastiCacheSubnetGroup
        VpcSecurityGroupIds:
          - !Ref CacheSecurityGroup

Getting this error while setting up a VPC using this plugin:

service: commercetools

plugins:
  - serverless-vpc-plugin

provider:
  name: aws
  region: eu-central-1
  stage: stg
  vpc:
    securityGroupIds:
      -  # plugin will add LambdaExecutionSecurityGroup to this list
    subnetIds:
      -  # plugin will add the "Application" subnets to this list

custom:
  vpcConfig:
    enabled: true
    cidrBlock: '10.30.0.0/16'
    createNatGateway: 3
    createNetworkAcl: false
    createDbSubnet: false
    createFlowLogs: false
    createBastionHost: false
    createNatInstance: false
    createParameters: false
    services:
      - kms
      - secretsmanager
    exportOutputs: false

I'm fairly new to VPC's and haven't tried to configure this but I ask anyways.

Wouldn't Egress-Only Internet Gateways be a viable alternative to nat gatways, provided all target networks use ipv6?

Is there a way to reuse the same VPCs for all the stages or environments?

This is related to #36, the AMI ID in this line is hardcoded, hence can only bring up from the us-east-1 region.

I'm struggling to connect a lambda to an RDS instance deployed into the RDS DB group. The Lambda is deployed into the Application Subnet

Issue seems to be IP blocked and Lambdas cannot get access to the IP range of the RDS DB Subnet. The Lambda can correctly reach the internet.

Any advice or tips for troubleshooting would be appreciated.

service: test-vpc

plugins:
  - serverless-vpc-plugin

provider:
  name: aws
  stage: dev
  region: eu-central-1
custom:
  dotenv: 
    logging: true
  REGION: ${opt:region, self:provider.region}
  STAGE: ${opt:stage, self:provider.stage}
  vpcConfig:
    cidrBlock: '10.0.0.0/16'
 
    # if createNatGateway is a boolean "true", a NAT Gateway and EIP will be provisioned in each zone 
    # if createNatGateway is a number, that number of NAT Gateways will be provisioned 
    # createNatGateway: 1
 
    # When enabled, the DB subnet will only be accessible from the Application subnet 
    # Both the Public and Application subnets will be accessible from 0.0.0.0/0 
    createNetworkAcl: true
 
    # Whether to create the DB subnet 
    createDbSubnet: true
 
    # Whether to enable VPC flow logging to an S3 bucket 
    createFlowLogs: false

 
    # Whether to create a bastion host 
    createBastionHost: true
    bastionHostKeyName: REPLACED# required if creating a bastion host 
 
    # Whether to create a NAT instance 
    createNatInstance: true
 
    # Whether to create AWS Systems Manager (SSM) Parameters 
    createParameters: true
 
    # Optionally specify AZs (defaults to auto-discover all availabile AZs) 
    zones:
      - eu-central-1a
      - eu-central-1b
 
    # By default, S3 and DynamoDB endpoints will be available within the VPC 
    # see https://docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints.html 
    # for a list of available service endpoints to provision within the VPC 
    # (varies per region) 
    services:
      - kms
      - secretsmanager
 
    # Optionally specify subnet groups to create. If not provided, subnet groups 
    # for RDS, Redshift, ElasticCache and DAX will be provisioned. 
    subnetGroups:
      - rds
 
    # Whether to export stack outputs so it may be consumed by other stacks 
    exportOutputs: true

Currently, each stage provisions the VPC settings again, is it possible to reuse the same VPC for all stages? Is there some trickery that could be done?

Getting this error while setting up a VPC using this plugin:

provider:
  name: aws
  runtime: nodejs12.x
  lambdaHashingVersion: '20201221'
  stage: ${opt:stage, 'dev'}
  region: ${file(serverless.env.yml):${self:custom.stage}.REGION}
  vpc:
    securityGroupIds:
      - sg-xxxxxxxx
    subnetIds:
      - subnet-xxxxxx
      - subnet-xxxxxx
      - subnet-xxxxxx

Both subnet and security groups are there in the same VPC.

rolling back to 0.10.0 resolves an issue

Running with SLS_DEBUG=* does not help much:

When I try to use createBastionHost and bastionHostKeyName, deploy fails with

BastionAutoScalingGroup | CREATE_FAILED

When using the plugin: serverless-plugin-split-stacks many parameters cannot be referenced on the second function. The first function in the serverless.yml will include the proper parameters, however, 2nd+ functions will not contain the proper resolutions.

It appears that this is happening due to the way they are exported. As such, you will receive an error: "Template format error: Unresolved resource dependencies [AppSubnet1Parameter, AppSubnet3Parameter, AppSubnet4Parameter, AppSubnet2Parameter, LambdaExecutionSecurityGroupParameter] in the Resources block of the template." when defining out the remainder of the handlers.

The following is my custom configuration:

custom:
  splitStacks:
    perFunction: true
    perType: false
    perGroupFunction: false
  vpcConfig:
    cidrBlock: '10.10.0.0/16'
    createBastionHost: false
    createDbSubnet: true
    createFlowLogs: false
    createNatGateway: true
    createNatInstance: false
    createNetworkAcl: true
    zones:
      - us-west-2a
      - us-west-2b
      - us-west-2c
      - us-west-2d
    subnetGroups:
      - rds
    exportOutputs: true

The function definitions are rather simplistic:

functions:
  networkList:
    handler: handler.networkList
    events:
      - http:
          method: get
          path: network
          cors: true
  networkRead:
    handler: handler.networkRead
    events:
      - http:
          method: get
          path: network/{id}
          cors: true
          request:
            parameters:
              path:
                id: true

The outputted information from the split stack shows that the parameters section within the first function contains the proper parameter types, whereas the second function will attempt to refer to the parameters but not actually define them.

Hi there - first of all, thanks for this plugin! It saves a lot of config time!

I was wondering if you could provide a little more examples on how to get the NAT to work. In short, I need to have a Lambda that can write to EFS and reach the internet. If I just create my own VPC for the EFS then the lambda can read/write to it, but I can't get out to the internet. I was hoping I could use this plugin to not have to manually set up all the NAT stuff, but I'm getting lost.

NAT, subnets, routes... none are my forte really, so hoping for a little help/example.

Placing the EFS on the AppSubnet, results in the lambda not being able to connect to it. Here's what I got so far:

resources:
  Resources:
    SharedFs:
      Type: AWS::EFS::FileSystem
      Properties:
        BackupPolicy:
          Status: ENABLED
        PerformanceMode: generalPurpose
        Encrypted: true
        LifecyclePolicies:
          - TransitionToIA: AFTER_30_DAYS

    MountTarget:
      Type: AWS::EFS::MountTarget
      Properties:
        FileSystemId: !Ref SharedFs
        SubnetId: !Ref AppSubnet1
        SecurityGroups:
          - !Ref AppSecurityGroup

    AccessPointResource:
      Type: AWS::EFS::AccessPoint
      Properties:
        FileSystemId: !Ref SharedFs
        PosixUser:
          Uid: 1001
          Gid: 1001
        RootDirectory:
          CreationInfo:
            OwnerGid: 1001
            OwnerUid: 1001
            Permissions: 770
          Path: /efs

But then I get:
The function couldn't connect to the Amazon EFS file system with access point.......... Check your network configuration and try again. (Service: AWSLambda; Status Code: 408; Error Code: EFSMountConnectivityException

Should it even be on the AppSubnet? Or use the public subnet?

I am trying to disable the plugin for some stages by providing enabled: false (keeping the other parameters same. ). But it's still generating VPC and subnets when I run serverless deploy. I also tried hardcoding the value like this: (just making sure, I am not doing something wrong while doing it conditionally)

  vpcConfig:
    # Whether plugin is enabled. Can be used to selectively disable plugin
    # on certain stages or configurations. Defaults to true.
    enabled: false

    cidrBlock: '10.0.0.0/16'
.....

I would like to setup VPC peering with this plugin

provider:
    vpc:
        securityGroupIds:
            - ${AppSecurityGroup}
        subnetIds: ${AppSubnets~split}

custom:
    vpcConfig:
        cidrBlock: '10.0.0.0/16'
        createNatGateway: 1
        createDbSubnet: false

seems to be working.
But when I want to use it for my resources its broken.

resources:
    Resources:
        MountTargetA:
            Type: AWS::EFS::MountTarget
            Properties:
                SubnetId: ${AppSubnet{1}}
                SecurityGroups:
                    - ${AppSecurityGroup}
        MountTargetB:
            Type: AWS::EFS::MountTarget
            Properties:
                SubnetId: ${AppSubnet{2}}
                SecurityGroups:
                    - ${AppSecurityGroup}
        MountTargetC:
            Type: AWS::EFS::MountTarget
            Properties:
                SubnetId: ${AppSubnet{3}}
                SecurityGroups:
                    - ${AppSecurityGroup}

Error message:

An error occurred: EFSCacheMountTargetResourceA - 2 validation errors detected: Value '${AppSubnet{1}}' at 'subnetId' failed to satisfy constraint: Member must satisfy regular expression pattern: ^subnet-[0-9a-f]{8,40}$; Value '[${AppSecurityGroup}]' at 'securityGroups' failed to satisfy constraint: Member must satisfy constraint: [Member must have length less than or equal to 43, Member must have length greater than or equal to 11, Member must satisfy regular expression pattern: ^sg-[0-9a-f]{8,40}] (Service: AmazonElasticFileSystem; Status Code: 400; Error Code: ValidationException; Request ID: 569b3c04-42f3-47db-9cb8-1c630b090b28; Proxy: null).

Not working with ${self:provider.vpc.subnetIds.1}, too.

When I don't list availability zones, the plugin uses all 6 in us-east1. Is this overkill? Would a better default be to randomly pick 3?

When using this plugin, the default package exclude option no longer works as described here:

https://serverless.com/framework/docs/providers/aws/guide/packaging/

This is my config, but src is still being included in the artifact.

package:
  exclude:
    - src/**