smoketurner / serverless-vpc-plugin Goto Github PK
View Code? Open in Web Editor NEWServerless Plugin to create a VPC
License: MIT License
Serverless Plugin to create a VPC
License: MIT License
Something in a recent Serverless upgrade broke aws-sdk-mock so the tests are failing
Hi,
I try to to create the stack with only ONE availability zone.
In my serverless.yml I do something like:
vpcConfig:
createNatGateway: true
zones:
- eu-central-1b
But I get an error when I deploy -->
Error: The CloudFormation template is invalid: Unresolved resource dependencies [RDSSubnetGroup] in the Outputs block of the template
All is okay when I omit the zone entry. In this case three external IPs are created. (But I want only one.)
What is the problem?
Hey, thanks for this awesome plugin. I use it all the time. Currently, I am including serverless-vpc-plugin
in the same stack with my microservices. This is great because your plugin automatically added all Subnets and SGs to my functions. However, as I'm sure you know, that takes up about ~70-80 of the 200 resource limit.
So in an attempt to free up some space in my CF templates I am researching how I can separate VPC resources into a separate serverless.yaml, export the values needed, and declare them in my microservice's serverless.yaml
files.
You may recommend doing that this way:
provider:
vpc:
subnetIds:
- 'Fn::ImportValue': myvpcstack-${opt:stage}-AppSubnet1
- 'Fn::ImportValue': myvpcstack-${opt:stage}-AppSubnet2
- ...
But I think we can do better than that. We should be able to dynamically discover the subnets that were created, rather than "hardcoding" them. The biggest problem this causes is that not all regions have the same number of AZs(subnets). If I had AppSubnet[1-6] declared, and tried to launch it in an non-US AWS region, it would error because that region only has 3 AZs(subnets) available.
Please create a new stack output, call it AllAppSubnets
(?), and the value should be all AppSubnets{i}
as a single string separated by commas. In us-east-1
this would have 6 values. E.g. subnet-1234, subnet-2345, subnet-3456, subnet-4567, subnet-5678, subnet-6789
This is a common pattern used in serverless.yaml
for this type of situation:
vpc:
subnetIds:
"Fn::Split":
- ","
- 'Fn::ImportValue': myvpcstack-${opt:stage}-AllAppSubnets
Possible to set up a multi-AZ public & private subnet VPC? E.g.:
public 1a subnet,
private 1a subnet,
public 1b subnet,
private 1b subnet,
...
Thanks!
Hi,
Are there any plans to include DocumentDB as part of the supported services? Its' CloudFormation needs are very similar to RDS. I think it would be somewhat simple.
Regards
Hi there,
I was wondering if there's a way to opt specific functions out of the global VPC config that this plugin creates?
Use case is: I have some functions that are replicated to Lambda @ Edge, and thus the source functions are not allowed to be inside a VPC.
What I'm doing currently is adding the following "empty" VPC config to the function:
vpc:
securityGroupIds: []
subnetIds: []
This seems to work sometimes (haven't fully tested it), but is there maybe a better way?
Seems to be a near identical error to ##382
Attempting to setup a super basic VPC just an elastic ip, public and app subnets. No db subnet nor Redshift, ElasticCache and Dax subnets getting an error message about RedshiftSubnetGroup.
Error: The CloudFormation template is invalid: Unresolved resource dependencies [RedshiftSubnetGroup] in the Outputs block of the template
vpcConfig:
cidrBlock: '10.0.0.0/16'
createNatGateway: 1
createNetworkAcl: true
createDbSubnet: false
createParameters: true
exportOutputs: true
The cloudformation-template-update-stack.json mentions all three subnet groups RedshiftSubnetGroup, ElastiCacheSubnetGroup, DAXSubnetGroup:
"RedshiftSubnetGroup": {
"Description": "Subnet Group for redshift",
"Value": {
"Ref": "RedshiftSubnetGroup"
},
"Export": {
"Name": {
"Fn::Sub": "${AWS::StackName}-RedshiftSubnetGroup"
}
}
},
"ElastiCacheSubnetGroup": {
"Description": "Subnet Group for elasticache",
"Value": {
"Ref": "ElastiCacheSubnetGroup"
},
"Export": {
"Name": {
"Fn::Sub": "${AWS::StackName}-ElastiCacheSubnetGroup"
}
}
},
"DAXSubnetGroup": {
"Description": "Subnet Group for dax",
"Value": {
"Ref": "DAXSubnetGroup"
},
"Export": {
"Name": {
"Fn::Sub": "${AWS::StackName}-DAXSubnetGroup"
}
}
},
If I set createDbSubnet
to false
, and createParameters
to true
, sls deploy
fails with this message:
An error occurred: ParameterDBSubnets - Property validation failure: [Length of value for property {/Value} is less than minimum allowed length {1}].
It seems you're trying to publish DBSubnet parameter even if createDbSubnet
is set to false
If an RDS database is provisioned within the DB subnets and network ACLs are being used, only hosts or lambda functions within the Application subnets can access the database.
If a bastion host is provisioned, we should also allow the Public subnets access to the DB subnets through the DB network ACLs.
We created something like this:
plugins:
- serverless-vpc-plugin
provider:
name: aws
runtime: nodejs12.x
region: eu-central-1
custom:
vpcConfig:
cidrBlock: "172.16.0.0/16"
createNatGateway: 2
createNetworkAcl: true
createDbSubnet: false # <-- NOTE: we have db subnets disabled
createFlowLogs: true
zones:
- eu-central-1a
- eu-central-1b
This creates a broken template, as it has outputs for db subnets, but no actual subnet:
"Outputs": {
"RDSSubnetGroup": {
"Description": "Subnet Group for rds",
"Value": {
"Ref": "RDSSubnetGroup"
}
},
The deployment fails with:
Error: The CloudFormation template is invalid: Unresolved resource dependencies [RDSSubnetGroup] in the Outputs block of the template
This works with only one AZ. Then no subnet group nor output will be created.
After deployment only Lambda Execution security group is assigned to lambda function. However function cannot connect to ElastiCache in this group. I had to manually add default security group to lambda, and create ElastiCache cluster with default security group in order to connect to redis properly.
It would be nice if we could specify an option to automatically assign vpc's default security group beside LambdaExec security group to lambda function during deployment .
Hi,
Is there a way to prevent the plugin from creating a DB subnet ?
I have no RDS and I only want 2 subnets: private and public. My lambda will run in the private subnet and route internet traffic via NAT with EIP in the public subnet.
In the interest of keeping my AWS resource config minimal, simple and secure I'd rather it didn't create the DB subnet.
My test config is below, and event when I specifying only one subnet, it creates three (db, app and public):
In provider
:
vpc:
securityGroupIds:
- Ref: LambdaExecutionSecurityGroup
subnetIds:
- Ref: AppSubnet1
In custom
:
vpcConfig:
cidrBlock: "10.0.0.0/16"
useNatGateway: true
zones:
- eu-west-1a
services:
- kms
- secretsmanager
Thanks in advance.
This is more of a question than an issue - after making changes in the console to the app security group to allow outbound access for RDS, if I redeploy the vpc in the future will it overwrite/remove those changes?
Create Network ACL's around each subnet as follows:
In #39, I added the ability to create a server in the Public subnet which acts as both a NAT Instance and a Bastion Host for the Application subnets.
Rethinking this, I should separate the two functions since they serve separate purposes from a security perspective (separation of duties):
Provisioning an actual bastion host requires providing the SSH key pair name to use when creating the EC2 instance. The NAT instance doesn't require SSH access or access to the server at all (except for troubleshooting purposes).
Since the actual request in #36 was for a NAT instance, I think I can start on that first.
Add an example project to test IAM access information from within the VPC to S3. This will help ensure that the NACL, Security Groups, VPC Endpoint and S3 bucket policies all work as designed.
An error occurred: BastionSecurityGroup - CIDR block 208.157.160.147
/32 is malformed (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue;
When using the useNatGateway
option, it will create a gateways per zone (so when using 6 zones - it will create 6 gateways + 6 EIP).
Sometimes, we need a limited number of EIP's.
Maybe useNatGateway
should be either a boolean or a number, so when using a number it will limit the number of gateways:
useNatGateway: 2
Maybe do this in a separate project, but:
I am using your plugin for a while now I think it is really a great plugin. (especially the example helped me a lot)
I tries to update to 1.0.1 from 0.10.0 and Cloudformation had a problem to delete the LambdaExecutionSecurityGroup. (error: resource sg-xxx has a dependent object) Cloudformation retried this for a while and then the deployment succeded.
The problem now is that the lambda cannot access the RDS instance. (I am using a RDS instance not a cluster)
DBSecurityGroup:
Type: 'AWS::EC2::SecurityGroup'
Properties:
GroupDescription: Access to Postgres
VpcId:
Ref: VPC
SecurityGroupIngress:
- SourceSecurityGroupId:
Ref: BastionHostSecurityGroup
Description: Bastion access to Postgres
IpProtocol: tcp
FromPort: 5432
ToPort: 5432
- SourceSecurityGroupId:
Ref: LambdaExecutionSecurityGroup
Description: Lambda access to Postgres
IpProtocol: tcp
FromPort: 5432
ToPort: 5432
Tags:
- Key: Name
Value:
'Fn::Join':
- '-'
- - Ref: 'AWS::StackName'
- rds
and I am using it in the DB instance like this:
DBInstance:
Type: 'AWS::RDS::DBInstance'
Properties:
AllocatedStorage: 20
AllowMajorVersionUpgrade: true
AutoMinorVersionUpgrade: true
DBInstanceClass: db.t2.micro
Port: 5432
StorageType: gp2
BackupRetentionPeriod: 7
PreferredBackupWindow: '00:25-00:55'
PreferredMaintenanceWindow: 'mon:03:37-mon:04:07'
DBName: postgres
Engine: postgres
EngineVersion: '12.3'
LicenseModel: postgresql-license
DBSubnetGroupName:
Ref: RDSSubnetGroup
VPCSecurityGroups:
- Ref: DBSecurityGroup
AvailabilityZone:
'Fn::Select':
- 0
- 'Fn::GetAZs': ''
CopyTagsToSnapshot: true
MasterUsername:
'Fn::Join':
- ''
- - '{{resolve:secretsmanager:'
- Ref: DBSecret
- ':SecretString:username}}'
MasterUserPassword:
'Fn::Join':
- ''
- - '{{resolve:secretsmanager:'
- Ref: DBSecret
- ':SecretString:password}}'
MultiAZ: false
PubliclyAccessible: false
Tags:
- Key: Name
Value:
'Fn::Join':
- '-'
- - Ref: 'AWS::StackName'
- rds
After the deployment I inspected the DBSecurityGroup in the AWS console and the two ingress security groups were successfully applied.
I rolled back for now, but I don't get why this is not working.
Your newest release v1.0.0 causes sls to fail with this error:
TypeError: awsService[method] is not a function
Using this very simple config:
service: vpc
provider:
name: aws
plugins:
- serverless-vpc-plugin
custom:
vpcConfig:
cidrBlock: '10.0.0.0/16'
exportOutputs: true
There's a good chance I screwed something up after not fully comprehending the breaking changes that v1.0.0 brings. But if I screwed it up, chances are someone else will screw it up too. They'll google the error and his issue will guide them.
Hi,
I've got the following set up:
infrastructure/serverless.yml
apiLambda/serverless.yml
infrastructure/serverless.yml includes the RDS/IAM.yml like in your example.
Only the infrastructure.yml has the vpc plugin installed but I'm struggling to get apiLambda/serverless.yml to connect to the other VPC. I have outputExports set to true and tried setting:
provider:
vpc: STACK-NAME-VPC
But when the lambda is deployed I can see that the VPC hasn't been connected.
Any ideas on how I can get to fix this would help me out a lot! Let me know if I can help by giving anymore information
First of all, this plugin is amazing!
Now I just want to make sure I'm not overthinking this: if I deploy with this plugin, and I create a RDS instance inside the "db" subnet, then Lambda cannot access the database by default, correct?
The only way I managed to make it work was:
plugins:
- serverless-vpc-plugin
custom:
vpcConfig:
resources:
Resources:
# RDS databases should be put inside this security group
DBSecurityGroup:
Type: 'AWS::EC2::SecurityGroup'
Properties:
GroupDescription: 'RDS database access from Lambda'
VpcId: !Ref VPC
SecurityGroupIngress:
- Description: 'Allow inbound MySQL access from Lambda'
FromPort: 3306
IpProtocol: tcp
# Accept connections from the Lambda security group
SourceSecurityGroupId: !GetAtt LambdaExecutionSecurityGroup.GroupId
ToPort: 3306
i.e. I have to create a security group, and then use this one when creating the RDS instance.
Is that correct?
And if yes, would it make sense to have the plugin create this security group?
It is not clear, in the README, that the comment on line 7 of the sample code actually means "delete the following five lines that are here as examples of would be here otherwise". Not deleting them will throw errors in deployment.
Hi,
This looks like a great plugin, and might be exactly what I've been searching for. Thanks for taking the time to create and share it! But unfortunately I'm stumped in terms of usage.
I've read the README and I've looked through (and deployed) the example directory and those gives me some guidance, but is there something else at a high levelย โย a blog post or something โ that I haven't seen in terms of documentation? I feel like perhaps I'm missing something obvious.
What is the purpose/design of the various subnets? What should I put into "public" vs "app" vs "db"? Are there others I've missed? What's the rationale behind this design? I'm guessing all this might have been in the link to the now shut-down Skyliner. I've searched the web and found nothing on it.
Also, how do I use the various subnets, security groups, etc this plugin creates? I presume there's a way to !Ref
them as needed, but how? What names does it create?
Is there any documentation on the bastion host? I can see this as highly useful, but I can't seem to work out how to use it. I set it up but I get an error saying the key doesn't exist? I presume bastionHostKeyName
must refer to something external, but what?
And finally, FYI I pulled down the repo and tried to run sls deploy
on the example directory but it fails out of the box right now due to an error on line 62 of serverless.yml
:
The CloudFormation template is invalid: Template error: every Fn::GetAtt object requires two non-empty parameters, the resource name and the resource attribute
The corrected line that works should be:
'Fn::GetAtt': [ EFSAccessPoint, Arn ]
This was run on my environment:
Your Environment Information ---------------------------
Operating System: darwin
Node Version: 14.4.0
Framework Version: 2.54.0 (local)
Plugin Version: 5.4.3
SDK Version: 4.2.6
Components Version: 3.15.1
Again, thanks for the plugin. I know this post may seem negative, but I really wish to get some better understanding so I can get some value out of it, and perhaps I can contribute to the project in terms of improved docs.
Cheers, Scott
Not able to export the CFN Output for SubnetIDs which is required for cross referencing with other stack. Added the tag in the serverless.yml but somehow its not converting the custom outputs. Using the below serverless.yml
service: vpc-creation-service
plugins:
provider:
name: aws
region: ap-southeast-2
vpc:
securityGroupIds:
- # plugin will add LambdaExecutionSecurityGroup to this list
subnetIds:
- # plugin will add the "Application" subnets to this list
custom:
vpcConfig:
cidrBlock: '172.1.0.0/16'
# if createNatGateway is a boolean "true", a NAT Gateway and EIP will be provisioned in each zone
# if createNatGateway is a number, that number of NAT Gateways will be provisioned
createNatGateway: 2
# When enabled, the DB subnet will only be accessible from the Application subnet
# Both the Public and Application subnets will be accessible from 0.0.0.0/0
createNetworkAcl: false
# Whether to create the DB subnet
#createDbSubnet: true
# Whether to enable VPC flow logging to an S3 bucket
createFlowLogs: false
# Whether to create a bastion host
createBastionHost: false
bastionHostKeyName: MyKey # required if creating a bastion host
# Whether to create a NAT instance
createNatInstance: false
zones:
- ap-southeast-2a
- ap-southeast-2b
# By default, S3 and DynamoDB endpoints will be available within the VPC
# see https://docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints.html
# for a list of available service endpoints to provision within the VPC
# (varies per region)
# Output of created architecture
Outputs:
AppSubnet1:
Value:
Ref: AppSubnet1
AppSubnet2:
Value:
Ref: AppSubnet2
PublicSubnet1:
Value:
Ref: PublicSubnet1
PublicSubnet2:
Value:
Ref: PublicSubnet2
DynamodbVPCEndpoint:
Value:
Ref: DynamodbVPCEndpoint
S3VPCEndpoint:
Value:
Ref: S3VPCEndpoint
LambdaExecutionSecurityGroupId:
Value:
Ref: LambdaExecutionSecurityGroupId
vpc:
Value:
Ref: vpc
Version: 0.9.3
Configuration is empty:
plugins:
- serverless-vpc-plugin
custom:
vpcConfig:
With the above configuration, running serverless deploy
fails with:
An error occurred: PublicRoute2 - route table rtb-03701c7a6171e6432 and network gateway igw-0138227911f6b43af belong to different networks (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue; Request ID: 12b44f4c-f15b-4217-ab15-3ffc888c9d5f).
diff /tmp/old.js node_modules/serverless-vpc-plugin/src/vpc.js
195a196
> DependsOn: ['InternetGatewayAttachment'],
Hi,
I have the problem that the EIP address has another value when I drop and recreate the stack.
I want use the same IP any time I have to recreate the stack.
I have in my infrastructure ec2-instances with EIPs. Could I assign this IP-addresses here too?
I see in natgw.js
function buildEIP(position) {
const cfName = `EIP${position}`;
return {
[cfName]: {
Type: 'AWS::EC2::EIP',
Properties: {
Domain: 'vpc'
},
},
};
}
Could I add another line
function buildEIP(position) {
const cfName = `EIP${position}`;
return {
[cfName]: {
Type: 'AWS::EC2::EIP',
Properties: {
Domain: 'vpc',
InstanceId : 'logical-name-of-myec2-instance'
},
},
};
}
or is this idea totally wrong?
In https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-eip.html
I see a parameter "PublicIpv4Pool" but I do not own a address pool. However I could create a further EIP in aws-console. But how could I assign/use this EIP in this stack?
As bastion instances are not used heavily it makes sense to be able to stop them to reduce costs.
ie after deployment of vpc and using bastion for setup purposes and initial checks, stop the instance in the ec2 console.
It was previously possible but I cannot remember which version of this plugin.
Currently the Bastion and NatInstance ec2 instances throw this error when trying to stop them
"Failed to stop the instance i-xxxxxxxxx
You can't stop the Spot Instance 'i-xxxxxx' because it is associated with a one-time Spot Instance request. You can only stop Spot Instances associated with persistent Spot Instance requests"
Can the spot instance type be adjusted to allow this please?
By default, without a NAT Instance
or NAT Gateway
, the subnets associated with the functions should route all traffic to the Internet Gateway
. Hence the functions should have the ability to make outbound requests. However, all the outgoing traffics are blocked and timeout.
This issue can be resolved by provisioning a NAT instance/gateway in v0.6.0.
serverless.yml
service: serverless-vpc-plugin-example
frameworkVersion: '>=1.40.0 <2.0.0'
provider:
name: aws
runtime: nodejs8.10
stage: dev
region: us-west-2
memorySize: 128
timeout: 6
functions:
hello:
handler: handler.hello
events:
- http:
path: /
method: get
custom:
vpcConfig:
cidrBlock: '10.123.0.0/16'
# createNatInstance: true
plugins:
- serverless-vpc-plugin
- serverless-webpack
handler.js
'use strict'
const axios = require('axios')
module.exports.hello = async (event) => {
const response = await axios.post('https://httpbin.org/post', { some: 'data' })
return {
statusCode: 200,
body: JSON.stringify({
message: 'Go Serverless v1.0! Your function executed successfully!',
input: event,
response: response.data
})
}
}
With NAT instance:
{
"message": "Go Serverless v1.0! Your function executed successfully!",
"input": {
...
},
"response": {
"args": {},
"data": "{\"some\":\"data\"}",
"files": {},
"form": {},
"headers": {
"Accept": "application/json, text/plain, */*",
"Content-Length": "15",
"Content-Type": "application/json;charset=utf-8",
"Host": "httpbin.org",
"User-Agent": "axios/0.18.0"
},
"json": {
"some": "data"
},
"origin": "...",
"url": "https://httpbin.org/post"
}
}
Without NAT instance:
{
"message": "Internal server error"
}
Sometimes, you don't want to pay for the NATGateway especially in the dev environment :P
Looks like an interesting plugin and I'm trying to give it a go.
Using version: "serverless-vpc-plugin": "^1.0.0"
My config looks like this but get an error thrown:
An error occurred: BastionSecurityGroup - CIDR block ##.##.##.###
/32 is malformed (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue; Request ID: #########-####-####-####-###########; Proxy: null).
vpcConfig:
cidrBlock: '10.0.0.0/16'
# if createNatGateway is a boolean "true", a NAT Gateway and EIP will be provisioned in each zone
# if createNatGateway is a number, that number of NAT Gateways will be provisioned
# createNatGateway: 2
# When enabled, the DB subnet will only be accessible from the Application subnet
# Both the Public and Application subnets will be accessible from 0.0.0.0/0
createNetworkAcl: true
# Whether to create the DB subnet
createDbSubnet: true
# Whether to enable VPC flow logging to an S3 bucket
createFlowLogs: false
# Whether to create a bastion host
createBastionHost: true
bastionHostKeyName: desktop # required if creating a bastion host
# Whether to create a NAT instance
createNatInstance: true
# Whether to create AWS Systems Manager (SSM) Parameters
createParameters: true
# Optionally specify AZs (defaults to auto-discover all availabile AZs)
zones:
- eu-central-1a
- eu-central-1b
# By default, S3 and DynamoDB endpoints will be available within the VPC
# see https://docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints.html
# for a list of available service endpoints to provision within the VPC
# (varies per region)
services:
- kms
- secretsmanager
# Optionally specify subnet groups to create. If not provided, subnet groups
# for RDS, Redshift, ElasticCache and DAX will be provisioned.
subnetGroups:
- rds
# Whether to export stack outputs so it may be consumed by other stacks
exportOutputs: true
Any thoughts on what is going on?
Hi, thanks for this awesome plugin!
Is it possible to obtain the list of all the required IAM permissions in order to fully deploy a serverless solution with VPC enabled?
Thank you.
Using serverless-vpc-plugin
I'm having trouble connecting it all. I have a Lambda-function triggered via API Gateway that I want to execute a call to elastiCache. It deploys just fine but my lambda-function just timeouts after performing a redisClient.get
... ๐ข I think the problem lies in missing access credentials for my lambda to the VPC where the elastiCache resides.
I have scavenged the darkest corners of the deep web in search for a solution for more than two full days, I have come up short... I need your help! ๐ค
Here's my serverless.yml
for the network:
service: obb-network
plugins:
- serverless-vpc-plugin
provider:
name: aws
runtime: nodejs12.x
stage: dev
region: eu-central-1
profile:
obb
custom:
vpcConfig:
cidrBlock: '10.0.0.0/16'
createNatGateway: false
createNetworkAcl: false
createDbSubnet: true
createFlowLogs: false
createBastionHost: false
createNatInstance: false
subnetGroups:
- elasticache
exportOutputs: true
And here's my serverless.yml
for the API Gateway and elastiCache:
service: obb-api
custom:
networkStack: obb-network-${self:provider.stage} # this links the above network serverless
provider:
name: aws
runtime: nodejs12.x
stage: dev
region: eu-central-1
profile:
obb
environment:
redisHost:
"Fn::GetAtt": [ElasticCacheCluster, RedisEndpoint.Address]
iamRoleStatements: # perhaps this will provide passage to elastiCache
- Effect: Allow
Action:
- "elasticache:*"
Resource: "*"
functions:
hello:
handler: hello.main
events:
- http:
path: /
method: get
cors: true
authorizer: aws_iam
role: LambdaVPCAccessExecutionRole # perhaps this will provide passage to elastiCache
vpc:
securityGroupIds:
- Fn::GetAtt: CacheSecurityGroup.GroupId
subnetIds:
- Fn::ImportValue: !Sub ${self:custom.networkStack}-ElastiCacheSubnetGroup
# Role to append to lambda in order to connect to ElastiCache, I guess
LambdaVPCAccessExecutionRole:
Type: "AWS::IAM::Role"
Properties:
RoleName: "${self:service}-${self:provider.stage}-${self:provider.region}-LambdaVPCAccessExecutionRole"
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: "lambda.amazonaws.com"
Action: "sts:AssumeRole"
Policies:
- PolicyName: LambdaVPCAccessExecutionRolePolicy
PolicyDocument:
Statement:
- Effect: Allow
Action:
- "elasticache:*"
Resource: "*"
ManagedPolicyArns:
- "arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole"
# Elasticache needs an inbound rule to open port 6379, I guess
CacheSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Ingress for Redis Cluster
VpcId:
Fn::ImportValue: !Sub ${self:custom.networkStack}-VPC
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 6379
ToPort: 6379
SourceSecurityGroupId:
Fn::ImportValue: !Sub ${self:custom.networkStack}-LambdaExecutionSecurityGroupId
# Set up elasticache cluster, I guess
ElasticCacheCluster:
Type: AWS::ElastiCache::CacheCluster
Properties:
AutoMinorVersionUpgrade: true
Engine: redis
CacheNodeType: cache.t2.micro
NumCacheNodes: 1
CacheSubnetGroupName:
Fn::ImportValue: !Sub ${self:custom.networkStack}-ElastiCacheSubnetGroup
VpcSecurityGroupIds:
- !Ref CacheSecurityGroup
Getting this error while setting up a VPC using this plugin:
service: commercetools
plugins:
- serverless-vpc-plugin
provider:
name: aws
region: eu-central-1
stage: stg
vpc:
securityGroupIds:
- # plugin will add LambdaExecutionSecurityGroup to this list
subnetIds:
- # plugin will add the "Application" subnets to this list
custom:
vpcConfig:
enabled: true
cidrBlock: '10.30.0.0/16'
createNatGateway: 3
createNetworkAcl: false
createDbSubnet: false
createFlowLogs: false
createBastionHost: false
createNatInstance: false
createParameters: false
services:
- kms
- secretsmanager
exportOutputs: false
I'm fairly new to VPC's and haven't tried to configure this but I ask anyways.
Wouldn't Egress-Only Internet Gateways be a viable alternative to nat gatways, provided all target networks use ipv6?
Is there a way to reuse the same VPCs for all the stages or environments?
I'm struggling to connect a lambda to an RDS instance deployed into the RDS DB group. The Lambda is deployed into the Application Subnet
Issue seems to be IP blocked and Lambdas cannot get access to the IP range of the RDS DB Subnet. The Lambda can correctly reach the internet.
Any advice or tips for troubleshooting would be appreciated.
service: test-vpc
plugins:
- serverless-vpc-plugin
provider:
name: aws
stage: dev
region: eu-central-1
custom:
dotenv:
logging: true
REGION: ${opt:region, self:provider.region}
STAGE: ${opt:stage, self:provider.stage}
vpcConfig:
cidrBlock: '10.0.0.0/16'
# if createNatGateway is a boolean "true", a NAT Gateway and EIP will be provisioned in each zone
# if createNatGateway is a number, that number of NAT Gateways will be provisioned
# createNatGateway: 1
# When enabled, the DB subnet will only be accessible from the Application subnet
# Both the Public and Application subnets will be accessible from 0.0.0.0/0
createNetworkAcl: true
# Whether to create the DB subnet
createDbSubnet: true
# Whether to enable VPC flow logging to an S3 bucket
createFlowLogs: false
# Whether to create a bastion host
createBastionHost: true
bastionHostKeyName: REPLACED# required if creating a bastion host
# Whether to create a NAT instance
createNatInstance: true
# Whether to create AWS Systems Manager (SSM) Parameters
createParameters: true
# Optionally specify AZs (defaults to auto-discover all availabile AZs)
zones:
- eu-central-1a
- eu-central-1b
# By default, S3 and DynamoDB endpoints will be available within the VPC
# see https://docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints.html
# for a list of available service endpoints to provision within the VPC
# (varies per region)
services:
- kms
- secretsmanager
# Optionally specify subnet groups to create. If not provided, subnet groups
# for RDS, Redshift, ElasticCache and DAX will be provisioned.
subnetGroups:
- rds
# Whether to export stack outputs so it may be consumed by other stacks
exportOutputs: true
Currently, each stage provisions the VPC settings again, is it possible to reuse the same VPC for all stages? Is there some trickery that could be done?
Getting this error while setting up a VPC using this plugin:
provider:
name: aws
runtime: nodejs12.x
lambdaHashingVersion: '20201221'
stage: ${opt:stage, 'dev'}
region: ${file(serverless.env.yml):${self:custom.stage}.REGION}
vpc:
securityGroupIds:
- sg-xxxxxxxx
subnetIds:
- subnet-xxxxxx
- subnet-xxxxxx
- subnet-xxxxxx
Both subnet and security groups are there in the same VPC.
When I try to use createBastionHost
and bastionHostKeyName
, deploy fails with
BastionAutoScalingGroup | CREATE_FAILED
When using the plugin: serverless-plugin-split-stacks
many parameters cannot be referenced on the second function. The first function in the serverless.yml will include the proper parameters, however, 2nd+ functions will not contain the proper resolutions.
It appears that this is happening due to the way they are exported. As such, you will receive an error: "Template format error: Unresolved resource dependencies [AppSubnet1Parameter, AppSubnet3Parameter, AppSubnet4Parameter, AppSubnet2Parameter, LambdaExecutionSecurityGroupParameter] in the Resources block of the template." when defining out the remainder of the handlers.
The following is my custom configuration:
custom:
splitStacks:
perFunction: true
perType: false
perGroupFunction: false
vpcConfig:
cidrBlock: '10.10.0.0/16'
createBastionHost: false
createDbSubnet: true
createFlowLogs: false
createNatGateway: true
createNatInstance: false
createNetworkAcl: true
zones:
- us-west-2a
- us-west-2b
- us-west-2c
- us-west-2d
subnetGroups:
- rds
exportOutputs: true
The function definitions are rather simplistic:
functions:
networkList:
handler: handler.networkList
events:
- http:
method: get
path: network
cors: true
networkRead:
handler: handler.networkRead
events:
- http:
method: get
path: network/{id}
cors: true
request:
parameters:
path:
id: true
The outputted information from the split stack shows that the parameters section within the first function contains the proper parameter types, whereas the second function will attempt to refer to the parameters but not actually define them.
Hi there - first of all, thanks for this plugin! It saves a lot of config time!
I was wondering if you could provide a little more examples on how to get the NAT to work. In short, I need to have a Lambda that can write to EFS and reach the internet. If I just create my own VPC for the EFS then the lambda can read/write to it, but I can't get out to the internet. I was hoping I could use this plugin to not have to manually set up all the NAT stuff, but I'm getting lost.
NAT, subnets, routes... none are my forte really, so hoping for a little help/example.
Placing the EFS on the AppSubnet, results in the lambda not being able to connect to it. Here's what I got so far:
resources:
Resources:
SharedFs:
Type: AWS::EFS::FileSystem
Properties:
BackupPolicy:
Status: ENABLED
PerformanceMode: generalPurpose
Encrypted: true
LifecyclePolicies:
- TransitionToIA: AFTER_30_DAYS
MountTarget:
Type: AWS::EFS::MountTarget
Properties:
FileSystemId: !Ref SharedFs
SubnetId: !Ref AppSubnet1
SecurityGroups:
- !Ref AppSecurityGroup
AccessPointResource:
Type: AWS::EFS::AccessPoint
Properties:
FileSystemId: !Ref SharedFs
PosixUser:
Uid: 1001
Gid: 1001
RootDirectory:
CreationInfo:
OwnerGid: 1001
OwnerUid: 1001
Permissions: 770
Path: /efs
But then I get:
The function couldn't connect to the Amazon EFS file system with access point.......... Check your network configuration and try again. (Service: AWSLambda; Status Code: 408; Error Code: EFSMountConnectivityException
Should it even be on the AppSubnet? Or use the public subnet?
I am trying to disable the plugin for some stages by providing enabled: false
(keeping the other parameters same. ). But it's still generating VPC and subnets when I run serverless deploy
. I also tried hardcoding the value like this: (just making sure, I am not doing something wrong while doing it conditionally)
vpcConfig:
# Whether plugin is enabled. Can be used to selectively disable plugin
# on certain stages or configurations. Defaults to true.
enabled: false
cidrBlock: '10.0.0.0/16'
.....
I would like to setup VPC peering with this plugin
provider:
vpc:
securityGroupIds:
- ${AppSecurityGroup}
subnetIds: ${AppSubnets~split}
custom:
vpcConfig:
cidrBlock: '10.0.0.0/16'
createNatGateway: 1
createDbSubnet: false
seems to be working.
But when I want to use it for my resources its broken.
resources:
Resources:
MountTargetA:
Type: AWS::EFS::MountTarget
Properties:
SubnetId: ${AppSubnet{1}}
SecurityGroups:
- ${AppSecurityGroup}
MountTargetB:
Type: AWS::EFS::MountTarget
Properties:
SubnetId: ${AppSubnet{2}}
SecurityGroups:
- ${AppSecurityGroup}
MountTargetC:
Type: AWS::EFS::MountTarget
Properties:
SubnetId: ${AppSubnet{3}}
SecurityGroups:
- ${AppSecurityGroup}
Error message:
An error occurred: EFSCacheMountTargetResourceA - 2 validation errors detected: Value '${AppSubnet{1}}' at 'subnetId' failed to satisfy constraint: Member must satisfy regular expression pattern: ^subnet-[0-9a-f]{8,40}$; Value '[${AppSecurityGroup}]' at 'securityGroups' failed to satisfy constraint: Member must satisfy constraint: [Member must have length less than or equal to 43, Member must have length greater than or equal to 11, Member must satisfy regular expression pattern: ^sg-[0-9a-f]{8,40}] (Service: AmazonElasticFileSystem; Status Code: 400; Error Code: ValidationException; Request ID: 569b3c04-42f3-47db-9cb8-1c630b090b28; Proxy: null).
Not working with ${self:provider.vpc.subnetIds.1}
, too.
When I don't list availability zones, the plugin uses all 6 in us-east1. Is this overkill? Would a better default be to randomly pick 3?
When using this plugin, the default package exclude option no longer works as described here:
https://serverless.com/framework/docs/providers/aws/guide/packaging/
This is my config, but src is still being included in the artifact.
package:
exclude:
- src/**
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.