agutoli / serverless-layers

Serverless.js plugin implementing AWS Lambda Layers, reducing lambda size, warm-up, and deployment time.

Serverless Layers

• Automatically attaches layers to the provider and for each function
  • Skips functions with no other layers as they will use the layer(s) we added to the provider
• Creates a new layer's version when dependencies are updated
• Does not publish a new layer if dependencies are unchanged
• Drastically reduces lambda size
• Reduces deployment time
• Allows sharing of the same layers (libraries) among all lambda functions

Options

• NodeJS
• Ruby
• Python

Common Requirements

• AWS only (sorry)
• Serverless >= 1.34.0 (layers support)

Install

npm install -D serverless-layers

or

serverless plugin install --name serverless-layers

Add the plugin to your serverless.yml file:

Single layer config

Example:

plugins:
  - serverless-layers
  
custom:
  serverless-layers:
    functions: # optional
      - my_func2
    dependenciesPath: ./package.json

functions:
  my_func1:
    handler: handler.hello
  my_func2:
    handler: handler.hello

Multiple layers config

Example:

plugins:
  - serverless-layers
  
custom:
  serverless-layers:
    # applies for all lambdas 
    - common:
        dependenciesPath: ./my-folder/package.json
    # apply for foo only
    - foo:
        functions:
          - foo
        dependenciesPath: my-folder/package-foo.json
    - staticArn:
        functions:
          - foo
          - bar
        arn: arn:aws:lambda:us-east-1:<your_account>:layer:node-v13-11-0:5

functions:
  foo:
    handler: handler.hello
  bar:
    handler: handler.hello

Option	Type	Default	Description
compileDir	string	.serverless	Compilation directory
layersDeploymentBucket	string		Specify a bucket to upload lambda layers. Required if deploymentBucket is not defined.
customInstallationCommand	string		Specify a custom command to install dependencies, e.g., MY_ENV=1 npm --proxy http://myproxy.com i -P
customHash	string		Specify a custom string that, once changed, will force a new build of the layer
retainVersions	int	null	Number of layer versions to keep; older versions will be removed after deployments

NodeJS

Requirements

• Node >= v6.10.3
• NPM >= 3.10.10
• A valid package.json file

Options

Option	Type	Default	Description
packageManager	string	npm	Possible values: npm, yarn
packagePath	string	package.json	(DEPRECATED): Available for <= 1.5.0; for versions >= 2.x, use compatibleRuntimes
dependenciesPath	string	package.json	Note: >= 2.x versions. You can specify a custom path for your package.json
compatibleRuntimes	array	['nodejs']	Possible values: nodejs, nodejs10.x, nodejs12.x
layerOptimization.cleanupPatterns	array	check	Pattern of files to cleanup in the layer artifact before uploading it

Ruby

Requirements

• Ruby >= 2.5
• A valid Gemfile file

Options

Option	Type	Default	Description
packageManager	string	bundle	Possible values: bundle
dependenciesPath	string	Gemfile	Note: Available for >= 2.x versions. You can specify a custom path for your Gemfile
compatibleRuntimes	array	['ruby']	Possible values: ruby2.5, ruby2.7
layerOptimization.cleanupPatterns	array	check	Pattern of files to cleanup in the layer artifact before uploading it

Python

Requirements

• Python >= 2.7
• A valid requirements.txt file

Options

Option	Type	Default	Description
packageManager	string	pip	Possible values: pip
dependenciesPath	string	requirements.txt	Note: Available for >= 2.x versions. You can specify a custom path for your requirements.txt
compatibleRuntimes	array	['python']	Possible values: python2.7, python3.x
layerOptimization.cleanupPatterns	array	check	Pattern of files to cleanup in the layer artifact before uploading it

Default Serverless Setup

This plugin will set up the following options automatically if not specified in serverless.yml.

Option	Type	Default
package.individually	bool	false
package.patterns	array	['node_modules/**']
package.excludeDevDependencies	bool	false

Minimal Policy permissions for CI/CD IAM users

serverless-layers-policy.json

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Effect":"Allow",
         "Action":[
            "s3:PutObject",
            "s3:GetObject"
         ],
         "Resource": "arn:aws:s3:::examplebucket"
      },
      {
         "Effect":"Allow",
         "Action":[
            "cloudformation:DescribeStacks"
         ],
         "Resource": "*"
      },
      {
         "Effect":"Allow",
         "Action":[
            "lambda:PublishLayerVersion"
         ],
         "Resource": "*"
      }
   ]
}

License

MIT

Contributors

Yes, thank you! This plugin is community-driven, and most of its features are from different authors. Please update the docs and tests and add your name to the package.json file. We try to follow Airbnb's JavaScript Style Guide.

Made with contributors-img.

serverless-layers's Issues

Check package.json dependencies changes return false, but always deploy

Adding this package makes deploying fail

Error: ... ○ "undefined" is not supported (yet).

Config:

custom:
  serverless-layers:
    dependenciesPath: requirements.txt
    compatibleRuntimes: ["python3.8"]

Env:

Operating System: win32 Node Version: 12.16.1 Framework Version: 1.71.3 Plugin Version: 3.6.12 SDK Version: 2.3.0 Components Version: Unavailable

[LayersPlugin]: Error: Can not find yarn.lock!

despite the yarn.lock existing, I am getting this error when I try to use yarn.lock in custom:packagePath:

Feature request: allow add more static layer

for example:

- staticArn:
        functions:
          - func
        arn:
          - arn:aws:lambda:ap-northeast-1:xxx:layer:linux-utility:8
          - arn:aws:lambda:ap-northeast-1:xxx:layer:python3:1

Does not use latest layer if layer was created during build

If you run a serverless build and a new layer is created based on changes in layer dependencies, and then run serverless deploy independently using the package just built, it will say no changes and use the current layer attached to the lambda function. For example, If layer version 1 is currently attached to the lambda function and i change a dependency, i first run serverless package --package package to create the package and version 2 of the layer will be created based on the changed dependencies. When i run serverless deploy --package package it will say 'no changes detected' and keep version 1 attached, even though version 2 was just created in the build step. It looks like in the CloudFormationService.js file, its getting the outputs from cfn, but the cfn isnt updated when i do a sls build

Compatibility with serverless-deployment-buckets

I've been having issues with the hook order between this plugin and serverless-deployment-bucket.

The serverless-deployment-bucket plugin creates the S3 deployment bucket for the deployment, and is performed on the hook before:package:setupProviderConfiguration.

serverless-layers fires much earlier in the process during before:package:initialize. The layer is attempted to be uploaded before the bucket can be created, and the exception halts the process.

Is there a more appropriate hook for uploading the layer after the provider is fully configured?

this.provider.getDeploymentPrefix is not a function

I've followed the instructions. The relevant parts of my serverless.yml are:

provider:
  deploymentBucket: #required!
    name: "layers_deployment_bucket"

plugins:
  - serverless-layers

package:
  individually: false
  exclude:
    - node_modules/**
  excludeDevDependencies: true

But when I try to deploy I get:

Type Error ---------------------------------------------

this.provider.getDeploymentPrefix is not a function

TypeError: arn.replace is not a function

When using serverless core layers feature in combinaision with this plugin a "TypeError: arn.replace is not a function" is thrown by logArn() function

Sample configuration:

Pre-build layer

layers:
postgresqlDriver:
package:
artifact: layer-linux-postgresql-driver.zip

Plugin configuration

custom:

AWS layers for Python requirements

serverless-layers:
# Commons for all lambdas
- common:
compileDir: .serverless/layers/common
dependenciesPath: src/layers/common/requirements.txt
# Per function requirements
- tool:
compileDir: .serverless/layers/tool
functions:
- items_get
dependenciesPath: src/layers/tool/requirements.txt

Function with prebuilt layer + "serverless-layers" plugin layers

functions:
items_get:
handler: src/v1/items/get/get.get
layers:
- { Ref: PostgresqlDriverLambdaLayer }

Error:

TypeError: arn.replace is not a function
at ServerlessLayers.logArn (C:\Graphane\serverless\api-mds-private\node_modules\serverless-layers\lib\index.js:779:18)
at forEach (C:\Graphane\serverless\api-mds-private\node_modules\serverless-layers\src\index.js:406:76)
at Array.forEach ()
at forEach (C:\Graphane\serverless\api-mds-private\node_modules\serverless-layers\src\index.js:401:14)
at Array.forEach ()
at ServerlessLayers._callee7$ (C:\Graphane\serverless\api-mds-private\node_modules\serverless-layers\src\index.js:392:41)
at tryCatch (C:\Graphane\serverless\api-mds-private\node_modules\regenerator-runtime\runtime.js:63:40)
at Generator.invoke [as _invoke] (C:\Graphane\serverless\api-mds-private\node_modules\regenerator-runtime\runtime.js:293:22)
at Generator.next (C:\Graphane\serverless\api-mds-private\node_modules\regenerator-runtime\runtime.js:118:21)
at asyncGeneratorStep (C:\Graphane\serverless\api-mds-private\node_modules@babel\runtime\helpers\asyncToGenerator.js:3:24)
at _next (C:\Graphane\serverless\api-mds-private\node_modules@babel\runtime\helpers\asyncToGenerator.js:25:9)
at C:\Graphane\serverless\api-mds-private\node_modules@babel\runtime\helpers\asyncToGenerator.js:32:7
at new Promise ()
at ServerlessLayers. (C:\Graphane\serverless\api-mds-private\node_modules@babel\runtime\helpers\asyncToGenerator.js:21:12)
at ServerlessLayers.finalizeDeploy (C:\Graphane\serverless\api-mds-private\node_modules\serverless-layers\lib\index.js:735:32)
at ServerlessLayers. (C:\Graphane\serverless\api-mds-private\node_modules\serverless-layers\src\index.js:34:26)
at ServerlessLayers.tryCatcher (C:\Graphane\serverless\api-mds-private\node_modules\bluebird\js\release\util.js:16:23)
at Promise._settlePromiseFromHandler (C:\Graphane\serverless\api-mds-private\node_modules\bluebird\js\release\promise.js:547:31)
at Promise._settlePromise (C:\Graphane\serverless\api-mds-private\node_modules\bluebird\js\release\promise.js:604:18)
at Promise._settlePromise0 (C:\Graphane\serverless\api-mds-private\node_modules\bluebird\js\release\promise.js:649:10)
at Promise._settlePromises (C:\Graphane\serverless\api-mds-private\node_modules\bluebird\js\release\promise.js:729:18)
at _drainQueueStep (C:\Graphane\serverless\api-mds-private\node_modules\bluebird\js\release\async.js:93:12)
at _drainQueue (C:\Graphane\serverless\api-mds-private\node_modules\bluebird\js\release\async.js:86:9)
at Async._drainQueues (C:\Graphane\serverless\api-mds-private\node_modules\bluebird\js\release\async.js:102:5)
at Immediate.Async.drainQueues (C:\Graphane\serverless\api-mds-private\node_modules\bluebird\js\release\async.js:15:14)
at processImmediate (internal/timers.js:456:21)

Private npm package does not work

Hi, I'm using some private packages from npm and after I upgraded from version 1.5.0 to the latest available I'm having problems with 404 error in my private packages. Can you help me?

Problem with windows style file paths when uploading to s3

Hello,

Firstly, thank you for this amazing plug-in!

I have trouble using the package on windows due to windows file paths are constructed by \ characters.

When uploading the layers zip file and package.json file, the files are uploaded as single files on the main bucket folder, the nesting folders are not constucted.

The files are formed as

serverless\{service_name}\{stage}\layers\package.json

and

serverless\{service_name}\{stage}\layers\{service_name}-{stage}.zip

I guess they are supposed to be uploaded as a folder structure.

Thanks in advance.

Add support for pnpm

Currently only support npm, yarn

I'll submit a PR

Deploying existing package

Deploying an existing service package using the command sls deploy --package .serverless results in a TypeError: Cannot read property 'getOutputs' of undefined. This is because the separate deploy command hasn't initialized the cloudFormation service (or any other service), and this.cloudFormationService is undefined.

Add more documentation

Thanks for the contribution with the plugin. I would suggest to add more documentation on how to use the plugin, how it works and a couple of serverless.yml examples using the plugin.

Associate multiple layers with different functions

Currently in my serverless.yml i have 2 functions defined but i only want my custom layer to be associated with 1 of the functions. Is it possible to define multiple layers and choose which layer is associated with which lambda?

layersDeploymentBucket isn't specified

Thanks for a good plugin!

Even though I set "layersDeploymentBucket" according the README, I got the message "Error: Please, you should specify "deploymentBucket" or "layersDeploymentBucket" option for this plugin!"

Isn't "custom.serverless-layers.layersDeploymentBucket" overwritten with null when "provider.deploymentBucket" isn't set?

serverless-layers/src/index.js

Line 70 in d2f0c36

getSettings() {

Should not install dev dependencies

At the moment, because of a small bug, the list of dependencies being passed to npm-install-package is [], which leads to it making the call npm i (which installs all packages, including dev dependencies). In one of my projects this pushes us over the 250MB maximum limit, whereas without dev dependencies I'm at 130MB.

Feature request - delete previous layer version

This is a feature request to support deleting previous layer versions when a new version is deployed

Same feature as option "retain" in core serverless
https://www.serverless.com/framework/docs/providers/aws/guide/layers/

2.2.0 broken - new layer created each time

Hi,

I've recetenly migrated from 1.x branch to 2.2.0 and seems there is a bug. Looks like each time severless is called new layer gets created.
I've downloaded s3 manifest and diff/md5 compared it with local manifest and they are same

kowal@LAPTOP-UJ4M2HCE:/mnt/c/Users/marci/IdeaProjects/comm$ aws s3 cp s3://serverless-layers/serverless/stack/staging/layers/package.json from3
download: s3://serverless-layers/serverless/stack/staging/layers/package.json to ./from3
kowal@LAPTOP-UJ4M2HCE:/mnt/c/Users/marci/IdeaProjects/comm$ diff package.json from3
kowal@LAPTOP-UJ4M2HCE:/mnt/c/Users/marci/IdeaProjects/comm$ md5sum package.json 
1377a1a91e3e2115d4e3960c22adf311  package.json
kowal@LAPTOP-UJ4M2HCE:/mnt/c/Users/marci/IdeaProjects/comm$ md5sum from3
1377a1a91e3e2115d4e3960c22adf311  from3
kowal@LAPTOP-UJ4M2HCE:/mnt/c/Users/marci/IdeaProjects/comm$ sls deploy --stage staging 

Serverless: [ LayersPlugin ]: => common
... ○ Downloading __meta__/manifest-settings.json from bucket...
... ○ Downloading package.json from bucket...
... ○ Comparing package.json dependencies...
... ○  Changes identified ! Re-installing...

On 1.x branch new layer was created only if there was a change in a package.json file

My config:

custom:
  serverless-layers:
    - common:
        packageManager: npm
        dependenciesPath: package.json
        layersDeploymentBucket: 2wayapp-serverless-layers
        compatibleRuntimes: ['nodejs']
        compileDir: .serverless-layers/${self:service}

tried with
dependenciesPath: ./package.json
but no luck either

With SLS_DEBUG=true I was able to check if remote package.json file is correct (in fact I've used it to download remote file with aws s3 cli)

Need help to use this plugin with serverless-webpack

I am also looking to use this plugin for our serverless project with TypeScript. We use serverless-webpack plugin to bundle the TypeScript code and also have an option to include/exclude node_modules from the service .zip file.

Basically, we can know what exactly packages need for the current build, so I am trying to see whether we can create a temporary package.json, install dependencies and create layer and deploy.

TypeError: Cannot read properties of undefined (reading 'pipes')

Hey—I'm getting the following error:

  TypeError: Cannot read properties of undefined (reading 'pipes')
      at ReadStream.Readable.pipe (node:internal/streams/readable:658:13)
      at Readable.<anonymous> (/Users/(...)/Projects/(...)/api/node_modules/lazystream/lib/lazystream.js:33:12)
      at Readable.instance.<computed> (/Users/(...)/Projects/(...)/api/node_modules/lazystream/lib/lazystream.js:18:14)
      at Readable.read (/Users/(...)/Projects/(...)/api/node_modules/lazystream/node_modules/readable-stream/lib/_stream_readable.js:443:10)
      at resume_ (/Users/(...)/Projects/(...)/api/node_modules/lazystream/node_modules/readable-stream/lib/_stream_readable.js:790:12)
      at processTicksAndRejections (node:internal/process/task_queues:83:21)
      at runNextTicks (node:internal/process/task_queues:65:3)
      at processImmediate (node:internal/timers:437:9)

I've tried debugging what's the root cause, but no luck.

Versions:

api % cat node_modules/serverless-layers/node_modules/archiver/package.json |jq .version
"3.1.1"
api % cat node_modules/serverless-layers/package.json |jq .version
"2.5.1"
api % cat node_modules/serverless/package.json |jq .version
"2.67.0"

Settings (serverless.yml):

serverless-layers:
  packageManager: npm
  compileDir: .serverless
  forceInstall: ${env:SERVERLESS_LAYERS_FORCE_INSTALL, false}
  dependenciesPath: package.json
  compatibleRuntimes: ["nodejs"]
  layersDeploymentBucket: (...)
  # Why ../../..? Because the script runs on `.serverless/layers/nodejs`
  customInstallationCommand: "../../../scripts/build/install-node-modules.sh"

install-node-modules.sh

echo "Installing dependencies: $(pwd)"
npm install --loglevel=error --no-optional --only=production --audit=false
npm prune --production --audit=false

echo "Running https://github.com/tj/node-prune"
node-prune --exclude "*.d.ts" ./node_modules

Custom environment setup

Hi,
I am using pupperteer. During deployment I need to run following command :

rm -rf node_modules/; env PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 npm install; serverless deploy;
However when I use serverless-layers, node-modules are re-installing. and I have no control to set environment. Kindly guide.

serverless.yml provider and region issue

If the provider region is using an argument/variable and it is not defined, the default is not used and serverless commands hang.

This is how to reproduce it:

• In serverless.yml set the region under provider with an optional:

provider:
    region: ${opt:stage, "us-east-1"}

• run serverless print to see it hang. Any sls command will hang:
  sls print

• To unblock it, just make sure the provider region has a value:
  sls print --region=us-east-1

We will need to still try and see where this is being caused in the code, but thought this might help anyone else seeing this issue in the meantime.

2.2.0 Broken when single layer config has 2 or more options

Hi,

For example, Single layer config from README followings:

custom:
  serverless-layers:
    functions: # optional
      - my_func2
    dependenciesPath: ./package.json

In this case occurs error It looks like you are trying to setup a list of layers, no? from version 2.2.0.

This validation logic was added by this commit.

const keys = Object.keys(inboundSettings); are functions and dependenciesPath.

2.2.0 not support for single layer config that has 2 or more options?

provider.deploymentBucket is specified, yet error is reported that it isn't

Using serverless 0.70.0 and the latest serverless-layers plugin

error:

Serverless: [ LayersPlugin ]: => default
 
 Error ---------------------------------------------------
 
  Error: Please, you should specify "deploymentBucket" or "layersDeploymentBucket" option for this plugin!

serverless.yml snippets:

provider:
   ...
  deploymentBucket:
    maxPreviousDeploymentArtifacts: 10
    blockPublicAccess: true
    serverSideEncryption: AES256
    tags: # Tags that will be added to each of the deployment resources
      MyEnv: ${opt:stage, 'dev'}
      MyApp: ${self:service.name}

....

custom:
  serverless-layers:
    dependenciesPath: ./package.json
    compatibleRuntimes: 
      - nodejs14.x

python packages accumulate

I'm trying to create separate layers for dependencies in a python3.8-based project, as not all functions need all dependencies.
I was surprised by the sizes of the generated zip files so I downloaded them from the bucket and realized that dependencies were being accumulated in the order of the list.

Each of these requirements.txt files only contains a single requirement (python-jose, pyjwt[crypto] and requests respectively) and while the python-jose layer works correctly, pyjwt-crypto contains both pyjwt[crypto] and python-jose and the requests layer contains all 3 dependencies.

package:
  individually: true
  exclude:
    - ./**

custom:
  serverless-layers:
    - python-jose:
        functions:
          - function1
        dependenciesPath: layers/python-jose/requirements.txt
    - pyjwt-crypto:
        functions:
          - function2
        dependenciesPath: layers/pyjwt-crypto/requirements.txt
    - requests:
        functions:
          - function1
          - function3
          - function4
        dependenciesPath: layers/requests/requirements.txt

Plugin does not respect serverSideEncryption flag

When the deployment bucket has encryption enabled:

deploymentBucket:
    name: exampleBucket
    serverSideEncryption: AES256

The plugin does not respect the serverSideEncryption flag and attempts to upload without encryption resulting in an error.
The plugin should also support a similar object on the layersDeploymentBucket:

serverless-layers:
    packageManager: npm
    layersDeploymentBucket:
        name: exampleBucket
        serverSideEncryption: AES256

This feature is desired as some organizations enforce encryption on all buckets as a security guideline

str.indexOf is not a function

Serverless: [ LayersPlugin ]: => default
... ○ Downloading package.json from bucket...
... ○ package.json does not exists at bucket...
... ⊗ =============================================================
... ⊗ NOTE: You're currently using incompatible version [v14.15.1]
... ⊗ =============================================================

... ○ Changes identified ! Re-installing...
... ∅ [warning] ".npmrc" file does not exists!
... ∅ [warning] "yarn.lock" file does not exists!
npm WARN [email protected] No description

added 180 packages from 151 contributors and audited 2546 packages in 14.671s

28 packages are looking for funding
run npm fund for details

found 72 vulnerabilities (1 low, 68 moderate, 3 high)
run npm audit fix to fix them, or npm audit for details

... ○ Created layer package /Users/orrlevinger/projects/BlindChat/blind-chat-backend/.serverless/sls-blind-chat-dev-stack-nodejs-default.zip (24.6 MB)
... ○ Uploading layer package...
str.indexOf is not a function

AWS Error: Function code combined with layers exceeds the maximum allowed size of 262144000 bytes

Issue

Just dropping this down somewhere to note / for people with this problem. Not sure if it's a bug or expected behaviour.

I tried to introduce serverless-layers to my existing project and got:

An error occurred: *** - Function code combined with layers exceeds the maximum allowed size of 262144000 bytes

The zip of the code was now only 40kb and the layer 40mb (150mb decompressed). However the previous bundle code size was pretty large and I think CloudFormation was trying to first apply the layer to the function before updating the function code which resulted in the error above.

Workaround

I deleted the stack and re-created it. An alternative (although not tested) update the code to something with no imports deploy. Once successful try and add layers to the project.

npm v7 workspace dependencies not bundled

I'm trying out npm v7 workspaces - where you can have multiple packages in a monorepo that are automatically linked together. You normally don't need to specify the other modules in package.json, they are simply made available to each package via symlinks.

Unfortunately when the node_modules layer for my project is generated by this plugin, it doesn't include the linked modules.

More details here: https://github.com/npm/rfcs/blob/26e8ac6ee176943d6522d5d057fab05e37655e1c/accepted/0000-workspaces.md

It's the symlinking that gets lost when the layer is created.

sls remove does not delete created layers

After I removed the service with sls remove the layers created by the serverless-layers plugin are not cleaned up.
Also the resources created in the Layer Bucket are not cleaned up.

Bucket doesn't exist

Took a bit of work but got this plugin mostly working with my project (see my PR). Filing an issue because the instructions specify that deploymentBucket is a required attribute. However, it gets into the deployment step and fails because I'm trying to apply it to a new service that doesn't have a deployment bucket created on S3 yet. Do I have to create the S3 bucket manually, or is there some way to keep it automated the way that serverless framework usually works?

Right now trying to deploy fails for lack of bucket.

Serverless Error ---------------------------------------

  Could not locate deployment bucket. Error: The specified bucket does not exist

Force excluding / including packages

How should I go about excluding aws-sdk from the layer?

While using this plugin it's no longer applying layers from `provider.layers`

Consider this serverless.yml:

service: bla

plugins:
 - serverless-layers

provider:
     layers:
        - userLayerARN
layers:
    - userLayerConfig
functions:
   lambda1:
        handler: handler.layer1

userLayer will never be bound to the lambda1 function because this plugin adds the nodejs layer to function.layers which takes priority over provider functions.

the only solution I can think of is:

• check before we add the layers
  1. if we have no function layers add the nodejs layer to the provider
  2. if there are function layers add the nodejs layer to functions (current behavior)
     or we can have a setting that decides whether this plugin adds to provider.layers or function.layers
     until then we can implement the following warning: if there are provider layers log output to warn the user that they will be ignored for all functions because this plugin adds layers to functions

Obviously there is no hurry on this. this is none blocking because I can add my self created layers to the functions.

If you point me in the right direction I can send you a PR for my suggested solution above
I am trying to find if I have access to read+write from providers.layers in relateLayerWithFunctions but I can't find it
Edit: found it PR incoming soon

Minimal configuration?

My lambda layer is getting created nicely, but all of its contents are still getting included in the main lambda deployment! That is, the lambda itself is still as large as it was before I set up a layer.

Besides including the plugin, I just added the tiniest config to my serverless.yml, in the custom section:

  serverless-layers:
    layersDeploymentBucket: foo-lambda-layers

What am I doing wrong?

[LayersPlugin]: package.json does not exists at bucket...

I am getting an error when trying to deploy. I am using a newly created bucket.

`excludeDevDependencies: true` seems pointless and time-consuming

I don't see the point of this. From what I understand, the existing node_modules folder will be ignored completely and will be installed using customInstallationCommand, which for me is yarn -p. Having excludeDevDependencies set to true will only spend computing time on something that will be package ignored.
I was able to save 10+ seconds setting this flag to false.

Comparing package.json does not work properly if package.json is in another directory

If i specify a different dependenciesPath other than the current directory, it will always compare the package.json in s3 to the package.json in the current directory, resulting in a new layer being created every single deploy. It should compare the package.json in the dependenciesPath specified

TypeError: Cannot read property 'pipesCount' of undefined

TypeScript nestjs application. I can deploy fine with serverless-layers disabled.
Using "serverless-layers": "^2.2.4"

❯ sls deploy


Serverless: [ LayersPlugin ]: => default
... ○ Downloading package.json from bucket...
... ○ Comparing package.json dependencies...
... ○  Changes identified ! Re-installing...
... ∅ [warning] ".npmrc" file does not exists!
... ∅ [warning] "yarn.lock" file does not exists!
npm WARN [email protected] No repository field.
npm WARN [email protected] No license field.


> @nestjs/[email protected] postinstall /Users/cyber/dev/jb/demo/aws-node-typescript-nest/.serverless/layers/nodejs/node_modules/@nestjs/core
> opencollective || exit 0

added 243 packages from 445 contributors and audited 2810 packages in 12.682s

4 packages are looking for funding
  run `npm fund` for details

found 1400 vulnerabilities (64 low, 1336 high)
  run `npm audit fix` to fix them, or `npm audit` for details


  Type Error ---------------------------------------------

  TypeError: Cannot read property 'pipesCount' of undefined
      at ReadStream.Readable.pipe (_stream_readable.js:623:17)
      at Readable.<anonymous> (/Users/cyber/dev/jb/demo/aws-node-typescript-nest/node_modules/lazystream/lib/lazystream.js:33:12)
      at Readable.instance.<computed> (/Users/cyber/dev/jb/demo/aws-node-typescript-nest/node_modules/lazystream/lib/lazystream.js:18:14)
      at Readable.read (/Users/cyber/dev/jb/demo/aws-node-typescript-nest/node_modules/lazystream/node_modules/readable-stream/lib/_stream_readable.js:443:10)
      at resume_ (/Users/cyber/dev/jb/demo/aws-node-typescript-nest/node_modules/lazystream/node_modules/readable-stream/lib/_stream_readable.js:790:12)
      at processTicksAndRejections (internal/process/task_queues.js:84:21)
      at runNextTicks (internal/process/task_queues.js:66:3)
      at processImmediate (internal/timers.js:429:9)
      at process.topLevelDomainCallback (domain.js:137:15)

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information ---------------------------
     Operating System:          darwin
     Node Version:              12.16.3
     Framework Version:         1.71.3
     Plugin Version:            3.6.12
     SDK Version:               2.3.0
     Components Version:        2.30.11

service:
  name: aws-node-typescript-nest

plugins:
  - serverless-offline
  - serverless-layers
  - serverless-plugin-typescript

provider:
  name: aws
  runtime: nodejs12.x
  deploymentBucket: aaaaa-layers
  region: eu-west-1

functions:
  app:
    handler: src/index.handler
    events:
      - http:
          cors: true
          path: "/"
          method: any
      - http:
          cors: true
          path: "{proxy+}"
          method: any

Private npm package support

Hi

I'm using some private npm packages in my project.

Even though, we are using this command echo "//registry.npmjs.org/:_authToken=$(NPM-TOKEN)" > ~/.npmrc, it cannot find our private npm packages.

Layer versions are not updated to the latest version number

When updating our lambdas the layer version for each lambda is not changed into the latest version.

I just added a new npm module to our package.json and the module created a new layer package and uploaded. However the reference to the new layer version is not updated and is still referencing the old layer number.

Changes indication

Hello!

First of all, thank you for this project! It saves my life for years now!

I had to start updating an old project today I and that project was using v1. When I saw that v2 was available, I could not find any patch notes nor any documentation on what changed from one version to another.

What changed from v1 to v2? Do I need to pay attention to anything before upgrading so I don't break the project?
It would be great to have some patch notes for the new versions.

Imposible to use with ci deployment

Why and when?

1. During a deployment to production, I use npm ci to install trusted versions of dependencies.
   The plugin, regardless, tries to install dependencies again, which install the latest version from package.json.
2. When I tried using "customInstallationCommand" parameter, I got an error,
   npm ERR! cipm can only install packages with an existing package-lock.json or npm-shrinkwrap.json with lockfileVersion >= 1. Run an install with npm@5 or later to generate it, then try again.

Possibly because of the fact that npm when being used programmatically is not reading lock file in the base folder automatically.

Possible solution:

• option to disable package installation
• option to compare package-lock.json or npm-shrinkwrap.json

Incompatible with dynamic deployment buckets

If no deployment bucket is configured, sls deploy will create a bucket dynamically, but this plugin does not pick up that bucket name and instead fails the entire deploy. This plugin would be very useful to package node_modules as a dedicated layer and slim down individual function packages.

Allow layer creation of user built modules

I am new to serverless and from the jump, I noticed that layers using the plugin "serverless-layers" cannot include user-defined modules (common code).

Please add support to allow layer creation from folders (which can also include requirements.txt)

Output pythonDashdefaultLambdaLayerQualifiedArn is already deployed in another alias with a different configuration. Either you change your output to match the other definition, or you change the logical resource id to deploy your output separately.

Feature request - Allow to specify "compatible architectures"

This is a feature request to support specifying the value of "compatible architectures"

Same feature as option "compatibleArchitectures" in core serverless
https://www.serverless.com/framework/docs/providers/aws/guide/layers/

Support lockfiles

This is a fantastic plugin, and there is just one thing that is making me hesitant about using it in production.

As far as I can see, if a change is made to package-lock.json / yarn.lock (meaning a different version of a dependency is installed) this is not picked up by the plugin and the layer is not recreated.

Are there any plans to support lockfiles? I had a quick look at the code but could not see an obvious way to do it.

always use latest layer version

When i redeploy my lambda it uses the layer version 1 even though i the latest version is 2 after changing the package.json
i changed it manually and it works but the plugin should do it for me

Feature request - support prebuilt archive to create layer

This is a feature request to support prebuilt zip archive to create layer

This is already available in base serverless:
https://www.serverless.com/framework/docs/providers/aws/guide/layers/

Npm dependencies installed locally do not create a new layer

Hi, it's me again, I would like to know if there is already a way to force the installation of the premises and the creation of a new layer when deploying.

I'm using a locally installed dependency to develop before publishing it on npm and every time I update it and try to deploy it a new layer is not created.

Looking at the code and thinking about a possible solution it seems that if you deploy using the command sls deploy --force-install it is possible to identify the argument --force-install in the options property in the main plugin class. I don't know if edde is the most appropriate way to do this.

This is a print of the index.js main function, explaining a possible solution that seems to work for me.