claygregory / serverless-prune-plugin Goto Github PK

It appears that the behavior of this plugin is to delete old lambda function versions without deleting their s3 object. This does prevent the old versions from counting toward the s3 code size limit, but some users may also want to delete the functions' s3 objects as well to keep s3 costs down.

It looks like passing includeLayers: true parameter doesn't remove the old layers created by the "serverless-layers" plugin.

Is it possible to add support for this plug in as well.

I came across this log output today:

service-name: Recoverable error occurred (Rate exceeded), sleeping for ~4 seconds. Try 1 of 20
service-name: Recoverable error occurred (Rate exceeded), sleeping for ~7 seconds. Try 1 of 20
service-name: Recoverable error occurred (Rate exceeded), sleeping for ~6 seconds. Try 1 of 20
service-name: ✔ Pruning of functions complete
service-name: No changes to deploy. Deployment skipped. (36s)

The service was not deployed since there were no changes but the pruning still executed and did nothing other than query AWS for the version of all lambda functions in the service and running in to a AWS rate limit.

I think this could be avoided with a check in index.js postDeploy, similar to what Serverless detects and does internally:

if (this.options.noDeploy === true || this.serverless.service.provider.shouldNotDeploy) {
      return BbPromise.resolve();
    }

What do you think?

It could either mean that pruning is disabled or that no old versions will be kept.

This is not clear to me

I've been trying to use this command to resolve "code limit storage" error. But unfortunately, nothing happens when I run the command sls prune -n <number>. The command runs for less than 5 seconds and exits telling pruning complete. Any help would be highly appreciable.

Deprecation warning: CLI options definitions were upgraded with "type" property (which could be one of "string", "boolean", "multiple"). Below listed plugins do not predefine type for introduced options:
- Prune for "number", "stage", "region", "function", "layer", "includeLayers", "dryRun"
Please report this issue in plugin issue tracker.
Starting with next major release, this will be communicated with a thrown error

Framework Core: 2.72.2 (local)
Plugin: 5.5.4
SDK: 4.3.0
Components: 3.18.2

It happens when this.log is called from the plugin.
As log isn't injected in the plugin constructor (maybe it is in v2 or v3), it uses serverless.cli.log.
But this.log then calls serverless.cli.log (from CLI.js):

  log(message, entity, opts) {
    const underline = opts ? opts.underline : false;
    const bold = opts ? opts.bold : false;
    const color = opts ? opts.color : null;

    let print = chalk.yellow;

    if (color) print = chalk.keyword(color);
    if (underline) print = print.underline;
    if (bold) print = print.bold;

    this.consoleLog(`${entity || 'Serverless'}: ${print(message)}`);
  }

In that context, it seems this is affected to Prune so this.consoleLog does not exist.
To fix it, I guess the faster will be to create a consoleLog method in the Prune class, should I submit a PR then?

Here is the error.

  TypeError: this.consoleLog is not a function
      at Prune.log (/Users/j0k/.nvm/versions/node/v12.22.7/lib/node_modules/serverless/lib/classes/CLI.js:414:10)
      at Prune.logNotice (/Users/j0k/Sites/20mn/heimdall/node_modules/serverless-prune-plugin/index.js:348:15)
      at Prune.cliPrune (/Users/j0k/Sites/20mn/heimdall/node_modules/serverless-prune-plugin/index.js:102:12)
      at /Users/j0k/.nvm/versions/node/v12.22.7/lib/node_modules/serverless/lib/classes/PluginManager.js:498:55
      at tryCatcher (/Users/j0k/.nvm/versions/node/v12.22.7/lib/node_modules/serverless/node_modules/bluebird/js/release/util.js:16:23)
      at Object.gotValue (/Users/j0k/.nvm/versions/node/v12.22.7/lib/node_modules/serverless/node_modules/bluebird/js/release/reduce.js:168:18)
      at Object.gotAccum (/Users/j0k/.nvm/versions/node/v12.22.7/lib/node_modules/serverless/node_modules/bluebird/js/release/reduce.js:155:25)
      at Object.tryCatcher (/Users/j0k/.nvm/versions/node/v12.22.7/lib/node_modules/serverless/node_modules/bluebird/js/release/util.js:16:23)
      at Promise._settlePromiseFromHandler (/Users/j0k/.nvm/versions/node/v12.22.7/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:547:31)
      at Promise._settlePromise (/Users/j0k/.nvm/versions/node/v12.22.7/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:604:18)
      at Promise._settlePromiseCtx (/Users/j0k/.nvm/versions/node/v12.22.7/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:641:10)
      at _drainQueueStep (/Users/j0k/.nvm/versions/node/v12.22.7/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:97:12)
      at _drainQueue (/Users/j0k/.nvm/versions/node/v12.22.7/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:86:9)
      at Async._drainQueues (/Users/j0k/.nvm/versions/node/v12.22.7/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:102:5)
      at Immediate.Async.drainQueues [as _onImmediate] (/Users/j0k/.nvm/versions/node/v12.22.7/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:15:14)
      at processImmediate (internal/timers.js:461:21)

Hi 👋

As far as I understand using this plugin in conjunction with versionFunctions: false is useless, am I right?

Hey,
This plugin was working perfectly with nodejs 6.x and 7.x
But when I went back to 4.3, I get the following error:

sls prune -n 4
 
  Syntax Error -------------------------------------------
 
     Unexpected token [

Which I think is due to the following line:
https://github.com/claygregory/serverless-prune-plugin/blob/master/index.js#L57

Any chance you could modify the .each block to be backwards compatible?

Since you're using Travis, you could also add a few older versions to your .travis.yml file. Maybe test the same versions that the serverless project does?

Thanks!

I was surprised to see auto pruning being attempted when running with the --noDeploy flag.

Problem Statement :

In serveless.yml we can configure multiple lambda functions under the functions key.

But now if we use the serverless-prune-plugin then the prune setting would get applied to all the lambdas.

How to restrict the pruning to just a specific lambda inside this serverless.yml file.?

Note : This needs to be done for automatic pruning.

When sls deploy is run with auto prune turned on and 5 versions, I see the pruning actions being fired in CloudFormation, but they are not completing - instead getting DELETE_SKIPPED. I do not have any DeletionPolicy set for these resources (lambda functions and associated api), which was my first guess. I am going to dig deeper, was just wondering if anyone else has run into this issue.

Hello guy,

I received that kind of error:
Lambda was unable to delete LAMBADAQUALIFIEDARN:2 because it is a replicated function. Please see our documentation for Deleting Lambda@Edge Functions and Replicas.

Is there any way do delete lambda @ edge non-used version(s) or some way to prevent plugin from deleting only lambda @ edge versions?

hi, Clay Gregory.
I used this plugin in my serverless project.

Actually I need to manage manay serverless projects and each of them has several Lambda functions.

And I deploy these serverless projects by using CloudFormation in CodePipeline deploy stage.
So in my scenario:
①I need to sls package to get CF template file
② upload jars to a specified location in S3
③ use CF with the template file generated in step ① to update a stack.

I update serverless.yml with

custom:
  prune:
    automatic: true
    number: 10

In this plugin ReadMe file, this config will automatic pruning. but it doesn't work in my situation.
Any idea?

I am on the serverless 2.54.0 version and receiving the following console warning around using this plugin.

CLI options definitions were upgraded with "type" property (which could be one of "string", "boolean", "multiple"). Below listed plugins do not predefine type for introduced options:
 - Prune for "number", "stage", "region", "function", "layer", "includeLayers", "dryRun"
Please report this issue in plugin issue tracker.
Starting with next major release, this will be communicated with a thrown error.

Can we please fix this issue?

I noticed the readme mentions it targets Serverless 1.x. Is there anything preventing it working on 2.x?

Thanks!

Hi,
first of all great plugin, but there could be more documentiation regarding the options:

E.g. The --stage option is also available not sure if there are more options possible.
I just tried it and was glad it worked.

When auto pruning, there should be minimal logs. Consider adding a debug or verbose option for automatic pruning to get detailed logs, like what's currently output.

With the current output, the prune logs push the deployed service information off the screen for big projects.

Hi,

When enabling automatic layer pruning in the serverless project, we get an error. We are unsure if this is a bug or a misconfiguration.

Steps to reproduce

• Create a project with a function and a layer
• Add the following configuration in the serverless.yml file:

custom:
  prune:
    includeLayers: true
    automatic: true
    number: 5

Expected result

• Plugin should automatically prune old versions of the layers and the functions

Actual result

  Serverless Error ---------------------------------------

  ServerlessError: Missing required key 'LayerName' in params
    at promise.catch.err (/Users/bernatrafales/.nvm/versions/node/v10.7.0/lib/node_modules/serverless/lib/plugins/aws/provider/awsProvider.js:316:27)
    at process._tickCallback (internal/process/next_tick.js:68:7)

Do we have to explicitly list all layers we want the plugin to prune or should the plugin figure it out itself?

Thanks.

Serverless version information

sls --version                                                                                                                                                                                                                                                                                                
Framework Core: 1.50.0
Plugin: 1.3.8
SDK: 2.1.0

Plugin version information
1.4.0

Hi,

First, awesome plugin! I'm using serverless 1.9.0 and did 'sls prune -n=1', however, I cannot deploy my lambdas anymore because I'm getting UPDATE_ROLLBACK_FAILED and my CloudFormation on AWS dashboard is broken. Any ideas? As the project is not in production yet, it's ok to remove and recreate the stack again, but that would cause some problems if that would happen on production.

Can we make the number key accept object so that we can specify values for multiple envs

custom:
  prune:
    automatic: true
    number:
      dev: 1
      prod: 3
      other: 3

Also supporting old behavior

custom:
  prune:
    automatic: true
    number: 3

IMHO This helps in a more declarative process in CI/CD

And also if this is acceptable I would like to work on it.

Deprecation warning: CLI options definitions were upgraded with "type" property (which could be one of "string", "boolean", "multiple"). Below listed plugins do not predefine type for introduced options:
- Prune for "number", "stage", "region", "function", "layer", "includeLayers", "dryRun"
Please report this issue in plugin issue tracker.

I'm guessing that the plugin does an alphabetical sort, not numeric, when deciding what to delete, so 9 and 8 appear to be more recent than e.g. 64 and 63

First, thanks for the amazing plugin!

I've installed it in my project and run the sls prune -s prod -n 10.
Most of my project lambdas had 240 versions, and I thought it would remove 230 of them, keeping only the newest 10. But instead, it removed the oldest 10 40, and kept 200 most recent ones.

The docs on README says the following

This will delete all but the n-most recent versions of each function deployed. Versions referenced by an alias are automatically preserved.

The plugin lets you manually prune to the specified number of versions, but would it be possible to add a command that would list the number of versions per function, prior to just choosing a number and commencing prune?

Not sure if this would really be that desirable, but it's a little frustrating that Serverless/AWS in general don't surface those numbers easily.

It would be pretty useful if we could set a custom var or something representing n last versions and have the plugin automatically prune instead of only being able to do it via CLI.

Hi 👋 , Serverless Framework core team here!

We're working on the v3 beta version right now. The v3 will include a redesigned CLI output:

You can try the v3 beta with: npm i serverless@pre-3

To help plugins integrate with that new design (and build a great user experience), we've added new APIs for plugins:

• Video introduction (Youtube)
• Documentation of the new API

Any feedback is appreciated! If you want to update the plugin to use these new APIs, feel free to ping me if you have any questions!

If you don't have time to update the plugin, let me know, we can send a pull request!

First of all thanks for the plugin, we reached the limit of our account and couldn't deploy. But now after prunning we get a lot of these errors when trying to do a deploy:

ERROR in ./admin_approve.js
Module not found: Error: Can't resolve 'source-map-support/register' in '/Users/LuisMiguelDelgado/Projects/Kickass/kada/kada-serverless/src/services/demos'
 @ ./admin_approve.js 89:0-38

ERROR in ./admin_activity.js
Module not found: Error: Can't resolve 'source-map-support/register' in '/Users/LuisMiguelDelgado/Projects/Kickass/kada/kada-serverless/src/services/demos'
 @ ./admin_activity.js 108:0-38

ERROR in ./email.js
Module not found: Error: Can't resolve 'source-map-support/register' in '/Users/LuisMiguelDelgado/Projects/Kickass/kada/kada-serverless/src/services/demos'
 @ ./email.js 100:0-38

ERROR in ./admin_reject.js
Module not found: Error: Can't resolve 'source-map-support/register' in '/Users/LuisMiguelDelgado/Projects/Kickass/kada/kada-serverless/src/services/demos'
 @ ./admin_reject.js 115:0-38

Apparently it can't resolve any dependency issues and we can't deploy after prunning for the first time, has anyone else had this happen?

As native Promise is available everywhere, we should get rid of bluebird and replace it with native promises

$ node --version
v16.13.1

$ npm --version
8.1.2

Packages Used:
"serverless": "^2.52.1",
"serverless-prune-versions": "^1.0.4",

CLI options definitions were upgraded with "type" property (which could be one of "string", "boolean", "multiple"). Below listed plugins do not predefine type for introduced options:

• PrunePlugin for "number", "stage", "region", "function", "layer", "includeLayers", "dryRun"

Please report this issue in plugin issue tracker.

I am using serverless v2.39.2.

When using the prune plugin with CLI options in the custom section of my serverless file, I get the following warning:

Deprecation warning: CLI options definitions were upgraded with "type" property (which could be one of "string", "boolean", "multiple"). Below listed plugins do not predefine type for introduced options: - Prune for "number", "stage", "region", "function", "layer", "includeLayers", "dryRun" Please report this issue in plugin issue tracker. Starting with next major release, this will be communicated with a thrown error. More Info: https://www.serverless.com/framework/docs/deprecations/#CLI_OPTIONS_SCHEMA

Steps to reproduce:

yarn add --dev serverless-prune-plugin
// Add plugin to serverless file
sls prune -n 10 --stage {stage}  --dryRun 

All versions for all functions in file will be marked as selected for deletion. Running without --dryRun will delete all versions.

Environment Info:

Serverless: 2.4
yarn: 1.22.10
serverless-prune-plugin: ^2.0.1
Ubuntu: 20.04.2 LTS

serverless doctor gives that:

CLI options definitions were upgraded with "type" property (which could be one of "string", "boolean", "multiple"). Below listed plugins do not predefine type for introduced options:
 - PrunePlugin for "number", "stage", "region", "function", "layer", "includeLayers", "dryRun"

if serverless-prune-plugin is compatible with serverless version 3.19,.

I got this error for functions while deploying.
##[error]Prune: Running post-deployment pruning
##[error]Prune: Querying for deployed function versions
##[error]Prune: [functionName] has 3 additional versions published and 0 aliases, 1 version selected for deletion
##[error]Prune: Deleting Function [functionName] v374...

configuration:
Framework Core: 3.19.0
Plugin: 6.2.2
SDK: 4.3.2

serverless.yml
plugins:

• serverless-prune-plugin

custom:
prune:
automatic: true
number: 2

Right now if you try to use it will serverless 4, it shows the following error when installing:

npm i
npm ERR! code ERESOLVE
npm ERR! ERESOLVE could not resolve
npm ERR!
npm ERR! While resolving: [email protected]
npm ERR! Found: [email protected]
npm ERR! node_modules/serverless
npm ERR!   dev serverless@"^4.0.23" from the root project
npm ERR!   peer serverless@">= 1.26.0" from [email protected]
npm ERR!   node_modules/serverless-plugin-canary-deployments
npm ERR!     dev serverless-plugin-canary-deployments@"^0.8.0" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer serverless@"1 || 2 || 3" from [email protected]
npm ERR! node_modules/serverless-prune-plugin
npm ERR!   dev serverless-prune-plugin@"^2.0.2" from the root project
npm ERR!
npm ERR! Conflicting peer dependency: [email protected]
npm ERR! node_modules/serverless
npm ERR!   peer serverless@"1 || 2 || 3" from [email protected]
npm ERR!   node_modules/serverless-prune-plugin
npm ERR!     dev serverless-prune-plugin@"^2.0.2" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.