Comments (19)
@ptimson I found a different workaround that might be slightly better, but still requires downtime of the service. I commented out following lines after each function:
layers:
- {Ref: PythonRequirementsLambdaLayer}
for all of my functions. This was my one change and I deployed like this once. Once that deployed successfully I added those lines back and was able to deploy again.
from serverless-layers.
Solution that worked for me:
- Remove/Comment lambda layer refs in function declaration
- Deploy
- Add/Uncomment lambda layer refs in function declaration
Just wanted to say Thank You as this solved my problem today as well.
from serverless-layers.
Same, this worked for chalice too! <3
from serverless-layers.
+1
Solution that worked for me: - Remove/Comment lambda layer refs in function declaration - Deploy - Add/Uncomment lambda layer refs in function declaration
It's sad that this still appears to be the only solution. Makes creating new deployments a much more complicated ordeal now that you need to choreograph new layer uploads and function references as two distinct steps.
from serverless-layers.
Hi @joelash, I'm got some time to investigate this issue as well, still not 100% sure yet why happening. But I reckon you guys could try using --no-compile
to avoid generating *.pyc
files. It would reduce considerably the layer size.
custom:
serverless-layers:
dependenciesPath: requirements.txt
packageManagerExtraArgs: '--no-compile --no-color'
compatibleRuntimes: ["python3.8"]
from serverless-layers.
+1
Solution that worked for me:
- Remove/Comment lambda layer refs in function declaration
- Deploy
- Add/Uncomment lambda layer refs in function declaration
from serverless-layers.
+1
Solution that worked for me: - Remove/Comment lambda layer refs in function declaration - Deploy - Add/Uncomment lambda layer refs in function declaration
Thank you βΒ this worked for me, too.
from serverless-layers.
Thanks for this @ptimson. I just hit this exact issues and I'm curious if anyone else who has seen this has other workarounds. Deleting my stack is not easily done at this time given the usage of it in a production environment.
from serverless-layers.
@joelash Are you able to deploy it removing a large library first? I don't really have another solution I assume AWS haven't fixed this yet. Would be keen to know if anyone else has another solution though like you asked but I wouldn't hold out too many hopes!
from serverless-layers.
@ptimson @joelash any luck with this, I'm facing same thing and both:
- removing imports
- removing layers definition from lambda functions, deploy then add it again and redeploy
didn't solve the problem!
from serverless-layers.
We faced the exact same problem when using the UnitedIncome/serverless-python-requirements. Since our workload is being used in production by many many users we had to slim down our dependencies in a previous deployment (and the requirements layer .zip
was very helpful to figure out the large ones) and then push the PR with the Lambda Layer.
I don't think there is other way of doing it with the current inner workings of CloudFormation. We also think this is the way CF works: first attach the new Lambda Layer to the current Lambda Function, then update the code.
from serverless-layers.
Any solution - Tried few things and none of them work:
- Removed the existing layers and redeployed - CF got created but adding the layers back throws the same error
- Added the S3 zip location for the layers and referred them while creating the Layers. Still getting the same error .
Any help would be highly appreciated
from serverless-layers.
Did anyone work out how to get around this, seeing it with nodejs too
from serverless-layers.
@rcoundon and @Soni1712 did you see my comment above? It worked, but it created a bit of downtime during the deploys
from serverless-layers.
@rcoundon and @Soni1712 did you see my comment above? It worked, but it created a bit of downtime during the deploys
Yes, I did, thank you. It was basically what I ended up with but was curious if anyone had a better approach.
from serverless-layers.
@ptimson I found a different workaround that might be slightly better, but still requires downtime of the service. I commented out following lines after each function:
layers: - {Ref: PythonRequirementsLambdaLayer}for all of my functions. This was my one change and I deployed like this once. Once that deployed successfully I added those lines back and was able to deploy again.
This worked! thanks @joelash , but can you explain why? Because technically, in both cases, both zips (layers and functions) uploaded. So I'm confused, why this works :)
from serverless-layers.
@haktan-suren this is my best recollection of why this works, but keep in mind it's been 14ish months since I found this workaround. Let me know if this makes sense or not.
I believe what this did was cause your functions to be deployed without dependencies. Since no function references the -{Ref: PythonRequirementsLambdaLayer}
the layer does not get deployed. This made the function package much smaller since all dependencies were missing.
I think the way the deploy works is that the layers get deployed first. So the size is computed by newLayers + currentlyDeployedFunctions. The issues came about that since the currently deployed functions also had those dependencies package. By deploying the functions without dependencies and without layers you get a smaller size for the "currentlyDeployedFunctions". Then when you do the secondary deploy with using the layers and the layers get deployed the computed size this accurate.
from serverless-layers.
For us, deleting the CloudFormation stack and redeploying worked. I think what @ptimson wrote is correct:
I think CloudFormation was trying to first apply the layer to the function before updating the function code which resulted in the error above.
This is very easy to reproduce. (1) Deploy a lambda with an unzipped size close to 250MB (using the serverless-python-requirements
plugin). (2) Deploy the same function, but now instead of serverless-python-requirements
use serverless-layers
.
I think a combination of numpy
pyarrow
pandas
pycountry
and pydantic
will be enough to get to 250MB
from serverless-layers.
Have run in to this a couple of times now and wanted to share the workaround that worked for us.
We are deploying using the Serverless Framework and essentially what we do is rename the Lambda in the config file. This creates a brand new Lambda, instead of trying to modify the existing Lambda. We are running the Lambda in a Step Function so we update that to use the new Lambda name.
This all seems to work although with minor disruption while the changeover happens, I think due to the way things align in Step Functions. It is preferable to removing and re-adding layers or doing a remove/redeploy as our deployments take several minutes and would result in considerable down time.
Anyway, this might be an option for anyone in this situation and might work with other triggers.
from serverless-layers.
Related Issues (20)
- Layer version not updated when using provision concurency HOT 1
- Command failed: yarn --production HOT 3
- Wrong layer is used HOT 6
- Release new version?
- Not able to detect change in package.json
- Possible to works with workspaces ?
- Python 3.10 and 3.11 support
- Mysteriously increased my layers deployment
- npm warnings for --only=prod
- excluding "aws-dk" broken the build HOT 2
- Python Layers using other layers packages HOT 3
- Is there a way to combine 2 package.json for a single function?
- Hello, can you support runtime: provided.al2023
- node_modules not included in layers HOT 7
- package.json file is not actually copied over in WSL2 HOT 1
- copyProjectFile should NOT call init() every time
- Bug in version 2.8.5 HOT 2
- "Could not find API configuration lambda-2015-03-31" in v2.8.5
- Creating layer fails in versions > 2.8.2 HOT 1
- Run layers command out of serverless hooks
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from serverless-layers.