ph200 / sharp-layer

npm sharp for AWS Lambda layers, separated builds for arm64 and x64, minified and bundled with esbuild

sharp for AWS Lambda Layers

About

The prebuilt sharp node module for AWS Lambda layer.

Features

• Built and tested automatically using GitHub Actions
• Automatically releases sharp updates with GitHub Actions
• Separated builds for arm64 and x64
• Minified and bundled with esbuild
• Minimum 6.98 MB zip file to optimize cold start time

Why use a bundled Lambda function? / Why separate build for arm64?

Please check out Optimizing Node.js dependencies in AWS Lambda for details. A bundled and minified lambda function can be up to 70% faster for cold starts. The package size is also crucial for cold start performance.

Download

Releases

Download latest release-arm64.zip or release-x64.zip

Usage

import sharp from 'sharp'

Check out aws: Creating and sharing Lambda layers for more details.

This package can be used with sst. Check out docs.sst.dev: Lambda Layers and sst.dev: Resize Images for examples.

Setting arm64 for sst functions

function: {
  handler: '{handler}',
  runtime: 'nodejs18.x',
  architecture: 'arm_64',
  nodejs: {
    esbuild: {
      external: ['sharp'],
    },
  },
  layers: [
    new lambda.LayerVersion(stack, 'SharpLayer', {
      code: lambda.Code.fromAsset('layers/sharp'),
      compatibleArchitectures: [
        lambda.Architecture.ARM_64
      ]
    }),
  ]
}

Setting up a lambda layer for AWS SAM

Providing a zip file locally actually works, even though it's not mentioned in the documentation.

  ## Lambda
  ImageFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: image-lambda/
      Handler: app.handler
      Runtime: nodejs18.x
      Architectures:
        - arm64
      Timeout: 30
      MemorySize: 1024
      Layers:
        - !Ref SharpLayer
    Metadata:
      BuildMethod: esbuild
      BuildProperties:
        # Check these two issues for problems related to esm and esbuild
        # https://github.com/evanw/esbuild/issues/1921
        # https://github.com/evanw/esbuild/pull/2067#issuecomment-1503688128
        # Switch to cjs when esm doesn't work
        Format: esm
        OutExtension:
          - .js=.mjs
        EntryPoints:
          - app.ts
        External:
          - '@aws-sdk/*' # @aws-sdk 3.x is installed globally for nodejs18.x
          - sharp # use layer
  ## Lambda layer
  SharpLayer:
    Type: AWS::Serverless::LayerVersion
    Properties:
      LayerName: sharp
      ContentUri: layers/sharp/release-arm64.zip # zip
      CompatibleArchitectures:
        - arm64
      CompatibleRuntimes:
        - nodejs18.x
        - nodejs16.x

Build

Fork this repo -> Actions -> Run build.yml

References

Umkus/lambda-layer-sharp - another maintained sharp lambda layer

aws: Creating and sharing Lambda layers

aws: Working with layers

aws: Building layers

sharp: Installation - AWS Lambda

sharp-layer's Issues

/var/task/main: /lib64/libc.so.6: version `GLIBC_2.34' not found

Awesome project 👍

Before all I must say that I'm not directly using your pre-build releases. So feel free to close this issue if you think it's not adequate.

I've copy/pasted your repository and adapted it for my needs but I'm getting the following runtime errors when running my Lambda:

/var/task/main: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by /var/task/main)
/var/task/main: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by /var/task/main)

What could be wrong?

This is my install script:

SHARP_VERSION="0.33.2"

echo ""
echo "Install and compile sharp v${SHARP_VERSION} for AWS Lambda Linux AMI environment"

# Install and compile Sharp for the AWS Lambda Linux AMI environment.
# Tells to explicitly install libvips in node_modules instead of using global libvips
SHARP_IGNORE_GLOBAL_LIBVIPS=1 \
npm install sharp@${SHARP_VERSION} \
  --save=false \
  --os=linux \
  --cpu=arm64 \
  --libc=glibc

# Remove musl-based binary
rm -rf ./node_modules/@img/*-linuxmusl*

And my build script:

# Build sharp using ESBuild
./node_modules/.bin/esbuild \
  --bundle ./node_modules/sharp/ \
  --outfile=index.js \
  --minify \
  --format=cjs \
  --platform=node

# Move index.js
mkdir -p ./dist/nodejs/node_modules/sharp/lib
mv index.js ./dist/nodejs/node_modules/sharp/lib/

# Copy
cp -a node_modules/sharp/package.json ./dist/nodejs/node_modules/sharp/
cp -a node_modules/sharp/LICENSE ./dist/nodejs/node_modules/sharp/
cp -a node_modules/@img ./dist/nodejs/node_modules/

I'm running everything using GitHub Actions using ubuntu-22.04 the same as you.
Along with actions/setup-node@v4

- name: Set up Node.js
  uses: actions/setup-node@v4
  with:
    node-version: 20.x

Is it because I'm building using ubuntu-22.04 that uses libc6-dev version 2.35? 🤔

Name	Version
libc++-dev	1:14.0-55~exp2
libc++abi-dev	1:14.0-55~exp2
libc6-dev	2.35-0ubuntu3.6
libcurl4	7.81.0-1ubuntu1.15

You are using the same ubuntu-22.04 so how is it working for you? 🤔 (is it?)

Bug. Both `linux` & `linuxmusl` are installed

Steps to reproduce:

• download latest release from this repo
• observe both linux & linuxmusl are installed

Shouldn't only linux be installed under @img?

Lambda bundled with webpack can't import sharp module

Using webpack (5.88.2) with your 0.32.6 release (https://github.com/pH200/sharp-layer/releases/download/0.32.6/release-x64.zip). I have the same sharp package installed locally (linux/x86).

I have created the layer using the zip referenced above (nodejs 18, same as local) and added to the Lambda that needs it. Source has been bundled with webpack.

Receiving the following: Runtime.ImportModuleError: Error: Cannot find module 'sharp'.

I think it may have to do with the externals property used by webpack. I have followed the instructions from the sharp documentation: https://sharp.pixelplumbing.com/install#aws-lambda. Any experience here?

Here is the full CloudWatch log error:

{
    "errorType": "Runtime.ImportModuleError",
    "errorMessage": "Error: Cannot find module 'sharp'\nRequire stack:\n- /var/task/EventIotTriggerPreTraffic.js\n- /var/runtime/index.mjs",
    "stack": [
        "Runtime.ImportModuleError: Error: Cannot find module 'sharp'",
        "Require stack:",
        "- /var/task/EventIotTriggerPreTraffic.js",
        "- /var/runtime/index.mjs",
        "    at _loadUserApp (file:///var/runtime/index.mjs:1087:17)",
        "    at async UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1119:21)",
        "    at async start (file:///var/runtime/index.mjs:1282:23)",
        "    at async file:///var/runtime/index.mjs:1288:1"
    ]
}

Webpack externals config:

externals: {
    'sharp': 'commonjs sharp'
}

Also tried (path from exec ls -FlaR /opt on ephemeral storage):

externals: {
    'sharp': '/opt/nodejs/node18/node_modules/sharp'
}

Still no HEIF support?

Hi, thanks for your work @pH200

I have installed the layer and attached it to my main application but I am still getting the HEIF not supported error.

I am reading on the libvips site that the HEIF is now supported. Do you know anything about that support?

Many users are using iPhones and that is the default format on new iPhones now and it would be bad to have some image handling system without that support.