Please help me out with deployment issue? about sweph

[krisztianb]

I guess this is happening while running "npm install"?

sweph contains a C wrapper around the swiss emphemeris library. It is compiled during installation. For this you need the C compiler installed on your runtime environment.

I'm using "sweph" on a Heroku server which is using Amazon's EC2 cloud-computing platform and have no problems there.

[timotejroiko]

Indeed, the module contains native C code that needs to be compiled, which requires proper build tools for your environment. I have never tested on AWS lambdas, but assuming they run on amazon linux, common guides suggest using:

yum install gcc44 gcc-c++ libgcc44 cmake

or

yum groupinstall -y "Development Tools"

to install the proper build tools.

Another amazon documentation page suggests pre-building your package on AWS SAM before transferring it to your lambda:

If your deployment package contains native libraries, you can build the deployment package with AWS Serverless Application Model (AWS SAM). You can use the AWS SAM CLI sam build command with the --use-container to create your deployment package. This option builds a deployment package inside a Docker image that is compatible with the Lambda execution environment.

give those a try

[bizprat]

Hi, I tried to build the app in docker container with amazonlinux environment for nodejs

I created a Dockerfile, and build the image. But when I tried running the container from that, I am getting this error:

[Nest] 28  - 09/01/2022, 3:28:58 PM     LOG [NestFactory] Starting Nest application...
node: symbol lookup error: /app/node_modules/sweph/build/Release/sweph.node: undefined symbol: swe_set_ephe_path

I have set the ephemeris path in my code, which is working fine on the host machine.

import { set_ephe_path } from 'sweph';
import { join } from 'path';

set_ephe_path(join(__dirname, '..', 'libs', 'ephemeris'));

Here is my Dockerfile:

FROM public.ecr.aws/lambda/nodejs:16

RUN yum update -y
RUN yum install python37 -y
RUN yum groupinstall -y "Development Tools"

WORKDIR /app
COPY . /app

RUN npm install -g @nestjs/cli
RUN npm install
RUN npm run build:webpack

ENTRYPOINT ["npm"]
CMD ["start"]

EXPOSE 3000

[bizprat]

I guess this is happening while running "npm install"?

No this is happening when a request is made to Lambda function

sweph contains a C wrapper around the swiss emphemeris library. It is compiled during installation. For this you need the C compiler installed on your runtime environment.

I'm using "sweph" on a Heroku server which is using Amazon's EC2 cloud-computing platform and have no problems there.

Yes, it works with EC2, but not with Lambda.

[krisztianb]

Hmmm... Can you tell me what /var/task/node_modules/sweph/build/Release/sweph.node is?

[bizprat]

Hmmm... Can you tell me what /var/task/node_modules/sweph/build/Release/sweph.node is?

It's a native node library build by sweph package

[bizprat]

I tried with another official amazon linux environment with proper build tools but still had no success

yum groupinstall -y "Development Tools"

Deployment was successful and working on the local machine also, got the error when I request the Lambda URL.

2022-09-01T15:38:11.889Z	undefined	ERROR	Uncaught Exception 	{
    "errorType": "Error",
    "errorMessage": "/lib64/libm.so.6: version `GLIBC_2.29' not found (required by /var/task/node_modules/sweph/build/Release/sweph.node)",
    "code": "ERR_DLOPEN_FAILED",
    "stack": [
        "Error: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by /var/task/node_modules/sweph/build/Release/sweph.node)",
        "    at Object.Module._extensions..node (node:internal/modules/cjs/loader:1189:18)",
        "    at Module.load (node:internal/modules/cjs/loader:981:32)",
        "    at Function.Module._load (node:internal/modules/cjs/loader:822:12)",
        "    at Module.require (node:internal/modules/cjs/loader:1005:19)",
        "    at require (node:internal/modules/cjs/helpers:102:18)",
        "    at Object.<anonymous> (/var/task/node_modules/sweph/index.js:3:15)",
        "    at Module._compile (node:internal/modules/cjs/loader:1105:14)",
        "    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)",
        "    at Module.load (node:internal/modules/cjs/loader:981:32)",
        "    at Function.Module._load (node:internal/modules/cjs/loader:822:12)"
    ]
}

Dockerfile

FROM amazonlinux:2022.0.20220824.0

SHELL ["/bin/bash", "-c"] 

RUN yum update -y
RUN yum install nodejs npm -y
RUN yum install python3 -y
RUN yum groupinstall -y "Development Tools"

ENV APP_PATH /app
WORKDIR ${APP_PATH}

ADD package.json /tmp/package.json
RUN cd /tmp && npm install
RUN cp -a /tmp/node_modules ${APP_PATH}


COPY . /app
COPY ./.serverless/.aws/credentials /root/.aws/credentials

RUN npm install -g @nestjs/cli
RUN npm install -g serverless
RUN npm run build:webpack


ENTRYPOINT ["npm"]
CMD ["start"]

[timotejroiko]

Are you doing this inside an EC2 instance or just running docker locally? Try creating an EC2 instance running Amazon Linux, build and test your app in there, then once it works, zip the whole thing and deploy the whole zip file to lambda

[bizprat]

It will run on EC2 as I have checked on heroku.

Can you please help me with this error:

node: symbol lookup error: /app/node_modules/sweph/build/Release/sweph.node: undefined symbol: swe_set_ephe_path

[timotejroiko]

the error means the source files were not properly included during the build process so the symbols are missing.

I can try setting up a lambda myself and see if i can get it to work but will take a while as im not used to AWS

[bizprat]

the error means the source files were not properly included during the build process so the symbols are missing.

I can try setting up a lambda myself and see if I can get it to work but will take a while as I'm not used to AWS

@timotejroiko
Thank you for so much help.
I will also update you here if I get a solution.

[i6mi6]

Facing the same problem with ElasticBeanstalk

node: symbol lookup error: /app/node_modules/sweph/build/Release/sweph.node: undefined symbol: swe_set_ephe_path

A workaround is to use https://github.com/mivion/swisseph instead

[timotejroiko]

Hello, apologies for the long wait, i am attempting to debug this issue now and have found that apparently amazon linux does something that breaks compiler optimizations like -flto

I will try adding an option to disable optimizations at install time, let you guys know soon, in the mean while can you test something for me?

try adding this to your npm install script in your docker image:

CPPFLAGS="-flto -ffat-lto-objects" npm install

EDIT: all optimizations are now removed, try installing from the github main branch timotejroiko/sweph

[timotejroiko]

UPDATE:

I was able to run it in a lambda using the following docker image (git can be removed once the update goes to npm)

FROM public.ecr.aws/lambda/nodejs:latest-x86_64

RUN yum -y install python3 make gcc gcc-c++ git

# Copy function code
COPY index.js package.json ${LAMBDA_TASK_ROOT}

RUN npm install
  
# Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)
CMD [ "index.handler" ]

and package.json

{
  "dependencies": {
    "sweph": "github:timotejroiko/sweph"
  }
}

Let me know if it works for you too, then i'll push the update to npm

[timotejroiko]

UPDATE 2:

With a friend's help we discovered that the cause of all this is that Amazon Linux puts gcc plugins in a non-standard location, so the whole issue can be fixed by symlinking the lto plugin to the standard location as follows:

mkdir /usr/lib/bfd-plugins/
ln -s /usr/libexec/gcc/x86_64-redhat-linux/7/liblto_plugin.so /usr/lib/bfd-plugins/liblto_plugin.so

/usr/lib/bfd-plugins is the standard location for gcc plugins, and /usr/libexec/gcc/x86_64-redhat-linux/7/liblto_plugin.so is the location where i found the plugin in my Amazon Linux docker container (could be different for you), so i created a symlink from there to the standard location and everything works now, with optimizations and everything.

[timotejroiko]

Released a new version on NPM today with the LTO flags disabled, which should now work on AWS lambda by default.

Gonna close this for now, let me know if the issue persists.