Unable to re-build with node-oracledb dependency about nodejs-buildpack

[warroyo]

@codeguruRahul

here are the changes I made to get this to work. I have listed the files and additions I made below.

lib/environment.sh

this is a function that will set up your environment variables for all of the oracle libraries

add_oracle_env(){
    local build_dir="$1"
        export LD_LIBRARY_PATH=$build_dir/oracle/instantclient:${LD_LIBRARY_PATH:-}
    export OCI_LIB_DIR=$build_dir/oracle/instantclient
    export OCI_INC_DIR=$build_dir/oracle/instantclient/sdk/include
  }

bin/compile.sh

add this line to the create_env() function. this executes the function above with the paramteter for the build directory

add_oracle_env "$BUILD_DIR/.heroku"

add this line to the top of the build_dependencies() function. this executes the function that will install the actual oracle libraries.

install_oracle_libraries "$BUILD_DIR/.heroku"

profile/nodejs.sh

add these into this file, i added them right after the NODE_ENV is exported.

export LD_LIBRARY_PATH=$HOME/.heroku/oracle/instantclient:${LD_LIBRARY_PATH:-}
export OCI_LIB_DIR=$HOME/.heroku/oracle/instantclient
export OCI_INC_DIR=$HOME/.heroku/oracle/instantclient/sdk/include

lib/dependecies.sh

create this function, I did this at the top of the file, it should not matter where you put it. this is executed in the compile script, that we added above. you will need to replace the download urls with proper locations that you can access. I downloaded the instantclient-basic and sdk and put them, into artifactory.

install_oracle_libraries(){
  echo $HOME
  local build_dir=${1:-}
  echo "Installing oracle libraries"
  mkdir -p $build_dir/oracle
  cd $build_dir/oracle
  local basic_download_url="https://myartifatcory.com/artifactory/thirdParty/oracle/instantclient-basic-linux.x64-12.1.0.2.0.zip"
  local sdk_download_url="https://myartifatcory.com/artifactory/thirdParty/oracle/instantclient-sdk-linux.x64-12.1.0.2.0.zip"
  curl -k "$basic_download_url" --silent --fail --retry 5 --retry-max-time 15 -o instantclient-basic.zip
  echo "Downloaded [$basic_download_url]"
  curl -k "$sdk_download_url" --silent --fail --retry 5 --retry-max-time 15 -o instantclient-sdk.zip
  echo "Downloaded [$sdk_download_url]"
  echo "unzipping libraries"
  unzip instantclient-basic.zip
  unzip instantclient-sdk.zip
  mv instantclient_12_1 instantclient
  cd instantclient
  ln -s libclntsh.so.12.1 libclntsh.so
}

you should be able to build the buildpack after this and it will bundle in the oracle libraries and npm will know about them.

let me know if there is anything that is not clear.

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/121983337

The labels on this github issue will be updated when the story is started.

[RochesterinNYC]

Hi @warroyo, if there are Oracle system libraries that you're missing that you need during the Nodejs buildpack compilation step, you can either:

• Fork the nodejs buildpack and install these oracle libraries in the bin/compile script of your fork before they are needed.
• Create your own stacks that includes these libraries.
• Ask around on cf-dev if others are interested or have use cases for adding these oracle libraries to stacks.

We are also exploring a feature narrative around pre-staging runtime hooks that might fulfill this use case and would allow you to specify that you want to install these oracle libraries before the staging/compilation step for this app.

[warroyo]

@RochesterinNYC thank you for giving me some direction around this issue. I think the first option is what I try for now.

[warroyo]

I was able to get this working by modifying the compile script.

[geeky-ant]

@warroyo Can you please give me some idea in detail how did you fix it? What changes you did in compile file before creating your own buildpack. I am also facing the same issue.

[geeky-ant]

@warroyo Thankyou so much...Its working :)

[warroyo]

@RochesterinNYC any chance this could be added to the core code base? I'm sure it could be slightly modified to have a flag so that it could be optional if people do not need the oracle libs.

[totapari]

Hi @warroyo ,
Firstly thank you for list of file and additions for setting up the oracle libraries..,
After setting up all the files and additions mention above build failed after Installing oracle libraries line.
logs:
-----> Installing binaries
engines.node (package.json): 4.6.0
engines.npm (package.json): 4.0.3
Downloading and installing node 4.6.0...
Downloading and installing npm 4.0.3 (replacing version 2.15.9)...
-----> Restoring cache
Skipping cache restore (disabled by config)
/app
Installing oracle libraries
! Push rejected, failed to compile Node.js app.
! Push failed

[totapari]

Hi @warroyo ,
Firstly thank you for list of file and additions for setting up the oracle libraries..,
After setting up all the files and additions mention above build failed after Installing oracle libraries line.
logs:
-----> Installing binaries
engines.node (package.json): 4.6.0
engines.npm (package.json): 4.0.3
Downloading and installing node 4.6.0...
Downloading and installing npm 4.0.3 (replacing version 2.15.9)...
-----> Restoring cache
Skipping cache restore (disabled by config)
/app
Installing oracle libraries
! Push rejected, failed to compile Node.js app.
! Push failed

[th1agarajan]

@warroyo

I am trying to connect oracle DB from NodeJs. My program was running perfectly in local but when I tried to push the code to PCF(pivotal cloud foundry) I am getting the following error "ORA-12541: TNS:no listener"

I forked the nodejs buildpack and installed Oracle libraries

My build pack with Oracle libraries can be found here
https://github.com/sirish595/nodejs-oracledb-buildpack

I followed the instructions mentioned in the below link by warroyo
#65

Can you help me out?

i see the following line in bin/compile.sh and profile/nodejs.sh

bin/compile.sh

add_oracle_env "$BUILD_DIR/.heroku"

profile/nodejs.sh

export LD_LIBRARY_PATH=$HOME/.heroku/oracle/instantclient:${LD_LIBRARY_PATH:-}
export OCI_LIB_DIR=$HOME/.heroku/oracle/instantclient
export OCI_INC_DIR=$HOME/.heroku/oracle/instantclient/sdk/include

is it something specific to heroku. Do I need to make changes according to cloud foundry?

[dgodd]

@th1agarajan I'm not an oracle specialist, but the error sounds like the problem is that your app can't talk to the database. Do you have firewalls on CF? oracle database? You could cf ssh into your app and see if you can telnet to the oracle host/port

[th1agarajan]

The code snipped by @warroyo is based on heroku. Will it work on Cloud Foundry?

[sclevine]

The code provided by @warroyo is based on an old version of the Cloud Foundry buildpack that used a .heroku subdirectory in the build directory. Your fork also appears to be based on the old version of the buildpack.

Multi-buildpack support should allow you to do this without forking the buildpack. See: https://docs.cloudfoundry.org/buildpacks/custom.html

[warroyo]

@sclevine is correct, the comment i made above was written for the old buildpack , the buildpack has been switched over to use libbuildpack so my notes will not longer work. if you are using the older version of the buildpack it will work on CF I made the modifications while testing on CF and have not tried it on heroku. I will also note that I built the buildpack in offline mode and deployed it.

@th1agarajan as for your error i would have to agree with @dgodd if the app is starting up and failing with that error, there is likely a firewall issue/network connectivity issue. if there was a buildpack issue it would fail before it even attempted to start the application.

[th1agarajan]

@warroyo Thank you for looking into this. I'm able to connect oracle DB from spring boot service deployed in same PCF instance. Can you provide a GitHub link where I can get the latest NodeJS Cloud Foundry buildpack with Oracle support? Or a link where I can get the steps to create one. The link (https://docs.cloudfoundry.org/buildpacks/custom.html) provide by @sclevine is very high level. I am not able to follow. Your help is highly appreciated.

[th1agarajan]

Finally, i fixed the issue. It was because of the PCF space. I tried pushing the code to different space and it worked perfectly. Thank you for your help.