Stack Manager: ruby gems support? about vertx-stack

[vietj]

so you mean to be able to configure the stack manager to use a specific repository ?

[StephenOTT]

So basically being able to use the same pattern as:

<dependencies>
  <dependency>
    <groupId>rubygems</groupId>
    <artifactId>rails</artifactId>
    <version>.3.0.3</version>
    <type>gem</type>
  </dependency>
  ...
</dependencies>

from: http://rubygems-proxy.torquebox.org
but of course in the json format that stack manager uses.

The Idea being that when using the ruby lang you can download the Gem Dependencies the same way Maven Jar deps are downloaded with the stack manager's resolution process.

Edit: In this scenario the stack manager would support the Maven Repo for the Jars and the Ruby Gems at the same time. (maybe different objects in the vertx-stack.json file?)

[StephenOTT]

@vietj okay i went about this in a different way. I was thinking that taking advantage of ruby's bundler would be better.

So i put this together:

Dockerfile:

...
# Current workaround to make bundler function
ENV PATH=$PATH:/root/.gem/jruby/2.3.0/bin

#Install bundler
RUN java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S gem install --user-install --no-document bundler

# Show gems list so we can confirm bundler is installed
RUN java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S gem list -d

RUN java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S bundle install --path=${VERTICLE_HOME}/bundle --binstubs --gemfile=$VERTICLE_HOME/Gemfile

# Use this when you want to test the path resolution of the search for bundle 
# RUN java -Djruby.debug.scriptResolution=true -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S bundle install --path=${VERTICLE_HOME}/bundle --binstubs --gemfile=$VERTICLE_HOME/Gemfile
...

Full dockerfile example (where the Stack Manager vertx-stack.json file has the vertx-lang-ruby dependency:

# Notice the difference between VERTX_HOME and VERTICLE_HOME.  
# VERTX_HOME is the location of the vertex executable/command

# Alpine cannot be used because Jruby requires cglib to install bundler
FROM vertx/vertx3
# FROM vertx/vertx3-alpine

ENV VERTICLE_HOME /usr/verticles

COPY logging.properties ${VERTICLE_HOME}/logging.properties
ENV VERTX_JUL_CONFIG $VERTICLE_HOME/logging.properties 

# Setup Stack Manager for Vertx
COPY vertx-stack.json ${VERTX_HOME}/vertx-stack.json
# Download and resolve dependencies listed in stack manager
RUN vertx resolve && rm -rf ${HOME}/.m2 


COPY config.json ${VERTICLE_HOME}/config.json

# Setup the cluster Hazelcast configuration
COPY cluster.xml ${VERTICLE_HOME}/cluster.xml

# Copies the starting verticles to the home directory
COPY $VERTICLE_NAME $VERTICLE_HOME/$VERTICLE_NAME
# Copies the sub-verticles into a verticles directory
COPY verticles $VERTICLE_HOME/verticles

COPY Gemfile $VERTICLE_HOME/Gemfile


# Current workaround to make bundler function
ENV PATH=$PATH:/root/.gem/jruby/2.3.0/bin

#Install bundler
RUN java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S gem install --user-install --no-document bundler

# Show gems list so we can confirm bundler is installed
RUN java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S gem list -d

RUN java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S bundle install --path=${VERTICLE_HOME}/bundle --binstubs --gemfile=$VERTICLE_HOME/Gemfile

# Use this when you want to test the path resolution of the search for bundle 
# RUN java -Djruby.debug.scriptResolution=true -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S bundle install --path=${VERTICLE_HOME}/bundle --binstubs --gemfile=$VERTICLE_HOME/Gemfile


# Starts the starting verticle in the home directory
WORKDIR $VERTICLE_HOME
ENTRYPOINT ["sh", "-c"]
# CMD ["exec vertx run $VERTICLE_NAME -cp $VERTICLE_HOME/* $VERTX_ARGS"]
CMD ["exec vertx run $VERTICLE_NAME -cp $VERTICLE_HOME/* -conf config.json $VERTX_ARGS"]

initial design was based on:
https://spin.atomicobject.com/2013/04/24/bundler-jruby-complete/
jruby/jruby#3186 (comment)
jruby/jruby#2488
shoes/shoes4#1168

has not been fully tested yet, but its a start to get the gems in the VERTICLE_HOME.

I think could also do a deps resolution similar to Stack Manager, where we do a bundle package and delete the rest of the files.
http://bundler.io/v1.16/man/bundle-package.1.html

Current console output of the build is:

Building cam-worker_01
Step 1/18 : FROM vertx/vertx3
 ---> e841617fbb56
Step 2/18 : ENV VERTICLE_HOME /usr/verticles
 ---> Using cache
 ---> 44cf2fa51e16
Step 3/18 : COPY logging.properties ${VERTICLE_HOME}/logging.properties
 ---> Using cache
 ---> afa9955265cd
Step 4/18 : ENV VERTX_JUL_CONFIG $VERTICLE_HOME/logging.properties
 ---> Using cache
 ---> 468580126dfe
Step 5/18 : COPY vertx-stack.json ${VERTX_HOME}/vertx-stack.json
 ---> Using cache
 ---> 9658255bf461
Step 6/18 : RUN vertx resolve && rm -rf ${HOME}/.m2
 ---> Using cache
 ---> 346bb8120aa6
Step 7/18 : COPY config.json ${VERTICLE_HOME}/config.json
 ---> Using cache
 ---> 8081e9c47544
Step 8/18 : COPY cluster.xml ${VERTICLE_HOME}/cluster.xml
 ---> Using cache
 ---> 47f2ca8e5d0a
Step 9/18 : COPY $VERTICLE_NAME $VERTICLE_HOME/$VERTICLE_NAME
 ---> a115d4a400ff
Step 10/18 : COPY verticles $VERTICLE_HOME/verticles
 ---> 889284e09f38
Step 11/18 : COPY Gemfile $VERTICLE_HOME/Gemfile
 ---> 56bc678b2bf7
Step 12/18 : ENV PATH=$PATH:/root/.gem/jruby/2.3.0/bin
 ---> Running in b64a637ddd82
Removing intermediate container b64a637ddd82
 ---> d4e3671da0e1
Step 13/18 : RUN java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S gem install --user-install --no-document bundler
 ---> Running in 8d318a04ec10
Successfully installed bundler-1.16.1
1 gem installed
Removing intermediate container 8d318a04ec10
 ---> e4d7ae4eb4a1
Step 14/18 : RUN java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S gem list -d
 ---> Running in 3ba5eab496ed
bundler (1.16.1)
    Authors: André Arko, Samuel Giddins, Chris Morris, James Wen, Tim
    Moore, André Medeiros, Jessica Lynn Suttles, Terence Lee, Carl
    Lerche, Yehuda Katz
    Homepage: http://bundler.io
    License: MIT
    Installed at: /root/.gem/jruby/2.3.0

    The best way to manage your application's dependencies

did_you_mean (1.0.1)
    Author: Yuki Nishijima
    Homepage: https://github.com/yuki24/did_you_mean
    License: MIT
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    "Did you mean?" experience in Ruby

jar-dependencies (0.3.10)
    Author: christian meier
    Homepage: https://github.com/mkristian/jar-dependencies
    License: MIT
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    manage jar dependencies for gems

jruby-openssl (0.9.21)
    Platform: java
    Authors: Ola Bini, JRuby contributors
    Homepage: https://github.com/jruby/jruby-openssl
    Licenses: EPL-1.0, GPL-2.0, LGPL-2.1
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    JRuby OpenSSL

jruby-readline (1.1.1)
    Platform: java
    Author: JRuby contributors
    Homepage: https://github.com/jruby/jruby
    Licenses: EPL-1.0, GPL-2.0, LGPL-2.1
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    JRuby Readline

json (1.8.3)
    Platform: java
    Author: Daniel Luz
    Homepage: http://json-jruby.rubyforge.org/
    License: Ruby
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    JSON implementation for JRuby

minitest (5.4.1)
    Author: Ryan Davis
    Homepage: https://github.com/seattlerb/minitest
    License: MIT
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    minitest provides a complete suite of testing facilities supporting
    TDD, BDD, mocking, and benchmarking

net-telnet (0.1.1)
    Author: SHIBATA Hiroshi
    Homepage: https://github.com/ruby/net-telnet
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    Provides telnet client functionality.

power_assert (0.2.3)
    Author: Kazuki Tsujimoto
    Homepage: https://github.com/k-tsj/power_assert
    Licenses: 2-clause BSDL, Ruby's
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    Power Assert for Ruby

psych (2.2.4)
    Platform: java
    Authors: Aaron Patterson, SHIBATA Hiroshi, Charles Oliver Nutter
    Homepage: https://github.com/ruby/psych
    License: MIT
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    Psych is a YAML parser and emitter

rake (10.4.2)
    Authors: Eric Hodel, Jim Weirich
    Homepage: https://github.com/ruby/rake
    License: MIT
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    Rake is a Make-like program implemented in Ruby

rdoc (4.2.0)
    Authors: Eric Hodel, Dave Thomas, Phil Hagelberg, Tony Strauss
    Homepage: http://docs.seattlerb.org/rdoc
    License: Ruby
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    RDoc produces HTML and command-line documentation for Ruby projects

test-unit (3.1.1)
    Authors: Kouhei Sutou, Haruka Yoshihara
    Homepage: http://test-unit.github.io/
    Licenses: Ruby, PSFL
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    An xUnit family unit testing framework for Ruby.
Removing intermediate container 3ba5eab496ed
 ---> fd59804051cd
Step 15/18 : RUN java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S bundle install --path=${VERTICLE_HOME}/bundle --binstubs --gemfile=$VERTICLE_HOME/Gemfile
 ---> Running in 93dce86c9812
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
installing your bundle as root will break this application for all non-root
users on this machine.
Fetching gem metadata from https://rubygems.org/....
Resolving dependencies.....
Fetching clavius 1.0.3
Installing clavius 1.0.3
Fetching thread_safe 0.3.6 (java)
Installing thread_safe 0.3.6 (java)
Fetching tzinfo 1.2.5
Installing tzinfo 1.2.5
Fetching biz 1.7.0
Installing biz 1.7.0
Using bundler 1.16.1
Bundle complete! 1 Gemfile dependency, 5 gems now installed.
Bundled gems are installed into `/usr/verticles/bundle`
Removing intermediate container 93dce86c9812
 ---> 7b0823bc7062
Step 16/18 : WORKDIR $VERTICLE_HOME
Removing intermediate container d4b8d1e54269
 ---> a4107ce92f4d
Step 17/18 : ENTRYPOINT ["sh", "-c"]
 ---> Running in 11ea7da0e520
Removing intermediate container 11ea7da0e520
 ---> 35db3050a133
Step 18/18 : CMD ["exec vertx run $VERTICLE_NAME -cp $VERTICLE_HOME/* -conf config.json $VERTX_ARGS"]
 ---> Running in 4ae4dac2c982
Removing intermediate container 4ae4dac2c982
 ---> 221df32ab9e3
Successfully built 221df32ab9e3
Successfully tagged camundaworkervertx_cam-worker_01:latest
Creating camundaworkervertx_cam-worker_01_1 ... done
Attaching to camundaworkervertx_cam-worker_01_1
cam-worker_01_1  | Starting Primary Verticle
cam-worker_01_1  | Succeeded in deploying verticle
cam-worker_01_1  | Starting fetch-and-lock Verticle
cam-worker_01_1  | Starting httpserver Verticle
cam-worker_01_1  | Starting task-complete Verticle
cam-worker_01_1  | Starting calculate-numbers Verticle

Few issues:

1. Setting of the path: ENV PATH=$PATH:/root/.gem/jruby/2.3.0/bin in the docker file, "feels" wrong.. Better way to do this?

2. Bundler is running as root and we see this warning message appear in console. Assume this is a message/error that should be resolved.

How it currently works:

You add a Gemfile file to your $VERTICLE_HOME, in this case mine looks like:

source 'https://rubygems.org'
 
gem "biz"

Additions that could be added in this model was moved forward:

1. Run Bundle install for multiple gemfiles, so each verticle could have their own gem files for better portability.

[StephenOTT]

updated example for simplicity

Building cam-worker_01
Step 1/18 : FROM vertx/vertx3
 ---> e841617fbb56
Step 2/18 : ENV VERTICLE_HOME /usr/verticles
 ---> Using cache
 ---> 44cf2fa51e16
Step 3/18 : COPY logging.properties ${VERTICLE_HOME}/logging.properties
 ---> Using cache
 ---> afa9955265cd
Step 4/18 : ENV VERTX_JUL_CONFIG $VERTICLE_HOME/logging.properties
 ---> Using cache
 ---> 468580126dfe
Step 5/18 : COPY vertx-stack.json ${VERTX_HOME}/vertx-stack.json
 ---> Using cache
 ---> 9658255bf461
Step 6/18 : RUN vertx resolve && rm -rf ${HOME}/.m2
 ---> Using cache
 ---> 346bb8120aa6
Step 7/18 : COPY config.json ${VERTICLE_HOME}/config.json
 ---> Using cache
 ---> 8081e9c47544
Step 8/18 : COPY cluster.xml ${VERTICLE_HOME}/cluster.xml
 ---> Using cache
 ---> 47f2ca8e5d0a
Step 9/18 : COPY $VERTICLE_NAME $VERTICLE_HOME/$VERTICLE_NAME
 ---> 5723ba685104
Step 10/18 : COPY verticles $VERTICLE_HOME/verticles
 ---> f50bc5bccaa0
Step 11/18 : COPY Gemfile $VERTICLE_HOME/Gemfile
 ---> d88b3a12c65a
Step 12/18 : ENV PATH=$PATH:/root/.gem/jruby/2.3.0/bin
 ---> Running in a4c1f00f4363
Removing intermediate container a4c1f00f4363
 ---> c4927c5c10c2
Step 13/18 : RUN java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S gem install --user-install --no-document bundler
 ---> Running in 95fc2f473661
Successfully installed bundler-1.16.1
1 gem installed
Removing intermediate container 95fc2f473661
 ---> 6cfe30e7f36f
Step 14/18 : RUN java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S gem list -d
 ---> Running in 50256ce3a260
bundler (1.16.1)
    Authors: André Arko, Samuel Giddins, Chris Morris, James Wen, Tim
    Moore, André Medeiros, Jessica Lynn Suttles, Terence Lee, Carl
    Lerche, Yehuda Katz
    Homepage: http://bundler.io
    License: MIT
    Installed at: /root/.gem/jruby/2.3.0

    The best way to manage your application's dependencies

did_you_mean (1.0.1)
    Author: Yuki Nishijima
    Homepage: https://github.com/yuki24/did_you_mean
    License: MIT
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    "Did you mean?" experience in Ruby

jar-dependencies (0.3.10)
    Author: christian meier
    Homepage: https://github.com/mkristian/jar-dependencies
    License: MIT
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    manage jar dependencies for gems

jruby-openssl (0.9.21)
    Platform: java
    Authors: Ola Bini, JRuby contributors
    Homepage: https://github.com/jruby/jruby-openssl
    Licenses: EPL-1.0, GPL-2.0, LGPL-2.1
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    JRuby OpenSSL

jruby-readline (1.1.1)
    Platform: java
    Author: JRuby contributors
    Homepage: https://github.com/jruby/jruby
    Licenses: EPL-1.0, GPL-2.0, LGPL-2.1
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    JRuby Readline

json (1.8.3)
    Platform: java
    Author: Daniel Luz
    Homepage: http://json-jruby.rubyforge.org/
    License: Ruby
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    JSON implementation for JRuby

minitest (5.4.1)
    Author: Ryan Davis
    Homepage: https://github.com/seattlerb/minitest
    License: MIT
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    minitest provides a complete suite of testing facilities supporting
    TDD, BDD, mocking, and benchmarking

net-telnet (0.1.1)
    Author: SHIBATA Hiroshi
    Homepage: https://github.com/ruby/net-telnet
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    Provides telnet client functionality.

power_assert (0.2.3)
    Author: Kazuki Tsujimoto
    Homepage: https://github.com/k-tsj/power_assert
    Licenses: 2-clause BSDL, Ruby's
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    Power Assert for Ruby

psych (2.2.4)
    Platform: java
    Authors: Aaron Patterson, SHIBATA Hiroshi, Charles Oliver Nutter
    Homepage: https://github.com/ruby/psych
    License: MIT
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    Psych is a YAML parser and emitter

rake (10.4.2)
    Authors: Eric Hodel, Jim Weirich
    Homepage: https://github.com/ruby/rake
    License: MIT
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    Rake is a Make-like program implemented in Ruby

rdoc (4.2.0)
    Authors: Eric Hodel, Dave Thomas, Phil Hagelberg, Tony Strauss
    Homepage: http://docs.seattlerb.org/rdoc
    License: Ruby
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    RDoc produces HTML and command-line documentation for Ruby projects

test-unit (3.1.1)
    Authors: Kouhei Sutou, Haruka Yoshihara
    Homepage: http://test-unit.github.io/
    Licenses: Ruby, PSFL
    Installed at (default): uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared

    An xUnit family unit testing framework for Ruby.
Removing intermediate container 50256ce3a260
 ---> 89de443b068e
Step 15/18 : RUN java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S bundle install --path=${VERTICLE_HOME}/bundle --binstubs --gemfile=$VERTICLE_HOME/Gemfile
 ---> Running in ab5ccb1541b4
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
installing your bundle as root will break this application for all non-root
users on this machine.
Fetching gem metadata from https://rubygems.org/...............
Resolving dependencies......
Fetching concurrent-ruby 1.0.5 (java)
Installing concurrent-ruby 1.0.5 (java)
Fetching i18n 0.9.5
Installing i18n 0.9.5
Fetching minitest 5.11.3
Installing minitest 5.11.3
Fetching thread_safe 0.3.6 (java)
Installing thread_safe 0.3.6 (java)
Fetching tzinfo 1.2.5
Installing tzinfo 1.2.5
Fetching activesupport 5.1.5
Installing activesupport 5.1.5
Using bundler 1.16.1
Fetching working_hours 1.1.3
Installing working_hours 1.1.3
Bundle complete! 1 Gemfile dependency, 8 gems now installed.
Bundled gems are installed into `/usr/verticles/bundle`
Removing intermediate container ab5ccb1541b4
 ---> a30deeebefcf
Step 16/18 : WORKDIR $VERTICLE_HOME
Removing intermediate container aafaefe2663d
 ---> 64bd489dc98d
Step 17/18 : ENTRYPOINT ["sh", "-c"]
 ---> Running in c477d0eb4f8f
Removing intermediate container c477d0eb4f8f
 ---> 751b8efffe10
Step 18/18 : CMD ["exec vertx run $VERTICLE_NAME -cp $VERTICLE_HOME/* -conf config.json $VERTX_ARGS"]
 ---> Running in 3fad3374f7dc
Removing intermediate container 3fad3374f7dc
 ---> 230f4e0bbcd2
Successfully built 230f4e0bbcd2
Successfully tagged camundaworkervertx_cam-worker_01:latest
Creating camundaworkervertx_cam-worker_01_1 ... done
Attaching to camundaworkervertx_cam-worker_01_1
cam-worker_01_1  | Starting Primary Verticle
cam-worker_01_1  | Succeeded in deploying verticle
cam-worker_01_1  | Starting calculate-numbers Verticle
cam-worker_01_1  | Starting httpserver Verticle
cam-worker_01_1  | Starting task-complete Verticle
cam-worker_01_1  | Starting fetch-and-lock Verticle
cam-worker_01_1  | 2018-02-23 17:53:15 -0500
cam-worker_01_1  | 2018-02-23 11:00:00 -0500
cam-worker_01_1  | 2018-02-23 09:15:00 -0500

Main Verticle:

...
var rubyOptions = {
  "config": {
    "GEM_PATH": "bundle/jruby/2.3.0/"
  },
  "worker" : true
}
vertx.deployVerticle('verticles/ruby-biz/ruby-biz.rb', rubyOptions)

Ruby File:

require 'working_hours'


# Configure working hours
WorkingHours::Config.working_hours = {
  :tue => {'09:00' => '12:00', '13:00' => '17:00'},
  :wed => {'09:00' => '12:00', '13:00' => '17:00'},
  :thu => {'09:00' => '12:00', '13:00' => '17:00'},
  :fri => {'09:00' => '12:00', '13:00' => '17:05:30'},
  :sat => {'19:00' => '24:00'}
}

# Configure timezone (uses activesupport, defaults to UTC)
WorkingHours::Config.time_zone = 'Eastern Time (US & Canada)'

# Configure holidays
WorkingHours::Config.holidays = [Date.new(2018, 02, 22)]


puts 1.working.day.from_now
puts 2.working.hours.from_now
puts 15.working.minutes.from_now

Gemfile:

source 'https://rubygems.org'
 
gem "working_hours"

where working_hours is: https://github.com/Intrepidd/working_hours. Switched from biz because biz was throwing some error about invalid date. not worth troubleshooting the specific lib.

• add support for deployment tag of bundle install

[StephenOTT]

@vietj if you have some ideas for best approaches for optimizations, that would be great! happy to do a PR.

[StephenOTT]

Note for future testing:
Use the following as basis for seeing how jruby is resolving the executable gem location

RUN java -Djruby.debug.scriptResolution=true -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S bundle install --path=${VERTICLE_HOME}/bundle --binstubs --gemfile=$VERTICLE_HOME/Gemfile

specifically: -Djruby.debug.scriptResolution=true

[StephenOTT]

When testing for where gems are being installed, use the following

# Show gems list so we can confirm bundler is installed. Used for troubleshooting where Bundler gets installed:
 RUN java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S gem list -d

the -d will display the details and therefore print out the install path of the gems. THis is important for debugging to understand where bundler gets installed vs the gems that come with jruby-complete

[StephenOTT]

Updated Dockerfile with cleaner install:

# Notice the difference between VERTX_HOME and VERTICLE_HOME.  
# VERTX_HOME is the location of the vertex executable/command

# Alpine cannot be used because Jruby requires cglib to install bundler
FROM vertx/vertx3
# FROM vertx/vertx3-alpine

ENV VERTICLE_HOME /usr/verticles

COPY logging.properties ${VERTICLE_HOME}/logging.properties
ENV VERTX_JUL_CONFIG $VERTICLE_HOME/logging.properties 

# Setup Stack Manager for Vertx
COPY vertx-stack.json ${VERTX_HOME}/vertx-stack.json
# Download and resolve dependencies listed in stack manager
RUN vertx resolve && rm -rf ${HOME}/.m2 


COPY config.json ${VERTICLE_HOME}/config.json

# Setup the cluster Hazelcast configuration
COPY cluster.xml ${VERTICLE_HOME}/cluster.xml

# Copies the starting verticles to the home directory
COPY $VERTICLE_NAME $VERTICLE_HOME/$VERTICLE_NAME
# Copies the sub-verticles into a verticles directory
COPY verticles $VERTICLE_HOME/verticles

# -- RUBY GEMFILE ISTALL: START --
# Copy Gemfile into VERTICLE_HOME
COPY Gemfile $VERTICLE_HOME/Gemfile

# Set Path for where bundler will get installed
# This is set before bundler install to prevent post-install warning
ENV PATH=$PATH:/root/.gem/jruby/2.3.0/bin

# Install Bundler gem
RUN java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S gem install --user-install --no-document bundler

# Install gems in gemfile located in VERTICLE_HOME
# @TODO Make --path location configurable
RUN BUNDLE_SILENCE_ROOT_WARNING=true java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S bundle install --path=${VERTICLE_HOME}/bundle --gemfile=$VERTICLE_HOME/Gemfile

# Print Gemfile.lock file for logging purposes, so we know what was installed by bundle install
RUN cat $VERTICLE_HOME/Gemfile.lock
# -- RUBY GEMFILE INSTALL: END --

# Starts the starting verticle in the home directory
WORKDIR $VERTICLE_HOME
ENTRYPOINT ["sh", "-c"]
# CMD ["exec vertx run $VERTICLE_NAME -cp $VERTICLE_HOME/* $VERTX_ARGS"]
CMD ["exec vertx run $VERTICLE_NAME -cp $VERTICLE_HOME/* -conf config.json $VERTX_ARGS"]

Console Output:

Building cam-worker_01
Step 1/18 : FROM vertx/vertx3
 ---> e841617fbb56
Step 2/18 : ENV VERTICLE_HOME /usr/verticles
 ---> Using cache
 ---> 44cf2fa51e16
Step 3/18 : COPY logging.properties ${VERTICLE_HOME}/logging.properties
 ---> Using cache
 ---> afa9955265cd
Step 4/18 : ENV VERTX_JUL_CONFIG $VERTICLE_HOME/logging.properties
 ---> Using cache
 ---> 468580126dfe
Step 5/18 : COPY vertx-stack.json ${VERTX_HOME}/vertx-stack.json
 ---> Using cache
 ---> 9658255bf461
Step 6/18 : RUN vertx resolve && rm -rf ${HOME}/.m2
 ---> Using cache
 ---> 346bb8120aa6
Step 7/18 : COPY config.json ${VERTICLE_HOME}/config.json
 ---> Using cache
 ---> 8081e9c47544
Step 8/18 : COPY cluster.xml ${VERTICLE_HOME}/cluster.xml
 ---> Using cache
 ---> 47f2ca8e5d0a
Step 9/18 : COPY $VERTICLE_NAME $VERTICLE_HOME/$VERTICLE_NAME
 ---> 0f828cc56402
Step 10/18 : COPY verticles $VERTICLE_HOME/verticles
 ---> 0ed803699d5f
Step 11/18 : COPY Gemfile $VERTICLE_HOME/Gemfile
 ---> b0546b5aa8a9
Step 12/18 : ENV PATH=$PATH:/root/.gem/jruby/2.3.0/bin
 ---> Running in 1252b03599ca
Removing intermediate container 1252b03599ca
 ---> 558baba051c4
Step 13/18 : RUN java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S gem install --user-install --no-document bundler
 ---> Running in c4fbac1f8b10
Successfully installed bundler-1.16.1
1 gem installed
Removing intermediate container c4fbac1f8b10
 ---> dcbd6c07d3ec
Step 14/18 : RUN BUNDLE_SILENCE_ROOT_WARNING=true java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S bundle install --path=${VERTICLE_HOME}/bundle --gemfile=$VERTICLE_HOME/Gemfile
 ---> Running in cb295810f9af
Fetching gem metadata from https://rubygems.org/...............
Resolving dependencies......
Fetching concurrent-ruby 1.0.5 (java)
Installing concurrent-ruby 1.0.5 (java)
Fetching i18n 0.9.5
Installing i18n 0.9.5
Fetching minitest 5.11.3
Installing minitest 5.11.3
Fetching thread_safe 0.3.6 (java)
Installing thread_safe 0.3.6 (java)
Fetching tzinfo 1.2.5
Installing tzinfo 1.2.5
Fetching activesupport 5.1.5
Installing activesupport 5.1.5
Using bundler 1.16.1
Fetching working_hours 1.1.3
Installing working_hours 1.1.3
Bundle complete! 1 Gemfile dependency, 8 gems now installed.
Bundled gems are installed into `/usr/verticles/bundle`
Removing intermediate container cb295810f9af
 ---> 13a6915a4793
Step 15/18 : RUN cat $VERTICLE_HOME/Gemfile.lock
 ---> Running in a1fc115edd38
GEM
  remote: https://rubygems.org/
  specs:
    activesupport (5.1.5)
      concurrent-ruby (~> 1.0, >= 1.0.2)
      i18n (~> 0.7)
      minitest (~> 5.1)
      tzinfo (~> 1.1)
    concurrent-ruby (1.0.5)
    concurrent-ruby (1.0.5-java)
    i18n (0.9.5)
      concurrent-ruby (~> 1.0)
    minitest (5.11.3)
    thread_safe (0.3.6)
    thread_safe (0.3.6-java)
    tzinfo (1.2.5)
      thread_safe (~> 0.1)
    working_hours (1.1.3)
      activesupport (>= 3.2)
      tzinfo

PLATFORMS
  java
  ruby

DEPENDENCIES
  working_hours

BUNDLED WITH
   1.16.1
Removing intermediate container a1fc115edd38
 ---> 28f0f5a7bd2e
Step 16/18 : WORKDIR $VERTICLE_HOME
Removing intermediate container 70d1e02caabe
 ---> a3899e4a19bd
Step 17/18 : ENTRYPOINT ["sh", "-c"]
 ---> Running in 7557f1f729a8
Removing intermediate container 7557f1f729a8
 ---> 3eaadffcce66
Step 18/18 : CMD ["exec vertx run $VERTICLE_NAME -cp $VERTICLE_HOME/* -conf config.json $VERTX_ARGS"]
 ---> Running in 369d7b3b831b
Removing intermediate container 369d7b3b831b
 ---> fca0b9cc2dc8
Successfully built fca0b9cc2dc8
Successfully tagged camundaworkervertx_cam-worker_01:latest
Creating camundaworkervertx_cam-worker_01_1 ... done
Attaching to camundaworkervertx_cam-worker_01_1
cam-worker_01_1  | Starting Primary Verticle
cam-worker_01_1  | Succeeded in deploying verticle
cam-worker_01_1  | Starting task-complete Verticle
cam-worker_01_1  | Starting httpserver Verticle
cam-worker_01_1  | Starting calculate-numbers Verticle
cam-worker_01_1  | Starting fetch-and-lock Verticle
cam-worker_01_1  | 2018-02-24 12:13:44 -0500
cam-worker_01_1  | 2018-02-23 15:00:00 -0500
cam-worker_01_1  | 2018-02-23 13:15:00 -0500

Notable updates:

RUN BUNDLE_SILENCE_ROOT_WARNING=true java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S bundle install --path=${VERTICLE_HOME}/bundle --gemfile=$VERTICLE_HOME/Gemfile

The env var BUNDLE_SILENCE_ROOT_WARNING=true is submitted with bundler commands to prevent the Root warning.

and

After the run of bundle install, we do a RUN cat $VERTICLE_HOME/Gemfile.lock to print the lockfile contents to the console for logging purposes so we know exactly what was loaded into the env.

[StephenOTT]

Can see a working example at:

.rb file: https://github.com/DigitalState/camunda-worker-vertx/blob/feature-ruby-rubygems-stackmanager/worker/verticles/ruby-biz/ruby-biz.rb

gemfile: https://github.com/DigitalState/camunda-worker-vertx/blob/feature-ruby-rubygems-stackmanager/worker/Gemfile

dockerfile: https://github.com/DigitalState/camunda-worker-vertx/blob/feature-ruby-rubygems-stackmanager/worker/Dockerfile

[vietj]

@StephenOTT do you think it would be possible to move the vertx .rb files in gems instead of jars ?

[vietj]

so still requires the jars (for the java code) but rb code would be in gems instead

[StephenOTT]

Not sure what you mean.

[vietj]

to have vertx for (j)ruby fully contained in gems file instead of jar files. not sure it makes sense though.

[StephenOTT]

I don't know enough about jruby to comment on the viability of that. What's your thinking/reasoning?

At the moment my understanding of the stack is: everything is executed in the jvm. Therefore you need a executable for the jvm. Jruby-complete offers a compartmentalized jruby executable in the jar. The addition gems I am downloading are outside the scope of jruby-complete and more inside of the scope of vertex instance and potentially vertex verticles.

If you are not using the jruby-complete jar, are you looking to install ruby rvm/jruby as a more integrated install in the OS? If yes, what's the value/goal?

[StephenOTT]

Additional cleanup for dockerfile to reduce the amount of intermediate containers:

# Notice the difference between VERTX_HOME and VERTICLE_HOME.  
# VERTX_HOME is the location of the vertex executable/command

# Alpine cannot be used because Jruby requires cglib to install bundler
FROM vertx/vertx3:3.5.1
# FROM vertx/vertx3-alpine

ENV VERTICLE_HOME /usr/verticles

COPY logging.properties ${VERTICLE_HOME}/logging.properties
ENV VERTX_JUL_CONFIG $VERTICLE_HOME/logging.properties 

# Setup Stack Manager for Vertx
COPY vertx-stack.json ${VERTX_HOME}/vertx-stack.json
# Download and resolve dependencies listed in stack manager
RUN vertx resolve && rm -rf ${HOME}/.m2 

# config.json file that will be used in the --conf argument of 'exec vertx run'
COPY config.json ${VERTICLE_HOME}/config.json

# Setup the cluster Hazelcast configuration
COPY cluster.xml ${VERTICLE_HOME}/cluster.xml

# Copies the starting verticles to the home directory
COPY $VERTICLE_NAME $VERTICLE_HOME/$VERTICLE_NAME
# Copies the sub-verticles into a verticles directory
COPY verticles $VERTICLE_HOME/verticles

# -- RUBY GEMFILE INSTALL: START --
# Copy Gemfile into VERTICLE_HOME
COPY Gemfile $VERTICLE_HOME/Gemfile

# Set Path for where bundler will get installed
# This is set before bundler install to prevent post-install warning
ENV PATH=$PATH:/root/.gem/jruby/2.3.0/bin

RUN java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S gem install --user-install --no-document bundler \
 && BUNDLE_SILENCE_ROOT_WARNING=true java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S bundle install --path=${VERTICLE_HOME}/bundle --gemfile=$VERTICLE_HOME/Gemfile \
 && cat $VERTICLE_HOME/Gemfile.lock 
# -- RUBY GEMFILE INSTALL: END --

# Starts the starting verticle in the home directory
WORKDIR $VERTICLE_HOME
ENTRYPOINT ["sh", "-c"]
# CMD ["exec vertx run $VERTICLE_NAME -cp $VERTICLE_HOME/* $VERTX_ARGS"]
CMD ["exec vertx run $VERTICLE_NAME -cp $VERTICLE_HOME/* -conf config.json $VERTX_ARGS"]

specific focus on the following section:

# -- RUBY GEMFILE INSTALL: START --
# Copy Gemfile into VERTICLE_HOME
COPY Gemfile $VERTICLE_HOME/Gemfile

# Set Path for where bundler will get installed
# This is set before bundler install to prevent post-install warning
ENV PATH=$PATH:/root/.gem/jruby/2.3.0/bin

RUN java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S gem install --user-install --no-document bundler \
 && BUNDLE_SILENCE_ROOT_WARNING=true java -jar ${VERTX_HOME}/lib/jruby-complete-9.1.13.0.jar -S bundle install --path=${VERTICLE_HOME}/bundle --gemfile=$VERTICLE_HOME/Gemfile \
 && cat $VERTICLE_HOME/Gemfile.lock 
# -- RUBY GEMFILE INSTALL: END --

[StephenOTT]

@vietj just following up on your preview comment

[StephenOTT]

Added a PR to vertx examples repo vert-x3/vertx-examples#272

[StephenOTT]

On related notes,

two issues i have noticed:

1. On slower machines (like when developing locally) the startup of a jRuby verticle (especially when its the main verticle being started up through the vertx run command, will cause event loop blocking beyond the 200ms limit and throw warnings. Is there a way to run vertx run that ignores the event loop blocking? Async Start as per: http://vertx.io/docs/vertx-core/ruby/#_asynchronous_verticle_start_and_stop does not seem possible when starting the primary verticle with vertx run in the CLI. is there other options?

2. The GEM_PATH example usage in the CLI as shown here: http://vertx.io/docs/vertx-core/ruby/#_verticle_deployment_options does not work. The CLI throws a expecting double quote object error that appears related to jackson parsing (based on some googling of the error)

@vietj are these known issues?

[vietj]

1. yes
2. I don't think so