stevenhaddox / rvm_fw

RVM::FW - Exposing hidden Rubies for firewalled RVMs

RVM::FW Edition

RVM::FW works really well to make it possible to use RVM (or rbenv even) inside a restrictive LAN or Firewall just like you do at home.

Setup RVM::FW

1. Clone RVM::FW:

    $ git clone git://github.com/stevenhaddox/rvm_fw.git
    $ bundle install
    # For Ruby < 1.9.3 RVM::FW runs in production mode only:
    $ bundle install --without development test
   

2. Create & modify config/rubies.yml.

3. Download Rubiez and packages:

    $ bundle exec rake boot:strap
    # Or, to keep pre-downloaded Rubies and just add to them:
    $ KEEP=true bundle exe rake boot:strap
    # Launch the local webrick server:
    $ bundle exec rackup config.ru  # Ensure you have configured your `.env` / ENV params appropriately
   

4. Archive your local setup of RVM::FW and import it into your network.

5. Deploy! RVM::FW is a simple Sinatra application so you can deploy it anywhere you have Ruby or Rack available internally!

How to Use RVM::FW Once It's Deployed

View a demo to see RVM::FW's user views & instructions for how to setup and use RVM internally. Obviously there are no rubies due to file-size / RVM itself existing on the real Internet.

Just visit: http(s)://<your_host>:<port>/db to get a plain-text file that your users need to copy and paste into: ~/.rvm/user/db. This file overrides the defaults built into RVM's ~/.rvm/config/db and will point it to your RVM::FW instance to download it's rubies.

You can also visit: http(s)://<your_host>:<port>/known to get a plain-text file that a user can be put into: ~.rvm/config/known. This will provide a more a accurate list of available rubies when a user runs rvm list known.

Easter Egg

Bonus: If you're an rbenv fan but still stuck in a restricted environment you can use RVM::FW as a simple way to host a central location to keep your Ruby source code. Accessing your desired version of Ruby for rbenv is as simple as querying http(s)://<your_host>:<port>/public/rubies/<ruby_platform>/<ruby_specific_path>. All the Ruby source paths & packages are easily discoverable within the config/rubies.yml file.

Adding More Rubiez

We're looking into rendering the views for db.erb and known.erb dynamically based upon your custom configuration or the default configuration of config/rubies.yml, but until then you have to manually update and maintain the following files:

• customize config/rubies.yml (defaults to: config/rubies.yml.example)
• update views/db.erb
• update views/known.erb

Any rubies that are found in /public/rubies will be rendered dynamically in the /rubies route on the site. The MD5 sums will also be calculated dynamically to ensure users can compare original MD5s vs possibly modified RVM::FW MD5s.

Enterprise Concerns

I've recently hit an issue when trying to import certain rubies into my corporate environment. The virus scanner fails on several files. I've tried to add these files to a flagged_file list in the config folder and written a bash script that will remove those files and repackage the ruby. All of these files seem Rdoc or test related thus far so the only real con is that the MD5 sum of the file changes, but at least it allows for automated importing of rubies for those who may hit similar issues with either the ClamAV or McAfee Enterprise virus scanners.

To remove these files from your packages be sure to add any files you need to your config/flagged_files.txt file unless you are only using the default rubies.yml file. Then run:

    $ ./scripts/enterprisify.sh

This process will take a while as it has to extract each ruby / package file, scan for matching flagged files, remove them and then re-compress or restore the original file as needed.

Development

CSS Styles

Stylesheets are handled via the gems: bootstrap-sass && compass.

To update bootstrap, update the version of bootstrap-sass and run bundle update

If you modfy the variables (sass/_boostrap_variables.scss) or the main application styles (sass/_rvmfw.scss) then be sure to update the generated CSS files with the appropriate environment command:

$ bundle exec compass compile --output-style compressed -e production
$ bundle exec compass compile --output-style nested -e development

I Want to Help!

Add your idea or feature requests to the issue tracker or Fork RVM::FW on GitHub and send me a pull request!

Special Thanks

Special thanks to those who've contributed and helped me maintain RVM::FW over the years.

rvm_fw's Issues

RVM 1.0.18 not downloading Rubies from RVM::FW properly

Looks like RVM thinks that a 404 page not found is still successful enough to download. We need to identify the proper error to return when a file is not found that RVM is looking for (e.g. file.tar.bz2 is often sought & incorrectly downloads a 404 HTML page before the real file.tar.gz is queried for).

Update db.txt file for RVM 1.0.18

http://github.com/stevenhaddox/rvm_fw/blob/master/views/db.txt needs to utilize the proper syntax and be updated for the new "known" Ruby versions we choose to support from issue #1. The proper syntax of the current version of RVM (1.0.18) is shown here: http://github.com/wayneeseguin/rvm/blob/master/config/db

Add a TL;DR section

The installation instructions can be a little confusing for new users and having a TL;DR section how to install RVM and configure the db/known files would make a nice addition to the top of the index.haml.

Update install script / RVM package location to reflect 1.6.2 install script

RVM::FW install script is very different from the install script RVM now uses, we need to update it and modify it to point to RVM::FW setup or modify where RVM::FW RVM package is to work with install script expectations.

Need to update RVM::FW to work with RVM 1.0.16's known rubies

Utilize: http://github.com/wayneeseguin/rvm/blob/master/config/known to help us identify which versions of Ruby are worth keeping in RVM::FW.

I don't have the largest Github plan and so we need to be specific as to which Rubies will be most used... initial thoughts from me are:

# Rubies
ruby_1.9.1_patch_level=378
ruby_1.9.2_patch_level=0
ruby_1.8.5_patch_level=231
ruby_1.8.6_patch_level=399
ruby_1.8.7_patch_level=299
ruby_1.8.7_patch_level=302
rubygems_1.3.6_url=http://rubyforge.org/frs/download.php/69365
rubygems_1.3.7_url=http://rvm.beginrescueend.com/src/
rbx_1.1.0_patch_level=20100923
ree_1.8.7_patch_level=2010.02
jruby_version=1.5.3
macruby_version=0.7.1
#maglev_version=24407  
#ironruby_version=1.0

These versions need to be placed into: rvm_fw_root/public/rubies/ and older / non-working versions (e.g. IronRuby, plaintext typo files from RailsConf) deleted.

Also, make sure that you maintain any folder structure currently in place. The folder name "prefixes" come from RVM's abbreviation for the Ruby version in order to ensure consistency.

This also requires us properly updating the user custom config file:
http://github.com/stevenhaddox/rvm_fw/blob/master/views/db.txt

Migrate to TOML

Convert config files to TOML for easier readability & better security. Ain't no need for YAML.

Dynamically generate .rvm/config/user file

Currently the user config file (http://github.com/stevenhaddox/rvm_fw/blob/master/views/db.txt) is static. We need to dynamically generate the rvm-fw.local with the server it is being rendered from and then render it on-the-fly.

Add support for common ruby packages that RVM allows installation of

RVM 1.0.16 allows installation of the following packages (see: http://github.com/wayneeseguin/rvm/blob/master/config/db):

# Packages
readline_url=ftp://ftp.gnu.org/gnu/readline
libiconv_url=http://ftp.gnu.org/pub/gnu/libiconv
curl_url=http://curl.haxx.se/download
openssl_url=http://www.openssl.org/source
zlib_url=http://www.zlib.net
autoconf_url=ftp.gnu.org/gnu/autoconf
ncurses_url=http://ftp.gnu.org/pub/gnu/ncurses
pkg-config_url=http://pkgconfig.freedesktop.org/releases
gettext_url=ftp://ftp.gnu.org/pub/gnu/gettext
libxml2_url=ftp://xmlsoft.org/libxml2/
glib_url=http://ftp.gnome.org/pub/gnome/sources/glib/2.23
mono_url=http://ftp.novell.com/pub/mono/sources/mono/
llvm_url=https://llvm.org/svn/llvm-project/llvm/trunk

I think we need to try and support those as much as possible within RVM::FW. I think we can probably pass on mono and llvm, but the rest all seem pretty useful for the majority of enterprise developers.

rubygems version list depends on github.com

the scripts/rubygems has two functions that dependon github.com, in particular rubygems_version_list: https://github.com/wayneeseguin/rvm/blob/master/scripts/rubygems#L80

when installing ruby-1.9.3-pXYZ I get an error because it can't get the checksum for "rubygems-.tgz", notice the rubygems version is missing.

Not sure how to "fix" this since it's a core rvm script, might have to package a "fixed" version of rvm that wouldn't depend on github.com or other external services.

Heroku instance missing stable version tar

When trying to install on a fresh ubuntu precise vm, I get this:

vagrant@precise32:~$ bash < <( curl http://rvm-fw.herokuapp.com/releases/rvm-install-latest )
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   910  100   910    0     0   2891      0 --:--:-- --:--:-- --:--:--  7844
Installing RVM Version: 1.18.14
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2126  100  2126    0     0   5523      0 --:--:-- --:--:-- --:--:-- 12730

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
bash: ./scripts/install: No such file or directory

Investigating a bit, it seems that this line fails because 1.18.14.tar.gz doesn't exist on the server:

vagrant@precise32:~$ curl "http://rvm-fw.herokuapp.com/rubies/packages/rvm/1.18.14.tar.gz"                              
<html>
  <head>
    <title>RVM::FW - Hidden Rubies for RVM</title>
    <meta content='text/html; charset=utf-8' http-equiv='Content-Type'>
    <meta viewport='width=device-width, initial-scale=1.0'>
    <link href='/stylesheets/styles.css' media='screen, projection' rel='stylesheet' type='text/css'>
    <!--[if lt IE 9]>
      <script src='/javascripts/html5shiv.js'></script>
    <![endif]-->
    <link href='images/favicon.png' rel='shortcut icon'>
  </head>
  <body>
    <div id='wrap'>
      <div class='navbar navbar-fixed-top'>
        <div class='navbar-inner'>
          <div class='container'>
            <button class='btn btn-navbar' data_target='.nav-collapse' data_toggle='collapse' type='button'>
              <span class='icon-bar'></span>
              <span class='icon-bar'></span>
              <span class='icon-bar'></span>
            </button>
            <a class='brand' href='/'>RVM::FW</a>
            <div class='nav-collapse collapse'>
              <ul class='nav'>
                <li>
                  <a href='/files'>Rubies</a>
                </li>
                <li>
                  <a href='/db'>DB</a>
                </li>
                <li>
                  <a href='/known'>Known</a>
                </li>
              </ul>
            </div>
          </div>
        </div>
      </div>
      <div class='container'>
        <header>
          <h1 class='title'>RVM::FW</h1>
          <h2 class='description'>Exposing hidden Rubies for firewalled RVMs</h2>
        </header>
        <article>
          <h2>You've dug in the wrong spot - there are no rubies here</h2>
          <p>
            Keep digging though! The treasure has to be somewhere!
          </p>
        </article>
      </div>
      <div id='push'></div>
    </div>
    <footer id='footer'>
      <div class='container'>
        <p>
          <a href='https://github.com/stevenhaddox/rvm_fw'>Fork RVM::FW on GitHub</a>
        </p>
      </div>
    </footer>
    <script src='/javascripts/jquery-1.9.1.min.js'></script>
    <script src='/javascripts/bootstrap-collapse.js'></script>
  </body>
</html>

Should http://rvm-fw.herokuapp.com/releases/stable-version.txt return rvm-1.18.14 instead of 1.18.14?

Fix bugs in menu for IE

IE doesn't display the Bootstrap menu properly. Fix it.

Add system-wide-install script support

https://github.com/wayneeseguin/rvm/raw/master/contrib/install-system-wide

Investigate `rvm pkg install openssl` failure when not online

Investigate rvm pkg install openssl failure when not online. Currently it causes automated installs to fail pretty fiercely due to a very ugly error code (not to mention the huge delay waiting for the timeout before it finally fails).

Finish rake boot:strap

This task is currently in progress (and working minus efficient file downloads) and lives in:

lib/tasks/bootstrap.rb

Just need to add the download code and it's done ;)

Create RVM::FW Installer / Update script

Have it run the steps of installing / updating RVM via RVM::FW and updating the ~/.rvm/user/db and ~/.rvm/user/known files from the most recent RVM::FW versions.

Leaning towards the following URL options:

http(s)://[rvm_fw_url]
  /go
  /install
  /update
  /auto
  /lazy

Other ideas / votes? @mindscratch @adamonduty?

Collapse menu JS errors with smaller widths

One of the images is returning 404 when resizing the browser window to < 760ish piexles. The image exists under public/images. Figure this bug out...

Update RVM::FW to handle Ruby SemVer as of 2.1.0+

Currently rvm in use (1.18.14) expects -p0 to be set on new versions of Ruby. As of 2.1.0 Ruby (MRI) is no longer using patches and has switched to Semantic Versioning (woot!)

We either need to update RVM::FW to a newer RVM that can handle this and modify our known and db views respectively as well as update how all the rvm pkg install stuff works in the newer RVMs with autolibs

OR

We need to create a task that automatically symlinks MRI Rubies >= 2.1.0 to ruby--p0 so older RVM can still download extract the files properly.

Obviously the former is preferred, but I won't have time to look into that for at least a month or so as that's a pretty big task in comparison.

Outdated version of JSON gem fails on bundle install

rvm_fw/Gemfile.lock

Line 54 in 579bb7d

json (1.8.3)

Just needs a bundle update json and it fixes the following issue:

generator.c:861:25: error: ‘rb_cFixnum’ undeclared (first use in this function)
} else if (klass == rb_cFixnum) {```

Needs Tests

RSpec & Cucumber support were added today...

Anyone want to pitch in?

:)

Install script fetches 404 html instead of tarball

Following the install directions, I'm getting this error:

tar: Unrecognized archive format

If I'm not horribly confused, it's because the install script references

http://rvm-fw.herokuapp.com/rubies/packages/rvm/${stable_version}.tar.gz

which fetches stable-version.txt to evaluate into

http://rvm-fw.herokuapp.com/rubies/packages/rvm/1.18.14.tar.gz

which does not resolve to a tarball but rather the message that your ruby is in another castle:

You've dug in the wrong spot - there are no rubies here
Keep digging though! The treasure has to be somewhere!

confusingly, the rubies page links to

http://rvm-fw.herokuapp.com/rubies/packages/rvm/rvm-1.18.14.tar.gz (note the rvm- prefix)

but this url returns the same 404.

Implement the md5 view

The app has a route for getting a file containing the md5 file used by RVM and it should be implemented.

Without it sometimes (maybe most or all of the time) RVM will not trust the download unless you include the "--verify-downloads 1" option, such as rvm install ruby-2.1.1 --verify-downloads 1.

Add an announcement via config file

Create a way to have some kind of announcement / blurb specific for each RVM::FW instance via the config file. Currently it's deployed as a sample application on Heroku but there's no way to inform users there aren't actually any rubies / packages included with it as it's just a dummy demo of the UI / instructions users will see.

Update RVM::FW installer script to use rvm's offline mode?

Thanks to @mpapis, I'm now aware of how to install rvm offline. Rather than trying to deduce how to keep up with RVM's ever more capable and quick moving installer from the past few years I should tweak the RVM::FW installer script to install rvm via offline mode.

It would be nice to investigate offline mode more thoroughly and see how it would work in relation to RVM::FW or if it's just another alternative to RVM::FW...

Modify how MD5 values are accessed

Currently (application.rb) MD5 values are generated manually and then stuck into a hash of rubies by version that contains path-prefix (see issue #1) and MD5 hash value.

We should modify this so that we extract MD5 hashes from: http://github.com/wayneeseguin/rvm/blob/master/config/md5. This will probably mean that we need to copy the md5 file from RVM itself into our project as we cannot assume the server hosting the sinatra app will have a local RVM install. It will also prevent us from having to update MD5 values by hand / including them in application.rb and is worth parsing the md5 file RVM provides for us I think.

These MD5 values are currently displayed in this view: http://github.com/stevenhaddox/rvm_fw/blob/master/views/files.haml

Rake Bootstrap

The rake bootstrap task should by default download and install the newest Rubies only. If provided with an argument (or perhaps as rake bootstrap:all) it should pull down all of the Rubies currently listed by RVM (not firewall).

Add installation of ruby example

Add an example how to install a ruby, this will help new users.

Dynamically Render ruby / package views

The rendering of /db and /known should all happen dynamically based upon the configuration.