jstorimer / spin

Spin speeds up your Rails testing workflow by preloading your Rails environment.

Spin

Spin speeds up your Rails testing workflow.

By preloading your Rails environment in one process and then using fork(2) for each test run you don't load the same code over and over and over... Spin works with an autotest(ish) workflow.

Installation

Spin is available as a rubygem.

gem i spin

Spin is a tool for Rails 3 apps. It is compatible with the following testing libraries:

• any version of test/unit or MiniTest
• RSpec 2.x

Usage

There are two components to Spin, a server and client. The server has to be running for anything interesting to happen. You can start the Spin server from your Rails.root with the following command:

spin serve

As soon as the server is running it will be ready to accept from clients. You can use the following command to specify a file for the server to load:

spin push test/unit/product_test.rb

Or push multiple files to be loaded at once:

spin push test/unit/product_test.rb test/unit/shop_test.rb test/unit/cart_test.rb

Or, when using RSpec, run the whole suite:

spin push spec

Running a single RSpec example by adding a line number is also possible, e.g:

spin push spec/models/user_spec.rb:14

If you experience issues with test_helper.rb not being available you may need to add your test directory to the load path using the -I option:

spin serve -Itest

Send a SIGQUIT to spin serve (Ctrl+\) if you want to re-run the last files that were ran via spin push [files].

With Kicker

As mentioned, this tool works best with an autotest(ish) workflow. I haven't actually used with with autotest itself, but it works great with kicker. Here's the suggested workflow for a Rails app:

1. Start up the spin server

   spin serve

2. Start up kicker using the custom binary option (and any other options you want)

   kicker -r rails -b 'spin push'

3. Faster testing workflow!

Motivation

A few months back I did an experiment. I opened up the source code to my local copy of the ActiveRecord gem. I added a line at the top of active_record/base that incremented a counter in Redis each time it was evaluated. After about a week that counter was well above 2000!

How did I load the ActiveRecord gem over 2000 times in one week? Autotest. I was using it all day while developing. The Rails version that the app was tracking doesn't change very often, yet I had to load the same code over and over again.

Given that there's no way to compile Ruby code into a faster representation I immediately thought of fork(2). I just need a process to load up Rails and wait around until I need it. When I want to run the tests I just fork(2) that idle process and run the test. Then I only have to load Rails once at the start of my workflow, fork(2) takes care of sharing the code with each child process.

I threw together the first version of this project in about 20 minutes and noticed an immediate difference in the speed of my testing workflow. Did I mention that I work on a big app? It takes about 10 seconds(!) to load Rails and all of the gem dependencies. With a bit more hacking I was able to get the idle process to load both Rails and my application dependencies, so each test run just initializes the application and loads the files needed for the test run.

(10 seconds saved per test run) x (2000 test runs per week) = (lots of time saved!)

How is it different from Spork?

There's another project (spork) that aims to solve the same problem, but takes a different approach.

1. It's unobtrusive.

   Your application needs to know about Spork, Spin works entirely outside of your application.

   You'll need to add spork to your Gemfile and introduce your test_helper.rb to spork. Spork needs to know details about your app's loading process.

   Spin is designed so that your app never has to know about it. You can use Spin to run your tests while the rest of your team doesn't even know that Spin exists.

2. It's simple.

   Spin should work out of the box with any Rails app. No custom configuration required.

3. It doesn't do any crazy monkey patching.

Docs

Rocco-annotated source:

• spin
  • spin serve
  • spin push

Hacking

I take pull requests, and it's commit bit, and there are no tests.

Related Projects

If Spin isn't scratching your itch then one of these projects might:

• guard-spin
• Spork
• TestR
• Zeus
• Spring

spin's Issues

Incorrectly identifying RSpec testing suite

I am seeing the following error in my Rails 3.0.9 project:

no such file to load -- rspec/autorun (LoadError)

The issue originates from line 85 in spin.rb:

if defined?(RSpec)

My project is not using RSpec itself, but is using the rspec-expectations gem in conjunction with Test::Unit.

Is there some other way of testing for an RSpec testing suite. Perhaps it should be an explicit option when the spin server is started?

Spin loading, but not running test

I can get the spin server running and attempting to run tests, but it seems to just hang for me:

Terminal 1

jspradlin:~/code/opower_workspace/jonesy [git: spin] $ spin serve -Itest
Preloaded Rails env in 8.29096007347107s...

Loading ["test/unit/relationship_test.rb"]

Terminal 2

jspradlin:~/code/opower_workspace/jonesy [git: spin] $ spin push test/unit/relationship_test.rb
Spinning up test/unit/relationship_test.rb

Platform: Snow Leopard
test-unit (2.4.2)
ruby 1.9.2p290
rails (3.0.9)

missing test_helper

I have installed spin and kicker

I am using rails 2.3.14 and copied environment.rb to application.rb

Spin and kicker started successful, but error occures when trying to run a test:

Executing: ruby -r test/functional/user_controller_test.rb -e ''
./test/functional/user_controller_test.rb:1:in `require': no such file to load -- test_helper (LoadError)
from ./test/functional/user_controller_test.rb:1

Not sure if this is Spin or Kicker issue.

Please check,
Grzegorz

console

spin console

Since we already have the environment preloaded, why couldn't we use that to fork off a rails console?

Reloading dependencies

Any thoughts on how can I make spin reload dependencies that were loaded with "require_dependency"? I have tried a couple things without success:

• set cache_classes = false in the test env
• added before/after fork hooks with:

ActiveSupport::DescendantsTracker.clear
ActiveSupport::Dependencies.clear
ActionDispatch::Reloader.cleanup!
ActionDispatch::Reloader.prepare!

Using spin to test a Rails Engine

Best practice for testing Rails Engines seems to be placing a dummy app inside the test directory. I would love to use Spin to speed up my Engine tests, but the path to application.rb is hard-coded in the serve method. A CLI variable to override that value should allow me to use spin in this configuration.

Spin Push with invalid file or directory

When running spin push invalid_name it will pass an empty array to be tested.

any growl support in plans?

RSpec reporting the wrong run time

I'm trying Spin for the first time. Using RSpec, the run time report seems to be showing the elapsed time since the pre-forked Rails process was started.

Has anyone seen this? Is it a bug in Spin, in RSpec or could it be something in this Rails app? (I get the same result with spin serve --rspec as well as through Guard)

✗ guard
Guard::Spin Initialized
Guard is now watching at '/Users/ashleymoran/Documents/PatchSpace/Development/bc'
Bundle already up-to-date
Starting Spin
LiveReload 1.6 is waiting for a browser to connect.
Preloaded Rails env in 9.81913s...
Spinning up spec/models/note_spec.rb

Loading ["spec/models/note_spec.rb"]
No examples matched {:wip=>true}. Running all.
No examples found.
  0 examples:  100% |==========================================| Time: 00:00:00

Finished in 14.07 seconds
0 examples, 0 failures
Spinning up spec/models/note_spec.rb

Loading ["spec/models/note_spec.rb"]
No examples matched {:wip=>true}. Running all.
No examples found.
  0 examples:  100% |==========================================| Time: 00:00:00

Finished in 39.17 seconds
0 examples, 0 failures
Spinning up spec/models/note_spec.rb

Loading ["spec/models/note_spec.rb"]
No examples matched {:wip=>true}. Running all.
No examples found.
  0 examples:  100% |==========================================| Time: 00:00:00

Finished in 196.8 seconds
0 examples, 0 failures

No way to run whole test-unit suite

With rspec you can do spin push spec, but there is no (documented) equivalent for test-unit.

license

What is the license of spin?

Having explicit license info somewhere in repository will help somebody making packages for distributions such as Gentoo Portage system or Debian apt packaging system.

RSpec code is always run

I always see 0 examples, 0 failures in the output even when I'm only running test-unit tests. Even --test-unit doesn't stop this:

$ spin serve --push-results -t --test-unit -Itest:lib

followed by:

$ spin push test/unit/person_test.rb
Spinning up test/unit/person_test.rb tty?

Loading ["test/unit/person_test.rb"]
Rack::File headers parameter replaces cache_control after Rack 1.5.
No examples found.


Finished in 0.00009 seconds
0 examples, 0 failures
Loaded suite /home/adam/.rvm/gems/ruby-1.9.3-p392@eventbook/bin/spin
Started
..................

Finished in 0.912380817 seconds.

18 tests, 77 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
0% passed

19.73 tests/s, 84.39 assertions/s
Coverage report generated for RSpec, Unit Tests to /home/adam/music/RotC/eventbook/coverage. 393 / 780 LOC (50.38%) covered.

and it even lets me run the whole RSpec suite despite --test-unit being used:

$ spin push spec                    
Spinning up spec tty?

Loading ["spec"]
Rack::File headers parameter replaces cache_control after Rack 1.5.
.....

Finished in 0.12828 seconds
5 examples, 0 failures
Coverage report generated for RSpec, Unit Tests to /home/adam/music/RotC/eventbook/coverage. 393 / 780 LOC (50.38%) covered.

spin push spec doesn't work

When running the full test suite with rspec, I obtain an error:

$ spin push spec

Loading ["spec"]
/home/aymeric/.rvm/gems/ruby-1.9.3-p194/gems/activesupport-3.2.13/lib/active_support/dependencies.rb:251:in `require': cannot load such file -- /home/aymeric/dev/pm-platform/modules/wisdom/spec (LoadError)
    from /home/aymeric/.rvm/gems/ruby-1.9.3-p194/gems/activesupport-3.2.13/lib/active_support/dependencies.rb:251:in `block in require'
    from /home/aymeric/.rvm/gems/ruby-1.9.3-p194/gems/activesupport-3.2.13/lib/active_support/dependencies.rb:236:in `load_dependency'
    from /home/aymeric/.rvm/gems/ruby-1.9.3-p194/gems/activesupport-3.2.13/lib/active_support/dependencies.rb:251:in `require'
    from /home/aymeric/.rvm/gems/ruby-1.9.3-p194/gems/spin-0.6.0/lib/spin.rb:278:in `block (2 levels) in fork_and_run'

I have the same error when I point out a line to test, like

$ spin push spec/lib/worker/manager_spec.rb:15

Loading ["spec/lib/worker/manager_spec.rb:15"]
/home/aymeric/.rvm/gems/ruby-1.9.3-p194/gems/activesupport-3.2.13/lib/active_support/dependencies.rb:251:in `require': cannot load such file -- /home/aymeric/dev/pm-platform/modules/wisdom/spec/lib/worker/manager_spec.rb:15 (LoadError)
    from /home/aymeric/.rvm/gems/ruby-1.9.3-p194/gems/activesupport-3.2.13/lib/active_support/dependencies.rb:251:in `block in require'

By specifying a precise pattern, it works

$ spin push spec/models/*.rb

Any idea ?

Does it work with RSpec?

I have an error but before sharing it just wanted to be sure that RSpec is compatible with spin

[Feature] Collect 'time' to run the test

It would be nice to collect the time it takes to run specs through the 'push' command. However, the collection should be done after the tests have finished running.

It might sound silly and I know any CI server would accomplish the same, but I am throwing it out there for some community bashing.

README: Mention Guard-Spin

Mention Guard-Spin as a possible replacement for Kicker because it uses Guard that supports OS X, Linux, Windows, and Polls for other operating systems.

Stop Spin Command

I've made changes for the https://github.com/vizjerai/guard-spin gem but there isn't a good way to stop Spin.

Currently the gem stops the UnixSocket directly and Spin pukes when it tries to run the loop and the socket is no longer there.

I would rather have a simple command to stop Spin instead of copying internal methods that could change in the future.

Running single tests cases

spin push test/unit/product_test.rb -n /testing_price/

Allow multiple files to be pushed at once

Currently the client will support this but the server won't

[Feature] Pick config from .spin file

Just like rspec, a user should be able to define serve command settings in a .spin file. For example:

--push-results
-t

The code to accomplish this would be trivial and I did add it to my fork. Probably, be part of the next release.

Edit: Update the title

slow tests

I try use spin with rails 3.2.8 but after spin serve and pushing specs with spin push spec/some/spec.rb simple test pass about 20-30 seconds (when just rspec spec/some/spec.rb it takes about 44 seconds to load). This spec take only 8 sec as rspec says to me.

I see when I push spec to spin it still load some environment from spec_helper. It normal or I have some wrong configuration in rspec?

Specifying the stream to output the test results

I'm wondering out loud if there would be a way to get the test ouput to appear in the output stream of the spin push command as opposed to the output of the spin serve command.

I'm happy to send in a pull request but I need a bit of help understand how the output streams work. Is this even possible?

Exception running with no .spin.rb

$ spin serve
/Users/luca/.rvm/gems/ruby-1.9.3-p194/gems/spin-0.5.0/bin/spin:294:in `load': cannot load such file -- .spin.rb (LoadError)
    from /Users/luca/.rvm/gems/ruby-1.9.3-p194/gems/spin-0.5.0/bin/spin:294:in `parse_hook_file'
    from /Users/luca/.rvm/gems/ruby-1.9.3-p194/gems/spin-0.5.0/bin/spin:63:in `serve'
    from /Users/luca/.rvm/gems/ruby-1.9.3-p194/gems/spin-0.5.0/bin/spin:351:in `<top (required)>'
    from /Users/luca/.rvm/gems/ruby-1.9.3-p194/bin/spin:23:in `load'
    from /Users/luca/.rvm/gems/ruby-1.9.3-p194/bin/spin:23:in `<main>'

Just creating an empty file seems to solve it:

$ touch .spin.rb
$ spin serve
Preloaded Rails env in 7.618661s...

Error raised when non-existent files are passed

When non-existent files are passed files_to_load get populated with nil's. It seems accounted for but the corresponding line of code doesn't work due to either past refactoring or typo.

https://github.com/jstorimer/spin/blob/master/bin/spin#L233

end.compact.uniq

should be

end

files_to_load = files_to_load.compact.uniq

or

end

files_to_load.compact!
files_to_load.uniq!

compact! returns nil when there was nothing to compact, so compact!.uniq! chain doesn't work.

Using --push-results and pry breakpoints doesn't show any output

I've recently discovered the --push-results option, which is great.` However, pry breakpoints don't work when using this option:

before_session hook failed: Errno::ENOTTY: Inappropriate ioctl for device
~/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/pry-0.9.12.2/lib/pry/terminal.rb:45:in `winsize'
(see _pry_.hooks.errors to debug)

And then the console doesn't give any output after typing ruby methods there.

Breakpoints work perfectly fine when running the server without the --push-results option.

Support for running a single test within files

I usually run a single example or example group with rspec by passing the line-number, if we try to do that with spin the file does not get pushed because it checks to see if user.rb:12 exists.

rspec spec/models/user.rb:12

Allow specifying RSpec command line options

Spin will ignore any arguments passed to spin serve that are not paths to files:

spin/bin/spin

Line 191 in 72fbfc6

f = files_to_load.select { |f| File.exist?(f.split(':')[0].to_s) }.uniq.join(SEPARATOR)

This is unfortunate, as it prevents passing RSpec options to spin serve. I propose spin support a way to send in ARGV options that get set as the ARGV in the forked process.

Maybe we could use this style:

spin push -- --color --profile --tag type:model

Thoughts?

Add a -I option for appending to the load path

Just like Ruby. Folks are having issues with test_helper not being on the load path.

Conflicts with minitest

My test/minitest_helper.rb:

ENV["RAILS_ENV"] = "test"
require File.expand_path('../../config/environment', __FILE__)

require "minitest/autorun"
require "minitest/rails"

# Add `gem "minitest-rails-capybara"` to the test group of your Gemfile
# and uncomment the following if you want Capybara feature tests
# require "minitest/rails/capybara"

# Uncomment if you want awesome colorful output
require "minitest/pride"

class ActiveSupport::TestCase
  # Setup all fixtures in test/fixtures/*.(yml|csv) for all tests in alphabetical order.
  # fixtures :all

  # Add more helper methods to be used by all tests here...
end

The test (test/models/user_test.rb)

# encoding: utf-8
require 'minitest_helper'

describe User do
...

Spin output:

Loading ["test/models/user_test.rb"]
/Users/kir/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/activesupport-3.2.11/lib/active_support/dependencies.rb:251:in `require': cannot load such file -- minitest_helper (LoadError)

I use Rails 3.2.11 and spin 0.6

Errno::ENOENT when `spin push` while `spin serve` hasn't started yet

After update to the latest version of spin, when I forgot to serve up spin, I get an error spin/bin/spin:249:in initialize: No such file or directory ... (Errno::ENOENT) instead of the message: Connection was refused. Have you started up spine serve yet?

I've fixed this issue, but can't pull request it, because I already have a open pull request and github is suggesting me to put commits together.

Is it worth considering supporting rails 5 with Spin?

Spin on JRuby?

Is it possible to implement support for JRuby in Spin?

Here's what happens now:

ubuntu% spin serve
Preloaded Rails env in 8.853999853134155s...
The signal QUIT is in use by the JVM and will not work correctly on this platform

on another terminal:

spin push spec/

on serve terminal:

NotImplementedError: fork is not available on this platform
          fork at org/jruby/RubyKernel.java:1782
  fork_and_run at ~/.rvm/gems/jruby-1.6.7.2/gems/spin-0.4.6/bin/spin:172
         serve at ~/.rvm/gems/jruby-1.6.7.2/gems/spin-0.4.6/bin/spin:144
          loop at org/jruby/RubyKernel.java:1410
         serve at ~/.rvm/gems/jruby-1.6.7.2/gems/spin-0.4.6/bin/spin:116
        (root) at ~/.rvm/gems/jruby-1.6.7.2/gems/spin-0.4.6/bin/spin:301
          load at org/jruby/RubyKernel.java:1068
               at ~/.rvm/gems/jruby-1.6.7.2/gems/spin-0.4.6/bin/spin:19
          eval at org/jruby/RubyKernel.java:1088
        (root) at ~/.rvm/gems/jruby-1.6.7.2/bin/ruby_noexec_wrapper:14

Batched test runs

1. Start up the spin server
2. Push up a test file
3. In the meantime, push up three more test files
4. Rather than doing three separate test runs the server should batch the three test cases to run at the same time

undefined local variable or method `conn'

When using --push-results I get an error

$ spin serve -Itest --test-unit --push-results
Preloaded Rails env in 2.5115768909454346s...
Pushing test results back to push processes
/Users/ben/.rvm/gems/ruby-1.9.2-p290@reader2/gems/spin-0.4.1/bin/spin:153:in `block in fork_and_run': undefined local variable or method `conn' for main:Object (NameError)
        from /Users/ben/.rvm/gems/ruby-1.9.2-p290@reader2/gems/spin-0.4.1/bin/spin:148:in `fork'
        from /Users/ben/.rvm/gems/ruby-1.9.2-p290@reader2/gems/spin-0.4.1/bin/spin:148:in `fork_and_run'
        from /Users/ben/.rvm/gems/ruby-1.9.2-p290@reader2/gems/spin-0.4.1/bin/spin:122:in `block in serve'
        from /Users/ben/.rvm/gems/ruby-1.9.2-p290@reader2/gems/spin-0.4.1/bin/spin:98:in `loop'
        from /Users/ben/.rvm/gems/ruby-1.9.2-p290@reader2/gems/spin-0.4.1/bin/spin:98:in `serve'
        from /Users/ben/.rvm/gems/ruby-1.9.2-p290@reader2/gems/spin-0.4.1/bin/spin:245:in `<top (required)>'
        from /Users/ben/.rvm/gems/ruby-1.9.2-p290@reader2/bin/spin:19:in `load'
        from /Users/ben/.rvm/gems/ruby-1.9.2-p290@reader2/bin/spin:19:in `<main>

empty -e argument from kicker not working

kicker sends a -e '' that spin does not like.

has anyone come up with a solution between kicker and spin?

Adding 'safe' frameworks to the preloader

So now that I've slowed down everyone's test cycles by removing rspec/rails from the preloader (#60), it would probably be a good idea to try to figure out exactly what can be preloaded safely.

Obviously the safest option is not to preload anything - that way you're guaranteed to have the same load ordering as just running via rspec. But on my SSD'd MacPro, I see roughly 1 second wasted every test run due to loading ActiveRecord::Base, ActionController::Base and all their dependencies.

At least for my app, requiring active_record/base & action_controller/base seems safe, and recovers a decent chunk of the time lost by not loading rspec/rails. But to be honest, the rails initialization process and dependency ordering makes my head hurt - I couldn't guarantee it's not going to break stuff, particularly in the face of plugins doing crazy shit. Also, maybe not all apps want to load ActiveRecord?

We could just hand the responsibility off to the user : if they'd like to try to speed things up, they can just add something like the following to .spin.rb :

Spin.hook(:after_preload) do
  require 'action_controller/base'
end

but that seems like a cop-out.

One final option : make rspec/rails safe to load at any time. It seems like maybe rspec could delay some of its loading until the corresponding Rails framework loads - eg in example.rb :

ActiveSupport.on_load(:action_controller) do
  require 'rspec/rails/example/controller_example_group'
end
ActiveSupport.on_load(:action_view) do
  require 'rspec/rails/example/view_example_group'
end

etc. I suspect that leads to a rabbit hole of pain, though.

Any thoughts? Anyone know a particular rails core member who might be able to offer advice?

Namespaced constants issue

I using spin with rspec and a rails 3 application that contains namespaced controllers. Controllers that are not namespace run fine, but for namespaced ones I'm getting a load_missing_constant error.

[Spin] Loading ["spec/controllers/api/users_controller_spec.rb:1"] 
~/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/activesupport-3.2.14/lib/
active_support/dependencies.rb:503:in `load_missing_constant': Expected ~/app/controllers/api/users_controller.rb to define Api::UsersController (LoadError)

Any ideas?

Changelog it outdated

Any chance you can update it?
Thanks a lot

View specs failing with spin (rails 3.2.1)

I have a really weird spin behavior. When running a view spec with rspec spec/views/edit.html.erb_spec.rb all examples pass. But when starting a spin server and pushing the same spec to spin, all examples fail.

I really do not know where to start investigating, but it would be great if someone could give me a hint.

Rails 2 support

cc @mathieul

continued from #5