drugpl / bbq Goto Github PK

Add support for RSpec.

By default Bbq tests run with use_transactional_fixtures = true. Accessing database within test case will run in a transaction, separate from application server (at least when running Selenium). As a result, changes made by test won't be visible to the application.

Although direct access to the database is discouraged in acceptance tests, sometimes it's very useful and not harmful. Creating admin user account might be a good example.

Possible solutions include

1. setting use_transactional_fixtures = false inside Bbq::TestCase
2. at least documenting this behavior in Readme, to minimize users confusion

Make sure TestClient accepts plain http headers. In case of rack-test driver convert them to CGI style.

Steps to reproduce:

1. cd to bbq (Engine home) directory, and run rake test, so that test-unit and rspec flavored tests are created in bbq/test/dummy/test and bbq/test/dummy/spec.

2. Add this scenario to the spec file, /test/dummy/spec/acceptance/implicit_user_eyes_spec.rb:

scenario 'should see pink within unicorns' do
    user = Bbq::TestUser.new
    user.visit "/ponycorns"
    user.see?("Pink", :within => "#unicorns")
end

3. cd to bbq/test/dummy and run rspec spec/acceptance/implicit_user_eyes_spec.rb

4. See that test passes (correctly):

..

Finished in 0.4461 seconds
2 examples, 0 failures

5. Now change scenario to this:

scenario 'should see pink within unicorns' do
    user = Bbq::TestUser.new
    user.visit "/ponycorns"
    user.see?("FooPink", :within => "#unicorns")
end

6. And again run rspec spec/acceptance/implicit_user_eyes_spec.rb, and see that test passes (incorrectly):

..

Finished in 0.4467 seconds
2 examples, 0 failures

7. Now change scenario to this:

scenario 'should see pink within unicorns' do
    user = Bbq::TestUser.new
    user.visit "/ponycorns"
    user.see?("Pink", :within => "#Foonicorns")
end

8. Again run rspec spec/acceptance/implicit_user_eyes_spec.rb, but this time see that the test finally fails (correctly):

.F

Failures:

  1) implicit user eyes should see pink within unicorns  
 Failure/Error: user.see?("Pink", :within => "#Foonicorns")
 Capybara::ElementNotFound:
   Unable to find css "#Foonicorns"
 # (eval):2:in `find'
 # /Users/afinetooth/Sites/bbq/lib/bbq/test_user/capybara_dsl.rb:9:in `within'
 # /Users/afinetooth/Sites/bbq/lib/bbq/test_user/within.rb:24:in `using_within'
 # (eval):2:in `see?'
 # ./spec/acceptance/implicit_user_eyes_spec.rb:20:in `block (2 levels) in <top (required)>'

Finished in 0.46488 seconds
2 examples, 1 failure

Failed examples:

rspec ./spec/acceptance/implicit_user_eyes_spec.rb:17 # implicit user eyes should see pink within unicorns

9. Verify that step (6) should have failed because "FooPink" was not to be found within the "#unicorns" element. Apparently, the bbq dsl is not properly conveying the :within scope to capybara's has_content? and has_no_content? methods via bbq's see? and not_see? methods.

10. Verify that :within scope is working for test-unit flavored tests, by changing line 57 in /bbq/test/unit/bbq_test_user_test.rb from:

assert @user.see?("Pink", :within => "#unicorns") 

to:

assert @user.see?("FooPink", :within => "#unicorns")

Then run rake test from bbq (Engine home) directory, and see that the test fails as follows:

Started
..............................F.....
Finished in 93.573554 seconds.

  1) Failure:
test_user_eyes_within_scope(BbqTestUserTest) [test/unit/bbq_test_user_test.rb:57]:
Failed assertion, no message given.

36 tests, 117 assertions, 1 failures, 0 errors, 0 skips

NOTE: This is backed up by my experience with my own project, using bbq engine as a gem.

The problem is that every actor (user) object created with bbq for interaction with site has a new capybara session. Probably it is not a problem for rack test-however it is for selenium because browser windows hangs around until all tests are executed.

My current workaround:

class Service::TreeTest < CapybaraTest
  def TestUser(*params, &block)
    user = Bbq::TestUser.new(*params, &block)
    @bbq_test_users ||= []
    @bbq_test_users << user
    return user
  end
  ::Bbq::TestUser.class_eval do
    def mode
      page.mode
    end
  end

  ::Capybara::Selenium::Driver.class_eval do
    def browser?
      !!@browser
    end
  end
  teardown do
    @bbq_test_users ||= []
    @bbq_test_users.select{|u| u.mode == :selenium}.each do |u|
      u.page.driver.quit if u.page.driver.browser? # http://img98.imageshack.us/img98/8158/lawglupcze.png
    end
  end

 test "something" do
   user = TestUser(:driver => :selenium) # instead of  user = user = Bbq::TestUser.new(:driver => :selenium)
 end
end

Just saw this blog. I might try to add some syntax sugar similar to this.

http://blog.railsware.com/2012/01/08/capybara-with-givenwhenthen-steps-in-acceptance-testing/

If anybody else would be interested in this feature I would take a stab at it.

Is it sensible in your opinion to run tests against vendored apps on every major release of Rails (and possibly Sinatra) ?

Ability to run read-only side-scenarios branching from base scenario.

When running generators peek into Rails config to see which test framework is set

Basically two problems:

• when blank test user is created

NameError: uninitialized class variable @@_callbacks in Bbq::TestUser
    /home/sensei/Code/bbq/lib/bbq/test_user.rb:20:in `initialize'

• when devise helper was included in other subclass of test user:

NoMethodError: undefined method `devise_authentication_key=' for #<TestUser:0x00000003cd9210>
    /home/sensei/Code/bbq/lib/bbq/devise.rb:15:in `init'
    /home/sensei/Code/bbq/lib/bbq/test_user.rb:21:in `block in initialize'
    /home/sensei/Code/bbq/lib/bbq/test_user.rb:20:in `each'
    /home/sensei/Code/bbq/lib/bbq/test_user.rb:20:in `initialize'

Things like:

• https://github.com/drugpl/bbq/blob/master/lib/bbq/rails/routes.rb
• https://github.com/drugpl/bbq/blob/master/lib/bbq/railtie.rb

are useful but totally not essential to core. This would help to resolve dependency problems as well and move responsibility to test compatibility with Rails to that gem.

Currently bbq creates new Capybara session for each TestUser in tests. That means running separate browser process when using Selenium. Spawning those processes takes a considerable amount of time on each tests run.

I think we could speed up our tests and also use less memory if we reuse Capybara sessions between tests. A few weeks ago I wrote a sample implementation: https://gist.github.com/1456271

I'm pretty sure it could be improved. (I don't like the idea of passing TestUser class, but I needed Bbq::TestUser#page to call reset!; maybe we need an object encapsulating Capybara session and pass that object to TestUser.new). Also, this implementation needs thorough testing. I recall it didn't work quite well in some cases...

What do you think?

Add generator bbq:install as advertised in README. This should:

• guide user to add bbq/{unit,spec} to {test,spec}_helper.rb
• create {test,spec}/acceptance/
• add rake task to run acceptance tests, i.e rake {test,spec}:acceptance

Optionally it would be nice to generate TestUser stub in {test,spec}/support/test_user.rb inheriting after Bbq::TestUser and having register/login/logout actions if some auth system is present (Devise will be good for start).

Currently TestUser exposes options attribute publicly. There are no clear use cases for me to have this method public. In fact this causes method conflict in TestClient where options is one of http methods actions.

In order to maintain consistency, TestUser can use @options ivar internally instead of accessor or we can rename and protect accessor. This also means deprecating old options accessor.

What do you think?

Object#tap is not available in Ruby 1.8.7 without ActiveSupport, so we should not use it.

This should have been caught by tests for Sinatra app, so make sure those tests run and catch such issues in the future.

Rails generator to provide test case stubs, i.e:

rails g bbq:feature --unit comments

results in file test/acceptance/comments_test.rb:

class CommentsFeature < Bbq::TestCase
  scenario "the truth" do
    assert true
  end
end

Rails' URL helpers are part of user's application. Therefore it should not be accessible from integration test suite, where plain paths should be used to keep routes be tested within integration test suite.

Remove rails' url helpers support

Here's how it might look like: https://gist.github.com/1212860

We could provide default implementation of Bbq::TestClient, similar to Bbq::TestUser.

First of all, thanks for bbq! Very excited to work with it more, but having a couple of issues.

I'll submit them separately. The first is minor, but curious to me.

Getting this single failure when I run tests:

---------------------------------------------------------------
# Running tests:

....................F...............

Finished tests in 50.777182s, 0.7090 tests/s, 2.7965 assertions/s.

  1) Failure:
test_api_client(BbqTestUnitTest) [test/unit/bbq_test_unit_test.rb:159]:
Expected /5 tests, 15 assertions, 0 failures, 0 errors/ to match "Run options: \n\n# Running tests:\n\n.....\n\nFinished tests in 0.087016s, 57.4607 tests/s, 206.8585 assertions/s.\n\n5 tests, 18 assertions, 0 failures, 0 errors, 0 skips\n".

36 tests, 142 assertions, 1 failures, 0 errors, 0 skips
---------------------------------------------------------------

It's curious because I count 15 assertions in the method, test_api_client, in the file, test/unit/bbq_test_unit_test.rb, as well as in the file, test/dummy/test/acceptance/api_test.rb, which is generated by it.

Nevertheless, the correct number of assertions seems to be 18.

Spent some time with it, but not enough to understand why.

Devise helper [https://github.com/drugpl/bbq/blob/master/lib/bbq/devise.rb] is nice but I don't see it used that often and would like to liberate core from this dependency on devise.

Waiting for API proposals.

user.recives_email?(text)
user.clicks_email_link(text)

user.emails.last.has?(text)
user.emails.last.click_link(text)

Quote from capybara list:

A while ago I made two tiny gems inspired on the email_spec gem for
testing email content, clicking links in emails etc in Capybara
https://github.com/suhrawardi/capybara_email
https://github.com/suhrawardi/cache_mailer

Didn't really have the need and the time to expand these further. But
you might check whether it suits your need.
Questions etc are welcome!

jarra [email protected]

Review README and Roundtrip app for example usage. Also the more examples, the better.

hi link in Related examples:
https://github.com/pawelpacana/eventmachine-bbq-example
results in 404

Bbq doesn't work with the latest capybara (2.1).

TestUser::CapybaraDSL module seems not to be required any more - you can include Capybara::DSL instead. The session pool is also possibly broken - it cannot set proper driver.