stepful / cyperful

Interactive system testing UI for capybara

Cyperful

The Capybara visual debugger

An addon for Ruby Capybara system tests that adds the great DX of tools like Cypress.io.

cyperful-demo.mp4

Features

• View the list of steps in a test, with live updates as the test runs
• Pause the test at each step and interact with the page
• View API requests and console logs between steps
• Auto-restart the test when the source code is modified
• Records a video of the test so you can preview the finished test at that point in time (experimental)
• View and run ANY test suite from the UI (coming soon)
• Record browser interactions to save as ruby code (coming later)

Framework support

🚨 Cyperful is in BETA and you will likely encounter issues/bugs! We currently only support: Rails 7.1, Ruby 3.2.1, Minitest/RSpec, capybara, and selenium with Chrome.

Try it out, then please open a Github issue if you'd like to see support for any other specific frameworks/setups.

Installation

First, install the gem (with require=false):

group :test do
  gem "cyperful", require: false
end

Follow the instructions for your test framework:

RSpec

In your rails_helper.rb file, add the following:

CYPERFUL = !!ENV["CYPERFUL"]

require "cyperful/rspec" if CYPERFUL

RSpec.configure do |config|
  # make sure we setup the browser-driver BEFORE Cyperful's setup
  config.prepend_before(:example, type: :system) do
    # Cyperful only supports Selenium + Chrome
    driven_by :selenium, using: :chrome, screen_size: [1400, 1400]
  end

  # ...
end

Minitest

In your application_system_test_case.rb file, add the following:

CYPERFUL = !!ENV["CYPERFUL"]

require "cyperful/minitest" if CYPERFUL

class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
  include Cyperful::Minitest::SystemTestHelper if CYPERFUL

  # Cyperful only supports Selenium + Chrome
  driven_by :selenium, using: :chrome, screen_size: [1400, 1400]

  # ...
end

Usage

Run a test with CYPERFUL env var, e.g.:

CYPERFUL=1 rails test test/system/my_test.rb:123

🚨 IMPORTANT NOTE: Cyperful currently works best when you run a single test at a time i.e. your test file contains only 1 test OR you specify a single test with <filename>:<line_number>. A better DX for running multiple sequential tests is coming soon.

Config

Config options:

• reload_test_files (default: true) - Reset the test when the test's source code is modified.
• auto_run_on_reload (default: true) - Automatically start the test when the source code is modified. Only applies if reload_test_files is true.
• history_recording (default: true) - Record a video of the test while it is running for debugging.

You can set these options on the Cyperful.config object after requiring the gem, e.g.:

Cyperful.config.history_recording = false

Development

# in a terminal, run the frontend dev server.
# this will watch for changes and rebuild the frontend
cd cyperful
pnpm run dev

# in another terminal, run any test.
# prepend `CYPERFUL_DEV=1` to tell cyperful to look at the
# dev server instead of the prebuilt frontend assets.
cd my_test_app
CYPERFUL_DEV=1 CYPERFUL=1 rails test test/system/my_test.rb

cyperful's Issues

add Rspec compatibility

Context: Right now we only support MiniTest, because that's what I use for my projects :)

Tasks:

• split into separate imports: cyperful/minitest & cyperful/rspec
• inject into Rspec setup/teardown flow (easy)
• parse rspec test files (hard)
  • describe, it, context, etc.
• add tracing on all Rspec commands (hard)
  • e.g. expect(page).to have_content("foo")
  • This was pretty easy on MiniTest because we only needed to look at methods starting with assert_, and all tests were direct methods of a ActiveSupport::TestClass class

UPDATE: I am actively working on this, a PoC should be ready soonish.

Rails not initialized at Bundle require-time

When cyperful is required by my Gemfile (e.g. I don't have require: false set), it tries to hook into Rails before Rails itself hasn't initialized yet. Looking briefly at the code, it seems like this middleware may need to be added in a railtie initializer

$ CYPERFUL=1 bin/rails test test/system/program_admin_test.rb:60
/Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/cyperful-0.1.9/lib/cyperful/framework_injections.rb:104:in `<main>': undefined method `config' for nil (NoMethodError)

  Rails.application.config.content_security_policy do |policy|
                   ^^^^^^^
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundled_gems.rb:74:in `require'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundled_gems.rb:74:in `block (2 levels) in replace_require'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bootsnap-1.18.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/zeitwerk-2.6.13/lib/zeitwerk/kernel.rb:34:in `require'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/cyperful-0.1.9/lib/cyperful.rb:68:in `<main>'
        from <internal:/Users/justin/.rbenv/versions/3.3.0/lib/ruby/site_ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
        from <internal:/Users/justin/.rbenv/versions/3.3.0/lib/ruby/site_ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundled_gems.rb:74:in `block (2 levels) in replace_require'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/zeitwerk-2.6.13/lib/zeitwerk/kernel.rb:34:in `require'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/runtime.rb:60:in `block (2 levels) in require'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/runtime.rb:55:in `each'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/runtime.rb:55:in `block in require'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/runtime.rb:44:in `each'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler/runtime.rb:44:in `require'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/lib/bundler.rb:187:in `require'
        from /Users/justin/code/searls/grog/config/application.rb:19:in `<main>'
        from /Users/justin/code/searls/grog/config/environment.rb:2:in `require_relative'
        from /Users/justin/code/searls/grog/config/environment.rb:2:in `<main>'
        from /Users/justin/code/searls/grog/test/test_helper.rb:2:in `require_relative'
        from /Users/justin/code/searls/grog/test/test_helper.rb:2:in `<main>'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundled_gems.rb:74:in `require'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundled_gems.rb:74:in `block (2 levels) in replace_require'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bootsnap-1.18.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
        from /Users/justin/code/searls/grog/test/application_system_test_case.rb:1:in `<main>'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundled_gems.rb:74:in `require'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundled_gems.rb:74:in `block (2 levels) in replace_require'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bootsnap-1.18.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
        from /Users/justin/code/searls/grog/test/system/program_admin_test.rb:1:in `<main>'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundled_gems.rb:74:in `require'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundled_gems.rb:74:in `block (2 levels) in replace_require'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bootsnap-1.18.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/bundler/gems/rails-02f6c2913b75/railties/lib/rails/test_unit/runner.rb:61:in `block in load_tests'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/bundler/gems/rails-02f6c2913b75/railties/lib/rails/test_unit/runner.rb:60:in `each'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/bundler/gems/rails-02f6c2913b75/railties/lib/rails/test_unit/runner.rb:60:in `load_tests'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/bundler/gems/rails-02f6c2913b75/railties/lib/rails/test_unit/runner.rb:52:in `run'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/bundler/gems/rails-02f6c2913b75/railties/lib/rails/commands/test/test_command.rb:33:in `perform'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/thor-1.3.1/lib/thor/command.rb:28:in `run'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/thor-1.3.1/lib/thor/invocation.rb:127:in `invoke_command'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/bundler/gems/rails-02f6c2913b75/railties/lib/rails/command/base.rb:178:in `invoke_command'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/thor-1.3.1/lib/thor.rb:527:in `dispatch'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/bundler/gems/rails-02f6c2913b75/railties/lib/rails/command/base.rb:73:in `perform'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/bundler/gems/rails-02f6c2913b75/railties/lib/rails/command.rb:71:in `block in invoke'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/bundler/gems/rails-02f6c2913b75/railties/lib/rails/command.rb:149:in `with_argv'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/bundler/gems/rails-02f6c2913b75/railties/lib/rails/command.rb:69:in `invoke'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/bundler/gems/rails-02f6c2913b75/railties/lib/rails/commands.rb:18:in `<main>'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundled_gems.rb:74:in `require'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundled_gems.rb:74:in `block (2 levels) in replace_require'
        from /Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bootsnap-1.18.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
        from bin/rails:4:in `<main>'

In the meantime, it'd probably be best to suggest users add Cyperful to their Gemfile with require: false:

group :test do
  # …
  gem "cyperful", require: false
end

Unable to find css "iframe#scenario-frame"

Ran into this issue running a single test (as suggested in the README).

What I experienced:

• Roughly 30 second startup time (usually a few seconds)
• Test opened to the first visited page
• Then the Cyperful runner launched
• Then nothing happened
• Clicking the play button resulted in the first visited page being shown again

Trace:

# Running:

Capybara starting Puma...
* Version 6.4.2, codename: The Eagle of Durango
* Min threads: 0, max threads: 4
* Listening on http://127.0.0.1:49232
* Listening on http://[::1]:49232
Setting up Cyperful for: ProgramAdminTest#test_make_a_program
Cyperful server started: http://localhost:3004
Found 42 steps:
  1.  visit: 6:4
  2.  click_on: 8:4
  3.  click_on: 9:4
  4.  click_on: 10:4
  5.  fill_in: 11:4
  6.  fill_in: 12:4
  7.  attach_file: 13:4
  8.  fill_in: 14:4
  9.  fill_in: 15:4
 10.  fill_in: 16:4
 11.  check: 17:4
 12.  click_on: 18:4
 13.  assert_text: 19:4
 14.  click_on: 21:4
 15.  click_on: 22:4
 16.  click_on: 23:4
 17.  fill_in: 24:4
 18.  fill_in: 25:4
 19.  fill_in: 26:4
 20.  check: 27:4
 21.  check: 28:4
 22.  attach_file: 30:4
 23.  attach_file: 31:4
 24.  select: 32:4
 25.  click_on: 33:4
 26.  assert_text: 34:4
 27.  click_on: 36:4
 28.  click_on: 37:4
 29.  click: 38:4
 30.  fill_in: 39:4
 31.  attach_file: 40:4
 32.  attach_file: 41:4
 33.  check: 42:4
 34.  check: 43:4
 35.  check: 44:4
 36.  select: 45:4
 37.  fill_in: 46:4
 38.  click: 48:4
 39.  select: 52:6
 40.  click: 55:4
 41.  fill_in: 56:4
 42.  first: 57:11

Driving iframe...
Driving iframe...


Test failed with error:
Unable to find css "iframe#scenario-frame"
/Users/justin/code/searls/grog/test/system/program_admin_test.rb:5:in `test_make_a_program'
/Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/minitest-5.21.2/lib/minitest/test.rb:94:in `block (3 levels) in run'
/Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/minitest-5.21.2/lib/minitest/test.rb:191:in `capture_exceptions'
/Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/minitest-5.21.2/lib/minitest/test.rb:89:in `block (2 levels) in run'
/Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/minitest-5.21.2/lib/minitest.rb:303:in `time_it'
/Users/justin/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/minitest-5.21.2/lib/minitest/test.rb:88:in `block in run'
Cyperful teardown complete. Waiting for command...