donv / capybara-screenshot-diff

Track changes in Capybara screenshots

Capybara::Screenshot::Diff

Ever wondered what your project looked like two years ago? To answer that, you start taking screen shots during your tests. Capybara provides the save_screenshot method for this. Very good.

Ever introduced a graphical change unintended? Never want it to happen again? Then this gem is for you! Use this gem to detect changes in your pages by taking screen shots and comparing them to the previous revision.

Installation

Add these lines to your application's Gemfile:

gem 'capybara-screenshot-diff'
gem 'oily_png', platform: :ruby

And then execute:

$ bundle

Or install it yourself as:

$ gem install capybara-screenshot-diff

Requirements

capybara-screenshot-diff supports the currently supported versions of Ruby and Rails:

• Ruby 3.0+ (https://www.ruby-lang.org/en/downloads/branches/)

• Rails 6.1+ (https://guides.rubyonrails.org/maintenance_policy.html)

• [for :vips driver] libvips 8.9 or later, see the libvips install instructions

Usage

Minitest

In your test class, include the Capybara::Screenshot::Diff module:

class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
  include Capybara::Screenshot::Diff
  # ...
end

RSpec

describe 'Permissions admin', type: :feature, js: true do

  include Capybara::Screenshot::Diff

  it 'works with permissions' do
    visit('/')
    screenshot 'home_page'
  end

end

But it's better to include it within your *_helper.rb file so that it can used anywhere in your feature specs.

# spec/feature_helper.rb
require 'capybara/screenshot/diff'

RSpec.configure do |config|
  config.include Capybara::Screenshot::Diff
end

Cucumber

Load Cucumber support by adding the following line (typically to your features/support/env.rb file):

require 'capybara/screenshot/diff/cucumber'

And in the steps you can use:

Then('I should not see any visual difference') do
  screenshot 'homepage'
end

Taking screenshots

Add screenshot '<my_feature>' to your tests. The screenshot will be saved in the doc/screenshots directory.

Change your existing save_screenshot calls to screenshot

test 'my useful feature' do
  visit '/'
  screenshot 'welcome_index'
  click_button 'Useful feature'
  screenshot 'feature_index'
  click_button 'Perform action'
  screenshot 'action_performed'
end

This will produce a sequence of images like this

doc
  screenshots
    action_performed
    feature_index
    welcome_index

To store the screen shot history, add the doc/screenshots directory to your version control system (git, svn, etc).

Screen shots are compared to the previously COMMITTED version of the same screen shot.

Screenshot groups

Commonly it is useful to group screenshots around a feature, and record them as a sequence. To do this, add a screenshot_group call to the start of your test.

test 'my useful feature' do
  screenshot_group 'useful_feature'
  visit '/'
  screenshot 'welcome_index'
  click_button 'Useful feature'
  screenshot 'feature_index'
  click_button 'Perform action'
  screenshot 'action_performed'
end

This will produce a sequence of images like this

doc
  screenshots
    useful_feature
      00-welcome_index
      01-feature_index
      02-action_performed

All files in the screenshot group directory will be deleted when screenshot_group is called.

Screenshot sections

You can introduce another level above the screenshot group called a screenshot_section. The section name is inserted just before the group name in the save path. If called in the setup of the test, all screenshots in that test will get the same prefix:

setup do
  screenshot_section 'my_feature'
end

test 'my subfeature' do
  screenshot_group 'subfeature'
  visit '/feature'
  click_button 'Interesting button'
  screenshot 'subfeature_index'
  click_button 'Perform action'
  screenshot 'action_performed'
end

This will produce a sequence of images like this

doc
  screenshots
    my_feature
      subfeature
        00-subfeature_index
        01-action_performed

Setting screenshot_section and/or screenshot_group for all tests

Setting the screenshot_section and/or screenshot_group for all tests can be done in the super class setup:

class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
  setup do
    screenshot_section class_name.underscore.sub(/(_feature|_system)?_test$/, '')
    screenshot_group name[5..-1]
  end
end

screenshot_section and/or screenshot_group can still be overridden in each test.

Capturing one area instead of the whole page

You can crop images before comparison to be run, by providing region to crop as [left, top, right, bottom] or by css selector like body .tag

test 'the cool' do
  visit '/feature'
  screenshot 'cool_element', crop: '#my_element'
end

Note: When using a retina device screenshots dimensions might be off. If you are using (headless) chrome you can prevent this by setting the force-device-scale-factor argument to 1.

For Rails system specs using selenium you can do so for example by using the following snippet:

driven_by :selenium, using: :chrome_headless do |options|
  options.args << '--force-device-scale-factor=1'
end

Multiple Capybara drivers

Often it is useful to test your app using different browsers. To avoid the screenshots for different Capybara drivers to overwrite each other, set

Capybara::Screenshot.add_driver_path = true

The example above will then save your screenshots like this (for poltergeist and selenium):

doc
  screenshots
    poltergeist
      useful_feature
        00-welcome_index
        01-feature_index
        02-action_performed
    selenium
      useful_feature
        00-welcome_index
        01-feature_index
        02-action_performed

Multiple OSs

If you run your tests on multiple operating systems, you will most likely find the screen shots differ. To avoid the screenshots for different OSs to overwrite each other, set

Capybara::Screenshot.add_os_path = true

The example above will then save your screenshots like this (for Linux and Windows):

doc
  screenshots
    linux
      useful_feature
        00-welcome_index
        01-feature_index
        02-action_performed
    windows
      useful_feature
        00-welcome_index
        01-feature_index
        02-action_performed

If you combine this config with the add_driver_path config, the driver will be put in front of the OS name.

Screen size

You can specify the desired screen size using

Capybara::Screenshot.window_size = [1024, 768]

This will force the screen shots to the given size, and skip taking screen shots unless the desired window size can be achieved.

Disabling screen shots

If you want to skip taking screen shots, set

Capybara::Screenshot.enabled = false

You can of course set this by an environment variable

Capybara::Screenshot.enabled = ENV['TAKE_SCREENSHOTS']

Disabling diff

If you want to skip the assertion for change in the screen shot, set

Capybara::Screenshot::Diff.enabled = false

Using an environment variable

Capybara::Screenshot::Diff.enabled = ENV['COMPARE_SCREENSHOTS']

Tolerate screenshot differences

To allow screenshot differences, but still fail on functional errors, you can set the following option:

Capybara::Screenshot::Diff.fail_on_difference = false

It defaults to true. This can be useful in continuous integration to a generate a screenshot difference report while still reporting functional errors.

Does not tolerate new screenshots

To fail the test if a new screenshot is taken, set the following option:

Capybara::Screenshot::Diff.fail_if_new = true

If fail_if_new is set to true, the test will fail if a new screenshot is taken that does not have a corresponding previous image to compare against. This can be useful in situations where you want to ensure that every screenshot taken by your tests corresponds to an expected state of your application.

Screen shot save path

By default, Capybara::Screenshot::Diff saves screenshots to a doc/screenshots folder, relative to either Rails.root (if you're in Rails), or your current directory otherwise.

If you want to change where screenshots are saved to, then there are two configuration options that that are relevant.

The most likely one you'll want to modify is ...

Capybara::Screenshot.save_path = "other/path"

The save_path option is relative to Capybara::Screenshot.root.

Capybara::Screenshot.root defaults to either Rails.root (if you're in Rails) or your current directory. You can change it to something entirely different if necessary, such as when using an alternative web framework.

Capybara::Screenshot.root = Hanami.root

Screen shot stability

To ensure that animations are finished before saving a screen shot, you can add a stability time limit. If the stability time limit is set, a second screen shot will be taken and compared to the first. This is repeated until two subsequent screen shots are identical.

Capybara::Screenshot.stability_time_limit = 0.1

This can be overridden on a single screenshot:

test 'stability_time_limit' do
  visit '/'
  screenshot 'index', stability_time_limit: 0.5
end

Maximum wait limit

When the stability_time_limit is set, but no stable screenshot can be taken, a timeout occurs. The timeout occurs after Capybara.default_max_wait_time, but can be overridden by an option.

test 'max wait time' do
  visit '/'
  screenshot 'index', wait: 20.seconds
end

Hiding the caret for active input elements

In Chrome the screenshot includes the blinking input cursor. This can make it impossible to get a stable screenshot. To get around this you can set the hide caret option:

Capybara::Screenshot.hide_caret = true

This will make the cursor (caret) transparent (invisible), so the blinking does not delay the screen shot.

Removing focus from the active element

Another way to avoid the cursor blinking is to set the blur_active_element option:

Capybara::Screenshot.blur_active_element = true

This will remove the focus from the active element, removing the blinking cursor.

Allowed color distance

Sometimes you want to allow small differences in the images. For example, Chrome renders the same page slightly differently sometimes. You can set set the color difference threshold for the comparison using the color_distance_limit option to the screenshot method:

test 'color threshold' do
  visit '/'
  screenshot 'index', color_distance_limit: 30
end

The difference is calculated as the eucledian distance. You can also set this globally:

Capybara::Screenshot::Diff.color_distance_limit = 42

Allowed shift distance

Sometimes you want to allow small movements in the images. For example, jquer-tablesorter renders the same table slightly differently sometimes. You can set set the shift distance threshold for the comparison using the shift_distance_limit option to the screenshot method:

test 'color threshold' do
  visit '/'
  screenshot 'index', shift_distance_limit: 2
end

The difference is calculated as maximum distance in either the X or the Y axis. You can also set this globally:

Capybara::Screenshot::Diff.shift_distance_limit = 1

Note: For each increase in shift_distance_limit more pixels are searched for a matching color value, and this will impact performance severely if a match cannot be found.

If shift_distance_limit is nil shift distance is not measured. If shift_distance_limit is set, even to 0, shift distance is measured and reported on image differences.

Allowed difference size

You can set set a threshold for the differing area size for the comparison using the area_size_limit option to the screenshot method:

test 'area threshold' do
  visit '/'
  screenshot 'index', area_size_limit: 17
end

The difference is calculated as width * height. You can also set this globally:

Capybara::Screenshot::Diff.area_size_limit = 42

Skipping an area

Sometimes you have expected change that you want to ignore. You can use the skip_area option with [left, top, right, bottom] or css selector like '#footer' or '.container .skipped_element' to the screenshot method to ignore an area. Be aware that if the selector is not in the page then the library will wait the default wait time for it to appear. Therefore, it is best to only use css selectors for skip_areas you know will be in the page:

test 'unstable area' do
  visit '/'
  screenshot 'index', skip_area: [[17, 6, 27, 16], '.container .skipped_element', '#footer']
end

The arguments are [left, top, right, bottom] for the area you want to ignore. You can also set this globally:

Capybara::Screenshot::Diff.skip_area = [0, 0, 64, 48]

If you need to ignore multiple areas:

screenshot 'index', skip_area: [[0, 0, 64, 48], [17, 6, 27, 16], 'css_selector .element']

Available Image Processing Drivers

There are several image processing supported by this gem. There are several options to setup active driver: :auto, :chunky_png and :vips.

• :auto - will try to load :vips if there is gem ruby-vips, in other cases will load :chunky_png
• :chunky_png and :vips will load correspondent driver

Enable VIPS image processing

Vips driver provides a faster comparison, and could be enabled by adding ruby-vips to Gemfile.

If need to setup explicitly Vips driver, there are several ways to do this:

• Globally: Capybara::Screenshot::Diff.driver = :vips
• Per screenshot option: screenshot 'index', driver: :vips

With enabled VIPS there are new alternatives to process differences, which easier to find and support. For example, shift_distance_limit is very heavy operation. Instead better to use median_filter_window_size.

Tolerance level (vips only)

You can set a “tolerance” anywhere from 0% to 100%. This is the amount of change that's allowable. If the screenshot has changed by more than that amount, it'll flag it as a failure.

This is alternative to "Allowed difference size", only the difference that area calculates including valid pixels. But "tolerance" compares only different pixels.

You can use the tolerance option to the screenshot method to set level:

test 'unstable area' do
  visit '/'
  screenshot 'index', tolerance: 0.3
end

You can also set this globally:

Capybara::Screenshot::Diff.tolerance = 0.3

Median filter size (vips only)

This is an alternative to "Allowed shift distance", but much faster. You can find more about this strategy on Median Filter. Think about this like smoothing of the image, before comparison.

You can use the median_filter_window_size option to the screenshot method to set level:

test 'unstable area' do
  visit '/'
  screenshot 'index', median_filter_window_size: 2
end

Skipping stack frames in the error output

If you would like to override the screenshot method or for some other reason would like to skip stack frames when reporting image differences, you can use the skip_stack_frames option:

test 'test visiting the index' do
  visit root_path
  screenshot :index
end

private

def screenshot(name, **options)
  super(name, skip_stack_frames: 1, **options)
end

Screenshot Format

You can specify the format of the screenshots taken by setting the screenshot_format option. By default, the format is set to "png". However, you can change this to any format supported by your image processing driver. For example, to set the format to "webp", you can do the following:

Capybara::Screenshot.screenshot_format = "webp"

Customize Capybara#screenshot options

Allow to bypass screenshot options to Capybara driver.

# To create full page screenshots for Selenium
Capybara::Screenshot.capybara_screenshot_options[:full_page] = true

screenshot('index', median_filter_window_size: 2, capybara_screenshot_options: {full_page: false})

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake test to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install.

To release a new version, update the version number in lib/capybara/screenshot/diff/version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/donv/capybara-screenshot-diff. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.

License

The gem is available as open source under the terms of the MIT License.

capybara-screenshot-diff's Issues

Crop is wrong by a factor of 2

I am using your library on an iMac and suspect that the screen pixel density somehow is not taken into account when using the crop option. I am using vips as a driver.

This doesn't work:

screenshot "test", crop: '#my_element', driver: :vips

This works:

my_element.evaluate_script("this.getBoundingClientRect()").symbolize_keys => {left:, top:, right:, bottom:, width:, height:}
screenshot "test", crop: [left * 2, top * 2, right * 2, bottom * 2]

If you point me in the right direction I can prepare a PR to fix this. From what I understand you always take a "fullscreen" screenshot and then crop afterwards using the driver. In that case I suppose it is the driver crop call which needs to take this into account. I wouldn't really fiddle with determining the pixel density but rather offload this to the driver. However I wasn't able to find anything related to that in the vips docs.

The other option I can imagine is that the screenshot is taken at "normal" pixel density but vips crops it using the screen pixel density.

Let me know how I can be of help.

Add Support for Capybara 3.0

Hi,
I would like to use this project with a newer version of Capybara (3.0.2). Is it possible to test it with 3.0.2 and change the dependency?

Thank you!

DSL methods not available when using Capybara and RSpec

Not sure if this is expected to work? But Capybara::Screenshot::Diff is defined for me, but the DSL methods such as screenshot are not. I am currently working in a fork to determine what exactly is wrong - I suspect they may need to be added to Capybara::DSL?

Feature Request: I want to easily see the areas of the screenshot that are different

First off, thank you so much for making this gem. It is very useful.

When I worked at Google we had a similar tool. It had a useful feature that overlayed the two screenshots on top of each other making the top one slightly transparent so you could see where the diffs were.

It could also highlight diff areas with a semi-transparent rectangle, making them easier to spot.

Either of both of these two features would make comparing different screenshots much easier.

Publish new releases by GitHub Actions

Technical References:

• https://github.com/actions/publish-action

WIP: Adds integration tests for JavaScript Code

Is it possible to use gem with Cucumber + Capybara + RSpec?

Discussed in #71

^{Originally posted by gsguma February 24, 2023}
I'm trying to implement this gem using Cucumber + Capybara + RSpec, but it didn't work. Is it possible to use this gem in the cucumber steps?

I tried the following changes:

env.rb:

require 'capybara/screenshot/diff'

...

RSpec.configure do |config|
  config.include Capybara::Screenshot::Diff
end

test_steps.rb:

And(/^visit google$/) do
  visit 'https://google.com'

  screenshot 'test'
end

Gemfile:

  ...

  gem 'capybara-screenshot-diff'
  gem 'oily_png', platform: :ruby

  ...

And I'm getting the error undefined method 'screenshot' for #<...+Capybara::Screenshot::Diff+Capybara::Screenshot::Os+Capybara::RSpecMatchers+Capybara::DSLRSpecProxyInstaller+Capybara::DSL+RSpec::Matchers+Cucumber::Glue::ProtoWorld+ ...> (NoMethodError)

If this gem is cucumber compatible, do you have any idea that can help me?

Thank you in advance!

Remove dependency on Rails to run tests

Load failure if vips is not installed

If vips is not installed, the gem will not load.

This needs a test in travis.

Allow to skip area or crop by css selectors

Will be great to provide CSS selectors which should be excluded from the comparison, or selectors for containers to crop.

We can do this as separate libs or helpers. Also will be great to provide them as default, in order to skip footers and navigations and compare only the content part.

Screenshot of a specified element

It would be awesome if we could take a screenshot of only specified element. Like canvas. Testing canvas by checking diff in screenshot would be very reasonable.

cannot load such file -- capybara-screenshot-diff (LoadError)

gem install capybara-screenshot-diff
Fetching capybara-screenshot-diff-1.5.4.gem
Successfully installed capybara-screenshot-diff-1.5.4
Parsing documentation for capybara-screenshot-diff-1.5.4
Installing ri documentation for capybara-screenshot-diff-1.5.4
Done installing documentation for capybara-screenshot-diff after 0 seconds
1 gem installed

gem which capybara-screen-diff
ERROR:  Can't find Ruby library file or shared library capybara-screen-diff

bundle exec cucumber test.feature
cannot load such file -- capybara-screenshot-diff (LoadError)

uninitialized constant Capybara::Screenshot

Any reason why I get this error when I add include Capybara::Screenshot::Diff

Bundler installed your gem without issues

$ bundle install
...
Using capybara 3.10.0
Using chunky_png 1.3.11
Using capybara-screenshot-diff 0.13.1
Using ffi 1.9.25
Using childprocess 0.9.0
Using chromedriver-helper 2.1.0
...

But when I run the spec, I get this error for the include line

spec/t1.rb:12:in `block in <top (required)>': uninitialized constant Capybara::Screenshot (NameError)
	from /Users/ibarraalfonso/.rvm/gems/ruby-2.3.1/gems/rspec-core-3.5.4/lib/rspec/core/example_group.rb:385:in `module_exec'

I tried adding

require 'capybara-screenshot-diff'

but get a Cannot load file error

I also tried adding

require 'capybara/screenshot/diff/version'
require 'capybara/screenshot/diff/image_compare'
require 'capybara/screenshot/diff/test_methods'

This results in

NoMethodError:
       undefined method `screenshot' for #<RSpec::ExampleGroups::HomePage:0x007fa7bcc7c048>
     # ./spec/t1.rb:36:in `block (2 levels) in <top (required)>'
     # /Users/ibarraalfonso/.rvm/gems/ruby-2.3.1/gems/rspec-retry-0.5.2/lib/rspec/retry.rb:112:in `block in run'

MORE INVESTIGATIONS:
I tried directly using some of the class methods I found in the rubydocs and they seem to work fine.

pry(#<StageSite>)> aa = Capybara::Screenshot::Diff::ImageCompare.new('pone.png', 'ptwo.png')
=> #<Capybara::Screenshot::Diff::ImageCompare:0x007fa3679072e0
 @annotated_new_file_name="pone_1.png~",
 @annotated_old_file_name="pone_0.png~",
 @area_size_limit=nil,
 @bottom=nil,
 @color_distance_limit=nil,
 @dimensions=nil,
 @left=nil,
 @max_color_distance=nil,
 @max_shift_distance=nil,
 @new_file_name="pone.png",
 @old_file_name="ptwo.png",
 @right=nil,
 @shift_distance_limit=nil,
 @top=nil>

[6] pry(#<StageSite>)> aa.different?
=> false

I'm quite sure your gem is installed since after the pry line, I checked if I can see access the methods

  pry> Capybara::Screenshot::Diff.methods
   => [:delegate,
 :module_exec,
 :class_exec,
 :module_eval,
...```

But it does not have a screenshot method?

Image Minification

Hey @donv back at it with this thing :) You'd mentioned you used this for your company and personal projects. I was curious if you'd run into any situations where you'd want to minify the png's before comparing them to limit the file-size stored in the git repo?

I was thinking of some sort of minify: true config, since it does (obviously) add a delay while the images are optimized. Potentially using something like https://github.com/toy/image_optim can shrink the file-sizes down a lot (> 50% in some cases). What do you think?

assert_image_not changed with rspec usage

Hi,
I am trying to incorporate this to gem to work with rspec. In test_methods.rb I found assert_image_not changed to compare two screenshots and was not able to find instructions for this in readme. I am stuck with this. I would also appreciate if more documentation is provided for comparing two screenshots.Can you give an example of how to use assert_image_not changed with rspec?

Auto-switch driver

Hi @pftg !

I'd like to auto-switch the backend driver based on availability of gens and libraries.

The driver config should allow :auto in addition to the current :chunky_png and :vip.

If :auto is selected, and it should be the default, we should try to select chunky_png if the chunky_png gem is available or VIP if the VIP gem is avalable.

If neither chunky_png nor VIP is available, we should fail with a USAGE message stating that either chunky_png or VIP is required.

If an actual driver is selected by the developer, we should not fall back to another driver, but fail if it is not present or has trouble loading.

With this function, neither chunky_png nor VIP should be an install requirement for the gem directly, although one of them will be needed for proper function.

@pftg Are you up for implementing this? With this in place, I'd like to push a new release.

Add option to set screenshot group automatically based on the test name

This can be done today in each test class with

setup { screenshot_group @NAME[5..-1] }

I propose an option

Capybara::Screenshot::Diff.default_group_name { @NAME[5..-1] }

This should adjust to the different test frameworks.

The screenshot group name of each test can still be overridden by calling screenshot_group in the test.

wrong number of arguments (given 7, expected 6) for decode_png_image_pass

Hi, I get this error when i run the spec.
I have this in my Gemfile

  gem "capybara-screenshot-diff"
  gem "oily_png", platform: :ruby

I installed libvips

In my spec

        visit admin_users_path

        screenshot("welcome_index")

Failure/Error: decode_png_image_pass(stream, width, height, color_mode, depth, 0, decoding_palette)
                                                                                                   
ArgumentError:                                                                                     
  wrong number of arguments (given 7, expected 6)      

     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/chunky_png-1.4.0/lib/chunky_png/canvas/png_decoding.rb:120:in `decode_png_image_pass'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/chunky_png-1.4.0/lib/chunky_png/canvas/png_decoding.rb:120:in `decode_png_without_interlacing'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/chunky_png-1.4.0/lib/chunky_png/canvas/png_decoding.rb:99:in `decode_png_pixelstream'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/chunky_png-1.4.0/lib/chunky_png/canvas/png_decoding.rb:81:in `from_datastream'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/chunky_png-1.4.0/lib/chunky_png/image.rb:74:in `from_datastream'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/chunky_png-1.4.0/lib/chunky_png/canvas/png_decoding.rb:36:in `from_blob'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/capybara-screenshot-diff-1.7.1/lib/capybara/screenshot/diff/drivers/chunky_png_driver.rb:364:in `_load_images'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/capybara-screenshot-diff-1.7.1/lib/capybara/screenshot/diff/drivers/chunky_png_driver.rb:40:in `load_images'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/capybara-screenshot-diff-1.7.1/lib/capybara/screenshot/diff/image_compare.rb:81:in `different?'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/capybara-screenshot-diff-1.7.1/lib/capybara/screenshot/diff/test_methods.rb:96:in `assert_image_not_changed'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/capybara-screenshot-diff-1.7.1/lib/capybara/screenshot/diff.rb:99:in `block in track_failures'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/capybara-screenshot-diff-1.7.1/lib/capybara/screenshot/diff.rb:98:in `map'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/capybara-screenshot-diff-1.7.1/lib/capybara/screenshot/diff.rb:98:in `track_failures'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/capybara-screenshot-diff-1.7.1/lib/capybara/screenshot/diff.rb:87:in `block in included'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/example.rb:457:in `instance_exec'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/example.rb:457:in `instance_exec'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/hooks.rb:372:in `run'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/hooks.rb:529:in `block in run_owned_hooks_for'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/hooks.rb:528:in `each'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/hooks.rb:528:in `run_owned_hooks_for'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/hooks.rb:615:in `block in run_example_hooks_for'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/hooks.rb:614:in `each'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/hooks.rb:614:in `run_example_hooks_for'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/hooks.rb:485:in `run'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/example.rb:518:in `run_after_example'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/example.rb:283:in `block in run'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/example.rb:511:in `block in with_around_and_singleton_context_hooks'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/example.rb:468:in `block in with_around_example_hooks'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/hooks.rb:486:in `block in run'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/hooks.rb:626:in `block in run_around_example_hooks_for'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/example.rb:352:in `call'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-rails-6.0.3/lib/rspec/rails/example/system_example_group.rb:120:in `block (2 levels) in <module:SystemExampleGroup>'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/example.rb:457:in `instance_exec'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/example.rb:457:in `instance_exec'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/hooks.rb:390:in `execute_with'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/hooks.rb:628:in `block (2 levels) in run_around_example_hooks_for'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/example.rb:352:in `call'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-rails-6.0.3/lib/rspec/rails/adapters.rb:75:in `block (2 levels) in <module:MinitestLifecycleAdapter>'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/example.rb:457:in `instance_exec'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/example.rb:457:in `instance_exec'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/hooks.rb:390:in `execute_with'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/hooks.rb:628:in `block (2 levels) in run_around_example_hooks_for'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/example.rb:352:in `call'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/webmock-3.18.1/lib/webmock/rspec.rb:37:in `block (2 levels) in <main>'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/example.rb:457:in `instance_exec'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/example.rb:457:in `instance_exec'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/hooks.rb:390:in `execute_with'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/hooks.rb:628:in `block (2 levels) in run_around_example_hooks_for'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/example.rb:352:in `call'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/hooks.rb:629:in `run_around_example_hooks_for'
     # /Users/vucinjo/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2/lib/rspec/core/hooks.rb:486:in `run'
                                            

Any idea?
Thanks in advance

Compare without annotations for quick equal

In order to take a stable screenshot, we compare several shots and as soon we confirm that shots are stable, that there are no changes between them then we start the comparison with annotations enabled.

There is an idea to improve the current quick_equal:

Build checksum for the image with filters enabled and without. Then compare only checksums previous with the current shot.
This reduces the time on loading the second image to confirm that images are different.

Question: Why delete all screenshots in a screenshot group?

Hey @donv! Excited to get started using this. Seems like it may solve the exact problem we're looking for.

I'm curious, reading over the docs I noticed it says: All files in the screenshot group directory will be deleted when screenshot_group is called..

If it deletes all of the files in a screenshot group when it's called, how can they be used to compare against the new versions? The way I envisioned using screenshot group is like saying, "I want all of these tests to be grouped into a folder like parent/00_child_test.png, parent/01_child_other_test.png, etc.. Then when running that same test with screenshot_group 'parent' and a child test, it would look for that and compare accordingly.

Maybe I'm not understanding groups correctly, but either way I'd love to hear more about why they're deleted when that's called, and the correct way to use groups but still allow for diff comparison.

Thanks!

Tolerance erroring out

Adding the option tolerance without vips, I get this error:

 NoMethodError:
       undefined method `size' for #<Capybara::Screenshot::Diff::Drivers::ChunkyPNGDriver

Should we raise an error saying that translate option is incompatible with Chunky PNG driver (like noted in the README)?

Is it possible for capybara-screenshot-diff NOT to try to require minitest?

My problem is that I'm not using minitest in my Rails project, but it's being pulled in as a dependency by ActiveSupport and so this [1] test returns true and capybara-screenshot-diff thinks its in MiniTest context, which it isn't. I mean isn't it better to just check const_defined?(MiniTest) or something similar here: https://github.com/donv/capybara-screenshot-diff/blob/6a113e26f5e1fb664f0c359511744e00c9e3e89b/lib/capybara/screenshot/diff/drivers/utils.rb#L40C1-L46

[1] https://github.com/donv/capybara-screenshot-diff/blob/master/lib/capybara/screenshot/diff.rb#L104

skip_area with CSS selector not covering full area with vips driver

I'm using skip_area: [". admin-order-qr-code"] to generate the screnshots below. As you can see from the diff files, the skip area is missing the bottom part of the QR code. The gist has the HTML + CSS and using Chrome's inspect tool I can see that the QR Code is fully contained within its parent, so it's not an overflow issue. Furthermore, if I switch to chunky_png instead of VIPS then no diff is detected.

https://gist.github.com/alan-pie/86ac2d3c90e46032d7325c439b01bf1e

How do you assert image difference

I followed the read me , but it does not explain how to actually check the image differences . In my rspec script, I called screenshot 'test.png' , it creates a screen shot just fine, but when I switch CSS and color and re run , it does not detect changes

Source line for screenshot call is missing in the error report.

The screenshot diff error report must include which test source line the screenshot was called from.

Silently fails if screenshot is the wrong size

If the screenshot is not the right size (for example if the window size is altered during the test), capybara-screenshot-diff does not take a screenshot, it just returns early. This means that a test like this:

# somewhere in setup:
Capybara::Screenshot.window_size = [800, 600]

#test:
it 'works with permissions' do
  visit('/')
  page.current_window.resize_to(500, 500)
  screenshot 'home_page'
end

Will always pass, even if the screenshots would not match.

This example is a bit contrived but there are a lot of scenarios where resolution might not end up matching and a developer may fail to notice that screenshots are not actually being taken.

NoMethodError: undefined method `screenshot' for #<AcceptanceTest:0x0>

NoMethodError: undefined method `screenshot' for #<AcceptanceTest:0x000000047356c8>
NoMethodError: undefined method `screenshot_group' for #<AcceptanceTest:0x000000047356c8>

for

class AcceptanceTest < ActionDispatch::IntegrationTest
  include Capybara::DSL

  test "the truth" do
    visit('/')
    screenshot_group 'useful_feature'
    screenshot 'home_page'
  end
end

Rails 4.2
Gemfile.lock.zip

Feature: allow to use ruby-vips or imagemagick for performance

There is a performance issue on finding changed parts of the screenshots. We do a brute force solution. For images with 1400x1400, we will have 1400 * 1400 cycles. We could improve that by finding a difference with ready solutions and with compiled native libs. That should improve test speed.

Cons of this that we will have more dependencies and variations, which increases complexity.

The idea is to introduce a bridge pattern with several implementations that should be selected in the configuration.

Error implementing lib

I'm trying to implement this capybara screenshot diff, but I'm getting an error

I added these lines on gem file

gem 'capybara-screenshot-diff'
gem 'oily_png', platform: :ruby

I rode the command
bundle

In my testes steps file I tried it:

require 'capybara/screenshot/diff'


test 'my useful feature' do
  visit '/'
  include Capybara::Screenshot::Diff
  screenshot 'welcome_index'
end

but I received and error:
undefined method 'screenshot' for #......(NoMethodError)

so I changed to:

require 'capybara/screenshot/diff'
include Capybara::Screenshot::Diff

test 'my useful feature' do
  visit '/'
  screenshot 'welcome_index'
end

but I received and error:
undefined method 'setup' for Object:Class (NoMethodError)

I've tried reinstalling and redoing the process, but I always get these same types of errors, how can I fix it?