dequelabs / axe-core-gems Goto Github PK

• Link to live demo (if applicable)

- `axe-matchers` version: X.Y.Z and other relevant notes.
- Browser and Assistive Technology versions (if any).
- Tooling details (if any)

Steps to reproduce:

1.Pulled the axe-core-gems code(axe-core v4.1.0)
2. bundles installed
3. navigated to axe-core-cucumber/e2e/cucumber
for the test file: "http://qateam.dequecloud.com/attest/api/test.html"
When I tried to analyse the page looks like analyze() is running on empty page(though it has launched the correct page first, later empty screen is shown).. so it is not giving voilations for empty page.
Test feature:
Given I am a visitor
When I visit "http://qateam.dequecloud.com/attest/api/test.html"
Scenario: Test whole page (known to be inaccessible, should fail)
Then the page should be axe clean

Hey everyone,

Thanks for this gem. It's pretty good and It has helped us a lot.

I used this gem in some of the pages from my application and I fixed all the issues, it turns out I discovered this chrome extension: https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd. When I use the extension it finds other issues that this gem didn't find.

Should I trust on the chrome extension?

Hi,

We are using aXe-core api to hit our application using phantomJS and it is working perfectly. We have BDD testing in Selenium, Cucumber and Grunt. Does axe-matchers supports this, do you have any documentation for same, i can see only Ruby Rails and Cucumber configuration.

Thanks for help.

Dependabot can't resolve your Ruby dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Bundler::GemNotFound with message: Could not find gem 'axe-core-api' in source at `..`.
The source does not contain any versions of 'axe-core-api'

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Tried 2 cases:
1: adding gemfile, proper require and the configure params. Failure...
2: Straight from : unzipping the package, going to examples and running the cucumber in the Watir folder. Failure..

e:\QA\Cucumber\axe-matchers-master\examples\watir>cucumber
undefined method configure' for Axe:Module (NoMethodError) e:/QA/Cucumber/axe-matchers-master/examples/watir/features/support/env.rb:10:in <top (required)>'
C:/ruby200/lib/ruby/gems/2.0.0/gems/cucumber-2.3.2/lib/cucumber/rb_support/rb_la
nguage.rb:96:in `load'

Dependabot can't resolve your Ruby dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Bundler::GemNotFound with message: Could not find gem 'axe-core-api' in source at `..`.
The source does not contain any versions of 'axe-core-api'

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Dependabot couldn't find a Gemfile for this project.

Dependabot requires a Gemfile to evaluate your project's current Ruby dependencies. It had expected to find one at the path: /Gemfile.

If this isn't a Ruby project, or if it is a library, you may wish to disable updates for it from within Dependabot.

View the update logs.

Dependabot can't resolve your Ruby dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Bundler::GemNotFound with message: Could not find gem 'axe-core-api' in source at `..`.
The source does not contain any versions of 'axe-core-api'

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

CircleCI caching of dependencies

When upgrading to use [email protected] we will need to add the ability to configure frame origin

Malformed results being returned from axe-core cause errors while rendering the failure messages within axe-matchers. The results object is defined in axe-core https://github.com/dequelabs/axe-core/blob/master/doc/API.md#results-object.

Axe-matchers should gracefully handle situations when there is information missing from the audit results.

Hello,

We have to use Minitest when writing system tests and Rspec when writing accessibility tests (using axe). We're repeating our tests in both a lot of the times. I was wondering if there were any plans to have an axe-minitest so we can move our tests over to Minitest?

Thanks.

Expectation: upgrade to use [email protected] and axe-core#runPartial()

Actual: currently uses [email protected]

Gem Version-> 2.4.1
Steps to Reproduce

1. Add the gem to the repository Gemfile
2. Add the include to the step definitions require 'axe/cucumber/step_definitions' to features/support/env.rb
3. Trying to run a rake task after this caused a error when its trying to resolve the require statement in the env.rb file

Workaround:

1. Move the require 'axe/cucumber/step_definitions' line to the step definition file that has some of your step definitions for the tests that use this gem

error stack:
rake aborted! NoMethodError: undefined method Then' for main:Object
/usr/local/rvm/gems/ruby-2.5.1@qa_shared_shelf/gems/axe-matchers-2.4.1/lib/axe/cucumber/step_definitions.rb:3:in <top (required)>' /Users/jshah/forum-ui/ui_tests/features/support/env.rb:7:in require'
/Users/jshah/forum-ui/ui_tests/features/support/env.rb:7:in <top (required)>' /Users/jshah/forum-ui/ui_tests/Rakefile:4:in require'
/Users/jshah/forum-ui/ui_tests/Rakefile:4:in <top (required)>' /usr/local/rvm/gems/ruby-2.5.1@qa_shared_shelf/gems/rake-12.3.3/lib/rake/rake_module.rb:29:in load'
/usr/local/rvm/gems/ruby-2.5.1@qa_shared_shelf/gems/rake-12.3.3/lib/rake/rake_module.rb:29:in load_rakefile' /usr/local/rvm/gems/ruby-2.5.1@qa_shared_shelf/gems/rake-12.3.3/lib/rake/application.rb:703:in raw_load_rakefile'
/usr/local/rvm/gems/ruby-2.5.1@qa_shared_shelf/gems/rake-12.3.3/lib/rake/application.rb:104:in block in load_rakefile' /usr/local/rvm/gems/ruby-2.5.1@qa_shared_shelf/gems/rake-12.3.3/lib/rake/application.rb:186:in standard_exception_handling'
/usr/local/rvm/gems/ruby-2.5.1@qa_shared_shelf/gems/rake-12.3.3/lib/rake/application.rb:103:in load_rakefile' /usr/local/rvm/gems/ruby-2.5.1@qa_shared_shelf/gems/rake-12.3.3/lib/rake/application.rb:82:in block in run'
/usr/local/rvm/gems/ruby-2.5.1@qa_shared_shelf/gems/rake-12.3.3/lib/rake/application.rb:186:in standard_exception_handling' /usr/local/rvm/gems/ruby-2.5.1@qa_shared_shelf/gems/rake-12.3.3/lib/rake/application.rb:80:in run'
/usr/local/rvm/gems/ruby-2.5.1@qa_shared_shelf/gems/rake-12.3.3/exe/rake:27:in <top (required)>' /usr/local/rvm/gems/ruby-2.5.1@qa_shared_shelf/bin/rake:23:in load'
/usr/local/rvm/gems/ruby-2.5.1@qa_shared_shelf/bin/rake:23:in <main>'

• Link to live demo (if applicable)

- `axe-matchers` version: X.Y.Z and other relevant notes.
- Browser and Assistive Technology versions (if any).
- Tooling details (if any)

• Remove any publish steps from CI that could push either a next version or actual version of axe-matchers accidentally to rubygems. - #100
• Set up directory structure for split gems per [proposal][https://github.com/dequelabs/html-team-proposals/pull/135/files#diff-5c8c48ed15bd394d11c60d223543e7a7R1]. Each of these directories will have its own gemspec file, if that directory is to produce a gem
• Split core/ common functions
  • loaders
  • hooks
  • axe api wrappers
  • web driver adapters
• Split webdrivers to be its own gems (1 webdriver per PR)
  • axe-core-selenium
  • axe-core-watir
  • axe-core-capybara
• Split testing framework integrations (1 testing framework per PR)
  • axe-rspec
  • axe-cucumber
• Set up build script (Rake or Bazel) to create each of these gems
• Introduce a README to reflect the proposal
• Remove top level gemspec name
• Set up examples to use updated gems and new API
• Set up CI

Note: Given the nature of the work, tests will fail during this change when between PR's

Steps to reproduce:

1. Bundles installed from release branch axe-core-gems
2. Verified the axe-core version , 4.2.0
3. On bundle exec cucumber in capabara driver:
   violations list is not expected.

@javascript
Feature: Example using default Capybara setup
  Default driver is :rack_test, uses :selenium by default for any test tagged
  with @javascript.

  Background:
    Given I am a visitor
    When I visit "http://qateam.dequecloud.com/attest/api/test.html"

  Scenario: Test whole page (known to be inaccessible, should fail)
    Then the page should be axe clean according to: wcag2a

Output:

bundle exec cucumber
*** WARNING: You must use ANSICON 1.31 or higher (https://github.com/adoxa/ansicon/) to get coloured output on Windows
@javascript
Feature: Example using default Capybara setup
Default driver is :rack_test, uses :selenium by default for any test tagged
with @javascript.

Background: # features/a11y.feature:6
Given I am a visitor # features/step_definitions/steps.rb:1
When I visit "http://qateam.dequecloud.com/attest/api/test.html" # features/step_definitions/steps.rb:4

Scenario: Test whole page (known to be inaccessible, should fail) # features/a11y.feature:10
Then the page should be axe clean according to: wcag2a # C:/Users/DQ_Santosh/Documents/Releases/Axe-core-4.2/axe-core-gems-release-2020-05-06/axe-core-gems-release-2020-05-06/packages/axe-core-cucumber/lib/axe-cucumber-steps.rb:3

  Found 2 accessibility violations:

  1) document-title: Documents must have <title> element to aid in navigation (serious)
      https://dequeuniversity.com/rules/axe/4.2/document-title?application=axeAPI
      The following 1 node violate this rule:

          Selector: html
          HTML: <html><head></head><body></body></html>
          Fix any of the following:
          - Document does not have a non-empty <title> element

  2) html-has-lang: <html> element must have a lang attribute (serious)
      https://dequeuniversity.com/rules/axe/4.2/html-has-lang?application=axeAPI
      The following 1 node violate this rule:

          Selector: html
          HTML: <html><head></head><body></body></html>
          Fix any of the following:
          - The <html> element does not have a lang attribute

  Invocation: axe.run({"runOnly":{"type":"tag","values":["wcag2a"]}}, callback); (RuntimeError)
  C:/Users/DQ_user/Documents/Releases/Axe-core-4.2/axe-core-gems-release-2020-05-06/axe-core-gems-release-2020-05-06/packages/axe-core-api/lib/axe/expectation.rb:4:in `assert'
  C:/Users/DQ_user/Documents/Releases/Axe-core-4.2/axe-core-gems-release-2020-05-06/axe-core-gems-release-2020-05-06/packages/axe-core-cucumber/lib/axe-cucumber.rb:65:in `assert_accessibility'
  features/a11y.feature:13:in `the page should be axe clean according to: wcag2a'

Failing Scenarios:
cucumber features/a11y.feature:10 # Scenario: Test whole page (known to be inaccessible, should fail)

1 scenario (1 failed)
3 steps (1 failed, 2 passed)
0m15.368s

Expected: 
```
 Violation of "aria-allowed-attr" with 1 occurrences!
    Ensures ARIA attributes are allowed for an element's role. Correct invalid elements at:
     - #wcag2a-fail > a[href="\#"][role="button"]
    For details, see: https://dequeuniversity.com/rules/axe/4.2/aria-allowed-attr

    Violation of "label" with 1 occurrences!
    Ensures every form element has a label. Correct invalid elements at:
     - input[name="lastname"]
    For details, see: https://dequeuniversity.com/rules/axe/4.2/label

2 Accessibility issues detected.

and also for the scenario

Background:
    Given I am a visitor
    When I visit "http://qateam.dequecloud.com/attest/api/test.html"

  Scenario: Test whole page (known to be inaccessible, should fail)
    Then the page should be axe clean

output

bundle exec cucumber
*** WARNING: You must use ANSICON 1.31 or higher (https://github.com/adoxa/ansicon/) to get coloured output on Windows
@javascript
Feature: Example using default Capybara setup
  Default driver is :rack_test, uses :selenium by default for any test tagged
  with @javascript.

  Background:                                                        # features/a11y.feature:6
    Given I am a visitor                                             # features/step_definitions/steps.rb:1
    When I visit "http://qateam.dequecloud.com/attest/api/test.html" # features/step_definitions/steps.rb:4

  Scenario: Test whole page (known to be inaccessible, should fail) # features/a11y.feature:10
    Then the page should be axe clean                               # C:/Users/DQ_Santosh/Documents/Releases/Axe-core-4.2/axe-core-gems-release-2020-05-06/axe-core-gems-release-2020-05-06/packages/axe-core-cucumber/lib/axe-cucumber-steps.rb:3

      Found 4 accessibility violations:

      1) document-title: Documents must have <title> element to aid in navigation (serious)
          https://dequeuniversity.com/rules/axe/4.2/document-title?application=axeAPI
          The following 1 node violate this rule:

              Selector: html
              HTML: <html><head></head><body></body></html>
              Fix any of the following:
              - Document does not have a non-empty <title> element

      2) html-has-lang: <html> element must have a lang attribute (serious)
          https://dequeuniversity.com/rules/axe/4.2/html-has-lang?application=axeAPI
          The following 1 node violate this rule:

              Selector: html
              HTML: <html><head></head><body></body></html>
              Fix any of the following:
              - The <html> element does not have a lang attribute

      3) landmark-one-main: Document should have one main landmark (moderate)
          https://dequeuniversity.com/rules/axe/4.2/landmark-one-main?application=axeAPI
          The following 1 node violate this rule:

              Selector: html
              HTML: <html><head></head><body></body></html>
              Fix all of the following:
              - Document does not have a main landmark

      4) page-has-heading-one: Page should contain a level-one heading (moderate)
          https://dequeuniversity.com/rules/axe/4.2/page-has-heading-one?application=axeAPI
          The following 1 node violate this rule:

              Selector: html
              HTML: <html><head></head><body></body></html>
              Fix all of the following:
              - Page must have a level-one heading

      Invocation: axe.run(callback); (RuntimeError)
      C:/Users/DQ_user/Documents/Releases/Axe-core-4.2/axe-core-gems-release-2020-05-06/axe-core-gems-release-2020-05-06/packages/axe-core-api/lib/axe/expectation.rb:4:in `assert'
      C:/Users/DQ_user/Documents/Releases/Axe-core-4.2/axe-core-gems-release-2020-05-06/axe-core-gems-release-2020-05-06/packages/axe-core-cucumber/lib/axe-cucumber.rb:65:in `assert_accessibility'
      features/a11y.feature:11:in `the page should be axe clean'

Failing Scenarios:
cucumber features/a11y.feature:10 # Scenario: Test whole page (known to be inaccessible, should fail)

1 scenario (1 failed)
3 steps (1 failed, 2 passed)
0m13.934s

Expected:

Violation of "aria-allowed-attr" with 1 occurrences!
    Ensures ARIA attributes are allowed for an element's role. Correct invalid elements at:
     - #wcag2a-fail > a[href="\#"][role="button"]
    For details, see: https://dequeuniversity.com/rules/axe/4.2/aria-allowed-attr

 Violation of "color-contrast" with 1 occurrences!
    Ensures the contrast between foreground and background colors meets WCAG 2 AA contrast ratio thresholds. Correct invalid elements at:
     - #wcag2aa-fail > p
    For details, see: https://dequeuniversity.com/rules/axe/4.2/color-contrast

Violation of "heading-order" with 1 occurrences!
    Ensures the order of headings is semantically correct. Correct invalid elements at:
     - h6
    For details, see: https://dequeuniversity.com/rules/axe/4.2/heading-order

 Violation of "label" with 1 occurrences!
    Ensures every form element has a label. Correct invalid elements at:
     - input[name="lastname"]
    For details, see: https://dequeuniversity.com/rules/axe/4.2/label

4 Accessibility issues detected.

Noite:
Watir driver is working fine

We have axe-core running on the front end, along with axe-matchers in CI. Currently, we have two different sets of configuration and two different versions of axe-core.

I'm wondering if there's a way to have axe-matchers use the same script and configuration that we are already loading?

Please see the same issue I posted here https://github.com/evcohen/eslint-plugin-jsx-a11y/issues/630

A menu item, acting as a submenu given a role of menuitem should have an aria-expanded attribute. I am being thrown an error when I do this.

Something like this raises an error:

<ul role="menubar">
  <li>
    <a role="menuitem" aria-expanded="false" href="someurl.com">Some URL</a>
      <ul role="menu">
        <li><a href="https://google.com">Google.com</a></li>
        <li><a href="https://github.com">Github.com</a></li>
      </ul>
  </li>
</ul>

The attribute aria-expanded is not supported by the role menuitem.

According to w3c/aria#454 this is valid aria spec.

This is also shown on an example on W2
https://www.w3.org/TR/wai-aria-practices/examples/menubar/menubar-1/menubar-1.html

Am using axe-matchers 1.3.2 , when I do Axe::Cucumber.configure as mentioned in your docs - am running into this issue that NoMethodError. Any help?

Steps to reproduce:

1.Pulled the axe-core-gems code(axe-core v4.1.0)
2. bundles installed
3. navigated to axe-core-cucumber/e2e/watir (also issue with axe-core-rspec/e2e/capybara)
for the test file: "http://qateam.dequecloud.com/attest/api/test.html"
When I tried to analyse the page for specific rules,
Given I am a visitor
When I visit "http://qateam.dequecloud.com/attest/api/test.html"
Scenario: Test whole page (known to be inaccessible, should fail)
Then the page should be axe clean checking: aria-allowed-attr, label

before :each do
visit "http://qateam.dequecloud.com/attest/api/test.html"
end

it "is known to be inaccessible, should fail" do
expect(page).to be_axe_clean.checking :label
end
(here it need to correct the readME file too for command correction )
Actual:
Its not checking the specific rules, but checking entire results.

• finishRun error handling
  • asserts createWindow or window.open returns a valid handle
  • wraps switchToWindow in a try / catch
  • wraps call to private finishRun in try / catch with a message that points to the error handling doc
• Error handling doc

Dependabot can't resolve your Ruby dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Bundler::GemNotFound with message: Could not find gem 'axe-core-api' in source at `..`.
The source does not contain any versions of 'axe-core-api'

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

- `axe-core-rspec (4.2.0)` was working on 4.1.0

Error:

undefined method `find_elements' for #<WebDriverScriptAdapter::FrameAdapter::WatirAdapter:0x00007fb1464f54a8>

On callee:

expect(entry_point).to be_axe_clean.according_to(:wcag2a, :wcag2aa)

Upgrading from 4.1 to 4.2.

as part of upgrade watir went to (6.19.1) - from 6.19.0. Is there a locked dep missing?

While running Axe-matcher with Cucumber. Always get warning in the Pretty format even though standard check is passing

Feature: Color Contrast

Scenario Outline: Page should have valid color combinations according to BBC accessibility standards # features/10_color_contrast.feature:3
Given I am on the page # features/10_color_contrast.feature:5
Then page should have all relevant "" for the accessibility guideline # features/10_color_contrast.feature:6

Examples:
  | attributes     |
  |No response from frame:  [object HTMLIFrameElement]

accesskeys |
|No response from frame: [object HTMLIFrameElement]
color-contrast |

Can we suppress this warnings?

Great library. We have a long way to go fixing up accessibility so I want to tackle it guideline-by-guideline and make sure our specs will fail if anyone makes changes that fail one of the guidelines I've already dealt with. To do that I'll need to turn on specific rules one by one.

I can't see documentation for how I can configure axe-matchers to include/exclude specific rules, or set an impact level to test for (Critical/Serious/Minor).

• Is it possible to configure like that?
• If so, is it documented somewhere? I couldn't find anything linked from the readme.

Hi there --
We leverage axe-matchers extensively through our rspec suite.
When profiling, we noticed that each accessibility check was taking 20 seconds -- These calls ended up taking up 30-45% of our suite's total run time.

This call to find iframes waits for Capybara's default_max_wait_time which in our case was 20 seconds.

https://github.com/dequelabs/axe-matchers/blob/97dad5b952342e45f903d3b042062d4fee18e031/lib/webdriver_script_adapter/frame_adapter.rb#L52

I'm wondering if iframe auditing can be turned off by default?
Or maybe it could just be called out in the docs boldly with instructions for how to disable it Axe::Configuration.from_hash(skip_iframes: true)?

Any other ideas?

When writing an RSpec Test with axe-matchers, the according_to parameter should throw an error or print a warning if an unknown ruleset is passed.

I believe this would be helpful to prevent false positive tests that are a result of a typo. For example if a user tests:

expect(page).to be_accessible.according_to(:best-practices)

this expectation will pass 100% of the time, no matter how inaccessible the page is. The user is never warned that best-practices is not a valid ruleset (the correct name is best-practice, singular).

- `axe-matchers` Version: 2.5.0
- Firefox Version: 60.9.0esr
- Selenium Webdriver Version: 4.0.0.alpha3 (same results with 3.142.6)
- Geckodriver Version: 0.26.0

I have added the following to my env.rb for Cucumber:

require 'axe/cucumber/step_definitions'

I also wrote a very simple Feature:

@javascript
Feature: Accessibility

  Scenario: Sign in page
    Given I am on the sign in page    # This only does a "visit /signin"
    Then the page should be accessible

When I run this I receive the following error:

@javascript
Feature: Accessibility

Capybara starting Puma...
* Version 3.11.2 , codename: Love Song
* Min threads: 0, max threads: 4
* Listening on tcp://127.0.0.1:40085
  Scenario: Sign in page               # features/accessibility.feature:8
    Given I am on the sign in page     # features/step_definitions/sessions_steps.rb:5
    Then the page should be accessible # axe-matchers-2.5.0/lib/axe/cucumber/step_definitions.rb:3
      ReferenceError: __magic__ is not defined (Selenium::WebDriver::Error::JavascriptError)
      execute_script @, line 0
      inline javascript, line 12
      src: "undefined"
      Stack:
      e/<[2]</<@http://127.0.0.1:40085/signin:12:2953
      e/<[2]<@http://127.0.0.1:40085/signin:12:2173
      l@http://127.0.0.1:40085/signin:12:1243
      l/<@http://127.0.0.1:40085/signin:12:1285
      e/<[1]<@http://127.0.0.1:40085/signin:12:1865
      l@http://127.0.0.1:40085/signin:12:1243
      o@http://127.0.0.1:40085/signin:12:1409
      e@http://127.0.0.1:40085/signin:12:1008
      @http://127.0.0.1:40085/signin:12:11
      @http://127.0.0.1:40085/signin:1:2
      features/accessibility.feature:10:in `Then the page should be accessible'

Failing Scenarios:
cucumber features/accessibility.feature:8 # Scenario: Sign in page

1 scenario (1 failed)
2 steps (1 failed, 1 passed)
0m2.879s

If I run the feature without the @javascript tag I receive:

Feature: Accessibility

  Scenario: Sign in page               # features/accessibility.feature:7
    Given I am on the sign in page     # features/step_definitions/sessions_steps.rb:5
    Then the page should be accessible # axe-matchers-2.5.0/lib/axe/cucumber/step_definitions.rb:3
      Capybara::Driver::Base#evaluate_script (Capybara::NotSupportedByDriverError)
      features/accessibility.feature:9:in `Then the page should be accessible'

Failing Scenarios:
cucumber features/accessibility.feature:7 # Scenario: Sign in page

1 scenario (1 failed)
2 steps (1 failed, 1 passed)
0m0.170s

Any idea what I can do to proceed? Thank you so much for this amazing gem! I can't wait to use it further.

Dependabot couldn't authenticate with agora.dequecloud.com/artifactory/api/npm/dequelabs/.

Dependabot tried to authenticate with the details you previously provided, but authentication failed. If they are no longer valid you will need to provide Dependabot with new credentials.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

Steps to reproduce:

1. Pulled the axe-core-gems code(axe-core v4.1.0)
2. bundles installed
3. navigated to axe-core-cucumber/e2e/capybara
   for the test file: "http://qateam.dequecloud.com/attest/api/test.html"
   Its giving the results excluding "iframe" related issues.
   Actual:
   5 types issues found:
4. aria-allowed-attr
   2.autocomplete-valid
5. color-contrast
6. heading-order
7. label

Expected:

1. aria-allowed-attr
   2.autocomplete-valid
2. color-contrast
3. heading-order
4. label
5. dl-item (iframe related ids are there)
6. image-alt (iframe related ids are there)
   Actual:
   issue-1.txt
   Expected:
   expectedForissue-1.txt

Note: same issue seeing with "rspec" too

I'm trying to run this on my CI, however on the build step i always get this error
cannot load such file -- axe-cucumber-steps (LoadError)

Some Gemfile imports

Using axe-core-api 4.2.1
Using axe-core-cucumber 4.2.1
Using axe-core-selenium 4.2.1

Looks like the step.rb is using the regex with the name capture groups and cucumber stop supporting the named capture groups and hence cucumber integration with axe-matchers is broken if you are using cucumber > 2.2

cucumber/common#329

class Step
      REGEX = /^

      # require initial phrasing, with 'not' to negate the matcher
      (?-x:the page should(? not)? be accessible)

      # optionally specify which subtree to check, via CSS selector
      (?-x:;? within "(?.*?)")?

      # optionally specify subtrees to be excluded, via CSS selector
      (?-x:;?(?: but)? excluding "(?.*?)")?

      # optionally specify ruleset via list of comma-separated tags
      (?-x:;? according to: (?.*?))?

      # optionally specify rules to check as comma-separated list of rule ids
      # in addition to default ruleset or explicit ruleset specified above via tags
      # if the 'only' keyword is supplied, then *only* the listed rules are checked, not *additionally*
      (?-x:;?(?: and)? checking(? only)?: (?.*?))?

      # optionally specify rules to skip as comma-separated list of rule ids
      (?-x:;?(?: but)? skipping: (?.*?))?

      # optionally specify custom options to pass directly to axe-core as a yaml-parsed hash or json string
      (?-x:;? with options: (?.*?))?

      $/x

Named capture groups are not supported. See https://github.com/cucumber/cucumber/issues/329 (Cucumber::CucumberExpressions::CucumberExpressionError)
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-expressions-6.0.1/lib/cucumber/cucumber_expressions/tree_regexp.rb:38:in `block in initialize'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-expressions-6.0.1/lib/cucumber/cucumber_expressions/tree_regexp.rb:17:in `each'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-expressions-6.0.1/lib/cucumber/cucumber_expressions/tree_regexp.rb:17:in `each_with_index'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-expressions-6.0.1/lib/cucumber/cucumber_expressions/tree_regexp.rb:17:in `initialize'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-expressions-6.0.1/lib/cucumber/cucumber_expressions/regular_expression.rb:12:in `new'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-expressions-6.0.1/lib/cucumber/cucumber_expressions/regular_expression.rb:12:in `initialize'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-3.1.2/lib/cucumber/glue/registry_and_more.rb:155:in `new'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-3.1.2/lib/cucumber/glue/registry_and_more.rb:155:in `create_expression'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-3.1.2/lib/cucumber/glue/step_definition.rb:30:in `new'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-3.1.2/lib/cucumber/glue/registry_and_more.rb:83:in `register_rb_step_definition'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-3.1.2/lib/cucumber/glue/dsl.rb:31:in `register_rb_step_definition'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-3.1.2/lib/cucumber/glue/dsl.rb:133:in `register_rb_step_definition'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/axe-matchers-2.3.0/lib/axe/cucumber/step_definitions.rb:3:in `'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:135:in `require'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:135:in `rescue in require'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:39:in `require'
/Users/cp74367/Documents/BK/RubymineProjects/BitBucket/ace/powr/features/support/env.rb:38:in `'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-3.1.2/lib/cucumber/glue/registry_and_more.rb:107:in `load'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-3.1.2/lib/cucumber/glue/registry_and_more.rb:107:in `load_code_file'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-3.1.2/lib/cucumber/runtime/support_code.rb:144:in `load_file'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-3.1.2/lib/cucumber/runtime/support_code.rb:85:in `block in load_files!'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-3.1.2/lib/cucumber/runtime/support_code.rb:84:in `each'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-3.1.2/lib/cucumber/runtime/support_code.rb:84:in `load_files!'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-3.1.2/lib/cucumber/runtime.rb:272:in `load_step_definitions'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-3.1.2/lib/cucumber/runtime.rb:68:in `run!'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-3.1.2/lib/cucumber/cli/main.rb:34:in `execute!'
/Users/cp74367/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/cucumber-3.1.2/bin/cucumber:9:in `'
/Users/cp74367/.rbenv/versions/2.5.1/bin/cucumber:23:in `load'
/Users/cp74367/.rbenv/versions/2.5.1/bin/cucumber:23:in `'

I was trying to setup Axe in my Minitest test suite (following the general idea presented here), and I hit an error that took me a while to track down. I was using this as my helper method:

def assert_accessible(page)
  matcher = Axe::Matchers::BeAxeClean.new

  assert(matcher.matches?(page), matcher.failure_message)
end

and in my assert call I kept getting this error:

NoMethodError: undefined method `reject' for nil:NilClass

After digging into the backtrace and your source code, I found that the source of the problem was building up the failure_message for the audit results:

In my case, I was hitting a problem when inspecting the CheckedNode results:

Specifically, none at this point was nil.

I saw at the top of the file that you were using Virtus to declare that this value should be an Array, and thus should default to being an empty array and not nil. In experimenting, I found that it was becoming nil because of the nullify_blank option being used in the ValueObject setup:

When I manually switched this to false, my tests started working as expected.

I am not opening a PR immediately, however, because I'm not sure why this option was added in the first place and whether changing it might break something else. So I thought I would inquire first.

Given I have a step
"Then the page should be accessible according to: wcag2a, wcag2aa but skipping: color-contrast"
Then I get the response
"Elements must have sufficient color contrast:" and so on.

I see that it calls

Invocation: axe.a11yCheck({}, {"runOnly":{"type":"tag","values":["wcag2a","wcag2aa"]},"rules":{"color-contrast":{"enabled":false}}}, callback); (RuntimeError)

So it's a problem in the axe.a11yCheck method.

https://app.circleci.com/pipelines/github/dequelabs/axe-core-gems/241/workflows/bda7b04e-d512-4de2-a0e2-62ed4215a812/jobs/653

• Link to live demo (if applicable)

- `axe-core-rspec` version: 4.2.1
- Testing with chromedriver, Chrome 93

I have a section that has a background image that is fairly dark. On top of this is a transparent p that contains an a with the text "Application Info`, which also has a transparent background -- that is, the background is very evident directly behind the text. The text itself is color #6dccf6. The image background is defined as

      background: no-repeat top left / cover url(asset-path("hero-background.jpg")) #000000;

I get the error:

               Selector: a[href="\#"]:nth-child(2)
               HTML: <a href="#">Application Info</a>
               Fix any of the following:
               - Element has insufficient color contrast of 1.8 (foreground color: #6dccf6, background color: #ffffff, font size: 9.0pt (12px), font weight: bold). Expected contrast ratio of 4.5:1

But there is nothing behind the a that has a background color of #ffffff. Incidentally, there is other text inside the p that is color: #ffffff and is not getting this error.

Why am I getting this specific error when I should not be?

Firstly, thank you for the great tool.

I'm looking to update from axe-matchers to the newer axe-core-... libraries but noticed that the axe-core-api is installing the capybara, selenium-webdriver and watir gems. Is there a reason they are explicitly referenced/installed there? I couldn't find any direct references to those gems (being loaded) in the axe-core-api. I see that they are referenced in the specs, however it would seem more appropriate to include them in the Gemfile?

Let's say I have the rule color-contrast failing inside div.some-class.

I can't fix this right now for whatever reason, and want to be able to skip it while checking the rest of the page.

I can say expect(page).to be_axe_clean.excluding(".some-class"), and the test will pass again -- but now it's not checking anything inside .some-class, ideally I'd like it to check other rules, just not color-contrast.

I can say be_axe_clean.skipping(:"color-contrast"), but now it's not checking color-contrast anywhere, ideally I'd like it to check color-contrast in other parts of the page, just not in div.some-class.

Is there any way to tell it to skip a specific rule only inside a specific selector? Can/should there be?

These of course don't work, but one could imagine API like:

# not actually existing API, does not work
expect(page).to be_axe_clean.skipping({rule: :"color-contrast", selector: ".some_class"})
expect(page).to be_axe_clean.excluding({selector: ".some-class", rule: :"color-contrast"})

There are currently integrations available for Capybara, Cucumber, RSpec, Selenium and Watir. It might be good to have additional integrations added to this package. We are open for public contributions of additional integrations.

• Implements recursive runPartial
  • injects and configures axe on each frame (exception: top-level iframe since axe might already be injected there from testing legacy code path)
  • calls axe.utils.getFrameContexts on each frame
  • runs axe.runPartial on each frame
  • finds each iframe using axe.utils.shadowSelect
  • calls function again with new iframe and context
  • returns null if iframe errors for any reason and does not process children iframes
• implements finishRun
  • opens new tab and navigates to blank page (using .goto or .navigate, etc.)
  • injects and configures axe
  • calls axe.finishRun and passes results from all iframes
  • closes blank page and returns driver context to original page
• calls axe.run in legacy axe versions (where axe.runPartial is not defined)
• implements all fixture tests
  • analyze - throws if axe errors out on the top window
  • analyze - throws when injecting a problematic source
  • analyze - throws when a setup fails
  • analyze - properly isolates the call to axe.finishRun
  • analyze - returns correct results metadata
  • frame tests - injects into nested iframes
  • frame tests - injects into nested frameset
  • frame tests - should work on shadow DOM iframes
  • frame tests - reports erroring frames in frame-tested
  • frame tests - returns the same results from runPartial as from legacy mode
  • for versions without axe.runPartial - can run
  • for versions without axe.runPartial - throws if the top level errors
  • for versions without axe.runPartial -reports frame-testedhelpUrl`

• Change log generation should be automated.

Dependabot can't resolve your Ruby dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Bundler::GemNotFound with message: Could not find gem 'axe-core-api' in source at `..`.
The source does not contain any versions of 'axe-core-api'

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

We need to be able to test iframes with the sandbox attribute:

<iframe sandbox src="..."></iframe>

We have a solution for this in axe-webdriverjs that we can copy here.

Hi,
I'm testing axe with Selenium/Axe using RSPEC. I'm attempting to construct an axe accessibility RSpec check:
expect(@lti).to be_accessible.within(iframe: 'tool_content', selectr: 'learn-app--main-content')
@LTI is a webdriver instance:
=> #<LtiA11y:0x00007ff2a22d0180 @driver=#<Selenium::WebDriver::Chrome::Driver:0x66afd90ae96e316 browser=:chrome>>
When invoke the expect, I get this message:

WebDriverScriptAdapter::WebDriverError:
       WebDriver must respond to #execute_script

I'm following the instructions from this page:
https://github.com/dequelabs/axe-matchers#installation
Then for RSpec:
https://github.com/dequelabs/axe-matchers/blob/develop/docs/RSpec.md

Any pointers appreciated. I feel like I may have missed something in my setup
Thanks,
Eddie

I'm on MacOS Mohave 10.14.1

axe-matchers (2.2.1)
Chrome Version 70.0.3538.110
selenium-webdriver (3.14.1, 3.11.0)
rspec (3.8.0, 3.7.0, 3.6.0)
ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-darwin17]

• Link to live demo (if applicable)

Error:-
    undefined method `switch_to_frame' for # WebDriverScriptAdapter::FrameAdapter::CapybaraAdapter

On callee:-
   within_frame(frame)

Upgrading from 4.1 to 4.2.

dumb_delegator gem still in 0.8.1 version, is that causing the issue?

As of Cucumber-expression 5.0.14, named capture groups are no longer supported, due to it being buggy in Ruby. This means that axe-matchers' Cucumber selector will not function with Cucumber 3.1.0. The regex will need to be replace. In the mean time people should stay on Cucumber 3.0.0.

Similar to this PR we should find <frame> elements and inject axe-core