quailjs / quail

Accessibility testing in the browser and on the server.

Deprecation

All projects have a useful lifespan. Quail was born during a time when accessibility testing, as a discipline, was maturing. There were numerous teams building solutions in parallel -- Quail was just one of these. In the end, we were all orbiting around the same general approach. Technology may have been a distinguishing factor a couple years ago, but even these advantages have largely dissolved as solutions have evolved and improved over time.

Given the realities of the market and the limited time that the Quail team can devote to this project, we are initializing deprecation for this project. Folks are welcome to fork it or volunteer to maintain it, but realistically, there are better options out there.

If you are looking for one, we recommend aXe: the Accessibility Engine by Deque Labs.

Quail: Accessibility Information Library

The project website is quailjs.io.

Quail is a Node module and a jQuery plugin that lets you easily check HTML for adherence to accessibility standards. It comes with over 200 tests which implement Open Accessibility Tests and comes with WCAG 1.0, WCAG 2.0, and Section 508 guidelines.

Developers can build their own guidelines, or easily build a custom guideline that integrates with their project. While the project supports checking entire HTML pages, integration with a CMS to check partial HTML content is probably the most popular use case.

Notice for developers working in the 2.2.x version.

Please base your changes on the master-2.2.x branch.

Requirements

You will need the following installed on your system to run Quail.

1. Node.js
2. Grunt & Grunt CLI
3. Bower
4. Java JDK (To run Selenium Server)
5. Karma

Using Quail

Setup

Run the following commands to build Quail.

git clone https://github.com/quailjs/quail.git
cd quail
npm install
bower install
grunt build

Using Quail from the command line

To see the Quail CLI API, run the following. You can also add the Quail bin directory to your PATH if you are so inclined.

./bin/quail --help

Once the command is set up, you can run the following to test any addressable web page.

./bin/quail eval http://placekitten.com

Write the results as a JSON object out to file in an existing directory.

./bin/quail evaluate http://placekitten.com -o ./analyses

You can also pass configurations to the evaluator.

./bin/quail evaluate http://placekitten.com -c ~/path/to/myconfig.json -o ./analyses

This is the default set of configurations. Your configurations will replace the defaults.

{
  "phantomjs": {
    "resourceTimeout": 5000
  },
  "blacklists": {
    "domains": [
      "fbstatic.com",
      "facebook.com",
      "twitter.com",
      "google-analytics.com",
      "googleadservices.com",
      "googlesyndication.com",
      "perfectaudience.com",
      "typekit.com",
      "sharethis.com",
      "doubleclick.com",
      "optimizely.com",
      "gigya.com"
    ],
    "mimetypes": [
      "application/x-shockwave-flash",
      "application/(ms)?(word|powerpoint|excel)"
    ],
    "headers": []
  }
}

The phantomjs configurations affect the PhantomJS runtime.

The blacklists block resource resource requests from PhantomJS by the resource's domain, its mime type or a header name in the request. Currently the only header value that can be blocked is the value of Accept (mime type).

To work on an existing assessment in a browser (so that breakpoints can be set in the console), run the following.

./bin/quail dev aMustHaveTitle

The following is experimental, which means it probably won't work. You can generate a new assessment with this command.

./bin/quail gen someNewAssessment

Testing

Core unit tests in Karma

./node_modules/karma/bin/karma start ./config/karma-unit.conf.js

Testing assessments in Selenium

All assessments. This takes about 10 minutes to run.

node ./test/assessmentSpecs/testRunner.js

An individual assessment. This takes about 2 seconds to run.

node ./test/assessmentSpecs/testRunner.js -I aMustHaveTitle

In a specific browser.

_BROWSER=chrome node ./test/assessmentSpecs/testRunner.js

Documentation

Full documentation is available via readthedocs.org.

Contributing

Pull requests should be made against the master branch.

We are currently looking for the following types of contritions.

1. Help with our current Milestone objectives.
2. JavaScript enthusiasts who want a chance to build out:
3. A dependency injection load pattern: Replace our load-order-depending spinup with an Inversion of Control pattern
4. A pub/sub pattern: Refactor the custom event bubbling code in the core Classes (e.g. TestCollection) to use a real event pub/sub library
5. Improve an NPM package, better build and test scripts, etc: Improve the Quail NPM package

Credits

• Quail is maintained by Jesse Renée Beach
• The Kwaliteitsinstituut Nederlandse Gemeenten (KING), who have funded many developer hours.
• The CKEditor development team for their contributions.

References

HTML Test Suite for WCAG 2.0, Sorted by Guideline

Legal

QUAIL is covered under the MIT License, and is copyright (c) 2015 by Jesse Beach. Current license is at http://quailjs.org/license.

quail's Issues

Need to deal with framesets

Because of the scoping of frames within jQuery all frame tests are faililng.

Web subtitle support

When checking for subtitles, please follow:

Youtube API
WebVTT - http://dev.w3.org/html5/webvtt/
JS options like:
http://www.storiesinflight.com/js_videosub/index.html
http://www.bluishcoder.co.nz/2008/12/srt-subtitles-with-html5-video.html

Create wiki that links OAC file names to test numbers

Test for WCAG technique H64: Using the title attribute of the frame and iframe elements

Create a test for the following WCAG 2.0 technique: http://www.w3.org/TR/WCAG20-TECHS/H64.html

We already have a test for frames, just need one for iframes.

Approach to integrating with Drupal's CKEditor Module

So to try to bring this into the CKEditor module for Drupal the plan would be to add the quail plugin to the CKEditor's plugin directory sites/all/modules/contrib/ckeditor/plugins/

Edit the config file sites/all/modules/contrib/ckeditor/ckeditor.config.js

config.extraPlugins = += (config.extraPlugins ? ',quail' : 'quail' );
CKEDITOR.plugins.addExternal('quail', Drupal.settings.ckeditor.module_path + '/plugins/quail/');  
window.alert(Drupal.settings.ckeditor.module_path + '/plugins/quail/');

That still doesn't load the quail.js but this would be housed in sites/all/libraries/quail/

I think it would also be required to login to Drupal and check it as an enabled plugin admin/config/content/ckeditor/edit/Full

Is there anything I'm missing in the general approach?

placeholder for file size should be expanded

Things like "15k bytes" are caught, but not "15 kilobytes"

Add general test for layout tables

We should add another test on top of the OAC that just says No Tables For Layout Period.

Merge abbr/acronym tests

The real thing we are testing for are acronyms, abbreviations are more complicated.

Discussion: Moving from "severity" to "testability"

The term "severity" is actually inaccurate in QUAIL, as severity might be different for different standards. What we are really meaning is "testability" i.e. is this a test that is 100% accurate, mostly accurate, or just so impossible to do through automated testing that it just outlines a place where a manual reviewer should look further?

I'm proposing that we change from "severity" to "testability" - which thankfully is an actual word. I can make it also map to "severity" for API purposes, so this could happen safely in a 2.1 version.

False positive in http://drupala11y.org/

Text has appropriate contrast
html
core/modules/system/templates/html.html.twig

Seems to be fine. "Skip to Main Content is clearly visible"

That being said, according to Color Analyzer - https://chrome.google.com/webstore/detail/color-contrast-analyzer/dagdlcijhfbmgkjokkjicnnfimlebcll

The other text in the header isn't accessible. Only outlined text is accessible.

Change license to MIT

To facilitate quail being adopted into more projects, while retaining the GPL option for downstream implementations, I would like to investigate moving to an MIT license. There doesn't see to be any issues with moving licenses even for projects that already implement quail. Thoughts?

Move to use require.js

To cut back on size and including tools not needed for most tests, start to break out to require.js modules.

Test for technique for F89: using null alt on an image where the image is the only content in a link

Create a test for wcag technique F89.

http://www.w3.org/TR/WCAG20-TECHS/F89.html

Merge label tests

All the tests about input, radio elements, etc. might need to be merged into a single test that all form elements have an associated label element. This would prevent having to create more tests for every HTML5 element, and sorta-future-proof these tests.

Img alt should only care about http/https:// images

Lots of plugins like skype insert images into content with different prefixes (i.e. "skype-ie-addon-data://"). Only images in content that have http protocol should be checked.

More than one "section clearfix" in Drupal 8 template.

There are 3 instances of class "section clearfix" in:
core/themes/bartik/templates/page.html.twig

Is it possible to get line numbers here http://drupala11y.org/

It's not clear to me that this is an issue, but want clarification about which one I'm referring to first.

Is that really your cat?

Just checked out the latest code. So neat that you're using AE as an example.

So, how do we apply this now. In AE I could see the errors and saw you just included it:
<script src="../src/quail.js"></script>

But I didn't see anyting with the simple.html version.

Great to see progress!

imgNonDecorativeHasAlt should check image size

It should be that any image over a certain size is flagged instead of any image without alt.

http://quail-lib.org/tests/any-non-decorative-images-should-have-non-empty-alt-attribute

Test for WCAG technique H65: Using the title attribute to identify form controls when the label element cannot be used

Create a test for the following WCAG 2.0 technique: http://www.w3.org/TR/WCAG20-TECHS/H65.html

Change build system, add /dist directory

To facilitate an automated build system, break out the quail /src directory into core, libs, and custom test callbacks. This will also allow wrapping the library for different contexts, like cheerio.

For the time being, final files should be compiled to both the /src and /dist directories since some projects are expecting that, but we can plan to sunset this in a future release.

Update test.json to be valid

Changing resources/guidelines/test.json to the following will make it valid and parsable;

[
"imgAttributeAltExists",
"imgAttributeAltNotFilename",
"imgAttributeAltTooLong",
"imgNonDecorativeHasAlt"
]

Discussion: Removing certain OAC tests, revisiting alignment with WCAG/508

QUAIL started off using only the Open Accessibility Check framework, but quickly outgrew those, as they did not capture common best practices, and were basically written to be implemented in PHP or Java, which meant that certain things like CSS style checking would be difficult.

I'd like to open the discussion in this issue about how we should approach tests in general in QUAIL in the future. I like the model of shipping with a ton of tests, and allowing implementers to actually choose the tests they want to use; however, a saner starting point might be helpful.

Here's my thoughts:

1. I'll put together a wiki of test-to-standard alignment. I already have most of that data, but it will help to have more community involvement, and to ensure we have proper test coverage. I'll start by focusing on WCAG 2.
2. We cull QUAIL of tests that do not necessarily align with any best practice or standard. These tests should first just be disabled by the default guidelines, then deprecated, and finally removed by version 2.5.
3. We find a better way to name tests than the kludgy-QUAIL-name system. This is going to have to be something that is done for a major 3.0 release, I'm thinking namespacing tests to the guidelines or specification, then doing something to link common tests together.

Thoughts? Discussion? I'll link to the wiki soon.

Test for technique F89: using null alt on an image where the image is the only content in a link

The test aMustContainText would capture this, but we should have a test that just fails on images within links that have no or null alt and are the only elements within a link.

Examples on http://quail-lib.org/

Just wanted to add a todo to add some examples either up on http://quail-lib.org/ or even hosted by github. Pretty sure they provide this.

Just going to play around with it a bit more in jsfiddle to see if I can get it working as I want it to with CKEditor. Have the simple one basically working here - http://jsfiddle.net/mgifford/C7N7r/

I think that the jsonPath is the line that's mucking up:
http://jsfiddle.net/mgifford/C7N7r/

test for WCAG technique H58: Using language attributes to identify changes in the human language

Check that the human language of the content of the element is the same as the inherited language for the element.

This can be done with a guesslanguage script. A guesslanguage script checks characters (chinese, japanese) used, and checks for regular words ('and', 'und', 'et' etc).

Here is one we can implement:
https://github.com/richtr/guessLanguage.js

If no guideline is provided, fallback to all provided tests.

Most users when implementing quail will be loading their own tests object, not using the full tests.json file provided with quail. In these cases, it would make senes to just use the keys of the provided tests object to define the guideline.

Allow adding additional tests and callbacks more easily

Right now, if someone wants to add their own tests, they have to patch them into the results from the tests.json file. Instead, users should be able to add more test definitions to an additional customTests option.

Options within labels should not count as readable text

If a label wraps an associated select element, the option elements within the select should not be considered readable text.

imgImportantNoSpacerAlt should use CSS width in php land

Need to add support for the width and height attribute in phpQuery.

Tags to tests.json

In order to help CMS integration, tests.json should include tags that can be used to differentiate tests. Here's a few:

1. content - This test is appropriate for checking content, as opposed to things that are usually in the or related to doctypes.
2. deprecated - Tests for things that are deprecated HTML elements, i.e. frames, blink.
3. image - Related to images
4. header - Related to headers
5. forms
6. structure - Things about page structure
7. aria - Aria-related things like appropriate roles
8. forms - Labels, inputs, etc.

accessibilityTests isn't *really* optional

According to the docs (https://quail-accessibility-information-library.readthedocs.org/en/latest/) 'AccessibilityTests' is optional. Though, if you omit it, you get js errors

"Uncaught TypeError: Cannot read property 'type' of undefined "

Line 84 in quail.js is the problem;

var testType = quail.accessibilityTests[testName].type;

I'm not sure if the docs might be out of date? I know they didnt really cover much of what was in the examples.

Add fleish-kincade readability level to jquery tests.

aMustContainText should ignore anchors

If an a tag has no "href" attribute, then it should not throw aMustContainText.

Cleanup or remove unused OAC Tests

Some OAC tests which are not being tested for anymore should have their files removed.

Change how errors are logged and reported

When a test is flagged, instead of adding it to an array, call a method which will log it in an object for later use, and then also calls any registered callbacks so they can deal with it immediately.

Comparison with HTML_CodeSniffer?

What do you see as the difference between Quail and HTML Code Sniffer?

https://github.com/squizlabs/HTML_CodeSniffer

Add karma support for testing

To test cross-browser glitches (esp. handling of ad-hoc DOM changes after page load), add Karma and grunt-karma testrunner to quail.

Missing tests

Bring back a handful of tests that were lost in the move to jQuery:

documentIsReadable
documentIsWrittenClearly
imgAltTextNotRedundant
selectJumpMenus
svgContainsTitle
textIsNotSmall
videosEmbeddedOrLinkedNeedCaptions

Tests to be sunset

This is an issue to track tests that should probably be sunset, either because they are excess fluff from OAC, or test for things that are rarely run across these days.

These should probably be moved to a separate file rather than being removed altogether. Part of me wants to just say any test with is not at least a testability greater than .1 should go away. Lots of these are tests that are too general to test for and should be broken up.

• addressForAuthorMustBeValid
• appletContainsTextEquivalentInAlt
• appletContainsTextEquivalent
• appletProvidesMechanismToReturnToParent
• appletUIMustBeAccessible
• appletsDoNotFlicker
• appletsDoneUseColorAlone
• basefontIsNotUsed
• cssDocumentMakesSenseStyleTurnedOff
• frameRelationshipsMustBeDescribed
• frameSrcIsAccessible
• framesetIsNotUsed
• frameTitlesDescribeFunction
• frameTitlesNotEmpty
• frameTitlesNotPlaceholder
• framesHaveATitle
• framesetMustHaveNoFramesSection
• imgHasLongDesc
• imgImportantNoSpacerAlt
• imgNotReferredToByColorAlone
• imgServerSideMapNotUsed
• All the "tabindex" tests
• inputElementsDontHaveAlt
• inputImageAltIdentifiesPurpose
• inputImageNotDecorative
• linkUsedForAlternateContent
• linkUsedToDescribeNavigation
• objectDoesNotFlicker
• objectLinkToMultimediaHasTextTranscript
• objectInterfaceIsAccessible
• objectMustHaveTitle
• objectMustHaveEmbed
• objectMustHaveValidTitle
• objectProvidesMechanismToReturnToParent
• objectTextUpdatesWhenObjectChanges
• objectShouldHaveLongDescription
• objectUIMustBeAccessible
• objectWithClassIDHasNoText
• scriptContentAccessibleWithScriptsTurnedOff
• scriptInBodyMustHaveNoscript
• scriptUIMustBeAccessible
• scriptsDoNotFlicker
• scriptsDoNotUseColorAlone
• selectWithOptionsHasOptgroup
• siteMap
• tabIndexFollowsLogicalOrder
• tableLayoutHasNoCaption
• tableLayoutDataShouldNotHaveTh
• tableLayoutHasNoSummary
• tableSummaryIsSufficient

Move from PHP query to querypath

Need to port CSS management over to querypath as well.

New event: pre-test fail filter

Add a new option that's a callback which allows users to basically say "no, this item is fine." While we can do this through the selector in jQuery, sometimes there are complicated use cases where pre-filtering items would make sense.

README or INSTALL for TinyMCE

I'm eager to try to test this. Curious what the install process is.

Test for appropriate th as first cell with row scope

If the first cells are bold or in other ways distinguish from other row cells consistently across all rows, it should be a th with a scope attribute set to "row".

Make dependent libraries case-insensitive.

The hasEventListener library's file name is different than the load even listener callback, causing issues with case sensitive OSes. This is per: http://drupal.org/node/2005480

Test for Assigning aria button role to a nonfocusable item

Color contrast for non-text elements

Per https://drupal.org/node/2006310

It appears that elements containing no visible text fail color contrast tests.

Test for WCAG technique H58: Using language attributes to identify changes in the human language

Create a test for the following WCAG 2.0 technique: http://www.w3.org/TR/WCAG20-TECHS/H58.html

Crawling with QUAIL

I understand QUAIL to be primarily used as a real time editing review for user generated content.

However, are there any reasons why you couldn't use it to crawl a site (there must be a Mozilla Plugin for this) and record the errors for everything related to http://example.org

Get rid of pxtoem and hasEventListener dependencies

pxtoem only has one method we need and can be included in the core file, while hasEventListener can be folded into quail itself.

Test for orphaned content n aria roles are used, but some content is not in any region