w3c / wai-atag-report-tool

Repository for ATAG Report Tool

ATAG Report Tool (ART)

With this tool, people can report on how accessible their authoring tool is. It guides them through the Success Criteria of the Authoring Tool Accessibility Guidelines 2.0 (ATAG 2.0), allowing to record a result and observation for each. It outputs to JSON and HTML.

Development

This application is built with Svelte. To run it locally, you need to clone it this repository, have Node installed and then run this in the project's directory:

npm install

This may take a while the first time, but it only needs to be done once.

Then, to build the app with Rollup, and serve it on a local dev server, run:

npm run dev

Navigate to localhost:5000. You should see your app running. Edit a component file in src, save it, and reload the page to see your changes.

Deployment

To create an optimised version of the app:

npm run build

Preview on Netlify

On merge to the main branch, a preview is created on https://wai-atag-report-tool.netlify.app.

On PR to the main branch, a Netlify PR preview is created, see the PR for the URI.

Release to W3C/WAI website

The W3C/WAI site serves what's on GitHub pages. To release to GitHub pages, create a new release. This should trigger a deploy action.

Environment-specific variables in HTML and JavaScript (.svelte)

In some deployment contexts, things like paths may need to vary. There are two steps to this:

1. Add environment specific settings to config/[environment-name].json, for example { pathPrefix: "/WAI" } or { imageDir: "/images" }
2. Before running npm run build or npm run dev, set export NODE_ENV=[environment-name]

Using in HTML

Use variables in HTML, with {{ variable-name }}, for instance {{ pathPrefix }}. If you need these non-escaped, use triple brackets, for instance {{{ pathPrefix }}}.

Mustache replaces the variables in src/index.html and places the resulting HTML in public/index.html.

Using in JavaScript

In JavaScript, __buildEnv__ is replaced with the name of the build environment using the replace plugin for rollup.

To use variables, this is how you can import the JSON file that your build environment needs:

import vars from "../../config/__buildEnv__.json";

The vars are now in the vars object, you can reference them with vars.variableName, for instance vars.pathPrefix.

Data structure

ATAG Specification

This project has a JavaScript representation of the ATAG 2.0 specification, atag.js.

This is an example of everything that can exist in this object, formatted as yaml:

principle:
  handle: Short name for principle
  text: Full text of principle
  num: Unique reference for this item, e.g. A.1
  guidelines:
   - id: “Slug” for this item, e.g. “web-based-accessible”
     num: Unique reference for this item, e.g. A.1.1
     handle: Short name
     text: Full text for this guideline
     successcriteria:
       - id: short-name
         num: Unique reference for this item, e.g. A.1.1.1
         handle: Short name
         text: Full text for this success criterion
         level: Level, e.g. “Level A”
         details:
           - type: note
             handle: Note 1
             text: Full note text
           - type: olist
             items:
               - handle: Ordered Item
                 text: Full text for item
               - handle: Ordered Item 2
                 text: Full text for item 2
           - type: ulist
              items:
                - handle: Unordered item
                  text: Full text for item
                - handle: Unordered item
                  text: Full text for item 2

Evaluation object

There is also an object that contains the current evaluation. It is based on the structure of ATAG and created in createCleanEvaluation.

Here is an example of what that object looks like, for reference:

evaluation:
  evaluationData:
    successcriterionId: 
      - id: successcriterion.id,
      - num: unique reference to the criterion (like “A.3.1”)
      - handle: short reference to the criterion (like “Keyboard Access (Minimum)”)
      - result: any of Passed, Failed, Cannot Tell, Not Applicable, Not Checked. Defaults to "Not checked".
      - observations: string of Markdown. Defaults to null.
      - level: level this criterion applies to (like “A”). Note: some can apply to any of A, AA or AAA (“multi-level SCs”).
      - evaluatedLevel: level this criterion is evaluated as. Same as level, except for multi-level SCs, where it is picked from a dropdown.
  meta:
    name:
      id: "name"
      value: null
    website: 
      id: "website",
      value: null,
    executiveSummary: 
      id: "executiveSummary"
      value: null
    evaluatorName:
      id: "evaluatorName"
      value: null
    evaluatorOrg:
      id: "evaluatorOrg"
      value: null
    evaluationId:
      id: "evaluationId"
      value: defaults to date stamp set at generation of report (`Date.now()`)
    conformanceTarget: 
      id: "conformanceTarget"
      value: defaults to "AA", used to decide which criteria in scope
    createdWith: 
      id: "createdWith",
      value: set to version number in package.json at generation of report

wai-atag-report-tool's Issues

Title/Name of tool

I see this as a placeholder:
ATAG Conformance Evaluation Report Tool (ACERT)

That seems unnecessarily complex. Also, I've heard discomfort that "ACERT" is too similar to "CERTification".

Brainstorms for other names:

• ATAG Report Tool (ART) ... (I notice this is used throughout the survey :-)
• ... (will add more)

Use simplified header

We're currently using the full WAI website header. That's a lot of links and a lot of space taking attention away from the main event, this resource may qualify for using a (yet to be determined) smaller WAI header.

See issue w3c/wai-website#195 for the wider WAI issue.

cc @nitedog

Add expand/collapse all buttons to Start page

reported by Helen and Kris Anne

"Your name" and "Your organization"

On "Your Evaluation" page, I'm not sure about the use of "your". While we want to avoid the use of passive language where we can, there are often reasons why "your" may not apply in this context. For example, I'm hired to do an evaluation for an organization not for my own.

I suggest:

• Name of evaluator (may also be the name of an organization)
• Organization (for whom the evaluation is carried out)

Also, change "About your evaluation" to "About the authoring tool" and "About you" to "About the evaluation".

PS: note misspelling "organisation" instead of "organization" (US instead of GB spelling)

Add “Not Applicable” as evaluation result

Because

some checkpoints relate to web based tools and others are application based tools

Add Tool URL to “Your Evaluation”

Is it worth (or has it been considered) adding in the URL for the tool being evaluated?

A.2.1: Explain “time based media” better (in info icon)

Suggested by Jennifer in January survey:

• location: section under “A.2.1: Make alternative content available to authors” under “Text Alternatives for Rendered Non-Text Content” under “Alternatives for Rendered Time-Based Media”
• current wording: “Time-based” media
  *suggested revision: Add an explanation or examples of “time-based media” in case users are not sure. Add this in the sentence or in the info icon?

Add information icon / tooltip to SCs

This could be a placeholder in the first release

"Your Evaluation" page name seems confusing

The name of the page "Your Evaluation" seems confusing (in the navigation and main heading). Unfortunately I don't have a better suggestion at the moment.

Render ATAG's lists and notes separately

Add Markdown support to 'Observations'

So that people can add things like links, images and lists to make their report easier to act on.

Display steps in sub navigation under “Evaluate”

Use new design component for navigation/progress

We're now reusing regular WAI sub navigation…

…but functionally, this is something different, so it warrants a design specific for that function.

Remove word ‘step’ everywhere

(reported by @nitedog)

Make explicit that tool is not automated

(reported by @slhenry)

Use visible labels for result form fields

Both for the multiple choice and the text area, use visible labels to make it clearer what information is to be filled in. (cc @nitedog)

Add final step

Final step may display results (or a placeholder), allow for downloads, etc.

Display current principle as page title

Rather than having the same page title throughout all steps, have the current step as page title. (cc @nitedog)

Some SCs show Notes button despite having no notes

For instance, A.2.1.2 in step 2

Show steps in progress bar, make them accessible from progress bar

Currently, the progress bar just shows progress.

This task is to:

• display steps (number or number plus full name if possible),
• allow user to go to a specific step by clicking on it (to allow for browsing the tool)

Add first “welcome” step

Support internationalization

I noticed that some labels/dialogs/texts are embedded in the content, which may make this tool harder to translate into other languages? Would be good to have just a single JSON or other config file that people can translate without needing to fiddle around with any source code.

A.3.2: content missing

Jennifer commented:

*location: Section under “A.3.2: Provide authors with enough time” under “Timing Adjustable”

• current wording: “The authoring tool does not include time limits or at least one of the following is true:”
• suggested wording: There is no content after the word “true”, only the word “Level A”, as if something is missing.

Add more meta fields to "Start” page

I suggest adding the following metadata as well (all optional):

• Version of authoring tool
• Release date of authoring tool
• Date of evaluation

Replace short handle in page heading with actual ATAG Principle

This is part of issues #56 and #57 but also an issue of it's own:

Most of the text comes from ATAG directly, except for the Principles, which is confusing to me. Especially the link "More details: ..." seems misleading because there is actually no more details for the Principle, just the Guidelines and Success Criteria which are included in the tool anyway.

I think it would be much clearer to simply use the ATAG Principle as is without the subsequent link, because it would be much clearer that all the text is simply from ATAG itself.

That is, instead of saying:

UI operable

More details: A.3 Editing-views are operable

Simply have:

A.3: Editing-views are operable

Use literal ATAG copy

Update page title after route

The <title> does not reflect current page except on the first one. This is a known technical limitation of the “router” component, that we can and should work around.

(related to #12)

Add expand/collapse functionality so that we can display not all of SC text

better representation of progress in the navigation bar

Wondering if there can be some sort of progress representation in the navigation bar? For example to have small triangles from one step to the next, rather than each step being just a box?

Missing w3c.json

See also https://w3c.github.io/w3c.json.html

B.1.2: Explain “transformations” and “web content transformations” better (in info icon)

Suggested by Jennifer in January survey:

location: Section under “B.1.2: Ensure that accessibility information is preserved” under “Restructuring and Recoding Transformations”

• current wording: “If the authoring tool provides restructuring transformations or re-coding transformations, and if equivalent mechanisms exist in the web content technology of the output, then at least one of the following is true:”
  *suggested wording: Can more explanation be added to the sentence or in the info icon on the definition of “transformations”?

*location: Section under “B.1.2: Ensure that accessibility information is preserved” under “Text Alternatives for Non-Text Content are Preserved”

• current wording: “If the authoring tool provides web content transformations that preserve non-text content in the output, then any text alternatives for that non-text content are also preserved, if equivalent mechanisms exist in the web content technology of the output.”
  *suggested wording: Can more explanation be added to the sentence or in the info icon on the definition of “web content transformations”?

Should we link to individual SCs?

cc @nitedog

Why?

• We have the name of number of the SC displayed anyway, so might as well save people a Google search and have it be a direct link

Why not?

• Too many links
• We already have the full text in the report tool, finding it in the spec doesn't add anything

Save evaluation locally

Using localStorage or some other mechanism, so that people don't need to do their evaluation in one sitting.

(cc @yatil)

Improve "evaluation box"

In addition to the "Your Evaluation" page, there is also a "Your evaluation" sidebar/alert. This is in itself confusing (see issue #59 for more on this topic). This issues is about the "Your evaluation" sidebar/alert alone.

On my configuration, I have to switch to full screen for it to appear on the left. It otherwise appears at the top on every single page and it is very unclear what it does. I finally pressed "new evaluation" and now it constantly just says "Set up information about your evaluation" on every single page but does not indicate how to do that setup, nor does it provide a way to remove that quite intrusive alert.

I'm not sure what the solution is but feel that there is little use of that sidebar/alert at all.

mismatch of page handles/titles

Example of current naming:

• Page title: "UI operable - ATAG Conformance Evaluation Report Tool"
• Navigation: "A.3"
• Main heading: "UI operable"
• Forward/back button: "UI operable"

I think this lack of naming association in the navigation is confusing. Also the mismatch between the titles used in this tool and ATAG itself is confusing.

Important assumption: I believe that initially the short handles like "UI operable" were created for use in the navigation but now the navigation uses the ATAG handles for the Principles like "A.3", and the short handles now seem more confusing than helpful.

Suggested renaming to improve consistency:

• Page title: "A.3 - ATAG Conformance Evaluation Report Tool"
• Navigation: "A.3"
• Main heading: "A.3: Editing-views are operable"
• Forward/back button: "A.3"

Allow submitting to Authoring Tools List

Question from Kris Anne:

once they have a conformance report will they be able to put their tool on the Authoring Tools List?

Suggestion from Kevin:

on completion ask if the user wants to submit to the Authoring Tools list

So, the suggestion is to allow users to submit to the authoring tools list directly from this report tool.

Structuring results table

Right now all results are presented in one long table. I wonder if this table should be divided into Principles at least? Maybe also Guidelines?

Use Success Criteria, not just guidelines

Include import JSON functionality

Revise content on start page

cc @nitedog

Add full WAI website header (hardcoded)

Manage focus after route

Render HTML version in the “Results” step

Just like we do in the WCAG-EM Report Tool.

cc @nitedog

Some fields not announced in NVDA

(bug reported via email by Sylvie, to be investigated)

Add CVS export support

[minor] use of "Start" in page title

Most page titles use page name followed by "ATAG Conformance Evaluation Report Tool", which makes sense for a multi-page form. However, for the first page, I wonder if we need any such prefix? Maybe "ATAG Conformance Evaluation Report Tool" is sufficient without "Start" or such?

Convert data format to use EARL with JSON-LD

See docs about use of EARL in WCAG EM Report Tool

inconsistent use of ATAG numbering

Example of current numbering:

• Navigation: "A.3"
• Main heading: "UI operable" (with reference to A.3 below that)
• Guideline: "A.3.1: Provide keyboard access to authoring features"
• Success Criteria: "Keyboard Access (Minimum)" (with references to "A.3.1.1 in later links and form labels)
• Forward/back button: "UI operable"

This inconsistent explicit and implicit use of ATAG numbering is confusing. I find myself consistently trying to process what comes from ATAG and what not, and how the different pieces of content relate to each other. In particular, the introduction of new handles (ie. "UI operable") that is not part of ATAG and inconsistently associated with A.3 (see issue #56) increases this confusion.

Suggested numbering:

• Navigation: "A.3"
• Main heading: "A.3: Editing-views are operable"
• Guideline: "A.3.1: Provide keyboard access to authoring features"
• Success Criteria: "A.3.1.1 Keyboard Access (Minimum)"
• Forward/back button: "A.3"

[minor] avoid use of "step" in the URI

We do not really have steps in this tool, so maybe should also avoid it in the URI? Maybe just use the same page name, like A.3 or A3 instead?

Add link to relevant “Implementing” from SC

(as mentioned in #22 (comment))