wicg / aom

Accessibility Object Model

The Accessibility Object Model (AOM)

This effort aims to create a JavaScript API to allow developers to modify (and eventually explore) the accessibility tree for an HTML page.

Editors:

• Alice Boxhall, Google, [email protected]
• James Craig, Apple, [email protected]
• Dominic Mazzoni, Google, [email protected]
• Alexander Surkov, Mozilla, [email protected]

Details

Read the Explainer

Read the Spec - note, not up-to-date

Can I Use AOM in Chrome, Safari, or Firefox today?

Interactive Slide Deck (for Chrome)

Tic-tac-toe demo (for Chrome)

Accessibility (ARIA) Notification API

Note notification-api.md is outdated. For the up-to-date explainer of this initiative, please click here.

This effort aims to create a JavaScript API to allow developers to initiate notifications that confirm user actions (not necessarily tied to direct UI changes).

Editors:

• Sara Tang, Microsoft, [email protected]
• Doug Geoffray, Microsoft, [email protected]
• Alison Maher, Microsoft, [email protected]
• Travis Leithead, Microsoft, [email protected]
• Daniel Libby, Microsoft

Details

Read the Explainer

aom's Issues

Consider having "labeledBy" as an alias for "labelledBy" or throwing if the wrong one is used

The US vs British spelling thing is annoying. For an API like this, you could just treat the two as aliases, or have the single-'l' version throw if someone tries to use it. The idea is to avoid the failure mode when a property set is silently ignored because only one 'l' was used.

Simple Typos in Accessibility Object Model spec

https://a11y-api.github.io/a11y-api/spec/
06 September 2016

Just a few simple typos:

• Accessiblity
• AccessibileNode
• a AccessibleNodeList ( an )
• an relationship ( x 2 )
• happend
• labeledBy ( x 2 )
• Relationship attributes that is the inverse of this one.. (can't tell if it's singular or plural)
• targer
• expore

Use case: explore accessibility API for existing document

Existing web accessibility APIs are complicated and it isn't always obvious to web authors how declarative accessibility markup affects the actual accessibility properties exposed to clients via native accessibility APIs.

One use of a scripting web accessibility API would be for a web author to explore the resulting accessibility tree for an existing web page and the properties of those objects.

Possible Use Case: Text Structure

This is similar-ish to Issue #4, which says that AT needs to be able to query focus, selection start & end, etc. in a canvas [or contenteditable] text editor.

AT also needs to be able to query which word is at a given position in the text editor.
(Not sure if "character/sentence/paragraph/etc at position" needs to be queried also, but perhaps).

This use case is best described by a recent email conversation I had with a screen reader user regarding our Code Editor widget...

Me:

I don't like the way navigating by words with ctrl+right arrow works with a screen reader, because the browser tells the screen reader where the word boundaries are, and browsers don't expect all of the special symbols that are found in code. So things like brackets, dots, equals signs, etc, cause the screen reader to repeat some words/symbols and skip others.
Here's a demo: http://libingw.github.io/OrionCodeEdit/
It works better in Firefox than in Chrome. I tried it with both NVDA and JAWS (configured to speak all punctuation).
Type up/down arrow to hear complete lines of code read correctly, and then type ctrl+right/left arrow to move by words.
(Note that you can type ctrl+M to change the tab key mode from "insert tabs" to "navigate to next tab stop" in order to get out of the editor so that it is not a keyboard trap).
Is this odd-sounding word navigation something screen reader users would expect in a web-based code editor? Do you think they would just not use it? (Since line-based navigation, typing characters, moving by character, etc work well).

Screen reader user's reply:

This is an interesting problem, that impacts so much more than code editing. You are correct, in browsers word by word navigation does not read well. For that matter, it is horrible and frustrating. This is partly why it is so difficult to just say that we will move document or spreadsheet editing into the cloud on a web based platform. Theoretically it is possible, but it is not productive. I avoid word by word navigation as much as I can, it is not worth it.
I am a JAWS user, and I also have the fortune of owning a braille display. I use my braille display to position my cursor anywhere I need on a given line, and for working with code, I would say it is mandatory. I just can't imagine working with code in any other way, especially when it comes to languages where spacing matters.

Here's what's happening in a bit more detail. Assume Firefox and IAccessible2, for example. When the user types Ctrl+right arrow, our Code Editor moves the caret forward by one variable or symbol, depending on the line of code. The ctrl+right arrow is passed through to the AT, which sends IAccessibleText::textAtOffset [1] with the caret offset and a boundary type of "word" [2] to find out what the new word in front of the caret is. Then the browser answers for us what it thinks the word is, but it is wrong because it doesn't realize that if, for example, the caret is between x and = in x= then the answer should just be = and not x=. Somehow, the browser needs to inform us that the AT is asking for a word, so that we can answer what that (code) "word" is.

I'm not quite sure what the API for this would be if it was included in the AOM. I realize that this is "Defer for now" future work, but I just wanted to put it out there for consideration.

[1] IAccessibleText::textAtOffset(offset, boundaryType) -> startOffset, endOffset, text http://accessibility.linuxfoundation.org/a11yspecs/ia2/docs/html/interface_i_accessible_text.html#a9bd84b099ac7ce2435280549e90c8827
[2] IA2_TEXT_BOUNDARY_WORD http://accessibility.linuxfoundation.org/a11yspecs/ia2/docs/html/_accessible_text_8idl.html#ad3da05c9bed9f44b1e52eb4adb2924f6

accessibleNode definition is highly imprecise

" the accessible node associated with this DOM node" is not defined anywhere. Which nodes get this? Comment nodes? When is it created---at Node creation time? When the Node is inserted into the document?

"is not currently rendered as part of a web page" is also imprecise. What about things that are offscreen? If I had to guess I'd imagine you mean that its shadow-including root is a document and that document is the active document of some browsing context.

"when called with the name of an accessibility attribute"

What happens if it is called with a string that's not the name of an accessibility attribute?

Additive Behaviors (aka control patterns, delegate protocols, behavioral mixins, etc)

There is a shared desire for additive behaviors, taking the form of multiple proposals.

1. Microsoft UIAutomation "control patterns" InvokePattern, etc. https://msdn.microsoft.com/en-us/library/ms752362(v=vs.110).aspx
2. Cocoa Protocol examples for view delegates
   • NSTableViewDataSource e.g. tableView:objectValueForTableColumn:row: https://developer.apple.com/library/mac/documentation/Cocoa/Reference/NSTableViewDelegate_Protocol/
     https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/TableView/Introduction/Introduction.html#//apple_ref/doc/uid/10000026i
   • iOS UIAccessibilityAction Protocol: e.g. accessibilityIncrement on sliders https://developer.apple.com/library/prerelease/ios/documentation/UIKit/Reference/UIAccessibilityAction_Protocol/index.html#//apple_ref/occ/instm/NSObject/accessibilityIncrement
   • OS X NSAccessibility Protocols e.g. accessibilityPerformIncrement on sliders https://developer.apple.com/library/mac/documentation/AppKit/Reference/NSAccessibilitySlider_Protocol/index.html#//apple_ref/occ/intfm/NSAccessibilitySlider/accessibilityPerformIncrement
3. General JavaScript framework behavioral mix-ins
   • Ember http://emberjs.com/api/classes/Ember.Mixin.html
   • React https://facebook.github.io/react/docs/reusable-components.html#mixins
   • Angular https://www.google.com/search?q=Angular+mixins
4. Class-less JavaScript using apply() to inherit functionality from other objects
   • http://blog.berniesumption.com/software/inheritance-is-evil-and-must-be-destroyed/
   • http://www.crockford.com/javascript/inheritance.html

The biggest challenge here is that, despite mix-ins being common in JavaScript frameworks, there is no precedent for this in DOM or another web standard, so we want to be cautious that any proposal fits well within the Web Platform.

use case: additive behaviors (aka patterns)

The best example of this is a clickable heading
<h2 onclick="doSomehting();">My Heading</h2>

If you add a role to this to tell the user it's clickable, that role overrides the heading role.
<h2 role="button" onclick="doSomehting();">My Heading</h2>

But, usually we want it to be both clickable and a heading.

The UIA concept of a Control Pattern allows adding a set of related methods, states and properties, without changing the Control Type (similar to role). In this case, we would add the Invoke pattern, which includes an Invoke method.

Here's an overview of UIA Patterns:
https://msdn.microsoft.com/en-us/library/windows/desktop/ee671194(v=vs.85).aspx

Accessibility input events probably don't belong here

I think these should still be in the main DOM tree, not the accessibility tree. This reminds me of the old "intention events" proposal, which I cannot find.

In general, telling authors to have two parallel hierarchies in which to listen for events does not make sense. And, since these events will only be triggered for UA created accessible nodes, they always have a corresponding DOM node the intention event could go to instead. The oft-used example of incrementing a slider is valuable and can work entirely in the normal DOM tree.

Use case: Events / actions from accessibility clients

One of the major gaps in ARIA is that a custom control can't be manipulated via accessibility actions.

For example, on almost all platforms, a native HTML slider can be modified by AT, typically some sort of setValue action. The browser receives the setValue on the accessible element and modifies the HTML control directly.

An ARIA slider doesn't work that way:

If AT or an automation client tries to call setValue on this object, nothing happens.

A scripting API for web accessibility could address this gap, allowing the web author to respond to these actions directly.

Events

• Finite vocabulary of a11y only events
• target and bubble in a11y tree (not capture for now)
• then fire fallback DOM event if event propagation not stopped

Need mapping of a11y events to fallback DOM event.

Extensible events is a potential future goal but explicit non-goal for V1.

Consider roleDescription for 1.0

Raising roleDescription (currently in future work) as an issue b/c "role description" is in ARIA 1.1 and implemented on Apple's iOS and macOS platforms. For example, in iWork (native), we can and sometimes do use custom role descriptions for elements like "slide" (on a group or other container role), "chart" (on an image role), etc. Without this web feature, we would not be able to achieve the same end-user result in iWork for iCloud versions of Keynote, Numbers, etc. Inclusion of this feature on any platform seems trivial once the ARIA equivalent has been implemented.

If no other editors step forth with a +1, we'll leave it in the future work section.

Accessible Node's source

The spec says:

    Let an <a>AccessibleNode</a>'s <dfn>source</dfn> be an internal
     dictionary containing attributes that the author has set.
     Attributes in an AccessibleNode's <em>source</em>, when valid,
    supercede all other rules in the computation of that attribute.

I wonder if this should be defined at part of the model. Also, internal slots are usually marked up as [[source]], as per ES spec.

Use case: Only compute accessibility attributes when needed

Note: this use case does not seem to have consensus. May be worth further discussion, but it seems to be not one that we want to use to motivate this spec.

Most users don't have accessibility support enabled most of the time. Many web developers have expressed a desire to keep accessibility attributes out of the DOM for aesthetic and performance reasons.

ARIA is sometimes quite verbose. As an example, implementing a dynamically growing list box might require the "role", "aria-setsize", "aria-posinset", and "aria-selected" attributes on each item in the list. Many authors feel this is redundant and error-prone, and being able to implement the same support via scripting would be an improvement. Sometimes the amount of ARIA in a document represents a significant fraction of the total byte size of the served HTML, and being able to represent this using scripts would be more efficient.

There can be some performance cost to keeping accessibility attributes up-to-date in the DOM, which is wasted effort if there are no clients of this information. By making it possible to implement accessibility via scripting, no accessibility information would need to be provided unless it's needed, so users without any assistive technology running wouldn't incur any performance cost.

One side effect of making this possible is that web authors would now have an accurate way of determining if accessibility is enabled. They would not necessarily know what type of assistive technology is running, if any, but I mention this just because it has been a potential privacy concern in the past. I think we shouldn't ignore this issue and instead stress that there are lots of clients of accessibility APIs including testing and automation, so developers wouldn't be able to learn very much personal information.

Formalize the "source" manipulation

Treat it as a map with keys. Right now it says things like "stores the provided value in the AccessibleNode's source." Under what key? Stores what value? You want something like

The setAttribute(attributeName, value) method performs the following steps:

1. If there is no entry in this AccessibleNode's source with key attributeName, create such an entry.
2. Set the value of the entry in this AccessibleNode's source with key attributeName to value.

For more map verbiage examples see all uses of "module map" or "when-defined promise map" in https://html.spec.whatwg.org/

1.4 Terms should be "1.4 Model"

This section really defines the highly level model that underpins the API.

Terms are randomly capitalized

In the "Terms" section, some terms are randomly capitalized (while others are normal lowercase term definitions). Please keep everything lowercase unless it is a proper noun.

"checked" under Future Work has compelling use cases

In particular, making a custom element checkbox control.

setRelation does the same thing as setAttribute

This might be related to #21 and #22. But both of them store the value in the source, which seems like it would cause collisions.

The console log for deleting the cache is in the wrong spot

The console log will always display, because it comes before the conditional check to see if it should delete this cache

Criteria for Inclusion + Criteria for Exclusion or Objection

These sections should be moved to a CONTRIBUTING.md file - they will eventually need to be removed from the spec. We can just point people to the CONTRIBUTING file.

Listening to AOM events is a potential privacy concern

We may need to ask a user's permission before a web page is allowed to listen for accessibility input events

Idea: The first time an accessibility input event is actually received by the browser, AND there's an event listener that would have caught it, we pop up a prompt asking the user if they would like that site to be allowed to listen to accessibility input events. That first event would be lost (or handled as default).

Sample test cases for 1.0

Try to keep this to a minimal scope for 1.0.

In other words, an accessible rich text editor, though desirable, is out of scope because text editing, input methods, and accessible range tracking would balloon the scope too large, too quickly.

AccessibleDOMNode : AccessibleNode is confusing

Can we please nave these things differently? It's quite confusing, particularly as all DOM nodes are by definition accessible.

Do not call public APIs from spec methods

Setting the attribute must invoke setAttribute() with the first argument being the attribute name, and the second argument the given value. Getting the attribute must invoke getAttribute() with the argument being the attribute name, and return the result.

This means if I do axNode.setAttribute = function () { throw new Error("boo"); } you must call that and rethrow the error. That's bad.

Instead, create internal algorithms ("set the accessibility attribute") which both the setter and setAttribute can refer to.

"2. Tutorial" should move to explainer.md

It is customary to have an explainer.md for tutorial-like material. We should move that stuff into such a file.

Enums in the Accessibility attribute table

If those enums are not already in other specs, please formally define them in this spec.

eg.

enum { "both", "inline", "list", "none" }

AccessibleNodeList API feedback

In general a lot of this feedback is about not using the same name as DOM APIs but with different semantics or parameters. This feedback is cross-cutting with #25.

• getNode is useless. Just have an unnamed indexed getter.
• Do not overload remove().
• Consider following the Array API more closely. If you use [LegacyArrayClass] you'll get a bunch of methods for free, using names that JS developers are more used to (e.g. push instead of append, splice instead of insertBefore/remove/replace).

labelledBy/describedBy seems bonkers

In particular, setting an AccessibleNode's labelledBy or describedBy attributes will have no effect on its accessible name or description. Set the label and description attributes directly if you want those to change.

Why even include these attributes then? At the very least make them not settable.

High-level goals

What are our high level goals, and what are their relative priorities - i.e. if they come into conflict for a particular feature, how do we trade off?

Feature detection

Discoverability (in inspector/devtools)

Consistency with ARIA

Support for unimplemented/draft attributes

Strong typing

Helps fix bad authoring

Consider allowing property setting through the constructor

var axVirtualButton = new AccessibleNode("button");
axVirtualButton.label = "Enable Audio Descriptions";
axVirtualButton.offsetWidth = 224;
axVirtualButton.offsetHeight = 72;

could become

var axVirtualButton = new AccessibleNode("button", {
  label: "Enable Audio Descriptions",
  offsetWidth: 224,
  offsetHeight: 72
});

Two ways for doing the same thing?

In relation to:

            partial interface AccessibleNode {
              attribute typename attributeName;
            };

The spec says:

Setting the attribute must invoke setAttribute() with the first argument being the attribute name, and the second argument the given value. Getting the attribute must invoke getAttribute() with the argument being the attribute name, and return the result.

This doesn't strike me as good design? There should not be two ways of doing things. If calling one invokes the other (setting the attribute calls setAttribute) then please remove one of them.

What are the use cases for changing an accessible node's role?

It seems plausible to me that they exist, but I can't think of any...

This informs a number of the existing design choices, e.g. how certain properties only work when the role is set in a certain way.

Accessing the AOM must not enable browser accessibility support

Enabling "native" browser accessibility support has performance and memory implications.

Simply setting or accessing the AOM should not do this.

This needs to be clear in the spec.

Don't use set/getAttribute for things that are not content attributes

This is overly confusing. We already have "IDL attributes" and "content attributes". We don't need "accessibility attributes".

Not sure what the right replacement word is (I guess "state" and "property" are already taken in ARIA?) but we can't overload attribute like this. I wonder if just get and set would work for the method names?

Keep process stuff out of the spec

I'd suggest keeping sections 1.2 and 1.3 in the readme or in another markdown document in the repo (maybe CONTRIBUTING.md).

Use case: Override accessible focus and selection

If a web author is going to be able to create virtual accessible objects that don't correspond to DOM elements, that implies that the author needs to be in control of what accessible element is focused, and where the text selection is.

Note that it isn't sufficient for there to simply be an accessible focus event. AT needs to be able to query what object has focus at any time. A web author should be able to return any accessible object on the page, which may not correspond to the document.activeElement. Same with selection - if text is selected inside a canvas-based text editor, the author should be able to implement functions that return the selection start and end in the accessibility tree, and other information about the selection, like its bounding box, line breaks, etc.

A few points

This is really excellent and very well explained. I think it will address the pain points currently experienced. Adding imperative programmatic access opens the AT API up nicely. I did one wonder about making Aria extendable but this goes much further (may still need custom roles).

I'm supplying comments here for explainer.md as it's mostly discussion but if you'd prefer a PR that's no prolems

• "Web apps that push the boundaries of what's possible on the web struggle to make them accessible ": them -> themselves?
• "an element might not have an accessible node if it's not ..., or if it's not currently displayed" Surely not true? The "availability of a node in other modes supported via the platform A11y API should not depend on the visual mode. There are current use cases of making extra info available to ATs that is visually hidden. The concept of Virtual AOM nodes also indicates that is so.
• "Our decision is that properties set on an accessible node are not reflected in the HTML element," I think that is probably the correct decision but is likely to cause the odd temporary confusion as devs get used to it.. The answer is probably to make it VERY explicit :) If you allows the trees to have different structures it should be less of a surprise.

Once again it's great to see this cross browser accessibility work happening! It can only benefit AT users and allows web based ATs to become more powerful too. Looking forward to playing with it.

AccessibleNode set/getAttribute and set/getRelation

These are both Map like structures, so they should perhaps be defined as:

interface AccessibleNode : EventTarget {
  attribute DOMString role;

  readonly attribute AccessibilityMap attributes;
  readonly attribute AccessibilityMap relations;

  void focus();
  void announce(DOMString message);
};

[Constructor(OpenEndedDictionary<DOMString>)]
interface AccessibilityMap {
  maplike<DOMString, DOMString>;
};

/cc @domenic

Use Case: Accessibility Notifications

Use Case: Accessibility Notifications

Currently, web authors primarily alert assistive technologies to important document change events by moving focus around. For example, new content in a single-page web application is loaded so the author forces focus there. This means ATs have to heuristically determine what happened and why it's relevant to the user. iOS has several author-supplied notifications that would be useful in the context of web accessibility.

Announcement (e.g. Speak this text) and the callback AnnouncementDidFinish
LayoutChanged
ScreenChanged (single-page web apps could trigger this when loading new content)

Relationship of Accessible Input Events and IndieUI Events?

More of a question perhaps, but: what is the relationship between Accessible Input Events and IndieUI Events https://www.w3.org/TR/indie-ui-events/ ?

I seem to remember that the scope for IndieUI Events was scaled back, but I thought the whole idea of having events relating to intent, rather than input-device-specific handlers, was going to be done by IndieUI.

What is the point of offsetParent/offsetTop/offsetLeft?

These properties seem extraordinarily out of place, and encroaching on the territory of CSS. What purpose do they serve? The spec gives no normative requirements for how UAs are supposed to interpret them.

Agenda for F2F

Web A11y API F2F

May 11 - 12, 2016
Google San Francisco
345 Spear St, San Francisco, CA 94105

Goal

Reach enough consensus to "go public" as either a working group or incubator group.

Day 1: Wednesday, May 11th, 2016

9:00
Meet at 345 Spear St, SF, 4th floor lobby, get badges, walk to meeting room.
Continental breakfast and coffee will be in the meeting room.

9:30
Introductions

IRC channel: #web-a11y-api on irc.freenode.net
Can use webchat.freenode.net to connect.

9:45
Demo of web accessibility API in Chrome

10:15
Brainstorming

• Concerns with technical details of current draft spec
• Title of spec and name of JS object
• Featured use cases
• Criteria for 1.0 inclusion

11:30
Breakouts

• Group 1: Technical spec, code snippets
• Group 2: Working group organization, criteria for inclusion, text of public announcement

12:30
Lunch @ Google

2:00
Breakouts

3:30
Summary of breakouts

4:30
Break / free time

6:00
Optional social dinner out (location TBD)

Day 2: Thursday May 12th, 2016

9:30-10:15: Basic and aspirational use cases

• Basic: must be supported in V1
• Aspirational: must not be ruled out from future revisions

10:15-11:00 Breakouts

11:00-11:20: Break

11:20-12:30: Review API

• Name of API
• Name of object
• Line by line consensus

12:30: Lunch @ Google

2:00: Next steps

• Action items
• Whether to go public (yes / no)
• Spec editors
• Title of spec and name of JS object
• Featured use cases
• Criteria for 1.0 inclusion
• Text of public announcement

3:30
Adjourn

Consider more restrictive numeric types

E.g. it seems strange that you allow negative column count, or that column counts > 2^32 wrap back to -2^32.

See also https://w3ctag.github.io/design-principles/#numeric-types

Use case: Create virtual accessibility trees

Use case: Create virtual accessibility trees...

Canvas and WebGL are the obvious candidates, but there are also examples of DOM-based views whose element order and context is arbitrary (e.g. some complex SVG-based information graphics). It'd be useful to have a JavaScript object that can respond to some or all of the accessibility API calls for the DOM element and any virtual accessibility element subtree.

Mutations and events

The spec defines:

          interface AccessibleInputEvent : Event {
          };

          interface AccessibleSetValueEvent : AccessibleInputEvent {
            attribute DOMString value;
          };

Firstly, it seems odd to just have AccessibleInputEvent as an abstract class with no attributes or methods?

Secondly, AccessibleSetValueEvent is a "mutation event", thus should be modeled in the same way as other mutations in the platform. Maybe we want mutation observers here?

Consider not creating new indexed collection types (AccessibleNodeList)

This deserves some discussion from people like @annevk. In general we are trying to move away from these unfortunate legacy types. However, this case seems hard to do otherwise.

Some alternatives I can imagine:

• Use get(0)/set(0) instead of indexed getters/setters
• Have a .nodes collection which is a FrozenArray<AccessibleNode> that returns a new value each time the child list changes
• Have .children be a FrozenArray<AccessibleNode> and move all manipulation methods to the AccessibleNode (probably suffixed with Child).

Neither of these are great.

Use case: Replace IDREF DOM content attribute associations with a JavaScript element reference

Use case: Replace IDREF content attr associations with a JavaScript element reference

Some HTML and ARIA features require reference to another element by IDREF, but it's problematic in some contexts to require the accessibility code to generate IDs. For example, some JS frameworks generated IDs automatically and make maintaining the value of those IDs purposefully obtuse. Likewise, in some extremes cases, the expectation of IDREFs requires generating thousands of DOM modifications which can lead to serious performance problems in the application.

It's be useful to associate related elements using a JavaScript object reference.

First thoughts on AccessibleElement interface

(Just copying stuff over from email as I find it easier to read in this format)

Security and privacy

Capture potential security and privacy issues in comments below.

How to handle: current proposal is to show a permission bubble along the lines of the geolocation API.