Use case: Create virtual accessibility trees about aom

[minorninth]

Here's the other text I wrote up, which says basically the same thing. These should probably be merged.

Some authors create web interfaces using an HTML Canvas or some other direct-drawing API to render an interface instead of using native HTML elements. Currently the only way to make this type of solution accessible is by creating hidden DOM elements for accessibility, either as part of the Canvas fallback content, or just with DOM elements placed behind the visual UI in the Z-order. This is an ugly hack and it's wasteful, as there's a lot of overhead to adding DOM elements to the page - it impacts layout, style resolution, etc.

Instead, it should be possible to create virtual accessibility objects for a portion of a page's accessibility tree.

[cookiecrook]

I've invited Giorgio Natili from McGraw Hill to comment on this use case. He's implemented a pretty solid Canvas-based web app using the "Ghost DOM" approach (Giorgio's name for the technique we used to call 'Shadow DOM' before it meant web components). He's starting to hit the boundaries of tedium in that API, and requested a something like this JavaScript-only accessibility controller for Canvas or WebGL.

[minorninth]

Great! Welcome, Giorgio.

On Thu, Feb 18, 2016 at 11:03 AM James Craig [email protected]
wrote:

I've invited Giorgio Natili from McGraw Hill to comment on this use case.
He's implemented a pretty solid Canvas-based web app using the "ghost DOM"
approach (what we used to call 'shadow DOM' before it meant web
components). He's starting to hit the boundaries of tedium in that API, and
requested a something like this JavaScript-only accessibility controller
for Canvas or WebGL.

—
Reply to this email directly or view it on GitHub
#1 (comment).

[GiorgioNatili]

@cookiecrook thanks for inviting me to this discussion! @minorninth thanks! :)

Let me try to recap the main issues we are getting and then guys, let me know if you need further info.

Context
The solution currently adopted in McGraw-Hill Education uses PixiJS to render through web standards an app built with a functional programming language we own.
PixiJS, on its side, uses the canvas to provide the UI and the content through the 2d or 3d context (the central feature of our web app is to deliver textbooks to the users).
In both cases, the app navigation and the content are not exposed to any assistive technology.

Implemented Solution
Our web app presents the user interface and the content to assistive technology duplicating the rendered ones into standard HTML tags (aka GhostDom).

The main issues we are encountering are:

• Positioning
  When the user navigate through UI elements with keyboard, the web app should clearly highlight the selected element. The actual implementation uses CSS pixel precision positioning to achieve this requirement, and we are encountering some performances degradation while the screen resize. Even more, the code base is starting to be hard to be maintained across different screens and devices.
• Content
  The content is not correctly exposed to the screen readers. Often, it happens that we are not able to expose it in a linear way.
• Hidden Controls
  To avoid DOM repainting, the dev team decided to keep most of the elements in the page and then expose them to the assistive technologies.
  We are having hard time to expose and hide these elements properly.

Let me know if this is valuable for you and what else I can do to support this group.