willowtreeapps / vocable-android

Vocable for Android

Vocable AAC for Android

Empowering people to communicate with care takers and loved ones.

Pre-release builds are available through Firebase here

Contents

• What is Vocable?
• Features
• Roadmap
• Translations
• Contributing
• Requirements
• Credits
• License

What is Vocable?

Vocable AAC allows those with conditions such as MS, stroke, ALS, or spinal cord injuries to communicate using an app that tracks head movements, without the need to spend tens of thousands of dollars on technology to do so.

Features

Multimodal User Interface

Vocable uses ARCore to track the user's head movements and understand where the user is looking at on the screen. This allows the app to be used completely hands-free: users can look around the screen and make selections by lingering their gaze at a particular element.

For users with more mobility, the app can be operated by touch.

Saved Phrases

Use a list of common phrases provided by speech language pathologists, or create and save your own.

Full QWERTY Keyboard

Type with your head or your hands.

Roadmap

For the current progress on features, please visit the project board.

For a high-level roadmap, see the Vocable Roadmap

Translations

We'd love to translate Vocable into as many languages as possible. If you'd like to help translate, please visit our Crowdin project. Thanks for helping people communicate all around the world! 🌎🌍🌏

Contributing

We love contributions! To get started, please see our Contributing Guidelines.

Device Requirements

• Android devices with ARCore

Credits

Matt Kubota, Kyle Ohanian, Duncan Lewis, Ameir Al-Zoubi, and many more from WillowTree 💙.

License

vocable-android is released under the MIT license. See LICENSE for details.

Other Variants

vocable-ios is available on Apple Play Store and is also open-source.

vocable-android's Issues

Settings alert

Users need to be prompted that by selecting Privacy Policy or Contact Developers, that they will be taken outside of the app and no longer able to use the control system used within the app.

Users need the option to proceed or cancel (stay in the app).

Change dwell time

Users will have the ability to increase or decrease the dwell time from the settings. Dwell time options are:

(TBD)

"Please to meet" you should be "Pleased to meet you"

Timing and Sensitivity Screen

The timing and sensitivity screen allows users to adjust dwell time ("Hover Time") and tracking speed ("Cursor Sensitivity").

Audio feedback for navigation

Audio feedback during scrolling and button selection which can be activated or deactivated from the application settings

Alert button is cutoff (handset)

See attached screenshot

Add, edit and remove custom categories

Acceptance Criteria
Users will have the ability to edit custom categories.

1. Users can edit the name and location of custom categories.
2. Only custom category names can be edited.
3. Only custom categories can be removed or hidden. Preloaded categories can only be hidden.
   Note: The keyboard for editing category name will not have the speak button.
4. Custom categories can be added. When they are added, they are moved to the last position in the list of shown categories.

Design

Move category location

Users can move the location of the category by changing where the category lies within the list of categories in Settings.

Keyboard support for additional language characters

The keyboard will need to be able to support varying numbers of characters for different languages. This feature is for the architecture to support the different numbers, sizes and character types.

Reset

The ability to reset the app to the original default settings and content.

Multiple buttons can be pressed at once

Multiple buttons can be touched at the same time. This is mostly an issue on the keyboard screen.

Error states

Regression testing and implementation of missed error states. See https://docs.google.com/spreadsheets/d/1z2NVSfh3UTRzi1twRA6UC2xrEi4vwqIfxDplNX6f8Vo/edit#gid=0

History

Ability to view and reuse past inputs

Ability to edit custom phrases

Users will have the ability to edit previously added custom words and phrases.

Dark mode

The app UI will support a device that is set in dark mode.

Smart device integration

The ability for a user to control smart devices using the commands within the app.
Ex. "Turn off the lights" integrates with Google Home to turn off the lights in the room
According to an assistive technologist at Duke, some tools do this already by just having a category of items with "Alexa" tagged on to the beginning.

Voice integration

2 voice integrated features:

1. The ability to control using voice, possibly using numbers or some other short hand to select phrases.
2. The ability for the app to listen to conversations occurring.

Crashing with the same saying in edit mode

Number pad

Placeholder text is cut off and falls below the margin

Handset, landscape

Photo upload for button use

The ability to upload custom photos from the user's device to use for labeling buttons

Selection mode screen

Selection Mode Screen:
This is a screen within Settings that allows users to change selection method.

Move closer alert is cut off

See attached screenshots in portrait and landscape

Delete preloaded phrases

As a vocable mobile app user, I would like to be able to delete preloaded phrases for any of the preset categories.

Acceptance Criteria
Users will have the ability to remove the preloaded phrases from the application and/or replace them with custom phrases.

Categories displayed resets when switching from portrait to landscape, or landscape to portrait

STR:

1. Hold mobile device in portrait mode
2. Tap the right arrow button a few times to display a different category
3. Change view to landscape

Expected: The category visible in portrait mode is still visible in landscape

Observed: The category displayed resets to show the first one in the list, "General"

This also occurs with the page of phrases you are looking at

Spoken Translations of Phrases

Patients using the vocable app may need phrases translated into another language to assist caregivers that do not speak the patient's first language.

My Sayings settings screen

The My Sayings settings screen will allow users to add, edit and delete custom created phrases from the My Sayings Category.

Work included in this issue: Navigating to and displaying the screen. The ability to add, edit and delete are encompassed in other issues.

Onboarding

When the user launches the app for the first time, they will be guided through an instructional onboarding experience.

Edit preloaded categories

As a vocable app, I would like to be able to edit the preloaded categories

Acceptance Criteria

• Users will have the ability to edit preloaded categories from within the Settings > Categories screen
• Show the current name for the category in the edit screen
• if the user does not save the changes show a message that changes will be discarded (can we do this?)
• Add an option to save the changes

Design
Need design updates

Delete confirmation

A delete confirmation when the user deletes a phrase from my sayings.

Delete confirmation will be an actionable modal allowing the user to delete or cancel upon selection.

Use of head tracking for volume control

The ability for the end user to use head tracking as a way to control the volume of the spoken output.

Hit area seems small on alerts

The hit area on alerts seems small, making it difficult to dwell on.

FAQ's in app

As a user of the vocable mobile app, I would like to be able view FAQ's and troubleshooting tips of common app issues from within the app so that i can quickly find answers to them

Acceptance Criteria

• Add a new section within the settings of the App to view FAQ's
• When user taps on this FAQ option, show a list of common issues or FAQ's
• Tapping on any of the items in the list will display details related to the option selected
• User will be able to use a back arrow to navigate back into the list, settings

Design
Need Design updates for these screens

Content
Need to determine the list for FAQ

Localization - Support for German

As a vocable app user, I would like to be able to view and operate the vocable app in German when my device language is set to German

Acceptance Criteria

• View the preset categories, phrases in German
• Keyboard for German language phrase entry /edit support.

Support for other languages. Potentially:

Spanish
French
German
Italian

Alternative control scheme support

The ability for an end user to use head tracking or touch with different control schemes such as scanning, joystick method and more.

Please to meet you -> Pleased to meet you

Ability to rearrange presets

The ability for caregivers and end users to rearrange where the presets are presented on the screen.

My Sayings edit screen margins

TABLET: left margin is greater than right margin

Add custom phrases to My Sayings from My Sayings settings screen.

Translation

The ability for a user to submit an input in one language and present a spoken output in a different language

Delete custom phrases

Users will have the ability to delete previously created custom phrases.

For v1:

1. Users will navigate to Settings
2. Open the My Sayings menu
3. Delete option will be available for each phrase

Platform specific e-mail subject format for feedback

Email subject format is e.g. Feedback for Android Vocable 1.2.0-503

Notes/large text input

The ability for users to create notes and longer text strings (paragraphs, etc.)

Calibration

The ability for users to recalibrate when head tracking system to better align with their movements

Settings screen components

The Settings Screen will contain the following components:

1. Privacy policy
2. Contact devs link
3. Edit my sayings
4. Timing and Sensitivity
5. Selection Mode
6. Categories
7. Reset App Settings

In-app view of privacy policy and methods to contact support

The ability for users to use head tracking to view the privacy policy and contact support

Empty States

The following empty state screens have been added in figma:

• Assist Mode - Frequently Used Empty States
• Assist Mode - My Sayings Empty States
• Settings - Edit Mode - My Sayings

Pointer disappears when trying to navigate to the lower portion of the screen in portrait mode.

1. using a handset device in portrait, attempt to navigate to the arrows at the bottom of the screen.
2. The pointer disappears from the screen as the user moves their head towards the bottom of the screen.
   Expected: pointer should be able to reach the pagination tiles
   Actual: cursor is only capable of reaching the second column of the last row.

Hide and show categories

A user can choose what preloaded categories to show or hide from the setting screen. They can hide all the preloaded categories, they can hide a subset of the categories, or they can show them all.

1. Selecting to show or hide a category is done on an individual basis (i.e. a user must go into each category to show or hide; there is no option to multi-select)
2. If a category gets hidden, it gets moved to the end of the list of all categories (shown and hidden).
3. If a category goes from hidden to shown, it moves to the end of the list of categories being shown.

Letter placement within buttons

In portrait: the bottom of each letter touches the bottom of the button

In landscape: Letters are closer to the bottom of the button than the top.

This presents a major issue on devices with smaller screen sizes as the letters are cut off and can be hard to distinguish. From a Galaxy S7: