ldw-sh-cn / emotiface

A bright and colorful game using facial recognition and emotion detection AI to have fun making faces! Entertaining for all types, and especially useful for people with autism and/or asperger's to learn and practice emotional recognition.

Emotiface (Web App/Game)

https://naeohmi.github.io/emotiface/

Table of Contents:

1. App/Game description
2. App Flow Chart
3. App Flow/Pseudo Code
4. User Stories
5. Wireframes
6. Tech Stack
7. Research on Facial Recognition/Emotion Detection
8. Screenshots from Game
9. See Project

App/Game Description:

A bright and colorful game using facial recognition and emotion detection AI to have fun making faces!

Useful and fun for people with autism and/or asperger's to learn and practice emotional recognition.

App Flow Chart:

App Flow/Pseudo Code:

1. Display opening landing page with instructions and start button
2. Player clicks start button
3. Live webcam video is turned on
4. Player makes a face conveying an extreme emotion
5. If face is found (from facial recognition detection) -> then display the score of each emotion, as returned from AI, in a pop up modal window
6. If face is NOT found (from facial recognition detection) -> then alert user the face was not found, and refresh the screen to allow user to try again

User Stories:

Target User Demographic = young people with Autism or Asperger's who want to practice reading facial expressions and have fun figuring out emotions and learning how their facial expressions might be perceived by others.

1. As a user, I want to play a game with bright fun colors and simple, easy to understand button prompts.
2. As a user, I want to take a photo of a face and see what emotions are found from that face.
3. As a user, I want to play simply for my own personal enjoyment and edification. I don't want scores or competition.
4. As a user, I want to see emoticons for each emotion checked so I can understand why that emotion was found, from the picture I took.
5. As a user, I want to know if no face was found in the picture and be able to quickly and easily take another picture.

Wire Frames:

Home/Start Landing Page:

Game In Play - Current Face (with webcam):

Tech Stack:

Core Stack:

• React
• Node
• Express
• HTML
• CSS
• jQuery
• Ajax
• Git
• PostgreSQL
• AWS

Middleware:

• React Webcam
• React-Bootstrap
• Isomorphic fetch
• Axios
• pg-promise
• bluebird

APIs:

• Cloudinary
• Kairos Facial Recognition API
• Kairos Emotion Detection API

Research on Facial Recognition/Emotion Detection:

JavaScript Libraries:

• https://github.com/auduno/clmtrackr

Emotion Detection API tools:

• Lambdal Facial Recognition API (https://lambdal.com/face-recognition-api#src)

• Microsoft Azure Cognitive Services - Emotion (https://azure.microsoft.com/en-us/services/cognitive-services/emotion/)

• Google Cloud Vision Facial Recognition AI SDK (https://cloud.google.com/vision/)

• Affectiva Emotion SDK (https://affectiva.readme.io/docs/getting-started-with-the-emotion-sdk-for-the-cloud)
• Clarifai Video Recognition AI SDK (https://developer.clarifai.com/guide/public-models#public-models)

Screenshots from Emotiface:

Play Game Page:

(current version of the display emotions and score modal)

(older version of the display score screen)

About Page:

Home Page:

End Page:

See the project here:

Live Deployment of App: https://naeohmi.github.io/emotiface/

Download Project & Install

1. Git clone or download this project
2. Make sure to npm install
3. Run npm start and app should be available on localhost:3000

Thanks :)

Go Back to Table of Contents