byteofkathy / manuela

GT robohacks 2023

Setup

Requirements

Python 3.10 or higher is required.

This is not a tutorial for python, but if you don't have it installed, you can get it from python.org.

Installation

1. Clone the repository

2. Create a virtual environment with python -m venv venv

|⚠️ Note: Replace python with your alias for your installation of python

3. Activate the virtual environment with . ./venv/bin/activate for linux or ./venv/Scripts/activate on Windows

4. Install the requirements with pip install -r requirements.txt

5. You're done! (the test files should not be run with python, instead they should be run with pytest)

Pushing

1. Run black . to format the code

2. Run pytest to make sure the tests pass

3. Add the required dependencies with pip freeze > requirements.txt

4. Add the required dependencies for GHA with pip freeze > actionsReq.txt and take out the pyaudio dependency and any failing dependencies

5. Commit and push

manuela's Issues

Find a way to get text strings from audio input

Record some test cases to use for speech to text recognizer

Part 1

• create a live speech to text recognizer
• successfully recognize speech from a mic
• Output the speech as text

Part 2

• Record some speech test cases
• Have Manuela respond accordingly

Part 3

• Grab some pictures for testing
• Have Manuela respond accordingly

Finale

• Have Manuela respond accordingly to live input (camera + voice)
• suggestion to add a debug flag to print what the speech to text detects.

implement faces & mood feature

• read in an image
• successfully recognize a mood
• offer a suggestions based on the user's mood

Fix speech recognizer

implement text recognition and a suggestion based on the text

• successfully recognize text
• put together a list of responses to each mood
• respond correctly based on the mood detected

test webhook

update readme to include instructions to run project

Instructions to

• setup project
• run project
• push to project

Find dataset/examples for ppl faces/mood to test the face and image detection

Research what has been done & future work that the research suggests

discussion board ab it here

Take input for Manuela using microphone

Currently, input is taken through the terminal. We should integrate speech-to-text.

Find a way to detect emotion from text input

Preferably using a prebuilt package

Find Models for mood/face detection