rezar / odbot

ODBot

Please take a look at our wiki before getting starting!

This is a snapshot version for Respeaker 4 Mic Hat for RPI version. Later version will use Respeaker Mic Array v2.0

Getting Started

This project uses Seeed Studio's 4 mic hat and Respeaker Mic Array v2.0. For more information on this device, refer to Seeed studio's wiki page (Product page).

http://wiki.seeedstudio.com/ReSpeaker_Mic_Array_v2.0/

Python 2.7 is used for this project and virtualenv is strongly recommended to isolate python environment.

Voice-engine is used as a voice interface and for DOA (Direction of Arrival)

CMU Sphinx and pocketsphinx is used for speech recognition.

Installation

1. Install necessary drivers
2. Install virtualenv (optional)
3. Install all dependencies
4. git clone https://github.com/Rezar/ODBot.git
5. Apply voice-engine patch
6. Modify keyphrase.list as necessary (optional)

Dependencies

• Respeaker 4mic hat - Uses spidev and gpiozero.
  • Snowboy and Google Assistant library installation is unnecessary.
• Voice-engine - Used as a voice interface
  • Voice-engine needs to be modified in order to be used with pocketsphinx in this project. Please refer to instructions below after installation.
  • Make sure the right DOA code is being used. This should be used for 4 mic hat.
• Pocketsphinx - Used for STT (Speech-To-Text)
  • Refer to CMU Sphinx for more details

Drivers

• seeed-voicecard driver - Drivers for quad AC108 ADC.

Topology

Voice-engine Audio Feed Flow

Source -> ChannelPicker -> PocketSphinx
  |
  v
 DOA

Voice Command Flow

Human Voice -> Source -> ChannelPicker -> PocketSphinx -> KWS
                                              /             \
                                            DOA            Arduino Serial Communication
											                 \
															 Arduino Command Recognizer
															

How to build the ODbot (cooking recipe)

https://drive.google.com/open?id=1q2JHpqTQjX9pdawrUT7o65O4GIdZvtEY

odbot's Issues

Issue with frame size

Originally, frame sizes were around 10 - 20ms. However, to implement KWS, frame size was increased significantly so that keywords wouldn't get cut off. This worked well with pocketsphinx KWS but since voice-engine's DOA was created to take 10~20ms of frame time, DOA is not outputting accurate results. Same goes with VAD. Although we're not using it at the moment, in order to use VAD, we need to cut this frame time.