Coder Social home page Coder Social logo

alexa-voice-service-client's Introduction

Alexa Voice Service Client

code-climate-image circle-ci-image codecov-image

Python Client for Alexa Voice Service (AVS)


Installation

pip install alexa_client

or if you want to run the demos:

pip install alexa_client[demo]

Usage

File audio

from alexa_client import AlexaClient

client = AlexaClient(
    client_id='my-client-id',
    secret='my-secret',
    refresh_token='my-refresh-token',
)
client.connect()  # authenticate and other handshaking steps
with open('./tests/resources/alexa_what_time_is_it.wav', 'rb') as f:
    for i, directive in enumerate(client.send_audio_file(f)):
        if directive.name in ['Speak', 'Play']:
            with open(f'./output_{i}.mp3', 'wb') as f:
                f.write(directive.audio_attachment)

Now listen to output_0.wav and Alexa should tell you the time.

Microphone audio

import io

from alexa_client import AlexaClient
import pyaudio


def callback(in_data, frame_count, time_info, status):
    buffer.write(in_data)
    return (in_data, pyaudio.paContinue)

p = pyaudio.PyAudio()
stream = p.open(
    format=pyaudio.paInt16,
    channels=1,
    rate=16000,
    input=True,
    stream_callback=callback,
)

client = AlexaClient(
    client_id='my-client-id',
    secret='my-secret',
    refresh_token='my-refresh-token',
)

buffer = io.BytesIO()
try:
    stream.start_stream()
    print('listening. Press CTRL + C to exit.')
    client.connect()
    for i, directive in enumerate(client.send_audio_file(buffer)):
        if directive.name in ['Speak', 'Play']:
            with open(f'./output_{i}.mp3', 'wb') as f:
                f.write(directive.audio_attachment)
finally:
    stream.stop_stream()
    stream.close()
    p.terminate()

Multi-step requests

An Alexa command may relate to a previous command e.g,

[you] "Alexa, play twenty questions" [Alexa] "Is it a animal, mineral, or vegetable?" [you] "Mineral" [Alexa] "Is it valuable" [you] "No" [Alexa] "is it..."

This can be achieved by passing the same dialog request ID to multiple send_audio_file calls:

from alexa_client.alexa_client import helpers

dialog_request_id = helpers.generate_unique_id()
directives_one = client.send_audio_file(audio_one, dialog_request_id=dialog_request_id)
directives_two = client.send_audio_file(audio_two, dialog_request_id=dialog_request_id)
directives_three = client.send_audio_file(audio_three, dialog_request_id=dialog_request_id)

Run the streaming microphone audio demo to use this feature:

pip install alexa_client[demo]
python -m alexa_client.demo.streaming_microphone \
    --client-id="{enter-client-id-here}" \
    --client-secret="{enter-client-secret-here}" \
    --refresh-token="{enter-refresh-token-here}"

ASR Profiles

Automatic Speech Recognition (ASR) profiles optimized for user speech from varying distances. By default CLOSE_TALK is used but this can be specified:

from alexa_client import constants

client.send_audio_file(
    audio_file=audio_file,
    distance_profile=constants.NEAR_FIELD,  # or constants.FAR_FIELD
)

Audio format

By default PCM audio format is assumed, but OPUS can be specified:

from alexa_client import constants

client.send_audio_file(
    audio_file=audio_file,
    audio_format=constants.OPUS,
)

When PCM format is specified the audio should be 16bit Linear PCM (LPCM16), 16kHz sample rate, single-channel, and little endian.

When OPUS forat is specified the audio should be 16bit Opus, 16kHz sample rate, 32k bit rate, and little endian.

Base URL

base_url can be set to improve latency. Choose a region closest to your location.

from alexa_client.alexa_client import constants

client = AlexaClient(
    client_id='my-client-id',
    secret='my-secret',
    refresh_token='my-refresh-token',
    base_url=constants.BASE_URL_ASIA
)

The default base URL is Europe. The available constants are BASE_URL_EUROPE, BASE_URL_ASIA and BASE_URL_NORTH_AMERICA but you can pass any string if required.

Read more

Authentication

To use AVS you must first have a developer account. Then register your product here. Choose "Application" under "Is your product an app or a device"?

The client requires your client_id, secret and refresh_token:

client kwarg Notes
client_id Retrieve by clicking on the your product listed here
secret Retrieve by clicking on the your product listed here
refresh_token You must generate this. See below

Refresh token

You will need to login to Amazon via a web browser to get your refresh token.

To enable this first go here and click on your product to set some security settings under Security Profile:

setting value
Allowed Origins http://localhost:9000
Allowed Return URLs http://localhost:9000/callback/

Note what you entered for Product ID under Product Information, as this will be used as the device-type-id (case sensitive!)

Then run:

python -m alexa_client.refreshtoken.serve \
    --device-type-id="{enter-device-type-id-here}" \
    --client-id="{enter-client-id-here}" \
    --client-secret="{enter-client-secret-here}"

Follow the on-screen instructions shown at http://localhost:9000 in your web browser. On completion Amazon will return your refresh_token - which you will require to send audio or recorded voice.

Steaming audio to AVS

AlexaClient.send_audio_file streaming uploads a file-like object to AVS for great latency. The file-like object can be an actual file on your filesystem, an in-memory BytesIo buffer containing audio from your microphone, or even audio streaming from your browser over a websocket in real-time.

Persistent AVS connection

Calling AlexaClient.connect creates a persistent connection to AVS. A thread runs that pings AVS after 4 minutes of no request being made to AVS. This prevents the connection getting forcefully closed due to inactivity.

Unit test

To run the unit tests, call the following commands:

git clone [email protected]:richtier/alexa-voice-service-client.git
make test_requirements
pytest

Other projects

This library is used by alexa-browser-client, which allows you to talk to Alexa from your browser.

alexa-voice-service-client's People

Contributors

kelvinn avatar koshyviv avatar richtier avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

alexa-voice-service-client's Issues

Voice response is empty

I am using Azure text to speech to test this client. I used Azure to generate the voice of the following command with "riff-16khz-16bit-mono-pcm" output format.

"Alexa, play True or False",
"Alexa, start Song Quiz",
"Alexa, ask Baby Groot what time is it",
"Alexa, Open Eighties Letters",
"Alexa, ask Question of the Day.",
"Alexa, Play Jeopardy!",

"Alexa, play Twenty Questions.",
"Alexa, open Heads Up! and play Superstars",
"Alexa, play Geo Quiz",
"Alexa, Open Alien Simulator"

The first 6 commands return legit response.
I don't know why, but the last 4 commands always return an empty response.

This may not be related to this repo, but do you happen to know why?
I tried to change the voice volume and make Azure to speak it slower. None of this worked.

Make connection profile configurable

I need to customized the profile configurations of connection, set it to NEAR_FIELD or FAR_FIELD but can not do it on current version.
refer https://developer.amazon.com/docs/alexa-voice-service/speechrecognizer.html#profile

i fork a branch is not a better choice.

suggest to add a class member named as profile, default set to 'CLOSE_TALK', so that we can set it
https://github.com/richtier/alexa-voice-service-client/blob/master/alexa_client/alexa_client/connection.py Line 99
'payload': {
'profile': 'CLOSE_TALK',
'format': 'AUDIO_L16_RATE_16000_CHANNELS_1'
}
Line 12
From
host = 'avs-alexa-eu.amazon.com'
To
host = 'avs-alexa-eu.amazon.com'
profile = 'CLOSE_TALK'

Multi-step requests triggers fallback from the second step for the customized skill

I have a custom skill in alexa account. I have triggered it from microphone by following the instructions and executed --> alexa_client.demo.streaming_microphone along with required client parameters. I'm able to invoke the skill & getting the response for the first command, but from the second command it's triggering fallback. I tried to print the directive response to check the dialogRequestId, it is same.

I tried for "Alexa, Open twenty questions" skill too. Unable to play continuously. Connected is termination after responding to first question

Can't get payload when request using file - 'Directive' object is not subscriptable

Environment :
Python 3.6.9 on Ubuntu 16.04

I think this bug related to multipart from request. For each request I get the header name, but the payload is missing. When I print the whole directive response, it shows the payload but the Directive class can't get it

{'header': {'dialogRequestId': '....', 'messageId': .... 'name': 'RenderTemplate', 'namespace': 'TemplateRuntime'}, 'payload': {'skillIcon': ...... 'textField': 'this', 'title': {'mainTitle': 'Transcription', 'subTitle': 'Transcribe Test'}, 'token':....', 'type': 'BodyTemplate1'}}

Seems the Alexa return the payload, but the directive class can't get it

assert response.status in [http.client.NO_CONTENT, http.client.OK] AssertionError

I am pretty sure I did all steps right
still i am getting this error
Traceback (most recent call last):
File "alexa.py", line 8, in
client.connect() # authenticate and other handshaking steps
File "C:\Users\Harshit.dell-PC\AppData\Local\Programs\Python\Python36\lib\site-packages\alexa_client\alexa_client\client.py", line 33, in connect
self.synchronise_device_state()
File "C:\Users\Harshit.dell-PC\AppData\Local\Programs\Python\Python36\lib\site-packages\alexa_client\alexa_client\client.py", line 49, in synchronise_device_state
device_state=self.device_manager.get_device_state(),
File "C:\Users\Harshit.dell-PC\AppData\Local\Programs\Python\Python36\lib\site-packages\alexa_client\alexa_client\connection.py", line 74, in synchronise_device_state
assert response.status in [http.client.NO_CONTENT, http.client.OK]
AssertionError

send_audio_file returnes NoneType.

I tried your example of using audio files but I get the error:
TypeError: 'NoneType' object is not iterable

when I run the script.
Is this bug known? And if so, how can I fix it?

Simple auto file send not returning?

i installed and ran your audio file demo ... the program does not return however it does create the output mp3 file ... is this the correct or is there something wrong with my installation ?

Encounter http.client.NO_CONTENT error

Hello,

I want to use this library to generate some answer waves of a question set.
But I encounter http.client.NO_CONTENT.
I have successful generated the refresh token and register a product.
Could you give me some help on this issue? Thanks.

Traceback (most recent call last):
  File "test.py", line 8, in <module>
    alexa_client.connect()  # authenticate and other handshaking steps
  File "/home/mstts/Documents/Amazon/alexa-voice-service-client/avs_client/avs_client/client.py", line 43, in connect
    self.synchronise_device_state()
  File "/home/mstts/Documents/Amazon/alexa-voice-service-client/avs_client/avs_client/client.py", line 55, in synchronise_device_state
    device_state=self.device_manager.get_device_state(),
  File "/home/mstts/Documents/Amazon/alexa-voice-service-client/avs_client/avs_client/connection.py", line 72, in synchronise_device_state
    assert response.status == http.client.NO_CONTENT
AssertionError

Not able to receive data on downchannel stream

Hello.
I am using the library and my compliments for this, is very well done.

By the way i have a problem:
The Post stream with audio events are working pretty good.
But after i am enabling the downchannel i am not anymore able to receive event on this, is that normal?
Did anyone find any solution about this?
If i try a read on the downchannel all the program is blocked

Thanks for caring,

Alessio

Add support for initiator in ExpectSpeech directive.

Currently during the multi-turn requests, the AVS client is not sending initiator.

But for supporting multi-turn AVS expects initiator (when it's being send with previous AVS response).

From AVS documentation

If present, the initiator object included in the payload of the ExpectSpeech directive must be passed back to Alexa as the initiator object in the following Recognize event. If initiator is absent from the payload, the following Recognize event should not include initiator.

Details can be found here https://developer.amazon.com/docs/alexa-voice-service/speechrecognizer.html#expectspeech

AVS client could not install

Could not find a version that satisfies the requirement avs_client (from versions: )
No matching distribution found for avs_client

That's the error I keep getting whenever I run the install command

Client.send_audio_file returning None

When running the sample code given in the readme, i get the following error:
Traceback (most recent call last): File "/root/Desktop/python/Project-Mark/trial.py", line 12, in <module> for i, directive in enumerate(client.send_audio_file(f)): TypeError: 'NoneType' object is not iterable

Alexa Voice Service returning new response to Sync Device State

When running the streaming_microphone demo (or the Alexa Browser client), the sync device state will error out with a failed assertion. Debug inspection showed it is returning an OK response.

โ†’ python3  -m alexa_client.demo.streaming_microphone  --client-id="amzn1.application-**" --client-secret="**" --refresh-token='**'
listening. Press CTRL + C to exit.
Say something to Alexa.
Traceback (most recent call last):
  File "/anaconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/anaconda3/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/alexa-voice-service-client/alexa_client/demo/streaming_microphone.py", line 84, in <module>
    refresh_token=parsed.refresh_token,
  File "/alexa-voice-service-client/alexa_client/demo/streaming_microphone.py", line 42, in main
    alexa_client.connect()
  File "/alexa-voice-service-client/alexa_client/alexa_client/client.py", line 35, in connect
    self.synchronise_device_state()
  File "/alexa-voice-service-client/alexa_client/alexa_client/client.py", line 51, in synchronise_device_state
    device_state=self.device_manager.get_device_state(),
  File "/alexa-voice-service-client/alexa_client/alexa_client/connection.py", line 77, in synchronise_device_state
    assert response.status in ( http.client.NO_CONTENT, )
AssertionError

Problem getting refresh token

When I run
python ./avs_client/refreshtoken/serve.py \ --device-type-id=enter-device-type-id-here \ --client-id=enter-client-id-here \ --client-secret=enter-client-secret-here

I assume the --device-type-id is the "product id"
I have added "https://localhost:9000" to allowed origin and added "https://localhost:9000/callback/" to allowed return urls.

But when I run the server.py and go to http:localhost:9000.

It always give me the error.

We're sorry!
An error occurred when we tried to process your request. Rest assured, we're already working on the problem and expect to resolve it shortly.

Hide Details
Error Summary
400 Bad Request
The redirect URI you provided has not been whitelisted for your application. Please add your redirect URI in the 'Allowed Return URLs' section under 'Web Settings' for your Security Profile on Amazon Developer Portal.
Request Details
client_id=amzn1.application-oa2-client.31d6fe43e21d4554bceea25fd258e1d6
scope=alexa%3Aall
scope_data=%7B%22alexa%3Aall%22%3A+%7B%22productID%22%3A+%22bottled_joy%22%2C+%22productInstanceAttributes%22%3A+%7B%22deviceSerialNumber%22%3A+%22001%22%7D%7D%7D
response_type=code
redirect_uri=http%3A%2F%2F1.0.0.127.in-addr.arpa%3A9000%2Fcallback%2F

No user authentication

The authentication API you've provided allows you to set the client_id, secret, and refresh_token for the product credentials, but there doesn't seem to be a way to bind my Amazon user credentials to this device (as you can with an e.g., an Echo Dot).

I'm attempting to use your Python module to test not-yet-published Alexa skills. Even though I've registered the built-in app to create this Python script using the same Amazon developer account as the skill, the app doesn't see it when I query. I assume the issue is that I haven't actually logged into this new "device" using my Amazon credentials.

It's fairly easy to get an access_token and refresh_token after authenticating with Amazon via code-based linking authorization (spawning a webpage, giving it a short code, etc.). I just need some way to hook this into this API.

Can you confirm whether there's a way to do what I want?

Stream forcefully closed

I'm attempting to create an AWS lambda function that calls the .connect() method and then asks a question. When I submit a single question at a time, it appears to work. When submitting multiple requests, I'm getting a "Stream forcefully closed" error in hyper.http20 when getting the response. Is it possible for the client id, client secret and refresh token to be used by multiple connections concurrently? Any thoughts on what might cause this error?

send text to avs

How to send text to Alexa instead of audio file? Is there any way?

TypeError: 'NoneType' object is not iterable

Hi Team,
I have used amazon polly to generate the wav file and passed it to the alexa client, but it's giving me the error as

for i, directive in enumerate(client.send_audio_file(f)):
TypeError: 'NoneType' object is not iterable

Amazon polly code:
import boto3
import os

os.environ['AWS_PROFILE'] = "NitinVermaAskCli"
os.environ['AWS_DEFAULT_REGION'] = "us-east-1"
polly_client = boto3.Session(aws_access_key_id='AKIAZYOFB32NZRDUP',
aws_secret_access_key='jTuAUFmtQte4LfG0QIVuHS+E0kh1qJ3XVBmw',
region_name='us-west-1').client('polly')

response = polly_client.synthesize_speech(VoiceId='Joanna',
OutputFormat='mp3',
Text = 'Alexa, ask doctor connect')

file = open('C:/Users/nitin.verma/PycharmProjects/AwsPolly/resources/alexa_doctor_connect.wav', 'wb')
file.write(response['AudioStream'].read())
file.close()

Alexa_Client
from alexa_client import AlexaClient

client = AlexaClient(
client_id='amzn1.application-oa2-client.ce399dfcf5cf8b66c5d0c4cd1b03',
secret='53a7c2f016764402f6c4bcf53ce7e8a0f4b9a37dfa46bf2fb31e3ccf099',
refresh_token='Atzr|IwEBIEiWGBIY5fgZPrjzbdLgJfGdLGYHHInC6gDOTgTTGuCMM2H9aQWQ-7MZFn9NUmZXn-B_IHrRhG1WAi4geiE6PbohWIY81cMtK0tFAkYzkhGMDQNNXDoVMsVk1rTNr354ZopD3IQ5r1WSko8XzHLir-RskaaUERKyqEsOIEvqYlozmP6VjELnAJvglQYawtJFSf24QvmoUsFqUXdZ87eIudvWWMh-i37D25fWLQo230LzLf5D9IABshQs7nra3PFF2cacKU1Q3IL3op7738tPzxf4q32GGECsvi5lXQHk0NlYU7JgjIa-adoKSIToE9ZHV2kf2-XB5jdqlpelMBeoB6d8wH9XjXB4ZpP-RNNx9EEAaP9p7YJ04EMY',
)
client.connect() # authenticate and other handshaking steps
with open('C:/Users/nitin.verma/PycharmProjects/AwsPolly/resources/alexa_doctor_connect.wav', 'rb') as f:
for i, directive in enumerate(client.send_audio_file(f)):
if directive.name in ['Speak', 'Play']:
with open(f'./output_{i}.mp3', 'wb') as f:
f.write(directive.audio_attachment)
print('done')

Requesting you to please help me out to resolve this.

Custom skill not being invoked

I'm able to get responses for generic statements as -

What is the time now
How is the weather

But I'm not able to use my custom skill. Any idea on how i can proceed?

Thanks for the amazing library though :)

get alert and notification directives

Hello.
For example, I sent request as "countdown 10 seconds" and timer was created.
I got response as "10 seconds, starting now".
I checked that timer was created in Alexa.amazon.com.
So, I sent request and get response synchronously.
how to get other directives (asynch alert responses) ?
I don't get this directive.

AttributeError: 'NoneType' object has no attribute 'request'

Hi Team,

I tried with Multi-step requests, but I'm getting 'NoneType' object has no attribute 'request' error. Below is my code.

Code

from alexa_client import AlexaClient
from alexa_client.alexa_client import helpers

client = AlexaClient(
client_id='amzn1.application-oa2-client.ce399dfcf5c958b66c5d0c4cd1b03',
secret='53a7c2f016764402f6c4bcf53ce7e8a0f47b9a37dfa46bf2fb31e3ccf099',
refresh_token='Atzr|IwEBIEiWGBIY5fgZPrjzbdLgJfGdHInC6gDOTgTTGuCMM2H9aQWQ-7MZFn9NUmZXn-B_IHrRhG1WAi4geiE6PbohWIY81cMtK0tFAkYzkhGMDQNNXDoVMsVk1rTNr354ZopD3IQ5r1WSko8XzHLir-RskaaUERKyqEsOIEvqYlozmP6VjELnAJvglQYawtJFSf24QUsF24jYROqUXdZ87eIudvWWMh-i37D25fWLQo230LzLf5D9IABshQs7nra3PFF2cacKUcRkZ3IL3op7738tPzxf4q32GGECsvi5lXQHk0NlYU7JgjIa-adoKSIToE9ZHV2kf2-XB5jdqlpelMBeoB6d8wH9XjXB4ZpP-RNNx9EEAaP9p7YJ04EMY',
)

dialog_request_id = helpers.generate_unique_id()
directives_one = client.send_audio_file('C:/Users/nitin.verma/PycharmProjects/AwsPolly/resources/alexa_what_time_is_it.wav', dialog_request_id=dialog_request_id)
directives_two = client.send_audio_file('C:/Users/nitin.verma/PycharmProjects/AwsPolly/resources/alexa_go_to_dark_forest.wav', dialog_request_id=dialog_request_id)
directives_three = client.send_audio_file('C:/Users/nitin.verma/PycharmProjects/AwsPolly/resources/alexa_harumph.wav', dialog_request_id=dialog_request_id)

Text response from AVS

Hi,

As per the following example of yours, the response is stored as an mp3 file. I want text as a response. How can I get that ?

with open('./tests/resources/alexa_what_time_is_it.wav', 'rb') as f:
resp = client.send_audio_file(f)
for i, directive in enumerate(resp):
if directive.name in ['Speak', 'Play']:
with open(f'./output_{i}.mp3', 'wb') as f:
f.write(directive.audio_attachment)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.