voice comparisons should be case insensitive, right now the skill responses are being changed to lower case but the expected values are not, which results in false cases of bad validation results.

When we cannot detect a skill name in the first command, we say:
Security token lacks sufficient permissions to invoke "" skill.

Instead, if the skill name we come up with is blank, we should instruct the user to use the "open skill" syntax. For example. It should say instead:
The first input should open the skill - e.g. 'open my skill'

Description

From: https://github.com/bespoken/virtual-device/issues/260

The stt flag should also be exposed via the virtual device SDK. It should be able to be set similar to the other optional flags on the virtual device:
https://github.com/bespoken/virtual-device-sdk/blob/master/src/VirtualDevice.ts#L12

Values specified in our homophone list should be matched case-insensitively.

For example, if someone defines a homophone like this:

{
  homophones: {
    "Kelvie": ["Kelly"]
  } 
}

We should replace either "kelly" or "Kelly" with Kelvie in the transcript from the virtual device. This is just a matter of setting the i flag on the regexp here:
https://github.com/bespoken/virtual-device-sdk/blob/master/src/VirtualDevice.ts#L398

SilentEcho -> VirtualDevice
SilentEchoScript -> VirtualDeviceScript
SilentEchoValidator -> VirtualDeviceValidator

Relates to https://github.com/bespoken/technical-debt/issues/2

Is your feature request related to a problem? Please describe.
Once we have an asynchronous batch process, passing a conversation id parameter allows to use an specific conversation id, this is useful to allow the use from skill testing or similar as a sync process but still be able to check the status on a separate process.

Describe the solution you'd like
Add a optional conversation id parameter to the batch_process exported function

Describe alternatives you've considered
Search by token and timestamp is not really efficient for live polling of the process status.

Additional context
Requires bespoken/virtual-device#198 is finished as a prerequisite

Overview

Create prettified HTML output as part of running a SilentEchoScript.

It should be broken down by sequence, with each test within the sequence listed.

Tests that are a success should be output with a checkmark and a green font.
Tests that are a failure should be output with an "X" mark and a red font.

Example

For the following input script:

"Open We Study Billionaires": "Welcome"
"Play": http://first.com/file.mp3
"Next": http://second.com/file.mp3

"Open We Study Billionaires": "Hi"

Output:
Overall:
4 tests, 3 succeeded, 1 failed

Time:
7/5/2017 11:02:00 UTC

Sequence 1:

Sequence 2:

Note - this is a low-fidelity mockup - please feel free to suggest improvements for the real UI.

Signature

The Validator should have a method "prettifyAsHTML(result: ValidatorResult): string"
The method takes the validator result and returns the HTML string.

For some reason, no does not get recognized for me, but "alexa no" does.

When the user enters no, we should automatically translate it to Alexa no.

To test this, try this sequence:
"tell we study billionaires to play"
"open we study billionaires"
"no"

It does not recognize the "no" in response to the yes or no question from We Study Billionaires ("Would you like to resume?").

Saying "alexa no" does work though.

The message parser, when checking for skill access, does not look for all the skill open synonyms. They are:
Launch
Tell
Ask

Also, they are being handled in a case-sensitive way - they should not be. "Open" and "open" should both be accepted.

When running a test if the token used is in trial mode and expired we need to show an specific error mentioning the expire status like we do on the skill testing ml.

Steps To Reproduce

1. Run a test with expired token
2. wait for it to finish
3. see that there is a general undefined error

Expected behavior

The error should say that the trial token is expired

Actual behavior

The error is a general undefined error

Is your feature request related to a problem? Please describe.
When using tests for Google virtual devices, the session does not end in between calls, so consecutive tests of the same interaction will fail if the google action is not exited explicitly.

Describe the solution you'd like
The virtual device endpoints have a parameter called new_conversation (must be set to true) that will force start a new conversation with the Google Assistant SDK. We should expose that parameter as we do for message, locale, voiceId and phrases for sequential validation. That way, we can set it to true on the first interaction with the virtual device, and false on the following.

As for batch validation, we should always set it to true.

Additional context
Please inform @AnkRaiza when a new release is done so that we can include this in dashboard.

Overview

The validator will take a series of tests and execute them using the SilentEcho service.

Test Structure

A test sequence is made up of an array of ValidatorTest objects.

ValidatorTest has the following properties:

• input
• comparison
• expectedTranscript [optional]
• expectedStreamURL [optional]

For now, the only value for comparison is "contains".

Signature

The signature of the Validator will require the following methods:

1. Constructor with token
   The token is used for accessing the Silent Echo virtual device
2. execute(tests: Array): ValidatorResult
   An array of tests - each sequence in the script should be turned into an array of simple objects that describe the tests. A Validator test would contain properties for:

Test Output

Test output will be in well structured array of objects, one for each test. Each test result object will have:

1. The sequence number of the tests
2. The index number of the test within the sequence
3. The absolute index of the test within the file
4. The input of the test
5. The actual output of the test (entire payload - transcript, transcriptURL, and streamURL)
6. The result of the test (success or failure)

Additionally, there should be a overall result object (ValidatorResult) that contains:

1. Success or failure of the tests overall (failure if any test fails)

Other Notes

The Validator objects are part of the definitions for the silent-echo-sdk library, and should be exported as part of Index.ts file.

Whenever I use the node.js SDK, the audio response is empty.

Right now before publishing we have to manually build the TS files to ensure the latest version of the js files are published. Adding a preversion step should avoid posible publishing errors.

**Is your feature request related to a problem? **
https://github.com/bespoken/virtual-device/issues/287

Description
Add a "screenMode" property and tests for it that will set the "screenMode" parameter when calling the virtual device process or batch process endpoints.

To close out any existing session that may have been running

We should add an "addListener" that calls back with these events:
ON_MESSAGE
Data:
phrase uttered

ON_RESULT
Data:
payload from silent echo for the call

Is your feature request related to a problem? Please describe.
There is no documentation on the latest added support for conversation

Describe the solution you'd like
documentation on async mode, how it changes the batch_process method and the new getConversationResults method

• Add session reset (that does "alexa quit")
  Should happen automatically on first call, as well as be a method
• Automatically do lowercase on transcript
• Better error handling
  Right now, just exits altogether, so not clear what went wrong
  Would be better to handle the error internally, and show a friendly message about which message failed as well as any information about why

In German the utterance Ja (yes) is not correctly processed. It works well using a longer utterance like "alexa ja". To workaroung this VD should automatically add "alexa " in front of one-word utterances.

Is your feature request related to a problem? Please describe.
I'd like to be able to control which account I'm linked to before I initiate the test.

Describe the solution you'd like
It would be great if there was an additional parameter that would let me pass in a bearer token (similar to how you can with the unit test library), that would allow me to conduct the account linking on my own through my own scripting means and then to simply pass the bearer token that I want the skill to utilize.

Describe alternatives you've considered
I'm not aware of any other way to accomplish this. In fact, I'm not even sure if it's possible.

Return the card URL in the payload

• Rename the repo: bespoken/silent-echo-sdk -> bespoken/virtual-device-sdk.
• Reconfigure CircleCi & Codecov to use new repo name: bespoken/virtual-device-sdk.

Seems we are using some >4 features. No need for this.

When a stream is returned in the streamURL field, verify that the stream is actually active.

This involves actually loading the stream and seeing that it returns data. Redirects should be followed.

If the end result is not a 200 or data being returned, it should be considered a failure.

https://circleci.com/docs/2.0/migrating-from-1-2/

Description:

When using messages and phrases like "what is rock & roll?", they are being sent to the virtual device as it is. This means, that they are interpreted as parameters and so we only receive "what is rock".

Environment:

• Version: 1.5.3

Steps To Reproduce

Steps to reproduce the behavior:

1. Use the message function to send the message "what is rock & roll"

Expected behavior

The message and phrases should be correctly encoded.

Actual behavior

The message and phrases are not correctly encoded and are sent as they are.

Additional context

This ticket was born from https://github.com/bespoken/dashboard/issues/1014
The solution should be to encode message and phrases here:

Please write tests to confirm it works.

Description:

The current implementation works only when virtual device is in an https environment. The port is hardcoded to 443:

We need to change it so that it also works with custom urls that don't use HTTPS.

Steps To Reproduce

Steps to reproduce the behavior:

1. Have an instance of virtual device running at http://localhost:3000
2. set the env variable VIRTUAL_DEVICE_BASE_URL to that url
3. do a request (from bst)

Expected behavior

The response should be returned correctly.

Actual behavior

We get an error response.

Additional context

We need to remove the hardcoded 443 port on VirtualDevice.ts and probably change the library used from https to http.

Also update bst with a new version once this is done.

the expected vs the actual response comparison while validating should be case insensitive.

@chris-ramon Can we provide an error message if the Vendor ID is invalid?

What do we get back from SMAPI if the vendor ID is bad?

Summary

Add a new class in the SDK called DeviceLocation, its constructor should receive two parameters Lat and Lng. These will correspond with the lat and lng parameters described here:

https://github.com/bespoken/virtual-device/issues/274

Use a decimal type for both parameters and add tests for it.

Hey all. Playing with the SDK and noticed that I was unable to extract the "caption" property from the IVirtualDeviceResult type as shown in the docs.

• At the moment we parse correctly the following script structure:

  "hi": "*"
  

• We want to improve the script parsing so we remove the white spaces between : character, therefore the next script must be valid:

  "hi" : "*"
  

Overview

The SilentEchoScript component provides an easy way to interact with the SilentEchoValidator. It will take a simple text file that describes a series of tests and execute them using the SilentEchoValidator service.

Format

The format of a Validator Script will be:

<Input>: <ExpectedOutput>
<Input>: <ExpectedOutput>

<Input>: <ExpectedOutput>
...

An input is a phrase in quotes:
"Hi"
"Hello There"

An expected output is either a phrase or URL:
"Hi"
http://test.com/my/audio/file.mp3

A blank line represents a new sequence of tests. Tests that are part of a sequence will be run as part of a single session.

Example

"Open We Study Billionaires": "Welcome"
"Play": http://first.com/file.mp3
"Next": http://second.com/file.mp3

"Open We Study Billionaires": "Hi"

In this test, the skill We Study Billionaires will be issued the three commands "Open", "Play" and "next" in quick succession.

The second "Open" will be sent after a pause. It will act after the first session has concluded.

The exact duration of the pause between sessions needs to be determined - we do not have precise session-handling at this point. I would suggest starting with a value of 10 seconds.

Test Rules

All tests will use an implied "contains" match. If the actual output contains the expected output, it will be considered a match.

URL comparisons will be performed against the Stream URL.

All comparisons are case-insensitive (note: this is a feature of the Validator - nothing needs to be done by the Script component to implement this).

Test Output

The test output will be the ValidatorResult object from the SilentEchoValidator

Signature

The signature of the SilentEchoScript will require the following methods:

1. Constructor with token
   The token is used for accessing the Silent Echo virtual device

2. execute(scriptContents: string)
   The contents of a test to execute

3. validate(scriptContents: string): (string | undefined)
   Run this to ensure that the test script is in a valid format. Undefined means it is valid, a string contains the error message, if there is an error.

Other Notes

The Validator objects are part of the definitions for the silent-echo-sdk library, and should be exported as part of Index.ts file.

As part of the migration to circle ci 2.0, had to skip some test because they were failing

test/VirtualDeviceTest.ts

• "Should have stream URL"
• "Should have debug info"

TBD

relates to https://github.com/bespoken/dashboard/issues/972

Per this conversation:

There is not a way to test that the session ended successfully right now when an "exit" or "quit" is included in the script. Perhaps this should be covered by *? Or an explicit sessionEnded property?

If the user specifies a line like:
"Hi": "*"
or
"Hi": ""

It should just run the command without doing any test on it (it should just show the output and automatically flag it as green).

Is your feature request related to a problem? Please describe.
We can send audio in base64 format inside the payload for our virtual-device. We want to make it easier by adding this inside this library.

Describe the solution you'd like
Our message and batch process should be able to receive an audio file, it can have two different inputs

• A local path, in this scenario we just need to convert it to base64 and generate the payload.
• A URL, in this scenario we need to download the file, convert it to base64, generate the payload and then delete the file.

Since our inputs don't have to match 1 to 1 with what we send to virtual device we can have the IMessage interface have two optional values: audioPath and audioURL, depending on which one is present we can process and convert to a payload to send to virtual device.

For the batch process, the audio should be processed in parallel.

Is your feature request related to a problem? Please describe.
https://github.com/bespoken/virtual-device/issues/285

Describe the solution you'd like
Add an optional parameter when calling the virtual device endpoint for a client. The default value should be "sdk" but it should allow overwriting that value.

Talk to Juan and Joel about this.

We want to move more things to our rancher environment.