Network error while using VADTriggerAndroidASR Profile about spokestack-android

[space-pope]

Thanks for submitting an issue! Since this appears to be a compatibility issue with Android's onboard ASR, could you let us know which model of device you're testing with?

We've found the onboard ASR to have spotty compatibility thus far—Spokestack does indeed use its own AudioRecord instance to process audio from the microphone. On newer devices, this appears to work seamlessly, but go back even a few years, and there are hiccups.

Due to its hit-or-miss device support and tendency to not play nice with other libraries, we tend to recommend onboard ASR as more of a demo feature if you have a newer device to run the demo on. Google Cloud Speech and Azure Speech Services both have free tiers that could also be used for demonstration on an older device.

All that said, we're still interested in getting as much info as possible and reducing all the incompatibilities we can, so let us know what device you're using, and we'll see if there's anything we can do.

[karthikpala]

Hey @space-pope - Thanks for the detailed explanation.

I am using Samsung tab S3 (SM-T820) running on Android -9

[space-pope]

Thanks for the info! I haven't gotten to test on a Samsung device yet, so it'll be good to see if we can get this sorted. A few things to check/try:

1. Ensure the tablet has an internet connection (I know, I know; I just have to ask and make sure)
2. There's an old issue on a Cordova plugin that mentions switching the speech API. This would make sense as a possible source of the problem, as device manufacturers are allowed to implement the Android SpeechRecognizer interface themselves.
3. See if you have an offline speech recognition model, and if not, download one for English. More details on the process are on p. 31 of the manual (hope that's the right model).

NETWORK_ERROR is one I haven't seen in the wild before; with the last out-of-date device I tested on (way older than yours), I consistently got a SERVER_ERROR for what I assume was an abandoned speech recognition service. This is speculation, but I wouldn't be surprised if those 2 Google error codes are responses to different classes of HTTP response from the ASR server.

Anyway, let me know if any of these things help at all.

[karthikpala]

I have implemented the Android's ASR as a separate class and it is working fine without any error on this device.
As I was trying to implement continuous voice recognition, I came across this library which has the capability to have voice activity detection.

So here is my checklist for your questions

1. Check
2. Check - as it is working with plain implementation of speech recogniser API
3. Check

[space-pope]

Ah, got it—so this is probably similar to the microphone control issues we've seen with Pixel 1 devices, just with a different error code (those fail to capture the audio and throw a timeout error after a few seconds).

Unfortunately, I don't have any better answer for you right now. Spokestack uses SpeechRecognizer in a way that I don't think quite aligns with Google's intent for it; I've tried various ways of relinquishing control of the microphone before starting up SpeechRecognizer, but nothing has worked yet. I consider this a bug in Spokestack, but due to the things I mentioned earlier about SpeechRecognizer support across devices, it doesn't have the highest priority right now. I'll leave this one open for the time being, though, and either way update it if I learn anything new.

The goal of continuous speech recognition you mentioned falls into the same bucket of "not how Google wants you to do SpeechRecognizer things"—you can operate it semi-continuously, but I'm sure you've already run into the UX problem of the start/stop beeps and all the hacky ways of disabling them. My recommendation would be to look into a cloud ASR provider—though I know the free on-device ASR is a great selling point. Spokestack might have a different solution to this problem in the future.

[karthikpala]

Yeah I guess google doesn't want us to use continuous voice recognition using SpeechRecognizer

I haven't started working on the google cloud's speech API - but does it support offline mode without any network connection?

[space-pope]

Sadly, no; it'd be cloud-only. Offline ASR requires a full model (usually a neural one) on the device, and Google doesn't have much of an incentive to offer those for free, or even for a reasonable price, since usage would be effectively infinite.

[space-pope]

@karthikpala I know it's been awhile, but we were able to do some further troubleshooting on this and get the on-device ASR working for a bunch of devices where it was previously failing. We'd love to see if it works for your Samsung device as well if you're still working on this problem.

None of the devices we've tested on were failing with your specific error code before this fix, so there might be a separate issue in play. Since the vanilla SpeechRecognizer was working for you, though, I'm hopeful that this will fix your error as well.

The fix is in version 5.6.0. I'll go ahead and close this issue since it's been this long, but feel free to reopen if you try this out and run into a problem.