microsoft / cognitive-services-sdk-react-native-example

Example repo integrating the JavaScript Speech SDK with a react native that runs on Android and iOS

cognitive-services-sdk-react-native-example

Sample repo for integrating JS Speech SDK with react-native

• clone repo
• cd cognitive-services-sdk-react-native-example\ExampleTSProject (windows)
• npx react-native run-android or npx react-native run-ios

cognitive-services-sdk-react-native-example's Issues

new NativeEventEmitter() Warning

WARN new NativeEventEmitter() was called with a non-null argument without the required removeListeners method.

I am using the Microphone continuous async example in a React Native VoiceComponent. At the top I am ignoring these logs on the VoiceComponent:

import { LogBox } from 'react-native';
LogBox.ignoreLogs(['new NativeEventEmitter', 'Require cycle']); 

When I call the VoiceComponent within another component in the return object, I get these warnings. The error is triggered when I navigate to the component with the VoiceComponent

LogBox.ignoreAllLogs(); as well does not work.

Continuous speech recognition not working? Possible causes

Issue opened for documentation.

Common causes:
Do you have developer mode on in the android device? Double check it is on
USB debugging on? just in case
Permissions granted to app? (All permission should be granted to the application)
Is the service key in?
Region typed properly? eastus not useast
Speechrecog language set right? en-US not us-en etc
Wifi is on?
Mic is on?

Error message: failed to start on an uninitialized audiostream - turn on mic permissions in settings.

doesn't work anymore can you please maintain this repo

This repo is missing important files

There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.

Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.

Merge this pull request

This repo is missing a LICENSE file

This repository is currently missing a LICENSE file.

A license helps users understand how to use your project in a compliant manner. You can find the standard MIT license Microsoft uses at: https://github.com/microsoft/repo-templates/blob/main/shared/LICENSE.

If you would like to learn more about open source licenses, please visit the document at https://aka.ms/license (Microsoft-internal guidance).

No audio output with SpeechSynthesizer

Hello,

I am trying to get SpeechSynthesizer working in React Native. I set up code as outlined in this example. It looks like the audio is running. I get the +++ onAudioEnd console log after a moment. However, I hear no sound. I tried this on the simulator and a real device and of course made sure that the volume was turned up.

const speechConfig = SpeechConfig.fromSubscription(
      this.state.subscriptionKey,
      this.state.region,
    );
    speechConfig.speechSynthesisLanguage = 'en-US';
    speechConfig.speechSynthesisVoiceName = 'en-US-JennyMultilingualNeural';

    const player = new SpeakerAudioDestination();
    player.onAudioEnd = () => {
      console.log('+++ onAudioEnd');
    };

    const audioConfigOutput = AudioConfig.fromSpeakerOutput(player);

    const synthesizer = new SpeechSynthesizer(speechConfig, audioConfigOutput);

    synthesizer.speakSsmlAsync(
      `<speak version="1.0" xmlns="http://www.w3.org/2001/10/synthesis" xmlns:mstts="https://www.w3.org/2001/mstts" xml:lang="en-US">
        <voice name="en-US-AshleyNeural">
          <mstts:express-as style="normal">
          ${'Hello, here is my question!'}
          </mstts:express-as>
        </voice>
      </speak>`,
      result => {
        if (result) {
          synthesizer.close();
          return result.audioData;
        }
        return undefined;
      },
      error => {
        synthesizer.close();
        console.log(
          '+++ error',
          new Error(`Error while saying question: ${error}`),
        );
      },
    );

This code was pasted inside the existing startRecognition() function in place of the existing code in that function.

Any ideas?

Speech to Text Freezes in Background Mode

I've encountered an issue with the Speech to Text functionality in our app. When the app is in the foreground, everything works perfectly. However, the problem arises when the app goes into the background state. The translation process seems to get stuck, and it only resumes when I bring the app back to the foreground.

Issue Details:
Speech to Text works as expected when the app is in the foreground.
However, when the app goes into the background, the translation process encounters issues.
The translation remains stuck until the app is brought back to the foreground.

Request for Assistance:
I'd like to explore the possibility of running Speech to Text even when the app is in the background. Could you please provide guidance on how to achieve this or if there's a workaround for this issue?

Your help would be greatly appreciated in resolving this matter.

Speech to text not working, showing listening

I am sharing my code below, all the permissions are set and required dependencies are imported. Whenever the app is started and button is presses, it only shows "Listening" no speech result. Please help with this

import React, {Component} from 'react';
import {SPEECH_KEY, SPEECH_REGION} from '@env';
import {
Button,
Pressable,
SafeAreaView,
ScrollView,
StatusBar,
StyleSheet,
Text,
PermissionsAndroid,
TextInput,
TouchableOpacity,
useColorScheme,
View,
} from 'react-native';
import 'react-native-get-random-values';
import 'node-libs-react-native/globals';

import {
AudioConfig,
AudioInputStream,
AudioStreamFormat,
CancellationDetails,
CancellationReason,
NoMatchDetails,
NoMatchReason,
ResultReason,
SpeechConfig,
SpeechRecognizer,
} from 'microsoft-cognitiveservices-speech-sdk';

import LiveAudioStream from 'react-native-live-audio-stream';

export default App = () => {
LiveAudioStream.init({
sampleRate: 16000,
bufferSize: 4096,
channels: 1,
bitsPerChannel: 16,
audioSource: 6,
});

const pushStream = AudioInputStream.createPushStream();

LiveAudioStream.on('data', data => {
const pcmData = Buffer.from(data, 'base64');
pushStream.write(pcmData);
});

const speechConfig = SpeechConfig.fromSubscription(SPEECH_KEY, SPEECH_REGION);
speechConfig.speechRecognitionLanguage = 'en-US';
const audioConfig = AudioConfig.fromStreamInput(
pushStream,
AudioStreamFormat.getWaveFormatPCM(16000, 16, 1),
);
const recognizer = new SpeechRecognizer(speechConfig, audioConfig);

recognizer.recognizing = (s, e) => {
console.log(RECOGNIZING: Text=${e.result.text});
};

recognizer.recognized = (s, e) => {
console.log(RECOGNIZED: Text=${e.result.text});
console.log(e.result);
}

recognizer.startContinuousRecognitionAsync();

return (
<SafeAreaView
style={{flexGrow: 1, justifyContent: 'center', alignItems: 'center'}}>
<Pressable
style={{padding: 15, backgroundColor: 'white', borderRadius: 15}}
onPress={() => {
console.log('Listening');
LiveAudioStream.start();
}}>
<Text style={{color: 'black'}}>Micstream start

<Pressable
style={{padding: 15, backgroundColor: 'white', borderRadius: 15}}
onPress={() => {
console.log('Stopping');
LiveAudioStream.stop();
}}>
<Text style={{color: 'black'}}>Micstream stop


);
};

assemble error

when i run './gradlew assemble':

Task :app:createBundleReleaseJsAndAssets FAILED

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.

• What went wrong:
  Execution failed for task ':app:createBundleReleaseJsAndAssets'.

Process 'command '/Users/zhaoweibo/zjs/cognitive-services-sdk-react-native-example/ExampleTSProject/node_modules/react-native/sdks/hermesc/osx-bin/hermesc'' finished with non-zero exit value 2
2: Task failed with an exception.

---

• What went wrong:
  java.lang.StackOverflowError (no error message)

but 'yarn run andorid' is ok