Comments (7)
Hi @LiamMorrow, thanks for using JS Speech SDK and writing this issue up. We use ArrayBuffer because it's a base JS type, and thus accessible without conversion for our customers using our library as a browser include. AFAIK Buffer is Node-only, correct?
from cognitive-services-speech-sdk-js.
Ah of course that makes sense for the typing, thanks for getting back to me so soon!
I guess we're a little surprised by the runtime behavior, where we wouldn't have expected that it should throw, as Buffer
has the same interface as ArrayBuffer
, and can more or less be dropped in with no issues.
Just for reference, we've attemped to tackle this by performing a runtime typecheck on the buffers returned from our streams and if they're not an ArrayBuffer
, we create a new one with the data from the original Buffer, however this seemed to blow out our memory usage significantly.
Perhaps instead of checking that the data is an instanceof ArrayBuffer
it could be more of a duck type check?
Relevant code
if (messageType === MessageType.Binary && body && !(body instanceof ArrayBuffer)) {
throw new InvalidOperationError("Payload must be ArrayBuffer");
from cognitive-services-speech-sdk-js.
Duck typing could work, Buffer instances have the byteOffset property, while ArrayBuffer instances do not...but you're not able to use the .buffer property on the buffer to get the underlying ArrayBuffer? (ala this comment)?
from cognitive-services-speech-sdk-js.
but you're not able to use the .buffer property on the buffer to get the underlying ArrayBuffer?
So while that returns the underlying ArrayBuffer
, the Buffer
we have might be just a slice over that underlying ArrayBuffer
(hence byteOffset and length).
For instance if a Buffer
is allocated, the node runtime may not provide an instance of an ArrayBuffer
, instead it may return a Buffer
that is a view over a larger ArrayBuffer
from the pool. So while buffer.size
might be for example 100
, buffer.buffer.size
could be something like 1024
, and the underlying buffer would contain garbage unrelated data.
At one stage, before we wrote the buffer into the PushStream we did do something like this (pseudocode):
ensureArrayBuffer(buf: Buffer) : ArrayBuffer{
if(buf instanceof ArrayBuffer) return buf
const underlyingBuffer = buf.buffer
if(underlyingBuffer.size === buf.size) return underlyingBuffer
const definitelyAnArrayBuffer = new ArrayBuffer(buf.size)
buf.copy(definitelyAnArrayBuffer)
return definitelyAnArrayBuffer
}
which did mean we never got this error, but as mentioned earlier it ended up blowing out our memory usage significantly (which we're not too sure why to be honest)
from cognitive-services-speech-sdk-js.
which did mean we never got this error, but as mentioned earlier it ended up blowing out our memory usage significantly (which we're not too sure why to be honest)
Ah right, Node tries to make raw data handling easier, which ends up making use cases like yours difficult. Sounds familiar.
The memory spike does sound puzzling, I'd do some A-B profiling to figure out if that new ArrayBuffer() is the culprit, and if so, naively try force deleting it after slicing it into the pushStream.
from cognitive-services-speech-sdk-js.
Ah right, Node tries to make raw data handling easier, which ends up making use cases like yours difficult. Sounds familiar.
Yeah it's a fun time.
The memory spike does sound puzzling, I'd do some A-B profiling to figure out if that new ArrayBuffer() is the culprit
Yeah at the time we were actually combating some other memory usage issues when streaming multiple hours of audio, but were able to reliably reproduce an OOM when flagging on the above code.
and if so, naively try force deleting it after slicing it into the pushStream.
Unfortunately if I'm understanding you correctly, I don't think that's feasible even if we could force a deallocation of the buffer after calling write, the PushStream is implemented as a queue (through Stream
) with as far as we can tell, no backpressure, so while write may have completed, it may not have actually been sent to azure yet.
from cognitive-services-speech-sdk-js.
The code I had in mind:
function getCopyOfArrayBuffer() {
// what you previously constructed
}
eventHandler.on('some_event', () => {
const arrayBuffer = getCopyOfArrayBuffer();
pushStream.write(arrayBuffer.slice());
} // arrayBuffer should be gc-ed here, but obv. the copy lives on while the sdk processes the audio data
from cognitive-services-speech-sdk-js.
Related Issues (20)
- Illegal Invocation Error When Using Speech SDK in Cloudflare Workers Environment HOT 5
- [Bug]: 2 Node [s] with type [Others] should not contain node [voice] with type [Media] HOT 2
- [Bug]: No way to determine when the produced audio has completed HOT 2
- [Bug]: Websocket 404 in Firefox HOT 15
- [Bug]: 3D Blendshape Data Not Generating for Super Realistic Voices HOT 8
- I'm looking for a way to adjust these threshold values depending on the country, but I haven't found any options or settings for that. HOT 2
- [Bug]: JS SpeechSDK.AudioConfig.fromDefaultMicrophoneInput capturing Teams/Zoom call speaker sounds where as JAVA SpeechSDK.AudioConfig.fromDefaultMicrophoneInput not HOT 8
- How do I get the speaker's name from SpeechSynthesizer events?
- [Bug]: SDK Crashes HOT 1
- [Bug]: Speech translation dynamic addTargetLanguage fails after no speech for 1 min HOT 2
- [Bug]: Firefox WebSocket HTTP/2 Issue: App Malfunction When Engine Started and Stopped Multiple Times HOT 8
- [Bug]: Browser Unable to Decode and Play Partial Speech Segments due to Missing Header Information HOT 1
- [Bug]: Azure Speech Recognition Not Converting Speech to Text for Chinese Language HOT 9
- [Doc]: TTS batch synthesis maximum JSON payload size HOT 1
- [Bug]: SpeakerAudioDestination > onAudioEnd does not work
- Seeking Advice on Optimizing Azure Speech Services Region Handling HOT 1
- [Bug]: SSML bookmark tag returning junk value with a specifc voice HOT 6
- [Bug]: bookmark and punctuation timestamp misaligned HOT 2
- [Bug]: Refused to create a worker from 'data:text/javascript;base64,
- [Bug]: SpeechSDK not work on Iphone's safari, but work on Mac
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cognitive-services-speech-sdk-js.