Input overflow about porcupine

[kenarsa]

I think you are correct. The Python demo runs slow on an RPi zero. Could you try adding:

frames_per_buffer=4096,

to pa.open() call in porcupine_demo.py ?

Also, could you please provide me few pieces of information to reproduce the issue:

1- What is the sample rate for the microphone? If it is not 16000, are you resampling using an ALSA plug?
2- What is your version of pyaudio?

Thanks!

[tfontoura]

I changed frames_per_buffer up and down, but no luck.

1. No resampling, although I'm using dsnoop in order to have more than one script capturing audio, This is what I get from hw_params for the mic card:

access: MMAP_INTERLEAVED
format: S16_LE
subformat: STD
channels: 2
rate: 48000 (48000/1)
period_size: 6000
buffer_size: 18000

2. PyAudio==0.2.11
   PySoundFile==0.9.0.post1

[kenarsa]

OK. I haven't worked with dsnoop before. So this could be from my lack of knowledge ...

But from what I can see the sample rate for your HW is 48000. Porcupine consumes 16000. So there needs to be resampling happening somewhere. Otherwise, we are feeding 48000 audio into Porcupine. That could be a reason as we are doing 3 times more processing. Thoughts?

The other obvious thing we can look at is CPU usage. Can you tell me what is the average CPU usage without running the Python demon and what is it when running it?

[tfontoura]

Thanks for your help, A.
I believe snowboy (that works in the same pi, but obviously kept turned off while testing porcupine), uses the same sample rate (16000) but has no issues with same default device, both using pyaudio (would pyaudio be the responsible for rate resampling?).
CPU is fine < 34% there's no increase that causes it to stop with overflow.
If I lower the rate and sample rate to 8000 in the demo, it doesn't stop - but it doesn't recognize hot word (probably because it needs 16000 as you mentioned). If I enter 16000, it breaks with overflow. Frame buffer to > 4096 doesn't help.
Using default values, within the seconds that it is working, it did recognize hot word before crashing with overflow.

[kenarsa]

We are using pyaudio 0.2.8 and it does not resample at all. In fact, without an ALSA plugin connecting a mic with a different sampling rate will fail initialization of pyaudio for 16000 recording.

Let's do these two things:

1- How did you install pyaudio on your RPi? I am going to try to do that and see if I can reproduce the issue.
2- If you disable the call to Porcupine in the while loop

result = porcupine.process(pcm)

do you still get the error? think should help isolate the problem.

Since your CPU usage does not go up much this really shouldn't happen ....

[tfontoura]

1. Pyaudio was installed months ago, I believe we've built it, but I'm not sure. It was required by other modules at that time.
   Dsnoop alsa plugin was required because using HW device (without dsnoop) the detection locks it, not allowing access from the STT to the capture device. Actually I tried connecting HW device directly to Porcupine demo, but had the same error.
2. Removing the mentioned line caused a "not defined" error, so I changed it to result = "" #porcupine.process(pcm) but the error still occurs. Same overflow error.

[kenarsa]

OK. This is useful. Porcupine is not the reason for causing the overflow error. The demo right now is simply reading audio in chunks from PyAudio and do nothing with it. Are you using the "--output_path"? If yes maybe remove that so we don't even spend time writing into a file.

I would suggest these following steps:
1- Can you create a simple python script and read audio using PyAudio in a while loop? Does that give you the overflow error?
2- Would it be possible to try Porcupine's demo using PyAudio 0.2.8? You can install it using sudo apt-get install python3-pyaudio

Let me know how it goes.

[amiranhari]

@tfontoura Could you please pass the "stream_callback" optional argument to the pa.open() at line 90 of porcupine_demo.py and remove the while True loop instead? That would do the operation in non-blocking mode and might solve your issue. You basically need to read "porcupine.frame_length" samples from the buffer and call the porcupine.process, same as the code in the while loop.

Hope that resolves your issue.

[tfontoura]

I was able to remove the error message/throw error by changing line 104 to
pcm = audio_stream.read(porcupine.frame_length,exception_on_overflow = False)
now it doesn't stop and I can see that the real CPU usage that is~55% . It seems to be detecting hotword fine, however I have to change sensitivity to 1 (is that normal?). Not sure if detection is reliable though, will do some more testing.
The command line I'm using is:
python demo/python/porcupine_demo.py --keyword_file_path resources/keyword_files/alexa_raspberrypi.ppn --sensitivity 1

PS: @amiranhari I'm not sure what you want me to do, can you please post a small sample code?
@kenarsa : (2) I can't change pyaudio version, as it could break working things. (1) Will try that.

[kenarsa]

I think whats happening is that the buffer overflow is happening and it is just not throwing exceptions. So we are passing incorrect (chopped) audio to Porcupine. That explains why it does not work well ...

[tfontoura]

I agree, @kenarsa . I must solve the pyaudio issue.

[amiranhari]

@tfontoura I meant something like the attached diff.
rp0.txt

[tfontoura]

I applied the diff - the error is gone. However, now how do I send the audio stream to porcupine in order to detect the hotword??

[tfontoura]

Oh, I see. it calls the function _audio_callback now. No overflow error now, but no detection as well.

[amiranhari]

The callback function calls the Porcupine library to detect the keyword and is working on my Mac. I would recommend:

1. Clean your repo and apply the change and run it on your laptop and get it working.
2. If everything is working then try and run it on RP 0 and add some logs to make sure you are passing audio data to the Porcupine engine.
3. make sure the sampling rate is 16000

[tfontoura]

Ok, thanks. Setting rate=16000 hardcoded (# 3 above) did the trick, detection is working now.

[amiranhari]

Glad that it worked. Please close the issue if it's resolved.

[EgorLakomkin]

1. dsnoop

@tfontoura, I am curious how did you use dsnoop with porcupine hotword detection? Does it work with pyAudio?

[tfontoura]

@EgorLakomkin, they do work, it's just a matter of selecting the right input for pyaudio (dsnoop as the input).