python demo crashed about porcupine

[kenarsa]

Could you do me a favor and tell me what is the CPU usage without running the demo and when running the demo? I think what is happening is that CPU usage goes up and audio buffer overflows. Let's verify it first.

[fquirin]

top says 1% idle 55% when Picovoice is running. I'll try to keep an eye on that.

[kenarsa]

OK. How long does it run before crashing? Is it intermittent?

[fquirin]

Unfortunately the crash message has no timestamp so I have to check this again, but it looks like its happening after around 20min. Btw I have music running in the background (for testing and entertainment purposes) so this might play a role for the audio buffer maybe....
... uups it just crashed after 8min with CPU load at 55% (at least a few seconds before).

[kenarsa]

Got it. We had this issue before on RPi zero. Could you please follow this issue: #10

I think you find the solution there.

[fquirin]

I'm not sure if I got everything right in the other issue but I've changed the python_demo code to this (starting from line 82):

num_keywords = len(self._keyword_file_paths)
		
        def _audio_callback(in_data, frame_count, time_info, status):
			if frame_count >= porcupine.frame_length:
				pcm = struct.unpack_from("h" * porcupine.frame_length, in_data)
				result = porcupine.process(pcm)
				if num_keywords == 1 and result:
					print('[%s] detected keyword' % str(datetime.now()))
				elif num_keywords > 1 and result >= 0:
					print('[%s] detected keyword #%d' % (str(datetime.now()), result))
			return None, pyaudio.paContinue

        porcupine = None
        pa = None
        audio_stream = None
        try:
            porcupine = Porcupine(
                library_path=self._library_path,
                model_file_path=self._model_file_path,
                keyword_file_paths=self._keyword_file_paths,
                sensitivities=[self._sensitivity] * num_keywords)

            pa = pyaudio.PyAudio()
            audio_stream = pa.open(
                rate=porcupine.sample_rate,
				#rate=16000,
                channels=1,
                format=pyaudio.paInt16,
                input=True,
                frames_per_buffer=porcupine.frame_length,
                input_device_index=self._input_device_index,
				stream_callback=_audio_callback)

            while True:
                pass

But I get no results anymore and when I quit the demo I get a message Segmentation fault :-(

[kenarsa]

Could you please revert the code to original status and only change the following line:

frames_per_buffer=porcupine.frame_length,

to:

frames_per_buffer=4096,

[fquirin]

running stable for an hour now ... lets see :-)

[fquirin]

I'm still having some crashes from time to time but they appear in a much more irregular pattern :-( It was running for 5h fine then crashed then crashed after 10min again then ran fine again until I quit manually.

Just a quick question, when we replace frames_per_buffer=... what do we do with the lines:

 while True:
                pcm = audio_stream.read(porcupine.frame_length)
                pcm = struct.unpack_from("h" * porcupine.frame_length, pcm)

Leave them or change them too?
I've tried to change them too on my Raspberry Pi Zero which seems to lead to no detection anymore.

[fquirin]

I took some time to better understand PyAudio and realized that I needed the non-blocking method anyways so I ran over this again and came up with a working version of my previous attempt. Here is the complete new "run" method:

        num_keywords = len(self._keyword_file_paths)
		
        def _audio_callback(in_data, frame_count, time_info, status):
            if frame_count >= porcupine.frame_length:
                pcm = struct.unpack_from("h" * porcupine.frame_length, in_data)
                result = porcupine.process(pcm)
                if num_keywords == 1 and result:
                    print('[%s] detected keyword' % str(datetime.now()))
                    # self.sepia_remote.trigger_microphone()
                elif num_keywords > 1 and result >= 0:
                    print('[%s] detected keyword #%d' % (str(datetime.now()), result))
            
            return (None, pyaudio.paContinue)

        porcupine = None
        pa = None
        audio_stream = None
        try:
            porcupine = Porcupine(
                library_path=self._library_path,
                model_file_path=self._model_file_path,
                keyword_file_paths=self._keyword_file_paths,
                sensitivities=[self._sensitivity] * num_keywords)

            pa = pyaudio.PyAudio()
            sample_rate = porcupine.sample_rate
            num_channels = 1
            audio_format = pyaudio.paInt16
            if not self.frame_length:
                frame_length = porcupine.frame_length   # frame_length = 4096
            else:
                frame_length = self.frame_length
            audio_stream = pa.open(
                rate=sample_rate,
                channels=num_channels,
                format=audio_format,
                input=True,
                frames_per_buffer=frame_length,
                input_device_index=self._input_device_index,
				stream_callback=_audio_callback)

            audio_stream.start_stream()

            print("Started porcupine with following settings:")
            if self._input_device_index:
                print("Input device: %d (check with --show_audio_devices_info)" % self._input_device_index)
            else:
                print("Input device: default (check with --show_audio_devices_info)")
            print("Sample-rate: %d" % sample_rate)
            print("Channels: %d" % num_channels)
            print("Format: %d" % audio_format)
            print("Frame-length: %d" % frame_length)
            print("Keyword file(s): %s" % self._keyword_file_paths)
            print("Waiting for keywords ...\n")

            while True:
                time.sleep(0.1)

        except KeyboardInterrupt:
            print('stopping ...')
        finally:
            if porcupine is not None:
                porcupine.delete()

            if audio_stream is not None:
                audio_stream.stop_stream()
                audio_stream.close()

            if pa is not None:
                pa.terminate()

            if self._output_path is not None and len(self._recorded_frames) > 0:
                recorded_audio = np.concatenate(self._recorded_frames, axis=0).astype(np.int16)
                soundfile.write(self._output_path, recorded_audio, samplerate=sample_rate, subtype='PCM_16')

So far it is working on my Windows and Raspberry Pi Zero, but I'll have to test it for a longer period to see if I get buffer overflows again.
Will keep you updated.

[kenarsa]

Perfect. I think this could work better on Raspberry Pi zero that has a weaker CPU compared to other variants. Quick question, did it change the response time?

Also, if you are happy with the performance and it works consistently would be great if you can add it back to the repo. I think the community would really appreciate it. If you are up for it let me know and we can discuss the best way forward. cheers!

[fquirin]

Response time seems to be unchanged and it runs pretty stable :-)
I have a fork of your repo and could do a pull request (guess I don't have the rights currently) or I could post the file here as ZIP ... whatever you prefer.

[EDIT]
Just added the file anyways ^^

porcupine_demo_non_blocking.zip

[kenarsa]

That is awesome! I think you can do a pull request. PR would be better as we can go through the review process, your name will be added to contributor list as well ;-)

Let me know if you face any problems submitting a PR and I will get to it.

[fquirin]

Sorry I was on holiday for a few days :-) Pull request was created now.

[kenarsa]

Hope you had fun. Thanks a lot. I will look into it ASAP.

[fquirin]

Changes are merged, my Zero runs stable, issue closed :-)