Just want to ask if there is actually a way to speed up result generation? Since --threads flag is more like to limit the threads. Is there any way or configuration that might speed up the process? Thanks man!

Greetings. I'm in love with what you created :), its astounding.

Anyway i have a small suggestion, can i somehow integrate the voice recognition with the text-to-speech using google cloud? i mean this one : https://cloud.google.com/speech-to-text/docs/languages

And i can specify which letters represent what image is displayed?

When reading a WAVE audio file, WaveFileReader performs a superfluous seek operation for every single sample. Performing this operation only when necessary results in a significant speedup.

As I know, GMOCK is only for Test purpose.
When I uncheck it from cmake, cmake configure always is error.
CMake Error at CMakeLists.txt:97 (set_target_properties):
set_target_properties Can not find target to add properties to: gmock

Wow this looks great!
Just wondering about language support. You have any plans to support other languages than English?

I'm planing to do LipSync for my game in Unity that has animations done in Spine (Esoteric Software).

Hi, I would like to use rhubarb to extract the mouth shape on the real time audio stream, but I don't know how to do this? I have downloaded the source code, but I can find the Makefile. How can I build the project by myself.

Rhubarb binary by default is installed to /usr/local/. I'm pretty sure that correct location should be /usr/local/bin/

Previous version of utf8proc could only be used as dynamic libraries. To use it as a static library, I had to patch it.

The latest version of utf8proc introduces a toggle. I should switch to this version and get rid of the patch.

I took a piece of speech and use rhubarb to extract the mouth shape and put the mouth shape change into an animation video. It seems to match for less than 50% of the speech. Would you mind take a look what's wrong? The animated video

0.00 X
0.56 C
0.79 B
1.37 D
1.57 C
......

I just put the mouth shape X from 0 to 0.56 seconds and C from 0.56 to 0.79 seconds. Is that the right way of creating the animation? I feel like the sample rate is a bit low since when the speech speed is fast many syllables are not recognized. Is there an option to adjust the sampling rate?

Thanks

I have been working to have Rhubarb help me convert an audio podcast into lip sync animation data for a podcast animation project. These podcasts all start with about 10 seconds of audio, and although some run through Rhubarb ok, most end up with a crash. I was able to isolate the problem after reviewing some debug logs to the first part of the podcast, where the music is located before the speaking begins. If I remove the first 10 seconds from the ~10 minute podcast, Rhubard works fine, and the output works very well. So, I was able to create a small testcase by clipping out the first 20 seconds of audio into a wave file which exhibits the crash. I am using Rhubarb version 1.6.0-win32. Attached is a tarball with the wav file, and the log. I am running Rhubarb like this:

./tools/rhubarb-lip-sync-1.6.0-win32/rhubarb.exe podcast3_first20s.wav --logFile podcast3_first20s.log --logLevel Debug

rhubarb_first20_crash.tar.gz

In voiceActivityDetection.cpp, the activity timeline is modified while being iterated. This results in undefined behavior. On Windows, the code seems to work; on OS X, it results in a segmentation fault.

This may be the same bug @RankoR observed in #62.

I'm now working on a project which need to grab mouth shape info from an audio. I already did the sound recognition and get the script by using sphinx. I'd like to use rhubarb to get the mouth shape but instead of taking the audio as input, I'm wondering if I can use the script which I already got from sphinx. I noticed this " -d , --dialogFile " but I don't know what's the file like. It will be helpful if you can tell more about dialogFile. Does it have time info matches with the script in dialogFile? It will be great if there's an example or more reference for the dialogFile. Any suggestions or advice will be helpful, really appreciate!

Hi.
I found your Github with the Rhubarb Lip Sync app on it and I was wondering if you could give me some advice.

I do animatronics for Cos-Play and other amateur/hobbies applications. One that I have been working on for a long time is a way to take a continuous real time speech stream from a microphone, and generate a number corresponding to the lip shape. From there a secondary processer can take this number and manipulate servo’s to give an approximation on an animatronic character.

For instance.. When they are making a movie where the main character is a werewolf consisting of an actor in the suit, and a team of puppeteers conforming the lips and facial expressions using remote control. All of this has to be carefully scripted or put in afterwards as CGI.

However take the same character and put them in a live performance situation i.e. a Cos-Play convention and you do not have that flexibility. I already have ways to pick up the actors facial movements underneath the costume. But getting a small self-contain Real Time Lip Sync is beyond me.

Could Rhubarb be compiled in Mono to run on an RPi or similar. The output being just a stream of numbers covering the lip shapes. The input would be a microphone worn by the actor.

Any suggestions would be greatly appreciated.

Many thanks
Dave

I'm using the AfterEffects script to animate a recording. When Rhubarb is done generating the animation, a new window opens with some debug output.

In rare cases, mouth animation before a dialog pause can be wrong. This is the case if the mouth ought to be opening and closing rapidly just before the pause. The optimization algorithm will then try to stretch those last lip flaps over the entire duration of the pause.

Hi do you have support for an after effects script that parses the xml to time remapped keyframes?

I'm adding Rhubarb Lip-Sync (still a WIP) support to my Game Engine:
http://docs.korge.soywiz.com/audio/lipsync

And I wanted to let you now about it, and to ask a question:

Is it ok to host the binary files of Rhubarb in one of my repositories, download them with a gradle task/intelliJ plugin and run them locally to generate lipsync files from audio files?

Also, can I use those mouth images/link Rhubarb Lip-Sync 1.0 demo video as I'm doing in the documentation?

Would it be possible to implement an output-format for MOHO? This program (former animate studio) is very often used an some other programs use the same format.
It is similar to the stanbdartd format but it uses frames instead of seconds.
The first line of a Moho Switchdatafile (*.dat) must have the content "MohoSwitch1"

best regards
Norbert Linda

Daniel,

I Like rhubarb and very grateful to you for making it open source. But I have a gripe with it, this goes for most lips animation software actually. Anyway, here it is:

The generated animation is fine for 2d cartoon, but for 3d/realistic looking subjects, it looks like watching a stop motion animation. Because transitioning from mouth shape A to B happens instantly. This makes rhubarb very limiting. It would be great, there was an option to add more mouth shapes related to each mouth shapes so that the animation looks more smooth.

Can the animation made more smooth if we used more mouth shapes? for example, for transitioning from mouth shape A to B, maybe use more mouth shapes for A, for example, lips_AB1, lispAB2, lipsAB3 . Would it work?

I am using the Spine extension. Did you write the spine extension in Java? I see that you have written the after effects extension in javascript. Since I am a javascript developer, can I modify the after effects extension to use it with Spine?

Related #28

Thank You

Hi can you please let us know how to use this in linux , i tried running the executable in linux ubuntu , but its not working ,

Hello,

Very interesting project! I am trying to use a wave file to extract phonemes as well. In your project you can extract target shapes rather than phonemes. Is it possible to modify the program to support extraction of phonemes given a phoneme set (e.g. Arpabet, IPA)?

Constantinos

Issue:
karoshi@karoshi-HP-Pavilion-15-Notebook-PC:~/LipSync/rhubarb-lip-sync/build$ ./rhubarb ../../test.wav
Analyzing input file [#-------------------] 6% -Segmentation fault (core dumped)

Input: wav file checked with avprobe.
avprobe version 11.2-6:11.2-1, Copyright (c) 2007-2014 the Libav developers
built on Jan 18 2015 05:12:33 with gcc 4.9.2 (Ubuntu 4.9.2-10ubuntu2)
[wav @ 0x1f31980] max_analyze_duration 5000000 reached
Input #0, wav, from '../../test.wav':
Duration: 00:00:09.91, bitrate: 705 kb/s
Stream #0.0: Audio: pcm_s16le, 44100 Hz, 1 channels, s16, 705 kb/s

avprobe output

When use gdb to get backtrace:

(gdb) backtrace
#0 0x00007ffff79079a0 in std::string::find_first_of(char const*, unsigned long, unsigned long) const () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#1 0x00000000007f0e41 in boost::filesystem::path::m_path_iterator_increment(boost::filesystem::path::iterator&) ()
#2 0x00000000007e98cd in boost::filesystem::detail::canonical(boost::filesystem::path const&, boost::filesystem::path const&, boost::system::error_code*) ()
#3 0x0000000000764e86 in boost::filesystem::canonical(boost::filesystem::path const&, boost::filesystem::path const&) ()
#4 0x00000000007646b1 in getBinPath() ()
#5 0x000000000077e334 in getBinaryNameabi:cxx11 ()
#6 0x000000000077e760 in NiceCmdLineOutput::printShortUsage(TCLAP::CmdLineInterface&, std::ostream&) const ()
#7 0x000000000077e62c in NiceCmdLineOutput::failure(TCLAP::CmdLineInterface&, TCLAP::ArgException&) ()
#8 0x00000000006cd4c6 in main ()

When Rhubarb Lip Sync for Spine is run from a JAR file (as opposed to a .class file), it fails to determine its binary directory. This results in the generic error message "Error performing lip sync for event ."

To fix this, a more robust may of determining the binary directory needs to be implemented.

Hi, I'm on the most recent version of OSX and the most recent downloadable binary trying to use the CLI to generate a lipsync from a Google API text to speech wav and the dialog text.
The command I'm using is:
rhubarb input.wav -f json -d input.dialog --extendedShapes GX

I'm getting this output:
Generating lip sync data for "input.wav". Progress: [#-------------------] 6% [Error] "acmod.c", line 83: Folder '/usr/local/bin/res/sphinx/acoustic-model' does not contain acoustic model definition 'mdef' Progress (cont'd): [#-------------------] 6% [Error] "acmod.c", line 83: Folder '/usr/local/bin/res/sphinx/acoustic-model' does not contain acoustic model definition 'mdef' Progress (cont'd): [#-------------------] 6% [Error] "acmod.c", line 83: Folder '/usr/local/bin/res/sphinx/acoustic-model' does not contain acoustic model definition 'mdef' Progress (cont'd): [#-------------------] 6% [Error] "acmod.c", line 83: Folder '/usr/local/bin/res/sphinx/acoustic-model' does not contain acoustic model definition 'mdef' Progress (cont'd): [#-------------------] 6% [Error] "acmod.c", line 83: Folder '/usr/local/bin/res/sphinx/acoustic-model' does not contain acoustic model definition 'mdef' Progress (cont'd): [#-------------------] 6% [Error] "acmod.c", line 83: Folder '/usr/local/bin/res/sphinx/acoustic-model' does not contain acoustic model definition 'mdef' Progress (cont'd): [#-------------------] 6% [Error] "acmod.c", line 83: Folder '/usr/local/bin/res/sphinx/acoustic-model' does not contain acoustic model definition 'mdef' Progress (cont'd): [#-------------------] 6% [Error] "acmod.c", line 83: Folder '/usr/local/bin/res/sphinx/acoustic-model' does not contain acoustic model definition 'mdef' Progress (cont'd): [#-------------------] 6% [Fatal] Application terminating with error: Error processing file "1552349650_dinosaureggsoatmeal.wav". Error performing speech recognition via PocketSphinx tools. Error creating speech decoder.
From what I can tell after researching this seems to be pocketsphinx looking in the wrong location for the acoustic model. Indeed, the referenced folder doesn't exist and homebrew puts sphinx elsewhere, but I can't seem to find any way to customize the directory for rhubarb.

I also get the same output when I start the program with -r phonetic, which I hoped might avoid calling on the sphinx acoustic model.

Any idea of the best way to direct rhubarb to the acoustic model's actual directory, or what else might be going on here?

Can we get a build with phonetic recognizer please? Latest build doesnt have that support. I tried downloading the project and building it myself, but didnt manage to get far. It kept saying there were some libraries or paths missing.

Hi,

I'm actually encounter some difficulties to use Rhubarb in After Effect.

Binary is installed etc, everything should work as expected but we've got the following error "No animation result, animation was probably canceled" : We have a mouth composition with A to F (on file per frame) and a wave audio file. Is there any step by step documentation that we could find ?

Any help will be very appreciated :)

Hi,
I'm trying the latest version of rhubarb with Magix Vegas 16, have created the xml file and selected the image file, when I press OK, I get the following Error : Image file name doesn't have expected format.
Have tried with BMP , PNG and JPG what should I do ?

Best regards

Stdout output is OK, but that is printed to stderr:

rhubarb(7885,0x7fffabb45380) malloc: *** error for object 0x627261: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug

macOS 10.3.6, build from master

Rhubarb Lip Sync uses word-based speech recognition. That works well for English dialog. For non-English dialog, however, phonetic recognition might work better. Rather than try to extract English words from non-English speech, this will extract phonemes.

I'm planning to add a CLI option to switch to phonetic recognition.

This is only a temporary solution. In the long run, I still plan to implement full (word-based) recognition for languages other than English (see #5).

The XML and JSON exporters add metadata to the output file, including the path of the audio file. If the audio file is given as a relative path on the command line, it is also written as a relative path. This means that code interpreting that information may not be able to find the original audio file.

When using the Vegas plugin on such an XML file, it responds with the (not really helpful) message "Media stream not specified."

As a fix, the exporters should always write the absolute file path.

Hello,

thanks for providing such a useful tool. I unfortunately came across a problem when using the extra/EsotericSoftwareSpine tool:

here's the underlying error tips:

KEVINXZHANG-MC4:EsotericSoftwareSpine rdm$ java -jar rhubarb-for-spine-1.9.1.jar 
objc[67391]: Class FIFinderSyncExtensionHost is implemented in both /System/Library/PrivateFrameworks/FinderKit.framework/Versions/A/FinderKit (0x7fff9ab9c1d0) and /System/Library/PrivateFrameworks/FileProvider.framework/OverrideBundles/FinderSyncCollaborationFileProviderOverride.bundle/Contents/MacOS/FinderSyncCollaborationFileProviderOverride (0x132f8fdc8). One of the two will be used. Which one is undefined.
com.rhubarb_lip_sync.rhubarb_for_spine.EndUserException: " is not a valid Shape value.
	at com.rhubarb_lip_sync.rhubarb_for_spine.RhubarbTask.call(RhubarbTask.kt:54)
	at com.rhubarb_lip_sync.rhubarb_for_spine.AudioFileModel$startAnimation$wrapperTask$1.run(AudioFileModel.kt:152)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
com.rhubarb_lip_sync.rhubarb_for_spine.EndUserException: " is not a valid Shape value.
	at com.rhubarb_lip_sync.rhubarb_for_spine.RhubarbTask.call(RhubarbTask.kt:54)
	at com.rhubarb_lip_sync.rhubarb_for_spine.AudioFileModel$startAnimation$wrapperTask$1.run(AudioFileModel.kt:152)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

And if it helps, I’ve attached the spine I'm trying to animate:

spineboy.zip

As you can see I just modified the example spineboy given by EsotericSoftware a little bit. Let me know if I need to give you more details. Thank you~

Would it be practical to add functionality to export the times at which words are spoken in the optionally provided dialogue file? This would be a really useful feature for printing text at the precise moment that a character speaks it.

Really enjoying rhubarb,
I need to use it multiple times on the same project, but everytime I import rhubarb from the scripting (Magix Vegas 16) it exits my current project and starts a new one. Any way to bypass it and have multiple instances of rhubarb running on the same project?

Best Regards

Hi. Trying to run rhubarb on blender 2.79 but it doesnt work. Ive created a pose library, uploaded a .wav audio file and when pressing Rhubarb lipsync button it pops the error: "OSError: [Errno 8] Exec format error"

Im Using Ubuntu 16.04.
Yes, User Preferences is correct

No, rhubarb does not run from command line, no additional error messages, only "Exec format error"

No additional error messages when running Blender from command line
Thanks for advise

Rhubarb is a little slow, I think its becuase the time pocketSphinx take to recognize the words in the audio file. In my project, I will use google speech to text, which will output a something like this:

[
             {
                "startTime": "1.300s",
                "endTime": "1.400s",
                "word": "Four"
              },
              {
                "startTime": "1.400s",
                "endTime": "1.600s",
                "word": "score"
              },
              {
                "startTime": "1.600s",
                "endTime": "1.600s",
                "word": "and"
              },
              {
                "startTime": "1.600s",
                "endTime": "1.900s",
                "word": "twenty"
              }
]

What file/function do I look into inside this repo to see how rhubarb converts the words to mouth shapes directly. I mean what function does rhubarb uses to convert words to shapes eg:

0.00	A
0.05	B
0.63	C
0.70	B
0.84	F
0.98	D

I looked into rhubarb-lip-sync/rhubarb/src/core/ and could not figure it out. So Far I understand that, the words are being converted to "DARPA phonetic alphabet" first and then converted to mouth Shapes. eg: AA becomes Shape A, EH becomes shape C etc..

Can you kindly provide a high level overview of what process rhubarb follows to convert words to shapes?

Thanks

Hello,
I've created small demo using the library, both with and without using dialogFile, and the end result looks like it skipping frames.
is there i possibility to specify sensitivity or mouth motion speed?
thank you very much.

Hey, thanks for the awesome work you do. However ive had some problems with breathing sounds registering as phonemes and sometimes in clear long phonemes (like holding AAH, or OOH for a long time) it can switch to other phonemes that arent there for a brief period. Can we please get some sort of adjustable sensitivity of the detection algorithm or something along those lines?

Hi this may be a noob question but I'm trying to use Rhubarb with After effects.
The directions say that there is an executable called rhubarb.exe

Do I need to compile to project to create this file?

There is a rare corner case that isn't handled correctly by the algorithm that optimizes the animation timing. As a result, Rhubarb will crash for some WAVE file with this error message (numbers may differ):

Time range start must not be less than end. Start: 115cs, end: 113cs

For some developers, working with uncompressed WAVE files may not be an option. For instance, users of Visionaire Studio typically work with Ogg Vorbis files throughout development. Similarly, Spine supports .ogg files, but Rhubarb for Spine does not.

Rhubarb should support .ogg files (Ogg Vorbis) as an alternative to .wav files.

Hi - I'm not sure is I should be asking this of you or the creator of the Blender addon...

I've downloaded and unzipped Rhubarb, and installed this blender addon:
https://github.com/scaredyfish/blender-rhubarb-lipsync
but when I come to select the .exe file I can't find one! There's a batch file, but no .exe.

Do you know of this addon?

By the way, I'm currently in the middle of creating a series of tutorials on creating simple automated lipsinc in Blender called Lipsinc for the Lazy in Blender. If I can get rhubarb working, it will make it obsolete! :S but I don't really mind. I want to get it working, and do a tutorial on it. I would also be interested in doing a skype interview if that interests you.

I am getting different results on OSX vs linux, should this be expected?
OSX is the better result.

Linux:
{
"metadata": {
"soundFile": "audio.wav",
"duration": 3.56
},
"mouthCues": [
{ "start": 0.00, "end": 0.09, "value": "X" },
{ "start": 0.09, "end": 0.30, "value": "B" },
{ "start": 0.30, "end": 0.37, "value": "C" },
{ "start": 0.37, "end": 0.44, "value": "B" },
{ "start": 0.44, "end": 0.51, "value": "C" },
{ "start": 0.51, "end": 0.72, "value": "D" },
{ "start": 0.72, "end": 0.80, "value": "C" },
{ "start": 0.80, "end": 1.12, "value": "X" },
{ "start": 1.12, "end": 1.26, "value": "B" },
{ "start": 1.26, "end": 1.54, "value": "C" },
{ "start": 1.54, "end": 2.83, "value": "X" },
{ "start": 2.83, "end": 2.89, "value": "C" },
{ "start": 2.89, "end": 3.09, "value": "G" },
{ "start": 3.09, "end": 3.23, "value": "B" },
{ "start": 3.23, "end": 3.56, "value": "X" }
]
}

OSX:
{
"metadata": {
"soundFile": "audio.wav",
"duration": 3.56
},
"mouthCues": [
{ "start": 0.00, "end": 0.09, "value": "X" },
{ "start": 0.09, "end": 0.30, "value": "B" },
{ "start": 0.30, "end": 0.37, "value": "C" },
{ "start": 0.37, "end": 0.44, "value": "B" },
{ "start": 0.44, "end": 0.51, "value": "C" },
{ "start": 0.51, "end": 0.72, "value": "D" },
{ "start": 0.72, "end": 0.80, "value": "C" },
{ "start": 0.80, "end": 1.12, "value": "X" },
{ "start": 1.12, "end": 1.26, "value": "B" },
{ "start": 1.26, "end": 1.54, "value": "C" },
{ "start": 1.54, "end": 2.30, "value": "X" },
{ "start": 2.30, "end": 2.46, "value": "B" },
{ "start": 2.46, "end": 2.60, "value": "D" },
{ "start": 2.60, "end": 2.67, "value": "B" },
{ "start": 2.67, "end": 2.88, "value": "C" },
{ "start": 2.88, "end": 3.02, "value": "B" },
{ "start": 3.02, "end": 3.56, "value": "X" }
]
}

sudo apt-get -y install libboost-all-dev fetches Boost 1.58 and there was change in some templates in Boost 1.56 thus compilation fails.

Relevant links : http://lists.boost.org/boost-users/2014/08/82693.php and https://www.facebook.com/libracode/posts/1339039419469305

[ 13%] Building CXX object CMakeFiles/rhubarb-exporters.dir/src/exporters/XmlExporter.cpp.o
In file included from /usr/include/boost/property_tree/detail/xml_parser_utils.hpp:15:0,
                 from /usr/include/boost/property_tree/detail/xml_parser_write.hpp:15,
                 from /usr/include/boost/property_tree/xml_parser.hpp:15,
                 from /home/ubuntu/rhubarb-lip-sync/src/exporters/XmlExporter.cpp:3:
/usr/include/boost/property_tree/detail/xml_parser_writer_settings.hpp: In instantiation of ‘class boost::property_tree::xml_parser::xml_writer_settings<char>’:
/home/ubuntu/rhubarb-lip-sync/src/exporters/XmlExporter.cpp:28:53:   required from here
/usr/include/boost/property_tree/detail/xml_parser_writer_settings.hpp:38:35: error: ‘char’ is not a class, struct, or union type
  typedef typename Str::value_type Ch;
                                   ^
/usr/include/boost/property_tree/detail/xml_parser_writer_settings.hpp:40:9: error: ‘char’ is not a class, struct, or union type
         xml_writer_settings(Ch inchar = Ch(' '),
         ^
/usr/include/boost/property_tree/detail/xml_parser_writer_settings.hpp:50:33: error: ‘char’ is not a class, struct, or union type
         typename Str::size_type indent_count;
                                 ^
/home/ubuntu/rhubarb-lip-sync/src/exporters/XmlExporter.cpp: In member function ‘virtual void XmlExporter::exportAnimation(const boost::filesystem::path&, JoiningContinuousTimeline<Shape>&, const ShapeSet&, std::ostream&)’:
/home/ubuntu/rhubarb-lip-sync/src/exporters/XmlExporter.cpp:28:53: error: no matching function for call to ‘boost::property_tree::xml_parser::xml_writer_settings<char>::xml_writer_settings(char, int)’
  write_xml(outputStream, tree, writer_setting(' ', 2));
                                                     ^
In file included from /usr/include/boost/property_tree/detail/xml_parser_utils.hpp:15:0,
                 from /usr/include/boost/property_tree/detail/xml_parser_write.hpp:15,
                 from /usr/include/boost/property_tree/xml_parser.hpp:15,
                 from /home/ubuntu/rhubarb-lip-sync/src/exporters/XmlExporter.cpp:3:
/usr/include/boost/property_tree/detail/xml_parser_writer_settings.hpp:36:11: note: candidate: boost::property_tree::xml_parser::xml_writer_settings<char>::xml_writer_settings()
     class xml_writer_settings
           ^
/usr/include/boost/property_tree/detail/xml_parser_writer_settings.hpp:36:11: note:   candidate expects 0 arguments, 2 provided
/usr/include/boost/property_tree/detail/xml_parser_writer_settings.hpp:36:11: note: candidate: constexpr boost::property_tree::xml_parser::xml_writer_settings<char>::xml_writer_settings(const boost::property_tree::xml_parser::xml_writer_settings<char>&)
/usr/include/boost/property_tree/detail/xml_parser_writer_settings.hpp:36:11: note:   candidate expects 1 argument, 2 provided
/usr/include/boost/property_tree/detail/xml_parser_writer_settings.hpp:36:11: note: candidate: constexpr boost::property_tree::xml_parser::xml_writer_settings<char>::xml_writer_settings(boost::property_tree::xml_parser::xml_writer_settings<char>&&)
/usr/include/boost/property_tree/detail/xml_parser_writer_settings.hpp:36:11: note:   candidate expects 1 argument, 2 provided
CMakeFiles/rhubarb-exporters.dir/build.make:134: recipe for target 'CMakeFiles/rhubarb-exporters.dir/src/exporters/XmlExporter.cpp.o' failed
make[2]: *** [CMakeFiles/rhubarb-exporters.dir/src/exporters/XmlExporter.cpp.o] Error 1
CMakeFiles/Makefile2:73: recipe for target 'CMakeFiles/rhubarb-exporters.dir/all' failed
make[1]: *** [CMakeFiles/rhubarb-exporters.dir/all] Error 2
Makefile:149: recipe for target 'all' failed
make: *** [all] Error 2

See image

Special characters (non-ASCII Unicode) in file paths are handled correctly for the input file and the log file. However, they don't work for the output file (at least on Windows). Specifying -o der-frühe-vogel.xml results in a file der-frühe-vogel.xml.

Hi.
A couple of questions.
Firstly - would this run on a RPi 3 or smaller.
Secondly can this be run in a continuous speech environment.

I am looking at ways to implement this in a real time environment where an 'actor' is wearing a mascot style costume that has servo's around the characters lips. When the actor speaks the suit's mouth will give an approximate shape of the spoken sound.

Many many thanks
Dave

If a recording starts with a pause, the resulting animation should start with a closed mouth (X shape). For some recordings, however, the animation wrongly starts with a B shape.

This is caused by the way Rhubarb uses WebRTC for voice activity detection. For some recordings, WebRTC erroneously detects a noise sound at the very beginning of the recording, which Rhubarb then translates into a B shape.

This is an awesome project. Great work!

You've stated that you're getting about 75% accuracy, That means that at least 25% are still inaccurate. How do you tweak those manually? To modify the output files by hand is tedious and error-prone. Do you have a GUI for that?

Cheers