bornfree / talking_avatar

A nice 3D avatar that can speak input text with facial expressions

Talking avatar

A ThreeJS-powered virtual human being that uses a set of neat Azure APIs to do some talking!

ta.webm

Add Chat GPT to the mix and maybe you can have for yourself a nice face to chat with. 🙂

Run

$ yarn install
$ yarn start

Backend

You'll also need https://github.com/bornfree/talking_avatar_backend for the text to speech conversion.

talking_avatar's Issues

Video Download option

Hi there, I am working on this project.

Could you please tell me how can I download the video that I create?

Blend json - how to create?

Noticed you are using .json for animation?

what software did you use to create then export the .json animation?

I’ve tried a few software packages and cannot locate on to export the .json animation data

Example only shows the model's hair

Hi,

Thank you for a nice example on integrating the text to audio.

I can receive the audio and Viseme, but the model is not loading correctly.

Would you be so kind to show me how to display the entire model?

Speak button not working?

hi, how come when I click on the Speak button, it does not talk? the button flickers (I think it says running or a fraction of a second). see video. thanks

https://www.loom.com/share/4f3444181adb46d881dfc5e456ab8ef8?sid=e2b1650f-b922-4f42-9cbe-711a1c5735e5

THREE.WebGLRenderer: Context Lost.

The voice file has been produce but hit the error and the page become grey after click speak.
three.module.js:46363 Uncaught TypeError: Cannot read properties of undefined (reading 'uuid') at AnimationMixer.clipAction (three.module.js:46363:1) at App.js:262:1 at arrayEach (lodash.js:530:1) at Function.forEach (lodash.js:9410:1) at App.js:261:1 at commitHookEffectListMount (react-reconciler.development.js:14669:1) at commitPassiveMountOnFiber (react-reconciler.development.js:16531:1) at commitPassiveMountEffects_complete (react-reconciler.development.js:16495:1) at commitPassiveMountEffects_begin (react-reconciler.development.js:16482:1) at commitPassiveMountEffects (react-reconciler.development.js:16470:1) clipAction @ three.module.js:46363 (anonymous) @ App.js:262 arrayEach @ lodash.js:530 forEach @ lodash.js:9410 (anonymous) @ App.js:261 commitHookEffectListMount @ react-reconciler.development.js:14669 commitPassiveMountOnFiber @ react-reconciler.development.js:16531 commitPassiveMountEffects_complete @ react-reconciler.development.js:16495 commitPassiveMountEffects_begin @ react-reconciler.development.js:16482 commitPassiveMountEffects @ react-reconciler.development.js:16470 flushPassiveEffectsImpl @ react-reconciler.development.js:19142 flushPassiveEffects @ react-reconciler.development.js:19095 (anonymous) @ react-reconciler.development.js:18891 workLoop @ scheduler.development.js:266 flushWork @ scheduler.development.js:239 performWorkUntilDeadline @ scheduler.development.js:533

How to create character ?

Everythings looks fine but looking to change digital human character. Can you please provide documentation how can we create ?

Avatar is not speaking

Hi @bornfree ,

I have npm install and npm start the code. The avatar is not speaking is it require to put the Azure Speech Service Key and Region? If yes, where should I put? I couldn't find the KEY that I need to put.

Feature Request : Hindi Language Support

Please tell me how to add hindi language support

Not speaking in Mozila

It works very well in Microsoft Edge, and in Chrome the avatar take a little bit late than the sound. The wors in Mozila it is not speaking at all, just running.

—-

cuustom character

hello
this is awesome project
how can I create my own character? I need Men , Women and child character
regards

Are there any completely open-source implementation?

Are there any open source implementation of the whole pipeline needed for such Avatar Video that can be run locally and/or deployed to different cloud?

2d characters

The azure documentation mentions that 2d characters can work with this. Have you tried it?

less teeth?

I was wondering how to show less of teeth of the model ? (maybe just me but it feels too much )

2d Character option

Hi there, great initiative.

I was just wondering how to integrate 2d characters in the system?