numessanguis / facsvatar

An Open Source Modular Framework From Face to FACS Based Avatar Animation (Unity3D / Blender)

Notes 2023-01-12

• I wanted to simplify the quick start and smooth some rough edges before releasing version v0.4.0, but life held me up and I never merged this into master/main branch. This version is in all aspects better than v0.3.4, so I'm merging it after all in its current state.
• Version v0.5.0 is on its way though (no release date yet). Small spoilers:
  • Python version 3.10+
  • Support for Blender 3.3 LTS (2023-06-17) Updated FACSvatar-Blender Blender add-on to support Blender LTS v3.3 & v3.6
  • Will have a proper GUI (written in Vue3 (JavaScript Framework)) that communicates with Python.

What is FACSvatar? (v0.4.0-Alpha)

FACSvatar is An Open Source Modular Framework for Real-Time FACS based Facial Animation

Or in plain English:

Track facial expressions with any software and visualize that data on any avatar in real-time, powered by the FACS representation. No more need to modify your avatar to support your tracking software. All written in your favorite programming language, on any OS, and across machines.

Muscle image source.

• Facial Action Coding System (FACS): A description of how muscle groups in the human face contract/relax to make any facial configuration possible. (learn more).
  • Action Unit (AU): The strength of contraction of a single muscle group.
• Modular: Software and OS independent. You only need to know what data goes in and what comes out.
• Extendable: Write your code, add a ZeroMQ message socket, and let it talk to other modules.
• Real-time: Create lively avatars that respond to your user.
• Machine/Deep Learning: Input/output data-fied facial configurations.

(Above demo video link: https://www.youtube.com/watch?v=J2FvrIl-ypU)

Message to:

• Animators: Copy facial expressions from a video/webcam to your avatar.
• Affective Computing: Enable Human-Agent Interaction (HAI) by inputting your human-analysis into a ML-model, output FACS values, and have your Embodied Conversational Agent (ECA) display it.
• Psychologists: Create stimuli with the same facial configurations across avatars of different sex, age and ethnicity.

FACSvatar is already operable with:

• Tracking software:
  • OpenFace: Extract facial AUs from videos/webcam.
• Visualization software:
  • Blender 2.80+
    • MB-Lab 3D avatar generation
  • Unity3D
  • FACSHuman
• Modules for additional data processing, and allowing m trackers - to - n avatars (modules folder)
• ZeroMQ: This framework's glue, allowing modules to communicate with each other.
• Containerization with Docker to run FACSvatar modules everywhere.

Disclaimers: This is an open-source project, hopefully being flexible enough for your facial animation needs. This is not software supported by a company / commercially, but by users like you. If you need some new capability, you likely have to code it yourself (or ask/hire someone), but questions for guidance are always welcome (make a GitHub issue)! For commercial usage, please check the license page. Read more about FACSvatar's limitations (TODO doc link).

Full documentation

Read the Docs: https://facsvatar.readthedocs.io/

Paper

Please cite the following paper when using this framework in a paper:

van der Struijk, Stef and Huang, Hung-Hsuan and Mirzaei, Maryam Sadat and Nishida, Toyoaki "FACSvatar: An Open Source Modular Framework for Real-Time FACS based Facial Animation" In Proceedings of 18th ACM International Conference on Intelligent Virtual Agents (pp. 159-164). ACM, 2018.

• DOI: https://doi.org/10.1145/10.1145/3267851.3267918
• ISBN: 978-1-4503-6013-5/18/11

New in v0.4.0-alpha (2020-07-??) TODO UNFINISHED

• COMPLETE re-write of the documentation: Check it out!
• Python modules:
  • Standardization pass over all modules / code clean-up
  • Consistency fix: ROUTER / DEALER sockets use JSON formatted data
  • DOC string per class and function
  • Logger instead of print() statements
  • Debug as option to enable logger
  • File structure for proper import of modules / pip?
  • Use config file (in addition to command line arguments) + config filepath argument
• Easy run: Docker container per module + Docker Compose
• Demo video

See all changelogs

Quickstart

FACSvatar is tested on Ubuntu and Windows, but should work on MacOS.

This quickstart has 2 parts:

1. Start FACSvatar modules using Docker - modules in containers (see here for Python instructions)
2. Visualize in Unity3D or Blender

Dockerized modules

0. Downloads - Go to the release page of this GitHub repo and download:

   • (Real-time only) openface_2.1.0_zeromq.zip
     • Unzip and execute download_models.sh or .ps1 to download trained models
   • Windows 7 / 8 / 10 Home version <2004 : unity_FACSvatar_standalone_docker-ip.zip
   • Windows 10 Home v2004+ / Pro / Enterprise / Education: unity_FACSvatar_standalone.zip
   • Windows / Linux / Mac: Unity3D editor (documentation)
   • Source code (zip / tar.gz) or download this repository with:
     • git clone https://github.com/NumesSanguis/FACSvatar.git
     • Press the green Clone or Download button on this page --> Download ZIP

1. Docker Install - Let's you execute applications without worrying about OS or programming language.

   • General Docker instructions
   • Docker Toolbox for Windows 7/8/10 Home version <2004
   • Docker for Windows 10 Home v2004+
   • Docker for Windows 10 Pro, Enterprise or Education
   • Ubuntu: Docker and docker-compose and sudo usermod -a -G docker $USER

2. Docker Modules - Open a terminal (W7/8: cmd.exe / W10: PowerShell) and navigate to folder FACSvatar/modules, then execute:

   1. docker-compose pull (Downloads FACSvatar Docker containers)
   2. docker-compose up (Starts downloaded Docker containers)

3. See visualization engine instructions

Offline version:

4. Open a 2nd terminal in folder FACSvatar/modules and execute: docker-compose exec facsvatar_facsfromcsv bash
5. Inside Docker container - Start facial animation with: python main.py --pub_ip facsvatar_bridge

With webcam for real-time (Windows-only for now):

4. Navigate inside folder openface_x.x.x_zeromq
5. (Windows 7/8/10 Home version <2004 - only) Get Docker machine ip by opening a 2nd terminal and execute: docker-machine ip (likely to be 192.168.99.100)
6. (Windows 7/8/10 Home version <2004 - only) Open config.xml, change <IP>127.0.0.1</IP> to <IP>machine ip from step 3</IP> (<IP>192.168.99.100</IP>) and save and close.
7. Double click OpenFaceOffline.exe –> menu: File –> Open Webcam

Visualization engines

Unity3D

Tested on version: 2018.2.20f1

1. Open the folder unity_FACSvatar as a project with Unity3D
2. Press play (now it's waiting for facial data)

OR (Windows-only TODO):

1. Navigate inside unzipped folder unity_FACSvatar_standalone(_docker-ip) and double-click unity_FACSvatar.exe

Extra: Use the numbers 0, 1, 2 on your keyboard to change camera.

FACSvatar Blender add-on

Follow instructions here: https://github.com/NumesSanguis/FACSvatar-Blender

Quickstart video

See the quickstart video (:warning: note that the Blender script part is outdated (from 15:15) due the new FACSvatar Blender add-on):

Find out more

Full documentation

Read the FACSvatar documentation!

2017 promotion poster (English & 日本語)

facsvatar's Issues

Recording animation curves in Unity

Hi NumesSanguis

Generally animators like to record curves in their software for ease of editing.

The following Github project is a class to record animation curves in Unity and may be helpful to you.

Code
(https://github.com/DrunkenMonday/AnimationRecorder)

Article
https://www.gamedev.net/articles/programming/general-and-gameplay-programming/how-we-optimized-ragdoll-death-animation-in-unity-r4717/

Cheers

Issue with changing bone.rotation_mode to 'XYZ'

I tried implementing FACSvatar with Blender 2.82 with newer version of MB-LAB (1.7.8) and the following issue arises:

Traceback (most recent call last):
File "<blender_console>", line 1, in
File "path/FACSvatar-master/blender/facsvatar_zeromq.py", line 243, in
bpy.ops.wm.facsvatar_zeromq()
File "/Applications/Blender 2.82.app/Contents/Resources/2.82/scripts/modules/bpy/ops.py", line 201, in call
ret = op_call(self.idname_py(), None, kw)
RuntimeError: Error: Traceback (most recent call last):
File "path/FACSvatar-master/blender/facsvatar_zeromq.py", line 41, in init
self.find_MBLabModel()
File "path/FACSvatar-master/blender/facsvatar_zeromq.py", line 58, in find_MBLabModel
bone.rotation_mode = 'XYZ'
AttributeError: Writing to ID classes in this context is not allowed: MBlab_sk1618120982.624969, Object datablock, error setting PoseBone.rotation_mode

location: /Applications/Blender 2.82.app/Contents/Resources/2.82/scripts/modules/bpy/ops.py:201

This issue doesn't arise with Blender 2.79 and MB-LAB 1.6.1a.
I suspect the source of issue is this:
https://wiki.blender.org/wiki/Reference/Release_Notes/2.80/Python_API/Animation_API

However I am not sure how to fix this. Any suggestions?

Blender Problem With Facsvatar

Hi Nunes
I have been following your video trying to set up Facsvatar with Blender 2.80 and 2.95b on Windows 10. I have got the 3 Anaconda windows working as in the video but when I enter the first two lines from facsvatar_zeromq.py in the Blender python console, Blender does not freeze but the script runs to error as below and I am at a loss as to what is wrong. Both environments have python (3.7 and 3.5 respectively) and pyzmq.
I’m not too tech savvy – this is the first time I’ve used Anaconda and environments so I would appreciate any simple guidance you can give.
That said what a marvelous project you have.

script = "C:\Users\mojay\FACSvatar-master (7)\blender\facsvatar_zeromq.py"
exec(compile(open(script).read(), script, 'exec'))

Traceback (most recent call last):
File "<blender_console>", line 1, in
File "C:\Users\mojay\FACSvatar-master (7)\blender\facsvatar_zeromq.py", line 9, in
import zmq
File "C:\Users\mojay\Anaconda3\envs\blender\Lib\site-packages\zmq_init_.py", line 47, in
from zmq import backend
File "C:\Users\mojay\Anaconda3\envs\blender\Lib\site-packages\zmq\backend_init_.py", line 40, in reraise(*exc_info)
File "C:\Users\mojay\Anaconda3\envs\blender\Lib\site-packages\zmq\utils\sixcerpt.py", line 34, in reraise raise value
File "C:\Users\mojay\Anaconda3\envs\blender\Lib\site-packages\zmq\backend_init_.py", line 27, in ns = select_backend(first)
File "C:\Users\mojay\Anaconda3\envs\blender\Lib\site-packages\zmq\backend\select.py", line 27, in select_backend mod = import(name, fromlist=public_api)
File "C:\Users\mojay\Anaconda3\envs\blender\Lib\site-packages\zmq\backend\cython_init.py", line 6, in from . import (constants, error, message, context,

ImportError: cannot import name 'constants' from 'zmq.backend.cython' (C:\Users\mojay\Anaconda3\envs\blender\Lib\site-packages\zmq\backend\cython_init_.py)

Regards
Mojay

How to enhance AUs from within Facsvatar? (Fix the notebook GUI)

Hi there,

The set_new_multiplier function in smooth_data.py multiplies the value of AU45 by 1.5. I can change this value, and this seems to work. But I am having trouble adding multipliers for other AU's. Should this be possible or not?

I have tried this by adding a similar line below the one for AU45, so for example:
self.multiplier[15] = 2.0

But this causes an error in the Process_bridge module:

Traceback (most recent call last):
File "main.py", line 43, in
from smooth_data import SmoothData
File "...\FACSvatar\modules\process_bridge\smooth_data.py", line 40
self.multiplier[15] = 2.0
^
TabError: inconsistent use of tabs and spaces in indentation

I have also tried to change the line for AU45 into self.multiplier[15:17] = [2.0,1.5]

But this gives the following error:

Traceback (most recent call last):
File "main.py", line 43, in
from smooth_data import SmoothData
File "...\FACSvatar\modules\process_bridge\smooth_data.py", line 34
def set_new_multiplier(self no_of_columns=17):
^
SyntaxError: invalid syntax

If it is possible to set additional multipliers, can you show me how to do this?
If not, is there another way to enhance AUs? (other than editing the CSV file from OpenFace)

How to use FACSvatar system with 2 or more computers

how to use FACSvatar(0.3.4) system with 2 computers.
I used windows 10 Enterprise .

-Machine 1 , Unity3D ( ZeroMQFACSvatar's parameter "sub_to_ip" is Machine2_IP ) & dcoker-compose.
-Machine 2 , OpenFaceOffline (Machine1_IP in config.xml)

It's work for me .

You can check which you can communicate with machine following command ( you need to use docker-compose on Machine2 ) .

cd FACSvatar/modules/test_msg
python main.py --sub_ip ip_machine_2 --sub_port module_port

Use Google's project MediaPipe Face Mesh as FACS tracker

Overview

As @fire shared in issue 27, Google has released MediaPipe Face Mesh as a Python library (code).

As this solution seems to be fast and is Apache-2.0 Licensed, it's interesting to consider this as a replacement for OpenFace. The tracking output is 468 3D coordinates (x, y, z) for a single mask (1 image or 1 frame in a video), which looks like:

To be useful for this project, the changes in these masks need to be mapped to Action Unit (AU) values according to the Facial Action Coding System (FACS) model.
More info - Demonstration of facial movement per AU.

This can be tricky, because the deformation in the tracking mesh when 2 AUs are active at the same time, won't be always just be adding the deformation of the AUs individually (no linear relation between e.g. AU12+AU14 compared to AU12 and AU14 separately). In practice a linear approach might be good enough though.

A paper by OpenFace on how they did Facial Action Unit detection:

• Cross-dataset learning and person-specific normalisation for automatic Action Unit detection Tadas Baltrušaitis, Marwa Mahmoud, and Peter Robinson in Facial Expression Recognition and Analysis Challenge, IEEE International Conference on Automatic Face and Gesture Recognition, 2015

MediaPipe Face Mesh output

A single face mesh tracking a single person in 1 image/frame has 468*3 values. A single landmark:

x: 0.4966168701648712
y: 0.694128692150116
z: -0.11288105696439743

Note that this output is normalized between [-1, 1].

Full landmark output single mesh (CLICK ME)

landmark {
  x: 0.4966168701648712
  y: 0.694128692150116
  z: -0.11288105696439743
}
landmark {
  x: 0.4816158711910248
  y: 0.6072000861167908
  z: -0.21528172492980957
}
landmark {
  x: 0.48926183581352234
  y: 0.6384062170982361
  z: -0.1202792227268219
}
landmark {
  x: 0.4422526955604553
  y: 0.5044909715652466
  z: -0.15802232921123505
}
landmark {
  x: 0.47746509313583374
  y: 0.5734008550643921
  z: -0.22716723382472992
}
landmark {
  x: 0.47410061955451965
  y: 0.5310384035110474
  z: -0.2105686366558075
}
landmark {
  x: 0.46738678216934204
  y: 0.4304444193840027
  z: -0.10389776527881622
}
landmark {
  x: 0.21002808213233948
  y: 0.4479711651802063
  z: 0.06500452011823654
}
landmark {
  x: 0.45994189381599426
  y: 0.353225439786911
  z: -0.07487861067056656
}
landmark {
  x: 0.45476841926574707
  y: 0.3084910809993744
  z: -0.08058346062898636
}
landmark {
  x: 0.44055771827697754
  y: 0.14465117454528809
  z: -0.038242045789957047
}
landmark {
  x: 0.49868014454841614
  y: 0.7090015411376953
  z: -0.10894951224327087
}
landmark {
  x: 0.5007204413414001
  y: 0.719012439250946
  z: -0.09897053241729736
}
landmark {
  x: 0.5020402073860168
  y: 0.7213298082351685
  z: -0.08386010676622391
}
landmark {
  x: 0.5109937787055969
  y: 0.7895573973655701
  z: -0.05537683516740799
}
landmark {
  x: 0.5119189023971558
  y: 0.8027503490447998
  z: -0.06261561065912247
}
landmark {
  x: 0.5133177638053894
  y: 0.8185326457023621
  z: -0.0709986463189125
}
landmark {
  x: 0.5152280330657959
  y: 0.8369268774986267
  z: -0.06422298401594162
}
landmark {
  x: 0.5181258916854858
  y: 0.8634383678436279
  z: -0.03230631723999977
}
landmark {
  x: 0.48471203446388245
  y: 0.6251006722450256
  z: -0.19659900665283203
}
landmark {
  x: 0.4531421959400177
  y: 0.6249157190322876
  z: -0.143781840801239
}
landmark {
  x: 0.06447488069534302
  y: 0.339648574590683
  z: 0.25507593154907227
}
landmark {
  x: 0.31962549686431885
  y: 0.46791741251945496
  z: 0.02018388733267784
}
landmark {
  x: 0.28245049715042114
  y: 0.47231966257095337
  z: 0.025203807279467583
}
landmark {
  x: 0.24612677097320557
  y: 0.47224801778793335
  z: 0.0373702310025692
}
landmark {
  x: 0.19500768184661865
  y: 0.45980608463287354
  z: 0.07198791205883026
}
landmark {
  x: 0.349759966135025
  y: 0.45816195011138916
  z: 0.022274039685726166
}
landmark {
  x: 0.26011133193969727
  y: 0.389184832572937
  z: 0.010085818357765675
}
landmark {
  x: 0.2995329797267914
  y: 0.389138400554657
  z: 0.008145865984261036
}
landmark {
  x: 0.22314757108688354
  y: 0.39661189913749695
  z: 0.02493373677134514
}
landmark {
  x: 0.19949522614479065
  y: 0.40996548533439636
  z: 0.0419645681977272
}
landmark {
  x: 0.16591835021972656
  y: 0.48124581575393677
  z: 0.09229511767625809
}
landmark {
  x: 0.3812064528465271
  y: 0.9071664214134216
  z: 0.021440977230668068
}
landmark {
  x: 0.19616428017616272
  y: 0.4427569508552551
  z: 0.07880867272615433
}
landmark {
  x: 0.062373995780944824
  y: 0.48197007179260254
  z: 0.26922407746315
}
landmark {
  x: 0.12454494833946228
  y: 0.465819776058197
  z: 0.13364587724208832
}
landmark {
  x: 0.2905968725681305
  y: 0.5917161703109741
  z: -0.022236289456486702
}
landmark {
  x: 0.4381605386734009
  y: 0.6930720210075378
  z: -0.10508478432893753
}
landmark {
  x: 0.4468565285205841
  y: 0.7214252352714539
  z: -0.08921617269515991
}
landmark {
  x: 0.38093671202659607
  y: 0.7045804262161255
  z: -0.07571420818567276
}
landmark {
  x: 0.3459462523460388
  y: 0.717242956161499
  z: -0.033613234758377075
}
landmark {
  x: 0.4050546884536743
  y: 0.7251520156860352
  z: -0.06517799943685532
}
landmark {
  x: 0.37378859519958496
  y: 0.7300823926925659
  z: -0.02689925581216812
}
landmark {
  x: 0.3168252110481262
  y: 0.7854228019714355
  z: 0.044089797884225845
}
landmark {
  x: 0.45147567987442017
  y: 0.6086890697479248
  z: -0.21051371097564697
}
landmark {
  x: 0.4435341954231262
  y: 0.5764272809028625
  z: -0.22110529243946075
}
landmark {
  x: 0.13545703887939453
  y: 0.3788211941719055
  z: 0.056899067014455795
}
landmark {
  x: 0.3665546476840973
  y: 0.5107970833778381
  z: -0.027126528322696686
}
landmark {
  x: 0.36413857340812683
  y: 0.6033427715301514
  z: -0.11284682154655457
}
landmark {
  x: 0.36061158776283264
  y: 0.5862759947776794
  z: -0.10040289163589478
}
landmark {
  x: 0.1746581494808197
  y: 0.5945844054222107
  z: 0.03535617142915726
}
landmark {
  x: 0.4421435296535492
  y: 0.5369409322738647
  z: -0.1966618001461029
}
landmark {
  x: 0.2176908552646637
  y: 0.33583539724349976
  z: -0.014324278570711613
}
landmark {
  x: 0.16782251000404358
  y: 0.35119521617889404
  z: 0.01882544904947281
}
landmark {
  x: 0.09129291772842407
  y: 0.2760707437992096
  z: 0.179345965385437
}
landmark {
  x: 0.3814221918582916
  y: 0.3533117473125458
  z: -0.05903608724474907
}
landmark {
  x: 0.3350331783294678
  y: 0.3979620933532715
  z: 0.016856539994478226
}
landmark {
  x: 0.2812685966491699
  y: 0.745897650718689
  z: 0.04884406924247742
}
landmark {
  x: 0.1452232003211975
  y: 0.7861756086349487
  z: 0.3580620586872101
}
landmark {
  x: 0.394314706325531
  y: 0.619657576084137
  z: -0.10019056499004364
}
landmark {
  x: 0.4270922541618347
  y: 0.6279042959213257
  z: -0.10671922564506531
}
landmark {
  x: 0.307145357131958
  y: 0.7430539727210999
  z: 0.049138471484184265
}
landmark {
  x: 0.32793134450912476
  y: 0.7431167960166931
  z: 0.03924649953842163
}
landmark {
  x: 0.14394527673721313
  y: 0.3307373821735382
  z: 0.03859829157590866
}
landmark {
  x: 0.3629467487335205
  y: 0.6155763268470764
  z: -0.09240224212408066
}
landmark {
  x: 0.28702664375305176
  y: 0.3346884250640869
  z: -0.04128420352935791
}
landmark {
  x: 0.27642199397087097
  y: 0.30713507533073425
  z: -0.04866618663072586
}
landmark {
  x: 0.22702038288116455
  y: 0.1813761293888092
  z: 0.023917950689792633
}
landmark {
  x: 0.11776086688041687
  y: 0.3024034798145294
  z: 0.10050833225250244
}
landmark {
  x: 0.2545614540576935
  y: 0.24051368236541748
  z: -0.01349948812276125
}
landmark {
  x: 0.11170089244842529
  y: 0.36651477217674255
  z: 0.09082090854644775
}
landmark {
  x: 0.08806124329566956
  y: 0.35485026240348816
  z: 0.16924317181110382
}
landmark {
  x: 0.44142016768455505
  y: 0.71026611328125
  z: -0.10029156506061554
}
landmark {
  x: 0.39390215277671814
  y: 0.7174962162971497
  z: -0.07107415795326233
}
landmark {
  x: 0.35953131318092346
  y: 0.7237300276756287
  z: -0.032877374440431595
}
landmark {
  x: 0.40349727869033813
  y: 0.6242803335189819
  z: -0.09295294433832169
}
landmark {
  x: 0.3165252208709717
  y: 0.7428818941116333
  z: 0.043773844838142395
}
landmark {
  x: 0.3406851887702942
  y: 0.7669117450714111
  z: 0.023747634142637253
}
landmark {
  x: 0.33578944206237793
  y: 0.7433992028236389
  z: 0.03797805681824684
}
landmark {
  x: 0.4140060544013977
  y: 0.6110385656356812
  z: -0.15295974910259247
}
landmark {
  x: 0.38158589601516724
  y: 0.7267777323722839
  z: -0.022487040609121323
}
landmark {
  x: 0.41331616044044495
  y: 0.724240779876709
  z: -0.0519661121070385
}
landmark {
  x: 0.4527042806148529
  y: 0.7233290076255798
  z: -0.07439269125461578
}
landmark {
  x: 0.45655256509780884
  y: 0.8648825287818909
  z: -0.02545144222676754
}
landmark {
  x: 0.4548318684101105
  y: 0.8358229398727417
  z: -0.05548474192619324
}
landmark {
  x: 0.4551684558391571
  y: 0.8169925212860107
  z: -0.059868987649679184
}
landmark {
  x: 0.457076758146286
  y: 0.8008837699890137
  z: -0.05086479336023331
}
landmark {
  x: 0.4605860710144043
  y: 0.7895804643630981
  z: -0.0449383407831192
}
landmark {
  x: 0.38548389077186584
  y: 0.7749221324920654
  z: 9.880734432954341e-05
}
landmark {
  x: 0.3780013620853424
  y: 0.7787157893180847
  z: -0.005501376464962959
}
landmark {
  x: 0.36844146251678467
  y: 0.7868350148200989
  z: -0.009895708411931992
}
landmark {
  x: 0.3618044853210449
  y: 0.7972811460494995
  z: -0.0017165766330435872
}
landmark {
  x: 0.3194640278816223
  y: 0.6933881044387817
  z: -0.021864671260118484
}
landmark {
  x: 0.08811995387077332
  y: 0.6305686235427856
  z: 0.4192250669002533
}
landmark {
  x: 0.48704424500465393
  y: 0.6317288279533386
  z: -0.14750073850154877
}
landmark {
  x: 0.36362749338150024
  y: 0.76473468542099
  z: 0.025699688121676445
}
landmark {
  x: 0.35120195150375366
  y: 0.7645043134689331
  z: 0.02254905179142952
}
landmark {
  x: 0.4399850070476532
  y: 0.6405894756317139
  z: -0.10764143615961075
}
landmark {
  x: 0.375720739364624
  y: 0.6322762966156006
  z: -0.06489422917366028
}
landmark {
  x: 0.4340003728866577
  y: 0.634506344795227
  z: -0.10850734263658524
}
landmark {
  x: 0.32653310894966125
  y: 0.5300822854042053
  z: -0.012731536291539669
}
landmark {
  x: 0.2632318139076233
  y: 0.5526027083396912
  z: 0.00027940748259425163
}
landmark {
  x: 0.3542672097682953
  y: 0.5996798276901245
  z: -0.07833056151866913
}
landmark {
  x: 0.14197686314582825
  y: 0.2222341001033783
  z: 0.09827900677919388
}
landmark {
  x: 0.16972631216049194
  y: 0.26265406608581543
  z: 0.04034826159477234
}
landmark {
  x: 0.20060905814170837
  y: 0.3098430633544922
  z: -0.006581175606697798
}
landmark {
  x: 0.35248780250549316
  y: 0.8181872367858887
  z: 0.022957870736718178
}
landmark {
  x: 0.36348414421081543
  y: 0.3094140589237213
  z: -0.07173669338226318
}
landmark {
  x: 0.3437490463256836
  y: 0.23109906911849976
  z: -0.047013651579618454
}
landmark {
  x: 0.3235037922859192
  y: 0.15789145231246948
  z: -0.02233033999800682
}
landmark {
  x: 0.21419644355773926
  y: 0.4688110947608948
  z: 0.05552274361252785
}
landmark {
  x: 0.12787866592407227
  y: 0.5059182047843933
  z: 0.1152096688747406
}
landmark {
  x: 0.36812901496887207
  y: 0.44856852293014526
  z: 0.025466639548540115
}
landmark {
  x: 0.15955564379692078
  y: 0.42317861318588257
  z: 0.07820062339305878
}
landmark {
  x: 0.3913554549217224
  y: 0.4891124963760376
  z: -0.04222859442234039
}
landmark {
  x: 0.38729652762413025
  y: 0.5918189883232117
  z: -0.15105858445167542
}
landmark {
  x: 0.09255272150039673
  y: 0.5330958962440491
  z: 0.16015303134918213
}
landmark {
  x: 0.1535634994506836
  y: 0.5218596458435059
  z: 0.07908810675144196
}
landmark {
  x: 0.20109590888023376
  y: 0.5274339914321899
  z: 0.04150325804948807
}
landmark {
  x: 0.27242377400398254
  y: 0.5198872685432434
  z: 0.020176483318209648
}
landmark {
  x: 0.326151967048645
  y: 0.5059536695480347
  z: 0.008998933248221874
}
landmark {
  x: 0.36502182483673096
  y: 0.48930275440216064
  z: -0.0038086730055510998
}
landmark {
  x: 0.4342425763607025
  y: 0.44238021969795227
  z: -0.08627469837665558
}
landmark {
  x: 0.09740403294563293
  y: 0.5966350436210632
  z: 0.1479249894618988
}
landmark {
  x: 0.12877383828163147
  y: 0.42056652903556824
  z: 0.09765920042991638
}
landmark {
  x: 0.46656739711761475
  y: 0.6248356699943542
  z: -0.19346565008163452
}
landmark {
  x: 0.3698480725288391
  y: 0.5399408936500549
  z: -0.046744342893362045
}
landmark {
  x: 0.0581088662147522
  y: 0.4841393232345581
  z: 0.3943922519683838
}
landmark {
  x: 0.3915879726409912
  y: 0.47015851736068726
  z: -0.011566780507564545
}
landmark {
  x: 0.3505880832672119
  y: 0.6014127135276794
  z: -0.03828176483511925
}
landmark {
  x: 0.18235325813293457
  y: 0.44442564249038696
  z: 0.08718197047710419
}
landmark {
  x: 0.38435113430023193
  y: 0.5708659887313843
  z: -0.13555653393268585
}
landmark {
  x: 0.1089932918548584
  y: 0.7077124118804932
  z: 0.39826714992523193
}
landmark {
  x: 0.3669191300868988
  y: 0.4378426671028137
  z: 0.03585735708475113
}
landmark {
  x: 0.41278910636901855
  y: 0.5510349273681641
  z: -0.17305362224578857
}
landmark {
  x: 0.22341036796569824
  y: 0.8427634239196777
  z: 0.1659410148859024
}
landmark {
  x: 0.24016138911247253
  y: 0.8845473527908325
  z: 0.22497878968715668
}
landmark {
  x: 0.07366946339607239
  y: 0.6184796690940857
  z: 0.28494521975517273
}
landmark {
  x: 0.16882428526878357
  y: 0.8082045912742615
  z: 0.208804190158844
}
landmark {
  x: 0.07401168346405029
  y: 0.41425713896751404
  z: 0.23039989173412323
}
landmark {
  x: 0.38395026326179504
  y: 0.9384512305259705
  z: 0.038852691650390625
}
landmark {
  x: 0.4726627469062805
  y: 0.6314657926559448
  z: -0.14449647068977356
}
landmark {
  x: 0.3361490070819855
  y: 0.5637073516845703
  z: -0.03239408880472183
}
landmark {
  x: 0.09436336159706116
  y: 0.47562286257743835
  z: 0.1737174242734909
}
landmark {
  x: 0.24819177389144897
  y: 0.4541197419166565
  z: 0.03995387256145477
}
landmark {
  x: 0.28098762035369873
  y: 0.45491164922714233
  z: 0.02891157753765583
}
landmark {
  x: 0.33149421215057373
  y: 0.7714042663574219
  z: 0.0297850389033556
}
landmark {
  x: 0.1068907082080841
  y: 0.6629747748374939
  z: 0.15780605375766754
}
landmark {
  x: 0.4563775956630707
  y: 0.9773697257041931
  z: 0.038378626108169556
}
landmark {
  x: 0.34909868240356445
  y: 0.9424223899841309
  z: 0.12394849210977554
}
landmark {
  x: 0.30104225873947144
  y: 0.9184457063674927
  z: 0.1700422614812851
}
landmark {
  x: 0.4470309317111969
  y: 0.22389227151870728
  z: -0.061588384211063385
}
landmark {
  x: 0.5305874347686768
  y: 0.9793791770935059
  z: 0.02257702499628067
}
landmark {
  x: 0.3115778863430023
  y: 0.4516664147377014
  z: 0.02560044266283512
}
landmark {
  x: 0.34036505222320557
  y: 0.44540536403656006
  z: 0.029373547062277794
}
landmark {
  x: 0.35830408334732056
  y: 0.4412270188331604
  z: 0.03582789748907089
}
landmark {
  x: 0.10178148746490479
  y: 0.41643524169921875
  z: 0.13728414475917816
}
landmark {
  x: 0.3333684802055359
  y: 0.4224480390548706
  z: 0.023977747187018394
}
landmark {
  x: 0.30116382241249084
  y: 0.4180963933467865
  z: 0.019164299592375755
}
landmark {
  x: 0.2705670893192291
  y: 0.41848498582839966
  z: 0.022922012954950333
}
landmark {
  x: 0.23962080478668213
  y: 0.42437681555747986
  z: 0.03419533744454384
}
landmark {
  x: 0.21877902746200562
  y: 0.4323749542236328
  z: 0.047679878771305084
}
landmark {
  x: 0.0542866587638855
  y: 0.40489107370376587
  z: 0.33343178033828735
}
landmark {
  x: 0.2258961796760559
  y: 0.45137596130371094
  z: 0.0531965009868145
}
landmark {
  x: 0.492154061794281
  y: 0.6571369767189026
  z: -0.10802311450242996
}
landmark {
  x: 0.35693883895874023
  y: 0.6745429039001465
  z: -0.05522865429520607
}
landmark {
  x: 0.39345255494117737
  y: 0.6159301996231079
  z: -0.11440737545490265
}
landmark {
  x: 0.43464553356170654
  y: 0.6622862219810486
  z: -0.10015508532524109
}
landmark {
  x: 0.4644954800605774
  y: 0.3917056918144226
  z: -0.07517721503973007
}
landmark {
  x: 0.28190237283706665
  y: 0.879287838935852
  z: 0.1305902898311615
}
landmark {
  x: 0.33249735832214355
  y: 0.9081810712814331
  z: 0.08741878718137741
}
landmark {
  x: 0.44906002283096313
  y: 0.9580975770950317
  z: -0.003724485170096159
}
landmark {
  x: 0.1908087432384491
  y: 0.8453988432884216
  z: 0.29279109835624695
}
landmark {
  x: 0.35627108812332153
  y: 0.431122362613678
  z: 0.03127270191907883
}
landmark {
  x: 0.41472727060317993
  y: 0.49441081285476685
  z: -0.09768738597631454
}
landmark {
  x: 0.5268312692642212
  y: 0.9583680629730225
  z: -0.01725558005273342
}
landmark {
  x: 0.3982726037502289
  y: 0.9635181427001953
  z: 0.07418964058160782
}
landmark {
  x: 0.09216967225074768
  y: 0.6903148293495178
  z: 0.27550944685935974
}
landmark {
  x: 0.41915491223335266
  y: 0.7839378118515015
  z: -0.02422095090150833
}
landmark {
  x: 0.4124149680137634
  y: 0.7920427918434143
  z: -0.02981782704591751
}
landmark {
  x: 0.4065060317516327
  y: 0.8049683570861816
  z: -0.03447248786687851
}
landmark {
  x: 0.4030742049217224
  y: 0.8215083479881287
  z: -0.02789442613720894
}
landmark {
  x: 0.39689958095550537
  y: 0.8466787338256836
  z: -0.003043192205950618
}
landmark {
  x: 0.3447519540786743
  y: 0.7349370121955872
  z: 0.008834567852318287
}
landmark {
  x: 0.3311789631843567
  y: 0.7321010828018188
  z: 0.010857325978577137
}
landmark {
  x: 0.3195076584815979
  y: 0.7279363870620728
  z: 0.0107242651283741
}
landmark {
  x: 0.2904130518436432
  y: 0.7160662412643433
  z: 0.01643066480755806
}
landmark {
  x: 0.15070724487304688
  y: 0.6711384654045105
  z: 0.0781637653708458
}
landmark {
  x: 0.4137824773788452
  y: 0.4639121890068054
  z: -0.06148932874202728
}
landmark {
  x: 0.3897281885147095
  y: 0.4090869128704071
  z: 0.00622992729768157
}
landmark {
  x: 0.36975330114364624
  y: 0.41790977120399475
  z: 0.02349216863512993
}
landmark {
  x: 0.35541102290153503
  y: 0.7306694984436035
  z: 0.010990123264491558
}
landmark {
  x: 0.164760559797287
  y: 0.7537547945976257
  z: 0.13850508630275726
}
landmark {
  x: 0.41884663701057434
  y: 0.40231144428253174
  z: -0.046456996351480484
}
landmark {
  x: 0.38834986090660095
  y: 0.8745599389076233
  z: 0.010446273721754551
}
landmark {
  x: 0.4720120429992676
  y: 0.4969934821128845
  z: -0.17284521460533142
}
landmark {
  x: 0.43800702691078186
  y: 0.4754502773284912
  z: -0.125311017036438
}
landmark {
  x: 0.46979230642318726
  y: 0.4655134677886963
  z: -0.1363074779510498
}
landmark {
  x: 0.3925747275352478
  y: 0.544338583946228
  z: -0.09969377517700195
}
landmark {
  x: 0.5232637524604797
  y: 0.9283697009086609
  z: -0.03395295888185501
}
landmark {
  x: 0.5202719569206238
  y: 0.8925750255584717
  z: -0.03351389244198799
}
landmark {
  x: 0.45128118991851807
  y: 0.8941555619239807
  z: -0.02066991664469242
}
landmark {
  x: 0.2987120747566223
  y: 0.8035737872123718
  z: 0.06246912106871605
}
landmark {
  x: 0.32278764247894287
  y: 0.628929078578949
  z: -0.02323312498629093
}
landmark {
  x: 0.3410077691078186
  y: 0.8435471057891846
  z: 0.04044443741440773
}
landmark {
  x: 0.2297494113445282
  y: 0.6335470676422119
  z: -0.0015491320518776774
}
landmark {
  x: 0.28704535961151123
  y: 0.6627032160758972
  z: -0.005966142285615206
}
landmark {
  x: 0.2043558955192566
  y: 0.6869828104972839
  z: 0.03605557605624199
}
landmark {
  x: 0.44654372334480286
  y: 0.9296964406967163
  z: -0.01958140730857849
}
landmark {
  x: 0.3718560039997101
  y: 0.5627002120018005
  z: -0.07194902747869492
}
landmark {
  x: 0.2805273234844208
  y: 0.8365193605422974
  z: 0.09268306195735931
}
landmark {
  x: 0.3318604826927185
  y: 0.8726451396942139
  z: 0.06202562153339386
}
landmark {
  x: 0.25948265194892883
  y: 0.760127067565918
  z: 0.06235231086611748
}
landmark {
  x: 0.12568554282188416
  y: 0.7197843790054321
  z: 0.17079582810401917
}
landmark {
  x: 0.21877965331077576
  y: 0.7787196636199951
  z: 0.09545453637838364
}
landmark {
  x: 0.12219688296318054
  y: 0.7549031376838684
  z: 0.25807473063468933
}
landmark {
  x: 0.25807932019233704
  y: 0.7004751563072205
  z: 0.02069525420665741
}
landmark {
  x: 0.3937021791934967
  y: 0.5172567367553711
  z: -0.07004125416278839
}
landmark {
  x: 0.40121617913246155
  y: 0.6062255501747131
  z: -0.1600225418806076
}
landmark {
  x: 0.3769870102405548
  y: 0.6137251853942871
  z: -0.11848873645067215
}
landmark {
  x: 0.4142044186592102
  y: 0.5831300020217896
  z: -0.1871527135372162
}
landmark {
  x: 0.3571854829788208
  y: 0.3837273418903351
  z: -0.0017102970741689205
}
landmark {
  x: 0.2956382930278778
  y: 0.3691142201423645
  z: -0.003822853323072195
}
landmark {
  x: 0.2431805431842804
  y: 0.3689405918121338
  z: 0.0017890947638079524
}
landmark {
  x: 0.20002737641334534
  y: 0.3778613805770874
  z: 0.0207314882427454
}
landmark {
  x: 0.17119255661964417
  y: 0.39538031816482544
  z: 0.046379540115594864
}
landmark {
  x: 0.15769413113594055
  y: 0.4544641971588135
  z: 0.10427386313676834
}
landmark {
  x: 0.06413432955741882
  y: 0.5501613616943359
  z: 0.28021129965782166
}
landmark {
  x: 0.18775427341461182
  y: 0.49140042066574097
  z: 0.07144352793693542
}
landmark {
  x: 0.22720354795455933
  y: 0.49667030572891235
  z: 0.04704747349023819
}
landmark {
  x: 0.27706509828567505
  y: 0.49465706944465637
  z: 0.029038190841674805
}
landmark {
  x: 0.32410991191864014
  y: 0.48578399419784546
  z: 0.018875757232308388
}
landmark {
  x: 0.3597770035266876
  y: 0.4729805588722229
  z: 0.014622870832681656
}
landmark {
  x: 0.3826742172241211
  y: 0.4602789282798767
  z: 0.01001096609979868
}
landmark {
  x: 0.07308945059776306
  y: 0.5572210550308228
  z: 0.4202417731285095
}
landmark {
  x: 0.3781541585922241
  y: 0.6203401684761047
  z: -0.09991267323493958
}
landmark {
  x: 0.41601312160491943
  y: 0.5217124223709106
  z: -0.13004370033740997
}
landmark {
  x: 0.42697784304618835
  y: 0.60621577501297
  z: -0.19082331657409668
}
landmark {
  x: 0.44973915815353394
  y: 0.6207560896873474
  z: -0.17728112637996674
}
landmark {
  x: 0.42836445569992065
  y: 0.6117011904716492
  z: -0.17246049642562866
}
landmark {
  x: 0.3879000246524811
  y: 0.6280851364135742
  z: -0.08750980347394943
}
landmark {
  x: 0.45556584000587463
  y: 0.6235464215278625
  z: -0.18769052624702454
}
landmark {
  x: 0.4613252580165863
  y: 0.6296263337135315
  z: -0.14377787709236145
}
landmark {
  x: 0.3795766532421112
  y: 0.43754279613494873
  z: 0.0279754139482975
}
landmark {
  x: 0.39896324276924133
  y: 0.44095879793167114
  z: 0.003275311319157481
}
landmark {
  x: 0.4094507694244385
  y: 0.44369012117385864
  z: -0.02484704740345478
}
landmark {
  x: 0.2062455117702484
  y: 0.43818193674087524
  z: 0.06118197739124298
}
landmark {
  x: 0.18186673521995544
  y: 0.4249523878097534
  z: 0.06467242538928986
}
landmark {
  x: 0.5035853981971741
  y: 0.4999598562717438
  z: -0.16247594356536865
}
landmark {
  x: 0.7413772344589233
  y: 0.40681785345077515
  z: 0.03183317929506302
}
landmark {
  x: 0.5206395387649536
  y: 0.6203845143318176
  z: -0.14702844619750977
}
landmark {
  x: 0.8921987414360046
  y: 0.27469971776008606
  z: 0.20559513568878174
}
landmark {
  x: 0.6345242261886597
  y: 0.443726509809494
  z: -0.0012637971667572856
}
landmark {
  x: 0.6732664704322815
  y: 0.4422304034233093
  z: -0.0007405339856632054
}
landmark {
  x: 0.7107070684432983
  y: 0.4366280734539032
  z: 0.007454411592334509
}
landmark {
  x: 0.7611681222915649
  y: 0.41643911600112915
  z: 0.03679535910487175
}
landmark {
  x: 0.6014629602432251
  y: 0.4388699531555176
  z: 0.005170207004994154
}
landmark {
  x: 0.6754707098007202
  y: 0.35596051812171936
  z: -0.016539370641112328
}
landmark {
  x: 0.6357946395874023
  y: 0.362026184797287
  z: -0.012676878832280636
}
landmark {
  x: 0.7149510979652405
  y: 0.35779958963394165
  z: -0.005732965189963579
}
landmark {
  x: 0.7429794669151306
  y: 0.3676159083843231
  z: 0.00850668828934431
}
landmark {
  x: 0.798031747341156
  y: 0.4332960247993469
  z: 0.054415538907051086
}
landmark {
  x: 0.6666548848152161
  y: 0.8896511793136597
  z: 0.006440763361752033
}
landmark {
  x: 0.7551484107971191
  y: 0.39951521158218384
  z: 0.044434018433094025
}
landmark {
  x: 0.9230958223342896
  y: 0.41696596145629883
  z: 0.21836701035499573
}
landmark {
  x: 0.8410305976867676
  y: 0.41109052300453186
  z: 0.09126061201095581
}
landmark {
  x: 0.6870469450950623
  y: 0.5627361536026001
  z: -0.04433140903711319
}
landmark {
  x: 0.5558522939682007
  y: 0.6847144365310669
  z: -0.11138556152582169
}
landmark {
  x: 0.5563403964042664
  y: 0.7140500545501709
  z: -0.0948861837387085
}
landmark {
  x: 0.6192103028297424
  y: 0.688189685344696
  z: -0.08818458765745163
}
landmark {
  x: 0.6611127853393555
  y: 0.6955099701881409
  z: -0.05048443749547005
}
landmark {
  x: 0.6017453074455261
  y: 0.7119958996772766
  z: -0.07660623639822006
}
landmark {
  x: 0.6376242637634277
  y: 0.7120493650436401
  z: -0.04177974909543991
}
landmark {
  x: 0.7118440866470337
  y: 0.7589071989059448
  z: 0.023515647277235985
}
landmark {
  x: 0.5126743316650391
  y: 0.6043348908424377
  z: -0.21332086622714996
}
landmark {
  x: 0.5123692154884338
  y: 0.5711033940315247
  z: -0.22434364259243011
}
landmark {
  x: 0.8047051429748535
  y: 0.3262219727039337
  z: 0.017701640725135803
}
landmark {
  x: 0.5927931070327759
  y: 0.49367523193359375
  z: -0.041927769780159
}
landmark {
  x: 0.6082389950752258
  y: 0.5855504274368286
  z: -0.12623192369937897
}
landmark {
  x: 0.6092568635940552
  y: 0.5680200457572937
  z: -0.11419199407100677
}
landmark {
  x: 0.8092576861381531
  y: 0.5479861497879028
  z: -0.0014363480731844902
}
landmark {
  x: 0.507614016532898
  y: 0.5319874286651611
  z: -0.20096458494663239
}
landmark {
  x: 0.7031187415122986
  y: 0.2961500585079193
  z: -0.0406055711209774
}
landmark {
  x: 0.760782778263092
  y: 0.3038061857223511
  z: -0.014503950253129005
}
landmark {
  x: 0.8425694704055786
  y: 0.21568787097930908
  z: 0.13522790372371674
}
landmark {
  x: 0.5385988354682922
  y: 0.34032565355300903
  z: -0.06717018038034439
}
landmark {
  x: 0.6027284860610962
  y: 0.3763411343097687
  z: 0.00015584345965180546
}
landmark {
  x: 0.7395942211151123
  y: 0.7144449949264526
  z: 0.02430148981511593
}
landmark {
  x: 0.9136649370193481
  y: 0.7333142757415771
  z: 0.31204113364219666
}
landmark {
  x: 0.5830571055412292
  y: 0.6060899496078491
  z: -0.11036499589681625
}
landmark {
  x: 0.551142156124115
  y: 0.6191096901893616
  z: -0.11160721629858017
}
landmark {
  x: 0.7133134603500366
  y: 0.7155593037605286
  z: 0.027367359027266502
}
landmark {
  x: 0.6942542791366577
  y: 0.7182605862617493
  z: 0.019502172246575356
}
landmark {
  x: 0.7833856344223022
  y: 0.2792634069919586
  z: 0.0030102706514298916
}
landmark {
  x: 0.6151416301727295
  y: 0.5973612666130066
  z: -0.10532160103321075
}
landmark {
  x: 0.6298211812973022
  y: 0.30626943707466125
  z: -0.059158291667699814
}
landmark {
  x: 0.6340736746788025
  y: 0.27706363797187805
  z: -0.06689538061618805
}
landmark {
  x: 0.6665512323379517
  y: 0.14459574222564697
  z: -0.0008641325403004885
}
landmark {
  x: 0.8120237588882446
  y: 0.24646785855293274
  z: 0.06073474511504173
}
landmark {
  x: 0.6456632018089294
  y: 0.20777574181556702
  z: -0.03507254645228386
}
landmark {
  x: 0.830237865447998
  y: 0.3101172149181366
  z: 0.048747479915618896
}
landmark {
  x: 0.8616198301315308
  y: 0.29404959082603455
  z: 0.12381526082754135
}
landmark {
  x: 0.5572219491004944
  y: 0.702317476272583
  z: -0.10649286210536957
}
landmark {
  x: 0.6097401976585388
  y: 0.7025388479232788
  z: -0.08312451094388962
}
landmark {
  x: 0.6494945883750916
  y: 0.7038677930831909
  z: -0.04906775429844856
}
landmark {
  x: 0.5753908157348633
  y: 0.6119466423988342
  z: -0.10112249851226807
}
landmark {
  x: 0.7044703364372253
  y: 0.716620683670044
  z: 0.023401401937007904
}
landmark {
  x: 0.6838364601135254
  y: 0.7439297437667847
  z: 0.006240763235837221
}
landmark {
  x: 0.6874526739120483
  y: 0.7196060419082642
  z: 0.018882296979427338
}
landmark {
  x: 0.5566620826721191
  y: 0.6008176207542419
  z: -0.16197162866592407
}
landmark {
  x: 0.630964994430542
  y: 0.7098885178565979
  z: -0.036925382912158966
}
landmark {
  x: 0.5949345827102661
  y: 0.7120267152786255
  z: -0.062326859682798386
}
landmark {
  x: 0.5532914996147156
  y: 0.7164126634597778
  z: -0.08051052689552307
}
landmark {
  x: 0.5808660387992859
  y: 0.8571116924285889
  z: -0.031214017421007156
}
landmark {
  x: 0.5760382413864136
  y: 0.8280496001243591
  z: -0.06238376349210739
}
landmark {
  x: 0.571864902973175
  y: 0.8095350861549377
  z: -0.06722287088632584
}
landmark {
  x: 0.5670864582061768
  y: 0.7940603494644165
  z: -0.05826123431324959
}
landmark {
  x: 0.5617002248764038
  y: 0.7833285927772522
  z: -0.05180101841688156
}
landmark {
  x: 0.6394295692443848
  y: 0.7579297423362732
  z: -0.014121092855930328
}
landmark {
  x: 0.6471433639526367
  y: 0.7607876062393188
  z: -0.0198096614331007
}
landmark {
  x: 0.6577990651130676
  y: 0.7677105665206909
  z: -0.024386810138821602
}
landmark {
  x: 0.6662388443946838
  y: 0.77713942527771
  z: -0.01703520677983761
}
landmark {
  x: 0.681681215763092
  y: 0.6677284836769104
  z: -0.04070958122611046
}
landmark {
  x: 0.9427776336669922
  y: 0.568905770778656
  z: 0.3679347634315491
}
landmark {
  x: 0.6616540551185608
  y: 0.7447880506515503
  z: 0.00876859575510025
}
landmark {
  x: 0.6731821298599243
  y: 0.7429101467132568
  z: 0.005481828935444355
}
landmark {
  x: 0.5410251021385193
  y: 0.6335012912750244
  z: -0.11239670217037201
}
landmark {
  x: 0.6075263619422913
  y: 0.6156387329101562
  z: -0.07554959505796432
}
landmark {
  x: 0.5455762147903442
  y: 0.6267488598823547
  z: -0.11317762732505798
}
landmark {
  x: 0.6382776498794556
  y: 0.5067825317382812
  z: -0.03245118632912636
}
landmark {
  x: 0.7076281905174255
  y: 0.5195570588111877
  z: -0.026095494627952576
}
landmark {
  x: 0.6209862232208252
  y: 0.5803111791610718
  z: -0.09298138320446014
}
landmark {
  x: 0.7698853611946106
  y: 0.17035594582557678
  z: 0.06264714896678925
}
landmark {
  x: 0.7429320812225342
  y: 0.2153264582157135
  z: 0.008848011493682861
}
landmark {
  x: 0.7155844569206238
  y: 0.26735252141952515
  z: -0.034542590379714966
}
landmark {
  x: 0.6803761720657349
  y: 0.796616792678833
  z: 0.005564854014664888
}
landmark {
  x: 0.545661449432373
  y: 0.29406824707984924
  z: -0.0806029886007309
}
landmark {
  x: 0.5515839457511902
  y: 0.2136651575565338
  z: -0.05765911936759949
}
landmark {
  x: 0.5609226226806641
  y: 0.13781699538230896
  z: -0.034965626895427704
}
landmark {
  x: 0.7430508732795715
  y: 0.4281330108642578
  z: 0.022452041506767273
}
landmark {
  x: 0.844697117805481
  y: 0.4517696499824524
  z: 0.07315035909414291
}
landmark {
  x: 0.5802938938140869
  y: 0.4321155548095703
  z: 0.010565190576016903
}
landmark {
  x: 0.7901830077171326
  y: 0.3747103214263916
  z: 0.04007326439023018
}
landmark {
  x: 0.5615787506103516
  y: 0.4760693609714508
  z: -0.053956516087055206
}
landmark {
  x: 0.5785065293312073
  y: 0.5778008699417114
  z: -0.16147805750370026
}
landmark {
  x: 0.8918441534042358
  y: 0.4735209345817566
  z: 0.11267603933811188
}
landmark {
  x: 0.8184541463851929
  y: 0.4718348979949951
  z: 0.03932137042284012
}
landmark {
  x: 0.768517255783081
  y: 0.4847654700279236
  z: 0.007178612053394318
}
landmark {
  x: 0.6929854154586792
  y: 0.48831745982170105
  z: -0.006274426821619272
}
landmark {
  x: 0.6349441409111023
  y: 0.48269224166870117
  z: -0.011449138633906841
}
landmark {
  x: 0.591370165348053
  y: 0.4720451235771179
  z: -0.01922423392534256
}
landmark {
  x: 0.5042835474014282
  y: 0.4369489550590515
  z: -0.09103569388389587
}
landmark {
  x: 0.8984358310699463
  y: 0.5377718210220337
  z: 0.10063814371824265
}
landmark {
  x: 0.8244988918304443
  y: 0.3667304515838623
  z: 0.05605992674827576
}
landmark {
  x: 0.5030688643455505
  y: 0.62248295545578
  z: -0.19541682302951813
}
landmark {
  x: 0.5940446257591248
  y: 0.5232166647911072
  z: -0.061043690890073776
}
landmark {
  x: 0.9397686123847961
  y: 0.41795897483825684
  z: 0.3411884307861328
}
landmark {
  x: 0.5593149065971375
  y: 0.4572816789150238
  z: -0.02309775911271572
}
landmark {
  x: 0.6279747486114502
  y: 0.5811028480529785
  z: -0.052767619490623474
}
landmark {
  x: 0.7707387208938599
  y: 0.39914965629577637
  z: 0.05126340687274933
}
landmark {
  x: 0.5788552165031433
  y: 0.5565616488456726
  z: -0.14665107429027557
}
landmark {
  x: 0.9362665414810181
  y: 0.6493462920188904
  z: 0.34945130348205566
}
landmark {
  x: 0.580301821231842
  y: 0.4213067293167114
  z: 0.020215287804603577
}
landmark {
  x: 0.5423721075057983
  y: 0.5413954854011536
  z: -0.18082544207572937
}
landmark {
  x: 0.827576756477356
  y: 0.8029569983482361
  z: 0.13183309137821198
}
landmark {
  x: 0.826129674911499
  y: 0.847174346446991
  z: 0.19045203924179077
}
landmark {
  x: 0.9401122331619263
  y: 0.5557646751403809
  z: 0.2341695874929428
}
landmark {
  x: 0.8798336982727051
  y: 0.7598730325698853
  z: 0.16860802471637726
}
landmark {
  x: 0.8934718370437622
  y: 0.35113173723220825
  z: 0.18171405792236328
}
landmark {
  x: 0.6714124083518982
  y: 0.9213674068450928
  z: 0.0226733461022377
}
landmark {
  x: 0.5012639164924622
  y: 0.6293689608573914
  z: -0.14582233130931854
}
landmark {
  x: 0.6347501277923584
  y: 0.5416210889816284
  z: -0.049733251333236694
}
landmark {
  x: 0.8785412311553955
  y: 0.41621002554893494
  z: 0.12761190533638
}
landmark {
  x: 0.7031115293502808
  y: 0.4189247786998749
  z: 0.01105754915624857
}
landmark {
  x: 0.6701067686080933
  y: 0.4246559143066406
  z: 0.003528991714119911
}
landmark {
  x: 0.693998396396637
  y: 0.747170627117157
  z: 0.011281153187155724
}
landmark {
  x: 0.904225766658783
  y: 0.6057406663894653
  z: 0.11217426508665085
}
landmark {
  x: 0.6056143641471863
  y: 0.968888521194458
  z: 0.029406843706965446
}
landmark {
  x: 0.7156542539596558
  y: 0.9208263158798218
  z: 0.10170795768499374
}
landmark {
  x: 0.7654732465744019
  y: 0.8901197910308838
  z: 0.14202842116355896
}
landmark {
  x: 0.6382739543914795
  y: 0.4263345003128052
  z: 0.003714349353685975
}
landmark {
  x: 0.608167827129364
  y: 0.42451217770576477
  z: 0.01025357935577631
}
landmark {
  x: 0.5894688963890076
  y: 0.4233325719833374
  z: 0.01901676133275032
}
landmark {
  x: 0.8561718463897705
  y: 0.35812079906463623
  z: 0.09244342148303986
}
landmark {
  x: 0.610427737236023
  y: 0.40064048767089844
  z: 0.004954040050506592
}
landmark {
  x: 0.6419333219528198
  y: 0.39145931601524353
  z: -0.0035158481914550066
}
landmark {
  x: 0.6723035573959351
  y: 0.38722050189971924
  z: -0.0033594227861613035
}
landmark {
  x: 0.7050137519836426
  y: 0.38829874992370605
  z: 0.004632687196135521
}
landmark {
  x: 0.7280253767967224
  y: 0.3929325044155121
  z: 0.0157832819968462
}
landmark {
  x: 0.922637403011322
  y: 0.3383057117462158
  z: 0.2819123864173889
}
landmark {
  x: 0.7254979610443115
  y: 0.41271597146987915
  z: 0.021820107474923134
}
landmark {
  x: 0.636895477771759
  y: 0.6547951698303223
  z: -0.06942592561244965
}
landmark {
  x: 0.5817331075668335
  y: 0.6023143529891968
  z: -0.12369497120380402
}
landmark {
  x: 0.5518733859062195
  y: 0.6538554430007935
  z: -0.10611211508512497
}
landmark {
  x: 0.7720415592193604
  y: 0.8480919003486633
  z: 0.10292784124612808
}
landmark {
  x: 0.7224955558776855
  y: 0.8843857049942017
  z: 0.06571002304553986
}
landmark {
  x: 0.6056313514709473
  y: 0.949116587638855
  z: -0.012368883937597275
}
landmark {
  x: 0.8745077848434448
  y: 0.7999281883239746
  z: 0.251703143119812
}
landmark {
  x: 0.5891892910003662
  y: 0.4131283760070801
  z: 0.013970219530165195
}
landmark {
  x: 0.5339846611022949
  y: 0.4853363633155823
  z: -0.10536916553974152
}
landmark {
  x: 0.6645001769065857
  y: 0.9482040405273438
  z: 0.05757834389805794
}
landmark {
  x: 0.9371163845062256
  y: 0.6303572654724121
  z: 0.22724252939224243
}
landmark {
  x: 0.6042358875274658
  y: 0.7720387578010559
  z: -0.035778649151325226
}
landmark {
  x: 0.612248957157135
  y: 0.7790498733520508
  z: -0.04143768548965454
}
landmark {
  x: 0.6202538013458252
  y: 0.7911738753318787
  z: -0.04681113734841347
}
landmark {
  x: 0.6267709732055664
  y: 0.8070510029792786
  z: -0.040287576615810394
}
landmark {
  x: 0.6387888193130493
  y: 0.831239640712738
  z: -0.016214247792959213
}
landmark {
  x: 0.6726184487342834
  y: 0.7126936912536621
  z: -0.00914271455258131
}
landmark {
  x: 0.6843034029006958
  y: 0.7081231474876404
  z: -0.008467559702694416
}
landmark {
  x: 0.6945748329162598
  y: 0.7022805213928223
  z: -0.009355273097753525
}
landmark {
  x: 0.7200194597244263
  y: 0.6861039400100708
  z: -0.006583949085325003
}
landmark {
  x: 0.8542413711547852
  y: 0.6205570101737976
  z: 0.038892485201358795
}
landmark {
  x: 0.5316597819328308
  y: 0.45482513308525085
  z: -0.06983059644699097
}
landmark {
  x: 0.5485436916351318
  y: 0.3964008688926697
  z: -0.0035277754068374634
}
landmark {
  x: 0.5719646215438843
  y: 0.40180742740631104
  z: 0.009555423632264137
}
landmark {
  x: 0.6626864671707153
  y: 0.7099243998527527
  z: -0.0063500674441456795
}
landmark {
  x: 0.8646278977394104
  y: 0.7047960758209229
  z: 0.100004643201828
}
landmark {
  x: 0.5146359801292419
  y: 0.39463502168655396
  z: -0.0523100420832634
}
landmark {
  x: 0.6529819369316101
  y: 0.8578700423240662
  z: -0.0028281020931899548
}
landmark {
  x: 0.5042141675949097
  y: 0.4702150225639343
  z: -0.129316508769989
}
landmark {
  x: 0.5673672556877136
  y: 0.5313471555709839
  z: -0.10971097648143768
}
landmark {
  x: 0.5904504060745239
  y: 0.885657787322998
  z: -0.0272047258913517
}
landmark {
  x: 0.7341678738594055
  y: 0.7744194865226746
  z: 0.03884495422244072
}
landmark {
  x: 0.6638760566711426
  y: 0.6042520999908447
  z: -0.04157112166285515
}
landmark {
  x: 0.6976938247680664
  y: 0.8204447031021118
  z: 0.02156316302716732
}
landmark {
  x: 0.7590912580490112
  y: 0.5951480865478516
  z: -0.02994963340461254
}
landmark {
  x: 0.7086312174797058
  y: 0.6325660347938538
  z: -0.028385769575834274
}
landmark {
  x: 0.8003910779953003
  y: 0.6445255279541016
  z: 0.004138569813221693
}
landmark {
  x: 0.6013050675392151
  y: 0.9202969074249268
  z: -0.027641113847494125
}
landmark {
  x: 0.5956445336341858
  y: 0.5460949540138245
  z: -0.08496230840682983
}
landmark {
  x: 0.7609846591949463
  y: 0.8049650192260742
  z: 0.06619337201118469
}
landmark {
  x: 0.713670015335083
  y: 0.8484628796577454
  z: 0.04197729751467705
}
landmark {
  x: 0.7646187543869019
  y: 0.7252383828163147
  z: 0.034109391272068024
}
landmark {
  x: 0.899861216545105
  y: 0.6650470495223999
  z: 0.1269931197166443
}
landmark {
  x: 0.811811089515686
  y: 0.737910807132721
  z: 0.0630282610654831
}
landmark {
  x: 0.9194583296775818
  y: 0.6992863416671753
  z: 0.2123991996049881
}
landmark {
  x: 0.7495314478874207
  y: 0.6656218767166138
  z: -0.005609575193375349
}
landmark {
  x: 0.5628350973129272
  y: 0.5045331716537476
  z: -0.0812133252620697
}
landmark {
  x: 0.5678873062133789
  y: 0.5942215919494629
  z: -0.16940903663635254
}
landmark {
  x: 0.5977365374565125
  y: 0.597644031047821
  z: -0.12983109056949615
}
landmark {
  x: 0.5464522242546082
  y: 0.5735775828361511
  z: -0.194813534617424
}
landmark {
  x: 0.5745513439178467
  y: 0.36621636152267456
  z: -0.015208940021693707
}
landmark {
  x: 0.6327033042907715
  y: 0.341604083776474
  z: -0.023593256250023842
}
landmark {
  x: 0.6859936118125916
  y: 0.3331960439682007
  z: -0.023968541994690895
}
landmark {
  x: 0.7332467436790466
  y: 0.33573848009109497
  z: -0.011009084060788155
}
landmark {
  x: 0.7689438462257385
  y: 0.348848432302475
  z: 0.011183288879692554
}
landmark {
  x: 0.8013812899589539
  y: 0.40506184101104736
  z: 0.06539134681224823
}
landmark {
  x: 0.9359502792358398
  y: 0.48567837476730347
  z: 0.22824189066886902
}
landmark {
  x: 0.7763141393661499
  y: 0.44680675864219666
  z: 0.035724811255931854
}
landmark {
  x: 0.7362672686576843
  y: 0.4579707980155945
  z: 0.01578447036445141
}
landmark {
  x: 0.6839812994003296
  y: 0.4636895954608917
  z: 0.002962634898722172
}
landmark {
  x: 0.6337649822235107
  y: 0.46220487356185913
  z: -0.001766429515555501
}
landmark {
  x: 0.5947312116622925
  y: 0.45500612258911133
  z: -0.0008954411023296416
}
landmark {
  x: 0.5681023001670837
  y: 0.4459196925163269
  z: -0.00256124185398221
}
landmark {
  x: 0.9421491622924805
  y: 0.4931836724281311
  z: 0.3675062358379364
}
landmark {
  x: 0.5998513698577881
  y: 0.6042654514312744
  z: -0.11095136404037476
}
landmark {
  x: 0.5358161330223083
  y: 0.5127532482147217
  z: -0.13812489807605743
}
landmark {
  x: 0.5388860106468201
  y: 0.5981953740119934
  z: -0.19681857526302338
}
landmark {
  x: 0.5211951732635498
  y: 0.6159141659736633
  z: -0.1806589514017105
}
landmark {
  x: 0.5407022833824158
  y: 0.6038531064987183
  z: -0.17878368496894836
}
landmark {
  x: 0.5928868651390076
  y: 0.6134583950042725
  z: -0.09697843343019485
}
landmark {
  x: 0.5145755410194397
  y: 0.6193801164627075
  z: -0.190590038895607
}
landmark {
  x: 0.5130746960639954
  y: 0.6262079477310181
  z: -0.14627952873706818
}
landmark {
  x: 0.5662755370140076
  y: 0.42279717326164246
  z: 0.015079694800078869
}
landmark {
  x: 0.5467715263366699
  y: 0.4294167459011078
  z: -0.006547094322741032
}
landmark {
  x: 0.5349453687667847
  y: 0.4338741898536682
  z: -0.032735634595155716
}
landmark {
  x: 0.7428998947143555
  y: 0.39666956663131714
  z: 0.02821102924644947
}
landmark {
  x: 0.7656942009925842
  y: 0.37987834215164185
  z: 0.02971372753381729
}

Tracking caveats

As of v0.8.7 of Face Mesh, it seems they don't support asymmetric facial expressions (raising only 1 eyebrow or only the right corner of your lips).

Extra info

How to get the landmark coordinates out of the face mesh object: https://stackoverflow.com/questions/67141844/how-do-i-get-the-coordinates-of-face-mash-landmarks-in-mediapipe

Next steps

1. The creation of a mapping between the 468 3D coordinates (landmarks) output of Face Mesh to AU values (OpenFace uses: 1, 2, 4, 5, 6, 7, 9, 10, 12, 14, 15, 17, 20, 23, 25, 26, 28, and 45).

This might be a research project on itself. Some thoughts:

• Manual approach: I imagine the cleanest data would be mask generated from a picture of a resting face and a mask generated from a picture with 1 AU at maximum value (maximum muscle contraction of that muscle group). Or a video of a person moving a single AU, but that seems hard to map by hand.
• Machine learning approach: Generate data by running this solution on videos annotated with AU values. Then creating a ML model that learns the mapping from landmarks to AU values.

Unable to link app with Unity

Hello,
I am facing wierd issue. The offline mode works fine but when I try to run it real time, it doesn't seem to connect with openface app. There is error showing anywhere whatsoever, still not working. Need some guidance.

Thanks in advance

Documentation suggestions

Hi NumesSanguis

This project looks great!

May I suggest the following documentation changes in the How to Run guide?
1: Due to the use of function async Python version 3.5 or greater is required
2: you include six and pandas libraries as python dependencies
3: user to load Unity Scene MB_demo_asian_female_ready
4: Terminal: python pub_facs.py (/modules/01_facs-from-csv/) be moved to Step 4 (it runs before Terminal: python pub_blend.py (/modules/02_facs-to-blendshapes/) is loaded)

Cheers!

Problem with face bones

Hello, i've tried with a custom model with blend shapes for the face (and with another with bones), but it only moves the head, it doesnt change the eyes, mouth etc

FACSvatar with Blender 3.3

Hi Numes,

I'm unable to run the FACSvatar_zeromq.py script from blender while the same script and pyzmq version works well with blender 2.79.

I'm getting the following error:
Traceback (most recent call last): File "<blender_console>", line 1, in <module> File "/home/mon/FACSvatar/blender/facsvatar_zeromq.py", line 9, in <module> import zmq File "/home/mon/anaconda3/envs/blender_demo/lib/python3.5/site-packages/zmq/backend/cython/__init__.py", line 6, in <module> from . import (constants, error, message, context, ImportError: cannot import name 'constants' from partially initialized module 'zmq.backend.cython' (most likely due to a circular import) (/home/mon/anaconda3/envs/blender_demo/lib/python3.5/site-packages/zmq/backend/cython/__init__.py)

I've tried multiple things to resolve this but with no success.

Has anyone tried using FACSvatar visualization with newer versions of blender(3.3 or any other).

Please let me know if you're using FACSvatar with blender and which version of pyzmq??

Thanks

Some confusion about n_array

Hi NumesSanguis
I‘m learning the bridge module about how to smooth data.I notice that in softmax_smooth2 function the first number of n_array is set to 1.But in facsvatar paper, the smoothing formula shows n start at 0.I wonder about the difference between the two ways.Hope you could give me some suggestions.
Thanks again.

Hi author

Help Getting AUs Into Blender

Hi Numes
Many thanks for your Blender addon. I can now get Blender objects moving as you described but I cannot get Facsvatar to input au’s into Blender. I have the three Facsvatar env terminals going as in your latest Blender add-on video (plus bzmq and Blender envs) but the data is not showing up in Blender. I notice in the video you have a slightly different panel in Blender titled FACSvatar whereas the version I have is headed BlenderZMQ. Would this make the difference? Also I am not sure what port numbers I should be using. I would appreciate any guidance you can give me to get expressions working in Blender with auto keyframing.
Thanks again and I wish you a very merry Christmas.

Facsdnnfacs for dynamic AUs?

Hi

In the description you wrote:

"Deep Neural Network generation of facial expressions for Human-Agent Interaction (See modules/process_facsdnnfacs)"

I just quickly checked out the module. How is it supposed to work for dynamic facial expression?

I can input emotion label or corresponding Facs list and it outputs me a new more dynamic and "realistic" AU dict kinda realtime?

Thank you for an answer

How should I get python output?

Thank you for your work, which has helped me a lot.
As you know, openface has not provided python binding library yet, but I need to use some data identified by it. I see that your work can already output openface data in python. If I want to use these data in Python, what should I do?

Openface License Information. No Commercial Use Without Purchase, $18k Per Year

I just learned that Openface, which this project depends on, cannot be used for commercial purposes without purchasing a license.
Important points from the license:

The non-exclusive commercial license requires a non-refundable annual royalty 

(USD $10,000 for OpenFace Light,
USD$15,000 for OpenFace Landmark,
USD$18,000 for OpenFace Full Suite: see each offering for details.
The license is non-negotiable.

Information required to complete the license:

Legal company name
State/country of incorporation
Type of corporation
Principal address of corporation
Name and title of person who will sign the license
Title of the person who will sign the license
Address for notices, including name and email of person to be notified

https://cmu.flintbox.com/#technologies/5c5e7fee-6a24-467b-bb5f-eb2f72119e59

Thank you again for making this. I just felt this is an important point to note, as i was rather far into using this when i discovered that i had to pay someone else to be able to use it commercially.

Avatar Face isn't moving at all

Following on from TadasBaltrusaitis/OpenFace#492.

I pasted the MainWindow.xaml.cs straight in and it worked fine, though I do still get the same crash when trying to load webcams so I'll wait for a response on my other issue over there. TadasBaltrusaitis/OpenFace#506

Using cd to change the working directory to the folder containing each of the python files before running them seems to have done the trick. I now get both the n_proxy_m_bus.py and pub_facs.py scrolling through lots of data then eventually stopping, but the head in Unity still doesn't do anything. I get the Debug.Logs in ZeroMQFACSvatar.cs for "Setting up subscriber sock" and "sub socket initiliased", but uncommenting the one for "Received messages:" never gets called.

openface_2.1.0_zeromq generated .csv data incompatibility with input_facsfromcsv

The .csv data generated by openface_2.1.0_zeromg is not compatible with the input_facsfromcsv interpreter. The demo.csv data works perfectly, and captures keyframes. I can also locally stream from OpenFaceOffline to blender directly from a video file, but this method does not capture keyframes

EDIT: this method is only capturing keyframes if you start the OpenFace program after you set the Blender plugin to connected and tick insert keyframes. If the Openface program was open before you set the plugin, it seems as though the signal to capture frames does not get through. So, in order to get key frame capture when streaming a video from OpenFace, it seems you must have blender open and set to connected, and insert key frames checked before you open OpenFace at all.

When trying to send a .csv i generated with the same OpenFace through input_facsfromcsv/main.py, errors are as follows

frame   face_id   timestamp   confidence   success   gaze_0_x  ...   AU20_c   AU23_c   AU25_c   AU26_c   AU28_c   AU45_c
0      1         0       0.259         0.98         1   0.072391  ...      0.0      1.0      0.0      0.0      0.0      0.0
1      2         0       1.411         0.98         1   0.081622  ...      0.0      1.0      0.0      0.0      0.0      0.0
2      3         0       1.446         0.98         1   0.174707  ...      0.0      1.0      0.0      0.0      0.0      0.0
3      4         0       1.494         0.98         1   0.179774  ...      0.0      1.0      0.0      0.0      0.0      0.0
4      5         0       1.535         0.98         1   0.197015  ...      0.0      1.0      0.0      0.0      0.0      0.0

[5 rows x 714 columns]
Data rows in data frame: 2579
C:\Users\MyAccountName\anaconda3\lib\site-packages\pandas\core\frame.py:4133: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  errors=errors,
FRAME TRACKER: 0
ERROR:asyncio:Task exception was never retrieved
future: <Task finished coro=<FACSvatarMessages.facs_pub() done, defined at main.py:382> exception=KeyError('timestamp')>
Traceback (most recent call last):
  File "C:\Users\MyAccountName\anaconda3\lib\site-packages\pandas\core\indexes\base.py", line 4411, in get_value
    return libindex.get_value_at(s, key)
  File "pandas\_libs\index.pyx", line 44, in pandas._libs.index.get_value_at
  File "pandas\_libs\index.pyx", line 45, in pandas._libs.index.get_value_at
  File "pandas\_libs\util.pxd", line 98, in pandas._libs.util.get_value_at
  File "pandas\_libs\util.pxd", line 83, in pandas._libs.util.validate_indexer
TypeError: 'str' object cannot be interpreted as an integer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 388, in facs_pub
    async for msg in self.openface_msg.msg_gen():
  File "main.py", line 283, in msg_gen
    async for i, msg in self.msg_from_csv(csv_group):
  File "main.py", line 343, in msg_from_csv
    ofmsg.set_msg(frame_tracker)
  File "main.py", line 200, in set_msg
    self.msg['timestamp'] = row['timestamp']
  File "C:\Users\MyAccountName\anaconda3\lib\site-packages\pandas\core\series.py", line 871, in __getitem__
    result = self.index.get_value(self, key)
  File "C:\Users\MyAccountName\anaconda3\lib\site-packages\pandas\core\indexes\base.py", line 4419, in get_value
    raise e1
  File "C:\Users\MyAccountName\anaconda3\lib\site-packages\pandas\core\indexes\base.py", line 4405, in get_value
    return self._engine.get_value(s, k, tz=getattr(series.dtype, "tz", None))
  File "pandas\_libs\index.pyx", line 80, in pandas._libs.index.IndexEngine.get_value
  File "pandas\_libs\index.pyx", line 90, in pandas._libs.index.IndexEngine.get_value
  File "pandas\_libs\index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
  File "pandas\_libs\hashtable_class_helper.pxi", line 1618, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas\_libs\hashtable_class_helper.pxi", line 1626, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'timestamp'

Thank you, and all who are involved in this project and OpenFace itself, very much for your talent and hard work. This really is an amazing project with top tier performance.

How avatar's setting in Make Human (FACS Human)

I want to use the avatar made by me .
I installed Make Human and FACS Human plugin .
However there is a lot of choices (for example , bone type) .
For my reference , I want to know what setting do you use .

get face point

This is a great project. I wonder can I only get AUs from openface with zeromq? Can I get 68 face points from it?

Hello, could you upload training data to github? Microsoft Onedriver and Dropbox are very difficult to download.

Hello, could you upload training data to github? Microsoft Onedriver and Dropbox are very difficult to download.

[FeatureRequest] Input: csv & fbx -> Output fbx_blendshape

Please create a cli where you input your csv and your face-fbx or body+face-fbx and it outputs a fbx with blendshapes applied, with rotating eyes and head movement data.

Alternatively do it with dae.

Can I fix fps of OpenFaceoffline ?

Frame rate of OpenFaceOffline is depend on movie's quality so I open a movie with OpenFaceOffline but it has possible to be longer movie than original one .
For example , I open 5 seconds movie with OpenFaceOffline , but it's low fps so avatar's face moves more than 5 seconds as a result .
Can I fix fps ?

Is this cpu based or gpu?

As far as I understand docker does not use your native gpu. With the exception of nvidia docker. Would there be a big difference in performance and accuracy using GPU?

How can I save the data captured by camera through OpenFaceOffline into a csv file?

Will there be a windows stand-alone for Blender?

I see the standalone for Unity. Do you use this version in blender as well. I would like to run the windows version to the blender add-on without having to install docker.

Thanks

Feature Request: Reallusion iClone and Character Creator 3 support

I am still trying to wrap my head around these types of programs but it would be great to have the ability to use any Character Creator 3 character and create facial animations that will work with that character in iClone.

https://www.reallusion.com/character-creator/ like MB-Lab

https://www.reallusion.com/iclone/ animation product that uses Character Creator 3 characters

I am not a really a programmer so setting up these types of projects but it would be nice to be able to use this with your own Characters or Characters from Daz,CC3,Poser. A one click import into which ever app you want to use. I can't help with the project from a programmers point of view but I would be willing to donate a few dollars for continued improvements. Ultimately I would like to be able to use FACSvatar, Openpose Openface or combination to easily create full body mocap from a video or webcam that can be used with any program or character. As a solo creator and newbie to 3D I can't afford a thousand dollars or more to get this functionality. So if I can help in any way to continue this project and hopefully expand to full body mocap please let me know!!

Thanks for the awesome work!

Error running MB_demo_asian_femaile_ready scene

Hi NumesSanguis

I get the following 715 errors when using How to Run directions

JsonReaderException: Unexpected character encountered while parsing value: h. Path '', line 0, position 0.
Newtonsoft.Json.JsonTextReader.ParseValue () (at <97722d3abc9f4cf69f9e21e6770081b3>:0)
Newtonsoft.Json.JsonTextReader.Read () (at <97722d3abc9f4cf69f9e21e6770081b3>:0)
Newtonsoft.Json.Linq.JObject.Load (Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Linq.JsonLoadSettings settings) (at <97722d3abc9f4cf69f9e21e6770081b3>:0)
Newtonsoft.Json.Linq.JObject.Parse (System.String json, Newtonsoft.Json.Linq.JsonLoadSettings settings) (at <97722d3abc9f4cf69f9e21e6770081b3>:0)
Newtonsoft.Json.Linq.JObject.Parse (System.String json) (at <97722d3abc9f4cf69f9e21e6770081b3>:0)
ZeroMQFACSvatar.HandleMessage (System.String subMsg) (at Assets/Scripts/ZeroMQFACSvatar.cs:85)
NetMqListener.Update () (at Assets/Scripts/ZeroMQFACSvatar.cs:45)
ZeroMQFACSvatar.Update () (at Assets/Scripts/ZeroMQFACSvatar.cs:104)

Windows 10 Home
Unity 2017.3.1f1
Current libzmq version is 4.1.6
Current pyzmq version is 17.0.0
Python 3.6.5

Her head moves but she looks very unwell :)

Please let me know if I can provide any further information.

Cheers

Blender avatar is not receiving data.

Background rotation in visualizing pose angles

I am trying to visualize videos from Openface using pose angles.
The issue I'm having is that the background also rotates along the character's face. How can I stop the background rotation and visualize only character head motion?
I'd be thankful for any help in this regard.

Can FACSvatar be used for lip sync?

Hi NumesSanguis

I hope it is OK to ask questions here? If not I can go to Blender Artists or what ever is better.

Have you tried using FACSvatar for real time lip sync facial motion capture?

I am not sure if I understand that FACSvatar can be used in real time or if the output from OpenFace must be processed by the Python scripts first before routing the output to the Game Engine.

Cheers

How to add more AUs?

Hi NumesSanguis,
Thanks for your sharing.
To realize more detailed expressions of the avatar in Unity 3D，I think I need to transition more AUs to Blendshapes，which means more AU0X.json in file FACSvatar-master\modules\process_facstoblend\au_json (only 20 .json files there).
if I thought right，where can I get more AU0X.json files，e.g AU07.json ？Or should I write these files manually？if so, is there any mapping laws between AUs and blendshapes that I can follow at?
Thanks again.

No json file found for AU07

Hi all,

Not sure if anyone is still checking this repos, but after installing OpenFace (works!) I see in the console that modules-facsvatar_facstoblend-1 reports: No json file found for AU07.
When looking in the AU_predictor folders I indeed do not see AU07 files in the 'disfa' folders.

Not having the detection of AU07 happers the detection as the Anger expression relies heavily on AU07.

Anyone know if I can find/add the missing files?

Cheers!

How to read csv output by OpenFaceOffline ?

OpenFaceOffline can output csv as a result of face recognition .
I can manipulate avatar's face on Unity3D by reading csv with main.py .
However , csv output by OpenFaceOffline has too argument to use main.py .

1. Can I make main.py read csv output by OpenFaceOffline without format by change the code of main.py ?
2. Is it same that csv output by OpenFaceOffline and csv can read by main.py (for example , demo.csv has only three gaze parameter , three pose parameter and AU1_r ~ AU45_r , but csv output by OpenFaceOffline has more parameters .) ?

Character model replacement

I hope to replace the model with something else, what work do I need to do?
I'd be thankful for any help in this regard.

How to apply it to my webcam ?

Hi NumesSanguis

I have followed the instruction 'simple how to run' but I don't know how to apply it to my webcam.
So, I wanna get advice.