strange shape question about tf_flame

[TimoBolkart]

Sorry for the late response. I did play with the landmarks a bit. The reason for the strage shape is that the model priors of shape / expression are too low, so the landmarks are tightly fit, but the reconstructed shape is unrealistic.

One thing to get better eye blinks reconstruction, you can use an additional loss to measure the distance between the upper / lower eyelid landmarks for the projected model landmarks and the target landmarks, and minimizing the difference between them. The loss (i.e. lmk_eye_dist) would be something like that:

'model_lmks_dist = tf.subtract(tf.gather(lmks_proj_2d, [20, 21, 26, 27]), tf.gather(lmks_proj_2d, [24, 23, 30, 29])) target_lmks_dist = target_2d_lmks[[20, 21, 26, 27]] - target_2d_lmks[[24, 23, 30, 29]] lmk_eye_dist = 1.0*tf.reduce_sum(tf.square(tf.subtract(model_lmks_dist, target_lmks_dist))) / (factor ** 2)`

Another problem is that while FLAME is able to blink with the eyelids, but the blink of both eyes is quite correlated in the training data. Therefore, blinking only with one eye is difficult. You could use these additive blendshapes for left eye and right eye with weight [0,1] to close either eye for the FLAME template.

[TimoBolkart]

Can you share the set of landmarks? Then I can have a look into it.

[flynnamy]

Hi,please try this landmarks file! @TimoBolkart
wink.zip

[flynnamy]

Hi, do you try this landmark file?Any suggestions? @TimoBolkart