Alignment: Corner case when fiducial points are outside of the image about openface

[bamos]

Hi @Strateus - why is there a big probability the points will be outside the range? Do you have examples where this is failing for reasonable images? I haven't extensively profiled or tested this portion of code, but the alignment code works for 13081 out of 13233 images from the LFW dataset.

This code portion checks if the boundary points of the detected landmarks are outside of the image, which should only happen in extreme cases like if somebody's face is outside of the image. There may be heuristics that can improve this, like filling in the missing portions with black pixels or using image completion.

[Strateus]

Well, i used my web camera, and added 30% margins to face, and still after warping there were points out of warped image all the time i tried, maybe 5% it was ok, but rest failed.
Anyways i would consider this check to be more thorough since you transform only on eyes and nose. Just thoughts, i elminated this check locally and will be testing it.

[bamos]

Code reference:

openface/openface/alignment/naive_dlib.py

Line 140 in 3dba58e

if 0 <= l <= w and 0 <= r <= w and 0 <= b <= h and 0 <= t <= h:

[bamos]

Be careful when expanding the margins since the neural network nn4.v1.t7 is trained with no margin added. I wouldn't be surprised if this severely hurts the accuracy.

You need to train a new neural network if you want to add margins.
Note that in a few experiments I ran, adding a margin didn't improve the accuracy,
which is why there is no margin in nn4.v1.t7.

Also, won't eliminating the check cause array out of bounds errors in:

cwImg = cv2.resize(warpedImg[t:b, l:r], (size, size))

[Strateus]

Yea i got the point about network. Accuracy is not my primary concern now.
As for eliminating, i did it like this:

    l = 0 if l < 0 else l
    r = w if r > w else r
    t = 0 if t < 0 else t
    b = h if b > h else b

[bamos]

Fixed with the new alignment technique.