yihuacheng / gazetr Goto Github PK

Hi, the GazeTR model outputs the yaw and pitch for a given face image, but if we wanted to calculate the angular error and possibly the PoG, what's the process of doing it, the GazeTR paper doesn't provide details about it (apologies if I missed it). I'm specifically working with the Gaze360 dataset.

I did find the methods "gazeto3d" and "angular" in gtools.py. Am I supposed to input the yaw and pitch to the "gazeto3d" method and then use the output and the 3d gaze ground truth to generate the angular error directly, or is there some step I'm skipping?

Thanks in advance.

Hi, Yihua, thanks for the great job regarding appearance-based gaze-estimation. I have gone through the review paper and codes in Gazehub. It seems that the way to acquireing Rs and Ts from CCS to SCS is not mentioned. It would be appreciated if you could elaborate a bit more on that or provide any reference link or paper. Thanks in advance.

Hi @yihuacheng,
I trained your pre-trained model on MPIIFaceGaze. I haven't made any changes in the script for training as well as pre-processing of dataset. I performed the leave-one-person-out evaluation on this dataset as mentioned in your paper.
I am using PyTorch 1.7.0.
I got the following best angular errors for respective person:

The mean of this best angular errors comes out to be 4.254, which is far away from the reported 4.00 error.
Please let me know if I am missing something over here. Also, help me to reproduce the reported results.

Hi, thanks for the nice and clean repo. I have two questions here.

#1. I am trying to use your code on my own images for other purposes. Since there is no ground truth available, I am wondering whether you could provide the code to draw the red line as you show in the paper figure so that I could approximately evaluate the performance.

#2. For my own images, what is the preprocessing in addition to resizing the face image to 224 x 224?

I find that gaze360's baseline used a LSTM to sum up 7 frames info. But it seems that GazeTR only use only one frame to do that?

Thank you for your great job!
I trained your pre-trained model on RT-Gene. But i find the result is far away from the result of paper,the result is about 13, I used the face image as input to get the gaze angle directly. I want to know how to train on RT-Gene in your trial.

hello，陈老师，
我在下载xgaze的预处理代码，但是显示无法加载pdf，老师是否方便共享下。谢谢。邮箱地址[email protected]

I think many of the gaze estimation related works misunderstand the term.

0. yaw pitch is different from spherical coordinate, so your function is wrong
1. most of the dataset's gaze3d label is from camera's coordinate, so you can't just transform your output to 3d and then calculate the arccos

The training results I obtained using the MPII dataset, such as [80] Total Num: 45000, avg: 0.41229396885316455, how is it concluded that the training result is around four degrees?我用mpii数据集得到的训练结果如[80] Total Num: 45000, avg: 0.41229396885316455，请问是怎样得出训练结果为四度左右的

Hi,

I have some problems about the MPII dataset processing and the use of MPII data in the GazeTR model.

In GazeTR reader.py, you define the decode function for MPII like this:

def Decode_MPII(line):
    anno = edict()
    anno.face, anno.lefteye, anno.righteye = line[0], line[1], line[2]
    anno.name = line[3]

    anno.gaze3d, anno.head3d = line[5], line[6]
    anno.gaze2d, anno.head2d = line[7], line[8]
    return anno

And in the data_processing_mpii.py file you provided, you process and write the annotations into the format like this:

outfile.write("Face Left Right Grid Origin whicheye 2DPoint HeadRot HeadTrans ratio FaceCorner LeftEyeCorner RightEyeCorner\n")

If you indeed used the same code to process the MPII dataset for training the GazeTR model, then this is not right.
You can see that anno.gaze2d = line[7] which is actaully corresponding to HeadRot.

Could you please give some explanation about how to correctly use the data_processing code and how to load the data in GazeTR?

Thank you very much and best regards

In your paper, it is said that "we change the output of GazeTR-Hybrid to perform POG estimation". May I ask How should the output be changed?

Hi, thank you for your work.
I am trying to use your code but I encountered some problems.
The code/link you provided for preprocessing the dataset is not complete. More specifically, the codes provided here: http://phi-ai.buaa.edu.cn/Gazehub/2D-dataset/ is not complete. It lacks some modules, like gtools, which is needed in the data_processing_mpii.py. I suppose the gtools is not a standard python lib, but a file you wrote. I tried to search for the gtools lib on the internet, and installed one, but it doesn't work.

Can you provide the complete code for data processing. Or maybe would it be possible that you directly release your processed datasets?

Thank you and best regards.

Hi, the data preprocessing link for xgaze doesn't seem to exist, where can I get the code for it?

Hi, yihuacheng. I am too poor to train the model of pure ViT using EHT-XGaze. But I want to know the error of ETH-XGaze dataset using pure ViT model. Can you share the Pre-trained pure ViT model using EHT-XGaze? Thank you very much!!!

Thanks for your great work.
The output of GazeTR is yaw and pitch angle in head coordinate system.

• How to calculate the PoG from the yaw and pitch angles?
• For that, how calibrate camera and monitor?

Could you possibly share all details about it?

Thanks in advance.

Thank you for your great work! Would you like to ask if the code can be trained in parallel on multiple Gpus? How do I need to change the code? Looking forward to your reply!