shaoanlu / gazeml-keras

A keras port of swook/GazeML for pupil, iris and eye-lid detection

👋Hello, I'm Shao-An!

• 🔭 Working as a Control Software Engineer.
• 🌏 Grew up in Taiwan, residing in the vibrant city of Tokyo, Japan.
• ⭐ Interested in Control Systems, Optimization, and Deep Generative Models.
• 🌱 Currently engaged in the exploration of Software Architecture within Robotic Systems.

Recent Activity

1. 🎉 Merged PR #1 in shaoanlu/CBF_QP_safety_filter
2. 💪 Opened PR #1 in shaoanlu/CBF_QP_safety_filter
3. 💪 Opened PR #234 in qpsolvers/qpsolvers
4. 🎉 Merged PR #6 in shaoanlu/qpsolvers
5. 💪 Opened PR #6 in shaoanlu/qpsolvers

Projects

• Control and Optimization
  • Diffusion policy for Quadrotor Control: Implements cutting-edge diffusion models for imitation learning in quadrotor control.
  • Safety Filter for Collision Avoidance: Employs Control Barrier Functions (CBF) as safety filters for collision avoidance.
  • Motion Planning with MPPI: Integrates Model Predictive Control and CBF in a sampling-based approach for motion planning.
• Deep Learning and Computer Vision
  • Few-Shot Face Translation: Features Generative Adversarial Networks trained on the VGGFace2 dataset for face-translation with few-shot examples.
  • Face Swapping GAN: Applies Generative Adversarial Networks for face-swapping in images.
  • Facial Analysis Toolbox: Offers a collection of deep learning models for facial analysis in Keras.

gazeml-keras's Issues

Perfomance diff bw GazeML vs GazeML-Keras?

Hi .

Thanks for great work.

Since I am beginner, I would like to know what is the performance diff bw this GazeML-Keras vs GazeML(swook).

Is there any significant reason to choose this repo?

please advise.

Also, I convert demo_colab.ipynb to .py file and then run on the ubuntu But it seems Not running.

Gaze Estimation Part

Hi,

Thank you for sharing code. But in demo, you only generate 18 heatmaps of eyes.
How to calculate gaze directions via 18 heatmaps?

I found part of code which should be useful: https://github.com/swook/GazeML/blob/master/src/elg_demo.py#L174

Where is the output of `eye_radius`?

Hi,

According to GazeML's Implementation, the output of network should be 18 heatmaps and 1 eye_radius. https://github.com/swook/GazeML/blob/master/src/models/elg.py#L123

Did you implement this in your keras model?

Thank you!

[REQUEST]Update to TF3

Hello,
Would update the code to use TensorFlow 3?

How to get gaze estimation from landmarks?

i have 4 questions about the paper:
1.The original paper says that the author trained a SVR model from MPIIGaze datasite. How to do with that? Useing the GazeML model to predict landmarks on MPII Gaze data images and use the detection landmarks results as the ground truth to train a SVR?
2.why not directly using Unityeyes dataset 's landmark and its gaze vector ground truth to train the SVR?
3.why use SVR? using several FC layers to regress result is ok or not?
4. what does the "calabration with 20 or more samples" mean in author's paper, calabrating to get
camera paramters or get what? don't understand exactlly.

i would appreciate very very very much if you could answer my question,thank you.

Training script

Can you provide the training script for this demo?

AttributeError: module 'keras.backend' has no attribute 'get_session'

When trying to run the demo Colaboratory getting this error:

AttributeError: module 'keras.backend' has no attribute 'get_session'

What is the license for this project?

Could you please update a License for this project?

The previous project GazeML use MIT license. Do you think that we can continue to research on this project with MIT license?

Thanks.

How to convert trained model from GazeML to Keras?

Hi @shaoanlu ,
sorry for disturbing. I am wondering that how to convert trained model from GazeML to Keras, Like you did in this repo? I currently trained a model from GazeML, but I would like to test in your code.
Could you guide me how to achieve this?
Thank you.