severin-lemaignan / gazr

3D head pose estimation using monocular vision

gazr

gazr is a library and a set of tools for real-time face tracking and gaze estimation from a monocular camera (typically, a webcam) or a RGB-D stream (3D camera).

It natively supports ROS.

Currently, it only performs 6D head pose estimation. Eye orientation based on pupil tracking is being worked on.

If you plan to use this library for academic purposes, we kindly request you to cite our work.

Head pose estimation

This library (libhead_pose_estimation.so) performs 3D head pose estimation based on the fantastic dlib face detector and a bit of OpenCV's solvePnP magic (it uses adult male anthropometric data to match a real 3D head to the projected image).

The library returns a 4x4 transformation matrix.

It supports detection and tracking of multiple faces at the same time, and runs on-line, but it does not provide face identification/recognition.

3D facial features extraction

If provided with an RGB-D (color + depth) stream, the library can extract and compute the 3D localisation of 68 facial landmarks (cf screenshot above).

Note that this feature is currently only available for ROS.

Installation

Note: The library has only been tested on Linux. We can only provide limited support for other operating systems!

Pre-requisites

You need to download and extract Dlib somewhere. This application requires dlib >= 18.18. On Ubuntu 16.04 and above, sudo apt-get install libdlib-dev

You also need OpenCV. On Ubuntu, sudo apt-get install libopencv-dev. The code has been last tested with OpenCV4. Previous versions of OpenCV might or might not work.

Installation

The library uses a standard CMake workflow:

$ mkdir build && cd build
$ cmake -DCMAKE_BUILD_TYPE=Release ..
$ make

Do not forget to enable Release mode, otherwise the performances will be terrible.

You can test the result with:

$ ./gazr_show_head_pose ../share/shape_predictor_68_face_landmarks.dat

Finally, if desired, you can install the library and the executables:

$ make install

By default, only the sample tools are built (CMake parameter WITH_TOOLS=ON). See below for ROS support.

ROS support

Installation

The ROS wrapper provides a convenient node that exposes each detected face as a TF frame.

Before building gazr with the ROS wrapper, make sure that you have installed the following ROS- < distro > package dependencies, where < distro > is the ROS distribution in your machine.

The code has been tested up to ROS noetic, but should work for any ROS1 distribution.

For example, this is the case for ROS-kinetic distribution:

sudo apt-get install ros-kinetic-roscpp ros-kinetic-tf ros-kinetic-std-msgs ros-kinetic-visualization-msgs ros-kinetic-sensor-msgs ros-kinetic-cv-bridge ros-kinetic-image-transport ros-kinetic-image-geometry

The compilation of the ROS wrapper is disabled by default. You can enable it with:

cmake -DWITH_ROS=ON

Usage

Once installed with ROS support, you can launch gazr with a monocular RGB stream with:

$ roslaunch gazr gazr.launch

The estimated TF frames of the heads will then be broadcasted as soon as detected.

The number of detected faces is published on /gazr/detected_faces/count and if gazr has been compiled with the flag DEBUG_OUTPUT=TRUE, then the detected features can be seen on the topic /gazr/detected_faces/image.

To process a depth stream as well, run:

$ roslaunch gazr gazr.launch with_depth:=true

The facial features are published as a PointCloud2 message on the /gazr/facial_features topic.

You can get the full list of arguments by typing:

$ roslaunch gazr gazr.launch --ros-args

Importantly, you might want to remap the rgb and depth topics to your liking.

Stand-alone tools

Example - show head pose

Run ./gazr_show_head_pose ../share/shape_predictor_68_face_landmarks.dat to test the library. You should get something very similar to the picture above.

Example - estimate head pose on image/images

Run ./gazr_estimate_head_pose ../share/shape_predictor_68_face_landmarks.dat frame.jpg to print the head pose detected in frame.jpg.

Run ./gazr_estimate_head_pose ../share/shape_predictor_68_face_landmarks.dat image_file_names.txt to print the head pose detected in each image file listed in image_file_names.txt (image file names written in new lines).

gazr's Issues

gazr live_polt.py error

I'm trying to plot the roll yaw , and pitch using live_plot.py

I have used this :

tofi@tofi:~/gazr-master/build$

./gazr_estimate_head_direction -c --model=shape_predictor_68_face_landmarks.dat | python -u ../tools/live_plot.py
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injectorboost::program_options::unknown_option >'
what(): unrecognised option '-c'
/usr/lib/python2.7/dist-packages/pkg_resources.py:1031: UserWarning: /home/tofi/.python-eggs is writable by group/others and vulnerable to attack when used with get_resource_filename. Consider a more secure location (set with .set_extraction_path or the PYTHON_EGG_CACHE environment variable).
warnings.warn(msg, UserWarning)
Traceback (most recent call last):
File "../tools/live_plot.py", line 22, in
data = eval(line)
File "", line 0

^

SyntaxError: unexpected EOF while parsing

and also I tried :

tofi@tofi:~/gazr-master/build$ ./estimate_gaze_direction -c --model=shape_predictor_68_face_landmarks.dat | python -u ../tools/live_plot.py

bash: ./estimate_gaze_direction: No such file or directory
/usr/lib/python2.7/dist-packages/pkg_resources.py:1031: UserWarning: /home/tofi/.python-eggs is writable by group/others and vulnerable to attack when used with get_resource_filename. Consider a more secure location (set with .set_extraction_path or the PYTHON_EGG_CACHE environment variable).
warnings.warn(msg, UserWarning)
Traceback (most recent call last):
File "../tools/live_plot.py", line 22, in
data = eval(line)
File "", line 0

^

SyntaxError: unexpected EOF while parsin

estimation speed extremely slow

I am able to compile and use gazr fine without any bugs in ROS but the run time is extremely slow.
The facial features topic only publishes once every few seconds (highly variable as well).
This is while gazr uses up 100% of a CPU core.

Is this sort of delay normal? I thought that gazr could be used for online applications.

I already built gazr with the following flag
-DCMAKE_BUILD_TYPE=Release

webcam video is very slow

I have compiled both dlib and gazr in release mode :

mkdir build
cd build
cmake ..
cmake --build . --config Release

but when I run

./gazr_show_head_pose --model ../share/shape_predictor_68_face_landmarks.dat

the video from webcam is very slow

Can't find ros/ros.h

Hi I'm having an issue where make displays the following error:
gazr/src/head_pose_estimation.cpp:4:21: fatal error: ros/ros.h: No such file or directory
This appears despite ROS being installed

About roll, yaw, pitch by Python implementaiton

Hello I follow your c++ code to implementation the pose estimation,
I want to get the face pose, range from +90 to -90, like the following picture

I use the six landmark and their world coordinate to get pose

image_points = np.array([
                            (landmarks[4], landmarks[5]),     # Nose tip
                            (landmarks[10], landmarks[11]),   # Chin
                            (landmarks[0], landmarks[1]),     # Left eye left corner
                            (landmarks[2], landmarks[3]),     # Right eye right corne
                            (landmarks[6], landmarks[7]),     # Left Mouth corner
                            (landmarks[8], landmarks[9])      # Right mouth corner
                        ], dtype="double")

    # 3D model points.
    model_points = np.array([
                            (0.0, 0.0, 0.0),             # Nose tip
                            (0.0, -330.0, -65.0),        # Chin
                            (-165.0, 170.0, -135.0),     # Left eye left corner
                            (165.0, 170.0, -135.0),      # Right eye right corne
                            (-150.0, -150.0, -125.0),    # Left Mouth corner
                            (150.0, -150.0, -125.0)      # Right mouth corner                         
                        ])

(success, rotation_vector, translation_vector) = cv2.solvePnP(model_points, image_points, camera_matrix, dist_coeffs, flags=cv2.CV_ITERATIVE)
rvec_matrix = cv2.Rodrigues(rotation_vector)[0]
proj_matrix = np.hstack((rvec_matrix, translation_vector))
eulerAngles = -cv2.decomposeProjectionMatrix(proj_matrix)[6] 
yaw   = eulerAngles[1]
pitch = eulerAngles[0]
roll  = eulerAngles[2]
if pitch > 0:
  pitch = 180 - pitch
elif pitch < 0:
  pitch = -180 - pitch
yaw = -yaw 

But I have some problem, each case(yaw, pitch, row) is correct, for example, In face roll case, roll is true, but, pitch is false(pitch range will almost 180). Could u give me some advise? Thanks.

./gazr_show_head_pose ... is not working properly

gazr has been successfully built with dlib-19.4 on Ubuntu 14.04 x64 with OpenCV version: 3.2.0 and Boost version: 1.54. For building dlib-19.4, it has been used "sudo cmake --build . --target install"

Then, I try to test gazr as suggested

$ ./gazr_show_head_pose --model ../share/shape_predictor_68_face_landmarks.dat  
Setting the optical center to (320, 240)  
Processing time for this frame: 3464.05ms  

However, it only generates the previous two lines and I can also observe a quick
view of the window for a very short time.

I did another test with gazr_estimate_head_direction and live_plot.py with

$ ./gazr_estimate_head_direction --model=../share/shape_predictor_68_face_landmarks.dat | python -u ../tools/live_plot.py

It seems to be working since the traces of Pitch Yaw Roll changed over time when
the webcam is pointing my face but the processing is very slowly.

I reckon, I am missing something to make it work. Do you have any comments?

P.S. "./webcam_face_pose_ex" in "~/dlib/dlib-19.4/examples/build" is working fine.

Machine Features

Webcam

$ lsusb
Bus 002 Device 010: ID 046d:0991 Logitech, Inc. QuickCam Pro for Notebooks

Machine

$ less /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 37
model name      : Intel(R) Core(TM) i5 CPU         650  @ 3.20GHz
stepping        : 2
microcode       : 0xc
cpu MHz         : 1197.000

$ lshw -class memory
WARNING: you should run this program as super-user.
  *-memory                
       description: System memory
       physical id: 0
       size: 3751MiB

I appreciate your time and help.

Make Error: head_pose_estimation.cpp: error: reference to 'line' is ambiguous

Running into compile issues around opencv syntax in source (please see below):

Error example:
"/home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:178:13: error: reference to 'line' is ambiguous
line(result, feature_points[i], feature_points[i-1], line_color, 2, CV_AA);"

---

nvidia@tegra-ubuntu:~/cviz/gazr/build$ cmake -DWITH_TOOLS=TRUE ..
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Warning at /usr/local/lib/cmake/dlib/dlibConfig.cmake:42 (message):
The variable 'dlib_INCLUDE_DIRS' is deprecated! Instead, simply use
target_link_libraries(your_app dlib::dlib). See
http://dlib.net/examples/CMakeLists.txt.html for an example.
Call Stack (most recent call first):
CMakeLists.txt:9999 (__deprecated_var)

-- Using CATKIN_DEVEL_PREFIX: /home/nvidia/cviz/gazr/build/devel
-- Using CMAKE_PREFIX_PATH: /home/nvidia/jetsonbot/devel;/opt/ros/kinetic
-- This workspace overlays: /home/nvidia/jetsonbot/devel;/opt/ros/kinetic
-- Found PythonInterp: /usr/bin/python (found version "2.7.12")
-- Using PYTHON_EXECUTABLE: /usr/bin/python
-- Using Debian Python package layout
-- Using empy: /usr/bin/empy
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: /home/nvidia/cviz/gazr/build/test_results
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found gtest sources under '/usr/src/gtest': gtests will be built
-- Using Python nosetests: /usr/bin/nosetests-2.7
-- catkin 0.7.6
-- Using these message generators: gencpp;geneus;genlisp;gennodejs;genpy
-- Found OpenCV: /opt/ros/kinetic (found version "3.3.1") found components: core imgproc calib3d
-- OpenCV version: 3.3.1
CMake Warning at /usr/local/lib/cmake/dlib/dlibConfig.cmake:42 (message):
The variable 'dlib_LIBRARIES' is deprecated! Instead, simply use
target_link_libraries(your_app dlib::dlib). See
http://dlib.net/examples/CMakeLists.txt.html for an example.
Call Stack (most recent call first):
CMakeLists.txt:9999 (__deprecated_var)

-- Found OpenCV: /opt/ros/kinetic (found version "3.3.1") found components: core imgproc calib3d highgui imgcodecs videoio
-- Boost version: 1.60.0
-- Found the following Boost libraries:
-- program_options
-- Configuring done
-- Generating done
-- Build files have been written to: /home/nvidia/cviz/gazr/build

---

nvidia@tegra-ubuntu:~/cviz/gazr/build$ make
Scanning dependencies of target gazr
Scanning dependencies of target estimate_focus
[ 7%] Building CXX object CMakeFiles/estimate_focus.dir/src/estimate_focus.cpp.o
[ 14%] Building CXX object CMakeFiles/gazr.dir/src/head_pose_estimation.cpp.o
[ 21%] Linking CXX executable devel/lib/gazr/estimate_focus
/home/nvidia/cviz/gazr/src/head_pose_estimation.cpp: In member function 'void HeadPoseEstimation::drawFeatures(const std::vector<std::vector<cv::Point_ > >&, cv::Mat&) const':
/home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:175:13: error: reference to 'line' is ambiguous
line(result, feature_points[i], feature_points[i-1], line_color, 2, CV_AA);
^
In file included from /usr/local/include/opencv2/imgproc/imgproc.hpp:48:0,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:5:
/usr/local/include/opencv2/imgproc.hpp:4182:19: note: candidates are: void cv::line(cv::InputOutputArray, cv::Point, cv::Point, const Scalar&, int, int, int)
CV_EXPORTS_W void line(InputOutputArray img, Point pt1, Point pt2, const Scalar& color,
^
In file included from /usr/local/include/dlib/geometry.h:11:0,
from /usr/local/include/dlib/optimization/max_sum_submatrix.h:10,
from /usr/local/include/dlib/optimization.h:15,
from /usr/local/include/dlib/statistics/vector_normalizer_frobmetric.h:8,
from /usr/local/include/dlib/statistics.h:13,
from /usr/local/include/dlib/image_transforms/assign_image.h:8,
from /usr/local/include/dlib/image_transforms/spatial_filtering.h:15,
from /usr/local/include/dlib/image_processing/scan_image.h:13,
from /usr/local/include/dlib/image_processing.h:11,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.hpp:6,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:11:
/usr/local/include/dlib/geometry/line.h:15:11: note: class dlib::line
class line
^
/home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:178:13: error: reference to 'line' is ambiguous
line(result, feature_points[i], feature_points[i-1], line_color, 2, CV_AA);
^
In file included from /usr/local/include/opencv2/imgproc/imgproc.hpp:48:0,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:5:
/usr/local/include/opencv2/imgproc.hpp:4182:19: note: candidates are: void cv::line(cv::InputOutputArray, cv::Point, cv::Point, const Scalar&, int, int, int)
CV_EXPORTS_W void line(InputOutputArray img, Point pt1, Point pt2, const Scalar& color,
^
In file included from /usr/local/include/dlib/geometry.h:11:0,
from /usr/local/include/dlib/optimization/max_sum_submatrix.h:10,
from /usr/local/include/dlib/optimization.h:15,
from /usr/local/include/dlib/statistics/vector_normalizer_frobmetric.h:8,
from /usr/local/include/dlib/statistics.h:13,
from /usr/local/include/dlib/image_transforms/assign_image.h:8,
from /usr/local/include/dlib/image_transforms/spatial_filtering.h:15,
from /usr/local/include/dlib/image_processing/scan_image.h:13,
from /usr/local/include/dlib/image_processing.h:11,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.hpp:6,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:11:
/usr/local/include/dlib/geometry/line.h:15:11: note: class dlib::line
class line
^
/home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:181:13: error: reference to 'line' is ambiguous
line(result, feature_points[i], feature_points[i-1], line_color, 2, CV_AA);
^
In file included from /usr/local/include/opencv2/imgproc/imgproc.hpp:48:0,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:5:
/usr/local/include/opencv2/imgproc.hpp:4182:19: note: candidates are: void cv::line(cv::InputOutputArray, cv::Point, cv::Point, const Scalar&, int, int, int)
CV_EXPORTS_W void line(InputOutputArray img, Point pt1, Point pt2, const Scalar& color,
^
In file included from /usr/local/include/dlib/geometry.h:11:0,
from /usr/local/include/dlib/optimization/max_sum_submatrix.h:10,
from /usr/local/include/dlib/optimization.h:15,
from /usr/local/include/dlib/statistics/vector_normalizer_frobmetric.h:8,
from /usr/local/include/dlib/statistics.h:13,
from /usr/local/include/dlib/image_transforms/assign_image.h:8,
from /usr/local/include/dlib/image_transforms/spatial_filtering.h:15,
from /usr/local/include/dlib/image_processing/scan_image.h:13,
from /usr/local/include/dlib/image_processing.h:11,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.hpp:6,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:11:
/usr/local/include/dlib/geometry/line.h:15:11: note: class dlib::line
class line
^
/home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:183:13: error: reference to 'line' is ambiguous
line(result, feature_points[i], feature_points[i-1], line_color, 2, CV_AA);
^
In file included from /usr/local/include/opencv2/imgproc/imgproc.hpp:48:0,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:5:
/usr/local/include/opencv2/imgproc.hpp:4182:19: note: candidates are: void cv::line(cv::InputOutputArray, cv::Point, cv::Point, const Scalar&, int, int, int)
CV_EXPORTS_W void line(InputOutputArray img, Point pt1, Point pt2, const Scalar& color,
^
In file included from /usr/local/include/dlib/geometry.h:11:0,
from /usr/local/include/dlib/optimization/max_sum_submatrix.h:10,
from /usr/local/include/dlib/optimization.h:15,
from /usr/local/include/dlib/statistics/vector_normalizer_frobmetric.h:8,
from /usr/local/include/dlib/statistics.h:13,
from /usr/local/include/dlib/image_transforms/assign_image.h:8,
from /usr/local/include/dlib/image_transforms/spatial_filtering.h:15,
from /usr/local/include/dlib/image_processing/scan_image.h:13,
from /usr/local/include/dlib/image_processing.h:11,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.hpp:6,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:11:
/usr/local/include/dlib/geometry/line.h:15:11: note: class dlib::line
class line
^
/home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:185:13: error: reference to 'line' is ambiguous
line(result, feature_points[i], feature_points[i-1], line_color, 2, CV_AA);
^
In file included from /usr/local/include/opencv2/imgproc/imgproc.hpp:48:0,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:5:
/usr/local/include/opencv2/imgproc.hpp:4182:19: note: candidates are: void cv::line(cv::InputOutputArray, cv::Point, cv::Point, const Scalar&, int, int, int)
CV_EXPORTS_W void line(InputOutputArray img, Point pt1, Point pt2, const Scalar& color,
^
In file included from /usr/local/include/dlib/geometry.h:11:0,
from /usr/local/include/dlib/optimization/max_sum_submatrix.h:10,
from /usr/local/include/dlib/optimization.h:15,
from /usr/local/include/dlib/statistics/vector_normalizer_frobmetric.h:8,
from /usr/local/include/dlib/statistics.h:13,
from /usr/local/include/dlib/image_transforms/assign_image.h:8,
from /usr/local/include/dlib/image_transforms/spatial_filtering.h:15,
from /usr/local/include/dlib/image_processing/scan_image.h:13,
from /usr/local/include/dlib/image_processing.h:11,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.hpp:6,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:11:
/usr/local/include/dlib/geometry/line.h:15:11: note: class dlib::line
class line
^
/home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:186:9: error: reference to 'line' is ambiguous
line(result, feature_points[30], feature_points[35], line_color, 2, CV_AA);
^
In file included from /usr/local/include/opencv2/imgproc/imgproc.hpp:48:0,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:5:
/usr/local/include/opencv2/imgproc.hpp:4182:19: note: candidates are: void cv::line(cv::InputOutputArray, cv::Point, cv::Point, const Scalar&, int, int, int)
CV_EXPORTS_W void line(InputOutputArray img, Point pt1, Point pt2, const Scalar& color,
^
In file included from /usr/local/include/dlib/geometry.h:11:0,
from /usr/local/include/dlib/optimization/max_sum_submatrix.h:10,
from /usr/local/include/dlib/optimization.h:15,
from /usr/local/include/dlib/statistics/vector_normalizer_frobmetric.h:8,
from /usr/local/include/dlib/statistics.h:13,
from /usr/local/include/dlib/image_transforms/assign_image.h:8,
from /usr/local/include/dlib/image_transforms/spatial_filtering.h:15,
from /usr/local/include/dlib/image_processing/scan_image.h:13,
from /usr/local/include/dlib/image_processing.h:11,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.hpp:6,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:11:
/usr/local/include/dlib/geometry/line.h:15:11: note: class dlib::line
class line
^
/home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:189:13: error: reference to 'line' is ambiguous
line(result, feature_points[i], feature_points[i-1], line_color, 2, CV_AA);
^
In file included from /usr/local/include/opencv2/imgproc/imgproc.hpp:48:0,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:5:
/usr/local/include/opencv2/imgproc.hpp:4182:19: note: candidates are: void cv::line(cv::InputOutputArray, cv::Point, cv::Point, const Scalar&, int, int, int)
CV_EXPORTS_W void line(InputOutputArray img, Point pt1, Point pt2, const Scalar& color,
^
In file included from /usr/local/include/dlib/geometry.h:11:0,
from /usr/local/include/dlib/optimization/max_sum_submatrix.h:10,
from /usr/local/include/dlib/optimization.h:15,
from /usr/local/include/dlib/statistics/vector_normalizer_frobmetric.h:8,
from /usr/local/include/dlib/statistics.h:13,
from /usr/local/include/dlib/image_transforms/assign_image.h:8,
from /usr/local/include/dlib/image_transforms/spatial_filtering.h:15,
from /usr/local/include/dlib/image_processing/scan_image.h:13,
from /usr/local/include/dlib/image_processing.h:11,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.hpp:6,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:11:
/usr/local/include/dlib/geometry/line.h:15:11: note: class dlib::line
class line
^
/home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:190:9: error: reference to 'line' is ambiguous
line(result, feature_points[36], feature_points[41], line_color, 2, CV_AA);
^
In file included from /usr/local/include/opencv2/imgproc/imgproc.hpp:48:0,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:5:
/usr/local/include/opencv2/imgproc.hpp:4182:19: note: candidates are: void cv::line(cv::InputOutputArray, cv::Point, cv::Point, const Scalar&, int, int, int)
CV_EXPORTS_W void line(InputOutputArray img, Point pt1, Point pt2, const Scalar& color,
^
In file included from /usr/local/include/dlib/geometry.h:11:0,
from /usr/local/include/dlib/optimization/max_sum_submatrix.h:10,
from /usr/local/include/dlib/optimization.h:15,
from /usr/local/include/dlib/statistics/vector_normalizer_frobmetric.h:8,
from /usr/local/include/dlib/statistics.h:13,
from /usr/local/include/dlib/image_transforms/assign_image.h:8,
from /usr/local/include/dlib/image_transforms/spatial_filtering.h:15,
from /usr/local/include/dlib/image_processing/scan_image.h:13,
from /usr/local/include/dlib/image_processing.h:11,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.hpp:6,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:11:
/usr/local/include/dlib/geometry/line.h:15:11: note: class dlib::line
class line
^
/home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:193:13: error: reference to 'line' is ambiguous
line(result, feature_points[i], feature_points[i-1], line_color, 2, CV_AA);
^
In file included from /usr/local/include/opencv2/imgproc/imgproc.hpp:48:0,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:5:
/usr/local/include/opencv2/imgproc.hpp:4182:19: note: candidates are: void cv::line(cv::InputOutputArray, cv::Point, cv::Point, const Scalar&, int, int, int)
CV_EXPORTS_W void line(InputOutputArray img, Point pt1, Point pt2, const Scalar& color,
^
In file included from /usr/local/include/dlib/geometry.h:11:0,
from /usr/local/include/dlib/optimization/max_sum_submatrix.h:10,
from /usr/local/include/dlib/optimization.h:15,
from /usr/local/include/dlib/statistics/vector_normalizer_frobmetric.h:8,
from /usr/local/include/dlib/statistics.h:13,
from /usr/local/include/dlib/image_transforms/assign_image.h:8,
from /usr/local/include/dlib/image_transforms/spatial_filtering.h:15,
from /usr/local/include/dlib/image_processing/scan_image.h:13,
from /usr/local/include/dlib/image_processing.h:11,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.hpp:6,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:11:
/usr/local/include/dlib/geometry/line.h:15:11: note: class dlib::line
class line
^
/home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:194:9: error: reference to 'line' is ambiguous
line(result, feature_points[42], feature_points[47], line_color, 2, CV_AA);
^
In file included from /usr/local/include/opencv2/imgproc/imgproc.hpp:48:0,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:5:
/usr/local/include/opencv2/imgproc.hpp:4182:19: note: candidates are: void cv::line(cv::InputOutputArray, cv::Point, cv::Point, const Scalar&, int, int, int)
CV_EXPORTS_W void line(InputOutputArray img, Point pt1, Point pt2, const Scalar& color,
^
In file included from /usr/local/include/dlib/geometry.h:11:0,
from /usr/local/include/dlib/optimization/max_sum_submatrix.h:10,
from /usr/local/include/dlib/optimization.h:15,
from /usr/local/include/dlib/statistics/vector_normalizer_frobmetric.h:8,
from /usr/local/include/dlib/statistics.h:13,
from /usr/local/include/dlib/image_transforms/assign_image.h:8,
from /usr/local/include/dlib/image_transforms/spatial_filtering.h:15,
from /usr/local/include/dlib/image_processing/scan_image.h:13,
from /usr/local/include/dlib/image_processing.h:11,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.hpp:6,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:11:
/usr/local/include/dlib/geometry/line.h:15:11: note: class dlib::line
class line
^
/home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:197:13: error: reference to 'line' is ambiguous
line(result, feature_points[i], feature_points[i-1], line_color, 2, CV_AA);
^
In file included from /usr/local/include/opencv2/imgproc/imgproc.hpp:48:0,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:5:
/usr/local/include/opencv2/imgproc.hpp:4182:19: note: candidates are: void cv::line(cv::InputOutputArray, cv::Point, cv::Point, const Scalar&, int, int, int)
CV_EXPORTS_W void line(InputOutputArray img, Point pt1, Point pt2, const Scalar& color,
^
In file included from /usr/local/include/dlib/geometry.h:11:0,
from /usr/local/include/dlib/optimization/max_sum_submatrix.h:10,
from /usr/local/include/dlib/optimization.h:15,
from /usr/local/include/dlib/statistics/vector_normalizer_frobmetric.h:8,
from /usr/local/include/dlib/statistics.h:13,
from /usr/local/include/dlib/image_transforms/assign_image.h:8,
from /usr/local/include/dlib/image_transforms/spatial_filtering.h:15,
from /usr/local/include/dlib/image_processing/scan_image.h:13,
from /usr/local/include/dlib/image_processing.h:11,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.hpp:6,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:11:
/usr/local/include/dlib/geometry/line.h:15:11: note: class dlib::line
class line
^
/home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:198:9: error: reference to 'line' is ambiguous
line(result, feature_points[48], feature_points[59], line_color, 2, CV_AA);
^
In file included from /usr/local/include/opencv2/imgproc/imgproc.hpp:48:0,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:5:
/usr/local/include/opencv2/imgproc.hpp:4182:19: note: candidates are: void cv::line(cv::InputOutputArray, cv::Point, cv::Point, const Scalar&, int, int, int)
CV_EXPORTS_W void line(InputOutputArray img, Point pt1, Point pt2, const Scalar& color,
^
In file included from /usr/local/include/dlib/geometry.h:11:0,
from /usr/local/include/dlib/optimization/max_sum_submatrix.h:10,
from /usr/local/include/dlib/optimization.h:15,
from /usr/local/include/dlib/statistics/vector_normalizer_frobmetric.h:8,
from /usr/local/include/dlib/statistics.h:13,
from /usr/local/include/dlib/image_transforms/assign_image.h:8,
from /usr/local/include/dlib/image_transforms/spatial_filtering.h:15,
from /usr/local/include/dlib/image_processing/scan_image.h:13,
from /usr/local/include/dlib/image_processing.h:11,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.hpp:6,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:11:
/usr/local/include/dlib/geometry/line.h:15:11: note: class dlib::line
class line
^
/home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:201:13: error: reference to 'line' is ambiguous
line(result, feature_points[i], feature_points[i-1], line_color, 2, CV_AA);
^
In file included from /usr/local/include/opencv2/imgproc/imgproc.hpp:48:0,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:5:
/usr/local/include/opencv2/imgproc.hpp:4182:19: note: candidates are: void cv::line(cv::InputOutputArray, cv::Point, cv::Point, const Scalar&, int, int, int)
CV_EXPORTS_W void line(InputOutputArray img, Point pt1, Point pt2, const Scalar& color,
^
In file included from /usr/local/include/dlib/geometry.h:11:0,
from /usr/local/include/dlib/optimization/max_sum_submatrix.h:10,
from /usr/local/include/dlib/optimization.h:15,
from /usr/local/include/dlib/statistics/vector_normalizer_frobmetric.h:8,
from /usr/local/include/dlib/statistics.h:13,
from /usr/local/include/dlib/image_transforms/assign_image.h:8,
from /usr/local/include/dlib/image_transforms/spatial_filtering.h:15,
from /usr/local/include/dlib/image_processing/scan_image.h:13,
from /usr/local/include/dlib/image_processing.h:11,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.hpp:6,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:11:
/usr/local/include/dlib/geometry/line.h:15:11: note: class dlib::line
class line
^
/home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:202:9: error: reference to 'line' is ambiguous
line(result, feature_points[60], feature_points[67], line_color, 2, CV_AA);
^
In file included from /usr/local/include/opencv2/imgproc/imgproc.hpp:48:0,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:5:
/usr/local/include/opencv2/imgproc.hpp:4182:19: note: candidates are: void cv::line(cv::InputOutputArray, cv::Point, cv::Point, const Scalar&, int, int, int)
CV_EXPORTS_W void line(InputOutputArray img, Point pt1, Point pt2, const Scalar& color,
^
In file included from /usr/local/include/dlib/geometry.h:11:0,
from /usr/local/include/dlib/optimization/max_sum_submatrix.h:10,
from /usr/local/include/dlib/optimization.h:15,
from /usr/local/include/dlib/statistics/vector_normalizer_frobmetric.h:8,
from /usr/local/include/dlib/statistics.h:13,
from /usr/local/include/dlib/image_transforms/assign_image.h:8,
from /usr/local/include/dlib/image_transforms/spatial_filtering.h:15,
from /usr/local/include/dlib/image_processing/scan_image.h:13,
from /usr/local/include/dlib/image_processing.h:11,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.hpp:6,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:11:
/usr/local/include/dlib/geometry/line.h:15:11: note: class dlib::line
class line
^
/home/nvidia/cviz/gazr/src/head_pose_estimation.cpp: In member function 'void HeadPoseEstimation::drawPose(const head_pose&, size_t, cv::Mat&) const':
/home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:254:5: error: reference to 'line' is ambiguous
line(result, projected_axes[0], projected_axes[3], x_axis_color,2,CV_AA);
^
In file included from /usr/local/include/opencv2/imgproc/imgproc.hpp:48:0,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:5:
/usr/local/include/opencv2/imgproc.hpp:4182:19: note: candidates are: void cv::line(cv::InputOutputArray, cv::Point, cv::Point, const Scalar&, int, int, int)
CV_EXPORTS_W void line(InputOutputArray img, Point pt1, Point pt2, const Scalar& color,
^
In file included from /usr/local/include/dlib/geometry.h:11:0,
from /usr/local/include/dlib/optimization/max_sum_submatrix.h:10,
from /usr/local/include/dlib/optimization.h:15,
from /usr/local/include/dlib/statistics/vector_normalizer_frobmetric.h:8,
from /usr/local/include/dlib/statistics.h:13,
from /usr/local/include/dlib/image_transforms/assign_image.h:8,
from /usr/local/include/dlib/image_transforms/spatial_filtering.h:15,
from /usr/local/include/dlib/image_processing/scan_image.h:13,
from /usr/local/include/dlib/image_processing.h:11,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.hpp:6,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:11:
/usr/local/include/dlib/geometry/line.h:15:11: note: class dlib::line
class line
^
/home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:255:5: error: reference to 'line' is ambiguous
line(result, projected_axes[0], projected_axes[2], y_axis_color,2,CV_AA);
^
In file included from /usr/local/include/opencv2/imgproc/imgproc.hpp:48:0,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:5:
/usr/local/include/opencv2/imgproc.hpp:4182:19: note: candidates are: void cv::line(cv::InputOutputArray, cv::Point, cv::Point, const Scalar&, int, int, int)
CV_EXPORTS_W void line(InputOutputArray img, Point pt1, Point pt2, const Scalar& color,
^
In file included from /usr/local/include/dlib/geometry.h:11:0,
from /usr/local/include/dlib/optimization/max_sum_submatrix.h:10,
from /usr/local/include/dlib/optimization.h:15,
from /usr/local/include/dlib/statistics/vector_normalizer_frobmetric.h:8,
from /usr/local/include/dlib/statistics.h:13,
from /usr/local/include/dlib/image_transforms/assign_image.h:8,
from /usr/local/include/dlib/image_transforms/spatial_filtering.h:15,
from /usr/local/include/dlib/image_processing/scan_image.h:13,
from /usr/local/include/dlib/image_processing.h:11,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.hpp:6,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:11:
/usr/local/include/dlib/geometry/line.h:15:11: note: class dlib::line
class line
^
/home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:256:5: error: reference to 'line' is ambiguous
line(result, projected_axes[0], projected_axes[1], z_axis_color,2,CV_AA);
^
In file included from /usr/local/include/opencv2/imgproc/imgproc.hpp:48:0,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:5:
/usr/local/include/opencv2/imgproc.hpp:4182:19: note: candidates are: void cv::line(cv::InputOutputArray, cv::Point, cv::Point, const Scalar&, int, int, int)
CV_EXPORTS_W void line(InputOutputArray img, Point pt1, Point pt2, const Scalar& color,
^
In file included from /usr/local/include/dlib/geometry.h:11:0,
from /usr/local/include/dlib/optimization/max_sum_submatrix.h:10,
from /usr/local/include/dlib/optimization.h:15,
from /usr/local/include/dlib/statistics/vector_normalizer_frobmetric.h:8,
from /usr/local/include/dlib/statistics.h:13,
from /usr/local/include/dlib/image_transforms/assign_image.h:8,
from /usr/local/include/dlib/image_transforms/spatial_filtering.h:15,
from /usr/local/include/dlib/image_processing/scan_image.h:13,
from /usr/local/include/dlib/image_processing.h:11,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.hpp:6,
from /home/nvidia/cviz/gazr/src/head_pose_estimation.cpp:11:
/usr/local/include/dlib/geometry/line.h:15:11: note: class dlib::line
class line
^
[ 21%] Built target estimate_focus
CMakeFiles/gazr.dir/build.make:62: recipe for target 'CMakeFiles/gazr.dir/src/head_pose_estimation.cpp.o' failed
make[2]: *** [CMakeFiles/gazr.dir/src/head_pose_estimation.cpp.o] Error 1
CMakeFiles/Makefile2:1047: recipe for target 'CMakeFiles/gazr.dir/all' failed
make[1]: *** [CMakeFiles/gazr.dir/all] Error 2
Makefile:138: recipe for target 'all' failed
make: *** [all] Error 2

Facial landmarks

HI

Sorry may be it is not the right location to ask , but I couldn't find an answer for my question.
how the landmark points on the face such as the eyes can be viewed ? e.g how to show the pixel locations of these landmarks ?

Thanks

Estimation speed is a little slow

I run the gazr_show_head_pose and get the estimation speed of 8s per frame. how can I speed it up?

OpenBLAS library not found.

whenever compling i get this error

/usr/lib/x86_64-linux-gnu/libopenblas.so', needed by '/home/catkin_vis/devel/lib/libgazr.so'.

and then i created a static link to the openblas library folder and then i got this error.

ibdlib.a(base64_kernel_1.cpp.o): relocation R_X86_64_PC32 against symbol `_ZTVN4dlib6base64E' can not be used when making a shared object; recompile with -fPIC

is there a reason why it was not linked from the start?

am using RTX3070 11.1 CUDA.

devel/lib/libgazr.so: undefined reference to

Hello, While trying to build, I found a MAKE error and upgraded the OpenCV version to 4.4.0.
After that, when I tried to build again, this error appeared again. What's the problem?

[ 14%] Built target gazr
[ 21%] Linking CXX executable devel/lib/gazr/gazr_estimate_head_direction
devel/lib/libgazr.so: undefined reference to `cblas_dgemm'

devel/lib/libgazr.so: undefined reference to `dgesvd_'
collect2: error: ld returned 1 exit status
CMakeFiles/gazr_estimate_head_direction.dir/build.make:104: recipe for target 'devel/lib/gazr/gazr_estimate_head_direction' failed
make[2]: *** [devel/lib/gazr/gazr_estimate_head_direction] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/gazr_estimate_head_direction.dir/all' failed
make[1]: *** [CMakeFiles/gazr_estimate_head_direction.dir/all] Error 2
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2

Compiling problems with dlib

Hi,

I'm trying to compile the project. I would like to use it as a ROS package in my workspace, so I'm building with catkin build gazr. I installed dlib with sudo apt-get install libdlib-dev. I'm on ROS Melodic.

The first building gives me this error:

/home/alberto/tiago_dual_public_ws/src/gazr/src/head_pose_estimation.cpp: In member function ‘std::vector<std::vector<cv::Point_<int> > > HeadPoseEstimation::update(cv::InputArray)’:
/home/alberto/tiago_dual_public_ws/src/gazr/src/head_pose_estimation.cpp:51:20: error: ‘cvIplImage’ was not declared in this scope
     auto ipl_img = cvIplImage(image);
                    ^~~~~~~~~~
/home/alberto/tiago_dual_public_ws/src/gazr/src/head_pose_estimation.cpp:51:20: note: suggested alternative: ‘IplImage’
     auto ipl_img = cvIplImage(image);
                    ^~~~~~~~~~
                    IplImage

I easily solved following the suggestion of the compiler. Then I got:

/usr/bin/ld: /usr/lib/libdlib.a(threads_kernel_shared.o): relocation R_X86_64_PC32 against symbol `_ZN4dlib21threads_kernel_shared14thread_starterEPv' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
make[2]: *** [/home/alberto/tiago_dual_public_ws/devel/.private/gazr/lib/libgazr.so] Error 1
make[1]: *** [CMakeFiles/gazr.dir/all] Error 2
make: *** [all] Error 2

I don't know how to solve it. It seems is it due to a wrong compilation of dlib. Can someone help me?

ROS Kinetic dependencies are not working with gazr

Having seen the ROS package dependencies at CMakeLists.txt, I tried to install the following packages on ROS kinetic. However, it seems that gazr has been designed to work with another distribution for ROS because the following ones are not available for kinetic:

sudo apt-get install ros-kinetic-std_msgs
sudo apt-get install ros-kinetic-image_transport
sudo apt-get install ros-kinetic-image_geometry
sudo apt-get install ros-kinetic-cv_bridge
sudo apt-get install ros-kinetic-sensor_msgs
sudo apt-get install ros-kinetic-visualization_msgs

It would be good to add a description in the README.md for the ROS distribution which works with gazr.

NB. For the time being, I am learning quite a lot from gazr, but if there is a way that I can contribute by having general instructions, I will happily give it a go to build the packages dependencies on ROS Kinetic.

Yaw, roll and pitch angle

Hi,

I am trying to determine head yaw, roll and pitch angles using your solution. However, the results are very bad. I assume that the input parameters (e.g. focal length) are not set properly.

Thus, I have the following questions: do you have any tips how to estimate the focal length for the webcam in my laptop? What steps should I take to adjust the algorithm for my data?

Hope you can help me.

Best,
Michal

Dependencies issues with dlibConfig.cmake when building gazr using dlib-19.4 in Release mode

I am trying to build dlib-19.4 in release mode to increase the processing
speed using "cmake --build . --config Release" but I have been facing the following dependencies
issues with dlibConfig.cmake:

$ cmake -Ddlib_DIR=/home/map479-admin/dlib/dlib-19.4/build/dlib/config ..
.
.
.
CMake Error at /home/map479-admin/dlib/dlib-19.4/build/dlib/config/dlibConfig.cmake:35 (include):
  include could not find load file:

    /home/map479-admin/dlib/dlib-19.4/build/dlib/config/dlib.cmake
Call Stack (most recent call first):
  CMakeLists.txt:11 (find_package)


CMake Error at /home/map479-admin/dlib/dlib-19.4/build/dlib/config/dlibConfig.cmake:42 (include):
  include could not find load file:

    /home/map479-admin/dlib/dlib-19.4/include/dlib/cmake_utils/use_cpp_11.cmake
Call Stack (most recent call first):
  CMakeLists.txt:11 (find_package)

To solve the previous problem I installed dlib-19.4 with
"sudo cmake --build . --target install" which is not the release mode.

However, I want to install both dlib and gazr in release mode.

Do you have any suggestions?

Can you recommend a camera to get reliable data for multiple user gaze estimation?

Hiya,

I am not really sure whether this is the place to post this question, if not, please let me know!

Having said that, I want to mention that I like what you are doing with gazr that I will denitifely work with your project for multiple user gaze estimation to which I am wondering which brand of web cam do you recomend to work with gazr? Also, what is the maximum number of users that gazr can estimate?

Cheerio,
Miguel

YPR range

Hello Severin,
I have two questions.
What is the YPR range that can be obtained ?.Also did you test it with any standard dataset?

/home/tofi/gazr/tools/estimate_head_direction.cpp:71:5: error: ‘VideoCapture’ was not declared in this scope VideoCapture video_in; ^

Hi , i'm getting this error during make :

I'm using opencv 3.1 is that causes this error ?

tofi@tofi:~/gazr/build$ make
Scanning dependencies of target gazr
[ 25%] Building CXX object CMakeFiles/gazr.dir/src/head_pose_estimation.cpp.o
Linking CXX shared library libgazr.so
[ 25%] Built target gazr
Scanning dependencies of target gazr_benchmark_head_pose_single_frame
[ 50%] Building CXX object CMakeFiles/gazr_benchmark_head_pose_single_frame.dir/tools/benchmark_head_pose_estimation_single_frame.cpp.o
Linking CXX executable gazr_benchmark_head_pose_single_frame
[ 50%] Built target gazr_benchmark_head_pose_single_frame
Scanning dependencies of target gazr_estimate_head_direction
[ 75%] Building CXX object CMakeFiles/gazr_estimate_head_direction.dir/tools/estimate_head_direction.cpp.o
/home/tofi/gazr/tools/estimate_head_direction.cpp: In function ‘int main(int, char**)’:
/home/tofi/gazr/tools/estimate_head_direction.cpp:71:5: error: ‘VideoCapture’ was not declared in this scope
VideoCapture video_in;
^
/home/tofi/gazr/tools/estimate_head_direction.cpp:71:18: error: expected ‘;’ before ‘video_in’
VideoCapture video_in;
^
/home/tofi/gazr/tools/estimate_head_direction.cpp:74:9: error: ‘video_in’ was not declared in this scope
video_in = VideoCapture(0);
^
/home/tofi/gazr/tools/estimate_head_direction.cpp:77:22: error: ‘CV_CAP_PROP_FRAME_WIDTH’ was not declared in this scope
video_in.set(CV_CAP_PROP_FRAME_WIDTH, 640);
^
/home/tofi/gazr/tools/estimate_head_direction.cpp:78:22: error: ‘CV_CAP_PROP_FRAME_HEIGHT’ was not declared in this scope
video_in.set(CV_CAP_PROP_FRAME_HEIGHT, 480);
^
/home/tofi/gazr/tools/estimate_head_direction.cpp:105:23: error: ‘video_in’ was not declared in this scope
auto ok = video_in.read(frame);
^
/home/tofi/gazr/tools/estimate_head_direction.cpp:149:48: error: ‘imshow’ was not declared in this scope
imshow("headpose", estimator._debug);
^
/home/tofi/gazr/tools/estimate_head_direction.cpp:151:27: error: ‘waitKey’ was not declared in this scope
waitKey(10);
^
/home/tofi/gazr/tools/estimate_head_direction.cpp:154:33: error: ‘waitKey’ was not declared in this scope
while(waitKey(10) != 1048603) {}
^
make[2]: *** [CMakeFiles/gazr_estimate_head_direction.dir/tools/estimate_head_direction.cpp.o] Error 1
make[1]: *** [CMakeFiles/gazr_estimate_head_direction.dir/all] Error 2
make: *** [all] Error 2

REQUIRED process [camera/gazr-2] has died!

Hello everyone,

I have followed the instructions and are now stuck at [camera/gazr-2] dying during roslaunch. Any ideas, what I should check? My camera is a simple USB camera (Logitech) and I am on Ubuntu 18.04 with ROS-Melodic.

All the best
Marcus

Getting results in Gazr

So, everything is installed and working as best as I can tell. Here is what I got so far:

using the ROS node:
roslaunch gazr gazr.launch
Output:

SUMMARY
========

PARAMETERS
 * /camera/gazr/face_model: /home/argos/catki...
 * /camera/gazr/prefix: face
 * /camera/gazr/with_depth: False
 * /rosdistro: melodic
 * /rosversion: 1.14.3

NODES
  /camera/
    gazr (gazr/estimate)

auto-starting new master
process[master]: started with pid [9688]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to e8951794-cd63-11e9-a309-a0a4c57336f1
process[rosout-1]: started with pid [9699]
started core service [/rosout]
process[camera/gazr-2]: started with pid [9705]
[ INFO] [1567416460.577348598]: Initializing the face detector with the model /home/argos/catkin_ws/src/gazr/shape_predictor_68_face_landmarks.dat...
[ INFO] [1567416462.437761834]: RGB-only estimator successfully initialized.
TF frames of detected faces will be published when detected,
as well as the nb of detected faces on /nb_detected_faces.

The camera is not turning on and if I check the rostopics, none of them are publishing.

---

If I try the stand alone tools:
./gazr_show_head_pose ../../../../shape_predictor_68_face_landmarks.dat
Output:

Corrupt JPEG data: 7 extraneous bytes before marker 0xd6
Processing time for this frame: 2252.85ms

I can see that the camera is turning on briefly, but apart from that nothing seems to be happening.

Thanks for the the help,
Best
Marcus

How to use this in Gazebo or ignition gazebo?

Compiling problems with openCV?

...@host:~/workarea/gazr$ mkdir build
...@host:~/workarea/gazr$ cd build
...@host:~/workarea/gazr/build$ cmake .. -DWITH_ROS=OFF -DWITH_TOOLS=TRUE
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- OpenCV version: 2.4.9.1
-- Boost version: 1.58.0
-- Found the following Boost libraries:
--   program_options
-- Configuring done
-- Generating done
-- Build files have been written to: /home/.../workarea/gazr/build
...@host:~/workarea/gazr/build$ make
Scanning dependencies of target gazr
[ 12%] Building CXX object CMakeFiles/gazr.dir/src/head_pose_estimation.cpp.o
/home/.../workarea/gazr/src/head_pose_estimation.cpp: In member function ‘std::vector<std::vector<cv::Point_<int> > > HeadPoseEstimation::update(cv::InputArray)’:
/home/.../workarea/gazr/src/head_pose_estimation.cpp:51:36: error: ‘cvIplImage’ was not declared in this scope
     auto ipl_img = cvIplImage(image);
                                    ^
CMakeFiles/gazr.dir/build.make:62: ターゲット 'CMakeFiles/gazr.dir/src/head_pose_estimation.cpp.o' のレシピで失敗しました
make[2]: *** [CMakeFiles/gazr.dir/src/head_pose_estimation.cpp.o] エラー 1
CMakeFiles/Makefile2:67: ターゲット 'CMakeFiles/gazr.dir/all' のレシピで失敗しました
make[1]: *** [CMakeFiles/gazr.dir/all] エラー 2
Makefile:83: ターゲット 'all' のレシピで失敗しました
make: *** [all] エラー 2
...@host:~/workarea/gazr/build$ 

make install error

HI severin:
When I'm going to implement the "make install" steps, it keep telling me there is a error in show_head_pose.cpp file. The warning is as follows:
error: ‘class HeadPoseEstimation’ has no member named ‘_debug’ imshow("headpose", estimator._debug);
how can I solve this problem?

Could not find a package configuration file provided by "dlib"

Hi

I have installed dlib and it works fine ,

when I cmake gazr this error is shown :

 ubuntu@ubuntu~/gazr/build$ cmake ..

CMake Error at CMakeLists.txt:6 (find_package):
  By not providing "Finddlib.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "dlib", but
  CMake did not find one.

  Could not find a package configuration file provided by "dlib" with any of
  the following names:

    dlibConfig.cmake
    dlib-config.cmake

can you help please to resolve this

What is the difference between this and attention tracker?

Any notable differences?
Thanks!

make - fatal error: ros/ros.h: No such file or directory

When I was trying to compile the files, I met an error.
I am using Ubuntu 16.04, ROS Kinetic Kame, dlib 19.7

d@D:~/Documents/gazr/build$ cmake ..
CMake Warning at /usr/local/lib/cmake/dlib/dlibConfig.cmake:42 (message):
  The variable 'dlib_INCLUDE_DIRS' is deprecated! Instead, simply use
  target_link_libraries(your_app dlib::dlib).  See
  http://dlib.net/examples/CMakeLists.txt.html for an example.
Call Stack (most recent call first):
  CMakeLists.txt:9999 (__deprecated_var)

-- Found OpenCV: /opt/ros/kinetic (found version "3.2.0") found components:  core imgproc calib3d highgui 
-- OpenCV version: 3.2.0
CMake Warning at /usr/local/lib/cmake/dlib/dlibConfig.cmake:42 (message):
  The variable 'dlib_LIBRARIES' is deprecated! Instead, simply use
  target_link_libraries(your_app dlib::dlib).  See
  http://dlib.net/examples/CMakeLists.txt.html for an example.
Call Stack (most recent call first):
  CMakeLists.txt:9999 (__deprecated_var)


-- Found OpenCV: /opt/ros/kinetic (found version "3.2.0") found components:  core imgproc calib3d highgui imgcodecs videoio 
-- Boost version: 1.58.0
-- Found the following Boost libraries:
--   program_options
-- Configuring done
-- Generating done
-- Build files have been written to: /home/dai/Documents/gazr/build

d@D:~/Documents/gazr/build$ make
[ 12%] Building CXX object CMakeFiles/gazr.dir/src/head_pose_estimation.cpp.o
/home/dai/Documents/gazr/src/head_pose_estimation.cpp:4:21: fatal error: ros/ros.h: No such file or directory
compilation terminated.
CMakeFiles/gazr.dir/build.make:62: recipe for target 'CMakeFiles/gazr.dir/src/head_pose_estimation.cpp.o' failed
make[2]: *** [CMakeFiles/gazr.dir/src/head_pose_estimation.cpp.o] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/gazr.dir/all' failed
make[1]: *** [CMakeFiles/gazr.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

Thanks for any hints!

dlib link failure

Hi,

I encountered a compilation (link) problem with x86_64 environment:

cmake --version
cmake version 3.5.1

cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.3 LTS"

gcc --version
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.5) 5.4.0 20160609

I built dlib with:

cmake -DCMAKE_INSTALL_PREFIX:PATH=/home/takotakot/dlib_install/ -DBUILD_SHARED_LIBS=ON ..
make
make install

gazr with:

dlib_DIR=/home/takotakot/dlib_install/ cmake .. -DWITH_ROS=false
make

And get

/usr/bin/ld: /home/tako/dlib_install/lib/libdlib.a(base64_kernel_1.cpp.o): relocation R_X86_64_32S against `_ZTVN4dlib6base64E' can not be used when making a shared object; recompile with -fPIC

The content of CMakeFiles/gazr.dir/link.txt is

/usr/bin/c++  -fPIC   -shared -Wl,-soname,libgazr.so -o libgazr.so CMakeFiles/gazr.dir/src/head_pose_estimation.cpp.o /home/takotakot/dlib_install/lib/libdlib.a /usr/local/lib/libopencv_calib3d.so.2.4.13 -lpthread -lnsl -lpng -lz -ljpeg /usr/local/lib/libopencv_features2d.so.2.4.13 /usr/local/lib/libopencv_flann.so.2.4.13 /usr/local/lib/libopencv_highgui.so.2.4.13 /usr/local/lib/libopencv_imgproc.so.2.4.13 /usr/local/lib/libopencv_core.so.2.4.13 -ldl -lm -lpthread -lrt -Wl,-rpath,/usr/local/lib

When I changed /home/takotakot/dlib_install/lib/libdlib.a to /home/takotakot/dlib_install/lib/libdlib.so manually, I could comiple libgazr.so .

Is this a gazr or cmake (or other tools) problem? If it could be fixed with changing some cmake configuration of gazr, I want to fix. But I'm sorry that I don't familiar with cmake.

Thanks