Currently, only one hand at a time detect. We need support for two hands simultaneously especially when two hands are involved in an egocentric task.

I try to build code, it fail and stop at Convert.h because of lacking RealSense/SampleReader.h. I also cannot find it at Real Sense SDK 2. Where can I found it ?

Arthur

There are many hardcoded windows based paths to data and datasets. The data is not provided.
There is no hint on how to convert SMPL models to PCD that is used by PCL.
It would be great if a sample video was included

Hi,

I am aware that the hand detector is for egocentric point of view but I am wondering if it could be also used from a cenital point of view. Specifically, is the SVM classifier method only trained with egocentric data? Any of the fingers and/or wrist estimators makes an assumption of a egocentric view?

Thank you for the project!

hi~
It is very impressive of your work , thank you for sharing. I am trying to see what my D435 camera can do, but it doesn't work very well.
I can't successfully configure the code following the guide OpenARK CMAKE build instructions for Windows .
It says Could NOT find RSSDK2(missing:RSSDK2_LIBRARY). Maybe something is wrong with my setting of environment variables? Because I can configure with RSSDK successfully via the same process.
any help thx! Orz

OpenARK/include/DetectionParams.h:110:28: warning: implicit conversion from 'double' to 'int' changes value from 1.6 to 1 [-Wliteral-conversion]
int handMaxDepth = 1.6;

Following Build Instructions for Windows Using Pre-Built Dependencies, when I executated cmake .. -G"Visual Studio 14 2015 Win64", it said CMake Error at CMakeLists.txt:61 (include): include could not find load file: C:\Program Files\OpenARK-Deps/CMakeLists.txt, and it could not find dependencies like OpenCV, etc.

When I examined the folder C:\Program Files\OpenARK-Deps, there was not a cmake file. I would like to ask whether the dependency installer configures the cmake file (although the instruction says it is) or what I did wrong. Thank you.

I have just run the HandDemo.cpp, but the result proved to be poor. I trust this project must run perfect in your lab.but it didn't work in my computer and i wanted to know the reason.

Highly recommend having ubuntu dependencies packed in a docker so that user can simply download a docker and start to use.

I had to increase the stack size to 20000000 so I wouldn't get stackoverflow for SR300 camera hand detection. This happens due to the recursive floodfill algorithm. We would need an irrecursive algorithm to replace it to avoid stackoverflow.

Here's an example of the depth image visualization from the Visualizer class in OpenARK. We need to improve this so that there is more contrast and hand and its fingers are more visually visible to the user.

I'm trying to use the Unity DLLs in the 0.9.3 Beta release, but I am getting this error in Unity 2019.2:

Plugins: Failed to load 'Assets/Plugins/openark_unity_0_9_3_native.dll' because one or more of its dependencies could not be loaded.
OpenARK.Detector:.ctor(Boolean)

And this error when I try it in 2017 or 2018:

DllNotFoundException: openark_unity_0_9_3_native

I have tried changing the API compatibility level, and have made sure that the DLLs are set to the correct platform.

Hi, running the sample on windows 10.
I have 2 realsense D435i cameras. With one of the all the frames are dropped, and then there is an exception (sometimes). With the other one I get "no meta data".
The realsense viewer and the hand demo work fine.
Any idea what can cause this?

When testing the OpenARK in Vicon room with Vicon MoCap cameras, there will be lots of flashes in the Intel RealSense SR300 camera and as a result the hand detection system will not work due to lots of noise and flicker.
In the case OpenARK needs to be run in the presence of Vicon Motion Capture cameras, new methods should be used to handle the flickery output from Intel RealSense SR300 camera.
Here's a video of what happens to SR300 output:
https://www.youtube.com/watch?v=eVhrkgsW6Uc

Hello，thanks for your great code！

I compile this code in win10 with vs2015 and the provided pre-build .exe file, following the instruction of "documentation/Windows-build-instructions-quick.md ". I suffered from difficulties when building Open3D 0.9.0 and 0.8.0, and the current version I used is 0.7.0.

There are 2 main errors I met.

Fisrt, “OpenARK\Source\SegmentedMesh.cpp(324): error C2660: "open3d::io::WriteTriangleMeshToPLY": The function does not accept 8 parameters”。

Second, “OpenARK\Source\SlamDemo435i.cpp(174): error C2039: "isReset": not a member of "okvis::ThreadedKFVio"

And I wondered, which is correct version of Open3D?

Looking forward to your answers. Thanks!

I have to claim, the algorithm is really sucks~, after being built successfully, I tested on realsensesdk 2, the hand detector demo, the result is really poor~. definitely it would not be deserved.

Hi, while trying to build OpenARK_SLAM_demo on Windows 10 linking fails with "unable to open blas.lib".
I can't find this file anywhere in the dependencies. The closest I can get is libblas.lib in the suitesparse dependency. Any idea why this file is in the project?

After i have tried to run HandDemo.cpp, I found this project only recognizes the hand 40-60cm from the camera. So this project has a high demand for distance.

Hello
I am trying to build the project in ubuntu 16.04. I have a pmd picoflexx camera. I tried to change the cmake file for pmd in cmake modules folder, but I confused. When I download sdk from pmd, I get sdk named "royale"(I think it is the newer version of pmdsdk), which works fine in my other applications with pmd camera, but it does not contain headers such as pmdsdk2.h. Is there a way to use my camera with this project?

The Problem

I have been working on some features about OpenARK. However, I find it hard to merge in.

I may achieve this by adding some code snippets into main.cpp to demonstrate them. But what if more features are coming in, which is definitely gonna happen, can we keep on modifying this file which may cause a dramatic increase in size and decrease in scalability?

Is there an elegant way to handle it?

There's a significant lag between the image in the Result window and the image in the Contour window. Please see the below link in 1080P fullscreen to observe the problem:
https://www.youtube.com/watch?v=UmG5UfE3hSo&feature=youtu.be&hd=1
According to the following screencast, we don't have the same problem with Intel RealSense and there's no lag between the Result window and Contour window for the hand.
https://www.youtube.com/watch?v=pi-puHjhEag&feature=youtu.be&hd=1
*Possible cause of bug is the use of pyrup to scale the acquired image 4x.

1- ***************************************************************************************************
I get the following compile error during the build with cmake.
I am using Eigen 3.3.4 as mentioned in the installation guide. Any idea how to solve?

OpenARK/include/Util.h: In function ‘Eigen::Matrix<_Scalar, 3, 1> ark::util::rotate(const Eigen::Matrix<_Scalar, 3, 1>&, const Quat_T&)’:
OpenARK/include/Util.h:655:62: error: expected primary-expression before ‘>’ token
             const Eigen::Matrix<T, 3, 1> & u = q.vec().cast<T>();

2- ****************************************************************************************************
I am wondering where pcl and Eigen are included in /include/Util.h. My compiler cannot find them

Dear all,

Some error happens when building . May be wrong version of gcc?
Thanks!

[ 69%] Building CXX object CMakeFiles/OpenARK.dir/Visualizer.cpp.o
In file included from /home/xx/Tool/OpenARK/MockCamera.cpp:1:0:
/home/xx/Tool/OpenARK/include/MockCamera.h:3:22: fatal error: filesystem: No such file or directory
compilation terminated.
CMakeFiles/OpenARK.dir/build.make:244: recipe for target 'CMakeFiles/OpenARK.dir/MockCamera.cpp.o' failed
make[2]: *** [CMakeFiles/OpenARK.dir/MockCamera.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....

Sometimes I get access violation for this line:
cv::Vec3f handPos = handAverager.addDataPoint(objects[handObjectIndex].getHand().fingers_xyz[0]);

This usually happens when my hand is not initially captured by the camera and I suddenly bring my hand in front of camera.
Another screenshot:

{sampleFrequency=4 rejectionThreshold=0.10000000000000001 currentValue={...} ...}
handpose: {val=0x00000038aafff2f0 {-0.0131008998, 0.000879988074, 0.304594696} }

As can be seen by running the OpenARK_test for folders P4, P5, P6, and P7 of CVAR dataset, in the case of too much clutter, the hand is not detected.

*Possible solution proposed by Joe Menke: when detecting objects as hand, only select the one that is the closest to the user based on the depth value after comparing all of the objects centroids depth values.