denistome / lifting-from-the-deep-release Goto Github PK

I operate the code python demo.py while a issue is occuring. And the issue as fellows:

My environment: python 2.7; GPU:1060 &1080ti; ubuntu16.04

Can you help me?

Hello,

I used the 3D pose estimator code to identify the points and am trying to convert them into real world co-ordinates. In order to do this, I need to know the depth value.

Any thoughts on how I can get the depth value? Thanks.

In my understanding, an input image would go through the PersonNet (to find out position of each person), PoseNet (to get the most confident pixel as the location of each landmark), and Prob3dPose (to calculate the 3D pose).

In Prob3dPose, the paper introduced steps on lifting 2D belief-maps into 3D, projecting to a new 2D pose belief maps and training weights for 2D fusion. Where can we find codes on this part?

Please help. Thank you.

How does the model define the coordinates system?

Hi ,
I tried to run your code but it reaches to this error. What is the source of the error ?

Best regards
Sarah

sara@sara:~/Downloads/Lifting-from-the-Deep-release-master$ python Applications/Lifting-from-the-Deep/demo.py
Traceback (most recent call last):
File "Applications/Lifting-from-the-Deep/demo.py", line 11, in
from lifting import PoseEstimator
ImportError: No module named lifting

Hi Denis - Thank you for sharing this project.

I've run into an issue in which the model outputs highly skewed y coordinates for certain images. For example, the model outputs a reasonable re-construction of the first frame (img) in this sequence:

pose_3d = [[[ 165.8230328    67.03754957 -205.89986905  -32.38983883  302.10612467
     91.22563632  280.07067535   64.23923688 -121.04723249 -166.27962564
   -226.57145921   80.90181175  263.25173031  314.5440196  -233.47394329
   -260.15381899 -319.60138574]
  [ 209.65279238  314.71824629   82.10550074  165.22295977  104.58738359
   -168.25258589   13.24661237   87.6175417  -111.29930409 -200.36746041
   -205.78091077 -181.83588106 -160.04598113 -278.14880424   28.59255825
    181.96800836   99.68732301]
  [ -29.95290344  -30.35298069   -5.77654454 -337.00434106    6.86051195
   -103.72445841 -435.77584961  155.03801527  311.75958399  306.73182102
    533.42573399  223.34929849  -75.80560911 -363.73045755  260.07321576
     10.41625411 -163.05433098]]]

However, in other frames, the x and z coordinates remain consistent with similar frames, but the y coordinates are dramatically skewed, such as here (img):

pose_3d = [[[ 1.02271246e+02  9.62935992e+01 -2.34439449e+02 -1.05166807e+02
    2.98286838e+02  1.64104641e+02  3.41578049e+02 -1.27844847e+01
   -1.25346562e+02 -1.31937004e+02 -3.06537751e+02  2.64495353e+00
    2.90037744e+02  4.65259697e+02 -2.01997730e+02 -3.24498281e+02
   -3.60220340e+02]
  [-7.38312954e+04 -7.38708949e+04 -7.38310055e+04 -7.38060559e+04
   -7.37916961e+04 -7.37781771e+04 -7.38124515e+04 -7.36989474e+04
   -7.35605069e+04 -7.36014401e+04 -7.35446308e+04 -7.35147100e+04
   -7.34832985e+04 -7.35327440e+04 -7.36254187e+04 -7.39279633e+04
   -7.40327591e+04]
  [-2.10067792e+02 -2.62270589e+02 -3.37599039e+02 -7.12890082e+02
   -2.10924765e+02 -4.33412911e+02 -8.77269711e+02 -1.90837834e+01
    1.60809907e+02  3.17632726e+02  3.83645266e+02  1.99421824e+02
    3.73765312e+02  5.77611265e+02  1.04109928e+02  3.57356140e+02
    6.42948964e+02]]]

The y coordinates are showing up skewed by 2 orders of magnitude and some other factor I can't determine. This happens for around 20% of the images that I try. Do you have any idea what the problem is and how I could address it?

I want to obtain new model parameters that suit my data. How to get my own prob_model_params.mat?

Can this to do real-time?

I met an issue when I run the demo.py file. For some pictures, it works, but for some pictures, it gave an error.
How can I fix it?
Expect your answer!

Traceback (most recent call last):
File "demo1.py", line 68, in
sys.exit(main())
File "demo1.py", line 40, in main
pose_2d, visibility, pose_3d = pose_estimator.estimate(image)
File "/home/suiishii/Lifting-from-the-Deep-release/packages/lifting/_pose_estimator.py", line 144, in estimate
pose_3d = self.poseLifting.compute_3d(transformed_pose2d, weights)
File "/home/suiishii/Lifting-from-the-Deep-release/packages/lifting/utils/prob_model.py", line 256, in compute_3d
norm_pose, _ = Prob3dPose.normalise_data(reg_joints, weights)
File "/home/suiishii/Lifting-from-the-Deep-release/packages/lifting/utils/prob_model.py", line 107, in normalise_data
d2 = d2.reshape(d2.shape[0], -1, 2).transpose(0, 2, 1)
ValueError: total size of new array must be unchanged

Can you update the model in the repo. It says the demo model is not the accurate one used to generate results in the paper.

Hi
I had been reading , that the reason why it is very slow , is the fact the 2D , detections take the most of the time
issue stated here. Could you point , where one can change the input to for other 2D detectors to work ?

My second question , is that is there some limitation on the input picture ? It seems for some of the pictures it seem to lifiting perfectly and sometimes it returns an error

`Traceback (most recent call last):
  File "demo_modify.py", line 67, in <module>
    sys.exit(main())
  File "demo_modify.py", line 31, in main
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)  # conversion to rgb
cv2.error: OpenCV(4.1.0) /io/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

Hi I tried constructing a skeletal figure based on the connection values given in packages/lifting/utils/draw.py. To me first point (position 0) seems to be the hip midpoint and eight seems to be the neck. Can you confirm this?

Thanks,
Mritula

Hi,

I am able to generate the output using your code and I would like to take this to the next step. As this is not quite an issue per se, could you please share your email id so that I can share the details? Thanks.

Hi Denis, thanks for the great work.

Could you kindly share the aligned 3d poses or the code with which you perform the alignment?

Also, if I correctly understand how the alignment works, you are learning matrices e (of dimension J x 3L) and A (of dimension N x J), and N rotation matrices Ri (that are all about the y axis only), so that you can obtain every training set pose Pi by multiplying every element of the matrix e by the coefficients of a row ai of A, addind that to the mean pose of the training data and finally rotating it by Ri.

Once the optimization is done you can replace every training pose Pi with its `non-rotated' version (mu + ai * e) which basically points in the same direction of mu.

Could you please confirm my description is correct? If yes, could you explain why you need the noise variance sigma? If not, could you please help me understand?

Thanks a lot!

python3 pt3-webcam3D.py
Traceback (most recent call last):
File "pt3-webcam3D.py", line 17, in
from lifting.prob_model import Prob3dPose
ModuleNotFoundError: No module named 'lifting'

Please can someone help in that??Also I am not able to access the updated version of the repo..If someone have it,can somebody please send it??
Thanks in advance!!

Hi,

I was wondering, can you please release the training code as well?

Thanks!

How to train the model / use this code to estimate the pose of animals, birds and reptiles?

It is a amazing results of your wonderful code. how could we get the training code？

Hi,
As Python 2 cannot be used with COLAB I am getting the following errors: Can someone please let e know how to overcome it? Thanks In advance!

WARNING:tensorflow:Entity <bound method Conv.call of <tensorflow.python.layers.convolutional.Conv2D object at 0x7efc3ff15a10>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method Conv.call of <tensorflow.python.layers.convolutional.Conv2D object at 0x7efc3ff15a10>>: AttributeError: module 'gast' has no attribute 'Str'
WARNING:tensorflow:From /content/Lifting-from-the-Deep-release/packages/lifting/utils/cpm.py:167: The name tf.floordiv is deprecated. Please use tf.math.floordiv instead.

WARNING:tensorflow:From /content/Lifting-from-the-Deep-release/packages/lifting/utils/cpm.py:168: The name tf.mod is deprecated. Please use tf.math.mod instead.

2022-05-29 15:35:30.628677: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2022-05-29 15:35:30.632187: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2199995000 Hz
2022-05-29 15:35:30.632383: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x13a7340 executing computations on platform Host. Devices:
2022-05-29 15:35:30.632415: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): ,
2022-05-29 15:35:30.865153: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
WARNING:tensorflow:From /content/Lifting-from-the-Deep-release/packages/lifting/_pose_estimator.py:91: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
tcmalloc: large alloc 1248370688 bytes == 0x43c54000 @ 0x7efc8da251e7 0x7efc68f9ac05 0x7efc69a6f8fe 0x7efc6be4ba20 0x7efc6c44eb00 0x7efc6c4500ce 0x7efc6c4c0ab1 0x7efc6c4c4276 0x7efc6c4c4c03 0x7efc65aa1136 0x7efc65a915c5 0x7efc65b3b3ae 0x7efc65b38228 0x7efc8c307a50 0x7efc8d3e96db 0x7efc8d72261f
/content/Lifting-from-the-Deep-release/packages/lifting/utils/upright_fast.py:206: FutureWarning: rcond parameter will change to the default of machine precision times max(M, N) where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass rcond=None, to keep using the old, explicitly pass rcond=-1.
p_copy.T, res[j].T)

I am having success with accurate 2D detections and a 3D model that looks well-posed.
However upon closer inspections it seems that the left and right limbs differ in size greatly for a single frame. Is this expected behavior, or should I be seeing right-side limbs that are almost similar in size to the left-side limbs?

Also, what is the right way to scale the 3D points to real-word co-ordinates if i know my camera parameters?

How do I run it in a realtime video?

If the person in the picture is not standing (such as push-ups or inverted), it can not be promoted to 3d.
Will you follow up on this project so that it can support the actions of various scenes?

Hi,
I am getting the below error when I am trying to use pose estimator on images. Kindly suggest how to overcome this.

Error converting shape to tensorshape: only size-1 arrays can be converted to scalars

Thanks
Mritula

in normalise_data
d2 = d2.reshape(d2.shape[0], -1, 2).transpose(0, 2, 1)
ValueError: cannot reshape array of size 0 into shape (0,newaxis,2)

Hi,
I am getting the above error when giving a jpeg image for reconstruction. How am I to overcome this error?#
Thanks,
Mritula

Hi, thanks for your amazing work! But the model run for real-time maybe a little slow, i just know you have made a cython optimized version code , is the code open or is convenient for you to share the code with a link? Thanks in advance!

Dear author
Thanks for supplying so wonderful work. I run demo.py on my win10, I met an error, that is:

WARNING:tensorflow:From D:\Anaconda\lib\site-packages\tensorflow\python\training\saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
2019-11-20 17:22:47.504774: W tensorflow/core/framework/op_kernel.cc:1502] OP_REQUIRES failed at save_restore_v2_ops.cc:184 : Out of range: Read fewer bytes than requested
Traceback (most recent call last):
File "D:\Anaconda\lib\site-packages\tensorflow\python\client\session.py", line 1356, in _do_call
return fn(*args)
File "D:\Anaconda\lib\site-packages\tensorflow\python\client\session.py", line 1341, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "D:\Anaconda\lib\site-packages\tensorflow\python\client\session.py", line 1429, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.OutOfRangeError: Read fewer bytes than requested
[[{{node save/RestoreV2}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:/pycharm/teacher_analyse/Lifting-from-the-Deep/applications/demo.py", line 67, in
sys.exit(main())
File "D:/pycharm/teacher_analyse/Lifting-from-the-Deep/applications/demo.py", line 36, in main
pose_estimator.initialise()
File "D:\pycharm\teacher_analyse\Lifting-from-the-Deep\packages\lifting_pose_estimator.py", line 92, in initialise
saver.restore(sess, self.session_path)
File "D:\Anaconda\lib\site-packages\tensorflow\python\training\saver.py", line 1286, in restore
{self.saver_def.filename_tensor_name: save_path})
File "D:\Anaconda\lib\site-packages\tensorflow\python\client\session.py", line 950, in run
run_metadata_ptr)
File "D:\Anaconda\lib\site-packages\tensorflow\python\client\session.py", line 1173, in _run
feed_dict_tensor, options, run_metadata)
File "D:\Anaconda\lib\site-packages\tensorflow\python\client\session.py", line 1350, in _do_run
run_metadata)
File "D:\Anaconda\lib\site-packages\tensorflow\python\client\session.py", line 1370, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: Read fewer bytes than requested
[[node save/RestoreV2 (defined at \pycharm\teacher_analyse\Lifting-from-the-Deep\packages\lifting_pose_estimator.py:91) ]]

Original stack trace for 'save/RestoreV2':
File "/pycharm/teacher_analyse/Lifting-from-the-Deep/applications/demo.py", line 67, in
sys.exit(main())
File "/pycharm/teacher_analyse/Lifting-from-the-Deep/applications/demo.py", line 36, in main
pose_estimator.initialise()
File "\pycharm\teacher_analyse\Lifting-from-the-Deep\packages\lifting_pose_estimator.py", line 91, in initialise
saver = tf.compat.v1.train.Saver()
File "\Anaconda\lib\site-packages\tensorflow\python\training\saver.py", line 825, in init
self.build()
File "\Anaconda\lib\site-packages\tensorflow\python\training\saver.py", line 837, in build
self._build(self._filename, build_save=True, build_restore=True)
File "\Anaconda\lib\site-packages\tensorflow\python\training\saver.py", line 875, in _build
build_restore=build_restore)
File "\Anaconda\lib\site-packages\tensorflow\python\training\saver.py", line 508, in _build_internal
restore_sequentially, reshape)
File "\Anaconda\lib\site-packages\tensorflow\python\training\saver.py", line 328, in _AddRestoreOps
restore_sequentially)
File "\Anaconda\lib\site-packages\tensorflow\python\training\saver.py", line 575, in bulk_restore
return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
File "\Anaconda\lib\site-packages\tensorflow\python\ops\gen_io_ops.py", line 1779, in restore_v2
name=name)
File "\Anaconda\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "\Anaconda\lib\site-packages\tensorflow\python\util\deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "\Anaconda\lib\site-packages\tensorflow\python\framework\ops.py", line 3616, in create_op
op_def=op_def)
File "\Anaconda\lib\site-packages\tensorflow\python\framework\ops.py", line 2005, in init
self._traceback = tf_stack.extract_stack()

How to solve it? Can you help me? Thanks!

Hello! I'm trying to change the code to realtime 3d estimation. And I found that if my camera catches no people in one frame, this code will have this error:
ValueError: cannot reshape array of size 0 into shape (0,newaxis,2)
I don't have methods to fix or pass it..... :(
This code must guarantee some people in the image?

Error: (-215:Assertion failed) 0 < thickness && thickness <= MAX_THICKNESS in function 'cv::line'

I can't seem to run the 3D pose figure, while the pose_3d results are calculated ok.

Tried both

plot_pose(Prob3dPose.centre_all(single_3D))
plot_pose(single_3D)

Running python 2.7.15 and matplotlib v2.1.0

Hi,
Thanks for the wonderful application. I tried running demo.py and im getting the below mentioned error. I have installed Tensorflow1.12.0 and Opencv. Do let me know how to overcome this.

Thanks,
Mritula

Error:
(Lift_Deep) E:\Lift_From_Deep\applications>python demo.py
Traceback (most recent call last):
File "demo.py", line 11, in
from lifting import PoseEstimator
File "E:\Lift_From_Deep/packages\lifting_init_.py", line 1, in
from .pose_estimator import *
File "E:\Lift_From_Deep/packages\lifting_pose_estimator.py", line 7, in
from . import utils
File "E:\Lift_From_Deep/packages\lifting\utils_init.py", line 9, in
from .cpm import *
File "E:\Lift_From_Deep/packages\lifting\utils\cpm.py", line 5, in
import tensorflow as tf
File "C:\Users\user\Anaconda3\envs\Lift_Deep\lib\site-packages\tensorflow_init_.py", line 24, in
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File "C:\Users\user\Anaconda3\envs\Lift_Deep\lib\site-packages\tensorflow\python_init_.py", line 49, in
from tensorflow.python import pywrap_tensorflow
File "C:\Users\user\Anaconda3\envs\Lift_Deep\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 74, in
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "C:\Users\user\Anaconda3\envs\Lift_Deep\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 58, in
from tensorflow.python.pywrap_tensorflow_internal import *
File "C:\Users\user\Anaconda3\envs\Lift_Deep\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 28, in
_pywrap_tensorflow_internal = swig_import_helper()
File "C:\Users\user\Anaconda3\envs\Lift_Deep\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 20, in swig_import_helper
import _pywrap_tensorflow_internal
ImportError: No module named _pywrap_tensorflow_internal

Failed to load the native TensorFlow runtime.
See https://www.tensorflow.org/install/errors

for some common reasons and solutions. Include the entire stack trace
above this error message when asking for help.

Hello, I try to use the demo.py to get results but for some pictures, it give me the error as show as below.
How can I fix it?
Expect your answer.

Traceback (most recent call last):
File "demo.py", line 89, in
sys.exit(main())
File "demo.py", line 54, in main
pose_2d, visibility, pose_3d = pose_estimator.estimate(image)
File "/home/suiishii/Lifting-from-the-Deep-release/packages/lifting/_pose_estimator.py", line 144, in estimate
pose_3d = self.poseLifting.compute_3d(transformed_pose2d, weights)
File "/home/suiishii/Lifting-from-the-Deep-release/packages/lifting/utils/prob_model.py", line 256, in compute_3d
norm_pose, _ = Prob3dPose.normalise_data(reg_joints, weights)
File "/home/suiishii/Lifting-from-the-Deep-release/packages/lifting/utils/prob_model.py", line 107, in normalise_data
d2 = d2.reshape(d2.shape[0], -1, 2).transpose(0, 2, 1)
ValueError: total size of new array must be unchanged

Hi, I was trying to run the demo on a MacBook Pro 2016 base model and the inference_pose takes about 86 seconds per image. I was wondering if this is to be expected?
p.s. Going through the network defined in the CPM.py file, it looks like there's a number of conv_2d's happening not followed by pooling, so I imagine the resulting calculations are quite heavy.
p.s.2 I re-ran it using an optimised build of TensorFlow and the inference_pose ran in ~30 seconds.

Hello @DenisTome,

Loving your work on this repo. I was thinking of trying out this output in SMPL. Not sure how to map your coordinates to SMPL.

Regards.

I've used it on several images and, while some work perfectly, there are a couple of them that trigger a very weird error. Do you know why this might happen and how to fix this issue?

File "demo.py", line 67, in

pose3D = poseLifting.compute_3d(pose2D, weights)

File "/home/ubuntu/projects/Lifting-from-the-Deep-release/utils/prob_model.py", line 219, in compute_3d

norm_pose, _ = Prob3dPose.normalise_data(reg_joints, weights)

File "/home/ubuntu/projects/Lifting-from-the-Deep-release/utils/prob_model.py", line 100, in normalise_data

m2 = d2[:, 1, idx_consider].min(1) / 2.0

File "/usr/local/lib/python2.7/dist-packages/numpy/core/_methods.py", line 29, in _amin

return umr_minimum(a, axis, None, out, keepdims)

ValueError: zero-size array to reduction operation minimum which has no identity

Hi, author
I didn't explain my problem clearly before. The problem I face now is when I test my images, some of them are person with no face , that is back to the camera. In this situation, the model cannot detect person in images. Can model solve this problem? Thanks!

Hi,I try to run your demo on some low resolution person full body image(128*64) .it shows:
File "demo.py", line 57, in
sys.exit(main())
File "demo.py", line 32, in main
pose_2d, visibility, pose_3d = pose_estimator.estimate(image)
File "Lifting-from-the-Deep-release-master/graph_functions.py", line 136, in estimate
pose_3d = self.poseLifting.compute_3d(transformed_pose2d, weights)
File "Lifting-from-the-Deep-release-master/utils/prob_model.py", line 221, in compute_3d
norm_pose, _ = Prob3dPose.normalise_data(reg_joints, weights)
File "Lifting-from-the-Deep-release-master/utils/prob_model.py", line 95, in normalise_data
d2 = d2.reshape(d2.shape[0], -1, 2).transpose(0, 2, 1)
ValueError: cannot reshape array of size 0 into shape (0,newaxis,2)

is that means it did not get the keypoint?

For some images (haven't figured out the correlation) when using draw_limbs, terminal throws this error:

_COLORS[lid], LIMB_DRAW_SIZE*_NORMALISATION_FACTOR , 16)
cv2.error: OpenCV(4.0.1) /tmp/opencv-20190105-31032-o160to/opencv-4.0.1/modules/imgproc/src/drawing.cpp:1811: error: (-215:Assertion failed) 0 < thickness && thickness <= MAX_THICKNESS in function 'line'

There seems to be no logical explanation why plotting works fine for some images while not for others...

Dear author
Thanks for sharing so interesting work. I met an error when I ran demo.py, the error like this:

No visible people in the image. Change CENTER_TR in packages/lifting/utils/config.py ...

But in my picture, there is one person. How this happened？Can you give me some help? Thanks!

how do i run setup.sh and demo file

Hi,

I'm testing the project on an image sequence, and while I manage to successfully get poses for the sequence, they are always centered to origin. I tried to initialise a PoseEstimator for the whole sequence or one per frame without much differences. Am I missing something obvious?

Cheers,

Thomas

I got this error: "Segmentation fault (core dumped)" when running your scripts on the cpu.

Hello!

I would like to know what the minimum requirements on the os are, such as cpu, ram, graphics card...

Thanks!

I ran your code on GTX-1080ti and it was hardly giving 5 fps after muting visualization. I have generated a frozen graph from meta data and then tried to convert that graph to tensor-rt. But it failed to convert. Can you share frozen graph of your model or upload a converted tensor-rt model?

I'm getting the following error when running with a particular frame from a video:

File "demo.py", line 149, in <module>
pose3D = poseLifting.compute_3d(pose2D, weights)
File "/home/ubuntu/projects/Lifting-from-the-Deep-release/utils/prob_model.py", line 231, in compute_3d
pose_3d = self.create_rec(norm_pose, weights)*_SCALE_3D
File "/home/ubuntu/projects/Lifting-from-the-Deep-release/utils/prob_model.py", line 191, in create_rec
depth_reg=0, cap_scale=-0.001, scale_mean=-0.003)
File "/home/ubuntu/projects/Lifting-from-the-Deep-release/utils/prob_model.py", line 154, in affine_estimate
interval=0.01, depth_reg=depth_reg, scale_prior=scale_mean)
File "upright_fast.pyx", line 76, in upright_fast.pick_e (upright_fast.cpp:4074)
File "upright_fast.pyx", line 263, in upright_fast.estimate_a_and_r_with_res_weights (upright_fast.cpp:8240)
ValueError: setting an array element with a sequence.

Any idea what might cause it?

It works really well with full body people even playing soccer.
Any advises to deal with invisible body parts in a picture(like just upper bodies shown in a frame)??

Hi , I am currently looking into python implementationo posenet to do 2D pose estimation.
posenet-python. In it , the following decode single person. The following return the 17 keypoints instance_keypoint_coords and pose confidence. instance_keypoint_scores .

Where can we input to get the corresponding 3d estimation ?

Can it also be extended to videos ?

Greeting! I am running the 3D pose by this paper (Lifting from the deep), the 2D human joints are from Oppenpose (https://github.com/ildoonet/tf-pose-estimation)

I test each frame I need approximate 0.05s to process the 2D joints, while it requires near 10s to transform 3D, so it is impossible to construct real-time system. However, I see the demo in Lifting, it is real-time, how do you do that? And may I ask how to speed up the 3D transform ? Thanks!

You would need a big dataset with 3D data (either captured or synthetically generated) and follow the same steps described in the paper

I have 1 GB of snake image dataset, which itself contains different gait/pose pics, but the problem when i tried to use this model is, it couldnt recognise the pictures, either due to thickness, or it just couldnt find any humans in image.

INFO:tensorflow:Restoring parameters from /content/saved_sessions/init_session/init No visible people in the image. Change CENTER_TR in packages/lifting/utils/config.py ... An exception has occurred, use %tb to see the full traceback.

@DenisTome

Originally posted by @ArimaValanImmanuel in #56 (comment)

Hi Denis,
I am trying to map the 2d joints, from another 2d detector. I note that transformed_pose2d is a 1x14x2 array . I was wondering what is the joint order that is expected. I have the following two standard detectors with joint order .I was wondering what is the joint order that is expected so that
I can map another 2d joint detector .

`  For StackedHourglass
  0  'RFoot'
  1  'RKnee'
  2  'RHip'
  3  'LHip'
  4  'LKnee'
  5  'LFoot',
  6  'Hip'
  7  'Spine'
  8  'Thorax'
  9  'Head'
  10  'RWrist'
  11  'RElbow'
  12  'Rshoulder'
  13  'LShoulder'
  14  'LElbow'
  15  'LWrist'
  
  For Posenet 
  'nose',             0
  'leftEye',          1
  'rightEye',         2
  'leftEar',          3
  'rightEar',         4
  'leftShoulder',     5
  'rightShoulder',    6
  'leftElbow',        7
  'rightElbow',       8
  'leftWrist',        9
  'rightWrist',      10 
  'leftHip',         11  
  'rightHip',        12
  'leftKnee', '      13
  rightKnee',        14
  'leftAnkle',       15
  'rightAnkle'       16
`