Problem with the pretrained models: about rp-vio

[sudarshan-s-harithas]

Hi,

It appears that you have given the wrong path to the pre-trained model.

Please try the following:

1. A pre-trained model is present in the cloned repository at rp-vio/plane_segmentation/pretrained_model/ you have to run this file.
2. In the inference step the path required for the ckpt_file will typically look like this, for the given name in which our model is saved

--ckpt_file=/rp-vio/plane_segmentation/pretrained_model/model

3. Ensure that the installations are working fine try importing tensorflow, keras, numpy. The code is developed on tensorflow1 ensure you have appropriate installations.

Try the above steps if the issue persists please post the entire command you used to run the script and the complete error message.

Thank You

[Gatsby23]

Hi,

It appears that you have given the wrong path to the pre-trained model.

Please try the following:

1. A pre-trained model is present in the cloned repository at rp-vio/plane_segmentation/pretrained_model/ you have to run this file.
2. In the inference step the path required for the ckpt_file will typically look like this, for the given name in which our model is saved

--ckpt_file=/rp-vio/plane_segmentation/pretrained_model/model

1. Ensure that the installations are working fine try importing tensorflow, keras, numpy. The code is developed on tensorflow1 ensure you have appropriate installations.

Try the above steps if the issue persists please post the entire command you used to run the script and the complete error message.

Thank You

Thank you very much. Here is my command:

 python inference.py --dataset=/home/robotics/Data/Kitti_Data/dataset/sequences/00 --output_dir=/home/robotics/Data/Kitti_Data/dataset/sequences/00/segment_planes --test_list=/home/robotics/Data/Kitti_Data/dataset/sequences/00/test.txt --ckpt_file=pretrained_model/model --use_preprocessed=true 

and I got error like this:

/home/robotics/Application/anaconda3/envs/plane_seg/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:469: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/robotics/Application/anaconda3/envs/plane_seg/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:470: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/robotics/Application/anaconda3/envs/plane_seg/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:471: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/robotics/Application/anaconda3/envs/plane_seg/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:472: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/robotics/Application/anaconda3/envs/plane_seg/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:473: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/robotics/Application/anaconda3/envs/plane_seg/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:476: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
0
2021-06-29 17:59:02.985633: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
processing model: 0/5
No.0 batch cost_time: 0.090702/img
Traceback (most recent call last):
  File "inference.py", line 411, in <module>
    tf.app.run()
  File "/home/robotics/Application/anaconda3/envs/plane_seg/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 48, in run
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "inference.py", line 334, in main
    scipy.misc.imsave(visual_path + pic_name, color_plane_mask)
AttributeError: module 'scipy.misc' has no attribute 'imsave'

I'm sure my scipy version is: 0.19.1, which is in the requirements.txt. But here also remind me lack of attribute 'imsave'

[sudarshan-s-harithas]

Hi,

Even our system is working with scipy version 0.19.1 and I have faced no issues.

Can you please try to install the previous version of scipy (0.17.0) as mentioned in one of these answers.

Or you can try to use imageio as described here, and even opencv can be tried with their cv2.imwrite( ...) function.

I can try to reproduce this error at our end and can get back in case we find the error.

Thank You

[Gatsby23]

Hi,

Even our system is working with scipy version 0.19.1 and I have faced no issues.

Can you please try to install the previous version of scipy (0.17.0) as mentioned in one of these answers.

Or you can try to use imageio as described here, and even opencv can be tried with their cv2.imwrite( ...) function.

I can try to reproduce this error at our end and can get back in case we find the error.

Thank You

OK, I'll give it a try. But I really don't know why install with requirements, I met this error:

Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - gnutls==3.6.13=h85f3911_1
  - lame==3.100=h14c3975_1001
  - cycler==0.10.0=py_2
  - libxcb==1.13=h14c3975_1002
  - icu==67.1=he1b5a44_0
  - protobuf==3.13.0.1=py36h201607c_1
  - libprotobuf==3.13.0.1=h8b12597_0
  - libpq==12.3=h255efa7_3
  - xorg-kbproto==1.0.7=h14c3975_1002
  - xorg-libx11==1.6.12=h516909a_0
  - libxkbcommon==0.10.0=he1b5a44_0
  - werkzeug==1.0.1=pyh9f0ad1d_0
  - numpy==1.17.0=py36h95a1406_0
  - mysql-libs==8.0.21=hf3661c5_2
  - hdf5==1.10.6=nompi_h3c11f04_101
  - nss==3.59=h2c00c37_0
  - pthread-stubs==0.4=h36c2ea0_1001
  - webencodings==0.5.1=py_1
  - xorg-libxrender==0.9.10=h516909a_1002
  - six==1.15.0=pyh9f0ad1d_0
  - joblib==1.0.0=pyhd8ed1ab_0
  - plyfile==0.7.2=pyh9f0ad1d_0
  - libuuid==2.32.1=h7f98852_1000
  - libclang==10.0.1=default_hde54327_1
  - pyqtchart==5.12=py36h7ec31b9_6
  - libtiff==4.1.0=h4f3a223_6
  - _libgcc_mutex==0.1=conda_forge
  - libwebp-base==1.1.0=h36c2ea0_3
  - libpng==1.6.37=h21135ba_2
  - mkl_fft==1.1.0=py36hc1659b7_1
  - gst-plugins-base==1.14.5=h0935bb2_2
  - html5lib==0.9999999=py36_0
  - dbus==1.13.6=hfdff14a_1
  - addict==2.3.0=py36h9f0ad1d_2
  - lz4-c==1.9.2=he1b5a44_3
  - libssh2==1.9.0=hab1572f_5
  - matplotlib==3.3.3=py36h5fab9bb_0
  - gettext==0.19.8.1=hf34092f_1004
  - openh264==2.1.1=h8b12597_0
  - pcre==8.44=he1b5a44_0
  - matplotlib-base==3.3.3=py36he12231b_0
  - pyqt==5.12.3=py36h5fab9bb_6
  - gmp==6.2.1=h58526e2_0
  - mysql-common==8.0.21=2
  - markdown==3.3.3=pyh9f0ad1d_0
  - xorg-libsm==1.2.3=h84519dc_1000
  - liblapack==3.8.0=14_openblas
  - mkl_random==1.1.0=py36hb3f55d8_0
  - harfbuzz==2.7.2=ha5b49bf_1
  - qt==5.12.9=h1f2b2cb_0
  - opencv==4.5.0=py36_4
  - importlib-metadata==3.3.0=py36h5fab9bb_2
  - libcblas==3.8.0=14_openblas
  - jpeg==9d=h36c2ea0_0
  - pyqtwebengine==5.12.1=py36h7ec31b9_6
  - cairo==1.16.0=h9f066cc_1006
  - libev==4.33=h516909a_1
  - pyqt-impl==5.12.3=py36h7ec31b9_6
  - freetype==2.10.4=h7ca028e_0
  - libgomp==9.3.0=h5dbcf3e_17
  - pydensecrf==1.0rc2=pypi_0
  - graphite2==1.3.13=h58526e2_1001
  - libopencv==4.5.0=py36_4
  - libffi==3.2.1=he1b5a44_1007
  - libevent==2.1.10=hcdb4288_3
  - nettle==3.6=he412f7d_0
  - libiconv==1.16=h516909a_0
  - zipp==3.4.0=py_0
  - xorg-renderproto==0.11.1=h14c3975_1002
  - open3d==0.11.2=py36_0
  - bzip2==1.0.8=h7f98852_4
  - liblapacke==3.8.0=14_openblas
  - py-opencv==4.5.0=py36he448a4c_4
  - xorg-xproto==7.0.31=h7f98852_1007
  - bleach==1.5.0=py36_0
  - xorg-libxext==1.3.4=h516909a_0
  - tornado==6.1=py36h1d69622_0
  - blas==2.14=openblas
  - python==3.6.11=h425cb1d_0_cpython
  - libnghttp2==1.41.0=h8cfc5f6_2
  - python_abi==3.6=1_cp36m
  - libcurl==7.71.1=hcdd3856_8
  - pyqt5-sip==4.19.18=py36hc4f0c31_6
  - c-ares==1.17.1=h36c2ea0_0
  - libglib==2.66.3=hbe7bbb4_0
  - libgcc-ng==9.3.0=h5dbcf3e_17
  - krb5==1.17.2=h926e7f8_0
  - x264==1!152.20180806=h14c3975_0
  - fontconfig==2.13.1=h7e3eb15_1002
  - pyparsing==2.4.7=pyh9f0ad1d_0
  - pyyaml==5.3.1=py36he6145b8_1
  - libopenblas==0.3.7=h5ec1e0e_6
  - pillow==8.0.1=py36h10ecd5c_0
  - glib==2.66.3=h58526e2_0
  - xorg-libxau==1.0.9=h14c3975_0
  - gstreamer==1.14.5=h36ae1b5_2
  - xorg-libxdmcp==1.1.3=h516909a_0
  - libblas==3.8.0=14_openblas
  - ffmpeg==4.3.1=h3215721_1
  - jasper==1.900.1=h07fcdf6_1006
  - kiwisolver==1.3.1=py36h51d7077_0
  - libllvm10==10.0.1=he513fc3_3
  - nspr==4.29=he1b5a44_1
  - zstd==1.4.5=h6597ccf_2
  - expat==2.2.9=he1b5a44_2
  - xorg-libice==1.0.10=h516909a_0
  - xorg-xextproto==7.3.0=h14c3975_1002
  - libxml2==2.9.10=h68273f3_2
  - olefile==0.46=pyh9f0ad1d_1
  - tqdm==4.55.0=pyhd8ed1ab_0
  - libstdcxx-ng==9.3.0=h2ae2ef3_17
  - pixman==0.40.0=h36c2ea0_0
  - lcms2==2.11=hcbb858e_1

Current channels:

  - https://repo.anaconda.com/pkgs/main/linux-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-64
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

[Gatsby23]

Hi,

Even our system is working with scipy version 0.19.1 and I have faced no issues.

Can you please try to install the previous version of scipy (0.17.0) as mentioned in one of these answers.

Or you can try to use imageio as described here, and even opencv can be tried with their cv2.imwrite( ...) function.

I can try to reproduce this error at our end and can get back in case we find the error.

Thank You

When I install the scipy==0.17.0, I met this error:

Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - scipy==0.17.0

Current channels:

  - https://repo.anaconda.com/pkgs/main/linux-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-64
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

[Gatsby23]

Hi,

It appears that you have given the wrong path to the pre-trained model.

Please try the following:

1. A pre-trained model is present in the cloned repository at rp-vio/plane_segmentation/pretrained_model/ you have to run this file.
2. In the inference step the path required for the ckpt_file will typically look like this, for the given name in which our model is saved

--ckpt_file=/rp-vio/plane_segmentation/pretrained_model/model

1. Ensure that the installations are working fine try importing tensorflow, keras, numpy. The code is developed on tensorflow1 ensure you have appropriate installations.

Try the above steps if the issue persists please post the entire command you used to run the script and the complete error message.

Thank You

Another thing is the image size. If I wanna to resize image width and height, how should I modify the inference code ? When I set the image width and image height directly like 341 and 1726. But it shows error like this:

ValueError: Dimension 1 in both shapes must be equal, but are 48 and 47 for 'plane_predication/depth_net/mask/concat_1' (op: 'ConcatV2') with input shapes: [4,48,156,128], [4,47,156,128], [] and with computed input tensors: input[2] = <3>.

[karnikram]

Another thing is the image size. If I wanna to resize image width and height, how should I modify the inference code ?

This is not easy to do and we don't recommend as you have to adjust the dimensions of all the layers. Please resize your input image.

[sudarshan-s-harithas]

Hi,

The inference works on fixed input dimensions of (192,320).

But you can always upscale the output plane detection image to your desired shape using the opencv resize function.

A sample code can be found here.

[Gatsby23]

Another thing is the image size. If I wanna to resize image width and height, how should I modify the inference code ?

This is not easy to do and we don't recommend as you have to adjust the dimensions of all the layers. Please resize your input image.

Question is that when you rescale the image, the camera intrinsic will satisfize or not and to my knowledge, it will also impact the localization accuracy ?

[karnikram]

This resizing is only for the model, the output of the model is again upsampled to the original image size before passing it to the VIO pipeline.

[Gatsby23]

This resizing is only for the model, the output of the model is again upsampled to the original image size before passing it to the VIO pipeline.

OK, It means that the output visual image is not the input of the VIO system ？

[karnikram]

The output from the model is a mask image. This output is upsampled to the original image size, and then passed together with the original RBG image to the VIO system as a ROS bagfile.

[calmelo]

Hi,
Even our system is working with scipy version 0.19.1 and I have faced no issues.
Can you please try to install the previous version of scipy (0.17.0) as mentioned in one of these answers.
Or you can try to use imageio as described here, and even opencv can be tried with their cv2.imwrite( ...) function.
I can try to reproduce this error at our end and can get back in case we find the error.
Thank You

OK, I'll give it a try. But I really don't know why install with requirements, I met this error:

Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - gnutls==3.6.13=h85f3911_1
  - lame==3.100=h14c3975_1001
  - cycler==0.10.0=py_2
  - libxcb==1.13=h14c3975_1002
  - icu==67.1=he1b5a44_0
  - protobuf==3.13.0.1=py36h201607c_1
  - libprotobuf==3.13.0.1=h8b12597_0
  - libpq==12.3=h255efa7_3
  - xorg-kbproto==1.0.7=h14c3975_1002
  - xorg-libx11==1.6.12=h516909a_0
  - libxkbcommon==0.10.0=he1b5a44_0
  - werkzeug==1.0.1=pyh9f0ad1d_0
  - numpy==1.17.0=py36h95a1406_0
  - mysql-libs==8.0.21=hf3661c5_2
  - hdf5==1.10.6=nompi_h3c11f04_101
  - nss==3.59=h2c00c37_0
  - pthread-stubs==0.4=h36c2ea0_1001
  - webencodings==0.5.1=py_1
  - xorg-libxrender==0.9.10=h516909a_1002
  - six==1.15.0=pyh9f0ad1d_0
  - joblib==1.0.0=pyhd8ed1ab_0
  - plyfile==0.7.2=pyh9f0ad1d_0
  - libuuid==2.32.1=h7f98852_1000
  - libclang==10.0.1=default_hde54327_1
  - pyqtchart==5.12=py36h7ec31b9_6
  - libtiff==4.1.0=h4f3a223_6
  - _libgcc_mutex==0.1=conda_forge
  - libwebp-base==1.1.0=h36c2ea0_3
  - libpng==1.6.37=h21135ba_2
  - mkl_fft==1.1.0=py36hc1659b7_1
  - gst-plugins-base==1.14.5=h0935bb2_2
  - html5lib==0.9999999=py36_0
  - dbus==1.13.6=hfdff14a_1
  - addict==2.3.0=py36h9f0ad1d_2
  - lz4-c==1.9.2=he1b5a44_3
  - libssh2==1.9.0=hab1572f_5
  - matplotlib==3.3.3=py36h5fab9bb_0
  - gettext==0.19.8.1=hf34092f_1004
  - openh264==2.1.1=h8b12597_0
  - pcre==8.44=he1b5a44_0
  - matplotlib-base==3.3.3=py36he12231b_0
  - pyqt==5.12.3=py36h5fab9bb_6
  - gmp==6.2.1=h58526e2_0
  - mysql-common==8.0.21=2
  - markdown==3.3.3=pyh9f0ad1d_0
  - xorg-libsm==1.2.3=h84519dc_1000
  - liblapack==3.8.0=14_openblas
  - mkl_random==1.1.0=py36hb3f55d8_0
  - harfbuzz==2.7.2=ha5b49bf_1
  - qt==5.12.9=h1f2b2cb_0
  - opencv==4.5.0=py36_4
  - importlib-metadata==3.3.0=py36h5fab9bb_2
  - libcblas==3.8.0=14_openblas
  - jpeg==9d=h36c2ea0_0
  - pyqtwebengine==5.12.1=py36h7ec31b9_6
  - cairo==1.16.0=h9f066cc_1006
  - libev==4.33=h516909a_1
  - pyqt-impl==5.12.3=py36h7ec31b9_6
  - freetype==2.10.4=h7ca028e_0
  - libgomp==9.3.0=h5dbcf3e_17
  - pydensecrf==1.0rc2=pypi_0
  - graphite2==1.3.13=h58526e2_1001
  - libopencv==4.5.0=py36_4
  - libffi==3.2.1=he1b5a44_1007
  - libevent==2.1.10=hcdb4288_3
  - nettle==3.6=he412f7d_0
  - libiconv==1.16=h516909a_0
  - zipp==3.4.0=py_0
  - xorg-renderproto==0.11.1=h14c3975_1002
  - open3d==0.11.2=py36_0
  - bzip2==1.0.8=h7f98852_4
  - liblapacke==3.8.0=14_openblas
  - py-opencv==4.5.0=py36he448a4c_4
  - xorg-xproto==7.0.31=h7f98852_1007
  - bleach==1.5.0=py36_0
  - xorg-libxext==1.3.4=h516909a_0
  - tornado==6.1=py36h1d69622_0
  - blas==2.14=openblas
  - python==3.6.11=h425cb1d_0_cpython
  - libnghttp2==1.41.0=h8cfc5f6_2
  - python_abi==3.6=1_cp36m
  - libcurl==7.71.1=hcdd3856_8
  - pyqt5-sip==4.19.18=py36hc4f0c31_6
  - c-ares==1.17.1=h36c2ea0_0
  - libglib==2.66.3=hbe7bbb4_0
  - libgcc-ng==9.3.0=h5dbcf3e_17
  - krb5==1.17.2=h926e7f8_0
  - x264==1!152.20180806=h14c3975_0
  - fontconfig==2.13.1=h7e3eb15_1002
  - pyparsing==2.4.7=pyh9f0ad1d_0
  - pyyaml==5.3.1=py36he6145b8_1
  - libopenblas==0.3.7=h5ec1e0e_6
  - pillow==8.0.1=py36h10ecd5c_0
  - glib==2.66.3=h58526e2_0
  - xorg-libxau==1.0.9=h14c3975_0
  - gstreamer==1.14.5=h36ae1b5_2
  - xorg-libxdmcp==1.1.3=h516909a_0
  - libblas==3.8.0=14_openblas
  - ffmpeg==4.3.1=h3215721_1
  - jasper==1.900.1=h07fcdf6_1006
  - kiwisolver==1.3.1=py36h51d7077_0
  - libllvm10==10.0.1=he513fc3_3
  - nspr==4.29=he1b5a44_1
  - zstd==1.4.5=h6597ccf_2
  - expat==2.2.9=he1b5a44_2
  - xorg-libice==1.0.10=h516909a_0
  - xorg-xextproto==7.3.0=h14c3975_1002
  - libxml2==2.9.10=h68273f3_2
  - olefile==0.46=pyh9f0ad1d_1
  - tqdm==4.55.0=pyhd8ed1ab_0
  - libstdcxx-ng==9.3.0=h2ae2ef3_17
  - pixman==0.40.0=h36c2ea0_0
  - lcms2==2.11=hcbb858e_1

Current channels:

  - https://repo.anaconda.com/pkgs/main/linux-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-64
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

Have you resolved the problem? I meet the problem too.