I want to render my model with cad option ,however, i have used DWG, DXF,3DS and STL model type and have error as follow

(aae) gaobo@gaobo-ubuntu:~/SSD_ws$ ae_train exp_group/my_autoencoder -d
128 128 3
[[8, 8], [16, 16], [32, 32], [64, 64]]
(?, 128, 128, 3)
(?, 128, 128, 3)
Traceback (most recent call last): |
File "/home/gaobo/.local/bin/ae_train", line 10, in
sys.exit(main())
File "/home/gaobo/.local/lib/python2.7/site-packages/auto_pose/ae/ae_train.py", line 90, in main
dataset.get_training_images(dataset_path, args)
File "/home/gaobo/.local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 93, in get_training_images
self.render_training_images()
File "/home/gaobo/.local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 245, in render_training_images
bgr_x, depth_x = self.renderer.render(
File "/home/gaobo/.local/lib/python2.7/site-packages/auto_pose/ae/utils.py", line 15, in decorator
setattr(self, attribute, function(self))
File "/home/gaobo/.local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 68, in renderer
float(self._kw['vertex_scale'])
File "/home/gaobo/.local/lib/python2.7/site-packages/auto_pose/meshrenderer/meshrenderer.py", line 37, in init
vert_norms = gu.geo.load_meshes(models_cad_files, vertex_tmp_store_folder, recalculate_normals=True)
File "/home/gaobo/.local/lib/python2.7/site-packages/auto_pose/meshrenderer/gl_utils/geometry.py", line 54, in load_meshes
scene = pyassimp.load(model_path, pyassimp.postprocess.aiProcess_Triangulate)
File "/home/gaobo/anaconda3/envs/aae/lib/python2.7/site-packages/pyassimp/core.py", line 320, in load
raise AssimpError('Could not import file!')
pyassimp.errors.AssimpError: Could not import file!

I just want to know which type of model can be used with cad option

Is there a way adapt it to support a 16bit depth image. If I use it directly it does not work, as seen by the print out in icp.py in the icp_refinement method:

real min max x -18383.785420814358 16734.17997734934
real min max y -18381.14515417148 12457.235152135814
real min max z 74000.0 149000.0
syn min max x -43.666910471988324 7.3976425377620245
syn min max y -15.081647411564028 32.52808661089217
syn min max z 270.7138977050781 311.607055664062

If i just load it as 8bit the numbers look closer i guess:

real min max x -123.38111020680778 122.1472991047397
real min max y -123.36339029645289 122.12975639348836
real min max z 1000.0 1000.0
syn min max x -43.666910471988324 7.3976425377620245
syn min max y -15.081647411564028 32.52808661089217
syn min max z 270.7138977050781 311.6070556640625

However when I inspect the 8bit image i do not get a depth difference in my object due to the lower depth resolution. Im therefore pretty sure i need the 16 bit depth.

I am currently just testing by loading image files with opencv, so they load as uint8 and uint16.

Hi,
I want to reproduce results in paper. So I trained a retinanet following the instruction:
1. Generate a training dataset by running 'python detection_utils/generate_sixd_train.py'
2. Train a retinanet according to ' https://github.com/fizyr/keras-retinanet'
However, I only get a mean ap 18% on TLESS dataset at the threshold IOU=0.5
So, What could be the problem? I would be grateful if you could provide detailed configuration of training retinanet.

I followed README to install the packages but got the following error:

'glCreateTextures' is not defined.

@MartinSmeyer Do you have any ideas?

Hi, after I create the training config file, and insert the paths of my 3d model, when I run "ae_train exp_group/my_autoencoder -d" I get following ValueError :

128 128 3
[[8, 8], [16, 16], [32, 32], [64, 64]]
(?, 128, 128, 3)
(?, 128, 128, 3)
Traceback (most recent call last):20000 ] | | (ETA: --:--:--)
File "/localhome/demo/autoencoder_6d_pose_estimation/venv/bin/ae_train", line 11, in
load_entry_point('auto-pose==0.9', 'console_scripts', 'ae_train')()
File "/localhome/demo/autoencoder_6d_pose_estimation/venv/local/lib/python2.7/site-packages/auto_pose/ae/ae_train.py", line 90, in main
dataset.get_training_images(dataset_path, args)
File "/localhome/demo/autoencoder_6d_pose_estimation/venv/local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 93, in get_training_images
self.render_training_images()
File "/localhome/demo/autoencoder_6d_pose_estimation/venv/local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 246, in render_training_images
bgr_x, depth_x = self.renderer.render(
File "/localhome/demo/autoencoder_6d_pose_estimation/venv/local/lib/python2.7/site-packages/auto_pose/ae/utils.py", line 15, in decorator
setattr(self, attribute, function(self))
File "/localhome/demo/autoencoder_6d_pose_estimation/venv/local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 75, in renderer
float(self._kw['vertex_scale'])
File "/localhome/demo/autoencoder_6d_pose_estimation/venv/local/lib/python2.7/site-packages/auto_pose/meshrenderer/meshrenderer_phong.py", line 48, in init
for vertex, normal, color, faces in attributes:
ValueError: need more than 3 values to unpack

It' look like the attributes are not correctly loaded, as I check the attributes in debug mode the size of attributes is 3 not 4. The 3d model I use is T-LESS models_cad/obj_01.ply. I tried other .ply meshes and have also problems.
the background image: VOCdevkit/VOC2012/JPEGImages/'.jpg

System Info
My environment:
I'm using pycharm virtualenvironment

• GPU: GeForce GTX 1050
• Python 2.7
• PyOpenGL 3.1.3b2
• PyOpenGl-accelerate 3.1.3b1

thanx..

Hello, the YCB-video dataset is becoming a standard SIXD benchmark for deep learning today. Did you have tested your approach on it? If so, could you provide the corresponding numbers?

How to get the 1-dim latent code for visualization? The original latent code from the encoder is a multi-dim vector.

Models_Reconst works well, but Models_CAD show:
" No handlers could be found for logger "OpenGL.arrays.numpymodule"ETA: --:--:-- " Segmentation fault (core dumped) "
I have change the MODEL from "reconst" to " cad" in the .cfg file

Hi. I struggled so much and finally I came into be able to train this model(I added normals by using meshlab).
https://www.laserdesign.com/sample-files/hot-red-pepper/

The result is like this.

But when I train a cube-like model I got this error.
https://github.com/skn047/machine_learning/blob/master/red_object_meshlab.ply

root@6e76b1f6d984:/home/kuromiya/Documents/AugmentedAutoencoder# ae_train exp_group/my_autoencoder -d

Traceback (most recent call last): |
File "/home/kuromiya/.local/bin/ae_train", line 10, in
sys.exit(main())
File "/root/.local/lib/python2.7/site-packages/auto_pose/ae/ae_train.py", line 90, in main
dataset.get_training_images(dataset_path, args)
File "/root/.local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 93, in get_training_images
self.render_training_images()
File "/root/.local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 245, in render_training_images
bgr_x, depth_x = self.renderer.render(
File "/root/.local/lib/python2.7/site-packages/auto_pose/ae/utils.py", line 15, in decorator
setattr(self, attribute, function(self))
File "/root/.local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 68, in renderer
float(self._kw['vertex_scale'])
File "/root/.local/lib/python2.7/site-packages/auto_pose/meshrenderer/meshrenderer.py", line 37, in init
vert_norms = gu.geo.load_meshes(models_cad_files, vertex_tmp_store_folder, recalculate_normals=True)
File "/root/.local/lib/python2.7/site-packages/auto_pose/meshrenderer/gl_utils/geometry.py", line 57, in load_meshes
normals = calc_normals(vertices) if recalculate_normals else mesh.normals
File "/root/.local/lib/python2.7/site-packages/auto_pose/meshrenderer/gl_utils/geometry.py", line 69, in calc_normals
v3 = vertices[i+2]
IndexError: index 8 is out of bounds for axis 0 with size 8

How can I solve this problem ?

Thanks for the code for evaluation. I have trained it on a custom object:

I tested it on 3 samples(first row) and the result is (second row):

The middle object has an incorrect output.
What is your idea on this issue?

HI！
I want yo use ICP algorithm to refine R and t,but there are some questions that bother me...
I konw there is an ICP option in the cfg file and i can set it to TRUE,but what else should i do?
we know that a rgbd camera have a rgb camera and a depth camera,and the function process_pose has two parameters:color_img, depth_img.Should i align depth_img with color_img first?

1. I trained Autoencoder for my 3d model
2. I trained keras-retinanet for detecting my object.
   How can I evaluate the whole system to get 6DOF of the object in a single RGB image?
   Thank you!

I was wondering if this would be possible to run in a docker container by enabling gui in the container:
http://wiki.ros.org/docker/Tutorials/GUI

I have played a bit with it, but I am running into some issues - not sure if they are related to being in a docker container or not.

This is the output from running the ae_train.py
128 128 3
[[8, 8], [16, 16], [32, 32], [64, 64]]
(?, 128, 128, 3)
(?, 128, 128, 3)
libGL error: No matching fbConfigs or visuals found | 0 / 20000 ETA: --:--:--
libGL error: failed to load driver: swrast
Traceback (most recent call last):
File "ae_train.py", line 160, in
main()
File "ae_train.py", line 90, in main
dataset.get_training_images(dataset_path, args)
File "/app/AugmentedAutoencoder-master/auto_pose/ae/dataset.py", line 93, in get_training_images
self.render_training_images()
File "/app/AugmentedAutoencoder-master/auto_pose/ae/dataset.py", line 245, in render_training_images
bgr_x, depth_x = self.renderer.render(
File "/app/AugmentedAutoencoder-master/auto_pose/ae/utils.py", line 15, in decorator
setattr(self, attribute, function(self))
File "/app/AugmentedAutoencoder-master/auto_pose/ae/dataset.py", line 75, in renderer
float(self._kw['vertex_scale'])
File "/app/AugmentedAutoencoder-master/auto_pose/ae/meshrenderer/meshrenderer_phong.py", line 22, in init
self._fbo = gu.Framebuffer( { GL_COLOR_ATTACHMENT0: gu.Texture(GL_TEXTURE_2D, 1, GL_RGB8, W, H),
File "/app/AugmentedAutoencoder-master/auto_pose/ae/meshrenderer/gl_utils/texture.py", line 11, in init
glCreateTextures(tex_type, len(self.__id), self.__id)
File "src/latebind.pyx", line 32, in OpenGL_accelerate.latebind.LateBind.call
File "src/wrapper.pyx", line 311, in OpenGL_accelerate.wrapper.Wrapper.call
File "/root/.local/lib/python2.7/site-packages/OpenGL/platform/baseplatform.py", line 414, in call
self.name, self.name,
OpenGL.error.NullFunctionError: Attempt to call an undefined function glCreateTextures, check for bool(glCreateTextures) before calling

I couldn't run the script because of a Segmentation Fault caused by glfw.CreateWindow(1, 1, "", None) in file glfw_off_screen_context.py (line 14).

The message in the terminal is

Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)

System Info

• Linux version: 14.04, with local display connected.
• Python version: 2.7 (system interpreter, no virtualenv)
• nvidia-smi output:

  +-----------------------------------------------------------------------------+
  | NVIDIA-SMI 390.87                 Driver Version: 390.87                    |
  |-------------------------------+----------------------+----------------------+
  | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
  | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
  |===============================+======================+======================|
  |   0  GeForce GTX 770     Off  | 00000000:01:00.0 N/A |                  N/A |
  | 17%   40C    P8    N/A /  N/A |    351MiB /  4034MiB |     N/A      Default |
  +-------------------------------+----------------------+----------------------+
  |   1  GeForce GTX 1070    Off  | 00000000:02:00.0 Off |                  N/A |
  |  0%   38C    P8     6W / 180W |      2MiB /  8119MiB |      0%      Default |
  +-------------------------------+----------------------+----------------------+
  

• glmark2 output:

  =======================================================
      glmark2 2012.08
  =======================================================
      OpenGL Information
      GL_VENDOR:     NVIDIA Corporation
      GL_RENDERER:   GeForce GTX 770/PCIe/SSE2
      GL_VERSION:    4.6.0 NVIDIA 390.87
  =======================================================
  
  

• glxinfo output:

  name of display: :0
  display: :0  screen: 0
  direct rendering: Yes
  server glx vendor string: NVIDIA Corporation
  server glx version string: 1.4
  client glx vendor string: NVIDIA Corporation
  client glx version string: 1.4
  GLX version: 1.4
  OpenGL vendor string: NVIDIA Corporation
  OpenGL renderer string: GeForce GTX 770/PCIe/SSE2
  OpenGL core profile version string: 4.3.0 NVIDIA 390.87
  OpenGL core profile shading language version string: 4.30 NVIDIA via Cg compiler
  OpenGL core profile context flags: (none)
  OpenGL core profile profile mask: core profile
  OpenGL version string: 4.6.0 NVIDIA 390.87
  OpenGL shading language version string: 4.60 NVIDIA
  OpenGL context flags: (none)
  OpenGL profile mask: (none)
  

• libglfw3 version: 3.2.1 (Compile from source using cmake)
• pip show cyglfw3 output:

  Name: cyglfw3
  Version: 3.1.0.2
  Summary: Python bindings for GLFW 3+ using Cython
  Home-page: http://github.com/adamlwgriffiths/cyglfw3
  Author: Adam Griffiths
  Author-email: UNKNOWN
  License: BSD
  Location: /usr/local/lib/python2.7/dist-packages
  Requires: Cython
  

• the cfg file:

  MODEL_PATH: /home/me/HardDisk/Datasets/6DB/public/datasets/hinterstoisser/models/obj_02.ply
  BACKGROUND_IMAGES_GLOB: /home/me/HardDisk/Datasets/VOC/VOCdevkit/VOC2007/JPEGImages/
  

  (the rest remains unchanged)

And actually it dose't seem like creating a window here is the key to the problem, 'cause I also create a python script with following lines:

import logging as log
import os

import os
import numpy as np
from OpenGL.GL import *

import gl_utils as gu
from pysixd import misc

import numpy as np
import time
import hashlib
import glob
import os
import progressbar
import cv2

from OpenGL.GL import *
import cyglfw3 as glfw
class OffscreenContext(object):
    def __init__(self):
        assert glfw.Init(), 'Glfw Init failed!'
        glfw.WindowHint(glfw.VISIBLE, False);
        self._offscreen_context = glfw.CreateWindow(1, 1, "", None)
        assert self._offscreen_context, 'Could not create Offscreen Context!'
        glfw.MakeContextCurrent(self._offscreen_context)
        print("All good!")

if __name__ == '__main__':
    context = OffscreenContext()

This file runs perfectly fine without any segmentation fault.

Any suggestion?

Cheers

It seems there is a bug in SceneRenderer.

BTW: what is the difference between meshrenderer_phong and meshrenderer?
And what is the difference between cad and reconst models?
Why do you use meshrenderer_phong for reconst models and meshrender for cad models?
In LINEMOD, is the model_type "reconst" or "cad"?

Are you thinking about sharing the code of the work done in 'Multi-path Learning for Object Pose Estimation Across Domains'?

Hi!

I trained the AAE with a cad leaf model, which I set VERTEX_SCALE: 80, AUXILIARY_MASK: True, BOOTSTRAP_RATIO: 10. The training image is showed below:

But when I test in images with aae_image.py, the results are not correct. For example, this is one of the images I used:

The predict image is

It is obviously that the predict result is not correct.
Could you give me some suggestion for this? Thanks a lot.

How do I train with more than one models? For there is only one model that can be placed in to the config file.

Hi. There is a problem when I generate and check training data.
As the picture shows, the blue part of my model becomes transparent.

Could you give any suggestions? Thanks in advance.
And my model is here.
model.tar.gz

I replaced the "obj_5.ply" model to my own ply model, which has the same format with the model in the models_reconst.However, it looks wrong when I run "ae_train exp_group/my_autoencoder -d"
Why?

Hi, I followed the instruction and I got this problem. I did pip install pyopengl ((py2.7_edc) edison@amax-server:~/edison/aae/AugmentedAutoencoder/AE_WORKSPACE_PATH$ pip install pyopengl
Requirement already satisfied: pyopengl in /home/edison/py2.7_edc/lib/python2.7/site-packages (3.1.0)
)

This is the problem, Can you please help me? Thank you.

(py2.7_edc) edison@amax-server:~/edison/aae/AugmentedAutoencoder/AE_WORKSPACE_PATH$ ae_train exp_group/my_autoencoder -d
128 128 3
[[8, 8], [16, 16], [32, 32], [64, 64]]
(?, 128, 128, 3)
(?, 128, 128, 3)
Traceback (most recent call last):20000 ] | | (ETA: --:--:--)
File "/home/edison/py2.7_edc/bin/ae_train", line 11, in
sys.exit(main())
File "/home/edison/py2.7_edc/local/lib/python2.7/site-packages/auto_pose/ae/ae_train.py", line 90, in main
dataset.get_training_images(dataset_path, args)
File "/home/edison/py2.7_edc/local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 93, in get_training_images
self.render_training_images()
File "/home/edison/py2.7_edc/local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 246, in render_training_images
bgr_x, depth_x = self.renderer.render(
File "/home/edison/py2.7_edc/local/lib/python2.7/site-packages/auto_pose/ae/utils.py", line 15, in decorator
setattr(self, attribute, function(self))
File "/home/edison/py2.7_edc/local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 75, in renderer
float(self._kw['vertex_scale'])
File "/home/edison/py2.7_edc/local/lib/python2.7/site-packages/auto_pose/meshrenderer/meshrenderer_phong.py", line 22, in init
self._fbo = gu.Framebuffer( { GL_COLOR_ATTACHMENT0: gu.Texture(GL_TEXTURE_2D, 1, GL_RGB8, W, H),
File "/home/edison/py2.7_edc/local/lib/python2.7/site-packages/auto_pose/meshrenderer/gl_utils/texture.py", line 11, in init
glCreateTextures(tex_type, len(self.__id), self.__id)
NameError: global name 'glCreateTextures' is not defined

Hi,
I am wondering if it will be an option to share with me with some trained model or complete package files that are necessary to run your's aae experiment?
If you have a Docker i will be very helpfull.

Hi!
I want to konw the meaning of train_x,mask_x,noof_obj_pixels,train_y,and hashlib.md5(...).hexdigest().
Looking forward to your explanation!

I'm trying to run ae_train exp_group/my_autoencoder -d with a .ply model of mine (which is not a CAD model) and I receive the following error:

Instructions for updating:
Use the retry module or similar alternatives.
128 128 3
[[8, 8], [16, 16], [32, 32], [64, 64]]
(?, 128, 128, 3)
(?, 128, 128, 3)
Traceback (most recent call last):20000 ] |                                                                       | (ETA:  --:--:--) 
  File "/home/username/.local/bin/ae_train", line 10, in <module>
    sys.exit(main())
  File "/home/username/.local/lib/python2.7/site-packages/auto_pose/ae/ae_train.py", line 90, in main
    dataset.get_training_images(dataset_path, args)
  File "/home/username/.local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 93, in get_training_images
    self.render_training_images()
  File "/home/username/.local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 246, in render_training_images
    bgr_x, depth_x = self.renderer.render( 
  File "/home/username/.local/lib/python2.7/site-packages/auto_pose/ae/utils.py", line 15, in decorator
    setattr(self, attribute, function(self))
  File "/home/username/.local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 75, in renderer
    float(self._kw['vertex_scale'])
  File "/home/username/.local/lib/python2.7/site-packages/auto_pose/meshrenderer/meshrenderer_phong.py", line 45, in __init__
    for vertex, normal, color, faces in attributes:
ValueError: need more than 3 values to unpack

Note that the suggestion reported in #7, did not solve my problem. And if I change it to cad in the config file, I get a Segmentation Fault.

In order to help debugging, I am also sharing my .ply model with you: bay.ply

My Blender can visualize this .ply, so I don't think the problem lies within there.

hello!

in issue 22, i raised a question about what to do after training the aae. According to your helpful reply, i followed the github of keras-retinanet and successfully trained my detect retinanet, which is a h5 document. Then i revised the aae_retina_webcam.cfg, it is as follows:

[MODEL]
gpu_memory_fraction = 0.9
[DATA]
color_format = bgr
color_data_type = np.float32
depth_data_type = np.float32
[AAE]
experiments = ['exp_group/my_autoencoder']
upright = False
topk = 1
[DETECTOR]
detector_model_path = /home/zelong/Desktop/keras-retinanet-master/snapshots/after.h5
backbone = resnet50
class_names = [0]
nms_threshold = 0.5
det_threshold = 0.8
max_detections = 3 #300
[CAMERA]
width = 960
height = 720
K_test = [810.4968405 ,0.,487.55096072, 0., 810.61326022 ,354.6674888 , 0., 0., 1.]
camPose = False
[ICP]
icp = False

then i run this code: python aae_retina_webcam_pose.py -test_config aae_retina_webcam.cfg -vis

and the result is:
Using TensorFlow backend.
2019-03-24 17:07:37.816778: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-03-24 17:07:37.909418: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:898] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-03-24 17:07:37.910052: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 0 with properties:
name: GeForce GTX 1050 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.62
pciBusID: 0000:01:00.0
totalMemory: 3.94GiB freeMemory: 3.48GiB
2019-03-24 17:07:37.910068: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0
2019-03-24 17:07:38.125015: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-03-24 17:07:38.125048: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929] 0
2019-03-24 17:07:38.125073: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0: N
2019-03-24 17:07:38.125287: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3634 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1050 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1)
/home/zelong/.conda/envs/aae/lib/python2.7/site-packages/keras/engine/saving.py:293: UserWarning: No training configuration found in save file: the model was not compiled. Compile it manually.
warnings.warn('No training configuration found in save file: '
/home/zelong/Desktop/path/to/autoencoder_ws/experiments/exp_group/my_autoencoder/my_autoencoder.cfg

Start video stream with shape: 640,480
N/A% (0 of 1) | | Elapsed Time: 0:00:00 ETA: --:--:--Segmentation fault (core dumped)

i have tried to find solutions, but failed, may i bother you to help me find what's wrong with this?

the aae and retinanet work properly alone.

Thank you very much!

If you have any questions, feel free to create an issue with the tag [question].
If you wish to suggest an enhancement or feature request, add the tag [feature request].
If you are submitting a bug report, please fill in the following details.

Describe the bug
A clear and concise description of what the bug is.

System Info
Describe the characteristic of your environment:

• GPU models and configuration
• Python version
• OpenGL version
• Versions of any other relevant libraries

Additional context
Add any other context about the problem here.

I want to use the camera to detect multiple object poses. I have trained 3 models and introduced them in aae_retina_webcam.cfg:
Experiments = ['exp_group/my_autoencoder', 'exp_group/my_autoencoder-1', 'exp_group/my_autoencoder-2'],
The retinanet network has also been trained and has been verified to correctly identify these models in the picture, as introduced in aae_retina_webcam.cfg as follows:
[DETECTOR]
Detector_model_path = /home/a/Downloads/keras-retinanet-master/newmodel.h5
Backbone = resnet50
Class_names = ['1','5','6']
However, in video stream detection aae_retina_webcam_pose.py, only one model can always be detected.
Is the model name trained in AAE consistent with the model name of the retinanet training? For example, the model name is ‘1.ply’ and in retinanet is '1’.

I just tried to install and run the scripts on a new pc but I am running into this error when running the ae_train exp_group/my_autoencoder -d:

File "src/errorchecker.pyx", line 53, in OpenGL_accelerate.errorchecker._ErrorChecker.glCheckError
OpenGL.error.GLError: GLError(
err = 1282,
description = 'invalid operation',
baseOperation = glNamedFramebufferTexture,
cArguments = (
array([1], dtype=uint32),
GL_COLOR_ATTACHMENT0,
1,
0,
)
)

I tried using pip to install pyopengl, and also tried it by using the branch directly, which fixed another opegl bug: https://github.com/mcfletch/PyOpenGL

I am using CUDA 10.1 and tensorflow 1.14, and the nvidia 390.87 driver on a Quadro P2000 GPU.

Hi, After training. I try to do the testing for duck in LINEMOD dataset. (edison@amax-server:~/edison/aae/AugmentedAutoencoder/auto_pose/test$ python aae_image.py exp_group/my_autoencoder -f /home/edison/edison/aae/AugmentedAutoencoder/LINEMOD/duck/JPEGImages
) However, I get this error. Could you please give me a help? Thank you.

Traceback (most recent call last):
File "aae_image.py", line 50, in
pred_view = dataset.render_rot( R,downSample = 1)
File "/home/edison/anaconda2/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 193, in render_rot
bgr_y, depth_y = self.renderer.render(
File "/home/edison/anaconda2/lib/python2.7/site-packages/auto_pose/ae/utils.py", line 15, in decorator
setattr(self, attribute, function(self))
File "/home/edison/anaconda2/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 62, in renderer
from auto_pose.meshrenderer import meshrenderer, meshrenderer_phong
File "/home/edison/anaconda2/lib/python2.7/site-packages/auto_pose/meshrenderer/meshrenderer.py", line 7, in
import gl_utils as gu
File "/home/edison/anaconda2/lib/python2.7/site-packages/auto_pose/meshrenderer/gl_utils/init.py", line 3, in
from .glfw_offscreen_context import OffscreenContext
File "/home/edison/anaconda2/lib/python2.7/site-packages/auto_pose/meshrenderer/gl_utils/glfw_offscreen_context.py", line 7, in
import cyglfw3 as glfw
File "/home/edison/anaconda2/lib/python2.7/site-packages/cyglfw3/init.py", line 7, in
from .glfw3 import *
ImportError: /home/edison/anaconda2/lib/python2.7/site-packages/cyglfw3/glfw3.so: undefined symbol: PyFPE_jbuf

After I type ae_train exp_group/my_autoencoder -d
Traceback (most recent call last): | 0 / 20000 ETA: --:--:--
File "/media/data_1/home/zelin/.local/bin/ae_train", line 11, in
sys.exit(main())
File "/media/data_1/home/zelin/.local/lib/python2.7/site-packages/auto_pose/ae/ae_train.py", line 90, in main
dataset.get_training_images(dataset_path, args)
File "/media/data_1/home/zelin/.local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 93, in get_training_images
self.render_training_images()
File "/media/data_1/home/zelin/.local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 245, in render_training_images
bgr_x, depth_x = self.renderer.render(
File "/media/data_1/home/zelin/.local/lib/python2.7/site-packages/auto_pose/ae/utils.py", line 15, in decorator
setattr(self, attribute, function(self))
File "/media/data_1/home/zelin/.local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 75, in renderer
float(self._kw['vertex_scale'])
File "/media/data_1/home/zelin/.local/lib/python2.7/site-packages/auto_pose/meshrenderer/meshrenderer_phong.py", line 18, in init
self._context = gu.OffscreenContext()
File "/media/data_1/home/zelin/.local/lib/python2.7/site-packages/auto_pose/meshrenderer/gl_utils/glfw_offscreen_context.py", line 12, in init
assert glfw.Init(), 'Glfw Init failed!'
AssertionError: Glfw Init failed!

Describe the bug

1. After the output picture success display when I use "ae_train exp_group/my_autoencoder -d", the program wait and throw me "Qt: Cannot set locale modifiers"
2. When I try the order "ae_train exp_group/my_autoencoder" with obj_05.ply model, I meet the "Attempted to use a closed Session" problem from "queue.py". The problem not happens when I use tensorflow CPU version, but happens in GPU version, from 1.7.0 to 1.12.0.
   I don't know if these two problems have relationship, so I submit them together. Very expecting your solutions!

Problem Info

Exception in thread Thread-3 (and other threads):
Traceback (most recent call last):
File "/home//anaconda3/envs//lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/home//anaconda3/envs//lib/python2.7/threading.py", line 754, in run
self.__target(self.__args, **self.__kwargs)
File "/home//.local/lib/python2.7/site-packages/auto_pose/ae/queue.py", line 66, in run
session.run(self.enqueue_op, feed_dict)
File "/home//anaconda3/envs//lib/python2.7/site-packages/tensorflow/python/client/session.py", line 900, in run
run_metadata_ptr)
File "/home//anaconda3/envs//lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1058, in _run
raise RuntimeError('Attempted to use a closed Session.')
RuntimeError: Attempted to use a closed Session.

Describe the characteristic of your environment:

• GPU : GTX1080
• Python2.7
• Latest pyOpenGL
• Ubuntu 16.04
• Try Tensorflow-gpu from 1.7.0 to 1.12.0 with "conda install" order

Thank you again for your wonderful work.

After trained with LINEMOD driller.ply, the reconstruction image is a pure black image.

I have tried to set BOOTSTRAP_RATIO = 10 to train, it still reconstruct dark image after 30,000 iterations of training.

It seems that it is because the original rendering image is dark, and it is even darker when some augmentation added to it.

Is there a way to render a brighter image? If possible, could you share your training config file for LINEMOD driller for us?

My training config file for LINEMOD driller.ply:

[Paths]
MODEL_PATH: /ACCV12/models/driller.ply
BACKGROUND_IMAGES_GLOB: /VOCdevkit/VOC2007/JPEGImages/*.jpg

[Dataset]
MODEL: reconst
H: 128
W: 128
C: 3
RADIUS: 700
RENDER_DIMS: (640, 480)
# RENDER_DIMS: (Width, Height)
K: [572.41140, 0.0, 325.26110, 0.0, 573.57043, 242.04899, 0.0, 0.0, 1.0]
# Scale vertices to mm
VERTEX_SCALE: 1
ANTIALIASING: 8
PAD_FACTOR: 1.2
CLIP_NEAR: 10
CLIP_FAR: 10000
NOOF_TRAINING_IMGS: 30000
NOOF_BG_IMGS: 10000

[Augmentation]
REALISTIC_OCCLUSION: False
SQUARE_OCCLUSION: False
MAX_REL_OFFSET: 0.20
CODE: Sequential([
	#Sometimes(0.5, PerspectiveTransform(0.05)),
	#Sometimes(0.5, CropAndPad(percent=(-0.05, 0.1))),
	Sometimes(0.5, Affine(scale=(1.0, 1.2))),
	Sometimes(0.5, CoarseDropout( p=0.2, size_percent=0.05) ),
	Sometimes(0.5, GaussianBlur(1.2*np.random.rand())),
    Sometimes(0.5, Add((0, 25), per_channel=0.3)),
    Sometimes(0.3, Invert(0.2, per_channel=True)),
    #Sometimes(0.5, Multiply((0.9, 1.2), per_channel=0.5)),
    Sometimes(0.5, Multiply((1, 1.2))),
    Sometimes(0.5, ContrastNormalization((0.5, 2.2), per_channel=0.3))
	], random_order=False)

[Embedding]
EMBED_BB: True
MIN_N_VIEWS: 2562
NUM_CYCLO: 36

[Network]
BATCH_NORMALIZATION: False
AUXILIARY_MASK: True
VARIATIONAL: 0
LOSS: L2
BOOTSTRAP_RATIO: 10
NORM_REGULARIZE: 0
LATENT_SPACE_SIZE: 128
NUM_FILTER: [128, 256, 512, 512]
STRIDES: [2, 2, 2, 2]
KERNEL_SIZE_ENCODER: 5
KERNEL_SIZE_DECODER: 5


[Training]
OPTIMIZER: Adam
NUM_ITER: 30000
BATCH_SIZE: 64
LEARNING_RATE: 2e-4
SAVE_INTERVAL: 5000

[Queue]
# OPENGL_RENDER_QUEUE_SIZE: 500
NUM_THREADS: 10
QUEUE_SIZE: 50

The preparation commands are somehow unclear.

Starting from:

export AE_WORKSPACE_PATH=/path/to/autoencoder_ws

Is this workspace supposed to be where we clone the repository? If not, we need to create another folder for this. Is this the case? Please clarify this.

Moving on:

The command of ae_init_workspace returns ae_init_workspace: command not found, which means due to the unclear instructions, something is not really installed right. Whatever the workspace is at, whether it's a separate folder or it's the cloned repository, this error is thrown anyway.

Last but not least:

(if installed locally, make sure .local/bin/ is in your PATH), if installed what exactly? A lot of installations through pip were done, are you referring to those? If not, what installation are you referring to? The setup.py of the repository?

A clearer set of instructions here would be really, really useful.

I think I have got right images.

when I test the model, the evaluation is very bad.
python aae_image.py exp_group/my_autoencoder -f /home/leviathan/PycharmProjects/test_img/size743/

The original cad model don't have colour vertex, but when I edit the 'rescont' to 'cad', I got Segmentation Fault (core dumped), just like #10 and #9 ,so I add the colour vertex as 255.the code can run right.

this is my ply file, no colour version and colour version
ply.zip

I don't know how to solve it ,please help me. tank you very much!

Hello, thank you for your wonderful work! Recently I tried to reproduce the results on T-LESS. More specifically, the "w/ GT 2D BBs" part of AAE without ICP. For training and evaling, I both used the same parameters in README.md. And I succeeded in reproducing the result of object 05(I got recall of vsd 66.1, which is similar to the result 66.7 reported in article). But when I used the same parameters for object 19 and 20, I only got 13.1 and 11.2 while the results in article are 50.75 and 37.75. I repeated the experiments twice, but no difference seemed to be seen. Can you tell some skills in adjusting the parameters? Or can you share your trained models or estimated pose files on T-LESS? Thank you for your assistance!

After I type ae_train exp_group/my_autoencoder -d
Traceback (most recent call last): | 0 / 20000 ETA: --:--:--
File "/media/data_1/home/zelin/.local/bin/ae_train", line 11, in
sys.exit(main())
File "/media/data_1/home/zelin/.local/lib/python2.7/site-packages/auto_pose/ae/ae_train.py", line 90, in main
dataset.get_training_images(dataset_path, args)
File "/media/data_1/home/zelin/.local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 93, in get_training_images
self.render_training_images()
File "/media/data_1/home/zelin/.local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 245, in render_training_images
bgr_x, depth_x = self.renderer.render(
File "/media/data_1/home/zelin/.local/lib/python2.7/site-packages/auto_pose/ae/utils.py", line 15, in decorator
setattr(self, attribute, function(self))
File "/media/data_1/home/zelin/.local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 75, in renderer
float(self._kw['vertex_scale'])
File "/media/data_1/home/zelin/.local/lib/python2.7/site-packages/auto_pose/meshrenderer/meshrenderer_phong.py", line 18, in init
self._context = gu.OffscreenContext()
File "/media/data_1/home/zelin/.local/lib/python2.7/site-packages/auto_pose/meshrenderer/gl_utils/glfw_offscreen_context.py", line 12, in init
assert glfw.Init(), 'Glfw Init failed!'
AssertionError: Glfw Init failed!

Describe the bug
When increasing H and W in the autoencoder config file (my_autoencoder.cfg) to for example 256, and I try to train, it goes through rendering and loading background images, but when it comes to training it says:
Killed
and exits

I've been able to expand the resolution to 220 without this error, but above this it will not train.

System Info
Describe the characteristic of your environment:

• GPU models and configuration: GTX1080
• Python version: 2.7
• OpenGL version: downloaded from here https://github.com/mcfletch/PyOpenGL

Testing as "Augmented Autoencoder only" means there is no SSD detector used and won't detect the object in the image or video ? and only calculate the 3D Orientation ?
And I have to add my own detector model to the code (aae_webcam.py) ?

Thank you for your brilliant masterpiece and making it open-sourced.
I have successfully trained AAE on TejaniDB object "milk", when I run ae_eval, the code warns me not enough visible points, then the image shown does not render the estimation object. I checked the code it is in auto_pose/eval/icp_utils.py, line 265.

    if len(real_depth_pts) < len(synthetic_pts)/8.:
        print 'not enough visible points'

It seems that the number of real depth points is not enough. I have changed the if statement to len(real_depth_pts) < len(synthetic_pts)/800., but the warning still exists.
What should I do?
Thank you in advance!

I have searched among the closed issues and see one reply from the author saying the Offscreen rendering is not supported and a display is needed. That was several months ago, so now has there been any change? Could it be possible for me to run this project on a machine without display??

Thank you very much!

Hi, I' following the documentation and encountered problems:
In section "Evaluate and visualize 6D pose estimation of AAE with ground truth bounding boxes" after I run the following command:

ae_eval exp_group/my_autoencoder evaluation --eval_cfg eval_group/eval_my_autoencoder.cfg

(I didn't modify the configuration file template, since estimate_bbs is already set to False)
I get following errors :

Processing: /localhome/demo/autoencoder_6d_pose_estimation/AugmentedAutoencoder_ws/cfg_eval/eval_results/hodan-iros15_tless_primesense
test_primesense
Loading object models...

Done.
{1: [5, 7, 9, 17, 18, 20], 2: [1, 12, 20, 9], 3: [9, 12, 20, 7], 4: [9, 18, 20, 5, 17], 5: [11, 2, 3, 4], 6: [2, 6], 7: [17, 2, 12, 18, 6], 8: [11, 3, 4], 9: [17, 18, 11, 12, 5], 10: [16, 11, 5], 11: [16, 3, 6], 12: [16, 3, 6], 13: [16, 19, 7], 14: [16, 19, 7], 15: [16, 19, 7], 16: [16, 19, 7], 17: [16, 19, 7], 18: [19, 3, 7], 19: [8, 10, 13, 14], 20: [8, 10, 13, 14], 21: [8, 10, 13], 22: [8, 10, 14], 23: [8, 10, 13, 14], 24: [8, 10, 19, 14], 25: [1, 15], 26: [4, 15], 27: [5, 15], 28: [4, 13, 15], 29: [1, 15], 30: [1, 19, 15]}
test_primesense
<backports.configparser.ConfigParser object at 0x7f631558cf50>
128 128 3
[[8, 8], [16, 16], [32, 32], [64, 64]]
(?, 128, 128, 3)
(?, 128, 128, 3)
2019-02-25 17:57:11.296768: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-02-25 17:57:11.373947: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:964] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-02-25 17:57:11.374378: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties: 
name: GeForce GTX 1050 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.506
pciBusID: 0000:01:00.0
totalMemory: 3.94GiB freeMemory: 2.89GiB
2019-02-25 17:57:11.374393: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2019-02-25 17:57:11.542753: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-02-25 17:57:11.542794: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988]      0 
2019-02-25 17:57:11.542799: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0:   N 
2019-02-25 17:57:11.542967: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2019 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1050 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1)
11
scene_id: 11
eval_args: <backports.configparser.ConfigParser object at 0x7f631558cf50>
test_primesense
dataset_name: tless
cam_type: primesense
test_primesense

Traceback (most recent call last):
  File "/localhome/demo/autoencoder_6d_pose_estimation/venv/bin/ae_eval", line 11, in <module>
    load_entry_point('auto-pose==0.9', 'console_scripts', 'ae_eval')()
  File "/localhome/demo/autoencoder_6d_pose_estimation/venv/local/lib/python2.7/site-packages/auto_pose/eval/ae_eval.py", line 114, in main
    test_imgs = eval_utils.load_scenes(scene_id, eval_args)
  File "/localhome/demo/autoencoder_6d_pose_estimation/venv/local/lib/python2.7/site-packages/auto_pose/eval/eval_utils.py", line 152, in load_scenes
    noof_imgs = noof_scene_views(scene_id, eval_args)
  File "/localhome/demo/autoencoder_6d_pose_estimation/venv/local/lib/python2.7/site-packages/auto_pose/eval/eval_utils.py", line 138, in noof_scene_views
    noof_imgs = len(os.listdir(os.path.join(p['base_path'], p['test_dir'], '{:02d}', 'rgb').format(scene_id)))
KeyError: 'test_dir'

I set up my sixd_toolkit as follows:

pip install -r requirements.txt
in dataset_params.py:
common_base_path = '/localhome/demo/autoencoder_6d_pose_estimation/t-less/t-less_v2/t-less_v2/'
tless_tk_path = '/localhome/demo/autoencoder_6d_pose_estimation/t-less_toolkit/'

Thanks a lot if someone can help me out..

System Info

• GPU 1050 4G
• Python version:2.7.12
• PyOpenGL version: 3.1.3b2

by the way it seems that there's a little mistake in
Create the evaluation config file:

should be:  mkdir $AE_WORKSPACE_PATH/cfg_eval/eval_group,  instead of ../eval_cfg/..
cp $AE_WORKSPACE_PATH/cfg_eval/eval_template.cfg $AE_WORKSPACE_PATH/eval_cfg/eval_group/eval_my_autoencoder.cfg
gedit $AE_WORKSPACE_PATH/cfg_eval/eval_group/eval_my_autoencoder.cfg ---- instead of ../cfg/..

Describe the bug

Hi, I try to use depth image to refine the result, device is Kinect v2 and I use pylibfreenect2 to get RGB-D images in python.

Here is the problem:

**Traceback (most recent call last):
File "/home/a/.vscode/extensions/ms-python.python-2019.10.41019/pythonFiles/ptvsd_launcher.py", line 43, in
main(ptvsdArgs)
File "/home/a/.vscode/extensions/ms-python.python-2019.10.41019/pythonFiles/lib/python/old_ptvsd/ptvsd/main.py", line 432, in main
run()
File "/home/a/.vscode/extensions/ms-python.python-2019.10.41019/pythonFiles/lib/python/old_ptvsd/ptvsd/main.py", line 316, in run_file
runpy.run_path(target, run_name='main')
File "/home/a/anaconda3/envs/py27/lib/python2.7/runpy.py", line 252, in run_path
return _run_module_code(code, init_globals, run_name, path_name)
File "/home/a/anaconda3/envs/py27/lib/python2.7/runpy.py", line 82, in _run_module_code
mod_name, mod_fname, mod_loader, pkg_name)
File "/home/a/anaconda3/envs/py27/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/a/Downloads/AugmentedAutoencoder-master/auto_pose/test/aae_retina_webcam_pose_kinect.py", line 166, in
all_pose_estimates, all_class_idcs = ae_pose_est.process_pose(boxes, labels, image0, depth1)
File "/home/a/Downloads/AugmentedAutoencoder-master/auto_pose/test/aae_retina_pose_estimator.py", line 212, in process_pose
R_est, t_est = self.icp_handle.icp_refinement(depth_crop, R_est, t_est, self._camK, (W,H), clas_idx=clas_idx, depth_only=True)
File "/home/a/.local/lib/python2.7/site-packages/auto_pose/icp/icp.py", line 177, in icp_refinement
print 'real min max x', np.min(real_depth_pts[:,0]), np.max(real_depth_pts[:,0])
File "/home/a/anaconda3/envs/py27/lib/python2.7/site-packages/numpy/core/fromnumeric.py", line 2618, in amin
initial=initial)
File "/home/a/anaconda3/envs/py27/lib/python2.7/site-packages/numpy/core/fromnumeric.py", line 86, in _wrapreduction
return ufunc.reduce(obj, axis, dtype, out, passkwargs)
ValueError: zero-size array to reduction operation minimum which has no identity

the depth_image.shape is [542,414], and I save the depth image data like below:
……………………
0.000000000000000000e+00 1.922912353515625000e+03 0.000000000000000000e+00 1.884088623046875000e+03 0.000000000000000000e+00 1.822018188476562500e+03 1.898113159179687500e+03 1.828330566406250000e+03 0.000000000000000000e+00 1.919650512695312500e+03 1.870128906250000000e+03 0.000000000000000000e+00
……………………
I think the depth data is in mm.
And the RGB image can be used normally

I'm quiet confused,could you please give me some advice?
Thanks a lot!!!

I have installed the pyopengl as issue#1 mentioned. However, I get this problem and I dont know why. Could you please help me? Thank you.

(py2.7_edc) edison@amax-server:~/edison/aae/AugmentedAutoencoder/AE_WORKSPACE_PATH$ ae_train exp_group/my_autoencoder -d
128 128 3
[[8, 8], [16, 16], [32, 32], [64, 64]]
(?, 128, 128, 3)
(?, 128, 128, 3)
No handlers could be found for logger "OpenGL.acceleratesupport" | (ETA: --:--:--)
Traceback (most recent call last):
File "/home/edison/py2.7_edc/bin/ae_train", line 11, in
sys.exit(main())
File "/home/edison/py2.7_edc/local/lib/python2.7/site-packages/auto_pose/ae/ae_train.py", line 90, in main
dataset.get_training_images(dataset_path, args)
File "/home/edison/py2.7_edc/local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 93, in get_training_images
self.render_training_images()
File "/home/edison/py2.7_edc/local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 246, in render_training_images
bgr_x, depth_x = self.renderer.render(
File "/home/edison/py2.7_edc/local/lib/python2.7/site-packages/auto_pose/ae/utils.py", line 15, in decorator
setattr(self, attribute, function(self))
File "/home/edison/py2.7_edc/local/lib/python2.7/site-packages/auto_pose/ae/dataset.py", line 75, in renderer
float(self._kw['vertex_scale'])
File "/home/edison/py2.7_edc/local/lib/python2.7/site-packages/auto_pose/meshrenderer/meshrenderer_phong.py", line 24, in init
GL_DEPTH_ATTACHMENT: gu.Renderbuffer(GL_DEPTH_COMPONENT32F, W, H) } )
File "/home/edison/py2.7_edc/local/lib/python2.7/site-packages/auto_pose/meshrenderer/gl_utils/fbo.py", line 24, in init
raise RuntimeError('Framebuffer not complete.')
RuntimeError: Framebuffer not complete.

I've been trying to train a model with a custom .ply. Visually the model is correctly being used to generate training images. However, the auto-encoder generates only black images. The training configuration is the used in your results? Any hyper parameter worth changing?

EDIT:
Actually, setting BatchNorm to True changes the behavior of the model. I'll update with new results

[question]
hello!
your work is quite wonderful!
I would like to use a camera to take a picture of an object(just one kind of object),then get its 6d information through your nice work. Following the readme, i have managed testing Augmented Autoencoder only, code is:python aae_image.py. input a picture,output a SO3 rotation matrix, work properly. After that i feel confused about what to do next to get the total 6d information, should i choose one of these two methods?
Option 1: Train a RetinaNet Model from
Option 2: Using the Google Detection API with Fixes.
I am new to this area and have few skill, could you please tell me which one is easy to realize? in my case only one kind object need to be detected so i think the detect network is no necessary to be complicated.
Another question is during the training of detect network, can i use generate_sixd_train.py this code to generate training data instead of marking labels by hand?
In short,
1:which option is easy
2:whether use generate_sixd_train.py
As you see, my English is quite poor, but I sincerely appreciate your help and kindness!

Thanks a lot for the nice work, but there's some information missing related to generating training data for ssd, hope followings will save time for others who are also interested in this work:

for generating training data for ssd, instead using "detection_utils/generate_sixd_train.py", "generate_syn_det_train.py" should be used. And the commanline configuration looks as follows:

python generate_syn_det_train.py \
     --output_path=/path/to/training_data_ssd  \  <==output dir given by user
     --model=/path/to/model_dir/t-less/t-less_v2/models_cad/  \ <==model data dir
     --num=3000 \ <==how many data u want to generate
      --scale=1 \ <==scale factors, In case of a model in meters, use 1000
      --vocpath=/localhome/demo/autoencoder_6d_pose_estimation/backgrounimage/VOCdevkit/VOC2012/JPEGImages  \ <==background pictures location
      --model_type=cad  <==when with textures use 'reconst'

after running, xml annotations and training images will be generated nicely.

run 'export PYOPENGL_PLATFORM='egl'', and then run "ae_train exp_group/my_autoencoder -d"
but it occured to have this error:
OpenGL.error.GLError: GLError(
err = 12297,
baseOperation = eglCreateContext,
cArguments = (
<OpenGL._opaque.EGLDisplay_pointer object at 0x7fec2fd053b0>,
<OpenGL._opaque.EGLConfig_pointer object at 0x7fec2fd05320>,
<OpenGL._opaque.EGLContext_pointer object at 0x7fec2fdc5560>,
<OpenGL.arrays.lists.c_int_Array_7 object at 0x7fec2fd054d0>,
),
result = <OpenGL._opaque.EGLContext_pointer object at 0x7fec2fd057a0>
)

Describe the bug

1. I can't use "ae_train exp_group/my_autoencoder -d" becauese " command not found"
   so I use "python ae_train exp_group/my_autoencoder -d"
   2.When I use "python ae_train exp_group/my_autoencoder -d", after loading pictures, the program quit in "reconstr_train = sess.run(decoder.x, feed_dict={queue.x:this_x})" (line 140 in ae_train.py)
   the bug info is "Segmentation fault (core dumped)"
   3.And when I check the shape of this_x and queue.x, the output is "('queue.x', TensorShape([Dimension(None), Dimension(128), Dimension(128), Dimension(3)]))
   ('this_x', (128, 128, 3))"
   4.I have also changed the batch size and the number of pictures, it seems not to be the memory problem of GPU

System Info

• GPU GeForce GTX 1080
• python2.7
• tensorflow 1.9.0
• Ubuntu 16.04

I keep getting "Segmentation Fault" whatever the model I feed in. I tried 5 so far, what's going on I have no idea, but they weren't scaled. I think this can be registered as a bug.

Instructions for updating:
Use the retry module or similar alternatives.
128 128 3
[[8, 8], [16, 16], [32, 32], [64, 64]]
(?, 128, 128, 3)
(?, 128, 128, 3)
Segmentation fault (core dumped)  

Here are some models I tried: model3, model4, model5

Here is my hw setup:

Kernel: 4.15.0-45-generic x86_64 bits: 64 gcc: 7.3.0
Desktop: Gnome 3.28.3 (Gtk 2.24.32) 
Distro: Ubuntu 18.04.1 LTS
Machine:   Device: desktop 
Mobo: ASUSTeK 
model: P9X79 PRO v: Rev 1.xx serial: N/A
BIOS: American Megatrends v: 2002 date: 06/18/2012
CPU:       6 core Intel Core i7-3930K (-MT-MCP-) arch: Sandy Bridge rev.7 cache: 12288 KB
           flags: (lm nx sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx) bmips: 38425
           clock speeds: max: 3800 MHz 1: 1406 MHz 2: 1313 MHz 3: 1401 MHz 4: 1240 MHz 5: 1232 MHz
           6: 1405 MHz 7: 1327 MHz 8: 1524 MHz 9: 1503 MHz 10: 1377 MHz 11: 1935 MHz 12: 1285 MHz
Graphics:  Card: NVIDIA GP106 [GeForce GTX 1060 6GB] bus-ID: 01:00.0
           Display Server: x11 (X.Org 1.19.6 )
           drivers: nvidia (unloaded: modesetting,fbdev,vesa,nouveau)
           Resolution: [email protected], [email protected]
           OpenGL: renderer: GeForce GTX 1060 6GB/PCIe/SSE2
           version: 4.6.0 NVIDIA 390.77 Direct Render: Yes
Audio:     Card-1 NVIDIA GP106 High Def. Audio Controller driver: snd_hda_intel bus-ID: 01:00.1
           Card-2 Intel C600/X79 series High Def. Audio Controller
           driver: snd_hda_intel bus-ID: 00:1b.0
           Card-3 Logitech Headset H340 driver: USB Audio usb-ID: 002-005
           Card-4 Logitech Webcam C270 driver: USB Audio usb-ID: 002-006
           Sound: Advanced Linux Sound Architecture v: k4.15.0-45-generic
Network:   Card: Intel 82579V Gigabit Network Connection
           driver: e1000e v: 3.2.6-k port: f040 bus-ID: 00:19.0
           IF: eno1 state: up speed: 1000 Mbps duplex: full mac: 30:85:a9:98:15:7f
Drives:    HDD Total Size: 250.1GB (47.3% used)
           ID-1: /dev/sda model: Samsung_SSD_850 size: 250.1GB temp: 0C
Partition: ID-1: / size: 229G used: 111G (51%) fs: ext4 dev: /dev/sda1
RAID:      No RAID devices: /proc/mdstat, md_mod kernel module present
Sensors:   System Temperatures: cpu: 30.0C mobo: N/A gpu: 1.0:35C
           Fan Speeds (in rpm): cpu: 0
Info:      Processes: 418 Uptime: 2:59 Memory: 4648.7/15982.4MB
           Init: systemd runlevel: 5 Gcc sys: 6.5.0 Client: Shell (bash 4.4.191) inxi: 2.3.56 

when i train an autoencoder with linemod datasets,i change the cfg like this:

[Dataset]
MODEL: reconst
H: 128
W: 128
C: 3
RADIUS: 500
RENDER_DIMS: (650, 484)
K: [572.4114, 0, 325.2611, 0, 573.5704, 242.0489, 0, 0, 1]

Scale vertices to mm

VERTEX_SCALE: 10
ANTIALIASING: 8
PAD_FACTOR: 1.2
CLIP_NEAR: 10
CLIP_FAR: 10000
NOOF_TRAINING_IMGS: 20000
NOOF_BG_IMGS: 15000

The vertex scale of linemod is cm, so i change the vertex_scale, the inner cammera matrix of linemod is K: [572.4114, 0, 325.2611, 0, 573.5704, 242.0489, 0, 0, 1], and so on as the render_dims.
However, i still encounter the problem that "object in rendering not visible,have you scaled the vertices to mm?".
Are there any mistakes with the parameters i changed? Any parameters else i need to change?
Thanks for your reply!