Coder Social home page Coder Social logo

tum-vision / mono_dataset_code Goto Github PK

View Code? Open in Web Editor NEW
276.0 276.0 119.0 2.65 MB

Code for Monocular Visual Odometry Dataset - https://vision.cs.tum.edu/data/datasets/mono-dataset

License: BSD 3-Clause "New" or "Revised" License

CMake 6.93% MATLAB 10.10% C++ 82.97%

mono_dataset_code's People

Contributors

jakobengel avatar vladyslavusenko avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mono_dataset_code's Issues

pcalib.txt and vignette.png generation on TUM VO dataset

Hello,

I am trying to build pcalib.txt and vignette.png on one of the sequences of TUM dataset. Using ./responseCalib I am able to generate pcalib.txt on one of the sequence but it isn't the same as the original pcalib.txt. I am not able to understand why is it different. Not only I get non-increasing values also all the values lie in 100...255 range.

Also I am not able to generate vignette.png using ./vignetteCalib on the same sequence using the original pcalib.txt since the pcalib generated using ./responseCalib is not correct.
(PS: I have aruco already installed and tried creating marker board example after installation. That works fine.)

I also encounter an error when I run ./playDataset on TUM dataset sequence. The error looks like

Screenshot from 2021-10-07 12-42-33

Has anyone encountered some similar problem or have some solution to this?

Thank you!

Unable to run playDataset with mono_dataset sequences

Hello
I successfully compiled " mono dataset " but when i am trying to run it with the command
./playDataset X/sequence_01 but getting the error that :-

bash: ./playDataset: No such file or directory

Can anyone tell me where this file is ,i am unable to find it.
Thanks .

playDataset can't read calibration file

There seems to be a bug with the camera calibration reader in which no float is correctly read. Here's my output, using the sequence 45

Playback dataset /home/alejandro/Downloads/sequence_45/!
Load Dataset /home/alejandro/Downloads/sequence_45/: found 3000 files in folder /images; assuming that all images are there.
0.535719308086809       0.669566858850269       0.493248545285398       0.500408664348414       0.897966326944875
inputCalibration before
0
0
0
0
0
inputCalibration after
0
0
0
0
0
Failed to read camera calibration (invalid format?)
Calibration file: /home/alejandro/Downloads/sequence_45/camera.txt
Reading Photometric Calibration from file /home/alejandro/Downloads/sequence_45/pcalib.txt
Reading Vignette Image from /home/alejandro/Downloads/sequence_45/vignette.png
PhotometricUndistorter: Invalid vignette image size! got 1280 x 1024, expected 0 x 0. Set vignette to 1.
Dataset /home/alejandro/Downloads/sequence_45/: Got 3000 files!
Rectified Images: -1 x 0. K:
2.10578e-08 4.56277e-41 1.41933e-37
4.56277e-41 1.41933e-37           0
2.10578e-08           0           0

Original Images: 0 x 0. omega=0,000000 K:
0 0 0
0 0 0
0 0 0

ERROR: expected cv-mat to have dimensions 0 x 0; found 1280 x 1024 (image /home/alejandro/Downloads/sequence_45/images/00000.jpg)!
Segmentation fault (core dumped)

I added those lines for debugging purposes. Seems like
std::sscanf(l1.c_str(),"%f %f %f %f %f", &inputCalibration[0], &inputCalibration[1], &inputCalibration[2], &inputCalibration[3], &inputCalibration[4]);
at FOVundistorter.cpp isn't parsing the calibration file correctly. I've tried with other sequences to no avail.

Do not know how undistort an image

I am sorry I did not get it how to undistort the images using this code. I did not find the distortion coefficients either. So, where can I find it? Like k1, k2, k3, p1 and p2?

false result of responseCalib

pcalib.txt
g-10
Hi!
I used your program “responseCalib” to test my data and get the response function as shown in the figure:
I have a question, why is the beginning of the first few data is -nan?
and the picture is missing a part of the left.
my data is:
Exposure Time increased from 0.1ms to 20ms. And each collection of 10 pictures, exposure time increased by 0.1ms. so I save 2000 pictures with Exposure Time.
( My camera can only get 0.1ms accuracy )
Thank you very much!

Uninitialized values in undistorter

Vignette calibration crashed with this error:

OpenCV Error: Assertion failed (s >= 0) in setSize, file /tmp/binarydeb/ros-kinetic-opencv3-3.2.0/modules/core/src/matrix.cpp, line 307 terminate called after throwing an instance of 'cv::Exception' what(): /tmp/binarydeb/ros-kinetic-opencv3-3.2.0/modules/core/src/matrix.cpp:307: error: (-215) s >= 0 in function setSize

I was using following camera calibration:

395.006308 404.392711 314.697857 174.704342 0.0
856 480
none
856 480

The problem is in FOVUndistorter.cpp on the line 99. Undistorter returns and never reads output width and height which later results in error.

I suggest simply initializing out_width and out_height with values of in_width and in_height respectively.
Patch is attached.
patch.txt

Vignette calibration problem

I am running vignetteCalib on my own dataset - just 30 images to test the approach.
The program produces only black vignette and following output:

Load Dataset samples_vignette3/: found 30 files in folder /images; assuming that all images are there.
Input resolution: 856 480
Input Calibration (fx fy cx cy): 338.125397 194.108505 269.381378 83.858086 1.000000
NO RECTIFICATION
Reading Photometric Calibration from file samples_vignette3/pcalib.txt
Reading Vignette Image from samples_vignette3/vignette.png
PhotometricUndistorter: Invalid vignette image size! got 0 x 0, expected 856 x 480. Set vignette to 1.
Dataset samples_vignette3/: Got 30 files!
SEQUENCE NAME: samples_vignette3/!
plane image values 10000000000.000000 - -10000000000.000000!
0.000000 residual terms => -nan
0.000000 residual terms => -nan
plane image values 10000000000.000000 - -10000000000.000000!
0.000000 residual terms => -nan
0.000000 residual terms => -nan
...

the output is in the same fashion until the end.

Any idea what might be wrong with my dataset?
Is it possible that it does not work because it is not composed from video capture? (I can only get exposure times for the third column of times.txt when shooting images one by one with the camera)

Good RMSE values

I was wondering what are typical RMSE values on a good calibration.

Camera response calibration

Hi, we have some problems with reproducing the experiment. For the inverse response calibration, it doesn't appear to be monotonic, so we think that some errors exist in our data. Therefore, we want to ask some help for capturing test data:
1)There are some image algorithms for improving the image quality in our camera,like AWB, Positional Gain Adjustment, Adaptive Local Tone Mapping ,Color Correcton and Aperture Correction. Except the gamma correction , whether the above mentioned algorithoms should be disabled?Or have you some suggestions about the camera parameter set-up, which one should be enable or which one should be disable?
2) For recording the images, range of exposure in our camera is 0.19ms up to 31ms, but we can't get the overexposed images, if we haven't the overexposed images, whether it's a factor influencing the response funtion generation?
3)Our camera is rolling shutter, does it matter?
Please help.Thanks

Unable to run playDataset with mono_dataset sequences

Hello.
I compiled the mono_dataset_code successfully. Now I want to run ./playDataset for the sequences available in mono_dataset. ( https://vision.cs.tum.edu/data/datasets/mono-dataset )
I get some different errors while trying to run it on some of the sequences.
For the first sequence, I run:
./playDataset /home/dso/sequence_01/

It shows this message:

Playback dataset /home/dso/sequence_01/!
Load Dataset /home/dso/sequence_01/: found 4757 files in folder /images; assuming that all images are there.
Failed to read camera calibration (invalid format?)
Calibration file: /home/dso/sequence_01/camera.txt
Reading Photometric Calibration from file /home/dso/sequence_01/pcalib.txt
Reading Vignette Image from /home/dso/sequence_01/vignette.png
PhotometricUndistorter: Invalid vignette image size! got 1280 x 1024, expected 9 x 0. Set vignette to 1.
Dataset /home/dso/sequence_01/: Got 4757 files!
Rectified Images: 21199456 x 0. K:
-8.12748e-30 0 3.53393e-38
4.58953e-41 3.57133e-38 0
3.59111e-38 0 -nan
_
Original Images: 9 x 0. omega=0,000000 K:
0 8.96831e-44 0
1.06809e-05 0 3.5766e-38
2.89901e+29 6.72623e-44 0

ERROR: expected cv-mat to have dimensions 9 x 0; found 1280 x 1024 (image /home/dso/sequence_01/images/00000.jpg)!
Segmentation fault (core dumped)

Does anyone know what the problem is?
Thanks a lot.

question about the evaluation result of dso and orbslam

hello
I've read your paper "A Photometrically Calibrated Benchmark For Monocular Visual Odometry".
In section 3.2, figure 8 you compared the result of dso and orbslam by 500 runs.
The figure is upload below:
xp hy 3d w94tka ym ii d
However i have a question here: from the figure we can see that as the number of runs increases, the error increases correspondingly, but when the number of run increases from approximately 480 to 500, the error increases so fast. I think it's not so feasible or may i just can't explain this phenomenon.
In the paper there are a lot of figures like this having the same result. Could you please explain this phenomenon? Thank you so much!
best
chenchr

camera ISP affected the manual calibration

We are doing the manual photometric calibration on our embedded system, so far we still have two problems:

  1. Change camera exposure time costs 0.5 hour to execute
  2. The ISP for camera will automatically adjust the exposure time if it is over exposure, and also ISP will adjust other parameters to affect Vignette calibration.

How shall we handle these problems? Please help to advise, thanks

Checkerboard size

Where is it possible to get the checkerboard sizes for sequences calib_narrow_checkerboard1/2 and wide_checkerboard1/2 ?
Nothing on the webpage and the paper doesn't seem to contain any information.

Inv. response function U is not monotonic

Hello there, thanks for open-source.
I have a question that the inverse response function U is not monotonic based on my dataset (more than 1000 images, static scene and different exposure time from 0.05ms to 20ms). You said in this case, "it" should be smoothed or perturbed on your paper, and what is "it"?
I have modified some code like "G[255]" to "Gmax"(which is the max value in the G[]), is there some other codes I need to modified?
I'm sure that there are many different objects in the static scene, and the exposure time are incremental.

Best wishes.

responseCalib error : buffer still too small. abort.

I tried to make pcalib.txt for my custom data. I prepared images.zip, camera.txt, times.txt.

Data:
images.zip

camera.txt
RadTan 0.39296 0.70102 0.49574 0.51231 0.00018 -0.01646 0.00351 0.000281
1280 720
crop
1280 720

times.txt
00001 1659702911.5288742000 16.7
00002 1659702911.5455742000 16.7
00003 1659702911.5622742000 16.7
00004 1659702911.5789742000 16.7
00005 1659702911.5956743000 16.7
00006 1659702911.6123743000 16.7
00007 1659702911.6290743000 16.7
00008 1659702911.6457744000 16.7
00009 1659702911.6624744000 16.7
00010 1659702911.6791744000 16.7
00011 1659702911.6958745000 16.7
00012 1659702911.7125745000 16.7
00013 1659702911.7292745000 16.7
00014 1659702911.7459745000 16.7
....

input:
~/mono_dataset_code/bin$ ./responseCalib ../dataset_0805/

output:
Load Dataset ../dataset_0805/: found no in folder /images; assuming that images are zipped.
got 587 entries and 587 files from zipfile!
Failed to read camera calibration (invalid format?)
Calibration file: ../dataset_0805/camera.txt
Reading Photometric Calibration from file ../dataset_0805/pcalib.txt
PhotometricUndistorter: Could not open file!
Dataset ../dataset_0805/: Got 587 files!
read 10000/10000 bytes for file 00001.png. increase buffer!!
buffer still to small (read 100000/100000). abort.

I don't know how to solve this problem.

create tum dataset for my video

I want to convert my own video data into a folder structure like TUM benchmarks. So you want to create times.txt, camera.txt, statistic.txt and times.txt. how do I do this automatically? or what do the columns in statistic.txt and times.txt mean?

fx fy cx cy k1 k2 p1 p2 inputWidth inputHeight "crop" / "full" / "none" / "e1 e2 e3 e4 0" outputWidth outputHeight
0.349153000000000 0.436593000000000 0.493140000000000 0.499021000000000 0.933271000000000
???

Camera response calibration

Hi,
I've a problem with calculating inverse camera response function on my own data set.
the ICRF doesn't appear to be monotonic (or normal) at all, knowing that I've tested the code on different data sets that I've created on my own.
I've attached a sample of (G) that appears to me in the following link and times.txt file.
range of exposure time in my camera is 0.01 ms up to 100 ms.
CRF : https://ibb.co/eSj2A6
times.txt: https://file.io/XZMlAm

response function calibration with own dataset

Hello, @NikolausDemmel i want to do response function calibration with my webcam which is auto exposure and a Mono industrial camera with manual exposure setting. How can i generate dataset with different exposure time as suggested in algorithm.
Thanks in advance.

Calibrate response receives infinite results

Hi

I'm trying to calibrate the response of my camera. When I run the program, I found the results as follow,
Screenshot from 2020-06-28 20-50-45

It shows the irradiance and inverse response goes to infinite.
My guess is it related to the mean irradiance of all images, which range between [2.5, 182.6]. However, the exposure I set is between [0.06ms 200ms], with 167 images and exposure in multiplicative increments of 1.05. It is definitely overexposed at large expo time, and visually black at low expo time. I'm not sure what's wrong with this image data.

Meanwhile, the following points are especially noticed and ensured,

  • a constant scene with not moving objects and change light.
  • the compiled program worked correctly on the "narrow_sweep3" dataset.

Please let me if you have any advice.

Thanks

How to calibrate camera?

I use example from OpenCV for camera calibration. After calibrate matrix check it on learned images. All looks fine. Also get 5 values for distortion, but not use anywhere.

After this try to adapt to yours format and check on playDataset example.

double fx = cameraMatrix.at<double>(0, 0) / (double)imageSize.width;
double fy = cameraMatrix.at<double>(1, 1) / (double)imageSize.height;
double cx = (cameraMatrix.at<double>(0, 2) + 0.5) / (double)imageSize.width;
double cy = (cameraMatrix.at<double>(1, 2) + 0.5) / (double)imageSize.height;

Camera matrix looks like:

0.820361393435219	0.461474391295798	0.495088499159556	0.529959721497076	0
1280 720
crop
1280 720

After undistortion nothing changed. If I set omega to some value, undistortion apply but with strong skew of image. So how you calculate values for camera.txt file?

What is the unit of fx an fy?

In my case:
Camera.fx: 1138.042594
Camera.fy: 1152.150891
Camera.cx: 314.573090
Camera.cy: 273.959896
unit is pixel.

What about the TUM mono dataset?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.