Disclaimer:
- These days, I found this code to be too slow for the turnover rate in deep learning development. I personally switched to using numba as done in https://github.com/traveller59/second.pytorch to compute AP in much faster times (~10 seconds per full run on validation set). I recommend you take a look if you need faster code with python integration.
Updates:
- 17/12/2017: Initial release.
- 18/02/2020: This code complies with the 40 recall point change on the official KITTI website.
- 18/02/2020: I am not actively developing this repository anymore. This will be the last update for the time being.
This code is based on Bo Li's repository: https://github.com/prclibo/kitti_eval with the main differences being some code cleanup and additional AHS metric described in our paper: Joint 3D Proposal Generation and Object Detection from View Aggregation
evaluate_object_3d_offline.cpp evaluates your KITTI detection locally on
your own computer using your validation data selected from KITTI training dataset, with the following metrics:
- overlap on image (AP)
- oriented overlap on image (AOS)
- overlap on ground-plane (AP)
- oriented overlap on ground-plane (AHS)
- overlap in 3D (AP)
- oriented overlap in 3D (3D AHS)
Clone the repo using:
git clone https://github.com/asharakeh/kitti_native_evaluation.git
From inside the main folder do:
cmake ./make
Run the evalutaion by:
./evaluate_object_3d_offline groundtruth_dir result_dir
Note that you don't have to detect over all KITTI training data. The evaluator only evaluates samples whose result files exist.
The detection format should be simillar to the KITTI dataset label format with 15 columns representing:
| Values | Name | Description |
|---|---|---|
| 1 | type | Describes the type of object: 'Car', 'Van', 'Truck', 'Pedestrian', 'Person_sitting', 'Cyclist', 'Tram', 'Misc' or 'DontCare' |
| 1 | truncated | -1 |
| 1 | occluded | -1 |
| 1 | alpha | Observation angle of object, ranging [-pi..pi] |
| 4 | bbox | 2D bounding box of object in the image (0-based index): contains left, top, right, bottom pixel coordinates |
| 3 | dimensions | 3D object dimensions: height, width, length (in meters) |
| 3 | location | 3D object location x,y,z in camera coordinates (in meters) |
| 1 | rotation_y | Rotation ry around Y-axis in camera coordinates [-pi..pi] |
| 1 | score | Only for results: Float, indicating confidence in detection, needed for p/r curves, higher is better. |
Example:
| type | truncated | occluded | alpha | bbox | dimensions | location | rotation_y | score |
|---|---|---|---|---|---|---|---|---|
| Pedestrian | -1 | -1 | 0.29 | 873.70 152.10 933.44 256.07 | 1.87 0.50 0.90 | 5.42 1.50 13.43 | 0.67 | 0.99 |
If you are using this code, please cite our paper:
Joint 3D Proposal Generation and Object Detection from View Aggregation
@article{ku2017joint, title={Joint 3D Proposal Generation and Object Detection from View Aggregation}, author={Ku, Jason and Mozifian, Melissa and Lee, Jungwook and Harakeh, Ali and Waslander, Steven}, journal={arXiv preprint arXiv:1712.02294}, year={2017} }
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent