ranjaykrishna / densevid_eval

Evaluation code for Dense-Captioning Events in Videos

Dense Captioning Events in Video - Evaluation Code

Usage

First, clone this repository and make sure that all the submodules are also cloned properly.

git clone --recursive https://github.com/ranjaykrishna/densevid_eval.git

Next, download the dataset using

./download.sh

Finally, test that the evaluator runs by testing it on our sample_submission.json file by calling:

python evaluate.py

You are now all set to produce your own dense captioning results for videos and use this code to evaluate your mode:

python evaluate.py -s YOUR_SUBMISSION_FILE.JSON

Paper

Visit our project page and read our paper for details.

Citation

@inproceedings{krishna2017dense,
    title={Dense-Captioning Events in Videos},
    author={Krishna, Ranjay and Hata, Kenji and Ren, Frederic and Fei-Fei, Li and Niebles, Juan Carlos},
    booktitle={ArXiv},
    year={2017}
}

License

MIT License copyright Ranjay Krishna

Contributions

Feel free to send pull requests to help write example code, contribute patches, document methods, etc. You can reach me through:

twitter: @RanjayKrishna

email: ranjaykrishna [at] stanford [dot] edu

densevid_eval's Issues

Code to generate dense captions

Hello,

Is there a near-term plan to release the code to generate dense captions for videos?

Thanks,

KeyError while running evaluate.py

Hi,

Thank you for sharing your code! I was trying to run 'python evaluate.py' to test whether the evaluator runs, but I got some KeyError shown as following. I am wondering did anyone ever encountered similar problems? And if so, how did you solve it?

I think this problem is somehow related to the .jar files in the COCO Evaluation because before the error, I also got something like this:


However, I just have no clue how to debug this. Really appreciated if anyone can help!

How can i reproduce your experiment

Hi ranjay
I want to train a model and caption my own video,Do you have any plans to release the training model in the future?

Does the function "evaluate_detection()" give wrong return order of precision and recall?

Hi Ranjay,
I found the function evaluate_detection() gives possibly wrong output order of precision and recall.

This means that precision and recall might be misused in other functions.
Should it be changed to " return sum(precision) / len(precision), sum(recall) / len(recall) " ?
Did your paper report the right result (recall instead of precision) ?

how can i use the code process my video?(.mp4 or others)

how can i use the code process my video?

evaluation bug

Hi ranjay,

I noticed a potential bug in your evaluation code as follows:
https://github.com/ranjaykrishna/densevid_eval/blob/master/evaluate.py#L196

It only utilizes the temporal proposals in the first groundtruth file to decide whether to add the garbage sentence rather than all groundtruth files.
So if the order of the groundtruth files changes, the evaluation results are different.
I think this part of code might be put outside of the for loop of groundtruth files.