got a strange mAP about pytorch_yolov3

[hirotomusiker]

Thank you!
The aP calculation you have observed is not an error.
yolov3.weight is trained on the same coco dataset (by the author) but uses the different train/val split.

darknet

train : trainvalno5k
val : 5k
that can be obtained from:
https://github.com/pjreddie/darknet/blob/master/scripts/get_coco_dataset.sh#L22-L23

Ours

train : train2017
val : val2017
that can be obtained from:
http://cocodataset.org/#download

Because there is overlap between trainvalno5k and val2017, the aP calculation result using yolov3.weight on val2017 is higher than the reported one.

[deeppower]

@hirotomusiker
Thanks a lot!
Using yolov3.weight on 5k, the mAP[IOU=0.5] is 0.547.

[AsiamGit]

thanks a lot~

[deeppower]

@TakashiNinjouji Hi! Do you know where can i find the json file of 5k.txt?

[hirotomusiker]

Hi,
you have to convert 5k.txt to json format.
datadir is the path of the directory where you have 5k.txt, instances_train2017.json and instances_val2017.json.

1. Load image ids from 5k.txt

import json
with open(datadir + "5k.txt", "r") as f:
    txt_5k = f.read()
txt_5k = txt_5k.split('\n')

ids_5k = [int(w.split('_')[-1].split('.')[0]) for w in txt_5k[:-1]]

2. extract annotations and image information from train2017 and val2017

train = json.load(open(datadir + "/instances_train2017.json"))
val = json.load(open(datadir + "/instances_val2017.json"))
images_ids = []
annot_ids = []

for an in train['annotations']:
    if an['image_id'] in ids_5k:
        annot_ids.append(an)
        
for im in train['images']:
    if im['id'] in ids_5k:
        images_ids.append(im)

for an in val['annotations']:
    if an['image_id'] in ids_5k:
        annot_ids.append(an)
        
for im in val['images']:
    if im['id'] in ids_5k:
        images_ids.append(im)

3. save val5k file

val['images'] = images_ids
val['annotations'] = annot_ids

json.dump(val, open(datadir + "/instances_val5k.json", "w"))

Hope it will work for you 😃

[hirotomusiker]

Perfect!
thank you!