cesc-park / crcn Goto Github PK

Where is the evaluation code to get the results reported in the paper ?

It seems that the access of this dataset is invalid? Could you please tell me how to get this dataset? Thanks~

In training process, the epoch is only 5 and why using stages ?

I find it difficult in differentiating training and testing data.
Testing data supposed to contain image sets. But example_test.json has single image instances. Correct me if I am wrong.
Can you help me to prepare for new dataset. ?

As stated in the paper, "Each algorithm retrieves the best sequences from training database for a query image sequence, ... Since the training and test data are disjoint, each algorithm can only retrieve similar (but not identical) sentences at best", then how should I compute the Retrieval metrics (R@1, R@5,..)?
@cesc-park can you help me with the question? Thanks a lot.

I find that the validation loss will increase after only 3 stages. Does the model have heavy over-fitting problem ?

Train on 5823 samples, validate on 648 samples
Epoch 0
5823/5823 [==============================] - 322s - loss: 14098.2655 - val. loss: 197782.5469
Epoch 1
5823/5823 [==============================] - 322s - loss: 5548.3110 - val. loss: 143680.6094
Epoch 2
5823/5823 [==============================] - 321s - loss: 3508.6181 - val. loss: 119612.8281
Epoch 3
5823/5823 [==============================] - 320s - loss: 2460.4376 - val. loss: 108164.6172
Epoch 4
5823/5823 [==============================] - 322s - loss: 1882.4434 - val. loss: 103302.0938
Checkpoint saved
.
.
.
Number of stage 3
Train on 5823 samples, validate on 648 samples
Epoch 0
5823/5823 [==============================] - 322s - loss: 678.7651 - val. loss: 91479.6172
Epoch 1
5823/5823 [==============================] - 321s - loss: 619.2658 - val. loss: 95824.9844
Epoch 2
5823/5823 [==============================] - 321s - loss: 572.6705 - val. loss: 102938.8516
Epoch 3
5823/5823 [==============================] - 322s - loss: 525.9343 - val. loss: 104955.4688
Epoch 4
5823/5823 [==============================] - 321s - loss: 481.5498 - val. loss: 102355.6406
Checkpoint saved
Number of stage 4
Train on 5823 samples, validate on 648 samples
Epoch 0
5823/5823 [==============================] - 322s - loss: 441.9791 - val. loss: 102219.4531
Epoch 1
5823/5823 [==============================] - 321s - loss: 430.3209 - val. loss: 111248.1094
Epoch 2
5823/5823 [==============================] - 320s - loss: 400.2233 - val. loss: 109078.7812
Epoch 3
5823/5823 [==============================] - 322s - loss: 393.7606 - val. loss: 103563.5000
Epoch 4
5823/5823 [==============================] - 321s - loss: 382.1619 - val. loss: 108578.5625
Checkpoint saved
.
.
.
Number of stage 19
Train on 5823 samples, validate on 648 samples
Epoch 0
5823/5823 [==============================] - 394s - loss: 231.7763 - val. loss: 252069.9531
Epoch 1
5823/5823 [==============================] - 388s - loss: 229.1790 - val. loss: 280088.4375
Epoch 2
5823/5823 [==============================] - 399s - loss: 231.8644 - val. loss: 245616.1719
Epoch 3
5823/5823 [==============================] - 412s - loss: 233.6944 - val. loss: 222301.6875
Epoch 4
5823/5823 [==============================] - 398s - loss: 228.1494 - val. loss: 291693.0938
Checkpoint saved