eljandoubi / image-captioning

Uses a CNN Encoder and a RNN Decoder to generate captions for input images.

Image-Captioning

Uses a CNN Encoder and a RNN Decoder to generate captions for input images.

Here's a sumary of the steps involved.

• Dataset used is the COCO data set by Microsoft.
• Feature vectors for images are generated using a CNN based on the ResNet architecture by Google.
• Word embeddings are generated from captions for training images. NLTK was used for working with processing of captions.
• Implemented an RNN decoder using LSTM cells.
• Trained the network for nearly 9 hrs using GPU to achieve average loss of about 2.8.
• Obtained outputs for some test images to understand efficiency of the trained network.