amaiasalvador / imcap_keras Goto Github PK
View Code? Open in Web Editor NEWImage captioning with spatial attention using keras with tensorflow backend
Image captioning with spatial attention using keras with tensorflow backend
Hi. I've seen that you have manually created the LSTM? Why didn't you use the LSTM provided by Keras, and used it's memory cell output to construct the sentinel?
Here: https://github.com/amaiasalvador/imcap_keras/blob/master/imcap/test.py#L52
lang_model is not declared and also the language_model module is never used in the file.
PS. I'm working on a similar project and I'm trying to understand how to move from training to test because during training it's necessary to feed the lstm with the word sequence, whereas during inference that's not available. What's your approach?
It looks like the generator function has an issue with getting images and captions from the data file:
Line 110:
batch_ims = ims[batch_idxs,:,:,:]
Lines 122-123:
label_start_ixs = starts[batch_idxs]
label_end_ixs = ends[batch_idxs]
The indexing in data file differs from indexing in splits, so I think the lines should be revised to get images and captions from the appropriate splits. For instance, it could be done by using the following lines:
Line 110:
batch_ims = ims[[u+sample_list[0] for u in batch_idxs],:,:,:]
Lines 122-123:
label_start_ixs = starts[[u+sample_list[0] for u in batch_idxs]]
label_end_ixs = ends[[u+sample_list[0] for u in batch_idxs]]
Could you please upload the weights for a pre-trained model, it would be a huge help thanks
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.