div99 / image-captioning Goto Github PK
View Code? Open in Web Editor NEWImage Captioning with Keras
License: MIT License
Image Captioning with Keras
License: MIT License
I'd like to use this code in my project, however I'm getting an error when running eval_model.py
:
ValueError: Input 0 of layer dense_1 is incompatible with the layer: expected axis -1 of input shape to have value 4096 but received input with shape [None, 1000]
I'm thinking that the structure of the network might have changed since 2018, which is when the pretrained weights are from. Could you possibly upload the latest version of the weights if you still have them lying around?
Thanks a bunch!
I am studying Image Caption and very interested in your project. Do you have a model that incorporates attention mechanisms? Thank you very much!
Is it possible for you to share the h5 file so that we can try it in Python?
When I am running command python prepare_data.py
getting below error
Using TensorFlow backend.
Traceback (most recent call last):
File "prepare_data.py", line 132, in
features = extract_features(directory)
File "prepare_data.py", line 38, in extract_features
model = Model(inputs=model.inputs, outputs=model.layers[-1].output)
File "C:\Users\XX\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "C:\Users\XX\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\engine\network.py", line 91, in init
self._init_graph_network(*args, **kwargs)
File "C:\Users\XX\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\engine\network.py", line 251, in _init_graph_network
input_shapes=[x._keras_shape for x in self.inputs],
File "C:\Users\XX\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\engine\network.py", line 251, in
input_shapes=[x._keras_shape for x in self.inputs],
AttributeError: 'Tensor' object has no attribute '_keras_shape'
vocab_size = len(tokenizer.word_index) + 1
请问这个地方为什么要+1呢?
After excuting: prepare_data.py and train_model.py while I was trying to run eval_model.py on few images to find caption, I received the below Error:
Traceback (most recent call last):
File "eval_model.py", line 127, in <module>
model = load_model(filename)
File "./.env/lib/python3.8/site-packages/tensorflow/python/keras/saving/save.py", line 206, in load_model
return hdf5_format.load_model_from_hdf5(filepath, custom_objects,
File "./.env/lib/python3.8/site-packages/tensorflow/python/keras/saving/hdf5_format.py", line 182, in load_model_from_hdf5
model_config = json_utils.decode(model_config.decode('utf-8'))
AttributeError: 'str' object has no attribute 'decode'
After model creation, when the first epoch starts, I get this error:
tensorflow.python.framework.errors_impl.InvalidArgumentError: Matrix size-incompatible: In[0]: [47,1000], In[1]: [4096,256]
[[node model/dense/Relu
Can you help me with this please? Seems like some size issue of tensor.
Error Traceback:
input_1 (InputLayer) [(None, 4096)] 0 []
dropout (Dropout) (None, 4096) 0 ['input_1[0][0]']
dense (Dense) (None, 256) 1048832 ['dropout[0][0]']
input_2 (InputLayer) [(None, 34)] 0 []
repeat_vector (RepeatVector) (None, 34, 256) 0 ['dense[0][0]']
embedding (Embedding) (None, 34, 256) 1940224 ['input_2[0][0]']
concatenate (Concatenate) (None, 34, 512) 0 ['repeat_vector[0][0]',
'embedding[0][0]']
lstm (LSTM) (None, 500) 2026000 ['concatenate[0][0]']
dense_1 (Dense) (None, 7579) 3797079 ['lstm[0][0]']
==================================================================================================
Total params: 8,812,135
Trainable params: 8,812,135
Non-trainable params: 0
None
train_model.py:48: UserWarning: Model.fit_generator
is deprecated and will be removed in a future version. Please use Model.fit
, which supports generators.
callbacks=[checkpoint], validation_data=val_generator, validation_steps=val_steps)
Epoch 1/20
Traceback (most recent call last):
File "train_model.py", line 58, in
train_model(epochs=20)
File "train_model.py", line 48, in train_model
callbacks=[checkpoint], validation_data=val_generator, validation_steps=val_steps)
File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\engine\training.py", line 2030, in fit_generator
initial_epoch=initial_epoch)
File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\tensorflow\python\eager\execute.py", line 59, in quick_execute
inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Matrix size-incompatible: In[0]: [47,1000], In[1]: [4096,256]
[[node model/dense/Relu
(defined at C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\backend.py:4867)
]] [Op:__inference_train_function_7367]
Errors may have originated from an input operation.
Input Source operations connected to node model/dense/Relu:
In[0] model/dense/BiasAdd (defined at C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\layers\core\dense.py:210)
Operation defined at: (most recent call last)
File "train_model.py", line 58, in
train_model(epochs=20)File "train_model.py", line 48, in train_model
callbacks=[checkpoint], validation_data=val_generator, validation_steps=val_steps)File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\engine\training.py", line 2030, in fit_generator
initial_epoch=initial_epoch)File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\engine\training.py", line 1216, in fit
tmp_logs = self.train_function(iterator)File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\engine\training.py", line 878, in train_function
return step_function(self, iterator)File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\engine\training.py", line 867, in step_function
outputs = model.distribute_strategy.run(run_step, args=(data,))File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\engine\training.py", line 860, in run_step
outputs = model.train_step(data)File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\engine\training.py", line 808, in train_step
y_pred = self(x, training=True)File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\engine\base_layer.py", line 1083, in call
outputs = call_fn(inputs, *args, **kwargs)File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\utils\traceback_utils.py", line 92, in error_handler
return fn(*args, **kwargs)File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\engine\functional.py", line 452, in call
inputs, training=training, mask=mask)File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\engine\functional.py", line 589, in _run_internal_graph
outputs = node.layer(*args, **kwargs)File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\engine\base_layer.py", line 1083, in call
outputs = call_fn(inputs, *args, **kwargs)File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\utils\traceback_utils.py", line 92, in error_handler
return fn(*args, **kwargs)File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\layers\core\dense.py", line 213, in call
outputs = self.activation(outputs)File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\activations.py", line 311, in relu
return backend.relu(x, alpha=alpha, max_value=max_value, threshold=threshold)File "C:\Users\kingo\miniconda3\envs\icg\lib\site-packages\keras\backend.py", line 4867, in relu
x = tf.nn.relu(x)
2021-11-27 03:22:33.545141: W tensorflow/core/kernels/data/generator_dataset_op.cc:107] Error occurred when finalizing GeneratorDataset iterator: FAILED_PRECONDITION: Python interpreter state is not initialized. The process may be terminated.
[[{{node PyFunc}}]]
FileNotFoundError: [Errno 2] No such file or directory: 'models/tokenizer.pkl'
Any idea why test samples are not read?
Where can I get those .pkl files ???? Thank you
Do you have Documentation of this project?
Can you provide me the documentation of this project please?
Received the below error while running eval_model.py
raise ValueError('Input ' + str(input_index) +
ValueError: Input 0 is incompatible with layer model_1: expected shape=(None, 4096), found shape=(None, 1000)
The error is on:
File "eval_model.py", line 134, in <module>
captions = generate_desc(model, tokenizer, photo, index_word, max_length)
File "eval_model.py", line 56, in generate_desc
y_pred = model.predict([photo,sequence], verbose=0)[0]
May I ask how much loss and val_loss have you got? I have got a val_loss of 3.736 which I thought is not good enough.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.