harishb97 / im2latex-tensorflow-2

TensorFlow-2 implementation of Im2Latex deep learning model described in HarvardNLP paper "What You Get Is What You See: A Visual Markup Decompiler"

Im2Latex-TensorFlow-2

---

TensorFlow-2 implementation of Im2Latex deep learning model for latex code generation from images of mathematical expressions described in HarvardNLP paper "What You Get Is What You See: A Visual Markup Decompiler"

What You Get Is What You See: A Visual Markup Decompiler  
Yuntian Deng, Anssi Kanervisto, and Alexander M. Rush
http://arxiv.org/pdf/1609.04938v1.pdf

This is a general-purpose, deep learning-based system to decompile an image into presentational markup. For example, we can infer the LaTeX or HTML source from a rendered image.

Training data

---

Source im2latex-100k dataset has been preprocessed and resized as suitable for the model. Download the data from this link and move to "images" folder before training.

Sample results

---

Training and evaluating the model

---

Step to train and evaluate the model has been given in the notebooks "im2latex_train.ipynb" and "im2latex_test.ipynb" respectively

Model Performance

---

BLEU score

1. Validation dataset (10340 images): 84.44%
2. Test dataset (9340 images): 84.30%

Validation and train perplexity

Exact match accuracy

Preferred versions:

---

• Tensorflow 2.8.0
• Numpy 1.21.6

Previous implementations:

---

1. Original implementation by HarvardNLP in Torch (Lua)
2. TensorFlow-1 implementation

im2latex-tensorflow-2's Issues

logits and labels must have the same first dimension, got logits shape [20,1024] and labels shape [2780]

Hello, after I loaded the data set, the training model encountered an error, I hope to get your help, thank you.

tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
(0) Invalid argument: logits and labels must have the same first dimension, got logits shape [20,1024] and labels shape [2780]
[[node loss_func_1/sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits (defined at E:\Im2Latex-TensorFlow-2\im2latex.py:248) ]]
[[Func/cond_6/then/_1847/input/_3044/_492]]
(1) Invalid argument: logits and labels must have the same first dimension, got logits shape [20,1024] and labels shape [2780]
[[node loss_func_1/sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits (defined at E:\Im2Latex-TensorFlow-2\im2latex.py:248) ]]