accuracy only about 4% after 10k epochs about tensorflowhandwritingrecognition

[frereit]

Hello. I suspect a problem with how you preprocessed the data. Did you follow my tutorial Preprocessing the data to generate the .npy files? If you want to, you can e-mail me your .npy data, and I will check them to make sure they are correct.

[ivbhatt]

Thanks for such prompt reply!! the .npy files are too large to upload from our internet connection instead, what if i send you the "modified" data_handler.py and rename_images.py modifications are done because we're trying to run this on a linux machine..(hence change from ".split("\\")[1] to split("/")[2])

On Sat, Apr 14, 2018 at 4:28 PM, Frederik Reiter ***@***.***> wrote: Hello. I suspect a problem with how you preprocessed the data. Did you follow my tutorial Preprocessing the data <https://github.com/frereit/TensorflowHandwritingRecognition/blob/master/Preprocessing%20the%20data.md> to generate the .npy files? If you want to, you can e-mail me your .npy data, and I will check them to make sure they are correct. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ATavTmMRIPRIRTxpcZaeyW9MHmZ6rFz8ks5todZIgaJpZM4TVBgC> .

import os from shutil import copyfile def get_class(str): return str.split("/")[2] def main(): by_merge_dir = "./by_merge" output_dir = "./data/" index = 0 class_index = -1 counter = 0 n_copied = 0 classes = [] """ for subdir, dirs, files in os.walk(by_merge_dir): print subdir #s, get_class(subdir) """ for subdir, dirs, files in os.walk(by_merge_dir): for file in files: if get_class(subdir) not in classes: print (get_class(subdir)) classes.append(get_class(subdir)) class_index += 1 index = 0 if counter % 8 == 0 and file.endswith(".png"): copyfile(os.path.join(subdir, file),os.path.join(output_dir, "class_{}_index_{}.png".format(class_index, index))) print("Copied " + os.path.join(subdir, file) + " to " + os.path.join(output_dir, "class_{}_index_{}.png".format(class_index, index))) index += 1 n_copied += 1 counter += 1 print(classes) print("Total images: " + str(n_copied)) if __name__ == "__main__": main() import numpy as np import cv2 import os import sys def get_2d_array(im_path): """Return image scaled with factor 0.5 as numpy array.""" im_color = cv2.imread(im_path) # Read the image as a numpy array. # Shape = (64,64,3) (x_pixels,y_pixels, color_channels) im_color = cv2.resize(im_color, (32,32)) # Rescale the image # Shape = (32,32,3) (x_pixels*scale, y_pixels*scale, color_channels) im = np.zeros(shape=(32,32,1)) # Create an empty array with the final shape (32,32,1) for i, x in enumerate(im_color): # Fill the array for n, y in enumerate(x): # Note: We cannot use cv2.cvtColor(im_color, cv2.COLOR_BGR2GRAY), because im[i][n][0] = (y[0] + y[1] + y[2]) / 3 # that will return an array with the shape (32,32), but we need return im # an array with shape (32,32,1) def get_label(name): """Returns label number for file name >> get_label("class_10_Index_3454.jpeg") 10 """ return int(name.split("_")[1]) def main(): n_labels = 47 n_images = 101785 path = "./data" # Path to images images = np.zeros(shape=(n_images, 32, 32, 1)) # Array with all images labels = np.zeros(shape=(n_images, n_labels)) # Array with all labels, one-hot encoded # Convert all images to numpy arrays for i, file in enumerate(os.listdir(path)): label = get_label(file) image = get_2d_array(os.path.join(path, file)) images[i] = image labels[i, label] = 1 print(str(float(i) / n_images * 100) + "% done") # Save arrays np.save("nist_labels_32x32", labels) np.save("nist_images_32x32", images) if __name__ == "__main__": main()

[frereit]

Could you run this script and send me the output?

import numpy as np
images = np.load("nist_images_32x32.npy")
labels = np.load("nist_labels_32x32.npy")
print(images.shape)
print(labels.shape)

[ivbhatt]

Sure on it Ishan Bhatt, Dept. of Computer Engg., Birla Vishawakarma Mahavidyalaya, V. V. Nagar.

…

On Sat, Apr 14, 2018, 4:50 PM Frederik Reiter ***@***.***> wrote: Could you run this script and send me the output? import numpy as np images = np.load("nist_images_32x32.npy") labels = np.load("nist_labels_32x32.npy") print(images.shape) print(labels.shape) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ATavTp5M8ZU70O2W02XGpiWOZCr2bmEBks5toduIgaJpZM4TVBgC> .

[ivbhatt]

Output:: (101785, 32,32,1) (101785,47) Ishan Bhatt, Dept. of Computer Engg., Birla Vishawakarma Mahavidyalaya, V. V. Nagar. On Sat, Apr 14, 2018, 4:51 PM Ishan Bhatt <[email protected]> wrote:

…

Sure on it Ishan Bhatt, Dept. of Computer Engg., Birla Vishawakarma Mahavidyalaya, V. V. Nagar. On Sat, Apr 14, 2018, 4:50 PM Frederik Reiter ***@***.***> wrote: > Could you run this script and send me the output? > > import numpy as np > images = np.load("nist_images_32x32.npy") > labels = np.load("nist_labels_32x32.npy") > print(images.shape) > print(labels.shape) > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#3 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/ATavTp5M8ZU70O2W02XGpiWOZCr2bmEBks5toduIgaJpZM4TVBgC> > . >

[frereit]

Okay, thank you. I will recreate that and check if I get it to work with the latest commit. I'll get back to you later.

[ivbhatt]

The output of the script that you sent seems just as it should be to me. Am I right?? Ishan Bhatt, Dept. of Computer Engg., Birla Vishawakarma Mahavidyalaya, V. V. Nagar.

…

On Sat, Apr 14, 2018, 5:01 PM Frederik Reiter ***@***.***> wrote: Okay, thank you. I will recreate that and check if I get it to work with the latest commit. I'll get back to you later. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ATavTpdrg9QvDRg3TcVyKYFOb6k_c1ppks5tod32gaJpZM4TVBgC> .

[frereit]

Yeah, that seems correct. I'll check my code; I was messing around with it to test some new stuff, maybe I accidentally committed non-functional code.

[ivbhatt]

Thankyou. waiting anxiously to hear from you again.

…

On Sat, Apr 14, 2018 at 5:08 PM, Frederik Reiter ***@***.***> wrote: Yeah, that seems correct. I'll check my code; I was messing around with it to test some new stuff, maybe I accidentally committed non-functional code. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ATavTk_sRVeeDidLQ3Zpb9byPzgs0llJks5tod-ZgaJpZM4TVBgC> .

[frereit]

Hmm. So I just ran rename_images.py and the script I asked you to run, and my output is:

(101784, 32, 32, 1)
(101784, 47)

I wonder if there is a difference between running that script on Linux and Windows, but a difference of 1 image cannot affect the accuracy. I'll run the training for 15,000 epochs. Currently, I suspect that the optimizer is configured faulty; Maybe the learning rate is too low. Did you run training_32x32.py with the default learning rate of 0.001?

[frereit]

Huh, after only 800 steps with the latest public training_32x32.py I got an accuracy of 60.5%. I'll check the script on Linux now, but at least on windows, the newest commit is working fine.

2018-04-14T14:34:58.860221 Loading data...
2018-04-14T14:34:59.305223 Data loaded... Shuffling...
2018-04-14T14:34:59.774221 Shuffled!
2018-04-14T14:34:59.774221 Performed train-test split
2018-04-14T14:35:00.380722 Model created!
2018-04-14T14:35:01.430837 Variables initialized!
2018-04-14T14:35:01.430837 Training for 15000 epochs.
2018-04-14T14:35:03.682129 Step: 0; Accuracy: 0.06;
2018-04-14T14:36:14.368702 Step: 200; Accuracy: 0.15; Time (200 Steps): 70.6860728263855
2018-04-14T14:37:25.080658 Step: 400; Accuracy: 0.38; Time (200 Steps): 70.71145606040955
2018-04-14T14:38:35.513445 Step: 600; Accuracy: 0.42; Time (200 Steps): 70.43228793144226
2018-04-14T14:39:45.505531 Step: 800; Accuracy: 0.52; Time (200 Steps): 69.99208521842957
2018-04-14T14:40:55.696452 Step: 1000; Accuracy: 0.605; Time (200 Steps): 70.19092130661011

[frereit]

Are you using tensorflow cpu or tensorflow gpu with cuDNN?

[ivbhatt]

About the discrepancy of one image.. We changed a line in rename_images.py: n_images = 101785 (I don't remember exactly, but we incremented n_images by 1 because ls -l | wc -l (Linux command to count the no. Of files in a directory gave the output 101786; one line of summary and total 101785 images in "data") Next. I didn't know tensor flow had a stable distribution for windows! We did not change the learning rate. We have tensorflow for GPU with cudnn installed. However, there was one issue.. nn.softmax_cross_entropy_with_logits_v2() We couldn't find it, so we used nn.softmax_cross_entropy_with_logits() By "latest commit" you mean the one that was 29 days ago right.? Thanks. Your help is much appreciated! Ishan Bhatt, Dept. of Computer Engg., Birla Vishawakarma Mahavidyalaya, V. V. Nagar.

…

On Sat, Apr 14, 2018, 6:16 PM Frederik Reiter ***@***.***> wrote: Are you using tensorflow cpu or tensorflow gpu with cuDNN? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ATavTjGPIGUPEJfJnjxsZkHO5FL0uK6jks5toe-wgaJpZM4TVBgC> .

[frereit]

Okay. Yes, I am talking about the commit from 29 days ago. About the small change in cross entropy calculation:
I used _v2(), because tensorflow was giving me a Warning that nn.softmax_cross_entropy_with_logits() was deprecated:

WARNING:tensorflow:From training_32x32.py:161: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.
Instructions for updating:

Future major versions of TensorFlow will allow gradients to flow
into the labels input on backprop by default.

See tf.nn.softmax_cross_entropy_with_logits_v2.

I now switched it back to tf.nn.softmax_cross_entropy_with_logits and I still achieve a 25% accuracy after only 400 epochs. Since you dind't find nn.softmax_cross_entropy_with_logits_v2(), maybe your tensorflow version is outdated? For me, python -c "import tensorflow as tf; print(tf.__version__)" in terminal returns 1.6.0.

[ivbhatt]

I'll reply to you in about 90 minutes. Ishan Bhatt, Dept. of Computer Engg., Birla Vishawakarma Mahavidyalaya, V. V. Nagar.

…

On Sat, Apr 14, 2018, 6:58 PM Frederik Reiter ***@***.***> wrote: Okay. Yes, I am talking about the commit from 29 days ago. About the small change in cross entropy calculation: I used _v2(), because tensorflow was giving me a Warning that nn.softmax_cross_entropy_with_logits() was deprecated: WARNING:tensorflow:From training_32x32.py:161: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version. Instructions for updating: Future major versions of TensorFlow will allow gradients to flow into the labels input on backprop by default. See tf.nn.softmax_cross_entropy_with_logits_v2. I now switched it back to tf.nn.softmax_cross_entropy_with_logits and I still achieve a 25% accuracy after only 400 epochs. Since you dind't find nn.softmax_cross_entropy_with_logits_v2(), maybe your tensorflow version is outdated? For me, python -c "import tensorflow as tf; print(tf.__version__)" in terminal returns 1.6.0. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ATavTh39_34j2sKEtm5M4mr6jtsK5kE2ks5tofmUgaJpZM4TVBgC> .

[ivbhatt]

Hey, this just struck me, can you send me the trained model files with a good enough accuracy. ie. 32x32_2conv_32_64_1norm_1024.ckpt.meta And others.. Since, we're just not able get the accuracy over 7% .. It may be helpful for others too, if you put this up in the repository itself. Thanks in anticipation. Ishan Bhatt, Dept. of Computer Engg., Birla Vishawakarma Mahavidyalaya, V. V. Nagar. On Sat, Apr 14, 2018, 7:10 PM Ishan Bhatt <[email protected]> wrote:

…

I'll reply to you in about 90 minutes. Ishan Bhatt, Dept. of Computer Engg., Birla Vishawakarma Mahavidyalaya, V. V. Nagar. On Sat, Apr 14, 2018, 6:58 PM Frederik Reiter ***@***.***> wrote: > Okay. Yes, I am talking about the commit from 29 days ago. About the > small change in cross entropy calculation: > I used _v2(), because tensorflow was giving me a Warning that > nn.softmax_cross_entropy_with_logits() was deprecated: > > WARNING:tensorflow:From training_32x32.py:161: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version. > Instructions for updating: > > Future major versions of TensorFlow will allow gradients to flow > into the labels input on backprop by default. > > See tf.nn.softmax_cross_entropy_with_logits_v2. > > I now switched it back to tf.nn.softmax_cross_entropy_with_logits and I > still achieve a 25% accuracy after only 400 epochs. Since you dind't find > nn.softmax_cross_entropy_with_logits_v2(), maybe your tensorflow version > is outdated? For me, python -c "import tensorflow as tf; > print(tf.__version__)" in terminal returns 1.6.0. > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#3 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/ATavTh39_34j2sKEtm5M4mr6jtsK5kE2ks5tofmUgaJpZM4TVBgC> > . >

[frereit]

Okay, I now committed a model which works with the graph from the latest commit. It was trained over 15,000 epochs and has reached the best accuracy I was able to achieve with my computing power and experience: 0.85. I trained this model on Linux with the tensorflow CPU version.
I'd still be interested in your tensorflow version python -c "import tensorflow as tf; print(tf.__version__)", because I am very interested in your problem.

[ivbhatt]

Tensorflow version is 1.0.1 1.0.1 Ishan Bhatt, Dept. of Computer Engg., Birla Vishawakarma Mahavidyalaya, V. V. Nagar.

…

On Sun, Apr 15, 2018, 12:19 AM Frederik Reiter ***@***.***> wrote: Okay, I now committed a model which works with the graph from the latest commit. It was trained over 15,000 epochs and has reached the best accuracy I was able to achieve with my computing power and experience: 0.85. I trained this model on Linux with the tensorflow CPU version. I'd still be interested in your tensorflow version python -c "import tensorflow as tf; print(tf.__version__)", because I am very interested in your problem. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ATavToNH8Xne5S-Fw4pBptE6geTTUeYyks5tokS2gaJpZM4TVBgC> .

[frereit]

Hmm. Maybe that is the problem, but I doubt it. There has to be some problem on your side as I just confirmed the code is working fine with tensorflow CPU on windows and Linux (Ubuntu) and tensorflow GPU on windows. I didn't check tensorflow GPU on Linux, because I couldn't get it to work.

[garchit33]

@IshanBhattOfficial I got an error :

File "F:\TensorflowHandwritingRecognition-master\training_32x32.py", line 211, in
main()
File "F:\TensorflowHandwritingRecognition-master\training_32x32.py", line 184, in main
epochs = int(sys.argv[1])
IndexError: list index out of range

Can you please help me with this asap.

[ivbhatt]

Have you specified "number of epochs" as a command line argument?
It looks like you're using Windows (and an IDE)
If using PyCharm, here's how to specific command line arguments:

https://i.stack.imgur.com/6KQF2.gif

For other IDE, a quick Google search should cut it.

[garchit33]

Got it ... Thanks!!

[garchit33]

@IshanBhattOfficial Have you got the correct accuracy.....How much you have got??

[sanksys]

@frereit
i got below error in training_32x32.py

2019-01-23T16:39:17.376486 Loading data...
2019-01-23T16:39:30.611663 Data loaded... Shuffling...
2019-01-23T16:39:31.936904 Shuffled!
2019-01-23T16:39:31.937902 Performed train-test split
2019-01-23T16:39:32.497435 Model created!
2019-01-23T16:39:40.379682 Variables initialized!
2019-01-23T16:39:40.379682 Training for 15000 epochs.
Traceback (most recent call last):

File "", line 222, in
main()

File "", line 199, in main
summary, _ = sess.run([merged, train], feed_dict={x: batch[0], y_true: batch[1], hold_prob: 0.5})
InvalidArgumentError: You must feed a value for placeholder tensor 'x' with dtype float and shape [?,32,32,1]
[[Node: x = Placeholderdtype=DT_FLOAT, shape=[?,32,32,1], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

i have attached the variable explorer image
i am using Anaconda Spyder with python 3.6 and TF 1.10.0