packtpublishing / deep-learning-with-tensorflow-2-and-keras

Deep Learning with TensorFlow 2 and Keras, published by Packt

Deep Learning with TensorFlow 2 and Keras - 2nd Edition

This is the code repository for Deep Learning with TensorFlow 2 and Keras - 2nd Edition, published by Packt. It contains all the supporting project files necessary to work through the book from start to finish.

About the Book

Deep Learning with TensorFlow 2 and Keras, 2nd edition teaches deep learning techniques alongside TensorFlow (TF) and Keras. The book introduces neural networks with TensorFlow, runs through the main applications, covers two working example apps, and then dives into TF and cloudin production, TF mobile, and using TensorFlow with AutoML.

Instructions and Navigation

All of the code is organized into folders. Each folder starts with a number followed by the application name. For example, Chapter 2.

The code will look like the following:

@tf.function
def fn(input, state):
    return cell(input, state)

input = tf.zeros([100, 100])
state = [tf.zeros([100, 100])] * 2
# warmup
cell(input, state)
fn(input, state)

Install the latest version of conda, tensorflow, h5py, opencv

conda update conda
conda update --all
pip install --upgrade tensorflow
pip install --upgrade h5py
pip install opencv-python

Incoming fixes:

This is a compiled list of errors reported to us through Amazon with their solutions and ETA for fixes. Thanks to Sam S. for highlighting these:

From Jan 26, 2020 Review Notes:

• Some programs are giving "cublas64-100.dll" file not found error. Is it possible for authors to zip this dll file and post it on this book's Github page please?

This doesnt look like it's depend on autors, but on the specific setup used by the reader. An additional information is needed from reader on this.

From Jan 25, 2020 Review Notes:

• Ran cifar10_predict.py program of page 131. It ran without errors and gave output results. However, the output gave [4 4]. This output is saying both "standing cat imge" and "dog image" belong to same class of four. This result may be wrong, due to one or both of the following reasons:
  • Model file "cifar10_weights.h5" used by this program is wrong?
  • Accuracy of training program that generated this model file is very low?

All the above is fixed.

Questions are:

• Which is the traing program that generated the above model file?
• Is it the program on pages 128 and 129?

This is based on the pre-trained model contained in TF/Keras

• Program on page 129 is saving to "model.h5" file.
• I ran the program on page 129 and renamed the model file "model.h5" as "cifar10_weights.h5".
• Then I ran program on page 131 and getting following error: ValueError: You are trying to load a weight file containing 13 layers into a model with 6 layers.
• Authors need to fix these errors please?
• Fix model file names of programs on pages 129 and 131 please?

The READ.ME file in Chapter 4 clarifies how to install the model with pre-trained weights.

Thanks again Sam S. we really appreciate your feedback and we'd love to get in touch to say thank you personally! Please feel free if you would like to contact me as the book's Producer, Ben Renow-Clarke, at [email protected] be great to connect!

Antonio Gulli	Amita Kapoor	Sujit Pal

Related Products

• Python Machine Learning – Third Edition

• AI Crash Course

• Dancing with Qubits

Download a free PDF

If you have already purchased a print or Kindle version of this book, you can get a DRM-free PDF version at no cost.
Simply click on the link to claim your free PDF.

https://packt.link/free-ebook/9781838823412

deep-learning-with-tensorflow-2-and-keras's Issues

wrong hidden layer dim in denoising autoencoder

Based on the definition of denoising auto encoder, the hidden layer's dim should > input layer,

is this a bug in the 2nd cell at

https://github.com/PacktPublishing/Deep-Learning-with-TensorFlow-2-and-Keras/blob/master/Chapter%209/DenoisingAutoencoder.ipynb

Deep Convolutional GAN Poor Result Quality

First, I read you book and really enjoyed it. I liked how clearly you explained concepts and provided code within the text. I learned a lot.

Github code: The code on GitHub is https://github.com/PacktPublishing/Deep-Learning-with-TensorFlow-2-and-Keras/blob/master/Chapter%206/DCGAN.ipynb
Book: Deep Learning with TensorFlow 2 and Keras Second Edition

I am working on recreating the deep convolutional GAN starting on page 198 and finding the quality of results changes drastically run to run. I would like to rule out a few discrepancies between the book and the current GitHub code.

1. On page 199, the book says the learning rate is 0.002. Two zeros after the decimal place. On GitHub and in book’s code, the learning rate 0.0002. Three zeros after the decimal place. Which one is correct?

2. On page 200, the text says noise is 100 dimensions. In the book, the code is aligned to this. Z has a default value of 100 and is unchanged when the instance is created. On GitHub, the default is 10. Not 100. Is one value preferred over the other? Is this an insignificant choice?

3. For the vanilla gan, the discriminator's weights are set to trainable before calling discriminator.train_on_batch. Then the weights are reverted back to not trainable when gan.train_on_batch is called. For the deep convolutional GAN, the discriminator’s weights are never set to trainable. Was this on purpose?

4. For the vanilla gan, the discriminator receives one set of fake data. Then a different random sample is created and the GAN is trained. For the deep convolutional gan, the discriminator and gan are trained on the same random sample of noise. Is there a reason for the difference? Is this an insignificant choice?

Chapter 6 DCGAN.ipynb not working as expected any more

I use Python 3.8 and Tensorflow 2.4 and I see that at about 1000 epochs discriminator's accuracy reaches 100% and the generator's loss significantly increases and the quality of the images produced by this DCGAN goes down and never recovers. I have tried multiple times, both on my work computer and with the use of colab. In other words the training goes very differently than in https://github.com/PacktPublishing/Deep-Learning-with-TensorFlow-2-and-Keras/blob/master/Chapter%206/DCGAN.ipynb. My presumption is that some improvements in TensorFlow 2.4 cause the generator to train too well and/or too quickly. Any ideas how to fix DCGAN.ipynb?

Wrong in alice_text_generator.py

alice_text_generator.py:
open(p, mode='r').read() -> open(p, mode='r', encoding='utf-8').read()

Wrong function parameter order in seq2seq

Deep-Learning-with-TensorFlow-2-and-Keras/Chapter 8/seq2seq_wo_attn.py

Line 47 in b263947

class Encoder(tf.keras.Model):

The definition is

class Encoder(tf.keras.Model):
    def __init__(self, vocab_size, num_timesteps, 
            embedding_dim, encoder_dim, **kwargs):

However, the encoder is create by

Deep-Learning-with-TensorFlow-2-and-Keras/Chapter 8/seq2seq_wo_attn.py

Line 217 in b263947

encoder = Encoder(vocab_size_en+1, embedding_dim, maxlen_en, encoder_dim)

I think the correct order should be

class Encoder(tf.keras.Model):
    def __init__(self, vocab_size, embedding_dim, num_timesteps,
            encoder_dim, **kwargs):

Error at Chapter 7 Skipgram model (tf2_skipgram_model.py)

import tensorflow as tf

class SkipgramModel(tf.keras.Model):
    def __init__(self, vocab_sz, embed_sz, window_sz, **kwargs):
        super(SkipgramModel, self).__init__(**kwargs)
        self.word_model = tf.keras.Sequential([
            tf.keras.layers.Embedding(
                input_dim=vocab_sz,
                output_dim=embed_sz,
                embeddings_initializer="glorot_uniform",
                input_length=1
            ),
            tf.keras.layers.Reshape((embed_sz,))
        ])
        self.context_model = tf.keras.Sequential([
            tf.keras.layers.Embedding(
                input_dim=vocab_sz,
                output_dim=embed_sz,
                embeddings_initializer="glorot_uniform",
                input_length=1
            ),
            tf.keras.layers.Reshape((embed_sz,))
        ])
        self.merge = tf.keras.layers.Dot(axes=0)
        self.dense = tf.keras.layers.Dense(1,
                kernel_initializer="glorot_uniform",
                activation="sigmoid"
        )

    def call(self, input):
        word, context = input
        word_emb = self.word_model(word)
        context_emb = self.context_model(context)
        x = self.merge([word_emb, context_emb])
        x = self.dense(x)
        return x


VOCAB_SIZE = 5000
EMBED_SIZE = 300
WINDOW_SIZE = 1  # 3 word window, 1 on left, 1 on right

model = SkipgramModel(VOCAB_SIZE, EMBED_SIZE, WINDOW_SIZE)
model.build(input_shape=[(None, VOCAB_SIZE), (None, VOCAB_SIZE)])
model.compile(optimizer=tf.optimizers.Adam(),
    loss="categorical_crossentropy",
    metrics=["accuracy"])

model.summary()

# train the model here

# retrieve embeddings from trained model
word_model = model.layers[0]
word_emb_layer = word_model.layers[0]
emb_weights = None
for weight in word_emb_layer.weights:
    if weight.name == "embedding/embeddings:0":
        emb_weights = weight.numpy()
print(emb_weights, emb_weights.shape)

---

ValueError Traceback (most recent call last)
in
42
43 model = SkipgramModel(VOCAB_SIZE, EMBED_SIZE, WINDOW_SIZE)
---> 44 model.build(input_shape=[(None, VOCAB_SIZE), (None, VOCAB_SIZE)])
45 model.compile(optimizer=tf.optimizers.Adam(),
46 loss="categorical_crossentropy",

C:\Anaconda\envs\tensorflow2_py37\lib\site-packages\tensorflow_core\python\keras\engine\network.py in build(self, input_shape)
680 'method accepts an inputs argument.')
681 try:
--> 682 self.call(x, **kwargs)
683 except (errors.InvalidArgumentError, TypeError):
684 raise ValueError('You cannot build your model by calling build '

in call(self, input)
32 word_emb = self.word_model(word)
33 context_emb = self.context_model(context)
---> 34 x = self.merge([word_emb, context_emb])
35 x = self.dense(x)
36 return x

C:\Anaconda\envs\tensorflow2_py37\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py in call(self, inputs, *args, **kwargs)
776 outputs = base_layer_utils.mark_as_return(outputs, acd)
777 else:
--> 778 outputs = call_fn(cast_inputs, *args, **kwargs)
779
780 except errors.OperatorNotAllowedInGraphError as e:

C:\Anaconda\envs\tensorflow2_py37\lib\site-packages\tensorflow_core\python\keras\layers\merge.py in call(self, inputs)
179 return y
180 else:
--> 181 return self._merge_function(inputs)
182
183 @tf_utils.shape_type_conversion

C:\Anaconda\envs\tensorflow2_py37\lib\site-packages\tensorflow_core\python\keras\layers\merge.py in _merge_function(self, inputs)
533 x1 = nn.l2_normalize(x1, axis=axes[0])
534 x2 = nn.l2_normalize(x2, axis=axes[1])
--> 535 output = K.batch_dot(x1, x2, axes)
536 return output
537

C:\Anaconda\envs\tensorflow2_py37\lib\site-packages\tensorflow_core\python\keras\backend.py in batch_dot(x, y, axes)
1785 # sanity checks
1786 if 0 in axes:
-> 1787 raise ValueError('Cannot perform batch_dot over axis 0. '
1788 'If your inputs are not batched, '
1789 'add a dummy batch dimension to your '

ValueError: Cannot perform batch_dot over axis 0. If your inputs are not batched, add a dummy batch dimension to your inputs using K.expand_dims(x, 0)