据我所见,作者在所有
@tf.function
def call(self, x):
h1 = self.l1_conv(x)
h1_pool = self.pool(h1)
h2 = self.l2_conv(h1_pool)
h2_pool = self.pool(h2)
flat_h = self.flat(h2_pool)
dense1 = self.dense1(flat_h)
logits = self.dense2(dense1)
probs = tf.nn.softmax(logits, axis=-1)
return probs
call函数前面,都会添加@tf.function,但这样似乎会在使用dropout和BatchNormalization时出现问题
具体问题报错如下(dropout):
Epoch 1/10
1/Unknown - 0s 490ms/step
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
…………………………这是华丽的省略号…………………………………………
_SymbolicException: Inputs to eager execution function cannot be Keras symbolic tensors, but found [<tf.Tensor 'keras_learning_phase:0' shape=() dtype=bool>]
具体问题报错如下(dropout):
---------------------------------------------------------------------------
InaccessibleTensorError Traceback (most recent call last)
<ipython-input-23-12a7599f911a> in <module>
4 train_ds, test_ds = cifar10_dataset()
5 model.is_train=True
----> 6 model.fit(train_ds, epochs=10)
7 model.is_train=False
8 model.evaluate(test_ds)
……………………………………这是华丽的省略号……………………………………………………
InaccessibleTensorError: The tensor 'Tensor("batch_normalization_24/batch_normalization_24_trainable:0", dtype=bool)' cannot be accessed here: it is defined in another function or code block. Use return values, explicit Python locals or TensorFlow collections to access it. Defined in: FuncGraph(name=call, id=2038740490784); accessed from: FuncGraph(name=keras_graph, id=2033355272936).
当我经过一天的反复试错之后(所以给加鸡腿🍖不?),发现锅在这个@tf.function,只要去掉,就可以顺利运行,代码如下(dropout):
def __init__(self):
super(myConvModel, self).__init__()
self.l1_conv = Conv2D(filters=32,
kernel_size=(5, 5),
activation='relu', padding='same')
self.l2_conv = Conv2D(filters=64,
kernel_size=(3, 3),
activation='relu',padding='same')
self.pool = MaxPooling2D(pool_size=(2, 2), strides=2)
self.l3_conv = Conv2D(filters=128,
kernel_size=(3, 3),
activation='relu', padding='same')
self.l4_conv = Conv2D(filters=128,
kernel_size=(3, 3),
activation='relu',padding='same')
self.drop=Dropout(0.5)
self.flat = Flatten()
self.dense1 = layers.Dense(100, activation='tanh')
self.dense2 = layers.Dense(10)
def call(self, x):
h1 = self.drop(self.l1_conv(x))
h1_pool = self.pool(h1)
h2 = self.drop(self.l2_conv(h1_pool))
h2_pool = self.pool(h2)
h3 = self.drop(self.l3_conv(h2_pool))
h4 = self.drop(self.l4_conv(h3))
h4_pool=self.pool(h4)
flat_h = self.flat(h4_pool)
dense1 = self.dense1(flat_h)
logits = self.dense2(dense1)
probs = tf.nn.softmax(logits, axis=-1)
return probs
Epoch 1/10
200/200 [==============================] - 4s 20ms/step - loss: 1.9084 - accuracy: 0.2942
Epoch 2/10
200/200 [==============================] - 3s 16ms/step - loss: 1.5522 - accuracy: 0.4316
Epoch 3/10
200/200 [==============================] - 3s 16ms/step - loss: 1.4182 - accuracy: 0.4845
Epoch 4/10
200/200 [==============================] - 3s 16ms/step - loss: 1.3322 - accuracy: 0.5161
Epoch 5/10
200/200 [==============================] - 3s 16ms/step - loss: 1.2656 - accuracy: 0.5444
Epoch 6/10
200/200 [==============================] - 3s 17ms/step - loss: 1.1927 - accuracy: 0.5668
Epoch 7/10
200/200 [==============================] - 3s 17ms/step - loss: 1.1255 - accuracy: 0.5969
Epoch 8/10
200/200 [==============================] - 3s 17ms/step - loss: 1.0688 - accuracy: 0.6187
Epoch 9/10
200/200 [==============================] - 3s 17ms/step - loss: 1.0302 - accuracy: 0.6305
Epoch 10/10
200/200 [==============================] - 3s 16ms/step - loss: 0.9981 - accuracy: 0.6439
1/1 [==============================] - 1s 574ms/step - loss: 1.1820 - accuracy: 0.5783
[1.1819568872451782, 0.5783]
代码如下(BatchNormalization):
def __init__(self):
super(myConvModel, self).__init__()
self.l1_conv = Conv2D(filters=32,
kernel_size=(5, 5),
activation='relu', padding='same')
self.l2_conv = Conv2D(filters=64,
kernel_size=(3, 3),
activation='relu',padding='same')
self.pool = MaxPooling2D(pool_size=(2, 2), strides=2)
self.l3_conv = Conv2D(filters=128,
kernel_size=(3, 3),
activation='relu', padding='same')
self.l4_conv = Conv2D(filters=128,
kernel_size=(3, 3),
activation='relu',padding='same')
self.bn1=BatchNormalization()
self.bn2=BatchNormalization()
self.bn3=BatchNormalization()
self.bn4=BatchNormalization()
self.flat = Flatten()
self.dense1 = layers.Dense(100, activation='tanh')
self.dense2 = layers.Dense(10)
def call(self, x):
h1 = self.bn1(self.l1_conv(x))
h1_pool = self.pool(h1)
h2 = self.bn2(self.l2_conv(h1_pool))
h2_pool = self.pool(h2)
h3 = self.bn3(self.l3_conv(h2_pool))
h4 = self.bn4(self.l4_conv(h3))
h4_pool=self.pool(h4)
flat_h = self.flat(h4_pool)
dense1 = self.dense1(flat_h)
logits = self.dense2(dense1)
probs = tf.nn.softmax(logits, axis=-1)
return probs
Epoch 1/10
200/200 [==============================] - 4s 20ms/step - loss: 1.5309 - accuracy: 0.4478404 -
Epoch 2/10
200/200 [==============================] - 3s 15ms/step - loss: 1.1861 - accuracy: 0.5785
Epoch 3/10
200/200 [==============================] - 3s 15ms/step - loss: 0.9970 - accuracy: 0.6466
Epoch 4/10
200/200 [==============================] - 3s 15ms/step - loss: 0.8535 - accuracy: 0.6977
Epoch 5/10
200/200 [==============================] - 3s 15ms/step - loss: 0.7295 - accuracy: 0.7469
Epoch 6/10
200/200 [==============================] - 3s 15ms/step - loss: 0.6173 - accuracy: 0.7850
Epoch 7/10
200/200 [==============================] - 3s 15ms/step - loss: 0.5188 - accuracy: 0.8222
Epoch 8/10
200/200 [==============================] - 3s 15ms/step - loss: 0.4174 - accuracy: 0.8595
Epoch 9/10
200/200 [==============================] - 3s 16ms/step - loss: 0.3266 - accuracy: 0.8929
Epoch 10/10
200/200 [==============================] - 3s 16ms/step - loss: 0.2286 - accuracy: 0.9290
1/1 [==============================] - 1s 708ms/step - loss: 1.1636 - accuracy: 0.6567
[1.163588285446167, 0.6567]
此外,这是官方示例代码:
可以看见,在call前面没有@标签的,我个人猜测,这是因为@标签,破坏了一些东西,导致这类需要超参数控制训练和预测时的参数training,出现了问题,而无法train下去,这是我个人浅显的认知,如果作者有确切的答案,希望告知!谢谢!