tf.keras.layers.Dense leads to significant differences between CPU and GPU runs of the model implementation code about tensorflow

[PhyllisJi]

Data from /Users/pinji/Desktop/MoCoDiff/tf-0513/tensorflow-LeNet/LeNet-12-654/case/tensorflow_cpu/output.npz:
Array name: output
[[-0.19694163, -0.09662387, -0.2447289 , ..., -0.03054091, -0.01255638,
0.11781679],
[-0.09944421, -0.1292284 , -0.20278051, ..., -0.01450334, 0.1526829 ,
0.2054872 ],
[ 0.07628068, -0.14517091, -0.16096437, ..., -0.21423727, 0.14428714,
0.09608107],
...,
[-0.20587711, -0.10656235, -0.33036825, ..., -0.13541238, 0.3215404 ,
0.2165656 ],
[-0.11513966, -0.1102111 , -0.3434586 , ..., -0.19525276, 0.08722814,
-0.05503507],
[-0.09822497, -0.10671525, -0.2368006 , ..., -0.16467465, 0.15050802,
0.10809175]]

========================================
Data from /Users/pinji/Desktop/MoCoDiff/tf-0513/tensorflow-LeNet/LeNet-12-654/case/tensorflow_gpu/output.npz:
Array name: output
[[-0.19693479, -0.09660047, -0.24478191, ..., -0.03068957, -0.01259471,
0.1177731 ],
[-0.09948827, -0.12920822, -0.20286846, ..., -0.01475166, 0.15262538,
0.20549278],
[ 0.07635015, -0.14510253, -0.16103148, ..., -0.21449052, 0.14431402,
0.09607705],
...,
[-0.20597562, -0.1065662 , -0.33040133, ..., -0.13553414, 0.32143065,
0.2166584 ],
[-0.1152329 , -0.11016633, -0.34351912, ..., -0.19536608, 0.08703801,
-0.05509032],
[-0.09832728, -0.10680126, -0.2368007 , ..., -0.1649086 , 0.1503351 ,
0.10806311]]

========================================
543 diff: 0.055949583649635315
cpu-543: [-0.1139553 -0.04668004 -0.4390249 0.20775127 0.0899936 0.22854462
-0.46038878 -0.15526699 0.23111053 -0.03327706]
gpu-543: [-0.12328502 -0.0404007 -0.4342668 0.21254592 0.05579592 0.24491714
-0.479826 -0.15429592 0.17516094 -0.05355967]

[tilakrayal]

@PhyllisJi,
I tried to execute the mentioned code. Kindly find the gist of it here. In the given code snippet you have defined the class and its methods but are not calling them anywhere. Also try to execute the code with the keras3.0 which is default for the TensorFlow 2.16 and let us know if you are facing the same issue. Thank you!